2024-04-25T09:47:39.380 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:47:39.380 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} --verbose --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039 -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/orig.config.yaml 2024-04-25T09:47:39.424 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:47:39.499 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673039' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG+YL4I7Su40jh+Xep6ALSRVogUymF8B8SQK6N9Pzc+CIIti8+X7NMBP9JoTeL/1ngw4FHqQRhyZeRdQ9u4EkjY= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE67rz3gBp+UYTCR0nL1yumg2dDICEjI5e5neAvMcscRyI6LMon5FIPkeN7zQz/NHykplfLXQuqieQhKU+lsdNY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:47:39.500 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:47:39.500 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:47:39.501 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:47:39.502 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:47:39.526 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:47:39.526 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:47:39.526 INFO:teuthology.packaging:ref: None 2024-04-25T09:47:39.526 INFO:teuthology.packaging:tag: None 2024-04-25T09:47:39.526 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:47:39.526 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:47:39.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:47:39.686 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:47:39.687 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:47:39.694 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:47:39.694 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:47:39.731 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:47:39.750 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:47:39.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:47:39.775 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:42:31.428604', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG+YL4I7Su40jh+Xep6ALSRVogUymF8B8SQK6N9Pzc+CIIti8+X7NMBP9JoTeL/1ngw4FHqQRhyZeRdQ9u4EkjY='} 2024-04-25T09:47:39.794 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:42:31.427277', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE67rz3gBp+UYTCR0nL1yumg2dDICEjI5e5neAvMcscRyI6LMon5FIPkeN7zQz/NHykplfLXQuqieQhKU+lsdNY='} 2024-04-25T09:47:39.795 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:47:39.800 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:47:39.801 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:47:39.801 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:47:39.879 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbdbf7e7160>, signals=[15]) 2024-04-25T09:47:39.879 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:47:39.885 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:47:39.885 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:47:39.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:47:39.963 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:47:39.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:47:40.039 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:47:40.046 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-04-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-04-25T09:47:40.064 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-04-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-04-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-04-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:47:40.122 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-04-25T09:47:40.139 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-04-25T09:47:40.157 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-04-25T09:47:40.157 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-04-25T09:47:40.215 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-04-25T09:47:40.215 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-04-25T09:47:40.215 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-04-25T09:47:40.215 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:47:40.215 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-04-25T09:47:40.215 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:47:40.216 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:47:40.216 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-04-25T09:47:40.239 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:47:40.249 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:47:40.281 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:47:40.282 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:47:40.284 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:47:40.300 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:47:40.305 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:47:40.305 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:47:40.341 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:47:40.371 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:47:40.416 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:47:40.431 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:47:40.431 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T09:47:40.431 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T09:47:40.432 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:47:40.432 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:47:40.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:47:40.450 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-04-25T09:47:40.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:47:40.451 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-04-25T09:47:40.507 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:47:40.507 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-437.el9.x86_64 2024-04-25T09:47:40.507 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-04-25T09:47:40.509 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:47:40.509 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-437.el9.x86_64 2024-04-25T09:47:40.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-04-25T09:47:42.742 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T09:47:42.773 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T09:47:44.626 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T09:47:45.010 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T09:47:48.744 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T09:47:49.044 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-25T09:47:50.328 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-04-25T09:47:51.094 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-25T09:47:51.815 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T09:47:52.540 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T09:47:56.812 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T09:47:57.611 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 9.4 kB/s | 1.7 kB 00:00 2024-04-25T09:47:58.677 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:47:58.677 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:47:58.765 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-25T09:47:58.766 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-25T09:47:58.766 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-25T09:47:58.893 DEBUG:teuthology.orchestra.run.smithi148:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:47:59.396 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:47:56 AM UTC. 2024-04-25T09:47:59.485 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T09:47:59.485 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T09:47:59.520 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-25T09:47:59.533 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-25T09:47:59.533 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-04-25T09:47:59.533 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout:Reinstalling: 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 255 k 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 0 2024-04-25T09:47:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:Operation aborted. 2024-04-25T09:47:59.578 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-25T09:47:59.579 INFO:teuthology.orchestra.run.smithi188.stdout:Nothing to do. 2024-04-25T09:47:59.579 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-25T09:47:59.592 INFO:teuthology.orchestra.run.smithi148.stdout:Is this ok [y/N]: 2024-04-25T09:47:59.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum reinstall -y kernel || true 2024-04-25T09:47:59.695 DEBUG:teuthology.orchestra.run.smithi188:> echo no | sudo yum reinstall kernel || true 2024-04-25T09:48:00.086 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 09:47:56 AM UTC. 2024-04-25T09:48:00.194 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:47:57 AM UTC. 2024-04-25T09:48:00.205 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-25T09:48:00.205 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:Reinstalling: 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 255 k 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 0 2024-04-25T09:48:00.206 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-25T09:48:00.312 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-25T09:48:00.335 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-25T09:48:00.335 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-04-25T09:48:00.335 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-25T09:48:00.335 INFO:teuthology.orchestra.run.smithi188.stdout:Reinstalling: 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 255 k 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 0 2024-04-25T09:48:00.336 INFO:teuthology.orchestra.run.smithi188.stderr:Operation aborted. 2024-04-25T09:48:00.398 INFO:teuthology.orchestra.run.smithi188.stdout:Is this ok [y/N]: 2024-04-25T09:48:00.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum reinstall -y kernel || true 2024-04-25T09:48:00.440 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-25T09:48:00.440 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:48:00.440 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.1 MB/s | 255 kB 00:00 2024-04-25T09:48:00.461 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-25T09:48:00.640 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-25T09:48:00.640 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-25T09:48:00.645 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-25T09:48:00.645 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-25T09:48:00.782 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-25T09:48:00.882 INFO:teuthology.orchestra.run.smithi148.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:48:00.883 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:48:00.895 INFO:teuthology.orchestra.run.smithi188.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 09:47:57 AM UTC. 2024-04-25T09:48:01.011 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout:Reinstalling: 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:48:01.012 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-25T09:48:01.013 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-04-25T09:48:01.013 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:48:01.013 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 255 k 2024-04-25T09:48:01.013 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 0 2024-04-25T09:48:01.013 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-25T09:48:01.599 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:48:01.599 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:48:01.713 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-437.el9.x86_64.rpm 774 kB/s | 255 kB 00:00 2024-04-25T09:48:01.714 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:48:01.714 INFO:teuthology.orchestra.run.smithi188.stdout:Total 363 kB/s | 255 kB 00:00 2024-04-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-25T09:48:01.903 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-25T09:48:01.903 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-25T09:48:01.909 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-25T09:48:01.909 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-25T09:48:02.056 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-25T09:48:02.131 INFO:teuthology.orchestra.run.smithi188.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:48:02.131 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:48:02.292 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:48:02.292 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:48:02.292 INFO:teuthology.orchestra.run.smithi148.stdout:Reinstalled: 2024-04-25T09:48:02.292 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:48:02.292 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:48:02.292 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-25T09:48:02.478 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:48:02.524 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:48:02.525 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:48:02.525 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:48:02.525 DEBUG:teuthology.parallel:result is None 2024-04-25T09:48:02.791 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:48:02.791 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T09:48:03.514 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T09:48:03.514 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:48:03.514 INFO:teuthology.orchestra.run.smithi188.stdout:Reinstalled: 2024-04-25T09:48:03.514 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T09:48:03.514 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:48:03.514 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-25T09:48:03.677 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T09:48:03.765 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T09:48:03.766 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T09:48:03.766 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T09:48:03.766 DEBUG:teuthology.parallel:result is None 2024-04-25T09:48:03.766 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:48:03.774 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:48:03.774 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:48:03.776 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:48:03.792 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:48:03.798 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:48:03.845 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:48:03.846 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:48:03.849 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:48:03.881 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:48:03.896 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:48:03.896 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:48:03.918 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:48:03.958 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:48:03.959 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:48:03.973 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:48:03.974 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:48:03.976 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:48:03.983 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:48:03.983 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:48:04.018 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:48:04.043 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:48:04.052 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:48:04.052 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:48:04.085 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:48:04.099 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:48:04.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:48:04.233 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:48:04.312 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:48:04.357 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:48:04.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:48:04.422 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:48:04.462 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:48:04.516 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:48:04.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:48:04.635 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:48:04.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:48:04.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-04-25T09:48:04.700 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-04-25T09:48:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:48:04.781 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:48:05.136 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:48:05.145 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:48:05.145 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:48:05.191 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:48:05.240 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-04-25T09:48:05.301 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-04-25T09:48:05.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-04-25T09:48:05.356 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-04-25T09:48:05.358 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:48:05.358 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-04-25T09:48:05.381 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-04-25T09:48:05.381 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-04-25T09:48:05.404 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-04-25T09:48:05.404 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:48:05.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:48:05.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:48:05.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:48:05.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:48:05.471 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:48:05.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:48:05.511 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:48:05.535 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:48:05.579 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:48:05.596 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:48:05.597 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi148.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:51:33.554 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-04-25T09:51:33.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-25T09:51:33.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:33.631 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-04-25T09:51:33.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-25T09:51:33.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-25T09:51:33.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:33.785 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-25T09:51:33.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-25T09:51:33.866 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:51:33.879 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:51:33.880 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:51:33.880 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:51:33.883 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:51:33.883 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:51:33.910 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:51:33.924 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:51:33.935 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:51:33.946 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-04-25T09:51:33.948 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:51:33.957 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-04-25T09:51:33.970 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:51:33.971 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-04-25T09:51:33.982 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-04-25T09:51:33.983 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:51:33.995 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:51:34.008 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:51:34.042 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:51:34.045 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:51:34.045 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-25T09:51:34.046 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.046 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.046 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.046 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.062 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:51:34.065 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:51:34.065 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-25T09:51:34.065 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.065 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.065 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.065 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:51:34.066 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:51:34.075 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:51:34.075 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:51:34.075 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:51:34.075 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:51:34.081 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:51:34.081 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:51:34.081 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:51:34.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:51:34.084 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:51:34.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:51:34.222 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:51:34.222 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:51:34.258 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:51:34.259 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:51:34.343 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:51:34.343 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:51:34.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:51:34.368 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:51:34.368 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:51:34.370 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:51:34.370 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:51:34.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:51:34.395 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:51:34.395 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:51:34.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:51:34.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:51:34.506 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:51:34.536 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:51:34.570 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-04-25T09:51:34.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-04-25T09:51:34.599 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-04-25T09:51:34.601 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-04-25T09:51:34.794 INFO:teuthology.orchestra.run.smithi188.stdout:48 files removed 2024-04-25T09:51:34.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:51:34.825 INFO:teuthology.orchestra.run.smithi148.stdout:48 files removed 2024-04-25T09:51:34.851 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:51:35.454 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 218 kB/s | 82 kB 00:00 2024-04-25T09:51:35.460 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 233 kB/s | 82 kB 00:00 2024-04-25T09:51:36.160 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-04-25T09:51:36.214 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-04-25T09:51:36.628 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-25T09:51:37.328 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T09:51:38.881 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 737 B/s | 1.9 kB 00:02 2024-04-25T09:51:39.539 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T09:51:41.280 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-25T09:51:42.315 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T09:51:45.230 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-25T09:51:46.716 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.2 MB 00:00 2024-04-25T09:51:46.981 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-25T09:51:48.092 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-04-25T09:51:49.260 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-04-25T09:51:50.951 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 21 MB 00:02 2024-04-25T09:51:54.307 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-25T09:51:55.992 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-25T09:51:56.433 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:51:56.434 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:51:56.445 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:51:56.446 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:51:56.447 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:51:56.447 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:51:56.449 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:51:56.451 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:51:56.452 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:51:56.513 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-25T09:51:56.522 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-25T09:51:56.522 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:51:56.523 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:51:56.524 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:51:56.525 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:51:56.526 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:51:56.527 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:51:56.528 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:51:56.529 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:51:56.530 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:51:56.531 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:51:56.532 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:51:56.533 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:51:56.534 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 4 Packages 2024-04-25T09:51:56.535 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:51:56.536 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 215 M 2024-04-25T09:51:56.536 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-25T09:51:57.392 INFO:teuthology.orchestra.run.smithi148.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-25T09:51:57.650 INFO:teuthology.orchestra.run.smithi148.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 3.4 MB/s | 887 kB 00:00 2024-04-25T09:51:57.734 INFO:teuthology.orchestra.run.smithi148.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.7 MB/s | 145 kB 00:00 2024-04-25T09:51:57.959 INFO:teuthology.orchestra.run.smithi148.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 7.2 MB/s | 5.3 MB 00:00 2024-04-25T09:51:58.109 INFO:teuthology.orchestra.run.smithi148.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 5.7 MB/s | 2.1 MB 00:00 2024-04-25T09:51:58.151 INFO:teuthology.orchestra.run.smithi148.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-25T09:51:58.152 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:51:58.152 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:51:58.163 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:51:58.164 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:51:58.165 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:51:58.165 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:51:58.167 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:51:58.169 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:51:58.170 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:51:58.230 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:51:58.239 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:51:58.240 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:51:58.241 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-04-25T09:51:58.242 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:51:58.243 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:51:58.244 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:51:58.245 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:51:58.246 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:51:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:51:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:51:58.249 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:51:58.250 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout:Install 148 Packages 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 4 Packages 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:51:58.251 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 215 M 2024-04-25T09:51:58.252 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-25T09:51:58.818 INFO:teuthology.orchestra.run.smithi148.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-25T09:51:59.018 INFO:teuthology.orchestra.run.smithi188.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-25T09:51:59.835 INFO:teuthology.orchestra.run.smithi148.stdout:(8/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x86 10 MB/s | 17 MB 00:01 2024-04-25T09:51:59.902 INFO:teuthology.orchestra.run.smithi188.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.0 MB/s | 887 kB 00:00 2024-04-25T09:51:59.910 INFO:teuthology.orchestra.run.smithi148.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 337 kB/s | 25 kB 00:00 2024-04-25T09:52:00.044 INFO:teuthology.orchestra.run.smithi188.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.0 MB/s | 145 kB 00:00 2024-04-25T09:52:00.686 INFO:teuthology.orchestra.run.smithi148.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 5.2 MB/s | 9.8 MB 00:01 2024-04-25T09:52:00.786 INFO:teuthology.orchestra.run.smithi148.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.2 MB/s | 220 kB 00:00 2024-04-25T09:52:00.869 INFO:teuthology.orchestra.run.smithi148.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 325 kB/s | 27 kB 00:00 2024-04-25T09:52:00.944 INFO:teuthology.orchestra.run.smithi148.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 416 kB/s | 31 kB 00:00 2024-04-25T09:52:01.077 INFO:teuthology.orchestra.run.smithi188.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 2.3 MB/s | 5.3 MB 00:02 2024-04-25T09:52:01.119 INFO:teuthology.orchestra.run.smithi188.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 2.1 MB 00:01 2024-04-25T09:52:01.119 INFO:teuthology.orchestra.run.smithi148.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 4.2 MB/s | 744 kB 00:00 2024-04-25T09:52:01.195 INFO:teuthology.orchestra.run.smithi148.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.1 MB/s | 163 kB 00:00 2024-04-25T09:52:01.270 INFO:teuthology.orchestra.run.smithi148.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.6 MB/s | 127 kB 00:00 2024-04-25T09:52:01.553 INFO:teuthology.orchestra.run.smithi188.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T09:52:01.621 INFO:teuthology.orchestra.run.smithi148.stdout:(17/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 4.5 MB/s | 20 MB 00:04 2024-04-25T09:52:01.654 INFO:teuthology.orchestra.run.smithi148.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 1.3 MB/s | 501 kB 00:00 2024-04-25T09:52:01.738 INFO:teuthology.orchestra.run.smithi148.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 539 kB/s | 45 kB 00:00 2024-04-25T09:52:01.838 INFO:teuthology.orchestra.run.smithi148.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.3 MB/s | 131 kB 00:00 2024-04-25T09:52:01.930 INFO:teuthology.orchestra.run.smithi148.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.7 MB/s | 162 kB 00:00 2024-04-25T09:52:02.172 INFO:teuthology.orchestra.run.smithi148.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 1.3 MB/s | 321 kB 00:00 2024-04-25T09:52:02.464 INFO:teuthology.orchestra.run.smithi148.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 1.0 MB/s | 298 kB 00:00 2024-04-25T09:52:02.556 INFO:teuthology.orchestra.run.smithi148.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T09:52:02.631 INFO:teuthology.orchestra.run.smithi148.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T09:52:02.653 INFO:teuthology.orchestra.run.smithi188.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 3.1 MB/s | 4.7 MB 00:01 2024-04-25T09:52:02.824 INFO:teuthology.orchestra.run.smithi148.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 4.3 MB/s | 5.1 MB 00:01 2024-04-25T09:52:02.907 INFO:teuthology.orchestra.run.smithi148.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T09:52:02.974 INFO:teuthology.orchestra.run.smithi148.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 473 kB/s | 31 kB 00:00 2024-04-25T09:52:03.057 INFO:teuthology.orchestra.run.smithi148.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T09:52:03.458 INFO:teuthology.orchestra.run.smithi148.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 4.5 MB/s | 1.8 MB 00:00 2024-04-25T09:52:03.641 INFO:teuthology.orchestra.run.smithi148.stdout:(31/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 3.1 MB/s | 3.1 MB 00:01 2024-04-25T09:52:03.792 INFO:teuthology.orchestra.run.smithi148.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.6 MB/s | 248 kB 00:00 2024-04-25T09:52:03.867 INFO:teuthology.orchestra.run.smithi148.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 658 kB/s | 49 kB 00:00 2024-04-25T09:52:03.942 INFO:teuthology.orchestra.run.smithi148.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 203 kB/s | 15 kB 00:00 2024-04-25T09:52:04.076 INFO:teuthology.orchestra.run.smithi148.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 1.9 MB/s | 264 kB 00:00 2024-04-25T09:52:04.352 INFO:teuthology.orchestra.run.smithi148.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 2.7 MB/s | 748 kB 00:00 2024-04-25T09:52:04.419 INFO:teuthology.orchestra.run.smithi148.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 274 kB/s | 18 kB 00:00 2024-04-25T09:52:04.510 INFO:teuthology.orchestra.run.smithi148.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 879 kB/s | 80 kB 00:00 2024-04-25T09:52:04.610 INFO:teuthology.orchestra.run.smithi148.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-04-25T09:52:04.677 INFO:teuthology.orchestra.run.smithi148.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T09:52:04.753 INFO:teuthology.orchestra.run.smithi148.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi148.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T09:52:04.928 INFO:teuthology.orchestra.run.smithi148.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T09:52:04.995 INFO:teuthology.orchestra.run.smithi148.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T09:52:05.062 INFO:teuthology.orchestra.run.smithi148.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T09:52:05.137 INFO:teuthology.orchestra.run.smithi148.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T09:52:05.329 INFO:teuthology.orchestra.run.smithi148.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-04-25T09:52:05.394 INFO:teuthology.orchestra.run.smithi188.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 3.0 MB/s | 20 MB 00:06 2024-04-25T09:52:05.413 INFO:teuthology.orchestra.run.smithi148.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-25T09:52:05.453 INFO:teuthology.orchestra.run.smithi188.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 434 kB/s | 25 kB 00:00 2024-04-25T09:52:05.463 INFO:teuthology.orchestra.run.smithi148.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-25T09:52:05.547 INFO:teuthology.orchestra.run.smithi148.stdout:(50/152): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-04-25T09:52:05.572 INFO:teuthology.orchestra.run.smithi148.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 198 kB/s | 21 kB 00:00 2024-04-25T09:52:05.622 INFO:teuthology.orchestra.run.smithi148.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-25T09:52:05.636 INFO:teuthology.orchestra.run.smithi188.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 3.3 MB/s | 9.8 MB 00:02 2024-04-25T09:52:05.728 INFO:teuthology.orchestra.run.smithi188.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.4 MB/s | 220 kB 00:00 2024-04-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi148.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-25T09:52:05.773 INFO:teuthology.orchestra.run.smithi148.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-04-25T09:52:05.778 INFO:teuthology.orchestra.run.smithi188.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 544 kB/s | 27 kB 00:00 2024-04-25T09:52:05.798 INFO:teuthology.orchestra.run.smithi148.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-04-25T09:52:05.815 INFO:teuthology.orchestra.run.smithi148.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-04-25T09:52:05.828 INFO:teuthology.orchestra.run.smithi188.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 624 kB/s | 31 kB 00:00 2024-04-25T09:52:05.849 INFO:teuthology.orchestra.run.smithi148.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T09:52:05.874 INFO:teuthology.orchestra.run.smithi148.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-25T09:52:05.908 INFO:teuthology.orchestra.run.smithi148.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T09:52:05.950 INFO:teuthology.orchestra.run.smithi148.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-25T09:52:05.987 INFO:teuthology.orchestra.run.smithi188.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 4.6 MB/s | 744 kB 00:00 2024-04-25T09:52:06.000 INFO:teuthology.orchestra.run.smithi148.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-04-25T09:52:06.025 INFO:teuthology.orchestra.run.smithi148.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T09:52:06.037 INFO:teuthology.orchestra.run.smithi188.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 163 kB 00:00 2024-04-25T09:52:06.067 INFO:teuthology.orchestra.run.smithi148.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-25T09:52:06.093 INFO:teuthology.orchestra.run.smithi148.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-04-25T09:52:06.096 INFO:teuthology.orchestra.run.smithi188.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 2.1 MB/s | 127 kB 00:00 2024-04-25T09:52:06.118 INFO:teuthology.orchestra.run.smithi148.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 699 kB/s | 35 kB 00:00 2024-04-25T09:52:06.213 INFO:teuthology.orchestra.run.smithi188.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 4.2 MB/s | 501 kB 00:00 2024-04-25T09:52:06.226 INFO:teuthology.orchestra.run.smithi148.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-04-25T09:52:06.302 INFO:teuthology.orchestra.run.smithi148.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-25T09:52:06.369 INFO:teuthology.orchestra.run.smithi148.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-25T09:52:06.419 INFO:teuthology.orchestra.run.smithi148.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-25T09:52:06.578 INFO:teuthology.orchestra.run.smithi148.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 37 MB/s | 5.8 MB 00:00 2024-04-25T09:52:06.645 INFO:teuthology.orchestra.run.smithi148.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T09:52:06.695 INFO:teuthology.orchestra.run.smithi148.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T09:52:06.747 INFO:teuthology.orchestra.run.smithi188.stdout:(18/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 3.2 MB/s | 17 MB 00:05 2024-04-25T09:52:06.754 INFO:teuthology.orchestra.run.smithi148.stdout:(73/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-04-25T09:52:06.797 INFO:teuthology.orchestra.run.smithi188.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 910 kB/s | 45 kB 00:00 2024-04-25T09:52:06.804 INFO:teuthology.orchestra.run.smithi148.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T09:52:06.847 INFO:teuthology.orchestra.run.smithi188.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 2.6 MB/s | 131 kB 00:00 2024-04-25T09:52:06.879 INFO:teuthology.orchestra.run.smithi148.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-04-25T09:52:06.899 INFO:teuthology.orchestra.run.smithi188.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 3.0 MB/s | 162 kB 00:00 2024-04-25T09:52:06.904 INFO:teuthology.orchestra.run.smithi148.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T09:52:06.929 INFO:teuthology.orchestra.run.smithi148.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-25T09:52:06.955 INFO:teuthology.orchestra.run.smithi148.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T09:52:06.980 INFO:teuthology.orchestra.run.smithi148.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T09:52:07.005 INFO:teuthology.orchestra.run.smithi148.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-25T09:52:07.025 INFO:teuthology.orchestra.run.smithi188.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.5 MB/s | 321 kB 00:00 2024-04-25T09:52:07.072 INFO:teuthology.orchestra.run.smithi148.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T09:52:07.148 INFO:teuthology.orchestra.run.smithi148.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-04-25T09:52:07.173 INFO:teuthology.orchestra.run.smithi148.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T09:52:07.198 INFO:teuthology.orchestra.run.smithi148.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T09:52:07.215 INFO:teuthology.orchestra.run.smithi148.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T09:52:07.217 INFO:teuthology.orchestra.run.smithi188.stdout:(23/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 5.1 MB/s | 5.1 MB 00:01 2024-04-25T09:52:07.242 INFO:teuthology.orchestra.run.smithi188.stdout:(24/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 298 kB 00:00 2024-04-25T09:52:07.249 INFO:teuthology.orchestra.run.smithi148.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T09:52:07.259 INFO:teuthology.orchestra.run.smithi188.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 2.3 MB/s | 99 kB 00:00 2024-04-25T09:52:07.274 INFO:teuthology.orchestra.run.smithi148.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T09:52:07.293 INFO:teuthology.orchestra.run.smithi188.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T09:52:07.299 INFO:teuthology.orchestra.run.smithi148.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T09:52:07.324 INFO:teuthology.orchestra.run.smithi148.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T09:52:07.335 INFO:teuthology.orchestra.run.smithi188.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 4.0 MB/s | 170 kB 00:00 2024-04-25T09:52:07.350 INFO:teuthology.orchestra.run.smithi148.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T09:52:07.376 INFO:teuthology.orchestra.run.smithi188.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 755 kB/s | 31 kB 00:00 2024-04-25T09:52:07.400 INFO:teuthology.orchestra.run.smithi148.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-25T09:52:07.427 INFO:teuthology.orchestra.run.smithi188.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 2.7 MB/s | 138 kB 00:00 2024-04-25T09:52:07.450 INFO:teuthology.orchestra.run.smithi148.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-04-25T09:52:07.503 INFO:teuthology.orchestra.run.smithi148.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.1 MB/s | 221 kB 00:00 2024-04-25T09:52:07.553 INFO:teuthology.orchestra.run.smithi148.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-25T09:52:07.603 INFO:teuthology.orchestra.run.smithi148.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T09:52:07.653 INFO:teuthology.orchestra.run.smithi148.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T09:52:07.712 INFO:teuthology.orchestra.run.smithi148.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-04-25T09:52:07.728 INFO:teuthology.orchestra.run.smithi188.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 6.6 MB/s | 3.1 MB 00:00 2024-04-25T09:52:07.762 INFO:teuthology.orchestra.run.smithi148.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T09:52:07.795 INFO:teuthology.orchestra.run.smithi188.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 4.8 MB/s | 1.8 MB 00:00 2024-04-25T09:52:07.813 INFO:teuthology.orchestra.run.smithi148.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T09:52:07.863 INFO:teuthology.orchestra.run.smithi148.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-04-25T09:52:07.878 INFO:teuthology.orchestra.run.smithi188.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.9 MB/s | 248 kB 00:00 2024-04-25T09:52:07.928 INFO:teuthology.orchestra.run.smithi188.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 990 kB/s | 49 kB 00:00 2024-04-25T09:52:07.979 INFO:teuthology.orchestra.run.smithi188.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 307 kB/s | 15 kB 00:00 2024-04-25T09:52:08.054 INFO:teuthology.orchestra.run.smithi188.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.4 MB/s | 264 kB 00:00 2024-04-25T09:52:08.196 INFO:teuthology.orchestra.run.smithi188.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 5.2 MB/s | 748 kB 00:00 2024-04-25T09:52:08.246 INFO:teuthology.orchestra.run.smithi188.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 367 kB/s | 18 kB 00:00 2024-04-25T09:52:08.322 INFO:teuthology.orchestra.run.smithi148.stdout:(101/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 5.6 MB/s | 47 MB 00:08 2024-04-25T09:52:08.355 INFO:teuthology.orchestra.run.smithi188.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T09:52:08.472 INFO:teuthology.orchestra.run.smithi188.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-25T09:52:08.472 INFO:teuthology.orchestra.run.smithi148.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-25T09:52:08.497 INFO:teuthology.orchestra.run.smithi148.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-04-25T09:52:08.522 INFO:teuthology.orchestra.run.smithi188.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T09:52:08.572 INFO:teuthology.orchestra.run.smithi188.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T09:52:08.648 INFO:teuthology.orchestra.run.smithi188.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T09:52:08.731 INFO:teuthology.orchestra.run.smithi188.stdout:(43/152): ceph-mgr-diskprediction-local-19.0.0- 7.4 MB/s | 7.4 MB 00:01 2024-04-25T09:52:08.756 INFO:teuthology.orchestra.run.smithi188.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 983 kB/s | 106 kB 00:00 2024-04-25T09:52:08.789 INFO:teuthology.orchestra.run.smithi148.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-25T09:52:08.798 INFO:teuthology.orchestra.run.smithi188.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T09:52:08.849 INFO:teuthology.orchestra.run.smithi188.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T09:52:08.874 INFO:teuthology.orchestra.run.smithi188.stdout:(47/152): python3-pycparser-2.20-6.el9.noarch.r 948 kB/s | 135 kB 00:00 2024-04-25T09:52:08.889 INFO:teuthology.orchestra.run.smithi148.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-04-25T09:52:08.908 INFO:teuthology.orchestra.run.smithi188.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 11 MB/s | 689 kB 00:00 2024-04-25T09:52:08.950 INFO:teuthology.orchestra.run.smithi188.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T09:52:09.000 INFO:teuthology.orchestra.run.smithi188.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 7.3 MB/s | 943 kB 00:00 2024-04-25T09:52:09.026 INFO:teuthology.orchestra.run.smithi188.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-25T09:52:09.051 INFO:teuthology.orchestra.run.smithi188.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T09:52:09.076 INFO:teuthology.orchestra.run.smithi188.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-25T09:52:09.126 INFO:teuthology.orchestra.run.smithi188.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T09:52:09.177 INFO:teuthology.orchestra.run.smithi188.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T09:52:09.202 INFO:teuthology.orchestra.run.smithi188.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-25T09:52:09.243 INFO:teuthology.orchestra.run.smithi188.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-25T09:52:09.269 INFO:teuthology.orchestra.run.smithi188.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-04-25T09:52:09.303 INFO:teuthology.orchestra.run.smithi188.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T09:52:09.345 INFO:teuthology.orchestra.run.smithi188.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T09:52:09.370 INFO:teuthology.orchestra.run.smithi188.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-04-25T09:52:09.395 INFO:teuthology.orchestra.run.smithi188.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T09:52:09.421 INFO:teuthology.orchestra.run.smithi188.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T09:52:09.446 INFO:teuthology.orchestra.run.smithi188.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T09:52:09.471 INFO:teuthology.orchestra.run.smithi188.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-25T09:52:09.622 INFO:teuthology.orchestra.run.smithi188.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-25T09:52:09.697 INFO:teuthology.orchestra.run.smithi188.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-25T09:52:09.739 INFO:teuthology.orchestra.run.smithi188.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T09:52:09.773 INFO:teuthology.orchestra.run.smithi188.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T09:52:09.798 INFO:teuthology.orchestra.run.smithi188.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T09:52:09.865 INFO:teuthology.orchestra.run.smithi188.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T09:52:09.957 INFO:teuthology.orchestra.run.smithi188.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-04-25T09:52:09.973 INFO:teuthology.orchestra.run.smithi188.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-04-25T09:52:10.041 INFO:teuthology.orchestra.run.smithi188.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T09:52:10.158 INFO:teuthology.orchestra.run.smithi188.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-25T09:52:10.183 INFO:teuthology.orchestra.run.smithi188.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-25T09:52:10.200 INFO:teuthology.orchestra.run.smithi188.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-04-25T09:52:10.217 INFO:teuthology.orchestra.run.smithi188.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 567 kB/s | 19 kB 00:00 2024-04-25T09:52:10.242 INFO:teuthology.orchestra.run.smithi188.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T09:52:10.268 INFO:teuthology.orchestra.run.smithi188.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi188.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T09:52:10.402 INFO:teuthology.orchestra.run.smithi188.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T09:52:10.427 INFO:teuthology.orchestra.run.smithi188.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T09:52:10.452 INFO:teuthology.orchestra.run.smithi188.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T09:52:10.477 INFO:teuthology.orchestra.run.smithi188.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-04-25T09:52:10.503 INFO:teuthology.orchestra.run.smithi188.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T09:52:10.528 INFO:teuthology.orchestra.run.smithi188.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T09:52:10.553 INFO:teuthology.orchestra.run.smithi188.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T09:52:10.579 INFO:teuthology.orchestra.run.smithi188.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T09:52:10.604 INFO:teuthology.orchestra.run.smithi188.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T09:52:10.671 INFO:teuthology.orchestra.run.smithi188.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 626 kB/s | 42 kB 00:00 2024-04-25T09:52:10.721 INFO:teuthology.orchestra.run.smithi188.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-25T09:52:10.772 INFO:teuthology.orchestra.run.smithi188.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T09:52:10.822 INFO:teuthology.orchestra.run.smithi188.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-25T09:52:10.989 INFO:teuthology.orchestra.run.smithi188.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-04-25T09:52:11.014 INFO:teuthology.orchestra.run.smithi188.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-04-25T09:52:11.039 INFO:teuthology.orchestra.run.smithi188.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T09:52:11.064 INFO:teuthology.orchestra.run.smithi188.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T09:52:11.090 INFO:teuthology.orchestra.run.smithi188.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T09:52:11.115 INFO:teuthology.orchestra.run.smithi188.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T09:52:11.140 INFO:teuthology.orchestra.run.smithi188.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T09:52:11.191 INFO:teuthology.orchestra.run.smithi188.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-04-25T09:52:11.266 INFO:teuthology.orchestra.run.smithi188.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-25T09:52:11.708 INFO:teuthology.orchestra.run.smithi188.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T09:52:11.733 INFO:teuthology.orchestra.run.smithi188.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 662 kB/s | 309 kB 00:00 2024-04-25T09:52:11.842 INFO:teuthology.orchestra.run.smithi188.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-25T09:52:12.009 INFO:teuthology.orchestra.run.smithi188.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-04-25T09:52:12.134 INFO:teuthology.orchestra.run.smithi188.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-04-25T09:52:12.243 INFO:teuthology.orchestra.run.smithi188.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-04-25T09:52:12.310 INFO:teuthology.orchestra.run.smithi188.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-04-25T09:52:12.427 INFO:teuthology.orchestra.run.smithi188.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 824 kB/s | 151 kB 00:00 2024-04-25T09:52:12.502 INFO:teuthology.orchestra.run.smithi188.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-04-25T09:52:12.636 INFO:teuthology.orchestra.run.smithi188.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-04-25T09:52:12.652 INFO:teuthology.orchestra.run.smithi188.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-04-25T09:52:12.719 INFO:teuthology.orchestra.run.smithi188.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-04-25T09:52:12.736 INFO:teuthology.orchestra.run.smithi188.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T09:52:12.861 INFO:teuthology.orchestra.run.smithi188.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-25T09:52:12.904 INFO:teuthology.orchestra.run.smithi188.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-25T09:52:12.962 INFO:teuthology.orchestra.run.smithi188.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T09:52:12.973 INFO:teuthology.orchestra.run.smithi148.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 69 kB/s | 309 kB 00:04 2024-04-25T09:52:12.987 INFO:teuthology.orchestra.run.smithi188.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 423 kB/s | 35 kB 00:00 2024-04-25T09:52:13.006 INFO:teuthology.orchestra.run.smithi148.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 121 kB/s | 548 kB 00:04 2024-04-25T09:52:13.031 INFO:teuthology.orchestra.run.smithi148.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-04-25T09:52:13.054 INFO:teuthology.orchestra.run.smithi188.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T09:52:13.113 INFO:teuthology.orchestra.run.smithi188.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-25T09:52:13.147 INFO:teuthology.orchestra.run.smithi188.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T09:52:13.165 INFO:teuthology.orchestra.run.smithi148.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-04-25T09:52:13.190 INFO:teuthology.orchestra.run.smithi148.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-04-25T09:52:13.197 INFO:teuthology.orchestra.run.smithi188.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-25T09:52:13.224 INFO:teuthology.orchestra.run.smithi148.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T09:52:13.230 INFO:teuthology.orchestra.run.smithi188.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-25T09:52:13.297 INFO:teuthology.orchestra.run.smithi188.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T09:52:13.373 INFO:teuthology.orchestra.run.smithi188.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-04-25T09:52:13.398 INFO:teuthology.orchestra.run.smithi188.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-25T09:52:13.482 INFO:teuthology.orchestra.run.smithi188.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-04-25T09:52:13.532 INFO:teuthology.orchestra.run.smithi188.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-25T09:52:13.949 INFO:teuthology.orchestra.run.smithi188.stdout:(131/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 5.6 MB/s | 47 MB 00:08 2024-04-25T09:52:13.983 INFO:teuthology.orchestra.run.smithi188.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 128 kB/s | 58 kB 00:00 2024-04-25T09:52:13.999 INFO:teuthology.orchestra.run.smithi188.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 152 kB/s | 79 kB 00:00 2024-04-25T09:52:14.066 INFO:teuthology.orchestra.run.smithi188.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-25T09:52:14.092 INFO:teuthology.orchestra.run.smithi188.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 978 kB/s | 90 kB 00:00 2024-04-25T09:52:14.150 INFO:teuthology.orchestra.run.smithi188.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-25T09:52:14.175 INFO:teuthology.orchestra.run.smithi188.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-04-25T09:52:14.234 INFO:teuthology.orchestra.run.smithi188.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T09:52:14.259 INFO:teuthology.orchestra.run.smithi188.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-25T09:52:14.326 INFO:teuthology.orchestra.run.smithi188.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2024-04-25T09:52:14.351 INFO:teuthology.orchestra.run.smithi188.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-25T09:52:14.377 INFO:teuthology.orchestra.run.smithi188.stdout:(142/152): python3-pecan-1.4.2-3.el9.noarch.rpm 637 kB/s | 272 kB 00:00 2024-04-25T09:52:14.410 INFO:teuthology.orchestra.run.smithi188.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T09:52:14.436 INFO:teuthology.orchestra.run.smithi188.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-25T09:52:14.494 INFO:teuthology.orchestra.run.smithi188.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T09:52:14.520 INFO:teuthology.orchestra.run.smithi188.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T09:52:14.545 INFO:teuthology.orchestra.run.smithi188.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-04-25T09:52:14.629 INFO:teuthology.orchestra.run.smithi188.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T09:52:14.812 INFO:teuthology.orchestra.run.smithi188.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-04-25T09:52:14.921 INFO:teuthology.orchestra.run.smithi188.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T09:52:15.038 INFO:teuthology.orchestra.run.smithi188.stdout:(151/152): librados2-19.0.0-1628.g97ae6cc9.el9. 6.5 MB/s | 3.4 MB 00:00 2024-04-25T09:52:15.129 INFO:teuthology.orchestra.run.smithi188.stdout:(152/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-04-25T09:52:15.133 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:52:15.133 INFO:teuthology.orchestra.run.smithi188.stdout:Total 13 MB/s | 215 MB 00:16 2024-04-25T09:52:16.115 INFO:teuthology.orchestra.run.smithi148.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 190 kB/s | 548 kB 00:02 2024-04-25T09:52:16.157 INFO:teuthology.orchestra.run.smithi148.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 282 kB/s | 838 kB 00:02 2024-04-25T09:52:16.216 INFO:teuthology.orchestra.run.smithi148.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 617 kB/s | 4.4 MB 00:07 2024-04-25T09:52:16.237 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-25T09:52:16.381 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-25T09:52:16.381 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-25T09:52:16.450 INFO:teuthology.orchestra.run.smithi148.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 64 kB/s | 15 kB 00:00 2024-04-25T09:52:16.475 INFO:teuthology.orchestra.run.smithi148.stdout:(116/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 121 kB/s | 43 kB 00:00 2024-04-25T09:52:16.500 INFO:teuthology.orchestra.run.smithi148.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 94 kB/s | 32 kB 00:00 2024-04-25T09:52:18.110 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-25T09:52:18.110 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-25T09:52:19.483 INFO:teuthology.orchestra.run.smithi148.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 57 kB/s | 172 kB 00:03 2024-04-25T09:52:19.508 INFO:teuthology.orchestra.run.smithi148.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 101 kB/s | 303 kB 00:03 2024-04-25T09:52:19.533 INFO:teuthology.orchestra.run.smithi148.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 118 kB/s | 358 kB 00:03 2024-04-25T09:52:21.059 INFO:teuthology.orchestra.run.smithi148.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 22 kB/s | 35 kB 00:01 2024-04-25T09:52:21.084 INFO:teuthology.orchestra.run.smithi148.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 130 kB/s | 206 kB 00:01 2024-04-25T09:52:21.143 INFO:teuthology.orchestra.run.smithi148.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.3 MB/s | 2.0 MB 00:01 2024-04-25T09:52:21.376 INFO:teuthology.orchestra.run.smithi148.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 37 kB/s | 11 kB 00:00 2024-04-25T09:52:21.393 INFO:teuthology.orchestra.run.smithi148.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 71 kB/s | 18 kB 00:00 2024-04-25T09:52:21.419 INFO:teuthology.orchestra.run.smithi148.stdout:(126/152): python3-grpcio-tools-1.46.7-10.el9.x 402 kB/s | 144 kB 00:00 2024-04-25T09:52:21.452 INFO:teuthology.orchestra.run.smithi148.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-04-25T09:52:21.469 INFO:teuthology.orchestra.run.smithi148.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T09:52:21.486 INFO:teuthology.orchestra.run.smithi148.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-04-25T09:52:21.603 INFO:teuthology.orchestra.run.smithi148.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-04-25T09:52:21.628 INFO:teuthology.orchestra.run.smithi148.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 555 kB/s | 79 kB 00:00 2024-04-25T09:52:21.662 INFO:teuthology.orchestra.run.smithi148.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-04-25T09:52:23.620 INFO:teuthology.orchestra.run.smithi148.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 8.4 kB/s | 16 kB 00:01 2024-04-25T09:52:23.637 INFO:teuthology.orchestra.run.smithi148.stdout:(134/152): python3-natsort-7.1.1-5.el9.noarch.r 28 kB/s | 58 kB 00:02 2024-04-25T09:52:23.662 INFO:teuthology.orchestra.run.smithi148.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 134 kB/s | 272 kB 00:02 2024-04-25T09:52:24.011 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-25T09:52:24.138 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:52:24.382 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:52:24.437 INFO:teuthology.orchestra.run.smithi148.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 35 kB/s | 28 kB 00:00 2024-04-25T09:52:24.444 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:52:24.454 INFO:teuthology.orchestra.run.smithi148.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 39 kB/s | 31 kB 00:00 2024-04-25T09:52:24.480 INFO:teuthology.orchestra.run.smithi148.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 105 kB/s | 90 kB 00:00 2024-04-25T09:52:24.485 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:52:24.505 INFO:teuthology.orchestra.run.smithi148.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T09:52:24.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:52:24.530 INFO:teuthology.orchestra.run.smithi148.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-04-25T09:52:24.554 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:52:24.555 INFO:teuthology.orchestra.run.smithi148.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T09:52:24.814 INFO:teuthology.orchestra.run.smithi148.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 165 kB/s | 51 kB 00:00 2024-04-25T09:52:24.931 INFO:teuthology.orchestra.run.smithi148.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 239 kB/s | 90 kB 00:00 2024-04-25T09:52:24.941 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:52:24.956 INFO:teuthology.orchestra.run.smithi148.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 539 kB/s | 230 kB 00:00 2024-04-25T09:52:24.981 INFO:teuthology.orchestra.run.smithi148.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-25T09:52:24.985 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:52:25.007 INFO:teuthology.orchestra.run.smithi148.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T09:52:25.032 INFO:teuthology.orchestra.run.smithi148.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi148.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-25T09:52:25.237 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:52:25.265 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:52:25.299 INFO:teuthology.orchestra.run.smithi148.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-04-25T09:52:25.421 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:52:25.458 INFO:teuthology.orchestra.run.smithi148.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-04-25T09:52:25.471 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:52:25.499 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:52:25.500 INFO:teuthology.orchestra.run.smithi148.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-25T09:52:25.603 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:52:25.608 INFO:teuthology.orchestra.run.smithi148.stdout:(152/152): librados2-19.0.0-1628.g97ae6cc9.el9. 5.6 MB/s | 3.4 MB 00:00 2024-04-25T09:52:25.610 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:52:25.611 INFO:teuthology.orchestra.run.smithi148.stdout:Total 7.4 MB/s | 215 MB 00:29 2024-04-25T09:52:25.625 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:52:25.679 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:52:25.701 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:52:25.724 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:52:25.910 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:52:25.942 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:52:26.049 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:52:26.148 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:52:26.192 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:52:26.226 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:52:26.301 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:52:26.334 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:52:26.377 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:52:26.410 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:52:26.438 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:52:26.508 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:52:26.561 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:52:26.591 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:52:26.626 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:52:26.702 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:52:26.713 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-25T09:52:26.737 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:52:26.781 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:52:26.809 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:52:26.859 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-25T09:52:26.860 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-25T09:52:26.881 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:52:26.924 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:52:27.000 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:52:27.136 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:52:27.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:52:27.737 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:52:27.823 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:52:27.852 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:52:27.894 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:52:27.933 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:52:27.975 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:52:28.130 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:52:28.186 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:52:28.241 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:52:28.323 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:52:28.361 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:52:28.398 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:52:28.446 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:52:28.481 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:52:28.524 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:52:28.566 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:52:28.700 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-25T09:52:28.701 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-25T09:52:28.705 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:52:28.755 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:52:28.785 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:52:28.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:52:28.923 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:52:28.960 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:52:29.000 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:52:29.026 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:52:29.051 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:52:29.127 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:52:29.255 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:52:29.291 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:52:29.320 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:52:29.354 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:52:29.393 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:52:29.433 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:52:29.463 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:52:29.498 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:52:29.531 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:52:29.573 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:52:29.607 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:52:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:52:29.738 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:52:29.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:52:29.817 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:52:29.916 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:52:29.971 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:52:30.904 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:52:30.960 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:52:31.060 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:52:31.103 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:52:31.290 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:52:31.318 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:52:31.395 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:52:32.057 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:52:32.167 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:52:33.427 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:52:33.522 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:52:33.559 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:52:33.592 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:52:33.904 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:52:33.946 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:52:34.037 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:52:34.068 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:52:34.104 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:52:34.559 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:52:34.591 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:52:34.646 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:52:34.662 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-25T09:52:34.676 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:52:34.762 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:52:34.809 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:52:34.879 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:52:34.913 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:52:34.938 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:52:34.981 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:52:35.356 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:52:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:52:35.527 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:52:35.557 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:52:35.639 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:52:35.678 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:52:35.829 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:52:35.874 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:52:35.903 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:52:36.005 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:52:36.028 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:52:36.091 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:52:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:52:36.139 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:52:36.479 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:52:36.574 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:52:36.620 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:52:36.654 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:52:36.719 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:52:36.753 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:52:36.797 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:52:36.829 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:52:36.857 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:52:36.902 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:52:36.926 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:52:36.988 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:52:37.006 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:52:37.018 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:52:37.053 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:52:37.129 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:52:37.351 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:52:37.427 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:52:37.601 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:52:37.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:52:37.662 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:52:37.693 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:52:37.694 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:52:37.694 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:52:37.694 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:37.727 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:52:38.196 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:52:38.282 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:52:38.311 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:52:38.354 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:52:38.392 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:52:38.442 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:52:38.601 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:52:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:52:38.837 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:52:38.873 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:52:38.921 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:52:38.956 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:52:38.999 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:52:39.042 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:52:39.179 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:52:39.255 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:52:39.277 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:52:39.318 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:52:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:52:39.452 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:52:39.492 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:52:39.517 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:52:39.542 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:52:39.617 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:52:39.745 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:52:39.783 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:52:39.810 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:52:39.845 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:52:39.884 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:52:39.915 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:52:39.945 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:52:39.980 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:52:40.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:52:40.055 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:52:40.090 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:52:40.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:52:40.236 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:52:40.268 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:52:40.314 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:52:40.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:52:40.460 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:52:41.199 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:52:41.249 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:52:41.283 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:52:41.384 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:52:41.433 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:52:41.520 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:52:41.568 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:52:41.748 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:52:41.782 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:52:41.869 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:52:42.526 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:52:42.648 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:52:43.897 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:52:43.978 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:52:44.022 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:52:44.377 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:52:44.467 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:52:44.498 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:52:44.534 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:52:44.989 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:52:45.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:52:45.083 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:52:45.106 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:52:47.341 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:52:47.561 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:52:48.104 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:52:48.147 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:52:48.246 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:52:48.246 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:52:48.249 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:52:48.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:52:48.273 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:52:49.176 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:52:49.176 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-04-25T09:52:49.176 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-04-25T09:52:49.177 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-04-25T09:52:49.177 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-04-25T09:52:49.177 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-04-25T09:52:49.177 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-04-25T09:52:49.177 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-04-25T09:52:49.177 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:49.350 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:52:49.365 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:52:49.365 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:49.365 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:52:49.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:50.853 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:52:50.878 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:52:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:52:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:52:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:52:50.879 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:51.086 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:52:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:52:51.109 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:51.109 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:52:51.109 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:52:51.109 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:52:51.109 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:52:51.576 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:52:51.576 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:51.576 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:52:51.576 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:52:51.577 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:52:51.577 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:51.701 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:52:51.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:52:51.814 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:52:52.000 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:52:52.055 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:52:52.133 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:52:52.222 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:52:52.363 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:52:52.405 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:52:52.437 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:52:52.465 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:52:52.873 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:52:52.905 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:52:52.936 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:52:52.997 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:52:53.028 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:52:53.259 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:52:53.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:52:54.257 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:52:54.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:52:54.390 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:52:54.551 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:52:54.573 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:52:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:52:54.598 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:54.598 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:52:54.598 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:52:54.598 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:52:54.598 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:54.640 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:52:54.652 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:52:54.797 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:52:55.650 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:52:55.684 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:52:55.707 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:52:55.707 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:55.707 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:52:55.707 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:52:55.707 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:52:55.708 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:55.752 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:52:55.774 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:52:55.774 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:55.774 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:52:55.774 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:55.819 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:52:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:52:55.842 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:55.842 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:52:55.842 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:52:55.842 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:52:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:56.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:52:56.167 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:52:56.168 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:56.168 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:52:56.168 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:52:56.168 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:52:56.168 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:52:59.677 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-04-25T09:52:59.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:52:59.844 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:52:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T09:52:59.864 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:52:59.864 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T09:52:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:00.607 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:53:00.648 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:53:00.688 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:53:00.721 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:53:00.756 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:53:00.941 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:53:00.986 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:53:01.016 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:53:01.371 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:53:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T09:53:01.397 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:01.397 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T09:53:01.397 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:53:01.397 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T09:53:01.397 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:01.605 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:53:01.628 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T09:53:01.628 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:01.628 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T09:53:01.628 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:53:01.628 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T09:53:01.628 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:01.644 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:53:01.689 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:53:01.709 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:53:02.072 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:53:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T09:53:02.095 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:02.095 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T09:53:02.095 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:53:02.095 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T09:53:02.095 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:02.202 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T09:53:02.252 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T09:53:02.307 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T09:53:02.495 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T09:53:02.540 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T09:53:02.544 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:53:02.618 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T09:53:02.645 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:53:02.685 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:53:02.707 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:53:02.769 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:53:02.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:53:02.815 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:53:02.849 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:53:02.890 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T09:53:02.899 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:53:02.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T09:53:02.932 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:53:02.932 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:53:02.949 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T09:53:03.019 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:53:03.089 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:53:03.392 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:53:03.423 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T09:53:03.454 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T09:53:03.515 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T09:53:03.562 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:53:03.794 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T09:53:03.822 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:53:04.802 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T09:53:04.830 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:53:04.934 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T09:53:05.102 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T09:53:05.124 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:53:05.147 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T09:53:05.147 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:05.147 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T09:53:05.147 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:53:05.147 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T09:53:05.147 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:05.201 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:53:05.211 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T09:53:05.309 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T09:53:05.357 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T09:53:06.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T09:53:06.252 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:53:06.275 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T09:53:06.276 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:06.276 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T09:53:06.276 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:53:06.276 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T09:53:06.276 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:06.320 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:53:06.342 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T09:53:06.342 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:06.342 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T09:53:06.342 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:06.387 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:53:06.410 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T09:53:06.410 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:06.411 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T09:53:06.411 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:53:06.411 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T09:53:06.411 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:06.721 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:53:06.744 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T09:53:06.744 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T09:53:06.744 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T09:53:06.744 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:53:06.744 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T09:53:06.745 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:53:08.682 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:53:08.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:53:08.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:53:08.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:53:08.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:53:08.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:53:08.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:53:08.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:53:08.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:53:08.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:53:08.703 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:53:08.703 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:53:08.703 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:53:08.703 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:53:08.703 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:53:08.703 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:53:08.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:53:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:53:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:53:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:53:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:53:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:53:08.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:53:08.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:53:08.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:53:08.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:53:08.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:53:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:53:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:53:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:53:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:53:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:53:08.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:53:08.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:53:08.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:53:08.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:53:08.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:53:08.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:53:08.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:53:08.710 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:53:08.710 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:53:08.710 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:53:08.710 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:53:08.710 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:53:08.710 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:53:08.711 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:53:08.711 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:53:08.711 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:53:08.711 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:53:10.763 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:53:10.763 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:10.763 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-04-25T09:53:10.764 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.764 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.764 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:53:10.764 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:53:10.764 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-04-25T09:53:10.764 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:53:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:53:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:53:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:53:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.766 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.766 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.766 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.766 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.766 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.767 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:53:10.768 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:53:10.769 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:53:10.770 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:53:10.771 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:53:10.772 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:53:10.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:53:10.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:53:10.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:53:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:53:10.777 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:53:10.778 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:53:10.778 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:53:10.778 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:10.778 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-25T09:53:11.169 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:11.208 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T09:53:11.249 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T09:53:11.290 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T09:53:11.322 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T09:53:11.349 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T09:53:11.541 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T09:53:11.586 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T09:53:11.616 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T09:53:12.238 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:53:12.272 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:53:12.292 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T09:53:13.139 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T09:53:13.235 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T09:53:13.277 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T09:53:13.359 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T09:53:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T09:53:13.389 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T09:53:13.506 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:53:13.506 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:53:13.580 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T09:53:13.654 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:53:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:53:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T09:53:18.788 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T09:53:18.789 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T09:53:18.790 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T09:53:18.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T09:53:18.792 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T09:53:18.793 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T09:53:18.794 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T09:53:18.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T09:53:18.796 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T09:53:18.797 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T09:53:18.798 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T09:53:18.799 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T09:53:18.800 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T09:53:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T09:53:20.968 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.969 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.970 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T09:53:20.971 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.972 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T09:53:20.973 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T09:53:20.974 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T09:53:20.975 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T09:53:20.976 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T09:53:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T09:53:20.977 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T09:53:20.978 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T09:53:20.979 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T09:53:20.980 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T09:53:20.981 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T09:53:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T09:53:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T09:53:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T09:53:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T09:53:20.982 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:20.982 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-25T09:53:21.338 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:21.339 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:53:21.486 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:53:21.507 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:53:21.508 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:53:21.508 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:53:21.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:53:21.655 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T09:53:21.681 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:53:21.681 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T09:53:21.681 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T09:53:21.683 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T09:53:21.683 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:21.684 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:53:21.711 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:21.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T09:53:21.751 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T09:53:21.751 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:21.751 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:53:21.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:53:21.840 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:21.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T09:53:21.868 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T09:53:21.934 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T09:53:21.934 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:21.934 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:53:21.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:53:22.021 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:22.021 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T09:53:22.046 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T09:53:22.111 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T09:53:22.111 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:22.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:53:22.134 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:53:22.198 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:22.198 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T09:53:22.223 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T09:53:22.288 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T09:53:22.379 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T09:53:22.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-04-25T09:53:22.381 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-04-25T09:53:22.405 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T09:53:22.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:53:22.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T09:53:22.470 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T09:53:22.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:53:22.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T09:53:22.541 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T09:53:22.541 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T09:53:22.541 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T09:53:22.541 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:22.542 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:53:22.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:22.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:22.724 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:22.724 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T09:53:22.740 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:22.850 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:22.880 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T09:53:22.880 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T09:53:22.880 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673039', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG+YL4I7Su40jh+Xep6ALSRVogUymF8B8SQK6N9Pzc+CIIti8+X7NMBP9JoTeL/1ngw4FHqQRhyZeRdQ9u4EkjY=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE67rz3gBp+UYTCR0nL1yumg2dDICEjI5e5neAvMcscRyI6LMon5FIPkeN7zQz/NHykplfLXQuqieQhKU+lsdNY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T09:53:22.881 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:53:22.883 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T09:53:22.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:53:22.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T09:53:22.994 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:22.994 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:53:23.017 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:53:23.017 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-04-25T09:53:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:53:23.071 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T09:53:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:23.071 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-25 09:53:14.033692603 +0000 2024-04-25T09:53:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-25 09:51:14.738187972 +0000 2024-04-25T09:53:23.072 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-25 09:51:14.738187972 +0000 2024-04-25T09:53:23.072 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-25 09:51:14.738187972 +0000 2024-04-25T09:53:23.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:53:23.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T09:53:23.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T09:53:23.134 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000282856 s, 1.8 MB/s 2024-04-25T09:53:23.135 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:53:23.192 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-25 09:53:14.033692603 +0000 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-25 09:51:15.077178892 +0000 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-25 09:51:15.077178892 +0000 2024-04-25T09:53:23.247 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-25 09:51:15.077178892 +0000 2024-04-25T09:53:23.248 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:53:23.310 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T09:53:23.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T09:53:23.311 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000175557 s, 2.9 MB/s 2024-04-25T09:53:23.313 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:53:23.368 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-04-25T09:53:23.423 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-25 09:53:14.034692578 +0000 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-25 09:51:15.415169838 +0000 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-25 09:51:15.415169838 +0000 2024-04-25T09:53:23.424 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-25 09:51:15.415169838 +0000 2024-04-25T09:53:23.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:53:23.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T09:53:23.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T09:53:23.487 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000209005 s, 2.4 MB/s 2024-04-25T09:53:23.489 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:53:23.544 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-04-25T09:53:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:53:23.598 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-25 09:53:14.034692578 +0000 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-25 09:51:15.745160999 +0000 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-25 09:51:15.745160999 +0000 2024-04-25T09:53:23.599 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-25 09:51:15.745160999 +0000 2024-04-25T09:53:23.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:53:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T09:53:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T09:53:23.661 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000198824 s, 2.6 MB/s 2024-04-25T09:53:23.662 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:53:23.717 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:23.717 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:23.718 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T09:53:23.735 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T09:53:23.736 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-04-25T09:53:23.793 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-25 09:53:03.460925037 +0000 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-25 09:51:14.775964321 +0000 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-25 09:51:14.775964321 +0000 2024-04-25T09:53:23.794 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-25 09:51:14.775964321 +0000 2024-04-25T09:53:23.795 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T09:53:23.860 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-25T09:53:23.860 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-25T09:53:23.860 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000346052 s, 1.5 MB/s 2024-04-25T09:53:23.861 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T09:53:23.919 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-04-25T09:53:23.975 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-25 09:53:03.461925018 +0000 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-25 09:51:15.130957159 +0000 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-25 09:51:15.130957159 +0000 2024-04-25T09:53:23.976 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-25 09:51:15.130957159 +0000 2024-04-25T09:53:23.976 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T09:53:24.040 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-25T09:53:24.040 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-25T09:53:24.041 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000220336 s, 2.3 MB/s 2024-04-25T09:53:24.042 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T09:53:24.099 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-04-25T09:53:24.155 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-25 09:53:03.461925018 +0000 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-25 09:51:15.495949797 +0000 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-25 09:51:15.495949797 +0000 2024-04-25T09:53:24.156 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-25 09:51:15.495949797 +0000 2024-04-25T09:53:24.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T09:53:24.220 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-25T09:53:24.220 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-25T09:53:24.220 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000168114 s, 3.0 MB/s 2024-04-25T09:53:24.221 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T09:53:24.279 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-25 09:53:03.462924999 +0000 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-25 09:51:15.855942535 +0000 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-25 09:51:15.855942535 +0000 2024-04-25T09:53:24.335 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-25 09:51:15.855942535 +0000 2024-04-25T09:53:24.336 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T09:53:24.398 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-25T09:53:24.398 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-25T09:53:24.398 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000210557 s, 2.4 MB/s 2024-04-25T09:53:24.399 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T09:53:24.455 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:24.456 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:53:24.456 INFO:tasks.ceph:Generating config... 2024-04-25T09:53:24.458 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T09:53:24.458 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T09:53:24.458 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T09:53:24.458 INFO:tasks.ceph:[client] pid file = 2024-04-25T09:53:24.459 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T09:53:24.459 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T09:53:24.459 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T09:53:24.459 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T09:53:24.459 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T09:53:24.460 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T09:53:24.461 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T09:53:24.461 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T09:53:24.461 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T09:53:24.461 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T09:53:24.461 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T09:53:24.461 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T09:53:24.462 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T09:53:24.462 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T09:53:24.462 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T09:53:24.462 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T09:53:24.462 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T09:53:24.463 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T09:53:24.464 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T09:53:24.464 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T09:53:24.464 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T09:53:24.464 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T09:53:24.464 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T09:53:24.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T09:53:24.521 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T09:53:24.525 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T09:53:24.567 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:53:24.632 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.148'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-04-25T09:53:24.633 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T09:53:24.633 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:24.633 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:53:24.688 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.148 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:24.773 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:24.773 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: generated fsid 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T09:53:24.773 INFO:teuthology.orchestra.run.smithi148.stdout:setting min_mon_release = quincy 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 0 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:fsid 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:last_changed 2024-04-25T09:53:24.772073+0000 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:created 2024-04-25T09:53:24.772073+0000 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:min_mon_release 17 (quincy) 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:election_strategy: 1 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:0: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.a 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-04-25T09:53:24.774 INFO:teuthology.orchestra.run.smithi148.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-04-25T09:53:24.775 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T09:53:24.776 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T09:53:24.790 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 086ed9ea-68e4-4e62-83c5-c0821c62dd17... 2024-04-25T09:53:24.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:53:24.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:53:24.871 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.872 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:53:24.873 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-25T09:53:24.874 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:53:24.875 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-25T09:53:24.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-25T09:53:24.877 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:53:24.878 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:53:24.879 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-25T09:53:24.880 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-04-25T09:53:24.881 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: mds cache max size = 1000 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: mds log max segments = 1 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = True 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T09:53:24.882 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-25T09:53:24.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.884 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.885 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-25T09:53:24.886 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:53:24.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T09:53:24.888 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:53:24.889 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.890 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-25T09:53:24.891 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-25T09:53:24.892 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: mds cache max size = 1000 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: mds log max segments = 1 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T09:53:24.893 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T09:53:24.895 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T09:53:24.895 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T09:53:24.987 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T09:53:24.988 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:24.988 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T09:53:25.002 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:25.002 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T09:53:25.057 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:53:25.057 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:25.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:53:25.057 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:53:25.130 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:25.130 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:25.185 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:53:25.185 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:25.185 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T09:53:25.185 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T09:53:25.222 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:25.223 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:25.280 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T09:53:25.280 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T09:53:25.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:53:25.329 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T09:53:25.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:53:25.390 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T09:53:25.393 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T09:53:25.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:53:25.441 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T09:53:25.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T09:53:25.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:53:25.556 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T09:53:25.558 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T09:53:25.581 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:53:25.632 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T09:53:25.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T09:53:25.659 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:53:25.746 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T09:53:25.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T09:53:25.774 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T09:53:25.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T09:53:25.813 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T09:53:25.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T09:53:25.904 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T09:53:25.916 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T09:53:25.955 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T09:53:25.966 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T09:53:26.049 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T09:53:26.060 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T09:53:26.061 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T09:53:26.061 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T09:53:26.084 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:26.085 INFO:tasks.ceph:role: osd.0 2024-04-25T09:53:26.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:53:26.085 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:26.156 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:26.157 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:26.157 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:26.157 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:26.661 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-25T09:53:26.664 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-25T09:53:26.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T09:53:26.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T09:53:26.851 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T09:53:26.917 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:26.918 INFO:tasks.ceph:role: osd.1 2024-04-25T09:53:26.918 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:53:26.918 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:26.990 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:26.990 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:27.433 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-25T09:53:27.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-25T09:53:27.435 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T09:53:27.529 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T09:53:27.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T09:53:27.621 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:27.621 INFO:tasks.ceph:role: osd.2 2024-04-25T09:53:27.621 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:53:27.621 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:27.692 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:27.693 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:28.185 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-25T09:53:28.186 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-25T09:53:28.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T09:53:28.288 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T09:53:28.317 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T09:53:28.380 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:28.380 INFO:tasks.ceph:role: osd.3 2024-04-25T09:53:28.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:53:28.381 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:53:28.452 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:28.452 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:28.453 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:28.453 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:28.453 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:28.453 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:28.453 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:28.453 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:28.970 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-25T09:53:28.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-25T09:53:28.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T09:53:29.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T09:53:29.099 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:29.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:29.182+0000 7fc961617640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T09:53:29.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:29.186+0000 7fc961617640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T09:53:29.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:29.186+0000 7fc961617640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:53:29.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:29.186+0000 7fc961617640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:53:29.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:29.186+0000 7fc961617640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T09:53:29.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:29.192+0000 7fc961617640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T09:53:32.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T09:53:32.074 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:32.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:32.160+0000 7f8d833a3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T09:53:32.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:32.160+0000 7f8d833a3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T09:53:32.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:32.160+0000 7f8d833a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:53:32.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:32.160+0000 7f8d833a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:53:32.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:32.160+0000 7f8d833a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T09:53:32.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:32.161+0000 7f8d833a3640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T09:53:35.016 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T09:53:35.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:35.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:35.131+0000 7fcea1bc4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T09:53:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:35.132+0000 7fcea1bc4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T09:53:35.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:35.132+0000 7fcea1bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:53:35.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:35.132+0000 7fcea1bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:53:35.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:35.132+0000 7fcea1bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T09:53:35.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:35.132+0000 7fcea1bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T09:53:37.981 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T09:53:38.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:38.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:38.095+0000 7f3b466dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T09:53:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:38.095+0000 7f3b466dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T09:53:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:38.095+0000 7f3b466dd640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:53:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:38.095+0000 7f3b466dd640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:53:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:38.095+0000 7f3b466dd640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T09:53:38.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:38.096+0000 7f3b466dd640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T09:53:40.955 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T09:53:40.982 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T09:53:41.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:41.013 INFO:tasks.ceph:role: osd.4 2024-04-25T09:53:41.013 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:53:41.013 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T09:53:41.088 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:41.088 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:41.088 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:41.088 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:41.089 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:41.089 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:41.089 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:41.089 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:41.089 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:41.089 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:41.616 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-25T09:53:41.619 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-25T09:53:41.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T09:53:41.758 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T09:53:41.792 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T09:53:41.858 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:41.858 INFO:tasks.ceph:role: osd.5 2024-04-25T09:53:41.858 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:53:41.858 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T09:53:41.931 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:41.931 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:41.931 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:41.931 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:41.931 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:41.931 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:41.932 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:41.932 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:41.932 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:41.932 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:42.383 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-25T09:53:42.385 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-25T09:53:42.385 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T09:53:42.496 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T09:53:42.529 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T09:53:42.595 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:42.595 INFO:tasks.ceph:role: osd.6 2024-04-25T09:53:42.595 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:53:42.595 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T09:53:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:42.671 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:42.671 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:42.671 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:42.671 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:43.149 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-25T09:53:43.151 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-25T09:53:43.151 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T09:53:43.286 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T09:53:43.319 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T09:53:43.387 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T09:53:43.387 INFO:tasks.ceph:role: osd.7 2024-04-25T09:53:43.387 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:53:43.387 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T09:53:43.460 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T09:53:43.461 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T09:53:43.949 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-04-25T09:53:43.952 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-04-25T09:53:43.952 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T09:53:44.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T09:53:44.120 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:44.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:44.230+0000 7f5678c0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T09:53:44.232 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:44.231+0000 7f5678c0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T09:53:44.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:44.231+0000 7f5678c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:53:44.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:44.231+0000 7f5678c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:53:44.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:44.231+0000 7f5678c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T09:53:44.233 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:44.231+0000 7f5678c0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T09:53:47.085 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T09:53:47.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:47.203 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:47.201+0000 7fc52bfd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T09:53:47.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:47.202+0000 7fc52bfd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T09:53:47.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:47.202+0000 7fc52bfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:53:47.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:47.202+0000 7fc52bfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:53:47.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:47.202+0000 7fc52bfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T09:53:47.204 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:47.203+0000 7fc52bfd8640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T09:53:50.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T09:53:50.070 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:50.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:50.156+0000 7fa25bd4c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T09:53:50.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:50.156+0000 7fa25bd4c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T09:53:50.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:50.156+0000 7fa25bd4c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:53:50.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:50.156+0000 7fa25bd4c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:53:50.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:50.156+0000 7fa25bd4c640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T09:53:50.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:50.157+0000 7fa25bd4c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T09:53:52.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:52.886 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:53:53.009 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T09:53:53.037 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:53.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:53.161+0000 7fb8c8c2f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T09:53:53.165 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:53.161+0000 7fb8c8c2f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T09:53:53.165 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:53.161+0000 7fb8c8c2f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:53:53.165 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:53.162+0000 7fb8c8c2f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:53:53.165 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:53.162+0000 7fb8c8c2f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T09:53:53.165 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:53.162+0000 7fb8c8c2f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T09:53:56.007 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T09:53:56.070 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T09:53:56.071 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T09:53:56.102 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.102 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T09:53:56.173 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T09:53:56.240 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.240 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T09:53:56.306 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T09:53:56.371 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T09:53:56.437 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.437 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T09:53:56.501 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T09:53:56.526 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.526 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T09:53:56.590 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.590 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T09:53:56.655 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.655 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T09:53:56.718 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.718 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T09:53:56.781 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.782 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T09:53:56.849 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.849 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T09:53:56.912 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.912 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T09:53:56.930 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:56.930 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T09:53:56.986 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:56.986 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T09:53:57.001 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:57.001 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T09:53:57.055 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T09:53:57.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:53:57.058 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T09:53:57.098 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2024-04-25T09:53:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ== 2024-04-25T09:53:57.098 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-04-25T09:53:57.098 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg== 2024-04-25T09:53:57.099 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ== 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw== 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg== 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg== 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A== 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2024-04-25T09:53:57.100 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw== 2024-04-25T09:53:57.101 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-04-25T09:53:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg== 2024-04-25T09:53:57.101 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2024-04-25T09:53:57.101 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg== 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ== 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA== 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw== 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-04-25T09:53:57.117 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ== 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-04-25T09:53:57.118 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg== 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2024-04-25T09:53:57.119 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg== 2024-04-25T09:53:57.120 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:53:57.144 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:53:57.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.265 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.319 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.370 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.418 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.420 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.471 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:53:57.522 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T09:53:57.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.573 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.624 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T09:53:57.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.729 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.731 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.780 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.834 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T09:53:57.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:57.885 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:57.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:57.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:57.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:57.994 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:58.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:58.047 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T09:53:58.098 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T09:53:58.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T09:53:58.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:53:58.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T09:53:58.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T09:53:58.263 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:53:58.356 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T09:53:58.379 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T09:53:58.444 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T09:53:58.540 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T09:53:58.564 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:58.567 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T09:53:58.618 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T09:53:58.619 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T09:53:58.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T09:53:58.621 INFO:tasks.ceph.mon.a:Started 2024-04-25T09:53:58.621 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T09:53:58.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T09:53:58.661 INFO:tasks.ceph.mon.b:Started 2024-04-25T09:53:58.661 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T09:53:58.661 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T09:53:58.663 INFO:tasks.ceph.mon.c:Started 2024-04-25T09:53:58.663 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T09:53:58.664 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T09:53:58.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T09:53:58.666 INFO:tasks.ceph.mgr.y:Started 2024-04-25T09:53:58.666 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T09:53:58.666 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T09:53:58.668 INFO:tasks.ceph.mgr.x:Started 2024-04-25T09:53:58.668 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T09:53:58.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T09:53:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.763+0000 7f399ef0a640 1 Processor -- start 2024-04-25T09:53:58.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.763+0000 7f399ef0a640 1 -- start start 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399ef0a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 0x7f39980cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399ef0a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 0x7f39980cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399ef0a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39980afc80 con 0x7f39980cf980 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399ef0a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39980ce4d0 con 0x7f39980cefb0 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399ef0a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39980ce7b0 con 0x7f399815de80 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f3997fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 0x7f39980cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399d4a7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f39980cf980 0x7f399806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55426/0 (socket says 172.21.15.148:55426) 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f399d4a7640 1 -- 172.21.15.148:0/1110148757 learned_addr learned my addr 172.21.15.148:0/1110148757 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.764+0000 7f3997fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 0x7f39980cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38122/0 (socket says 172.21.15.148:38122) 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f3997fff640 1 -- 172.21.15.148:0/1110148757 >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 msgr2=0x7f39980cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f3997fff640 1 -- 172.21.15.148:0/1110148757 >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 msgr2=0x7f39980cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f3997fff640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 0x7f39980cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 0x7f39980cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f3997fff640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 0x7f39980cf440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:53:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f399cca6640 1 -- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 msgr2=0x7f39980cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T09:53:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f399cca6640 1 -- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 msgr2=0x7f39980cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T09:53:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 0x7f39980cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T09:53:58.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.765+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 0x7f39980cadb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T09:53:58.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.926+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008276059 0 0) 0x7f39980afc80 con 0x7f39980cf980 2024-04-25T09:53:58.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.926+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3984003520 con 0x7f39980cf980 2024-04-25T09:53:58.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 692265589 0 0) 0x7f3984003520 con 0x7f39980cf980 2024-04-25T09:53:58.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 msgr2=0x7f39980cadb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:58.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.188:3300/0 conn(0x7f399815de80 0x7f39980cadb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 msgr2=0x7f39980cf440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 --2- 172.21.15.148:0/1110148757 >> v2:172.21.15.148:3300/0 conn(0x7f39980cefb0 0x7f39980cf440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980cea90 con 0x7f39980cf980 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3990001670 con 0x7f39980cf980 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 >> v1:172.21.15.148:6789/0 conn(0x7f39980cf980 legacy=0x7f399806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840039b0 0x7f3984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984004800 0x7f3984004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39840051c0 0x7f398400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980ce4d0 con 0x7f39840039b0 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980ce7b0 con 0x7f3984004800 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398400c0c0 con 0x7f39840051c0 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840039b0 0x7f3984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399d4a7640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39840051c0 0x7f398400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f3997fff640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984004800 0x7f3984004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399cca6640 1 -- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39840051c0 msgr2=0x7f398400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39840051c0 0x7f398400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399cca6640 1 -- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984004800 msgr2=0x7f3984004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984004800 0x7f3984004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399cca6640 1 -- 172.21.15.148:0/1110148757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984003520 con 0x7f39840039b0 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.927+0000 7f399d4a7640 1 -- 172.21.15.148:0/1110148757 reap_dead start 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.928+0000 7f399cca6640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840039b0 0x7f3984003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f398000b9e0 tx=0x7f3984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dfde97799f9b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.928+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980030070 con 0x7f39840039b0 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.928+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398002adb0 con 0x7f39840039b0 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.928+0000 7f39977fe640 1 -- 172.21.15.148:0/1110148757 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398000b5e0 con 0x7f39840039b0 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 -- 172.21.15.148:0/1110148757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840039b0 msgr2=0x7f3984003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840039b0 0x7f3984003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f398000b9e0 tx=0x7f3984005830 comp rx=0 tx=0).stop 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 -- 172.21.15.148:0/1110148757 shutdown_connections 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 --2- 172.21.15.148:0/1110148757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840039b0 0x7f3984003dd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 -- 172.21.15.148:0/1110148757 >> 172.21.15.148:0/1110148757 conn(0x7f39980b2390 msgr2=0x7f39980b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:58.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 -- 172.21.15.148:0/1110148757 shutdown_connections 2024-04-25T09:53:58.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.929+0000 7f399ef0a640 1 -- 172.21.15.148:0/1110148757 wait complete. 2024-04-25T09:53:58.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.930+0000 7f399ef0a640 1 Processor -- start 2024-04-25T09:53:58.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.930+0000 7f399ef0a640 1 -- start start 2024-04-25T09:53:58.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399ef0a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840039b0 0x7f39980c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399ef0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 0x7f399815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399ef0a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39980cac80 0x7f39980cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:58.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399ef0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980b3030 con 0x7f39981548c0 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399ef0a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980b2eb0 con 0x7f39840039b0 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399ef0a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39980c5600 con 0x7f39980cac80 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 0x7f399815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 0x7f399815bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38144/0 (socket says 172.21.15.148:38144) 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 -- 172.21.15.148:0/1468382137 learned_addr learned my addr 172.21.15.148:0/1468382137 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399cca6640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840039b0 0x7f39980c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399d4a7640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39980cac80 0x7f39980cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 -- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39980cac80 msgr2=0x7f39980cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39980cac80 0x7f39980cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 -- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840039b0 msgr2=0x7f39980c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840039b0 0x7f39980c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f3997fff640 1 -- 172.21.15.148:0/1468382137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980c8710 con 0x7f39981548c0 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.931+0000 7f399d4a7640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39980cac80 0x7f39980cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:53:58.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f3997fff640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 0x7f399815bc50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f398800f150 tx=0x7f398800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880145e0 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398801a070 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39980c89c0 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880149a0 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39980cbfc0 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3988015470 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.932+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f398801e780 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.933+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39980cbdb0 con 0x7f39981548c0 2024-04-25T09:53:58.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:58.935+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f39880158f0 con 0x7f39981548c0 2024-04-25T09:53:59.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.002+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3998073a00 con 0x7f39981548c0 2024-04-25T09:53:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.004+0000 7f39957fa640 1 -- 172.21.15.148:0/1468382137 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3988015ad0 con 0x7f39981548c0 2024-04-25T09:53:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:adjusted tunables profile to default 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 msgr2=0x7f399815bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 0x7f399815bc50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f398800f150 tx=0x7f398800f610 comp rx=0 tx=0).stop 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 shutdown_connections 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39980cac80 0x7f39980cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39981548c0 0x7f399815bc50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 --2- 172.21.15.148:0/1468382137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840039b0 0x7f39980c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 >> 172.21.15.148:0/1468382137 conn(0x7f39980b2390 msgr2=0x7f39980b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 shutdown_connections 2024-04-25T09:53:59.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:53:59.005+0000 7f399ef0a640 1 -- 172.21.15.148:0/1468382137 wait complete. 2024-04-25T09:53:59.018 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T09:53:59.018 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T09:53:59.019 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:59.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T09:53:59.037 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:53:59.035+0000 7f9c730b2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:53:59.042 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:59.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T09:53:59.046 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:53:59.045+0000 7fcb42893040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T09:53:59.070 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:59.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T09:53:59.135 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:53:59.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T09:53:59.191 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:53:59.189+0000 7f9c730b2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:53:59.199 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:59.199 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T09:53:59.212 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:53:59.210+0000 7fcb42893040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:53:59.222 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:59.223 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T09:53:59.247 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:53:59.244+0000 7f9c730b2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:53:59.266 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:53:59.265+0000 7fcb42893040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T09:53:59.270 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:59.270 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T09:53:59.333 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:53:59.334 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T09:53:59.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 99e54694-af1a-42d6-bdcf-6f34b9ecb4d4 0 2024-04-25T09:53:59.495 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:53:59.492+0000 7f9c730b2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:53:59.512 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:53:59.510+0000 7fcb42893040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T09:53:59.535 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.533+0000 7fa0682c4640 1 Processor -- start 2024-04-25T09:53:59.536 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.533+0000 7fa0682c4640 1 -- start start 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa0682c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0600c6010 0x7fa0600ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa0682c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0600c6940 con 0x7fa0600cd520 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa0682c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06015add0 con 0x7fa06015a9f0 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa0682c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0600b0020 con 0x7fa0600c6010 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa06585f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0600c6010 0x7fa0600ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa06585f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0600c6010 0x7fa0600ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39020/0 (socket says 172.21.15.188:39020) 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa06585f640 1 -- 172.21.15.188:0/1885998763 learned_addr learned my addr 172.21.15.188:0/1885998763 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa06585f640 1 -- 172.21.15.188:0/1885998763 >> v1:172.21.15.188:6789/0 conn(0x7fa06015a9f0 legacy=0x7fa0600cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa06585f640 1 -- 172.21.15.188:0/1885998763 >> v1:172.21.15.148:6789/0 conn(0x7fa0600cd520 legacy=0x7fa0600cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.534+0000 7fa06585f640 1 -- 172.21.15.188:0/1885998763 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0600c6660 con 0x7fa0600c6010 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.535+0000 7fa06585f640 1 --2- 172.21.15.188:0/1885998763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0600c6010 0x7fa0600ccfe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa050009870 tx=0x7fa05002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31918823ccef430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.535+0000 7fa06505e640 1 -- 172.21.15.188:0/1885998763 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050037070 con 0x7fa0600c6010 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.535+0000 7fa06505e640 1 -- 172.21.15.188:0/1885998763 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05002ae50 con 0x7fa0600c6010 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.535+0000 7fa06505e640 1 -- 172.21.15.188:0/1885998763 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050032860 con 0x7fa0600c6010 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 -- 172.21.15.188:0/1885998763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0600c6010 msgr2=0x7fa0600ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 --2- 172.21.15.188:0/1885998763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0600c6010 0x7fa0600ccfe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa050009870 tx=0x7fa05002a300 comp rx=0 tx=0).stop 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 -- 172.21.15.188:0/1885998763 shutdown_connections 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 --2- 172.21.15.188:0/1885998763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0600c6010 0x7fa0600ccfe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 -- 172.21.15.188:0/1885998763 >> 172.21.15.188:0/1885998763 conn(0x7fa0600b2390 msgr2=0x7fa0600b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 -- 172.21.15.188:0/1885998763 shutdown_connections 2024-04-25T09:53:59.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.536+0000 7fa0682c4640 1 -- 172.21.15.188:0/1885998763 wait complete. 2024-04-25T09:53:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.537+0000 7fa0682c4640 1 Processor -- start 2024-04-25T09:53:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.537+0000 7fa0682c4640 1 -- start start 2024-04-25T09:53:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa0682c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0600c6010 0x7fa06014d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa0682c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 0x7fa06006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa0682c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa06015a9f0 0x7fa06006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa0682c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600cb230 con 0x7fa0600c6010 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa0682c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600cb5a0 con 0x7fa06015a9f0 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa0682c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600cc140 con 0x7fa0600cd520 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa06585f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 0x7fa06006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa06015a9f0 0x7fa06006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa06585f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 0x7fa06006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50834/0 (socket says 172.21.15.188:50834) 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa06585f640 1 -- 172.21.15.188:0/901934311 learned_addr learned my addr 172.21.15.188:0/901934311 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066060640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0600c6010 0x7fa06014d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 -- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 msgr2=0x7fa06006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 0x7fa06006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 -- 172.21.15.188:0/901934311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0600c6010 msgr2=0x7fa06014d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0600c6010 0x7fa06014d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 -- 172.21.15.188:0/901934311 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0601f17c0 con 0x7fa06015a9f0 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa06585f640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 0x7fa06006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.538+0000 7fa066861640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa06015a9f0 0x7fa06006b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00c690 tx=0x7fa05c00cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c003b70 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c003d10 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0601f1a80 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c005310 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601f2040 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa05c0054b0 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.539+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa05c012830 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.540+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06006cc80 con 0x7fa06015a9f0 2024-04-25T09:53:59.544 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:53:59.541+0000 7f9c730b2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:53:59.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.542+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa0600c6940 con 0x7fa06015a9f0 2024-04-25T09:53:59.558 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:53:59.556+0000 7fcb42893040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T09:53:59.616 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.614+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "99e54694-af1a-42d6-bdcf-6f34b9ecb4d4", "id": 0} v 0) v1 -- 0x7fa0600b21b0 con 0x7fa06015a9f0 2024-04-25T09:53:59.622 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.620+0000 7fa056ffd640 1 -- 172.21.15.188:0/901934311 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99e54694-af1a-42d6-bdcf-6f34b9ecb4d4", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa0600b21b0 con 0x7fa06015a9f0 2024-04-25T09:53:59.623 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa06015a9f0 msgr2=0x7fa06006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa06015a9f0 0x7fa06006b120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00c690 tx=0x7fa05c00cb50 comp rx=0 tx=0).stop 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 shutdown_connections 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa06015a9f0 0x7fa06006b120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0600cd520 0x7fa06006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 --2- 172.21.15.188:0/901934311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0600c6010 0x7fa06014d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 >> 172.21.15.188:0/901934311 conn(0x7fa0600b2390 msgr2=0x7fa0600b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 shutdown_connections 2024-04-25T09:53:59.624 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.621+0000 7fa0682c4640 1 -- 172.21.15.188:0/901934311 wait complete. 2024-04-25T09:53:59.634 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 833ab044-9d43-4c94-88b5-db5debf42a89 1 2024-04-25T09:53:59.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.728+0000 7f787deee640 1 Processor -- start 2024-04-25T09:53:59.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.728+0000 7f787deee640 1 -- start start 2024-04-25T09:53:59.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.729+0000 7f787deee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7878166a10 0x7f7878168e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.729+0000 7f787deee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780afc80 con 0x7f78780cadb0 2024-04-25T09:53:59.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.729+0000 7f787deee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780ce4d0 con 0x7f787815de80 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.729+0000 7f787deee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780cb240 con 0x7f7878166a10 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.729+0000 7f78777fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f787815de80 0x7f78780cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:47678/0 (socket says 172.21.15.188:47678) 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.729+0000 7f78777fe640 1 -- 172.21.15.188:0/2767551698 learned_addr learned my addr 172.21.15.188:0/2767551698 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7877fff640 1 --2- 172.21.15.188:0/2767551698 >> v2:172.21.15.148:3300/0 conn(0x7f7878166a10 0x7f7878168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233442101 0 0) 0x7f78780ce4d0 con 0x7f787815de80 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7864003520 con 0x7f787815de80 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734967208 0 0) 0x7f78780afc80 con 0x7f78780cadb0 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78780ce4d0 con 0x7f78780cadb0 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1429227457 0 0) 0x7f7864003520 con 0x7f787815de80 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7877fff640 1 -- 172.21.15.188:0/2767551698 >> v1:172.21.15.188:6789/0 conn(0x7f787815de80 legacy=0x7f78780cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7877fff640 1 -- 172.21.15.188:0/2767551698 >> v1:172.21.15.148:6789/0 conn(0x7f78780cadb0 legacy=0x7f78780cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.730+0000 7f7877fff640 1 -- 172.21.15.188:0/2767551698 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780ce7b0 con 0x7f7878166a10 2024-04-25T09:53:59.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.731+0000 7f7877fff640 1 --2- 172.21.15.188:0/2767551698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f7878168e00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f787002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c73f31c2eec4afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.731+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868001670 con 0x7f7878166a10 2024-04-25T09:53:59.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.731+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870007540 con 0x7f7878166a10 2024-04-25T09:53:59.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.731+0000 7f7876ffd640 1 -- 172.21.15.188:0/2767551698 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870007910 con 0x7f7878166a10 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.731+0000 7f787deee640 1 -- 172.21.15.188:0/2767551698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 msgr2=0x7f7878168e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.731+0000 7f787deee640 1 --2- 172.21.15.188:0/2767551698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f7878168e00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f787002a610 comp rx=0 tx=0).stop 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.732+0000 7f787deee640 1 -- 172.21.15.188:0/2767551698 shutdown_connections 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.732+0000 7f787deee640 1 --2- 172.21.15.188:0/2767551698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f7878168e00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.732+0000 7f787deee640 1 -- 172.21.15.188:0/2767551698 >> 172.21.15.188:0/2767551698 conn(0x7f78780b2390 msgr2=0x7f78780b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.732+0000 7f787deee640 1 -- 172.21.15.188:0/2767551698 shutdown_connections 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.732+0000 7f787deee640 1 -- 172.21.15.188:0/2767551698 wait complete. 2024-04-25T09:53:59.738 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 Processor -- start 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 -- start start 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 0x7f78781524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f787815de80 0x7f78780ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f78781f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b2eb0 con 0x7f7878166a10 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787815e260 con 0x7f787815de80 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f787deee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b3030 con 0x7f78780cadb0 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f78777fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 0x7f78781524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f786ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f787815de80 0x7f78780ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.733+0000 7f78777fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 0x7f78781524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50850/0 (socket says 172.21.15.188:50850) 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 -- 172.21.15.188:0/2559554262 learned_addr learned my addr 172.21.15.188:0/2559554262 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f7877fff640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f78781f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 -- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f787815de80 msgr2=0x7f78780ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.739 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f787815de80 0x7f78780ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 -- 172.21.15.188:0/2559554262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 msgr2=0x7f78781f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f78781f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 -- 172.21.15.188:0/2559554262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780ae020 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f786ffff640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f787815de80 0x7f78780ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f78777fe640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 0x7f78781524c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f786800c6d0 tx=0x7f786800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868018070 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78680072d0 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780ae300 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680047e0 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7878152a00 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f786801b070 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.734+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f786801d440 con 0x7f78780cadb0 2024-04-25T09:53:59.740 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.735+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7878073a00 con 0x7f78780cadb0 2024-04-25T09:53:59.741 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.737+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f786800a640 con 0x7f78780cadb0 2024-04-25T09:53:59.812 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.809+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "833ab044-9d43-4c94-88b5-db5debf42a89", "id": 1} v 0) v1 -- 0x7f7878166460 con 0x7f78780cadb0 2024-04-25T09:53:59.817 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.814+0000 7f7874ff9640 1 -- 172.21.15.188:0/2559554262 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "833ab044-9d43-4c94-88b5-db5debf42a89", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7868004980 con 0x7f78780cadb0 2024-04-25T09:53:59.817 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-25T09:53:59.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 msgr2=0x7f78781524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 0x7f78781524c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f786800c6d0 tx=0x7f786800cb90 comp rx=0 tx=0).stop 2024-04-25T09:53:59.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 shutdown_connections 2024-04-25T09:53:59.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7878166a10 0x7f78781f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f787815de80 0x7f78780ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 --2- 172.21.15.188:0/2559554262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78780cadb0 0x7f78781524c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 >> 172.21.15.188:0/2559554262 conn(0x7f78780b2390 msgr2=0x7f78780b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 shutdown_connections 2024-04-25T09:53:59.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.815+0000 7f787deee640 1 -- 172.21.15.188:0/2559554262 wait complete. 2024-04-25T09:53:59.828 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 976d340c-b8e6-4f86-805f-7402637d96e6 2 2024-04-25T09:53:59.927 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.924+0000 7eff1e25d640 1 Processor -- start 2024-04-25T09:53:59.927 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.924+0000 7eff1e25d640 1 -- start start 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff1e25d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff18161240 0x7eff180b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff1e25d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff180b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff1e25d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff180cf1d0 con 0x7eff18161890 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff1e25d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff180b7120 con 0x7eff18161240 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff1e25d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff18161620 con 0x7eff180b54a0 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff18161240 0x7eff180b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.929 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff17fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff180b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff18161240 0x7eff180b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39046/0 (socket says 172.21.15.188:39046) 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 -- 172.21.15.188:0/3956775472 learned_addr learned my addr 172.21.15.188:0/3956775472 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff167fc640 1 -- 172.21.15.188:0/3956775472 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098708491 0 0) 0x7eff180cf1d0 con 0x7eff18161890 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 -- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 msgr2=0x7eff180b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 --2- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff180b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 -- 172.21.15.188:0/3956775472 >> v1:172.21.15.148:6789/0 conn(0x7eff18161890 legacy=0x7eff180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff177fe640 1 -- 172.21.15.188:0/3956775472 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18161e00 con 0x7eff18161240 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.925+0000 7eff17fff640 1 --2- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff180b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.926+0000 7eff177fe640 1 --2- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161240 0x7eff180b6be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff08009870 tx=0x7eff0802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c198dc7b2f2bf7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.926+0000 7eff167fc640 1 -- 172.21.15.188:0/3956775472 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08037070 con 0x7eff18161240 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.926+0000 7eff167fc640 1 -- 172.21.15.188:0/3956775472 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08032440 con 0x7eff18161240 2024-04-25T09:53:59.930 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.926+0000 7eff167fc640 1 -- 172.21.15.188:0/3956775472 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08032810 con 0x7eff18161240 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 -- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161240 msgr2=0x7eff180b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 --2- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161240 0x7eff180b6be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff08009870 tx=0x7eff0802a250 comp rx=0 tx=0).stop 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 -- 172.21.15.188:0/3956775472 shutdown_connections 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 --2- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff180b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 --2- 172.21.15.188:0/3956775472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161240 0x7eff180b6be0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 -- 172.21.15.188:0/3956775472 >> 172.21.15.188:0/3956775472 conn(0x7eff180d04a0 msgr2=0x7eff180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 -- 172.21.15.188:0/3956775472 shutdown_connections 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.927+0000 7eff1e25d640 1 -- 172.21.15.188:0/3956775472 wait complete. 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.928+0000 7eff1e25d640 1 Processor -- start 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 -- start start 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff181f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.933 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff18161240 0x7eff180c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161890 0x7eff180cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181626a0 con 0x7eff18161240 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181623c0 con 0x7eff18161890 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff1e25d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff180d15f0 con 0x7eff180b54a0 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff177fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff181f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff17fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161890 0x7eff180cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff177fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff181f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50858/0 (socket says 172.21.15.188:50858) 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff177fe640 1 -- 172.21.15.188:0/1852750172 learned_addr learned my addr 172.21.15.188:0/1852750172 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.929+0000 7eff16ffd640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff18161240 0x7eff180c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff177fe640 1 -- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161890 msgr2=0x7eff180cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff177fe640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161890 0x7eff180cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.934 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff177fe640 1 -- 172.21.15.188:0/1852750172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff18161240 msgr2=0x7eff180c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:53:59.935 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff177fe640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff18161240 0x7eff180c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:53:59.935 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff177fe640 1 -- 172.21.15.188:0/1852750172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff180d1a00 con 0x7eff180b54a0 2024-04-25T09:53:59.935 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff17fff640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161890 0x7eff180cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:59.935 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff16ffd640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff18161240 0x7eff180c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:53:59.935 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.930+0000 7eff177fe640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff181f4bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff08000c00 tx=0x7eff080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08037040 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08042070 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080308a0 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff181f5130 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff180d1f50 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7eff08030a40 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.931+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7eff08030d30 con 0x7eff180b54a0 2024-04-25T09:53:59.936 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.932+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff180b5ff0 con 0x7eff180b54a0 2024-04-25T09:53:59.937 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:53:59.934+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7eff180b5ff0 con 0x7eff180b54a0 2024-04-25T09:54:00.010 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.007+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "976d340c-b8e6-4f86-805f-7402637d96e6", "id": 2} v 0) v1 -- 0x7eff180ca360 con 0x7eff180b54a0 2024-04-25T09:54:00.015 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.013+0000 7efefffff640 1 -- 172.21.15.188:0/1852750172 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "976d340c-b8e6-4f86-805f-7402637d96e6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7eff08047460 con 0x7eff180b54a0 2024-04-25T09:54:00.015 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-04-25T09:54:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 msgr2=0x7eff181f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff181f4bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff08000c00 tx=0x7eff080046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.016 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 shutdown_connections 2024-04-25T09:54:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff18161890 0x7eff180cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff18161240 0x7eff180c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 --2- 172.21.15.188:0/1852750172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff180b54a0 0x7eff181f4bf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 >> 172.21.15.188:0/1852750172 conn(0x7eff180d04a0 msgr2=0x7eff1815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 shutdown_connections 2024-04-25T09:54:00.017 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.014+0000 7eff1e25d640 1 -- 172.21.15.188:0/1852750172 wait complete. 2024-04-25T09:54:00.026 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 0b8d5dda-2380-4190-95e8-3084f6a71da4 3 2024-04-25T09:54:00.124 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 Processor -- start 2024-04-25T09:54:00.124 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 -- start start 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 0x7fc8840b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 0x7fc8840b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8840cf1d0 con 0x7fc884161240 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8840b7120 con 0x7fc8840b54a0 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc88a65d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc884161620 con 0x7fc884161890 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc888bfa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 0x7fc8840b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.122+0000 7fc8837fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 0x7fc8840b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc888bfa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 0x7fc8840b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39074/0 (socket says 172.21.15.188:39074) 2024-04-25T09:54:00.127 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc888bfa640 1 -- 172.21.15.188:0/3496277945 learned_addr learned my addr 172.21.15.188:0/3496277945 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc8837fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 0x7fc8840b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50874/0 (socket says 172.21.15.188:50874) 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc8837fe640 1 -- 172.21.15.188:0/3496277945 >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 msgr2=0x7fc8840b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc8837fe640 1 --2- 172.21.15.188:0/3496277945 >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 0x7fc8840b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc8837fe640 1 -- 172.21.15.188:0/3496277945 >> v1:172.21.15.148:6789/0 conn(0x7fc884161240 legacy=0x7fc8840ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc8837fe640 1 -- 172.21.15.188:0/3496277945 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884161e00 con 0x7fc884161890 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc888bfa640 1 --2- 172.21.15.188:0/3496277945 >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 0x7fc8840b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.123+0000 7fc8837fe640 1 --2- 172.21.15.188:0/3496277945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 0x7fc8840b6be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8780097a0 tx=0x7fc87802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0c505cd8aef2c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc882ffd640 1 -- 172.21.15.188:0/3496277945 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878037070 con 0x7fc884161890 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc882ffd640 1 -- 172.21.15.188:0/3496277945 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc878032440 con 0x7fc884161890 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc882ffd640 1 -- 172.21.15.188:0/3496277945 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878032810 con 0x7fc884161890 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 -- 172.21.15.188:0/3496277945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 msgr2=0x7fc8840b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 --2- 172.21.15.188:0/3496277945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 0x7fc8840b6be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8780097a0 tx=0x7fc87802a180 comp rx=0 tx=0).stop 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 -- 172.21.15.188:0/3496277945 shutdown_connections 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 --2- 172.21.15.188:0/3496277945 >> v2:172.21.15.188:3300/0 conn(0x7fc8840b54a0 0x7fc8840b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.128 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 --2- 172.21.15.188:0/3496277945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161890 0x7fc8840b6be0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.129 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 -- 172.21.15.188:0/3496277945 >> 172.21.15.188:0/3496277945 conn(0x7fc8840d04a0 msgr2=0x7fc8840d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.129 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 -- 172.21.15.188:0/3496277945 shutdown_connections 2024-04-25T09:54:00.129 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.124+0000 7fc88a65d640 1 -- 172.21.15.188:0/3496277945 wait complete. 2024-04-25T09:54:00.131 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.125+0000 7fc88a65d640 1 Processor -- start 2024-04-25T09:54:00.131 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 -- start start 2024-04-25T09:54:00.131 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8840b54a0 0x7fc8840cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.131 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 0x7fc8840c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.131 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884161890 0x7fc884151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841626a0 con 0x7fc8840b54a0 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841623c0 con 0x7fc884161890 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc88a65d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840d15f0 con 0x7fc884161240 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc888bfa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884161890 0x7fc884151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.126+0000 7fc8837fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 0x7fc8840c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 0x7fc8840c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50886/0 (socket says 172.21.15.188:50886) 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 -- 172.21.15.188:0/1821705436 learned_addr learned my addr 172.21.15.188:0/1821705436 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc883fff640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8840b54a0 0x7fc8840cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 -- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884161890 msgr2=0x7fc884151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884161890 0x7fc884151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 -- 172.21.15.188:0/1821705436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8840b54a0 msgr2=0x7fc8840cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8840b54a0 0x7fc8840cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 -- 172.21.15.188:0/1821705436 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840c9030 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc8837fe640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 0x7fc8840c8af0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc878000c00 tx=0x7fc8780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878041070 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87803c870 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8840cc830 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878037040 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8840ccdf0 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc878044070 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.127+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fc87803d3f0 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.128+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc884073a00 con 0x7fc884161240 2024-04-25T09:54:00.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.130+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc87803cc60 con 0x7fc884161240 2024-04-25T09:54:00.206 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.202+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0b8d5dda-2380-4190-95e8-3084f6a71da4", "id": 3} v 0) v1 -- 0x7fc88406d240 con 0x7fc884161240 2024-04-25T09:54:00.210 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.208+0000 7fc880ff9640 1 -- 172.21.15.188:0/1821705436 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0b8d5dda-2380-4190-95e8-3084f6a71da4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc878046450 con 0x7fc884161240 2024-04-25T09:54:00.210 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 msgr2=0x7fc8840c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 0x7fc8840c8af0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc878000c00 tx=0x7fc8780046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 shutdown_connections 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884161890 0x7fc884151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc884161240 0x7fc8840c8af0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 --2- 172.21.15.188:0/1821705436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8840b54a0 0x7fc8840cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 >> 172.21.15.188:0/1821705436 conn(0x7fc8840d04a0 msgr2=0x7fc8840ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 shutdown_connections 2024-04-25T09:54:00.212 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.209+0000 7fc88a65d640 1 -- 172.21.15.188:0/1821705436 wait complete. 2024-04-25T09:54:00.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 05f1febd-ccd2-4ec3-9e25-e8cbf5569152 4 2024-04-25T09:54:00.250 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.248+0000 7fcb42893040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:54:00.296 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.293+0000 7f9c730b2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T09:54:00.319 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.316+0000 7fbf09ecb640 1 Processor -- start 2024-04-25T09:54:00.319 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.316+0000 7fbf09ecb640 1 -- start start 2024-04-25T09:54:00.320 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf09ecb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf040cf600 0x7fbf040cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.320 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf09ecb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040c75e0 con 0x7fbf040cbf20 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf09ecb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040c78c0 con 0x7fbf0415bb50 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf09ecb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf040c8680 con 0x7fbf040cf600 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf02ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf040cf600 0x7fbf040cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf02ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf040cf600 0x7fbf040cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39094/0 (socket says 172.21.15.188:39094) 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf02ffd640 1 -- 172.21.15.188:0/1451682563 learned_addr learned my addr 172.21.15.188:0/1451682563 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf037fe640 1 --1- 172.21.15.188:0/1451682563 >> v1:172.21.15.188:6789/0 conn(0x7fbf0415bb50 0x7fbf040c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:47680/0 (socket says 172.21.15.188:47680) 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf02ffd640 1 -- 172.21.15.188:0/1451682563 >> v1:172.21.15.188:6789/0 conn(0x7fbf0415bb50 legacy=0x7fbf040c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf02ffd640 1 -- 172.21.15.188:0/1451682563 >> v1:172.21.15.148:6789/0 conn(0x7fbf040cbf20 legacy=0x7fbf040cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.317+0000 7fbf02ffd640 1 -- 172.21.15.188:0/1451682563 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040c7ba0 con 0x7fbf040cf600 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.318+0000 7fbf02ffd640 1 --2- 172.21.15.188:0/1451682563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf040cfa40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009870 tx=0x7fbef402a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=948808340b875eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.318+0000 7fbf027fc640 1 -- 172.21.15.188:0/1451682563 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4037070 con 0x7fbf040cf600 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.318+0000 7fbf027fc640 1 -- 172.21.15.188:0/1451682563 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4032640 con 0x7fbf040cf600 2024-04-25T09:54:00.321 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.318+0000 7fbf027fc640 1 -- 172.21.15.188:0/1451682563 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4032a10 con 0x7fbf040cf600 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 -- 172.21.15.188:0/1451682563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 msgr2=0x7fbf040cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 --2- 172.21.15.188:0/1451682563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf040cfa40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009870 tx=0x7fbef402a3c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 -- 172.21.15.188:0/1451682563 shutdown_connections 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 --2- 172.21.15.188:0/1451682563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf040cfa40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 -- 172.21.15.188:0/1451682563 >> 172.21.15.188:0/1451682563 conn(0x7fbf0406f500 msgr2=0x7fbf0406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 -- 172.21.15.188:0/1451682563 shutdown_connections 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.319+0000 7fbf09ecb640 1 -- 172.21.15.188:0/1451682563 wait complete. 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.320+0000 7fbf09ecb640 1 Processor -- start 2024-04-25T09:54:00.323 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.320+0000 7fbf09ecb640 1 -- start start 2024-04-25T09:54:00.325 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf09ecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf040cbf20 0x7fbf04154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf09ecb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf0414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf09ecb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf0415bb50 0x7fbf0414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf09ecb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040c8160 con 0x7fbf0415bb50 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf09ecb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040cff80 con 0x7fbf040cf600 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf09ecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040c72a0 con 0x7fbf040cbf20 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf0414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf040cbf20 0x7fbf04154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf0414e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39110/0 (socket says 172.21.15.188:39110) 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 -- 172.21.15.188:0/492452319 learned_addr learned my addr 172.21.15.188:0/492452319 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf03fff640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf0415bb50 0x7fbf0414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 -- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf040cbf20 msgr2=0x7fbf04154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf040cbf20 0x7fbf04154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.326 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 -- 172.21.15.188:0/492452319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf0415bb50 msgr2=0x7fbf0414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf0415bb50 0x7fbf0414ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 -- 172.21.15.188:0/492452319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040c8a70 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.321+0000 7fbf02ffd640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf0414e680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef403adc0 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4045450 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4045840 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf040c8d50 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf040c9310 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbef4032500 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.322+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbef404edd0 con 0x7fbf040cf600 2024-04-25T09:54:00.327 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.323+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf04073a00 con 0x7fbf040cf600 2024-04-25T09:54:00.329 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.326+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbef4053080 con 0x7fbf040cf600 2024-04-25T09:54:00.337 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.335+0000 7fcb42893040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:54:00.379 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.377+0000 7fcb42893040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:54:00.389 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.386+0000 7f9c730b2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T09:54:00.401 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.399+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "05f1febd-ccd2-4ec3-9e25-e8cbf5569152", "id": 4} v 0) v1 -- 0x7fbf040cf150 con 0x7fbf040cf600 2024-04-25T09:54:00.407 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.404+0000 7fbeebfff640 1 -- 172.21.15.188:0/492452319 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05f1febd-ccd2-4ec3-9e25-e8cbf5569152", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbef404d3a0 con 0x7fbf040cf600 2024-04-25T09:54:00.407 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 msgr2=0x7fbf0414e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf0414e680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000c00 tx=0x7fbef40046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 shutdown_connections 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf0415bb50 0x7fbf0414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.408 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf040cf600 0x7fbf0414e680 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.409 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 --2- 172.21.15.188:0/492452319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf040cbf20 0x7fbf04154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.409 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 >> 172.21.15.188:0/492452319 conn(0x7fbf0406f500 msgr2=0x7fbf040d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.409 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 shutdown_connections 2024-04-25T09:54:00.409 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.405+0000 7fbf09ecb640 1 -- 172.21.15.188:0/492452319 wait complete. 2024-04-25T09:54:00.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 4f810a22-a237-42ea-af28-df38627332fa 5 2024-04-25T09:54:00.436 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.433+0000 7f9c730b2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T09:54:00.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.513+0000 7f93fc03a640 1 Processor -- start 2024-04-25T09:54:00.515 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.513+0000 7f93fc03a640 1 -- start start 2024-04-25T09:54:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fc03a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93f415dbb0 0x7f93f406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fc03a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f406c300 0x7f93f406db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fc03a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40cae60 con 0x7f93f40cbf00 2024-04-25T09:54:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fc03a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f40cb140 con 0x7f93f415dbb0 2024-04-25T09:54:00.517 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fc03a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cb420 con 0x7f93f406c300 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fa5d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f406c300 0x7f93f406db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93f95d5640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f93f40cbf00 0x7f93f406bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:47694/0 (socket says 172.21.15.188:47694) 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93f95d5640 1 -- 172.21.15.188:0/3544922020 learned_addr learned my addr 172.21.15.188:0/3544922020 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93f9dd6640 1 --2- 172.21.15.188:0/3544922020 >> v2:172.21.15.148:3300/0 conn(0x7f93f415dbb0 0x7f93f406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93f8dd4640 1 -- 172.21.15.188:0/3544922020 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499775402 0 0) 0x7f93f40cae60 con 0x7f93f40cbf00 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93f8dd4640 1 -- 172.21.15.188:0/3544922020 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93e0003520 con 0x7f93f40cbf00 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fa5d7640 1 -- 172.21.15.188:0/3544922020 >> v2:172.21.15.148:3300/0 conn(0x7f93f415dbb0 msgr2=0x7f93f406fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fa5d7640 1 --2- 172.21.15.188:0/3544922020 >> v2:172.21.15.148:3300/0 conn(0x7f93f415dbb0 0x7f93f406fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fa5d7640 1 -- 172.21.15.188:0/3544922020 >> v1:172.21.15.188:6789/0 conn(0x7f93f40cbf00 legacy=0x7f93f406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.514+0000 7f93fa5d7640 1 -- 172.21.15.188:0/3544922020 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40cb700 con 0x7f93f406c300 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.515+0000 7f93fa5d7640 1 --2- 172.21.15.188:0/3544922020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f406c300 0x7f93f406db50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009d90 tx=0x7f93ec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed82957efa42f979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.515+0000 7f93f8dd4640 1 -- 172.21.15.188:0/3544922020 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec007740 con 0x7f93f406c300 2024-04-25T09:54:00.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.515+0000 7f93f8dd4640 1 -- 172.21.15.188:0/3544922020 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec0078e0 con 0x7f93f406c300 2024-04-25T09:54:00.519 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.515+0000 7f93f8dd4640 1 -- 172.21.15.188:0/3544922020 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec007cb0 con 0x7f93f406c300 2024-04-25T09:54:00.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.515+0000 7f93fc03a640 1 -- 172.21.15.188:0/3544922020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f406c300 msgr2=0x7f93f406db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.521 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.515+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3544922020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f406c300 0x7f93f406db50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009d90 tx=0x7f93ec02a660 comp rx=0 tx=0).stop 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.516+0000 7f93fc03a640 1 -- 172.21.15.188:0/3544922020 shutdown_connections 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.516+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3544922020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f406c300 0x7f93f406db50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.516+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3544922020 >> v2:172.21.15.148:3300/0 conn(0x7f93f415dbb0 0x7f93f406fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.516+0000 7f93fc03a640 1 -- 172.21.15.188:0/3544922020 >> 172.21.15.188:0/3544922020 conn(0x7f93f4071af0 msgr2=0x7f93f4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.516+0000 7f93fc03a640 1 -- 172.21.15.188:0/3544922020 shutdown_connections 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.516+0000 7f93fc03a640 1 -- 172.21.15.188:0/3544922020 wait complete. 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.517+0000 7f93fc03a640 1 Processor -- start 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.517+0000 7f93fc03a640 1 -- start start 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93f406c300 0x7f93f40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 0x7f93f40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f415dbb0 0x7f93f414f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40cab10 con 0x7f93f406c300 2024-04-25T09:54:00.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f415df90 con 0x7f93f415dbb0 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40703e0 con 0x7f93f40cbf00 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 0x7f93f40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fa5d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f415dbb0 0x7f93f414f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 0x7f93f40b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50920/0 (socket says 172.21.15.188:50920) 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 -- 172.21.15.188:0/3793979728 learned_addr learned my addr 172.21.15.188:0/3793979728 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f9dd6640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93f406c300 0x7f93f40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 -- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f415dbb0 msgr2=0x7f93f414f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f415dbb0 0x7f93f414f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 -- 172.21.15.188:0/3793979728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93f406c300 msgr2=0x7f93f40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93f406c300 0x7f93f40c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 -- 172.21.15.188:0/3793979728 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f415f1c0 con 0x7f93f40cbf00 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93f95d5640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 0x7f93f40b4fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93e400c460 tx=0x7f93e400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4017070 con 0x7f93f40cbf00 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e4005ce0 con 0x7f93f40cbf00 2024-04-25T09:54:00.523 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f40c9c70 con 0x7f93f40cbf00 2024-04-25T09:54:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4007290 con 0x7f93f40cbf00 2024-04-25T09:54:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.518+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f40c9ef0 con 0x7f93f40cbf00 2024-04-25T09:54:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.519+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f93e401a070 con 0x7f93f40cbf00 2024-04-25T09:54:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.519+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f93e4012980 con 0x7f93f40cbf00 2024-04-25T09:54:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.519+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4073a00 con 0x7f93f40cbf00 2024-04-25T09:54:00.524 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.521+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f93e4021080 con 0x7f93f40cbf00 2024-04-25T09:54:00.545 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.543+0000 7fcb42893040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:54:00.596 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.593+0000 7fcb42893040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:54:00.596 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.593+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4f810a22-a237-42ea-af28-df38627332fa", "id": 5} v 0) v1 -- 0x7f93f415f390 con 0x7f93f40cbf00 2024-04-25T09:54:00.598 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.596+0000 7f93ea7fc640 1 -- 172.21.15.188:0/3793979728 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f810a22-a237-42ea-af28-df38627332fa", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f93f40c9c70 con 0x7f93f40cbf00 2024-04-25T09:54:00.599 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 msgr2=0x7f93f40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 0x7f93f40b4fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93e400c460 tx=0x7f93e400c920 comp rx=0 tx=0).stop 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 shutdown_connections 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f415dbb0 0x7f93f414f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f40cbf00 0x7f93f40b4fd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 --2- 172.21.15.188:0/3793979728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93f406c300 0x7f93f40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 >> 172.21.15.188:0/3793979728 conn(0x7f93f4071af0 msgr2=0x7f93f406db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 shutdown_connections 2024-04-25T09:54:00.600 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.597+0000 7f93fc03a640 1 -- 172.21.15.188:0/3793979728 wait complete. 2024-04-25T09:54:00.608 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.606+0000 7f9c730b2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T09:54:00.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 9dbe47fa-4206-44be-8ac0-908c472bff8a 6 2024-04-25T09:54:00.647 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.645+0000 7fcb42893040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:54:00.664 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.662+0000 7f9c730b2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T09:54:00.720 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.717+0000 7f9c730b2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T09:54:00.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.745+0000 7f01403b0640 1 Processor -- start 2024-04-25T09:54:00.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.745+0000 7f01403b0640 1 -- start start 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f01403b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 0x7f01380c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f01403b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01380c70d0 0x7f01380cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f01403b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01380c75e0 con 0x7f01380cfb40 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f01403b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01380c8680 con 0x7f013815bb50 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f01403b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380c78c0 con 0x7f01380c70d0 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013e14c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 0x7f01380c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013d94b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01380c70d0 0x7f01380cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013e14c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 0x7f01380c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39132/0 (socket says 172.21.15.188:39132) 2024-04-25T09:54:00.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013e14c640 1 -- 172.21.15.188:0/3958787199 learned_addr learned my addr 172.21.15.188:0/3958787199 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013d94b640 1 -- 172.21.15.188:0/3958787199 >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 msgr2=0x7f01380c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013d94b640 1 --2- 172.21.15.188:0/3958787199 >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 0x7f01380c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013d94b640 1 -- 172.21.15.188:0/3958787199 >> v1:172.21.15.148:6789/0 conn(0x7f01380cfb40 legacy=0x7f0138070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013d94b640 1 -- 172.21.15.188:0/3958787199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380c7ba0 con 0x7f01380c70d0 2024-04-25T09:54:00.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.746+0000 7f013e14c640 1 --2- 172.21.15.188:0/3958787199 >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 0x7f01380c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:00.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.747+0000 7f013d94b640 1 --2- 172.21.15.188:0/3958787199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01380c70d0 0x7f01380cf600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01340097a0 tx=0x7f013402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a735b512d2e63f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.747+0000 7f013d14a640 1 -- 172.21.15.188:0/3958787199 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134037070 con 0x7f01380c70d0 2024-04-25T09:54:00.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.747+0000 7f013d14a640 1 -- 172.21.15.188:0/3958787199 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134032440 con 0x7f01380c70d0 2024-04-25T09:54:00.751 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.747+0000 7f013d14a640 1 -- 172.21.15.188:0/3958787199 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134032810 con 0x7f01380c70d0 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 -- 172.21.15.188:0/3958787199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01380c70d0 msgr2=0x7f01380cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 --2- 172.21.15.188:0/3958787199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01380c70d0 0x7f01380cf600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01340097a0 tx=0x7f013402a210 comp rx=0 tx=0).stop 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 -- 172.21.15.188:0/3958787199 shutdown_connections 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 --2- 172.21.15.188:0/3958787199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01380c70d0 0x7f01380cf600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 --2- 172.21.15.188:0/3958787199 >> v2:172.21.15.188:3300/0 conn(0x7f013815bb50 0x7f01380c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 -- 172.21.15.188:0/3958787199 >> 172.21.15.188:0/3958787199 conn(0x7f013806f500 msgr2=0x7f013806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 -- 172.21.15.188:0/3958787199 shutdown_connections 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.748+0000 7f01403b0640 1 -- 172.21.15.188:0/3958787199 wait complete. 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.749+0000 7f01403b0640 1 Processor -- start 2024-04-25T09:54:00.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.749+0000 7f01403b0640 1 -- start start 2024-04-25T09:54:00.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 0x7f01380b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01380cfb40 0x7f013814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f013815bb50 0x7f013815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013815f6d0 con 0x7f01380cfb40 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380c8160 con 0x7f01380c70d0 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013815f850 con 0x7f013815bb50 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e14c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 0x7f01380b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f013815bb50 0x7f013815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e14c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 0x7f01380b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39136/0 (socket says 172.21.15.188:39136) 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e14c640 1 -- 172.21.15.188:0/4154745752 learned_addr learned my addr 172.21.15.188:0/4154745752 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013d94b640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01380cfb40 0x7f013814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 -- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 msgr2=0x7f01380b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 0x7f01380b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 -- 172.21.15.188:0/4154745752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01380cfb40 msgr2=0x7f013814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01380cfb40 0x7f013814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 -- 172.21.15.188:0/4154745752 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013814c710 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e14c640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 0x7f01380b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f013e94d640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f013815bb50 0x7f013815e4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f013000c690 tx=0x7f013000cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130003b70 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130003d10 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01380b4bb0 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130005310 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.750+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01380b5450 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.751+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f01300054b0 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.751+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f0130012b30 con 0x7f013815bb50 2024-04-25T09:54:00.756 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.752+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01380b4fa0 con 0x7f013815bb50 2024-04-25T09:54:00.757 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.754+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0130012d80 con 0x7f013815bb50 2024-04-25T09:54:00.758 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:00.756+0000 7fcb42893040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:54:00.828 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.825+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9dbe47fa-4206-44be-8ac0-908c472bff8a", "id": 6} v 0) v1 -- 0x7f01380cf150 con 0x7f013815bb50 2024-04-25T09:54:00.833 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.831+0000 7f012effd640 1 -- 172.21.15.188:0/4154745752 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9dbe47fa-4206-44be-8ac0-908c472bff8a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0130012500 con 0x7f013815bb50 2024-04-25T09:54:00.833 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-04-25T09:54:00.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.831+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f013815bb50 msgr2=0x7f013815e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.831+0000 7f01403b0640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f013815bb50 0x7f013815e4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f013000c690 tx=0x7f013000cb50 comp rx=0 tx=0).stop 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 shutdown_connections 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f013815bb50 0x7f013815e4f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01380cfb40 0x7f013814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 --2- 172.21.15.188:0/4154745752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01380c70d0 0x7f01380b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 >> 172.21.15.188:0/4154745752 conn(0x7f013806f500 msgr2=0x7f01380ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 shutdown_connections 2024-04-25T09:54:00.835 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.832+0000 7f01403b0640 1 -- 172.21.15.188:0/4154745752 wait complete. 2024-04-25T09:54:00.838 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:00.835+0000 7f9c730b2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T09:54:00.844 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 8be29075-43eb-44ff-8f48-96316cff73f7 7 2024-04-25T09:54:00.982 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.979+0000 7f439423a640 1 Processor -- start 2024-04-25T09:54:00.983 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.980+0000 7f439423a640 1 -- start start 2024-04-25T09:54:00.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.980+0000 7f439423a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.980+0000 7f439423a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 0x7f438c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f439423a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438c0afc80 con 0x7f438c0cefb0 2024-04-25T09:54:00.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f439423a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438c0ce4d0 con 0x7f438c0ccfe0 2024-04-25T09:54:00.986 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f439423a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0ce7b0 con 0x7f438c15de80 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f43927d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 0x7f438c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50954/0 (socket says 172.21.15.188:50954) 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f43927d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 0x7f438c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:39148/0 (socket says 172.21.15.188:39148) 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 -- 172.21.15.188:0/2831546918 learned_addr learned my addr 172.21.15.188:0/2831546918 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 -- 172.21.15.188:0/2831546918 >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 msgr2=0x7f438c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 --2- 172.21.15.188:0/2831546918 >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 0x7f438c0cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 -- 172.21.15.188:0/2831546918 >> v1:172.21.15.148:6789/0 conn(0x7f438c0cefb0 legacy=0x7f438c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f4391fd6640 1 -- 172.21.15.188:0/2831546918 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0cea90 con 0x7f438c15de80 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.981+0000 7f43927d7640 1 --2- 172.21.15.188:0/2831546918 >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 0x7f438c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f4391fd6640 1 --2- 172.21.15.188:0/2831546918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0cadb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4388009870 tx=0x7f438802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=846b284b1c9a5a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f4390fd4640 1 -- 172.21.15.188:0/2831546918 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388037070 con 0x7f438c15de80 2024-04-25T09:54:00.987 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f4390fd4640 1 -- 172.21.15.188:0/2831546918 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4388032440 con 0x7f438c15de80 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f4390fd4640 1 -- 172.21.15.188:0/2831546918 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388032810 con 0x7f438c15de80 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f439423a640 1 -- 172.21.15.188:0/2831546918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 msgr2=0x7f438c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f439423a640 1 --2- 172.21.15.188:0/2831546918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0cadb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4388009870 tx=0x7f438802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f439423a640 1 -- 172.21.15.188:0/2831546918 shutdown_connections 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f439423a640 1 --2- 172.21.15.188:0/2831546918 >> v2:172.21.15.188:3300/0 conn(0x7f438c0ccfe0 0x7f438c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f439423a640 1 --2- 172.21.15.188:0/2831546918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0cadb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.982+0000 7f439423a640 1 -- 172.21.15.188:0/2831546918 >> 172.21.15.188:0/2831546918 conn(0x7f438c0b2390 msgr2=0x7f438c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.983+0000 7f439423a640 1 -- 172.21.15.188:0/2831546918 shutdown_connections 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.983+0000 7f439423a640 1 -- 172.21.15.188:0/2831546918 wait complete. 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 Processor -- start 2024-04-25T09:54:00.988 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 -- start start 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f438c0ccfe0 0x7f438c150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f438c0cefb0 0x7f438c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:00.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0cfca0 con 0x7f438c0ccfe0 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0cfb20 con 0x7f438c0cefb0 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f439423a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0b2eb0 con 0x7f438c15de80 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.984+0000 7f43927d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43917d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f438c0cefb0 0x7f438c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:50962/0 (socket says 172.21.15.188:50962) 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 -- 172.21.15.188:0/2568515378 learned_addr learned my addr 172.21.15.188:0/2568515378 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f4391fd6640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f438c0ccfe0 0x7f438c150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 -- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f438c0cefb0 msgr2=0x7f438c0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f438c0cefb0 0x7f438c0b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 -- 172.21.15.188:0/2568515378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f438c0ccfe0 msgr2=0x7f438c150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f438c0ccfe0 0x7f438c150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 -- 172.21.15.188:0/2568515378 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0b0590 con 0x7f438c15de80 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43917d5640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f438c0cefb0 0x7f438c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43927d7640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0c7830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f438400efc0 tx=0x7f4384009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:00.991 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384018030 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438400cde0 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384003bb0 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c0b08d0 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c0b02b0 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4384005d10 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.985+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f438401db60 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.986+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c073a00 con 0x7f438c15de80 2024-04-25T09:54:00.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:00.990+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4384022070 con 0x7f438c15de80 2024-04-25T09:54:01.064 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.061+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8be29075-43eb-44ff-8f48-96316cff73f7", "id": 7} v 0) v1 -- 0x7f438c161750 con 0x7f438c15de80 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.072+0000 7f43827fc640 1 -- 172.21.15.188:0/2568515378 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8be29075-43eb-44ff-8f48-96316cff73f7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4384010040 con 0x7f438c15de80 2024-04-25T09:54:01.075 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-04-25T09:54:01.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 msgr2=0x7f438c0c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0c7830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f438400efc0 tx=0x7f4384009f90 comp rx=0 tx=0).stop 2024-04-25T09:54:01.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 shutdown_connections 2024-04-25T09:54:01.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438c15de80 0x7f438c0c7830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f438c0cefb0 0x7f438c0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 --2- 172.21.15.188:0/2568515378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f438c0ccfe0 0x7f438c150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 >> 172.21.15.188:0/2568515378 conn(0x7f438c0b2390 msgr2=0x7f438c0cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 shutdown_connections 2024-04-25T09:54:01.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-25T09:54:01.073+0000 7f439423a640 1 -- 172.21.15.188:0/2568515378 wait complete. 2024-04-25T09:54:01.086 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T09:54:01.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T09:54:01.089 INFO:tasks.ceph.osd.0:Started 2024-04-25T09:54:01.089 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T09:54:01.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T09:54:01.091 INFO:tasks.ceph.osd.1:Started 2024-04-25T09:54:01.091 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T09:54:01.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T09:54:01.093 INFO:tasks.ceph.osd.2:Started 2024-04-25T09:54:01.094 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T09:54:01.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T09:54:01.096 INFO:tasks.ceph.osd.3:Started 2024-04-25T09:54:01.096 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T09:54:01.096 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T09:54:01.098 INFO:tasks.ceph.osd.4:Started 2024-04-25T09:54:01.098 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T09:54:01.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T09:54:01.101 INFO:tasks.ceph.osd.5:Started 2024-04-25T09:54:01.101 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T09:54:01.101 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T09:54:01.103 INFO:tasks.ceph.osd.6:Started 2024-04-25T09:54:01.103 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T09:54:01.104 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T09:54:01.106 INFO:tasks.ceph.osd.7:Started 2024-04-25T09:54:01.107 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:01.163 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:01.161+0000 7fcb42893040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:54:01.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.249+0000 7f8c888ee640 1 Processor -- start 2024-04-25T09:54:01.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.249+0000 7f8c888ee640 1 -- start start 2024-04-25T09:54:01.253 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:01.250+0000 7f9c730b2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.250+0000 7f8c888ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c8415da80 0x7f8c840b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.250+0000 7f8c888ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ad5a0 0x7f8c8406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.250+0000 7f8c888ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c8406df50 0x7f8c8406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.250+0000 7f8c888ee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c840b4370 con 0x7f8c8406df50 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.250+0000 7f8c888ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8415de60 con 0x7f8c8415da80 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.250+0000 7f8c888ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b3760 con 0x7f8c840ad5a0 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c8406df50 0x7f8c8406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c8406df50 0x7f8c8406bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38194/0 (socket says 172.21.15.148:38194) 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 -- 172.21.15.148:0/338767893 learned_addr learned my addr 172.21.15.148:0/338767893 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c8259c640 1 --2- 172.21.15.148:0/338767893 >> v2:172.21.15.188:3300/0 conn(0x7f8c8415da80 0x7f8c840b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c81d9b640 1 --2- 172.21.15.148:0/338767893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ad5a0 0x7f8c8406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 -- 172.21.15.148:0/338767893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ad5a0 msgr2=0x7f8c8406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 --2- 172.21.15.148:0/338767893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ad5a0 0x7f8c8406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 -- 172.21.15.148:0/338767893 >> v2:172.21.15.188:3300/0 conn(0x7f8c8415da80 msgr2=0x7f8c840b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 --2- 172.21.15.148:0/338767893 >> v2:172.21.15.188:3300/0 conn(0x7f8c8415da80 0x7f8c840b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 -- 172.21.15.148:0/338767893 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840b3a40 con 0x7f8c8406df50 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.251+0000 7f8c82d9d640 1 --2- 172.21.15.148:0/338767893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8406df50 0x7f8c8406bbf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c840b5500 tx=0x7f8c7c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235dbdf6314e72af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c8159a640 1 -- 172.21.15.148:0/338767893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c036070 con 0x7f8c8406df50 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c8159a640 1 -- 172.21.15.148:0/338767893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c031600 con 0x7f8c8406df50 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c8159a640 1 -- 172.21.15.148:0/338767893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0319d0 con 0x7f8c8406df50 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 -- 172.21.15.148:0/338767893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8406df50 msgr2=0x7f8c8406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 --2- 172.21.15.148:0/338767893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8406df50 0x7f8c8406bbf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c840b5500 tx=0x7f8c7c02a730 comp rx=0 tx=0).stop 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 -- 172.21.15.148:0/338767893 shutdown_connections 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 --2- 172.21.15.148:0/338767893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8406df50 0x7f8c8406bbf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 --2- 172.21.15.148:0/338767893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ad5a0 0x7f8c8406da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 --2- 172.21.15.148:0/338767893 >> v2:172.21.15.188:3300/0 conn(0x7f8c8415da80 0x7f8c840b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 -- 172.21.15.148:0/338767893 >> 172.21.15.148:0/338767893 conn(0x7f8c84071b00 msgr2=0x7f8c84071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 -- 172.21.15.148:0/338767893 shutdown_connections 2024-04-25T09:54:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.252+0000 7f8c888ee640 1 -- 172.21.15.148:0/338767893 wait complete. 2024-04-25T09:54:01.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.254+0000 7f8c888ee640 1 Processor -- start 2024-04-25T09:54:01.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.254+0000 7f8c888ee640 1 -- start start 2024-04-25T09:54:01.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.254+0000 7f8c888ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c8406df50 0x7f8c84151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.254+0000 7f8c888ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 0x7f8c8406f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c888ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c8415da80 0x7f8c8406f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c888ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b3350 con 0x7f8c840ad5a0 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c888ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b4000 con 0x7f8c8406df50 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c888ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b34d0 con 0x7f8c8415da80 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c81d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 0x7f8c8406f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c81d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 0x7f8c8406f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38210/0 (socket says 172.21.15.148:38210) 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c81d9b640 1 -- 172.21.15.148:0/2494454582 learned_addr learned my addr 172.21.15.148:0/2494454582 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c8406df50 0x7f8c84151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c82d9d640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c8415da80 0x7f8c8406f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 -- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c8415da80 msgr2=0x7f8c8406f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c8415da80 0x7f8c8406f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 -- 172.21.15.148:0/2494454582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 msgr2=0x7f8c8406f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 0x7f8c8406f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 -- 172.21.15.148:0/2494454582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840cd560 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c81d9b640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 0x7f8c8406f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c8259c640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c8406df50 0x7f8c84151b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800e870 tx=0x7f8c7800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78017070 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800ee60 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8415ecd0 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800cb60 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.255+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8415f290 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.256+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8c7801a070 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.256+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8c78013b40 con 0x7f8c8406df50 2024-04-25T09:54:01.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.256+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c84073990 con 0x7f8c8406df50 2024-04-25T09:54:01.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.258+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8c7800cdf0 con 0x7f8c8406df50 2024-04-25T09:54:01.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.320+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c8406af60 con 0x7f8c8406df50 2024-04-25T09:54:01.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.320+0000 7f8c72ffd640 1 -- 172.21.15.148:0/2494454582 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f8c78013dc0 con 0x7f8c8406df50 2024-04-25T09:54:01.323 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:01.323 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":10,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:01.064025+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c8406df50 msgr2=0x7f8c84151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c8406df50 0x7f8c84151b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800e870 tx=0x7f8c7800ed30 comp rx=0 tx=0).stop 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 shutdown_connections 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c8415da80 0x7f8c8406f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c840ad5a0 0x7f8c8406f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 --2- 172.21.15.148:0/2494454582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c8406df50 0x7f8c84151b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 >> 172.21.15.148:0/2494454582 conn(0x7f8c84071b00 msgr2=0x7f8c840acdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 shutdown_connections 2024-04-25T09:54:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:01.321+0000 7f8c888ee640 1 -- 172.21.15.148:0/2494454582 wait complete. 2024-04-25T09:54:01.334 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T09:54:01.334 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T09:54:01.456 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:01.454+0000 7fcb42893040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:54:01.509 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:01.507+0000 7fcb42893040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:54:01.511 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:01.509+0000 7f9c730b2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T09:54:01.564 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:01.561+0000 7f9c730b2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T09:54:01.623 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:01.621+0000 7fcb42893040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:54:01.680 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:01.678+0000 7f9c730b2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T09:54:01.694 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-04-25T09:54:01.692+0000 7fbc40da7640 -1 Falling back to public interface 2024-04-25T09:54:01.705 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-04-25T09:54:01.703+0000 7fc2f761c640 -1 Falling back to public interface 2024-04-25T09:54:01.707 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-25T09:54:01.705+0000 7f1a90643640 -1 Falling back to public interface 2024-04-25T09:54:01.715 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-25T09:54:01.713+0000 7f4eb041c640 -1 Falling back to public interface 2024-04-25T09:54:01.717 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-04-25T09:54:01.715+0000 7fb570443640 -1 Falling back to public interface 2024-04-25T09:54:01.717 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-25T09:54:01.715+0000 7f680b216640 -1 Falling back to public interface 2024-04-25T09:54:01.718 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-04-25T09:54:01.716+0000 7ff413a0e640 -1 Falling back to public interface 2024-04-25T09:54:01.727 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-25T09:54:01.725+0000 7fdec182c640 -1 Falling back to public interface 2024-04-25T09:54:01.903 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:01.901+0000 7fcb42893040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:54:01.952 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:01.949+0000 7fcb42893040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:54:01.964 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:01.961+0000 7f9c730b2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T09:54:02.013 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.010+0000 7f9c730b2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T09:54:02.109 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:02.107+0000 7fcb42893040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:54:02.141 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:02.169 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.166+0000 7f9c730b2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T09:54:02.231 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.229+0000 7f5752207640 1 Processor -- start 2024-04-25T09:54:02.232 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.229+0000 7f5752207640 1 -- start start 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5752207640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c1448b0 0x7f574c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5752207640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f574c0728f0 0x7f574c0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5752207640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0b2080 con 0x7f574c06ab90 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5752207640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574c0d0d80 con 0x7f574c0728f0 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5752207640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0d0730 con 0x7f574c1448b0 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f574c0728f0 0x7f574c0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f574c0728f0 0x7f574c0b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38216/0 (socket says 172.21.15.148:38216) 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 -- 172.21.15.148:0/1126476707 learned_addr learned my addr 172.21.15.148:0/1126476707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f574b7fe640 1 --2- 172.21.15.148:0/1126476707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c1448b0 0x7f574c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 -- 172.21.15.148:0/1126476707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c1448b0 msgr2=0x7f574c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 --2- 172.21.15.148:0/1126476707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c1448b0 0x7f574c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.234 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 -- 172.21.15.148:0/1126476707 >> v1:172.21.15.188:6789/0 conn(0x7f574c06ab90 legacy=0x7f574c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.235 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.230+0000 7f5742ffd640 1 -- 172.21.15.148:0/1126476707 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0d0a10 con 0x7f574c0728f0 2024-04-25T09:54:02.235 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.231+0000 7f5742ffd640 1 --2- 172.21.15.148:0/1126476707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c0728f0 0x7f574c0b1980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5734009870 tx=0x7f573402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29e4ebbe4105ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.235 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.231+0000 7f574affd640 1 -- 172.21.15.148:0/1126476707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734037070 con 0x7f574c0728f0 2024-04-25T09:54:02.235 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.231+0000 7f574affd640 1 -- 172.21.15.148:0/1126476707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734032440 con 0x7f574c0728f0 2024-04-25T09:54:02.235 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.231+0000 7f574affd640 1 -- 172.21.15.148:0/1126476707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734032810 con 0x7f574c0728f0 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 -- 172.21.15.148:0/1126476707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c0728f0 msgr2=0x7f574c0b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 --2- 172.21.15.148:0/1126476707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c0728f0 0x7f574c0b1980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5734009870 tx=0x7f573402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 -- 172.21.15.148:0/1126476707 shutdown_connections 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 --2- 172.21.15.148:0/1126476707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c0728f0 0x7f574c0b1980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 --2- 172.21.15.148:0/1126476707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c1448b0 0x7f574c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 -- 172.21.15.148:0/1126476707 >> 172.21.15.148:0/1126476707 conn(0x7f574c0b5e50 msgr2=0x7f574c099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 -- 172.21.15.148:0/1126476707 shutdown_connections 2024-04-25T09:54:02.238 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.232+0000 7f5752207640 1 -- 172.21.15.148:0/1126476707 wait complete. 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.233+0000 7f5752207640 1 Processor -- start 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.233+0000 7f5752207640 1 -- start start 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5752207640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 0x7f574c15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5752207640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0728f0 0x7f574c0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5752207640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c1448b0 0x7f574c09c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5752207640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06b1a0 con 0x7f574c06ab90 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5752207640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06b020 con 0x7f574c1448b0 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5752207640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06b320 con 0x7f574c0728f0 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 0x7f574c15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 0x7f574c15b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38218/0 (socket says 172.21.15.148:38218) 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 -- 172.21.15.148:0/3397065347 learned_addr learned my addr 172.21.15.148:0/3397065347 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f5742ffd640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0728f0 0x7f574c0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574bfff640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c1448b0 0x7f574c09c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:02.239 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 -- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0728f0 msgr2=0x7f574c0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0728f0 0x7f574c0cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 -- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c1448b0 msgr2=0x7f574c09c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.234+0000 7f574b7fe640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c1448b0 0x7f574c09c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f574b7fe640 1 -- 172.21.15.148:0/3397065347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0cd880 con 0x7f574c06ab90 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f574b7fe640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 0x7f574c15b240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f573c00e8a0 tx=0x7f573c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c017070 con 0x7f574c06ab90 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c00c940 con 0x7f574c06ab90 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c0cdb60 con 0x7f574c06ab90 2024-04-25T09:54:02.240 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c00cd30 con 0x7f574c06ab90 2024-04-25T09:54:02.241 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c0ce120 con 0x7f574c06ab90 2024-04-25T09:54:02.241 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f573c01a070 con 0x7f574c06ab90 2024-04-25T09:54:02.241 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.235+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f573c01cb20 con 0x7f574c06ab90 2024-04-25T09:54:02.241 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.236+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c04ef20 con 0x7f574c06ab90 2024-04-25T09:54:02.241 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.238+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f573c01c450 con 0x7f574c06ab90 2024-04-25T09:54:02.251 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:02.249+0000 7fcb42893040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:54:02.303 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.300+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f574c06eb50 con 0x7f574c06ab90 2024-04-25T09:54:02.303 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.300+0000 7f5748ff9640 1 -- 172.21.15.148:0/3397065347 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f573c01c630 con 0x7f574c06ab90 2024-04-25T09:54:02.303 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:02.300+0000 7fcb42893040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:54:02.303 INFO:teuthology.misc.health.smithi148.stdout: 2024-04-25T09:54:02.304 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":10,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:01.064025+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:02.304 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 msgr2=0x7f574c15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:02.304 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 0x7f574c15b240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f573c00e8a0 tx=0x7f573c00ed60 comp rx=0 tx=0).stop 2024-04-25T09:54:02.304 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 shutdown_connections 2024-04-25T09:54:02.305 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574c1448b0 0x7f574c09c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.305 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f574c0728f0 0x7f574c0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.305 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 --2- 172.21.15.148:0/3397065347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f574c06ab90 0x7f574c15b240 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:02.305 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 >> 172.21.15.148:0/3397065347 conn(0x7f574c0b5e50 msgr2=0x7f574c06f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:02.305 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 shutdown_connections 2024-04-25T09:54:02.305 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:02.301+0000 7f5752207640 1 -- 172.21.15.148:0/3397065347 wait complete. 2024-04-25T09:54:02.305 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.303+0000 7f9c730b2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T09:54:02.313 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:54:02.355 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.353+0000 7f9c730b2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T09:54:02.375 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:02.373+0000 7fcb42893040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:54:02.420 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:02.417+0000 7fcb42893040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:54:02.424 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.422+0000 7f9c730b2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T09:54:02.469 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.466+0000 7f9c730b2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T09:54:02.895 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:02.893+0000 7fcb42893040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:54:02.946 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:02.944+0000 7f9c730b2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T09:54:03.010 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:03.008+0000 7fcb42893040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:54:03.065 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-04-25T09:54:03.063+0000 7f9c730b2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T09:54:04.028 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:04.026+0000 7fcae3cfc640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T09:54:04.028 INFO:tasks.ceph.mgr.y.smithi148.stderr:2024-04-25T09:54:04.026+0000 7fcae3cfc640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T09:54:05.177 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-04-25T09:54:05.175+0000 7ff413a0e640 -1 osd.3 0 log_to_monitors true 2024-04-25T09:54:05.179 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-25T09:54:05.177+0000 7f1a90643640 -1 osd.4 0 log_to_monitors true 2024-04-25T09:54:05.214 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-04-25T09:54:05.212+0000 7fb570443640 -1 osd.0 0 log_to_monitors true 2024-04-25T09:54:05.233 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-04-25T09:54:05.230+0000 7fc2f761c640 -1 osd.2 0 log_to_monitors true 2024-04-25T09:54:05.233 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-04-25T09:54:05.231+0000 7fbc40da7640 -1 osd.1 0 log_to_monitors true 2024-04-25T09:54:05.253 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-25T09:54:05.250+0000 7fdec182c640 -1 osd.7 0 log_to_monitors true 2024-04-25T09:54:05.256 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-25T09:54:05.253+0000 7f680b216640 -1 osd.6 0 log_to_monitors true 2024-04-25T09:54:05.260 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-25T09:54:05.257+0000 7f4eb041c640 -1 osd.5 0 log_to_monitors true 2024-04-25T09:54:08.966 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-25T09:54:08.964+0000 7fdebd64c640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T09:54:08.986 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-04-25T09:54:08.984+0000 7fdeb4c58640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:08.992 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-04-25T09:54:08.988+0000 7fb56c3d9640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T09:54:09.007 INFO:tasks.ceph.osd.0.smithi148.stderr:2024-04-25T09:54:09.005+0000 7fb5641f9640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.116 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-04-25T09:54:09.114+0000 7ff40f3df640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T09:54:09.120 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-25T09:54:09.117+0000 7f1a8c5d9640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T09:54:09.120 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:09.132 INFO:tasks.ceph.osd.3.smithi148.stderr:2024-04-25T09:54:09.130+0000 7ff4071ff640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.134 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-25T09:54:09.132+0000 7f4eabd0c640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T09:54:09.136 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-04-25T09:54:09.134+0000 7f1a843f9640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.146 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-04-25T09:54:09.144+0000 7f4ea3b2c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.153 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-04-25T09:54:09.151+0000 7fc2f3059640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T09:54:09.171 INFO:tasks.ceph.osd.2.smithi148.stderr:2024-04-25T09:54:09.169+0000 7fc2eae79640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.177 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-04-25T09:54:09.175+0000 7fbc3cd3d640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T09:54:09.191 INFO:tasks.ceph.osd.1.smithi148.stderr:2024-04-25T09:54:09.189+0000 7fbc34b5d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.217 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.214+0000 7f03a016a640 1 Processor -- start 2024-04-25T09:54:09.217 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f03a016a640 1 -- start start 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f03a016a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03980af350 0x7f0398162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f03a016a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03980af730 con 0x7f039806e500 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f03a016a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039806ab90 con 0x7f039815e180 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f03a016a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039815e6f0 con 0x7f03980af350 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f039df06640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03980af350 0x7f0398162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f039e707640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f039806e500 0x7f039806b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54418/0 (socket says 172.21.15.148:54418) 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f039e707640 1 -- 172.21.15.148:0/1412233433 learned_addr learned my addr 172.21.15.148:0/1412233433 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.215+0000 7f039df06640 1 --2- 172.21.15.148:0/1412233433 >> v2:172.21.15.148:3300/0 conn(0x7f03980af350 0x7f0398162180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36224/0 (socket says 172.21.15.148:36224) 2024-04-25T09:54:09.218 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.216+0000 7f039df06640 1 -- 172.21.15.148:0/1412233433 >> v1:172.21.15.188:6789/0 conn(0x7f039815e180 legacy=0x7f039806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.219 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.216+0000 7f039df06640 1 -- 172.21.15.148:0/1412233433 >> v1:172.21.15.148:6789/0 conn(0x7f039806e500 legacy=0x7f039806b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.219 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.216+0000 7f039df06640 1 -- 172.21.15.148:0/1412233433 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039815e9d0 con 0x7f03980af350 2024-04-25T09:54:09.219 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.216+0000 7f039cf04640 1 -- 172.21.15.148:0/1412233433 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731763297 0 0) 0x7f0388001490 con 0x7f039806e500 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.216+0000 7f039df06640 1 --2- 172.21.15.148:0/1412233433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03980af350 0x7f0398162180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03900097a0 tx=0x7f039002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9c0552c5d51e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f039cf04640 1 -- 172.21.15.148:0/1412233433 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390037070 con 0x7f03980af350 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f039cf04640 1 -- 172.21.15.148:0/1412233433 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390032440 con 0x7f03980af350 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f039cf04640 1 -- 172.21.15.148:0/1412233433 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390032810 con 0x7f03980af350 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 -- 172.21.15.148:0/1412233433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03980af350 msgr2=0x7f0398162180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 --2- 172.21.15.148:0/1412233433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03980af350 0x7f0398162180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03900097a0 tx=0x7f039002a230 comp rx=0 tx=0).stop 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 -- 172.21.15.148:0/1412233433 shutdown_connections 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 --2- 172.21.15.148:0/1412233433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03980af350 0x7f0398162180 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 -- 172.21.15.148:0/1412233433 >> 172.21.15.148:0/1412233433 conn(0x7f03980cfe90 msgr2=0x7f03980d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 -- 172.21.15.148:0/1412233433 shutdown_connections 2024-04-25T09:54:09.220 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.217+0000 7f03a016a640 1 -- 172.21.15.148:0/1412233433 wait complete. 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.218+0000 7f03a016a640 1 Processor -- start 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.218+0000 7f03a016a640 1 -- start start 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f03a016a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f039806e500 0x7f0398151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f03a016a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03980af350 0x7f03981599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f03a016a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 0x7f03980b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f03a016a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039806ebc0 con 0x7f039815e180 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f03a016a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039815ef90 con 0x7f039806e500 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f03a016a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039806b7a0 con 0x7f03980af350 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 0x7f03980b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 0x7f03980b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36230/0 (socket says 172.21.15.148:36230) 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 -- 172.21.15.148:0/2905204800 learned_addr learned my addr 172.21.15.148:0/2905204800 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039df06640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f039806e500 0x7f0398151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.223 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039d705640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03980af350 0x7f03981599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 -- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03980af350 msgr2=0x7f03981599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03980af350 0x7f03981599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 -- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f039806e500 msgr2=0x7f0398151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f039806e500 0x7f0398151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.219+0000 7f039e707640 1 -- 172.21.15.148:0/2905204800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398152080 con 0x7f039815e180 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.220+0000 7f039e707640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 0x7f03980b09e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f038800c7f0 tx=0x7f038800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.220+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388018070 con 0x7f039815e180 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.220+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038800a5e0 con 0x7f039815e180 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.220+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038800a9d0 con 0x7f039815e180 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.220+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0398159f30 con 0x7f039815e180 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.220+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0398158f10 con 0x7f039815e180 2024-04-25T09:54:09.224 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.221+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039804ef20 con 0x7f039815e180 2024-04-25T09:54:09.225 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.221+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f038801b070 con 0x7f039815e180 2024-04-25T09:54:09.225 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.221+0000 7f03967fc640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f038406b440 0x7f038406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:09.225 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.222+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f0388088890 con 0x7f039815e180 2024-04-25T09:54:09.225 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.222+0000 7f039df06640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f038406b440 0x7f038406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:09.225 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.222+0000 7f039df06640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f038406b440 0x7f038406d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0390034000 tx=0x7f039002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:09.228 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.226+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03980af730 con 0x7f039815e180 2024-04-25T09:54:09.271 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-25T09:54:09.268+0000 7f6806c2a640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T09:54:09.289 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-04-25T09:54:09.287+0000 7f67fea4a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T09:54:09.400 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.398+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0398066eb0 con 0x7f039815e180 2024-04-25T09:54:09.401 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.399+0000 7f03967fc640 1 -- 172.21.15.148:0/2905204800 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f038805bbb0 con 0x7f039815e180 2024-04-25T09:54:09.401 INFO:teuthology.misc.health.smithi148.stdout: 2024-04-25T09:54:09.401 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":12,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:07.032465+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:09.403 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f038406b440 msgr2=0x7f038406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.403 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f038406b440 0x7f038406d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0390034000 tx=0x7f039002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:09.403 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 msgr2=0x7f03980b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:09.403 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 0x7f03980b09e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f038800c7f0 tx=0x7f038800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.403 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 shutdown_connections 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f038406b440 0x7f038406d900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f039815e180 0x7f03980b09e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03980af350 0x7f03981599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 --2- 172.21.15.148:0/2905204800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f039806e500 0x7f0398151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 >> 172.21.15.148:0/2905204800 conn(0x7f03980cfe90 msgr2=0x7f03980d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 shutdown_connections 2024-04-25T09:54:09.404 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:09.401+0000 7f03a016a640 1 -- 172.21.15.148:0/2905204800 wait complete. 2024-04-25T09:54:09.413 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T09:54:16.219 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:16.319 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.317+0000 7fbf56b1d640 1 Processor -- start 2024-04-25T09:54:16.319 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.317+0000 7fbf56b1d640 1 -- start start 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf56b1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf5015cd40 0x7fbf500cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf56b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf500ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf56b1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf500cc060 con 0x7fbf501561e0 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf56b1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500cf940 con 0x7fbf5015cd40 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf56b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500ceae0 con 0x7fbf500ce150 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf5015cd40 0x7fbf500cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.321 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf5015cd40 0x7fbf500cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51278/0 (socket says 172.21.15.148:51278) 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 -- 172.21.15.148:0/3253151245 learned_addr learned my addr 172.21.15.148:0/3253151245 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf550ba640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf500ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 -- 172.21.15.148:0/3253151245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 msgr2=0x7fbf500ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf500ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 -- 172.21.15.148:0/3253151245 >> v1:172.21.15.188:6789/0 conn(0x7fbf501561e0 legacy=0x7fbf500cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf4ffff640 1 -- 172.21.15.148:0/3253151245 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500cc300 con 0x7fbf5015cd40 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.318+0000 7fbf550ba640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf500ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf4ffff640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf500cf400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009870 tx=0x7fbf3c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c2dd281f816b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf4f7fe640 1 -- 172.21.15.148:0/3253151245 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c037070 con 0x7fbf5015cd40 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf4f7fe640 1 -- 172.21.15.148:0/3253151245 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c032440 con 0x7fbf5015cd40 2024-04-25T09:54:16.322 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf4f7fe640 1 -- 172.21.15.148:0/3253151245 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c032810 con 0x7fbf5015cd40 2024-04-25T09:54:16.325 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf56b1d640 1 -- 172.21.15.148:0/3253151245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 msgr2=0x7fbf500cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.325 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf500cf400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009870 tx=0x7fbf3c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.325 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf56b1d640 1 -- 172.21.15.148:0/3253151245 shutdown_connections 2024-04-25T09:54:16.325 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf500cf400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.325 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/3253151245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf500ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.319+0000 7fbf56b1d640 1 -- 172.21.15.148:0/3253151245 >> 172.21.15.148:0/3253151245 conn(0x7fbf500b3460 msgr2=0x7fbf500b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.320+0000 7fbf56b1d640 1 -- 172.21.15.148:0/3253151245 shutdown_connections 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.320+0000 7fbf56b1d640 1 -- 172.21.15.148:0/3253151245 wait complete. 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.321+0000 7fbf56b1d640 1 Processor -- start 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.321+0000 7fbf56b1d640 1 -- start start 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf5015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf501561e0 0x7fbf501f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf50157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500ccba0 con 0x7fbf5015cd40 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500cc8c0 con 0x7fbf501561e0 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501565c0 con 0x7fbf500ce150 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf50157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.326 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf50157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51284/0 (socket says 172.21.15.148:51284) 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 -- 172.21.15.148:0/774345043 learned_addr learned my addr 172.21.15.148:0/774345043 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf4ffff640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf501561e0 0x7fbf501f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf548b9640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf5015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 -- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 msgr2=0x7fbf5015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf5015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 -- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf501561e0 msgr2=0x7fbf501f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf501561e0 0x7fbf501f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 -- 172.21.15.148:0/774345043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50156a40 con 0x7fbf5015cd40 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf4ffff640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf501561e0 0x7fbf501f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf550ba640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf50157d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf48007ef0 tx=0x7fbf4800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4801a070 con 0x7fbf5015cd40 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf48003b90 con 0x7fbf5015cd40 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf50156d20 con 0x7fbf5015cd40 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48004030 con 0x7fbf5015cd40 2024-04-25T09:54:16.327 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.322+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf501572e0 con 0x7fbf5015cd40 2024-04-25T09:54:16.328 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.323+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf500b77e0 con 0x7fbf5015cd40 2024-04-25T09:54:16.328 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.324+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf4801d070 con 0x7fbf5015cd40 2024-04-25T09:54:16.328 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.324+0000 7fbf4d7fa640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3406b500 0x7fbf3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.328 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.324+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf4808a060 con 0x7fbf5015cd40 2024-04-25T09:54:16.329 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.324+0000 7fbf548b9640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3406b500 0x7fbf3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.329 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.325+0000 7fbf548b9640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3406b500 0x7fbf3406d9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf44000f60 tx=0x7fbf44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.331 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.329+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf500b77e0 con 0x7fbf5015cd40 2024-04-25T09:54:16.502 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.500+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf5004ef20 con 0x7fbf5015cd40 2024-04-25T09:54:16.503 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.501+0000 7fbf4d7fa640 1 -- 172.21.15.148:0/774345043 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7fbf48010070 con 0x7fbf5015cd40 2024-04-25T09:54:16.503 INFO:teuthology.misc.health.smithi148.stdout: 2024-04-25T09:54:16.503 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":16,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:13.076119+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:16.505 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.502+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3406b500 msgr2=0x7fbf3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.505 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.502+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3406b500 0x7fbf3406d9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf44000f60 tx=0x7fbf44009040 comp rx=0 tx=0).stop 2024-04-25T09:54:16.505 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.502+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 msgr2=0x7fbf50157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.505 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf50157d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf48007ef0 tx=0x7fbf4800d4a0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.505 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 shutdown_connections 2024-04-25T09:54:16.505 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3406b500 0x7fbf3406d9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.506 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5015cd40 0x7fbf50157d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.506 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf501561e0 0x7fbf501f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.506 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 --2- 172.21.15.148:0/774345043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf500ce150 0x7fbf5015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.506 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 >> 172.21.15.148:0/774345043 conn(0x7fbf500b3460 msgr2=0x7fbf50073660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.506 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 shutdown_connections 2024-04-25T09:54:16.506 INFO:teuthology.misc.health.smithi148.stderr:2024-04-25T09:54:16.503+0000 7fbf56b1d640 1 -- 172.21.15.148:0/774345043 wait complete. 2024-04-25T09:54:16.515 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T09:54:16.515 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T09:54:16.515 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:54:16.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:54:16.517 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:54:16.517 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:54:16.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:54:16.519 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:54:16.520 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:54:16.520 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:54:16.523 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:54:16.523 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:54:16.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:54:16.525 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:54:16.526 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T09:54:16.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:16.590 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:54:16.590 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:54:16.605 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T09:54:16.605 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:54:16.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.625+0000 7f05ff84a640 1 Processor -- start 2024-04-25T09:54:16.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.625+0000 7f05ff84a640 1 -- start start 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.626+0000 7f05ff84a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f80b2cb0 0x7f05f80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.626+0000 7f05ff84a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f80b97a0 con 0x7f05f815dbe0 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fcde5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f80b2cb0 0x7f05f80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fdde7640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f05f80d1020 0x7f05f80d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:35016/0 (socket says 172.21.15.148:35016) 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fd5e6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f05f815dbe0 0x7f05f80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44290/0 (socket says 172.21.15.148:44290) 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05ff84a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f80d1c50 con 0x7f05f80d1020 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fdde7640 1 -- 172.21.15.148:0/2862014730 learned_addr learned my addr 172.21.15.148:0/2862014730 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05ff84a640 1 -- 172.21.15.148:0/2862014730 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f815dfc0 con 0x7f05f80b2cb0 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05effff640 1 -- 172.21.15.148:0/2862014730 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199649154 0 0) 0x7f05f80b97a0 con 0x7f05f815dbe0 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05effff640 1 -- 172.21.15.148:0/2862014730 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e4003520 con 0x7f05f815dbe0 2024-04-25T09:54:16.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05effff640 1 -- 172.21.15.148:0/2862014730 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609717305 0 0) 0x7f05f80d1c50 con 0x7f05f80d1020 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fcde5640 1 -- 172.21.15.148:0/2862014730 >> v1:172.21.15.188:6789/0 conn(0x7f05f80d1020 legacy=0x7f05f80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fcde5640 1 -- 172.21.15.148:0/2862014730 >> v1:172.21.15.148:6789/0 conn(0x7f05f815dbe0 legacy=0x7f05f80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.627+0000 7f05fcde5640 1 -- 172.21.15.148:0/2862014730 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80d1f30 con 0x7f05f80b2cb0 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05fcde5640 1 --2- 172.21.15.148:0/2862014730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f80b2cb0 0x7f05f80d2730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05e80097d0 tx=0x7f05e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f4fcf0ee408f64b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05effff640 1 -- 172.21.15.148:0/2862014730 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037070 con 0x7f05f80b2cb0 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05effff640 1 -- 172.21.15.148:0/2862014730 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8032440 con 0x7f05f80b2cb0 2024-04-25T09:54:16.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05effff640 1 -- 172.21.15.148:0/2862014730 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8032810 con 0x7f05f80b2cb0 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 -- 172.21.15.148:0/2862014730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f80b2cb0 msgr2=0x7f05f80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2862014730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f80b2cb0 0x7f05f80d2730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05e80097d0 tx=0x7f05e802a210 comp rx=0 tx=0).stop 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 -- 172.21.15.148:0/2862014730 shutdown_connections 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2862014730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f80b2cb0 0x7f05f80d2730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 -- 172.21.15.148:0/2862014730 >> 172.21.15.148:0/2862014730 conn(0x7f05f8061a20 msgr2=0x7f05f8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 -- 172.21.15.148:0/2862014730 shutdown_connections 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.628+0000 7f05ff84a640 1 -- 172.21.15.148:0/2862014730 wait complete. 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.629+0000 7f05ff84a640 1 Processor -- start 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 -- start start 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 0x7f05f8158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05f80d1020 0x7f05f814b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f815dbe0 0x7f05f8160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80b9420 con 0x7f05f80b2cb0 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80b92a0 con 0x7f05f80d1020 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05ff84a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80b6ab0 con 0x7f05f815dbe0 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05fd5e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 0x7f05f8158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05fd5e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 0x7f05f8158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51336/0 (socket says 172.21.15.148:51336) 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05fd5e6640 1 -- 172.21.15.148:0/2408010276 learned_addr learned my addr 172.21.15.148:0/2408010276 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05fdde7640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f815dbe0 0x7f05f8160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.630+0000 7f05fcde5640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05f80d1020 0x7f05f814b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fd5e6640 1 -- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f815dbe0 msgr2=0x7f05f8160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fd5e6640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f815dbe0 0x7f05f8160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fd5e6640 1 -- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05f80d1020 msgr2=0x7f05f814b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fd5e6640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05f80d1020 0x7f05f814b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fd5e6640 1 -- 172.21.15.148:0/2408010276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8157440 con 0x7f05f80b2cb0 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fcde5640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05f80d1020 0x7f05f814b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05fd5e6640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 0x7f05f8158200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c390 tx=0x7f05f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40074e0 con 0x7f05f80b2cb0 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f40047e0 con 0x7f05f80b2cb0 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4004bd0 con 0x7f05f80b2cb0 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f8157720 con 0x7f05f80b2cb0 2024-04-25T09:54:16.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.631+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f81f7850 con 0x7f05f80b2cb0 2024-04-25T09:54:16.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.632+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f8073990 con 0x7f05f80b2cb0 2024-04-25T09:54:16.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.632+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05f401a020 con 0x7f05f80b2cb0 2024-04-25T09:54:16.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.633+0000 7f05edffb640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.633+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05f4089560 con 0x7f05f80b2cb0 2024-04-25T09:54:16.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.633+0000 7f05fcde5640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.633+0000 7f05fcde5640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05dc06b4d0 0x7f05dc06d990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05e8034000 tx=0x7f05e802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.637+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05f4058880 con 0x7f05f80b2cb0 2024-04-25T09:54:16.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.808+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05f80d3400 con 0x7f05f80b2cb0 2024-04-25T09:54:16.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.809+0000 7f05edffb640 1 -- 172.21.15.148:0/2408010276 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7f05f4058180 con 0x7f05f80b2cb0 2024-04-25T09:54:16.811 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:16.811 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:13.076119+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05dc06b4d0 msgr2=0x7f05dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05dc06b4d0 0x7f05dc06d990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05e8034000 tx=0x7f05e802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 msgr2=0x7f05f8158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 0x7f05f8158200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f05f400c390 tx=0x7f05f400c850 comp rx=0 tx=0).stop 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 shutdown_connections 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05dc06b4d0 0x7f05dc06d990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05f815dbe0 0x7f05f8160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05f80d1020 0x7f05f814b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 --2- 172.21.15.148:0/2408010276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05f80b2cb0 0x7f05f8158200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 >> 172.21.15.148:0/2408010276 conn(0x7f05f8061a20 msgr2=0x7f05f80b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 shutdown_connections 2024-04-25T09:54:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.811+0000 7f05ff84a640 1 -- 172.21.15.148:0/2408010276 wait complete. 2024-04-25T09:54:16.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:16.824 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:16.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.922+0000 7f5a64b5a640 1 Processor -- start 2024-04-25T09:54:16.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.922+0000 7f5a64b5a640 1 -- start start 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a64b5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6015dc20 0x7f5a600ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a64b5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a60161710 0x7f5a600cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a64b5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6015e000 con 0x7f5a600b9330 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a64b5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a600d1de0 con 0x7f5a60161710 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a64b5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60161c60 con 0x7f5a6015dc20 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a60161710 0x7f5a600cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a60161710 0x7f5a600cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51350/0 (socket says 172.21.15.148:51350) 2024-04-25T09:54:16.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/1734807321 learned_addr learned my addr 172.21.15.148:0/1734807321 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5e59c640 1 --2- 172.21.15.148:0/1734807321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6015dc20 0x7f5a600ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/1734807321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6015dc20 msgr2=0x7f5a600ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 --2- 172.21.15.148:0/1734807321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6015dc20 0x7f5a600ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/1734807321 >> v1:172.21.15.188:6789/0 conn(0x7f5a600b9330 legacy=0x7f5a6006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.923+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/1734807321 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a60161f40 con 0x7f5a60161710 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.924+0000 7f5a5dd9b640 1 --2- 172.21.15.148:0/1734807321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a60161710 0x7f5a600cebb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009870 tx=0x7f5a5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32902c8064a8ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a5d59a640 1 -- 172.21.15.148:0/1734807321 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54037070 con 0x7f5a60161710 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a5d59a640 1 -- 172.21.15.148:0/1734807321 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54032440 con 0x7f5a60161710 2024-04-25T09:54:16.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a5d59a640 1 -- 172.21.15.148:0/1734807321 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54032810 con 0x7f5a60161710 2024-04-25T09:54:16.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 -- 172.21.15.148:0/1734807321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a60161710 msgr2=0x7f5a600cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/1734807321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a60161710 0x7f5a600cebb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009870 tx=0x7f5a5402a270 comp rx=0 tx=0).stop 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 -- 172.21.15.148:0/1734807321 shutdown_connections 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/1734807321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a60161710 0x7f5a600cebb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/1734807321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6015dc20 0x7f5a600ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 -- 172.21.15.148:0/1734807321 >> 172.21.15.148:0/1734807321 conn(0x7f5a600d0420 msgr2=0x7f5a600d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 -- 172.21.15.148:0/1734807321 shutdown_connections 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.925+0000 7f5a64b5a640 1 -- 172.21.15.148:0/1734807321 wait complete. 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 Processor -- start 2024-04-25T09:54:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 -- start start 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a600b9330 0x7f5a6014cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 0x7f5a60160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a60161710 0x7f5a60146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600b5e50 con 0x7f5a6015dc20 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60162500 con 0x7f5a60161710 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.927+0000 7f5a64b5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600b5fd0 con 0x7f5a600b9330 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 0x7f5a60160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 0x7f5a60160610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51352/0 (socket says 172.21.15.148:51352) 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/2128449744 learned_addr learned my addr 172.21.15.148:0/2128449744 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5e59c640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a600b9330 0x7f5a6014cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5ed9d640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a60161710 0x7f5a60146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a600b9330 msgr2=0x7f5a6014cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a600b9330 0x7f5a6014cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a60161710 msgr2=0x7f5a60146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a60161710 0x7f5a60146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 -- 172.21.15.148:0/2128449744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600ca1b0 con 0x7f5a6015dc20 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5e59c640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a600b9330 0x7f5a6014cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a5dd9b640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 0x7f5a60160610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000c00 tx=0x7f5a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54037040 con 0x7f5a6015dc20 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54042070 con 0x7f5a6015dc20 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a600c9110 con 0x7f5a6015dc20 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54030880 con 0x7f5a6015dc20 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.928+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a600c96d0 con 0x7f5a6015dc20 2024-04-25T09:54:16.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.929+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a60073a00 con 0x7f5a6015dc20 2024-04-25T09:54:16.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.929+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a54030a20 con 0x7f5a6015dc20 2024-04-25T09:54:16.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.930+0000 7f5a4effd640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4006b4d0 0x7f5a4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:16.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.930+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5a540aa770 con 0x7f5a6015dc20 2024-04-25T09:54:16.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.930+0000 7f5a5e59c640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4006b4d0 0x7f5a4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:16.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.930+0000 7f5a5e59c640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4006b4d0 0x7f5a4006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009a20 tx=0x7f5a50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:16.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:16.934+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a54079940 con 0x7f5a6015dc20 2024-04-25T09:54:17.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.107+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a600d16b0 con 0x7f5a6015dc20 2024-04-25T09:54:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.107+0000 7f5a4effd640 1 -- 172.21.15.148:0/2128449744 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a54079240 con 0x7f5a6015dc20 2024-04-25T09:54:17.110 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:54:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4006b4d0 msgr2=0x7f5a4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4006b4d0 0x7f5a4006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009a20 tx=0x7f5a50008040 comp rx=0 tx=0).stop 2024-04-25T09:54:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 msgr2=0x7f5a60160610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 0x7f5a60160610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000c00 tx=0x7f5a540046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 shutdown_connections 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4006b4d0 0x7f5a4006d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a60161710 0x7f5a60146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6015dc20 0x7f5a60160610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 --2- 172.21.15.148:0/2128449744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a600b9330 0x7f5a6014cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 >> 172.21.15.148:0/2128449744 conn(0x7f5a600d0420 msgr2=0x7f5a600d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 shutdown_connections 2024-04-25T09:54:17.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.109+0000 7f5a64b5a640 1 -- 172.21.15.148:0/2128449744 wait complete. 2024-04-25T09:54:17.122 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.217+0000 7f0883ccb640 1 Processor -- start 2024-04-25T09:54:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.217+0000 7f0883ccb640 1 -- start start 2024-04-25T09:54:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0883ccb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f087c15dc00 0x7f087c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0883ccb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c0d1dd0 0x7f087c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0883ccb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c15dfe0 con 0x7f087c0ceb20 2024-04-25T09:54:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0883ccb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0d00c0 con 0x7f087c15dc00 2024-04-25T09:54:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0883ccb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0d03a0 con 0x7f087c0d1dd0 2024-04-25T09:54:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0881266640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f087c0ceb20 0x7f087c0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44294/0 (socket says 172.21.15.148:44294) 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0881266640 1 -- 172.21.15.148:0/3590115208 learned_addr learned my addr 172.21.15.148:0/3590115208 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847320060 0 0) 0x7f086c001110 con 0x7f087c0ceb20 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0881a67640 1 --2- 172.21.15.148:0/3590115208 >> v2:172.21.15.188:3300/0 conn(0x7f087c15dc00 0x7f087c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0868003520 con 0x7f087c0ceb20 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0882268640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c0d1dd0 0x7f087c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3869130983 0 0) 0x7f0868003520 con 0x7f087c0ceb20 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c0d1dd0 msgr2=0x7f087c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c0d1dd0 0x7f087c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 >> v2:172.21.15.188:3300/0 conn(0x7f087c15dc00 msgr2=0x7f087c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 --2- 172.21.15.148:0/3590115208 >> v2:172.21.15.188:3300/0 conn(0x7f087c15dc00 0x7f087c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0cc440 con 0x7f087c0ceb20 2024-04-25T09:54:17.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f086c001840 con 0x7f087c0ceb20 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.218+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 >> v1:172.21.15.148:6789/0 conn(0x7f087c0ceb20 legacy=0x7f087c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0880a65640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08680039b0 0x7f0868003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0880a65640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08680043a0 0x7f0868004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0880a65640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0868004fa0 0x7f086800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0d00c0 con 0x7f08680043a0 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0d03a0 con 0x7f0868004fa0 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086800bf30 con 0x7f08680039b0 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0882268640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c0d1dd0 0x7f087c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08680043a0 0x7f0868004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881a67640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08680039b0 0x7f0868003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0882268640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0868004fa0 0x7f086800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 -- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08680039b0 msgr2=0x7f0868003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08680039b0 0x7f0868003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 -- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0868004fa0 msgr2=0x7f086800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0868004fa0 0x7f086800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 -- 172.21.15.148:0/3590115208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868003520 con 0x7f08680043a0 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.219+0000 7f0881266640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08680043a0 0x7f0868004880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c3e0 tx=0x7f0868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4604fbfcca8c1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c032020 con 0x7f08680043a0 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c005af0 con 0x7f08680043a0 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0880a65640 1 -- 172.21.15.148:0/3590115208 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00c0e0 con 0x7f08680043a0 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0883ccb640 1 -- 172.21.15.148:0/3590115208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08680043a0 msgr2=0x7f0868004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0883ccb640 1 --2- 172.21.15.148:0/3590115208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08680043a0 0x7f0868004880 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f086c00c3e0 tx=0x7f0868005830 comp rx=0 tx=0).stop 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0882268640 1 -- 172.21.15.148:0/3590115208 reap_dead start 2024-04-25T09:54:17.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0883ccb640 1 -- 172.21.15.148:0/3590115208 shutdown_connections 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0883ccb640 1 -- 172.21.15.148:0/3590115208 >> 172.21.15.148:0/3590115208 conn(0x7f087c06f5d0 msgr2=0x7f087c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0883ccb640 1 -- 172.21.15.148:0/3590115208 shutdown_connections 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.220+0000 7f0883ccb640 1 -- 172.21.15.148:0/3590115208 wait complete. 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.221+0000 7f0883ccb640 1 Processor -- start 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.221+0000 7f0883ccb640 1 -- start start 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0883ccb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08680043a0 0x7f087c1e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0883ccb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 0x7f087c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0883ccb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c15d230 0x7f087c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0883ccb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0cfd90 con 0x7f087c15a260 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0883ccb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0cc130 con 0x7f08680043a0 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0883ccb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0cdd70 con 0x7f087c15d230 2024-04-25T09:54:17.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0881266640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 0x7f087c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0881266640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 0x7f087c15ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51366/0 (socket says 172.21.15.148:51366) 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0881266640 1 -- 172.21.15.148:0/410841039 learned_addr learned my addr 172.21.15.148:0/410841039 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0881a67640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08680043a0 0x7f087c1e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.222+0000 7f0882268640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c15d230 0x7f087c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881266640 1 -- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c15d230 msgr2=0x7f087c14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881266640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c15d230 0x7f087c14bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881266640 1 -- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08680043a0 msgr2=0x7f087c1e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881266640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08680043a0 0x7f087c1e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881266640 1 -- 172.21.15.148:0/410841039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c1e86d0 con 0x7f087c15a260 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881a67640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08680043a0 0x7f087c1e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0881266640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 0x7f087c15ccf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f086c031040 tx=0x7f086c001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c032030 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c007270 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c1e89b0 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c007620 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.223+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c1e8f70 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.224+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c0cfb60 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.224+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f086c035020 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.225+0000 7f08727fc640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f086006b4d0 0x7f086006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.225+0000 7f0881a67640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f086006b4d0 0x7f086006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.225+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f086c0aa260 con 0x7f087c15a260 2024-04-25T09:54:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.225+0000 7f0881a67640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f086006b4d0 0x7f086006d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08740074d0 tx=0x7f087400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:17.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.229+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f087c0cfb60 con 0x7f087c15a260 2024-04-25T09:54:17.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.436+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087c0c66c0 con 0x7f087c15a260 2024-04-25T09:54:17.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.436+0000 7f08727fc640 1 -- 172.21.15.148:0/410841039 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f087c15dfe0 con 0x7f087c15a260 2024-04-25T09:54:17.439 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:17.439 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T09:54:17.439 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 2 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f086006b4d0 msgr2=0x7f086006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f086006b4d0 0x7f086006d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08740074d0 tx=0x7f087400b040 comp rx=0 tx=0).stop 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 msgr2=0x7f087c15ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 0x7f087c15ccf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f086c031040 tx=0x7f086c001c10 comp rx=0 tx=0).stop 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 shutdown_connections 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f086006b4d0 0x7f086006d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f087c15d230 0x7f087c14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f087c15a260 0x7f087c15ccf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 --2- 172.21.15.148:0/410841039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08680043a0 0x7f087c1e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:17.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 >> 172.21.15.148:0/410841039 conn(0x7f087c06f5d0 msgr2=0x7f087c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:17.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 shutdown_connections 2024-04-25T09:54:17.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:17.438+0000 7f0883ccb640 1 -- 172.21.15.148:0/410841039 wait complete. 2024-04-25T09:54:19.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:19.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.558+0000 7ffa27b30640 1 Processor -- start 2024-04-25T09:54:19.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.558+0000 7ffa27b30640 1 -- start start 2024-04-25T09:54:19.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa27b30640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa200b9730 0x7ffa200ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa27b30640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa200ceb90 0x7ffa200d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa27b30640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa200cf040 con 0x7ffa2015da70 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa27b30640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa201625e0 con 0x7ffa200b9730 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa27b30640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20161d50 con 0x7ffa200ceb90 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa200b9730 0x7ffa200ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffa200b9730 0x7ffa200ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51380/0 (socket says 172.21.15.148:51380) 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa258cc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffa2015da70 0x7ffa20161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44296/0 (socket says 172.21.15.148:44296) 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 -- 172.21.15.148:0/1940138868 learned_addr learned my addr 172.21.15.148:0/1940138868 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa260cd640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa200ceb90 0x7ffa200d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 -- 172.21.15.148:0/1940138868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa200ceb90 msgr2=0x7ffa200d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa200ceb90 0x7ffa200d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 -- 172.21.15.148:0/1940138868 >> v1:172.21.15.148:6789/0 conn(0x7ffa2015da70 legacy=0x7ffa20161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa250cb640 1 -- 172.21.15.148:0/1940138868 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20162020 con 0x7ffa200b9730 2024-04-25T09:54:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.559+0000 7ffa260cd640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa200ceb90 0x7ffa200d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.560+0000 7ffa250cb640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200b9730 0x7ffa200ce650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009870 tx=0x7ffa1002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b51caaca083eec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.560+0000 7ffa248ca640 1 -- 172.21.15.148:0/1940138868 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10037070 con 0x7ffa200b9730 2024-04-25T09:54:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.560+0000 7ffa248ca640 1 -- 172.21.15.148:0/1940138868 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa10032440 con 0x7ffa200b9730 2024-04-25T09:54:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.560+0000 7ffa248ca640 1 -- 172.21.15.148:0/1940138868 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10032810 con 0x7ffa200b9730 2024-04-25T09:54:19.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 -- 172.21.15.148:0/1940138868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200b9730 msgr2=0x7ffa200ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200b9730 0x7ffa200ce650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009870 tx=0x7ffa1002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 -- 172.21.15.148:0/1940138868 shutdown_connections 2024-04-25T09:54:19.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa200ceb90 0x7ffa200d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 --2- 172.21.15.148:0/1940138868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200b9730 0x7ffa200ce650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 -- 172.21.15.148:0/1940138868 >> 172.21.15.148:0/1940138868 conn(0x7ffa20061a20 msgr2=0x7ffa20061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 -- 172.21.15.148:0/1940138868 shutdown_connections 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.561+0000 7ffa27b30640 1 -- 172.21.15.148:0/1940138868 wait complete. 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.562+0000 7ffa27b30640 1 Processor -- start 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.562+0000 7ffa27b30640 1 -- start start 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa27b30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 0x7ffa2014eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa27b30640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2015da70 0x7ffa2015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa27b30640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2015e620 0x7ffa20160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa27b30640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200d1dd0 con 0x7ffa200ceb90 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa27b30640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200d1af0 con 0x7ffa2015da70 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa27b30640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2015de50 con 0x7ffa2015e620 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 0x7ffa2014eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 0x7ffa2014eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51394/0 (socket says 172.21.15.148:51394) 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 -- 172.21.15.148:0/4181228824 learned_addr learned my addr 172.21.15.148:0/4181228824 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:19.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa260cd640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2015e620 0x7ffa20160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa250cb640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2015da70 0x7ffa2015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 -- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2015e620 msgr2=0x7ffa20160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2015e620 0x7ffa20160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 -- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2015da70 msgr2=0x7ffa2015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2015da70 0x7ffa2015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 -- 172.21.15.148:0/4181228824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2015eba0 con 0x7ffa200ceb90 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa250cb640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2015da70 0x7ffa2015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa260cd640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2015e620 0x7ffa20160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.563+0000 7ffa258cc640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 0x7ffa2014eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c3c0 tx=0x7ffa1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.564+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0074e0 con 0x7ffa200ceb90 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.564+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c007680 con 0x7ffa200ceb90 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.564+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2015ee80 con 0x7ffa200ceb90 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.564+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c004a00 con 0x7ffa200ceb90 2024-04-25T09:54:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.564+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2015f440 con 0x7ffa200ceb90 2024-04-25T09:54:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.565+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa20073990 con 0x7ffa200ceb90 2024-04-25T09:54:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.565+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa1c01a020 con 0x7ffa200ceb90 2024-04-25T09:54:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.565+0000 7ffa167fc640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa0406b4d0 0x7ffa0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.566+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffa1c089590 con 0x7ffa200ceb90 2024-04-25T09:54:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.566+0000 7ffa250cb640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa0406b4d0 0x7ffa0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.566+0000 7ffa250cb640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa0406b4d0 0x7ffa0406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa10034000 tx=0x7ffa1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.570+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa1c0587f0 con 0x7ffa200ceb90 2024-04-25T09:54:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.774+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa200d1570 con 0x7ffa200ceb90 2024-04-25T09:54:19.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.774+0000 7ffa167fc640 1 -- 172.21.15.148:0/4181228824 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7ffa1c0580f0 con 0x7ffa200ceb90 2024-04-25T09:54:19.777 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:19.777 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T09:54:19.777 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 3 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa0406b4d0 msgr2=0x7ffa0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa0406b4d0 0x7ffa0406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa10034000 tx=0x7ffa1002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 msgr2=0x7ffa2014eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 0x7ffa2014eed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00c3c0 tx=0x7ffa1c00c880 comp rx=0 tx=0).stop 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 shutdown_connections 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa0406b4d0 0x7ffa0406d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2015e620 0x7ffa20160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa2015da70 0x7ffa2015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 --2- 172.21.15.148:0/4181228824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa200ceb90 0x7ffa2014eed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 >> 172.21.15.148:0/4181228824 conn(0x7ffa20061a20 msgr2=0x7ffa2015bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 shutdown_connections 2024-04-25T09:54:19.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.776+0000 7ffa27b30640 1 -- 172.21.15.148:0/4181228824 wait complete. 2024-04-25T09:54:19.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:19.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.883+0000 7f55ec4b3640 1 Processor -- start 2024-04-25T09:54:19.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.883+0000 7f55ec4b3640 1 -- start start 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.884+0000 7f55ec4b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55e415ba00 0x7f55e409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.884+0000 7f55ec4b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f55e41602e0 0x7f55e40ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.884+0000 7f55ec4b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e40cd710 con 0x7f55e415f5f0 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.884+0000 7f55ec4b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e4099b70 con 0x7f55e415ba00 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.884+0000 7f55ec4b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e4099e50 con 0x7f55e41602e0 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.884+0000 7f55ea24f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55e415ba00 0x7f55e409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55e415ba00 0x7f55e409a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51402/0 (socket says 172.21.15.148:51402) 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 -- 172.21.15.148:0/3449784117 learned_addr learned my addr 172.21.15.148:0/3449784117 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55eaa50640 1 --1- 172.21.15.148:0/3449784117 >> v1:172.21.15.148:6789/0 conn(0x7f55e415f5f0 0x7f55e40c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44304/0 (socket says 172.21.15.148:44304) 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55e9a4e640 1 --2- 172.21.15.148:0/3449784117 >> v2:172.21.15.188:3300/0 conn(0x7f55e41602e0 0x7f55e40ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 -- 172.21.15.148:0/3449784117 >> v2:172.21.15.188:3300/0 conn(0x7f55e41602e0 msgr2=0x7f55e40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 --2- 172.21.15.148:0/3449784117 >> v2:172.21.15.188:3300/0 conn(0x7f55e41602e0 0x7f55e40ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 -- 172.21.15.148:0/3449784117 >> v1:172.21.15.148:6789/0 conn(0x7f55e415f5f0 legacy=0x7f55e40c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55e924d640 1 -- 172.21.15.148:0/3449784117 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837429123 0 0) 0x7f55e0001110 con 0x7f55e415f5f0 2024-04-25T09:54:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 -- 172.21.15.148:0/3449784117 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e409a130 con 0x7f55e415ba00 2024-04-25T09:54:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.885+0000 7f55ea24f640 1 --2- 172.21.15.148:0/3449784117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e409a930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55e40b7830 tx=0x7f55dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116ca9f824635d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55e924d640 1 -- 172.21.15.148:0/3449784117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc036070 con 0x7f55e415ba00 2024-04-25T09:54:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55e924d640 1 -- 172.21.15.148:0/3449784117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc031600 con 0x7f55e415ba00 2024-04-25T09:54:19.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55e924d640 1 -- 172.21.15.148:0/3449784117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0319d0 con 0x7f55e415ba00 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3449784117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 msgr2=0x7f55e409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3449784117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e409a930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f55e40b7830 tx=0x7f55dc02a610 comp rx=0 tx=0).stop 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3449784117 shutdown_connections 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3449784117 >> v2:172.21.15.188:3300/0 conn(0x7f55e41602e0 0x7f55e40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3449784117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e409a930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3449784117 >> 172.21.15.148:0/3449784117 conn(0x7f55e40b3270 msgr2=0x7f55e40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3449784117 shutdown_connections 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.886+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3449784117 wait complete. 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.887+0000 7f55ec4b3640 1 Processor -- start 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.887+0000 7f55ec4b3640 1 -- start start 2024-04-25T09:54:19.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ec4b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ec4b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55e415f5f0 0x7f55e40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ec4b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55e41602e0 0x7f55e40b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ec4b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e415bde0 con 0x7f55e415ba00 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ec4b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40cd2e0 con 0x7f55e415f5f0 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ec4b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40cb3f0 con 0x7f55e41602e0 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51412/0 (socket says 172.21.15.148:51412) 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 -- 172.21.15.148:0/3689124710 learned_addr learned my addr 172.21.15.148:0/3689124710 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55e9a4e640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55e415f5f0 0x7f55e40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55eaa50640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55e41602e0 0x7f55e40b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 -- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55e41602e0 msgr2=0x7f55e40b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55e41602e0 0x7f55e40b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 -- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55e415f5f0 msgr2=0x7f55e40caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:19.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.888+0000 7f55ea24f640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55e415f5f0 0x7f55e40caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55ea24f640 1 -- 172.21.15.148:0/3689124710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e40c9e10 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55eaa50640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55e41602e0 0x7f55e40b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55ea24f640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e415a160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000f80 tx=0x7f55dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc041020 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc037040 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc036050 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e40ca0f0 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.889+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e40ca6b0 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.890+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e4073990 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.890+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55dc007c50 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.891+0000 7f55daffd640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55c806b4d0 0x7f55c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.891+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55dc0aa5d0 con 0x7f55e415ba00 2024-04-25T09:54:19.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.891+0000 7f55e9a4e640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55c806b4d0 0x7f55c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:19.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.891+0000 7f55e9a4e640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55c806b4d0 0x7f55c806d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55e41f7260 tx=0x7f55d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:19.895+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55dc04b060 con 0x7f55e415ba00 2024-04-25T09:54:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.066+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55e40b40a0 con 0x7f55e415ba00 2024-04-25T09:54:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.067+0000 7f55daffd640 1 -- 172.21.15.148:0/3689124710 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12932 (secure 0 0 0) 0x7f55dc079830 con 0x7f55e415ba00 2024-04-25T09:54:20.069 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:20.069 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:13.076119+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55c806b4d0 msgr2=0x7f55c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55c806b4d0 0x7f55c806d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55e41f7260 tx=0x7f55d4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 msgr2=0x7f55e415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e415a160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000f80 tx=0x7f55dc004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 shutdown_connections 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55c806b4d0 0x7f55c806d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55e41602e0 0x7f55e40b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55e415f5f0 0x7f55e40caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 --2- 172.21.15.148:0/3689124710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55e415ba00 0x7f55e415a160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 >> 172.21.15.148:0/3689124710 conn(0x7f55e40b3270 msgr2=0x7f55e40cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 shutdown_connections 2024-04-25T09:54:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.069+0000 7f55ec4b3640 1 -- 172.21.15.148:0/3689124710 wait complete. 2024-04-25T09:54:20.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:54:20.082 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:20.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.179+0000 7fd72b6c4640 1 Processor -- start 2024-04-25T09:54:20.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.179+0000 7fd72b6c4640 1 -- start start 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.180+0000 7fd72b6c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 0x7fd7240d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.180+0000 7fd72b6c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72415e000 con 0x7fd72415dc20 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.180+0000 7fd72b6c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240d1de0 con 0x7fd724161710 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.180+0000 7fd72b6c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd724161c60 con 0x7fd7240ced80 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.180+0000 7fd729460640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd72415dc20 0x7fd7240ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44316/0 (socket says 172.21.15.148:44316) 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.180+0000 7fd729460640 1 -- 172.21.15.148:0/1902066688 learned_addr learned my addr 172.21.15.148:0/1902066688 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd728c5f640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 0x7fd7240d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516797218 0 0) 0x7fd72415e000 con 0x7fd72415dc20 2024-04-25T09:54:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710003520 con 0x7fd72415dc20 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126885749 0 0) 0x7fd7240d1de0 con 0x7fd724161710 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd72415e000 con 0x7fd724161710 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267570893 0 0) 0x7fd710003520 con 0x7fd72415dc20 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 msgr2=0x7fd7240d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 0x7fd7240d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 >> v1:172.21.15.188:6789/0 conn(0x7fd724161710 legacy=0x7fd7240d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd724161f40 con 0x7fd72415dc20 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd7200019f0 con 0x7fd72415dc20 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 >> v1:172.21.15.148:6789/0 conn(0x7fd72415dc20 legacy=0x7fd7240ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 0x7fd710003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd710004120 0x7fd710004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd710004d40 0x7fd7100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd724161c60 con 0x7fd710004d40 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bb00 con 0x7fd710004120 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bd00 con 0x7fd7240ced80 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729c61640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd710004d40 0x7fd7100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729460640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 0x7fd710003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd728c5f640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd710004120 0x7fd710004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729c61640 1 -- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 msgr2=0x7fd710003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729c61640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7240ced80 0x7fd710003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729c61640 1 -- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd710004120 msgr2=0x7fd710004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729c61640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd710004120 0x7fd710004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.181+0000 7fd729c61640 1 -- 172.21.15.148:0/1902066688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710003520 con 0x7fd710004d40 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.182+0000 7fd729c61640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd710004d40 0x7fd7100051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0018d0 tx=0x7fd710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5dd2a21fea87d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.182+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0014f0 con 0x7fd710004d40 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.182+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c0052d0 con 0x7fd710004d40 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.182+0000 7fd71bfff640 1 -- 172.21.15.148:0/1902066688 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c02aca0 con 0x7fd710004d40 2024-04-25T09:54:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd72b6c4640 1 -- 172.21.15.148:0/1902066688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd710004d40 msgr2=0x7fd7100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/1902066688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd710004d40 0x7fd7100051c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0018d0 tx=0x7fd710005830 comp rx=0 tx=0).stop 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd729c61640 1 -- 172.21.15.148:0/1902066688 reap_dead start 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd72b6c4640 1 -- 172.21.15.148:0/1902066688 shutdown_connections 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd72b6c4640 1 -- 172.21.15.148:0/1902066688 >> 172.21.15.148:0/1902066688 conn(0x7fd7240d0420 msgr2=0x7fd7240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd72b6c4640 1 -- 172.21.15.148:0/1902066688 shutdown_connections 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.183+0000 7fd72b6c4640 1 -- 172.21.15.148:0/1902066688 wait complete. 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.184+0000 7fd72b6c4640 1 Processor -- start 2024-04-25T09:54:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.184+0000 7fd72b6c4640 1 -- start start 2024-04-25T09:54:20.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd710004d40 0x7fd7240cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 0x7fd724160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd72415c160 0x7fd72414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b5e50 con 0x7fd72415bd00 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd724162500 con 0x7fd72415c160 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b5fd0 con 0x7fd710004d40 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 0x7fd724160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 0x7fd724160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51424/0 (socket says 172.21.15.148:51424) 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 -- 172.21.15.148:0/764211645 learned_addr learned my addr 172.21.15.148:0/764211645 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd729460640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd710004d40 0x7fd7240cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd729c61640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd72415c160 0x7fd72414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 -- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd710004d40 msgr2=0x7fd7240cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd710004d40 0x7fd7240cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 -- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd72415c160 msgr2=0x7fd72414e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd72415c160 0x7fd72414e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 -- 172.21.15.148:0/764211645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240cab80 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd729c61640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd72415c160 0x7fd72414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd728c5f640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 0x7fd724160970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7140119a0 tx=0x7fd714011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400faf0 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400fc90 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7240cae60 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.185+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714014440 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.186+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7240cb420 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.187+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd724073a00 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.187+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd714012040 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.187+0000 7fd719ffb640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd70806b4d0 0x7fd70806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.187+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd71408a620 con 0x7fd72415bd00 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.187+0000 7fd729460640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd70806b4d0 0x7fd70806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.188+0000 7fd729460640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd70806b4d0 0x7fd70806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd720004c20 tx=0x7fd720006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.191+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd714019030 con 0x7fd72415bd00 2024-04-25T09:54:20.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.360+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7240cdcb0 con 0x7fd72415bd00 2024-04-25T09:54:20.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.360+0000 7fd719ffb640 1 -- 172.21.15.148:0/764211645 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd714059760 con 0x7fd72415bd00 2024-04-25T09:54:20.363 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:54:20.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.362+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd70806b4d0 msgr2=0x7fd70806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.362+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd70806b4d0 0x7fd70806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd720004c20 tx=0x7fd720006040 comp rx=0 tx=0).stop 2024-04-25T09:54:20.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.362+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 msgr2=0x7fd724160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.362+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 0x7fd724160970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7140119a0 tx=0x7fd714011e60 comp rx=0 tx=0).stop 2024-04-25T09:54:20.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 shutdown_connections 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd70806b4d0 0x7fd70806d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd72415c160 0x7fd72414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd72415bd00 0x7fd724160970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 --2- 172.21.15.148:0/764211645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd710004d40 0x7fd7240cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 >> 172.21.15.148:0/764211645 conn(0x7fd7240d0420 msgr2=0x7fd7240d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 shutdown_connections 2024-04-25T09:54:20.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.363+0000 7fd72b6c4640 1 -- 172.21.15.148:0/764211645 wait complete. 2024-04-25T09:54:20.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:54:20.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.472+0000 7fb343cee640 1 Processor -- start 2024-04-25T09:54:20.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.472+0000 7fb343cee640 1 -- start start 2024-04-25T09:54:20.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb343cee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 0x7fb33c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb343cee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c0b97a0 con 0x7fb33c15dbe0 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb343cee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c0d1c50 con 0x7fb33c0b2cb0 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb343cee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c15dfc0 con 0x7fb33c0d1020 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb341a8a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb33c15dbe0 0x7fb33c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44320/0 (socket says 172.21.15.148:44320) 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb341a8a640 1 -- 172.21.15.148:0/2700192238 learned_addr learned my addr 172.21.15.148:0/2700192238 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426789656 0 0) 0x7fb33c0b97a0 con 0x7fb33c15dbe0 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb328003520 con 0x7fb33c15dbe0 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb34228b640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 0x7fb33c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036883052 0 0) 0x7fb328003520 con 0x7fb33c15dbe0 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 msgr2=0x7fb33c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 0x7fb33c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 >> v1:172.21.15.188:6789/0 conn(0x7fb33c0b2cb0 legacy=0x7fb33c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0d1f30 con 0x7fb33c15dbe0 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb338001670 con 0x7fb33c15dbe0 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 >> v1:172.21.15.148:6789/0 conn(0x7fb33c15dbe0 legacy=0x7fb33c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280039e0 0x7fb328003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 0x7fb3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32800ba70 con 0x7fb328004800 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32800bc20 con 0x7fb3280039e0 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32800be20 con 0x7fb33c0d1020 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.473+0000 7fb341289640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341a8a640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280039e0 0x7fb328003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb34228b640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 0x7fb3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341289640 1 -- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 msgr2=0x7fb3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341289640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0d1020 0x7fb3280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341289640 1 -- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280039e0 msgr2=0x7fb328003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341289640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3280039e0 0x7fb328003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341289640 1 -- 172.21.15.148:0/2700192238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328003520 con 0x7fb328004800 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb341289640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb328004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0d3930 tx=0x7fb328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbccc87b541b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.474+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c001370 con 0x7fb328004800 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c0052d0 con 0x7fb328004800 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb340a88640 1 -- 172.21.15.148:0/2700192238 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c02ab70 con 0x7fb328004800 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb343cee640 1 -- 172.21.15.148:0/2700192238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 msgr2=0x7fb328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb343cee640 1 --2- 172.21.15.148:0/2700192238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb328004c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0d3930 tx=0x7fb328005830 comp rx=0 tx=0).stop 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb34228b640 1 -- 172.21.15.148:0/2700192238 reap_dead start 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb343cee640 1 -- 172.21.15.148:0/2700192238 shutdown_connections 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb343cee640 1 -- 172.21.15.148:0/2700192238 >> 172.21.15.148:0/2700192238 conn(0x7fb33c061a20 msgr2=0x7fb33c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb343cee640 1 -- 172.21.15.148:0/2700192238 shutdown_connections 2024-04-25T09:54:20.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.475+0000 7fb343cee640 1 -- 172.21.15.148:0/2700192238 wait complete. 2024-04-25T09:54:20.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.476+0000 7fb343cee640 1 Processor -- start 2024-04-25T09:54:20.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.476+0000 7fb343cee640 1 -- start start 2024-04-25T09:54:20.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb343cee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb33c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb343cee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb33c1560a0 0x7fb33c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb343cee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0b42c0 0x7fb33c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb343cee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b92a0 con 0x7fb328004800 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb343cee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0d1a00 con 0x7fb33c1560a0 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb343cee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b9420 con 0x7fb33c0b42c0 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb33c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb33c15a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51442/0 (socket says 172.21.15.148:51442) 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 -- 172.21.15.148:0/3684441724 learned_addr learned my addr 172.21.15.148:0/3684441724 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341289640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb33c1560a0 0x7fb33c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb34228b640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0b42c0 0x7fb33c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 -- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0b42c0 msgr2=0x7fb33c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0b42c0 0x7fb33c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 -- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb33c1560a0 msgr2=0x7fb33c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb33c1560a0 0x7fb33c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 -- 172.21.15.148:0/3684441724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c156560 con 0x7fb328004800 2024-04-25T09:54:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341289640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb33c1560a0 0x7fb33c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.477+0000 7fb341a8a640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb33c15a6b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3380072c0 tx=0x7fb338007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.478+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800f960 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.478+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338018070 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.478+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c156840 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.478+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800fd60 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.478+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c156e00 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.479+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c073990 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.479+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb33801d460 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.479+0000 7fb3327fc640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb32006b500 0x7fb32006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.479+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb338089e10 con 0x7fb328004800 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.479+0000 7fb341289640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb32006b500 0x7fb32006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.480+0000 7fb341289640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb32006b500 0x7fb32006d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb32c032000 tx=0x7fb32c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.483+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb338059070 con 0x7fb328004800 2024-04-25T09:54:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.655+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb33c15b570 con 0x7fb328004800 2024-04-25T09:54:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.656+0000 7fb3327fc640 1 -- 172.21.15.148:0/3684441724 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb338058970 con 0x7fb328004800 2024-04-25T09:54:20.658 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:20.658 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:54:20.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:54:20.659 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T09:54:20.659 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T09:54:20.659 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:54:20.659 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:54:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb32006b500 msgr2=0x7fb32006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb32006b500 0x7fb32006d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb32c032000 tx=0x7fb32c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 msgr2=0x7fb33c15a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb33c15a6b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3380072c0 tx=0x7fb338007780 comp rx=0 tx=0).stop 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 shutdown_connections 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb32006b500 0x7fb32006d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb33c0b42c0 0x7fb33c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb33c1560a0 0x7fb33c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 --2- 172.21.15.148:0/3684441724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb328004800 0x7fb33c15a6b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 >> 172.21.15.148:0/3684441724 conn(0x7fb33c061a20 msgr2=0x7fb33c15ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 shutdown_connections 2024-04-25T09:54:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.658+0000 7fb343cee640 1 -- 172.21.15.148:0/3684441724 wait complete. 2024-04-25T09:54:20.670 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:54:20.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:54:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.769+0000 7feef465d640 1 Processor -- start 2024-04-25T09:54:20.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.769+0000 7feef465d640 1 -- start start 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef465d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feeec161240 0x7feeec0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef465d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec0b54a0 0x7feeec0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef465d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec0cf1d0 con 0x7feeec161890 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef465d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0b7120 con 0x7feeec161240 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef465d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec161620 con 0x7feeec0b54a0 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef23f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feeec161240 0x7feeec0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.770+0000 7feef1bf8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feeec161890 0x7feeec0ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44330/0 (socket says 172.21.15.148:44330) 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef1bf8640 1 -- 172.21.15.148:0/1082643923 learned_addr learned my addr 172.21.15.148:0/1082643923 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef2bfa640 1 --2- 172.21.15.148:0/1082643923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec0b54a0 0x7feeec0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef23f9640 1 -- 172.21.15.148:0/1082643923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec0b54a0 msgr2=0x7feeec0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:20.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef23f9640 1 --2- 172.21.15.148:0/1082643923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec0b54a0 0x7feeec0b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef23f9640 1 -- 172.21.15.148:0/1082643923 >> v1:172.21.15.148:6789/0 conn(0x7feeec161890 legacy=0x7feeec0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef23f9640 1 -- 172.21.15.148:0/1082643923 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec161e00 con 0x7feeec161240 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef13f7640 1 -- 172.21.15.148:0/1082643923 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650435007 0 0) 0x7feeec0cf1d0 con 0x7feeec161890 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.771+0000 7feef23f9640 1 --2- 172.21.15.148:0/1082643923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161240 0x7feeec0b6be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c75bf953b13a308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef13f7640 1 -- 172.21.15.148:0/1082643923 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8037070 con 0x7feeec161240 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef13f7640 1 -- 172.21.15.148:0/1082643923 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8032440 con 0x7feeec161240 2024-04-25T09:54:20.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef13f7640 1 -- 172.21.15.148:0/1082643923 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8032810 con 0x7feeec161240 2024-04-25T09:54:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 -- 172.21.15.148:0/1082643923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161240 msgr2=0x7feeec0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 --2- 172.21.15.148:0/1082643923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161240 0x7feeec0b6be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802a180 comp rx=0 tx=0).stop 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 -- 172.21.15.148:0/1082643923 shutdown_connections 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 --2- 172.21.15.148:0/1082643923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec0b54a0 0x7feeec0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 --2- 172.21.15.148:0/1082643923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161240 0x7feeec0b6be0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 -- 172.21.15.148:0/1082643923 >> 172.21.15.148:0/1082643923 conn(0x7feeec0d04a0 msgr2=0x7feeec0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 -- 172.21.15.148:0/1082643923 shutdown_connections 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.772+0000 7feef465d640 1 -- 172.21.15.148:0/1082643923 wait complete. 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.773+0000 7feef465d640 1 Processor -- start 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.773+0000 7feef465d640 1 -- start start 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef465d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec0b54a0 0x7feeec151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef465d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec161240 0x7feeec1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef465d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 0x7feeec0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef465d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1626a0 con 0x7feeec161890 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef465d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec1623c0 con 0x7feeec0b54a0 2024-04-25T09:54:20.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef465d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0d15f0 con 0x7feeec161240 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 0x7feeec0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 0x7feeec0cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51448/0 (socket says 172.21.15.148:51448) 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 -- 172.21.15.148:0/940684937 learned_addr learned my addr 172.21.15.148:0/940684937 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef1bf8640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec161240 0x7feeec1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 -- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec161240 msgr2=0x7feeec1f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec161240 0x7feeec1f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef23f9640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec0b54a0 0x7feeec151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 -- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec0b54a0 msgr2=0x7feeec151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec0b54a0 0x7feeec151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.774+0000 7feef2bfa640 1 -- 172.21.15.148:0/940684937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec152450 con 0x7feeec161890 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.775+0000 7feef2bfa640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 0x7feeec0cc3e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feee400cda0 tx=0x7feee400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.775+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4003b90 con 0x7feeec161890 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.775+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4003d30 con 0x7feeec161890 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.775+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec1f76c0 con 0x7feeec161890 2024-04-25T09:54:20.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.775+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4004240 con 0x7feeec161890 2024-04-25T09:54:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.775+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec0c8c60 con 0x7feeec161890 2024-04-25T09:54:20.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.776+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec0c8a80 con 0x7feeec161890 2024-04-25T09:54:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.776+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feee40043e0 con 0x7feeec161890 2024-04-25T09:54:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.777+0000 7feee2ffd640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.777+0000 7feef23f9640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.777+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feee408a940 con 0x7feeec161890 2024-04-25T09:54:20.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.777+0000 7feef23f9640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feee8034000 tx=0x7feee802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:20.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.781+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feeec0c8a80 con 0x7feeec161890 2024-04-25T09:54:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:20.951+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feeec073a00 con 0x7feeec161890 2024-04-25T09:54:21.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.938+0000 7feee2ffd640 1 -- 172.21.15.148:0/940684937 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7feeec152450 con 0x7feeec161890 2024-04-25T09:54:21.941 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T09:54:21.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.941+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 msgr2=0x7feed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.941+0000 7feef465d640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feee8034000 tx=0x7feee802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:21.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.941+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 msgr2=0x7feeec0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:21.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.941+0000 7feef465d640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 0x7feeec0cc3e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feee400cda0 tx=0x7feee400a660 comp rx=0 tx=0).stop 2024-04-25T09:54:21.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 shutdown_connections 2024-04-25T09:54:21.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec161890 0x7feeec0cc3e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec161240 0x7feeec1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 --2- 172.21.15.148:0/940684937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec0b54a0 0x7feeec151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:21.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 >> 172.21.15.148:0/940684937 conn(0x7feeec0d04a0 msgr2=0x7feeec0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:21.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 shutdown_connections 2024-04-25T09:54:21.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:21.942+0000 7feef465d640 1 -- 172.21.15.148:0/940684937 wait complete. 2024-04-25T09:54:21.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T09:54:22.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 Processor -- start 2024-04-25T09:54:22.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 -- start start 2024-04-25T09:54:22.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff0406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff041616f0 0x7eff04161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7eff040ce0a0 0x7eff040ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff041614d0 con 0x7eff040ce0a0 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040c6960 con 0x7eff041616f0 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.068+0000 7eff0a23f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff04162260 con 0x7eff041610f0 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7eff040ce0a0 0x7eff040ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7eff040ce0a0 0x7eff040ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51456/0 (socket says 172.21.15.148:51456) 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 -- 172.21.15.148:0/925608016 learned_addr learned my addr 172.21.15.148:0/925608016 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff037fe640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff0406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7efefbfff640 1 --2- 172.21.15.148:0/925608016 >> v2:172.21.15.188:3300/0 conn(0x7eff041616f0 0x7eff04161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 -- 172.21.15.148:0/925608016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 msgr2=0x7eff0406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff0406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 -- 172.21.15.148:0/925608016 >> v2:172.21.15.188:3300/0 conn(0x7eff041616f0 msgr2=0x7eff04161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 --2- 172.21.15.148:0/925608016 >> v2:172.21.15.188:3300/0 conn(0x7eff041616f0 0x7eff04161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 -- 172.21.15.148:0/925608016 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04162540 con 0x7eff040ce0a0 2024-04-25T09:54:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7efefbfff640 1 --2- 172.21.15.148:0/925608016 >> v2:172.21.15.188:3300/0 conn(0x7eff041616f0 0x7eff04161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff037fe640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff0406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.069+0000 7eff03fff640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff040ce0a0 0x7eff040ce530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7eff04072020 tx=0x7efefc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181edac7c2dff035 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff02ffd640 1 -- 172.21.15.148:0/925608016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc036070 con 0x7eff040ce0a0 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff02ffd640 1 -- 172.21.15.148:0/925608016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc031600 con 0x7eff040ce0a0 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff02ffd640 1 -- 172.21.15.148:0/925608016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc0319d0 con 0x7eff040ce0a0 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 -- 172.21.15.148:0/925608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff040ce0a0 msgr2=0x7eff040ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff040ce0a0 0x7eff040ce530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7eff04072020 tx=0x7efefc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 -- 172.21.15.148:0/925608016 shutdown_connections 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff040ce0a0 0x7eff040ce530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 --2- 172.21.15.148:0/925608016 >> v2:172.21.15.188:3300/0 conn(0x7eff041616f0 0x7eff04161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 --2- 172.21.15.148:0/925608016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff0406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 -- 172.21.15.148:0/925608016 >> 172.21.15.148:0/925608016 conn(0x7eff0406f500 msgr2=0x7eff0406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:22.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 -- 172.21.15.148:0/925608016 shutdown_connections 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.070+0000 7eff0a23f640 1 -- 172.21.15.148:0/925608016 wait complete. 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.071+0000 7eff0a23f640 1 Processor -- start 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 -- start start 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff040ce0a0 0x7eff040c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 0x7eff041f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff040cf690 con 0x7eff041616f0 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff040cfce0 con 0x7eff040ce0a0 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff0a23f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0406c700 con 0x7eff041610f0 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 0x7eff041f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 0x7eff041f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51462/0 (socket says 172.21.15.148:51462) 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff03fff640 1 -- 172.21.15.148:0/3435828352 learned_addr learned my addr 172.21.15.148:0/3435828352 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7efefbfff640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.072+0000 7eff037fe640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff040ce0a0 0x7eff040c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff03fff640 1 -- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 msgr2=0x7eff040b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff03fff640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff040b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff03fff640 1 -- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff040ce0a0 msgr2=0x7eff040c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff03fff640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff040ce0a0 0x7eff040c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff03fff640 1 -- 172.21.15.148:0/3435828352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff041f4210 con 0x7eff041616f0 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff03fff640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 0x7eff041f3700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efefc009f40 tx=0x7efefc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc036050 con 0x7eff041616f0 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc042070 con 0x7eff041616f0 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff041f44f0 con 0x7eff041616f0 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc009510 con 0x7eff041616f0 2024-04-25T09:54:22.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.073+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff040c76e0 con 0x7eff041616f0 2024-04-25T09:54:22.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.074+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff040b4520 con 0x7eff041616f0 2024-04-25T09:54:22.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.075+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efefc007bd0 con 0x7eff041616f0 2024-04-25T09:54:22.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.075+0000 7eff00ff9640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efee406b500 0x7efee406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:22.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.075+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7efefc0aadb0 con 0x7eff041616f0 2024-04-25T09:54:22.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.075+0000 7eff037fe640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efee406b500 0x7efee406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:22.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.075+0000 7eff037fe640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efee406b500 0x7efee406d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff041f12f0 tx=0x7efef4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:22.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.079+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efefc079850 con 0x7eff041616f0 2024-04-25T09:54:22.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.249+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7eff04073a00 con 0x7eff041616f0 2024-04-25T09:54:22.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.940+0000 7eff00ff9640 1 -- 172.21.15.148:0/3435828352 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7efefc07d8f0 con 0x7eff041616f0 2024-04-25T09:54:22.944 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T09:54:22.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efee406b500 msgr2=0x7efee406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efee406b500 0x7efee406d9c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff041f12f0 tx=0x7efef4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 msgr2=0x7eff041f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 0x7eff041f3700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efefc009f40 tx=0x7efefc004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 shutdown_connections 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efee406b500 0x7efee406d9c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff041616f0 0x7eff041f3700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff041610f0 0x7eff040b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 --2- 172.21.15.148:0/3435828352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff040ce0a0 0x7eff040c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 >> 172.21.15.148:0/3435828352 conn(0x7eff0406f500 msgr2=0x7eff040d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 shutdown_connections 2024-04-25T09:54:22.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:22.944+0000 7eff0a23f640 1 -- 172.21.15.148:0/3435828352 wait complete. 2024-04-25T09:54:22.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:54:23.077 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:23.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.075+0000 7fd222a5d640 1 Processor -- start 2024-04-25T09:54:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.075+0000 7fd222a5d640 1 -- start start 2024-04-25T09:54:23.080 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd222a5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd21c161710 0x7fd21c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd222a5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c15e000 con 0x7fd21c0ced80 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd222a5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c0d1de0 con 0x7fd21c15dc20 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd222a5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21c161c60 con 0x7fd21c161710 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd21b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd21c0ced80 0x7fd21c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44340/0 (socket says 172.21.15.148:44340) 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd21b7fe640 1 -- 172.21.15.148:0/3490533211 learned_addr learned my addr 172.21.15.148:0/3490533211 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd220ffa640 1 --2- 172.21.15.148:0/3490533211 >> v2:172.21.15.188:3300/0 conn(0x7fd21c161710 0x7fd21c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd21affd640 1 -- 172.21.15.148:0/3490533211 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150755583 0 0) 0x7fd21c15e000 con 0x7fd21c0ced80 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.076+0000 7fd21affd640 1 -- 172.21.15.148:0/3490533211 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd204003520 con 0x7fd21c0ced80 2024-04-25T09:54:23.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.077+0000 7fd21affd640 1 -- 172.21.15.148:0/3490533211 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448600612 0 0) 0x7fd204003520 con 0x7fd21c0ced80 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.077+0000 7fd220ffa640 1 -- 172.21.15.148:0/3490533211 >> v1:172.21.15.188:6789/0 conn(0x7fd21c15dc20 legacy=0x7fd21c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.077+0000 7fd220ffa640 1 -- 172.21.15.148:0/3490533211 >> v1:172.21.15.148:6789/0 conn(0x7fd21c0ced80 legacy=0x7fd21c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.077+0000 7fd220ffa640 1 -- 172.21.15.148:0/3490533211 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c161f40 con 0x7fd21c161710 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.077+0000 7fd220ffa640 1 --2- 172.21.15.148:0/3490533211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c161710 0x7fd21c0d29f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd214009d90 tx=0x7fd21402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db146b25c6bedc32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd21affd640 1 -- 172.21.15.148:0/3490533211 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214035020 con 0x7fd21c161710 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd21affd640 1 -- 172.21.15.148:0/3490533211 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214007540 con 0x7fd21c161710 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd21affd640 1 -- 172.21.15.148:0/3490533211 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210001670 con 0x7fd21c161710 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 -- 172.21.15.148:0/3490533211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c161710 msgr2=0x7fd21c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3490533211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c161710 0x7fd21c0d29f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd214009d90 tx=0x7fd21402a5b0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 -- 172.21.15.148:0/3490533211 shutdown_connections 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3490533211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c161710 0x7fd21c0d29f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 -- 172.21.15.148:0/3490533211 >> 172.21.15.148:0/3490533211 conn(0x7fd21c0d0420 msgr2=0x7fd21c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 -- 172.21.15.148:0/3490533211 shutdown_connections 2024-04-25T09:54:23.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.078+0000 7fd222a5d640 1 -- 172.21.15.148:0/3490533211 wait complete. 2024-04-25T09:54:23.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.079+0000 7fd222a5d640 1 Processor -- start 2024-04-25T09:54:23.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 -- start start 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c0ced80 0x7fd21c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c15dc20 0x7fd21c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 0x7fd21c1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0b5e50 con 0x7fd21c161710 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c162500 con 0x7fd21c15dc20 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.080+0000 7fd222a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0b5fd0 con 0x7fd21c0ced80 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 0x7fd21c1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 0x7fd21c1551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51476/0 (socket says 172.21.15.148:51476) 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 -- 172.21.15.148:0/3413948567 learned_addr learned my addr 172.21.15.148:0/3413948567 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd21b7fe640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c15dc20 0x7fd21c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd21bfff640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c0ced80 0x7fd21c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 -- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c0ced80 msgr2=0x7fd21c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c0ced80 0x7fd21c0c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 -- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c15dc20 msgr2=0x7fd21c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c15dc20 0x7fd21c160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 -- 172.21.15.148:0/3413948567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c0c89f0 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd220ffa640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 0x7fd21c1551e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd214009eb0 tx=0x7fd214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214009450 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214031c40 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214007e50 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c0c7950 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.081+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c0c7f10 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.083+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c073a00 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.083+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd21402ad10 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.083+0000 7fd218ff9640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.083+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd2140aaf90 con 0x7fd21c161710 2024-04-25T09:54:23.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.083+0000 7fd21bfff640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:23.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.084+0000 7fd21bfff640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd20c001900 tx=0x7fd20c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:23.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.088+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd214041020 con 0x7fd21c161710 2024-04-25T09:54:23.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.297+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd21c0d16b0 con 0x7fd21c161710 2024-04-25T09:54:23.969 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T09:54:23.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.965+0000 7fd218ff9640 1 -- 172.21.15.148:0/3413948567 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd214079940 con 0x7fd21c161710 2024-04-25T09:54:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.970+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 msgr2=0x7fd20806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd20c001900 tx=0x7fd20c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 msgr2=0x7fd21c1551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 0x7fd21c1551e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd214009eb0 tx=0x7fd214004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 shutdown_connections 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd20c001900 tx=0x7fd20c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c161710 0x7fd21c1551e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c15dc20 0x7fd21c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 --2- 172.21.15.148:0/3413948567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c0ced80 0x7fd21c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:23.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.971+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 >> 172.21.15.148:0/3413948567 conn(0x7fd21c0d0420 msgr2=0x7fd21c0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.972+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 shutdown_connections 2024-04-25T09:54:23.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:23.973+0000 7fd222a5d640 1 -- 172.21.15.148:0/3413948567 wait complete. 2024-04-25T09:54:23.993 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T09:54:23.993 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T09:54:24.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 Processor -- start 2024-04-25T09:54:24.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 -- start start 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439006abe0 0x7f439006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43900ad570 0x7f439006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439006b500 con 0x7f439015dd80 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439006e410 con 0x7f43900ad570 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.093+0000 7f4395d70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43900b3930 con 0x7f439006abe0 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43900ad570 0x7f439006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439006abe0 0x7f439006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438effd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f439015dd80 0x7f43900b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:49372/0 (socket says 172.21.15.148:49372) 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438effd640 1 -- 172.21.15.148:0/1554391177 learned_addr learned my addr 172.21.15.148:0/1554391177 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438ffff640 1 -- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439006abe0 msgr2=0x7f439006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:24.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438ffff640 1 --2- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439006abe0 0x7f439006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438ffff640 1 -- 172.21.15.148:0/1554391177 >> v1:172.21.15.188:6789/0 conn(0x7f439015dd80 legacy=0x7f43900b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438ffff640 1 -- 172.21.15.148:0/1554391177 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43900b3c10 con 0x7f43900ad570 2024-04-25T09:54:24.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.094+0000 7f438e7fc640 1 -- 172.21.15.148:0/1554391177 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320944599 0 0) 0x7f439006b500 con 0x7f439015dd80 2024-04-25T09:54:24.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.095+0000 7f438ffff640 1 --2- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43900ad570 0x7f439006ee60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4388009d90 tx=0x7f438802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e9fcf7a6186f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:24.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.095+0000 7f438e7fc640 1 -- 172.21.15.148:0/1554391177 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388007740 con 0x7f43900ad570 2024-04-25T09:54:24.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.095+0000 7f438e7fc640 1 -- 172.21.15.148:0/1554391177 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43880078e0 con 0x7f43900ad570 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.095+0000 7f438e7fc640 1 -- 172.21.15.148:0/1554391177 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388007cb0 con 0x7f43900ad570 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 -- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43900ad570 msgr2=0x7f439006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 --2- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43900ad570 0x7f439006ee60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4388009d90 tx=0x7f438802a660 comp rx=0 tx=0).stop 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 -- 172.21.15.148:0/1554391177 shutdown_connections 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 --2- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43900ad570 0x7f439006ee60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 --2- 172.21.15.148:0/1554391177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439006abe0 0x7f439006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 -- 172.21.15.148:0/1554391177 >> 172.21.15.148:0/1554391177 conn(0x7f43900b5af0 msgr2=0x7f43900b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 -- 172.21.15.148:0/1554391177 shutdown_connections 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.096+0000 7f4395d70640 1 -- 172.21.15.148:0/1554391177 wait complete. 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.097+0000 7f4395d70640 1 Processor -- start 2024-04-25T09:54:24.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.097+0000 7f4395d70640 1 -- start start 2024-04-25T09:54:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439006abe0 0x7f4390141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43900ad570 0x7f43901f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 0x7f43901f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4390070280 con 0x7f439015dd80 2024-04-25T09:54:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439006e940 con 0x7f439006abe0 2024-04-25T09:54:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4390070400 con 0x7f43900ad570 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 0x7f43901f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 0x7f43901f01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44574/0 (socket says 172.21.15.148:44574) 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 -- 172.21.15.148:0/648896687 learned_addr learned my addr 172.21.15.148:0/648896687 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438f7fe640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439006abe0 0x7f4390141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438effd640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43900ad570 0x7f43901f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 -- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43900ad570 msgr2=0x7f43901f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43900ad570 0x7f43901f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 -- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439006abe0 msgr2=0x7f4390141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439006abe0 0x7f4390141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 -- 172.21.15.148:0/648896687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4390141e40 con 0x7f439015dd80 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f438ffff640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 0x7f43901f01a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4388009d90 tx=0x7f4388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388040070 con 0x7f439015dd80 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4388031c40 con 0x7f439015dd80 2024-04-25T09:54:24.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43901541f0 con 0x7f439015dd80 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.098+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388009450 con 0x7f439015dd80 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.099+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43901547b0 con 0x7f439015dd80 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.099+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4390073a00 con 0x7f439015dd80 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.100+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4388039040 con 0x7f439015dd80 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.100+0000 7f4377fff640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f437c06b4d0 0x7f437c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.100+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43880ab3d0 con 0x7f439015dd80 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.100+0000 7f438f7fe640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f437c06b4d0 0x7f437c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.101+0000 7f438f7fe640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f437c06b4d0 0x7f437c06d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4380009ad0 tx=0x7f4380008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.105+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f438807a2e0 con 0x7f439015dd80 2024-04-25T09:54:24.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.283+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f43901f7600 con 0x7f439015dd80 2024-04-25T09:54:24.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.971+0000 7f4377fff640 1 -- 172.21.15.148:0/648896687 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f4388079be0 con 0x7f439015dd80 2024-04-25T09:54:24.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f437c06b4d0 msgr2=0x7f437c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f437c06b4d0 0x7f437c06d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4380009ad0 tx=0x7f4380008040 comp rx=0 tx=0).stop 2024-04-25T09:54:24.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 msgr2=0x7f43901f01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:24.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 0x7f43901f01a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4388009d90 tx=0x7f4388004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 shutdown_connections 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f437c06b4d0 0x7f437c06d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f439015dd80 0x7f43901f01a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43900ad570 0x7f43901f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 --2- 172.21.15.148:0/648896687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439006abe0 0x7f4390141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 >> 172.21.15.148:0/648896687 conn(0x7f43900b5af0 msgr2=0x7f439006ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 shutdown_connections 2024-04-25T09:54:24.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:24.974+0000 7f4395d70640 1 -- 172.21.15.148:0/648896687 wait complete. 2024-04-25T09:54:24.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:54:25.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.097+0000 7f37d963a640 1 Processor -- start 2024-04-25T09:54:25.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.097+0000 7f37d963a640 1 -- start start 2024-04-25T09:54:25.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d963a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37d4161710 0x7f37d40d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d963a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d415e000 con 0x7f37d40ced80 2024-04-25T09:54:25.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d963a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d40d1de0 con 0x7f37d415dc20 2024-04-25T09:54:25.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d963a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d4161c60 con 0x7f37d4161710 2024-04-25T09:54:25.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37d4161710 0x7f37d40d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37cbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f37d40ced80 0x7f37d40d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51120/0 (socket says 172.21.15.148:51120) 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37d4161710 0x7f37d40d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44584/0 (socket says 172.21.15.148:44584) 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37cbfff640 1 -- 172.21.15.148:0/3315986939 learned_addr learned my addr 172.21.15.148:0/3315986939 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d37fe640 1 -- 172.21.15.148:0/3315986939 >> v1:172.21.15.188:6789/0 conn(0x7f37d415dc20 legacy=0x7f37d40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d37fe640 1 -- 172.21.15.148:0/3315986939 >> v1:172.21.15.148:6789/0 conn(0x7f37d40ced80 legacy=0x7f37d40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d37fe640 1 -- 172.21.15.148:0/3315986939 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d4161f40 con 0x7f37d4161710 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.098+0000 7f37d27fc640 1 -- 172.21.15.148:0/3315986939 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292717729 0 0) 0x7f37d415e000 con 0x7f37d40ced80 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d37fe640 1 --2- 172.21.15.148:0/3315986939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d4161710 0x7f37d40d29f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009d90 tx=0x7f37cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e270b48720f8fe6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d27fc640 1 -- 172.21.15.148:0/3315986939 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc007740 con 0x7f37d4161710 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d27fc640 1 -- 172.21.15.148:0/3315986939 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc0078e0 con 0x7f37d4161710 2024-04-25T09:54:25.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d27fc640 1 -- 172.21.15.148:0/3315986939 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc007cb0 con 0x7f37d4161710 2024-04-25T09:54:25.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d963a640 1 -- 172.21.15.148:0/3315986939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d4161710 msgr2=0x7f37d40d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d963a640 1 --2- 172.21.15.148:0/3315986939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d4161710 0x7f37d40d29f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009d90 tx=0x7f37cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d963a640 1 -- 172.21.15.148:0/3315986939 shutdown_connections 2024-04-25T09:54:25.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d963a640 1 --2- 172.21.15.148:0/3315986939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d4161710 0x7f37d40d29f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.099+0000 7f37d963a640 1 -- 172.21.15.148:0/3315986939 >> 172.21.15.148:0/3315986939 conn(0x7f37d40d0420 msgr2=0x7f37d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:25.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.100+0000 7f37d963a640 1 -- 172.21.15.148:0/3315986939 shutdown_connections 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.100+0000 7f37d963a640 1 -- 172.21.15.148:0/3315986939 wait complete. 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 Processor -- start 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 -- start start 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37d40ced80 0x7f37d40caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 0x7f37d415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37d4161710 0x7f37d415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40b5e50 con 0x7f37d415dc20 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4162500 con 0x7f37d4161710 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37d963a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40b5fd0 con 0x7f37d40ced80 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 0x7f37d415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 0x7f37d415a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44600/0 (socket says 172.21.15.148:44600) 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.101+0000 7f37cbfff640 1 -- 172.21.15.148:0/366792423 learned_addr learned my addr 172.21.15.148:0/366792423 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37d2ffd640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37d40ced80 0x7f37d40caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37d37fe640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37d4161710 0x7f37d415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cbfff640 1 -- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37d40ced80 msgr2=0x7f37d40caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cbfff640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37d40ced80 0x7f37d40caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cbfff640 1 -- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37d4161710 msgr2=0x7f37d415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cbfff640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37d4161710 0x7f37d415aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cbfff640 1 -- 172.21.15.148:0/366792423 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40cb610 con 0x7f37d415dc20 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37d37fe640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37d4161710 0x7f37d415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37d2ffd640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37d40ced80 0x7f37d40caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cbfff640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 0x7f37d415a5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c460 tx=0x7f37bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc018070 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc007500 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d40cb8f0 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc004a20 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.102+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d40cd440 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.103+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4073a00 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.103+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37bc01b070 con 0x7f37d415dc20 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.104+0000 7f37cb7fe640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37b406b4d0 0x7f37b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.104+0000 7f37d2ffd640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37b406b4d0 0x7f37b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:25.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.104+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f37bc089570 con 0x7f37d415dc20 2024-04-25T09:54:25.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.104+0000 7f37d2ffd640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37b406b4d0 0x7f37b406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f37c4003eb0 tx=0x7f37c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:25.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.108+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37bc058350 con 0x7f37d415dc20 2024-04-25T09:54:25.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:25.278+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f37d40d16b0 con 0x7f37d415dc20 2024-04-25T09:54:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.983+0000 7f37cb7fe640 1 -- 172.21.15.148:0/366792423 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f37bc057c50 con 0x7f37d415dc20 2024-04-25T09:54:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data_ec' created 2024-04-25T09:54:26.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.988+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37b406b4d0 msgr2=0x7f37b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.988+0000 7f37d963a640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37b406b4d0 0x7f37b406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f37c4003eb0 tx=0x7f37c4009040 comp rx=0 tx=0).stop 2024-04-25T09:54:26.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.988+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 msgr2=0x7f37d415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:26.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.988+0000 7f37d963a640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 0x7f37d415a5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00c460 tx=0x7f37bc00c920 comp rx=0 tx=0).stop 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.989+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 shutdown_connections 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.989+0000 7f37d963a640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37b406b4d0 0x7f37b406d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.989+0000 7f37d963a640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37d4161710 0x7f37d415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.989+0000 7f37d963a640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37d415dc20 0x7f37d415a5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.989+0000 7f37d963a640 1 --2- 172.21.15.148:0/366792423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37d40ced80 0x7f37d40caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.989+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 >> 172.21.15.148:0/366792423 conn(0x7f37d40d0420 msgr2=0x7f37d415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.990+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 shutdown_connections 2024-04-25T09:54:26.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:26.990+0000 7f37d963a640 1 -- 172.21.15.148:0/366792423 wait complete. 2024-04-25T09:54:27.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T09:54:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.114+0000 7f658799e640 1 Processor -- start 2024-04-25T09:54:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.114+0000 7f658799e640 1 -- start start 2024-04-25T09:54:27.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f658799e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65800ced80 0x7f65800d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f658799e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6580161710 0x7f65800d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f658799e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658015e000 con 0x7f658015dc20 2024-04-25T09:54:27.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f658799e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65800d1de0 con 0x7f6580161710 2024-04-25T09:54:27.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f658799e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6580161c60 con 0x7f65800ced80 2024-04-25T09:54:27.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f6585f3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6580161710 0x7f65800d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f6585f3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6580161710 0x7f65800d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44602/0 (socket says 172.21.15.148:44602) 2024-04-25T09:54:27.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.115+0000 7f6585f3b640 1 -- 172.21.15.148:0/3497055208 learned_addr learned my addr 172.21.15.148:0/3497055208 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:27.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6584f39640 1 --2- 172.21.15.148:0/3497055208 >> v2:172.21.15.188:3300/0 conn(0x7f65800ced80 0x7f65800d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6585f3b640 1 -- 172.21.15.148:0/3497055208 >> v2:172.21.15.188:3300/0 conn(0x7f65800ced80 msgr2=0x7f65800d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6585f3b640 1 --2- 172.21.15.148:0/3497055208 >> v2:172.21.15.188:3300/0 conn(0x7f65800ced80 0x7f65800d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6585f3b640 1 -- 172.21.15.148:0/3497055208 >> v1:172.21.15.188:6789/0 conn(0x7f658015dc20 legacy=0x7f65800ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:27.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6585f3b640 1 -- 172.21.15.148:0/3497055208 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580161f40 con 0x7f6580161710 2024-04-25T09:54:27.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6585f3b640 1 --2- 172.21.15.148:0/3497055208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6580161710 0x7f65800d2820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f658006da90 tx=0x7f657802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fef665c7cc8779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6577fff640 1 -- 172.21.15.148:0/3497055208 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578036070 con 0x7f6580161710 2024-04-25T09:54:27.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6577fff640 1 -- 172.21.15.148:0/3497055208 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6578031600 con 0x7f6580161710 2024-04-25T09:54:27.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.116+0000 7f6577fff640 1 -- 172.21.15.148:0/3497055208 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65780319d0 con 0x7f6580161710 2024-04-25T09:54:27.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 -- 172.21.15.148:0/3497055208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6580161710 msgr2=0x7f65800d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 --2- 172.21.15.148:0/3497055208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6580161710 0x7f65800d2820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f658006da90 tx=0x7f657802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 -- 172.21.15.148:0/3497055208 shutdown_connections 2024-04-25T09:54:27.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 --2- 172.21.15.148:0/3497055208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6580161710 0x7f65800d2820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 --2- 172.21.15.148:0/3497055208 >> v2:172.21.15.188:3300/0 conn(0x7f65800ced80 0x7f65800d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 -- 172.21.15.148:0/3497055208 >> 172.21.15.148:0/3497055208 conn(0x7f65800d0420 msgr2=0x7f65800d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:27.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 -- 172.21.15.148:0/3497055208 shutdown_connections 2024-04-25T09:54:27.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.117+0000 7f658799e640 1 -- 172.21.15.148:0/3497055208 wait complete. 2024-04-25T09:54:27.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.118+0000 7f658799e640 1 Processor -- start 2024-04-25T09:54:27.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.118+0000 7f658799e640 1 -- start start 2024-04-25T09:54:27.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658799e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65800ced80 0x7f658015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658799e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 0x7f658015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658799e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6580161710 0x7f65801607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658799e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800b5e50 con 0x7f658015dc20 2024-04-25T09:54:27.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658799e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6580162500 con 0x7f6580161710 2024-04-25T09:54:27.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658799e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65800b5fd0 con 0x7f65800ced80 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 0x7f658015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 0x7f658015bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44612/0 (socket says 172.21.15.148:44612) 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 -- 172.21.15.148:0/1376595014 learned_addr learned my addr 172.21.15.148:0/1376595014 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f658573a640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65800ced80 0x7f658015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6585f3b640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6580161710 0x7f65801607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 -- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65800ced80 msgr2=0x7f658015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65800ced80 0x7f658015add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 -- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6580161710 msgr2=0x7f65801607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6580161710 0x7f65801607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 -- 172.21.15.148:0/1376595014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658015c350 con 0x7f658015dc20 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6585f3b640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6580161710 0x7f65801607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.119+0000 7f6584f39640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 0x7f658015bcc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f657000d5f0 tx=0x7f657000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.120+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700125f0 con 0x7f658015dc20 2024-04-25T09:54:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.120+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570019070 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.120+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658015a2f0 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.120+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700129d0 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.120+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580160d20 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.121+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6580073a00 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.121+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6570012b70 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.122+0000 7f6575ffb640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f656406b4d0 0x7f656406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.122+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f657008a850 con 0x7f658015dc20 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.122+0000 7f658573a640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f656406b4d0 0x7f656406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.122+0000 7f658573a640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f656406b4d0 0x7f656406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f657c006fd0 tx=0x7f657c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.126+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6570059440 con 0x7f658015dc20 2024-04-25T09:54:27.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.295+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f65800d16b0 con 0x7f658015dc20 2024-04-25T09:54:27.602 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-25T09:54:27.600+0000 7f6ac0491640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:54:27.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.988+0000 7f6575ffb640 1 -- 172.21.15.148:0/1376595014 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6570058d40 con 0x7f658015dc20 2024-04-25T09:54:27.991 INFO:teuthology.orchestra.run.smithi148.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.991+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f656406b4d0 msgr2=0x7f656406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.991+0000 7f658799e640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f656406b4d0 0x7f656406d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f657c006fd0 tx=0x7f657c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.991+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 msgr2=0x7f658015bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.991+0000 7f658799e640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 0x7f658015bcc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f657000d5f0 tx=0x7f657000dab0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 shutdown_connections 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f656406b4d0 0x7f656406d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6580161710 0x7f65801607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f658015dc20 0x7f658015bcc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 --2- 172.21.15.148:0/1376595014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65800ced80 0x7f658015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 >> 172.21.15.148:0/1376595014 conn(0x7f65800d0420 msgr2=0x7f65800d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:28.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.993+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 shutdown_connections 2024-04-25T09:54:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:27.999+0000 7f658799e640 1 -- 172.21.15.148:0/1376595014 wait complete. 2024-04-25T09:54:28.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T09:54:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.126+0000 7f005b6fc640 1 Processor -- start 2024-04-25T09:54:28.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.126+0000 7f005b6fc640 1 -- start start 2024-04-25T09:54:28.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.126+0000 7f005b6fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00541446d0 0x7f00540b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f005b6fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00540cccc0 con 0x7f005406e8a0 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f005b6fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00540d0460 con 0x7f0054144d50 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f005b6fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00540b2b20 con 0x7f00541446d0 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f0059c99640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f005406e8a0 0x7f00540cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51126/0 (socket says 172.21.15.148:51126) 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f0059c99640 1 -- 172.21.15.148:0/1739997793 learned_addr learned my addr 172.21.15.148:0/1739997793 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f0059498640 1 --2- 172.21.15.148:0/1739997793 >> v2:172.21.15.188:3300/0 conn(0x7f00541446d0 0x7f00540b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f004bfff640 1 -- 172.21.15.148:0/1739997793 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113347061 0 0) 0x7f00540cccc0 con 0x7f005406e8a0 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f004bfff640 1 -- 172.21.15.148:0/1739997793 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0040003520 con 0x7f005406e8a0 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f004bfff640 1 -- 172.21.15.148:0/1739997793 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740852906 0 0) 0x7f0040003520 con 0x7f005406e8a0 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f0059498640 1 -- 172.21.15.148:0/1739997793 >> v1:172.21.15.188:6789/0 conn(0x7f0054144d50 legacy=0x7f00540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f0059498640 1 -- 172.21.15.148:0/1739997793 >> v1:172.21.15.148:6789/0 conn(0x7f005406e8a0 legacy=0x7f00540cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.127+0000 7f0059498640 1 -- 172.21.15.148:0/1739997793 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540cc140 con 0x7f00541446d0 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.128+0000 7f0059498640 1 --2- 172.21.15.148:0/1739997793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00541446d0 0x7f00540b3f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f004c009d90 tx=0x7f004c02a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b0b77eb4fae9a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.128+0000 7f004bfff640 1 -- 172.21.15.148:0/1739997793 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c035020 con 0x7f00541446d0 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.128+0000 7f004bfff640 1 -- 172.21.15.148:0/1739997793 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c007540 con 0x7f00541446d0 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.128+0000 7f004bfff640 1 -- 172.21.15.148:0/1739997793 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050001670 con 0x7f00541446d0 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 -- 172.21.15.148:0/1739997793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00541446d0 msgr2=0x7f00540b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 --2- 172.21.15.148:0/1739997793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00541446d0 0x7f00540b3f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f004c009d90 tx=0x7f004c02a5b0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 -- 172.21.15.148:0/1739997793 shutdown_connections 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 --2- 172.21.15.148:0/1739997793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00541446d0 0x7f00540b3f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 -- 172.21.15.148:0/1739997793 >> 172.21.15.148:0/1739997793 conn(0x7f005406abe0 msgr2=0x7f005406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 -- 172.21.15.148:0/1739997793 shutdown_connections 2024-04-25T09:54:28.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.129+0000 7f005b6fc640 1 -- 172.21.15.148:0/1739997793 wait complete. 2024-04-25T09:54:28.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.130+0000 7f005b6fc640 1 Processor -- start 2024-04-25T09:54:28.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.130+0000 7f005b6fc640 1 -- start start 2024-04-25T09:54:28.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005406e8a0 0x7f00540ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:28.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 0x7f00540a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0054144d50 0x7f00540aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540cc700 con 0x7f00541446d0 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005406d400 con 0x7f005406e8a0 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00540cc9e0 con 0x7f0054144d50 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 0x7f00540a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 0x7f00540a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44622/0 (socket says 172.21.15.148:44622) 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 -- 172.21.15.148:0/949957675 learned_addr learned my addr 172.21.15.148:0/949957675 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0059498640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005406e8a0 0x7f00540ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0059c99640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0054144d50 0x7f00540aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 -- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0054144d50 msgr2=0x7f00540aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0054144d50 0x7f00540aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 -- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005406e8a0 msgr2=0x7f00540ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005406e8a0 0x7f00540ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 -- 172.21.15.148:0/949957675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540a9890 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0058c97640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 0x7f00540a8eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f004400c570 tx=0x7f004400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044005480 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00440035f0 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00540a9b70 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.131+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00440039c0 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.132+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005413bb80 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.133+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005413bd80 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.133+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0044003b60 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.133+0000 7f0049ffb640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f003806b670 0x7f003806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.133+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f004408a080 con 0x7f00541446d0 2024-04-25T09:54:28.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.133+0000 7f0059498640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f003806b670 0x7f003806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:28.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.134+0000 7f0059498640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f003806b670 0x7f003806db30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f004c033000 tx=0x7f004c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:28.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.137+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f005413bd80 con 0x7f00541446d0 2024-04-25T09:54:28.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.346+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f0054073a00 con 0x7f00541446d0 2024-04-25T09:54:28.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.990+0000 7f0049ffb640 1 -- 172.21.15.148:0/949957675 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0044058620 con 0x7f00541446d0 2024-04-25T09:54:28.993 INFO:teuthology.orchestra.run.smithi148.stderr:added data pool 4 to fsmap 2024-04-25T09:54:28.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f003806b670 msgr2=0x7f003806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f003806b670 0x7f003806db30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f004c033000 tx=0x7f004c02f000 comp rx=0 tx=0).stop 2024-04-25T09:54:28.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 msgr2=0x7f00540a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 0x7f00540a8eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f004400c570 tx=0x7f004400ca30 comp rx=0 tx=0).stop 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 shutdown_connections 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f003806b670 0x7f003806db30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0054144d50 0x7f00540aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00541446d0 0x7f00540a8eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 --2- 172.21.15.148:0/949957675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005406e8a0 0x7f00540ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 >> 172.21.15.148:0/949957675 conn(0x7f005406abe0 msgr2=0x7f0054143550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 shutdown_connections 2024-04-25T09:54:28.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:28.994+0000 7f005b6fc640 1 -- 172.21.15.148:0/949957675 wait complete. 2024-04-25T09:54:29.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:29.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.109+0000 7f6d989b0640 1 Processor -- start 2024-04-25T09:54:29.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.109+0000 7f6d989b0640 1 -- start start 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d989b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d940af720 0x7f6d940afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d989b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9406fe00 con 0x7f6d940d02f0 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d989b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d940700e0 con 0x7f6d94158220 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d989b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d940b3670 con 0x7f6d940af720 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d91d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d940af720 0x7f6d940afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d91d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6d940af720 0x7f6d940afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44626/0 (socket says 172.21.15.148:44626) 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d91d9b640 1 -- 172.21.15.148:0/898222500 learned_addr learned my addr 172.21.15.148:0/898222500 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.110+0000 7f6d92d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6d940d02f0 0x7f6d940d0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51130/0 (socket says 172.21.15.148:51130) 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d9159a640 1 -- 172.21.15.148:0/898222500 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837180962 0 0) 0x7f6d9406fe00 con 0x7f6d940d02f0 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d9159a640 1 -- 172.21.15.148:0/898222500 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d7c003520 con 0x7f6d940d02f0 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d91d9b640 1 -- 172.21.15.148:0/898222500 >> v1:172.21.15.188:6789/0 conn(0x7f6d94158220 legacy=0x7f6d9415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d91d9b640 1 -- 172.21.15.148:0/898222500 >> v1:172.21.15.148:6789/0 conn(0x7f6d940d02f0 legacy=0x7f6d940d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d91d9b640 1 -- 172.21.15.148:0/898222500 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940703c0 con 0x7f6d940af720 2024-04-25T09:54:29.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d91d9b640 1 --2- 172.21.15.148:0/898222500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d940af720 0x7f6d940afba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009870 tx=0x7f6d8802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c555148f1420793d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.111+0000 7f6d9159a640 1 -- 172.21.15.148:0/898222500 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88037070 con 0x7f6d940af720 2024-04-25T09:54:29.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d9159a640 1 -- 172.21.15.148:0/898222500 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8802ae50 con 0x7f6d940af720 2024-04-25T09:54:29.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d9159a640 1 -- 172.21.15.148:0/898222500 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88032860 con 0x7f6d940af720 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 -- 172.21.15.148:0/898222500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d940af720 msgr2=0x7f6d940afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 --2- 172.21.15.148:0/898222500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d940af720 0x7f6d940afba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009870 tx=0x7f6d8802a300 comp rx=0 tx=0).stop 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 -- 172.21.15.148:0/898222500 shutdown_connections 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 --2- 172.21.15.148:0/898222500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d940af720 0x7f6d940afba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 -- 172.21.15.148:0/898222500 >> 172.21.15.148:0/898222500 conn(0x7f6d940b2370 msgr2=0x7f6d940b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 -- 172.21.15.148:0/898222500 shutdown_connections 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.112+0000 7f6d989b0640 1 -- 172.21.15.148:0/898222500 wait complete. 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.113+0000 7f6d989b0640 1 Processor -- start 2024-04-25T09:54:29.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.113+0000 7f6d989b0640 1 -- start start 2024-04-25T09:54:29.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d989b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d940af720 0x7f6d9414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d989b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d940d02f0 0x7f6d940cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d989b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 0x7f6d940cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d989b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94158600 con 0x7f6d94158220 2024-04-25T09:54:29.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d989b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94070a10 con 0x7f6d940af720 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d989b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94072960 con 0x7f6d940d02f0 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d92d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 0x7f6d940cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d92d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 0x7f6d940cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44632/0 (socket says 172.21.15.148:44632) 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d92d9d640 1 -- 172.21.15.148:0/3953340657 learned_addr learned my addr 172.21.15.148:0/3953340657 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d91d9b640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d940d02f0 0x7f6d940cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d9259c640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d940af720 0x7f6d9414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.114+0000 7f6d92d9d640 1 -- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d940d02f0 msgr2=0x7f6d940cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d92d9d640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d940d02f0 0x7f6d940cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d92d9d640 1 -- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d940af720 msgr2=0x7f6d9414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d92d9d640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d940af720 0x7f6d9414b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d92d9d640 1 -- 172.21.15.148:0/3953340657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940cdf30 con 0x7f6d94158220 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d92d9d640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 0x7f6d940cfc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c460 tx=0x7f6d8c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c018070 con 0x7f6d94158220 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c007480 con 0x7f6d94158220 2024-04-25T09:54:29.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d940ce210 con 0x7f6d94158220 2024-04-25T09:54:29.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0047e0 con 0x7f6d94158220 2024-04-25T09:54:29.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.115+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d940ce7d0 con 0x7f6d94158220 2024-04-25T09:54:29.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.116+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d94073990 con 0x7f6d94158220 2024-04-25T09:54:29.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.116+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d8c01b070 con 0x7f6d94158220 2024-04-25T09:54:29.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.117+0000 7f6d82ffd640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d7406b4d0 0x7f6d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.117+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6d8c089730 con 0x7f6d94158220 2024-04-25T09:54:29.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.117+0000 7f6d9259c640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d7406b4d0 0x7f6d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.117+0000 7f6d9259c640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d7406b4d0 0x7f6d7406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d84001d40 tx=0x7f6d84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.121+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d8c058300 con 0x7f6d94158220 2024-04-25T09:54:29.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.328+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d941560c0 con 0x7f6d94158220 2024-04-25T09:54:29.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.328+0000 7f6d82ffd640 1 -- 172.21.15.148:0/3953340657 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f6d8c057c00 con 0x7f6d94158220 2024-04-25T09:54:29.331 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:29.331 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:28.348024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T09:54:29.331 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 7 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d7406b4d0 msgr2=0x7f6d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d7406b4d0 0x7f6d7406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d84001d40 tx=0x7f6d84009040 comp rx=0 tx=0).stop 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 msgr2=0x7f6d940cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 0x7f6d940cfc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c460 tx=0x7f6d8c00c920 comp rx=0 tx=0).stop 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 shutdown_connections 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d7406b4d0 0x7f6d7406d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d94158220 0x7f6d940cfc70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d940d02f0 0x7f6d940cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 --2- 172.21.15.148:0/3953340657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d940af720 0x7f6d9414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 >> 172.21.15.148:0/3953340657 conn(0x7f6d940b2370 msgr2=0x7f6d9415d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 shutdown_connections 2024-04-25T09:54:29.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.330+0000 7f6d989b0640 1 -- 172.21.15.148:0/3953340657 wait complete. 2024-04-25T09:54:29.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.439+0000 7f03e4af5640 1 Processor -- start 2024-04-25T09:54:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.439+0000 7f03e4af5640 1 -- start start 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.439+0000 7f03e4af5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 0x7f03e0162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.439+0000 7f03e4af5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e0162690 con 0x7f03e00b6cb0 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.439+0000 7f03e4af5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e00ceee0 con 0x7f03e00b5c30 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.439+0000 7f03e4af5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00d1d10 con 0x7f03e015db70 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03ddd9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f03e00b6cb0 0x7f03e00b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51132/0 (socket says 172.21.15.148:51132) 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03ddd9b640 1 -- 172.21.15.148:0/2550525239 learned_addr learned my addr 172.21.15.148:0/2550525239 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03de59c640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 0x7f03e0162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214568293 0 0) 0x7f03e0162690 con 0x7f03e00b6cb0 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c8003520 con 0x7f03e00b6cb0 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3566494105 0 0) 0x7f03c8003520 con 0x7f03e00b6cb0 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 msgr2=0x7f03e0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 0x7f03e0162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 >> v1:172.21.15.188:6789/0 conn(0x7f03e00b5c30 legacy=0x7f03e00d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00d1ff0 con 0x7f03e00b6cb0 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f03d4001670 con 0x7f03e00b6cb0 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 >> v1:172.21.15.148:6789/0 conn(0x7f03e00b6cb0 legacy=0x7f03e00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 0x7f03c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03c8003f20 0x7f03c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03c8004d40 0x7f03c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00d1d10 con 0x7f03c8003f20 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c800bad0 con 0x7f03c8004d40 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c800bcd0 con 0x7f03e015db70 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03ddd9b640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03c8003f20 0x7f03c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03de59c640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 0x7f03c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.440+0000 7f03ded9d640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03c8004d40 0x7f03c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ddd9b640 1 -- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 msgr2=0x7f03c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ddd9b640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03e015db70 0x7f03c80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ddd9b640 1 -- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03c8004d40 msgr2=0x7f03c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ddd9b640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03c8004d40 0x7f03c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ddd9b640 1 -- 172.21.15.148:0/2550525239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8003520 con 0x7f03c8003f20 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ded9d640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03c8004d40 0x7f03c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03ddd9b640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03c8003f20 0x7f03c8004800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03d400c210 tx=0x7f03c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2f8426e05fedb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4007d10 con 0x7f03c8003f20 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d4005ad0 con 0x7f03c8003f20 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.441+0000 7f03dd59a640 1 -- 172.21.15.148:0/2550525239 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4007070 con 0x7f03c8003f20 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03e4af5640 1 -- 172.21.15.148:0/2550525239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03c8003f20 msgr2=0x7f03c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03e4af5640 1 --2- 172.21.15.148:0/2550525239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03c8003f20 0x7f03c8004800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03d400c210 tx=0x7f03c8005830 comp rx=0 tx=0).stop 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03ded9d640 1 -- 172.21.15.148:0/2550525239 reap_dead start 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03e4af5640 1 -- 172.21.15.148:0/2550525239 shutdown_connections 2024-04-25T09:54:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03e4af5640 1 -- 172.21.15.148:0/2550525239 >> 172.21.15.148:0/2550525239 conn(0x7f03e00d0520 msgr2=0x7f03e00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03e4af5640 1 -- 172.21.15.148:0/2550525239 shutdown_connections 2024-04-25T09:54:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.442+0000 7f03e4af5640 1 -- 172.21.15.148:0/2550525239 wait complete. 2024-04-25T09:54:29.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.443+0000 7f03e4af5640 1 Processor -- start 2024-04-25T09:54:29.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.443+0000 7f03e4af5640 1 -- start start 2024-04-25T09:54:29.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03e4af5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03c8003f20 0x7f03e00c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03e4af5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e014f190 0x7f03e0155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03e4af5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 0x7f03e0160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03e4af5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00ce530 con 0x7f03e0160860 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03e4af5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e015df50 con 0x7f03e014f190 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03e4af5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e00cf1c0 con 0x7f03c8003f20 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 0x7f03e0160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 0x7f03e0160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44656/0 (socket says 172.21.15.148:44656) 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 -- 172.21.15.148:0/3681523912 learned_addr learned my addr 172.21.15.148:0/3681523912 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03de59c640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03c8003f20 0x7f03e00c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ddd9b640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e014f190 0x7f03e0155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 -- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03c8003f20 msgr2=0x7f03e00c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03c8003f20 0x7f03e00c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 -- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e014f190 msgr2=0x7f03e0155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e014f190 0x7f03e0155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.444+0000 7f03ded9d640 1 -- 172.21.15.148:0/3681523912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00c9b20 con 0x7f03e0160860 2024-04-25T09:54:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.445+0000 7f03ded9d640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 0x7f03e0160ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03e00d1540 tx=0x7f03d800ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.445+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8019070 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.445+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d8006b50 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.445+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e015f7c0 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.445+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8003970 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.445+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e015fd80 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.446+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e0073990 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.446+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03d801c070 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.447+0000 7f03ceffd640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03c006b4d0 0x7f03c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.447+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f03d8089890 con 0x7f03e0160860 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.447+0000 7f03de59c640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03c006b4d0 0x7f03c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.447+0000 7f03de59c640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03c006b4d0 0x7f03c006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009b00 tx=0x7f03d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.451+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03e00c9b20 con 0x7f03e0160860 2024-04-25T09:54:29.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.622+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03e00d1820 con 0x7f03e0160860 2024-04-25T09:54:29.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.623+0000 7f03ceffd640 1 -- 172.21.15.148:0/3681523912 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18575 (secure 0 0 0) 0x7f03d80583d0 con 0x7f03e0160860 2024-04-25T09:54:29.625 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:29.626 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:28.347823+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:29.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03c006b4d0 msgr2=0x7f03c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03c006b4d0 0x7f03c006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009b00 tx=0x7f03d0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:29.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 msgr2=0x7f03e0160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 0x7f03e0160ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03e00d1540 tx=0x7f03d800ebe0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 shutdown_connections 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03c006b4d0 0x7f03c006d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03e0160860 0x7f03e0160ce0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03e014f190 0x7f03e0155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 --2- 172.21.15.148:0/3681523912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03c8003f20 0x7f03e00c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 >> 172.21.15.148:0/3681523912 conn(0x7f03e00d0520 msgr2=0x7f03e00c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 shutdown_connections 2024-04-25T09:54:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.625+0000 7f03e4af5640 1 -- 172.21.15.148:0/3681523912 wait complete. 2024-04-25T09:54:29.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:29.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.733+0000 7f8d317d5640 1 Processor -- start 2024-04-25T09:54:29.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.733+0000 7f8d317d5640 1 -- start start 2024-04-25T09:54:29.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d317d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 0x7f8d2c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d317d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0b97a0 con 0x7f8d2c15dbe0 2024-04-25T09:54:29.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d317d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0d1c50 con 0x7f8d2c0d1020 2024-04-25T09:54:29.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d317d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15dfc0 con 0x7f8d2c0b2cb0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d2affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8d2c15dbe0 0x7f8d2c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51136/0 (socket says 172.21.15.148:51136) 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d2affd640 1 -- 172.21.15.148:0/2486725434 learned_addr learned my addr 172.21.15.148:0/2486725434 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512154077 0 0) 0x7f8d2c0b97a0 con 0x7f8d2c15dbe0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d14003520 con 0x7f8d2c15dbe0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d2a7fc640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 0x7f8d2c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.734+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2662411797 0 0) 0x7f8d14003520 con 0x7f8d2c15dbe0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 msgr2=0x7f8d2c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 0x7f8d2c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 >> v1:172.21.15.188:6789/0 conn(0x7f8d2c0d1020 legacy=0x7f8d2c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0d1f30 con 0x7f8d2c15dbe0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f8d1c001670 con 0x7f8d2c15dbe0 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 >> v1:172.21.15.148:6789/0 conn(0x7f8d2c15dbe0 legacy=0x7f8d2c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 0x7f8d140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d14003f60 0x7f8d14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d14004d40 0x7f8d140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1400ba70 con 0x7f8d14004d40 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1400bc20 con 0x7f8d14003f60 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1400be20 con 0x7f8d2c0b2cb0 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2b7fe640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d14004d40 0x7f8d140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2affd640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 0x7f8d140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2a7fc640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d14003f60 0x7f8d14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2b7fe640 1 -- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 msgr2=0x7f8d140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2b7fe640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c0b2cb0 0x7f8d140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2b7fe640 1 -- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d14003f60 msgr2=0x7f8d14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2b7fe640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d14003f60 0x7f8d14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.735+0000 7f8d2b7fe640 1 -- 172.21.15.148:0/2486725434 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d14003520 con 0x7f8d14004d40 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d2b7fe640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d14004d40 0x7f8d140051c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d24001e20 tx=0x7f8d14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0493f0bb098345 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24001a70 con 0x7f8d14004d40 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2402ae40 con 0x7f8d14004d40 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d29ffb640 1 -- 172.21.15.148:0/2486725434 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24003bb0 con 0x7f8d14004d40 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d317d5640 1 -- 172.21.15.148:0/2486725434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d14004d40 msgr2=0x7f8d140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2486725434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d14004d40 0x7f8d140051c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8d24001e20 tx=0x7f8d14005830 comp rx=0 tx=0).stop 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d2b7fe640 1 -- 172.21.15.148:0/2486725434 reap_dead start 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d317d5640 1 -- 172.21.15.148:0/2486725434 shutdown_connections 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d317d5640 1 -- 172.21.15.148:0/2486725434 >> 172.21.15.148:0/2486725434 conn(0x7f8d2c061a20 msgr2=0x7f8d2c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d317d5640 1 -- 172.21.15.148:0/2486725434 shutdown_connections 2024-04-25T09:54:29.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.736+0000 7f8d317d5640 1 -- 172.21.15.148:0/2486725434 wait complete. 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 Processor -- start 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 -- start start 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 0x7f8d2c1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c15aa70 0x7f8d2c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c157240 0x7f8d2c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b92a0 con 0x7f8d2c154850 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0d1a00 con 0x7f8d2c15aa70 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.738+0000 7f8d317d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b9420 con 0x7f8d2c157240 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 0x7f8d2c1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 0x7f8d2c1564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44676/0 (socket says 172.21.15.148:44676) 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 -- 172.21.15.148:0/2651115606 learned_addr learned my addr 172.21.15.148:0/2651115606 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:29.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2a7fc640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c15aa70 0x7f8d2c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2b7fe640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c157240 0x7f8d2c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 -- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c157240 msgr2=0x7f8d2c1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c157240 0x7f8d2c1576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 -- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c15aa70 msgr2=0x7f8d2c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c15aa70 0x7f8d2c15aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 -- 172.21.15.148:0/2651115606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c154c30 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d2affd640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 0x7f8d2c1564a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007290 tx=0x7f8d1c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00f960 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c018070 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c15b510 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00fd60 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.739+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c1559c0 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.740+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c073990 con 0x7f8d2c154850 2024-04-25T09:54:29.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.741+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d1c01d460 con 0x7f8d2c154850 2024-04-25T09:54:29.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.741+0000 7f8d137fe640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d1806b500 0x7f8d1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:29.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.741+0000 7f8d2a7fc640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d1806b500 0x7f8d1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:29.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.741+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8d1c08a650 con 0x7f8d2c154850 2024-04-25T09:54:29.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.741+0000 7f8d2a7fc640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d1806b500 0x7f8d1806d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c15a830 tx=0x7f8d20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:29.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.745+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d1c059190 con 0x7f8d2c154850 2024-04-25T09:54:29.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.913+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d2c0b2790 con 0x7f8d2c154850 2024-04-25T09:54:29.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.915+0000 7f8d137fe640 1 -- 172.21.15.148:0/2651115606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18575 (secure 0 0 0) 0x7f8d1c058a90 con 0x7f8d2c154850 2024-04-25T09:54:29.917 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:29.917 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:28.347823+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d1806b500 msgr2=0x7f8d1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d1806b500 0x7f8d1806d9c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c15a830 tx=0x7f8d20008040 comp rx=0 tx=0).stop 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 msgr2=0x7f8d2c1564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 0x7f8d2c1564a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007290 tx=0x7f8d1c007750 comp rx=0 tx=0).stop 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 shutdown_connections 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d1806b500 0x7f8d1806d9c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d2c157240 0x7f8d2c1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d2c15aa70 0x7f8d2c15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.916+0000 7f8d317d5640 1 --2- 172.21.15.148:0/2651115606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d2c154850 0x7f8d2c1564a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:29.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.917+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 >> 172.21.15.148:0/2651115606 conn(0x7f8d2c061a20 msgr2=0x7f8d2c0b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:29.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.917+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 shutdown_connections 2024-04-25T09:54:29.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:29.917+0000 7f8d317d5640 1 -- 172.21.15.148:0/2651115606 wait complete. 2024-04-25T09:54:29.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.044+0000 7f7925e07640 1 Processor -- start 2024-04-25T09:54:30.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.044+0000 7f7925e07640 1 -- start start 2024-04-25T09:54:30.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7925e07640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f792015dbe0 0x7f79200d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7925e07640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79200b2cb0 0x7f79200d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7925e07640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200b97a0 con 0x7f79200b92a0 2024-04-25T09:54:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7925e07640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792015dfc0 con 0x7f79200b2cb0 2024-04-25T09:54:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7925e07640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79200d1a00 con 0x7f792015dbe0 2024-04-25T09:54:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7917fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79200b2cb0 0x7f79200d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7917fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79200b2cb0 0x7f79200d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44692/0 (socket says 172.21.15.148:44692) 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f7917fff640 1 -- 172.21.15.148:0/2899562376 learned_addr learned my addr 172.21.15.148:0/2899562376 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.045+0000 7f791f7fe640 1 --2- 172.21.15.148:0/2899562376 >> v2:172.21.15.188:3300/0 conn(0x7f792015dbe0 0x7f79200d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f7917fff640 1 -- 172.21.15.148:0/2899562376 >> v2:172.21.15.188:3300/0 conn(0x7f792015dbe0 msgr2=0x7f79200d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f7917fff640 1 --2- 172.21.15.148:0/2899562376 >> v2:172.21.15.188:3300/0 conn(0x7f792015dbe0 0x7f79200d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f7917fff640 1 -- 172.21.15.148:0/2899562376 >> v1:172.21.15.188:6789/0 conn(0x7f79200b92a0 legacy=0x7f79200b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f7917fff640 1 -- 172.21.15.148:0/2899562376 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200d1c50 con 0x7f79200b2cb0 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f791f7fe640 1 --2- 172.21.15.148:0/2899562376 >> v2:172.21.15.188:3300/0 conn(0x7f792015dbe0 0x7f79200d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f7917fff640 1 --2- 172.21.15.148:0/2899562376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79200b2cb0 0x7f79200d0e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f792006c140 tx=0x7f790802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8687ef71f2d4e27e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f791effd640 1 -- 172.21.15.148:0/2899562376 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908037070 con 0x7f79200b2cb0 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f791effd640 1 -- 172.21.15.148:0/2899562376 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908032440 con 0x7f79200b2cb0 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.046+0000 7f791effd640 1 -- 172.21.15.148:0/2899562376 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908032810 con 0x7f79200b2cb0 2024-04-25T09:54:30.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 -- 172.21.15.148:0/2899562376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79200b2cb0 msgr2=0x7f79200d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 --2- 172.21.15.148:0/2899562376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79200b2cb0 0x7f79200d0e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f792006c140 tx=0x7f790802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 -- 172.21.15.148:0/2899562376 shutdown_connections 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 --2- 172.21.15.148:0/2899562376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79200b2cb0 0x7f79200d0e50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 --2- 172.21.15.148:0/2899562376 >> v2:172.21.15.188:3300/0 conn(0x7f792015dbe0 0x7f79200d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 -- 172.21.15.148:0/2899562376 >> 172.21.15.148:0/2899562376 conn(0x7f7920061a20 msgr2=0x7f7920061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 -- 172.21.15.148:0/2899562376 shutdown_connections 2024-04-25T09:54:30.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.047+0000 7f7925e07640 1 -- 172.21.15.148:0/2899562376 wait complete. 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.048+0000 7f7925e07640 1 Processor -- start 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.048+0000 7f7925e07640 1 -- start start 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7925e07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79200b2cb0 0x7f792015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7925e07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79200b92a0 0x7f79200c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7925e07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 0x7f79200c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7925e07640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200b6ab0 con 0x7f792015dbe0 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7925e07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200d2210 con 0x7f79200b92a0 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7925e07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79200b6c30 con 0x7f79200b2cb0 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 0x7f79200c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 0x7f79200c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44694/0 (socket says 172.21.15.148:44694) 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 -- 172.21.15.148:0/359668650 learned_addr learned my addr 172.21.15.148:0/359668650 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791f7fe640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79200b2cb0 0x7f792015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f7917fff640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79200b92a0 0x7f79200c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 -- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79200b2cb0 msgr2=0x7f792015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79200b2cb0 0x7f792015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 -- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79200b92a0 msgr2=0x7f79200c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79200b92a0 0x7f79200c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 -- 172.21.15.148:0/359668650 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79201f7b80 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.049+0000 7f791ffff640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 0x7f79200c9810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f791800c730 tx=0x7f791800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.050+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918003b70 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.050+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918003d10 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.050+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79200caca0 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.050+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800aa90 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.050+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792015b1d0 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.050+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7920073990 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.051+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f791800ac30 con 0x7f792015dbe0 2024-04-25T09:54:30.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.051+0000 7f791cff9640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f790006b4d0 0x7f790006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.051+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f791808a250 con 0x7f792015dbe0 2024-04-25T09:54:30.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.051+0000 7f791f7fe640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f790006b4d0 0x7f790006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.052+0000 7f791f7fe640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f790006b4d0 0x7f790006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7910006e80 tx=0x7f7910008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.056+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79180587d0 con 0x7f792015dbe0 2024-04-25T09:54:30.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.265+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f792015d120 con 0x7f792015dbe0 2024-04-25T09:54:30.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.265+0000 7f791cff9640 1 -- 172.21.15.148:0/359668650 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f791805c870 con 0x7f792015dbe0 2024-04-25T09:54:30.269 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:30.269 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:28.348024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T09:54:30.271 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 7 2024-04-25T09:54:30.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f790006b4d0 msgr2=0x7f790006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f790006b4d0 0x7f790006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7910006e80 tx=0x7f7910008040 comp rx=0 tx=0).stop 2024-04-25T09:54:30.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 msgr2=0x7f79200c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 0x7f79200c9810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f791800c730 tx=0x7f791800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 shutdown_connections 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f790006b4d0 0x7f790006d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f792015dbe0 0x7f79200c9810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79200b92a0 0x7f79200c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 --2- 172.21.15.148:0/359668650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79200b2cb0 0x7f792015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 >> 172.21.15.148:0/359668650 conn(0x7f7920061a20 msgr2=0x7f792006c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 shutdown_connections 2024-04-25T09:54:30.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.267+0000 7f7925e07640 1 -- 172.21.15.148:0/359668650 wait complete. 2024-04-25T09:54:30.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.374+0000 7feace742640 1 Processor -- start 2024-04-25T09:54:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.374+0000 7feace742640 1 -- start start 2024-04-25T09:54:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feace742640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feac815dbe0 0x7feac80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feace742640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b2cb0 0x7feac80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feace742640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac80b97a0 con 0x7feac80b92a0 2024-04-25T09:54:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feace742640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac815dfc0 con 0x7feac815dbe0 2024-04-25T09:54:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feace742640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80d1a00 con 0x7feac80b2cb0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feac7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feac815dbe0 0x7feac80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feac7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feac815dbe0 0x7feac80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44700/0 (socket says 172.21.15.148:44700) 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.375+0000 7feac7fff640 1 -- 172.21.15.148:0/1422161157 learned_addr learned my addr 172.21.15.148:0/1422161157 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac77fe640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b2cb0 0x7feac80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac7fff640 1 -- 172.21.15.148:0/1422161157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b2cb0 msgr2=0x7feac80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac7fff640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b2cb0 0x7feac80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac7fff640 1 -- 172.21.15.148:0/1422161157 >> v1:172.21.15.188:6789/0 conn(0x7feac80b92a0 legacy=0x7feac80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac7fff640 1 -- 172.21.15.148:0/1422161157 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80d1c50 con 0x7feac815dbe0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac77fe640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b2cb0 0x7feac80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac7fff640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d0910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feab8009870 tx=0x7feab802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943fc600fcc422e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac6ffd640 1 -- 172.21.15.148:0/1422161157 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8037070 con 0x7feac815dbe0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac6ffd640 1 -- 172.21.15.148:0/1422161157 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8032440 con 0x7feac815dbe0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.376+0000 7feac6ffd640 1 -- 172.21.15.148:0/1422161157 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8032810 con 0x7feac815dbe0 2024-04-25T09:54:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 -- 172.21.15.148:0/1422161157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 msgr2=0x7feac80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d0910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feab8009870 tx=0x7feab802a210 comp rx=0 tx=0).stop 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 -- 172.21.15.148:0/1422161157 shutdown_connections 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b2cb0 0x7feac80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 --2- 172.21.15.148:0/1422161157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d0910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 -- 172.21.15.148:0/1422161157 >> 172.21.15.148:0/1422161157 conn(0x7feac8061a20 msgr2=0x7feac8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 -- 172.21.15.148:0/1422161157 shutdown_connections 2024-04-25T09:54:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.377+0000 7feace742640 1 -- 172.21.15.148:0/1422161157 wait complete. 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.378+0000 7feace742640 1 Processor -- start 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.378+0000 7feace742640 1 -- start start 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feace742640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac80b2cb0 0x7feac80c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feace742640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b92a0 0x7feac81f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feace742640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feace742640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80b6ab0 con 0x7feac815dbe0 2024-04-25T09:54:30.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feace742640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80d2210 con 0x7feac80b2cb0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feace742640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80b6c30 con 0x7feac80b92a0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44704/0 (socket says 172.21.15.148:44704) 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 -- 172.21.15.148:0/1534622535 learned_addr learned my addr 172.21.15.148:0/1534622535 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feac77fe640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b92a0 0x7feac81f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feac7fff640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac80b2cb0 0x7feac80c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 -- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b92a0 msgr2=0x7feac81f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b92a0 0x7feac81f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 -- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac80b2cb0 msgr2=0x7feac80c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac80b2cb0 0x7feac80c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.379+0000 7feacccdf640 1 -- 172.21.15.148:0/1534622535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac80cc5a0 con 0x7feac815dbe0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.380+0000 7feacccdf640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d3960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feac000c6d0 tx=0x7feac000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.380+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0016020 con 0x7feac815dbe0 2024-04-25T09:54:30.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.380+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac000a840 con 0x7feac815dbe0 2024-04-25T09:54:30.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.380+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac81f47a0 con 0x7feac815dbe0 2024-04-25T09:54:30.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.380+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac000ac70 con 0x7feac815dbe0 2024-04-25T09:54:30.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.380+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac80b57d0 con 0x7feac815dbe0 2024-04-25T09:54:30.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.381+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac8073990 con 0x7feac815dbe0 2024-04-25T09:54:30.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.381+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feac0019020 con 0x7feac815dbe0 2024-04-25T09:54:30.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.382+0000 7feac4ff9640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab406b4d0 0x7feab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:30.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.382+0000 7feac7fff640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab406b4d0 0x7feab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:30.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.382+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feac008a890 con 0x7feac815dbe0 2024-04-25T09:54:30.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.382+0000 7feac7fff640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab406b4d0 0x7feab406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feab8034000 tx=0x7feab802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:30.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.386+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feac0020020 con 0x7feac815dbe0 2024-04-25T09:54:30.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.556+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac80b7990 con 0x7feac815dbe0 2024-04-25T09:54:30.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.557+0000 7feac4ff9640 1 -- 172.21.15.148:0/1534622535 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7feac80cc5a0 con 0x7feac815dbe0 2024-04-25T09:54:30.560 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:30.560 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab406b4d0 msgr2=0x7feab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab406b4d0 0x7feab406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feab8034000 tx=0x7feab802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 msgr2=0x7feac80d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d3960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feac000c6d0 tx=0x7feac000cb90 comp rx=0 tx=0).stop 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 shutdown_connections 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab406b4d0 0x7feab406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac815dbe0 0x7feac80d3960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac80b92a0 0x7feac81f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 --2- 172.21.15.148:0/1534622535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feac80b2cb0 0x7feac80c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:30.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.559+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 >> 172.21.15.148:0/1534622535 conn(0x7feac8061a20 msgr2=0x7feac80b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:30.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.560+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 shutdown_connections 2024-04-25T09:54:30.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:30.560+0000 7feace742640 1 -- 172.21.15.148:0/1534622535 wait complete. 2024-04-25T09:54:30.976 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:54:23.300442+0000', 'modified': '2024-04-25T09:54:28.348024+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/1500268127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 1500268127}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 1500268127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T09:54:30.976 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:54:30.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:54:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.076+0000 7f4641bc1640 1 Processor -- start 2024-04-25T09:54:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.076+0000 7f4641bc1640 1 -- start start 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f4641bc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f463c06e360 0x7f463c06e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f4641bc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 0x7f463c06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f4641bc1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c15e020 con 0x7f463c06ec80 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f4641bc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463c06b890 con 0x7f463c06e360 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f4641bc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c06ac80 con 0x7f463c06bd20 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f463affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f463c06ec80 0x7f463c0b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51150/0 (socket says 172.21.15.148:51150) 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f463affd640 1 -- 172.21.15.148:0/1493045321 learned_addr learned my addr 172.21.15.148:0/1493045321 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863457311 0 0) 0x7f463c15e020 con 0x7f463c06ec80 2024-04-25T09:54:31.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1493045321 >> v2:172.21.15.188:3300/0 conn(0x7f463c06e360 0x7f463c06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f463bfff640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 0x7f463c06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.077+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4624003520 con 0x7f463c06ec80 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642573990 0 0) 0x7f4624003520 con 0x7f463c06ec80 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 msgr2=0x7f463c06c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 0x7f463c06c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 >> v2:172.21.15.188:3300/0 conn(0x7f463c06e360 msgr2=0x7f463c06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1493045321 >> v2:172.21.15.188:3300/0 conn(0x7f463c06e360 0x7f463c06e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c06af60 con 0x7f463c06ec80 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4630001670 con 0x7f463c06ec80 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 >> v1:172.21.15.148:6789/0 conn(0x7f463c06ec80 legacy=0x7f463c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 0x7f4624004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624004d40 0x7f46240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c06b890 con 0x7f4624004d40 2024-04-25T09:54:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c06ac80 con 0x7f46240039b0 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462400b9c0 con 0x7f463c06bd20 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463bfff640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624004d40 0x7f46240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463affd640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 0x7f4624004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463bfff640 1 -- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 msgr2=0x7f4624004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463bfff640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c06bd20 0x7f4624004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463bfff640 1 -- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240039b0 msgr2=0x7f4624003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463bfff640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46240039b0 0x7f4624003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.078+0000 7f463bfff640 1 -- 172.21.15.148:0/1493045321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624003520 con 0x7f4624004d40 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f463bfff640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624004d40 0x7f46240051c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4634009fc0 tx=0x7f4624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9c7870d6dd02aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634009c10 con 0x7f4624004d40 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4634036070 con 0x7f4624004d40 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f463a7fc640 1 -- 172.21.15.148:0/1493045321 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634031880 con 0x7f4624004d40 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f4641bc1640 1 -- 172.21.15.148:0/1493045321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624004d40 msgr2=0x7f46240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f4641bc1640 1 --2- 172.21.15.148:0/1493045321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624004d40 0x7f46240051c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4634009fc0 tx=0x7f4624005830 comp rx=0 tx=0).stop 2024-04-25T09:54:31.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f463bfff640 1 -- 172.21.15.148:0/1493045321 reap_dead start 2024-04-25T09:54:31.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f4641bc1640 1 -- 172.21.15.148:0/1493045321 shutdown_connections 2024-04-25T09:54:31.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.079+0000 7f4641bc1640 1 -- 172.21.15.148:0/1493045321 >> 172.21.15.148:0/1493045321 conn(0x7f463c0cfec0 msgr2=0x7f463c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.080+0000 7f4641bc1640 1 -- 172.21.15.148:0/1493045321 shutdown_connections 2024-04-25T09:54:31.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.080+0000 7f4641bc1640 1 -- 172.21.15.148:0/1493045321 wait complete. 2024-04-25T09:54:31.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.081+0000 7f4641bc1640 1 Processor -- start 2024-04-25T09:54:31.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.081+0000 7f4641bc1640 1 -- start start 2024-04-25T09:54:31.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624004d40 0x7f463c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c0b08d0 0x7f463c13d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 0x7f463c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0b2fd0 con 0x7f463c1545f0 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c06b520 con 0x7f463c0b08d0 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0b3150 con 0x7f4624004d40 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 0x7f463c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 0x7f463c154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44724/0 (socket says 172.21.15.148:44724) 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 -- 172.21.15.148:0/627838067 learned_addr learned my addr 172.21.15.148:0/627838067 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463affd640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c0b08d0 0x7f463c13d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463b7fe640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624004d40 0x7f463c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 -- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624004d40 msgr2=0x7f463c0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624004d40 0x7f463c0aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 -- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c0b08d0 msgr2=0x7f463c13d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c0b08d0 0x7f463c13d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 -- 172.21.15.148:0/627838067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0af070 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463affd640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c0b08d0 0x7f463c13d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463b7fe640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624004d40 0x7f463c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f463bfff640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 0x7f463c154a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4634033040 tx=0x7f4634009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4623fff640 1 -- 172.21.15.148:0/627838067 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634009c10 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4623fff640 1 -- 172.21.15.148:0/627838067 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4634031dd0 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c0af340 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4623fff640 1 -- 172.21.15.148:0/627838067 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634003bd0 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.082+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c0af900 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.084+0000 7f4623fff640 1 -- 172.21.15.148:0/627838067 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4634005d10 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.084+0000 7f4623fff640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.084+0000 7f4623fff640 1 -- 172.21.15.148:0/627838067 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46340abbf0 con 0x7f463c1545f0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.084+0000 7f463b7fe640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.084+0000 7f463b7fe640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f462c009bc0 tx=0x7f462c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.085+0000 7f464499e740 1 Processor -- start 2024-04-25T09:54:31.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.085+0000 7f464499e740 1 -- start start 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f464499e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a25504f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25506d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a255024f10 0x55a25506b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f464499e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a254f87a10 con 0x55a25504f230 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a254f30b30 con 0x55a25506cad0 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a254f83c50 con 0x55a255024f10 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a25504f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a25504f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44738/0 (socket says 172.21.15.148:44738) 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 -- 172.21.15.148:0/1630800610 learned_addr learned my addr 172.21.15.148:0/1630800610 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463bfff640 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a255024f10 0x55a25506b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463affd640 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25506d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 -- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a255024f10 msgr2=0x55a25506b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a255024f10 0x55a25506b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 -- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 msgr2=0x55a25506d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25506d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 -- 172.21.15.148:0/1630800610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a254f45e10 con 0x55a25504f230 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a25504f610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f462c014f00 tx=0x7f462c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98003ef812b954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f46217fa640 1 -- 172.21.15.148:0/1630800610 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c027070 con 0x55a25504f230 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f46217fa640 1 -- 172.21.15.148:0/1630800610 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c016e40 con 0x55a25504f230 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.086+0000 7f46217fa640 1 -- 172.21.15.148:0/1630800610 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c023860 con 0x55a25504f230 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 -- 172.21.15.148:0/1630800610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 msgr2=0x55a25504f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a25504f610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f462c014f00 tx=0x7f462c017000 comp rx=0 tx=0).stop 2024-04-25T09:54:31.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 -- 172.21.15.148:0/1630800610 shutdown_connections 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a255024f10 0x55a25506b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25506d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 --2- 172.21.15.148:0/1630800610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a25504f610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 -- 172.21.15.148:0/1630800610 >> 172.21.15.148:0/1630800610 conn(0x55a255021840 msgr2=0x55a25506a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 -- 172.21.15.148:0/1630800610 shutdown_connections 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.087+0000 7f464499e740 1 -- 172.21.15.148:0/1630800610 wait complete. 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.088+0000 7f464499e740 1 -- start start 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.088+0000 7f464499e740 1 Processor -- start 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a255050fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25503a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25503cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a255025cb0 con 0x55a25504f230 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a254f440c0 con 0x55a25506cad0 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a25501f770 con 0x55a25503d980 2024-04-25T09:54:31.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 10 client.0 _ll_register_callbacks cb 0x55a254f37ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a255050fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a255050fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44752/0 (socket says 172.21.15.148:44752) 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 -- 172.21.15.148:0/3604907834 learned_addr learned my addr 172.21.15.148:0/3604907834 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 -- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 msgr2=0x55a25503cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25503cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 -- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 msgr2=0x55a25503a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25503a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 -- 172.21.15.148:0/3604907834 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a254c9c2d0 con 0x55a25504f230 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f463b7fe640 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a255050fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f462c008010 tx=0x7f462c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0615c582e6971bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f461a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f461a7fc640 1 -- 172.21.15.148:0/3604907834 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c039060 con 0x55a25504f230 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f461a7fc640 1 -- 172.21.15.148:0/3604907834 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f462c02c860 con 0x55a25504f230 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a254fe6920 con 0x55a25504f230 2024-04-25T09:54:31.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f461a7fc640 1 -- 172.21.15.148:0/3604907834 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f462c03f050 con 0x55a25504f230 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 10 client.4428 fetch_fsmap learned FSMap version 7 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a254d666b0 con 0x55a25504f230 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.089+0000 7f461a7fc640 1 -- 172.21.15.148:0/3604907834 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a254fe6920 con 0x55a25504f230 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f461a7fc640 1 -- 172.21.15.148:0/3604907834 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f462c042020 con 0x55a25504f230 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f464499e740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f464499e740 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f464499e740 20 client.4428 populate_metadata read hostname 'smithi148' 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f464499e740 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a25507e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f464499e740 4 client.4428 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 --> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a254f94f50 con 0x55a25507de70 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f463affd640 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a25507e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f461a7fc640 10 client.4428 ms_handle_connect on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.090+0000 7f463affd640 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a25507e250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb074045e5ffc6fb server_cookie=b31901801101b3e6 in_seq=0 out_seq=0 2024-04-25T09:54:31.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f461a7fc640 1 -- 172.21.15.148:0/3604907834 <== mds.0 v2:172.21.15.188:6833/1500268127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a254f94f50 con 0x55a25507de70 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f461a7fc640 10 client.4428 handle_command_reply: tid=0 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 client.4428 shutdown 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 msgr2=0x55a255050fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a255050fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f462c008010 tx=0x7f462c004730 comp rx=0 tx=0).stop 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 shutdown_connections 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a25507e250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25503cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f461a7fc640 0 client.4428 ms_handle_reset on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25506cad0 0x55a25503a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 --2- 172.21.15.148:0/3604907834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25504f230 0x55a255050fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 >> 172.21.15.148:0/3604907834 conn(0x55a255024f10 msgr2=0x55a25506de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 shutdown_connections 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 1 -- 172.21.15.148:0/3604907834 wait complete. 2024-04-25T09:54:31.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.091+0000 7f464499e740 20 client.4428 trim_cache size 0 max 16384 2024-04-25T09:54:31.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.106+0000 7f464499e740 1 Processor -- start 2024-04-25T09:54:31.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.106+0000 7f464499e740 1 -- start start 2024-04-25T09:54:31.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f464499e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 0x55a255023f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255024450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25506cad0 0x55a255026250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f464499e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c200b40 con 0x55a25503d980 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c1fff40 con 0x55a25504f230 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c200540 con 0x55a25506cad0 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 0x55a255023f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 0x55a255023f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44758/0 (socket says 172.21.15.148:44758) 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.107+0000 7f463b7fe640 1 -- 172.21.15.148:0/126464302 learned_addr learned my addr 172.21.15.148:0/126464302 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f463b7fe640 1 -- 172.21.15.148:0/126464302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25506cad0 msgr2=0x55a255026250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f463b7fe640 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25506cad0 0x55a255026250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f463b7fe640 1 -- 172.21.15.148:0/126464302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 msgr2=0x55a255024450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f463b7fe640 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255024450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f463b7fe640 1 -- 172.21.15.148:0/126464302 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a255069c40 con 0x55a25503d980 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f463b7fe640 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 0x55a255023f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f462c016dc0 tx=0x7f462c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcf5802d7fbb958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f4620ff9640 1 -- 172.21.15.148:0/126464302 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c048070 con 0x55a25503d980 2024-04-25T09:54:31.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f4620ff9640 1 -- 172.21.15.148:0/126464302 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c02c660 con 0x55a25503d980 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f4620ff9640 1 -- 172.21.15.148:0/126464302 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c027050 con 0x55a25503d980 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f464499e740 1 -- 172.21.15.148:0/126464302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 msgr2=0x55a255023f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f464499e740 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 0x55a255023f10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f462c016dc0 tx=0x7f462c02e040 comp rx=0 tx=0).stop 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f464499e740 1 -- 172.21.15.148:0/126464302 shutdown_connections 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f464499e740 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25506cad0 0x55a255026250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f464499e740 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255024450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.108+0000 7f464499e740 1 --2- 172.21.15.148:0/126464302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25503d980 0x55a255023f10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.109+0000 7f464499e740 1 -- 172.21.15.148:0/126464302 >> 172.21.15.148:0/126464302 conn(0x55a255024f10 msgr2=0x55a25506c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.109+0000 7f464499e740 1 -- 172.21.15.148:0/126464302 shutdown_connections 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.109+0000 7f464499e740 1 -- 172.21.15.148:0/126464302 wait complete. 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.109+0000 7f464499e740 1 -- start start 2024-04-25T09:54:31.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.109+0000 7f464499e740 1 Processor -- start 2024-04-25T09:54:31.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25504d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255064710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 0x55a255064c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a255051af0 con 0x55a25506cad0 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25503af10 con 0x55a25504f230 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a25503c510 con 0x55a25503d980 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 10 client.0 _ll_register_callbacks cb 0x55a25501fc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 0x55a255064c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 0x55a255064c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44770/0 (socket says 172.21.15.148:44770) 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 -- 172.21.15.148:0/4041186661 learned_addr learned my addr 172.21.15.148:0/4041186661 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463b7fe640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25504d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463affd640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255064710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 -- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 msgr2=0x55a25504d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25504d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 -- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 msgr2=0x55a255064710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255064710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 -- 172.21.15.148:0/4041186661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2550e0a30 con 0x55a25506cad0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463affd640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255064710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f463bfff640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 0x55a255064c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4634079dc0 tx=0x7f4634079ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3c58ba28dae02b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f461a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f461a7fc640 1 -- 172.21.15.148:0/4041186661 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340827d0 con 0x55a25506cad0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f461a7fc640 1 -- 172.21.15.148:0/4041186661 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4634043380 con 0x55a25506cad0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.110+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f462c02c660 con 0x55a25506cad0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f461a7fc640 1 -- 172.21.15.148:0/4041186661 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4634098050 con 0x55a25506cad0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 10 client.4434 fetch_fsmap learned FSMap version 7 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a255039c90 con 0x55a25506cad0 2024-04-25T09:54:31.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f461a7fc640 1 -- 172.21.15.148:0/4041186661 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c02c660 con 0x55a25506cad0 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f461a7fc640 1 -- 172.21.15.148:0/4041186661 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f463404d070 con 0x55a25506cad0 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 10 client.4434 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 20 client.4434 populate_metadata read hostname 'smithi148' 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a255060340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 4 client.4434 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 --> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a25504fb30 con 0x55a25507de70 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f463b7fe640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a255060340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f463b7fe640 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a255060340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d285ddc75b55393 server_cookie=b9e6b99bcdaf46f9 in_seq=0 out_seq=0 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f461a7fc640 10 client.4434 ms_handle_connect on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f461a7fc640 1 -- 172.21.15.148:0/4041186661 <== mds.0 v2:172.21.15.188:6833/1500268127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55a25504fb30 con 0x55a25507de70 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f461a7fc640 10 client.4434 handle_command_reply: tid=0 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.111+0000 7f464499e740 1 client.4434 shutdown 2024-04-25T09:54:31.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 msgr2=0x55a255064c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 0x55a255064c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4634079dc0 tx=0x7f4634079ae0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 shutdown_connections 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x55a25507de70 0x55a255060340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a25506cad0 0x55a255064c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a25504f230 0x55a255064710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f461a7fc640 0 client.4434 ms_handle_reset on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 --2- 172.21.15.148:0/4041186661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a25503d980 0x55a25504d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 >> 172.21.15.148:0/4041186661 conn(0x55a255024f10 msgr2=0x55a25503fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 shutdown_connections 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 1 -- 172.21.15.148:0/4041186661 wait complete. 2024-04-25T09:54:31.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f464499e740 20 client.4434 trim_cache size 0 max 16384 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4291, 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 7.1437304529999999, 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 7, 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 25, 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 23, 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 14.506720097000001 2024-04-25T09:54:31.122 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:54:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 msgr2=0x7f462806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f462c009bc0 tx=0x7f462c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 msgr2=0x7f463c154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 0x7f463c154a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4634033040 tx=0x7f4634009750 comp rx=0 tx=0).stop 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 shutdown_connections 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c1545f0 0x7f463c154a70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c0b08d0 0x7f463c13d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 --2- 172.21.15.148:0/627838067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624004d40 0x7f463c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 >> 172.21.15.148:0/627838067 conn(0x7f463c0cfec0 msgr2=0x7f463c0b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 shutdown_connections 2024-04-25T09:54:31.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.112+0000 7f4641bc1640 1 -- 172.21.15.148:0/627838067 wait complete. 2024-04-25T09:54:31.125 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:54:31.125 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:31.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.219+0000 7f6ab1c3a640 1 Processor -- start 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.219+0000 7f6ab1c3a640 1 -- start start 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6ab1c3a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6aac15dc90 0x7f6aac1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6ab1c3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6aac0d1810 0x7f6aac0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6ab1c3a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac0cce50 con 0x7f6aac0cc8e0 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6ab1c3a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0d21c0 con 0x7f6aac0d1810 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6ab1c3a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac161af0 con 0x7f6aac15dc90 2024-04-25T09:54:31.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6aac0d1810 0x7f6aac0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6aac0d1810 0x7f6aac0d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44776/0 (socket says 172.21.15.148:44776) 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 -- 172.21.15.148:0/1626400559 learned_addr learned my addr 172.21.15.148:0/1626400559 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aab7fe640 1 --2- 172.21.15.148:0/1626400559 >> v2:172.21.15.188:3300/0 conn(0x7f6aac15dc90 0x7f6aac1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 -- 172.21.15.148:0/1626400559 >> v2:172.21.15.188:3300/0 conn(0x7f6aac15dc90 msgr2=0x7f6aac1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 --2- 172.21.15.148:0/1626400559 >> v2:172.21.15.188:3300/0 conn(0x7f6aac15dc90 0x7f6aac1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 -- 172.21.15.148:0/1626400559 >> v1:172.21.15.188:6789/0 conn(0x7f6aac0cc8e0 legacy=0x7f6aac15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aabfff640 1 -- 172.21.15.148:0/1626400559 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac161dd0 con 0x7f6aac0d1810 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.220+0000 7f6aab7fe640 1 --2- 172.21.15.148:0/1626400559 >> v2:172.21.15.188:3300/0 conn(0x7f6aac15dc90 0x7f6aac1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.221+0000 7f6aabfff640 1 --2- 172.21.15.148:0/1626400559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac0d1810 0x7f6aac0d1c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6aac06b820 tx=0x7f6aa402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4455bb179fb0b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.221+0000 7f6aaa7fc640 1 -- 172.21.15.148:0/1626400559 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4036070 con 0x7f6aac0d1810 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.221+0000 7f6aaa7fc640 1 -- 172.21.15.148:0/1626400559 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa4031600 con 0x7f6aac0d1810 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.221+0000 7f6aaa7fc640 1 -- 172.21.15.148:0/1626400559 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40319d0 con 0x7f6aac0d1810 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/1626400559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac0d1810 msgr2=0x7f6aac0d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/1626400559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac0d1810 0x7f6aac0d1c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6aac06b820 tx=0x7f6aa402a660 comp rx=0 tx=0).stop 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/1626400559 shutdown_connections 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/1626400559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac0d1810 0x7f6aac0d1c80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/1626400559 >> v2:172.21.15.188:3300/0 conn(0x7f6aac15dc90 0x7f6aac1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/1626400559 >> 172.21.15.148:0/1626400559 conn(0x7f6aac06f590 msgr2=0x7f6aac06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/1626400559 shutdown_connections 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.222+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/1626400559 wait complete. 2024-04-25T09:54:31.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.223+0000 7f6ab1c3a640 1 Processor -- start 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.223+0000 7f6ab1c3a640 1 -- start start 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aac0cc8e0 0x7f6aac155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aac0d1810 0x7f6aac0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 0x7f6aac1e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15e070 con 0x7f6aac15dc90 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0cd130 con 0x7f6aac0cc8e0 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac15e1f0 con 0x7f6aac0d1810 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 0x7f6aac1e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 0x7f6aac1e9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44792/0 (socket says 172.21.15.148:44792) 2024-04-25T09:54:31.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 -- 172.21.15.148:0/2179892336 learned_addr learned my addr 172.21.15.148:0/2179892336 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aab7fe640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aac0cc8e0 0x7f6aac155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aaaffd640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aac0d1810 0x7f6aac0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 -- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aac0d1810 msgr2=0x7f6aac0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aac0d1810 0x7f6aac0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 -- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aac0cc8e0 msgr2=0x7f6aac155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aac0cc8e0 0x7f6aac155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 -- 172.21.15.148:0/2179892336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac0ca690 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6aabfff640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 0x7f6aac1e9600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009f40 tx=0x7f6aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4036050 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa4042070 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4009510 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.224+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac1e7460 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.225+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac1e7a20 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.225+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac1e7250 con 0x7f6aac15dc90 2024-04-25T09:54:31.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.226+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6aa4007bf0 con 0x7f6aac15dc90 2024-04-25T09:54:31.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.226+0000 7f6a93fff640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a9806b4d0 0x7f6a9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:31.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.226+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6aa40ab4b0 con 0x7f6aac15dc90 2024-04-25T09:54:31.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.226+0000 7f6aab7fe640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a9806b4d0 0x7f6a9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:31.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.227+0000 7f6aab7fe640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a9806b4d0 0x7f6a9806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0b2320 tx=0x7f6a9c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:31.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.230+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6aa407a110 con 0x7f6aac15dc90 2024-04-25T09:54:31.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.435+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6aac073990 con 0x7f6aac15dc90 2024-04-25T09:54:31.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.435+0000 7f6a93fff640 1 -- 172.21.15.148:0/2179892336 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6aa4079a10 con 0x7f6aac15dc90 2024-04-25T09:54:31.438 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a9806b4d0 msgr2=0x7f6a9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a9806b4d0 0x7f6a9806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0b2320 tx=0x7f6a9c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 msgr2=0x7f6aac1e9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 0x7f6aac1e9600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009f40 tx=0x7f6aa4004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 shutdown_connections 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a9806b4d0 0x7f6a9806d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6aac15dc90 0x7f6aac1e9600 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aac0d1810 0x7f6aac0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 --2- 172.21.15.148:0/2179892336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aac0cc8e0 0x7f6aac155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 >> 172.21.15.148:0/2179892336 conn(0x7f6aac06f590 msgr2=0x7f6aac15b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 shutdown_connections 2024-04-25T09:54:31.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:31.437+0000 7f6ab1c3a640 1 -- 172.21.15.148:0/2179892336 wait complete. 2024-04-25T09:54:31.450 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:54:31.450 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:54:31.450 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:54:31.450 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T09:54:31.450 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:31.451 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:54:31.451 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:54:31.451 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:54:31.451 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T09:54:31.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.451 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:54:31.466 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:31.466 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:31.467 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:54:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42764sec preferred_lft 42764sec 2024-04-25T09:54:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:54:31.468 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:31.468 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:54:31.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:54:31.469 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:31.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.607 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:54:31.629 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:54:31.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.632 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:54:31.708 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:54:31.709 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:54:31.709 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:54:31.709 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:54:31.709 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:54:31.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.710 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:31.710 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:31.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:54:31.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:54:31.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:54:31.710 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:31.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.931 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:54:31.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:31.945 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-04-25T09:54:32.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.000 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:32.000 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:32.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:54:32.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T09:54:32.001 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:32.077 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T09:54:32.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.077 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:32.077 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:32.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T09:54:32.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:54:32.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T09:54:32.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T09:54:32.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T09:54:32.078 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:32.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.197 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:32.197 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:32.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:54:32.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:54:32.197 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:32.233 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T09:54:32.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:32.233 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:32.288 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T09:54:32.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:32.288 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:32.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-04-25T09:54:32.411 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:54:32.468 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-04-25T09:54:32.468 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-04-25T09:54:32.468 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-04-25T09:54:32.468 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-04-25T09:54:32.469 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-04-25T09:54:32.470 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:54:32.471 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:54:32.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.472 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-04-25T09:54:32.527 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:54:32.527 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T09:54:32.569 DEBUG:teuthology.orchestra.run.smithi148:> sudo modprobe fuse 2024-04-25T09:54:32.594 DEBUG:teuthology.orchestra.run.smithi148:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/proc 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/sys 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/dev 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/security 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/shm 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/pts 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/run 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/cgroup 2024-04-25T09:54:32.650 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/pstore 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/bpf 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/ 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/selinux 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/mqueue 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/dev/hugepages 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/debug 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/tracing 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/kernel/config 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/sys/fs/fuse/connections 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:54:32.651 INFO:teuthology.orchestra.run.smithi148.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/run/user/1000 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run.smithi148.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:54:32.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.652 DEBUG:teuthology.orchestra.run.smithi148:> ls /sys/fs/fuse/connections 2024-04-25T09:54:32.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:2024-04-25T09:54:32.659+0000 7eff5811d480 -1 init, newargv = 0x55bec9e2d230 newargc=15 2024-04-25T09:54:32.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[42142]: starting ceph client 2024-04-25T09:54:32.668 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[42142]: starting fuse 2024-04-25T09:54:32.677 INFO:teuthology.orchestra.run.smithi148.stdout:43 2024-04-25T09:54:32.677 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:54:32.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:54:32.677 DEBUG:teuthology.orchestra.run.smithi148:> import glob 2024-04-25T09:54:32.677 DEBUG:teuthology.orchestra.run.smithi148:> import re 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> import subprocess 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> def _find_admin_socket(client_name): 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> files = glob.glob(asok_path) 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> # Given a non-glob path, it better be there 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> if "*" not in asok_path: 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> assert(len(files) == 1) 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> return files[0] 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> for f in files: 2024-04-25T09:54:32.678 DEBUG:teuthology.orchestra.run.smithi148:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> contents = proc_f.read() 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> if mountpoint in contents: 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> return f 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> print(_find_admin_socket("client.admin")) 2024-04-25T09:54:32.679 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:54:32.794 INFO:teuthology.orchestra.run.smithi148.stdout:/var/run/ceph/ceph-client.admin.42142.asok 2024-04-25T09:54:32.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:54:32.805 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42142.asok 2024-04-25T09:54:32.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.805 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42142.asok status 2024-04-25T09:54:32.939 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "metadata": { 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "entity_id": "admin", 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "hostname": "smithi148", 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "pid": "42142", 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "root": "/" 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_count": 0, 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "dentry_pinned_count": 0, 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4449, 2024-04-25T09:54:32.940 INFO:teuthology.orchestra.run.smithi148.stdout: "inst": { 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "name": { 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "client", 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "num": 4449 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2816421634 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": { 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "type": "v1", 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:54:32.941 INFO:teuthology.orchestra.run.smithi148.stdout: "nonce": 2816421634 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "inst_str": "client.4449 192.168.0.1:0/2816421634", 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "addr_str": "192.168.0.1:0/2816421634", 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "inode_count": 1, 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_epoch": 7, 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch": 25, 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "blocklisted": false, 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs" 2024-04-25T09:54:32.942 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:54:32.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.948 DEBUG:teuthology.orchestra.run.smithi148:> stat --file-system '--printf=%T 2024-04-25T09:54:32.948 DEBUG:teuthology.orchestra.run.smithi148:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:32.963 INFO:teuthology.orchestra.run.smithi148.stdout:fuseblk 2024-04-25T09:54:32.964 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:32.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:32.964 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:33.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:33.030 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T09:54:33.137 INFO:teuthology.orchestra.run.smithi148.stdout:# file: . 2024-04-25T09:54:33.137 INFO:teuthology.orchestra.run.smithi148.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T09:54:33.137 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:33.138 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:54:33.138 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:33.153 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:33.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.287+0000 7f889461f640 1 Processor -- start 2024-04-25T09:54:33.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.287+0000 7f889461f640 1 -- start start 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f889461f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f888c15dd30 0x7f888c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f889461f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f888c0d0ad0 0x7f888c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f889461f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888c0d1df0 con 0x7f888c0b9770 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f889461f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888c0d20d0 con 0x7f888c15dd30 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f889461f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888c0d23b0 con 0x7f888c0d0ad0 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f88923bb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f888c15dd30 0x7f888c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f8891bba640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f888c0b9770 0x7f888c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51166/0 (socket says 172.21.15.148:51166) 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f8891bba640 1 -- 172.21.15.148:0/3023895354 learned_addr learned my addr 172.21.15.148:0/3023895354 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.288+0000 7f88923bb640 1 --2- 172.21.15.148:0/3023895354 >> v2:172.21.15.148:3300/0 conn(0x7f888c15dd30 0x7f888c0b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44806/0 (socket says 172.21.15.148:44806) 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f8892bbc640 1 --2- 172.21.15.148:0/3023895354 >> v2:172.21.15.188:3300/0 conn(0x7f888c0d0ad0 0x7f888c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f88923bb640 1 -- 172.21.15.148:0/3023895354 >> v2:172.21.15.188:3300/0 conn(0x7f888c0d0ad0 msgr2=0x7f888c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:33.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f88923bb640 1 --2- 172.21.15.148:0/3023895354 >> v2:172.21.15.188:3300/0 conn(0x7f888c0d0ad0 0x7f888c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f88923bb640 1 -- 172.21.15.148:0/3023895354 >> v1:172.21.15.148:6789/0 conn(0x7f888c0b9770 legacy=0x7f888c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f88923bb640 1 -- 172.21.15.148:0/3023895354 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0d2690 con 0x7f888c15dd30 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f88913b9640 1 -- 172.21.15.148:0/3023895354 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52774507 0 0) 0x7f888c0d1df0 con 0x7f888c0b9770 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.289+0000 7f88923bb640 1 --2- 172.21.15.148:0/3023895354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c15dd30 0x7f888c0b9230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f888c0ce200 tx=0x7f888802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cbe1e07587f61d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f88913b9640 1 -- 172.21.15.148:0/3023895354 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888007740 con 0x7f888c15dd30 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f88913b9640 1 -- 172.21.15.148:0/3023895354 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880078e0 con 0x7f888c15dd30 2024-04-25T09:54:33.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f88913b9640 1 -- 172.21.15.148:0/3023895354 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888007cb0 con 0x7f888c15dd30 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 -- 172.21.15.148:0/3023895354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c15dd30 msgr2=0x7f888c0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 --2- 172.21.15.148:0/3023895354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c15dd30 0x7f888c0b9230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f888c0ce200 tx=0x7f888802a560 comp rx=0 tx=0).stop 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 -- 172.21.15.148:0/3023895354 shutdown_connections 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 --2- 172.21.15.148:0/3023895354 >> v2:172.21.15.188:3300/0 conn(0x7f888c0d0ad0 0x7f888c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 --2- 172.21.15.148:0/3023895354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c15dd30 0x7f888c0b9230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 -- 172.21.15.148:0/3023895354 >> 172.21.15.148:0/3023895354 conn(0x7f888c061c30 msgr2=0x7f888c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 -- 172.21.15.148:0/3023895354 shutdown_connections 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.290+0000 7f889461f640 1 -- 172.21.15.148:0/3023895354 wait complete. 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.291+0000 7f889461f640 1 Processor -- start 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.291+0000 7f889461f640 1 -- start start 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f889461f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 0x7f888c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f889461f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c0d0ad0 0x7f888c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f889461f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c15dd30 0x7f888c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f889461f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0d1b90 con 0x7f888c0b9770 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f889461f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0d1a10 con 0x7f888c15dd30 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f889461f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888c0d2e90 con 0x7f888c0d0ad0 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 0x7f888c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 0x7f888c1530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44812/0 (socket says 172.21.15.148:44812) 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 -- 172.21.15.148:0/1475709617 learned_addr learned my addr 172.21.15.148:0/1475709617 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f8891bba640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c0d0ad0 0x7f888c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f8892bbc640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c15dd30 0x7f888c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 -- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c0d0ad0 msgr2=0x7f888c155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c0d0ad0 0x7f888c155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 -- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c15dd30 msgr2=0x7f888c1563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c15dd30 0x7f888c1563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.292+0000 7f88923bb640 1 -- 172.21.15.148:0/1475709617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c158d70 con 0x7f888c0b9770 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.293+0000 7f88923bb640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 0x7f888c1530f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f888802ab50 tx=0x7f8888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.293+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888803f070 con 0x7f888c0b9770 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.293+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8888031c40 con 0x7f888c0b9770 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.293+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c159050 con 0x7f888c0b9770 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.293+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888043410 con 0x7f888c0b9770 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.293+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c159610 con 0x7f888c0b9770 2024-04-25T09:54:33.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.294+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c073990 con 0x7f888c0b9770 2024-04-25T09:54:33.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.294+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88880435b0 con 0x7f888c0b9770 2024-04-25T09:54:33.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.295+0000 7f8882ffd640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f887006b4d0 0x7f887006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.295+0000 7f8891bba640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f887006b4d0 0x7f887006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.295+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f88880ab840 con 0x7f888c0b9770 2024-04-25T09:54:33.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.295+0000 7f8891bba640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f887006b4d0 0x7f887006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f888c0b5db0 tx=0x7f887c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.299+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8888079d30 con 0x7f888c0b9770 2024-04-25T09:54:33.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.472+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f888c0d4600 con 0x7f888c0b9770 2024-04-25T09:54:33.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.473+0000 7f8882ffd640 1 -- 172.21.15.148:0/1475709617 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f888807ddd0 con 0x7f888c0b9770 2024-04-25T09:54:33.476 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:33.476 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f887006b4d0 msgr2=0x7f887006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f887006b4d0 0x7f887006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f888c0b5db0 tx=0x7f887c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 msgr2=0x7f888c1530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 0x7f888c1530f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f888802ab50 tx=0x7f8888004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 shutdown_connections 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f887006b4d0 0x7f887006d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f888c15dd30 0x7f888c1563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f888c0d0ad0 0x7f888c155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 --2- 172.21.15.148:0/1475709617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f888c0b9770 0x7f888c1530f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 >> 172.21.15.148:0/1475709617 conn(0x7f888c061c30 msgr2=0x7f888c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:33.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 shutdown_connections 2024-04-25T09:54:33.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.475+0000 7f889461f640 1 -- 172.21.15.148:0/1475709617 wait complete. 2024-04-25T09:54:33.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:33.489 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:33.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.587+0000 7fdf5949e640 1 Processor -- start 2024-04-25T09:54:33.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.587+0000 7fdf5949e640 1 -- start start 2024-04-25T09:54:33.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf5949e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5415dc20 0x7fdf540d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf5949e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf540d1ce0 0x7fdf540b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf5949e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540d2120 con 0x7fdf540ce670 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf5949e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540d23a0 con 0x7fdf540d1ce0 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf5949e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540d2680 con 0x7fdf5415dc20 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf537fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdf540ce670 0x7fdf5406c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56956/0 (socket says 172.21.15.148:56956) 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf537fe640 1 -- 172.21.15.148:0/2670439851 learned_addr learned my addr 172.21.15.148:0/2670439851 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf52ffd640 1 --2- 172.21.15.148:0/2670439851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5415dc20 0x7fdf540d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf527fc640 1 --2- 172.21.15.148:0/2670439851 >> v2:172.21.15.188:3300/0 conn(0x7fdf540d1ce0 0x7fdf540b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf51ffb640 1 -- 172.21.15.148:0/2670439851 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666267943 0 0) 0x7fdf4c001490 con 0x7fdf540ce670 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf51ffb640 1 -- 172.21.15.148:0/2670439851 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c003520 con 0x7fdf540ce670 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf52ffd640 1 -- 172.21.15.148:0/2670439851 >> v2:172.21.15.188:3300/0 conn(0x7fdf540d1ce0 msgr2=0x7fdf540b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf52ffd640 1 --2- 172.21.15.148:0/2670439851 >> v2:172.21.15.188:3300/0 conn(0x7fdf540d1ce0 0x7fdf540b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.588+0000 7fdf52ffd640 1 -- 172.21.15.148:0/2670439851 >> v1:172.21.15.148:6789/0 conn(0x7fdf540ce670 legacy=0x7fdf5406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.589+0000 7fdf52ffd640 1 -- 172.21.15.148:0/2670439851 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540d2960 con 0x7fdf5415dc20 2024-04-25T09:54:33.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.589+0000 7fdf52ffd640 1 --2- 172.21.15.148:0/2670439851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5415dc20 0x7fdf540d3440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80a2607fc65372d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.589+0000 7fdf51ffb640 1 -- 172.21.15.148:0/2670439851 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44037070 con 0x7fdf5415dc20 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.589+0000 7fdf51ffb640 1 -- 172.21.15.148:0/2670439851 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4402ae50 con 0x7fdf5415dc20 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.589+0000 7fdf51ffb640 1 -- 172.21.15.148:0/2670439851 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44032640 con 0x7fdf5415dc20 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 -- 172.21.15.148:0/2670439851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5415dc20 msgr2=0x7fdf540d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 --2- 172.21.15.148:0/2670439851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5415dc20 0x7fdf540d3440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097a0 tx=0x7fdf4402a100 comp rx=0 tx=0).stop 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 -- 172.21.15.148:0/2670439851 shutdown_connections 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 --2- 172.21.15.148:0/2670439851 >> v2:172.21.15.188:3300/0 conn(0x7fdf540d1ce0 0x7fdf540b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 --2- 172.21.15.148:0/2670439851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5415dc20 0x7fdf540d3440 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 -- 172.21.15.148:0/2670439851 >> 172.21.15.148:0/2670439851 conn(0x7fdf54061a20 msgr2=0x7fdf54061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 -- 172.21.15.148:0/2670439851 shutdown_connections 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.590+0000 7fdf5949e640 1 -- 172.21.15.148:0/2670439851 wait complete. 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.591+0000 7fdf5949e640 1 Processor -- start 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.591+0000 7fdf5949e640 1 -- start start 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf5949e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 0x7fdf54160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf5949e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf540d1ce0 0x7fdf5415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf5949e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5415dc20 0x7fdf54149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf5949e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540ceb00 con 0x7fdf540ce670 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf5949e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540d2f20 con 0x7fdf5415dc20 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf5949e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540cec80 con 0x7fdf540d1ce0 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 0x7fdf54160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 0x7fdf54160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57226/0 (socket says 172.21.15.148:57226) 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 -- 172.21.15.148:0/3711011241 learned_addr learned my addr 172.21.15.148:0/3711011241 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:33.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf537fe640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5415dc20 0x7fdf54149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf527fc640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf540d1ce0 0x7fdf5415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 -- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf540d1ce0 msgr2=0x7fdf5415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf540d1ce0 0x7fdf5415f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 -- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5415dc20 msgr2=0x7fdf54149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5415dc20 0x7fdf54149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.592+0000 7fdf52ffd640 1 -- 172.21.15.148:0/3711011241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540b4430 con 0x7fdf540ce670 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf527fc640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf540d1ce0 0x7fdf5415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf537fe640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5415dc20 0x7fdf54149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf52ffd640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 0x7fdf54160a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf4402a140 tx=0x7fdf440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44041070 con 0x7fdf540ce670 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4403c460 con 0x7fdf540ce670 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5415eb90 con 0x7fdf540ce670 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44037040 con 0x7fdf540ce670 2024-04-25T09:54:33.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.593+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5415f150 con 0x7fdf540ce670 2024-04-25T09:54:33.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.594+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf54073a00 con 0x7fdf540ce670 2024-04-25T09:54:33.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.594+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf44044070 con 0x7fdf540ce670 2024-04-25T09:54:33.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.595+0000 7fdf3b7fe640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4006b500 0x7fdf4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.595+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdf440aaa70 con 0x7fdf540ce670 2024-04-25T09:54:33.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.595+0000 7fdf527fc640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4006b500 0x7fdf4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.595+0000 7fdf527fc640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4006b500 0x7fdf4006d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf540b6020 tx=0x7fdf48009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.599+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf44079110 con 0x7fdf540ce670 2024-04-25T09:54:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.769+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf5406cd90 con 0x7fdf540ce670 2024-04-25T09:54:33.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.769+0000 7fdf3b7fe640 1 -- 172.21.15.148:0/3711011241 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf4407d1b0 con 0x7fdf540ce670 2024-04-25T09:54:33.772 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4006b500 msgr2=0x7fdf4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4006b500 0x7fdf4006d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf540b6020 tx=0x7fdf48009190 comp rx=0 tx=0).stop 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 msgr2=0x7fdf54160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 0x7fdf54160a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf4402a140 tx=0x7fdf440046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 shutdown_connections 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4006b500 0x7fdf4006d9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5415dc20 0x7fdf54149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf540d1ce0 0x7fdf5415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 --2- 172.21.15.148:0/3711011241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf540ce670 0x7fdf54160a60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 >> 172.21.15.148:0/3711011241 conn(0x7fdf54061a20 msgr2=0x7fdf540c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:33.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 shutdown_connections 2024-04-25T09:54:33.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.771+0000 7fdf5949e640 1 -- 172.21.15.148:0/3711011241 wait complete. 2024-04-25T09:54:33.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 Processor -- start 2024-04-25T09:54:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 -- start start 2024-04-25T09:54:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe59815bb50 0x7fe5980c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5980c70d0 0x7fe5980cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980cfb40 0x7fe598070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980c8680 con 0x7fe59815bb50 2024-04-25T09:54:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980c75e0 con 0x7fe5980c70d0 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.883+0000 7fe59d55c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980c78c0 con 0x7fe5980cfb40 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe59815bb50 0x7fe5980c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe59815bb50 0x7fe5980c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57228/0 (socket says 172.21.15.148:57228) 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 -- 172.21.15.148:0/3008639287 learned_addr learned my addr 172.21.15.148:0/3008639287 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe5967fc640 1 --2- 172.21.15.148:0/3008639287 >> v2:172.21.15.188:3300/0 conn(0x7fe5980c70d0 0x7fe5980cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe5977fe640 1 --2- 172.21.15.148:0/3008639287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980cfb40 0x7fe598070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 -- 172.21.15.148:0/3008639287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980cfb40 msgr2=0x7fe598070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 --2- 172.21.15.148:0/3008639287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980cfb40 0x7fe598070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 -- 172.21.15.148:0/3008639287 >> v2:172.21.15.188:3300/0 conn(0x7fe5980c70d0 msgr2=0x7fe5980cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 --2- 172.21.15.148:0/3008639287 >> v2:172.21.15.188:3300/0 conn(0x7fe5980c70d0 0x7fe5980cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe596ffd640 1 -- 172.21.15.148:0/3008639287 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5980c7ba0 con 0x7fe59815bb50 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.884+0000 7fe5967fc640 1 --2- 172.21.15.148:0/3008639287 >> v2:172.21.15.188:3300/0 conn(0x7fe5980c70d0 0x7fe5980cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe596ffd640 1 --2- 172.21.15.148:0/3008639287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe59815bb50 0x7fe5980c6b90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe59806f220 tx=0x7fe58802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762f8bdd7e528800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe595ffb640 1 -- 172.21.15.148:0/3008639287 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588037070 con 0x7fe59815bb50 2024-04-25T09:54:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe595ffb640 1 -- 172.21.15.148:0/3008639287 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588032440 con 0x7fe59815bb50 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe595ffb640 1 -- 172.21.15.148:0/3008639287 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588032810 con 0x7fe59815bb50 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 -- 172.21.15.148:0/3008639287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe59815bb50 msgr2=0x7fe5980c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 --2- 172.21.15.148:0/3008639287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe59815bb50 0x7fe5980c6b90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe59806f220 tx=0x7fe58802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 -- 172.21.15.148:0/3008639287 shutdown_connections 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 --2- 172.21.15.148:0/3008639287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980cfb40 0x7fe598070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 --2- 172.21.15.148:0/3008639287 >> v2:172.21.15.188:3300/0 conn(0x7fe5980c70d0 0x7fe5980cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 --2- 172.21.15.148:0/3008639287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe59815bb50 0x7fe5980c6b90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 -- 172.21.15.148:0/3008639287 >> 172.21.15.148:0/3008639287 conn(0x7fe59806f500 msgr2=0x7fe59806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 -- 172.21.15.148:0/3008639287 shutdown_connections 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.885+0000 7fe59d55c640 1 -- 172.21.15.148:0/3008639287 wait complete. 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 Processor -- start 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 -- start start 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980c70d0 0x7fe59815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 0x7fe5981f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe59815bb50 0x7fe5981f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59815f6d0 con 0x7fe5980cfb40 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980c8160 con 0x7fe59815bb50 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.887+0000 7fe59d55c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59815f850 con 0x7fe5980c70d0 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 0x7fe5981f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 0x7fe5981f0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57242/0 (socket says 172.21.15.148:57242) 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 -- 172.21.15.148:0/619845636 learned_addr learned my addr 172.21.15.148:0/619845636 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5977fe640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe59815bb50 0x7fe5981f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe596ffd640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980c70d0 0x7fe59815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 -- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980c70d0 msgr2=0x7fe59815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980c70d0 0x7fe59815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 -- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe59815bb50 msgr2=0x7fe5981f1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe59815bb50 0x7fe5981f1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 -- 172.21.15.148:0/619845636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59815d390 con 0x7fe5980cfb40 2024-04-25T09:54:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe5967fc640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 0x7fe5981f0d20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d800 tx=0x7fe58c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c018030 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c0096a0 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c009870 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59815cdd0 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.888+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe598150da0 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.889+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598073a00 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.890+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe58c01b070 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.890+0000 7fe57f7fe640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe58406b4d0 0x7fe58406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.890+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe58c08a650 con 0x7fe5980cfb40 2024-04-25T09:54:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.890+0000 7fe596ffd640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe58406b4d0 0x7fe58406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:33.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.890+0000 7fe596ffd640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe58406b4d0 0x7fe58406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe588034000 tx=0x7fe58802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:33.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:33.894+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe58c059220 con 0x7fe5980cfb40 2024-04-25T09:54:34.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.064+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5980cf150 con 0x7fe5980cfb40 2024-04-25T09:54:34.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.065+0000 7fe57f7fe640 1 -- 172.21.15.148:0/619845636 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe58c058b20 con 0x7fe5980cfb40 2024-04-25T09:54:34.067 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:34.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe58406b4d0 msgr2=0x7fe58406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe58406b4d0 0x7fe58406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe588034000 tx=0x7fe58802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 msgr2=0x7fe5981f0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 0x7fe5981f0d20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d800 tx=0x7fe58c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 shutdown_connections 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe58406b4d0 0x7fe58406d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe59815bb50 0x7fe5981f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5980cfb40 0x7fe5981f0d20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 --2- 172.21.15.148:0/619845636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5980c70d0 0x7fe59815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 >> 172.21.15.148:0/619845636 conn(0x7fe59806f500 msgr2=0x7fe5980b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 shutdown_connections 2024-04-25T09:54:34.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.067+0000 7fe59d55c640 1 -- 172.21.15.148:0/619845636 wait complete. 2024-04-25T09:54:34.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.177+0000 7f2b61b09640 1 Processor -- start 2024-04-25T09:54:34.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.178+0000 7f2b61b09640 1 -- start start 2024-04-25T09:54:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.178+0000 7f2b61b09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 0x7f2b5c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.178+0000 7f2b61b09640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b5c0cdbf0 0x7f2b5c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.178+0000 7f2b61b09640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c0c6940 con 0x7f2b5c0cd520 2024-04-25T09:54:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.178+0000 7f2b61b09640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c0b0020 con 0x7f2b5c0cdbf0 2024-04-25T09:54:34.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.178+0000 7f2b61b09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0cb5a0 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b52ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b5c0cdbf0 0x7f2b5c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 0x7f2b5c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 0x7f2b5c0c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:43884/0 (socket says 172.21.15.148:43884) 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 -- 172.21.15.148:0/1994098431 learned_addr learned my addr 172.21.15.148:0/1994098431 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 -- 172.21.15.148:0/1994098431 >> v2:172.21.15.188:3300/0 conn(0x7f2b5c0cdbf0 msgr2=0x7f2b5c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 --2- 172.21.15.148:0/1994098431 >> v2:172.21.15.188:3300/0 conn(0x7f2b5c0cdbf0 0x7f2b5c0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 -- 172.21.15.148:0/1994098431 >> v1:172.21.15.188:6789/0 conn(0x7f2b5c0cd520 legacy=0x7f2b5c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.179+0000 7f2b5b7fe640 1 -- 172.21.15.148:0/1994098431 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c15add0 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.180+0000 7f2b5b7fe640 1 --2- 172.21.15.148:0/1994098431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 0x7f2b5c0c6010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009870 tx=0x7f2b4c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607bd89343a2c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.180+0000 7f2b5affd640 1 -- 172.21.15.148:0/1994098431 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c037070 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.180+0000 7f2b5affd640 1 -- 172.21.15.148:0/1994098431 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c032440 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.180+0000 7f2b5affd640 1 -- 172.21.15.148:0/1994098431 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c032810 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 -- 172.21.15.148:0/1994098431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 msgr2=0x7f2b5c0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 --2- 172.21.15.148:0/1994098431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 0x7f2b5c0c6010 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009870 tx=0x7f2b4c02a270 comp rx=0 tx=0).stop 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 -- 172.21.15.148:0/1994098431 shutdown_connections 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 --2- 172.21.15.148:0/1994098431 >> v2:172.21.15.188:3300/0 conn(0x7f2b5c0cdbf0 0x7f2b5c0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 --2- 172.21.15.148:0/1994098431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c15a9f0 0x7f2b5c0c6010 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 -- 172.21.15.148:0/1994098431 >> 172.21.15.148:0/1994098431 conn(0x7f2b5c0b2390 msgr2=0x7f2b5c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 -- 172.21.15.148:0/1994098431 shutdown_connections 2024-04-25T09:54:34.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.181+0000 7f2b61b09640 1 -- 172.21.15.148:0/1994098431 wait complete. 2024-04-25T09:54:34.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.182+0000 7f2b61b09640 1 Processor -- start 2024-04-25T09:54:34.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.182+0000 7f2b61b09640 1 -- start start 2024-04-25T09:54:34.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b61b09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c0cd520 0x7f2b5c14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b61b09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5c0cdbf0 0x7f2b5c159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b61b09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 0x7f2b5c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b61b09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0cb230 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b61b09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0caf50 con 0x7f2b5c0cdbf0 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b61b09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0ce070 con 0x7f2b5c0cd520 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 0x7f2b5c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 0x7f2b5c159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57258/0 (socket says 172.21.15.148:57258) 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 -- 172.21.15.148:0/284985454 learned_addr learned my addr 172.21.15.148:0/284985454 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b52ffd640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5c0cdbf0 0x7f2b5c159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5b7fe640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c0cd520 0x7f2b5c14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 -- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c0cd520 msgr2=0x7f2b5c14d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c0cd520 0x7f2b5c14d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 -- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5c0cdbf0 msgr2=0x7f2b5c159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5c0cdbf0 0x7f2b5c159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 -- 172.21.15.148:0/284985454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c14f9e0 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b5bfff640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 0x7f2b5c159dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c7a0 tx=0x7f2b5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.183+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54003b70 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.184+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b54003d30 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.184+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400aa90 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.184+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c14fcc0 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.184+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c150280 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.184+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c073a00 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.185+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b5400ac30 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.185+0000 7f2b58ff9640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.185+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2b5408a350 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.185+0000 7f2b5b7fe640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.186+0000 7f2b5b7fe640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c034000 tx=0x7f2b4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.189+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b5c0c6940 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.358+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b5c0cfb40 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.359+0000 7f2b58ff9640 1 -- 172.21.15.148:0/284985454 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b54058840 con 0x7f2b5c15a9f0 2024-04-25T09:54:34.361 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:54:34.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b3c06b4d0 msgr2=0x7f2b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c034000 tx=0x7f2b4c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:34.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 msgr2=0x7f2b5c159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 0x7f2b5c159dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400c7a0 tx=0x7f2b5400cc60 comp rx=0 tx=0).stop 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 shutdown_connections 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b3c06b4d0 0x7f2b3c06d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b5c15a9f0 0x7f2b5c159dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5c0cdbf0 0x7f2b5c159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 --2- 172.21.15.148:0/284985454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b5c0cd520 0x7f2b5c14d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 >> 172.21.15.148:0/284985454 conn(0x7f2b5c0b2390 msgr2=0x7f2b5c06fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 shutdown_connections 2024-04-25T09:54:34.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.361+0000 7f2b61b09640 1 -- 172.21.15.148:0/284985454 wait complete. 2024-04-25T09:54:34.374 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:34.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.471+0000 7fc5341b7640 1 Processor -- start 2024-04-25T09:54:34.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.471+0000 7fc5341b7640 1 -- start start 2024-04-25T09:54:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc5341b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc52c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc5341b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0d2420 con 0x7fc52c0d0ab0 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc5341b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0d2700 con 0x7fc52c0d1f30 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc5341b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0d3030 con 0x7fc52c15dac0 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc532754640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc52c0d0ab0 0x7fc52c0b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56972/0 (socket says 172.21.15.148:56972) 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc532754640 1 -- 172.21.15.148:0/1959084063 learned_addr learned my addr 172.21.15.148:0/1959084063 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc531f53640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc52c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259035078 0 0) 0x7fc52c0d2420 con 0x7fc52c0d0ab0 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc518003520 con 0x7fc52c0d0ab0 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708484441 0 0) 0x7fc52c0d2700 con 0x7fc52c0d1f30 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc52c0d2420 con 0x7fc52c0d1f30 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625442507 0 0) 0x7fc518003520 con 0x7fc52c0d0ab0 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 msgr2=0x7fc52c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc52c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 >> v1:172.21.15.188:6789/0 conn(0x7fc52c0d1f30 legacy=0x7fc52c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0d29e0 con 0x7fc52c0d0ab0 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc524001970 con 0x7fc52c0d0ab0 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc531f53640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc52c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 >> v1:172.21.15.148:6789/0 conn(0x7fc52c0d0ab0 legacy=0x7fc52c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518003b70 0x7fc518003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518004800 0x7fc518004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.472+0000 7fc530f51640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc5180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0d3030 con 0x7fc518003b70 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51800ba70 con 0x7fc518004800 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51800bc70 con 0x7fc52c15dac0 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518003b70 0x7fc518003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531752640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518004800 0x7fc518004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc532754640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc5180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 -- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 msgr2=0x7fc5180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c15dac0 0x7fc5180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 -- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518004800 msgr2=0x7fc518004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518004800 0x7fc518004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 -- 172.21.15.148:0/1959084063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518003520 con 0x7fc518003b70 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.473+0000 7fc531f53640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518003b70 0x7fc518003f90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc528009c60 tx=0x7fc518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eec879f59c830c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528036070 con 0x7fc518003b70 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc528031400 con 0x7fc518003b70 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc530f51640 1 -- 172.21.15.148:0/1959084063 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528009860 con 0x7fc518003b70 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc5341b7640 1 -- 172.21.15.148:0/1959084063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518003b70 msgr2=0x7fc518003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc5341b7640 1 --2- 172.21.15.148:0/1959084063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc518003b70 0x7fc518003f90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc528009c60 tx=0x7fc518005830 comp rx=0 tx=0).stop 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc532754640 1 -- 172.21.15.148:0/1959084063 reap_dead start 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc5341b7640 1 -- 172.21.15.148:0/1959084063 shutdown_connections 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc5341b7640 1 -- 172.21.15.148:0/1959084063 >> 172.21.15.148:0/1959084063 conn(0x7fc52c061c30 msgr2=0x7fc52c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc5341b7640 1 -- 172.21.15.148:0/1959084063 shutdown_connections 2024-04-25T09:54:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.474+0000 7fc5341b7640 1 -- 172.21.15.148:0/1959084063 wait complete. 2024-04-25T09:54:34.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.475+0000 7fc5341b7640 1 Processor -- start 2024-04-25T09:54:34.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 -- start start 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518003b70 0x7fc52c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 0x7fc52c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c1f71d0 0x7fc52c1f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0b97d0 con 0x7fc52c0b5520 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c15dea0 con 0x7fc518003b70 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc5341b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0b9950 con 0x7fc52c1f71d0 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc531752640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 0x7fc52c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc531752640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 0x7fc52c0b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57288/0 (socket says 172.21.15.148:57288) 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc531752640 1 -- 172.21.15.148:0/3811881647 learned_addr learned my addr 172.21.15.148:0/3811881647 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc531f53640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518003b70 0x7fc52c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.476+0000 7fc532754640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c1f71d0 0x7fc52c1f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531752640 1 -- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c1f71d0 msgr2=0x7fc52c1f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531752640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c1f71d0 0x7fc52c1f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531752640 1 -- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518003b70 msgr2=0x7fc52c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531752640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518003b70 0x7fc52c0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531752640 1 -- 172.21.15.148:0/3811881647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0b44f0 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531f53640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518003b70 0x7fc52c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc531752640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 0x7fc52c0b59a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00f7d0 tx=0x7fc51c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c01a070 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00fe80 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c0b4760 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c006820 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.477+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c0b4d20 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.478+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c073a00 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.478+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51c01d070 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.479+0000 7fc5227fc640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc51006b500 0x7fc51006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.479+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc51c089cc0 con 0x7fc52c0b5520 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.479+0000 7fc531f53640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc51006b500 0x7fc51006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.479+0000 7fc531f53640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc51006b500 0x7fc51006d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc528009c60 tx=0x7fc528008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.483+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc51c058800 con 0x7fc52c0b5520 2024-04-25T09:54:34.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.651+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc52c15cf50 con 0x7fc52c0b5520 2024-04-25T09:54:34.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.651+0000 7fc5227fc640 1 -- 172.21.15.148:0/3811881647 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc51c058100 con 0x7fc52c0b5520 2024-04-25T09:54:34.654 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc51006b500 msgr2=0x7fc51006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc51006b500 0x7fc51006d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc528009c60 tx=0x7fc528008030 comp rx=0 tx=0).stop 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 msgr2=0x7fc52c0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 0x7fc52c0b59a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00f7d0 tx=0x7fc51c00fc90 comp rx=0 tx=0).stop 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 shutdown_connections 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc51006b500 0x7fc51006d9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c1f71d0 0x7fc52c1f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c0b5520 0x7fc52c0b59a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 --2- 172.21.15.148:0/3811881647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc518003b70 0x7fc52c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 >> 172.21.15.148:0/3811881647 conn(0x7fc52c061c30 msgr2=0x7fc52c0b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 shutdown_connections 2024-04-25T09:54:34.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.653+0000 7fc5341b7640 1 -- 172.21.15.148:0/3811881647 wait complete. 2024-04-25T09:54:34.667 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.763+0000 7fbc00b32640 1 Processor -- start 2024-04-25T09:54:34.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.763+0000 7fbc00b32640 1 -- start start 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbc00b32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d0ab0 0x7fbbfc168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbc00b32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0d2420 con 0x7fbbfc15dac0 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbc00b32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc0d2700 con 0x7fbbfc0d34c0 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbc00b32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0d3030 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbfa59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbbfc15dac0 0x7fbbfc0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56986/0 (socket says 172.21.15.148:56986) 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbfa59c640 1 -- 172.21.15.148:0/2216853015 learned_addr learned my addr 172.21.15.148:0/2216853015 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbfad9d640 1 --2- 172.21.15.148:0/2216853015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d0ab0 0x7fbbfc168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262447870 0 0) 0x7fbbfc0d2420 con 0x7fbbfc15dac0 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe8003520 con 0x7fbbfc15dac0 2024-04-25T09:54:34.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694674913 0 0) 0x7fbbfc0d2700 con 0x7fbbfc0d34c0 2024-04-25T09:54:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbfc0d2420 con 0x7fbbfc0d34c0 2024-04-25T09:54:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbfad9d640 1 -- 172.21.15.148:0/2216853015 >> v1:172.21.15.188:6789/0 conn(0x7fbbfc0d34c0 legacy=0x7fbbfc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955206772 0 0) 0x7fbbe8003520 con 0x7fbbfc15dac0 2024-04-25T09:54:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbfad9d640 1 -- 172.21.15.148:0/2216853015 >> v1:172.21.15.148:6789/0 conn(0x7fbbfc15dac0 legacy=0x7fbbfc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.764+0000 7fbbfad9d640 1 -- 172.21.15.148:0/2216853015 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0d29e0 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.765+0000 7fbbfad9d640 1 --2- 172.21.15.148:0/2216853015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d0ab0 0x7fbbfc168670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0d31b0 tx=0x7fbbf402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb178c3fc1b9084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.765+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec001670 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.765+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf402ae50 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.765+0000 7fbbf9d9b640 1 -- 172.21.15.148:0/2216853015 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4007740 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 -- 172.21.15.148:0/2216853015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d0ab0 msgr2=0x7fbbfc168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2216853015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d0ab0 0x7fbbfc168670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0d31b0 tx=0x7fbbf402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 -- 172.21.15.148:0/2216853015 shutdown_connections 2024-04-25T09:54:34.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2216853015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d0ab0 0x7fbbfc168670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 -- 172.21.15.148:0/2216853015 >> 172.21.15.148:0/2216853015 conn(0x7fbbfc061c30 msgr2=0x7fbbfc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 -- 172.21.15.148:0/2216853015 shutdown_connections 2024-04-25T09:54:34.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.766+0000 7fbc00b32640 1 -- 172.21.15.148:0/2216853015 wait complete. 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.767+0000 7fbc00b32640 1 Processor -- start 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.767+0000 7fbc00b32640 1 -- start start 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbc00b32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbfc0d0ab0 0x7fbbfc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbc00b32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d34c0 0x7fbbfc1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbc00b32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 0x7fbbfc0b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbc00b32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc15dea0 con 0x7fbbfc15dac0 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbc00b32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0d2100 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbc00b32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0b97d0 con 0x7fbbfc0d34c0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 0x7fbbfc0b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbfc0d0ab0 0x7fbbfc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 0x7fbbfc0b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57296/0 (socket says 172.21.15.148:57296) 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfad9d640 1 -- 172.21.15.148:0/2625762103 learned_addr learned my addr 172.21.15.148:0/2625762103 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbf3fff640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d34c0 0x7fbbfc1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfa59c640 1 -- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d34c0 msgr2=0x7fbbfc1568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfa59c640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d34c0 0x7fbbfc1568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfa59c640 1 -- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 msgr2=0x7fbbfc0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfa59c640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 0x7fbbfc0b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.768+0000 7fbbfa59c640 1 -- 172.21.15.148:0/2625762103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc0b3560 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbbf3fff640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d34c0 0x7fbbfc1568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbbfad9d640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 0x7fbbfc0b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbbfa59c640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbfc0d0ab0 0x7fbbfc152aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007fd0 tx=0x7fbbec00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec017070 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec005ce0 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc0b3800 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec007310 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.769+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc0b3dc0 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.770+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc073a00 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.770+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbec01a070 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.771+0000 7fbbf2ffd640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:34.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.771+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbbec089b70 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.771+0000 7fbbf3fff640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:34.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.771+0000 7fbbf3fff640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc1f6c10 tx=0x7fbbe4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:34.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.775+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbec0581b0 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.958+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbbfc0cefa0 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.959+0000 7fbbf2ffd640 1 -- 172.21.15.148:0/2625762103 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbbec05c250 con 0x7fbbfc0d0ab0 2024-04-25T09:54:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:54:34.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbdc06b4d0 msgr2=0x7fbbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc1f6c10 tx=0x7fbbe4009040 comp rx=0 tx=0).stop 2024-04-25T09:54:34.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbfc0d0ab0 msgr2=0x7fbbfc152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:34.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbfc0d0ab0 0x7fbbfc152aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbbec007fd0 tx=0x7fbbec00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 shutdown_connections 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbdc06b4d0 0x7fbbdc06d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbfc15dac0 0x7fbbfc0b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbfc0d34c0 0x7fbbfc1568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.960+0000 7fbc00b32640 1 --2- 172.21.15.148:0/2625762103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbfc0d0ab0 0x7fbbfc152aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.961+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 >> 172.21.15.148:0/2625762103 conn(0x7fbbfc061c30 msgr2=0x7fbbfc0c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.961+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 shutdown_connections 2024-04-25T09:54:34.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:34.961+0000 7fbc00b32640 1 -- 172.21.15.148:0/2625762103 wait complete. 2024-04-25T09:54:34.974 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:54:34.974 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:34.989 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.126+0000 7f4c6f5f4640 1 Processor -- start 2024-04-25T09:54:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.126+0000 7f4c6f5f4640 1 -- start start 2024-04-25T09:54:35.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6f5f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c680afc90 0x7f4c680ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6f5f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680cef80 con 0x7f4c680c36d0 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6f5f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6806f880 con 0x7f4c6806f3c0 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6f5f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680cf220 con 0x7f4c680afc90 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6db91640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c680afc90 0x7f4c680ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6d390640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4c680c36d0 0x7f4c680c3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56998/0 (socket says 172.21.15.148:56998) 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6d390640 1 -- 172.21.15.148:0/1274634386 learned_addr learned my addr 172.21.15.148:0/1274634386 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6db91640 1 -- 172.21.15.148:0/1274634386 >> v1:172.21.15.188:6789/0 conn(0x7f4c6806f3c0 legacy=0x7f4c680cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6db91640 1 -- 172.21.15.148:0/1274634386 >> v1:172.21.15.148:6789/0 conn(0x7f4c680c36d0 legacy=0x7f4c680c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c6db91640 1 -- 172.21.15.148:0/1274634386 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680cf500 con 0x7f4c680afc90 2024-04-25T09:54:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.127+0000 7f4c5ffff640 1 -- 172.21.15.148:0/1274634386 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308400500 0 0) 0x7f4c680cef80 con 0x7f4c680c36d0 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c6db91640 1 --2- 172.21.15.148:0/1274634386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680afc90 0x7f4c680ad570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009dc0 tx=0x7f4c6002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82df1e2d80f25370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c5ffff640 1 -- 172.21.15.148:0/1274634386 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007740 con 0x7f4c680afc90 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c5ffff640 1 -- 172.21.15.148:0/1274634386 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c600078e0 con 0x7f4c680afc90 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c5ffff640 1 -- 172.21.15.148:0/1274634386 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007cb0 con 0x7f4c680afc90 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/1274634386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680afc90 msgr2=0x7f4c680ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/1274634386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680afc90 0x7f4c680ad570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009dc0 tx=0x7f4c6002a640 comp rx=0 tx=0).stop 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/1274634386 shutdown_connections 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/1274634386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680afc90 0x7f4c680ad570 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.128+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/1274634386 >> 172.21.15.148:0/1274634386 conn(0x7f4c680b2290 msgr2=0x7f4c680b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.129+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/1274634386 shutdown_connections 2024-04-25T09:54:35.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.129+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/1274634386 wait complete. 2024-04-25T09:54:35.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 Processor -- start 2024-04-25T09:54:35.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 -- start start 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6806f3c0 0x7f4c6815eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c680afc90 0x7f4c680cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 0x7f4c680b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6806fce0 con 0x7f4c680c36d0 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6806fb60 con 0x7f4c680afc90 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6f5f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681657f0 con 0x7f4c6806f3c0 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6db91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 0x7f4c680b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6db91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 0x7f4c680b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57312/0 (socket says 172.21.15.148:57312) 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.130+0000 7f4c6db91640 1 -- 172.21.15.148:0/721390957 learned_addr learned my addr 172.21.15.148:0/721390957 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6cb8f640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c680afc90 0x7f4c680cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6d390640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6806f3c0 0x7f4c6815eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6db91640 1 -- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6806f3c0 msgr2=0x7f4c6815eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6db91640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6806f3c0 0x7f4c6815eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6db91640 1 -- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c680afc90 msgr2=0x7f4c680cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6db91640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c680afc90 0x7f4c680cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6db91640 1 -- 172.21.15.148:0/721390957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680cbd40 con 0x7f4c680c36d0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6db91640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 0x7f4c680b54a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009ee0 tx=0x7f4c60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60040070 con 0x7f4c680c36d0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c60031bc0 con 0x7f4c680c36d0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60045460 con 0x7f4c680c36d0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680cc020 con 0x7f4c680c36d0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.131+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680cc5e0 con 0x7f4c680c36d0 2024-04-25T09:54:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.132+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68073990 con 0x7f4c680c36d0 2024-04-25T09:54:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.133+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c60039040 con 0x7f4c680c36d0 2024-04-25T09:54:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.133+0000 7f4c5dffb640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4c06b500 0x7f4c4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.133+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c600abd20 con 0x7f4c680c36d0 2024-04-25T09:54:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.134+0000 7f4c6d390640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4c06b500 0x7f4c4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.134+0000 7f4c6d390640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4c06b500 0x7f4c4c06d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c64007fd0 tx=0x7f4c64007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.138+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c6002ae20 con 0x7f4c680c36d0 2024-04-25T09:54:35.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.309+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c68161900 con 0x7f4c680c36d0 2024-04-25T09:54:35.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.310+0000 7f4c5dffb640 1 -- 172.21.15.148:0/721390957 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f4c6007a360 con 0x7f4c680c36d0 2024-04-25T09:54:35.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:35.313 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:35.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.312+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4c06b500 msgr2=0x7f4c4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.312+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4c06b500 0x7f4c4c06d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c64007fd0 tx=0x7f4c64007de0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.312+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 msgr2=0x7f4c680b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.312+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 0x7f4c680b54a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009ee0 tx=0x7f4c60004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:35.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 shutdown_connections 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4c06b500 0x7f4c4c06d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c680c36d0 0x7f4c680b54a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c680afc90 0x7f4c680cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 --2- 172.21.15.148:0/721390957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6806f3c0 0x7f4c6815eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 >> 172.21.15.148:0/721390957 conn(0x7f4c680b2290 msgr2=0x7f4c681623b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 shutdown_connections 2024-04-25T09:54:35.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.313+0000 7f4c6f5f4640 1 -- 172.21.15.148:0/721390957 wait complete. 2024-04-25T09:54:35.326 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:35.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:35.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.424+0000 7f734fb9e640 1 Processor -- start 2024-04-25T09:54:35.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.424+0000 7f734fb9e640 1 -- start start 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734fb9e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f734815dc20 0x7f73480ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734fb9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734806cda0 con 0x7f7348161710 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734fb9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734815e000 con 0x7f73480d32f0 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734fb9e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73480d1de0 con 0x7f734815dc20 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734d139640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7348161710 0x7f734806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57004/0 (socket says 172.21.15.148:57004) 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734d139640 1 -- 172.21.15.148:0/2479322214 learned_addr learned my addr 172.21.15.148:0/2479322214 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:35.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734d93a640 1 --2- 172.21.15.148:0/2479322214 >> v2:172.21.15.188:3300/0 conn(0x7f734815dc20 0x7f73480ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560995904 0 0) 0x7f734806cda0 con 0x7f7348161710 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.425+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7334003520 con 0x7f7348161710 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4290346666 0 0) 0x7f7334003520 con 0x7f7348161710 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 >> v2:172.21.15.188:3300/0 conn(0x7f734815dc20 msgr2=0x7f73480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 --2- 172.21.15.148:0/2479322214 >> v2:172.21.15.188:3300/0 conn(0x7f734815dc20 0x7f73480ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 >> v1:172.21.15.188:6789/0 conn(0x7f73480d32f0 legacy=0x7f734806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7348161c60 con 0x7f7348161710 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7338001670 con 0x7f7348161710 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 >> v1:172.21.15.148:6789/0 conn(0x7f7348161710 legacy=0x7f734806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7334004800 0x7f7334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73340051c0 0x7f733400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480d1de0 con 0x7f73340051c0 2024-04-25T09:54:35.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733400c060 con 0x7f73340039b0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733400c260 con 0x7f7334004800 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734e13b640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73340051c0 0x7f733400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734d93a640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734d139640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7334004800 0x7f7334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734e13b640 1 -- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7334004800 msgr2=0x7f7334004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734e13b640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7334004800 0x7f7334004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734e13b640 1 -- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340039b0 msgr2=0x7f7334003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734e13b640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.426+0000 7f734e13b640 1 -- 172.21.15.148:0/2479322214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334003520 con 0x7f73340051c0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.427+0000 7f734e13b640 1 -- 172.21.15.148:0/2479322214 reap_dead start 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.427+0000 7f734e13b640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73340051c0 0x7f733400b9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7340001720 tx=0x7f7334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ca5be7178aefd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.427+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340001370 con 0x7f73340051c0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.427+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73400052d0 con 0x7f73340051c0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.427+0000 7f734c938640 1 -- 172.21.15.148:0/2479322214 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734002abc0 con 0x7f73340051c0 2024-04-25T09:54:35.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 -- 172.21.15.148:0/2479322214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73340051c0 msgr2=0x7f733400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73340051c0 0x7f733400b9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7340001720 tx=0x7f7334005830 comp rx=0 tx=0).stop 2024-04-25T09:54:35.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 -- 172.21.15.148:0/2479322214 shutdown_connections 2024-04-25T09:54:35.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2479322214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73340051c0 0x7f733400b9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 -- 172.21.15.148:0/2479322214 >> 172.21.15.148:0/2479322214 conn(0x7f73480d0420 msgr2=0x7f73480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:35.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 -- 172.21.15.148:0/2479322214 shutdown_connections 2024-04-25T09:54:35.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.428+0000 7f734fb9e640 1 -- 172.21.15.148:0/2479322214 wait complete. 2024-04-25T09:54:35.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.429+0000 7f734fb9e640 1 Processor -- start 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.429+0000 7f734fb9e640 1 -- start start 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.429+0000 7f734fb9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340051c0 0x7f73480c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.429+0000 7f734fb9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 0x7f73480caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.429+0000 7f734fb9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480b39c0 0x7f73480b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734fb9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348162500 con 0x7f73481f2d80 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734fb9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7348162220 con 0x7f73480b39c0 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734fb9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480b5e50 con 0x7f73340051c0 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 0x7f73480caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 0x7f73480caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57330/0 (socket says 172.21.15.148:57330) 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 -- 172.21.15.148:0/2245271188 learned_addr learned my addr 172.21.15.148:0/2245271188 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734e13b640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480b39c0 0x7f73480b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d93a640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340051c0 0x7f73480c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 -- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340051c0 msgr2=0x7f73480c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340051c0 0x7f73480c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 -- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480b39c0 msgr2=0x7f73480b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480b39c0 0x7f73480b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 -- 172.21.15.148:0/2245271188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73481f3200 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734d139640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 0x7f73480caae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73380072f0 tx=0x7f73380077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733801a070 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800f8e0 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73480ccff0 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800fcb0 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.430+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73480cd5b0 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.431+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7348073a00 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.432+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f733801d070 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.432+0000 7f733e7fc640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732c06b4d0 0x7f732c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.432+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7338089f80 con 0x7f73481f2d80 2024-04-25T09:54:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.432+0000 7f734d93a640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732c06b4d0 0x7f732c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.432+0000 7f734d93a640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732c06b4d0 0x7f732c06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7344009fd0 tx=0x7f734400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.436+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7338058ac0 con 0x7f73481f2d80 2024-04-25T09:54:35.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.605+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73480c74d0 con 0x7f73481f2d80 2024-04-25T09:54:35.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.606+0000 7f733e7fc640 1 -- 172.21.15.148:0/2245271188 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73380583c0 con 0x7f73481f2d80 2024-04-25T09:54:35.608 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:54:35.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732c06b4d0 msgr2=0x7f732c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732c06b4d0 0x7f732c06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7344009fd0 tx=0x7f734400b040 comp rx=0 tx=0).stop 2024-04-25T09:54:35.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 msgr2=0x7f73480caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 0x7f73480caae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73380072f0 tx=0x7f73380077b0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 shutdown_connections 2024-04-25T09:54:35.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732c06b4d0 0x7f732c06d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480b39c0 0x7f73480b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73481f2d80 0x7f73480caae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 --2- 172.21.15.148:0/2245271188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73340051c0 0x7f73480c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 >> 172.21.15.148:0/2245271188 conn(0x7f73480d0420 msgr2=0x7f73480d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 shutdown_connections 2024-04-25T09:54:35.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.608+0000 7f734fb9e640 1 -- 172.21.15.148:0/2245271188 wait complete. 2024-04-25T09:54:35.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:35.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 Processor -- start 2024-04-25T09:54:35.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 -- start start 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d640cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d640ccfe0 0x7f3d640cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640afc80 con 0x7f3d6415de80 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640cb210 con 0x7f3d640ccfe0 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d697e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640ce4d0 con 0x7f3d640cadb0 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.719+0000 7f3d637fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d640ccfe0 0x7f3d640cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d62ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3d6415de80 0x7f3d640cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57016/0 (socket says 172.21.15.148:57016) 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d62ffd640 1 -- 172.21.15.148:0/619276251 learned_addr learned my addr 172.21.15.148:0/619276251 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d627fc640 1 --2- 172.21.15.148:0/619276251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d640cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d637fe640 1 -- 172.21.15.148:0/619276251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 msgr2=0x7f3d640cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d637fe640 1 --2- 172.21.15.148:0/619276251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d640cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d637fe640 1 -- 172.21.15.148:0/619276251 >> v1:172.21.15.148:6789/0 conn(0x7f3d6415de80 legacy=0x7f3d640cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d637fe640 1 -- 172.21.15.148:0/619276251 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640ce7b0 con 0x7f3d640ccfe0 2024-04-25T09:54:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d61ffb640 1 -- 172.21.15.148:0/619276251 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364386526 0 0) 0x7f3d640afc80 con 0x7f3d6415de80 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.720+0000 7f3d637fe640 1 --2- 172.21.15.148:0/619276251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d640ccfe0 0x7f3d640cd470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0097a0 tx=0x7f3d5c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4874bfc0b26b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d61ffb640 1 -- 172.21.15.148:0/619276251 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c037070 con 0x7f3d640ccfe0 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d61ffb640 1 -- 172.21.15.148:0/619276251 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c032440 con 0x7f3d640ccfe0 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d61ffb640 1 -- 172.21.15.148:0/619276251 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c032810 con 0x7f3d640ccfe0 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 -- 172.21.15.148:0/619276251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d640ccfe0 msgr2=0x7f3d640cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 --2- 172.21.15.148:0/619276251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d640ccfe0 0x7f3d640cd470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0097a0 tx=0x7f3d5c02a180 comp rx=0 tx=0).stop 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 -- 172.21.15.148:0/619276251 shutdown_connections 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 --2- 172.21.15.148:0/619276251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d640ccfe0 0x7f3d640cd470 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 --2- 172.21.15.148:0/619276251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d640cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 -- 172.21.15.148:0/619276251 >> 172.21.15.148:0/619276251 conn(0x7f3d640b2390 msgr2=0x7f3d640b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 -- 172.21.15.148:0/619276251 shutdown_connections 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.721+0000 7f3d697e0640 1 -- 172.21.15.148:0/619276251 wait complete. 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.722+0000 7f3d697e0640 1 Processor -- start 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 -- start start 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d64157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d640ccfe0 0x7f3d64151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 0x7f3d640c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6415e260 con 0x7f3d6415de80 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640cfc00 con 0x7f3d640ccfe0 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d697e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b2eb0 con 0x7f3d640cadb0 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 0x7f3d640c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 0x7f3d640c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57352/0 (socket says 172.21.15.148:57352) 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d637fe640 1 -- 172.21.15.148:0/1831650487 learned_addr learned my addr 172.21.15.148:0/1831650487 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d62ffd640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d64157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.723+0000 7f3d627fc640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d640ccfe0 0x7f3d64151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d637fe640 1 -- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 msgr2=0x7f3d64157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d637fe640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d64157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d637fe640 1 -- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d640ccfe0 msgr2=0x7f3d64151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d637fe640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d640ccfe0 0x7f3d64151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d637fe640 1 -- 172.21.15.148:0/1831650487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d641590a0 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d637fe640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 0x7f3d640c9e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0098c0 tx=0x7f3d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c037040 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c042070 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c030b50 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d64159380 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.724+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d640ca380 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.725+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d64073a00 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.725+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d5c030cf0 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.726+0000 7f3d4b7fe640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d5006b4d0 0x7f3d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.726+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d5c0ab060 con 0x7f3d6415de80 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.726+0000 7f3d62ffd640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d5006b4d0 0x7f3d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:35.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.726+0000 7f3d62ffd640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d5006b4d0 0x7f3d5006d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d540012e0 tx=0x7f3d54005eb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:35.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.730+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d5c032e20 con 0x7f3d6415de80 2024-04-25T09:54:35.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.901+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d640c6d00 con 0x7f3d6415de80 2024-04-25T09:54:35.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.901+0000 7f3d4b7fe640 1 -- 172.21.15.148:0/1831650487 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d5c045360 con 0x7f3d6415de80 2024-04-25T09:54:35.904 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:35.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.903+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d5006b4d0 msgr2=0x7f3d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.903+0000 7f3d697e0640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d5006b4d0 0x7f3d5006d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d540012e0 tx=0x7f3d54005eb0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.903+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 msgr2=0x7f3d640c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.903+0000 7f3d697e0640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 0x7f3d640c9e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0098c0 tx=0x7f3d5c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 shutdown_connections 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d5006b4d0 0x7f3d5006d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d6415de80 0x7f3d640c9e40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d640ccfe0 0x7f3d64151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 --2- 172.21.15.148:0/1831650487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d640cadb0 0x7f3d64157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 >> 172.21.15.148:0/1831650487 conn(0x7f3d640b2390 msgr2=0x7f3d640cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 shutdown_connections 2024-04-25T09:54:35.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:35.904+0000 7f3d697e0640 1 -- 172.21.15.148:0/1831650487 wait complete. 2024-04-25T09:54:35.917 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.016+0000 7f3df9b68640 1 Processor -- start 2024-04-25T09:54:36.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.016+0000 7f3df9b68640 1 -- start start 2024-04-25T09:54:36.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df9b68640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 0x7f3df40cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df9b68640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40afc80 con 0x7f3df415de80 2024-04-25T09:54:36.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df9b68640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40ce4d0 con 0x7f3df40ccfe0 2024-04-25T09:54:36.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df9b68640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40cb210 con 0x7f3df40cadb0 2024-04-25T09:54:36.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3df415de80 0x7f3df40cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57022/0 (socket says 172.21.15.148:57022) 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df37fe640 1 -- 172.21.15.148:0/3956443433 learned_addr learned my addr 172.21.15.148:0/3956443433 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.017+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 0x7f3df40cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237591501 0 0) 0x7f3df40afc80 con 0x7f3df415de80 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de0003520 con 0x7f3df415de80 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3849283869 0 0) 0x7f3de0003520 con 0x7f3df415de80 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 msgr2=0x7f3df40cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 0x7f3df40cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 >> v1:172.21.15.188:6789/0 conn(0x7f3df40ccfe0 legacy=0x7f3df40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40ce7b0 con 0x7f3df415de80 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3de40019f0 con 0x7f3df415de80 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 >> v1:172.21.15.148:6789/0 conn(0x7f3df415de80 legacy=0x7f3df40cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de00039e0 0x7f3de0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 0x7f3de0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3de0004d40 0x7f3de00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40cb210 con 0x7f3de0004d40 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de000baa0 con 0x7f3de00039e0 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de000bca0 con 0x7f3df40cadb0 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3de0004d40 0x7f3de00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de00039e0 0x7f3de0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 0x7f3de0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df3fff640 1 -- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 msgr2=0x7f3de0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df40cadb0 0x7f3de0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df3fff640 1 -- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de00039e0 msgr2=0x7f3de0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de00039e0 0x7f3de0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.018+0000 7f3df3fff640 1 -- 172.21.15.148:0/3956443433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de0003520 con 0x7f3de0004d40 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.019+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3de0004d40 0x7f3de00051c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3dec001720 tx=0x7f3de0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08368ca449eac49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.019+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec001370 con 0x7f3de0004d40 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.019+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec0052d0 con 0x7f3de0004d40 2024-04-25T09:54:36.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.019+0000 7f3df27fc640 1 -- 172.21.15.148:0/3956443433 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec02abc0 con 0x7f3de0004d40 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df9b68640 1 -- 172.21.15.148:0/3956443433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3de0004d40 msgr2=0x7f3de00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3956443433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3de0004d40 0x7f3de00051c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3dec001720 tx=0x7f3de0005830 comp rx=0 tx=0).stop 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df3fff640 1 -- 172.21.15.148:0/3956443433 reap_dead start 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df9b68640 1 -- 172.21.15.148:0/3956443433 shutdown_connections 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df9b68640 1 -- 172.21.15.148:0/3956443433 >> 172.21.15.148:0/3956443433 conn(0x7f3df40b2390 msgr2=0x7f3df40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df9b68640 1 -- 172.21.15.148:0/3956443433 shutdown_connections 2024-04-25T09:54:36.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.020+0000 7f3df9b68640 1 -- 172.21.15.148:0/3956443433 wait complete. 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.021+0000 7f3df9b68640 1 Processor -- start 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.021+0000 7f3df9b68640 1 -- start start 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de0004d40 0x7f3df40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 0x7f3df406f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df406f9a0 0x7f3df40c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b2eb0 con 0x7f3df4158190 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df415e260 con 0x7f3de0004d40 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b3030 con 0x7f3df406f9a0 2024-04-25T09:54:36.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 0x7f3df406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 0x7f3df406f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57362/0 (socket says 172.21.15.148:57362) 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 -- 172.21.15.148:0/3589674385 learned_addr learned my addr 172.21.15.148:0/3589674385 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de0004d40 0x7f3df40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df406f9a0 0x7f3df40c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 -- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df406f9a0 msgr2=0x7f3df40c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df406f9a0 0x7f3df40c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 -- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de0004d40 msgr2=0x7f3df40b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de0004d40 0x7f3df40b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 -- 172.21.15.148:0/3589674385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df41f6a60 con 0x7f3df4158190 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df3fff640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df406f9a0 0x7f3df40c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df2ffd640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 0x7f3df406f460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0119a0 tx=0x7f3ddc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00faf0 con 0x7f3df4158190 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc00fc90 con 0x7f3df4158190 2024-04-25T09:54:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41f6d40 con 0x7f3df4158190 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc014440 con 0x7f3df4158190 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.022+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41f7300 con 0x7f3df4158190 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.023+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df406fdb0 con 0x7f3df4158190 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.024+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ddc012040 con 0x7f3df4158190 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.024+0000 7f3debfff640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd406b4d0 0x7f3dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.024+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd406b4d0 0x7f3dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.024+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ddc08abe0 con 0x7f3df4158190 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.025+0000 7f3df37fe640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd406b4d0 0x7f3dd406d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3df40ca830 tx=0x7f3de4006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.028+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3df406fdb0 con 0x7f3df4158190 2024-04-25T09:54:36.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.206+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3df4073a00 con 0x7f3df4158190 2024-04-25T09:54:36.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.206+0000 7f3debfff640 1 -- 172.21.15.148:0/3589674385 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ddc019030 con 0x7f3df4158190 2024-04-25T09:54:36.209 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:54:36.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.208+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd406b4d0 msgr2=0x7f3dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.208+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd406b4d0 0x7f3dd406d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3df40ca830 tx=0x7f3de4006040 comp rx=0 tx=0).stop 2024-04-25T09:54:36.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.208+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 msgr2=0x7f3df406f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.208+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 0x7f3df406f460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0119a0 tx=0x7f3ddc011e60 comp rx=0 tx=0).stop 2024-04-25T09:54:36.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 shutdown_connections 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd406b4d0 0x7f3dd406d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3df406f9a0 0x7f3df40c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3df4158190 0x7f3df406f460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 --2- 172.21.15.148:0/3589674385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3de0004d40 0x7f3df40b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 >> 172.21.15.148:0/3589674385 conn(0x7f3df40b2390 msgr2=0x7f3df415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 shutdown_connections 2024-04-25T09:54:36.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.209+0000 7f3df9b68640 1 -- 172.21.15.148:0/3589674385 wait complete. 2024-04-25T09:54:36.222 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:36.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 Processor -- start 2024-04-25T09:54:36.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 -- start start 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc718161890 0x7fc7180b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc7180b54a0 0x7fc7180b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7180cf1d0 con 0x7fc718161240 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180b7120 con 0x7fc718161890 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.320+0000 7fc71e66b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180b5e30 con 0x7fc7180b54a0 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc718161890 0x7fc7180b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc718161890 0x7fc7180b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57378/0 (socket says 172.21.15.148:57378) 2024-04-25T09:54:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 -- 172.21.15.148:0/455552464 learned_addr learned my addr 172.21.15.148:0/455552464 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc71cc08640 1 --2- 172.21.15.148:0/455552464 >> v2:172.21.15.188:3300/0 conn(0x7fc7180b54a0 0x7fc7180b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 -- 172.21.15.148:0/455552464 >> v2:172.21.15.188:3300/0 conn(0x7fc7180b54a0 msgr2=0x7fc7180b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 --2- 172.21.15.148:0/455552464 >> v2:172.21.15.188:3300/0 conn(0x7fc7180b54a0 0x7fc7180b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 -- 172.21.15.148:0/455552464 >> v1:172.21.15.188:6789/0 conn(0x7fc718161240 legacy=0x7fc7180ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.321+0000 7fc7177fe640 1 -- 172.21.15.148:0/455552464 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718161620 con 0x7fc718161890 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc7177fe640 1 --2- 172.21.15.148:0/455552464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161890 0x7fc7180b6be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc71808ce10 tx=0x7fc70802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94ac6fb9373a693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc716ffd640 1 -- 172.21.15.148:0/455552464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708037070 con 0x7fc718161890 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc716ffd640 1 -- 172.21.15.148:0/455552464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708032440 con 0x7fc718161890 2024-04-25T09:54:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc716ffd640 1 -- 172.21.15.148:0/455552464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708032810 con 0x7fc718161890 2024-04-25T09:54:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 -- 172.21.15.148:0/455552464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161890 msgr2=0x7fc7180b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 --2- 172.21.15.148:0/455552464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161890 0x7fc7180b6be0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc71808ce10 tx=0x7fc70802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 -- 172.21.15.148:0/455552464 shutdown_connections 2024-04-25T09:54:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 --2- 172.21.15.148:0/455552464 >> v2:172.21.15.188:3300/0 conn(0x7fc7180b54a0 0x7fc7180b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 --2- 172.21.15.148:0/455552464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161890 0x7fc7180b6be0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 -- 172.21.15.148:0/455552464 >> 172.21.15.148:0/455552464 conn(0x7fc7180d04a0 msgr2=0x7fc7180d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 -- 172.21.15.148:0/455552464 shutdown_connections 2024-04-25T09:54:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.322+0000 7fc71e66b640 1 -- 172.21.15.148:0/455552464 wait complete. 2024-04-25T09:54:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.323+0000 7fc71e66b640 1 Processor -- start 2024-04-25T09:54:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 -- start start 2024-04-25T09:54:36.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7180b54a0 0x7fc7181f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 0x7fc7180cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc718161890 0x7fc7180c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181623c0 con 0x7fc718161240 2024-04-25T09:54:36.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181620e0 con 0x7fc718161890 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc71e66b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181626a0 con 0x7fc7180b54a0 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc7177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 0x7fc7180cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc7177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 0x7fc7180cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57386/0 (socket says 172.21.15.148:57386) 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.324+0000 7fc7177fe640 1 -- 172.21.15.148:0/103560304 learned_addr learned my addr 172.21.15.148:0/103560304 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc71cc08640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc718161890 0x7fc7180c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc717fff640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7180b54a0 0x7fc7181f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc7177fe640 1 -- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7180b54a0 msgr2=0x7fc7181f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc7177fe640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7180b54a0 0x7fc7181f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc7177fe640 1 -- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc718161890 msgr2=0x7fc7180c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc7177fe640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc718161890 0x7fc7180c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc7177fe640 1 -- 172.21.15.148:0/103560304 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180d1960 con 0x7fc718161240 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc71cc08640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc718161890 0x7fc7180c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc7177fe640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 0x7fc7180cd2d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc70802a2b0 tx=0x7fc7080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708037040 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708042070 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708030880 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7181f3ee0 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.325+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7181f44a0 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.326+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7180d1b30 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.327+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc708030a20 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.327+0000 7fc714ff9640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc70406b4d0 0x7fc70406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.327+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc7080aad20 con 0x7fc718161240 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.327+0000 7fc717fff640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc70406b4d0 0x7fc70406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.327+0000 7fc717fff640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc70406b4d0 0x7fc70406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc70c007fd0 tx=0x7fc70c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.331+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7080798f0 con 0x7fc718161240 2024-04-25T09:54:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.501+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc718073a00 con 0x7fc718161240 2024-04-25T09:54:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.502+0000 7fc714ff9640 1 -- 172.21.15.148:0/103560304 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc7080791f0 con 0x7fc718161240 2024-04-25T09:54:36.504 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.503+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc70406b4d0 msgr2=0x7fc70406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.503+0000 7fc71e66b640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc70406b4d0 0x7fc70406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc70c007fd0 tx=0x7fc70c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.503+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 msgr2=0x7fc7180cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.503+0000 7fc71e66b640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 0x7fc7180cd2d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc70802a2b0 tx=0x7fc7080046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.503+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 shutdown_connections 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.503+0000 7fc71e66b640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc70406b4d0 0x7fc70406d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.504+0000 7fc71e66b640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc718161890 0x7fc7180c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.504+0000 7fc71e66b640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc718161240 0x7fc7180cd2d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.504+0000 7fc71e66b640 1 --2- 172.21.15.148:0/103560304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7180b54a0 0x7fc7181f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.504+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 >> 172.21.15.148:0/103560304 conn(0x7fc7180d04a0 msgr2=0x7fc7180b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.504+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 shutdown_connections 2024-04-25T09:54:36.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.504+0000 7fc71e66b640 1 -- 172.21.15.148:0/103560304 wait complete. 2024-04-25T09:54:36.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:54:36.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.614+0000 7f2268eb3640 1 Processor -- start 2024-04-25T09:54:36.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.614+0000 7f2268eb3640 1 -- start start 2024-04-25T09:54:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f2268eb3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22640b5620 0x7f22640b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f2268eb3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226406cf50 con 0x7f226415b7d0 2024-04-25T09:54:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f2268eb3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22640ce530 con 0x7f226406c2d0 2024-04-25T09:54:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f2268eb3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22640ce810 con 0x7f22640b5620 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f226259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f226415b7d0 0x7f22640cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57030/0 (socket says 172.21.15.148:57030) 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f225bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22640b5620 0x7f22640b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f226259c640 1 -- 172.21.15.148:0/3779368472 learned_addr learned my addr 172.21.15.148:0/3779368472 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f2261d9b640 1 -- 172.21.15.148:0/3779368472 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433115617 0 0) 0x7f226406cf50 con 0x7f226415b7d0 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f2261d9b640 1 -- 172.21.15.148:0/3779368472 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2250003520 con 0x7f226415b7d0 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f225bfff640 1 -- 172.21.15.148:0/3779368472 >> v1:172.21.15.188:6789/0 conn(0x7f226406c2d0 legacy=0x7f226406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f225bfff640 1 -- 172.21.15.148:0/3779368472 >> v1:172.21.15.148:6789/0 conn(0x7f226415b7d0 legacy=0x7f22640cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.615+0000 7f225bfff640 1 -- 172.21.15.148:0/3779368472 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22640ceaf0 con 0x7f22640b5620 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.616+0000 7f225bfff640 1 --2- 172.21.15.148:0/3779368472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f22640b5ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f224c0097a0 tx=0x7f224c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3a8af6be08ff6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.616+0000 7f2261d9b640 1 -- 172.21.15.148:0/3779368472 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c037070 con 0x7f22640b5620 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.616+0000 7f2261d9b640 1 -- 172.21.15.148:0/3779368472 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c032440 con 0x7f22640b5620 2024-04-25T09:54:36.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.616+0000 7f2261d9b640 1 -- 172.21.15.148:0/3779368472 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c032810 con 0x7f22640b5620 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.616+0000 7f2268eb3640 1 -- 172.21.15.148:0/3779368472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 msgr2=0x7f22640b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.616+0000 7f2268eb3640 1 --2- 172.21.15.148:0/3779368472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f22640b5ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f224c0097a0 tx=0x7f224c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.617+0000 7f2268eb3640 1 -- 172.21.15.148:0/3779368472 shutdown_connections 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.617+0000 7f2268eb3640 1 --2- 172.21.15.148:0/3779368472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f22640b5ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.617+0000 7f2268eb3640 1 -- 172.21.15.148:0/3779368472 >> 172.21.15.148:0/3779368472 conn(0x7f22640d0800 msgr2=0x7f22640d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.617+0000 7f2268eb3640 1 -- 172.21.15.148:0/3779368472 shutdown_connections 2024-04-25T09:54:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.617+0000 7f2268eb3640 1 -- 172.21.15.148:0/3779368472 wait complete. 2024-04-25T09:54:36.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.618+0000 7f2268eb3640 1 Processor -- start 2024-04-25T09:54:36.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.618+0000 7f2268eb3640 1 -- start start 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226406c2d0 0x7f22640b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f2264158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226415b7d0 0x7f22640c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226415bd30 con 0x7f22640b5620 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226415bbb0 con 0x7f226406c2d0 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22640d1f40 con 0x7f226415b7d0 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f2264158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f2264158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57404/0 (socket says 172.21.15.148:57404) 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 -- 172.21.15.148:0/2054903755 learned_addr learned my addr 172.21.15.148:0/2054903755 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f226259c640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226406c2d0 0x7f22640b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 -- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226415b7d0 msgr2=0x7f22640c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2262d9d640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226415b7d0 0x7f22640c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226415b7d0 0x7f22640c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 -- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226406c2d0 msgr2=0x7f22640b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226406c2d0 0x7f22640b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 -- 172.21.15.148:0/2054903755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22640b7490 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f226259c640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226406c2d0 0x7f22640b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225bfff640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f2264158e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f224c0098c0 tx=0x7f224c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c037040 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c042070 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22640b7770 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c030940 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.619+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22640b7d30 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.620+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226415f030 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.621+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f224c030ae0 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.621+0000 7f225affd640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f224406b4d0 0x7f224406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.621+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f224c0aae70 con 0x7f22640b5620 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.621+0000 7f226259c640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f224406b4d0 0x7f224406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:36.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.622+0000 7f226259c640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f224406b4d0 0x7f224406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2254005880 tx=0x7f2254009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:36.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.625+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f226415f030 con 0x7f22640b5620 2024-04-25T09:54:36.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.810+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f226406b210 con 0x7f22640b5620 2024-04-25T09:54:36.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.810+0000 7f225affd640 1 -- 172.21.15.148:0/2054903755 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f224c045360 con 0x7f22640b5620 2024-04-25T09:54:36.813 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f224406b4d0 msgr2=0x7f224406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f224406b4d0 0x7f224406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2254005880 tx=0x7f2254009040 comp rx=0 tx=0).stop 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 msgr2=0x7f2264158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f2264158e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f224c0098c0 tx=0x7f224c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 shutdown_connections 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f224406b4d0 0x7f224406d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f226415b7d0 0x7f22640c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22640b5620 0x7f2264158e40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 --2- 172.21.15.148:0/2054903755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f226406c2d0 0x7f22640b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 >> 172.21.15.148:0/2054903755 conn(0x7f22640d0800 msgr2=0x7f226415afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 shutdown_connections 2024-04-25T09:54:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:36.812+0000 7f2268eb3640 1 -- 172.21.15.148:0/2054903755 wait complete. 2024-04-25T09:54:36.825 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi148.front.sepia.ceph.com... 2024-04-25T09:54:36.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:36.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T09:54:36.848 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi148.stderr:ceph-fuse[42142]: fuse finished with error 0 and tester_r 0 2024-04-25T09:54:36.849 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:54:42.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:54:42.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:42.951 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T09:54:42.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:43.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.121+0000 7ff84b646640 1 Processor -- start 2024-04-25T09:54:43.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.121+0000 7ff84b646640 1 -- start start 2024-04-25T09:54:43.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff84b646640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff844164970 0x7ff844166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff84b646640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff844144b30 con 0x7ff8440ce920 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff84b646640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8440712d0 con 0x7ff844144750 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff84b646640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440715b0 con 0x7ff844164970 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff848be1640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff8440ce920 0x7ff8440cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57032/0 (socket says 172.21.15.148:57032) 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff848be1640 1 -- 172.21.15.148:0/3828843759 learned_addr learned my addr 172.21.15.148:0/3828843759 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173788170 0 0) 0x7ff844144b30 con 0x7ff8440ce920 2024-04-25T09:54:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff849be3640 1 --2- 172.21.15.148:0/3828843759 >> v2:172.21.15.188:3300/0 conn(0x7ff844164970 0x7ff844166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.122+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff830003520 con 0x7ff8440ce920 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589611462 0 0) 0x7ff830003520 con 0x7ff8440ce920 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 >> v2:172.21.15.188:3300/0 conn(0x7ff844164970 msgr2=0x7ff844166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 --2- 172.21.15.148:0/3828843759 >> v2:172.21.15.188:3300/0 conn(0x7ff844164970 0x7ff844166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 >> v1:172.21.15.188:6789/0 conn(0x7ff844144750 legacy=0x7ff84406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844071890 con 0x7ff8440ce920 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff834001670 con 0x7ff8440ce920 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 >> v1:172.21.15.148:6789/0 conn(0x7ff8440ce920 legacy=0x7ff8440cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300039b0 0x7ff830003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8300051c0 0x7ff83000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440715b0 con 0x7ff830004800 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83000c160 con 0x7ff8300039b0 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83000c360 con 0x7ff8300051c0 2024-04-25T09:54:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff848be1640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff849be3640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8300051c0 0x7ff83000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff8493e2640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300039b0 0x7ff830003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff848be1640 1 -- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8300051c0 msgr2=0x7ff83000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff848be1640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8300051c0 0x7ff83000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff848be1640 1 -- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300039b0 msgr2=0x7ff830003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff848be1640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300039b0 0x7ff830003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff848be1640 1 -- 172.21.15.148:0/3828843759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830003520 con 0x7ff830004800 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff849be3640 1 -- 172.21.15.148:0/3828843759 reap_dead start 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.123+0000 7ff8493e2640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300039b0 0x7ff830003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.124+0000 7ff848be1640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff830004c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff83400c240 tx=0x7ff830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55a65378df1efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.124+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834007d10 con 0x7ff830004800 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.124+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff834005ad0 con 0x7ff830004800 2024-04-25T09:54:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.124+0000 7ff83bfff640 1 -- 172.21.15.148:0/3828843759 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff834007070 con 0x7ff830004800 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 -- 172.21.15.148:0/3828843759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 msgr2=0x7ff830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff830004c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff83400c240 tx=0x7ff830005830 comp rx=0 tx=0).stop 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 -- 172.21.15.148:0/3828843759 shutdown_connections 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 --2- 172.21.15.148:0/3828843759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff830004c80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 -- 172.21.15.148:0/3828843759 >> 172.21.15.148:0/3828843759 conn(0x7ff84406abe0 msgr2=0x7ff84406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 -- 172.21.15.148:0/3828843759 shutdown_connections 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.125+0000 7ff84b646640 1 -- 172.21.15.148:0/3828843759 wait complete. 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.126+0000 7ff84b646640 1 Processor -- start 2024-04-25T09:54:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.126+0000 7ff84b646640 1 -- start start 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff844143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff844153f80 0x7ff8441543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8440b2db0 0x7ff8440b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8441455d0 con 0x7ff830004800 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff844145450 con 0x7ff8440b2db0 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84415c000 con 0x7ff844153f80 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff844143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff844143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57418/0 (socket says 172.21.15.148:57418) 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 -- 172.21.15.148:0/1759985949 learned_addr learned my addr 172.21.15.148:0/1759985949 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff848be1640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff844153f80 0x7ff8441543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 -- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff844153f80 msgr2=0x7ff8441543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff849be3640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8440b2db0 0x7ff8440b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff844153f80 0x7ff8441543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 -- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8440b2db0 msgr2=0x7ff8440b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8440b2db0 0x7ff8440b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 -- 172.21.15.148:0/1759985949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff844154920 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff8493e2640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff844143650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff84000e4b0 tx=0x7ff84000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400052e0 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff840007020 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840007210 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.127+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8440b3b20 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.128+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8440b40e0 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.129+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844073990 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.129+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff84001a030 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.129+0000 7ff839ffb640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff82806b500 0x7ff82806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.129+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff840089b00 con 0x7ff830004800 2024-04-25T09:54:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.129+0000 7ff848be1640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff82806b500 0x7ff82806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.130+0000 7ff848be1640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff82806b500 0x7ff82806d9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff834032000 tx=0x7ff83402d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.133+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff840021020 con 0x7ff830004800 2024-04-25T09:54:43.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.303+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8440af8c0 con 0x7ff830004800 2024-04-25T09:54:43.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.304+0000 7ff839ffb640 1 -- 172.21.15.148:0/1759985949 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7ff840058640 con 0x7ff830004800 2024-04-25T09:54:43.307 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:43.307 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:43.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff82806b500 msgr2=0x7ff82806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff82806b500 0x7ff82806d9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff834032000 tx=0x7ff83402d000 comp rx=0 tx=0).stop 2024-04-25T09:54:43.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 msgr2=0x7ff844143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff844143650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff84000e4b0 tx=0x7ff84000e970 comp rx=0 tx=0).stop 2024-04-25T09:54:43.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 shutdown_connections 2024-04-25T09:54:43.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff82806b500 0x7ff82806d9c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8440b2db0 0x7ff8440b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff844153f80 0x7ff8441543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 --2- 172.21.15.148:0/1759985949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830004800 0x7ff844143650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.306+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 >> 172.21.15.148:0/1759985949 conn(0x7ff84406abe0 msgr2=0x7ff84415a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.307+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 shutdown_connections 2024-04-25T09:54:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.307+0000 7ff84b646640 1 -- 172.21.15.148:0/1759985949 wait complete. 2024-04-25T09:54:43.320 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:43.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.414+0000 7fdf69196640 1 Processor -- start 2024-04-25T09:54:43.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf69196640 1 -- start start 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf69196640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf6406ce10 0x7fdf640d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf69196640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf6409a450 0x7fdf640cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf69196640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6406c220 con 0x7fdf6415dc70 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf69196640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640d0c90 con 0x7fdf6409a450 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf69196640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640d0080 con 0x7fdf6406ce10 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf6359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf6409a450 0x7fdf640cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.415+0000 7fdf62d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdf6415dc70 0x7fdf6406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57044/0 (socket says 172.21.15.148:57044) 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf62d9d640 1 -- 172.21.15.148:0/3255376857 learned_addr learned my addr 172.21.15.148:0/3255376857 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/3255376857 >> v2:172.21.15.188:3300/0 conn(0x7fdf6406ce10 0x7fdf640d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf6359e640 1 -- 172.21.15.148:0/3255376857 >> v2:172.21.15.188:3300/0 conn(0x7fdf6406ce10 msgr2=0x7fdf640d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf6359e640 1 --2- 172.21.15.148:0/3255376857 >> v2:172.21.15.188:3300/0 conn(0x7fdf6406ce10 0x7fdf640d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf6359e640 1 -- 172.21.15.148:0/3255376857 >> v1:172.21.15.148:6789/0 conn(0x7fdf6415dc70 legacy=0x7fdf6406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf6359e640 1 -- 172.21.15.148:0/3255376857 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640d0360 con 0x7fdf6409a450 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf6259c640 1 -- 172.21.15.148:0/3255376857 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496668585 0 0) 0x7fdf6406c220 con 0x7fdf6415dc70 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.416+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/3255376857 >> v2:172.21.15.188:3300/0 conn(0x7fdf6406ce10 0x7fdf640d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf6359e640 1 --2- 172.21.15.148:0/3255376857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6409a450 0x7fdf640cf9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf640733b0 tx=0x7fdf5c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ecf6e15ad83af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf6259c640 1 -- 172.21.15.148:0/3255376857 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c007740 con 0x7fdf6409a450 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf6259c640 1 -- 172.21.15.148:0/3255376857 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c0078e0 con 0x7fdf6409a450 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf6259c640 1 -- 172.21.15.148:0/3255376857 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c007cb0 con 0x7fdf6409a450 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 -- 172.21.15.148:0/3255376857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6409a450 msgr2=0x7fdf640cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 --2- 172.21.15.148:0/3255376857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6409a450 0x7fdf640cf9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdf640733b0 tx=0x7fdf5c02a680 comp rx=0 tx=0).stop 2024-04-25T09:54:43.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 -- 172.21.15.148:0/3255376857 shutdown_connections 2024-04-25T09:54:43.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 --2- 172.21.15.148:0/3255376857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6409a450 0x7fdf640cf9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 --2- 172.21.15.148:0/3255376857 >> v2:172.21.15.188:3300/0 conn(0x7fdf6406ce10 0x7fdf640d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 -- 172.21.15.148:0/3255376857 >> 172.21.15.148:0/3255376857 conn(0x7fdf640b3270 msgr2=0x7fdf640b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 -- 172.21.15.148:0/3255376857 shutdown_connections 2024-04-25T09:54:43.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.417+0000 7fdf69196640 1 -- 172.21.15.148:0/3255376857 wait complete. 2024-04-25T09:54:43.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 Processor -- start 2024-04-25T09:54:43.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 -- start start 2024-04-25T09:54:43.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 0x7fdf64151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf6409a450 0x7fdf640b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415dc70 0x7fdf64070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415e050 con 0x7fdf6406ce10 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640d0920 con 0x7fdf6409a450 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf69196640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640cbdd0 con 0x7fdf6415dc70 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf62d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 0x7fdf64151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf62d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 0x7fdf64151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57444/0 (socket says 172.21.15.148:57444) 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.419+0000 7fdf62d9d640 1 -- 172.21.15.148:0/3458527477 learned_addr learned my addr 172.21.15.148:0/3458527477 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf6359e640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415dc70 0x7fdf64070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf6409a450 0x7fdf640b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf62d9d640 1 -- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415dc70 msgr2=0x7fdf64070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf62d9d640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415dc70 0x7fdf64070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf62d9d640 1 -- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf6409a450 msgr2=0x7fdf640b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf62d9d640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf6409a450 0x7fdf640b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf62d9d640 1 -- 172.21.15.148:0/3458527477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640b52b0 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf62d9d640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 0x7fdf64151af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400c430 tx=0x7fdf5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540074e0 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540047e0 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf640b5590 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54004bd0 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.420+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf640b5b50 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.421+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf640b5f90 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.421+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf54011070 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.422+0000 7fdf5b7fe640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4406b4d0 0x7fdf4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.422+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4406b4d0 0x7fdf4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.422+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdf5408a150 con 0x7fdf6406ce10 2024-04-25T09:54:43.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.422+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4406b4d0 0x7fdf4406d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf64070ef0 tx=0x7fdf4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.426+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf5400e280 con 0x7fdf6406ce10 2024-04-25T09:54:43.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.596+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf64073990 con 0x7fdf6406ce10 2024-04-25T09:54:43.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.598+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/3458527477 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fdf54058c90 con 0x7fdf6406ce10 2024-04-25T09:54:43.600 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:43.600 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:43.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4406b4d0 msgr2=0x7fdf4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4406b4d0 0x7fdf4406d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf64070ef0 tx=0x7fdf4c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 msgr2=0x7fdf64151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 0x7fdf64151af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400c430 tx=0x7fdf5400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 shutdown_connections 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4406b4d0 0x7fdf4406d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf6415dc70 0x7fdf64070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf6409a450 0x7fdf640b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 --2- 172.21.15.148:0/3458527477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf6406ce10 0x7fdf64151af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 >> 172.21.15.148:0/3458527477 conn(0x7fdf640b3270 msgr2=0x7fdf640ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 shutdown_connections 2024-04-25T09:54:43.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.600+0000 7fdf69196640 1 -- 172.21.15.148:0/3458527477 wait complete. 2024-04-25T09:54:43.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:54:43.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.710+0000 7f1b9e9f9640 1 Processor -- start 2024-04-25T09:54:43.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.710+0000 7f1b9e9f9640 1 -- start start 2024-04-25T09:54:43.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b9e9f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b9815dc20 0x7f1b980ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b9e9f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b980d32f0 0x7f1b980cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b9e9f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9815e000 con 0x7f1b98161710 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b9e9f9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980d1de0 con 0x7f1b9815dc20 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b9e9f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98161c60 con 0x7f1b980d32f0 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b9815dc20 0x7f1b980ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b9815dc20 0x7f1b980ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33224/0 (socket says 172.21.15.148:33224) 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 -- 172.21.15.148:0/96135716 learned_addr learned my addr 172.21.15.148:0/96135716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b9cf96640 1 --2- 172.21.15.148:0/96135716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b980d32f0 0x7f1b980cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 -- 172.21.15.148:0/96135716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b980d32f0 msgr2=0x7f1b980cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 --2- 172.21.15.148:0/96135716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b980d32f0 0x7f1b980cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 -- 172.21.15.148:0/96135716 >> v1:172.21.15.188:6789/0 conn(0x7f1b98161710 legacy=0x7f1b9806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.711+0000 7f1b97fff640 1 -- 172.21.15.148:0/96135716 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b98161f40 con 0x7f1b9815dc20 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.712+0000 7f1b97fff640 1 --2- 172.21.15.148:0/96135716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b9815dc20 0x7f1b980ce670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009870 tx=0x7f1b8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6509048a51ef9dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.712+0000 7f1b96ffd640 1 -- 172.21.15.148:0/96135716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88037070 con 0x7f1b9815dc20 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.712+0000 7f1b96ffd640 1 -- 172.21.15.148:0/96135716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88032440 con 0x7f1b9815dc20 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.712+0000 7f1b96ffd640 1 -- 172.21.15.148:0/96135716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88032810 con 0x7f1b9815dc20 2024-04-25T09:54:43.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.712+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/96135716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b9815dc20 msgr2=0x7f1b980ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.712+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/96135716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b9815dc20 0x7f1b980ce670 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009870 tx=0x7f1b8802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.713+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/96135716 shutdown_connections 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.713+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/96135716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b980d32f0 0x7f1b980cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.713+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/96135716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b9815dc20 0x7f1b980ce670 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.713+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/96135716 >> 172.21.15.148:0/96135716 conn(0x7f1b980d0420 msgr2=0x7f1b980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.713+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/96135716 shutdown_connections 2024-04-25T09:54:43.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.713+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/96135716 wait complete. 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.714+0000 7f1b9e9f9640 1 Processor -- start 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.714+0000 7f1b9e9f9640 1 -- start start 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 0x7f1b980cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815dc20 0x7f1b98160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b98161710 0x7f1b981f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b5e50 con 0x7f1b980d32f0 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98162500 con 0x7f1b98161710 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b5fd0 con 0x7f1b9815dc20 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 0x7f1b980cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 0x7f1b980cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33230/0 (socket says 172.21.15.148:33230) 2024-04-25T09:54:43.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 -- 172.21.15.148:0/2997488276 learned_addr learned my addr 172.21.15.148:0/2997488276 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b977fe640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815dc20 0x7f1b98160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9cf96640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b98161710 0x7f1b981f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 -- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815dc20 msgr2=0x7f1b98160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815dc20 0x7f1b98160590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 -- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b98161710 msgr2=0x7f1b981f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b98161710 0x7f1b981f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 -- 172.21.15.148:0/2997488276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980cd260 con 0x7f1b980d32f0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b977fe640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815dc20 0x7f1b98160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9cf96640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b98161710 0x7f1b981f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b97fff640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 0x7f1b980cc000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b8802a2b0 tx=0x7f1b880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88041020 con 0x7f1b980d32f0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b880306c0 con 0x7f1b980d32f0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88037040 con 0x7f1b980d32f0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b980cd500 con 0x7f1b980d32f0 2024-04-25T09:54:43.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.715+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b980cb800 con 0x7f1b980d32f0 2024-04-25T09:54:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.717+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b980cf940 con 0x7f1b980d32f0 2024-04-25T09:54:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.717+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b88044020 con 0x7f1b980d32f0 2024-04-25T09:54:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.717+0000 7f1b94ff9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b8406b4d0 0x7f1b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.717+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1b880aacb0 con 0x7f1b980d32f0 2024-04-25T09:54:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.718+0000 7f1b977fe640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b8406b4d0 0x7f1b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:43.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.718+0000 7f1b977fe640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b8406b4d0 0x7f1b8406d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b98161210 tx=0x7f1b8c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:43.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.722+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b980cf940 con 0x7f1b980d32f0 2024-04-25T09:54:43.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:43.927+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b980d16b0 con 0x7f1b980d32f0 2024-04-25T09:54:44.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.096+0000 7f1b94ff9640 1 -- 172.21.15.148:0/2997488276 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1b8804b030 con 0x7f1b980d32f0 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b8406b4d0 msgr2=0x7f1b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b8406b4d0 0x7f1b8406d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b98161210 tx=0x7f1b8c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 msgr2=0x7f1b980cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 0x7f1b980cc000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b8802a2b0 tx=0x7f1b880046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 shutdown_connections 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b8406b4d0 0x7f1b8406d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b98161710 0x7f1b981f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815dc20 0x7f1b98160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 --2- 172.21.15.148:0/2997488276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d32f0 0x7f1b980cc000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 >> 172.21.15.148:0/2997488276 conn(0x7f1b980d0420 msgr2=0x7f1b980b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 shutdown_connections 2024-04-25T09:54:44.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.098+0000 7f1b9e9f9640 1 -- 172.21.15.148:0/2997488276 wait complete. 2024-04-25T09:54:44.112 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T09:54:44.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:54:44.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.209+0000 7f32d9134640 1 Processor -- start 2024-04-25T09:54:44.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d9134640 1 -- start start 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d9134640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32d4070dc0 0x7f32d406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d9134640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32d40711a0 0x7f32d40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d9134640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40d1770 con 0x7f32d40b4830 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d9134640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d41622b0 con 0x7f32d40711a0 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d9134640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d40b10f0 con 0x7f32d4070dc0 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.210+0000 7f32d259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32d40711a0 0x7f32d40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d359e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f32d40b4830 0x7f32d40b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48656/0 (socket says 172.21.15.148:48656) 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32d40711a0 0x7f32d40b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33246/0 (socket says 172.21.15.148:33246) 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d359e640 1 -- 172.21.15.148:0/3808447170 learned_addr learned my addr 172.21.15.148:0/3808447170 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d2d9d640 1 --2- 172.21.15.148:0/3808447170 >> v2:172.21.15.188:3300/0 conn(0x7f32d4070dc0 0x7f32d406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d259c640 1 -- 172.21.15.148:0/3808447170 >> v2:172.21.15.188:3300/0 conn(0x7f32d4070dc0 msgr2=0x7f32d406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:44.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d259c640 1 --2- 172.21.15.148:0/3808447170 >> v2:172.21.15.188:3300/0 conn(0x7f32d4070dc0 0x7f32d406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d259c640 1 -- 172.21.15.148:0/3808447170 >> v1:172.21.15.148:6789/0 conn(0x7f32d40b4830 legacy=0x7f32d40b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d1d9b640 1 -- 172.21.15.148:0/3808447170 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100012923 0 0) 0x7f32d40d1770 con 0x7f32d40b4830 2024-04-25T09:54:44.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d259c640 1 -- 172.21.15.148:0/3808447170 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4162740 con 0x7f32d40711a0 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.211+0000 7f32d259c640 1 --2- 172.21.15.148:0/3808447170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d40711a0 0x7f32d40b07a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f32d4070a00 tx=0x7f32bc02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c93124484ff948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d1d9b640 1 -- 172.21.15.148:0/3808447170 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc037070 con 0x7f32d40711a0 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d1d9b640 1 -- 172.21.15.148:0/3808447170 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32bc02add0 con 0x7f32d40711a0 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d1d9b640 1 -- 172.21.15.148:0/3808447170 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32bc032660 con 0x7f32d40711a0 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 -- 172.21.15.148:0/3808447170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d40711a0 msgr2=0x7f32d40b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 --2- 172.21.15.148:0/3808447170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d40711a0 0x7f32d40b07a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f32d4070a00 tx=0x7f32bc02a080 comp rx=0 tx=0).stop 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 -- 172.21.15.148:0/3808447170 shutdown_connections 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 --2- 172.21.15.148:0/3808447170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d40711a0 0x7f32d40b07a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 --2- 172.21.15.148:0/3808447170 >> v2:172.21.15.188:3300/0 conn(0x7f32d4070dc0 0x7f32d406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 -- 172.21.15.148:0/3808447170 >> 172.21.15.148:0/3808447170 conn(0x7f32d40b3530 msgr2=0x7f32d40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 -- 172.21.15.148:0/3808447170 shutdown_connections 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.212+0000 7f32d9134640 1 -- 172.21.15.148:0/3808447170 wait complete. 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 Processor -- start 2024-04-25T09:54:44.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 -- start start 2024-04-25T09:54:44.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 0x7f32d4148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d40711a0 0x7f32d41f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d40b4830 0x7f32d41f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4161f40 con 0x7f32d4070dc0 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4161c60 con 0x7f32d40b4830 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d9134640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4071630 con 0x7f32d40711a0 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 0x7f32d4148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.214+0000 7f32d2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 0x7f32d4148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33258/0 (socket says 172.21.15.148:33258) 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 -- 172.21.15.148:0/2290974630 learned_addr learned my addr 172.21.15.148:0/2290974630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d359e640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d40b4830 0x7f32d41f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d259c640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d40711a0 0x7f32d41f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 -- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d40711a0 msgr2=0x7f32d41f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d40711a0 0x7f32d41f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 -- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d40b4830 msgr2=0x7f32d41f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d40b4830 0x7f32d41f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 -- 172.21.15.148:0/2290974630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40cd570 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d359e640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d40b4830 0x7f32d41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d259c640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d40711a0 0x7f32d41f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d2d9d640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 0x7f32d4148890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32cc007f20 tx=0x7f32cc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc014620 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc019070 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d4148dd0 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc014a00 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.215+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d4149200 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.216+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4073a00 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.217+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32cc014ba0 con 0x7f32d4070dc0 2024-04-25T09:54:44.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.217+0000 7f32cb7fe640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32b406b4d0 0x7f32b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.217+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f32cc08ae50 con 0x7f32d4070dc0 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.217+0000 7f32d259c640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32b406b4d0 0x7f32b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:44.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.217+0000 7f32d259c640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32b406b4d0 0x7f32b406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32bc034000 tx=0x7f32bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.221+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32cc059990 con 0x7f32d4070dc0 2024-04-25T09:54:44.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:44.429+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f32d40cf830 con 0x7f32d4070dc0 2024-04-25T09:54:45.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.101+0000 7f32cb7fe640 1 -- 172.21.15.148:0/2290974630 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f32cc059290 con 0x7f32d4070dc0 2024-04-25T09:54:45.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32b406b4d0 msgr2=0x7f32b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32b406b4d0 0x7f32b406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32bc034000 tx=0x7f32bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:45.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 msgr2=0x7f32d4148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 0x7f32d4148890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32cc007f20 tx=0x7f32cc00e570 comp rx=0 tx=0).stop 2024-04-25T09:54:45.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 shutdown_connections 2024-04-25T09:54:45.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32b406b4d0 0x7f32b406d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32d40b4830 0x7f32d41f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32d40711a0 0x7f32d41f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 --2- 172.21.15.148:0/2290974630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32d4070dc0 0x7f32d4148890 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 >> 172.21.15.148:0/2290974630 conn(0x7f32d40b3530 msgr2=0x7f32d40cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 shutdown_connections 2024-04-25T09:54:45.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.103+0000 7f32d9134640 1 -- 172.21.15.148:0/2290974630 wait complete. 2024-04-25T09:54:45.117 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:54:45.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:45.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.213+0000 7f9b332f5640 1 Processor -- start 2024-04-25T09:54:45.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.213+0000 7f9b332f5640 1 -- start start 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b332f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b2c15db70 0x7f9b2c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b332f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9b2c0b5c30 0x7f9b2c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b332f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c162690 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b332f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0ceee0 con 0x7f9b2c15db70 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b332f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0d1d10 con 0x7f9b2c0b5c30 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b31091640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b2c15db70 0x7f9b2c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b31091640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b2c15db70 0x7f9b2c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33262/0 (socket says 172.21.15.148:33262) 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b31091640 1 -- 172.21.15.148:0/3982638847 learned_addr learned my addr 172.21.15.148:0/3982638847 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b31091640 1 -- 172.21.15.148:0/3982638847 >> v2:172.21.15.188:3300/0 conn(0x7f9b2c0b5c30 msgr2=0x7f9b2c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b31091640 1 --2- 172.21.15.148:0/3982638847 >> v2:172.21.15.188:3300/0 conn(0x7f9b2c0b5c30 0x7f9b2c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.214+0000 7f9b31091640 1 -- 172.21.15.148:0/3982638847 >> v1:172.21.15.188:6789/0 conn(0x7f9b2c0b6cb0 legacy=0x7f9b2c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.215+0000 7f9b31091640 1 -- 172.21.15.148:0/3982638847 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0d1ff0 con 0x7f9b2c15db70 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.215+0000 7f9b31091640 1 --2- 172.21.15.148:0/3982638847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c15db70 0x7f9b2c162150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c08ccc0 tx=0x7f9b2802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204f35635f37bc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.215+0000 7f9b23fff640 1 -- 172.21.15.148:0/3982638847 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28037070 con 0x7f9b2c15db70 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.215+0000 7f9b23fff640 1 -- 172.21.15.148:0/3982638847 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b28032440 con 0x7f9b2c15db70 2024-04-25T09:54:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.215+0000 7f9b23fff640 1 -- 172.21.15.148:0/3982638847 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28032810 con 0x7f9b2c15db70 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 -- 172.21.15.148:0/3982638847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c15db70 msgr2=0x7f9b2c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 --2- 172.21.15.148:0/3982638847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c15db70 0x7f9b2c162150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c08ccc0 tx=0x7f9b2802a250 comp rx=0 tx=0).stop 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 -- 172.21.15.148:0/3982638847 shutdown_connections 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 --2- 172.21.15.148:0/3982638847 >> v2:172.21.15.188:3300/0 conn(0x7f9b2c0b5c30 0x7f9b2c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 --2- 172.21.15.148:0/3982638847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c15db70 0x7f9b2c162150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 -- 172.21.15.148:0/3982638847 >> 172.21.15.148:0/3982638847 conn(0x7f9b2c0d0520 msgr2=0x7f9b2c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 -- 172.21.15.148:0/3982638847 shutdown_connections 2024-04-25T09:54:45.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.216+0000 7f9b332f5640 1 -- 172.21.15.148:0/3982638847 wait complete. 2024-04-25T09:54:45.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.217+0000 7f9b332f5640 1 Processor -- start 2024-04-25T09:54:45.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.217+0000 7f9b332f5640 1 -- start start 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b332f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b2c0b5c30 0x7f9b2c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b332f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 0x7f9b2c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b332f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b2c15db70 0x7f9b2c14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b332f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0ce530 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b332f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c15df50 con 0x7f9b2c15db70 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b332f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cf1c0 con 0x7f9b2c0b5c30 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 0x7f9b2c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 0x7f9b2c0cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33278/0 (socket says 172.21.15.148:33278) 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 -- 172.21.15.148:0/1851629334 learned_addr learned my addr 172.21.15.148:0/1851629334 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b31892640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b2c15db70 0x7f9b2c14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b31091640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b2c0b5c30 0x7f9b2c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 -- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b2c0b5c30 msgr2=0x7f9b2c0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b2c0b5c30 0x7f9b2c0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 -- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b2c15db70 msgr2=0x7f9b2c14edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b2c15db70 0x7f9b2c14edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 -- 172.21.15.148:0/1851629334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0cb050 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b31892640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b2c15db70 0x7f9b2c14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.218+0000 7f9b30890640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 0x7f9b2c0cddf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00c400 tx=0x7f9b1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.219+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c017070 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.219+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0035f0 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.219+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c003a20 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.219+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c0cb330 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.219+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c0cb8f0 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.220+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c073990 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.220+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b1c01a070 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.220+0000 7f9b21ffb640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b1006b4d0 0x7f9b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.220+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9b1c0897b0 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.220+0000 7f9b31091640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b1006b4d0 0x7f9b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.221+0000 7f9b31091640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b1006b4d0 0x7f9b1006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b28034000 tx=0x7f9b2802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.224+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b1c057d30 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.431+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b2c06cea0 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.431+0000 7f9b21ffb640 1 -- 172.21.15.148:0/1851629334 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f9b1c05bdd0 con 0x7f9b2c0b6cb0 2024-04-25T09:54:45.434 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:45.434 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:45.434 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b1006b4d0 msgr2=0x7f9b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b1006b4d0 0x7f9b1006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9b28034000 tx=0x7f9b2802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 msgr2=0x7f9b2c0cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 0x7f9b2c0cddf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00c400 tx=0x7f9b1c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 shutdown_connections 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b1006b4d0 0x7f9b1006d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b2c15db70 0x7f9b2c14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b2c0b6cb0 0x7f9b2c0cddf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 --2- 172.21.15.148:0/1851629334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b2c0b5c30 0x7f9b2c0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 >> 172.21.15.148:0/1851629334 conn(0x7f9b2c0d0520 msgr2=0x7f9b2c15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 shutdown_connections 2024-04-25T09:54:45.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.433+0000 7f9b332f5640 1 -- 172.21.15.148:0/1851629334 wait complete. 2024-04-25T09:54:45.447 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:45.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.540+0000 7f477cc22640 1 Processor -- start 2024-04-25T09:54:45.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.541+0000 7f477cc22640 1 -- start start 2024-04-25T09:54:45.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.541+0000 7f477cc22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 0x7f477806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.541+0000 7f477cc22640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4778072620 0x7f4778072a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.541+0000 7f477cc22640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47780cfa60 con 0x7f477815b9c0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.541+0000 7f477cc22640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477806b580 con 0x7f4778072620 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.541+0000 7f477cc22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47780cfd40 con 0x7f477806abe0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f477659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f477815b9c0 0x7f477815bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48668/0 (socket says 172.21.15.148:48668) 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f477659c640 1 -- 172.21.15.148:0/4131237481 learned_addr learned my addr 172.21.15.148:0/4131237481 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f476ffff640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 0x7f477806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4776d9d640 1 --2- 172.21.15.148:0/4131237481 >> v2:172.21.15.188:3300/0 conn(0x7f4778072620 0x7f4778072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224220767 0 0) 0x7f47780cfa60 con 0x7f477815b9c0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4764003520 con 0x7f477815b9c0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006812992 0 0) 0x7f4764003520 con 0x7f477815b9c0 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 msgr2=0x7f477806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 0x7f477806b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 >> v2:172.21.15.188:3300/0 conn(0x7f4778072620 msgr2=0x7f4778072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 --2- 172.21.15.148:0/4131237481 >> v2:172.21.15.188:3300/0 conn(0x7f4778072620 0x7f4778072a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47780d0020 con 0x7f477815b9c0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4768001670 con 0x7f477815b9c0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4776d9d640 1 --2- 172.21.15.148:0/4131237481 >> v2:172.21.15.188:3300/0 conn(0x7f4778072620 0x7f4778072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 >> v1:172.21.15.148:6789/0 conn(0x7f477815b9c0 legacy=0x7f477815bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47640039b0 0x7f4764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 0x7f47640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47780cfd40 con 0x7f4764004800 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476400b9c0 con 0x7f47640039b0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476400bb40 con 0x7f477806abe0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f476ffff640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f477659c640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47640039b0 0x7f4764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.542+0000 7f4776d9d640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 0x7f47640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f476ffff640 1 -- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 msgr2=0x7f47640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f476ffff640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f477806abe0 0x7f47640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f476ffff640 1 -- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47640039b0 msgr2=0x7f4764003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f476ffff640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47640039b0 0x7f4764003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f476ffff640 1 -- 172.21.15.148:0/4131237481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764003520 con 0x7f4764004800 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f476ffff640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4764004c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4760009ac0 tx=0x7f4764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443dd67c5ea2c070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760036070 con 0x7f4764004800 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760031400 con 0x7f4764004800 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.543+0000 7f4775d9b640 1 -- 172.21.15.148:0/4131237481 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760009690 con 0x7f4764004800 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f477cc22640 1 -- 172.21.15.148:0/4131237481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 msgr2=0x7f4764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f477cc22640 1 --2- 172.21.15.148:0/4131237481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4764004c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4760009ac0 tx=0x7f4764005830 comp rx=0 tx=0).stop 2024-04-25T09:54:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f4776d9d640 1 -- 172.21.15.148:0/4131237481 reap_dead start 2024-04-25T09:54:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f477cc22640 1 -- 172.21.15.148:0/4131237481 shutdown_connections 2024-04-25T09:54:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f477cc22640 1 -- 172.21.15.148:0/4131237481 >> 172.21.15.148:0/4131237481 conn(0x7f47780b2370 msgr2=0x7f47780b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f477cc22640 1 -- 172.21.15.148:0/4131237481 shutdown_connections 2024-04-25T09:54:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.544+0000 7f477cc22640 1 -- 172.21.15.148:0/4131237481 wait complete. 2024-04-25T09:54:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.545+0000 7f477cc22640 1 Processor -- start 2024-04-25T09:54:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.545+0000 7f477cc22640 1 -- start start 2024-04-25T09:54:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477cc22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4778151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477cc22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4778150b30 0x7f4778159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477cc22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47781597d0 0x7f477815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477cc22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778072fd0 con 0x7f4764004800 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477cc22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47780cf710 con 0x7f47781597d0 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477cc22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4778073150 con 0x7f4778150b30 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4778151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4778151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33302/0 (socket says 172.21.15.148:33302) 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 -- 172.21.15.148:0/3400634671 learned_addr learned my addr 172.21.15.148:0/3400634671 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f4776d9d640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47781597d0 0x7f477815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f476ffff640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4778150b30 0x7f4778159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 -- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4778150b30 msgr2=0x7f4778159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4778150b30 0x7f4778159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 -- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47781597d0 msgr2=0x7f477815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47781597d0 0x7f477815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 -- 172.21.15.148:0/3400634671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778159ce0 con 0x7f4764004800 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f476ffff640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4778150b30 0x7f4778159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.546+0000 7f477659c640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4778151320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4768007340 tx=0x7f4768007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.547+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476801a070 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.547+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476800f8e0 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.547+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477815c6e0 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.547+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476800fcb0 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.547+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477815cca0 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.548+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4778073990 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.548+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f476801d070 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.549+0000 7f476effd640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f475806b4d0 0x7f475806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.549+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4768089e40 con 0x7f4764004800 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.549+0000 7f476ffff640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f475806b4d0 0x7f475806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.549+0000 7f476ffff640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f475806b4d0 0x7f475806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4760033000 tx=0x7f4760008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.553+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47680583c0 con 0x7f4764004800 2024-04-25T09:54:45.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.723+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4778157ad0 con 0x7f4764004800 2024-04-25T09:54:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.724+0000 7f476effd640 1 -- 172.21.15.148:0/3400634671 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f476805c460 con 0x7f4764004800 2024-04-25T09:54:45.727 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:45.727 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f475806b4d0 msgr2=0x7f475806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f475806b4d0 0x7f475806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4760033000 tx=0x7f4760008030 comp rx=0 tx=0).stop 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 msgr2=0x7f4778151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4778151320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4768007340 tx=0x7f4768007800 comp rx=0 tx=0).stop 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 shutdown_connections 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f475806b4d0 0x7f475806d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47781597d0 0x7f477815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4778150b30 0x7f4778159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 --2- 172.21.15.148:0/3400634671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4764004800 0x7f4778151320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 >> 172.21.15.148:0/3400634671 conn(0x7f47780b2370 msgr2=0x7f47780d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 shutdown_connections 2024-04-25T09:54:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.726+0000 7f477cc22640 1 -- 172.21.15.148:0/3400634671 wait complete. 2024-04-25T09:54:45.740 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:54:45.740 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.835+0000 7fcbd6553640 1 Processor -- start 2024-04-25T09:54:45.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.835+0000 7fcbd6553640 1 -- start start 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.835+0000 7fcbd6553640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbd015db70 0x7fcbd0162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.835+0000 7fcbd6553640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcbd00b5c30 0x7fcbd00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd6553640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd0162690 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd6553640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd00ceee0 con 0x7fcbd00b5c30 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd6553640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd00d1d10 con 0x7fcbd015db70 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcbd00b5c30 0x7fcbd00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcbd00b5c30 0x7fcbd00d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33318/0 (socket says 172.21.15.148:33318) 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 -- 172.21.15.148:0/1036360071 learned_addr learned my addr 172.21.15.148:0/1036360071 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:45.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbcffff640 1 --2- 172.21.15.148:0/1036360071 >> v2:172.21.15.188:3300/0 conn(0x7fcbd015db70 0x7fcbd0162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 -- 172.21.15.148:0/1036360071 >> v2:172.21.15.188:3300/0 conn(0x7fcbd015db70 msgr2=0x7fcbd0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 --2- 172.21.15.148:0/1036360071 >> v2:172.21.15.188:3300/0 conn(0x7fcbd015db70 0x7fcbd0162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 -- 172.21.15.148:0/1036360071 >> v1:172.21.15.188:6789/0 conn(0x7fcbd00b6cb0 legacy=0x7fcbd00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbd4af0640 1 -- 172.21.15.148:0/1036360071 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00d1ff0 con 0x7fcbd00b5c30 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.836+0000 7fcbcffff640 1 --2- 172.21.15.148:0/1036360071 >> v2:172.21.15.188:3300/0 conn(0x7fcbd015db70 0x7fcbd0162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd4af0640 1 --2- 172.21.15.148:0/1036360071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b5c30 0x7fcbd00d27f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00b6570 tx=0x7fcbc802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96be0bbbd2f2fe83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbceffd640 1 -- 172.21.15.148:0/1036360071 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8036070 con 0x7fcbd00b5c30 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbceffd640 1 -- 172.21.15.148:0/1036360071 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc8031600 con 0x7fcbd00b5c30 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbceffd640 1 -- 172.21.15.148:0/1036360071 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80319d0 con 0x7fcbd00b5c30 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 -- 172.21.15.148:0/1036360071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b5c30 msgr2=0x7fcbd00d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 --2- 172.21.15.148:0/1036360071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b5c30 0x7fcbd00d27f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00b6570 tx=0x7fcbc802a730 comp rx=0 tx=0).stop 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 -- 172.21.15.148:0/1036360071 shutdown_connections 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 --2- 172.21.15.148:0/1036360071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b5c30 0x7fcbd00d27f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 --2- 172.21.15.148:0/1036360071 >> v2:172.21.15.188:3300/0 conn(0x7fcbd015db70 0x7fcbd0162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 -- 172.21.15.148:0/1036360071 >> 172.21.15.148:0/1036360071 conn(0x7fcbd00d0520 msgr2=0x7fcbd00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:45.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.837+0000 7fcbd6553640 1 -- 172.21.15.148:0/1036360071 shutdown_connections 2024-04-25T09:54:45.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.838+0000 7fcbd6553640 1 -- 172.21.15.148:0/1036360071 wait complete. 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 Processor -- start 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 -- start start 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00b5c30 0x7fcbd0151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 0x7fcbd01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd015db70 0x7fcbd0150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00ce530 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd015df50 con 0x7fcbd00b5c30 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.839+0000 7fcbd6553640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd00cf1c0 con 0x7fcbd015db70 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 0x7fcbd01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 0x7fcbd01562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33334/0 (socket says 172.21.15.148:33334) 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 -- 172.21.15.148:0/195864577 learned_addr learned my addr 172.21.15.148:0/195864577 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcffff640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00b5c30 0x7fcbd0151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbd4af0640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd015db70 0x7fcbd0150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 -- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd015db70 msgr2=0x7fcbd0150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd015db70 0x7fcbd0150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 -- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00b5c30 msgr2=0x7fcbd0151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00b5c30 0x7fcbd0151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 -- 172.21.15.148:0/195864577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0152a50 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcffff640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00b5c30 0x7fcbd0151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbcf7fe640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 0x7fcbd01562c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c440 tx=0x7fcbc4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40052d0 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc4005470 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd0160fe0 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4007260 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.840+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd0152530 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.841+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd0152320 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.842+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbc4007400 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.842+0000 7fcbccff9640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbbc06b500 0x7fcbbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:45.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.842+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcbc4089a10 con 0x7fcbd00b6cb0 2024-04-25T09:54:45.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.842+0000 7fcbcffff640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbbc06b500 0x7fcbbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:45.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.843+0000 7fcbcffff640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbbc06b500 0x7fcbbc06d9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000a990 tx=0x7fcbc0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:45.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:45.846+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbc4057e70 con 0x7fcbd00b6cb0 2024-04-25T09:54:46.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.014+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbd0152700 con 0x7fcbbc06b500 2024-04-25T09:54:46.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.022+0000 7fcbccff9640 1 -- 172.21.15.148:0/195864577 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fcbd0152700 con 0x7fcbbc06b500 2024-04-25T09:54:46.029 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:46.031 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:54:46.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbbc06b500 msgr2=0x7fcbbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbbc06b500 0x7fcbbc06d9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000a990 tx=0x7fcbc0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:46.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 msgr2=0x7fcbd01562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 0x7fcbd01562c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c440 tx=0x7fcbc4001e70 comp rx=0 tx=0).stop 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 shutdown_connections 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbbc06b500 0x7fcbbc06d9c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbd015db70 0x7fcbd0150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcbd00b6cb0 0x7fcbd01562c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 --2- 172.21.15.148:0/195864577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbd00b5c30 0x7fcbd0151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 >> 172.21.15.148:0/195864577 conn(0x7fcbd00d0520 msgr2=0x7fcbd01598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 shutdown_connections 2024-04-25T09:54:46.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.026+0000 7fcbd6553640 1 -- 172.21.15.148:0/195864577 wait complete. 2024-04-25T09:54:46.039 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T09:54:45.024391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086122+0000","last_change":"2024-04-25T09:54:22.962688+0000","last_active":"2024-04-25T09:54:30.086122+0000","last_peered":"2024-04-25T09:54:30.086122+0000","last_clean":"2024-04-25T09:54:30.086122+0000","last_became_active":"2024-04-25T09:54:22.962426+0000","last_became_peered":"2024-04-25T09:54:22.962426+0000","last_unstale":"2024-04-25T09:54:30.086122+0000","last_undegraded":"2024-04-25T09:54:30.086122+0000","last_fullsized":"2024-04-25T09:54:30.086122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:21.016302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995003+0000","last_change":"2024-04-25T09:54:23.969918+0000","last_active":"2024-04-25T09:54:29.995003+0000","last_peered":"2024-04-25T09:54:29.995003+0000","last_clean":"2024-04-25T09:54:29.995003+0000","last_became_active":"2024-04-25T09:54:23.969631+0000","last_became_peered":"2024-04-25T09:54:23.969631+0000","last_unstale":"2024-04-25T09:54:29.995003+0000","last_undegraded":"2024-04-25T09:54:29.995003+0000","last_fullsized":"2024-04-25T09:54:29.995003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994303+0000","last_change":"2024-04-25T09:54:28.397901+0000","last_active":"2024-04-25T09:54:29.994303+0000","last_peered":"2024-04-25T09:54:29.994303+0000","last_clean":"2024-04-25T09:54:29.994303+0000","last_became_active":"2024-04-25T09:54:28.397492+0000","last_became_peered":"2024-04-25T09:54:28.397492+0000","last_unstale":"2024-04-25T09:54:29.994303+0000","last_undegraded":"2024-04-25T09:54:29.994303+0000","last_fullsized":"2024-04-25T09:54:29.994303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:14.336697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995250+0000","last_change":"2024-04-25T09:54:22.958345+0000","last_active":"2024-04-25T09:54:29.995250+0000","last_peered":"2024-04-25T09:54:29.995250+0000","last_clean":"2024-04-25T09:54:29.995250+0000","last_became_active":"2024-04-25T09:54:22.958021+0000","last_became_peered":"2024-04-25T09:54:22.958021+0000","last_unstale":"2024-04-25T09:54:29.995250+0000","last_undegraded":"2024-04-25T09:54:29.995250+0000","last_fullsized":"2024-04-25T09:54:29.995250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:02.625621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085905+0000","last_change":"2024-04-25T09:54:23.970381+0000","last_active":"2024-04-25T09:54:30.085905+0000","last_peered":"2024-04-25T09:54:30.085905+0000","last_clean":"2024-04-25T09:54:30.085905+0000","last_became_active":"2024-04-25T09:54:23.970077+0000","last_became_peered":"2024-04-25T09:54:23.970077+0000","last_unstale":"2024-04-25T09:54:30.085905+0000","last_undegraded":"2024-04-25T09:54:30.085905+0000","last_fullsized":"2024-04-25T09:54:30.085905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:16.071642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995354+0000","last_change":"2024-04-25T09:54:28.011784+0000","last_active":"2024-04-25T09:54:29.995354+0000","last_peered":"2024-04-25T09:54:29.995354+0000","last_clean":"2024-04-25T09:54:29.995354+0000","last_became_active":"2024-04-25T09:54:28.011370+0000","last_became_peered":"2024-04-25T09:54:28.011370+0000","last_unstale":"2024-04-25T09:54:29.995354+0000","last_undegraded":"2024-04-25T09:54:29.995354+0000","last_fullsized":"2024-04-25T09:54:29.995354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088011+0000","last_change":"2024-04-25T09:54:22.965546+0000","last_active":"2024-04-25T09:54:30.088011+0000","last_peered":"2024-04-25T09:54:30.088011+0000","last_clean":"2024-04-25T09:54:30.088011+0000","last_became_active":"2024-04-25T09:54:22.965088+0000","last_became_peered":"2024-04-25T09:54:22.965088+0000","last_unstale":"2024-04-25T09:54:30.088011+0000","last_undegraded":"2024-04-25T09:54:30.088011+0000","last_fullsized":"2024-04-25T09:54:30.088011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:23.982573+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:23.982304+0000","last_became_peered":"2024-04-25T09:54:23.982304+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995558+0000","last_change":"2024-04-25T09:54:28.012195+0000","last_active":"2024-04-25T09:54:29.995558+0000","last_peered":"2024-04-25T09:54:29.995558+0000","last_clean":"2024-04-25T09:54:29.995558+0000","last_became_active":"2024-04-25T09:54:28.011667+0000","last_became_peered":"2024-04-25T09:54:28.011667+0000","last_unstale":"2024-04-25T09:54:29.995558+0000","last_undegraded":"2024-04-25T09:54:29.995558+0000","last_fullsized":"2024-04-25T09:54:29.995558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087654+0000","last_change":"2024-04-25T09:54:22.967092+0000","last_active":"2024-04-25T09:54:30.087654+0000","last_peered":"2024-04-25T09:54:30.087654+0000","last_clean":"2024-04-25T09:54:30.087654+0000","last_became_active":"2024-04-25T09:54:22.966690+0000","last_became_peered":"2024-04-25T09:54:22.966690+0000","last_unstale":"2024-04-25T09:54:30.087654+0000","last_undegraded":"2024-04-25T09:54:30.087654+0000","last_fullsized":"2024-04-25T09:54:30.087654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087289+0000","last_change":"2024-04-25T09:54:23.979036+0000","last_active":"2024-04-25T09:54:30.087289+0000","last_peered":"2024-04-25T09:54:30.087289+0000","last_clean":"2024-04-25T09:54:30.087289+0000","last_became_active":"2024-04-25T09:54:23.978775+0000","last_became_peered":"2024-04-25T09:54:23.978775+0000","last_unstale":"2024-04-25T09:54:30.087289+0000","last_undegraded":"2024-04-25T09:54:30.087289+0000","last_fullsized":"2024-04-25T09:54:30.087289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995398+0000","last_change":"2024-04-25T09:54:28.006708+0000","last_active":"2024-04-25T09:54:29.995398+0000","last_peered":"2024-04-25T09:54:29.995398+0000","last_clean":"2024-04-25T09:54:29.995398+0000","last_became_active":"2024-04-25T09:54:28.006351+0000","last_became_peered":"2024-04-25T09:54:28.006351+0000","last_unstale":"2024-04-25T09:54:29.995398+0000","last_undegraded":"2024-04-25T09:54:29.995398+0000","last_fullsized":"2024-04-25T09:54:29.995398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085947+0000","last_change":"2024-04-25T09:54:23.983452+0000","last_active":"2024-04-25T09:54:30.085947+0000","last_peered":"2024-04-25T09:54:30.085947+0000","last_clean":"2024-04-25T09:54:30.085947+0000","last_became_active":"2024-04-25T09:54:23.983150+0000","last_became_peered":"2024-04-25T09:54:23.983150+0000","last_unstale":"2024-04-25T09:54:30.085947+0000","last_undegraded":"2024-04-25T09:54:30.085947+0000","last_fullsized":"2024-04-25T09:54:30.085947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087835+0000","last_change":"2024-04-25T09:54:22.965628+0000","last_active":"2024-04-25T09:54:30.087835+0000","last_peered":"2024-04-25T09:54:30.087835+0000","last_clean":"2024-04-25T09:54:30.087835+0000","last_became_active":"2024-04-25T09:54:22.965294+0000","last_became_peered":"2024-04-25T09:54:22.965294+0000","last_unstale":"2024-04-25T09:54:30.087835+0000","last_undegraded":"2024-04-25T09:54:30.087835+0000","last_fullsized":"2024-04-25T09:54:30.087835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996313+0000","last_change":"2024-04-25T09:54:28.011196+0000","last_active":"2024-04-25T09:54:29.996313+0000","last_peered":"2024-04-25T09:54:29.996313+0000","last_clean":"2024-04-25T09:54:29.996313+0000","last_became_active":"2024-04-25T09:54:28.010638+0000","last_became_peered":"2024-04-25T09:54:28.010638+0000","last_unstale":"2024-04-25T09:54:29.996313+0000","last_undegraded":"2024-04-25T09:54:29.996313+0000","last_fullsized":"2024-04-25T09:54:29.996313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994468+0000","last_change":"2024-04-25T09:54:23.975061+0000","last_active":"2024-04-25T09:54:29.994468+0000","last_peered":"2024-04-25T09:54:29.994468+0000","last_clean":"2024-04-25T09:54:29.994468+0000","last_became_active":"2024-04-25T09:54:23.974793+0000","last_became_peered":"2024-04-25T09:54:23.974793+0000","last_unstale":"2024-04-25T09:54:29.994468+0000","last_undegraded":"2024-04-25T09:54:29.994468+0000","last_fullsized":"2024-04-25T09:54:29.994468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086013+0000","last_change":"2024-04-25T09:54:22.965353+0000","last_active":"2024-04-25T09:54:30.086013+0000","last_peered":"2024-04-25T09:54:30.086013+0000","last_clean":"2024-04-25T09:54:30.086013+0000","last_became_active":"2024-04-25T09:54:22.965038+0000","last_became_peered":"2024-04-25T09:54:22.965038+0000","last_unstale":"2024-04-25T09:54:30.086013+0000","last_undegraded":"2024-04-25T09:54:30.086013+0000","last_fullsized":"2024-04-25T09:54:30.086013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996196+0000","last_change":"2024-04-25T09:54:28.005662+0000","last_active":"2024-04-25T09:54:29.996196+0000","last_peered":"2024-04-25T09:54:29.996196+0000","last_clean":"2024-04-25T09:54:29.996196+0000","last_became_active":"2024-04-25T09:54:28.005233+0000","last_became_peered":"2024-04-25T09:54:28.005233+0000","last_unstale":"2024-04-25T09:54:29.996196+0000","last_undegraded":"2024-04-25T09:54:29.996196+0000","last_fullsized":"2024-04-25T09:54:29.996196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994816+0000","last_change":"2024-04-25T09:54:23.969844+0000","last_active":"2024-04-25T09:54:29.994816+0000","last_peered":"2024-04-25T09:54:29.994816+0000","last_clean":"2024-04-25T09:54:29.994816+0000","last_became_active":"2024-04-25T09:54:23.969261+0000","last_became_peered":"2024-04-25T09:54:23.969261+0000","last_unstale":"2024-04-25T09:54:29.994816+0000","last_undegraded":"2024-04-25T09:54:29.994816+0000","last_fullsized":"2024-04-25T09:54:29.994816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086366+0000","last_change":"2024-04-25T09:54:22.961764+0000","last_active":"2024-04-25T09:54:30.086366+0000","last_peered":"2024-04-25T09:54:30.086366+0000","last_clean":"2024-04-25T09:54:30.086366+0000","last_became_active":"2024-04-25T09:54:22.961455+0000","last_became_peered":"2024-04-25T09:54:22.961455+0000","last_unstale":"2024-04-25T09:54:30.086366+0000","last_undegraded":"2024-04-25T09:54:30.086366+0000","last_fullsized":"2024-04-25T09:54:30.086366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087099+0000","last_change":"2024-04-25T09:54:28.395240+0000","last_active":"2024-04-25T09:54:30.087099+0000","last_peered":"2024-04-25T09:54:30.087099+0000","last_clean":"2024-04-25T09:54:30.087099+0000","last_became_active":"2024-04-25T09:54:28.394866+0000","last_became_peered":"2024-04-25T09:54:28.394866+0000","last_unstale":"2024-04-25T09:54:30.087099+0000","last_undegraded":"2024-04-25T09:54:30.087099+0000","last_fullsized":"2024-04-25T09:54:30.087099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995771+0000","last_change":"2024-04-25T09:54:23.973436+0000","last_active":"2024-04-25T09:54:29.995771+0000","last_peered":"2024-04-25T09:54:29.995771+0000","last_clean":"2024-04-25T09:54:29.995771+0000","last_became_active":"2024-04-25T09:54:23.973139+0000","last_became_peered":"2024-04-25T09:54:23.973139+0000","last_unstale":"2024-04-25T09:54:29.995771+0000","last_undegraded":"2024-04-25T09:54:29.995771+0000","last_fullsized":"2024-04-25T09:54:29.995771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086953+0000","last_change":"2024-04-25T09:54:22.963445+0000","last_active":"2024-04-25T09:54:30.086953+0000","last_peered":"2024-04-25T09:54:30.086953+0000","last_clean":"2024-04-25T09:54:30.086953+0000","last_became_active":"2024-04-25T09:54:22.963190+0000","last_became_peered":"2024-04-25T09:54:22.963190+0000","last_unstale":"2024-04-25T09:54:30.086953+0000","last_undegraded":"2024-04-25T09:54:30.086953+0000","last_fullsized":"2024-04-25T09:54:30.086953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996305+0000","last_change":"2024-04-25T09:54:28.011236+0000","last_active":"2024-04-25T09:54:29.996305+0000","last_peered":"2024-04-25T09:54:29.996305+0000","last_clean":"2024-04-25T09:54:29.996305+0000","last_became_active":"2024-04-25T09:54:28.010633+0000","last_became_peered":"2024-04-25T09:54:28.010633+0000","last_unstale":"2024-04-25T09:54:29.996305+0000","last_undegraded":"2024-04-25T09:54:29.996305+0000","last_fullsized":"2024-04-25T09:54:29.996305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086754+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:30.086754+0000","last_peered":"2024-04-25T09:54:30.086754+0000","last_clean":"2024-04-25T09:54:30.086754+0000","last_became_active":"2024-04-25T09:54:23.970411+0000","last_became_peered":"2024-04-25T09:54:23.970411+0000","last_unstale":"2024-04-25T09:54:30.086754+0000","last_undegraded":"2024-04-25T09:54:30.086754+0000","last_fullsized":"2024-04-25T09:54:30.086754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:20.071529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086786+0000","last_change":"2024-04-25T09:54:22.961585+0000","last_active":"2024-04-25T09:54:30.086786+0000","last_peered":"2024-04-25T09:54:30.086786+0000","last_clean":"2024-04-25T09:54:30.086786+0000","last_became_active":"2024-04-25T09:54:22.961286+0000","last_became_peered":"2024-04-25T09:54:22.961286+0000","last_unstale":"2024-04-25T09:54:30.086786+0000","last_undegraded":"2024-04-25T09:54:30.086786+0000","last_fullsized":"2024-04-25T09:54:30.086786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995453+0000","last_change":"2024-04-25T09:54:28.396021+0000","last_active":"2024-04-25T09:54:29.995453+0000","last_peered":"2024-04-25T09:54:29.995453+0000","last_clean":"2024-04-25T09:54:29.995453+0000","last_became_active":"2024-04-25T09:54:28.394961+0000","last_became_peered":"2024-04-25T09:54:28.394961+0000","last_unstale":"2024-04-25T09:54:29.995453+0000","last_undegraded":"2024-04-25T09:54:29.995453+0000","last_fullsized":"2024-04-25T09:54:29.995453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995224+0000","last_change":"2024-04-25T09:54:23.981659+0000","last_active":"2024-04-25T09:54:29.995224+0000","last_peered":"2024-04-25T09:54:29.995224+0000","last_clean":"2024-04-25T09:54:29.995224+0000","last_became_active":"2024-04-25T09:54:23.981345+0000","last_became_peered":"2024-04-25T09:54:23.981345+0000","last_unstale":"2024-04-25T09:54:29.995224+0000","last_undegraded":"2024-04-25T09:54:29.995224+0000","last_fullsized":"2024-04-25T09:54:29.995224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086841+0000","last_change":"2024-04-25T09:54:22.961799+0000","last_active":"2024-04-25T09:54:30.086841+0000","last_peered":"2024-04-25T09:54:30.086841+0000","last_clean":"2024-04-25T09:54:30.086841+0000","last_became_active":"2024-04-25T09:54:22.961205+0000","last_became_peered":"2024-04-25T09:54:22.961205+0000","last_unstale":"2024-04-25T09:54:30.086841+0000","last_undegraded":"2024-04-25T09:54:30.086841+0000","last_fullsized":"2024-04-25T09:54:30.086841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994244+0000","last_change":"2024-04-25T09:54:28.397432+0000","last_active":"2024-04-25T09:54:29.994244+0000","last_peered":"2024-04-25T09:54:29.994244+0000","last_clean":"2024-04-25T09:54:29.994244+0000","last_became_active":"2024-04-25T09:54:28.396060+0000","last_became_peered":"2024-04-25T09:54:28.396060+0000","last_unstale":"2024-04-25T09:54:29.994244+0000","last_undegraded":"2024-04-25T09:54:29.994244+0000","last_fullsized":"2024-04-25T09:54:29.994244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:18.141331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086215+0000","last_change":"2024-04-25T09:54:23.964362+0000","last_active":"2024-04-25T09:54:30.086215+0000","last_peered":"2024-04-25T09:54:30.086215+0000","last_clean":"2024-04-25T09:54:30.086215+0000","last_became_active":"2024-04-25T09:54:23.964097+0000","last_became_peered":"2024-04-25T09:54:23.964097+0000","last_unstale":"2024-04-25T09:54:30.086215+0000","last_undegraded":"2024-04-25T09:54:30.086215+0000","last_fullsized":"2024-04-25T09:54:30.086215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088121+0000","last_change":"2024-04-25T09:54:22.967489+0000","last_active":"2024-04-25T09:54:30.088121+0000","last_peered":"2024-04-25T09:54:30.088121+0000","last_clean":"2024-04-25T09:54:30.088121+0000","last_became_active":"2024-04-25T09:54:22.967191+0000","last_became_peered":"2024-04-25T09:54:22.967191+0000","last_unstale":"2024-04-25T09:54:30.088121+0000","last_undegraded":"2024-04-25T09:54:30.088121+0000","last_fullsized":"2024-04-25T09:54:30.088121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086235+0000","last_change":"2024-04-25T09:54:28.394981+0000","last_active":"2024-04-25T09:54:30.086235+0000","last_peered":"2024-04-25T09:54:30.086235+0000","last_clean":"2024-04-25T09:54:30.086235+0000","last_became_active":"2024-04-25T09:54:28.394572+0000","last_became_peered":"2024-04-25T09:54:28.394572+0000","last_unstale":"2024-04-25T09:54:30.086235+0000","last_undegraded":"2024-04-25T09:54:30.086235+0000","last_fullsized":"2024-04-25T09:54:30.086235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.977167+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.976581+0000","last_became_peered":"2024-04-25T09:54:23.976581+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086309+0000","last_change":"2024-04-25T09:54:22.957000+0000","last_active":"2024-04-25T09:54:30.086309+0000","last_peered":"2024-04-25T09:54:30.086309+0000","last_clean":"2024-04-25T09:54:30.086309+0000","last_became_active":"2024-04-25T09:54:22.956737+0000","last_became_peered":"2024-04-25T09:54:22.956737+0000","last_unstale":"2024-04-25T09:54:30.086309+0000","last_undegraded":"2024-04-25T09:54:30.086309+0000","last_fullsized":"2024-04-25T09:54:30.086309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085588+0000","last_change":"2024-04-25T09:54:28.009926+0000","last_active":"2024-04-25T09:54:30.085588+0000","last_peered":"2024-04-25T09:54:30.085588+0000","last_clean":"2024-04-25T09:54:30.085588+0000","last_became_active":"2024-04-25T09:54:28.009486+0000","last_became_peered":"2024-04-25T09:54:28.009486+0000","last_unstale":"2024-04-25T09:54:30.085588+0000","last_undegraded":"2024-04-25T09:54:30.085588+0000","last_fullsized":"2024-04-25T09:54:30.085588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996132+0000","last_change":"2024-04-25T09:54:23.974850+0000","last_active":"2024-04-25T09:54:29.996132+0000","last_peered":"2024-04-25T09:54:29.996132+0000","last_clean":"2024-04-25T09:54:29.996132+0000","last_became_active":"2024-04-25T09:54:23.974014+0000","last_became_peered":"2024-04-25T09:54:23.974014+0000","last_unstale":"2024-04-25T09:54:29.996132+0000","last_undegraded":"2024-04-25T09:54:29.996132+0000","last_fullsized":"2024-04-25T09:54:29.996132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087242+0000","last_change":"2024-04-25T09:54:22.966563+0000","last_active":"2024-04-25T09:54:30.087242+0000","last_peered":"2024-04-25T09:54:30.087242+0000","last_clean":"2024-04-25T09:54:30.087242+0000","last_became_active":"2024-04-25T09:54:22.966272+0000","last_became_peered":"2024-04-25T09:54:22.966272+0000","last_unstale":"2024-04-25T09:54:30.087242+0000","last_undegraded":"2024-04-25T09:54:30.087242+0000","last_fullsized":"2024-04-25T09:54:30.087242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086121+0000","last_change":"2024-04-25T09:54:28.013312+0000","last_active":"2024-04-25T09:54:30.086121+0000","last_peered":"2024-04-25T09:54:30.086121+0000","last_clean":"2024-04-25T09:54:30.086121+0000","last_became_active":"2024-04-25T09:54:28.012953+0000","last_became_peered":"2024-04-25T09:54:28.012953+0000","last_unstale":"2024-04-25T09:54:30.086121+0000","last_undegraded":"2024-04-25T09:54:30.086121+0000","last_fullsized":"2024-04-25T09:54:30.086121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087003+0000","last_change":"2024-04-25T09:54:23.966466+0000","last_active":"2024-04-25T09:54:30.087003+0000","last_peered":"2024-04-25T09:54:30.087003+0000","last_clean":"2024-04-25T09:54:30.087003+0000","last_became_active":"2024-04-25T09:54:23.966129+0000","last_became_peered":"2024-04-25T09:54:23.966129+0000","last_unstale":"2024-04-25T09:54:30.087003+0000","last_undegraded":"2024-04-25T09:54:30.087003+0000","last_fullsized":"2024-04-25T09:54:30.087003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086581+0000","last_change":"2024-04-25T09:54:22.963150+0000","last_active":"2024-04-25T09:54:30.086581+0000","last_peered":"2024-04-25T09:54:30.086581+0000","last_clean":"2024-04-25T09:54:30.086581+0000","last_became_active":"2024-04-25T09:54:22.962832+0000","last_became_peered":"2024-04-25T09:54:22.962832+0000","last_unstale":"2024-04-25T09:54:30.086581+0000","last_undegraded":"2024-04-25T09:54:30.086581+0000","last_fullsized":"2024-04-25T09:54:30.086581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087126+0000","last_change":"2024-04-25T09:54:28.011051+0000","last_active":"2024-04-25T09:54:30.087126+0000","last_peered":"2024-04-25T09:54:30.087126+0000","last_clean":"2024-04-25T09:54:30.087126+0000","last_became_active":"2024-04-25T09:54:28.010629+0000","last_became_peered":"2024-04-25T09:54:28.010629+0000","last_unstale":"2024-04-25T09:54:30.087126+0000","last_undegraded":"2024-04-25T09:54:30.087126+0000","last_fullsized":"2024-04-25T09:54:30.087126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085547+0000","last_change":"2024-04-25T09:54:23.973638+0000","last_active":"2024-04-25T09:54:30.085547+0000","last_peered":"2024-04-25T09:54:30.085547+0000","last_clean":"2024-04-25T09:54:30.085547+0000","last_became_active":"2024-04-25T09:54:23.973349+0000","last_became_peered":"2024-04-25T09:54:23.973349+0000","last_unstale":"2024-04-25T09:54:30.085547+0000","last_undegraded":"2024-04-25T09:54:30.085547+0000","last_fullsized":"2024-04-25T09:54:30.085547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087072+0000","last_change":"2024-04-25T09:54:22.966560+0000","last_active":"2024-04-25T09:54:30.087072+0000","last_peered":"2024-04-25T09:54:30.087072+0000","last_clean":"2024-04-25T09:54:30.087072+0000","last_became_active":"2024-04-25T09:54:22.966299+0000","last_became_peered":"2024-04-25T09:54:22.966299+0000","last_unstale":"2024-04-25T09:54:30.087072+0000","last_undegraded":"2024-04-25T09:54:30.087072+0000","last_fullsized":"2024-04-25T09:54:30.087072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086483+0000","last_change":"2024-04-25T09:54:28.395179+0000","last_active":"2024-04-25T09:54:30.086483+0000","last_peered":"2024-04-25T09:54:30.086483+0000","last_clean":"2024-04-25T09:54:30.086483+0000","last_became_active":"2024-04-25T09:54:28.394767+0000","last_became_peered":"2024-04-25T09:54:28.394767+0000","last_unstale":"2024-04-25T09:54:30.086483+0000","last_undegraded":"2024-04-25T09:54:30.086483+0000","last_fullsized":"2024-04-25T09:54:30.086483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994411+0000","last_change":"2024-04-25T09:54:23.969956+0000","last_active":"2024-04-25T09:54:29.994411+0000","last_peered":"2024-04-25T09:54:29.994411+0000","last_clean":"2024-04-25T09:54:29.994411+0000","last_became_active":"2024-04-25T09:54:23.969643+0000","last_became_peered":"2024-04-25T09:54:23.969643+0000","last_unstale":"2024-04-25T09:54:29.994411+0000","last_undegraded":"2024-04-25T09:54:29.994411+0000","last_fullsized":"2024-04-25T09:54:29.994411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086619+0000","last_change":"2024-04-25T09:54:22.965368+0000","last_active":"2024-04-25T09:54:30.086619+0000","last_peered":"2024-04-25T09:54:30.086619+0000","last_clean":"2024-04-25T09:54:30.086619+0000","last_became_active":"2024-04-25T09:54:22.965051+0000","last_became_peered":"2024-04-25T09:54:22.965051+0000","last_unstale":"2024-04-25T09:54:30.086619+0000","last_undegraded":"2024-04-25T09:54:30.086619+0000","last_fullsized":"2024-04-25T09:54:30.086619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087177+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:54:30.087177+0000","last_peered":"2024-04-25T09:54:30.087177+0000","last_clean":"2024-04-25T09:54:30.087177+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:54:30.087177+0000","last_undegraded":"2024-04-25T09:54:30.087177+0000","last_fullsized":"2024-04-25T09:54:30.087177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085975+0000","last_change":"2024-04-25T09:54:28.396320+0000","last_active":"2024-04-25T09:54:30.085975+0000","last_peered":"2024-04-25T09:54:30.085975+0000","last_clean":"2024-04-25T09:54:30.085975+0000","last_became_active":"2024-04-25T09:54:28.395895+0000","last_became_peered":"2024-04-25T09:54:28.395895+0000","last_unstale":"2024-04-25T09:54:30.085975+0000","last_undegraded":"2024-04-25T09:54:30.085975+0000","last_fullsized":"2024-04-25T09:54:30.085975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087913+0000","last_change":"2024-04-25T09:54:23.983959+0000","last_active":"2024-04-25T09:54:30.087913+0000","last_peered":"2024-04-25T09:54:30.087913+0000","last_clean":"2024-04-25T09:54:30.087913+0000","last_became_active":"2024-04-25T09:54:23.983669+0000","last_became_peered":"2024-04-25T09:54:23.983669+0000","last_unstale":"2024-04-25T09:54:30.087913+0000","last_undegraded":"2024-04-25T09:54:30.087913+0000","last_fullsized":"2024-04-25T09:54:30.087913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087174+0000","last_change":"2024-04-25T09:54:22.966591+0000","last_active":"2024-04-25T09:54:30.087174+0000","last_peered":"2024-04-25T09:54:30.087174+0000","last_clean":"2024-04-25T09:54:30.087174+0000","last_became_active":"2024-04-25T09:54:22.966301+0000","last_became_peered":"2024-04-25T09:54:22.966301+0000","last_unstale":"2024-04-25T09:54:30.087174+0000","last_undegraded":"2024-04-25T09:54:30.087174+0000","last_fullsized":"2024-04-25T09:54:30.087174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085806+0000","last_change":"2024-04-25T09:54:28.395577+0000","last_active":"2024-04-25T09:54:30.085806+0000","last_peered":"2024-04-25T09:54:30.085806+0000","last_clean":"2024-04-25T09:54:30.085806+0000","last_became_active":"2024-04-25T09:54:28.395067+0000","last_became_peered":"2024-04-25T09:54:28.395067+0000","last_unstale":"2024-04-25T09:54:30.085806+0000","last_undegraded":"2024-04-25T09:54:30.085806+0000","last_fullsized":"2024-04-25T09:54:30.085806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086598+0000","last_change":"2024-04-25T09:54:23.975660+0000","last_active":"2024-04-25T09:54:30.086598+0000","last_peered":"2024-04-25T09:54:30.086598+0000","last_clean":"2024-04-25T09:54:30.086598+0000","last_became_active":"2024-04-25T09:54:23.975394+0000","last_became_peered":"2024-04-25T09:54:23.975394+0000","last_unstale":"2024-04-25T09:54:30.086598+0000","last_undegraded":"2024-04-25T09:54:30.086598+0000","last_fullsized":"2024-04-25T09:54:30.086598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087150+0000","last_change":"2024-04-25T09:54:22.959855+0000","last_active":"2024-04-25T09:54:30.087150+0000","last_peered":"2024-04-25T09:54:30.087150+0000","last_clean":"2024-04-25T09:54:30.087150+0000","last_became_active":"2024-04-25T09:54:22.959556+0000","last_became_peered":"2024-04-25T09:54:22.959556+0000","last_unstale":"2024-04-25T09:54:30.087150+0000","last_undegraded":"2024-04-25T09:54:30.087150+0000","last_fullsized":"2024-04-25T09:54:30.087150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086831+0000","last_change":"2024-04-25T09:54:28.010760+0000","last_active":"2024-04-25T09:54:30.086831+0000","last_peered":"2024-04-25T09:54:30.086831+0000","last_clean":"2024-04-25T09:54:30.086831+0000","last_became_active":"2024-04-25T09:54:28.010391+0000","last_became_peered":"2024-04-25T09:54:28.010391+0000","last_unstale":"2024-04-25T09:54:30.086831+0000","last_undegraded":"2024-04-25T09:54:30.086831+0000","last_fullsized":"2024-04-25T09:54:30.086831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088301+0000","last_change":"2024-04-25T09:54:23.982380+0000","last_active":"2024-04-25T09:54:30.088301+0000","last_peered":"2024-04-25T09:54:30.088301+0000","last_clean":"2024-04-25T09:54:30.088301+0000","last_became_active":"2024-04-25T09:54:23.982076+0000","last_became_peered":"2024-04-25T09:54:23.982076+0000","last_unstale":"2024-04-25T09:54:30.088301+0000","last_undegraded":"2024-04-25T09:54:30.088301+0000","last_fullsized":"2024-04-25T09:54:30.088301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086229+0000","last_change":"2024-04-25T09:54:22.963068+0000","last_active":"2024-04-25T09:54:30.086229+0000","last_peered":"2024-04-25T09:54:30.086229+0000","last_clean":"2024-04-25T09:54:30.086229+0000","last_became_active":"2024-04-25T09:54:22.962607+0000","last_became_peered":"2024-04-25T09:54:22.962607+0000","last_unstale":"2024-04-25T09:54:30.086229+0000","last_undegraded":"2024-04-25T09:54:30.086229+0000","last_fullsized":"2024-04-25T09:54:30.086229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:28.396123+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:28.395141+0000","last_became_peered":"2024-04-25T09:54:28.395141+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087759+0000","last_change":"2024-04-25T09:54:23.981922+0000","last_active":"2024-04-25T09:54:30.087759+0000","last_peered":"2024-04-25T09:54:30.087759+0000","last_clean":"2024-04-25T09:54:30.087759+0000","last_became_active":"2024-04-25T09:54:23.981624+0000","last_became_peered":"2024-04-25T09:54:23.981624+0000","last_unstale":"2024-04-25T09:54:30.087759+0000","last_undegraded":"2024-04-25T09:54:30.087759+0000","last_fullsized":"2024-04-25T09:54:30.087759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996103+0000","last_change":"2024-04-25T09:54:22.958111+0000","last_active":"2024-04-25T09:54:29.996103+0000","last_peered":"2024-04-25T09:54:29.996103+0000","last_clean":"2024-04-25T09:54:29.996103+0000","last_became_active":"2024-04-25T09:54:22.957822+0000","last_became_peered":"2024-04-25T09:54:22.957822+0000","last_unstale":"2024-04-25T09:54:29.996103+0000","last_undegraded":"2024-04-25T09:54:29.996103+0000","last_fullsized":"2024-04-25T09:54:29.996103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:05.312329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995523+0000","last_change":"2024-04-25T09:54:28.011885+0000","last_active":"2024-04-25T09:54:29.995523+0000","last_peered":"2024-04-25T09:54:29.995523+0000","last_clean":"2024-04-25T09:54:29.995523+0000","last_became_active":"2024-04-25T09:54:28.011498+0000","last_became_peered":"2024-04-25T09:54:28.011498+0000","last_unstale":"2024-04-25T09:54:29.995523+0000","last_undegraded":"2024-04-25T09:54:29.995523+0000","last_fullsized":"2024-04-25T09:54:29.995523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087609+0000","last_change":"2024-04-25T09:54:23.982839+0000","last_active":"2024-04-25T09:54:30.087609+0000","last_peered":"2024-04-25T09:54:30.087609+0000","last_clean":"2024-04-25T09:54:30.087609+0000","last_became_active":"2024-04-25T09:54:23.982542+0000","last_became_peered":"2024-04-25T09:54:23.982542+0000","last_unstale":"2024-04-25T09:54:30.087609+0000","last_undegraded":"2024-04-25T09:54:30.087609+0000","last_fullsized":"2024-04-25T09:54:30.087609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085485+0000","last_change":"2024-04-25T09:54:22.954113+0000","last_active":"2024-04-25T09:54:30.085485+0000","last_peered":"2024-04-25T09:54:30.085485+0000","last_clean":"2024-04-25T09:54:30.085485+0000","last_became_active":"2024-04-25T09:54:22.953819+0000","last_became_peered":"2024-04-25T09:54:22.953819+0000","last_unstale":"2024-04-25T09:54:30.085485+0000","last_undegraded":"2024-04-25T09:54:30.085485+0000","last_fullsized":"2024-04-25T09:54:30.085485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086548+0000","last_change":"2024-04-25T09:54:28.008178+0000","last_active":"2024-04-25T09:54:30.086548+0000","last_peered":"2024-04-25T09:54:30.086548+0000","last_clean":"2024-04-25T09:54:30.086548+0000","last_became_active":"2024-04-25T09:54:28.007737+0000","last_became_peered":"2024-04-25T09:54:28.007737+0000","last_unstale":"2024-04-25T09:54:30.086548+0000","last_undegraded":"2024-04-25T09:54:30.086548+0000","last_fullsized":"2024-04-25T09:54:30.086548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085923+0000","last_change":"2024-04-25T09:54:23.977488+0000","last_active":"2024-04-25T09:54:30.085923+0000","last_peered":"2024-04-25T09:54:30.085923+0000","last_clean":"2024-04-25T09:54:30.085923+0000","last_became_active":"2024-04-25T09:54:23.977190+0000","last_became_peered":"2024-04-25T09:54:23.977190+0000","last_unstale":"2024-04-25T09:54:30.085923+0000","last_undegraded":"2024-04-25T09:54:30.085923+0000","last_fullsized":"2024-04-25T09:54:30.085923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086516+0000","last_change":"2024-04-25T09:54:22.959330+0000","last_active":"2024-04-25T09:54:30.086516+0000","last_peered":"2024-04-25T09:54:30.086516+0000","last_clean":"2024-04-25T09:54:30.086516+0000","last_became_active":"2024-04-25T09:54:22.959026+0000","last_became_peered":"2024-04-25T09:54:22.959026+0000","last_unstale":"2024-04-25T09:54:30.086516+0000","last_undegraded":"2024-04-25T09:54:30.086516+0000","last_fullsized":"2024-04-25T09:54:30.086516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995258+0000","last_change":"2024-04-25T09:54:28.396288+0000","last_active":"2024-04-25T09:54:29.995258+0000","last_peered":"2024-04-25T09:54:29.995258+0000","last_clean":"2024-04-25T09:54:29.995258+0000","last_became_active":"2024-04-25T09:54:28.395940+0000","last_became_peered":"2024-04-25T09:54:28.395940+0000","last_unstale":"2024-04-25T09:54:29.995258+0000","last_undegraded":"2024-04-25T09:54:29.995258+0000","last_fullsized":"2024-04-25T09:54:29.995258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:05.296954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087789+0000","last_change":"2024-04-25T09:54:23.982575+0000","last_active":"2024-04-25T09:54:30.087789+0000","last_peered":"2024-04-25T09:54:30.087789+0000","last_clean":"2024-04-25T09:54:30.087789+0000","last_became_active":"2024-04-25T09:54:23.982274+0000","last_became_peered":"2024-04-25T09:54:23.982274+0000","last_unstale":"2024-04-25T09:54:30.087789+0000","last_undegraded":"2024-04-25T09:54:30.087789+0000","last_fullsized":"2024-04-25T09:54:30.087789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087152+0000","last_change":"2024-04-25T09:54:22.962975+0000","last_active":"2024-04-25T09:54:30.087152+0000","last_peered":"2024-04-25T09:54:30.087152+0000","last_clean":"2024-04-25T09:54:30.087152+0000","last_became_active":"2024-04-25T09:54:22.962642+0000","last_became_peered":"2024-04-25T09:54:22.962642+0000","last_unstale":"2024-04-25T09:54:30.087152+0000","last_undegraded":"2024-04-25T09:54:30.087152+0000","last_fullsized":"2024-04-25T09:54:30.087152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994645+0000","last_change":"2024-04-25T09:54:28.395146+0000","last_active":"2024-04-25T09:54:29.994645+0000","last_peered":"2024-04-25T09:54:29.994645+0000","last_clean":"2024-04-25T09:54:29.994645+0000","last_became_active":"2024-04-25T09:54:28.394756+0000","last_became_peered":"2024-04-25T09:54:28.394756+0000","last_unstale":"2024-04-25T09:54:29.994645+0000","last_undegraded":"2024-04-25T09:54:29.994645+0000","last_fullsized":"2024-04-25T09:54:29.994645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:05.993975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086033+0000","last_change":"2024-04-25T09:54:23.972232+0000","last_active":"2024-04-25T09:54:30.086033+0000","last_peered":"2024-04-25T09:54:30.086033+0000","last_clean":"2024-04-25T09:54:30.086033+0000","last_became_active":"2024-04-25T09:54:23.971948+0000","last_became_peered":"2024-04-25T09:54:23.971948+0000","last_unstale":"2024-04-25T09:54:30.086033+0000","last_undegraded":"2024-04-25T09:54:30.086033+0000","last_fullsized":"2024-04-25T09:54:30.086033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:10.704903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086681+0000","last_change":"2024-04-25T09:54:22.957583+0000","last_active":"2024-04-25T09:54:30.086681+0000","last_peered":"2024-04-25T09:54:30.086681+0000","last_clean":"2024-04-25T09:54:30.086681+0000","last_became_active":"2024-04-25T09:54:22.957190+0000","last_became_peered":"2024-04-25T09:54:22.957190+0000","last_unstale":"2024-04-25T09:54:30.086681+0000","last_undegraded":"2024-04-25T09:54:30.086681+0000","last_fullsized":"2024-04-25T09:54:30.086681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:21.147671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087171+0000","last_change":"2024-04-25T09:54:28.395989+0000","last_active":"2024-04-25T09:54:30.087171+0000","last_peered":"2024-04-25T09:54:30.087171+0000","last_clean":"2024-04-25T09:54:30.087171+0000","last_became_active":"2024-04-25T09:54:28.395578+0000","last_became_peered":"2024-04-25T09:54:28.395578+0000","last_unstale":"2024-04-25T09:54:30.087171+0000","last_undegraded":"2024-04-25T09:54:30.087171+0000","last_fullsized":"2024-04-25T09:54:30.087171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087863+0000","last_change":"2024-04-25T09:54:23.984390+0000","last_active":"2024-04-25T09:54:30.087863+0000","last_peered":"2024-04-25T09:54:30.087863+0000","last_clean":"2024-04-25T09:54:30.087863+0000","last_became_active":"2024-04-25T09:54:23.981528+0000","last_became_peered":"2024-04-25T09:54:23.981528+0000","last_unstale":"2024-04-25T09:54:30.087863+0000","last_undegraded":"2024-04-25T09:54:30.087863+0000","last_fullsized":"2024-04-25T09:54:30.087863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:33.133289+0000","last_change":"2024-04-25T09:54:22.962540+0000","last_active":"2024-04-25T09:54:33.133289+0000","last_peered":"2024-04-25T09:54:33.133289+0000","last_clean":"2024-04-25T09:54:33.133289+0000","last_became_active":"2024-04-25T09:54:22.962223+0000","last_became_peered":"2024-04-25T09:54:22.962223+0000","last_unstale":"2024-04-25T09:54:33.133289+0000","last_undegraded":"2024-04-25T09:54:33.133289+0000","last_fullsized":"2024-04-25T09:54:33.133289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085807+0000","last_change":"2024-04-25T09:54:28.012670+0000","last_active":"2024-04-25T09:54:30.085807+0000","last_peered":"2024-04-25T09:54:30.085807+0000","last_clean":"2024-04-25T09:54:30.085807+0000","last_became_active":"2024-04-25T09:54:28.011580+0000","last_became_peered":"2024-04-25T09:54:28.011580+0000","last_unstale":"2024-04-25T09:54:30.085807+0000","last_undegraded":"2024-04-25T09:54:30.085807+0000","last_fullsized":"2024-04-25T09:54:30.085807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996273+0000","last_change":"2024-04-25T09:54:23.981044+0000","last_active":"2024-04-25T09:54:29.996273+0000","last_peered":"2024-04-25T09:54:29.996273+0000","last_clean":"2024-04-25T09:54:29.996273+0000","last_became_active":"2024-04-25T09:54:23.980733+0000","last_became_peered":"2024-04-25T09:54:23.980733+0000","last_unstale":"2024-04-25T09:54:29.996273+0000","last_undegraded":"2024-04-25T09:54:29.996273+0000","last_fullsized":"2024-04-25T09:54:29.996273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087059+0000","last_change":"2024-04-25T09:54:22.965478+0000","last_active":"2024-04-25T09:54:30.087059+0000","last_peered":"2024-04-25T09:54:30.087059+0000","last_clean":"2024-04-25T09:54:30.087059+0000","last_became_active":"2024-04-25T09:54:22.965191+0000","last_became_peered":"2024-04-25T09:54:22.965191+0000","last_unstale":"2024-04-25T09:54:30.087059+0000","last_undegraded":"2024-04-25T09:54:30.087059+0000","last_fullsized":"2024-04-25T09:54:30.087059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995924+0000","last_change":"2024-04-25T09:54:28.003916+0000","last_active":"2024-04-25T09:54:29.995924+0000","last_peered":"2024-04-25T09:54:29.995924+0000","last_clean":"2024-04-25T09:54:29.995924+0000","last_became_active":"2024-04-25T09:54:28.003512+0000","last_became_peered":"2024-04-25T09:54:28.003512+0000","last_unstale":"2024-04-25T09:54:29.995924+0000","last_undegraded":"2024-04-25T09:54:29.995924+0000","last_fullsized":"2024-04-25T09:54:29.995924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086045+0000","last_change":"2024-04-25T09:54:23.969795+0000","last_active":"2024-04-25T09:54:30.086045+0000","last_peered":"2024-04-25T09:54:30.086045+0000","last_clean":"2024-04-25T09:54:30.086045+0000","last_became_active":"2024-04-25T09:54:23.969543+0000","last_became_peered":"2024-04-25T09:54:23.969543+0000","last_unstale":"2024-04-25T09:54:30.086045+0000","last_undegraded":"2024-04-25T09:54:30.086045+0000","last_fullsized":"2024-04-25T09:54:30.086045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086869+0000","last_change":"2024-04-25T09:54:22.965431+0000","last_active":"2024-04-25T09:54:30.086869+0000","last_peered":"2024-04-25T09:54:30.086869+0000","last_clean":"2024-04-25T09:54:30.086869+0000","last_became_active":"2024-04-25T09:54:22.965115+0000","last_became_peered":"2024-04-25T09:54:22.965115+0000","last_unstale":"2024-04-25T09:54:30.086869+0000","last_undegraded":"2024-04-25T09:54:30.086869+0000","last_fullsized":"2024-04-25T09:54:30.086869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994658+0000","last_change":"2024-04-25T09:54:28.011370+0000","last_active":"2024-04-25T09:54:29.994658+0000","last_peered":"2024-04-25T09:54:29.994658+0000","last_clean":"2024-04-25T09:54:29.994658+0000","last_became_active":"2024-04-25T09:54:28.011006+0000","last_became_peered":"2024-04-25T09:54:28.011006+0000","last_unstale":"2024-04-25T09:54:29.994658+0000","last_undegraded":"2024-04-25T09:54:29.994658+0000","last_fullsized":"2024-04-25T09:54:29.994658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:05.906706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086261+0000","last_change":"2024-04-25T09:54:23.983800+0000","last_active":"2024-04-25T09:54:30.086261+0000","last_peered":"2024-04-25T09:54:30.086261+0000","last_clean":"2024-04-25T09:54:30.086261+0000","last_became_active":"2024-04-25T09:54:23.983518+0000","last_became_peered":"2024-04-25T09:54:23.983518+0000","last_unstale":"2024-04-25T09:54:30.086261+0000","last_undegraded":"2024-04-25T09:54:30.086261+0000","last_fullsized":"2024-04-25T09:54:30.086261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086666+0000","last_change":"2024-04-25T09:54:22.966898+0000","last_active":"2024-04-25T09:54:30.086666+0000","last_peered":"2024-04-25T09:54:30.086666+0000","last_clean":"2024-04-25T09:54:30.086666+0000","last_became_active":"2024-04-25T09:54:22.966638+0000","last_became_peered":"2024-04-25T09:54:22.966638+0000","last_unstale":"2024-04-25T09:54:30.086666+0000","last_undegraded":"2024-04-25T09:54:30.086666+0000","last_fullsized":"2024-04-25T09:54:30.086666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994713+0000","last_change":"2024-04-25T09:54:28.397771+0000","last_active":"2024-04-25T09:54:29.994713+0000","last_peered":"2024-04-25T09:54:29.994713+0000","last_clean":"2024-04-25T09:54:29.994713+0000","last_became_active":"2024-04-25T09:54:28.397413+0000","last_became_peered":"2024-04-25T09:54:28.397413+0000","last_unstale":"2024-04-25T09:54:29.994713+0000","last_undegraded":"2024-04-25T09:54:29.994713+0000","last_fullsized":"2024-04-25T09:54:29.994713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:07.468831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086335+0000","last_change":"2024-04-25T09:54:28.396249+0000","last_active":"2024-04-25T09:54:30.086335+0000","last_peered":"2024-04-25T09:54:30.086335+0000","last_clean":"2024-04-25T09:54:30.086335+0000","last_became_active":"2024-04-25T09:54:28.395904+0000","last_became_peered":"2024-04-25T09:54:28.395904+0000","last_unstale":"2024-04-25T09:54:30.086335+0000","last_undegraded":"2024-04-25T09:54:30.086335+0000","last_fullsized":"2024-04-25T09:54:30.086335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086193+0000","last_change":"2024-04-25T09:54:23.972012+0000","last_active":"2024-04-25T09:54:30.086193+0000","last_peered":"2024-04-25T09:54:30.086193+0000","last_clean":"2024-04-25T09:54:30.086193+0000","last_became_active":"2024-04-25T09:54:23.971731+0000","last_became_peered":"2024-04-25T09:54:23.971731+0000","last_unstale":"2024-04-25T09:54:30.086193+0000","last_undegraded":"2024-04-25T09:54:30.086193+0000","last_fullsized":"2024-04-25T09:54:30.086193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086458+0000","last_change":"2024-04-25T09:54:22.965400+0000","last_active":"2024-04-25T09:54:30.086458+0000","last_peered":"2024-04-25T09:54:30.086458+0000","last_clean":"2024-04-25T09:54:30.086458+0000","last_became_active":"2024-04-25T09:54:22.965099+0000","last_became_peered":"2024-04-25T09:54:22.965099+0000","last_unstale":"2024-04-25T09:54:30.086458+0000","last_undegraded":"2024-04-25T09:54:30.086458+0000","last_fullsized":"2024-04-25T09:54:30.086458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086552+0000","last_change":"2024-04-25T09:54:28.010667+0000","last_active":"2024-04-25T09:54:30.086552+0000","last_peered":"2024-04-25T09:54:30.086552+0000","last_clean":"2024-04-25T09:54:30.086552+0000","last_became_active":"2024-04-25T09:54:28.010243+0000","last_became_peered":"2024-04-25T09:54:28.010243+0000","last_unstale":"2024-04-25T09:54:30.086552+0000","last_undegraded":"2024-04-25T09:54:30.086552+0000","last_fullsized":"2024-04-25T09:54:30.086552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994577+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:29.994577+0000","last_peered":"2024-04-25T09:54:29.994577+0000","last_clean":"2024-04-25T09:54:29.994577+0000","last_became_active":"2024-04-25T09:54:23.970422+0000","last_became_peered":"2024-04-25T09:54:23.970422+0000","last_unstale":"2024-04-25T09:54:29.994577+0000","last_undegraded":"2024-04-25T09:54:29.994577+0000","last_fullsized":"2024-04-25T09:54:29.994577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994799+0000","last_change":"2024-04-25T09:54:22.960394+0000","last_active":"2024-04-25T09:54:29.994799+0000","last_peered":"2024-04-25T09:54:29.994799+0000","last_clean":"2024-04-25T09:54:29.994799+0000","last_became_active":"2024-04-25T09:54:22.960128+0000","last_became_peered":"2024-04-25T09:54:22.960128+0000","last_unstale":"2024-04-25T09:54:29.994799+0000","last_undegraded":"2024-04-25T09:54:29.994799+0000","last_fullsized":"2024-04-25T09:54:29.994799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086079+0000","last_change":"2024-04-25T09:54:22.962677+0000","last_active":"2024-04-25T09:54:30.086079+0000","last_peered":"2024-04-25T09:54:30.086079+0000","last_clean":"2024-04-25T09:54:30.086079+0000","last_became_active":"2024-04-25T09:54:22.962351+0000","last_became_peered":"2024-04-25T09:54:22.962351+0000","last_unstale":"2024-04-25T09:54:30.086079+0000","last_undegraded":"2024-04-25T09:54:30.086079+0000","last_fullsized":"2024-04-25T09:54:30.086079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086533+0000","last_change":"2024-04-25T09:54:23.973688+0000","last_active":"2024-04-25T09:54:30.086533+0000","last_peered":"2024-04-25T09:54:30.086533+0000","last_clean":"2024-04-25T09:54:30.086533+0000","last_became_active":"2024-04-25T09:54:23.973424+0000","last_became_peered":"2024-04-25T09:54:23.973424+0000","last_unstale":"2024-04-25T09:54:30.086533+0000","last_undegraded":"2024-04-25T09:54:30.086533+0000","last_fullsized":"2024-04-25T09:54:30.086533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086930+0000","last_change":"2024-04-25T09:54:28.016164+0000","last_active":"2024-04-25T09:54:30.086930+0000","last_peered":"2024-04-25T09:54:30.086930+0000","last_clean":"2024-04-25T09:54:30.086930+0000","last_became_active":"2024-04-25T09:54:28.015634+0000","last_became_peered":"2024-04-25T09:54:28.015634+0000","last_unstale":"2024-04-25T09:54:30.086930+0000","last_undegraded":"2024-04-25T09:54:30.086930+0000","last_fullsized":"2024-04-25T09:54:30.086930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.111330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994585+0000","last_change":"2024-04-25T09:54:22.962095+0000","last_active":"2024-04-25T09:54:29.994585+0000","last_peered":"2024-04-25T09:54:29.994585+0000","last_clean":"2024-04-25T09:54:29.994585+0000","last_became_active":"2024-04-25T09:54:22.961801+0000","last_became_peered":"2024-04-25T09:54:22.961801+0000","last_unstale":"2024-04-25T09:54:29.994585+0000","last_undegraded":"2024-04-25T09:54:29.994585+0000","last_fullsized":"2024-04-25T09:54:29.994585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995467+0000","last_change":"2024-04-25T09:54:23.971722+0000","last_active":"2024-04-25T09:54:29.995467+0000","last_peered":"2024-04-25T09:54:29.995467+0000","last_clean":"2024-04-25T09:54:29.995467+0000","last_became_active":"2024-04-25T09:54:23.971437+0000","last_became_peered":"2024-04-25T09:54:23.971437+0000","last_unstale":"2024-04-25T09:54:29.995467+0000","last_undegraded":"2024-04-25T09:54:29.995467+0000","last_fullsized":"2024-04-25T09:54:29.995467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:28.933707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994926+0000","last_change":"2024-04-25T09:54:28.016742+0000","last_active":"2024-04-25T09:54:29.994926+0000","last_peered":"2024-04-25T09:54:29.994926+0000","last_clean":"2024-04-25T09:54:29.994926+0000","last_became_active":"2024-04-25T09:54:28.016017+0000","last_became_peered":"2024-04-25T09:54:28.016017+0000","last_unstale":"2024-04-25T09:54:29.994926+0000","last_undegraded":"2024-04-25T09:54:29.994926+0000","last_fullsized":"2024-04-25T09:54:29.994926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:51.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085710+0000","last_change":"2024-04-25T09:54:23.976967+0000","last_active":"2024-04-25T09:54:30.085710+0000","last_peered":"2024-04-25T09:54:30.085710+0000","last_clean":"2024-04-25T09:54:30.085710+0000","last_became_active":"2024-04-25T09:54:23.976684+0000","last_became_peered":"2024-04-25T09:54:23.976684+0000","last_unstale":"2024-04-25T09:54:30.085710+0000","last_undegraded":"2024-04-25T09:54:30.085710+0000","last_fullsized":"2024-04-25T09:54:30.085710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086490+0000","last_change":"2024-04-25T09:54:23.968076+0000","last_active":"2024-04-25T09:54:30.086490+0000","last_peered":"2024-04-25T09:54:30.086490+0000","last_clean":"2024-04-25T09:54:30.086490+0000","last_became_active":"2024-04-25T09:54:23.967823+0000","last_became_peered":"2024-04-25T09:54:23.967823+0000","last_unstale":"2024-04-25T09:54:30.086490+0000","last_undegraded":"2024-04-25T09:54:30.086490+0000","last_fullsized":"2024-04-25T09:54:30.086490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087064+0000","last_change":"2024-04-25T09:54:23.974924+0000","last_active":"2024-04-25T09:54:30.087064+0000","last_peered":"2024-04-25T09:54:30.087064+0000","last_clean":"2024-04-25T09:54:30.087064+0000","last_became_active":"2024-04-25T09:54:23.974647+0000","last_became_peered":"2024-04-25T09:54:23.974647+0000","last_unstale":"2024-04-25T09:54:30.087064+0000","last_undegraded":"2024-04-25T09:54:30.087064+0000","last_fullsized":"2024-04-25T09:54:30.087064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086431+0000","last_change":"2024-04-25T09:54:23.974599+0000","last_active":"2024-04-25T09:54:30.086431+0000","last_peered":"2024-04-25T09:54:30.086431+0000","last_clean":"2024-04-25T09:54:30.086431+0000","last_became_active":"2024-04-25T09:54:23.973958+0000","last_became_peered":"2024-04-25T09:54:23.973958+0000","last_unstale":"2024-04-25T09:54:30.086431+0000","last_undegraded":"2024-04-25T09:54:30.086431+0000","last_fullsized":"2024-04-25T09:54:30.086431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994701+0000","last_change":"2024-04-25T09:54:23.976111+0000","last_active":"2024-04-25T09:54:29.994701+0000","last_peered":"2024-04-25T09:54:29.994701+0000","last_clean":"2024-04-25T09:54:29.994701+0000","last_became_active":"2024-04-25T09:54:23.975817+0000","last_became_peered":"2024-04-25T09:54:23.975817+0000","last_unstale":"2024-04-25T09:54:29.994701+0000","last_undegraded":"2024-04-25T09:54:29.994701+0000","last_fullsized":"2024-04-25T09:54:29.994701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086971+0000","last_change":"2024-04-25T09:54:23.967777+0000","last_active":"2024-04-25T09:54:30.086971+0000","last_peered":"2024-04-25T09:54:30.086971+0000","last_clean":"2024-04-25T09:54:30.086971+0000","last_became_active":"2024-04-25T09:54:23.967099+0000","last_became_peered":"2024-04-25T09:54:23.967099+0000","last_unstale":"2024-04-25T09:54:30.086971+0000","last_undegraded":"2024-04-25T09:54:30.086971+0000","last_fullsized":"2024-04-25T09:54:30.086971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086631+0000","last_change":"2024-04-25T09:54:23.982361+0000","last_active":"2024-04-25T09:54:30.086631+0000","last_peered":"2024-04-25T09:54:30.086631+0000","last_clean":"2024-04-25T09:54:30.086631+0000","last_became_active":"2024-04-25T09:54:23.982020+0000","last_became_peered":"2024-04-25T09:54:23.982020+0000","last_unstale":"2024-04-25T09:54:30.086631+0000","last_undegraded":"2024-04-25T09:54:30.086631+0000","last_fullsized":"2024-04-25T09:54:30.086631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995121+0000","last_change":"2024-04-25T09:54:23.976994+0000","last_active":"2024-04-25T09:54:29.995121+0000","last_peered":"2024-04-25T09:54:29.995121+0000","last_clean":"2024-04-25T09:54:29.995121+0000","last_became_active":"2024-04-25T09:54:23.976707+0000","last_became_peered":"2024-04-25T09:54:23.976707+0000","last_unstale":"2024-04-25T09:54:29.995121+0000","last_undegraded":"2024-04-25T09:54:29.995121+0000","last_fullsized":"2024-04-25T09:54:29.995121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086418+0000","last_change":"2024-04-25T09:54:23.977123+0000","last_active":"2024-04-25T09:54:30.086418+0000","last_peered":"2024-04-25T09:54:30.086418+0000","last_clean":"2024-04-25T09:54:30.086418+0000","last_became_active":"2024-04-25T09:54:23.976810+0000","last_became_peered":"2024-04-25T09:54:23.976810+0000","last_unstale":"2024-04-25T09:54:30.086418+0000","last_undegraded":"2024-04-25T09:54:30.086418+0000","last_fullsized":"2024-04-25T09:54:30.086418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994535+0000","last_change":"2024-04-25T09:54:23.976035+0000","last_active":"2024-04-25T09:54:29.994535+0000","last_peered":"2024-04-25T09:54:29.994535+0000","last_clean":"2024-04-25T09:54:29.994535+0000","last_became_active":"2024-04-25T09:54:23.975755+0000","last_became_peered":"2024-04-25T09:54:23.975755+0000","last_unstale":"2024-04-25T09:54:29.994535+0000","last_undegraded":"2024-04-25T09:54:29.994535+0000","last_fullsized":"2024-04-25T09:54:29.994535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086969+0000","last_change":"2024-04-25T09:54:23.973340+0000","last_active":"2024-04-25T09:54:30.086969+0000","last_peered":"2024-04-25T09:54:30.086969+0000","last_clean":"2024-04-25T09:54:30.086969+0000","last_became_active":"2024-04-25T09:54:23.973010+0000","last_became_peered":"2024-04-25T09:54:23.973010+0000","last_unstale":"2024-04-25T09:54:30.086969+0000","last_undegraded":"2024-04-25T09:54:30.086969+0000","last_fullsized":"2024-04-25T09:54:30.086969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086318+0000","last_change":"2024-04-25T09:54:23.980705+0000","last_active":"2024-04-25T09:54:30.086318+0000","last_peered":"2024-04-25T09:54:30.086318+0000","last_clean":"2024-04-25T09:54:30.086318+0000","last_became_active":"2024-04-25T09:54:23.980411+0000","last_became_peered":"2024-04-25T09:54:23.980411+0000","last_unstale":"2024-04-25T09:54:30.086318+0000","last_undegraded":"2024-04-25T09:54:30.086318+0000","last_fullsized":"2024-04-25T09:54:30.086318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086259+0000","last_change":"2024-04-25T09:54:23.984091+0000","last_active":"2024-04-25T09:54:30.086259+0000","last_peered":"2024-04-25T09:54:30.086259+0000","last_clean":"2024-04-25T09:54:30.086259+0000","last_became_active":"2024-04-25T09:54:23.983834+0000","last_became_peered":"2024-04-25T09:54:23.983834+0000","last_unstale":"2024-04-25T09:54:30.086259+0000","last_undegraded":"2024-04-25T09:54:30.086259+0000","last_fullsized":"2024-04-25T09:54:30.086259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088036+0000","last_change":"2024-04-25T09:54:23.983353+0000","last_active":"2024-04-25T09:54:30.088036+0000","last_peered":"2024-04-25T09:54:30.088036+0000","last_clean":"2024-04-25T09:54:30.088036+0000","last_became_active":"2024-04-25T09:54:23.983043+0000","last_became_peered":"2024-04-25T09:54:23.983043+0000","last_unstale":"2024-04-25T09:54:30.088036+0000","last_undegraded":"2024-04-25T09:54:30.088036+0000","last_fullsized":"2024-04-25T09:54:30.088036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086130+0000","last_change":"2024-04-25T09:54:23.977174+0000","last_active":"2024-04-25T09:54:30.086130+0000","last_peered":"2024-04-25T09:54:30.086130+0000","last_clean":"2024-04-25T09:54:30.086130+0000","last_became_active":"2024-04-25T09:54:23.976903+0000","last_became_peered":"2024-04-25T09:54:23.976903+0000","last_unstale":"2024-04-25T09:54:30.086130+0000","last_undegraded":"2024-04-25T09:54:30.086130+0000","last_fullsized":"2024-04-25T09:54:30.086130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.972492+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.972188+0000","last_became_peered":"2024-04-25T09:54:23.972188+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086368+0000","last_change":"2024-04-25T09:54:23.978371+0000","last_active":"2024-04-25T09:54:30.086368+0000","last_peered":"2024-04-25T09:54:30.086368+0000","last_clean":"2024-04-25T09:54:30.086368+0000","last_became_active":"2024-04-25T09:54:23.978079+0000","last_became_peered":"2024-04-25T09:54:23.978079+0000","last_unstale":"2024-04-25T09:54:30.086368+0000","last_undegraded":"2024-04-25T09:54:30.086368+0000","last_fullsized":"2024-04-25T09:54:30.086368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086280+0000","last_change":"2024-04-25T09:54:23.976031+0000","last_active":"2024-04-25T09:54:30.086280+0000","last_peered":"2024-04-25T09:54:30.086280+0000","last_clean":"2024-04-25T09:54:30.086280+0000","last_became_active":"2024-04-25T09:54:23.975750+0000","last_became_peered":"2024-04-25T09:54:23.975750+0000","last_unstale":"2024-04-25T09:54:30.086280+0000","last_undegraded":"2024-04-25T09:54:30.086280+0000","last_fullsized":"2024-04-25T09:54:30.086280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085955+0000","last_change":"2024-04-25T09:54:23.973561+0000","last_active":"2024-04-25T09:54:30.085955+0000","last_peered":"2024-04-25T09:54:30.085955+0000","last_clean":"2024-04-25T09:54:30.085955+0000","last_became_active":"2024-04-25T09:54:23.973312+0000","last_became_peered":"2024-04-25T09:54:23.973312+0000","last_unstale":"2024-04-25T09:54:30.085955+0000","last_undegraded":"2024-04-25T09:54:30.085955+0000","last_fullsized":"2024-04-25T09:54:30.085955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994677+0000","last_change":"2024-04-25T09:54:23.969586+0000","last_active":"2024-04-25T09:54:29.994677+0000","last_peered":"2024-04-25T09:54:29.994677+0000","last_clean":"2024-04-25T09:54:29.994677+0000","last_became_active":"2024-04-25T09:54:23.969067+0000","last_became_peered":"2024-04-25T09:54:23.969067+0000","last_unstale":"2024-04-25T09:54:29.994677+0000","last_undegraded":"2024-04-25T09:54:29.994677+0000","last_fullsized":"2024-04-25T09:54:29.994677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995863+0000","last_change":"2024-04-25T09:54:23.972621+0000","last_active":"2024-04-25T09:54:29.995863+0000","last_peered":"2024-04-25T09:54:29.995863+0000","last_clean":"2024-04-25T09:54:29.995863+0000","last_became_active":"2024-04-25T09:54:23.972338+0000","last_became_peered":"2024-04-25T09:54:23.972338+0000","last_unstale":"2024-04-25T09:54:29.995863+0000","last_undegraded":"2024-04-25T09:54:29.995863+0000","last_fullsized":"2024-04-25T09:54:29.995863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086120+0000","last_change":"2024-04-25T09:54:23.971085+0000","last_active":"2024-04-25T09:54:30.086120+0000","last_peered":"2024-04-25T09:54:30.086120+0000","last_clean":"2024-04-25T09:54:30.086120+0000","last_became_active":"2024-04-25T09:54:23.970354+0000","last_became_peered":"2024-04-25T09:54:23.970354+0000","last_unstale":"2024-04-25T09:54:30.086120+0000","last_undegraded":"2024-04-25T09:54:30.086120+0000","last_fullsized":"2024-04-25T09:54:30.086120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086765+0000","last_change":"2024-04-25T09:54:23.970478+0000","last_active":"2024-04-25T09:54:30.086765+0000","last_peered":"2024-04-25T09:54:30.086765+0000","last_clean":"2024-04-25T09:54:30.086765+0000","last_became_active":"2024-04-25T09:54:23.970181+0000","last_became_peered":"2024-04-25T09:54:23.970181+0000","last_unstale":"2024-04-25T09:54:30.086765+0000","last_undegraded":"2024-04-25T09:54:30.086765+0000","last_fullsized":"2024-04-25T09:54:30.086765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088285+0000","last_change":"2024-04-25T09:54:23.983654+0000","last_active":"2024-04-25T09:54:30.088285+0000","last_peered":"2024-04-25T09:54:30.088285+0000","last_clean":"2024-04-25T09:54:30.088285+0000","last_became_active":"2024-04-25T09:54:23.983388+0000","last_became_peered":"2024-04-25T09:54:23.983388+0000","last_unstale":"2024-04-25T09:54:30.088285+0000","last_undegraded":"2024-04-25T09:54:30.088285+0000","last_fullsized":"2024-04-25T09:54:30.088285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994460+0000","last_change":"2024-04-25T09:54:23.970049+0000","last_active":"2024-04-25T09:54:29.994460+0000","last_peered":"2024-04-25T09:54:29.994460+0000","last_clean":"2024-04-25T09:54:29.994460+0000","last_became_active":"2024-04-25T09:54:23.969605+0000","last_became_peered":"2024-04-25T09:54:23.969605+0000","last_unstale":"2024-04-25T09:54:29.994460+0000","last_undegraded":"2024-04-25T09:54:29.994460+0000","last_fullsized":"2024-04-25T09:54:29.994460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087574+0000","last_change":"2024-04-25T09:54:23.980721+0000","last_active":"2024-04-25T09:54:30.087574+0000","last_peered":"2024-04-25T09:54:30.087574+0000","last_clean":"2024-04-25T09:54:30.087574+0000","last_became_active":"2024-04-25T09:54:23.980440+0000","last_became_peered":"2024-04-25T09:54:23.980440+0000","last_unstale":"2024-04-25T09:54:30.087574+0000","last_undegraded":"2024-04-25T09:54:30.087574+0000","last_fullsized":"2024-04-25T09:54:30.087574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086883+0000","last_change":"2024-04-25T09:54:23.967904+0000","last_active":"2024-04-25T09:54:30.086883+0000","last_peered":"2024-04-25T09:54:30.086883+0000","last_clean":"2024-04-25T09:54:30.086883+0000","last_became_active":"2024-04-25T09:54:23.967585+0000","last_became_peered":"2024-04-25T09:54:23.967585+0000","last_unstale":"2024-04-25T09:54:30.086883+0000","last_undegraded":"2024-04-25T09:54:30.086883+0000","last_fullsized":"2024-04-25T09:54:30.086883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087670+0000","last_change":"2024-04-25T09:54:23.978911+0000","last_active":"2024-04-25T09:54:30.087670+0000","last_peered":"2024-04-25T09:54:30.087670+0000","last_clean":"2024-04-25T09:54:30.087670+0000","last_became_active":"2024-04-25T09:54:23.978625+0000","last_became_peered":"2024-04-25T09:54:23.978625+0000","last_unstale":"2024-04-25T09:54:30.087670+0000","last_undegraded":"2024-04-25T09:54:30.087670+0000","last_fullsized":"2024-04-25T09:54:30.087670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086202+0000","last_change":"2024-04-25T09:54:23.976392+0000","last_active":"2024-04-25T09:54:30.086202+0000","last_peered":"2024-04-25T09:54:30.086202+0000","last_clean":"2024-04-25T09:54:30.086202+0000","last_became_active":"2024-04-25T09:54:23.976117+0000","last_became_peered":"2024-04-25T09:54:23.976117+0000","last_unstale":"2024-04-25T09:54:30.086202+0000","last_undegraded":"2024-04-25T09:54:30.086202+0000","last_fullsized":"2024-04-25T09:54:30.086202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085739+0000","last_change":"2024-04-25T09:54:23.967710+0000","last_active":"2024-04-25T09:54:30.085739+0000","last_peered":"2024-04-25T09:54:30.085739+0000","last_clean":"2024-04-25T09:54:30.085739+0000","last_became_active":"2024-04-25T09:54:23.967075+0000","last_became_peered":"2024-04-25T09:54:23.967075+0000","last_unstale":"2024-04-25T09:54:30.085739+0000","last_undegraded":"2024-04-25T09:54:30.085739+0000","last_fullsized":"2024-04-25T09:54:30.085739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086748+0000","last_change":"2024-04-25T09:54:23.968977+0000","last_active":"2024-04-25T09:54:30.086748+0000","last_peered":"2024-04-25T09:54:30.086748+0000","last_clean":"2024-04-25T09:54:30.086748+0000","last_became_active":"2024-04-25T09:54:23.966936+0000","last_became_peered":"2024-04-25T09:54:23.966936+0000","last_unstale":"2024-04-25T09:54:30.086748+0000","last_undegraded":"2024-04-25T09:54:30.086748+0000","last_fullsized":"2024-04-25T09:54:30.086748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088179+0000","last_change":"2024-04-25T09:54:23.987914+0000","last_active":"2024-04-25T09:54:30.088179+0000","last_peered":"2024-04-25T09:54:30.088179+0000","last_clean":"2024-04-25T09:54:30.088179+0000","last_became_active":"2024-04-25T09:54:23.985923+0000","last_became_peered":"2024-04-25T09:54:23.985923+0000","last_unstale":"2024-04-25T09:54:30.088179+0000","last_undegraded":"2024-04-25T09:54:30.088179+0000","last_fullsized":"2024-04-25T09:54:30.088179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:46.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:46.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.174+0000 7f4a05922640 1 Processor -- start 2024-04-25T09:54:46.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.175+0000 7f4a05922640 1 -- start start 2024-04-25T09:54:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.175+0000 7f4a05922640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a000ad780 0x7f4a00152060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.175+0000 7f4a05922640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0006f9c0 con 0x7f4a000af270 2024-04-25T09:54:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.175+0000 7f4a05922640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000af8a0 con 0x7f4a0015c1d0 2024-04-25T09:54:46.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.175+0000 7f4a05922640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a00160370 con 0x7f4a000ad780 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.175+0000 7f49ff7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a000ad780 0x7f4a00152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49ff7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a000ad780 0x7f4a00152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33348/0 (socket says 172.21.15.148:33348) 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49fe7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4a000af270 0x7f4a000ad070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48678/0 (socket says 172.21.15.148:48678) 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49ff7fe640 1 -- 172.21.15.148:0/138924587 learned_addr learned my addr 172.21.15.148:0/138924587 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49ff7fe640 1 -- 172.21.15.148:0/138924587 >> v1:172.21.15.188:6789/0 conn(0x7f4a0015c1d0 legacy=0x7f4a0015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49ff7fe640 1 -- 172.21.15.148:0/138924587 >> v1:172.21.15.148:6789/0 conn(0x7f4a000af270 legacy=0x7f4a000ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49ff7fe640 1 -- 172.21.15.148:0/138924587 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0015f5b0 con 0x7f4a000ad780 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.176+0000 7f49ff7fe640 1 --2- 172.21.15.148:0/138924587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000ad780 0x7f4a00152060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e80 tx=0x7f49f002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f2e603b22a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f49fdffb640 1 -- 172.21.15.148:0/138924587 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0036070 con 0x7f4a000ad780 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f49fdffb640 1 -- 172.21.15.148:0/138924587 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f0031600 con 0x7f4a000ad780 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f49fdffb640 1 -- 172.21.15.148:0/138924587 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00319d0 con 0x7f4a000ad780 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 -- 172.21.15.148:0/138924587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000ad780 msgr2=0x7f4a00152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 --2- 172.21.15.148:0/138924587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000ad780 0x7f4a00152060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e80 tx=0x7f49f002a750 comp rx=0 tx=0).stop 2024-04-25T09:54:46.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 -- 172.21.15.148:0/138924587 shutdown_connections 2024-04-25T09:54:46.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 --2- 172.21.15.148:0/138924587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000ad780 0x7f4a00152060 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 -- 172.21.15.148:0/138924587 >> 172.21.15.148:0/138924587 conn(0x7f4a000cfe10 msgr2=0x7f4a000d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 -- 172.21.15.148:0/138924587 shutdown_connections 2024-04-25T09:54:46.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.177+0000 7f4a05922640 1 -- 172.21.15.148:0/138924587 wait complete. 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.178+0000 7f4a05922640 1 Processor -- start 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 -- start start 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a000ad780 0x7f4a0014e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 0x7f4a000cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a0015c1d0 0x7f4a0015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0015fe50 con 0x7f4a000af270 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0015fb70 con 0x7f4a000ad780 2024-04-25T09:54:46.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f4a05922640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006bbf0 con 0x7f4a0015c1d0 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f49fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 0x7f4a000cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f49fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 0x7f4a000cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33362/0 (socket says 172.21.15.148:33362) 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.179+0000 7f49fe7fc640 1 -- 172.21.15.148:0/3313396157 learned_addr learned my addr 172.21.15.148:0/3313396157 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49feffd640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a000ad780 0x7f4a0014e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49ff7fe640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a0015c1d0 0x7f4a0015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49fe7fc640 1 -- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a0015c1d0 msgr2=0x7f4a0015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49fe7fc640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a0015c1d0 0x7f4a0015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49fe7fc640 1 -- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a000ad780 msgr2=0x7f4a0014e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49fe7fc640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a000ad780 0x7f4a0014e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49fe7fc640 1 -- 172.21.15.148:0/3313396157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000ccd90 con 0x7f4a000af270 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49fe7fc640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 0x7f4a000cc440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f49f400c4d0 tx=0x7f49f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4018070 con 0x7f4a000af270 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4007480 con 0x7f4a000af270 2024-04-25T09:54:46.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a000cd070 con 0x7f4a000af270 2024-04-25T09:54:46.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40049e0 con 0x7f4a000af270 2024-04-25T09:54:46.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.180+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a000cd630 con 0x7f4a000af270 2024-04-25T09:54:46.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.181+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a000cd800 con 0x7f4a000af270 2024-04-25T09:54:46.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.181+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49f401b070 con 0x7f4a000af270 2024-04-25T09:54:46.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.182+0000 7f49e7fff640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49ec06b4d0 0x7f49ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.182+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49f40898f0 con 0x7f4a000af270 2024-04-25T09:54:46.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.182+0000 7f49feffd640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49ec06b4d0 0x7f49ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.182+0000 7f49feffd640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49ec06b4d0 0x7f49ec06d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007fd0 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.186+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a0006f9c0 con 0x7f4a000af270 2024-04-25T09:54:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.356+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a00061b70 con 0x7f49ec06b4d0 2024-04-25T09:54:46.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.363+0000 7f49e7fff640 1 -- 172.21.15.148:0/3313396157 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f4a00061b70 con 0x7f49ec06b4d0 2024-04-25T09:54:46.370 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:46.371 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:54:46.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49ec06b4d0 msgr2=0x7f49ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49ec06b4d0 0x7f49ec06d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007fd0 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-04-25T09:54:46.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 msgr2=0x7f4a000cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 0x7f4a000cc440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f49f400c4d0 tx=0x7f49f400c990 comp rx=0 tx=0).stop 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 shutdown_connections 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49ec06b4d0 0x7f49ec06d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a0015c1d0 0x7f4a0015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a000af270 0x7f4a000cc440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 --2- 172.21.15.148:0/3313396157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a000ad780 0x7f4a0014e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 >> 172.21.15.148:0/3313396157 conn(0x7f4a000cfe10 msgr2=0x7f4a0006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 shutdown_connections 2024-04-25T09:54:46.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.367+0000 7f4a05922640 1 -- 172.21.15.148:0/3313396157 wait complete. 2024-04-25T09:54:46.383 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T09:54:45.024391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086122+0000","last_change":"2024-04-25T09:54:22.962688+0000","last_active":"2024-04-25T09:54:30.086122+0000","last_peered":"2024-04-25T09:54:30.086122+0000","last_clean":"2024-04-25T09:54:30.086122+0000","last_became_active":"2024-04-25T09:54:22.962426+0000","last_became_peered":"2024-04-25T09:54:22.962426+0000","last_unstale":"2024-04-25T09:54:30.086122+0000","last_undegraded":"2024-04-25T09:54:30.086122+0000","last_fullsized":"2024-04-25T09:54:30.086122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:21.016302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995003+0000","last_change":"2024-04-25T09:54:23.969918+0000","last_active":"2024-04-25T09:54:29.995003+0000","last_peered":"2024-04-25T09:54:29.995003+0000","last_clean":"2024-04-25T09:54:29.995003+0000","last_became_active":"2024-04-25T09:54:23.969631+0000","last_became_peered":"2024-04-25T09:54:23.969631+0000","last_unstale":"2024-04-25T09:54:29.995003+0000","last_undegraded":"2024-04-25T09:54:29.995003+0000","last_fullsized":"2024-04-25T09:54:29.995003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994303+0000","last_change":"2024-04-25T09:54:28.397901+0000","last_active":"2024-04-25T09:54:29.994303+0000","last_peered":"2024-04-25T09:54:29.994303+0000","last_clean":"2024-04-25T09:54:29.994303+0000","last_became_active":"2024-04-25T09:54:28.397492+0000","last_became_peered":"2024-04-25T09:54:28.397492+0000","last_unstale":"2024-04-25T09:54:29.994303+0000","last_undegraded":"2024-04-25T09:54:29.994303+0000","last_fullsized":"2024-04-25T09:54:29.994303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:14.336697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995250+0000","last_change":"2024-04-25T09:54:22.958345+0000","last_active":"2024-04-25T09:54:29.995250+0000","last_peered":"2024-04-25T09:54:29.995250+0000","last_clean":"2024-04-25T09:54:29.995250+0000","last_became_active":"2024-04-25T09:54:22.958021+0000","last_became_peered":"2024-04-25T09:54:22.958021+0000","last_unstale":"2024-04-25T09:54:29.995250+0000","last_undegraded":"2024-04-25T09:54:29.995250+0000","last_fullsized":"2024-04-25T09:54:29.995250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:02.625621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085905+0000","last_change":"2024-04-25T09:54:23.970381+0000","last_active":"2024-04-25T09:54:30.085905+0000","last_peered":"2024-04-25T09:54:30.085905+0000","last_clean":"2024-04-25T09:54:30.085905+0000","last_became_active":"2024-04-25T09:54:23.970077+0000","last_became_peered":"2024-04-25T09:54:23.970077+0000","last_unstale":"2024-04-25T09:54:30.085905+0000","last_undegraded":"2024-04-25T09:54:30.085905+0000","last_fullsized":"2024-04-25T09:54:30.085905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:16.071642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995354+0000","last_change":"2024-04-25T09:54:28.011784+0000","last_active":"2024-04-25T09:54:29.995354+0000","last_peered":"2024-04-25T09:54:29.995354+0000","last_clean":"2024-04-25T09:54:29.995354+0000","last_became_active":"2024-04-25T09:54:28.011370+0000","last_became_peered":"2024-04-25T09:54:28.011370+0000","last_unstale":"2024-04-25T09:54:29.995354+0000","last_undegraded":"2024-04-25T09:54:29.995354+0000","last_fullsized":"2024-04-25T09:54:29.995354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088011+0000","last_change":"2024-04-25T09:54:22.965546+0000","last_active":"2024-04-25T09:54:30.088011+0000","last_peered":"2024-04-25T09:54:30.088011+0000","last_clean":"2024-04-25T09:54:30.088011+0000","last_became_active":"2024-04-25T09:54:22.965088+0000","last_became_peered":"2024-04-25T09:54:22.965088+0000","last_unstale":"2024-04-25T09:54:30.088011+0000","last_undegraded":"2024-04-25T09:54:30.088011+0000","last_fullsized":"2024-04-25T09:54:30.088011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:23.982573+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:23.982304+0000","last_became_peered":"2024-04-25T09:54:23.982304+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995558+0000","last_change":"2024-04-25T09:54:28.012195+0000","last_active":"2024-04-25T09:54:29.995558+0000","last_peered":"2024-04-25T09:54:29.995558+0000","last_clean":"2024-04-25T09:54:29.995558+0000","last_became_active":"2024-04-25T09:54:28.011667+0000","last_became_peered":"2024-04-25T09:54:28.011667+0000","last_unstale":"2024-04-25T09:54:29.995558+0000","last_undegraded":"2024-04-25T09:54:29.995558+0000","last_fullsized":"2024-04-25T09:54:29.995558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087654+0000","last_change":"2024-04-25T09:54:22.967092+0000","last_active":"2024-04-25T09:54:30.087654+0000","last_peered":"2024-04-25T09:54:30.087654+0000","last_clean":"2024-04-25T09:54:30.087654+0000","last_became_active":"2024-04-25T09:54:22.966690+0000","last_became_peered":"2024-04-25T09:54:22.966690+0000","last_unstale":"2024-04-25T09:54:30.087654+0000","last_undegraded":"2024-04-25T09:54:30.087654+0000","last_fullsized":"2024-04-25T09:54:30.087654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087289+0000","last_change":"2024-04-25T09:54:23.979036+0000","last_active":"2024-04-25T09:54:30.087289+0000","last_peered":"2024-04-25T09:54:30.087289+0000","last_clean":"2024-04-25T09:54:30.087289+0000","last_became_active":"2024-04-25T09:54:23.978775+0000","last_became_peered":"2024-04-25T09:54:23.978775+0000","last_unstale":"2024-04-25T09:54:30.087289+0000","last_undegraded":"2024-04-25T09:54:30.087289+0000","last_fullsized":"2024-04-25T09:54:30.087289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995398+0000","last_change":"2024-04-25T09:54:28.006708+0000","last_active":"2024-04-25T09:54:29.995398+0000","last_peered":"2024-04-25T09:54:29.995398+0000","last_clean":"2024-04-25T09:54:29.995398+0000","last_became_active":"2024-04-25T09:54:28.006351+0000","last_became_peered":"2024-04-25T09:54:28.006351+0000","last_unstale":"2024-04-25T09:54:29.995398+0000","last_undegraded":"2024-04-25T09:54:29.995398+0000","last_fullsized":"2024-04-25T09:54:29.995398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085947+0000","last_change":"2024-04-25T09:54:23.983452+0000","last_active":"2024-04-25T09:54:30.085947+0000","last_peered":"2024-04-25T09:54:30.085947+0000","last_clean":"2024-04-25T09:54:30.085947+0000","last_became_active":"2024-04-25T09:54:23.983150+0000","last_became_peered":"2024-04-25T09:54:23.983150+0000","last_unstale":"2024-04-25T09:54:30.085947+0000","last_undegraded":"2024-04-25T09:54:30.085947+0000","last_fullsized":"2024-04-25T09:54:30.085947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087835+0000","last_change":"2024-04-25T09:54:22.965628+0000","last_active":"2024-04-25T09:54:30.087835+0000","last_peered":"2024-04-25T09:54:30.087835+0000","last_clean":"2024-04-25T09:54:30.087835+0000","last_became_active":"2024-04-25T09:54:22.965294+0000","last_became_peered":"2024-04-25T09:54:22.965294+0000","last_unstale":"2024-04-25T09:54:30.087835+0000","last_undegraded":"2024-04-25T09:54:30.087835+0000","last_fullsized":"2024-04-25T09:54:30.087835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996313+0000","last_change":"2024-04-25T09:54:28.011196+0000","last_active":"2024-04-25T09:54:29.996313+0000","last_peered":"2024-04-25T09:54:29.996313+0000","last_clean":"2024-04-25T09:54:29.996313+0000","last_became_active":"2024-04-25T09:54:28.010638+0000","last_became_peered":"2024-04-25T09:54:28.010638+0000","last_unstale":"2024-04-25T09:54:29.996313+0000","last_undegraded":"2024-04-25T09:54:29.996313+0000","last_fullsized":"2024-04-25T09:54:29.996313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994468+0000","last_change":"2024-04-25T09:54:23.975061+0000","last_active":"2024-04-25T09:54:29.994468+0000","last_peered":"2024-04-25T09:54:29.994468+0000","last_clean":"2024-04-25T09:54:29.994468+0000","last_became_active":"2024-04-25T09:54:23.974793+0000","last_became_peered":"2024-04-25T09:54:23.974793+0000","last_unstale":"2024-04-25T09:54:29.994468+0000","last_undegraded":"2024-04-25T09:54:29.994468+0000","last_fullsized":"2024-04-25T09:54:29.994468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086013+0000","last_change":"2024-04-25T09:54:22.965353+0000","last_active":"2024-04-25T09:54:30.086013+0000","last_peered":"2024-04-25T09:54:30.086013+0000","last_clean":"2024-04-25T09:54:30.086013+0000","last_became_active":"2024-04-25T09:54:22.965038+0000","last_became_peered":"2024-04-25T09:54:22.965038+0000","last_unstale":"2024-04-25T09:54:30.086013+0000","last_undegraded":"2024-04-25T09:54:30.086013+0000","last_fullsized":"2024-04-25T09:54:30.086013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996196+0000","last_change":"2024-04-25T09:54:28.005662+0000","last_active":"2024-04-25T09:54:29.996196+0000","last_peered":"2024-04-25T09:54:29.996196+0000","last_clean":"2024-04-25T09:54:29.996196+0000","last_became_active":"2024-04-25T09:54:28.005233+0000","last_became_peered":"2024-04-25T09:54:28.005233+0000","last_unstale":"2024-04-25T09:54:29.996196+0000","last_undegraded":"2024-04-25T09:54:29.996196+0000","last_fullsized":"2024-04-25T09:54:29.996196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994816+0000","last_change":"2024-04-25T09:54:23.969844+0000","last_active":"2024-04-25T09:54:29.994816+0000","last_peered":"2024-04-25T09:54:29.994816+0000","last_clean":"2024-04-25T09:54:29.994816+0000","last_became_active":"2024-04-25T09:54:23.969261+0000","last_became_peered":"2024-04-25T09:54:23.969261+0000","last_unstale":"2024-04-25T09:54:29.994816+0000","last_undegraded":"2024-04-25T09:54:29.994816+0000","last_fullsized":"2024-04-25T09:54:29.994816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086366+0000","last_change":"2024-04-25T09:54:22.961764+0000","last_active":"2024-04-25T09:54:30.086366+0000","last_peered":"2024-04-25T09:54:30.086366+0000","last_clean":"2024-04-25T09:54:30.086366+0000","last_became_active":"2024-04-25T09:54:22.961455+0000","last_became_peered":"2024-04-25T09:54:22.961455+0000","last_unstale":"2024-04-25T09:54:30.086366+0000","last_undegraded":"2024-04-25T09:54:30.086366+0000","last_fullsized":"2024-04-25T09:54:30.086366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087099+0000","last_change":"2024-04-25T09:54:28.395240+0000","last_active":"2024-04-25T09:54:30.087099+0000","last_peered":"2024-04-25T09:54:30.087099+0000","last_clean":"2024-04-25T09:54:30.087099+0000","last_became_active":"2024-04-25T09:54:28.394866+0000","last_became_peered":"2024-04-25T09:54:28.394866+0000","last_unstale":"2024-04-25T09:54:30.087099+0000","last_undegraded":"2024-04-25T09:54:30.087099+0000","last_fullsized":"2024-04-25T09:54:30.087099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995771+0000","last_change":"2024-04-25T09:54:23.973436+0000","last_active":"2024-04-25T09:54:29.995771+0000","last_peered":"2024-04-25T09:54:29.995771+0000","last_clean":"2024-04-25T09:54:29.995771+0000","last_became_active":"2024-04-25T09:54:23.973139+0000","last_became_peered":"2024-04-25T09:54:23.973139+0000","last_unstale":"2024-04-25T09:54:29.995771+0000","last_undegraded":"2024-04-25T09:54:29.995771+0000","last_fullsized":"2024-04-25T09:54:29.995771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086953+0000","last_change":"2024-04-25T09:54:22.963445+0000","last_active":"2024-04-25T09:54:30.086953+0000","last_peered":"2024-04-25T09:54:30.086953+0000","last_clean":"2024-04-25T09:54:30.086953+0000","last_became_active":"2024-04-25T09:54:22.963190+0000","last_became_peered":"2024-04-25T09:54:22.963190+0000","last_unstale":"2024-04-25T09:54:30.086953+0000","last_undegraded":"2024-04-25T09:54:30.086953+0000","last_fullsized":"2024-04-25T09:54:30.086953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996305+0000","last_change":"2024-04-25T09:54:28.011236+0000","last_active":"2024-04-25T09:54:29.996305+0000","last_peered":"2024-04-25T09:54:29.996305+0000","last_clean":"2024-04-25T09:54:29.996305+0000","last_became_active":"2024-04-25T09:54:28.010633+0000","last_became_peered":"2024-04-25T09:54:28.010633+0000","last_unstale":"2024-04-25T09:54:29.996305+0000","last_undegraded":"2024-04-25T09:54:29.996305+0000","last_fullsized":"2024-04-25T09:54:29.996305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086754+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:30.086754+0000","last_peered":"2024-04-25T09:54:30.086754+0000","last_clean":"2024-04-25T09:54:30.086754+0000","last_became_active":"2024-04-25T09:54:23.970411+0000","last_became_peered":"2024-04-25T09:54:23.970411+0000","last_unstale":"2024-04-25T09:54:30.086754+0000","last_undegraded":"2024-04-25T09:54:30.086754+0000","last_fullsized":"2024-04-25T09:54:30.086754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:20.071529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086786+0000","last_change":"2024-04-25T09:54:22.961585+0000","last_active":"2024-04-25T09:54:30.086786+0000","last_peered":"2024-04-25T09:54:30.086786+0000","last_clean":"2024-04-25T09:54:30.086786+0000","last_became_active":"2024-04-25T09:54:22.961286+0000","last_became_peered":"2024-04-25T09:54:22.961286+0000","last_unstale":"2024-04-25T09:54:30.086786+0000","last_undegraded":"2024-04-25T09:54:30.086786+0000","last_fullsized":"2024-04-25T09:54:30.086786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995453+0000","last_change":"2024-04-25T09:54:28.396021+0000","last_active":"2024-04-25T09:54:29.995453+0000","last_peered":"2024-04-25T09:54:29.995453+0000","last_clean":"2024-04-25T09:54:29.995453+0000","last_became_active":"2024-04-25T09:54:28.394961+0000","last_became_peered":"2024-04-25T09:54:28.394961+0000","last_unstale":"2024-04-25T09:54:29.995453+0000","last_undegraded":"2024-04-25T09:54:29.995453+0000","last_fullsized":"2024-04-25T09:54:29.995453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995224+0000","last_change":"2024-04-25T09:54:23.981659+0000","last_active":"2024-04-25T09:54:29.995224+0000","last_peered":"2024-04-25T09:54:29.995224+0000","last_clean":"2024-04-25T09:54:29.995224+0000","last_became_active":"2024-04-25T09:54:23.981345+0000","last_became_peered":"2024-04-25T09:54:23.981345+0000","last_unstale":"2024-04-25T09:54:29.995224+0000","last_undegraded":"2024-04-25T09:54:29.995224+0000","last_fullsized":"2024-04-25T09:54:29.995224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086841+0000","last_change":"2024-04-25T09:54:22.961799+0000","last_active":"2024-04-25T09:54:30.086841+0000","last_peered":"2024-04-25T09:54:30.086841+0000","last_clean":"2024-04-25T09:54:30.086841+0000","last_became_active":"2024-04-25T09:54:22.961205+0000","last_became_peered":"2024-04-25T09:54:22.961205+0000","last_unstale":"2024-04-25T09:54:30.086841+0000","last_undegraded":"2024-04-25T09:54:30.086841+0000","last_fullsized":"2024-04-25T09:54:30.086841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994244+0000","last_change":"2024-04-25T09:54:28.397432+0000","last_active":"2024-04-25T09:54:29.994244+0000","last_peered":"2024-04-25T09:54:29.994244+0000","last_clean":"2024-04-25T09:54:29.994244+0000","last_became_active":"2024-04-25T09:54:28.396060+0000","last_became_peered":"2024-04-25T09:54:28.396060+0000","last_unstale":"2024-04-25T09:54:29.994244+0000","last_undegraded":"2024-04-25T09:54:29.994244+0000","last_fullsized":"2024-04-25T09:54:29.994244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:18.141331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086215+0000","last_change":"2024-04-25T09:54:23.964362+0000","last_active":"2024-04-25T09:54:30.086215+0000","last_peered":"2024-04-25T09:54:30.086215+0000","last_clean":"2024-04-25T09:54:30.086215+0000","last_became_active":"2024-04-25T09:54:23.964097+0000","last_became_peered":"2024-04-25T09:54:23.964097+0000","last_unstale":"2024-04-25T09:54:30.086215+0000","last_undegraded":"2024-04-25T09:54:30.086215+0000","last_fullsized":"2024-04-25T09:54:30.086215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088121+0000","last_change":"2024-04-25T09:54:22.967489+0000","last_active":"2024-04-25T09:54:30.088121+0000","last_peered":"2024-04-25T09:54:30.088121+0000","last_clean":"2024-04-25T09:54:30.088121+0000","last_became_active":"2024-04-25T09:54:22.967191+0000","last_became_peered":"2024-04-25T09:54:22.967191+0000","last_unstale":"2024-04-25T09:54:30.088121+0000","last_undegraded":"2024-04-25T09:54:30.088121+0000","last_fullsized":"2024-04-25T09:54:30.088121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086235+0000","last_change":"2024-04-25T09:54:28.394981+0000","last_active":"2024-04-25T09:54:30.086235+0000","last_peered":"2024-04-25T09:54:30.086235+0000","last_clean":"2024-04-25T09:54:30.086235+0000","last_became_active":"2024-04-25T09:54:28.394572+0000","last_became_peered":"2024-04-25T09:54:28.394572+0000","last_unstale":"2024-04-25T09:54:30.086235+0000","last_undegraded":"2024-04-25T09:54:30.086235+0000","last_fullsized":"2024-04-25T09:54:30.086235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.977167+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.976581+0000","last_became_peered":"2024-04-25T09:54:23.976581+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086309+0000","last_change":"2024-04-25T09:54:22.957000+0000","last_active":"2024-04-25T09:54:30.086309+0000","last_peered":"2024-04-25T09:54:30.086309+0000","last_clean":"2024-04-25T09:54:30.086309+0000","last_became_active":"2024-04-25T09:54:22.956737+0000","last_became_peered":"2024-04-25T09:54:22.956737+0000","last_unstale":"2024-04-25T09:54:30.086309+0000","last_undegraded":"2024-04-25T09:54:30.086309+0000","last_fullsized":"2024-04-25T09:54:30.086309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085588+0000","last_change":"2024-04-25T09:54:28.009926+0000","last_active":"2024-04-25T09:54:30.085588+0000","last_peered":"2024-04-25T09:54:30.085588+0000","last_clean":"2024-04-25T09:54:30.085588+0000","last_became_active":"2024-04-25T09:54:28.009486+0000","last_became_peered":"2024-04-25T09:54:28.009486+0000","last_unstale":"2024-04-25T09:54:30.085588+0000","last_undegraded":"2024-04-25T09:54:30.085588+0000","last_fullsized":"2024-04-25T09:54:30.085588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996132+0000","last_change":"2024-04-25T09:54:23.974850+0000","last_active":"2024-04-25T09:54:29.996132+0000","last_peered":"2024-04-25T09:54:29.996132+0000","last_clean":"2024-04-25T09:54:29.996132+0000","last_became_active":"2024-04-25T09:54:23.974014+0000","last_became_peered":"2024-04-25T09:54:23.974014+0000","last_unstale":"2024-04-25T09:54:29.996132+0000","last_undegraded":"2024-04-25T09:54:29.996132+0000","last_fullsized":"2024-04-25T09:54:29.996132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087242+0000","last_change":"2024-04-25T09:54:22.966563+0000","last_active":"2024-04-25T09:54:30.087242+0000","last_peered":"2024-04-25T09:54:30.087242+0000","last_clean":"2024-04-25T09:54:30.087242+0000","last_became_active":"2024-04-25T09:54:22.966272+0000","last_became_peered":"2024-04-25T09:54:22.966272+0000","last_unstale":"2024-04-25T09:54:30.087242+0000","last_undegraded":"2024-04-25T09:54:30.087242+0000","last_fullsized":"2024-04-25T09:54:30.087242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086121+0000","last_change":"2024-04-25T09:54:28.013312+0000","last_active":"2024-04-25T09:54:30.086121+0000","last_peered":"2024-04-25T09:54:30.086121+0000","last_clean":"2024-04-25T09:54:30.086121+0000","last_became_active":"2024-04-25T09:54:28.012953+0000","last_became_peered":"2024-04-25T09:54:28.012953+0000","last_unstale":"2024-04-25T09:54:30.086121+0000","last_undegraded":"2024-04-25T09:54:30.086121+0000","last_fullsized":"2024-04-25T09:54:30.086121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087003+0000","last_change":"2024-04-25T09:54:23.966466+0000","last_active":"2024-04-25T09:54:30.087003+0000","last_peered":"2024-04-25T09:54:30.087003+0000","last_clean":"2024-04-25T09:54:30.087003+0000","last_became_active":"2024-04-25T09:54:23.966129+0000","last_became_peered":"2024-04-25T09:54:23.966129+0000","last_unstale":"2024-04-25T09:54:30.087003+0000","last_undegraded":"2024-04-25T09:54:30.087003+0000","last_fullsized":"2024-04-25T09:54:30.087003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086581+0000","last_change":"2024-04-25T09:54:22.963150+0000","last_active":"2024-04-25T09:54:30.086581+0000","last_peered":"2024-04-25T09:54:30.086581+0000","last_clean":"2024-04-25T09:54:30.086581+0000","last_became_active":"2024-04-25T09:54:22.962832+0000","last_became_peered":"2024-04-25T09:54:22.962832+0000","last_unstale":"2024-04-25T09:54:30.086581+0000","last_undegraded":"2024-04-25T09:54:30.086581+0000","last_fullsized":"2024-04-25T09:54:30.086581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087126+0000","last_change":"2024-04-25T09:54:28.011051+0000","last_active":"2024-04-25T09:54:30.087126+0000","last_peered":"2024-04-25T09:54:30.087126+0000","last_clean":"2024-04-25T09:54:30.087126+0000","last_became_active":"2024-04-25T09:54:28.010629+0000","last_became_peered":"2024-04-25T09:54:28.010629+0000","last_unstale":"2024-04-25T09:54:30.087126+0000","last_undegraded":"2024-04-25T09:54:30.087126+0000","last_fullsized":"2024-04-25T09:54:30.087126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085547+0000","last_change":"2024-04-25T09:54:23.973638+0000","last_active":"2024-04-25T09:54:30.085547+0000","last_peered":"2024-04-25T09:54:30.085547+0000","last_clean":"2024-04-25T09:54:30.085547+0000","last_became_active":"2024-04-25T09:54:23.973349+0000","last_became_peered":"2024-04-25T09:54:23.973349+0000","last_unstale":"2024-04-25T09:54:30.085547+0000","last_undegraded":"2024-04-25T09:54:30.085547+0000","last_fullsized":"2024-04-25T09:54:30.085547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087072+0000","last_change":"2024-04-25T09:54:22.966560+0000","last_active":"2024-04-25T09:54:30.087072+0000","last_peered":"2024-04-25T09:54:30.087072+0000","last_clean":"2024-04-25T09:54:30.087072+0000","last_became_active":"2024-04-25T09:54:22.966299+0000","last_became_peered":"2024-04-25T09:54:22.966299+0000","last_unstale":"2024-04-25T09:54:30.087072+0000","last_undegraded":"2024-04-25T09:54:30.087072+0000","last_fullsized":"2024-04-25T09:54:30.087072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086483+0000","last_change":"2024-04-25T09:54:28.395179+0000","last_active":"2024-04-25T09:54:30.086483+0000","last_peered":"2024-04-25T09:54:30.086483+0000","last_clean":"2024-04-25T09:54:30.086483+0000","last_became_active":"2024-04-25T09:54:28.394767+0000","last_became_peered":"2024-04-25T09:54:28.394767+0000","last_unstale":"2024-04-25T09:54:30.086483+0000","last_undegraded":"2024-04-25T09:54:30.086483+0000","last_fullsized":"2024-04-25T09:54:30.086483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994411+0000","last_change":"2024-04-25T09:54:23.969956+0000","last_active":"2024-04-25T09:54:29.994411+0000","last_peered":"2024-04-25T09:54:29.994411+0000","last_clean":"2024-04-25T09:54:29.994411+0000","last_became_active":"2024-04-25T09:54:23.969643+0000","last_became_peered":"2024-04-25T09:54:23.969643+0000","last_unstale":"2024-04-25T09:54:29.994411+0000","last_undegraded":"2024-04-25T09:54:29.994411+0000","last_fullsized":"2024-04-25T09:54:29.994411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086619+0000","last_change":"2024-04-25T09:54:22.965368+0000","last_active":"2024-04-25T09:54:30.086619+0000","last_peered":"2024-04-25T09:54:30.086619+0000","last_clean":"2024-04-25T09:54:30.086619+0000","last_became_active":"2024-04-25T09:54:22.965051+0000","last_became_peered":"2024-04-25T09:54:22.965051+0000","last_unstale":"2024-04-25T09:54:30.086619+0000","last_undegraded":"2024-04-25T09:54:30.086619+0000","last_fullsized":"2024-04-25T09:54:30.086619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087177+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:54:30.087177+0000","last_peered":"2024-04-25T09:54:30.087177+0000","last_clean":"2024-04-25T09:54:30.087177+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:54:30.087177+0000","last_undegraded":"2024-04-25T09:54:30.087177+0000","last_fullsized":"2024-04-25T09:54:30.087177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085975+0000","last_change":"2024-04-25T09:54:28.396320+0000","last_active":"2024-04-25T09:54:30.085975+0000","last_peered":"2024-04-25T09:54:30.085975+0000","last_clean":"2024-04-25T09:54:30.085975+0000","last_became_active":"2024-04-25T09:54:28.395895+0000","last_became_peered":"2024-04-25T09:54:28.395895+0000","last_unstale":"2024-04-25T09:54:30.085975+0000","last_undegraded":"2024-04-25T09:54:30.085975+0000","last_fullsized":"2024-04-25T09:54:30.085975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087913+0000","last_change":"2024-04-25T09:54:23.983959+0000","last_active":"2024-04-25T09:54:30.087913+0000","last_peered":"2024-04-25T09:54:30.087913+0000","last_clean":"2024-04-25T09:54:30.087913+0000","last_became_active":"2024-04-25T09:54:23.983669+0000","last_became_peered":"2024-04-25T09:54:23.983669+0000","last_unstale":"2024-04-25T09:54:30.087913+0000","last_undegraded":"2024-04-25T09:54:30.087913+0000","last_fullsized":"2024-04-25T09:54:30.087913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087174+0000","last_change":"2024-04-25T09:54:22.966591+0000","last_active":"2024-04-25T09:54:30.087174+0000","last_peered":"2024-04-25T09:54:30.087174+0000","last_clean":"2024-04-25T09:54:30.087174+0000","last_became_active":"2024-04-25T09:54:22.966301+0000","last_became_peered":"2024-04-25T09:54:22.966301+0000","last_unstale":"2024-04-25T09:54:30.087174+0000","last_undegraded":"2024-04-25T09:54:30.087174+0000","last_fullsized":"2024-04-25T09:54:30.087174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085806+0000","last_change":"2024-04-25T09:54:28.395577+0000","last_active":"2024-04-25T09:54:30.085806+0000","last_peered":"2024-04-25T09:54:30.085806+0000","last_clean":"2024-04-25T09:54:30.085806+0000","last_became_active":"2024-04-25T09:54:28.395067+0000","last_became_peered":"2024-04-25T09:54:28.395067+0000","last_unstale":"2024-04-25T09:54:30.085806+0000","last_undegraded":"2024-04-25T09:54:30.085806+0000","last_fullsized":"2024-04-25T09:54:30.085806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086598+0000","last_change":"2024-04-25T09:54:23.975660+0000","last_active":"2024-04-25T09:54:30.086598+0000","last_peered":"2024-04-25T09:54:30.086598+0000","last_clean":"2024-04-25T09:54:30.086598+0000","last_became_active":"2024-04-25T09:54:23.975394+0000","last_became_peered":"2024-04-25T09:54:23.975394+0000","last_unstale":"2024-04-25T09:54:30.086598+0000","last_undegraded":"2024-04-25T09:54:30.086598+0000","last_fullsized":"2024-04-25T09:54:30.086598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087150+0000","last_change":"2024-04-25T09:54:22.959855+0000","last_active":"2024-04-25T09:54:30.087150+0000","last_peered":"2024-04-25T09:54:30.087150+0000","last_clean":"2024-04-25T09:54:30.087150+0000","last_became_active":"2024-04-25T09:54:22.959556+0000","last_became_peered":"2024-04-25T09:54:22.959556+0000","last_unstale":"2024-04-25T09:54:30.087150+0000","last_undegraded":"2024-04-25T09:54:30.087150+0000","last_fullsized":"2024-04-25T09:54:30.087150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086831+0000","last_change":"2024-04-25T09:54:28.010760+0000","last_active":"2024-04-25T09:54:30.086831+0000","last_peered":"2024-04-25T09:54:30.086831+0000","last_clean":"2024-04-25T09:54:30.086831+0000","last_became_active":"2024-04-25T09:54:28.010391+0000","last_became_peered":"2024-04-25T09:54:28.010391+0000","last_unstale":"2024-04-25T09:54:30.086831+0000","last_undegraded":"2024-04-25T09:54:30.086831+0000","last_fullsized":"2024-04-25T09:54:30.086831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088301+0000","last_change":"2024-04-25T09:54:23.982380+0000","last_active":"2024-04-25T09:54:30.088301+0000","last_peered":"2024-04-25T09:54:30.088301+0000","last_clean":"2024-04-25T09:54:30.088301+0000","last_became_active":"2024-04-25T09:54:23.982076+0000","last_became_peered":"2024-04-25T09:54:23.982076+0000","last_unstale":"2024-04-25T09:54:30.088301+0000","last_undegraded":"2024-04-25T09:54:30.088301+0000","last_fullsized":"2024-04-25T09:54:30.088301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086229+0000","last_change":"2024-04-25T09:54:22.963068+0000","last_active":"2024-04-25T09:54:30.086229+0000","last_peered":"2024-04-25T09:54:30.086229+0000","last_clean":"2024-04-25T09:54:30.086229+0000","last_became_active":"2024-04-25T09:54:22.962607+0000","last_became_peered":"2024-04-25T09:54:22.962607+0000","last_unstale":"2024-04-25T09:54:30.086229+0000","last_undegraded":"2024-04-25T09:54:30.086229+0000","last_fullsized":"2024-04-25T09:54:30.086229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:28.396123+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:28.395141+0000","last_became_peered":"2024-04-25T09:54:28.395141+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087759+0000","last_change":"2024-04-25T09:54:23.981922+0000","last_active":"2024-04-25T09:54:30.087759+0000","last_peered":"2024-04-25T09:54:30.087759+0000","last_clean":"2024-04-25T09:54:30.087759+0000","last_became_active":"2024-04-25T09:54:23.981624+0000","last_became_peered":"2024-04-25T09:54:23.981624+0000","last_unstale":"2024-04-25T09:54:30.087759+0000","last_undegraded":"2024-04-25T09:54:30.087759+0000","last_fullsized":"2024-04-25T09:54:30.087759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996103+0000","last_change":"2024-04-25T09:54:22.958111+0000","last_active":"2024-04-25T09:54:29.996103+0000","last_peered":"2024-04-25T09:54:29.996103+0000","last_clean":"2024-04-25T09:54:29.996103+0000","last_became_active":"2024-04-25T09:54:22.957822+0000","last_became_peered":"2024-04-25T09:54:22.957822+0000","last_unstale":"2024-04-25T09:54:29.996103+0000","last_undegraded":"2024-04-25T09:54:29.996103+0000","last_fullsized":"2024-04-25T09:54:29.996103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:05.312329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995523+0000","last_change":"2024-04-25T09:54:28.011885+0000","last_active":"2024-04-25T09:54:29.995523+0000","last_peered":"2024-04-25T09:54:29.995523+0000","last_clean":"2024-04-25T09:54:29.995523+0000","last_became_active":"2024-04-25T09:54:28.011498+0000","last_became_peered":"2024-04-25T09:54:28.011498+0000","last_unstale":"2024-04-25T09:54:29.995523+0000","last_undegraded":"2024-04-25T09:54:29.995523+0000","last_fullsized":"2024-04-25T09:54:29.995523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087609+0000","last_change":"2024-04-25T09:54:23.982839+0000","last_active":"2024-04-25T09:54:30.087609+0000","last_peered":"2024-04-25T09:54:30.087609+0000","last_clean":"2024-04-25T09:54:30.087609+0000","last_became_active":"2024-04-25T09:54:23.982542+0000","last_became_peered":"2024-04-25T09:54:23.982542+0000","last_unstale":"2024-04-25T09:54:30.087609+0000","last_undegraded":"2024-04-25T09:54:30.087609+0000","last_fullsized":"2024-04-25T09:54:30.087609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085485+0000","last_change":"2024-04-25T09:54:22.954113+0000","last_active":"2024-04-25T09:54:30.085485+0000","last_peered":"2024-04-25T09:54:30.085485+0000","last_clean":"2024-04-25T09:54:30.085485+0000","last_became_active":"2024-04-25T09:54:22.953819+0000","last_became_peered":"2024-04-25T09:54:22.953819+0000","last_unstale":"2024-04-25T09:54:30.085485+0000","last_undegraded":"2024-04-25T09:54:30.085485+0000","last_fullsized":"2024-04-25T09:54:30.085485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086548+0000","last_change":"2024-04-25T09:54:28.008178+0000","last_active":"2024-04-25T09:54:30.086548+0000","last_peered":"2024-04-25T09:54:30.086548+0000","last_clean":"2024-04-25T09:54:30.086548+0000","last_became_active":"2024-04-25T09:54:28.007737+0000","last_became_peered":"2024-04-25T09:54:28.007737+0000","last_unstale":"2024-04-25T09:54:30.086548+0000","last_undegraded":"2024-04-25T09:54:30.086548+0000","last_fullsized":"2024-04-25T09:54:30.086548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085923+0000","last_change":"2024-04-25T09:54:23.977488+0000","last_active":"2024-04-25T09:54:30.085923+0000","last_peered":"2024-04-25T09:54:30.085923+0000","last_clean":"2024-04-25T09:54:30.085923+0000","last_became_active":"2024-04-25T09:54:23.977190+0000","last_became_peered":"2024-04-25T09:54:23.977190+0000","last_unstale":"2024-04-25T09:54:30.085923+0000","last_undegraded":"2024-04-25T09:54:30.085923+0000","last_fullsized":"2024-04-25T09:54:30.085923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086516+0000","last_change":"2024-04-25T09:54:22.959330+0000","last_active":"2024-04-25T09:54:30.086516+0000","last_peered":"2024-04-25T09:54:30.086516+0000","last_clean":"2024-04-25T09:54:30.086516+0000","last_became_active":"2024-04-25T09:54:22.959026+0000","last_became_peered":"2024-04-25T09:54:22.959026+0000","last_unstale":"2024-04-25T09:54:30.086516+0000","last_undegraded":"2024-04-25T09:54:30.086516+0000","last_fullsized":"2024-04-25T09:54:30.086516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995258+0000","last_change":"2024-04-25T09:54:28.396288+0000","last_active":"2024-04-25T09:54:29.995258+0000","last_peered":"2024-04-25T09:54:29.995258+0000","last_clean":"2024-04-25T09:54:29.995258+0000","last_became_active":"2024-04-25T09:54:28.395940+0000","last_became_peered":"2024-04-25T09:54:28.395940+0000","last_unstale":"2024-04-25T09:54:29.995258+0000","last_undegraded":"2024-04-25T09:54:29.995258+0000","last_fullsized":"2024-04-25T09:54:29.995258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:05.296954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087789+0000","last_change":"2024-04-25T09:54:23.982575+0000","last_active":"2024-04-25T09:54:30.087789+0000","last_peered":"2024-04-25T09:54:30.087789+0000","last_clean":"2024-04-25T09:54:30.087789+0000","last_became_active":"2024-04-25T09:54:23.982274+0000","last_became_peered":"2024-04-25T09:54:23.982274+0000","last_unstale":"2024-04-25T09:54:30.087789+0000","last_undegraded":"2024-04-25T09:54:30.087789+0000","last_fullsized":"2024-04-25T09:54:30.087789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087152+0000","last_change":"2024-04-25T09:54:22.962975+0000","last_active":"2024-04-25T09:54:30.087152+0000","last_peered":"2024-04-25T09:54:30.087152+0000","last_clean":"2024-04-25T09:54:30.087152+0000","last_became_active":"2024-04-25T09:54:22.962642+0000","last_became_peered":"2024-04-25T09:54:22.962642+0000","last_unstale":"2024-04-25T09:54:30.087152+0000","last_undegraded":"2024-04-25T09:54:30.087152+0000","last_fullsized":"2024-04-25T09:54:30.087152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994645+0000","last_change":"2024-04-25T09:54:28.395146+0000","last_active":"2024-04-25T09:54:29.994645+0000","last_peered":"2024-04-25T09:54:29.994645+0000","last_clean":"2024-04-25T09:54:29.994645+0000","last_became_active":"2024-04-25T09:54:28.394756+0000","last_became_peered":"2024-04-25T09:54:28.394756+0000","last_unstale":"2024-04-25T09:54:29.994645+0000","last_undegraded":"2024-04-25T09:54:29.994645+0000","last_fullsized":"2024-04-25T09:54:29.994645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:05.993975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086033+0000","last_change":"2024-04-25T09:54:23.972232+0000","last_active":"2024-04-25T09:54:30.086033+0000","last_peered":"2024-04-25T09:54:30.086033+0000","last_clean":"2024-04-25T09:54:30.086033+0000","last_became_active":"2024-04-25T09:54:23.971948+0000","last_became_peered":"2024-04-25T09:54:23.971948+0000","last_unstale":"2024-04-25T09:54:30.086033+0000","last_undegraded":"2024-04-25T09:54:30.086033+0000","last_fullsized":"2024-04-25T09:54:30.086033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:10.704903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086681+0000","last_change":"2024-04-25T09:54:22.957583+0000","last_active":"2024-04-25T09:54:30.086681+0000","last_peered":"2024-04-25T09:54:30.086681+0000","last_clean":"2024-04-25T09:54:30.086681+0000","last_became_active":"2024-04-25T09:54:22.957190+0000","last_became_peered":"2024-04-25T09:54:22.957190+0000","last_unstale":"2024-04-25T09:54:30.086681+0000","last_undegraded":"2024-04-25T09:54:30.086681+0000","last_fullsized":"2024-04-25T09:54:30.086681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:21.147671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087171+0000","last_change":"2024-04-25T09:54:28.395989+0000","last_active":"2024-04-25T09:54:30.087171+0000","last_peered":"2024-04-25T09:54:30.087171+0000","last_clean":"2024-04-25T09:54:30.087171+0000","last_became_active":"2024-04-25T09:54:28.395578+0000","last_became_peered":"2024-04-25T09:54:28.395578+0000","last_unstale":"2024-04-25T09:54:30.087171+0000","last_undegraded":"2024-04-25T09:54:30.087171+0000","last_fullsized":"2024-04-25T09:54:30.087171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087863+0000","last_change":"2024-04-25T09:54:23.984390+0000","last_active":"2024-04-25T09:54:30.087863+0000","last_peered":"2024-04-25T09:54:30.087863+0000","last_clean":"2024-04-25T09:54:30.087863+0000","last_became_active":"2024-04-25T09:54:23.981528+0000","last_became_peered":"2024-04-25T09:54:23.981528+0000","last_unstale":"2024-04-25T09:54:30.087863+0000","last_undegraded":"2024-04-25T09:54:30.087863+0000","last_fullsized":"2024-04-25T09:54:30.087863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:33.133289+0000","last_change":"2024-04-25T09:54:22.962540+0000","last_active":"2024-04-25T09:54:33.133289+0000","last_peered":"2024-04-25T09:54:33.133289+0000","last_clean":"2024-04-25T09:54:33.133289+0000","last_became_active":"2024-04-25T09:54:22.962223+0000","last_became_peered":"2024-04-25T09:54:22.962223+0000","last_unstale":"2024-04-25T09:54:33.133289+0000","last_undegraded":"2024-04-25T09:54:33.133289+0000","last_fullsized":"2024-04-25T09:54:33.133289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085807+0000","last_change":"2024-04-25T09:54:28.012670+0000","last_active":"2024-04-25T09:54:30.085807+0000","last_peered":"2024-04-25T09:54:30.085807+0000","last_clean":"2024-04-25T09:54:30.085807+0000","last_became_active":"2024-04-25T09:54:28.011580+0000","last_became_peered":"2024-04-25T09:54:28.011580+0000","last_unstale":"2024-04-25T09:54:30.085807+0000","last_undegraded":"2024-04-25T09:54:30.085807+0000","last_fullsized":"2024-04-25T09:54:30.085807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996273+0000","last_change":"2024-04-25T09:54:23.981044+0000","last_active":"2024-04-25T09:54:29.996273+0000","last_peered":"2024-04-25T09:54:29.996273+0000","last_clean":"2024-04-25T09:54:29.996273+0000","last_became_active":"2024-04-25T09:54:23.980733+0000","last_became_peered":"2024-04-25T09:54:23.980733+0000","last_unstale":"2024-04-25T09:54:29.996273+0000","last_undegraded":"2024-04-25T09:54:29.996273+0000","last_fullsized":"2024-04-25T09:54:29.996273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087059+0000","last_change":"2024-04-25T09:54:22.965478+0000","last_active":"2024-04-25T09:54:30.087059+0000","last_peered":"2024-04-25T09:54:30.087059+0000","last_clean":"2024-04-25T09:54:30.087059+0000","last_became_active":"2024-04-25T09:54:22.965191+0000","last_became_peered":"2024-04-25T09:54:22.965191+0000","last_unstale":"2024-04-25T09:54:30.087059+0000","last_undegraded":"2024-04-25T09:54:30.087059+0000","last_fullsized":"2024-04-25T09:54:30.087059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995924+0000","last_change":"2024-04-25T09:54:28.003916+0000","last_active":"2024-04-25T09:54:29.995924+0000","last_peered":"2024-04-25T09:54:29.995924+0000","last_clean":"2024-04-25T09:54:29.995924+0000","last_became_active":"2024-04-25T09:54:28.003512+0000","last_became_peered":"2024-04-25T09:54:28.003512+0000","last_unstale":"2024-04-25T09:54:29.995924+0000","last_undegraded":"2024-04-25T09:54:29.995924+0000","last_fullsized":"2024-04-25T09:54:29.995924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086045+0000","last_change":"2024-04-25T09:54:23.969795+0000","last_active":"2024-04-25T09:54:30.086045+0000","last_peered":"2024-04-25T09:54:30.086045+0000","last_clean":"2024-04-25T09:54:30.086045+0000","last_became_active":"2024-04-25T09:54:23.969543+0000","last_became_peered":"2024-04-25T09:54:23.969543+0000","last_unstale":"2024-04-25T09:54:30.086045+0000","last_undegraded":"2024-04-25T09:54:30.086045+0000","last_fullsized":"2024-04-25T09:54:30.086045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086869+0000","last_change":"2024-04-25T09:54:22.965431+0000","last_active":"2024-04-25T09:54:30.086869+0000","last_peered":"2024-04-25T09:54:30.086869+0000","last_clean":"2024-04-25T09:54:30.086869+0000","last_became_active":"2024-04-25T09:54:22.965115+0000","last_became_peered":"2024-04-25T09:54:22.965115+0000","last_unstale":"2024-04-25T09:54:30.086869+0000","last_undegraded":"2024-04-25T09:54:30.086869+0000","last_fullsized":"2024-04-25T09:54:30.086869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994658+0000","last_change":"2024-04-25T09:54:28.011370+0000","last_active":"2024-04-25T09:54:29.994658+0000","last_peered":"2024-04-25T09:54:29.994658+0000","last_clean":"2024-04-25T09:54:29.994658+0000","last_became_active":"2024-04-25T09:54:28.011006+0000","last_became_peered":"2024-04-25T09:54:28.011006+0000","last_unstale":"2024-04-25T09:54:29.994658+0000","last_undegraded":"2024-04-25T09:54:29.994658+0000","last_fullsized":"2024-04-25T09:54:29.994658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:05.906706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086261+0000","last_change":"2024-04-25T09:54:23.983800+0000","last_active":"2024-04-25T09:54:30.086261+0000","last_peered":"2024-04-25T09:54:30.086261+0000","last_clean":"2024-04-25T09:54:30.086261+0000","last_became_active":"2024-04-25T09:54:23.983518+0000","last_became_peered":"2024-04-25T09:54:23.983518+0000","last_unstale":"2024-04-25T09:54:30.086261+0000","last_undegraded":"2024-04-25T09:54:30.086261+0000","last_fullsized":"2024-04-25T09:54:30.086261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086666+0000","last_change":"2024-04-25T09:54:22.966898+0000","last_active":"2024-04-25T09:54:30.086666+0000","last_peered":"2024-04-25T09:54:30.086666+0000","last_clean":"2024-04-25T09:54:30.086666+0000","last_became_active":"2024-04-25T09:54:22.966638+0000","last_became_peered":"2024-04-25T09:54:22.966638+0000","last_unstale":"2024-04-25T09:54:30.086666+0000","last_undegraded":"2024-04-25T09:54:30.086666+0000","last_fullsized":"2024-04-25T09:54:30.086666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994713+0000","last_change":"2024-04-25T09:54:28.397771+0000","last_active":"2024-04-25T09:54:29.994713+0000","last_peered":"2024-04-25T09:54:29.994713+0000","last_clean":"2024-04-25T09:54:29.994713+0000","last_became_active":"2024-04-25T09:54:28.397413+0000","last_became_peered":"2024-04-25T09:54:28.397413+0000","last_unstale":"2024-04-25T09:54:29.994713+0000","last_undegraded":"2024-04-25T09:54:29.994713+0000","last_fullsized":"2024-04-25T09:54:29.994713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:07.468831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086335+0000","last_change":"2024-04-25T09:54:28.396249+0000","last_active":"2024-04-25T09:54:30.086335+0000","last_peered":"2024-04-25T09:54:30.086335+0000","last_clean":"2024-04-25T09:54:30.086335+0000","last_became_active":"2024-04-25T09:54:28.395904+0000","last_became_peered":"2024-04-25T09:54:28.395904+0000","last_unstale":"2024-04-25T09:54:30.086335+0000","last_undegraded":"2024-04-25T09:54:30.086335+0000","last_fullsized":"2024-04-25T09:54:30.086335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086193+0000","last_change":"2024-04-25T09:54:23.972012+0000","last_active":"2024-04-25T09:54:30.086193+0000","last_peered":"2024-04-25T09:54:30.086193+0000","last_clean":"2024-04-25T09:54:30.086193+0000","last_became_active":"2024-04-25T09:54:23.971731+0000","last_became_peered":"2024-04-25T09:54:23.971731+0000","last_unstale":"2024-04-25T09:54:30.086193+0000","last_undegraded":"2024-04-25T09:54:30.086193+0000","last_fullsized":"2024-04-25T09:54:30.086193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086458+0000","last_change":"2024-04-25T09:54:22.965400+0000","last_active":"2024-04-25T09:54:30.086458+0000","last_peered":"2024-04-25T09:54:30.086458+0000","last_clean":"2024-04-25T09:54:30.086458+0000","last_became_active":"2024-04-25T09:54:22.965099+0000","last_became_peered":"2024-04-25T09:54:22.965099+0000","last_unstale":"2024-04-25T09:54:30.086458+0000","last_undegraded":"2024-04-25T09:54:30.086458+0000","last_fullsized":"2024-04-25T09:54:30.086458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086552+0000","last_change":"2024-04-25T09:54:28.010667+0000","last_active":"2024-04-25T09:54:30.086552+0000","last_peered":"2024-04-25T09:54:30.086552+0000","last_clean":"2024-04-25T09:54:30.086552+0000","last_became_active":"2024-04-25T09:54:28.010243+0000","last_became_peered":"2024-04-25T09:54:28.010243+0000","last_unstale":"2024-04-25T09:54:30.086552+0000","last_undegraded":"2024-04-25T09:54:30.086552+0000","last_fullsized":"2024-04-25T09:54:30.086552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994577+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:29.994577+0000","last_peered":"2024-04-25T09:54:29.994577+0000","last_clean":"2024-04-25T09:54:29.994577+0000","last_became_active":"2024-04-25T09:54:23.970422+0000","last_became_peered":"2024-04-25T09:54:23.970422+0000","last_unstale":"2024-04-25T09:54:29.994577+0000","last_undegraded":"2024-04-25T09:54:29.994577+0000","last_fullsized":"2024-04-25T09:54:29.994577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994799+0000","last_change":"2024-04-25T09:54:22.960394+0000","last_active":"2024-04-25T09:54:29.994799+0000","last_peered":"2024-04-25T09:54:29.994799+0000","last_clean":"2024-04-25T09:54:29.994799+0000","last_became_active":"2024-04-25T09:54:22.960128+0000","last_became_peered":"2024-04-25T09:54:22.960128+0000","last_unstale":"2024-04-25T09:54:29.994799+0000","last_undegraded":"2024-04-25T09:54:29.994799+0000","last_fullsized":"2024-04-25T09:54:29.994799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086079+0000","last_change":"2024-04-25T09:54:22.962677+0000","last_active":"2024-04-25T09:54:30.086079+0000","last_peered":"2024-04-25T09:54:30.086079+0000","last_clean":"2024-04-25T09:54:30.086079+0000","last_became_active":"2024-04-25T09:54:22.962351+0000","last_became_peered":"2024-04-25T09:54:22.962351+0000","last_unstale":"2024-04-25T09:54:30.086079+0000","last_undegraded":"2024-04-25T09:54:30.086079+0000","last_fullsized":"2024-04-25T09:54:30.086079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086533+0000","last_change":"2024-04-25T09:54:23.973688+0000","last_active":"2024-04-25T09:54:30.086533+0000","last_peered":"2024-04-25T09:54:30.086533+0000","last_clean":"2024-04-25T09:54:30.086533+0000","last_became_active":"2024-04-25T09:54:23.973424+0000","last_became_peered":"2024-04-25T09:54:23.973424+0000","last_unstale":"2024-04-25T09:54:30.086533+0000","last_undegraded":"2024-04-25T09:54:30.086533+0000","last_fullsized":"2024-04-25T09:54:30.086533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086930+0000","last_change":"2024-04-25T09:54:28.016164+0000","last_active":"2024-04-25T09:54:30.086930+0000","last_peered":"2024-04-25T09:54:30.086930+0000","last_clean":"2024-04-25T09:54:30.086930+0000","last_became_active":"2024-04-25T09:54:28.015634+0000","last_became_peered":"2024-04-25T09:54:28.015634+0000","last_unstale":"2024-04-25T09:54:30.086930+0000","last_undegraded":"2024-04-25T09:54:30.086930+0000","last_fullsized":"2024-04-25T09:54:30.086930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.111330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994585+0000","last_change":"2024-04-25T09:54:22.962095+0000","last_active":"2024-04-25T09:54:29.994585+0000","last_peered":"2024-04-25T09:54:29.994585+0000","last_clean":"2024-04-25T09:54:29.994585+0000","last_became_active":"2024-04-25T09:54:22.961801+0000","last_became_peered":"2024-04-25T09:54:22.961801+0000","last_unstale":"2024-04-25T09:54:29.994585+0000","last_undegraded":"2024-04-25T09:54:29.994585+0000","last_fullsized":"2024-04-25T09:54:29.994585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995467+0000","last_change":"2024-04-25T09:54:23.971722+0000","last_active":"2024-04-25T09:54:29.995467+0000","last_peered":"2024-04-25T09:54:29.995467+0000","last_clean":"2024-04-25T09:54:29.995467+0000","last_became_active":"2024-04-25T09:54:23.971437+0000","last_became_peered":"2024-04-25T09:54:23.971437+0000","last_unstale":"2024-04-25T09:54:29.995467+0000","last_undegraded":"2024-04-25T09:54:29.995467+0000","last_fullsized":"2024-04-25T09:54:29.995467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:28.933707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994926+0000","last_change":"2024-04-25T09:54:28.016742+0000","last_active":"2024-04-25T09:54:29.994926+0000","last_peered":"2024-04-25T09:54:29.994926+0000","last_clean":"2024-04-25T09:54:29.994926+0000","last_became_active":"2024-04-25T09:54:28.016017+0000","last_became_peered":"2024-04-25T09:54:28.016017+0000","last_unstale":"2024-04-25T09:54:29.994926+0000","last_undegraded":"2024-04-25T09:54:29.994926+0000","last_fullsized":"2024-04-25T09:54:29.994926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:51.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085710+0000","last_change":"2024-04-25T09:54:23.976967+0000","last_active":"2024-04-25T09:54:30.085710+0000","last_peered":"2024-04-25T09:54:30.085710+0000","last_clean":"2024-04-25T09:54:30.085710+0000","last_became_active":"2024-04-25T09:54:23.976684+0000","last_became_peered":"2024-04-25T09:54:23.976684+0000","last_unstale":"2024-04-25T09:54:30.085710+0000","last_undegraded":"2024-04-25T09:54:30.085710+0000","last_fullsized":"2024-04-25T09:54:30.085710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086490+0000","last_change":"2024-04-25T09:54:23.968076+0000","last_active":"2024-04-25T09:54:30.086490+0000","last_peered":"2024-04-25T09:54:30.086490+0000","last_clean":"2024-04-25T09:54:30.086490+0000","last_became_active":"2024-04-25T09:54:23.967823+0000","last_became_peered":"2024-04-25T09:54:23.967823+0000","last_unstale":"2024-04-25T09:54:30.086490+0000","last_undegraded":"2024-04-25T09:54:30.086490+0000","last_fullsized":"2024-04-25T09:54:30.086490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087064+0000","last_change":"2024-04-25T09:54:23.974924+0000","last_active":"2024-04-25T09:54:30.087064+0000","last_peered":"2024-04-25T09:54:30.087064+0000","last_clean":"2024-04-25T09:54:30.087064+0000","last_became_active":"2024-04-25T09:54:23.974647+0000","last_became_peered":"2024-04-25T09:54:23.974647+0000","last_unstale":"2024-04-25T09:54:30.087064+0000","last_undegraded":"2024-04-25T09:54:30.087064+0000","last_fullsized":"2024-04-25T09:54:30.087064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086431+0000","last_change":"2024-04-25T09:54:23.974599+0000","last_active":"2024-04-25T09:54:30.086431+0000","last_peered":"2024-04-25T09:54:30.086431+0000","last_clean":"2024-04-25T09:54:30.086431+0000","last_became_active":"2024-04-25T09:54:23.973958+0000","last_became_peered":"2024-04-25T09:54:23.973958+0000","last_unstale":"2024-04-25T09:54:30.086431+0000","last_undegraded":"2024-04-25T09:54:30.086431+0000","last_fullsized":"2024-04-25T09:54:30.086431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994701+0000","last_change":"2024-04-25T09:54:23.976111+0000","last_active":"2024-04-25T09:54:29.994701+0000","last_peered":"2024-04-25T09:54:29.994701+0000","last_clean":"2024-04-25T09:54:29.994701+0000","last_became_active":"2024-04-25T09:54:23.975817+0000","last_became_peered":"2024-04-25T09:54:23.975817+0000","last_unstale":"2024-04-25T09:54:29.994701+0000","last_undegraded":"2024-04-25T09:54:29.994701+0000","last_fullsized":"2024-04-25T09:54:29.994701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086971+0000","last_change":"2024-04-25T09:54:23.967777+0000","last_active":"2024-04-25T09:54:30.086971+0000","last_peered":"2024-04-25T09:54:30.086971+0000","last_clean":"2024-04-25T09:54:30.086971+0000","last_became_active":"2024-04-25T09:54:23.967099+0000","last_became_peered":"2024-04-25T09:54:23.967099+0000","last_unstale":"2024-04-25T09:54:30.086971+0000","last_undegraded":"2024-04-25T09:54:30.086971+0000","last_fullsized":"2024-04-25T09:54:30.086971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086631+0000","last_change":"2024-04-25T09:54:23.982361+0000","last_active":"2024-04-25T09:54:30.086631+0000","last_peered":"2024-04-25T09:54:30.086631+0000","last_clean":"2024-04-25T09:54:30.086631+0000","last_became_active":"2024-04-25T09:54:23.982020+0000","last_became_peered":"2024-04-25T09:54:23.982020+0000","last_unstale":"2024-04-25T09:54:30.086631+0000","last_undegraded":"2024-04-25T09:54:30.086631+0000","last_fullsized":"2024-04-25T09:54:30.086631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995121+0000","last_change":"2024-04-25T09:54:23.976994+0000","last_active":"2024-04-25T09:54:29.995121+0000","last_peered":"2024-04-25T09:54:29.995121+0000","last_clean":"2024-04-25T09:54:29.995121+0000","last_became_active":"2024-04-25T09:54:23.976707+0000","last_became_peered":"2024-04-25T09:54:23.976707+0000","last_unstale":"2024-04-25T09:54:29.995121+0000","last_undegraded":"2024-04-25T09:54:29.995121+0000","last_fullsized":"2024-04-25T09:54:29.995121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086418+0000","last_change":"2024-04-25T09:54:23.977123+0000","last_active":"2024-04-25T09:54:30.086418+0000","last_peered":"2024-04-25T09:54:30.086418+0000","last_clean":"2024-04-25T09:54:30.086418+0000","last_became_active":"2024-04-25T09:54:23.976810+0000","last_became_peered":"2024-04-25T09:54:23.976810+0000","last_unstale":"2024-04-25T09:54:30.086418+0000","last_undegraded":"2024-04-25T09:54:30.086418+0000","last_fullsized":"2024-04-25T09:54:30.086418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994535+0000","last_change":"2024-04-25T09:54:23.976035+0000","last_active":"2024-04-25T09:54:29.994535+0000","last_peered":"2024-04-25T09:54:29.994535+0000","last_clean":"2024-04-25T09:54:29.994535+0000","last_became_active":"2024-04-25T09:54:23.975755+0000","last_became_peered":"2024-04-25T09:54:23.975755+0000","last_unstale":"2024-04-25T09:54:29.994535+0000","last_undegraded":"2024-04-25T09:54:29.994535+0000","last_fullsized":"2024-04-25T09:54:29.994535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086969+0000","last_change":"2024-04-25T09:54:23.973340+0000","last_active":"2024-04-25T09:54:30.086969+0000","last_peered":"2024-04-25T09:54:30.086969+0000","last_clean":"2024-04-25T09:54:30.086969+0000","last_became_active":"2024-04-25T09:54:23.973010+0000","last_became_peered":"2024-04-25T09:54:23.973010+0000","last_unstale":"2024-04-25T09:54:30.086969+0000","last_undegraded":"2024-04-25T09:54:30.086969+0000","last_fullsized":"2024-04-25T09:54:30.086969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086318+0000","last_change":"2024-04-25T09:54:23.980705+0000","last_active":"2024-04-25T09:54:30.086318+0000","last_peered":"2024-04-25T09:54:30.086318+0000","last_clean":"2024-04-25T09:54:30.086318+0000","last_became_active":"2024-04-25T09:54:23.980411+0000","last_became_peered":"2024-04-25T09:54:23.980411+0000","last_unstale":"2024-04-25T09:54:30.086318+0000","last_undegraded":"2024-04-25T09:54:30.086318+0000","last_fullsized":"2024-04-25T09:54:30.086318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086259+0000","last_change":"2024-04-25T09:54:23.984091+0000","last_active":"2024-04-25T09:54:30.086259+0000","last_peered":"2024-04-25T09:54:30.086259+0000","last_clean":"2024-04-25T09:54:30.086259+0000","last_became_active":"2024-04-25T09:54:23.983834+0000","last_became_peered":"2024-04-25T09:54:23.983834+0000","last_unstale":"2024-04-25T09:54:30.086259+0000","last_undegraded":"2024-04-25T09:54:30.086259+0000","last_fullsized":"2024-04-25T09:54:30.086259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088036+0000","last_change":"2024-04-25T09:54:23.983353+0000","last_active":"2024-04-25T09:54:30.088036+0000","last_peered":"2024-04-25T09:54:30.088036+0000","last_clean":"2024-04-25T09:54:30.088036+0000","last_became_active":"2024-04-25T09:54:23.983043+0000","last_became_peered":"2024-04-25T09:54:23.983043+0000","last_unstale":"2024-04-25T09:54:30.088036+0000","last_undegraded":"2024-04-25T09:54:30.088036+0000","last_fullsized":"2024-04-25T09:54:30.088036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086130+0000","last_change":"2024-04-25T09:54:23.977174+0000","last_active":"2024-04-25T09:54:30.086130+0000","last_peered":"2024-04-25T09:54:30.086130+0000","last_clean":"2024-04-25T09:54:30.086130+0000","last_became_active":"2024-04-25T09:54:23.976903+0000","last_became_peered":"2024-04-25T09:54:23.976903+0000","last_unstale":"2024-04-25T09:54:30.086130+0000","last_undegraded":"2024-04-25T09:54:30.086130+0000","last_fullsized":"2024-04-25T09:54:30.086130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.972492+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.972188+0000","last_became_peered":"2024-04-25T09:54:23.972188+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086368+0000","last_change":"2024-04-25T09:54:23.978371+0000","last_active":"2024-04-25T09:54:30.086368+0000","last_peered":"2024-04-25T09:54:30.086368+0000","last_clean":"2024-04-25T09:54:30.086368+0000","last_became_active":"2024-04-25T09:54:23.978079+0000","last_became_peered":"2024-04-25T09:54:23.978079+0000","last_unstale":"2024-04-25T09:54:30.086368+0000","last_undegraded":"2024-04-25T09:54:30.086368+0000","last_fullsized":"2024-04-25T09:54:30.086368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086280+0000","last_change":"2024-04-25T09:54:23.976031+0000","last_active":"2024-04-25T09:54:30.086280+0000","last_peered":"2024-04-25T09:54:30.086280+0000","last_clean":"2024-04-25T09:54:30.086280+0000","last_became_active":"2024-04-25T09:54:23.975750+0000","last_became_peered":"2024-04-25T09:54:23.975750+0000","last_unstale":"2024-04-25T09:54:30.086280+0000","last_undegraded":"2024-04-25T09:54:30.086280+0000","last_fullsized":"2024-04-25T09:54:30.086280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085955+0000","last_change":"2024-04-25T09:54:23.973561+0000","last_active":"2024-04-25T09:54:30.085955+0000","last_peered":"2024-04-25T09:54:30.085955+0000","last_clean":"2024-04-25T09:54:30.085955+0000","last_became_active":"2024-04-25T09:54:23.973312+0000","last_became_peered":"2024-04-25T09:54:23.973312+0000","last_unstale":"2024-04-25T09:54:30.085955+0000","last_undegraded":"2024-04-25T09:54:30.085955+0000","last_fullsized":"2024-04-25T09:54:30.085955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994677+0000","last_change":"2024-04-25T09:54:23.969586+0000","last_active":"2024-04-25T09:54:29.994677+0000","last_peered":"2024-04-25T09:54:29.994677+0000","last_clean":"2024-04-25T09:54:29.994677+0000","last_became_active":"2024-04-25T09:54:23.969067+0000","last_became_peered":"2024-04-25T09:54:23.969067+0000","last_unstale":"2024-04-25T09:54:29.994677+0000","last_undegraded":"2024-04-25T09:54:29.994677+0000","last_fullsized":"2024-04-25T09:54:29.994677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995863+0000","last_change":"2024-04-25T09:54:23.972621+0000","last_active":"2024-04-25T09:54:29.995863+0000","last_peered":"2024-04-25T09:54:29.995863+0000","last_clean":"2024-04-25T09:54:29.995863+0000","last_became_active":"2024-04-25T09:54:23.972338+0000","last_became_peered":"2024-04-25T09:54:23.972338+0000","last_unstale":"2024-04-25T09:54:29.995863+0000","last_undegraded":"2024-04-25T09:54:29.995863+0000","last_fullsized":"2024-04-25T09:54:29.995863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086120+0000","last_change":"2024-04-25T09:54:23.971085+0000","last_active":"2024-04-25T09:54:30.086120+0000","last_peered":"2024-04-25T09:54:30.086120+0000","last_clean":"2024-04-25T09:54:30.086120+0000","last_became_active":"2024-04-25T09:54:23.970354+0000","last_became_peered":"2024-04-25T09:54:23.970354+0000","last_unstale":"2024-04-25T09:54:30.086120+0000","last_undegraded":"2024-04-25T09:54:30.086120+0000","last_fullsized":"2024-04-25T09:54:30.086120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086765+0000","last_change":"2024-04-25T09:54:23.970478+0000","last_active":"2024-04-25T09:54:30.086765+0000","last_peered":"2024-04-25T09:54:30.086765+0000","last_clean":"2024-04-25T09:54:30.086765+0000","last_became_active":"2024-04-25T09:54:23.970181+0000","last_became_peered":"2024-04-25T09:54:23.970181+0000","last_unstale":"2024-04-25T09:54:30.086765+0000","last_undegraded":"2024-04-25T09:54:30.086765+0000","last_fullsized":"2024-04-25T09:54:30.086765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088285+0000","last_change":"2024-04-25T09:54:23.983654+0000","last_active":"2024-04-25T09:54:30.088285+0000","last_peered":"2024-04-25T09:54:30.088285+0000","last_clean":"2024-04-25T09:54:30.088285+0000","last_became_active":"2024-04-25T09:54:23.983388+0000","last_became_peered":"2024-04-25T09:54:23.983388+0000","last_unstale":"2024-04-25T09:54:30.088285+0000","last_undegraded":"2024-04-25T09:54:30.088285+0000","last_fullsized":"2024-04-25T09:54:30.088285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994460+0000","last_change":"2024-04-25T09:54:23.970049+0000","last_active":"2024-04-25T09:54:29.994460+0000","last_peered":"2024-04-25T09:54:29.994460+0000","last_clean":"2024-04-25T09:54:29.994460+0000","last_became_active":"2024-04-25T09:54:23.969605+0000","last_became_peered":"2024-04-25T09:54:23.969605+0000","last_unstale":"2024-04-25T09:54:29.994460+0000","last_undegraded":"2024-04-25T09:54:29.994460+0000","last_fullsized":"2024-04-25T09:54:29.994460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087574+0000","last_change":"2024-04-25T09:54:23.980721+0000","last_active":"2024-04-25T09:54:30.087574+0000","last_peered":"2024-04-25T09:54:30.087574+0000","last_clean":"2024-04-25T09:54:30.087574+0000","last_became_active":"2024-04-25T09:54:23.980440+0000","last_became_peered":"2024-04-25T09:54:23.980440+0000","last_unstale":"2024-04-25T09:54:30.087574+0000","last_undegraded":"2024-04-25T09:54:30.087574+0000","last_fullsized":"2024-04-25T09:54:30.087574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086883+0000","last_change":"2024-04-25T09:54:23.967904+0000","last_active":"2024-04-25T09:54:30.086883+0000","last_peered":"2024-04-25T09:54:30.086883+0000","last_clean":"2024-04-25T09:54:30.086883+0000","last_became_active":"2024-04-25T09:54:23.967585+0000","last_became_peered":"2024-04-25T09:54:23.967585+0000","last_unstale":"2024-04-25T09:54:30.086883+0000","last_undegraded":"2024-04-25T09:54:30.086883+0000","last_fullsized":"2024-04-25T09:54:30.086883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087670+0000","last_change":"2024-04-25T09:54:23.978911+0000","last_active":"2024-04-25T09:54:30.087670+0000","last_peered":"2024-04-25T09:54:30.087670+0000","last_clean":"2024-04-25T09:54:30.087670+0000","last_became_active":"2024-04-25T09:54:23.978625+0000","last_became_peered":"2024-04-25T09:54:23.978625+0000","last_unstale":"2024-04-25T09:54:30.087670+0000","last_undegraded":"2024-04-25T09:54:30.087670+0000","last_fullsized":"2024-04-25T09:54:30.087670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086202+0000","last_change":"2024-04-25T09:54:23.976392+0000","last_active":"2024-04-25T09:54:30.086202+0000","last_peered":"2024-04-25T09:54:30.086202+0000","last_clean":"2024-04-25T09:54:30.086202+0000","last_became_active":"2024-04-25T09:54:23.976117+0000","last_became_peered":"2024-04-25T09:54:23.976117+0000","last_unstale":"2024-04-25T09:54:30.086202+0000","last_undegraded":"2024-04-25T09:54:30.086202+0000","last_fullsized":"2024-04-25T09:54:30.086202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085739+0000","last_change":"2024-04-25T09:54:23.967710+0000","last_active":"2024-04-25T09:54:30.085739+0000","last_peered":"2024-04-25T09:54:30.085739+0000","last_clean":"2024-04-25T09:54:30.085739+0000","last_became_active":"2024-04-25T09:54:23.967075+0000","last_became_peered":"2024-04-25T09:54:23.967075+0000","last_unstale":"2024-04-25T09:54:30.085739+0000","last_undegraded":"2024-04-25T09:54:30.085739+0000","last_fullsized":"2024-04-25T09:54:30.085739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086748+0000","last_change":"2024-04-25T09:54:23.968977+0000","last_active":"2024-04-25T09:54:30.086748+0000","last_peered":"2024-04-25T09:54:30.086748+0000","last_clean":"2024-04-25T09:54:30.086748+0000","last_became_active":"2024-04-25T09:54:23.966936+0000","last_became_peered":"2024-04-25T09:54:23.966936+0000","last_unstale":"2024-04-25T09:54:30.086748+0000","last_undegraded":"2024-04-25T09:54:30.086748+0000","last_fullsized":"2024-04-25T09:54:30.086748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088179+0000","last_change":"2024-04-25T09:54:23.987914+0000","last_active":"2024-04-25T09:54:30.088179+0000","last_peered":"2024-04-25T09:54:30.088179+0000","last_clean":"2024-04-25T09:54:30.088179+0000","last_became_active":"2024-04-25T09:54:23.985923+0000","last_became_peered":"2024-04-25T09:54:23.985923+0000","last_unstale":"2024-04-25T09:54:30.088179+0000","last_undegraded":"2024-04-25T09:54:30.088179+0000","last_fullsized":"2024-04-25T09:54:30.088179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:46.388 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:54:46.388 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T09:54:46.388 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T09:54:46.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T09:54:46.389 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T09:54:46.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.518+0000 7f5e69efc640 1 Processor -- start 2024-04-25T09:54:46.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.518+0000 7f5e69efc640 1 -- start start 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e69efc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5e640cc6c0 0x7f5e64071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e69efc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5e6415b320 0x7f5e64071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e69efc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e64071ed0 con 0x7f5e6413d190 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e69efc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e64071cb0 con 0x7f5e6415b320 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e69efc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6413d570 con 0x7f5e640cc6c0 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5e6415b320 0x7f5e64071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5e6415b320 0x7f5e64071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33364/0 (socket says 172.21.15.148:33364) 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e637fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5e6413d190 0x7f5e640cbfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48694/0 (socket says 172.21.15.148:48694) 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 -- 172.21.15.148:0/4241423193 learned_addr learned my addr 172.21.15.148:0/4241423193 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:46.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e62ffd640 1 --2- 172.21.15.148:0/4241423193 >> v2:172.21.15.188:3300/0 conn(0x7f5e640cc6c0 0x7f5e64071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 -- 172.21.15.148:0/4241423193 >> v2:172.21.15.188:3300/0 conn(0x7f5e640cc6c0 msgr2=0x7f5e64071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:46.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 --2- 172.21.15.148:0/4241423193 >> v2:172.21.15.188:3300/0 conn(0x7f5e640cc6c0 0x7f5e64071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 -- 172.21.15.148:0/4241423193 >> v1:172.21.15.148:6789/0 conn(0x7f5e6413d190 legacy=0x7f5e640cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e63fff640 1 -- 172.21.15.148:0/4241423193 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6415a840 con 0x7f5e6415b320 2024-04-25T09:54:46.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.519+0000 7f5e627fc640 1 -- 172.21.15.148:0/4241423193 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601758193 0 0) 0x7f5e64071ed0 con 0x7f5e6413d190 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.520+0000 7f5e63fff640 1 --2- 172.21.15.148:0/4241423193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6415b320 0x7f5e64071770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e6406f6f0 tx=0x7f5e5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f543e6f2a4f89b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.520+0000 7f5e627fc640 1 -- 172.21.15.148:0/4241423193 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c036070 con 0x7f5e6415b320 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.520+0000 7f5e627fc640 1 -- 172.21.15.148:0/4241423193 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c031600 con 0x7f5e6415b320 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.520+0000 7f5e627fc640 1 -- 172.21.15.148:0/4241423193 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0319d0 con 0x7f5e6415b320 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 -- 172.21.15.148:0/4241423193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6415b320 msgr2=0x7f5e64071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 --2- 172.21.15.148:0/4241423193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6415b320 0x7f5e64071770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e6406f6f0 tx=0x7f5e5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 -- 172.21.15.148:0/4241423193 shutdown_connections 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 --2- 172.21.15.148:0/4241423193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6415b320 0x7f5e64071770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 --2- 172.21.15.148:0/4241423193 >> v2:172.21.15.188:3300/0 conn(0x7f5e640cc6c0 0x7f5e64071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 -- 172.21.15.148:0/4241423193 >> 172.21.15.148:0/4241423193 conn(0x7f5e6406abe0 msgr2=0x7f5e6406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 -- 172.21.15.148:0/4241423193 shutdown_connections 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.521+0000 7f5e69efc640 1 -- 172.21.15.148:0/4241423193 wait complete. 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.522+0000 7f5e69efc640 1 Processor -- start 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.522+0000 7f5e69efc640 1 -- start start 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 0x7f5e641556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6413d190 0x7f5e64139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e6415b320 0x7f5e64137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640cf620 con 0x7f5e640cc6c0 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6415ae00 con 0x7f5e6413d190 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e640cf7a0 con 0x7f5e6415b320 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 0x7f5e641556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 0x7f5e641556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33370/0 (socket says 172.21.15.148:33370) 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 -- 172.21.15.148:0/97572279 learned_addr learned my addr 172.21.15.148:0/97572279 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:46.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e62ffd640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6413d190 0x7f5e64139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e63fff640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e6415b320 0x7f5e64137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 -- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e6415b320 msgr2=0x7f5e64137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e6415b320 0x7f5e64137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 -- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6413d190 msgr2=0x7f5e64139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6413d190 0x7f5e64139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 -- 172.21.15.148:0/97572279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64156510 con 0x7f5e640cc6c0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e63fff640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e6415b320 0x7f5e64137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e637fe640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 0x7f5e641556f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c4d0 tx=0x7f5e5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54018070 con 0x7f5e640cc6c0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54007500 con 0x7f5e640cc6c0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e64157e30 con 0x7f5e640cc6c0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004a20 con 0x7f5e640cc6c0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.523+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e64156230 con 0x7f5e640cc6c0 2024-04-25T09:54:46.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.524+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6414bec0 con 0x7f5e640cc6c0 2024-04-25T09:54:46.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.525+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e5401b070 con 0x7f5e640cc6c0 2024-04-25T09:54:46.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.525+0000 7f5e5bfff640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5e4406b4d0 0x7f5e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.525+0000 7f5e62ffd640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5e4406b4d0 0x7f5e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.525+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5e54089840 con 0x7f5e640cc6c0 2024-04-25T09:54:46.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.526+0000 7f5e62ffd640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5e4406b4d0 0x7f5e4406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e6415ca90 tx=0x7f5e4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.529+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e6414bec0 con 0x7f5e640cc6c0 2024-04-25T09:54:46.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.745+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5e64155d40 con 0x7f5e640cc6c0 2024-04-25T09:54:46.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.748+0000 7f5e5bfff640 1 -- 172.21.15.148:0/97572279 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171791 (secure 0 0 0) 0x7f5e54058380 con 0x7f5e640cc6c0 2024-04-25T09:54:46.753 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:46.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5e4406b4d0 msgr2=0x7f5e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5e4406b4d0 0x7f5e4406d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e6415ca90 tx=0x7f5e4c008040 comp rx=0 tx=0).stop 2024-04-25T09:54:46.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 msgr2=0x7f5e641556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 0x7f5e641556f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c4d0 tx=0x7f5e5400c990 comp rx=0 tx=0).stop 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 shutdown_connections 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5e4406b4d0 0x7f5e4406d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e6415b320 0x7f5e64137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e6413d190 0x7f5e64139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 --2- 172.21.15.148:0/97572279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e640cc6c0 0x7f5e641556f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 >> 172.21.15.148:0/97572279 conn(0x7f5e6406abe0 msgr2=0x7f5e64154bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 shutdown_connections 2024-04-25T09:54:46.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.751+0000 7f5e69efc640 1 -- 172.21.15.148:0/97572279 wait complete. 2024-04-25T09:54:46.764 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":2250922117},{"type":"v1","addr":"172.21.15.148:6833","nonce":2250922117}]},"active_addr":"172.21.15.148:6833/2250922117","active_change":"2024-04-25T09:54:03.010366+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1441979306}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3152143178}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":814094421}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1038702464}]}]} 2024-04-25T09:54:46.768 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T09:54:46.768 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T09:54:46.768 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.863+0000 7f6965de0640 1 Processor -- start 2024-04-25T09:54:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.863+0000 7f6965de0640 1 -- start start 2024-04-25T09:54:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f6965de0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f696015db70 0x7f6960162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f6965de0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6960162690 con 0x7f69600b5c30 2024-04-25T09:54:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f6965de0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69600ceee0 con 0x7f69600b6cb0 2024-04-25T09:54:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f6965de0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69600d1d10 con 0x7f696015db70 2024-04-25T09:54:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f695ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f69600b5c30 0x7f69600d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48698/0 (socket says 172.21.15.148:48698) 2024-04-25T09:54:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f695ffff640 1 -- 172.21.15.148:0/3279512652 learned_addr learned my addr 172.21.15.148:0/3279512652 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f695f7fe640 1 --2- 172.21.15.148:0/3279512652 >> v2:172.21.15.188:3300/0 conn(0x7f696015db70 0x7f6960162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378692663 0 0) 0x7f6960162690 con 0x7f69600b5c30 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.864+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6948003520 con 0x7f69600b5c30 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3271860577 0 0) 0x7f6948003520 con 0x7f69600b5c30 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 >> v2:172.21.15.188:3300/0 conn(0x7f696015db70 msgr2=0x7f6960162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 --2- 172.21.15.148:0/3279512652 >> v2:172.21.15.188:3300/0 conn(0x7f696015db70 0x7f6960162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 >> v1:172.21.15.188:6789/0 conn(0x7f69600b6cb0 legacy=0x7f69600b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600d1ff0 con 0x7f69600b5c30 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f69580019f0 con 0x7f69600b5c30 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 >> v1:172.21.15.148:6789/0 conn(0x7f69600b5c30 legacy=0x7f69600d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69480039b0 0x7f6948003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6948004800 0x7f6948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69480051c0 0x7f694800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600d1d10 con 0x7f69480039b0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694800c060 con 0x7f6948004800 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694800c260 con 0x7f69480051c0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695f7fe640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69480039b0 0x7f6948003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695effd640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6948004800 0x7f6948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695ffff640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69480051c0 0x7f694800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695f7fe640 1 -- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69480051c0 msgr2=0x7f694800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695f7fe640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69480051c0 0x7f694800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695f7fe640 1 -- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6948004800 msgr2=0x7f6948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695f7fe640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6948004800 0x7f6948004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695f7fe640 1 -- 172.21.15.148:0/3279512652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6948003520 con 0x7f69480039b0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.865+0000 7f695ffff640 1 -- 172.21.15.148:0/3279512652 reap_dead start 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f695f7fe640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69480039b0 0x7f6948003dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6950009ac0 tx=0x7f6948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6391b9d4da39234d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950036070 con 0x7f69480039b0 2024-04-25T09:54:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950031400 con 0x7f69480039b0 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f695e7fc640 1 -- 172.21.15.148:0/3279512652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950009690 con 0x7f69480039b0 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 -- 172.21.15.148:0/3279512652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69480039b0 msgr2=0x7f6948003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69480039b0 0x7f6948003dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6950009ac0 tx=0x7f6948005830 comp rx=0 tx=0).stop 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 -- 172.21.15.148:0/3279512652 shutdown_connections 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 --2- 172.21.15.148:0/3279512652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69480039b0 0x7f6948003dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 -- 172.21.15.148:0/3279512652 >> 172.21.15.148:0/3279512652 conn(0x7f69600d0520 msgr2=0x7f69600d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 -- 172.21.15.148:0/3279512652 shutdown_connections 2024-04-25T09:54:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.866+0000 7f6965de0640 1 -- 172.21.15.148:0/3279512652 wait complete. 2024-04-25T09:54:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.868+0000 7f6965de0640 1 Processor -- start 2024-04-25T09:54:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.868+0000 7f6965de0640 1 -- start start 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.868+0000 7f6965de0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69480039b0 0x7f69601e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.868+0000 7f6965de0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 0x7f69601f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6965de0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69601f3b70 0x7f6960149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6965de0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600ce530 con 0x7f69600cb280 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6965de0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696015df50 con 0x7f69480039b0 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6965de0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69600cf1c0 con 0x7f69601f3b70 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 0x7f69601f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 0x7f69601f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33378/0 (socket says 172.21.15.148:33378) 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 -- 172.21.15.148:0/1697224874 learned_addr learned my addr 172.21.15.148:0/1697224874 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 -- 172.21.15.148:0/1697224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69601f3b70 msgr2=0x7f6960149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69601f3b70 0x7f6960149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 -- 172.21.15.148:0/1697224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69480039b0 msgr2=0x7f69601e63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69480039b0 0x7f69601e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 -- 172.21.15.148:0/1697224874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69601e6920 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f695effd640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 0x7f69601f3630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f695400f7f0 tx=0x7f695400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540050d0 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954018070 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954005490 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69600cb7b0 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.869+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69600cbd70 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.870+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960073990 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.871+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f695400d6b0 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.871+0000 7f6947fff640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f694c06b4d0 0x7f694c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.871+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f695408a2d0 con 0x7f69600cb280 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.871+0000 7f695f7fe640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f694c06b4d0 0x7f694c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:46.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.872+0000 7f695f7fe640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f694c06b4d0 0x7f694c06d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6950033000 tx=0x7f695002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:46.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:46.875+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6954058e10 con 0x7f69600cb280 2024-04-25T09:54:47.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.051+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69600d0e20 con 0x7f69600cb280 2024-04-25T09:54:47.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.052+0000 7f6947fff640 1 -- 172.21.15.148:0/1697224874 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f6954081e20 con 0x7f69600cb280 2024-04-25T09:54:47.055 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:47.055 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:47.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f694c06b4d0 msgr2=0x7f694c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f694c06b4d0 0x7f694c06d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6950033000 tx=0x7f695002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 msgr2=0x7f69601f3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 0x7f69601f3630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f695400f7f0 tx=0x7f695400fcb0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 shutdown_connections 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f694c06b4d0 0x7f694c06d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69601f3b70 0x7f6960149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69600cb280 0x7f69601f3630 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 --2- 172.21.15.148:0/1697224874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69480039b0 0x7f69601e63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 >> 172.21.15.148:0/1697224874 conn(0x7f69600d0520 msgr2=0x7f69600c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 shutdown_connections 2024-04-25T09:54:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.054+0000 7f6965de0640 1 -- 172.21.15.148:0/1697224874 wait complete. 2024-04-25T09:54:47.068 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T09:54:47.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.164+0000 7fb7108cb640 1 Processor -- start 2024-04-25T09:54:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.164+0000 7fb7108cb640 1 -- start start 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.164+0000 7fb7108cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb7108cb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb70c0d0ad0 0x7fb70c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb7108cb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c0d1df0 con 0x7fb70c0b9770 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb7108cb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c0d20d0 con 0x7fb70c0d0ad0 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb7108cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0d23b0 con 0x7fb70c15dd30 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70a59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70ad9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb70c0d0ad0 0x7fb70c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70a59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c0b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56212/0 (socket says 172.21.15.148:56212) 2024-04-25T09:54:47.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70ad9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb70c0d0ad0 0x7fb70c0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57290/0 (socket says 172.21.15.148:57290) 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70a59c640 1 -- 172.21.15.148:0/554303733 learned_addr learned my addr 172.21.15.148:0/554303733 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70ad9d640 1 -- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 msgr2=0x7fb70c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70ad9d640 1 --2- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70ad9d640 1 -- 172.21.15.148:0/554303733 >> v1:172.21.15.188:6789/0 conn(0x7fb70c0b9770 legacy=0x7fb70c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70ad9d640 1 -- 172.21.15.148:0/554303733 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0d2690 con 0x7fb70c0d0ad0 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70959a640 1 -- 172.21.15.148:0/554303733 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905396051 0 0) 0x7fb70c0d1df0 con 0x7fb70c0b9770 2024-04-25T09:54:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.165+0000 7fb70a59c640 1 --2- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.166+0000 7fb70ad9d640 1 --2- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c0b6a40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009d90 tx=0x7fb6fc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ce2d822ee5025f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.166+0000 7fb70959a640 1 -- 172.21.15.148:0/554303733 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc031600 con 0x7fb70c0d0ad0 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.166+0000 7fb70959a640 1 -- 172.21.15.148:0/554303733 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc0317a0 con 0x7fb70c0d0ad0 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.166+0000 7fb70959a640 1 -- 172.21.15.148:0/554303733 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc031b70 con 0x7fb70c0d0ad0 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 -- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 msgr2=0x7fb70c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 --2- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c0b6a40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009d90 tx=0x7fb6fc02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 -- 172.21.15.148:0/554303733 shutdown_connections 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 --2- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c0b6a40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 --2- 172.21.15.148:0/554303733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 -- 172.21.15.148:0/554303733 >> 172.21.15.148:0/554303733 conn(0x7fb70c061c30 msgr2=0x7fb70c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 -- 172.21.15.148:0/554303733 shutdown_connections 2024-04-25T09:54:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.167+0000 7fb7108cb640 1 -- 172.21.15.148:0/554303733 wait complete. 2024-04-25T09:54:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.168+0000 7fb7108cb640 1 Processor -- start 2024-04-25T09:54:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.168+0000 7fb7108cb640 1 -- start start 2024-04-25T09:54:47.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb7108cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 0x7fb70c0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb7108cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb7108cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c1f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb7108cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0d1b90 con 0x7fb70c0b9770 2024-04-25T09:54:47.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb7108cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0d1a10 con 0x7fb70c0d0ad0 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb7108cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0d2e90 con 0x7fb70c15dd30 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 0x7fb70c0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb709d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 0x7fb70c0b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33382/0 (socket says 172.21.15.148:33382) 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 -- 172.21.15.148:0/4175793730 learned_addr learned my addr 172.21.15.148:0/4175793730 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c1f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 -- 172.21.15.148:0/4175793730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 msgr2=0x7fb70c1f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c1f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 -- 172.21.15.148:0/4175793730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 msgr2=0x7fb70c1592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c1592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.169+0000 7fb70a59c640 1 -- 172.21.15.148:0/4175793730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0b4420 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.170+0000 7fb70a59c640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 0x7fb70c0b3ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb70400e9e0 tx=0x7fb70400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.170+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704017070 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.170+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400c9d0 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.170+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c0ab250 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.170+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400cd80 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.170+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c0cbb70 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.171+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c0cf840 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.171+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb70401a070 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.171+0000 7fb702ffd640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb6ec06b4d0 0x7fb6ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.172+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb70408a6d0 con 0x7fb70c0b9770 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.172+0000 7fb709d9b640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb6ec06b4d0 0x7fb6ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.172+0000 7fb709d9b640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb6ec06b4d0 0x7fb6ec06d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0ce8f0 tx=0x7fb6f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.176+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb70c0cf840 con 0x7fb70c0b9770 2024-04-25T09:54:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.346+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb70c073990 con 0x7fb70c0b9770 2024-04-25T09:54:47.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.347+0000 7fb702ffd640 1 -- 172.21.15.148:0/4175793730 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fb70c0b4420 con 0x7fb70c0b9770 2024-04-25T09:54:47.350 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:47.350 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb6ec06b4d0 msgr2=0x7fb6ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb6ec06b4d0 0x7fb6ec06d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0ce8f0 tx=0x7fb6f4009040 comp rx=0 tx=0).stop 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 msgr2=0x7fb70c0b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 0x7fb70c0b3ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb70400e9e0 tx=0x7fb70400eea0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 shutdown_connections 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb6ec06b4d0 0x7fb6ec06d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb70c15dd30 0x7fb70c1f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb70c0d0ad0 0x7fb70c1592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 --2- 172.21.15.148:0/4175793730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb70c0b9770 0x7fb70c0b3ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 >> 172.21.15.148:0/4175793730 conn(0x7fb70c061c30 msgr2=0x7fb70c06c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 shutdown_connections 2024-04-25T09:54:47.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.349+0000 7fb7108cb640 1 -- 172.21.15.148:0/4175793730 wait complete. 2024-04-25T09:54:47.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T09:54:47.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T09:54:47.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T09:54:47.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T09:54:47.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T09:54:47.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T09:54:47.366 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T09:54:47.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T09:54:47.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.527+0000 7ff89b0c0640 1 Processor -- start 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.527+0000 7ff89b0c0640 1 -- start start 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.528+0000 7ff89b0c0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89406bbf0 0x7ff89406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.528+0000 7ff89b0c0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff8940ad9b0 0x7ff8940ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.528+0000 7ff89b0c0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940b3be0 con 0x7ff8940b3e30 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.528+0000 7ff89b0c0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff894157d80 con 0x7ff8940ad9b0 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.528+0000 7ff89b0c0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89415a4a0 con 0x7ff89406bbf0 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.528+0000 7ff8998bd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff8940b3e30 0x7ff8940b4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43078/0 (socket says 172.21.15.148:43078) 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff8998bd640 1 -- 172.21.15.148:0/2855915868 learned_addr learned my addr 172.21.15.148:0/2855915868 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff89a0be640 1 --2- 172.21.15.148:0/2855915868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89406bbf0 0x7ff89406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff89a8bf640 1 --2- 172.21.15.148:0/2855915868 >> v2:172.21.15.148:3300/0 conn(0x7ff8940ad9b0 0x7ff8940ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff89a0be640 1 -- 172.21.15.148:0/2855915868 >> v2:172.21.15.148:3300/0 conn(0x7ff8940ad9b0 msgr2=0x7ff8940ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff89a0be640 1 --2- 172.21.15.148:0/2855915868 >> v2:172.21.15.148:3300/0 conn(0x7ff8940ad9b0 0x7ff8940ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff89a0be640 1 -- 172.21.15.148:0/2855915868 >> v1:172.21.15.188:6789/0 conn(0x7ff8940b3e30 legacy=0x7ff8940b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.529+0000 7ff89a0be640 1 -- 172.21.15.148:0/2855915868 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89415a780 con 0x7ff89406bbf0 2024-04-25T09:54:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.530+0000 7f0b74d61640 1 Processor -- start 2024-04-25T09:54:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.530+0000 7f0b74d61640 1 -- start start 2024-04-25T09:54:47.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.530+0000 7ff89a0be640 1 --2- 172.21.15.148:0/2855915868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89406bbf0 0x7ff89406bfd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8900097a0 tx=0x7ff89002a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e70cc2c85281eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.531+0000 7f4bbc04d640 1 Processor -- start 2024-04-25T09:54:47.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.531+0000 7f4bbc04d640 1 -- start start 2024-04-25T09:54:47.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7ff32279e640 1 Processor -- start 2024-04-25T09:54:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b74d61640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0b68054b10 0x7f0b68051ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b74d61640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b68053bc0 0x7f0b68051ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b74d61640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b68054fa0 con 0x7f0b680e1270 2024-04-25T09:54:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b74d61640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b680e2660 con 0x7f0b68054b10 2024-04-25T09:54:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b74d61640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b68055280 con 0x7f0b68053bc0 2024-04-25T09:54:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7ff32279e640 1 -- start start 2024-04-25T09:54:47.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b68053bc0 0x7f0b68051ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b68053bc0 0x7f0b68051ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57298/0 (socket says 172.21.15.148:57298) 2024-04-25T09:54:47.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/3668938619 learned_addr learned my addr 172.21.15.148:0/3668938619 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f4bbc04d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 0x7f4bb406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6dd9b640 1 --2- 172.21.15.148:0/3668938619 >> v2:172.21.15.148:3300/0 conn(0x7f0b68054b10 0x7f0b68051ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f4bbc04d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4bb40cfd80 0x7f4bb406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f4bbc04d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb4161a50 con 0x7f4bb40cc8b0 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f4bbc04d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb4161800 con 0x7f4bb40cfd80 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f4bbc04d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb4161d30 con 0x7f4bb4161100 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb9de9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 0x7f4bb406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb95e8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4bb40cc8b0 0x7f4bb406d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48700/0 (socket says 172.21.15.148:48700) 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb95e8640 1 -- 172.21.15.148:0/2542798184 learned_addr learned my addr 172.21.15.148:0/2542798184 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb9de9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 0x7f4bb406e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56246/0 (socket says 172.21.15.148:56246) 2024-04-25T09:54:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bba5ea640 1 --2- 172.21.15.148:0/2542798184 >> v2:172.21.15.188:3300/0 conn(0x7f4bb40cfd80 0x7f4bb406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f97c950e640 1 Processor -- start 2024-04-25T09:54:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f97c950e640 1 -- start start 2024-04-25T09:54:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb9de9640 1 -- 172.21.15.148:0/2542798184 >> v2:172.21.15.188:3300/0 conn(0x7f4bb40cfd80 msgr2=0x7f4bb406ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb9de9640 1 --2- 172.21.15.148:0/2542798184 >> v2:172.21.15.188:3300/0 conn(0x7f4bb40cfd80 0x7f4bb406ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb9de9640 1 -- 172.21.15.148:0/2542798184 >> v1:172.21.15.148:6789/0 conn(0x7f4bb40cc8b0 legacy=0x7f4bb406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f4bb9de9640 1 -- 172.21.15.148:0/2542798184 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4162010 con 0x7f4bb4161100 2024-04-25T09:54:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7ff32279e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff31400a550 0x7ff3140e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/3668938619 >> v2:172.21.15.148:3300/0 conn(0x7f0b68054b10 msgr2=0x7f0b68051ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 --2- 172.21.15.148:0/3668938619 >> v2:172.21.15.148:3300/0 conn(0x7f0b68054b10 0x7f0b68051ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/3668938619 >> v1:172.21.15.188:6789/0 conn(0x7f0b680e1270 legacy=0x7f0b680534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.532+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/3668938619 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68055560 con 0x7f0b68053bc0 2024-04-25T09:54:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7f0b6dd9b640 1 --2- 172.21.15.148:0/3668938619 >> v2:172.21.15.148:3300/0 conn(0x7f0b68054b10 0x7f0b68051ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.533+0000 7ff31b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff31400a550 0x7ff3140e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff32279e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314051cc0 con 0x7ff3140e59e0 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff31b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff31400a550 0x7ff3140e4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57314/0 (socket says 172.21.15.148:57314) 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff31b7fe640 1 -- 172.21.15.148:0/1806640725 learned_addr learned my addr 172.21.15.148:0/1806640725 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff314051fa0 con 0x7ff3140e3fa0 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff314052280 con 0x7ff31400a550 2024-04-25T09:54:47.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff8990bc640 1 -- 172.21.15.148:0/2855915868 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890037070 con 0x7ff89406bbf0 2024-04-25T09:54:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff8990bc640 1 -- 172.21.15.148:0/2855915868 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89002ad30 con 0x7ff89406bbf0 2024-04-25T09:54:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f0b6ed9d640 1 --2- 172.21.15.148:0/3668938619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 0x7f0b68051ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b70068fd0 tx=0x7f0b700a51c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3591cd07184d185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f4bb9de9640 1 --2- 172.21.15.148:0/2542798184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 0x7f4bb406e4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e7e5189f446cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f97c950e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c415dbd0 0x7f97c4161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f97c950e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97c40b5ae0 0x7f97c40d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f97c950e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c4161e70 con 0x7f97c40b95c0 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f97c950e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c40d1ce0 con 0x7f97c40b5ae0 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f97c950e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c4162150 con 0x7f97c415dbd0 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff31b7fe640 1 -- 172.21.15.148:0/1806640725 >> v1:172.21.15.188:6789/0 conn(0x7ff3140e3fa0 legacy=0x7ff3140e09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff31b7fe640 1 -- 172.21.15.148:0/1806640725 >> v1:172.21.15.148:6789/0 conn(0x7ff3140e59e0 legacy=0x7ff314009e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff31b7fe640 1 -- 172.21.15.148:0/1806640725 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314052560 con 0x7ff31400a550 2024-04-25T09:54:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7ff8990bc640 1 -- 172.21.15.148:0/2855915868 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890032640 con 0x7ff89406bbf0 2024-04-25T09:54:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f4bb8de7640 1 -- 172.21.15.148:0/2542798184 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0037070 con 0x7f4bb4161100 2024-04-25T09:54:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.534+0000 7f97c37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f97c40b95c0 0x7f97c40b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48714/0 (socket says 172.21.15.148:48714) 2024-04-25T09:54:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c37fe640 1 -- 172.21.15.148:0/4242463845 learned_addr learned my addr 172.21.15.148:0/4242463845 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f4bb8de7640 1 -- 172.21.15.148:0/2542798184 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb0032440 con 0x7f4bb4161100 2024-04-25T09:54:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f4bb8de7640 1 -- 172.21.15.148:0/2542798184 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0037070 con 0x7f4bb4161100 2024-04-25T09:54:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 -- 172.21.15.148:0/2855915868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89406bbf0 msgr2=0x7ff89406bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c3fff640 1 --2- 172.21.15.148:0/4242463845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c415dbd0 0x7f97c4161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c8d0d640 1 --2- 172.21.15.148:0/4242463845 >> v2:172.21.15.188:3300/0 conn(0x7f97c40b5ae0 0x7f97c40d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f4c76c8c640 1 Processor -- start 2024-04-25T09:54:47.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c2ffd640 1 -- 172.21.15.148:0/4242463845 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077153953 0 0) 0x7f97c4161e70 con 0x7f97c40b95c0 2024-04-25T09:54:47.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c2ffd640 1 -- 172.21.15.148:0/4242463845 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b0003520 con 0x7f97c40b95c0 2024-04-25T09:54:47.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7fa4d6922640 1 Processor -- start 2024-04-25T09:54:47.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 --2- 172.21.15.148:0/2855915868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89406bbf0 0x7ff89406bfd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8900097a0 tx=0x7ff89002a0a0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 -- 172.21.15.148:0/2855915868 shutdown_connections 2024-04-25T09:54:47.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 --2- 172.21.15.148:0/2855915868 >> v2:172.21.15.148:3300/0 conn(0x7ff8940ad9b0 0x7ff8940ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 --2- 172.21.15.148:0/2855915868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff89406bbf0 0x7ff89406bfd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 -- 172.21.15.148:0/2855915868 >> 172.21.15.148:0/2855915868 conn(0x7ff894071ad0 msgr2=0x7ff894071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7fa4d6922640 1 -- start start 2024-04-25T09:54:47.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f0b6d59a640 1 -- 172.21.15.148:0/3668938619 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700b2070 con 0x7f0b68053bc0 2024-04-25T09:54:47.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f0b6d59a640 1 -- 172.21.15.148:0/3668938619 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700ad440 con 0x7f0b68053bc0 2024-04-25T09:54:47.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f0b6d59a640 1 -- 172.21.15.148:0/3668938619 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700ad870 con 0x7f0b68053bc0 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 -- 172.21.15.148:0/2542798184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 msgr2=0x7f4bb406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 --2- 172.21.15.148:0/2542798184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 0x7f4bb406e4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002a180 comp rx=0 tx=0).stop 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 -- 172.21.15.148:0/2542798184 shutdown_connections 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 --2- 172.21.15.148:0/2542798184 >> v2:172.21.15.188:3300/0 conn(0x7f4bb40cfd80 0x7f4bb406ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 --2- 172.21.15.148:0/2542798184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb4161100 0x7f4bb406e4f0 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002a180 comp rx=0 tx=0).stop 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 -- 172.21.15.148:0/2542798184 >> 172.21.15.148:0/2542798184 conn(0x7f4bb406abe0 msgr2=0x7f4bb406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f4c76c8c640 1 -- start start 2024-04-25T09:54:47.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c3fff640 1 -- 172.21.15.148:0/4242463845 >> v2:172.21.15.188:3300/0 conn(0x7f97c40b5ae0 msgr2=0x7f97c40d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c3fff640 1 --2- 172.21.15.148:0/4242463845 >> v2:172.21.15.188:3300/0 conn(0x7f97c40b5ae0 0x7f97c40d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7ff31b7fe640 1 --2- 172.21.15.148:0/1806640725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140e4d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3080097f0 tx=0x7ff30802a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd14395b414b231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c3fff640 1 -- 172.21.15.148:0/4242463845 >> v1:172.21.15.148:6789/0 conn(0x7f97c40b95c0 legacy=0x7f97c40b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7f97c3fff640 1 -- 172.21.15.148:0/4242463845 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4162430 con 0x7f97c415dbd0 2024-04-25T09:54:47.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f97c3fff640 1 --2- 172.21.15.148:0/4242463845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c415dbd0 0x7f97c4161620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009d90 tx=0x7f97b402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8155995c303afa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7ff31affd640 1 -- 172.21.15.148:0/1806640725 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308037070 con 0x7ff31400a550 2024-04-25T09:54:47.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f97c2ffd640 1 -- 172.21.15.148:0/4242463845 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4007740 con 0x7f97c415dbd0 2024-04-25T09:54:47.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f97c2ffd640 1 -- 172.21.15.148:0/4242463845 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b40078e0 con 0x7f97c415dbd0 2024-04-25T09:54:47.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f97c2ffd640 1 -- 172.21.15.148:0/4242463845 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4007740 con 0x7f97c415dbd0 2024-04-25T09:54:47.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7ff31affd640 1 -- 172.21.15.148:0/1806640725 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30802adc0 con 0x7ff31400a550 2024-04-25T09:54:47.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7ff31affd640 1 -- 172.21.15.148:0/1806640725 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308032660 con 0x7ff31400a550 2024-04-25T09:54:47.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7fa4d6922640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4d00b5bb0 0x7fa4d00b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c76c8c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4d6922640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d00cf1d0 con 0x7fa4d0161240 2024-04-25T09:54:47.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c75c8a640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4c7006f470 0x7f4c701543d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43106/0 (socket says 172.21.15.148:43106) 2024-04-25T09:54:47.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f0b74d61640 1 -- 172.21.15.148:0/3668938619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 msgr2=0x7f0b68051ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f0b74d61640 1 --2- 172.21.15.148:0/3668938619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 0x7f0b68051ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b70068fd0 tx=0x7f0b700a51c0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4d6922640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d0161620 con 0x7fa4d0161890 2024-04-25T09:54:47.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4d6922640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d00b7120 con 0x7fa4d00b5bb0 2024-04-25T09:54:47.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4d4ebf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4d00b5bb0 0x7fa4d00b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c76c8c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c70061600 0x7f4c7006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4d4ebf640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4d00b5bb0 0x7fa4d00b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57336/0 (socket says 172.21.15.148:57336) 2024-04-25T09:54:47.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4d4ebf640 1 -- 172.21.15.148:0/3442375652 learned_addr learned my addr 172.21.15.148:0/3442375652 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c75c8a640 1 -- 172.21.15.148:0/3383700398 learned_addr learned my addr 172.21.15.148:0/3383700398 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f0b74d61640 1 -- 172.21.15.148:0/3668938619 shutdown_connections 2024-04-25T09:54:47.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f0b74d61640 1 --2- 172.21.15.148:0/3668938619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 0x7f0b68051ff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f0b74d61640 1 --2- 172.21.15.148:0/3668938619 >> v2:172.21.15.148:3300/0 conn(0x7f0b68054b10 0x7f0b68051ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f0b74d61640 1 -- 172.21.15.148:0/3668938619 >> 172.21.15.148:0/3668938619 conn(0x7f0b68020000 msgr2=0x7f0b68056d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c75489640 1 --2- 172.21.15.148:0/3383700398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c700ae0d0 con 0x7f4c7006f470 2024-04-25T09:54:47.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 -- 172.21.15.148:0/2855915868 shutdown_connections 2024-04-25T09:54:47.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.535+0000 7ff89b0c0640 1 -- 172.21.15.148:0/2855915868 wait complete. 2024-04-25T09:54:47.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7ff89b0c0640 1 Processor -- start 2024-04-25T09:54:47.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7ff89b0c0640 1 -- start start 2024-04-25T09:54:47.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c700ae3b0 con 0x7f4c70061600 2024-04-25T09:54:47.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700ae690 con 0x7f4c7015e670 2024-04-25T09:54:47.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f72cc9b0640 1 Processor -- start 2024-04-25T09:54:47.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4ceffd640 1 -- 172.21.15.148:0/3442375652 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785735329 0 0) 0x7fa4d00cf1d0 con 0x7fa4d0161240 2024-04-25T09:54:47.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7fa4ceffd640 1 -- 172.21.15.148:0/3442375652 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4bc003520 con 0x7fa4d0161240 2024-04-25T09:54:47.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c74c88640 1 -- 172.21.15.148:0/3383700398 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022785726 0 0) 0x7f4c700ae0d0 con 0x7f4c7006f470 2024-04-25T09:54:47.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c74c88640 1 -- 172.21.15.148:0/3383700398 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c5c005730 con 0x7f4c7006f470 2024-04-25T09:54:47.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7fa4d4ebf640 1 -- 172.21.15.148:0/3442375652 >> v1:172.21.15.188:6789/0 conn(0x7fa4d0161890 legacy=0x7fa4d00b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7fa4d4ebf640 1 -- 172.21.15.148:0/3442375652 >> v1:172.21.15.148:6789/0 conn(0x7fa4d0161240 legacy=0x7fa4d00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7fa4d4ebf640 1 -- 172.21.15.148:0/3442375652 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0161e00 con 0x7fa4d00b5bb0 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 -- 172.21.15.148:0/4242463845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c415dbd0 msgr2=0x7f97c4161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 --2- 172.21.15.148:0/4242463845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c415dbd0 0x7f97c4161620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009d90 tx=0x7f97b402a680 comp rx=0 tx=0).stop 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89b0c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff89406bbf0 0x7ff89414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89b0c0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940ad9b0 0x7ff894150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89b0c0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940b3e30 0x7ff89406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89b0c0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89415b020 con 0x7ff89406bbf0 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89b0c0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89415ad40 con 0x7ff8940ad9b0 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89b0c0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89406c510 con 0x7ff8940b3e30 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 -- 172.21.15.148:0/4242463845 shutdown_connections 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 --2- 172.21.15.148:0/4242463845 >> v2:172.21.15.188:3300/0 conn(0x7f97c40b5ae0 0x7f97c40d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 --2- 172.21.15.148:0/4242463845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c415dbd0 0x7f97c4161620 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 -- 172.21.15.148:0/4242463845 >> 172.21.15.148:0/4242463845 conn(0x7f97c40d03d0 msgr2=0x7f97c40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f72cc9b0640 1 -- start start 2024-04-25T09:54:47.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89a0be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff89406bbf0 0x7ff89414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89a0be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff89406bbf0 0x7ff89414dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33422/0 (socket says 172.21.15.148:33422) 2024-04-25T09:54:47.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89a0be640 1 -- 172.21.15.148:0/1148440159 learned_addr learned my addr 172.21.15.148:0/1148440159 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff89a8bf640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940b3e30 0x7ff89406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 -- 172.21.15.148:0/4242463845 shutdown_connections 2024-04-25T09:54:47.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7ff8998bd640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940ad9b0 0x7ff894150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a0be640 1 -- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940b3e30 msgr2=0x7ff89406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a0be640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940b3e30 0x7ff89406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a0be640 1 -- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940ad9b0 msgr2=0x7ff894150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a0be640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940ad9b0 0x7ff894150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a0be640 1 -- 172.21.15.148:0/1148440159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894099f70 con 0x7ff89406bbf0 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.537+0000 7f4c75489640 1 -- 172.21.15.148:0/3383700398 >> v2:172.21.15.148:3300/0 conn(0x7f4c70061600 msgr2=0x7f4c7006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4c7648b640 1 --2- 172.21.15.148:0/3383700398 >> v2:172.21.15.148:3300/0 conn(0x7f4c70061600 0x7f4c7006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4c75489640 1 --2- 172.21.15.148:0/3383700398 >> v2:172.21.15.148:3300/0 conn(0x7f4c70061600 0x7f4c7006bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4c75489640 1 -- 172.21.15.148:0/3383700398 >> v1:172.21.15.188:6789/0 conn(0x7f4c7006f470 legacy=0x7f4c701543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4c75489640 1 -- 172.21.15.148:0/3383700398 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700ae970 con 0x7f4c7015e670 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a8bf640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940b3e30 0x7ff89406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:47.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4c7648b640 1 --2- 172.21.15.148:0/3383700398 >> v2:172.21.15.148:3300/0 conn(0x7f4c70061600 0x7f4c7006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:47.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 -- 172.21.15.148:0/2542798184 shutdown_connections 2024-04-25T09:54:47.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.536+0000 7f4bbc04d640 1 -- 172.21.15.148:0/2542798184 wait complete. 2024-04-25T09:54:47.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4bbc04d640 1 Processor -- start 2024-04-25T09:54:47.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f4bbc04d640 1 -- start start 2024-04-25T09:54:47.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4c75489640 1 --2- 172.21.15.148:0/3383700398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7015ad00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c600097a0 tx=0x7f4c6002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177b94e786ebdb28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.538+0000 7f97c950e640 1 -- 172.21.15.148:0/4242463845 wait complete. 2024-04-25T09:54:47.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7fa4d4ebf640 1 --2- 172.21.15.148:0/3442375652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 0x7fa4d00b6be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009870 tx=0x7fa4b802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53972b2984ca12d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7ff89a0be640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff89406bbf0 0x7ff89414dc00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098c0 tx=0x7ff8900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4c74c88640 1 -- 172.21.15.148:0/3383700398 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60037070 con 0x7f4c7015e670 2024-04-25T09:54:47.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4c74c88640 1 -- 172.21.15.148:0/3383700398 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c60032440 con 0x7f4c7015e670 2024-04-25T09:54:47.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4c74c88640 1 -- 172.21.15.148:0/3383700398 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6002ae60 con 0x7f4c7015e670 2024-04-25T09:54:47.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72cc9b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72c806ea90 0x7f72c8071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72cc9b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72c806d2d0 0x7f72c806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72c659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f72c8161fc0 0x7f72c81623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48724/0 (socket says 172.21.15.148:48724) 2024-04-25T09:54:47.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4bbc04d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb40cc8b0 0x7f4bb42045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4bbc04d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfd80 0x7f4bb4208ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72c6d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72c806d2d0 0x7f72c806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72c6d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72c806d2d0 0x7f72c806d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33436/0 (socket says 172.21.15.148:33436) 2024-04-25T09:54:47.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72c659c640 1 -- 172.21.15.148:0/389306285 learned_addr learned my addr 172.21.15.148:0/389306285 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f4bbc04d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb408a570 0x7f4bb415db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c8162ba0 con 0x7f72c8161fc0 2024-04-25T09:54:47.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c8165be0 con 0x7f72c806d2d0 2024-04-25T09:54:47.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.539+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c806dc40 con 0x7f72c806ea90 2024-04-25T09:54:47.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f72c5d9b640 1 --2- 172.21.15.148:0/389306285 >> v2:172.21.15.188:3300/0 conn(0x7f72c806ea90 0x7f72c8071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f0b74d61640 1 -- 172.21.15.148:0/3668938619 shutdown_connections 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bbc04d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb41614e0 con 0x7f4bb40cfd80 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bbc04d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb40cd5a0 con 0x7f4bb40cc8b0 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bbc04d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb4071ec0 con 0x7f4bb408a570 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bb95e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfd80 0x7f4bb4208ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 msgr2=0x7ff3140e4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bb95e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfd80 0x7f4bb4208ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33442/0 (socket says 172.21.15.148:33442) 2024-04-25T09:54:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bb95e8640 1 -- 172.21.15.148:0/1737374683 learned_addr learned my addr 172.21.15.148:0/1737374683 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7ff32279e640 1 --2- 172.21.15.148:0/1806640725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140e4d70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3080097f0 tx=0x7ff30802a0b0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bba5ea640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb408a570 0x7f4bb415db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890037030 con 0x7ff89406bbf0 2024-04-25T09:54:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f97c950e640 1 Processor -- start 2024-04-25T09:54:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 msgr2=0x7f4c7015ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4c76c8c640 1 --2- 172.21.15.148:0/3383700398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7015ad00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4c600097a0 tx=0x7f4c6002a110 comp rx=0 tx=0).stop 2024-04-25T09:54:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7fa4ceffd640 1 -- 172.21.15.148:0/3442375652 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8037070 con 0x7fa4d00b5bb0 2024-04-25T09:54:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bb9de9640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb40cc8b0 0x7f4bb42045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7fa4ceffd640 1 -- 172.21.15.148:0/3442375652 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b8032440 con 0x7fa4d00b5bb0 2024-04-25T09:54:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 shutdown_connections 2024-04-25T09:54:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bba5ea640 1 -- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb40cc8b0 msgr2=0x7f4bb42045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bba5ea640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb40cc8b0 0x7f4bb42045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bba5ea640 1 -- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfd80 msgr2=0x7f4bb4208ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bba5ea640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfd80 0x7f4bb4208ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4bba5ea640 1 -- 172.21.15.148:0/1737374683 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb408ade0 con 0x7f4bb408a570 2024-04-25T09:54:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4c76c8c640 1 --2- 172.21.15.148:0/3383700398 >> v2:172.21.15.148:3300/0 conn(0x7f4c70061600 0x7f4c7006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 shutdown_connections 2024-04-25T09:54:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7ff32279e640 1 --2- 172.21.15.148:0/1806640725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140e4d70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 >> 172.21.15.148:0/1806640725 conn(0x7ff3140537e0 msgr2=0x7ff314053bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4c76c8c640 1 --2- 172.21.15.148:0/3383700398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7015ad00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 >> 172.21.15.148:0/3383700398 conn(0x7f4c70071880 msgr2=0x7f4c70071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7fa4ceffd640 1 -- 172.21.15.148:0/3442375652 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8032810 con 0x7fa4d00b5bb0 2024-04-25T09:54:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f0b74d61640 1 -- 172.21.15.148:0/3668938619 wait complete. 2024-04-25T09:54:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f72c6d9d640 1 -- 172.21.15.148:0/389306285 >> v2:172.21.15.188:3300/0 conn(0x7f72c806ea90 msgr2=0x7f72c8071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f72c6d9d640 1 --2- 172.21.15.148:0/389306285 >> v2:172.21.15.188:3300/0 conn(0x7f72c806ea90 0x7f72c8071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f72c6d9d640 1 -- 172.21.15.148:0/389306285 >> v1:172.21.15.148:6789/0 conn(0x7f72c8161fc0 legacy=0x7f72c81623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7f72c6d9d640 1 -- 172.21.15.148:0/389306285 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c8162e80 con 0x7f72c806d2d0 2024-04-25T09:54:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f72c6d9d640 1 --2- 172.21.15.148:0/389306285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c806d2d0 0x7f72c806d700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72c80713b0 tx=0x7f72b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720f8d8e6df093ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7ff89b0c0640 1 -- 172.21.15.148:0/1148440159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89409a250 con 0x7ff89406bbf0 2024-04-25T09:54:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.540+0000 7ff89b0c0640 1 -- 172.21.15.148:0/1148440159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89409a810 con 0x7ff89406bbf0 2024-04-25T09:54:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff890038040 con 0x7ff89406bbf0 2024-04-25T09:54:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89003c660 con 0x7ff89406bbf0 2024-04-25T09:54:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f72c559a640 1 -- 172.21.15.148:0/389306285 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8037070 con 0x7f72c806d2d0 2024-04-25T09:54:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f72c559a640 1 -- 172.21.15.148:0/389306285 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b8032440 con 0x7f72c806d2d0 2024-04-25T09:54:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f97c950e640 1 -- start start 2024-04-25T09:54:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f4bba5ea640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb408a570 0x7f4bb415db50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00eb70 tx=0x7f4bac00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff894073990 con 0x7ff89406bbf0 2024-04-25T09:54:47.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7fa4d6922640 1 -- 172.21.15.148:0/3442375652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 msgr2=0x7fa4d00b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7fa4d6922640 1 --2- 172.21.15.148:0/3442375652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 0x7fa4d00b6be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009870 tx=0x7fa4b802a270 comp rx=0 tx=0).stop 2024-04-25T09:54:47.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7fa4d6922640 1 -- 172.21.15.148:0/3442375652 shutdown_connections 2024-04-25T09:54:47.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 shutdown_connections 2024-04-25T09:54:47.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac018020 con 0x7f4bb408a570 2024-04-25T09:54:47.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7fa4d6922640 1 --2- 172.21.15.148:0/3442375652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 0x7fa4d00b6be0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac00cd20 con 0x7f4bb408a570 2024-04-25T09:54:47.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0146c0 con 0x7f4bb408a570 2024-04-25T09:54:47.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7fa4d6922640 1 -- 172.21.15.148:0/3442375652 >> 172.21.15.148:0/3442375652 conn(0x7fa4d00d04a0 msgr2=0x7fa4d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f4c76c8c640 1 -- 172.21.15.148:0/3383700398 wait complete. 2024-04-25T09:54:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f0b74d61640 1 Processor -- start 2024-04-25T09:54:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f0b74d61640 1 -- start start 2024-04-25T09:54:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f97c950e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97c40b5ae0 0x7f97c41f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.541+0000 7f72c559a640 1 -- 172.21.15.148:0/389306285 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b802ae60 con 0x7f72c806d2d0 2024-04-25T09:54:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7fa4d6922640 1 -- 172.21.15.148:0/3442375652 shutdown_connections 2024-04-25T09:54:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7fa4d6922640 1 -- 172.21.15.148:0/3442375652 wait complete. 2024-04-25T09:54:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 shutdown_connections 2024-04-25T09:54:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7f97c950e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c40b95c0 0x7f97c4160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.542+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff890043020 con 0x7ff89406bbf0 2024-04-25T09:54:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c950e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c415dbd0 0x7f97c4160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97c40b5ae0 0x7f97c41f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff32279e640 1 -- 172.21.15.148:0/1806640725 wait complete. 2024-04-25T09:54:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff88affd640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff87806b520 0x7ff87806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb40b2410 con 0x7f4bb408a570 2024-04-25T09:54:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb40b29b0 con 0x7f4bb408a570 2024-04-25T09:54:47.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff8900a9fd0 con 0x7ff89406bbf0 2024-04-25T09:54:47.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff88affd640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7ff878074ca0 0x7ff8780770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97c40b5ae0 0x7f97c41f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33458/0 (socket says 172.21.15.148:33458) 2024-04-25T09:54:47.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b74d61640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 0x7f0b680e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c3fff640 1 -- 172.21.15.148:0/456914199 learned_addr learned my addr 172.21.15.148:0/456914199 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b74d61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b680e1270 0x7f0b680e3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 --> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8780777b0 con 0x7ff878074ca0 2024-04-25T09:54:47.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8900ac5f0 con 0x7ff89406bbf0 2024-04-25T09:54:47.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40d26b0 con 0x7f97c40b5ae0 2024-04-25T09:54:47.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b74d61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b680cc6e0 0x7f0b6804c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c4161b60 con 0x7f97c415dbd0 2024-04-25T09:54:47.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40d2830 con 0x7f97c40b95c0 2024-04-25T09:54:47.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c806d2d0 msgr2=0x7f72c806d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/389306285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c806d2d0 0x7f72c806d700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72c80713b0 tx=0x7f72b802a110 comp rx=0 tx=0).stop 2024-04-25T09:54:47.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f4c76c8c640 1 Processor -- start 2024-04-25T09:54:47.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f97c37fe640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c40b95c0 0x7f97c4160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7ff89a8bf640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7ff878074ca0 0x7ff8780770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b74d61640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680577e0 con 0x7f0b680cc6e0 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b74d61640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68057660 con 0x7f0b68053bc0 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b74d61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680e4cf0 con 0x7f0b680e1270 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b6ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b680cc6e0 0x7f0b6804c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b6ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b680cc6e0 0x7f0b6804c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33470/0 (socket says 172.21.15.148:33470) 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/646121934 learned_addr learned my addr 172.21.15.148:0/646121934 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b6e59c640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 0x7f0b680e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.543+0000 7f0b6dd9b640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b680e1270 0x7f0b680e3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c8d0d640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c415dbd0 0x7f97c4160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f4c76c8c640 1 -- start start 2024-04-25T09:54:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c37fe640 1 -- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c415dbd0 msgr2=0x7f97c4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c37fe640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c415dbd0 0x7f97c4160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c37fe640 1 -- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97c40b5ae0 msgr2=0x7f97c41f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c37fe640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97c40b5ae0 0x7f97c41f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c37fe640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c415ee70 con 0x7f97c40b95c0 2024-04-25T09:54:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7ff8998bd640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff87806b520 0x7ff87806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7fa4d6922640 1 Processor -- start 2024-04-25T09:54:47.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f97c37fe640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c40b95c0 0x7f97c4160210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c430 tx=0x7f97ac00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/646121934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b680e1270 msgr2=0x7f0b680e3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f0b6ed9d640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b680e1270 0x7f0b680e3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/646121934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 msgr2=0x7f0b680e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f0b6ed9d640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b68053bc0 0x7f0b680e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/646121934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6804abf0 con 0x7f0b680cc6e0 2024-04-25T09:54:47.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f0b6ed9d640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b680cc6e0 0x7f0b6804c040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b70068d50 tx=0x7f0b70068960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 shutdown_connections 2024-04-25T09:54:47.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/389306285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c806d2d0 0x7f72c806d700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/389306285 >> v2:172.21.15.188:3300/0 conn(0x7f72c806ea90 0x7f72c8071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 >> 172.21.15.148:0/389306285 conn(0x7f72c806abe0 msgr2=0x7f72c806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700b2050 con 0x7f0b680cc6e0 2024-04-25T09:54:47.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700b3040 con 0x7f0b680cc6e0 2024-04-25T09:54:47.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700acae0 con 0x7f0b680cc6e0 2024-04-25T09:54:47.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7fa4d6922640 1 -- start start 2024-04-25T09:54:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7ff89a8bf640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7ff878074ca0 0x7ff8780770e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b680e4b20 con 0x7f0b680cc6e0 2024-04-25T09:54:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 shutdown_connections 2024-04-25T09:54:47.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac017070 con 0x7f97c40b95c0 2024-04-25T09:54:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac005ce0 con 0x7f97c40b95c0 2024-04-25T09:54:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac017070 con 0x7f97c40b95c0 2024-04-25T09:54:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7ff32279e640 1 Processor -- start 2024-04-25T09:54:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7ff32279e640 1 -- start start 2024-04-25T09:54:47.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6804d490 con 0x7f0b680cc6e0 2024-04-25T09:54:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.544+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bb4073a00 con 0x7f4bb408a570 2024-04-25T09:54:47.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7ff8998bd640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff87806b520 0x7ff87806d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8941fb1f0 tx=0x7ff88c005b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c76c8c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c70061600 0x7f4c70200940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c76c8c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7006f470 0x7f4c7009c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c76c8c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7009cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c76c8c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70061bd0 con 0x7f4c70061600 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c76c8c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70061a50 con 0x7f4c7006f470 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c76c8c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7006c130 con 0x7f4c7015e670 2024-04-25T09:54:47.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c75c8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c70061600 0x7f4c70200940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c75c8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c70061600 0x7f4c70200940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33486/0 (socket says 172.21.15.148:33486) 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.545+0000 7f4c75c8a640 1 -- 172.21.15.148:0/232298327 learned_addr learned my addr 172.21.15.148:0/232298327 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4c75c8a640 1 -- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 msgr2=0x7f4c7009cec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4c75c8a640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c7015e670 0x7f4c7009cec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4c75c8a640 1 -- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7006f470 msgr2=0x7f4c7009c980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4c75c8a640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c7006f470 0x7f4c7009c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4c75c8a640 1 -- 172.21.15.148:0/232298327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700ce9d0 con 0x7f4c70061600 2024-04-25T09:54:47.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7fa4d6922640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 0x7fa4d015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7fa4d6922640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4d0161240 0x7fa4d014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4c75c8a640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c70061600 0x7f4c70200940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c430 tx=0x7f4c6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f72cc9b0640 1 -- 172.21.15.148:0/389306285 wait complete. 2024-04-25T09:54:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bac005d10 con 0x7f4bb408a570 2024-04-25T09:54:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7fa4d6922640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d0161890 0x7fa4d014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7fa4cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4d0161240 0x7fa4d014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7fa4cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4d0161240 0x7fa4d014eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33488/0 (socket says 172.21.15.148:33488) 2024-04-25T09:54:47.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7fa4cf7fe640 1 -- 172.21.15.148:0/2913191116 learned_addr learned my addr 172.21.15.148:0/2913191116 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff32279e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4baa7fc640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b5a0 0x7f4b9806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c017070 con 0x7f4c70061600 2024-04-25T09:54:47.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c005ce0 con 0x7f4c70061600 2024-04-25T09:54:47.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0072d0 con 0x7f4c70061600 2024-04-25T09:54:47.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01626a0 con 0x7fa4d0161240 2024-04-25T09:54:47.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4c76c8c640 1 -- 172.21.15.148:0/232298327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c700cde50 con 0x7f4c70061600 2024-04-25T09:54:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff32279e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3140e3fa0 0x7ff3140dc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4c76c8c640 1 -- 172.21.15.148:0/232298327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c700ce410 con 0x7f4c70061600 2024-04-25T09:54:47.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f72cc9b0640 1 Processor -- start 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c415f150 con 0x7f97c40b95c0 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c415f710 con 0x7f97c40b95c0 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f97c3fff640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97c40b5ae0 0x7f97c41f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff31b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3140e3fa0 0x7ff3140dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff31b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3140e3fa0 0x7ff3140dc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56314/0 (socket says 172.21.15.148:56314) 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff31bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff31bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140d85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57378/0 (socket says 172.21.15.148:57378) 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff32279e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3140e59e0 0x7ff31404e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7ff320d3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3140e59e0 0x7ff31404e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b700c0020 con 0x7f0b680cc6e0 2024-04-25T09:54:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4bac08b190 con 0x7f4bb408a570 2024-04-25T09:54:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.547+0000 7f4bb9de9640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b5a0 0x7f4b9806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.546+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== osd.5 v2:172.21.15.188:6808/3945194105 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff8780777b0 con 0x7ff878074ca0 2024-04-25T09:54:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f4c76c8c640 1 -- 172.21.15.148:0/232298327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c70073990 con 0x7f4c70061600 2024-04-25T09:54:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f4baa7fc640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f4b98074d20 0x7f4b98077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 --> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bac012e50 con 0x7f4b98074d20 2024-04-25T09:54:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bac092050 con 0x7f4bb408a570 2024-04-25T09:54:47.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f0b5effd640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0b5006b5a0 0x7f0b5006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7fa4d4ebf640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d0161890 0x7fa4d014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f4bb95e8640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f4b98074d20 0x7f4b98077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f0b6e59c640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0b5006b5a0 0x7f0b5006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b701253d0 con 0x7f0b680cc6e0 2024-04-25T09:54:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d01623c0 con 0x7fa4d00b5bb0 2024-04-25T09:54:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d00d15f0 con 0x7fa4d0161890 2024-04-25T09:54:47.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f4bb9de9640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b5a0 0x7f4b9806da60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0034000 tx=0x7f4bb002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7ff32279e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140518d0 con 0x7ff3140e59e0 2024-04-25T09:54:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7ff32279e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314051750 con 0x7ff31400a550 2024-04-25T09:54:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7ff32279e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff314051a50 con 0x7ff3140e3fa0 2024-04-25T09:54:47.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7ff320d3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3140e59e0 0x7ff31404e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33504/0 (socket says 172.21.15.148:33504) 2024-04-25T09:54:47.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7ff320d3b640 1 -- 172.21.15.148:0/3897803446 learned_addr learned my addr 172.21.15.148:0/3897803446 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f0b6e59c640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0b5006b5a0 0x7f0b5006da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b68053680 tx=0x7f0b64009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f0b74d61640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f0b68054b10 0x7f0b680e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 --> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b68057440 con 0x7f0b68054b10 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.548+0000 7f72cc9b0640 1 -- start start 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f72cc9b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72c806d2d0 0x7f72c80afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f72cc9b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72c806ea90 0x7f72c815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7fa4cf7fe640 1 -- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d0161890 msgr2=0x7fa4d014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7fa4cf7fe640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4d0161890 0x7fa4d014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7fa4cf7fe640 1 -- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 msgr2=0x7fa4d015eaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7fa4cf7fe640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4d00b5bb0 0x7fa4d015eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7fa4cf7fe640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d00b4f30 con 0x7fa4d0161240 2024-04-25T09:54:47.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f72c659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72c806d2d0 0x7f72c80afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7f72c659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72c806d2d0 0x7f72c80afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57398/0 (socket says 172.21.15.148:57398) 2024-04-25T09:54:47.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72c659c640 1 -- 172.21.15.148:0/2859378607 learned_addr learned my addr 172.21.15.148:0/2859378607 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7fa4cf7fe640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4d0161240 0x7fa4d014eb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800ba50 tx=0x7fa4c800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c8161fc0 0x7f72c8156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8019070 con 0x7fa4d0161240 2024-04-25T09:54:47.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f0b6dd9b640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f0b68054b10 0x7f0b680e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c6c01a070 con 0x7f4c70061600 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72c5d9b640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72c806ea90 0x7f72c815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7ff320d3b640 1 -- 172.21.15.148:0/3897803446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3140e3fa0 msgr2=0x7ff3140dc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7ff320d3b640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3140e3fa0 0x7ff3140dc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7ff320d3b640 1 -- 172.21.15.148:0/3897803446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 msgr2=0x7ff3140d85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7ff320d3b640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff31400a550 0x7ff3140d85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.549+0000 7ff320d3b640 1 -- 172.21.15.148:0/3897803446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31404fec0 con 0x7ff3140e59e0 2024-04-25T09:54:47.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7ff320d3b640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3140e59e0 0x7ff31404e7b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff31c06ab40 tx=0x7ff31c0686c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c800cc10 con 0x7fa4d0161240 2024-04-25T09:54:47.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80719c0 con 0x7f72c8161fc0 2024-04-25T09:54:47.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c667fc640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c5406b5a0 0x7f4c5406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8163440 con 0x7f72c806d2d0 2024-04-25T09:54:47.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8071b40 con 0x7f72c806ea90 2024-04-25T09:54:47.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c6c089830 con 0x7f4c70061600 2024-04-25T09:54:47.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c75489640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c5406b5a0 0x7f4c5406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c667fc640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f4c54074ca0 0x7f4c540770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 --> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c6c014ec0 con 0x7f4c54074ca0 2024-04-25T09:54:47.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c6c053e60 con 0x7f4c70061600 2024-04-25T09:54:47.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f4bb95e8640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f4b98074d20 0x7f4b98077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f0b6dd9b640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f0b68054b10 0x7f0b680e5080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f4c7648b640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f4c54074ca0 0x7f4c540770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c073280 con 0x7ff3140e59e0 2024-04-25T09:54:47.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c092400 con 0x7ff3140e59e0 2024-04-25T09:54:47.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3140dce90 con 0x7ff3140e59e0 2024-04-25T09:54:47.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97c40b3aa0 con 0x7f97c40b95c0 2024-04-25T09:54:47.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f4c75489640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c5406b5a0 0x7f4c5406da60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c60034000 tx=0x7f4c6002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c092870 con 0x7ff3140e59e0 2024-04-25T09:54:47.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31404ad20 con 0x7ff3140e59e0 2024-04-25T09:54:47.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8014440 con 0x7fa4d0161240 2024-04-25T09:54:47.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97ac01a070 con 0x7f97c40b95c0 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.550+0000 7f72c6d9d640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c8161fc0 0x7f72c8156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f72c659c640 1 -- 172.21.15.148:0/2859378607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72c806ea90 msgr2=0x7f72c815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f72c659c640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72c806ea90 0x7f72c815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f72c659c640 1 -- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c8161fc0 msgr2=0x7f72c8156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f72c659c640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c8161fc0 0x7f72c8156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f72c659c640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c81fb0b0 con 0x7f72c806d2d0 2024-04-25T09:54:47.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f72c6d9d640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72c8161fc0 0x7f72c8156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:47.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d00b5210 con 0x7fa4d0161240 2024-04-25T09:54:47.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.551+0000 7f4c7648b640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f4c54074ca0 0x7f4c540770e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== osd.4 v2:172.21.15.188:6800/3303846211 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4bac012e50 con 0x7f4b98074d20 2024-04-25T09:54:47.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c0ff9640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97a406b8d0 0x7f97a406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f97ac005ae0 con 0x7f97c40b95c0 2024-04-25T09:54:47.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d00b57d0 con 0x7fa4d0161240 2024-04-25T09:54:47.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c3fff640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97a406b8d0 0x7f97a406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c0ff9640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f97a4074fd0 0x7f97a4077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97a4077ae0 con 0x7f97a4074fd0 2024-04-25T09:54:47.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97ac090050 con 0x7f97c40b95c0 2024-04-25T09:54:47.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.552+0000 7f97c8d0d640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f97a4074fd0 0x7f97a4077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.553+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff314005810 con 0x7ff3140e59e0 2024-04-25T09:54:47.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.553+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== osd.3 v2:172.21.15.148:6819/1795140808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0b68057440 con 0x7f0b68054b10 2024-04-25T09:54:47.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.553+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== osd.7 v2:172.21.15.188:6824/678316061 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4c6c014ec0 con 0x7f4c54074ca0 2024-04-25T09:54:47.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.553+0000 7f97c8d0d640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f97a4074fd0 0x7f97a4077410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.553+0000 7f97c3fff640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97a406b8d0 0x7f97a406dd90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009bd0 tx=0x7f97b402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.553+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa4d0073a00 con 0x7fa4d0161240 2024-04-25T09:54:47.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.554+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== osd.1 v2:172.21.15.148:6800/3547190582 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f97a4077ae0 con 0x7f97a4074fd0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.554+0000 7f72c659c640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72c806d2d0 0x7f72c80afb40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c480 tx=0x7f72bc00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.554+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc017070 con 0x7f72c806d2d0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.554+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc005ce0 con 0x7f72c806d2d0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.554+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0072f0 con 0x7f72c806d2d0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.555+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c81f82f0 con 0x7f72c806d2d0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.555+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c81f88b0 con 0x7f72c806d2d0 2024-04-25T09:54:47.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.555+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4c8003970 con 0x7fa4d0161240 2024-04-25T09:54:47.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.555+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff31c092a10 con 0x7ff3140e59e0 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.555+0000 7fa4ccff9640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa4b006b500 0x7fa4b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.555+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa4c808a5a0 con 0x7fa4d0161240 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7fa4ccff9640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7fa4b0074c80 0x7fa4b00770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4b0077790 con 0x7fa4b0074c80 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c808a990 con 0x7fa4d0161240 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7fa4d4ebf640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7fa4b0074c80 0x7fa4b00770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7fa4cffff640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa4b006b500 0x7fa4b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7ff318ff9640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff30006b5a0 0x7ff30006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7ff31bfff640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff30006b5a0 0x7ff30006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff31c104c30 con 0x7ff3140e59e0 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7ff318ff9640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7ff300074d20 0x7ff300077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.557+0000 7ff31b7fe640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7ff300074d20 0x7ff300077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.557+0000 7ff31b7fe640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7ff300074d20 0x7ff300077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.557+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 --> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff31c08cec0 con 0x7ff300074d20 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.558+0000 7ff31bfff640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff30006b5a0 0x7ff30006da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff310000f60 tx=0x7ff310009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.558+0000 7fa4cffff640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa4b006b500 0x7fa4b006d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00015e0 tx=0x7fa4c00078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.556+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72c8073a00 con 0x7f72c806d2d0 2024-04-25T09:54:47.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.558+0000 7fa4d4ebf640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7fa4b0074c80 0x7fa4b00770c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.558+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72bc01a070 con 0x7f72c806d2d0 2024-04-25T09:54:47.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.558+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff31c107290 con 0x7ff3140e59e0 2024-04-25T09:54:47.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== osd.2 v2:172.21.15.148:6808/1311261585 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff31c08cec0 con 0x7ff300074d20 2024-04-25T09:54:47.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7f72b6ffd640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72a806b5a0 0x7f72a806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f72bc089a30 con 0x7f72c806d2d0 2024-04-25T09:54:47.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7f72c5d9b640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72a806b5a0 0x7f72a806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7f72b6ffd640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7f72a8074d20 0x7f72a8077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72bc01eec0 con 0x7f72a8074d20 2024-04-25T09:54:47.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72bc089e20 con 0x7f72c806d2d0 2024-04-25T09:54:47.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.559+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== osd.6 v2:172.21.15.188:6816/1362851547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa4b0077790 con 0x7fa4b0074c80 2024-04-25T09:54:47.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.560+0000 7f72c5d9b640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72a806b5a0 0x7f72a806da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72c8155ca0 tx=0x7f72c0003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.560+0000 7f72c6d9d640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7f72a8074d20 0x7f72a8077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.561+0000 7f72c6d9d640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7f72a8074d20 0x7f72a8077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.562+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== osd.0 v2:172.21.15.148:6816/2808668116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f72bc01eec0 con 0x7f72a8074d20 2024-04-25T09:54:47.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.578+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 --> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8940cfb40 con 0x7ff878074ca0 2024-04-25T09:54:47.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.578+0000 7ff88affd640 1 -- 172.21.15.148:0/1148440159 <== osd.5 v2:172.21.15.188:6808/3945194105 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8940cfb40 con 0x7ff878074ca0 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.579+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7ff878074ca0 msgr2=0x7ff8780770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.579+0000 7ff888ff9640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7ff878074ca0 0x7ff8780770e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.579+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff87806b520 msgr2=0x7ff87806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.579+0000 7ff888ff9640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff87806b520 0x7ff87806d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff8941fb1f0 tx=0x7ff88c005b70 comp rx=0 tx=0).stop 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.579+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff89406bbf0 msgr2=0x7ff89414dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.580+0000 7ff888ff9640 1 --2- 172.21.15.148:0/1148440159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff89406bbf0 0x7ff89414dc00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098c0 tx=0x7ff8900046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.580+0000 7ff89a8bf640 1 -- 172.21.15.148:0/1148440159 reap_dead start 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.580+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 shutdown_connections 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.580+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 >> 172.21.15.148:0/1148440159 conn(0x7ff894071ad0 msgr2=0x7ff89415c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.581+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 shutdown_connections 2024-04-25T09:54:47.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.581+0000 7ff888ff9640 1 -- 172.21.15.148:0/1148440159 wait complete. 2024-04-25T09:54:47.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.584+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 --> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4bb415e2e0 con 0x7f4b98074d20 2024-04-25T09:54:47.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.584+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 --> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97c40d1520 con 0x7f97a4074fd0 2024-04-25T09:54:47.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.584+0000 7f4c76c8c640 1 -- 172.21.15.148:0/232298327 --> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c700ce7b0 con 0x7f4c54074ca0 2024-04-25T09:54:47.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.584+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 --> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b680542e0 con 0x7f0b68054b10 2024-04-25T09:54:47.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f97c0ff9640 1 -- 172.21.15.148:0/456914199 <== osd.1 v2:172.21.15.148:6800/3547190582 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f97c40d1520 con 0x7f97a4074fd0 2024-04-25T09:54:47.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4baa7fc640 1 -- 172.21.15.148:0/1737374683 <== osd.4 v2:172.21.15.188:6800/3303846211 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4bb415e2e0 con 0x7f4b98074d20 2024-04-25T09:54:47.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4c667fc640 1 -- 172.21.15.148:0/232298327 <== osd.7 v2:172.21.15.188:6824/678316061 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c700ce7b0 con 0x7f4c54074ca0 2024-04-25T09:54:47.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f0b5effd640 1 -- 172.21.15.148:0/646121934 <== osd.3 v2:172.21.15.148:6819/1795140808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b680542e0 con 0x7f0b68054b10 2024-04-25T09:54:47.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f4b98074d20 msgr2=0x7f4b98077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bbc04d640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f4b98074d20 0x7f4b98077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f97a4074fd0 msgr2=0x7f97a4077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f97c950e640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f97a4074fd0 0x7f97a4077410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b5a0 msgr2=0x7f4b9806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bbc04d640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b5a0 0x7f4b9806da60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0034000 tx=0x7f4bb002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:47.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb408a570 msgr2=0x7f4bb415db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bbc04d640 1 --2- 172.21.15.148:0/1737374683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb408a570 0x7f4bb415db50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00eb70 tx=0x7f4bac00c490 comp rx=0 tx=0).stop 2024-04-25T09:54:47.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f4bba5ea640 1 -- 172.21.15.148:0/1737374683 reap_dead start 2024-04-25T09:54:47.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 shutdown_connections 2024-04-25T09:54:47.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 >> 172.21.15.148:0/1737374683 conn(0x7f4bb406abe0 msgr2=0x7f4bb40ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97a406b8d0 msgr2=0x7f97a406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.585+0000 7f97c950e640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97a406b8d0 0x7f97a406dd90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009bd0 tx=0x7f97b402f000 comp rx=0 tx=0).stop 2024-04-25T09:54:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c40b95c0 msgr2=0x7f97c4160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f97c950e640 1 --2- 172.21.15.148:0/456914199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c40b95c0 0x7f97c4160210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c430 tx=0x7f97ac00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f97c8d0d640 1 -- 172.21.15.148:0/456914199 reap_dead start 2024-04-25T09:54:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 shutdown_connections 2024-04-25T09:54:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 >> 172.21.15.148:0/456914199 conn(0x7f97c40d03d0 msgr2=0x7f97c40b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f0b68054b10 msgr2=0x7f0b680e5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f0b74d61640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f0b68054b10 0x7f0b680e5080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 shutdown_connections 2024-04-25T09:54:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4bbc04d640 1 -- 172.21.15.148:0/1737374683 wait complete. 2024-04-25T09:54:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f4c54074ca0 msgr2=0x7f4c540770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c53fff640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f4c54074ca0 0x7f4c540770e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c5406b5a0 msgr2=0x7f4c5406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0b5006b5a0 msgr2=0x7f0b5006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f0b74d61640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0b5006b5a0 0x7f0b5006da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0b68053680 tx=0x7f0b64009060 comp rx=0 tx=0).stop 2024-04-25T09:54:47.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c53fff640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c5406b5a0 0x7f4c5406da60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c60034000 tx=0x7f4c6002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b680cc6e0 msgr2=0x7f0b6804c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c70061600 msgr2=0x7f4c70200940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f0b74d61640 1 --2- 172.21.15.148:0/646121934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b680cc6e0 0x7f0b6804c040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b70068d50 tx=0x7f0b70068960 comp rx=0 tx=0).stop 2024-04-25T09:54:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c53fff640 1 --2- 172.21.15.148:0/232298327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c70061600 0x7f4c70200940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c430 tx=0x7f4c6c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.586+0000 7f4c7648b640 1 -- 172.21.15.148:0/232298327 reap_dead start 2024-04-25T09:54:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f0b6ed9d640 1 -- 172.21.15.148:0/646121934 reap_dead start 2024-04-25T09:54:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 shutdown_connections 2024-04-25T09:54:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 shutdown_connections 2024-04-25T09:54:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 >> 172.21.15.148:0/232298327 conn(0x7f4c70071880 msgr2=0x7f4c70074060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 >> 172.21.15.148:0/646121934 conn(0x7f0b68020000 msgr2=0x7f0b6800c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 shutdown_connections 2024-04-25T09:54:47.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f0b74d61640 1 -- 172.21.15.148:0/646121934 wait complete. 2024-04-25T09:54:47.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 shutdown_connections 2024-04-25T09:54:47.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.587+0000 7f4c53fff640 1 -- 172.21.15.148:0/232298327 wait complete. 2024-04-25T09:54:47.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.588+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 shutdown_connections 2024-04-25T09:54:47.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.589+0000 7f97c950e640 1 -- 172.21.15.148:0/456914199 wait complete. 2024-04-25T09:54:47.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.589+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4d0073a00 con 0x7fa4b0074c80 2024-04-25T09:54:47.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.589+0000 7fa4ccff9640 1 -- 172.21.15.148:0/2913191116 <== osd.6 v2:172.21.15.188:6816/1362851547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa4d0073a00 con 0x7fa4b0074c80 2024-04-25T09:54:47.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.591+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7fa4b0074c80 msgr2=0x7fa4b00770c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.591+0000 7fa4d6922640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7fa4b0074c80 0x7fa4b00770c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.591+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa4b006b500 msgr2=0x7fa4b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.591+0000 7fa4d6922640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa4b006b500 0x7fa4b006d9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa4c00015e0 tx=0x7fa4c00078f0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.591+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4d0161240 msgr2=0x7fa4d014eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.592+0000 7fa4d6922640 1 --2- 172.21.15.148:0/2913191116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4d0161240 0x7fa4d014eb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800ba50 tx=0x7fa4c800bf10 comp rx=0 tx=0).stop 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.592+0000 7fa4d4ebf640 1 -- 172.21.15.148:0/2913191116 reap_dead start 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.592+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 shutdown_connections 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.592+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 >> 172.21.15.148:0/2913191116 conn(0x7fa4d00d04a0 msgr2=0x7fa4d015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.592+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 --> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f72c81fb280 con 0x7f72a8074d20 2024-04-25T09:54:47.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.592+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 shutdown_connections 2024-04-25T09:54:47.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.593+0000 7fa4d6922640 1 -- 172.21.15.148:0/2913191116 wait complete. 2024-04-25T09:54:47.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.593+0000 7f72b6ffd640 1 -- 172.21.15.148:0/2859378607 <== osd.0 v2:172.21.15.148:6816/2808668116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f72c81fb280 con 0x7f72a8074d20 2024-04-25T09:54:47.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.594+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 --> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff314051320 con 0x7ff300074d20 2024-04-25T09:54:47.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.594+0000 7ff318ff9640 1 -- 172.21.15.148:0/3897803446 <== osd.2 v2:172.21.15.148:6808/1311261585 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff314051320 con 0x7ff300074d20 2024-04-25T09:54:47.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.594+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7f72a8074d20 msgr2=0x7f72a8077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.594+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7f72a8074d20 0x7f72a8077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72a806b5a0 msgr2=0x7f72a806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72a806b5a0 0x7f72a806da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72c8155ca0 tx=0x7f72c0003990 comp rx=0 tx=0).stop 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72c806d2d0 msgr2=0x7f72c80afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 --2- 172.21.15.148:0/2859378607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72c806d2d0 0x7f72c80afb40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c480 tx=0x7f72bc00c940 comp rx=0 tx=0).stop 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72c6d9d640 1 -- 172.21.15.148:0/2859378607 reap_dead start 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 shutdown_connections 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 >> 172.21.15.148:0/2859378607 conn(0x7f72c806abe0 msgr2=0x7f72c80cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 shutdown_connections 2024-04-25T09:54:47.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7ff300074d20 msgr2=0x7ff300077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.595+0000 7ff32279e640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7ff300074d20 0x7ff300077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7f72cc9b0640 1 -- 172.21.15.148:0/2859378607 wait complete. 2024-04-25T09:54:47.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff30006b5a0 msgr2=0x7ff30006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff30006b5a0 0x7ff30006da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff310000f60 tx=0x7ff310009040 comp rx=0 tx=0).stop 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3140e59e0 msgr2=0x7ff31404e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 --2- 172.21.15.148:0/3897803446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3140e59e0 0x7ff31404e7b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff31c06ab40 tx=0x7ff31c0686c0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff320d3b640 1 -- 172.21.15.148:0/3897803446 reap_dead start 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 shutdown_connections 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 >> 172.21.15.148:0/3897803446 conn(0x7ff3140537e0 msgr2=0x7ff314009e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.596+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 shutdown_connections 2024-04-25T09:54:47.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.597+0000 7ff32279e640 1 -- 172.21.15.148:0/3897803446 wait complete. 2024-04-25T09:54:47.672 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-04-25T09:54:47.672 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:54:47.673 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:47.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:54:47.674 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:47.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:54:47.675 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-04-25T09:54:47.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:54:47.676 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:47.676 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:54:47.677 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:47.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:54:47.679 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:47.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:54:47.680 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:47.680 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:54:47.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.876+0000 7fb909917640 1 Processor -- start 2024-04-25T09:54:47.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.876+0000 7fb909917640 1 -- start start 2024-04-25T09:54:47.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7faea384c640 1 Processor -- start 2024-04-25T09:54:47.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7faea384c640 1 -- start start 2024-04-25T09:54:47.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7f2ea4d73640 1 Processor -- start 2024-04-25T09:54:47.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7f2ea4d73640 1 -- start start 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7fb909917640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb904159b20 0x7fb9040b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7fb909917640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 0x7fb90406e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7fb909917640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb904159f00 con 0x7fb90406b700 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7fb909917640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040b3460 con 0x7fb904159b20 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.877+0000 7fb909917640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b2850 con 0x7fb90406dbc0 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 0x7fb90406e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 0x7fb90406e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56342/0 (socket says 172.21.15.148:56342) 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 -- 172.21.15.148:0/2779494761 learned_addr learned my addr 172.21.15.148:0/2779494761 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 -- 172.21.15.148:0/2779494761 >> v2:172.21.15.188:3300/0 conn(0x7fb904159b20 msgr2=0x7fb9040b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 --2- 172.21.15.148:0/2779494761 >> v2:172.21.15.188:3300/0 conn(0x7fb904159b20 0x7fb9040b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 -- 172.21.15.148:0/2779494761 >> v1:172.21.15.188:6789/0 conn(0x7fb90406b700 legacy=0x7fb904070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb901ffb640 1 -- 172.21.15.148:0/2779494761 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895719908 0 0) 0x7fb904159f00 con 0x7fb90406b700 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2ea4d73640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ea006b950 0x7f2ea00b2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2ea4d73640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea0165950 con 0x7f2ea006abe0 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2ea4d73640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea0165c30 con 0x7f2ea0161cb0 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2ea4d73640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea006f230 con 0x7f2ea006b950 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2e9effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ea006b950 0x7f2ea00b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2e9effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ea006b950 0x7f2ea00b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33522/0 (socket says 172.21.15.148:33522) 2024-04-25T09:54:47.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2e9effd640 1 -- 172.21.15.148:0/34769377 learned_addr learned my addr 172.21.15.148:0/34769377 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7fb9027fc640 1 -- 172.21.15.148:0/2779494761 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9040b2b30 con 0x7fb90406dbc0 2024-04-25T09:54:47.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7faea384c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae94054970 0x7fae94050290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7faea384c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae94039480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.878+0000 7faea15e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae94054970 0x7fae94050290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea384c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae94050d40 con 0x7fae94008de0 2024-04-25T09:54:47.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea384c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae94050fe0 con 0x7fae94054970 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea384c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940085e0 con 0x7fae94009270 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea0de7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fae94008de0 0x7fae94038d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48738/0 (socket says 172.21.15.148:48738) 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea0de7640 1 -- 172.21.15.148:0/2496207783 learned_addr learned my addr 172.21.15.148:0/2496207783 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea15e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae94054970 0x7fae94050290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33518/0 (socket says 172.21.15.148:33518) 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea15e8640 1 -- 172.21.15.148:0/2496207783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 msgr2=0x7fae94039480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea15e8640 1 --2- 172.21.15.148:0/2496207783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae94039480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea15e8640 1 -- 172.21.15.148:0/2496207783 >> v1:172.21.15.148:6789/0 conn(0x7fae94008de0 legacy=0x7fae94038d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7fae93fff640 1 -- 172.21.15.148:0/2496207783 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694231922 0 0) 0x7fae94050d40 con 0x7fae94008de0 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7faea15e8640 1 -- 172.21.15.148:0/2496207783 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940088c0 con 0x7fae94054970 2024-04-25T09:54:47.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f198df1d640 1 Processor -- start 2024-04-25T09:54:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2e9effd640 1 -- 172.21.15.148:0/34769377 >> v1:172.21.15.188:6789/0 conn(0x7f2ea0161cb0 legacy=0x7f2ea006eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2e9effd640 1 -- 172.21.15.148:0/34769377 >> v1:172.21.15.148:6789/0 conn(0x7f2ea006abe0 legacy=0x7f2ea0162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f4caff7e640 1 Processor -- start 2024-04-25T09:54:47.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.879+0000 7f2e9effd640 1 -- 172.21.15.148:0/34769377 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea0165f10 con 0x7f2ea006b950 2024-04-25T09:54:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7f8487da9640 1 Processor -- start 2024-04-25T09:54:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7f4caff7e640 1 -- start start 2024-04-25T09:54:47.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7fb9027fc640 1 --2- 172.21.15.148:0/2779494761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 0x7fb90406e020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009e50 tx=0x7fb8fc02a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d4b7ab277c52cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7f8487da9640 1 -- start start 2024-04-25T09:54:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7fb901ffb640 1 -- 172.21.15.148:0/2779494761 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc036070 con 0x7fb90406dbc0 2024-04-25T09:54:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7fb901ffb640 1 -- 172.21.15.148:0/2779494761 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8fc031600 con 0x7fb90406dbc0 2024-04-25T09:54:47.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7fb901ffb640 1 -- 172.21.15.148:0/2779494761 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc031a10 con 0x7fb90406dbc0 2024-04-25T09:54:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7f198df1d640 1 -- start start 2024-04-25T09:54:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.880+0000 7ffbd5d55640 1 Processor -- start 2024-04-25T09:54:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7ffbd5d55640 1 -- start start 2024-04-25T09:54:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7faea15e8640 1 --2- 172.21.15.148:0/2496207783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 0x7fae94050290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae9c02d370 tx=0x7fae9c0a50b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a29815b2d65501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fae93fff640 1 -- 172.21.15.148:0/2496207783 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b2070 con 0x7fae94054970 2024-04-25T09:54:47.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7f8487da9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84800cf640 0x7f848006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7f8487da9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84800ac8b0 0x7f84800accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7f2e9effd640 1 --2- 172.21.15.148:0/34769377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea006b950 0x7f2ea00b2860 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009870 tx=0x7f2e9402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442aa6188753d91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fb909917640 1 -- 172.21.15.148:0/2779494761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 msgr2=0x7fb90406e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fb909917640 1 --2- 172.21.15.148:0/2779494761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 0x7fb90406e020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009e50 tx=0x7fb8fc02a730 comp rx=0 tx=0).stop 2024-04-25T09:54:47.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7f4caff7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ca8157700 0x7f4ca815e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7f198df1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 0x7f1980053650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fb909917640 1 -- 172.21.15.148:0/2779494761 shutdown_connections 2024-04-25T09:54:47.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fb909917640 1 --2- 172.21.15.148:0/2779494761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90406dbc0 0x7fb90406e020 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fb909917640 1 --2- 172.21.15.148:0/2779494761 >> v2:172.21.15.188:3300/0 conn(0x7fb904159b20 0x7fb9040b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.881+0000 7fb909917640 1 -- 172.21.15.148:0/2779494761 >> 172.21.15.148:0/2779494761 conn(0x7fb90406abe0 msgr2=0x7fb90406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f19877fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f19800eabb0 0x7f198004f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43154/0 (socket says 172.21.15.148:43154) 2024-04-25T09:54:47.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485b45640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f848015b710 0x7f848015e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48762/0 (socket says 172.21.15.148:48762) 2024-04-25T09:54:47.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485b45640 1 -- 172.21.15.148:0/4114857321 learned_addr learned my addr 172.21.15.148:0/4114857321 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f2e9e7fc640 1 -- 172.21.15.148:0/34769377 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94037070 con 0x7f2ea006b950 2024-04-25T09:54:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f2e9e7fc640 1 -- 172.21.15.148:0/34769377 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94032440 con 0x7f2ea006b950 2024-04-25T09:54:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f2e9e7fc640 1 -- 172.21.15.148:0/34769377 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94032810 con 0x7f2ea006b950 2024-04-25T09:54:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485344640 1 --2- 172.21.15.148:0/4114857321 >> v2:172.21.15.148:3300/0 conn(0x7f84800cf640 0x7f848006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f198df1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f198003a370 0x7f1980053b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f19877fe640 1 -- 172.21.15.148:0/311350602 learned_addr learned my addr 172.21.15.148:0/311350602 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7fae93fff640 1 -- 172.21.15.148:0/2496207783 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0ad440 con 0x7fae94054970 2024-04-25T09:54:47.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800cfad0 con 0x7f848015b710 2024-04-25T09:54:47.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848006f190 con 0x7f84800cf640 2024-04-25T09:54:47.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848015e490 con 0x7f84800ac8b0 2024-04-25T09:54:47.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7fae93fff640 1 -- 172.21.15.148:0/2496207783 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b2070 con 0x7fae94054970 2024-04-25T09:54:47.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f4cadd1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ca8157700 0x7f4ca815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f1986ffd640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 0x7f1980053650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7ffbd5d55640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbc80e7700 0x7ffbc804ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8486346640 1 --2- 172.21.15.148:0/4114857321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84800ac8b0 0x7f84800accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f115edd5640 1 Processor -- start 2024-04-25T09:54:47.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f4cadd1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ca8157700 0x7f4ca815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57414/0 (socket says 172.21.15.148:57414) 2024-04-25T09:54:47.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1980034430 con 0x7f19800eabb0 2024-04-25T09:54:47.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198003aff0 con 0x7f198003a370 2024-04-25T09:54:47.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f4cad519640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4ca80cd2e0 0x7f4ca80ae830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48778/0 (socket says 172.21.15.148:48778) 2024-04-25T09:54:47.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f4cadd1a640 1 -- 172.21.15.148:0/3174044472 learned_addr learned my addr 172.21.15.148:0/3174044472 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7ffbcffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffbc80589b0 0x7ffbc8053520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48794/0 (socket says 172.21.15.148:48794) 2024-04-25T09:54:47.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1980034710 con 0x7f198004fe40 2024-04-25T09:54:47.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955643050 0 0) 0x7f1980034430 con 0x7f19800eabb0 2024-04-25T09:54:47.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1978003520 con 0x7f19800eabb0 2024-04-25T09:54:47.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8484b43640 1 -- 172.21.15.148:0/4114857321 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851157163 0 0) 0x7f84800cfad0 con 0x7f848015b710 2024-04-25T09:54:47.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7ffbd4d53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbc80e7700 0x7ffbc804ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8484b43640 1 -- 172.21.15.148:0/4114857321 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f846c003520 con 0x7f848015b710 2024-04-25T09:54:47.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7fb909917640 1 -- 172.21.15.148:0/2779494761 shutdown_connections 2024-04-25T09:54:47.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f198ccbb640 1 --2- 172.21.15.148:0/311350602 >> v2:172.21.15.148:3300/0 conn(0x7f198003a370 0x7f1980053b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485344640 1 -- 172.21.15.148:0/4114857321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84800ac8b0 msgr2=0x7f84800accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485344640 1 --2- 172.21.15.148:0/4114857321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84800ac8b0 0x7f84800accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485344640 1 -- 172.21.15.148:0/4114857321 >> v1:172.21.15.148:6789/0 conn(0x7f848015b710 legacy=0x7f848015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7ffbd4d53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbc80e7700 0x7ffbc804ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33552/0 (socket says 172.21.15.148:33552) 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7ffbd5554640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ffbc8053c30 0x7ffbc8007010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43162/0 (socket says 172.21.15.148:43162) 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7ffbcffff640 1 -- 172.21.15.148:0/1757073174 learned_addr learned my addr 172.21.15.148:0/1757073174 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc8057d80 con 0x7ffbc80589b0 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc8006250 con 0x7ffbc8053c30 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc8006530 con 0x7ffbc80e7700 2024-04-25T09:54:47.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f8485344640 1 -- 172.21.15.148:0/4114857321 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800cfd80 con 0x7f84800cf640 2024-04-25T09:54:47.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7faea384c640 1 -- 172.21.15.148:0/2496207783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 msgr2=0x7fae94050290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7faea384c640 1 --2- 172.21.15.148:0/2496207783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 0x7fae94050290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fae9c02d370 tx=0x7fae9c0a50b0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 -- 172.21.15.148:0/34769377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea006b950 msgr2=0x7f2ea00b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718617428 0 0) 0x7f1978003520 con 0x7f19800eabb0 2024-04-25T09:54:47.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 --2- 172.21.15.148:0/34769377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea006b950 0x7f2ea00b2860 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009870 tx=0x7f2e9402a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca8071c50 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca8071f30 con 0x7f4ca80aef40 2024-04-25T09:54:47.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca815e7d0 con 0x7f4ca8157700 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 -- 172.21.15.148:0/34769377 shutdown_connections 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 --2- 172.21.15.148:0/34769377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea006b950 0x7f2ea00b2860 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 -- 172.21.15.148:0/34769377 >> 172.21.15.148:0/34769377 conn(0x7f2ea00b1560 msgr2=0x7f2ea00b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 msgr2=0x7f1980053650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 0x7f1980053650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 >> v2:172.21.15.148:3300/0 conn(0x7f198003a370 msgr2=0x7f1980053b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 --2- 172.21.15.148:0/311350602 >> v2:172.21.15.148:3300/0 conn(0x7f198003a370 0x7f1980053b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980052e50 con 0x7f19800eabb0 2024-04-25T09:54:47.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f19880627f0 con 0x7f19800eabb0 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7f115edd5640 1 -- start start 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f115edd5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f115806ea90 0x7f1158071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f115edd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f115806d2d0 0x7f115806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f115edd5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1158162ba0 con 0x7f1158161fc0 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f115edd5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115806dc80 con 0x7f115806ea90 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 >> v1:172.21.15.188:6789/0 conn(0x7f19800eabb0 legacy=0x7f198004f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19780039b0 0x7f1978003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f115edd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1158165be0 con 0x7f115806d2d0 2024-04-25T09:54:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 0x7f1978004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1978004d40 0x7f19780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980034710 con 0x7f19780039b0 2024-04-25T09:54:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197800b9c0 con 0x7f1978004d40 2024-04-25T09:54:47.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197800bb40 con 0x7f198004fe40 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f4cacd18640 1 -- 172.21.15.148:0/3174044472 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163100501 0 0) 0x7f4ca8071c50 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f4cacd18640 1 -- 172.21.15.148:0/3174044472 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c94005730 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 -- 172.21.15.148:0/34769377 shutdown_connections 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.883+0000 7f2ea4d73640 1 -- 172.21.15.148:0/34769377 wait complete. 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7faea384c640 1 -- 172.21.15.148:0/2496207783 shutdown_connections 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7faea384c640 1 --2- 172.21.15.148:0/2496207783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae94039480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7faea384c640 1 --2- 172.21.15.148:0/2496207783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 0x7fae94050290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7faea384c640 1 -- 172.21.15.148:0/2496207783 >> 172.21.15.148:0/2496207783 conn(0x7fae94055c40 msgr2=0x7fae94056050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/1757073174 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71819732 0 0) 0x7ffbc8057d80 con 0x7ffbc80589b0 2024-04-25T09:54:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/1757073174 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc0003520 con 0x7ffbc80589b0 2024-04-25T09:54:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f115cb71640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1158161fc0 0x7f11581623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43174/0 (socket says 172.21.15.148:43174) 2024-04-25T09:54:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f115cb71640 1 -- 172.21.15.148:0/1736584558 learned_addr learned my addr 172.21.15.148:0/1736584558 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:47.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f115d372640 1 --2- 172.21.15.148:0/1736584558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f115806d2d0 0x7f115806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f1986ffd640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 0x7f1978004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f19877fe640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19780039b0 0x7f1978003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f8485344640 1 --2- 172.21.15.148:0/4114857321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f848006ec50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84700097a0 tx=0x7f847002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dba747c61a92f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f1157fff640 1 --2- 172.21.15.148:0/1736584558 >> v2:172.21.15.148:3300/0 conn(0x7f115806ea90 0x7f1158071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.882+0000 7fb909917640 1 -- 172.21.15.148:0/2779494761 wait complete. 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7fb909917640 1 Processor -- start 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7fb909917640 1 -- start start 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7ffbd4d53640 1 -- 172.21.15.148:0/1757073174 >> v1:172.21.15.188:6789/0 conn(0x7ffbc8053c30 legacy=0x7ffbc8007010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7ffbd4d53640 1 -- 172.21.15.148:0/1757073174 >> v1:172.21.15.148:6789/0 conn(0x7ffbc80589b0 legacy=0x7ffbc8053520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7ffbd4d53640 1 -- 172.21.15.148:0/1757073174 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8006810 con 0x7ffbc80e7700 2024-04-25T09:54:47.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f115d372640 1 -- 172.21.15.148:0/1736584558 >> v2:172.21.15.148:3300/0 conn(0x7f115806ea90 msgr2=0x7f1158071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f115d372640 1 --2- 172.21.15.148:0/1736584558 >> v2:172.21.15.148:3300/0 conn(0x7f115806ea90 0x7f1158071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f115d372640 1 -- 172.21.15.148:0/1736584558 >> v1:172.21.15.188:6789/0 conn(0x7f1158161fc0 legacy=0x7f11581623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f115d372640 1 -- 172.21.15.148:0/1736584558 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158162e80 con 0x7f115806d2d0 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f19877fe640 1 -- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 msgr2=0x7f1978004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f19877fe640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198004fe40 0x7f1978004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f19877fe640 1 -- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1978004d40 msgr2=0x7f19780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:47.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f198ccbb640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1978004d40 0x7f19780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f8484b43640 1 -- 172.21.15.148:0/4114857321 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470037070 con 0x7f84800cf640 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f8484b43640 1 -- 172.21.15.148:0/4114857321 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470032440 con 0x7f84800cf640 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f8484b43640 1 -- 172.21.15.148:0/4114857321 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470032810 con 0x7f84800cf640 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f4cadd1a640 1 -- 172.21.15.148:0/3174044472 >> v1:172.21.15.188:6789/0 conn(0x7f4ca80aef40 legacy=0x7f4ca815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f4cadd1a640 1 -- 172.21.15.148:0/3174044472 >> v1:172.21.15.148:6789/0 conn(0x7f4ca80cd2e0 legacy=0x7f4ca80ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f4cadd1a640 1 -- 172.21.15.148:0/3174044472 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca815da90 con 0x7f4ca8157700 2024-04-25T09:54:47.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f4cadd1a640 1 --2- 172.21.15.148:0/3174044472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca8157700 0x7f4ca815e290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c3a0 tx=0x7f4ca402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9aae74fdfc83ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7ffbd4d53640 1 --2- 172.21.15.148:0/1757073174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc80e7700 0x7ffbc804ca20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0097a0 tx=0x7ffbbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f509f710f009b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f19877fe640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1978004d40 0x7f19780051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.884+0000 7f19877fe640 1 -- 172.21.15.148:0/311350602 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978003520 con 0x7f19780039b0 2024-04-25T09:54:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f4cacd18640 1 -- 172.21.15.148:0/3174044472 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4008df0 con 0x7f4ca8157700 2024-04-25T09:54:47.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f19877fe640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19780039b0 0x7f1978003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f198804f440 tx=0x7f1978005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60c23c1f5d1f7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f4cacd18640 1 -- 172.21.15.148:0/3174044472 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca402a9c0 con 0x7f4ca8157700 2024-04-25T09:54:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/1757073174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc037070 con 0x7ffbc80e7700 2024-04-25T09:54:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/1757073174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc032440 con 0x7ffbc80e7700 2024-04-25T09:54:47.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f4cacd18640 1 -- 172.21.15.148:0/3174044472 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca402ad90 con 0x7f4ca8157700 2024-04-25T09:54:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f115d372640 1 --2- 172.21.15.148:0/1736584558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f115806d2d0 0x7f115806d740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11500097a0 tx=0x7f115002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5ba259f69250f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19880ad020 con 0x7f19780039b0 2024-04-25T09:54:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198806aca0 con 0x7f19780039b0 2024-04-25T09:54:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f19867fc640 1 -- 172.21.15.148:0/311350602 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988073990 con 0x7f19780039b0 2024-04-25T09:54:47.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 msgr2=0x7f848006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f8487da9640 1 --2- 172.21.15.148:0/4114857321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f848006ec50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f84700097a0 tx=0x7f847002a180 comp rx=0 tx=0).stop 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb909917640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb90406b700 0x7fb9041423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb909917640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90406dbc0 0x7fb90415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb909917640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb904159b20 0x7fb904144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb909917640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b2470 con 0x7fb90406b700 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb909917640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b30f0 con 0x7fb90406dbc0 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb909917640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040b25f0 con 0x7fb904159b20 2024-04-25T09:54:47.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9027fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90406dbc0 0x7fb90415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f2ea4d73640 1 Processor -- start 2024-04-25T09:54:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.885+0000 7f2ea4d73640 1 -- start start 2024-04-25T09:54:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9037fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb904159b20 0x7fb904144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9027fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90406dbc0 0x7fb90415a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57426/0 (socket says 172.21.15.148:57426) 2024-04-25T09:54:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9027fc640 1 -- 172.21.15.148:0/2342329234 learned_addr learned my addr 172.21.15.148:0/2342329234 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb902ffd640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb90406b700 0x7fb9041423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f11577fe640 1 -- 172.21.15.148:0/1736584558 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150037070 con 0x7f115806d2d0 2024-04-25T09:54:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f11577fe640 1 -- 172.21.15.148:0/1736584558 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115002ac90 con 0x7f115806d2d0 2024-04-25T09:54:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f11577fe640 1 -- 172.21.15.148:0/1736584558 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150032660 con 0x7f115806d2d0 2024-04-25T09:54:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca8157700 msgr2=0x7f4ca815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f4caff7e640 1 --2- 172.21.15.148:0/3174044472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca8157700 0x7f4ca815e290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c3a0 tx=0x7f4ca402a140 comp rx=0 tx=0).stop 2024-04-25T09:54:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/1757073174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc037070 con 0x7ffbc80e7700 2024-04-25T09:54:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 shutdown_connections 2024-04-25T09:54:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f4caff7e640 1 --2- 172.21.15.148:0/3174044472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca8157700 0x7f4ca815e290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 >> 172.21.15.148:0/3174044472 conn(0x7f4ca806abe0 msgr2=0x7f4ca806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9037fe640 1 -- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90406dbc0 msgr2=0x7fb90415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9037fe640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90406dbc0 0x7fb90415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9037fe640 1 -- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb90406b700 msgr2=0x7fb9041423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9037fe640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb90406b700 0x7fb9041423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb9037fe640 1 -- 172.21.15.148:0/2342329234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9041428f0 con 0x7fb904159b20 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7fb902ffd640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb90406b700 0x7fb9041423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:47.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7faea384c640 1 -- 172.21.15.148:0/2496207783 shutdown_connections 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7faea384c640 1 -- 172.21.15.148:0/2496207783 wait complete. 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.886+0000 7f2ea4d73640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 0x7f2ea014cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2ea4d73640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea0161cb0 0x7f2ea015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2ea4d73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea0163670 0x7f2ea015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2ea4d73640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea006b010 con 0x7f2ea0161cb0 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2ea4d73640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea0162090 con 0x7f2ea006abe0 2024-04-25T09:54:47.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 0x7f2ea014cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19780039b0 msgr2=0x7f1978003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198df1d640 1 --2- 172.21.15.148:0/311350602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19780039b0 0x7f1978003dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f198804f440 tx=0x7f1978005830 comp rx=0 tx=0).stop 2024-04-25T09:54:47.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 shutdown_connections 2024-04-25T09:54:47.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7fb9037fe640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb904159b20 0x7fb904144db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c3e0 tx=0x7fb8f800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198ccbb640 1 -- 172.21.15.148:0/311350602 reap_dead start 2024-04-25T09:54:47.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f4caff7e640 1 -- 172.21.15.148:0/3174044472 wait complete. 2024-04-25T09:54:47.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 0x7f2ea014cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57428/0 (socket says 172.21.15.148:57428) 2024-04-25T09:54:47.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 -- 172.21.15.148:0/3194889570 learned_addr learned my addr 172.21.15.148:0/3194889570 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2ea4d73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea006b190 con 0x7f2ea0163670 2024-04-25T09:54:47.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 shutdown_connections 2024-04-25T09:54:47.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 >> 172.21.15.148:0/311350602 conn(0x7f19800550e0 msgr2=0x7f19800554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 shutdown_connections 2024-04-25T09:54:47.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f8487da9640 1 --2- 172.21.15.148:0/4114857321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84800ac8b0 0x7f84800accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 shutdown_connections 2024-04-25T09:54:47.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f8487da9640 1 --2- 172.21.15.148:0/4114857321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f848006ec50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc80e7700 msgr2=0x7ffbc804ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/1757073174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc80e7700 0x7ffbc804ca20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0097a0 tx=0x7ffbbc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f115edd5640 1 -- 172.21.15.148:0/1736584558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f115806d2d0 msgr2=0x7f115806d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f115edd5640 1 --2- 172.21.15.148:0/1736584558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f115806d2d0 0x7f115806d740 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f11500097a0 tx=0x7f115002a000 comp rx=0 tx=0).stop 2024-04-25T09:54:47.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 >> 172.21.15.148:0/4114857321 conn(0x7f848006abe0 msgr2=0x7f848006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f115edd5640 1 -- 172.21.15.148:0/1736584558 shutdown_connections 2024-04-25T09:54:47.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f115edd5640 1 --2- 172.21.15.148:0/1736584558 >> v2:172.21.15.148:3300/0 conn(0x7f115806ea90 0x7f1158071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f115edd5640 1 --2- 172.21.15.148:0/1736584558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f115806d2d0 0x7f115806d740 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f115edd5640 1 -- 172.21.15.148:0/1736584558 >> 172.21.15.148:0/1736584558 conn(0x7f115806abe0 msgr2=0x7f115806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f198df1d640 1 -- 172.21.15.148:0/311350602 wait complete. 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9effd640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea0161cb0 0x7f2ea015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9ffff640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea0163670 0x7f2ea015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 -- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea0163670 msgr2=0x7f2ea015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea0163670 0x7f2ea015e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 -- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea0161cb0 msgr2=0x7f2ea015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea0161cb0 0x7f2ea015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7f2e9f7fe640 1 -- 172.21.15.148:0/3194889570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea01625d0 con 0x7f2ea006abe0 2024-04-25T09:54:47.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 shutdown_connections 2024-04-25T09:54:47.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/1757073174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc80e7700 0x7ffbc804ca20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 >> 172.21.15.148:0/1757073174 conn(0x7ffbc8058ed0 msgr2=0x7ffbc80592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:47.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8017070 con 0x7fb904159b20 2024-04-25T09:54:47.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 shutdown_connections 2024-04-25T09:54:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7f115edd5640 1 -- 172.21.15.148:0/1736584558 shutdown_connections 2024-04-25T09:54:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f80035f0 con 0x7fb904159b20 2024-04-25T09:54:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.887+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80039e0 con 0x7fb904159b20 2024-04-25T09:54:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb904168360 con 0x7fb904159b20 2024-04-25T09:54:47.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7f2e9f7fe640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 0x7f2ea014cb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b3f0 tx=0x7f2e9000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 shutdown_connections 2024-04-25T09:54:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7f8487da9640 1 -- 172.21.15.148:0/4114857321 wait complete. 2024-04-25T09:54:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f4caff7e640 1 Processor -- start 2024-04-25T09:54:47.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb904166a60 con 0x7fb904159b20 2024-04-25T09:54:47.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7faea384c640 1 Processor -- start 2024-04-25T09:54:47.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7f115edd5640 1 -- 172.21.15.148:0/1736584558 wait complete. 2024-04-25T09:54:47.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7faea384c640 1 -- start start 2024-04-25T09:54:47.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f198df1d640 1 Processor -- start 2024-04-25T09:54:47.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f4caff7e640 1 -- start start 2024-04-25T09:54:47.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f115edd5640 1 Processor -- start 2024-04-25T09:54:47.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90015020 con 0x7f2ea006abe0 2024-04-25T09:54:47.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90011830 con 0x7f2ea006abe0 2024-04-25T09:54:47.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90011c20 con 0x7f2ea006abe0 2024-04-25T09:54:47.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f2ea4d73640 1 -- 172.21.15.148:0/3194889570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea01628b0 con 0x7f2ea006abe0 2024-04-25T09:54:47.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f2ea4d73640 1 -- 172.21.15.148:0/3194889570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea0162e70 con 0x7f2ea006abe0 2024-04-25T09:54:47.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.888+0000 7ffbd5d55640 1 -- 172.21.15.148:0/1757073174 wait complete. 2024-04-25T09:54:47.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7ffbd5d55640 1 Processor -- start 2024-04-25T09:54:47.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7faea384c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 0x7fae940d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7f115edd5640 1 -- start start 2024-04-25T09:54:47.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.889+0000 7f198df1d640 1 -- start start 2024-04-25T09:54:47.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7f198df1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 0x7f19800d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea384c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae940e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7f4caff7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ca80aef40 0x7f4ca81f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea15e8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 0x7fae940d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea15e8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 0x7fae940d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57438/0 (socket says 172.21.15.148:57438) 2024-04-25T09:54:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea15e8640 1 -- 172.21.15.148:0/3925183587 learned_addr learned my addr 172.21.15.148:0/3925183587 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea0de7640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae940e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7ffbd5d55640 1 -- start start 2024-04-25T09:54:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd5d55640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 0x7ffbc804bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd5d55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc80589b0 0x7ffbc8051600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 0x7fae940e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115edd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f115806d2d0 0x7f1158153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115edd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f115806ea90 0x7f115814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115edd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 0x7f11580aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115edd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580719c0 con 0x7f115806ea90 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115edd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158163440 con 0x7f115806d2d0 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115edd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1158071b40 con 0x7f1158161fc0 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4cadd1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ca80aef40 0x7f4ca81f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7f8487da9640 1 Processor -- start 2024-04-25T09:54:47.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7f8487da9640 1 -- start start 2024-04-25T09:54:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.890+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8f801a070 con 0x7fb904159b20 2024-04-25T09:54:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94053a80 con 0x7fae94054970 2024-04-25T09:54:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940507d0 con 0x7fae94008de0 2024-04-25T09:54:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4cadd1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ca80aef40 0x7f4ca81f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56440/0 (socket says 172.21.15.148:56440) 2024-04-25T09:54:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae94053c00 con 0x7fae94009270 2024-04-25T09:54:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4caff7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca80cd2e0 0x7f4ca8142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd5d55640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc8051b40 0x7ffbc80d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4cadd1a640 1 -- 172.21.15.148:0/2992072195 learned_addr learned my addr 172.21.15.148:0/2992072195 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7fb8ebfff640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f006b5a0 0x7fb8f006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb8f8089880 con 0x7fb904159b20 2024-04-25T09:54:47.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/2342329234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb904073990 con 0x7fb904159b20 2024-04-25T09:54:47.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115d372640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 0x7f11580aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115cb71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f115806d2d0 0x7f1158153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd4d53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 0x7ffbc804bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd4d53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 0x7ffbc804bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56452/0 (socket says 172.21.15.148:56452) 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbcffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc80589b0 0x7ffbc8051600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbcffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc80589b0 0x7ffbc8051600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57474/0 (socket says 172.21.15.148:57474) 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f8487da9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 0x7f84801572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f8487da9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f84800a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f8487da9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848015b710 0x7f84800a9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd5d55640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8057990 con 0x7ffbc8051b40 2024-04-25T09:54:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd5d55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc804cf60 con 0x7ffbc80589b0 2024-04-25T09:54:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd5d55640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8057b10 con 0x7ffbc8053c30 2024-04-25T09:54:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115d372640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 0x7f11580aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56448/0 (socket says 172.21.15.148:56448) 2024-04-25T09:54:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f115d372640 1 -- 172.21.15.148:0/1955189684 learned_addr learned my addr 172.21.15.148:0/1955189684 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7ffbd4d53640 1 -- 172.21.15.148:0/2109887808 learned_addr learned my addr 172.21.15.148:0/2109887808 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8487da9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848015baf0 con 0x7f84800cf640 2024-04-25T09:54:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8487da9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800d0340 con 0x7f84800ac8b0 2024-04-25T09:54:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8487da9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848015bc70 con 0x7f848015b710 2024-04-25T09:54:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198df1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19800d7300 0x7f19800d92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198df1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800d7800 0x7f1980045590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 0x7f19800d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 0x7f19800d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56418/0 (socket says 172.21.15.148:56418) 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 -- 172.21.15.148:0/775999767 learned_addr learned my addr 172.21.15.148:0/775999767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198003a8c0 con 0x7f19800d7300 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800eaf90 con 0x7f19800d7800 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4cad519640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca80cd2e0 0x7f4ca8142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4caff7e640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca8157700 0x7f4ca815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4caff7e640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80718e0 con 0x7f4ca8157700 2024-04-25T09:54:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4caff7e640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca815d840 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7fb902ffd640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f006b5a0 0x7fb8f006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198003aa40 con 0x7f19800d6f20 2024-04-25T09:54:47.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198ccbb640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800d7800 0x7f1980045590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f4caff7e640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca8157ae0 con 0x7f4ca80aef40 2024-04-25T09:54:47.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7faea15e8640 1 -- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 msgr2=0x7fae940e0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 -- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800d7800 msgr2=0x7f1980045590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800d7800 0x7f1980045590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 -- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19800d7300 msgr2=0x7f19800d92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19800d7300 0x7f19800d92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f19877fe640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19800d7cc0 con 0x7f19800d6f20 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.891+0000 7f198ccbb640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800d7800 0x7f1980045590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f19877fe640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 0x7f19800d8d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f198804fcd0 tx=0x7f1980034600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485b45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 0x7f84801572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7faea15e8640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae940e0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7faea15e8640 1 -- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 msgr2=0x7fae940e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7faea15e8640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 0x7fae940e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7faea15e8640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9404b770 con 0x7fae94008de0 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485b45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 0x7f84801572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57490/0 (socket says 172.21.15.148:57490) 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485b45640 1 -- 172.21.15.148:0/4136746644 learned_addr learned my addr 172.21.15.148:0/4136746644 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7ffbd4d53640 1 -- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc80589b0 msgr2=0x7ffbc8051600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7ffbd4d53640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc80589b0 0x7ffbc8051600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7ffbd4d53640 1 -- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc8051b40 msgr2=0x7ffbc80d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7ffbd4d53640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc8051b40 0x7ffbc80d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8486346640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848015b710 0x7f84800a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7ffbd4d53640 1 -- 172.21.15.148:0/2109887808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc80523b0 con 0x7ffbc8053c30 2024-04-25T09:54:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7faea15e8640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 0x7fae940d1f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae9c02d370 tx=0x7fae9c068960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f4cae51b640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca8157700 0x7f4ca815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f1157fff640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f115806ea90 0x7f115814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f4cad519640 1 -- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ca80aef40 msgr2=0x7f4ca81f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7fb902ffd640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f006b5a0 0x7fb8f006da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb904150990 tx=0x7fb8f4009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485b45640 1 -- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848015b710 msgr2=0x7f84800a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485b45640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848015b710 0x7f84800a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485344640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f84800a8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f115d372640 1 -- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f115806d2d0 msgr2=0x7f1158153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f115d372640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f115806d2d0 0x7f1158153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f115d372640 1 -- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f115806ea90 msgr2=0x7f115814b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f115d372640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f115806ea90 0x7f115814b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f115d372640 1 -- 172.21.15.148:0/1955189684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580a9bc0 con 0x7f1158161fc0 2024-04-25T09:54:47.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f8485b45640 1 -- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 msgr2=0x7f84800a8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f8485b45640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f84800a8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7ffbd4d53640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 0x7ffbc804bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc000c00 tx=0x7ffbbc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f8485b45640 1 -- 172.21.15.148:0/4136746644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84801577f0 con 0x7f84800ac8b0 2024-04-25T09:54:47.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f2ea4d73640 1 -- 172.21.15.148:0/3194889570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e80005290 con 0x7f2ea006abe0 2024-04-25T09:54:47.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0bd020 con 0x7fae94008de0 2024-04-25T09:54:47.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f4cad519640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ca80aef40 0x7f4ca81f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f4cad519640 1 -- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca8157700 msgr2=0x7f4ca815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:47.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f4cad519640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca8157700 0x7f4ca815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:47.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.892+0000 7f4cad519640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8142880 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4cae51b640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca8157700 0x7f4ca815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0acbe0 con 0x7fae94008de0 2024-04-25T09:54:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0b2070 con 0x7fae94008de0 2024-04-25T09:54:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f115d372640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 0x7f11580aac60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1150009890 tx=0x7f11500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f8485344640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f84800a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4cad519640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca80cd2e0 0x7f4ca8142340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c430 tx=0x7f4c9800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9404ba10 con 0x7fae94008de0 2024-04-25T09:54:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f8485b45640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 0x7f84801572b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c390 tx=0x7f847c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980074e0 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c980076a0 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e90018020 con 0x7f2ea006abe0 2024-04-25T09:54:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4caff7e640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca81f10b0 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4caff7e640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca81f1670 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98004a20 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9404bfd0 con 0x7fae94008de0 2024-04-25T09:54:47.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.893+0000 7f2e87fff640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b520 0x7f2e8c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.894+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c017070 con 0x7f84800ac8b0 2024-04-25T09:54:47.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.894+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800ab880 con 0x7f84800ac8b0 2024-04-25T09:54:47.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.894+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2e90089ff0 con 0x7f2ea006abe0 2024-04-25T09:54:47.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.894+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800abe40 con 0x7f84800ac8b0 2024-04-25T09:54:47.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.894+0000 7f2e9effd640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b520 0x7f2e8c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.894+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae940057d0 con 0x7fae94008de0 2024-04-25T09:54:47.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.895+0000 7f2e9effd640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b520 0x7f2e8c06d9e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e94034000 tx=0x7f2e9402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.895+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae9c0c0020 con 0x7fae94008de0 2024-04-25T09:54:47.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.895+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8073a00 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.895+0000 7fae91ffb640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae8006b520 0x7fae8006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.895+0000 7faea0de7640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae8006b520 0x7fae8006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.895+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8480073a00 con 0x7f84800ac8b0 2024-04-25T09:54:47.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.896+0000 7faea0de7640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae8006b520 0x7fae8006d9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae940dfa60 tx=0x7fae8c007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19880ad070 con 0x7f19800d6f20 2024-04-25T09:54:47.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc037050 con 0x7ffbc8053c30 2024-04-25T09:54:47.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc8052690 con 0x7ffbc8053c30 2024-04-25T09:54:47.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198806ae50 con 0x7f19800d6f20 2024-04-25T09:54:47.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988073990 con 0x7f19800d6f20 2024-04-25T09:54:47.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc8052c50 con 0x7ffbc8053c30 2024-04-25T09:54:47.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150040070 con 0x7f1158161fc0 2024-04-25T09:54:47.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19800d7fa0 con 0x7f19800d6f20 2024-04-25T09:54:47.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800d8560 con 0x7f19800d6f20 2024-04-25T09:54:47.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9041428f0 con 0x7fb904159b20 2024-04-25T09:54:47.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11580a9ea0 con 0x7f1158161fc0 2024-04-25T09:54:47.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11580aa460 con 0x7f1158161fc0 2024-04-25T09:54:47.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150038040 con 0x7f1158161fc0 2024-04-25T09:54:47.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc038040 con 0x7ffbc8053c30 2024-04-25T09:54:47.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.897+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0327f0 con 0x7ffbc8053c30 2024-04-25T09:54:47.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.898+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150037070 con 0x7f1158161fc0 2024-04-25T09:54:47.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.898+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1980008ff0 con 0x7f19800d6f20 2024-04-25T09:54:47.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.898+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc8005380 con 0x7ffbc8053c30 2024-04-25T09:54:47.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.898+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c005290 con 0x7f1158161fc0 2024-04-25T09:54:47.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.898+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e9001f030 con 0x7f2ea006abe0 2024-04-25T09:54:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c005ce0 con 0x7f84800ac8b0 2024-04-25T09:54:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c021060 con 0x7f84800ac8b0 2024-04-25T09:54:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbbc045020 con 0x7ffbc8053c30 2024-04-25T09:54:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae9c0f1ca0 con 0x7fae94008de0 2024-04-25T09:54:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19880b0020 con 0x7f19800d6f20 2024-04-25T09:54:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7ffbcd7fa640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbb406b4d0 0x7ffbb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.899+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffbbc0aab90 con 0x7ffbc8053c30 2024-04-25T09:54:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7ffbcffff640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbb406b4d0 0x7ffbb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f115002ab50 con 0x7f1158161fc0 2024-04-25T09:54:47.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f1973fff640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f196c06b5a0 0x7f196c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1988125880 con 0x7f19800d6f20 2024-04-25T09:54:47.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f1986ffd640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f196c06b5a0 0x7f196c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c9801a020 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f11557fa640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f113806b5a0 0x7f113806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7ffbcffff640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbb406b4d0 0x7ffbb406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffbc804f070 tx=0x7ffbc4007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f115cb71640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f113806b5a0 0x7f113806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.900+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f11500aaf60 con 0x7f1158161fc0 2024-04-25T09:54:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f1986ffd640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f196c06b5a0 0x7f196c06da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19800e7040 tx=0x7f197c0095d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f4c9e7fc640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c8c06b4d0 0x7f4c8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c9808a4e0 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f4cadd1a640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c8c06b4d0 0x7f4c8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f847c01a070 con 0x7f84800ac8b0 2024-04-25T09:54:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f115cb71640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f113806b5a0 0x7f113806da60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1140000f60 tx=0x7f1140007c40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f84767fc640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.901+0000 7f4cadd1a640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c8c06b4d0 0x7f4c8c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4000c00 tx=0x7f4ca400c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.902+0000 7f8485344640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.902+0000 7f8485344640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f846406b4d0 0x7f846406d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8470034000 tx=0x7f847002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.903+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11500d88c0 con 0x7f1158161fc0 2024-04-25T09:54:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.904+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae9c0c7020 con 0x7fae94008de0 2024-04-25T09:54:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.908+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbbc0797f0 con 0x7ffbc8053c30 2024-04-25T09:54:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.908+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f847c056670 con 0x7f84800ac8b0 2024-04-25T09:54:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.908+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c98021070 con 0x7f4ca80cd2e0 2024-04-25T09:54:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.912+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19880f43a0 con 0x7f19800d6f20 2024-04-25T09:54:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:47.913+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f847c0585d0 con 0x7f84800ac8b0 2024-04-25T09:54:48.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.305+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/2342329234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb9041509c0 con 0x7fb904159b20 2024-04-25T09:54:48.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.305+0000 7fb8ebfff640 1 -- 172.21.15.148:0/2342329234 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb8f8057d50 con 0x7fb904159b20 2024-04-25T09:54:48.308 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f006b5a0 msgr2=0x7fb8f006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f006b5a0 0x7fb8f006da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb904150990 tx=0x7fb8f4009060 comp rx=0 tx=0).stop 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb904159b20 msgr2=0x7fb904144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb904159b20 0x7fb904144db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c3e0 tx=0x7fb8f800c8a0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 shutdown_connections 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f006b5a0 0x7fb8f006da60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb904159b20 0x7fb904144db0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90406dbc0 0x7fb90415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 --2- 172.21.15.148:0/2342329234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb90406b700 0x7fb9041423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.309+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 >> 172.21.15.148:0/2342329234 conn(0x7fb90406abe0 msgr2=0x7fb9040b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.310+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 shutdown_connections 2024-04-25T09:54:48.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.310+0000 7fb909917640 1 -- 172.21.15.148:0/2342329234 wait complete. 2024-04-25T09:54:48.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.315+0000 7f2ea4d73640 1 -- 172.21.15.148:0/3194889570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2e80005850 con 0x7f2ea006abe0 2024-04-25T09:54:48.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.316+0000 7f2e87fff640 1 -- 172.21.15.148:0/3194889570 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e90058c10 con 0x7f2ea006abe0 2024-04-25T09:54:48.319 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-04-25T09:54:48.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.319+0000 7f2e85ffb640 1 -- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b520 msgr2=0x7f2e8c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b520 0x7f2e8c06d9e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2e94034000 tx=0x7f2e9402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:48.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 -- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 msgr2=0x7f2ea014cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 0x7f2ea014cb60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b3f0 tx=0x7f2e9000b8b0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 -- 172.21.15.148:0/3194889570 shutdown_connections 2024-04-25T09:54:48.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b520 0x7f2e8c06d9e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea0163670 0x7f2ea015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea0161cb0 0x7f2ea015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 --2- 172.21.15.148:0/3194889570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea006abe0 0x7f2ea014cb60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.320+0000 7f2e85ffb640 1 -- 172.21.15.148:0/3194889570 >> 172.21.15.148:0/3194889570 conn(0x7f2ea00b1560 msgr2=0x7f2ea015f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.321+0000 7f2e85ffb640 1 -- 172.21.15.148:0/3194889570 shutdown_connections 2024-04-25T09:54:48.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.321+0000 7f2e85ffb640 1 -- 172.21.15.148:0/3194889570 wait complete. 2024-04-25T09:54:48.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.323+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f113c005850 con 0x7f1158161fc0 2024-04-25T09:54:48.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.324+0000 7f11557fa640 1 -- 172.21.15.148:0/1955189684 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1150063030 con 0x7f1158161fc0 2024-04-25T09:54:48.327 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-04-25T09:54:48.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.324+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffbc8052f90 con 0x7ffbc8053c30 2024-04-25T09:54:48.330 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:48.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.325+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2109887808 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbbc0790f0 con 0x7ffbc8053c30 2024-04-25T09:54:48.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.327+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4ca80cf480 con 0x7f4ca80cd2e0 2024-04-25T09:54:48.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-25T09:54:48.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.327+0000 7f4c9e7fc640 1 -- 172.21.15.148:0/2992072195 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c98059000 con 0x7f4ca80cd2e0 2024-04-25T09:54:48.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f113806b5a0 msgr2=0x7f113806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f113806b5a0 0x7f113806da60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1140000f60 tx=0x7f1140007c40 comp rx=0 tx=0).stop 2024-04-25T09:54:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 msgr2=0x7f11580aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 0x7f11580aac60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1150009890 tx=0x7f11500046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.332 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-04-25T09:54:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 shutdown_connections 2024-04-25T09:54:48.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f113806b5a0 0x7f113806da60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbb406b4d0 msgr2=0x7ffbb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbb406b4d0 0x7ffbb406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffbc804f070 tx=0x7ffbc4007d30 comp rx=0 tx=0).stop 2024-04-25T09:54:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fae9404cfb0 con 0x7fae94008de0 2024-04-25T09:54:48.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 msgr2=0x7ffbc804bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 0x7ffbc804bfa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc000c00 tx=0x7ffbbc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1158161fc0 0x7f11580aac60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f115806ea90 0x7f115814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 --2- 172.21.15.148:0/1955189684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f115806d2d0 0x7f1158153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 >> 172.21.15.148:0/1955189684 conn(0x7f115806abe0 msgr2=0x7f1158164bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 shutdown_connections 2024-04-25T09:54:48.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.328+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f19800059b0 con 0x7f19800d6f20 2024-04-25T09:54:48.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbb406b4d0 0x7ffbb406d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbc8051b40 0x7ffbc80d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbc80589b0 0x7ffbc8051600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 --2- 172.21.15.148:0/2109887808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbc8053c30 0x7ffbc804bfa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 >> 172.21.15.148:0/2109887808 conn(0x7ffbc8058ed0 msgr2=0x7ffbc8056a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 shutdown_connections 2024-04-25T09:54:48.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7f115edd5640 1 -- 172.21.15.148:0/1955189684 wait complete. 2024-04-25T09:54:48.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7fae91ffb640 1 -- 172.21.15.148:0/3925183587 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae9c0f4300 con 0x7fae94008de0 2024-04-25T09:54:48.337 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-04-25T09:54:48.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7f1973fff640 1 -- 172.21.15.148:0/775999767 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f19880627f0 con 0x7f19800d6f20 2024-04-25T09:54:48.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 shutdown_connections 2024-04-25T09:54:48.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.329+0000 7ffbd5d55640 1 -- 172.21.15.148:0/2109887808 wait complete. 2024-04-25T09:54:48.339 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c8c06b4d0 msgr2=0x7f4c8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c8c06b4d0 0x7f4c8c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4000c00 tx=0x7f4ca400c040 comp rx=0 tx=0).stop 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca80cd2e0 msgr2=0x7f4ca8142340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca80cd2e0 0x7f4ca8142340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c430 tx=0x7f4c9800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 shutdown_connections 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c8c06b4d0 0x7f4c8c06d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca8157700 0x7f4ca815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ca80cd2e0 0x7f4ca8142340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 --2- 172.21.15.148:0/2992072195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ca80aef40 0x7f4ca81f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 >> 172.21.15.148:0/2992072195 conn(0x7f4ca806abe0 msgr2=0x7f4ca80b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 shutdown_connections 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.331+0000 7f4c8bfff640 1 -- 172.21.15.148:0/2992072195 wait complete. 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.332+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f84800cf400 con 0x7f84800ac8b0 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae8006b520 msgr2=0x7fae8006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae8006b520 0x7fae8006d9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae940dfa60 tx=0x7fae8c007d50 comp rx=0 tx=0).stop 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 msgr2=0x7fae940d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 0x7fae940d1f70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fae9c02d370 tx=0x7fae9c068960 comp rx=0 tx=0).stop 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.333+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f196c06b5a0 msgr2=0x7f196c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.333+0000 7f198df1d640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f196c06b5a0 0x7f196c06da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19800e7040 tx=0x7f197c0095d0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.333+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 msgr2=0x7f19800d8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.333+0000 7f198df1d640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 0x7f19800d8d60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f198804fcd0 tx=0x7f1980034600 comp rx=0 tx=0).stop 2024-04-25T09:54:48.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.333+0000 7f84767fc640 1 -- 172.21.15.148:0/4136746644 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f847c05c670 con 0x7f84800ac8b0 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 shutdown_connections 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae8006b520 0x7fae8006d9e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae94054970 0x7fae940e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 shutdown_connections 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f196c06b5a0 0x7f196c06da60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19800d7800 0x7f1980045590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19800d7300 0x7f19800d92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 --2- 172.21.15.148:0/775999767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19800d6f20 0x7f19800d8d60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 >> 172.21.15.148:0/775999767 conn(0x7f19800550e0 msgr2=0x7f19800e8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae94009270 0x7fae940e0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 --2- 172.21.15.148:0/3925183587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94008de0 0x7fae940d1f70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 >> 172.21.15.148:0/3925183587 conn(0x7fae94055c40 msgr2=0x7fae94056020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 shutdown_connections 2024-04-25T09:54:48.345 INFO:teuthology.orchestra.run.smithi148.stdout:55834574857 2024-04-25T09:54:48.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.334+0000 7f198df1d640 1 -- 172.21.15.148:0/775999767 wait complete. 2024-04-25T09:54:48.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.335+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 shutdown_connections 2024-04-25T09:54:48.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.335+0000 7faea384c640 1 -- 172.21.15.148:0/3925183587 wait complete. 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.338+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f846406b4d0 msgr2=0x7f846406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.338+0000 7f8487da9640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f846406b4d0 0x7f846406d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8470034000 tx=0x7f847002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.338+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 msgr2=0x7f84801572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.338+0000 7f8487da9640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 0x7f84801572b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c390 tx=0x7f847c00c850 comp rx=0 tx=0).stop 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 shutdown_connections 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848015b710 0x7f84800a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84800cf640 0x7f84800a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:48.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 --2- 172.21.15.148:0/4136746644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84800ac8b0 0x7f84801572b0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c390 tx=0x7f847c00c850 comp rx=0 tx=0).stop 2024-04-25T09:54:48.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 >> 172.21.15.148:0/4136746644 conn(0x7f848006abe0 msgr2=0x7f84800b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:48.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.339+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 shutdown_connections 2024-04-25T09:54:48.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:48.340+0000 7f8487da9640 1 -- 172.21.15.148:0/4136746644 wait complete. 2024-04-25T09:54:48.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-04-25T09:54:48.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-04-25T09:54:48.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-25T09:54:48.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-04-25T09:54:48.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2024-04-25T09:54:48.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-04-25T09:54:48.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-25T09:54:49.332 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T09:54:49.349 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T09:54:49.353 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T09:54:49.355 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T09:54:49.355 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T09:54:49.357 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T09:54:49.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T09:54:49.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T09:54:49.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.481+0000 7ff13167f640 1 Processor -- start 2024-04-25T09:54:49.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.481+0000 7ff13167f640 1 -- start start 2024-04-25T09:54:49.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.482+0000 7ff13167f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff12c06ae50 0x7ff12c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.482+0000 7ff13167f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff12c0c36d0 0x7ff12c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.482+0000 7ff13167f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 0x7ff12c0cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.482+0000 7ff13167f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12c06ecd0 con 0x7ff12c06ae50 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.482+0000 7ff13167f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12c1623b0 con 0x7ff12c0c36d0 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.482+0000 7ff13167f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0cd740 con 0x7ff12c06b770 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 0x7ff12c0cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 0x7ff12c0cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56482/0 (socket says 172.21.15.148:56482) 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 -- 172.21.15.148:0/1525249031 learned_addr learned my addr 172.21.15.148:0/1525249031 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff12b7fe640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.188:3300/0 conn(0x7ff12c0c36d0 0x7ff12c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff12bfff640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.148:3300/0 conn(0x7ff12c06ae50 0x7ff12c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 -- 172.21.15.148:0/1525249031 >> v2:172.21.15.188:3300/0 conn(0x7ff12c0c36d0 msgr2=0x7ff12c06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.188:3300/0 conn(0x7ff12c0c36d0 0x7ff12c06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 -- 172.21.15.148:0/1525249031 >> v2:172.21.15.148:3300/0 conn(0x7ff12c06ae50 msgr2=0x7ff12c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.148:3300/0 conn(0x7ff12c06ae50 0x7ff12c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.483+0000 7ff130e7e640 1 -- 172.21.15.148:0/1525249031 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c0cda20 con 0x7ff12c06b770 2024-04-25T09:54:49.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.484+0000 7ff12bfff640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.148:3300/0 conn(0x7ff12c06ae50 0x7ff12c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:49.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.486+0000 7ff130e7e640 1 --2- 172.21.15.148:0/1525249031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 0x7ff12c0cec00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0c72a0 tx=0x7ff12404c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe018f5ab8d2d7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.486+0000 7ff12affd640 1 -- 172.21.15.148:0/1525249031 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124058070 con 0x7ff12c06b770 2024-04-25T09:54:49.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.486+0000 7ff12affd640 1 -- 172.21.15.148:0/1525249031 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124053600 con 0x7ff12c06b770 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.487+0000 7ff13167f640 1 -- 172.21.15.148:0/1525249031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 msgr2=0x7ff12c0cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.487+0000 7ff13167f640 1 --2- 172.21.15.148:0/1525249031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 0x7ff12c0cec00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0c72a0 tx=0x7ff12404c6d0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 -- 172.21.15.148:0/1525249031 shutdown_connections 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 --2- 172.21.15.148:0/1525249031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06b770 0x7ff12c0cec00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.188:3300/0 conn(0x7ff12c0c36d0 0x7ff12c06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 --2- 172.21.15.148:0/1525249031 >> v2:172.21.15.148:3300/0 conn(0x7ff12c06ae50 0x7ff12c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 -- 172.21.15.148:0/1525249031 >> 172.21.15.148:0/1525249031 conn(0x7ff12c099a40 msgr2=0x7ff12c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 -- 172.21.15.148:0/1525249031 shutdown_connections 2024-04-25T09:54:49.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.488+0000 7ff13167f640 1 -- 172.21.15.148:0/1525249031 wait complete. 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.490+0000 7ff13167f640 1 Processor -- start 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.490+0000 7ff13167f640 1 -- start start 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.491+0000 7ff13167f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06ae50 0x7ff12c08aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.491+0000 7ff13167f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 0x7ff12c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.491+0000 7ff13167f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0c36d0 0x7ff12c0c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.492+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 0x7ff12c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.492+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 0x7ff12c0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33670/0 (socket says 172.21.15.148:33670) 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.492+0000 7ff13167f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0c3ab0 con 0x7ff12c06b770 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.492+0000 7ff13167f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c06e960 con 0x7ff12c0c36d0 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.492+0000 7ff13167f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0cf140 con 0x7ff12c06ae50 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.492+0000 7ff12b7fe640 1 -- 172.21.15.148:0/4215049323 learned_addr learned my addr 172.21.15.148:0/4215049323 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff130e7e640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0c36d0 0x7ff12c0c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff12b7fe640 1 -- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06ae50 msgr2=0x7ff12c08aa60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff12b7fe640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06ae50 0x7ff12c08aa60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff12b7fe640 1 -- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0c36d0 msgr2=0x7ff12c0c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff12b7fe640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0c36d0 0x7ff12c0c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff12b7fe640 1 -- 172.21.15.148:0/4215049323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c08a810 con 0x7ff12c06b770 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.493+0000 7ff12b7fe640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 0x7ff12c0c9110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9b0 tx=0x7ff11c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.495+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c017070 con 0x7ff12c06b770 2024-04-25T09:54:49.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.495+0000 7ff13167f640 1 -- 172.21.15.148:0/4215049323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c15d120 con 0x7ff12c06b770 2024-04-25T09:54:49.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.495+0000 7ff13167f640 1 -- 172.21.15.148:0/4215049323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c0c7a00 con 0x7ff12c06b770 2024-04-25T09:54:49.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.498+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c00c7f0 con 0x7ff12c06b770 2024-04-25T09:54:49.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.498+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00cbe0 con 0x7ff12c06b770 2024-04-25T09:54:49.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.498+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff11c01a070 con 0x7ff12c06b770 2024-04-25T09:54:49.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.499+0000 7ff128ff9640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff11806b5d0 0x7ff11806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.499+0000 7ff12bfff640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff11806b5d0 0x7ff11806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.499+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff11c08a3d0 con 0x7ff12c06b770 2024-04-25T09:54:49.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.500+0000 7ff12bfff640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff11806b5d0 0x7ff11806da90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff120009690 tx=0x7ff120008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.501+0000 7ff13167f640 1 -- 172.21.15.148:0/4215049323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c005290 con 0x7ff12c06b770 2024-04-25T09:54:49.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.510+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff11c058f80 con 0x7ff12c06b770 2024-04-25T09:54:49.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.543+0000 7f8caf59e640 1 Processor -- start 2024-04-25T09:54:49.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.543+0000 7f1a426a3640 1 Processor -- start 2024-04-25T09:54:49.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.544+0000 7f8caf59e640 1 -- start start 2024-04-25T09:54:49.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.544+0000 7f1a426a3640 1 -- start start 2024-04-25T09:54:49.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f2eae14e640 1 Processor -- start 2024-04-25T09:54:49.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f9de8cac640 1 Processor -- start 2024-04-25T09:54:49.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f9de8cac640 1 -- start start 2024-04-25T09:54:49.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f8caf59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ca804b3c0 0x7f8ca8034020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f8caf59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca8054c10 con 0x7f8ca8006a50 2024-04-25T09:54:49.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f8caed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ca8006a50 0x7f8ca8006e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48810/0 (socket says 172.21.15.148:48810) 2024-04-25T09:54:49.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f8caed9d640 1 -- 172.21.15.148:0/2567690046 learned_addr learned my addr 172.21.15.148:0/2567690046 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:49.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca8053a50 con 0x7f8ca804f4d0 2024-04-25T09:54:49.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca80062e0 con 0x7f8ca804b3c0 2024-04-25T09:54:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568878163 0 0) 0x7f8ca8054c10 con 0x7f8ca8006a50 2024-04-25T09:54:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a426a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c157700 0x7f1a3c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a426a3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a3c0aef40 0x7f1a3c0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a426a3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c071c50 con 0x7f1a3c0cd2e0 2024-04-25T09:54:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a426a3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c15e7d0 con 0x7f1a3c0aef40 2024-04-25T09:54:49.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a426a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c15d840 con 0x7f1a3c157700 2024-04-25T09:54:49.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f47c53f4640 1 Processor -- start 2024-04-25T09:54:49.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c98003520 con 0x7f8ca8006a50 2024-04-25T09:54:49.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f9de8cac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9de4071630 0x7f9de40b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f9de1d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9de4147630 0x7f9de4071a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48816/0 (socket says 172.21.15.148:48816) 2024-04-25T09:54:49.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f9de1d9b640 1 -- 172.21.15.148:0/2467305290 learned_addr learned my addr 172.21.15.148:0/2467305290 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:49.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f9de8cac640 1 --2- 172.21.15.148:0/2467305290 >> v2:172.21.15.148:3300/0 conn(0x7f9de415eb70 0x7f9de406d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de40b2ba0 con 0x7f9de4147630 2024-04-25T09:54:49.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f9de2d9d640 1 --2- 172.21.15.148:0/2467305290 >> v2:172.21.15.148:3300/0 conn(0x7f9de415eb70 0x7f9de406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de4147490 con 0x7f9de415eb70 2024-04-25T09:54:49.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de40b2e80 con 0x7f9de4071630 2024-04-25T09:54:49.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f9de259c640 1 --2- 172.21.15.148:0/2467305290 >> v2:172.21.15.188:3300/0 conn(0x7f9de4071630 0x7f9de40b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f8cae59c640 1 --2- 172.21.15.148:0/2567690046 >> v2:172.21.15.148:3300/0 conn(0x7f8ca804b3c0 0x7f8ca8034020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.545+0000 7f2eae14e640 1 -- start start 2024-04-25T09:54:49.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f2eae14e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ea815dbd0 0x7f2ea8161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f2eae14e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea8161e70 con 0x7f2ea80b95c0 2024-04-25T09:54:49.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f2eae14e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea8162150 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f2eae14e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80d1ce0 con 0x7f2ea815dbd0 2024-04-25T09:54:49.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f2ead14c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ea815dbd0 0x7f2ea8161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f2ead14c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2ea815dbd0 0x7f2ea8161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33680/0 (socket says 172.21.15.148:33680) 2024-04-25T09:54:49.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f2ead14c640 1 -- 172.21.15.148:0/1791080248 learned_addr learned my addr 172.21.15.148:0/1791080248 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f2e9ffff640 1 -- 172.21.15.148:0/1791080248 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027274173 0 0) 0x7f2ea8161e70 con 0x7f2ea80b95c0 2024-04-25T09:54:49.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f2e9ffff640 1 -- 172.21.15.148:0/1791080248 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e94003520 con 0x7f2ea80b95c0 2024-04-25T09:54:49.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f2e9ffff640 1 -- 172.21.15.148:0/1791080248 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2596747244 0 0) 0x7f2e94003520 con 0x7f2ea80b95c0 2024-04-25T09:54:49.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f47c53f4640 1 -- start start 2024-04-25T09:54:49.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f47c53f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 0x7f47b8034010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f47c53f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f47b8006a40 0x7f47b8006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a3b7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1a3c0cd2e0 0x7f1a3c0ae830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43176/0 (socket says 172.21.15.148:43176) 2024-04-25T09:54:49.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a3b7fe640 1 -- 172.21.15.148:0/507082342 learned_addr learned my addr 172.21.15.148:0/507082342 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:49.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.546+0000 7f1a3bfff640 1 --2- 172.21.15.148:0/507082342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c157700 0x7f1a3c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f1a3affd640 1 -- 172.21.15.148:0/507082342 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326269509 0 0) 0x7f1a3c071c50 con 0x7f1a3c0cd2e0 2024-04-25T09:54:49.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f1a3affd640 1 -- 172.21.15.148:0/507082342 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a24003520 con 0x7f1a3c0cd2e0 2024-04-25T09:54:49.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f1a3bfff640 1 -- 172.21.15.148:0/507082342 >> v2:172.21.15.188:3300/0 conn(0x7f1a3c0aef40 msgr2=0x7f1a3c0cd760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T09:54:49.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f1a3bfff640 1 --2- 172.21.15.148:0/507082342 >> v2:172.21.15.188:3300/0 conn(0x7f1a3c0aef40 0x7f1a3c0cd760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f1a3bfff640 1 -- 172.21.15.148:0/507082342 >> v1:172.21.15.188:6789/0 conn(0x7f1a3c0cd2e0 legacy=0x7f1a3c0ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.547+0000 7f1a3bfff640 1 -- 172.21.15.148:0/507082342 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c071f30 con 0x7f1a3c157700 2024-04-25T09:54:49.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f2ead14c640 1 -- 172.21.15.148:0/1791080248 >> v1:172.21.15.188:6789/0 conn(0x7f2ea80b5ae0 legacy=0x7f2ea80d2170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T09:54:49.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f9de259c640 1 -- 172.21.15.148:0/2467305290 >> v2:172.21.15.148:3300/0 conn(0x7f9de415eb70 msgr2=0x7f9de406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f9de259c640 1 --2- 172.21.15.148:0/2467305290 >> v2:172.21.15.148:3300/0 conn(0x7f9de415eb70 0x7f9de406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f47c53f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b8054c00 con 0x7f47b804f4c0 2024-04-25T09:54:49.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f47c53f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b80062d0 con 0x7f47b8006a40 2024-04-25T09:54:49.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f47c53f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8053a40 con 0x7f47b804b3b0 2024-04-25T09:54:49.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f9de259c640 1 -- 172.21.15.148:0/2467305290 >> v1:172.21.15.148:6789/0 conn(0x7f9de4147630 legacy=0x7f9de4071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952482100 0 0) 0x7f8ca8053a50 con 0x7f8ca804f4d0 2024-04-25T09:54:49.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f2ead14c640 1 -- 172.21.15.148:0/1791080248 >> v1:172.21.15.148:6789/0 conn(0x7f2ea80b95c0 legacy=0x7f2ea80b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f9de259c640 1 -- 172.21.15.148:0/2467305290 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40b3160 con 0x7f9de4071630 2024-04-25T09:54:49.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca8054c10 con 0x7f8ca804f4d0 2024-04-25T09:54:49.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f1a3bfff640 1 --2- 172.21.15.148:0/507082342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c157700 0x7f1a3c15e290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010c00 tx=0x7f1a2c032b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c60b103ce88e7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7fbf1a970640 1 Processor -- start 2024-04-25T09:54:49.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7fbf1a970640 1 -- start start 2024-04-25T09:54:49.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4173646592 0 0) 0x7f8ca8054c10 con 0x7f8ca804f4d0 2024-04-25T09:54:49.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 >> v2:172.21.15.148:3300/0 conn(0x7f8ca804b3c0 msgr2=0x7f8ca8034020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f1a3affd640 1 -- 172.21.15.148:0/507082342 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c0348c0 con 0x7f1a3c157700 2024-04-25T09:54:49.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 --2- 172.21.15.148:0/2567690046 >> v2:172.21.15.148:3300/0 conn(0x7f8ca804b3c0 0x7f8ca8034020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 >> v1:172.21.15.148:6789/0 conn(0x7f8ca8006a50 legacy=0x7f8ca8006e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f1a3affd640 1 -- 172.21.15.148:0/507082342 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c034a60 con 0x7f1a3c157700 2024-04-25T09:54:49.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f1a3affd640 1 -- 172.21.15.148:0/507082342 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c034e10 con 0x7f1a3c157700 2024-04-25T09:54:49.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8053d30 con 0x7f8ca804f4d0 2024-04-25T09:54:49.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.548+0000 7f2ead14c640 1 -- 172.21.15.148:0/1791080248 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8162430 con 0x7f2ea815dbd0 2024-04-25T09:54:49.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f8ca4001670 con 0x7f8ca804f4d0 2024-04-25T09:54:49.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 >> v1:172.21.15.188:6789/0 conn(0x7f8ca804f4d0 legacy=0x7f8ca804f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c98003b40 0x7f8c98003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980051c0 0x7f8c9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f9de259c640 1 --2- 172.21.15.148:0/2467305290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40b3c40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9de40d0b80 tx=0x7f9dd402a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10db2be186cf66a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f565af77640 1 Processor -- start 2024-04-25T09:54:49.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f565af77640 1 -- start start 2024-04-25T09:54:49.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80062e0 con 0x7f8c98003b40 2024-04-25T09:54:49.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800c0f0 con 0x7f8c980051c0 2024-04-25T09:54:49.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9800c2f0 con 0x7f8c98004800 2024-04-25T09:54:49.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f2ead14c640 1 --2- 172.21.15.148:0/1791080248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea815dbd0 0x7f2ea8161620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009870 tx=0x7f2ea402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c06de761dda3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f9de159a640 1 -- 172.21.15.148:0/2467305290 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4037070 con 0x7f9de4071630 2024-04-25T09:54:49.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8caed9d640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980051c0 0x7f8c9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f9de159a640 1 -- 172.21.15.148:0/2467305290 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd402acc0 con 0x7f9de4071630 2024-04-25T09:54:49.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f9de159a640 1 -- 172.21.15.148:0/2467305290 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4037070 con 0x7f9de4071630 2024-04-25T09:54:49.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f1a426a3640 1 -- 172.21.15.148:0/507082342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c157700 msgr2=0x7f1a3c15e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f1a426a3640 1 --2- 172.21.15.148:0/507082342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c157700 0x7f1a3c15e290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010c00 tx=0x7f1a2c032b60 comp rx=0 tx=0).stop 2024-04-25T09:54:49.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f1a426a3640 1 -- 172.21.15.148:0/507082342 shutdown_connections 2024-04-25T09:54:49.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f1a426a3640 1 --2- 172.21.15.148:0/507082342 >> v2:172.21.15.188:3300/0 conn(0x7f1a3c0aef40 0x7f1a3c0cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f1a426a3640 1 --2- 172.21.15.148:0/507082342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c157700 0x7f1a3c15e290 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010c00 tx=0x7f1a2c032b60 comp rx=0 tx=0).stop 2024-04-25T09:54:49.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.549+0000 7f1a426a3640 1 -- 172.21.15.148:0/507082342 >> 172.21.15.148:0/507082342 conn(0x7f1a3c06abe0 msgr2=0x7f1a3c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f1a426a3640 1 -- 172.21.15.148:0/507082342 shutdown_connections 2024-04-25T09:54:49.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf1a970640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbf140b29a0 0x7fbf140b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf1a970640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf140713d0 0x7fbf14071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf13fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fbf1415a030 0x7fbf1415ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43180/0 (socket says 172.21.15.148:43180) 2024-04-25T09:54:49.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f1a426a3640 1 -- 172.21.15.148:0/507082342 wait complete. 2024-04-25T09:54:49.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 0x7f47b8034010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 0x7f47b8034010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56500/0 (socket says 172.21.15.148:56500) 2024-04-25T09:54:49.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 -- 172.21.15.148:0/2328017860 learned_addr learned my addr 172.21.15.148:0/2328017860 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf13fff640 1 -- 172.21.15.148:0/605701901 learned_addr learned my addr 172.21.15.148:0/605701901 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:49.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f8cadd9b640 1 -- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980051c0 msgr2=0x7f8c9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980051c0 0x7f8c9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f8cadd9b640 1 -- 172.21.15.148:0/2567690046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c98003b40 msgr2=0x7f8c98003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c98003b40 0x7f8c98003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f8cadd9b640 1 -- 172.21.15.148:0/2567690046 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8054c10 con 0x7f8c98004800 2024-04-25T09:54:49.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1415a410 con 0x7fbf1415a030 2024-04-25T09:54:49.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f565af77640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f56540cfc30 0x7f5654166410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f565af77640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f56540b3120 0x7f565406d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1406edc0 con 0x7fbf140b29a0 2024-04-25T09:54:49.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f565af77640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5654162bb0 con 0x7f56541623a0 2024-04-25T09:54:49.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f565af77640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56541592e0 con 0x7f56540cfc30 2024-04-25T09:54:49.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f565af77640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5654162e90 con 0x7f56540b3120 2024-04-25T09:54:49.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf14071d40 con 0x7fbf140713d0 2024-04-25T09:54:49.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7fbf18f0d640 1 --2- 172.21.15.148:0/605701901 >> v2:172.21.15.188:3300/0 conn(0x7fbf140713d0 0x7fbf14071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f8caed9d640 1 -- 172.21.15.148:0/2567690046 reap_dead start 2024-04-25T09:54:49.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f5658d13640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f56540cfc30 0x7f5654166410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f5658d13640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f56540cfc30 0x7f5654166410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33736/0 (socket says 172.21.15.148:33736) 2024-04-25T09:54:49.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f5658d13640 1 -- 172.21.15.148:0/3731728974 learned_addr learned my addr 172.21.15.148:0/3731728974 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f2e9ffff640 1 -- 172.21.15.148:0/1791080248 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4037070 con 0x7f2ea815dbd0 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f2e9ffff640 1 -- 172.21.15.148:0/1791080248 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea402ace0 con 0x7f2ea815dbd0 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 msgr2=0x7f9de40b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 --2- 172.21.15.148:0/2467305290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40b3c40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9de40d0b80 tx=0x7f9dd402a030 comp rx=0 tx=0).stop 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f5659514640 1 --2- 172.21.15.148:0/3731728974 >> v2:172.21.15.188:3300/0 conn(0x7f56540b3120 0x7f565406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f5658d13640 1 -- 172.21.15.148:0/3731728974 >> v2:172.21.15.188:3300/0 conn(0x7f56540b3120 msgr2=0x7f565406d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f5658d13640 1 --2- 172.21.15.148:0/3731728974 >> v2:172.21.15.188:3300/0 conn(0x7f56540b3120 0x7f565406d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f5658d13640 1 -- 172.21.15.148:0/3731728974 >> v1:172.21.15.188:6789/0 conn(0x7f56541623a0 legacy=0x7f56540b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8c98004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4003c60 tx=0x7f8c98005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27bb4a1253401ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 shutdown_connections 2024-04-25T09:54:49.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 --2- 172.21.15.148:0/2467305290 >> v2:172.21.15.148:3300/0 conn(0x7f9de415eb70 0x7f9de406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 --2- 172.21.15.148:0/2467305290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40b3c40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 >> 172.21.15.148:0/2467305290 conn(0x7f9de40b1790 msgr2=0x7f9de40b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f5658d13640 1 -- 172.21.15.148:0/3731728974 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5654163170 con 0x7f56540cfc30 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47bf7fe640 1 --2- 172.21.15.148:0/2328017860 >> v2:172.21.15.148:3300/0 conn(0x7f47b8006a40 0x7f47b8006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 -- 172.21.15.148:0/2328017860 >> v2:172.21.15.148:3300/0 conn(0x7f47b8006a40 msgr2=0x7f47b8006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 --2- 172.21.15.148:0/2328017860 >> v2:172.21.15.148:3300/0 conn(0x7f47b8006a40 0x7f47b8006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 -- 172.21.15.148:0/2328017860 >> v1:172.21.15.148:6789/0 conn(0x7f47b804f4c0 legacy=0x7f47b804f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.550+0000 7f47beffd640 1 -- 172.21.15.148:0/2328017860 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8053d20 con 0x7f47b804b3b0 2024-04-25T09:54:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 shutdown_connections 2024-04-25T09:54:49.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4003830 con 0x7f8c98004800 2024-04-25T09:54:49.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f9de8cac640 1 -- 172.21.15.148:0/2467305290 wait complete. 2024-04-25T09:54:49.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f2e9ffff640 1 -- 172.21.15.148:0/1791080248 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98001650 con 0x7f2ea815dbd0 2024-04-25T09:54:49.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca402f070 con 0x7f8c98004800 2024-04-25T09:54:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280391067 0 0) 0x7fbf1415a410 con 0x7fbf1415a030 2024-04-25T09:54:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbefc003520 con 0x7fbf1415a030 2024-04-25T09:54:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f8cad59a640 1 -- 172.21.15.148:0/2567690046 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4004640 con 0x7f8c98004800 2024-04-25T09:54:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf137fe640 1 --2- 172.21.15.148:0/605701901 >> v2:172.21.15.148:3300/0 conn(0x7fbf140b29a0 0x7fbf140b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789089358 0 0) 0x7fbefc003520 con 0x7fbf1415a030 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 >> v2:172.21.15.188:3300/0 conn(0x7fbf140713d0 msgr2=0x7fbf14071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 --2- 172.21.15.148:0/605701901 >> v2:172.21.15.188:3300/0 conn(0x7fbf140713d0 0x7fbf14071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 >> v2:172.21.15.148:3300/0 conn(0x7fbf140b29a0 msgr2=0x7fbf140b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 --2- 172.21.15.148:0/605701901 >> v2:172.21.15.148:3300/0 conn(0x7fbf140b29a0 0x7fbf140b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140b3a90 con 0x7fbf1415a030 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbf04001670 con 0x7fbf1415a030 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 >> v1:172.21.15.188:6789/0 conn(0x7fbf1415a030 legacy=0x7fbf1415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbefc0039e0 0x7fbefc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f2eae14e640 1 -- 172.21.15.148:0/1791080248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea815dbd0 msgr2=0x7f2ea8161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f2eae14e640 1 --2- 172.21.15.148:0/1791080248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea815dbd0 0x7f2ea8161620 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4009870 tx=0x7f2ea402a250 comp rx=0 tx=0).stop 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbefc004800 0x7fbefc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbefc0051c0 0x7fbefc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14071d40 con 0x7fbefc0039e0 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc00c010 con 0x7fbefc0051c0 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc00c210 con 0x7fbefc004800 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf13fff640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbefc0039e0 0x7fbefc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf18f0d640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbefc0051c0 0x7fbefc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.551+0000 7f1a426a3640 1 Processor -- start 2024-04-25T09:54:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f1a426a3640 1 -- start start 2024-04-25T09:54:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f47beffd640 1 --2- 172.21.15.148:0/2328017860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 0x7f47b8034010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009870 tx=0x7f47b002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d31dcd14c692701d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f2eae14e640 1 -- 172.21.15.148:0/1791080248 shutdown_connections 2024-04-25T09:54:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f47bdffb640 1 -- 172.21.15.148:0/2328017860 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0037070 con 0x7f47b804b3b0 2024-04-25T09:54:49.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f47bdffb640 1 -- 172.21.15.148:0/2328017860 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0032440 con 0x7f47b804b3b0 2024-04-25T09:54:49.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f47bdffb640 1 -- 172.21.15.148:0/2328017860 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0032810 con 0x7f47b804b3b0 2024-04-25T09:54:49.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f2eae14e640 1 --2- 172.21.15.148:0/1791080248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea815dbd0 0x7f2ea8161620 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f2eae14e640 1 -- 172.21.15.148:0/1791080248 >> 172.21.15.148:0/1791080248 conn(0x7f2ea80d03d0 msgr2=0x7f2ea80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 msgr2=0x7f8c98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f2eae14e640 1 -- 172.21.15.148:0/1791080248 shutdown_connections 2024-04-25T09:54:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8c98004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4003c60 tx=0x7f8c98005830 comp rx=0 tx=0).stop 2024-04-25T09:54:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f2eae14e640 1 -- 172.21.15.148:0/1791080248 wait complete. 2024-04-25T09:54:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7f5658d13640 1 --2- 172.21.15.148:0/3731728974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540cfc30 0x7f5654166410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5654070f30 tx=0x7f564802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce31a5b5026467c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 shutdown_connections 2024-04-25T09:54:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2567690046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8c98004c80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f56537fe640 1 -- 172.21.15.148:0/3731728974 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648037070 con 0x7f56540cfc30 2024-04-25T09:54:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f56537fe640 1 -- 172.21.15.148:0/3731728974 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5648032440 con 0x7f56540cfc30 2024-04-25T09:54:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f56537fe640 1 -- 172.21.15.148:0/3731728974 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648032810 con 0x7f56540cfc30 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 >> 172.21.15.148:0/2567690046 conn(0x7f8ca800a400 msgr2=0x7f8ca800a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a426a3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 0x7f1a3c146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a426a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c0cd2e0 0x7f1a3c15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a426a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a3c150c80 0x7f1a3c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a426a3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0718e0 con 0x7f1a3c0aef40 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a426a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c15dd70 con 0x7f1a3c150c80 2024-04-25T09:54:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a426a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c157ae0 con 0x7f1a3c0cd2e0 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf13fff640 1 -- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbefc004800 msgr2=0x7fbefc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf13fff640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbefc004800 0x7fbefc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf13fff640 1 -- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbefc0051c0 msgr2=0x7fbefc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf13fff640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbefc0051c0 0x7fbefc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf13fff640 1 -- 172.21.15.148:0/605701901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf140b3a90 con 0x7fbefc0039e0 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.552+0000 7fbf18f0d640 1 -- 172.21.15.148:0/605701901 reap_dead start 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 shutdown_connections 2024-04-25T09:54:49.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f9de8cac640 1 Processor -- start 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f8caf59e640 1 -- 172.21.15.148:0/2567690046 wait complete. 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 -- 172.21.15.148:0/3731728974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540cfc30 msgr2=0x7f5654166410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 --2- 172.21.15.148:0/3731728974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540cfc30 0x7f5654166410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5654070f30 tx=0x7f564802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 -- 172.21.15.148:0/3731728974 shutdown_connections 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 --2- 172.21.15.148:0/3731728974 >> v2:172.21.15.188:3300/0 conn(0x7f56540b3120 0x7f565406d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 --2- 172.21.15.148:0/3731728974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540cfc30 0x7f5654166410 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 -- 172.21.15.148:0/3731728974 >> 172.21.15.148:0/3731728974 conn(0x7f565406abe0 msgr2=0x7f565406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 -- 172.21.15.148:0/3731728974 shutdown_connections 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f565af77640 1 -- 172.21.15.148:0/3731728974 wait complete. 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 0x7f1a3c146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a3bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 0x7f1a3c146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33756/0 (socket says 172.21.15.148:33756) 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a3bfff640 1 -- 172.21.15.148:0/3448588436 learned_addr learned my addr 172.21.15.148:0/3448588436 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a40c40640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a3c150c80 0x7f1a3c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f1a3b7fe640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c0cd2e0 0x7f1a3c15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f1a3bfff640 1 -- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c0cd2e0 msgr2=0x7f1a3c15c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f1a3bfff640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c0cd2e0 0x7f1a3c15c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f1a3bfff640 1 -- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a3c150c80 msgr2=0x7f1a3c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f1a3bfff640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a3c150c80 0x7f1a3c15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f1a3bfff640 1 -- 172.21.15.148:0/3448588436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c1511f0 con 0x7f1a3c0aef40 2024-04-25T09:54:49.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f47c53f4640 1 -- 172.21.15.148:0/2328017860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 msgr2=0x7f47b8034010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f47c53f4640 1 --2- 172.21.15.148:0/2328017860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 0x7f47b8034010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009870 tx=0x7f47b002a210 comp rx=0 tx=0).stop 2024-04-25T09:54:49.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7fbf13fff640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbefc0039e0 0x7fbefc003e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c240 tx=0x7fbefc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5260f46b7098b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.553+0000 7f9de8cac640 1 -- start start 2024-04-25T09:54:49.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f2eae14e640 1 Processor -- start 2024-04-25T09:54:49.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f1a3bfff640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 0x7f1a3c146070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010d20 tx=0x7f1a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c043040 con 0x7f1a3c0aef40 2024-04-25T09:54:49.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c04d070 con 0x7f1a3c0aef40 2024-04-25T09:54:49.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c007e20 con 0x7f1a3c0aef40 2024-04-25T09:54:49.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f9de8cac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c1f3160 con 0x7f1a3c0aef40 2024-04-25T09:54:49.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c1f3720 con 0x7f1a3c0aef40 2024-04-25T09:54:49.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f9de8cac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de4147630 0x7f9de40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f9de259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f9de259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57596/0 (socket says 172.21.15.148:57596) 2024-04-25T09:54:49.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f9de259c640 1 -- 172.21.15.148:0/19601386 learned_addr learned my addr 172.21.15.148:0/19601386 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9de415eb70 0x7f9de415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de406dbd0 con 0x7f9de4147630 2024-04-25T09:54:49.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f8caf59e640 1 Processor -- start 2024-04-25T09:54:49.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.554+0000 7f2eae14e640 1 -- start start 2024-04-25T09:54:49.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f8caf59e640 1 -- start start 2024-04-25T09:54:49.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de2d9d640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9de415eb70 0x7f9de415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04007d30 con 0x7fbefc0039e0 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf04005ad0 con 0x7fbefc0039e0 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.555+0000 7fbf12ffd640 1 -- 172.21.15.148:0/605701901 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04007070 con 0x7fbefc0039e0 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbefc0039e0 msgr2=0x7fbefc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7fbf1a970640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbefc0039e0 0x7fbefc003e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c240 tx=0x7fbefc005830 comp rx=0 tx=0).stop 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de415f000 con 0x7f9de4071630 2024-04-25T09:54:49.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de406dd50 con 0x7f9de415eb70 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 shutdown_connections 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7fbf1a970640 1 --2- 172.21.15.148:0/605701901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbefc0039e0 0x7fbefc003e00 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c240 tx=0x7fbefc005830 comp rx=0 tx=0).stop 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 >> 172.21.15.148:0/605701901 conn(0x7fbf140b1560 msgr2=0x7fbf140b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f47c53f4640 1 -- 172.21.15.148:0/2328017860 shutdown_connections 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f47c53f4640 1 --2- 172.21.15.148:0/2328017860 >> v2:172.21.15.148:3300/0 conn(0x7f47b8006a40 0x7f47b8006ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f47c53f4640 1 --2- 172.21.15.148:0/2328017860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b804b3b0 0x7f47b8034010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f47c53f4640 1 -- 172.21.15.148:0/2328017860 >> 172.21.15.148:0/2328017860 conn(0x7f47b800a3f0 msgr2=0x7f47b800a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2eae14e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 0x7f2ea81f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2eae14e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea80b95c0 0x7f2ea80c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f565af77640 1 Processor -- start 2024-04-25T09:54:49.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2eae14e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea815dbd0 0x7f2ea8160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2eae14e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80d2880 con 0x7f2ea80b95c0 2024-04-25T09:54:49.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2eae14e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8161b60 con 0x7f2ea815dbd0 2024-04-25T09:54:49.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2eae14e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80d2a00 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de1d9b640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de4147630 0x7f9de40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f565af77640 1 -- start start 2024-04-25T09:54:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de2d9d640 1 -- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 msgr2=0x7f9de40cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de2d9d640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de2d9d640 1 -- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de4147630 msgr2=0x7f9de40cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2ead14c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 0x7f2ea81f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de2d9d640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de4147630 0x7f9de40cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2ead14c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 0x7f2ea81f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56556/0 (socket says 172.21.15.148:56556) 2024-04-25T09:54:49.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de2d9d640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40cd640 con 0x7f9de415eb70 2024-04-25T09:54:49.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f2ead14c640 1 -- 172.21.15.148:0/143349089 learned_addr learned my addr 172.21.15.148:0/143349089 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de1d9b640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de4147630 0x7f9de40cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.556+0000 7f9de259c640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f2eac94b640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea80b95c0 0x7f2ea80c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f47c53f4640 1 -- 172.21.15.148:0/2328017860 shutdown_connections 2024-04-25T09:54:49.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 shutdown_connections 2024-04-25T09:54:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f8caf59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8ca80d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7fbf1a970640 1 -- 172.21.15.148:0/605701901 wait complete. 2024-04-25T09:54:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f2ead14c640 1 -- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea815dbd0 msgr2=0x7f2ea8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f2ead14c640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea815dbd0 0x7f2ea8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f2ead14c640 1 -- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea80b95c0 msgr2=0x7f2ea80c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f2ead14c640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea80b95c0 0x7f2ea80c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f2ead14c640 1 -- 172.21.15.148:0/143349089 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80c9460 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f9de2d9d640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9de415eb70 0x7f9de415dbb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00eba0 tx=0x7f9ddc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8ca80d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8ca80d9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56562/0 (socket says 172.21.15.148:56562) 2024-04-25T09:54:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c073a00 con 0x7f1a3c0aef40 2024-04-25T09:54:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f47c53f4640 1 -- 172.21.15.148:0/2328017860 wait complete. 2024-04-25T09:54:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8caf59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca80d7bf0 0x7f8ca804bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.557+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc018020 con 0x7f9de415eb70 2024-04-25T09:54:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8cae59c640 1 -- 172.21.15.148:0/2562272574 learned_addr learned my addr 172.21.15.148:0/2562272574 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc00cd20 con 0x7f9de415eb70 2024-04-25T09:54:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de40cd910 con 0x7f9de415eb70 2024-04-25T09:54:49.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f2ead14c640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 0x7f2ea81f8c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4000c00 tx=0x7f2ea40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de40cf600 con 0x7f9de415eb70 2024-04-25T09:54:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4037040 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4004250 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4003610 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f565af77640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540b3120 0x7f5654165d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca80d7bf0 0x7f8ca804bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca80e5ee0 0x7f8ca80e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f5658d13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540b3120 0x7f5654165d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8caed9d640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca80e5ee0 0x7f8ca80e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca8054830 con 0x7f8ca80e5ee0 2024-04-25T09:54:49.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca804b7a0 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80549b0 con 0x7f8c98004800 2024-04-25T09:54:49.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea81eb8d0 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea815eab0 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a2c003610 con 0x7f1a3c0aef40 2024-04-25T09:54:49.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.558+0000 7f1a38ff9640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a2806b520 0x7f1a2806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0146c0 con 0x7f9de415eb70 2024-04-25T09:54:49.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f47c53f4640 1 Processor -- start 2024-04-25T09:54:49.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1a2c0b2f60 con 0x7f1a3c0aef40 2024-04-25T09:54:49.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f1a3b7fe640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a2806b520 0x7f1a2806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7fbf1a970640 1 Processor -- start 2024-04-25T09:54:49.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f47c53f4640 1 -- start start 2024-04-25T09:54:49.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cadd9b640 1 -- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 msgr2=0x7f8ca80d9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8ca80d9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cadd9b640 1 -- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca80e5ee0 msgr2=0x7f8ca80e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca80e5ee0 0x7f8ca80e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cadd9b640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80d8070 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f1a3b7fe640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a2806b520 0x7f1a2806d9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c15cfe0 tx=0x7f1a30005b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cae59c640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8ca80d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:49.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8caed9d640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca80e5ee0 0x7f8ca80e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:49.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ddc005d10 con 0x7f9de415eb70 2024-04-25T09:54:49.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f8cadd9b640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca80d7bf0 0x7f8ca804bf50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4008fd0 tx=0x7f8ca40036b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e88005290 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7f47c53f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b8006a40 0x7f47b8035780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.560+0000 7fbf1a970640 1 -- start start 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f565af77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56540cfc30 0x7f56541f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f565af77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56541623a0 0x7f56541f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f565af77640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565406dc70 con 0x7f56540b3120 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f5658d13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540b3120 0x7f5654165d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33800/0 (socket says 172.21.15.148:33800) 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f5658d13640 1 -- 172.21.15.148:0/928039859 learned_addr learned my addr 172.21.15.148:0/928039859 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654162820 con 0x7f56540cfc30 2024-04-25T09:54:49.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.559+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565406ddf0 con 0x7f56541623a0 2024-04-25T09:54:49.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47c53f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47b804b3b0 0x7f47b8189110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4008020 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca40081c0 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4008020 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca803a9b0 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47beffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b8006a40 0x7f47b8035780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47c53f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b804f4c0 0x7f47b8039760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47c53f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8050050 con 0x7f47b804f4c0 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47c53f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b804b790 con 0x7f47b804b3b0 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47beffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b8006a40 0x7f47b8035780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56566/0 (socket says 172.21.15.148:56566) 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47c53f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8054820 con 0x7f47b8006a40 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47beffd640 1 -- 172.21.15.148:0/1818944268 learned_addr learned my addr 172.21.15.148:0/1818944268 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47be7fc640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47b804b3b0 0x7f47b8189110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc8005290 con 0x7f9de415eb70 2024-04-25T09:54:49.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f47bf7fe640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b804f4c0 0x7f47b8039760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f5653fff640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56540cfc30 0x7f56541f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.561+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca804de30 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f5653fff640 1 -- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56541623a0 msgr2=0x7f56541f70b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T09:54:49.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f5653fff640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56541623a0 0x7f56541f70b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f5653fff640 1 -- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540b3120 msgr2=0x7f5654165d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f5653fff640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540b3120 0x7f5654165d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f5653fff640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56541493e0 con 0x7f56540cfc30 2024-04-25T09:54:49.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf1a970640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf141ef240 0x7fbf1415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf1a970640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf14140b70 0x7fbf14140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf1a970640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf14141530 0x7fbf140af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f5653fff640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56540cfc30 0x7f56541f6b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f564000c390 tx=0x7f564000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf141ef240 0x7fbf1415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf13fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf141ef240 0x7fbf1415c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56572/0 (socket says 172.21.15.148:56572) 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf13fff640 1 -- 172.21.15.148:0/1032455618 learned_addr learned my addr 172.21.15.148:0/1032455618 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140b42f0 con 0x7fbf14141530 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf140b4010 con 0x7fbf14140b70 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f47bf7fe640 1 -- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b8006a40 msgr2=0x7f47b8035780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f47bf7fe640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b8006a40 0x7f47b8035780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f47bf7fe640 1 -- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47b804b3b0 msgr2=0x7f47b8189110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f47bf7fe640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47b804b3b0 0x7f47b8189110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1415a680 con 0x7fbf141ef240 2024-04-25T09:54:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf137fe640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf14140b70 0x7fbf14140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f47bf7fe640 1 -- 172.21.15.148:0/1818944268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8189ad0 con 0x7f47b804f4c0 2024-04-25T09:54:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f47be7fc640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47b804b3b0 0x7f47b8189110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:49.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640005220 con 0x7f56540cfc30 2024-04-25T09:54:49.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7fbf18f0d640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf14141530 0x7fbf140af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f9dd2ffd640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dc406b5a0 0x7f9dc406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ea403a070 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.562+0000 7f2e9dffb640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b5a0 0x7f2e8c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f47bf7fe640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b804f4c0 0x7f47b8039760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47c004f440 tx=0x7f47c0067fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf18f0d640 1 -- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf141ef240 msgr2=0x7fbf1415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf18f0d640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf141ef240 0x7fbf1415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2ea40aaac0 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f9de259c640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dc406b5a0 0x7f9dc406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca8008e20 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9ddc08b0b0 con 0x7f9de415eb70 2024-04-25T09:54:49.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f2eac94b640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b5a0 0x7f2e8c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf18f0d640 1 -- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf14140b70 msgr2=0x7fbf14140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf18f0d640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf14140b70 0x7fbf14140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf18f0d640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf141419b0 con 0x7fbf14141530 2024-04-25T09:54:49.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf18f0d640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf14141530 0x7fbf140af220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b25c0 tx=0x7fbf0c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7fbf137fe640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf14140b70 0x7fbf14140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:49.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56400053c0 con 0x7f56540cfc30 2024-04-25T09:54:49.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640007230 con 0x7f56540cfc30 2024-04-25T09:54:49.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5654148d60 con 0x7f56540cfc30 2024-04-25T09:54:49.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.563+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5654146420 con 0x7f56540cfc30 2024-04-25T09:54:49.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.564+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ca4046020 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.564+0000 7f9de259c640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dc406b5a0 0x7f9dc406da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ae40 tx=0x7f9dd402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.564+0000 7f2eac94b640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b5a0 0x7f2e8c06da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80c96d0 tx=0x7f2e98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.564+0000 7f8c9effd640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b900 0x7f8c9006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.564+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8ca4001ba0 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.564+0000 7f8cae59c640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b900 0x7f8c9006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.565+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5654073990 con 0x7f56540cfc30 2024-04-25T09:54:49.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.565+0000 7f8cae59c640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b900 0x7f8c9006ddc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00690f0 tx=0x7f8cb008c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5640019020 con 0x7f56540cfc30 2024-04-25T09:54:49.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f56517fa640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f563806b520 0x7f563806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f56400895b0 con 0x7f56540cfc30 2024-04-25T09:54:49.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0097070 con 0x7f47b804f4c0 2024-04-25T09:54:49.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0074610 con 0x7f47b804f4c0 2024-04-25T09:54:49.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0092630 con 0x7f47b804f4c0 2024-04-25T09:54:49.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f5658d13640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f563806b520 0x7f563806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a2c081a80 con 0x7f1a3c0aef40 2024-04-25T09:54:49.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b8189db0 con 0x7f47b804f4c0 2024-04-25T09:54:49.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b8035cc0 con 0x7f47b804f4c0 2024-04-25T09:54:49.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011e10 con 0x7fbf14141530 2024-04-25T09:54:49.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1415ce60 con 0x7fbf14141530 2024-04-25T09:54:49.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c01b070 con 0x7fbf14141530 2024-04-25T09:54:49.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c003990 con 0x7fbf14141530 2024-04-25T09:54:49.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.566+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1415d3e0 con 0x7fbf14141530 2024-04-25T09:54:49.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.567+0000 7f5658d13640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f563806b520 0x7f563806d9e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5648009630 tx=0x7f564802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.568+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b80057d0 con 0x7f47b804f4c0 2024-04-25T09:54:49.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.568+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf14071f20 con 0x7fbf14141530 2024-04-25T09:54:49.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.569+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47c009a070 con 0x7f47b804f4c0 2024-04-25T09:54:49.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7f47a77fe640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47ac06b520 0x7f47ac06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf0c005d10 con 0x7fbf14141530 2024-04-25T09:54:49.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7f47beffd640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47ac06b520 0x7f47ac06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f47c01052a0 con 0x7f47b804f4c0 2024-04-25T09:54:49.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7f47beffd640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47ac06b520 0x7f47ac06d9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009630 tx=0x7f47b002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ea80c9460 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7fbf10ff9640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf0006b550 0x7fbf0006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbf0c08b670 con 0x7fbf14141530 2024-04-25T09:54:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.570+0000 7fbf13fff640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf0006b550 0x7fbf0006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.571+0000 7fbf13fff640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf0006b550 0x7fbf0006da10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf1415e6c0 tx=0x7fbf0402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.574+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ca80d8070 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.579+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47c008f070 con 0x7f47b804f4c0 2024-04-25T09:54:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.580+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ddc022030 con 0x7f9de415eb70 2024-04-25T09:54:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.583+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56541493e0 con 0x7f56540cfc30 2024-04-25T09:54:49.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.586+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf0c05a290 con 0x7fbf14141530 2024-04-25T09:54:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.908+0000 7ff13167f640 1 -- 172.21.15.148:0/4215049323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff10c005580 con 0x7ff12c06b770 2024-04-25T09:54:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.909+0000 7ff128ff9640 1 -- 172.21.15.148:0/4215049323 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff11c058880 con 0x7ff12c06b770 2024-04-25T09:54:49.911 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-04-25T09:54:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.912+0000 7ff1127fc640 1 -- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff11806b5d0 msgr2=0x7ff11806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.912+0000 7ff1127fc640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff11806b5d0 0x7ff11806da90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff120009690 tx=0x7ff120008040 comp rx=0 tx=0).stop 2024-04-25T09:54:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.912+0000 7ff1127fc640 1 -- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 msgr2=0x7ff12c0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.912+0000 7ff1127fc640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 0x7ff12c0c9110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9b0 tx=0x7ff11c00ee70 comp rx=0 tx=0).stop 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 -- 172.21.15.148:0/4215049323 shutdown_connections 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff11806b5d0 0x7ff11806da90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0c36d0 0x7ff12c0c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff12c06b770 0x7ff12c0c9110 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 --2- 172.21.15.148:0/4215049323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff12c06ae50 0x7ff12c08aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 -- 172.21.15.148:0/4215049323 >> 172.21.15.148:0/4215049323 conn(0x7ff12c099a40 msgr2=0x7ff12c0ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 -- 172.21.15.148:0/4215049323 shutdown_connections 2024-04-25T09:54:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.913+0000 7ff1127fc640 1 -- 172.21.15.148:0/4215049323 wait complete. 2024-04-25T09:54:49.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-25T09:54:49.933 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.949+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2e88005580 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.950+0000 7f2e9dffb640 1 -- 172.21.15.148:0/143349089 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ea4041020 con 0x7f2ea80b5ae0 2024-04-25T09:54:49.954 INFO:teuthology.orchestra.run.smithi148.stdout:55834574859 2024-04-25T09:54:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.952+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b5a0 msgr2=0x7f2e8c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.952+0000 7f2eae14e640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b5a0 0x7f2e8c06da60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80c96d0 tx=0x7f2e98009040 comp rx=0 tx=0).stop 2024-04-25T09:54:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.952+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 msgr2=0x7f2ea81f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.952+0000 7f2eae14e640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 0x7f2ea81f8c70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4000c00 tx=0x7f2ea40046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 shutdown_connections 2024-04-25T09:54:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e8c06b5a0 0x7f2e8c06da60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea815dbd0 0x7f2ea8160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ea80b95c0 0x7f2ea80c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 --2- 172.21.15.148:0/143349089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea80b5ae0 0x7f2ea81f8c70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 >> 172.21.15.148:0/143349089 conn(0x7f2ea80d03d0 msgr2=0x7f2ea80b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 shutdown_connections 2024-04-25T09:54:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.953+0000 7f2eae14e640 1 -- 172.21.15.148:0/143349089 wait complete. 2024-04-25T09:54:49.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T09:54:49.966 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.974+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1a3c0cfc40 con 0x7f1a3c0aef40 2024-04-25T09:54:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.975+0000 7f1a38ff9640 1 -- 172.21.15.148:0/3448588436 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a2c081380 con 0x7f1a3c0aef40 2024-04-25T09:54:49.977 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:49.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.977+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a2806b520 msgr2=0x7f1a2806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.977+0000 7f1a426a3640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a2806b520 0x7f1a2806d9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c15cfe0 tx=0x7f1a30005b50 comp rx=0 tx=0).stop 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.977+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 msgr2=0x7f1a3c146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.977+0000 7f1a426a3640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 0x7f1a3c146070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c010d20 tx=0x7f1a2c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 shutdown_connections 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a2806b520 0x7f1a2806d9e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a3c150c80 0x7f1a3c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3c0cd2e0 0x7f1a3c15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 --2- 172.21.15.148:0/3448588436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0aef40 0x7f1a3c146070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 >> 172.21.15.148:0/3448588436 conn(0x7f1a3c06abe0 msgr2=0x7f1a3c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 shutdown_connections 2024-04-25T09:54:49.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.978+0000 7f1a426a3640 1 -- 172.21.15.148:0/3448588436 wait complete. 2024-04-25T09:54:49.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.979+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8ca80e68a0 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.980+0000 7f8c9effd640 1 -- 172.21.15.148:0/2562272574 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ca80d8070 con 0x7f8ca80d7bf0 2024-04-25T09:54:49.982 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b900 msgr2=0x7f8c9006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b900 0x7f8c9006ddc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00690f0 tx=0x7f8cb008c040 comp rx=0 tx=0).stop 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca80d7bf0 msgr2=0x7f8ca804bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca80d7bf0 0x7f8ca804bf50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4008fd0 tx=0x7f8ca40036b0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 shutdown_connections 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b900 0x7f8c9006ddc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca80e5ee0 0x7f8ca80e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca80d7bf0 0x7f8ca804bf50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 --2- 172.21.15.148:0/2562272574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98004800 0x7f8ca80d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 >> 172.21.15.148:0/2562272574 conn(0x7f8ca800a400 msgr2=0x7f8ca80357b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 shutdown_connections 2024-04-25T09:54:49.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.983+0000 7f8caf59e640 1 -- 172.21.15.148:0/2562272574 wait complete. 2024-04-25T09:54:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.990+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f565406bf80 con 0x7f56540cfc30 2024-04-25T09:54:49.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.991+0000 7f56517fa640 1 -- 172.21.15.148:0/928039859 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56400581d0 con 0x7f56540cfc30 2024-04-25T09:54:49.994 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.993+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f563806b520 msgr2=0x7f563806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.993+0000 7f565af77640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f563806b520 0x7f563806d9e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5648009630 tx=0x7f564802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.993+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56540cfc30 msgr2=0x7f56541f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.993+0000 7f565af77640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56540cfc30 0x7f56541f6b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f564000c390 tx=0x7f564000c850 comp rx=0 tx=0).stop 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 shutdown_connections 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f563806b520 0x7f563806d9e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56541623a0 0x7f56541f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56540cfc30 0x7f56541f6b70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 --2- 172.21.15.148:0/928039859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56540b3120 0x7f5654165d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:49.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 >> 172.21.15.148:0/928039859 conn(0x7f565406abe0 msgr2=0x7f56540af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:49.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f47b8054220 con 0x7f47b804f4c0 2024-04-25T09:54:49.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 shutdown_connections 2024-04-25T09:54:49.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f565af77640 1 -- 172.21.15.148:0/928039859 wait complete. 2024-04-25T09:54:49.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.994+0000 7f47a77fe640 1 -- 172.21.15.148:0/1818944268 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47c00680e0 con 0x7f47b804f4c0 2024-04-25T09:54:49.998 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:49.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-04-25T09:54:49.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.995+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbf141571c0 con 0x7fbf14141530 2024-04-25T09:54:49.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.996+0000 7fbf10ff9640 1 -- 172.21.15.148:0/1032455618 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf0c05a290 con 0x7fbf14141530 2024-04-25T09:54:50.000 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:50.000 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:50.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-25T09:54:50.001 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:50.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47ac06b520 msgr2=0x7f47ac06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47ac06b520 0x7f47ac06d9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009630 tx=0x7f47b002e000 comp rx=0 tx=0).stop 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b804f4c0 msgr2=0x7f47b8039760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b804f4c0 0x7f47b8039760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47c004f440 tx=0x7f47c0067fb0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 shutdown_connections 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47ac06b520 0x7f47ac06d9e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47b804f4c0 0x7f47b8039760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47b804b3b0 0x7f47b8189110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 --2- 172.21.15.148:0/1818944268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47b8006a40 0x7f47b8035780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 >> 172.21.15.148:0/1818944268 conn(0x7f47b800a3f0 msgr2=0x7f47b80e5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 shutdown_connections 2024-04-25T09:54:50.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.997+0000 7f47c53f4640 1 -- 172.21.15.148:0/1818944268 wait complete. 2024-04-25T09:54:50.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf0006b550 msgr2=0x7fbf0006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf0006b550 0x7fbf0006da10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf1415e6c0 tx=0x7fbf0402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:50.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf14141530 msgr2=0x7fbf140af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf14141530 0x7fbf140af220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b25c0 tx=0x7fbf0c0115a0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9dc8005580 con 0x7f9de415eb70 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 shutdown_connections 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf0006b550 0x7fbf0006da10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf14141530 0x7fbf140af220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf14140b70 0x7fbf14140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 --2- 172.21.15.148:0/1032455618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf141ef240 0x7fbf1415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 >> 172.21.15.148:0/1032455618 conn(0x7fbf140b1560 msgr2=0x7fbf140ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 shutdown_connections 2024-04-25T09:54:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.998+0000 7fbf1a970640 1 -- 172.21.15.148:0/1032455618 wait complete. 2024-04-25T09:54:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:49.999+0000 7f9dd2ffd640 1 -- 172.21.15.148:0/19601386 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ddc010040 con 0x7f9de415eb70 2024-04-25T09:54:50.005 INFO:teuthology.orchestra.run.smithi148.stdout:55834574858 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dc406b5a0 msgr2=0x7f9dc406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dc406b5a0 0x7f9dc406da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ae40 tx=0x7f9dd402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9de415eb70 msgr2=0x7f9de415dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9de415eb70 0x7f9de415dbb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00eba0 tx=0x7f9ddc00c490 comp rx=0 tx=0).stop 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 shutdown_connections 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dc406b5a0 0x7f9dc406da60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9de415eb70 0x7f9de415dbb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9de4147630 0x7f9de40cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 --2- 172.21.15.148:0/19601386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9de4071630 0x7f9de40cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 >> 172.21.15.148:0/19601386 conn(0x7f9de40b1790 msgr2=0x7f9de40af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 shutdown_connections 2024-04-25T09:54:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.001+0000 7f9de8cac640 1 -- 172.21.15.148:0/19601386 wait complete. 2024-04-25T09:54:50.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-25T09:54:50.012 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:50.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-25T09:54:50.014 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:50.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T09:54:50.019 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:50.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T09:54:50.020 DEBUG:teuthology.parallel:result is None 2024-04-25T09:54:50.021 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:54:50.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.157+0000 7f49356a1640 1 Processor -- start 2024-04-25T09:54:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.157+0000 7f49356a1640 1 -- start start 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.157+0000 7f49356a1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49300d1020 0x7f49300d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.157+0000 7f49356a1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300b97a0 con 0x7f493015dbe0 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.157+0000 7f49356a1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300d1c50 con 0x7f49300b2cb0 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.157+0000 7f49356a1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493015dfc0 con 0x7f49300d1020 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49300d1020 0x7f49300d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f493015dbe0 0x7f49300d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48844/0 (socket says 172.21.15.148:48844) 2024-04-25T09:54:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492effd640 1 -- 172.21.15.148:0/2320015335 learned_addr learned my addr 172.21.15.148:0/2320015335 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492f7fe640 1 -- 172.21.15.148:0/2320015335 >> v1:172.21.15.188:6789/0 conn(0x7f49300b2cb0 legacy=0x7f49300d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492f7fe640 1 -- 172.21.15.148:0/2320015335 >> v1:172.21.15.148:6789/0 conn(0x7f493015dbe0 legacy=0x7f49300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492f7fe640 1 -- 172.21.15.148:0/2320015335 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300d1f30 con 0x7f49300d1020 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.158+0000 7f492dffb640 1 -- 172.21.15.148:0/2320015335 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239903151 0 0) 0x7f49300b97a0 con 0x7f493015dbe0 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f492f7fe640 1 --2- 172.21.15.148:0/2320015335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300d1020 0x7f49300d2e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4928009d90 tx=0x7f492802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94df5ebcd60e7de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f492dffb640 1 -- 172.21.15.148:0/2320015335 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928007740 con 0x7f49300d1020 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f492dffb640 1 -- 172.21.15.148:0/2320015335 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49280078e0 con 0x7f49300d1020 2024-04-25T09:54:50.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f492dffb640 1 -- 172.21.15.148:0/2320015335 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928007cb0 con 0x7f49300d1020 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 -- 172.21.15.148:0/2320015335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300d1020 msgr2=0x7f49300d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 --2- 172.21.15.148:0/2320015335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300d1020 0x7f49300d2e40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4928009d90 tx=0x7f492802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 -- 172.21.15.148:0/2320015335 shutdown_connections 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 --2- 172.21.15.148:0/2320015335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300d1020 0x7f49300d2e40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 -- 172.21.15.148:0/2320015335 >> 172.21.15.148:0/2320015335 conn(0x7f4930061a20 msgr2=0x7f4930061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 -- 172.21.15.148:0/2320015335 shutdown_connections 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.159+0000 7f49356a1640 1 -- 172.21.15.148:0/2320015335 wait complete. 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 Processor -- start 2024-04-25T09:54:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 -- start start 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300b2cb0 0x7f49300cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49300d1020 0x7f49300caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 0x7f49300c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300b92a0 con 0x7f493015dbe0 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300d1a00 con 0x7f49300b2cb0 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f49356a1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49300b9420 con 0x7f49300d1020 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 0x7f49300c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 0x7f49300c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33854/0 (socket says 172.21.15.148:33854) 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.161+0000 7f492f7fe640 1 -- 172.21.15.148:0/882305886 learned_addr learned my addr 172.21.15.148:0/882305886 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492effd640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300b2cb0 0x7f49300cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492e7fc640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49300d1020 0x7f49300caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492f7fe640 1 -- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49300d1020 msgr2=0x7f49300caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492f7fe640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49300d1020 0x7f49300caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492f7fe640 1 -- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300b2cb0 msgr2=0x7f49300cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492f7fe640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300b2cb0 0x7f49300cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492f7fe640 1 -- 172.21.15.148:0/882305886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300cb1c0 con 0x7f493015dbe0 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492effd640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300b2cb0 0x7f49300cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f492f7fe640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 0x7f49300c95e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4928009d90 tx=0x7f4928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928040070 con 0x7f493015dbe0 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928031c90 con 0x7f493015dbe0 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492803c640 con 0x7f493015dbe0 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300cb4a0 con 0x7f493015dbe0 2024-04-25T09:54:50.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.162+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300cba60 con 0x7f493015dbe0 2024-04-25T09:54:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.163+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930073990 con 0x7f493015dbe0 2024-04-25T09:54:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.164+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4928039040 con 0x7f493015dbe0 2024-04-25T09:54:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.164+0000 7f49177fe640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491c06b500 0x7f491c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.164+0000 7f492effd640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491c06b500 0x7f491c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.164+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49280abb20 con 0x7f493015dbe0 2024-04-25T09:54:50.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.164+0000 7f492effd640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491c06b500 0x7f491c06d9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4920007fd0 tx=0x7f4920007dc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.168+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f492804b340 con 0x7f493015dbe0 2024-04-25T09:54:50.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.340+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49300b3d20 con 0x7f491c06b500 2024-04-25T09:54:50.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.348+0000 7f49177fe640 1 -- 172.21.15.148:0/882305886 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f49300b3d20 con 0x7f491c06b500 2024-04-25T09:54:50.354 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:50.356 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491c06b500 msgr2=0x7f491c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491c06b500 0x7f491c06d9c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4920007fd0 tx=0x7f4920007dc0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 msgr2=0x7f49300c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 0x7f49300c95e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4928009d90 tx=0x7f4928004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 shutdown_connections 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491c06b500 0x7f491c06d9c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f493015dbe0 0x7f49300c95e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49300d1020 0x7f49300caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 --2- 172.21.15.148:0/882305886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300b2cb0 0x7f49300cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 >> 172.21.15.148:0/882305886 conn(0x7f4930061a20 msgr2=0x7f49300b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 shutdown_connections 2024-04-25T09:54:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.351+0000 7f49356a1640 1 -- 172.21.15.148:0/882305886 wait complete. 2024-04-25T09:54:50.365 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T09:54:49.025642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003382"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086122+0000","last_change":"2024-04-25T09:54:22.962688+0000","last_active":"2024-04-25T09:54:30.086122+0000","last_peered":"2024-04-25T09:54:30.086122+0000","last_clean":"2024-04-25T09:54:30.086122+0000","last_became_active":"2024-04-25T09:54:22.962426+0000","last_became_peered":"2024-04-25T09:54:22.962426+0000","last_unstale":"2024-04-25T09:54:30.086122+0000","last_undegraded":"2024-04-25T09:54:30.086122+0000","last_fullsized":"2024-04-25T09:54:30.086122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:21.016302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995003+0000","last_change":"2024-04-25T09:54:23.969918+0000","last_active":"2024-04-25T09:54:29.995003+0000","last_peered":"2024-04-25T09:54:29.995003+0000","last_clean":"2024-04-25T09:54:29.995003+0000","last_became_active":"2024-04-25T09:54:23.969631+0000","last_became_peered":"2024-04-25T09:54:23.969631+0000","last_unstale":"2024-04-25T09:54:29.995003+0000","last_undegraded":"2024-04-25T09:54:29.995003+0000","last_fullsized":"2024-04-25T09:54:29.995003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994303+0000","last_change":"2024-04-25T09:54:28.397901+0000","last_active":"2024-04-25T09:54:29.994303+0000","last_peered":"2024-04-25T09:54:29.994303+0000","last_clean":"2024-04-25T09:54:29.994303+0000","last_became_active":"2024-04-25T09:54:28.397492+0000","last_became_peered":"2024-04-25T09:54:28.397492+0000","last_unstale":"2024-04-25T09:54:29.994303+0000","last_undegraded":"2024-04-25T09:54:29.994303+0000","last_fullsized":"2024-04-25T09:54:29.994303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:14.336697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995250+0000","last_change":"2024-04-25T09:54:22.958345+0000","last_active":"2024-04-25T09:54:29.995250+0000","last_peered":"2024-04-25T09:54:29.995250+0000","last_clean":"2024-04-25T09:54:29.995250+0000","last_became_active":"2024-04-25T09:54:22.958021+0000","last_became_peered":"2024-04-25T09:54:22.958021+0000","last_unstale":"2024-04-25T09:54:29.995250+0000","last_undegraded":"2024-04-25T09:54:29.995250+0000","last_fullsized":"2024-04-25T09:54:29.995250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:02.625621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085905+0000","last_change":"2024-04-25T09:54:23.970381+0000","last_active":"2024-04-25T09:54:30.085905+0000","last_peered":"2024-04-25T09:54:30.085905+0000","last_clean":"2024-04-25T09:54:30.085905+0000","last_became_active":"2024-04-25T09:54:23.970077+0000","last_became_peered":"2024-04-25T09:54:23.970077+0000","last_unstale":"2024-04-25T09:54:30.085905+0000","last_undegraded":"2024-04-25T09:54:30.085905+0000","last_fullsized":"2024-04-25T09:54:30.085905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:16.071642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995354+0000","last_change":"2024-04-25T09:54:28.011784+0000","last_active":"2024-04-25T09:54:29.995354+0000","last_peered":"2024-04-25T09:54:29.995354+0000","last_clean":"2024-04-25T09:54:29.995354+0000","last_became_active":"2024-04-25T09:54:28.011370+0000","last_became_peered":"2024-04-25T09:54:28.011370+0000","last_unstale":"2024-04-25T09:54:29.995354+0000","last_undegraded":"2024-04-25T09:54:29.995354+0000","last_fullsized":"2024-04-25T09:54:29.995354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088011+0000","last_change":"2024-04-25T09:54:22.965546+0000","last_active":"2024-04-25T09:54:30.088011+0000","last_peered":"2024-04-25T09:54:30.088011+0000","last_clean":"2024-04-25T09:54:30.088011+0000","last_became_active":"2024-04-25T09:54:22.965088+0000","last_became_peered":"2024-04-25T09:54:22.965088+0000","last_unstale":"2024-04-25T09:54:30.088011+0000","last_undegraded":"2024-04-25T09:54:30.088011+0000","last_fullsized":"2024-04-25T09:54:30.088011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:23.982573+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:23.982304+0000","last_became_peered":"2024-04-25T09:54:23.982304+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995558+0000","last_change":"2024-04-25T09:54:28.012195+0000","last_active":"2024-04-25T09:54:29.995558+0000","last_peered":"2024-04-25T09:54:29.995558+0000","last_clean":"2024-04-25T09:54:29.995558+0000","last_became_active":"2024-04-25T09:54:28.011667+0000","last_became_peered":"2024-04-25T09:54:28.011667+0000","last_unstale":"2024-04-25T09:54:29.995558+0000","last_undegraded":"2024-04-25T09:54:29.995558+0000","last_fullsized":"2024-04-25T09:54:29.995558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087654+0000","last_change":"2024-04-25T09:54:22.967092+0000","last_active":"2024-04-25T09:54:30.087654+0000","last_peered":"2024-04-25T09:54:30.087654+0000","last_clean":"2024-04-25T09:54:30.087654+0000","last_became_active":"2024-04-25T09:54:22.966690+0000","last_became_peered":"2024-04-25T09:54:22.966690+0000","last_unstale":"2024-04-25T09:54:30.087654+0000","last_undegraded":"2024-04-25T09:54:30.087654+0000","last_fullsized":"2024-04-25T09:54:30.087654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087289+0000","last_change":"2024-04-25T09:54:23.979036+0000","last_active":"2024-04-25T09:54:30.087289+0000","last_peered":"2024-04-25T09:54:30.087289+0000","last_clean":"2024-04-25T09:54:30.087289+0000","last_became_active":"2024-04-25T09:54:23.978775+0000","last_became_peered":"2024-04-25T09:54:23.978775+0000","last_unstale":"2024-04-25T09:54:30.087289+0000","last_undegraded":"2024-04-25T09:54:30.087289+0000","last_fullsized":"2024-04-25T09:54:30.087289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995398+0000","last_change":"2024-04-25T09:54:28.006708+0000","last_active":"2024-04-25T09:54:29.995398+0000","last_peered":"2024-04-25T09:54:29.995398+0000","last_clean":"2024-04-25T09:54:29.995398+0000","last_became_active":"2024-04-25T09:54:28.006351+0000","last_became_peered":"2024-04-25T09:54:28.006351+0000","last_unstale":"2024-04-25T09:54:29.995398+0000","last_undegraded":"2024-04-25T09:54:29.995398+0000","last_fullsized":"2024-04-25T09:54:29.995398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085947+0000","last_change":"2024-04-25T09:54:23.983452+0000","last_active":"2024-04-25T09:54:30.085947+0000","last_peered":"2024-04-25T09:54:30.085947+0000","last_clean":"2024-04-25T09:54:30.085947+0000","last_became_active":"2024-04-25T09:54:23.983150+0000","last_became_peered":"2024-04-25T09:54:23.983150+0000","last_unstale":"2024-04-25T09:54:30.085947+0000","last_undegraded":"2024-04-25T09:54:30.085947+0000","last_fullsized":"2024-04-25T09:54:30.085947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087835+0000","last_change":"2024-04-25T09:54:22.965628+0000","last_active":"2024-04-25T09:54:30.087835+0000","last_peered":"2024-04-25T09:54:30.087835+0000","last_clean":"2024-04-25T09:54:30.087835+0000","last_became_active":"2024-04-25T09:54:22.965294+0000","last_became_peered":"2024-04-25T09:54:22.965294+0000","last_unstale":"2024-04-25T09:54:30.087835+0000","last_undegraded":"2024-04-25T09:54:30.087835+0000","last_fullsized":"2024-04-25T09:54:30.087835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996313+0000","last_change":"2024-04-25T09:54:28.011196+0000","last_active":"2024-04-25T09:54:29.996313+0000","last_peered":"2024-04-25T09:54:29.996313+0000","last_clean":"2024-04-25T09:54:29.996313+0000","last_became_active":"2024-04-25T09:54:28.010638+0000","last_became_peered":"2024-04-25T09:54:28.010638+0000","last_unstale":"2024-04-25T09:54:29.996313+0000","last_undegraded":"2024-04-25T09:54:29.996313+0000","last_fullsized":"2024-04-25T09:54:29.996313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994468+0000","last_change":"2024-04-25T09:54:23.975061+0000","last_active":"2024-04-25T09:54:29.994468+0000","last_peered":"2024-04-25T09:54:29.994468+0000","last_clean":"2024-04-25T09:54:29.994468+0000","last_became_active":"2024-04-25T09:54:23.974793+0000","last_became_peered":"2024-04-25T09:54:23.974793+0000","last_unstale":"2024-04-25T09:54:29.994468+0000","last_undegraded":"2024-04-25T09:54:29.994468+0000","last_fullsized":"2024-04-25T09:54:29.994468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086013+0000","last_change":"2024-04-25T09:54:22.965353+0000","last_active":"2024-04-25T09:54:30.086013+0000","last_peered":"2024-04-25T09:54:30.086013+0000","last_clean":"2024-04-25T09:54:30.086013+0000","last_became_active":"2024-04-25T09:54:22.965038+0000","last_became_peered":"2024-04-25T09:54:22.965038+0000","last_unstale":"2024-04-25T09:54:30.086013+0000","last_undegraded":"2024-04-25T09:54:30.086013+0000","last_fullsized":"2024-04-25T09:54:30.086013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996196+0000","last_change":"2024-04-25T09:54:28.005662+0000","last_active":"2024-04-25T09:54:29.996196+0000","last_peered":"2024-04-25T09:54:29.996196+0000","last_clean":"2024-04-25T09:54:29.996196+0000","last_became_active":"2024-04-25T09:54:28.005233+0000","last_became_peered":"2024-04-25T09:54:28.005233+0000","last_unstale":"2024-04-25T09:54:29.996196+0000","last_undegraded":"2024-04-25T09:54:29.996196+0000","last_fullsized":"2024-04-25T09:54:29.996196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994816+0000","last_change":"2024-04-25T09:54:23.969844+0000","last_active":"2024-04-25T09:54:29.994816+0000","last_peered":"2024-04-25T09:54:29.994816+0000","last_clean":"2024-04-25T09:54:29.994816+0000","last_became_active":"2024-04-25T09:54:23.969261+0000","last_became_peered":"2024-04-25T09:54:23.969261+0000","last_unstale":"2024-04-25T09:54:29.994816+0000","last_undegraded":"2024-04-25T09:54:29.994816+0000","last_fullsized":"2024-04-25T09:54:29.994816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086366+0000","last_change":"2024-04-25T09:54:22.961764+0000","last_active":"2024-04-25T09:54:30.086366+0000","last_peered":"2024-04-25T09:54:30.086366+0000","last_clean":"2024-04-25T09:54:30.086366+0000","last_became_active":"2024-04-25T09:54:22.961455+0000","last_became_peered":"2024-04-25T09:54:22.961455+0000","last_unstale":"2024-04-25T09:54:30.086366+0000","last_undegraded":"2024-04-25T09:54:30.086366+0000","last_fullsized":"2024-04-25T09:54:30.086366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087099+0000","last_change":"2024-04-25T09:54:28.395240+0000","last_active":"2024-04-25T09:54:30.087099+0000","last_peered":"2024-04-25T09:54:30.087099+0000","last_clean":"2024-04-25T09:54:30.087099+0000","last_became_active":"2024-04-25T09:54:28.394866+0000","last_became_peered":"2024-04-25T09:54:28.394866+0000","last_unstale":"2024-04-25T09:54:30.087099+0000","last_undegraded":"2024-04-25T09:54:30.087099+0000","last_fullsized":"2024-04-25T09:54:30.087099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995771+0000","last_change":"2024-04-25T09:54:23.973436+0000","last_active":"2024-04-25T09:54:29.995771+0000","last_peered":"2024-04-25T09:54:29.995771+0000","last_clean":"2024-04-25T09:54:29.995771+0000","last_became_active":"2024-04-25T09:54:23.973139+0000","last_became_peered":"2024-04-25T09:54:23.973139+0000","last_unstale":"2024-04-25T09:54:29.995771+0000","last_undegraded":"2024-04-25T09:54:29.995771+0000","last_fullsized":"2024-04-25T09:54:29.995771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086953+0000","last_change":"2024-04-25T09:54:22.963445+0000","last_active":"2024-04-25T09:54:30.086953+0000","last_peered":"2024-04-25T09:54:30.086953+0000","last_clean":"2024-04-25T09:54:30.086953+0000","last_became_active":"2024-04-25T09:54:22.963190+0000","last_became_peered":"2024-04-25T09:54:22.963190+0000","last_unstale":"2024-04-25T09:54:30.086953+0000","last_undegraded":"2024-04-25T09:54:30.086953+0000","last_fullsized":"2024-04-25T09:54:30.086953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996305+0000","last_change":"2024-04-25T09:54:28.011236+0000","last_active":"2024-04-25T09:54:29.996305+0000","last_peered":"2024-04-25T09:54:29.996305+0000","last_clean":"2024-04-25T09:54:29.996305+0000","last_became_active":"2024-04-25T09:54:28.010633+0000","last_became_peered":"2024-04-25T09:54:28.010633+0000","last_unstale":"2024-04-25T09:54:29.996305+0000","last_undegraded":"2024-04-25T09:54:29.996305+0000","last_fullsized":"2024-04-25T09:54:29.996305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086754+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:30.086754+0000","last_peered":"2024-04-25T09:54:30.086754+0000","last_clean":"2024-04-25T09:54:30.086754+0000","last_became_active":"2024-04-25T09:54:23.970411+0000","last_became_peered":"2024-04-25T09:54:23.970411+0000","last_unstale":"2024-04-25T09:54:30.086754+0000","last_undegraded":"2024-04-25T09:54:30.086754+0000","last_fullsized":"2024-04-25T09:54:30.086754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:20.071529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086786+0000","last_change":"2024-04-25T09:54:22.961585+0000","last_active":"2024-04-25T09:54:30.086786+0000","last_peered":"2024-04-25T09:54:30.086786+0000","last_clean":"2024-04-25T09:54:30.086786+0000","last_became_active":"2024-04-25T09:54:22.961286+0000","last_became_peered":"2024-04-25T09:54:22.961286+0000","last_unstale":"2024-04-25T09:54:30.086786+0000","last_undegraded":"2024-04-25T09:54:30.086786+0000","last_fullsized":"2024-04-25T09:54:30.086786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995453+0000","last_change":"2024-04-25T09:54:28.396021+0000","last_active":"2024-04-25T09:54:29.995453+0000","last_peered":"2024-04-25T09:54:29.995453+0000","last_clean":"2024-04-25T09:54:29.995453+0000","last_became_active":"2024-04-25T09:54:28.394961+0000","last_became_peered":"2024-04-25T09:54:28.394961+0000","last_unstale":"2024-04-25T09:54:29.995453+0000","last_undegraded":"2024-04-25T09:54:29.995453+0000","last_fullsized":"2024-04-25T09:54:29.995453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995224+0000","last_change":"2024-04-25T09:54:23.981659+0000","last_active":"2024-04-25T09:54:29.995224+0000","last_peered":"2024-04-25T09:54:29.995224+0000","last_clean":"2024-04-25T09:54:29.995224+0000","last_became_active":"2024-04-25T09:54:23.981345+0000","last_became_peered":"2024-04-25T09:54:23.981345+0000","last_unstale":"2024-04-25T09:54:29.995224+0000","last_undegraded":"2024-04-25T09:54:29.995224+0000","last_fullsized":"2024-04-25T09:54:29.995224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086841+0000","last_change":"2024-04-25T09:54:22.961799+0000","last_active":"2024-04-25T09:54:30.086841+0000","last_peered":"2024-04-25T09:54:30.086841+0000","last_clean":"2024-04-25T09:54:30.086841+0000","last_became_active":"2024-04-25T09:54:22.961205+0000","last_became_peered":"2024-04-25T09:54:22.961205+0000","last_unstale":"2024-04-25T09:54:30.086841+0000","last_undegraded":"2024-04-25T09:54:30.086841+0000","last_fullsized":"2024-04-25T09:54:30.086841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994244+0000","last_change":"2024-04-25T09:54:28.397432+0000","last_active":"2024-04-25T09:54:29.994244+0000","last_peered":"2024-04-25T09:54:29.994244+0000","last_clean":"2024-04-25T09:54:29.994244+0000","last_became_active":"2024-04-25T09:54:28.396060+0000","last_became_peered":"2024-04-25T09:54:28.396060+0000","last_unstale":"2024-04-25T09:54:29.994244+0000","last_undegraded":"2024-04-25T09:54:29.994244+0000","last_fullsized":"2024-04-25T09:54:29.994244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:18.141331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086215+0000","last_change":"2024-04-25T09:54:23.964362+0000","last_active":"2024-04-25T09:54:30.086215+0000","last_peered":"2024-04-25T09:54:30.086215+0000","last_clean":"2024-04-25T09:54:30.086215+0000","last_became_active":"2024-04-25T09:54:23.964097+0000","last_became_peered":"2024-04-25T09:54:23.964097+0000","last_unstale":"2024-04-25T09:54:30.086215+0000","last_undegraded":"2024-04-25T09:54:30.086215+0000","last_fullsized":"2024-04-25T09:54:30.086215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088121+0000","last_change":"2024-04-25T09:54:22.967489+0000","last_active":"2024-04-25T09:54:30.088121+0000","last_peered":"2024-04-25T09:54:30.088121+0000","last_clean":"2024-04-25T09:54:30.088121+0000","last_became_active":"2024-04-25T09:54:22.967191+0000","last_became_peered":"2024-04-25T09:54:22.967191+0000","last_unstale":"2024-04-25T09:54:30.088121+0000","last_undegraded":"2024-04-25T09:54:30.088121+0000","last_fullsized":"2024-04-25T09:54:30.088121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086235+0000","last_change":"2024-04-25T09:54:28.394981+0000","last_active":"2024-04-25T09:54:30.086235+0000","last_peered":"2024-04-25T09:54:30.086235+0000","last_clean":"2024-04-25T09:54:30.086235+0000","last_became_active":"2024-04-25T09:54:28.394572+0000","last_became_peered":"2024-04-25T09:54:28.394572+0000","last_unstale":"2024-04-25T09:54:30.086235+0000","last_undegraded":"2024-04-25T09:54:30.086235+0000","last_fullsized":"2024-04-25T09:54:30.086235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.977167+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.976581+0000","last_became_peered":"2024-04-25T09:54:23.976581+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086309+0000","last_change":"2024-04-25T09:54:22.957000+0000","last_active":"2024-04-25T09:54:30.086309+0000","last_peered":"2024-04-25T09:54:30.086309+0000","last_clean":"2024-04-25T09:54:30.086309+0000","last_became_active":"2024-04-25T09:54:22.956737+0000","last_became_peered":"2024-04-25T09:54:22.956737+0000","last_unstale":"2024-04-25T09:54:30.086309+0000","last_undegraded":"2024-04-25T09:54:30.086309+0000","last_fullsized":"2024-04-25T09:54:30.086309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085588+0000","last_change":"2024-04-25T09:54:28.009926+0000","last_active":"2024-04-25T09:54:30.085588+0000","last_peered":"2024-04-25T09:54:30.085588+0000","last_clean":"2024-04-25T09:54:30.085588+0000","last_became_active":"2024-04-25T09:54:28.009486+0000","last_became_peered":"2024-04-25T09:54:28.009486+0000","last_unstale":"2024-04-25T09:54:30.085588+0000","last_undegraded":"2024-04-25T09:54:30.085588+0000","last_fullsized":"2024-04-25T09:54:30.085588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996132+0000","last_change":"2024-04-25T09:54:23.974850+0000","last_active":"2024-04-25T09:54:29.996132+0000","last_peered":"2024-04-25T09:54:29.996132+0000","last_clean":"2024-04-25T09:54:29.996132+0000","last_became_active":"2024-04-25T09:54:23.974014+0000","last_became_peered":"2024-04-25T09:54:23.974014+0000","last_unstale":"2024-04-25T09:54:29.996132+0000","last_undegraded":"2024-04-25T09:54:29.996132+0000","last_fullsized":"2024-04-25T09:54:29.996132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087242+0000","last_change":"2024-04-25T09:54:22.966563+0000","last_active":"2024-04-25T09:54:30.087242+0000","last_peered":"2024-04-25T09:54:30.087242+0000","last_clean":"2024-04-25T09:54:30.087242+0000","last_became_active":"2024-04-25T09:54:22.966272+0000","last_became_peered":"2024-04-25T09:54:22.966272+0000","last_unstale":"2024-04-25T09:54:30.087242+0000","last_undegraded":"2024-04-25T09:54:30.087242+0000","last_fullsized":"2024-04-25T09:54:30.087242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086121+0000","last_change":"2024-04-25T09:54:28.013312+0000","last_active":"2024-04-25T09:54:30.086121+0000","last_peered":"2024-04-25T09:54:30.086121+0000","last_clean":"2024-04-25T09:54:30.086121+0000","last_became_active":"2024-04-25T09:54:28.012953+0000","last_became_peered":"2024-04-25T09:54:28.012953+0000","last_unstale":"2024-04-25T09:54:30.086121+0000","last_undegraded":"2024-04-25T09:54:30.086121+0000","last_fullsized":"2024-04-25T09:54:30.086121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087003+0000","last_change":"2024-04-25T09:54:23.966466+0000","last_active":"2024-04-25T09:54:30.087003+0000","last_peered":"2024-04-25T09:54:30.087003+0000","last_clean":"2024-04-25T09:54:30.087003+0000","last_became_active":"2024-04-25T09:54:23.966129+0000","last_became_peered":"2024-04-25T09:54:23.966129+0000","last_unstale":"2024-04-25T09:54:30.087003+0000","last_undegraded":"2024-04-25T09:54:30.087003+0000","last_fullsized":"2024-04-25T09:54:30.087003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086581+0000","last_change":"2024-04-25T09:54:22.963150+0000","last_active":"2024-04-25T09:54:30.086581+0000","last_peered":"2024-04-25T09:54:30.086581+0000","last_clean":"2024-04-25T09:54:30.086581+0000","last_became_active":"2024-04-25T09:54:22.962832+0000","last_became_peered":"2024-04-25T09:54:22.962832+0000","last_unstale":"2024-04-25T09:54:30.086581+0000","last_undegraded":"2024-04-25T09:54:30.086581+0000","last_fullsized":"2024-04-25T09:54:30.086581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087126+0000","last_change":"2024-04-25T09:54:28.011051+0000","last_active":"2024-04-25T09:54:30.087126+0000","last_peered":"2024-04-25T09:54:30.087126+0000","last_clean":"2024-04-25T09:54:30.087126+0000","last_became_active":"2024-04-25T09:54:28.010629+0000","last_became_peered":"2024-04-25T09:54:28.010629+0000","last_unstale":"2024-04-25T09:54:30.087126+0000","last_undegraded":"2024-04-25T09:54:30.087126+0000","last_fullsized":"2024-04-25T09:54:30.087126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085547+0000","last_change":"2024-04-25T09:54:23.973638+0000","last_active":"2024-04-25T09:54:30.085547+0000","last_peered":"2024-04-25T09:54:30.085547+0000","last_clean":"2024-04-25T09:54:30.085547+0000","last_became_active":"2024-04-25T09:54:23.973349+0000","last_became_peered":"2024-04-25T09:54:23.973349+0000","last_unstale":"2024-04-25T09:54:30.085547+0000","last_undegraded":"2024-04-25T09:54:30.085547+0000","last_fullsized":"2024-04-25T09:54:30.085547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087072+0000","last_change":"2024-04-25T09:54:22.966560+0000","last_active":"2024-04-25T09:54:30.087072+0000","last_peered":"2024-04-25T09:54:30.087072+0000","last_clean":"2024-04-25T09:54:30.087072+0000","last_became_active":"2024-04-25T09:54:22.966299+0000","last_became_peered":"2024-04-25T09:54:22.966299+0000","last_unstale":"2024-04-25T09:54:30.087072+0000","last_undegraded":"2024-04-25T09:54:30.087072+0000","last_fullsized":"2024-04-25T09:54:30.087072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086483+0000","last_change":"2024-04-25T09:54:28.395179+0000","last_active":"2024-04-25T09:54:30.086483+0000","last_peered":"2024-04-25T09:54:30.086483+0000","last_clean":"2024-04-25T09:54:30.086483+0000","last_became_active":"2024-04-25T09:54:28.394767+0000","last_became_peered":"2024-04-25T09:54:28.394767+0000","last_unstale":"2024-04-25T09:54:30.086483+0000","last_undegraded":"2024-04-25T09:54:30.086483+0000","last_fullsized":"2024-04-25T09:54:30.086483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994411+0000","last_change":"2024-04-25T09:54:23.969956+0000","last_active":"2024-04-25T09:54:29.994411+0000","last_peered":"2024-04-25T09:54:29.994411+0000","last_clean":"2024-04-25T09:54:29.994411+0000","last_became_active":"2024-04-25T09:54:23.969643+0000","last_became_peered":"2024-04-25T09:54:23.969643+0000","last_unstale":"2024-04-25T09:54:29.994411+0000","last_undegraded":"2024-04-25T09:54:29.994411+0000","last_fullsized":"2024-04-25T09:54:29.994411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086619+0000","last_change":"2024-04-25T09:54:22.965368+0000","last_active":"2024-04-25T09:54:30.086619+0000","last_peered":"2024-04-25T09:54:30.086619+0000","last_clean":"2024-04-25T09:54:30.086619+0000","last_became_active":"2024-04-25T09:54:22.965051+0000","last_became_peered":"2024-04-25T09:54:22.965051+0000","last_unstale":"2024-04-25T09:54:30.086619+0000","last_undegraded":"2024-04-25T09:54:30.086619+0000","last_fullsized":"2024-04-25T09:54:30.086619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087177+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:54:30.087177+0000","last_peered":"2024-04-25T09:54:30.087177+0000","last_clean":"2024-04-25T09:54:30.087177+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:54:30.087177+0000","last_undegraded":"2024-04-25T09:54:30.087177+0000","last_fullsized":"2024-04-25T09:54:30.087177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085975+0000","last_change":"2024-04-25T09:54:28.396320+0000","last_active":"2024-04-25T09:54:30.085975+0000","last_peered":"2024-04-25T09:54:30.085975+0000","last_clean":"2024-04-25T09:54:30.085975+0000","last_became_active":"2024-04-25T09:54:28.395895+0000","last_became_peered":"2024-04-25T09:54:28.395895+0000","last_unstale":"2024-04-25T09:54:30.085975+0000","last_undegraded":"2024-04-25T09:54:30.085975+0000","last_fullsized":"2024-04-25T09:54:30.085975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087913+0000","last_change":"2024-04-25T09:54:23.983959+0000","last_active":"2024-04-25T09:54:30.087913+0000","last_peered":"2024-04-25T09:54:30.087913+0000","last_clean":"2024-04-25T09:54:30.087913+0000","last_became_active":"2024-04-25T09:54:23.983669+0000","last_became_peered":"2024-04-25T09:54:23.983669+0000","last_unstale":"2024-04-25T09:54:30.087913+0000","last_undegraded":"2024-04-25T09:54:30.087913+0000","last_fullsized":"2024-04-25T09:54:30.087913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087174+0000","last_change":"2024-04-25T09:54:22.966591+0000","last_active":"2024-04-25T09:54:30.087174+0000","last_peered":"2024-04-25T09:54:30.087174+0000","last_clean":"2024-04-25T09:54:30.087174+0000","last_became_active":"2024-04-25T09:54:22.966301+0000","last_became_peered":"2024-04-25T09:54:22.966301+0000","last_unstale":"2024-04-25T09:54:30.087174+0000","last_undegraded":"2024-04-25T09:54:30.087174+0000","last_fullsized":"2024-04-25T09:54:30.087174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085806+0000","last_change":"2024-04-25T09:54:28.395577+0000","last_active":"2024-04-25T09:54:30.085806+0000","last_peered":"2024-04-25T09:54:30.085806+0000","last_clean":"2024-04-25T09:54:30.085806+0000","last_became_active":"2024-04-25T09:54:28.395067+0000","last_became_peered":"2024-04-25T09:54:28.395067+0000","last_unstale":"2024-04-25T09:54:30.085806+0000","last_undegraded":"2024-04-25T09:54:30.085806+0000","last_fullsized":"2024-04-25T09:54:30.085806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086598+0000","last_change":"2024-04-25T09:54:23.975660+0000","last_active":"2024-04-25T09:54:30.086598+0000","last_peered":"2024-04-25T09:54:30.086598+0000","last_clean":"2024-04-25T09:54:30.086598+0000","last_became_active":"2024-04-25T09:54:23.975394+0000","last_became_peered":"2024-04-25T09:54:23.975394+0000","last_unstale":"2024-04-25T09:54:30.086598+0000","last_undegraded":"2024-04-25T09:54:30.086598+0000","last_fullsized":"2024-04-25T09:54:30.086598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087150+0000","last_change":"2024-04-25T09:54:22.959855+0000","last_active":"2024-04-25T09:54:30.087150+0000","last_peered":"2024-04-25T09:54:30.087150+0000","last_clean":"2024-04-25T09:54:30.087150+0000","last_became_active":"2024-04-25T09:54:22.959556+0000","last_became_peered":"2024-04-25T09:54:22.959556+0000","last_unstale":"2024-04-25T09:54:30.087150+0000","last_undegraded":"2024-04-25T09:54:30.087150+0000","last_fullsized":"2024-04-25T09:54:30.087150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086831+0000","last_change":"2024-04-25T09:54:28.010760+0000","last_active":"2024-04-25T09:54:30.086831+0000","last_peered":"2024-04-25T09:54:30.086831+0000","last_clean":"2024-04-25T09:54:30.086831+0000","last_became_active":"2024-04-25T09:54:28.010391+0000","last_became_peered":"2024-04-25T09:54:28.010391+0000","last_unstale":"2024-04-25T09:54:30.086831+0000","last_undegraded":"2024-04-25T09:54:30.086831+0000","last_fullsized":"2024-04-25T09:54:30.086831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088301+0000","last_change":"2024-04-25T09:54:23.982380+0000","last_active":"2024-04-25T09:54:30.088301+0000","last_peered":"2024-04-25T09:54:30.088301+0000","last_clean":"2024-04-25T09:54:30.088301+0000","last_became_active":"2024-04-25T09:54:23.982076+0000","last_became_peered":"2024-04-25T09:54:23.982076+0000","last_unstale":"2024-04-25T09:54:30.088301+0000","last_undegraded":"2024-04-25T09:54:30.088301+0000","last_fullsized":"2024-04-25T09:54:30.088301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086229+0000","last_change":"2024-04-25T09:54:22.963068+0000","last_active":"2024-04-25T09:54:30.086229+0000","last_peered":"2024-04-25T09:54:30.086229+0000","last_clean":"2024-04-25T09:54:30.086229+0000","last_became_active":"2024-04-25T09:54:22.962607+0000","last_became_peered":"2024-04-25T09:54:22.962607+0000","last_unstale":"2024-04-25T09:54:30.086229+0000","last_undegraded":"2024-04-25T09:54:30.086229+0000","last_fullsized":"2024-04-25T09:54:30.086229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:28.396123+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:28.395141+0000","last_became_peered":"2024-04-25T09:54:28.395141+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087759+0000","last_change":"2024-04-25T09:54:23.981922+0000","last_active":"2024-04-25T09:54:30.087759+0000","last_peered":"2024-04-25T09:54:30.087759+0000","last_clean":"2024-04-25T09:54:30.087759+0000","last_became_active":"2024-04-25T09:54:23.981624+0000","last_became_peered":"2024-04-25T09:54:23.981624+0000","last_unstale":"2024-04-25T09:54:30.087759+0000","last_undegraded":"2024-04-25T09:54:30.087759+0000","last_fullsized":"2024-04-25T09:54:30.087759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996103+0000","last_change":"2024-04-25T09:54:22.958111+0000","last_active":"2024-04-25T09:54:29.996103+0000","last_peered":"2024-04-25T09:54:29.996103+0000","last_clean":"2024-04-25T09:54:29.996103+0000","last_became_active":"2024-04-25T09:54:22.957822+0000","last_became_peered":"2024-04-25T09:54:22.957822+0000","last_unstale":"2024-04-25T09:54:29.996103+0000","last_undegraded":"2024-04-25T09:54:29.996103+0000","last_fullsized":"2024-04-25T09:54:29.996103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:05.312329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995523+0000","last_change":"2024-04-25T09:54:28.011885+0000","last_active":"2024-04-25T09:54:29.995523+0000","last_peered":"2024-04-25T09:54:29.995523+0000","last_clean":"2024-04-25T09:54:29.995523+0000","last_became_active":"2024-04-25T09:54:28.011498+0000","last_became_peered":"2024-04-25T09:54:28.011498+0000","last_unstale":"2024-04-25T09:54:29.995523+0000","last_undegraded":"2024-04-25T09:54:29.995523+0000","last_fullsized":"2024-04-25T09:54:29.995523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087609+0000","last_change":"2024-04-25T09:54:23.982839+0000","last_active":"2024-04-25T09:54:30.087609+0000","last_peered":"2024-04-25T09:54:30.087609+0000","last_clean":"2024-04-25T09:54:30.087609+0000","last_became_active":"2024-04-25T09:54:23.982542+0000","last_became_peered":"2024-04-25T09:54:23.982542+0000","last_unstale":"2024-04-25T09:54:30.087609+0000","last_undegraded":"2024-04-25T09:54:30.087609+0000","last_fullsized":"2024-04-25T09:54:30.087609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085485+0000","last_change":"2024-04-25T09:54:22.954113+0000","last_active":"2024-04-25T09:54:30.085485+0000","last_peered":"2024-04-25T09:54:30.085485+0000","last_clean":"2024-04-25T09:54:30.085485+0000","last_became_active":"2024-04-25T09:54:22.953819+0000","last_became_peered":"2024-04-25T09:54:22.953819+0000","last_unstale":"2024-04-25T09:54:30.085485+0000","last_undegraded":"2024-04-25T09:54:30.085485+0000","last_fullsized":"2024-04-25T09:54:30.085485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086548+0000","last_change":"2024-04-25T09:54:28.008178+0000","last_active":"2024-04-25T09:54:30.086548+0000","last_peered":"2024-04-25T09:54:30.086548+0000","last_clean":"2024-04-25T09:54:30.086548+0000","last_became_active":"2024-04-25T09:54:28.007737+0000","last_became_peered":"2024-04-25T09:54:28.007737+0000","last_unstale":"2024-04-25T09:54:30.086548+0000","last_undegraded":"2024-04-25T09:54:30.086548+0000","last_fullsized":"2024-04-25T09:54:30.086548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085923+0000","last_change":"2024-04-25T09:54:23.977488+0000","last_active":"2024-04-25T09:54:30.085923+0000","last_peered":"2024-04-25T09:54:30.085923+0000","last_clean":"2024-04-25T09:54:30.085923+0000","last_became_active":"2024-04-25T09:54:23.977190+0000","last_became_peered":"2024-04-25T09:54:23.977190+0000","last_unstale":"2024-04-25T09:54:30.085923+0000","last_undegraded":"2024-04-25T09:54:30.085923+0000","last_fullsized":"2024-04-25T09:54:30.085923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086516+0000","last_change":"2024-04-25T09:54:22.959330+0000","last_active":"2024-04-25T09:54:30.086516+0000","last_peered":"2024-04-25T09:54:30.086516+0000","last_clean":"2024-04-25T09:54:30.086516+0000","last_became_active":"2024-04-25T09:54:22.959026+0000","last_became_peered":"2024-04-25T09:54:22.959026+0000","last_unstale":"2024-04-25T09:54:30.086516+0000","last_undegraded":"2024-04-25T09:54:30.086516+0000","last_fullsized":"2024-04-25T09:54:30.086516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995258+0000","last_change":"2024-04-25T09:54:28.396288+0000","last_active":"2024-04-25T09:54:29.995258+0000","last_peered":"2024-04-25T09:54:29.995258+0000","last_clean":"2024-04-25T09:54:29.995258+0000","last_became_active":"2024-04-25T09:54:28.395940+0000","last_became_peered":"2024-04-25T09:54:28.395940+0000","last_unstale":"2024-04-25T09:54:29.995258+0000","last_undegraded":"2024-04-25T09:54:29.995258+0000","last_fullsized":"2024-04-25T09:54:29.995258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:05.296954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087789+0000","last_change":"2024-04-25T09:54:23.982575+0000","last_active":"2024-04-25T09:54:30.087789+0000","last_peered":"2024-04-25T09:54:30.087789+0000","last_clean":"2024-04-25T09:54:30.087789+0000","last_became_active":"2024-04-25T09:54:23.982274+0000","last_became_peered":"2024-04-25T09:54:23.982274+0000","last_unstale":"2024-04-25T09:54:30.087789+0000","last_undegraded":"2024-04-25T09:54:30.087789+0000","last_fullsized":"2024-04-25T09:54:30.087789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087152+0000","last_change":"2024-04-25T09:54:22.962975+0000","last_active":"2024-04-25T09:54:30.087152+0000","last_peered":"2024-04-25T09:54:30.087152+0000","last_clean":"2024-04-25T09:54:30.087152+0000","last_became_active":"2024-04-25T09:54:22.962642+0000","last_became_peered":"2024-04-25T09:54:22.962642+0000","last_unstale":"2024-04-25T09:54:30.087152+0000","last_undegraded":"2024-04-25T09:54:30.087152+0000","last_fullsized":"2024-04-25T09:54:30.087152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994645+0000","last_change":"2024-04-25T09:54:28.395146+0000","last_active":"2024-04-25T09:54:29.994645+0000","last_peered":"2024-04-25T09:54:29.994645+0000","last_clean":"2024-04-25T09:54:29.994645+0000","last_became_active":"2024-04-25T09:54:28.394756+0000","last_became_peered":"2024-04-25T09:54:28.394756+0000","last_unstale":"2024-04-25T09:54:29.994645+0000","last_undegraded":"2024-04-25T09:54:29.994645+0000","last_fullsized":"2024-04-25T09:54:29.994645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:05.993975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086033+0000","last_change":"2024-04-25T09:54:23.972232+0000","last_active":"2024-04-25T09:54:30.086033+0000","last_peered":"2024-04-25T09:54:30.086033+0000","last_clean":"2024-04-25T09:54:30.086033+0000","last_became_active":"2024-04-25T09:54:23.971948+0000","last_became_peered":"2024-04-25T09:54:23.971948+0000","last_unstale":"2024-04-25T09:54:30.086033+0000","last_undegraded":"2024-04-25T09:54:30.086033+0000","last_fullsized":"2024-04-25T09:54:30.086033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:10.704903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086681+0000","last_change":"2024-04-25T09:54:22.957583+0000","last_active":"2024-04-25T09:54:30.086681+0000","last_peered":"2024-04-25T09:54:30.086681+0000","last_clean":"2024-04-25T09:54:30.086681+0000","last_became_active":"2024-04-25T09:54:22.957190+0000","last_became_peered":"2024-04-25T09:54:22.957190+0000","last_unstale":"2024-04-25T09:54:30.086681+0000","last_undegraded":"2024-04-25T09:54:30.086681+0000","last_fullsized":"2024-04-25T09:54:30.086681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:21.147671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087171+0000","last_change":"2024-04-25T09:54:28.395989+0000","last_active":"2024-04-25T09:54:30.087171+0000","last_peered":"2024-04-25T09:54:30.087171+0000","last_clean":"2024-04-25T09:54:30.087171+0000","last_became_active":"2024-04-25T09:54:28.395578+0000","last_became_peered":"2024-04-25T09:54:28.395578+0000","last_unstale":"2024-04-25T09:54:30.087171+0000","last_undegraded":"2024-04-25T09:54:30.087171+0000","last_fullsized":"2024-04-25T09:54:30.087171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087863+0000","last_change":"2024-04-25T09:54:23.984390+0000","last_active":"2024-04-25T09:54:30.087863+0000","last_peered":"2024-04-25T09:54:30.087863+0000","last_clean":"2024-04-25T09:54:30.087863+0000","last_became_active":"2024-04-25T09:54:23.981528+0000","last_became_peered":"2024-04-25T09:54:23.981528+0000","last_unstale":"2024-04-25T09:54:30.087863+0000","last_undegraded":"2024-04-25T09:54:30.087863+0000","last_fullsized":"2024-04-25T09:54:30.087863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:33.133289+0000","last_change":"2024-04-25T09:54:22.962540+0000","last_active":"2024-04-25T09:54:33.133289+0000","last_peered":"2024-04-25T09:54:33.133289+0000","last_clean":"2024-04-25T09:54:33.133289+0000","last_became_active":"2024-04-25T09:54:22.962223+0000","last_became_peered":"2024-04-25T09:54:22.962223+0000","last_unstale":"2024-04-25T09:54:33.133289+0000","last_undegraded":"2024-04-25T09:54:33.133289+0000","last_fullsized":"2024-04-25T09:54:33.133289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085807+0000","last_change":"2024-04-25T09:54:28.012670+0000","last_active":"2024-04-25T09:54:30.085807+0000","last_peered":"2024-04-25T09:54:30.085807+0000","last_clean":"2024-04-25T09:54:30.085807+0000","last_became_active":"2024-04-25T09:54:28.011580+0000","last_became_peered":"2024-04-25T09:54:28.011580+0000","last_unstale":"2024-04-25T09:54:30.085807+0000","last_undegraded":"2024-04-25T09:54:30.085807+0000","last_fullsized":"2024-04-25T09:54:30.085807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996273+0000","last_change":"2024-04-25T09:54:23.981044+0000","last_active":"2024-04-25T09:54:29.996273+0000","last_peered":"2024-04-25T09:54:29.996273+0000","last_clean":"2024-04-25T09:54:29.996273+0000","last_became_active":"2024-04-25T09:54:23.980733+0000","last_became_peered":"2024-04-25T09:54:23.980733+0000","last_unstale":"2024-04-25T09:54:29.996273+0000","last_undegraded":"2024-04-25T09:54:29.996273+0000","last_fullsized":"2024-04-25T09:54:29.996273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087059+0000","last_change":"2024-04-25T09:54:22.965478+0000","last_active":"2024-04-25T09:54:30.087059+0000","last_peered":"2024-04-25T09:54:30.087059+0000","last_clean":"2024-04-25T09:54:30.087059+0000","last_became_active":"2024-04-25T09:54:22.965191+0000","last_became_peered":"2024-04-25T09:54:22.965191+0000","last_unstale":"2024-04-25T09:54:30.087059+0000","last_undegraded":"2024-04-25T09:54:30.087059+0000","last_fullsized":"2024-04-25T09:54:30.087059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995924+0000","last_change":"2024-04-25T09:54:28.003916+0000","last_active":"2024-04-25T09:54:29.995924+0000","last_peered":"2024-04-25T09:54:29.995924+0000","last_clean":"2024-04-25T09:54:29.995924+0000","last_became_active":"2024-04-25T09:54:28.003512+0000","last_became_peered":"2024-04-25T09:54:28.003512+0000","last_unstale":"2024-04-25T09:54:29.995924+0000","last_undegraded":"2024-04-25T09:54:29.995924+0000","last_fullsized":"2024-04-25T09:54:29.995924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086045+0000","last_change":"2024-04-25T09:54:23.969795+0000","last_active":"2024-04-25T09:54:30.086045+0000","last_peered":"2024-04-25T09:54:30.086045+0000","last_clean":"2024-04-25T09:54:30.086045+0000","last_became_active":"2024-04-25T09:54:23.969543+0000","last_became_peered":"2024-04-25T09:54:23.969543+0000","last_unstale":"2024-04-25T09:54:30.086045+0000","last_undegraded":"2024-04-25T09:54:30.086045+0000","last_fullsized":"2024-04-25T09:54:30.086045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086869+0000","last_change":"2024-04-25T09:54:22.965431+0000","last_active":"2024-04-25T09:54:30.086869+0000","last_peered":"2024-04-25T09:54:30.086869+0000","last_clean":"2024-04-25T09:54:30.086869+0000","last_became_active":"2024-04-25T09:54:22.965115+0000","last_became_peered":"2024-04-25T09:54:22.965115+0000","last_unstale":"2024-04-25T09:54:30.086869+0000","last_undegraded":"2024-04-25T09:54:30.086869+0000","last_fullsized":"2024-04-25T09:54:30.086869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994658+0000","last_change":"2024-04-25T09:54:28.011370+0000","last_active":"2024-04-25T09:54:29.994658+0000","last_peered":"2024-04-25T09:54:29.994658+0000","last_clean":"2024-04-25T09:54:29.994658+0000","last_became_active":"2024-04-25T09:54:28.011006+0000","last_became_peered":"2024-04-25T09:54:28.011006+0000","last_unstale":"2024-04-25T09:54:29.994658+0000","last_undegraded":"2024-04-25T09:54:29.994658+0000","last_fullsized":"2024-04-25T09:54:29.994658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:05.906706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086261+0000","last_change":"2024-04-25T09:54:23.983800+0000","last_active":"2024-04-25T09:54:30.086261+0000","last_peered":"2024-04-25T09:54:30.086261+0000","last_clean":"2024-04-25T09:54:30.086261+0000","last_became_active":"2024-04-25T09:54:23.983518+0000","last_became_peered":"2024-04-25T09:54:23.983518+0000","last_unstale":"2024-04-25T09:54:30.086261+0000","last_undegraded":"2024-04-25T09:54:30.086261+0000","last_fullsized":"2024-04-25T09:54:30.086261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086666+0000","last_change":"2024-04-25T09:54:22.966898+0000","last_active":"2024-04-25T09:54:30.086666+0000","last_peered":"2024-04-25T09:54:30.086666+0000","last_clean":"2024-04-25T09:54:30.086666+0000","last_became_active":"2024-04-25T09:54:22.966638+0000","last_became_peered":"2024-04-25T09:54:22.966638+0000","last_unstale":"2024-04-25T09:54:30.086666+0000","last_undegraded":"2024-04-25T09:54:30.086666+0000","last_fullsized":"2024-04-25T09:54:30.086666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994713+0000","last_change":"2024-04-25T09:54:28.397771+0000","last_active":"2024-04-25T09:54:29.994713+0000","last_peered":"2024-04-25T09:54:29.994713+0000","last_clean":"2024-04-25T09:54:29.994713+0000","last_became_active":"2024-04-25T09:54:28.397413+0000","last_became_peered":"2024-04-25T09:54:28.397413+0000","last_unstale":"2024-04-25T09:54:29.994713+0000","last_undegraded":"2024-04-25T09:54:29.994713+0000","last_fullsized":"2024-04-25T09:54:29.994713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:07.468831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086335+0000","last_change":"2024-04-25T09:54:28.396249+0000","last_active":"2024-04-25T09:54:30.086335+0000","last_peered":"2024-04-25T09:54:30.086335+0000","last_clean":"2024-04-25T09:54:30.086335+0000","last_became_active":"2024-04-25T09:54:28.395904+0000","last_became_peered":"2024-04-25T09:54:28.395904+0000","last_unstale":"2024-04-25T09:54:30.086335+0000","last_undegraded":"2024-04-25T09:54:30.086335+0000","last_fullsized":"2024-04-25T09:54:30.086335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086193+0000","last_change":"2024-04-25T09:54:23.972012+0000","last_active":"2024-04-25T09:54:30.086193+0000","last_peered":"2024-04-25T09:54:30.086193+0000","last_clean":"2024-04-25T09:54:30.086193+0000","last_became_active":"2024-04-25T09:54:23.971731+0000","last_became_peered":"2024-04-25T09:54:23.971731+0000","last_unstale":"2024-04-25T09:54:30.086193+0000","last_undegraded":"2024-04-25T09:54:30.086193+0000","last_fullsized":"2024-04-25T09:54:30.086193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086458+0000","last_change":"2024-04-25T09:54:22.965400+0000","last_active":"2024-04-25T09:54:30.086458+0000","last_peered":"2024-04-25T09:54:30.086458+0000","last_clean":"2024-04-25T09:54:30.086458+0000","last_became_active":"2024-04-25T09:54:22.965099+0000","last_became_peered":"2024-04-25T09:54:22.965099+0000","last_unstale":"2024-04-25T09:54:30.086458+0000","last_undegraded":"2024-04-25T09:54:30.086458+0000","last_fullsized":"2024-04-25T09:54:30.086458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086552+0000","last_change":"2024-04-25T09:54:28.010667+0000","last_active":"2024-04-25T09:54:30.086552+0000","last_peered":"2024-04-25T09:54:30.086552+0000","last_clean":"2024-04-25T09:54:30.086552+0000","last_became_active":"2024-04-25T09:54:28.010243+0000","last_became_peered":"2024-04-25T09:54:28.010243+0000","last_unstale":"2024-04-25T09:54:30.086552+0000","last_undegraded":"2024-04-25T09:54:30.086552+0000","last_fullsized":"2024-04-25T09:54:30.086552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994577+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:29.994577+0000","last_peered":"2024-04-25T09:54:29.994577+0000","last_clean":"2024-04-25T09:54:29.994577+0000","last_became_active":"2024-04-25T09:54:23.970422+0000","last_became_peered":"2024-04-25T09:54:23.970422+0000","last_unstale":"2024-04-25T09:54:29.994577+0000","last_undegraded":"2024-04-25T09:54:29.994577+0000","last_fullsized":"2024-04-25T09:54:29.994577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994799+0000","last_change":"2024-04-25T09:54:22.960394+0000","last_active":"2024-04-25T09:54:29.994799+0000","last_peered":"2024-04-25T09:54:29.994799+0000","last_clean":"2024-04-25T09:54:29.994799+0000","last_became_active":"2024-04-25T09:54:22.960128+0000","last_became_peered":"2024-04-25T09:54:22.960128+0000","last_unstale":"2024-04-25T09:54:29.994799+0000","last_undegraded":"2024-04-25T09:54:29.994799+0000","last_fullsized":"2024-04-25T09:54:29.994799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086079+0000","last_change":"2024-04-25T09:54:22.962677+0000","last_active":"2024-04-25T09:54:30.086079+0000","last_peered":"2024-04-25T09:54:30.086079+0000","last_clean":"2024-04-25T09:54:30.086079+0000","last_became_active":"2024-04-25T09:54:22.962351+0000","last_became_peered":"2024-04-25T09:54:22.962351+0000","last_unstale":"2024-04-25T09:54:30.086079+0000","last_undegraded":"2024-04-25T09:54:30.086079+0000","last_fullsized":"2024-04-25T09:54:30.086079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086533+0000","last_change":"2024-04-25T09:54:23.973688+0000","last_active":"2024-04-25T09:54:30.086533+0000","last_peered":"2024-04-25T09:54:30.086533+0000","last_clean":"2024-04-25T09:54:30.086533+0000","last_became_active":"2024-04-25T09:54:23.973424+0000","last_became_peered":"2024-04-25T09:54:23.973424+0000","last_unstale":"2024-04-25T09:54:30.086533+0000","last_undegraded":"2024-04-25T09:54:30.086533+0000","last_fullsized":"2024-04-25T09:54:30.086533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086930+0000","last_change":"2024-04-25T09:54:28.016164+0000","last_active":"2024-04-25T09:54:30.086930+0000","last_peered":"2024-04-25T09:54:30.086930+0000","last_clean":"2024-04-25T09:54:30.086930+0000","last_became_active":"2024-04-25T09:54:28.015634+0000","last_became_peered":"2024-04-25T09:54:28.015634+0000","last_unstale":"2024-04-25T09:54:30.086930+0000","last_undegraded":"2024-04-25T09:54:30.086930+0000","last_fullsized":"2024-04-25T09:54:30.086930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.111330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994585+0000","last_change":"2024-04-25T09:54:22.962095+0000","last_active":"2024-04-25T09:54:29.994585+0000","last_peered":"2024-04-25T09:54:29.994585+0000","last_clean":"2024-04-25T09:54:29.994585+0000","last_became_active":"2024-04-25T09:54:22.961801+0000","last_became_peered":"2024-04-25T09:54:22.961801+0000","last_unstale":"2024-04-25T09:54:29.994585+0000","last_undegraded":"2024-04-25T09:54:29.994585+0000","last_fullsized":"2024-04-25T09:54:29.994585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995467+0000","last_change":"2024-04-25T09:54:23.971722+0000","last_active":"2024-04-25T09:54:29.995467+0000","last_peered":"2024-04-25T09:54:29.995467+0000","last_clean":"2024-04-25T09:54:29.995467+0000","last_became_active":"2024-04-25T09:54:23.971437+0000","last_became_peered":"2024-04-25T09:54:23.971437+0000","last_unstale":"2024-04-25T09:54:29.995467+0000","last_undegraded":"2024-04-25T09:54:29.995467+0000","last_fullsized":"2024-04-25T09:54:29.995467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:28.933707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994926+0000","last_change":"2024-04-25T09:54:28.016742+0000","last_active":"2024-04-25T09:54:29.994926+0000","last_peered":"2024-04-25T09:54:29.994926+0000","last_clean":"2024-04-25T09:54:29.994926+0000","last_became_active":"2024-04-25T09:54:28.016017+0000","last_became_peered":"2024-04-25T09:54:28.016017+0000","last_unstale":"2024-04-25T09:54:29.994926+0000","last_undegraded":"2024-04-25T09:54:29.994926+0000","last_fullsized":"2024-04-25T09:54:29.994926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:51.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085710+0000","last_change":"2024-04-25T09:54:23.976967+0000","last_active":"2024-04-25T09:54:30.085710+0000","last_peered":"2024-04-25T09:54:30.085710+0000","last_clean":"2024-04-25T09:54:30.085710+0000","last_became_active":"2024-04-25T09:54:23.976684+0000","last_became_peered":"2024-04-25T09:54:23.976684+0000","last_unstale":"2024-04-25T09:54:30.085710+0000","last_undegraded":"2024-04-25T09:54:30.085710+0000","last_fullsized":"2024-04-25T09:54:30.085710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086490+0000","last_change":"2024-04-25T09:54:23.968076+0000","last_active":"2024-04-25T09:54:30.086490+0000","last_peered":"2024-04-25T09:54:30.086490+0000","last_clean":"2024-04-25T09:54:30.086490+0000","last_became_active":"2024-04-25T09:54:23.967823+0000","last_became_peered":"2024-04-25T09:54:23.967823+0000","last_unstale":"2024-04-25T09:54:30.086490+0000","last_undegraded":"2024-04-25T09:54:30.086490+0000","last_fullsized":"2024-04-25T09:54:30.086490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087064+0000","last_change":"2024-04-25T09:54:23.974924+0000","last_active":"2024-04-25T09:54:30.087064+0000","last_peered":"2024-04-25T09:54:30.087064+0000","last_clean":"2024-04-25T09:54:30.087064+0000","last_became_active":"2024-04-25T09:54:23.974647+0000","last_became_peered":"2024-04-25T09:54:23.974647+0000","last_unstale":"2024-04-25T09:54:30.087064+0000","last_undegraded":"2024-04-25T09:54:30.087064+0000","last_fullsized":"2024-04-25T09:54:30.087064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086431+0000","last_change":"2024-04-25T09:54:23.974599+0000","last_active":"2024-04-25T09:54:30.086431+0000","last_peered":"2024-04-25T09:54:30.086431+0000","last_clean":"2024-04-25T09:54:30.086431+0000","last_became_active":"2024-04-25T09:54:23.973958+0000","last_became_peered":"2024-04-25T09:54:23.973958+0000","last_unstale":"2024-04-25T09:54:30.086431+0000","last_undegraded":"2024-04-25T09:54:30.086431+0000","last_fullsized":"2024-04-25T09:54:30.086431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994701+0000","last_change":"2024-04-25T09:54:23.976111+0000","last_active":"2024-04-25T09:54:29.994701+0000","last_peered":"2024-04-25T09:54:29.994701+0000","last_clean":"2024-04-25T09:54:29.994701+0000","last_became_active":"2024-04-25T09:54:23.975817+0000","last_became_peered":"2024-04-25T09:54:23.975817+0000","last_unstale":"2024-04-25T09:54:29.994701+0000","last_undegraded":"2024-04-25T09:54:29.994701+0000","last_fullsized":"2024-04-25T09:54:29.994701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086971+0000","last_change":"2024-04-25T09:54:23.967777+0000","last_active":"2024-04-25T09:54:30.086971+0000","last_peered":"2024-04-25T09:54:30.086971+0000","last_clean":"2024-04-25T09:54:30.086971+0000","last_became_active":"2024-04-25T09:54:23.967099+0000","last_became_peered":"2024-04-25T09:54:23.967099+0000","last_unstale":"2024-04-25T09:54:30.086971+0000","last_undegraded":"2024-04-25T09:54:30.086971+0000","last_fullsized":"2024-04-25T09:54:30.086971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086631+0000","last_change":"2024-04-25T09:54:23.982361+0000","last_active":"2024-04-25T09:54:30.086631+0000","last_peered":"2024-04-25T09:54:30.086631+0000","last_clean":"2024-04-25T09:54:30.086631+0000","last_became_active":"2024-04-25T09:54:23.982020+0000","last_became_peered":"2024-04-25T09:54:23.982020+0000","last_unstale":"2024-04-25T09:54:30.086631+0000","last_undegraded":"2024-04-25T09:54:30.086631+0000","last_fullsized":"2024-04-25T09:54:30.086631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995121+0000","last_change":"2024-04-25T09:54:23.976994+0000","last_active":"2024-04-25T09:54:29.995121+0000","last_peered":"2024-04-25T09:54:29.995121+0000","last_clean":"2024-04-25T09:54:29.995121+0000","last_became_active":"2024-04-25T09:54:23.976707+0000","last_became_peered":"2024-04-25T09:54:23.976707+0000","last_unstale":"2024-04-25T09:54:29.995121+0000","last_undegraded":"2024-04-25T09:54:29.995121+0000","last_fullsized":"2024-04-25T09:54:29.995121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086418+0000","last_change":"2024-04-25T09:54:23.977123+0000","last_active":"2024-04-25T09:54:30.086418+0000","last_peered":"2024-04-25T09:54:30.086418+0000","last_clean":"2024-04-25T09:54:30.086418+0000","last_became_active":"2024-04-25T09:54:23.976810+0000","last_became_peered":"2024-04-25T09:54:23.976810+0000","last_unstale":"2024-04-25T09:54:30.086418+0000","last_undegraded":"2024-04-25T09:54:30.086418+0000","last_fullsized":"2024-04-25T09:54:30.086418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994535+0000","last_change":"2024-04-25T09:54:23.976035+0000","last_active":"2024-04-25T09:54:29.994535+0000","last_peered":"2024-04-25T09:54:29.994535+0000","last_clean":"2024-04-25T09:54:29.994535+0000","last_became_active":"2024-04-25T09:54:23.975755+0000","last_became_peered":"2024-04-25T09:54:23.975755+0000","last_unstale":"2024-04-25T09:54:29.994535+0000","last_undegraded":"2024-04-25T09:54:29.994535+0000","last_fullsized":"2024-04-25T09:54:29.994535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086969+0000","last_change":"2024-04-25T09:54:23.973340+0000","last_active":"2024-04-25T09:54:30.086969+0000","last_peered":"2024-04-25T09:54:30.086969+0000","last_clean":"2024-04-25T09:54:30.086969+0000","last_became_active":"2024-04-25T09:54:23.973010+0000","last_became_peered":"2024-04-25T09:54:23.973010+0000","last_unstale":"2024-04-25T09:54:30.086969+0000","last_undegraded":"2024-04-25T09:54:30.086969+0000","last_fullsized":"2024-04-25T09:54:30.086969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086318+0000","last_change":"2024-04-25T09:54:23.980705+0000","last_active":"2024-04-25T09:54:30.086318+0000","last_peered":"2024-04-25T09:54:30.086318+0000","last_clean":"2024-04-25T09:54:30.086318+0000","last_became_active":"2024-04-25T09:54:23.980411+0000","last_became_peered":"2024-04-25T09:54:23.980411+0000","last_unstale":"2024-04-25T09:54:30.086318+0000","last_undegraded":"2024-04-25T09:54:30.086318+0000","last_fullsized":"2024-04-25T09:54:30.086318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086259+0000","last_change":"2024-04-25T09:54:23.984091+0000","last_active":"2024-04-25T09:54:30.086259+0000","last_peered":"2024-04-25T09:54:30.086259+0000","last_clean":"2024-04-25T09:54:30.086259+0000","last_became_active":"2024-04-25T09:54:23.983834+0000","last_became_peered":"2024-04-25T09:54:23.983834+0000","last_unstale":"2024-04-25T09:54:30.086259+0000","last_undegraded":"2024-04-25T09:54:30.086259+0000","last_fullsized":"2024-04-25T09:54:30.086259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088036+0000","last_change":"2024-04-25T09:54:23.983353+0000","last_active":"2024-04-25T09:54:30.088036+0000","last_peered":"2024-04-25T09:54:30.088036+0000","last_clean":"2024-04-25T09:54:30.088036+0000","last_became_active":"2024-04-25T09:54:23.983043+0000","last_became_peered":"2024-04-25T09:54:23.983043+0000","last_unstale":"2024-04-25T09:54:30.088036+0000","last_undegraded":"2024-04-25T09:54:30.088036+0000","last_fullsized":"2024-04-25T09:54:30.088036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086130+0000","last_change":"2024-04-25T09:54:23.977174+0000","last_active":"2024-04-25T09:54:30.086130+0000","last_peered":"2024-04-25T09:54:30.086130+0000","last_clean":"2024-04-25T09:54:30.086130+0000","last_became_active":"2024-04-25T09:54:23.976903+0000","last_became_peered":"2024-04-25T09:54:23.976903+0000","last_unstale":"2024-04-25T09:54:30.086130+0000","last_undegraded":"2024-04-25T09:54:30.086130+0000","last_fullsized":"2024-04-25T09:54:30.086130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.972492+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.972188+0000","last_became_peered":"2024-04-25T09:54:23.972188+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086368+0000","last_change":"2024-04-25T09:54:23.978371+0000","last_active":"2024-04-25T09:54:30.086368+0000","last_peered":"2024-04-25T09:54:30.086368+0000","last_clean":"2024-04-25T09:54:30.086368+0000","last_became_active":"2024-04-25T09:54:23.978079+0000","last_became_peered":"2024-04-25T09:54:23.978079+0000","last_unstale":"2024-04-25T09:54:30.086368+0000","last_undegraded":"2024-04-25T09:54:30.086368+0000","last_fullsized":"2024-04-25T09:54:30.086368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086280+0000","last_change":"2024-04-25T09:54:23.976031+0000","last_active":"2024-04-25T09:54:30.086280+0000","last_peered":"2024-04-25T09:54:30.086280+0000","last_clean":"2024-04-25T09:54:30.086280+0000","last_became_active":"2024-04-25T09:54:23.975750+0000","last_became_peered":"2024-04-25T09:54:23.975750+0000","last_unstale":"2024-04-25T09:54:30.086280+0000","last_undegraded":"2024-04-25T09:54:30.086280+0000","last_fullsized":"2024-04-25T09:54:30.086280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085955+0000","last_change":"2024-04-25T09:54:23.973561+0000","last_active":"2024-04-25T09:54:30.085955+0000","last_peered":"2024-04-25T09:54:30.085955+0000","last_clean":"2024-04-25T09:54:30.085955+0000","last_became_active":"2024-04-25T09:54:23.973312+0000","last_became_peered":"2024-04-25T09:54:23.973312+0000","last_unstale":"2024-04-25T09:54:30.085955+0000","last_undegraded":"2024-04-25T09:54:30.085955+0000","last_fullsized":"2024-04-25T09:54:30.085955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994677+0000","last_change":"2024-04-25T09:54:23.969586+0000","last_active":"2024-04-25T09:54:29.994677+0000","last_peered":"2024-04-25T09:54:29.994677+0000","last_clean":"2024-04-25T09:54:29.994677+0000","last_became_active":"2024-04-25T09:54:23.969067+0000","last_became_peered":"2024-04-25T09:54:23.969067+0000","last_unstale":"2024-04-25T09:54:29.994677+0000","last_undegraded":"2024-04-25T09:54:29.994677+0000","last_fullsized":"2024-04-25T09:54:29.994677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995863+0000","last_change":"2024-04-25T09:54:23.972621+0000","last_active":"2024-04-25T09:54:29.995863+0000","last_peered":"2024-04-25T09:54:29.995863+0000","last_clean":"2024-04-25T09:54:29.995863+0000","last_became_active":"2024-04-25T09:54:23.972338+0000","last_became_peered":"2024-04-25T09:54:23.972338+0000","last_unstale":"2024-04-25T09:54:29.995863+0000","last_undegraded":"2024-04-25T09:54:29.995863+0000","last_fullsized":"2024-04-25T09:54:29.995863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086120+0000","last_change":"2024-04-25T09:54:23.971085+0000","last_active":"2024-04-25T09:54:30.086120+0000","last_peered":"2024-04-25T09:54:30.086120+0000","last_clean":"2024-04-25T09:54:30.086120+0000","last_became_active":"2024-04-25T09:54:23.970354+0000","last_became_peered":"2024-04-25T09:54:23.970354+0000","last_unstale":"2024-04-25T09:54:30.086120+0000","last_undegraded":"2024-04-25T09:54:30.086120+0000","last_fullsized":"2024-04-25T09:54:30.086120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086765+0000","last_change":"2024-04-25T09:54:23.970478+0000","last_active":"2024-04-25T09:54:30.086765+0000","last_peered":"2024-04-25T09:54:30.086765+0000","last_clean":"2024-04-25T09:54:30.086765+0000","last_became_active":"2024-04-25T09:54:23.970181+0000","last_became_peered":"2024-04-25T09:54:23.970181+0000","last_unstale":"2024-04-25T09:54:30.086765+0000","last_undegraded":"2024-04-25T09:54:30.086765+0000","last_fullsized":"2024-04-25T09:54:30.086765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088285+0000","last_change":"2024-04-25T09:54:23.983654+0000","last_active":"2024-04-25T09:54:30.088285+0000","last_peered":"2024-04-25T09:54:30.088285+0000","last_clean":"2024-04-25T09:54:30.088285+0000","last_became_active":"2024-04-25T09:54:23.983388+0000","last_became_peered":"2024-04-25T09:54:23.983388+0000","last_unstale":"2024-04-25T09:54:30.088285+0000","last_undegraded":"2024-04-25T09:54:30.088285+0000","last_fullsized":"2024-04-25T09:54:30.088285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994460+0000","last_change":"2024-04-25T09:54:23.970049+0000","last_active":"2024-04-25T09:54:29.994460+0000","last_peered":"2024-04-25T09:54:29.994460+0000","last_clean":"2024-04-25T09:54:29.994460+0000","last_became_active":"2024-04-25T09:54:23.969605+0000","last_became_peered":"2024-04-25T09:54:23.969605+0000","last_unstale":"2024-04-25T09:54:29.994460+0000","last_undegraded":"2024-04-25T09:54:29.994460+0000","last_fullsized":"2024-04-25T09:54:29.994460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087574+0000","last_change":"2024-04-25T09:54:23.980721+0000","last_active":"2024-04-25T09:54:30.087574+0000","last_peered":"2024-04-25T09:54:30.087574+0000","last_clean":"2024-04-25T09:54:30.087574+0000","last_became_active":"2024-04-25T09:54:23.980440+0000","last_became_peered":"2024-04-25T09:54:23.980440+0000","last_unstale":"2024-04-25T09:54:30.087574+0000","last_undegraded":"2024-04-25T09:54:30.087574+0000","last_fullsized":"2024-04-25T09:54:30.087574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086883+0000","last_change":"2024-04-25T09:54:23.967904+0000","last_active":"2024-04-25T09:54:30.086883+0000","last_peered":"2024-04-25T09:54:30.086883+0000","last_clean":"2024-04-25T09:54:30.086883+0000","last_became_active":"2024-04-25T09:54:23.967585+0000","last_became_peered":"2024-04-25T09:54:23.967585+0000","last_unstale":"2024-04-25T09:54:30.086883+0000","last_undegraded":"2024-04-25T09:54:30.086883+0000","last_fullsized":"2024-04-25T09:54:30.086883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087670+0000","last_change":"2024-04-25T09:54:23.978911+0000","last_active":"2024-04-25T09:54:30.087670+0000","last_peered":"2024-04-25T09:54:30.087670+0000","last_clean":"2024-04-25T09:54:30.087670+0000","last_became_active":"2024-04-25T09:54:23.978625+0000","last_became_peered":"2024-04-25T09:54:23.978625+0000","last_unstale":"2024-04-25T09:54:30.087670+0000","last_undegraded":"2024-04-25T09:54:30.087670+0000","last_fullsized":"2024-04-25T09:54:30.087670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086202+0000","last_change":"2024-04-25T09:54:23.976392+0000","last_active":"2024-04-25T09:54:30.086202+0000","last_peered":"2024-04-25T09:54:30.086202+0000","last_clean":"2024-04-25T09:54:30.086202+0000","last_became_active":"2024-04-25T09:54:23.976117+0000","last_became_peered":"2024-04-25T09:54:23.976117+0000","last_unstale":"2024-04-25T09:54:30.086202+0000","last_undegraded":"2024-04-25T09:54:30.086202+0000","last_fullsized":"2024-04-25T09:54:30.086202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085739+0000","last_change":"2024-04-25T09:54:23.967710+0000","last_active":"2024-04-25T09:54:30.085739+0000","last_peered":"2024-04-25T09:54:30.085739+0000","last_clean":"2024-04-25T09:54:30.085739+0000","last_became_active":"2024-04-25T09:54:23.967075+0000","last_became_peered":"2024-04-25T09:54:23.967075+0000","last_unstale":"2024-04-25T09:54:30.085739+0000","last_undegraded":"2024-04-25T09:54:30.085739+0000","last_fullsized":"2024-04-25T09:54:30.085739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086748+0000","last_change":"2024-04-25T09:54:23.968977+0000","last_active":"2024-04-25T09:54:30.086748+0000","last_peered":"2024-04-25T09:54:30.086748+0000","last_clean":"2024-04-25T09:54:30.086748+0000","last_became_active":"2024-04-25T09:54:23.966936+0000","last_became_peered":"2024-04-25T09:54:23.966936+0000","last_unstale":"2024-04-25T09:54:30.086748+0000","last_undegraded":"2024-04-25T09:54:30.086748+0000","last_fullsized":"2024-04-25T09:54:30.086748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088179+0000","last_change":"2024-04-25T09:54:23.987914+0000","last_active":"2024-04-25T09:54:30.088179+0000","last_peered":"2024-04-25T09:54:30.088179+0000","last_clean":"2024-04-25T09:54:30.088179+0000","last_became_active":"2024-04-25T09:54:23.985923+0000","last_became_peered":"2024-04-25T09:54:23.985923+0000","last_unstale":"2024-04-25T09:54:30.088179+0000","last_undegraded":"2024-04-25T09:54:30.088179+0000","last_fullsized":"2024-04-25T09:54:30.088179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:50.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:54:50.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.500+0000 7f14c6c5d640 1 Processor -- start 2024-04-25T09:54:50.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.500+0000 7f14c6c5d640 1 -- start start 2024-04-25T09:54:50.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.501+0000 7f14c6c5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14c00d0b10 0x7f14c00ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.501+0000 7f14c6c5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c00b5f00 0x7f14c00d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.501+0000 7f14c6c5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c00cec00 con 0x7f14c015db60 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.501+0000 7f14c6c5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c00ce930 con 0x7f14c00d0b10 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.501+0000 7f14c6c5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00b6450 con 0x7f14c00b5f00 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14c00d0b10 0x7f14c00ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14c51fa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c00b5f00 0x7f14c00d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14c00d0b10 0x7f14c00ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57646/0 (socket says 172.21.15.148:57646) 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14c49f9640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f14c015db60 0x7f14c00d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43204/0 (socket says 172.21.15.148:43204) 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 -- 172.21.15.148:0/3499530287 learned_addr learned my addr 172.21.15.148:0/3499530287 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 -- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c00b5f00 msgr2=0x7f14c00d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 --2- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c00b5f00 0x7f14c00d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 -- 172.21.15.148:0/3499530287 >> v1:172.21.15.188:6789/0 conn(0x7f14c015db60 legacy=0x7f14c00d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bf7fe640 1 -- 172.21.15.148:0/3499530287 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241273586 0 0) 0x7f14c00cec00 con 0x7f14c015db60 2024-04-25T09:54:50.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14bffff640 1 -- 172.21.15.148:0/3499530287 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c00ceee0 con 0x7f14c00d0b10 2024-04-25T09:54:50.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.502+0000 7f14c51fa640 1 --2- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c00b5f00 0x7f14c00d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:50.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14bffff640 1 --2- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00ce3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14b40098d0 tx=0x7f14b402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531e2b5376e59c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14bf7fe640 1 -- 172.21.15.148:0/3499530287 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4037070 con 0x7f14c00d0b10 2024-04-25T09:54:50.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14bf7fe640 1 -- 172.21.15.148:0/3499530287 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b4032440 con 0x7f14c00d0b10 2024-04-25T09:54:50.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14bf7fe640 1 -- 172.21.15.148:0/3499530287 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4032810 con 0x7f14c00d0b10 2024-04-25T09:54:50.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 msgr2=0x7f14c00ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00ce3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14b40098d0 tx=0x7f14b402a230 comp rx=0 tx=0).stop 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3499530287 shutdown_connections 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00ce3f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3499530287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c00b5f00 0x7f14c00d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.503+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3499530287 >> 172.21.15.148:0/3499530287 conn(0x7f14c0061600 msgr2=0x7f14c0061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.504+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3499530287 shutdown_connections 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.504+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3499530287 wait complete. 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.505+0000 7f14c6c5d640 1 Processor -- start 2024-04-25T09:54:50.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.505+0000 7f14c6c5d640 1 -- start start 2024-04-25T09:54:50.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 0x7f14c0151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c015db60 0x7f14c00c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c015df40 con 0x7f14c00b5f00 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00d1d50 con 0x7f14c00d0b10 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c015e0c0 con 0x7f14c015db60 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 0x7f14c0151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 0x7f14c0151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33856/0 (socket says 172.21.15.148:33856) 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 -- 172.21.15.148:0/3186562684 learned_addr learned my addr 172.21.15.148:0/3186562684 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14bffff640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c51fa640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c015db60 0x7f14c00c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 -- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c015db60 msgr2=0x7f14c00c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c015db60 0x7f14c00c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 -- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 msgr2=0x7f14c00c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 -- 172.21.15.148:0/3186562684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c00c8270 con 0x7f14c00b5f00 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c51fa640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c015db60 0x7f14c00c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c49f9640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 0x7f14c0151870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c430 tx=0x7f14b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b00074e0 con 0x7f14c00b5f00 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b00047e0 con 0x7f14c00b5f00 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0004bd0 con 0x7f14c00b5f00 2024-04-25T09:54:50.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c0156bd0 con 0x7f14c00b5f00 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.506+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c0157190 con 0x7f14c00b5f00 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.507+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c0073270 con 0x7f14c00b5f00 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.508+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14b0011070 con 0x7f14c00b5f00 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.508+0000 7f14bd7fa640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f14ac06b4d0 0x7f14ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.508+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f14b008a120 con 0x7f14c00b5f00 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.508+0000 7f14bffff640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f14ac06b4d0 0x7f14ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.509+0000 7f14bffff640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f14ac06b4d0 0x7f14ac06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14b4034000 tx=0x7f14b402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.512+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14b0058c60 con 0x7f14c00b5f00 2024-04-25T09:54:50.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.679+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14c0073990 con 0x7f14ac06b4d0 2024-04-25T09:54:50.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.686+0000 7f14bd7fa640 1 -- 172.21.15.148:0/3186562684 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f14c0073990 con 0x7f14ac06b4d0 2024-04-25T09:54:50.692 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:50.694 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f14ac06b4d0 msgr2=0x7f14ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f14ac06b4d0 0x7f14ac06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14b4034000 tx=0x7f14b402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 msgr2=0x7f14c0151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 0x7f14c0151870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c430 tx=0x7f14b000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 shutdown_connections 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f14ac06b4d0 0x7f14ac06d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14c015db60 0x7f14c00c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14c00d0b10 0x7f14c00c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 --2- 172.21.15.148:0/3186562684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14c00b5f00 0x7f14c0151870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 >> 172.21.15.148:0/3186562684 conn(0x7f14c0061600 msgr2=0x7f14c015c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 shutdown_connections 2024-04-25T09:54:50.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.690+0000 7f14c6c5d640 1 -- 172.21.15.148:0/3186562684 wait complete. 2024-04-25T09:54:50.703 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T09:54:49.025642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003382"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086122+0000","last_change":"2024-04-25T09:54:22.962688+0000","last_active":"2024-04-25T09:54:30.086122+0000","last_peered":"2024-04-25T09:54:30.086122+0000","last_clean":"2024-04-25T09:54:30.086122+0000","last_became_active":"2024-04-25T09:54:22.962426+0000","last_became_peered":"2024-04-25T09:54:22.962426+0000","last_unstale":"2024-04-25T09:54:30.086122+0000","last_undegraded":"2024-04-25T09:54:30.086122+0000","last_fullsized":"2024-04-25T09:54:30.086122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:21.016302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995003+0000","last_change":"2024-04-25T09:54:23.969918+0000","last_active":"2024-04-25T09:54:29.995003+0000","last_peered":"2024-04-25T09:54:29.995003+0000","last_clean":"2024-04-25T09:54:29.995003+0000","last_became_active":"2024-04-25T09:54:23.969631+0000","last_became_peered":"2024-04-25T09:54:23.969631+0000","last_unstale":"2024-04-25T09:54:29.995003+0000","last_undegraded":"2024-04-25T09:54:29.995003+0000","last_fullsized":"2024-04-25T09:54:29.995003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994303+0000","last_change":"2024-04-25T09:54:28.397901+0000","last_active":"2024-04-25T09:54:29.994303+0000","last_peered":"2024-04-25T09:54:29.994303+0000","last_clean":"2024-04-25T09:54:29.994303+0000","last_became_active":"2024-04-25T09:54:28.397492+0000","last_became_peered":"2024-04-25T09:54:28.397492+0000","last_unstale":"2024-04-25T09:54:29.994303+0000","last_undegraded":"2024-04-25T09:54:29.994303+0000","last_fullsized":"2024-04-25T09:54:29.994303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:14.336697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995250+0000","last_change":"2024-04-25T09:54:22.958345+0000","last_active":"2024-04-25T09:54:29.995250+0000","last_peered":"2024-04-25T09:54:29.995250+0000","last_clean":"2024-04-25T09:54:29.995250+0000","last_became_active":"2024-04-25T09:54:22.958021+0000","last_became_peered":"2024-04-25T09:54:22.958021+0000","last_unstale":"2024-04-25T09:54:29.995250+0000","last_undegraded":"2024-04-25T09:54:29.995250+0000","last_fullsized":"2024-04-25T09:54:29.995250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:02.625621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085905+0000","last_change":"2024-04-25T09:54:23.970381+0000","last_active":"2024-04-25T09:54:30.085905+0000","last_peered":"2024-04-25T09:54:30.085905+0000","last_clean":"2024-04-25T09:54:30.085905+0000","last_became_active":"2024-04-25T09:54:23.970077+0000","last_became_peered":"2024-04-25T09:54:23.970077+0000","last_unstale":"2024-04-25T09:54:30.085905+0000","last_undegraded":"2024-04-25T09:54:30.085905+0000","last_fullsized":"2024-04-25T09:54:30.085905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:16.071642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995354+0000","last_change":"2024-04-25T09:54:28.011784+0000","last_active":"2024-04-25T09:54:29.995354+0000","last_peered":"2024-04-25T09:54:29.995354+0000","last_clean":"2024-04-25T09:54:29.995354+0000","last_became_active":"2024-04-25T09:54:28.011370+0000","last_became_peered":"2024-04-25T09:54:28.011370+0000","last_unstale":"2024-04-25T09:54:29.995354+0000","last_undegraded":"2024-04-25T09:54:29.995354+0000","last_fullsized":"2024-04-25T09:54:29.995354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088011+0000","last_change":"2024-04-25T09:54:22.965546+0000","last_active":"2024-04-25T09:54:30.088011+0000","last_peered":"2024-04-25T09:54:30.088011+0000","last_clean":"2024-04-25T09:54:30.088011+0000","last_became_active":"2024-04-25T09:54:22.965088+0000","last_became_peered":"2024-04-25T09:54:22.965088+0000","last_unstale":"2024-04-25T09:54:30.088011+0000","last_undegraded":"2024-04-25T09:54:30.088011+0000","last_fullsized":"2024-04-25T09:54:30.088011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:23.982573+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:23.982304+0000","last_became_peered":"2024-04-25T09:54:23.982304+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995558+0000","last_change":"2024-04-25T09:54:28.012195+0000","last_active":"2024-04-25T09:54:29.995558+0000","last_peered":"2024-04-25T09:54:29.995558+0000","last_clean":"2024-04-25T09:54:29.995558+0000","last_became_active":"2024-04-25T09:54:28.011667+0000","last_became_peered":"2024-04-25T09:54:28.011667+0000","last_unstale":"2024-04-25T09:54:29.995558+0000","last_undegraded":"2024-04-25T09:54:29.995558+0000","last_fullsized":"2024-04-25T09:54:29.995558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087654+0000","last_change":"2024-04-25T09:54:22.967092+0000","last_active":"2024-04-25T09:54:30.087654+0000","last_peered":"2024-04-25T09:54:30.087654+0000","last_clean":"2024-04-25T09:54:30.087654+0000","last_became_active":"2024-04-25T09:54:22.966690+0000","last_became_peered":"2024-04-25T09:54:22.966690+0000","last_unstale":"2024-04-25T09:54:30.087654+0000","last_undegraded":"2024-04-25T09:54:30.087654+0000","last_fullsized":"2024-04-25T09:54:30.087654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087289+0000","last_change":"2024-04-25T09:54:23.979036+0000","last_active":"2024-04-25T09:54:30.087289+0000","last_peered":"2024-04-25T09:54:30.087289+0000","last_clean":"2024-04-25T09:54:30.087289+0000","last_became_active":"2024-04-25T09:54:23.978775+0000","last_became_peered":"2024-04-25T09:54:23.978775+0000","last_unstale":"2024-04-25T09:54:30.087289+0000","last_undegraded":"2024-04-25T09:54:30.087289+0000","last_fullsized":"2024-04-25T09:54:30.087289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995398+0000","last_change":"2024-04-25T09:54:28.006708+0000","last_active":"2024-04-25T09:54:29.995398+0000","last_peered":"2024-04-25T09:54:29.995398+0000","last_clean":"2024-04-25T09:54:29.995398+0000","last_became_active":"2024-04-25T09:54:28.006351+0000","last_became_peered":"2024-04-25T09:54:28.006351+0000","last_unstale":"2024-04-25T09:54:29.995398+0000","last_undegraded":"2024-04-25T09:54:29.995398+0000","last_fullsized":"2024-04-25T09:54:29.995398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085947+0000","last_change":"2024-04-25T09:54:23.983452+0000","last_active":"2024-04-25T09:54:30.085947+0000","last_peered":"2024-04-25T09:54:30.085947+0000","last_clean":"2024-04-25T09:54:30.085947+0000","last_became_active":"2024-04-25T09:54:23.983150+0000","last_became_peered":"2024-04-25T09:54:23.983150+0000","last_unstale":"2024-04-25T09:54:30.085947+0000","last_undegraded":"2024-04-25T09:54:30.085947+0000","last_fullsized":"2024-04-25T09:54:30.085947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087835+0000","last_change":"2024-04-25T09:54:22.965628+0000","last_active":"2024-04-25T09:54:30.087835+0000","last_peered":"2024-04-25T09:54:30.087835+0000","last_clean":"2024-04-25T09:54:30.087835+0000","last_became_active":"2024-04-25T09:54:22.965294+0000","last_became_peered":"2024-04-25T09:54:22.965294+0000","last_unstale":"2024-04-25T09:54:30.087835+0000","last_undegraded":"2024-04-25T09:54:30.087835+0000","last_fullsized":"2024-04-25T09:54:30.087835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996313+0000","last_change":"2024-04-25T09:54:28.011196+0000","last_active":"2024-04-25T09:54:29.996313+0000","last_peered":"2024-04-25T09:54:29.996313+0000","last_clean":"2024-04-25T09:54:29.996313+0000","last_became_active":"2024-04-25T09:54:28.010638+0000","last_became_peered":"2024-04-25T09:54:28.010638+0000","last_unstale":"2024-04-25T09:54:29.996313+0000","last_undegraded":"2024-04-25T09:54:29.996313+0000","last_fullsized":"2024-04-25T09:54:29.996313+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994468+0000","last_change":"2024-04-25T09:54:23.975061+0000","last_active":"2024-04-25T09:54:29.994468+0000","last_peered":"2024-04-25T09:54:29.994468+0000","last_clean":"2024-04-25T09:54:29.994468+0000","last_became_active":"2024-04-25T09:54:23.974793+0000","last_became_peered":"2024-04-25T09:54:23.974793+0000","last_unstale":"2024-04-25T09:54:29.994468+0000","last_undegraded":"2024-04-25T09:54:29.994468+0000","last_fullsized":"2024-04-25T09:54:29.994468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086013+0000","last_change":"2024-04-25T09:54:22.965353+0000","last_active":"2024-04-25T09:54:30.086013+0000","last_peered":"2024-04-25T09:54:30.086013+0000","last_clean":"2024-04-25T09:54:30.086013+0000","last_became_active":"2024-04-25T09:54:22.965038+0000","last_became_peered":"2024-04-25T09:54:22.965038+0000","last_unstale":"2024-04-25T09:54:30.086013+0000","last_undegraded":"2024-04-25T09:54:30.086013+0000","last_fullsized":"2024-04-25T09:54:30.086013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996196+0000","last_change":"2024-04-25T09:54:28.005662+0000","last_active":"2024-04-25T09:54:29.996196+0000","last_peered":"2024-04-25T09:54:29.996196+0000","last_clean":"2024-04-25T09:54:29.996196+0000","last_became_active":"2024-04-25T09:54:28.005233+0000","last_became_peered":"2024-04-25T09:54:28.005233+0000","last_unstale":"2024-04-25T09:54:29.996196+0000","last_undegraded":"2024-04-25T09:54:29.996196+0000","last_fullsized":"2024-04-25T09:54:29.996196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994816+0000","last_change":"2024-04-25T09:54:23.969844+0000","last_active":"2024-04-25T09:54:29.994816+0000","last_peered":"2024-04-25T09:54:29.994816+0000","last_clean":"2024-04-25T09:54:29.994816+0000","last_became_active":"2024-04-25T09:54:23.969261+0000","last_became_peered":"2024-04-25T09:54:23.969261+0000","last_unstale":"2024-04-25T09:54:29.994816+0000","last_undegraded":"2024-04-25T09:54:29.994816+0000","last_fullsized":"2024-04-25T09:54:29.994816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086366+0000","last_change":"2024-04-25T09:54:22.961764+0000","last_active":"2024-04-25T09:54:30.086366+0000","last_peered":"2024-04-25T09:54:30.086366+0000","last_clean":"2024-04-25T09:54:30.086366+0000","last_became_active":"2024-04-25T09:54:22.961455+0000","last_became_peered":"2024-04-25T09:54:22.961455+0000","last_unstale":"2024-04-25T09:54:30.086366+0000","last_undegraded":"2024-04-25T09:54:30.086366+0000","last_fullsized":"2024-04-25T09:54:30.086366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087099+0000","last_change":"2024-04-25T09:54:28.395240+0000","last_active":"2024-04-25T09:54:30.087099+0000","last_peered":"2024-04-25T09:54:30.087099+0000","last_clean":"2024-04-25T09:54:30.087099+0000","last_became_active":"2024-04-25T09:54:28.394866+0000","last_became_peered":"2024-04-25T09:54:28.394866+0000","last_unstale":"2024-04-25T09:54:30.087099+0000","last_undegraded":"2024-04-25T09:54:30.087099+0000","last_fullsized":"2024-04-25T09:54:30.087099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995771+0000","last_change":"2024-04-25T09:54:23.973436+0000","last_active":"2024-04-25T09:54:29.995771+0000","last_peered":"2024-04-25T09:54:29.995771+0000","last_clean":"2024-04-25T09:54:29.995771+0000","last_became_active":"2024-04-25T09:54:23.973139+0000","last_became_peered":"2024-04-25T09:54:23.973139+0000","last_unstale":"2024-04-25T09:54:29.995771+0000","last_undegraded":"2024-04-25T09:54:29.995771+0000","last_fullsized":"2024-04-25T09:54:29.995771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086953+0000","last_change":"2024-04-25T09:54:22.963445+0000","last_active":"2024-04-25T09:54:30.086953+0000","last_peered":"2024-04-25T09:54:30.086953+0000","last_clean":"2024-04-25T09:54:30.086953+0000","last_became_active":"2024-04-25T09:54:22.963190+0000","last_became_peered":"2024-04-25T09:54:22.963190+0000","last_unstale":"2024-04-25T09:54:30.086953+0000","last_undegraded":"2024-04-25T09:54:30.086953+0000","last_fullsized":"2024-04-25T09:54:30.086953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:10.955157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996305+0000","last_change":"2024-04-25T09:54:28.011236+0000","last_active":"2024-04-25T09:54:29.996305+0000","last_peered":"2024-04-25T09:54:29.996305+0000","last_clean":"2024-04-25T09:54:29.996305+0000","last_became_active":"2024-04-25T09:54:28.010633+0000","last_became_peered":"2024-04-25T09:54:28.010633+0000","last_unstale":"2024-04-25T09:54:29.996305+0000","last_undegraded":"2024-04-25T09:54:29.996305+0000","last_fullsized":"2024-04-25T09:54:29.996305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086754+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:30.086754+0000","last_peered":"2024-04-25T09:54:30.086754+0000","last_clean":"2024-04-25T09:54:30.086754+0000","last_became_active":"2024-04-25T09:54:23.970411+0000","last_became_peered":"2024-04-25T09:54:23.970411+0000","last_unstale":"2024-04-25T09:54:30.086754+0000","last_undegraded":"2024-04-25T09:54:30.086754+0000","last_fullsized":"2024-04-25T09:54:30.086754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:20.071529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086786+0000","last_change":"2024-04-25T09:54:22.961585+0000","last_active":"2024-04-25T09:54:30.086786+0000","last_peered":"2024-04-25T09:54:30.086786+0000","last_clean":"2024-04-25T09:54:30.086786+0000","last_became_active":"2024-04-25T09:54:22.961286+0000","last_became_peered":"2024-04-25T09:54:22.961286+0000","last_unstale":"2024-04-25T09:54:30.086786+0000","last_undegraded":"2024-04-25T09:54:30.086786+0000","last_fullsized":"2024-04-25T09:54:30.086786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995453+0000","last_change":"2024-04-25T09:54:28.396021+0000","last_active":"2024-04-25T09:54:29.995453+0000","last_peered":"2024-04-25T09:54:29.995453+0000","last_clean":"2024-04-25T09:54:29.995453+0000","last_became_active":"2024-04-25T09:54:28.394961+0000","last_became_peered":"2024-04-25T09:54:28.394961+0000","last_unstale":"2024-04-25T09:54:29.995453+0000","last_undegraded":"2024-04-25T09:54:29.995453+0000","last_fullsized":"2024-04-25T09:54:29.995453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995224+0000","last_change":"2024-04-25T09:54:23.981659+0000","last_active":"2024-04-25T09:54:29.995224+0000","last_peered":"2024-04-25T09:54:29.995224+0000","last_clean":"2024-04-25T09:54:29.995224+0000","last_became_active":"2024-04-25T09:54:23.981345+0000","last_became_peered":"2024-04-25T09:54:23.981345+0000","last_unstale":"2024-04-25T09:54:29.995224+0000","last_undegraded":"2024-04-25T09:54:29.995224+0000","last_fullsized":"2024-04-25T09:54:29.995224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086841+0000","last_change":"2024-04-25T09:54:22.961799+0000","last_active":"2024-04-25T09:54:30.086841+0000","last_peered":"2024-04-25T09:54:30.086841+0000","last_clean":"2024-04-25T09:54:30.086841+0000","last_became_active":"2024-04-25T09:54:22.961205+0000","last_became_peered":"2024-04-25T09:54:22.961205+0000","last_unstale":"2024-04-25T09:54:30.086841+0000","last_undegraded":"2024-04-25T09:54:30.086841+0000","last_fullsized":"2024-04-25T09:54:30.086841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994244+0000","last_change":"2024-04-25T09:54:28.397432+0000","last_active":"2024-04-25T09:54:29.994244+0000","last_peered":"2024-04-25T09:54:29.994244+0000","last_clean":"2024-04-25T09:54:29.994244+0000","last_became_active":"2024-04-25T09:54:28.396060+0000","last_became_peered":"2024-04-25T09:54:28.396060+0000","last_unstale":"2024-04-25T09:54:29.994244+0000","last_undegraded":"2024-04-25T09:54:29.994244+0000","last_fullsized":"2024-04-25T09:54:29.994244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:18.141331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086215+0000","last_change":"2024-04-25T09:54:23.964362+0000","last_active":"2024-04-25T09:54:30.086215+0000","last_peered":"2024-04-25T09:54:30.086215+0000","last_clean":"2024-04-25T09:54:30.086215+0000","last_became_active":"2024-04-25T09:54:23.964097+0000","last_became_peered":"2024-04-25T09:54:23.964097+0000","last_unstale":"2024-04-25T09:54:30.086215+0000","last_undegraded":"2024-04-25T09:54:30.086215+0000","last_fullsized":"2024-04-25T09:54:30.086215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088121+0000","last_change":"2024-04-25T09:54:22.967489+0000","last_active":"2024-04-25T09:54:30.088121+0000","last_peered":"2024-04-25T09:54:30.088121+0000","last_clean":"2024-04-25T09:54:30.088121+0000","last_became_active":"2024-04-25T09:54:22.967191+0000","last_became_peered":"2024-04-25T09:54:22.967191+0000","last_unstale":"2024-04-25T09:54:30.088121+0000","last_undegraded":"2024-04-25T09:54:30.088121+0000","last_fullsized":"2024-04-25T09:54:30.088121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.805416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086235+0000","last_change":"2024-04-25T09:54:28.394981+0000","last_active":"2024-04-25T09:54:30.086235+0000","last_peered":"2024-04-25T09:54:30.086235+0000","last_clean":"2024-04-25T09:54:30.086235+0000","last_became_active":"2024-04-25T09:54:28.394572+0000","last_became_peered":"2024-04-25T09:54:28.394572+0000","last_unstale":"2024-04-25T09:54:30.086235+0000","last_undegraded":"2024-04-25T09:54:30.086235+0000","last_fullsized":"2024-04-25T09:54:30.086235+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.977167+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.976581+0000","last_became_peered":"2024-04-25T09:54:23.976581+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086309+0000","last_change":"2024-04-25T09:54:22.957000+0000","last_active":"2024-04-25T09:54:30.086309+0000","last_peered":"2024-04-25T09:54:30.086309+0000","last_clean":"2024-04-25T09:54:30.086309+0000","last_became_active":"2024-04-25T09:54:22.956737+0000","last_became_peered":"2024-04-25T09:54:22.956737+0000","last_unstale":"2024-04-25T09:54:30.086309+0000","last_undegraded":"2024-04-25T09:54:30.086309+0000","last_fullsized":"2024-04-25T09:54:30.086309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085588+0000","last_change":"2024-04-25T09:54:28.009926+0000","last_active":"2024-04-25T09:54:30.085588+0000","last_peered":"2024-04-25T09:54:30.085588+0000","last_clean":"2024-04-25T09:54:30.085588+0000","last_became_active":"2024-04-25T09:54:28.009486+0000","last_became_peered":"2024-04-25T09:54:28.009486+0000","last_unstale":"2024-04-25T09:54:30.085588+0000","last_undegraded":"2024-04-25T09:54:30.085588+0000","last_fullsized":"2024-04-25T09:54:30.085588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996132+0000","last_change":"2024-04-25T09:54:23.974850+0000","last_active":"2024-04-25T09:54:29.996132+0000","last_peered":"2024-04-25T09:54:29.996132+0000","last_clean":"2024-04-25T09:54:29.996132+0000","last_became_active":"2024-04-25T09:54:23.974014+0000","last_became_peered":"2024-04-25T09:54:23.974014+0000","last_unstale":"2024-04-25T09:54:29.996132+0000","last_undegraded":"2024-04-25T09:54:29.996132+0000","last_fullsized":"2024-04-25T09:54:29.996132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087242+0000","last_change":"2024-04-25T09:54:22.966563+0000","last_active":"2024-04-25T09:54:30.087242+0000","last_peered":"2024-04-25T09:54:30.087242+0000","last_clean":"2024-04-25T09:54:30.087242+0000","last_became_active":"2024-04-25T09:54:22.966272+0000","last_became_peered":"2024-04-25T09:54:22.966272+0000","last_unstale":"2024-04-25T09:54:30.087242+0000","last_undegraded":"2024-04-25T09:54:30.087242+0000","last_fullsized":"2024-04-25T09:54:30.087242+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086121+0000","last_change":"2024-04-25T09:54:28.013312+0000","last_active":"2024-04-25T09:54:30.086121+0000","last_peered":"2024-04-25T09:54:30.086121+0000","last_clean":"2024-04-25T09:54:30.086121+0000","last_became_active":"2024-04-25T09:54:28.012953+0000","last_became_peered":"2024-04-25T09:54:28.012953+0000","last_unstale":"2024-04-25T09:54:30.086121+0000","last_undegraded":"2024-04-25T09:54:30.086121+0000","last_fullsized":"2024-04-25T09:54:30.086121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:19.258904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087003+0000","last_change":"2024-04-25T09:54:23.966466+0000","last_active":"2024-04-25T09:54:30.087003+0000","last_peered":"2024-04-25T09:54:30.087003+0000","last_clean":"2024-04-25T09:54:30.087003+0000","last_became_active":"2024-04-25T09:54:23.966129+0000","last_became_peered":"2024-04-25T09:54:23.966129+0000","last_unstale":"2024-04-25T09:54:30.087003+0000","last_undegraded":"2024-04-25T09:54:30.087003+0000","last_fullsized":"2024-04-25T09:54:30.087003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086581+0000","last_change":"2024-04-25T09:54:22.963150+0000","last_active":"2024-04-25T09:54:30.086581+0000","last_peered":"2024-04-25T09:54:30.086581+0000","last_clean":"2024-04-25T09:54:30.086581+0000","last_became_active":"2024-04-25T09:54:22.962832+0000","last_became_peered":"2024-04-25T09:54:22.962832+0000","last_unstale":"2024-04-25T09:54:30.086581+0000","last_undegraded":"2024-04-25T09:54:30.086581+0000","last_fullsized":"2024-04-25T09:54:30.086581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087126+0000","last_change":"2024-04-25T09:54:28.011051+0000","last_active":"2024-04-25T09:54:30.087126+0000","last_peered":"2024-04-25T09:54:30.087126+0000","last_clean":"2024-04-25T09:54:30.087126+0000","last_became_active":"2024-04-25T09:54:28.010629+0000","last_became_peered":"2024-04-25T09:54:28.010629+0000","last_unstale":"2024-04-25T09:54:30.087126+0000","last_undegraded":"2024-04-25T09:54:30.087126+0000","last_fullsized":"2024-04-25T09:54:30.087126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085547+0000","last_change":"2024-04-25T09:54:23.973638+0000","last_active":"2024-04-25T09:54:30.085547+0000","last_peered":"2024-04-25T09:54:30.085547+0000","last_clean":"2024-04-25T09:54:30.085547+0000","last_became_active":"2024-04-25T09:54:23.973349+0000","last_became_peered":"2024-04-25T09:54:23.973349+0000","last_unstale":"2024-04-25T09:54:30.085547+0000","last_undegraded":"2024-04-25T09:54:30.085547+0000","last_fullsized":"2024-04-25T09:54:30.085547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087072+0000","last_change":"2024-04-25T09:54:22.966560+0000","last_active":"2024-04-25T09:54:30.087072+0000","last_peered":"2024-04-25T09:54:30.087072+0000","last_clean":"2024-04-25T09:54:30.087072+0000","last_became_active":"2024-04-25T09:54:22.966299+0000","last_became_peered":"2024-04-25T09:54:22.966299+0000","last_unstale":"2024-04-25T09:54:30.087072+0000","last_undegraded":"2024-04-25T09:54:30.087072+0000","last_fullsized":"2024-04-25T09:54:30.087072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086483+0000","last_change":"2024-04-25T09:54:28.395179+0000","last_active":"2024-04-25T09:54:30.086483+0000","last_peered":"2024-04-25T09:54:30.086483+0000","last_clean":"2024-04-25T09:54:30.086483+0000","last_became_active":"2024-04-25T09:54:28.394767+0000","last_became_peered":"2024-04-25T09:54:28.394767+0000","last_unstale":"2024-04-25T09:54:30.086483+0000","last_undegraded":"2024-04-25T09:54:30.086483+0000","last_fullsized":"2024-04-25T09:54:30.086483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:30.028837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994411+0000","last_change":"2024-04-25T09:54:23.969956+0000","last_active":"2024-04-25T09:54:29.994411+0000","last_peered":"2024-04-25T09:54:29.994411+0000","last_clean":"2024-04-25T09:54:29.994411+0000","last_became_active":"2024-04-25T09:54:23.969643+0000","last_became_peered":"2024-04-25T09:54:23.969643+0000","last_unstale":"2024-04-25T09:54:29.994411+0000","last_undegraded":"2024-04-25T09:54:29.994411+0000","last_fullsized":"2024-04-25T09:54:29.994411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086619+0000","last_change":"2024-04-25T09:54:22.965368+0000","last_active":"2024-04-25T09:54:30.086619+0000","last_peered":"2024-04-25T09:54:30.086619+0000","last_clean":"2024-04-25T09:54:30.086619+0000","last_became_active":"2024-04-25T09:54:22.965051+0000","last_became_peered":"2024-04-25T09:54:22.965051+0000","last_unstale":"2024-04-25T09:54:30.086619+0000","last_undegraded":"2024-04-25T09:54:30.086619+0000","last_fullsized":"2024-04-25T09:54:30.086619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087177+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:54:30.087177+0000","last_peered":"2024-04-25T09:54:30.087177+0000","last_clean":"2024-04-25T09:54:30.087177+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:54:30.087177+0000","last_undegraded":"2024-04-25T09:54:30.087177+0000","last_fullsized":"2024-04-25T09:54:30.087177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085975+0000","last_change":"2024-04-25T09:54:28.396320+0000","last_active":"2024-04-25T09:54:30.085975+0000","last_peered":"2024-04-25T09:54:30.085975+0000","last_clean":"2024-04-25T09:54:30.085975+0000","last_became_active":"2024-04-25T09:54:28.395895+0000","last_became_peered":"2024-04-25T09:54:28.395895+0000","last_unstale":"2024-04-25T09:54:30.085975+0000","last_undegraded":"2024-04-25T09:54:30.085975+0000","last_fullsized":"2024-04-25T09:54:30.085975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:18.160554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087913+0000","last_change":"2024-04-25T09:54:23.983959+0000","last_active":"2024-04-25T09:54:30.087913+0000","last_peered":"2024-04-25T09:54:30.087913+0000","last_clean":"2024-04-25T09:54:30.087913+0000","last_became_active":"2024-04-25T09:54:23.983669+0000","last_became_peered":"2024-04-25T09:54:23.983669+0000","last_unstale":"2024-04-25T09:54:30.087913+0000","last_undegraded":"2024-04-25T09:54:30.087913+0000","last_fullsized":"2024-04-25T09:54:30.087913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087174+0000","last_change":"2024-04-25T09:54:22.966591+0000","last_active":"2024-04-25T09:54:30.087174+0000","last_peered":"2024-04-25T09:54:30.087174+0000","last_clean":"2024-04-25T09:54:30.087174+0000","last_became_active":"2024-04-25T09:54:22.966301+0000","last_became_peered":"2024-04-25T09:54:22.966301+0000","last_unstale":"2024-04-25T09:54:30.087174+0000","last_undegraded":"2024-04-25T09:54:30.087174+0000","last_fullsized":"2024-04-25T09:54:30.087174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085806+0000","last_change":"2024-04-25T09:54:28.395577+0000","last_active":"2024-04-25T09:54:30.085806+0000","last_peered":"2024-04-25T09:54:30.085806+0000","last_clean":"2024-04-25T09:54:30.085806+0000","last_became_active":"2024-04-25T09:54:28.395067+0000","last_became_peered":"2024-04-25T09:54:28.395067+0000","last_unstale":"2024-04-25T09:54:30.085806+0000","last_undegraded":"2024-04-25T09:54:30.085806+0000","last_fullsized":"2024-04-25T09:54:30.085806+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:40.978481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086598+0000","last_change":"2024-04-25T09:54:23.975660+0000","last_active":"2024-04-25T09:54:30.086598+0000","last_peered":"2024-04-25T09:54:30.086598+0000","last_clean":"2024-04-25T09:54:30.086598+0000","last_became_active":"2024-04-25T09:54:23.975394+0000","last_became_peered":"2024-04-25T09:54:23.975394+0000","last_unstale":"2024-04-25T09:54:30.086598+0000","last_undegraded":"2024-04-25T09:54:30.086598+0000","last_fullsized":"2024-04-25T09:54:30.086598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087150+0000","last_change":"2024-04-25T09:54:22.959855+0000","last_active":"2024-04-25T09:54:30.087150+0000","last_peered":"2024-04-25T09:54:30.087150+0000","last_clean":"2024-04-25T09:54:30.087150+0000","last_became_active":"2024-04-25T09:54:22.959556+0000","last_became_peered":"2024-04-25T09:54:22.959556+0000","last_unstale":"2024-04-25T09:54:30.087150+0000","last_undegraded":"2024-04-25T09:54:30.087150+0000","last_fullsized":"2024-04-25T09:54:30.087150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:11.538574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086831+0000","last_change":"2024-04-25T09:54:28.010760+0000","last_active":"2024-04-25T09:54:30.086831+0000","last_peered":"2024-04-25T09:54:30.086831+0000","last_clean":"2024-04-25T09:54:30.086831+0000","last_became_active":"2024-04-25T09:54:28.010391+0000","last_became_peered":"2024-04-25T09:54:28.010391+0000","last_unstale":"2024-04-25T09:54:30.086831+0000","last_undegraded":"2024-04-25T09:54:30.086831+0000","last_fullsized":"2024-04-25T09:54:30.086831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088301+0000","last_change":"2024-04-25T09:54:23.982380+0000","last_active":"2024-04-25T09:54:30.088301+0000","last_peered":"2024-04-25T09:54:30.088301+0000","last_clean":"2024-04-25T09:54:30.088301+0000","last_became_active":"2024-04-25T09:54:23.982076+0000","last_became_peered":"2024-04-25T09:54:23.982076+0000","last_unstale":"2024-04-25T09:54:30.088301+0000","last_undegraded":"2024-04-25T09:54:30.088301+0000","last_fullsized":"2024-04-25T09:54:30.088301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086229+0000","last_change":"2024-04-25T09:54:22.963068+0000","last_active":"2024-04-25T09:54:30.086229+0000","last_peered":"2024-04-25T09:54:30.086229+0000","last_clean":"2024-04-25T09:54:30.086229+0000","last_became_active":"2024-04-25T09:54:22.962607+0000","last_became_peered":"2024-04-25T09:54:22.962607+0000","last_unstale":"2024-04-25T09:54:30.086229+0000","last_undegraded":"2024-04-25T09:54:30.086229+0000","last_fullsized":"2024-04-25T09:54:30.086229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086295+0000","last_change":"2024-04-25T09:54:28.396123+0000","last_active":"2024-04-25T09:54:30.086295+0000","last_peered":"2024-04-25T09:54:30.086295+0000","last_clean":"2024-04-25T09:54:30.086295+0000","last_became_active":"2024-04-25T09:54:28.395141+0000","last_became_peered":"2024-04-25T09:54:28.395141+0000","last_unstale":"2024-04-25T09:54:30.086295+0000","last_undegraded":"2024-04-25T09:54:30.086295+0000","last_fullsized":"2024-04-25T09:54:30.086295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:29.121658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087759+0000","last_change":"2024-04-25T09:54:23.981922+0000","last_active":"2024-04-25T09:54:30.087759+0000","last_peered":"2024-04-25T09:54:30.087759+0000","last_clean":"2024-04-25T09:54:30.087759+0000","last_became_active":"2024-04-25T09:54:23.981624+0000","last_became_peered":"2024-04-25T09:54:23.981624+0000","last_unstale":"2024-04-25T09:54:30.087759+0000","last_undegraded":"2024-04-25T09:54:30.087759+0000","last_fullsized":"2024-04-25T09:54:30.087759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:30.744040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996103+0000","last_change":"2024-04-25T09:54:22.958111+0000","last_active":"2024-04-25T09:54:29.996103+0000","last_peered":"2024-04-25T09:54:29.996103+0000","last_clean":"2024-04-25T09:54:29.996103+0000","last_became_active":"2024-04-25T09:54:22.957822+0000","last_became_peered":"2024-04-25T09:54:22.957822+0000","last_unstale":"2024-04-25T09:54:29.996103+0000","last_undegraded":"2024-04-25T09:54:29.996103+0000","last_fullsized":"2024-04-25T09:54:29.996103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:05.312329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995523+0000","last_change":"2024-04-25T09:54:28.011885+0000","last_active":"2024-04-25T09:54:29.995523+0000","last_peered":"2024-04-25T09:54:29.995523+0000","last_clean":"2024-04-25T09:54:29.995523+0000","last_became_active":"2024-04-25T09:54:28.011498+0000","last_became_peered":"2024-04-25T09:54:28.011498+0000","last_unstale":"2024-04-25T09:54:29.995523+0000","last_undegraded":"2024-04-25T09:54:29.995523+0000","last_fullsized":"2024-04-25T09:54:29.995523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087609+0000","last_change":"2024-04-25T09:54:23.982839+0000","last_active":"2024-04-25T09:54:30.087609+0000","last_peered":"2024-04-25T09:54:30.087609+0000","last_clean":"2024-04-25T09:54:30.087609+0000","last_became_active":"2024-04-25T09:54:23.982542+0000","last_became_peered":"2024-04-25T09:54:23.982542+0000","last_unstale":"2024-04-25T09:54:30.087609+0000","last_undegraded":"2024-04-25T09:54:30.087609+0000","last_fullsized":"2024-04-25T09:54:30.087609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085485+0000","last_change":"2024-04-25T09:54:22.954113+0000","last_active":"2024-04-25T09:54:30.085485+0000","last_peered":"2024-04-25T09:54:30.085485+0000","last_clean":"2024-04-25T09:54:30.085485+0000","last_became_active":"2024-04-25T09:54:22.953819+0000","last_became_peered":"2024-04-25T09:54:22.953819+0000","last_unstale":"2024-04-25T09:54:30.085485+0000","last_undegraded":"2024-04-25T09:54:30.085485+0000","last_fullsized":"2024-04-25T09:54:30.085485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086548+0000","last_change":"2024-04-25T09:54:28.008178+0000","last_active":"2024-04-25T09:54:30.086548+0000","last_peered":"2024-04-25T09:54:30.086548+0000","last_clean":"2024-04-25T09:54:30.086548+0000","last_became_active":"2024-04-25T09:54:28.007737+0000","last_became_peered":"2024-04-25T09:54:28.007737+0000","last_unstale":"2024-04-25T09:54:30.086548+0000","last_undegraded":"2024-04-25T09:54:30.086548+0000","last_fullsized":"2024-04-25T09:54:30.086548+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:20.576777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085923+0000","last_change":"2024-04-25T09:54:23.977488+0000","last_active":"2024-04-25T09:54:30.085923+0000","last_peered":"2024-04-25T09:54:30.085923+0000","last_clean":"2024-04-25T09:54:30.085923+0000","last_became_active":"2024-04-25T09:54:23.977190+0000","last_became_peered":"2024-04-25T09:54:23.977190+0000","last_unstale":"2024-04-25T09:54:30.085923+0000","last_undegraded":"2024-04-25T09:54:30.085923+0000","last_fullsized":"2024-04-25T09:54:30.085923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086516+0000","last_change":"2024-04-25T09:54:22.959330+0000","last_active":"2024-04-25T09:54:30.086516+0000","last_peered":"2024-04-25T09:54:30.086516+0000","last_clean":"2024-04-25T09:54:30.086516+0000","last_became_active":"2024-04-25T09:54:22.959026+0000","last_became_peered":"2024-04-25T09:54:22.959026+0000","last_unstale":"2024-04-25T09:54:30.086516+0000","last_undegraded":"2024-04-25T09:54:30.086516+0000","last_fullsized":"2024-04-25T09:54:30.086516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995258+0000","last_change":"2024-04-25T09:54:28.396288+0000","last_active":"2024-04-25T09:54:29.995258+0000","last_peered":"2024-04-25T09:54:29.995258+0000","last_clean":"2024-04-25T09:54:29.995258+0000","last_became_active":"2024-04-25T09:54:28.395940+0000","last_became_peered":"2024-04-25T09:54:28.395940+0000","last_unstale":"2024-04-25T09:54:29.995258+0000","last_undegraded":"2024-04-25T09:54:29.995258+0000","last_fullsized":"2024-04-25T09:54:29.995258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:05.296954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087789+0000","last_change":"2024-04-25T09:54:23.982575+0000","last_active":"2024-04-25T09:54:30.087789+0000","last_peered":"2024-04-25T09:54:30.087789+0000","last_clean":"2024-04-25T09:54:30.087789+0000","last_became_active":"2024-04-25T09:54:23.982274+0000","last_became_peered":"2024-04-25T09:54:23.982274+0000","last_unstale":"2024-04-25T09:54:30.087789+0000","last_undegraded":"2024-04-25T09:54:30.087789+0000","last_fullsized":"2024-04-25T09:54:30.087789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:04.346172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087152+0000","last_change":"2024-04-25T09:54:22.962975+0000","last_active":"2024-04-25T09:54:30.087152+0000","last_peered":"2024-04-25T09:54:30.087152+0000","last_clean":"2024-04-25T09:54:30.087152+0000","last_became_active":"2024-04-25T09:54:22.962642+0000","last_became_peered":"2024-04-25T09:54:22.962642+0000","last_unstale":"2024-04-25T09:54:30.087152+0000","last_undegraded":"2024-04-25T09:54:30.087152+0000","last_fullsized":"2024-04-25T09:54:30.087152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994645+0000","last_change":"2024-04-25T09:54:28.395146+0000","last_active":"2024-04-25T09:54:29.994645+0000","last_peered":"2024-04-25T09:54:29.994645+0000","last_clean":"2024-04-25T09:54:29.994645+0000","last_became_active":"2024-04-25T09:54:28.394756+0000","last_became_peered":"2024-04-25T09:54:28.394756+0000","last_unstale":"2024-04-25T09:54:29.994645+0000","last_undegraded":"2024-04-25T09:54:29.994645+0000","last_fullsized":"2024-04-25T09:54:29.994645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:05.993975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086033+0000","last_change":"2024-04-25T09:54:23.972232+0000","last_active":"2024-04-25T09:54:30.086033+0000","last_peered":"2024-04-25T09:54:30.086033+0000","last_clean":"2024-04-25T09:54:30.086033+0000","last_became_active":"2024-04-25T09:54:23.971948+0000","last_became_peered":"2024-04-25T09:54:23.971948+0000","last_unstale":"2024-04-25T09:54:30.086033+0000","last_undegraded":"2024-04-25T09:54:30.086033+0000","last_fullsized":"2024-04-25T09:54:30.086033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:10.704903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086681+0000","last_change":"2024-04-25T09:54:22.957583+0000","last_active":"2024-04-25T09:54:30.086681+0000","last_peered":"2024-04-25T09:54:30.086681+0000","last_clean":"2024-04-25T09:54:30.086681+0000","last_became_active":"2024-04-25T09:54:22.957190+0000","last_became_peered":"2024-04-25T09:54:22.957190+0000","last_unstale":"2024-04-25T09:54:30.086681+0000","last_undegraded":"2024-04-25T09:54:30.086681+0000","last_fullsized":"2024-04-25T09:54:30.086681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:21.147671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087171+0000","last_change":"2024-04-25T09:54:28.395989+0000","last_active":"2024-04-25T09:54:30.087171+0000","last_peered":"2024-04-25T09:54:30.087171+0000","last_clean":"2024-04-25T09:54:30.087171+0000","last_became_active":"2024-04-25T09:54:28.395578+0000","last_became_peered":"2024-04-25T09:54:28.395578+0000","last_unstale":"2024-04-25T09:54:30.087171+0000","last_undegraded":"2024-04-25T09:54:30.087171+0000","last_fullsized":"2024-04-25T09:54:30.087171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:36.837954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087863+0000","last_change":"2024-04-25T09:54:23.984390+0000","last_active":"2024-04-25T09:54:30.087863+0000","last_peered":"2024-04-25T09:54:30.087863+0000","last_clean":"2024-04-25T09:54:30.087863+0000","last_became_active":"2024-04-25T09:54:23.981528+0000","last_became_peered":"2024-04-25T09:54:23.981528+0000","last_unstale":"2024-04-25T09:54:30.087863+0000","last_undegraded":"2024-04-25T09:54:30.087863+0000","last_fullsized":"2024-04-25T09:54:30.087863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:33.133289+0000","last_change":"2024-04-25T09:54:22.962540+0000","last_active":"2024-04-25T09:54:33.133289+0000","last_peered":"2024-04-25T09:54:33.133289+0000","last_clean":"2024-04-25T09:54:33.133289+0000","last_became_active":"2024-04-25T09:54:22.962223+0000","last_became_peered":"2024-04-25T09:54:22.962223+0000","last_unstale":"2024-04-25T09:54:33.133289+0000","last_undegraded":"2024-04-25T09:54:33.133289+0000","last_fullsized":"2024-04-25T09:54:33.133289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.973886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085807+0000","last_change":"2024-04-25T09:54:28.012670+0000","last_active":"2024-04-25T09:54:30.085807+0000","last_peered":"2024-04-25T09:54:30.085807+0000","last_clean":"2024-04-25T09:54:30.085807+0000","last_became_active":"2024-04-25T09:54:28.011580+0000","last_became_peered":"2024-04-25T09:54:28.011580+0000","last_unstale":"2024-04-25T09:54:30.085807+0000","last_undegraded":"2024-04-25T09:54:30.085807+0000","last_fullsized":"2024-04-25T09:54:30.085807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.996273+0000","last_change":"2024-04-25T09:54:23.981044+0000","last_active":"2024-04-25T09:54:29.996273+0000","last_peered":"2024-04-25T09:54:29.996273+0000","last_clean":"2024-04-25T09:54:29.996273+0000","last_became_active":"2024-04-25T09:54:23.980733+0000","last_became_peered":"2024-04-25T09:54:23.980733+0000","last_unstale":"2024-04-25T09:54:29.996273+0000","last_undegraded":"2024-04-25T09:54:29.996273+0000","last_fullsized":"2024-04-25T09:54:29.996273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087059+0000","last_change":"2024-04-25T09:54:22.965478+0000","last_active":"2024-04-25T09:54:30.087059+0000","last_peered":"2024-04-25T09:54:30.087059+0000","last_clean":"2024-04-25T09:54:30.087059+0000","last_became_active":"2024-04-25T09:54:22.965191+0000","last_became_peered":"2024-04-25T09:54:22.965191+0000","last_unstale":"2024-04-25T09:54:30.087059+0000","last_undegraded":"2024-04-25T09:54:30.087059+0000","last_fullsized":"2024-04-25T09:54:30.087059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995924+0000","last_change":"2024-04-25T09:54:28.003916+0000","last_active":"2024-04-25T09:54:29.995924+0000","last_peered":"2024-04-25T09:54:29.995924+0000","last_clean":"2024-04-25T09:54:29.995924+0000","last_became_active":"2024-04-25T09:54:28.003512+0000","last_became_peered":"2024-04-25T09:54:28.003512+0000","last_unstale":"2024-04-25T09:54:29.995924+0000","last_undegraded":"2024-04-25T09:54:29.995924+0000","last_fullsized":"2024-04-25T09:54:29.995924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:23.127479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086045+0000","last_change":"2024-04-25T09:54:23.969795+0000","last_active":"2024-04-25T09:54:30.086045+0000","last_peered":"2024-04-25T09:54:30.086045+0000","last_clean":"2024-04-25T09:54:30.086045+0000","last_became_active":"2024-04-25T09:54:23.969543+0000","last_became_peered":"2024-04-25T09:54:23.969543+0000","last_unstale":"2024-04-25T09:54:30.086045+0000","last_undegraded":"2024-04-25T09:54:30.086045+0000","last_fullsized":"2024-04-25T09:54:30.086045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086869+0000","last_change":"2024-04-25T09:54:22.965431+0000","last_active":"2024-04-25T09:54:30.086869+0000","last_peered":"2024-04-25T09:54:30.086869+0000","last_clean":"2024-04-25T09:54:30.086869+0000","last_became_active":"2024-04-25T09:54:22.965115+0000","last_became_peered":"2024-04-25T09:54:22.965115+0000","last_unstale":"2024-04-25T09:54:30.086869+0000","last_undegraded":"2024-04-25T09:54:30.086869+0000","last_fullsized":"2024-04-25T09:54:30.086869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.326768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994658+0000","last_change":"2024-04-25T09:54:28.011370+0000","last_active":"2024-04-25T09:54:29.994658+0000","last_peered":"2024-04-25T09:54:29.994658+0000","last_clean":"2024-04-25T09:54:29.994658+0000","last_became_active":"2024-04-25T09:54:28.011006+0000","last_became_peered":"2024-04-25T09:54:28.011006+0000","last_unstale":"2024-04-25T09:54:29.994658+0000","last_undegraded":"2024-04-25T09:54:29.994658+0000","last_fullsized":"2024-04-25T09:54:29.994658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:05.906706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086261+0000","last_change":"2024-04-25T09:54:23.983800+0000","last_active":"2024-04-25T09:54:30.086261+0000","last_peered":"2024-04-25T09:54:30.086261+0000","last_clean":"2024-04-25T09:54:30.086261+0000","last_became_active":"2024-04-25T09:54:23.983518+0000","last_became_peered":"2024-04-25T09:54:23.983518+0000","last_unstale":"2024-04-25T09:54:30.086261+0000","last_undegraded":"2024-04-25T09:54:30.086261+0000","last_fullsized":"2024-04-25T09:54:30.086261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086666+0000","last_change":"2024-04-25T09:54:22.966898+0000","last_active":"2024-04-25T09:54:30.086666+0000","last_peered":"2024-04-25T09:54:30.086666+0000","last_clean":"2024-04-25T09:54:30.086666+0000","last_became_active":"2024-04-25T09:54:22.966638+0000","last_became_peered":"2024-04-25T09:54:22.966638+0000","last_unstale":"2024-04-25T09:54:30.086666+0000","last_undegraded":"2024-04-25T09:54:30.086666+0000","last_fullsized":"2024-04-25T09:54:30.086666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.929702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994713+0000","last_change":"2024-04-25T09:54:28.397771+0000","last_active":"2024-04-25T09:54:29.994713+0000","last_peered":"2024-04-25T09:54:29.994713+0000","last_clean":"2024-04-25T09:54:29.994713+0000","last_became_active":"2024-04-25T09:54:28.397413+0000","last_became_peered":"2024-04-25T09:54:28.397413+0000","last_unstale":"2024-04-25T09:54:29.994713+0000","last_undegraded":"2024-04-25T09:54:29.994713+0000","last_fullsized":"2024-04-25T09:54:29.994713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:07.468831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086335+0000","last_change":"2024-04-25T09:54:28.396249+0000","last_active":"2024-04-25T09:54:30.086335+0000","last_peered":"2024-04-25T09:54:30.086335+0000","last_clean":"2024-04-25T09:54:30.086335+0000","last_became_active":"2024-04-25T09:54:28.395904+0000","last_became_peered":"2024-04-25T09:54:28.395904+0000","last_unstale":"2024-04-25T09:54:30.086335+0000","last_undegraded":"2024-04-25T09:54:30.086335+0000","last_fullsized":"2024-04-25T09:54:30.086335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086193+0000","last_change":"2024-04-25T09:54:23.972012+0000","last_active":"2024-04-25T09:54:30.086193+0000","last_peered":"2024-04-25T09:54:30.086193+0000","last_clean":"2024-04-25T09:54:30.086193+0000","last_became_active":"2024-04-25T09:54:23.971731+0000","last_became_peered":"2024-04-25T09:54:23.971731+0000","last_unstale":"2024-04-25T09:54:30.086193+0000","last_undegraded":"2024-04-25T09:54:30.086193+0000","last_fullsized":"2024-04-25T09:54:30.086193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086458+0000","last_change":"2024-04-25T09:54:22.965400+0000","last_active":"2024-04-25T09:54:30.086458+0000","last_peered":"2024-04-25T09:54:30.086458+0000","last_clean":"2024-04-25T09:54:30.086458+0000","last_became_active":"2024-04-25T09:54:22.965099+0000","last_became_peered":"2024-04-25T09:54:22.965099+0000","last_unstale":"2024-04-25T09:54:30.086458+0000","last_undegraded":"2024-04-25T09:54:30.086458+0000","last_fullsized":"2024-04-25T09:54:30.086458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086552+0000","last_change":"2024-04-25T09:54:28.010667+0000","last_active":"2024-04-25T09:54:30.086552+0000","last_peered":"2024-04-25T09:54:30.086552+0000","last_clean":"2024-04-25T09:54:30.086552+0000","last_became_active":"2024-04-25T09:54:28.010243+0000","last_became_peered":"2024-04-25T09:54:28.010243+0000","last_unstale":"2024-04-25T09:54:30.086552+0000","last_undegraded":"2024-04-25T09:54:30.086552+0000","last_fullsized":"2024-04-25T09:54:30.086552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:55.089097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994577+0000","last_change":"2024-04-25T09:54:23.970661+0000","last_active":"2024-04-25T09:54:29.994577+0000","last_peered":"2024-04-25T09:54:29.994577+0000","last_clean":"2024-04-25T09:54:29.994577+0000","last_became_active":"2024-04-25T09:54:23.970422+0000","last_became_peered":"2024-04-25T09:54:23.970422+0000","last_unstale":"2024-04-25T09:54:29.994577+0000","last_undegraded":"2024-04-25T09:54:29.994577+0000","last_fullsized":"2024-04-25T09:54:29.994577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994799+0000","last_change":"2024-04-25T09:54:22.960394+0000","last_active":"2024-04-25T09:54:29.994799+0000","last_peered":"2024-04-25T09:54:29.994799+0000","last_clean":"2024-04-25T09:54:29.994799+0000","last_became_active":"2024-04-25T09:54:22.960128+0000","last_became_peered":"2024-04-25T09:54:22.960128+0000","last_unstale":"2024-04-25T09:54:29.994799+0000","last_undegraded":"2024-04-25T09:54:29.994799+0000","last_fullsized":"2024-04-25T09:54:29.994799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:33.803787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086079+0000","last_change":"2024-04-25T09:54:22.962677+0000","last_active":"2024-04-25T09:54:30.086079+0000","last_peered":"2024-04-25T09:54:30.086079+0000","last_clean":"2024-04-25T09:54:30.086079+0000","last_became_active":"2024-04-25T09:54:22.962351+0000","last_became_peered":"2024-04-25T09:54:22.962351+0000","last_unstale":"2024-04-25T09:54:30.086079+0000","last_undegraded":"2024-04-25T09:54:30.086079+0000","last_fullsized":"2024-04-25T09:54:30.086079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:11.165570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086533+0000","last_change":"2024-04-25T09:54:23.973688+0000","last_active":"2024-04-25T09:54:30.086533+0000","last_peered":"2024-04-25T09:54:30.086533+0000","last_clean":"2024-04-25T09:54:30.086533+0000","last_became_active":"2024-04-25T09:54:23.973424+0000","last_became_peered":"2024-04-25T09:54:23.973424+0000","last_unstale":"2024-04-25T09:54:30.086533+0000","last_undegraded":"2024-04-25T09:54:30.086533+0000","last_fullsized":"2024-04-25T09:54:30.086533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086930+0000","last_change":"2024-04-25T09:54:28.016164+0000","last_active":"2024-04-25T09:54:30.086930+0000","last_peered":"2024-04-25T09:54:30.086930+0000","last_clean":"2024-04-25T09:54:30.086930+0000","last_became_active":"2024-04-25T09:54:28.015634+0000","last_became_peered":"2024-04-25T09:54:28.015634+0000","last_unstale":"2024-04-25T09:54:30.086930+0000","last_undegraded":"2024-04-25T09:54:30.086930+0000","last_fullsized":"2024-04-25T09:54:30.086930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.111330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994585+0000","last_change":"2024-04-25T09:54:22.962095+0000","last_active":"2024-04-25T09:54:29.994585+0000","last_peered":"2024-04-25T09:54:29.994585+0000","last_clean":"2024-04-25T09:54:29.994585+0000","last_became_active":"2024-04-25T09:54:22.961801+0000","last_became_peered":"2024-04-25T09:54:22.961801+0000","last_unstale":"2024-04-25T09:54:29.994585+0000","last_undegraded":"2024-04-25T09:54:29.994585+0000","last_fullsized":"2024-04-25T09:54:29.994585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:21.935503+0000","last_clean_scrub_stamp":"2024-04-25T09:54:21.935503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:01.533820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995467+0000","last_change":"2024-04-25T09:54:23.971722+0000","last_active":"2024-04-25T09:54:29.995467+0000","last_peered":"2024-04-25T09:54:29.995467+0000","last_clean":"2024-04-25T09:54:29.995467+0000","last_became_active":"2024-04-25T09:54:23.971437+0000","last_became_peered":"2024-04-25T09:54:23.971437+0000","last_unstale":"2024-04-25T09:54:29.995467+0000","last_undegraded":"2024-04-25T09:54:29.995467+0000","last_fullsized":"2024-04-25T09:54:29.995467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:28.933707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994926+0000","last_change":"2024-04-25T09:54:28.016742+0000","last_active":"2024-04-25T09:54:29.994926+0000","last_peered":"2024-04-25T09:54:29.994926+0000","last_clean":"2024-04-25T09:54:29.994926+0000","last_became_active":"2024-04-25T09:54:28.016017+0000","last_became_peered":"2024-04-25T09:54:28.016017+0000","last_unstale":"2024-04-25T09:54:29.994926+0000","last_undegraded":"2024-04-25T09:54:29.994926+0000","last_fullsized":"2024-04-25T09:54:29.994926+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:26.979196+0000","last_clean_scrub_stamp":"2024-04-25T09:54:26.979196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:51.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085710+0000","last_change":"2024-04-25T09:54:23.976967+0000","last_active":"2024-04-25T09:54:30.085710+0000","last_peered":"2024-04-25T09:54:30.085710+0000","last_clean":"2024-04-25T09:54:30.085710+0000","last_became_active":"2024-04-25T09:54:23.976684+0000","last_became_peered":"2024-04-25T09:54:23.976684+0000","last_unstale":"2024-04-25T09:54:30.085710+0000","last_undegraded":"2024-04-25T09:54:30.085710+0000","last_fullsized":"2024-04-25T09:54:30.085710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086490+0000","last_change":"2024-04-25T09:54:23.968076+0000","last_active":"2024-04-25T09:54:30.086490+0000","last_peered":"2024-04-25T09:54:30.086490+0000","last_clean":"2024-04-25T09:54:30.086490+0000","last_became_active":"2024-04-25T09:54:23.967823+0000","last_became_peered":"2024-04-25T09:54:23.967823+0000","last_unstale":"2024-04-25T09:54:30.086490+0000","last_undegraded":"2024-04-25T09:54:30.086490+0000","last_fullsized":"2024-04-25T09:54:30.086490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.809421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087064+0000","last_change":"2024-04-25T09:54:23.974924+0000","last_active":"2024-04-25T09:54:30.087064+0000","last_peered":"2024-04-25T09:54:30.087064+0000","last_clean":"2024-04-25T09:54:30.087064+0000","last_became_active":"2024-04-25T09:54:23.974647+0000","last_became_peered":"2024-04-25T09:54:23.974647+0000","last_unstale":"2024-04-25T09:54:30.087064+0000","last_undegraded":"2024-04-25T09:54:30.087064+0000","last_fullsized":"2024-04-25T09:54:30.087064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:37.726625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086431+0000","last_change":"2024-04-25T09:54:23.974599+0000","last_active":"2024-04-25T09:54:30.086431+0000","last_peered":"2024-04-25T09:54:30.086431+0000","last_clean":"2024-04-25T09:54:30.086431+0000","last_became_active":"2024-04-25T09:54:23.973958+0000","last_became_peered":"2024-04-25T09:54:23.973958+0000","last_unstale":"2024-04-25T09:54:30.086431+0000","last_undegraded":"2024-04-25T09:54:30.086431+0000","last_fullsized":"2024-04-25T09:54:30.086431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994701+0000","last_change":"2024-04-25T09:54:23.976111+0000","last_active":"2024-04-25T09:54:29.994701+0000","last_peered":"2024-04-25T09:54:29.994701+0000","last_clean":"2024-04-25T09:54:29.994701+0000","last_became_active":"2024-04-25T09:54:23.975817+0000","last_became_peered":"2024-04-25T09:54:23.975817+0000","last_unstale":"2024-04-25T09:54:29.994701+0000","last_undegraded":"2024-04-25T09:54:29.994701+0000","last_fullsized":"2024-04-25T09:54:29.994701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086971+0000","last_change":"2024-04-25T09:54:23.967777+0000","last_active":"2024-04-25T09:54:30.086971+0000","last_peered":"2024-04-25T09:54:30.086971+0000","last_clean":"2024-04-25T09:54:30.086971+0000","last_became_active":"2024-04-25T09:54:23.967099+0000","last_became_peered":"2024-04-25T09:54:23.967099+0000","last_unstale":"2024-04-25T09:54:30.086971+0000","last_undegraded":"2024-04-25T09:54:30.086971+0000","last_fullsized":"2024-04-25T09:54:30.086971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086631+0000","last_change":"2024-04-25T09:54:23.982361+0000","last_active":"2024-04-25T09:54:30.086631+0000","last_peered":"2024-04-25T09:54:30.086631+0000","last_clean":"2024-04-25T09:54:30.086631+0000","last_became_active":"2024-04-25T09:54:23.982020+0000","last_became_peered":"2024-04-25T09:54:23.982020+0000","last_unstale":"2024-04-25T09:54:30.086631+0000","last_undegraded":"2024-04-25T09:54:30.086631+0000","last_fullsized":"2024-04-25T09:54:30.086631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995121+0000","last_change":"2024-04-25T09:54:23.976994+0000","last_active":"2024-04-25T09:54:29.995121+0000","last_peered":"2024-04-25T09:54:29.995121+0000","last_clean":"2024-04-25T09:54:29.995121+0000","last_became_active":"2024-04-25T09:54:23.976707+0000","last_became_peered":"2024-04-25T09:54:23.976707+0000","last_unstale":"2024-04-25T09:54:29.995121+0000","last_undegraded":"2024-04-25T09:54:29.995121+0000","last_fullsized":"2024-04-25T09:54:29.995121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086418+0000","last_change":"2024-04-25T09:54:23.977123+0000","last_active":"2024-04-25T09:54:30.086418+0000","last_peered":"2024-04-25T09:54:30.086418+0000","last_clean":"2024-04-25T09:54:30.086418+0000","last_became_active":"2024-04-25T09:54:23.976810+0000","last_became_peered":"2024-04-25T09:54:23.976810+0000","last_unstale":"2024-04-25T09:54:30.086418+0000","last_undegraded":"2024-04-25T09:54:30.086418+0000","last_fullsized":"2024-04-25T09:54:30.086418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994535+0000","last_change":"2024-04-25T09:54:23.976035+0000","last_active":"2024-04-25T09:54:29.994535+0000","last_peered":"2024-04-25T09:54:29.994535+0000","last_clean":"2024-04-25T09:54:29.994535+0000","last_became_active":"2024-04-25T09:54:23.975755+0000","last_became_peered":"2024-04-25T09:54:23.975755+0000","last_unstale":"2024-04-25T09:54:29.994535+0000","last_undegraded":"2024-04-25T09:54:29.994535+0000","last_fullsized":"2024-04-25T09:54:29.994535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:07.194799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086969+0000","last_change":"2024-04-25T09:54:23.973340+0000","last_active":"2024-04-25T09:54:30.086969+0000","last_peered":"2024-04-25T09:54:30.086969+0000","last_clean":"2024-04-25T09:54:30.086969+0000","last_became_active":"2024-04-25T09:54:23.973010+0000","last_became_peered":"2024-04-25T09:54:23.973010+0000","last_unstale":"2024-04-25T09:54:30.086969+0000","last_undegraded":"2024-04-25T09:54:30.086969+0000","last_fullsized":"2024-04-25T09:54:30.086969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086318+0000","last_change":"2024-04-25T09:54:23.980705+0000","last_active":"2024-04-25T09:54:30.086318+0000","last_peered":"2024-04-25T09:54:30.086318+0000","last_clean":"2024-04-25T09:54:30.086318+0000","last_became_active":"2024-04-25T09:54:23.980411+0000","last_became_peered":"2024-04-25T09:54:23.980411+0000","last_unstale":"2024-04-25T09:54:30.086318+0000","last_undegraded":"2024-04-25T09:54:30.086318+0000","last_fullsized":"2024-04-25T09:54:30.086318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086259+0000","last_change":"2024-04-25T09:54:23.984091+0000","last_active":"2024-04-25T09:54:30.086259+0000","last_peered":"2024-04-25T09:54:30.086259+0000","last_clean":"2024-04-25T09:54:30.086259+0000","last_became_active":"2024-04-25T09:54:23.983834+0000","last_became_peered":"2024-04-25T09:54:23.983834+0000","last_unstale":"2024-04-25T09:54:30.086259+0000","last_undegraded":"2024-04-25T09:54:30.086259+0000","last_fullsized":"2024-04-25T09:54:30.086259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:07.661233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088036+0000","last_change":"2024-04-25T09:54:23.983353+0000","last_active":"2024-04-25T09:54:30.088036+0000","last_peered":"2024-04-25T09:54:30.088036+0000","last_clean":"2024-04-25T09:54:30.088036+0000","last_became_active":"2024-04-25T09:54:23.983043+0000","last_became_peered":"2024-04-25T09:54:23.983043+0000","last_unstale":"2024-04-25T09:54:30.088036+0000","last_undegraded":"2024-04-25T09:54:30.088036+0000","last_fullsized":"2024-04-25T09:54:30.088036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:10.359311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086130+0000","last_change":"2024-04-25T09:54:23.977174+0000","last_active":"2024-04-25T09:54:30.086130+0000","last_peered":"2024-04-25T09:54:30.086130+0000","last_clean":"2024-04-25T09:54:30.086130+0000","last_became_active":"2024-04-25T09:54:23.976903+0000","last_became_peered":"2024-04-25T09:54:23.976903+0000","last_unstale":"2024-04-25T09:54:30.086130+0000","last_undegraded":"2024-04-25T09:54:30.086130+0000","last_fullsized":"2024-04-25T09:54:30.086130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:12.542579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087382+0000","last_change":"2024-04-25T09:54:23.972492+0000","last_active":"2024-04-25T09:54:30.087382+0000","last_peered":"2024-04-25T09:54:30.087382+0000","last_clean":"2024-04-25T09:54:30.087382+0000","last_became_active":"2024-04-25T09:54:23.972188+0000","last_became_peered":"2024-04-25T09:54:23.972188+0000","last_unstale":"2024-04-25T09:54:30.087382+0000","last_undegraded":"2024-04-25T09:54:30.087382+0000","last_fullsized":"2024-04-25T09:54:30.087382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:51.049409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086368+0000","last_change":"2024-04-25T09:54:23.978371+0000","last_active":"2024-04-25T09:54:30.086368+0000","last_peered":"2024-04-25T09:54:30.086368+0000","last_clean":"2024-04-25T09:54:30.086368+0000","last_became_active":"2024-04-25T09:54:23.978079+0000","last_became_peered":"2024-04-25T09:54:23.978079+0000","last_unstale":"2024-04-25T09:54:30.086368+0000","last_undegraded":"2024-04-25T09:54:30.086368+0000","last_fullsized":"2024-04-25T09:54:30.086368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086280+0000","last_change":"2024-04-25T09:54:23.976031+0000","last_active":"2024-04-25T09:54:30.086280+0000","last_peered":"2024-04-25T09:54:30.086280+0000","last_clean":"2024-04-25T09:54:30.086280+0000","last_became_active":"2024-04-25T09:54:23.975750+0000","last_became_peered":"2024-04-25T09:54:23.975750+0000","last_unstale":"2024-04-25T09:54:30.086280+0000","last_undegraded":"2024-04-25T09:54:30.086280+0000","last_fullsized":"2024-04-25T09:54:30.086280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:34.673061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085955+0000","last_change":"2024-04-25T09:54:23.973561+0000","last_active":"2024-04-25T09:54:30.085955+0000","last_peered":"2024-04-25T09:54:30.085955+0000","last_clean":"2024-04-25T09:54:30.085955+0000","last_became_active":"2024-04-25T09:54:23.973312+0000","last_became_peered":"2024-04-25T09:54:23.973312+0000","last_unstale":"2024-04-25T09:54:30.085955+0000","last_undegraded":"2024-04-25T09:54:30.085955+0000","last_fullsized":"2024-04-25T09:54:30.085955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:00.438251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994677+0000","last_change":"2024-04-25T09:54:23.969586+0000","last_active":"2024-04-25T09:54:29.994677+0000","last_peered":"2024-04-25T09:54:29.994677+0000","last_clean":"2024-04-25T09:54:29.994677+0000","last_became_active":"2024-04-25T09:54:23.969067+0000","last_became_peered":"2024-04-25T09:54:23.969067+0000","last_unstale":"2024-04-25T09:54:29.994677+0000","last_undegraded":"2024-04-25T09:54:29.994677+0000","last_fullsized":"2024-04-25T09:54:29.994677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:24.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.995863+0000","last_change":"2024-04-25T09:54:23.972621+0000","last_active":"2024-04-25T09:54:29.995863+0000","last_peered":"2024-04-25T09:54:29.995863+0000","last_clean":"2024-04-25T09:54:29.995863+0000","last_became_active":"2024-04-25T09:54:23.972338+0000","last_became_peered":"2024-04-25T09:54:23.972338+0000","last_unstale":"2024-04-25T09:54:29.995863+0000","last_undegraded":"2024-04-25T09:54:29.995863+0000","last_fullsized":"2024-04-25T09:54:29.995863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:11.959162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086120+0000","last_change":"2024-04-25T09:54:23.971085+0000","last_active":"2024-04-25T09:54:30.086120+0000","last_peered":"2024-04-25T09:54:30.086120+0000","last_clean":"2024-04-25T09:54:30.086120+0000","last_became_active":"2024-04-25T09:54:23.970354+0000","last_became_peered":"2024-04-25T09:54:23.970354+0000","last_unstale":"2024-04-25T09:54:30.086120+0000","last_undegraded":"2024-04-25T09:54:30.086120+0000","last_fullsized":"2024-04-25T09:54:30.086120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:59.560612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086765+0000","last_change":"2024-04-25T09:54:23.970478+0000","last_active":"2024-04-25T09:54:30.086765+0000","last_peered":"2024-04-25T09:54:30.086765+0000","last_clean":"2024-04-25T09:54:30.086765+0000","last_became_active":"2024-04-25T09:54:23.970181+0000","last_became_peered":"2024-04-25T09:54:23.970181+0000","last_unstale":"2024-04-25T09:54:30.086765+0000","last_undegraded":"2024-04-25T09:54:30.086765+0000","last_fullsized":"2024-04-25T09:54:30.086765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:37.132496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088285+0000","last_change":"2024-04-25T09:54:23.983654+0000","last_active":"2024-04-25T09:54:30.088285+0000","last_peered":"2024-04-25T09:54:30.088285+0000","last_clean":"2024-04-25T09:54:30.088285+0000","last_became_active":"2024-04-25T09:54:23.983388+0000","last_became_peered":"2024-04-25T09:54:23.983388+0000","last_unstale":"2024-04-25T09:54:30.088285+0000","last_undegraded":"2024-04-25T09:54:30.088285+0000","last_fullsized":"2024-04-25T09:54:30.088285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:14.120866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:29.994460+0000","last_change":"2024-04-25T09:54:23.970049+0000","last_active":"2024-04-25T09:54:29.994460+0000","last_peered":"2024-04-25T09:54:29.994460+0000","last_clean":"2024-04-25T09:54:29.994460+0000","last_became_active":"2024-04-25T09:54:23.969605+0000","last_became_peered":"2024-04-25T09:54:23.969605+0000","last_unstale":"2024-04-25T09:54:29.994460+0000","last_undegraded":"2024-04-25T09:54:29.994460+0000","last_fullsized":"2024-04-25T09:54:29.994460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.403701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087574+0000","last_change":"2024-04-25T09:54:23.980721+0000","last_active":"2024-04-25T09:54:30.087574+0000","last_peered":"2024-04-25T09:54:30.087574+0000","last_clean":"2024-04-25T09:54:30.087574+0000","last_became_active":"2024-04-25T09:54:23.980440+0000","last_became_peered":"2024-04-25T09:54:23.980440+0000","last_unstale":"2024-04-25T09:54:30.087574+0000","last_undegraded":"2024-04-25T09:54:30.087574+0000","last_fullsized":"2024-04-25T09:54:30.087574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086883+0000","last_change":"2024-04-25T09:54:23.967904+0000","last_active":"2024-04-25T09:54:30.086883+0000","last_peered":"2024-04-25T09:54:30.086883+0000","last_clean":"2024-04-25T09:54:30.086883+0000","last_became_active":"2024-04-25T09:54:23.967585+0000","last_became_peered":"2024-04-25T09:54:23.967585+0000","last_unstale":"2024-04-25T09:54:30.086883+0000","last_undegraded":"2024-04-25T09:54:30.086883+0000","last_fullsized":"2024-04-25T09:54:30.086883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:11.278492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.087670+0000","last_change":"2024-04-25T09:54:23.978911+0000","last_active":"2024-04-25T09:54:30.087670+0000","last_peered":"2024-04-25T09:54:30.087670+0000","last_clean":"2024-04-25T09:54:30.087670+0000","last_became_active":"2024-04-25T09:54:23.978625+0000","last_became_peered":"2024-04-25T09:54:23.978625+0000","last_unstale":"2024-04-25T09:54:30.087670+0000","last_undegraded":"2024-04-25T09:54:30.087670+0000","last_fullsized":"2024-04-25T09:54:30.087670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086202+0000","last_change":"2024-04-25T09:54:23.976392+0000","last_active":"2024-04-25T09:54:30.086202+0000","last_peered":"2024-04-25T09:54:30.086202+0000","last_clean":"2024-04-25T09:54:30.086202+0000","last_became_active":"2024-04-25T09:54:23.976117+0000","last_became_peered":"2024-04-25T09:54:23.976117+0000","last_unstale":"2024-04-25T09:54:30.086202+0000","last_undegraded":"2024-04-25T09:54:30.086202+0000","last_fullsized":"2024-04-25T09:54:30.086202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:18.245586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.085739+0000","last_change":"2024-04-25T09:54:23.967710+0000","last_active":"2024-04-25T09:54:30.085739+0000","last_peered":"2024-04-25T09:54:30.085739+0000","last_clean":"2024-04-25T09:54:30.085739+0000","last_became_active":"2024-04-25T09:54:23.967075+0000","last_became_peered":"2024-04-25T09:54:23.967075+0000","last_unstale":"2024-04-25T09:54:30.085739+0000","last_undegraded":"2024-04-25T09:54:30.085739+0000","last_fullsized":"2024-04-25T09:54:30.085739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:22.329777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.086748+0000","last_change":"2024-04-25T09:54:23.968977+0000","last_active":"2024-04-25T09:54:30.086748+0000","last_peered":"2024-04-25T09:54:30.086748+0000","last_clean":"2024-04-25T09:54:30.086748+0000","last_became_active":"2024-04-25T09:54:23.966936+0000","last_became_peered":"2024-04-25T09:54:23.966936+0000","last_unstale":"2024-04-25T09:54:30.086748+0000","last_undegraded":"2024-04-25T09:54:30.086748+0000","last_fullsized":"2024-04-25T09:54:30.086748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:44.221960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T09:54:30.088179+0000","last_change":"2024-04-25T09:54:23.987914+0000","last_active":"2024-04-25T09:54:30.088179+0000","last_peered":"2024-04-25T09:54:30.088179+0000","last_clean":"2024-04-25T09:54:30.088179+0000","last_became_active":"2024-04-25T09:54:23.985923+0000","last_became_peered":"2024-04-25T09:54:23.985923+0000","last_unstale":"2024-04-25T09:54:30.088179+0000","last_undegraded":"2024-04-25T09:54:30.088179+0000","last_fullsized":"2024-04-25T09:54:30.088179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:22.939508+0000","last_clean_scrub_stamp":"2024-04-25T09:54:22.939508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:36.938793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:54:50.708 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:54:50.708 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T09:54:50.708 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T09:54:50.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:54:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.838+0000 7f32e6363640 1 Processor -- start 2024-04-25T09:54:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.838+0000 7f32e6363640 1 -- start start 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.838+0000 7f32e6363640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32e00ce920 0x7f32e00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.838+0000 7f32e6363640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e0144b30 con 0x7f32e006d290 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.838+0000 7f32e6363640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e00712d0 con 0x7f32e0144750 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.838+0000 7f32e6363640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e00715b0 con 0x7f32e00ce920 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32df7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32e00ce920 0x7f32e00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32df7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f32e00ce920 0x7f32e00cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33870/0 (socket says 172.21.15.148:33870) 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32df7fe640 1 -- 172.21.15.148:0/708701786 learned_addr learned my addr 172.21.15.148:0/708701786 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32e4900640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f32e006d290 0x7f32e006d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48860/0 (socket says 172.21.15.148:48860) 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32df7fe640 1 -- 172.21.15.148:0/708701786 >> v1:172.21.15.188:6789/0 conn(0x7f32e0144750 legacy=0x7f32e006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32df7fe640 1 -- 172.21.15.148:0/708701786 >> v1:172.21.15.148:6789/0 conn(0x7f32e006d290 legacy=0x7f32e006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32df7fe640 1 -- 172.21.15.148:0/708701786 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0071890 con 0x7f32e00ce920 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.839+0000 7f32deffd640 1 -- 172.21.15.148:0/708701786 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379893735 0 0) 0x7f32e0144b30 con 0x7f32e006d290 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32df7fe640 1 --2- 172.21.15.148:0/708701786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e00ce920 0x7f32e00cedb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009870 tx=0x7f32c802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cb334f2fff4210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32deffd640 1 -- 172.21.15.148:0/708701786 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8037070 con 0x7f32e00ce920 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32deffd640 1 -- 172.21.15.148:0/708701786 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8032440 con 0x7f32e00ce920 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32deffd640 1 -- 172.21.15.148:0/708701786 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8032810 con 0x7f32e00ce920 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 -- 172.21.15.148:0/708701786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e00ce920 msgr2=0x7f32e00cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 --2- 172.21.15.148:0/708701786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e00ce920 0x7f32e00cedb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009870 tx=0x7f32c802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 -- 172.21.15.148:0/708701786 shutdown_connections 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 --2- 172.21.15.148:0/708701786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e00ce920 0x7f32e00cedb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 -- 172.21.15.148:0/708701786 >> 172.21.15.148:0/708701786 conn(0x7f32e006abe0 msgr2=0x7f32e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 -- 172.21.15.148:0/708701786 shutdown_connections 2024-04-25T09:54:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.840+0000 7f32e6363640 1 -- 172.21.15.148:0/708701786 wait complete. 2024-04-25T09:54:50.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.841+0000 7f32e6363640 1 Processor -- start 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 -- start start 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 0x7f32e015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00ce920 0x7f32e015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e0144750 0x7f32e00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e006f020 con 0x7f32e006d290 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e006ece0 con 0x7f32e00ce920 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32e6363640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0145450 con 0x7f32e0144750 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32dffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 0x7f32e015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32dffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 0x7f32e015c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33886/0 (socket says 172.21.15.148:33886) 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.842+0000 7f32dffff640 1 -- 172.21.15.148:0/1229861480 learned_addr learned my addr 172.21.15.148:0/1229861480 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32df7fe640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00ce920 0x7f32e015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32e4900640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e0144750 0x7f32e00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dffff640 1 -- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e0144750 msgr2=0x7f32e00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dffff640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e0144750 0x7f32e00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dffff640 1 -- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00ce920 msgr2=0x7f32e015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dffff640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00ce920 0x7f32e015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dffff640 1 -- 172.21.15.148:0/1229861480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00b2160 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32df7fe640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00ce920 0x7f32e015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32e4900640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e0144750 0x7f32e00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dffff640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 0x7f32e015c000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f32d000c460 tx=0x7f32d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0016020 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d0012870 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0012c60 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e015c970 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.843+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e015cf30 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.844+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0073990 con 0x7f32e006d290 2024-04-25T09:54:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.845+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32d0019020 con 0x7f32e006d290 2024-04-25T09:54:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.845+0000 7f32dcff9640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32c006b4d0 0x7f32c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.845+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f32d008a260 con 0x7f32e006d290 2024-04-25T09:54:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.845+0000 7f32df7fe640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32c006b4d0 0x7f32c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.846+0000 7f32df7fe640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32c006b4d0 0x7f32c006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32c8034000 tx=0x7f32c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:50.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:50.849+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32d0020030 con 0x7f32e006d290 2024-04-25T09:54:51.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.056+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f32e00d0510 con 0x7f32e006d290 2024-04-25T09:54:51.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.057+0000 7f32dcff9640 1 -- 172.21.15.148:0/1229861480 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f32d0058da0 con 0x7f32e006d290 2024-04-25T09:54:51.060 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:51.060 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32c006b4d0 msgr2=0x7f32c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32c006b4d0 0x7f32c006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32c8034000 tx=0x7f32c802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 msgr2=0x7f32e015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 0x7f32e015c000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f32d000c460 tx=0x7f32d000c920 comp rx=0 tx=0).stop 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 shutdown_connections 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f32c006b4d0 0x7f32c006d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e0144750 0x7f32e00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00ce920 0x7f32e015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 --2- 172.21.15.148:0/1229861480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32e006d290 0x7f32e015c000 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 >> 172.21.15.148:0/1229861480 conn(0x7f32e006abe0 msgr2=0x7f32e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 shutdown_connections 2024-04-25T09:54:51.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.059+0000 7f32e6363640 1 -- 172.21.15.148:0/1229861480 wait complete. 2024-04-25T09:54:51.072 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T09:54:51.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:51.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.169+0000 7f4d6f0b6640 1 Processor -- start 2024-04-25T09:54:51.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.169+0000 7f4d6f0b6640 1 -- start start 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6f0b6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d68144750 0x7f4d6806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6f0b6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4d6806eb10 0x7f4d680ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6f0b6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d68144b30 con 0x7f4d680cee60 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6f0b6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d680712d0 con 0x7f4d6806eb10 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6f0b6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d680715b0 con 0x7f4d68144750 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4d6806eb10 0x7f4d680ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4d6806eb10 0x7f4d680ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33894/0 (socket says 172.21.15.148:33894) 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 -- 172.21.15.148:0/1098223504 learned_addr learned my addr 172.21.15.148:0/1098223504 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6ce52640 1 --2- 172.21.15.148:0/1098223504 >> v2:172.21.15.188:3300/0 conn(0x7f4d68144750 0x7f4d6806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 -- 172.21.15.148:0/1098223504 >> v2:172.21.15.188:3300/0 conn(0x7f4d68144750 msgr2=0x7f4d6806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 --2- 172.21.15.148:0/1098223504 >> v2:172.21.15.188:3300/0 conn(0x7f4d68144750 0x7f4d6806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 -- 172.21.15.148:0/1098223504 >> v1:172.21.15.188:6789/0 conn(0x7f4d680cee60 legacy=0x7f4d6815f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d67fff640 1 -- 172.21.15.148:0/1098223504 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68071890 con 0x7f4d6806eb10 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.170+0000 7f4d6ce52640 1 --2- 172.21.15.148:0/1098223504 >> v2:172.21.15.188:3300/0 conn(0x7f4d68144750 0x7f4d6806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:51.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d67fff640 1 --2- 172.21.15.148:0/1098223504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d6806eb10 0x7f4d680ce920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d6806f5d0 tx=0x7f4d5802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49157a659dd8a367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d677fe640 1 -- 172.21.15.148:0/1098223504 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58036070 con 0x7f4d6806eb10 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d677fe640 1 -- 172.21.15.148:0/1098223504 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58031600 con 0x7f4d6806eb10 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d677fe640 1 -- 172.21.15.148:0/1098223504 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580319d0 con 0x7f4d6806eb10 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/1098223504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d6806eb10 msgr2=0x7f4d680ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/1098223504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d6806eb10 0x7f4d680ce920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d6806f5d0 tx=0x7f4d5802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/1098223504 shutdown_connections 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/1098223504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d6806eb10 0x7f4d680ce920 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/1098223504 >> v2:172.21.15.188:3300/0 conn(0x7f4d68144750 0x7f4d6806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/1098223504 >> 172.21.15.148:0/1098223504 conn(0x7f4d6806abe0 msgr2=0x7f4d6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/1098223504 shutdown_connections 2024-04-25T09:54:51.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.171+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/1098223504 wait complete. 2024-04-25T09:54:51.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.172+0000 7f4d6f0b6640 1 Processor -- start 2024-04-25T09:54:51.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 -- start start 2024-04-25T09:54:51.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6806eb10 0x7f4d6815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 0x7f4d68156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68144750 0x7f4d6815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68145450 con 0x7f4d680cee60 2024-04-25T09:54:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6806f020 con 0x7f4d6806eb10 2024-04-25T09:54:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d6f0b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681455d0 con 0x7f4d68144750 2024-04-25T09:54:51.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 0x7f4d68156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.173+0000 7f4d67fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 0x7f4d68156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33910/0 (socket says 172.21.15.148:33910) 2024-04-25T09:54:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 -- 172.21.15.148:0/3777872944 learned_addr learned my addr 172.21.15.148:0/3777872944 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d6ce52640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6806eb10 0x7f4d6815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d6d653640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68144750 0x7f4d6815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 -- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68144750 msgr2=0x7f4d6815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68144750 0x7f4d6815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 -- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6806eb10 msgr2=0x7f4d6815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6806eb10 0x7f4d6815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 -- 172.21.15.148:0/3777872944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68157170 con 0x7f4d680cee60 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d6d653640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68144750 0x7f4d6815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d6ce52640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6806eb10 0x7f4d6815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d67fff640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 0x7f4d68156c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009f40 tx=0x7f4d58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58036050 con 0x7f4d680cee60 2024-04-25T09:54:51.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58042070 con 0x7f4d680cee60 2024-04-25T09:54:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6815f1d0 con 0x7f4d680cee60 2024-04-25T09:54:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58009510 con 0x7f4d680cee60 2024-04-25T09:54:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.174+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6815f790 con 0x7f4d680cee60 2024-04-25T09:54:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.175+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68073990 con 0x7f4d680cee60 2024-04-25T09:54:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.176+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d58007c50 con 0x7f4d680cee60 2024-04-25T09:54:51.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.176+0000 7f4d657fa640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4d4806b500 0x7f4d4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.176+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4d580ab420 con 0x7f4d680cee60 2024-04-25T09:54:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.176+0000 7f4d6ce52640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4d4806b500 0x7f4d4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.177+0000 7f4d6ce52640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4d4806b500 0x7f4d4806d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000aa30 tx=0x7f4d60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.180+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d58079ff0 con 0x7f4d680cee60 2024-04-25T09:54:51.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.352+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d6806afc0 con 0x7f4d680cee60 2024-04-25T09:54:51.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.353+0000 7f4d657fa640 1 -- 172.21.15.148:0/3777872944 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f4d580798f0 con 0x7f4d680cee60 2024-04-25T09:54:51.356 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:51.356 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:51.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.355+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4d4806b500 msgr2=0x7f4d4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.355+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4d4806b500 0x7f4d4806d9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000aa30 tx=0x7f4d60008040 comp rx=0 tx=0).stop 2024-04-25T09:54:51.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.355+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 msgr2=0x7f4d68156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.355+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 0x7f4d68156c30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009f40 tx=0x7f4d58004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:51.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 shutdown_connections 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4d4806b500 0x7f4d4806d9c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d68144750 0x7f4d6815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d680cee60 0x7f4d68156c30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 --2- 172.21.15.148:0/3777872944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d6806eb10 0x7f4d6815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 >> 172.21.15.148:0/3777872944 conn(0x7f4d6806abe0 msgr2=0x7f4d6813fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 shutdown_connections 2024-04-25T09:54:51.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.356+0000 7f4d6f0b6640 1 -- 172.21.15.148:0/3777872944 wait complete. 2024-04-25T09:54:51.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:51.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:51.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.467+0000 7fd21f15c640 1 Processor -- start 2024-04-25T09:54:51.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.467+0000 7fd21f15c640 1 -- start start 2024-04-25T09:54:51.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.467+0000 7fd21f15c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2180b5570 0x7fd21806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21f15c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2180d1d50 con 0x7fd218158230 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21f15c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2180d2020 con 0x7fd21806c330 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21f15c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd218158610 con 0x7fd2180b5570 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21d6f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2180b5570 0x7fd21806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21cef8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd218158230 0x7fd2180d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48866/0 (socket says 172.21.15.148:48866) 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21d6f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2180b5570 0x7fd21806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33918/0 (socket says 172.21.15.148:33918) 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21cef8640 1 -- 172.21.15.148:0/2943350284 learned_addr learned my addr 172.21.15.148:0/2943350284 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21d6f9640 1 -- 172.21.15.148:0/2943350284 >> v1:172.21.15.188:6789/0 conn(0x7fd21806c330 legacy=0x7fd2180b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd2177fe640 1 -- 172.21.15.148:0/2943350284 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989872143 0 0) 0x7fd2180d1d50 con 0x7fd218158230 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21d6f9640 1 -- 172.21.15.148:0/2943350284 >> v1:172.21.15.148:6789/0 conn(0x7fd218158230 legacy=0x7fd2180d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.468+0000 7fd21d6f9640 1 -- 172.21.15.148:0/2943350284 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180d2300 con 0x7fd2180b5570 2024-04-25T09:54:51.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21d6f9640 1 --2- 172.21.15.148:0/2943350284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2180b5570 0x7fd21806f4c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd210009d90 tx=0x7fd21002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94635991bb91464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd2177fe640 1 -- 172.21.15.148:0/2943350284 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210007740 con 0x7fd2180b5570 2024-04-25T09:54:51.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd2177fe640 1 -- 172.21.15.148:0/2943350284 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2100078e0 con 0x7fd2180b5570 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd2177fe640 1 -- 172.21.15.148:0/2943350284 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210007cb0 con 0x7fd2180b5570 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 -- 172.21.15.148:0/2943350284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2180b5570 msgr2=0x7fd21806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 --2- 172.21.15.148:0/2943350284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2180b5570 0x7fd21806f4c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd210009d90 tx=0x7fd21002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 -- 172.21.15.148:0/2943350284 shutdown_connections 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 --2- 172.21.15.148:0/2943350284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2180b5570 0x7fd21806f4c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 -- 172.21.15.148:0/2943350284 >> 172.21.15.148:0/2943350284 conn(0x7fd218061600 msgr2=0x7fd2180619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 -- 172.21.15.148:0/2943350284 shutdown_connections 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.469+0000 7fd21f15c640 1 -- 172.21.15.148:0/2943350284 wait complete. 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.470+0000 7fd21f15c640 1 Processor -- start 2024-04-25T09:54:51.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 -- start start 2024-04-25T09:54:51.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21806c330 0x7fd2180cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 0x7fd21815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218143180 0x7fd21815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180d25e0 con 0x7fd218158230 2024-04-25T09:54:51.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21806bea0 con 0x7fd218143180 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd21f15c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21815e320 con 0x7fd21806c330 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd217fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 0x7fd21815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd217fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 0x7fd21815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33928/0 (socket says 172.21.15.148:33928) 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.471+0000 7fd217fff640 1 -- 172.21.15.148:0/875843885 learned_addr learned my addr 172.21.15.148:0/875843885 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd217fff640 1 -- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21806c330 msgr2=0x7fd2180cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd21cef8640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21806c330 0x7fd2180cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd21d6f9640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218143180 0x7fd21815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd217fff640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21806c330 0x7fd2180cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd217fff640 1 -- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218143180 msgr2=0x7fd21815e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd217fff640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218143180 0x7fd21815e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd217fff640 1 -- 172.21.15.148:0/875843885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21815cf20 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd21d6f9640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218143180 0x7fd21815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd21cef8640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21806c330 0x7fd2180cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd217fff640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 0x7fd21815c9e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c3a0 tx=0x7fd20400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204017070 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2040052f0 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204003640 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21815d1c0 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.472+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd218158790 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.473+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd218073a00 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.473+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd20401a070 con 0x7fd218158230 2024-04-25T09:54:51.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.474+0000 7fd2157fa640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.474+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd204089730 con 0x7fd218158230 2024-04-25T09:54:51.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.474+0000 7fd21cef8640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.474+0000 7fd21cef8640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd20c000f60 tx=0x7fd20c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.478+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd204057cb0 con 0x7fd218158230 2024-04-25T09:54:51.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.647+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd21806e530 con 0x7fd218158230 2024-04-25T09:54:51.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.648+0000 7fd2157fa640 1 -- 172.21.15.148:0/875843885 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd20405bd50 con 0x7fd218158230 2024-04-25T09:54:51.650 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:54:51.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd1fc06b4d0 msgr2=0x7fd1fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd20c000f60 tx=0x7fd20c009040 comp rx=0 tx=0).stop 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 msgr2=0x7fd21815c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 0x7fd21815c9e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c3a0 tx=0x7fd20400c860 comp rx=0 tx=0).stop 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 shutdown_connections 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218143180 0x7fd21815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd218158230 0x7fd21815c9e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 --2- 172.21.15.148:0/875843885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21806c330 0x7fd2180cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 >> 172.21.15.148:0/875843885 conn(0x7fd218061600 msgr2=0x7fd2180b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 shutdown_connections 2024-04-25T09:54:51.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.650+0000 7fd21f15c640 1 -- 172.21.15.148:0/875843885 wait complete. 2024-04-25T09:54:51.663 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:51.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.760+0000 7fbc8d058640 1 Processor -- start 2024-04-25T09:54:51.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.760+0000 7fbc8d058640 1 -- start start 2024-04-25T09:54:51.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.761+0000 7fbc8d058640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc880d00e0 0x7fbc880ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.761+0000 7fbc8d058640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880cb190 con 0x7fbc8815bb20 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.761+0000 7fbc8d058640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880cb470 con 0x7fbc880ccdd0 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.761+0000 7fbc8d058640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880cb750 con 0x7fbc880d00e0 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.761+0000 7fbc86d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbc8815bb20 0x7fbc880cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48882/0 (socket says 172.21.15.148:48882) 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.761+0000 7fbc86d9d640 1 -- 172.21.15.148:0/3351739833 learned_addr learned my addr 172.21.15.148:0/3351739833 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979280639 0 0) 0x7fbc880cb190 con 0x7fbc8815bb20 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8759e640 1 --2- 172.21.15.148:0/3351739833 >> v2:172.21.15.188:3300/0 conn(0x7fbc880d00e0 0x7fbc880ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc70003520 con 0x7fbc8815bb20 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037975938 0 0) 0x7fbc70003520 con 0x7fbc8815bb20 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 >> v2:172.21.15.188:3300/0 conn(0x7fbc880d00e0 msgr2=0x7fbc880ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 --2- 172.21.15.148:0/3351739833 >> v2:172.21.15.188:3300/0 conn(0x7fbc880d00e0 0x7fbc880ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 >> v1:172.21.15.188:6789/0 conn(0x7fbc880ccdd0 legacy=0x7fbc880cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880cba30 con 0x7fbc8815bb20 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbc78001670 con 0x7fbc8815bb20 2024-04-25T09:54:51.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 >> v1:172.21.15.148:6789/0 conn(0x7fbc8815bb20 legacy=0x7fbc880cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880cb750 con 0x7fbc70004800 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7000c160 con 0x7fbc700039b0 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7000c360 con 0x7fbc700051c0 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc86d9d640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8759e640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8659c640 1 -- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc700051c0 msgr2=0x7fbc7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8659c640 1 -- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc700039b0 msgr2=0x7fbc70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8659c640 1 -- 172.21.15.148:0/3351739833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70003520 con 0x7fbc70004800 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8759e640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.762+0000 7fbc8759e640 1 -- 172.21.15.148:0/3351739833 reap_dead start 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.763+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc70004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c001750 tx=0x7fbc70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337f2f63f8049707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.763+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c001370 con 0x7fbc70004800 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.763+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0052d0 con 0x7fbc70004800 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.763+0000 7fbc85d9b640 1 -- 172.21.15.148:0/3351739833 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c02abc0 con 0x7fbc70004800 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 -- 172.21.15.148:0/3351739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 msgr2=0x7fbc70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc70004c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c001750 tx=0x7fbc70005830 comp rx=0 tx=0).stop 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 -- 172.21.15.148:0/3351739833 shutdown_connections 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3351739833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc70004c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 -- 172.21.15.148:0/3351739833 >> 172.21.15.148:0/3351739833 conn(0x7fbc880b2390 msgr2=0x7fbc880b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 -- 172.21.15.148:0/3351739833 shutdown_connections 2024-04-25T09:54:51.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.764+0000 7fbc8d058640 1 -- 172.21.15.148:0/3351739833 wait complete. 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.765+0000 7fbc8d058640 1 Processor -- start 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.765+0000 7fbc8d058640 1 -- start start 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8d058640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc880c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8d058640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc880c7360 0x7fbc8814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8d058640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc880c8370 0x7fbc88159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8d058640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8815c080 con 0x7fbc70004800 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8d058640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8815bf00 con 0x7fbc880c8370 2024-04-25T09:54:51.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8d058640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880cadb0 con 0x7fbc880c7360 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc880c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc880c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33944/0 (socket says 172.21.15.148:33944) 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 -- 172.21.15.148:0/3660922430 learned_addr learned my addr 172.21.15.148:0/3660922430 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc880c7360 0x7fbc8814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc8759e640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc880c8370 0x7fbc88159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 -- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc880c7360 msgr2=0x7fbc8814e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc880c7360 0x7fbc8814e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 -- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc880c8370 msgr2=0x7fbc88159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc880c8370 0x7fbc88159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.766+0000 7fbc86d9d640 1 -- 172.21.15.148:0/3660922430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880c78d0 con 0x7fbc70004800 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.767+0000 7fbc86d9d640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc880c6840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbc78004f20 tx=0x7fbc780074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.767+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800f7b0 con 0x7fbc70004800 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.767+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78018070 con 0x7fbc70004800 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.767+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8806f460 con 0x7fbc70004800 2024-04-25T09:54:51.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.767+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800fbb0 con 0x7fbc70004800 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.767+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc880c7b70 con 0x7fbc70004800 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.768+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc88073a00 con 0x7fbc70004800 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.768+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc7800fd50 con 0x7fbc70004800 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.769+0000 7fbc6f7fe640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc7406b4d0 0x7fbc7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.769+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc7406b4d0 0x7fbc7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.769+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbc7808a360 con 0x7fbc70004800 2024-04-25T09:54:51.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.769+0000 7fbc8659c640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc7406b4d0 0x7fbc7406d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c032000 tx=0x7fbc7c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:51.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.773+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc78058850 con 0x7fbc70004800 2024-04-25T09:54:51.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.944+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc8814e930 con 0x7fbc70004800 2024-04-25T09:54:51.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.944+0000 7fbc6f7fe640 1 -- 172.21.15.148:0/3660922430 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc7805c8f0 con 0x7fbc70004800 2024-04-25T09:54:51.947 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc7406b4d0 msgr2=0x7fbc7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc7406b4d0 0x7fbc7406d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c032000 tx=0x7fbc7c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 msgr2=0x7fbc880c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc880c6840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbc78004f20 tx=0x7fbc780074f0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 shutdown_connections 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc7406b4d0 0x7fbc7406d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc880c8370 0x7fbc88159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc880c7360 0x7fbc8814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 --2- 172.21.15.148:0/3660922430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc70004800 0x7fbc880c6840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:51.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 >> 172.21.15.148:0/3660922430 conn(0x7fbc880b2390 msgr2=0x7fbc880b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:51.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 shutdown_connections 2024-04-25T09:54:51.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:51.946+0000 7fbc8d058640 1 -- 172.21.15.148:0/3660922430 wait complete. 2024-04-25T09:54:51.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:52.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.060+0000 7f357a69e640 1 Processor -- start 2024-04-25T09:54:52.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.060+0000 7f357a69e640 1 -- start start 2024-04-25T09:54:52.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f357a69e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f357415b7d0 0x7f35740cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f357a69e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35740b5620 0x7f35740b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f357a69e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357406cf50 con 0x7f357406c2d0 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f357a69e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740ce530 con 0x7f357415b7d0 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f357a69e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740ce810 con 0x7f35740b5620 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f357415b7d0 0x7f35740cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f357415b7d0 0x7f35740cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33948/0 (socket says 172.21.15.148:33948) 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 -- 172.21.15.148:0/3251104087 learned_addr learned my addr 172.21.15.148:0/3251104087 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f35737fe640 1 --2- 172.21.15.148:0/3251104087 >> v2:172.21.15.188:3300/0 conn(0x7f35740b5620 0x7f35740b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 -- 172.21.15.148:0/3251104087 >> v2:172.21.15.188:3300/0 conn(0x7f35740b5620 msgr2=0x7f35740b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 --2- 172.21.15.148:0/3251104087 >> v2:172.21.15.188:3300/0 conn(0x7f35740b5620 0x7f35740b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 -- 172.21.15.148:0/3251104087 >> v1:172.21.15.188:6789/0 conn(0x7f357406c2d0 legacy=0x7f357406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f3573fff640 1 -- 172.21.15.148:0/3251104087 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740ceaf0 con 0x7f357415b7d0 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.061+0000 7f35737fe640 1 --2- 172.21.15.148:0/3251104087 >> v2:172.21.15.188:3300/0 conn(0x7f35740b5620 0x7f35740b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f3573fff640 1 --2- 172.21.15.148:0/3251104087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cf010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f35740ce180 tx=0x7f356802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d9aa0abb2eacc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f3572ffd640 1 -- 172.21.15.148:0/3251104087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568037070 con 0x7f357415b7d0 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f3572ffd640 1 -- 172.21.15.148:0/3251104087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3568032440 con 0x7f357415b7d0 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f3572ffd640 1 -- 172.21.15.148:0/3251104087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568032810 con 0x7f357415b7d0 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 -- 172.21.15.148:0/3251104087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 msgr2=0x7f35740cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 --2- 172.21.15.148:0/3251104087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cf010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f35740ce180 tx=0x7f356802a230 comp rx=0 tx=0).stop 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 -- 172.21.15.148:0/3251104087 shutdown_connections 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 --2- 172.21.15.148:0/3251104087 >> v2:172.21.15.188:3300/0 conn(0x7f35740b5620 0x7f35740b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 --2- 172.21.15.148:0/3251104087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cf010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 -- 172.21.15.148:0/3251104087 >> 172.21.15.148:0/3251104087 conn(0x7f35740d0800 msgr2=0x7f35740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.062+0000 7f357a69e640 1 -- 172.21.15.148:0/3251104087 shutdown_connections 2024-04-25T09:54:52.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.063+0000 7f357a69e640 1 -- 172.21.15.148:0/3251104087 wait complete. 2024-04-25T09:54:52.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 Processor -- start 2024-04-25T09:54:52.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 -- start start 2024-04-25T09:54:52.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357406c2d0 0x7f357414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740b5620 0x7f3574149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740cf6d0 con 0x7f357415b7d0 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740cf550 con 0x7f35740b5620 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.064+0000 7f357a69e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357415bbb0 con 0x7f357406c2d0 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33964/0 (socket says 172.21.15.148:33964) 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 -- 172.21.15.148:0/268012504 learned_addr learned my addr 172.21.15.148:0/268012504 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f35737fe640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740b5620 0x7f3574149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3573fff640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357406c2d0 0x7f357414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 -- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357406c2d0 msgr2=0x7f357414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357406c2d0 0x7f357414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 -- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740b5620 msgr2=0x7f3574149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740b5620 0x7f3574149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 -- 172.21.15.148:0/268012504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357414ba90 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3573fff640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357406c2d0 0x7f357414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3578c3b640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cd570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f356c00c780 tx=0x7f356c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c004b10 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c004cb0 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00aa90 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357414bd70 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.065+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35740cd050 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.066+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740c8340 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.067+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f356c00ac30 con 0x7f357415b7d0 2024-04-25T09:54:52.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.067+0000 7f3570ff9640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f356006b4d0 0x7f356006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.067+0000 7f3573fff640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f356006b4d0 0x7f356006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.067+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f356c08a6b0 con 0x7f357415b7d0 2024-04-25T09:54:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.068+0000 7f3573fff640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f356006b4d0 0x7f356006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3568034000 tx=0x7f356802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.071+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f356c020070 con 0x7f357415b7d0 2024-04-25T09:54:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.247+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35740739e0 con 0x7f357415b7d0 2024-04-25T09:54:52.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.248+0000 7f3570ff9640 1 -- 172.21.15.148:0/268012504 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f356c0591f0 con 0x7f357415b7d0 2024-04-25T09:54:52.251 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:54:52.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f356006b4d0 msgr2=0x7f356006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f356006b4d0 0x7f356006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3568034000 tx=0x7f356802e000 comp rx=0 tx=0).stop 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 msgr2=0x7f35740cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cd570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f356c00c780 tx=0x7f356c00cc40 comp rx=0 tx=0).stop 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 shutdown_connections 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f356006b4d0 0x7f356006d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357415b7d0 0x7f35740cd570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35740b5620 0x7f3574149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 --2- 172.21.15.148:0/268012504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357406c2d0 0x7f357414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 >> 172.21.15.148:0/268012504 conn(0x7f35740d0800 msgr2=0x7f35740d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 shutdown_connections 2024-04-25T09:54:52.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.250+0000 7f357a69e640 1 -- 172.21.15.148:0/268012504 wait complete. 2024-04-25T09:54:52.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:52.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.361+0000 7f01c86ee640 1 Processor -- start 2024-04-25T09:54:52.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.361+0000 7f01c86ee640 1 -- start start 2024-04-25T09:54:52.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c86ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01c01446d0 0x7f01c00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c86ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01c006e8a0 0x7f01c00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c86ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c00cccc0 con 0x7f01c0144d50 2024-04-25T09:54:52.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c86ee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c00b2b20 con 0x7f01c01446d0 2024-04-25T09:54:52.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c86ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c006d400 con 0x7f01c006e8a0 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c648a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01c01446d0 0x7f01c00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c648a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01c01446d0 0x7f01c00b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33976/0 (socket says 172.21.15.148:33976) 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.362+0000 7f01c648a640 1 -- 172.21.15.148:0/1155868553 learned_addr learned my addr 172.21.15.148:0/1155868553 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c6c8b640 1 --2- 172.21.15.148:0/1155868553 >> v2:172.21.15.188:3300/0 conn(0x7f01c006e8a0 0x7f01c00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c648a640 1 -- 172.21.15.148:0/1155868553 >> v2:172.21.15.188:3300/0 conn(0x7f01c006e8a0 msgr2=0x7f01c00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c648a640 1 --2- 172.21.15.148:0/1155868553 >> v2:172.21.15.188:3300/0 conn(0x7f01c006e8a0 0x7f01c00b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c648a640 1 -- 172.21.15.148:0/1155868553 >> v1:172.21.15.188:6789/0 conn(0x7f01c0144d50 legacy=0x7f01c00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c648a640 1 -- 172.21.15.148:0/1155868553 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c00cc140 con 0x7f01c01446d0 2024-04-25T09:54:52.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c648a640 1 --2- 172.21.15.148:0/1155868553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00b3f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01c0071200 tx=0x7f01bc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53de2510eb6ac1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.363+0000 7f01c5488640 1 -- 172.21.15.148:0/1155868553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc036070 con 0x7f01c01446d0 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c5488640 1 -- 172.21.15.148:0/1155868553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc031600 con 0x7f01c01446d0 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c5488640 1 -- 172.21.15.148:0/1155868553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0319d0 con 0x7f01c01446d0 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 -- 172.21.15.148:0/1155868553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 msgr2=0x7f01c00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 --2- 172.21.15.148:0/1155868553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00b3f20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01c0071200 tx=0x7f01bc02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 -- 172.21.15.148:0/1155868553 shutdown_connections 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 --2- 172.21.15.148:0/1155868553 >> v2:172.21.15.188:3300/0 conn(0x7f01c006e8a0 0x7f01c00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 --2- 172.21.15.148:0/1155868553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00b3f20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 -- 172.21.15.148:0/1155868553 >> 172.21.15.148:0/1155868553 conn(0x7f01c006abe0 msgr2=0x7f01c006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 -- 172.21.15.148:0/1155868553 shutdown_connections 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.364+0000 7f01c86ee640 1 -- 172.21.15.148:0/1155868553 wait complete. 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.365+0000 7f01c86ee640 1 Processor -- start 2024-04-25T09:54:52.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.365+0000 7f01c86ee640 1 -- start start 2024-04-25T09:54:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c86ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c006e8a0 0x7f01c01434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c86ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c86ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c0144d50 0x7f01c00ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c86ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c00cc700 con 0x7f01c01446d0 2024-04-25T09:54:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c86ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c00b49a0 con 0x7f01c0144d50 2024-04-25T09:54:52.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c86ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c00cc9e0 con 0x7f01c006e8a0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33990/0 (socket says 172.21.15.148:33990) 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 -- 172.21.15.148:0/3541343391 learned_addr learned my addr 172.21.15.148:0/3541343391 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c648a640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c006e8a0 0x7f01c01434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c6c8b640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c0144d50 0x7f01c00ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 -- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c006e8a0 msgr2=0x7f01c01434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c006e8a0 0x7f01c01434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 -- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c0144d50 msgr2=0x7f01c00ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c0144d50 0x7f01c00ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.366+0000 7f01c5c89640 1 -- 172.21.15.148:0/3541343391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c00ce260 con 0x7f01c01446d0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.367+0000 7f01c5c89640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00cd2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b3e0 tx=0x7f01b000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.367+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0015020 con 0x7f01c01446d0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.367+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b0011850 con 0x7f01c01446d0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.367+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c00cdca0 con 0x7f01c01446d0 2024-04-25T09:54:52.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.367+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0011c40 con 0x7f01c01446d0 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.367+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c00ac950 con 0x7f01c01446d0 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.368+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c00ace40 con 0x7f01c01446d0 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.368+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01b0018020 con 0x7f01c01446d0 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.369+0000 7f01b6ffd640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a406b4d0 0x7f01a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.369+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f01b008a3d0 con 0x7f01c01446d0 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.369+0000 7f01c648a640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a406b4d0 0x7f01a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.369+0000 7f01c648a640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a406b4d0 0x7f01a406d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f01bc033000 tx=0x7f01bc009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.373+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01b001f020 con 0x7f01c01446d0 2024-04-25T09:54:52.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.542+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f01c0073a00 con 0x7f01c01446d0 2024-04-25T09:54:52.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.543+0000 7f01b6ffd640 1 -- 172.21.15.148:0/3541343391 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f01b00172c0 con 0x7f01c01446d0 2024-04-25T09:54:52.546 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:52.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a406b4d0 msgr2=0x7f01a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a406b4d0 0x7f01a406d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f01bc033000 tx=0x7f01bc009000 comp rx=0 tx=0).stop 2024-04-25T09:54:52.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 msgr2=0x7f01c00cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00cd2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b3e0 tx=0x7f01b000b8a0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 shutdown_connections 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a406b4d0 0x7f01a406d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c0144d50 0x7f01c00ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01c01446d0 0x7f01c00cd2e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 --2- 172.21.15.148:0/3541343391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c006e8a0 0x7f01c01434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 >> 172.21.15.148:0/3541343391 conn(0x7f01c006abe0 msgr2=0x7f01c00cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 shutdown_connections 2024-04-25T09:54:52.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.545+0000 7f01c86ee640 1 -- 172.21.15.148:0/3541343391 wait complete. 2024-04-25T09:54:52.558 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:52.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.653+0000 7f5628438640 1 Processor -- start 2024-04-25T09:54:52.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.653+0000 7f5628438640 1 -- start start 2024-04-25T09:54:52.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f5628438640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 0x7f56200c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f5628438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 0x7f56200ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f5628438640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562015dfe0 con 0x7f56200ceb20 2024-04-25T09:54:52.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f5628438640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56200d00c0 con 0x7f562015dc00 2024-04-25T09:54:52.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f5628438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200d03a0 con 0x7f56200d1dd0 2024-04-25T09:54:52.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f56269d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 0x7f56200ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f56261d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 0x7f56200c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f56269d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 0x7f56200ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:56668/0 (socket says 172.21.15.148:56668) 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f56259d3640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f56200ceb20 0x7f56200d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43264/0 (socket says 172.21.15.148:43264) 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f56261d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 0x7f56200c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57780/0 (socket says 172.21.15.148:57780) 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.654+0000 7f56269d5640 1 -- 172.21.15.148:0/3634749353 learned_addr learned my addr 172.21.15.148:0/3634749353 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56269d5640 1 -- 172.21.15.148:0/3634749353 >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 msgr2=0x7f56200c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56269d5640 1 --2- 172.21.15.148:0/3634749353 >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 0x7f56200c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56269d5640 1 -- 172.21.15.148:0/3634749353 >> v1:172.21.15.188:6789/0 conn(0x7f56200ceb20 legacy=0x7f56200d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56269d5640 1 -- 172.21.15.148:0/3634749353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200cc440 con 0x7f56200d1dd0 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56261d4640 1 --2- 172.21.15.148:0/3634749353 >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 0x7f56200c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:52.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56269d5640 1 --2- 172.21.15.148:0/3634749353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 0x7f56200ccc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5618009e50 tx=0x7f561802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce4a54a9aaef109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56251d2640 1 -- 172.21.15.148:0/3634749353 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5618036070 con 0x7f56200d1dd0 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56251d2640 1 -- 172.21.15.148:0/3634749353 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5618031600 con 0x7f56200d1dd0 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.655+0000 7f56251d2640 1 -- 172.21.15.148:0/3634749353 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56180319d0 con 0x7f56200d1dd0 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 -- 172.21.15.148:0/3634749353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 msgr2=0x7f56200ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 --2- 172.21.15.148:0/3634749353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 0x7f56200ccc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5618009e50 tx=0x7f561802a710 comp rx=0 tx=0).stop 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 -- 172.21.15.148:0/3634749353 shutdown_connections 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 --2- 172.21.15.148:0/3634749353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56200d1dd0 0x7f56200ccc40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 --2- 172.21.15.148:0/3634749353 >> v2:172.21.15.188:3300/0 conn(0x7f562015dc00 0x7f56200c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 -- 172.21.15.148:0/3634749353 >> 172.21.15.148:0/3634749353 conn(0x7f562006f5d0 msgr2=0x7f562006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 -- 172.21.15.148:0/3634749353 shutdown_connections 2024-04-25T09:54:52.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.656+0000 7f5628438640 1 -- 172.21.15.148:0/3634749353 wait complete. 2024-04-25T09:54:52.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.657+0000 7f5628438640 1 Processor -- start 2024-04-25T09:54:52.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.657+0000 7f5628438640 1 -- start start 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f5628438640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200ceb20 0x7f5620151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f5628438640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 0x7f56201560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f5628438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562015dc00 0x7f5620156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f5628438640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cfd90 con 0x7f56200d1dd0 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f5628438640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cc130 con 0x7f56200ceb20 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f5628438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cdd70 con 0x7f562015dc00 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 0x7f56201560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 0x7f56201560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34006/0 (socket says 172.21.15.148:34006) 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 -- 172.21.15.148:0/1942376837 learned_addr learned my addr 172.21.15.148:0/1942376837 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56269d5640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562015dc00 0x7f5620156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56261d4640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200ceb20 0x7f5620151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 -- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562015dc00 msgr2=0x7f5620156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562015dc00 0x7f5620156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 -- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200ceb20 msgr2=0x7f5620151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200ceb20 0x7f5620151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.658+0000 7f56259d3640 1 -- 172.21.15.148:0/1942376837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200cb6d0 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.659+0000 7f56259d3640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 0x7f56201560c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f561000c400 tx=0x7f561000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.659+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56100074e0 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.659+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610007680 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.659+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56200cb110 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.659+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610004a00 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.659+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5620157d10 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.660+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5620073990 con 0x7f56200d1dd0 2024-04-25T09:54:52.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.660+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f561001a020 con 0x7f56200d1dd0 2024-04-25T09:54:52.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.661+0000 7f5616ffd640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560406b4d0 0x7f560406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.661+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5610089ba0 con 0x7f56200d1dd0 2024-04-25T09:54:52.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.661+0000 7f56261d4640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560406b4d0 0x7f560406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.661+0000 7f56261d4640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560406b4d0 0x7f560406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5620156d10 tx=0x7f561c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.665+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f562015dfe0 con 0x7f56200d1dd0 2024-04-25T09:54:52.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.871+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56200ce340 con 0x7f56200d1dd0 2024-04-25T09:54:52.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.872+0000 7f5616ffd640 1 -- 172.21.15.148:0/1942376837 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f5610089020 con 0x7f56200d1dd0 2024-04-25T09:54:52.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:52.875 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:52.875 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:54:52.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560406b4d0 msgr2=0x7f560406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560406b4d0 0x7f560406d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5620156d10 tx=0x7f561c009190 comp rx=0 tx=0).stop 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 msgr2=0x7f56201560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 0x7f56201560c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f561000c400 tx=0x7f561000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 shutdown_connections 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560406b4d0 0x7f560406d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562015dc00 0x7f5620156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56200d1dd0 0x7f56201560c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 --2- 172.21.15.148:0/1942376837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200ceb20 0x7f5620151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 >> 172.21.15.148:0/1942376837 conn(0x7f562006f5d0 msgr2=0x7f56200c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 shutdown_connections 2024-04-25T09:54:52.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.874+0000 7f5628438640 1 -- 172.21.15.148:0/1942376837 wait complete. 2024-04-25T09:54:52.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:52.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.981+0000 7f812fbc8640 1 Processor -- start 2024-04-25T09:54:52.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.982+0000 7f812fbc8640 1 -- start start 2024-04-25T09:54:52.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.982+0000 7f812fbc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81280ce7a0 0x7f81280cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.982+0000 7f812fbc8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81280cd6e0 con 0x7f81280cdb40 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.982+0000 7f812fbc8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81280cc640 con 0x7f812815b950 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.982+0000 7f812fbc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81280cc3f0 con 0x7f81280ce7a0 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812e165640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f81280cdb40 0x7f81280c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48894/0 (socket says 172.21.15.148:48894) 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812e165640 1 -- 172.21.15.148:0/4288580420 learned_addr learned my addr 172.21.15.148:0/4288580420 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812d163640 1 --2- 172.21.15.148:0/4288580420 >> v2:172.21.15.188:3300/0 conn(0x7f81280ce7a0 0x7f81280cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834354319 0 0) 0x7f81280cd6e0 con 0x7f81280cdb40 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8114003520 con 0x7f81280cdb40 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111007750 0 0) 0x7f81280cc640 con 0x7f812815b950 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81280cd6e0 con 0x7f812815b950 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989224860 0 0) 0x7f8114003520 con 0x7f81280cdb40 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 >> v2:172.21.15.188:3300/0 conn(0x7f81280ce7a0 msgr2=0x7f81280cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 --2- 172.21.15.148:0/4288580420 >> v2:172.21.15.188:3300/0 conn(0x7f81280ce7a0 0x7f81280cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 >> v1:172.21.15.188:6789/0 conn(0x7f812815b950 legacy=0x7f81280ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280cc920 con 0x7f81280cdb40 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f8124001670 con 0x7f81280cdb40 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 >> v1:172.21.15.148:6789/0 conn(0x7f81280cdb40 legacy=0x7f81280c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8114003b90 0x7f8114003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8114004800 0x7f8114004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f811400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280cc3f0 con 0x7f81140051c0 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811400bfe0 con 0x7f8114003b90 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811400c1e0 con 0x7f8114004800 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812d163640 1 --2- 172.21.15.148:0/4288580420 >> v2:172.21.15.188:3300/0 conn(0x7f81280ce7a0 0x7f81280cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812e165640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f811400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.983+0000 7f812d964640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8114003b90 0x7f8114003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812d163640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8114004800 0x7f8114004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 -- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8114004800 msgr2=0x7f8114004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8114004800 0x7f8114004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 -- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8114003b90 msgr2=0x7f8114003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8114003b90 0x7f8114003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 -- 172.21.15.148:0/4288580420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114003520 con 0x7f81140051c0 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 -- 172.21.15.148:0/4288580420 reap_dead start 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812e165640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f811400b9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8124007fc0 tx=0x7f8114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a798003fba1349f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124007be0 con 0x7f81140051c0 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124005ad0 con 0x7f81140051c0 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.984+0000 7f812c962640 1 -- 172.21.15.148:0/4288580420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124007070 con 0x7f81140051c0 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 -- 172.21.15.148:0/4288580420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 msgr2=0x7f811400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f811400b9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8124007fc0 tx=0x7f8114005830 comp rx=0 tx=0).stop 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 -- 172.21.15.148:0/4288580420 shutdown_connections 2024-04-25T09:54:52.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 --2- 172.21.15.148:0/4288580420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f811400b9c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 -- 172.21.15.148:0/4288580420 >> 172.21.15.148:0/4288580420 conn(0x7f81280b33e0 msgr2=0x7f81280b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:52.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 -- 172.21.15.148:0/4288580420 shutdown_connections 2024-04-25T09:54:52.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.985+0000 7f812fbc8640 1 -- 172.21.15.148:0/4288580420 wait complete. 2024-04-25T09:54:52.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.986+0000 7f812fbc8640 1 Processor -- start 2024-04-25T09:54:52.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.986+0000 7f812fbc8640 1 -- start start 2024-04-25T09:54:52.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812fbc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f8128148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812fbc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81281e6310 0x7f81281e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812fbc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8128148a60 0x7f8128148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812fbc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280cd1c0 con 0x7f81140051c0 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812fbc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280ccee0 con 0x7f8128148a60 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812fbc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280d02d0 con 0x7f81281e6310 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f8128148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f8128148140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34018/0 (socket says 172.21.15.148:34018) 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 -- 172.21.15.148:0/2031835334 learned_addr learned my addr 172.21.15.148:0/2031835334 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d163640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81281e6310 0x7f81281e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812e165640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8128148a60 0x7f8128148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 -- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81281e6310 msgr2=0x7f81281e6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81281e6310 0x7f81281e6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 -- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8128148a60 msgr2=0x7f8128148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8128148a60 0x7f8128148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.987+0000 7f812d964640 1 -- 172.21.15.148:0/2031835334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81281e5270 con 0x7f81140051c0 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.988+0000 7f812d964640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f8128148140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f812000fb10 tx=0x7f812000ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.988+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812001a070 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.988+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120006d60 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.988+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81281e5550 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.988+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120003b90 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.988+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81281e5b10 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.989+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8128073990 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.989+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f812001d070 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.990+0000 7f811e7fc640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f810c06b500 0x7f810c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.990+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f812008a020 con 0x7f81140051c0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.990+0000 7f812d163640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f810c06b500 0x7f810c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:52.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.990+0000 7f812d163640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f810c06b500 0x7f810c06d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81280cf010 tx=0x7f8118008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:52.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:52.994+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8120058480 con 0x7f81140051c0 2024-04-25T09:54:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.167+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81280b1730 con 0x7f81140051c0 2024-04-25T09:54:53.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.168+0000 7f811e7fc640 1 -- 172.21.15.148:0/2031835334 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7f812005c520 con 0x7f81140051c0 2024-04-25T09:54:53.171 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:53.171 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:53.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f810c06b500 msgr2=0x7f810c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f810c06b500 0x7f810c06d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f81280cf010 tx=0x7f8118008040 comp rx=0 tx=0).stop 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 msgr2=0x7f8128148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f8128148140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f812000fb10 tx=0x7f812000ff10 comp rx=0 tx=0).stop 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 shutdown_connections 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f810c06b500 0x7f810c06d9c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8128148a60 0x7f8128148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81281e6310 0x7f81281e6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 --2- 172.21.15.148:0/2031835334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81140051c0 0x7f8128148140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 >> 172.21.15.148:0/2031835334 conn(0x7f81280b33e0 msgr2=0x7f8128070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 shutdown_connections 2024-04-25T09:54:53.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.170+0000 7f812fbc8640 1 -- 172.21.15.148:0/2031835334 wait complete. 2024-04-25T09:54:53.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:54:53.184 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:54:53.227 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:53.230 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:54:53.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.283+0000 7fb7b9f70640 1 Processor -- start 2024-04-25T09:54:53.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.283+0000 7fb7b9f70640 1 -- start start 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b9f70640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb7b40ccdd0 0x7fb7b40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b9f70640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7b40cdcd0 0x7fb7b40cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b9f70640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40cb190 con 0x7fb7b415bb20 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b9f70640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40d0450 con 0x7fb7b40cdcd0 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b9f70640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40cb470 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7b40cdcd0 0x7fb7b40cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb7b415bb20 0x7fb7b40cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:48904/0 (socket says 172.21.15.148:48904) 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.284+0000 7fb7b37fe640 1 -- 172.21.15.148:0/3668684518 learned_addr learned my addr 172.21.15.148:0/3668684518 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:53.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b3fff640 1 -- 172.21.15.148:0/3668684518 >> v2:172.21.15.188:3300/0 conn(0x7fb7b40ccdd0 msgr2=0x7fb7b40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b3fff640 1 --2- 172.21.15.148:0/3668684518 >> v2:172.21.15.188:3300/0 conn(0x7fb7b40ccdd0 0x7fb7b40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b3fff640 1 -- 172.21.15.148:0/3668684518 >> v1:172.21.15.148:6789/0 conn(0x7fb7b415bb20 legacy=0x7fb7b40cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b3fff640 1 -- 172.21.15.148:0/3668684518 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40cb750 con 0x7fb7b40cdcd0 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b27fc640 1 -- 172.21.15.148:0/3668684518 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476641062 0 0) 0x7fb7b40cb190 con 0x7fb7b415bb20 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b3fff640 1 --2- 172.21.15.148:0/3668684518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b40cff10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4071680 tx=0x7fb7ac02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53aa824c701feb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b27fc640 1 -- 172.21.15.148:0/3668684518 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac007740 con 0x7fb7b40cdcd0 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b27fc640 1 -- 172.21.15.148:0/3668684518 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0078e0 con 0x7fb7b40cdcd0 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.285+0000 7fb7b27fc640 1 -- 172.21.15.148:0/3668684518 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac007cb0 con 0x7fb7b40cdcd0 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 -- 172.21.15.148:0/3668684518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40cdcd0 msgr2=0x7fb7b40cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/3668684518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b40cff10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4071680 tx=0x7fb7ac02a5f0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 -- 172.21.15.148:0/3668684518 shutdown_connections 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/3668684518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b40cff10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/3668684518 >> v2:172.21.15.188:3300/0 conn(0x7fb7b40ccdd0 0x7fb7b40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 -- 172.21.15.148:0/3668684518 >> 172.21.15.148:0/3668684518 conn(0x7fb7b40b2390 msgr2=0x7fb7b40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 -- 172.21.15.148:0/3668684518 shutdown_connections 2024-04-25T09:54:53.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.286+0000 7fb7b9f70640 1 -- 172.21.15.148:0/3668684518 wait complete. 2024-04-25T09:54:53.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.287+0000 7fb7b9f70640 1 Processor -- start 2024-04-25T09:54:53.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.287+0000 7fb7b9f70640 1 -- start start 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b9f70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 0x7fb7b41f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b9f70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b41f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b9f70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b415bb20 0x7fb7b41484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b9f70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b415bf00 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b9f70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cbd10 con 0x7fb7b40cdcd0 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b9f70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b415c080 con 0x7fb7b415bb20 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 0x7fb7b41f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 0x7fb7b41f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34034/0 (socket says 172.21.15.148:34034) 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b37fe640 1 -- 172.21.15.148:0/2026497936 learned_addr learned my addr 172.21.15.148:0/2026497936 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b2ffd640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b41f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b3fff640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b415bb20 0x7fb7b41484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b37fe640 1 -- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b415bb20 msgr2=0x7fb7b41484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.288+0000 7fb7b37fe640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b415bb20 0x7fb7b41484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b37fe640 1 -- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b40cdcd0 msgr2=0x7fb7b41f0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b37fe640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b41f0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b37fe640 1 -- 172.21.15.148:0/2026497936 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4147d30 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b2ffd640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b41f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b37fe640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 0x7fb7b41f4810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c430 tx=0x7fb7a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40074e0 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a40047e0 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b41f4d50 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4004bd0 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.289+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41f5300 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.290+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b4073a00 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.290+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7a4011070 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.291+0000 7fb79bfff640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7a006b4d0 0x7fb7a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.291+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb7a408a200 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.291+0000 7fb7b2ffd640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7a006b4d0 0x7fb7a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.291+0000 7fb7b2ffd640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7a006b4d0 0x7fb7a006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7b415a090 tx=0x7fb7a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.295+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7a4058d40 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.467+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7b406e5b0 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.468+0000 7fb79bfff640 1 -- 172.21.15.148:0/2026497936 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7a4058640 con 0x7fb7b40ccdd0 2024-04-25T09:54:53.470 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7a006b4d0 msgr2=0x7fb7a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7a006b4d0 0x7fb7a006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7b415a090 tx=0x7fb7a8008040 comp rx=0 tx=0).stop 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 msgr2=0x7fb7b41f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 0x7fb7b41f4810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c430 tx=0x7fb7a400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 shutdown_connections 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7a006b4d0 0x7fb7a006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7b415bb20 0x7fb7b41484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7b40cdcd0 0x7fb7b41f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 --2- 172.21.15.148:0/2026497936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7b40ccdd0 0x7fb7b41f4810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 >> 172.21.15.148:0/2026497936 conn(0x7fb7b40b2390 msgr2=0x7fb7b4156360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 shutdown_connections 2024-04-25T09:54:53.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.470+0000 7fb7b9f70640 1 -- 172.21.15.148:0/2026497936 wait complete. 2024-04-25T09:54:53.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:54:53.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.621+0000 7f4fdc2ac640 1 Processor -- start 2024-04-25T09:54:53.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.621+0000 7f4fdc2ac640 1 -- start start 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.622+0000 7f4fdc2ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 0x7f4fd406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.622+0000 7f4fdc2ac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40d1770 con 0x7f4fd40711a0 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.622+0000 7f4fdc2ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd4162740 con 0x7f4fd415deb0 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.622+0000 7f4fdc2ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40b0eb0 con 0x7f4fd4070dc0 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9847640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4fd40711a0 0x7f4fd40b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51346/0 (socket says 172.21.15.148:51346) 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9847640 1 -- 172.21.15.148:0/3735845048 learned_addr learned my addr 172.21.15.148:0/3735845048 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fda048640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 0x7f4fd406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406154633 0 0) 0x7f4fd40d1770 con 0x7f4fd40711a0 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc0003520 con 0x7f4fd40711a0 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677047745 0 0) 0x7f4fc0003520 con 0x7f4fd40711a0 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 msgr2=0x7f4fd406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 0x7f4fd406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 >> v1:172.21.15.188:6789/0 conn(0x7f4fd415deb0 legacy=0x7f4fd415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4161980 con 0x7f4fd40711a0 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4fc4001670 con 0x7f4fd40711a0 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 >> v1:172.21.15.148:6789/0 conn(0x7f4fd40711a0 legacy=0x7f4fd40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 0x7f4fc00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0003f60 0x7f4fc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc0004d40 0x7f4fc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40b0eb0 con 0x7f4fc0004d40 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc000bc00 con 0x7f4fc0003f60 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc000be00 con 0x7f4fd4070dc0 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fda849640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc0004d40 0x7f4fc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fda048640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 0x7f4fc00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.623+0000 7f4fd9847640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0003f60 0x7f4fc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fda849640 1 -- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 msgr2=0x7f4fc00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fda849640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4070dc0 0x7f4fc00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fda849640 1 -- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0003f60 msgr2=0x7f4fc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fda849640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0003f60 0x7f4fc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fda849640 1 -- 172.21.15.148:0/3735845048 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0003520 con 0x7f4fc0004d40 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fda849640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc0004d40 0x7f4fc00051c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc001e50 tx=0x7f4fc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524f0bdb3dec1afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc001a70 con 0x7f4fc0004d40 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc003970 con 0x7f4fc0004d40 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.624+0000 7f4fd9046640 1 -- 172.21.15.148:0/3735845048 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc003d60 con 0x7f4fc0004d40 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/3735845048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc0004d40 msgr2=0x7f4fc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/3735845048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fc0004d40 0x7f4fc00051c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc001e50 tx=0x7f4fc0005830 comp rx=0 tx=0).stop 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fda849640 1 -- 172.21.15.148:0/3735845048 reap_dead start 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/3735845048 shutdown_connections 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/3735845048 >> 172.21.15.148:0/3735845048 conn(0x7f4fd40b3530 msgr2=0x7f4fd40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/3735845048 shutdown_connections 2024-04-25T09:54:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.625+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/3735845048 wait complete. 2024-04-25T09:54:53.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.626+0000 7f4fdc2ac640 1 Processor -- start 2024-04-25T09:54:53.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.626+0000 7f4fdc2ac640 1 -- start start 2024-04-25T09:54:53.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fdc2ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0004d40 0x7f4fd414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fdc2ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 0x7f4fd4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fdc2ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4151690 0x7f4fd4151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fdc2ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd40b10f0 con 0x7f4fd4155110 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fdc2ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41622b0 con 0x7f4fc0004d40 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fdc2ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4161f40 con 0x7f4fd4151690 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 0x7f4fd4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 0x7f4fd4155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47350/0 (socket says 172.21.15.148:47350) 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 -- 172.21.15.148:0/1056183504 learned_addr learned my addr 172.21.15.148:0/1056183504 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fda849640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4151690 0x7f4fd4151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fda048640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0004d40 0x7f4fd414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 -- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4151690 msgr2=0x7f4fd4151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4151690 0x7f4fd4151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 -- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0004d40 msgr2=0x7f4fd414e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0004d40 0x7f4fd414e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 -- 172.21.15.148:0/1056183504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4150150 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fda048640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0004d40 0x7f4fd414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.627+0000 7f4fd9847640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 0x7f4fd4155590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007340 tx=0x7f4fc4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.628+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400f8c0 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.628+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4018070 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.628+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd4150430 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.628+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400fcc0 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.628+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41509f0 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.628+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd4073a00 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.629+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fc401d460 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.629+0000 7f4fcaffd640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4fb806b4d0 0x7f4fb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.629+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4fc408a630 con 0x7f4fd4155110 2024-04-25T09:54:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.629+0000 7f4fda048640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4fb806b4d0 0x7f4fb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.630+0000 7f4fda048640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4fb806b4d0 0x7f4fb806d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009bc0 tx=0x7f4fd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.634+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fc400fe60 con 0x7f4fd4155110 2024-04-25T09:54:53.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.805+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fd4150bc0 con 0x7f4fd4155110 2024-04-25T09:54:53.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.805+0000 7f4fcaffd640 1 -- 172.21.15.148:0/1056183504 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fc4058bb0 con 0x7f4fd4155110 2024-04-25T09:54:53.808 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4fb806b4d0 msgr2=0x7f4fb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4fb806b4d0 0x7f4fb806d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009bc0 tx=0x7f4fd0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 msgr2=0x7f4fd4155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 0x7f4fd4155590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007340 tx=0x7f4fc4007800 comp rx=0 tx=0).stop 2024-04-25T09:54:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 shutdown_connections 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4fb806b4d0 0x7f4fb806d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fd4151690 0x7f4fd4151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fd4155110 0x7f4fd4155590 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 --2- 172.21.15.148:0/1056183504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc0004d40 0x7f4fd414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 >> 172.21.15.148:0/1056183504 conn(0x7f4fd40b3530 msgr2=0x7f4fd40b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 shutdown_connections 2024-04-25T09:54:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.807+0000 7f4fdc2ac640 1 -- 172.21.15.148:0/1056183504 wait complete. 2024-04-25T09:54:53.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:54:53.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.918+0000 7fdcff4cd640 1 Processor -- start 2024-04-25T09:54:53.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.918+0000 7fdcff4cd640 1 -- start start 2024-04-25T09:54:53.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcff4cd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf8158230 0x7fdcf80d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcff4cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcf80d1b80 0x7fdcf806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcff4cd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcf806c870 0x7fdcf80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcff4cd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf8158610 con 0x7fdcf80d1b80 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcff4cd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf806bea0 con 0x7fdcf806c870 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcff4cd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80d2020 con 0x7fdcf8158230 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcf80d1b80 0x7fdcf806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcf80d1b80 0x7fdcf806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47354/0 (socket says 172.21.15.148:47354) 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 -- 172.21.15.148:0/1993568457 learned_addr learned my addr 172.21.15.148:0/1993568457 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfda6a640 1 --2- 172.21.15.148:0/1993568457 >> v2:172.21.15.188:3300/0 conn(0x7fdcf806c870 0x7fdcf80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfd269640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf8158230 0x7fdcf80d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 -- 172.21.15.148:0/1993568457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf8158230 msgr2=0x7fdcf80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf8158230 0x7fdcf80d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 -- 172.21.15.148:0/1993568457 >> v2:172.21.15.188:3300/0 conn(0x7fdcf806c870 msgr2=0x7fdcf80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 --2- 172.21.15.148:0/1993568457 >> v2:172.21.15.188:3300/0 conn(0x7fdcf806c870 0x7fdcf80b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfca68640 1 -- 172.21.15.148:0/1993568457 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80d2300 con 0x7fdcf80d1b80 2024-04-25T09:54:53.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.919+0000 7fdcfd269640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf8158230 0x7fdcf80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:53.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.920+0000 7fdcfca68640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf806c330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80d13e0 tx=0x7fdce802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef75cc367fa329c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.920+0000 7fdceffff640 1 -- 172.21.15.148:0/1993568457 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004030 con 0x7fdcf80d1b80 2024-04-25T09:54:53.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.920+0000 7fdceffff640 1 -- 172.21.15.148:0/1993568457 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce80041d0 con 0x7fdcf80d1b80 2024-04-25T09:54:53.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.920+0000 7fdceffff640 1 -- 172.21.15.148:0/1993568457 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8003610 con 0x7fdcf80d1b80 2024-04-25T09:54:53.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 -- 172.21.15.148:0/1993568457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 msgr2=0x7fdcf806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf806c330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80d13e0 tx=0x7fdce802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 -- 172.21.15.148:0/1993568457 shutdown_connections 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/1993568457 >> v2:172.21.15.188:3300/0 conn(0x7fdcf806c870 0x7fdcf80b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf806c330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/1993568457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf8158230 0x7fdcf80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 -- 172.21.15.148:0/1993568457 >> 172.21.15.148:0/1993568457 conn(0x7fdcf8061600 msgr2=0x7fdcf80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 -- 172.21.15.148:0/1993568457 shutdown_connections 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.921+0000 7fdcff4cd640 1 -- 172.21.15.148:0/1993568457 wait complete. 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.922+0000 7fdcff4cd640 1 Processor -- start 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.922+0000 7fdcff4cd640 1 -- start start 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcff4cd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf806c870 0x7fdcf81e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcff4cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf80cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcff4cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf8158230 0x7fdcf814c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcff4cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf815e320 con 0x7fdcf80d1b80 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcff4cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80d25e0 con 0x7fdcf8158230 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcff4cd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf815e4a0 con 0x7fdcf806c870 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf80cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf80cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47360/0 (socket says 172.21.15.148:47360) 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 -- 172.21.15.148:0/496771912 learned_addr learned my addr 172.21.15.148:0/496771912 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfd269640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf806c870 0x7fdcf81e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfda6a640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf8158230 0x7fdcf814c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 -- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf806c870 msgr2=0x7fdcf81e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf806c870 0x7fdcf81e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 -- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf8158230 msgr2=0x7fdcf814c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf8158230 0x7fdcf814c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 -- 172.21.15.148:0/496771912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80cc850 con 0x7fdcf80d1b80 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.923+0000 7fdcfca68640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf80cc020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdce802a200 tx=0x7fdce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.924+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8041020 con 0x7fdcf80d1b80 2024-04-25T09:54:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.924+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce8003c50 con 0x7fdcf80d1b80 2024-04-25T09:54:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.924+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce8004030 con 0x7fdcf80d1b80 2024-04-25T09:54:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.924+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf80ccb30 con 0x7fdcf80d1b80 2024-04-25T09:54:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.924+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf80cd0f0 con 0x7fdcf80d1b80 2024-04-25T09:54:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.925+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf8073a00 con 0x7fdcf80d1b80 2024-04-25T09:54:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.925+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdce8044020 con 0x7fdcf80d1b80 2024-04-25T09:54:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.926+0000 7fdcedffb640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.926+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdce80aaed0 con 0x7fdcf80d1b80 2024-04-25T09:54:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.926+0000 7fdcfd269640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.926+0000 7fdcfd269640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400aae0 tx=0x7fdcf4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:53.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:53.930+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdce804b030 con 0x7fdcf80d1b80 2024-04-25T09:54:54.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.098+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcf80b4ff0 con 0x7fdcf80d1b80 2024-04-25T09:54:54.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.099+0000 7fdcedffb640 1 -- 172.21.15.148:0/496771912 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdce8079450 con 0x7fdcf80d1b80 2024-04-25T09:54:54.101 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:54:54.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcdc06b4d0 msgr2=0x7fdcdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdcf400aae0 tx=0x7fdcf4008040 comp rx=0 tx=0).stop 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 msgr2=0x7fdcf80cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf80cc020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdce802a200 tx=0x7fdce80046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 shutdown_connections 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcdc06b4d0 0x7fdcdc06d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf8158230 0x7fdcf814c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcf80d1b80 0x7fdcf80cc020 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 --2- 172.21.15.148:0/496771912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcf806c870 0x7fdcf81e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 >> 172.21.15.148:0/496771912 conn(0x7fdcf8061600 msgr2=0x7fdcf815d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 shutdown_connections 2024-04-25T09:54:54.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.101+0000 7fdcff4cd640 1 -- 172.21.15.148:0/496771912 wait complete. 2024-04-25T09:54:54.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:54:54.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.212+0000 7f3903bbd640 1 Processor -- start 2024-04-25T09:54:54.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.212+0000 7f3903bbd640 1 -- start start 2024-04-25T09:54:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3903bbd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38fc0d34c0 0x7f38fc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3903bbd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0d2420 con 0x7f38fc0d08e0 2024-04-25T09:54:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3903bbd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0d2700 con 0x7f38fc15dac0 2024-04-25T09:54:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3903bbd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0d0d50 con 0x7f38fc0d34c0 2024-04-25T09:54:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f390215a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f38fc0d08e0 0x7f38fc0b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51362/0 (socket says 172.21.15.148:51362) 2024-04-25T09:54:54.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f390215a640 1 -- 172.21.15.148:0/900010041 learned_addr learned my addr 172.21.15.148:0/900010041 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3901158640 1 --2- 172.21.15.148:0/900010041 >> v2:172.21.15.188:3300/0 conn(0x7f38fc0d34c0 0x7f38fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376195789 0 0) 0x7f38fc0d2420 con 0x7f38fc0d08e0 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.213+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e8003520 con 0x7f38fc0d08e0 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207398283 0 0) 0x7f38fc0d2700 con 0x7f38fc15dac0 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38fc0d2420 con 0x7f38fc15dac0 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3901158640 1 -- 172.21.15.148:0/900010041 >> v1:172.21.15.188:6789/0 conn(0x7f38fc15dac0 legacy=0x7f38fc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3901158640 1 -- 172.21.15.148:0/900010041 >> v1:172.21.15.148:6789/0 conn(0x7f38fc0d08e0 legacy=0x7f38fc0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3901158640 1 -- 172.21.15.148:0/900010041 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0d29e0 con 0x7f38fc0d34c0 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3901158640 1 --2- 172.21.15.148:0/900010041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc0d03a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009790 tx=0x7f38ec02a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4cce0477a002167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec037070 con 0x7f38fc0d34c0 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.214+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec032440 con 0x7f38fc0d34c0 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3900957640 1 -- 172.21.15.148:0/900010041 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec032810 con 0x7f38fc0d34c0 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 -- 172.21.15.148:0/900010041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 msgr2=0x7f38fc0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 --2- 172.21.15.148:0/900010041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc0d03a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009790 tx=0x7f38ec02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 -- 172.21.15.148:0/900010041 shutdown_connections 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 --2- 172.21.15.148:0/900010041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc0d03a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 -- 172.21.15.148:0/900010041 >> 172.21.15.148:0/900010041 conn(0x7f38fc061c30 msgr2=0x7f38fc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 -- 172.21.15.148:0/900010041 shutdown_connections 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.215+0000 7f3903bbd640 1 -- 172.21.15.148:0/900010041 wait complete. 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.216+0000 7f3903bbd640 1 Processor -- start 2024-04-25T09:54:54.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.216+0000 7f3903bbd640 1 -- start start 2024-04-25T09:54:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3903bbd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38fc0d08e0 0x7f38fc152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3903bbd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc15eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3903bbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 0x7f38fc0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3903bbd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0d2100 con 0x7f38fc15dac0 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3903bbd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0d3030 con 0x7f38fc0d34c0 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3903bbd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc15dea0 con 0x7f38fc0d08e0 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 0x7f38fc0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 0x7f38fc0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47376/0 (socket says 172.21.15.148:47376) 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 -- 172.21.15.148:0/3215554880 learned_addr learned my addr 172.21.15.148:0/3215554880 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3901158640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc15eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 -- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38fc0d08e0 msgr2=0x7f38fc152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f3901959640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38fc0d08e0 0x7f38fc152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38fc0d08e0 0x7f38fc152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 -- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 msgr2=0x7f38fc15eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.217+0000 7f390215a640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc15eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f390215a640 1 -- 172.21.15.148:0/3215554880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc160680 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f390215a640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 0x7f38fc0b33c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38f400cbc0 tx=0x7f38f400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4017070 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400ace0 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4012600 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0cb760 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.218+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc160920 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.219+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc073a00 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.219+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38f401a070 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.220+0000 7f38f27fc640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38e006b500 0x7f38e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.220+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f38f408a280 con 0x7f38fc15dac0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.220+0000 7f3901959640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38e006b500 0x7f38e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.220+0000 7f3901959640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38e006b500 0x7f38e006d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f38f8007fd0 tx=0x7f38f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.224+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38f40586f0 con 0x7f38fc15dac0 2024-04-25T09:54:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.395+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f38fc0b2800 con 0x7f38fc15dac0 2024-04-25T09:54:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.396+0000 7f38f27fc640 1 -- 172.21.15.148:0/3215554880 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f38f405c790 con 0x7f38fc15dac0 2024-04-25T09:54:54.399 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38e006b500 msgr2=0x7f38e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38e006b500 0x7f38e006d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f38f8007fd0 tx=0x7f38f8009040 comp rx=0 tx=0).stop 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 msgr2=0x7f38fc0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 0x7f38fc0b33c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38f400cbc0 tx=0x7f38f400a430 comp rx=0 tx=0).stop 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 shutdown_connections 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38e006b500 0x7f38e006d9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc15dac0 0x7f38fc0b33c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc0d34c0 0x7f38fc15eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 --2- 172.21.15.148:0/3215554880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38fc0d08e0 0x7f38fc152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 >> 172.21.15.148:0/3215554880 conn(0x7f38fc061c30 msgr2=0x7f38fc0d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 shutdown_connections 2024-04-25T09:54:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.398+0000 7f3903bbd640 1 -- 172.21.15.148:0/3215554880 wait complete. 2024-04-25T09:54:54.412 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:54.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.506+0000 7f3e1f05a640 1 Processor -- start 2024-04-25T09:54:54.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.507+0000 7f3e1f05a640 1 -- start start 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.507+0000 7f3e1f05a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e1815db60 0x7f3e180ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.507+0000 7f3e1f05a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.507+0000 7f3e1f05a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e180cec00 con 0x7f3e180b5f00 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.507+0000 7f3e1f05a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e180ce930 con 0x7f3e1815db60 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.507+0000 7f3e1f05a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180b6450 con 0x7f3e180d0b10 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e1815db60 0x7f3e180ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1d5f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e1815db60 0x7f3e180ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:56768/0 (socket says 172.21.15.148:56768) 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1d5f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:41090/0 (socket says 172.21.15.148:41090) 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/1911897748 learned_addr learned my addr 172.21.15.148:0/1911897748 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e17fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3e180b5f00 0x7f3e180d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:40404/0 (socket says 172.21.15.148:40404) 2024-04-25T09:54:54.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 msgr2=0x7f3e180d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 --2- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/1911897748 >> v1:172.21.15.188:6789/0 conn(0x7f3e180b5f00 legacy=0x7f3e180d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/1911897748 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180ceee0 con 0x7f3e1815db60 2024-04-25T09:54:54.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.508+0000 7f3e1d5f7640 1 --2- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.509+0000 7f3e1cdf6640 1 --2- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e180ce3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0098a0 tx=0x7f3e0c02a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4547b140b66e8bce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.509+0000 7f3e177fe640 1 -- 172.21.15.148:0/1911897748 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c037070 con 0x7f3e1815db60 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.509+0000 7f3e177fe640 1 -- 172.21.15.148:0/1911897748 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c032440 con 0x7f3e1815db60 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.509+0000 7f3e177fe640 1 -- 172.21.15.148:0/1911897748 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c032810 con 0x7f3e1815db60 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 -- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 msgr2=0x7f3e180ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e180ce3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0098a0 tx=0x7f3e0c02a200 comp rx=0 tx=0).stop 2024-04-25T09:54:54.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 -- 172.21.15.148:0/1911897748 shutdown_connections 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/1911897748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e180ce3f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 -- 172.21.15.148:0/1911897748 >> 172.21.15.148:0/1911897748 conn(0x7f3e18061600 msgr2=0x7f3e18061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 -- 172.21.15.148:0/1911897748 shutdown_connections 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.510+0000 7f3e1f05a640 1 -- 172.21.15.148:0/1911897748 wait complete. 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.511+0000 7f3e1f05a640 1 Processor -- start 2024-04-25T09:54:54.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.511+0000 7f3e1f05a640 1 -- start start 2024-04-25T09:54:54.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 0x7f3e18151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e1815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1815df40 con 0x7f3e180b5f00 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180d1d50 con 0x7f3e1815db60 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1815e0c0 con 0x7f3e180d0b10 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 0x7f3e18151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 0x7f3e18151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47382/0 (socket says 172.21.15.148:47382) 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/3323056174 learned_addr learned my addr 172.21.15.148:0/3323056174 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1d5f7640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e1815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e17fff640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 msgr2=0x7f3e180cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 msgr2=0x7f3e1815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e1815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 -- 172.21.15.148:0/3323056174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180cb080 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1cdf6640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 0x7f3e18151070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0098a0 tx=0x7f3e0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c037040 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c030440 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1815e3d0 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c030830 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.512+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1815e990 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.513+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e18073270 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.514+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e0c044030 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.514+0000 7f3e157fa640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dfc06b4d0 0x7f3dfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.514+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3e0c0aa380 con 0x7f3e180b5f00 2024-04-25T09:54:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.514+0000 7f3e17fff640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dfc06b4d0 0x7f3dfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.515+0000 7f3e17fff640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dfc06b4d0 0x7f3dfc06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e180b8e80 tx=0x7f3e04007cc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.518+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e0c04b060 con 0x7f3e180b5f00 2024-04-25T09:54:54.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.724+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e18073990 con 0x7f3e180b5f00 2024-04-25T09:54:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.724+0000 7f3e157fa640 1 -- 172.21.15.148:0/3323056174 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f3e0c078f50 con 0x7f3e180b5f00 2024-04-25T09:54:54.727 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:54.727 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:54.727 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:54:54.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dfc06b4d0 msgr2=0x7f3dfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dfc06b4d0 0x7f3dfc06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e180b8e80 tx=0x7f3e04007cc0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 msgr2=0x7f3e18151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 0x7f3e18151070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0098a0 tx=0x7f3e0c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 shutdown_connections 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dfc06b4d0 0x7f3dfc06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.726+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e1815db60 0x7f3e1815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.727+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e180d0b10 0x7f3e180cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.727+0000 7f3e1f05a640 1 --2- 172.21.15.148:0/3323056174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e180b5f00 0x7f3e18151070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.727+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 >> 172.21.15.148:0/3323056174 conn(0x7f3e18061600 msgr2=0x7f3e180d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.727+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 shutdown_connections 2024-04-25T09:54:54.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.727+0000 7f3e1f05a640 1 -- 172.21.15.148:0/3323056174 wait complete. 2024-04-25T09:54:54.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:54:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.837+0000 7fca08841640 1 Processor -- start 2024-04-25T09:54:54.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.837+0000 7fca08841640 1 -- start start 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.838+0000 7fca08841640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca040cf340 0x7fca040c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.838+0000 7fca08841640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca040c7380 con 0x7fca0415b8f0 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.838+0000 7fca08841640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca040c7660 con 0x7fca040b3af0 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.838+0000 7fca08841640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040d03c0 con 0x7fca040cf340 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.838+0000 7fca0259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fca0415b8f0 0x7fca040b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51378/0 (socket says 172.21.15.148:51378) 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.838+0000 7fca0259c640 1 -- 172.21.15.148:0/1776051522 learned_addr learned my addr 172.21.15.148:0/1776051522 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca02d9d640 1 --2- 172.21.15.148:0/1776051522 >> v2:172.21.15.188:3300/0 conn(0x7fca040cf340 0x7fca040c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655395441 0 0) 0x7fca040c7380 con 0x7fca0415b8f0 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ec003520 con 0x7fca0415b8f0 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2864163315 0 0) 0x7fc9ec003520 con 0x7fca0415b8f0 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 >> v2:172.21.15.188:3300/0 conn(0x7fca040cf340 msgr2=0x7fca040c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 --2- 172.21.15.148:0/1776051522 >> v2:172.21.15.188:3300/0 conn(0x7fca040cf340 0x7fca040c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 >> v1:172.21.15.188:6789/0 conn(0x7fca040b3af0 legacy=0x7fca040cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040c7940 con 0x7fca0415b8f0 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc9f40019f0 con 0x7fca0415b8f0 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 >> v1:172.21.15.148:6789/0 conn(0x7fca0415b8f0 legacy=0x7fca040b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040d03c0 con 0x7fc9ec0039b0 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec00c070 con 0x7fc9ec0051c0 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec00c270 con 0x7fc9ec004800 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0259c640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca02d9d640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca01d9b640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0259c640 1 -- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec004800 msgr2=0x7fc9ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0259c640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec004800 0x7fc9ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0259c640 1 -- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9ec0051c0 msgr2=0x7fc9ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0259c640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca0259c640 1 -- 172.21.15.148:0/1776051522 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec003520 con 0x7fc9ec0039b0 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca02d9d640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9ec0051c0 0x7fc9ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.839+0000 7fca02d9d640 1 -- 172.21.15.148:0/1776051522 reap_dead start 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.840+0000 7fca0259c640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400a840 tx=0x7fc9ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d720307aa82061b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.840+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400a460 con 0x7fc9ec0039b0 2024-04-25T09:54:54.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.840+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f402f070 con 0x7fc9ec0039b0 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.840+0000 7fca0159a640 1 -- 172.21.15.148:0/1776051522 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f402bcd0 con 0x7fc9ec0039b0 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 -- 172.21.15.148:0/1776051522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ec0039b0 msgr2=0x7fc9ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400a840 tx=0x7fc9ec005830 comp rx=0 tx=0).stop 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 -- 172.21.15.148:0/1776051522 shutdown_connections 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 --2- 172.21.15.148:0/1776051522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9ec0039b0 0x7fc9ec003dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 -- 172.21.15.148:0/1776051522 >> 172.21.15.148:0/1776051522 conn(0x7fca040d1800 msgr2=0x7fca040d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 -- 172.21.15.148:0/1776051522 shutdown_connections 2024-04-25T09:54:54.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.841+0000 7fca08841640 1 -- 172.21.15.148:0/1776051522 wait complete. 2024-04-25T09:54:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.842+0000 7fca08841640 1 Processor -- start 2024-04-25T09:54:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.842+0000 7fca08841640 1 -- start start 2024-04-25T09:54:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec0039b0 0x7fca0414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 0x7fca040b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0415d7e0 0x7fca0415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0415bcd0 con 0x7fca040b22f0 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040c7f90 con 0x7fca0415d7e0 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0415be50 con 0x7fc9ec0039b0 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 0x7fca040b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 0x7fca040b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47402/0 (socket says 172.21.15.148:47402) 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 -- 172.21.15.148:0/4282986231 learned_addr learned my addr 172.21.15.148:0/4282986231 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 -- 172.21.15.148:0/4282986231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec0039b0 msgr2=0x7fca0414d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec0039b0 0x7fca0414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 -- 172.21.15.148:0/4282986231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0415d7e0 msgr2=0x7fca0415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0415d7e0 0x7fca0415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 -- 172.21.15.148:0/4282986231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0414db20 con 0x7fca040b22f0 2024-04-25T09:54:54.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca01d9b640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 0x7fca040b2770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800e4d0 tx=0x7fc9f800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80053a0 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f8018070 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8003810 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca041f1a10 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.843+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca041f1fd0 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.844+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca04073990 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.845+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9f80039b0 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.845+0000 7fc9f2ffd640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9e406b500 0x7fc9e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.845+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc9f808a1b0 con 0x7fca040b22f0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.845+0000 7fca0259c640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9e406b500 0x7fc9e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:54.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.846+0000 7fca0259c640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9e406b500 0x7fc9e406d9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4033000 tx=0x7fc9f402d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:54.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:54.849+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca0414db20 con 0x7fca040b22f0 2024-04-25T09:54:55.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.024+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca040c85d0 con 0x7fca040b22f0 2024-04-25T09:54:55.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.025+0000 7fc9f2ffd640 1 -- 172.21.15.148:0/4282986231 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18562 (secure 0 0 0) 0x7fc9f8058cf0 con 0x7fca040b22f0 2024-04-25T09:54:55.028 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:55.028 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:54:29.991334+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9e406b500 msgr2=0x7fc9e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9e406b500 0x7fc9e406d9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4033000 tx=0x7fc9f402d000 comp rx=0 tx=0).stop 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 msgr2=0x7fca040b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 0x7fca040b2770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800e4d0 tx=0x7fc9f800e990 comp rx=0 tx=0).stop 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 shutdown_connections 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9e406b500 0x7fc9e406d9c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0415d7e0 0x7fca0415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca040b22f0 0x7fca040b2770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 --2- 172.21.15.148:0/4282986231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9ec0039b0 0x7fca0414d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 >> 172.21.15.148:0/4282986231 conn(0x7fca040d1800 msgr2=0x7fca040d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 shutdown_connections 2024-04-25T09:54:55.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.027+0000 7fca08841640 1 -- 172.21.15.148:0/4282986231 wait complete. 2024-04-25T09:54:55.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:54:55.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 Processor -- start 2024-04-25T09:54:55.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 -- start start 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98100b5f00 0x7f98100d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f981015df40 0x7f98100d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98100ce930 con 0x7f98100b5f00 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98100b6450 con 0x7f981015df40 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f9816238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98100cec00 con 0x7f981015db60 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f980effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98100b5f00 0x7f98100d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.137+0000 7f980effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98100b5f00 0x7f98100d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47404/0 (socket says 172.21.15.148:47404) 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 -- 172.21.15.148:0/3139317574 learned_addr learned my addr 172.21.15.148:0/3139317574 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980ffff640 1 --2- 172.21.15.148:0/3139317574 >> v2:172.21.15.188:3300/0 conn(0x7f981015df40 0x7f98100d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980f7fe640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 -- 172.21.15.148:0/3139317574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 msgr2=0x7f98100ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 -- 172.21.15.148:0/3139317574 >> v2:172.21.15.188:3300/0 conn(0x7f981015df40 msgr2=0x7f98100d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 --2- 172.21.15.148:0/3139317574 >> v2:172.21.15.188:3300/0 conn(0x7f981015df40 0x7f98100d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 -- 172.21.15.148:0/3139317574 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98100ceee0 con 0x7f98100b5f00 2024-04-25T09:54:55.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980f7fe640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:55.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.138+0000 7f980effd640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98100d0400 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9810161890 tx=0x7f980402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b8b86125bf42de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.139+0000 7f980e7fc640 1 -- 172.21.15.148:0/3139317574 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804037070 con 0x7f98100b5f00 2024-04-25T09:54:55.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.139+0000 7f980e7fc640 1 -- 172.21.15.148:0/3139317574 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9804032440 con 0x7f98100b5f00 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.139+0000 7f980e7fc640 1 -- 172.21.15.148:0/3139317574 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804032810 con 0x7f98100b5f00 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 -- 172.21.15.148:0/3139317574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 msgr2=0x7f98100d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98100d0400 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9810161890 tx=0x7f980402a180 comp rx=0 tx=0).stop 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 -- 172.21.15.148:0/3139317574 shutdown_connections 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 --2- 172.21.15.148:0/3139317574 >> v2:172.21.15.188:3300/0 conn(0x7f981015df40 0x7f98100d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98100d0400 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 --2- 172.21.15.148:0/3139317574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 -- 172.21.15.148:0/3139317574 >> 172.21.15.148:0/3139317574 conn(0x7f9810061600 msgr2=0x7f9810061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 -- 172.21.15.148:0/3139317574 shutdown_connections 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.140+0000 7f9816238640 1 -- 172.21.15.148:0/3139317574 wait complete. 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.141+0000 7f9816238640 1 Processor -- start 2024-04-25T09:54:55.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.141+0000 7f9816238640 1 -- start start 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f9816238640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98101f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f9816238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f9816238640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f981015df40 0x7f9810160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f9816238640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810161950 con 0x7f98100b5f00 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f9816238640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98100d1d50 con 0x7f981015df40 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f9816238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9810161ad0 con 0x7f981015db60 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98101f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98101f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47410/0 (socket says 172.21.15.148:47410) 2024-04-25T09:54:55.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 -- 172.21.15.148:0/3046735653 learned_addr learned my addr 172.21.15.148:0/3046735653 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980ffff640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f981015df40 0x7f9810160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980effd640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 -- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 msgr2=0x7f98100ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 -- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f981015df40 msgr2=0x7f9810160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f981015df40 0x7f9810160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 -- 172.21.15.148:0/3046735653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98101f4a50 con 0x7f98100b5f00 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f980f7fe640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98101f4510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f980000e840 tx=0x7f980000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.142+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000f040 con 0x7f98100b5f00 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.143+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9800004030 con 0x7f98100b5f00 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.143+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800004400 con 0x7f98100b5f00 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.143+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981015e950 con 0x7f98100b5f00 2024-04-25T09:54:55.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.143+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981015ef10 con 0x7f98100b5f00 2024-04-25T09:54:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.144+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9810073270 con 0x7f98100b5f00 2024-04-25T09:54:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.144+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f980000c6d0 con 0x7f98100b5f00 2024-04-25T09:54:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.144+0000 7f97f7fff640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97fc06b500 0x7f97fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.145+0000 7f980effd640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97fc06b500 0x7f97fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.145+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f980008a0f0 con 0x7f98100b5f00 2024-04-25T09:54:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.145+0000 7f980effd640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97fc06b500 0x7f97fc06d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f980402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.149+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98101f4a50 con 0x7f98100b5f00 2024-04-25T09:54:55.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.354+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9810073990 con 0x7f98100b5f00 2024-04-25T09:54:55.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.354+0000 7f97f7fff640 1 -- 172.21.15.148:0/3046735653 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f9800017020 con 0x7f98100b5f00 2024-04-25T09:54:55.357 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:54:55.357 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:54:55.358 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:54:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97fc06b500 msgr2=0x7f97fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97fc06b500 0x7f97fc06d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f980402e000 comp rx=0 tx=0).stop 2024-04-25T09:54:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 msgr2=0x7f98101f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98101f4510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f980000e840 tx=0x7f980000ed00 comp rx=0 tx=0).stop 2024-04-25T09:54:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 shutdown_connections 2024-04-25T09:54:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f97fc06b500 0x7f97fc06d9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f981015df40 0x7f9810160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981015db60 0x7f98100ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 --2- 172.21.15.148:0/3046735653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98100b5f00 0x7f98101f4510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.356+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 >> 172.21.15.148:0/3046735653 conn(0x7f9810061600 msgr2=0x7f98100b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.357+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 shutdown_connections 2024-04-25T09:54:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.357+0000 7f9816238640 1 -- 172.21.15.148:0/3046735653 wait complete. 2024-04-25T09:54:55.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:54:23.300442+0000', 'modified': '2024-04-25T09:54:44.431176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/1500268127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 1500268127}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 1500268127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:54:55.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:54:55.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T09:54:55.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.871+0000 7f48fe815640 1 Processor -- start 2024-04-25T09:54:55.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.871+0000 7f48fe815640 1 -- start start 2024-04-25T09:54:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fe815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f815dac0 0x7f48f80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fe815640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f48f80d1f30 0x7f48f80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fe815640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f80d34c0 0x7f48f80d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fe815640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f80d3030 con 0x7f48f80d34c0 2024-04-25T09:54:55.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fe815640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f80d2420 con 0x7f48f80d1f30 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fe815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80d2700 con 0x7f48f815dac0 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f80d34c0 0x7f48f80d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f80d34c0 0x7f48f80d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47418/0 (socket says 172.21.15.148:47418) 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2319173079 learned_addr learned my addr 172.21.15.148:0/2319173079 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48f77fe640 1 --2- 172.21.15.148:0/2319173079 >> v2:172.21.15.188:3300/0 conn(0x7f48f80d1f30 0x7f48f80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48f7fff640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f815dac0 0x7f48f80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2319173079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f815dac0 msgr2=0x7f48f80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f815dac0 0x7f48f80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2319173079 >> v2:172.21.15.188:3300/0 conn(0x7f48f80d1f30 msgr2=0x7f48f80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/2319173079 >> v2:172.21.15.188:3300/0 conn(0x7f48f80d1f30 0x7f48f80d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2319173079 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80d29e0 con 0x7f48f80d34c0 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.872+0000 7f48f7fff640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f815dac0 0x7f48f80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:55.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.873+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f80d34c0 0x7f48f80d08e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48f80b6550 tx=0x7f48f002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5427b174cd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.873+0000 7f48f6ffd640 1 -- 172.21.15.148:0/2319173079 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0036070 con 0x7f48f80d34c0 2024-04-25T09:54:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.873+0000 7f48f6ffd640 1 -- 172.21.15.148:0/2319173079 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0031600 con 0x7f48f80d34c0 2024-04-25T09:54:55.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.873+0000 7f48f6ffd640 1 -- 172.21.15.148:0/2319173079 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00319d0 con 0x7f48f80d34c0 2024-04-25T09:54:55.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.873+0000 7f48fe815640 1 -- 172.21.15.148:0/2319173079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f80d34c0 msgr2=0x7f48f80d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.873+0000 7f48fe815640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f80d34c0 0x7f48f80d08e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f48f80b6550 tx=0x7f48f002a730 comp rx=0 tx=0).stop 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 -- 172.21.15.148:0/2319173079 shutdown_connections 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f80d34c0 0x7f48f80d08e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 --2- 172.21.15.148:0/2319173079 >> v2:172.21.15.188:3300/0 conn(0x7f48f80d1f30 0x7f48f80d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 --2- 172.21.15.148:0/2319173079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f815dac0 0x7f48f80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 -- 172.21.15.148:0/2319173079 >> 172.21.15.148:0/2319173079 conn(0x7f48f8061c30 msgr2=0x7f48f806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 -- 172.21.15.148:0/2319173079 shutdown_connections 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.874+0000 7f48fe815640 1 -- 172.21.15.148:0/2319173079 wait complete. 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.875+0000 7f48fe815640 1 Processor -- start 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.875+0000 7f48fe815640 1 -- start start 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f80d1f30 0x7f48f815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f80d34c0 0x7f48f8153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 0x7f48f81544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80b97d0 con 0x7f48f815dac0 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f815dea0 con 0x7f48f80d1f30 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80b9950 con 0x7f48f80d34c0 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 0x7f48f81544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 0x7f48f81544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47430/0 (socket says 172.21.15.148:47430) 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2683844359 learned_addr learned my addr 172.21.15.148:0/2683844359 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48f7fff640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f80d1f30 0x7f48f815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48f77fe640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f80d34c0 0x7f48f8153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f80d34c0 msgr2=0x7f48f8153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f80d34c0 0x7f48f8153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f80d1f30 msgr2=0x7f48f815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f80d1f30 0x7f48f815f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 -- 172.21.15.148:0/2683844359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f815f7e0 con 0x7f48f815dac0 2024-04-25T09:54:55.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 0x7f48f81544b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009f40 tx=0x7f48f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48f4ff9640 1 -- 172.21.15.148:0/2683844359 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0036050 con 0x7f48f815dac0 2024-04-25T09:54:55.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48f4ff9640 1 -- 172.21.15.148:0/2683844359 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0042070 con 0x7f48f815dac0 2024-04-25T09:54:55.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48f4ff9640 1 -- 172.21.15.148:0/2683844359 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0009510 con 0x7f48f815dac0 2024-04-25T09:54:55.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f815fac0 con 0x7f48f815dac0 2024-04-25T09:54:55.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.876+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f8160080 con 0x7f48f815dac0 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.878+0000 7f48f4ff9640 1 -- 172.21.15.148:0/2683844359 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48f0007bf0 con 0x7f48f815dac0 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.878+0000 7f48f4ff9640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48e406b4d0 0x7f48e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.878+0000 7f48f4ff9640 1 -- 172.21.15.148:0/2683844359 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48f00ab350 con 0x7f48f815dac0 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.878+0000 7f48f7fff640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48e406b4d0 0x7f48e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.879+0000 7f48f7fff640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48e406b4d0 0x7f48e406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48e800b7c0 tx=0x7f48e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.879+0000 7f49015f9740 1 Processor -- start 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.879+0000 7f49015f9740 1 -- start start 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f49015f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 0x556613e25020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613ddcf10 0x556613e23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f49015f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556613d3fa10 con 0x556613e24ad0 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556613ce8b30 con 0x556613e07230 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556613d3bc50 con 0x556613ddcf10 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 0x556613e25020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 0x556613e25020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47446/0 (socket says 172.21.15.148:47446) 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 -- 172.21.15.148:0/39941106 learned_addr learned my addr 172.21.15.148:0/39941106 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613ddcf10 0x556613e23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f7fff640 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 -- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613ddcf10 msgr2=0x556613e23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613ddcf10 0x556613e23190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 -- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 msgr2=0x556613e07610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e07610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 -- 172.21.15.148:0/39941106 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556613cfde10 con 0x556613e24ad0 2024-04-25T09:54:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f7fff640 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e07610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48f77fe640 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 0x556613e25020 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0098e0 tx=0x7f48ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69041fd5f77d69d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48ddffb640 1 -- 172.21.15.148:0/39941106 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00f040 con 0x556613e24ad0 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48ddffb640 1 -- 172.21.15.148:0/39941106 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec018070 con 0x556613e24ad0 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.880+0000 7f48ddffb640 1 -- 172.21.15.148:0/39941106 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00e750 con 0x556613e24ad0 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 -- 172.21.15.148:0/39941106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 msgr2=0x556613e25020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 0x556613e25020 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0098e0 tx=0x7f48ec009da0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 -- 172.21.15.148:0/39941106 shutdown_connections 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613ddcf10 0x556613e23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e24ad0 0x556613e25020 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 --2- 172.21.15.148:0/39941106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e07610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 -- 172.21.15.148:0/39941106 >> 172.21.15.148:0/39941106 conn(0x556613dd9840 msgr2=0x556613e22040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 -- 172.21.15.148:0/39941106 shutdown_connections 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.881+0000 7f49015f9740 1 -- 172.21.15.148:0/39941106 wait complete. 2024-04-25T09:54:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.882+0000 7f49015f9740 1 -- start start 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.882+0000 7f49015f9740 1 Processor -- start 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.882+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e03aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556613dddcb0 con 0x556613e07ef0 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556613cfc0c0 con 0x556613e07230 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556613dd7770 con 0x556613e24ad0 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 10 client.0 _ll_register_callbacks cb 0x556613cefec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e05570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47462/0 (socket says 172.21.15.148:47462) 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 -- 172.21.15.148:0/3676283173 learned_addr learned my addr 172.21.15.148:0/3676283173 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48f7fff640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48f77fe640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e03aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 -- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 msgr2=0x556613e03aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e03aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 -- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 msgr2=0x556613e192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 -- 172.21.15.148:0/3676283173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556613a542d0 con 0x556613e07ef0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48f77fe640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e03aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e05570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f48f002a770 tx=0x7f48f0045360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c74832db8cfca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48d6ffd640 1 -- 172.21.15.148:0/3676283173 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0079820 con 0x556613e07ef0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48d6ffd640 1 -- 172.21.15.148:0/3676283173 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48f00911c0 con 0x556613e07ef0 2024-04-25T09:54:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48ec018070 con 0x556613e07ef0 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48d6ffd640 1 -- 172.21.15.148:0/3676283173 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f007d970 con 0x556613e07ef0 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 10 client.4857 fetch_fsmap learned FSMap version 9 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x556613b1e6b0 con 0x556613e07ef0 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.883+0000 7f48d6ffd640 1 -- 172.21.15.148:0/3676283173 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec018070 con 0x556613e07ef0 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f48d6ffd640 1 -- 172.21.15.148:0/3676283173 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f48f00a9120 con 0x556613e07ef0 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f49015f9740 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f49015f9740 10 client.4857 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f49015f9740 20 client.4857 populate_metadata read hostname 'smithi148' 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f49015f9740 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613e3dd30 0x556613e40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f49015f9740 4 client.4857 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 --> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556613d9e920 con 0x556613e3dd30 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f48f7fff640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613e3dd30 0x556613e40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f48d6ffd640 10 client.4857 ms_handle_connect on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.884+0000 7f48f7fff640 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613e3dd30 0x556613e40120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8441423f9fc5f80f server_cookie=38c7b22a19e5bbae in_seq=0 out_seq=0 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f48d6ffd640 1 -- 172.21.15.148:0/3676283173 <== mds.0 v2:172.21.15.188:6833/1500268127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556613d9e920 con 0x556613e3dd30 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f48d6ffd640 10 client.4857 handle_command_reply: tid=0 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 client.4857 shutdown 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 msgr2=0x556613e05570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e05570 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f48f002a770 tx=0x7f48f0045360 comp rx=0 tx=0).stop 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 shutdown_connections 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613e3dd30 0x556613e40120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e05570 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e03aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f48d6ffd640 0 client.4857 ms_handle_reset on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 --2- 172.21.15.148:0/3676283173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 >> 172.21.15.148:0/3676283173 conn(0x556613ddcf10 msgr2=0x556613dd6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 shutdown_connections 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 1 -- 172.21.15.148:0/3676283173 wait complete. 2024-04-25T09:54:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.885+0000 7f49015f9740 20 client.4857 trim_cache size 0 max 16384 2024-04-25T09:54:55.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.901+0000 7f49015f9740 1 Processor -- start 2024-04-25T09:54:55.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.901+0000 7f49015f9740 1 -- start start 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f49015f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e07ef0 0x556613e22200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e24ad0 0x556613e04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f49015f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8200430 con 0x556613e07230 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81ff830 con 0x556613e24ad0 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81ffe30 con 0x556613e07ef0 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47470/0 (socket says 172.21.15.148:47470) 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 -- 172.21.15.148:0/3924419738 learned_addr learned my addr 172.21.15.148:0/3924419738 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 -- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e07ef0 msgr2=0x556613e22200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e07ef0 0x556613e22200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 -- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e24ad0 msgr2=0x556613e04260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e24ad0 0x556613e04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e24ad0 0x556613e04260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 -- 172.21.15.148:0/3924419738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556613e07980 con 0x556613e07230 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e24ad0 0x556613e04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48f7fff640 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 0x556613e21190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48e800d590 tx=0x7f48e8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f02df0caf107c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48dd7fa640 1 -- 172.21.15.148:0/3924419738 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e802c070 con 0x556613e07230 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48dd7fa640 1 -- 172.21.15.148:0/3924419738 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8012b80 con 0x556613e07230 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.902+0000 7f48dd7fa640 1 -- 172.21.15.148:0/3924419738 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8021400 con 0x556613e07230 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 -- 172.21.15.148:0/3924419738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 msgr2=0x556613e21190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 0x556613e21190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48e800d590 tx=0x7f48e8013000 comp rx=0 tx=0).stop 2024-04-25T09:54:55.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 -- 172.21.15.148:0/3924419738 shutdown_connections 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e24ad0 0x556613e04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e07ef0 0x556613e22200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 --2- 172.21.15.148:0/3924419738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 -- 172.21.15.148:0/3924419738 >> 172.21.15.148:0/3924419738 conn(0x556613ddcf10 msgr2=0x556613ddd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 -- 172.21.15.148:0/3924419738 shutdown_connections 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 -- 172.21.15.148:0/3924419738 wait complete. 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.903+0000 7f49015f9740 1 -- start start 2024-04-25T09:54:55.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 Processor -- start 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e18e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556613e08e90 con 0x556613e07ef0 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556613dddcb0 con 0x556613e07230 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556613e35f70 con 0x556613e24ad0 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f49015f9740 10 client.0 _ll_register_callbacks cb 0x556613dd7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f48f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e18e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f48f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e18e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47474/0 (socket says 172.21.15.148:47474) 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.904+0000 7f48f77fe640 1 -- 172.21.15.148:0/4251523716 learned_addr learned my addr 172.21.15.148:0/4251523716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f7fff640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48fcdb2640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f77fe640 1 -- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 msgr2=0x556613e193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f77fe640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f77fe640 1 -- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 msgr2=0x556613e21190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f77fe640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f77fe640 1 -- 172.21.15.148:0/4251523716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556613e093f0 con 0x556613e07ef0 2024-04-25T09:54:55.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f7fff640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48f77fe640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e18e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48ec014000 tx=0x7f48ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5268c4c96efd59e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48d6ffd640 1 -- 172.21.15.148:0/4251523716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0094b0 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48d6ffd640 1 -- 172.21.15.148:0/4251523716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48ec026df0 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48e8012b80 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48d6ffd640 1 -- 172.21.15.148:0/4251523716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ec006050 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 10 client.4863 fetch_fsmap learned FSMap version 9 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x556613e216d0 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48d6ffd640 1 -- 172.21.15.148:0/4251523716 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8012b80 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f48d6ffd640 1 -- 172.21.15.148:0/4251523716 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f48ec012070 con 0x556613e07ef0 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 10 client.4863 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 20 client.4863 populate_metadata read hostname 'smithi148' 2024-04-25T09:54:55.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613ddb840 0x556613e04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 4 client.4863 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.905+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 --> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556613d4cf50 con 0x556613ddb840 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f48f7fff640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613ddb840 0x556613e04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f48f7fff640 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613ddb840 0x556613e04260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99aec1c2d9ad348d server_cookie=b49f2a3ba8a16e29 in_seq=0 out_seq=0 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f48d6ffd640 10 client.4863 ms_handle_connect on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f48d6ffd640 1 -- 172.21.15.148:0/4251523716 <== mds.0 v2:172.21.15.188:6833/1500268127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x556613d4cf50 con 0x556613ddb840 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f48d6ffd640 10 client.4863 handle_command_reply: tid=0 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 client.4863 shutdown 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 msgr2=0x556613e18e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e18e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48ec014000 tx=0x7f48ec0046c0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 shutdown_connections 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:6833/1500268127,v1:172.21.15.188:6835/1500268127] conn(0x556613ddb840 0x556613e04260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556613e24ad0 0x556613e193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556613e07ef0 0x556613e18e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f48d6ffd640 0 client.4863 ms_handle_reset on v2:172.21.15.188:6833/1500268127 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 --2- 172.21.15.148:0/4251523716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556613e07230 0x556613e21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 >> 172.21.15.148:0/4251523716 conn(0x556613ddcf10 msgr2=0x556613e24080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 shutdown_connections 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.906+0000 7f49015f9740 1 -- 172.21.15.148:0/4251523716 wait complete. 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f49015f9740 20 client.4863 trim_cache size 0 max 16384 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 4291, 2024-04-25T09:54:55.916 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 31.938416543999999, 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 9, 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 25, 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 39.301407546999997 2024-04-25T09:54:55.917 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48e406b4d0 msgr2=0x7f48e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48e406b4d0 0x7f48e406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48e800b7c0 tx=0x7f48e8008040 comp rx=0 tx=0).stop 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 msgr2=0x7f48f81544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 0x7f48f81544b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009f40 tx=0x7f48f0004a40 comp rx=0 tx=0).stop 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 shutdown_connections 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48e406b4d0 0x7f48e406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f815dac0 0x7f48f81544b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f80d34c0 0x7f48f8153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 --2- 172.21.15.148:0/2683844359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f80d1f30 0x7f48f815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:55.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 >> 172.21.15.148:0/2683844359 conn(0x7f48f8061c30 msgr2=0x7f48f80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:55.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 shutdown_connections 2024-04-25T09:54:55.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:55.907+0000 7f48fe815640 1 -- 172.21.15.148:0/2683844359 wait complete. 2024-04-25T09:54:55.920 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T09:54:55.934 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T09:54:55.934 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T09:54:55.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:55.934 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:54:55.949 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T09:54:55.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:55.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T09:54:56.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.013 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-04-25T09:54:56.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.107 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:54:56.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-04-25T09:54:56.188 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-04-25T09:54:56.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:54:56.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.189 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-25T09:54:56.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.208 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-04-25T09:54:56.280 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-04-25T09:54:56.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:54:56.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.282 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-25T09:54:56.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.339 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-04-25T09:54:56.407 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-04-25T09:54:56.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:54:56.409 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T09:54:56.409 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:54:56.409 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:56.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:54:56.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.504+0000 7ffbdfa4a640 1 Processor -- start 2024-04-25T09:54:56.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.504+0000 7ffbdfa4a640 1 -- start start 2024-04-25T09:54:56.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbdfa4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbd80ce590 0x7ffbd8165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbdfa4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8071480 con 0x7ffbd80cfec0 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbdfa4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8071760 con 0x7ffbd8154ee0 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbddfe7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffbd80ce590 0x7ffbd8165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbdcfe5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffbd80cfec0 0x7ffbd80cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51384/0 (socket says 172.21.15.148:51384) 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbdfa4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd8072090 con 0x7ffbd80ce590 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbdcfe5640 1 -- 172.21.15.148:0/2623136815 learned_addr learned my addr 172.21.15.148:0/2623136815 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbddfe7640 1 -- 172.21.15.148:0/2623136815 >> v1:172.21.15.188:6789/0 conn(0x7ffbd8154ee0 legacy=0x7ffbd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbddfe7640 1 -- 172.21.15.148:0/2623136815 >> v1:172.21.15.148:6789/0 conn(0x7ffbd80cfec0 legacy=0x7ffbd80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbcffff640 1 -- 172.21.15.148:0/2623136815 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243874294 0 0) 0x7ffbd8071480 con 0x7ffbd80cfec0 2024-04-25T09:54:56.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.505+0000 7ffbddfe7640 1 -- 172.21.15.148:0/2623136815 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8071a40 con 0x7ffbd80ce590 2024-04-25T09:54:56.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.506+0000 7ffbddfe7640 1 --2- 172.21.15.148:0/2623136815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd8165900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009d90 tx=0x7ffbd002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319532952e03943a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.506+0000 7ffbcffff640 1 -- 172.21.15.148:0/2623136815 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0007740 con 0x7ffbd80ce590 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.506+0000 7ffbcffff640 1 -- 172.21.15.148:0/2623136815 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd00078e0 con 0x7ffbd80ce590 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.506+0000 7ffbcffff640 1 -- 172.21.15.148:0/2623136815 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0007cb0 con 0x7ffbd80ce590 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/2623136815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 msgr2=0x7ffbd8165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/2623136815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd8165900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009d90 tx=0x7ffbd002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/2623136815 shutdown_connections 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/2623136815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd8165900 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/2623136815 >> 172.21.15.148:0/2623136815 conn(0x7ffbd806abe0 msgr2=0x7ffbd806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/2623136815 shutdown_connections 2024-04-25T09:54:56.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.507+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/2623136815 wait complete. 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.508+0000 7ffbdfa4a640 1 Processor -- start 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.508+0000 7ffbdfa4a640 1 -- start start 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd80cfec0 0x7ffbd81423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbd8154ee0 0x7ffbd80b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81552c0 con 0x7ffbd80ce590 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8071230 con 0x7ffbd80cfec0 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8155440 con 0x7ffbd8154ee0 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd813f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47494/0 (socket says 172.21.15.148:47494) 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 -- 172.21.15.148:0/1607429364 learned_addr learned my addr 172.21.15.148:0/1607429364 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdcfe5640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd80cfec0 0x7ffbd81423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbddfe7640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbd8154ee0 0x7ffbd80b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 -- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbd8154ee0 msgr2=0x7ffbd80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbd8154ee0 0x7ffbd80b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 -- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd80cfec0 msgr2=0x7ffbd81423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd80cfec0 0x7ffbd81423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 -- 172.21.15.148:0/1607429364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8142910 con 0x7ffbd80ce590 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdcfe5640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd80cfec0 0x7ffbd81423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbddfe7640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbd8154ee0 0x7ffbd80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdd7e6640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd813f190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400c400 tx=0x7ffbd400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4017070 con 0x7ffbd80ce590 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd40052d0 con 0x7ffbd80ce590 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd8155960 con 0x7ffbd80ce590 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4003830 con 0x7ffbd80ce590 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.509+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd8155f20 con 0x7ffbd80ce590 2024-04-25T09:54:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.510+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd8073990 con 0x7ffbd80ce590 2024-04-25T09:54:56.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.511+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbd401a070 con 0x7ffbd80ce590 2024-04-25T09:54:56.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.511+0000 7ffbcdffb640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:56.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.511+0000 7ffbdcfe5640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:56.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.511+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffbd4089810 con 0x7ffbd80ce590 2024-04-25T09:54:56.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.512+0000 7ffbdcfe5640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8156930 tx=0x7ffbc8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:56.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.516+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbd40583e0 con 0x7ffbd80ce590 2024-04-25T09:54:56.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.721+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffbd806d360 con 0x7ffbd80ce590 2024-04-25T09:54:56.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.721+0000 7ffbcdffb640 1 -- 172.21.15.148:0/1607429364 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ffbd4087e20 con 0x7ffbd80ce590 2024-04-25T09:54:56.724 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:54:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 msgr2=0x7ffbbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8156930 tx=0x7ffbc8009040 comp rx=0 tx=0).stop 2024-04-25T09:54:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 msgr2=0x7ffbd813f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd813f190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400c400 tx=0x7ffbd400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 shutdown_connections 2024-04-25T09:54:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbd8154ee0 0x7ffbd80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd80cfec0 0x7ffbd81423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 --2- 172.21.15.148:0/1607429364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd80ce590 0x7ffbd813f190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 >> 172.21.15.148:0/1607429364 conn(0x7ffbd806abe0 msgr2=0x7ffbd815c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 shutdown_connections 2024-04-25T09:54:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:56.723+0000 7ffbdfa4a640 1 -- 172.21.15.148:0/1607429364 wait complete. 2024-04-25T09:54:56.736 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:54:56.737 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:56.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.738 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:54:56.752 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:56.752 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:56.753 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:56.754 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:54:56.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:54:56.755 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:56.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.846 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:54:56.868 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:54:56.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.870 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:54:56.926 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:54:56.926 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:54:56.926 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:54:56.926 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:54:56.926 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:54:56.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:56.927 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:56.927 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:56.927 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:54:56.927 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:54:56.927 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:54:56.927 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:57.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.010 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:54:57.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.065 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-04-25T09:54:57.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.120 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:57.121 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:57.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:54:57.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T09:54:57.121 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:57.197 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T09:54:57.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.197 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:57.197 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:57.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T09:54:57.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:54:57.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T09:54:57.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T09:54:57.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T09:54:57.198 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:57.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.306 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:57.306 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:57.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:54:57.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:54:57.306 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:57.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:57.344 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:57.399 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:54:57.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:57.399 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:57.454 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:54:57.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.479 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:54:57.536 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:54:57.536 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:54:57.536 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:54:57.640 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:54:57.641 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:54:57.641 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:54:57.641 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:54:57.641 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:54:57.645 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:54:57.646 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:54:57.647 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:54:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:54:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:54:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:54:57.647 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:54:57.648 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:54:57.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.648 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:54:57.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.712 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:57.768 INFO:teuthology.orchestra.run.smithi148.stdout:client4556 2024-04-25T09:54:57.769 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:54:57.769 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:57.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.824 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:57.881 INFO:teuthology.orchestra.run.smithi148.stdout:client4556 2024-04-25T09:54:57.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.881 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:57.937 INFO:teuthology.orchestra.run.smithi148.stdout:client4556 2024-04-25T09:54:57.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:57.937 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/086ed9ea-68e4-4e62-83c5-c0821c62dd17.client4556/status') 2024-04-25T09:54:58.000 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.4556 (3)172.21.15.148:0/2174727057 2024-04-25T09:54:58.000 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-04-25T09:54:58.000 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-04-25T09:54:58.001 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-04-25T09:54:58.001 INFO:teuthology.orchestra.run.smithi148.stderr:71 bytes copied, 5.4525e-05 s, 1.3 MB/s 2024-04-25T09:54:58.002 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T09:54:58.002 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:54:58.002 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:54:58.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.137+0000 7fa9b205a640 1 Processor -- start 2024-04-25T09:54:58.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.137+0000 7fa9b205a640 1 -- start start 2024-04-25T09:54:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.138+0000 7fa9b205a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9ac15dbe0 0x7fa9ac0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.138+0000 7fa9b205a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.138+0000 7fa9b205a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac0b97a0 con 0x7fa9ac0b92a0 2024-04-25T09:54:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.138+0000 7fa9b205a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac15dfc0 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.138+0000 7fa9b205a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0d1a00 con 0x7fa9ac0b2cb0 2024-04-25T09:54:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9ac15dbe0 0x7fa9ac0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9aaffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9ac15dbe0 0x7fa9ac0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:56888/0 (socket says 172.21.15.148:56888) 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 -- 172.21.15.148:0/1825228557 learned_addr learned my addr 172.21.15.148:0/1825228557 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 -- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b2cb0 msgr2=0x7fa9ac0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 --2- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 -- 172.21.15.148:0/1825228557 >> v1:172.21.15.188:6789/0 conn(0x7fa9ac0b92a0 legacy=0x7fa9ac0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9ab7fe640 1 -- 172.21.15.148:0/1825228557 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0d1c50 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.139+0000 7fa9aaffd640 1 --2- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9ab7fe640 1 --2- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac0d0910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009870 tx=0x7fa99c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5753eb97300dc09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9aa7fc640 1 -- 172.21.15.148:0/1825228557 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c037070 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9aa7fc640 1 -- 172.21.15.148:0/1825228557 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c032440 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9aa7fc640 1 -- 172.21.15.148:0/1825228557 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c032810 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 -- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac15dbe0 msgr2=0x7fa9ac0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 --2- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac0d0910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009870 tx=0x7fa99c02a250 comp rx=0 tx=0).stop 2024-04-25T09:54:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 -- 172.21.15.148:0/1825228557 shutdown_connections 2024-04-25T09:54:58.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 --2- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 --2- 172.21.15.148:0/1825228557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac0d0910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 -- 172.21.15.148:0/1825228557 >> 172.21.15.148:0/1825228557 conn(0x7fa9ac061a20 msgr2=0x7fa9ac061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:58.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 -- 172.21.15.148:0/1825228557 shutdown_connections 2024-04-25T09:54:58.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.140+0000 7fa9b205a640 1 -- 172.21.15.148:0/1825228557 wait complete. 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.142+0000 7fa9b205a640 1 Processor -- start 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.142+0000 7fa9b205a640 1 -- start start 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9b205a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9b205a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b92a0 0x7fa9ac1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9b205a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9b205a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0b6ab0 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9b205a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0d2210 con 0x7fa9ac0b2cb0 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9b205a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0b6c30 con 0x7fa9ac0b92a0 2024-04-25T09:54:58.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47506/0 (socket says 172.21.15.148:47506) 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 -- 172.21.15.148:0/720496554 learned_addr learned my addr 172.21.15.148:0/720496554 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9aaffd640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b92a0 0x7fa9ac1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9ab7fe640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 -- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b92a0 msgr2=0x7fa9ac1522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b92a0 0x7fa9ac1522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 -- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac0b2cb0 msgr2=0x7fa9ac0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 -- 172.21.15.148:0/720496554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac150380 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.143+0000 7fa9abfff640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac152820 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c730 tx=0x7fa9a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.145+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4003b70 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.145+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a4003d10 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.145+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac0c9e70 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.145+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400aa90 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.145+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac1531a0 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.146+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac073990 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.146+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9a400ac30 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.147+0000 7fa993fff640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa99806b4d0 0x7fa99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.147+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa9a408a360 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.147+0000 7fa9ab7fe640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa99806b4d0 0x7fa99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.147+0000 7fa9ab7fe640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa99806b4d0 0x7fa99806d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa99c034000 tx=0x7fa99c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:58.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.151+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9ac150380 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.360+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9ac0d2c50 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.360+0000 7fa993fff640 1 -- 172.21.15.148:0/720496554 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa9a40587c0 con 0x7fa9ac15dbe0 2024-04-25T09:54:58.363 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa99806b4d0 msgr2=0x7fa99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa99806b4d0 0x7fa99806d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa99c034000 tx=0x7fa99c02e000 comp rx=0 tx=0).stop 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 msgr2=0x7fa9ac152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac152820 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c730 tx=0x7fa9a400cbf0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 shutdown_connections 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa99806b4d0 0x7fa99806d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9ac15dbe0 0x7fa9ac152820 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9ac0b92a0 0x7fa9ac1522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 --2- 172.21.15.148:0/720496554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9ac0b2cb0 0x7fa9ac0c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 >> 172.21.15.148:0/720496554 conn(0x7fa9ac061a20 msgr2=0x7fa9ac160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 shutdown_connections 2024-04-25T09:54:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:58.362+0000 7fa9b205a640 1 -- 172.21.15.148:0/720496554 wait complete. 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:54:58.378 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:54:58.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.379 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:58.394 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42737sec preferred_lft 42737sec 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:54:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link tentative 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.396 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:54:58.456 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:54:58.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.459 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:54:58.514 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:54:58.515 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:54:58.515 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:54:58.515 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:54:58.515 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:54:58.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.515 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:58.515 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:58.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:54:58.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:54:58.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:54:58.516 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:58.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.599 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:54:58.654 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:54:58.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.655 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-04-25T09:54:58.709 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T09:54:58.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.709 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:58.709 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:58.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:54:58.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T09:54:58.710 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:58.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:54:58.847 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:58.847 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether c2:3c:af:e2:05:a4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c03c:afff:fee2:5a4/64 scope link 2024-04-25T09:54:58.848 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T09:54:58.912 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:54:58.912 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:54:58.912 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:54:58.912 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether c2:3c:af:e2:05:a4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c03c:afff:fee2:5a4/64 scope link 2024-04-25T09:54:58.913 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:54:58.914 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T09:54:58.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T09:54:58.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T09:54:58.916 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:59.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.022 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:54:59.022 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:54:59.022 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2024-04-25T09:54:59.022 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:54:59.022 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:54:59.057 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:59.057 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T09:54:59.116 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T09:54:59.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:54:59.117 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:54:59.172 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:54:59.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T09:54:59.254 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T09:54:59.254 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T09:54:59.254 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:54:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:54:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T09:54:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T09:54:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:54:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:54:59.271 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:54:59.272 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:54:59.273 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:54:59.274 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:54:59.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T09:54:59.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.339 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:59.395 INFO:teuthology.orchestra.run.smithi148.stdout:client4756 2024-04-25T09:54:59.395 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:54:59.395 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:54:59.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.451 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:59.508 INFO:teuthology.orchestra.run.smithi148.stdout:client4756 2024-04-25T09:54:59.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.508 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:54:59.564 INFO:teuthology.orchestra.run.smithi148.stdout:client4756 2024-04-25T09:54:59.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.565 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/086ed9ea-68e4-4e62-83c5-c0821c62dd17.client4756/status') 2024-04-25T09:54:59.628 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.4756 (3)172.21.15.148:0/464104506 2024-04-25T09:54:59.629 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-04-25T09:54:59.629 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-04-25T09:54:59.629 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-04-25T09:54:59.629 INFO:teuthology.orchestra.run.smithi148.stderr:70 bytes copied, 2.2239e-05 s, 3.1 MB/s 2024-04-25T09:54:59.630 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T09:54:59.630 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:54:59.630 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:54:59.631 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:54:59.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.764+0000 7fd9fb283640 1 Processor -- start 2024-04-25T09:54:59.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.765+0000 7fd9fb283640 1 -- start start 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.765+0000 7fd9fb283640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9f41602e0 0x7fd9f40ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.765+0000 7fd9fb283640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f415f5f0 0x7fd9f415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.765+0000 7fd9fb283640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f40cd710 con 0x7fd9f415ba00 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.765+0000 7fd9fb283640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f415ff80 con 0x7fd9f415f5f0 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.765+0000 7fd9fb283640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f4099b70 con 0x7fd9f41602e0 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f9820640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f415f5f0 0x7fd9f415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f901f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9f415ba00 0x7fd9f409a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51388/0 (socket says 172.21.15.148:51388) 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f901f640 1 -- 172.21.15.148:0/2340265483 learned_addr learned my addr 172.21.15.148:0/2340265483 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f9820640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f415f5f0 0x7fd9f415fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47516/0 (socket says 172.21.15.148:47516) 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f881e640 1 --2- 172.21.15.148:0/2340265483 >> v2:172.21.15.188:3300/0 conn(0x7fd9f41602e0 0x7fd9f40ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f9820640 1 -- 172.21.15.148:0/2340265483 >> v2:172.21.15.188:3300/0 conn(0x7fd9f41602e0 msgr2=0x7fd9f40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f9820640 1 --2- 172.21.15.148:0/2340265483 >> v2:172.21.15.188:3300/0 conn(0x7fd9f41602e0 0x7fd9f40ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f9820640 1 -- 172.21.15.148:0/2340265483 >> v1:172.21.15.148:6789/0 conn(0x7fd9f415ba00 legacy=0x7fd9f409a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9e7fff640 1 -- 172.21.15.148:0/2340265483 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809720421 0 0) 0x7fd9e8001110 con 0x7fd9f415ba00 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f9820640 1 -- 172.21.15.148:0/2340265483 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4099e50 con 0x7fd9f415f5f0 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.766+0000 7fd9f881e640 1 --2- 172.21.15.148:0/2340265483 >> v2:172.21.15.188:3300/0 conn(0x7fd9f41602e0 0x7fd9f40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9f9820640 1 --2- 172.21.15.148:0/2340265483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415f5f0 0x7fd9f415fa40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40b7830 tx=0x7fd9ec02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624ca73423b99e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9e7fff640 1 -- 172.21.15.148:0/2340265483 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec007740 con 0x7fd9f415f5f0 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9e7fff640 1 -- 172.21.15.148:0/2340265483 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0078e0 con 0x7fd9f415f5f0 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9e7fff640 1 -- 172.21.15.148:0/2340265483 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec007cb0 con 0x7fd9f415f5f0 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 -- 172.21.15.148:0/2340265483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415f5f0 msgr2=0x7fd9f415fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 --2- 172.21.15.148:0/2340265483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415f5f0 0x7fd9f415fa40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40b7830 tx=0x7fd9ec02a560 comp rx=0 tx=0).stop 2024-04-25T09:54:59.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 -- 172.21.15.148:0/2340265483 shutdown_connections 2024-04-25T09:54:59.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 --2- 172.21.15.148:0/2340265483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415f5f0 0x7fd9f415fa40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 --2- 172.21.15.148:0/2340265483 >> v2:172.21.15.188:3300/0 conn(0x7fd9f41602e0 0x7fd9f40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 -- 172.21.15.148:0/2340265483 >> 172.21.15.148:0/2340265483 conn(0x7fd9f40b3270 msgr2=0x7fd9f40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 -- 172.21.15.148:0/2340265483 shutdown_connections 2024-04-25T09:54:59.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.767+0000 7fd9fb283640 1 -- 172.21.15.148:0/2340265483 wait complete. 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 Processor -- start 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 -- start start 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 0x7fd9f40b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f415f5f0 0x7fd9f40b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f41602e0 0x7fd9f40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f415bde0 con 0x7fd9f415ba00 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40cd2e0 con 0x7fd9f41602e0 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9fb283640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40b1920 con 0x7fd9f415f5f0 2024-04-25T09:54:59.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9f901f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 0x7fd9f40b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9f901f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 0x7fd9f40b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47522/0 (socket says 172.21.15.148:47522) 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.769+0000 7fd9f901f640 1 -- 172.21.15.148:0/4278565788 learned_addr learned my addr 172.21.15.148:0/4278565788 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f881e640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f415f5f0 0x7fd9f40b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f901f640 1 -- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f415f5f0 msgr2=0x7fd9f40b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f9820640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f41602e0 0x7fd9f40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f901f640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f415f5f0 0x7fd9f40b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f901f640 1 -- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f41602e0 msgr2=0x7fd9f40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f901f640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f41602e0 0x7fd9f40c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f901f640 1 -- 172.21.15.148:0/4278565788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40b1640 con 0x7fd9f415ba00 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f881e640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f415f5f0 0x7fd9f40b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f9820640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f41602e0 0x7fd9f40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9f901f640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 0x7fd9f40b53f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c780 tx=0x7fd9e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8018070 con 0x7fd9f415ba00 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e80072d0 con 0x7fd9f415ba00 2024-04-25T09:54:59.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f40c8b90 con 0x7fd9f415ba00 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80047e0 con 0x7fd9f415ba00 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.770+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f40c9150 con 0x7fd9f415ba00 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.771+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f4067be0 con 0x7fd9f415ba00 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.771+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9e801b070 con 0x7fd9f415ba00 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.772+0000 7fd9e5ffb640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d806b4d0 0x7fd9d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.772+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd9e80898a0 con 0x7fd9f415ba00 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.772+0000 7fd9f881e640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d806b4d0 0x7fd9d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:54:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.772+0000 7fd9f881e640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d806b4d0 0x7fd9d806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0006fd0 tx=0x7fd9f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:54:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.776+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9f4067be0 con 0x7fd9f415ba00 2024-04-25T09:54:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.981+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9f40ae790 con 0x7fd9f415ba00 2024-04-25T09:54:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.982+0000 7fd9e5ffb640 1 -- 172.21.15.148:0/4278565788 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd9f40cd710 con 0x7fd9f415ba00 2024-04-25T09:54:59.985 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d806b4d0 msgr2=0x7fd9d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d806b4d0 0x7fd9d806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0006fd0 tx=0x7fd9f0008040 comp rx=0 tx=0).stop 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 msgr2=0x7fd9f40b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 0x7fd9f40b53f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c780 tx=0x7fd9e800cc40 comp rx=0 tx=0).stop 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 shutdown_connections 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d806b4d0 0x7fd9d806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f41602e0 0x7fd9f40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f415f5f0 0x7fd9f40b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 --2- 172.21.15.148:0/4278565788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f415ba00 0x7fd9f40b53f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 >> 172.21.15.148:0/4278565788 conn(0x7fd9f40b3270 msgr2=0x7fd9f40c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:54:59.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 shutdown_connections 2024-04-25T09:54:59.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:54:59.984+0000 7fd9fb283640 1 -- 172.21.15.148:0/4278565788 wait complete. 2024-04-25T09:54:59.997 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:54:59.997 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:54:59.997 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-04-25T09:54:59.997 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T09:54:59.997 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T09:54:59.997 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:54:59.998 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:54:59.998 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:54:59.998 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:54:59.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:54:59.998 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-04-25T09:55:00.017 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:00.018 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::9028:5c12:d0ba:a8f0/64 scope link noprefixroute 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:00.019 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T09:55:00.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T09:55:00.020 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:00.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.376 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:55:00.401 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-25T09:55:00.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.403 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-04-25T09:55:00.473 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-04-25T09:55:00.474 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:55:00.474 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:55:00.474 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:55:00.474 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:55:00.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.474 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:00.474 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:00.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:55:00.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:55:00.475 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:55:00.475 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:00.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.714 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-25T09:55:00.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.729 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-04-25T09:55:00.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.783 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:00.783 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:00.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:55:00.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T09:55:00.784 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:00.859 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T09:55:00.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T09:55:00.859 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:00.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:00.981 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:00.981 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:00.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-04-25T09:55:00.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T09:55:00.981 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:01.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:01.017 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T09:55:01.071 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T09:55:01.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:01.071 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:55:01.126 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:55:01.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.126 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T09:55:01.212 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T09:55:01.212 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T09:55:01.212 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-25T09:55:01.318 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T09:55:01.318 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T09:55:01.318 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-25T09:55:01.318 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:55:01.318 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:55:01.333 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:55:01.334 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T09:55:01.334 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:55:01.334 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:55:01.334 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:55:01.334 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:55:01.336 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:55:01.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.336 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T09:55:01.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.359 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:01.415 INFO:teuthology.orchestra.run.smithi188.stdout:client4565 2024-04-25T09:55:01.416 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:01.416 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:01.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.471 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:01.527 INFO:teuthology.orchestra.run.smithi188.stdout:client4565 2024-04-25T09:55:01.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.528 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:01.585 INFO:teuthology.orchestra.run.smithi188.stdout:client4565 2024-04-25T09:55:01.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.586 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/086ed9ea-68e4-4e62-83c5-c0821c62dd17.client4565/status') 2024-04-25T09:55:01.649 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4565 (3)192.168.0.1:0/821164471 2024-04-25T09:55:01.649 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-04-25T09:55:01.649 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-04-25T09:55:01.649 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-04-25T09:55:01.650 INFO:teuthology.orchestra.run.smithi188.stderr:68 bytes copied, 2.0601e-05 s, 3.3 MB/s 2024-04-25T09:55:01.651 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T09:55:01.651 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T09:55:01.651 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:55:01.651 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:55:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.756+0000 7ff9daecb640 1 Processor -- start 2024-04-25T09:55:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.756+0000 7ff9daecb640 1 -- start start 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9daecb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9d40d0170 0x7ff9d40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9daecb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d40c8030 con 0x7ff9d415da70 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9daecb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d40c8310 con 0x7ff9d40ce700 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9daecb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d40ceb30 con 0x7ff9d40d0170 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9d40d0170 0x7ff9d40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff9d40d0170 0x7ff9d40ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47532/0 (socket says 172.21.15.148:47532) 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d3fff640 1 -- 172.21.15.148:0/3120377835 learned_addr learned my addr 172.21.15.148:0/3120377835 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d3fff640 1 -- 172.21.15.148:0/3120377835 >> v1:172.21.15.188:6789/0 conn(0x7ff9d40ce700 legacy=0x7ff9d406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d3fff640 1 -- 172.21.15.148:0/3120377835 >> v1:172.21.15.148:6789/0 conn(0x7ff9d415da70 legacy=0x7ff9d40c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d3fff640 1 -- 172.21.15.148:0/3120377835 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40c85f0 con 0x7ff9d40d0170 2024-04-25T09:55:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.757+0000 7ff9d37fe640 1 -- 172.21.15.148:0/3120377835 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994118695 0 0) 0x7ff9d40c8030 con 0x7ff9d415da70 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.758+0000 7ff9d3fff640 1 --2- 172.21.15.148:0/3120377835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d40d0170 0x7ff9d40ce1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1553daee1382b45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.758+0000 7ff9d37fe640 1 -- 172.21.15.148:0/3120377835 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8037070 con 0x7ff9d40d0170 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.758+0000 7ff9d37fe640 1 -- 172.21.15.148:0/3120377835 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8032440 con 0x7ff9d40d0170 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.758+0000 7ff9d37fe640 1 -- 172.21.15.148:0/3120377835 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8032810 con 0x7ff9d40d0170 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 -- 172.21.15.148:0/3120377835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d40d0170 msgr2=0x7ff9d40ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 --2- 172.21.15.148:0/3120377835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d40d0170 0x7ff9d40ce1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009870 tx=0x7ff9c802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 -- 172.21.15.148:0/3120377835 shutdown_connections 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 --2- 172.21.15.148:0/3120377835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d40d0170 0x7ff9d40ce1c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 -- 172.21.15.148:0/3120377835 >> 172.21.15.148:0/3120377835 conn(0x7ff9d40b33e0 msgr2=0x7ff9d40b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 -- 172.21.15.148:0/3120377835 shutdown_connections 2024-04-25T09:55:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.759+0000 7ff9daecb640 1 -- 172.21.15.148:0/3120377835 wait complete. 2024-04-25T09:55:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.760+0000 7ff9daecb640 1 Processor -- start 2024-04-25T09:55:01.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.760+0000 7ff9daecb640 1 -- start start 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9daecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9d40ce700 0x7ff9d414f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9daecb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d40d0170 0x7ff9d40c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9daecb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 0x7ff9d40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9daecb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d415de50 con 0x7ff9d415da70 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9daecb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d406be10 con 0x7ff9d40d0170 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9daecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d406ca40 con 0x7ff9d40ce700 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9d9468640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 0x7ff9d40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9d9468640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 0x7ff9d40c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47546/0 (socket says 172.21.15.148:47546) 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9d9468640 1 -- 172.21.15.148:0/1732891999 learned_addr learned my addr 172.21.15.148:0/1732891999 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9d3fff640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d40d0170 0x7ff9d40c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9d8c67640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9d40ce700 0x7ff9d414f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.761+0000 7ff9d9468640 1 -- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9d40ce700 msgr2=0x7ff9d414f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d9468640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9d40ce700 0x7ff9d414f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d9468640 1 -- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d40d0170 msgr2=0x7ff9d40c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d9468640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d40d0170 0x7ff9d40c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d9468640 1 -- 172.21.15.148:0/1732891999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40c91f0 con 0x7ff9d415da70 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d9468640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 0x7ff9d40c9fd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c780 tx=0x7ff9cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc017070 con 0x7ff9d415da70 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc003da0 con 0x7ff9d415da70 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0052b0 con 0x7ff9d415da70 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d40c94d0 con 0x7ff9d415da70 2024-04-25T09:55:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.762+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d40c9a90 con 0x7ff9d415da70 2024-04-25T09:55:01.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.763+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d4073990 con 0x7ff9d415da70 2024-04-25T09:55:01.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.763+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9cc01a070 con 0x7ff9d415da70 2024-04-25T09:55:01.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.764+0000 7ff9d17fa640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9c006b4d0 0x7ff9c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:01.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.764+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff9cc089b00 con 0x7ff9d415da70 2024-04-25T09:55:01.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.764+0000 7ff9d8c67640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9c006b4d0 0x7ff9c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:01.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.764+0000 7ff9d8c67640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9c006b4d0 0x7ff9c006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4000f60 tx=0x7ff9c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.768+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9cc057ff0 con 0x7ff9d415da70 2024-04-25T09:55:01.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.973+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9d40d2640 con 0x7ff9d415da70 2024-04-25T09:55:01.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.973+0000 7ff9d17fa640 1 -- 172.21.15.148:0/1732891999 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff9cc05c090 con 0x7ff9d415da70 2024-04-25T09:55:01.976 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T09:55:01.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9c006b4d0 msgr2=0x7ff9c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9c006b4d0 0x7ff9c006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4000f60 tx=0x7ff9c4009040 comp rx=0 tx=0).stop 2024-04-25T09:55:01.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 msgr2=0x7ff9d40c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 0x7ff9d40c9fd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c780 tx=0x7ff9cc00cc40 comp rx=0 tx=0).stop 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 shutdown_connections 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9c006b4d0 0x7ff9c006d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9d415da70 0x7ff9d40c9fd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9d40d0170 0x7ff9d40c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 --2- 172.21.15.148:0/1732891999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9d40ce700 0x7ff9d414f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.975+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 >> 172.21.15.148:0/1732891999 conn(0x7ff9d40b33e0 msgr2=0x7ff9d406c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.976+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 shutdown_connections 2024-04-25T09:55:01.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:01.976+0000 7ff9daecb640 1 -- 172.21.15.148:0/1732891999 wait complete. 2024-04-25T09:55:01.989 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:55:01.989 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:55:01.989 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-04-25T09:55:01.989 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T09:55:01.989 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T09:55:01.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:55:01.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:55:01.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:55:01.990 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:55:01.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:01.990 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:02.006 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.007 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42706sec preferred_lft 42706sec 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::9028:5c12:d0ba:a8f0/64 scope link noprefixroute 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether b6:60:38:14:33:52 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::88e5:73ff:fe0e:e456/64 scope link 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether b6:60:38:14:33:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:55:02.008 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::b460:38ff:fe14:3352/64 scope link tentative 2024-04-25T09:55:02.009 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.009 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:55:02.069 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-04-25T09:55:02.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.071 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-04-25T09:55:02.127 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-04-25T09:55:02.127 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:55:02.127 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:55:02.127 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:55:02.127 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:55:02.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.128 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:02.128 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:02.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:55:02.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:55:02.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:55:02.128 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:02.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.213 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-25T09:55:02.227 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:55:02.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.228 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-04-25T09:55:02.282 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T09:55:02.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.283 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:02.283 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:02.283 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:55:02.283 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T09:55:02.283 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:02.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.358 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:55:02.420 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:02.420 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether d6:fc:49:ec:57:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::d4fc:49ff:feec:57b1/64 scope link tentative 2024-04-25T09:55:02.421 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.422 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:55:02.486 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether d6:fc:49:ec:57:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T09:55:02.487 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T09:55:02.487 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.487 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::d4fc:49ff:feec:57b1/64 scope link tentative 2024-04-25T09:55:02.487 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:02.488 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T09:55:02.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T09:55:02.488 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:02.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.595 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-04-25T09:55:02.595 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-04-25T09:55:02.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2024-04-25T09:55:02.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T09:55:02.595 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-04-25T09:55:02.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:02.629 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T09:55:02.684 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T09:55:02.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:02.685 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:55:02.739 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:55:02.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.740 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T09:55:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T09:55:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T09:55:02.820 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-25T09:55:02.829 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T09:55:02.829 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T09:55:02.829 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-25T09:55:02.829 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:55:02.829 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:55:02.838 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T09:55:02.838 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T09:55:02.838 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-04-25T09:55:02.838 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:55:02.839 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:55:02.839 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:55:02.839 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:55:02.839 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:55:02.840 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T09:55:02.840 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:55:02.840 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:55:02.840 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:55:02.840 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:55:02.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:55:02.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T09:55:02.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:02.905 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:02.963 INFO:teuthology.orchestra.run.smithi188.stdout:client4571 2024-04-25T09:55:02.963 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:02.963 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:03.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:03.018 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:03.074 INFO:teuthology.orchestra.run.smithi188.stdout:client4571 2024-04-25T09:55:03.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:03.075 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:03.130 INFO:teuthology.orchestra.run.smithi188.stdout:client4571 2024-04-25T09:55:03.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:03.130 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/086ed9ea-68e4-4e62-83c5-c0821c62dd17.client4571/status') 2024-04-25T09:55:03.193 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4571 (3)192.168.0.2:0/2029706557 2024-04-25T09:55:03.193 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-04-25T09:55:03.194 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-04-25T09:55:03.194 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-04-25T09:55:03.194 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 2.2027e-05 s, 3.1 MB/s 2024-04-25T09:55:03.195 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T09:55:03.210 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:03.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.309+0000 7f90e4653640 1 Processor -- start 2024-04-25T09:55:03.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.309+0000 7f90e4653640 1 -- start start 2024-04-25T09:55:03.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e4653640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc154ee0 0x7f90dc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e4653640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90dc071480 con 0x7f90dc0ce590 2024-04-25T09:55:03.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e4653640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90dc071760 con 0x7f90dc0cfcf0 2024-04-25T09:55:03.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e4653640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc072090 con 0x7f90dc154ee0 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e2bf0640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f90dc0ce590 0x7f90dc06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51416/0 (socket says 172.21.15.148:51416) 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e2bf0640 1 -- 172.21.15.148:0/1709940024 learned_addr learned my addr 172.21.15.148:0/1709940024 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e23ef640 1 --2- 172.21.15.148:0/1709940024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc154ee0 0x7f90dc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.310+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344052107 0 0) 0x7f90dc071480 con 0x7f90dc0ce590 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c8003520 con 0x7f90dc0ce590 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787363754 0 0) 0x7f90dc071760 con 0x7f90dc0cfcf0 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90dc071480 con 0x7f90dc0cfcf0 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e23ef640 1 -- 172.21.15.148:0/1709940024 >> v1:172.21.15.188:6789/0 conn(0x7f90dc0cfcf0 legacy=0x7f90dc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e23ef640 1 -- 172.21.15.148:0/1709940024 >> v1:172.21.15.148:6789/0 conn(0x7f90dc0ce590 legacy=0x7f90dc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e23ef640 1 -- 172.21.15.148:0/1709940024 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc071a40 con 0x7f90dc154ee0 2024-04-25T09:55:03.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.311+0000 7f90e23ef640 1 --2- 172.21.15.148:0/1709940024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc154ee0 0x7f90dc0cf7b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009dc0 tx=0x7f90d802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c3b2d9511702355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8007740 con 0x7f90dc154ee0 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80078e0 con 0x7f90dc154ee0 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e13ed640 1 -- 172.21.15.148:0/1709940024 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8007cb0 con 0x7f90dc154ee0 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 -- 172.21.15.148:0/1709940024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc154ee0 msgr2=0x7f90dc0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 --2- 172.21.15.148:0/1709940024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc154ee0 0x7f90dc0cf7b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009dc0 tx=0x7f90d802a6d0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 -- 172.21.15.148:0/1709940024 shutdown_connections 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 --2- 172.21.15.148:0/1709940024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc154ee0 0x7f90dc0cf7b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 -- 172.21.15.148:0/1709940024 >> 172.21.15.148:0/1709940024 conn(0x7f90dc06abe0 msgr2=0x7f90dc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 -- 172.21.15.148:0/1709940024 shutdown_connections 2024-04-25T09:55:03.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.312+0000 7f90e4653640 1 -- 172.21.15.148:0/1709940024 wait complete. 2024-04-25T09:55:03.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.313+0000 7f90e4653640 1 Processor -- start 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 -- start start 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90dc0ce590 0x7f90dc150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc0cfcf0 0x7f90dc1f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 0x7f90dc14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc071230 con 0x7f90dc154ee0 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc1552c0 con 0x7f90dc0ce590 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e4653640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc155440 con 0x7f90dc0cfcf0 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e2bf0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 0x7f90dc14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e2bf0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 0x7f90dc14dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47558/0 (socket says 172.21.15.148:47558) 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e2bf0640 1 -- 172.21.15.148:0/2652294597 learned_addr learned my addr 172.21.15.148:0/2652294597 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e1bee640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc0cfcf0 0x7f90dc1f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.314+0000 7f90e23ef640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90dc0ce590 0x7f90dc150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e2bf0640 1 -- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc0cfcf0 msgr2=0x7f90dc1f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e2bf0640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc0cfcf0 0x7f90dc1f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e2bf0640 1 -- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90dc0ce590 msgr2=0x7f90dc150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e2bf0640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90dc0ce590 0x7f90dc150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e2bf0640 1 -- 172.21.15.148:0/2652294597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc159bc0 con 0x7f90dc154ee0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e2bf0640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 0x7f90dc14dd70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90d400c460 tx=0x7f90d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4017070 con 0x7f90dc154ee0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d4005ce0 con 0x7f90dc154ee0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90dc159ea0 con 0x7f90dc154ee0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4007270 con 0x7f90dc154ee0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.315+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc15a460 con 0x7f90dc154ee0 2024-04-25T09:55:03.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.316+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90dc073990 con 0x7f90dc154ee0 2024-04-25T09:55:03.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.317+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90d401a070 con 0x7f90dc154ee0 2024-04-25T09:55:03.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.317+0000 7f90d2ffd640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.317+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f90d40897c0 con 0x7f90dc154ee0 2024-04-25T09:55:03.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.317+0000 7f90e23ef640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.317+0000 7f90e23ef640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90c006b4d0 0x7f90c006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f90d8034000 tx=0x7f90d802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.321+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90d4057c70 con 0x7f90dc154ee0 2024-04-25T09:55:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.491+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90dc06d360 con 0x7f90dc154ee0 2024-04-25T09:55:03.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.492+0000 7f90d2ffd640 1 -- 172.21.15.148:0/2652294597 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18966 (secure 0 0 0) 0x7f90d405bd10 con 0x7f90dc154ee0 2024-04-25T09:55:03.496 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:03.496 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:03.152782+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90c006b4d0 msgr2=0x7f90c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90c006b4d0 0x7f90c006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f90d8034000 tx=0x7f90d802f000 comp rx=0 tx=0).stop 2024-04-25T09:55:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 msgr2=0x7f90dc14dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 0x7f90dc14dd70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90d400c460 tx=0x7f90d400c920 comp rx=0 tx=0).stop 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 shutdown_connections 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90dc154ee0 0x7f90dc14dd70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90dc0cfcf0 0x7f90dc1f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 --2- 172.21.15.148:0/2652294597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90dc0ce590 0x7f90dc150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 >> 172.21.15.148:0/2652294597 conn(0x7f90dc06abe0 msgr2=0x7f90dc15d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 shutdown_connections 2024-04-25T09:55:03.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.494+0000 7f90e4653640 1 -- 172.21.15.148:0/2652294597 wait complete. 2024-04-25T09:55:03.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:03.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.607+0000 7f50680cb640 1 Processor -- start 2024-04-25T09:55:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.607+0000 7f50680cb640 1 -- start start 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f50680cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d0ab0 0x7f5060168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f50680cb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600d2420 con 0x7f506015dac0 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f50680cb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600d2700 con 0x7f50600d34c0 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f50680cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600d3030 con 0x7f50600d0ab0 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5065e67640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f506015dac0 0x7f50600d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43084/0 (socket says 172.21.15.148:43084) 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5065e67640 1 -- 172.21.15.148:0/2032747934 learned_addr learned my addr 172.21.15.148:0/2032747934 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5066668640 1 --2- 172.21.15.148:0/2032747934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d0ab0 0x7f5060168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446981009 0 0) 0x7f50600d2420 con 0x7f506015dac0 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f504c003520 con 0x7f506015dac0 2024-04-25T09:55:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063480558 0 0) 0x7f50600d2700 con 0x7f50600d34c0 2024-04-25T09:55:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50600d2420 con 0x7f50600d34c0 2024-04-25T09:55:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5066668640 1 -- 172.21.15.148:0/2032747934 >> v1:172.21.15.188:6789/0 conn(0x7f50600d34c0 legacy=0x7f50600d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5066668640 1 -- 172.21.15.148:0/2032747934 >> v1:172.21.15.148:6789/0 conn(0x7f506015dac0 legacy=0x7f50600d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3620654493 0 0) 0x7f504c003520 con 0x7f506015dac0 2024-04-25T09:55:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.608+0000 7f5066668640 1 -- 172.21.15.148:0/2032747934 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600d29e0 con 0x7f50600d0ab0 2024-04-25T09:55:03.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.609+0000 7f5066668640 1 --2- 172.21.15.148:0/2032747934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d0ab0 0x7f5060168670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50600d31b0 tx=0x7f505802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6f2f4dbb5b681a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.609+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c001670 con 0x7f50600d0ab0 2024-04-25T09:55:03.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.609+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505802ae50 con 0x7f50600d0ab0 2024-04-25T09:55:03.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.609+0000 7f5064e65640 1 -- 172.21.15.148:0/2032747934 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058007740 con 0x7f50600d0ab0 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 -- 172.21.15.148:0/2032747934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d0ab0 msgr2=0x7f5060168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 --2- 172.21.15.148:0/2032747934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d0ab0 0x7f5060168670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f50600d31b0 tx=0x7f505802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 -- 172.21.15.148:0/2032747934 shutdown_connections 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 --2- 172.21.15.148:0/2032747934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d0ab0 0x7f5060168670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 -- 172.21.15.148:0/2032747934 >> 172.21.15.148:0/2032747934 conn(0x7f5060061c30 msgr2=0x7f506006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 -- 172.21.15.148:0/2032747934 shutdown_connections 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.610+0000 7f50680cb640 1 -- 172.21.15.148:0/2032747934 wait complete. 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.611+0000 7f50680cb640 1 Processor -- start 2024-04-25T09:55:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.611+0000 7f50680cb640 1 -- start start 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50600d0ab0 0x7f5060153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d34c0 0x7f506015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 0x7f50601566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506015dea0 con 0x7f506015dac0 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600d2100 con 0x7f50600d0ab0 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600b97d0 con 0x7f50600d34c0 2024-04-25T09:55:03.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 0x7f50601566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 0x7f50601566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51214/0 (socket says 172.21.15.148:51214) 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 -- 172.21.15.148:0/691749341 learned_addr learned my addr 172.21.15.148:0/691749341 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5065e67640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50600d0ab0 0x7f5060153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5065666640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d34c0 0x7f506015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 -- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d34c0 msgr2=0x7f506015eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d34c0 0x7f506015eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 -- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50600d0ab0 msgr2=0x7f5060153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50600d0ab0 0x7f5060153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 -- 172.21.15.148:0/691749341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600b4450 con 0x7f506015dac0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f5066668640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 0x7f50601566b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5058009eb0 tx=0x7f5058004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058040070 con 0x7f506015dac0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505803c830 con 0x7f506015dac0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50600b4730 con 0x7f506015dac0 2024-04-25T09:55:03.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505803cc00 con 0x7f506015dac0 2024-04-25T09:55:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.612+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50600b4cf0 con 0x7f506015dac0 2024-04-25T09:55:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.613+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5060073a00 con 0x7f506015dac0 2024-04-25T09:55:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.614+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5058039040 con 0x7f506015dac0 2024-04-25T09:55:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.614+0000 7f50567fc640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f504406b4d0 0x7f504406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.614+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f50580ab880 con 0x7f506015dac0 2024-04-25T09:55:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.614+0000 7f5065e67640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f504406b4d0 0x7f504406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.615+0000 7f5065e67640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f504406b4d0 0x7f504406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f505c0011e0 tx=0x7f505c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.619+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f505807a2f0 con 0x7f506015dac0 2024-04-25T09:55:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.790+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50601536c0 con 0x7f506015dac0 2024-04-25T09:55:03.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.790+0000 7f50567fc640 1 -- 172.21.15.148:0/691749341 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5058038e10 con 0x7f506015dac0 2024-04-25T09:55:03.794 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f504406b4d0 msgr2=0x7f504406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f504406b4d0 0x7f504406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f505c0011e0 tx=0x7f505c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 msgr2=0x7f50601566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 0x7f50601566b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5058009eb0 tx=0x7f5058004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 shutdown_connections 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f504406b4d0 0x7f504406d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f506015dac0 0x7f50601566b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50600d34c0 0x7f506015eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 --2- 172.21.15.148:0/691749341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50600d0ab0 0x7f5060153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 >> 172.21.15.148:0/691749341 conn(0x7f5060061c30 msgr2=0x7f50600c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 shutdown_connections 2024-04-25T09:55:03.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.792+0000 7f50680cb640 1 -- 172.21.15.148:0/691749341 wait complete. 2024-04-25T09:55:03.807 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:03.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.903+0000 7f4937658640 1 Processor -- start 2024-04-25T09:55:03.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.903+0000 7f4937658640 1 -- start start 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4937658640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f493015ba80 0x7f49300cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4937658640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49300cf250 0x7f49300b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4937658640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493015ce20 con 0x7f493015c990 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4937658640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4930070130 con 0x7f49300cf250 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4937658640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493006fe50 con 0x7f493015ba80 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49300cf250 0x7f49300b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49300cf250 0x7f49300b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51220/0 (socket says 172.21.15.148:51220) 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 -- 172.21.15.148:0/2192111508 learned_addr learned my addr 172.21.15.148:0/2192111508 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f49353f4640 1 --2- 172.21.15.148:0/2192111508 >> v2:172.21.15.188:3300/0 conn(0x7f493015ba80 0x7f49300cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 -- 172.21.15.148:0/2192111508 >> v2:172.21.15.188:3300/0 conn(0x7f493015ba80 msgr2=0x7f49300cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 --2- 172.21.15.148:0/2192111508 >> v2:172.21.15.188:3300/0 conn(0x7f493015ba80 0x7f49300cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 -- 172.21.15.148:0/2192111508 >> v1:172.21.15.188:6789/0 conn(0x7f493015c990 legacy=0x7f49300ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:03.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.904+0000 7f4935bf5640 1 -- 172.21.15.148:0/2192111508 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300cc8b0 con 0x7f49300cf250 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.905+0000 7f4935bf5640 1 --2- 172.21.15.148:0/2192111508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f49300b2a90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4930071270 tx=0x7f492802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c841d5c7938cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.905+0000 7f4927fff640 1 -- 172.21.15.148:0/2192111508 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928037070 con 0x7f49300cf250 2024-04-25T09:55:03.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.905+0000 7f4927fff640 1 -- 172.21.15.148:0/2192111508 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928032440 con 0x7f49300cf250 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.905+0000 7f4927fff640 1 -- 172.21.15.148:0/2192111508 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928032810 con 0x7f49300cf250 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 -- 172.21.15.148:0/2192111508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 msgr2=0x7f49300b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 --2- 172.21.15.148:0/2192111508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f49300b2a90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4930071270 tx=0x7f492802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 -- 172.21.15.148:0/2192111508 shutdown_connections 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 --2- 172.21.15.148:0/2192111508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f49300b2a90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 --2- 172.21.15.148:0/2192111508 >> v2:172.21.15.188:3300/0 conn(0x7f493015ba80 0x7f49300cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 -- 172.21.15.148:0/2192111508 >> 172.21.15.148:0/2192111508 conn(0x7f493006abe0 msgr2=0x7f493006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 -- 172.21.15.148:0/2192111508 shutdown_connections 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.906+0000 7f4937658640 1 -- 172.21.15.148:0/2192111508 wait complete. 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.907+0000 7f4937658640 1 Processor -- start 2024-04-25T09:55:03.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.907+0000 7f4937658640 1 -- start start 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f4930156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f493015ba80 0x7f49300b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493015c990 0x7f49300a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493006fb00 con 0x7f49300cf250 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300cce70 con 0x7f493015c990 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493006b850 con 0x7f493015ba80 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f4930156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f4930156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51232/0 (socket says 172.21.15.148:51232) 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 -- 172.21.15.148:0/3819448952 learned_addr learned my addr 172.21.15.148:0/3819448952 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4935bf5640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493015c990 0x7f49300a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4934bf3640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f493015ba80 0x7f49300b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 -- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f493015ba80 msgr2=0x7f49300b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f493015ba80 0x7f49300b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 -- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493015c990 msgr2=0x7f49300a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493015c990 0x7f49300a8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 -- 172.21.15.148:0/3819448952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300b3560 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f49353f4640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f4930156b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f492c00ce10 tx=0x7f492c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c003b90 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c003d30 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300a91f0 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.908+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c004240 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.909+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300b3ae0 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.910+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930073a00 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.910+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f492c0043e0 con 0x7f49300cf250 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.910+0000 7f4925ffb640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491406b4d0 0x7f491406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:03.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.910+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f492c08b0a0 con 0x7f49300cf250 2024-04-25T09:55:03.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.910+0000 7f4934bf3640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491406b4d0 0x7f491406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:03.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.911+0000 7f4934bf3640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491406b4d0 0x7f491406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4920007fd0 tx=0x7f4920009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:03.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:03.914+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f492c01d340 con 0x7f49300cf250 2024-04-25T09:55:04.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.089+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f493006b630 con 0x7f49300cf250 2024-04-25T09:55:04.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.089+0000 7f4925ffb640 1 -- 172.21.15.148:0/3819448952 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f492c059ba0 con 0x7f49300cf250 2024-04-25T09:55:04.093 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491406b4d0 msgr2=0x7f491406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491406b4d0 0x7f491406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4920007fd0 tx=0x7f4920009040 comp rx=0 tx=0).stop 2024-04-25T09:55:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 msgr2=0x7f4930156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f4930156b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f492c00ce10 tx=0x7f492c00a660 comp rx=0 tx=0).stop 2024-04-25T09:55:04.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 shutdown_connections 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f491406b4d0 0x7f491406d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f493015c990 0x7f49300a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f493015ba80 0x7f49300b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 --2- 172.21.15.148:0/3819448952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49300cf250 0x7f4930156b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 >> 172.21.15.148:0/3819448952 conn(0x7f493006abe0 msgr2=0x7f49300cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 shutdown_connections 2024-04-25T09:55:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.091+0000 7f4937658640 1 -- 172.21.15.148:0/3819448952 wait complete. 2024-04-25T09:55:04.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:04.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.205+0000 7fdf6293b640 1 Processor -- start 2024-04-25T09:55:04.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.205+0000 7fdf6293b640 1 -- start start 2024-04-25T09:55:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf6293b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf5c15bb20 0x7fdf5c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf6293b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf6293b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0cb190 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf6293b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cb470 con 0x7fdf5c15bb20 2024-04-25T09:55:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf6293b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cb750 con 0x7fdf5c0ccdd0 2024-04-25T09:55:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf5c15bb20 0x7fdf5c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf5c15bb20 0x7fdf5c0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51236/0 (socket says 172.21.15.148:51236) 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 -- 172.21.15.148:0/1770625184 learned_addr learned my addr 172.21.15.148:0/1770625184 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5b7fe640 1 --2- 172.21.15.148:0/1770625184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 -- 172.21.15.148:0/1770625184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 msgr2=0x7fdf5c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/1770625184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c0cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 -- 172.21.15.148:0/1770625184 >> v1:172.21.15.188:6789/0 conn(0x7fdf5c0cdcd0 legacy=0x7fdf5c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.206+0000 7fdf5bfff640 1 -- 172.21.15.148:0/1770625184 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0cba30 con 0x7fdf5c15bb20 2024-04-25T09:55:04.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.207+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/1770625184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c0cc230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009870 tx=0x7fdf5002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c536eabd1c01ec6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.207+0000 7fdf5affd640 1 -- 172.21.15.148:0/1770625184 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50037070 con 0x7fdf5c15bb20 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.207+0000 7fdf5affd640 1 -- 172.21.15.148:0/1770625184 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50032440 con 0x7fdf5c15bb20 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.207+0000 7fdf5affd640 1 -- 172.21.15.148:0/1770625184 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50032810 con 0x7fdf5c15bb20 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 -- 172.21.15.148:0/1770625184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c15bb20 msgr2=0x7fdf5c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 --2- 172.21.15.148:0/1770625184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c0cc230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009870 tx=0x7fdf5002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 -- 172.21.15.148:0/1770625184 shutdown_connections 2024-04-25T09:55:04.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 --2- 172.21.15.148:0/1770625184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c0cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 --2- 172.21.15.148:0/1770625184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c0cc230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 -- 172.21.15.148:0/1770625184 >> 172.21.15.148:0/1770625184 conn(0x7fdf5c0b2390 msgr2=0x7fdf5c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 -- 172.21.15.148:0/1770625184 shutdown_connections 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.208+0000 7fdf6293b640 1 -- 172.21.15.148:0/1770625184 wait complete. 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.209+0000 7fdf6293b640 1 Processor -- start 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.209+0000 7fdf6293b640 1 -- start start 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf6293b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf6293b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 0x7fdf5c15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf6293b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf6293b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c15bf00 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf6293b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cc770 con 0x7fdf5c15bb20 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf6293b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c15c080 con 0x7fdf5c0ccdd0 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 0x7fdf5c15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 0x7fdf5c15e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51238/0 (socket says 172.21.15.148:51238) 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/4053063299 learned_addr learned my addr 172.21.15.148:0/4053063299 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:04.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf60ed8640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 msgr2=0x7fdf5c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5c15bb20 msgr2=0x7fdf5c156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 -- 172.21.15.148:0/4053063299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0b44f0 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.210+0000 7fdf5b7fe640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 0x7fdf5c15e8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00d800 tx=0x7fdf4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.211+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c00b6c0 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.211+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c019070 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.211+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c158880 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.211+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c00ba80 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.211+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c0b4a90 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.212+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c073a00 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.212+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf4c00bc20 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.213+0000 7fdf58ff9640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4806b4d0 0x7fdf4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.213+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4806b4d0 0x7fdf4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.213+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fdf4c08a970 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.213+0000 7fdf5bfff640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4806b4d0 0x7fdf4806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf50034000 tx=0x7fdf5002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.217+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf4c059470 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.390+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf5c15b340 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.390+0000 7fdf58ff9640 1 -- 172.21.15.148:0/4053063299 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf4c058d70 con 0x7fdf5c0cdcd0 2024-04-25T09:55:04.395 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:04.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.392+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4806b4d0 msgr2=0x7fdf4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.392+0000 7fdf6293b640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4806b4d0 0x7fdf4806d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf50034000 tx=0x7fdf5002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 msgr2=0x7fdf5c15e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 0x7fdf5c15e8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00d800 tx=0x7fdf4c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 shutdown_connections 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdf4806b4d0 0x7fdf4806d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf5c15bb20 0x7fdf5c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf5c0cdcd0 0x7fdf5c15e8f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 --2- 172.21.15.148:0/4053063299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf5c0ccdd0 0x7fdf5c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 >> 172.21.15.148:0/4053063299 conn(0x7fdf5c0b2390 msgr2=0x7fdf5c0cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 shutdown_connections 2024-04-25T09:55:04.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.393+0000 7fdf6293b640 1 -- 172.21.15.148:0/4053063299 wait complete. 2024-04-25T09:55:04.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:04.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.506+0000 7f027d29e640 1 Processor -- start 2024-04-25T09:55:04.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.506+0000 7f027d29e640 1 -- start start 2024-04-25T09:55:04.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f027d29e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f02780ce230 0x7f02780cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f027d29e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0278072280 con 0x7f02780cc4f0 2024-04-25T09:55:04.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f027d29e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780714c0 con 0x7f027815b270 2024-04-25T09:55:04.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f027d29e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0278071270 con 0x7f02780ce230 2024-04-25T09:55:04.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f0276ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f02780cc4f0 0x7f02780cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43086/0 (socket says 172.21.15.148:43086) 2024-04-25T09:55:04.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f0276ffd640 1 -- 172.21.15.148:0/2720331552 learned_addr learned my addr 172.21.15.148:0/2720331552 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:04.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f026ffff640 1 --2- 172.21.15.148:0/2720331552 >> v2:172.21.15.188:3300/0 conn(0x7f02780ce230 0x7f02780cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f02767fc640 1 -- 172.21.15.148:0/2720331552 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025872314 0 0) 0x7f0278072280 con 0x7f02780cc4f0 2024-04-25T09:55:04.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f02767fc640 1 -- 172.21.15.148:0/2720331552 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0264003520 con 0x7f02780cc4f0 2024-04-25T09:55:04.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f02767fc640 1 -- 172.21.15.148:0/2720331552 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222314230 0 0) 0x7f0264003520 con 0x7f02780cc4f0 2024-04-25T09:55:04.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f026ffff640 1 -- 172.21.15.148:0/2720331552 >> v1:172.21.15.188:6789/0 conn(0x7f027815b270 legacy=0x7f027815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f026ffff640 1 -- 172.21.15.148:0/2720331552 >> v1:172.21.15.148:6789/0 conn(0x7f02780cc4f0 legacy=0x7f02780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.507+0000 7f026ffff640 1 -- 172.21.15.148:0/2720331552 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780717a0 con 0x7f02780ce230 2024-04-25T09:55:04.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.508+0000 7f026ffff640 1 --2- 172.21.15.148:0/2720331552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02780ce230 0x7f02780cc8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02600097a0 tx=0x7f026002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3f6413248bb773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.508+0000 7f02767fc640 1 -- 172.21.15.148:0/2720331552 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260037070 con 0x7f02780ce230 2024-04-25T09:55:04.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.508+0000 7f02767fc640 1 -- 172.21.15.148:0/2720331552 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026002ac50 con 0x7f02780ce230 2024-04-25T09:55:04.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.508+0000 7f02767fc640 1 -- 172.21.15.148:0/2720331552 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268001670 con 0x7f02780ce230 2024-04-25T09:55:04.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 -- 172.21.15.148:0/2720331552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02780ce230 msgr2=0x7f02780cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 --2- 172.21.15.148:0/2720331552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02780ce230 0x7f02780cc8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02600097a0 tx=0x7f026002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 -- 172.21.15.148:0/2720331552 shutdown_connections 2024-04-25T09:55:04.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 --2- 172.21.15.148:0/2720331552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02780ce230 0x7f02780cc8d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 -- 172.21.15.148:0/2720331552 >> 172.21.15.148:0/2720331552 conn(0x7f027806abe0 msgr2=0x7f027806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 -- 172.21.15.148:0/2720331552 shutdown_connections 2024-04-25T09:55:04.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.509+0000 7f027d29e640 1 -- 172.21.15.148:0/2720331552 wait complete. 2024-04-25T09:55:04.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.510+0000 7f027d29e640 1 Processor -- start 2024-04-25T09:55:04.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.510+0000 7f027d29e640 1 -- start start 2024-04-25T09:55:04.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 0x7f027813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02780ce230 0x7f02781e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027815b270 0x7f02781e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027806e8a0 con 0x7f02780cc4f0 2024-04-25T09:55:04.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0278071d60 con 0x7f027815b270 2024-04-25T09:55:04.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780ae500 con 0x7f02780ce230 2024-04-25T09:55:04.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 0x7f027813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 0x7f027813ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51244/0 (socket says 172.21.15.148:51244) 2024-04-25T09:55:04.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 -- 172.21.15.148:0/3348912314 learned_addr learned my addr 172.21.15.148:0/3348912314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:04.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f02777fe640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027815b270 0x7f02781e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f026ffff640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02780ce230 0x7f02781e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 -- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02780ce230 msgr2=0x7f02781e71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02780ce230 0x7f02781e71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 -- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027815b270 msgr2=0x7f02781e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027815b270 0x7f02781e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 -- 172.21.15.148:0/3348912314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02781e6410 con 0x7f02780cc4f0 2024-04-25T09:55:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f0276ffd640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 0x7f027813ce20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f026800c5b0 tx=0x7f026800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268017070 con 0x7f02780cc4f0 2024-04-25T09:55:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268005ce0 con 0x7f02780cc4f0 2024-04-25T09:55:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02781e66f0 con 0x7f02780cc4f0 2024-04-25T09:55:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268007330 con 0x7f02780cc4f0 2024-04-25T09:55:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.511+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02781e6cb0 con 0x7f02780cc4f0 2024-04-25T09:55:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.512+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0278073a00 con 0x7f02780cc4f0 2024-04-25T09:55:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.513+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f026801a070 con 0x7f02780cc4f0 2024-04-25T09:55:04.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.513+0000 7f026f7fe640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f025806b4d0 0x7f025806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.513+0000 7f026ffff640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f025806b4d0 0x7f025806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.513+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f0268089ad0 con 0x7f02780cc4f0 2024-04-25T09:55:04.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.514+0000 7f026ffff640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f025806b4d0 0x7f025806d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0260034000 tx=0x7f026002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.517+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02680585d0 con 0x7f02780cc4f0 2024-04-25T09:55:04.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.688+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f027806d3a0 con 0x7f02780cc4f0 2024-04-25T09:55:04.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.688+0000 7f026f7fe640 1 -- 172.21.15.148:0/3348912314 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0268057ed0 con 0x7f02780cc4f0 2024-04-25T09:55:04.692 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:04.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f025806b4d0 msgr2=0x7f025806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f025806b4d0 0x7f025806d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0260034000 tx=0x7f026002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 msgr2=0x7f027813ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 0x7f027813ce20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f026800c5b0 tx=0x7f026800ca70 comp rx=0 tx=0).stop 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 shutdown_connections 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f025806b4d0 0x7f025806d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f027815b270 0x7f02781e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02780ce230 0x7f02781e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 --2- 172.21.15.148:0/3348912314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02780cc4f0 0x7f027813ce20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 >> 172.21.15.148:0/3348912314 conn(0x7f027806abe0 msgr2=0x7f027806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 shutdown_connections 2024-04-25T09:55:04.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.691+0000 7f027d29e640 1 -- 172.21.15.148:0/3348912314 wait complete. 2024-04-25T09:55:04.705 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.801+0000 7f4cd08f5640 1 Processor -- start 2024-04-25T09:55:04.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.801+0000 7f4cd08f5640 1 -- start start 2024-04-25T09:55:04.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cd08f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ccc144750 0x7f4ccc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cd08f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cd08f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc144b30 con 0x7f4ccc0cee60 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cd08f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc0712d0 con 0x7f4ccc144750 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cd08f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0715b0 con 0x7f4ccc06eb10 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cca59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ccc144750 0x7f4ccc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4ccad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4ccc0cee60 0x7f4ccc15f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43100/0 (socket says 172.21.15.148:43100) 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cca59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ccc144750 0x7f4ccc06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51250/0 (socket says 172.21.15.148:51250) 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4ccad9d640 1 -- 172.21.15.148:0/3377926280 learned_addr learned my addr 172.21.15.148:0/3377926280 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cc9d9b640 1 --2- 172.21.15.148:0/3377926280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cca59c640 1 -- 172.21.15.148:0/3377926280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 msgr2=0x7f4ccc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cca59c640 1 --2- 172.21.15.148:0/3377926280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cca59c640 1 -- 172.21.15.148:0/3377926280 >> v1:172.21.15.148:6789/0 conn(0x7f4ccc0cee60 legacy=0x7f4ccc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cca59c640 1 -- 172.21.15.148:0/3377926280 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc071890 con 0x7f4ccc144750 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.802+0000 7f4cc959a640 1 -- 172.21.15.148:0/3377926280 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48205438 0 0) 0x7f4ccc144b30 con 0x7f4ccc0cee60 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.803+0000 7f4cca59c640 1 --2- 172.21.15.148:0/3377926280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc06e5d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009d90 tx=0x7f4cbc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c668f2620e619256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.803+0000 7f4cc959a640 1 -- 172.21.15.148:0/3377926280 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc007740 con 0x7f4ccc144750 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.803+0000 7f4cc959a640 1 -- 172.21.15.148:0/3377926280 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc0078e0 con 0x7f4ccc144750 2024-04-25T09:55:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.803+0000 7f4cc959a640 1 -- 172.21.15.148:0/3377926280 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc007cb0 con 0x7f4ccc144750 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3377926280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 msgr2=0x7f4ccc06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3377926280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc06e5d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009d90 tx=0x7f4cbc02a660 comp rx=0 tx=0).stop 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3377926280 shutdown_connections 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3377926280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3377926280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc06e5d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3377926280 >> 172.21.15.148:0/3377926280 conn(0x7f4ccc06abe0 msgr2=0x7f4ccc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3377926280 shutdown_connections 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.804+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3377926280 wait complete. 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.805+0000 7f4cd08f5640 1 Processor -- start 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.805+0000 7f4cd08f5640 1 -- start start 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cd08f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc13b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cd08f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0cee60 0x7f4ccc0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cd08f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc1f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cd08f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc145450 con 0x7f4ccc144750 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cd08f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc06f020 con 0x7f4ccc0cee60 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cd08f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc1455d0 con 0x7f4ccc06eb10 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc1f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc1f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51254/0 (socket says 172.21.15.148:51254) 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 -- 172.21.15.148:0/3948817589 learned_addr learned my addr 172.21.15.148:0/3948817589 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cca59c640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc13b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cc9d9b640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0cee60 0x7f4ccc0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 -- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 msgr2=0x7f4ccc13b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc13b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 -- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0cee60 msgr2=0x7f4ccc0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0cee60 0x7f4ccc0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 -- 172.21.15.148:0/3948817589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc13a000 con 0x7f4ccc144750 2024-04-25T09:55:04.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4cc9d9b640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0cee60 0x7f4ccc0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.806+0000 7f4ccad9d640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc1f1910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c480 tx=0x7f4cc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.807+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4018070 con 0x7f4ccc144750 2024-04-25T09:55:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.807+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc4007500 con 0x7f4ccc144750 2024-04-25T09:55:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.807+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc13a2e0 con 0x7f4ccc144750 2024-04-25T09:55:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.807+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4004a20 con 0x7f4ccc144750 2024-04-25T09:55:04.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.807+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc13a8a0 con 0x7f4ccc144750 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.808+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc073990 con 0x7f4ccc144750 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.808+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cc401b070 con 0x7f4ccc144750 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.808+0000 7f4cbaffd640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cac06b4d0 0x7f4cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.808+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f4cc4089890 con 0x7f4ccc144750 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.808+0000 7f4cca59c640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cac06b4d0 0x7f4cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:04.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.809+0000 7f4cca59c640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cac06b4d0 0x7f4cac06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc034000 tx=0x7f4cbc033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:04.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.812+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cc4058450 con 0x7f4ccc144750 2024-04-25T09:55:04.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.982+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ccc1408e0 con 0x7f4ccc144750 2024-04-25T09:55:04.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.983+0000 7f4cbaffd640 1 -- 172.21.15.148:0/3948817589 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7f4cc4057d50 con 0x7f4ccc144750 2024-04-25T09:55:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:04.987 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cac06b4d0 msgr2=0x7f4cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cac06b4d0 0x7f4cac06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc034000 tx=0x7f4cbc033040 comp rx=0 tx=0).stop 2024-04-25T09:55:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 msgr2=0x7f4ccc1f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc1f1910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c480 tx=0x7f4cc400c940 comp rx=0 tx=0).stop 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 shutdown_connections 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cac06b4d0 0x7f4cac06d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ccc144750 0x7f4ccc1f1910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ccc0cee60 0x7f4ccc0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 --2- 172.21.15.148:0/3948817589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ccc06eb10 0x7f4ccc13b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.985+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 >> 172.21.15.148:0/3948817589 conn(0x7f4ccc06abe0 msgr2=0x7f4ccc0cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.986+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 shutdown_connections 2024-04-25T09:55:04.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:04.986+0000 7f4cd08f5640 1 -- 172.21.15.148:0/3948817589 wait complete. 2024-04-25T09:55:05.002 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:05.003 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:05.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.100+0000 7f3857cbd640 1 Processor -- start 2024-04-25T09:55:05.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.100+0000 7f3857cbd640 1 -- start start 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.101+0000 7f3857cbd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38501610f0 0x7f385006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.101+0000 7f3857cbd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 0x7f3850161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.101+0000 7f3857cbd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38500c6960 con 0x7f38501616f0 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.101+0000 7f3857cbd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38501614d0 con 0x7f38501610f0 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.101+0000 7f3857cbd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3850162260 con 0x7f38500ce7b0 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855a59640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38501610f0 0x7f385006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f385625a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 0x7f3850161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855258640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f38501616f0 0x7f38500ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:35886/0 (socket says 172.21.15.148:35886) 2024-04-25T09:55:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855258640 1 -- 172.21.15.148:0/4046919463 learned_addr learned my addr 172.21.15.148:0/4046919463 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855a59640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38501610f0 0x7f385006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:44410/0 (socket says 172.21.15.148:44410) 2024-04-25T09:55:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f385625a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 0x7f3850161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:34088/0 (socket says 172.21.15.148:34088) 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855a59640 1 -- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 msgr2=0x7f3850161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855a59640 1 --2- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 0x7f3850161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855a59640 1 -- 172.21.15.148:0/4046919463 >> v1:172.21.15.188:6789/0 conn(0x7f38501616f0 legacy=0x7f38500ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3854a57640 1 -- 172.21.15.148:0/4046919463 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810173118 0 0) 0x7f38500c6960 con 0x7f38501616f0 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f3855a59640 1 -- 172.21.15.148:0/4046919463 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850162540 con 0x7f38501610f0 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.102+0000 7f385625a640 1 --2- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 0x7f3850161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.103+0000 7f3855a59640 1 --2- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f385006c1c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f384c009d90 tx=0x7f384c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5957d78cc6b2d800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.103+0000 7f3854a57640 1 -- 172.21.15.148:0/4046919463 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c007740 con 0x7f38501610f0 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.103+0000 7f3854a57640 1 -- 172.21.15.148:0/4046919463 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c0078e0 con 0x7f38501610f0 2024-04-25T09:55:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.103+0000 7f3854a57640 1 -- 172.21.15.148:0/4046919463 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c007cb0 con 0x7f38501610f0 2024-04-25T09:55:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 -- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 msgr2=0x7f385006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 --2- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f385006c1c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f384c009d90 tx=0x7f384c02a660 comp rx=0 tx=0).stop 2024-04-25T09:55:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 -- 172.21.15.148:0/4046919463 shutdown_connections 2024-04-25T09:55:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 --2- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38500ce7b0 0x7f3850161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 --2- 172.21.15.148:0/4046919463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f385006c1c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 -- 172.21.15.148:0/4046919463 >> 172.21.15.148:0/4046919463 conn(0x7f385006f500 msgr2=0x7f385006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 -- 172.21.15.148:0/4046919463 shutdown_connections 2024-04-25T09:55:05.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.104+0000 7f3857cbd640 1 -- 172.21.15.148:0/4046919463 wait complete. 2024-04-25T09:55:05.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.105+0000 7f3857cbd640 1 Processor -- start 2024-04-25T09:55:05.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.105+0000 7f3857cbd640 1 -- start start 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.105+0000 7f3857cbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 0x7f385014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f38500b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38501616f0 0x7f38500c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500cf690 con 0x7f38500ce7b0 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500cfce0 con 0x7f38501610f0 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385006c700 con 0x7f38501616f0 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 0x7f385014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 0x7f385014b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51264/0 (socket says 172.21.15.148:51264) 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 -- 172.21.15.148:0/1010712928 learned_addr learned my addr 172.21.15.148:0/1010712928 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f385625a640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38501616f0 0x7f38500c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855258640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f38500b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 -- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38501616f0 msgr2=0x7f38500c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38501616f0 0x7f38500c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 -- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 msgr2=0x7f38500b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f38500b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 -- 172.21.15.148:0/1010712928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385015fb10 con 0x7f38500ce7b0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3855a59640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 0x7f385014b270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f384c009d90 tx=0x7f384c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c040070 con 0x7f38500ce7b0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c031c40 con 0x7f38500ce7b0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38500b0d50 con 0x7f38500ce7b0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c009450 con 0x7f38500ce7b0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.106+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385015fdd0 con 0x7f38500ce7b0 2024-04-25T09:55:05.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.107+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3850073a00 con 0x7f38500ce7b0 2024-04-25T09:55:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.108+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f384c039040 con 0x7f38500ce7b0 2024-04-25T09:55:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.108+0000 7f38467fc640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f383406b500 0x7f383406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.108+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f384c0ab900 con 0x7f38500ce7b0 2024-04-25T09:55:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.108+0000 7f3855258640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f383406b500 0x7f383406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.109+0000 7f3855258640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f383406b500 0x7f383406d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3850155b70 tx=0x7f3840007b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.112+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f384c07a260 con 0x7f38500ce7b0 2024-04-25T09:55:05.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.287+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f385006f8e0 con 0x7f38500ce7b0 2024-04-25T09:55:05.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.287+0000 7f38467fc640 1 -- 172.21.15.148:0/1010712928 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f384c079b60 con 0x7f38500ce7b0 2024-04-25T09:55:05.291 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f383406b500 msgr2=0x7f383406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f383406b500 0x7f383406d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3850155b70 tx=0x7f3840007b70 comp rx=0 tx=0).stop 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 msgr2=0x7f385014b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 0x7f385014b270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f384c009d90 tx=0x7f384c004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 shutdown_connections 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f383406b500 0x7f383406d9c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38501616f0 0x7f38500c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38501610f0 0x7f38500b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 --2- 172.21.15.148:0/1010712928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38500ce7b0 0x7f385014b270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 >> 172.21.15.148:0/1010712928 conn(0x7f385006f500 msgr2=0x7f38500b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 shutdown_connections 2024-04-25T09:55:05.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.289+0000 7f3857cbd640 1 -- 172.21.15.148:0/1010712928 wait complete. 2024-04-25T09:55:05.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:05.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.401+0000 7fee3b012640 1 Processor -- start 2024-04-25T09:55:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.401+0000 7fee3b012640 1 -- start start 2024-04-25T09:55:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee3b012640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fee3406ccd0 0x7fee340b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee3b012640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 0x7fee3415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee3b012640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee340d1770 con 0x7fee34070dc0 2024-04-25T09:55:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee3b012640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee341622b0 con 0x7fee3406ccd0 2024-04-25T09:55:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee3b012640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b10f0 con 0x7fee3415deb0 2024-04-25T09:55:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee38dae640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee34070dc0 0x7fee340711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43114/0 (socket says 172.21.15.148:43114) 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee38dae640 1 -- 172.21.15.148:0/1993433903 learned_addr learned my addr 172.21.15.148:0/1993433903 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee395af640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 0x7fee3415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee33fff640 1 --2- 172.21.15.148:0/1993433903 >> v2:172.21.15.188:3300/0 conn(0x7fee3406ccd0 0x7fee340b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610518663 0 0) 0x7fee340d1770 con 0x7fee34070dc0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee24003520 con 0x7fee34070dc0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3385325505 0 0) 0x7fee24003520 con 0x7fee34070dc0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 msgr2=0x7fee3415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 0x7fee3415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 >> v2:172.21.15.188:3300/0 conn(0x7fee3406ccd0 msgr2=0x7fee340b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 --2- 172.21.15.148:0/1993433903 >> v2:172.21.15.188:3300/0 conn(0x7fee3406ccd0 0x7fee340b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34162740 con 0x7fee34070dc0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fee28001670 con 0x7fee34070dc0 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 >> v1:172.21.15.148:6789/0 conn(0x7fee34070dc0 legacy=0x7fee340711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee240039b0 0x7fee24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 0x7fee24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee24004d40 0x7fee240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee341622b0 con 0x7fee240039b0 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b10f0 con 0x7fee24004d40 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2400ba40 con 0x7fee3415deb0 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.402+0000 7fee38dae640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee240039b0 0x7fee24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee33fff640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 0x7fee24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee395af640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee24004d40 0x7fee240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee38dae640 1 -- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 msgr2=0x7fee24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee38dae640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee3415deb0 0x7fee24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee38dae640 1 -- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee24004d40 msgr2=0x7fee240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee38dae640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee24004d40 0x7fee240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee38dae640 1 -- 172.21.15.148:0/1993433903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24003520 con 0x7fee240039b0 2024-04-25T09:55:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee395af640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee24004d40 0x7fee240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee38dae640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee240039b0 0x7fee24003dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee28007fd0 tx=0x7fee24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3788d54e12b54ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.403+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28007d50 con 0x7fee240039b0 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee28005ad0 con 0x7fee240039b0 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee337fe640 1 -- 172.21.15.148:0/1993433903 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28007070 con 0x7fee240039b0 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee3b012640 1 -- 172.21.15.148:0/1993433903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee240039b0 msgr2=0x7fee24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee3b012640 1 --2- 172.21.15.148:0/1993433903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee240039b0 0x7fee24003dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee28007fd0 tx=0x7fee24005830 comp rx=0 tx=0).stop 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee395af640 1 -- 172.21.15.148:0/1993433903 reap_dead start 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee3b012640 1 -- 172.21.15.148:0/1993433903 shutdown_connections 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee3b012640 1 -- 172.21.15.148:0/1993433903 >> 172.21.15.148:0/1993433903 conn(0x7fee340b3530 msgr2=0x7fee340b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee3b012640 1 -- 172.21.15.148:0/1993433903 shutdown_connections 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.404+0000 7fee3b012640 1 -- 172.21.15.148:0/1993433903 wait complete. 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.405+0000 7fee3b012640 1 Processor -- start 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.405+0000 7fee3b012640 1 -- start start 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee3b012640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee240039b0 0x7fee34147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee3b012640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee340cd3a0 0x7fee340b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee3b012640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 0x7fee340cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee3b012640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34161f40 con 0x7fee340b19c0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee3b012640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34161c60 con 0x7fee240039b0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee3b012640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340b5840 con 0x7fee340cd3a0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee395af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 0x7fee340cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee395af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 0x7fee340cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51282/0 (socket says 172.21.15.148:51282) 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee395af640 1 -- 172.21.15.148:0/36905969 learned_addr learned my addr 172.21.15.148:0/36905969 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee38dae640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee240039b0 0x7fee34147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.406+0000 7fee33fff640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee340cd3a0 0x7fee340b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee395af640 1 -- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee340cd3a0 msgr2=0x7fee340b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee395af640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee340cd3a0 0x7fee340b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee395af640 1 -- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee240039b0 msgr2=0x7fee34147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee395af640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee240039b0 0x7fee34147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee395af640 1 -- 172.21.15.148:0/36905969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3415e510 con 0x7fee340b19c0 2024-04-25T09:55:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee395af640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 0x7fee340cbc00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee3415e290 tx=0x7fee2c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c01a030 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c010d80 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3415e7f0 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c003bd0 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.407+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3415edb0 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.408+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee340b2c70 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.408+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee2c01d070 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.409+0000 7fee317fa640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b500 0x7fee1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.409+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fee2c08aa50 con 0x7fee340b19c0 2024-04-25T09:55:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.409+0000 7fee38dae640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b500 0x7fee1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.409+0000 7fee38dae640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b500 0x7fee1806d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee28033000 tx=0x7fee28009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.413+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee340b2c70 con 0x7fee340b19c0 2024-04-25T09:55:05.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.586+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee34073a00 con 0x7fee340b19c0 2024-04-25T09:55:05.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.587+0000 7fee317fa640 1 -- 172.21.15.148:0/36905969 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee2c059550 con 0x7fee340b19c0 2024-04-25T09:55:05.590 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b500 msgr2=0x7fee1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b500 0x7fee1806d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee28033000 tx=0x7fee28009030 comp rx=0 tx=0).stop 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 msgr2=0x7fee340cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 0x7fee340cbc00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee3415e290 tx=0x7fee2c010590 comp rx=0 tx=0).stop 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 shutdown_connections 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b500 0x7fee1806d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee340b19c0 0x7fee340cbc00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee340cd3a0 0x7fee340b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 --2- 172.21.15.148:0/36905969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee240039b0 0x7fee34147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 >> 172.21.15.148:0/36905969 conn(0x7fee340b3530 msgr2=0x7fee340b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 shutdown_connections 2024-04-25T09:55:05.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.589+0000 7fee3b012640 1 -- 172.21.15.148:0/36905969 wait complete. 2024-04-25T09:55:05.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:05.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.699+0000 7fd6621a9640 1 Processor -- start 2024-04-25T09:55:05.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.699+0000 7fd6621a9640 1 -- start start 2024-04-25T09:55:05.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd6621a9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd65c0cfa40 0x7fd65c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd6621a9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0c75e0 con 0x7fd65c0cf600 2024-04-25T09:55:05.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd6621a9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0c78c0 con 0x7fd65c15bb50 2024-04-25T09:55:05.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd6621a9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c0c8680 con 0x7fd65c0cfa40 2024-04-25T09:55:05.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd65bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd65c0cfa40 0x7fd65c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd652ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd65c0cf600 0x7fd65c0cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43130/0 (socket says 172.21.15.148:43130) 2024-04-25T09:55:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.700+0000 7fd652ffd640 1 -- 172.21.15.148:0/2713520120 learned_addr learned my addr 172.21.15.148:0/2713520120 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.701+0000 7fd65bfff640 1 -- 172.21.15.148:0/2713520120 >> v1:172.21.15.188:6789/0 conn(0x7fd65c15bb50 legacy=0x7fd65c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.701+0000 7fd65bfff640 1 -- 172.21.15.148:0/2713520120 >> v1:172.21.15.148:6789/0 conn(0x7fd65c0cf600 legacy=0x7fd65c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.701+0000 7fd65affd640 1 -- 172.21.15.148:0/2713520120 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554675534 0 0) 0x7fd65c0c75e0 con 0x7fd65c0cf600 2024-04-25T09:55:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.701+0000 7fd65bfff640 1 -- 172.21.15.148:0/2713520120 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0c7ba0 con 0x7fd65c0cfa40 2024-04-25T09:55:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.701+0000 7fd65bfff640 1 --2- 172.21.15.148:0/2713520120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c0cc630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd654009d90 tx=0x7fd65402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0f4c2e1c6ba394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd65affd640 1 -- 172.21.15.148:0/2713520120 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654037070 con 0x7fd65c0cfa40 2024-04-25T09:55:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd65affd640 1 -- 172.21.15.148:0/2713520120 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654007740 con 0x7fd65c0cfa40 2024-04-25T09:55:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd65affd640 1 -- 172.21.15.148:0/2713520120 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654007b10 con 0x7fd65c0cfa40 2024-04-25T09:55:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 -- 172.21.15.148:0/2713520120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 msgr2=0x7fd65c0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 --2- 172.21.15.148:0/2713520120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c0cc630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd654009d90 tx=0x7fd65402a780 comp rx=0 tx=0).stop 2024-04-25T09:55:05.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 -- 172.21.15.148:0/2713520120 shutdown_connections 2024-04-25T09:55:05.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 --2- 172.21.15.148:0/2713520120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c0cc630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 -- 172.21.15.148:0/2713520120 >> 172.21.15.148:0/2713520120 conn(0x7fd65c06f500 msgr2=0x7fd65c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 -- 172.21.15.148:0/2713520120 shutdown_connections 2024-04-25T09:55:05.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.702+0000 7fd6621a9640 1 -- 172.21.15.148:0/2713520120 wait complete. 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.703+0000 7fd6621a9640 1 Processor -- start 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 -- start start 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd65c0cf600 0x7fd65c14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65c15bb50 0x7fd65c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0c72a0 con 0x7fd65c0cfa40 2024-04-25T09:55:05.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0c8160 con 0x7fd65c0cf600 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd6621a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0cfe90 con 0x7fd65c15bb50 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd652ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd652ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c06d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51294/0 (socket says 172.21.15.148:51294) 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.704+0000 7fd652ffd640 1 -- 172.21.15.148:0/571376621 learned_addr learned my addr 172.21.15.148:0/571376621 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd65bfff640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65c15bb50 0x7fd65c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd65b7fe640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd65c0cf600 0x7fd65c14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd652ffd640 1 -- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65c15bb50 msgr2=0x7fd65c0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd652ffd640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65c15bb50 0x7fd65c0cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd652ffd640 1 -- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd65c0cf600 msgr2=0x7fd65c14f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd652ffd640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd65c0cf600 0x7fd65c14f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd652ffd640 1 -- 172.21.15.148:0/571376621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c06da90 con 0x7fd65c0cfa40 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd652ffd640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c06d240 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c4a0 tx=0x7fd64400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644018070 con 0x7fd65c0cfa40 2024-04-25T09:55:05.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6440074e0 con 0x7fd65c0cfa40 2024-04-25T09:55:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c0cbcd0 con 0x7fd65c0cfa40 2024-04-25T09:55:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644004a20 con 0x7fd65c0cfa40 2024-04-25T09:55:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.705+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c0cc290 con 0x7fd65c0cfa40 2024-04-25T09:55:05.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.706+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c073a00 con 0x7fd65c0cfa40 2024-04-25T09:55:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.706+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd64401b070 con 0x7fd65c0cfa40 2024-04-25T09:55:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.707+0000 7fd658ff9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd63c06b4d0 0x7fd63c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.707+0000 7fd65b7fe640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd63c06b4d0 0x7fd63c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.707+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd644089980 con 0x7fd65c0cfa40 2024-04-25T09:55:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.707+0000 7fd65b7fe640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd63c06b4d0 0x7fd63c06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd65c155340 tx=0x7fd64c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:05.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.711+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6440583f0 con 0x7fd65c0cfa40 2024-04-25T09:55:05.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.881+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd65c0cf150 con 0x7fd65c0cfa40 2024-04-25T09:55:05.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.881+0000 7fd658ff9640 1 -- 172.21.15.148:0/571376621 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd644057cf0 con 0x7fd65c0cfa40 2024-04-25T09:55:05.885 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:05.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd63c06b4d0 msgr2=0x7fd63c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd63c06b4d0 0x7fd63c06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd65c155340 tx=0x7fd64c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:05.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 msgr2=0x7fd65c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:05.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c06d240 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd64400c4a0 tx=0x7fd64400c960 comp rx=0 tx=0).stop 2024-04-25T09:55:05.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 shutdown_connections 2024-04-25T09:55:05.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd63c06b4d0 0x7fd63c06d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd65c15bb50 0x7fd65c0cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd65c0cfa40 0x7fd65c06d240 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 --2- 172.21.15.148:0/571376621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd65c0cf600 0x7fd65c14f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:05.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.883+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 >> 172.21.15.148:0/571376621 conn(0x7fd65c06f500 msgr2=0x7fd65c0b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:05.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.884+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 shutdown_connections 2024-04-25T09:55:05.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.884+0000 7fd6621a9640 1 -- 172.21.15.148:0/571376621 wait complete. 2024-04-25T09:55:05.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:05.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.994+0000 7f6afbc83640 1 Processor -- start 2024-04-25T09:55:05.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.994+0000 7f6afbc83640 1 -- start start 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6afbc83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6af40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6afbc83640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af415e020 con 0x7f6af406bd20 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6afbc83640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af406ac80 con 0x7f6af406e360 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6afbc83640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af406b890 con 0x7f6af40b38d0 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6afa220640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6af406bd20 0x7f6af40acea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43132/0 (socket says 172.21.15.148:43132) 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6afa220640 1 -- 172.21.15.148:0/3769102590 learned_addr learned my addr 172.21.15.148:0/3769102590 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:06.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6af921e640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6af40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902201563 0 0) 0x7f6af415e020 con 0x7f6af406bd20 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae0003520 con 0x7f6af406bd20 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241976361 0 0) 0x7f6ae0003520 con 0x7f6af406bd20 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 msgr2=0x7f6af40b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.995+0000 7f6af8a1d640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6af40b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 >> v1:172.21.15.188:6789/0 conn(0x7f6af406e360 legacy=0x7f6af406e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af406af60 con 0x7f6af406bd20 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6af0001670 con 0x7f6af406bd20 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 >> v1:172.21.15.148:6789/0 conn(0x7f6af406bd20 legacy=0x7f6af40acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6ae00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae0003f60 0x7f6ae0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0004d40 0x7f6ae00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af406b890 con 0x7f6ae0003f60 2024-04-25T09:55:06.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae000bb30 con 0x7f6ae0004d40 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae000bd30 con 0x7f6af40b38d0 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af921e640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae0003f60 0x7f6ae0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af9a1f640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6ae00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6afa220640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0004d40 0x7f6ae00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af921e640 1 -- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 msgr2=0x7f6ae00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af921e640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6ae00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af921e640 1 -- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0004d40 msgr2=0x7f6ae00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af921e640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0004d40 0x7f6ae00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af921e640 1 -- 172.21.15.148:0/3769102590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae0003520 con 0x7f6ae0003f60 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6afa220640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0004d40 0x7f6ae00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.996+0000 7f6af9a1f640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af40b38d0 0x7f6ae00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6af921e640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae0003f60 0x7f6ae0004800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ac0 tx=0x7f6ae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eb9b7d3cae9f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4036070 con 0x7f6ae0003f60 2024-04-25T09:55:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae4031400 con 0x7f6ae0003f60 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6af8a1d640 1 -- 172.21.15.148:0/3769102590 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4009690 con 0x7f6ae0003f60 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afbc83640 1 -- 172.21.15.148:0/3769102590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae0003f60 msgr2=0x7f6ae0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3769102590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae0003f60 0x7f6ae0004800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009ac0 tx=0x7f6ae0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afa220640 1 -- 172.21.15.148:0/3769102590 reap_dead start 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afbc83640 1 -- 172.21.15.148:0/3769102590 shutdown_connections 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afbc83640 1 -- 172.21.15.148:0/3769102590 >> 172.21.15.148:0/3769102590 conn(0x7f6af40cfec0 msgr2=0x7f6af40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afbc83640 1 -- 172.21.15.148:0/3769102590 shutdown_connections 2024-04-25T09:55:06.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.997+0000 7f6afbc83640 1 -- 172.21.15.148:0/3769102590 wait complete. 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.998+0000 7f6afbc83640 1 Processor -- start 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 -- start start 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0003f60 0x7f6af40c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af414c530 0x7f6af40b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 0x7f6af4153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af406ee50 con 0x7f6af41531b0 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af406b520 con 0x7f6ae0003f60 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afbc83640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40b2fd0 con 0x7f6af414c530 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afa220640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 0x7f6af4153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afa220640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 0x7f6af4153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51314/0 (socket says 172.21.15.148:51314) 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:05.999+0000 7f6afa220640 1 -- 172.21.15.148:0/3513859331 learned_addr learned my addr 172.21.15.148:0/3513859331 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6af921e640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af414c530 0x7f6af40b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afa220640 1 -- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af414c530 msgr2=0x7f6af40b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afa220640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af414c530 0x7f6af40b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afa220640 1 -- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0003f60 msgr2=0x7f6af40c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6af9a1f640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0003f60 0x7f6af40c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afa220640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0003f60 0x7f6af40c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afa220640 1 -- 172.21.15.148:0/3513859331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af414d040 con 0x7f6af41531b0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afa220640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 0x7f6af4153630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6af409df90 tx=0x7f6af000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000c5b0 con 0x7f6af41531b0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0018070 con 0x7f6af41531b0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40afa30 con 0x7f6af41531b0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000c9b0 con 0x7f6af41531b0 2024-04-25T09:55:06.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.000+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af40afff0 con 0x7f6af41531b0 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.001+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af4153b70 con 0x7f6af41531b0 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.001+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6af000cb50 con 0x7f6af41531b0 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.002+0000 7f6aea7fc640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ad806b500 0x7f6ad806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.002+0000 7f6af9a1f640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ad806b500 0x7f6ad806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.002+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f6af008a100 con 0x7f6af41531b0 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.002+0000 7f6af9a1f640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ad806b500 0x7f6ad806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007fd0 tx=0x7f6aec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.006+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6af0058520 con 0x7f6af41531b0 2024-04-25T09:55:06.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.179+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6af4073a00 con 0x7f6af41531b0 2024-04-25T09:55:06.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.179+0000 7f6aea7fc640 1 -- 172.21.15.148:0/3513859331 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6af005c5c0 con 0x7f6af41531b0 2024-04-25T09:55:06.183 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:06.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ad806b500 msgr2=0x7f6ad806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ad806b500 0x7f6ad806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6aec007fd0 tx=0x7f6aec009040 comp rx=0 tx=0).stop 2024-04-25T09:55:06.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 msgr2=0x7f6af4153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 0x7f6af4153630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6af409df90 tx=0x7f6af000eb90 comp rx=0 tx=0).stop 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 shutdown_connections 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ad806b500 0x7f6ad806d9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6af41531b0 0x7f6af4153630 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6af414c530 0x7f6af40b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 --2- 172.21.15.148:0/3513859331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ae0003f60 0x7f6af40c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 >> 172.21.15.148:0/3513859331 conn(0x7f6af40cfec0 msgr2=0x7f6af406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 shutdown_connections 2024-04-25T09:55:06.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.181+0000 7f6afbc83640 1 -- 172.21.15.148:0/3513859331 wait complete. 2024-04-25T09:55:06.196 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:06.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.290+0000 7fd83e053640 1 Processor -- start 2024-04-25T09:55:06.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.290+0000 7fd83e053640 1 -- start start 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd83e053640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8380cedf0 0x7fd8381616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd83e053640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8380b93a0 0x7fd8380d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd83e053640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83815e000 con 0x7fd83815dc20 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd83e053640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380b6bb0 con 0x7fd8380cedf0 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd83e053640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380b6030 con 0x7fd8380b93a0 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd836ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8380cedf0 0x7fd8381616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd8377fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd83815dc20 0x7fd8380ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43140/0 (socket says 172.21.15.148:43140) 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd836ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd8380cedf0 0x7fd8381616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51318/0 (socket says 172.21.15.148:51318) 2024-04-25T09:55:06.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd8377fe640 1 -- 172.21.15.148:0/880654040 learned_addr learned my addr 172.21.15.148:0/880654040 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.291+0000 7fd837fff640 1 --2- 172.21.15.148:0/880654040 >> v2:172.21.15.188:3300/0 conn(0x7fd8380b93a0 0x7fd8380d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd836ffd640 1 -- 172.21.15.148:0/880654040 >> v2:172.21.15.188:3300/0 conn(0x7fd8380b93a0 msgr2=0x7fd8380d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd836ffd640 1 --2- 172.21.15.148:0/880654040 >> v2:172.21.15.188:3300/0 conn(0x7fd8380b93a0 0x7fd8380d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd836ffd640 1 -- 172.21.15.148:0/880654040 >> v1:172.21.15.148:6789/0 conn(0x7fd83815dc20 legacy=0x7fd8380ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd836ffd640 1 -- 172.21.15.148:0/880654040 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd838161de0 con 0x7fd8380cedf0 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd8367fc640 1 -- 172.21.15.148:0/880654040 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659469375 0 0) 0x7fd828001490 con 0x7fd83815dc20 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd837fff640 1 --2- 172.21.15.148:0/880654040 >> v2:172.21.15.188:3300/0 conn(0x7fd8380b93a0 0x7fd8380d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd836ffd640 1 --2- 172.21.15.148:0/880654040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd8381616b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd8380cdff0 tx=0x7fd82c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba06410c791aff1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd8367fc640 1 -- 172.21.15.148:0/880654040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037070 con 0x7fd8380cedf0 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd8367fc640 1 -- 172.21.15.148:0/880654040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c032440 con 0x7fd8380cedf0 2024-04-25T09:55:06.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.292+0000 7fd8367fc640 1 -- 172.21.15.148:0/880654040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c032810 con 0x7fd8380cedf0 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 -- 172.21.15.148:0/880654040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 msgr2=0x7fd8381616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 --2- 172.21.15.148:0/880654040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd8381616b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd8380cdff0 tx=0x7fd82c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 -- 172.21.15.148:0/880654040 shutdown_connections 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 --2- 172.21.15.148:0/880654040 >> v2:172.21.15.188:3300/0 conn(0x7fd8380b93a0 0x7fd8380d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 --2- 172.21.15.148:0/880654040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd8381616b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 -- 172.21.15.148:0/880654040 >> 172.21.15.148:0/880654040 conn(0x7fd8380d0490 msgr2=0x7fd8380d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 -- 172.21.15.148:0/880654040 shutdown_connections 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.293+0000 7fd83e053640 1 -- 172.21.15.148:0/880654040 wait complete. 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.294+0000 7fd83e053640 1 Processor -- start 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.294+0000 7fd83e053640 1 -- start start 2024-04-25T09:55:06.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd83e053640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380b93a0 0x7fd8380cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd83e053640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd838156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd83e053640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83815dc20 0x7fd8380cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd83e053640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380d2a50 con 0x7fd8380cedf0 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd83e053640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8381623a0 con 0x7fd83815dc20 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd83e053640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380d2bd0 con 0x7fd8380b93a0 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd838156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd838156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51322/0 (socket says 172.21.15.148:51322) 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 -- 172.21.15.148:0/3991165084 learned_addr learned my addr 172.21.15.148:0/3991165084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd8377fe640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380b93a0 0x7fd8380cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 -- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380b93a0 msgr2=0x7fd8380cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380b93a0 0x7fd8380cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 -- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83815dc20 msgr2=0x7fd8380cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83815dc20 0x7fd8380cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.295+0000 7fd836ffd640 1 -- 172.21.15.148:0/3991165084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380cd960 con 0x7fd8380cedf0 2024-04-25T09:55:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.296+0000 7fd836ffd640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd838156ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009890 tx=0x7fd82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.296+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037040 con 0x7fd8380cedf0 2024-04-25T09:55:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.296+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c042070 con 0x7fd8380cedf0 2024-04-25T09:55:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.296+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8380cdc40 con 0x7fd8380cedf0 2024-04-25T09:55:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.296+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c030a50 con 0x7fd8380cedf0 2024-04-25T09:55:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.296+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd838157420 con 0x7fd8380cedf0 2024-04-25T09:55:06.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.297+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838073990 con 0x7fd8380cedf0 2024-04-25T09:55:06.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.297+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd82c030bf0 con 0x7fd8380cedf0 2024-04-25T09:55:06.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.298+0000 7fd81ffff640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd82406b4d0 0x7fd82406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.298+0000 7fd8377fe640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd82406b4d0 0x7fd82406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.298+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd82c0ab090 con 0x7fd8380cedf0 2024-04-25T09:55:06.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.298+0000 7fd8377fe640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd82406b4d0 0x7fd82406d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd8280012e0 tx=0x7fd828009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.302+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd82c045360 con 0x7fd8380cedf0 2024-04-25T09:55:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.475+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8380cde10 con 0x7fd8380cedf0 2024-04-25T09:55:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.476+0000 7fd81ffff640 1 -- 172.21.15.148:0/3991165084 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7fd82c079b90 con 0x7fd8380cedf0 2024-04-25T09:55:06.480 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:06.480 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd82406b4d0 msgr2=0x7fd82406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd82406b4d0 0x7fd82406d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd8280012e0 tx=0x7fd828009040 comp rx=0 tx=0).stop 2024-04-25T09:55:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 msgr2=0x7fd838156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd838156ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009890 tx=0x7fd82c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 shutdown_connections 2024-04-25T09:55:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd82406b4d0 0x7fd82406d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83815dc20 0x7fd8380cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8380cedf0 0x7fd838156ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 --2- 172.21.15.148:0/3991165084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380b93a0 0x7fd8380cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 >> 172.21.15.148:0/3991165084 conn(0x7fd8380d0490 msgr2=0x7fd8380b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 shutdown_connections 2024-04-25T09:55:06.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.478+0000 7fd83e053640 1 -- 172.21.15.148:0/3991165084 wait complete. 2024-04-25T09:55:06.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:06.494 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:06.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.590+0000 7f1cf110a640 1 Processor -- start 2024-04-25T09:55:06.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.590+0000 7f1cf110a640 1 -- start start 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1cf110a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1cec1610f0 0x7f1cec06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1cf110a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cec0ce7b0 0x7f1cec161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1cf110a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cec0c6960 con 0x7f1cec1616f0 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1cf110a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cec1614d0 con 0x7f1cec0ce7b0 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1cf110a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cec162260 con 0x7f1cec1610f0 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1ceb59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cec0ce7b0 0x7f1cec161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1ceb59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cec0ce7b0 0x7f1cec161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51334/0 (socket says 172.21.15.148:51334) 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1ceb59e640 1 -- 172.21.15.148:0/3219422098 learned_addr learned my addr 172.21.15.148:0/3219422098 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.591+0000 7f1cead9d640 1 --2- 172.21.15.148:0/3219422098 >> v2:172.21.15.188:3300/0 conn(0x7f1cec1610f0 0x7f1cec06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ceb59e640 1 -- 172.21.15.148:0/3219422098 >> v2:172.21.15.188:3300/0 conn(0x7f1cec1610f0 msgr2=0x7f1cec06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ceb59e640 1 --2- 172.21.15.148:0/3219422098 >> v2:172.21.15.188:3300/0 conn(0x7f1cec1610f0 0x7f1cec06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ceb59e640 1 -- 172.21.15.148:0/3219422098 >> v1:172.21.15.188:6789/0 conn(0x7f1cec1616f0 legacy=0x7f1cec0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ceb59e640 1 -- 172.21.15.148:0/3219422098 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec162540 con 0x7f1cec0ce7b0 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ceb59e640 1 --2- 172.21.15.148:0/3219422098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec161b50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1cec071fb0 tx=0x7f1ce402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202786528529afa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ce9d9b640 1 -- 172.21.15.148:0/3219422098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4037070 con 0x7f1cec0ce7b0 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ce9d9b640 1 -- 172.21.15.148:0/3219422098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4032440 con 0x7f1cec0ce7b0 2024-04-25T09:55:06.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.592+0000 7f1ce9d9b640 1 -- 172.21.15.148:0/3219422098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4032810 con 0x7f1cec0ce7b0 2024-04-25T09:55:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 -- 172.21.15.148:0/3219422098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec0ce7b0 msgr2=0x7f1cec161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 --2- 172.21.15.148:0/3219422098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec161b50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1cec071fb0 tx=0x7f1ce402a270 comp rx=0 tx=0).stop 2024-04-25T09:55:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 -- 172.21.15.148:0/3219422098 shutdown_connections 2024-04-25T09:55:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 --2- 172.21.15.148:0/3219422098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec161b50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 --2- 172.21.15.148:0/3219422098 >> v2:172.21.15.188:3300/0 conn(0x7f1cec1610f0 0x7f1cec06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 -- 172.21.15.148:0/3219422098 >> 172.21.15.148:0/3219422098 conn(0x7f1cec06f500 msgr2=0x7f1cec06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 -- 172.21.15.148:0/3219422098 shutdown_connections 2024-04-25T09:55:06.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.593+0000 7f1cf110a640 1 -- 172.21.15.148:0/3219422098 wait complete. 2024-04-25T09:55:06.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.594+0000 7f1cf110a640 1 Processor -- start 2024-04-25T09:55:06.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 -- start start 2024-04-25T09:55:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cec1610f0 0x7f1cec1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 0x7f1cec0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0cf690 con 0x7f1cec1616f0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec0cfce0 con 0x7f1cec0ce7b0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cf110a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec06c700 con 0x7f1cec1610f0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1ceb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 0x7f1cec0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1ceb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 0x7f1cec0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51338/0 (socket says 172.21.15.148:51338) 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1ceb59e640 1 -- 172.21.15.148:0/1184154019 learned_addr learned my addr 172.21.15.148:0/1184154019 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cead9d640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.595+0000 7f1cea59c640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cec1610f0 0x7f1cec1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1ceb59e640 1 -- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cec1610f0 msgr2=0x7f1cec1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1ceb59e640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cec1610f0 0x7f1cec1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1ceb59e640 1 -- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cec0ce7b0 msgr2=0x7f1cec0b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1ceb59e640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec0b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1ceb59e640 1 -- 172.21.15.148:0/1184154019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0b5710 con 0x7f1cec1616f0 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1cea59c640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cec1610f0 0x7f1cec1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:06.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1ceb59e640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 0x7f1cec0aea30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40090b0 tx=0x7f1ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4041020 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce403d640 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cec0b1150 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4037040 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.596+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec0ae010 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.597+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cec073a00 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.597+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ce4044020 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.598+0000 7f1cd37fe640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cd806b4d0 0x7f1cd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.598+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f1ce40aac30 con 0x7f1cec1616f0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.598+0000 7f1cead9d640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cd806b4d0 0x7f1cd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.598+0000 7f1cead9d640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cd806b4d0 0x7f1cd806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0113e0 tx=0x7f1cdc00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.602+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ce404b030 con 0x7f1cec1616f0 2024-04-25T09:55:06.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.775+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cec0ae250 con 0x7f1cec1616f0 2024-04-25T09:55:06.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.775+0000 7f1cd37fe640 1 -- 172.21.15.148:0/1184154019 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ce40796b0 con 0x7f1cec1616f0 2024-04-25T09:55:06.779 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cd806b4d0 msgr2=0x7f1cd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cd806b4d0 0x7f1cd806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0113e0 tx=0x7f1cdc00b040 comp rx=0 tx=0).stop 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 msgr2=0x7f1cec0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 0x7f1cec0aea30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40090b0 tx=0x7f1ce40046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 shutdown_connections 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cd806b4d0 0x7f1cd806d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec1616f0 0x7f1cec0aea30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cec1610f0 0x7f1cec1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 --2- 172.21.15.148:0/1184154019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cec0ce7b0 0x7f1cec0b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 >> 172.21.15.148:0/1184154019 conn(0x7f1cec06f500 msgr2=0x7f1cec0712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 shutdown_connections 2024-04-25T09:55:06.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.777+0000 7f1cf110a640 1 -- 172.21.15.148:0/1184154019 wait complete. 2024-04-25T09:55:06.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:06.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.890+0000 7f607761f640 1 Processor -- start 2024-04-25T09:55:06.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.890+0000 7f607761f640 1 -- start start 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f607761f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f60700ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f607761f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60700cf320 con 0x7f607015b980 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f607761f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60700cf040 con 0x7f607015b5a0 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f607761f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6070161970 con 0x7f60700b5c40 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f6075bbc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f607015b980 0x7f60700ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43150/0 (socket says 172.21.15.148:43150) 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f6075bbc640 1 -- 172.21.15.148:0/1931484153 learned_addr learned my addr 172.21.15.148:0/1931484153 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f6074bba640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f60700ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.891+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917376569 0 0) 0x7f6068001490 con 0x7f607015b980 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f605c003520 con 0x7f607015b980 2024-04-25T09:55:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717833755 0 0) 0x7f60700cf040 con 0x7f607015b5a0 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6068001490 con 0x7f607015b5a0 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 908281729 0 0) 0x7f605c003520 con 0x7f607015b980 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 msgr2=0x7f60700ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f60700ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 >> v1:172.21.15.188:6789/0 conn(0x7f607015b5a0 legacy=0x7f60700b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070161c50 con 0x7f607015b980 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6068001bc0 con 0x7f607015b980 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 >> v1:172.21.15.148:6789/0 conn(0x7f607015b980 legacy=0x7f60700ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c003b70 0x7f605c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f605c004800 0x7f605c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f605c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070161970 con 0x7f605c003b70 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c00baa0 con 0x7f605c004800 2024-04-25T09:55:06.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c00bca0 con 0x7f60700b5c40 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f60753bb640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c003b70 0x7f605c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6075bbc640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f605c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6074bba640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f605c004800 0x7f605c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f60753bb640 1 -- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 msgr2=0x7f605c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f60753bb640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f605c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f60753bb640 1 -- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f605c004800 msgr2=0x7f605c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f60753bb640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f605c004800 0x7f605c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f60753bb640 1 -- 172.21.15.148:0/1931484153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c003520 con 0x7f605c003b70 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6075bbc640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60700b5c40 0x7f605c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.892+0000 7f6074bba640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f605c004800 0x7f605c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f60753bb640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c003b70 0x7f605c003f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f606c0018a0 tx=0x7f605c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1972fa37317d73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0014f0 con 0x7f605c003b70 2024-04-25T09:55:06.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c0052d0 con 0x7f605c003b70 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f6067fff640 1 -- 172.21.15.148:0/1931484153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c02ac40 con 0x7f605c003b70 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f607761f640 1 -- 172.21.15.148:0/1931484153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c003b70 msgr2=0x7f605c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f607761f640 1 --2- 172.21.15.148:0/1931484153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c003b70 0x7f605c003f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f606c0018a0 tx=0x7f605c005830 comp rx=0 tx=0).stop 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.893+0000 7f6075bbc640 1 -- 172.21.15.148:0/1931484153 reap_dead start 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.894+0000 7f607761f640 1 -- 172.21.15.148:0/1931484153 shutdown_connections 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.894+0000 7f607761f640 1 -- 172.21.15.148:0/1931484153 >> 172.21.15.148:0/1931484153 conn(0x7f60700d0530 msgr2=0x7f60700d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.894+0000 7f607761f640 1 -- 172.21.15.148:0/1931484153 shutdown_connections 2024-04-25T09:55:06.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.894+0000 7f607761f640 1 -- 172.21.15.148:0/1931484153 wait complete. 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.895+0000 7f607761f640 1 Processor -- start 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.895+0000 7f607761f640 1 -- start start 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f605c003b70 0x7f607014bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 0x7f607015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070160500 0x7f60700d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701625d0 con 0x7f6070157be0 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070162450 con 0x7f6070160500 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6070162750 con 0x7f605c003b70 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 0x7f607015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 0x7f607015ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51344/0 (socket says 172.21.15.148:51344) 2024-04-25T09:55:06.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 -- 172.21.15.148:0/2547157487 learned_addr learned my addr 172.21.15.148:0/2547157487 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6075bbc640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070160500 0x7f60700d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f60753bb640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f605c003b70 0x7f607014bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 -- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f605c003b70 msgr2=0x7f607014bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f605c003b70 0x7f607014bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 -- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070160500 msgr2=0x7f60700d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070160500 0x7f60700d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 -- 172.21.15.148:0/2547157487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60701f76a0 con 0x7f6070157be0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6075bbc640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070160500 0x7f60700d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f60753bb640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f605c003b70 0x7f607014bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6074bba640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 0x7f607015ffc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f60600116d0 tx=0x7f6060011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060019070 con 0x7f6070157be0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060004220 con 0x7f6070157be0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60701f7940 con 0x7f6070157be0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.896+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060005990 con 0x7f6070157be0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.897+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701f7f00 con 0x7f6070157be0 2024-04-25T09:55:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.898+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6070073a00 con 0x7f6070157be0 2024-04-25T09:55:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.898+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6060005b30 con 0x7f6070157be0 2024-04-25T09:55:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.898+0000 7f6065ffb640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f605406b4d0 0x7f605406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.899+0000 7f60753bb640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f605406b4d0 0x7f605406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.899+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f606008ab10 con 0x7f6070157be0 2024-04-25T09:55:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.899+0000 7f60753bb640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f605406b4d0 0x7f605406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f606c032000 tx=0x7f606c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:06.903+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6060005e20 con 0x7f6070157be0 2024-04-25T09:55:07.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.074+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60700d17c0 con 0x7f6070157be0 2024-04-25T09:55:07.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.075+0000 7f6065ffb640 1 -- 172.21.15.148:0/2547157487 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6060059580 con 0x7f6070157be0 2024-04-25T09:55:07.078 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.076+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f605406b4d0 msgr2=0x7f605406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.076+0000 7f607761f640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f605406b4d0 0x7f605406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f606c032000 tx=0x7f606c009030 comp rx=0 tx=0).stop 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.076+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 msgr2=0x7f607015ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 0x7f607015ffc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f60600116d0 tx=0x7f6060011b90 comp rx=0 tx=0).stop 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 shutdown_connections 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f605406b4d0 0x7f605406d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6070160500 0x7f60700d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6070157be0 0x7f607015ffc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 --2- 172.21.15.148:0/2547157487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f605c003b70 0x7f607014bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 >> 172.21.15.148:0/2547157487 conn(0x7f60700d0530 msgr2=0x7f60701566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 shutdown_connections 2024-04-25T09:55:07.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.077+0000 7f607761f640 1 -- 172.21.15.148:0/2547157487 wait complete. 2024-04-25T09:55:07.091 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:07.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.189+0000 7fe778d0a640 1 Processor -- start 2024-04-25T09:55:07.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.189+0000 7fe778d0a640 1 -- start start 2024-04-25T09:55:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe778d0a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe778d0a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77406cf50 con 0x7fe77415b7d0 2024-04-25T09:55:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe778d0a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740ce530 con 0x7fe77406c2d0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe778d0a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740b5fc0 con 0x7fe7740b5620 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe77259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe77415b7d0 0x7fe7740cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43152/0 (socket says 172.21.15.148:43152) 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe77259c640 1 -- 172.21.15.148:0/3956971820 learned_addr learned my addr 172.21.15.148:0/3956971820 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133482703 0 0) 0x7fe77406cf50 con 0x7fe77415b7d0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe760003520 con 0x7fe77415b7d0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe76bfff640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1296865110 0 0) 0x7fe760003520 con 0x7fe77415b7d0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 msgr2=0x7fe7740b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 >> v1:172.21.15.188:6789/0 conn(0x7fe77406c2d0 legacy=0x7fe77406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740ce810 con 0x7fe77415b7d0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe764001670 con 0x7fe77415b7d0 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 >> v1:172.21.15.148:6789/0 conn(0x7fe77415b7d0 legacy=0x7fe7740cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe760004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760004d40 0x7fe7600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76000bb50 con 0x7fe760004010 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76000bd00 con 0x7fe760004d40 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76000bf00 con 0x7fe7740b5620 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.190+0000 7fe76bfff640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe760004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe77259c640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe772d9d640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760004d40 0x7fe7600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe76bfff640 1 -- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 msgr2=0x7fe7600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe76bfff640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe76bfff640 1 -- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760004d40 msgr2=0x7fe7600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:07.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe76bfff640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe760004d40 0x7fe7600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe76bfff640 1 -- 172.21.15.148:0/3956971820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760003520 con 0x7fe760004010 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.191+0000 7fe76bfff640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe760004800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00ba30 tx=0x7fe760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbe6c7736905568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c030070 con 0x7fe760004010 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c02ade0 con 0x7fe760004010 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe771d9b640 1 -- 172.21.15.148:0/3956971820 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00b600 con 0x7fe760004010 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe778d0a640 1 -- 172.21.15.148:0/3956971820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 msgr2=0x7fe760004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe778d0a640 1 --2- 172.21.15.148:0/3956971820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe760004800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00ba30 tx=0x7fe760005830 comp rx=0 tx=0).stop 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe772d9d640 1 -- 172.21.15.148:0/3956971820 reap_dead start 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe778d0a640 1 -- 172.21.15.148:0/3956971820 shutdown_connections 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe778d0a640 1 -- 172.21.15.148:0/3956971820 >> 172.21.15.148:0/3956971820 conn(0x7fe7740d0800 msgr2=0x7fe7740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe778d0a640 1 -- 172.21.15.148:0/3956971820 shutdown_connections 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.192+0000 7fe778d0a640 1 -- 172.21.15.148:0/3956971820 wait complete. 2024-04-25T09:55:07.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.193+0000 7fe778d0a640 1 Processor -- start 2024-04-25T09:55:07.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 -- start start 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe7740b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741f6b70 0x7fe7741f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe774149c20 0x7fe77414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77415bbb0 con 0x7fe760004010 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77406d230 con 0x7fe7741f6b70 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe778d0a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77415bd30 con 0x7fe774149c20 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe77259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe7740b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe77259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe7740b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51360/0 (socket says 172.21.15.148:51360) 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.194+0000 7fe77259c640 1 -- 172.21.15.148:0/4129448818 learned_addr learned my addr 172.21.15.148:0/4129448818 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe77259c640 1 -- 172.21.15.148:0/4129448818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe774149c20 msgr2=0x7fe77414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe77259c640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe774149c20 0x7fe77414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe77259c640 1 -- 172.21.15.148:0/4129448818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741f6b70 msgr2=0x7fe7741f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe77259c640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741f6b70 0x7fe7741f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe77259c640 1 -- 172.21.15.148:0/4129448818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7741f70e0 con 0x7fe760004010 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe77259c640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe7740b8a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe7640073a0 tx=0x7fe764007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76401a070 con 0x7fe760004010 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76400f8e0 con 0x7fe760004010 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7741f73c0 con 0x7fe760004010 2024-04-25T09:55:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76400fc90 con 0x7fe760004010 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.195+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741f7980 con 0x7fe760004010 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.196+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe774073a00 con 0x7fe760004010 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.196+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe76401d070 con 0x7fe760004010 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.197+0000 7fe76affd640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75406b4d0 0x7fe75406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.197+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fe76408a060 con 0x7fe760004010 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.197+0000 7fe76bfff640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75406b4d0 0x7fe75406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.197+0000 7fe76bfff640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75406b4d0 0x7fe75406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00ba00 tx=0x7fe75c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.201+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe764058ad0 con 0x7fe760004010 2024-04-25T09:55:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.368+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe77415ec80 con 0x7fe760004010 2024-04-25T09:55:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.369+0000 7fe76affd640 1 -- 172.21.15.148:0/4129448818 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7640583d0 con 0x7fe760004010 2024-04-25T09:55:07.373 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75406b4d0 msgr2=0x7fe75406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75406b4d0 0x7fe75406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00ba00 tx=0x7fe75c02c000 comp rx=0 tx=0).stop 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 msgr2=0x7fe7740b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe7740b8a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe7640073a0 tx=0x7fe764007860 comp rx=0 tx=0).stop 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 shutdown_connections 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75406b4d0 0x7fe75406d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe774149c20 0x7fe77414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741f6b70 0x7fe7741f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 --2- 172.21.15.148:0/4129448818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004010 0x7fe7740b8a60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 >> 172.21.15.148:0/4129448818 conn(0x7fe7740d0800 msgr2=0x7fe7740b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 shutdown_connections 2024-04-25T09:55:07.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.371+0000 7fe778d0a640 1 -- 172.21.15.148:0/4129448818 wait complete. 2024-04-25T09:55:07.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:07.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.481+0000 7fc27f517640 1 Processor -- start 2024-04-25T09:55:07.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.481+0000 7fc27f517640 1 -- start start 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27f517640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2781610f0 0x7fc27806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27f517640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc2781616f0 0x7fc278161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27f517640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2780c6960 con 0x7fc2780ce0a0 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27f517640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2781614d0 con 0x7fc2781616f0 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27f517640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc278162260 con 0x7fc2781610f0 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27cab2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc2781616f0 0x7fc278161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27dab4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc2780ce0a0 0x7fc2780d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43162/0 (socket says 172.21.15.148:43162) 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.482+0000 7fc27dab4640 1 -- 172.21.15.148:0/481665417 learned_addr learned my addr 172.21.15.148:0/481665417 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27d2b3640 1 --2- 172.21.15.148:0/481665417 >> v2:172.21.15.188:3300/0 conn(0x7fc2781610f0 0x7fc27806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27cab2640 1 -- 172.21.15.148:0/481665417 >> v2:172.21.15.188:3300/0 conn(0x7fc2781610f0 msgr2=0x7fc27806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27cab2640 1 --2- 172.21.15.148:0/481665417 >> v2:172.21.15.188:3300/0 conn(0x7fc2781610f0 0x7fc27806c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27cab2640 1 -- 172.21.15.148:0/481665417 >> v1:172.21.15.148:6789/0 conn(0x7fc2780ce0a0 legacy=0x7fc2780d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc26ffff640 1 -- 172.21.15.148:0/481665417 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629854701 0 0) 0x7fc2780c6960 con 0x7fc2780ce0a0 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27cab2640 1 -- 172.21.15.148:0/481665417 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278162540 con 0x7fc2781616f0 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27d2b3640 1 --2- 172.21.15.148:0/481665417 >> v2:172.21.15.188:3300/0 conn(0x7fc2781610f0 0x7fc27806c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.483+0000 7fc27cab2640 1 --2- 172.21.15.148:0/481665417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781616f0 0x7fc278161b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc278071fb0 tx=0x7fc26802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e4fcf895a8ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc26ffff640 1 -- 172.21.15.148:0/481665417 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268037070 con 0x7fc2781616f0 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc26ffff640 1 -- 172.21.15.148:0/481665417 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268032440 con 0x7fc2781616f0 2024-04-25T09:55:07.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc26ffff640 1 -- 172.21.15.148:0/481665417 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268032810 con 0x7fc2781616f0 2024-04-25T09:55:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 -- 172.21.15.148:0/481665417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781616f0 msgr2=0x7fc278161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 --2- 172.21.15.148:0/481665417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781616f0 0x7fc278161b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc278071fb0 tx=0x7fc26802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 -- 172.21.15.148:0/481665417 shutdown_connections 2024-04-25T09:55:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 --2- 172.21.15.148:0/481665417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781616f0 0x7fc278161b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 --2- 172.21.15.148:0/481665417 >> v2:172.21.15.188:3300/0 conn(0x7fc2781610f0 0x7fc27806c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 -- 172.21.15.148:0/481665417 >> 172.21.15.148:0/481665417 conn(0x7fc27806f500 msgr2=0x7fc27806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 -- 172.21.15.148:0/481665417 shutdown_connections 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.484+0000 7fc27f517640 1 -- 172.21.15.148:0/481665417 wait complete. 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.485+0000 7fc27f517640 1 Processor -- start 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.485+0000 7fc27f517640 1 -- start start 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27f517640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2780ce0a0 0x7fc2780b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27f517640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 0x7fc278144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27f517640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2781616f0 0x7fc278147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27f517640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780cf690 con 0x7fc2781610f0 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27f517640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2780cfce0 con 0x7fc2780ce0a0 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27f517640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27806c700 con 0x7fc2781616f0 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 0x7fc278144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 0x7fc278144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51382/0 (socket says 172.21.15.148:51382) 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 -- 172.21.15.148:0/51572950 learned_addr learned my addr 172.21.15.148:0/51572950 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27dab4640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2781616f0 0x7fc278147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27d2b3640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2780ce0a0 0x7fc2780b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 -- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2781616f0 msgr2=0x7fc278147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2781616f0 0x7fc278147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 -- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2780ce0a0 msgr2=0x7fc2780b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2780ce0a0 0x7fc2780b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.486+0000 7fc27cab2640 1 -- 172.21.15.148:0/51572950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2781449d0 con 0x7fc2781610f0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc27d2b3640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2780ce0a0 0x7fc2780b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc27cab2640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 0x7fc278144490 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc268009890 tx=0x7fc2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268037040 con 0x7fc2781610f0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268042070 con 0x7fc2781610f0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc278147c70 con 0x7fc2781610f0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268030a50 con 0x7fc2781610f0 2024-04-25T09:55:07.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.487+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2780b6300 con 0x7fc2781610f0 2024-04-25T09:55:07.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.488+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc278073a00 con 0x7fc2781610f0 2024-04-25T09:55:07.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.488+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc268030bf0 con 0x7fc2781610f0 2024-04-25T09:55:07.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.489+0000 7fc26dffb640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25c06b500 0x7fc25c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.489+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fc2680ab0d0 con 0x7fc2781610f0 2024-04-25T09:55:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.489+0000 7fc27d2b3640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25c06b500 0x7fc25c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.489+0000 7fc27d2b3640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25c06b500 0x7fc25c06d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc27400d430 tx=0x7fc274007530 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.493+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc268079b40 con 0x7fc2781610f0 2024-04-25T09:55:07.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.661+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc2780d25a0 con 0x7fc2781610f0 2024-04-25T09:55:07.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.661+0000 7fc26dffb640 1 -- 172.21.15.148:0/51572950 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc268079440 con 0x7fc2781610f0 2024-04-25T09:55:07.665 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:07.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.663+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25c06b500 msgr2=0x7fc25c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.663+0000 7fc27f517640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25c06b500 0x7fc25c06d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc27400d430 tx=0x7fc274007530 comp rx=0 tx=0).stop 2024-04-25T09:55:07.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 msgr2=0x7fc278144490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 0x7fc278144490 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc268009890 tx=0x7fc2680046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 shutdown_connections 2024-04-25T09:55:07.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25c06b500 0x7fc25c06d9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2781616f0 0x7fc278147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2781610f0 0x7fc278144490 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 --2- 172.21.15.148:0/51572950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2780ce0a0 0x7fc2780b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 >> 172.21.15.148:0/51572950 conn(0x7fc27806f500 msgr2=0x7fc2780c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 shutdown_connections 2024-04-25T09:55:07.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.664+0000 7fc27f517640 1 -- 172.21.15.148:0/51572950 wait complete. 2024-04-25T09:55:07.678 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:07.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.772+0000 7f6b7ee76640 1 Processor -- start 2024-04-25T09:55:07.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.772+0000 7f6b7ee76640 1 -- start start 2024-04-25T09:55:07.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7ee76640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b7815ba00 0x7f6b7809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7ee76640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b781602e0 0x7f6b780ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7ee76640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780cd710 con 0x7f6b7815f5f0 2024-04-25T09:55:07.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7ee76640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b78099b70 con 0x7f6b7815ba00 2024-04-25T09:55:07.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7ee76640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b78099e50 con 0x7f6b781602e0 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b7815ba00 0x7f6b7809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6b7815ba00 0x7f6b7809a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51388/0 (socket says 172.21.15.148:51388) 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 -- 172.21.15.148:0/2021572697 learned_addr learned my addr 172.21.15.148:0/2021572697 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 -- 172.21.15.148:0/2021572697 >> v2:172.21.15.188:3300/0 conn(0x7f6b781602e0 msgr2=0x7f6b780ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b77fff640 1 --2- 172.21.15.148:0/2021572697 >> v2:172.21.15.188:3300/0 conn(0x7f6b781602e0 0x7f6b780ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 --2- 172.21.15.148:0/2021572697 >> v2:172.21.15.188:3300/0 conn(0x7f6b781602e0 0x7f6b780ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 -- 172.21.15.148:0/2021572697 >> v1:172.21.15.188:6789/0 conn(0x7f6b7815f5f0 legacy=0x7f6b780c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b7cc12640 1 -- 172.21.15.148:0/2021572697 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7809a130 con 0x7f6b7815ba00 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.773+0000 7f6b77fff640 1 --2- 172.21.15.148:0/2021572697 >> v2:172.21.15.188:3300/0 conn(0x7f6b781602e0 0x7f6b780ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b7cc12640 1 --2- 172.21.15.148:0/2021572697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815ba00 0x7f6b7809a930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b780b7830 tx=0x7f6b6c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9b51577f891f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b777fe640 1 -- 172.21.15.148:0/2021572697 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c037070 con 0x7f6b7815ba00 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b777fe640 1 -- 172.21.15.148:0/2021572697 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c02ae20 con 0x7f6b7815ba00 2024-04-25T09:55:07.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b777fe640 1 -- 172.21.15.148:0/2021572697 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c032660 con 0x7f6b7815ba00 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b7ee76640 1 -- 172.21.15.148:0/2021572697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815ba00 msgr2=0x7f6b7809a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/2021572697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815ba00 0x7f6b7809a930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b780b7830 tx=0x7f6b6c02a110 comp rx=0 tx=0).stop 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b7ee76640 1 -- 172.21.15.148:0/2021572697 shutdown_connections 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/2021572697 >> v2:172.21.15.188:3300/0 conn(0x7f6b781602e0 0x7f6b780ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.774+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/2021572697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815ba00 0x7f6b7809a930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.775+0000 7f6b7ee76640 1 -- 172.21.15.148:0/2021572697 >> 172.21.15.148:0/2021572697 conn(0x7f6b780b3270 msgr2=0x7f6b780b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.775+0000 7f6b7ee76640 1 -- 172.21.15.148:0/2021572697 shutdown_connections 2024-04-25T09:55:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.775+0000 7f6b7ee76640 1 -- 172.21.15.148:0/2021572697 wait complete. 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.776+0000 7f6b7ee76640 1 Processor -- start 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.776+0000 7f6b7ee76640 1 -- start start 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b7815ba00 0x7f6b7814db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 0x7f6b781e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b781602e0 0x7f6b780cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7815bde0 con 0x7f6b7815f5f0 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780cd2e0 con 0x7f6b781602e0 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7814e090 con 0x7f6b7815ba00 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 0x7f6b781e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 0x7f6b781e67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51404/0 (socket says 172.21.15.148:51404) 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 -- 172.21.15.148:0/3748880980 learned_addr learned my addr 172.21.15.148:0/3748880980 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7cc12640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b7815ba00 0x7f6b7814db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7d413640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b781602e0 0x7f6b780cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 -- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b7815ba00 msgr2=0x7f6b7814db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b7815ba00 0x7f6b7814db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 -- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b781602e0 msgr2=0x7f6b780cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b781602e0 0x7f6b780cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 -- 172.21.15.148:0/3748880980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b781e70b0 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7d413640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b781602e0 0x7f6b780cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b77fff640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 0x7f6b781e67d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c7f0 tx=0x7f6b6400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64015070 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b64007a30 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b781e7390 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b64007e00 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.777+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b780b6040 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.778+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b78073990 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.779+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b64018070 con 0x7f6b7815f5f0 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.779+0000 7f6b757fa640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.779+0000 7f6b7cc12640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:07.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.779+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f6b6408a560 con 0x7f6b7815f5f0 2024-04-25T09:55:07.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.780+0000 7f6b7cc12640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c034000 tx=0x7f6b6c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:07.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.783+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b64058980 con 0x7f6b7815f5f0 2024-04-25T09:55:07.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.955+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b780b40a0 con 0x7f6b7815f5f0 2024-04-25T09:55:07.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.956+0000 7f6b757fa640 1 -- 172.21.15.148:0/3748880980 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7f6b6405ca20 con 0x7f6b7815f5f0 2024-04-25T09:55:07.959 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:07.960 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b5c06b4d0 msgr2=0x7f6b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c034000 tx=0x7f6b6c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 msgr2=0x7f6b781e67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 0x7f6b781e67d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400c7f0 tx=0x7f6b6400cbf0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 shutdown_connections 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b5c06b4d0 0x7f6b5c06d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b781602e0 0x7f6b780cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b7815f5f0 0x7f6b781e67d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 --2- 172.21.15.148:0/3748880980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b7815ba00 0x7f6b7814db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 >> 172.21.15.148:0/3748880980 conn(0x7f6b780b3270 msgr2=0x7f6b7815e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 shutdown_connections 2024-04-25T09:55:07.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:07.958+0000 7f6b7ee76640 1 -- 172.21.15.148:0/3748880980 wait complete. 2024-04-25T09:55:07.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:07.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:08.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.071+0000 7fcd7684a640 1 Processor -- start 2024-04-25T09:55:08.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.071+0000 7fcd7684a640 1 -- start start 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd7684a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd7015dac0 0x7fcd700d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd7684a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcd700d1f30 0x7fcd700d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd7684a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700d2420 con 0x7fcd700d34c0 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd7684a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700d3030 con 0x7fcd700d1f30 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd7684a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700d2700 con 0x7fcd7015dac0 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd6f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcd700d1f30 0x7fcd700d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd6f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcd700d1f30 0x7fcd700d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51414/0 (socket says 172.21.15.148:51414) 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/4099900098 learned_addr learned my addr 172.21.15.148:0/4099900098 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.072+0000 7fcd6ffff640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd7015dac0 0x7fcd700d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/4099900098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd7015dac0 msgr2=0x7fcd700d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6f7fe640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd7015dac0 0x7fcd700d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/4099900098 >> v1:172.21.15.188:6789/0 conn(0x7fcd700d34c0 legacy=0x7fcd700b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/4099900098 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700d29e0 con 0x7fcd700d1f30 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6ffff640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd7015dac0 0x7fcd700d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6f7fe640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d1f30 0x7fcd700d03a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009860 tx=0x7fcd6402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877dd8110ff37e70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6effd640 1 -- 172.21.15.148:0/4099900098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64037070 con 0x7fcd700d1f30 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6effd640 1 -- 172.21.15.148:0/4099900098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64032440 con 0x7fcd700d1f30 2024-04-25T09:55:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.073+0000 7fcd6effd640 1 -- 172.21.15.148:0/4099900098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64032810 con 0x7fcd700d1f30 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 -- 172.21.15.148:0/4099900098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d1f30 msgr2=0x7fcd700d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d1f30 0x7fcd700d03a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009860 tx=0x7fcd6402a290 comp rx=0 tx=0).stop 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 -- 172.21.15.148:0/4099900098 shutdown_connections 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d1f30 0x7fcd700d03a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 --2- 172.21.15.148:0/4099900098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd7015dac0 0x7fcd700d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 -- 172.21.15.148:0/4099900098 >> 172.21.15.148:0/4099900098 conn(0x7fcd70061c30 msgr2=0x7fcd7006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 -- 172.21.15.148:0/4099900098 shutdown_connections 2024-04-25T09:55:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.074+0000 7fcd7684a640 1 -- 172.21.15.148:0/4099900098 wait complete. 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.075+0000 7fcd7684a640 1 Processor -- start 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.075+0000 7fcd7684a640 1 -- start start 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd7684a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd700d1f30 0x7fcd700cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd7684a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 0x7fcd70149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd7684a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7015dac0 0x7fcd700cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd7684a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b97d0 con 0x7fcd700d34c0 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd7684a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015dea0 con 0x7fcd7015dac0 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd7684a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b9950 con 0x7fcd700d1f30 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 0x7fcd70149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 0x7fcd70149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51426/0 (socket says 172.21.15.148:51426) 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/2539739731 learned_addr learned my addr 172.21.15.148:0/2539739731 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6ffff640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd700d1f30 0x7fcd700cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd74de7640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7015dac0 0x7fcd700cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd700d1f30 msgr2=0x7fcd700cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd700d1f30 0x7fcd700cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7015dac0 msgr2=0x7fcd700cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.076+0000 7fcd6f7fe640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7015dac0 0x7fcd700cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd6f7fe640 1 -- 172.21.15.148:0/2539739731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7015faf0 con 0x7fcd700d34c0 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd74de7640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7015dac0 0x7fcd700cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd6f7fe640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 0x7fcd70149ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009860 tx=0x7fcd640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64041020 con 0x7fcd700d34c0 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64030970 con 0x7fcd700d34c0 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7015fdd0 con 0x7fcd700d34c0 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64037040 con 0x7fcd700d34c0 2024-04-25T09:55:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.077+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7014dd30 con 0x7fcd700d34c0 2024-04-25T09:55:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.078+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd70073a00 con 0x7fcd700d34c0 2024-04-25T09:55:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.078+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd64044020 con 0x7fcd700d34c0 2024-04-25T09:55:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.079+0000 7fcd6cff9640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcd5c06b4d0 0x7fcd5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.079+0000 7fcd6ffff640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcd5c06b4d0 0x7fcd5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.079+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fcd640aacb0 con 0x7fcd700d34c0 2024-04-25T09:55:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.079+0000 7fcd6ffff640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcd5c06b4d0 0x7fcd5c06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000a950 tx=0x7fcd60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.083+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd6404b070 con 0x7fcd700d34c0 2024-04-25T09:55:08.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.253+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd7015ffa0 con 0x7fcd700d34c0 2024-04-25T09:55:08.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.254+0000 7fcd6cff9640 1 -- 172.21.15.148:0/2539739731 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd64079720 con 0x7fcd700d34c0 2024-04-25T09:55:08.258 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcd5c06b4d0 msgr2=0x7fcd5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcd5c06b4d0 0x7fcd5c06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000a950 tx=0x7fcd60008040 comp rx=0 tx=0).stop 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 msgr2=0x7fcd70149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 0x7fcd70149ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009860 tx=0x7fcd640046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 shutdown_connections 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcd5c06b4d0 0x7fcd5c06d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd7015dac0 0x7fcd700cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd700d34c0 0x7fcd70149ba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 --2- 172.21.15.148:0/2539739731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd700d1f30 0x7fcd700cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 >> 172.21.15.148:0/2539739731 conn(0x7fcd70061c30 msgr2=0x7fcd700b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 shutdown_connections 2024-04-25T09:55:08.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.256+0000 7fcd7684a640 1 -- 172.21.15.148:0/2539739731 wait complete. 2024-04-25T09:55:08.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:08.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.367+0000 7fad8babc640 1 Processor -- start 2024-04-25T09:55:08.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.368+0000 7fad8babc640 1 -- start start 2024-04-25T09:55:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.368+0000 7fad8babc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad840cadb0 0x7fad840cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.368+0000 7fad8babc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad840ccfe0 0x7fad840cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.368+0000 7fad8babc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840afc80 con 0x7fad8415de80 2024-04-25T09:55:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.368+0000 7fad8babc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad840cb240 con 0x7fad840cadb0 2024-04-25T09:55:08.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.368+0000 7fad8babc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840ce4d0 con 0x7fad840ccfe0 2024-04-25T09:55:08.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad840cadb0 0x7fad840cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad8a059640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad840ccfe0 0x7fad840cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad840cadb0 0x7fad840cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:44580/0 (socket says 172.21.15.148:44580) 2024-04-25T09:55:08.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 -- 172.21.15.148:0/518688713 learned_addr learned my addr 172.21.15.148:0/518688713 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 -- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad840ccfe0 msgr2=0x7fad840cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 --2- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad840ccfe0 0x7fad840cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 -- 172.21.15.148:0/518688713 >> v1:172.21.15.188:6789/0 conn(0x7fad8415de80 legacy=0x7fad840cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.369+0000 7fad89057640 1 -- 172.21.15.148:0/518688713 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840ce7b0 con 0x7fad840cadb0 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.370+0000 7fad89057640 1 --2- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad840cf6c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad74009870 tx=0x7fad7402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3600a09bee96e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.370+0000 7fad88856640 1 -- 172.21.15.148:0/518688713 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74037070 con 0x7fad840cadb0 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.370+0000 7fad88856640 1 -- 172.21.15.148:0/518688713 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74032440 con 0x7fad840cadb0 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.370+0000 7fad88856640 1 -- 172.21.15.148:0/518688713 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74032810 con 0x7fad840cadb0 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 -- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 msgr2=0x7fad840cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 --2- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad840cf6c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad74009870 tx=0x7fad7402a270 comp rx=0 tx=0).stop 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 -- 172.21.15.148:0/518688713 shutdown_connections 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 --2- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad840cf6c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 --2- 172.21.15.148:0/518688713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad840ccfe0 0x7fad840cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 -- 172.21.15.148:0/518688713 >> 172.21.15.148:0/518688713 conn(0x7fad840b2390 msgr2=0x7fad840b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 -- 172.21.15.148:0/518688713 shutdown_connections 2024-04-25T09:55:08.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.371+0000 7fad8babc640 1 -- 172.21.15.148:0/518688713 wait complete. 2024-04-25T09:55:08.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.372+0000 7fad8babc640 1 Processor -- start 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.372+0000 7fad8babc640 1 -- start start 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad84152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 0x7fad8415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad8415de80 0x7fad8415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8415e260 con 0x7fad840ccfe0 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad840cfc00 con 0x7fad840cadb0 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840b2eb0 con 0x7fad8415de80 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 0x7fad8415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 0x7fad8415cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51434/0 (socket says 172.21.15.148:51434) 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 -- 172.21.15.148:0/606498628 learned_addr learned my addr 172.21.15.148:0/606498628 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89858640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad84152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8a059640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad8415de80 0x7fad8415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 -- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad8415de80 msgr2=0x7fad8415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad8415de80 0x7fad8415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 -- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 msgr2=0x7fad84152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad84152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 -- 172.21.15.148:0/606498628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad841f38e0 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89858640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad84152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad89057640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 0x7fad8415cbe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad74000c00 tx=0x7fad740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74037040 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74042070 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad841f3ba0 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74030880 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.373+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad841f4160 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.374+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad84073a00 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.375+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad74030a20 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.375+0000 7fad7a7fc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6806b4d0 0x7fad6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.375+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fad740aae80 con 0x7fad840ccfe0 2024-04-25T09:55:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.375+0000 7fad89858640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6806b4d0 0x7fad6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.376+0000 7fad89858640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6806b4d0 0x7fad6806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad840d0220 tx=0x7fad80009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.379+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad740798f0 con 0x7fad840ccfe0 2024-04-25T09:55:08.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.548+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad840d1470 con 0x7fad840ccfe0 2024-04-25T09:55:08.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.549+0000 7fad7a7fc640 1 -- 172.21.15.148:0/606498628 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad740791f0 con 0x7fad840ccfe0 2024-04-25T09:55:08.553 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6806b4d0 msgr2=0x7fad6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6806b4d0 0x7fad6806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad840d0220 tx=0x7fad80009040 comp rx=0 tx=0).stop 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 msgr2=0x7fad8415cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 0x7fad8415cbe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad74000c00 tx=0x7fad740046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 shutdown_connections 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6806b4d0 0x7fad6806d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad8415de80 0x7fad8415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad840ccfe0 0x7fad8415cbe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 --2- 172.21.15.148:0/606498628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad840cadb0 0x7fad84152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 >> 172.21.15.148:0/606498628 conn(0x7fad840b2390 msgr2=0x7fad840b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 shutdown_connections 2024-04-25T09:55:08.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.551+0000 7fad8babc640 1 -- 172.21.15.148:0/606498628 wait complete. 2024-04-25T09:55:08.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:08.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.662+0000 7f35d719e640 1 Processor -- start 2024-04-25T09:55:08.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.662+0000 7f35d719e640 1 -- start start 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35d719e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35d00cadb0 0x7f35d00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35d719e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35d00ccfe0 0x7f35d00cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35d719e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00afc80 con 0x7f35d015de80 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35d719e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d00cb240 con 0x7f35d00cadb0 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35d719e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d00ce4d0 con 0x7f35d00ccfe0 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35cffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35d00cadb0 0x7f35d00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35cffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35d00cadb0 0x7f35d00cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51444/0 (socket says 172.21.15.148:51444) 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35d4f3a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f35d015de80 0x7f35d00cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43172/0 (socket says 172.21.15.148:43172) 2024-04-25T09:55:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.663+0000 7f35cffff640 1 -- 172.21.15.148:0/3553890733 learned_addr learned my addr 172.21.15.148:0/3553890733 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35d573b640 1 --2- 172.21.15.148:0/3553890733 >> v2:172.21.15.188:3300/0 conn(0x7f35d00ccfe0 0x7f35d00cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35cffff640 1 -- 172.21.15.148:0/3553890733 >> v2:172.21.15.188:3300/0 conn(0x7f35d00ccfe0 msgr2=0x7f35d00cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35cffff640 1 --2- 172.21.15.148:0/3553890733 >> v2:172.21.15.188:3300/0 conn(0x7f35d00ccfe0 0x7f35d00cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35cffff640 1 -- 172.21.15.148:0/3553890733 >> v1:172.21.15.148:6789/0 conn(0x7f35d015de80 legacy=0x7f35d00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35cffff640 1 -- 172.21.15.148:0/3553890733 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00ce7b0 con 0x7f35d00cadb0 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35cf7fe640 1 -- 172.21.15.148:0/3553890733 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564757565 0 0) 0x7f35d00afc80 con 0x7f35d015de80 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.664+0000 7f35cffff640 1 --2- 172.21.15.148:0/3553890733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d00cf6c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f35d006e2a0 tx=0x7f35c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5963160f8aefb144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35cf7fe640 1 -- 172.21.15.148:0/3553890733 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4037070 con 0x7f35d00cadb0 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35cf7fe640 1 -- 172.21.15.148:0/3553890733 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c4032440 con 0x7f35d00cadb0 2024-04-25T09:55:08.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35cf7fe640 1 -- 172.21.15.148:0/3553890733 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4032810 con 0x7f35d00cadb0 2024-04-25T09:55:08.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 -- 172.21.15.148:0/3553890733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 msgr2=0x7f35d00cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 --2- 172.21.15.148:0/3553890733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d00cf6c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f35d006e2a0 tx=0x7f35c402a270 comp rx=0 tx=0).stop 2024-04-25T09:55:08.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 -- 172.21.15.148:0/3553890733 shutdown_connections 2024-04-25T09:55:08.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 --2- 172.21.15.148:0/3553890733 >> v2:172.21.15.188:3300/0 conn(0x7f35d00ccfe0 0x7f35d00cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 --2- 172.21.15.148:0/3553890733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d00cf6c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 -- 172.21.15.148:0/3553890733 >> 172.21.15.148:0/3553890733 conn(0x7f35d00b2390 msgr2=0x7f35d00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 -- 172.21.15.148:0/3553890733 shutdown_connections 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.665+0000 7f35d719e640 1 -- 172.21.15.148:0/3553890733 wait complete. 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.666+0000 7f35d719e640 1 Processor -- start 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.666+0000 7f35d719e640 1 -- start start 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d719e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d719e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d00ccfe0 0x7f35d00b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d719e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d015de80 0x7f35d00b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d719e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d015e260 con 0x7f35d00cadb0 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d719e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00cfc00 con 0x7f35d015de80 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d719e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00b2eb0 con 0x7f35d00ccfe0 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d4f3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d4f3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d0155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51458/0 (socket says 172.21.15.148:51458) 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d4f3a640 1 -- 172.21.15.148:0/188102046 learned_addr learned my addr 172.21.15.148:0/188102046 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35d573b640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d015de80 0x7f35d00b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.667+0000 7f35cffff640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d00ccfe0 0x7f35d00b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d4f3a640 1 -- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d00ccfe0 msgr2=0x7f35d00b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d4f3a640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d00ccfe0 0x7f35d00b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d4f3a640 1 -- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d015de80 msgr2=0x7f35d00b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d4f3a640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d015de80 0x7f35d00b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d4f3a640 1 -- 172.21.15.148:0/188102046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00b0f70 con 0x7f35d00cadb0 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35cffff640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d00ccfe0 0x7f35d00b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d573b640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d015de80 0x7f35d00b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d4f3a640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d0155240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c430 tx=0x7f35c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c00074e0 con 0x7f35d00cadb0 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c00047e0 con 0x7f35d00cadb0 2024-04-25T09:55:08.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0004bd0 con 0x7f35d00cadb0 2024-04-25T09:55:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d0155780 con 0x7f35d00cadb0 2024-04-25T09:55:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.668+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d00b01b0 con 0x7f35d00cadb0 2024-04-25T09:55:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.669+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d0073a00 con 0x7f35d00cadb0 2024-04-25T09:55:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.669+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35c0011070 con 0x7f35d00cadb0 2024-04-25T09:55:08.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.670+0000 7f35cd7fa640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35bc06b4d0 0x7f35bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.670+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f35c008a240 con 0x7f35d00cadb0 2024-04-25T09:55:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.670+0000 7f35cffff640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35bc06b4d0 0x7f35bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.670+0000 7f35cffff640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35bc06b4d0 0x7f35bc06d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f35c4034000 tx=0x7f35c402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.675+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35c0058d40 con 0x7f35d00cadb0 2024-04-25T09:55:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.843+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35d00b2a50 con 0x7f35d00cadb0 2024-04-25T09:55:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.844+0000 7f35cd7fa640 1 -- 172.21.15.148:0/188102046 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35c0058640 con 0x7f35d00cadb0 2024-04-25T09:55:08.847 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35bc06b4d0 msgr2=0x7f35bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35bc06b4d0 0x7f35bc06d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f35c4034000 tx=0x7f35c402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 msgr2=0x7f35d0155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d0155240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c430 tx=0x7f35c000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 shutdown_connections 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35bc06b4d0 0x7f35bc06d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d015de80 0x7f35d00b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d00ccfe0 0x7f35d00b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 --2- 172.21.15.148:0/188102046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d00cadb0 0x7f35d0155240 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 >> 172.21.15.148:0/188102046 conn(0x7f35d00b2390 msgr2=0x7f35d015c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 shutdown_connections 2024-04-25T09:55:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.846+0000 7f35d719e640 1 -- 172.21.15.148:0/188102046 wait complete. 2024-04-25T09:55:08.860 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:08.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.958+0000 7f82eb59e640 1 Processor -- start 2024-04-25T09:55:08.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.958+0000 7f82eb59e640 1 -- start start 2024-04-25T09:55:08.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.959+0000 7f82eb59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.959+0000 7f82eb59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f82ec0b4740 0x7f82ec0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.959+0000 7f82eb59e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ec06f830 con 0x7f82ec06f340 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.959+0000 7f82eb59e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ec070440 con 0x7f82ec0b4740 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.959+0000 7f82eb59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec06fb10 con 0x7f82ec15a7a0 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.959+0000 7f82ead9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f82ec0b4740 0x7f82ec0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82e9d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f82ec06f340 0x7f82ec0b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43178/0 (socket says 172.21.15.148:43178) 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82e9d9b640 1 -- 172.21.15.148:0/1442115508 learned_addr learned my addr 172.21.15.148:0/1442115508 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82ea59c640 1 --2- 172.21.15.148:0/1442115508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82ead9d640 1 -- 172.21.15.148:0/1442115508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 msgr2=0x7f82ec06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82ead9d640 1 --2- 172.21.15.148:0/1442115508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82ead9d640 1 -- 172.21.15.148:0/1442115508 >> v1:172.21.15.148:6789/0 conn(0x7f82ec06f340 legacy=0x7f82ec0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82ead9d640 1 -- 172.21.15.148:0/1442115508 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec06fdf0 con 0x7f82ec0b4740 2024-04-25T09:55:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.960+0000 7f82e959a640 1 -- 172.21.15.148:0/1442115508 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369024256 0 0) 0x7f82ec06f830 con 0x7f82ec06f340 2024-04-25T09:55:08.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82ead9d640 1 --2- 172.21.15.148:0/1442115508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec0b4740 0x7f82ec0ae060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82e40097a0 tx=0x7f82e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cea7aedb8bbaf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82e959a640 1 -- 172.21.15.148:0/1442115508 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4037070 con 0x7f82ec0b4740 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82e959a640 1 -- 172.21.15.148:0/1442115508 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4032440 con 0x7f82ec0b4740 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82e959a640 1 -- 172.21.15.148:0/1442115508 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4032810 con 0x7f82ec0b4740 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 -- 172.21.15.148:0/1442115508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec0b4740 msgr2=0x7f82ec0ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1442115508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec0b4740 0x7f82ec0ae060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82e40097a0 tx=0x7f82e402a180 comp rx=0 tx=0).stop 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 -- 172.21.15.148:0/1442115508 shutdown_connections 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1442115508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec0b4740 0x7f82ec0ae060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1442115508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 -- 172.21.15.148:0/1442115508 >> 172.21.15.148:0/1442115508 conn(0x7f82ec0b5a40 msgr2=0x7f82ec0b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 -- 172.21.15.148:0/1442115508 shutdown_connections 2024-04-25T09:55:08.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.961+0000 7f82eb59e640 1 -- 172.21.15.148:0/1442115508 wait complete. 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 Processor -- start 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 -- start start 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 0x7f82ec1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82ec0b4740 0x7f82ec0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec0b36b0 con 0x7f82ec06f340 2024-04-25T09:55:08.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec15ab80 con 0x7f82ec0b4740 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.963+0000 7f82eb59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec0b3830 con 0x7f82ec15a7a0 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 0x7f82ec1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 0x7f82ec1505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51480/0 (socket says 172.21.15.148:51480) 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 -- 172.21.15.148:0/1617346126 learned_addr learned my addr 172.21.15.148:0/1617346126 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 -- 172.21.15.148:0/1617346126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 msgr2=0x7f82ec0b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 -- 172.21.15.148:0/1617346126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82ec0b4740 msgr2=0x7f82ec0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82ec0b4740 0x7f82ec0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 -- 172.21.15.148:0/1617346126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec151020 con 0x7f82ec06f340 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82ea59c640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 0x7f82ec1505a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82dc007e40 tx=0x7f82dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc014620 con 0x7f82ec06f340 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc019070 con 0x7f82ec06f340 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec14e210 con 0x7f82ec06f340 2024-04-25T09:55:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0149e0 con 0x7f82ec06f340 2024-04-25T09:55:08.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.964+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec150ae0 con 0x7f82ec06f340 2024-04-25T09:55:08.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.965+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ec150d70 con 0x7f82ec06f340 2024-04-25T09:55:08.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.966+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82dc014b80 con 0x7f82ec06f340 2024-04-25T09:55:08.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.966+0000 7f82daffd640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f82cc06b4d0 0x7f82cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:08.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.966+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f82dc08af70 con 0x7f82ec06f340 2024-04-25T09:55:08.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.966+0000 7f82e9d9b640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f82cc06b4d0 0x7f82cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:08.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.966+0000 7f82e9d9b640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f82cc06b4d0 0x7f82cc06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82e00011e0 tx=0x7f82e0007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:08.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:08.970+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82dc059950 con 0x7f82ec06f340 2024-04-25T09:55:09.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.139+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f82ec073990 con 0x7f82ec06f340 2024-04-25T09:55:09.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.140+0000 7f82daffd640 1 -- 172.21.15.148:0/1617346126 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f82dc059250 con 0x7f82ec06f340 2024-04-25T09:55:09.143 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:09.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f82cc06b4d0 msgr2=0x7f82cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f82cc06b4d0 0x7f82cc06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82e00011e0 tx=0x7f82e0007920 comp rx=0 tx=0).stop 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 msgr2=0x7f82ec1505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 0x7f82ec1505a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82dc007e40 tx=0x7f82dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 shutdown_connections 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f82cc06b4d0 0x7f82cc06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82ec15a7a0 0x7f82ec0b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82ec0b4740 0x7f82ec0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 --2- 172.21.15.148:0/1617346126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82ec06f340 0x7f82ec1505a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 >> 172.21.15.148:0/1617346126 conn(0x7f82ec0b5a40 msgr2=0x7f82ec158450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 shutdown_connections 2024-04-25T09:55:09.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.142+0000 7f82eb59e640 1 -- 172.21.15.148:0/1617346126 wait complete. 2024-04-25T09:55:09.165 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T09:55:09.165 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:55:09.166 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:09.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.261+0000 7f297964a640 1 Processor -- start 2024-04-25T09:55:09.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.261+0000 7f297964a640 1 -- start start 2024-04-25T09:55:09.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.262+0000 7f297964a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 0x7f29740cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.262+0000 7f297964a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297406abe0 con 0x7f297415dce0 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.262+0000 7f297964a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29740ccc50 con 0x7f29740af970 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.262+0000 7f297964a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297406b4f0 con 0x7f29740d0010 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.262+0000 7f2972ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f297415dce0 0x7f29740cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43188/0 (socket says 172.21.15.148:43188) 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.262+0000 7f2972ffd640 1 -- 172.21.15.148:0/1003855736 learned_addr learned my addr 172.21.15.148:0/1003855736 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 0x7f29740cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376461394 0 0) 0x7f297406abe0 con 0x7f297415dce0 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f295c003520 con 0x7f297415dce0 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 704400021 0 0) 0x7f295c003520 con 0x7f297415dce0 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 msgr2=0x7f29740cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 0x7f29740cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 >> v1:172.21.15.188:6789/0 conn(0x7f29740af970 legacy=0x7f29740ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297406aea0 con 0x7f297415dce0 2024-04-25T09:55:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f29640019f0 con 0x7f297415dce0 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 >> v1:172.21.15.148:6789/0 conn(0x7f297415dce0 legacy=0x7f29740cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 0x7f295c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f295c003f60 0x7f295c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f295c004d40 0x7f295c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297406b4f0 con 0x7f295c004d40 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c00bb50 con 0x7f295c003f60 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c00bd50 con 0x7f29740d0010 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f295c004d40 0x7f295c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f2972ffd640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 0x7f295c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29727fc640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f295c003f60 0x7f295c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 -- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 msgr2=0x7f295c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29740d0010 0x7f295c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 -- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f295c003f60 msgr2=0x7f295c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f295c003f60 0x7f295c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.263+0000 7f29737fe640 1 -- 172.21.15.148:0/1003855736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c003520 con 0x7f295c004d40 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.264+0000 7f29737fe640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f295c004d40 0x7f295c0051c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f296c009ac0 tx=0x7f295c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5351c473ed2b8198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.264+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c036070 con 0x7f295c004d40 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.264+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c031400 con 0x7f295c004d40 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.264+0000 7f2971ffb640 1 -- 172.21.15.148:0/1003855736 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c009690 con 0x7f295c004d40 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f297964a640 1 -- 172.21.15.148:0/1003855736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f295c004d40 msgr2=0x7f295c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f297964a640 1 --2- 172.21.15.148:0/1003855736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f295c004d40 0x7f295c0051c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f296c009ac0 tx=0x7f295c005830 comp rx=0 tx=0).stop 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f29737fe640 1 -- 172.21.15.148:0/1003855736 reap_dead start 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f297964a640 1 -- 172.21.15.148:0/1003855736 shutdown_connections 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f297964a640 1 -- 172.21.15.148:0/1003855736 >> 172.21.15.148:0/1003855736 conn(0x7f29740b2290 msgr2=0x7f29740b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:09.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f297964a640 1 -- 172.21.15.148:0/1003855736 shutdown_connections 2024-04-25T09:55:09.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.265+0000 7f297964a640 1 -- 172.21.15.148:0/1003855736 wait complete. 2024-04-25T09:55:09.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.266+0000 7f297964a640 1 Processor -- start 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.266+0000 7f297964a640 1 -- start start 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f297964a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 0x7f2974146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f297964a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2974145e40 0x7f297415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f297964a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974147e90 0x7f297415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f297964a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297415e0c0 con 0x7f2974145a60 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f297964a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297406b980 con 0x7f2974147e90 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f297964a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2974146df0 con 0x7f2974145e40 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 0x7f2974146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 0x7f2974146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51494/0 (socket says 172.21.15.148:51494) 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 -- 172.21.15.148:0/1177626267 learned_addr learned my addr 172.21.15.148:0/1177626267 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f29727fc640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2974145e40 0x7f297415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f29737fe640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974147e90 0x7f297415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 -- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2974145e40 msgr2=0x7f297415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2974145e40 0x7f297415c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 -- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974147e90 msgr2=0x7f297415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974147e90 0x7f297415c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 -- 172.21.15.148:0/1177626267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29741470d0 con 0x7f2974145a60 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f29727fc640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2974145e40 0x7f297415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f29737fe640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974147e90 0x7f297415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.267+0000 7f2972ffd640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 0x7f2974146330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2964001af0 tx=0x7f2964001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.268+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964017020 con 0x7f2974145a60 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.268+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2964013600 con 0x7f2974145a60 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.268+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29741473b0 con 0x7f2974145a60 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.268+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29640139d0 con 0x7f2974145a60 2024-04-25T09:55:09.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.268+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2974147970 con 0x7f2974145a60 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.268+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297406cc80 con 0x7f2974145a60 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.269+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f296401a020 con 0x7f2974145a60 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.269+0000 7f295b7fe640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296006b4d0 0x7f296006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.269+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f296408b560 con 0x7f2974145a60 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.269+0000 7f29727fc640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296006b4d0 0x7f296006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.270+0000 7f29727fc640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296006b4d0 0x7f296006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f297415cf60 tx=0x7f2968009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.274+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2964021020 con 0x7f2974145a60 2024-04-25T09:55:09.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.445+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2974073990 con 0x7f2974145a60 2024-04-25T09:55:09.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.446+0000 7f295b7fe640 1 -- 172.21.15.148:0/1177626267 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7f2964059fd0 con 0x7f2974145a60 2024-04-25T09:55:09.449 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:09.449 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296006b4d0 msgr2=0x7f296006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296006b4d0 0x7f296006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f297415cf60 tx=0x7f2968009040 comp rx=0 tx=0).stop 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 msgr2=0x7f2974146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 0x7f2974146330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2964001af0 tx=0x7f2964001b20 comp rx=0 tx=0).stop 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 shutdown_connections 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296006b4d0 0x7f296006d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2974147e90 0x7f297415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2974145e40 0x7f297415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 --2- 172.21.15.148:0/1177626267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2974145a60 0x7f2974146330 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 >> 172.21.15.148:0/1177626267 conn(0x7f29740b2290 msgr2=0x7f29740cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 shutdown_connections 2024-04-25T09:55:09.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.448+0000 7f297964a640 1 -- 172.21.15.148:0/1177626267 wait complete. 2024-04-25T09:55:09.462 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:09.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:09.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.559+0000 7f9c27a1f640 1 Processor -- start 2024-04-25T09:55:09.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.560+0000 7f9c27a1f640 1 -- start start 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.560+0000 7f9c27a1f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c200ce230 0x7f9c200cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.560+0000 7f9c27a1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c2015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.560+0000 7f9c27a1f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c20072280 con 0x7f9c200cc4f0 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.560+0000 7f9c27a1f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c20071270 con 0x7f9c200ce230 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.560+0000 7f9c27a1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200714c0 con 0x7f9c2015b270 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c24fba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c200ce230 0x7f9c200cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c257bb640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9c200cc4f0 0x7f9c200cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43194/0 (socket says 172.21.15.148:43194) 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c257bb640 1 -- 172.21.15.148:0/912937128 learned_addr learned my addr 172.21.15.148:0/912937128 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c25fbc640 1 --2- 172.21.15.148:0/912937128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c2015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c24fba640 1 -- 172.21.15.148:0/912937128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 msgr2=0x7f9c2015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c24fba640 1 --2- 172.21.15.148:0/912937128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c2015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c24fba640 1 -- 172.21.15.148:0/912937128 >> v1:172.21.15.148:6789/0 conn(0x7f9c200cc4f0 legacy=0x7f9c200cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c17fff640 1 -- 172.21.15.148:0/912937128 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260991270 0 0) 0x7f9c20072280 con 0x7f9c200cc4f0 2024-04-25T09:55:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.561+0000 7f9c24fba640 1 -- 172.21.15.148:0/912937128 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c200717a0 con 0x7f9c200ce230 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c24fba640 1 --2- 172.21.15.148:0/912937128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c200cc8d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9c100097a0 tx=0x7f9c1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b004c5ad562e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c17fff640 1 -- 172.21.15.148:0/912937128 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10037070 con 0x7f9c200ce230 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c17fff640 1 -- 172.21.15.148:0/912937128 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10032440 con 0x7f9c200ce230 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c17fff640 1 -- 172.21.15.148:0/912937128 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10032810 con 0x7f9c200ce230 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 -- 172.21.15.148:0/912937128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 msgr2=0x7f9c200cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/912937128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c200cc8d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9c100097a0 tx=0x7f9c1002a180 comp rx=0 tx=0).stop 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 -- 172.21.15.148:0/912937128 shutdown_connections 2024-04-25T09:55:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/912937128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c2015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/912937128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c200cc8d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 -- 172.21.15.148:0/912937128 >> 172.21.15.148:0/912937128 conn(0x7f9c2006abe0 msgr2=0x7f9c2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 -- 172.21.15.148:0/912937128 shutdown_connections 2024-04-25T09:55:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.562+0000 7f9c27a1f640 1 -- 172.21.15.148:0/912937128 wait complete. 2024-04-25T09:55:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 Processor -- start 2024-04-25T09:55:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 -- start start 2024-04-25T09:55:09.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200cc4f0 0x7f9c2014c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c2013a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c20155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2015bbf0 con 0x7f9c200ce230 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20071d60 con 0x7f9c200cc4f0 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c27a1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2015bd70 con 0x7f9c2015b270 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c24fba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c2013a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c24fba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c2013a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51510/0 (socket says 172.21.15.148:51510) 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.564+0000 7f9c24fba640 1 -- 172.21.15.148:0/3080758022 learned_addr learned my addr 172.21.15.148:0/3080758022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c24fba640 1 -- 172.21.15.148:0/3080758022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 msgr2=0x7f9c20155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c24fba640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c20155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c24fba640 1 -- 172.21.15.148:0/3080758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200cc4f0 msgr2=0x7f9c2014c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c24fba640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200cc4f0 0x7f9c2014c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c24fba640 1 -- 172.21.15.148:0/3080758022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20155ad0 con 0x7f9c200ce230 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c24fba640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c2013a370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098c0 tx=0x7f9c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10037040 con 0x7f9c200ce230 2024-04-25T09:55:09.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10042070 con 0x7f9c200ce230 2024-04-25T09:55:09.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2014cca0 con 0x7f9c200ce230 2024-04-25T09:55:09.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10030bb0 con 0x7f9c200ce230 2024-04-25T09:55:09.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.565+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2014d260 con 0x7f9c200ce230 2024-04-25T09:55:09.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.566+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c20073a00 con 0x7f9c200ce230 2024-04-25T09:55:09.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.567+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c10047460 con 0x7f9c200ce230 2024-04-25T09:55:09.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.567+0000 7f9c15ffb640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0406b4d0 0x7f9c0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.567+0000 7f9c257bb640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0406b4d0 0x7f9c0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.567+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f9c100ab180 con 0x7f9c200ce230 2024-04-25T09:55:09.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.568+0000 7f9c257bb640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0406b4d0 0x7f9c0406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c007fd0 tx=0x7f9c1c007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.571+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c10032e20 con 0x7f9c200ce230 2024-04-25T09:55:09.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.739+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c2006d3a0 con 0x7f9c200ce230 2024-04-25T09:55:09.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.740+0000 7f9c15ffb640 1 -- 172.21.15.148:0/3080758022 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c10079c80 con 0x7f9c200ce230 2024-04-25T09:55:09.743 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0406b4d0 msgr2=0x7f9c0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0406b4d0 0x7f9c0406d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c007fd0 tx=0x7f9c1c007960 comp rx=0 tx=0).stop 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 msgr2=0x7f9c2013a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c2013a370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098c0 tx=0x7f9c100046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 shutdown_connections 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0406b4d0 0x7f9c0406d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2015b270 0x7f9c20155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c200ce230 0x7f9c2013a370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 --2- 172.21.15.148:0/3080758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c200cc4f0 0x7f9c2014c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 >> 172.21.15.148:0/3080758022 conn(0x7f9c2006abe0 msgr2=0x7f9c200b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:09.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 shutdown_connections 2024-04-25T09:55:09.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.742+0000 7f9c27a1f640 1 -- 172.21.15.148:0/3080758022 wait complete. 2024-04-25T09:55:09.756 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:09.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.853+0000 7f0a61d9d640 1 Processor -- start 2024-04-25T09:55:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.853+0000 7f0a61d9d640 1 -- start start 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a61d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c158230 0x7f0a5c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a61d9d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c0d2020 con 0x7f0a5c0b4e60 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a61d9d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c0d2300 con 0x7f0a5c0d1b80 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a61d9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c158610 con 0x7f0a5c158230 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0a5c0b4e60 0x7f0a5c06f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43198/0 (socket says 172.21.15.148:43198) 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5bfff640 1 -- 172.21.15.148:0/2412489680 learned_addr learned my addr 172.21.15.148:0/2412489680 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5b7fe640 1 --2- 172.21.15.148:0/2412489680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c158230 0x7f0a5c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5a7fc640 1 -- 172.21.15.148:0/2412489680 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316665971 0 0) 0x7f0a5c0d2020 con 0x7f0a5c0b4e60 2024-04-25T09:55:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5a7fc640 1 -- 172.21.15.148:0/2412489680 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a44003520 con 0x7f0a5c0b4e60 2024-04-25T09:55:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5b7fe640 1 -- 172.21.15.148:0/2412489680 >> v1:172.21.15.188:6789/0 conn(0x7f0a5c0d1b80 legacy=0x7f0a5c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5a7fc640 1 -- 172.21.15.148:0/2412489680 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731286631 0 0) 0x7f0a5c0d2300 con 0x7f0a5c0d1b80 2024-04-25T09:55:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5b7fe640 1 -- 172.21.15.148:0/2412489680 >> v1:172.21.15.148:6789/0 conn(0x7f0a5c0b4e60 legacy=0x7f0a5c06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.854+0000 7f0a5b7fe640 1 -- 172.21.15.148:0/2412489680 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c06bb30 con 0x7f0a5c158230 2024-04-25T09:55:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.855+0000 7f0a5b7fe640 1 --2- 172.21.15.148:0/2412489680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c158230 0x7f0a5c0d1640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0097d0 tx=0x7f0a4c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c24e7afa53a77cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.855+0000 7f0a5a7fc640 1 -- 172.21.15.148:0/2412489680 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c037070 con 0x7f0a5c158230 2024-04-25T09:55:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.855+0000 7f0a5a7fc640 1 -- 172.21.15.148:0/2412489680 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c032440 con 0x7f0a5c158230 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.855+0000 7f0a5a7fc640 1 -- 172.21.15.148:0/2412489680 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c032810 con 0x7f0a5c158230 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 -- 172.21.15.148:0/2412489680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c158230 msgr2=0x7f0a5c0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/2412489680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c158230 0x7f0a5c0d1640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0097d0 tx=0x7f0a4c02a210 comp rx=0 tx=0).stop 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 -- 172.21.15.148:0/2412489680 shutdown_connections 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/2412489680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c158230 0x7f0a5c0d1640 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 -- 172.21.15.148:0/2412489680 >> 172.21.15.148:0/2412489680 conn(0x7f0a5c061600 msgr2=0x7f0a5c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 -- 172.21.15.148:0/2412489680 shutdown_connections 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.856+0000 7f0a61d9d640 1 -- 172.21.15.148:0/2412489680 wait complete. 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.857+0000 7f0a61d9d640 1 Processor -- start 2024-04-25T09:55:09.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.857+0000 7f0a61d9d640 1 -- start start 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c0b4e60 0x7f0a5c146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a5c0d1b80 0x7f0a5c1f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 0x7f0a5c1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c06ca40 con 0x7f0a5c158230 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c06bea0 con 0x7f0a5c0d1b80 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c15e320 con 0x7f0a5c0b4e60 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 0x7f0a5c1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 0x7f0a5c1f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51520/0 (socket says 172.21.15.148:51520) 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 -- 172.21.15.148:0/1319656145 learned_addr learned my addr 172.21.15.148:0/1319656145 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5affd640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a5c0d1b80 0x7f0a5c1f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5b7fe640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c0b4e60 0x7f0a5c146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 -- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c0b4e60 msgr2=0x7f0a5c146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c0b4e60 0x7f0a5c146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 -- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a5c0d1b80 msgr2=0x7f0a5c1f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a5c0d1b80 0x7f0a5c1f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 -- 172.21.15.148:0/1319656145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c1594a0 con 0x7f0a5c158230 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5affd640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a5c0d1b80 0x7f0a5c1f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a5bfff640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 0x7f0a5c1f70c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400caa0 tx=0x7f0a5400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54017070 con 0x7f0a5c158230 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5400ac00 con 0x7f0a5c158230 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c159780 con 0x7f0a5c158230 2024-04-25T09:55:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.858+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54012400 con 0x7f0a5c158230 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.859+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c0cf210 con 0x7f0a5c158230 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.860+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c06de50 con 0x7f0a5c158230 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.860+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a5401a070 con 0x7f0a5c158230 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.860+0000 7f0a43fff640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a4806b500 0x7f0a4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.860+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f0a5408a310 con 0x7f0a5c158230 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.860+0000 7f0a5b7fe640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a4806b500 0x7f0a4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.861+0000 7f0a5b7fe640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a4806b500 0x7f0a4806d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c034000 tx=0x7f0a4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:09.864+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a54058730 con 0x7f0a5c158230 2024-04-25T09:55:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.034+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a5c073a00 con 0x7f0a5c158230 2024-04-25T09:55:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.034+0000 7f0a43fff640 1 -- 172.21.15.148:0/1319656145 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a5405c7d0 con 0x7f0a5c158230 2024-04-25T09:55:10.037 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a4806b500 msgr2=0x7f0a4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a4806b500 0x7f0a4806d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c034000 tx=0x7f0a4c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 msgr2=0x7f0a5c1f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 0x7f0a5c1f70c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400caa0 tx=0x7f0a5400cf60 comp rx=0 tx=0).stop 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 shutdown_connections 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a4806b500 0x7f0a4806d9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a5c158230 0x7f0a5c1f70c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a5c0d1b80 0x7f0a5c1f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 --2- 172.21.15.148:0/1319656145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a5c0b4e60 0x7f0a5c146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 >> 172.21.15.148:0/1319656145 conn(0x7f0a5c061600 msgr2=0x7f0a5c0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 shutdown_connections 2024-04-25T09:55:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.036+0000 7f0a61d9d640 1 -- 172.21.15.148:0/1319656145 wait complete. 2024-04-25T09:55:10.050 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:10.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.148+0000 7f3b08fc4640 1 Processor -- start 2024-04-25T09:55:10.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.148+0000 7f3b08fc4640 1 -- start start 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.149+0000 7f3b08fc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b040cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.149+0000 7f3b08fc4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b040cbf20 0x7f3b040cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.149+0000 7f3b08fc4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b040c75e0 con 0x7f3b0415bb50 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.149+0000 7f3b08fc4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b040c8680 con 0x7f3b040cbf20 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.149+0000 7f3b08fc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040cff80 con 0x7f3b040cf600 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.149+0000 7f3b0259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3b0415bb50 0x7f3b040c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43212/0 (socket says 172.21.15.148:43212) 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b0259c640 1 -- 172.21.15.148:0/1664817899 learned_addr learned my addr 172.21.15.148:0/1664817899 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b01d9b640 1 --2- 172.21.15.148:0/1664817899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b040cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b02d9d640 1 --2- 172.21.15.148:0/1664817899 >> v2:172.21.15.188:3300/0 conn(0x7f3b040cbf20 0x7f3b040cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b0159a640 1 -- 172.21.15.148:0/1664817899 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360677502 0 0) 0x7f3b040c75e0 con 0x7f3b0415bb50 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b0159a640 1 -- 172.21.15.148:0/1664817899 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aec003520 con 0x7f3b0415bb50 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b01d9b640 1 -- 172.21.15.148:0/1664817899 >> v2:172.21.15.188:3300/0 conn(0x7f3b040cbf20 msgr2=0x7f3b040cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b01d9b640 1 --2- 172.21.15.148:0/1664817899 >> v2:172.21.15.188:3300/0 conn(0x7f3b040cbf20 0x7f3b040cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b0159a640 1 -- 172.21.15.148:0/1664817899 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199955688 0 0) 0x7f3aec003520 con 0x7f3b0415bb50 2024-04-25T09:55:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b01d9b640 1 -- 172.21.15.148:0/1664817899 >> v1:172.21.15.148:6789/0 conn(0x7f3b0415bb50 legacy=0x7f3b040c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.150+0000 7f3b01d9b640 1 -- 172.21.15.148:0/1664817899 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040c78c0 con 0x7f3b040cf600 2024-04-25T09:55:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.151+0000 7f3b01d9b640 1 --2- 172.21.15.148:0/1664817899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b040cfa40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3af80097a0 tx=0x7f3af802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=103792f82045b5f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.151+0000 7f3b0159a640 1 -- 172.21.15.148:0/1664817899 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8037070 con 0x7f3b040cf600 2024-04-25T09:55:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.151+0000 7f3b0159a640 1 -- 172.21.15.148:0/1664817899 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af802ac30 con 0x7f3b040cf600 2024-04-25T09:55:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.151+0000 7f3b0159a640 1 -- 172.21.15.148:0/1664817899 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40019f0 con 0x7f3b040cf600 2024-04-25T09:55:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.151+0000 7f3b08fc4640 1 -- 172.21.15.148:0/1664817899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 msgr2=0x7f3b040cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.151+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/1664817899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b040cfa40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3af80097a0 tx=0x7f3af802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.152+0000 7f3b08fc4640 1 -- 172.21.15.148:0/1664817899 shutdown_connections 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.152+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/1664817899 >> v2:172.21.15.188:3300/0 conn(0x7f3b040cbf20 0x7f3b040cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.152+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/1664817899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b040cfa40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.152+0000 7f3b08fc4640 1 -- 172.21.15.148:0/1664817899 >> 172.21.15.148:0/1664817899 conn(0x7f3b0406f500 msgr2=0x7f3b0406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.152+0000 7f3b08fc4640 1 -- 172.21.15.148:0/1664817899 shutdown_connections 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.152+0000 7f3b08fc4640 1 -- 172.21.15.148:0/1664817899 wait complete. 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.153+0000 7f3b08fc4640 1 Processor -- start 2024-04-25T09:55:10.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.153+0000 7f3b08fc4640 1 -- start start 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b040cbf20 0x7f3b04148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b04159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 0x7f3b04159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040c8160 con 0x7f3b0415bb50 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040c7e80 con 0x7f3b040cbf20 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040c72a0 con 0x7f3b040cf600 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 0x7f3b04159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 0x7f3b04159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51534/0 (socket says 172.21.15.148:51534) 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 -- 172.21.15.148:0/936992975 learned_addr learned my addr 172.21.15.148:0/936992975 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 -- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 msgr2=0x7f3b04159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b01d9b640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b04159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b0259c640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b040cbf20 0x7f3b04148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b04159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 -- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b040cbf20 msgr2=0x7f3b04148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b040cbf20 0x7f3b04148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 -- 172.21.15.148:0/936992975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040b5530 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b01d9b640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b04159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b0259c640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b040cbf20 0x7f3b04148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b02d9d640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 0x7f3b04159a60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00e960 tx=0x7f3afc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc017070 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00c980 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b040b4770 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00cd70 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.154+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b040b4d30 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.156+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b04073a00 con 0x7f3b0415bb50 2024-04-25T09:55:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.156+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3afc01a070 con 0x7f3b0415bb50 2024-04-25T09:55:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.156+0000 7f3af2ffd640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.156+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f3afc08a580 con 0x7f3b0415bb50 2024-04-25T09:55:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.156+0000 7f3b0259c640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.157+0000 7f3b0259c640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ae406b4d0 0x7f3ae406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3af4002230 tx=0x7f3af4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.160+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3afc059080 con 0x7f3b0415bb50 2024-04-25T09:55:10.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.331+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b040b2f70 con 0x7f3b0415bb50 2024-04-25T09:55:10.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.331+0000 7f3af2ffd640 1 -- 172.21.15.148:0/936992975 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3afc058980 con 0x7f3b0415bb50 2024-04-25T09:55:10.335 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ae406b4d0 msgr2=0x7f3ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ae406b4d0 0x7f3ae406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3af4002230 tx=0x7f3af4009040 comp rx=0 tx=0).stop 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 msgr2=0x7f3b04159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 0x7f3b04159a60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00e960 tx=0x7f3afc00ee20 comp rx=0 tx=0).stop 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 shutdown_connections 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b0415bb50 0x7f3b04159a60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b040cf600 0x7f3b04159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 --2- 172.21.15.148:0/936992975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b040cbf20 0x7f3b04148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 >> 172.21.15.148:0/936992975 conn(0x7f3b0406f500 msgr2=0x7f3b040c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 shutdown_connections 2024-04-25T09:55:10.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.333+0000 7f3b08fc4640 1 -- 172.21.15.148:0/936992975 wait complete. 2024-04-25T09:55:10.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:10.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.443+0000 7f797022a640 1 Processor -- start 2024-04-25T09:55:10.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.443+0000 7f797022a640 1 -- start start 2024-04-25T09:55:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f797022a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f796806c330 0x7f79680b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f797022a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680d1d50 con 0x7f796806c7e0 2024-04-25T09:55:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f797022a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680d2020 con 0x7f7968158230 2024-04-25T09:55:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f797022a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7968158610 con 0x7f796806c330 2024-04-25T09:55:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f796e7c7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f796806c7e0 0x7f796806f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43222/0 (socket says 172.21.15.148:43222) 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f796e7c7640 1 -- 172.21.15.148:0/2896888006 learned_addr learned my addr 172.21.15.148:0/2896888006 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f796d7c5640 1 --2- 172.21.15.148:0/2896888006 >> v2:172.21.15.188:3300/0 conn(0x7f796806c330 0x7f79680b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f796cfc4640 1 -- 172.21.15.148:0/2896888006 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475098950 0 0) 0x7f79680d1d50 con 0x7f796806c7e0 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.444+0000 7f796cfc4640 1 -- 172.21.15.148:0/2896888006 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7954003520 con 0x7f796806c7e0 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.445+0000 7f796d7c5640 1 -- 172.21.15.148:0/2896888006 >> v1:172.21.15.188:6789/0 conn(0x7f7968158230 legacy=0x7f79680d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.445+0000 7f796d7c5640 1 -- 172.21.15.148:0/2896888006 >> v1:172.21.15.148:6789/0 conn(0x7f796806c7e0 legacy=0x7f796806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.445+0000 7f796cfc4640 1 -- 172.21.15.148:0/2896888006 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049426507 0 0) 0x7f7954003520 con 0x7f796806c7e0 2024-04-25T09:55:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.445+0000 7f796d7c5640 1 -- 172.21.15.148:0/2896888006 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680d2300 con 0x7f796806c330 2024-04-25T09:55:10.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.445+0000 7f796d7c5640 1 --2- 172.21.15.148:0/2896888006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f79680b4e60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7958009790 tx=0x7f795802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333f6d2dc534d12a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f796cfc4640 1 -- 172.21.15.148:0/2896888006 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958037070 con 0x7f796806c330 2024-04-25T09:55:10.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f796cfc4640 1 -- 172.21.15.148:0/2896888006 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795802ac30 con 0x7f796806c330 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f796cfc4640 1 -- 172.21.15.148:0/2896888006 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960001670 con 0x7f796806c330 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 -- 172.21.15.148:0/2896888006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 msgr2=0x7f79680b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 --2- 172.21.15.148:0/2896888006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f79680b4e60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7958009790 tx=0x7f795802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 -- 172.21.15.148:0/2896888006 shutdown_connections 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 --2- 172.21.15.148:0/2896888006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f79680b4e60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 -- 172.21.15.148:0/2896888006 >> 172.21.15.148:0/2896888006 conn(0x7f7968061600 msgr2=0x7f79680619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 -- 172.21.15.148:0/2896888006 shutdown_connections 2024-04-25T09:55:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.446+0000 7f797022a640 1 -- 172.21.15.148:0/2896888006 wait complete. 2024-04-25T09:55:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.447+0000 7f797022a640 1 Processor -- start 2024-04-25T09:55:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.447+0000 7f797022a640 1 -- start start 2024-04-25T09:55:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f797022a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f796814a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f797022a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 0x7f79680cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f797022a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968158230 0x7f796815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f797022a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680d25e0 con 0x7f796806c7e0 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f797022a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796806bea0 con 0x7f796806c330 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f797022a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796815e320 con 0x7f7968158230 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 0x7f79680cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 0x7f79680cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51540/0 (socket says 172.21.15.148:51540) 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 -- 172.21.15.148:0/2248140930 learned_addr learned my addr 172.21.15.148:0/2248140930 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796dfc6640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f796814a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796e7c7640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968158230 0x7f796815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 -- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968158230 msgr2=0x7f796815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968158230 0x7f796815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 -- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 msgr2=0x7f796814a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f796814a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.448+0000 7f796d7c5640 1 -- 172.21.15.148:0/2248140930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680cc060 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f796dfc6640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f796814a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f796d7c5640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 0x7f79680cd570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79580098b0 tx=0x7f79580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958042020 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958003c40 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79680cc340 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958037040 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.449+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79680ccd70 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.450+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968073a00 con 0x7f796806c7e0 2024-04-25T09:55:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.450+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7958045020 con 0x7f796806c7e0 2024-04-25T09:55:10.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.451+0000 7f795e7fc640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f794c06b4d0 0x7f794c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.451+0000 7f796dfc6640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f794c06b4d0 0x7f794c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.451+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f79580aad70 con 0x7f796806c7e0 2024-04-25T09:55:10.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.451+0000 7f796dfc6640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f794c06b4d0 0x7f794c06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7968158f20 tx=0x7f7964009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.455+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f795804c030 con 0x7f796806c7e0 2024-04-25T09:55:10.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.624+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f796806e530 con 0x7f796806c7e0 2024-04-25T09:55:10.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.624+0000 7f795e7fc640 1 -- 172.21.15.148:0/2248140930 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7958079900 con 0x7f796806c7e0 2024-04-25T09:55:10.627 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f794c06b4d0 msgr2=0x7f794c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f794c06b4d0 0x7f794c06d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7968158f20 tx=0x7f7964009040 comp rx=0 tx=0).stop 2024-04-25T09:55:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 msgr2=0x7f79680cd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 0x7f79680cd570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79580098b0 tx=0x7f79580046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 shutdown_connections 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f794c06b4d0 0x7f794c06d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7968158230 0x7f796815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f796806c7e0 0x7f79680cd570 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 --2- 172.21.15.148:0/2248140930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f796806c330 0x7f796814a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 >> 172.21.15.148:0/2248140930 conn(0x7f7968061600 msgr2=0x7f79680b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 shutdown_connections 2024-04-25T09:55:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.626+0000 7f797022a640 1 -- 172.21.15.148:0/2248140930 wait complete. 2024-04-25T09:55:10.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:55:10.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.734+0000 7f47af9d5640 1 Processor -- start 2024-04-25T09:55:10.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.735+0000 7f47af9d5640 1 -- start start 2024-04-25T09:55:10.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.735+0000 7f47af9d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47a8072d30 0x7f47a806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.735+0000 7f47af9d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a8073150 con 0x7f47a806abe0 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.735+0000 7f47af9d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80cfa60 con 0x7f47a815b9c0 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.735+0000 7f47af9d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a80cfd40 con 0x7f47a8072d30 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f47acf70640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f47a806abe0 0x7f47a8072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43226/0 (socket says 172.21.15.148:43226) 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f47acf70640 1 -- 172.21.15.148:0/3511653010 learned_addr learned my addr 172.21.15.148:0/3511653010 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f47adf72640 1 --2- 172.21.15.148:0/3511653010 >> v2:172.21.15.188:3300/0 conn(0x7f47a8072d30 0x7f47a806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111836414 0 0) 0x7f47a8073150 con 0x7f47a806abe0 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4794003520 con 0x7f47a806abe0 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1690833049 0 0) 0x7f4794003520 con 0x7f47a806abe0 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 >> v2:172.21.15.188:3300/0 conn(0x7f47a8072d30 msgr2=0x7f47a806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 --2- 172.21.15.148:0/3511653010 >> v2:172.21.15.188:3300/0 conn(0x7f47a8072d30 0x7f47a806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 >> v1:172.21.15.188:6789/0 conn(0x7f47a815b9c0 legacy=0x7f47a815bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80d0020 con 0x7f47a806abe0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f47a4001670 con 0x7f47a806abe0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 >> v1:172.21.15.148:6789/0 conn(0x7f47a806abe0 legacy=0x7f47a8072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47940039b0 0x7f4794003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4794004800 0x7f4794004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940051c0 0x7f479400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80cfd40 con 0x7f47940051c0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479400c060 con 0x7f4794004800 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479400c260 con 0x7f47940039b0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f47adf72640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940051c0 0x7f479400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f47acf70640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4794004800 0x7f4794004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.736+0000 7f47ad771640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47940039b0 0x7f4794003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 -- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47940039b0 msgr2=0x7f4794003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47940039b0 0x7f4794003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 -- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4794004800 msgr2=0x7f4794004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4794004800 0x7f4794004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 -- 172.21.15.148:0/3511653010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4794003520 con 0x7f47940051c0 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 -- 172.21.15.148:0/3511653010 reap_dead start 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f47adf72640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940051c0 0x7f479400b9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fc0 tx=0x7f4794005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6559938d22677df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0035020 con 0x7f47940051c0 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0003bd0 con 0x7f47940051c0 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.737+0000 7f479bfff640 1 -- 172.21.15.148:0/3511653010 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0009c10 con 0x7f47940051c0 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 -- 172.21.15.148:0/3511653010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940051c0 msgr2=0x7f479400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940051c0 0x7f479400b9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fc0 tx=0x7f4794005830 comp rx=0 tx=0).stop 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 -- 172.21.15.148:0/3511653010 shutdown_connections 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 --2- 172.21.15.148:0/3511653010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47940051c0 0x7f479400b9c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 -- 172.21.15.148:0/3511653010 >> 172.21.15.148:0/3511653010 conn(0x7f47a80b2370 msgr2=0x7f47a80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 -- 172.21.15.148:0/3511653010 shutdown_connections 2024-04-25T09:55:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.738+0000 7f47af9d5640 1 -- 172.21.15.148:0/3511653010 wait complete. 2024-04-25T09:55:10.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.739+0000 7f47af9d5640 1 Processor -- start 2024-04-25T09:55:10.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.739+0000 7f47af9d5640 1 -- start start 2024-04-25T09:55:10.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47af9d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47940051c0 0x7f47a8141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47af9d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 0x7f47a815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47af9d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47a8151860 0x7f47a8151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47af9d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a806b070 con 0x7f47a815cdf0 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47af9d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80cf710 con 0x7f47940051c0 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47af9d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a806b1f0 con 0x7f47a8151860 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 0x7f47a815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 0x7f47a815d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51560/0 (socket says 172.21.15.148:51560) 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 -- 172.21.15.148:0/1699501979 learned_addr learned my addr 172.21.15.148:0/1699501979 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47adf72640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47a8151860 0x7f47a8151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47ad771640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47940051c0 0x7f47a8141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 -- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47a8151860 msgr2=0x7f47a8151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47a8151860 0x7f47a8151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 -- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47940051c0 msgr2=0x7f47a8141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47940051c0 0x7f47a8141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.740+0000 7f47acf70640 1 -- 172.21.15.148:0/1699501979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a815d870 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.741+0000 7f47acf70640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 0x7f47a815d270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47a40072f0 tx=0x7f47a40077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.741+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a401a070 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.741+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a4007e00 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.741+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a815db50 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.741+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400fb00 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.741+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a815e110 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.742+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a8073990 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.742+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47a401d070 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.743+0000 7f4799ffb640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f478c06b4d0 0x7f478c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.743+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f47a4089df0 con 0x7f47a815cdf0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.743+0000 7f47ad771640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f478c06b4d0 0x7f478c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.743+0000 7f47ad771640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f478c06b4d0 0x7f478c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.747+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47a4058210 con 0x7f47a815cdf0 2024-04-25T09:55:10.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:10.985+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f47a80b30b0 con 0x7f47a815cdf0 2024-04-25T09:55:11.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.186+0000 7f4799ffb640 1 -- 172.21.15.148:0/1699501979 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v67) v1 ==== 151+0+0 (secure 0 0 0) 0x7f47a405c2b0 con 0x7f47a815cdf0 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f478c06b4d0 msgr2=0x7f478c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f478c06b4d0 0x7f478c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 msgr2=0x7f47a815d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 0x7f47a815d270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f47a40072f0 tx=0x7f47a40077b0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 shutdown_connections 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f478c06b4d0 0x7f478c06d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47a8151860 0x7f47a8151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a815cdf0 0x7f47a815d270 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 --2- 172.21.15.148:0/1699501979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47940051c0 0x7f47a8141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 >> 172.21.15.148:0/1699501979 conn(0x7f47a80b2370 msgr2=0x7f47a80b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 shutdown_connections 2024-04-25T09:55:11.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.188+0000 7f47af9d5640 1 -- 172.21.15.148:0/1699501979 wait complete. 2024-04-25T09:55:11.202 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.300+0000 7fa9eac6f640 1 Processor -- start 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.300+0000 7fa9eac6f640 1 -- start start 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9eac6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9e40b5620 0x7fa9e40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9eac6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e406c2d0 0x7fa9e406c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9eac6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e406cc80 con 0x7fa9e415b7d0 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9eac6f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e406cf50 con 0x7fa9e40b5620 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9eac6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40ce530 con 0x7fa9e406c2d0 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9e40b5620 0x7fa9e40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9e40b5620 0x7fa9e40b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51562/0 (socket says 172.21.15.148:51562) 2024-04-25T09:55:11.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 -- 172.21.15.148:0/486791172 learned_addr learned my addr 172.21.15.148:0/486791172 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:11.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e920c640 1 --2- 172.21.15.148:0/486791172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e406c2d0 0x7fa9e406c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 -- 172.21.15.148:0/486791172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e406c2d0 msgr2=0x7fa9e406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:11.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 --2- 172.21.15.148:0/486791172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e406c2d0 0x7fa9e406c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 -- 172.21.15.148:0/486791172 >> v1:172.21.15.188:6789/0 conn(0x7fa9e415b7d0 legacy=0x7fa9e40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:11.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.301+0000 7fa9e3fff640 1 -- 172.21.15.148:0/486791172 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40ce810 con 0x7fa9e40b5620 2024-04-25T09:55:11.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9e3fff640 1 --2- 172.21.15.148:0/486791172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40b5ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009870 tx=0x7fa9d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d261984441247997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:11.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9e37fe640 1 -- 172.21.15.148:0/486791172 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4037070 con 0x7fa9e40b5620 2024-04-25T09:55:11.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9e37fe640 1 -- 172.21.15.148:0/486791172 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4032440 con 0x7fa9e40b5620 2024-04-25T09:55:11.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9e37fe640 1 -- 172.21.15.148:0/486791172 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4032810 con 0x7fa9e40b5620 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9eac6f640 1 -- 172.21.15.148:0/486791172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 msgr2=0x7fa9e40b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/486791172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40b5ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009870 tx=0x7fa9d402a210 comp rx=0 tx=0).stop 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9eac6f640 1 -- 172.21.15.148:0/486791172 shutdown_connections 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/486791172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40b5ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.302+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/486791172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e406c2d0 0x7fa9e406c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.303+0000 7fa9eac6f640 1 -- 172.21.15.148:0/486791172 >> 172.21.15.148:0/486791172 conn(0x7fa9e40d0800 msgr2=0x7fa9e40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.303+0000 7fa9eac6f640 1 -- 172.21.15.148:0/486791172 shutdown_connections 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.303+0000 7fa9eac6f640 1 -- 172.21.15.148:0/486791172 wait complete. 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.304+0000 7fa9eac6f640 1 Processor -- start 2024-04-25T09:55:11.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.304+0000 7fa9eac6f640 1 -- start start 2024-04-25T09:55:11.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e406c2d0 0x7fa9e41f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e415b7d0 0x7fa9e40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e415bbb0 con 0x7fa9e40b5620 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e406d230 con 0x7fa9e406c2d0 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e415bd30 con 0x7fa9e415b7d0 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51564/0 (socket says 172.21.15.148:51564) 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 -- 172.21.15.148:0/1728090013 learned_addr learned my addr 172.21.15.148:0/1728090013 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 -- 172.21.15.148:0/1728090013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e415b7d0 msgr2=0x7fa9e40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e415b7d0 0x7fa9e40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 -- 172.21.15.148:0/1728090013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e406c2d0 msgr2=0x7fa9e41f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e406c2d0 0x7fa9e41f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 -- 172.21.15.148:0/1728090013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e41f3fc0 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e3fff640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40cd690 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009870 tx=0x7fa9d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4041020 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4030970 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4037040 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e41f42a0 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.305+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e40cd170 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.306+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e4073a00 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.307+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9d4044020 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.307+0000 7fa9e17fa640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.307+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fa9d40ab010 con 0x7fa9e40b5620 2024-04-25T09:55:11.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.307+0000 7fa9e8a0b640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.308+0000 7fa9e8a0b640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c406b4d0 0x7fa9c406d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00ab90 tx=0x7fa9dc009230 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:11.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.311+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9d404b030 con 0x7fa9e40b5620 2024-04-25T09:55:11.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.480+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa9e40ccc60 con 0x7fa9e40b5620 2024-04-25T09:55:11.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.484+0000 7fa9e17fa640 1 -- 172.21.15.148:0/1728090013 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9d4079a80 con 0x7fa9e40b5620 2024-04-25T09:55:11.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c406b4d0 msgr2=0x7fa9c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c406b4d0 0x7fa9c406d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00ab90 tx=0x7fa9dc009230 comp rx=0 tx=0).stop 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 msgr2=0x7fa9e40cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40cd690 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009870 tx=0x7fa9d40046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 shutdown_connections 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c406b4d0 0x7fa9c406d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e415b7d0 0x7fa9e40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9e40b5620 0x7fa9e40cd690 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 --2- 172.21.15.148:0/1728090013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e406c2d0 0x7fa9e41f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 >> 172.21.15.148:0/1728090013 conn(0x7fa9e40d0800 msgr2=0x7fa9e4158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 shutdown_connections 2024-04-25T09:55:11.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.486+0000 7fa9eac6f640 1 -- 172.21.15.148:0/1728090013 wait complete. 2024-04-25T09:55:11.499 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:55:11.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:11.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:55:11.584 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:55:11.585 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:11.585 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:11.600 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:11.600 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:11.655 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:11.655 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:11.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:11.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.844+0000 7f761bcbc640 1 Processor -- start 2024-04-25T09:55:11.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.844+0000 7f761bcbc640 1 -- start start 2024-04-25T09:55:11.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761bcbc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f761406b580 0x7f76140cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761bcbc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76140cffa0 0x7f761415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761bcbc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7614161960 con 0x7f761406b1a0 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761bcbc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76140cbd50 con 0x7f76140cffa0 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761bcbc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76140700d0 con 0x7f761406b580 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76140cffa0 0x7f761415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f76140cffa0 0x7f761415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51574/0 (socket says 172.21.15.148:51574) 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 -- 172.21.15.148:0/1675370245 learned_addr learned my addr 172.21.15.148:0/1675370245 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f7619257640 1 --2- 172.21.15.148:0/1675370245 >> v2:172.21.15.188:3300/0 conn(0x7f761406b580 0x7f76140cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 -- 172.21.15.148:0/1675370245 >> v2:172.21.15.188:3300/0 conn(0x7f761406b580 msgr2=0x7f76140cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 --2- 172.21.15.148:0/1675370245 >> v2:172.21.15.188:3300/0 conn(0x7f761406b580 0x7f76140cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 -- 172.21.15.148:0/1675370245 >> v1:172.21.15.188:6789/0 conn(0x7f761406b1a0 legacy=0x7f761406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f761a259640 1 -- 172.21.15.148:0/1675370245 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7614161c40 con 0x7f76140cffa0 2024-04-25T09:55:11.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.845+0000 7f7619257640 1 --2- 172.21.15.148:0/1675370245 >> v2:172.21.15.188:3300/0 conn(0x7f761406b580 0x7f76140cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:11.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761a259640 1 --2- 172.21.15.148:0/1675370245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cffa0 0x7f761415d370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76140cbed0 tx=0x7f760c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ef5f552cfb6836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:11.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f7618a56640 1 -- 172.21.15.148:0/1675370245 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c036070 con 0x7f76140cffa0 2024-04-25T09:55:11.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f7618a56640 1 -- 172.21.15.148:0/1675370245 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c031600 con 0x7f76140cffa0 2024-04-25T09:55:11.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f7618a56640 1 -- 172.21.15.148:0/1675370245 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c0319d0 con 0x7f76140cffa0 2024-04-25T09:55:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761bcbc640 1 -- 172.21.15.148:0/1675370245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cffa0 msgr2=0x7f761415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761bcbc640 1 --2- 172.21.15.148:0/1675370245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cffa0 0x7f761415d370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f76140cbed0 tx=0x7f760c02a640 comp rx=0 tx=0).stop 2024-04-25T09:55:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761bcbc640 1 -- 172.21.15.148:0/1675370245 shutdown_connections 2024-04-25T09:55:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761bcbc640 1 --2- 172.21.15.148:0/1675370245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76140cffa0 0x7f761415d370 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761bcbc640 1 --2- 172.21.15.148:0/1675370245 >> v2:172.21.15.188:3300/0 conn(0x7f761406b580 0x7f76140cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.846+0000 7f761bcbc640 1 -- 172.21.15.148:0/1675370245 >> 172.21.15.148:0/1675370245 conn(0x7f76140b2220 msgr2=0x7f76140b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.847+0000 7f761bcbc640 1 -- 172.21.15.148:0/1675370245 shutdown_connections 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.847+0000 7f761bcbc640 1 -- 172.21.15.148:0/1675370245 wait complete. 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 Processor -- start 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 -- start start 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 0x7f7614157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761406b580 0x7f76140b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140cffa0 0x7f76140c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7614162200 con 0x7f761406b1a0 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76140d03f0 con 0x7f76140cffa0 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f761bcbc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7614162720 con 0x7f761406b580 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f7619a58640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 0x7f7614157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f7619a58640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 0x7f7614157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51586/0 (socket says 172.21.15.148:51586) 2024-04-25T09:55:11.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.848+0000 7f7619a58640 1 -- 172.21.15.148:0/2557056565 learned_addr learned my addr 172.21.15.148:0/2557056565 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f7619a58640 1 -- 172.21.15.148:0/2557056565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761406b580 msgr2=0x7f76140b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f7619a58640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761406b580 0x7f76140b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f7619a58640 1 -- 172.21.15.148:0/2557056565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140cffa0 msgr2=0x7f76140c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f7619a58640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140cffa0 0x7f76140c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f7619a58640 1 -- 172.21.15.148:0/2557056565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140c5500 con 0x7f761406b1a0 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f7619a58640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 0x7f7614157ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f761000b3f0 tx=0x7f761000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610015020 con 0x7f761406b1a0 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7610011830 con 0x7f761406b1a0 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7610011c20 con 0x7f761406b1a0 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76140c7860 con 0x7f761406b1a0 2024-04-25T09:55:11.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.849+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76140c7e20 con 0x7f761406b1a0 2024-04-25T09:55:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.850+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761406efc0 con 0x7f761406b1a0 2024-04-25T09:55:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.850+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7610018020 con 0x7f761406b1a0 2024-04-25T09:55:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.851+0000 7f760a7fc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f75f806b4d0 0x7f75f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.851+0000 7f7619257640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f75f806b4d0 0x7f75f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.851+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f761008a2a0 con 0x7f761406b1a0 2024-04-25T09:55:11.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.851+0000 7f7619257640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f75f806b4d0 0x7f75f806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7604006fd0 tx=0x7f7604008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:11.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:11.855+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f761001f030 con 0x7f761406b1a0 2024-04-25T09:55:12.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.033+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f761406d370 con 0x7f761406b1a0 2024-04-25T09:55:12.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.034+0000 7f760a7fc640 1 -- 172.21.15.148:0/2557056565 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7f76100172c0 con 0x7f761406b1a0 2024-04-25T09:55:12.037 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:12.037 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:12.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f75f806b4d0 msgr2=0x7f75f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f75f806b4d0 0x7f75f806d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7604006fd0 tx=0x7f7604008040 comp rx=0 tx=0).stop 2024-04-25T09:55:12.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 msgr2=0x7f7614157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 0x7f7614157ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f761000b3f0 tx=0x7f761000b8b0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 shutdown_connections 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f75f806b4d0 0x7f75f806d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140cffa0 0x7f76140c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f761406b580 0x7f76140b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 --2- 172.21.15.148:0/2557056565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761406b1a0 0x7f7614157ad0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 >> 172.21.15.148:0/2557056565 conn(0x7f76140b2220 msgr2=0x7f76140b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 shutdown_connections 2024-04-25T09:55:12.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.036+0000 7f761bcbc640 1 -- 172.21.15.148:0/2557056565 wait complete. 2024-04-25T09:55:12.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:12.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:12.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.148+0000 7fd1b3ef5640 1 Processor -- start 2024-04-25T09:55:12.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.148+0000 7fd1b3ef5640 1 -- start start 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.149+0000 7fd1b3ef5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 0x7fd1ac0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.149+0000 7fd1b3ef5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0c75e0 con 0x7fd1ac0cf600 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.149+0000 7fd1b3ef5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0c78c0 con 0x7fd1ac15bb50 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.149+0000 7fd1b3ef5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c8680 con 0x7fd1ac0cfa40 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.149+0000 7fd1b1490640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd1ac0cf600 0x7fd1ac0cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43234/0 (socket says 172.21.15.148:43234) 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.149+0000 7fd1b1490640 1 -- 172.21.15.148:0/3837569812 learned_addr learned my addr 172.21.15.148:0/3837569812 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b2492640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 0x7fd1ac0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235140590 0 0) 0x7fd1ac0c75e0 con 0x7fd1ac0cf600 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd198003520 con 0x7fd1ac0cf600 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3765559056 0 0) 0x7fd198003520 con 0x7fd1ac0cf600 2024-04-25T09:55:12.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 msgr2=0x7fd1ac0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 0x7fd1ac0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 >> v1:172.21.15.188:6789/0 conn(0x7fd1ac15bb50 legacy=0x7fd1ac0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0c7ba0 con 0x7fd1ac0cf600 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd19c001670 con 0x7fd1ac0cf600 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 >> v1:172.21.15.148:6789/0 conn(0x7fd1ac0cf600 legacy=0x7fd1ac0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 0x7fd198004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c8680 con 0x7fd1980039e0 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800bb00 con 0x7fd198004d40 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800bd00 con 0x7fd1ac0cfa40 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1490640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 0x7fd198004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b2492640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1c91640 1 -- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 msgr2=0x7fd198004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac0cfa40 0x7fd198004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1c91640 1 -- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd198004d40 msgr2=0x7fd1980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b1c91640 1 -- 172.21.15.148:0/3837569812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198003520 con 0x7fd1980039e0 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.150+0000 7fd1b2492640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.151+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8001e20 tx=0x7fd198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fb6dc479dc0f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.151+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8001a70 con 0x7fd1980039e0 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.151+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a802ae40 con 0x7fd1980039e0 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.151+0000 7fd1b0c8f640 1 -- 172.21.15.148:0/3837569812 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8003bb0 con 0x7fd1980039e0 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/3837569812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1980039e0 msgr2=0x7fd198003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/3837569812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8001e20 tx=0x7fd198005830 comp rx=0 tx=0).stop 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b2492640 1 -- 172.21.15.148:0/3837569812 reap_dead start 2024-04-25T09:55:12.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/3837569812 shutdown_connections 2024-04-25T09:55:12.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/3837569812 >> 172.21.15.148:0/3837569812 conn(0x7fd1ac06f500 msgr2=0x7fd1ac06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/3837569812 shutdown_connections 2024-04-25T09:55:12.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.152+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/3837569812 wait complete. 2024-04-25T09:55:12.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.153+0000 7fd1b3ef5640 1 Processor -- start 2024-04-25T09:55:12.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.153+0000 7fd1b3ef5640 1 -- start start 2024-04-25T09:55:12.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b3ef5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1980039e0 0x7fd1ac14bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b3ef5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 0x7fd1ac155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b3ef5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac14d880 0x7fd1ac155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b3ef5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0cfed0 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b3ef5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c8160 con 0x7fd1980039e0 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b3ef5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c72a0 con 0x7fd1ac14d880 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 0x7fd1ac155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 0x7fd1ac155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51600/0 (socket says 172.21.15.148:51600) 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 -- 172.21.15.148:0/423807009 learned_addr learned my addr 172.21.15.148:0/423807009 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b2492640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac14d880 0x7fd1ac155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1980039e0 0x7fd1ac14bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 -- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac14d880 msgr2=0x7fd1ac155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac14d880 0x7fd1ac155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 -- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1980039e0 msgr2=0x7fd1ac14bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1980039e0 0x7fd1ac14bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 -- 172.21.15.148:0/423807009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac14c8e0 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1b1490640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 0x7fd1ac155030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0011e0 tx=0x7fd19c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00f960 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.154+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c018070 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.155+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac1f5390 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.155+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00fd60 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.155+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac1f5950 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.155+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac073a00 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.156+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd19c01d460 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.156+0000 7fd1a27fc640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd19006b500 0x7fd19006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.156+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd19c08a540 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.156+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd19006b500 0x7fd19006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.157+0000 7fd1b1c91640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd19006b500 0x7fd19006d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8032000 tx=0x7fd1a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.160+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd19c059040 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.333+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1ac0ce700 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.333+0000 7fd1a27fc640 1 -- 172.21.15.148:0/423807009 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd19c058940 con 0x7fd1ac14d3b0 2024-04-25T09:55:12.336 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd19006b500 msgr2=0x7fd19006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd19006b500 0x7fd19006d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8032000 tx=0x7fd1a802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 msgr2=0x7fd1ac155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 0x7fd1ac155030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0011e0 tx=0x7fd19c007720 comp rx=0 tx=0).stop 2024-04-25T09:55:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 shutdown_connections 2024-04-25T09:55:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd19006b500 0x7fd19006d9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac14d880 0x7fd1ac155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1ac14d3b0 0x7fd1ac155030 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 --2- 172.21.15.148:0/423807009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1980039e0 0x7fd1ac14bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 >> 172.21.15.148:0/423807009 conn(0x7fd1ac06f500 msgr2=0x7fd1ac0cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 shutdown_connections 2024-04-25T09:55:12.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.335+0000 7fd1b3ef5640 1 -- 172.21.15.148:0/423807009 wait complete. 2024-04-25T09:55:12.349 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.446+0000 7f2fc82b3640 1 Processor -- start 2024-04-25T09:55:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.446+0000 7f2fc82b3640 1 -- start start 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc82b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fc015b7d0 0x7f2fc00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc82b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2fc00b5620 0x7f2fc00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc82b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc006cf50 con 0x7f2fc006c2d0 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc82b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00ce530 con 0x7f2fc015b7d0 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc82b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00ce810 con 0x7f2fc00b5620 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc604f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fc015b7d0 0x7f2fc00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc6850640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2fc006c2d0 0x7f2fc006c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43244/0 (socket says 172.21.15.148:43244) 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc6850640 1 -- 172.21.15.148:0/3681541030 learned_addr learned my addr 172.21.15.148:0/3681541030 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc604f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2fc015b7d0 0x7f2fc00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51616/0 (socket says 172.21.15.148:51616) 2024-04-25T09:55:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc604f640 1 -- 172.21.15.148:0/3681541030 >> v2:172.21.15.188:3300/0 conn(0x7f2fc00b5620 msgr2=0x7f2fc00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:12.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc604f640 1 --2- 172.21.15.148:0/3681541030 >> v2:172.21.15.188:3300/0 conn(0x7f2fc00b5620 0x7f2fc00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc604f640 1 -- 172.21.15.148:0/3681541030 >> v1:172.21.15.148:6789/0 conn(0x7f2fc006c2d0 legacy=0x7f2fc006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc504d640 1 -- 172.21.15.148:0/3681541030 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986431446 0 0) 0x7f2fc006cf50 con 0x7f2fc006c2d0 2024-04-25T09:55:12.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.447+0000 7f2fc604f640 1 -- 172.21.15.148:0/3681541030 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00ceaf0 con 0x7f2fc015b7d0 2024-04-25T09:55:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.448+0000 7f2fc604f640 1 --2- 172.21.15.148:0/3681541030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cf010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00ce180 tx=0x7f2fbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2853e03c96ed90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.448+0000 7f2fc504d640 1 -- 172.21.15.148:0/3681541030 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc037070 con 0x7f2fc015b7d0 2024-04-25T09:55:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.448+0000 7f2fc504d640 1 -- 172.21.15.148:0/3681541030 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc032440 con 0x7f2fc015b7d0 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.448+0000 7f2fc504d640 1 -- 172.21.15.148:0/3681541030 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc032810 con 0x7f2fc015b7d0 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.448+0000 7f2fc82b3640 1 -- 172.21.15.148:0/3681541030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc015b7d0 msgr2=0x7f2fc00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.448+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/3681541030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cf010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00ce180 tx=0x7f2fbc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.449+0000 7f2fc82b3640 1 -- 172.21.15.148:0/3681541030 shutdown_connections 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.449+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/3681541030 >> v2:172.21.15.188:3300/0 conn(0x7f2fc00b5620 0x7f2fc00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.449+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/3681541030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cf010 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.449+0000 7f2fc82b3640 1 -- 172.21.15.148:0/3681541030 >> 172.21.15.148:0/3681541030 conn(0x7f2fc00d0800 msgr2=0x7f2fc00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.449+0000 7f2fc82b3640 1 -- 172.21.15.148:0/3681541030 shutdown_connections 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.449+0000 7f2fc82b3640 1 -- 172.21.15.148:0/3681541030 wait complete. 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 Processor -- start 2024-04-25T09:55:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 -- start start 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 0x7f2fc0157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc00b5620 0x7f2fc00c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00cf6d0 con 0x7f2fc006c2d0 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00cf550 con 0x7f2fc015b7d0 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.450+0000 7f2fc82b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc015bbb0 con 0x7f2fc00b5620 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 0x7f2fc0157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 0x7f2fc0157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51622/0 (socket says 172.21.15.148:51622) 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 -- 172.21.15.148:0/123908384 learned_addr learned my addr 172.21.15.148:0/123908384 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 -- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc00b5620 msgr2=0x7f2fc00c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc584e640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc00b5620 0x7f2fc00c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc6850640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc00b5620 0x7f2fc00c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 -- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc015b7d0 msgr2=0x7f2fc00cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 -- 172.21.15.148:0/123908384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00c7590 con 0x7f2fc006c2d0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc584e640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc00b5620 0x7f2fc00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc6850640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc604f640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 0x7f2fc0157bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc000c00 tx=0x7f2fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc041070 con 0x7f2fc006c2d0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc03c950 con 0x7f2fc006c2d0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00cd190 con 0x7f2fc006c2d0 2024-04-25T09:55:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc037040 con 0x7f2fc006c2d0 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.451+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00cd750 con 0x7f2fc006c2d0 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.452+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc0073a00 con 0x7f2fc006c2d0 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.453+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fbc044070 con 0x7f2fc006c2d0 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.453+0000 7f2fb6ffd640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2fa406b4d0 0x7f2fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.453+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f2fbc0ab150 con 0x7f2fc006c2d0 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.453+0000 7f2fc584e640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2fa406b4d0 0x7f2fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.453+0000 7f2fc584e640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2fa406b4d0 0x7f2fa406d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.457+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fbc079d70 con 0x7f2fc006c2d0 2024-04-25T09:55:12.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.628+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fc00ce060 con 0x7f2fc006c2d0 2024-04-25T09:55:12.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.628+0000 7f2fb6ffd640 1 -- 172.21.15.148:0/123908384 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fbc079670 con 0x7f2fc006c2d0 2024-04-25T09:55:12.631 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:12.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2fa406b4d0 msgr2=0x7f2fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2fa406b4d0 0x7f2fa406d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).stop 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 msgr2=0x7f2fc0157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 0x7f2fc0157bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc000c00 tx=0x7f2fbc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 shutdown_connections 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2fa406b4d0 0x7f2fa406d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc015b7d0 0x7f2fc00cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc00b5620 0x7f2fc00c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 --2- 172.21.15.148:0/123908384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2fc006c2d0 0x7f2fc0157bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 >> 172.21.15.148:0/123908384 conn(0x7f2fc00d0800 msgr2=0x7f2fc015b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 shutdown_connections 2024-04-25T09:55:12.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.630+0000 7f2fc82b3640 1 -- 172.21.15.148:0/123908384 wait complete. 2024-04-25T09:55:12.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:12.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.741+0000 7f7c2a691640 1 Processor -- start 2024-04-25T09:55:12.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.741+0000 7f7c2a691640 1 -- start start 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.741+0000 7f7c2a691640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2415dac0 0x7f7c240d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c2a691640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7c240d1f30 0x7f7c240d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c2a691640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240d2420 con 0x7f7c240d34c0 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c2a691640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c240d3030 con 0x7f7c240d1f30 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c2a691640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240d2700 con 0x7f7c2415dac0 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c237fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7c240d1f30 0x7f7c240d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2415dac0 0x7f7c240d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c28c2e640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7c240d34c0 0x7f7c240b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:36016/0 (socket says 172.21.15.148:36016) 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c28c2e640 1 -- 172.21.15.148:0/1055163417 learned_addr learned my addr 172.21.15.148:0/1055163417 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c22ffd640 1 -- 172.21.15.148:0/1055163417 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332431957 0 0) 0x7f7c240d2420 con 0x7f7c240d34c0 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c22ffd640 1 -- 172.21.15.148:0/1055163417 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c0c003520 con 0x7f7c240d34c0 2024-04-25T09:55:12.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c23fff640 1 -- 172.21.15.148:0/1055163417 >> v2:172.21.15.188:3300/0 conn(0x7f7c240d1f30 msgr2=0x7f7c240d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c23fff640 1 --2- 172.21.15.148:0/1055163417 >> v2:172.21.15.188:3300/0 conn(0x7f7c240d1f30 0x7f7c240d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c23fff640 1 -- 172.21.15.148:0/1055163417 >> v1:172.21.15.188:6789/0 conn(0x7f7c240d34c0 legacy=0x7f7c240b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c23fff640 1 -- 172.21.15.148:0/1055163417 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240d29e0 con 0x7f7c2415dac0 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.742+0000 7f7c237fe640 1 --2- 172.21.15.148:0/1055163417 >> v2:172.21.15.188:3300/0 conn(0x7f7c240d1f30 0x7f7c240d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.743+0000 7f7c23fff640 1 --2- 172.21.15.148:0/1055163417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2415dac0 0x7f7c240d19f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c140097a0 tx=0x7f7c1402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dd941e38d857aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.743+0000 7f7c22ffd640 1 -- 172.21.15.148:0/1055163417 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14037070 con 0x7f7c2415dac0 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.743+0000 7f7c22ffd640 1 -- 172.21.15.148:0/1055163417 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c14032440 con 0x7f7c2415dac0 2024-04-25T09:55:12.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.743+0000 7f7c22ffd640 1 -- 172.21.15.148:0/1055163417 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c140327f0 con 0x7f7c2415dac0 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 -- 172.21.15.148:0/1055163417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2415dac0 msgr2=0x7f7c240d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 --2- 172.21.15.148:0/1055163417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2415dac0 0x7f7c240d19f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c140097a0 tx=0x7f7c1402a210 comp rx=0 tx=0).stop 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 -- 172.21.15.148:0/1055163417 shutdown_connections 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 --2- 172.21.15.148:0/1055163417 >> v2:172.21.15.188:3300/0 conn(0x7f7c240d1f30 0x7f7c240d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 --2- 172.21.15.148:0/1055163417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2415dac0 0x7f7c240d19f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 -- 172.21.15.148:0/1055163417 >> 172.21.15.148:0/1055163417 conn(0x7f7c24061c30 msgr2=0x7f7c2406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 -- 172.21.15.148:0/1055163417 shutdown_connections 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.744+0000 7f7c2a691640 1 -- 172.21.15.148:0/1055163417 wait complete. 2024-04-25T09:55:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.745+0000 7f7c2a691640 1 Processor -- start 2024-04-25T09:55:12.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.745+0000 7f7c2a691640 1 -- start start 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c2a691640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c240d1f30 0x7f7c240c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c2a691640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c240d34c0 0x7f7c240cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c2a691640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 0x7f7c2415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c2a691640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240b97d0 con 0x7f7c2415dac0 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c2a691640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2415dea0 con 0x7f7c240d1f30 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c2a691640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240b9950 con 0x7f7c240d34c0 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 0x7f7c2415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 0x7f7c2415a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51636/0 (socket says 172.21.15.148:51636) 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 -- 172.21.15.148:0/2489978432 learned_addr learned my addr 172.21.15.148:0/2489978432 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c237fe640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c240d34c0 0x7f7c240cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c23fff640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c240d1f30 0x7f7c240c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 -- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c240d34c0 msgr2=0x7f7c240cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c240d34c0 0x7f7c240cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 -- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c240d1f30 msgr2=0x7f7c240c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c240d1f30 0x7f7c240c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.746+0000 7f7c28c2e640 1 -- 172.21.15.148:0/2489978432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240cbe20 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.747+0000 7f7c28c2e640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 0x7f7c2415a920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00cbe0 tx=0x7f7c1c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.747+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c017070 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.747+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c00ad40 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.747+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c240cc100 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.747+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c012600 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.747+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c240c7dd0 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.748+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c24073a00 con 0x7f7c2415dac0 2024-04-25T09:55:12.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.748+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c1c01a070 con 0x7f7c2415dac0 2024-04-25T09:55:12.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.749+0000 7f7c20ff9640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c1006b4d0 0x7f7c1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:12.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.749+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f7c1c08a270 con 0x7f7c2415dac0 2024-04-25T09:55:12.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.749+0000 7f7c23fff640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c1006b4d0 0x7f7c1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:12.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.749+0000 7f7c23fff640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c1006b4d0 0x7f7c1006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c14034000 tx=0x7f7c1402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.753+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c1c058720 con 0x7f7c2415dac0 2024-04-25T09:55:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.922+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c2415d2c0 con 0x7f7c2415dac0 2024-04-25T09:55:12.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.922+0000 7f7c20ff9640 1 -- 172.21.15.148:0/2489978432 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c1c05c7c0 con 0x7f7c2415dac0 2024-04-25T09:55:12.925 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c1006b4d0 msgr2=0x7f7c1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c1006b4d0 0x7f7c1006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c14034000 tx=0x7f7c1402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 msgr2=0x7f7c2415a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 0x7f7c2415a920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00cbe0 tx=0x7f7c1c00a430 comp rx=0 tx=0).stop 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 shutdown_connections 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c1006b4d0 0x7f7c1006d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c2415dac0 0x7f7c2415a920 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c240d34c0 0x7f7c240cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 --2- 172.21.15.148:0/2489978432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c240d1f30 0x7f7c240c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.924+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 >> 172.21.15.148:0/2489978432 conn(0x7f7c24061c30 msgr2=0x7f7c240b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.925+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 shutdown_connections 2024-04-25T09:55:12.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:12.925+0000 7f7c2a691640 1 -- 172.21.15.148:0/2489978432 wait complete. 2024-04-25T09:55:12.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:13.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.036+0000 7f50d05c8640 1 Processor -- start 2024-04-25T09:55:13.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.036+0000 7f50d05c8640 1 -- start start 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50d05c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c80d34c0 0x7f50c80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50d05c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50c80d08e0 0x7f50c80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50d05c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c80d2420 con 0x7f50c815dac0 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50d05c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80d3030 con 0x7f50c80d08e0 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50d05c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80d2700 con 0x7f50c80d34c0 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50c80d08e0 0x7f50c80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50c80d08e0 0x7f50c80b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51648/0 (socket says 172.21.15.148:51648) 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 -- 172.21.15.148:0/2875968549 learned_addr learned my addr 172.21.15.148:0/2875968549 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ce364640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50c815dac0 0x7f50c80d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43248/0 (socket says 172.21.15.148:43248) 2024-04-25T09:55:13.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 -- 172.21.15.148:0/2875968549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c80d34c0 msgr2=0x7f50c80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50cdb63640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c80d34c0 0x7f50c80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50cd362640 1 -- 172.21.15.148:0/2875968549 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024048091 0 0) 0x7f50c80d2420 con 0x7f50c815dac0 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c80d34c0 0x7f50c80d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 -- 172.21.15.148:0/2875968549 >> v1:172.21.15.148:6789/0 conn(0x7f50c815dac0 legacy=0x7f50c80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50ceb65640 1 -- 172.21.15.148:0/2875968549 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80d29e0 con 0x7f50c80d08e0 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.037+0000 7f50cdb63640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c80d34c0 0x7f50c80d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.038+0000 7f50ceb65640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d08e0 0x7f50c80b6990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009e50 tx=0x7f50c002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1be80ac7d0cdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.038+0000 7f50cd362640 1 -- 172.21.15.148:0/2875968549 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0036070 con 0x7f50c80d08e0 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.038+0000 7f50cd362640 1 -- 172.21.15.148:0/2875968549 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0031600 con 0x7f50c80d08e0 2024-04-25T09:55:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.038+0000 7f50cd362640 1 -- 172.21.15.148:0/2875968549 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00319d0 con 0x7f50c80d08e0 2024-04-25T09:55:13.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 -- 172.21.15.148:0/2875968549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d08e0 msgr2=0x7f50c80b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d08e0 0x7f50c80b6990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009e50 tx=0x7f50c002a7a0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 -- 172.21.15.148:0/2875968549 shutdown_connections 2024-04-25T09:55:13.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d08e0 0x7f50c80b6990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2875968549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c80d34c0 0x7f50c80d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 -- 172.21.15.148:0/2875968549 >> 172.21.15.148:0/2875968549 conn(0x7f50c8061c30 msgr2=0x7f50c806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 -- 172.21.15.148:0/2875968549 shutdown_connections 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.039+0000 7f50d05c8640 1 -- 172.21.15.148:0/2875968549 wait complete. 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.040+0000 7f50d05c8640 1 Processor -- start 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.040+0000 7f50d05c8640 1 -- start start 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50d05c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c80d08e0 0x7f50c80b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50d05c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 0x7f50c80cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50d05c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c815dac0 0x7f50c81f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50d05c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c815dea0 con 0x7f50c80d34c0 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50d05c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80d2100 con 0x7f50c80d08e0 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50d05c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80b97d0 con 0x7f50c815dac0 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 0x7f50c80cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 0x7f50c80cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51664/0 (socket says 172.21.15.148:51664) 2024-04-25T09:55:13.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 -- 172.21.15.148:0/2881510697 learned_addr learned my addr 172.21.15.148:0/2881510697 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50ceb65640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c815dac0 0x7f50c81f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50ce364640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c80d08e0 0x7f50c80b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 -- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c815dac0 msgr2=0x7f50c81f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c815dac0 0x7f50c81f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 -- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c80d08e0 msgr2=0x7f50c80b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c80d08e0 0x7f50c80b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 -- 172.21.15.148:0/2881510697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80b4b30 con 0x7f50c80d34c0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.041+0000 7f50cdb63640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 0x7f50c80cc820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c7f0 tx=0x7f50b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.042+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8005b90 con 0x7f50c80d34c0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.042+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8005d30 con 0x7f50c80d34c0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.042+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8007a70 con 0x7f50c80d34c0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.042+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c80b4e10 con 0x7f50c80d34c0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.042+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c80b6180 con 0x7f50c80d34c0 2024-04-25T09:55:13.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.043+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c815f700 con 0x7f50c80d34c0 2024-04-25T09:55:13.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.043+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50b8007c10 con 0x7f50c80d34c0 2024-04-25T09:55:13.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.043+0000 7f50beffd640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50ac06b4d0 0x7f50ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.044+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f50b808a630 con 0x7f50c80d34c0 2024-04-25T09:55:13.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.044+0000 7f50ce364640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50ac06b4d0 0x7f50ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.044+0000 7f50ce364640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50ac06b4d0 0x7f50ac06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000f60 tx=0x7f50c4007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.048+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50b8015020 con 0x7f50c80d34c0 2024-04-25T09:55:13.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.223+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f50c8073a00 con 0x7f50c80d34c0 2024-04-25T09:55:13.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.223+0000 7f50beffd640 1 -- 172.21.15.148:0/2881510697 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f50b8059130 con 0x7f50c80d34c0 2024-04-25T09:55:13.226 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50ac06b4d0 msgr2=0x7f50ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50ac06b4d0 0x7f50ac06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50c4000f60 tx=0x7f50c4007c00 comp rx=0 tx=0).stop 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 msgr2=0x7f50c80cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 0x7f50c80cc820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f50b800c7f0 tx=0x7f50b800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 shutdown_connections 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50ac06b4d0 0x7f50ac06d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c815dac0 0x7f50c81f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50c80d34c0 0x7f50c80cc820 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 --2- 172.21.15.148:0/2881510697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c80d08e0 0x7f50c80b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 >> 172.21.15.148:0/2881510697 conn(0x7f50c8061c30 msgr2=0x7f50c80b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 shutdown_connections 2024-04-25T09:55:13.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.226+0000 7f50d05c8640 1 -- 172.21.15.148:0/2881510697 wait complete. 2024-04-25T09:55:13.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:55:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.334+0000 7ff0bd85d640 1 Processor -- start 2024-04-25T09:55:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.334+0000 7ff0bd85d640 1 -- start start 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0bd85d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0b815dc20 0x7ff0b80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0bd85d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0b80d32f0 0x7ff0b80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0bd85d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b815e000 con 0x7ff0b8161710 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0bd85d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b80d1de0 con 0x7ff0b815dc20 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0bd85d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b8161c60 con 0x7ff0b80d32f0 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0b815dc20 0x7ff0b80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0ae7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0b8161710 0x7ff0b806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43250/0 (socket says 172.21.15.148:43250) 2024-04-25T09:55:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0b815dc20 0x7ff0b80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51666/0 (socket says 172.21.15.148:51666) 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0ae7fc640 1 -- 172.21.15.148:0/2417469524 learned_addr learned my addr 172.21.15.148:0/2417469524 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b77fe640 1 --2- 172.21.15.148:0/2417469524 >> v2:172.21.15.188:3300/0 conn(0x7ff0b80d32f0 0x7ff0b80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/2417469524 >> v2:172.21.15.188:3300/0 conn(0x7ff0b80d32f0 msgr2=0x7ff0b80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b6ffd640 1 --2- 172.21.15.148:0/2417469524 >> v2:172.21.15.188:3300/0 conn(0x7ff0b80d32f0 0x7ff0b80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/2417469524 >> v1:172.21.15.148:6789/0 conn(0x7ff0b8161710 legacy=0x7ff0b806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b67fc640 1 -- 172.21.15.148:0/2417469524 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478570820 0 0) 0x7ff0b815e000 con 0x7ff0b8161710 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/2417469524 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8161f40 con 0x7ff0b815dc20 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.335+0000 7ff0b77fe640 1 --2- 172.21.15.148:0/2417469524 >> v2:172.21.15.188:3300/0 conn(0x7ff0b80d32f0 0x7ff0b80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.336+0000 7ff0b6ffd640 1 --2- 172.21.15.148:0/2417469524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b815dc20 0x7ff0b80ce670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80b6910 tx=0x7ff0a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81593c268d9566da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.336+0000 7ff0b67fc640 1 -- 172.21.15.148:0/2417469524 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8037070 con 0x7ff0b815dc20 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.336+0000 7ff0b67fc640 1 -- 172.21.15.148:0/2417469524 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8032440 con 0x7ff0b815dc20 2024-04-25T09:55:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.336+0000 7ff0b67fc640 1 -- 172.21.15.148:0/2417469524 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8032810 con 0x7ff0b815dc20 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 -- 172.21.15.148:0/2417469524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b815dc20 msgr2=0x7ff0b80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/2417469524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b815dc20 0x7ff0b80ce670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80b6910 tx=0x7ff0a802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 -- 172.21.15.148:0/2417469524 shutdown_connections 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/2417469524 >> v2:172.21.15.188:3300/0 conn(0x7ff0b80d32f0 0x7ff0b80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/2417469524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b815dc20 0x7ff0b80ce670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 -- 172.21.15.148:0/2417469524 >> 172.21.15.148:0/2417469524 conn(0x7ff0b80d0420 msgr2=0x7ff0b80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 -- 172.21.15.148:0/2417469524 shutdown_connections 2024-04-25T09:55:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.337+0000 7ff0bd85d640 1 -- 172.21.15.148:0/2417469524 wait complete. 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.338+0000 7ff0bd85d640 1 Processor -- start 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.338+0000 7ff0bd85d640 1 -- start start 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0bd85d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 0x7ff0b80b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0bd85d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b815dc20 0x7ff0b8156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0bd85d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b8161710 0x7ff0b8160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0bd85d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80b5e50 con 0x7ff0b80d32f0 2024-04-25T09:55:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0bd85d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8162500 con 0x7ff0b8161710 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0bd85d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80b5fd0 con 0x7ff0b815dc20 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 0x7ff0b80b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 0x7ff0b80b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51670/0 (socket says 172.21.15.148:51670) 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/669752146 learned_addr learned my addr 172.21.15.148:0/669752146 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/669752146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b815dc20 msgr2=0x7ff0b8156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b815dc20 0x7ff0b8156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/669752146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b8161710 msgr2=0x7ff0b8160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b8161710 0x7ff0b8160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 -- 172.21.15.148:0/669752146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80b17f0 con 0x7ff0b80d32f0 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.339+0000 7ff0b6ffd640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 0x7ff0b80b48f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009890 tx=0x7ff0a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.340+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8037040 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.340+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8042070 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.340+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b80b1ad0 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.340+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8030a70 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.340+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b80b2090 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.341+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80d4760 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.341+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0a8030c10 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.342+0000 7ff0b4ff9640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff09806b4d0 0x7ff09806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.342+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7ff0a80ab0e0 con 0x7ff0b80d32f0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.342+0000 7ff0ae7fc640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff09806b4d0 0x7ff09806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.342+0000 7ff0ae7fc640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff09806b4d0 0x7ff09806d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0007fd0 tx=0x7ff0a0007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.346+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0b80d4760 con 0x7ff0b80d32f0 2024-04-25T09:55:13.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.526+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0b8073a00 con 0x7ff0b80d32f0 2024-04-25T09:55:13.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.527+0000 7ff0b4ff9640 1 -- 172.21.15.148:0/669752146 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff0a8045360 con 0x7ff0b80d32f0 2024-04-25T09:55:13.530 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff09806b4d0 msgr2=0x7ff09806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff09806b4d0 0x7ff09806d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0007fd0 tx=0x7ff0a0007960 comp rx=0 tx=0).stop 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 msgr2=0x7ff0b80b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 0x7ff0b80b48f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009890 tx=0x7ff0a80046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 shutdown_connections 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff09806b4d0 0x7ff09806d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0b8161710 0x7ff0b8160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0b815dc20 0x7ff0b8156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 --2- 172.21.15.148:0/669752146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0b80d32f0 0x7ff0b80b48f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 >> 172.21.15.148:0/669752146 conn(0x7ff0b80d0420 msgr2=0x7ff0b80b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 shutdown_connections 2024-04-25T09:55:13.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.529+0000 7ff0bd85d640 1 -- 172.21.15.148:0/669752146 wait complete. 2024-04-25T09:55:13.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:55:13.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:13.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T09:55:13.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:55:13.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:13.568 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:55:13.624 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:13.624 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:13.643 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:13.643 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:13.702 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:13.702 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:13.761 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.857+0000 7fce4e509640 1 Processor -- start 2024-04-25T09:55:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.857+0000 7fce4e509640 1 -- start start 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.857+0000 7fce4e509640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce4815dbe0 0x7fce480d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.857+0000 7fce4e509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce480b2cb0 0x7fce480d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce4e509640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce480b92a0 0x7fce480b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce4e509640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4815dfc0 con 0x7fce4815dbe0 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce4e509640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce480d1a00 con 0x7fce480b92a0 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce4e509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480b97a0 con 0x7fce480b2cb0 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce4815dbe0 0x7fce480d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce4815dbe0 0x7fce480d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35246/0 (socket says 172.21.15.148:35246) 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 -- 172.21.15.148:0/1846106759 learned_addr learned my addr 172.21.15.148:0/1846106759 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce477fe640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce480b2cb0 0x7fce480d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce4caa6640 1 --2- 172.21.15.148:0/1846106759 >> v2:172.21.15.188:3300/0 conn(0x7fce480b92a0 0x7fce480b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 -- 172.21.15.148:0/1846106759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce480b2cb0 msgr2=0x7fce480d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce480b2cb0 0x7fce480d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 -- 172.21.15.148:0/1846106759 >> v2:172.21.15.188:3300/0 conn(0x7fce480b92a0 msgr2=0x7fce480b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 --2- 172.21.15.148:0/1846106759 >> v2:172.21.15.188:3300/0 conn(0x7fce480b92a0 0x7fce480b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce47fff640 1 -- 172.21.15.148:0/1846106759 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480d1c50 con 0x7fce4815dbe0 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce477fe640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce480b2cb0 0x7fce480d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.858+0000 7fce4caa6640 1 --2- 172.21.15.148:0/1846106759 >> v2:172.21.15.188:3300/0 conn(0x7fce480b92a0 0x7fce480b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce47fff640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815dbe0 0x7fce480d0910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce4806c640 tx=0x7fce3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351d91d5fc2d63a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce46ffd640 1 -- 172.21.15.148:0/1846106759 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c037070 con 0x7fce4815dbe0 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce46ffd640 1 -- 172.21.15.148:0/1846106759 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c032440 con 0x7fce4815dbe0 2024-04-25T09:55:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce46ffd640 1 -- 172.21.15.148:0/1846106759 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c032810 con 0x7fce4815dbe0 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 -- 172.21.15.148:0/1846106759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815dbe0 msgr2=0x7fce480d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815dbe0 0x7fce480d0910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce4806c640 tx=0x7fce3c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 -- 172.21.15.148:0/1846106759 shutdown_connections 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 --2- 172.21.15.148:0/1846106759 >> v2:172.21.15.188:3300/0 conn(0x7fce480b92a0 0x7fce480b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce480b2cb0 0x7fce480d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 --2- 172.21.15.148:0/1846106759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce4815dbe0 0x7fce480d0910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 -- 172.21.15.148:0/1846106759 >> 172.21.15.148:0/1846106759 conn(0x7fce48061a20 msgr2=0x7fce48061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 -- 172.21.15.148:0/1846106759 shutdown_connections 2024-04-25T09:55:13.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.859+0000 7fce4e509640 1 -- 172.21.15.148:0/1846106759 wait complete. 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 Processor -- start 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 -- start start 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce480b2cb0 0x7fce480b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 0x7fce480caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce4815dbe0 0x7fce4815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480b6ab0 con 0x7fce480b92a0 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480d2210 con 0x7fce480b2cb0 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.861+0000 7fce4e509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480b6c30 con 0x7fce4815dbe0 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 0x7fce480caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 0x7fce480caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35250/0 (socket says 172.21.15.148:35250) 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 -- 172.21.15.148:0/4047083992 learned_addr learned my addr 172.21.15.148:0/4047083992 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce4caa6640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce4815dbe0 0x7fce4815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce47fff640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce480b2cb0 0x7fce480b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 -- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce4815dbe0 msgr2=0x7fce4815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce4815dbe0 0x7fce4815fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 -- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce480b2cb0 msgr2=0x7fce480b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce480b2cb0 0x7fce480b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 -- 172.21.15.148:0/4047083992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480b5100 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce47fff640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce480b2cb0 0x7fce480b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce477fe640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 0x7fce480caa50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c7f0 tx=0x7fce3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38015070 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce380079d0 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce480b32e0 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38007da0 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.862+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce481f3d70 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.863+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce48073990 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.864+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce38018070 con 0x7fce480b92a0 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.864+0000 7fce44ff9640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce3406b4d0 0x7fce3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:13.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.864+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fce3808a4c0 con 0x7fce480b92a0 2024-04-25T09:55:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.864+0000 7fce47fff640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce3406b4d0 0x7fce3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:13.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.864+0000 7fce47fff640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce3406b4d0 0x7fce3406d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce3c034000 tx=0x7fce3c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:13.868+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce38058fc0 con 0x7fce480b92a0 2024-04-25T09:55:14.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.039+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce480b5330 con 0x7fce480b92a0 2024-04-25T09:55:14.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.040+0000 7fce44ff9640 1 -- 172.21.15.148:0/4047083992 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7fce380588c0 con 0x7fce480b92a0 2024-04-25T09:55:14.043 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:14.043 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce3406b4d0 msgr2=0x7fce3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce3406b4d0 0x7fce3406d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce3c034000 tx=0x7fce3c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 msgr2=0x7fce480caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 0x7fce480caa50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fce3800c7f0 tx=0x7fce3800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 shutdown_connections 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce3406b4d0 0x7fce3406d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce4815dbe0 0x7fce4815fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b92a0 0x7fce480caa50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 --2- 172.21.15.148:0/4047083992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce480b2cb0 0x7fce480b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 >> 172.21.15.148:0/4047083992 conn(0x7fce48061a20 msgr2=0x7fce4815b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 shutdown_connections 2024-04-25T09:55:14.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.042+0000 7fce4e509640 1 -- 172.21.15.148:0/4047083992 wait complete. 2024-04-25T09:55:14.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:14.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:14.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.153+0000 7ff9676fc640 1 Processor -- start 2024-04-25T09:55:14.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.153+0000 7ff9676fc640 1 -- start start 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff9676fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff960070dc0 0x7ff96006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff9676fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9600d1770 con 0x7ff9600711a0 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff9676fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff960162740 con 0x7ff96015deb0 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff9676fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9600b0eb0 con 0x7ff960070dc0 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff965498640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff960070dc0 0x7ff96006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff964c97640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff9600711a0 0x7ff9600b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55530/0 (socket says 172.21.15.148:55530) 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff965498640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff960070dc0 0x7ff96006ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35254/0 (socket says 172.21.15.148:35254) 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff964c97640 1 -- 172.21.15.148:0/1057550108 learned_addr learned my addr 172.21.15.148:0/1057550108 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff965498640 1 -- 172.21.15.148:0/1057550108 >> v1:172.21.15.188:6789/0 conn(0x7ff96015deb0 legacy=0x7ff96015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff965498640 1 -- 172.21.15.148:0/1057550108 >> v1:172.21.15.148:6789/0 conn(0x7ff9600711a0 legacy=0x7ff9600b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff957fff640 1 -- 172.21.15.148:0/1057550108 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431413815 0 0) 0x7ff9600d1770 con 0x7ff9600711a0 2024-04-25T09:55:14.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.154+0000 7ff965498640 1 -- 172.21.15.148:0/1057550108 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960161980 con 0x7ff960070dc0 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.155+0000 7ff965498640 1 --2- 172.21.15.148:0/1057550108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff96006ccd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0097a0 tx=0x7ff95c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdf77e050f253ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.155+0000 7ff957fff640 1 -- 172.21.15.148:0/1057550108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff960070dc0 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.155+0000 7ff957fff640 1 -- 172.21.15.148:0/1057550108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c02adc0 con 0x7ff960070dc0 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.155+0000 7ff957fff640 1 -- 172.21.15.148:0/1057550108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032660 con 0x7ff960070dc0 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 -- 172.21.15.148:0/1057550108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 msgr2=0x7ff96006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 --2- 172.21.15.148:0/1057550108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff96006ccd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0097a0 tx=0x7ff95c02a0b0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 -- 172.21.15.148:0/1057550108 shutdown_connections 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 --2- 172.21.15.148:0/1057550108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff96006ccd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 -- 172.21.15.148:0/1057550108 >> 172.21.15.148:0/1057550108 conn(0x7ff9600b3530 msgr2=0x7ff9600b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 -- 172.21.15.148:0/1057550108 shutdown_connections 2024-04-25T09:55:14.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.156+0000 7ff9676fc640 1 -- 172.21.15.148:0/1057550108 wait complete. 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.157+0000 7ff9676fc640 1 Processor -- start 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.157+0000 7ff9676fc640 1 -- start start 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff9601f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9600711a0 0x7ff9601583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff96015deb0 0x7ff960158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600b10f0 con 0x7ff960070dc0 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9601622b0 con 0x7ff96015deb0 2024-04-25T09:55:14.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff960161f40 con 0x7ff9600711a0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff9601f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff9601f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35264/0 (socket says 172.21.15.148:35264) 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 -- 172.21.15.148:0/3921125227 learned_addr learned my addr 172.21.15.148:0/3921125227 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff964c97640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9600711a0 0x7ff9601583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965c99640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff96015deb0 0x7ff960158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 -- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9600711a0 msgr2=0x7ff9601583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9600711a0 0x7ff9601583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 -- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff96015deb0 msgr2=0x7ff960158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff96015deb0 0x7ff960158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 -- 172.21.15.148:0/3921125227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96015f8e0 con 0x7ff960070dc0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff964c97640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9600711a0 0x7ff9601583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965c99640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff96015deb0 0x7ff960158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff965498640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff9601f6f00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0098c0 tx=0x7ff95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c041070 con 0x7ff960070dc0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c03c670 con 0x7ff960070dc0 2024-04-25T09:55:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96015fbc0 con 0x7ff960070dc0 2024-04-25T09:55:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.158+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037040 con 0x7ff960070dc0 2024-04-25T09:55:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.159+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff960156420 con 0x7ff960070dc0 2024-04-25T09:55:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.160+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9600b7b00 con 0x7ff960070dc0 2024-04-25T09:55:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.160+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff95c044070 con 0x7ff960070dc0 2024-04-25T09:55:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.160+0000 7ff955ffb640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff94406b4d0 0x7ff94406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.160+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7ff95c0aae30 con 0x7ff960070dc0 2024-04-25T09:55:14.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.160+0000 7ff964c97640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff94406b4d0 0x7ff94406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.161+0000 7ff964c97640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff94406b4d0 0x7ff94406d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9600b3380 tx=0x7ff950009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.164+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9600b7b00 con 0x7ff960070dc0 2024-04-25T09:55:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.333+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9600d01c0 con 0x7ff960070dc0 2024-04-25T09:55:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.333+0000 7ff955ffb640 1 -- 172.21.15.148:0/3921125227 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9600d01c0 con 0x7ff960070dc0 2024-04-25T09:55:14.336 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.335+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff94406b4d0 msgr2=0x7ff94406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.335+0000 7ff9676fc640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff94406b4d0 0x7ff94406d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9600b3380 tx=0x7ff950009040 comp rx=0 tx=0).stop 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.335+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 msgr2=0x7ff9601f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.335+0000 7ff9676fc640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff9601f6f00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0098c0 tx=0x7ff95c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.335+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 shutdown_connections 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff94406b4d0 0x7ff94406d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff96015deb0 0x7ff960158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9600711a0 0x7ff9601583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 --2- 172.21.15.148:0/3921125227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff960070dc0 0x7ff9601f6f00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 >> 172.21.15.148:0/3921125227 conn(0x7ff9600b3530 msgr2=0x7ff96015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 shutdown_connections 2024-04-25T09:55:14.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.336+0000 7ff9676fc640 1 -- 172.21.15.148:0/3921125227 wait complete. 2024-04-25T09:55:14.349 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:14.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.447+0000 7fb545153640 1 Processor -- start 2024-04-25T09:55:14.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.447+0000 7fb545153640 1 -- start start 2024-04-25T09:55:14.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.447+0000 7fb545153640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb54015b980 0x7fb5400cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.447+0000 7fb545153640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb540072280 con 0x7fb5400ce230 2024-04-25T09:55:14.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.447+0000 7fb545153640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5400714c0 con 0x7fb5400cc4f0 2024-04-25T09:55:14.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.447+0000 7fb545153640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb540071270 con 0x7fb54015b980 2024-04-25T09:55:14.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53e59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb5400ce230 0x7fb54015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55536/0 (socket says 172.21.15.148:55536) 2024-04-25T09:55:14.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53e59c640 1 -- 172.21.15.148:0/793787297 learned_addr learned my addr 172.21.15.148:0/793787297 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368203192 0 0) 0x7fb540072280 con 0x7fb5400ce230 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53f59e640 1 --2- 172.21.15.148:0/793787297 >> v2:172.21.15.188:3300/0 conn(0x7fb54015b980 0x7fb5400cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb528003520 con 0x7fb5400ce230 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806808601 0 0) 0x7fb528003520 con 0x7fb5400ce230 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 >> v2:172.21.15.188:3300/0 conn(0x7fb54015b980 msgr2=0x7fb5400cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 --2- 172.21.15.148:0/793787297 >> v2:172.21.15.188:3300/0 conn(0x7fb54015b980 0x7fb5400cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 >> v1:172.21.15.188:6789/0 conn(0x7fb5400cc4f0 legacy=0x7fb5400cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400717a0 con 0x7fb5400ce230 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb534001670 con 0x7fb5400ce230 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 >> v1:172.21.15.148:6789/0 conn(0x7fb5400ce230 legacy=0x7fb54015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb528003a00 0x7fb528003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528004800 0x7fb528004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5280051c0 0x7fb52800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540071270 con 0x7fb528003a00 2024-04-25T09:55:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800c0c0 con 0x7fb5280051c0 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800c2c0 con 0x7fb528004800 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb528003a00 0x7fb528003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53f59e640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5280051c0 0x7fb52800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.448+0000 7fb53e59c640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528004800 0x7fb528004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53ed9d640 1 -- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528004800 msgr2=0x7fb528004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528004800 0x7fb528004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53ed9d640 1 -- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5280051c0 msgr2=0x7fb52800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5280051c0 0x7fb52800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53ed9d640 1 -- 172.21.15.148:0/793787297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528003520 con 0x7fb528003a00 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53f59e640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5280051c0 0x7fb52800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53f59e640 1 -- 172.21.15.148:0/793787297 reap_dead start 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb528003a00 0x7fb528003e20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb530001750 tx=0x7fb528005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0e640a195ab961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530001370 con 0x7fb528003a00 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300052d0 con 0x7fb528003a00 2024-04-25T09:55:14.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.449+0000 7fb53dd9b640 1 -- 172.21.15.148:0/793787297 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53002abc0 con 0x7fb528003a00 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 -- 172.21.15.148:0/793787297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb528003a00 msgr2=0x7fb528003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb528003a00 0x7fb528003e20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb530001750 tx=0x7fb528005830 comp rx=0 tx=0).stop 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 -- 172.21.15.148:0/793787297 shutdown_connections 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 --2- 172.21.15.148:0/793787297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb528003a00 0x7fb528003e20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 -- 172.21.15.148:0/793787297 >> 172.21.15.148:0/793787297 conn(0x7fb54006abe0 msgr2=0x7fb54006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 -- 172.21.15.148:0/793787297 shutdown_connections 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.450+0000 7fb545153640 1 -- 172.21.15.148:0/793787297 wait complete. 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.451+0000 7fb545153640 1 Processor -- start 2024-04-25T09:55:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.451+0000 7fb545153640 1 -- start start 2024-04-25T09:55:14.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb545153640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528003a00 0x7fb540151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb545153640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb540157da0 0x7fb540141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb545153640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 0x7fb5400b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb545153640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400cc8d0 con 0x7fb5400b3530 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb545153640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540071d60 con 0x7fb540157da0 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb545153640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400cca50 con 0x7fb528003a00 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb53f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 0x7fb5400b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb53f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 0x7fb5400b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35278/0 (socket says 172.21.15.148:35278) 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb53f59e640 1 -- 172.21.15.148:0/3753707595 learned_addr learned my addr 172.21.15.148:0/3753707595 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528003a00 0x7fb540151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.452+0000 7fb53e59c640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb540157da0 0x7fb540141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb53f59e640 1 -- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528003a00 msgr2=0x7fb540151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb53f59e640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528003a00 0x7fb540151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb53f59e640 1 -- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb540157da0 msgr2=0x7fb540141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb53f59e640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb540157da0 0x7fb540141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb53f59e640 1 -- 172.21.15.148:0/3753707595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540152510 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb53f59e640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 0x7fb5400b39b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb540151fa0 tx=0x7fb538010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538010da0 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538003990 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540141560 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538003d80 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.453+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb540152a90 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.454+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5400b3ef0 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.454+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb53801e020 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.455+0000 7fb5277fe640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb52c06b500 0x7fb52c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.455+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb52c06b500 0x7fb52c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.455+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fb53808aaa0 con 0x7fb5400b3530 2024-04-25T09:55:14.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.455+0000 7fb53ed9d640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb52c06b500 0x7fb52c06d9c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb530032000 tx=0x7fb53002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.459+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb538058ec0 con 0x7fb5400b3530 2024-04-25T09:55:14.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.628+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb540073a00 con 0x7fb5400b3530 2024-04-25T09:55:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.628+0000 7fb5277fe640 1 -- 172.21.15.148:0/3753707595 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb53805cf60 con 0x7fb5400b3530 2024-04-25T09:55:14.631 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:14.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb52c06b500 msgr2=0x7fb52c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb52c06b500 0x7fb52c06d9c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb530032000 tx=0x7fb53002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:14.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 msgr2=0x7fb5400b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 0x7fb5400b39b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb540151fa0 tx=0x7fb538010590 comp rx=0 tx=0).stop 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 shutdown_connections 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb52c06b500 0x7fb52c06d9c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5400b3530 0x7fb5400b39b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb540157da0 0x7fb540141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 --2- 172.21.15.148:0/3753707595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb528003a00 0x7fb540151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 >> 172.21.15.148:0/3753707595 conn(0x7fb54006abe0 msgr2=0x7fb5400cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 shutdown_connections 2024-04-25T09:55:14.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.630+0000 7fb545153640 1 -- 172.21.15.148:0/3753707595 wait complete. 2024-04-25T09:55:14.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:14.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.741+0000 7f7cbb69e640 1 Processor -- start 2024-04-25T09:55:14.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cbb69e640 1 -- start start 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cbb69e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7cb4158230 0x7f7cb40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cbb69e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7cb40d1b80 0x7f7cb406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cbb69e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40d2020 con 0x7f7cb406c870 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cbb69e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb4158610 con 0x7f7cb4158230 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cbb69e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb406bea0 con 0x7f7cb40d1b80 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.742+0000 7f7cb943a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7cb4158230 0x7f7cb40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb9c3b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7cb406c870 0x7f7cb40b4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55546/0 (socket says 172.21.15.148:55546) 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb9c3b640 1 -- 172.21.15.148:0/3940079224 learned_addr learned my addr 172.21.15.148:0/3940079224 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb943a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7cb4158230 0x7f7cb40d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35280/0 (socket says 172.21.15.148:35280) 2024-04-25T09:55:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb8c39640 1 --2- 172.21.15.148:0/3940079224 >> v2:172.21.15.188:3300/0 conn(0x7f7cb40d1b80 0x7f7cb406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb943a640 1 -- 172.21.15.148:0/3940079224 >> v2:172.21.15.188:3300/0 conn(0x7f7cb40d1b80 msgr2=0x7f7cb406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb943a640 1 --2- 172.21.15.148:0/3940079224 >> v2:172.21.15.188:3300/0 conn(0x7f7cb40d1b80 0x7f7cb406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb943a640 1 -- 172.21.15.148:0/3940079224 >> v1:172.21.15.148:6789/0 conn(0x7f7cb406c870 legacy=0x7f7cb40b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb943a640 1 -- 172.21.15.148:0/3940079224 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40d2300 con 0x7f7cb4158230 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.743+0000 7f7cb943a640 1 --2- 172.21.15.148:0/3940079224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb4158230 0x7f7cb40d1640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7cb415bcc0 tx=0x7f7cb002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27a54f710d9a840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cabfff640 1 -- 172.21.15.148:0/3940079224 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0037070 con 0x7f7cb4158230 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cabfff640 1 -- 172.21.15.148:0/3940079224 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0032440 con 0x7f7cb4158230 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cabfff640 1 -- 172.21.15.148:0/3940079224 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0032810 con 0x7f7cb4158230 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 -- 172.21.15.148:0/3940079224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb4158230 msgr2=0x7f7cb40d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/3940079224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb4158230 0x7f7cb40d1640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7cb415bcc0 tx=0x7f7cb002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 -- 172.21.15.148:0/3940079224 shutdown_connections 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/3940079224 >> v2:172.21.15.188:3300/0 conn(0x7f7cb40d1b80 0x7f7cb406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/3940079224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb4158230 0x7f7cb40d1640 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 -- 172.21.15.148:0/3940079224 >> 172.21.15.148:0/3940079224 conn(0x7f7cb4061600 msgr2=0x7f7cb40619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 -- 172.21.15.148:0/3940079224 shutdown_connections 2024-04-25T09:55:14.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.744+0000 7f7cbb69e640 1 -- 172.21.15.148:0/3940079224 wait complete. 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.745+0000 7f7cbb69e640 1 Processor -- start 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.745+0000 7f7cbb69e640 1 -- start start 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cbb69e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 0x7f7cb41f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cbb69e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb40d1b80 0x7f7cb40cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cbb69e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb4158230 0x7f7cb40cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cbb69e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415e320 con 0x7f7cb406c870 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cbb69e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40d25e0 con 0x7f7cb40d1b80 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cbb69e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415e4a0 con 0x7f7cb4158230 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb943a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 0x7f7cb41f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb943a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 0x7f7cb41f3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35292/0 (socket says 172.21.15.148:35292) 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb943a640 1 -- 172.21.15.148:0/1078155357 learned_addr learned my addr 172.21.15.148:0/1078155357 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb8c39640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb40d1b80 0x7f7cb40cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb9c3b640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb4158230 0x7f7cb40cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb943a640 1 -- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb4158230 msgr2=0x7f7cb40cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.746+0000 7f7cb943a640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb4158230 0x7f7cb40cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7cb943a640 1 -- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb40d1b80 msgr2=0x7f7cb40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7cb943a640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb40d1b80 0x7f7cb40cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7cb943a640 1 -- 172.21.15.148:0/1078155357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb415b4e0 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7cb943a640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 0x7f7cb41f3440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0037040 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0042070 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0030bd0 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40cc410 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.747+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40cd950 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.748+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb40cdbe0 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.748+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cb0047460 con 0x7f7cb406c870 2024-04-25T09:55:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.749+0000 7f7ca9ffb640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c9806b4d0 0x7f7c9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.749+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f7cb00ab180 con 0x7f7cb406c870 2024-04-25T09:55:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.749+0000 7f7cb8c39640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c9806b4d0 0x7f7c9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.749+0000 7f7cb8c39640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c9806b4d0 0x7f7c9806d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7cb415bca0 tx=0x7f7ca4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:14.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.753+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cb0032e20 con 0x7f7cb406c870 2024-04-25T09:55:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.921+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cb406de50 con 0x7f7cb406c870 2024-04-25T09:55:14.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.922+0000 7f7ca9ffb640 1 -- 172.21.15.148:0/1078155357 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cb0079c80 con 0x7f7cb406c870 2024-04-25T09:55:14.924 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.923+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c9806b4d0 msgr2=0x7f7c9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c9806b4d0 0x7f7c9806d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7cb415bca0 tx=0x7f7ca4008040 comp rx=0 tx=0).stop 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 msgr2=0x7f7cb41f3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 0x7f7cb41f3440 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb00046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 shutdown_connections 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7c9806b4d0 0x7f7c9806d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cb4158230 0x7f7cb40cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cb40d1b80 0x7f7cb40cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 --2- 172.21.15.148:0/1078155357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cb406c870 0x7f7cb41f3440 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 >> 172.21.15.148:0/1078155357 conn(0x7f7cb4061600 msgr2=0x7f7cb415d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 shutdown_connections 2024-04-25T09:55:14.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:14.924+0000 7f7cbb69e640 1 -- 172.21.15.148:0/1078155357 wait complete. 2024-04-25T09:55:14.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.035+0000 7f44bf91d640 1 Processor -- start 2024-04-25T09:55:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.035+0000 7f44bf91d640 1 -- start start 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bf91d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44b81454a0 0x7f44b80aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bf91d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44b815c970 0x7f44b806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bf91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b806b120 0x7f44b815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bf91d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b80af540 con 0x7f44b815c970 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bf91d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b80af820 con 0x7f44b81454a0 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bf91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80afb00 con 0x7f44b806b120 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44b815c970 0x7f44b806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f44b815c970 0x7f44b806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35306/0 (socket says 172.21.15.148:35306) 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 -- 172.21.15.148:0/349350031 learned_addr learned my addr 172.21.15.148:0/349350031 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bdeba640 1 --2- 172.21.15.148:0/349350031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b806b120 0x7f44b815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 -- 172.21.15.148:0/349350031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b806b120 msgr2=0x7f44b815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:15.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bd6b9640 1 --2- 172.21.15.148:0/349350031 >> v2:172.21.15.188:3300/0 conn(0x7f44b81454a0 0x7f44b80aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 --2- 172.21.15.148:0/349350031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b806b120 0x7f44b815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 -- 172.21.15.148:0/349350031 >> v2:172.21.15.188:3300/0 conn(0x7f44b81454a0 msgr2=0x7f44b80aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 --2- 172.21.15.148:0/349350031 >> v2:172.21.15.188:3300/0 conn(0x7f44b81454a0 0x7f44b80aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.036+0000 7f44bceb8640 1 -- 172.21.15.148:0/349350031 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b80afde0 con 0x7f44b815c970 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.037+0000 7f44bceb8640 1 --2- 172.21.15.148:0/349350031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b815c970 0x7f44b806abe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f44b80b3320 tx=0x7f44a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edf7060c318805c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.037+0000 7f44affff640 1 -- 172.21.15.148:0/349350031 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8037070 con 0x7f44b815c970 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.037+0000 7f44affff640 1 -- 172.21.15.148:0/349350031 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a8032440 con 0x7f44b815c970 2024-04-25T09:55:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.037+0000 7f44affff640 1 -- 172.21.15.148:0/349350031 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8032810 con 0x7f44b815c970 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 -- 172.21.15.148:0/349350031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b815c970 msgr2=0x7f44b806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 --2- 172.21.15.148:0/349350031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b815c970 0x7f44b806abe0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f44b80b3320 tx=0x7f44a802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 -- 172.21.15.148:0/349350031 shutdown_connections 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 --2- 172.21.15.148:0/349350031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b806b120 0x7f44b815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 --2- 172.21.15.148:0/349350031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b815c970 0x7f44b806abe0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 --2- 172.21.15.148:0/349350031 >> v2:172.21.15.188:3300/0 conn(0x7f44b81454a0 0x7f44b80aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 -- 172.21.15.148:0/349350031 >> 172.21.15.148:0/349350031 conn(0x7f44b80b5af0 msgr2=0x7f44b80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 -- 172.21.15.148:0/349350031 shutdown_connections 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.038+0000 7f44bf91d640 1 -- 172.21.15.148:0/349350031 wait complete. 2024-04-25T09:55:15.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.039+0000 7f44bf91d640 1 Processor -- start 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.039+0000 7f44bf91d640 1 -- start start 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bf91d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 0x7f44b813f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bf91d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b81454a0 0x7f44b809a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bf91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b815c970 0x7f44b81e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bf91d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b815fba0 con 0x7f44b806b120 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bf91d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b815fa20 con 0x7f44b81454a0 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bf91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b815fd20 con 0x7f44b815c970 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 0x7f44b813f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 0x7f44b813f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35316/0 (socket says 172.21.15.148:35316) 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 -- 172.21.15.148:0/379130997 learned_addr learned my addr 172.21.15.148:0/379130997 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bdeba640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b815c970 0x7f44b81e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bceb8640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b81454a0 0x7f44b809a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 -- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b815c970 msgr2=0x7f44b81e5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b815c970 0x7f44b81e5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 -- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b81454a0 msgr2=0x7f44b809a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b81454a0 0x7f44b809a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 -- 172.21.15.148:0/379130997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8140950 con 0x7f44b806b120 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bceb8640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b81454a0 0x7f44b809a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.040+0000 7f44bd6b9640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 0x7f44b813f090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c7f0 tx=0x7f44b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.041+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4015070 con 0x7f44b806b120 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.041+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4004240 con 0x7f44b806b120 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.041+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b8140c30 con 0x7f44b806b120 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.041+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4007620 con 0x7f44b806b120 2024-04-25T09:55:15.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.041+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b81405e0 con 0x7f44b806b120 2024-04-25T09:55:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.042+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b806cc80 con 0x7f44b806b120 2024-04-25T09:55:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.042+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44b40079d0 con 0x7f44b806b120 2024-04-25T09:55:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.043+0000 7f44adffb640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f449c06b4d0 0x7f449c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.043+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f44b408aa90 con 0x7f44b806b120 2024-04-25T09:55:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.043+0000 7f44bceb8640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f449c06b4d0 0x7f449c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.043+0000 7f44bceb8640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f449c06b4d0 0x7f449c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f44a8034000 tx=0x7f44a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.047+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44b4059500 con 0x7f44b806b120 2024-04-25T09:55:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.216+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f44b8073a00 con 0x7f44b806b120 2024-04-25T09:55:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.217+0000 7f44adffb640 1 -- 172.21.15.148:0/379130997 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f44b4058e00 con 0x7f44b806b120 2024-04-25T09:55:15.220 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f449c06b4d0 msgr2=0x7f449c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f449c06b4d0 0x7f449c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f44a8034000 tx=0x7f44a802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 msgr2=0x7f44b813f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 0x7f44b813f090 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c7f0 tx=0x7f44b400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 shutdown_connections 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f449c06b4d0 0x7f449c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b815c970 0x7f44b81e5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b81454a0 0x7f44b809a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 --2- 172.21.15.148:0/379130997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b806b120 0x7f44b813f090 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 >> 172.21.15.148:0/379130997 conn(0x7f44b80b5af0 msgr2=0x7f44b80b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:15.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 shutdown_connections 2024-04-25T09:55:15.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.219+0000 7f44bf91d640 1 -- 172.21.15.148:0/379130997 wait complete. 2024-04-25T09:55:15.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:55:15.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.328+0000 7fae6731d640 1 Processor -- start 2024-04-25T09:55:15.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.328+0000 7fae6731d640 1 -- start start 2024-04-25T09:55:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae6731d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae6006e360 0x7fae6006e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae6731d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6015e020 con 0x7fae6006bd20 2024-04-25T09:55:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae6731d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6006ac80 con 0x7fae6006ec80 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae6731d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6006b890 con 0x7fae6006e360 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae650b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fae6006e360 0x7fae6006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae658ba640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fae6006bd20 0x7fae600acea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55558/0 (socket says 172.21.15.148:55558) 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae658ba640 1 -- 172.21.15.148:0/963144938 learned_addr learned my addr 172.21.15.148:0/963144938 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae650b9640 1 -- 172.21.15.148:0/963144938 >> v1:172.21.15.188:6789/0 conn(0x7fae6006ec80 legacy=0x7fae600b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae650b9640 1 -- 172.21.15.148:0/963144938 >> v1:172.21.15.148:6789/0 conn(0x7fae6006bd20 legacy=0x7fae600acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.329+0000 7fae650b9640 1 -- 172.21.15.148:0/963144938 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6006af60 con 0x7fae6006e360 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.330+0000 7fae650b9640 1 --2- 172.21.15.148:0/963144938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006e360 0x7fae6006e740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0097a0 tx=0x7fae5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34bb73bb2a9c283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.330+0000 7fae57fff640 1 -- 172.21.15.148:0/963144938 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c037070 con 0x7fae6006e360 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.330+0000 7fae57fff640 1 -- 172.21.15.148:0/963144938 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c032440 con 0x7fae6006e360 2024-04-25T09:55:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.330+0000 7fae57fff640 1 -- 172.21.15.148:0/963144938 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c032810 con 0x7fae6006e360 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.330+0000 7fae6731d640 1 -- 172.21.15.148:0/963144938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006e360 msgr2=0x7fae6006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.330+0000 7fae6731d640 1 --2- 172.21.15.148:0/963144938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006e360 0x7fae6006e740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0097a0 tx=0x7fae5c02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.331+0000 7fae6731d640 1 -- 172.21.15.148:0/963144938 shutdown_connections 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.331+0000 7fae6731d640 1 --2- 172.21.15.148:0/963144938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006e360 0x7fae6006e740 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.331+0000 7fae6731d640 1 -- 172.21.15.148:0/963144938 >> 172.21.15.148:0/963144938 conn(0x7fae600cfec0 msgr2=0x7fae600d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.331+0000 7fae6731d640 1 -- 172.21.15.148:0/963144938 shutdown_connections 2024-04-25T09:55:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.331+0000 7fae6731d640 1 -- 172.21.15.148:0/963144938 wait complete. 2024-04-25T09:55:15.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 Processor -- start 2024-04-25T09:55:15.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 -- start start 2024-04-25T09:55:15.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae6006bd20 0x7fae60144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae6006e360 0x7fae601f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 0x7fae601f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae600b2fd0 con 0x7fae6006ec80 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6006b520 con 0x7fae6006bd20 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.332+0000 7fae6731d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae600b3150 con 0x7fae6006e360 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 0x7fae601f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 0x7fae601f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35338/0 (socket says 172.21.15.148:35338) 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 -- 172.21.15.148:0/3574214606 learned_addr learned my addr 172.21.15.148:0/3574214606 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae650b9640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae6006bd20 0x7fae60144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae648b8640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae6006e360 0x7fae601f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 -- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae6006e360 msgr2=0x7fae601f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae6006e360 0x7fae601f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 -- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae6006bd20 msgr2=0x7fae60144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae6006bd20 0x7fae60144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 -- 172.21.15.148:0/3574214606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae60144c90 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae650b9640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae6006bd20 0x7fae60144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae658ba640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 0x7fae601f7340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fae5800cbc0 tx=0x7fae5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58017070 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5800ad40 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae60144f70 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58012600 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.333+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae60145530 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.334+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae60145a10 con 0x7fae6006ec80 2024-04-25T09:55:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.335+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae5801a070 con 0x7fae6006ec80 2024-04-25T09:55:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.335+0000 7fae55ffb640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae4406b4d0 0x7fae4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.335+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fae5808a270 con 0x7fae6006ec80 2024-04-25T09:55:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.335+0000 7fae650b9640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae4406b4d0 0x7fae4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.336+0000 7fae650b9640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae4406b4d0 0x7fae4406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.339+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae60145a10 con 0x7fae6006ec80 2024-04-25T09:55:15.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.520+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae60073a00 con 0x7fae6006ec80 2024-04-25T09:55:15.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.521+0000 7fae55ffb640 1 -- 172.21.15.148:0/3574214606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae58058690 con 0x7fae6006ec80 2024-04-25T09:55:15.524 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae4406b4d0 msgr2=0x7fae4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae4406b4d0 0x7fae4406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae5c034000 tx=0x7fae5c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 msgr2=0x7fae601f7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 0x7fae601f7340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fae5800cbc0 tx=0x7fae5800a430 comp rx=0 tx=0).stop 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 shutdown_connections 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fae4406b4d0 0x7fae4406d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6006ec80 0x7fae601f7340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae6006e360 0x7fae601f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 --2- 172.21.15.148:0/3574214606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae6006bd20 0x7fae60144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 >> 172.21.15.148:0/3574214606 conn(0x7fae600cfec0 msgr2=0x7fae600ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 shutdown_connections 2024-04-25T09:55:15.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.523+0000 7fae6731d640 1 -- 172.21.15.148:0/3574214606 wait complete. 2024-04-25T09:55:15.536 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T09:55:15.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:15.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T09:55:15.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:55:15.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:15.568 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:55:15.649 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:15.650 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:15.665 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:15.665 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:15.720 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:15.720 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:15.778 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.913+0000 7fa8752cd640 1 Processor -- start 2024-04-25T09:55:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.913+0000 7fa8752cd640 1 -- start start 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa8752cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa87015dd20 0x7fa8700b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa8752cd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8700d11b0 0x7fa8700d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa8752cd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87015eac0 con 0x7fa87015e100 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa8752cd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700d1c00 con 0x7fa87015dd20 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa8752cd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87015eda0 con 0x7fa8700d11b0 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa87015dd20 0x7fa8700b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa87015dd20 0x7fa8700b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35350/0 (socket says 172.21.15.148:35350) 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86e7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa87015e100 0x7fa8700d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55572/0 (socket says 172.21.15.148:55572) 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 -- 172.21.15.148:0/958235286 learned_addr learned my addr 172.21.15.148:0/958235286 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 -- 172.21.15.148:0/958235286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8700d11b0 msgr2=0x7fa8700d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 --2- 172.21.15.148:0/958235286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8700d11b0 0x7fa8700d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 -- 172.21.15.148:0/958235286 >> v1:172.21.15.148:6789/0 conn(0x7fa87015e100 legacy=0x7fa8700d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86effd640 1 -- 172.21.15.148:0/958235286 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87015f080 con 0x7fa87015dd20 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.914+0000 7fa86dffb640 1 -- 172.21.15.148:0/958235286 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829831223 0 0) 0x7fa87015eac0 con 0x7fa87015e100 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.915+0000 7fa86effd640 1 --2- 172.21.15.148:0/958235286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa87015dd20 0x7fa8700b96d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa86002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a922516573a4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.915+0000 7fa86dffb640 1 -- 172.21.15.148:0/958235286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860037070 con 0x7fa87015dd20 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.915+0000 7fa86dffb640 1 -- 172.21.15.148:0/958235286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860032440 con 0x7fa87015dd20 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.915+0000 7fa86dffb640 1 -- 172.21.15.148:0/958235286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860032810 con 0x7fa87015dd20 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 -- 172.21.15.148:0/958235286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa87015dd20 msgr2=0x7fa8700b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 --2- 172.21.15.148:0/958235286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa87015dd20 0x7fa8700b96d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa86002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 -- 172.21.15.148:0/958235286 shutdown_connections 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 --2- 172.21.15.148:0/958235286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8700d11b0 0x7fa8700d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 --2- 172.21.15.148:0/958235286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa87015dd20 0x7fa8700b96d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 -- 172.21.15.148:0/958235286 >> 172.21.15.148:0/958235286 conn(0x7fa870061a20 msgr2=0x7fa870061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 -- 172.21.15.148:0/958235286 shutdown_connections 2024-04-25T09:55:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.916+0000 7fa8752cd640 1 -- 172.21.15.148:0/958235286 wait complete. 2024-04-25T09:55:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.917+0000 7fa8752cd640 1 Processor -- start 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.917+0000 7fa8752cd640 1 -- start start 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa8752cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 0x7fa8701f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa8752cd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87015dd20 0x7fa8700c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa8752cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87015e100 0x7fa8700d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa8752cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87015e710 con 0x7fa8700d11b0 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa8752cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87015e590 con 0x7fa87015e100 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa8752cd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700c8bd0 con 0x7fa87015dd20 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 0x7fa8701f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 0x7fa8701f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35354/0 (socket says 172.21.15.148:35354) 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 -- 172.21.15.148:0/1037337428 learned_addr learned my addr 172.21.15.148:0/1037337428 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86f7fe640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87015e100 0x7fa8700d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 -- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87015dd20 msgr2=0x7fa8700c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87015dd20 0x7fa8700c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 -- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87015e100 msgr2=0x7fa8700d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87015e100 0x7fa8700d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 -- 172.21.15.148:0/1037337428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700c8ea0 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.918+0000 7fa86effd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 0x7fa8701f44d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa8600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.919+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860041070 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.919+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86003c870 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.919+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8700c9180 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.919+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860037040 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.919+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8700c9740 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.920+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa870073990 con 0x7fa8700d11b0 2024-04-25T09:55:15.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.920+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa860044070 con 0x7fa8700d11b0 2024-04-25T09:55:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.921+0000 7fa8577fe640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.921+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fa8600ab210 con 0x7fa8700d11b0 2024-04-25T09:55:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.921+0000 7fa86e7fc640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.921+0000 7fa86e7fc640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85c06b4d0 0x7fa85c06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa864000c00 tx=0x7fa8640079c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:15.925+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa860079c80 con 0x7fa8700d11b0 2024-04-25T09:55:16.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.098+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8700cc170 con 0x7fa8700d11b0 2024-04-25T09:55:16.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.100+0000 7fa8577fe640 1 -- 172.21.15.148:0/1037337428 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7fa860079580 con 0x7fa8700d11b0 2024-04-25T09:55:16.103 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:16.103 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:16.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85c06b4d0 msgr2=0x7fa85c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85c06b4d0 0x7fa85c06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa864000c00 tx=0x7fa8640079c0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 msgr2=0x7fa8701f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 0x7fa8701f44d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa8600046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 shutdown_connections 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa87015e100 0x7fa8700d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87015dd20 0x7fa8700c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 --2- 172.21.15.148:0/1037337428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8700d11b0 0x7fa8701f44d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 >> 172.21.15.148:0/1037337428 conn(0x7fa870061a20 msgr2=0x7fa8700b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 shutdown_connections 2024-04-25T09:55:16.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.102+0000 7fa8752cd640 1 -- 172.21.15.148:0/1037337428 wait complete. 2024-04-25T09:55:16.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:16.116 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:16.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.215+0000 7fd4fba38640 1 Processor -- start 2024-04-25T09:55:16.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.215+0000 7fd4fba38640 1 -- start start 2024-04-25T09:55:16.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4fba38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4f415b7d0 0x7fd4f40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4fba38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4f406c9e0 0x7fd4f40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4fba38640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f406cf50 con 0x7fd4f40b5620 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4fba38640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f40ce530 con 0x7fd4f415b7d0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4fba38640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f40ce810 con 0x7fd4f406c9e0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f97d4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4f415b7d0 0x7fd4f40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f8fd3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd4f40b5620 0x7fd4f406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55586/0 (socket says 172.21.15.148:55586) 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f8fd3640 1 -- 172.21.15.148:0/864631772 learned_addr learned my addr 172.21.15.148:0/864631772 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f97d4640 1 -- 172.21.15.148:0/864631772 >> v2:172.21.15.188:3300/0 conn(0x7fd4f406c9e0 msgr2=0x7fd4f40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f9fd5640 1 --2- 172.21.15.148:0/864631772 >> v2:172.21.15.188:3300/0 conn(0x7fd4f406c9e0 0x7fd4f40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f97d4640 1 --2- 172.21.15.148:0/864631772 >> v2:172.21.15.188:3300/0 conn(0x7fd4f406c9e0 0x7fd4f40b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f97d4640 1 -- 172.21.15.148:0/864631772 >> v1:172.21.15.148:6789/0 conn(0x7fd4f40b5620 legacy=0x7fd4f406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4ebfff640 1 -- 172.21.15.148:0/864631772 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980629770 0 0) 0x7fd4f406cf50 con 0x7fd4f40b5620 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.216+0000 7fd4f97d4640 1 -- 172.21.15.148:0/864631772 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40ceaf0 con 0x7fd4f415b7d0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4f97d4640 1 --2- 172.21.15.148:0/864631772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f415b7d0 0x7fd4f40cf010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4161890 tx=0x7fd4f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ace281a9e5140f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4ebfff640 1 -- 172.21.15.148:0/864631772 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0037070 con 0x7fd4f415b7d0 2024-04-25T09:55:16.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4ebfff640 1 -- 172.21.15.148:0/864631772 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0032440 con 0x7fd4f415b7d0 2024-04-25T09:55:16.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4ebfff640 1 -- 172.21.15.148:0/864631772 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0032810 con 0x7fd4f415b7d0 2024-04-25T09:55:16.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4fba38640 1 -- 172.21.15.148:0/864631772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f415b7d0 msgr2=0x7fd4f40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4fba38640 1 --2- 172.21.15.148:0/864631772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f415b7d0 0x7fd4f40cf010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4161890 tx=0x7fd4f002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4fba38640 1 -- 172.21.15.148:0/864631772 shutdown_connections 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4fba38640 1 --2- 172.21.15.148:0/864631772 >> v2:172.21.15.188:3300/0 conn(0x7fd4f406c9e0 0x7fd4f40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4fba38640 1 --2- 172.21.15.148:0/864631772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f415b7d0 0x7fd4f40cf010 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.217+0000 7fd4fba38640 1 -- 172.21.15.148:0/864631772 >> 172.21.15.148:0/864631772 conn(0x7fd4f40d0800 msgr2=0x7fd4f40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.218+0000 7fd4fba38640 1 -- 172.21.15.148:0/864631772 shutdown_connections 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.218+0000 7fd4fba38640 1 -- 172.21.15.148:0/864631772 wait complete. 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 Processor -- start 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 -- start start 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 0x7fd4f41f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f40b5620 0x7fd4f40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f415b7d0 0x7fd4f40b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40cf6d0 con 0x7fd4f406c9e0 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40cf550 con 0x7fd4f40b5620 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4fba38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f415bbb0 con 0x7fd4f415b7d0 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4f97d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 0x7fd4f41f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4f97d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 0x7fd4f41f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35368/0 (socket says 172.21.15.148:35368) 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.219+0000 7fd4f97d4640 1 -- 172.21.15.148:0/1615602274 learned_addr learned my addr 172.21.15.148:0/1615602274 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4f97d4640 1 -- 172.21.15.148:0/1615602274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f415b7d0 msgr2=0x7fd4f40b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4f97d4640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f415b7d0 0x7fd4f40b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4f97d4640 1 -- 172.21.15.148:0/1615602274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f40b5620 msgr2=0x7fd4f40cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4f97d4640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f40b5620 0x7fd4f40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4f97d4640 1 -- 172.21.15.148:0/1615602274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f40c9c60 con 0x7fd4f406c9e0 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4f97d4640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 0x7fd4f41f6a60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0037040 con 0x7fd4f406c9e0 2024-04-25T09:55:16.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0042070 con 0x7fd4f406c9e0 2024-04-25T09:55:16.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f40c9f40 con 0x7fd4f406c9e0 2024-04-25T09:55:16.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0030a50 con 0x7fd4f406c9e0 2024-04-25T09:55:16.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.220+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f41f6fa0 con 0x7fd4f406c9e0 2024-04-25T09:55:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.222+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4073a00 con 0x7fd4f406c9e0 2024-04-25T09:55:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.222+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4f0030bf0 con 0x7fd4f406c9e0 2024-04-25T09:55:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.222+0000 7fd4e9ffb640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4d806b500 0x7fd4d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.222+0000 7fd4f8fd3640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4d806b500 0x7fd4d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.222+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd4f00ab060 con 0x7fd4f406c9e0 2024-04-25T09:55:16.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.223+0000 7fd4f8fd3640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4d806b500 0x7fd4d806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007fd0 tx=0x7fd4e4007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.227+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4f0079c80 con 0x7fd4f406c9e0 2024-04-25T09:55:16.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.397+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4f40d0f60 con 0x7fd4f406c9e0 2024-04-25T09:55:16.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.398+0000 7fd4e9ffb640 1 -- 172.21.15.148:0/1615602274 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4f0079580 con 0x7fd4f406c9e0 2024-04-25T09:55:16.400 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4d806b500 msgr2=0x7fd4d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4d806b500 0x7fd4d806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007fd0 tx=0x7fd4e4007920 comp rx=0 tx=0).stop 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 msgr2=0x7fd4f41f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 0x7fd4f41f6a60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f00046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 shutdown_connections 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4d806b500 0x7fd4d806d9c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f415b7d0 0x7fd4f40b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f40b5620 0x7fd4f40cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 --2- 172.21.15.148:0/1615602274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4f406c9e0 0x7fd4f41f6a60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 >> 172.21.15.148:0/1615602274 conn(0x7fd4f40d0800 msgr2=0x7fd4f415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 shutdown_connections 2024-04-25T09:55:16.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.400+0000 7fd4fba38640 1 -- 172.21.15.148:0/1615602274 wait complete. 2024-04-25T09:55:16.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.511+0000 7f59c3f17640 1 Processor -- start 2024-04-25T09:55:16.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.511+0000 7f59c3f17640 1 -- start start 2024-04-25T09:55:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c3f17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc15b7d0 0x7f59bc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c3f17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59bc0b5620 0x7f59bc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c3f17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc06cf50 con 0x7f59bc06c2d0 2024-04-25T09:55:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c3f17640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0b5fc0 con 0x7f59bc0b5620 2024-04-25T09:55:16.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c3f17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0ce530 con 0x7f59bc15b7d0 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c14b2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59bc0b5620 0x7f59bc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c24b4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f59bc06c2d0 0x7f59bc06c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55590/0 (socket says 172.21.15.148:55590) 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c24b4640 1 -- 172.21.15.148:0/782071125 learned_addr learned my addr 172.21.15.148:0/782071125 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.512+0000 7f59c1cb3640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc15b7d0 0x7f59bc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c14b2640 1 -- 172.21.15.148:0/782071125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc15b7d0 msgr2=0x7f59bc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c14b2640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc15b7d0 0x7f59bc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c14b2640 1 -- 172.21.15.148:0/782071125 >> v1:172.21.15.148:6789/0 conn(0x7f59bc06c2d0 legacy=0x7f59bc06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c14b2640 1 -- 172.21.15.148:0/782071125 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0ce810 con 0x7f59bc0b5620 2024-04-25T09:55:16.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c1cb3640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc15b7d0 0x7f59bc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c14b2640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc0b5a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0097a0 tx=0x7f59ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111dbab1a9211f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c0cb1640 1 -- 172.21.15.148:0/782071125 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac037070 con 0x7f59bc0b5620 2024-04-25T09:55:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c0cb1640 1 -- 172.21.15.148:0/782071125 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac032440 con 0x7f59bc0b5620 2024-04-25T09:55:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.513+0000 7f59c0cb1640 1 -- 172.21.15.148:0/782071125 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac032810 con 0x7f59bc0b5620 2024-04-25T09:55:16.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 -- 172.21.15.148:0/782071125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 msgr2=0x7f59bc0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc0b5a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0097a0 tx=0x7f59ac02a180 comp rx=0 tx=0).stop 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 -- 172.21.15.148:0/782071125 shutdown_connections 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc0b5a80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 --2- 172.21.15.148:0/782071125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc15b7d0 0x7f59bc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 -- 172.21.15.148:0/782071125 >> 172.21.15.148:0/782071125 conn(0x7f59bc0d0800 msgr2=0x7f59bc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 -- 172.21.15.148:0/782071125 shutdown_connections 2024-04-25T09:55:16.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.514+0000 7f59c3f17640 1 -- 172.21.15.148:0/782071125 wait complete. 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.515+0000 7f59c3f17640 1 Processor -- start 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.515+0000 7f59c3f17640 1 -- start start 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c3f17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc06c2d0 0x7f59bc0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c3f17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c3f17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59bc15b7d0 0x7f59bc0b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c3f17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cf550 con 0x7f59bc0b5620 2024-04-25T09:55:16.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c3f17640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc06d230 con 0x7f59bc15b7d0 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c3f17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cf6d0 con 0x7f59bc06c2d0 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35388/0 (socket says 172.21.15.148:35388) 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 -- 172.21.15.148:0/3499221073 learned_addr learned my addr 172.21.15.148:0/3499221073 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c24b4640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59bc15b7d0 0x7f59bc0b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c1cb3640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc06c2d0 0x7f59bc0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 -- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc06c2d0 msgr2=0x7f59bc0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc06c2d0 0x7f59bc0c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 -- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59bc15b7d0 msgr2=0x7f59bc0b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59bc15b7d0 0x7f59bc0b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c14b2640 1 -- 172.21.15.148:0/3499221073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc1599d0 con 0x7f59bc0b5620 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c24b4640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59bc15b7d0 0x7f59bc0b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.516+0000 7f59c1cb3640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc06c2d0 0x7f59bc0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.517+0000 7f59c14b2640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc159490 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098c0 tx=0x7f59ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.517+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac037040 con 0x7f59bc0b5620 2024-04-25T09:55:16.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.517+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac030770 con 0x7f59bc0b5620 2024-04-25T09:55:16.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.517+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc0c9c70 con 0x7f59bc0b5620 2024-04-25T09:55:16.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.517+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac030b40 con 0x7f59bc0b5620 2024-04-25T09:55:16.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.517+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc0c7b50 con 0x7f59bc0b5620 2024-04-25T09:55:16.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.518+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc1616b0 con 0x7f59bc0b5620 2024-04-25T09:55:16.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.518+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59ac044020 con 0x7f59bc0b5620 2024-04-25T09:55:16.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.518+0000 7f59b27fc640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59a006b4d0 0x7f59a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.519+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f59ac0aabd0 con 0x7f59bc0b5620 2024-04-25T09:55:16.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.519+0000 7f59c1cb3640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59a006b4d0 0x7f59a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.519+0000 7f59c1cb3640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59a006b4d0 0x7f59a006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.523+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59bc1616b0 con 0x7f59bc0b5620 2024-04-25T09:55:16.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.694+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59bc073a00 con 0x7f59bc0b5620 2024-04-25T09:55:16.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.694+0000 7f59b27fc640 1 -- 172.21.15.148:0/3499221073 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59ac032e20 con 0x7f59bc0b5620 2024-04-25T09:55:16.697 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59a006b4d0 msgr2=0x7f59a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59a006b4d0 0x7f59a006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8008040 comp rx=0 tx=0).stop 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 msgr2=0x7f59bc159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc159490 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0098c0 tx=0x7f59ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 shutdown_connections 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59a006b4d0 0x7f59a006d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59bc15b7d0 0x7f59bc0b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59bc0b5620 0x7f59bc159490 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 --2- 172.21.15.148:0/3499221073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59bc06c2d0 0x7f59bc0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 >> 172.21.15.148:0/3499221073 conn(0x7f59bc0d0800 msgr2=0x7f59bc06c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 shutdown_connections 2024-04-25T09:55:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.696+0000 7f59c3f17640 1 -- 172.21.15.148:0/3499221073 wait complete. 2024-04-25T09:55:16.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:16.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.806+0000 7f348761f640 1 Processor -- start 2024-04-25T09:55:16.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.807+0000 7f348761f640 1 -- start start 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.807+0000 7f348761f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f34800ce230 0x7f34800cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.807+0000 7f348761f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3480072280 con 0x7f348015b270 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.807+0000 7f348761f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34800714c0 con 0x7f34800cc4f0 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.807+0000 7f348761f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3480071270 con 0x7f34800ce230 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3485bbc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f348015b270 0x7f348015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55594/0 (socket says 172.21.15.148:55594) 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3485bbc640 1 -- 172.21.15.148:0/755852583 learned_addr learned my addr 172.21.15.148:0/755852583 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3484bba640 1 --2- 172.21.15.148:0/755852583 >> v2:172.21.15.188:3300/0 conn(0x7f34800ce230 0x7f34800cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096354229 0 0) 0x7f3480072280 con 0x7f348015b270 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003520 con 0x7f348015b270 2024-04-25T09:55:16.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 838980033 0 0) 0x7f346c003520 con 0x7f348015b270 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 >> v2:172.21.15.188:3300/0 conn(0x7f34800ce230 msgr2=0x7f34800cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 --2- 172.21.15.148:0/755852583 >> v2:172.21.15.188:3300/0 conn(0x7f34800ce230 0x7f34800cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 >> v1:172.21.15.188:6789/0 conn(0x7f34800cc4f0 legacy=0x7f34800cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800717a0 con 0x7f348015b270 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f34780019f0 con 0x7f348015b270 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 >> v1:172.21.15.148:6789/0 conn(0x7f348015b270 legacy=0x7f348015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346c0051c0 0x7f346c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480071270 con 0x7f346c0051c0 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00c060 con 0x7f346c0039b0 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00c260 con 0x7f346c004800 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3485bbc640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346c0051c0 0x7f346c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f34853bb640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.808+0000 7f3484bba640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 -- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c004800 msgr2=0x7f346c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c004800 0x7f346c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 -- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0039b0 msgr2=0x7f346c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0039b0 0x7f346c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 -- 172.21.15.148:0/755852583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c003520 con 0x7f346c0051c0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 -- 172.21.15.148:0/755852583 reap_dead start 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3485bbc640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346c0051c0 0x7f346c00b9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f347800a860 tx=0x7f346c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcc50d976720ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800a480 con 0x7f346c0051c0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347802f070 con 0x7f346c0051c0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.809+0000 7f3473fff640 1 -- 172.21.15.148:0/755852583 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347802bcc0 con 0x7f346c0051c0 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 -- 172.21.15.148:0/755852583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346c0051c0 msgr2=0x7f346c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346c0051c0 0x7f346c00b9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f347800a860 tx=0x7f346c005830 comp rx=0 tx=0).stop 2024-04-25T09:55:16.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 -- 172.21.15.148:0/755852583 shutdown_connections 2024-04-25T09:55:16.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 --2- 172.21.15.148:0/755852583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f346c0051c0 0x7f346c00b9c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 -- 172.21.15.148:0/755852583 >> 172.21.15.148:0/755852583 conn(0x7f348006abe0 msgr2=0x7f348006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 -- 172.21.15.148:0/755852583 shutdown_connections 2024-04-25T09:55:16.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.810+0000 7f348761f640 1 -- 172.21.15.148:0/755852583 wait complete. 2024-04-25T09:55:16.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.811+0000 7f348761f640 1 Processor -- start 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.811+0000 7f348761f640 1 -- start start 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f348761f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0051c0 0x7f3480157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f348761f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34801f02b0 0x7f34800c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f348761f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 0x7f34800b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f348761f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34800cc8d0 con 0x7f34800b2d70 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f348761f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480071d60 con 0x7f346c0051c0 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f348761f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34800cca50 con 0x7f34801f02b0 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 0x7f34800b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 0x7f34800b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35410/0 (socket says 172.21.15.148:35410) 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 -- 172.21.15.148:0/4291643971 learned_addr learned my addr 172.21.15.148:0/4291643971 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f34853bb640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0051c0 0x7f3480157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3484bba640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34801f02b0 0x7f34800c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 -- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34801f02b0 msgr2=0x7f34800c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34801f02b0 0x7f34800c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 -- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0051c0 msgr2=0x7f3480157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0051c0 0x7f3480157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 -- 172.21.15.148:0/4291643971 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800c5b80 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.812+0000 7f3485bbc640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 0x7f34800b31f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f347800a980 tx=0x7f3480071690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.813+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478044070 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.813+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347802bdc0 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.813+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3480157980 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.813+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800a480 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.813+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3480155280 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.814+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3480073a00 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.814+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3478047070 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.815+0000 7f3471ffb640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f346406b500 0x7f346406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.815+0000 7f34853bb640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f346406b500 0x7f346406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.815+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f34780aa030 con 0x7f34800b2d70 2024-04-25T09:55:16.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.815+0000 7f34853bb640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f346406b500 0x7f346406d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f347c000c00 tx=0x7f347c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:16.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.819+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3478078570 con 0x7f34800b2d70 2024-04-25T09:55:16.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.986+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f348006d3a0 con 0x7f34800b2d70 2024-04-25T09:55:16.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.987+0000 7f3471ffb640 1 -- 172.21.15.148:0/4291643971 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f347807c610 con 0x7f34800b2d70 2024-04-25T09:55:16.990 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:16.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f346406b500 msgr2=0x7f346406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f346406b500 0x7f346406d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f347c000c00 tx=0x7f347c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 msgr2=0x7f34800b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 0x7f34800b31f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f347800a980 tx=0x7f3480071690 comp rx=0 tx=0).stop 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 shutdown_connections 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f346406b500 0x7f346406d9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34800b2d70 0x7f34800b31f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34801f02b0 0x7f34800c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 --2- 172.21.15.148:0/4291643971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0051c0 0x7f3480157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 >> 172.21.15.148:0/4291643971 conn(0x7f348006abe0 msgr2=0x7f34800b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 shutdown_connections 2024-04-25T09:55:16.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:16.989+0000 7f348761f640 1 -- 172.21.15.148:0/4291643971 wait complete. 2024-04-25T09:55:17.002 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:17.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 Processor -- start 2024-04-25T09:55:17.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 -- start start 2024-04-25T09:55:17.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b015de80 0x7f43b00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43b00cefb0 0x7f43b00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b00afc80 con 0x7f43b00cf980 2024-04-25T09:55:17.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00ce4d0 con 0x7f43b015de80 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.101+0000 7f43b7163640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00ce7b0 con 0x7f43b00cefb0 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b4eff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b015de80 0x7f43b00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b5700640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f43b00cf980 0x7f43b006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55606/0 (socket says 172.21.15.148:55606) 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b4eff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43b015de80 0x7f43b00cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35420/0 (socket says 172.21.15.148:35420) 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b5700640 1 -- 172.21.15.148:0/1580993150 learned_addr learned my addr 172.21.15.148:0/1580993150 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43affff640 1 --2- 172.21.15.148:0/1580993150 >> v2:172.21.15.188:3300/0 conn(0x7f43b00cefb0 0x7f43b00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b4eff640 1 -- 172.21.15.148:0/1580993150 >> v2:172.21.15.188:3300/0 conn(0x7f43b00cefb0 msgr2=0x7f43b00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b4eff640 1 --2- 172.21.15.148:0/1580993150 >> v2:172.21.15.188:3300/0 conn(0x7f43b00cefb0 0x7f43b00cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b4eff640 1 -- 172.21.15.148:0/1580993150 >> v1:172.21.15.148:6789/0 conn(0x7f43b00cf980 legacy=0x7f43b006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43af7fe640 1 -- 172.21.15.148:0/1580993150 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562956173 0 0) 0x7f43b00afc80 con 0x7f43b00cf980 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43b4eff640 1 -- 172.21.15.148:0/1580993150 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00cea90 con 0x7f43b015de80 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.102+0000 7f43affff640 1 --2- 172.21.15.148:0/1580993150 >> v2:172.21.15.188:3300/0 conn(0x7f43b00cefb0 0x7f43b00cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b4eff640 1 --2- 172.21.15.148:0/1580993150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b00cadb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f43b0062500 tx=0x7f43a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24792f230e2731d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43af7fe640 1 -- 172.21.15.148:0/1580993150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0037070 con 0x7f43b015de80 2024-04-25T09:55:17.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43af7fe640 1 -- 172.21.15.148:0/1580993150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a0032440 con 0x7f43b015de80 2024-04-25T09:55:17.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43af7fe640 1 -- 172.21.15.148:0/1580993150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0032810 con 0x7f43b015de80 2024-04-25T09:55:17.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 -- 172.21.15.148:0/1580993150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 msgr2=0x7f43b00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 --2- 172.21.15.148:0/1580993150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b00cadb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f43b0062500 tx=0x7f43a002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 -- 172.21.15.148:0/1580993150 shutdown_connections 2024-04-25T09:55:17.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 --2- 172.21.15.148:0/1580993150 >> v2:172.21.15.188:3300/0 conn(0x7f43b00cefb0 0x7f43b00cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 --2- 172.21.15.148:0/1580993150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b00cadb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 -- 172.21.15.148:0/1580993150 >> 172.21.15.148:0/1580993150 conn(0x7f43b00b2390 msgr2=0x7f43b00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 -- 172.21.15.148:0/1580993150 shutdown_connections 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.103+0000 7f43b7163640 1 -- 172.21.15.148:0/1580993150 wait complete. 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.104+0000 7f43b7163640 1 Processor -- start 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 -- start start 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cefb0 0x7f43b00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00cf980 0x7f43b01f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b01f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b3030 con 0x7f43b015de80 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b2eb0 con 0x7f43b00cf980 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b7163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c5600 con 0x7f43b00cefb0 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.105+0000 7f43b5700640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b01f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b01f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35426/0 (socket says 172.21.15.148:35426) 2024-04-25T09:55:17.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 -- 172.21.15.148:0/487638844 learned_addr learned my addr 172.21.15.148:0/487638844 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b4eff640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cefb0 0x7f43b00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43affff640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00cf980 0x7f43b01f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 -- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cefb0 msgr2=0x7f43b00b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cefb0 0x7f43b00b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 -- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00cf980 msgr2=0x7f43b01f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00cf980 0x7f43b01f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 -- 172.21.15.148:0/487638844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00b6540 con 0x7f43b015de80 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b4eff640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cefb0 0x7f43b00b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b5700640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b01f6700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c810 tx=0x7f43a800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8018070 con 0x7f43b015de80 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800a5e0 con 0x7f43b015de80 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b0157610 con 0x7f43b015de80 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800a9d0 con 0x7f43b015de80 2024-04-25T09:55:17.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.106+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b0154920 con 0x7f43b015de80 2024-04-25T09:55:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.107+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00739e0 con 0x7f43b015de80 2024-04-25T09:55:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.108+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43a801b070 con 0x7f43b015de80 2024-04-25T09:55:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.108+0000 7f43ad7fa640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f439c06b500 0x7f439c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.108+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f43a8089b00 con 0x7f43b015de80 2024-04-25T09:55:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.108+0000 7f43b4eff640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f439c06b500 0x7f439c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.109+0000 7f43b4eff640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f439c06b500 0x7f439c06d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43a0034000 tx=0x7f43a002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.112+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43b00b6540 con 0x7f43b015de80 2024-04-25T09:55:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.284+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f43b006d330 con 0x7f43b015de80 2024-04-25T09:55:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.285+0000 7f43ad7fa640 1 -- 172.21.15.148:0/487638844 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f43a8058600 con 0x7f43b015de80 2024-04-25T09:55:17.287 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:17.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.286+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f439c06b500 msgr2=0x7f439c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f439c06b500 0x7f439c06d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43a0034000 tx=0x7f43a002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 msgr2=0x7f43b01f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b01f6700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c810 tx=0x7f43a800ccd0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 shutdown_connections 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f439c06b500 0x7f439c06d9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43b015de80 0x7f43b01f6700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00cf980 0x7f43b01f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 --2- 172.21.15.148:0/487638844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cefb0 0x7f43b00b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 >> 172.21.15.148:0/487638844 conn(0x7f43b00b2390 msgr2=0x7f43b015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 shutdown_connections 2024-04-25T09:55:17.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.287+0000 7f43b7163640 1 -- 172.21.15.148:0/487638844 wait complete. 2024-04-25T09:55:17.300 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:55:17.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.395+0000 7f195bf3b640 1 Processor -- start 2024-04-25T09:55:17.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.396+0000 7f195bf3b640 1 -- start start 2024-04-25T09:55:17.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.396+0000 7f195bf3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19540ced80 0x7f19540d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.396+0000 7f195bf3b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1954161710 0x7f19540d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.396+0000 7f195bf3b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195415e000 con 0x7f195415dc20 2024-04-25T09:55:17.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.396+0000 7f195bf3b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19540d1de0 con 0x7f19540ced80 2024-04-25T09:55:17.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.396+0000 7f195bf3b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1954161c60 con 0x7f1954161710 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f19594d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19540ced80 0x7f19540d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f1959cd7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f195415dc20 0x7f19540ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55612/0 (socket says 172.21.15.148:55612) 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f1959cd7640 1 -- 172.21.15.148:0/1792339718 learned_addr learned my addr 172.21.15.148:0/1792339718 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f19594d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19540ced80 0x7f19540d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35430/0 (socket says 172.21.15.148:35430) 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f195a4d8640 1 --2- 172.21.15.148:0/1792339718 >> v2:172.21.15.188:3300/0 conn(0x7f1954161710 0x7f19540d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f19594d6640 1 -- 172.21.15.148:0/1792339718 >> v2:172.21.15.188:3300/0 conn(0x7f1954161710 msgr2=0x7f19540d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f19594d6640 1 --2- 172.21.15.148:0/1792339718 >> v2:172.21.15.188:3300/0 conn(0x7f1954161710 0x7f19540d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f1958cd5640 1 -- 172.21.15.148:0/1792339718 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801490764 0 0) 0x7f195415e000 con 0x7f195415dc20 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f19594d6640 1 -- 172.21.15.148:0/1792339718 >> v1:172.21.15.148:6789/0 conn(0x7f195415dc20 legacy=0x7f19540ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f19594d6640 1 -- 172.21.15.148:0/1792339718 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954161f40 con 0x7f19540ced80 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.397+0000 7f195a4d8640 1 --2- 172.21.15.148:0/1792339718 >> v2:172.21.15.188:3300/0 conn(0x7f1954161710 0x7f19540d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f19594d6640 1 --2- 172.21.15.148:0/1792339718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f19540d22e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19540b6570 tx=0x7f194402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebd82661c83ef61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f1958cd5640 1 -- 172.21.15.148:0/1792339718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944037070 con 0x7f19540ced80 2024-04-25T09:55:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f1958cd5640 1 -- 172.21.15.148:0/1792339718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1944032440 con 0x7f19540ced80 2024-04-25T09:55:17.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f1958cd5640 1 -- 172.21.15.148:0/1792339718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944032810 con 0x7f19540ced80 2024-04-25T09:55:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 -- 172.21.15.148:0/1792339718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 msgr2=0x7f19540d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 --2- 172.21.15.148:0/1792339718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f19540d22e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f19540b6570 tx=0x7f194402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 -- 172.21.15.148:0/1792339718 shutdown_connections 2024-04-25T09:55:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 --2- 172.21.15.148:0/1792339718 >> v2:172.21.15.188:3300/0 conn(0x7f1954161710 0x7f19540d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 --2- 172.21.15.148:0/1792339718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f19540d22e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 -- 172.21.15.148:0/1792339718 >> 172.21.15.148:0/1792339718 conn(0x7f19540d0420 msgr2=0x7f19540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 -- 172.21.15.148:0/1792339718 shutdown_connections 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.398+0000 7f195bf3b640 1 -- 172.21.15.148:0/1792339718 wait complete. 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 Processor -- start 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 -- start start 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f1954159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f195415dc20 0x7f19541f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1954161710 0x7f19540b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19540b5e50 con 0x7f19540ced80 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954162500 con 0x7f195415dc20 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f195bf3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19540b5fd0 con 0x7f1954161710 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f1959cd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f1954159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f1959cd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f1954159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35432/0 (socket says 172.21.15.148:35432) 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.400+0000 7f1959cd7640 1 -- 172.21.15.148:0/3416935011 learned_addr learned my addr 172.21.15.148:0/3416935011 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f1959cd7640 1 -- 172.21.15.148:0/3416935011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1954161710 msgr2=0x7f19540b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f1959cd7640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1954161710 0x7f19540b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f1959cd7640 1 -- 172.21.15.148:0/3416935011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f195415dc20 msgr2=0x7f19541f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f1959cd7640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f195415dc20 0x7f19541f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f1959cd7640 1 -- 172.21.15.148:0/3416935011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19541f48f0 con 0x7f19540ced80 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f1959cd7640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f1954159fa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f195000c460 tx=0x7f195000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950018070 con 0x7f19540ced80 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1950007500 con 0x7f19540ced80 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950004a20 con 0x7f19540ced80 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19540c7ad0 con 0x7f19540ced80 2024-04-25T09:55:17.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.401+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19540c8090 con 0x7f19540ced80 2024-04-25T09:55:17.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.402+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19540b2b70 con 0x7f19540ced80 2024-04-25T09:55:17.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.402+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1950004bc0 con 0x7f19540ced80 2024-04-25T09:55:17.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.403+0000 7f194a7fc640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f193806b4d0 0x7f193806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.403+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f1950089890 con 0x7f19540ced80 2024-04-25T09:55:17.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.403+0000 7f19594d6640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f193806b4d0 0x7f193806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.403+0000 7f19594d6640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f193806b4d0 0x7f193806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1944034000 tx=0x7f194402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.407+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19540b2b70 con 0x7f19540ced80 2024-04-25T09:55:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.588+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1954073a00 con 0x7f19540ced80 2024-04-25T09:55:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.589+0000 7f194a7fc640 1 -- 172.21.15.148:0/3416935011 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f19541f48f0 con 0x7f19540ced80 2024-04-25T09:55:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:55:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f193806b4d0 msgr2=0x7f193806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f193806b4d0 0x7f193806d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1944034000 tx=0x7f194402c000 comp rx=0 tx=0).stop 2024-04-25T09:55:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 msgr2=0x7f1954159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f1954159fa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f195000c460 tx=0x7f195000c920 comp rx=0 tx=0).stop 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 shutdown_connections 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f193806b4d0 0x7f193806d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1954161710 0x7f19540b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f195415dc20 0x7f19541f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 --2- 172.21.15.148:0/3416935011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19540ced80 0x7f1954159fa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.590+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 >> 172.21.15.148:0/3416935011 conn(0x7f19540d0420 msgr2=0x7f19540d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.591+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 shutdown_connections 2024-04-25T09:55:17.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.591+0000 7f195bf3b640 1 -- 172.21.15.148:0/3416935011 wait complete. 2024-04-25T09:55:17.604 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T09:55:17.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:17.604 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T09:55:17.629 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:55:17.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:17.629 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:55:17.684 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:17.684 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:17.703 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:17.703 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:17.761 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:55:17.761 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:17.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:17.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 Processor -- start 2024-04-25T09:55:17.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 -- start start 2024-04-25T09:55:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faca406abe0 0x7faca406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40ce330 0x7faca406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca41626e0 con 0x7faca40700a0 2024-04-25T09:55:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca406b500 con 0x7faca406abe0 2024-04-25T09:55:17.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7facaab89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca40cd270 con 0x7faca40ce330 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.915+0000 7faca8925640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faca406abe0 0x7faca406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca9126640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7faca40700a0 0x7faca406e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55628/0 (socket says 172.21.15.148:55628) 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca9126640 1 -- 172.21.15.148:0/1845837679 learned_addr learned my addr 172.21.15.148:0/1845837679 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca3fff640 1 --2- 172.21.15.148:0/1845837679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40ce330 0x7faca406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca8925640 1 -- 172.21.15.148:0/1845837679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40ce330 msgr2=0x7faca406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca8925640 1 --2- 172.21.15.148:0/1845837679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40ce330 0x7faca406fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca8925640 1 -- 172.21.15.148:0/1845837679 >> v1:172.21.15.148:6789/0 conn(0x7faca40700a0 legacy=0x7faca406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca37fe640 1 -- 172.21.15.148:0/1845837679 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455916344 0 0) 0x7faca41626e0 con 0x7faca40700a0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.916+0000 7faca8925640 1 -- 172.21.15.148:0/1845837679 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40cd550 con 0x7faca406abe0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7faca8925640 1 --2- 172.21.15.148:0/1845837679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca406abe0 0x7faca406afc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870b4d5dd1dd8b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7faca37fe640 1 -- 172.21.15.148:0/1845837679 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c031600 con 0x7faca406abe0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7faca37fe640 1 -- 172.21.15.148:0/1845837679 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c0317a0 con 0x7faca406abe0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7faca37fe640 1 -- 172.21.15.148:0/1845837679 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c031b70 con 0x7faca406abe0 2024-04-25T09:55:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 -- 172.21.15.148:0/1845837679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca406abe0 msgr2=0x7faca406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 --2- 172.21.15.148:0/1845837679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca406abe0 0x7faca406afc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009d90 tx=0x7fac9c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 -- 172.21.15.148:0/1845837679 shutdown_connections 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 --2- 172.21.15.148:0/1845837679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca40ce330 0x7faca406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 --2- 172.21.15.148:0/1845837679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca406abe0 0x7faca406afc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 -- 172.21.15.148:0/1845837679 >> 172.21.15.148:0/1845837679 conn(0x7faca40b2220 msgr2=0x7faca40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 -- 172.21.15.148:0/1845837679 shutdown_connections 2024-04-25T09:55:17.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.917+0000 7facaab89640 1 -- 172.21.15.148:0/1845837679 wait complete. 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.918+0000 7facaab89640 1 Processor -- start 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.918+0000 7facaab89640 1 -- start start 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7facaab89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406abe0 0x7faca41f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7facaab89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40700a0 0x7faca41f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7facaab89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 0x7faca415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7facaab89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4161780 con 0x7faca40ce330 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7facaab89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca40cdb10 con 0x7faca40700a0 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7facaab89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca406eb00 con 0x7faca406abe0 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7faca9126640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 0x7faca415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7faca9126640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 0x7faca415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35456/0 (socket says 172.21.15.148:35456) 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.919+0000 7faca9126640 1 -- 172.21.15.148:0/900554062 learned_addr learned my addr 172.21.15.148:0/900554062 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca8925640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406abe0 0x7faca41f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca3fff640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40700a0 0x7faca41f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca9126640 1 -- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406abe0 msgr2=0x7faca41f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca9126640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406abe0 0x7faca41f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca9126640 1 -- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40700a0 msgr2=0x7faca41f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca9126640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40700a0 0x7faca41f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca9126640 1 -- 172.21.15.148:0/900554062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca415ab70 con 0x7faca40ce330 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca3fff640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40700a0 0x7faca41f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca8925640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406abe0 0x7faca41f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca9126640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 0x7faca415a630 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c480 tx=0x7fac9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94018070 con 0x7faca40ce330 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac940074e0 con 0x7faca40ce330 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca415ae10 con 0x7faca40ce330 2024-04-25T09:55:17.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94004a20 con 0x7faca40ce330 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.920+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca415b3d0 con 0x7faca40ce330 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.921+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4073990 con 0x7faca40ce330 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.922+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac9401b070 con 0x7faca40ce330 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.922+0000 7faca17fa640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac8406b4d0 0x7fac8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.922+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fac94089810 con 0x7faca40ce330 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.922+0000 7faca8925640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac8406b4d0 0x7fac8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:17.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.922+0000 7faca8925640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac8406b4d0 0x7fac8406d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac9c034000 tx=0x7fac9c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:17.926+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac94058310 con 0x7faca40ce330 2024-04-25T09:55:18.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.097+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faca40b2ff0 con 0x7faca40ce330 2024-04-25T09:55:18.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.099+0000 7faca17fa640 1 -- 172.21.15.148:0/900554062 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7fac94057c10 con 0x7faca40ce330 2024-04-25T09:55:18.102 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:18.102 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac8406b4d0 msgr2=0x7fac8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac8406b4d0 0x7fac8406d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac9c034000 tx=0x7fac9c02f000 comp rx=0 tx=0).stop 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 msgr2=0x7faca415a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 0x7faca415a630 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c480 tx=0x7fac9400c940 comp rx=0 tx=0).stop 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 shutdown_connections 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac8406b4d0 0x7fac8406d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faca40ce330 0x7faca415a630 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca40700a0 0x7faca41f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 --2- 172.21.15.148:0/900554062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca406abe0 0x7faca41f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 >> 172.21.15.148:0/900554062 conn(0x7faca40b2220 msgr2=0x7faca40b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 shutdown_connections 2024-04-25T09:55:18.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.101+0000 7facaab89640 1 -- 172.21.15.148:0/900554062 wait complete. 2024-04-25T09:55:18.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:18.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:18.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.212+0000 7f7619a6f640 1 Processor -- start 2024-04-25T09:55:18.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.212+0000 7f7619a6f640 1 -- start start 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7619a6f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f761415b980 0x7f76140cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7619a6f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7614072280 con 0x7f76140cc4f0 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7619a6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76140714c0 con 0x7f76140ce230 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7619a6f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7614071270 con 0x7f761415b980 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f76137fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f76140cc4f0 0x7f76140cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55640/0 (socket says 172.21.15.148:55640) 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f76137fe640 1 -- 172.21.15.148:0/1965451498 learned_addr learned my addr 172.21.15.148:0/1965451498 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7613fff640 1 --2- 172.21.15.148:0/1965451498 >> v2:172.21.15.188:3300/0 conn(0x7f761415b980 0x7f76140cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7613fff640 1 -- 172.21.15.148:0/1965451498 >> v1:172.21.15.188:6789/0 conn(0x7f76140ce230 legacy=0x7f761415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7613fff640 1 -- 172.21.15.148:0/1965451498 >> v1:172.21.15.148:6789/0 conn(0x7f76140cc4f0 legacy=0x7f76140cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.213+0000 7f7613fff640 1 -- 172.21.15.148:0/1965451498 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140717a0 con 0x7f761415b980 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.214+0000 7f7613fff640 1 --2- 172.21.15.148:0/1965451498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761415b980 0x7f76140cc8d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f760c009d90 tx=0x7f760c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3766d4802277eac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.214+0000 7f76127fc640 1 -- 172.21.15.148:0/1965451498 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c007740 con 0x7f761415b980 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f76127fc640 1 -- 172.21.15.148:0/1965451498 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c0078e0 con 0x7f761415b980 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f76127fc640 1 -- 172.21.15.148:0/1965451498 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c007cb0 con 0x7f761415b980 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 -- 172.21.15.148:0/1965451498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761415b980 msgr2=0x7f76140cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1965451498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761415b980 0x7f76140cc8d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f760c009d90 tx=0x7f760c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 -- 172.21.15.148:0/1965451498 shutdown_connections 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1965451498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f761415b980 0x7f76140cc8d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 -- 172.21.15.148:0/1965451498 >> 172.21.15.148:0/1965451498 conn(0x7f761406abe0 msgr2=0x7f761406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 -- 172.21.15.148:0/1965451498 shutdown_connections 2024-04-25T09:55:18.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.215+0000 7f7619a6f640 1 -- 172.21.15.148:0/1965451498 wait complete. 2024-04-25T09:55:18.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.216+0000 7f7619a6f640 1 Processor -- start 2024-04-25T09:55:18.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 -- start start 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76140cc4f0 0x7f76141473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140ce230 0x7f76140af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 0x7f76141599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761406e8a0 con 0x7f761415b980 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7614071d60 con 0x7f76140ce230 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7619a6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76140ae500 con 0x7f76140cc4f0 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7613fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 0x7f76141599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7613fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 0x7f76141599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35472/0 (socket says 172.21.15.148:35472) 2024-04-25T09:55:18.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7613fff640 1 -- 172.21.15.148:0/1418340399 learned_addr learned my addr 172.21.15.148:0/1418340399 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f76137fe640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76140cc4f0 0x7f76141473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.217+0000 7f7612ffd640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140ce230 0x7f76140af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7613fff640 1 -- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76140cc4f0 msgr2=0x7f76141473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7613fff640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76140cc4f0 0x7f76141473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7613fff640 1 -- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140ce230 msgr2=0x7f76140af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7613fff640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140ce230 0x7f76140af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7613fff640 1 -- 172.21.15.148:0/1418340399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140b0210 con 0x7f761415b980 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7612ffd640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140ce230 0x7f76140af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7613fff640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 0x7f76141599a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f760c009eb0 tx=0x7f760c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c040070 con 0x7f761415b980 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760c031c90 con 0x7f761415b980 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76140b04f0 con 0x7f761415b980 2024-04-25T09:55:18.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760c03c640 con 0x7f761415b980 2024-04-25T09:55:18.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.218+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76140b0ab0 con 0x7f761415b980 2024-04-25T09:55:18.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.219+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76140ce960 con 0x7f761415b980 2024-04-25T09:55:18.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.219+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f760c039040 con 0x7f761415b980 2024-04-25T09:55:18.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.220+0000 7f75fbfff640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f760006b4d0 0x7f760006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.220+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f760c0abb80 con 0x7f761415b980 2024-04-25T09:55:18.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.220+0000 7f76137fe640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f760006b4d0 0x7f760006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.220+0000 7f76137fe640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f760006b4d0 0x7f760006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7604005880 tx=0x7f7604009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.224+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76140ce960 con 0x7f761415b980 2024-04-25T09:55:18.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.392+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76140b5a30 con 0x7f761415b980 2024-04-25T09:55:18.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.392+0000 7f75fbfff640 1 -- 172.21.15.148:0/1418340399 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76140b5a30 con 0x7f761415b980 2024-04-25T09:55:18.395 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f760006b4d0 msgr2=0x7f760006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f760006b4d0 0x7f760006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7604005880 tx=0x7f7604009040 comp rx=0 tx=0).stop 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 msgr2=0x7f76141599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 0x7f76141599a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f760c009eb0 tx=0x7f760c004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 shutdown_connections 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f760006b4d0 0x7f760006d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f761415b980 0x7f76141599a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76140ce230 0x7f76140af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 --2- 172.21.15.148:0/1418340399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76140cc4f0 0x7f76141473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 >> 172.21.15.148:0/1418340399 conn(0x7f761406abe0 msgr2=0x7f761415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 shutdown_connections 2024-04-25T09:55:18.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.394+0000 7f7619a6f640 1 -- 172.21.15.148:0/1418340399 wait complete. 2024-04-25T09:55:18.408 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:18.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.505+0000 7f8abfc3a640 1 Processor -- start 2024-04-25T09:55:18.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.505+0000 7f8abfc3a640 1 -- start start 2024-04-25T09:55:18.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abfc3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab8158230 0x7f8ab80d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abfc3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ab80b4e60 0x7f8ab80b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abfc3a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80d2020 con 0x7f8ab80d1b80 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abfc3a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab8158610 con 0x7f8ab80b4e60 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abfc3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab806bea0 con 0x7f8ab8158230 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ab80b4e60 0x7f8ab80b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ab80b4e60 0x7f8ab80b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35484/0 (socket says 172.21.15.148:35484) 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 -- 172.21.15.148:0/3776847795 learned_addr learned my addr 172.21.15.148:0/3776847795 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 -- 172.21.15.148:0/3776847795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab8158230 msgr2=0x7f8ab80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abd9d6640 1 --2- 172.21.15.148:0/3776847795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab8158230 0x7f8ab80d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 --2- 172.21.15.148:0/3776847795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab8158230 0x7f8ab80d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 -- 172.21.15.148:0/3776847795 >> v1:172.21.15.188:6789/0 conn(0x7f8ab80d1b80 legacy=0x7f8ab806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.506+0000 7f8abe1d7640 1 -- 172.21.15.148:0/3776847795 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80d2300 con 0x7f8ab80b4e60 2024-04-25T09:55:18.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.507+0000 7f8abe1d7640 1 --2- 172.21.15.148:0/3776847795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b4e60 0x7f8ab80b5270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009870 tx=0x7f8ab002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b8a01fbf5121dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.507+0000 7f8abc9d4640 1 -- 172.21.15.148:0/3776847795 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0037070 con 0x7f8ab80b4e60 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.507+0000 7f8abc9d4640 1 -- 172.21.15.148:0/3776847795 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab0032440 con 0x7f8ab80b4e60 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.507+0000 7f8abc9d4640 1 -- 172.21.15.148:0/3776847795 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0032810 con 0x7f8ab80b4e60 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3776847795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b4e60 msgr2=0x7f8ab80b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3776847795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b4e60 0x7f8ab80b5270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009870 tx=0x7f8ab002a210 comp rx=0 tx=0).stop 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3776847795 shutdown_connections 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3776847795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b4e60 0x7f8ab80b5270 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3776847795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab8158230 0x7f8ab80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3776847795 >> 172.21.15.148:0/3776847795 conn(0x7f8ab8061600 msgr2=0x7f8ab80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3776847795 shutdown_connections 2024-04-25T09:55:18.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.508+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3776847795 wait complete. 2024-04-25T09:55:18.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.509+0000 7f8abfc3a640 1 Processor -- start 2024-04-25T09:55:18.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.509+0000 7f8abfc3a640 1 -- start start 2024-04-25T09:55:18.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abfc3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b4e60 0x7f8ab81e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abfc3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 0x7f8ab81f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abfc3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab8158230 0x7f8ab81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abfc3a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80d25e0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abfc3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b57b0 con 0x7f8ab8158230 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abfc3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab806ca40 con 0x7f8ab80b4e60 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 0x7f8ab81f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 0x7f8ab81f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35498/0 (socket says 172.21.15.148:35498) 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 -- 172.21.15.148:0/3626017502 learned_addr learned my addr 172.21.15.148:0/3626017502 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abe1d7640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab8158230 0x7f8ab81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd9d6640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b4e60 0x7f8ab81e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 -- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b4e60 msgr2=0x7f8ab81e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b4e60 0x7f8ab81e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 -- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab8158230 msgr2=0x7f8ab81533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab8158230 0x7f8ab81533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 -- 172.21.15.148:0/3626017502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab81f4af0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abe1d7640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab8158230 0x7f8ab81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.510+0000 7f8abd1d5640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 0x7f8ab81f45b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8004640 tx=0x7f8aa800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.511+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8011790 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.511+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa8017070 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.511+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab81e5980 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.511+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8011bb0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.511+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab81e5f40 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.512+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab8073a00 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.512+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8aa8011d50 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.512+0000 7f8aae7fc640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.513+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f8aa808a0a0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.513+0000 7f8abd9d6640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.513+0000 7f8abd9d6640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80cacb0 tx=0x7f8ab40083c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.517+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ab81f4af0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.683+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ab8061dd0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.683+0000 7f8aae7fc640 1 -- 172.21.15.148:0/3626017502 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8aa8058ba0 con 0x7f8ab80d1b80 2024-04-25T09:55:18.686 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a9c06b500 msgr2=0x7f8a9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80cacb0 tx=0x7f8ab40083c0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 msgr2=0x7f8ab81f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 0x7f8ab81f45b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8004640 tx=0x7f8aa800c8e0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 shutdown_connections 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab8158230 0x7f8ab81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80d1b80 0x7f8ab81f45b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 --2- 172.21.15.148:0/3626017502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b4e60 0x7f8ab81e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.685+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 >> 172.21.15.148:0/3626017502 conn(0x7f8ab8061600 msgr2=0x7f8ab80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.686+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 shutdown_connections 2024-04-25T09:55:18.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.686+0000 7f8abfc3a640 1 -- 172.21.15.148:0/3626017502 wait complete. 2024-04-25T09:55:18.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:18.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 Processor -- start 2024-04-25T09:55:18.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 -- start start 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd11815bb50 0x7fd1180c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1180cfd10 0x7fd1180cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1180c75e0 con 0x7fd1180c70d0 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1180c8680 con 0x7fd1180cfd10 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd11d989640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180c78c0 con 0x7fd11815bb50 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.796+0000 7fd1177fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1180cfd10 0x7fd1180cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd1180cfd10 0x7fd1180cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35504/0 (socket says 172.21.15.148:35504) 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 -- 172.21.15.148:0/3805538565 learned_addr learned my addr 172.21.15.148:0/3805538565 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd116ffd640 1 --2- 172.21.15.148:0/3805538565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd11815bb50 0x7fd1180c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 -- 172.21.15.148:0/3805538565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd11815bb50 msgr2=0x7fd1180c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 --2- 172.21.15.148:0/3805538565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd11815bb50 0x7fd1180c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 -- 172.21.15.148:0/3805538565 >> v1:172.21.15.188:6789/0 conn(0x7fd1180c70d0 legacy=0x7fd1180cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 -- 172.21.15.148:0/3805538565 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180c7ba0 con 0x7fd1180cfd10 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.797+0000 7fd1177fe640 1 --2- 172.21.15.148:0/3805538565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180cfd10 0x7fd1180cbf20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd110009870 tx=0x7fd11002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92c7cf64ad10de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd115ffb640 1 -- 172.21.15.148:0/3805538565 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110037070 con 0x7fd1180cfd10 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd115ffb640 1 -- 172.21.15.148:0/3805538565 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11002ae50 con 0x7fd1180cfd10 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd115ffb640 1 -- 172.21.15.148:0/3805538565 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110032860 con 0x7fd1180cfd10 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 -- 172.21.15.148:0/3805538565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180cfd10 msgr2=0x7fd1180cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 --2- 172.21.15.148:0/3805538565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180cfd10 0x7fd1180cbf20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd110009870 tx=0x7fd11002a300 comp rx=0 tx=0).stop 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 -- 172.21.15.148:0/3805538565 shutdown_connections 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 --2- 172.21.15.148:0/3805538565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180cfd10 0x7fd1180cbf20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 --2- 172.21.15.148:0/3805538565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd11815bb50 0x7fd1180c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 -- 172.21.15.148:0/3805538565 >> 172.21.15.148:0/3805538565 conn(0x7fd11806f500 msgr2=0x7fd11806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 -- 172.21.15.148:0/3805538565 shutdown_connections 2024-04-25T09:55:18.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.798+0000 7fd11d989640 1 -- 172.21.15.148:0/3805538565 wait complete. 2024-04-25T09:55:18.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.799+0000 7fd11d989640 1 Processor -- start 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 -- start start 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 0x7fd1181484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1180cfd10 0x7fd1181e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11815bb50 0x7fd1180b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11815f6d0 con 0x7fd1180c70d0 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180c8160 con 0x7fd11815bb50 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd11d989640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11815f850 con 0x7fd1180cfd10 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd116ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 0x7fd1181484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd116ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 0x7fd1181484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35506/0 (socket says 172.21.15.148:35506) 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.800+0000 7fd116ffd640 1 -- 172.21.15.148:0/1095277029 learned_addr learned my addr 172.21.15.148:0/1095277029 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd1177fe640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11815bb50 0x7fd1180b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd1167fc640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1180cfd10 0x7fd1181e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd116ffd640 1 -- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1180cfd10 msgr2=0x7fd1181e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd116ffd640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1180cfd10 0x7fd1181e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd116ffd640 1 -- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11815bb50 msgr2=0x7fd1180b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd116ffd640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11815bb50 0x7fd1180b60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd116ffd640 1 -- 172.21.15.148:0/1095277029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1181e6d20 con 0x7fd1180c70d0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd1177fe640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11815bb50 0x7fd1180b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd1167fc640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1180cfd10 0x7fd1181e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd116ffd640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 0x7fd1181484b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd108007e60 tx=0x7fd10800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108014620 con 0x7fd1180c70d0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd108019070 con 0x7fd1180c70d0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1181e5c80 con 0x7fd1180c70d0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108014a20 con 0x7fd1180c70d0 2024-04-25T09:55:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.801+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1181e6240 con 0x7fd1180c70d0 2024-04-25T09:55:18.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.802+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11815b370 con 0x7fd1180c70d0 2024-04-25T09:55:18.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.802+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd108014bc0 con 0x7fd1180c70d0 2024-04-25T09:55:18.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.803+0000 7fd0fffff640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd10406b4d0 0x7fd10406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:18.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.803+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd10808ae70 con 0x7fd1180c70d0 2024-04-25T09:55:18.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.803+0000 7fd1167fc640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd10406b4d0 0x7fd10406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:18.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.803+0000 7fd1167fc640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd10406b4d0 0x7fd10406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1180ce5b0 tx=0x7fd10c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:18.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.807+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd11815b370 con 0x7fd1180c70d0 2024-04-25T09:55:18.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.977+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd1180d1410 con 0x7fd1180c70d0 2024-04-25T09:55:18.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.978+0000 7fd0fffff640 1 -- 172.21.15.148:0/1095277029 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1180d1410 con 0x7fd1180c70d0 2024-04-25T09:55:18.981 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd10406b4d0 msgr2=0x7fd10406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd10406b4d0 0x7fd10406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1180ce5b0 tx=0x7fd10c009040 comp rx=0 tx=0).stop 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 msgr2=0x7fd1181484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 0x7fd1181484b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd108007e60 tx=0x7fd10800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 shutdown_connections 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd10406b4d0 0x7fd10406d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd11815bb50 0x7fd1180b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1180cfd10 0x7fd1181e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 --2- 172.21.15.148:0/1095277029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1180c70d0 0x7fd1181484b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 >> 172.21.15.148:0/1095277029 conn(0x7fd11806f500 msgr2=0x7fd1180b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:18.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 shutdown_connections 2024-04-25T09:55:18.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:18.980+0000 7fd11d989640 1 -- 172.21.15.148:0/1095277029 wait complete. 2024-04-25T09:55:18.993 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:19.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.091+0000 7f2f61bd5640 1 Processor -- start 2024-04-25T09:55:19.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.091+0000 7f2f61bd5640 1 -- start start 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f61bd5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f5c15bb20 0x7f2f5c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f61bd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c0ccdd0 0x7f2f5c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f61bd5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5c0cb190 con 0x7f2f5c0cdcd0 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f61bd5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5c0cb470 con 0x7f2f5c15bb20 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f61bd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0cb750 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f5c15bb20 0x7f2f5c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f5c15bb20 0x7f2f5c0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35508/0 (socket says 172.21.15.148:35508) 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/3814264808 learned_addr learned my addr 172.21.15.148:0/3814264808 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f53fff640 1 --2- 172.21.15.148:0/3814264808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c0ccdd0 0x7f2f5c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/3814264808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c0ccdd0 msgr2=0x7f2f5c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:19.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 --2- 172.21.15.148:0/3814264808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c0ccdd0 0x7f2f5c0cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/3814264808 >> v1:172.21.15.188:6789/0 conn(0x7f2f5c0cdcd0 legacy=0x7f2f5c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.092+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/3814264808 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0cba30 con 0x7f2f5c15bb20 2024-04-25T09:55:19.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.093+0000 7f2f5b7fe640 1 --2- 172.21.15.148:0/3814264808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c15bb20 0x7f2f5c0cc230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009870 tx=0x7f2f4c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce78ab1eee75b83d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.093+0000 7f2f5affd640 1 -- 172.21.15.148:0/3814264808 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c037070 con 0x7f2f5c15bb20 2024-04-25T09:55:19.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.093+0000 7f2f5affd640 1 -- 172.21.15.148:0/3814264808 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c032440 con 0x7f2f5c15bb20 2024-04-25T09:55:19.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.093+0000 7f2f5affd640 1 -- 172.21.15.148:0/3814264808 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c032810 con 0x7f2f5c15bb20 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 -- 172.21.15.148:0/3814264808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c15bb20 msgr2=0x7f2f5c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/3814264808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c15bb20 0x7f2f5c0cc230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009870 tx=0x7f2f4c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 -- 172.21.15.148:0/3814264808 shutdown_connections 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/3814264808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c0ccdd0 0x7f2f5c0cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/3814264808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c15bb20 0x7f2f5c0cc230 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 -- 172.21.15.148:0/3814264808 >> 172.21.15.148:0/3814264808 conn(0x7f2f5c0b2390 msgr2=0x7f2f5c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 -- 172.21.15.148:0/3814264808 shutdown_connections 2024-04-25T09:55:19.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.094+0000 7f2f61bd5640 1 -- 172.21.15.148:0/3814264808 wait complete. 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.095+0000 7f2f61bd5640 1 Processor -- start 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.095+0000 7f2f61bd5640 1 -- start start 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 0x7f2f5c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f5c0cdcd0 0x7f2f5c0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c15bb20 0x7f2f5c0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c15bf00 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0cc770 con 0x7f2f5c0cdcd0 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c15c080 con 0x7f2f5c15bb20 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 0x7f2f5c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 0x7f2f5c06f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35516/0 (socket says 172.21.15.148:35516) 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/2450652453 learned_addr learned my addr 172.21.15.148:0/2450652453 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f53fff640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f5c0cdcd0 0x7f2f5c0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5bfff640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c15bb20 0x7f2f5c0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c15bb20 msgr2=0x7f2f5c0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c15bb20 0x7f2f5c0c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f5c0cdcd0 msgr2=0x7f2f5c0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f5c0cdcd0 0x7f2f5c0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 -- 172.21.15.148:0/2450652453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c1f1680 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f53fff640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f5c0cdcd0 0x7f2f5c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5bfff640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c15bb20 0x7f2f5c0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f5b7fe640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 0x7f2f5c06f7e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c000c00 tx=0x7f2f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c041020 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0306c0 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c1f1960 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.096+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c037040 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.097+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c1f1f20 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.097+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5c073a00 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.098+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f4c044020 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.098+0000 7f2f58ff9640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.099+0000 7f2f53fff640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.099+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f2f4c0aada0 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.099+0000 7f2f53fff640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c072650 tx=0x7f2f44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.103+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f4c04b030 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.274+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2f5c0cc260 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.274+0000 7f2f58ff9640 1 -- 172.21.15.148:0/2450652453 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2f4c0798a0 con 0x7f2f5c0ccdd0 2024-04-25T09:55:19.277 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2f3c06b4d0 msgr2=0x7f2f3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c072650 tx=0x7f2f44008040 comp rx=0 tx=0).stop 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 msgr2=0x7f2f5c06f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 0x7f2f5c06f7e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c000c00 tx=0x7f2f4c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 shutdown_connections 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f5c15bb20 0x7f2f5c0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f5c0cdcd0 0x7f2f5c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 --2- 172.21.15.148:0/2450652453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f5c0ccdd0 0x7f2f5c06f7e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 >> 172.21.15.148:0/2450652453 conn(0x7f2f5c0b2390 msgr2=0x7f2f5c06e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:19.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 shutdown_connections 2024-04-25T09:55:19.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.276+0000 7f2f61bd5640 1 -- 172.21.15.148:0/2450652453 wait complete. 2024-04-25T09:55:19.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:55:19.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.385+0000 7fbb1e26b640 1 Processor -- start 2024-04-25T09:55:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.385+0000 7fbb1e26b640 1 -- start start 2024-04-25T09:55:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1e26b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 0x7fbb180ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1e26b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb18161710 0x7fbb180cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1e26b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1815e000 con 0x7fbb180b9330 2024-04-25T09:55:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1e26b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180d1de0 con 0x7fbb18161710 2024-04-25T09:55:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1e26b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18161c60 con 0x7fbb1815dc20 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1c808640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbb180b9330 0x7fbb1806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55648/0 (socket says 172.21.15.148:55648) 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1c808640 1 -- 172.21.15.148:0/2575838097 learned_addr learned my addr 172.21.15.148:0/2575838097 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025465666 0 0) 0x7fbb1815e000 con 0x7fbb180b9330 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb00003520 con 0x7fbb180b9330 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb17fff640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 0x7fbb180ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb177fe640 1 --2- 172.21.15.148:0/2575838097 >> v2:172.21.15.188:3300/0 conn(0x7fbb18161710 0x7fbb180cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1103111635 0 0) 0x7fbb00003520 con 0x7fbb180b9330 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 msgr2=0x7fbb180ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 0x7fbb180ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 >> v2:172.21.15.188:3300/0 conn(0x7fbb18161710 msgr2=0x7fbb180cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 --2- 172.21.15.148:0/2575838097 >> v2:172.21.15.188:3300/0 conn(0x7fbb18161710 0x7fbb180cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18161f40 con 0x7fbb180b9330 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbb100019f0 con 0x7fbb180b9330 2024-04-25T09:55:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 >> v1:172.21.15.148:6789/0 conn(0x7fbb180b9330 legacy=0x7fbb1806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 0x7fbb000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb00003f30 0x7fbb00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180d1de0 con 0x7fbb00004d40 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18161c60 con 0x7fbb00003f30 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000b9c0 con 0x7fbb1815dc20 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.386+0000 7fbb1c808640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb177fe640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb00003f30 0x7fbb00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb17fff640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 0x7fbb000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb1c808640 1 -- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 msgr2=0x7fbb000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb1c808640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb1815dc20 0x7fbb000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb1c808640 1 -- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb00003f30 msgr2=0x7fbb00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb1c808640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb00003f30 0x7fbb00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb1c808640 1 -- 172.21.15.148:0/2575838097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00003520 con 0x7fbb00004d40 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.387+0000 7fbb1c808640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000a870 tx=0x7fbb00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740becb3a5950c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000a4c0 con 0x7fbb00004d40 2024-04-25T09:55:19.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1002f070 con 0x7fbb00004d40 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb16ffd640 1 -- 172.21.15.148:0/2575838097 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1002bcb0 con 0x7fbb00004d40 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1e26b640 1 -- 172.21.15.148:0/2575838097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 msgr2=0x7fbb000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/2575838097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb000051c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000a870 tx=0x7fbb00005830 comp rx=0 tx=0).stop 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1c808640 1 -- 172.21.15.148:0/2575838097 reap_dead start 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1e26b640 1 -- 172.21.15.148:0/2575838097 shutdown_connections 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1e26b640 1 -- 172.21.15.148:0/2575838097 >> 172.21.15.148:0/2575838097 conn(0x7fbb180d0420 msgr2=0x7fbb180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1e26b640 1 -- 172.21.15.148:0/2575838097 shutdown_connections 2024-04-25T09:55:19.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.388+0000 7fbb1e26b640 1 -- 172.21.15.148:0/2575838097 wait complete. 2024-04-25T09:55:19.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.389+0000 7fbb1e26b640 1 Processor -- start 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 -- start start 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb181544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb180caa70 0x7fbb180cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb180cb840 0x7fbb180cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180b5e50 con 0x7fbb00004d40 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18162500 con 0x7fbb180cb840 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1e26b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180b5fd0 con 0x7fbb180caa70 2024-04-25T09:55:19.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb181544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb181544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35540/0 (socket says 172.21.15.148:35540) 2024-04-25T09:55:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb17fff640 1 -- 172.21.15.148:0/3639374055 learned_addr learned my addr 172.21.15.148:0/3639374055 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb177fe640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb180caa70 0x7fbb180cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.390+0000 7fbb1c808640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb180cb840 0x7fbb180cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb17fff640 1 -- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb180caa70 msgr2=0x7fbb180cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb17fff640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb180caa70 0x7fbb180cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb17fff640 1 -- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb180cb840 msgr2=0x7fbb180cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb17fff640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb180cb840 0x7fbb180cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb17fff640 1 -- 172.21.15.148:0/3639374055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb181f3620 con 0x7fbb00004d40 2024-04-25T09:55:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb17fff640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb181544b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb080119a0 tx=0x7fbb08011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800faf0 con 0x7fbb00004d40 2024-04-25T09:55:19.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0800fc90 con 0x7fbb00004d40 2024-04-25T09:55:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c80a0 con 0x7fbb00004d40 2024-04-25T09:55:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08014440 con 0x7fbb00004d40 2024-04-25T09:55:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.391+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb180c8660 con 0x7fbb00004d40 2024-04-25T09:55:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.392+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb18073a00 con 0x7fbb00004d40 2024-04-25T09:55:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.392+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb08012040 con 0x7fbb00004d40 2024-04-25T09:55:19.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.393+0000 7fbb14ff9640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb0406b4d0 0x7fbb0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.393+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fbb0808ad10 con 0x7fbb00004d40 2024-04-25T09:55:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.393+0000 7fbb177fe640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb0406b4d0 0x7fbb0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.393+0000 7fbb177fe640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb0406b4d0 0x7fbb0406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb1815bf60 tx=0x7fbb0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.397+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb08019030 con 0x7fbb00004d40 2024-04-25T09:55:19.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.578+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb180d16b0 con 0x7fbb00004d40 2024-04-25T09:55:19.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.578+0000 7fbb14ff9640 1 -- 172.21.15.148:0/3639374055 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb08059130 con 0x7fbb00004d40 2024-04-25T09:55:19.581 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb0406b4d0 msgr2=0x7fbb0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb0406b4d0 0x7fbb0406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb1815bf60 tx=0x7fbb0c008040 comp rx=0 tx=0).stop 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 msgr2=0x7fbb181544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb181544b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb080119a0 tx=0x7fbb08011e60 comp rx=0 tx=0).stop 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 shutdown_connections 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb0406b4d0 0x7fbb0406d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb180cb840 0x7fbb180cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb180caa70 0x7fbb180cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 --2- 172.21.15.148:0/3639374055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb00004d40 0x7fbb181544b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 >> 172.21.15.148:0/3639374055 conn(0x7fbb180d0420 msgr2=0x7fbb180b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:19.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 shutdown_connections 2024-04-25T09:55:19.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.580+0000 7fbb1e26b640 1 -- 172.21.15.148:0/3639374055 wait complete. 2024-04-25T09:55:19.593 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T09:55:19.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:19.594 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T09:55:19.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T09:55:19.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:19.625 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:55:19.683 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:55:19.683 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:19.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.779+0000 7fe8016cb640 1 Processor -- start 2024-04-25T09:55:19.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.779+0000 7fe8016cb640 1 -- start start 2024-04-25T09:55:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.780+0000 7fe8016cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc15ec20 0x7fe7fc0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.780+0000 7fe8016cb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7fc06e430 0x7fe7fc06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.780+0000 7fe8016cb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc0b3e10 con 0x7fe7fc06abe0 2024-04-25T09:55:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.780+0000 7fe8016cb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b3590 con 0x7fe7fc06e430 2024-04-25T09:55:19.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.780+0000 7fe8016cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0b3850 con 0x7fe7fc15ec20 2024-04-25T09:55:19.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fa7fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc15ec20 0x7fe7fc0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fb7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7fc06e430 0x7fe7fc06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7faffd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe7fc06abe0 0x7fe7fc06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:41272/0 (socket says 172.21.15.148:41272) 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7faffd640 1 -- 172.21.15.148:0/1543792668 learned_addr learned my addr 172.21.15.148:0/1543792668 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7f9ffb640 1 -- 172.21.15.148:0/1543792668 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774796134 0 0) 0x7fe7ec001490 con 0x7fe7fc06abe0 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7f9ffb640 1 -- 172.21.15.148:0/1543792668 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e4003520 con 0x7fe7fc06abe0 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fa7fc640 1 -- 172.21.15.148:0/1543792668 >> v2:172.21.15.188:3300/0 conn(0x7fe7fc06e430 msgr2=0x7fe7fc06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fa7fc640 1 --2- 172.21.15.148:0/1543792668 >> v2:172.21.15.188:3300/0 conn(0x7fe7fc06e430 0x7fe7fc06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fa7fc640 1 -- 172.21.15.148:0/1543792668 >> v1:172.21.15.188:6789/0 conn(0x7fe7fc06abe0 legacy=0x7fe7fc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fa7fc640 1 -- 172.21.15.148:0/1543792668 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0b3b30 con 0x7fe7fc15ec20 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.781+0000 7fe7fb7fe640 1 --2- 172.21.15.148:0/1543792668 >> v2:172.21.15.188:3300/0 conn(0x7fe7fc06e430 0x7fe7fc06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.782+0000 7fe7fa7fc640 1 --2- 172.21.15.148:0/1543792668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc15ec20 0x7fe7fc0b5270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00097a0 tx=0x7fe7f002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c70971315db676f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.782+0000 7fe7f9ffb640 1 -- 172.21.15.148:0/1543792668 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0037070 con 0x7fe7fc15ec20 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.782+0000 7fe7f9ffb640 1 -- 172.21.15.148:0/1543792668 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0032440 con 0x7fe7fc15ec20 2024-04-25T09:55:19.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.782+0000 7fe7f9ffb640 1 -- 172.21.15.148:0/1543792668 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00327f0 con 0x7fe7fc15ec20 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 -- 172.21.15.148:0/1543792668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc15ec20 msgr2=0x7fe7fc0b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1543792668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc15ec20 0x7fe7fc0b5270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00097a0 tx=0x7fe7f002a180 comp rx=0 tx=0).stop 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 -- 172.21.15.148:0/1543792668 shutdown_connections 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1543792668 >> v2:172.21.15.188:3300/0 conn(0x7fe7fc06e430 0x7fe7fc06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1543792668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc15ec20 0x7fe7fc0b5270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 -- 172.21.15.148:0/1543792668 >> 172.21.15.148:0/1543792668 conn(0x7fe7fc0b5af0 msgr2=0x7fe7fc0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 -- 172.21.15.148:0/1543792668 shutdown_connections 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.783+0000 7fe8016cb640 1 -- 172.21.15.148:0/1543792668 wait complete. 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.784+0000 7fe8016cb640 1 Processor -- start 2024-04-25T09:55:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.784+0000 7fe8016cb640 1 -- start start 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe8016cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7fc06abe0 0x7fe7fc158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe8016cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc06e430 0x7fe7fc1f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe8016cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 0x7fe7fc159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe8016cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc070320 con 0x7fe7fc15ec20 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe8016cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc15e700 con 0x7fe7fc06abe0 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe8016cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0704a0 con 0x7fe7fc06e430 2024-04-25T09:55:19.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 0x7fe7fc159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 0x7fe7fc159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35550/0 (socket says 172.21.15.148:35550) 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 -- 172.21.15.148:0/1038524755 learned_addr learned my addr 172.21.15.148:0/1038524755 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7faffd640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7fc06abe0 0x7fe7fc158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fa7fc640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc06e430 0x7fe7fc1f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 -- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc06e430 msgr2=0x7fe7fc1f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc06e430 0x7fe7fc1f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 -- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7fc06abe0 msgr2=0x7fe7fc158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7fc06abe0 0x7fe7fc158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 -- 172.21.15.148:0/1038524755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc1f7e70 con 0x7fe7fc15ec20 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7faffd640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7fc06abe0 0x7fe7fc158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fa7fc640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc06e430 0x7fe7fc1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7fb7fe640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 0x7fe7fc159ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e960 tx=0x7fe7f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.785+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017070 con 0x7fe7fc15ec20 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.786+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4004020 con 0x7fe7fc15ec20 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.786+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc1f8150 con 0x7fe7fc15ec20 2024-04-25T09:55:19.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.786+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40043f0 con 0x7fe7fc15ec20 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.786+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc1449f0 con 0x7fe7fc15ec20 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.787+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc06cc80 con 0x7fe7fc15ec20 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.787+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7f400ca00 con 0x7fe7fc15ec20 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.787+0000 7fe7e37fe640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7e806b4d0 0x7fe7e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.788+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fe7f408a860 con 0x7fe7fc15ec20 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.788+0000 7fe7faffd640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7e806b4d0 0x7fe7e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:19.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.788+0000 7fe7faffd640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7e806b4d0 0x7fe7e806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec005c00 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:19.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.792+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7f40592d0 con 0x7fe7fc15ec20 2024-04-25T09:55:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:19.999+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7fc144810 con 0x7fe7fc15ec20 2024-04-25T09:55:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.000+0000 7fe7e37fe640 1 -- 172.21.15.148:0/1038524755 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fe7fc144810 con 0x7fe7fc15ec20 2024-04-25T09:55:20.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:20.003 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:55:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7e806b4d0 msgr2=0x7fe7e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7e806b4d0 0x7fe7e806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec005c00 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 msgr2=0x7fe7fc159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 0x7fe7fc159ee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400e960 tx=0x7fe7f400ee20 comp rx=0 tx=0).stop 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 shutdown_connections 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7e806b4d0 0x7fe7e806d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7fc15ec20 0x7fe7fc159ee0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7fc06e430 0x7fe7fc1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 --2- 172.21.15.148:0/1038524755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7fc06abe0 0x7fe7fc158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 >> 172.21.15.148:0/1038524755 conn(0x7fe7fc0b5af0 msgr2=0x7fe7fc0ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 shutdown_connections 2024-04-25T09:55:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.002+0000 7fe8016cb640 1 -- 172.21.15.148:0/1038524755 wait complete. 2024-04-25T09:55:20.016 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:20.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.111+0000 7f757c7e0640 1 Processor -- start 2024-04-25T09:55:20.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.111+0000 7f757c7e0640 1 -- start start 2024-04-25T09:55:20.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757c7e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f757415dc00 0x7f75740c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757c7e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740ceb20 0x7f75740d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757c7e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757415dfe0 con 0x7f75740d1c00 2024-04-25T09:55:20.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757c7e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740d2060 con 0x7f757415dc00 2024-04-25T09:55:20.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757c7e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740d00c0 con 0x7f75740ceb20 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f7579d7b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740ceb20 0x7f75740d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757a57c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f757415dc00 0x7f75740c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757ad7d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f75740d1c00 0x7f75740ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:41278/0 (socket says 172.21.15.148:41278) 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757ad7d640 1 -- 172.21.15.148:0/1439324188 learned_addr learned my addr 172.21.15.148:0/1439324188 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757957a640 1 -- 172.21.15.148:0/1439324188 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613554966 0 0) 0x7f756c001490 con 0x7f75740d1c00 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757957a640 1 -- 172.21.15.148:0/1439324188 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7560003520 con 0x7f75740d1c00 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f7579d7b640 1 -- 172.21.15.148:0/1439324188 >> v2:172.21.15.188:3300/0 conn(0x7f757415dc00 msgr2=0x7f75740c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f7579d7b640 1 --2- 172.21.15.148:0/1439324188 >> v2:172.21.15.188:3300/0 conn(0x7f757415dc00 0x7f75740c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f7579d7b640 1 -- 172.21.15.148:0/1439324188 >> v1:172.21.15.188:6789/0 conn(0x7f75740d1c00 legacy=0x7f75740ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f7579d7b640 1 -- 172.21.15.148:0/1439324188 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740d03a0 con 0x7f75740ceb20 2024-04-25T09:55:20.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.112+0000 7f757a57c640 1 --2- 172.21.15.148:0/1439324188 >> v2:172.21.15.188:3300/0 conn(0x7f757415dc00 0x7f75740c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:20.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.113+0000 7f7579d7b640 1 --2- 172.21.15.148:0/1439324188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740ceb20 0x7f75740d16c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75640097a0 tx=0x7f756402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90898e242b81e235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.113+0000 7f757957a640 1 -- 172.21.15.148:0/1439324188 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564037070 con 0x7f75740ceb20 2024-04-25T09:55:20.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.113+0000 7f757957a640 1 -- 172.21.15.148:0/1439324188 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564032440 con 0x7f75740ceb20 2024-04-25T09:55:20.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.113+0000 7f757957a640 1 -- 172.21.15.148:0/1439324188 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640327f0 con 0x7f75740ceb20 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 -- 172.21.15.148:0/1439324188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740ceb20 msgr2=0x7f75740d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 --2- 172.21.15.148:0/1439324188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740ceb20 0x7f75740d16c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75640097a0 tx=0x7f756402a180 comp rx=0 tx=0).stop 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 -- 172.21.15.148:0/1439324188 shutdown_connections 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 --2- 172.21.15.148:0/1439324188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740ceb20 0x7f75740d16c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 --2- 172.21.15.148:0/1439324188 >> v2:172.21.15.188:3300/0 conn(0x7f757415dc00 0x7f75740c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 -- 172.21.15.148:0/1439324188 >> 172.21.15.148:0/1439324188 conn(0x7f757406f5d0 msgr2=0x7f757406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 -- 172.21.15.148:0/1439324188 shutdown_connections 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.114+0000 7f757c7e0640 1 -- 172.21.15.148:0/1439324188 wait complete. 2024-04-25T09:55:20.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.115+0000 7f757c7e0640 1 Processor -- start 2024-04-25T09:55:20.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.115+0000 7f757c7e0640 1 -- start start 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757c7e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 0x7f75740c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757c7e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740d1c00 0x7f7574147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757c7e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757415dc00 0x7f7574159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757c7e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cc130 con 0x7f75740ceb20 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757c7e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cc720 con 0x7f75740d1c00 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757c7e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cfd90 con 0x7f757415dc00 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 0x7f75740c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 0x7f75740c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35554/0 (socket says 172.21.15.148:35554) 2024-04-25T09:55:20.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 -- 172.21.15.148:0/4128606817 learned_addr learned my addr 172.21.15.148:0/4128606817 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757ad7d640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757415dc00 0x7f7574159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f7579d7b640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740d1c00 0x7f7574147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 -- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757415dc00 msgr2=0x7f7574159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757415dc00 0x7f7574159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 -- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740d1c00 msgr2=0x7f7574147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740d1c00 0x7f7574147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757a57c640 1 -- 172.21.15.148:0/4128606817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75741598b0 con 0x7f75740ceb20 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f7579d7b640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740d1c00 0x7f7574147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.116+0000 7f757ad7d640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757415dc00 0x7f7574159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.117+0000 7f757a57c640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 0x7f75740c9900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f757000e960 tx=0x7f757000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.117+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570017070 con 0x7f75740ceb20 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.117+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000ca00 con 0x7f75740ceb20 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.117+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000cdd0 con 0x7f75740ceb20 2024-04-25T09:55:20.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.117+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75740c8b40 con 0x7f75740ceb20 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.117+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75740c9100 con 0x7f75740ceb20 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.118+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7574073990 con 0x7f75740ceb20 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.118+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f757001a070 con 0x7f75740ceb20 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.118+0000 7f756affd640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755806b4d0 0x7f755806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.119+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f757008a460 con 0x7f75740ceb20 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.119+0000 7f7579d7b640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755806b4d0 0x7f755806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.119+0000 7f7579d7b640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755806b4d0 0x7f755806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7564034000 tx=0x7f756402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.123+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75741598b0 con 0x7f75740ceb20 2024-04-25T09:55:20.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.294+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75740cfab0 con 0x7f75740ceb20 2024-04-25T09:55:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.295+0000 7f756affd640 1 -- 172.21.15.148:0/4128606817 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7f7570058f60 con 0x7f75740ceb20 2024-04-25T09:55:20.298 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:20.298 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755806b4d0 msgr2=0x7f755806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755806b4d0 0x7f755806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7564034000 tx=0x7f756402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 msgr2=0x7f75740c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 0x7f75740c9900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f757000e960 tx=0x7f757000ee20 comp rx=0 tx=0).stop 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 shutdown_connections 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755806b4d0 0x7f755806d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757415dc00 0x7f7574159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740d1c00 0x7f7574147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 --2- 172.21.15.148:0/4128606817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740ceb20 0x7f75740c9900 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 >> 172.21.15.148:0/4128606817 conn(0x7f757406f5d0 msgr2=0x7f75740b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 shutdown_connections 2024-04-25T09:55:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.297+0000 7f757c7e0640 1 -- 172.21.15.148:0/4128606817 wait complete. 2024-04-25T09:55:20.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:54:20.970314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:54:22.267323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T09:54:25.998970+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T09:55:20.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:20.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.408+0000 7f582235c640 1 Processor -- start 2024-04-25T09:55:20.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f582235c640 1 -- start start 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f582235c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f581c15de80 0x7f581c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f582235c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0cefb0 0x7f581c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f582235c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0afc80 con 0x7f581c0cf950 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f582235c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0ce4d0 con 0x7f581c15de80 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f582235c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0ce7b0 con 0x7f581c0cefb0 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.409+0000 7f581bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f581c15de80 0x7f581c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f581c15de80 0x7f581c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35570/0 (socket says 172.21.15.148:35570) 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 -- 172.21.15.148:0/1115730826 learned_addr learned my addr 172.21.15.148:0/1115730826 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581b7fe640 1 --2- 172.21.15.148:0/1115730826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0cefb0 0x7f581c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 -- 172.21.15.148:0/1115730826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0cefb0 msgr2=0x7f581c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 --2- 172.21.15.148:0/1115730826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0cefb0 0x7f581c0cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 -- 172.21.15.148:0/1115730826 >> v1:172.21.15.188:6789/0 conn(0x7f581c0cf950 legacy=0x7f581c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 -- 172.21.15.148:0/1115730826 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0cea90 con 0x7f581c15de80 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.410+0000 7f581bfff640 1 --2- 172.21.15.148:0/1115730826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c15de80 0x7f581c0cadb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f580c009870 tx=0x7f580c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5e1266809d1b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f581affd640 1 -- 172.21.15.148:0/1115730826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c037070 con 0x7f581c15de80 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f581affd640 1 -- 172.21.15.148:0/1115730826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c032440 con 0x7f581c15de80 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f581affd640 1 -- 172.21.15.148:0/1115730826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c032810 con 0x7f581c15de80 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 -- 172.21.15.148:0/1115730826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c15de80 msgr2=0x7f581c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 --2- 172.21.15.148:0/1115730826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c15de80 0x7f581c0cadb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f580c009870 tx=0x7f580c02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 -- 172.21.15.148:0/1115730826 shutdown_connections 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 --2- 172.21.15.148:0/1115730826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c0cefb0 0x7f581c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 --2- 172.21.15.148:0/1115730826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c15de80 0x7f581c0cadb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 -- 172.21.15.148:0/1115730826 >> 172.21.15.148:0/1115730826 conn(0x7f581c0b2390 msgr2=0x7f581c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 -- 172.21.15.148:0/1115730826 shutdown_connections 2024-04-25T09:55:20.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.411+0000 7f582235c640 1 -- 172.21.15.148:0/1115730826 wait complete. 2024-04-25T09:55:20.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.412+0000 7f582235c640 1 Processor -- start 2024-04-25T09:55:20.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 -- start start 2024-04-25T09:55:20.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c0cefb0 0x7f581c14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 0x7f581c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15de80 0x7f581c0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b3030 con 0x7f581c0cf950 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b2eb0 con 0x7f581c0cefb0 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f582235c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c5600 con 0x7f581c15de80 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f581b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 0x7f581c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.413+0000 7f581b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 0x7f581c0cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35576/0 (socket says 172.21.15.148:35576) 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 -- 172.21.15.148:0/3713060707 learned_addr learned my addr 172.21.15.148:0/3713060707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f58208f9640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15de80 0x7f581c0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581bfff640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c0cefb0 0x7f581c14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 -- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15de80 msgr2=0x7f581c0ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15de80 0x7f581c0ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 -- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c0cefb0 msgr2=0x7f581c14d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c0cefb0 0x7f581c14d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 -- 172.21.15.148:0/3713060707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0b38a0 con 0x7f581c0cf950 2024-04-25T09:55:20.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581bfff640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c0cefb0 0x7f581c14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f58208f9640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15de80 0x7f581c0ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f581b7fe640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 0x7f581c0cca10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f581000d830 tx=0x7f581000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000b6c0 con 0x7f581c0cf950 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810019070 con 0x7f581c0cf950 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581000bac0 con 0x7f581c0cf950 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c0b3b80 con 0x7f581c0cf950 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.414+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c0b4140 con 0x7f581c0cf950 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.415+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c073a00 con 0x7f581c0cf950 2024-04-25T09:55:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.416+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f581001e460 con 0x7f581c0cf950 2024-04-25T09:55:20.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.416+0000 7f5818ff9640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f580806b4d0 0x7f580806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.416+0000 7f581bfff640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f580806b4d0 0x7f580806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.416+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f581008ad20 con 0x7f581c0cf950 2024-04-25T09:55:20.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.417+0000 7f581bfff640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f580806b4d0 0x7f580806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f580c034000 tx=0x7f580c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.420+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5810059790 con 0x7f581c0cf950 2024-04-25T09:55:20.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.589+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f581c0b0b90 con 0x7f581c0cf950 2024-04-25T09:55:20.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.590+0000 7f5818ff9640 1 -- 172.21.15.148:0/3713060707 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5810059090 con 0x7f581c0cf950 2024-04-25T09:55:20.593 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f580806b4d0 msgr2=0x7f580806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f580806b4d0 0x7f580806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f580c034000 tx=0x7f580c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 msgr2=0x7f581c0cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 0x7f581c0cca10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f581000d830 tx=0x7f581000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 shutdown_connections 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f580806b4d0 0x7f580806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f581c15de80 0x7f581c0ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f581c0cf950 0x7f581c0cca10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 --2- 172.21.15.148:0/3713060707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f581c0cefb0 0x7f581c14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 >> 172.21.15.148:0/3713060707 conn(0x7f581c0b2390 msgr2=0x7f581c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 shutdown_connections 2024-04-25T09:55:20.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.592+0000 7f582235c640 1 -- 172.21.15.148:0/3713060707 wait complete. 2024-04-25T09:55:20.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:55:20.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.703+0000 7f9c7f9c4640 1 Processor -- start 2024-04-25T09:55:20.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.703+0000 7f9c7f9c4640 1 -- start start 2024-04-25T09:55:20.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7f9c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c780cadb0 0x7f9c780cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7f9c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c780ccfe0 0x7f9c780cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7f9c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c780afc80 con 0x7f9c7815de80 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7f9c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c780cb240 con 0x7f9c780cadb0 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7f9c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c780ce4d0 con 0x7f9c780ccfe0 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7cf5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c780cadb0 0x7f9c780cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7cf5f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9c780cadb0 0x7f9c780cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35588/0 (socket says 172.21.15.148:35588) 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.704+0000 7f9c7cf5f640 1 -- 172.21.15.148:0/947825277 learned_addr learned my addr 172.21.15.148:0/947825277 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.705+0000 7f9c7df61640 1 --2- 172.21.15.148:0/947825277 >> v2:172.21.15.188:3300/0 conn(0x7f9c780ccfe0 0x7f9c780cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.705+0000 7f9c7cf5f640 1 -- 172.21.15.148:0/947825277 >> v2:172.21.15.188:3300/0 conn(0x7f9c780ccfe0 msgr2=0x7f9c780cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.705+0000 7f9c7cf5f640 1 --2- 172.21.15.148:0/947825277 >> v2:172.21.15.188:3300/0 conn(0x7f9c780ccfe0 0x7f9c780cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.705+0000 7f9c7cf5f640 1 -- 172.21.15.148:0/947825277 >> v1:172.21.15.188:6789/0 conn(0x7f9c7815de80 legacy=0x7f9c780cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:20.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.705+0000 7f9c7cf5f640 1 -- 172.21.15.148:0/947825277 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780ce7b0 con 0x7f9c780cadb0 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.705+0000 7f9c7cf5f640 1 --2- 172.21.15.148:0/947825277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c780cadb0 0x7f9c780cf6c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9c7806e2a0 tx=0x7f9c6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3426fd2d076bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c6ffff640 1 -- 172.21.15.148:0/947825277 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68037070 con 0x7f9c780cadb0 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c6ffff640 1 -- 172.21.15.148:0/947825277 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c68032440 con 0x7f9c780cadb0 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c6ffff640 1 -- 172.21.15.148:0/947825277 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c68032810 con 0x7f9c780cadb0 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/947825277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c780cadb0 msgr2=0x7f9c780cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/947825277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c780cadb0 0x7f9c780cf6c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9c7806e2a0 tx=0x7f9c6802a230 comp rx=0 tx=0).stop 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/947825277 shutdown_connections 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/947825277 >> v2:172.21.15.188:3300/0 conn(0x7f9c780ccfe0 0x7f9c780cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/947825277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c780cadb0 0x7f9c780cf6c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/947825277 >> 172.21.15.148:0/947825277 conn(0x7f9c780b2390 msgr2=0x7f9c780b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/947825277 shutdown_connections 2024-04-25T09:55:20.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.706+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/947825277 wait complete. 2024-04-25T09:55:20.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.707+0000 7f9c7f9c4640 1 Processor -- start 2024-04-25T09:55:20.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.707+0000 7f9c7f9c4640 1 -- start start 2024-04-25T09:55:20.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7f9c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c780cadb0 0x7f9c780aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7f9c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c780ccfe0 0x7f9c780c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7f9c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 0x7f9c78159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7f9c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7815e260 con 0x7f9c7815de80 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7f9c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780cfc00 con 0x7f9c780ccfe0 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7f9c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b2eb0 con 0x7f9c780cadb0 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 0x7f9c78159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 0x7f9c78159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35602/0 (socket says 172.21.15.148:35602) 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 -- 172.21.15.148:0/3318647474 learned_addr learned my addr 172.21.15.148:0/3318647474 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7cf5f640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c780ccfe0 0x7f9c780c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7d760640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c780cadb0 0x7f9c780aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 -- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c780cadb0 msgr2=0x7f9c780aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c780cadb0 0x7f9c780aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 -- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c780ccfe0 msgr2=0x7f9c780c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c780ccfe0 0x7f9c780c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7df61640 1 -- 172.21.15.148:0/3318647474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780c8730 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.708+0000 7f9c7cf5f640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c780ccfe0 0x7f9c780c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.709+0000 7f9c7df61640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 0x7f9c78159020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000d800 tx=0x7f9c7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.709+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000b6c0 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.709+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70019070 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.709+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c780c8a10 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.709+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000bac0 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.709+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c780c8fd0 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.710+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c780c5600 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.710+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c7000bc60 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.711+0000 7f9c6dffb640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.711+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f9c7008aa70 con 0x7f9c7815de80 2024-04-25T09:55:20.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.711+0000 7f9c7d760640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:20.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.711+0000 7f9c7d760640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009f20 tx=0x7f9c740094d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.715+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c780c5600 con 0x7f9c7815de80 2024-04-25T09:55:20.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.884+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c78073a00 con 0x7f9c7815de80 2024-04-25T09:55:20.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.885+0000 7f9c6dffb640 1 -- 172.21.15.148:0/3318647474 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c700594e0 con 0x7f9c7815de80 2024-04-25T09:55:20.888 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c5c06b4d0 msgr2=0x7f9c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009f20 tx=0x7f9c740094d0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 msgr2=0x7f9c78159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 0x7f9c78159020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000d800 tx=0x7f9c7000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 shutdown_connections 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c5c06b4d0 0x7f9c5c06d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c7815de80 0x7f9c78159020 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c780ccfe0 0x7f9c780c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 --2- 172.21.15.148:0/3318647474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c780cadb0 0x7f9c780aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:20.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 >> 172.21.15.148:0/3318647474 conn(0x7f9c780b2390 msgr2=0x7f9c780cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:20.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 shutdown_connections 2024-04-25T09:55:20.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.887+0000 7f9c7f9c4640 1 -- 172.21.15.148:0/3318647474 wait complete. 2024-04-25T09:55:20.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:55:21.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 Processor -- start 2024-04-25T09:55:21.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 -- start start 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd95c15b7d0 0x7fd95c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd95c0b5620 0x7fd95c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c06cf50 con 0x7fd95c06c2d0 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95c0ce530 con 0x7fd95c15b7d0 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd962dd5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95c0ce810 con 0x7fd95c0b5620 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.998+0000 7fd960b71640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd95c15b7d0 0x7fd95c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd961372640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd95c06c2d0 0x7fd95c06c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55654/0 (socket says 172.21.15.148:55654) 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd961372640 1 -- 172.21.15.148:0/1726969623 learned_addr learned my addr 172.21.15.148:0/1726969623 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd95bfff640 1 --2- 172.21.15.148:0/1726969623 >> v2:172.21.15.188:3300/0 conn(0x7fd95c0b5620 0x7fd95c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd960b71640 1 -- 172.21.15.148:0/1726969623 >> v2:172.21.15.188:3300/0 conn(0x7fd95c0b5620 msgr2=0x7fd95c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd960b71640 1 --2- 172.21.15.148:0/1726969623 >> v2:172.21.15.188:3300/0 conn(0x7fd95c0b5620 0x7fd95c0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd960b71640 1 -- 172.21.15.148:0/1726969623 >> v1:172.21.15.148:6789/0 conn(0x7fd95c06c2d0 legacy=0x7fd95c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd95b7fe640 1 -- 172.21.15.148:0/1726969623 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552635339 0 0) 0x7fd95c06cf50 con 0x7fd95c06c2d0 2024-04-25T09:55:21.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd960b71640 1 -- 172.21.15.148:0/1726969623 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0ceaf0 con 0x7fd95c15b7d0 2024-04-25T09:55:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:20.999+0000 7fd960b71640 1 --2- 172.21.15.148:0/1726969623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c15b7d0 0x7fd95c0cf010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0ce180 tx=0x7fd95002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279d83198d9e9dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd95b7fe640 1 -- 172.21.15.148:0/1726969623 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950037070 con 0x7fd95c15b7d0 2024-04-25T09:55:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd95b7fe640 1 -- 172.21.15.148:0/1726969623 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950032440 con 0x7fd95c15b7d0 2024-04-25T09:55:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd95b7fe640 1 -- 172.21.15.148:0/1726969623 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950032810 con 0x7fd95c15b7d0 2024-04-25T09:55:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 -- 172.21.15.148:0/1726969623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c15b7d0 msgr2=0x7fd95c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 --2- 172.21.15.148:0/1726969623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c15b7d0 0x7fd95c0cf010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0ce180 tx=0x7fd95002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 -- 172.21.15.148:0/1726969623 shutdown_connections 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 --2- 172.21.15.148:0/1726969623 >> v2:172.21.15.188:3300/0 conn(0x7fd95c0b5620 0x7fd95c0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 --2- 172.21.15.148:0/1726969623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c15b7d0 0x7fd95c0cf010 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 -- 172.21.15.148:0/1726969623 >> 172.21.15.148:0/1726969623 conn(0x7fd95c0d0800 msgr2=0x7fd95c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 -- 172.21.15.148:0/1726969623 shutdown_connections 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.000+0000 7fd962dd5640 1 -- 172.21.15.148:0/1726969623 wait complete. 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.001+0000 7fd962dd5640 1 Processor -- start 2024-04-25T09:55:21.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.001+0000 7fd962dd5640 1 -- start start 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd962dd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 0x7fd95c0aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd962dd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd95c0b5620 0x7fd95c14fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd962dd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd95c15b7d0 0x7fd95c0d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd962dd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0cf6d0 con 0x7fd95c06c2d0 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd962dd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0cf550 con 0x7fd95c0b5620 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd962dd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c15bbb0 con 0x7fd95c15b7d0 2024-04-25T09:55:21.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd960b71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 0x7fd95c0aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd960b71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 0x7fd95c0aa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35622/0 (socket says 172.21.15.148:35622) 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.002+0000 7fd960b71640 1 -- 172.21.15.148:0/4073216891 learned_addr learned my addr 172.21.15.148:0/4073216891 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd95bfff640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd95c0b5620 0x7fd95c14fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd961372640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd95c15b7d0 0x7fd95c0d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd960b71640 1 -- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd95c15b7d0 msgr2=0x7fd95c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd960b71640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd95c15b7d0 0x7fd95c0d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd960b71640 1 -- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd95c0b5620 msgr2=0x7fd95c14fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd960b71640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd95c0b5620 0x7fd95c14fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd960b71640 1 -- 172.21.15.148:0/4073216891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0b8240 con 0x7fd95c06c2d0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd960b71640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 0x7fd95c0aa360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd950000c00 tx=0x7fd9500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950037040 con 0x7fd95c06c2d0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950042070 con 0x7fd95c06c2d0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c0b7f60 con 0x7fd95c06c2d0 2024-04-25T09:55:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950030bd0 con 0x7fd95c06c2d0 2024-04-25T09:55:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.003+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c0c7d00 con 0x7fd95c06c2d0 2024-04-25T09:55:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.004+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c073a00 con 0x7fd95c06c2d0 2024-04-25T09:55:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.005+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd950047460 con 0x7fd95c06c2d0 2024-04-25T09:55:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.005+0000 7fd9597fa640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94006b4d0 0x7fd94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.005+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd9500ab240 con 0x7fd95c06c2d0 2024-04-25T09:55:21.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.005+0000 7fd95bfff640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94006b4d0 0x7fd94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.005+0000 7fd95bfff640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94006b4d0 0x7fd94006d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0ca810 tx=0x7fd948008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.009+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd950032e20 con 0x7fd95c06c2d0 2024-04-25T09:55:21.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.182+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd95c0cdca0 con 0x7fd95c06c2d0 2024-04-25T09:55:21.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.182+0000 7fd9597fa640 1 -- 172.21.15.148:0/4073216891 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd950079cb0 con 0x7fd95c06c2d0 2024-04-25T09:55:21.185 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94006b4d0 msgr2=0x7fd94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94006b4d0 0x7fd94006d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0ca810 tx=0x7fd948008040 comp rx=0 tx=0).stop 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 msgr2=0x7fd95c0aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 0x7fd95c0aa360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd950000c00 tx=0x7fd9500046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 shutdown_connections 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94006b4d0 0x7fd94006d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd95c15b7d0 0x7fd95c0d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd95c0b5620 0x7fd95c14fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 --2- 172.21.15.148:0/4073216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd95c06c2d0 0x7fd95c0aa360 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 >> 172.21.15.148:0/4073216891 conn(0x7fd95c0d0800 msgr2=0x7fd95c1609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 shutdown_connections 2024-04-25T09:55:21.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.185+0000 7fd962dd5640 1 -- 172.21.15.148:0/4073216891 wait complete. 2024-04-25T09:55:21.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T09:55:21.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.296+0000 7f715770a640 1 Processor -- start 2024-04-25T09:55:21.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.296+0000 7f715770a640 1 -- start start 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f715770a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71500d34c0 0x7f71500d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f715770a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f71500d08e0 0x7f71500b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f715770a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71500d2420 con 0x7f715015dac0 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f715770a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71500d3030 con 0x7f71500d08e0 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f715770a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71500d2700 con 0x7f71500d34c0 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f71500d08e0 0x7f71500b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f71500d08e0 0x7f71500b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35624/0 (socket says 172.21.15.148:35624) 2024-04-25T09:55:21.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 -- 172.21.15.148:0/1291715311 learned_addr learned my addr 172.21.15.148:0/1291715311 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7154ca5640 1 --2- 172.21.15.148:0/1291715311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71500d34c0 0x7f71500d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 -- 172.21.15.148:0/1291715311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71500d34c0 msgr2=0x7f71500d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:21.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 --2- 172.21.15.148:0/1291715311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71500d34c0 0x7f71500d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 -- 172.21.15.148:0/1291715311 >> v1:172.21.15.188:6789/0 conn(0x7f715015dac0 legacy=0x7f71500d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.297+0000 7f7155ca7640 1 -- 172.21.15.148:0/1291715311 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500d29e0 con 0x7f71500d08e0 2024-04-25T09:55:21.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.298+0000 7f7155ca7640 1 --2- 172.21.15.148:0/1291715311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b6990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7148009e50 tx=0x7f714802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08cd1e4981f5bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.298+0000 7f7147fff640 1 -- 172.21.15.148:0/1291715311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148036070 con 0x7f71500d08e0 2024-04-25T09:55:21.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.298+0000 7f7147fff640 1 -- 172.21.15.148:0/1291715311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148031600 con 0x7f71500d08e0 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.298+0000 7f7147fff640 1 -- 172.21.15.148:0/1291715311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480319d0 con 0x7f71500d08e0 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 -- 172.21.15.148:0/1291715311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 msgr2=0x7f71500b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 --2- 172.21.15.148:0/1291715311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b6990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7148009e50 tx=0x7f714802a7a0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 -- 172.21.15.148:0/1291715311 shutdown_connections 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 --2- 172.21.15.148:0/1291715311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b6990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 --2- 172.21.15.148:0/1291715311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71500d34c0 0x7f71500d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 -- 172.21.15.148:0/1291715311 >> 172.21.15.148:0/1291715311 conn(0x7f7150061c30 msgr2=0x7f715006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 -- 172.21.15.148:0/1291715311 shutdown_connections 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.299+0000 7f715770a640 1 -- 172.21.15.148:0/1291715311 wait complete. 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.300+0000 7f715770a640 1 Processor -- start 2024-04-25T09:55:21.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.300+0000 7f715770a640 1 -- start start 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71500d34c0 0x7f7150152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f715015dac0 0x7f71501600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715015dea0 con 0x7f71500d08e0 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71500d2100 con 0x7f71500d34c0 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71500b97d0 con 0x7f715015dac0 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35636/0 (socket says 172.21.15.148:35636) 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 -- 172.21.15.148:0/898362678 learned_addr learned my addr 172.21.15.148:0/898362678 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f7155ca7640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f715015dac0 0x7f71501600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f7154ca5640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71500d34c0 0x7f7150152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 -- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f715015dac0 msgr2=0x7f71501600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f715015dac0 0x7f71501600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 -- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71500d34c0 msgr2=0x7f7150152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71500d34c0 0x7f7150152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 -- 172.21.15.148:0/898362678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71501609c0 con 0x7f71500d08e0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f71554a6640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b4ee0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c3c0 tx=0x7f714c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0052d0 con 0x7f71500d08e0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c005490 con 0x7f71500d08e0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7150160ca0 con 0x7f71500d08e0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.301+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c007260 con 0x7f71500d08e0 2024-04-25T09:55:21.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.302+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71500cbf70 con 0x7f71500d08e0 2024-04-25T09:55:21.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.302+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7150073a00 con 0x7f71500d08e0 2024-04-25T09:55:21.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.303+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f714c007400 con 0x7f71500d08e0 2024-04-25T09:55:21.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.303+0000 7f7145ffb640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f713406b4d0 0x7f713406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.303+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f714c089af0 con 0x7f71500d08e0 2024-04-25T09:55:21.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.303+0000 7f7154ca5640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f713406b4d0 0x7f713406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.304+0000 7f7154ca5640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f713406b4d0 0x7f713406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f71501571a0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.307+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f714c058560 con 0x7f71500d08e0 2024-04-25T09:55:21.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.476+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f715015ada0 con 0x7f71500d08e0 2024-04-25T09:55:21.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.476+0000 7f7145ffb640 1 -- 172.21.15.148:0/898362678 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f714c057e60 con 0x7f71500d08e0 2024-04-25T09:55:21.479 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f713406b4d0 msgr2=0x7f713406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f713406b4d0 0x7f713406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f71501571a0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 msgr2=0x7f71500b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b4ee0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c3c0 tx=0x7f714c00c880 comp rx=0 tx=0).stop 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 shutdown_connections 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f713406b4d0 0x7f713406d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f715015dac0 0x7f71501600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71500d34c0 0x7f7150152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 --2- 172.21.15.148:0/898362678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71500d08e0 0x7f71500b4ee0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 >> 172.21.15.148:0/898362678 conn(0x7f7150061c30 msgr2=0x7f71500c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 shutdown_connections 2024-04-25T09:55:21.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.478+0000 7f715770a640 1 -- 172.21.15.148:0/898362678 wait complete. 2024-04-25T09:55:21.492 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.586+0000 7f7876b5c640 1 Processor -- start 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.586+0000 7f7876b5c640 1 -- start start 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f7876b5c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78700d13f0 0x7f78700d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f7876b5c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78700d1dc0 0x7f78700b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f7876b5c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78700d0350 con 0x7f787015bd30 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f7876b5c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78700d0630 con 0x7f78700d1dc0 2024-04-25T09:55:21.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f7876b5c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78700d0910 con 0x7f78700d13f0 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78700d1dc0 0x7f78700b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78700d1dc0 0x7f78700b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35650/0 (socket says 172.21.15.148:35650) 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 -- 172.21.15.148:0/3523710744 learned_addr learned my addr 172.21.15.148:0/3523710744 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f786ffff640 1 --2- 172.21.15.148:0/3523710744 >> v2:172.21.15.188:3300/0 conn(0x7f78700d13f0 0x7f78700d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 -- 172.21.15.148:0/3523710744 >> v2:172.21.15.188:3300/0 conn(0x7f78700d13f0 msgr2=0x7f78700d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 --2- 172.21.15.148:0/3523710744 >> v2:172.21.15.188:3300/0 conn(0x7f78700d13f0 0x7f78700d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 -- 172.21.15.148:0/3523710744 >> v1:172.21.15.188:6789/0 conn(0x7f787015bd30 legacy=0x7f78700cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:21.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.587+0000 7f78750f9640 1 -- 172.21.15.148:0/3523710744 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700d0bf0 con 0x7f78700d1dc0 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f78750f9640 1 --2- 172.21.15.148:0/3523710744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78700d1dc0 0x7f78700b4a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f787015f7f0 tx=0x7f786802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a2432561455625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f786f7fe640 1 -- 172.21.15.148:0/3523710744 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868036070 con 0x7f78700d1dc0 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f786f7fe640 1 -- 172.21.15.148:0/3523710744 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868031600 con 0x7f78700d1dc0 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f786f7fe640 1 -- 172.21.15.148:0/3523710744 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680319d0 con 0x7f78700d1dc0 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f7876b5c640 1 -- 172.21.15.148:0/3523710744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78700d1dc0 msgr2=0x7f78700b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f7876b5c640 1 --2- 172.21.15.148:0/3523710744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78700d1dc0 0x7f78700b4a30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f787015f7f0 tx=0x7f786802a780 comp rx=0 tx=0).stop 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f7876b5c640 1 -- 172.21.15.148:0/3523710744 shutdown_connections 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f7876b5c640 1 --2- 172.21.15.148:0/3523710744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78700d1dc0 0x7f78700b4a30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f7876b5c640 1 --2- 172.21.15.148:0/3523710744 >> v2:172.21.15.188:3300/0 conn(0x7f78700d13f0 0x7f78700d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.588+0000 7f7876b5c640 1 -- 172.21.15.148:0/3523710744 >> 172.21.15.148:0/3523710744 conn(0x7f787006f590 msgr2=0x7f787006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.589+0000 7f7876b5c640 1 -- 172.21.15.148:0/3523710744 shutdown_connections 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.589+0000 7f7876b5c640 1 -- 172.21.15.148:0/3523710744 wait complete. 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.590+0000 7f7876b5c640 1 Processor -- start 2024-04-25T09:55:21.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.590+0000 7f7876b5c640 1 -- start start 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78700d13f0 0x7f787014b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78700d1dc0 0x7f78701f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 0x7f78700c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78700d0010 con 0x7f787015bd30 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787015c110 con 0x7f78700d13f0 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78700b73b0 con 0x7f78700d1dc0 2024-04-25T09:55:21.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 0x7f78700c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 0x7f78700c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35662/0 (socket says 172.21.15.148:35662) 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 -- 172.21.15.148:0/2093702895 learned_addr learned my addr 172.21.15.148:0/2093702895 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78748f8640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78700d13f0 0x7f787014b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 -- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78700d1dc0 msgr2=0x7f78701f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f786ffff640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78700d1dc0 0x7f78701f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78700d1dc0 0x7f78701f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 -- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78700d13f0 msgr2=0x7f787014b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78700d13f0 0x7f787014b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 -- 172.21.15.148:0/2093702895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700c8e90 con 0x7f787015bd30 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78748f8640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78700d13f0 0x7f787014b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f78750f9640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 0x7f78700c8950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7868009f40 tx=0x7f7868004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868036050 con 0x7f787015bd30 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7868042070 con 0x7f787015bd30 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78700c9110 con 0x7f787015bd30 2024-04-25T09:55:21.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868009510 con 0x7f787015bd30 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.591+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78700c96d0 con 0x7f787015bd30 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.592+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7870073990 con 0x7f787015bd30 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.593+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7868007bf0 con 0x7f787015bd30 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.593+0000 7f786d7fa640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f785406b4d0 0x7f785406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.593+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f78680ab3d0 con 0x7f787015bd30 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.593+0000 7f78748f8640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f785406b4d0 0x7f785406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.594+0000 7f78748f8640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f785406b4d0 0x7f785406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f786400a830 tx=0x7f7864008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.597+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78700c8e90 con 0x7f787015bd30 2024-04-25T09:55:21.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.803+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78700b2170 con 0x7f787015bd30 2024-04-25T09:55:21.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.804+0000 7f786d7fa640 1 -- 172.21.15.148:0/2093702895 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f7868045360 con 0x7f787015bd30 2024-04-25T09:55:21.806 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:21.807 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:55:21.807 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:55:21.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.805+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f785406b4d0 msgr2=0x7f785406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.805+0000 7f7876b5c640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f785406b4d0 0x7f785406d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f786400a830 tx=0x7f7864008040 comp rx=0 tx=0).stop 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.805+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 msgr2=0x7f78700c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.805+0000 7f7876b5c640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 0x7f78700c8950 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7868009f40 tx=0x7f7868004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.805+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 shutdown_connections 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f785406b4d0 0x7f785406d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787015bd30 0x7f78700c8950 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78700d1dc0 0x7f78701f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 --2- 172.21.15.148:0/2093702895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78700d13f0 0x7f787014b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 >> 172.21.15.148:0/2093702895 conn(0x7f787006f590 msgr2=0x7f78700b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 shutdown_connections 2024-04-25T09:55:21.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.806+0000 7f7876b5c640 1 -- 172.21.15.148:0/2093702895 wait complete. 2024-04-25T09:55:21.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:21.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.913+0000 7fef963b0640 1 Processor -- start 2024-04-25T09:55:21.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef963b0640 1 -- start start 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef963b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef9015a9b0 0x7fef900b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef963b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef900b4890 0x7fef900aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef963b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9015ad90 con 0x7fef9006ee40 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef963b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9006f370 con 0x7fef9015a9b0 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef963b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9006f650 con 0x7fef900b4890 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.914+0000 7fef8ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef9015a9b0 0x7fef900b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef9015a9b0 0x7fef900b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35668/0 (socket says 172.21.15.148:35668) 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 -- 172.21.15.148:0/51193267 learned_addr learned my addr 172.21.15.148:0/51193267 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef9494d640 1 --2- 172.21.15.148:0/51193267 >> v2:172.21.15.188:3300/0 conn(0x7fef900b4890 0x7fef900aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 -- 172.21.15.148:0/51193267 >> v2:172.21.15.188:3300/0 conn(0x7fef900b4890 msgr2=0x7fef900aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 --2- 172.21.15.148:0/51193267 >> v2:172.21.15.188:3300/0 conn(0x7fef900b4890 0x7fef900aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 -- 172.21.15.148:0/51193267 >> v1:172.21.15.188:6789/0 conn(0x7fef9006ee40 legacy=0x7fef900b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 -- 172.21.15.148:0/51193267 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9006f930 con 0x7fef9015a9b0 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.915+0000 7fef8ffff640 1 --2- 172.21.15.148:0/51193267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9015a9b0 0x7fef900b37b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef9006cf50 tx=0x7fef8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3dcfc8f732aa24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef8effd640 1 -- 172.21.15.148:0/51193267 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80037070 con 0x7fef9015a9b0 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef8effd640 1 -- 172.21.15.148:0/51193267 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80032440 con 0x7fef9015a9b0 2024-04-25T09:55:21.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef8effd640 1 -- 172.21.15.148:0/51193267 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80032810 con 0x7fef9015a9b0 2024-04-25T09:55:21.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 -- 172.21.15.148:0/51193267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9015a9b0 msgr2=0x7fef900b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 --2- 172.21.15.148:0/51193267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9015a9b0 0x7fef900b37b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef9006cf50 tx=0x7fef8002a270 comp rx=0 tx=0).stop 2024-04-25T09:55:21.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 -- 172.21.15.148:0/51193267 shutdown_connections 2024-04-25T09:55:21.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 --2- 172.21.15.148:0/51193267 >> v2:172.21.15.188:3300/0 conn(0x7fef900b4890 0x7fef900aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 --2- 172.21.15.148:0/51193267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9015a9b0 0x7fef900b37b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 -- 172.21.15.148:0/51193267 >> 172.21.15.148:0/51193267 conn(0x7fef900b5af0 msgr2=0x7fef900b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 -- 172.21.15.148:0/51193267 shutdown_connections 2024-04-25T09:55:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.916+0000 7fef963b0640 1 -- 172.21.15.148:0/51193267 wait complete. 2024-04-25T09:55:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.917+0000 7fef963b0640 1 Processor -- start 2024-04-25T09:55:21.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 -- start start 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9006ee40 0x7fef90143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 0x7fef9014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a9b0 0x7fef9014f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b3cf0 con 0x7fef900b4890 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9006fef0 con 0x7fef9015a9b0 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef963b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b3e70 con 0x7fef9006ee40 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 0x7fef9014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 0x7fef9014f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35674/0 (socket says 172.21.15.148:35674) 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.918+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2684668187 learned_addr learned my addr 172.21.15.148:0/2684668187 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9006ee40 msgr2=0x7fef90143740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef9494d640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a9b0 0x7fef9014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8ffff640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9006ee40 0x7fef90143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8f7fe640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9006ee40 0x7fef90143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a9b0 msgr2=0x7fef9014f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8f7fe640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a9b0 0x7fef9014f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8f7fe640 1 -- 172.21.15.148:0/2684668187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90148bb0 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8f7fe640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 0x7fef9014f110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c430 tx=0x7fef8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84011790 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84017070 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef90148e90 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84011bb0 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.919+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9015b9e0 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.920+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef90073990 con 0x7fef900b4890 2024-04-25T09:55:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.920+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef8401c460 con 0x7fef900b4890 2024-04-25T09:55:21.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.921+0000 7fef8cff9640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef7c06b4d0 0x7fef7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:21.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.921+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fef8408a470 con 0x7fef900b4890 2024-04-25T09:55:21.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.921+0000 7fef8ffff640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef7c06b4d0 0x7fef7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:21.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.921+0000 7fef8ffff640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef7c06b4d0 0x7fef7c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fef80034000 tx=0x7fef8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:21.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:21.925+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef84058f70 con 0x7fef900b4890 2024-04-25T09:55:22.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.095+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef9009e5a0 con 0x7fef900b4890 2024-04-25T09:55:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.096+0000 7fef8cff9640 1 -- 172.21.15.148:0/2684668187 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7fef84058870 con 0x7fef900b4890 2024-04-25T09:55:22.100 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:22.100 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:22.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef7c06b4d0 msgr2=0x7fef7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef7c06b4d0 0x7fef7c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fef80034000 tx=0x7fef8002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:22.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 msgr2=0x7fef9014f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 0x7fef9014f110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c430 tx=0x7fef8400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 shutdown_connections 2024-04-25T09:55:22.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef7c06b4d0 0x7fef7c06d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9015a9b0 0x7fef9014f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef900b4890 0x7fef9014f110 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 --2- 172.21.15.148:0/2684668187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9006ee40 0x7fef90143740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 >> 172.21.15.148:0/2684668187 conn(0x7fef900b5af0 msgr2=0x7fef9014e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:22.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 shutdown_connections 2024-04-25T09:55:22.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.098+0000 7fef963b0640 1 -- 172.21.15.148:0/2684668187 wait complete. 2024-04-25T09:55:22.113 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:55:22.113 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:22.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.207+0000 7f4a7b1d5640 1 Processor -- start 2024-04-25T09:55:22.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.207+0000 7f4a7b1d5640 1 -- start start 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a7b1d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 0x7f4a740cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a7b1d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a740cfe40 0x7f4a740d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a7b1d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a740b5a40 con 0x7f4a740d1930 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a7b1d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a740d0350 con 0x7f4a740cfe40 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a7b1d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a740d0630 con 0x7f4a7415bd30 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a79772640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4a740d1930 0x7f4a740ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55664/0 (socket says 172.21.15.148:55664) 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a79772640 1 -- 172.21.15.148:0/1525368480 learned_addr learned my addr 172.21.15.148:0/1525368480 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a78f71640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 0x7f4a740cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a73fff640 1 --2- 172.21.15.148:0/1525368480 >> v2:172.21.15.188:3300/0 conn(0x7f4a740cfe40 0x7f4a740d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.208+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398331159 0 0) 0x7f4a740b5a40 con 0x7f4a740d1930 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a5c003520 con 0x7f4a740d1930 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1088381067 0 0) 0x7f4a5c003520 con 0x7f4a740d1930 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 msgr2=0x7f4a740cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 0x7f4a740cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 >> v2:172.21.15.188:3300/0 conn(0x7f4a740cfe40 msgr2=0x7f4a740d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 --2- 172.21.15.148:0/1525368480 >> v2:172.21.15.188:3300/0 conn(0x7f4a740cfe40 0x7f4a740d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a740d0910 con 0x7f4a740d1930 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4a6c0019f0 con 0x7f4a740d1930 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 >> v1:172.21.15.148:6789/0 conn(0x7f4a740d1930 legacy=0x7f4a740ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0039b0 0x7f4a5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c004800 0x7f4a5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 0x7f4a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a740d0350 con 0x7f4a5c0039b0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a740d0630 con 0x7f4a5c004800 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c00b9c0 con 0x7f4a7415bd30 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a78f71640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0039b0 0x7f4a5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a73fff640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c004800 0x7f4a5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a79772640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 0x7f4a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a78f71640 1 -- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 msgr2=0x7f4a5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a78f71640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a7415bd30 0x7f4a5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a78f71640 1 -- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c004800 msgr2=0x7f4a5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a78f71640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c004800 0x7f4a5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a78f71640 1 -- 172.21.15.148:0/1525368480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c003520 con 0x7f4a5c0039b0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.209+0000 7f4a73fff640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c004800 0x7f4a5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.210+0000 7f4a78f71640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0039b0 0x7f4a5c003dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009ac0 tx=0x7f4a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcdb0298a9cf653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.210+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64036070 con 0x7f4a5c0039b0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.210+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a64031400 con 0x7f4a5c0039b0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.210+0000 7f4a737fe640 1 -- 172.21.15.148:0/1525368480 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64009690 con 0x7f4a5c0039b0 2024-04-25T09:55:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/1525368480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0039b0 msgr2=0x7f4a5c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/1525368480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0039b0 0x7f4a5c003dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009ac0 tx=0x7f4a5c005830 comp rx=0 tx=0).stop 2024-04-25T09:55:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a79772640 1 -- 172.21.15.148:0/1525368480 reap_dead start 2024-04-25T09:55:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/1525368480 shutdown_connections 2024-04-25T09:55:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/1525368480 >> 172.21.15.148:0/1525368480 conn(0x7f4a7406f590 msgr2=0x7f4a7406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/1525368480 shutdown_connections 2024-04-25T09:55:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.211+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/1525368480 wait complete. 2024-04-25T09:55:22.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.212+0000 7f4a7b1d5640 1 Processor -- start 2024-04-25T09:55:22.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.212+0000 7f4a7b1d5640 1 -- start start 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0039b0 0x7f4a741f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a740a8640 0x7f4a740b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 0x7f4a74159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7415c110 con 0x7f4a74159580 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a740d0ed0 con 0x7f4a740a8640 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a740b73b0 con 0x7f4a5c0039b0 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 0x7f4a74159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 0x7f4a74159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35706/0 (socket says 172.21.15.148:35706) 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 -- 172.21.15.148:0/993143049 learned_addr learned my addr 172.21.15.148:0/993143049 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a73fff640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a740a8640 0x7f4a740b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a78f71640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0039b0 0x7f4a741f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 -- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0039b0 msgr2=0x7f4a741f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0039b0 0x7f4a741f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 -- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a740a8640 msgr2=0x7f4a740b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a740a8640 0x7f4a740b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 -- 172.21.15.148:0/993143049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a740b0570 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a79772640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 0x7f4a74159a30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4a740b1390 tx=0x7f4a6c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00ea80 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00ec40 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a740a8e20 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c014430 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.213+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a741f2d90 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.214+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a74073990 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.215+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a6c0145f0 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.215+0000 7f4a717fa640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6006b500 0x7f4a6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.215+0000 7f4a78f71640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6006b500 0x7f4a6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.215+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f4a6c08a270 con 0x7f4a74159580 2024-04-25T09:55:22.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.216+0000 7f4a78f71640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6006b500 0x7f4a6006d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a64033000 tx=0x7f4a64008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.220+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a6c058ce0 con 0x7f4a74159580 2024-04-25T09:55:22.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.425+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a740b6ad0 con 0x7f4a74159580 2024-04-25T09:55:22.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.425+0000 7f4a717fa640 1 -- 172.21.15.148:0/993143049 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7f4a6c0585e0 con 0x7f4a74159580 2024-04-25T09:55:22.428 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:22.428 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:55:22.429 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:55:22.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6006b500 msgr2=0x7f4a6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6006b500 0x7f4a6006d9c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a64033000 tx=0x7f4a64008030 comp rx=0 tx=0).stop 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 msgr2=0x7f4a74159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 0x7f4a74159a30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4a740b1390 tx=0x7f4a6c007bc0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 shutdown_connections 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6006b500 0x7f4a6006d9c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a74159580 0x7f4a74159a30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a740a8640 0x7f4a740b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 --2- 172.21.15.148:0/993143049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0039b0 0x7f4a741f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.427+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 >> 172.21.15.148:0/993143049 conn(0x7f4a7406f590 msgr2=0x7f4a740b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.428+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 shutdown_connections 2024-04-25T09:55:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.428+0000 7f4a7b1d5640 1 -- 172.21.15.148:0/993143049 wait complete. 2024-04-25T09:55:22.441 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:22.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.535+0000 7fb959a37640 1 Processor -- start 2024-04-25T09:55:22.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.536+0000 7fb959a37640 1 -- start start 2024-04-25T09:55:22.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.536+0000 7fb959a37640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb95415dc00 0x7fb9540c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.536+0000 7fb959a37640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9540ceb20 0x7fb9540d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.536+0000 7fb959a37640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95415dfe0 con 0x7fb9540d1c00 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.536+0000 7fb959a37640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540d2060 con 0x7fb9540ceb20 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.536+0000 7fb959a37640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540d00c0 con 0x7fb95415dc00 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9540ceb20 0x7fb9540d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb9540ceb20 0x7fb9540d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35708/0 (socket says 172.21.15.148:35708) 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 -- 172.21.15.148:0/3935990862 learned_addr learned my addr 172.21.15.148:0/3935990862 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb952ffd640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb95415dc00 0x7fb9540c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 -- 172.21.15.148:0/3935990862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb95415dc00 msgr2=0x7fb9540c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb95415dc00 0x7fb9540c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 -- 172.21.15.148:0/3935990862 >> v1:172.21.15.188:6789/0 conn(0x7fb9540d1c00 legacy=0x7fb9540ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb94bfff640 1 -- 172.21.15.148:0/3935990862 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540d03a0 con 0x7fb9540ceb20 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.537+0000 7fb952ffd640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb95415dc00 0x7fb9540c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb94bfff640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9540ceb20 0x7fb9540d16c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009870 tx=0x7fb93c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa555f5b6c7c1edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb9527fc640 1 -- 172.21.15.148:0/3935990862 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c037070 con 0x7fb9540ceb20 2024-04-25T09:55:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb9527fc640 1 -- 172.21.15.148:0/3935990862 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c032440 con 0x7fb9540ceb20 2024-04-25T09:55:22.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb9527fc640 1 -- 172.21.15.148:0/3935990862 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c032810 con 0x7fb9540ceb20 2024-04-25T09:55:22.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 -- 172.21.15.148:0/3935990862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9540ceb20 msgr2=0x7fb9540d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9540ceb20 0x7fb9540d16c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009870 tx=0x7fb93c02a210 comp rx=0 tx=0).stop 2024-04-25T09:55:22.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 -- 172.21.15.148:0/3935990862 shutdown_connections 2024-04-25T09:55:22.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9540ceb20 0x7fb9540d16c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 --2- 172.21.15.148:0/3935990862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb95415dc00 0x7fb9540c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 -- 172.21.15.148:0/3935990862 >> 172.21.15.148:0/3935990862 conn(0x7fb95406f5d0 msgr2=0x7fb95406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 -- 172.21.15.148:0/3935990862 shutdown_connections 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.538+0000 7fb959a37640 1 -- 172.21.15.148:0/3935990862 wait complete. 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.539+0000 7fb959a37640 1 Processor -- start 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 -- start start 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9540ceb20 0x7fb9541f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9540d1c00 0x7fb9540ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 0x7fb9540caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cc130 con 0x7fb95415dc00 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cc720 con 0x7fb9540ceb20 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb959a37640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cfd90 con 0x7fb9540d1c00 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 0x7fb9540caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb9537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 0x7fb9540caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35724/0 (socket says 172.21.15.148:35724) 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.540+0000 7fb9537fe640 1 -- 172.21.15.148:0/2992247853 learned_addr learned my addr 172.21.15.148:0/2992247853 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:22.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb952ffd640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9540ceb20 0x7fb9541f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb94bfff640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9540d1c00 0x7fb9540ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb9537fe640 1 -- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9540d1c00 msgr2=0x7fb9540ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb9537fe640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9540d1c00 0x7fb9540ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb9537fe640 1 -- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9540ceb20 msgr2=0x7fb9541f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb9537fe640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9540ceb20 0x7fb9541f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb9537fe640 1 -- 172.21.15.148:0/2992247853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9541e93c0 con 0x7fb95415dc00 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb9537fe640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 0x7fb9540caca0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c7b0 tx=0x7fb94c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c004b60 con 0x7fb95415dc00 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c004d00 con 0x7fb95415dc00 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00aa90 con 0x7fb95415dc00 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9541e96a0 con 0x7fb95415dc00 2024-04-25T09:55:22.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.541+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9541e9c60 con 0x7fb95415dc00 2024-04-25T09:55:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.542+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb954073990 con 0x7fb95415dc00 2024-04-25T09:55:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.542+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb94c00ac30 con 0x7fb95415dc00 2024-04-25T09:55:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.543+0000 7fb950ff9640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.543+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fb94c08a9b0 con 0x7fb95415dc00 2024-04-25T09:55:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.543+0000 7fb952ffd640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.543+0000 7fb952ffd640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb94400b840 tx=0x7fb944008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.547+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb94c020070 con 0x7fb95415dc00 2024-04-25T09:55:22.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.758+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9540c66c0 con 0x7fb95415dc00 2024-04-25T09:55:22.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.758+0000 7fb950ff9640 1 -- 172.21.15.148:0/2992247853 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4943 (secure 0 0 0) 0x7fb94c059390 con 0x7fb95415dc00 2024-04-25T09:55:22.761 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:22.761 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/917156230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":917156230},{"type":"v1","addr":"172.21.15.188:6834","nonce":917156230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3969636216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3969636216},{"type":"v1","addr":"172.21.15.148:6836","nonce":3969636216}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2682437477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2682437477},{"type":"v1","addr":"172.21.15.148:6837","nonce":2682437477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:54:23.300442+0000","modified":"2024-04-25T09:54:44.431176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1500268127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1500268127},{"type":"v1","addr":"172.21.15.188:6835","nonce":1500268127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T09:55:22.762 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 9 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 msgr2=0x7fb93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb94400b840 tx=0x7fb944008040 comp rx=0 tx=0).stop 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 msgr2=0x7fb9540caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 0x7fb9540caca0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c7b0 tx=0x7fb94c00cc70 comp rx=0 tx=0).stop 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 shutdown_connections 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95415dc00 0x7fb9540caca0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9540d1c00 0x7fb9540ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 --2- 172.21.15.148:0/2992247853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9540ceb20 0x7fb9541f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.760+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 >> 172.21.15.148:0/2992247853 conn(0x7fb95406f5d0 msgr2=0x7fb9540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.761+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 shutdown_connections 2024-04-25T09:55:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.761+0000 7fb959a37640 1 -- 172.21.15.148:0/2992247853 wait complete. 2024-04-25T09:55:22.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:22.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.869+0000 7fd536d9e640 1 Processor -- start 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.869+0000 7fd536d9e640 1 -- start start 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd536d9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 0x7fd5301625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd536d9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5300cce50 con 0x7fd53015dc90 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd536d9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd530161af0 con 0x7fd5300cc8e0 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd536d9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd530161dd0 con 0x7fd5300d1f20 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd534b3a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd53015dc90 0x7fd53015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55668/0 (socket says 172.21.15.148:55668) 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd534b3a640 1 -- 172.21.15.148:0/1346185316 learned_addr learned my addr 172.21.15.148:0/1346185316 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd53533b640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 0x7fd5301625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017388313 0 0) 0x7fd5300cce50 con 0x7fd53015dc90 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd520003520 con 0x7fd53015dc90 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2676854170 0 0) 0x7fd520003520 con 0x7fd53015dc90 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 msgr2=0x7fd5301625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 0x7fd5301625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 >> v1:172.21.15.188:6789/0 conn(0x7fd5300cc8e0 legacy=0x7fd5300d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5301620b0 con 0x7fd53015dc90 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd524001670 con 0x7fd53015dc90 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 >> v1:172.21.15.148:6789/0 conn(0x7fd53015dc90 legacy=0x7fd53015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5200039e0 0x7fd520003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 0x7fd520004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd520004d40 0x7fd5200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd530161dd0 con 0x7fd5200039e0 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52000baa0 con 0x7fd520004d40 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52000bca0 con 0x7fd5300d1f20 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd534b3a640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5200039e0 0x7fd520003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.870+0000 7fd52ffff640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 0x7fd520004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd53533b640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd520004d40 0x7fd5200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd534b3a640 1 -- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 msgr2=0x7fd520004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd534b3a640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5300d1f20 0x7fd520004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd534b3a640 1 -- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd520004d40 msgr2=0x7fd5200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:22.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd534b3a640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd520004d40 0x7fd5200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd534b3a640 1 -- 172.21.15.148:0/1346185316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520003520 con 0x7fd5200039e0 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd534b3a640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5200039e0 0x7fd520003dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c240 tx=0x7fd520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fff43c8bf7b90b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524007d10 con 0x7fd5200039e0 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524005ad0 con 0x7fd5200039e0 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.871+0000 7fd52f7fe640 1 -- 172.21.15.148:0/1346185316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524007070 con 0x7fd5200039e0 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd536d9e640 1 -- 172.21.15.148:0/1346185316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5200039e0 msgr2=0x7fd520003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd536d9e640 1 --2- 172.21.15.148:0/1346185316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5200039e0 0x7fd520003dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c240 tx=0x7fd520005830 comp rx=0 tx=0).stop 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd53533b640 1 -- 172.21.15.148:0/1346185316 reap_dead start 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd536d9e640 1 -- 172.21.15.148:0/1346185316 shutdown_connections 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd536d9e640 1 -- 172.21.15.148:0/1346185316 >> 172.21.15.148:0/1346185316 conn(0x7fd53006f590 msgr2=0x7fd53006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd536d9e640 1 -- 172.21.15.148:0/1346185316 shutdown_connections 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.872+0000 7fd536d9e640 1 -- 172.21.15.148:0/1346185316 wait complete. 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.873+0000 7fd536d9e640 1 Processor -- start 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.873+0000 7fd536d9e640 1 -- start start 2024-04-25T09:55:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd536d9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5200039e0 0x7fd5300cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd536d9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd53014b990 0x7fd5301601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd536d9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 0x7fd53014b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd536d9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53015e1f0 con 0x7fd53014ac80 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd536d9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53015e070 con 0x7fd53014b990 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd536d9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301625d0 con 0x7fd5200039e0 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 0x7fd53014b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 0x7fd53014b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35744/0 (socket says 172.21.15.148:35744) 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 -- 172.21.15.148:0/4025134690 learned_addr learned my addr 172.21.15.148:0/4025134690 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd534b3a640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5200039e0 0x7fd5300cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd52ffff640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd53014b990 0x7fd5301601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 -- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5200039e0 msgr2=0x7fd5300cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5200039e0 0x7fd5300cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 -- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd53014b990 msgr2=0x7fd5301601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:22.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd53014b990 0x7fd5301601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd53533b640 1 -- 172.21.15.148:0/4025134690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53015ee50 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.874+0000 7fd52ffff640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd53014b990 0x7fd5301601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.875+0000 7fd53533b640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 0x7fd53014b100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd5300d27b0 tx=0x7fd5280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.875+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011e40 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.875+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52801b070 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.875+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53015f130 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.875+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528003990 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.875+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53015f6f0 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.876+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530073990 con 0x7fd53014ac80 2024-04-25T09:55:22.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.876+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd528005d10 con 0x7fd53014ac80 2024-04-25T09:55:22.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.876+0000 7fd52d7fa640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd51406b500 0x7fd51406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:22.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.877+0000 7fd534b3a640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd51406b500 0x7fd51406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:22.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.877+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7fd52808a890 con 0x7fd53014ac80 2024-04-25T09:55:22.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.877+0000 7fd534b3a640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd51406b500 0x7fd51406d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd524033000 tx=0x7fd524009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:22.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:22.881+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd528059270 con 0x7fd53014ac80 2024-04-25T09:55:23.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.053+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5300cf6b0 con 0x7fd53014ac80 2024-04-25T09:55:23.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.054+0000 7fd52d7fa640 1 -- 172.21.15.148:0/4025134690 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18966 (secure 0 0 0) 0x7fd528058b70 con 0x7fd53014ac80 2024-04-25T09:55:23.057 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:23.057 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:04.156217+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:54:20.970314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T09:54:22.267323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T09:54:25.998970+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1},{"from":7,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd51406b500 msgr2=0x7fd51406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd51406b500 0x7fd51406d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd524033000 tx=0x7fd524009030 comp rx=0 tx=0).stop 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 msgr2=0x7fd53014b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 0x7fd53014b100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd5300d27b0 tx=0x7fd5280115a0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 shutdown_connections 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd51406b500 0x7fd51406d9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd53014ac80 0x7fd53014b100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd53014b990 0x7fd5301601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 --2- 172.21.15.148:0/4025134690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5200039e0 0x7fd5300cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 >> 172.21.15.148:0/4025134690 conn(0x7fd53006f590 msgr2=0x7fd5300b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 shutdown_connections 2024-04-25T09:55:23.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.056+0000 7fd536d9e640 1 -- 172.21.15.148:0/4025134690 wait complete. 2024-04-25T09:55:23.070 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:55:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.165+0000 7f5ef275c640 1 Processor -- start 2024-04-25T09:55:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.165+0000 7f5ef275c640 1 -- start start 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef275c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eec15b5a0 0x7f5eec0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef275c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eec15b980 0x7f5eec0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef275c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec162450 0x7f5eec0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef275c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0cf320 con 0x7f5eec15b980 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef275c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0cf040 con 0x7f5eec15b5a0 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef275c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec161970 con 0x7f5eec162450 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eec15b980 0x7f5eec0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eec15b980 0x7f5eec0b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35750/0 (socket says 172.21.15.148:35750) 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/1035475654 learned_addr learned my addr 172.21.15.148:0/1035475654 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eebfff640 1 --2- 172.21.15.148:0/1035475654 >> v2:172.21.15.188:3300/0 conn(0x7f5eec15b5a0 0x7f5eec0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5ef0cf9640 1 --2- 172.21.15.148:0/1035475654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec162450 0x7f5eec0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/1035475654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec162450 msgr2=0x7f5eec0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 --2- 172.21.15.148:0/1035475654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec162450 0x7f5eec0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/1035475654 >> v2:172.21.15.188:3300/0 conn(0x7f5eec15b5a0 msgr2=0x7f5eec0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 --2- 172.21.15.148:0/1035475654 >> v2:172.21.15.188:3300/0 conn(0x7f5eec15b5a0 0x7f5eec0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.166+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/1035475654 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec161c50 con 0x7f5eec15b980 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.167+0000 7f5eeb7fe640 1 --2- 172.21.15.148:0/1035475654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0b5a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5eec08c0d0 tx=0x7f5ee002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479b80cf45de78e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.167+0000 7f5eeaffd640 1 -- 172.21.15.148:0/1035475654 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004030 con 0x7f5eec15b980 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.167+0000 7f5eeaffd640 1 -- 172.21.15.148:0/1035475654 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee00041d0 con 0x7f5eec15b980 2024-04-25T09:55:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.167+0000 7f5eeaffd640 1 -- 172.21.15.148:0/1035475654 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0003610 con 0x7f5eec15b980 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 -- 172.21.15.148:0/1035475654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 msgr2=0x7f5eec0b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 --2- 172.21.15.148:0/1035475654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0b5a70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5eec08c0d0 tx=0x7f5ee002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 -- 172.21.15.148:0/1035475654 shutdown_connections 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 --2- 172.21.15.148:0/1035475654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec162450 0x7f5eec0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 --2- 172.21.15.148:0/1035475654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0b5a70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 --2- 172.21.15.148:0/1035475654 >> v2:172.21.15.188:3300/0 conn(0x7f5eec15b5a0 0x7f5eec0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 -- 172.21.15.148:0/1035475654 >> 172.21.15.148:0/1035475654 conn(0x7f5eec0d0530 msgr2=0x7f5eec0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 -- 172.21.15.148:0/1035475654 shutdown_connections 2024-04-25T09:55:23.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.168+0000 7f5ef275c640 1 -- 172.21.15.148:0/1035475654 wait complete. 2024-04-25T09:55:23.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.169+0000 7f5ef275c640 1 Processor -- start 2024-04-25T09:55:23.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.169+0000 7f5ef275c640 1 -- start start 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5ef275c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec15b5a0 0x7f5eec1f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5ef275c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5ef275c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eec162450 0x7f5eec14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5ef275c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0ce9d0 con 0x7f5eec15b980 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5ef275c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0ce850 con 0x7f5eec162450 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5ef275c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0ceb50 con 0x7f5eec15b5a0 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:35758/0 (socket says 172.21.15.148:35758) 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/4078000118 learned_addr learned my addr 172.21.15.148:0/4078000118 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eebfff640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec15b5a0 0x7f5eec1f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec15b5a0 msgr2=0x7f5eec1f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec15b5a0 0x7f5eec1f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eec162450 msgr2=0x7f5eec14f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eec162450 0x7f5eec14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 -- 172.21.15.148:0/4078000118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0c9dc0 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.170+0000 7f5eeb7fe640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0c9880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ee002a200 tx=0x7f5ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.171+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0041020 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.171+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee0005e10 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.171+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004030 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.171+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec14fd20 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.171+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec0ca370 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.172+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec073a00 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.172+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ee0044020 con 0x7f5eec15b980 2024-04-25T09:55:23.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.172+0000 7f5ee8ff9640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ed806b4d0 0x7f5ed806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.173+0000 7f5eebfff640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ed806b4d0 0x7f5ed806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.173+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6743+0+0 (secure 0 0 0) 0x7f5ee00aaff0 con 0x7f5eec15b980 2024-04-25T09:55:23.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.173+0000 7f5eebfff640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ed806b4d0 0x7f5ed806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009990 tx=0x7f5edc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:23.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.177+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ee004b030 con 0x7f5eec15b980 2024-04-25T09:55:23.312 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:23.315 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:23.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.384+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5eec0d1d20 con 0x7f5eec15b980 2024-04-25T09:55:23.387 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:55:23.385+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:55:23.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.386+0000 7f5ee8ff9640 1 -- 172.21.15.148:0/4078000118 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5ee00453b0 con 0x7f5eec15b980 2024-04-25T09:55:23.389 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ed806b4d0 msgr2=0x7f5ed806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ed806b4d0 0x7f5ed806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009990 tx=0x7f5edc008040 comp rx=0 tx=0).stop 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 msgr2=0x7f5eec0c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0c9880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ee002a200 tx=0x7f5ee00046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 shutdown_connections 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ed806b4d0 0x7f5ed806d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eec162450 0x7f5eec14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec15b980 0x7f5eec0c9880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 --2- 172.21.15.148:0/4078000118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eec15b5a0 0x7f5eec1f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 >> 172.21.15.148:0/4078000118 conn(0x7f5eec0d0530 msgr2=0x7f5eec0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:23.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 shutdown_connections 2024-04-25T09:55:23.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.389+0000 7f5ef275c640 1 -- 172.21.15.148:0/4078000118 wait complete. 2024-04-25T09:55:23.398 INFO:tasks.ceph.mds.d.smithi188.stderr: -4955> 2024-04-25T09:54:27.600+0000 7f6ac0491640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:55:23.403 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:55:23.428 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:55:23.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.540+0000 7f69aa08f640 1 Processor -- start 2024-04-25T09:55:23.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.540+0000 7f69aa08f640 1 -- start start 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69aa08f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69a40cf250 0x7f69a40cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69aa08f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69a415c990 0x7f69a40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69aa08f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69a406fe50 con 0x7f69a415ba80 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69aa08f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a4070130 con 0x7f69a40cf250 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69aa08f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a40cc8b0 con 0x7f69a415c990 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69a40cf250 0x7f69a40cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f69a415ba80 0x7f69a40ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42590/0 (socket says 172.21.15.148:42590) 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f69a40cf250 0x7f69a40cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47420/0 (socket says 172.21.15.148:47420) 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a37fe640 1 -- 172.21.15.148:0/958379802 learned_addr learned my addr 172.21.15.148:0/958379802 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a3fff640 1 --2- 172.21.15.148:0/958379802 >> v2:172.21.15.188:3300/0 conn(0x7f69a415c990 0x7f69a40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a2ffd640 1 -- 172.21.15.148:0/958379802 >> v2:172.21.15.188:3300/0 conn(0x7f69a415c990 msgr2=0x7f69a40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a2ffd640 1 --2- 172.21.15.148:0/958379802 >> v2:172.21.15.188:3300/0 conn(0x7f69a415c990 0x7f69a40b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a2ffd640 1 -- 172.21.15.148:0/958379802 >> v1:172.21.15.148:6789/0 conn(0x7f69a415ba80 legacy=0x7f69a40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.541+0000 7f69a2ffd640 1 -- 172.21.15.148:0/958379802 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a40ccb90 con 0x7f69a40cf250 2024-04-25T09:55:23.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.542+0000 7f69a2ffd640 1 --2- 172.21.15.148:0/958379802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a40cf250 0x7f69a40cd390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f69a4072e50 tx=0x7f699802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b0c8a7f1c84f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:23.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.542+0000 7f69a27fc640 1 -- 172.21.15.148:0/958379802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998037070 con 0x7f69a40cf250 2024-04-25T09:55:23.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.542+0000 7f69a27fc640 1 -- 172.21.15.148:0/958379802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6998032440 con 0x7f69a40cf250 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.542+0000 7f69a27fc640 1 -- 172.21.15.148:0/958379802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998032810 con 0x7f69a40cf250 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 -- 172.21.15.148:0/958379802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a40cf250 msgr2=0x7f69a40cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 --2- 172.21.15.148:0/958379802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a40cf250 0x7f69a40cd390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f69a4072e50 tx=0x7f699802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 -- 172.21.15.148:0/958379802 shutdown_connections 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 --2- 172.21.15.148:0/958379802 >> v2:172.21.15.188:3300/0 conn(0x7f69a415c990 0x7f69a40b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 --2- 172.21.15.148:0/958379802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a40cf250 0x7f69a40cd390 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 -- 172.21.15.148:0/958379802 >> 172.21.15.148:0/958379802 conn(0x7f69a406abe0 msgr2=0x7f69a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 -- 172.21.15.148:0/958379802 shutdown_connections 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.543+0000 7f69aa08f640 1 -- 172.21.15.148:0/958379802 wait complete. 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.544+0000 7f69aa08f640 1 Processor -- start 2024-04-25T09:55:23.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.544+0000 7f69aa08f640 1 -- start start 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a40cf250 0x7f69a4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 0x7f69a4145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a415c990 0x7f69a40b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a406fb00 con 0x7f69a415ba80 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a415ce00 con 0x7f69a40cf250 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a406b850 con 0x7f69a415c990 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 0x7f69a4145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 0x7f69a4145450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47422/0 (socket says 172.21.15.148:47422) 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 -- 172.21.15.148:0/1115336288 learned_addr learned my addr 172.21.15.148:0/1115336288 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a37fe640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a40cf250 0x7f69a4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a3fff640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a415c990 0x7f69a40b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 -- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a415c990 msgr2=0x7f69a40b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a415c990 0x7f69a40b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 -- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a40cf250 msgr2=0x7f69a4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a40cf250 0x7f69a4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 -- 172.21.15.148:0/1115336288 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a4150fd0 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a3fff640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a415c990 0x7f69a40b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a37fe640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a40cf250 0x7f69a4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69a2ffd640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 0x7f69a4145450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6998000c00 tx=0x7f69980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998037040 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6998042070 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a41512b0 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.545+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998030bd0 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.546+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a4151870 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.547+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a40cd8e0 con 0x7f69a415ba80 2024-04-25T09:55:23.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.547+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6998047460 con 0x7f69a415ba80 2024-04-25T09:55:23.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.547+0000 7f698bfff640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f699006b500 0x7f699006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:23.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.547+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6829+0+0 (secure 0 0 0) 0x7f69980ab160 con 0x7f69a415ba80 2024-04-25T09:55:23.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.547+0000 7f69a37fe640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f699006b500 0x7f699006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:23.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.548+0000 7f69a37fe640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f699006b500 0x7f699006d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6994000f60 tx=0x7f6994009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:23.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.551+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6998032e20 con 0x7f69a415ba80 2024-04-25T09:55:23.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:23.761+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f69a40739e0 con 0x7f69a415ba80 2024-04-25T09:55:24.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.388+0000 7f698bfff640 1 -- 172.21.15.148:0/1115336288 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6998079c10 con 0x7f69a415ba80 2024-04-25T09:55:24.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f699006b500 msgr2=0x7f699006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:24.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f699006b500 0x7f699006d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6994000f60 tx=0x7f6994009040 comp rx=0 tx=0).stop 2024-04-25T09:55:24.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 msgr2=0x7f69a4145450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 0x7f69a4145450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6998000c00 tx=0x7f69980046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 shutdown_connections 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f699006b500 0x7f699006d9c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a415c990 0x7f69a40b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a415ba80 0x7f69a4145450 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 --2- 172.21.15.148:0/1115336288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a40cf250 0x7f69a4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.390+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 >> 172.21.15.148:0/1115336288 conn(0x7f69a406abe0 msgr2=0x7f69a40aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.391+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 shutdown_connections 2024-04-25T09:55:24.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.391+0000 7f69aa08f640 1 -- 172.21.15.148:0/1115336288 wait complete. 2024-04-25T09:55:24.406 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:55:24.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 Processor -- start 2024-04-25T09:55:24.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 -- start start 2024-04-25T09:55:24.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb940cfd30 0x7fdb9415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:24.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdb9409a090 0x7fdb9415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:24.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb940cc5e0 con 0x7fdb94099cb0 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb940cc3b0 con 0x7fdb9409a090 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.507+0000 7fdb99e9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940b2ee0 con 0x7fdb940cfd30 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb93fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdb9409a090 0x7fdb9415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb937fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdb94099cb0 0x7fdb940b27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42594/0 (socket says 172.21.15.148:42594) 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb937fe640 1 -- 172.21.15.148:0/391102286 learned_addr learned my addr 172.21.15.148:0/391102286 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb92ffd640 1 --2- 172.21.15.148:0/391102286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb940cfd30 0x7fdb9415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb93fff640 1 -- 172.21.15.148:0/391102286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb940cfd30 msgr2=0x7fdb9415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb93fff640 1 --2- 172.21.15.148:0/391102286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb940cfd30 0x7fdb9415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb93fff640 1 -- 172.21.15.148:0/391102286 >> v1:172.21.15.148:6789/0 conn(0x7fdb94099cb0 legacy=0x7fdb940b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb93fff640 1 -- 172.21.15.148:0/391102286 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940d0210 con 0x7fdb9409a090 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.508+0000 7fdb927fc640 1 -- 172.21.15.148:0/391102286 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918939168 0 0) 0x7fdb940cc5e0 con 0x7fdb94099cb0 2024-04-25T09:55:24.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb93fff640 1 --2- 172.21.15.148:0/391102286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb9409a090 0x7fdb9415f200 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009d90 tx=0x7fdb8c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8ab67f997eb032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:24.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb927fc640 1 -- 172.21.15.148:0/391102286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c031600 con 0x7fdb9409a090 2024-04-25T09:55:24.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb927fc640 1 -- 172.21.15.148:0/391102286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0317a0 con 0x7fdb9409a090 2024-04-25T09:55:24.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb927fc640 1 -- 172.21.15.148:0/391102286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c031b70 con 0x7fdb9409a090 2024-04-25T09:55:24.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 -- 172.21.15.148:0/391102286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb9409a090 msgr2=0x7fdb9415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/391102286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb9409a090 0x7fdb9415f200 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009d90 tx=0x7fdb8c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 -- 172.21.15.148:0/391102286 shutdown_connections 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/391102286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb9409a090 0x7fdb9415f200 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/391102286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb940cfd30 0x7fdb9415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 -- 172.21.15.148:0/391102286 >> 172.21.15.148:0/391102286 conn(0x7fdb940b3290 msgr2=0x7fdb940b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 -- 172.21.15.148:0/391102286 shutdown_connections 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.509+0000 7fdb99e9e640 1 -- 172.21.15.148:0/391102286 wait complete. 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.510+0000 7fdb99e9e640 1 Processor -- start 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 -- start start 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb94099cb0 0x7fdb9406cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb9409a090 0x7fdb941f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:24.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 0x7fdb940b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940d07d0 con 0x7fdb940cfd30 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940cc880 con 0x7fdb9409a090 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.511+0000 7fdb99e9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c52f0 con 0x7fdb94099cb0 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 0x7fdb940b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 0x7fdb940b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47440/0 (socket says 172.21.15.148:47440) 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 -- 172.21.15.148:0/527786033 learned_addr learned my addr 172.21.15.148:0/527786033 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb937fe640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb94099cb0 0x7fdb9406cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb92ffd640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb9409a090 0x7fdb941f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 -- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb94099cb0 msgr2=0x7fdb9406cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb94099cb0 0x7fdb9406cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 -- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb9409a090 msgr2=0x7fdb941f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb9409a090 0x7fdb941f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:24.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 -- 172.21.15.148:0/527786033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940b5550 con 0x7fdb940cfd30 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb93fff640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 0x7fdb940b4db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009d90 tx=0x7fdb8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c03f070 con 0x7fdb940cfd30 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c030be0 con 0x7fdb940cfd30 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9415c770 con 0x7fdb940cfd30 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c009450 con 0x7fdb940cfd30 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.512+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9415cd30 con 0x7fdb940cfd30 2024-04-25T09:55:24.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.513+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb94073a00 con 0x7fdb940cfd30 2024-04-25T09:55:24.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.514+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb8c003990 con 0x7fdb940cfd30 2024-04-25T09:55:24.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.514+0000 7fdb7bfff640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdb8006b4d0 0x7fdb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:24.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.514+0000 7fdb937fe640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdb8006b4d0 0x7fdb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:24.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.514+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6829+0+0 (secure 0 0 0) 0x7fdb8c0ab0d0 con 0x7fdb940cfd30 2024-04-25T09:55:24.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.515+0000 7fdb937fe640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdb8006b4d0 0x7fdb8006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb84007fd0 tx=0x7fdb84007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:24.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.519+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb8c079530 con 0x7fdb940cfd30 2024-04-25T09:55:24.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:24.690+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdb940cbad0 con 0x7fdb940cfd30 2024-04-25T09:55:25.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.397+0000 7fdb7bfff640 1 -- 172.21.15.148:0/527786033 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdb8c07d5d0 con 0x7fdb940cfd30 2024-04-25T09:55:25.400 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:55:25.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdb8006b4d0 msgr2=0x7fdb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdb8006b4d0 0x7fdb8006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdb84007fd0 tx=0x7fdb84007de0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 msgr2=0x7fdb940b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 0x7fdb940b4db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009d90 tx=0x7fdb8c004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 shutdown_connections 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdb8006b4d0 0x7fdb8006d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb940cfd30 0x7fdb940b4db0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb9409a090 0x7fdb941f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.400+0000 7fdb99e9e640 1 --2- 172.21.15.148:0/527786033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb94099cb0 0x7fdb9406cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.401+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 >> 172.21.15.148:0/527786033 conn(0x7fdb940b3290 msgr2=0x7fdb940c5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.401+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 shutdown_connections 2024-04-25T09:55:25.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.401+0000 7fdb99e9e640 1 -- 172.21.15.148:0/527786033 wait complete. 2024-04-25T09:55:25.416 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:55:25.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.517+0000 7fc96997e640 1 Processor -- start 2024-04-25T09:55:25.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.517+0000 7fc96997e640 1 -- start start 2024-04-25T09:55:25.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.517+0000 7fc96997e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9640ced80 0x7fc9640d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.517+0000 7fc96997e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96415e000 con 0x7fc964161710 2024-04-25T09:55:25.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.517+0000 7fc96997e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9640d1de0 con 0x7fc96415dc20 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.517+0000 7fc96997e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc964161c60 con 0x7fc9640ced80 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9637fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc964161710 0x7fc9640d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42598/0 (socket says 172.21.15.148:42598) 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9637fe640 1 -- 172.21.15.148:0/1875724827 learned_addr learned my addr 172.21.15.148:0/1875724827 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc95a7fc640 1 --2- 172.21.15.148:0/1875724827 >> v2:172.21.15.188:3300/0 conn(0x7fc9640ced80 0x7fc9640d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320540743 0 0) 0x7fc96415e000 con 0x7fc964161710 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc950003520 con 0x7fc964161710 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770084111 0 0) 0x7fc9640d1de0 con 0x7fc96415dc20 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc96415e000 con 0x7fc96415dc20 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2193278693 0 0) 0x7fc950003520 con 0x7fc964161710 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 >> v2:172.21.15.188:3300/0 conn(0x7fc9640ced80 msgr2=0x7fc9640d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 --2- 172.21.15.148:0/1875724827 >> v2:172.21.15.188:3300/0 conn(0x7fc9640ced80 0x7fc9640d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 >> v1:172.21.15.188:6789/0 conn(0x7fc96415dc20 legacy=0x7fc9640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964161f40 con 0x7fc964161710 2024-04-25T09:55:25.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc95c0019f0 con 0x7fc964161710 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 >> v1:172.21.15.148:6789/0 conn(0x7fc964161710 legacy=0x7fc9640d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc950003b90 0x7fc950003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950004800 0x7fc950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9500051c0 0x7fc95000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964161c60 con 0x7fc950003b90 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000c090 con 0x7fc950004800 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95000c290 con 0x7fc9500051c0 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc962ffd640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc950003b90 0x7fc950003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc95a7fc640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950004800 0x7fc950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.518+0000 7fc9637fe640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9500051c0 0x7fc95000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc962ffd640 1 -- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9500051c0 msgr2=0x7fc95000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc962ffd640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9500051c0 0x7fc95000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc962ffd640 1 -- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950004800 msgr2=0x7fc950004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc962ffd640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950004800 0x7fc950004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc962ffd640 1 -- 172.21.15.148:0/1875724827 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950003520 con 0x7fc950003b90 2024-04-25T09:55:25.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc9637fe640 1 -- 172.21.15.148:0/1875724827 reap_dead start 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc962ffd640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc950003b90 0x7fc950003f90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9540018d0 tx=0x7fc950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c247e0dcf3534d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540014f0 con 0x7fc950003b90 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9540052d0 con 0x7fc950003b90 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.519+0000 7fc9627fc640 1 -- 172.21.15.148:0/1875724827 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95402ac40 con 0x7fc950003b90 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 -- 172.21.15.148:0/1875724827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc950003b90 msgr2=0x7fc950003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc950003b90 0x7fc950003f90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9540018d0 tx=0x7fc950005830 comp rx=0 tx=0).stop 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 -- 172.21.15.148:0/1875724827 shutdown_connections 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 --2- 172.21.15.148:0/1875724827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc950003b90 0x7fc950003f90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 -- 172.21.15.148:0/1875724827 >> 172.21.15.148:0/1875724827 conn(0x7fc9640d0420 msgr2=0x7fc9640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 -- 172.21.15.148:0/1875724827 shutdown_connections 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.520+0000 7fc96997e640 1 -- 172.21.15.148:0/1875724827 wait complete. 2024-04-25T09:55:25.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.521+0000 7fc96997e640 1 Processor -- start 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.521+0000 7fc96997e640 1 -- start start 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc96997e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950003b90 0x7fc9640cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc96997e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9640ccb80 0x7fc9640d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc96997e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 0x7fc9641f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc96997e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640b5e50 con 0x7fc9640d3ec0 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc96997e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964162500 con 0x7fc950003b90 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc96997e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640b5fd0 con 0x7fc9640ccb80 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 0x7fc9641f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 0x7fc9641f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47460/0 (socket says 172.21.15.148:47460) 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 -- 172.21.15.148:0/4220494090 learned_addr learned my addr 172.21.15.148:0/4220494090 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc95a7fc640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9640ccb80 0x7fc9640d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc962ffd640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950003b90 0x7fc9640cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 -- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9640ccb80 msgr2=0x7fc9640d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9640ccb80 0x7fc9640d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 -- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950003b90 msgr2=0x7fc9640cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950003b90 0x7fc9640cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 -- 172.21.15.148:0/4220494090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640ab6a0 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc95a7fc640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9640ccb80 0x7fc9640d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.522+0000 7fc9637fe640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 0x7fc9641f36d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9640b3290 tx=0x7fc95c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.523+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00aa10 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.523+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c019070 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.523+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9640ab980 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.523+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00ae30 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.523+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9640cbe40 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.524+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc964073a00 con 0x7fc9640d3ec0 2024-04-25T09:55:25.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.524+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc95c01e460 con 0x7fc9640d3ec0 2024-04-25T09:55:25.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.524+0000 7fc960ff9640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94406b4d0 0x7fc94406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:25.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.525+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fc95c08a8a0 con 0x7fc9640d3ec0 2024-04-25T09:55:25.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.525+0000 7fc962ffd640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94406b4d0 0x7fc94406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:25.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.525+0000 7fc962ffd640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94406b4d0 0x7fc94406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc954032000 tx=0x7fc954009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:25.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.529+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc95c059470 con 0x7fc9640d3ec0 2024-04-25T09:55:25.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:25.700+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc9640aa630 con 0x7fc9640d3ec0 2024-04-25T09:55:26.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.402+0000 7fc960ff9640 1 -- 172.21.15.148:0/4220494090 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc95c058d70 con 0x7fc9640d3ec0 2024-04-25T09:55:26.405 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T09:55:26.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94406b4d0 msgr2=0x7fc94406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:26.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94406b4d0 0x7fc94406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc954032000 tx=0x7fc954009030 comp rx=0 tx=0).stop 2024-04-25T09:55:26.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 msgr2=0x7fc9641f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:26.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 0x7fc9641f36d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9640b3290 tx=0x7fc95c007bd0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 shutdown_connections 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94406b4d0 0x7fc94406d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640d3ec0 0x7fc9641f36d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9640ccb80 0x7fc9640d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 --2- 172.21.15.148:0/4220494090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc950003b90 0x7fc9640cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 >> 172.21.15.148:0/4220494090 conn(0x7fc9640d0420 msgr2=0x7fc9640b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 shutdown_connections 2024-04-25T09:55:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.404+0000 7fc96997e640 1 -- 172.21.15.148:0/4220494090 wait complete. 2024-04-25T09:55:26.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T09:55:26.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.520+0000 7f3803709640 1 Processor -- start 2024-04-25T09:55:26.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.520+0000 7f3803709640 1 -- start start 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3803709640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37fc15eec0 0x7f37fc1689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3803709640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc0d1770 con 0x7f37fc06ccd0 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3803709640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc162740 con 0x7f37fc070dc0 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3803709640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc0b0eb0 con 0x7f37fc15eec0 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3801ca6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37fc15eec0 0x7f37fc1689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3800ca4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f37fc06ccd0 0x7f37fc0b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42600/0 (socket says 172.21.15.148:42600) 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3801ca6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f37fc15eec0 0x7f37fc1689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47476/0 (socket says 172.21.15.148:47476) 2024-04-25T09:55:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3800ca4640 1 -- 172.21.15.148:0/4034819671 learned_addr learned my addr 172.21.15.148:0/4034819671 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3801ca6640 1 -- 172.21.15.148:0/4034819671 >> v1:172.21.15.188:6789/0 conn(0x7f37fc070dc0 legacy=0x7f37fc0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3801ca6640 1 -- 172.21.15.148:0/4034819671 >> v1:172.21.15.148:6789/0 conn(0x7f37fc06ccd0 legacy=0x7f37fc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f3801ca6640 1 -- 172.21.15.148:0/4034819671 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc161980 con 0x7f37fc15eec0 2024-04-25T09:55:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.521+0000 7f37f3fff640 1 -- 172.21.15.148:0/4034819671 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381296442 0 0) 0x7f37fc0d1770 con 0x7f37fc06ccd0 2024-04-25T09:55:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.522+0000 7f3801ca6640 1 --2- 172.21.15.148:0/4034819671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc15eec0 0x7f37fc1689d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009d90 tx=0x7f37f402a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9e1f6f45ce446f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.522+0000 7f37f3fff640 1 -- 172.21.15.148:0/4034819671 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4007740 con 0x7f37fc15eec0 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.522+0000 7f37f3fff640 1 -- 172.21.15.148:0/4034819671 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f40078e0 con 0x7f37fc15eec0 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.522+0000 7f37f3fff640 1 -- 172.21.15.148:0/4034819671 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4007cb0 con 0x7f37fc15eec0 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 -- 172.21.15.148:0/4034819671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc15eec0 msgr2=0x7f37fc1689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 --2- 172.21.15.148:0/4034819671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc15eec0 0x7f37fc1689d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009d90 tx=0x7f37f402a580 comp rx=0 tx=0).stop 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 -- 172.21.15.148:0/4034819671 shutdown_connections 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 --2- 172.21.15.148:0/4034819671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc15eec0 0x7f37fc1689d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 -- 172.21.15.148:0/4034819671 >> 172.21.15.148:0/4034819671 conn(0x7f37fc0b3530 msgr2=0x7f37fc0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 -- 172.21.15.148:0/4034819671 shutdown_connections 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.523+0000 7f3803709640 1 -- 172.21.15.148:0/4034819671 wait complete. 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.524+0000 7f3803709640 1 Processor -- start 2024-04-25T09:55:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.524+0000 7f3803709640 1 -- start start 2024-04-25T09:55:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37fc06ccd0 0x7f37fc1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 0x7f37fc0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37fc15eec0 0x7f37fc0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0b10f0 con 0x7f37fc070dc0 2024-04-25T09:55:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc1622b0 con 0x7f37fc06ccd0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc161f40 con 0x7f37fc15eec0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 0x7f37fc0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 0x7f37fc0cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47490/0 (socket says 172.21.15.148:47490) 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 -- 172.21.15.148:0/3135262536 learned_addr learned my addr 172.21.15.148:0/3135262536 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 -- 172.21.15.148:0/3135262536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37fc15eec0 msgr2=0x7f37fc0aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37fc15eec0 0x7f37fc0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 -- 172.21.15.148:0/3135262536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37fc06ccd0 msgr2=0x7f37fc1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37fc06ccd0 0x7f37fc1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 -- 172.21.15.148:0/3135262536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc152810 con 0x7f37fc070dc0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3800ca4640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 0x7f37fc0cdf30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c4d0 tx=0x7f37ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec018070 con 0x7f37fc070dc0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec007500 con 0x7f37fc070dc0 2024-04-25T09:55:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc0cbf70 con 0x7f37fc070dc0 2024-04-25T09:55:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.525+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec004a20 con 0x7f37fc070dc0 2024-04-25T09:55:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.526+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc0cc530 con 0x7f37fc070dc0 2024-04-25T09:55:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.527+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc1f3b70 con 0x7f37fc070dc0 2024-04-25T09:55:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.527+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37ec01b070 con 0x7f37fc070dc0 2024-04-25T09:55:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.527+0000 7f37f1ffb640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37e006b4d0 0x7f37e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:26.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.527+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f37ec089540 con 0x7f37fc070dc0 2024-04-25T09:55:26.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.527+0000 7f38014a5640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37e006b4d0 0x7f37e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:26.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.528+0000 7f38014a5640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37e006b4d0 0x7f37e006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f37f8003cc0 tx=0x7f37f80079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:26.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.532+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37fc1f3b70 con 0x7f37fc070dc0 2024-04-25T09:55:26.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:26.702+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37fc073a00 con 0x7f37fc070dc0 2024-04-25T09:55:27.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.406+0000 7f37f1ffb640 1 -- 172.21.15.148:0/3135262536 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f37ec058330 con 0x7f37fc070dc0 2024-04-25T09:55:27.410 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data_ec' removed 2024-04-25T09:55:27.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.409+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37e006b4d0 msgr2=0x7f37e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.409+0000 7f3803709640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37e006b4d0 0x7f37e006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f37f8003cc0 tx=0x7f37f80079b0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.409+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 msgr2=0x7f37fc0cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.409+0000 7f3803709640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 0x7f37fc0cdf30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00c4d0 tx=0x7f37ec00c990 comp rx=0 tx=0).stop 2024-04-25T09:55:27.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.410+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 shutdown_connections 2024-04-25T09:55:27.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.410+0000 7f3803709640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f37e006b4d0 0x7f37e006d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.410+0000 7f3803709640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37fc15eec0 0x7f37fc0aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.410+0000 7f3803709640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37fc070dc0 0x7f37fc0cdf30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.410+0000 7f3803709640 1 --2- 172.21.15.148:0/3135262536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37fc06ccd0 0x7f37fc1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.410+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 >> 172.21.15.148:0/3135262536 conn(0x7f37fc0b3530 msgr2=0x7f37fc0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:27.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.411+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 shutdown_connections 2024-04-25T09:55:27.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.411+0000 7f3803709640 1 -- 172.21.15.148:0/3135262536 wait complete. 2024-04-25T09:55:27.431 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:55:27.431 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:55:27.432 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:55:27.432 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:27.432 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:55:27.432 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:55:27.432 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:55:27.432 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:27.433 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:55:27.433 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:55:27.433 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:55:27.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:27.433 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:55:27.433 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:55:27.433 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:55:27.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:55:27.443 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:55:27.443 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:55:27.444 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:55:27.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:55:27.445 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:55:27.445 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:55:27.455 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:55:27.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:55:27.486 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:55:27.486 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:27.487 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:55:27.487 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:27.497 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:55:27.497 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:27.498 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:55:27.498 DEBUG:teuthology.parallel:result is None 2024-04-25T09:55:27.499 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:55:27.562 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:55:27.562 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:55:27.577 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:55:27.577 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T09:55:27.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.602+0000 7f833f04d640 1 Processor -- start 2024-04-25T09:55:27.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.602+0000 7f833f04d640 1 -- start start 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833f04d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833f04d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83380d2420 con 0x7f83380d0ab0 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833f04d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83380d2700 con 0x7f83380d1f30 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833f04d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83380d3030 con 0x7f833815dac0 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833d5ea640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f83380d0ab0 0x7f83380b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42628/0 (socket says 172.21.15.148:42628) 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833d5ea640 1 -- 172.21.15.148:0/777929641 learned_addr learned my addr 172.21.15.148:0/777929641 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.603+0000 7f833cde9640 1 --2- 172.21.15.148:0/777929641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929263512 0 0) 0x7f83380d2420 con 0x7f83380d0ab0 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8328003520 con 0x7f83380d0ab0 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471148870 0 0) 0x7f83380d2700 con 0x7f83380d1f30 2024-04-25T09:55:27.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83380d2420 con 0x7f83380d1f30 2024-04-25T09:55:27.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f833cde9640 1 -- 172.21.15.148:0/777929641 >> v1:172.21.15.188:6789/0 conn(0x7f83380d1f30 legacy=0x7f83380d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f833cde9640 1 -- 172.21.15.148:0/777929641 >> v1:172.21.15.148:6789/0 conn(0x7f83380d0ab0 legacy=0x7f83380b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f833cde9640 1 -- 172.21.15.148:0/777929641 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380d29e0 con 0x7f833815dac0 2024-04-25T09:55:27.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.604+0000 7f833cde9640 1 --2- 172.21.15.148:0/777929641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380d19f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83380d31b0 tx=0x7f832c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1464cff57b9436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:27.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c037070 con 0x7f833815dac0 2024-04-25T09:55:27.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c032440 con 0x7f833815dac0 2024-04-25T09:55:27.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f83377fe640 1 -- 172.21.15.148:0/777929641 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0327f0 con 0x7f833815dac0 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 -- 172.21.15.148:0/777929641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 msgr2=0x7f83380d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 --2- 172.21.15.148:0/777929641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380d19f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83380d31b0 tx=0x7f832c02a290 comp rx=0 tx=0).stop 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 -- 172.21.15.148:0/777929641 shutdown_connections 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 --2- 172.21.15.148:0/777929641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380d19f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 -- 172.21.15.148:0/777929641 >> 172.21.15.148:0/777929641 conn(0x7f8338061c30 msgr2=0x7f833806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 -- 172.21.15.148:0/777929641 shutdown_connections 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.605+0000 7f833f04d640 1 -- 172.21.15.148:0/777929641 wait complete. 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.606+0000 7f833f04d640 1 Processor -- start 2024-04-25T09:55:27.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.606+0000 7f833f04d640 1 -- start start 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833f04d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 0x7f8338156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833f04d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83380d1f30 0x7f8338152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833f04d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833f04d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83380b97d0 con 0x7f83380d0ab0 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833f04d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833815dea0 con 0x7f83380d1f30 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833f04d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83380b9950 con 0x7f833815dac0 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833cde9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 0x7f8338156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833cde9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 0x7f8338156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47534/0 (socket says 172.21.15.148:47534) 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833cde9640 1 -- 172.21.15.148:0/499125960 learned_addr learned my addr 172.21.15.148:0/499125960 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f8337fff640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83380d1f30 0x7f8338152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.607+0000 7f833d5ea640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:27.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833cde9640 1 -- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 msgr2=0x7f83380ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833cde9640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833cde9640 1 -- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83380d1f30 msgr2=0x7f8338152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833cde9640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83380d1f30 0x7f8338152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833cde9640 1 -- 172.21.15.148:0/499125960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380b49c0 con 0x7f83380d0ab0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f8337fff640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83380d1f30 0x7f8338152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833d5ea640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833cde9640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 0x7f8338156db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f832c0098c0 tx=0x7f832c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c037050 con 0x7f83380d0ab0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c043070 con 0x7f83380d0ab0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83380b4ca0 con 0x7f83380d0ab0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c03ca50 con 0x7f83380d0ab0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.608+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83380b5260 con 0x7f83380d0ab0 2024-04-25T09:55:27.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.609+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338073a00 con 0x7f83380d0ab0 2024-04-25T09:55:27.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.609+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f832c03cbf0 con 0x7f83380d0ab0 2024-04-25T09:55:27.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.610+0000 7f83357fa640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f831c06b4d0 0x7f831c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:27.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.610+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f832c0aad10 con 0x7f83380d0ab0 2024-04-25T09:55:27.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.610+0000 7f8337fff640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f831c06b4d0 0x7f831c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:27.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.610+0000 7f8337fff640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f831c06b4d0 0x7f831c06d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83381f6d10 tx=0x7f8324009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:27.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.614+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f832c046360 con 0x7f83380d0ab0 2024-04-25T09:55:27.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:27.797+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f833815ac00 con 0x7f83380d0ab0 2024-04-25T09:55:28.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.421+0000 7f83357fa640 1 -- 172.21.15.148:0/499125960 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f832c079e30 con 0x7f83380d0ab0 2024-04-25T09:55:28.425 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T09:55:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f831c06b4d0 msgr2=0x7f831c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f831c06b4d0 0x7f831c06d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83381f6d10 tx=0x7f8324009040 comp rx=0 tx=0).stop 2024-04-25T09:55:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 msgr2=0x7f8338156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 0x7f8338156db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f832c0098c0 tx=0x7f832c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 shutdown_connections 2024-04-25T09:55:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f831c06b4d0 0x7f831c06d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f833815dac0 0x7f83380ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83380d1f30 0x7f8338152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 --2- 172.21.15.148:0/499125960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83380d0ab0 0x7f8338156db0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 >> 172.21.15.148:0/499125960 conn(0x7f8338061c30 msgr2=0x7f83380b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:28.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 shutdown_connections 2024-04-25T09:55:28.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.424+0000 7f833f04d640 1 -- 172.21.15.148:0/499125960 wait complete. 2024-04-25T09:55:28.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:55:28.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.535+0000 7fbeae6fc640 1 Processor -- start 2024-04-25T09:55:28.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.535+0000 7fbeae6fc640 1 -- start start 2024-04-25T09:55:28.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.536+0000 7fbeae6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 0x7fbea80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.536+0000 7fbeae6fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea80b6220 con 0x7fbea80b2cb0 2024-04-25T09:55:28.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.536+0000 7fbeae6fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea80b97a0 con 0x7fbea80b92a0 2024-04-25T09:55:28.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.536+0000 7fbeae6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea815dfc0 con 0x7fbea815dbe0 2024-04-25T09:55:28.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.536+0000 7fbea77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbea80b2cb0 0x7fbea80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42642/0 (socket says 172.21.15.148:42642) 2024-04-25T09:55:28.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.536+0000 7fbea77fe640 1 -- 172.21.15.148:0/1123690739 learned_addr learned my addr 172.21.15.148:0/1123690739 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea7fff640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 0x7fbea80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626159005 0 0) 0x7fbea80b6220 con 0x7fbea80b2cb0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe90003520 con 0x7fbea80b2cb0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429825015 0 0) 0x7fbea80b97a0 con 0x7fbea80b92a0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea80b6220 con 0x7fbea80b92a0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3119641093 0 0) 0x7fbe90003520 con 0x7fbea80b2cb0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 msgr2=0x7fbea80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 0x7fbea80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 >> v1:172.21.15.188:6789/0 conn(0x7fbea80b92a0 legacy=0x7fbea80b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80d1c50 con 0x7fbea80b2cb0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbe9c001670 con 0x7fbea80b2cb0 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 >> v1:172.21.15.148:6789/0 conn(0x7fbea80b2cb0 legacy=0x7fbea80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 0x7fbe90003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbe90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9000bad0 con 0x7fbe90004120 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9000bc80 con 0x7fbe90004d40 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9000be80 con 0x7fbea815dbe0 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea77fe640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbe90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea7fff640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 0x7fbe90003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbeacc99640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea77fe640 1 -- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 msgr2=0x7fbe90003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea77fe640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea815dbe0 0x7fbe90003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea77fe640 1 -- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90004d40 msgr2=0x7fbe900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea77fe640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbea77fe640 1 -- 172.21.15.148:0/1123690739 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90003520 con 0x7fbe90004120 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.537+0000 7fbeacc99640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90004d40 0x7fbe900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:28.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.538+0000 7fbea77fe640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbe90004800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c0e0 tx=0x7fbe90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae02bbc3b31ed76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.538+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007c00 con 0x7fbe90004120 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.538+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c005ad0 con 0x7fbe90004120 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.538+0000 7fbea6ffd640 1 -- 172.21.15.148:0/1123690739 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007070 con 0x7fbe90004120 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeae6fc640 1 -- 172.21.15.148:0/1123690739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 msgr2=0x7fbe90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/1123690739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbe90004800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c0e0 tx=0x7fbe90005830 comp rx=0 tx=0).stop 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeacc99640 1 -- 172.21.15.148:0/1123690739 reap_dead start 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeae6fc640 1 -- 172.21.15.148:0/1123690739 shutdown_connections 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeae6fc640 1 -- 172.21.15.148:0/1123690739 >> 172.21.15.148:0/1123690739 conn(0x7fbea8061a20 msgr2=0x7fbea8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeae6fc640 1 -- 172.21.15.148:0/1123690739 shutdown_connections 2024-04-25T09:55:28.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.539+0000 7fbeae6fc640 1 -- 172.21.15.148:0/1123690739 wait complete. 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.540+0000 7fbeae6fc640 1 Processor -- start 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.540+0000 7fbeae6fc640 1 -- start start 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeae6fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbea8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeae6fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbea8150770 0x7fbea80b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeae6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea80b5830 0x7fbea815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeae6fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80d2210 con 0x7fbe90004120 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeae6fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80d1a00 con 0x7fbea8150770 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeae6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80b6ab0 con 0x7fbea80b5830 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbea8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbea8154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47552/0 (socket says 172.21.15.148:47552) 2024-04-25T09:55:28.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 -- 172.21.15.148:0/668287211 learned_addr learned my addr 172.21.15.148:0/668287211 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbeacc99640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea80b5830 0x7fbea815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea77fe640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbea8150770 0x7fbea80b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 -- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea80b5830 msgr2=0x7fbea815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea80b5830 0x7fbea815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 -- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbea8150770 msgr2=0x7fbea80b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbea8150770 0x7fbea80b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 -- 172.21.15.148:0/668287211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8153040 con 0x7fbe90004120 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea7fff640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbea8154d00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe98011a00 tx=0x7fbe98011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.541+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800fb50 con 0x7fbe90004120 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.542+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9800fcf0 con 0x7fbe90004120 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.542+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea8155240 con 0x7fbe90004120 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.542+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800fea0 con 0x7fbe90004120 2024-04-25T09:55:28.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.542+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea81535c0 con 0x7fbe90004120 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.542+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea8073990 con 0x7fbe90004120 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.543+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe98012040 con 0x7fbe90004120 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.543+0000 7fbea4ff9640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe9406b4d0 0x7fbe9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.543+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe9808a620 con 0x7fbe90004120 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.543+0000 7fbea77fe640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe9406b4d0 0x7fbe9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.544+0000 7fbea77fe640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe9406b4d0 0x7fbe9406d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c032000 tx=0x7fbe9c02d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:28.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.547+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe98019030 con 0x7fbe90004120 2024-04-25T09:55:28.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.780+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbea815ae40 con 0x7fbe90004120 2024-04-25T09:55:28.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.781+0000 7fbea4ff9640 1 -- 172.21.15.148:0/668287211 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fbe980596f0 con 0x7fbe90004120 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.785 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T09:55:28.786 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.787 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.788 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T09:55:28.789 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T09:55:28.791 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.792 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.793 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:55:28.794 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.795 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mds", 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQA2KCpmaPvHNhAAb6fU7Rg+izC/YDzwQkIJCA==", 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQA2KCpmlAPINhAAvgiDOClghZhvODOJ09DkLA==", 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.796 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-osd", 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQA2KCpm8gnINhAAOaDySHMne3Kw/CFxNh3RWQ==", 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQA2KCpmLxDINhAA9RHQdKrZMyhwAKcbTlgacw==", 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQA2KCpmsBbINhAAYIk1W0BADy+rQMPveDIbng==", 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQA2KCpmsh3INhAAR4a8mk+5Fnp6Jck+6LuAXQ==", 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.798 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T09:55:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T09:55:28.800 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:55:28.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe9406b4d0 msgr2=0x7fbe9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe9406b4d0 0x7fbe9406d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c032000 tx=0x7fbe9c02d000 comp rx=0 tx=0).stop 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 msgr2=0x7fbea8154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbea8154d00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe98011a00 tx=0x7fbe98011ec0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 shutdown_connections 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe9406b4d0 0x7fbe9406d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbea80b5830 0x7fbea815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbea8150770 0x7fbea80b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 --2- 172.21.15.148:0/668287211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe90004120 0x7fbea8154d00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 >> 172.21.15.148:0/668287211 conn(0x7fbea8061a20 msgr2=0x7fbea815b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 shutdown_connections 2024-04-25T09:55:28.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.783+0000 7fbeae6fc640 1 -- 172.21.15.148:0/668287211 wait complete. 2024-04-25T09:55:28.803 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T09:55:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.897+0000 7fc6b254c640 1 Processor -- start 2024-04-25T09:55:28.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.897+0000 7fc6b254c640 1 -- start start 2024-04-25T09:55:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b254c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc6ac15dc20 0x7fc6ac0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b254c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac161710 0x7fc6ac0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b254c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ac15e000 con 0x7fc6ac0b9330 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b254c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ac0d1de0 con 0x7fc6ac15dc20 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b254c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac161c60 con 0x7fc6ac161710 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6abfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc6ac15dc20 0x7fc6ac0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b0ae9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc6ac0b9330 0x7fc6ac06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42654/0 (socket says 172.21.15.148:42654) 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.898+0000 7fc6b0ae9640 1 -- 172.21.15.148:0/3788542092 learned_addr learned my addr 172.21.15.148:0/3788542092 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6abfff640 1 -- 172.21.15.148:0/3788542092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac161710 msgr2=0x7fc6ac0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6abfff640 1 --2- 172.21.15.148:0/3788542092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac161710 0x7fc6ac0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6abfff640 1 -- 172.21.15.148:0/3788542092 >> v1:172.21.15.148:6789/0 conn(0x7fc6ac0b9330 legacy=0x7fc6ac06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6aaffd640 1 -- 172.21.15.148:0/3788542092 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378984392 0 0) 0x7fc6ac15e000 con 0x7fc6ac0b9330 2024-04-25T09:55:28.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6abfff640 1 -- 172.21.15.148:0/3788542092 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac161f40 con 0x7fc6ac15dc20 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6abfff640 1 --2- 172.21.15.148:0/3788542092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ce670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0097a0 tx=0x7fc69c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05bfedc8e0649ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.899+0000 7fc6aaffd640 1 -- 172.21.15.148:0/3788542092 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c037070 con 0x7fc6ac15dc20 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6aaffd640 1 -- 172.21.15.148:0/3788542092 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c032440 con 0x7fc6ac15dc20 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6aaffd640 1 -- 172.21.15.148:0/3788542092 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c032810 con 0x7fc6ac15dc20 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 -- 172.21.15.148:0/3788542092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac15dc20 msgr2=0x7fc6ac0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3788542092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ce670 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0097a0 tx=0x7fc69c02a180 comp rx=0 tx=0).stop 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 -- 172.21.15.148:0/3788542092 shutdown_connections 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3788542092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac161710 0x7fc6ac0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3788542092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ce670 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 -- 172.21.15.148:0/3788542092 >> 172.21.15.148:0/3788542092 conn(0x7fc6ac0d0420 msgr2=0x7fc6ac0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 -- 172.21.15.148:0/3788542092 shutdown_connections 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.900+0000 7fc6b254c640 1 -- 172.21.15.148:0/3788542092 wait complete. 2024-04-25T09:55:28.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.901+0000 7fc6b254c640 1 Processor -- start 2024-04-25T09:55:28.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 -- start start 2024-04-25T09:55:28.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac0b9330 0x7fc6ac152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 0x7fc6ac15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0b5e50 con 0x7fc6ac161710 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac162500 con 0x7fc6ac15dc20 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b254c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac0b5fd0 con 0x7fc6ac0b9330 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b0ae9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 0x7fc6ac15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b0ae9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 0x7fc6ac15c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47560/0 (socket says 172.21.15.148:47560) 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.902+0000 7fc6b0ae9640 1 -- 172.21.15.148:0/3738823551 learned_addr learned my addr 172.21.15.148:0/3738823551 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6abfff640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac0b9330 0x7fc6ac152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6ab7fe640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b0ae9640 1 -- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac0b9330 msgr2=0x7fc6ac152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b0ae9640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac0b9330 0x7fc6ac152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b0ae9640 1 -- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6ac15dc20 msgr2=0x7fc6ac0ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b0ae9640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ab5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b0ae9640 1 -- 172.21.15.148:0/3738823551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac0aa500 con 0x7fc6ac161710 2024-04-25T09:55:28.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6abfff640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac0b9330 0x7fc6ac152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6ab7fe640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b0ae9640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 0x7fc6ac15c860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c7b0 tx=0x7fc6a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4018070 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a400a640 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac0aa7e0 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a400ce60 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.903+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac0aada0 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.904+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac0b8ef0 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.904+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6a401b070 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.905+0000 7fc6a8ff9640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc69806b4d0 0x7fc69806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.905+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc6a4089530 con 0x7fc6ac161710 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.905+0000 7fc6abfff640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc69806b4d0 0x7fc69806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:28.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.905+0000 7fc6abfff640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc69806b4d0 0x7fc69806d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc69c034000 tx=0x7fc69c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:28.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:28.909+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6ac0b8ef0 con 0x7fc6ac161710 2024-04-25T09:55:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.146+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc6ac0d16b0 con 0x7fc6ac161710 2024-04-25T09:55:29.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.148+0000 7fc6a8ff9640 1 -- 172.21.15.148:0/3738823551 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc6a4058600 con 0x7fc6ac161710 2024-04-25T09:55:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc69806b4d0 msgr2=0x7fc69806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc69806b4d0 0x7fc69806d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc69c034000 tx=0x7fc69c02e000 comp rx=0 tx=0).stop 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 msgr2=0x7fc6ac15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 0x7fc6ac15c860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c7b0 tx=0x7fc6a400cc70 comp rx=0 tx=0).stop 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 shutdown_connections 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc69806b4d0 0x7fc69806d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.150+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6ac161710 0x7fc6ac15c860 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.151+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6ac15dc20 0x7fc6ac0ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.151+0000 7fc6b254c640 1 --2- 172.21.15.148:0/3738823551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6ac0b9330 0x7fc6ac152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.151+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 >> 172.21.15.148:0/3738823551 conn(0x7fc6ac0d0420 msgr2=0x7fc6ac0c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.151+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 shutdown_connections 2024-04-25T09:55:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.151+0000 7fc6b254c640 1 -- 172.21.15.148:0/3738823551 wait complete. 2024-04-25T09:55:29.164 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T09:55:29.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.259+0000 7f4298b3b640 1 Processor -- start 2024-04-25T09:55:29.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.259+0000 7f4298b3b640 1 -- start start 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.260+0000 7f4298b3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f429415bb50 0x7f42940c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.260+0000 7f4298b3b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42940c75e0 con 0x7f42940c70d0 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.260+0000 7f4298b3b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42940c78c0 con 0x7f42940cfd10 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.260+0000 7f4298b3b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42940c8680 con 0x7f429415bb50 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.261+0000 7f429259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f429415bb50 0x7f42940c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.261+0000 7f4291d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f42940c70d0 0x7f42940cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42660/0 (socket says 172.21.15.148:42660) 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.261+0000 7f4291d9b640 1 -- 172.21.15.148:0/1109396435 learned_addr learned my addr 172.21.15.148:0/1109396435 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.261+0000 7f429259c640 1 -- 172.21.15.148:0/1109396435 >> v1:172.21.15.188:6789/0 conn(0x7f42940cfd10 legacy=0x7f42940cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.261+0000 7f429259c640 1 -- 172.21.15.148:0/1109396435 >> v1:172.21.15.148:6789/0 conn(0x7f42940c70d0 legacy=0x7f42940cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.261+0000 7f429259c640 1 -- 172.21.15.148:0/1109396435 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940c7ba0 con 0x7f429415bb50 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f429259c640 1 --2- 172.21.15.148:0/1109396435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f42940c6b90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f42840097a0 tx=0x7f428402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddd70a9b2aa51e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f429159a640 1 -- 172.21.15.148:0/1109396435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284037070 con 0x7f429415bb50 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f429159a640 1 -- 172.21.15.148:0/1109396435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4284032440 con 0x7f429415bb50 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f429159a640 1 -- 172.21.15.148:0/1109396435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284032810 con 0x7f429415bb50 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 -- 172.21.15.148:0/1109396435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 msgr2=0x7f42940c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 --2- 172.21.15.148:0/1109396435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f42940c6b90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f42840097a0 tx=0x7f428402a270 comp rx=0 tx=0).stop 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 -- 172.21.15.148:0/1109396435 shutdown_connections 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 --2- 172.21.15.148:0/1109396435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f42940c6b90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 -- 172.21.15.148:0/1109396435 >> 172.21.15.148:0/1109396435 conn(0x7f429406f500 msgr2=0x7f429406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 -- 172.21.15.148:0/1109396435 shutdown_connections 2024-04-25T09:55:29.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.262+0000 7f4298b3b640 1 -- 172.21.15.148:0/1109396435 wait complete. 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 Processor -- start 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 -- start start 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42940c70d0 0x7f4294159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42940cfd10 0x7f429413f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f429413f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429415f6d0 con 0x7f429415bb50 2024-04-25T09:55:29.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42940c8160 con 0x7f42940c70d0 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.264+0000 7f4298b3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429415f850 con 0x7f42940cfd10 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f429413f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f429413f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47578/0 (socket says 172.21.15.148:47578) 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 -- 172.21.15.148:0/3030352489 learned_addr learned my addr 172.21.15.148:0/3030352489 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f429259c640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42940c70d0 0x7f4294159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4291d9b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42940cfd10 0x7f429413f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 -- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42940cfd10 msgr2=0x7f429413f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42940cfd10 0x7f429413f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 -- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42940c70d0 msgr2=0x7f4294159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42940c70d0 0x7f4294159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 -- 172.21.15.148:0/3030352489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429414d7e0 con 0x7f429415bb50 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4291d9b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42940cfd10 0x7f429413f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:29.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4292d9d640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f429413f750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c700 tx=0x7f428c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c016020 con 0x7f429415bb50 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c00aa50 con 0x7f429415bb50 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429414dac0 con 0x7f429415bb50 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00ae40 con 0x7f429415bb50 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.265+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42941596e0 con 0x7f429415bb50 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.266+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4294073a00 con 0x7f429415bb50 2024-04-25T09:55:29.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.267+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f428c019020 con 0x7f429415bb50 2024-04-25T09:55:29.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.267+0000 7f428affd640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f427406b4d0 0x7f427406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.267+0000 7f429259c640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f427406b4d0 0x7f427406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.267+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f428c089e80 con 0x7f429415bb50 2024-04-25T09:55:29.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.268+0000 7f429259c640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f427406b4d0 0x7f427406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4284034000 tx=0x7f428402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.271+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f428c020030 con 0x7f429415bb50 2024-04-25T09:55:29.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.512+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f42940b7400 con 0x7f429415bb50 2024-04-25T09:55:29.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.513+0000 7f428affd640 1 -- 172.21.15.148:0/3030352489 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f428c058f50 con 0x7f429415bb50 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.515+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f427406b4d0 msgr2=0x7f427406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.515+0000 7f4298b3b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f427406b4d0 0x7f427406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4284034000 tx=0x7f428402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.515+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 msgr2=0x7f429413f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.515+0000 7f4298b3b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f429413f750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c700 tx=0x7f428c00cbc0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 shutdown_connections 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f427406b4d0 0x7f427406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f429415bb50 0x7f429413f750 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42940cfd10 0x7f429413f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 --2- 172.21.15.148:0/3030352489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42940c70d0 0x7f4294159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 >> 172.21.15.148:0/3030352489 conn(0x7f429406f500 msgr2=0x7f42940b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 shutdown_connections 2024-04-25T09:55:29.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.516+0000 7f4298b3b640 1 -- 172.21.15.148:0/3030352489 wait complete. 2024-04-25T09:55:29.529 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T09:55:29.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.625+0000 7f0a4425d640 1 Processor -- start 2024-04-25T09:55:29.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.625+0000 7f0a4425d640 1 -- start start 2024-04-25T09:55:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a4425d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c144d20 0x7f0a3c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a4425d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0a3c0714d0 0x7f0a3c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a4425d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c0ceba0 con 0x7f0a3c1454b0 2024-04-25T09:55:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a4425d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0b4830 con 0x7f0a3c0714d0 2024-04-25T09:55:29.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a4425d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0cee80 con 0x7f0a3c144d20 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0a3c0714d0 0x7f0a3c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0a3c0714d0 0x7f0a3c0b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47588/0 (socket says 172.21.15.148:47588) 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 -- 172.21.15.148:0/383203515 learned_addr learned my addr 172.21.15.148:0/383203515 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a41ff9640 1 --2- 172.21.15.148:0/383203515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c144d20 0x7f0a3c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 -- 172.21.15.148:0/383203515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c144d20 msgr2=0x7f0a3c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 --2- 172.21.15.148:0/383203515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c144d20 0x7f0a3c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 -- 172.21.15.148:0/383203515 >> v1:172.21.15.188:6789/0 conn(0x7f0a3c1454b0 legacy=0x7f0a3c070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:29.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.626+0000 7f0a427fa640 1 -- 172.21.15.148:0/383203515 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0cf160 con 0x7f0a3c0714d0 2024-04-25T09:55:29.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.627+0000 7f0a427fa640 1 --2- 172.21.15.148:0/383203515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c0b4ca0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009e50 tx=0x7f0a3402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171d69313b55162c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.627+0000 7f0a40ff7640 1 -- 172.21.15.148:0/383203515 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34036070 con 0x7f0a3c0714d0 2024-04-25T09:55:29.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.627+0000 7f0a40ff7640 1 -- 172.21.15.148:0/383203515 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34031600 con 0x7f0a3c0714d0 2024-04-25T09:55:29.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.627+0000 7f0a40ff7640 1 -- 172.21.15.148:0/383203515 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340319d0 con 0x7f0a3c0714d0 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 -- 172.21.15.148:0/383203515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c0714d0 msgr2=0x7f0a3c0b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 --2- 172.21.15.148:0/383203515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c0b4ca0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009e50 tx=0x7f0a3402a6c0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 -- 172.21.15.148:0/383203515 shutdown_connections 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 --2- 172.21.15.148:0/383203515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c0b4ca0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 --2- 172.21.15.148:0/383203515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c144d20 0x7f0a3c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 -- 172.21.15.148:0/383203515 >> 172.21.15.148:0/383203515 conn(0x7f0a3c0b3530 msgr2=0x7f0a3c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 -- 172.21.15.148:0/383203515 shutdown_connections 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.628+0000 7f0a4425d640 1 -- 172.21.15.148:0/383203515 wait complete. 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.629+0000 7f0a4425d640 1 Processor -- start 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.629+0000 7f0a4425d640 1 -- start start 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 0x7f0a3c1f10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c1454b0 0x7f0a3c1f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c145100 con 0x7f0a3c144d20 2024-04-25T09:55:29.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0ce890 con 0x7f0a3c0714d0 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c145280 con 0x7f0a3c1454b0 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 0x7f0a3c1f10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 0x7f0a3c1f10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47600/0 (socket says 172.21.15.148:47600) 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 -- 172.21.15.148:0/2375515717 learned_addr learned my addr 172.21.15.148:0/2375515717 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a41ff9640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a427fa640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c1454b0 0x7f0a3c1f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 -- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c1454b0 msgr2=0x7f0a3c1f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c1454b0 0x7f0a3c1f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 -- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c0714d0 msgr2=0x7f0a3c1f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c1f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 -- 172.21.15.148:0/2375515717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c13a8f0 con 0x7f0a3c144d20 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a417f8640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 0x7f0a3c1f10b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c430 tx=0x7f0a2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011790 con 0x7f0a3c144d20 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c017070 con 0x7f0a3c144d20 2024-04-25T09:55:29.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c011bb0 con 0x7f0a3c144d20 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c06bcf0 con 0x7f0a3c144d20 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.630+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c06c2b0 con 0x7f0a3c144d20 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.631+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c073a00 con 0x7f0a3c144d20 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.632+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a2c01c460 con 0x7f0a3c144d20 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.632+0000 7f0a327fc640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a2006b4d0 0x7f0a2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.632+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0a2c089ea0 con 0x7f0a3c144d20 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.632+0000 7f0a41ff9640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a2006b4d0 0x7f0a2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.633+0000 7f0a41ff9640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a2006b4d0 0x7f0a2006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a38006fd0 tx=0x7f0a38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.636+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a2c058f70 con 0x7f0a3c144d20 2024-04-25T09:55:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.871+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f0a3c0cf7f0 con 0x7f0a3c144d20 2024-04-25T09:55:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.873+0000 7f0a327fc640 1 -- 172.21.15.148:0/2375515717 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0a2c058870 con 0x7f0a3c144d20 2024-04-25T09:55:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a2006b4d0 msgr2=0x7f0a2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a2006b4d0 0x7f0a2006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a38006fd0 tx=0x7f0a38008040 comp rx=0 tx=0).stop 2024-04-25T09:55:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 msgr2=0x7f0a3c1f10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 0x7f0a3c1f10b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00c430 tx=0x7f0a2c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 shutdown_connections 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a2006b4d0 0x7f0a2006d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a3c1454b0 0x7f0a3c1f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c144d20 0x7f0a3c1f10b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 --2- 172.21.15.148:0/2375515717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a3c0714d0 0x7f0a3c1f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 >> 172.21.15.148:0/2375515717 conn(0x7f0a3c0b3530 msgr2=0x7f0a3c143b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 shutdown_connections 2024-04-25T09:55:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.875+0000 7f0a4425d640 1 -- 172.21.15.148:0/2375515717 wait complete. 2024-04-25T09:55:29.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T09:55:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.986+0000 7f2415ed8640 1 Processor -- start 2024-04-25T09:55:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.986+0000 7f2415ed8640 1 -- start start 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f2415ed8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f24101616f0 0x7f2410161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f2415ed8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24100ce0a0 0x7f24100ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f2415ed8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241006c8d0 con 0x7f24101610f0 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f2415ed8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24101614d0 con 0x7f24101616f0 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f2415ed8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24100c6960 con 0x7f24100ce0a0 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f240effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f24101616f0 0x7f2410161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f240effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f24101616f0 0x7f2410161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47604/0 (socket says 172.21.15.148:47604) 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.987+0000 7f240effd640 1 -- 172.21.15.148:0/3596395775 learned_addr learned my addr 172.21.15.148:0/3596395775 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.988+0000 7f240effd640 1 -- 172.21.15.148:0/3596395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24100ce0a0 msgr2=0x7f24100ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.988+0000 7f240ffff640 1 --2- 172.21.15.148:0/3596395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24100ce0a0 0x7f24100ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.988+0000 7f240effd640 1 --2- 172.21.15.148:0/3596395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24100ce0a0 0x7f24100ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.988+0000 7f240effd640 1 -- 172.21.15.148:0/3596395775 >> v1:172.21.15.188:6789/0 conn(0x7f24101610f0 legacy=0x7f241006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.988+0000 7f240effd640 1 -- 172.21.15.148:0/3596395775 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410162260 con 0x7f24101616f0 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.988+0000 7f240effd640 1 --2- 172.21.15.148:0/3596395775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24101616f0 0x7f2410161b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2404009870 tx=0x7f240402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd41fc22f91ca108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f240e7fc640 1 -- 172.21.15.148:0/3596395775 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404037070 con 0x7f24101616f0 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f240e7fc640 1 -- 172.21.15.148:0/3596395775 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404032440 con 0x7f24101616f0 2024-04-25T09:55:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f240e7fc640 1 -- 172.21.15.148:0/3596395775 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404032810 con 0x7f24101616f0 2024-04-25T09:55:29.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 -- 172.21.15.148:0/3596395775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24101616f0 msgr2=0x7f2410161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 --2- 172.21.15.148:0/3596395775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24101616f0 0x7f2410161b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2404009870 tx=0x7f240402a210 comp rx=0 tx=0).stop 2024-04-25T09:55:29.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 -- 172.21.15.148:0/3596395775 shutdown_connections 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 --2- 172.21.15.148:0/3596395775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24101616f0 0x7f2410161b50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 --2- 172.21.15.148:0/3596395775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24100ce0a0 0x7f24100ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 -- 172.21.15.148:0/3596395775 >> 172.21.15.148:0/3596395775 conn(0x7f241006f500 msgr2=0x7f241006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 -- 172.21.15.148:0/3596395775 shutdown_connections 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.989+0000 7f2415ed8640 1 -- 172.21.15.148:0/3596395775 wait complete. 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.990+0000 7f2415ed8640 1 Processor -- start 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 -- start start 2024-04-25T09:55:29.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 0x7f241015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24101610f0 0x7f241015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24101616f0 0x7f24101504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24100cfce0 con 0x7f24100ce0a0 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24100cfa00 con 0x7f24101610f0 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f2415ed8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24100cf690 con 0x7f24101616f0 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 0x7f241015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 0x7f241015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47606/0 (socket says 172.21.15.148:47606) 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f240f7fe640 1 -- 172.21.15.148:0/2239643468 learned_addr learned my addr 172.21.15.148:0/2239643468 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f240effd640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24101610f0 0x7f241015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.991+0000 7f240ffff640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24101616f0 0x7f24101504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240f7fe640 1 -- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24101616f0 msgr2=0x7f24101504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240f7fe640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24101616f0 0x7f24101504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240f7fe640 1 -- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24101610f0 msgr2=0x7f241015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240f7fe640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24101610f0 0x7f241015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240f7fe640 1 -- 172.21.15.148:0/2239643468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100b5640 con 0x7f24100ce0a0 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240ffff640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24101616f0 0x7f24101504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f240f7fe640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 0x7f241015c5f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f240000c460 tx=0x7f240000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400016020 con 0x7f24100ce0a0 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400012870 con 0x7f24100ce0a0 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24101f7e60 con 0x7f24100ce0a0 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400012c60 con 0x7f24100ce0a0 2024-04-25T09:55:29.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.992+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2410157250 con 0x7f24100ce0a0 2024-04-25T09:55:29.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.993+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2410157490 con 0x7f24100ce0a0 2024-04-25T09:55:29.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.993+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2400019020 con 0x7f24100ce0a0 2024-04-25T09:55:29.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.994+0000 7f23f7fff640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f23fc06b4d0 0x7f23fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:29.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.994+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2400089a30 con 0x7f24100ce0a0 2024-04-25T09:55:29.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.994+0000 7f240effd640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f23fc06b4d0 0x7f23fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:29.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.994+0000 7f240effd640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f23fc06b4d0 0x7f23fc06d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2404034000 tx=0x7f240402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:29.998+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2410157490 con 0x7f24100ce0a0 2024-04-25T09:55:30.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.233+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f24100cf150 con 0x7f24100ce0a0 2024-04-25T09:55:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.235+0000 7f23f7fff640 1 -- 172.21.15.148:0/2239643468 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2400020070 con 0x7f24100ce0a0 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f23fc06b4d0 msgr2=0x7f23fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f23fc06b4d0 0x7f23fc06d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2404034000 tx=0x7f240402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 msgr2=0x7f241015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 0x7f241015c5f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f240000c460 tx=0x7f240000c920 comp rx=0 tx=0).stop 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 shutdown_connections 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f23fc06b4d0 0x7f23fc06d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24101616f0 0x7f24101504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24101610f0 0x7f241015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 --2- 172.21.15.148:0/2239643468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24100ce0a0 0x7f241015c5f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 >> 172.21.15.148:0/2239643468 conn(0x7f241006f500 msgr2=0x7f241006fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:30.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 shutdown_connections 2024-04-25T09:55:30.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.237+0000 7f2415ed8640 1 -- 172.21.15.148:0/2239643468 wait complete. 2024-04-25T09:55:30.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T09:55:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.346+0000 7f16a9ab3640 1 Processor -- start 2024-04-25T09:55:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a9ab3640 1 -- start start 2024-04-25T09:55:30.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a9ab3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16a415e0e0 0x7f16a409a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a9ab3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a40d06c0 con 0x7f16a415dd00 2024-04-25T09:55:30.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a9ab3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a409a690 con 0x7f16a406bf00 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a9ab3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40d1440 con 0x7f16a415e0e0 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f16a415dd00 0x7f16a4099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42676/0 (socket says 172.21.15.148:42676) 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.347+0000 7f16a37fe640 1 -- 172.21.15.148:0/3668042755 learned_addr learned my addr 172.21.15.148:0/3668042755 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.348+0000 7f16a2ffd640 1 --2- 172.21.15.148:0/3668042755 >> v2:172.21.15.148:3300/0 conn(0x7f16a415e0e0 0x7f16a409a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.348+0000 7f16a27fc640 1 -- 172.21.15.148:0/3668042755 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942054637 0 0) 0x7f16a40d06c0 con 0x7f16a415dd00 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.348+0000 7f16a27fc640 1 -- 172.21.15.148:0/3668042755 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168c003520 con 0x7f16a415dd00 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.348+0000 7f16a2ffd640 1 -- 172.21.15.148:0/3668042755 >> v1:172.21.15.188:6789/0 conn(0x7f16a406bf00 legacy=0x7f16a406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.348+0000 7f16a2ffd640 1 -- 172.21.15.148:0/3668042755 >> v1:172.21.15.148:6789/0 conn(0x7f16a415dd00 legacy=0x7f16a4099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.348+0000 7f16a2ffd640 1 -- 172.21.15.148:0/3668042755 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40d0960 con 0x7f16a415e0e0 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a2ffd640 1 --2- 172.21.15.148:0/3668042755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a415e0e0 0x7f16a409a150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f16980097a0 tx=0x7f169802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f872e8da1e62efa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a27fc640 1 -- 172.21.15.148:0/3668042755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698037070 con 0x7f16a415e0e0 2024-04-25T09:55:30.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a27fc640 1 -- 172.21.15.148:0/3668042755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1698032440 con 0x7f16a415e0e0 2024-04-25T09:55:30.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a27fc640 1 -- 172.21.15.148:0/3668042755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698032810 con 0x7f16a415e0e0 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 -- 172.21.15.148:0/3668042755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a415e0e0 msgr2=0x7f16a409a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/3668042755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a415e0e0 0x7f16a409a150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f16980097a0 tx=0x7f169802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 -- 172.21.15.148:0/3668042755 shutdown_connections 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/3668042755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a415e0e0 0x7f16a409a150 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 -- 172.21.15.148:0/3668042755 >> 172.21.15.148:0/3668042755 conn(0x7f16a40b3290 msgr2=0x7f16a40b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 -- 172.21.15.148:0/3668042755 shutdown_connections 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.349+0000 7f16a9ab3640 1 -- 172.21.15.148:0/3668042755 wait complete. 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.350+0000 7f16a9ab3640 1 Processor -- start 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 -- start start 2024-04-25T09:55:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 0x7f16a4159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a415dd00 0x7f16a41f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a415e0e0 0x7f16a40c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a409a950 con 0x7f16a406bf00 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40d0f20 con 0x7f16a415dd00 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a9ab3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40b2b40 con 0x7f16a415e0e0 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 0x7f16a4159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 0x7f16a4159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47624/0 (socket says 172.21.15.148:47624) 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.351+0000 7f16a37fe640 1 -- 172.21.15.148:0/643308146 learned_addr learned my addr 172.21.15.148:0/643308146 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a37fe640 1 -- 172.21.15.148:0/643308146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a415e0e0 msgr2=0x7f16a40c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a37fe640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a415e0e0 0x7f16a40c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a37fe640 1 -- 172.21.15.148:0/643308146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a415dd00 msgr2=0x7f16a41f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a37fe640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a415dd00 0x7f16a41f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a37fe640 1 -- 172.21.15.148:0/643308146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a41f41b0 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a37fe640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 0x7f16a4159640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f169400c390 tx=0x7f169400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694017070 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694005ce0 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a41f4490 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940072f0 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.352+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a40b4820 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.353+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a4073a00 con 0x7f16a406bf00 2024-04-25T09:55:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.353+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169401a070 con 0x7f16a406bf00 2024-04-25T09:55:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.354+0000 7f168bfff640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.354+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f16940893d0 con 0x7f16a406bf00 2024-04-25T09:55:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.354+0000 7f16a2ffd640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.354+0000 7f16a2ffd640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f169006b4d0 0x7f169006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1698034000 tx=0x7f169802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.358+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16940584a0 con 0x7f16a406bf00 2024-04-25T09:55:30.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.593+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f16a40ccd90 con 0x7f16a406bf00 2024-04-25T09:55:30.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.595+0000 7f168bfff640 1 -- 172.21.15.148:0/643308146 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1694057da0 con 0x7f16a406bf00 2024-04-25T09:55:30.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f169006b4d0 msgr2=0x7f169006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f169006b4d0 0x7f169006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1698034000 tx=0x7f169802c000 comp rx=0 tx=0).stop 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 msgr2=0x7f16a4159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 0x7f16a4159640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f169400c390 tx=0x7f169400c850 comp rx=0 tx=0).stop 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 shutdown_connections 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.597+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a415e0e0 0x7f16a40c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.598+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a415dd00 0x7f16a41f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.598+0000 7f16a9ab3640 1 --2- 172.21.15.148:0/643308146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a406bf00 0x7f16a4159640 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.598+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 >> 172.21.15.148:0/643308146 conn(0x7f16a40b3290 msgr2=0x7f16a406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.598+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 shutdown_connections 2024-04-25T09:55:30.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.598+0000 7f16a9ab3640 1 -- 172.21.15.148:0/643308146 wait complete. 2024-04-25T09:55:30.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T09:55:30.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.706+0000 7fa929f7e640 1 Processor -- start 2024-04-25T09:55:30.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.706+0000 7fa929f7e640 1 -- start start 2024-04-25T09:55:30.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa929f7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa92415dac0 0x7fa9240d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa929f7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9240d1f30 0x7fa9240d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa929f7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9240d34c0 0x7fa9240d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa929f7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9240d3030 con 0x7fa9240d34c0 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa929f7e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9240d2420 con 0x7fa9240d1f30 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa929f7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240d2700 con 0x7fa92415dac0 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9240d34c0 0x7fa9240d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9240d34c0 0x7fa9240d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47632/0 (socket says 172.21.15.148:47632) 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 -- 172.21.15.148:0/2220466786 learned_addr learned my addr 172.21.15.148:0/2220466786 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 -- 172.21.15.148:0/2220466786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa92415dac0 msgr2=0x7fa9240d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 --2- 172.21.15.148:0/2220466786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa92415dac0 0x7fa9240d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 -- 172.21.15.148:0/2220466786 >> v2:172.21.15.188:3300/0 conn(0x7fa9240d1f30 msgr2=0x7fa9240d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:30.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 --2- 172.21.15.148:0/2220466786 >> v2:172.21.15.188:3300/0 conn(0x7fa9240d1f30 0x7fa9240d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.707+0000 7fa923fff640 1 -- 172.21.15.148:0/2220466786 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240d29e0 con 0x7fa9240d34c0 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.708+0000 7fa923fff640 1 --2- 172.21.15.148:0/2220466786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9240d34c0 0x7fa9240d08e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b6550 tx=0x7fa91c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb2376087de39c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.708+0000 7fa9227fc640 1 -- 172.21.15.148:0/2220466786 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c036070 con 0x7fa9240d34c0 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.708+0000 7fa9227fc640 1 -- 172.21.15.148:0/2220466786 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c031600 con 0x7fa9240d34c0 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.708+0000 7fa9227fc640 1 -- 172.21.15.148:0/2220466786 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0319d0 con 0x7fa9240d34c0 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.708+0000 7fa929f7e640 1 -- 172.21.15.148:0/2220466786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9240d34c0 msgr2=0x7fa9240d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.708+0000 7fa929f7e640 1 --2- 172.21.15.148:0/2220466786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9240d34c0 0x7fa9240d08e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b6550 tx=0x7fa91c02a730 comp rx=0 tx=0).stop 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 -- 172.21.15.148:0/2220466786 shutdown_connections 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 --2- 172.21.15.148:0/2220466786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9240d34c0 0x7fa9240d08e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 --2- 172.21.15.148:0/2220466786 >> v2:172.21.15.188:3300/0 conn(0x7fa9240d1f30 0x7fa9240d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 --2- 172.21.15.148:0/2220466786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa92415dac0 0x7fa9240d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 -- 172.21.15.148:0/2220466786 >> 172.21.15.148:0/2220466786 conn(0x7fa924061c30 msgr2=0x7fa92406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 -- 172.21.15.148:0/2220466786 shutdown_connections 2024-04-25T09:55:30.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.709+0000 7fa929f7e640 1 -- 172.21.15.148:0/2220466786 wait complete. 2024-04-25T09:55:30.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.710+0000 7fa929f7e640 1 Processor -- start 2024-04-25T09:55:30.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.710+0000 7fa929f7e640 1 -- start start 2024-04-25T09:55:30.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9240d1f30 0x7fa9240c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9240d34c0 0x7fa924155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 0x7fa9241f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240b97d0 con 0x7fa92415dac0 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92415dea0 con 0x7fa9240d1f30 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240b9950 con 0x7fa9240d34c0 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 0x7fa9241f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 0x7fa9241f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47642/0 (socket says 172.21.15.148:47642) 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 -- 172.21.15.148:0/4229818321 learned_addr learned my addr 172.21.15.148:0/4229818321 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 -- 172.21.15.148:0/4229818321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9240d34c0 msgr2=0x7fa924155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9240d34c0 0x7fa924155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 -- 172.21.15.148:0/4229818321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9240d1f30 msgr2=0x7fa9240c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9240d1f30 0x7fa9240c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 -- 172.21.15.148:0/4229818321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9241f8440 con 0x7fa92415dac0 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa923fff640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 0x7fa9241f3880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa91c034040 tx=0x7fa91c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c041020 con 0x7fa92415dac0 2024-04-25T09:55:30.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c0077a0 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9241f75e0 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c036040 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.711+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9241f7ba0 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.712+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa924073a00 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.713+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa91c044020 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.713+0000 7fa90bfff640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.713+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa91c0aadc0 con 0x7fa92415dac0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.713+0000 7fa9237fe640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:30.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.714+0000 7fa9237fe640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa91006b4d0 0x7fa91006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa914009670 tx=0x7fa914006bf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:30.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.717+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa91c04b030 con 0x7fa92415dac0 2024-04-25T09:55:30.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.955+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa9240b3570 con 0x7fa92415dac0 2024-04-25T09:55:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.957+0000 7fa90bfff640 1 -- 172.21.15.148:0/4229818321 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa91c079ec0 con 0x7fa92415dac0 2024-04-25T09:55:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa91006b4d0 msgr2=0x7fa91006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa91006b4d0 0x7fa91006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa914009670 tx=0x7fa914006bf0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 msgr2=0x7fa9241f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 0x7fa9241f3880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa91c034040 tx=0x7fa91c004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 shutdown_connections 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92415dac0 0x7fa9241f3880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9240d34c0 0x7fa924155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 --2- 172.21.15.148:0/4229818321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9240d1f30 0x7fa9240c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.959+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 >> 172.21.15.148:0/4229818321 conn(0x7fa924061c30 msgr2=0x7fa9240b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.960+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 shutdown_connections 2024-04-25T09:55:30.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:30.960+0000 7fa929f7e640 1 -- 172.21.15.148:0/4229818321 wait complete. 2024-04-25T09:55:30.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:31.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 Processor -- start 2024-04-25T09:55:31.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 -- start start 2024-04-25T09:55:31.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbe5815dc90 0x7fbe581625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbe580cc8e0 0x7fbe5815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe580d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe580d21c0 con 0x7fbe5815dc90 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5815f390 con 0x7fbe580cc8e0 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.069+0000 7fbe5c883640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580cce50 con 0x7fbe580d1810 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbe5815dc90 0x7fbe581625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbe5815dc90 0x7fbe581625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47648/0 (socket says 172.21.15.148:47648) 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 -- 172.21.15.148:0/4041705571 learned_addr learned my addr 172.21.15.148:0/4041705571 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe55d9b640 1 --2- 172.21.15.148:0/4041705571 >> v2:172.21.15.188:3300/0 conn(0x7fbe580cc8e0 0x7fbe5815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe56d9d640 1 --2- 172.21.15.148:0/4041705571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe580d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 -- 172.21.15.148:0/4041705571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 msgr2=0x7fbe580d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 --2- 172.21.15.148:0/4041705571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe580d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 -- 172.21.15.148:0/4041705571 >> v2:172.21.15.188:3300/0 conn(0x7fbe580cc8e0 msgr2=0x7fbe5815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 --2- 172.21.15.148:0/4041705571 >> v2:172.21.15.188:3300/0 conn(0x7fbe580cc8e0 0x7fbe5815ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.070+0000 7fbe5659c640 1 -- 172.21.15.148:0/4041705571 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58161af0 con 0x7fbe5815dc90 2024-04-25T09:55:31.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5659c640 1 --2- 172.21.15.148:0/4041705571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5815dc90 0x7fbe581625d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbe580c88a0 tx=0x7fbe4802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f06cadb74f0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5559a640 1 -- 172.21.15.148:0/4041705571 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48037070 con 0x7fbe5815dc90 2024-04-25T09:55:31.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5559a640 1 -- 172.21.15.148:0/4041705571 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48032440 con 0x7fbe5815dc90 2024-04-25T09:55:31.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5559a640 1 -- 172.21.15.148:0/4041705571 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48032810 con 0x7fbe5815dc90 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 -- 172.21.15.148:0/4041705571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5815dc90 msgr2=0x7fbe581625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 --2- 172.21.15.148:0/4041705571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5815dc90 0x7fbe581625d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbe580c88a0 tx=0x7fbe4802a140 comp rx=0 tx=0).stop 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 -- 172.21.15.148:0/4041705571 shutdown_connections 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 --2- 172.21.15.148:0/4041705571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe580d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 --2- 172.21.15.148:0/4041705571 >> v2:172.21.15.188:3300/0 conn(0x7fbe580cc8e0 0x7fbe5815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 --2- 172.21.15.148:0/4041705571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe5815dc90 0x7fbe581625d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 -- 172.21.15.148:0/4041705571 >> 172.21.15.148:0/4041705571 conn(0x7fbe5806f590 msgr2=0x7fbe5806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 -- 172.21.15.148:0/4041705571 shutdown_connections 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.071+0000 7fbe5c883640 1 -- 172.21.15.148:0/4041705571 wait complete. 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 Processor -- start 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 -- start start 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 0x7fbe58155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe5815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe5815dc90 0x7fbe580c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580cd130 con 0x7fbe580cc8e0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe581620b0 con 0x7fbe5815dc90 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.073+0000 7fbe5c883640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5815e070 con 0x7fbe580d1810 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 0x7fbe58155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 0x7fbe58155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47658/0 (socket says 172.21.15.148:47658) 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 -- 172.21.15.148:0/2513617702 learned_addr learned my addr 172.21.15.148:0/2513617702 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe56d9d640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe5815dc90 0x7fbe580c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 -- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 msgr2=0x7fbe5815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe5815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 -- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe5815dc90 msgr2=0x7fbe580c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe5815dc90 0x7fbe580c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 -- 172.21.15.148:0/2513617702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580c8ba0 con 0x7fbe580cc8e0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5659c640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 0x7fbe58155cd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbe48000c00 tx=0x7fbe480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48037040 con 0x7fbe580cc8e0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe48042070 con 0x7fbe580cc8e0 2024-04-25T09:55:31.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe580c8e80 con 0x7fbe580cc8e0 2024-04-25T09:55:31.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48030bb0 con 0x7fbe580cc8e0 2024-04-25T09:55:31.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.074+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580c9440 con 0x7fbe580cc8e0 2024-04-25T09:55:31.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.075+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe580c8930 con 0x7fbe580cc8e0 2024-04-25T09:55:31.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.076+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe48047460 con 0x7fbe580cc8e0 2024-04-25T09:55:31.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.076+0000 7fbe4effd640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe3806b4d0 0x7fbe3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.076+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe480aac00 con 0x7fbe580cc8e0 2024-04-25T09:55:31.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.076+0000 7fbe55d9b640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe3806b4d0 0x7fbe3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.077+0000 7fbe55d9b640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe3806b4d0 0x7fbe3806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006b10 tx=0x7fbe40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.080+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe480b1080 con 0x7fbe580cc8e0 2024-04-25T09:55:31.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.251+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe58073990 con 0x7fbe580cc8e0 2024-04-25T09:55:31.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.252+0000 7fbe4effd640 1 -- 172.21.15.148:0/2513617702 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13212 (secure 0 0 0) 0x7fbe48045360 con 0x7fbe580cc8e0 2024-04-25T09:55:31.254 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:31.254 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":35,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:28.428113+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:31.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe3806b4d0 msgr2=0x7fbe3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe3806b4d0 0x7fbe3806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbe40006b10 tx=0x7fbe40008040 comp rx=0 tx=0).stop 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 msgr2=0x7fbe58155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 0x7fbe58155cd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbe48000c00 tx=0x7fbe480046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 shutdown_connections 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbe3806b4d0 0x7fbe3806d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe5815dc90 0x7fbe580c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe580d1810 0x7fbe5815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 --2- 172.21.15.148:0/2513617702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe580cc8e0 0x7fbe58155cd0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 >> 172.21.15.148:0/2513617702 conn(0x7fbe5806f590 msgr2=0x7fbe580b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 shutdown_connections 2024-04-25T09:55:31.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.254+0000 7fbe5c883640 1 -- 172.21.15.148:0/2513617702 wait complete. 2024-04-25T09:55:31.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:55:31.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:55:31.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.364+0000 7f8f96209640 1 Processor -- start 2024-04-25T09:55:31.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.364+0000 7f8f96209640 1 -- start start 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f96209640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f900cc4f0 0x7f8f900cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f96209640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9015b980 0x7f8f900cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f96209640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f90072280 con 0x7f8f900cdb20 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f96209640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f90071270 con 0x7f8f900cc4f0 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f96209640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900714c0 con 0x7f8f9015b980 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8f900cdb20 0x7f8f9015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42690/0 (socket says 172.21.15.148:42690) 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8effd640 1 -- 172.21.15.148:0/945863812 learned_addr learned my addr 172.21.15.148:0/945863812 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723884206 0 0) 0x7f8f90072280 con 0x7f8f900cdb20 2024-04-25T09:55:31.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9015b980 0x7f8f900cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f78003520 con 0x7f8f900cdb20 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/945863812 >> v2:172.21.15.188:3300/0 conn(0x7f8f900cc4f0 0x7f8f900cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3254671974 0 0) 0x7f8f78003520 con 0x7f8f900cdb20 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9015b980 msgr2=0x7f8f900cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9015b980 0x7f8f900cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 >> v2:172.21.15.188:3300/0 conn(0x7f8f900cc4f0 msgr2=0x7f8f900cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.365+0000 7f8f8e7fc640 1 --2- 172.21.15.148:0/945863812 >> v2:172.21.15.188:3300/0 conn(0x7f8f900cc4f0 0x7f8f900cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900717a0 con 0x7f8f900cdb20 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f8f84001690 con 0x7f8f900cdb20 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 >> v1:172.21.15.148:6789/0 conn(0x7f8f900cdb20 legacy=0x7f8f9015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f780039b0 0x7f8f78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f78004800 0x7f8f78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90071270 con 0x7f8f780051c0 2024-04-25T09:55:31.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900714c0 con 0x7f8f780039b0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7800c000 con 0x7f8f78004800 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/945863812 >> v2:172.21.15.188:3300/0 conn(0x7f8f900cc4f0 0x7f8f900cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9015b980 0x7f8f900cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f780039b0 0x7f8f78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8effd640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f78004800 0x7f8f78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 -- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f78004800 msgr2=0x7f8f78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f78004800 0x7f8f78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 -- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f780039b0 msgr2=0x7f8f78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f780039b0 0x7f8f78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8ffff640 1 -- 172.21.15.148:0/945863812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78003520 con 0x7f8f780051c0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.366+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f780039b0 0x7f8f78003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f7800b9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009f00 tx=0x7f8f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a4ad3e7c44dca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88009b50 con 0x7f8f780051c0 2024-04-25T09:55:31.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88003990 con 0x7f8f780051c0 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f8e7fc640 1 -- 172.21.15.148:0/945863812 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88003d60 con 0x7f8f780051c0 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f96209640 1 -- 172.21.15.148:0/945863812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 msgr2=0x7f8f7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f96209640 1 --2- 172.21.15.148:0/945863812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f7800b9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009f00 tx=0x7f8f78005830 comp rx=0 tx=0).stop 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f8ffff640 1 -- 172.21.15.148:0/945863812 reap_dead start 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f96209640 1 -- 172.21.15.148:0/945863812 shutdown_connections 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f96209640 1 -- 172.21.15.148:0/945863812 >> 172.21.15.148:0/945863812 conn(0x7f8f9006abe0 msgr2=0x7f8f9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.367+0000 7f8f96209640 1 -- 172.21.15.148:0/945863812 shutdown_connections 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.368+0000 7f8f96209640 1 -- 172.21.15.148:0/945863812 wait complete. 2024-04-25T09:55:31.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.369+0000 7f8f96209640 1 Processor -- start 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.369+0000 7f8f96209640 1 -- start start 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.369+0000 7f8f96209640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f901462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.369+0000 7f8f96209640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9014f4d0 0x7f8f900cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f96209640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f900cfc50 0x7f8f900c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f96209640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900cdfb0 con 0x7f8f780051c0 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f96209640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f90071d60 con 0x7f8f900cfc50 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f96209640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900ce130 con 0x7f8f9014f4d0 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f901462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f901462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47682/0 (socket says 172.21.15.148:47682) 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 -- 172.21.15.148:0/3475561501 learned_addr learned my addr 172.21.15.148:0/3475561501 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:31.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f900cfc50 0x7f8f900c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8effd640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9014f4d0 0x7f8f900cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 -- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9014f4d0 msgr2=0x7f8f900cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9014f4d0 0x7f8f900cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 -- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f900cfc50 msgr2=0x7f8f900c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f900cfc50 0x7f8f900c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 -- 172.21.15.148:0/3475561501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f901570d0 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8ffff640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f900cfc50 0x7f8f900c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8effd640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9014f4d0 0x7f8f900cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f8f7fe640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f901462a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f80010890 tx=0x7f8f80010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8001a020 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8000eb20 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f901573b0 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000f040 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.370+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f900ca060 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.371+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f90073a00 con 0x7f8f780051c0 2024-04-25T09:55:31.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.372+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f8001d020 con 0x7f8f780051c0 2024-04-25T09:55:31.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.372+0000 7f8f77fff640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f7c06b4d0 0x7f8f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.372+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8f8008a610 con 0x7f8f780051c0 2024-04-25T09:55:31.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.372+0000 7f8f8effd640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f7c06b4d0 0x7f8f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.373+0000 7f8f8effd640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f7c06b4d0 0x7f8f7c06d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f901f6c70 tx=0x7f8f84009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.376+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f80024030 con 0x7f8f780051c0 2024-04-25T09:55:31.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.546+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f9006d3a0 con 0x7f8f780051c0 2024-04-25T09:55:31.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.547+0000 7f8f77fff640 1 -- 172.21.15.148:0/3475561501 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f800122d0 con 0x7f8f780051c0 2024-04-25T09:55:31.550 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f7c06b4d0 msgr2=0x7f8f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f7c06b4d0 0x7f8f7c06d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8f901f6c70 tx=0x7f8f84009040 comp rx=0 tx=0).stop 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 msgr2=0x7f8f901462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f901462a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f80010890 tx=0x7f8f80010d50 comp rx=0 tx=0).stop 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 shutdown_connections 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f7c06b4d0 0x7f8f7c06d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f900cfc50 0x7f8f900c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9014f4d0 0x7f8f900cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 --2- 172.21.15.148:0/3475561501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f780051c0 0x7f8f901462a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 >> 172.21.15.148:0/3475561501 conn(0x7f8f9006abe0 msgr2=0x7f8f90154490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 shutdown_connections 2024-04-25T09:55:31.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.549+0000 7f8f96209640 1 -- 172.21.15.148:0/3475561501 wait complete. 2024-04-25T09:55:31.563 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:55:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.657+0000 7fc27865d640 1 Processor -- start 2024-04-25T09:55:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.657+0000 7fc27865d640 1 -- start start 2024-04-25T09:55:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc27865d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2700b6cb0 0x7fc2700b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc27865d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700b5a60 0x7fc2700d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc27865d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2700ceee0 con 0x7fc27015db70 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc27865d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2700d1d10 con 0x7fc2700b6cb0 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc27865d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700d1ff0 con 0x7fc2700b5a60 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc275bf8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2700b6cb0 0x7fc2700b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc276bfa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700b5a60 0x7fc2700d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc2763f9640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc27015db70 0x7fc270162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56170/0 (socket says 172.21.15.148:56170) 2024-04-25T09:55:31.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc2763f9640 1 -- 172.21.15.148:0/2790379699 learned_addr learned my addr 172.21.15.148:0/2790379699 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc2753f7640 1 -- 172.21.15.148:0/2790379699 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602300345 0 0) 0x7fc2700ceee0 con 0x7fc27015db70 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.658+0000 7fc2753f7640 1 -- 172.21.15.148:0/2790379699 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc25c003520 con 0x7fc27015db70 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.659+0000 7fc276bfa640 1 -- 172.21.15.148:0/2790379699 >> v2:172.21.15.188:3300/0 conn(0x7fc2700b6cb0 msgr2=0x7fc2700b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.659+0000 7fc276bfa640 1 --2- 172.21.15.148:0/2790379699 >> v2:172.21.15.188:3300/0 conn(0x7fc2700b6cb0 0x7fc2700b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.659+0000 7fc276bfa640 1 -- 172.21.15.148:0/2790379699 >> v1:172.21.15.188:6789/0 conn(0x7fc27015db70 legacy=0x7fc270162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.659+0000 7fc276bfa640 1 -- 172.21.15.148:0/2790379699 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2700d22d0 con 0x7fc2700b5a60 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.659+0000 7fc275bf8640 1 --2- 172.21.15.148:0/2790379699 >> v2:172.21.15.188:3300/0 conn(0x7fc2700b6cb0 0x7fc2700b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.659+0000 7fc276bfa640 1 --2- 172.21.15.148:0/2790379699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700b5a60 0x7fc2700d27f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a8afcf1ebd8dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc2753f7640 1 -- 172.21.15.148:0/2790379699 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268007740 con 0x7fc2700b5a60 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc2753f7640 1 -- 172.21.15.148:0/2790379699 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2680078e0 con 0x7fc2700b5a60 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc2753f7640 1 -- 172.21.15.148:0/2790379699 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268007c90 con 0x7fc2700b5a60 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 -- 172.21.15.148:0/2790379699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700b5a60 msgr2=0x7fc2700d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 --2- 172.21.15.148:0/2790379699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700b5a60 0x7fc2700d27f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc268009d90 tx=0x7fc26802a660 comp rx=0 tx=0).stop 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 -- 172.21.15.148:0/2790379699 shutdown_connections 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 --2- 172.21.15.148:0/2790379699 >> v2:172.21.15.188:3300/0 conn(0x7fc2700b6cb0 0x7fc2700b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 --2- 172.21.15.148:0/2790379699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2700b5a60 0x7fc2700d27f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 -- 172.21.15.148:0/2790379699 >> 172.21.15.148:0/2790379699 conn(0x7fc2700d0520 msgr2=0x7fc2700d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 -- 172.21.15.148:0/2790379699 shutdown_connections 2024-04-25T09:55:31.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.660+0000 7fc27865d640 1 -- 172.21.15.148:0/2790379699 wait complete. 2024-04-25T09:55:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.661+0000 7fc27865d640 1 Processor -- start 2024-04-25T09:55:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.661+0000 7fc27865d640 1 -- start start 2024-04-25T09:55:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc27865d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 0x7fc27015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc27865d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2700b6cb0 0x7fc270155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc27865d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27015db70 0x7fc2700c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc27865d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700cf1c0 con 0x7fc2700b5a60 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc27865d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700ce530 con 0x7fc2700b6cb0 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc27865d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700cf340 con 0x7fc27015db70 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc2763f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 0x7fc27015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc2763f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 0x7fc27015f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47698/0 (socket says 172.21.15.148:47698) 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc2763f9640 1 -- 172.21.15.148:0/2523926802 learned_addr learned my addr 172.21.15.148:0/2523926802 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc275bf8640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2700b6cb0 0x7fc270155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.662+0000 7fc276bfa640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27015db70 0x7fc2700c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc2763f9640 1 -- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27015db70 msgr2=0x7fc2700c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc2763f9640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27015db70 0x7fc2700c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc2763f9640 1 -- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2700b6cb0 msgr2=0x7fc270155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc2763f9640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2700b6cb0 0x7fc270155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc2763f9640 1 -- 172.21.15.148:0/2523926802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270156170 con 0x7fc2700b5a60 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc275bf8640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2700b6cb0 0x7fc270155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc276bfa640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27015db70 0x7fc2700c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc2763f9640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 0x7fc27015f200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c430 tx=0x7fc26c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c017070 con 0x7fc2700b5a60 2024-04-25T09:55:31.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c005ce0 con 0x7fc2700b5a60 2024-04-25T09:55:31.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0072f0 con 0x7fc2700b5a60 2024-04-25T09:55:31.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27015f800 con 0x7fc2700b5a60 2024-04-25T09:55:31.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.663+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27015fdc0 con 0x7fc2700b5a60 2024-04-25T09:55:31.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.664+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc270073990 con 0x7fc2700b5a60 2024-04-25T09:55:31.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.664+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc26c01a070 con 0x7fc2700b5a60 2024-04-25T09:55:31.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.665+0000 7fc266ffd640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25406b500 0x7fc25406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.665+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc26c0894d0 con 0x7fc2700b5a60 2024-04-25T09:55:31.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.665+0000 7fc275bf8640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25406b500 0x7fc25406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.665+0000 7fc275bf8640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25406b500 0x7fc25406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc2700b9820 tx=0x7fc260008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.669+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc26c058510 con 0x7fc2700b5a60 2024-04-25T09:55:31.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.839+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc2700d1820 con 0x7fc2700b5a60 2024-04-25T09:55:31.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.840+0000 7fc266ffd640 1 -- 172.21.15.148:0/2523926802 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc26c057e10 con 0x7fc2700b5a60 2024-04-25T09:55:31.842 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:31.843 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:55:31.843 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:55:31.843 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T09:55:31.843 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T09:55:31.843 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:55:31.843 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.841+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25406b500 msgr2=0x7fc25406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.841+0000 7fc27865d640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25406b500 0x7fc25406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc2700b9820 tx=0x7fc260008040 comp rx=0 tx=0).stop 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.841+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 msgr2=0x7fc27015f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.841+0000 7fc27865d640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 0x7fc27015f200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c430 tx=0x7fc26c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 shutdown_connections 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc25406b500 0x7fc25406d9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc27015db70 0x7fc2700c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2700b6cb0 0x7fc270155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 --2- 172.21.15.148:0/2523926802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2700b5a60 0x7fc27015f200 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 >> 172.21.15.148:0/2523926802 conn(0x7fc2700d0520 msgr2=0x7fc2700d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 shutdown_connections 2024-04-25T09:55:31.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.842+0000 7fc27865d640 1 -- 172.21.15.148:0/2523926802 wait complete. 2024-04-25T09:55:31.855 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:55:31.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:55:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.954+0000 7f46265b0640 1 Processor -- start 2024-04-25T09:55:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f46265b0640 1 -- start start 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f46265b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f462015dc20 0x7f46200ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f46265b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f46200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f46265b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462015e000 con 0x7f4620161710 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f46265b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46200d1de0 con 0x7f462015dc20 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f46265b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4620161c60 con 0x7f46200d32f0 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f461ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f462015dc20 0x7f46200ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f461f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4620161710 0x7f462006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42696/0 (socket says 172.21.15.148:42696) 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.955+0000 7f461f7fe640 1 -- 172.21.15.148:0/4182012774 learned_addr learned my addr 172.21.15.148:0/4182012774 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.956+0000 7f461ffff640 1 -- 172.21.15.148:0/4182012774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 msgr2=0x7f46200cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.956+0000 7f461ffff640 1 --2- 172.21.15.148:0/4182012774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f46200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.956+0000 7f461ffff640 1 -- 172.21.15.148:0/4182012774 >> v1:172.21.15.148:6789/0 conn(0x7f4620161710 legacy=0x7f462006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.956+0000 7f461ffff640 1 -- 172.21.15.148:0/4182012774 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620161f40 con 0x7f462015dc20 2024-04-25T09:55:31.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.956+0000 7f461effd640 1 -- 172.21.15.148:0/4182012774 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543287813 0 0) 0x7f462015e000 con 0x7f4620161710 2024-04-25T09:55:31.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.956+0000 7f461ffff640 1 --2- 172.21.15.148:0/4182012774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46200ce670 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f46100097a0 tx=0x7f461002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812c64598e311e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f461effd640 1 -- 172.21.15.148:0/4182012774 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610037070 con 0x7f462015dc20 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f461effd640 1 -- 172.21.15.148:0/4182012774 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610032440 con 0x7f462015dc20 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f461effd640 1 -- 172.21.15.148:0/4182012774 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610032810 con 0x7f462015dc20 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 -- 172.21.15.148:0/4182012774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 msgr2=0x7f46200ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 --2- 172.21.15.148:0/4182012774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46200ce670 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f46100097a0 tx=0x7f461002a180 comp rx=0 tx=0).stop 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 -- 172.21.15.148:0/4182012774 shutdown_connections 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 --2- 172.21.15.148:0/4182012774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f46200cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 --2- 172.21.15.148:0/4182012774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46200ce670 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 -- 172.21.15.148:0/4182012774 >> 172.21.15.148:0/4182012774 conn(0x7f46200d0420 msgr2=0x7f46200d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 -- 172.21.15.148:0/4182012774 shutdown_connections 2024-04-25T09:55:31.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.957+0000 7f46265b0640 1 -- 172.21.15.148:0/4182012774 wait complete. 2024-04-25T09:55:31.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.958+0000 7f46265b0640 1 Processor -- start 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 -- start start 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f462015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46201f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620161710 0x7f46201e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46200b5e50 con 0x7f462015dc20 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620162500 con 0x7f4620161710 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f46265b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46200b5fd0 con 0x7f46200d32f0 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46201f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46201f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47714/0 (socket says 172.21.15.148:47714) 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.959+0000 7f461f7fe640 1 -- 172.21.15.148:0/2695518708 learned_addr learned my addr 172.21.15.148:0/2695518708 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f4624b4d640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620161710 0x7f46201e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461ffff640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f462015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461f7fe640 1 -- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 msgr2=0x7f462015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461f7fe640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f462015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461f7fe640 1 -- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620161710 msgr2=0x7f46201e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461f7fe640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620161710 0x7f46201e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461f7fe640 1 -- 172.21.15.148:0/2695518708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200cac80 con 0x7f462015dc20 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461f7fe640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46201f3e20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f461400c460 tx=0x7f461400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614018070 con 0x7f462015dc20 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614007500 con 0x7f462015dc20 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46200caf60 con 0x7f462015dc20 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614004a20 con 0x7f462015dc20 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.960+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46201f4360 con 0x7f462015dc20 2024-04-25T09:55:31.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.961+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4620161450 con 0x7f462015dc20 2024-04-25T09:55:31.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.962+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f461401b070 con 0x7f462015dc20 2024-04-25T09:55:31.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.962+0000 7f461cff9640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f460c06b4d0 0x7f460c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:31.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.962+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4614089300 con 0x7f462015dc20 2024-04-25T09:55:31.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.962+0000 7f461ffff640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f460c06b4d0 0x7f460c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:31.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.962+0000 7f461ffff640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f460c06b4d0 0x7f460c06d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4610034000 tx=0x7f461002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:31.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:31.966+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4620161450 con 0x7f462015dc20 2024-04-25T09:55:32.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.139+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4620068370 con 0x7f462015dc20 2024-04-25T09:55:32.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.960+0000 7f461cff9640 1 -- 172.21.15.148:0/2695518708 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4620068370 con 0x7f462015dc20 2024-04-25T09:55:32.963 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T09:55:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f460c06b4d0 msgr2=0x7f460c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f460c06b4d0 0x7f460c06d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4610034000 tx=0x7f461002e000 comp rx=0 tx=0).stop 2024-04-25T09:55:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 msgr2=0x7f46201f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46201f3e20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f461400c460 tx=0x7f461400c920 comp rx=0 tx=0).stop 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 shutdown_connections 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f460c06b4d0 0x7f460c06d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4620161710 0x7f46201e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f462015dc20 0x7f46201f3e20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 --2- 172.21.15.148:0/2695518708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46200d32f0 0x7f462015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.963+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 >> 172.21.15.148:0/2695518708 conn(0x7f46200d0420 msgr2=0x7f46200b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.964+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 shutdown_connections 2024-04-25T09:55:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:32.964+0000 7f46265b0640 1 -- 172.21.15.148:0/2695518708 wait complete. 2024-04-25T09:55:32.983 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:55:33.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.090+0000 7f58c4770640 1 Processor -- start 2024-04-25T09:55:33.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.090+0000 7f58c4770640 1 -- start start 2024-04-25T09:55:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c4770640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58bc0ce230 0x7f58bc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c4770640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc15b270 0x7f58bc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c4770640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc072280 con 0x7f58bc0cc4f0 2024-04-25T09:55:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c4770640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc071270 con 0x7f58bc0ce230 2024-04-25T09:55:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c4770640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0714c0 con 0x7f58bc15b270 2024-04-25T09:55:33.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58bc0ce230 0x7f58bc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58bc0ce230 0x7f58bc0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:60624/0 (socket says 172.21.15.148:60624) 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 -- 172.21.15.148:0/540777517 learned_addr learned my addr 172.21.15.148:0/540777517 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c250c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f58bc0cc4f0 0x7f58bc0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56174/0 (socket says 172.21.15.148:56174) 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c2d0d640 1 --2- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc15b270 0x7f58bc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 -- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc15b270 msgr2=0x7f58bc15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 --2- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc15b270 0x7f58bc15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 -- 172.21.15.148:0/540777517 >> v1:172.21.15.188:6789/0 conn(0x7f58bc0cc4f0 legacy=0x7f58bc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c150a640 1 -- 172.21.15.148:0/540777517 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160485180 0 0) 0x7f58bc072280 con 0x7f58bc0cc4f0 2024-04-25T09:55:33.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.091+0000 7f58c1d0b640 1 -- 172.21.15.148:0/540777517 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0717a0 con 0x7f58bc0ce230 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.092+0000 7f58c1d0b640 1 --2- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc0ce230 0x7f58bc0cc8d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009870 tx=0x7f58ac02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2fa71df24c60ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.092+0000 7f58c150a640 1 -- 172.21.15.148:0/540777517 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac037070 con 0x7f58bc0ce230 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.092+0000 7f58c150a640 1 -- 172.21.15.148:0/540777517 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac032440 con 0x7f58bc0ce230 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.092+0000 7f58c150a640 1 -- 172.21.15.148:0/540777517 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac032810 con 0x7f58bc0ce230 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 -- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc0ce230 msgr2=0x7f58bc0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 --2- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc0ce230 0x7f58bc0cc8d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009870 tx=0x7f58ac02a270 comp rx=0 tx=0).stop 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 -- 172.21.15.148:0/540777517 shutdown_connections 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 --2- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc0ce230 0x7f58bc0cc8d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 --2- 172.21.15.148:0/540777517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc15b270 0x7f58bc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 -- 172.21.15.148:0/540777517 >> 172.21.15.148:0/540777517 conn(0x7f58bc06abe0 msgr2=0x7f58bc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:33.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 -- 172.21.15.148:0/540777517 shutdown_connections 2024-04-25T09:55:33.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.093+0000 7f58c4770640 1 -- 172.21.15.148:0/540777517 wait complete. 2024-04-25T09:55:33.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.094+0000 7f58c4770640 1 Processor -- start 2024-04-25T09:55:33.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.094+0000 7f58c4770640 1 -- start start 2024-04-25T09:55:33.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c4770640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0cc4f0 0x7f58bc14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c4770640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 0x7f58bc1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c4770640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc15b270 0x7f58bc06db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c4770640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc15bbf0 con 0x7f58bc0ce230 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c4770640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc071d60 con 0x7f58bc15b270 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c4770640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc15bd70 con 0x7f58bc0cc4f0 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 0x7f58bc1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 0x7f58bc1483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47720/0 (socket says 172.21.15.148:47720) 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 -- 172.21.15.148:0/2985304096 learned_addr learned my addr 172.21.15.148:0/2985304096 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c2d0d640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc15b270 0x7f58bc06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c250c640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0cc4f0 0x7f58bc14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 -- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0cc4f0 msgr2=0x7f58bc14b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:33.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0cc4f0 0x7f58bc14b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 -- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc15b270 msgr2=0x7f58bc06db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc15b270 0x7f58bc06db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 -- 172.21.15.148:0/2985304096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc14bf00 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.095+0000 7f58c1d0b640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 0x7f58bc1483b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f58ac035040 tx=0x7f58ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.096+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac037040 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.096+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac004030 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.096+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004420 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.096+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc149f70 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.096+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1488f0 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.097+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc073a00 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.097+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58ac03a070 con 0x7f58bc0ce230 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.097+0000 7f58b2ffd640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a006b4d0 0x7f58a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:33.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.097+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f58ac0aa620 con 0x7f58bc0ce230 2024-04-25T09:55:33.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.098+0000 7f58c250c640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a006b4d0 0x7f58a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:33.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.098+0000 7f58c250c640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a006b4d0 0x7f58a006d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58b8007fd0 tx=0x7f58b8007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:33.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.102+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58ac041020 con 0x7f58bc0ce230 2024-04-25T09:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.273+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f58bc06d3a0 con 0x7f58bc0ce230 2024-04-25T09:55:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.968+0000 7f58b2ffd640 1 -- 172.21.15.148:0/2985304096 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f58ac078eb0 con 0x7f58bc0ce230 2024-04-25T09:55:33.971 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T09:55:33.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.971+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a006b4d0 msgr2=0x7f58a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:33.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.971+0000 7f58c4770640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a006b4d0 0x7f58a006d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58b8007fd0 tx=0x7f58b8007de0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 msgr2=0x7f58bc1483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 0x7f58bc1483b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f58ac035040 tx=0x7f58ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 shutdown_connections 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a006b4d0 0x7f58a006d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc15b270 0x7f58bc06db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc0ce230 0x7f58bc1483b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 --2- 172.21.15.148:0/2985304096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0cc4f0 0x7f58bc14b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 >> 172.21.15.148:0/2985304096 conn(0x7f58bc06abe0 msgr2=0x7f58bc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 shutdown_connections 2024-04-25T09:55:33.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:33.972+0000 7f58c4770640 1 -- 172.21.15.148:0/2985304096 wait complete. 2024-04-25T09:55:33.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:55:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.100+0000 7fcb70abc640 1 Processor -- start 2024-04-25T09:55:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb70abc640 1 -- start start 2024-04-25T09:55:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb70abc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcb6c0ce7b0 0x7fcb6c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb70abc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c06c8d0 con 0x7fcb6c1616f0 2024-04-25T09:55:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb70abc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0c6960 con 0x7fcb6c1610f0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb70abc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c1614d0 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb63fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb6c1616f0 0x7fcb6c0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37194/0 (socket says 172.21.15.148:37194) 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.101+0000 7fcb63fff640 1 -- 172.21.15.148:0/3094428992 learned_addr learned my addr 172.21.15.148:0/3094428992 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb6ad9d640 1 --2- 172.21.15.148:0/3094428992 >> v2:172.21.15.188:3300/0 conn(0x7fcb6c0ce7b0 0x7fcb6c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367497847 0 0) 0x7fcb6c06c8d0 con 0x7fcb6c1616f0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb58003520 con 0x7fcb6c1616f0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407672709 0 0) 0x7fcb6c0c6960 con 0x7fcb6c1610f0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb6c06c8d0 con 0x7fcb6c1610f0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb6ad9d640 1 -- 172.21.15.148:0/3094428992 >> v1:172.21.15.188:6789/0 conn(0x7fcb6c1610f0 legacy=0x7fcb6c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb6ad9d640 1 -- 172.21.15.148:0/3094428992 >> v1:172.21.15.148:6789/0 conn(0x7fcb6c1616f0 legacy=0x7fcb6c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.102+0000 7fcb6ad9d640 1 -- 172.21.15.148:0/3094428992 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c162260 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb6ad9d640 1 --2- 172.21.15.148:0/3094428992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c161b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009d90 tx=0x7fcb6402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c50b36bfb268d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64007740 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb640078e0 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb69d9b640 1 -- 172.21.15.148:0/3094428992 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64007c90 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 -- 172.21.15.148:0/3094428992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c0ce7b0 msgr2=0x7fcb6c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3094428992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c161b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009d90 tx=0x7fcb6402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 -- 172.21.15.148:0/3094428992 shutdown_connections 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3094428992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c161b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 -- 172.21.15.148:0/3094428992 >> 172.21.15.148:0/3094428992 conn(0x7fcb6c06f500 msgr2=0x7fcb6c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 -- 172.21.15.148:0/3094428992 shutdown_connections 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.103+0000 7fcb70abc640 1 -- 172.21.15.148:0/3094428992 wait complete. 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 Processor -- start 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 -- start start 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c1610f0 0x7fcb6c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb6c1616f0 0x7fcb6c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0cfce0 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0cfa00 con 0x7fcb6c1610f0 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.105+0000 7fcb70abc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0cf690 con 0x7fcb6c1616f0 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c0b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58452/0 (socket says 172.21.15.148:58452) 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 -- 172.21.15.148:0/3145997671 learned_addr learned my addr 172.21.15.148:0/3145997671 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb63fff640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c1610f0 0x7fcb6c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6ad9d640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb6c1616f0 0x7fcb6c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 -- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb6c1616f0 msgr2=0x7fcb6c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb6c1616f0 0x7fcb6c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 -- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c1610f0 msgr2=0x7fcb6c1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c1610f0 0x7fcb6c1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 -- 172.21.15.148:0/3145997671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c158b80 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6ad9d640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb6c1616f0 0x7fcb6c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb6a59c640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c0b64b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c500 tx=0x7fcb5c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0052d0 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c005490 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c012870 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c158e20 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.106+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c0b6d70 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.107+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c073a00 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.108+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb5c012a10 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.108+0000 7fcb62ffd640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb4c06b4d0 0x7fcb4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:34.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.108+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fcb5c08a020 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.108+0000 7fcb63fff640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb4c06b4d0 0x7fcb4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:34.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.109+0000 7fcb63fff640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb4c06b4d0 0x7fcb4c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0d0bd0 tx=0x7fcb54009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:34.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.112+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb5c058d80 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.318+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcb6c0cf150 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.970+0000 7fcb62ffd640 1 -- 172.21.15.148:0/3145997671 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcb5c058680 con 0x7fcb6c0ce7b0 2024-04-25T09:55:34.973 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb4c06b4d0 msgr2=0x7fcb4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb4c06b4d0 0x7fcb4c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0d0bd0 tx=0x7fcb54009040 comp rx=0 tx=0).stop 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 msgr2=0x7fcb6c0b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c0b64b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c500 tx=0x7fcb5c00c9c0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 shutdown_connections 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb4c06b4d0 0x7fcb4c06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb6c1616f0 0x7fcb6c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb6c1610f0 0x7fcb6c1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 --2- 172.21.15.148:0/3145997671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb6c0ce7b0 0x7fcb6c0b64b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:34.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 >> 172.21.15.148:0/3145997671 conn(0x7fcb6c06f500 msgr2=0x7fcb6c0c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:34.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 shutdown_connections 2024-04-25T09:55:34.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:34.973+0000 7fcb70abc640 1 -- 172.21.15.148:0/3145997671 wait complete. 2024-04-25T09:55:35.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:35.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.103+0000 7f5db623f640 1 Processor -- start 2024-04-25T09:55:35.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.103+0000 7f5db623f640 1 -- start start 2024-04-25T09:55:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5db623f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5db006abe0 0x7f5db006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5db623f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5db00ce330 0x7f5db006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5db623f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db01626e0 con 0x7f5db00700a0 2024-04-25T09:55:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5db623f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db006b500 con 0x7f5db006abe0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5db623f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db00cd270 con 0x7f5db00ce330 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5db006abe0 0x7f5db006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5db006abe0 0x7f5db006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58466/0 (socket says 172.21.15.148:58466) 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 -- 172.21.15.148:0/221370081 learned_addr learned my addr 172.21.15.148:0/221370081 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daeffd640 1 --2- 172.21.15.148:0/221370081 >> v2:172.21.15.188:3300/0 conn(0x7f5db00ce330 0x7f5db006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 -- 172.21.15.148:0/221370081 >> v2:172.21.15.188:3300/0 conn(0x7f5db00ce330 msgr2=0x7f5db006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 --2- 172.21.15.148:0/221370081 >> v2:172.21.15.188:3300/0 conn(0x7f5db00ce330 0x7f5db006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 -- 172.21.15.148:0/221370081 >> v1:172.21.15.188:6789/0 conn(0x7f5db00700a0 legacy=0x7f5db006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.104+0000 7f5daf7fe640 1 -- 172.21.15.148:0/221370081 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db00cd550 con 0x7f5db006abe0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.105+0000 7f5daf7fe640 1 --2- 172.21.15.148:0/221370081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db006abe0 0x7f5db006afc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5db006bbf0 tx=0x7f5da002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ae33cb80dfe55d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.105+0000 7f5dae7fc640 1 -- 172.21.15.148:0/221370081 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0037070 con 0x7f5db006abe0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.105+0000 7f5dae7fc640 1 -- 172.21.15.148:0/221370081 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da0032440 con 0x7f5db006abe0 2024-04-25T09:55:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.105+0000 7f5dae7fc640 1 -- 172.21.15.148:0/221370081 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0032810 con 0x7f5db006abe0 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 -- 172.21.15.148:0/221370081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db006abe0 msgr2=0x7f5db006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 --2- 172.21.15.148:0/221370081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db006abe0 0x7f5db006afc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5db006bbf0 tx=0x7f5da002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 -- 172.21.15.148:0/221370081 shutdown_connections 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 --2- 172.21.15.148:0/221370081 >> v2:172.21.15.188:3300/0 conn(0x7f5db00ce330 0x7f5db006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 --2- 172.21.15.148:0/221370081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db006abe0 0x7f5db006afc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 -- 172.21.15.148:0/221370081 >> 172.21.15.148:0/221370081 conn(0x7f5db00b2220 msgr2=0x7f5db00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 -- 172.21.15.148:0/221370081 shutdown_connections 2024-04-25T09:55:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.106+0000 7f5db623f640 1 -- 172.21.15.148:0/221370081 wait complete. 2024-04-25T09:55:35.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.107+0000 7f5db623f640 1 Processor -- start 2024-04-25T09:55:35.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.107+0000 7f5db623f640 1 -- start start 2024-04-25T09:55:35.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db006abe0 0x7f5db00aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db00700a0 0x7f5db01f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 0x7f5db00b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db0161780 con 0x7f5db00ce330 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db00cdb10 con 0x7f5db00700a0 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db006eb00 con 0x7f5db006abe0 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 0x7f5db00b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 0x7f5db00b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58476/0 (socket says 172.21.15.148:58476) 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 -- 172.21.15.148:0/2077284340 learned_addr learned my addr 172.21.15.148:0/2077284340 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 -- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db006abe0 msgr2=0x7f5db00aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db006abe0 0x7f5db00aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daeffd640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db00700a0 0x7f5db01f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 -- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db00700a0 msgr2=0x7f5db01f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db00700a0 0x7f5db01f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 -- 172.21.15.148:0/2077284340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db015ef20 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5daffff640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 0x7f5db00b1440 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5da800c7a0 tx=0x7f5da800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8003b70 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da8003d30 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db015f200 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da800aa90 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.108+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db00af470 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.109+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db0073990 con 0x7f5db00ce330 2024-04-25T09:55:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.110+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5da800ac30 con 0x7f5db00ce330 2024-04-25T09:55:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.110+0000 7f5d97fff640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d9c06b4d0 0x7f5d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.110+0000 7f5daf7fe640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d9c06b4d0 0x7f5d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.110+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5da808a250 con 0x7f5db00ce330 2024-04-25T09:55:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.111+0000 7f5daf7fe640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d9c06b4d0 0x7f5d9c06d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5da0034000 tx=0x7f5da002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.115+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5da8058fd0 con 0x7f5db00ce330 2024-04-25T09:55:35.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.285+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5db00b2ff0 con 0x7f5db00ce330 2024-04-25T09:55:35.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.286+0000 7f5d97fff640 1 -- 172.21.15.148:0/2077284340 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f5da80588d0 con 0x7f5db00ce330 2024-04-25T09:55:35.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:35.290 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":38,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:34.319496+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.288+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d9c06b4d0 msgr2=0x7f5d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.288+0000 7f5db623f640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d9c06b4d0 0x7f5d9c06d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5da0034000 tx=0x7f5da002c000 comp rx=0 tx=0).stop 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.288+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 msgr2=0x7f5db00b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.288+0000 7f5db623f640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 0x7f5db00b1440 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5da800c7a0 tx=0x7f5da800cc60 comp rx=0 tx=0).stop 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 shutdown_connections 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d9c06b4d0 0x7f5d9c06d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5db00ce330 0x7f5db00b1440 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5db00700a0 0x7f5db01f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 --2- 172.21.15.148:0/2077284340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5db006abe0 0x7f5db00aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 >> 172.21.15.148:0/2077284340 conn(0x7f5db00b2220 msgr2=0x7f5db0159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 shutdown_connections 2024-04-25T09:55:35.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.289+0000 7f5db623f640 1 -- 172.21.15.148:0/2077284340 wait complete. 2024-04-25T09:55:35.303 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.396+0000 7f8234d69640 1 Processor -- start 2024-04-25T09:55:35.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.396+0000 7f8234d69640 1 -- start start 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f8234d69640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8230154ee0 0x7f82300cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f8234d69640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82300cfcf0 0x7f82300cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f8234d69640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8230071480 con 0x7f82300ce3c0 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f8234d69640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82300ce830 con 0x7f8230154ee0 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f8234d69640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8230072090 con 0x7f82300cfcf0 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f822e59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8230154ee0 0x7f82300cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f822e59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8230154ee0 0x7f82300cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58488/0 (socket says 172.21.15.148:58488) 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f822e59c640 1 -- 172.21.15.148:0/4225152047 learned_addr learned my addr 172.21.15.148:0/4225152047 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.397+0000 7f822dd9b640 1 --2- 172.21.15.148:0/4225152047 >> v2:172.21.15.188:3300/0 conn(0x7f82300cfcf0 0x7f82300cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822e59c640 1 -- 172.21.15.148:0/4225152047 >> v2:172.21.15.188:3300/0 conn(0x7f82300cfcf0 msgr2=0x7f82300cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822e59c640 1 --2- 172.21.15.148:0/4225152047 >> v2:172.21.15.188:3300/0 conn(0x7f82300cfcf0 0x7f82300cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822e59c640 1 -- 172.21.15.148:0/4225152047 >> v1:172.21.15.188:6789/0 conn(0x7f82300ce3c0 legacy=0x7f823015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822e59c640 1 -- 172.21.15.148:0/4225152047 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8230071760 con 0x7f8230154ee0 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822e59c640 1 --2- 172.21.15.148:0/4225152047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8230154ee0 0x7f82300cf7b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f82300cf560 tx=0x7f822002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af6d5c05abc0aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822d59a640 1 -- 172.21.15.148:0/4225152047 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220036070 con 0x7f8230154ee0 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822d59a640 1 -- 172.21.15.148:0/4225152047 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8220031600 con 0x7f8230154ee0 2024-04-25T09:55:35.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.398+0000 7f822d59a640 1 -- 172.21.15.148:0/4225152047 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200319d0 con 0x7f8230154ee0 2024-04-25T09:55:35.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 -- 172.21.15.148:0/4225152047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8230154ee0 msgr2=0x7f82300cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 --2- 172.21.15.148:0/4225152047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8230154ee0 0x7f82300cf7b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f82300cf560 tx=0x7f822002a6f0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 -- 172.21.15.148:0/4225152047 shutdown_connections 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 --2- 172.21.15.148:0/4225152047 >> v2:172.21.15.188:3300/0 conn(0x7f82300cfcf0 0x7f82300cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 --2- 172.21.15.148:0/4225152047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8230154ee0 0x7f82300cf7b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 -- 172.21.15.148:0/4225152047 >> 172.21.15.148:0/4225152047 conn(0x7f823006abe0 msgr2=0x7f823006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 -- 172.21.15.148:0/4225152047 shutdown_connections 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.399+0000 7f8234d69640 1 -- 172.21.15.148:0/4225152047 wait complete. 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.400+0000 7f8234d69640 1 Processor -- start 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.400+0000 7f8234d69640 1 -- start start 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f8234d69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 0x7f8230141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f8234d69640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300cfcf0 0x7f8230144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f8234d69640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8230154ee0 0x7f82301f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f8234d69640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230071d20 con 0x7f82300ce3c0 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f8234d69640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230071230 con 0x7f8230154ee0 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f8234d69640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82301552c0 con 0x7f82300cfcf0 2024-04-25T09:55:35.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 0x7f8230141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 0x7f8230141c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58502/0 (socket says 172.21.15.148:58502) 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 -- 172.21.15.148:0/468075557 learned_addr learned my addr 172.21.15.148:0/468075557 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 -- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300cfcf0 msgr2=0x7f8230144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822dd9b640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300cfcf0 0x7f8230144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822ed9d640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8230154ee0 0x7f82301f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300cfcf0 0x7f8230144d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 -- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8230154ee0 msgr2=0x7f82301f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8230154ee0 0x7f82301f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.401+0000 7f822e59c640 1 -- 172.21.15.148:0/468075557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82301f1a90 con 0x7f82300ce3c0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.402+0000 7f822e59c640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 0x7f8230141c00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8220009f40 tx=0x7f8220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.402+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220009510 con 0x7f82300ce3c0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.402+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8220007a10 con 0x7f82300ce3c0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.402+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82301f14d0 con 0x7f82300ce3c0 2024-04-25T09:55:35.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.402+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220007e00 con 0x7f82300ce3c0 2024-04-25T09:55:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.402+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8230144570 con 0x7f82300ce3c0 2024-04-25T09:55:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.403+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8230073990 con 0x7f82300ce3c0 2024-04-25T09:55:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.403+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8220039400 con 0x7f82300ce3c0 2024-04-25T09:55:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.404+0000 7f821effd640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f821006b4d0 0x7f821006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.404+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82200ad180 con 0x7f82300ce3c0 2024-04-25T09:55:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.404+0000 7f822dd9b640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f821006b4d0 0x7f821006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.404+0000 7f822dd9b640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f821006b4d0 0x7f821006d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f823015db90 tx=0x7f8224008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.408+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f822007bf00 con 0x7f82300ce3c0 2024-04-25T09:55:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.577+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8230068070 con 0x7f82300ce3c0 2024-04-25T09:55:35.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.578+0000 7f821effd640 1 -- 172.21.15.148:0/468075557 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f822007b800 con 0x7f82300ce3c0 2024-04-25T09:55:35.581 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:35.581 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":38,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:34.319496+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:35.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f821006b4d0 msgr2=0x7f821006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f821006b4d0 0x7f821006d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f823015db90 tx=0x7f8224008040 comp rx=0 tx=0).stop 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 msgr2=0x7f8230141c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 0x7f8230141c00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8220009f40 tx=0x7f8220004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 shutdown_connections 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f821006b4d0 0x7f821006d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8230154ee0 0x7f82301f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300cfcf0 0x7f8230144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 --2- 172.21.15.148:0/468075557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82300ce3c0 0x7f8230141c00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 >> 172.21.15.148:0/468075557 conn(0x7f823006abe0 msgr2=0x7f823015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 shutdown_connections 2024-04-25T09:55:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.580+0000 7f8234d69640 1 -- 172.21.15.148:0/468075557 wait complete. 2024-04-25T09:55:35.594 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:55:35.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.691+0000 7ff048302640 1 Processor -- start 2024-04-25T09:55:35.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.691+0000 7ff048302640 1 -- start start 2024-04-25T09:55:35.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff048302640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0400cc4f0 0x7ff0400cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff048302640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff04015b980 0x7ff0400cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff048302640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff040072280 con 0x7ff0400cdb20 2024-04-25T09:55:35.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff048302640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff040071270 con 0x7ff04015b980 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff048302640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04015bd80 con 0x7ff0400cc4f0 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff04589d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0400cdb20 0x7ff04015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37206/0 (socket says 172.21.15.148:37206) 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff04689f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff04015b980 0x7ff0400cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.692+0000 7ff04589d640 1 -- 172.21.15.148:0/911016075 learned_addr learned my addr 172.21.15.148:0/911016075 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04509c640 1 -- 172.21.15.148:0/911016075 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261323823 0 0) 0x7ff040072280 con 0x7ff0400cdb20 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04609e640 1 --2- 172.21.15.148:0/911016075 >> v2:172.21.15.188:3300/0 conn(0x7ff0400cc4f0 0x7ff0400cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04509c640 1 -- 172.21.15.148:0/911016075 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff02c003520 con 0x7ff0400cdb20 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04689f640 1 -- 172.21.15.148:0/911016075 >> v2:172.21.15.188:3300/0 conn(0x7ff0400cc4f0 msgr2=0x7ff0400cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04689f640 1 --2- 172.21.15.148:0/911016075 >> v2:172.21.15.188:3300/0 conn(0x7ff0400cc4f0 0x7ff0400cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04689f640 1 -- 172.21.15.148:0/911016075 >> v1:172.21.15.148:6789/0 conn(0x7ff0400cdb20 legacy=0x7ff04015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04689f640 1 -- 172.21.15.148:0/911016075 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0400714c0 con 0x7ff04015b980 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04609e640 1 --2- 172.21.15.148:0/911016075 >> v2:172.21.15.188:3300/0 conn(0x7ff0400cc4f0 0x7ff0400cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.693+0000 7ff04689f640 1 --2- 172.21.15.148:0/911016075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff04015b980 0x7ff0400cdfb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff0400cd290 tx=0x7ff03802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2452a9fc4ce1bf66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff04509c640 1 -- 172.21.15.148:0/911016075 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038037070 con 0x7ff04015b980 2024-04-25T09:55:35.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff04509c640 1 -- 172.21.15.148:0/911016075 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038032440 con 0x7ff04015b980 2024-04-25T09:55:35.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff04509c640 1 -- 172.21.15.148:0/911016075 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038032810 con 0x7ff04015b980 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 -- 172.21.15.148:0/911016075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff04015b980 msgr2=0x7ff0400cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 --2- 172.21.15.148:0/911016075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff04015b980 0x7ff0400cdfb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff0400cd290 tx=0x7ff03802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 -- 172.21.15.148:0/911016075 shutdown_connections 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 --2- 172.21.15.148:0/911016075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff04015b980 0x7ff0400cdfb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 --2- 172.21.15.148:0/911016075 >> v2:172.21.15.188:3300/0 conn(0x7ff0400cc4f0 0x7ff0400cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 -- 172.21.15.148:0/911016075 >> 172.21.15.148:0/911016075 conn(0x7ff04006abe0 msgr2=0x7ff04006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 -- 172.21.15.148:0/911016075 shutdown_connections 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.694+0000 7ff048302640 1 -- 172.21.15.148:0/911016075 wait complete. 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.695+0000 7ff048302640 1 Processor -- start 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.695+0000 7ff048302640 1 -- start start 2024-04-25T09:55:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff048302640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 0x7ff0401478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff048302640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0400cdb20 0x7ff040159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff048302640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04015b980 0x7ff04015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff048302640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040071d60 con 0x7ff0400cc4f0 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff048302640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040071a80 con 0x7ff0400cdb20 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff048302640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400ae500 con 0x7ff04015b980 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 0x7ff0401478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 0x7ff0401478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58524/0 (socket says 172.21.15.148:58524) 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 -- 172.21.15.148:0/1162110192 learned_addr learned my addr 172.21.15.148:0/1162110192 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04689f640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04015b980 0x7ff04015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04589d640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0400cdb20 0x7ff040159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 -- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04015b980 msgr2=0x7ff04015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04015b980 0x7ff04015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 -- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0400cdb20 msgr2=0x7ff040159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0400cdb20 0x7ff040159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.696+0000 7ff04609e640 1 -- 172.21.15.148:0/1162110192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0401f3980 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.697+0000 7ff04609e640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 0x7ff0401478d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00ccb0 tx=0x7ff03c00a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.697+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c016070 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.697+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c00f600 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.697+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00f9f0 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.697+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0401f7aa0 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.697+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0400ac9e0 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.698+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff040073a00 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.698+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff03c00fb90 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.698+0000 7ff036ffd640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff02406b4d0 0x7ff02406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.699+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff03c08a890 con 0x7ff0400cc4f0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.699+0000 7ff04589d640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff02406b4d0 0x7ff02406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.699+0000 7ff04589d640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff02406b4d0 0x7ff02406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0400cc050 tx=0x7ff030009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:35.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.703+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff03c059610 con 0x7ff0400cc4f0 2024-04-25T09:55:35.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.907+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff04006ee40 con 0x7ff0400cc4f0 2024-04-25T09:55:35.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.984+0000 7ff036ffd640 1 -- 172.21.15.148:0/1162110192 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff03c058f10 con 0x7ff0400cc4f0 2024-04-25T09:55:35.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff02406b4d0 msgr2=0x7ff02406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff02406b4d0 0x7ff02406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0400cc050 tx=0x7ff030009040 comp rx=0 tx=0).stop 2024-04-25T09:55:35.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 msgr2=0x7ff0401478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:35.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 0x7ff0401478d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00ccb0 tx=0x7ff03c00a5d0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 shutdown_connections 2024-04-25T09:55:35.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff02406b4d0 0x7ff02406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff04015b980 0x7ff04015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0400cdb20 0x7ff040159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 --2- 172.21.15.148:0/1162110192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0400cc4f0 0x7ff0401478d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:35.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 >> 172.21.15.148:0/1162110192 conn(0x7ff04006abe0 msgr2=0x7ff04015b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:35.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 shutdown_connections 2024-04-25T09:55:35.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:35.987+0000 7ff048302640 1 -- 172.21.15.148:0/1162110192 wait complete. 2024-04-25T09:55:36.000 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:55:36.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:55:36.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.099+0000 7f21e9afc640 1 Processor -- start 2024-04-25T09:55:36.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.099+0000 7f21e9afc640 1 -- start start 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e9afc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21e40d34c0 0x7f21e40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e9afc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e40d08e0 0x7f21e40b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e9afc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40d2420 con 0x7f21e415dac0 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e9afc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40d3030 con 0x7f21e40d34c0 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e9afc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40d0d20 con 0x7f21e40d08e0 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21e40d34c0 0x7f21e40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f21e415dac0 0x7f21e40d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37208/0 (socket says 172.21.15.148:37208) 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e37fe640 1 -- 172.21.15.148:0/3331970974 learned_addr learned my addr 172.21.15.148:0/3331970974 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e3fff640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e40d08e0 0x7f21e40b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e2ffd640 1 -- 172.21.15.148:0/3331970974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e40d08e0 msgr2=0x7f21e40b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e2ffd640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e40d08e0 0x7f21e40b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e2ffd640 1 -- 172.21.15.148:0/3331970974 >> v1:172.21.15.148:6789/0 conn(0x7f21e415dac0 legacy=0x7f21e40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e27fc640 1 -- 172.21.15.148:0/3331970974 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828037970 0 0) 0x7f21e40d2420 con 0x7f21e415dac0 2024-04-25T09:55:36.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e2ffd640 1 -- 172.21.15.148:0/3331970974 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40d2700 con 0x7f21e40d34c0 2024-04-25T09:55:36.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.100+0000 7f21e3fff640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e40d08e0 0x7f21e40b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:36.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.101+0000 7f21e2ffd640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d34c0 0x7f21e40d03a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009790 tx=0x7f21d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c59aad1745e1ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.101+0000 7f21e27fc640 1 -- 172.21.15.148:0/3331970974 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8037070 con 0x7f21e40d34c0 2024-04-25T09:55:36.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.101+0000 7f21e27fc640 1 -- 172.21.15.148:0/3331970974 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8032440 con 0x7f21e40d34c0 2024-04-25T09:55:36.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.101+0000 7f21e27fc640 1 -- 172.21.15.148:0/3331970974 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8032810 con 0x7f21e40d34c0 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 -- 172.21.15.148:0/3331970974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d34c0 msgr2=0x7f21e40d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d34c0 0x7f21e40d03a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009790 tx=0x7f21d802a210 comp rx=0 tx=0).stop 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 -- 172.21.15.148:0/3331970974 shutdown_connections 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e40d08e0 0x7f21e40b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 --2- 172.21.15.148:0/3331970974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d34c0 0x7f21e40d03a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 -- 172.21.15.148:0/3331970974 >> 172.21.15.148:0/3331970974 conn(0x7f21e4061c30 msgr2=0x7f21e406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 -- 172.21.15.148:0/3331970974 shutdown_connections 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.102+0000 7f21e9afc640 1 -- 172.21.15.148:0/3331970974 wait complete. 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.103+0000 7f21e9afc640 1 Processor -- start 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.103+0000 7f21e9afc640 1 -- start start 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 0x7f21e4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e40d34c0 0x7f21e40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e415dac0 0x7f21e40b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40d2100 con 0x7f21e40d08e0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40d2cc0 con 0x7f21e40d34c0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e415dea0 con 0x7f21e415dac0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 0x7f21e4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 0x7f21e4154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58540/0 (socket says 172.21.15.148:58540) 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 -- 172.21.15.148:0/1629749606 learned_addr learned my addr 172.21.15.148:0/1629749606 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e3fff640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e415dac0 0x7f21e40b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 -- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e415dac0 msgr2=0x7f21e40b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e415dac0 0x7f21e40b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 -- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e40d34c0 msgr2=0x7f21e40b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e2ffd640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e40d34c0 0x7f21e40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e40d34c0 0x7f21e40b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 -- 172.21.15.148:0/1629749606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40c9740 con 0x7f21e40d08e0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e37fe640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 0x7f21e4154550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c460 tx=0x7f21d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4018070 con 0x7f21e40d08e0 2024-04-25T09:55:36.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d4007500 con 0x7f21e40d08e0 2024-04-25T09:55:36.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e40c9a20 con 0x7f21e40d08e0 2024-04-25T09:55:36.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4004a20 con 0x7f21e40d08e0 2024-04-25T09:55:36.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.104+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e40b4310 con 0x7f21e40d08e0 2024-04-25T09:55:36.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.105+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e4073a00 con 0x7f21e40d08e0 2024-04-25T09:55:36.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.106+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21d401b070 con 0x7f21e40d08e0 2024-04-25T09:55:36.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.106+0000 7f21cbfff640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21d006b500 0x7f21d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:36.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.106+0000 7f21e2ffd640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21d006b500 0x7f21d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:36.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.106+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21d4089670 con 0x7f21e40d08e0 2024-04-25T09:55:36.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.107+0000 7f21e2ffd640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21d006b500 0x7f21d006d9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f21d8034000 tx=0x7f21d802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:36.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.111+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21d40582d0 con 0x7f21e40d08e0 2024-04-25T09:55:36.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.322+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f21e40b4d70 con 0x7f21e40d08e0 2024-04-25T09:55:36.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.986+0000 7f21cbfff640 1 -- 172.21.15.148:0/1629749606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21d4057bd0 con 0x7f21e40d08e0 2024-04-25T09:55:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21d006b500 msgr2=0x7f21d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21d006b500 0x7f21d006d9c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f21d8034000 tx=0x7f21d802e000 comp rx=0 tx=0).stop 2024-04-25T09:55:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 msgr2=0x7f21e4154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 0x7f21e4154550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c460 tx=0x7f21d400c920 comp rx=0 tx=0).stop 2024-04-25T09:55:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 shutdown_connections 2024-04-25T09:55:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21d006b500 0x7f21d006d9c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e415dac0 0x7f21e40b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e40d34c0 0x7f21e40b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 --2- 172.21.15.148:0/1629749606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21e40d08e0 0x7f21e4154550 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 >> 172.21.15.148:0/1629749606 conn(0x7f21e4061c30 msgr2=0x7f21e40b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 shutdown_connections 2024-04-25T09:55:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:36.989+0000 7f21e9afc640 1 -- 172.21.15.148:0/1629749606 wait complete. 2024-04-25T09:55:37.004 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:55:37.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.101+0000 7fc7cac6f640 1 Processor -- start 2024-04-25T09:55:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.101+0000 7fc7cac6f640 1 -- start start 2024-04-25T09:55:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.101+0000 7fc7cac6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 0x7fc7c40d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.101+0000 7fc7cac6f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c40b9940 con 0x7fc7c415dd30 2024-04-25T09:55:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.101+0000 7fc7cac6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c40d1df0 con 0x7fc7c40d0900 2024-04-25T09:55:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.101+0000 7fc7cac6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40d0d70 con 0x7fc7c40d2e90 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c8a0b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc7c415dd30 0x7fc7c40b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37222/0 (socket says 172.21.15.148:37222) 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c8a0b640 1 -- 172.21.15.148:0/1517206419 learned_addr learned my addr 172.21.15.148:0/1517206419 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968943126 0 0) 0x7fc7c40b9940 con 0x7fc7c415dd30 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b4003520 con 0x7fc7c415dd30 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 0x7fc7c40d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1585689194 0 0) 0x7fc7b4003520 con 0x7fc7c415dd30 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 msgr2=0x7fc7c40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 0x7fc7c40d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 >> v1:172.21.15.188:6789/0 conn(0x7fc7c40d0900 legacy=0x7fc7c40b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40d20d0 con 0x7fc7c415dd30 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc7b8001670 con 0x7fc7c415dd30 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 >> v1:172.21.15.148:6789/0 conn(0x7fc7c415dd30 legacy=0x7fc7c40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 0x7fc7b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004010 0x7fc7b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b4004d40 0x7fc7b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40d0d70 con 0x7fc7b4004d40 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b400bc00 con 0x7fc7b4004010 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b400be00 con 0x7fc7c40d2e90 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c920c640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b4004d40 0x7fc7b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004010 0x7fc7b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.102+0000 7fc7c8a0b640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 0x7fc7b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c920c640 1 -- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 msgr2=0x7fc7b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c920c640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c40d2e90 0x7fc7b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c920c640 1 -- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004010 msgr2=0x7fc7b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c920c640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004010 0x7fc7b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c920c640 1 -- 172.21.15.148:0/1517206419 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4003520 con 0x7fc7b4004d40 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c920c640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b4004d40 0x7fc7b40051c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc001e50 tx=0x7fc7b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba2aacc1a6d1726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc001a70 con 0x7fc7b4004d40 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc003970 con 0x7fc7b4004d40 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.103+0000 7fc7c37fe640 1 -- 172.21.15.148:0/1517206419 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc003d60 con 0x7fc7b4004d40 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7cac6f640 1 -- 172.21.15.148:0/1517206419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b4004d40 msgr2=0x7fc7b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/1517206419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b4004d40 0x7fc7b40051c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc001e50 tx=0x7fc7b4005830 comp rx=0 tx=0).stop 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7c920c640 1 -- 172.21.15.148:0/1517206419 reap_dead start 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7cac6f640 1 -- 172.21.15.148:0/1517206419 shutdown_connections 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7cac6f640 1 -- 172.21.15.148:0/1517206419 >> 172.21.15.148:0/1517206419 conn(0x7fc7c4061c30 msgr2=0x7fc7c406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7cac6f640 1 -- 172.21.15.148:0/1517206419 shutdown_connections 2024-04-25T09:55:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.104+0000 7fc7cac6f640 1 -- 172.21.15.148:0/1517206419 wait complete. 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.105+0000 7fc7cac6f640 1 Processor -- start 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.105+0000 7fc7cac6f640 1 -- start start 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7cac6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004d40 0x7fc7c4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7cac6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c4160460 0x7fc7c415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7cac6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 0x7fc7c415b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7cac6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40d2970 con 0x7fc7c4156a30 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7cac6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40d2690 con 0x7fc7b4004d40 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7cac6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40d1a10 con 0x7fc7c4160460 2024-04-25T09:55:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 0x7fc7c415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 0x7fc7c415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58546/0 (socket says 172.21.15.148:58546) 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 -- 172.21.15.148:0/3288303527 learned_addr learned my addr 172.21.15.148:0/3288303527 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c8a0b640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004d40 0x7fc7c4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c4160460 0x7fc7c415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 -- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c4160460 msgr2=0x7fc7c415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c4160460 0x7fc7c415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 -- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004d40 msgr2=0x7fc7c4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.106+0000 7fc7c920c640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004d40 0x7fc7c4157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7c920c640 1 -- 172.21.15.148:0/3288303527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40b43b0 con 0x7fc7c4156a30 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c4160460 0x7fc7c415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7c920c640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 0x7fc7c415b280 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc001f70 tx=0x7fc7bc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc037040 con 0x7fc7c4156a30 2024-04-25T09:55:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc003e50 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c40b4690 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc007db0 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.107+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c415ffe0 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.108+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c4073990 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.108+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7bc042020 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.109+0000 7fc7c17fa640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7a806b500 0x7fc7a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.109+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7bc0a9a20 con 0x7fc7c4156a30 2024-04-25T09:55:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.109+0000 7fc7c8a0b640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7a806b500 0x7fc7a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.109+0000 7fc7c8a0b640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7a806b500 0x7fc7a806d9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8007cd0 tx=0x7fc7b8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.113+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7bc078760 con 0x7fc7c4156a30 2024-04-25T09:55:37.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.321+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7c40c76d0 con 0x7fc7c4156a30 2024-04-25T09:55:37.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.322+0000 7fc7c17fa640 1 -- 172.21.15.148:0/3288303527 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fc7bc078060 con 0x7fc7c4156a30 2024-04-25T09:55:37.325 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:37.325 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:34.319815+0000","modified":"2024-04-25T09:55:36.324069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/3412433349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3412433349},{"type":"v1","addr":"172.21.15.148:6836","nonce":3412433349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:55:37.325 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 16 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7a806b500 msgr2=0x7fc7a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7a806b500 0x7fc7a806d9c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8007cd0 tx=0x7fc7b8009040 comp rx=0 tx=0).stop 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 msgr2=0x7fc7c415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 0x7fc7c415b280 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc001f70 tx=0x7fc7bc0014a0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 shutdown_connections 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7a806b500 0x7fc7a806d9c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7c4156a30 0x7fc7c415b280 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c4160460 0x7fc7c415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 --2- 172.21.15.148:0/3288303527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b4004d40 0x7fc7c4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 >> 172.21.15.148:0/3288303527 conn(0x7fc7c4061c30 msgr2=0x7fc7c40cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 shutdown_connections 2024-04-25T09:55:37.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.324+0000 7fc7cac6f640 1 -- 172.21.15.148:0/3288303527 wait complete. 2024-04-25T09:55:37.338 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:55:37.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.431+0000 7f9a3a7b7640 1 Processor -- start 2024-04-25T09:55:37.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.431+0000 7f9a3a7b7640 1 -- start start 2024-04-25T09:55:37.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a3a7b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a340b2cb0 0x7f9a340d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a3a7b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a340d1020 0x7f9a340d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a3a7b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a340b97a0 con 0x7f9a3415dbe0 2024-04-25T09:55:37.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a3a7b7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3415dfc0 con 0x7f9a340d1020 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a3a7b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a340d1a00 con 0x7f9a340b2cb0 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a38d54640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a340d1020 0x7f9a340d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a33fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9a3415dbe0 0x7f9a340d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37234/0 (socket says 172.21.15.148:37234) 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.432+0000 7f9a33fff640 1 -- 172.21.15.148:0/3173900512 learned_addr learned my addr 172.21.15.148:0/3173900512 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a337fe640 1 --2- 172.21.15.148:0/3173900512 >> v2:172.21.15.188:3300/0 conn(0x7f9a340b2cb0 0x7f9a340d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a38d54640 1 -- 172.21.15.148:0/3173900512 >> v2:172.21.15.188:3300/0 conn(0x7f9a340b2cb0 msgr2=0x7f9a340d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a38d54640 1 --2- 172.21.15.148:0/3173900512 >> v2:172.21.15.188:3300/0 conn(0x7f9a340b2cb0 0x7f9a340d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a38d54640 1 -- 172.21.15.148:0/3173900512 >> v1:172.21.15.148:6789/0 conn(0x7f9a3415dbe0 legacy=0x7f9a340d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a32ffd640 1 -- 172.21.15.148:0/3173900512 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743402650 0 0) 0x7f9a340b97a0 con 0x7f9a3415dbe0 2024-04-25T09:55:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a38d54640 1 -- 172.21.15.148:0/3173900512 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340d1c50 con 0x7f9a340d1020 2024-04-25T09:55:37.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a38d54640 1 --2- 172.21.15.148:0/3173900512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340d1020 0x7f9a340d2c70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9a3406cff0 tx=0x7f9a2c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeff685f483fcacd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.433+0000 7f9a32ffd640 1 -- 172.21.15.148:0/3173900512 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c007740 con 0x7f9a340d1020 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a32ffd640 1 -- 172.21.15.148:0/3173900512 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c0078e0 con 0x7f9a340d1020 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a32ffd640 1 -- 172.21.15.148:0/3173900512 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c007cb0 con 0x7f9a340d1020 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/3173900512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340d1020 msgr2=0x7f9a340d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/3173900512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340d1020 0x7f9a340d2c70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9a3406cff0 tx=0x7f9a2c02a680 comp rx=0 tx=0).stop 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/3173900512 shutdown_connections 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/3173900512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340d1020 0x7f9a340d2c70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/3173900512 >> v2:172.21.15.188:3300/0 conn(0x7f9a340b2cb0 0x7f9a340d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/3173900512 >> 172.21.15.148:0/3173900512 conn(0x7f9a34061a20 msgr2=0x7f9a34061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/3173900512 shutdown_connections 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.434+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/3173900512 wait complete. 2024-04-25T09:55:37.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.435+0000 7f9a3a7b7640 1 Processor -- start 2024-04-25T09:55:37.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.435+0000 7f9a3a7b7640 1 -- start start 2024-04-25T09:55:37.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a3a7b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b2cb0 0x7f9a341533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a3a7b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340d1020 0x7f9a341538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a3a7b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a340b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a3a7b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340b92a0 con 0x7f9a3415dbe0 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a3a7b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340d2210 con 0x7f9a340b2cb0 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a3a7b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340b9420 con 0x7f9a340d1020 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a340b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a340b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58552/0 (socket says 172.21.15.148:58552) 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 -- 172.21.15.148:0/4214049333 learned_addr learned my addr 172.21.15.148:0/4214049333 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a33fff640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b2cb0 0x7f9a341533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a337fe640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340d1020 0x7f9a341538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 -- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340d1020 msgr2=0x7f9a341538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340d1020 0x7f9a341538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 -- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b2cb0 msgr2=0x7f9a341533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.436+0000 7f9a38d54640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b2cb0 0x7f9a341533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a38d54640 1 -- 172.21.15.148:0/4214049333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a34160680 con 0x7f9a3415dbe0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a33fff640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b2cb0 0x7f9a341533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a337fe640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340d1020 0x7f9a341538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a38d54640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a340b4370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009e80 tx=0x7f9a2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c040070 con 0x7f9a3415dbe0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c031c40 con 0x7f9a3415dbe0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a341f7a00 con 0x7f9a3415dbe0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c003990 con 0x7f9a3415dbe0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.437+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3415e710 con 0x7f9a3415dbe0 2024-04-25T09:55:37.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.438+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a34073990 con 0x7f9a3415dbe0 2024-04-25T09:55:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.438+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a2c039040 con 0x7f9a3415dbe0 2024-04-25T09:55:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.439+0000 7f9a30ff9640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a2006b4d0 0x7f9a2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.439+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a2c0ab410 con 0x7f9a3415dbe0 2024-04-25T09:55:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.439+0000 7f9a33fff640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a2006b4d0 0x7f9a2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.439+0000 7f9a33fff640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a2006b4d0 0x7f9a2006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000f60 tx=0x7f9a24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.443+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a2c07a070 con 0x7f9a3415dbe0 2024-04-25T09:55:37.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.616+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a3415ced0 con 0x7f9a3415dbe0 2024-04-25T09:55:37.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.617+0000 7f9a30ff9640 1 -- 172.21.15.148:0/4214049333 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f9a2c079970 con 0x7f9a3415dbe0 2024-04-25T09:55:37.620 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:37.620 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":39,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:55:35.981045+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:55:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a2006b4d0 msgr2=0x7f9a2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a2006b4d0 0x7f9a2006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000f60 tx=0x7f9a24009040 comp rx=0 tx=0).stop 2024-04-25T09:55:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 msgr2=0x7f9a340b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a340b4370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009e80 tx=0x7f9a2c004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 shutdown_connections 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a2006b4d0 0x7f9a2006d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a340b4370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340d1020 0x7f9a341538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 --2- 172.21.15.148:0/4214049333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b2cb0 0x7f9a341533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 >> 172.21.15.148:0/4214049333 conn(0x7f9a34061a20 msgr2=0x7f9a340c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 shutdown_connections 2024-04-25T09:55:37.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.619+0000 7f9a3a7b7640 1 -- 172.21.15.148:0/4214049333 wait complete. 2024-04-25T09:55:37.633 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:55:37.633 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:37.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 Processor -- start 2024-04-25T09:55:37.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 -- start start 2024-04-25T09:55:37.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 0x7f2ad80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2ad80d1020 0x7f2ad80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad80b97a0 con 0x7f2ad815dbe0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad815dfc0 con 0x7f2ad80d1020 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2add37e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80d1a00 con 0x7f2ad80b2cb0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2ad6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2ad815dbe0 0x7f2ad80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37238/0 (socket says 172.21.15.148:37238) 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.728+0000 7f2ad6ffd640 1 -- 172.21.15.148:0/1675188921 learned_addr learned my addr 172.21.15.148:0/1675188921 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373739525 0 0) 0x7f2ad80b97a0 con 0x7f2ad815dbe0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ac0003520 con 0x7f2ad815dbe0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 0x7f2ad80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad77fe640 1 --2- 172.21.15.148:0/1675188921 >> v2:172.21.15.188:3300/0 conn(0x7f2ad80d1020 0x7f2ad80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2564126327 0 0) 0x7f2ac0003520 con 0x7f2ad815dbe0 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 msgr2=0x7f2ad80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 0x7f2ad80d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 >> v2:172.21.15.188:3300/0 conn(0x7f2ad80d1020 msgr2=0x7f2ad80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 --2- 172.21.15.148:0/1675188921 >> v2:172.21.15.188:3300/0 conn(0x7f2ad80d1020 0x7f2ad80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad80d1c50 con 0x7f2ad815dbe0 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f2ac80019f0 con 0x7f2ad815dbe0 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 >> v1:172.21.15.148:6789/0 conn(0x7f2ad815dbe0 legacy=0x7f2ad80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 0x7f2ac00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ac0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac0004d40 0x7f2ac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80d1a00 con 0x7f2ac0003f30 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac000b9c0 con 0x7f2ac0004d40 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac000bb40 con 0x7f2ad80b2cb0 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ac0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad6ffd640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 0x7f2ac00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad77fe640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac0004d40 0x7f2ac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 -- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 msgr2=0x7f2ac00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad80b2cb0 0x7f2ac00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 -- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac0004d40 msgr2=0x7f2ac00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac0004d40 0x7f2ac00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.729+0000 7f2ad67fc640 1 -- 172.21.15.148:0/1675188921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac0003520 con 0x7f2ac0003f30 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.730+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ac0004800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009ac0 tx=0x7f2ac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5985315f0a3d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.730+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc036070 con 0x7f2ac0003f30 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.730+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc031400 con 0x7f2ac0003f30 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.730+0000 7f2ad5ffb640 1 -- 172.21.15.148:0/1675188921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc009690 con 0x7f2ac0003f30 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2add37e640 1 -- 172.21.15.148:0/1675188921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 msgr2=0x7f2ac0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2add37e640 1 --2- 172.21.15.148:0/1675188921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ac0004800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009ac0 tx=0x7f2ac0005830 comp rx=0 tx=0).stop 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2ad77fe640 1 -- 172.21.15.148:0/1675188921 reap_dead start 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2add37e640 1 -- 172.21.15.148:0/1675188921 shutdown_connections 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2add37e640 1 -- 172.21.15.148:0/1675188921 >> 172.21.15.148:0/1675188921 conn(0x7f2ad8061a20 msgr2=0x7f2ad8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2add37e640 1 -- 172.21.15.148:0/1675188921 shutdown_connections 2024-04-25T09:55:37.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.731+0000 7f2add37e640 1 -- 172.21.15.148:0/1675188921 wait complete. 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.732+0000 7f2add37e640 1 Processor -- start 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.732+0000 7f2add37e640 1 -- start start 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2add37e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ad8156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2add37e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ad8154f60 0x7f2ad81f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2add37e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad81f7180 0x7f2ad80b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2add37e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80b92a0 con 0x7f2ac0003f30 2024-04-25T09:55:37.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2add37e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80d2210 con 0x7f2ad8154f60 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2add37e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad80b9420 con 0x7f2ad81f7180 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ad8156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ad8156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58578/0 (socket says 172.21.15.148:58578) 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 -- 172.21.15.148:0/1838510428 learned_addr learned my addr 172.21.15.148:0/1838510428 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ad8154f60 0x7f2ad81f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad77fe640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad81f7180 0x7f2ad80b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 -- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad81f7180 msgr2=0x7f2ad80b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad81f7180 0x7f2ad80b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 -- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ad8154f60 msgr2=0x7f2ad81f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ad8154f60 0x7f2ad81f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 -- 172.21.15.148:0/1838510428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad8157450 con 0x7f2ac0003f30 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad77fe640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad81f7180 0x7f2ad80b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.733+0000 7f2ad6ffd640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ad8156e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007a20 tx=0x7f2ac800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.734+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8017020 con 0x7f2ac0003f30 2024-04-25T09:55:37.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.734+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac800ae00 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.734+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad8157730 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.734+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8013840 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.734+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad81f6c80 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.735+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad8073990 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.735+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ac801a020 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.735+0000 7f2abf7fe640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2ac406b500 0x7f2ac406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.735+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ac808adf0 con 0x7f2ac0003f30 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.735+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2ac406b500 0x7f2ac406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:37.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.736+0000 7f2ad67fc640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2ac406b500 0x7f2ac406d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2acc033000 tx=0x7f2acc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:37.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.739+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ac8021020 con 0x7f2ac0003f30 2024-04-25T09:55:37.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.904+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ad8155340 con 0x7f2ac406b500 2024-04-25T09:55:37.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.910+0000 7f2abf7fe640 1 -- 172.21.15.148:0/1838510428 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251884 (secure 0 0 0) 0x7f2ad8155340 con 0x7f2ac406b500 2024-04-25T09:55:37.916 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:37.917 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2ac406b500 msgr2=0x7f2ac406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2ac406b500 0x7f2ac406d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2acc033000 tx=0x7f2acc008030 comp rx=0 tx=0).stop 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 msgr2=0x7f2ad8156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ad8156e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8007a20 tx=0x7f2ac800a750 comp rx=0 tx=0).stop 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 shutdown_connections 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2ac406b500 0x7f2ac406d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ad81f7180 0x7f2ad80b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ad8154f60 0x7f2ad81f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 --2- 172.21.15.148:0/1838510428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ac0003f30 0x7f2ad8156e70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 >> 172.21.15.148:0/1838510428 conn(0x7f2ad8061a20 msgr2=0x7f2ad80d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 shutdown_connections 2024-04-25T09:55:37.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:37.913+0000 7f2add37e640 1 -- 172.21.15.148:0/1838510428 wait complete. 2024-04-25T09:55:37.927 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-25T09:55:37.034882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220732,"kb_used_data":4532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753988,"statfs":{"total":773094113280,"available":772868083712,"internally_reserved":0,"allocated":4640768,"data_stored":2663506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.073134"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005709+0000","last_change":"2024-04-25T09:55:34.998058+0000","last_active":"2024-04-25T09:55:35.005709+0000","last_peered":"2024-04-25T09:55:35.005709+0000","last_clean":"2024-04-25T09:55:35.005709+0000","last_became_active":"2024-04-25T09:55:34.997762+0000","last_became_peered":"2024-04-25T09:55:34.997762+0000","last_unstale":"2024-04-25T09:55:35.005709+0000","last_undegraded":"2024-04-25T09:55:35.005709+0000","last_fullsized":"2024-04-25T09:55:35.005709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:01.273548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.007976+0000","last_change":"2024-04-25T09:55:35.000950+0000","last_active":"2024-04-25T09:55:35.007976+0000","last_peered":"2024-04-25T09:55:35.007976+0000","last_clean":"2024-04-25T09:55:35.007976+0000","last_became_active":"2024-04-25T09:55:35.000684+0000","last_became_peered":"2024-04-25T09:55:35.000684+0000","last_unstale":"2024-04-25T09:55:35.007976+0000","last_undegraded":"2024-04-25T09:55:35.007976+0000","last_fullsized":"2024-04-25T09:55:35.007976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:53.496715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005649+0000","last_change":"2024-04-25T09:55:35.003813+0000","last_active":"2024-04-25T09:55:35.005649+0000","last_peered":"2024-04-25T09:55:35.005649+0000","last_clean":"2024-04-25T09:55:35.005649+0000","last_became_active":"2024-04-25T09:55:35.001566+0000","last_became_peered":"2024-04-25T09:55:35.001566+0000","last_unstale":"2024-04-25T09:55:35.005649+0000","last_undegraded":"2024-04-25T09:55:35.005649+0000","last_fullsized":"2024-04-25T09:55:35.005649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995849+0000","last_change":"2024-04-25T09:55:34.989140+0000","last_active":"2024-04-25T09:55:34.995849+0000","last_peered":"2024-04-25T09:55:34.995849+0000","last_clean":"2024-04-25T09:55:34.995849+0000","last_became_active":"2024-04-25T09:55:34.988837+0000","last_became_peered":"2024-04-25T09:55:34.988837+0000","last_unstale":"2024-04-25T09:55:34.995849+0000","last_undegraded":"2024-04-25T09:55:34.995849+0000","last_fullsized":"2024-04-25T09:55:34.995849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:14.451216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.989057+0000","last_change":"2024-04-25T09:55:34.987909+0000","last_active":"2024-04-25T09:55:34.989057+0000","last_peered":"2024-04-25T09:55:34.989057+0000","last_clean":"2024-04-25T09:55:34.989057+0000","last_became_active":"2024-04-25T09:55:34.987638+0000","last_became_peered":"2024-04-25T09:55:34.987638+0000","last_unstale":"2024-04-25T09:55:34.989057+0000","last_undegraded":"2024-04-25T09:55:34.989057+0000","last_fullsized":"2024-04-25T09:55:34.989057+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:12.889091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006455+0000","last_change":"2024-04-25T09:55:35.004395+0000","last_active":"2024-04-25T09:55:35.006455+0000","last_peered":"2024-04-25T09:55:35.006455+0000","last_clean":"2024-04-25T09:55:35.006455+0000","last_became_active":"2024-04-25T09:55:35.004131+0000","last_became_peered":"2024-04-25T09:55:35.004131+0000","last_unstale":"2024-04-25T09:55:35.006455+0000","last_undegraded":"2024-04-25T09:55:35.006455+0000","last_fullsized":"2024-04-25T09:55:35.006455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000474+0000","last_change":"2024-04-25T09:55:34.993774+0000","last_active":"2024-04-25T09:55:35.000474+0000","last_peered":"2024-04-25T09:55:35.000474+0000","last_clean":"2024-04-25T09:55:35.000474+0000","last_became_active":"2024-04-25T09:55:34.993499+0000","last_became_peered":"2024-04-25T09:55:34.993499+0000","last_unstale":"2024-04-25T09:55:35.000474+0000","last_undegraded":"2024-04-25T09:55:35.000474+0000","last_fullsized":"2024-04-25T09:55:35.000474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:20.783934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.007904+0000","last_change":"2024-04-25T09:55:35.003423+0000","last_active":"2024-04-25T09:55:35.007904+0000","last_peered":"2024-04-25T09:55:35.007904+0000","last_clean":"2024-04-25T09:55:35.007904+0000","last_became_active":"2024-04-25T09:55:35.003147+0000","last_became_peered":"2024-04-25T09:55:35.003147+0000","last_unstale":"2024-04-25T09:55:35.007904+0000","last_undegraded":"2024-04-25T09:55:35.007904+0000","last_fullsized":"2024-04-25T09:55:35.007904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:48.116086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005387+0000","last_change":"2024-04-25T09:55:35.002911+0000","last_active":"2024-04-25T09:55:35.005387+0000","last_peered":"2024-04-25T09:55:35.005387+0000","last_clean":"2024-04-25T09:55:35.005387+0000","last_became_active":"2024-04-25T09:55:35.002313+0000","last_became_peered":"2024-04-25T09:55:35.002313+0000","last_unstale":"2024-04-25T09:55:35.005387+0000","last_undegraded":"2024-04-25T09:55:35.005387+0000","last_fullsized":"2024-04-25T09:55:35.005387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000052+0000","last_change":"2024-04-25T09:55:34.989183+0000","last_active":"2024-04-25T09:55:35.000052+0000","last_peered":"2024-04-25T09:55:35.000052+0000","last_clean":"2024-04-25T09:55:35.000052+0000","last_became_active":"2024-04-25T09:55:34.988894+0000","last_became_peered":"2024-04-25T09:55:34.988894+0000","last_unstale":"2024-04-25T09:55:35.000052+0000","last_undegraded":"2024-04-25T09:55:35.000052+0000","last_fullsized":"2024-04-25T09:55:35.000052+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000537+0000","last_change":"2024-04-25T09:55:34.988436+0000","last_active":"2024-04-25T09:55:35.000537+0000","last_peered":"2024-04-25T09:55:35.000537+0000","last_clean":"2024-04-25T09:55:35.000537+0000","last_became_active":"2024-04-25T09:55:34.988190+0000","last_became_peered":"2024-04-25T09:55:34.988190+0000","last_unstale":"2024-04-25T09:55:35.000537+0000","last_undegraded":"2024-04-25T09:55:35.000537+0000","last_fullsized":"2024-04-25T09:55:35.000537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995203+0000","last_change":"2024-04-25T09:55:34.991544+0000","last_active":"2024-04-25T09:55:34.995203+0000","last_peered":"2024-04-25T09:55:34.995203+0000","last_clean":"2024-04-25T09:55:34.995203+0000","last_became_active":"2024-04-25T09:55:34.991265+0000","last_became_peered":"2024-04-25T09:55:34.991265+0000","last_unstale":"2024-04-25T09:55:34.995203+0000","last_undegraded":"2024-04-25T09:55:34.995203+0000","last_fullsized":"2024-04-25T09:55:34.995203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:08.435853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000524+0000","last_change":"2024-04-25T09:55:34.987801+0000","last_active":"2024-04-25T09:55:35.000524+0000","last_peered":"2024-04-25T09:55:35.000524+0000","last_clean":"2024-04-25T09:55:35.000524+0000","last_became_active":"2024-04-25T09:55:34.987541+0000","last_became_peered":"2024-04-25T09:55:34.987541+0000","last_unstale":"2024-04-25T09:55:35.000524+0000","last_undegraded":"2024-04-25T09:55:35.000524+0000","last_fullsized":"2024-04-25T09:55:35.000524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991204+0000","last_change":"2024-04-25T09:55:34.985185+0000","last_active":"2024-04-25T09:55:34.991204+0000","last_peered":"2024-04-25T09:55:34.991204+0000","last_clean":"2024-04-25T09:55:34.991204+0000","last_became_active":"2024-04-25T09:55:34.984913+0000","last_became_peered":"2024-04-25T09:55:34.984913+0000","last_unstale":"2024-04-25T09:55:34.991204+0000","last_undegraded":"2024-04-25T09:55:34.991204+0000","last_fullsized":"2024-04-25T09:55:34.991204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000453+0000","last_change":"2024-04-25T09:55:34.990337+0000","last_active":"2024-04-25T09:55:35.000453+0000","last_peered":"2024-04-25T09:55:35.000453+0000","last_clean":"2024-04-25T09:55:35.000453+0000","last_became_active":"2024-04-25T09:55:34.990083+0000","last_became_peered":"2024-04-25T09:55:34.990083+0000","last_unstale":"2024-04-25T09:55:35.000453+0000","last_undegraded":"2024-04-25T09:55:35.000453+0000","last_fullsized":"2024-04-25T09:55:35.000453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004811+0000","last_change":"2024-04-25T09:55:35.001561+0000","last_active":"2024-04-25T09:55:35.004811+0000","last_peered":"2024-04-25T09:55:35.004811+0000","last_clean":"2024-04-25T09:55:35.004811+0000","last_became_active":"2024-04-25T09:55:35.001257+0000","last_became_peered":"2024-04-25T09:55:35.001257+0000","last_unstale":"2024-04-25T09:55:35.004811+0000","last_undegraded":"2024-04-25T09:55:35.004811+0000","last_fullsized":"2024-04-25T09:55:35.004811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.003954+0000","last_change":"2024-04-25T09:55:35.000095+0000","last_active":"2024-04-25T09:55:35.003954+0000","last_peered":"2024-04-25T09:55:35.003954+0000","last_clean":"2024-04-25T09:55:35.003954+0000","last_became_active":"2024-04-25T09:55:34.999778+0000","last_became_peered":"2024-04-25T09:55:34.999778+0000","last_unstale":"2024-04-25T09:55:35.003954+0000","last_undegraded":"2024-04-25T09:55:35.003954+0000","last_fullsized":"2024-04-25T09:55:35.003954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998632+0000","last_change":"2024-04-25T09:55:34.997194+0000","last_active":"2024-04-25T09:55:34.998632+0000","last_peered":"2024-04-25T09:55:34.998632+0000","last_clean":"2024-04-25T09:55:34.998632+0000","last_became_active":"2024-04-25T09:55:34.996937+0000","last_became_peered":"2024-04-25T09:55:34.996937+0000","last_unstale":"2024-04-25T09:55:34.998632+0000","last_undegraded":"2024-04-25T09:55:34.998632+0000","last_fullsized":"2024-04-25T09:55:34.998632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005914+0000","last_change":"2024-04-25T09:55:35.002485+0000","last_active":"2024-04-25T09:55:35.005914+0000","last_peered":"2024-04-25T09:55:35.005914+0000","last_clean":"2024-04-25T09:55:35.005914+0000","last_became_active":"2024-04-25T09:55:35.001630+0000","last_became_peered":"2024-04-25T09:55:35.001630+0000","last_unstale":"2024-04-25T09:55:35.005914+0000","last_undegraded":"2024-04-25T09:55:35.005914+0000","last_fullsized":"2024-04-25T09:55:35.005914+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.695824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997639+0000","last_change":"2024-04-25T09:55:34.995872+0000","last_active":"2024-04-25T09:55:34.997639+0000","last_peered":"2024-04-25T09:55:34.997639+0000","last_clean":"2024-04-25T09:55:34.997639+0000","last_became_active":"2024-04-25T09:55:34.995508+0000","last_became_peered":"2024-04-25T09:55:34.995508+0000","last_unstale":"2024-04-25T09:55:34.997639+0000","last_undegraded":"2024-04-25T09:55:34.997639+0000","last_fullsized":"2024-04-25T09:55:34.997639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:20.981250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004064+0000","last_change":"2024-04-25T09:55:34.999549+0000","last_active":"2024-04-25T09:55:35.004064+0000","last_peered":"2024-04-25T09:55:35.004064+0000","last_clean":"2024-04-25T09:55:35.004064+0000","last_became_active":"2024-04-25T09:55:34.999262+0000","last_became_peered":"2024-04-25T09:55:34.999262+0000","last_unstale":"2024-04-25T09:55:35.004064+0000","last_undegraded":"2024-04-25T09:55:35.004064+0000","last_fullsized":"2024-04-25T09:55:35.004064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:36.816221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991258+0000","last_change":"2024-04-25T09:55:34.986306+0000","last_active":"2024-04-25T09:55:34.991258+0000","last_peered":"2024-04-25T09:55:34.991258+0000","last_clean":"2024-04-25T09:55:34.991258+0000","last_became_active":"2024-04-25T09:55:34.986024+0000","last_became_peered":"2024-04-25T09:55:34.986024+0000","last_unstale":"2024-04-25T09:55:34.991258+0000","last_undegraded":"2024-04-25T09:55:34.991258+0000","last_fullsized":"2024-04-25T09:55:34.991258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998336+0000","last_change":"2024-04-25T09:55:34.994839+0000","last_active":"2024-04-25T09:55:34.998336+0000","last_peered":"2024-04-25T09:55:34.998336+0000","last_clean":"2024-04-25T09:55:34.998336+0000","last_became_active":"2024-04-25T09:55:34.994499+0000","last_became_peered":"2024-04-25T09:55:34.994499+0000","last_unstale":"2024-04-25T09:55:34.998336+0000","last_undegraded":"2024-04-25T09:55:34.998336+0000","last_fullsized":"2024-04-25T09:55:34.998336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:04.584156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995665+0000","last_change":"2024-04-25T09:55:34.990426+0000","last_active":"2024-04-25T09:55:34.995665+0000","last_peered":"2024-04-25T09:55:34.995665+0000","last_clean":"2024-04-25T09:55:34.995665+0000","last_became_active":"2024-04-25T09:55:34.990169+0000","last_became_peered":"2024-04-25T09:55:34.990169+0000","last_unstale":"2024-04-25T09:55:34.995665+0000","last_undegraded":"2024-04-25T09:55:34.995665+0000","last_fullsized":"2024-04-25T09:55:34.995665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:58.440534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.989078+0000","last_change":"2024-04-25T09:55:34.987332+0000","last_active":"2024-04-25T09:55:34.989078+0000","last_peered":"2024-04-25T09:55:34.989078+0000","last_clean":"2024-04-25T09:55:34.989078+0000","last_became_active":"2024-04-25T09:55:34.987036+0000","last_became_peered":"2024-04-25T09:55:34.987036+0000","last_unstale":"2024-04-25T09:55:34.989078+0000","last_undegraded":"2024-04-25T09:55:34.989078+0000","last_fullsized":"2024-04-25T09:55:34.989078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:06.981986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006263+0000","last_change":"2024-04-25T09:55:35.002415+0000","last_active":"2024-04-25T09:55:35.006263+0000","last_peered":"2024-04-25T09:55:35.006263+0000","last_clean":"2024-04-25T09:55:35.006263+0000","last_became_active":"2024-04-25T09:55:35.002161+0000","last_became_peered":"2024-04-25T09:55:35.002161+0000","last_unstale":"2024-04-25T09:55:35.006263+0000","last_undegraded":"2024-04-25T09:55:35.006263+0000","last_fullsized":"2024-04-25T09:55:35.006263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.992291+0000","last_change":"2024-04-25T09:55:34.990711+0000","last_active":"2024-04-25T09:55:34.992291+0000","last_peered":"2024-04-25T09:55:34.992291+0000","last_clean":"2024-04-25T09:55:34.992291+0000","last_became_active":"2024-04-25T09:55:34.990449+0000","last_became_peered":"2024-04-25T09:55:34.990449+0000","last_unstale":"2024-04-25T09:55:34.992291+0000","last_undegraded":"2024-04-25T09:55:34.992291+0000","last_fullsized":"2024-04-25T09:55:34.992291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:07.494711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000573+0000","last_change":"2024-04-25T09:55:34.997116+0000","last_active":"2024-04-25T09:55:35.000573+0000","last_peered":"2024-04-25T09:55:35.000573+0000","last_clean":"2024-04-25T09:55:35.000573+0000","last_became_active":"2024-04-25T09:55:34.996585+0000","last_became_peered":"2024-04-25T09:55:34.996585+0000","last_unstale":"2024-04-25T09:55:35.000573+0000","last_undegraded":"2024-04-25T09:55:35.000573+0000","last_fullsized":"2024-04-25T09:55:35.000573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.902893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000414+0000","last_change":"2024-04-25T09:55:34.998486+0000","last_active":"2024-04-25T09:55:35.000414+0000","last_peered":"2024-04-25T09:55:35.000414+0000","last_clean":"2024-04-25T09:55:35.000414+0000","last_became_active":"2024-04-25T09:55:34.998204+0000","last_became_peered":"2024-04-25T09:55:34.998204+0000","last_unstale":"2024-04-25T09:55:35.000414+0000","last_undegraded":"2024-04-25T09:55:35.000414+0000","last_fullsized":"2024-04-25T09:55:35.000414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:08.435853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.996740+0000","last_change":"2024-04-25T09:55:34.995266+0000","last_active":"2024-04-25T09:55:34.996740+0000","last_peered":"2024-04-25T09:55:34.996740+0000","last_clean":"2024-04-25T09:55:34.996740+0000","last_became_active":"2024-04-25T09:55:34.994988+0000","last_became_peered":"2024-04-25T09:55:34.994988+0000","last_unstale":"2024-04-25T09:55:34.996740+0000","last_undegraded":"2024-04-25T09:55:34.996740+0000","last_fullsized":"2024-04-25T09:55:34.996740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:40.325973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998358+0000","last_change":"2024-04-25T09:55:34.995336+0000","last_active":"2024-04-25T09:55:34.998358+0000","last_peered":"2024-04-25T09:55:34.998358+0000","last_clean":"2024-04-25T09:55:34.998358+0000","last_became_active":"2024-04-25T09:55:34.995077+0000","last_became_peered":"2024-04-25T09:55:34.995077+0000","last_unstale":"2024-04-25T09:55:34.998358+0000","last_undegraded":"2024-04-25T09:55:34.998358+0000","last_fullsized":"2024-04-25T09:55:34.998358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.010639+0000","last_change":"2024-04-25T09:55:35.006392+0000","last_active":"2024-04-25T09:55:35.010639+0000","last_peered":"2024-04-25T09:55:35.010639+0000","last_clean":"2024-04-25T09:55:35.010639+0000","last_became_active":"2024-04-25T09:55:35.005962+0000","last_became_peered":"2024-04-25T09:55:35.005962+0000","last_unstale":"2024-04-25T09:55:35.010639+0000","last_undegraded":"2024-04-25T09:55:35.010639+0000","last_fullsized":"2024-04-25T09:55:35.010639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004003+0000","last_change":"2024-04-25T09:55:35.000104+0000","last_active":"2024-04-25T09:55:35.004003+0000","last_peered":"2024-04-25T09:55:35.004003+0000","last_clean":"2024-04-25T09:55:35.004003+0000","last_became_active":"2024-04-25T09:55:34.999833+0000","last_became_peered":"2024-04-25T09:55:34.999833+0000","last_unstale":"2024-04-25T09:55:35.004003+0000","last_undegraded":"2024-04-25T09:55:35.004003+0000","last_fullsized":"2024-04-25T09:55:35.004003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002113+0000","last_change":"2024-04-25T09:55:33.980326+0000","last_active":"2024-04-25T09:55:35.002113+0000","last_peered":"2024-04-25T09:55:35.002113+0000","last_clean":"2024-04-25T09:55:35.002113+0000","last_became_active":"2024-04-25T09:55:33.980059+0000","last_became_peered":"2024-04-25T09:55:33.980059+0000","last_unstale":"2024-04-25T09:55:35.002113+0000","last_undegraded":"2024-04-25T09:55:35.002113+0000","last_fullsized":"2024-04-25T09:55:35.002113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.008118+0000","last_change":"2024-04-25T09:55:35.005447+0000","last_active":"2024-04-25T09:55:35.008118+0000","last_peered":"2024-04-25T09:55:35.008118+0000","last_clean":"2024-04-25T09:55:35.008118+0000","last_became_active":"2024-04-25T09:55:35.004996+0000","last_became_peered":"2024-04-25T09:55:35.004996+0000","last_unstale":"2024-04-25T09:55:35.008118+0000","last_undegraded":"2024-04-25T09:55:35.008118+0000","last_fullsized":"2024-04-25T09:55:35.008118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001948+0000","last_change":"2024-04-25T09:55:33.988511+0000","last_active":"2024-04-25T09:55:35.001948+0000","last_peered":"2024-04-25T09:55:35.001948+0000","last_clean":"2024-04-25T09:55:35.001948+0000","last_became_active":"2024-04-25T09:55:33.988206+0000","last_became_peered":"2024-04-25T09:55:33.988206+0000","last_unstale":"2024-04-25T09:55:35.001948+0000","last_undegraded":"2024-04-25T09:55:35.001948+0000","last_fullsized":"2024-04-25T09:55:35.001948+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:19.979241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991101+0000","last_change":"2024-04-25T09:55:34.985951+0000","last_active":"2024-04-25T09:55:34.991101+0000","last_peered":"2024-04-25T09:55:34.991101+0000","last_clean":"2024-04-25T09:55:34.991101+0000","last_became_active":"2024-04-25T09:55:34.985681+0000","last_became_peered":"2024-04-25T09:55:34.985681+0000","last_unstale":"2024-04-25T09:55:34.991101+0000","last_undegraded":"2024-04-25T09:55:34.991101+0000","last_fullsized":"2024-04-25T09:55:34.991101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006781+0000","last_change":"2024-04-25T09:55:33.991186+0000","last_active":"2024-04-25T09:55:35.006781+0000","last_peered":"2024-04-25T09:55:35.006781+0000","last_clean":"2024-04-25T09:55:35.006781+0000","last_became_active":"2024-04-25T09:55:33.990904+0000","last_became_peered":"2024-04-25T09:55:33.990904+0000","last_unstale":"2024-04-25T09:55:35.006781+0000","last_undegraded":"2024-04-25T09:55:35.006781+0000","last_fullsized":"2024-04-25T09:55:35.006781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.582147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005731+0000","last_change":"2024-04-25T09:55:34.999727+0000","last_active":"2024-04-25T09:55:35.005731+0000","last_peered":"2024-04-25T09:55:35.005731+0000","last_clean":"2024-04-25T09:55:35.005731+0000","last_became_active":"2024-04-25T09:55:34.998967+0000","last_became_peered":"2024-04-25T09:55:34.998967+0000","last_unstale":"2024-04-25T09:55:35.005731+0000","last_undegraded":"2024-04-25T09:55:35.005731+0000","last_fullsized":"2024-04-25T09:55:35.005731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:15.133979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005653+0000","last_change":"2024-04-25T09:55:33.988594+0000","last_active":"2024-04-25T09:55:35.005653+0000","last_peered":"2024-04-25T09:55:35.005653+0000","last_clean":"2024-04-25T09:55:35.005653+0000","last_became_active":"2024-04-25T09:55:33.988321+0000","last_became_peered":"2024-04-25T09:55:33.988321+0000","last_unstale":"2024-04-25T09:55:35.005653+0000","last_undegraded":"2024-04-25T09:55:35.005653+0000","last_fullsized":"2024-04-25T09:55:35.005653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.576822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002499+0000","last_change":"2024-04-25T09:55:34.999393+0000","last_active":"2024-04-25T09:55:35.002499+0000","last_peered":"2024-04-25T09:55:35.002499+0000","last_clean":"2024-04-25T09:55:35.002499+0000","last_became_active":"2024-04-25T09:55:34.998891+0000","last_became_peered":"2024-04-25T09:55:34.998891+0000","last_unstale":"2024-04-25T09:55:35.002499+0000","last_undegraded":"2024-04-25T09:55:35.002499+0000","last_fullsized":"2024-04-25T09:55:35.002499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:04.584156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998903+0000","last_change":"2024-04-25T09:55:33.987385+0000","last_active":"2024-04-25T09:55:34.998903+0000","last_peered":"2024-04-25T09:55:34.998903+0000","last_clean":"2024-04-25T09:55:34.998903+0000","last_became_active":"2024-04-25T09:55:33.986702+0000","last_became_peered":"2024-04-25T09:55:33.986702+0000","last_unstale":"2024-04-25T09:55:34.998903+0000","last_undegraded":"2024-04-25T09:55:34.998903+0000","last_fullsized":"2024-04-25T09:55:34.998903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:07.433844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004190+0000","last_change":"2024-04-25T09:55:35.000315+0000","last_active":"2024-04-25T09:55:35.004190+0000","last_peered":"2024-04-25T09:55:35.004190+0000","last_clean":"2024-04-25T09:55:35.004190+0000","last_became_active":"2024-04-25T09:55:34.999725+0000","last_became_peered":"2024-04-25T09:55:34.999725+0000","last_unstale":"2024-04-25T09:55:35.004190+0000","last_undegraded":"2024-04-25T09:55:35.004190+0000","last_fullsized":"2024-04-25T09:55:35.004190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:39.381865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001843+0000","last_change":"2024-04-25T09:55:33.987960+0000","last_active":"2024-04-25T09:55:35.001843+0000","last_peered":"2024-04-25T09:55:35.001843+0000","last_clean":"2024-04-25T09:55:35.001843+0000","last_became_active":"2024-04-25T09:55:33.987681+0000","last_became_peered":"2024-04-25T09:55:33.987681+0000","last_unstale":"2024-04-25T09:55:35.001843+0000","last_undegraded":"2024-04-25T09:55:35.001843+0000","last_fullsized":"2024-04-25T09:55:35.001843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.979977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991230+0000","last_change":"2024-04-25T09:55:34.985424+0000","last_active":"2024-04-25T09:55:34.991230+0000","last_peered":"2024-04-25T09:55:34.991230+0000","last_clean":"2024-04-25T09:55:34.991230+0000","last_became_active":"2024-04-25T09:55:34.985132+0000","last_became_peered":"2024-04-25T09:55:34.985132+0000","last_unstale":"2024-04-25T09:55:34.991230+0000","last_undegraded":"2024-04-25T09:55:34.991230+0000","last_fullsized":"2024-04-25T09:55:34.991230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995356+0000","last_change":"2024-04-25T09:55:33.988746+0000","last_active":"2024-04-25T09:55:34.995356+0000","last_peered":"2024-04-25T09:55:34.995356+0000","last_clean":"2024-04-25T09:55:34.995356+0000","last_became_active":"2024-04-25T09:55:33.988411+0000","last_became_peered":"2024-04-25T09:55:33.988411+0000","last_unstale":"2024-04-25T09:55:34.995356+0000","last_undegraded":"2024-04-25T09:55:34.995356+0000","last_fullsized":"2024-04-25T09:55:34.995356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:00.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.007997+0000","last_change":"2024-04-25T09:55:35.004752+0000","last_active":"2024-04-25T09:55:35.007997+0000","last_peered":"2024-04-25T09:55:35.007997+0000","last_clean":"2024-04-25T09:55:35.007997+0000","last_became_active":"2024-04-25T09:55:35.004483+0000","last_became_peered":"2024-04-25T09:55:35.004483+0000","last_unstale":"2024-04-25T09:55:35.007997+0000","last_undegraded":"2024-04-25T09:55:35.007997+0000","last_fullsized":"2024-04-25T09:55:35.007997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:38.173441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.974033+0000","last_change":"2024-04-25T09:55:33.981430+0000","last_active":"2024-04-25T09:55:34.974033+0000","last_peered":"2024-04-25T09:55:34.974033+0000","last_clean":"2024-04-25T09:55:34.974033+0000","last_became_active":"2024-04-25T09:55:33.981055+0000","last_became_peered":"2024-04-25T09:55:33.981055+0000","last_unstale":"2024-04-25T09:55:34.974033+0000","last_undegraded":"2024-04-25T09:55:34.974033+0000","last_fullsized":"2024-04-25T09:55:34.974033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:07.433844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995879+0000","last_change":"2024-04-25T09:55:34.991343+0000","last_active":"2024-04-25T09:55:34.995879+0000","last_peered":"2024-04-25T09:55:34.995879+0000","last_clean":"2024-04-25T09:55:34.995879+0000","last_became_active":"2024-04-25T09:55:34.990776+0000","last_became_peered":"2024-04-25T09:55:34.990776+0000","last_unstale":"2024-04-25T09:55:34.995879+0000","last_undegraded":"2024-04-25T09:55:34.995879+0000","last_fullsized":"2024-04-25T09:55:34.995879+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:12.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.987241+0000","last_change":"2024-04-25T09:55:33.990153+0000","last_active":"2024-04-25T09:55:34.987241+0000","last_peered":"2024-04-25T09:55:34.987241+0000","last_clean":"2024-04-25T09:55:34.987241+0000","last_became_active":"2024-04-25T09:55:33.989882+0000","last_became_peered":"2024-04-25T09:55:33.989882+0000","last_unstale":"2024-04-25T09:55:34.987241+0000","last_undegraded":"2024-04-25T09:55:34.987241+0000","last_fullsized":"2024-04-25T09:55:34.987241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:56.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997448+0000","last_change":"2024-04-25T09:55:34.990739+0000","last_active":"2024-04-25T09:55:34.997448+0000","last_peered":"2024-04-25T09:55:34.997448+0000","last_clean":"2024-04-25T09:55:34.997448+0000","last_became_active":"2024-04-25T09:55:34.990102+0000","last_became_peered":"2024-04-25T09:55:34.990102+0000","last_unstale":"2024-04-25T09:55:34.997448+0000","last_undegraded":"2024-04-25T09:55:34.997448+0000","last_fullsized":"2024-04-25T09:55:34.997448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:08.435853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980322+0000","last_change":"2024-04-25T09:55:33.992403+0000","last_active":"2024-04-25T09:55:34.980322+0000","last_peered":"2024-04-25T09:55:34.980322+0000","last_clean":"2024-04-25T09:55:34.980322+0000","last_became_active":"2024-04-25T09:55:33.992080+0000","last_became_peered":"2024-04-25T09:55:33.992080+0000","last_unstale":"2024-04-25T09:55:34.980322+0000","last_undegraded":"2024-04-25T09:55:34.980322+0000","last_fullsized":"2024-04-25T09:55:34.980322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:00.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998566+0000","last_change":"2024-04-25T09:55:34.993226+0000","last_active":"2024-04-25T09:55:34.998566+0000","last_peered":"2024-04-25T09:55:34.998566+0000","last_clean":"2024-04-25T09:55:34.998566+0000","last_became_active":"2024-04-25T09:55:34.992947+0000","last_became_peered":"2024-04-25T09:55:34.992947+0000","last_unstale":"2024-04-25T09:55:34.998566+0000","last_undegraded":"2024-04-25T09:55:34.998566+0000","last_fullsized":"2024-04-25T09:55:34.998566+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:14.578831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001068+0000","last_change":"2024-04-25T09:55:33.991563+0000","last_active":"2024-04-25T09:55:35.001068+0000","last_peered":"2024-04-25T09:55:35.001068+0000","last_clean":"2024-04-25T09:55:35.001068+0000","last_became_active":"2024-04-25T09:55:33.991265+0000","last_became_peered":"2024-04-25T09:55:33.991265+0000","last_unstale":"2024-04-25T09:55:35.001068+0000","last_undegraded":"2024-04-25T09:55:35.001068+0000","last_fullsized":"2024-04-25T09:55:35.001068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.576822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.993706+0000","last_change":"2024-04-25T09:55:34.991737+0000","last_active":"2024-04-25T09:55:34.993706+0000","last_peered":"2024-04-25T09:55:34.993706+0000","last_clean":"2024-04-25T09:55:34.993706+0000","last_became_active":"2024-04-25T09:55:34.991452+0000","last_became_peered":"2024-04-25T09:55:34.991452+0000","last_unstale":"2024-04-25T09:55:34.993706+0000","last_undegraded":"2024-04-25T09:55:34.993706+0000","last_fullsized":"2024-04-25T09:55:34.993706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.010726+0000","last_change":"2024-04-25T09:55:33.990233+0000","last_active":"2024-04-25T09:55:35.010726+0000","last_peered":"2024-04-25T09:55:35.010726+0000","last_clean":"2024-04-25T09:55:35.010726+0000","last_became_active":"2024-04-25T09:55:33.989949+0000","last_became_peered":"2024-04-25T09:55:33.989949+0000","last_unstale":"2024-04-25T09:55:35.010726+0000","last_undegraded":"2024-04-25T09:55:35.010726+0000","last_fullsized":"2024-04-25T09:55:35.010726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:32.424425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.003299+0000","last_change":"2024-04-25T09:55:35.001157+0000","last_active":"2024-04-25T09:55:35.003299+0000","last_peered":"2024-04-25T09:55:35.003299+0000","last_clean":"2024-04-25T09:55:35.003299+0000","last_became_active":"2024-04-25T09:55:35.000900+0000","last_became_peered":"2024-04-25T09:55:35.000900+0000","last_unstale":"2024-04-25T09:55:35.003299+0000","last_undegraded":"2024-04-25T09:55:35.003299+0000","last_fullsized":"2024-04-25T09:55:35.003299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000300+0000","last_change":"2024-04-25T09:55:33.991510+0000","last_active":"2024-04-25T09:55:35.000300+0000","last_peered":"2024-04-25T09:55:35.000300+0000","last_clean":"2024-04-25T09:55:35.000300+0000","last_became_active":"2024-04-25T09:55:33.991171+0000","last_became_peered":"2024-04-25T09:55:33.991171+0000","last_unstale":"2024-04-25T09:55:35.000300+0000","last_undegraded":"2024-04-25T09:55:35.000300+0000","last_fullsized":"2024-04-25T09:55:35.000300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:34.886125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.003824+0000","last_change":"2024-04-25T09:55:34.993387+0000","last_active":"2024-04-25T09:55:35.003824+0000","last_peered":"2024-04-25T09:55:35.003824+0000","last_clean":"2024-04-25T09:55:35.003824+0000","last_became_active":"2024-04-25T09:55:34.993069+0000","last_became_peered":"2024-04-25T09:55:34.993069+0000","last_unstale":"2024-04-25T09:55:35.003824+0000","last_undegraded":"2024-04-25T09:55:35.003824+0000","last_fullsized":"2024-04-25T09:55:35.003824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:20.783934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.994313+0000","last_change":"2024-04-25T09:55:33.989882+0000","last_active":"2024-04-25T09:55:34.994313+0000","last_peered":"2024-04-25T09:55:34.994313+0000","last_clean":"2024-04-25T09:55:34.994313+0000","last_became_active":"2024-04-25T09:55:33.989619+0000","last_became_peered":"2024-04-25T09:55:33.989619+0000","last_unstale":"2024-04-25T09:55:34.994313+0000","last_undegraded":"2024-04-25T09:55:34.994313+0000","last_fullsized":"2024-04-25T09:55:34.994313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.576822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997355+0000","last_change":"2024-04-25T09:55:34.993510+0000","last_active":"2024-04-25T09:55:34.997355+0000","last_peered":"2024-04-25T09:55:34.997355+0000","last_clean":"2024-04-25T09:55:34.997355+0000","last_became_active":"2024-04-25T09:55:34.993214+0000","last_became_peered":"2024-04-25T09:55:34.993214+0000","last_unstale":"2024-04-25T09:55:34.997355+0000","last_undegraded":"2024-04-25T09:55:34.997355+0000","last_fullsized":"2024-04-25T09:55:34.997355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:01.725669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.978302+0000","last_change":"2024-04-25T09:55:33.980964+0000","last_active":"2024-04-25T09:55:34.978302+0000","last_peered":"2024-04-25T09:55:34.978302+0000","last_clean":"2024-04-25T09:55:34.978302+0000","last_became_active":"2024-04-25T09:55:33.980702+0000","last_became_peered":"2024-04-25T09:55:33.980702+0000","last_unstale":"2024-04-25T09:55:34.978302+0000","last_undegraded":"2024-04-25T09:55:34.978302+0000","last_fullsized":"2024-04-25T09:55:34.978302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:51.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005488+0000","last_change":"2024-04-25T09:55:34.997374+0000","last_active":"2024-04-25T09:55:35.005488+0000","last_peered":"2024-04-25T09:55:35.005488+0000","last_clean":"2024-04-25T09:55:35.005488+0000","last_became_active":"2024-04-25T09:55:34.997059+0000","last_became_peered":"2024-04-25T09:55:34.997059+0000","last_unstale":"2024-04-25T09:55:35.005488+0000","last_undegraded":"2024-04-25T09:55:35.005488+0000","last_fullsized":"2024-04-25T09:55:35.005488+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:36.816221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004870+0000","last_change":"2024-04-25T09:55:33.987327+0000","last_active":"2024-04-25T09:55:35.004870+0000","last_peered":"2024-04-25T09:55:35.004870+0000","last_clean":"2024-04-25T09:55:35.004870+0000","last_became_active":"2024-04-25T09:55:33.987063+0000","last_became_peered":"2024-04-25T09:55:33.987063+0000","last_unstale":"2024-04-25T09:55:35.004870+0000","last_undegraded":"2024-04-25T09:55:35.004870+0000","last_fullsized":"2024-04-25T09:55:35.004870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:19.979241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005306+0000","last_change":"2024-04-25T09:55:34.998184+0000","last_active":"2024-04-25T09:55:35.005306+0000","last_peered":"2024-04-25T09:55:35.005306+0000","last_clean":"2024-04-25T09:55:35.005306+0000","last_became_active":"2024-04-25T09:55:34.997923+0000","last_became_peered":"2024-04-25T09:55:34.997923+0000","last_unstale":"2024-04-25T09:55:35.005306+0000","last_undegraded":"2024-04-25T09:55:35.005306+0000","last_fullsized":"2024-04-25T09:55:35.005306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:39.381865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000280+0000","last_change":"2024-04-25T09:55:33.981963+0000","last_active":"2024-04-25T09:55:35.000280+0000","last_peered":"2024-04-25T09:55:35.000280+0000","last_clean":"2024-04-25T09:55:35.000280+0000","last_became_active":"2024-04-25T09:55:33.981647+0000","last_became_peered":"2024-04-25T09:55:33.981647+0000","last_unstale":"2024-04-25T09:55:35.000280+0000","last_undegraded":"2024-04-25T09:55:35.000280+0000","last_fullsized":"2024-04-25T09:55:35.000280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.582147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004952+0000","last_change":"2024-04-25T09:55:35.001579+0000","last_active":"2024-04-25T09:55:35.004952+0000","last_peered":"2024-04-25T09:55:35.004952+0000","last_clean":"2024-04-25T09:55:35.004952+0000","last_became_active":"2024-04-25T09:55:35.001312+0000","last_became_peered":"2024-04-25T09:55:35.001312+0000","last_unstale":"2024-04-25T09:55:35.004952+0000","last_undegraded":"2024-04-25T09:55:35.004952+0000","last_fullsized":"2024-04-25T09:55:35.004952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:15.133979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002726+0000","last_change":"2024-04-25T09:55:33.981893+0000","last_active":"2024-04-25T09:55:35.002726+0000","last_peered":"2024-04-25T09:55:35.002726+0000","last_clean":"2024-04-25T09:55:35.002726+0000","last_became_active":"2024-04-25T09:55:33.981615+0000","last_became_peered":"2024-04-25T09:55:33.981615+0000","last_unstale":"2024-04-25T09:55:35.002726+0000","last_undegraded":"2024-04-25T09:55:35.002726+0000","last_fullsized":"2024-04-25T09:55:35.002726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.582147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004083+0000","last_change":"2024-04-25T09:55:34.999272+0000","last_active":"2024-04-25T09:55:35.004083+0000","last_peered":"2024-04-25T09:55:35.004083+0000","last_clean":"2024-04-25T09:55:35.004083+0000","last_became_active":"2024-04-25T09:55:34.998960+0000","last_became_peered":"2024-04-25T09:55:34.998960+0000","last_unstale":"2024-04-25T09:55:35.004083+0000","last_undegraded":"2024-04-25T09:55:35.004083+0000","last_fullsized":"2024-04-25T09:55:35.004083+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:20.783934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.999190+0000","last_change":"2024-04-25T09:55:33.985741+0000","last_active":"2024-04-25T09:55:34.999190+0000","last_peered":"2024-04-25T09:55:34.999190+0000","last_clean":"2024-04-25T09:55:34.999190+0000","last_became_active":"2024-04-25T09:55:33.985408+0000","last_became_peered":"2024-04-25T09:55:33.985408+0000","last_unstale":"2024-04-25T09:55:34.999190+0000","last_undegraded":"2024-04-25T09:55:34.999190+0000","last_fullsized":"2024-04-25T09:55:34.999190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:11.974352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004064+0000","last_change":"2024-04-25T09:55:35.000267+0000","last_active":"2024-04-25T09:55:35.004064+0000","last_peered":"2024-04-25T09:55:35.004064+0000","last_clean":"2024-04-25T09:55:35.004064+0000","last_became_active":"2024-04-25T09:55:34.999953+0000","last_became_peered":"2024-04-25T09:55:34.999953+0000","last_unstale":"2024-04-25T09:55:35.004064+0000","last_undegraded":"2024-04-25T09:55:35.004064+0000","last_fullsized":"2024-04-25T09:55:35.004064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:26.184749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.994109+0000","last_change":"2024-04-25T09:55:33.990462+0000","last_active":"2024-04-25T09:55:34.994109+0000","last_peered":"2024-04-25T09:55:34.994109+0000","last_clean":"2024-04-25T09:55:34.994109+0000","last_became_active":"2024-04-25T09:55:33.990209+0000","last_became_peered":"2024-04-25T09:55:33.990209+0000","last_unstale":"2024-04-25T09:55:34.994109+0000","last_undegraded":"2024-04-25T09:55:34.994109+0000","last_fullsized":"2024-04-25T09:55:34.994109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:07.433844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001425+0000","last_change":"2024-04-25T09:55:34.997075+0000","last_active":"2024-04-25T09:55:35.001425+0000","last_peered":"2024-04-25T09:55:35.001425+0000","last_clean":"2024-04-25T09:55:35.001425+0000","last_became_active":"2024-04-25T09:55:34.996812+0000","last_became_peered":"2024-04-25T09:55:34.996812+0000","last_unstale":"2024-04-25T09:55:35.001425+0000","last_undegraded":"2024-04-25T09:55:35.001425+0000","last_fullsized":"2024-04-25T09:55:35.001425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.999500+0000","last_change":"2024-04-25T09:55:33.987566+0000","last_active":"2024-04-25T09:55:34.999500+0000","last_peered":"2024-04-25T09:55:34.999500+0000","last_clean":"2024-04-25T09:55:34.999500+0000","last_became_active":"2024-04-25T09:55:33.987311+0000","last_became_peered":"2024-04-25T09:55:33.987311+0000","last_unstale":"2024-04-25T09:55:34.999500+0000","last_undegraded":"2024-04-25T09:55:34.999500+0000","last_fullsized":"2024-04-25T09:55:34.999500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:14.842331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995309+0000","last_change":"2024-04-25T09:55:34.990488+0000","last_active":"2024-04-25T09:55:34.995309+0000","last_peered":"2024-04-25T09:55:34.995309+0000","last_clean":"2024-04-25T09:55:34.995309+0000","last_became_active":"2024-04-25T09:55:34.990222+0000","last_became_peered":"2024-04-25T09:55:34.990222+0000","last_unstale":"2024-04-25T09:55:34.995309+0000","last_undegraded":"2024-04-25T09:55:34.995309+0000","last_fullsized":"2024-04-25T09:55:34.995309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980732+0000","last_change":"2024-04-25T09:55:33.978187+0000","last_active":"2024-04-25T09:55:34.980732+0000","last_peered":"2024-04-25T09:55:34.980732+0000","last_clean":"2024-04-25T09:55:34.980732+0000","last_became_active":"2024-04-25T09:55:33.977894+0000","last_became_peered":"2024-04-25T09:55:33.977894+0000","last_unstale":"2024-04-25T09:55:34.980732+0000","last_undegraded":"2024-04-25T09:55:34.980732+0000","last_fullsized":"2024-04-25T09:55:34.980732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:42.818331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000095+0000","last_change":"2024-04-25T09:55:34.989516+0000","last_active":"2024-04-25T09:55:35.000095+0000","last_peered":"2024-04-25T09:55:35.000095+0000","last_clean":"2024-04-25T09:55:35.000095+0000","last_became_active":"2024-04-25T09:55:34.989220+0000","last_became_peered":"2024-04-25T09:55:34.989220+0000","last_unstale":"2024-04-25T09:55:35.000095+0000","last_undegraded":"2024-04-25T09:55:35.000095+0000","last_fullsized":"2024-04-25T09:55:35.000095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.981766+0000","last_change":"2024-04-25T09:55:33.988716+0000","last_active":"2024-04-25T09:55:34.981766+0000","last_peered":"2024-04-25T09:55:34.981766+0000","last_clean":"2024-04-25T09:55:34.981766+0000","last_became_active":"2024-04-25T09:55:33.988423+0000","last_became_peered":"2024-04-25T09:55:33.988423+0000","last_unstale":"2024-04-25T09:55:34.981766+0000","last_undegraded":"2024-04-25T09:55:34.981766+0000","last_fullsized":"2024-04-25T09:55:34.981766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:39.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.008017+0000","last_change":"2024-04-25T09:55:35.004735+0000","last_active":"2024-04-25T09:55:35.008017+0000","last_peered":"2024-04-25T09:55:35.008017+0000","last_clean":"2024-04-25T09:55:35.008017+0000","last_became_active":"2024-04-25T09:55:35.004116+0000","last_became_peered":"2024-04-25T09:55:35.004116+0000","last_unstale":"2024-04-25T09:55:35.008017+0000","last_undegraded":"2024-04-25T09:55:35.008017+0000","last_fullsized":"2024-04-25T09:55:35.008017+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:22.340985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980976+0000","last_change":"2024-04-25T09:55:33.987165+0000","last_active":"2024-04-25T09:55:34.980976+0000","last_peered":"2024-04-25T09:55:34.980976+0000","last_clean":"2024-04-25T09:55:34.980976+0000","last_became_active":"2024-04-25T09:55:33.986884+0000","last_became_peered":"2024-04-25T09:55:33.986884+0000","last_unstale":"2024-04-25T09:55:34.980976+0000","last_undegraded":"2024-04-25T09:55:34.980976+0000","last_fullsized":"2024-04-25T09:55:34.980976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:24.121708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006302+0000","last_change":"2024-04-25T09:55:35.005038+0000","last_active":"2024-04-25T09:55:35.006302+0000","last_peered":"2024-04-25T09:55:35.006302+0000","last_clean":"2024-04-25T09:55:35.006302+0000","last_became_active":"2024-04-25T09:55:35.004770+0000","last_became_peered":"2024-04-25T09:55:35.004770+0000","last_unstale":"2024-04-25T09:55:35.006302+0000","last_undegraded":"2024-04-25T09:55:35.006302+0000","last_fullsized":"2024-04-25T09:55:35.006302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000411+0000","last_change":"2024-04-25T09:55:33.991409+0000","last_active":"2024-04-25T09:55:35.000411+0000","last_peered":"2024-04-25T09:55:35.000411+0000","last_clean":"2024-04-25T09:55:35.000411+0000","last_became_active":"2024-04-25T09:55:33.991054+0000","last_became_peered":"2024-04-25T09:55:33.991054+0000","last_unstale":"2024-04-25T09:55:35.000411+0000","last_undegraded":"2024-04-25T09:55:35.000411+0000","last_fullsized":"2024-04-25T09:55:35.000411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:22.565667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997471+0000","last_change":"2024-04-25T09:55:34.990839+0000","last_active":"2024-04-25T09:55:34.997471+0000","last_peered":"2024-04-25T09:55:34.997471+0000","last_clean":"2024-04-25T09:55:34.997471+0000","last_became_active":"2024-04-25T09:55:34.990485+0000","last_became_peered":"2024-04-25T09:55:34.990485+0000","last_unstale":"2024-04-25T09:55:34.997471+0000","last_undegraded":"2024-04-25T09:55:34.997471+0000","last_fullsized":"2024-04-25T09:55:34.997471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:14.578831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.981479+0000","last_change":"2024-04-25T09:55:33.979697+0000","last_active":"2024-04-25T09:55:34.981479+0000","last_peered":"2024-04-25T09:55:34.981479+0000","last_clean":"2024-04-25T09:55:34.981479+0000","last_became_active":"2024-04-25T09:55:33.979428+0000","last_became_peered":"2024-04-25T09:55:33.979428+0000","last_unstale":"2024-04-25T09:55:34.981479+0000","last_undegraded":"2024-04-25T09:55:34.981479+0000","last_fullsized":"2024-04-25T09:55:34.981479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:20.317074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002129+0000","last_change":"2024-04-25T09:55:34.995172+0000","last_active":"2024-04-25T09:55:35.002129+0000","last_peered":"2024-04-25T09:55:35.002129+0000","last_clean":"2024-04-25T09:55:35.002129+0000","last_became_active":"2024-04-25T09:55:34.994915+0000","last_became_peered":"2024-04-25T09:55:34.994915+0000","last_unstale":"2024-04-25T09:55:35.002129+0000","last_undegraded":"2024-04-25T09:55:35.002129+0000","last_fullsized":"2024-04-25T09:55:35.002129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.902893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.982795+0000","last_change":"2024-04-25T09:55:33.991504+0000","last_active":"2024-04-25T09:55:34.982795+0000","last_peered":"2024-04-25T09:55:34.982795+0000","last_clean":"2024-04-25T09:55:34.982795+0000","last_became_active":"2024-04-25T09:55:33.991227+0000","last_became_peered":"2024-04-25T09:55:33.991227+0000","last_unstale":"2024-04-25T09:55:34.982795+0000","last_undegraded":"2024-04-25T09:55:34.982795+0000","last_fullsized":"2024-04-25T09:55:34.982795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:06.492702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001289+0000","last_change":"2024-04-25T09:55:34.996974+0000","last_active":"2024-04-25T09:55:35.001289+0000","last_peered":"2024-04-25T09:55:35.001289+0000","last_clean":"2024-04-25T09:55:35.001289+0000","last_became_active":"2024-04-25T09:55:34.996707+0000","last_became_peered":"2024-04-25T09:55:34.996707+0000","last_unstale":"2024-04-25T09:55:35.001289+0000","last_undegraded":"2024-04-25T09:55:35.001289+0000","last_fullsized":"2024-04-25T09:55:35.001289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:01.725669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.984041+0000","last_change":"2024-04-25T09:55:33.986830+0000","last_active":"2024-04-25T09:55:34.984041+0000","last_peered":"2024-04-25T09:55:34.984041+0000","last_clean":"2024-04-25T09:55:34.984041+0000","last_became_active":"2024-04-25T09:55:33.986526+0000","last_became_peered":"2024-04-25T09:55:33.986526+0000","last_unstale":"2024-04-25T09:55:34.984041+0000","last_undegraded":"2024-04-25T09:55:34.984041+0000","last_fullsized":"2024-04-25T09:55:34.984041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:39.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998603+0000","last_change":"2024-04-25T09:55:34.997229+0000","last_active":"2024-04-25T09:55:34.998603+0000","last_peered":"2024-04-25T09:55:34.998603+0000","last_clean":"2024-04-25T09:55:34.998603+0000","last_became_active":"2024-04-25T09:55:34.996966+0000","last_became_peered":"2024-04-25T09:55:34.996966+0000","last_unstale":"2024-04-25T09:55:34.998603+0000","last_undegraded":"2024-04-25T09:55:34.998603+0000","last_fullsized":"2024-04-25T09:55:34.998603+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.979154+0000","last_change":"2024-04-25T09:55:33.987722+0000","last_active":"2024-04-25T09:55:34.979154+0000","last_peered":"2024-04-25T09:55:34.979154+0000","last_clean":"2024-04-25T09:55:34.979154+0000","last_became_active":"2024-04-25T09:55:33.987222+0000","last_became_peered":"2024-04-25T09:55:33.987222+0000","last_unstale":"2024-04-25T09:55:34.979154+0000","last_undegraded":"2024-04-25T09:55:34.979154+0000","last_fullsized":"2024-04-25T09:55:34.979154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:58.688171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995730+0000","last_change":"2024-04-25T09:55:34.994148+0000","last_active":"2024-04-25T09:55:34.995730+0000","last_peered":"2024-04-25T09:55:34.995730+0000","last_clean":"2024-04-25T09:55:34.995730+0000","last_became_active":"2024-04-25T09:55:34.993866+0000","last_became_peered":"2024-04-25T09:55:34.993866+0000","last_unstale":"2024-04-25T09:55:34.995730+0000","last_undegraded":"2024-04-25T09:55:34.995730+0000","last_fullsized":"2024-04-25T09:55:34.995730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.974377+0000","last_change":"2024-04-25T09:55:33.989309+0000","last_active":"2024-04-25T09:55:34.974377+0000","last_peered":"2024-04-25T09:55:34.974377+0000","last_clean":"2024-04-25T09:55:34.974377+0000","last_became_active":"2024-04-25T09:55:33.989007+0000","last_became_peered":"2024-04-25T09:55:33.989007+0000","last_unstale":"2024-04-25T09:55:34.974377+0000","last_undegraded":"2024-04-25T09:55:34.974377+0000","last_fullsized":"2024-04-25T09:55:34.974377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:00.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.994639+0000","last_change":"2024-04-25T09:55:34.989680+0000","last_active":"2024-04-25T09:55:34.994639+0000","last_peered":"2024-04-25T09:55:34.994639+0000","last_clean":"2024-04-25T09:55:34.994639+0000","last_became_active":"2024-04-25T09:55:34.989391+0000","last_became_peered":"2024-04-25T09:55:34.989391+0000","last_unstale":"2024-04-25T09:55:34.994639+0000","last_undegraded":"2024-04-25T09:55:34.994639+0000","last_fullsized":"2024-04-25T09:55:34.994639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.981680+0000","last_change":"2024-04-25T09:55:33.990416+0000","last_active":"2024-04-25T09:55:34.981680+0000","last_peered":"2024-04-25T09:55:34.981680+0000","last_clean":"2024-04-25T09:55:34.981680+0000","last_became_active":"2024-04-25T09:55:33.989590+0000","last_became_peered":"2024-04-25T09:55:33.989590+0000","last_unstale":"2024-04-25T09:55:34.981680+0000","last_undegraded":"2024-04-25T09:55:34.981680+0000","last_fullsized":"2024-04-25T09:55:34.981680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:05.724588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004168+0000","last_change":"2024-04-25T09:55:34.999748+0000","last_active":"2024-04-25T09:55:35.004168+0000","last_peered":"2024-04-25T09:55:35.004168+0000","last_clean":"2024-04-25T09:55:35.004168+0000","last_became_active":"2024-04-25T09:55:34.999432+0000","last_became_peered":"2024-04-25T09:55:34.999432+0000","last_unstale":"2024-04-25T09:55:35.004168+0000","last_undegraded":"2024-04-25T09:55:35.004168+0000","last_fullsized":"2024-04-25T09:55:35.004168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:01.273548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980079+0000","last_change":"2024-04-25T09:55:33.992996+0000","last_active":"2024-04-25T09:55:34.980079+0000","last_peered":"2024-04-25T09:55:34.980079+0000","last_clean":"2024-04-25T09:55:34.980079+0000","last_became_active":"2024-04-25T09:55:33.992587+0000","last_became_peered":"2024-04-25T09:55:33.992587+0000","last_unstale":"2024-04-25T09:55:34.980079+0000","last_undegraded":"2024-04-25T09:55:34.980079+0000","last_fullsized":"2024-04-25T09:55:34.980079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:06.492702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.982035+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:55:34.982035+0000","last_peered":"2024-04-25T09:55:34.982035+0000","last_clean":"2024-04-25T09:55:34.982035+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:55:34.982035+0000","last_undegraded":"2024-04-25T09:55:34.982035+0000","last_fullsized":"2024-04-25T09:55:34.982035+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:37.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:55:38.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.064+0000 7fd0dae3a640 1 Processor -- start 2024-04-25T09:55:38.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.064+0000 7fd0dae3a640 1 -- start start 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0dae3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40cfec0 0x7fd0d40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0dae3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0d40ce3c0 0x7fd0d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0dae3a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d4071480 con 0x7fd0d4154ee0 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0dae3a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d4072090 con 0x7fd0d40ce3c0 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0dae3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4071230 con 0x7fd0d40cfec0 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d93d7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0d40ce3c0 0x7fd0d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d8bd6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd0d4154ee0 0x7fd0d40cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37240/0 (socket says 172.21.15.148:37240) 2024-04-25T09:55:38.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d8bd6640 1 -- 172.21.15.148:0/1312769995 learned_addr learned my addr 172.21.15.148:0/1312769995 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d3fff640 1 --2- 172.21.15.148:0/1312769995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40cfec0 0x7fd0d40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d93d7640 1 -- 172.21.15.148:0/1312769995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40cfec0 msgr2=0x7fd0d40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d93d7640 1 --2- 172.21.15.148:0/1312769995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40cfec0 0x7fd0d40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d93d7640 1 -- 172.21.15.148:0/1312769995 >> v1:172.21.15.148:6789/0 conn(0x7fd0d4154ee0 legacy=0x7fd0d40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d93d7640 1 -- 172.21.15.148:0/1312769995 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d4071760 con 0x7fd0d40ce3c0 2024-04-25T09:55:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.065+0000 7fd0d37fe640 1 -- 172.21.15.148:0/1312769995 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389668824 0 0) 0x7fd0d4071480 con 0x7fd0d4154ee0 2024-04-25T09:55:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.066+0000 7fd0d93d7640 1 --2- 172.21.15.148:0/1312769995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40ce3c0 0x7fd0d406d6f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009d90 tx=0x7fd0cc02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621eefb7319d9cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.066+0000 7fd0d37fe640 1 -- 172.21.15.148:0/1312769995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc032770 con 0x7fd0d40ce3c0 2024-04-25T09:55:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.066+0000 7fd0d37fe640 1 -- 172.21.15.148:0/1312769995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc038070 con 0x7fd0d40ce3c0 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.066+0000 7fd0d37fe640 1 -- 172.21.15.148:0/1312769995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc032b30 con 0x7fd0d40ce3c0 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 -- 172.21.15.148:0/1312769995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40ce3c0 msgr2=0x7fd0d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/1312769995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40ce3c0 0x7fd0d406d6f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009d90 tx=0x7fd0cc02c870 comp rx=0 tx=0).stop 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 -- 172.21.15.148:0/1312769995 shutdown_connections 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/1312769995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40ce3c0 0x7fd0d406d6f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/1312769995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40cfec0 0x7fd0d40cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 -- 172.21.15.148:0/1312769995 >> 172.21.15.148:0/1312769995 conn(0x7fd0d406abe0 msgr2=0x7fd0d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 -- 172.21.15.148:0/1312769995 shutdown_connections 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.067+0000 7fd0dae3a640 1 -- 172.21.15.148:0/1312769995 wait complete. 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.068+0000 7fd0dae3a640 1 Processor -- start 2024-04-25T09:55:38.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.068+0000 7fd0dae3a640 1 -- start start 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0dae3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40ce3c0 0x7fd0d40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0dae3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 0x7fd0d40a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0dae3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d4154ee0 0x7fd0d40a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0dae3a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4071d20 con 0x7fd0d40cfec0 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0dae3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40ce800 con 0x7fd0d4154ee0 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0dae3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d41552c0 con 0x7fd0d40ce3c0 2024-04-25T09:55:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 0x7fd0d40a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 0x7fd0d40a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58586/0 (socket says 172.21.15.148:58586) 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 -- 172.21.15.148:0/3711471549 learned_addr learned my addr 172.21.15.148:0/3711471549 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d93d7640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d4154ee0 0x7fd0d40a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 -- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40ce3c0 msgr2=0x7fd0d40ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d8bd6640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40ce3c0 0x7fd0d40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40ce3c0 0x7fd0d40ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 -- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d4154ee0 msgr2=0x7fd0d40a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d4154ee0 0x7fd0d40a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 -- 172.21.15.148:0/3711471549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d40aab20 con 0x7fd0d40cfec0 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d93d7640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d4154ee0 0x7fd0d40a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d3fff640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 0x7fd0d40a9040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c820 tx=0x7fd0c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.069+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8015070 con 0x7fd0d40cfec0 2024-04-25T09:55:38.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.070+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8004240 con 0x7fd0d40cfec0 2024-04-25T09:55:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.070+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d40a9de0 con 0x7fd0d40cfec0 2024-04-25T09:55:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.070+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800ce10 con 0x7fd0d40cfec0 2024-04-25T09:55:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.070+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d4146240 con 0x7fd0d40cfec0 2024-04-25T09:55:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.071+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d4073990 con 0x7fd0d40cfec0 2024-04-25T09:55:38.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.071+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0c8007830 con 0x7fd0d40cfec0 2024-04-25T09:55:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.071+0000 7fd0d17fa640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd0c006b4d0 0x7fd0c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.071+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0c808a260 con 0x7fd0d40cfec0 2024-04-25T09:55:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.071+0000 7fd0d8bd6640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd0c006b4d0 0x7fd0c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.072+0000 7fd0d8bd6640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd0c006b4d0 0x7fd0c006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4000f60 tx=0x7fd0c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.075+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0d40aab20 con 0x7fd0d40cfec0 2024-04-25T09:55:38.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.245+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0d406d360 con 0x7fd0c006b4d0 2024-04-25T09:55:38.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.251+0000 7fd0d17fa640 1 -- 172.21.15.148:0/3711471549 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251884 (secure 0 0 0) 0x7fd0d406d360 con 0x7fd0c006b4d0 2024-04-25T09:55:38.257 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:38.258 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:55:38.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.254+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd0c006b4d0 msgr2=0x7fd0c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.254+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd0c006b4d0 0x7fd0c006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4000f60 tx=0x7fd0c4009040 comp rx=0 tx=0).stop 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.254+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 msgr2=0x7fd0d40a9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.254+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 0x7fd0d40a9040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c820 tx=0x7fd0c800cce0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 shutdown_connections 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd0c006b4d0 0x7fd0c006d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d4154ee0 0x7fd0d40a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0d40cfec0 0x7fd0d40a9040 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 --2- 172.21.15.148:0/3711471549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d40ce3c0 0x7fd0d40ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 >> 172.21.15.148:0/3711471549 conn(0x7fd0d406abe0 msgr2=0x7fd0d415d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 shutdown_connections 2024-04-25T09:55:38.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.255+0000 7fd0dae3a640 1 -- 172.21.15.148:0/3711471549 wait complete. 2024-04-25T09:55:38.268 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-25T09:55:37.034882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220732,"kb_used_data":4532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753988,"statfs":{"total":773094113280,"available":772868083712,"internally_reserved":0,"allocated":4640768,"data_stored":2663506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.073134"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005709+0000","last_change":"2024-04-25T09:55:34.998058+0000","last_active":"2024-04-25T09:55:35.005709+0000","last_peered":"2024-04-25T09:55:35.005709+0000","last_clean":"2024-04-25T09:55:35.005709+0000","last_became_active":"2024-04-25T09:55:34.997762+0000","last_became_peered":"2024-04-25T09:55:34.997762+0000","last_unstale":"2024-04-25T09:55:35.005709+0000","last_undegraded":"2024-04-25T09:55:35.005709+0000","last_fullsized":"2024-04-25T09:55:35.005709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:01.273548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.007976+0000","last_change":"2024-04-25T09:55:35.000950+0000","last_active":"2024-04-25T09:55:35.007976+0000","last_peered":"2024-04-25T09:55:35.007976+0000","last_clean":"2024-04-25T09:55:35.007976+0000","last_became_active":"2024-04-25T09:55:35.000684+0000","last_became_peered":"2024-04-25T09:55:35.000684+0000","last_unstale":"2024-04-25T09:55:35.007976+0000","last_undegraded":"2024-04-25T09:55:35.007976+0000","last_fullsized":"2024-04-25T09:55:35.007976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:53.496715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005649+0000","last_change":"2024-04-25T09:55:35.003813+0000","last_active":"2024-04-25T09:55:35.005649+0000","last_peered":"2024-04-25T09:55:35.005649+0000","last_clean":"2024-04-25T09:55:35.005649+0000","last_became_active":"2024-04-25T09:55:35.001566+0000","last_became_peered":"2024-04-25T09:55:35.001566+0000","last_unstale":"2024-04-25T09:55:35.005649+0000","last_undegraded":"2024-04-25T09:55:35.005649+0000","last_fullsized":"2024-04-25T09:55:35.005649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995849+0000","last_change":"2024-04-25T09:55:34.989140+0000","last_active":"2024-04-25T09:55:34.995849+0000","last_peered":"2024-04-25T09:55:34.995849+0000","last_clean":"2024-04-25T09:55:34.995849+0000","last_became_active":"2024-04-25T09:55:34.988837+0000","last_became_peered":"2024-04-25T09:55:34.988837+0000","last_unstale":"2024-04-25T09:55:34.995849+0000","last_undegraded":"2024-04-25T09:55:34.995849+0000","last_fullsized":"2024-04-25T09:55:34.995849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:14.451216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.989057+0000","last_change":"2024-04-25T09:55:34.987909+0000","last_active":"2024-04-25T09:55:34.989057+0000","last_peered":"2024-04-25T09:55:34.989057+0000","last_clean":"2024-04-25T09:55:34.989057+0000","last_became_active":"2024-04-25T09:55:34.987638+0000","last_became_peered":"2024-04-25T09:55:34.987638+0000","last_unstale":"2024-04-25T09:55:34.989057+0000","last_undegraded":"2024-04-25T09:55:34.989057+0000","last_fullsized":"2024-04-25T09:55:34.989057+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:12.889091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006455+0000","last_change":"2024-04-25T09:55:35.004395+0000","last_active":"2024-04-25T09:55:35.006455+0000","last_peered":"2024-04-25T09:55:35.006455+0000","last_clean":"2024-04-25T09:55:35.006455+0000","last_became_active":"2024-04-25T09:55:35.004131+0000","last_became_peered":"2024-04-25T09:55:35.004131+0000","last_unstale":"2024-04-25T09:55:35.006455+0000","last_undegraded":"2024-04-25T09:55:35.006455+0000","last_fullsized":"2024-04-25T09:55:35.006455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000474+0000","last_change":"2024-04-25T09:55:34.993774+0000","last_active":"2024-04-25T09:55:35.000474+0000","last_peered":"2024-04-25T09:55:35.000474+0000","last_clean":"2024-04-25T09:55:35.000474+0000","last_became_active":"2024-04-25T09:55:34.993499+0000","last_became_peered":"2024-04-25T09:55:34.993499+0000","last_unstale":"2024-04-25T09:55:35.000474+0000","last_undegraded":"2024-04-25T09:55:35.000474+0000","last_fullsized":"2024-04-25T09:55:35.000474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:20.783934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.007904+0000","last_change":"2024-04-25T09:55:35.003423+0000","last_active":"2024-04-25T09:55:35.007904+0000","last_peered":"2024-04-25T09:55:35.007904+0000","last_clean":"2024-04-25T09:55:35.007904+0000","last_became_active":"2024-04-25T09:55:35.003147+0000","last_became_peered":"2024-04-25T09:55:35.003147+0000","last_unstale":"2024-04-25T09:55:35.007904+0000","last_undegraded":"2024-04-25T09:55:35.007904+0000","last_fullsized":"2024-04-25T09:55:35.007904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:48.116086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005387+0000","last_change":"2024-04-25T09:55:35.002911+0000","last_active":"2024-04-25T09:55:35.005387+0000","last_peered":"2024-04-25T09:55:35.005387+0000","last_clean":"2024-04-25T09:55:35.005387+0000","last_became_active":"2024-04-25T09:55:35.002313+0000","last_became_peered":"2024-04-25T09:55:35.002313+0000","last_unstale":"2024-04-25T09:55:35.005387+0000","last_undegraded":"2024-04-25T09:55:35.005387+0000","last_fullsized":"2024-04-25T09:55:35.005387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000052+0000","last_change":"2024-04-25T09:55:34.989183+0000","last_active":"2024-04-25T09:55:35.000052+0000","last_peered":"2024-04-25T09:55:35.000052+0000","last_clean":"2024-04-25T09:55:35.000052+0000","last_became_active":"2024-04-25T09:55:34.988894+0000","last_became_peered":"2024-04-25T09:55:34.988894+0000","last_unstale":"2024-04-25T09:55:35.000052+0000","last_undegraded":"2024-04-25T09:55:35.000052+0000","last_fullsized":"2024-04-25T09:55:35.000052+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000537+0000","last_change":"2024-04-25T09:55:34.988436+0000","last_active":"2024-04-25T09:55:35.000537+0000","last_peered":"2024-04-25T09:55:35.000537+0000","last_clean":"2024-04-25T09:55:35.000537+0000","last_became_active":"2024-04-25T09:55:34.988190+0000","last_became_peered":"2024-04-25T09:55:34.988190+0000","last_unstale":"2024-04-25T09:55:35.000537+0000","last_undegraded":"2024-04-25T09:55:35.000537+0000","last_fullsized":"2024-04-25T09:55:35.000537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995203+0000","last_change":"2024-04-25T09:55:34.991544+0000","last_active":"2024-04-25T09:55:34.995203+0000","last_peered":"2024-04-25T09:55:34.995203+0000","last_clean":"2024-04-25T09:55:34.995203+0000","last_became_active":"2024-04-25T09:55:34.991265+0000","last_became_peered":"2024-04-25T09:55:34.991265+0000","last_unstale":"2024-04-25T09:55:34.995203+0000","last_undegraded":"2024-04-25T09:55:34.995203+0000","last_fullsized":"2024-04-25T09:55:34.995203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:08.435853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000524+0000","last_change":"2024-04-25T09:55:34.987801+0000","last_active":"2024-04-25T09:55:35.000524+0000","last_peered":"2024-04-25T09:55:35.000524+0000","last_clean":"2024-04-25T09:55:35.000524+0000","last_became_active":"2024-04-25T09:55:34.987541+0000","last_became_peered":"2024-04-25T09:55:34.987541+0000","last_unstale":"2024-04-25T09:55:35.000524+0000","last_undegraded":"2024-04-25T09:55:35.000524+0000","last_fullsized":"2024-04-25T09:55:35.000524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991204+0000","last_change":"2024-04-25T09:55:34.985185+0000","last_active":"2024-04-25T09:55:34.991204+0000","last_peered":"2024-04-25T09:55:34.991204+0000","last_clean":"2024-04-25T09:55:34.991204+0000","last_became_active":"2024-04-25T09:55:34.984913+0000","last_became_peered":"2024-04-25T09:55:34.984913+0000","last_unstale":"2024-04-25T09:55:34.991204+0000","last_undegraded":"2024-04-25T09:55:34.991204+0000","last_fullsized":"2024-04-25T09:55:34.991204+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000453+0000","last_change":"2024-04-25T09:55:34.990337+0000","last_active":"2024-04-25T09:55:35.000453+0000","last_peered":"2024-04-25T09:55:35.000453+0000","last_clean":"2024-04-25T09:55:35.000453+0000","last_became_active":"2024-04-25T09:55:34.990083+0000","last_became_peered":"2024-04-25T09:55:34.990083+0000","last_unstale":"2024-04-25T09:55:35.000453+0000","last_undegraded":"2024-04-25T09:55:35.000453+0000","last_fullsized":"2024-04-25T09:55:35.000453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004811+0000","last_change":"2024-04-25T09:55:35.001561+0000","last_active":"2024-04-25T09:55:35.004811+0000","last_peered":"2024-04-25T09:55:35.004811+0000","last_clean":"2024-04-25T09:55:35.004811+0000","last_became_active":"2024-04-25T09:55:35.001257+0000","last_became_peered":"2024-04-25T09:55:35.001257+0000","last_unstale":"2024-04-25T09:55:35.004811+0000","last_undegraded":"2024-04-25T09:55:35.004811+0000","last_fullsized":"2024-04-25T09:55:35.004811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.003954+0000","last_change":"2024-04-25T09:55:35.000095+0000","last_active":"2024-04-25T09:55:35.003954+0000","last_peered":"2024-04-25T09:55:35.003954+0000","last_clean":"2024-04-25T09:55:35.003954+0000","last_became_active":"2024-04-25T09:55:34.999778+0000","last_became_peered":"2024-04-25T09:55:34.999778+0000","last_unstale":"2024-04-25T09:55:35.003954+0000","last_undegraded":"2024-04-25T09:55:35.003954+0000","last_fullsized":"2024-04-25T09:55:35.003954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998632+0000","last_change":"2024-04-25T09:55:34.997194+0000","last_active":"2024-04-25T09:55:34.998632+0000","last_peered":"2024-04-25T09:55:34.998632+0000","last_clean":"2024-04-25T09:55:34.998632+0000","last_became_active":"2024-04-25T09:55:34.996937+0000","last_became_peered":"2024-04-25T09:55:34.996937+0000","last_unstale":"2024-04-25T09:55:34.998632+0000","last_undegraded":"2024-04-25T09:55:34.998632+0000","last_fullsized":"2024-04-25T09:55:34.998632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005914+0000","last_change":"2024-04-25T09:55:35.002485+0000","last_active":"2024-04-25T09:55:35.005914+0000","last_peered":"2024-04-25T09:55:35.005914+0000","last_clean":"2024-04-25T09:55:35.005914+0000","last_became_active":"2024-04-25T09:55:35.001630+0000","last_became_peered":"2024-04-25T09:55:35.001630+0000","last_unstale":"2024-04-25T09:55:35.005914+0000","last_undegraded":"2024-04-25T09:55:35.005914+0000","last_fullsized":"2024-04-25T09:55:35.005914+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:42.695824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997639+0000","last_change":"2024-04-25T09:55:34.995872+0000","last_active":"2024-04-25T09:55:34.997639+0000","last_peered":"2024-04-25T09:55:34.997639+0000","last_clean":"2024-04-25T09:55:34.997639+0000","last_became_active":"2024-04-25T09:55:34.995508+0000","last_became_peered":"2024-04-25T09:55:34.995508+0000","last_unstale":"2024-04-25T09:55:34.997639+0000","last_undegraded":"2024-04-25T09:55:34.997639+0000","last_fullsized":"2024-04-25T09:55:34.997639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:20.981250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004064+0000","last_change":"2024-04-25T09:55:34.999549+0000","last_active":"2024-04-25T09:55:35.004064+0000","last_peered":"2024-04-25T09:55:35.004064+0000","last_clean":"2024-04-25T09:55:35.004064+0000","last_became_active":"2024-04-25T09:55:34.999262+0000","last_became_peered":"2024-04-25T09:55:34.999262+0000","last_unstale":"2024-04-25T09:55:35.004064+0000","last_undegraded":"2024-04-25T09:55:35.004064+0000","last_fullsized":"2024-04-25T09:55:35.004064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:36.816221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991258+0000","last_change":"2024-04-25T09:55:34.986306+0000","last_active":"2024-04-25T09:55:34.991258+0000","last_peered":"2024-04-25T09:55:34.991258+0000","last_clean":"2024-04-25T09:55:34.991258+0000","last_became_active":"2024-04-25T09:55:34.986024+0000","last_became_peered":"2024-04-25T09:55:34.986024+0000","last_unstale":"2024-04-25T09:55:34.991258+0000","last_undegraded":"2024-04-25T09:55:34.991258+0000","last_fullsized":"2024-04-25T09:55:34.991258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998336+0000","last_change":"2024-04-25T09:55:34.994839+0000","last_active":"2024-04-25T09:55:34.998336+0000","last_peered":"2024-04-25T09:55:34.998336+0000","last_clean":"2024-04-25T09:55:34.998336+0000","last_became_active":"2024-04-25T09:55:34.994499+0000","last_became_peered":"2024-04-25T09:55:34.994499+0000","last_unstale":"2024-04-25T09:55:34.998336+0000","last_undegraded":"2024-04-25T09:55:34.998336+0000","last_fullsized":"2024-04-25T09:55:34.998336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:04.584156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995665+0000","last_change":"2024-04-25T09:55:34.990426+0000","last_active":"2024-04-25T09:55:34.995665+0000","last_peered":"2024-04-25T09:55:34.995665+0000","last_clean":"2024-04-25T09:55:34.995665+0000","last_became_active":"2024-04-25T09:55:34.990169+0000","last_became_peered":"2024-04-25T09:55:34.990169+0000","last_unstale":"2024-04-25T09:55:34.995665+0000","last_undegraded":"2024-04-25T09:55:34.995665+0000","last_fullsized":"2024-04-25T09:55:34.995665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:58.440534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.989078+0000","last_change":"2024-04-25T09:55:34.987332+0000","last_active":"2024-04-25T09:55:34.989078+0000","last_peered":"2024-04-25T09:55:34.989078+0000","last_clean":"2024-04-25T09:55:34.989078+0000","last_became_active":"2024-04-25T09:55:34.987036+0000","last_became_peered":"2024-04-25T09:55:34.987036+0000","last_unstale":"2024-04-25T09:55:34.989078+0000","last_undegraded":"2024-04-25T09:55:34.989078+0000","last_fullsized":"2024-04-25T09:55:34.989078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:06.981986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006263+0000","last_change":"2024-04-25T09:55:35.002415+0000","last_active":"2024-04-25T09:55:35.006263+0000","last_peered":"2024-04-25T09:55:35.006263+0000","last_clean":"2024-04-25T09:55:35.006263+0000","last_became_active":"2024-04-25T09:55:35.002161+0000","last_became_peered":"2024-04-25T09:55:35.002161+0000","last_unstale":"2024-04-25T09:55:35.006263+0000","last_undegraded":"2024-04-25T09:55:35.006263+0000","last_fullsized":"2024-04-25T09:55:35.006263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.992291+0000","last_change":"2024-04-25T09:55:34.990711+0000","last_active":"2024-04-25T09:55:34.992291+0000","last_peered":"2024-04-25T09:55:34.992291+0000","last_clean":"2024-04-25T09:55:34.992291+0000","last_became_active":"2024-04-25T09:55:34.990449+0000","last_became_peered":"2024-04-25T09:55:34.990449+0000","last_unstale":"2024-04-25T09:55:34.992291+0000","last_undegraded":"2024-04-25T09:55:34.992291+0000","last_fullsized":"2024-04-25T09:55:34.992291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:07.494711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000573+0000","last_change":"2024-04-25T09:55:34.997116+0000","last_active":"2024-04-25T09:55:35.000573+0000","last_peered":"2024-04-25T09:55:35.000573+0000","last_clean":"2024-04-25T09:55:35.000573+0000","last_became_active":"2024-04-25T09:55:34.996585+0000","last_became_peered":"2024-04-25T09:55:34.996585+0000","last_unstale":"2024-04-25T09:55:35.000573+0000","last_undegraded":"2024-04-25T09:55:35.000573+0000","last_fullsized":"2024-04-25T09:55:35.000573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.902893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000414+0000","last_change":"2024-04-25T09:55:34.998486+0000","last_active":"2024-04-25T09:55:35.000414+0000","last_peered":"2024-04-25T09:55:35.000414+0000","last_clean":"2024-04-25T09:55:35.000414+0000","last_became_active":"2024-04-25T09:55:34.998204+0000","last_became_peered":"2024-04-25T09:55:34.998204+0000","last_unstale":"2024-04-25T09:55:35.000414+0000","last_undegraded":"2024-04-25T09:55:35.000414+0000","last_fullsized":"2024-04-25T09:55:35.000414+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:08.435853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.996740+0000","last_change":"2024-04-25T09:55:34.995266+0000","last_active":"2024-04-25T09:55:34.996740+0000","last_peered":"2024-04-25T09:55:34.996740+0000","last_clean":"2024-04-25T09:55:34.996740+0000","last_became_active":"2024-04-25T09:55:34.994988+0000","last_became_peered":"2024-04-25T09:55:34.994988+0000","last_unstale":"2024-04-25T09:55:34.996740+0000","last_undegraded":"2024-04-25T09:55:34.996740+0000","last_fullsized":"2024-04-25T09:55:34.996740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:40.325973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998358+0000","last_change":"2024-04-25T09:55:34.995336+0000","last_active":"2024-04-25T09:55:34.998358+0000","last_peered":"2024-04-25T09:55:34.998358+0000","last_clean":"2024-04-25T09:55:34.998358+0000","last_became_active":"2024-04-25T09:55:34.995077+0000","last_became_peered":"2024-04-25T09:55:34.995077+0000","last_unstale":"2024-04-25T09:55:34.998358+0000","last_undegraded":"2024-04-25T09:55:34.998358+0000","last_fullsized":"2024-04-25T09:55:34.998358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.010639+0000","last_change":"2024-04-25T09:55:35.006392+0000","last_active":"2024-04-25T09:55:35.010639+0000","last_peered":"2024-04-25T09:55:35.010639+0000","last_clean":"2024-04-25T09:55:35.010639+0000","last_became_active":"2024-04-25T09:55:35.005962+0000","last_became_peered":"2024-04-25T09:55:35.005962+0000","last_unstale":"2024-04-25T09:55:35.010639+0000","last_undegraded":"2024-04-25T09:55:35.010639+0000","last_fullsized":"2024-04-25T09:55:35.010639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004003+0000","last_change":"2024-04-25T09:55:35.000104+0000","last_active":"2024-04-25T09:55:35.004003+0000","last_peered":"2024-04-25T09:55:35.004003+0000","last_clean":"2024-04-25T09:55:35.004003+0000","last_became_active":"2024-04-25T09:55:34.999833+0000","last_became_peered":"2024-04-25T09:55:34.999833+0000","last_unstale":"2024-04-25T09:55:35.004003+0000","last_undegraded":"2024-04-25T09:55:35.004003+0000","last_fullsized":"2024-04-25T09:55:35.004003+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002113+0000","last_change":"2024-04-25T09:55:33.980326+0000","last_active":"2024-04-25T09:55:35.002113+0000","last_peered":"2024-04-25T09:55:35.002113+0000","last_clean":"2024-04-25T09:55:35.002113+0000","last_became_active":"2024-04-25T09:55:33.980059+0000","last_became_peered":"2024-04-25T09:55:33.980059+0000","last_unstale":"2024-04-25T09:55:35.002113+0000","last_undegraded":"2024-04-25T09:55:35.002113+0000","last_fullsized":"2024-04-25T09:55:35.002113+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.008118+0000","last_change":"2024-04-25T09:55:35.005447+0000","last_active":"2024-04-25T09:55:35.008118+0000","last_peered":"2024-04-25T09:55:35.008118+0000","last_clean":"2024-04-25T09:55:35.008118+0000","last_became_active":"2024-04-25T09:55:35.004996+0000","last_became_peered":"2024-04-25T09:55:35.004996+0000","last_unstale":"2024-04-25T09:55:35.008118+0000","last_undegraded":"2024-04-25T09:55:35.008118+0000","last_fullsized":"2024-04-25T09:55:35.008118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001948+0000","last_change":"2024-04-25T09:55:33.988511+0000","last_active":"2024-04-25T09:55:35.001948+0000","last_peered":"2024-04-25T09:55:35.001948+0000","last_clean":"2024-04-25T09:55:35.001948+0000","last_became_active":"2024-04-25T09:55:33.988206+0000","last_became_peered":"2024-04-25T09:55:33.988206+0000","last_unstale":"2024-04-25T09:55:35.001948+0000","last_undegraded":"2024-04-25T09:55:35.001948+0000","last_fullsized":"2024-04-25T09:55:35.001948+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:19.979241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991101+0000","last_change":"2024-04-25T09:55:34.985951+0000","last_active":"2024-04-25T09:55:34.991101+0000","last_peered":"2024-04-25T09:55:34.991101+0000","last_clean":"2024-04-25T09:55:34.991101+0000","last_became_active":"2024-04-25T09:55:34.985681+0000","last_became_peered":"2024-04-25T09:55:34.985681+0000","last_unstale":"2024-04-25T09:55:34.991101+0000","last_undegraded":"2024-04-25T09:55:34.991101+0000","last_fullsized":"2024-04-25T09:55:34.991101+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:06.726597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006781+0000","last_change":"2024-04-25T09:55:33.991186+0000","last_active":"2024-04-25T09:55:35.006781+0000","last_peered":"2024-04-25T09:55:35.006781+0000","last_clean":"2024-04-25T09:55:35.006781+0000","last_became_active":"2024-04-25T09:55:33.990904+0000","last_became_peered":"2024-04-25T09:55:33.990904+0000","last_unstale":"2024-04-25T09:55:35.006781+0000","last_undegraded":"2024-04-25T09:55:35.006781+0000","last_fullsized":"2024-04-25T09:55:35.006781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.582147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005731+0000","last_change":"2024-04-25T09:55:34.999727+0000","last_active":"2024-04-25T09:55:35.005731+0000","last_peered":"2024-04-25T09:55:35.005731+0000","last_clean":"2024-04-25T09:55:35.005731+0000","last_became_active":"2024-04-25T09:55:34.998967+0000","last_became_peered":"2024-04-25T09:55:34.998967+0000","last_unstale":"2024-04-25T09:55:35.005731+0000","last_undegraded":"2024-04-25T09:55:35.005731+0000","last_fullsized":"2024-04-25T09:55:35.005731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:15.133979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005653+0000","last_change":"2024-04-25T09:55:33.988594+0000","last_active":"2024-04-25T09:55:35.005653+0000","last_peered":"2024-04-25T09:55:35.005653+0000","last_clean":"2024-04-25T09:55:35.005653+0000","last_became_active":"2024-04-25T09:55:33.988321+0000","last_became_peered":"2024-04-25T09:55:33.988321+0000","last_unstale":"2024-04-25T09:55:35.005653+0000","last_undegraded":"2024-04-25T09:55:35.005653+0000","last_fullsized":"2024-04-25T09:55:35.005653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.576822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002499+0000","last_change":"2024-04-25T09:55:34.999393+0000","last_active":"2024-04-25T09:55:35.002499+0000","last_peered":"2024-04-25T09:55:35.002499+0000","last_clean":"2024-04-25T09:55:35.002499+0000","last_became_active":"2024-04-25T09:55:34.998891+0000","last_became_peered":"2024-04-25T09:55:34.998891+0000","last_unstale":"2024-04-25T09:55:35.002499+0000","last_undegraded":"2024-04-25T09:55:35.002499+0000","last_fullsized":"2024-04-25T09:55:35.002499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:04.584156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998903+0000","last_change":"2024-04-25T09:55:33.987385+0000","last_active":"2024-04-25T09:55:34.998903+0000","last_peered":"2024-04-25T09:55:34.998903+0000","last_clean":"2024-04-25T09:55:34.998903+0000","last_became_active":"2024-04-25T09:55:33.986702+0000","last_became_peered":"2024-04-25T09:55:33.986702+0000","last_unstale":"2024-04-25T09:55:34.998903+0000","last_undegraded":"2024-04-25T09:55:34.998903+0000","last_fullsized":"2024-04-25T09:55:34.998903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:07.433844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004190+0000","last_change":"2024-04-25T09:55:35.000315+0000","last_active":"2024-04-25T09:55:35.004190+0000","last_peered":"2024-04-25T09:55:35.004190+0000","last_clean":"2024-04-25T09:55:35.004190+0000","last_became_active":"2024-04-25T09:55:34.999725+0000","last_became_peered":"2024-04-25T09:55:34.999725+0000","last_unstale":"2024-04-25T09:55:35.004190+0000","last_undegraded":"2024-04-25T09:55:35.004190+0000","last_fullsized":"2024-04-25T09:55:35.004190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:39.381865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001843+0000","last_change":"2024-04-25T09:55:33.987960+0000","last_active":"2024-04-25T09:55:35.001843+0000","last_peered":"2024-04-25T09:55:35.001843+0000","last_clean":"2024-04-25T09:55:35.001843+0000","last_became_active":"2024-04-25T09:55:33.987681+0000","last_became_peered":"2024-04-25T09:55:33.987681+0000","last_unstale":"2024-04-25T09:55:35.001843+0000","last_undegraded":"2024-04-25T09:55:35.001843+0000","last_fullsized":"2024-04-25T09:55:35.001843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.979977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.991230+0000","last_change":"2024-04-25T09:55:34.985424+0000","last_active":"2024-04-25T09:55:34.991230+0000","last_peered":"2024-04-25T09:55:34.991230+0000","last_clean":"2024-04-25T09:55:34.991230+0000","last_became_active":"2024-04-25T09:55:34.985132+0000","last_became_peered":"2024-04-25T09:55:34.985132+0000","last_unstale":"2024-04-25T09:55:34.991230+0000","last_undegraded":"2024-04-25T09:55:34.991230+0000","last_fullsized":"2024-04-25T09:55:34.991230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995356+0000","last_change":"2024-04-25T09:55:33.988746+0000","last_active":"2024-04-25T09:55:34.995356+0000","last_peered":"2024-04-25T09:55:34.995356+0000","last_clean":"2024-04-25T09:55:34.995356+0000","last_became_active":"2024-04-25T09:55:33.988411+0000","last_became_peered":"2024-04-25T09:55:33.988411+0000","last_unstale":"2024-04-25T09:55:34.995356+0000","last_undegraded":"2024-04-25T09:55:34.995356+0000","last_fullsized":"2024-04-25T09:55:34.995356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:00.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.007997+0000","last_change":"2024-04-25T09:55:35.004752+0000","last_active":"2024-04-25T09:55:35.007997+0000","last_peered":"2024-04-25T09:55:35.007997+0000","last_clean":"2024-04-25T09:55:35.007997+0000","last_became_active":"2024-04-25T09:55:35.004483+0000","last_became_peered":"2024-04-25T09:55:35.004483+0000","last_unstale":"2024-04-25T09:55:35.007997+0000","last_undegraded":"2024-04-25T09:55:35.007997+0000","last_fullsized":"2024-04-25T09:55:35.007997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:38.173441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.974033+0000","last_change":"2024-04-25T09:55:33.981430+0000","last_active":"2024-04-25T09:55:34.974033+0000","last_peered":"2024-04-25T09:55:34.974033+0000","last_clean":"2024-04-25T09:55:34.974033+0000","last_became_active":"2024-04-25T09:55:33.981055+0000","last_became_peered":"2024-04-25T09:55:33.981055+0000","last_unstale":"2024-04-25T09:55:34.974033+0000","last_undegraded":"2024-04-25T09:55:34.974033+0000","last_fullsized":"2024-04-25T09:55:34.974033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:07.433844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995879+0000","last_change":"2024-04-25T09:55:34.991343+0000","last_active":"2024-04-25T09:55:34.995879+0000","last_peered":"2024-04-25T09:55:34.995879+0000","last_clean":"2024-04-25T09:55:34.995879+0000","last_became_active":"2024-04-25T09:55:34.990776+0000","last_became_peered":"2024-04-25T09:55:34.990776+0000","last_unstale":"2024-04-25T09:55:34.995879+0000","last_undegraded":"2024-04-25T09:55:34.995879+0000","last_fullsized":"2024-04-25T09:55:34.995879+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:12.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.987241+0000","last_change":"2024-04-25T09:55:33.990153+0000","last_active":"2024-04-25T09:55:34.987241+0000","last_peered":"2024-04-25T09:55:34.987241+0000","last_clean":"2024-04-25T09:55:34.987241+0000","last_became_active":"2024-04-25T09:55:33.989882+0000","last_became_peered":"2024-04-25T09:55:33.989882+0000","last_unstale":"2024-04-25T09:55:34.987241+0000","last_undegraded":"2024-04-25T09:55:34.987241+0000","last_fullsized":"2024-04-25T09:55:34.987241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:56.886723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997448+0000","last_change":"2024-04-25T09:55:34.990739+0000","last_active":"2024-04-25T09:55:34.997448+0000","last_peered":"2024-04-25T09:55:34.997448+0000","last_clean":"2024-04-25T09:55:34.997448+0000","last_became_active":"2024-04-25T09:55:34.990102+0000","last_became_peered":"2024-04-25T09:55:34.990102+0000","last_unstale":"2024-04-25T09:55:34.997448+0000","last_undegraded":"2024-04-25T09:55:34.997448+0000","last_fullsized":"2024-04-25T09:55:34.997448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:08.435853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980322+0000","last_change":"2024-04-25T09:55:33.992403+0000","last_active":"2024-04-25T09:55:34.980322+0000","last_peered":"2024-04-25T09:55:34.980322+0000","last_clean":"2024-04-25T09:55:34.980322+0000","last_became_active":"2024-04-25T09:55:33.992080+0000","last_became_peered":"2024-04-25T09:55:33.992080+0000","last_unstale":"2024-04-25T09:55:34.980322+0000","last_undegraded":"2024-04-25T09:55:34.980322+0000","last_fullsized":"2024-04-25T09:55:34.980322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:00.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998566+0000","last_change":"2024-04-25T09:55:34.993226+0000","last_active":"2024-04-25T09:55:34.998566+0000","last_peered":"2024-04-25T09:55:34.998566+0000","last_clean":"2024-04-25T09:55:34.998566+0000","last_became_active":"2024-04-25T09:55:34.992947+0000","last_became_peered":"2024-04-25T09:55:34.992947+0000","last_unstale":"2024-04-25T09:55:34.998566+0000","last_undegraded":"2024-04-25T09:55:34.998566+0000","last_fullsized":"2024-04-25T09:55:34.998566+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:14.578831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001068+0000","last_change":"2024-04-25T09:55:33.991563+0000","last_active":"2024-04-25T09:55:35.001068+0000","last_peered":"2024-04-25T09:55:35.001068+0000","last_clean":"2024-04-25T09:55:35.001068+0000","last_became_active":"2024-04-25T09:55:33.991265+0000","last_became_peered":"2024-04-25T09:55:33.991265+0000","last_unstale":"2024-04-25T09:55:35.001068+0000","last_undegraded":"2024-04-25T09:55:35.001068+0000","last_fullsized":"2024-04-25T09:55:35.001068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.576822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.993706+0000","last_change":"2024-04-25T09:55:34.991737+0000","last_active":"2024-04-25T09:55:34.993706+0000","last_peered":"2024-04-25T09:55:34.993706+0000","last_clean":"2024-04-25T09:55:34.993706+0000","last_became_active":"2024-04-25T09:55:34.991452+0000","last_became_peered":"2024-04-25T09:55:34.991452+0000","last_unstale":"2024-04-25T09:55:34.993706+0000","last_undegraded":"2024-04-25T09:55:34.993706+0000","last_fullsized":"2024-04-25T09:55:34.993706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.010726+0000","last_change":"2024-04-25T09:55:33.990233+0000","last_active":"2024-04-25T09:55:35.010726+0000","last_peered":"2024-04-25T09:55:35.010726+0000","last_clean":"2024-04-25T09:55:35.010726+0000","last_became_active":"2024-04-25T09:55:33.989949+0000","last_became_peered":"2024-04-25T09:55:33.989949+0000","last_unstale":"2024-04-25T09:55:35.010726+0000","last_undegraded":"2024-04-25T09:55:35.010726+0000","last_fullsized":"2024-04-25T09:55:35.010726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:32.424425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.003299+0000","last_change":"2024-04-25T09:55:35.001157+0000","last_active":"2024-04-25T09:55:35.003299+0000","last_peered":"2024-04-25T09:55:35.003299+0000","last_clean":"2024-04-25T09:55:35.003299+0000","last_became_active":"2024-04-25T09:55:35.000900+0000","last_became_peered":"2024-04-25T09:55:35.000900+0000","last_unstale":"2024-04-25T09:55:35.003299+0000","last_undegraded":"2024-04-25T09:55:35.003299+0000","last_fullsized":"2024-04-25T09:55:35.003299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000300+0000","last_change":"2024-04-25T09:55:33.991510+0000","last_active":"2024-04-25T09:55:35.000300+0000","last_peered":"2024-04-25T09:55:35.000300+0000","last_clean":"2024-04-25T09:55:35.000300+0000","last_became_active":"2024-04-25T09:55:33.991171+0000","last_became_peered":"2024-04-25T09:55:33.991171+0000","last_unstale":"2024-04-25T09:55:35.000300+0000","last_undegraded":"2024-04-25T09:55:35.000300+0000","last_fullsized":"2024-04-25T09:55:35.000300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:34.886125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.003824+0000","last_change":"2024-04-25T09:55:34.993387+0000","last_active":"2024-04-25T09:55:35.003824+0000","last_peered":"2024-04-25T09:55:35.003824+0000","last_clean":"2024-04-25T09:55:35.003824+0000","last_became_active":"2024-04-25T09:55:34.993069+0000","last_became_peered":"2024-04-25T09:55:34.993069+0000","last_unstale":"2024-04-25T09:55:35.003824+0000","last_undegraded":"2024-04-25T09:55:35.003824+0000","last_fullsized":"2024-04-25T09:55:35.003824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:20.783934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.994313+0000","last_change":"2024-04-25T09:55:33.989882+0000","last_active":"2024-04-25T09:55:34.994313+0000","last_peered":"2024-04-25T09:55:34.994313+0000","last_clean":"2024-04-25T09:55:34.994313+0000","last_became_active":"2024-04-25T09:55:33.989619+0000","last_became_peered":"2024-04-25T09:55:33.989619+0000","last_unstale":"2024-04-25T09:55:34.994313+0000","last_undegraded":"2024-04-25T09:55:34.994313+0000","last_fullsized":"2024-04-25T09:55:34.994313+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.576822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997355+0000","last_change":"2024-04-25T09:55:34.993510+0000","last_active":"2024-04-25T09:55:34.997355+0000","last_peered":"2024-04-25T09:55:34.997355+0000","last_clean":"2024-04-25T09:55:34.997355+0000","last_became_active":"2024-04-25T09:55:34.993214+0000","last_became_peered":"2024-04-25T09:55:34.993214+0000","last_unstale":"2024-04-25T09:55:34.997355+0000","last_undegraded":"2024-04-25T09:55:34.997355+0000","last_fullsized":"2024-04-25T09:55:34.997355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:01.725669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.978302+0000","last_change":"2024-04-25T09:55:33.980964+0000","last_active":"2024-04-25T09:55:34.978302+0000","last_peered":"2024-04-25T09:55:34.978302+0000","last_clean":"2024-04-25T09:55:34.978302+0000","last_became_active":"2024-04-25T09:55:33.980702+0000","last_became_peered":"2024-04-25T09:55:33.980702+0000","last_unstale":"2024-04-25T09:55:34.978302+0000","last_undegraded":"2024-04-25T09:55:34.978302+0000","last_fullsized":"2024-04-25T09:55:34.978302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:51.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005488+0000","last_change":"2024-04-25T09:55:34.997374+0000","last_active":"2024-04-25T09:55:35.005488+0000","last_peered":"2024-04-25T09:55:35.005488+0000","last_clean":"2024-04-25T09:55:35.005488+0000","last_became_active":"2024-04-25T09:55:34.997059+0000","last_became_peered":"2024-04-25T09:55:34.997059+0000","last_unstale":"2024-04-25T09:55:35.005488+0000","last_undegraded":"2024-04-25T09:55:35.005488+0000","last_fullsized":"2024-04-25T09:55:35.005488+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:36.816221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004870+0000","last_change":"2024-04-25T09:55:33.987327+0000","last_active":"2024-04-25T09:55:35.004870+0000","last_peered":"2024-04-25T09:55:35.004870+0000","last_clean":"2024-04-25T09:55:35.004870+0000","last_became_active":"2024-04-25T09:55:33.987063+0000","last_became_peered":"2024-04-25T09:55:33.987063+0000","last_unstale":"2024-04-25T09:55:35.004870+0000","last_undegraded":"2024-04-25T09:55:35.004870+0000","last_fullsized":"2024-04-25T09:55:35.004870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:19.979241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.005306+0000","last_change":"2024-04-25T09:55:34.998184+0000","last_active":"2024-04-25T09:55:35.005306+0000","last_peered":"2024-04-25T09:55:35.005306+0000","last_clean":"2024-04-25T09:55:35.005306+0000","last_became_active":"2024-04-25T09:55:34.997923+0000","last_became_peered":"2024-04-25T09:55:34.997923+0000","last_unstale":"2024-04-25T09:55:35.005306+0000","last_undegraded":"2024-04-25T09:55:35.005306+0000","last_fullsized":"2024-04-25T09:55:35.005306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:39.381865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000280+0000","last_change":"2024-04-25T09:55:33.981963+0000","last_active":"2024-04-25T09:55:35.000280+0000","last_peered":"2024-04-25T09:55:35.000280+0000","last_clean":"2024-04-25T09:55:35.000280+0000","last_became_active":"2024-04-25T09:55:33.981647+0000","last_became_peered":"2024-04-25T09:55:33.981647+0000","last_unstale":"2024-04-25T09:55:35.000280+0000","last_undegraded":"2024-04-25T09:55:35.000280+0000","last_fullsized":"2024-04-25T09:55:35.000280+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.582147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004952+0000","last_change":"2024-04-25T09:55:35.001579+0000","last_active":"2024-04-25T09:55:35.004952+0000","last_peered":"2024-04-25T09:55:35.004952+0000","last_clean":"2024-04-25T09:55:35.004952+0000","last_became_active":"2024-04-25T09:55:35.001312+0000","last_became_peered":"2024-04-25T09:55:35.001312+0000","last_unstale":"2024-04-25T09:55:35.004952+0000","last_undegraded":"2024-04-25T09:55:35.004952+0000","last_fullsized":"2024-04-25T09:55:35.004952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:15.133979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002726+0000","last_change":"2024-04-25T09:55:33.981893+0000","last_active":"2024-04-25T09:55:35.002726+0000","last_peered":"2024-04-25T09:55:35.002726+0000","last_clean":"2024-04-25T09:55:35.002726+0000","last_became_active":"2024-04-25T09:55:33.981615+0000","last_became_peered":"2024-04-25T09:55:33.981615+0000","last_unstale":"2024-04-25T09:55:35.002726+0000","last_undegraded":"2024-04-25T09:55:35.002726+0000","last_fullsized":"2024-04-25T09:55:35.002726+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:03.582147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004083+0000","last_change":"2024-04-25T09:55:34.999272+0000","last_active":"2024-04-25T09:55:35.004083+0000","last_peered":"2024-04-25T09:55:35.004083+0000","last_clean":"2024-04-25T09:55:35.004083+0000","last_became_active":"2024-04-25T09:55:34.998960+0000","last_became_peered":"2024-04-25T09:55:34.998960+0000","last_unstale":"2024-04-25T09:55:35.004083+0000","last_undegraded":"2024-04-25T09:55:35.004083+0000","last_fullsized":"2024-04-25T09:55:35.004083+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:20.783934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.999190+0000","last_change":"2024-04-25T09:55:33.985741+0000","last_active":"2024-04-25T09:55:34.999190+0000","last_peered":"2024-04-25T09:55:34.999190+0000","last_clean":"2024-04-25T09:55:34.999190+0000","last_became_active":"2024-04-25T09:55:33.985408+0000","last_became_peered":"2024-04-25T09:55:33.985408+0000","last_unstale":"2024-04-25T09:55:34.999190+0000","last_undegraded":"2024-04-25T09:55:34.999190+0000","last_fullsized":"2024-04-25T09:55:34.999190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:11.974352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004064+0000","last_change":"2024-04-25T09:55:35.000267+0000","last_active":"2024-04-25T09:55:35.004064+0000","last_peered":"2024-04-25T09:55:35.004064+0000","last_clean":"2024-04-25T09:55:35.004064+0000","last_became_active":"2024-04-25T09:55:34.999953+0000","last_became_peered":"2024-04-25T09:55:34.999953+0000","last_unstale":"2024-04-25T09:55:35.004064+0000","last_undegraded":"2024-04-25T09:55:35.004064+0000","last_fullsized":"2024-04-25T09:55:35.004064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:26.184749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.994109+0000","last_change":"2024-04-25T09:55:33.990462+0000","last_active":"2024-04-25T09:55:34.994109+0000","last_peered":"2024-04-25T09:55:34.994109+0000","last_clean":"2024-04-25T09:55:34.994109+0000","last_became_active":"2024-04-25T09:55:33.990209+0000","last_became_peered":"2024-04-25T09:55:33.990209+0000","last_unstale":"2024-04-25T09:55:34.994109+0000","last_undegraded":"2024-04-25T09:55:34.994109+0000","last_fullsized":"2024-04-25T09:55:34.994109+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:07.433844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001425+0000","last_change":"2024-04-25T09:55:34.997075+0000","last_active":"2024-04-25T09:55:35.001425+0000","last_peered":"2024-04-25T09:55:35.001425+0000","last_clean":"2024-04-25T09:55:35.001425+0000","last_became_active":"2024-04-25T09:55:34.996812+0000","last_became_peered":"2024-04-25T09:55:34.996812+0000","last_unstale":"2024-04-25T09:55:35.001425+0000","last_undegraded":"2024-04-25T09:55:35.001425+0000","last_fullsized":"2024-04-25T09:55:35.001425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:54.005356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.999500+0000","last_change":"2024-04-25T09:55:33.987566+0000","last_active":"2024-04-25T09:55:34.999500+0000","last_peered":"2024-04-25T09:55:34.999500+0000","last_clean":"2024-04-25T09:55:34.999500+0000","last_became_active":"2024-04-25T09:55:33.987311+0000","last_became_peered":"2024-04-25T09:55:33.987311+0000","last_unstale":"2024-04-25T09:55:34.999500+0000","last_undegraded":"2024-04-25T09:55:34.999500+0000","last_fullsized":"2024-04-25T09:55:34.999500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:14.842331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995309+0000","last_change":"2024-04-25T09:55:34.990488+0000","last_active":"2024-04-25T09:55:34.995309+0000","last_peered":"2024-04-25T09:55:34.995309+0000","last_clean":"2024-04-25T09:55:34.995309+0000","last_became_active":"2024-04-25T09:55:34.990222+0000","last_became_peered":"2024-04-25T09:55:34.990222+0000","last_unstale":"2024-04-25T09:55:34.995309+0000","last_undegraded":"2024-04-25T09:55:34.995309+0000","last_fullsized":"2024-04-25T09:55:34.995309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980732+0000","last_change":"2024-04-25T09:55:33.978187+0000","last_active":"2024-04-25T09:55:34.980732+0000","last_peered":"2024-04-25T09:55:34.980732+0000","last_clean":"2024-04-25T09:55:34.980732+0000","last_became_active":"2024-04-25T09:55:33.977894+0000","last_became_peered":"2024-04-25T09:55:33.977894+0000","last_unstale":"2024-04-25T09:55:34.980732+0000","last_undegraded":"2024-04-25T09:55:34.980732+0000","last_fullsized":"2024-04-25T09:55:34.980732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:42.818331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000095+0000","last_change":"2024-04-25T09:55:34.989516+0000","last_active":"2024-04-25T09:55:35.000095+0000","last_peered":"2024-04-25T09:55:35.000095+0000","last_clean":"2024-04-25T09:55:35.000095+0000","last_became_active":"2024-04-25T09:55:34.989220+0000","last_became_peered":"2024-04-25T09:55:34.989220+0000","last_unstale":"2024-04-25T09:55:35.000095+0000","last_undegraded":"2024-04-25T09:55:35.000095+0000","last_fullsized":"2024-04-25T09:55:35.000095+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:35.888134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.981766+0000","last_change":"2024-04-25T09:55:33.988716+0000","last_active":"2024-04-25T09:55:34.981766+0000","last_peered":"2024-04-25T09:55:34.981766+0000","last_clean":"2024-04-25T09:55:34.981766+0000","last_became_active":"2024-04-25T09:55:33.988423+0000","last_became_peered":"2024-04-25T09:55:33.988423+0000","last_unstale":"2024-04-25T09:55:34.981766+0000","last_undegraded":"2024-04-25T09:55:34.981766+0000","last_fullsized":"2024-04-25T09:55:34.981766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:39.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.008017+0000","last_change":"2024-04-25T09:55:35.004735+0000","last_active":"2024-04-25T09:55:35.008017+0000","last_peered":"2024-04-25T09:55:35.008017+0000","last_clean":"2024-04-25T09:55:35.008017+0000","last_became_active":"2024-04-25T09:55:35.004116+0000","last_became_peered":"2024-04-25T09:55:35.004116+0000","last_unstale":"2024-04-25T09:55:35.008017+0000","last_undegraded":"2024-04-25T09:55:35.008017+0000","last_fullsized":"2024-04-25T09:55:35.008017+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:22.340985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980976+0000","last_change":"2024-04-25T09:55:33.987165+0000","last_active":"2024-04-25T09:55:34.980976+0000","last_peered":"2024-04-25T09:55:34.980976+0000","last_clean":"2024-04-25T09:55:34.980976+0000","last_became_active":"2024-04-25T09:55:33.986884+0000","last_became_peered":"2024-04-25T09:55:33.986884+0000","last_unstale":"2024-04-25T09:55:34.980976+0000","last_undegraded":"2024-04-25T09:55:34.980976+0000","last_fullsized":"2024-04-25T09:55:34.980976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:24.121708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.006302+0000","last_change":"2024-04-25T09:55:35.005038+0000","last_active":"2024-04-25T09:55:35.006302+0000","last_peered":"2024-04-25T09:55:35.006302+0000","last_clean":"2024-04-25T09:55:35.006302+0000","last_became_active":"2024-04-25T09:55:35.004770+0000","last_became_peered":"2024-04-25T09:55:35.004770+0000","last_unstale":"2024-04-25T09:55:35.006302+0000","last_undegraded":"2024-04-25T09:55:35.006302+0000","last_fullsized":"2024-04-25T09:55:35.006302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:33.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.000411+0000","last_change":"2024-04-25T09:55:33.991409+0000","last_active":"2024-04-25T09:55:35.000411+0000","last_peered":"2024-04-25T09:55:35.000411+0000","last_clean":"2024-04-25T09:55:35.000411+0000","last_became_active":"2024-04-25T09:55:33.991054+0000","last_became_peered":"2024-04-25T09:55:33.991054+0000","last_unstale":"2024-04-25T09:55:35.000411+0000","last_undegraded":"2024-04-25T09:55:35.000411+0000","last_fullsized":"2024-04-25T09:55:35.000411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:22.565667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.997471+0000","last_change":"2024-04-25T09:55:34.990839+0000","last_active":"2024-04-25T09:55:34.997471+0000","last_peered":"2024-04-25T09:55:34.997471+0000","last_clean":"2024-04-25T09:55:34.997471+0000","last_became_active":"2024-04-25T09:55:34.990485+0000","last_became_peered":"2024-04-25T09:55:34.990485+0000","last_unstale":"2024-04-25T09:55:34.997471+0000","last_undegraded":"2024-04-25T09:55:34.997471+0000","last_fullsized":"2024-04-25T09:55:34.997471+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:14.578831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.981479+0000","last_change":"2024-04-25T09:55:33.979697+0000","last_active":"2024-04-25T09:55:34.981479+0000","last_peered":"2024-04-25T09:55:34.981479+0000","last_clean":"2024-04-25T09:55:34.981479+0000","last_became_active":"2024-04-25T09:55:33.979428+0000","last_became_peered":"2024-04-25T09:55:33.979428+0000","last_unstale":"2024-04-25T09:55:34.981479+0000","last_undegraded":"2024-04-25T09:55:34.981479+0000","last_fullsized":"2024-04-25T09:55:34.981479+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:20.317074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.002129+0000","last_change":"2024-04-25T09:55:34.995172+0000","last_active":"2024-04-25T09:55:35.002129+0000","last_peered":"2024-04-25T09:55:35.002129+0000","last_clean":"2024-04-25T09:55:35.002129+0000","last_became_active":"2024-04-25T09:55:34.994915+0000","last_became_peered":"2024-04-25T09:55:34.994915+0000","last_unstale":"2024-04-25T09:55:35.002129+0000","last_undegraded":"2024-04-25T09:55:35.002129+0000","last_fullsized":"2024-04-25T09:55:35.002129+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.902893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.982795+0000","last_change":"2024-04-25T09:55:33.991504+0000","last_active":"2024-04-25T09:55:34.982795+0000","last_peered":"2024-04-25T09:55:34.982795+0000","last_clean":"2024-04-25T09:55:34.982795+0000","last_became_active":"2024-04-25T09:55:33.991227+0000","last_became_peered":"2024-04-25T09:55:33.991227+0000","last_unstale":"2024-04-25T09:55:34.982795+0000","last_undegraded":"2024-04-25T09:55:34.982795+0000","last_fullsized":"2024-04-25T09:55:34.982795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:06.492702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.001289+0000","last_change":"2024-04-25T09:55:34.996974+0000","last_active":"2024-04-25T09:55:35.001289+0000","last_peered":"2024-04-25T09:55:35.001289+0000","last_clean":"2024-04-25T09:55:35.001289+0000","last_became_active":"2024-04-25T09:55:34.996707+0000","last_became_peered":"2024-04-25T09:55:34.996707+0000","last_unstale":"2024-04-25T09:55:35.001289+0000","last_undegraded":"2024-04-25T09:55:35.001289+0000","last_fullsized":"2024-04-25T09:55:35.001289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:01.725669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.984041+0000","last_change":"2024-04-25T09:55:33.986830+0000","last_active":"2024-04-25T09:55:34.984041+0000","last_peered":"2024-04-25T09:55:34.984041+0000","last_clean":"2024-04-25T09:55:34.984041+0000","last_became_active":"2024-04-25T09:55:33.986526+0000","last_became_peered":"2024-04-25T09:55:33.986526+0000","last_unstale":"2024-04-25T09:55:34.984041+0000","last_undegraded":"2024-04-25T09:55:34.984041+0000","last_fullsized":"2024-04-25T09:55:34.984041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:39.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.998603+0000","last_change":"2024-04-25T09:55:34.997229+0000","last_active":"2024-04-25T09:55:34.998603+0000","last_peered":"2024-04-25T09:55:34.998603+0000","last_clean":"2024-04-25T09:55:34.998603+0000","last_became_active":"2024-04-25T09:55:34.996966+0000","last_became_peered":"2024-04-25T09:55:34.996966+0000","last_unstale":"2024-04-25T09:55:34.998603+0000","last_undegraded":"2024-04-25T09:55:34.998603+0000","last_fullsized":"2024-04-25T09:55:34.998603+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:23.567676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.979154+0000","last_change":"2024-04-25T09:55:33.987722+0000","last_active":"2024-04-25T09:55:34.979154+0000","last_peered":"2024-04-25T09:55:34.979154+0000","last_clean":"2024-04-25T09:55:34.979154+0000","last_became_active":"2024-04-25T09:55:33.987222+0000","last_became_peered":"2024-04-25T09:55:33.987222+0000","last_unstale":"2024-04-25T09:55:34.979154+0000","last_undegraded":"2024-04-25T09:55:34.979154+0000","last_fullsized":"2024-04-25T09:55:34.979154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:58.688171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.995730+0000","last_change":"2024-04-25T09:55:34.994148+0000","last_active":"2024-04-25T09:55:34.995730+0000","last_peered":"2024-04-25T09:55:34.995730+0000","last_clean":"2024-04-25T09:55:34.995730+0000","last_became_active":"2024-04-25T09:55:34.993866+0000","last_became_peered":"2024-04-25T09:55:34.993866+0000","last_unstale":"2024-04-25T09:55:34.995730+0000","last_undegraded":"2024-04-25T09:55:34.995730+0000","last_fullsized":"2024-04-25T09:55:34.995730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.888732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.974377+0000","last_change":"2024-04-25T09:55:33.989309+0000","last_active":"2024-04-25T09:55:34.974377+0000","last_peered":"2024-04-25T09:55:34.974377+0000","last_clean":"2024-04-25T09:55:34.974377+0000","last_became_active":"2024-04-25T09:55:33.989007+0000","last_became_peered":"2024-04-25T09:55:33.989007+0000","last_unstale":"2024-04-25T09:55:34.974377+0000","last_undegraded":"2024-04-25T09:55:34.974377+0000","last_fullsized":"2024-04-25T09:55:34.974377+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:00.723660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.994639+0000","last_change":"2024-04-25T09:55:34.989680+0000","last_active":"2024-04-25T09:55:34.994639+0000","last_peered":"2024-04-25T09:55:34.994639+0000","last_clean":"2024-04-25T09:55:34.994639+0000","last_became_active":"2024-04-25T09:55:34.989391+0000","last_became_peered":"2024-04-25T09:55:34.989391+0000","last_unstale":"2024-04-25T09:55:34.994639+0000","last_undegraded":"2024-04-25T09:55:34.994639+0000","last_fullsized":"2024-04-25T09:55:34.994639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.676049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.981680+0000","last_change":"2024-04-25T09:55:33.990416+0000","last_active":"2024-04-25T09:55:34.981680+0000","last_peered":"2024-04-25T09:55:34.981680+0000","last_clean":"2024-04-25T09:55:34.981680+0000","last_became_active":"2024-04-25T09:55:33.989590+0000","last_became_peered":"2024-04-25T09:55:33.989590+0000","last_unstale":"2024-04-25T09:55:34.981680+0000","last_undegraded":"2024-04-25T09:55:34.981680+0000","last_fullsized":"2024-04-25T09:55:34.981680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:05.724588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:35.004168+0000","last_change":"2024-04-25T09:55:34.999748+0000","last_active":"2024-04-25T09:55:35.004168+0000","last_peered":"2024-04-25T09:55:35.004168+0000","last_clean":"2024-04-25T09:55:35.004168+0000","last_became_active":"2024-04-25T09:55:34.999432+0000","last_became_peered":"2024-04-25T09:55:34.999432+0000","last_unstale":"2024-04-25T09:55:35.004168+0000","last_undegraded":"2024-04-25T09:55:35.004168+0000","last_fullsized":"2024-04-25T09:55:35.004168+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:33.961582+0000","last_clean_scrub_stamp":"2024-04-25T09:55:33.961582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:01.273548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.980079+0000","last_change":"2024-04-25T09:55:33.992996+0000","last_active":"2024-04-25T09:55:34.980079+0000","last_peered":"2024-04-25T09:55:34.980079+0000","last_clean":"2024-04-25T09:55:34.980079+0000","last_became_active":"2024-04-25T09:55:33.992587+0000","last_became_peered":"2024-04-25T09:55:33.992587+0000","last_unstale":"2024-04-25T09:55:34.980079+0000","last_undegraded":"2024-04-25T09:55:34.980079+0000","last_fullsized":"2024-04-25T09:55:34.980079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:55:32.959573+0000","last_clean_scrub_stamp":"2024-04-25T09:55:32.959573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:06.492702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T09:55:34.982035+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:55:34.982035+0000","last_peered":"2024-04-25T09:55:34.982035+0000","last_clean":"2024-04-25T09:55:34.982035+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:55:34.982035+0000","last_undegraded":"2024-04-25T09:55:34.982035+0000","last_fullsized":"2024-04-25T09:55:34.982035+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":190535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":183346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":773714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:55:38.272 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:55:38.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:55:38.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.411+0000 7fe986529640 1 Processor -- start 2024-04-25T09:55:38.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.411+0000 7fe986529640 1 -- start start 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe986529640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe980070dc0 0x7fe98006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe986529640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9800711a0 0x7fe9800b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe986529640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe9800b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe986529640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9801622b0 con 0x7fe9800711a0 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe986529640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9800b10f0 con 0x7fe980070dc0 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe986529640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9800d1770 con 0x7fe9800b4830 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9800711a0 0x7fe9800b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9800711a0 0x7fe9800b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58594/0 (socket says 172.21.15.148:58594) 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 -- 172.21.15.148:0/209815626 learned_addr learned my addr 172.21.15.148:0/209815626 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:38.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97ffff640 1 --2- 172.21.15.148:0/209815626 >> v2:172.21.15.188:3300/0 conn(0x7fe980070dc0 0x7fe98006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 -- 172.21.15.148:0/209815626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 msgr2=0x7fe9800b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe984ac6640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe9800b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe9800b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 -- 172.21.15.148:0/209815626 >> v2:172.21.15.188:3300/0 conn(0x7fe980070dc0 msgr2=0x7fe98006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/209815626 >> v2:172.21.15.188:3300/0 conn(0x7fe980070dc0 0x7fe98006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97f7fe640 1 -- 172.21.15.148:0/209815626 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980162740 con 0x7fe9800711a0 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe97ffff640 1 --2- 172.21.15.148:0/209815626 >> v2:172.21.15.188:3300/0 conn(0x7fe980070dc0 0x7fe98006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.412+0000 7fe984ac6640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe9800b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.413+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9800711a0 0x7fe9800b07a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe980070a00 tx=0x7fe97402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c02d772ce800d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.413+0000 7fe97effd640 1 -- 172.21.15.148:0/209815626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974037070 con 0x7fe9800711a0 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.413+0000 7fe97effd640 1 -- 172.21.15.148:0/209815626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97402ae10 con 0x7fe9800711a0 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.413+0000 7fe97effd640 1 -- 172.21.15.148:0/209815626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974032660 con 0x7fe9800711a0 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 -- 172.21.15.148:0/209815626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9800711a0 msgr2=0x7fe9800b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9800711a0 0x7fe9800b07a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe980070a00 tx=0x7fe97402a100 comp rx=0 tx=0).stop 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 -- 172.21.15.148:0/209815626 shutdown_connections 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe9800b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 --2- 172.21.15.148:0/209815626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9800711a0 0x7fe9800b07a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 --2- 172.21.15.148:0/209815626 >> v2:172.21.15.188:3300/0 conn(0x7fe980070dc0 0x7fe98006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 -- 172.21.15.148:0/209815626 >> 172.21.15.148:0/209815626 conn(0x7fe9800b3530 msgr2=0x7fe9800b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 -- 172.21.15.148:0/209815626 shutdown_connections 2024-04-25T09:55:38.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.414+0000 7fe986529640 1 -- 172.21.15.148:0/209815626 wait complete. 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.415+0000 7fe986529640 1 Processor -- start 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.415+0000 7fe986529640 1 -- start start 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe986529640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 0x7fe9801522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe986529640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9800711a0 0x7fe9801555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe986529640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe980155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe986529640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe980161f40 con 0x7fe980070dc0 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe986529640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe980161c60 con 0x7fe9800711a0 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe986529640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe980071630 con 0x7fe9800b4830 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 0x7fe9801522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 0x7fe9801522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58602/0 (socket says 172.21.15.148:58602) 2024-04-25T09:55:38.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 -- 172.21.15.148:0/3999502336 learned_addr learned my addr 172.21.15.148:0/3999502336 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:38.422 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-04-25T09:55:38.416+0000 7f2aa6a38640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:55:38.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9800711a0 0x7fe9801555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe984ac6640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe980155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 -- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 msgr2=0x7fe980155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe980155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 -- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9800711a0 msgr2=0x7fe9801555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9800711a0 0x7fe9801555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 -- 172.21.15.148:0/3999502336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98015e590 con 0x7fe980070dc0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9800711a0 0x7fe9801555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97ffff640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 0x7fe9801522b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe97000c7c0 tx=0x7fe97000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.416+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970015070 con 0x7fe980070dc0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.417+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe970007a30 con 0x7fe980070dc0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.417+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe980153c70 con 0x7fe980070dc0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.417+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970007e00 con 0x7fe980070dc0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.417+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe980154230 con 0x7fe980070dc0 2024-04-25T09:55:38.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.418+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe980073a00 con 0x7fe980070dc0 2024-04-25T09:55:38.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.418+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe970018070 con 0x7fe980070dc0 2024-04-25T09:55:38.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.418+0000 7fe97cff9640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96c06b4d0 0x7fe96c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.418+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe97008a270 con 0x7fe980070dc0 2024-04-25T09:55:38.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.418+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96c06b4d0 0x7fe96c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.419+0000 7fe97f7fe640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96c06b4d0 0x7fe96c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe974034000 tx=0x7fe97402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.423+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe970058ff0 con 0x7fe980070dc0 2024-04-25T09:55:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.659+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9800b4e40 con 0x7fe980070dc0 2024-04-25T09:55:38.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.661+0000 7fe97cff9640 1 -- 172.21.15.148:0/3999502336 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe9700588f0 con 0x7fe980070dc0 2024-04-25T09:55:38.664 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T09:55:38.664 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T09:55:38.664 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T09:55:38.665 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T09:55:38.665 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:55:38.665 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96c06b4d0 msgr2=0x7fe96c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96c06b4d0 0x7fe96c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe974034000 tx=0x7fe97402e000 comp rx=0 tx=0).stop 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 msgr2=0x7fe9801522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 0x7fe9801522b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe97000c7c0 tx=0x7fe97000cc80 comp rx=0 tx=0).stop 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 shutdown_connections 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96c06b4d0 0x7fe96c06d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9800b4830 0x7fe980155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9800711a0 0x7fe9801555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 --2- 172.21.15.148:0/3999502336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe980070dc0 0x7fe9801522b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 >> 172.21.15.148:0/3999502336 conn(0x7fe9800b3530 msgr2=0x7fe98006ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 shutdown_connections 2024-04-25T09:55:38.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.664+0000 7fe986529640 1 -- 172.21.15.148:0/3999502336 wait complete. 2024-04-25T09:55:38.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:55:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.772+0000 7f30f92c4640 1 Processor -- start 2024-04-25T09:55:38.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.772+0000 7f30f92c4640 1 -- start start 2024-04-25T09:55:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f92c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30f415dbe0 0x7f30f40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f92c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 0x7f30f40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f92c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f40b97a0 con 0x7f30f40b92a0 2024-04-25T09:55:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f92c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f415dfc0 con 0x7f30f415dbe0 2024-04-25T09:55:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f92c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40d1a00 con 0x7f30f40b2cb0 2024-04-25T09:55:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f30f40b92a0 0x7f30f40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37252/0 (socket says 172.21.15.148:37252) 2024-04-25T09:55:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f37fe640 1 -- 172.21.15.148:0/3552818661 learned_addr learned my addr 172.21.15.148:0/3552818661 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:38.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049364094 0 0) 0x7f30f40b97a0 con 0x7f30f40b92a0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30dc003520 con 0x7f30f40b92a0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f2ffd640 1 --2- 172.21.15.148:0/3552818661 >> v2:172.21.15.188:3300/0 conn(0x7f30f415dbe0 0x7f30f40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.773+0000 7f30f27fc640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 0x7f30f40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2466755661 0 0) 0x7f30dc003520 con 0x7f30f40b92a0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 msgr2=0x7f30f40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 0x7f30f40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 >> v2:172.21.15.188:3300/0 conn(0x7f30f415dbe0 msgr2=0x7f30f40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 --2- 172.21.15.148:0/3552818661 >> v2:172.21.15.188:3300/0 conn(0x7f30f415dbe0 0x7f30f40d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40d1c50 con 0x7f30f40b92a0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f30ec0019f0 con 0x7f30f40b92a0 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 >> v1:172.21.15.148:6789/0 conn(0x7f30f40b92a0 legacy=0x7f30f40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 0x7f30dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc003f30 0x7f30dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40d1a00 con 0x7f30dc004d40 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc00b9c0 con 0x7f30dc003f30 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc00bb90 con 0x7f30f40b2cb0 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f37fe640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f2ffd640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 0x7f30dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f27fc640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc003f30 0x7f30dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f37fe640 1 -- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 msgr2=0x7f30dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f37fe640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f40b2cb0 0x7f30dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f37fe640 1 -- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc003f30 msgr2=0x7f30dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f37fe640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc003f30 0x7f30dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.774+0000 7f30f37fe640 1 -- 172.21.15.148:0/3552818661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc003520 con 0x7f30dc004d40 2024-04-25T09:55:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f37fe640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f30f40b96d0 tx=0x7f30dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4496dcbe069255e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00a4c0 con 0x7f30dc004d40 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec02f070 con 0x7f30dc004d40 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f1ffb640 1 -- 172.21.15.148:0/3552818661 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec02bc70 con 0x7f30dc004d40 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f92c4640 1 -- 172.21.15.148:0/3552818661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30dc004d40 msgr2=0x7f30dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3552818661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f30f40b96d0 tx=0x7f30dc005830 comp rx=0 tx=0).stop 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f37fe640 1 -- 172.21.15.148:0/3552818661 reap_dead start 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f92c4640 1 -- 172.21.15.148:0/3552818661 shutdown_connections 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f92c4640 1 -- 172.21.15.148:0/3552818661 >> 172.21.15.148:0/3552818661 conn(0x7f30f4061a20 msgr2=0x7f30f4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f92c4640 1 -- 172.21.15.148:0/3552818661 shutdown_connections 2024-04-25T09:55:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.775+0000 7f30f92c4640 1 -- 172.21.15.148:0/3552818661 wait complete. 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.777+0000 7f30f92c4640 1 Processor -- start 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.777+0000 7f30f92c4640 1 -- start start 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.777+0000 7f30f92c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc004d40 0x7f30f4160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 0x7f30f40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f41577e0 0x7f30f40cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40b6ab0 con 0x7f30f41f3710 2024-04-25T09:55:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40d2210 con 0x7f30dc004d40 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f40b6c30 con 0x7f30f41577e0 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 0x7f30f40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 0x7f30f40b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58620/0 (socket says 172.21.15.148:58620) 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 -- 172.21.15.148:0/3771398504 learned_addr learned my addr 172.21.15.148:0/3771398504 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 -- 172.21.15.148:0/3771398504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f41577e0 msgr2=0x7f30f40cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f41577e0 0x7f30f40cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 -- 172.21.15.148:0/3771398504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc004d40 msgr2=0x7f30f4160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc004d40 0x7f30f4160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 -- 172.21.15.148:0/3771398504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40b5870 con 0x7f30f41f3710 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f27fc640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 0x7f30f40b5100 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b820 tx=0x7f30e800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8018070 con 0x7f30f41f3710 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e800be10 con 0x7f30f41f3710 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f40b5b50 con 0x7f30f41f3710 2024-04-25T09:55:38.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8010be0 con 0x7f30f41f3710 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.778+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f40b3650 con 0x7f30f41f3710 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.779+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f4073990 con 0x7f30f41f3710 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.780+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30e801b070 con 0x7f30f41f3710 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.780+0000 7f30db7fe640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30e006b4d0 0x7f30e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.780+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30e808a6f0 con 0x7f30f41f3710 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.780+0000 7f30f2ffd640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30e006b4d0 0x7f30e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.780+0000 7f30f2ffd640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30e006b4d0 0x7f30e006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30e4006e80 tx=0x7f30e4006df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:38.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.784+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30e8059470 con 0x7f30f41f3710 2024-04-25T09:55:38.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.992+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30f40b8b90 con 0x7f30f41f3710 2024-04-25T09:55:38.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.993+0000 7f30db7fe640 1 -- 172.21.15.148:0/3771398504 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f30e8058d70 con 0x7f30f41f3710 2024-04-25T09:55:38.996 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:55:38.996 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:55:34.319815+0000","modified":"2024-04-25T09:55:36.324069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/3412433349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3412433349},{"type":"v1","addr":"172.21.15.148:6836","nonce":3412433349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:55:38.996 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 16 2024-04-25T09:55:38.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30e006b4d0 msgr2=0x7f30e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30e006b4d0 0x7f30e006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30e4006e80 tx=0x7f30e4006df0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 msgr2=0x7f30f40b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:38.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 0x7f30f40b5100 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b820 tx=0x7f30e800bce0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 shutdown_connections 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30e006b4d0 0x7f30e006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30f41577e0 0x7f30f40cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30f41f3710 0x7f30f40b5100 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 --2- 172.21.15.148:0/3771398504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30dc004d40 0x7f30f4160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 >> 172.21.15.148:0/3771398504 conn(0x7f30f4061a20 msgr2=0x7f30f40d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 shutdown_connections 2024-04-25T09:55:38.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:38.995+0000 7f30f92c4640 1 -- 172.21.15.148:0/3771398504 wait complete. 2024-04-25T09:55:39.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:55:34.319815+0000', 'modified': '2024-04-25T09:55:36.324069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5361}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5361': {'gid': 5361, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6836/3412433349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3412433349}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3412433349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:55:39.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:55:39.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:55:39.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.510+0000 7fc8451e0640 1 Processor -- start 2024-04-25T09:55:39.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.510+0000 7fc8451e0640 1 -- start start 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.510+0000 7fc8451e0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc84015b7d0 0x7fc8400cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.510+0000 7fc8451e0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84006cf50 con 0x7fc84006c9e0 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.510+0000 7fc8451e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400ce530 con 0x7fc8400b5620 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.510+0000 7fc8451e0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8400ce810 con 0x7fc84015b7d0 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc84015b7d0 0x7fc8400cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83f59e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc84006c9e0 0x7fc8400d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37264/0 (socket says 172.21.15.148:37264) 2024-04-25T09:55:39.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83f59e640 1 -- 172.21.15.148:0/1371614199 learned_addr learned my addr 172.21.15.148:0/1371614199 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1371614199 >> v1:172.21.15.188:6789/0 conn(0x7fc8400b5620 legacy=0x7fc84006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83dd9b640 1 -- 172.21.15.148:0/1371614199 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420829424 0 0) 0x7fc84006cf50 con 0x7fc84006c9e0 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1371614199 >> v1:172.21.15.148:6789/0 conn(0x7fc84006c9e0 legacy=0x7fc8400d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.511+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1371614199 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400ceaf0 con 0x7fc84015b7d0 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1371614199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84015b7d0 0x7fc8400cf010 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc830009d90 tx=0x7fc83002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9972cdf81ad2942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc83dd9b640 1 -- 172.21.15.148:0/1371614199 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830007740 con 0x7fc84015b7d0 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc83dd9b640 1 -- 172.21.15.148:0/1371614199 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8300078e0 con 0x7fc84015b7d0 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc83dd9b640 1 -- 172.21.15.148:0/1371614199 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830007cb0 con 0x7fc84015b7d0 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 -- 172.21.15.148:0/1371614199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84015b7d0 msgr2=0x7fc8400cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 --2- 172.21.15.148:0/1371614199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84015b7d0 0x7fc8400cf010 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc830009d90 tx=0x7fc83002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 -- 172.21.15.148:0/1371614199 shutdown_connections 2024-04-25T09:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 --2- 172.21.15.148:0/1371614199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84015b7d0 0x7fc8400cf010 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 -- 172.21.15.148:0/1371614199 >> 172.21.15.148:0/1371614199 conn(0x7fc8400d0800 msgr2=0x7fc8400d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 -- 172.21.15.148:0/1371614199 shutdown_connections 2024-04-25T09:55:39.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.512+0000 7fc8451e0640 1 -- 172.21.15.148:0/1371614199 wait complete. 2024-04-25T09:55:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 Processor -- start 2024-04-25T09:55:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 -- start start 2024-04-25T09:55:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 0x7fc8400c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8400b5620 0x7fc840155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc84015b7d0 0x7fc8400b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400cf6d0 con 0x7fc84006c9e0 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400cf550 con 0x7fc8400b5620 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.514+0000 7fc8451e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84015bbb0 con 0x7fc84015b7d0 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 0x7fc8400c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 0x7fc8400c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58640/0 (socket says 172.21.15.148:58640) 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 -- 172.21.15.148:0/491871443 learned_addr learned my addr 172.21.15.148:0/491871443 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83e59c640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8400b5620 0x7fc840155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83f59e640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc84015b7d0 0x7fc8400b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 -- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc84015b7d0 msgr2=0x7fc8400b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc84015b7d0 0x7fc8400b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 -- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8400b5620 msgr2=0x7fc840155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8400b5620 0x7fc840155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 -- 172.21.15.148:0/491871443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840158d70 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 0x7fc8400c9f20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc830009eb0 tx=0x7fc830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc8277fe640 1 -- 172.21.15.148:0/491871443 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc830040070 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc8277fe640 1 -- 172.21.15.148:0/491871443 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830031c40 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8400b7b40 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc8277fe640 1 -- 172.21.15.148:0/491871443 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83003c420 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.515+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8400b7de0 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.517+0000 7fc8277fe640 1 -- 172.21.15.148:0/491871443 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc830039040 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.517+0000 7fc8277fe640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc82c06b4d0 0x7fc82c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.517+0000 7fc8277fe640 1 -- 172.21.15.148:0/491871443 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8300ab9e0 con 0x7fc84006c9e0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.517+0000 7fc83e59c640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc82c06b4d0 0x7fc82c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.517+0000 7fc83e59c640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc82c06b4d0 0x7fc82c06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8401f6ad0 tx=0x7fc834009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.518+0000 7fc847fc4740 1 Processor -- start 2024-04-25T09:55:39.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.518+0000 7fc847fc4740 1 -- start start 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 0x563ab8e7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e4ff10 0x563ab8e96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc847fc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8db2a10 con 0x563ab8e7a230 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8d5bb30 con 0x563ab8e97ad0 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab8daec50 con 0x563ab8e4ff10 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 0x563ab8e7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 0x563ab8e7a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58644/0 (socket says 172.21.15.148:58644) 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 -- 172.21.15.148:0/802638464 learned_addr learned my addr 172.21.15.148:0/802638464 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 -- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e4ff10 msgr2=0x563ab8e96190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e4ff10 0x563ab8e96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83e59c640 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 -- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 msgr2=0x563ab8e98020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e98020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 -- 172.21.15.148:0/802638464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab8d70e10 con 0x563ab8e7a230 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83e59c640 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.519+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 0x563ab8e7a610 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc8300764c0 tx=0x7fc83007a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d08ead9d4d53d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc824ff9640 1 -- 172.21.15.148:0/802638464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300a8070 con 0x563ab8e7a230 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc824ff9640 1 -- 172.21.15.148:0/802638464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830007840 con 0x563ab8e7a230 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc824ff9640 1 -- 172.21.15.148:0/802638464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300766d0 con 0x563ab8e7a230 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 -- 172.21.15.148:0/802638464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 msgr2=0x563ab8e7a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 0x563ab8e7a610 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc8300764c0 tx=0x7fc83007a060 comp rx=0 tx=0).stop 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 -- 172.21.15.148:0/802638464 shutdown_connections 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e4ff10 0x563ab8e96190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 --2- 172.21.15.148:0/802638464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e7a230 0x563ab8e7a610 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 -- 172.21.15.148:0/802638464 >> 172.21.15.148:0/802638464 conn(0x563ab8e4c840 msgr2=0x563ab8e95040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 -- 172.21.15.148:0/802638464 shutdown_connections 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.520+0000 7fc847fc4740 1 -- 172.21.15.148:0/802638464 wait complete. 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.521+0000 7fc847fc4740 1 -- start start 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.521+0000 7fc847fc4740 1 Processor -- start 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8e67b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 0x563ab8e7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e787e0 0x563ab8e78c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8db2a10 con 0x563ab8e97ad0 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8d6f0c0 con 0x563ab8e7a230 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab8e50cb0 con 0x563ab8e787e0 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc847fc4740 10 client.0 _ll_register_callbacks cb 0x563ab8d62ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 0x563ab8e7aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 0x563ab8e7aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58660/0 (socket says 172.21.15.148:58660) 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 -- 172.21.15.148:0/1852777483 learned_addr learned my addr 172.21.15.148:0/1852777483 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8e67b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83f59e640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e787e0 0x563ab8e78c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 -- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e787e0 msgr2=0x563ab8e78c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e787e0 0x563ab8e78c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 -- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 msgr2=0x563ab8e67b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8e67b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 -- 172.21.15.148:0/1852777483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab8ac72d0 con 0x563ab8e97ad0 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83f59e640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e787e0 0x563ab8e78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 0x563ab8e7aef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc83400edc0 tx=0x7fc8340036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c6da29d71fedc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.522+0000 7fc81dffb640 1 -- 172.21.15.148:0/1852777483 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834007020 con 0x563ab8e97ad0 2024-04-25T09:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc81dffb640 1 -- 172.21.15.148:0/1852777483 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc834010040 con 0x563ab8e97ad0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc830007840 con 0x563ab8e97ad0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc81dffb640 1 -- 172.21.15.148:0/1852777483 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8340071c0 con 0x563ab8e97ad0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 10 client.5553 fetch_fsmap learned FSMap version 16 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x563ab8b916b0 con 0x563ab8e97ad0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc81dffb640 1 -- 172.21.15.148:0/1852777483 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830007840 con 0x563ab8e97ad0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc81dffb640 1 -- 172.21.15.148:0/1852777483 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc834014070 con 0x563ab8e97ad0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 10 client.5553 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 10 client.5553 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 20 client.5553 populate_metadata read hostname 'smithi148' 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e7d3f0 0x563ab8e7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 4 client.5553 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ab8e11920 con 0x563ab8e7d3f0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e7d3f0 0x563ab8e7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc81dffb640 10 client.5553 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.523+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e7d3f0 0x563ab8e7d810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59f3ab27b131696 server_cookie=d4996ca5f6f5c48c in_seq=0 out_seq=0 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc81dffb640 1 -- 172.21.15.148:0/1852777483 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ab8e11920 con 0x563ab8e7d3f0 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc81dffb640 10 client.5553 handle_command_reply: tid=0 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 client.5553 shutdown 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 msgr2=0x563ab8e7aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 0x563ab8e7aef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc83400edc0 tx=0x7fc8340036f0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 shutdown_connections 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e7d3f0 0x563ab8e7d810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e787e0 0x563ab8e78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e97ad0 0x563ab8e7aef0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc81dffb640 0 client.5553 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1852777483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8e67b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 >> 172.21.15.148:0/1852777483 conn(0x563ab8e4ff10 msgr2=0x563ab8e948c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 shutdown_connections 2024-04-25T09:55:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 1 -- 172.21.15.148:0/1852777483 wait complete. 2024-04-25T09:55:39.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.524+0000 7fc847fc4740 20 client.5553 trim_cache size 0 max 16384 2024-04-25T09:55:39.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 Processor -- start 2024-04-25T09:55:39.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 -- start start 2024-04-25T09:55:39.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e7a230 0x563ab8e49690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e8c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401ff640 con 0x563ab8e787e0 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401febc0 con 0x563ab8e97ad0 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8401ff040 con 0x563ab8e7a230 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e8bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58662/0 (socket says 172.21.15.148:58662) 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.540+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1521879675 learned_addr learned my addr 172.21.15.148:0/1521879675 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1521879675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e7a230 msgr2=0x563ab8e49690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e7a230 0x563ab8e49690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1521879675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 msgr2=0x563ab8e8c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e8c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1521879675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab8e4bd00 con 0x563ab8e787e0 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e8bbb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc830079fa0 tx=0x7fc83007e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba70d64c2249e40c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc81ffff640 1 -- 172.21.15.148:0/1521879675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300a8040 con 0x563ab8e787e0 2024-04-25T09:55:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc81ffff640 1 -- 172.21.15.148:0/1521879675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8300a9040 con 0x563ab8e787e0 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc81ffff640 1 -- 172.21.15.148:0/1521879675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83003a3b0 con 0x563ab8e787e0 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 -- 172.21.15.148:0/1521879675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 msgr2=0x563ab8e8bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e8bbb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc830079fa0 tx=0x7fc83007e000 comp rx=0 tx=0).stop 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 -- 172.21.15.148:0/1521879675 shutdown_connections 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e97ad0 0x563ab8e8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e7a230 0x563ab8e49690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1521879675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e8bbb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 -- 172.21.15.148:0/1521879675 >> 172.21.15.148:0/1521879675 conn(0x563ab8e4ff10 msgr2=0x563ab8e50310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 -- 172.21.15.148:0/1521879675 shutdown_connections 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.541+0000 7fc847fc4740 1 -- 172.21.15.148:0/1521879675 wait complete. 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.542+0000 7fc847fc4740 1 -- start start 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.542+0000 7fc847fc4740 1 Processor -- start 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e66b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8f025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e97ad0 0x563ab8e930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8e7e110 con 0x563ab8e787e0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8e7bef0 con 0x563ab8e7a230 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab8ea8f70 con 0x563ab8e97ad0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 10 client.0 _ll_register_callbacks cb 0x563ab8e4ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e66b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e66b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58668/0 (socket says 172.21.15.148:58668) 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1271252226 learned_addr learned my addr 172.21.15.148:0/1271252226 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83f59e640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e97ad0 0x563ab8e930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8f025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e97ad0 msgr2=0x563ab8e930f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e97ad0 0x563ab8e930f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 msgr2=0x563ab8f025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8f025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 -- 172.21.15.148:0/1271252226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab8e79540 con 0x563ab8e787e0 2024-04-25T09:55:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8f025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc83ed9d640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e66b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc8300a9e60 tx=0x7fc830007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5da6b595641ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc81dffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc81dffb640 1 -- 172.21.15.148:0/1271252226 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83009c070 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc81dffb640 1 -- 172.21.15.148:0/1271252226 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8300766d0 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.543+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8300a9040 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 1 -- 172.21.15.148:0/1271252226 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc830097050 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 10 client.5559 fetch_fsmap learned FSMap version 16 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x563ab8e79540 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 1 -- 172.21.15.148:0/1271252226 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8300a9040 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 1 -- 172.21.15.148:0/1271252226 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc83003a3b0 con 0x563ab8e787e0 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 10 client.5559 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 10 client.5559 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 20 client.5559 populate_metadata read hostname 'smithi148' 2024-04-25T09:55:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e49690 0x563ab8e49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 4 client.5559 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ab8dbff50 con 0x563ab8e49690 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e49690 0x563ab8e49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 10 client.5559 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc83e59c640 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e49690 0x563ab8e49a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02a2860dedcc577 server_cookie=39f124e2a595915a in_seq=0 out_seq=0 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 1 -- 172.21.15.148:0/1271252226 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563ab8dbff50 con 0x563ab8e49690 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 10 client.5559 handle_command_reply: tid=0 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 client.5559 shutdown 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 msgr2=0x563ab8e66b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e66b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc8300a9e60 tx=0x7fc830007fc0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 shutdown_connections 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x563ab8e49690 0x563ab8e49a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563ab8e97ad0 0x563ab8e930f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc81dffb640 0 client.5559 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ab8e7a230 0x563ab8f025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 --2- 172.21.15.148:0/1271252226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563ab8e787e0 0x563ab8e66b80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.544+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 >> 172.21.15.148:0/1271252226 conn(0x563ab8e4ff10 msgr2=0x563ab8eb7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 shutdown_connections 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc847fc4740 1 -- 172.21.15.148:0/1271252226 wait complete. 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc847fc4740 20 client.5559 trim_cache size 0 max 16384 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5361, 2024-04-25T09:55:39.555 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.5736677099999996, 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 16, 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 39, 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 39, 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 11.983445141000001 2024-04-25T09:55:39.556 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc82c06b4d0 msgr2=0x7fc82c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc82c06b4d0 0x7fc82c06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8401f6ad0 tx=0x7fc834009040 comp rx=0 tx=0).stop 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 msgr2=0x7fc8400c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 0x7fc8400c9f20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc830009eb0 tx=0x7fc830004a40 comp rx=0 tx=0).stop 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 shutdown_connections 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc82c06b4d0 0x7fc82c06d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc84015b7d0 0x7fc8400b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8400b5620 0x7fc840155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 --2- 172.21.15.148:0/491871443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84006c9e0 0x7fc8400c9f20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 >> 172.21.15.148:0/491871443 conn(0x7fc8400d0800 msgr2=0x7fc84015fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 shutdown_connections 2024-04-25T09:55:39.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.545+0000 7fc8451e0640 1 -- 172.21.15.148:0/491871443 wait complete. 2024-04-25T09:55:39.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:55:39.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.652+0000 7fe42d09c640 1 Processor -- start 2024-04-25T09:55:39.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.652+0000 7fe42d09c640 1 -- start start 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42d09c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe428158340 0x7fe4281621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42d09c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe42806b2f0 0x7fe42815e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42d09c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42815ecf0 con 0x7fe428157f60 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42d09c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42815efd0 con 0x7fe42806b2f0 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42d09c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe428161700 con 0x7fe428158340 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe42806b2f0 0x7fe42815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe426d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe428157f60 0x7fe42806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37270/0 (socket says 172.21.15.148:37270) 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe426d9d640 1 -- 172.21.15.148:0/894497873 learned_addr learned my addr 172.21.15.148:0/894497873 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:39.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42659c640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe428158340 0x7fe4281621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42759e640 1 -- 172.21.15.148:0/894497873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe428158340 msgr2=0x7fe4281621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42759e640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe428158340 0x7fe4281621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.653+0000 7fe42759e640 1 -- 172.21.15.148:0/894497873 >> v1:172.21.15.148:6789/0 conn(0x7fe428157f60 legacy=0x7fe42806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe42759e640 1 -- 172.21.15.148:0/894497873 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4281619e0 con 0x7fe42806b2f0 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe42659c640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe428158340 0x7fe4281621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe425d9b640 1 -- 172.21.15.148:0/894497873 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553015911 0 0) 0x7fe42815ecf0 con 0x7fe428157f60 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe42759e640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42806b2f0 0x7fe42815e090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe4200097a0 tx=0x7fe42002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a527e09b261148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe425d9b640 1 -- 172.21.15.148:0/894497873 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420037070 con 0x7fe42806b2f0 2024-04-25T09:55:39.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe425d9b640 1 -- 172.21.15.148:0/894497873 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42002ae50 con 0x7fe42806b2f0 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.654+0000 7fe425d9b640 1 -- 172.21.15.148:0/894497873 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420032660 con 0x7fe42806b2f0 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 -- 172.21.15.148:0/894497873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42806b2f0 msgr2=0x7fe42815e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42806b2f0 0x7fe42815e090 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe4200097a0 tx=0x7fe42002a100 comp rx=0 tx=0).stop 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 -- 172.21.15.148:0/894497873 shutdown_connections 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe42806b2f0 0x7fe42815e090 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 --2- 172.21.15.148:0/894497873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe428158340 0x7fe4281621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 -- 172.21.15.148:0/894497873 >> 172.21.15.148:0/894497873 conn(0x7fe4280b23e0 msgr2=0x7fe4280b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 -- 172.21.15.148:0/894497873 shutdown_connections 2024-04-25T09:55:39.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.655+0000 7fe42d09c640 1 -- 172.21.15.148:0/894497873 wait complete. 2024-04-25T09:55:39.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.656+0000 7fe42d09c640 1 Processor -- start 2024-04-25T09:55:39.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.656+0000 7fe42d09c640 1 -- start start 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42d09c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42806b2f0 0x7fe42814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42d09c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe428157f60 0x7fe4280b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42d09c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 0x7fe4280c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42d09c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42815e750 con 0x7fe4280b39d0 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42d09c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42815e5d0 con 0x7fe428157f60 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42d09c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42815e8d0 con 0x7fe42806b2f0 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 0x7fe4280c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 0x7fe4280c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58690/0 (socket says 172.21.15.148:58690) 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42759e640 1 -- 172.21.15.148:0/1665011918 learned_addr learned my addr 172.21.15.148:0/1665011918 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe42659c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe428157f60 0x7fe4280b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.657+0000 7fe426d9d640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42806b2f0 0x7fe42814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42759e640 1 -- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42806b2f0 msgr2=0x7fe42814d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42759e640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42806b2f0 0x7fe42814d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42759e640 1 -- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe428157f60 msgr2=0x7fe4280b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42759e640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe428157f60 0x7fe4280b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42759e640 1 -- 172.21.15.148:0/1665011918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4280ca6e0 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42759e640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 0x7fe4280c9a60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe420035040 tx=0x7fe4200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420037040 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe420004030 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4280ca9c0 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420004420 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.658+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4280caf80 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.659+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe428073990 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.659+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe420003610 con 0x7fe4280b39d0 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.660+0000 7fe40f7fe640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe41406b4d0 0x7fe41406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:39.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.660+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4200ab010 con 0x7fe4280b39d0 2024-04-25T09:55:39.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.660+0000 7fe426d9d640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe41406b4d0 0x7fe41406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:39.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.660+0000 7fe426d9d640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe41406b4d0 0x7fe41406d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4180012e0 tx=0x7fe418001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:39.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.664+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4200797d0 con 0x7fe4280b39d0 2024-04-25T09:55:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.871+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe42815ba20 con 0x7fe4280b39d0 2024-04-25T09:55:39.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.872+0000 7fe40f7fe640 1 -- 172.21.15.148:0/1665011918 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe42007d870 con 0x7fe4280b39d0 2024-04-25T09:55:39.875 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:55:39.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe41406b4d0 msgr2=0x7fe41406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe41406b4d0 0x7fe41406d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4180012e0 tx=0x7fe418001700 comp rx=0 tx=0).stop 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 msgr2=0x7fe4280c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 0x7fe4280c9a60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe420035040 tx=0x7fe4200046c0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 shutdown_connections 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe41406b4d0 0x7fe41406d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4280b39d0 0x7fe4280c9a60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe428157f60 0x7fe4280b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 --2- 172.21.15.148:0/1665011918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe42806b2f0 0x7fe42814d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 >> 172.21.15.148:0/1665011918 conn(0x7fe4280b23e0 msgr2=0x7fe4280d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 shutdown_connections 2024-04-25T09:55:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:39.874+0000 7fe42d09c640 1 -- 172.21.15.148:0/1665011918 wait complete. 2024-04-25T09:55:39.887 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:55:39.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:55:39.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:55:39.887 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:55:39.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:39.888 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:55:39.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:55:39.888 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:55:39.888 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:39.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:39.888 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:55:39.903 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:39.904 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42696sec preferred_lft 42696sec 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T09:55:39.905 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:55:39.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:39.906 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:55:39.965 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:55:39.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:39.971 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:55:40.027 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:55:40.027 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:55:40.027 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:55:40.027 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:55:40.027 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:55:40.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:40.027 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:55:40.027 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:55:40.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:55:40.027 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:55:40.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:55:40.028 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:55:40.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:40.111 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:55:40.125 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:55:40.125 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:55:40.126 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:40.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:40.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:55:40.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:40.190 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:40.244 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:55:40.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:55:40.244 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:40.299 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:55:40.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:40.299 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:55:40.380 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:55:40.380 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:55:40.381 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:55:40.389 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:55:40.394 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:55:40.395 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:55:40.397 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:55:40.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:40.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:55:40.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:40.461 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:55:40.521 INFO:teuthology.orchestra.run.smithi148.stdout:client5571 2024-04-25T09:55:40.521 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:40.521 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:40.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:55:40.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.714+0000 7f0d2625a640 1 Processor -- start 2024-04-25T09:55:40.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.714+0000 7f0d2625a640 1 -- start start 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.715+0000 7f0d2625a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 0x7f0d201689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.715+0000 7f0d2625a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d200d1770 con 0x7f0d2006ccd0 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.715+0000 7f0d2625a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d20162740 con 0x7f0d20070dc0 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.715+0000 7f0d2625a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200b0eb0 con 0x7f0d2015eec0 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.715+0000 7f0d1effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0d2006ccd0 0x7f0d200b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37282/0 (socket says 172.21.15.148:37282) 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.715+0000 7f0d1effd640 1 -- 172.21.15.148:0/2330832599 learned_addr learned my addr 172.21.15.148:0/2330832599 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1ffff640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 0x7f0d201689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635305375 0 0) 0x7f0d200d1770 con 0x7f0d2006ccd0 2024-04-25T09:55:40.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d08003520 con 0x7f0d2006ccd0 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 450045083 0 0) 0x7f0d08003520 con 0x7f0d2006ccd0 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 msgr2=0x7f0d201689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 0x7f0d201689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 >> v1:172.21.15.188:6789/0 conn(0x7f0d20070dc0 legacy=0x7f0d200711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20161980 con 0x7f0d2006ccd0 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f0d14001670 con 0x7f0d2006ccd0 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 >> v1:172.21.15.148:6789/0 conn(0x7f0d2006ccd0 legacy=0x7f0d200b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080039e0 0x7f0d08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 0x7f0d08005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200b0eb0 con 0x7f0d080039e0 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800baf0 con 0x7f0d08004800 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0800bcf0 con 0x7f0d2015eec0 2024-04-25T09:55:40.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080039e0 0x7f0d08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1effd640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1ffff640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 0x7f0d08005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1f7fe640 1 -- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 msgr2=0x7f0d08005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2015eec0 0x7f0d08005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1f7fe640 1 -- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d08004800 msgr2=0x7f0d08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1f7fe640 1 -- 172.21.15.148:0/2330832599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08003520 con 0x7f0d080039e0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.716+0000 7f0d1effd640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d08004800 0x7f0d08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.717+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080039e0 0x7f0d08003dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d10001860 tx=0x7f0d08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1252b169a06db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.717+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10037070 con 0x7f0d080039e0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.717+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1002ac30 con 0x7f0d080039e0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.717+0000 7f0d1e7fc640 1 -- 172.21.15.148:0/2330832599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10001430 con 0x7f0d080039e0 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d2625a640 1 -- 172.21.15.148:0/2330832599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080039e0 msgr2=0x7f0d08003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d2625a640 1 --2- 172.21.15.148:0/2330832599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d080039e0 0x7f0d08003dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d10001860 tx=0x7f0d08005830 comp rx=0 tx=0).stop 2024-04-25T09:55:40.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d1ffff640 1 -- 172.21.15.148:0/2330832599 reap_dead start 2024-04-25T09:55:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d2625a640 1 -- 172.21.15.148:0/2330832599 shutdown_connections 2024-04-25T09:55:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d2625a640 1 -- 172.21.15.148:0/2330832599 >> 172.21.15.148:0/2330832599 conn(0x7f0d200b3530 msgr2=0x7f0d200b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d2625a640 1 -- 172.21.15.148:0/2330832599 shutdown_connections 2024-04-25T09:55:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.718+0000 7f0d2625a640 1 -- 172.21.15.148:0/2330832599 wait complete. 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.719+0000 7f0d2625a640 1 Processor -- start 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.719+0000 7f0d2625a640 1 -- start start 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d2625a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d080039e0 0x7f0d201f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d2625a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147980 0x7f0d201f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d2625a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 0x7f0d200d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d2625a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200b10f0 con 0x7f0d201f31d0 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d2625a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201622b0 con 0x7f0d080039e0 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d2625a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20161f40 con 0x7f0d20147980 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 0x7f0d200d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 0x7f0d200d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58708/0 (socket says 172.21.15.148:58708) 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 -- 172.21.15.148:0/3614785608 learned_addr learned my addr 172.21.15.148:0/3614785608 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:55:40.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d080039e0 0x7f0d201f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1effd640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147980 0x7f0d201f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 -- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147980 msgr2=0x7f0d201f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147980 0x7f0d201f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 -- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d080039e0 msgr2=0x7f0d201f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d080039e0 0x7f0d201f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 -- 172.21.15.148:0/3614785608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200cf340 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d080039e0 0x7f0d201f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.720+0000 7f0d1ffff640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 0x7f0d200d0100 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0d2015f2d0 tx=0x7f0d180115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.721+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18003990 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.721+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1801b070 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.721+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d200cf620 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.721+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18003b70 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.721+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d200cfbe0 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.721+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d200cfde0 con 0x7f0d201f31d0 2024-04-25T09:55:40.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.722+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d18005d10 con 0x7f0d201f31d0 2024-04-25T09:55:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.722+0000 7f0d07fff640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:55:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.722+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d1808a920 con 0x7f0d201f31d0 2024-04-25T09:55:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.722+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:55:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.723+0000 7f0d1f7fe640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d10009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:55:40.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.727+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d200cfde0 con 0x7f0d201f31d0 2024-04-25T09:55:40.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:40.933+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d200b6610 con 0x7f0d201f31d0 2024-04-25T09:55:41.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.005+0000 7f0d07fff640 1 -- 172.21.15.148:0/3614785608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7f0d18025020 con 0x7f0d201f31d0 2024-04-25T09:55:41.008 INFO:teuthology.orchestra.run.smithi148.stderr:enabled new snapshots 2024-04-25T09:55:41.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d0c06b4d0 msgr2=0x7f0d0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d10009030 comp rx=0 tx=0).stop 2024-04-25T09:55:41.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 msgr2=0x7f0d200d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 0x7f0d200d0100 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0d2015f2d0 tx=0x7f0d180115a0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 shutdown_connections 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d0c06b4d0 0x7f0d0c06d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d201f31d0 0x7f0d200d0100 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147980 0x7f0d201f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 --2- 172.21.15.148:0/3614785608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d080039e0 0x7f0d201f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 >> 172.21.15.148:0/3614785608 conn(0x7f0d200b3530 msgr2=0x7f0d200740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.007+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 shutdown_connections 2024-04-25T09:55:41.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:41.008+0000 7f0d2625a640 1 -- 172.21.15.148:0/3614785608 wait complete. 2024-04-25T09:55:41.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:41.021 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:55:41.041 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:55:41.041 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:55:41.097 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:55:41.097 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:55:41.097 DEBUG:teuthology.orchestra.run.smithi148:> import uuid 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> n = 2000 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> create_hard_links = False 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> if 3 > 0: 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> create_hard_links = True 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> dpath = os.path.dirname(path) 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> fnameprefix = os.path.basename(path) 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> for i in range(n): 2024-04-25T09:55:41.098 DEBUG:teuthology.orchestra.run.smithi148:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> f.write(f"{i}") 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> f.flush() 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f.fileno()) 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink(fpath) 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(dirfd) 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> if create_hard_links: 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> for j in range(3): 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(dirfd) 2024-04-25T09:55:41.099 DEBUG:teuthology.orchestra.run.smithi148:> finally: 2024-04-25T09:55:41.100 DEBUG:teuthology.orchestra.run.smithi148:> os.close(dirfd) 2024-04-25T09:55:41.100 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:55:53.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:53.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:55:57.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:55:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:55:57.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:57.131 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:55:57.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:55:57.363 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:56:23.549 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:23.552 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:23.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:23.651 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:56:23.694 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:23.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:56:23.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.831+0000 7ff06017e640 1 Processor -- start 2024-04-25T09:56:23.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.831+0000 7ff06017e640 1 -- start start 2024-04-25T09:56:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff06017e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff058070dc0 0x7ff05806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff06017e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0580711a0 0x7ff0580b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff06017e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0580d1770 con 0x7ff0580b4830 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff06017e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0581622b0 con 0x7ff0580711a0 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff06017e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0580b10f0 con 0x7ff058070dc0 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05d719640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0580711a0 0x7ff0580b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05e71b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0580b4830 0x7ff0580b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52328/0 (socket says 172.21.15.148:52328) 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05d719640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0580711a0 0x7ff0580b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37364/0 (socket says 172.21.15.148:37364) 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05e71b640 1 -- 172.21.15.148:0/665968153 learned_addr learned my addr 172.21.15.148:0/665968153 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05df1a640 1 --2- 172.21.15.148:0/665968153 >> v2:172.21.15.188:3300/0 conn(0x7ff058070dc0 0x7ff05806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05d719640 1 -- 172.21.15.148:0/665968153 >> v2:172.21.15.188:3300/0 conn(0x7ff058070dc0 msgr2=0x7ff05806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.832+0000 7ff05d719640 1 --2- 172.21.15.148:0/665968153 >> v2:172.21.15.188:3300/0 conn(0x7ff058070dc0 0x7ff05806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.833+0000 7ff05d719640 1 -- 172.21.15.148:0/665968153 >> v1:172.21.15.148:6789/0 conn(0x7ff0580b4830 legacy=0x7ff0580b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.833+0000 7ff05d719640 1 -- 172.21.15.148:0/665968153 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058162740 con 0x7ff0580711a0 2024-04-25T09:56:23.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.833+0000 7ff05d719640 1 --2- 172.21.15.148:0/665968153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580711a0 0x7ff0580b07a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff058070a00 tx=0x7ff04802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a8e5083c06fac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.833+0000 7ff05cf18640 1 -- 172.21.15.148:0/665968153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048037070 con 0x7ff0580711a0 2024-04-25T09:56:23.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.833+0000 7ff05cf18640 1 -- 172.21.15.148:0/665968153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04802add0 con 0x7ff0580711a0 2024-04-25T09:56:23.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.833+0000 7ff05cf18640 1 -- 172.21.15.148:0/665968153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048032660 con 0x7ff0580711a0 2024-04-25T09:56:23.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 -- 172.21.15.148:0/665968153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580711a0 msgr2=0x7ff0580b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 --2- 172.21.15.148:0/665968153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580711a0 0x7ff0580b07a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff058070a00 tx=0x7ff04802a080 comp rx=0 tx=0).stop 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 -- 172.21.15.148:0/665968153 shutdown_connections 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 --2- 172.21.15.148:0/665968153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0580711a0 0x7ff0580b07a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 --2- 172.21.15.148:0/665968153 >> v2:172.21.15.188:3300/0 conn(0x7ff058070dc0 0x7ff05806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 -- 172.21.15.148:0/665968153 >> 172.21.15.148:0/665968153 conn(0x7ff0580b3530 msgr2=0x7ff0580b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 -- 172.21.15.148:0/665968153 shutdown_connections 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.834+0000 7ff06017e640 1 -- 172.21.15.148:0/665968153 wait complete. 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.835+0000 7ff06017e640 1 Processor -- start 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.835+0000 7ff06017e640 1 -- start start 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff06017e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 0x7ff0581481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff06017e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0580711a0 0x7ff058160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff06017e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0580b4830 0x7ff058160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff06017e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff058161f40 con 0x7ff058070dc0 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff06017e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff058161c60 con 0x7ff0580711a0 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff06017e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff058071630 con 0x7ff0580b4830 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 0x7ff0581481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 0x7ff0581481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37366/0 (socket says 172.21.15.148:37366) 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 -- 172.21.15.148:0/2760728208 learned_addr learned my addr 172.21.15.148:0/2760728208 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05d719640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0580711a0 0x7ff058160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05e71b640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0580b4830 0x7ff058160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 -- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0580b4830 msgr2=0x7ff058160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0580b4830 0x7ff058160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 -- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0580711a0 msgr2=0x7ff058160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0580711a0 0x7ff058160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 -- 172.21.15.148:0/2760728208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058156f20 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 0x7ff0581481a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff05400efc0 tx=0x7ff05400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.836+0000 7ff04e7fc640 1 -- 172.21.15.148:0/2760728208 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054018020 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.837+0000 7ff04e7fc640 1 -- 172.21.15.148:0/2760728208 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054014480 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.837+0000 7ff04e7fc640 1 -- 172.21.15.148:0/2760728208 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054014850 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.837+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff058157200 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.837+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0581486e0 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.838+0000 7ff04e7fc640 1 -- 172.21.15.148:0/2760728208 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff054005d10 con 0x7ff058070dc0 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.838+0000 7ff04e7fc640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff03c06b4d0 0x7ff03c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.839+0000 7ff04e7fc640 1 -- 172.21.15.148:0/2760728208 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff05408af20 con 0x7ff058070dc0 2024-04-25T09:56:23.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.839+0000 7ff05d719640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff03c06b4d0 0x7ff03c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.839+0000 7ff05d719640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff03c06b4d0 0x7ff03c06d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff048034000 tx=0x7ff04802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 Processor -- start 2024-04-25T09:56:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 -- start start 2024-04-25T09:56:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b6226ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b6226d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b62297be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b621cfb90 con 0x562b6226cca0 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62178e40 con 0x562b622a9370 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b621cbc30 con 0x562b62297760 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.840+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b6226ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b6226ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37374/0 (socket says 172.21.15.148:37374) 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 -- 172.21.15.148:0/2356012091 learned_addr learned my addr 172.21.15.148:0/2356012091 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05d719640 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b6226d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05e71b640 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b62297be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 -- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 msgr2=0x562b62297be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b62297be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 -- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 msgr2=0x562b6226d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b6226d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 -- 172.21.15.148:0/2356012091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b61fae7d0 con 0x562b6226cca0 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b6226ee00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff054089240 tx=0x7ff0540595b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e894d4e5646b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff03b7fe640 1 -- 172.21.15.148:0/2356012091 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054087070 con 0x562b6226cca0 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff03b7fe640 1 -- 172.21.15.148:0/2356012091 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054071070 con 0x562b6226cca0 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.841+0000 7ff03b7fe640 1 -- 172.21.15.148:0/2356012091 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054055b60 con 0x562b6226cca0 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 -- 172.21.15.148:0/2356012091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 msgr2=0x562b6226ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b6226ee00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff054089240 tx=0x7ff0540595b0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 -- 172.21.15.148:0/2356012091 shutdown_connections 2024-04-25T09:56:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b62297be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b6226d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 --2- 172.21.15.148:0/2356012091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b6226ee00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 -- 172.21.15.148:0/2356012091 >> 172.21.15.148:0/2356012091 conn(0x562b62234870 msgr2=0x562b622b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 -- 172.21.15.148:0/2356012091 shutdown_connections 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 -- 172.21.15.148:0/2356012091 wait complete. 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 -- start start 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.842+0000 7ff062f66740 1 Processor -- start 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b62299b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b622ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6218c3e0 con 0x562b6226cca0 2024-04-25T09:56:23.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b622952b0 con 0x562b622a9370 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b622a9a50 con 0x562b62297760 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff062f66740 10 client.0 _ll_register_callbacks cb 0x562b6218bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b62299b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b62299b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37384/0 (socket says 172.21.15.148:37384) 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff05df1a640 1 -- 172.21.15.148:0/3873610400 learned_addr learned my addr 172.21.15.148:0/3873610400 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff05d719640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.843+0000 7ff05e71b640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b622ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05df1a640 1 -- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 msgr2=0x562b622ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05df1a640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05df1a640 1 -- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 msgr2=0x562b622ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05df1a640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b622ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05df1a640 1 -- 172.21.15.148:0/3873610400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b61ea40f0 con 0x562b6226cca0 2024-04-25T09:56:23.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05df1a640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b62299b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff0540616d0 tx=0x7ff054000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4fe2a086106408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff038ff9640 1 -- 172.21.15.148:0/3873610400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054087070 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff038ff9640 1 -- 172.21.15.148:0/3873610400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff05405d360 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff054071070 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff038ff9640 1 -- 172.21.15.148:0/3873610400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0540780b0 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 10 client.5592 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562b6200ef60 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff038ff9640 1 -- 172.21.15.148:0/3873610400 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054071070 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff038ff9640 1 -- 172.21.15.148:0/3873610400 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff054011b40 con 0x562b6226cca0 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 10 client.5592 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 10 client.5592 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 20 client.5592 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:23.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b62282e20 0x562b62285940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 4 client.5592 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b621dd0d0 con 0x562b62282e20 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.844+0000 7ff05d719640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b62282e20 0x562b62285940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.845+0000 7ff038ff9640 10 client.5592 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.845+0000 7ff05d719640 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b62282e20 0x562b62285940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be579bddda2bf5f server_cookie=39e7527a1c15948 in_seq=0 out_seq=0 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.845+0000 7ff038ff9640 1 -- 172.21.15.148:0/3873610400 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b621dd0d0 con 0x562b62282e20 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.845+0000 7ff038ff9640 10 client.5592 handle_command_reply: tid=0 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.845+0000 7ff062f66740 1 client.5592 shutdown 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 msgr2=0x562b62299b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b62299b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff0540616d0 tx=0x7ff054000f40 comp rx=0 tx=0).stop 2024-04-25T09:56:23.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 shutdown_connections 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b62282e20 0x562b62285940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b622ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff038ff9640 0 client.5592 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 --2- 172.21.15.148:0/3873610400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b62299b50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 >> 172.21.15.148:0/3873610400 conn(0x562b62234870 msgr2=0x562b622948a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 shutdown_connections 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 1 -- 172.21.15.148:0/3873610400 wait complete. 2024-04-25T09:56:23.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.846+0000 7ff062f66740 20 client.5592 trim_cache size 0 max 16384 2024-04-25T09:56:23.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.860+0000 7ff062f66740 1 Processor -- start 2024-04-25T09:56:23.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.860+0000 7ff062f66740 1 -- start start 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff062f66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b623138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b62266ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff062f66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0582004e0 con 0x562b6226cca0 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0581ff6d0 con 0x562b622a9370 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05815f130 con 0x562b62297760 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b623138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b623138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37398/0 (socket says 172.21.15.148:37398) 2024-04-25T09:56:23.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 -- 172.21.15.148:0/2181739681 learned_addr learned my addr 172.21.15.148:0/2181739681 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05d719640 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05e71b640 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b62266ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 -- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 msgr2=0x562b622c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 -- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 msgr2=0x562b62266ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b62266ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 -- 172.21.15.148:0/2181739681 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b622addf0 con 0x562b6226cca0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05e71b640 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b62266ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b623138b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff05408b240 tx=0x7ff054084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42c3bd9976c4ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff03affd640 1 -- 172.21.15.148:0/2181739681 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054087070 con 0x562b6226cca0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff03affd640 1 -- 172.21.15.148:0/2181739681 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054009040 con 0x562b6226cca0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.861+0000 7ff03affd640 1 -- 172.21.15.148:0/2181739681 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054071040 con 0x562b6226cca0 2024-04-25T09:56:23.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 -- 172.21.15.148:0/2181739681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 msgr2=0x562b623138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b623138b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff05408b240 tx=0x7ff054084040 comp rx=0 tx=0).stop 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 -- 172.21.15.148:0/2181739681 shutdown_connections 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b622a9370 0x562b62266ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b62297760 0x562b622c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 --2- 172.21.15.148:0/2181739681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b6226cca0 0x562b623138b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 -- 172.21.15.148:0/2181739681 >> 172.21.15.148:0/2181739681 conn(0x562b62234870 msgr2=0x562b6229aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 -- 172.21.15.148:0/2181739681 shutdown_connections 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.862+0000 7ff062f66740 1 -- 172.21.15.148:0/2181739681 wait complete. 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.863+0000 7ff062f66740 1 -- start start 2024-04-25T09:56:23.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.863+0000 7ff062f66740 1 Processor -- start 2024-04-25T09:56:23.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.863+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b6226cca0 0x562b622ff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 0x562b622ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b622a9370 0x562b62284a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05815f730 con 0x562b62297760 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6229b930 con 0x562b6226cca0 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b623006e0 con 0x562b622a9370 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 10 client.0 _ll_register_callbacks cb 0x7ff054085070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 0x562b622ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 0x562b622ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37410/0 (socket says 172.21.15.148:37410) 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 -- 172.21.15.148:0/2729891880 learned_addr learned my addr 172.21.15.148:0/2729891880 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b6226cca0 0x562b622ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05e71b640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b622a9370 0x562b62284a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 -- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b622a9370 msgr2=0x562b62284a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b622a9370 0x562b62284a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 -- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b6226cca0 msgr2=0x562b622ff330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b6226cca0 0x562b622ff330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 -- 172.21.15.148:0/2729891880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b61e7d1b0 con 0x562b62297760 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff05d719640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 0x562b622ffbf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff04802a0c0 tx=0x7ff048004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2012e11c6ea70a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff038ff9640 1 -- 172.21.15.148:0/2729891880 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04803ee60 con 0x562b62297760 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff038ff9640 1 -- 172.21.15.148:0/2729891880 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff04804f350 con 0x562b62297760 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff054009040 con 0x562b62297760 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff038ff9640 1 -- 172.21.15.148:0/2729891880 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff04805d050 con 0x562b62297760 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 10 client.5598 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:23.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.864+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562b62300480 con 0x562b62297760 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff038ff9640 1 -- 172.21.15.148:0/2729891880 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054009040 con 0x562b62297760 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff038ff9640 1 -- 172.21.15.148:0/2729891880 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff048060020 con 0x562b62297760 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff062f66740 10 client.5598 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff062f66740 10 client.5598 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff062f66740 20 client.5598 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff062f66740 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b622acac0 0x562b622b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff062f66740 4 client.5598 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x562b621d7420 con 0x562b622acac0 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b622acac0 0x562b622b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff038ff9640 10 client.5598 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.865+0000 7ff05df1a640 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b622acac0 0x562b622b5b00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2acdbce1a74b803 server_cookie=eaccac5f1b623371 in_seq=0 out_seq=0 2024-04-25T09:56:23.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff038ff9640 1 -- 172.21.15.148:0/2729891880 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17921 (crc 0 0 0) 0x562b621d7420 con 0x562b622acac0 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff038ff9640 10 client.5598 handle_command_reply: tid=0 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 client.5598 shutdown 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 msgr2=0x562b622ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 0x562b622ffbf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff04802a0c0 tx=0x7ff048004730 comp rx=0 tx=0).stop 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 shutdown_connections 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x562b622acac0 0x562b622b5b00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b622a9370 0x562b62284a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562b62297760 0x562b622ffbf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff038ff9640 0 client.5598 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 --2- 172.21.15.148:0/2729891880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b6226cca0 0x562b622ff330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 >> 172.21.15.148:0/2729891880 conn(0x562b62234870 msgr2=0x562b62266b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 shutdown_connections 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 1 -- 172.21.15.148:0/2729891880 wait complete. 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff062f66740 20 client.5598 trim_cache size 0 max 16384 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2010, 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2011, 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 503208, 2024-04-25T09:56:23.877 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 36432070, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 9, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.270242306, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.156137783, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.040264749, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.062649485, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2009, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.416839539, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000207486 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.878 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000242, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000121 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1009, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1005, 2024-04-25T09:56:23.879 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 353493, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 20872167, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.172500187, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.109214219, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.027032830, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.035221638, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1005, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.172204432, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000171347 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.880 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 115888, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 17584 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 28576, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 51050, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 9196574, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 53289672, 2024-04-25T09:56:23.881 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 2.860352110, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 2.218940363, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.928703506, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.048660101, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 51049, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.228286016, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000024060 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:56:23.882 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 48831, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.007530732, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000154 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 105424 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-25T09:56:23.883 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1294, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.802515234, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.012984942 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1986, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.154603141, 2024-04-25T09:56:23.884 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000581371 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 25307, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 25307, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 25307, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 410.652899458, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.016226850 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-25T09:56:23.885 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 12, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 19874, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 19874, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 19874, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-25T09:56:23.886 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 77, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 77, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 45893, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 45467, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 107, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-25T09:56:23.887 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-25T09:56:23.888 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 1040, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 4038, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 3775, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 3255, 2024-04-25T09:56:23.889 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 58, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 13909 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 2000, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 168, 2024-04-25T09:56:23.890 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 4122, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 1758, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 2217, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-25T09:56:23.891 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 24471, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 24471, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-25T09:56:23.892 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 32, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 32, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 3, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 95943853, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2549, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 116.171553677, 2024-04-25T09:56:23.893 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.045575344 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 2014, 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 2014, 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 0, 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 13, 2024-04-25T09:56:23.894 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 13, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 0, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 19874, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 19875, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 1, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 77, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 7953, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 7876, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 113156, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215396 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 28180, 2024-04-25T09:56:23.895 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 25307, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 3, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2000, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.294412425, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000647206 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 29, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003797164, 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000130936 2024-04-25T09:56:23.896 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6000, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.780315172, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001130052 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7048, 2024-04-25T09:56:23.897 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.110144862, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000866933 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.898 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-25T09:56:23.899 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000038325, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000038325 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:23.900 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002908344, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002908344 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.211898011, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.211898011 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.901 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.488031897, 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.186003987 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-25T09:56:23.902 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2217, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 378.152432552, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.170569432 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.903 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027763602, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.027763602 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-25T09:56:23.904 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000249526, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000249526 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.905 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.906 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-25T09:56:23.907 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8000, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.580907578, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002072613 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.908 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 18579, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 18579, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 42, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.909 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:56:23.910 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:56:23.911 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:56:23.912 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 16426565, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 6153, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 5, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 5520, 2024-04-25T09:56:23.913 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 78, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 21377128, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 885039, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-25T09:56:23.914 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 4539, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 92219297, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 37, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 4539, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4539, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 22.189453540, 2024-04-25T09:56:23.915 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004888621 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4539, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4628 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 4539, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 50, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 4489, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-25T09:56:23.916 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 50, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 2599, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 32, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 1758, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 33, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-25T09:56:23.917 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-25T09:56:23.918 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-25T09:56:23.919 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 41, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 3, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 8, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 8, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 95, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 28 2024-04-25T09:56:23.920 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-25T09:56:23.921 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 32, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 1758, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 1758, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:23.922 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 31590, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 7636943, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 31590, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 31590, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 7636943, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.923 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T09:56:23.924 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4539, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 92410919, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4539, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 92410919, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-25T09:56:23.925 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4539, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 4539, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4539, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4539, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.926 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1758, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 191622, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1758, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 50, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 191622, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55eb5b2c26e0": { 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 84, 2024-04-25T09:56:23.928 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 24471, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 91749549, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 24471, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2549, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 91749465, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:23.929 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:23.930 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:23.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.930 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:23.930 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:56:23.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff03c06b4d0 msgr2=0x7ff03c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.866+0000 7ff06017e640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff03c06b4d0 0x7ff03c06d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff048034000 tx=0x7ff04802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 msgr2=0x7ff0581481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 0x7ff0581481a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff05400efc0 tx=0x7ff05400c490 comp rx=0 tx=0).stop 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 shutdown_connections 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff03c06b4d0 0x7ff03c06d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0580b4830 0x7ff058160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0580711a0 0x7ff058160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 --2- 172.21.15.148:0/2760728208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff058070dc0 0x7ff0581481a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 >> 172.21.15.148:0/2760728208 conn(0x7ff0580b3530 msgr2=0x7ff05806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 shutdown_connections 2024-04-25T09:56:23.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:23.867+0000 7ff06017e640 1 -- 172.21.15.148:0/2760728208 wait complete. 2024-04-25T09:56:23.933 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:56:23.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:23.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T09:56:24.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.030+0000 7f0c434b3640 1 Processor -- start 2024-04-25T09:56:24.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.030+0000 7f0c434b3640 1 -- start start 2024-04-25T09:56:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c434b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c3c099cb0 0x7f0c3c0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c434b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c0cfd30 0x7f0c3c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c434b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3c0cc5e0 con 0x7f0c3c09a090 2024-04-25T09:56:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c434b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c0cc3b0 con 0x7f0c3c099cb0 2024-04-25T09:56:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c434b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0d0210 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c4124f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c3c099cb0 0x7f0c3c0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c4124f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0c3c099cb0 0x7f0c3c0b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37426/0 (socket says 172.21.15.148:37426) 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.031+0000 7f0c4124f640 1 -- 172.21.15.148:0/710020516 learned_addr learned my addr 172.21.15.148:0/710020516 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c0cfd30 0x7f0c3c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c4124f640 1 -- 172.21.15.148:0/710020516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c0cfd30 msgr2=0x7f0c3c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c4124f640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c0cfd30 0x7f0c3c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c4124f640 1 -- 172.21.15.148:0/710020516 >> v1:172.21.15.188:6789/0 conn(0x7f0c3c09a090 legacy=0x7f0c3c15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c4124f640 1 -- 172.21.15.148:0/710020516 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0d04f0 con 0x7f0c3c099cb0 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c0cfd30 0x7f0c3c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c4124f640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c099cb0 0x7f0c3c0b27d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009870 tx=0x7f0c3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3fa445117787cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.032+0000 7f0c33fff640 1 -- 172.21.15.148:0/710020516 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38037070 con 0x7f0c3c099cb0 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c33fff640 1 -- 172.21.15.148:0/710020516 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c38032440 con 0x7f0c3c099cb0 2024-04-25T09:56:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c33fff640 1 -- 172.21.15.148:0/710020516 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38032810 con 0x7f0c3c099cb0 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 -- 172.21.15.148:0/710020516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c099cb0 msgr2=0x7f0c3c0b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c099cb0 0x7f0c3c0b27d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009870 tx=0x7f0c3802a180 comp rx=0 tx=0).stop 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 -- 172.21.15.148:0/710020516 shutdown_connections 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c0cfd30 0x7f0c3c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 --2- 172.21.15.148:0/710020516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c099cb0 0x7f0c3c0b27d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 -- 172.21.15.148:0/710020516 >> 172.21.15.148:0/710020516 conn(0x7f0c3c0b3290 msgr2=0x7f0c3c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 -- 172.21.15.148:0/710020516 shutdown_connections 2024-04-25T09:56:24.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.033+0000 7f0c434b3640 1 -- 172.21.15.148:0/710020516 wait complete. 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.034+0000 7f0c434b3640 1 Processor -- start 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.034+0000 7f0c434b3640 1 -- start start 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c434b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c099cb0 0x7f0c3c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c434b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c3c09a090 0x7f0c3c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c434b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 0x7f0c3c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c434b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0c52f0 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c434b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0d07d0 con 0x7f0c3c09a090 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c434b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0c5470 con 0x7f0c3c099cb0 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c41a50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 0x7f0c3c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c41a50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 0x7f0c3c15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37430/0 (socket says 172.21.15.148:37430) 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c41a50640 1 -- 172.21.15.148:0/528395022 learned_addr learned my addr 172.21.15.148:0/528395022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c4124f640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c099cb0 0x7f0c3c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.035+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c3c09a090 0x7f0c3c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c41a50640 1 -- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c099cb0 msgr2=0x7f0c3c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c41a50640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c099cb0 0x7f0c3c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c41a50640 1 -- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c3c09a090 msgr2=0x7f0c3c15b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c41a50640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c3c09a090 0x7f0c3c15b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c41a50640 1 -- 172.21.15.148:0/528395022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c1f3ad0 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c41a50640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 0x7f0c3c15b550 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c7b0 tx=0x7f0c3400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c31ffb640 1 -- 172.21.15.148:0/528395022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004b60 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c31ffb640 1 -- 172.21.15.148:0/528395022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34004d00 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c31ffb640 1 -- 172.21.15.148:0/528395022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3400aa90 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c1f3db0 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.036+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c0b43a0 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.037+0000 7f0c31ffb640 1 -- 172.21.15.148:0/528395022 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c3400ac30 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.038+0000 7f0c31ffb640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0c2006b4d0 0x7f0c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.038+0000 7f0c31ffb640 1 -- 172.21.15.148:0/528395022 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0c3408a680 con 0x7f0c3c0cfd30 2024-04-25T09:56:24.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.038+0000 7f0c4124f640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0c2006b4d0 0x7f0c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.038+0000 7f0c4124f640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0c2006b4d0 0x7f0c2006d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c38034000 tx=0x7f0c3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.039+0000 7f0c461b8740 1 Processor -- start 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.039+0000 7f0c461b8740 1 -- start start 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 0x5619e7d38f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36af0 0x5619e7d39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d09370 0x5619e7d0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c461b8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e7c71bc0 con 0x5619e7d36710 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e7c1ae70 con 0x5619e7d09370 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619e7c6dc60 con 0x5619e7d36af0 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 0x5619e7d38f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 0x5619e7d38f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37432/0 (socket says 172.21.15.148:37432) 2024-04-25T09:56:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 -- 172.21.15.148:0/3587451714 learned_addr learned my addr 172.21.15.148:0/3587451714 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 -- 172.21.15.148:0/3587451714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36af0 msgr2=0x5619e7d39460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36af0 0x5619e7d39460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 -- 172.21.15.148:0/3587451714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d09370 msgr2=0x5619e7d0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d09370 0x5619e7d0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 -- 172.21.15.148:0/3587451714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e7c3bad0 con 0x5619e7d36710 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c4124f640 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 0x5619e7d38f20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c38035040 tx=0x7f0c3802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d39aa2a14ea332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c1effd640 1 -- 172.21.15.148:0/3587451714 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38037040 con 0x5619e7d36710 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c1effd640 1 -- 172.21.15.148:0/3587451714 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3803e660 con 0x5619e7d36710 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.040+0000 7f0c1effd640 1 -- 172.21.15.148:0/3587451714 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3803ea10 con 0x5619e7d36710 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 -- 172.21.15.148:0/3587451714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 msgr2=0x5619e7d38f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 0x5619e7d38f20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c38035040 tx=0x7f0c3802c000 comp rx=0 tx=0).stop 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 -- 172.21.15.148:0/3587451714 shutdown_connections 2024-04-25T09:56:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d09370 0x5619e7d0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36af0 0x5619e7d39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3587451714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36710 0x5619e7d38f20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 -- 172.21.15.148:0/3587451714 >> 172.21.15.148:0/3587451714 conn(0x5619e7d0b5e0 msgr2=0x5619e7d53f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 -- 172.21.15.148:0/3587451714 shutdown_connections 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 -- 172.21.15.148:0/3587451714 wait complete. 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 -- start start 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.041+0000 7f0c461b8740 1 Processor -- start 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d2ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7d3bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e7d0e790 con 0x5619e7d36af0 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e7c104e0 con 0x5619e7d58c40 2024-04-25T09:56:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619e7d0ea70 con 0x5619e7d36710 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c461b8740 10 client.0 _ll_register_callbacks cb 0x5619e7c2dde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c40a4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7d3bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.042+0000 7f0c40a4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7d3bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37442/0 (socket says 172.21.15.148:37442) 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 -- 172.21.15.148:0/4186043127 learned_addr learned my addr 172.21.15.148:0/4186043127 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c41a50640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c4124f640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d2ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 -- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 msgr2=0x5619e7d2ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d2ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 -- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 msgr2=0x5619e7d573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 -- 172.21.15.148:0/4186043127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e7a52940 con 0x5619e7d36af0 2024-04-25T09:56:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7d3bd70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00de00 tx=0x7f0c2c00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c8888797d77fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c17fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c17fff640 1 -- 172.21.15.148:0/4186043127 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c018070 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c17fff640 1 -- 172.21.15.148:0/4186043127 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0c2c004220 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c3803e660 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c17fff640 1 -- 172.21.15.148:0/4186043127 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c2c022050 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c461b8740 10 client.5610 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5619e7ab30d0 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c17fff640 1 -- 172.21.15.148:0/4186043127 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3803e660 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c17fff640 1 -- 172.21.15.148:0/4186043127 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c2c025020 con 0x5619e7d36af0 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c461b8740 10 client.5610 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.043+0000 7f0c461b8740 10 client.5610 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c461b8740 20 client.5610 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c461b8740 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d56150 0x5619e7d56560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c461b8740 4 client.5610 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619e7c7f100 con 0x5619e7d56150 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c4124f640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d56150 0x5619e7d56560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c4124f640 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d56150 0x5619e7d56560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f1af4f9a93a8cc server_cookie=e8939625c082580a in_seq=0 out_seq=0 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c17fff640 10 client.5610 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c17fff640 1 -- 172.21.15.148:0/4186043127 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619e7c7f100 con 0x5619e7d56150 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c17fff640 10 client.5610 handle_command_reply: tid=0 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.044+0000 7f0c461b8740 1 client.5610 shutdown 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 msgr2=0x5619e7d3bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7d3bd70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00de00 tx=0x7f0c2c00f5b0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 shutdown_connections 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d56150 0x5619e7d56560 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c17fff640 0 client.5610 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7d3bd70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 --2- 172.21.15.148:0/4186043127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d2ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 >> 172.21.15.148:0/4186043127 conn(0x5619e7d09370 msgr2=0x5619e7d0a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 shutdown_connections 2024-04-25T09:56:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 1 -- 172.21.15.148:0/4186043127 wait complete. 2024-04-25T09:56:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.045+0000 7f0c461b8740 20 client.5610 trim_cache size 0 max 16384 2024-04-25T09:56:24.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 Processor -- start 2024-04-25T09:56:24.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 -- start start 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d3ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d36af0 0x5619e7d09a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 0x5619e7d2aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c200420 con 0x5619e7d58c40 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1ff650 con 0x5619e7d36af0 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.060+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c06c600 con 0x5619e7d36710 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 0x5619e7d2aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 0x5619e7d2aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37450/0 (socket says 172.21.15.148:37450) 2024-04-25T09:56:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 -- 172.21.15.148:0/3504426016 learned_addr learned my addr 172.21.15.148:0/3504426016 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c4124f640 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d3ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d36af0 0x5619e7d09a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 -- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 msgr2=0x5619e7d3ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d3ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 -- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d36af0 msgr2=0x5619e7d09a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d36af0 0x5619e7d09a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 -- 172.21.15.148:0/3504426016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e7946230 con 0x5619e7d58c40 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d36af0 0x5619e7d09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c41a50640 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 0x5619e7d2aaf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c3405f200 tx=0x7f0c3405f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237cbdab171dfc21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c1e7fc640 1 -- 172.21.15.148:0/3504426016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34058c50 con 0x5619e7d58c40 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c1e7fc640 1 -- 172.21.15.148:0/3504426016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3405cda0 con 0x5619e7d58c40 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.061+0000 7f0c1e7fc640 1 -- 172.21.15.148:0/3504426016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34061360 con 0x5619e7d58c40 2024-04-25T09:56:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 -- 172.21.15.148:0/3504426016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 msgr2=0x5619e7d2aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 0x5619e7d2aaf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c3405f200 tx=0x7f0c3405f040 comp rx=0 tx=0).stop 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 -- 172.21.15.148:0/3504426016 shutdown_connections 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d58c40 0x5619e7d2aaf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d36af0 0x5619e7d09a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 --2- 172.21.15.148:0/3504426016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d3ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 -- 172.21.15.148:0/3504426016 >> 172.21.15.148:0/3504426016 conn(0x5619e7d09370 msgr2=0x5619e7d39120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 -- 172.21.15.148:0/3504426016 shutdown_connections 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 -- 172.21.15.148:0/3504426016 wait complete. 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 -- start start 2024-04-25T09:56:24.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.062+0000 7f0c461b8740 1 Processor -- start 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d0d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7dc6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e7d09840 con 0x5619e7d36af0 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c06cc00 con 0x5619e7d58c40 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5619e7d3c4e0 con 0x5619e7d36710 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c461b8740 10 client.0 _ll_register_callbacks cb 0x7f0c2c00d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7dc6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7dc6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37458/0 (socket says 172.21.15.148:37458) 2024-04-25T09:56:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 -- 172.21.15.148:0/847621975 learned_addr learned my addr 172.21.15.148:0/847621975 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c4124f640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d0d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c41a50640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 -- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 msgr2=0x5619e7d0d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d0d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 -- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 msgr2=0x5619e7d54170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d54170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.063+0000 7f0c40a4e640 1 -- 172.21.15.148:0/847621975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e7dca4b0 con 0x5619e7d36af0 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c40a4e640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7dc6910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e010 tx=0x7f0c2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc40e34b3f1476ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c17fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c17fff640 1 -- 172.21.15.148:0/847621975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c034060 con 0x5619e7d36af0 2024-04-25T09:56:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c17fff640 1 -- 172.21.15.148:0/847621975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0c2c02f410 con 0x5619e7d36af0 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c3405cda0 con 0x5619e7d36af0 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c17fff640 1 -- 172.21.15.148:0/847621975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c2c03a050 con 0x5619e7d36af0 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 10 client.5616 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5619e7d21270 con 0x5619e7d36af0 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c17fff640 1 -- 172.21.15.148:0/847621975 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3405cda0 con 0x5619e7d36af0 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c17fff640 1 -- 172.21.15.148:0/847621975 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0c2c03d020 con 0x5619e7d36af0 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 10 client.5616 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 10 client.5616 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 20 client.5616 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d50790 0x5619e7d546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 4 client.5616 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x5619e7c79450 con 0x5619e7d50790 2024-04-25T09:56:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.064+0000 7f0c4124f640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d50790 0x5619e7d546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.065+0000 7f0c17fff640 10 client.5616 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.065+0000 7f0c4124f640 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d50790 0x5619e7d546b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a2b6e65aa1395c server_cookie=c16ebc86147e67a3 in_seq=0 out_seq=0 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c17fff640 1 -- 172.21.15.148:0/847621975 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5619e7c79450 con 0x5619e7d50790 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c17fff640 10 client.5616 handle_command_reply: tid=0 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 client.5616 shutdown 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 msgr2=0x5619e7dc6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7dc6910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e010 tx=0x7f0c2c008040 comp rx=0 tx=0).stop 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 shutdown_connections 2024-04-25T09:56:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x5619e7d50790 0x5619e7d546b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c17fff640 0 client.5616 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5619e7d58c40 0x5619e7d54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619e7d36af0 0x5619e7dc6910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 --2- 172.21.15.148:0/847621975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5619e7d36710 0x5619e7d0d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 >> 172.21.15.148:0/847621975 conn(0x5619e7d09370 msgr2=0x5619e7d2aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 shutdown_connections 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.069+0000 7f0c461b8740 1 -- 172.21.15.148:0/847621975 wait complete. 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c461b8740 20 client.5616 trim_cache size 0 max 16384 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:56:24.076 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:56:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "d51e611d-5c69-4c92-bce0-9c5c68ecbc7b", 2024-04-25T09:56:24.077 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:56:24.077 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0c2006b4d0 msgr2=0x7f0c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0c2006b4d0 0x7f0c2006d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c38034000 tx=0x7f0c3802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 msgr2=0x7f0c3c15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 0x7f0c3c15b550 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c7b0 tx=0x7f0c3400cc70 comp rx=0 tx=0).stop 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 shutdown_connections 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0c2006b4d0 0x7f0c2006d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c3c0cfd30 0x7f0c3c15b550 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c3c09a090 0x7f0c3c15b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 --2- 172.21.15.148:0/528395022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c3c099cb0 0x7f0c3c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 >> 172.21.15.148:0/528395022 conn(0x7f0c3c0b3290 msgr2=0x7f0c3c0704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 shutdown_connections 2024-04-25T09:56:24.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.070+0000 7f0c434b3640 1 -- 172.21.15.148:0/528395022 wait complete. 2024-04-25T09:56:24.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:24.085 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:56:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.207+0000 7f233949a640 1 Processor -- start 2024-04-25T09:56:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.207+0000 7f233949a640 1 -- start start 2024-04-25T09:56:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f233949a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f233406bff0 0x7f23340b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f233949a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23340b5970 0x7f23340d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f233949a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233415e110 con 0x7f233415dd30 2024-04-25T09:56:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f233949a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233406d000 con 0x7f233406bff0 2024-04-25T09:56:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f233949a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23340b2fd0 con 0x7f23340b5970 2024-04-25T09:56:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f23337fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f233406bff0 0x7f23340b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f23337fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f233406bff0 0x7f23340b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37466/0 (socket says 172.21.15.148:37466) 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f23337fe640 1 -- 172.21.15.148:0/530345438 learned_addr learned my addr 172.21.15.148:0/530345438 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.208+0000 7f2338c99640 1 --2- 172.21.15.148:0/530345438 >> v2:172.21.15.188:3300/0 conn(0x7f23340b5970 0x7f23340d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.209+0000 7f2332ffd640 1 -- 172.21.15.148:0/530345438 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668584304 0 0) 0x7f233415e110 con 0x7f233415dd30 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.209+0000 7f2332ffd640 1 -- 172.21.15.148:0/530345438 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c003520 con 0x7f233415dd30 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.209+0000 7f23337fe640 1 -- 172.21.15.148:0/530345438 >> v2:172.21.15.188:3300/0 conn(0x7f23340b5970 msgr2=0x7f23340d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.209+0000 7f23337fe640 1 --2- 172.21.15.148:0/530345438 >> v2:172.21.15.188:3300/0 conn(0x7f23340b5970 0x7f23340d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.209+0000 7f23337fe640 1 -- 172.21.15.148:0/530345438 >> v1:172.21.15.188:6789/0 conn(0x7f233415dd30 legacy=0x7f23340cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.209+0000 7f23337fe640 1 -- 172.21.15.148:0/530345438 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233406c480 con 0x7f233406bff0 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.210+0000 7f23337fe640 1 --2- 172.21.15.148:0/530345438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233406bff0 0x7f23340b5430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2334161760 tx=0x7f232802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2418728171aa0642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.210+0000 7f2332ffd640 1 -- 172.21.15.148:0/530345438 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328037070 con 0x7f233406bff0 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.210+0000 7f2332ffd640 1 -- 172.21.15.148:0/530345438 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328032440 con 0x7f233406bff0 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.210+0000 7f2332ffd640 1 -- 172.21.15.148:0/530345438 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328032810 con 0x7f233406bff0 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 -- 172.21.15.148:0/530345438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233406bff0 msgr2=0x7f23340b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 --2- 172.21.15.148:0/530345438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233406bff0 0x7f23340b5430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2334161760 tx=0x7f232802a270 comp rx=0 tx=0).stop 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 -- 172.21.15.148:0/530345438 shutdown_connections 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 --2- 172.21.15.148:0/530345438 >> v2:172.21.15.188:3300/0 conn(0x7f23340b5970 0x7f23340d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 --2- 172.21.15.148:0/530345438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f233406bff0 0x7f23340b5430 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 -- 172.21.15.148:0/530345438 >> 172.21.15.148:0/530345438 conn(0x7f23340d0610 msgr2=0x7f23340d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 -- 172.21.15.148:0/530345438 shutdown_connections 2024-04-25T09:56:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.211+0000 7f233949a640 1 -- 172.21.15.148:0/530345438 wait complete. 2024-04-25T09:56:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.213+0000 7f233949a640 1 Processor -- start 2024-04-25T09:56:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.214+0000 7f233949a640 1 -- start start 2024-04-25T09:56:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f233949a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23340b5970 0x7f23341f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f233949a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 0x7f2334150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f233949a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23341f6810 0x7f23340aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f233949a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233406cd20 con 0x7f23341f6810 2024-04-25T09:56:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f233949a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233406ca40 con 0x7f23340b5970 2024-04-25T09:56:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f233949a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340d26a0 con 0x7f233415dd30 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f2338c99640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23341f6810 0x7f23340aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 0x7f2334150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 0x7f2334150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:48484/0 (socket says 172.21.15.148:48484) 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 -- 172.21.15.148:0/1715948314 learned_addr learned my addr 172.21.15.148:0/1715948314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f2333fff640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23340b5970 0x7f23341f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 -- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23340b5970 msgr2=0x7f23341f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23340b5970 0x7f23341f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 -- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23341f6810 msgr2=0x7f23340aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23341f6810 0x7f23340aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 -- 172.21.15.148:0/1715948314 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334151360 con 0x7f233415dd30 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f2333fff640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23340b5970 0x7f23341f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f2338c99640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23341f6810 0x7f23340aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.215+0000 7f23337fe640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 0x7f2334150e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f232802a2b0 tx=0x7f23280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.216+0000 7f2330ff9640 1 -- 172.21.15.148:0/1715948314 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328041020 con 0x7f233415dd30 2024-04-25T09:56:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.216+0000 7f2330ff9640 1 -- 172.21.15.148:0/1715948314 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232803d620 con 0x7f233415dd30 2024-04-25T09:56:24.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.216+0000 7f2330ff9640 1 -- 172.21.15.148:0/1715948314 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328037040 con 0x7f233415dd30 2024-04-25T09:56:24.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.216+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23341f5770 con 0x7f233415dd30 2024-04-25T09:56:24.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.216+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23341f5d30 con 0x7f233415dd30 2024-04-25T09:56:24.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.218+0000 7f2330ff9640 1 -- 172.21.15.148:0/1715948314 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2328044020 con 0x7f233415dd30 2024-04-25T09:56:24.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.218+0000 7f2330ff9640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f232006b4d0 0x7f232006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.218+0000 7f2333fff640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f232006b4d0 0x7f232006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.218+0000 7f2330ff9640 1 -- 172.21.15.148:0/1715948314 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f23280aa930 con 0x7f233415dd30 2024-04-25T09:56:24.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.219+0000 7f2333fff640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f232006b4d0 0x7f232006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f2324009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.220+0000 7f233d3fe740 1 Processor -- start 2024-04-25T09:56:24.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.220+0000 7f233d3fe740 1 -- start start 2024-04-25T09:56:24.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae794a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae7948100 0x55dae794aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 0x55dae791e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f233d3fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae7883da0 con 0x55dae7947d20 2024-04-25T09:56:24.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae782cb80 con 0x55dae7948100 2024-04-25T09:56:24.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dae787fe40 con 0x55dae791a980 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2333fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae794a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 0x55dae791e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 0x55dae791e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:48490/0 (socket says 172.21.15.148:48490) 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2333fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae794a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37474/0 (socket says 172.21.15.148:37474) 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 -- 172.21.15.148:0/2764316126 learned_addr learned my addr 172.21.15.148:0/2764316126 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f23337fe640 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae7948100 0x55dae794aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 -- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae7948100 msgr2=0x55dae794aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae7948100 0x55dae794aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 -- 172.21.15.148:0/2764316126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 msgr2=0x55dae794a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae794a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.221+0000 7f2338c99640 1 -- 172.21.15.148:0/2764316126 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae7836f10 con 0x55dae791a980 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.222+0000 7f2338c99640 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 0x55dae791e190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e7f0 tx=0x7f232c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25c18834f2d76b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.222+0000 7f2319ffb640 1 -- 172.21.15.148:0/2764316126 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c01b030 con 0x55dae791a980 2024-04-25T09:56:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.222+0000 7f2319ffb640 1 -- 172.21.15.148:0/2764316126 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c007aa0 con 0x55dae791a980 2024-04-25T09:56:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.222+0000 7f2319ffb640 1 -- 172.21.15.148:0/2764316126 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c007e70 con 0x55dae791a980 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 -- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 msgr2=0x55dae791e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 0x55dae791e190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e7f0 tx=0x7f232c009750 comp rx=0 tx=0).stop 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 -- 172.21.15.148:0/2764316126 shutdown_connections 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae791a980 0x55dae791e190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae7948100 0x55dae794aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 --2- 172.21.15.148:0/2764316126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae794a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 -- 172.21.15.148:0/2764316126 >> 172.21.15.148:0/2764316126 conn(0x55dae791cbf0 msgr2=0x55dae7965530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 -- 172.21.15.148:0/2764316126 shutdown_connections 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.223+0000 7f233d3fe740 1 -- 172.21.15.148:0/2764316126 wait complete. 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.224+0000 7f233d3fe740 1 -- start start 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.224+0000 7f233d3fe740 1 Processor -- start 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 0x55dae7942590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 0x55dae793ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae79647b0 0x55dae7960400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae791fda0 con 0x55dae7948100 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae78224b0 con 0x55dae79647b0 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dae7920080 con 0x55dae7947d20 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f233d3fe740 10 client.0 _ll_register_callbacks cb 0x55dae7862180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f23337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 0x55dae793ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2333fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 0x55dae7942590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2333fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 0x55dae7942590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:48492/0 (socket says 172.21.15.148:48492) 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f23337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 0x55dae793ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37478/0 (socket says 172.21.15.148:37478) 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2333fff640 1 -- 172.21.15.148:0/3701182757 learned_addr learned my addr 172.21.15.148:0/3701182757 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2338c99640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae79647b0 0x55dae7960400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2333fff640 1 -- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae79647b0 msgr2=0x55dae7960400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2333fff640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae79647b0 0x55dae7960400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.225+0000 7f2333fff640 1 -- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 msgr2=0x55dae793ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2333fff640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 0x55dae793ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2333fff640 1 -- 172.21.15.148:0/3701182757 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dae76627d0 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2338c99640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae79647b0 0x55dae7960400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f23337fe640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 0x55dae793ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3301/0 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2333fff640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 0x55dae7942590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f23240149d0 tx=0x7f2324001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1411211010df14dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2312ffd640 1 -- 172.21.15.148:0/3701182757 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324025020 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f232c007aa0 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2312ffd640 1 -- 172.21.15.148:0/3701182757 <== mon.2 v2:172.21.15.188:3301/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2324013b40 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f2312ffd640 1 -- 172.21.15.148:0/3701182757 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f232402f050 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f233d3fe740 10 client.4910 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.226+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55dae75983f0 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f2312ffd640 1 -- 172.21.15.148:0/3701182757 <== mon.2 v2:172.21.15.188:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232401b070 con 0x55dae7947d20 2024-04-25T09:56:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f2312ffd640 1 -- 172.21.15.148:0/3701182757 <== mon.2 v2:172.21.15.188:3301/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2324032020 con 0x55dae7947d20 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f233d3fe740 10 client.4910 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f233d3fe740 10 client.4910 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f233d3fe740 20 client.4910 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f233d3fe740 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae793fd70 0x55dae7940150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f233d3fe740 4 client.4910 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f232c007aa0 con 0x55dae793fd70 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f23337fe640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae793fd70 0x55dae7940150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.227+0000 7f2312ffd640 10 client.4910 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.228+0000 7f23337fe640 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae793fd70 0x55dae7940150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936fdfa05dc828ac server_cookie=5710d4bfc7bd1f68 in_seq=0 out_seq=0 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f2312ffd640 1 -- 172.21.15.148:0/3701182757 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f232c007aa0 con 0x55dae793fd70 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f2312ffd640 10 client.4910 handle_command_reply: tid=0 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 client.4910 shutdown 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 msgr2=0x55dae7942590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 0x55dae7942590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f23240149d0 tx=0x7f2324001150 comp rx=0 tx=0).stop 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 shutdown_connections 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae793fd70 0x55dae7940150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae79647b0 0x55dae7960400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7948100 0x55dae793ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f2312ffd640 0 client.4910 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 --2- 172.21.15.148:0/3701182757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7947d20 0x55dae7942590 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 >> 172.21.15.148:0/3701182757 conn(0x55dae791a980 msgr2=0x55dae79463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 shutdown_connections 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 1 -- 172.21.15.148:0/3701182757 wait complete. 2024-04-25T09:56:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.229+0000 7f233d3fe740 20 client.4910 trim_cache size 0 max 16384 2024-04-25T09:56:24.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.250+0000 7f233d3fe740 1 Processor -- start 2024-04-25T09:56:24.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 -- start start 2024-04-25T09:56:24.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae7962530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae7974890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae793e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae79663d0 con 0x55dae7947d20 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340704d0 con 0x55dae793fd70 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233406f690 con 0x55dae7948100 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f23337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae7974890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f2338c99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae793e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f23337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae7974890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37484/0 (socket says 172.21.15.148:37484) 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f23337fe640 1 -- 172.21.15.148:0/631419957 learned_addr learned my addr 172.21.15.148:0/631419957 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.251+0000 7f2333fff640 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae7962530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f2338c99640 1 -- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 msgr2=0x55dae7962530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f2338c99640 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae7962530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f2338c99640 1 -- 172.21.15.148:0/631419957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 msgr2=0x55dae7974890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f2338c99640 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae7974890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f2338c99640 1 -- 172.21.15.148:0/631419957 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae75580f0 con 0x55dae7948100 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f23337fe640 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae7974890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f2338c99640 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae793e3e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e7c0 tx=0x7f232c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378e1acdc6cc693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f23197fa640 1 -- 172.21.15.148:0/631419957 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c003b90 con 0x55dae7948100 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f23197fa640 1 -- 172.21.15.148:0/631419957 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c007aa0 con 0x55dae7948100 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.252+0000 7f23197fa640 1 -- 172.21.15.148:0/631419957 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c004040 con 0x55dae7948100 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 -- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 msgr2=0x55dae793e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae793e3e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f232c00e7c0 tx=0x7f232c017000 comp rx=0 tx=0).stop 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 -- 172.21.15.148:0/631419957 shutdown_connections 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae793e3e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae7974890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 --2- 172.21.15.148:0/631419957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae7962530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 -- 172.21.15.148:0/631419957 >> 172.21.15.148:0/631419957 conn(0x55dae791a980 msgr2=0x55dae79421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 -- 172.21.15.148:0/631419957 shutdown_connections 2024-04-25T09:56:24.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.253+0000 7f233d3fe740 1 -- 172.21.15.148:0/631419957 wait complete. 2024-04-25T09:56:24.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.254+0000 7f233d3fe740 1 -- start start 2024-04-25T09:56:24.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.254+0000 7f233d3fe740 1 Processor -- start 2024-04-25T09:56:24.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae791ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae791b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae7966660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233406f090 con 0x55dae7947d20 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23341580a0 con 0x55dae793fd70 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dae79645c0 con 0x55dae7948100 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f23337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae791b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f233d3fe740 10 client.0 _ll_register_callbacks cb 0x7f232c009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.255+0000 7f2333fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae791ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae791b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37486/0 (socket says 172.21.15.148:37486) 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f2333fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae791ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:39578/0 (socket says 172.21.15.148:39578) 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 -- 172.21.15.148:0/4168689375 learned_addr learned my addr 172.21.15.148:0/4168689375 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f2338c99640 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae7966660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 -- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 msgr2=0x55dae7966660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae7966660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 -- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 msgr2=0x55dae791ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae791ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 -- 172.21.15.148:0/4168689375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dae791bc40 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f23337fe640 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae791b700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2328009630 tx=0x7f232803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2529eb00b15fd910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f2312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f2312ffd640 1 -- 172.21.15.148:0/4168689375 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23280791f0 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f232c007aa0 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.256+0000 7f2312ffd640 1 -- 172.21.15.148:0/4168689375 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2328079390 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f2312ffd640 1 -- 172.21.15.148:0/4168689375 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f232809e050 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 10 client.5634 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55dae791af60 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f2312ffd640 1 -- 172.21.15.148:0/4168689375 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c007aa0 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f2312ffd640 1 -- 172.21.15.148:0/4168689375 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f232803a030 con 0x55dae7947d20 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 10 client.5634 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:24.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 10 client.5634 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 20 client.5634 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae79647b0 0x55dae79743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 4 client.5634 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dae78912e0 con 0x55dae79647b0 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.257+0000 7f2333fff640 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae79647b0 0x55dae79743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.258+0000 7f2333fff640 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae79647b0 0x55dae79743d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7ed74d50d9b164 server_cookie=34ae3fa4e4f08e08 in_seq=0 out_seq=0 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.258+0000 7f2312ffd640 10 client.5634 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.258+0000 7f2312ffd640 1 -- 172.21.15.148:0/4168689375 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+275 (crc 0 0 0) 0x55dae78912e0 con 0x55dae79647b0 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.258+0000 7f2312ffd640 10 client.5634 handle_command_reply: tid=0 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.258+0000 7f233d3fe740 1 client.5634 shutdown 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 msgr2=0x55dae791b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae791b700 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2328009630 tx=0x7f232803f040 comp rx=0 tx=0).stop 2024-04-25T09:56:24.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 shutdown_connections 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55dae79647b0 0x55dae79743d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dae7948100 0x55dae7966660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dae7947d20 0x55dae791b700 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 --2- 172.21.15.148:0/4168689375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dae793fd70 0x55dae791ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 >> 172.21.15.148:0/4168689375 conn(0x55dae791a980 msgr2=0x55dae7919ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f2312ffd640 0 client.5634 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 shutdown_connections 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 1 -- 172.21.15.148:0/4168689375 wait complete. 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.259+0000 7f233d3fe740 20 client.5634 trim_cache size 0 max 16384 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:56:24.267 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "scrub active (1154 inodes in the stack)", 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: "d51e611d-5c69-4c92-bce0-9c5c68ecbc7b": { 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "d51e611d-5c69-4c92-bce0-9c5c68ecbc7b", 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:24.268 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:56:24.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f232006b4d0 msgr2=0x7f232006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f232006b4d0 0x7f232006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2324007fd0 tx=0x7f2324009040 comp rx=0 tx=0).stop 2024-04-25T09:56:24.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 msgr2=0x7f2334150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 0x7f2334150e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f232802a2b0 tx=0x7f23280046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 shutdown_connections 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f232006b4d0 0x7f232006d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23341f6810 0x7f23340aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f233415dd30 0x7f2334150e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 --2- 172.21.15.148:0/1715948314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23340b5970 0x7f23341f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 >> 172.21.15.148:0/1715948314 conn(0x7f23340d0610 msgr2=0x7f23340d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 shutdown_connections 2024-04-25T09:56:24.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:24.260+0000 7f233949a640 1 -- 172.21.15.148:0/1715948314 wait complete. 2024-04-25T09:56:24.278 INFO:tasks.cephfs.filesystem:scrub status for tag:d51e611d-5c69-4c92-bce0-9c5c68ecbc7b - {'path': '/', 'tag': 'd51e611d-5c69-4c92-bce0-9c5c68ecbc7b', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T09:56:53.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:53.663 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:56:54.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:54.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T09:56:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.388+0000 7f2095761640 1 Processor -- start 2024-04-25T09:56:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.388+0000 7f2095761640 1 -- start start 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2095761640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20900ced80 0x7f20900d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2095761640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209015e000 con 0x7f2090161710 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2095761640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20900d1de0 con 0x7f209015dc20 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2095761640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2090161c60 con 0x7f20900ced80 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2087fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20900ced80 0x7f20900d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f208f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2090161710 0x7f20900d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36406/0 (socket says 172.21.15.148:36406) 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f208f7fe640 1 -- 172.21.15.148:0/352227074 learned_addr learned my addr 172.21.15.148:0/352227074 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2087fff640 1 -- 172.21.15.148:0/352227074 >> v1:172.21.15.188:6789/0 conn(0x7f209015dc20 legacy=0x7f20900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2087fff640 1 -- 172.21.15.148:0/352227074 >> v1:172.21.15.148:6789/0 conn(0x7f2090161710 legacy=0x7f20900d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.389+0000 7f2087fff640 1 -- 172.21.15.148:0/352227074 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090161f40 con 0x7f20900ced80 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.390+0000 7f2087fff640 1 --2- 172.21.15.148:0/352227074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20900d22e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f20780097a0 tx=0x7f207802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969d3835fd3847a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.390+0000 7f208e7fc640 1 -- 172.21.15.148:0/352227074 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078037070 con 0x7f20900ced80 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.390+0000 7f208e7fc640 1 -- 172.21.15.148:0/352227074 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2078032440 con 0x7f20900ced80 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.390+0000 7f208e7fc640 1 -- 172.21.15.148:0/352227074 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078032810 con 0x7f20900ced80 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 -- 172.21.15.148:0/352227074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 msgr2=0x7f20900d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 --2- 172.21.15.148:0/352227074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20900d22e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f20780097a0 tx=0x7f207802a230 comp rx=0 tx=0).stop 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 -- 172.21.15.148:0/352227074 shutdown_connections 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 --2- 172.21.15.148:0/352227074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20900d22e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 -- 172.21.15.148:0/352227074 >> 172.21.15.148:0/352227074 conn(0x7f20900d0420 msgr2=0x7f20900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 -- 172.21.15.148:0/352227074 shutdown_connections 2024-04-25T09:56:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.391+0000 7f2095761640 1 -- 172.21.15.148:0/352227074 wait complete. 2024-04-25T09:56:54.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.392+0000 7f2095761640 1 Processor -- start 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.392+0000 7f2095761640 1 -- start start 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2095761640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20901f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2095761640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f209015dc20 0x7f20900c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2095761640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2090161710 0x7f20901606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2095761640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20900b5e50 con 0x7f20900ced80 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2095761640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090162500 con 0x7f2090161710 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2095761640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20900b5fd0 con 0x7f209015dc20 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20901f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20901f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43910/0 (socket says 172.21.15.148:43910) 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 -- 172.21.15.148:0/2581172009 learned_addr learned my addr 172.21.15.148:0/2581172009 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2087fff640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f209015dc20 0x7f20900c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208f7fe640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2090161710 0x7f20901606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 -- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f209015dc20 msgr2=0x7f20900c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f209015dc20 0x7f20900c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 -- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2090161710 msgr2=0x7f20901606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2090161710 0x7f20901606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 -- 172.21.15.148:0/2581172009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20901f7fa0 con 0x7f20900ced80 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f2087fff640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f209015dc20 0x7f20900c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208f7fe640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2090161710 0x7f20901606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f208effd640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20901f7a60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f208000c400 tx=0x7f208000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.393+0000 7f20877fe640 1 -- 172.21.15.148:0/2581172009 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800052d0 con 0x7f20900ced80 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.394+0000 7f20877fe640 1 -- 172.21.15.148:0/2581172009 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080005490 con 0x7f20900ced80 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.394+0000 7f20877fe640 1 -- 172.21.15.148:0/2581172009 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080007260 con 0x7f20900ced80 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.394+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209015f600 con 0x7f20900ced80 2024-04-25T09:56:54.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.394+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209015fbc0 con 0x7f20900ced80 2024-04-25T09:56:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.395+0000 7f20877fe640 1 -- 172.21.15.148:0/2581172009 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2080007400 con 0x7f20900ced80 2024-04-25T09:56:54.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.395+0000 7f20877fe640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207006b4d0 0x7f207006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.395+0000 7f20877fe640 1 -- 172.21.15.148:0/2581172009 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f2080089910 con 0x7f20900ced80 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.395+0000 7f2087fff640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207006b4d0 0x7f207006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.396+0000 7f2087fff640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207006b4d0 0x7f207006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2078034000 tx=0x7f207802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.396+0000 7f209845e740 1 Processor -- start 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 -- start start 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b19673770 0x558b19676f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b195dcda0 con 0x558b196a0b10 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19585b80 con 0x558b19673770 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b195d8e40 con 0x558b196a0ef0 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43918/0 (socket says 172.21.15.148:43918) 2024-04-25T09:56:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.397+0000 7f208effd640 1 -- 172.21.15.148:0/1378780370 learned_addr learned my addr 172.21.15.148:0/1378780370 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f208effd640 1 -- 172.21.15.148:0/1378780370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 msgr2=0x558b196a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f208effd640 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f208effd640 1 -- 172.21.15.148:0/1378780370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b19673770 msgr2=0x558b19676f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f208effd640 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b19673770 0x558b19676f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f208effd640 1 -- 172.21.15.148:0/1378780370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b1958ff10 con 0x558b196a0b10 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f208effd640 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196a3320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f208000cfd0 tx=0x7f208000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4263b69a377f591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f2084ff9640 1 -- 172.21.15.148:0/1378780370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800561d0 con 0x558b196a0b10 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f2084ff9640 1 -- 172.21.15.148:0/1378780370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080054540 con 0x558b196a0b10 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.398+0000 7f2084ff9640 1 -- 172.21.15.148:0/1378780370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208005e2b0 con 0x558b196a0b10 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 -- 172.21.15.148:0/1378780370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 msgr2=0x558b196a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196a3320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f208000cfd0 tx=0x7f208000a000 comp rx=0 tx=0).stop 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 -- 172.21.15.148:0/1378780370 shutdown_connections 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b19673770 0x558b19676f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 --2- 172.21.15.148:0/1378780370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196a3320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 -- 172.21.15.148:0/1378780370 >> 172.21.15.148:0/1378780370 conn(0x558b196759e0 msgr2=0x558b196be320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 -- 172.21.15.148:0/1378780370 shutdown_connections 2024-04-25T09:56:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 -- 172.21.15.148:0/1378780370 wait complete. 2024-04-25T09:56:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 -- start start 2024-04-25T09:56:54.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.399+0000 7f209845e740 1 Processor -- start 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0ef0 0x558b19672a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196977e0 0x558b19697c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19678b90 con 0x558b196a0b10 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1957b4b0 con 0x558b196a0ef0 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b19678e70 con 0x558b196977e0 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f209845e740 10 client.0 _ll_register_callbacks cb 0x558b195bb180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43924/0 (socket says 172.21.15.148:43924) 2024-04-25T09:56:54.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 -- 172.21.15.148:0/2526310915 learned_addr learned my addr 172.21.15.148:0/2526310915 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208f7fe640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196977e0 0x558b19697c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f2087fff640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0ef0 0x558b19672a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 -- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196977e0 msgr2=0x558b19697c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196977e0 0x558b19697c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 -- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0ef0 msgr2=0x558b19672a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0ef0 0x558b19672a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208effd640 1 -- 172.21.15.148:0/2526310915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b193bb7d0 con 0x558b196a0b10 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f208f7fe640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196977e0 0x558b19697c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.400+0000 7f2087fff640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0ef0 0x558b19672a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f208effd640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196bfe60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f208000cdc0 tx=0x7f2080000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8a64cb5d98a592 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f206dffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f206dffb640 1 -- 172.21.15.148:0/2526310915 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208005e070 con 0x558b196a0b10 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f206dffb640 1 -- 172.21.15.148:0/2526310915 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f208005e210 con 0x558b196a0b10 2024-04-25T09:56:54.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2080054540 con 0x558b196a0b10 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f206dffb640 1 -- 172.21.15.148:0/2526310915 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f208007d050 con 0x558b196a0b10 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 10 client.5649 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558b192f13f0 con 0x558b196a0b10 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f206dffb640 1 -- 172.21.15.148:0/2526310915 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080054540 con 0x558b196a0b10 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f206dffb640 1 -- 172.21.15.148:0/2526310915 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2080057320 con 0x558b196a0b10 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 10 client.5649 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 10 client.5649 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 20 client.5649 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b196ce330 0x558b196a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 4 client.5649 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b195ea2e0 con 0x558b196ce330 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.401+0000 7f2087fff640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b196ce330 0x558b196a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f206dffb640 10 client.5649 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f2087fff640 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b196ce330 0x558b196a4a70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff3d50ec2840ed4 server_cookie=e1efc3ae00745430 in_seq=0 out_seq=0 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f206dffb640 1 -- 172.21.15.148:0/2526310915 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b195ea2e0 con 0x558b196ce330 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f206dffb640 10 client.5649 handle_command_reply: tid=0 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 client.5649 shutdown 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 msgr2=0x558b196bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196bfe60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f208000cdc0 tx=0x7f2080000bc0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 shutdown_connections 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b196ce330 0x558b196a4a70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196977e0 0x558b19697c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0ef0 0x558b19672a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 --2- 172.21.15.148:0/2526310915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b196bfe60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f206dffb640 0 client.5649 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 >> 172.21.15.148:0/2526310915 conn(0x558b19673770 msgr2=0x558b196783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 shutdown_connections 2024-04-25T09:56:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 1 -- 172.21.15.148:0/2526310915 wait complete. 2024-04-25T09:56:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.402+0000 7f209845e740 20 client.5649 trim_cache size 0 max 16384 2024-04-25T09:56:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 Processor -- start 2024-04-25T09:56:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 -- start start 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196977e0 0x558b196a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b1969f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b1969f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2090200650 con 0x558b196a0b10 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20901ff830 con 0x558b196977e0 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2090159c60 con 0x558b196a0ef0 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f2087fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b1969f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f2087fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b1969f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43930/0 (socket says 172.21.15.148:43930) 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.418+0000 7f2087fff640 1 -- 172.21.15.148:0/924400972 learned_addr learned my addr 172.21.15.148:0/924400972 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f2087fff640 1 -- 172.21.15.148:0/924400972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 msgr2=0x558b1969f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f2087fff640 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b1969f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f2087fff640 1 -- 172.21.15.148:0/924400972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196977e0 msgr2=0x558b196a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f2087fff640 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196977e0 0x558b196a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f2087fff640 1 -- 172.21.15.148:0/924400972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b192b10f0 con 0x558b196a0b10 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f2087fff640 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b1969f190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2078000c00 tx=0x7f207802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ef4c3b152f99d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f206ffff640 1 -- 172.21.15.148:0/924400972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078051070 con 0x558b196a0b10 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f206ffff640 1 -- 172.21.15.148:0/924400972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207804b2d0 con 0x558b196a0b10 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f206ffff640 1 -- 172.21.15.148:0/924400972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078037040 con 0x558b196a0b10 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 -- 172.21.15.148:0/924400972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 msgr2=0x558b1969f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b1969f190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2078000c00 tx=0x7f207802c000 comp rx=0 tx=0).stop 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 -- 172.21.15.148:0/924400972 shutdown_connections 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b1969f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196a0b10 0x558b1969f190 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 --2- 172.21.15.148:0/924400972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196977e0 0x558b196a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.419+0000 7f209845e740 1 -- 172.21.15.148:0/924400972 >> 172.21.15.148:0/924400972 conn(0x558b19673770 msgr2=0x558b196a5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.420+0000 7f209845e740 1 -- 172.21.15.148:0/924400972 shutdown_connections 2024-04-25T09:56:54.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.420+0000 7f209845e740 1 -- 172.21.15.148:0/924400972 wait complete. 2024-04-25T09:56:54.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.420+0000 7f209845e740 1 -- start start 2024-04-25T09:56:54.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.420+0000 7f209845e740 1 Processor -- start 2024-04-25T09:56:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 0x558b196b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0b10 0x558b196a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b19673420 con 0x558b196977e0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209015a260 con 0x558b196a0b10 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b196770b0 con 0x558b196a0ef0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f209845e740 10 client.0 _ll_register_callbacks cb 0x7f20800541f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 0x558b196b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 0x558b196b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43932/0 (socket says 172.21.15.148:43932) 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 -- 172.21.15.148:0/825048896 learned_addr learned my addr 172.21.15.148:0/825048896 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f2087fff640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0b10 0x558b196a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208f7fe640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 -- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 msgr2=0x558b196a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 -- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0b10 msgr2=0x558b196a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0b10 0x558b196a6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 -- 172.21.15.148:0/825048896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b196a43e0 con 0x558b196977e0 2024-04-25T09:56:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f208effd640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 0x558b196b4530 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f208005e480 tx=0x7f2080087b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad346e9965c343ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f206dffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.421+0000 7f206dffb640 1 -- 172.21.15.148:0/825048896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080054540 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 1 -- 172.21.15.148:0/825048896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f2080053c20 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f207804b2d0 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 1 -- 172.21.15.148:0/825048896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f208005e280 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 10 client.5655 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558b19728cd0 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 1 -- 172.21.15.148:0/825048896 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207804b2d0 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 1 -- 172.21.15.148:0/825048896 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2080056d90 con 0x558b196977e0 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 10 client.5655 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 10 client.5655 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 20 client.5655 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b19722460 0x558b196be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 4 client.5655 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b195e4630 con 0x558b19722460 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f2087fff640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b19722460 0x558b196be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 10 client.5655 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f2087fff640 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b19722460 0x558b196be320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995bd462523c34d3 server_cookie=fbc205cb0f10620e in_seq=0 out_seq=0 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 1 -- 172.21.15.148:0/825048896 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558b195e4630 con 0x558b19722460 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f206dffb640 10 client.5655 handle_command_reply: tid=0 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 client.5655 shutdown 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 msgr2=0x558b196b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.422+0000 7f209845e740 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 0x558b196b4530 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f208005e480 tx=0x7f2080087b50 comp rx=0 tx=0).stop 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 shutdown_connections 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x558b19722460 0x558b196be320 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558b196a0ef0 0x558b196a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b196a0b10 0x558b196a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f206dffb640 0 client.5655 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 --2- 172.21.15.148:0/825048896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558b196977e0 0x558b196b4530 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 >> 172.21.15.148:0/825048896 conn(0x558b19673770 msgr2=0x558b19672810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 shutdown_connections 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 1 -- 172.21.15.148:0/825048896 wait complete. 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f209845e740 20 client.5655 trim_cache size 0 max 16384 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T09:56:54.432 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:56:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207006b4d0 msgr2=0x7f207006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207006b4d0 0x7f207006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2078034000 tx=0x7f207802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 msgr2=0x7f20901f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20901f7a60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f208000c400 tx=0x7f208000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 shutdown_connections 2024-04-25T09:56:54.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207006b4d0 0x7f207006d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2090161710 0x7f20901606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f209015dc20 0x7f20900c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 --2- 172.21.15.148:0/2581172009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20900ced80 0x7f20901f7a60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 >> 172.21.15.148:0/2581172009 conn(0x7f20900d0420 msgr2=0x7f20900b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 shutdown_connections 2024-04-25T09:56:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.423+0000 7f2095761640 1 -- 172.21.15.148:0/2581172009 wait complete. 2024-04-25T09:56:54.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:56:54.437 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:56:54.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T09:56:54.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.533+0000 7fdd64d5c640 1 Processor -- start 2024-04-25T09:56:54.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.533+0000 7fdd64d5c640 1 -- start start 2024-04-25T09:56:54.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd64d5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd60071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd64d5c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd600d2400 0x7fdd600cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd64d5c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600cf430 con 0x7fdd6015d160 2024-04-25T09:56:54.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd64d5c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6015cee0 con 0x7fdd600d2400 2024-04-25T09:56:54.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd64d5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600d0040 con 0x7fdd6015baf0 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd600d2400 0x7fdd600cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5ed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdd6015d160 0x7fdd600d04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36414/0 (socket says 172.21.15.148:36414) 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd600d2400 0x7fdd600cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43938/0 (socket says 172.21.15.148:43938) 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5ed9d640 1 -- 172.21.15.148:0/2056612363 learned_addr learned my addr 172.21.15.148:0/2056612363 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/2056612363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd60071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2056612363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 msgr2=0x7fdd60071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/2056612363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd60071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2056612363 >> v1:172.21.15.148:6789/0 conn(0x7fdd6015d160 legacy=0x7fdd600d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2056612363 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600cf710 con 0x7fdd600d2400 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.534+0000 7fdd5d59a640 1 -- 172.21.15.148:0/2056612363 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893003181 0 0) 0x7fdd600cf430 con 0x7fdd6015d160 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.535+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/2056612363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd600d2400 0x7fdd600cc020 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdd540097a0 tx=0x7fdd5402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ded3ffe0e56289b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.535+0000 7fdd5d59a640 1 -- 172.21.15.148:0/2056612363 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54037070 con 0x7fdd600d2400 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.535+0000 7fdd5d59a640 1 -- 172.21.15.148:0/2056612363 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5402ae50 con 0x7fdd600d2400 2024-04-25T09:56:54.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.535+0000 7fdd5d59a640 1 -- 172.21.15.148:0/2056612363 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54032660 con 0x7fdd600d2400 2024-04-25T09:56:54.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 -- 172.21.15.148:0/2056612363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd600d2400 msgr2=0x7fdd600cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/2056612363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd600d2400 0x7fdd600cc020 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdd540097a0 tx=0x7fdd5402a100 comp rx=0 tx=0).stop 2024-04-25T09:56:54.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 -- 172.21.15.148:0/2056612363 shutdown_connections 2024-04-25T09:56:54.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/2056612363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd600d2400 0x7fdd600cc020 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/2056612363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd60071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 -- 172.21.15.148:0/2056612363 >> 172.21.15.148:0/2056612363 conn(0x7fdd600d17c0 msgr2=0x7fdd600d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 -- 172.21.15.148:0/2056612363 shutdown_connections 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.536+0000 7fdd64d5c640 1 -- 172.21.15.148:0/2056612363 wait complete. 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.537+0000 7fdd64d5c640 1 Processor -- start 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.537+0000 7fdd64d5c640 1 -- start start 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd64d5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd601502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd64d5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 0x7fdd601561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd64d5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd601566e0 0x7fdd601533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd64d5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60071bd0 con 0x7fdd6015d160 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd64d5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600cfcd0 con 0x7fdd601566e0 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd64d5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60071d50 con 0x7fdd6015baf0 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 0x7fdd601561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 0x7fdd601561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43952/0 (socket says 172.21.15.148:43952) 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/4290004186 learned_addr learned my addr 172.21.15.148:0/4290004186 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd601502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5ed9d640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd601566e0 0x7fdd601533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 msgr2=0x7fdd601502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd601502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd601566e0 msgr2=0x7fdd601533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd601566e0 0x7fdd601533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/4290004186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600b6a60 con 0x7fdd6015d160 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.538+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 0x7fdd601561a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdd540098c0 tx=0x7fdd540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.539+0000 7fdd4effd640 1 -- 172.21.15.148:0/4290004186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54042070 con 0x7fdd6015d160 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.539+0000 7fdd4effd640 1 -- 172.21.15.148:0/4290004186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5403e670 con 0x7fdd6015d160 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.539+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd600b6d20 con 0x7fdd6015d160 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.539+0000 7fdd4effd640 1 -- 172.21.15.148:0/4290004186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54037040 con 0x7fdd6015d160 2024-04-25T09:56:54.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.539+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd600b72e0 con 0x7fdd6015d160 2024-04-25T09:56:54.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.540+0000 7fdd4effd640 1 -- 172.21.15.148:0/4290004186 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd54045070 con 0x7fdd6015d160 2024-04-25T09:56:54.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.540+0000 7fdd4effd640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd4006b4d0 0x7fdd4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.541+0000 7fdd4effd640 1 -- 172.21.15.148:0/4290004186 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdd540aabe0 con 0x7fdd6015d160 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.541+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd4006b4d0 0x7fdd4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.541+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd4006b4d0 0x7fdd4006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd50006fd0 tx=0x7fdd50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.541+0000 7fdd67a66740 1 Processor -- start 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 -- start start 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd8282e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd8281080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd81e3b90 con 0x55cfd82bd370 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd818ce40 con 0x55cfd8280ca0 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd81dfc30 con 0x55cfd82ab760 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd8281080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd8281080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43954/0 (socket says 172.21.15.148:43954) 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2817300427 learned_addr learned my addr 172.21.15.148:0/2817300427 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.542+0000 7fdd5ed9d640 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd8282e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 msgr2=0x55cfd82abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82abbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 msgr2=0x55cfd8282e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd8282e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/2817300427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd7fc27d0 con 0x55cfd82bd370 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd8282e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd8281080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdd54079050 tx=0x7fdd54078d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd059039fc6c2dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd3ffff640 1 -- 172.21.15.148:0/2817300427 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5404c050 con 0x55cfd82bd370 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd3ffff640 1 -- 172.21.15.148:0/2817300427 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5402ae50 con 0x55cfd82bd370 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.543+0000 7fdd3ffff640 1 -- 172.21.15.148:0/2817300427 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5407d4a0 con 0x55cfd82bd370 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 -- 172.21.15.148:0/2817300427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 msgr2=0x55cfd8281080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd8281080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdd54079050 tx=0x7fdd54078d70 comp rx=0 tx=0).stop 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 -- 172.21.15.148:0/2817300427 shutdown_connections 2024-04-25T09:56:54.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd8281080 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 --2- 172.21.15.148:0/2817300427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd8282e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 -- 172.21.15.148:0/2817300427 >> 172.21.15.148:0/2817300427 conn(0x55cfd8248870 msgr2=0x55cfd82c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 -- 172.21.15.148:0/2817300427 shutdown_connections 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 -- 172.21.15.148:0/2817300427 wait complete. 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 -- start start 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.544+0000 7fdd67a66740 1 Processor -- start 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd82d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd82d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd81e3b90 con 0x55cfd82ab760 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd81a03e0 con 0x55cfd82bd370 2024-04-25T09:56:54.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd82d8b60 con 0x55cfd8280ca0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd67a66740 10 client.0 _ll_register_callbacks cb 0x55cfd819fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd82d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd82d6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43960/0 (socket says 172.21.15.148:43960) 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/3428938096 learned_addr learned my addr 172.21.15.148:0/3428938096 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.545+0000 7fdd5ed9d640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd82d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 msgr2=0x55cfd82c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 msgr2=0x55cfd82d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd82d6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/3428938096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd7eb80f0 con 0x55cfd82ab760 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd82d6670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdd54079050 tx=0x7fdd54032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc04d34dd189d331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/3428938096 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54081a60 con 0x55cfd82ab760 2024-04-25T09:56:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/3428938096 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdd540815d0 con 0x55cfd82ab760 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd5402ae50 con 0x55cfd82ab760 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/3428938096 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd5407d4a0 con 0x55cfd82ab760 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 10 client.5670 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55cfd8022f60 con 0x55cfd82ab760 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/3428938096 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5402ae50 con 0x55cfd82ab760 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/3428938096 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd540787a0 con 0x55cfd82ab760 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 10 client.5670 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 10 client.5670 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 20 client.5670 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82da160 0x55cfd82da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 4 client.5670 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.546+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfd81f10d0 con 0x55cfd82da160 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82da160 0x55cfd82da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd3d7fa640 10 client.5670 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82da160 0x55cfd82da570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c8423d1dcbc44e server_cookie=cfb85ba762349a7a in_seq=0 out_seq=0 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/3428938096 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfd81f10d0 con 0x55cfd82da160 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd3d7fa640 10 client.5670 handle_command_reply: tid=0 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd67a66740 1 client.5670 shutdown 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 msgr2=0x55cfd82d6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.547+0000 7fdd67a66740 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd82d6670 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdd54079050 tx=0x7fdd54032f50 comp rx=0 tx=0).stop 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 shutdown_connections 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82da160 0x55cfd82da570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd82d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd82d6670 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd3d7fa640 0 client.5670 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 --2- 172.21.15.148:0/3428938096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 >> 172.21.15.148:0/3428938096 conn(0x55cfd8248870 msgr2=0x55cfd82c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 shutdown_connections 2024-04-25T09:56:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 1 -- 172.21.15.148:0/3428938096 wait complete. 2024-04-25T09:56:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.548+0000 7fdd67a66740 20 client.5670 trim_cache size 0 max 16384 2024-04-25T09:56:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 Processor -- start 2024-04-25T09:56:54.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 -- start start 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd832fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd8330350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601fe080 con 0x55cfd82ab760 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601fd240 con 0x55cfd82bd370 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60155790 con 0x55cfd8280ca0 2024-04-25T09:56:54.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd832fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd5dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd832fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43970/0 (socket says 172.21.15.148:43970) 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.562+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/1032561565 learned_addr learned my addr 172.21.15.148:0/1032561565 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/1032561565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 msgr2=0x55cfd82bdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/1032561565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 msgr2=0x55cfd8330350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd8330350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd5dd9b640 1 -- 172.21.15.148:0/1032561565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd82da8e0 con 0x55cfd82ab760 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd832fe10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdd54077880 tx=0x7fdd540a4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cddc17032178fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd3f7fe640 1 -- 172.21.15.148:0/1032561565 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5404c070 con 0x55cfd82ab760 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd3f7fe640 1 -- 172.21.15.148:0/1032561565 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5402ae50 con 0x55cfd82ab760 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd3f7fe640 1 -- 172.21.15.148:0/1032561565 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54081a60 con 0x55cfd82ab760 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 -- 172.21.15.148:0/1032561565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 msgr2=0x55cfd832fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd832fe10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdd54077880 tx=0x7fdd540a4040 comp rx=0 tx=0).stop 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 -- 172.21.15.148:0/1032561565 shutdown_connections 2024-04-25T09:56:54.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd82bd370 0x55cfd8330350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82ab760 0x55cfd832fe10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 --2- 172.21.15.148:0/1032561565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd8280ca0 0x55cfd82bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.563+0000 7fdd67a66740 1 -- 172.21.15.148:0/1032561565 >> 172.21.15.148:0/1032561565 conn(0x55cfd8248870 msgr2=0x55cfd8282ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.564+0000 7fdd67a66740 1 -- 172.21.15.148:0/1032561565 shutdown_connections 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.564+0000 7fdd67a66740 1 -- 172.21.15.148:0/1032561565 wait complete. 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.564+0000 7fdd67a66740 1 -- start start 2024-04-25T09:56:54.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.564+0000 7fdd67a66740 1 Processor -- start 2024-04-25T09:56:54.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd82c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd82d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601fcc70 con 0x55cfd82bd370 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd82ae140 con 0x55cfd8280ca0 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd82dfeb0 con 0x55cfd82ab760 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd67a66740 10 client.0 _ll_register_callbacks cb 0x55cfd82816d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd82d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd82d77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43974/0 (socket says 172.21.15.148:43974) 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 -- 172.21.15.148:0/397635710 learned_addr learned my addr 172.21.15.148:0/397635710 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd82c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5dd9b640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 -- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 msgr2=0x55cfd82ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 -- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 msgr2=0x55cfd82c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd82c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 -- 172.21.15.148:0/397635710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd7e911b0 con 0x55cfd82bd370 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd82c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.565+0000 7fdd5ed9d640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd82d77d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdd580145c0 tx=0x7fdd58014a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad50af5f63134f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/397635710 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580167b0 con 0x55cfd82bd370 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/397635710 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdd58016950 con 0x55cfd82bd370 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd5402ae50 con 0x55cfd82bd370 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/397635710 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd58026050 con 0x55cfd82bd370 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 10 client.5676 fetch_fsmap learned FSMap version 17 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55cfd82af6e0 con 0x55cfd82bd370 2024-04-25T09:56:54.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/397635710 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5402ae50 con 0x55cfd82bd370 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/397635710 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd58029020 con 0x55cfd82bd370 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 10 client.5676 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 10 client.5676 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 20 client.5676 populate_metadata read hostname 'smithi148' 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82a8bb0 0x55cfd82d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 4 client.5676 mds_command: new command op to 5361 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 --> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55cfd81eb420 con 0x55cfd82a8bb0 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82a8bb0 0x55cfd82d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd3d7fa640 10 client.5676 ms_handle_connect on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.566+0000 7fdd5e59c640 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82a8bb0 0x55cfd82d7d10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e102b4816fee74 server_cookie=4621720be308285b in_seq=0 out_seq=0 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd3d7fa640 1 -- 172.21.15.148:0/397635710 <== mds.0 v2:172.21.15.148:6834/3412433349 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17932 (crc 0 0 0) 0x55cfd81eb420 con 0x55cfd82a8bb0 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd3d7fa640 10 client.5676 handle_command_reply: tid=0 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 client.5676 shutdown 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 msgr2=0x55cfd82d77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd82d77d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdd580145c0 tx=0x7fdd58014a80 comp rx=0 tx=0).stop 2024-04-25T09:56:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 shutdown_connections 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:6834/3412433349,v1:172.21.15.148:6836/3412433349] conn(0x55cfd82a8bb0 0x55cfd82d7d10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cfd82bd370 0x55cfd82d77d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd3d7fa640 0 client.5676 ms_handle_reset on v2:172.21.15.148:6834/3412433349 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cfd82ab760 0x55cfd82ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 --2- 172.21.15.148:0/397635710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfd8280ca0 0x55cfd82c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 >> 172.21.15.148:0/397635710 conn(0x55cfd8248870 msgr2=0x55cfd82c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 shutdown_connections 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 1 -- 172.21.15.148:0/397635710 wait complete. 2024-04-25T09:56:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.567+0000 7fdd67a66740 20 client.5676 trim_cache size 0 max 16384 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3642, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3643, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 917812, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 37752374, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 17, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.377366682, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.202354822, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.064317184, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.093484011, 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:56:54.578 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3641, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.541656553, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000148765 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000940, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000094 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2640, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2636, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 768443, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 21936114, 2024-04-25T09:56:54.579 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.276120032, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.152535283, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.049458783, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.066892865, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2636, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.275438128, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000104490 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.580 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 119760, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 23808 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 29409, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 51857, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 9616670, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 53765387, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 2.913169643, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 2.235435765, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.942302269, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.068062663, 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:56:54.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 51856, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.286062855, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000024800 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 48932, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.007550788, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000154 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 116224 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-25T09:56:54.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2675, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 17.196279169, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006428515 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4255, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.421631794, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000334108 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.583 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 25307, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 25307, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 25307, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 410.652899458, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.016226850 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 12, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-25T09:56:54.584 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 17874, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 17874, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 17874, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 2, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 2, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 45894, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 45468, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-25T09:56:54.585 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 107, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-25T09:56:54.586 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 13, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 2000, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 1040, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 4038, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:56:54.587 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 3775, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 3255, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 59, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 13909 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 4122, 2024-04-25T09:56:54.588 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 3758, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 2217, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-25T09:56:54.589 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 26480, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 26480, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 36, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 36, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 4, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-25T09:56:54.590 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 97159283, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2554, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 116.207949824, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.045500371 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 14, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 2016, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 2002, 2024-04-25T09:56:54.591 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 13, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 13, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 0, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 17874, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 19875, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 2001, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 2, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 7953, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 7951, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 102016, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215396 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 28180, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-25T09:56:54.592 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 25307, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 5, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2000, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.294412425, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000647206 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 29, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003797164, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000130936 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-25T09:56:54.593 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6000, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.780315172, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001130052 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7048, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.110144862, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000866933 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.594 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.595 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000038325, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000038325 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.596 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002908344, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002908344 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.211898011, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.211898011 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-25T09:56:54.597 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.488031897, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.186003987 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2217, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 378.152432552, 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.170569432 2024-04-25T09:56:54.598 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.599 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.027763602, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.027763602 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000249526, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000249526 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-25T09:56:54.600 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.601 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8000, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.580907578, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002072613 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:56:54.602 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 18579, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 18579, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 72, 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-25T09:56:54.603 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:56:54.604 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:56:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 250233, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 92, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 5, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-25T09:56:54.606 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 5624, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 80, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 13458757, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 661231, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-25T09:56:54.607 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 8580, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 93649448, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 37, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 8580, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8580, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 33.547173880, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003909927 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-25T09:56:54.608 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8580, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 8671 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 8580, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 2073, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 6507, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 58, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 2610, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 35, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 1758, 2024-04-25T09:56:54.609 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 2034, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 2015, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-25T09:56:54.610 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-25T09:56:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 43, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 5, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 8, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 8, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 99, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 29 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-25T09:56:54.612 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 32, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 3758, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 3758, 2024-04-25T09:56:54.613 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 35686, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 8517876, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 35686, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 35686, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8517876, 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:54.614 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8580, 2024-04-25T09:56:54.615 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 94043070, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8580, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 94043070, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T09:56:54.616 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8580, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 8580, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8580, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8580, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:54.617 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 3758, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 393622, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 3758, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 56, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 393622, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.618 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55eb5b2c26e0": { 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 26480, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 92964979, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 26480, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2554, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 92964979, 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:56:54.619 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:56:54.620 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:56:54.620 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:56:54.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.620 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:56:54.620 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:56:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd4006b4d0 msgr2=0x7fdd4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd4006b4d0 0x7fdd4006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd50006fd0 tx=0x7fdd50008040 comp rx=0 tx=0).stop 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 msgr2=0x7fdd601561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 0x7fdd601561a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdd540098c0 tx=0x7fdd540046c0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 shutdown_connections 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd4006b4d0 0x7fdd4006d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd601566e0 0x7fdd601533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd6015d160 0x7fdd601561a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 --2- 172.21.15.148:0/4290004186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd6015baf0 0x7fdd601502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 >> 172.21.15.148:0/4290004186 conn(0x7fdd600d17c0 msgr2=0x7fdd600ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 shutdown_connections 2024-04-25T09:56:54.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.568+0000 7fdd64d5c640 1 -- 172.21.15.148:0/4290004186 wait complete. 2024-04-25T09:56:54.623 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:56:54.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:56:54.668 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:56:54.713 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:56:54.713 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:56:54.713 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:54.729 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:56:54.729 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:54.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:54.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.918+0000 7f651f16a640 1 Processor -- start 2024-04-25T09:56:54.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.918+0000 7f651f16a640 1 -- start start 2024-04-25T09:56:54.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651f16a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65180d0aa0 0x7f65180b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651f16a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 0x7f65180b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651f16a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65180b3710 con 0x7f651815a790 2024-04-25T09:56:54.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651f16a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65180cf0e0 con 0x7f65180d0aa0 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651f16a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cf3c0 con 0x7f651806fea0 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651cf06640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f651815a790 0x7f65180d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36422/0 (socket says 172.21.15.148:36422) 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651cf06640 1 -- 172.21.15.148:0/170399459 learned_addr learned my addr 172.21.15.148:0/170399459 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f6517fff640 1 --2- 172.21.15.148:0/170399459 >> v2:172.21.15.188:3300/0 conn(0x7f65180d0aa0 0x7f65180b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.919+0000 7f651d707640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 0x7f65180b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598305717 0 0) 0x7f65180b3710 con 0x7f651815a790 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6500003520 con 0x7f651815a790 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754802998 0 0) 0x7f6500003520 con 0x7f651815a790 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 msgr2=0x7f65180b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 0x7f65180b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 >> v2:172.21.15.188:3300/0 conn(0x7f65180d0aa0 msgr2=0x7f65180b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 --2- 172.21.15.148:0/170399459 >> v2:172.21.15.188:3300/0 conn(0x7f65180d0aa0 0x7f65180b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180cf6a0 con 0x7f651815a790 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6508001670 con 0x7f651815a790 2024-04-25T09:56:54.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 >> v1:172.21.15.148:6789/0 conn(0x7f651815a790 legacy=0x7f65180d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65000039b0 0x7f6500003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 0x7f6500004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6500004d40 0x7f65000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cf0e0 con 0x7f65000039b0 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cf3c0 con 0x7f6500004d40 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650000b9c0 con 0x7f651806fea0 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651cf06640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65000039b0 0x7f6500003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f6517fff640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 0x7f6500004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651d707640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6500004d40 0x7f65000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651cf06640 1 -- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 msgr2=0x7f6500004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651cf06640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f651806fea0 0x7f6500004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651cf06640 1 -- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6500004d40 msgr2=0x7f65000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651cf06640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6500004d40 0x7f65000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.920+0000 7f651cf06640 1 -- 172.21.15.148:0/170399459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500003520 con 0x7f65000039b0 2024-04-25T09:56:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.921+0000 7f651cf06640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65000039b0 0x7f6500003dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6508007fd0 tx=0x7f6500005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a880009a35738e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.921+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508007d50 con 0x7f65000039b0 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.921+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6508005ad0 con 0x7f65000039b0 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.921+0000 7f65177fe640 1 -- 172.21.15.148:0/170399459 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508007070 con 0x7f65000039b0 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651f16a640 1 -- 172.21.15.148:0/170399459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65000039b0 msgr2=0x7f6500003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651f16a640 1 --2- 172.21.15.148:0/170399459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65000039b0 0x7f6500003dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6508007fd0 tx=0x7f6500005830 comp rx=0 tx=0).stop 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651d707640 1 -- 172.21.15.148:0/170399459 reap_dead start 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651f16a640 1 -- 172.21.15.148:0/170399459 shutdown_connections 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651f16a640 1 -- 172.21.15.148:0/170399459 >> 172.21.15.148:0/170399459 conn(0x7f65180b2370 msgr2=0x7f65180b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651f16a640 1 -- 172.21.15.148:0/170399459 shutdown_connections 2024-04-25T09:56:54.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.922+0000 7f651f16a640 1 -- 172.21.15.148:0/170399459 wait complete. 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.923+0000 7f651f16a640 1 Processor -- start 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.923+0000 7f651f16a640 1 -- start start 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65000039b0 0x7f651815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6518159110 0x7f651814be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 0x7f651814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651815ab70 con 0x7f6518159590 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65180cf980 con 0x7f6518159110 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651815acf0 con 0x7f65000039b0 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 0x7f651814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 0x7f651814c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43986/0 (socket says 172.21.15.148:43986) 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 -- 172.21.15.148:0/2880908898 learned_addr learned my addr 172.21.15.148:0/2880908898 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651cf06640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65000039b0 0x7f651815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f6517fff640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6518159110 0x7f651814be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 -- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65000039b0 msgr2=0x7f651815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65000039b0 0x7f651815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 -- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6518159110 msgr2=0x7f651814be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6518159110 0x7f651814be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 -- 172.21.15.148:0/2880908898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651815cbb0 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651cf06640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65000039b0 0x7f651815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651d707640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 0x7f651814c3a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f65180733a0 tx=0x7f651000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651000fde0 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651001d070 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510015670 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65181f42f0 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.924+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651815d130 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.925+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65180afcc0 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.926+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6510015810 con 0x7f6518159590 2024-04-25T09:56:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.926+0000 7f65157fa640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f650406b4d0 0x7f650406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:54.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.926+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f651008ad00 con 0x7f6518159590 2024-04-25T09:56:54.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.926+0000 7f651cf06640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f650406b4d0 0x7f650406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:54.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.927+0000 7f651cf06640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f650406b4d0 0x7f650406d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6508033000 tx=0x7f6508009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:54.931+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65180afcc0 con 0x7f6518159590 2024-04-25T09:56:55.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.109+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f651806d330 con 0x7f6518159590 2024-04-25T09:56:55.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.110+0000 7f65157fa640 1 -- 172.21.15.148:0/2880908898 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17104 (secure 0 0 0) 0x7f6510010040 con 0x7f6518159590 2024-04-25T09:56:55.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:55.114 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":43,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:56:33.717690+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f650406b4d0 msgr2=0x7f650406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f650406b4d0 0x7f650406d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6508033000 tx=0x7f6508009030 comp rx=0 tx=0).stop 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 msgr2=0x7f651814c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 0x7f651814c3a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f65180733a0 tx=0x7f651000f580 comp rx=0 tx=0).stop 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 shutdown_connections 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f650406b4d0 0x7f650406d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6518159590 0x7f651814c3a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6518159110 0x7f651814be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 --2- 172.21.15.148:0/2880908898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65000039b0 0x7f651815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 >> 172.21.15.148:0/2880908898 conn(0x7f65180b2370 msgr2=0x7f65180cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:55.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 shutdown_connections 2024-04-25T09:56:55.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.112+0000 7f651f16a640 1 -- 172.21.15.148:0/2880908898 wait complete. 2024-04-25T09:56:55.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:55:32.158059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:55:33.292007+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:55.127 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.224+0000 7fa04ca9e640 1 Processor -- start 2024-04-25T09:56:55.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.225+0000 7fa04ca9e640 1 -- start start 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.225+0000 7fa04ca9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 0x7fa0480c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.225+0000 7fa04ca9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0480c75e0 con 0x7fa0480cfd10 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.225+0000 7fa04ca9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0480c78c0 con 0x7fa0480c70d0 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.225+0000 7fa04ca9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480c8680 con 0x7fa04815bb50 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.225+0000 7fa046d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa0480cfd10 0x7fa0480cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36424/0 (socket says 172.21.15.148:36424) 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa046d9d640 1 -- 172.21.15.148:0/1627651460 learned_addr learned my addr 172.21.15.148:0/1627651460 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931618563 0 0) 0x7fa0480c75e0 con 0x7fa0480cfd10 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa034003520 con 0x7fa0480cfd10 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa04659c640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 0x7fa0480c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 522901390 0 0) 0x7fa034003520 con 0x7fa0480cfd10 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 msgr2=0x7fa0480c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 0x7fa0480c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 >> v1:172.21.15.188:6789/0 conn(0x7fa0480c70d0 legacy=0x7fa0480cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480c7ba0 con 0x7fa0480cfd10 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa0400019f0 con 0x7fa0480cfd10 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 >> v1:172.21.15.148:6789/0 conn(0x7fa0480cfd10 legacy=0x7fa0480cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0340039e0 0x7fa034003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa034004800 0x7fa034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 0x7fa0340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480c8680 con 0x7fa034004800 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03400bb00 con 0x7fa0340039e0 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03400bd00 con 0x7fa04815bb50 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa03ffff640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa034004800 0x7fa034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa04659c640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0340039e0 0x7fa034003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.226+0000 7fa046d9d640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 0x7fa0340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa03ffff640 1 -- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 msgr2=0x7fa0340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa03ffff640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815bb50 0x7fa0340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa03ffff640 1 -- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0340039e0 msgr2=0x7fa034003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa03ffff640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0340039e0 0x7fa034003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa03ffff640 1 -- 172.21.15.148:0/1627651460 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa034003520 con 0x7fa034004800 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa03ffff640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa034004800 0x7fa034004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0480c7510 tx=0x7fa034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258976bdb17ff514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030001370 con 0x7fa034004800 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0300052d0 con 0x7fa034004800 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.227+0000 7fa045d9b640 1 -- 172.21.15.148:0/1627651460 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03002ab70 con 0x7fa034004800 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa04ca9e640 1 -- 172.21.15.148:0/1627651460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa034004800 msgr2=0x7fa034004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/1627651460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa034004800 0x7fa034004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa0480c7510 tx=0x7fa034005830 comp rx=0 tx=0).stop 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa046d9d640 1 -- 172.21.15.148:0/1627651460 reap_dead start 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa04ca9e640 1 -- 172.21.15.148:0/1627651460 shutdown_connections 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa04ca9e640 1 -- 172.21.15.148:0/1627651460 >> 172.21.15.148:0/1627651460 conn(0x7fa04806f500 msgr2=0x7fa04806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:55.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa04ca9e640 1 -- 172.21.15.148:0/1627651460 shutdown_connections 2024-04-25T09:56:55.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.228+0000 7fa04ca9e640 1 -- 172.21.15.148:0/1627651460 wait complete. 2024-04-25T09:56:55.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.229+0000 7fa04ca9e640 1 Processor -- start 2024-04-25T09:56:55.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.229+0000 7fa04ca9e640 1 -- start start 2024-04-25T09:56:55.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04ca9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa034004800 0x7fa0480cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04ca9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480b55c0 0x7fa04815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04ca9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 0x7fa04815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04ca9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04815f6d0 con 0x7fa0480b5a40 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04ca9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480c8160 con 0x7fa0480b55c0 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04ca9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04815f850 con 0x7fa034004800 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 0x7fa04815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 0x7fa04815aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44004/0 (socket says 172.21.15.148:44004) 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 -- 172.21.15.148:0/737243151 learned_addr learned my addr 172.21.15.148:0/737243151 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa04659c640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa034004800 0x7fa0480cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa03ffff640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480b55c0 0x7fa04815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 -- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa034004800 msgr2=0x7fa0480cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa034004800 0x7fa0480cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 -- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480b55c0 msgr2=0x7fa04815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480b55c0 0x7fa04815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.230+0000 7fa046d9d640 1 -- 172.21.15.148:0/737243151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480b4520 con 0x7fa0480b5a40 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.231+0000 7fa046d9d640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 0x7fa04815aad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa048071f60 tx=0x7fa040007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.231+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000abc0 con 0x7fa0480b5a40 2024-04-25T09:56:55.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.231+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040018070 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.231+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0480b4800 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.231+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400133f0 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.231+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0480b4dc0 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.232+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa048073a00 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.232+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa040013590 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.233+0000 7fa03effd640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa02806b500 0x7fa02806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.233+0000 7fa04659c640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa02806b500 0x7fa02806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.233+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa04008a9b0 con 0x7fa0480b5a40 2024-04-25T09:56:55.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.233+0000 7fa04659c640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa02806b500 0x7fa02806d9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b00 tx=0x7fa038008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.237+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa040059670 con 0x7fa0480b5a40 2024-04-25T09:56:55.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.404+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0480cf150 con 0x7fa0480b5a40 2024-04-25T09:56:55.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.404+0000 7fa03effd640 1 -- 172.21.15.148:0/737243151 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa040058f70 con 0x7fa0480b5a40 2024-04-25T09:56:55.408 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa02806b500 msgr2=0x7fa02806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa02806b500 0x7fa02806d9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b00 tx=0x7fa038008040 comp rx=0 tx=0).stop 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 msgr2=0x7fa04815aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 0x7fa04815aad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa048071f60 tx=0x7fa040007b00 comp rx=0 tx=0).stop 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 shutdown_connections 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa02806b500 0x7fa02806d9c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0480b5a40 0x7fa04815aad0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480b55c0 0x7fa04815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 --2- 172.21.15.148:0/737243151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa034004800 0x7fa0480cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.406+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 >> 172.21.15.148:0/737243151 conn(0x7fa04806f500 msgr2=0x7fa048156400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:55.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.407+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 shutdown_connections 2024-04-25T09:56:55.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.407+0000 7fa04ca9e640 1 -- 172.21.15.148:0/737243151 wait complete. 2024-04-25T09:56:55.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:56:55.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.518+0000 7f543335c640 1 Processor -- start 2024-04-25T09:56:55.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.518+0000 7f543335c640 1 -- start start 2024-04-25T09:56:55.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f543335c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f542c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f543335c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0d2020 con 0x7f542c0d1b80 2024-04-25T09:56:55.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f543335c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0d2300 con 0x7f542c0b4e60 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f543335c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c158610 con 0x7f542c158230 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f54308f7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f542c0d1b80 0x7f542c06c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36436/0 (socket says 172.21.15.148:36436) 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f54308f7640 1 -- 172.21.15.148:0/2017792810 learned_addr learned my addr 172.21.15.148:0/2017792810 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f54310f8640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f542c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415057909 0 0) 0x7f542c0d2020 con 0x7f542c0d1b80 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5418003520 con 0x7f542c0d1b80 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897810234 0 0) 0x7f5418003520 con 0x7f542c0d1b80 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 msgr2=0x7f542c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f542c0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 >> v1:172.21.15.188:6789/0 conn(0x7f542c0b4e60 legacy=0x7f542c06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c06bb30 con 0x7f542c0d1b80 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f541c0017f0 con 0x7f542c0d1b80 2024-04-25T09:56:55.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 >> v1:172.21.15.148:6789/0 conn(0x7f542c0d1b80 legacy=0x7f542c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180039e0 0x7f5418003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5418004390 0x7f5418004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f54180050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c158610 con 0x7f54180039e0 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541800b9c0 con 0x7f5418004390 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541800bbc0 con 0x7f542c158230 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.519+0000 7f54310f8640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180039e0 0x7f5418003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54308f7640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5418004390 0x7f5418004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54318f9640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f54180050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54310f8640 1 -- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 msgr2=0x7f54180050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54310f8640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f54180050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54310f8640 1 -- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5418004390 msgr2=0x7f5418004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54310f8640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5418004390 0x7f5418004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54310f8640 1 -- 172.21.15.148:0/2017792810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418003520 con 0x7f54180039e0 2024-04-25T09:56:55.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54318f9640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c158230 0x7f54180050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.520+0000 7f54310f8640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180039e0 0x7f5418003dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5428009ac0 tx=0x7f5418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131410fd487ef700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428036070 con 0x7f54180039e0 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428031400 con 0x7f54180039e0 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f5423fff640 1 -- 172.21.15.148:0/2017792810 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428009690 con 0x7f54180039e0 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f543335c640 1 -- 172.21.15.148:0/2017792810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180039e0 msgr2=0x7f5418003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f543335c640 1 --2- 172.21.15.148:0/2017792810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54180039e0 0x7f5418003dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5428009ac0 tx=0x7f5418005830 comp rx=0 tx=0).stop 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f54318f9640 1 -- 172.21.15.148:0/2017792810 reap_dead start 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f543335c640 1 -- 172.21.15.148:0/2017792810 shutdown_connections 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f543335c640 1 -- 172.21.15.148:0/2017792810 >> 172.21.15.148:0/2017792810 conn(0x7f542c061600 msgr2=0x7f542c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f543335c640 1 -- 172.21.15.148:0/2017792810 shutdown_connections 2024-04-25T09:56:55.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.521+0000 7f543335c640 1 -- 172.21.15.148:0/2017792810 wait complete. 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.522+0000 7f543335c640 1 Processor -- start 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.522+0000 7f543335c640 1 -- start start 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f543335c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54180039e0 0x7f542c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f543335c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 0x7f542c1e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f543335c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c1e4600 0x7f542c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f543335c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c06ca40 con 0x7f542c145950 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f543335c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c06bea0 con 0x7f542c1e4600 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f543335c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15e320 con 0x7f54180039e0 2024-04-25T09:56:55.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 0x7f542c1e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 0x7f542c1e40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44010/0 (socket says 172.21.15.148:44010) 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 -- 172.21.15.148:0/3842362640 learned_addr learned my addr 172.21.15.148:0/3842362640 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54318f9640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c1e4600 0x7f542c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54310f8640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54180039e0 0x7f542c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 -- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54180039e0 msgr2=0x7f542c0cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54180039e0 0x7f542c0cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 -- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c1e4600 msgr2=0x7f542c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c1e4600 0x7f542c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.523+0000 7f54308f7640 1 -- 172.21.15.148:0/3842362640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c1461f0 con 0x7f542c145950 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.524+0000 7f54308f7640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 0x7f542c1e40c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f541c00f910 tx=0x7f541c00fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.524+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0072e0 con 0x7f542c145950 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.524+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c018070 con 0x7f542c145950 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.524+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00d7f0 con 0x7f542c145950 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.524+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c1464d0 con 0x7f542c145950 2024-04-25T09:56:55.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.524+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c146a90 con 0x7f542c145950 2024-04-25T09:56:55.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.525+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c06de50 con 0x7f542c145950 2024-04-25T09:56:55.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.525+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f541c00d990 con 0x7f542c145950 2024-04-25T09:56:55.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.526+0000 7f5421ffb640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541006b4d0 0x7f541006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.526+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f541c08a370 con 0x7f542c145950 2024-04-25T09:56:55.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.526+0000 7f54310f8640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541006b4d0 0x7f541006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.526+0000 7f54310f8640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541006b4d0 0x7f541006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5428009ac0 tx=0x7f5428008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.530+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f541c058aa0 con 0x7f542c145950 2024-04-25T09:56:55.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.701+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f542c1e4a10 con 0x7f542c145950 2024-04-25T09:56:55.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.701+0000 7f5421ffb640 1 -- 172.21.15.148:0/3842362640 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f541c05cb40 con 0x7f542c145950 2024-04-25T09:56:55.704 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541006b4d0 msgr2=0x7f541006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541006b4d0 0x7f541006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5428009ac0 tx=0x7f5428008030 comp rx=0 tx=0).stop 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 msgr2=0x7f542c1e40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 0x7f542c1e40c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f541c00f910 tx=0x7f541c00fdd0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 shutdown_connections 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541006b4d0 0x7f541006d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c1e4600 0x7f542c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c145950 0x7f542c1e40c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 --2- 172.21.15.148:0/3842362640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54180039e0 0x7f542c0cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 >> 172.21.15.148:0/3842362640 conn(0x7f542c061600 msgr2=0x7f542c0d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:55.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 shutdown_connections 2024-04-25T09:56:55.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.703+0000 7f543335c640 1 -- 172.21.15.148:0/3842362640 wait complete. 2024-04-25T09:56:55.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:56:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.830+0000 7f681f76a640 1 Processor -- start 2024-04-25T09:56:55.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.830+0000 7f681f76a640 1 -- start start 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681f76a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68181512a0 0x7f681815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681f76a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681815e6e0 con 0x7f68180cd8a0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681f76a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681815e9c0 con 0x7f68180cd4c0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681f76a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681815a9c0 con 0x7f68181512a0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681cd05640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f68180cd8a0 0x7f6818150b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36446/0 (socket says 172.21.15.148:36446) 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681cd05640 1 -- 172.21.15.148:0/3605615561 learned_addr learned my addr 172.21.15.148:0/3605615561 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356026920 0 0) 0x7f681815e6e0 con 0x7f68180cd8a0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f681dd07640 1 --2- 172.21.15.148:0/3605615561 >> v2:172.21.15.188:3300/0 conn(0x7f68181512a0 0x7f681815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6804003440 con 0x7f68180cd8a0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185449366 0 0) 0x7f6804003440 con 0x7f68180cd8a0 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 >> v2:172.21.15.188:3300/0 conn(0x7f68181512a0 msgr2=0x7f681815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 --2- 172.21.15.148:0/3605615561 >> v2:172.21.15.188:3300/0 conn(0x7f68181512a0 0x7f681815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 >> v1:172.21.15.188:6789/0 conn(0x7f68180cd4c0 legacy=0x7f681815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681815aca0 con 0x7f68180cd8a0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6808001670 con 0x7f68180cd8a0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 >> v1:172.21.15.148:6789/0 conn(0x7f68180cd8a0 legacy=0x7f6818150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6804003ad0 0x7f6804003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804006690 0x7f6804008ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68040092e0 0x7f680400b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681815a9c0 con 0x7f6804003ad0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680400c080 con 0x7f68040092e0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.831+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680400c280 con 0x7f6804006690 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6804003ad0 0x7f6804003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681cd05640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804006690 0x7f6804008ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681dd07640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68040092e0 0x7f680400b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 -- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804006690 msgr2=0x7f6804008ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804006690 0x7f6804008ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 -- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68040092e0 msgr2=0x7f680400b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68040092e0 0x7f680400b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 -- 172.21.15.148:0/3605615561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6804003440 con 0x7f6804003ad0 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681dd07640 1 -- 172.21.15.148:0/3605615561 reap_dead start 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.832+0000 7f681d506640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6804003ad0 0x7f6804003ef0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6814001750 tx=0x7f6804003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5493e93423186262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814001370 con 0x7f6804003ad0 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68140052d0 con 0x7f6804003ad0 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f680ffff640 1 -- 172.21.15.148:0/3605615561 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681402abc0 con 0x7f6804003ad0 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 -- 172.21.15.148:0/3605615561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6804003ad0 msgr2=0x7f6804003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6804003ad0 0x7f6804003ef0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6814001750 tx=0x7f6804003730 comp rx=0 tx=0).stop 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 -- 172.21.15.148:0/3605615561 shutdown_connections 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 --2- 172.21.15.148:0/3605615561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6804003ad0 0x7f6804003ef0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 -- 172.21.15.148:0/3605615561 >> 172.21.15.148:0/3605615561 conn(0x7f681806abe0 msgr2=0x7f681806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:55.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 -- 172.21.15.148:0/3605615561 shutdown_connections 2024-04-25T09:56:55.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.833+0000 7f681f76a640 1 -- 172.21.15.148:0/3605615561 wait complete. 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.834+0000 7f681f76a640 1 Processor -- start 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.834+0000 7f681f76a640 1 -- start start 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681f76a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804003ad0 0x7f6818152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681f76a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 0x7f68181590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681f76a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68181595f0 0x7f68180b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681f76a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681806ee20 con 0x7f681814e290 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681f76a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681806eca0 con 0x7f6804003ad0 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681f76a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681815b1c0 con 0x7f68181595f0 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681cd05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 0x7f68181590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681cd05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 0x7f68181590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44024/0 (socket says 172.21.15.148:44024) 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681cd05640 1 -- 172.21.15.148:0/1365773628 learned_addr learned my addr 172.21.15.148:0/1365773628 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681dd07640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68181595f0 0x7f68180b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681d506640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804003ad0 0x7f6818152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681cd05640 1 -- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68181595f0 msgr2=0x7f68180b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.835+0000 7f681cd05640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68181595f0 0x7f68180b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681cd05640 1 -- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804003ad0 msgr2=0x7f6818152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681cd05640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804003ad0 0x7f6818152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681cd05640 1 -- 172.21.15.148:0/1365773628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68181f3f60 con 0x7f681814e290 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681dd07640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68181595f0 0x7f68180b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:55.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681d506640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804003ad0 0x7f6818152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681cd05640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 0x7f68181590b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f680800e7f0 tx=0x7f680800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800eea0 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6808018070 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68181f4240 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800ca90 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.836+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68181f4800 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.837+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6818073a00 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.837+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f680800cc30 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.838+0000 7f680dffb640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67fc06b500 0x7f67fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.838+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f680808a100 con 0x7f681814e290 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.838+0000 7f681d506640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67fc06b500 0x7f67fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:55.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.838+0000 7f681d506640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67fc06b500 0x7f67fc06d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6814032000 tx=0x7f681402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:55.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:55.842+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6808058e50 con 0x7f681814e290 2024-04-25T09:56:56.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.012+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f681814fa70 con 0x7f681814e290 2024-04-25T09:56:56.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.012+0000 7f680dffb640 1 -- 172.21.15.148:0/1365773628 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6808058750 con 0x7f681814e290 2024-04-25T09:56:56.016 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67fc06b500 msgr2=0x7f67fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67fc06b500 0x7f67fc06d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6814032000 tx=0x7f681402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 msgr2=0x7f68181590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 0x7f68181590b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f680800e7f0 tx=0x7f680800ecb0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 shutdown_connections 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67fc06b500 0x7f67fc06d9c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68181595f0 0x7f68180b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f681814e290 0x7f68181590b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 --2- 172.21.15.148:0/1365773628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804003ad0 0x7f6818152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 >> 172.21.15.148:0/1365773628 conn(0x7f681806abe0 msgr2=0x7f681815b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 shutdown_connections 2024-04-25T09:56:56.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.014+0000 7f681f76a640 1 -- 172.21.15.148:0/1365773628 wait complete. 2024-04-25T09:56:56.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:56:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.123+0000 7f401468f640 1 Processor -- start 2024-04-25T09:56:56.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.123+0000 7f401468f640 1 -- start start 2024-04-25T09:56:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401468f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f400c158230 0x7f400c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401468f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c0d1b80 0x7f400c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401468f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400c0d2020 con 0x7f400c06c870 2024-04-25T09:56:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401468f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400c158610 con 0x7f400c158230 2024-04-25T09:56:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401468f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c06bea0 con 0x7f400c0d1b80 2024-04-25T09:56:56.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f400c158230 0x7f400c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f400c158230 0x7f400c0d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44026/0 (socket says 172.21.15.148:44026) 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 -- 172.21.15.148:0/2652745176 learned_addr learned my addr 172.21.15.148:0/2652745176 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f4011c2a640 1 --2- 172.21.15.148:0/2652745176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c0d1b80 0x7f400c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 -- 172.21.15.148:0/2652745176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c0d1b80 msgr2=0x7f400c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 --2- 172.21.15.148:0/2652745176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c0d1b80 0x7f400c06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 -- 172.21.15.148:0/2652745176 >> v1:172.21.15.188:6789/0 conn(0x7f400c06c870 legacy=0x7f400c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.124+0000 7f401242b640 1 -- 172.21.15.148:0/2652745176 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c0d2300 con 0x7f400c158230 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.125+0000 7f401242b640 1 --2- 172.21.15.148:0/2652745176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c158230 0x7f400c0d1640 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4008009870 tx=0x7f400802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722cd142700b4215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.125+0000 7f4011429640 1 -- 172.21.15.148:0/2652745176 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008037070 con 0x7f400c158230 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.125+0000 7f4011429640 1 -- 172.21.15.148:0/2652745176 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4008032440 con 0x7f400c158230 2024-04-25T09:56:56.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.125+0000 7f4011429640 1 -- 172.21.15.148:0/2652745176 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008032810 con 0x7f400c158230 2024-04-25T09:56:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 -- 172.21.15.148:0/2652745176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c158230 msgr2=0x7f400c0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 --2- 172.21.15.148:0/2652745176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c158230 0x7f400c0d1640 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4008009870 tx=0x7f400802a270 comp rx=0 tx=0).stop 2024-04-25T09:56:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 -- 172.21.15.148:0/2652745176 shutdown_connections 2024-04-25T09:56:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 --2- 172.21.15.148:0/2652745176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c0d1b80 0x7f400c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 --2- 172.21.15.148:0/2652745176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c158230 0x7f400c0d1640 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 -- 172.21.15.148:0/2652745176 >> 172.21.15.148:0/2652745176 conn(0x7f400c061600 msgr2=0x7f400c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 -- 172.21.15.148:0/2652745176 shutdown_connections 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.126+0000 7f401468f640 1 -- 172.21.15.148:0/2652745176 wait complete. 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.127+0000 7f401468f640 1 Processor -- start 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.127+0000 7f401468f640 1 -- start start 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f400c06c870 0x7f400c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 0x7f400c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c158230 0x7f400c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c15e320 con 0x7f400c0d1b80 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c0d25e0 con 0x7f400c06c870 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400c15e4a0 con 0x7f400c158230 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 0x7f400c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 0x7f400c159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44036/0 (socket says 172.21.15.148:44036) 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 -- 172.21.15.148:0/217199892 learned_addr learned my addr 172.21.15.148:0/217199892 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:56.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401242b640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f400c06c870 0x7f400c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4012c2c640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c158230 0x7f400c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 -- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c158230 msgr2=0x7f400c152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c158230 0x7f400c152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 -- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f400c06c870 msgr2=0x7f400c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f400c06c870 0x7f400c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 -- 172.21.15.148:0/217199892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c158790 con 0x7f400c0d1b80 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401242b640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f400c06c870 0x7f400c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4011c2a640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 0x7f400c159810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00d800 tx=0x7f3ffc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00b6c0 con 0x7f400c0d1b80 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc019070 con 0x7f400c0d1b80 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c158a50 con 0x7f400c0d1b80 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00baa0 con 0x7f400c0d1b80 2024-04-25T09:56:56.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.128+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c1592f0 con 0x7f400c0d1b80 2024-04-25T09:56:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.129+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c06de50 con 0x7f400c0d1b80 2024-04-25T09:56:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.130+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ffc00bc40 con 0x7f400c0d1b80 2024-04-25T09:56:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.130+0000 7f4002ffd640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ff006b4d0 0x7f3ff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.130+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f3ffc08a830 con 0x7f400c0d1b80 2024-04-25T09:56:56.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.130+0000 7f401242b640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ff006b4d0 0x7f3ff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.131+0000 7f401242b640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ff006b4d0 0x7f3ff006d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4008034000 tx=0x7f400802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.134+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ffc0594f0 con 0x7f400c0d1b80 2024-04-25T09:56:56.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.314+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f400c073a00 con 0x7f400c0d1b80 2024-04-25T09:56:56.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.315+0000 7f4002ffd640 1 -- 172.21.15.148:0/217199892 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3ffc058df0 con 0x7f400c0d1b80 2024-04-25T09:56:56.318 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:56:56.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ff006b4d0 msgr2=0x7f3ff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ff006b4d0 0x7f3ff006d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4008034000 tx=0x7f400802e000 comp rx=0 tx=0).stop 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 msgr2=0x7f400c159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 0x7f400c159810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00d800 tx=0x7f3ffc00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 shutdown_connections 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ff006b4d0 0x7f3ff006d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400c158230 0x7f400c152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400c0d1b80 0x7f400c159810 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 --2- 172.21.15.148:0/217199892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f400c06c870 0x7f400c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 >> 172.21.15.148:0/217199892 conn(0x7f400c061600 msgr2=0x7f400c156940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 shutdown_connections 2024-04-25T09:56:56.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.317+0000 7f401468f640 1 -- 172.21.15.148:0/217199892 wait complete. 2024-04-25T09:56:56.331 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:56:56.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:56.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:56:56.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:56:56.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:56:56.363 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:56:56.418 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:56:56.418 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:56.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:56:56.436 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:56:56.436 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:56.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:56:56.473 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:56:56.473 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:56:56.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:56:56.493 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:56.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.606+0000 7f50417b0640 1 Processor -- start 2024-04-25T09:56:56.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.606+0000 7f50417b0640 1 -- start start 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f50417b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f503c15dbe0 0x7f503c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f50417b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0b6220 con 0x7f503c0b92a0 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f50417b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0b97a0 con 0x7f503c0b2cb0 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f50417b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503c15dfc0 con 0x7f503c15dbe0 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f503b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f503c0b92a0 0x7f503c0b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36462/0 (socket says 172.21.15.148:36462) 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f503b7fe640 1 -- 172.21.15.148:0/2335985729 learned_addr learned my addr 172.21.15.148:0/2335985729 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f503affd640 1 --2- 172.21.15.148:0/2335985729 >> v2:172.21.15.188:3300/0 conn(0x7f503c15dbe0 0x7f503c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412012638 0 0) 0x7f503c0b6220 con 0x7f503c0b92a0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.607+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5024003520 con 0x7f503c0b92a0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612331037 0 0) 0x7f503c0b97a0 con 0x7f503c0b2cb0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c0b6220 con 0x7f503c0b2cb0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329980921 0 0) 0x7f5024003520 con 0x7f503c0b92a0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 >> v2:172.21.15.188:3300/0 conn(0x7f503c15dbe0 msgr2=0x7f503c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 --2- 172.21.15.148:0/2335985729 >> v2:172.21.15.188:3300/0 conn(0x7f503c15dbe0 0x7f503c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 >> v1:172.21.15.188:6789/0 conn(0x7f503c0b2cb0 legacy=0x7f503c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0d1c50 con 0x7f503c0b92a0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5034001670 con 0x7f503c0b92a0 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 >> v1:172.21.15.148:6789/0 conn(0x7f503c0b92a0 legacy=0x7f503c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5024003b40 0x7f5024003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5024004800 0x7f5024004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240051c0 0x7f502400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c0c0 con 0x7f50240051c0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c270 con 0x7f5024004800 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502400c470 con 0x7f5024003b40 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503b7fe640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240051c0 0x7f502400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503a7fc640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5024004800 0x7f5024004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503affd640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5024003b40 0x7f5024003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503b7fe640 1 -- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5024003b40 msgr2=0x7f5024003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503b7fe640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5024003b40 0x7f5024003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503b7fe640 1 -- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5024004800 msgr2=0x7f5024004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503b7fe640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5024004800 0x7f5024004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.608+0000 7f503b7fe640 1 -- 172.21.15.148:0/2335985729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024003520 con 0x7f50240051c0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f503b7fe640 1 -- 172.21.15.148:0/2335985729 reap_dead start 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f503b7fe640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240051c0 0x7f502400b9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5034007fd0 tx=0x7f5024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f3017a00029df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034007ca0 con 0x7f50240051c0 2024-04-25T09:56:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034005af0 con 0x7f50240051c0 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f5039ffb640 1 -- 172.21.15.148:0/2335985729 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034007070 con 0x7f50240051c0 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f50417b0640 1 -- 172.21.15.148:0/2335985729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240051c0 msgr2=0x7f502400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f50417b0640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240051c0 0x7f502400b9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5034007fd0 tx=0x7f5024005830 comp rx=0 tx=0).stop 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f50417b0640 1 -- 172.21.15.148:0/2335985729 shutdown_connections 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f50417b0640 1 --2- 172.21.15.148:0/2335985729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50240051c0 0x7f502400b9c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.609+0000 7f50417b0640 1 -- 172.21.15.148:0/2335985729 >> 172.21.15.148:0/2335985729 conn(0x7f503c061a20 msgr2=0x7f503c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.610+0000 7f50417b0640 1 -- 172.21.15.148:0/2335985729 shutdown_connections 2024-04-25T09:56:56.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.610+0000 7f50417b0640 1 -- 172.21.15.148:0/2335985729 wait complete. 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.611+0000 7f50417b0640 1 Processor -- start 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.611+0000 7f50417b0640 1 -- start start 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.611+0000 7f50417b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50240051c0 0x7f503c1f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f503c1ed760 0x7f503c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 0x7f503c1f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0d2210 con 0x7f503c15cab0 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0d1a00 con 0x7f503c1ed760 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0b6ab0 con 0x7f50240051c0 2024-04-25T09:56:56.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 0x7f503c1f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 0x7f503c1f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44056/0 (socket says 172.21.15.148:44056) 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 -- 172.21.15.148:0/313338953 learned_addr learned my addr 172.21.15.148:0/313338953 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503affd640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50240051c0 0x7f503c1f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503a7fc640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f503c1ed760 0x7f503c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 -- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50240051c0 msgr2=0x7f503c1f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50240051c0 0x7f503c1f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 -- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f503c1ed760 msgr2=0x7f503c15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f503c1ed760 0x7f503c15c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 -- 172.21.15.148:0/313338953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c1f4530 con 0x7f503c15cab0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503a7fc640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f503c1ed760 0x7f503c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f503b7fe640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 0x7f503c1f6b60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5034009000 tx=0x7f503c0b9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034042070 con 0x7f503c15cab0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50340047e0 con 0x7f503c15cab0 2024-04-25T09:56:56.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c0cad80 con 0x7f503c15cab0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034004bb0 con 0x7f503c15cab0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.612+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c0cb340 con 0x7f503c15cab0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.613+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c073990 con 0x7f503c15cab0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.614+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5034045070 con 0x7f503c15cab0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.614+0000 7f50237fe640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f502806b500 0x7f502806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.614+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f50340a9400 con 0x7f503c15cab0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.614+0000 7f503affd640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f502806b500 0x7f502806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.614+0000 7f503affd640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f502806b500 0x7f502806d9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f502c009fd0 tx=0x7f502c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.618+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f503c1f4530 con 0x7f503c15cab0 2024-04-25T09:56:56.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.822+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f503c0b7c40 con 0x7f503c15cab0 2024-04-25T09:56:56.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.823+0000 7f50237fe640 1 -- 172.21.15.148:0/313338953 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f50340780c0 con 0x7f503c15cab0 2024-04-25T09:56:56.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:56.827 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:55:34.319815+0000","modified":"2024-04-25T09:55:40.935056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/3412433349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3412433349},{"type":"v1","addr":"172.21.15.148:6836","nonce":3412433349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:56:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2024-04-25T09:56:56.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f502806b500 msgr2=0x7f502806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f502806b500 0x7f502806d9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f502c009fd0 tx=0x7f502c00b040 comp rx=0 tx=0).stop 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 msgr2=0x7f503c1f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 0x7f503c1f6b60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5034009000 tx=0x7f503c0b9970 comp rx=0 tx=0).stop 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 shutdown_connections 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f502806b500 0x7f502806d9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f503c15cab0 0x7f503c1f6b60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f503c1ed760 0x7f503c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 --2- 172.21.15.148:0/313338953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50240051c0 0x7f503c1f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 >> 172.21.15.148:0/313338953 conn(0x7f503c061a20 msgr2=0x7f503c15b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 shutdown_connections 2024-04-25T09:56:56.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.825+0000 7f50417b0640 1 -- 172.21.15.148:0/313338953 wait complete. 2024-04-25T09:56:56.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:56.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.934+0000 7ff49619e640 1 Processor -- start 2024-04-25T09:56:56.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.934+0000 7ff49619e640 1 -- start start 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.934+0000 7ff49619e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff49015dc90 0x7ff4901625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff49619e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4900d1810 0x7ff4900d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff49619e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900cce50 con 0x7ff4900cc8e0 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff49619e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4900d21c0 con 0x7ff4900d1810 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff49619e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff490161af0 con 0x7ff49015dc90 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4900d1810 0x7ff4900d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff4900cc8e0 0x7ff49015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36464/0 (socket says 172.21.15.148:36464) 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48effd640 1 -- 172.21.15.148:0/2862231367 learned_addr learned my addr 172.21.15.148:0/2862231367 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48f7fe640 1 --2- 172.21.15.148:0/2862231367 >> v2:172.21.15.188:3300/0 conn(0x7ff49015dc90 0x7ff4901625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48ffff640 1 -- 172.21.15.148:0/2862231367 >> v2:172.21.15.188:3300/0 conn(0x7ff49015dc90 msgr2=0x7ff4901625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48ffff640 1 --2- 172.21.15.148:0/2862231367 >> v2:172.21.15.188:3300/0 conn(0x7ff49015dc90 0x7ff4901625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48ffff640 1 -- 172.21.15.148:0/2862231367 >> v1:172.21.15.148:6789/0 conn(0x7ff4900cc8e0 legacy=0x7ff49015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.935+0000 7ff48ffff640 1 -- 172.21.15.148:0/2862231367 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490161dd0 con 0x7ff4900d1810 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff48ffff640 1 --2- 172.21.15.148:0/2862231367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900d1810 0x7ff4900d1c80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff49006b820 tx=0x7ff48802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82ab738f81ddb2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff48e7fc640 1 -- 172.21.15.148:0/2862231367 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488007740 con 0x7ff4900d1810 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff48e7fc640 1 -- 172.21.15.148:0/2862231367 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4880078e0 con 0x7ff4900d1810 2024-04-25T09:56:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff48e7fc640 1 -- 172.21.15.148:0/2862231367 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488007cb0 con 0x7ff4900d1810 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 -- 172.21.15.148:0/2862231367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900d1810 msgr2=0x7ff4900d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 --2- 172.21.15.148:0/2862231367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900d1810 0x7ff4900d1c80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff49006b820 tx=0x7ff48802a5f0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 -- 172.21.15.148:0/2862231367 shutdown_connections 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 --2- 172.21.15.148:0/2862231367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900d1810 0x7ff4900d1c80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 --2- 172.21.15.148:0/2862231367 >> v2:172.21.15.188:3300/0 conn(0x7ff49015dc90 0x7ff4901625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 -- 172.21.15.148:0/2862231367 >> 172.21.15.148:0/2862231367 conn(0x7ff49006f590 msgr2=0x7ff49006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 -- 172.21.15.148:0/2862231367 shutdown_connections 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.936+0000 7ff49619e640 1 -- 172.21.15.148:0/2862231367 wait complete. 2024-04-25T09:56:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.937+0000 7ff49619e640 1 Processor -- start 2024-04-25T09:56:56.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 -- start start 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 0x7ff4901e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900d1810 0x7ff4900ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49015dc90 0x7ff49015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49015e070 con 0x7ff4900cc8e0 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4900cd130 con 0x7ff4900d1810 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff49619e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49015e1f0 con 0x7ff49015dc90 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 0x7ff4901e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.938+0000 7ff48f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 0x7ff4901e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44078/0 (socket says 172.21.15.148:44078) 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 -- 172.21.15.148:0/507545105 learned_addr learned my addr 172.21.15.148:0/507545105 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 -- 172.21.15.148:0/507545105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49015dc90 msgr2=0x7ff49015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49015dc90 0x7ff49015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 -- 172.21.15.148:0/507545105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900d1810 msgr2=0x7ff4900ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900d1810 0x7ff4900ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 -- 172.21.15.148:0/507545105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900c9d80 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff48f7fe640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 0x7ff4901e7250 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff48000d620 tx=0x7ff48000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000b620 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480019070 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4901e78e0 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48000ba00 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.939+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4901e7ea0 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.940+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490073990 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.940+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff48000bba0 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.941+0000 7ff477fff640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff47c06b4d0 0x7ff47c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.941+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ff48008a770 con 0x7ff4900cc8e0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.941+0000 7ff48effd640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff47c06b4d0 0x7ff47c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:56.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.941+0000 7ff48effd640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff47c06b4d0 0x7ff47c06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff484000f60 tx=0x7ff484007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:56.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:56.945+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4800594c0 con 0x7ff4900cc8e0 2024-04-25T09:56:57.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.116+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4900cf6b0 con 0x7ff4900cc8e0 2024-04-25T09:56:57.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.117+0000 7ff477fff640 1 -- 172.21.15.148:0/507545105 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17104 (secure 0 0 0) 0x7ff480058dc0 con 0x7ff4900cc8e0 2024-04-25T09:56:57.120 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:57.121 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":43,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:56:33.717690+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff47c06b4d0 msgr2=0x7ff47c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff47c06b4d0 0x7ff47c06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff484000f60 tx=0x7ff484007a80 comp rx=0 tx=0).stop 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 msgr2=0x7ff4901e7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 0x7ff4901e7250 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff48000d620 tx=0x7ff48000dae0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 shutdown_connections 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff47c06b4d0 0x7ff47c06d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49015dc90 0x7ff49015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4900d1810 0x7ff4900ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 --2- 172.21.15.148:0/507545105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4900cc8e0 0x7ff4901e7250 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 >> 172.21.15.148:0/507545105 conn(0x7ff49006f590 msgr2=0x7ff4900b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:57.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 shutdown_connections 2024-04-25T09:56:57.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.119+0000 7ff49619e640 1 -- 172.21.15.148:0/507545105 wait complete. 2024-04-25T09:56:57.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:55:32.158059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:55:33.292007+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:56:57.134 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:56:57.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.230+0000 7fdefe658640 1 Processor -- start 2024-04-25T09:56:57.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdefe658640 1 -- start start 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdefe658640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdef8161890 0x7fdef80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdefe658640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdef80b54a0 0x7fdef80b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdefe658640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef80cf1d0 con 0x7fdef8161240 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdefe658640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80b7120 con 0x7fdef8161890 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdefe658640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80b5e30 con 0x7fdef80b54a0 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.231+0000 7fdef77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdef8161890 0x7fdef80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdef8161890 0x7fdef80b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44084/0 (socket says 172.21.15.148:44084) 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 -- 172.21.15.148:0/2905489745 learned_addr learned my addr 172.21.15.148:0/2905489745 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdefcbf5640 1 --2- 172.21.15.148:0/2905489745 >> v2:172.21.15.188:3300/0 conn(0x7fdef80b54a0 0x7fdef80b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 -- 172.21.15.148:0/2905489745 >> v2:172.21.15.188:3300/0 conn(0x7fdef80b54a0 msgr2=0x7fdef80b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:57.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 --2- 172.21.15.148:0/2905489745 >> v2:172.21.15.188:3300/0 conn(0x7fdef80b54a0 0x7fdef80b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 -- 172.21.15.148:0/2905489745 >> v1:172.21.15.188:6789/0 conn(0x7fdef8161240 legacy=0x7fdef80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 -- 172.21.15.148:0/2905489745 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8161620 con 0x7fdef8161890 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdefcbf5640 1 --2- 172.21.15.148:0/2905489745 >> v2:172.21.15.188:3300/0 conn(0x7fdef80b54a0 0x7fdef80b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.232+0000 7fdef77fe640 1 --2- 172.21.15.148:0/2905489745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80b6be0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdef808ce10 tx=0x7fdeec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290a255f0e0f17db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdef6ffd640 1 -- 172.21.15.148:0/2905489745 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec037070 con 0x7fdef8161890 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdef6ffd640 1 -- 172.21.15.148:0/2905489745 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec032440 con 0x7fdef8161890 2024-04-25T09:56:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdef6ffd640 1 -- 172.21.15.148:0/2905489745 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec032810 con 0x7fdef8161890 2024-04-25T09:56:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 -- 172.21.15.148:0/2905489745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 msgr2=0x7fdef80b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 --2- 172.21.15.148:0/2905489745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80b6be0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdef808ce10 tx=0x7fdeec02a270 comp rx=0 tx=0).stop 2024-04-25T09:56:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 -- 172.21.15.148:0/2905489745 shutdown_connections 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 --2- 172.21.15.148:0/2905489745 >> v2:172.21.15.188:3300/0 conn(0x7fdef80b54a0 0x7fdef80b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 --2- 172.21.15.148:0/2905489745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80b6be0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 -- 172.21.15.148:0/2905489745 >> 172.21.15.148:0/2905489745 conn(0x7fdef80d04a0 msgr2=0x7fdef80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 -- 172.21.15.148:0/2905489745 shutdown_connections 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.233+0000 7fdefe658640 1 -- 172.21.15.148:0/2905489745 wait complete. 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.234+0000 7fdefe658640 1 Processor -- start 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 -- start start 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef80b54a0 0x7fdef8154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8161240 0x7fdef81f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81623c0 con 0x7fdef8161890 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81620e0 con 0x7fdef80b54a0 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefe658640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81626a0 con 0x7fdef8161240 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefcbf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefcbf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44090/0 (socket says 172.21.15.148:44090) 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.235+0000 7fdefcbf5640 1 -- 172.21.15.148:0/1678321714 learned_addr learned my addr 172.21.15.148:0/1678321714 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefcbf5640 1 -- 172.21.15.148:0/1678321714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8161240 msgr2=0x7fdef81f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefcbf5640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8161240 0x7fdef81f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefcbf5640 1 -- 172.21.15.148:0/1678321714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef80b54a0 msgr2=0x7fdef8154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefcbf5640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef80b54a0 0x7fdef8154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefcbf5640 1 -- 172.21.15.148:0/1678321714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef81f8060 con 0x7fdef8161890 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefcbf5640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80cc740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdef0007ef0 tx=0x7fdef000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef001a070 con 0x7fdef8161890 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0003b90 con 0x7fdef8161890 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef81f72a0 con 0x7fdef8161890 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0004030 con 0x7fdef8161890 2024-04-25T09:56:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.236+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef81f7860 con 0x7fdef8161890 2024-04-25T09:56:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.237+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef80b30f0 con 0x7fdef8161890 2024-04-25T09:56:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.237+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdef001d070 con 0x7fdef8161890 2024-04-25T09:56:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.238+0000 7fdef4ff9640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdee406b4d0 0x7fdee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.238+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdef008a670 con 0x7fdef8161890 2024-04-25T09:56:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.238+0000 7fdef7fff640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdee406b4d0 0x7fdee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.238+0000 7fdef7fff640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdee406b4d0 0x7fdee406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdee8000f60 tx=0x7fdee80079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.242+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdef80b30f0 con 0x7fdef8161890 2024-04-25T09:56:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.411+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdef8158d30 con 0x7fdef8161890 2024-04-25T09:56:57.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.411+0000 7fdef4ff9640 1 -- 172.21.15.148:0/1678321714 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdef8158d30 con 0x7fdef8161890 2024-04-25T09:56:57.414 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdee406b4d0 msgr2=0x7fdee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdee406b4d0 0x7fdee406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdee8000f60 tx=0x7fdee80079d0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 msgr2=0x7fdef80cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80cc740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdef0007ef0 tx=0x7fdef000d4a0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 shutdown_connections 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdee406b4d0 0x7fdee406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdef8161890 0x7fdef80cc740 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdef8161240 0x7fdef81f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 --2- 172.21.15.148:0/1678321714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdef80b54a0 0x7fdef8154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 >> 172.21.15.148:0/1678321714 conn(0x7fdef80d04a0 msgr2=0x7fdef80d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 shutdown_connections 2024-04-25T09:56:57.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.413+0000 7fdefe658640 1 -- 172.21.15.148:0/1678321714 wait complete. 2024-04-25T09:56:57.427 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:56:57.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 Processor -- start 2024-04-25T09:56:57.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 -- start start 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 0x7fa5b00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 0x7fa5b00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b0072280 con 0x7fa5b015b270 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b00ce4c0 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.524+0000 7fa5b5e8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0071270 con 0x7fa5b00cdb20 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5af7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 0x7fa5b00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5aeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 0x7fa5b00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5affff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa5b015b270 0x7fa5b015b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:40276/0 (socket says 172.21.15.148:40276) 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5af7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 0x7fa5b00cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57888/0 (socket says 172.21.15.148:57888) 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5aeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 0x7fa5b00cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:50818/0 (socket says 172.21.15.148:50818) 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5affff640 1 -- 172.21.15.148:0/2787010552 learned_addr learned my addr 172.21.15.148:0/2787010552 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5ae7fc640 1 -- 172.21.15.148:0/2787010552 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659678078 0 0) 0x7fa5b0072280 con 0x7fa5b015b270 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5ae7fc640 1 -- 172.21.15.148:0/2787010552 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa598003520 con 0x7fa5b015b270 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5aeffd640 1 -- 172.21.15.148:0/2787010552 >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 msgr2=0x7fa5b00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5aeffd640 1 --2- 172.21.15.148:0/2787010552 >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 0x7fa5b00cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5aeffd640 1 -- 172.21.15.148:0/2787010552 >> v1:172.21.15.188:6789/0 conn(0x7fa5b015b270 legacy=0x7fa5b015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5aeffd640 1 -- 172.21.15.148:0/2787010552 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00714c0 con 0x7fa5b00cdb20 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.525+0000 7fa5af7fe640 1 --2- 172.21.15.148:0/2787010552 >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 0x7fa5b00cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.526+0000 7fa5aeffd640 1 --2- 172.21.15.148:0/2787010552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 0x7fa5b00cdf80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40097a0 tx=0x7fa5a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb210e11e1ec46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.526+0000 7fa5ae7fc640 1 -- 172.21.15.148:0/2787010552 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4037070 con 0x7fa5b00cdb20 2024-04-25T09:56:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.526+0000 7fa5ae7fc640 1 -- 172.21.15.148:0/2787010552 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a4032440 con 0x7fa5b00cdb20 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.526+0000 7fa5ae7fc640 1 -- 172.21.15.148:0/2787010552 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40327f0 con 0x7fa5b00cdb20 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.526+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/2787010552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 msgr2=0x7fa5b00cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.526+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/2787010552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 0x7fa5b00cdf80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40097a0 tx=0x7fa5a402a180 comp rx=0 tx=0).stop 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.527+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/2787010552 shutdown_connections 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.527+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/2787010552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b00cdb20 0x7fa5b00cdf80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.527+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/2787010552 >> v2:172.21.15.188:3300/0 conn(0x7fa5b00cc4f0 0x7fa5b00cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.527+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/2787010552 >> 172.21.15.148:0/2787010552 conn(0x7fa5b006abe0 msgr2=0x7fa5b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.527+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/2787010552 shutdown_connections 2024-04-25T09:56:57.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.527+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/2787010552 wait complete. 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.528+0000 7fa5b5e8f640 1 Processor -- start 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.528+0000 7fa5b5e8f640 1 -- start start 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 0x7fa5b013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b00cdb20 0x7fa5b01f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b015b270 0x7fa5b0159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0071a80 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b015bd90 con 0x7fa5b00cdb20 2024-04-25T09:56:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0071d60 con 0x7fa5b015b270 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 0x7fa5b013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 0x7fa5b013e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44096/0 (socket says 172.21.15.148:44096) 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 -- 172.21.15.148:0/3278011162 learned_addr learned my addr 172.21.15.148:0/3278011162 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5aeffd640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b00cdb20 0x7fa5b01f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5affff640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b015b270 0x7fa5b0159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 -- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b015b270 msgr2=0x7fa5b0159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b015b270 0x7fa5b0159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 -- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b00cdb20 msgr2=0x7fa5b01f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b00cdb20 0x7fa5b01f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 -- 172.21.15.148:0/3278011162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b014b820 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5af7fe640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 0x7fa5b013e290 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000e990 tx=0x7fa5a000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0017070 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a000c9e0 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b014bb00 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.529+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000cdb0 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.530+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b014c3a0 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.531+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b014bef0 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.531+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5a001a070 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.531+0000 7fa597fff640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.531+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa5a008a540 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.531+0000 7fa5aeffd640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.532+0000 7fa5aeffd640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4034000 tx=0x7fa5a402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.535+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5a0058bb0 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.706+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5b0073a00 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.706+0000 7fa597fff640 1 -- 172.21.15.148:0/3278011162 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5a005cc50 con 0x7fa5b00cc4f0 2024-04-25T09:56:57.709 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:56:57.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 msgr2=0x7fa59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4034000 tx=0x7fa5a402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 msgr2=0x7fa5b013e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 0x7fa5b013e290 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000e990 tx=0x7fa5a000ee50 comp rx=0 tx=0).stop 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 shutdown_connections 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5b015b270 0x7fa5b0159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b00cdb20 0x7fa5b01f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 --2- 172.21.15.148:0/3278011162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5b00cc4f0 0x7fa5b013e290 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 >> 172.21.15.148:0/3278011162 conn(0x7fa5b006abe0 msgr2=0x7fa5b015b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 shutdown_connections 2024-04-25T09:56:57.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.708+0000 7fa5b5e8f640 1 -- 172.21.15.148:0/3278011162 wait complete. 2024-04-25T09:56:57.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:56:57.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.819+0000 7f0aa0dab640 1 Processor -- start 2024-04-25T09:56:57.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.819+0000 7f0aa0dab640 1 -- start start 2024-04-25T09:56:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0aa0dab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a9c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0aa0dab640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0a9c06b2f0 0x7f0a9c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0aa0dab640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0b56e0 con 0x7f0a9c06f190 2024-04-25T09:56:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0aa0dab640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b3590 con 0x7f0a9c06b2f0 2024-04-25T09:56:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0aa0dab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b37e0 con 0x7f0a9c15dc70 2024-04-25T09:56:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a93fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0a9c06f190 0x7f0a9c06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36478/0 (socket says 172.21.15.148:36478) 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a93fff640 1 -- 172.21.15.148:0/1209750581 learned_addr learned my addr 172.21.15.148:0/1209750581 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a9a59c640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a9c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a9ad9d640 1 --2- 172.21.15.148:0/1209750581 >> v2:172.21.15.188:3300/0 conn(0x7f0a9c06b2f0 0x7f0a9c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360717223 0 0) 0x7f0a9c0b56e0 con 0x7f0a9c06f190 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a88003520 con 0x7f0a9c06f190 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2102285545 0 0) 0x7f0a88003520 con 0x7f0a9c06f190 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 msgr2=0x7f0a9c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a9c0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 >> v2:172.21.15.188:3300/0 conn(0x7f0a9c06b2f0 msgr2=0x7f0a9c0ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 --2- 172.21.15.148:0/1209750581 >> v2:172.21.15.188:3300/0 conn(0x7f0a9c06b2f0 0x7f0a9c0ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0b3ac0 con 0x7f0a9c06f190 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f0a84001670 con 0x7f0a9c06f190 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a9ad9d640 1 --2- 172.21.15.148:0/1209750581 >> v2:172.21.15.188:3300/0 conn(0x7f0a9c06b2f0 0x7f0a9c0ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:57.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a9a59c640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a9c0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 >> v1:172.21.15.148:6789/0 conn(0x7f0a9c06f190 legacy=0x7f0a9c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a880039b0 0x7f0a88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.820+0000 7f0a99d9b640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b3590 con 0x7f0a88004800 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b37e0 con 0x7f0a880039b0 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8800ba10 con 0x7f0a9c15dc70 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a9a59c640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a880039b0 0x7f0a88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a9ad9d640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 -- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 msgr2=0x7f0a880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 -- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a880039b0 msgr2=0x7f0a88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a880039b0 0x7f0a88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 -- 172.21.15.148:0/1209750581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88003520 con 0x7f0a88004800 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a9ad9d640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c15dc70 0x7f0a880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.821+0000 7f0a93fff640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a88004c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c290 tx=0x7f0a88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b554b39bc2678649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84007d50 con 0x7f0a88004800 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84005af0 con 0x7f0a88004800 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0a99d9b640 1 -- 172.21.15.148:0/1209750581 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84007070 con 0x7f0a88004800 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0aa0dab640 1 -- 172.21.15.148:0/1209750581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 msgr2=0x7f0a88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/1209750581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a88004c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c290 tx=0x7f0a88005830 comp rx=0 tx=0).stop 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0a9ad9d640 1 -- 172.21.15.148:0/1209750581 reap_dead start 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0aa0dab640 1 -- 172.21.15.148:0/1209750581 shutdown_connections 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0aa0dab640 1 -- 172.21.15.148:0/1209750581 >> 172.21.15.148:0/1209750581 conn(0x7f0a9c0b5af0 msgr2=0x7f0a9c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0aa0dab640 1 -- 172.21.15.148:0/1209750581 shutdown_connections 2024-04-25T09:56:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.822+0000 7f0aa0dab640 1 -- 172.21.15.148:0/1209750581 wait complete. 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 Processor -- start 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 -- start start 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a9c09caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c14b150 0x7f0a9c15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c15d250 0x7f0a9c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0701d0 con 0x7f0a88004800 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c15e050 con 0x7f0a9c15d250 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0aa0dab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c070350 con 0x7f0a9c14b150 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0a9a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a9c09caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0a9a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a9c09caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44122/0 (socket says 172.21.15.148:44122) 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0a9a59c640 1 -- 172.21.15.148:0/2717249023 learned_addr learned my addr 172.21.15.148:0/2717249023 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.824+0000 7f0a9ad9d640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c15d250 0x7f0a9c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a93fff640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c14b150 0x7f0a9c15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a9a59c640 1 -- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c14b150 msgr2=0x7f0a9c15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a9a59c640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c14b150 0x7f0a9c15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a9a59c640 1 -- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c15d250 msgr2=0x7f0a9c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a9a59c640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c15d250 0x7f0a9c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a9a59c640 1 -- 172.21.15.148:0/2717249023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c09ba50 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a93fff640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c14b150 0x7f0a9c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a9a59c640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a9c09caf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c011930 tx=0x7f0a8c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00faf0 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c00fc90 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c004030 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c09bd30 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.825+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c09c2f0 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.826+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c06cc80 con 0x7f0a88004800 2024-04-25T09:56:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.826+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a8c012040 con 0x7f0a88004800 2024-04-25T09:56:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.827+0000 7f0a92ffd640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a7c06b4d0 0x7f0a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.827+0000 7f0a93fff640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a7c06b4d0 0x7f0a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.827+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f0a8c08a600 con 0x7f0a88004800 2024-04-25T09:56:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.827+0000 7f0a93fff640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a7c06b4d0 0x7f0a7c06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0a84033000 tx=0x7f0a84009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:57.831+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a8c019070 con 0x7f0a88004800 2024-04-25T09:56:58.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.000+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a9c073a00 con 0x7f0a88004800 2024-04-25T09:56:58.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.000+0000 7f0a92ffd640 1 -- 172.21.15.148:0/2717249023 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a8c059350 con 0x7f0a88004800 2024-04-25T09:56:58.003 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:56:58.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a7c06b4d0 msgr2=0x7f0a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a7c06b4d0 0x7f0a7c06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0a84033000 tx=0x7f0a84009030 comp rx=0 tx=0).stop 2024-04-25T09:56:58.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 msgr2=0x7f0a9c09caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a9c09caf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c011930 tx=0x7f0a8c011df0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 shutdown_connections 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a7c06b4d0 0x7f0a7c06d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a9c15d250 0x7f0a9c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a9c14b150 0x7f0a9c15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 --2- 172.21.15.148:0/2717249023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a88004800 0x7f0a9c09caf0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 >> 172.21.15.148:0/2717249023 conn(0x7f0a9c0b5af0 msgr2=0x7f0a9c0ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 shutdown_connections 2024-04-25T09:56:58.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.002+0000 7f0aa0dab640 1 -- 172.21.15.148:0/2717249023 wait complete. 2024-04-25T09:56:58.016 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:58.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.110+0000 7fea3679e640 1 Processor -- start 2024-04-25T09:56:58.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.110+0000 7fea3679e640 1 -- start start 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea3679e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fea300ce920 0x7fea300cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea3679e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea3006d290 0x7fea3006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea3679e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea30144b30 con 0x7fea30144750 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea3679e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea300712d0 con 0x7fea300ce920 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea3679e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea300715b0 con 0x7fea3006d290 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea2f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fea300ce920 0x7fea300cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea2ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fea30144750 0x7fea3006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36488/0 (socket says 172.21.15.148:36488) 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.111+0000 7fea2ffff640 1 -- 172.21.15.148:0/4156036655 learned_addr learned my addr 172.21.15.148:0/4156036655 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2f7fe640 1 -- 172.21.15.148:0/4156036655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea3006d290 msgr2=0x7fea3006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea34d3b640 1 --2- 172.21.15.148:0/4156036655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea3006d290 0x7fea3006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2f7fe640 1 --2- 172.21.15.148:0/4156036655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea3006d290 0x7fea3006d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2effd640 1 -- 172.21.15.148:0/4156036655 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582291261 0 0) 0x7fea30144b30 con 0x7fea30144750 2024-04-25T09:56:58.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2f7fe640 1 -- 172.21.15.148:0/4156036655 >> v1:172.21.15.148:6789/0 conn(0x7fea30144750 legacy=0x7fea3006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2f7fe640 1 -- 172.21.15.148:0/4156036655 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30071890 con 0x7fea300ce920 2024-04-25T09:56:58.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2f7fe640 1 --2- 172.21.15.148:0/4156036655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea300ce920 0x7fea300cedb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea240097a0 tx=0x7fea2402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f67b294245bfe4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2effd640 1 -- 172.21.15.148:0/4156036655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24037070 con 0x7fea300ce920 2024-04-25T09:56:58.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2effd640 1 -- 172.21.15.148:0/4156036655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea24032440 con 0x7fea300ce920 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.112+0000 7fea2effd640 1 -- 172.21.15.148:0/4156036655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24032810 con 0x7fea300ce920 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 -- 172.21.15.148:0/4156036655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea300ce920 msgr2=0x7fea300cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 --2- 172.21.15.148:0/4156036655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea300ce920 0x7fea300cedb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea240097a0 tx=0x7fea2402a180 comp rx=0 tx=0).stop 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 -- 172.21.15.148:0/4156036655 shutdown_connections 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 --2- 172.21.15.148:0/4156036655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea3006d290 0x7fea3006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 --2- 172.21.15.148:0/4156036655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea300ce920 0x7fea300cedb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 -- 172.21.15.148:0/4156036655 >> 172.21.15.148:0/4156036655 conn(0x7fea3006abe0 msgr2=0x7fea3006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 -- 172.21.15.148:0/4156036655 shutdown_connections 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.113+0000 7fea3679e640 1 -- 172.21.15.148:0/4156036655 wait complete. 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.114+0000 7fea3679e640 1 Processor -- start 2024-04-25T09:56:58.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.114+0000 7fea3679e640 1 -- start start 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea3679e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 0x7fea3013e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea3679e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea300ce920 0x7fea300afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea3679e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea30144750 0x7fea3013aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea3679e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3006f020 con 0x7fea3006d290 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea3679e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3006ece0 con 0x7fea300ce920 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea3679e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea30145450 con 0x7fea30144750 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 0x7fea3013e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 0x7fea3013e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44134/0 (socket says 172.21.15.148:44134) 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 -- 172.21.15.148:0/3149082692 learned_addr learned my addr 172.21.15.148:0/3149082692 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea34d3b640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea30144750 0x7fea3013aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2f7fe640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea300ce920 0x7fea300afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 -- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea30144750 msgr2=0x7fea3013aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea30144750 0x7fea3013aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 -- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea300ce920 msgr2=0x7fea300afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea300ce920 0x7fea300afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2ffff640 1 -- 172.21.15.148:0/3149082692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea300b0440 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.115+0000 7fea2f7fe640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea300ce920 0x7fea300afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.116+0000 7fea2ffff640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 0x7fea3013e7c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea2000c4d0 tx=0x7fea2000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.116+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20018070 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.116+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20007500 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.116+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea30139f60 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.116+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20004a20 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.116+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea30157510 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.117+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea30071b70 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.117+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea2001b070 con 0x7fea3006d290 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.117+0000 7fea2cff9640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea1c06b4d0 0x7fea1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.117+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fea20089680 con 0x7fea3006d290 2024-04-25T09:56:58.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.118+0000 7fea2f7fe640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea1c06b4d0 0x7fea1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.118+0000 7fea2f7fe640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea1c06b4d0 0x7fea1c06d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea24034000 tx=0x7fea2402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.122+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea30071b70 con 0x7fea3006d290 2024-04-25T09:56:58.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.326+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea300cfab0 con 0x7fea3006d290 2024-04-25T09:56:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.326+0000 7fea2cff9640 1 -- 172.21.15.148:0/3149082692 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fea300cfab0 con 0x7fea3006d290 2024-04-25T09:56:58.330 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:58.330 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:55:34.319815+0000","modified":"2024-04-25T09:55:40.935056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/3412433349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3412433349},{"type":"v1","addr":"172.21.15.148:6836","nonce":3412433349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:56:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.328+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea1c06b4d0 msgr2=0x7fea1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.328+0000 7fea3679e640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea1c06b4d0 0x7fea1c06d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea24034000 tx=0x7fea2402e000 comp rx=0 tx=0).stop 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.328+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 msgr2=0x7fea3013e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.328+0000 7fea3679e640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 0x7fea3013e7c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea2000c4d0 tx=0x7fea2000c990 comp rx=0 tx=0).stop 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 shutdown_connections 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea1c06b4d0 0x7fea1c06d990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea30144750 0x7fea3013aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea300ce920 0x7fea300afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 --2- 172.21.15.148:0/3149082692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea3006d290 0x7fea3013e7c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 >> 172.21.15.148:0/3149082692 conn(0x7fea3006abe0 msgr2=0x7fea30143a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 shutdown_connections 2024-04-25T09:56:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.329+0000 7fea3679e640 1 -- 172.21.15.148:0/3149082692 wait complete. 2024-04-25T09:56:58.343 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:58.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.436+0000 7f02e6b90640 1 Processor -- start 2024-04-25T09:56:58.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.436+0000 7f02e6b90640 1 -- start start 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02e6b90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f02e00cc8e0 0x7f02e01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02e6b90640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e00cce50 con 0x7f02e00d1810 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02e6b90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e0161af0 con 0x7f02e015dc90 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02e6b90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e0161dd0 con 0x7f02e00cc8e0 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02e512d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f02e00d1810 0x7f02e00cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36496/0 (socket says 172.21.15.148:36496) 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02e512d640 1 -- 172.21.15.148:0/1539413141 learned_addr learned my addr 172.21.15.148:0/1539413141 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02dffff640 1 --2- 172.21.15.148:0/1539413141 >> v2:172.21.15.188:3300/0 conn(0x7f02e00cc8e0 0x7f02e01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198641965 0 0) 0x7f02e00cce50 con 0x7f02e00d1810 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d0003520 con 0x7f02e00d1810 2024-04-25T09:56:58.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 623700570 0 0) 0x7f02d0003520 con 0x7f02e00d1810 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 >> v2:172.21.15.188:3300/0 conn(0x7f02e00cc8e0 msgr2=0x7f02e01625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 --2- 172.21.15.148:0/1539413141 >> v2:172.21.15.188:3300/0 conn(0x7f02e00cc8e0 0x7f02e01625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 >> v1:172.21.15.188:6789/0 conn(0x7f02e015dc90 legacy=0x7f02e015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e01620b0 con 0x7f02e00d1810 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f02d80019f0 con 0x7f02e00d1810 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 >> v1:172.21.15.148:6789/0 conn(0x7f02e00d1810 legacy=0x7f02e00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d00039b0 0x7f02d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02d00043b0 0x7f02d0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d0005120 0x7f02d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e0161dd0 con 0x7f02d00043b0 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d000c060 con 0x7f02d0005120 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.437+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d000c260 con 0x7f02d00039b0 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02d00043b0 0x7f02d0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02e492c640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d00039b0 0x7f02d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02e512d640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d0005120 0x7f02d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 -- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d00039b0 msgr2=0x7f02d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d00039b0 0x7f02d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 -- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d0005120 msgr2=0x7f02d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d0005120 0x7f02d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02e512d640 1 -- 172.21.15.148:0/1539413141 reap_dead start 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 -- 172.21.15.148:0/1539413141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0003520 con 0x7f02d00043b0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.438+0000 7f02dffff640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02d00043b0 0x7f02d0004880 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009ac0 tx=0x7f02d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb33309b2f48bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc036070 con 0x7f02d00043b0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc031400 con 0x7f02d00043b0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02df7fe640 1 -- 172.21.15.148:0/1539413141 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc009690 con 0x7f02d00043b0 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 -- 172.21.15.148:0/1539413141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02d00043b0 msgr2=0x7f02d0004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02d00043b0 0x7f02d0004880 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009ac0 tx=0x7f02d0005830 comp rx=0 tx=0).stop 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 -- 172.21.15.148:0/1539413141 shutdown_connections 2024-04-25T09:56:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 --2- 172.21.15.148:0/1539413141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02d00043b0 0x7f02d0004880 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 -- 172.21.15.148:0/1539413141 >> 172.21.15.148:0/1539413141 conn(0x7f02e006f590 msgr2=0x7f02e006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 -- 172.21.15.148:0/1539413141 shutdown_connections 2024-04-25T09:56:58.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.439+0000 7f02e6b90640 1 -- 172.21.15.148:0/1539413141 wait complete. 2024-04-25T09:56:58.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.440+0000 7f02e6b90640 1 Processor -- start 2024-04-25T09:56:58.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.440+0000 7f02e6b90640 1 -- start start 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e6b90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d00043b0 0x7f02e0158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e6b90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02e0157780 0x7f02e00b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e6b90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 0x7f02e015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e6b90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e015e1f0 con 0x7f02e00b6650 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e6b90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e015e070 con 0x7f02d00043b0 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e6b90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e01625d0 con 0x7f02e0157780 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e512d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 0x7f02e015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e512d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 0x7f02e015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44140/0 (socket says 172.21.15.148:44140) 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e512d640 1 -- 172.21.15.148:0/4048785085 learned_addr learned my addr 172.21.15.148:0/4048785085 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02dffff640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02e0157780 0x7f02e00b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.441+0000 7f02e492c640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d00043b0 0x7f02e0158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e512d640 1 -- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02e0157780 msgr2=0x7f02e00b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e512d640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02e0157780 0x7f02e00b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e512d640 1 -- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d00043b0 msgr2=0x7f02e0158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e512d640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d00043b0 0x7f02e0158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e512d640 1 -- 172.21.15.148:0/4048785085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e01561d0 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e492c640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d00043b0 0x7f02e0158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e512d640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 0x7f02e015fc00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f02e00cdb80 tx=0x7f02d8007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800eb00 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800eca0 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e01564b0 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8014670 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.442+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e0156a70 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.443+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e0073990 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.443+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02d8014810 con 0x7f02e00b6650 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.444+0000 7f02dd7fa640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f02c406b500 0x7f02c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.444+0000 7f02e492c640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f02c406b500 0x7f02c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.444+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f02d808a000 con 0x7f02e00b6650 2024-04-25T09:56:58.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.444+0000 7f02e492c640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f02c406b500 0x7f02c406d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02e0159620 tx=0x7f02d4007b70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.448+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02d8058cc0 con 0x7f02e00b6650 2024-04-25T09:56:58.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.621+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02e006fbe0 con 0x7f02e00b6650 2024-04-25T09:56:58.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.622+0000 7f02dd7fa640 1 -- 172.21.15.148:0/4048785085 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17104 (secure 0 0 0) 0x7f02d80585c0 con 0x7f02e00b6650 2024-04-25T09:56:58.625 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:58.626 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":43,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:56:33.717690+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:58.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f02c406b500 msgr2=0x7f02c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f02c406b500 0x7f02c406d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02e0159620 tx=0x7f02d4007b70 comp rx=0 tx=0).stop 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 msgr2=0x7f02e015fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 0x7f02e015fc00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f02e00cdb80 tx=0x7f02d8007af0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 shutdown_connections 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f02c406b500 0x7f02c406d9c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02e00b6650 0x7f02e015fc00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02e0157780 0x7f02e00b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 --2- 172.21.15.148:0/4048785085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d00043b0 0x7f02e0158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 >> 172.21.15.148:0/4048785085 conn(0x7f02e006f590 msgr2=0x7f02e00d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 shutdown_connections 2024-04-25T09:56:58.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.624+0000 7f02e6b90640 1 -- 172.21.15.148:0/4048785085 wait complete. 2024-04-25T09:56:58.639 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:56:58.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:58.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.733+0000 7fe8947fe640 1 Processor -- start 2024-04-25T09:56:58.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.733+0000 7fe8947fe640 1 -- start start 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe8947fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe88c0cfec0 0x7fe88c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe8947fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0ce3c0 0x7fe88c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe8947fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c071480 con 0x7fe88c154ee0 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe8947fe640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c0ce830 con 0x7fe88c0cfec0 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe8947fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c072090 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe88c0cfec0 0x7fe88c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe88c0cfec0 0x7fe88c0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44142/0 (socket says 172.21.15.148:44142) 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 -- 172.21.15.148:0/640130057 learned_addr learned my addr 172.21.15.148:0/640130057 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe892d9b640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0ce3c0 0x7fe88c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 -- 172.21.15.148:0/640130057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0ce3c0 msgr2=0x7fe88c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0ce3c0 0x7fe88c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 -- 172.21.15.148:0/640130057 >> v1:172.21.15.188:6789/0 conn(0x7fe88c154ee0 legacy=0x7fe88c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe891d99640 1 -- 172.21.15.148:0/640130057 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c071760 con 0x7fe88c0cfec0 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.734+0000 7fe892d9b640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0ce3c0 0x7fe88c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.735+0000 7fe891d99640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0cfec0 0x7fe88c0cde80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009870 tx=0x7fe87c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192734bd555292bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.735+0000 7fe891598640 1 -- 172.21.15.148:0/640130057 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c037070 con 0x7fe88c0cfec0 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.735+0000 7fe891598640 1 -- 172.21.15.148:0/640130057 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c032440 con 0x7fe88c0cfec0 2024-04-25T09:56:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.735+0000 7fe891598640 1 -- 172.21.15.148:0/640130057 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c032810 con 0x7fe88c0cfec0 2024-04-25T09:56:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.735+0000 7fe8947fe640 1 -- 172.21.15.148:0/640130057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0cfec0 msgr2=0x7fe88c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.735+0000 7fe8947fe640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0cfec0 0x7fe88c0cde80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009870 tx=0x7fe87c02a270 comp rx=0 tx=0).stop 2024-04-25T09:56:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.736+0000 7fe8947fe640 1 -- 172.21.15.148:0/640130057 shutdown_connections 2024-04-25T09:56:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.736+0000 7fe8947fe640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0cfec0 0x7fe88c0cde80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.736+0000 7fe8947fe640 1 --2- 172.21.15.148:0/640130057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0ce3c0 0x7fe88c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.736+0000 7fe8947fe640 1 -- 172.21.15.148:0/640130057 >> 172.21.15.148:0/640130057 conn(0x7fe88c06abe0 msgr2=0x7fe88c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.736+0000 7fe8947fe640 1 -- 172.21.15.148:0/640130057 shutdown_connections 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.736+0000 7fe8947fe640 1 -- 172.21.15.148:0/640130057 wait complete. 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 Processor -- start 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 -- start start 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 0x7fe88c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0cfec0 0x7fe88c1f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe88c154ee0 0x7fe88c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c071d20 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c071230 con 0x7fe88c154ee0 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe8947fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1552c0 con 0x7fe88c0cfec0 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.737+0000 7fe89259a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 0x7fe88c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 0x7fe88c1555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44150/0 (socket says 172.21.15.148:44150) 2024-04-25T09:56:58.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 -- 172.21.15.148:0/307822025 learned_addr learned my addr 172.21.15.148:0/307822025 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe892d9b640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe88c154ee0 0x7fe88c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 -- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0cfec0 msgr2=0x7fe88c1f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0cfec0 0x7fe88c1f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 -- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe88c154ee0 msgr2=0x7fe88c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe88c154ee0 0x7fe88c14aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 -- 172.21.15.148:0/307822025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c151730 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe89259a640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 0x7fe88c1555d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c3c0 tx=0x7fe88800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880052d0 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe888005490 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c151a10 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888007260 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.738+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c158ea0 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.739+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c073990 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.740+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe888007400 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.740+0000 7fe882ffd640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe87006b4d0 0x7fe87006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.740+0000 7fe891d99640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe87006b4d0 0x7fe87006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.740+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fe888089810 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.741+0000 7fe891d99640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe87006b4d0 0x7fe87006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034000 tx=0x7fe87c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:58.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.745+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe888058560 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.949+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe88c06d360 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.949+0000 7fe882ffd640 1 -- 172.21.15.148:0/307822025 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fe888057e60 con 0x7fe88c0ce3c0 2024-04-25T09:56:58.953 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:58.953 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:55:34.319815+0000","modified":"2024-04-25T09:55:40.935056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/3412433349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3412433349},{"type":"v1","addr":"172.21.15.148:6836","nonce":3412433349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:56:58.953 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2024-04-25T09:56:58.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe87006b4d0 msgr2=0x7fe87006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe87006b4d0 0x7fe87006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe87c034000 tx=0x7fe87c02c000 comp rx=0 tx=0).stop 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 msgr2=0x7fe88c1555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 0x7fe88c1555d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c3c0 tx=0x7fe88800c880 comp rx=0 tx=0).stop 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 shutdown_connections 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe87006b4d0 0x7fe87006d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe88c154ee0 0x7fe88c14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe88c0cfec0 0x7fe88c1f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 --2- 172.21.15.148:0/307822025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe88c0ce3c0 0x7fe88c1555d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 >> 172.21.15.148:0/307822025 conn(0x7fe88c06abe0 msgr2=0x7fe88c06d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 shutdown_connections 2024-04-25T09:56:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:58.951+0000 7fe8947fe640 1 -- 172.21.15.148:0/307822025 wait complete. 2024-04-25T09:56:58.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:56:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.060+0000 7f151e4cb640 1 Processor -- start 2024-04-25T09:56:59.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.060+0000 7f151e4cb640 1 -- start start 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151e4cb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15180d2e90 0x7f15180d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151e4cb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15180d0900 0x7f15180b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151e4cb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15180b9940 con 0x7f151815dd30 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151e4cb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15180d0d40 con 0x7f15180d0900 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151e4cb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15180d1df0 con 0x7f15180d2e90 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15180d0900 0x7f15180b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15180d0900 0x7f15180b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44162/0 (socket says 172.21.15.148:44162) 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 -- 172.21.15.148:0/3354939732 learned_addr learned my addr 172.21.15.148:0/3354939732 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f15177fe640 1 --2- 172.21.15.148:0/3354939732 >> v2:172.21.15.188:3300/0 conn(0x7f15180d2e90 0x7f15180d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 -- 172.21.15.148:0/3354939732 >> v2:172.21.15.188:3300/0 conn(0x7f15180d2e90 msgr2=0x7f15180d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 --2- 172.21.15.148:0/3354939732 >> v2:172.21.15.188:3300/0 conn(0x7f15180d2e90 0x7f15180d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 -- 172.21.15.148:0/3354939732 >> v1:172.21.15.188:6789/0 conn(0x7f151815dd30 legacy=0x7f15180b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.061+0000 7f151ca68640 1 -- 172.21.15.148:0/3354939732 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180d20d0 con 0x7f15180d0900 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.062+0000 7f151ca68640 1 --2- 172.21.15.148:0/3354939732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d0900 0x7f15180b6a40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f15180b6540 tx=0x7f151002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f459eea4a7b198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.062+0000 7f1516ffd640 1 -- 172.21.15.148:0/3354939732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510036070 con 0x7f15180d0900 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.062+0000 7f1516ffd640 1 -- 172.21.15.148:0/3354939732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510031600 con 0x7f15180d0900 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.062+0000 7f1516ffd640 1 -- 172.21.15.148:0/3354939732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15100319d0 con 0x7f15180d0900 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.062+0000 7f151e4cb640 1 -- 172.21.15.148:0/3354939732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d0900 msgr2=0x7f15180b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.062+0000 7f151e4cb640 1 --2- 172.21.15.148:0/3354939732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d0900 0x7f15180b6a40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f15180b6540 tx=0x7f151002a5d0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.063+0000 7f151e4cb640 1 -- 172.21.15.148:0/3354939732 shutdown_connections 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.063+0000 7f151e4cb640 1 --2- 172.21.15.148:0/3354939732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d0900 0x7f15180b6a40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.063+0000 7f151e4cb640 1 --2- 172.21.15.148:0/3354939732 >> v2:172.21.15.188:3300/0 conn(0x7f15180d2e90 0x7f15180d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.063+0000 7f151e4cb640 1 -- 172.21.15.148:0/3354939732 >> 172.21.15.148:0/3354939732 conn(0x7f1518061c30 msgr2=0x7f151806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.063+0000 7f151e4cb640 1 -- 172.21.15.148:0/3354939732 shutdown_connections 2024-04-25T09:56:59.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.063+0000 7f151e4cb640 1 -- 172.21.15.148:0/3354939732 wait complete. 2024-04-25T09:56:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.064+0000 7f151e4cb640 1 Processor -- start 2024-04-25T09:56:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.064+0000 7f151e4cb640 1 -- start start 2024-04-25T09:56:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.064+0000 7f151e4cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15180d0900 0x7f1518152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 0x7f15180b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f151815dd30 0x7f151815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180d2970 con 0x7f15180d2e90 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180d2690 con 0x7f15180d0900 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15180d1a10 con 0x7f151815dd30 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 0x7f15180b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 0x7f15180b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44170/0 (socket says 172.21.15.148:44170) 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 -- 172.21.15.148:0/114016943 learned_addr learned my addr 172.21.15.148:0/114016943 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151ca68640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f151815dd30 0x7f151815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f1517fff640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15180d0900 0x7f1518152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 -- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f151815dd30 msgr2=0x7f151815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f151815dd30 0x7f151815bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 -- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15180d0900 msgr2=0x7f1518152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15180d0900 0x7f1518152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 -- 172.21.15.148:0/114016943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180b3ac0 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f1517fff640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15180d0900 0x7f1518152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f15177fe640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 0x7f15180b3150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d830 tx=0x7f150c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00b6c0 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c019070 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15180b3da0 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00baa0 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.065+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15180b4850 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.066+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1518073990 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.067+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f150c01e460 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.067+0000 7f1514ff9640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f150406b500 0x7f150406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.067+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f150c08aa40 con 0x7f15180d2e90 2024-04-25T09:56:59.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.067+0000 7f1517fff640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f150406b500 0x7f150406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.068+0000 7f1517fff640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f150406b500 0x7f150406d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1508006fd0 tx=0x7f1508008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.071+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f150c059700 con 0x7f15180d2e90 2024-04-25T09:56:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.278+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f151806b210 con 0x7f15180d2e90 2024-04-25T09:56:59.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.278+0000 7f1514ff9640 1 -- 172.21.15.148:0/114016943 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f150c08ae20 con 0x7f15180d2e90 2024-04-25T09:56:59.282 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:59.282 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:55:34.319815+0000","modified":"2024-04-25T09:55:40.935056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/3412433349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3412433349},{"type":"v1","addr":"172.21.15.148:6836","nonce":3412433349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T09:56:59.282 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 17 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f150406b500 msgr2=0x7f150406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f150406b500 0x7f150406d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1508006fd0 tx=0x7f1508008040 comp rx=0 tx=0).stop 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 msgr2=0x7f15180b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 0x7f15180b3150 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d830 tx=0x7f150c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 shutdown_connections 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f150406b500 0x7f150406d9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f151815dd30 0x7f151815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15180d2e90 0x7f15180b3150 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 --2- 172.21.15.148:0/114016943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15180d0900 0x7f1518152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 >> 172.21.15.148:0/114016943 conn(0x7f1518061c30 msgr2=0x7f15180b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 shutdown_connections 2024-04-25T09:56:59.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.280+0000 7f151e4cb640 1 -- 172.21.15.148:0/114016943 wait complete. 2024-04-25T09:56:59.294 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:56:59.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.389+0000 7fc867ba9640 1 Processor -- start 2024-04-25T09:56:59.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.389+0000 7fc867ba9640 1 -- start start 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc867ba9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc8601625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc867ba9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8600cce50 con 0x7fc86015dc90 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc867ba9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc860161af0 con 0x7fc8600cc8e0 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc867ba9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc860161dd0 con 0x7fc8600d1f20 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc865945640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc86015dc90 0x7fc86015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36506/0 (socket says 172.21.15.148:36506) 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc865945640 1 -- 172.21.15.148:0/1931320855 learned_addr learned my addr 172.21.15.148:0/1931320855 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315120053 0 0) 0x7fc8600cce50 con 0x7fc86015dc90 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc866146640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc8601625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc84c003520 con 0x7fc86015dc90 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3506678737 0 0) 0x7fc84c003520 con 0x7fc86015dc90 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 msgr2=0x7fc8601625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc8601625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 >> v1:172.21.15.188:6789/0 conn(0x7fc8600cc8e0 legacy=0x7fc8600d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8601620b0 con 0x7fc86015dc90 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc85c001670 con 0x7fc86015dc90 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc866146640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc8601625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 >> v1:172.21.15.148:6789/0 conn(0x7fc86015dc90 legacy=0x7fc86015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc84c0039e0 0x7fc84c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc84c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc84c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc860161dd0 con 0x7fc84c004800 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c005510 con 0x7fc84c0039e0 2024-04-25T09:56:59.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.390+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c00bf00 con 0x7fc8600d1f20 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc84c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865945640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc84c0039e0 0x7fc84c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc866146640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc84c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 -- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 msgr2=0x7fc84c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600d1f20 0x7fc84c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 -- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc84c0039e0 msgr2=0x7fc84c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc84c0039e0 0x7fc84c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 -- 172.21.15.148:0/1931320855 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c003520 con 0x7fc84c004800 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.391+0000 7fc865144640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc84c004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc850001750 tx=0x7fc84c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecabcc235322f1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850001370 con 0x7fc84c004800 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8500052d0 con 0x7fc84c004800 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc864943640 1 -- 172.21.15.148:0/1931320855 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85002abe0 con 0x7fc84c004800 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc867ba9640 1 -- 172.21.15.148:0/1931320855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 msgr2=0x7fc84c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1931320855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc84c004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc850001750 tx=0x7fc84c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc866146640 1 -- 172.21.15.148:0/1931320855 reap_dead start 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc867ba9640 1 -- 172.21.15.148:0/1931320855 shutdown_connections 2024-04-25T09:56:59.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc867ba9640 1 -- 172.21.15.148:0/1931320855 >> 172.21.15.148:0/1931320855 conn(0x7fc86006f590 msgr2=0x7fc86006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc867ba9640 1 -- 172.21.15.148:0/1931320855 shutdown_connections 2024-04-25T09:56:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.392+0000 7fc867ba9640 1 -- 172.21.15.148:0/1931320855 wait complete. 2024-04-25T09:56:59.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.393+0000 7fc867ba9640 1 Processor -- start 2024-04-25T09:56:59.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.393+0000 7fc867ba9640 1 -- start start 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc867ba9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc8601e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc867ba9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600c7db0 0x7fc8601e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc867ba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc860147bd0 0x7fc860148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc867ba9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86015e1f0 con 0x7fc84c004800 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc867ba9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86015e070 con 0x7fc860147bd0 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc867ba9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8601625d0 con 0x7fc8600c7db0 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc8601e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc8601e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44194/0 (socket says 172.21.15.148:44194) 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 -- 172.21.15.148:0/1272606345 learned_addr learned my addr 172.21.15.148:0/1272606345 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 -- 172.21.15.148:0/1272606345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600c7db0 msgr2=0x7fc8601e7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600c7db0 0x7fc8601e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 -- 172.21.15.148:0/1272606345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc860147bd0 msgr2=0x7fc860148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc860147bd0 0x7fc860148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.394+0000 7fc865945640 1 -- 172.21.15.148:0/1272606345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8601489a0 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.395+0000 7fc865945640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc8601e76f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0073a0 tx=0x7fc85c007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.395+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c01a070 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.395+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c00f860 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.395+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc860145b10 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.395+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00fc50 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.395+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8601460d0 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.396+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc860073990 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.396+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc85c01d070 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.396+0000 7fc8567fc640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84406b500 0x7fc84406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.397+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fc85c089be0 con 0x7fc84c004800 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.397+0000 7fc865144640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84406b500 0x7fc84406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.397+0000 7fc865144640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84406b500 0x7fc84406d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc850032000 tx=0x7fc85002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.401+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc85c058930 con 0x7fc84c004800 2024-04-25T09:56:59.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.574+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8600d1c20 con 0x7fc84c004800 2024-04-25T09:56:59.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.575+0000 7fc8567fc640 1 -- 172.21.15.148:0/1272606345 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17104 (secure 0 0 0) 0x7fc85c058230 con 0x7fc84c004800 2024-04-25T09:56:59.579 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:56:59.579 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":43,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:56:33.717690+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:55:32.158059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T09:55:33.292007+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.577+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84406b500 msgr2=0x7fc84406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.577+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84406b500 0x7fc84406d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc850032000 tx=0x7fc85002c000 comp rx=0 tx=0).stop 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.577+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 msgr2=0x7fc8601e76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.577+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc8601e76f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0073a0 tx=0x7fc85c007860 comp rx=0 tx=0).stop 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 shutdown_connections 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84406b500 0x7fc84406d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc860147bd0 0x7fc860148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8600c7db0 0x7fc8601e7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 --2- 172.21.15.148:0/1272606345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc84c004800 0x7fc8601e76f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 >> 172.21.15.148:0/1272606345 conn(0x7fc86006f590 msgr2=0x7fc8600cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 shutdown_connections 2024-04-25T09:56:59.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.578+0000 7fc867ba9640 1 -- 172.21.15.148:0/1272606345 wait complete. 2024-04-25T09:56:59.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:56:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 Processor -- start 2024-04-25T09:56:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 -- start start 2024-04-25T09:56:59.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f74162140 0x7f7f740cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 0x7f7f740caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7415e3a0 con 0x7f7f740d0a00 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f741625c0 con 0x7f7f74162140 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.687+0000 7f7f7a7fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740cd9d0 con 0x7f7f740cd540 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f73fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7f740d0a00 0x7f7f740d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36518/0 (socket says 172.21.15.148:36518) 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f73fff640 1 -- 172.21.15.148:0/2757845368 learned_addr learned my addr 172.21.15.148:0/2757845368 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f78d9b640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 0x7f7f740caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f737fe640 1 --2- 172.21.15.148:0/2757845368 >> v2:172.21.15.188:3300/0 conn(0x7f7f74162140 0x7f7f740cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805041506 0 0) 0x7f7f7415e3a0 con 0x7f7f740d0a00 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f5c003520 con 0x7f7f740d0a00 2024-04-25T09:56:59.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3454715982 0 0) 0x7f7f5c003520 con 0x7f7f740d0a00 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 msgr2=0x7f7f740caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 0x7f7f740caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 >> v2:172.21.15.188:3300/0 conn(0x7f7f74162140 msgr2=0x7f7f740cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 --2- 172.21.15.148:0/2757845368 >> v2:172.21.15.188:3300/0 conn(0x7f7f74162140 0x7f7f740cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740cdc70 con 0x7f7f740d0a00 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7f64001670 con 0x7f7f740d0a00 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 >> v1:172.21.15.148:6789/0 conn(0x7f7f740d0a00 legacy=0x7f7f740d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f5c0039b0 0x7f7f5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 0x7f7f5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741625c0 con 0x7f7f5c004800 2024-04-25T09:56:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740cd9d0 con 0x7f7f5c0039b0 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c00b9f0 con 0x7f7f740cd540 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f737fe640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f73fff640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f5c0039b0 0x7f7f5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.688+0000 7f7f78d9b640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 0x7f7f5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f737fe640 1 -- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 msgr2=0x7f7f5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f737fe640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f740cd540 0x7f7f5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f737fe640 1 -- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f5c0039b0 msgr2=0x7f7f5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f737fe640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f5c0039b0 0x7f7f5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f737fe640 1 -- 172.21.15.148:0/2757845368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c003520 con 0x7f7f5c004800 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f73fff640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f5c0039b0 0x7f7f5c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f737fe640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f5c004c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009ac0 tx=0x7f7f5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707f14898c527ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68036070 con 0x7f7f5c004800 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68031400 con 0x7f7f5c004800 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.689+0000 7f7f72ffd640 1 -- 172.21.15.148:0/2757845368 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68009690 con 0x7f7f5c004800 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/2757845368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 msgr2=0x7f7f5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/2757845368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f5c004c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009ac0 tx=0x7f7f5c005830 comp rx=0 tx=0).stop 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f78d9b640 1 -- 172.21.15.148:0/2757845368 reap_dead start 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/2757845368 shutdown_connections 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/2757845368 >> 172.21.15.148:0/2757845368 conn(0x7f7f740d17c0 msgr2=0x7f7f740d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/2757845368 shutdown_connections 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.690+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/2757845368 wait complete. 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.691+0000 7f7f7a7fe640 1 Processor -- start 2024-04-25T09:56:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.691+0000 7f7f7a7fe640 1 -- start start 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f7a7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f7412fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f7a7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f741e8710 0x7f7f741e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f7a7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f741e8b90 0x7f7f741e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f7a7fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74161940 con 0x7f7f5c004800 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f7a7fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74161660 con 0x7f7f741e8b90 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f7a7fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74161c20 con 0x7f7f741e8710 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f7412fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f7412fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44220/0 (socket says 172.21.15.148:44220) 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 -- 172.21.15.148:0/3418009746 learned_addr learned my addr 172.21.15.148:0/3418009746 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f737fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f741e8710 0x7f7f741e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f78d9b640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f741e8b90 0x7f7f741e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 -- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f741e8710 msgr2=0x7f7f741e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f741e8710 0x7f7f741e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 -- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f741e8b90 msgr2=0x7f7f741e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f741e8b90 0x7f7f741e97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.692+0000 7f7f73fff640 1 -- 172.21.15.148:0/3418009746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74143dc0 con 0x7f7f5c004800 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.693+0000 7f7f73fff640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f7412fd50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400e700 tx=0x7f7f6400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.693+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400c6c0 con 0x7f7f5c004800 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.693+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f64018070 con 0x7f7f5c004800 2024-04-25T09:56:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.693+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400ca60 con 0x7f7f5c004800 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.693+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f741467e0 con 0x7f7f5c004800 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.693+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f74130290 con 0x7f7f5c004800 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.694+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f74146d00 con 0x7f7f5c004800 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.694+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f6400cc00 con 0x7f7f5c004800 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.695+0000 7f7f70ff9640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f6006b4d0 0x7f7f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.695+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f7f6408a190 con 0x7f7f5c004800 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.695+0000 7f7f737fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f6006b4d0 0x7f7f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:56:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.695+0000 7f7f737fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f6006b4d0 0x7f7f6006d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f68033000 tx=0x7f7f68008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:56:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.699+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f74146d00 con 0x7f7f5c004800 2024-04-25T09:56:59.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.905+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7f74073a00 con 0x7f7f5c004800 2024-04-25T09:56:59.909 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:56:59.906+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:56:59.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.907+0000 7f7f70ff9640 1 -- 172.21.15.148:0/3418009746 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7f64058ee0 con 0x7f7f5c004800 2024-04-25T09:56:59.910 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f6006b4d0 msgr2=0x7f7f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f6006b4d0 0x7f7f6006d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f68033000 tx=0x7f7f68008030 comp rx=0 tx=0).stop 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 msgr2=0x7f7f7412fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f7412fd50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400e700 tx=0x7f7f6400ebc0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 shutdown_connections 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f6006b4d0 0x7f7f6006d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f741e8b90 0x7f7f741e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f741e8710 0x7f7f741e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 --2- 172.21.15.148:0/3418009746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f5c004800 0x7f7f7412fd50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:56:59.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.910+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 >> 172.21.15.148:0/3418009746 conn(0x7f7f740d17c0 msgr2=0x7f7f7415f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:56:59.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.911+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 shutdown_connections 2024-04-25T09:56:59.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:56:59.911+0000 7f7f7a7fe640 1 -- 172.21.15.148:0/3418009746 wait complete. 2024-04-25T09:56:59.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:56:59.957 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:57:00.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.027+0000 7f772624d640 1 Processor -- start 2024-04-25T09:57:00.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.027+0000 7f772624d640 1 -- start start 2024-04-25T09:57:00.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f772624d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f772006fbd0 0x7f77200ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f772624d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f77200ce730 0x7f77200b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f772624d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77200b4d80 con 0x7f7720155f80 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f772624d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77200b5390 con 0x7f772006fbd0 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f772624d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77200d07f0 con 0x7f77200ce730 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7720155f80 0x7f772006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36542/0 (socket says 172.21.15.148:36542) 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771f7fe640 1 -- 172.21.15.148:0/3983666368 learned_addr learned my addr 172.21.15.148:0/3983666368 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771effd640 1 --2- 172.21.15.148:0/3983666368 >> v2:172.21.15.148:3300/0 conn(0x7f772006fbd0 0x7f77200ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771e7fc640 1 -- 172.21.15.148:0/3983666368 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726586016 0 0) 0x7f77200b4d80 con 0x7f7720155f80 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771ffff640 1 --2- 172.21.15.148:0/3983666368 >> v2:172.21.15.188:3300/0 conn(0x7f77200ce730 0x7f77200b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771e7fc640 1 -- 172.21.15.148:0/3983666368 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7708003520 con 0x7f7720155f80 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771effd640 1 -- 172.21.15.148:0/3983666368 >> v2:172.21.15.188:3300/0 conn(0x7f77200ce730 msgr2=0x7f77200b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771effd640 1 --2- 172.21.15.148:0/3983666368 >> v2:172.21.15.188:3300/0 conn(0x7f77200ce730 0x7f77200b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771effd640 1 -- 172.21.15.148:0/3983666368 >> v1:172.21.15.148:6789/0 conn(0x7f7720155f80 legacy=0x7f772006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.028+0000 7f771effd640 1 -- 172.21.15.148:0/3983666368 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200cef70 con 0x7f772006fbd0 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.029+0000 7f771effd640 1 --2- 172.21.15.148:0/3983666368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772006fbd0 0x7f77200ce1f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f77200625b0 tx=0x7f771402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6707c20cc0b8cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.029+0000 7f771e7fc640 1 -- 172.21.15.148:0/3983666368 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714037070 con 0x7f772006fbd0 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.029+0000 7f771e7fc640 1 -- 172.21.15.148:0/3983666368 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7714032440 con 0x7f772006fbd0 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.029+0000 7f771e7fc640 1 -- 172.21.15.148:0/3983666368 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714032810 con 0x7f772006fbd0 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 -- 172.21.15.148:0/3983666368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772006fbd0 msgr2=0x7f77200ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 --2- 172.21.15.148:0/3983666368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772006fbd0 0x7f77200ce1f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f77200625b0 tx=0x7f771402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 -- 172.21.15.148:0/3983666368 shutdown_connections 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 --2- 172.21.15.148:0/3983666368 >> v2:172.21.15.188:3300/0 conn(0x7f77200ce730 0x7f77200b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 --2- 172.21.15.148:0/3983666368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f772006fbd0 0x7f77200ce1f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 -- 172.21.15.148:0/3983666368 >> 172.21.15.148:0/3983666368 conn(0x7f7720061600 msgr2=0x7f77200619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 -- 172.21.15.148:0/3983666368 shutdown_connections 2024-04-25T09:57:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.030+0000 7f772624d640 1 -- 172.21.15.148:0/3983666368 wait complete. 2024-04-25T09:57:00.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.031+0000 7f772624d640 1 Processor -- start 2024-04-25T09:57:00.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.031+0000 7f772624d640 1 -- start start 2024-04-25T09:57:00.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f772624d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f772006fbd0 0x7f77201400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f772624d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 0x7f772014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f772624d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7720155f80 0x7f772014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f772624d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77200cf250 con 0x7f77200ce730 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f772624d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7720156360 con 0x7f772006fbd0 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f772624d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77200cf3d0 con 0x7f7720155f80 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 0x7f772014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 0x7f772014e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44244/0 (socket says 172.21.15.148:44244) 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 -- 172.21.15.148:0/3098591142 learned_addr learned my addr 172.21.15.148:0/3098591142 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 -- 172.21.15.148:0/3098591142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7720155f80 msgr2=0x7f772014fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7720155f80 0x7f772014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 -- 172.21.15.148:0/3098591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f772006fbd0 msgr2=0x7f77201400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f772006fbd0 0x7f77201400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 -- 172.21.15.148:0/3098591142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720140620 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f771effd640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 0x7f772014e2e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7714000c00 tx=0x7f77140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714037040 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.032+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7714042070 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.033+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7720150c40 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.033+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714030960 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.033+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7720151200 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.034+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7720073a00 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.034+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7714030b00 con 0x7f77200ce730 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.034+0000 7f7707fff640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f770c06b4d0 0x7f770c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:00.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.035+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f77140aada0 con 0x7f77200ce730 2024-04-25T09:57:00.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.035+0000 7f771f7fe640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f770c06b4d0 0x7f770c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:00.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.035+0000 7f771f7fe640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f770c06b4d0 0x7f770c06d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7710000f60 tx=0x7f77100078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:00.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.039+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7714079ab0 con 0x7f77200ce730 2024-04-25T09:57:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.245+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f77200b1030 con 0x7f77200ce730 2024-04-25T09:57:00.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.909+0000 7f7707fff640 1 -- 172.21.15.148:0/3098591142 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f77140793b0 con 0x7f77200ce730 2024-04-25T09:57:00.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f770c06b4d0 msgr2=0x7f770c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f770c06b4d0 0x7f770c06d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7710000f60 tx=0x7f77100078f0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 msgr2=0x7f772014e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:00.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 0x7f772014e2e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7714000c00 tx=0x7f77140046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 shutdown_connections 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f770c06b4d0 0x7f770c06d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7720155f80 0x7f772014fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77200ce730 0x7f772014e2e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 --2- 172.21.15.148:0/3098591142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f772006fbd0 0x7f77201400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 >> 172.21.15.148:0/3098591142 conn(0x7f7720061600 msgr2=0x7f77200d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 shutdown_connections 2024-04-25T09:57:00.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:00.911+0000 7f772624d640 1 -- 172.21.15.148:0/3098591142 wait complete. 2024-04-25T09:57:00.927 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:57:01.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.067+0000 7f757af09640 1 Processor -- start 2024-04-25T09:57:01.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.067+0000 7f757af09640 1 -- start start 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f757af09640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f757415dac0 0x7f75740d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f757af09640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75740d0ab0 0x7f75740b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f757af09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75740d2420 con 0x7f75740d1f30 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f757af09640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740d3030 con 0x7f757415dac0 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f757af09640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740d2700 con 0x7f75740d0ab0 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7578ca5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f757415dac0 0x7f75740d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7573fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f75740d1f30 0x7f75740d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36552/0 (socket says 172.21.15.148:36552) 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7573fff640 1 -- 172.21.15.148:0/3766659857 learned_addr learned my addr 172.21.15.148:0/3766659857 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7578ca5640 1 -- 172.21.15.148:0/3766659857 >> v2:172.21.15.188:3300/0 conn(0x7f75740d0ab0 msgr2=0x7f75740b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7578ca5640 1 --2- 172.21.15.148:0/3766659857 >> v2:172.21.15.188:3300/0 conn(0x7f75740d0ab0 0x7f75740b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7578ca5640 1 -- 172.21.15.148:0/3766659857 >> v1:172.21.15.148:6789/0 conn(0x7f75740d1f30 legacy=0x7f75740d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f75737fe640 1 -- 172.21.15.148:0/3766659857 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484585959 0 0) 0x7f75740d2420 con 0x7f75740d1f30 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.068+0000 7f7578ca5640 1 -- 172.21.15.148:0/3766659857 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740d29e0 con 0x7f757415dac0 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.069+0000 7f7578ca5640 1 --2- 172.21.15.148:0/3766659857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757415dac0 0x7f75740d19f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f75740cf780 tx=0x7f756402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bce198f80de0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.069+0000 7f75737fe640 1 -- 172.21.15.148:0/3766659857 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564037070 con 0x7f757415dac0 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.069+0000 7f75737fe640 1 -- 172.21.15.148:0/3766659857 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564032440 con 0x7f757415dac0 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.069+0000 7f75737fe640 1 -- 172.21.15.148:0/3766659857 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564032810 con 0x7f757415dac0 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 -- 172.21.15.148:0/3766659857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757415dac0 msgr2=0x7f75740d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 --2- 172.21.15.148:0/3766659857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757415dac0 0x7f75740d19f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f75740cf780 tx=0x7f756402a230 comp rx=0 tx=0).stop 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 -- 172.21.15.148:0/3766659857 shutdown_connections 2024-04-25T09:57:01.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 --2- 172.21.15.148:0/3766659857 >> v2:172.21.15.188:3300/0 conn(0x7f75740d0ab0 0x7f75740b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 --2- 172.21.15.148:0/3766659857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757415dac0 0x7f75740d19f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 -- 172.21.15.148:0/3766659857 >> 172.21.15.148:0/3766659857 conn(0x7f7574061c30 msgr2=0x7f757406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 -- 172.21.15.148:0/3766659857 shutdown_connections 2024-04-25T09:57:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.070+0000 7f757af09640 1 -- 172.21.15.148:0/3766659857 wait complete. 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.071+0000 7f757af09640 1 Processor -- start 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.071+0000 7f757af09640 1 -- start start 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f757af09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740d0ab0 0x7f75741f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f757af09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 0x7f7574160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f757af09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757415dac0 0x7f75741607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f757af09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75740b97d0 con 0x7f75740d1f30 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f757af09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415dea0 con 0x7f757415dac0 2024-04-25T09:57:01.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f757af09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740b9950 con 0x7f75740d0ab0 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 0x7f7574160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 0x7f7574160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44250/0 (socket says 172.21.15.148:44250) 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 -- 172.21.15.148:0/3250222840 learned_addr learned my addr 172.21.15.148:0/3250222840 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f75794a6640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757415dac0 0x7f75741607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7578ca5640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740d0ab0 0x7f75741f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 -- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740d0ab0 msgr2=0x7f75741f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740d0ab0 0x7f75741f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 -- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757415dac0 msgr2=0x7f75741607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757415dac0 0x7f75741607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 -- 172.21.15.148:0/3250222840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75741f6b80 con 0x7f75740d1f30 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f75794a6640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757415dac0 0x7f75741607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:01.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.072+0000 7f7573fff640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 0x7f7574160280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f756800c4d0 tx=0x7f756800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.073+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568007480 con 0x7f75740d1f30 2024-04-25T09:57:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.073+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75680057a0 con 0x7f75740d1f30 2024-04-25T09:57:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.073+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568007620 con 0x7f75740d1f30 2024-04-25T09:57:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.073+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757415f0c0 con 0x7f75740d1f30 2024-04-25T09:57:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.073+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757415f680 con 0x7f75740d1f30 2024-04-25T09:57:01.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.074+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7574073a00 con 0x7f75740d1f30 2024-04-25T09:57:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.074+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7568005b50 con 0x7f75740d1f30 2024-04-25T09:57:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.075+0000 7f75717fa640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f756006b4d0 0x7f756006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.075+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f756808a020 con 0x7f75740d1f30 2024-04-25T09:57:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.075+0000 7f7578ca5640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f756006b4d0 0x7f756006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.075+0000 7f7578ca5640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f756006b4d0 0x7f756006d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7564034000 tx=0x7f756402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:01.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.079+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75680182b0 con 0x7f75740d1f30 2024-04-25T09:57:01.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.254+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75740d1700 con 0x7f75740d1f30 2024-04-25T09:57:01.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.913+0000 7f75717fa640 1 -- 172.21.15.148:0/3250222840 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7568058ca0 con 0x7f75740d1f30 2024-04-25T09:57:01.917 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:57:01.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f756006b4d0 msgr2=0x7f756006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:01.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f756006b4d0 0x7f756006d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7564034000 tx=0x7f756402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:01.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 msgr2=0x7f7574160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:01.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 0x7f7574160280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f756800c4d0 tx=0x7f756800c990 comp rx=0 tx=0).stop 2024-04-25T09:57:01.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 shutdown_connections 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f756006b4d0 0x7f756006d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757415dac0 0x7f75741607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75740d1f30 0x7f7574160280 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 --2- 172.21.15.148:0/3250222840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740d0ab0 0x7f75741f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 >> 172.21.15.148:0/3250222840 conn(0x7f7574061c30 msgr2=0x7f75740b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 shutdown_connections 2024-04-25T09:57:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:01.917+0000 7f757af09640 1 -- 172.21.15.148:0/3250222840 wait complete. 2024-04-25T09:57:01.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:57:02.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.034+0000 7fcaf6789640 1 Processor -- start 2024-04-25T09:57:02.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.034+0000 7fcaf6789640 1 -- start start 2024-04-25T09:57:02.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaf6789640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcaf006ccd0 0x7fcaf00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:02.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaf6789640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00d1770 con 0x7fcaf0070dc0 2024-04-25T09:57:02.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaf6789640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf0162740 con 0x7fcaf015deb0 2024-04-25T09:57:02.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaf6789640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf01622b0 con 0x7fcaf006ccd0 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaef7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcaf006ccd0 0x7fcaf00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaeffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcaf0070dc0 0x7fcaf00711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36562/0 (socket says 172.21.15.148:36562) 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaef7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcaf006ccd0 0x7fcaf00b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44262/0 (socket says 172.21.15.148:44262) 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaeffff640 1 -- 172.21.15.148:0/2027000 learned_addr learned my addr 172.21.15.148:0/2027000 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaef7fe640 1 -- 172.21.15.148:0/2027000 >> v1:172.21.15.188:6789/0 conn(0x7fcaf015deb0 legacy=0x7fcaf015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaef7fe640 1 -- 172.21.15.148:0/2027000 >> v1:172.21.15.148:6789/0 conn(0x7fcaf0070dc0 legacy=0x7fcaf00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.035+0000 7fcaef7fe640 1 -- 172.21.15.148:0/2027000 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0161980 con 0x7fcaf006ccd0 2024-04-25T09:57:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.036+0000 7fcaef7fe640 1 --2- 172.21.15.148:0/2027000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00b07a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcae40097a0 tx=0x7fcae402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9265476c2506c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.036+0000 7fcaeeffd640 1 -- 172.21.15.148:0/2027000 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4037070 con 0x7fcaf006ccd0 2024-04-25T09:57:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.036+0000 7fcaeeffd640 1 -- 172.21.15.148:0/2027000 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4032440 con 0x7fcaf006ccd0 2024-04-25T09:57:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.036+0000 7fcaeeffd640 1 -- 172.21.15.148:0/2027000 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae402ae60 con 0x7fcaf006ccd0 2024-04-25T09:57:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.036+0000 7fcaf6789640 1 -- 172.21.15.148:0/2027000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 msgr2=0x7fcaf00b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.037+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2027000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00b07a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcae40097a0 tx=0x7fcae402a110 comp rx=0 tx=0).stop 2024-04-25T09:57:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.037+0000 7fcaf6789640 1 -- 172.21.15.148:0/2027000 shutdown_connections 2024-04-25T09:57:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.037+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2027000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00b07a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.037+0000 7fcaf6789640 1 -- 172.21.15.148:0/2027000 >> 172.21.15.148:0/2027000 conn(0x7fcaf00b3530 msgr2=0x7fcaf00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.037+0000 7fcaf6789640 1 -- 172.21.15.148:0/2027000 shutdown_connections 2024-04-25T09:57:02.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.037+0000 7fcaf6789640 1 -- 172.21.15.148:0/2027000 wait complete. 2024-04-25T09:57:02.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.038+0000 7fcaf6789640 1 Processor -- start 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.038+0000 7fcaf6789640 1 -- start start 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaf0070dc0 0x7fcaf015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaf015deb0 0x7fcaf00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0161f40 con 0x7fcaf006ccd0 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b10f0 con 0x7fcaf0070dc0 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00b5840 con 0x7fcaf015deb0 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44274/0 (socket says 172.21.15.148:44274) 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 -- 172.21.15.148:0/2646739272 learned_addr learned my addr 172.21.15.148:0/2646739272 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaef7fe640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaf0070dc0 0x7fcaf015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf4d26640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaf015deb0 0x7fcaf00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 -- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaf015deb0 msgr2=0x7fcaf00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaf015deb0 0x7fcaf00ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 -- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaf0070dc0 msgr2=0x7fcaf015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaf0070dc0 0x7fcaf015c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 -- 172.21.15.148:0/2646739272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00cae00 con 0x7fcaf006ccd0 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaef7fe640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaf0070dc0 0x7fcaf015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf4d26640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaf015deb0 0x7fcaf00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaeffff640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00cbea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c4d0 tx=0x7fcae000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0017070 con 0x7fcaf006ccd0 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae0005ce0 con 0x7fcaf006ccd0 2024-04-25T09:57:02.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf00cb0e0 con 0x7fcaf006ccd0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0007270 con 0x7fcaf006ccd0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.039+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf00cb6a0 con 0x7fcaf006ccd0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.040+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0073a00 con 0x7fcaf006ccd0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.041+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcae001a070 con 0x7fcaf006ccd0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.041+0000 7fcaecff9640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcadc06b4d0 0x7fcadc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.041+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5778+0+0 (secure 0 0 0) 0x7fcae00894d0 con 0x7fcaf006ccd0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.041+0000 7fcaef7fe640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcadc06b4d0 0x7fcadc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.042+0000 7fcaef7fe640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcadc06b4d0 0x7fcadc06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcae4034000 tx=0x7fcae402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:02.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.045+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcae0057cb0 con 0x7fcaf006ccd0 2024-04-25T09:57:02.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.214+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcaf015d680 con 0x7fcaf006ccd0 2024-04-25T09:57:02.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.919+0000 7fcaecff9640 1 -- 172.21.15.148:0/2646739272 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcae005bd50 con 0x7fcaf006ccd0 2024-04-25T09:57:02.922 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcadc06b4d0 msgr2=0x7fcadc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcadc06b4d0 0x7fcadc06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcae4034000 tx=0x7fcae402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 msgr2=0x7fcaf00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00cbea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c4d0 tx=0x7fcae000c990 comp rx=0 tx=0).stop 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 shutdown_connections 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcadc06b4d0 0x7fcadc06d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaf015deb0 0x7fcaf00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaf0070dc0 0x7fcaf015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 --2- 172.21.15.148:0/2646739272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaf006ccd0 0x7fcaf00cbea0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 >> 172.21.15.148:0/2646739272 conn(0x7fcaf00b3530 msgr2=0x7fcaf00711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 shutdown_connections 2024-04-25T09:57:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:02.921+0000 7fcaf6789640 1 -- 172.21.15.148:0/2646739272 wait complete. 2024-04-25T09:57:02.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:57:03.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.037+0000 7fd46503a640 1 Processor -- start 2024-04-25T09:57:03.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.037+0000 7fd46503a640 1 -- start start 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd46503a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 0x7fd4600cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd46503a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd46015b980 0x7fd4600cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd46503a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd460072280 con 0x7fd4600cdb20 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd46503a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4600ce4c0 con 0x7fd46015b980 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd46503a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd460071270 con 0x7fd4600cc4f0 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd45ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 0x7fd4600cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd45f59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd46015b980 0x7fd4600cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd45ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 0x7fd4600cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:51030/0 (socket says 172.21.15.148:51030) 2024-04-25T09:57:03.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd45ed9d640 1 -- 172.21.15.148:0/560962084 learned_addr learned my addr 172.21.15.148:0/560962084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.038+0000 7fd45ed9d640 1 -- 172.21.15.148:0/560962084 >> v2:172.21.15.188:3300/0 conn(0x7fd46015b980 msgr2=0x7fd4600cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45ed9d640 1 --2- 172.21.15.148:0/560962084 >> v2:172.21.15.188:3300/0 conn(0x7fd46015b980 0x7fd4600cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45ed9d640 1 -- 172.21.15.148:0/560962084 >> v1:172.21.15.188:6789/0 conn(0x7fd4600cdb20 legacy=0x7fd46015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45ed9d640 1 -- 172.21.15.148:0/560962084 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600714c0 con 0x7fd4600cc4f0 2024-04-25T09:57:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45e59c640 1 -- 172.21.15.148:0/560962084 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776004456 0 0) 0x7fd460072280 con 0x7fd4600cdb20 2024-04-25T09:57:03.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45f59e640 1 --2- 172.21.15.148:0/560962084 >> v2:172.21.15.188:3300/0 conn(0x7fd46015b980 0x7fd4600cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45ed9d640 1 --2- 172.21.15.148:0/560962084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 0x7fd4600cc8d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd450009870 tx=0x7fd45002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3927aafb51f6dd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45e59c640 1 -- 172.21.15.148:0/560962084 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450037070 con 0x7fd4600cc4f0 2024-04-25T09:57:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.039+0000 7fd45e59c640 1 -- 172.21.15.148:0/560962084 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd450032440 con 0x7fd4600cc4f0 2024-04-25T09:57:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.040+0000 7fd45e59c640 1 -- 172.21.15.148:0/560962084 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450032810 con 0x7fd4600cc4f0 2024-04-25T09:57:03.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.040+0000 7fd46503a640 1 -- 172.21.15.148:0/560962084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 msgr2=0x7fd4600cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.040+0000 7fd46503a640 1 --2- 172.21.15.148:0/560962084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 0x7fd4600cc8d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd450009870 tx=0x7fd45002a250 comp rx=0 tx=0).stop 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.041+0000 7fd46503a640 1 -- 172.21.15.148:0/560962084 shutdown_connections 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.041+0000 7fd46503a640 1 --2- 172.21.15.148:0/560962084 >> v2:172.21.15.188:3300/0 conn(0x7fd46015b980 0x7fd4600cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.041+0000 7fd46503a640 1 --2- 172.21.15.148:0/560962084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cc4f0 0x7fd4600cc8d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.041+0000 7fd46503a640 1 -- 172.21.15.148:0/560962084 >> 172.21.15.148:0/560962084 conn(0x7fd46006abe0 msgr2=0x7fd46006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.042+0000 7fd46503a640 1 -- 172.21.15.148:0/560962084 shutdown_connections 2024-04-25T09:57:03.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.042+0000 7fd46503a640 1 -- 172.21.15.148:0/560962084 wait complete. 2024-04-25T09:57:03.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.043+0000 7fd46503a640 1 Processor -- start 2024-04-25T09:57:03.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.044+0000 7fd46503a640 1 -- start start 2024-04-25T09:57:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd46503a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 0x7fd460151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd46503a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cdb20 0x7fd460144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd45ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 0x7fd460151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd45ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 0x7fd460151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44276/0 (socket says 172.21.15.148:44276) 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd45ed9d640 1 -- 172.21.15.148:0/2690552147 learned_addr learned my addr 172.21.15.148:0/2690552147 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd46015b980 0x7fd460151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460071a80 con 0x7fd4600cc4f0 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46015bd90 con 0x7fd46015b980 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.045+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd460071d60 con 0x7fd4600cdb20 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45659c640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cdb20 0x7fd460144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45ed9d640 1 -- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cdb20 msgr2=0x7fd460144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45ed9d640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cdb20 0x7fd460144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45ed9d640 1 -- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd46015b980 msgr2=0x7fd460151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45ed9d640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd46015b980 0x7fd460151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45ed9d640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4601556c0 con 0x7fd4600cc4f0 2024-04-25T09:57:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.046+0000 7fd45ed9d640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 0x7fd460151510 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd450035040 tx=0x7fd4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.047+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450037040 con 0x7fd4600cc4f0 2024-04-25T09:57:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.047+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd450004230 con 0x7fd4600cc4f0 2024-04-25T09:57:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.047+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4601559a0 con 0x7fd4600cc4f0 2024-04-25T09:57:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.047+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd460155f60 con 0x7fd4600cc4f0 2024-04-25T09:57:03.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.047+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450003610 con 0x7fd4600cc4f0 2024-04-25T09:57:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.049+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd444005290 con 0x7fd4600cc4f0 2024-04-25T09:57:03.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.049+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4500037b0 con 0x7fd4600cc4f0 2024-04-25T09:57:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.050+0000 7fd457fff640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd44006b520 0x7fd44006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.050+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd4500aac70 con 0x7fd4600cc4f0 2024-04-25T09:57:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.050+0000 7fd45659c640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd44006b520 0x7fd44006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.051+0000 7fd45659c640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd44006b520 0x7fd44006d9e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd4600cbb60 tx=0x7fd448009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.056+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd450079e60 con 0x7fd4600cc4f0 2024-04-25T09:57:03.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.236+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd444005850 con 0x7fd4600cc4f0 2024-04-25T09:57:03.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.239+0000 7fd457fff640 1 -- 172.21.15.148:0/2690552147 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd450079760 con 0x7fd4600cc4f0 2024-04-25T09:57:03.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd44006b520 msgr2=0x7fd44006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd44006b520 0x7fd44006d9e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd4600cbb60 tx=0x7fd448009040 comp rx=0 tx=0).stop 2024-04-25T09:57:03.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 msgr2=0x7fd460151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 0x7fd460151510 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd450035040 tx=0x7fd4500046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 shutdown_connections 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd44006b520 0x7fd44006d9e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd46015b980 0x7fd460151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4600cdb20 0x7fd460144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 --2- 172.21.15.148:0/2690552147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4600cc4f0 0x7fd460151510 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 >> 172.21.15.148:0/2690552147 conn(0x7fd46006abe0 msgr2=0x7fd4600b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 shutdown_connections 2024-04-25T09:57:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.242+0000 7fd46503a640 1 -- 172.21.15.148:0/2690552147 wait complete. 2024-04-25T09:57:03.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T09:57:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.350+0000 7fc43fcb3640 1 Processor -- start 2024-04-25T09:57:03.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.350+0000 7fc43fcb3640 1 -- start start 2024-04-25T09:57:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43fcb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 0x7fc4380d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43fcb3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4380d1020 0x7fc4380d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43fcb3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4380b97a0 con 0x7fc43815dbe0 2024-04-25T09:57:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43fcb3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43815dfc0 con 0x7fc4380d1020 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43fcb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380d1a00 con 0x7fc4380b2cb0 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43da4f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc43815dbe0 0x7fc4380d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36578/0 (socket says 172.21.15.148:36578) 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.351+0000 7fc43da4f640 1 -- 172.21.15.148:0/2297778190 learned_addr learned my addr 172.21.15.148:0/2297778190 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651460744 0 0) 0x7fc4380b97a0 con 0x7fc43815dbe0 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc424003520 con 0x7fc43815dbe0 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43e250640 1 --2- 172.21.15.148:0/2297778190 >> v2:172.21.15.188:3300/0 conn(0x7fc4380d1020 0x7fc4380d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 0x7fc4380d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 509921844 0 0) 0x7fc424003520 con 0x7fc43815dbe0 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 msgr2=0x7fc4380d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 0x7fc4380d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 >> v2:172.21.15.188:3300/0 conn(0x7fc4380d1020 msgr2=0x7fc4380d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 --2- 172.21.15.148:0/2297778190 >> v2:172.21.15.188:3300/0 conn(0x7fc4380d1020 0x7fc4380d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380d1c50 con 0x7fc43815dbe0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc4340019f0 con 0x7fc43815dbe0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 >> v1:172.21.15.148:6789/0 conn(0x7fc43815dbe0 legacy=0x7fc4380d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 0x7fc4240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380d1a00 con 0x7fc424004800 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42400b9f0 con 0x7fc4240039b0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42400bbf0 con 0x7fc4380b2cb0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43e250640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 0x7fc4240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43da4f640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 -- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 msgr2=0x7fc4240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4380b2cb0 0x7fc4240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 -- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4240039b0 msgr2=0x7fc424003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.352+0000 7fc43d24e640 1 -- 172.21.15.148:0/2297778190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc424003520 con 0x7fc424004800 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.353+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc424004c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc42800ba00 tx=0x7fc424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b1b66973b90ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.353+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc428030070 con 0x7fc424004800 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.353+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42802ad80 con 0x7fc424004800 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.353+0000 7fc43ca4d640 1 -- 172.21.15.148:0/2297778190 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42800b600 con 0x7fc424004800 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2297778190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 msgr2=0x7fc424004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2297778190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc424004c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc42800ba00 tx=0x7fc424005830 comp rx=0 tx=0).stop 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43e250640 1 -- 172.21.15.148:0/2297778190 reap_dead start 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2297778190 shutdown_connections 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2297778190 >> 172.21.15.148:0/2297778190 conn(0x7fc438061a20 msgr2=0x7fc438061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2297778190 shutdown_connections 2024-04-25T09:57:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.354+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2297778190 wait complete. 2024-04-25T09:57:03.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.355+0000 7fc43fcb3640 1 Processor -- start 2024-04-25T09:57:03.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.355+0000 7fc43fcb3640 1 -- start start 2024-04-25T09:57:03.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43fcb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc4381f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43fcb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4381f3a00 0x7fc43815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43fcb3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43815efa0 0x7fc438160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43fcb3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380b92a0 con 0x7fc424004800 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43fcb3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380d2210 con 0x7fc43815efa0 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43fcb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380b9420 con 0x7fc4381f3a00 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc4381f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc4381f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44284/0 (socket says 172.21.15.148:44284) 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 -- 172.21.15.148:0/2788363682 learned_addr learned my addr 172.21.15.148:0/2788363682 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4381f3a00 0x7fc43815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43e250640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43815efa0 0x7fc438160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 -- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4381f3a00 msgr2=0x7fc43815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4381f3a00 0x7fc43815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 -- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43815efa0 msgr2=0x7fc438160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43815efa0 0x7fc438160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 -- 172.21.15.148:0/2788363682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438160f30 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43e250640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43815efa0 0x7fc438160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc43da4f640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc4381f6800 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc434007820 tx=0x7fc43400a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.356+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434017020 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.357+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4340133f0 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.357+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43815f5d0 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.357+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4340137e0 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.357+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4380cae40 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.358+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc438073990 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.358+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc43401a020 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.358+0000 7fc42e7fc640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc41c06b500 0x7fc41c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.359+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc43408aad0 con 0x7fc424004800 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.359+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc41c06b500 0x7fc41c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:03.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.359+0000 7fc43d24e640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc41c06b500 0x7fc41c06d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4240057e0 tx=0x7fc428030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:03.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.363+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc434021020 con 0x7fc424004800 2024-04-25T09:57:03.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.596+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7fc4380d31a0 con 0x7fc424004800 2024-04-25T09:57:03.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.923+0000 7fc42e7fc640 1 -- 172.21.15.148:0/2788363682 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v145) v1 ==== 148+0+0 (secure 0 0 0) 0x7fc434059b50 con 0x7fc424004800 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc41c06b500 msgr2=0x7fc41c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc41c06b500 0x7fc41c06d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc4240057e0 tx=0x7fc428030000 comp rx=0 tx=0).stop 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 msgr2=0x7fc4381f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc4381f6800 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc434007820 tx=0x7fc43400a750 comp rx=0 tx=0).stop 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 shutdown_connections 2024-04-25T09:57:03.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc41c06b500 0x7fc41c06d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43815efa0 0x7fc438160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4381f3a00 0x7fc43815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 --2- 172.21.15.148:0/2788363682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424004800 0x7fc4381f6800 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 >> 172.21.15.148:0/2788363682 conn(0x7fc438061a20 msgr2=0x7fc43815cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 shutdown_connections 2024-04-25T09:57:03.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:03.926+0000 7fc43fcb3640 1 -- 172.21.15.148:0/2788363682 wait complete. 2024-04-25T09:57:03.941 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T09:57:03.941 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:57:03.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.036+0000 7fd42f86f640 1 Processor -- start 2024-04-25T09:57:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.036+0000 7fd42f86f640 1 -- start start 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42f86f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280701e0 0x7fd4280cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42f86f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4280cbbf0 0x7fd4280cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42f86f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42806f6c0 con 0x7fd42815df00 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42f86f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42806f950 con 0x7fd4280cbbf0 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42f86f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4280cdf00 con 0x7fd4280701e0 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42de0c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4280cbbf0 0x7fd4280cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42de0c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd4280cbbf0 0x7fd4280cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46486/0 (socket says 172.21.15.148:46486) 2024-04-25T09:57:04.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.037+0000 7fd42de0c640 1 -- 172.21.15.148:0/49954767 learned_addr learned my addr 172.21.15.148:0/49954767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd42de0c640 1 -- 172.21.15.148:0/49954767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280701e0 msgr2=0x7fd4280cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd42de0c640 1 --2- 172.21.15.148:0/49954767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280701e0 0x7fd4280cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd42de0c640 1 -- 172.21.15.148:0/49954767 >> v1:172.21.15.188:6789/0 conn(0x7fd42815df00 legacy=0x7fd4280cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd42de0c640 1 -- 172.21.15.148:0/49954767 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280cece0 con 0x7fd4280cbbf0 2024-04-25T09:57:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd42de0c640 1 --2- 172.21.15.148:0/49954767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280cbbf0 0x7fd4280cc080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd42000bb10 tx=0x7fd42002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6716ed008bbd92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd41ffff640 1 -- 172.21.15.148:0/49954767 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd420032790 con 0x7fd4280cbbf0 2024-04-25T09:57:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.038+0000 7fd41ffff640 1 -- 172.21.15.148:0/49954767 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd420038070 con 0x7fd4280cbbf0 2024-04-25T09:57:04.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd41ffff640 1 -- 172.21.15.148:0/49954767 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd420032b50 con 0x7fd4280cbbf0 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 -- 172.21.15.148:0/49954767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280cbbf0 msgr2=0x7fd4280cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 --2- 172.21.15.148:0/49954767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280cbbf0 0x7fd4280cc080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd42000bb10 tx=0x7fd42002c8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 -- 172.21.15.148:0/49954767 shutdown_connections 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 --2- 172.21.15.148:0/49954767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280cbbf0 0x7fd4280cc080 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 --2- 172.21.15.148:0/49954767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280701e0 0x7fd4280cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 -- 172.21.15.148:0/49954767 >> 172.21.15.148:0/49954767 conn(0x7fd4280b2290 msgr2=0x7fd4280b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 -- 172.21.15.148:0/49954767 shutdown_connections 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.039+0000 7fd42f86f640 1 -- 172.21.15.148:0/49954767 wait complete. 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.040+0000 7fd42f86f640 1 Processor -- start 2024-04-25T09:57:04.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.040+0000 7fd42f86f640 1 -- start start 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42f86f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 0x7fd4281eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42f86f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280cbbf0 0x7fd428159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42f86f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42815df00 0x7fd42815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42f86f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42806fd00 con 0x7fd4280701e0 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42f86f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4280cf4e0 con 0x7fd42815df00 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42f86f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42806fe80 con 0x7fd4280cbbf0 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 0x7fd4281eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 0x7fd4281eeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46502/0 (socket says 172.21.15.148:46502) 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 -- 172.21.15.148:0/29109523 learned_addr learned my addr 172.21.15.148:0/29109523 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 -- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280cbbf0 msgr2=0x7fd428159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42ce0a640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280cbbf0 0x7fd428159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42de0c640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42815df00 0x7fd42815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280cbbf0 0x7fd428159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 -- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42815df00 msgr2=0x7fd42815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42815df00 0x7fd42815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.041+0000 7fd42d60b640 1 -- 172.21.15.148:0/29109523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4281f2a00 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.042+0000 7fd42d60b640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 0x7fd4281eeb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd42400ba80 tx=0x7fd42400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.042+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400cc00 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.042+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42400cda0 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.042+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4281f2ce0 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.042+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424003bb0 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.042+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4281f32a0 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.043+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd428073990 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.043+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd424014440 con 0x7fd4280701e0 2024-04-25T09:57:04.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.043+0000 7fd41dffb640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.044+0000 7fd42ce0a640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.044+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd424089fd0 con 0x7fd4280701e0 2024-04-25T09:57:04.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.044+0000 7fd42ce0a640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40c06b4d0 0x7fd40c06d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4280c9eb0 tx=0x7fd418008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.048+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd424058fc0 con 0x7fd4280701e0 2024-04-25T09:57:04.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.221+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd42806d370 con 0x7fd4280701e0 2024-04-25T09:57:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.222+0000 7fd41dffb640 1 -- 172.21.15.148:0/29109523 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13343 (secure 0 0 0) 0x7fd4240588c0 con 0x7fd4280701e0 2024-04-25T09:57:04.225 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:04.225 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":46,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:02.914895+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/3412433349":"2024-04-26T09:56:59.906889+0000","172.21.15.148:6834/3412433349":"2024-04-26T09:56:59.906889+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40c06b4d0 msgr2=0x7fd40c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40c06b4d0 0x7fd40c06d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4280c9eb0 tx=0x7fd418008040 comp rx=0 tx=0).stop 2024-04-25T09:57:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 msgr2=0x7fd4281eeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 0x7fd4281eeb90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd42400ba80 tx=0x7fd42400bf40 comp rx=0 tx=0).stop 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 shutdown_connections 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40c06b4d0 0x7fd40c06d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42815df00 0x7fd42815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4280cbbf0 0x7fd428159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 --2- 172.21.15.148:0/29109523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4280701e0 0x7fd4281eeb90 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 >> 172.21.15.148:0/29109523 conn(0x7fd4280b2290 msgr2=0x7fd4280cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 shutdown_connections 2024-04-25T09:57:04.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.224+0000 7fd42f86f640 1 -- 172.21.15.148:0/29109523 wait complete. 2024-04-25T09:57:04.239 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:04.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:04.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.343+0000 7f941e44a640 1 Processor -- start 2024-04-25T09:57:04.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.343+0000 7f941e44a640 1 -- start start 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941e44a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f941815dc20 0x7f94180ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941e44a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9418161710 0x7f94180cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941e44a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941815e000 con 0x7f94180b9330 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941e44a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180d1de0 con 0x7f9418161710 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941e44a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9418161c60 con 0x7f941815dc20 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941c9e7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f94180b9330 0x7f941806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56390/0 (socket says 172.21.15.148:56390) 2024-04-25T09:57:04.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f941c9e7640 1 -- 172.21.15.148:0/2556210817 learned_addr learned my addr 172.21.15.148:0/2556210817 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.344+0000 7f94177fe640 1 --2- 172.21.15.148:0/2556210817 >> v2:172.21.15.148:3300/0 conn(0x7f9418161710 0x7f94180cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f9417fff640 1 --2- 172.21.15.148:0/2556210817 >> v2:172.21.15.188:3300/0 conn(0x7f941815dc20 0x7f94180ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f94177fe640 1 -- 172.21.15.148:0/2556210817 >> v2:172.21.15.188:3300/0 conn(0x7f941815dc20 msgr2=0x7f94180ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f94177fe640 1 --2- 172.21.15.148:0/2556210817 >> v2:172.21.15.188:3300/0 conn(0x7f941815dc20 0x7f94180ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f94177fe640 1 -- 172.21.15.148:0/2556210817 >> v1:172.21.15.148:6789/0 conn(0x7f94180b9330 legacy=0x7f941806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f94177fe640 1 -- 172.21.15.148:0/2556210817 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9418161f40 con 0x7f9418161710 2024-04-25T09:57:04.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f9416ffd640 1 -- 172.21.15.148:0/2556210817 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605829303 0 0) 0x7f941815e000 con 0x7f94180b9330 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.345+0000 7f94177fe640 1 --2- 172.21.15.148:0/2556210817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418161710 0x7f94180cebb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f94180d1520 tx=0x7f940c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bbf9132ecd5e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f9416ffd640 1 -- 172.21.15.148:0/2556210817 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c037070 con 0x7f9418161710 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f9416ffd640 1 -- 172.21.15.148:0/2556210817 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c032440 con 0x7f9418161710 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f9416ffd640 1 -- 172.21.15.148:0/2556210817 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c032810 con 0x7f9418161710 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 -- 172.21.15.148:0/2556210817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418161710 msgr2=0x7f94180cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 --2- 172.21.15.148:0/2556210817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418161710 0x7f94180cebb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f94180d1520 tx=0x7f940c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 -- 172.21.15.148:0/2556210817 shutdown_connections 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 --2- 172.21.15.148:0/2556210817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9418161710 0x7f94180cebb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 --2- 172.21.15.148:0/2556210817 >> v2:172.21.15.188:3300/0 conn(0x7f941815dc20 0x7f94180ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 -- 172.21.15.148:0/2556210817 >> 172.21.15.148:0/2556210817 conn(0x7f94180d0420 msgr2=0x7f94180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 -- 172.21.15.148:0/2556210817 shutdown_connections 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.346+0000 7f941e44a640 1 -- 172.21.15.148:0/2556210817 wait complete. 2024-04-25T09:57:04.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.347+0000 7f941e44a640 1 Processor -- start 2024-04-25T09:57:04.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 -- start start 2024-04-25T09:57:04.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94180b9330 0x7f94181f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 0x7f9418160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9418161710 0x7f94180b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b5e50 con 0x7f941815dc20 2024-04-25T09:57:04.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418162500 con 0x7f94180b9330 2024-04-25T09:57:04.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f941e44a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180b5fd0 con 0x7f9418161710 2024-04-25T09:57:04.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f94177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 0x7f9418160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f94177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 0x7f9418160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46520/0 (socket says 172.21.15.148:46520) 2024-04-25T09:57:04.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.348+0000 7f94177fe640 1 -- 172.21.15.148:0/3625976461 learned_addr learned my addr 172.21.15.148:0/3625976461 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f94177fe640 1 -- 172.21.15.148:0/3625976461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9418161710 msgr2=0x7f94180b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f94177fe640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9418161710 0x7f94180b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f94177fe640 1 -- 172.21.15.148:0/3625976461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94180b9330 msgr2=0x7f94181f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f94177fe640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94180b9330 0x7f94181f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f94177fe640 1 -- 172.21.15.148:0/3625976461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941815fba0 con 0x7f941815dc20 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f94177fe640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 0x7f9418160940 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f940c000c00 tx=0x7f940c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c037040 con 0x7f941815dc20 2024-04-25T09:57:04.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c042070 con 0x7f941815dc20 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c030900 con 0x7f941815dc20 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941815fe80 con 0x7f941815dc20 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.349+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9418160440 con 0x7f941815dc20 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.350+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9418073a00 con 0x7f941815dc20 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.350+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f940c030aa0 con 0x7f941815dc20 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.351+0000 7f9414ff9640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940406b4d0 0x7f940406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.351+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f940c0aa930 con 0x7f941815dc20 2024-04-25T09:57:04.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.351+0000 7f9417fff640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940406b4d0 0x7f940406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.351+0000 7f9417fff640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940406b4d0 0x7f940406d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9408006a90 tx=0x7f9408006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.355+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f940c045360 con 0x7f941815dc20 2024-04-25T09:57:04.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.525+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94180d16b0 con 0x7f941815dc20 2024-04-25T09:57:04.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.526+0000 7f9414ff9640 1 -- 172.21.15.148:0/3625976461 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f940c0799b0 con 0x7f941815dc20 2024-04-25T09:57:04.529 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:57:04.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940406b4d0 msgr2=0x7f940406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940406b4d0 0x7f940406d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9408006a90 tx=0x7f9408006a20 comp rx=0 tx=0).stop 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 msgr2=0x7f9418160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 0x7f9418160940 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f940c000c00 tx=0x7f940c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 shutdown_connections 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940406b4d0 0x7f940406d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9418161710 0x7f94180b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f941815dc20 0x7f9418160940 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 --2- 172.21.15.148:0/3625976461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94180b9330 0x7f94181f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 >> 172.21.15.148:0/3625976461 conn(0x7f94180d0420 msgr2=0x7f94180d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 shutdown_connections 2024-04-25T09:57:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.528+0000 7f941e44a640 1 -- 172.21.15.148:0/3625976461 wait complete. 2024-04-25T09:57:04.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:57:04.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.636+0000 7fec08638640 1 Processor -- start 2024-04-25T09:57:04.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.636+0000 7fec08638640 1 -- start start 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec08638640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec000d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec08638640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec000b2cb0 0x7fec000d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec08638640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec000b97a0 con 0x7fec000b92a0 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec08638640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0015dfc0 con 0x7fec000b2cb0 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec08638640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec000d1a00 con 0x7fec0015dbe0 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec05bd3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec000b2cb0 0x7fec000d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.637+0000 7fec063d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec000d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.638+0000 7fec06bd5640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fec000b92a0 0x7fec000b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:37958/0 (socket says 172.21.15.148:37958) 2024-04-25T09:57:04.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.638+0000 7fec06bd5640 1 -- 172.21.15.148:0/2056255296 learned_addr learned my addr 172.21.15.148:0/2056255296 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.638+0000 7fec063d4640 1 -- 172.21.15.148:0/2056255296 >> v2:172.21.15.188:3300/0 conn(0x7fec000b2cb0 msgr2=0x7fec000d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.638+0000 7fec063d4640 1 --2- 172.21.15.148:0/2056255296 >> v2:172.21.15.188:3300/0 conn(0x7fec000b2cb0 0x7fec000d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.638+0000 7fec063d4640 1 -- 172.21.15.148:0/2056255296 >> v1:172.21.15.188:6789/0 conn(0x7fec000b92a0 legacy=0x7fec000b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.638+0000 7fec063d4640 1 -- 172.21.15.148:0/2056255296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000d1c50 con 0x7fec0015dbe0 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec063d4640 1 --2- 172.21.15.148:0/2056255296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec000d0910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d90 tx=0x7febfc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82bdec64fae055d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec053d2640 1 -- 172.21.15.148:0/2056255296 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc007740 con 0x7fec0015dbe0 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec053d2640 1 -- 172.21.15.148:0/2056255296 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc0078e0 con 0x7fec0015dbe0 2024-04-25T09:57:04.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec053d2640 1 -- 172.21.15.148:0/2056255296 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc007cb0 con 0x7fec0015dbe0 2024-04-25T09:57:04.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 -- 172.21.15.148:0/2056255296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 msgr2=0x7fec000d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 --2- 172.21.15.148:0/2056255296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec000d0910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d90 tx=0x7febfc02a660 comp rx=0 tx=0).stop 2024-04-25T09:57:04.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 -- 172.21.15.148:0/2056255296 shutdown_connections 2024-04-25T09:57:04.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 --2- 172.21.15.148:0/2056255296 >> v2:172.21.15.188:3300/0 conn(0x7fec000b2cb0 0x7fec000d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 --2- 172.21.15.148:0/2056255296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec000d0910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 -- 172.21.15.148:0/2056255296 >> 172.21.15.148:0/2056255296 conn(0x7fec00061a20 msgr2=0x7fec00061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 -- 172.21.15.148:0/2056255296 shutdown_connections 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.639+0000 7fec08638640 1 -- 172.21.15.148:0/2056255296 wait complete. 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 Processor -- start 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 -- start start 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 0x7fec000c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec000b92a0 0x7fec000cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec0015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec000b6ab0 con 0x7fec000b2cb0 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec000d2210 con 0x7fec000b92a0 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec08638640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec000b6c30 con 0x7fec0015dbe0 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec063d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 0x7fec000c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec063d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 0x7fec000c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46536/0 (socket says 172.21.15.148:46536) 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.641+0000 7fec063d4640 1 -- 172.21.15.148:0/2358346614 learned_addr learned my addr 172.21.15.148:0/2358346614 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec06bd5640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec0015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec05bd3640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec000b92a0 0x7fec000cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec063d4640 1 -- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 msgr2=0x7fec0015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec063d4640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec0015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec063d4640 1 -- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec000b92a0 msgr2=0x7fec000cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec063d4640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec000b92a0 0x7fec000cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec063d4640 1 -- 172.21.15.148:0/2358346614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000cafc0 con 0x7fec000b2cb0 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec06bd5640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec0015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec05bd3640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec000b92a0 0x7fec000cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec063d4640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 0x7fec000c9a70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d90 tx=0x7febfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc03f070 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc031c40 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec000cb2a0 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc009450 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.642+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec000cb860 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.643+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec00073990 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.643+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febfc044460 con 0x7fec000b2cb0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.644+0000 7febf6ffd640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febe406b4d0 0x7febe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.644+0000 7fec05bd3640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febe406b4d0 0x7febe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:04.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.644+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7febfc0ab900 con 0x7fec000b2cb0 2024-04-25T09:57:04.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.644+0000 7fec05bd3640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febe406b4d0 0x7febe406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec000b5e30 tx=0x7febf0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:04.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.648+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febfc07aaa0 con 0x7fec000b2cb0 2024-04-25T09:57:04.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.881+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fec000b88d0 con 0x7fec000b2cb0 2024-04-25T09:57:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.925+0000 7febf6ffd640 1 -- 172.21.15.148:0/2358346614 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v146) v1 ==== 146+0+0 (secure 0 0 0) 0x7febfc07a3a0 con 0x7fec000b2cb0 2024-04-25T09:57:04.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febe406b4d0 msgr2=0x7febe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febe406b4d0 0x7febe406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fec000b5e30 tx=0x7febf0008040 comp rx=0 tx=0).stop 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 msgr2=0x7fec000c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 0x7fec000c9a70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7febfc009d90 tx=0x7febfc004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 shutdown_connections 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febe406b4d0 0x7febe406d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec0015dbe0 0x7fec0015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec000b92a0 0x7fec000cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 --2- 172.21.15.148:0/2358346614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec000b2cb0 0x7fec000c9a70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 >> 172.21.15.148:0/2358346614 conn(0x7fec00061a20 msgr2=0x7fec0015b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:04.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 shutdown_connections 2024-04-25T09:57:04.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:04.927+0000 7fec08638640 1 -- 172.21.15.148:0/2358346614 wait complete. 2024-04-25T09:57:04.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:57:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.037+0000 7fe5ca607640 1 Processor -- start 2024-04-25T09:57:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.037+0000 7fe5ca607640 1 -- start start 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5ca607640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5c40d34c0 0x7fe5c40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5ca607640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5c40d08e0 0x7fe5c40b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5ca607640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40d2420 con 0x7fe5c415dac0 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5ca607640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c40d0d50 con 0x7fe5c40d34c0 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5ca607640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c40d3030 con 0x7fe5c40d08e0 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5c37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5c40d34c0 0x7fe5c40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5c37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5c40d34c0 0x7fe5c40d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46552/0 (socket says 172.21.15.148:46552) 2024-04-25T09:57:05.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.038+0000 7fe5c37fe640 1 -- 172.21.15.148:0/57091022 learned_addr learned my addr 172.21.15.148:0/57091022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c8ba4640 1 --2- 172.21.15.148:0/57091022 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40d08e0 0x7fe5c40b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c37fe640 1 -- 172.21.15.148:0/57091022 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40d08e0 msgr2=0x7fe5c40b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c37fe640 1 --2- 172.21.15.148:0/57091022 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40d08e0 0x7fe5c40b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c37fe640 1 -- 172.21.15.148:0/57091022 >> v1:172.21.15.188:6789/0 conn(0x7fe5c415dac0 legacy=0x7fe5c40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c37fe640 1 -- 172.21.15.148:0/57091022 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40d2700 con 0x7fe5c40d34c0 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c8ba4640 1 --2- 172.21.15.148:0/57091022 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40d08e0 0x7fe5c40b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c37fe640 1 --2- 172.21.15.148:0/57091022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40d03a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40d2fa0 tx=0x7fe5b802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28be3e8ba10023ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c2ffd640 1 -- 172.21.15.148:0/57091022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8004030 con 0x7fe5c40d34c0 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c2ffd640 1 -- 172.21.15.148:0/57091022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b80041d0 con 0x7fe5c40d34c0 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.039+0000 7fe5c2ffd640 1 -- 172.21.15.148:0/57091022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8003610 con 0x7fe5c40d34c0 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 -- 172.21.15.148:0/57091022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 msgr2=0x7fe5c40d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 --2- 172.21.15.148:0/57091022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40d03a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40d2fa0 tx=0x7fe5b802a290 comp rx=0 tx=0).stop 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 -- 172.21.15.148:0/57091022 shutdown_connections 2024-04-25T09:57:05.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 --2- 172.21.15.148:0/57091022 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40d08e0 0x7fe5c40b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 --2- 172.21.15.148:0/57091022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40d03a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 -- 172.21.15.148:0/57091022 >> 172.21.15.148:0/57091022 conn(0x7fe5c4061c30 msgr2=0x7fe5c406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 -- 172.21.15.148:0/57091022 shutdown_connections 2024-04-25T09:57:05.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.040+0000 7fe5ca607640 1 -- 172.21.15.148:0/57091022 wait complete. 2024-04-25T09:57:05.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.041+0000 7fe5ca607640 1 Processor -- start 2024-04-25T09:57:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.041+0000 7fe5ca607640 1 -- start start 2024-04-25T09:57:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5ca607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40d08e0 0x7fe5c4149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5ca607640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5ca607640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c415dac0 0x7fe5c40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5ca607640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40d2100 con 0x7fe5c40d34c0 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5ca607640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40d2cc0 con 0x7fe5c415dac0 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5ca607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c415dea0 con 0x7fe5c40d08e0 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46554/0 (socket says 172.21.15.148:46554) 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 -- 172.21.15.148:0/632921683 learned_addr learned my addr 172.21.15.148:0/632921683 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c8ba4640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c415dac0 0x7fe5c40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c3fff640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40d08e0 0x7fe5c4149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 -- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40d08e0 msgr2=0x7fe5c4149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40d08e0 0x7fe5c4149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 -- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c415dac0 msgr2=0x7fe5c40cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c415dac0 0x7fe5c40cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c37fe640 1 -- 172.21.15.148:0/632921683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c41f6c70 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.042+0000 7fe5c8ba4640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c415dac0 0x7fe5c40cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.043+0000 7fe5c37fe640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40cd190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8034040 tx=0x7fe5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.043+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8041030 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.043+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8003c50 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.043+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c41f6f10 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.043+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8004030 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.043+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c41f74d0 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.044+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4073a00 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.044+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5b8044070 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.045+0000 7fe5c0ff9640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5b006b500 0x7fe5b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.045+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe5b80aac70 con 0x7fe5c40d34c0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.045+0000 7fe5c3fff640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5b006b500 0x7fe5b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.045+0000 7fe5c3fff640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5b006b500 0x7fe5b006d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4006fd0 tx=0x7fe5b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.049+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5b8003e20 con 0x7fe5c40d34c0 2024-04-25T09:57:05.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.219+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe5c406b210 con 0x7fe5c40d34c0 2024-04-25T09:57:05.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.222+0000 7fe5c0ff9640 1 -- 172.21.15.148:0/632921683 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5b8079cf0 con 0x7fe5c40d34c0 2024-04-25T09:57:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5b006b500 msgr2=0x7fe5b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5b006b500 0x7fe5b006d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4006fd0 tx=0x7fe5b4008040 comp rx=0 tx=0).stop 2024-04-25T09:57:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 msgr2=0x7fe5c40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40cd190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8034040 tx=0x7fe5b80046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 shutdown_connections 2024-04-25T09:57:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5b006b500 0x7fe5b006d9c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c415dac0 0x7fe5c40cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5c40d34c0 0x7fe5c40cd190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.224+0000 7fe5ca607640 1 --2- 172.21.15.148:0/632921683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40d08e0 0x7fe5c4149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.225+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 >> 172.21.15.148:0/632921683 conn(0x7fe5c4061c30 msgr2=0x7fe5c40b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.225+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 shutdown_connections 2024-04-25T09:57:05.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.225+0000 7fe5ca607640 1 -- 172.21.15.148:0/632921683 wait complete. 2024-04-25T09:57:05.240 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:05.240 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:05.275 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:05.317 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:05.318 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:05.318 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:05.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:05.333 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:57:05.333 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:05.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:05.375 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:05.376 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:05.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:05.391 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:57:05.391 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:05.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:05.433 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:57:05.433 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:05.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.526+0000 7f3f23c5a640 1 Processor -- start 2024-04-25T09:57:05.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.526+0000 7f3f23c5a640 1 -- start start 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f23c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 0x7f3f1c0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f23c5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0d0350 con 0x7f3f1c0d1d90 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f23c5a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0d0630 con 0x7f3f1c15bd30 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f23c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d0910 con 0x7f3f1c0d13f0 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f221f7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3f1c0d1d90 0x7f3f1c0b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56404/0 (socket says 172.21.15.148:56404) 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f221f7640 1 -- 172.21.15.148:0/405286326 learned_addr learned my addr 172.21.15.148:0/405286326 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f211f5640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 0x7f3f1c0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856115097 0 0) 0x7f3f1c0d0350 con 0x7f3f1c0d1d90 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f08003520 con 0x7f3f1c0d1d90 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 238960257 0 0) 0x7f3f08003520 con 0x7f3f1c0d1d90 2024-04-25T09:57:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 msgr2=0x7f3f1c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 0x7f3f1c0d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 >> v1:172.21.15.188:6789/0 conn(0x7f3f1c15bd30 legacy=0x7f3f1c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0d0bf0 con 0x7f3f1c0d1d90 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3f140019f0 con 0x7f3f1c0d1d90 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 >> v1:172.21.15.148:6789/0 conn(0x7f3f1c0d1d90 legacy=0x7f3f1c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 0x7f3f080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f08004d40 0x7f3f080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d0910 con 0x7f3f08003f60 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.527+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0800bb30 con 0x7f3f08004d40 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0800bd30 con 0x7f3f1c0d13f0 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f211f5640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f219f6640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 0x7f3f080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f221f7640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f08004d40 0x7f3f080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f211f5640 1 -- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 msgr2=0x7f3f080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f211f5640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c0d13f0 0x7f3f080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f211f5640 1 -- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f08004d40 msgr2=0x7f3f080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f211f5640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f08004d40 0x7f3f080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.528+0000 7f3f211f5640 1 -- 172.21.15.148:0/405286326 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08003520 con 0x7f3f08003f60 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f211f5640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f08004800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009ac0 tx=0x7f3f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e055b6f2b62883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c036070 con 0x7f3f08003f60 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c031400 con 0x7f3f08003f60 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f209f4640 1 -- 172.21.15.148:0/405286326 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c009690 con 0x7f3f08003f60 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f23c5a640 1 -- 172.21.15.148:0/405286326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 msgr2=0x7f3f08004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/405286326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f08004800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009ac0 tx=0x7f3f08005830 comp rx=0 tx=0).stop 2024-04-25T09:57:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f221f7640 1 -- 172.21.15.148:0/405286326 reap_dead start 2024-04-25T09:57:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f23c5a640 1 -- 172.21.15.148:0/405286326 shutdown_connections 2024-04-25T09:57:05.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f23c5a640 1 -- 172.21.15.148:0/405286326 >> 172.21.15.148:0/405286326 conn(0x7f3f1c06f590 msgr2=0x7f3f1c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f23c5a640 1 -- 172.21.15.148:0/405286326 shutdown_connections 2024-04-25T09:57:05.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.529+0000 7f3f23c5a640 1 -- 172.21.15.148:0/405286326 wait complete. 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 Processor -- start 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 -- start start 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f1c14c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c14c0e0 0x7f3f1c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f1c1f3b10 0x7f3f1c1f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d0010 con 0x7f3f08003f60 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c15c110 con 0x7f3f1c1f3b10 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f23c5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b73b0 con 0x7f3f1c14c0e0 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f219f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f1c14c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f219f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f1c14c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46572/0 (socket says 172.21.15.148:46572) 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.531+0000 7f3f219f6640 1 -- 172.21.15.148:0/869385866 learned_addr learned my addr 172.21.15.148:0/869385866 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f221f7640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f1c1f3b10 0x7f3f1c1f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f219f6640 1 -- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c14c0e0 msgr2=0x7f3f1c14f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f211f5640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c14c0e0 0x7f3f1c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f219f6640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c14c0e0 0x7f3f1c14f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f219f6640 1 -- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f1c1f3b10 msgr2=0x7f3f1c1f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f219f6640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f1c1f3b10 0x7f3f1c1f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f219f6640 1 -- 172.21.15.148:0/869385866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c1f44e0 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f221f7640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f1c1f3b10 0x7f3f1c1f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f219f6640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f1c14c7a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800f880 tx=0x7f3f1800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18001d20 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f18018070 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1800d730 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c14f750 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.532+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c06bf40 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.533+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c0b71a0 con 0x7f3f08003f60 2024-04-25T09:57:05.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.534+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f1800d8d0 con 0x7f3f08003f60 2024-04-25T09:57:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.534+0000 7f3f127fc640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f0006b500 0x7f3f0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.534+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3f18089e20 con 0x7f3f08003f60 2024-04-25T09:57:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.534+0000 7f3f211f5640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f0006b500 0x7f3f0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.534+0000 7f3f211f5640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f0006b500 0x7f3f0006d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c033000 tx=0x7f3f0c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.538+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f1c0b71a0 con 0x7f3f08003f60 2024-04-25T09:57:05.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.743+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f1c15b590 con 0x7f3f08003f60 2024-04-25T09:57:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.744+0000 7f3f127fc640 1 -- 172.21.15.148:0/869385866 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f3f1c15b590 con 0x7f3f08003f60 2024-04-25T09:57:05.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:05.748 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1131173507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1131173507},{"type":"v1","addr":"172.21.15.148:6837","nonce":1131173507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5035,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1416463559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1416463559},{"type":"v1","addr":"172.21.15.188:6834","nonce":1416463559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5038,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2192446660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2192446660},{"type":"v1","addr":"172.21.15.188:6835","nonce":2192446660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5784,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1609237612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1609237612},{"type":"v1","addr":"172.21.15.148:6836","nonce":1609237612}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T09:57:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 19 2024-04-25T09:57:05.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.745+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f0006b500 msgr2=0x7f3f0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.745+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f0006b500 0x7f3f0006d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c033000 tx=0x7f3f0c008030 comp rx=0 tx=0).stop 2024-04-25T09:57:05.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.745+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 msgr2=0x7f3f1c14c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.745+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f1c14c7a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800f880 tx=0x7f3f1800fd40 comp rx=0 tx=0).stop 2024-04-25T09:57:05.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 shutdown_connections 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f0006b500 0x7f3f0006d9c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f1c1f3b10 0x7f3f1c1f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f1c14c0e0 0x7f3f1c14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 --2- 172.21.15.148:0/869385866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f08003f60 0x7f3f1c14c7a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 >> 172.21.15.148:0/869385866 conn(0x7f3f1c06f590 msgr2=0x7f3f1c0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 shutdown_connections 2024-04-25T09:57:05.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.746+0000 7f3f23c5a640 1 -- 172.21.15.148:0/869385866 wait complete. 2024-04-25T09:57:05.761 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:57:05.761 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:57:05.762 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:57:05.762 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:05.762 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:57:05.762 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:57:05.762 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:57:05.762 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:05.763 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:57:05.763 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:57:05.763 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:57:05.763 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:05.763 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:57:05.763 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:57:05.763 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:57:05.764 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:05.771 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:57:05.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:57:05.772 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:57:05.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:57:05.775 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:57:05.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:57:05.776 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:57:05.776 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:57:05.814 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:57:05.815 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:05.815 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:57:05.815 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:05.818 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:57:05.818 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:05.818 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:57:05.819 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:05.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:57:05.889 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:57:05.889 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:57:05.898 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:57:05.898 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T09:57:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.918+0000 7fa23917e640 1 Processor -- start 2024-04-25T09:57:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.918+0000 7fa23917e640 1 -- start start 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa23917e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2341620b0 0x7fa23406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa23917e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2340b5420 0x7fa2340b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa23917e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2340b5950 con 0x7fa2340b6d30 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa23917e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa234162490 con 0x7fa2341620b0 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa23917e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340b5f10 con 0x7fa2340b5420 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2341620b0 0x7fa23406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2341620b0 0x7fa23406c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46604/0 (socket says 172.21.15.148:46604) 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 -- 172.21.15.148:0/2280087194 learned_addr learned my addr 172.21.15.148:0/2280087194 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa22bfff640 1 --2- 172.21.15.148:0/2280087194 >> v2:172.21.15.188:3300/0 conn(0x7fa2340b5420 0x7fa2340b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 -- 172.21.15.148:0/2280087194 >> v2:172.21.15.188:3300/0 conn(0x7fa2340b5420 msgr2=0x7fa2340b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 --2- 172.21.15.148:0/2280087194 >> v2:172.21.15.188:3300/0 conn(0x7fa2340b5420 0x7fa2340b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 -- 172.21.15.148:0/2280087194 >> v1:172.21.15.188:6789/0 conn(0x7fa2340b6d30 legacy=0x7fa2340cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa232d9d640 1 -- 172.21.15.148:0/2280087194 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340b5c30 con 0x7fa2341620b0 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.919+0000 7fa22bfff640 1 --2- 172.21.15.148:0/2280087194 >> v2:172.21.15.188:3300/0 conn(0x7fa2340b5420 0x7fa2340b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:05.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa232d9d640 1 --2- 172.21.15.148:0/2280087194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2341620b0 0x7fa23406c990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa2340b9a90 tx=0x7fa22402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d841e3ba3d7fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23259c640 1 -- 172.21.15.148:0/2280087194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224037070 con 0x7fa2341620b0 2024-04-25T09:57:05.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23259c640 1 -- 172.21.15.148:0/2280087194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224032440 con 0x7fa2341620b0 2024-04-25T09:57:05.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23259c640 1 -- 172.21.15.148:0/2280087194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224032810 con 0x7fa2341620b0 2024-04-25T09:57:05.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23917e640 1 -- 172.21.15.148:0/2280087194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2341620b0 msgr2=0x7fa23406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23917e640 1 --2- 172.21.15.148:0/2280087194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2341620b0 0x7fa23406c990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa2340b9a90 tx=0x7fa22402a230 comp rx=0 tx=0).stop 2024-04-25T09:57:05.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23917e640 1 -- 172.21.15.148:0/2280087194 shutdown_connections 2024-04-25T09:57:05.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23917e640 1 --2- 172.21.15.148:0/2280087194 >> v2:172.21.15.188:3300/0 conn(0x7fa2340b5420 0x7fa2340b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.920+0000 7fa23917e640 1 --2- 172.21.15.148:0/2280087194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2341620b0 0x7fa23406c990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.921+0000 7fa23917e640 1 -- 172.21.15.148:0/2280087194 >> 172.21.15.148:0/2280087194 conn(0x7fa2340d0420 msgr2=0x7fa2340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.921+0000 7fa23917e640 1 -- 172.21.15.148:0/2280087194 shutdown_connections 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.921+0000 7fa23917e640 1 -- 172.21.15.148:0/2280087194 wait complete. 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.922+0000 7fa23917e640 1 Processor -- start 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.922+0000 7fa23917e640 1 -- start start 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 0x7fa2340719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340b6d30 0x7fa2341f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2341620b0 0x7fa234155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23406ced0 con 0x7fa2340b5420 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340d1a80 con 0x7fa2340b6d30 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340ce700 con 0x7fa2341620b0 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 0x7fa2340719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 0x7fa2340719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46620/0 (socket says 172.21.15.148:46620) 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 -- 172.21.15.148:0/2789765973 learned_addr learned my addr 172.21.15.148:0/2789765973 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:05.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 -- 172.21.15.148:0/2789765973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2341620b0 msgr2=0x7fa234155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2341620b0 0x7fa234155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 -- 172.21.15.148:0/2789765973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340b6d30 msgr2=0x7fa2341f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340b6d30 0x7fa2341f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 -- 172.21.15.148:0/2789765973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2341f8520 con 0x7fa2340b5420 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa232d9d640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 0x7fa2340719b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa2240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224041020 con 0x7fa2340b5420 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224030970 con 0x7fa2340b5420 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2341f8800 con 0x7fa2340b5420 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224037040 con 0x7fa2340b5420 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.923+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23406e030 con 0x7fa2340b5420 2024-04-25T09:57:05.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.924+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2341f8d20 con 0x7fa2340b5420 2024-04-25T09:57:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.925+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa224044020 con 0x7fa2340b5420 2024-04-25T09:57:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.925+0000 7fa22b7fe640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21406b4d0 0x7fa21406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.926+0000 7fa22bfff640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21406b4d0 0x7fa21406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.926+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa2240aaa00 con 0x7fa2340b5420 2024-04-25T09:57:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.926+0000 7fa22bfff640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21406b4d0 0x7fa21406d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006fd0 tx=0x7fa21c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:05.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:05.932+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2341f8d20 con 0x7fa2340b5420 2024-04-25T09:57:06.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.118+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa234073a00 con 0x7fa2340b5420 2024-04-25T09:57:06.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.952+0000 7fa22b7fe640 1 -- 172.21.15.148:0/2789765973 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa22404b030 con 0x7fa2340b5420 2024-04-25T09:57:06.956 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21406b4d0 msgr2=0x7fa21406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21406b4d0 0x7fa21406d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006fd0 tx=0x7fa21c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 msgr2=0x7fa2340719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 0x7fa2340719b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa2240046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 shutdown_connections 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21406b4d0 0x7fa21406d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2341620b0 0x7fa234155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340b6d30 0x7fa2341f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 --2- 172.21.15.148:0/2789765973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340b5420 0x7fa2340719b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 >> 172.21.15.148:0/2789765973 conn(0x7fa2340d0420 msgr2=0x7fa234160fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 shutdown_connections 2024-04-25T09:57:06.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:06.954+0000 7fa23917e640 1 -- 172.21.15.148:0/2789765973 wait complete. 2024-04-25T09:57:06.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:57:07.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 Processor -- start 2024-04-25T09:57:07.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 -- start start 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd21c15a790 0x7fd21c0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd21c06fea0 0x7fd21c0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c0b3710 con 0x7fd21c0af9c0 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21c0cf0e0 con 0x7fd21c15a790 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.068+0000 7fd22255c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21c0cf3c0 con 0x7fd21c06fea0 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd21c15a790 0x7fd21c0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd21c15a790 0x7fd21c0d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46634/0 (socket says 172.21.15.148:46634) 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 -- 172.21.15.148:0/4015588235 learned_addr learned my addr 172.21.15.148:0/4015588235 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21b7fe640 1 --2- 172.21.15.148:0/4015588235 >> v2:172.21.15.188:3300/0 conn(0x7fd21c06fea0 0x7fd21c0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 -- 172.21.15.148:0/4015588235 >> v2:172.21.15.188:3300/0 conn(0x7fd21c06fea0 msgr2=0x7fd21c0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 --2- 172.21.15.148:0/4015588235 >> v2:172.21.15.188:3300/0 conn(0x7fd21c06fea0 0x7fd21c0d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 -- 172.21.15.148:0/4015588235 >> v1:172.21.15.188:6789/0 conn(0x7fd21c0af9c0 legacy=0x7fd21c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.069+0000 7fd21bfff640 1 -- 172.21.15.148:0/4015588235 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c0cf6a0 con 0x7fd21c15a790 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd21bfff640 1 --2- 172.21.15.148:0/4015588235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c15a790 0x7fd21c0d0390 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd21c070780 tx=0x7fd20c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c489cd94d40bc483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd21affd640 1 -- 172.21.15.148:0/4015588235 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037070 con 0x7fd21c15a790 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd21affd640 1 -- 172.21.15.148:0/4015588235 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c032440 con 0x7fd21c15a790 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd21affd640 1 -- 172.21.15.148:0/4015588235 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c032810 con 0x7fd21c15a790 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 -- 172.21.15.148:0/4015588235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c15a790 msgr2=0x7fd21c0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 --2- 172.21.15.148:0/4015588235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c15a790 0x7fd21c0d0390 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd21c070780 tx=0x7fd20c02a270 comp rx=0 tx=0).stop 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 -- 172.21.15.148:0/4015588235 shutdown_connections 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 --2- 172.21.15.148:0/4015588235 >> v2:172.21.15.188:3300/0 conn(0x7fd21c06fea0 0x7fd21c0d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 --2- 172.21.15.148:0/4015588235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c15a790 0x7fd21c0d0390 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 -- 172.21.15.148:0/4015588235 >> 172.21.15.148:0/4015588235 conn(0x7fd21c0b2370 msgr2=0x7fd21c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 -- 172.21.15.148:0/4015588235 shutdown_connections 2024-04-25T09:57:07.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.070+0000 7fd22255c640 1 -- 172.21.15.148:0/4015588235 wait complete. 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.071+0000 7fd22255c640 1 Processor -- start 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 -- start start 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 0x7fd21c14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c0af9c0 0x7fd21c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c15a790 0x7fd21c1f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c15ab70 con 0x7fd21c06fea0 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0cf980 con 0x7fd21c0af9c0 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd22255c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c15acf0 con 0x7fd21c15a790 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.072+0000 7fd21bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 0x7fd21c14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 0x7fd21c14eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46648/0 (socket says 172.21.15.148:46648) 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 -- 172.21.15.148:0/1140926482 learned_addr learned my addr 172.21.15.148:0/1140926482 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21b7fe640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c0af9c0 0x7fd21c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 -- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c15a790 msgr2=0x7fd21c1f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd220af9640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c15a790 0x7fd21c1f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c15a790 0x7fd21c1f1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 -- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c0af9c0 msgr2=0x7fd21c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c0af9c0 0x7fd21c154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 -- 172.21.15.148:0/1140926482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c14f110 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd21bfff640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 0x7fd21c14eb90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a2b0 tx=0x7fd20c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037040 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c042070 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c1f1ad0 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c030880 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.073+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c1f2090 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.074+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c073990 con 0x7fd21c06fea0 2024-04-25T09:57:07.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.074+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd20c030a20 con 0x7fd21c06fea0 2024-04-25T09:57:07.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.075+0000 7fd218ff9640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.075+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd20c0aa820 con 0x7fd21c06fea0 2024-04-25T09:57:07.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.075+0000 7fd21b7fe640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.075+0000 7fd21b7fe640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0cd0a0 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.079+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd20c079870 con 0x7fd21c06fea0 2024-04-25T09:57:07.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.314+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd21c06d330 con 0x7fd21c06fea0 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.314+0000 7fd218ff9640 1 -- 172.21.15.148:0/1140926482 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd20c079170 con 0x7fd21c06fea0 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.319 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.320 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.321 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.322 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.323 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T09:57:07.324 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.325 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.326 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T09:57:07.327 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T09:57:07.328 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:57:07.329 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.330 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:07.331 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:07.332 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T09:57:07.332 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:07.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 msgr2=0x7fd20806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0cd0a0 tx=0x7fd210008040 comp rx=0 tx=0).stop 2024-04-25T09:57:07.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 msgr2=0x7fd21c14eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 0x7fd21c14eb90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd20c02a2b0 tx=0x7fd20c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 shutdown_connections 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd20806b4d0 0x7fd20806d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd21c15a790 0x7fd21c1f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd21c0af9c0 0x7fd21c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 --2- 172.21.15.148:0/1140926482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21c06fea0 0x7fd21c14eb90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 >> 172.21.15.148:0/1140926482 conn(0x7fd21c0b2370 msgr2=0x7fd21c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 shutdown_connections 2024-04-25T09:57:07.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.316+0000 7fd22255c640 1 -- 172.21.15.148:0/1140926482 wait complete. 2024-04-25T09:57:07.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:07.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.427+0000 7f109bedb640 1 Processor -- start 2024-04-25T09:57:07.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.428+0000 7f109bedb640 1 -- start start 2024-04-25T09:57:07.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.428+0000 7f109bedb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.428+0000 7f109bedb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f10940b2cb0 0x7f10940d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.428+0000 7f109bedb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10940b97a0 con 0x7f10940b92a0 2024-04-25T09:57:07.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.428+0000 7f109bedb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109415dfc0 con 0x7f10940b2cb0 2024-04-25T09:57:07.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.428+0000 7f109bedb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940d1a00 con 0x7f109415dbe0 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099c77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099476640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f10940b2cb0 0x7f10940d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f109a478640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f10940b92a0 0x7f10940b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:38014/0 (socket says 172.21.15.148:38014) 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f109a478640 1 -- 172.21.15.148:0/956070331 learned_addr learned my addr 172.21.15.148:0/956070331 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099c77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:37410/0 (socket says 172.21.15.148:37410) 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099476640 1 -- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 msgr2=0x7f10940d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099476640 1 --2- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099476640 1 -- 172.21.15.148:0/956070331 >> v1:172.21.15.188:6789/0 conn(0x7f10940b92a0 legacy=0x7f10940b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.429+0000 7f1099476640 1 -- 172.21.15.148:0/956070331 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940d1c50 con 0x7f10940b2cb0 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f1099476640 1 --2- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b2cb0 0x7f10940d0e50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f10880097a0 tx=0x7f108802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3241b194257101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f1098c75640 1 -- 172.21.15.148:0/956070331 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088037070 con 0x7f10940b2cb0 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f1098c75640 1 -- 172.21.15.148:0/956070331 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1088032440 con 0x7f10940b2cb0 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f1098c75640 1 -- 172.21.15.148:0/956070331 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088032810 con 0x7f10940b2cb0 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f109bedb640 1 -- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b2cb0 msgr2=0x7f10940d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f109bedb640 1 --2- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b2cb0 0x7f10940d0e50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f10880097a0 tx=0x7f108802a180 comp rx=0 tx=0).stop 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f109bedb640 1 -- 172.21.15.148:0/956070331 shutdown_connections 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f109bedb640 1 --2- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b2cb0 0x7f10940d0e50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f109bedb640 1 --2- 172.21.15.148:0/956070331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.430+0000 7f109bedb640 1 -- 172.21.15.148:0/956070331 >> 172.21.15.148:0/956070331 conn(0x7f1094061a20 msgr2=0x7f1094061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.431+0000 7f109bedb640 1 -- 172.21.15.148:0/956070331 shutdown_connections 2024-04-25T09:57:07.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.431+0000 7f109bedb640 1 -- 172.21.15.148:0/956070331 wait complete. 2024-04-25T09:57:07.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 Processor -- start 2024-04-25T09:57:07.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 -- start start 2024-04-25T09:57:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 0x7f10940c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b92a0 0x7f109415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940b6ab0 con 0x7f10940b2cb0 2024-04-25T09:57:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940d2210 con 0x7f10940b92a0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.432+0000 7f109bedb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940b6c30 con 0x7f109415dbe0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 0x7f10940c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 0x7f10940c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46654/0 (socket says 172.21.15.148:46654) 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 -- 172.21.15.148:0/2019308601 learned_addr learned my addr 172.21.15.148:0/2019308601 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f109a478640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099476640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b92a0 0x7f109415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 -- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 msgr2=0x7f10940b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 -- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b92a0 msgr2=0x7f109415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b92a0 0x7f109415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 -- 172.21.15.148:0/2019308601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10941534c0 con 0x7f10940b2cb0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f109a478640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099476640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b92a0 0x7f109415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f1099c77640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 0x7f10940c9350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f108c00efc0 tx=0x7f108c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c018020 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c014480 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10940c9890 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c014850 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.433+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109415f3b0 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.434+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1094073990 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.435+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f108c005d10 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.435+0000 7f10927fc640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108006b4d0 0x7f108006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.435+0000 7f1099476640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108006b4d0 0x7f108006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.435+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f108c08abc0 con 0x7f10940b2cb0 2024-04-25T09:57:07.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.436+0000 7f1099476640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108006b4d0 0x7f108006d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1088034000 tx=0x7f108802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.439+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f108c022070 con 0x7f10940b2cb0 2024-04-25T09:57:07.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.613+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10940b87f0 con 0x7f10940b2cb0 2024-04-25T09:57:07.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.614+0000 7f10927fc640 1 -- 172.21.15.148:0/2019308601 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13212 (secure 0 0 0) 0x7f10940c9890 con 0x7f10940b2cb0 2024-04-25T09:57:07.618 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:07.618 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":51,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:06.950399+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:07.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108006b4d0 msgr2=0x7f108006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108006b4d0 0x7f108006d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1088034000 tx=0x7f108802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:07.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 msgr2=0x7f10940c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 0x7f10940c9350 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f108c00efc0 tx=0x7f108c00c490 comp rx=0 tx=0).stop 2024-04-25T09:57:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 shutdown_connections 2024-04-25T09:57:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108006b4d0 0x7f108006d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109415dbe0 0x7f10940b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10940b92a0 0x7f109415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 --2- 172.21.15.148:0/2019308601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10940b2cb0 0x7f10940c9350 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 >> 172.21.15.148:0/2019308601 conn(0x7f1094061a20 msgr2=0x7f10940b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 shutdown_connections 2024-04-25T09:57:07.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.616+0000 7f109bedb640 1 -- 172.21.15.148:0/2019308601 wait complete. 2024-04-25T09:57:07.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:07.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.727+0000 7f9ef3815640 1 Processor -- start 2024-04-25T09:57:07.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.727+0000 7f9ef3815640 1 -- start start 2024-04-25T09:57:07.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef3815640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9eec070dc0 0x7f9eec06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef3815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0711a0 0x7f9eec0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef3815640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec0d1770 con 0x7f9eec0b4830 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef3815640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec1622b0 con 0x7f9eec070dc0 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef3815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0b10f0 con 0x7f9eec0711a0 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef0db0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0711a0 0x7f9eec0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef15b1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9eec070dc0 0x7f9eec06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef1db2640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9eec0b4830 0x7f9eec0b4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:38026/0 (socket says 172.21.15.148:38026) 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef1db2640 1 -- 172.21.15.148:0/1447654501 learned_addr learned my addr 172.21.15.148:0/1447654501 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef0db0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0711a0 0x7f9eec0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:37426/0 (socket says 172.21.15.148:37426) 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef15b1640 1 -- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0711a0 msgr2=0x7f9eec0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef15b1640 1 --2- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0711a0 0x7f9eec0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef15b1640 1 -- 172.21.15.148:0/1447654501 >> v1:172.21.15.188:6789/0 conn(0x7f9eec0b4830 legacy=0x7f9eec0b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.728+0000 7f9ef15b1640 1 -- 172.21.15.148:0/1447654501 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec162740 con 0x7f9eec070dc0 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.729+0000 7f9ef15b1640 1 --2- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec06ccd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80097a0 tx=0x7f9ee802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f320b6d685086907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.729+0000 7f9ee3fff640 1 -- 172.21.15.148:0/1447654501 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8037070 con 0x7f9eec070dc0 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.729+0000 7f9ee3fff640 1 -- 172.21.15.148:0/1447654501 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee802adb0 con 0x7f9eec070dc0 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.729+0000 7f9ee3fff640 1 -- 172.21.15.148:0/1447654501 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8032660 con 0x7f9eec070dc0 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 -- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 msgr2=0x7f9eec06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 --2- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec06ccd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80097a0 tx=0x7f9ee802a060 comp rx=0 tx=0).stop 2024-04-25T09:57:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 -- 172.21.15.148:0/1447654501 shutdown_connections 2024-04-25T09:57:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 --2- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0711a0 0x7f9eec0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 --2- 172.21.15.148:0/1447654501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec06ccd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 -- 172.21.15.148:0/1447654501 >> 172.21.15.148:0/1447654501 conn(0x7f9eec0b3530 msgr2=0x7f9eec0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 -- 172.21.15.148:0/1447654501 shutdown_connections 2024-04-25T09:57:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.730+0000 7f9ef3815640 1 -- 172.21.15.148:0/1447654501 wait complete. 2024-04-25T09:57:07.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.731+0000 7f9ef3815640 1 Processor -- start 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.731+0000 7f9ef3815640 1 -- start start 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef3815640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef3815640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 0x7f9eec158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef3815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0b4830 0x7f9eec158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef3815640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec161f40 con 0x7f9eec0711a0 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef3815640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec161c60 con 0x7f9eec070dc0 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef3815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec071600 con 0x7f9eec0b4830 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 0x7f9eec158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 0x7f9eec158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46668/0 (socket says 172.21.15.148:46668) 2024-04-25T09:57:07.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 -- 172.21.15.148:0/342583535 learned_addr learned my addr 172.21.15.148:0/342583535 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef15b1640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef1db2640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0b4830 0x7f9eec158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 -- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0b4830 msgr2=0x7f9eec158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0b4830 0x7f9eec158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 -- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 msgr2=0x7f9eec153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 -- 172.21.15.148:0/342583535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec1590e0 con 0x7f9eec0711a0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.732+0000 7f9ef0db0640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 0x7f9eec158660 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e9b0 tx=0x7f9edc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.733+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc017070 con 0x7f9eec0711a0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.733+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00c9a0 con 0x7f9eec0711a0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.733+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec0cac50 con 0x7f9eec0711a0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.733+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00cd90 con 0x7f9eec0711a0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.733+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec0cb210 con 0x7f9eec0711a0 2024-04-25T09:57:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.734+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec073a00 con 0x7f9eec0711a0 2024-04-25T09:57:07.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.734+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9edc01a070 con 0x7f9eec0711a0 2024-04-25T09:57:07.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.734+0000 7f9ee1ffb640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ed006b4d0 0x7f9ed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:07.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.735+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9edc08a080 con 0x7f9eec0711a0 2024-04-25T09:57:07.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.735+0000 7f9ef15b1640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ed006b4d0 0x7f9ed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:07.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.735+0000 7f9ef15b1640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ed006b4d0 0x7f9ed006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8034000 tx=0x7f9ee802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:07.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.739+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9edc0590c0 con 0x7f9eec0711a0 2024-04-25T09:57:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.907+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9eec15da00 con 0x7f9eec0711a0 2024-04-25T09:57:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.907+0000 7f9ee1ffb640 1 -- 172.21.15.148:0/342583535 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9edc0589c0 con 0x7f9eec0711a0 2024-04-25T09:57:07.912 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ed006b4d0 msgr2=0x7f9ed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ed006b4d0 0x7f9ed006d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8034000 tx=0x7f9ee802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 msgr2=0x7f9eec158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 0x7f9eec158660 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e9b0 tx=0x7f9edc00ee70 comp rx=0 tx=0).stop 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 shutdown_connections 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ed006b4d0 0x7f9ed006d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eec0b4830 0x7f9eec158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eec0711a0 0x7f9eec158660 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 --2- 172.21.15.148:0/342583535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eec070dc0 0x7f9eec153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 >> 172.21.15.148:0/342583535 conn(0x7f9eec0b3530 msgr2=0x7f9eec15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:07.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 shutdown_connections 2024-04-25T09:57:07.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:07.909+0000 7f9ef3815640 1 -- 172.21.15.148:0/342583535 wait complete. 2024-04-25T09:57:07.925 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:57:08.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.017+0000 7fd36711d640 1 Processor -- start 2024-04-25T09:57:08.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 -- start start 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd36006abe0 0x7fd36006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3600ce330 0x7fd36006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3600700a0 0x7fd36006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3601626e0 con 0x7fd36006abe0 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36006b500 con 0x7fd3600ce330 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd36711d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600cd270 con 0x7fd3600700a0 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.018+0000 7fd364eb9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd36006abe0 0x7fd36006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd36006abe0 0x7fd36006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46670/0 (socket says 172.21.15.148:46670) 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 -- 172.21.15.148:0/959392616 learned_addr learned my addr 172.21.15.148:0/959392616 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:08.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd3656ba640 1 --2- 172.21.15.148:0/959392616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3600700a0 0x7fd36006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd35ffff640 1 --2- 172.21.15.148:0/959392616 >> v2:172.21.15.188:3300/0 conn(0x7fd3600ce330 0x7fd36006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 -- 172.21.15.148:0/959392616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3600700a0 msgr2=0x7fd36006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 --2- 172.21.15.148:0/959392616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3600700a0 0x7fd36006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 -- 172.21.15.148:0/959392616 >> v2:172.21.15.188:3300/0 conn(0x7fd3600ce330 msgr2=0x7fd36006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 --2- 172.21.15.148:0/959392616 >> v2:172.21.15.188:3300/0 conn(0x7fd3600ce330 0x7fd36006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.019+0000 7fd364eb9640 1 -- 172.21.15.148:0/959392616 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600cd550 con 0x7fd36006abe0 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd364eb9640 1 --2- 172.21.15.148:0/959392616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd36006abe0 0x7fd36006afc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd36006bbf0 tx=0x7fd35402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0185809dc4bd78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd35f7fe640 1 -- 172.21.15.148:0/959392616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354037070 con 0x7fd36006abe0 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd35f7fe640 1 -- 172.21.15.148:0/959392616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd354032440 con 0x7fd36006abe0 2024-04-25T09:57:08.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd35f7fe640 1 -- 172.21.15.148:0/959392616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354032810 con 0x7fd36006abe0 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 -- 172.21.15.148:0/959392616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd36006abe0 msgr2=0x7fd36006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 --2- 172.21.15.148:0/959392616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd36006abe0 0x7fd36006afc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd36006bbf0 tx=0x7fd35402a140 comp rx=0 tx=0).stop 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 -- 172.21.15.148:0/959392616 shutdown_connections 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 --2- 172.21.15.148:0/959392616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3600700a0 0x7fd36006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 --2- 172.21.15.148:0/959392616 >> v2:172.21.15.188:3300/0 conn(0x7fd3600ce330 0x7fd36006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 --2- 172.21.15.148:0/959392616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd36006abe0 0x7fd36006afc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 -- 172.21.15.148:0/959392616 >> 172.21.15.148:0/959392616 conn(0x7fd3600b2220 msgr2=0x7fd3600b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 -- 172.21.15.148:0/959392616 shutdown_connections 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.020+0000 7fd36711d640 1 -- 172.21.15.148:0/959392616 wait complete. 2024-04-25T09:57:08.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.021+0000 7fd36711d640 1 Processor -- start 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 -- start start 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd36006abe0 0x7fd36009b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3600700a0 0x7fd36009d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 0x7fd360159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd360161780 con 0x7fd3600ce330 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600cdb10 con 0x7fd3600700a0 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd36711d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006e930 con 0x7fd36006abe0 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd3656ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 0x7fd360159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.022+0000 7fd3656ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 0x7fd360159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46676/0 (socket says 172.21.15.148:46676) 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 -- 172.21.15.148:0/2508610019 learned_addr learned my addr 172.21.15.148:0/2508610019 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 -- 172.21.15.148:0/2508610019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd36006abe0 msgr2=0x7fd36009b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd36006abe0 0x7fd36009b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 -- 172.21.15.148:0/2508610019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3600700a0 msgr2=0x7fd36009d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3600700a0 0x7fd36009d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 -- 172.21.15.148:0/2508610019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36009df60 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd3656ba640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 0x7fd360159c90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd35800eed0 tx=0x7fd35800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358018020 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd358014400 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36009e240 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3580147f0 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.023+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36015f390 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.024+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd360073990 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.024+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd358005d10 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.025+0000 7fd35d7fa640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd34406b500 0x7fd34406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.025+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd35808a600 con 0x7fd3600ce330 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.025+0000 7fd364eb9640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd34406b500 0x7fd34406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.025+0000 7fd364eb9640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd34406b500 0x7fd34406d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd35402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.029+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd358022070 con 0x7fd3600ce330 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.200+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd3600afe50 con 0x7fd3600ce330 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.201+0000 7fd35d7fa640 1 -- 172.21.15.148:0/2508610019 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd3580596d0 con 0x7fd3600ce330 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:08.205 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.202+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd34406b500 msgr2=0x7fd34406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.202+0000 7fd36711d640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd34406b500 0x7fd34406d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd35402c000 comp rx=0 tx=0).stop 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.202+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 msgr2=0x7fd360159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.202+0000 7fd36711d640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 0x7fd360159c90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd35800eed0 tx=0x7fd35800b480 comp rx=0 tx=0).stop 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 shutdown_connections 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd34406b500 0x7fd34406d9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3600ce330 0x7fd360159c90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3600700a0 0x7fd36009d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 --2- 172.21.15.148:0/2508610019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd36006abe0 0x7fd36009b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 >> 172.21.15.148:0/2508610019 conn(0x7fd3600b2220 msgr2=0x7fd3600b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 shutdown_connections 2024-04-25T09:57:08.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.203+0000 7fd36711d640 1 -- 172.21.15.148:0/2508610019 wait complete. 2024-04-25T09:57:08.218 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:57:08.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:57:08.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.314+0000 7f7b1fd49640 1 Processor -- start 2024-04-25T09:57:08.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.314+0000 7f7b1fd49640 1 -- start start 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1fd49640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 0x7f7b18162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1fd49640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1815e3a0 con 0x7f7b180d0de0 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1fd49640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b180cdc70 con 0x7f7b180cd710 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1fd49640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180cdf50 con 0x7f7b180d0a00 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1d2e4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7b180d0de0 0x7f7b180cd000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56432/0 (socket says 172.21.15.148:56432) 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1d2e4640 1 -- 172.21.15.148:0/2232629239 learned_addr learned my addr 172.21.15.148:0/2232629239 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1dae5640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 0x7f7b18162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189372308 0 0) 0x7f7b1815e3a0 con 0x7f7b180d0de0 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.315+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b04003520 con 0x7f7b180d0de0 2024-04-25T09:57:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338095590 0 0) 0x7f7b04003520 con 0x7f7b180d0de0 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 msgr2=0x7f7b18162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 0x7f7b18162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 >> v1:172.21.15.188:6789/0 conn(0x7f7b180cd710 legacy=0x7f7b180caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18161660 con 0x7f7b180d0de0 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7b08001670 con 0x7f7b180d0de0 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 >> v1:172.21.15.148:6789/0 conn(0x7f7b180d0de0 legacy=0x7f7b180cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040039e0 0x7f7b04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 0x7f7b04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b04004d40 0x7f7b040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180cdf50 con 0x7f7b04004d40 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0400baf0 con 0x7f7b040039e0 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0400bcf0 con 0x7f7b180d0a00 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b04004d40 0x7f7b040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1dae5640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040039e0 0x7f7b04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1d2e4640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 0x7f7b04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 msgr2=0x7f7b04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b180d0a00 0x7f7b04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040039e0 msgr2=0x7f7b04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b040039e0 0x7f7b04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.316+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/2232629239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04003520 con 0x7f7b04004d40 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b04004d40 0x7f7b040051c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001720 tx=0x7f7b04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eecb3895075d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10001370 con 0x7f7b04004d40 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b100052d0 con 0x7f7b04004d40 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1cae3640 1 -- 172.21.15.148:0/2232629239 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1002abc0 con 0x7f7b04004d40 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1fd49640 1 -- 172.21.15.148:0/2232629239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b04004d40 msgr2=0x7f7b040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/2232629239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b04004d40 0x7f7b040051c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001720 tx=0x7f7b04005830 comp rx=0 tx=0).stop 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.317+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/2232629239 reap_dead start 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.318+0000 7f7b1fd49640 1 -- 172.21.15.148:0/2232629239 shutdown_connections 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.318+0000 7f7b1fd49640 1 -- 172.21.15.148:0/2232629239 >> 172.21.15.148:0/2232629239 conn(0x7f7b180d17c0 msgr2=0x7f7b180d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.318+0000 7f7b1fd49640 1 -- 172.21.15.148:0/2232629239 shutdown_connections 2024-04-25T09:57:08.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.318+0000 7f7b1fd49640 1 -- 172.21.15.148:0/2232629239 wait complete. 2024-04-25T09:57:08.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 Processor -- start 2024-04-25T09:57:08.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 -- start start 2024-04-25T09:57:08.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1815d2a0 0x7f7b181f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b181f50b0 0x7f7b180b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 0x7f7b181f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b180d1260 con 0x7f7b180b24f0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18161c20 con 0x7f7b181f50b0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.319+0000 7f7b1fd49640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b181475d0 con 0x7f7b1815d2a0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 0x7f7b181f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 0x7f7b181f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46704/0 (socket says 172.21.15.148:46704) 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/1976959751 learned_addr learned my addr 172.21.15.148:0/1976959751 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1dae5640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1815d2a0 0x7f7b181f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1d2e4640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b181f50b0 0x7f7b180b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1815d2a0 msgr2=0x7f7b181f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1815d2a0 0x7f7b181f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b181f50b0 msgr2=0x7f7b180b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b181f50b0 0x7f7b180b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 -- 172.21.15.148:0/1976959751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b180b0f10 con 0x7f7b180b24f0 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1dae5640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1815d2a0 0x7f7b181f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1d2e4640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b181f50b0 0x7f7b180b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1e2e6640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 0x7f7b181f6ff0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001810 tx=0x7f7b10001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10001370 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1002be50 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b180b11f0 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10007a30 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.320+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b180b17b0 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.321+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b18073a00 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.322+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b10043020 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.322+0000 7f7b0e7fc640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.322+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7b100a8fa0 con 0x7f7b180b24f0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.322+0000 7f7b1dae5640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.322+0000 7f7b1dae5640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7afc06b4d0 0x7f7afc06d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7b140074d0 tx=0x7f7b1400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:08.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.326+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b10077fe0 con 0x7f7b180b24f0 2024-04-25T09:57:08.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.496+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7b18071d60 con 0x7f7b180b24f0 2024-04-25T09:57:08.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.964+0000 7f7b0e7fc640 1 -- 172.21.15.148:0/1976959751 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7b100778e0 con 0x7f7b180b24f0 2024-04-25T09:57:08.969 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.966+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7afc06b4d0 msgr2=0x7f7afc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.966+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7afc06b4d0 0x7f7afc06d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7b140074d0 tx=0x7f7b1400b040 comp rx=0 tx=0).stop 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.966+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 msgr2=0x7f7b181f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.966+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 0x7f7b181f6ff0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7b10001810 tx=0x7f7b10001120 comp rx=0 tx=0).stop 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 shutdown_connections 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b180b24f0 0x7f7b181f6ff0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b181f50b0 0x7f7b180b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 --2- 172.21.15.148:0/1976959751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1815d2a0 0x7f7b181f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 >> 172.21.15.148:0/1976959751 conn(0x7f7b180d17c0 msgr2=0x7f7b1815f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 shutdown_connections 2024-04-25T09:57:08.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:08.967+0000 7f7b1fd49640 1 -- 172.21.15.148:0/1976959751 wait complete. 2024-04-25T09:57:08.986 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:57:09.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.088+0000 7f21965d5640 1 Processor -- start 2024-04-25T09:57:09.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.088+0000 7f21965d5640 1 -- start start 2024-04-25T09:57:09.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f21965d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21900cd8a0 0x7f219015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f21965d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2190150b90 0x7f2190151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f21965d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219015e6e0 con 0x7f21900cd4c0 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f21965d5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2190151560 con 0x7f21900cd8a0 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f21965d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219015e9c0 con 0x7f2190150b90 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f218f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21900cd8a0 0x7f219015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f218f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21900cd8a0 0x7f219015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46716/0 (socket says 172.21.15.148:46716) 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.089+0000 7f218f7fe640 1 -- 172.21.15.148:0/4003994135 learned_addr learned my addr 172.21.15.148:0/4003994135 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218f7fe640 1 -- 172.21.15.148:0/4003994135 >> v2:172.21.15.188:3300/0 conn(0x7f2190150b90 msgr2=0x7f2190151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218f7fe640 1 --2- 172.21.15.148:0/4003994135 >> v2:172.21.15.188:3300/0 conn(0x7f2190150b90 0x7f2190151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218f7fe640 1 -- 172.21.15.148:0/4003994135 >> v1:172.21.15.188:6789/0 conn(0x7f21900cd4c0 legacy=0x7f219015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218f7fe640 1 -- 172.21.15.148:0/4003994135 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219015a9c0 con 0x7f21900cd8a0 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218f7fe640 1 --2- 172.21.15.148:0/4003994135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd8a0 0x7f219015b1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f21900cec10 tx=0x7f218402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20adf69bf77b8044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218effd640 1 -- 172.21.15.148:0/4003994135 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184037070 con 0x7f21900cd8a0 2024-04-25T09:57:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218effd640 1 -- 172.21.15.148:0/4003994135 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2184032440 con 0x7f21900cd8a0 2024-04-25T09:57:09.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.090+0000 7f218effd640 1 -- 172.21.15.148:0/4003994135 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2184032810 con 0x7f21900cd8a0 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 -- 172.21.15.148:0/4003994135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd8a0 msgr2=0x7f219015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 --2- 172.21.15.148:0/4003994135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd8a0 0x7f219015b1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f21900cec10 tx=0x7f218402a230 comp rx=0 tx=0).stop 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 -- 172.21.15.148:0/4003994135 shutdown_connections 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 --2- 172.21.15.148:0/4003994135 >> v2:172.21.15.188:3300/0 conn(0x7f2190150b90 0x7f2190151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 --2- 172.21.15.148:0/4003994135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd8a0 0x7f219015b1c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 -- 172.21.15.148:0/4003994135 >> 172.21.15.148:0/4003994135 conn(0x7f219006abe0 msgr2=0x7f219006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 -- 172.21.15.148:0/4003994135 shutdown_connections 2024-04-25T09:57:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.091+0000 7f21965d5640 1 -- 172.21.15.148:0/4003994135 wait complete. 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.092+0000 7f21965d5640 1 Processor -- start 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.092+0000 7f21965d5640 1 -- start start 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f21965d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 0x7f21901459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f21965d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900cd8a0 0x7f219014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f21965d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2190150b90 0x7f219015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f21965d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219015e370 con 0x7f21900cd4c0 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f21965d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219015a670 con 0x7f2190150b90 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f21965d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219006eca0 con 0x7f21900cd8a0 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 0x7f21901459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 0x7f21901459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46726/0 (socket says 172.21.15.148:46726) 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 -- 172.21.15.148:0/2479186370 learned_addr learned my addr 172.21.15.148:0/2479186370 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218f7fe640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900cd8a0 0x7f219014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f2194b72640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2190150b90 0x7f219015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 -- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900cd8a0 msgr2=0x7f219014b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900cd8a0 0x7f219014b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 -- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2190150b90 msgr2=0x7f219015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2190150b90 0x7f219015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f218ffff640 1 -- 172.21.15.148:0/2479186370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2190156610 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.093+0000 7f2194b72640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2190150b90 0x7f219015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.094+0000 7f218ffff640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 0x7f21901459b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f218000c440 tx=0x7f218000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.094+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2180017070 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.094+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21800035f0 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.094+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21901568f0 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.094+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21800039e0 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.094+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2190156270 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.095+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2190073a00 con 0x7f21900cd4c0 2024-04-25T09:57:09.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.095+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f218001a070 con 0x7f21900cd4c0 2024-04-25T09:57:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.095+0000 7f218cff9640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f217c06b4d0 0x7f217c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.096+0000 7f218f7fe640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f217c06b4d0 0x7f217c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.096+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f2180089410 con 0x7f21900cd4c0 2024-04-25T09:57:09.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.096+0000 7f218f7fe640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f217c06b4d0 0x7f217c06d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2184034000 tx=0x7f218402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.100+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21800010c0 con 0x7f21900cd4c0 2024-04-25T09:57:09.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.269+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f21900ccb10 con 0x7f21900cd4c0 2024-04-25T09:57:09.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.970+0000 7f218cff9640 1 -- 172.21.15.148:0/2479186370 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2180057d00 con 0x7f21900cd4c0 2024-04-25T09:57:09.975 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T09:57:09.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f217c06b4d0 msgr2=0x7f217c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f217c06b4d0 0x7f217c06d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2184034000 tx=0x7f218402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:09.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 msgr2=0x7f21901459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:09.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 0x7f21901459b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f218000c440 tx=0x7f218000c900 comp rx=0 tx=0).stop 2024-04-25T09:57:09.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 shutdown_connections 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f217c06b4d0 0x7f217c06d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2190150b90 0x7f219015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21900cd8a0 0x7f219014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 --2- 172.21.15.148:0/2479186370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21900cd4c0 0x7f21901459b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 >> 172.21.15.148:0/2479186370 conn(0x7f219006abe0 msgr2=0x7f219006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.973+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 shutdown_connections 2024-04-25T09:57:09.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:09.974+0000 7f21965d5640 1 -- 172.21.15.148:0/2479186370 wait complete. 2024-04-25T09:57:10.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:57:10.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5fbb7e640 1 Processor -- start 2024-04-25T09:57:10.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5fbb7e640 1 -- start start 2024-04-25T09:57:10.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5fbb7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 0x7fb5f406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5fbb7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40b4d80 con 0x7fb5f406fa00 2024-04-25T09:57:10.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5fbb7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40cef70 con 0x7fb5f40ce900 2024-04-25T09:57:10.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5fbb7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40b5390 con 0x7fb5f4155f80 2024-04-25T09:57:10.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f9119640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb5f406fa00 0x7fb5f40ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56438/0 (socket says 172.21.15.148:56438) 2024-04-25T09:57:10.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f9119640 1 -- 172.21.15.148:0/311294199 learned_addr learned my addr 172.21.15.148:0/311294199 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960372569 0 0) 0x7fb5f40b4d80 con 0x7fb5f406fa00 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e0003520 con 0x7fb5f406fa00 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f991a640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 0x7fb5f406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348203845 0 0) 0x7fb5e0003520 con 0x7fb5f406fa00 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 msgr2=0x7fb5f406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 0x7fb5f406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 >> v1:172.21.15.188:6789/0 conn(0x7fb5f40ce900 legacy=0x7fb5f40b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40b5020 con 0x7fb5f406fa00 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb5e40016c0 con 0x7fb5f406fa00 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.106+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 >> v1:172.21.15.148:6789/0 conn(0x7fb5f406fa00 legacy=0x7fb5f40ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f8918640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 0x7fb5e0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f8918640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e0003f80 0x7fb5e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f8918640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40b5390 con 0x7fb5e0004d40 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e000bad0 con 0x7fb5e0003f80 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e000bcd0 con 0x7fb5f4155f80 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5fa11b640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f991a640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 0x7fb5e0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5f9119640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e0003f80 0x7fb5e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5fa11b640 1 -- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 msgr2=0x7fb5e0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5fa11b640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4155f80 0x7fb5e0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5fa11b640 1 -- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e0003f80 msgr2=0x7fb5e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5fa11b640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e0003f80 0x7fb5e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.107+0000 7fb5fa11b640 1 -- 172.21.15.148:0/311294199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0003520 con 0x7fb5e0004d40 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fa11b640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5e00051c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec001e20 tx=0x7fb5e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62458bf57ebac283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec001a70 con 0x7fb5e0004d40 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec02ae40 con 0x7fb5e0004d40 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5f8918640 1 -- 172.21.15.148:0/311294199 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec003bb0 con 0x7fb5e0004d40 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/311294199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 msgr2=0x7fb5e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/311294199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5e00051c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec001e20 tx=0x7fb5e0005830 comp rx=0 tx=0).stop 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fa11b640 1 -- 172.21.15.148:0/311294199 reap_dead start 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/311294199 shutdown_connections 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/311294199 >> 172.21.15.148:0/311294199 conn(0x7fb5f4061600 msgr2=0x7fb5f40619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/311294199 shutdown_connections 2024-04-25T09:57:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.108+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/311294199 wait complete. 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 Processor -- start 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 -- start start 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5f413fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4140580 0x7fb5f41510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f4151620 0x7fb5f40b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40cf250 con 0x7fb5e0004d40 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40d07f0 con 0x7fb5f4151620 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fbb7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40cf3d0 con 0x7fb5f4140580 2024-04-25T09:57:10.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5f991a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5f413fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5f991a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5f413fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46748/0 (socket says 172.21.15.148:46748) 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5f991a640 1 -- 172.21.15.148:0/2072155790 learned_addr learned my addr 172.21.15.148:0/2072155790 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5f9119640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4140580 0x7fb5f41510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.110+0000 7fb5fa11b640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f4151620 0x7fb5f40b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f991a640 1 -- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4140580 msgr2=0x7fb5f41510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f991a640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4140580 0x7fb5f41510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f991a640 1 -- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f4151620 msgr2=0x7fb5f40b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f991a640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f4151620 0x7fb5f40b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f991a640 1 -- 172.21.15.148:0/2072155790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40b2340 con 0x7fb5e0004d40 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5fa11b640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f4151620 0x7fb5f40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f9119640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4140580 0x7fb5f41510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5f991a640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5f413fe90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00119a0 tx=0x7fb5f0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000faf0 con 0x7fb5e0004d40 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000fc90 con 0x7fb5e0004d40 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0014440 con 0x7fb5e0004d40 2024-04-25T09:57:10.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f40b2620 con 0x7fb5e0004d40 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.111+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f40b2be0 con 0x7fb5e0004d40 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.112+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f40b1030 con 0x7fb5e0004d40 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.112+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5f0012040 con 0x7fb5e0004d40 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.113+0000 7fb5ea7fc640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5d806b500 0x7fb5d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.113+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb5f008a9b0 con 0x7fb5e0004d40 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.113+0000 7fb5f9119640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5d806b500 0x7fb5d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.113+0000 7fb5f9119640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5d806b500 0x7fb5d806d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4154fe0 tx=0x7fb5e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:10.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.118+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5f0019030 con 0x7fb5e0004d40 2024-04-25T09:57:10.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.322+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb5f4067df0 con 0x7fb5e0004d40 2024-04-25T09:57:10.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.973+0000 7fb5ea7fc640 1 -- 172.21.15.148:0/2072155790 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb5f00590b0 con 0x7fb5e0004d40 2024-04-25T09:57:10.976 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T09:57:10.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5d806b500 msgr2=0x7fb5d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5d806b500 0x7fb5d806d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4154fe0 tx=0x7fb5e4009040 comp rx=0 tx=0).stop 2024-04-25T09:57:10.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 msgr2=0x7fb5f413fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5f413fe90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00119a0 tx=0x7fb5f0011e60 comp rx=0 tx=0).stop 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 shutdown_connections 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5d806b500 0x7fb5d806d9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f4151620 0x7fb5f40b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f4140580 0x7fb5f41510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 --2- 172.21.15.148:0/2072155790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5e0004d40 0x7fb5f413fe90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 >> 172.21.15.148:0/2072155790 conn(0x7fb5f4061600 msgr2=0x7fb5f415dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 shutdown_connections 2024-04-25T09:57:10.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:10.976+0000 7fb5fbb7e640 1 -- 172.21.15.148:0/2072155790 wait complete. 2024-04-25T09:57:11.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.106+0000 7fe9e499e640 1 Processor -- start 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.106+0000 7fe9e499e640 1 -- start start 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9e499e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9e015a790 0x7fe9e00d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9e499e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe9e006fea0 0x7fe9e00d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9e499e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e00b3710 con 0x7fe9e00af9c0 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9e499e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e00cf0e0 con 0x7fe9e015a790 2024-04-25T09:57:11.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9e499e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e00cf3c0 con 0x7fe9e006fea0 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9de59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9e015a790 0x7fe9e00d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9ded9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe9e00af9c0 0x7fe9e0072300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56440/0 (socket says 172.21.15.148:56440) 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9ded9d640 1 -- 172.21.15.148:0/574815711 learned_addr learned my addr 172.21.15.148:0/574815711 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.107+0000 7fe9de59c640 1 --2- 172.21.15.148:0/574815711 >> v2:172.21.15.148:3300/0 conn(0x7fe9e015a790 0x7fe9e00d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46756/0 (socket says 172.21.15.148:46756) 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9ddd9b640 1 --2- 172.21.15.148:0/574815711 >> v2:172.21.15.188:3300/0 conn(0x7fe9e006fea0 0x7fe9e00d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9de59c640 1 -- 172.21.15.148:0/574815711 >> v2:172.21.15.188:3300/0 conn(0x7fe9e006fea0 msgr2=0x7fe9e00d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9de59c640 1 --2- 172.21.15.148:0/574815711 >> v2:172.21.15.188:3300/0 conn(0x7fe9e006fea0 0x7fe9e00d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9de59c640 1 -- 172.21.15.148:0/574815711 >> v1:172.21.15.148:6789/0 conn(0x7fe9e00af9c0 legacy=0x7fe9e0072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9de59c640 1 -- 172.21.15.148:0/574815711 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00cf6a0 con 0x7fe9e015a790 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9de59c640 1 --2- 172.21.15.148:0/574815711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e00d0390 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0070780 tx=0x7fe9d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c6008bf584205a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.108+0000 7fe9dd59a640 1 -- 172.21.15.148:0/574815711 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0007740 con 0x7fe9e015a790 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9dd59a640 1 -- 172.21.15.148:0/574815711 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d00078e0 con 0x7fe9e015a790 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9dd59a640 1 -- 172.21.15.148:0/574815711 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0007cb0 con 0x7fe9e015a790 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 -- 172.21.15.148:0/574815711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 msgr2=0x7fe9e00d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 --2- 172.21.15.148:0/574815711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e00d0390 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0070780 tx=0x7fe9d002a680 comp rx=0 tx=0).stop 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 -- 172.21.15.148:0/574815711 shutdown_connections 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 --2- 172.21.15.148:0/574815711 >> v2:172.21.15.188:3300/0 conn(0x7fe9e006fea0 0x7fe9e00d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 --2- 172.21.15.148:0/574815711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e00d0390 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 -- 172.21.15.148:0/574815711 >> 172.21.15.148:0/574815711 conn(0x7fe9e00b2370 msgr2=0x7fe9e00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 -- 172.21.15.148:0/574815711 shutdown_connections 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.109+0000 7fe9e499e640 1 -- 172.21.15.148:0/574815711 wait complete. 2024-04-25T09:57:11.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.110+0000 7fe9e499e640 1 Processor -- start 2024-04-25T09:57:11.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 -- start start 2024-04-25T09:57:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9e006fea0 0x7fe9e01f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9e00af9c0 0x7fe9e015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e01e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e015ab70 con 0x7fe9e015a790 2024-04-25T09:57:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e00cf980 con 0x7fe9e006fea0 2024-04-25T09:57:11.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9e499e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e015acf0 con 0x7fe9e00af9c0 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e01e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e01e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46764/0 (socket says 172.21.15.148:46764) 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.111+0000 7fe9ded9d640 1 -- 172.21.15.148:0/3725659710 learned_addr learned my addr 172.21.15.148:0/3725659710 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ddd9b640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9e00af9c0 0x7fe9e015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9de59c640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9e006fea0 0x7fe9e01f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ded9d640 1 -- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9e00af9c0 msgr2=0x7fe9e015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ded9d640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9e00af9c0 0x7fe9e015c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ded9d640 1 -- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9e006fea0 msgr2=0x7fe9e01f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ded9d640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9e006fea0 0x7fe9e01f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ded9d640 1 -- 172.21.15.148:0/3725659710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e015d350 con 0x7fe9e015a790 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9de59c640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9e006fea0 0x7fe9e01f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ded9d640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e01e2ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800c3c0 tx=0x7fe9d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8017070 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d8005ce0 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e015d630 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8007310 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.112+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e013f2b0 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.113+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e0073990 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.113+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9d801a070 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.114+0000 7fe9ceffd640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe9c006b4d0 0x7fe9c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.114+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9d80896a0 con 0x7fe9e015a790 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.114+0000 7fe9de59c640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe9c006b4d0 0x7fe9c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.114+0000 7fe9de59c640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe9c006b4d0 0x7fe9c006d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0034000 tx=0x7fe9d002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.118+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9d8058390 con 0x7fe9e015a790 2024-04-25T09:57:11.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.290+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9e00b05c0 con 0x7fe9e015a790 2024-04-25T09:57:11.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.291+0000 7fe9ceffd640 1 -- 172.21.15.148:0/3725659710 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17003 (secure 0 0 0) 0x7fe9d8057c90 con 0x7fe9e015a790 2024-04-25T09:57:11.295 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:11.295 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":54,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:10.325125+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe9c006b4d0 msgr2=0x7fe9c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe9c006b4d0 0x7fe9c006d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0034000 tx=0x7fe9d002f000 comp rx=0 tx=0).stop 2024-04-25T09:57:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 msgr2=0x7fe9e01e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e01e2ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800c3c0 tx=0x7fe9d800c880 comp rx=0 tx=0).stop 2024-04-25T09:57:11.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 shutdown_connections 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe9c006b4d0 0x7fe9c006d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9e015a790 0x7fe9e01e2ce0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9e00af9c0 0x7fe9e015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 --2- 172.21.15.148:0/3725659710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9e006fea0 0x7fe9e01f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 >> 172.21.15.148:0/3725659710 conn(0x7fe9e00b2370 msgr2=0x7fe9e00b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.293+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 shutdown_connections 2024-04-25T09:57:11.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.294+0000 7fe9e499e640 1 -- 172.21.15.148:0/3725659710 wait complete. 2024-04-25T09:57:11.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:11.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.402+0000 7f38e66dd640 1 Processor -- start 2024-04-25T09:57:11.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.402+0000 7f38e66dd640 1 -- start start 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e66dd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38e00b5c30 0x7f38e0168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e66dd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e00ceee0 con 0x7f38e015db70 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e66dd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e00d1d10 con 0x7f38e00b6cb0 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e66dd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e00d3800 con 0x7f38e00b5c30 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e4c7a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38e00b5c30 0x7f38e0168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e4c7a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38e00b5c30 0x7f38e0168ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46770/0 (socket says 172.21.15.148:46770) 2024-04-25T09:57:11.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e4c7a640 1 -- 172.21.15.148:0/3492868241 learned_addr learned my addr 172.21.15.148:0/3492868241 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e4c7a640 1 -- 172.21.15.148:0/3492868241 >> v1:172.21.15.188:6789/0 conn(0x7f38e00b6cb0 legacy=0x7f38e00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e4c7a640 1 -- 172.21.15.148:0/3492868241 >> v1:172.21.15.148:6789/0 conn(0x7f38e015db70 legacy=0x7f38e0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38e4c7a640 1 -- 172.21.15.148:0/3492868241 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00d1ff0 con 0x7f38e00b5c30 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.403+0000 7f38deffd640 1 -- 172.21.15.148:0/3492868241 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696060730 0 0) 0x7f38e00ceee0 con 0x7f38e015db70 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.404+0000 7f38e4c7a640 1 --2- 172.21.15.148:0/3492868241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e00b5c30 0x7f38e0168ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f38e00d3980 tx=0x7f38d802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb602ecae1c23512 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.404+0000 7f38deffd640 1 -- 172.21.15.148:0/3492868241 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8036070 con 0x7f38e00b5c30 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.404+0000 7f38deffd640 1 -- 172.21.15.148:0/3492868241 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8031600 con 0x7f38e00b5c30 2024-04-25T09:57:11.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.404+0000 7f38deffd640 1 -- 172.21.15.148:0/3492868241 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80319d0 con 0x7f38e00b5c30 2024-04-25T09:57:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 -- 172.21.15.148:0/3492868241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e00b5c30 msgr2=0x7f38e0168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 --2- 172.21.15.148:0/3492868241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e00b5c30 0x7f38e0168ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f38e00d3980 tx=0x7f38d802a750 comp rx=0 tx=0).stop 2024-04-25T09:57:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 -- 172.21.15.148:0/3492868241 shutdown_connections 2024-04-25T09:57:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 --2- 172.21.15.148:0/3492868241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e00b5c30 0x7f38e0168ad0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 -- 172.21.15.148:0/3492868241 >> 172.21.15.148:0/3492868241 conn(0x7f38e00d0520 msgr2=0x7f38e00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 -- 172.21.15.148:0/3492868241 shutdown_connections 2024-04-25T09:57:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.405+0000 7f38e66dd640 1 -- 172.21.15.148:0/3492868241 wait complete. 2024-04-25T09:57:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.406+0000 7f38e66dd640 1 Processor -- start 2024-04-25T09:57:11.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.406+0000 7f38e66dd640 1 -- start start 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38e00b5c30 0x7f38e01f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38e00b6cb0 0x7f38e01f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 0x7f38e014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00ce530 con 0x7f38e015db70 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e015df50 con 0x7f38e00b5c30 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e00cf1c0 con 0x7f38e00b6cb0 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 0x7f38e014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 0x7f38e014be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46772/0 (socket says 172.21.15.148:46772) 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 -- 172.21.15.148:0/626043316 learned_addr learned my addr 172.21.15.148:0/626043316 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38df7fe640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38e00b6cb0 0x7f38e01f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 -- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38e00b6cb0 msgr2=0x7f38e01f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38dffff640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38e00b5c30 0x7f38e01f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38e00b6cb0 0x7f38e01f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 -- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38e00b5c30 msgr2=0x7f38e01f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38e00b5c30 0x7f38e01f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 -- 172.21.15.148:0/626043316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00cca20 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e4c7a640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 0x7f38e014be90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f38d8034040 tx=0x7f38d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8041070 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8003990 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e00ccd00 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8036040 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.407+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e00cd2c0 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.408+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e0073990 con 0x7f38e015db70 2024-04-25T09:57:11.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.409+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38d8007800 con 0x7f38e015db70 2024-04-25T09:57:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.409+0000 7f38dcff9640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38cc06b4d0 0x7f38cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.409+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38d80aafd0 con 0x7f38e015db70 2024-04-25T09:57:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.409+0000 7f38dffff640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38cc06b4d0 0x7f38cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.410+0000 7f38dffff640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38cc06b4d0 0x7f38cc06d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f38d0000f60 tx=0x7f38d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.414+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38d804b020 con 0x7f38e015db70 2024-04-25T09:57:11.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.584+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e00d1820 con 0x7f38e015db70 2024-04-25T09:57:11.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.585+0000 7f38dcff9640 1 -- 172.21.15.148:0/626043316 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f38d8079d80 con 0x7f38e015db70 2024-04-25T09:57:11.589 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:11.589 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":54,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:10.325125+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38cc06b4d0 msgr2=0x7f38cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38cc06b4d0 0x7f38cc06d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f38d0000f60 tx=0x7f38d0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 msgr2=0x7f38e014be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 0x7f38e014be90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f38d8034040 tx=0x7f38d8004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 shutdown_connections 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38cc06b4d0 0x7f38cc06d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38e015db70 0x7f38e014be90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38e00b6cb0 0x7f38e01f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 --2- 172.21.15.148:0/626043316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38e00b5c30 0x7f38e01f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 >> 172.21.15.148:0/626043316 conn(0x7f38e00d0520 msgr2=0x7f38e0160170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 shutdown_connections 2024-04-25T09:57:11.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.587+0000 7f38e66dd640 1 -- 172.21.15.148:0/626043316 wait complete. 2024-04-25T09:57:11.602 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:57:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.698+0000 7fb357a7c640 1 Processor -- start 2024-04-25T09:57:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.698+0000 7fb357a7c640 1 -- start start 2024-04-25T09:57:11.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb357a7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3500ced80 0x7fb3500d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb357a7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb350161710 0x7fb3500d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb357a7c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35015e000 con 0x7fb35015dc20 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb357a7c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3500d1de0 con 0x7fb350161710 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb357a7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb350161c60 con 0x7fb3500ced80 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb356019640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb350161710 0x7fb3500d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb356019640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb350161710 0x7fb3500d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46774/0 (socket says 172.21.15.148:46774) 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.699+0000 7fb356019640 1 -- 172.21.15.148:0/3575522546 learned_addr learned my addr 172.21.15.148:0/3575522546 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb355017640 1 --2- 172.21.15.148:0/3575522546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3500ced80 0x7fb3500d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb356019640 1 -- 172.21.15.148:0/3575522546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3500ced80 msgr2=0x7fb3500d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb356019640 1 --2- 172.21.15.148:0/3575522546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3500ced80 0x7fb3500d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb356019640 1 -- 172.21.15.148:0/3575522546 >> v1:172.21.15.188:6789/0 conn(0x7fb35015dc20 legacy=0x7fb3500ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb356019640 1 -- 172.21.15.148:0/3575522546 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb350161f40 con 0x7fb350161710 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb356019640 1 --2- 172.21.15.148:0/3575522546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb350161710 0x7fb3500d2820 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb348009870 tx=0x7fb34802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c97b75146c3a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.700+0000 7fb354816640 1 -- 172.21.15.148:0/3575522546 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348037070 con 0x7fb350161710 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb354816640 1 -- 172.21.15.148:0/3575522546 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb348032440 con 0x7fb350161710 2024-04-25T09:57:11.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb354816640 1 -- 172.21.15.148:0/3575522546 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348032810 con 0x7fb350161710 2024-04-25T09:57:11.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 -- 172.21.15.148:0/3575522546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb350161710 msgr2=0x7fb3500d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 --2- 172.21.15.148:0/3575522546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb350161710 0x7fb3500d2820 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb348009870 tx=0x7fb34802a270 comp rx=0 tx=0).stop 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 -- 172.21.15.148:0/3575522546 shutdown_connections 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 --2- 172.21.15.148:0/3575522546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb350161710 0x7fb3500d2820 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 --2- 172.21.15.148:0/3575522546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3500ced80 0x7fb3500d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 -- 172.21.15.148:0/3575522546 >> 172.21.15.148:0/3575522546 conn(0x7fb3500d0420 msgr2=0x7fb3500d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 -- 172.21.15.148:0/3575522546 shutdown_connections 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.701+0000 7fb357a7c640 1 -- 172.21.15.148:0/3575522546 wait complete. 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.702+0000 7fb357a7c640 1 Processor -- start 2024-04-25T09:57:11.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.702+0000 7fb357a7c640 1 -- start start 2024-04-25T09:57:11.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb357a7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3500ced80 0x7fb35015ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb357a7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 0x7fb35015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb357a7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb350161710 0x7fb3500d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb357a7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500b5e50 con 0x7fb35015dc20 2024-04-25T09:57:11.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb357a7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb350162500 con 0x7fb3500ced80 2024-04-25T09:57:11.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb357a7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3500b5fd0 con 0x7fb350161710 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 0x7fb35015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 0x7fb35015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46780/0 (socket says 172.21.15.148:46780) 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 -- 172.21.15.148:0/1157588247 learned_addr learned my addr 172.21.15.148:0/1157588247 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb356019640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb350161710 0x7fb3500d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355818640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3500ced80 0x7fb35015ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 -- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb350161710 msgr2=0x7fb3500d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb350161710 0x7fb3500d3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 -- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3500ced80 msgr2=0x7fb35015ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3500ced80 0x7fb35015ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.703+0000 7fb355017640 1 -- 172.21.15.148:0/1157588247 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3501e9270 con 0x7fb35015dc20 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.704+0000 7fb355017640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 0x7fb35015c410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb34000d620 tx=0x7fb34000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.704+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34000b620 con 0x7fb35015dc20 2024-04-25T09:57:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.704+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340019070 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.704+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35015e420 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.704+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34000ba00 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.704+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35015e9e0 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.705+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35015e230 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.705+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb34000bba0 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.706+0000 7fb3467fc640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb33406b4d0 0x7fb33406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.706+0000 7fb355818640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb33406b4d0 0x7fb33406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.706+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb34008a740 con 0x7fb35015dc20 2024-04-25T09:57:11.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.706+0000 7fb355818640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb33406b4d0 0x7fb33406d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb34c007fd0 tx=0x7fb34c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:11.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.710+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3400594c0 con 0x7fb35015dc20 2024-04-25T09:57:11.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.916+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb350073a00 con 0x7fb35015dc20 2024-04-25T09:57:11.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.989+0000 7fb3467fc640 1 -- 172.21.15.148:0/1157588247 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb340058dc0 con 0x7fb35015dc20 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb33406b4d0 msgr2=0x7fb33406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb33406b4d0 0x7fb33406d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb34c007fd0 tx=0x7fb34c009040 comp rx=0 tx=0).stop 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 msgr2=0x7fb35015c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 0x7fb35015c410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb34000d620 tx=0x7fb34000dae0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 shutdown_connections 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb33406b4d0 0x7fb33406d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb350161710 0x7fb3500d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35015dc20 0x7fb35015c410 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 --2- 172.21.15.148:0/1157588247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3500ced80 0x7fb35015ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 >> 172.21.15.148:0/1157588247 conn(0x7fb3500d0420 msgr2=0x7fb3500cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 shutdown_connections 2024-04-25T09:57:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:11.991+0000 7fb357a7c640 1 -- 172.21.15.148:0/1157588247 wait complete. 2024-04-25T09:57:12.005 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:57:12.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:57:12.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.102+0000 7fa81c122640 1 Processor -- start 2024-04-25T09:57:12.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.102+0000 7fa81c122640 1 -- start start 2024-04-25T09:57:12.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa81c122640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 0x7fa8140d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa81c122640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8140d08e0 0x7fa8140b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa81c122640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8140d2420 con 0x7fa81415dac0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa81c122640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140d3030 con 0x7fa8140d08e0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa81c122640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d2700 con 0x7fa8140d34c0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa819ebe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa81415dac0 0x7fa8140d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56458/0 (socket says 172.21.15.148:56458) 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa819ebe640 1 -- 172.21.15.148:0/2535287809 learned_addr learned my addr 172.21.15.148:0/2535287809 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424965295 0 0) 0x7fa8140d2420 con 0x7fa81415dac0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa8196bd640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 0x7fa8140d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa800003520 con 0x7fa81415dac0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa81a6bf640 1 --2- 172.21.15.148:0/2535287809 >> v2:172.21.15.188:3300/0 conn(0x7fa8140d08e0 0x7fa8140b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 379759468 0 0) 0x7fa800003520 con 0x7fa81415dac0 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 msgr2=0x7fa8140d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 0x7fa8140d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 >> v2:172.21.15.188:3300/0 conn(0x7fa8140d08e0 msgr2=0x7fa8140b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 --2- 172.21.15.148:0/2535287809 >> v2:172.21.15.188:3300/0 conn(0x7fa8140d08e0 0x7fa8140b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140d29e0 con 0x7fa81415dac0 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa810001670 con 0x7fa81415dac0 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 >> v1:172.21.15.148:6789/0 conn(0x7fa81415dac0 legacy=0x7fa8140d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.103+0000 7fa818ebc640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000039b0 0x7fa800003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa818ebc640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 0x7fa800004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa818ebc640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa800004d40 0x7fa8000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d3030 con 0x7fa8000039b0 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d2700 con 0x7fa800004d40 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80000b9c0 con 0x7fa8140d34c0 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000039b0 0x7fa800003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa8196bd640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 0x7fa800004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa81a6bf640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa800004d40 0x7fa8000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa819ebe640 1 -- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 msgr2=0x7fa800004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8140d34c0 0x7fa800004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa819ebe640 1 -- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa800004d40 msgr2=0x7fa8000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa800004d40 0x7fa8000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.104+0000 7fa819ebe640 1 -- 172.21.15.148:0/2535287809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa800003520 con 0x7fa8000039b0 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000039b0 0x7fa800003dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa81000c240 tx=0x7fa800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df95f35ddce4e901 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81002e070 con 0x7fa8000039b0 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810005af0 con 0x7fa8000039b0 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa818ebc640 1 -- 172.21.15.148:0/2535287809 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810007d30 con 0x7fa8000039b0 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81c122640 1 -- 172.21.15.148:0/2535287809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000039b0 msgr2=0x7fa800003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81c122640 1 --2- 172.21.15.148:0/2535287809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8000039b0 0x7fa800003dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa81000c240 tx=0x7fa800005830 comp rx=0 tx=0).stop 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81a6bf640 1 -- 172.21.15.148:0/2535287809 reap_dead start 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81c122640 1 -- 172.21.15.148:0/2535287809 shutdown_connections 2024-04-25T09:57:12.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81c122640 1 -- 172.21.15.148:0/2535287809 >> 172.21.15.148:0/2535287809 conn(0x7fa814061c30 msgr2=0x7fa81406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:12.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81c122640 1 -- 172.21.15.148:0/2535287809 shutdown_connections 2024-04-25T09:57:12.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.105+0000 7fa81c122640 1 -- 172.21.15.148:0/2535287809 wait complete. 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 Processor -- start 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 -- start start 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8000039b0 0x7fa8140c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa814156910 0x7fa814152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 0x7fa8141f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81415dea0 con 0x7fa8140cbaf0 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140d2100 con 0x7fa8000039b0 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81c122640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b97d0 con 0x7fa814156910 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81a6bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 0x7fa8141f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81a6bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 0x7fa8141f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46790/0 (socket says 172.21.15.148:46790) 2024-04-25T09:57:12.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.107+0000 7fa81a6bf640 1 -- 172.21.15.148:0/2104849360 learned_addr learned my addr 172.21.15.148:0/2104849360 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8000039b0 0x7fa8140c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa8196bd640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa814156910 0x7fa814152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81a6bf640 1 -- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa814156910 msgr2=0x7fa814152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81a6bf640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa814156910 0x7fa814152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81a6bf640 1 -- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8000039b0 msgr2=0x7fa8140c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81a6bf640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8000039b0 0x7fa8140c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81a6bf640 1 -- 172.21.15.148:0/2104849360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8141f7ba0 con 0x7fa8140cbaf0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8000039b0 0x7fa8140c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81a6bf640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 0x7fa8141f6c90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa81406d010 tx=0x7fa80c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00fd10 con 0x7fa8140cbaf0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c01d070 con 0x7fa8140cbaf0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8140c81b0 con 0x7fa8140cbaf0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c015670 con 0x7fa8140cbaf0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.108+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81415efa0 con 0x7fa8140cbaf0 2024-04-25T09:57:12.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.109+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814073a00 con 0x7fa8140cbaf0 2024-04-25T09:57:12.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.109+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa80c015810 con 0x7fa8140cbaf0 2024-04-25T09:57:12.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.110+0000 7fa80a7fc640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa7f806b4d0 0x7fa7f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:12.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.110+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa80c08ab20 con 0x7fa8140cbaf0 2024-04-25T09:57:12.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.110+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa7f806b4d0 0x7fa7f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:12.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.110+0000 7fa819ebe640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa7f806b4d0 0x7fa7f806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa810033000 tx=0x7fa810009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:12.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.114+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa80c010040 con 0x7fa8140cbaf0 2024-04-25T09:57:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.321+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa8140b3580 con 0x7fa8140cbaf0 2024-04-25T09:57:12.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.995+0000 7fa80a7fc640 1 -- 172.21.15.148:0/2104849360 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa80c059810 con 0x7fa8140cbaf0 2024-04-25T09:57:13.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa7f806b4d0 msgr2=0x7fa7f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa7f806b4d0 0x7fa7f806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa810033000 tx=0x7fa810009030 comp rx=0 tx=0).stop 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 msgr2=0x7fa8141f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 0x7fa8141f6c90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa81406d010 tx=0x7fa80c00f1b0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 shutdown_connections 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa7f806b4d0 0x7fa7f806d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8140cbaf0 0x7fa8141f6c90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa814156910 0x7fa814152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 --2- 172.21.15.148:0/2104849360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8000039b0 0x7fa8140c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 >> 172.21.15.148:0/2104849360 conn(0x7fa814061c30 msgr2=0x7fa8140b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 shutdown_connections 2024-04-25T09:57:13.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:12.997+0000 7fa81c122640 1 -- 172.21.15.148:0/2104849360 wait complete. 2024-04-25T09:57:13.012 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:57:13.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.108+0000 7ff79eff5640 1 Processor -- start 2024-04-25T09:57:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.108+0000 7ff79eff5640 1 -- start start 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79eff5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff79815dbe0 0x7ff7980d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79eff5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7980b2cb0 0x7ff7980d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79eff5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980b97a0 con 0x7ff7980b92a0 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79eff5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79815dfc0 con 0x7ff79815dbe0 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79eff5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980d1a00 con 0x7ff7980b2cb0 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79cd91640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff79815dbe0 0x7ff7980d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79cd91640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff79815dbe0 0x7ff7980d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46804/0 (socket says 172.21.15.148:46804) 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79cd91640 1 -- 172.21.15.148:0/1048807914 learned_addr learned my addr 172.21.15.148:0/1048807914 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.109+0000 7ff79d592640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff7980b92a0 0x7ff7980b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56466/0 (socket says 172.21.15.148:56466) 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff797fff640 1 --2- 172.21.15.148:0/1048807914 >> v2:172.21.15.188:3300/0 conn(0x7ff7980b2cb0 0x7ff7980d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff79cd91640 1 -- 172.21.15.148:0/1048807914 >> v2:172.21.15.188:3300/0 conn(0x7ff7980b2cb0 msgr2=0x7ff7980d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff79cd91640 1 --2- 172.21.15.148:0/1048807914 >> v2:172.21.15.188:3300/0 conn(0x7ff7980b2cb0 0x7ff7980d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff79cd91640 1 -- 172.21.15.148:0/1048807914 >> v1:172.21.15.148:6789/0 conn(0x7ff7980b92a0 legacy=0x7ff7980b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff79cd91640 1 -- 172.21.15.148:0/1048807914 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980d1c50 con 0x7ff79815dbe0 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff7977fe640 1 -- 172.21.15.148:0/1048807914 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296210689 0 0) 0x7ff7980b97a0 con 0x7ff7980b92a0 2024-04-25T09:57:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff797fff640 1 --2- 172.21.15.148:0/1048807914 >> v2:172.21.15.188:3300/0 conn(0x7ff7980b2cb0 0x7ff7980d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.110+0000 7ff79cd91640 1 --2- 172.21.15.148:0/1048807914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79815dbe0 0x7ff7980d0910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff79806cff0 tx=0x7ff78802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf4bb9146bd801d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff7977fe640 1 -- 172.21.15.148:0/1048807914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff79815dbe0 2024-04-25T09:57:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff7977fe640 1 -- 172.21.15.148:0/1048807914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff79815dbe0 2024-04-25T09:57:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff7977fe640 1 -- 172.21.15.148:0/1048807914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff79815dbe0 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 -- 172.21.15.148:0/1048807914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79815dbe0 msgr2=0x7ff7980d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 --2- 172.21.15.148:0/1048807914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79815dbe0 0x7ff7980d0910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff79806cff0 tx=0x7ff78802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 -- 172.21.15.148:0/1048807914 shutdown_connections 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 --2- 172.21.15.148:0/1048807914 >> v2:172.21.15.188:3300/0 conn(0x7ff7980b2cb0 0x7ff7980d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 --2- 172.21.15.148:0/1048807914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff79815dbe0 0x7ff7980d0910 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 -- 172.21.15.148:0/1048807914 >> 172.21.15.148:0/1048807914 conn(0x7ff798061a20 msgr2=0x7ff798061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 -- 172.21.15.148:0/1048807914 shutdown_connections 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.111+0000 7ff79eff5640 1 -- 172.21.15.148:0/1048807914 wait complete. 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.112+0000 7ff79eff5640 1 Processor -- start 2024-04-25T09:57:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.112+0000 7ff79eff5640 1 -- start start 2024-04-25T09:57:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79eff5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 0x7ff79815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79eff5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7980b92a0 0x7ff7981f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79eff5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff79815dbe0 0x7ff7980b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79eff5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980b6ab0 con 0x7ff7980b2cb0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79eff5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980d2210 con 0x7ff79815dbe0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79eff5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980b6c30 con 0x7ff7980b92a0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79cd91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 0x7ff79815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79cd91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 0x7ff79815a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46808/0 (socket says 172.21.15.148:46808) 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff79cd91640 1 -- 172.21.15.148:0/574366107 learned_addr learned my addr 172.21.15.148:0/574366107 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.113+0000 7ff797fff640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7980b92a0 0x7ff7981f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79cd91640 1 -- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7980b92a0 msgr2=0x7ff7981f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79d592640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff79815dbe0 0x7ff7980b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79cd91640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7980b92a0 0x7ff7981f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79cd91640 1 -- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff79815dbe0 msgr2=0x7ff7980b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79cd91640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff79815dbe0 0x7ff7980b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79cd91640 1 -- 172.21.15.148:0/574366107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7981f7960 con 0x7ff7980b2cb0 2024-04-25T09:57:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff797fff640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7980b92a0 0x7ff7981f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79d592640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff79815dbe0 0x7ff7980b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79cd91640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 0x7ff79815a400 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff78802a220 tx=0x7ff7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788041020 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7880306c0 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788037040 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff798146e70 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.114+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff798147430 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.115+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798073990 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.115+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff788044020 con 0x7ff7980b2cb0 2024-04-25T09:57:13.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.116+0000 7ff7957fa640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff78406b4d0 0x7ff78406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.116+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7880aab70 con 0x7ff7980b2cb0 2024-04-25T09:57:13.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.116+0000 7ff797fff640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff78406b4d0 0x7ff78406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.116+0000 7ff797fff640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff78406b4d0 0x7ff78406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff78c006fd0 tx=0x7ff78c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.120+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff78804b030 con 0x7ff7980b2cb0 2024-04-25T09:57:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.325+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff798062010 con 0x7ff7980b2cb0 2024-04-25T09:57:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.326+0000 7ff7957fa640 1 -- 172.21.15.148:0/574366107 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff788079a10 con 0x7ff7980b2cb0 2024-04-25T09:57:13.329 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:13.329 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff78406b4d0 msgr2=0x7ff78406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff78406b4d0 0x7ff78406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff78c006fd0 tx=0x7ff78c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 msgr2=0x7ff79815a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 0x7ff79815a400 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff78802a220 tx=0x7ff7880046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 shutdown_connections 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff78406b4d0 0x7ff78406d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff79815dbe0 0x7ff7980b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7980b92a0 0x7ff7981f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 --2- 172.21.15.148:0/574366107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7980b2cb0 0x7ff79815a400 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.327+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 >> 172.21.15.148:0/574366107 conn(0x7ff798061a20 msgr2=0x7ff79815acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.328+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 shutdown_connections 2024-04-25T09:57:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.328+0000 7ff79eff5640 1 -- 172.21.15.148:0/574366107 wait complete. 2024-04-25T09:57:13.342 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.436+0000 7f5eba3ee640 1 Processor -- start 2024-04-25T09:57:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.436+0000 7f5eba3ee640 1 -- start start 2024-04-25T09:57:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eba3ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eb415dc90 0x7f5eb41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eba3ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eb40cc8e0 0x7f5eb415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eba3ee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb415f390 con 0x7f5eb40d1810 2024-04-25T09:57:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eba3ee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb40cce50 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eba3ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb4161af0 con 0x7f5eb415dc90 2024-04-25T09:57:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eb40cc8e0 0x7f5eb415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5eb40cc8e0 0x7f5eb415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46816/0 (socket says 172.21.15.148:46816) 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb898b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5eb40d1810 0x7f5eb40cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56476/0 (socket says 172.21.15.148:56476) 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 -- 172.21.15.148:0/2199947195 learned_addr learned my addr 172.21.15.148:0/2199947195 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb3fff640 1 --2- 172.21.15.148:0/2199947195 >> v2:172.21.15.188:3300/0 conn(0x7f5eb415dc90 0x7f5eb41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 -- 172.21.15.148:0/2199947195 >> v2:172.21.15.188:3300/0 conn(0x7f5eb415dc90 msgr2=0x7f5eb41625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 --2- 172.21.15.148:0/2199947195 >> v2:172.21.15.188:3300/0 conn(0x7f5eb415dc90 0x7f5eb41625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 -- 172.21.15.148:0/2199947195 >> v1:172.21.15.148:6789/0 conn(0x7f5eb40d1810 legacy=0x7f5eb40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb2ffd640 1 -- 172.21.15.148:0/2199947195 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540622888 0 0) 0x7f5eac001490 con 0x7f5eb40d1810 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.437+0000 7f5eb37fe640 1 -- 172.21.15.148:0/2199947195 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb4161dd0 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.438+0000 7f5eb37fe640 1 --2- 172.21.15.148:0/2199947195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb415ee50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5eb406b820 tx=0x7f5ea802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c344a6a093377e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.438+0000 7f5eb2ffd640 1 -- 172.21.15.148:0/2199947195 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8037070 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.438+0000 7f5eb2ffd640 1 -- 172.21.15.148:0/2199947195 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8032440 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.438+0000 7f5eb2ffd640 1 -- 172.21.15.148:0/2199947195 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8032810 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 -- 172.21.15.148:0/2199947195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 msgr2=0x7f5eb415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/2199947195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb415ee50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5eb406b820 tx=0x7f5ea802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 -- 172.21.15.148:0/2199947195 shutdown_connections 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/2199947195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb415ee50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/2199947195 >> v2:172.21.15.188:3300/0 conn(0x7f5eb415dc90 0x7f5eb41625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 -- 172.21.15.148:0/2199947195 >> 172.21.15.148:0/2199947195 conn(0x7f5eb406f590 msgr2=0x7f5eb406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 -- 172.21.15.148:0/2199947195 shutdown_connections 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.439+0000 7f5eba3ee640 1 -- 172.21.15.148:0/2199947195 wait complete. 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.440+0000 7f5eba3ee640 1 Processor -- start 2024-04-25T09:57:13.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.440+0000 7f5eba3ee640 1 -- start start 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb40d1810 0x7f5eb41571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb415dc90 0x7f5eb40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb415e070 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb40cd130 con 0x7f5eb415dc90 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb415e1f0 con 0x7f5eb40d1810 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb4155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46830/0 (socket says 172.21.15.148:46830) 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 -- 172.21.15.148:0/721978243 learned_addr learned my addr 172.21.15.148:0/721978243 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 -- 172.21.15.148:0/721978243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb40d1810 msgr2=0x7f5eb41571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb40d1810 0x7f5eb41571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 -- 172.21.15.148:0/721978243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb415dc90 msgr2=0x7f5eb40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb415dc90 0x7f5eb40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 -- 172.21.15.148:0/721978243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40c98c0 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb3fff640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb4155210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400c7f0 tx=0x7f5ea400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4015070 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea4004220 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4007620 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb40c9ba0 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.441+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb40ca440 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.442+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb40c9f90 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.443+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ea40079d0 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.443+0000 7f5eb0ff9640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ea006b4d0 0x7f5ea006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.443+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ea408a6c0 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.443+0000 7f5eb37fe640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ea006b4d0 0x7f5ea006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.444+0000 7f5eb37fe640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ea006b4d0 0x7f5ea006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8034000 tx=0x7f5ea802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.448+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ea4059440 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.619+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eb4073990 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.620+0000 7f5eb0ff9640 1 -- 172.21.15.148:0/721978243 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f5ea4058d40 con 0x7f5eb40cc8e0 2024-04-25T09:57:13.624 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:13.624 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:11.985711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.622+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ea006b4d0 msgr2=0x7f5ea006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.622+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ea006b4d0 0x7f5ea006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8034000 tx=0x7f5ea802c000 comp rx=0 tx=0).stop 2024-04-25T09:57:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 msgr2=0x7f5eb4155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb4155210 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400c7f0 tx=0x7f5ea400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 shutdown_connections 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5ea006b4d0 0x7f5ea006d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb415dc90 0x7f5eb40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb40d1810 0x7f5eb41571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 --2- 172.21.15.148:0/721978243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eb40cc8e0 0x7f5eb4155210 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 >> 172.21.15.148:0/721978243 conn(0x7f5eb406f590 msgr2=0x7f5eb406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 shutdown_connections 2024-04-25T09:57:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.623+0000 7f5eba3ee640 1 -- 172.21.15.148:0/721978243 wait complete. 2024-04-25T09:57:13.637 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:57:13.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.731+0000 7fdde3afc640 1 Processor -- start 2024-04-25T09:57:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.731+0000 7fdde3afc640 1 -- start start 2024-04-25T09:57:13.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde3afc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdddc15da70 0x7fdddc161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde3afc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc162020 con 0x7fdddc0b9730 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde3afc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc162300 con 0x7fdddc161b80 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde3afc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc1625e0 con 0x7fdddc15da70 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1898640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdddc15da70 0x7fdddc161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1097640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdddc0b9730 0x7fdddc0ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38966/0 (socket says 172.21.15.148:38966) 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1097640 1 -- 172.21.15.148:0/2018376159 learned_addr learned my addr 172.21.15.148:0/2018376159 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1898640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdddc15da70 0x7fdddc161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43732/0 (socket says 172.21.15.148:43732) 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1898640 1 -- 172.21.15.148:0/2018376159 >> v1:172.21.15.188:6789/0 conn(0x7fdddc161b80 legacy=0x7fdddc0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1898640 1 -- 172.21.15.148:0/2018376159 >> v1:172.21.15.148:6789/0 conn(0x7fdddc0b9730 legacy=0x7fdddc0ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde0896640 1 -- 172.21.15.148:0/2018376159 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836679007 0 0) 0x7fdddc162020 con 0x7fdddc0b9730 2024-04-25T09:57:13.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.732+0000 7fdde1898640 1 -- 172.21.15.148:0/2018376159 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0d1af0 con 0x7fdddc15da70 2024-04-25T09:57:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.733+0000 7fdde1898640 1 --2- 172.21.15.148:0/2018376159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc161640 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fddd80097a0 tx=0x7fddd802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6e9b36fd6ba18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.733+0000 7fdde0896640 1 -- 172.21.15.148:0/2018376159 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8037070 con 0x7fdddc15da70 2024-04-25T09:57:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.733+0000 7fdde0896640 1 -- 172.21.15.148:0/2018376159 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8032440 con 0x7fdddc15da70 2024-04-25T09:57:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.733+0000 7fdde0896640 1 -- 172.21.15.148:0/2018376159 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8032810 con 0x7fdddc15da70 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 -- 172.21.15.148:0/2018376159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 msgr2=0x7fdddc161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 --2- 172.21.15.148:0/2018376159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc161640 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fddd80097a0 tx=0x7fddd802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 -- 172.21.15.148:0/2018376159 shutdown_connections 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 --2- 172.21.15.148:0/2018376159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc161640 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 -- 172.21.15.148:0/2018376159 >> 172.21.15.148:0/2018376159 conn(0x7fdddc061a20 msgr2=0x7fdddc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 -- 172.21.15.148:0/2018376159 shutdown_connections 2024-04-25T09:57:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.734+0000 7fdde3afc640 1 -- 172.21.15.148:0/2018376159 wait complete. 2024-04-25T09:57:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 Processor -- start 2024-04-25T09:57:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 -- start start 2024-04-25T09:57:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdddc161b80 0x7fdddc1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdddc1f7550 0x7fdddc1f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0ced60 con 0x7fdddc15da70 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc15de50 con 0x7fdddc161b80 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.735+0000 7fdde3afc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0ceee0 con 0x7fdddc1f7550 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43748/0 (socket says 172.21.15.148:43748) 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 -- 172.21.15.148:0/1428404447 learned_addr learned my addr 172.21.15.148:0/1428404447 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde2099640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdddc1f7550 0x7fdddc1f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1097640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdddc161b80 0x7fdddc1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 -- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdddc1f7550 msgr2=0x7fdddc1f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdddc1f7550 0x7fdddc1f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 -- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdddc161b80 msgr2=0x7fdddc1f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdddc161b80 0x7fdddc1f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 -- 172.21.15.148:0/1428404447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc15ebf0 con 0x7fdddc15da70 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1097640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdddc161b80 0x7fdddc1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:13.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde1898640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc158580 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fddd8000c00 tx=0x7fddd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8037040 con 0x7fdddc15da70 2024-04-25T09:57:13.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8042070 con 0x7fdddc15da70 2024-04-25T09:57:13.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8030a70 con 0x7fdddc15da70 2024-04-25T09:57:13.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc15eed0 con 0x7fdddc15da70 2024-04-25T09:57:13.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.736+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc15f490 con 0x7fdddc15da70 2024-04-25T09:57:13.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.737+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc074ee0 con 0x7fdddc15da70 2024-04-25T09:57:13.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.738+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddd8030c10 con 0x7fdddc15da70 2024-04-25T09:57:13.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.738+0000 7fddd27fc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:13.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.738+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddd80aae10 con 0x7fdddc15da70 2024-04-25T09:57:13.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.738+0000 7fdde1097640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:13.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.738+0000 7fdde1097640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddc006b4d0 0x7fddc006d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0c9c50 tx=0x7fddcc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:13.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.742+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdddc074ee0 con 0x7fdddc15da70 2024-04-25T09:57:13.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.910+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdddc1f7f20 con 0x7fddc006b4d0 2024-04-25T09:57:13.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.916+0000 7fddd27fc640 1 -- 172.21.15.148:0/1428404447 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7fdddc1f7f20 con 0x7fddc006b4d0 2024-04-25T09:57:13.922 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:13.924 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:57:13.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddc006b4d0 msgr2=0x7fddc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddc006b4d0 0x7fddc006d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0c9c50 tx=0x7fddcc009040 comp rx=0 tx=0).stop 2024-04-25T09:57:13.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 msgr2=0x7fdddc158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:13.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc158580 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fddd8000c00 tx=0x7fddd80046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 shutdown_connections 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdddc1f7550 0x7fdddc1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdddc161b80 0x7fdddc1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 --2- 172.21.15.148:0/1428404447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdddc15da70 0x7fdddc158580 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 >> 172.21.15.148:0/1428404447 conn(0x7fdddc061a20 msgr2=0x7fdddc0b7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 shutdown_connections 2024-04-25T09:57:13.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:13.919+0000 7fdde3afc640 1 -- 172.21.15.148:0/1428404447 wait complete. 2024-04-25T09:57:13.934 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-25T09:57:13.054699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276252,"kb_used_data":6004,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754698468,"statfs":{"total":773094113280,"available":772811231232,"internally_reserved":0,"allocated":6148096,"data_stored":3394291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.087327"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981364+0000","last_change":"2024-04-25T09:57:09.981364+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981364+0000","last_undegraded":"2024-04-25T09:57:09.981364+0000","last_fullsized":"2024-04-25T09:57:09.981364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980666+0000","last_change":"2024-04-25T09:57:09.980666+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980666+0000","last_undegraded":"2024-04-25T09:57:09.980666+0000","last_fullsized":"2024-04-25T09:57:09.980666+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987160+0000","last_change":"2024-04-25T09:57:09.987160+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987160+0000","last_undegraded":"2024-04-25T09:57:09.987160+0000","last_fullsized":"2024-04-25T09:57:09.987160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980352+0000","last_change":"2024-04-25T09:57:09.980352+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980352+0000","last_undegraded":"2024-04-25T09:57:09.980352+0000","last_fullsized":"2024-04-25T09:57:09.980352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976437+0000","last_change":"2024-04-25T09:57:09.976437+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976437+0000","last_undegraded":"2024-04-25T09:57:09.976437+0000","last_fullsized":"2024-04-25T09:57:09.976437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982151+0000","last_change":"2024-04-25T09:57:09.982151+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982151+0000","last_undegraded":"2024-04-25T09:57:09.982151+0000","last_fullsized":"2024-04-25T09:57:09.982151+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986707+0000","last_change":"2024-04-25T09:57:09.986707+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986707+0000","last_undegraded":"2024-04-25T09:57:09.986707+0000","last_fullsized":"2024-04-25T09:57:09.986707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987280+0000","last_change":"2024-04-25T09:57:09.987280+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987280+0000","last_undegraded":"2024-04-25T09:57:09.987280+0000","last_fullsized":"2024-04-25T09:57:09.987280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982972+0000","last_change":"2024-04-25T09:57:09.982972+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982972+0000","last_undegraded":"2024-04-25T09:57:09.982972+0000","last_fullsized":"2024-04-25T09:57:09.982972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983088+0000","last_change":"2024-04-25T09:57:09.983088+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983088+0000","last_undegraded":"2024-04-25T09:57:09.983088+0000","last_fullsized":"2024-04-25T09:57:09.983088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977943+0000","last_change":"2024-04-25T09:57:09.977943+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977943+0000","last_undegraded":"2024-04-25T09:57:09.977943+0000","last_fullsized":"2024-04-25T09:57:09.977943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979313+0000","last_change":"2024-04-25T09:57:09.979313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979313+0000","last_undegraded":"2024-04-25T09:57:09.979313+0000","last_fullsized":"2024-04-25T09:57:09.979313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983403+0000","last_change":"2024-04-25T09:57:09.983403+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983403+0000","last_undegraded":"2024-04-25T09:57:09.983403+0000","last_fullsized":"2024-04-25T09:57:09.983403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986278+0000","last_change":"2024-04-25T09:57:09.986278+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986278+0000","last_undegraded":"2024-04-25T09:57:09.986278+0000","last_fullsized":"2024-04-25T09:57:09.986278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979368+0000","last_change":"2024-04-25T09:57:09.979368+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979368+0000","last_undegraded":"2024-04-25T09:57:09.979368+0000","last_fullsized":"2024-04-25T09:57:09.979368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980509+0000","last_change":"2024-04-25T09:57:09.980509+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980509+0000","last_undegraded":"2024-04-25T09:57:09.980509+0000","last_fullsized":"2024-04-25T09:57:09.980509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987736+0000","last_change":"2024-04-25T09:57:09.987736+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987736+0000","last_undegraded":"2024-04-25T09:57:09.987736+0000","last_fullsized":"2024-04-25T09:57:09.987736+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978066+0000","last_change":"2024-04-25T09:57:09.978066+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978066+0000","last_undegraded":"2024-04-25T09:57:09.978066+0000","last_fullsized":"2024-04-25T09:57:09.978066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985583+0000","last_change":"2024-04-25T09:57:09.985583+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985583+0000","last_undegraded":"2024-04-25T09:57:09.985583+0000","last_fullsized":"2024-04-25T09:57:09.985583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982992+0000","last_change":"2024-04-25T09:57:09.982992+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982992+0000","last_undegraded":"2024-04-25T09:57:09.982992+0000","last_fullsized":"2024-04-25T09:57:09.982992+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981564+0000","last_change":"2024-04-25T09:57:09.981564+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981564+0000","last_undegraded":"2024-04-25T09:57:09.981564+0000","last_fullsized":"2024-04-25T09:57:09.981564+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980454+0000","last_change":"2024-04-25T09:57:09.980454+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980454+0000","last_undegraded":"2024-04-25T09:57:09.980454+0000","last_fullsized":"2024-04-25T09:57:09.980454+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977821+0000","last_change":"2024-04-25T09:57:09.977821+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977821+0000","last_undegraded":"2024-04-25T09:57:09.977821+0000","last_fullsized":"2024-04-25T09:57:09.977821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985146+0000","last_change":"2024-04-25T09:57:09.985146+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985146+0000","last_undegraded":"2024-04-25T09:57:09.985146+0000","last_fullsized":"2024-04-25T09:57:09.985146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985837+0000","last_change":"2024-04-25T09:57:09.985837+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985837+0000","last_undegraded":"2024-04-25T09:57:09.985837+0000","last_fullsized":"2024-04-25T09:57:09.985837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980855+0000","last_change":"2024-04-25T09:57:09.980855+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980855+0000","last_undegraded":"2024-04-25T09:57:09.980855+0000","last_fullsized":"2024-04-25T09:57:09.980855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980652+0000","last_change":"2024-04-25T09:57:09.980652+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980652+0000","last_undegraded":"2024-04-25T09:57:09.980652+0000","last_fullsized":"2024-04-25T09:57:09.980652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981345+0000","last_change":"2024-04-25T09:57:09.981345+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981345+0000","last_undegraded":"2024-04-25T09:57:09.981345+0000","last_fullsized":"2024-04-25T09:57:09.981345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984479+0000","last_change":"2024-04-25T09:57:09.984479+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984479+0000","last_undegraded":"2024-04-25T09:57:09.984479+0000","last_fullsized":"2024-04-25T09:57:09.984479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979739+0000","last_change":"2024-04-25T09:57:09.979739+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979739+0000","last_undegraded":"2024-04-25T09:57:09.979739+0000","last_fullsized":"2024-04-25T09:57:09.979739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984343+0000","last_change":"2024-04-25T09:57:09.984343+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984343+0000","last_undegraded":"2024-04-25T09:57:09.984343+0000","last_fullsized":"2024-04-25T09:57:09.984343+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978355+0000","last_change":"2024-04-25T09:57:09.978355+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978355+0000","last_undegraded":"2024-04-25T09:57:09.978355+0000","last_fullsized":"2024-04-25T09:57:09.978355+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980990+0000","last_change":"2024-04-25T09:57:09.980990+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980990+0000","last_undegraded":"2024-04-25T09:57:09.980990+0000","last_fullsized":"2024-04-25T09:57:09.980990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989713+0000","last_change":"2024-04-25T09:57:09.986934+0000","last_active":"2024-04-25T09:57:09.989713+0000","last_peered":"2024-04-25T09:57:09.989713+0000","last_clean":"2024-04-25T09:57:09.989713+0000","last_became_active":"2024-04-25T09:57:09.986610+0000","last_became_peered":"2024-04-25T09:57:09.986610+0000","last_unstale":"2024-04-25T09:57:09.989713+0000","last_undegraded":"2024-04-25T09:57:09.989713+0000","last_fullsized":"2024-04-25T09:57:09.989713+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:16.052402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977814+0000","last_change":"2024-04-25T09:57:09.977814+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977814+0000","last_undegraded":"2024-04-25T09:57:09.977814+0000","last_fullsized":"2024-04-25T09:57:09.977814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997056+0000","last_change":"2024-04-25T09:57:09.996107+0000","last_active":"2024-04-25T09:57:09.997056+0000","last_peered":"2024-04-25T09:57:09.997056+0000","last_clean":"2024-04-25T09:57:09.997056+0000","last_became_active":"2024-04-25T09:57:09.995840+0000","last_became_peered":"2024-04-25T09:57:09.995840+0000","last_unstale":"2024-04-25T09:57:09.997056+0000","last_undegraded":"2024-04-25T09:57:09.997056+0000","last_fullsized":"2024-04-25T09:57:09.997056+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980833+0000","last_change":"2024-04-25T09:57:09.980833+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980833+0000","last_undegraded":"2024-04-25T09:57:09.980833+0000","last_fullsized":"2024-04-25T09:57:09.980833+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.993934+0000","last_change":"2024-04-25T09:57:09.988639+0000","last_active":"2024-04-25T09:57:09.993934+0000","last_peered":"2024-04-25T09:57:09.993934+0000","last_clean":"2024-04-25T09:57:09.993934+0000","last_became_active":"2024-04-25T09:57:09.988362+0000","last_became_peered":"2024-04-25T09:57:09.988362+0000","last_unstale":"2024-04-25T09:57:09.993934+0000","last_undegraded":"2024-04-25T09:57:09.993934+0000","last_fullsized":"2024-04-25T09:57:09.993934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.037390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981254+0000","last_change":"2024-04-25T09:57:09.981254+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981254+0000","last_undegraded":"2024-04-25T09:57:09.981254+0000","last_fullsized":"2024-04-25T09:57:09.981254+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986629+0000","last_change":"2024-04-25T09:57:09.981554+0000","last_active":"2024-04-25T09:57:09.986629+0000","last_peered":"2024-04-25T09:57:09.986629+0000","last_clean":"2024-04-25T09:57:09.986629+0000","last_became_active":"2024-04-25T09:57:09.981290+0000","last_became_peered":"2024-04-25T09:57:09.981290+0000","last_unstale":"2024-04-25T09:57:09.986629+0000","last_undegraded":"2024-04-25T09:57:09.986629+0000","last_fullsized":"2024-04-25T09:57:09.986629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:02.596300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975566+0000","last_change":"2024-04-25T09:57:09.975566+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975566+0000","last_undegraded":"2024-04-25T09:57:09.975566+0000","last_fullsized":"2024-04-25T09:57:09.975566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997140+0000","last_change":"2024-04-25T09:57:09.995396+0000","last_active":"2024-04-25T09:57:09.997140+0000","last_peered":"2024-04-25T09:57:09.997140+0000","last_clean":"2024-04-25T09:57:09.997140+0000","last_became_active":"2024-04-25T09:57:09.995117+0000","last_became_peered":"2024-04-25T09:57:09.995117+0000","last_unstale":"2024-04-25T09:57:09.997140+0000","last_undegraded":"2024-04-25T09:57:09.997140+0000","last_fullsized":"2024-04-25T09:57:09.997140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:27.807847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982554+0000","last_change":"2024-04-25T09:57:09.982554+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982554+0000","last_undegraded":"2024-04-25T09:57:09.982554+0000","last_fullsized":"2024-04-25T09:57:09.982554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987303+0000","last_change":"2024-04-25T09:57:09.985140+0000","last_active":"2024-04-25T09:57:09.987303+0000","last_peered":"2024-04-25T09:57:09.987303+0000","last_clean":"2024-04-25T09:57:09.987303+0000","last_became_active":"2024-04-25T09:57:09.983854+0000","last_became_peered":"2024-04-25T09:57:09.983854+0000","last_unstale":"2024-04-25T09:57:09.987303+0000","last_undegraded":"2024-04-25T09:57:09.987303+0000","last_fullsized":"2024-04-25T09:57:09.987303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979321+0000","last_change":"2024-04-25T09:57:09.979321+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979321+0000","last_undegraded":"2024-04-25T09:57:09.979321+0000","last_fullsized":"2024-04-25T09:57:09.979321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987086+0000","last_change":"2024-04-25T09:57:09.985009+0000","last_active":"2024-04-25T09:57:09.987086+0000","last_peered":"2024-04-25T09:57:09.987086+0000","last_clean":"2024-04-25T09:57:09.987086+0000","last_became_active":"2024-04-25T09:57:09.984733+0000","last_became_peered":"2024-04-25T09:57:09.984733+0000","last_unstale":"2024-04-25T09:57:09.987086+0000","last_undegraded":"2024-04-25T09:57:09.987086+0000","last_fullsized":"2024-04-25T09:57:09.987086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.282213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978986+0000","last_change":"2024-04-25T09:57:09.978986+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978986+0000","last_undegraded":"2024-04-25T09:57:09.978986+0000","last_fullsized":"2024-04-25T09:57:09.978986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991792+0000","last_change":"2024-04-25T09:57:09.988494+0000","last_active":"2024-04-25T09:57:09.991792+0000","last_peered":"2024-04-25T09:57:09.991792+0000","last_clean":"2024-04-25T09:57:09.991792+0000","last_became_active":"2024-04-25T09:57:09.988229+0000","last_became_peered":"2024-04-25T09:57:09.988229+0000","last_unstale":"2024-04-25T09:57:09.991792+0000","last_undegraded":"2024-04-25T09:57:09.991792+0000","last_fullsized":"2024-04-25T09:57:09.991792+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975670+0000","last_change":"2024-04-25T09:57:09.975670+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975670+0000","last_undegraded":"2024-04-25T09:57:09.975670+0000","last_fullsized":"2024-04-25T09:57:09.975670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990292+0000","last_change":"2024-04-25T09:57:09.988417+0000","last_active":"2024-04-25T09:57:09.990292+0000","last_peered":"2024-04-25T09:57:09.990292+0000","last_clean":"2024-04-25T09:57:09.990292+0000","last_became_active":"2024-04-25T09:57:09.987870+0000","last_became_peered":"2024-04-25T09:57:09.987870+0000","last_unstale":"2024-04-25T09:57:09.990292+0000","last_undegraded":"2024-04-25T09:57:09.990292+0000","last_fullsized":"2024-04-25T09:57:09.990292+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.029172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978288+0000","last_change":"2024-04-25T09:57:09.978288+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978288+0000","last_undegraded":"2024-04-25T09:57:09.978288+0000","last_fullsized":"2024-04-25T09:57:09.978288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985102+0000","last_change":"2024-04-25T09:57:09.981400+0000","last_active":"2024-04-25T09:57:09.985102+0000","last_peered":"2024-04-25T09:57:09.985102+0000","last_clean":"2024-04-25T09:57:09.985102+0000","last_became_active":"2024-04-25T09:57:09.981081+0000","last_became_peered":"2024-04-25T09:57:09.981081+0000","last_unstale":"2024-04-25T09:57:09.985102+0000","last_undegraded":"2024-04-25T09:57:09.985102+0000","last_fullsized":"2024-04-25T09:57:09.985102+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983313+0000","last_change":"2024-04-25T09:57:09.983313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983313+0000","last_undegraded":"2024-04-25T09:57:09.983313+0000","last_fullsized":"2024-04-25T09:57:09.983313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992963+0000","last_change":"2024-04-25T09:57:09.991460+0000","last_active":"2024-04-25T09:57:09.992963+0000","last_peered":"2024-04-25T09:57:09.992963+0000","last_clean":"2024-04-25T09:57:09.992963+0000","last_became_active":"2024-04-25T09:57:09.991137+0000","last_became_peered":"2024-04-25T09:57:09.991137+0000","last_unstale":"2024-04-25T09:57:09.992963+0000","last_undegraded":"2024-04-25T09:57:09.992963+0000","last_fullsized":"2024-04-25T09:57:09.992963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.780147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976028+0000","last_change":"2024-04-25T09:57:09.976028+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976028+0000","last_undegraded":"2024-04-25T09:57:09.976028+0000","last_fullsized":"2024-04-25T09:57:09.976028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997454+0000","last_change":"2024-04-25T09:57:09.996022+0000","last_active":"2024-04-25T09:57:09.997454+0000","last_peered":"2024-04-25T09:57:09.997454+0000","last_clean":"2024-04-25T09:57:09.997454+0000","last_became_active":"2024-04-25T09:57:09.995252+0000","last_became_peered":"2024-04-25T09:57:09.995252+0000","last_unstale":"2024-04-25T09:57:09.997454+0000","last_undegraded":"2024-04-25T09:57:09.997454+0000","last_fullsized":"2024-04-25T09:57:09.997454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:03.771862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978626+0000","last_change":"2024-04-25T09:57:09.978626+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978626+0000","last_undegraded":"2024-04-25T09:57:09.978626+0000","last_fullsized":"2024-04-25T09:57:09.978626+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991523+0000","last_change":"2024-04-25T09:57:09.988947+0000","last_active":"2024-04-25T09:57:09.991523+0000","last_peered":"2024-04-25T09:57:09.991523+0000","last_clean":"2024-04-25T09:57:09.991523+0000","last_became_active":"2024-04-25T09:57:09.988669+0000","last_became_peered":"2024-04-25T09:57:09.988669+0000","last_unstale":"2024-04-25T09:57:09.991523+0000","last_undegraded":"2024-04-25T09:57:09.991523+0000","last_fullsized":"2024-04-25T09:57:09.991523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983786+0000","last_change":"2024-04-25T09:57:09.983786+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983786+0000","last_undegraded":"2024-04-25T09:57:09.983786+0000","last_fullsized":"2024-04-25T09:57:09.983786+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985528+0000","last_change":"2024-04-25T09:57:09.983965+0000","last_active":"2024-04-25T09:57:09.985528+0000","last_peered":"2024-04-25T09:57:09.985528+0000","last_clean":"2024-04-25T09:57:09.985528+0000","last_became_active":"2024-04-25T09:57:09.983606+0000","last_became_peered":"2024-04-25T09:57:09.983606+0000","last_unstale":"2024-04-25T09:57:09.985528+0000","last_undegraded":"2024-04-25T09:57:09.985528+0000","last_fullsized":"2024-04-25T09:57:09.985528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.673019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979732+0000","last_change":"2024-04-25T09:57:09.979732+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979732+0000","last_undegraded":"2024-04-25T09:57:09.979732+0000","last_fullsized":"2024-04-25T09:57:09.979732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989924+0000","last_change":"2024-04-25T09:57:09.986995+0000","last_active":"2024-04-25T09:57:09.989924+0000","last_peered":"2024-04-25T09:57:09.989924+0000","last_clean":"2024-04-25T09:57:09.989924+0000","last_became_active":"2024-04-25T09:57:09.986584+0000","last_became_peered":"2024-04-25T09:57:09.986584+0000","last_unstale":"2024-04-25T09:57:09.989924+0000","last_undegraded":"2024-04-25T09:57:09.989924+0000","last_fullsized":"2024-04-25T09:57:09.989924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:46.383736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984631+0000","last_change":"2024-04-25T09:57:09.984631+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984631+0000","last_undegraded":"2024-04-25T09:57:09.984631+0000","last_fullsized":"2024-04-25T09:57:09.984631+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997402+0000","last_change":"2024-04-25T09:57:09.995540+0000","last_active":"2024-04-25T09:57:09.997402+0000","last_peered":"2024-04-25T09:57:09.997402+0000","last_clean":"2024-04-25T09:57:09.997402+0000","last_became_active":"2024-04-25T09:57:09.995243+0000","last_became_peered":"2024-04-25T09:57:09.995243+0000","last_unstale":"2024-04-25T09:57:09.997402+0000","last_undegraded":"2024-04-25T09:57:09.997402+0000","last_fullsized":"2024-04-25T09:57:09.997402+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:32.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978354+0000","last_change":"2024-04-25T09:57:09.978354+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978354+0000","last_undegraded":"2024-04-25T09:57:09.978354+0000","last_fullsized":"2024-04-25T09:57:09.978354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997145+0000","last_change":"2024-04-25T09:57:09.995365+0000","last_active":"2024-04-25T09:57:09.997145+0000","last_peered":"2024-04-25T09:57:09.997145+0000","last_clean":"2024-04-25T09:57:09.997145+0000","last_became_active":"2024-04-25T09:57:09.995083+0000","last_became_peered":"2024-04-25T09:57:09.995083+0000","last_unstale":"2024-04-25T09:57:09.997145+0000","last_undegraded":"2024-04-25T09:57:09.997145+0000","last_fullsized":"2024-04-25T09:57:09.997145+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:57.260934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982863+0000","last_change":"2024-04-25T09:57:09.982863+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982863+0000","last_undegraded":"2024-04-25T09:57:09.982863+0000","last_fullsized":"2024-04-25T09:57:09.982863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986423+0000","last_change":"2024-04-25T09:57:09.984256+0000","last_active":"2024-04-25T09:57:09.986423+0000","last_peered":"2024-04-25T09:57:09.986423+0000","last_clean":"2024-04-25T09:57:09.986423+0000","last_became_active":"2024-04-25T09:57:09.983972+0000","last_became_peered":"2024-04-25T09:57:09.983972+0000","last_unstale":"2024-04-25T09:57:09.986423+0000","last_undegraded":"2024-04-25T09:57:09.986423+0000","last_fullsized":"2024-04-25T09:57:09.986423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:23.222733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975678+0000","last_change":"2024-04-25T09:57:09.975678+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975678+0000","last_undegraded":"2024-04-25T09:57:09.975678+0000","last_fullsized":"2024-04-25T09:57:09.975678+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992827+0000","last_change":"2024-04-25T09:57:09.991436+0000","last_active":"2024-04-25T09:57:09.992827+0000","last_peered":"2024-04-25T09:57:09.992827+0000","last_clean":"2024-04-25T09:57:09.992827+0000","last_became_active":"2024-04-25T09:57:09.991138+0000","last_became_peered":"2024-04-25T09:57:09.991138+0000","last_unstale":"2024-04-25T09:57:09.992827+0000","last_undegraded":"2024-04-25T09:57:09.992827+0000","last_fullsized":"2024-04-25T09:57:09.992827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:31.815985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982962+0000","last_change":"2024-04-25T09:57:09.982962+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982962+0000","last_undegraded":"2024-04-25T09:57:09.982962+0000","last_fullsized":"2024-04-25T09:57:09.982962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990260+0000","last_change":"2024-04-25T09:57:09.988005+0000","last_active":"2024-04-25T09:57:09.990260+0000","last_peered":"2024-04-25T09:57:09.990260+0000","last_clean":"2024-04-25T09:57:09.990260+0000","last_became_active":"2024-04-25T09:57:09.987706+0000","last_became_peered":"2024-04-25T09:57:09.987706+0000","last_unstale":"2024-04-25T09:57:09.990260+0000","last_undegraded":"2024-04-25T09:57:09.990260+0000","last_fullsized":"2024-04-25T09:57:09.990260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:48.802471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977199+0000","last_change":"2024-04-25T09:57:09.977199+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977199+0000","last_undegraded":"2024-04-25T09:57:09.977199+0000","last_fullsized":"2024-04-25T09:57:09.977199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994711+0000","last_change":"2024-04-25T09:57:09.990696+0000","last_active":"2024-04-25T09:57:09.994711+0000","last_peered":"2024-04-25T09:57:09.994711+0000","last_clean":"2024-04-25T09:57:09.994711+0000","last_became_active":"2024-04-25T09:57:09.990433+0000","last_became_peered":"2024-04-25T09:57:09.990433+0000","last_unstale":"2024-04-25T09:57:09.994711+0000","last_undegraded":"2024-04-25T09:57:09.994711+0000","last_fullsized":"2024-04-25T09:57:09.994711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980632+0000","last_change":"2024-04-25T09:57:09.980632+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980632+0000","last_undegraded":"2024-04-25T09:57:09.980632+0000","last_fullsized":"2024-04-25T09:57:09.980632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997423+0000","last_change":"2024-04-25T09:57:09.995784+0000","last_active":"2024-04-25T09:57:09.997423+0000","last_peered":"2024-04-25T09:57:09.997423+0000","last_clean":"2024-04-25T09:57:09.997423+0000","last_became_active":"2024-04-25T09:57:09.995393+0000","last_became_peered":"2024-04-25T09:57:09.995393+0000","last_unstale":"2024-04-25T09:57:09.997423+0000","last_undegraded":"2024-04-25T09:57:09.997423+0000","last_fullsized":"2024-04-25T09:57:09.997423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:44.157462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982756+0000","last_change":"2024-04-25T09:57:09.982756+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982756+0000","last_undegraded":"2024-04-25T09:57:09.982756+0000","last_fullsized":"2024-04-25T09:57:09.982756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994920+0000","last_change":"2024-04-25T09:57:09.993056+0000","last_active":"2024-04-25T09:57:09.994920+0000","last_peered":"2024-04-25T09:57:09.994920+0000","last_clean":"2024-04-25T09:57:09.994920+0000","last_became_active":"2024-04-25T09:57:09.992768+0000","last_became_peered":"2024-04-25T09:57:09.992768+0000","last_unstale":"2024-04-25T09:57:09.994920+0000","last_undegraded":"2024-04-25T09:57:09.994920+0000","last_fullsized":"2024-04-25T09:57:09.994920+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980965+0000","last_change":"2024-04-25T09:57:09.980965+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980965+0000","last_undegraded":"2024-04-25T09:57:09.980965+0000","last_fullsized":"2024-04-25T09:57:09.980965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994590+0000","last_change":"2024-04-25T09:57:09.987312+0000","last_active":"2024-04-25T09:57:09.994590+0000","last_peered":"2024-04-25T09:57:09.994590+0000","last_clean":"2024-04-25T09:57:09.994590+0000","last_became_active":"2024-04-25T09:57:09.987064+0000","last_became_peered":"2024-04-25T09:57:09.987064+0000","last_unstale":"2024-04-25T09:57:09.994590+0000","last_undegraded":"2024-04-25T09:57:09.994590+0000","last_fullsized":"2024-04-25T09:57:09.994590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.889960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977222+0000","last_change":"2024-04-25T09:57:09.977222+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977222+0000","last_undegraded":"2024-04-25T09:57:09.977222+0000","last_fullsized":"2024-04-25T09:57:09.977222+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986936+0000","last_change":"2024-04-25T09:57:09.985025+0000","last_active":"2024-04-25T09:57:09.986936+0000","last_peered":"2024-04-25T09:57:09.986936+0000","last_clean":"2024-04-25T09:57:09.986936+0000","last_became_active":"2024-04-25T09:57:09.984357+0000","last_became_peered":"2024-04-25T09:57:09.984357+0000","last_unstale":"2024-04-25T09:57:09.986936+0000","last_undegraded":"2024-04-25T09:57:09.986936+0000","last_fullsized":"2024-04-25T09:57:09.986936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:11.720778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981755+0000","last_change":"2024-04-25T09:57:09.981755+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981755+0000","last_undegraded":"2024-04-25T09:57:09.981755+0000","last_fullsized":"2024-04-25T09:57:09.981755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992985+0000","last_change":"2024-04-25T09:57:09.991923+0000","last_active":"2024-04-25T09:57:09.992985+0000","last_peered":"2024-04-25T09:57:09.992985+0000","last_clean":"2024-04-25T09:57:09.992985+0000","last_became_active":"2024-04-25T09:57:09.991642+0000","last_became_peered":"2024-04-25T09:57:09.991642+0000","last_unstale":"2024-04-25T09:57:09.992985+0000","last_undegraded":"2024-04-25T09:57:09.992985+0000","last_fullsized":"2024-04-25T09:57:09.992985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.657727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977462+0000","last_change":"2024-04-25T09:57:09.977462+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977462+0000","last_undegraded":"2024-04-25T09:57:09.977462+0000","last_fullsized":"2024-04-25T09:57:09.977462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.984066+0000","last_change":"2024-04-25T09:57:09.982146+0000","last_active":"2024-04-25T09:57:09.984066+0000","last_peered":"2024-04-25T09:57:09.984066+0000","last_clean":"2024-04-25T09:57:09.984066+0000","last_became_active":"2024-04-25T09:57:09.981876+0000","last_became_peered":"2024-04-25T09:57:09.981876+0000","last_unstale":"2024-04-25T09:57:09.984066+0000","last_undegraded":"2024-04-25T09:57:09.984066+0000","last_fullsized":"2024-04-25T09:57:09.984066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982765+0000","last_change":"2024-04-25T09:57:09.982765+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982765+0000","last_undegraded":"2024-04-25T09:57:09.982765+0000","last_fullsized":"2024-04-25T09:57:09.982765+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997380+0000","last_change":"2024-04-25T09:57:09.995462+0000","last_active":"2024-04-25T09:57:09.997380+0000","last_peered":"2024-04-25T09:57:09.997380+0000","last_clean":"2024-04-25T09:57:09.997380+0000","last_became_active":"2024-04-25T09:57:09.995179+0000","last_became_peered":"2024-04-25T09:57:09.995179+0000","last_unstale":"2024-04-25T09:57:09.997380+0000","last_undegraded":"2024-04-25T09:57:09.997380+0000","last_fullsized":"2024-04-25T09:57:09.997380+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.011448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977753+0000","last_change":"2024-04-25T09:57:09.977753+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977753+0000","last_undegraded":"2024-04-25T09:57:09.977753+0000","last_fullsized":"2024-04-25T09:57:09.977753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.996932+0000","last_change":"2024-04-25T09:57:09.994224+0000","last_active":"2024-04-25T09:57:09.996932+0000","last_peered":"2024-04-25T09:57:09.996932+0000","last_clean":"2024-04-25T09:57:09.996932+0000","last_became_active":"2024-04-25T09:57:09.993902+0000","last_became_peered":"2024-04-25T09:57:09.993902+0000","last_unstale":"2024-04-25T09:57:09.996932+0000","last_undegraded":"2024-04-25T09:57:09.996932+0000","last_fullsized":"2024-04-25T09:57:09.996932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983273+0000","last_change":"2024-04-25T09:57:09.983273+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983273+0000","last_undegraded":"2024-04-25T09:57:09.983273+0000","last_fullsized":"2024-04-25T09:57:09.983273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989862+0000","last_change":"2024-04-25T09:57:09.988098+0000","last_active":"2024-04-25T09:57:09.989862+0000","last_peered":"2024-04-25T09:57:09.989862+0000","last_clean":"2024-04-25T09:57:09.989862+0000","last_became_active":"2024-04-25T09:57:09.987672+0000","last_became_peered":"2024-04-25T09:57:09.987672+0000","last_unstale":"2024-04-25T09:57:09.989862+0000","last_undegraded":"2024-04-25T09:57:09.989862+0000","last_fullsized":"2024-04-25T09:57:09.989862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981392+0000","last_change":"2024-04-25T09:57:09.981392+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981392+0000","last_undegraded":"2024-04-25T09:57:09.981392+0000","last_fullsized":"2024-04-25T09:57:09.981392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987441+0000","last_change":"2024-04-25T09:57:09.985927+0000","last_active":"2024-04-25T09:57:09.987441+0000","last_peered":"2024-04-25T09:57:09.987441+0000","last_clean":"2024-04-25T09:57:09.987441+0000","last_became_active":"2024-04-25T09:57:09.985657+0000","last_became_peered":"2024-04-25T09:57:09.985657+0000","last_unstale":"2024-04-25T09:57:09.987441+0000","last_undegraded":"2024-04-25T09:57:09.987441+0000","last_fullsized":"2024-04-25T09:57:09.987441+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981050+0000","last_change":"2024-04-25T09:57:09.981050+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981050+0000","last_undegraded":"2024-04-25T09:57:09.981050+0000","last_fullsized":"2024-04-25T09:57:09.981050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991567+0000","last_change":"2024-04-25T09:57:09.990770+0000","last_active":"2024-04-25T09:57:09.991567+0000","last_peered":"2024-04-25T09:57:09.991567+0000","last_clean":"2024-04-25T09:57:09.991567+0000","last_became_active":"2024-04-25T09:57:09.990495+0000","last_became_peered":"2024-04-25T09:57:09.990495+0000","last_unstale":"2024-04-25T09:57:09.991567+0000","last_undegraded":"2024-04-25T09:57:09.991567+0000","last_fullsized":"2024-04-25T09:57:09.991567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.981492+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:09.981492+0000","last_peered":"2024-04-25T09:57:09.981492+0000","last_clean":"2024-04-25T09:57:09.981492+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:09.981492+0000","last_undegraded":"2024-04-25T09:57:09.981492+0000","last_fullsized":"2024-04-25T09:57:09.981492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":866229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1224704,"data_stored":872896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:13.939 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:14.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.067+0000 7fc03c970640 1 Processor -- start 2024-04-25T09:57:14.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.067+0000 7fc03c970640 1 -- start start 2024-04-25T09:57:14.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.068+0000 7fc03c970640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0380b2cb0 0x7fc0380d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.068+0000 7fc03c970640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380b97a0 con 0x7fc03815dbe0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.068+0000 7fc03c970640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380d1c50 con 0x7fc0380d1020 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.068+0000 7fc03c970640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03815dfc0 con 0x7fc0380b2cb0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.068+0000 7fc03659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc03815dbe0 0x7fc0380d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38970/0 (socket says 172.21.15.148:38970) 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.068+0000 7fc03659c640 1 -- 172.21.15.148:0/1840835558 learned_addr learned my addr 172.21.15.148:0/1840835558 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882164804 0 0) 0x7fc0380b97a0 con 0x7fc03815dbe0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc020003520 con 0x7fc03815dbe0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc035d9b640 1 --2- 172.21.15.148:0/1840835558 >> v2:172.21.15.188:3300/0 conn(0x7fc0380b2cb0 0x7fc0380d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2000364785 0 0) 0x7fc020003520 con 0x7fc03815dbe0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 >> v2:172.21.15.188:3300/0 conn(0x7fc0380b2cb0 msgr2=0x7fc0380d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 --2- 172.21.15.148:0/1840835558 >> v2:172.21.15.188:3300/0 conn(0x7fc0380b2cb0 0x7fc0380d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 >> v1:172.21.15.188:6789/0 conn(0x7fc0380d1020 legacy=0x7fc0380d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0380d1f30 con 0x7fc03815dbe0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc0280019f0 con 0x7fc03815dbe0 2024-04-25T09:57:14.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 >> v1:172.21.15.148:6789/0 conn(0x7fc03815dbe0 legacy=0x7fc0380d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0200039b0 0x7fc020003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc020004800 0x7fc020004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0200051c0 0x7fc02000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000c0e0 con 0x7fc0200039b0 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000c290 con 0x7fc020004800 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000c490 con 0x7fc0200051c0 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03659c640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0200039b0 0x7fc020003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc035d9b640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc020004800 0x7fc020004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc036d9d640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0200051c0 0x7fc02000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03659c640 1 -- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0200051c0 msgr2=0x7fc02000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03659c640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0200051c0 0x7fc02000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03659c640 1 -- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc020004800 msgr2=0x7fc020004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03659c640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc020004800 0x7fc020004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc03659c640 1 -- 172.21.15.148:0/1840835558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020003520 con 0x7fc0200039b0 2024-04-25T09:57:14.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc035d9b640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc020004800 0x7fc020004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.069+0000 7fc036d9d640 1 -- 172.21.15.148:0/1840835558 reap_dead start 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.070+0000 7fc03659c640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0200039b0 0x7fc020003dd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc02800a860 tx=0x7fc020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf4c5a49fe803d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.070+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800a480 con 0x7fc0200039b0 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.070+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02802f070 con 0x7fc0200039b0 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.070+0000 7fc03559a640 1 -- 172.21.15.148:0/1840835558 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02802bd40 con 0x7fc0200039b0 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 -- 172.21.15.148:0/1840835558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0200039b0 msgr2=0x7fc020003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0200039b0 0x7fc020003dd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc02800a860 tx=0x7fc020005830 comp rx=0 tx=0).stop 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 -- 172.21.15.148:0/1840835558 shutdown_connections 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 --2- 172.21.15.148:0/1840835558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0200039b0 0x7fc020003dd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 -- 172.21.15.148:0/1840835558 >> 172.21.15.148:0/1840835558 conn(0x7fc038061a20 msgr2=0x7fc038061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 -- 172.21.15.148:0/1840835558 shutdown_connections 2024-04-25T09:57:14.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.071+0000 7fc03c970640 1 -- 172.21.15.148:0/1840835558 wait complete. 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.072+0000 7fc03c970640 1 Processor -- start 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.072+0000 7fc03c970640 1 -- start start 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03c970640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0200039b0 0x7fc03814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03c970640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 0x7fc0380b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03c970640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc03815f550 0x7fc03815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03c970640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380b92a0 con 0x7fc0380cbe70 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03c970640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380d1a00 con 0x7fc0200039b0 2024-04-25T09:57:14.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03c970640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380b9420 con 0x7fc03815f550 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 0x7fc0380b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 0x7fc0380b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43772/0 (socket says 172.21.15.148:43772) 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 -- 172.21.15.148:0/761520374 learned_addr learned my addr 172.21.15.148:0/761520374 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03659c640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0200039b0 0x7fc03814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc036d9d640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc03815f550 0x7fc03815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 -- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc03815f550 msgr2=0x7fc03815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc03815f550 0x7fc03815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 -- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0200039b0 msgr2=0x7fc03814c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0200039b0 0x7fc03814c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc035d9b640 1 -- 172.21.15.148:0/761520374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03814c700 con 0x7fc0380cbe70 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.073+0000 7fc03659c640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0200039b0 0x7fc03814c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.074+0000 7fc035d9b640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 0x7fc0380b32e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00f490 tx=0x7fc02c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.074+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c018020 con 0x7fc0380cbe70 2024-04-25T09:57:14.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.074+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c014930 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.074+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0380b3be0 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.074+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c014d20 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.074+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0380b4710 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.075+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc038073990 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.075+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc02c01b020 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.076+0000 7fc026ffd640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc01806b4d0 0x7fc01806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.076+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc02c08a6a0 con 0x7fc0380cbe70 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.076+0000 7fc03659c640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc01806b4d0 0x7fc01806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.076+0000 7fc03659c640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc01806b4d0 0x7fc01806d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc028033000 tx=0x7fc028038040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.080+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc02c022020 con 0x7fc0380cbe70 2024-04-25T09:57:14.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.247+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0380d0ab0 con 0x7fc01806b4d0 2024-04-25T09:57:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.253+0000 7fc026ffd640 1 -- 172.21.15.148:0/761520374 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7fc028082380 con 0x7fc01806b4d0 2024-04-25T09:57:14.259 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:14.260 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:57:14.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc01806b4d0 msgr2=0x7fc01806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc01806b4d0 0x7fc01806d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc028033000 tx=0x7fc028038040 comp rx=0 tx=0).stop 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 msgr2=0x7fc0380b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 0x7fc0380b32e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00f490 tx=0x7fc02c00f950 comp rx=0 tx=0).stop 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 shutdown_connections 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc01806b4d0 0x7fc01806d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc03815f550 0x7fc03815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0380cbe70 0x7fc0380b32e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 --2- 172.21.15.148:0/761520374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0200039b0 0x7fc03814c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 >> 172.21.15.148:0/761520374 conn(0x7fc038061a20 msgr2=0x7fc0380d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 shutdown_connections 2024-04-25T09:57:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.256+0000 7fc03c970640 1 -- 172.21.15.148:0/761520374 wait complete. 2024-04-25T09:57:14.270 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-25T09:57:13.054699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276252,"kb_used_data":6004,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754698468,"statfs":{"total":773094113280,"available":772811231232,"internally_reserved":0,"allocated":6148096,"data_stored":3394291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.087327"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981364+0000","last_change":"2024-04-25T09:57:09.981364+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981364+0000","last_undegraded":"2024-04-25T09:57:09.981364+0000","last_fullsized":"2024-04-25T09:57:09.981364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980666+0000","last_change":"2024-04-25T09:57:09.980666+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980666+0000","last_undegraded":"2024-04-25T09:57:09.980666+0000","last_fullsized":"2024-04-25T09:57:09.980666+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987160+0000","last_change":"2024-04-25T09:57:09.987160+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987160+0000","last_undegraded":"2024-04-25T09:57:09.987160+0000","last_fullsized":"2024-04-25T09:57:09.987160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980352+0000","last_change":"2024-04-25T09:57:09.980352+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980352+0000","last_undegraded":"2024-04-25T09:57:09.980352+0000","last_fullsized":"2024-04-25T09:57:09.980352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976437+0000","last_change":"2024-04-25T09:57:09.976437+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976437+0000","last_undegraded":"2024-04-25T09:57:09.976437+0000","last_fullsized":"2024-04-25T09:57:09.976437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982151+0000","last_change":"2024-04-25T09:57:09.982151+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982151+0000","last_undegraded":"2024-04-25T09:57:09.982151+0000","last_fullsized":"2024-04-25T09:57:09.982151+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986707+0000","last_change":"2024-04-25T09:57:09.986707+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986707+0000","last_undegraded":"2024-04-25T09:57:09.986707+0000","last_fullsized":"2024-04-25T09:57:09.986707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987280+0000","last_change":"2024-04-25T09:57:09.987280+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987280+0000","last_undegraded":"2024-04-25T09:57:09.987280+0000","last_fullsized":"2024-04-25T09:57:09.987280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982972+0000","last_change":"2024-04-25T09:57:09.982972+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982972+0000","last_undegraded":"2024-04-25T09:57:09.982972+0000","last_fullsized":"2024-04-25T09:57:09.982972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983088+0000","last_change":"2024-04-25T09:57:09.983088+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983088+0000","last_undegraded":"2024-04-25T09:57:09.983088+0000","last_fullsized":"2024-04-25T09:57:09.983088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977943+0000","last_change":"2024-04-25T09:57:09.977943+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977943+0000","last_undegraded":"2024-04-25T09:57:09.977943+0000","last_fullsized":"2024-04-25T09:57:09.977943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979313+0000","last_change":"2024-04-25T09:57:09.979313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979313+0000","last_undegraded":"2024-04-25T09:57:09.979313+0000","last_fullsized":"2024-04-25T09:57:09.979313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983403+0000","last_change":"2024-04-25T09:57:09.983403+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983403+0000","last_undegraded":"2024-04-25T09:57:09.983403+0000","last_fullsized":"2024-04-25T09:57:09.983403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986278+0000","last_change":"2024-04-25T09:57:09.986278+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986278+0000","last_undegraded":"2024-04-25T09:57:09.986278+0000","last_fullsized":"2024-04-25T09:57:09.986278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979368+0000","last_change":"2024-04-25T09:57:09.979368+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979368+0000","last_undegraded":"2024-04-25T09:57:09.979368+0000","last_fullsized":"2024-04-25T09:57:09.979368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980509+0000","last_change":"2024-04-25T09:57:09.980509+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980509+0000","last_undegraded":"2024-04-25T09:57:09.980509+0000","last_fullsized":"2024-04-25T09:57:09.980509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987736+0000","last_change":"2024-04-25T09:57:09.987736+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987736+0000","last_undegraded":"2024-04-25T09:57:09.987736+0000","last_fullsized":"2024-04-25T09:57:09.987736+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978066+0000","last_change":"2024-04-25T09:57:09.978066+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978066+0000","last_undegraded":"2024-04-25T09:57:09.978066+0000","last_fullsized":"2024-04-25T09:57:09.978066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985583+0000","last_change":"2024-04-25T09:57:09.985583+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985583+0000","last_undegraded":"2024-04-25T09:57:09.985583+0000","last_fullsized":"2024-04-25T09:57:09.985583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982992+0000","last_change":"2024-04-25T09:57:09.982992+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982992+0000","last_undegraded":"2024-04-25T09:57:09.982992+0000","last_fullsized":"2024-04-25T09:57:09.982992+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981564+0000","last_change":"2024-04-25T09:57:09.981564+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981564+0000","last_undegraded":"2024-04-25T09:57:09.981564+0000","last_fullsized":"2024-04-25T09:57:09.981564+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980454+0000","last_change":"2024-04-25T09:57:09.980454+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980454+0000","last_undegraded":"2024-04-25T09:57:09.980454+0000","last_fullsized":"2024-04-25T09:57:09.980454+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977821+0000","last_change":"2024-04-25T09:57:09.977821+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977821+0000","last_undegraded":"2024-04-25T09:57:09.977821+0000","last_fullsized":"2024-04-25T09:57:09.977821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985146+0000","last_change":"2024-04-25T09:57:09.985146+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985146+0000","last_undegraded":"2024-04-25T09:57:09.985146+0000","last_fullsized":"2024-04-25T09:57:09.985146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985837+0000","last_change":"2024-04-25T09:57:09.985837+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985837+0000","last_undegraded":"2024-04-25T09:57:09.985837+0000","last_fullsized":"2024-04-25T09:57:09.985837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980855+0000","last_change":"2024-04-25T09:57:09.980855+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980855+0000","last_undegraded":"2024-04-25T09:57:09.980855+0000","last_fullsized":"2024-04-25T09:57:09.980855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980652+0000","last_change":"2024-04-25T09:57:09.980652+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980652+0000","last_undegraded":"2024-04-25T09:57:09.980652+0000","last_fullsized":"2024-04-25T09:57:09.980652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981345+0000","last_change":"2024-04-25T09:57:09.981345+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981345+0000","last_undegraded":"2024-04-25T09:57:09.981345+0000","last_fullsized":"2024-04-25T09:57:09.981345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984479+0000","last_change":"2024-04-25T09:57:09.984479+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984479+0000","last_undegraded":"2024-04-25T09:57:09.984479+0000","last_fullsized":"2024-04-25T09:57:09.984479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979739+0000","last_change":"2024-04-25T09:57:09.979739+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979739+0000","last_undegraded":"2024-04-25T09:57:09.979739+0000","last_fullsized":"2024-04-25T09:57:09.979739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984343+0000","last_change":"2024-04-25T09:57:09.984343+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984343+0000","last_undegraded":"2024-04-25T09:57:09.984343+0000","last_fullsized":"2024-04-25T09:57:09.984343+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978355+0000","last_change":"2024-04-25T09:57:09.978355+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978355+0000","last_undegraded":"2024-04-25T09:57:09.978355+0000","last_fullsized":"2024-04-25T09:57:09.978355+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980990+0000","last_change":"2024-04-25T09:57:09.980990+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980990+0000","last_undegraded":"2024-04-25T09:57:09.980990+0000","last_fullsized":"2024-04-25T09:57:09.980990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989713+0000","last_change":"2024-04-25T09:57:09.986934+0000","last_active":"2024-04-25T09:57:09.989713+0000","last_peered":"2024-04-25T09:57:09.989713+0000","last_clean":"2024-04-25T09:57:09.989713+0000","last_became_active":"2024-04-25T09:57:09.986610+0000","last_became_peered":"2024-04-25T09:57:09.986610+0000","last_unstale":"2024-04-25T09:57:09.989713+0000","last_undegraded":"2024-04-25T09:57:09.989713+0000","last_fullsized":"2024-04-25T09:57:09.989713+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:16.052402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977814+0000","last_change":"2024-04-25T09:57:09.977814+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977814+0000","last_undegraded":"2024-04-25T09:57:09.977814+0000","last_fullsized":"2024-04-25T09:57:09.977814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997056+0000","last_change":"2024-04-25T09:57:09.996107+0000","last_active":"2024-04-25T09:57:09.997056+0000","last_peered":"2024-04-25T09:57:09.997056+0000","last_clean":"2024-04-25T09:57:09.997056+0000","last_became_active":"2024-04-25T09:57:09.995840+0000","last_became_peered":"2024-04-25T09:57:09.995840+0000","last_unstale":"2024-04-25T09:57:09.997056+0000","last_undegraded":"2024-04-25T09:57:09.997056+0000","last_fullsized":"2024-04-25T09:57:09.997056+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980833+0000","last_change":"2024-04-25T09:57:09.980833+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980833+0000","last_undegraded":"2024-04-25T09:57:09.980833+0000","last_fullsized":"2024-04-25T09:57:09.980833+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.993934+0000","last_change":"2024-04-25T09:57:09.988639+0000","last_active":"2024-04-25T09:57:09.993934+0000","last_peered":"2024-04-25T09:57:09.993934+0000","last_clean":"2024-04-25T09:57:09.993934+0000","last_became_active":"2024-04-25T09:57:09.988362+0000","last_became_peered":"2024-04-25T09:57:09.988362+0000","last_unstale":"2024-04-25T09:57:09.993934+0000","last_undegraded":"2024-04-25T09:57:09.993934+0000","last_fullsized":"2024-04-25T09:57:09.993934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.037390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981254+0000","last_change":"2024-04-25T09:57:09.981254+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981254+0000","last_undegraded":"2024-04-25T09:57:09.981254+0000","last_fullsized":"2024-04-25T09:57:09.981254+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986629+0000","last_change":"2024-04-25T09:57:09.981554+0000","last_active":"2024-04-25T09:57:09.986629+0000","last_peered":"2024-04-25T09:57:09.986629+0000","last_clean":"2024-04-25T09:57:09.986629+0000","last_became_active":"2024-04-25T09:57:09.981290+0000","last_became_peered":"2024-04-25T09:57:09.981290+0000","last_unstale":"2024-04-25T09:57:09.986629+0000","last_undegraded":"2024-04-25T09:57:09.986629+0000","last_fullsized":"2024-04-25T09:57:09.986629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:02.596300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975566+0000","last_change":"2024-04-25T09:57:09.975566+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975566+0000","last_undegraded":"2024-04-25T09:57:09.975566+0000","last_fullsized":"2024-04-25T09:57:09.975566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997140+0000","last_change":"2024-04-25T09:57:09.995396+0000","last_active":"2024-04-25T09:57:09.997140+0000","last_peered":"2024-04-25T09:57:09.997140+0000","last_clean":"2024-04-25T09:57:09.997140+0000","last_became_active":"2024-04-25T09:57:09.995117+0000","last_became_peered":"2024-04-25T09:57:09.995117+0000","last_unstale":"2024-04-25T09:57:09.997140+0000","last_undegraded":"2024-04-25T09:57:09.997140+0000","last_fullsized":"2024-04-25T09:57:09.997140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:27.807847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982554+0000","last_change":"2024-04-25T09:57:09.982554+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982554+0000","last_undegraded":"2024-04-25T09:57:09.982554+0000","last_fullsized":"2024-04-25T09:57:09.982554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987303+0000","last_change":"2024-04-25T09:57:09.985140+0000","last_active":"2024-04-25T09:57:09.987303+0000","last_peered":"2024-04-25T09:57:09.987303+0000","last_clean":"2024-04-25T09:57:09.987303+0000","last_became_active":"2024-04-25T09:57:09.983854+0000","last_became_peered":"2024-04-25T09:57:09.983854+0000","last_unstale":"2024-04-25T09:57:09.987303+0000","last_undegraded":"2024-04-25T09:57:09.987303+0000","last_fullsized":"2024-04-25T09:57:09.987303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979321+0000","last_change":"2024-04-25T09:57:09.979321+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979321+0000","last_undegraded":"2024-04-25T09:57:09.979321+0000","last_fullsized":"2024-04-25T09:57:09.979321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987086+0000","last_change":"2024-04-25T09:57:09.985009+0000","last_active":"2024-04-25T09:57:09.987086+0000","last_peered":"2024-04-25T09:57:09.987086+0000","last_clean":"2024-04-25T09:57:09.987086+0000","last_became_active":"2024-04-25T09:57:09.984733+0000","last_became_peered":"2024-04-25T09:57:09.984733+0000","last_unstale":"2024-04-25T09:57:09.987086+0000","last_undegraded":"2024-04-25T09:57:09.987086+0000","last_fullsized":"2024-04-25T09:57:09.987086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.282213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978986+0000","last_change":"2024-04-25T09:57:09.978986+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978986+0000","last_undegraded":"2024-04-25T09:57:09.978986+0000","last_fullsized":"2024-04-25T09:57:09.978986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991792+0000","last_change":"2024-04-25T09:57:09.988494+0000","last_active":"2024-04-25T09:57:09.991792+0000","last_peered":"2024-04-25T09:57:09.991792+0000","last_clean":"2024-04-25T09:57:09.991792+0000","last_became_active":"2024-04-25T09:57:09.988229+0000","last_became_peered":"2024-04-25T09:57:09.988229+0000","last_unstale":"2024-04-25T09:57:09.991792+0000","last_undegraded":"2024-04-25T09:57:09.991792+0000","last_fullsized":"2024-04-25T09:57:09.991792+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975670+0000","last_change":"2024-04-25T09:57:09.975670+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975670+0000","last_undegraded":"2024-04-25T09:57:09.975670+0000","last_fullsized":"2024-04-25T09:57:09.975670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990292+0000","last_change":"2024-04-25T09:57:09.988417+0000","last_active":"2024-04-25T09:57:09.990292+0000","last_peered":"2024-04-25T09:57:09.990292+0000","last_clean":"2024-04-25T09:57:09.990292+0000","last_became_active":"2024-04-25T09:57:09.987870+0000","last_became_peered":"2024-04-25T09:57:09.987870+0000","last_unstale":"2024-04-25T09:57:09.990292+0000","last_undegraded":"2024-04-25T09:57:09.990292+0000","last_fullsized":"2024-04-25T09:57:09.990292+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.029172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978288+0000","last_change":"2024-04-25T09:57:09.978288+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978288+0000","last_undegraded":"2024-04-25T09:57:09.978288+0000","last_fullsized":"2024-04-25T09:57:09.978288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985102+0000","last_change":"2024-04-25T09:57:09.981400+0000","last_active":"2024-04-25T09:57:09.985102+0000","last_peered":"2024-04-25T09:57:09.985102+0000","last_clean":"2024-04-25T09:57:09.985102+0000","last_became_active":"2024-04-25T09:57:09.981081+0000","last_became_peered":"2024-04-25T09:57:09.981081+0000","last_unstale":"2024-04-25T09:57:09.985102+0000","last_undegraded":"2024-04-25T09:57:09.985102+0000","last_fullsized":"2024-04-25T09:57:09.985102+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983313+0000","last_change":"2024-04-25T09:57:09.983313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983313+0000","last_undegraded":"2024-04-25T09:57:09.983313+0000","last_fullsized":"2024-04-25T09:57:09.983313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992963+0000","last_change":"2024-04-25T09:57:09.991460+0000","last_active":"2024-04-25T09:57:09.992963+0000","last_peered":"2024-04-25T09:57:09.992963+0000","last_clean":"2024-04-25T09:57:09.992963+0000","last_became_active":"2024-04-25T09:57:09.991137+0000","last_became_peered":"2024-04-25T09:57:09.991137+0000","last_unstale":"2024-04-25T09:57:09.992963+0000","last_undegraded":"2024-04-25T09:57:09.992963+0000","last_fullsized":"2024-04-25T09:57:09.992963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.780147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976028+0000","last_change":"2024-04-25T09:57:09.976028+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976028+0000","last_undegraded":"2024-04-25T09:57:09.976028+0000","last_fullsized":"2024-04-25T09:57:09.976028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997454+0000","last_change":"2024-04-25T09:57:09.996022+0000","last_active":"2024-04-25T09:57:09.997454+0000","last_peered":"2024-04-25T09:57:09.997454+0000","last_clean":"2024-04-25T09:57:09.997454+0000","last_became_active":"2024-04-25T09:57:09.995252+0000","last_became_peered":"2024-04-25T09:57:09.995252+0000","last_unstale":"2024-04-25T09:57:09.997454+0000","last_undegraded":"2024-04-25T09:57:09.997454+0000","last_fullsized":"2024-04-25T09:57:09.997454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:03.771862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978626+0000","last_change":"2024-04-25T09:57:09.978626+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978626+0000","last_undegraded":"2024-04-25T09:57:09.978626+0000","last_fullsized":"2024-04-25T09:57:09.978626+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991523+0000","last_change":"2024-04-25T09:57:09.988947+0000","last_active":"2024-04-25T09:57:09.991523+0000","last_peered":"2024-04-25T09:57:09.991523+0000","last_clean":"2024-04-25T09:57:09.991523+0000","last_became_active":"2024-04-25T09:57:09.988669+0000","last_became_peered":"2024-04-25T09:57:09.988669+0000","last_unstale":"2024-04-25T09:57:09.991523+0000","last_undegraded":"2024-04-25T09:57:09.991523+0000","last_fullsized":"2024-04-25T09:57:09.991523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983786+0000","last_change":"2024-04-25T09:57:09.983786+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983786+0000","last_undegraded":"2024-04-25T09:57:09.983786+0000","last_fullsized":"2024-04-25T09:57:09.983786+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985528+0000","last_change":"2024-04-25T09:57:09.983965+0000","last_active":"2024-04-25T09:57:09.985528+0000","last_peered":"2024-04-25T09:57:09.985528+0000","last_clean":"2024-04-25T09:57:09.985528+0000","last_became_active":"2024-04-25T09:57:09.983606+0000","last_became_peered":"2024-04-25T09:57:09.983606+0000","last_unstale":"2024-04-25T09:57:09.985528+0000","last_undegraded":"2024-04-25T09:57:09.985528+0000","last_fullsized":"2024-04-25T09:57:09.985528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.673019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979732+0000","last_change":"2024-04-25T09:57:09.979732+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979732+0000","last_undegraded":"2024-04-25T09:57:09.979732+0000","last_fullsized":"2024-04-25T09:57:09.979732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989924+0000","last_change":"2024-04-25T09:57:09.986995+0000","last_active":"2024-04-25T09:57:09.989924+0000","last_peered":"2024-04-25T09:57:09.989924+0000","last_clean":"2024-04-25T09:57:09.989924+0000","last_became_active":"2024-04-25T09:57:09.986584+0000","last_became_peered":"2024-04-25T09:57:09.986584+0000","last_unstale":"2024-04-25T09:57:09.989924+0000","last_undegraded":"2024-04-25T09:57:09.989924+0000","last_fullsized":"2024-04-25T09:57:09.989924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:46.383736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984631+0000","last_change":"2024-04-25T09:57:09.984631+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984631+0000","last_undegraded":"2024-04-25T09:57:09.984631+0000","last_fullsized":"2024-04-25T09:57:09.984631+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997402+0000","last_change":"2024-04-25T09:57:09.995540+0000","last_active":"2024-04-25T09:57:09.997402+0000","last_peered":"2024-04-25T09:57:09.997402+0000","last_clean":"2024-04-25T09:57:09.997402+0000","last_became_active":"2024-04-25T09:57:09.995243+0000","last_became_peered":"2024-04-25T09:57:09.995243+0000","last_unstale":"2024-04-25T09:57:09.997402+0000","last_undegraded":"2024-04-25T09:57:09.997402+0000","last_fullsized":"2024-04-25T09:57:09.997402+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:32.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978354+0000","last_change":"2024-04-25T09:57:09.978354+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978354+0000","last_undegraded":"2024-04-25T09:57:09.978354+0000","last_fullsized":"2024-04-25T09:57:09.978354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997145+0000","last_change":"2024-04-25T09:57:09.995365+0000","last_active":"2024-04-25T09:57:09.997145+0000","last_peered":"2024-04-25T09:57:09.997145+0000","last_clean":"2024-04-25T09:57:09.997145+0000","last_became_active":"2024-04-25T09:57:09.995083+0000","last_became_peered":"2024-04-25T09:57:09.995083+0000","last_unstale":"2024-04-25T09:57:09.997145+0000","last_undegraded":"2024-04-25T09:57:09.997145+0000","last_fullsized":"2024-04-25T09:57:09.997145+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:57.260934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982863+0000","last_change":"2024-04-25T09:57:09.982863+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982863+0000","last_undegraded":"2024-04-25T09:57:09.982863+0000","last_fullsized":"2024-04-25T09:57:09.982863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986423+0000","last_change":"2024-04-25T09:57:09.984256+0000","last_active":"2024-04-25T09:57:09.986423+0000","last_peered":"2024-04-25T09:57:09.986423+0000","last_clean":"2024-04-25T09:57:09.986423+0000","last_became_active":"2024-04-25T09:57:09.983972+0000","last_became_peered":"2024-04-25T09:57:09.983972+0000","last_unstale":"2024-04-25T09:57:09.986423+0000","last_undegraded":"2024-04-25T09:57:09.986423+0000","last_fullsized":"2024-04-25T09:57:09.986423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:23.222733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975678+0000","last_change":"2024-04-25T09:57:09.975678+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975678+0000","last_undegraded":"2024-04-25T09:57:09.975678+0000","last_fullsized":"2024-04-25T09:57:09.975678+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992827+0000","last_change":"2024-04-25T09:57:09.991436+0000","last_active":"2024-04-25T09:57:09.992827+0000","last_peered":"2024-04-25T09:57:09.992827+0000","last_clean":"2024-04-25T09:57:09.992827+0000","last_became_active":"2024-04-25T09:57:09.991138+0000","last_became_peered":"2024-04-25T09:57:09.991138+0000","last_unstale":"2024-04-25T09:57:09.992827+0000","last_undegraded":"2024-04-25T09:57:09.992827+0000","last_fullsized":"2024-04-25T09:57:09.992827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:31.815985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982962+0000","last_change":"2024-04-25T09:57:09.982962+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982962+0000","last_undegraded":"2024-04-25T09:57:09.982962+0000","last_fullsized":"2024-04-25T09:57:09.982962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990260+0000","last_change":"2024-04-25T09:57:09.988005+0000","last_active":"2024-04-25T09:57:09.990260+0000","last_peered":"2024-04-25T09:57:09.990260+0000","last_clean":"2024-04-25T09:57:09.990260+0000","last_became_active":"2024-04-25T09:57:09.987706+0000","last_became_peered":"2024-04-25T09:57:09.987706+0000","last_unstale":"2024-04-25T09:57:09.990260+0000","last_undegraded":"2024-04-25T09:57:09.990260+0000","last_fullsized":"2024-04-25T09:57:09.990260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:48.802471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977199+0000","last_change":"2024-04-25T09:57:09.977199+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977199+0000","last_undegraded":"2024-04-25T09:57:09.977199+0000","last_fullsized":"2024-04-25T09:57:09.977199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994711+0000","last_change":"2024-04-25T09:57:09.990696+0000","last_active":"2024-04-25T09:57:09.994711+0000","last_peered":"2024-04-25T09:57:09.994711+0000","last_clean":"2024-04-25T09:57:09.994711+0000","last_became_active":"2024-04-25T09:57:09.990433+0000","last_became_peered":"2024-04-25T09:57:09.990433+0000","last_unstale":"2024-04-25T09:57:09.994711+0000","last_undegraded":"2024-04-25T09:57:09.994711+0000","last_fullsized":"2024-04-25T09:57:09.994711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980632+0000","last_change":"2024-04-25T09:57:09.980632+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980632+0000","last_undegraded":"2024-04-25T09:57:09.980632+0000","last_fullsized":"2024-04-25T09:57:09.980632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997423+0000","last_change":"2024-04-25T09:57:09.995784+0000","last_active":"2024-04-25T09:57:09.997423+0000","last_peered":"2024-04-25T09:57:09.997423+0000","last_clean":"2024-04-25T09:57:09.997423+0000","last_became_active":"2024-04-25T09:57:09.995393+0000","last_became_peered":"2024-04-25T09:57:09.995393+0000","last_unstale":"2024-04-25T09:57:09.997423+0000","last_undegraded":"2024-04-25T09:57:09.997423+0000","last_fullsized":"2024-04-25T09:57:09.997423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:44.157462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982756+0000","last_change":"2024-04-25T09:57:09.982756+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982756+0000","last_undegraded":"2024-04-25T09:57:09.982756+0000","last_fullsized":"2024-04-25T09:57:09.982756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994920+0000","last_change":"2024-04-25T09:57:09.993056+0000","last_active":"2024-04-25T09:57:09.994920+0000","last_peered":"2024-04-25T09:57:09.994920+0000","last_clean":"2024-04-25T09:57:09.994920+0000","last_became_active":"2024-04-25T09:57:09.992768+0000","last_became_peered":"2024-04-25T09:57:09.992768+0000","last_unstale":"2024-04-25T09:57:09.994920+0000","last_undegraded":"2024-04-25T09:57:09.994920+0000","last_fullsized":"2024-04-25T09:57:09.994920+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980965+0000","last_change":"2024-04-25T09:57:09.980965+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980965+0000","last_undegraded":"2024-04-25T09:57:09.980965+0000","last_fullsized":"2024-04-25T09:57:09.980965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994590+0000","last_change":"2024-04-25T09:57:09.987312+0000","last_active":"2024-04-25T09:57:09.994590+0000","last_peered":"2024-04-25T09:57:09.994590+0000","last_clean":"2024-04-25T09:57:09.994590+0000","last_became_active":"2024-04-25T09:57:09.987064+0000","last_became_peered":"2024-04-25T09:57:09.987064+0000","last_unstale":"2024-04-25T09:57:09.994590+0000","last_undegraded":"2024-04-25T09:57:09.994590+0000","last_fullsized":"2024-04-25T09:57:09.994590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.889960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977222+0000","last_change":"2024-04-25T09:57:09.977222+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977222+0000","last_undegraded":"2024-04-25T09:57:09.977222+0000","last_fullsized":"2024-04-25T09:57:09.977222+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986936+0000","last_change":"2024-04-25T09:57:09.985025+0000","last_active":"2024-04-25T09:57:09.986936+0000","last_peered":"2024-04-25T09:57:09.986936+0000","last_clean":"2024-04-25T09:57:09.986936+0000","last_became_active":"2024-04-25T09:57:09.984357+0000","last_became_peered":"2024-04-25T09:57:09.984357+0000","last_unstale":"2024-04-25T09:57:09.986936+0000","last_undegraded":"2024-04-25T09:57:09.986936+0000","last_fullsized":"2024-04-25T09:57:09.986936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:11.720778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981755+0000","last_change":"2024-04-25T09:57:09.981755+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981755+0000","last_undegraded":"2024-04-25T09:57:09.981755+0000","last_fullsized":"2024-04-25T09:57:09.981755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992985+0000","last_change":"2024-04-25T09:57:09.991923+0000","last_active":"2024-04-25T09:57:09.992985+0000","last_peered":"2024-04-25T09:57:09.992985+0000","last_clean":"2024-04-25T09:57:09.992985+0000","last_became_active":"2024-04-25T09:57:09.991642+0000","last_became_peered":"2024-04-25T09:57:09.991642+0000","last_unstale":"2024-04-25T09:57:09.992985+0000","last_undegraded":"2024-04-25T09:57:09.992985+0000","last_fullsized":"2024-04-25T09:57:09.992985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.657727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977462+0000","last_change":"2024-04-25T09:57:09.977462+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977462+0000","last_undegraded":"2024-04-25T09:57:09.977462+0000","last_fullsized":"2024-04-25T09:57:09.977462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.984066+0000","last_change":"2024-04-25T09:57:09.982146+0000","last_active":"2024-04-25T09:57:09.984066+0000","last_peered":"2024-04-25T09:57:09.984066+0000","last_clean":"2024-04-25T09:57:09.984066+0000","last_became_active":"2024-04-25T09:57:09.981876+0000","last_became_peered":"2024-04-25T09:57:09.981876+0000","last_unstale":"2024-04-25T09:57:09.984066+0000","last_undegraded":"2024-04-25T09:57:09.984066+0000","last_fullsized":"2024-04-25T09:57:09.984066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982765+0000","last_change":"2024-04-25T09:57:09.982765+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982765+0000","last_undegraded":"2024-04-25T09:57:09.982765+0000","last_fullsized":"2024-04-25T09:57:09.982765+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997380+0000","last_change":"2024-04-25T09:57:09.995462+0000","last_active":"2024-04-25T09:57:09.997380+0000","last_peered":"2024-04-25T09:57:09.997380+0000","last_clean":"2024-04-25T09:57:09.997380+0000","last_became_active":"2024-04-25T09:57:09.995179+0000","last_became_peered":"2024-04-25T09:57:09.995179+0000","last_unstale":"2024-04-25T09:57:09.997380+0000","last_undegraded":"2024-04-25T09:57:09.997380+0000","last_fullsized":"2024-04-25T09:57:09.997380+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.011448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977753+0000","last_change":"2024-04-25T09:57:09.977753+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977753+0000","last_undegraded":"2024-04-25T09:57:09.977753+0000","last_fullsized":"2024-04-25T09:57:09.977753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.996932+0000","last_change":"2024-04-25T09:57:09.994224+0000","last_active":"2024-04-25T09:57:09.996932+0000","last_peered":"2024-04-25T09:57:09.996932+0000","last_clean":"2024-04-25T09:57:09.996932+0000","last_became_active":"2024-04-25T09:57:09.993902+0000","last_became_peered":"2024-04-25T09:57:09.993902+0000","last_unstale":"2024-04-25T09:57:09.996932+0000","last_undegraded":"2024-04-25T09:57:09.996932+0000","last_fullsized":"2024-04-25T09:57:09.996932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983273+0000","last_change":"2024-04-25T09:57:09.983273+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983273+0000","last_undegraded":"2024-04-25T09:57:09.983273+0000","last_fullsized":"2024-04-25T09:57:09.983273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989862+0000","last_change":"2024-04-25T09:57:09.988098+0000","last_active":"2024-04-25T09:57:09.989862+0000","last_peered":"2024-04-25T09:57:09.989862+0000","last_clean":"2024-04-25T09:57:09.989862+0000","last_became_active":"2024-04-25T09:57:09.987672+0000","last_became_peered":"2024-04-25T09:57:09.987672+0000","last_unstale":"2024-04-25T09:57:09.989862+0000","last_undegraded":"2024-04-25T09:57:09.989862+0000","last_fullsized":"2024-04-25T09:57:09.989862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981392+0000","last_change":"2024-04-25T09:57:09.981392+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981392+0000","last_undegraded":"2024-04-25T09:57:09.981392+0000","last_fullsized":"2024-04-25T09:57:09.981392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987441+0000","last_change":"2024-04-25T09:57:09.985927+0000","last_active":"2024-04-25T09:57:09.987441+0000","last_peered":"2024-04-25T09:57:09.987441+0000","last_clean":"2024-04-25T09:57:09.987441+0000","last_became_active":"2024-04-25T09:57:09.985657+0000","last_became_peered":"2024-04-25T09:57:09.985657+0000","last_unstale":"2024-04-25T09:57:09.987441+0000","last_undegraded":"2024-04-25T09:57:09.987441+0000","last_fullsized":"2024-04-25T09:57:09.987441+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981050+0000","last_change":"2024-04-25T09:57:09.981050+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981050+0000","last_undegraded":"2024-04-25T09:57:09.981050+0000","last_fullsized":"2024-04-25T09:57:09.981050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991567+0000","last_change":"2024-04-25T09:57:09.990770+0000","last_active":"2024-04-25T09:57:09.991567+0000","last_peered":"2024-04-25T09:57:09.991567+0000","last_clean":"2024-04-25T09:57:09.991567+0000","last_became_active":"2024-04-25T09:57:09.990495+0000","last_became_peered":"2024-04-25T09:57:09.990495+0000","last_unstale":"2024-04-25T09:57:09.991567+0000","last_undegraded":"2024-04-25T09:57:09.991567+0000","last_fullsized":"2024-04-25T09:57:09.991567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.981492+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:09.981492+0000","last_peered":"2024-04-25T09:57:09.981492+0000","last_clean":"2024-04-25T09:57:09.981492+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:09.981492+0000","last_undegraded":"2024-04-25T09:57:09.981492+0000","last_fullsized":"2024-04-25T09:57:09.981492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":866229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1224704,"data_stored":872896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:14.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.405+0000 7f7999d6a640 1 Processor -- start 2024-04-25T09:57:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.405+0000 7f7999d6a640 1 -- start start 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7999d6a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f799415ba00 0x7f799409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7999d6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f799415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7999d6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79940cd710 con 0x7f79941602e0 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7999d6a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79940cd4b0 con 0x7f799415ba00 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7999d6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7994099b70 con 0x7f799415f5f0 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79937fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f799415ba00 0x7f799409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7992ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f79941602e0 0x7f79940ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38976/0 (socket says 172.21.15.148:38976) 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7992ffd640 1 -- 172.21.15.148:0/261478894 learned_addr learned my addr 172.21.15.148:0/261478894 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:14.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79937fe640 1 --2- 172.21.15.148:0/261478894 >> v2:172.21.15.148:3300/0 conn(0x7f799415ba00 0x7f799409a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43782/0 (socket says 172.21.15.148:43782) 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f7993fff640 1 --2- 172.21.15.148:0/261478894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f799415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79937fe640 1 -- 172.21.15.148:0/261478894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 msgr2=0x7f799415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79937fe640 1 --2- 172.21.15.148:0/261478894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f799415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79937fe640 1 -- 172.21.15.148:0/261478894 >> v1:172.21.15.148:6789/0 conn(0x7f79941602e0 legacy=0x7f79940ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79927fc640 1 -- 172.21.15.148:0/261478894 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014085910 0 0) 0x7f7988001110 con 0x7f79941602e0 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.406+0000 7f79937fe640 1 -- 172.21.15.148:0/261478894 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994099e50 con 0x7f799415ba00 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.407+0000 7f79937fe640 1 --2- 172.21.15.148:0/261478894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799415ba00 0x7f799409a930 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f79840097a0 tx=0x7f798402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9ab538c5760273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.407+0000 7f79927fc640 1 -- 172.21.15.148:0/261478894 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984037070 con 0x7f799415ba00 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.407+0000 7f79927fc640 1 -- 172.21.15.148:0/261478894 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798402adb0 con 0x7f799415ba00 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.407+0000 7f79927fc640 1 -- 172.21.15.148:0/261478894 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984032660 con 0x7f799415ba00 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 -- 172.21.15.148:0/261478894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799415ba00 msgr2=0x7f799409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 --2- 172.21.15.148:0/261478894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799415ba00 0x7f799409a930 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f79840097a0 tx=0x7f798402a060 comp rx=0 tx=0).stop 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 -- 172.21.15.148:0/261478894 shutdown_connections 2024-04-25T09:57:14.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 --2- 172.21.15.148:0/261478894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f799415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 --2- 172.21.15.148:0/261478894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799415ba00 0x7f799409a930 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 -- 172.21.15.148:0/261478894 >> 172.21.15.148:0/261478894 conn(0x7f79940b3270 msgr2=0x7f79940b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 -- 172.21.15.148:0/261478894 shutdown_connections 2024-04-25T09:57:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.408+0000 7f7999d6a640 1 -- 172.21.15.148:0/261478894 wait complete. 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.409+0000 7f7999d6a640 1 Processor -- start 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.409+0000 7f7999d6a640 1 -- start start 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7999d6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f799415ba00 0x7f79941581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7999d6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f79940b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7999d6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 0x7f79940b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7999d6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799415bde0 con 0x7f79941602e0 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7999d6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799409ae70 con 0x7f799415ba00 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7999d6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7994158710 con 0x7f799415f5f0 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 0x7f79940b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 0x7f79940b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43784/0 (socket says 172.21.15.148:43784) 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 -- 172.21.15.148:0/3085633992 learned_addr learned my addr 172.21.15.148:0/3085633992 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7992ffd640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f79940b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f79937fe640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f799415ba00 0x7f79941581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 -- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 msgr2=0x7f79940b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f79940b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 -- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f799415ba00 msgr2=0x7f79941581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f799415ba00 0x7f79941581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 -- 172.21.15.148:0/3085633992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799415ea30 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f7993fff640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 0x7f79940b48e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f798c007e60 tx=0x7f798c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c014420 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.410+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c019070 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.411+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799415e190 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.411+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c014820 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.411+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799415e750 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.411+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799406b020 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.412+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f798c0149c0 con 0x7f79941602e0 2024-04-25T09:57:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.412+0000 7f797bfff640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f798006b4d0 0x7f798006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.412+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f798c08ac20 con 0x7f79941602e0 2024-04-25T09:57:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.412+0000 7f79937fe640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f798006b4d0 0x7f798006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.413+0000 7f79937fe640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f798006b4d0 0x7f798006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7984034000 tx=0x7f798402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.417+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f799406b020 con 0x7f79941602e0 2024-04-25T09:57:14.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.584+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79940d0130 con 0x7f798006b4d0 2024-04-25T09:57:14.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.589+0000 7f797bfff640 1 -- 172.21.15.148:0/3085633992 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7f79940d0130 con 0x7f798006b4d0 2024-04-25T09:57:14.596 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:14.597 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:57:14.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.592+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f798006b4d0 msgr2=0x7f798006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.592+0000 7f7999d6a640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f798006b4d0 0x7f798006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7984034000 tx=0x7f798402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:14.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 msgr2=0x7f79940b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 0x7f79940b48e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f798c007e60 tx=0x7f798c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 shutdown_connections 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f798006b4d0 0x7f798006d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79941602e0 0x7f79940b48e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f799415f5f0 0x7f79940b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 --2- 172.21.15.148:0/3085633992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f799415ba00 0x7f79941581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 >> 172.21.15.148:0/3085633992 conn(0x7f79940b3270 msgr2=0x7f79940c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 shutdown_connections 2024-04-25T09:57:14.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.593+0000 7f7999d6a640 1 -- 172.21.15.148:0/3085633992 wait complete. 2024-04-25T09:57:14.607 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-25T09:57:13.054699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276252,"kb_used_data":6004,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754698468,"statfs":{"total":773094113280,"available":772811231232,"internally_reserved":0,"allocated":6148096,"data_stored":3394291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.087327"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981364+0000","last_change":"2024-04-25T09:57:09.981364+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981364+0000","last_undegraded":"2024-04-25T09:57:09.981364+0000","last_fullsized":"2024-04-25T09:57:09.981364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980666+0000","last_change":"2024-04-25T09:57:09.980666+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980666+0000","last_undegraded":"2024-04-25T09:57:09.980666+0000","last_fullsized":"2024-04-25T09:57:09.980666+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987160+0000","last_change":"2024-04-25T09:57:09.987160+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987160+0000","last_undegraded":"2024-04-25T09:57:09.987160+0000","last_fullsized":"2024-04-25T09:57:09.987160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980352+0000","last_change":"2024-04-25T09:57:09.980352+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980352+0000","last_undegraded":"2024-04-25T09:57:09.980352+0000","last_fullsized":"2024-04-25T09:57:09.980352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976437+0000","last_change":"2024-04-25T09:57:09.976437+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976437+0000","last_undegraded":"2024-04-25T09:57:09.976437+0000","last_fullsized":"2024-04-25T09:57:09.976437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982151+0000","last_change":"2024-04-25T09:57:09.982151+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982151+0000","last_undegraded":"2024-04-25T09:57:09.982151+0000","last_fullsized":"2024-04-25T09:57:09.982151+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986707+0000","last_change":"2024-04-25T09:57:09.986707+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986707+0000","last_undegraded":"2024-04-25T09:57:09.986707+0000","last_fullsized":"2024-04-25T09:57:09.986707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987280+0000","last_change":"2024-04-25T09:57:09.987280+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987280+0000","last_undegraded":"2024-04-25T09:57:09.987280+0000","last_fullsized":"2024-04-25T09:57:09.987280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982972+0000","last_change":"2024-04-25T09:57:09.982972+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982972+0000","last_undegraded":"2024-04-25T09:57:09.982972+0000","last_fullsized":"2024-04-25T09:57:09.982972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983088+0000","last_change":"2024-04-25T09:57:09.983088+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983088+0000","last_undegraded":"2024-04-25T09:57:09.983088+0000","last_fullsized":"2024-04-25T09:57:09.983088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977943+0000","last_change":"2024-04-25T09:57:09.977943+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977943+0000","last_undegraded":"2024-04-25T09:57:09.977943+0000","last_fullsized":"2024-04-25T09:57:09.977943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979313+0000","last_change":"2024-04-25T09:57:09.979313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979313+0000","last_undegraded":"2024-04-25T09:57:09.979313+0000","last_fullsized":"2024-04-25T09:57:09.979313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983403+0000","last_change":"2024-04-25T09:57:09.983403+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983403+0000","last_undegraded":"2024-04-25T09:57:09.983403+0000","last_fullsized":"2024-04-25T09:57:09.983403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986278+0000","last_change":"2024-04-25T09:57:09.986278+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986278+0000","last_undegraded":"2024-04-25T09:57:09.986278+0000","last_fullsized":"2024-04-25T09:57:09.986278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979368+0000","last_change":"2024-04-25T09:57:09.979368+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979368+0000","last_undegraded":"2024-04-25T09:57:09.979368+0000","last_fullsized":"2024-04-25T09:57:09.979368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980509+0000","last_change":"2024-04-25T09:57:09.980509+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980509+0000","last_undegraded":"2024-04-25T09:57:09.980509+0000","last_fullsized":"2024-04-25T09:57:09.980509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987736+0000","last_change":"2024-04-25T09:57:09.987736+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987736+0000","last_undegraded":"2024-04-25T09:57:09.987736+0000","last_fullsized":"2024-04-25T09:57:09.987736+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978066+0000","last_change":"2024-04-25T09:57:09.978066+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978066+0000","last_undegraded":"2024-04-25T09:57:09.978066+0000","last_fullsized":"2024-04-25T09:57:09.978066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985583+0000","last_change":"2024-04-25T09:57:09.985583+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985583+0000","last_undegraded":"2024-04-25T09:57:09.985583+0000","last_fullsized":"2024-04-25T09:57:09.985583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982992+0000","last_change":"2024-04-25T09:57:09.982992+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982992+0000","last_undegraded":"2024-04-25T09:57:09.982992+0000","last_fullsized":"2024-04-25T09:57:09.982992+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981564+0000","last_change":"2024-04-25T09:57:09.981564+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981564+0000","last_undegraded":"2024-04-25T09:57:09.981564+0000","last_fullsized":"2024-04-25T09:57:09.981564+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980454+0000","last_change":"2024-04-25T09:57:09.980454+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980454+0000","last_undegraded":"2024-04-25T09:57:09.980454+0000","last_fullsized":"2024-04-25T09:57:09.980454+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977821+0000","last_change":"2024-04-25T09:57:09.977821+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977821+0000","last_undegraded":"2024-04-25T09:57:09.977821+0000","last_fullsized":"2024-04-25T09:57:09.977821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985146+0000","last_change":"2024-04-25T09:57:09.985146+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985146+0000","last_undegraded":"2024-04-25T09:57:09.985146+0000","last_fullsized":"2024-04-25T09:57:09.985146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985837+0000","last_change":"2024-04-25T09:57:09.985837+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985837+0000","last_undegraded":"2024-04-25T09:57:09.985837+0000","last_fullsized":"2024-04-25T09:57:09.985837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980855+0000","last_change":"2024-04-25T09:57:09.980855+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980855+0000","last_undegraded":"2024-04-25T09:57:09.980855+0000","last_fullsized":"2024-04-25T09:57:09.980855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980652+0000","last_change":"2024-04-25T09:57:09.980652+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980652+0000","last_undegraded":"2024-04-25T09:57:09.980652+0000","last_fullsized":"2024-04-25T09:57:09.980652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981345+0000","last_change":"2024-04-25T09:57:09.981345+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981345+0000","last_undegraded":"2024-04-25T09:57:09.981345+0000","last_fullsized":"2024-04-25T09:57:09.981345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984479+0000","last_change":"2024-04-25T09:57:09.984479+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984479+0000","last_undegraded":"2024-04-25T09:57:09.984479+0000","last_fullsized":"2024-04-25T09:57:09.984479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979739+0000","last_change":"2024-04-25T09:57:09.979739+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979739+0000","last_undegraded":"2024-04-25T09:57:09.979739+0000","last_fullsized":"2024-04-25T09:57:09.979739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984343+0000","last_change":"2024-04-25T09:57:09.984343+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984343+0000","last_undegraded":"2024-04-25T09:57:09.984343+0000","last_fullsized":"2024-04-25T09:57:09.984343+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978355+0000","last_change":"2024-04-25T09:57:09.978355+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978355+0000","last_undegraded":"2024-04-25T09:57:09.978355+0000","last_fullsized":"2024-04-25T09:57:09.978355+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980990+0000","last_change":"2024-04-25T09:57:09.980990+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980990+0000","last_undegraded":"2024-04-25T09:57:09.980990+0000","last_fullsized":"2024-04-25T09:57:09.980990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989713+0000","last_change":"2024-04-25T09:57:09.986934+0000","last_active":"2024-04-25T09:57:09.989713+0000","last_peered":"2024-04-25T09:57:09.989713+0000","last_clean":"2024-04-25T09:57:09.989713+0000","last_became_active":"2024-04-25T09:57:09.986610+0000","last_became_peered":"2024-04-25T09:57:09.986610+0000","last_unstale":"2024-04-25T09:57:09.989713+0000","last_undegraded":"2024-04-25T09:57:09.989713+0000","last_fullsized":"2024-04-25T09:57:09.989713+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:16.052402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977814+0000","last_change":"2024-04-25T09:57:09.977814+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977814+0000","last_undegraded":"2024-04-25T09:57:09.977814+0000","last_fullsized":"2024-04-25T09:57:09.977814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997056+0000","last_change":"2024-04-25T09:57:09.996107+0000","last_active":"2024-04-25T09:57:09.997056+0000","last_peered":"2024-04-25T09:57:09.997056+0000","last_clean":"2024-04-25T09:57:09.997056+0000","last_became_active":"2024-04-25T09:57:09.995840+0000","last_became_peered":"2024-04-25T09:57:09.995840+0000","last_unstale":"2024-04-25T09:57:09.997056+0000","last_undegraded":"2024-04-25T09:57:09.997056+0000","last_fullsized":"2024-04-25T09:57:09.997056+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980833+0000","last_change":"2024-04-25T09:57:09.980833+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980833+0000","last_undegraded":"2024-04-25T09:57:09.980833+0000","last_fullsized":"2024-04-25T09:57:09.980833+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.993934+0000","last_change":"2024-04-25T09:57:09.988639+0000","last_active":"2024-04-25T09:57:09.993934+0000","last_peered":"2024-04-25T09:57:09.993934+0000","last_clean":"2024-04-25T09:57:09.993934+0000","last_became_active":"2024-04-25T09:57:09.988362+0000","last_became_peered":"2024-04-25T09:57:09.988362+0000","last_unstale":"2024-04-25T09:57:09.993934+0000","last_undegraded":"2024-04-25T09:57:09.993934+0000","last_fullsized":"2024-04-25T09:57:09.993934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.037390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981254+0000","last_change":"2024-04-25T09:57:09.981254+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981254+0000","last_undegraded":"2024-04-25T09:57:09.981254+0000","last_fullsized":"2024-04-25T09:57:09.981254+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986629+0000","last_change":"2024-04-25T09:57:09.981554+0000","last_active":"2024-04-25T09:57:09.986629+0000","last_peered":"2024-04-25T09:57:09.986629+0000","last_clean":"2024-04-25T09:57:09.986629+0000","last_became_active":"2024-04-25T09:57:09.981290+0000","last_became_peered":"2024-04-25T09:57:09.981290+0000","last_unstale":"2024-04-25T09:57:09.986629+0000","last_undegraded":"2024-04-25T09:57:09.986629+0000","last_fullsized":"2024-04-25T09:57:09.986629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:02.596300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975566+0000","last_change":"2024-04-25T09:57:09.975566+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975566+0000","last_undegraded":"2024-04-25T09:57:09.975566+0000","last_fullsized":"2024-04-25T09:57:09.975566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997140+0000","last_change":"2024-04-25T09:57:09.995396+0000","last_active":"2024-04-25T09:57:09.997140+0000","last_peered":"2024-04-25T09:57:09.997140+0000","last_clean":"2024-04-25T09:57:09.997140+0000","last_became_active":"2024-04-25T09:57:09.995117+0000","last_became_peered":"2024-04-25T09:57:09.995117+0000","last_unstale":"2024-04-25T09:57:09.997140+0000","last_undegraded":"2024-04-25T09:57:09.997140+0000","last_fullsized":"2024-04-25T09:57:09.997140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:27.807847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982554+0000","last_change":"2024-04-25T09:57:09.982554+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982554+0000","last_undegraded":"2024-04-25T09:57:09.982554+0000","last_fullsized":"2024-04-25T09:57:09.982554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987303+0000","last_change":"2024-04-25T09:57:09.985140+0000","last_active":"2024-04-25T09:57:09.987303+0000","last_peered":"2024-04-25T09:57:09.987303+0000","last_clean":"2024-04-25T09:57:09.987303+0000","last_became_active":"2024-04-25T09:57:09.983854+0000","last_became_peered":"2024-04-25T09:57:09.983854+0000","last_unstale":"2024-04-25T09:57:09.987303+0000","last_undegraded":"2024-04-25T09:57:09.987303+0000","last_fullsized":"2024-04-25T09:57:09.987303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979321+0000","last_change":"2024-04-25T09:57:09.979321+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979321+0000","last_undegraded":"2024-04-25T09:57:09.979321+0000","last_fullsized":"2024-04-25T09:57:09.979321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987086+0000","last_change":"2024-04-25T09:57:09.985009+0000","last_active":"2024-04-25T09:57:09.987086+0000","last_peered":"2024-04-25T09:57:09.987086+0000","last_clean":"2024-04-25T09:57:09.987086+0000","last_became_active":"2024-04-25T09:57:09.984733+0000","last_became_peered":"2024-04-25T09:57:09.984733+0000","last_unstale":"2024-04-25T09:57:09.987086+0000","last_undegraded":"2024-04-25T09:57:09.987086+0000","last_fullsized":"2024-04-25T09:57:09.987086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.282213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978986+0000","last_change":"2024-04-25T09:57:09.978986+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978986+0000","last_undegraded":"2024-04-25T09:57:09.978986+0000","last_fullsized":"2024-04-25T09:57:09.978986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991792+0000","last_change":"2024-04-25T09:57:09.988494+0000","last_active":"2024-04-25T09:57:09.991792+0000","last_peered":"2024-04-25T09:57:09.991792+0000","last_clean":"2024-04-25T09:57:09.991792+0000","last_became_active":"2024-04-25T09:57:09.988229+0000","last_became_peered":"2024-04-25T09:57:09.988229+0000","last_unstale":"2024-04-25T09:57:09.991792+0000","last_undegraded":"2024-04-25T09:57:09.991792+0000","last_fullsized":"2024-04-25T09:57:09.991792+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975670+0000","last_change":"2024-04-25T09:57:09.975670+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975670+0000","last_undegraded":"2024-04-25T09:57:09.975670+0000","last_fullsized":"2024-04-25T09:57:09.975670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990292+0000","last_change":"2024-04-25T09:57:09.988417+0000","last_active":"2024-04-25T09:57:09.990292+0000","last_peered":"2024-04-25T09:57:09.990292+0000","last_clean":"2024-04-25T09:57:09.990292+0000","last_became_active":"2024-04-25T09:57:09.987870+0000","last_became_peered":"2024-04-25T09:57:09.987870+0000","last_unstale":"2024-04-25T09:57:09.990292+0000","last_undegraded":"2024-04-25T09:57:09.990292+0000","last_fullsized":"2024-04-25T09:57:09.990292+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.029172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978288+0000","last_change":"2024-04-25T09:57:09.978288+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978288+0000","last_undegraded":"2024-04-25T09:57:09.978288+0000","last_fullsized":"2024-04-25T09:57:09.978288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985102+0000","last_change":"2024-04-25T09:57:09.981400+0000","last_active":"2024-04-25T09:57:09.985102+0000","last_peered":"2024-04-25T09:57:09.985102+0000","last_clean":"2024-04-25T09:57:09.985102+0000","last_became_active":"2024-04-25T09:57:09.981081+0000","last_became_peered":"2024-04-25T09:57:09.981081+0000","last_unstale":"2024-04-25T09:57:09.985102+0000","last_undegraded":"2024-04-25T09:57:09.985102+0000","last_fullsized":"2024-04-25T09:57:09.985102+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983313+0000","last_change":"2024-04-25T09:57:09.983313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983313+0000","last_undegraded":"2024-04-25T09:57:09.983313+0000","last_fullsized":"2024-04-25T09:57:09.983313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992963+0000","last_change":"2024-04-25T09:57:09.991460+0000","last_active":"2024-04-25T09:57:09.992963+0000","last_peered":"2024-04-25T09:57:09.992963+0000","last_clean":"2024-04-25T09:57:09.992963+0000","last_became_active":"2024-04-25T09:57:09.991137+0000","last_became_peered":"2024-04-25T09:57:09.991137+0000","last_unstale":"2024-04-25T09:57:09.992963+0000","last_undegraded":"2024-04-25T09:57:09.992963+0000","last_fullsized":"2024-04-25T09:57:09.992963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.780147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976028+0000","last_change":"2024-04-25T09:57:09.976028+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976028+0000","last_undegraded":"2024-04-25T09:57:09.976028+0000","last_fullsized":"2024-04-25T09:57:09.976028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997454+0000","last_change":"2024-04-25T09:57:09.996022+0000","last_active":"2024-04-25T09:57:09.997454+0000","last_peered":"2024-04-25T09:57:09.997454+0000","last_clean":"2024-04-25T09:57:09.997454+0000","last_became_active":"2024-04-25T09:57:09.995252+0000","last_became_peered":"2024-04-25T09:57:09.995252+0000","last_unstale":"2024-04-25T09:57:09.997454+0000","last_undegraded":"2024-04-25T09:57:09.997454+0000","last_fullsized":"2024-04-25T09:57:09.997454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:03.771862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978626+0000","last_change":"2024-04-25T09:57:09.978626+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978626+0000","last_undegraded":"2024-04-25T09:57:09.978626+0000","last_fullsized":"2024-04-25T09:57:09.978626+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991523+0000","last_change":"2024-04-25T09:57:09.988947+0000","last_active":"2024-04-25T09:57:09.991523+0000","last_peered":"2024-04-25T09:57:09.991523+0000","last_clean":"2024-04-25T09:57:09.991523+0000","last_became_active":"2024-04-25T09:57:09.988669+0000","last_became_peered":"2024-04-25T09:57:09.988669+0000","last_unstale":"2024-04-25T09:57:09.991523+0000","last_undegraded":"2024-04-25T09:57:09.991523+0000","last_fullsized":"2024-04-25T09:57:09.991523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983786+0000","last_change":"2024-04-25T09:57:09.983786+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983786+0000","last_undegraded":"2024-04-25T09:57:09.983786+0000","last_fullsized":"2024-04-25T09:57:09.983786+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985528+0000","last_change":"2024-04-25T09:57:09.983965+0000","last_active":"2024-04-25T09:57:09.985528+0000","last_peered":"2024-04-25T09:57:09.985528+0000","last_clean":"2024-04-25T09:57:09.985528+0000","last_became_active":"2024-04-25T09:57:09.983606+0000","last_became_peered":"2024-04-25T09:57:09.983606+0000","last_unstale":"2024-04-25T09:57:09.985528+0000","last_undegraded":"2024-04-25T09:57:09.985528+0000","last_fullsized":"2024-04-25T09:57:09.985528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.673019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979732+0000","last_change":"2024-04-25T09:57:09.979732+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979732+0000","last_undegraded":"2024-04-25T09:57:09.979732+0000","last_fullsized":"2024-04-25T09:57:09.979732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989924+0000","last_change":"2024-04-25T09:57:09.986995+0000","last_active":"2024-04-25T09:57:09.989924+0000","last_peered":"2024-04-25T09:57:09.989924+0000","last_clean":"2024-04-25T09:57:09.989924+0000","last_became_active":"2024-04-25T09:57:09.986584+0000","last_became_peered":"2024-04-25T09:57:09.986584+0000","last_unstale":"2024-04-25T09:57:09.989924+0000","last_undegraded":"2024-04-25T09:57:09.989924+0000","last_fullsized":"2024-04-25T09:57:09.989924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:46.383736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984631+0000","last_change":"2024-04-25T09:57:09.984631+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984631+0000","last_undegraded":"2024-04-25T09:57:09.984631+0000","last_fullsized":"2024-04-25T09:57:09.984631+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997402+0000","last_change":"2024-04-25T09:57:09.995540+0000","last_active":"2024-04-25T09:57:09.997402+0000","last_peered":"2024-04-25T09:57:09.997402+0000","last_clean":"2024-04-25T09:57:09.997402+0000","last_became_active":"2024-04-25T09:57:09.995243+0000","last_became_peered":"2024-04-25T09:57:09.995243+0000","last_unstale":"2024-04-25T09:57:09.997402+0000","last_undegraded":"2024-04-25T09:57:09.997402+0000","last_fullsized":"2024-04-25T09:57:09.997402+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:32.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978354+0000","last_change":"2024-04-25T09:57:09.978354+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978354+0000","last_undegraded":"2024-04-25T09:57:09.978354+0000","last_fullsized":"2024-04-25T09:57:09.978354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997145+0000","last_change":"2024-04-25T09:57:09.995365+0000","last_active":"2024-04-25T09:57:09.997145+0000","last_peered":"2024-04-25T09:57:09.997145+0000","last_clean":"2024-04-25T09:57:09.997145+0000","last_became_active":"2024-04-25T09:57:09.995083+0000","last_became_peered":"2024-04-25T09:57:09.995083+0000","last_unstale":"2024-04-25T09:57:09.997145+0000","last_undegraded":"2024-04-25T09:57:09.997145+0000","last_fullsized":"2024-04-25T09:57:09.997145+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:57.260934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982863+0000","last_change":"2024-04-25T09:57:09.982863+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982863+0000","last_undegraded":"2024-04-25T09:57:09.982863+0000","last_fullsized":"2024-04-25T09:57:09.982863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986423+0000","last_change":"2024-04-25T09:57:09.984256+0000","last_active":"2024-04-25T09:57:09.986423+0000","last_peered":"2024-04-25T09:57:09.986423+0000","last_clean":"2024-04-25T09:57:09.986423+0000","last_became_active":"2024-04-25T09:57:09.983972+0000","last_became_peered":"2024-04-25T09:57:09.983972+0000","last_unstale":"2024-04-25T09:57:09.986423+0000","last_undegraded":"2024-04-25T09:57:09.986423+0000","last_fullsized":"2024-04-25T09:57:09.986423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:23.222733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975678+0000","last_change":"2024-04-25T09:57:09.975678+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975678+0000","last_undegraded":"2024-04-25T09:57:09.975678+0000","last_fullsized":"2024-04-25T09:57:09.975678+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992827+0000","last_change":"2024-04-25T09:57:09.991436+0000","last_active":"2024-04-25T09:57:09.992827+0000","last_peered":"2024-04-25T09:57:09.992827+0000","last_clean":"2024-04-25T09:57:09.992827+0000","last_became_active":"2024-04-25T09:57:09.991138+0000","last_became_peered":"2024-04-25T09:57:09.991138+0000","last_unstale":"2024-04-25T09:57:09.992827+0000","last_undegraded":"2024-04-25T09:57:09.992827+0000","last_fullsized":"2024-04-25T09:57:09.992827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:31.815985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982962+0000","last_change":"2024-04-25T09:57:09.982962+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982962+0000","last_undegraded":"2024-04-25T09:57:09.982962+0000","last_fullsized":"2024-04-25T09:57:09.982962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990260+0000","last_change":"2024-04-25T09:57:09.988005+0000","last_active":"2024-04-25T09:57:09.990260+0000","last_peered":"2024-04-25T09:57:09.990260+0000","last_clean":"2024-04-25T09:57:09.990260+0000","last_became_active":"2024-04-25T09:57:09.987706+0000","last_became_peered":"2024-04-25T09:57:09.987706+0000","last_unstale":"2024-04-25T09:57:09.990260+0000","last_undegraded":"2024-04-25T09:57:09.990260+0000","last_fullsized":"2024-04-25T09:57:09.990260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:48.802471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977199+0000","last_change":"2024-04-25T09:57:09.977199+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977199+0000","last_undegraded":"2024-04-25T09:57:09.977199+0000","last_fullsized":"2024-04-25T09:57:09.977199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994711+0000","last_change":"2024-04-25T09:57:09.990696+0000","last_active":"2024-04-25T09:57:09.994711+0000","last_peered":"2024-04-25T09:57:09.994711+0000","last_clean":"2024-04-25T09:57:09.994711+0000","last_became_active":"2024-04-25T09:57:09.990433+0000","last_became_peered":"2024-04-25T09:57:09.990433+0000","last_unstale":"2024-04-25T09:57:09.994711+0000","last_undegraded":"2024-04-25T09:57:09.994711+0000","last_fullsized":"2024-04-25T09:57:09.994711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980632+0000","last_change":"2024-04-25T09:57:09.980632+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980632+0000","last_undegraded":"2024-04-25T09:57:09.980632+0000","last_fullsized":"2024-04-25T09:57:09.980632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997423+0000","last_change":"2024-04-25T09:57:09.995784+0000","last_active":"2024-04-25T09:57:09.997423+0000","last_peered":"2024-04-25T09:57:09.997423+0000","last_clean":"2024-04-25T09:57:09.997423+0000","last_became_active":"2024-04-25T09:57:09.995393+0000","last_became_peered":"2024-04-25T09:57:09.995393+0000","last_unstale":"2024-04-25T09:57:09.997423+0000","last_undegraded":"2024-04-25T09:57:09.997423+0000","last_fullsized":"2024-04-25T09:57:09.997423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:44.157462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982756+0000","last_change":"2024-04-25T09:57:09.982756+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982756+0000","last_undegraded":"2024-04-25T09:57:09.982756+0000","last_fullsized":"2024-04-25T09:57:09.982756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994920+0000","last_change":"2024-04-25T09:57:09.993056+0000","last_active":"2024-04-25T09:57:09.994920+0000","last_peered":"2024-04-25T09:57:09.994920+0000","last_clean":"2024-04-25T09:57:09.994920+0000","last_became_active":"2024-04-25T09:57:09.992768+0000","last_became_peered":"2024-04-25T09:57:09.992768+0000","last_unstale":"2024-04-25T09:57:09.994920+0000","last_undegraded":"2024-04-25T09:57:09.994920+0000","last_fullsized":"2024-04-25T09:57:09.994920+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980965+0000","last_change":"2024-04-25T09:57:09.980965+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980965+0000","last_undegraded":"2024-04-25T09:57:09.980965+0000","last_fullsized":"2024-04-25T09:57:09.980965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994590+0000","last_change":"2024-04-25T09:57:09.987312+0000","last_active":"2024-04-25T09:57:09.994590+0000","last_peered":"2024-04-25T09:57:09.994590+0000","last_clean":"2024-04-25T09:57:09.994590+0000","last_became_active":"2024-04-25T09:57:09.987064+0000","last_became_peered":"2024-04-25T09:57:09.987064+0000","last_unstale":"2024-04-25T09:57:09.994590+0000","last_undegraded":"2024-04-25T09:57:09.994590+0000","last_fullsized":"2024-04-25T09:57:09.994590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.889960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977222+0000","last_change":"2024-04-25T09:57:09.977222+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977222+0000","last_undegraded":"2024-04-25T09:57:09.977222+0000","last_fullsized":"2024-04-25T09:57:09.977222+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986936+0000","last_change":"2024-04-25T09:57:09.985025+0000","last_active":"2024-04-25T09:57:09.986936+0000","last_peered":"2024-04-25T09:57:09.986936+0000","last_clean":"2024-04-25T09:57:09.986936+0000","last_became_active":"2024-04-25T09:57:09.984357+0000","last_became_peered":"2024-04-25T09:57:09.984357+0000","last_unstale":"2024-04-25T09:57:09.986936+0000","last_undegraded":"2024-04-25T09:57:09.986936+0000","last_fullsized":"2024-04-25T09:57:09.986936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:11.720778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981755+0000","last_change":"2024-04-25T09:57:09.981755+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981755+0000","last_undegraded":"2024-04-25T09:57:09.981755+0000","last_fullsized":"2024-04-25T09:57:09.981755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992985+0000","last_change":"2024-04-25T09:57:09.991923+0000","last_active":"2024-04-25T09:57:09.992985+0000","last_peered":"2024-04-25T09:57:09.992985+0000","last_clean":"2024-04-25T09:57:09.992985+0000","last_became_active":"2024-04-25T09:57:09.991642+0000","last_became_peered":"2024-04-25T09:57:09.991642+0000","last_unstale":"2024-04-25T09:57:09.992985+0000","last_undegraded":"2024-04-25T09:57:09.992985+0000","last_fullsized":"2024-04-25T09:57:09.992985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.657727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977462+0000","last_change":"2024-04-25T09:57:09.977462+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977462+0000","last_undegraded":"2024-04-25T09:57:09.977462+0000","last_fullsized":"2024-04-25T09:57:09.977462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.984066+0000","last_change":"2024-04-25T09:57:09.982146+0000","last_active":"2024-04-25T09:57:09.984066+0000","last_peered":"2024-04-25T09:57:09.984066+0000","last_clean":"2024-04-25T09:57:09.984066+0000","last_became_active":"2024-04-25T09:57:09.981876+0000","last_became_peered":"2024-04-25T09:57:09.981876+0000","last_unstale":"2024-04-25T09:57:09.984066+0000","last_undegraded":"2024-04-25T09:57:09.984066+0000","last_fullsized":"2024-04-25T09:57:09.984066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982765+0000","last_change":"2024-04-25T09:57:09.982765+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982765+0000","last_undegraded":"2024-04-25T09:57:09.982765+0000","last_fullsized":"2024-04-25T09:57:09.982765+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997380+0000","last_change":"2024-04-25T09:57:09.995462+0000","last_active":"2024-04-25T09:57:09.997380+0000","last_peered":"2024-04-25T09:57:09.997380+0000","last_clean":"2024-04-25T09:57:09.997380+0000","last_became_active":"2024-04-25T09:57:09.995179+0000","last_became_peered":"2024-04-25T09:57:09.995179+0000","last_unstale":"2024-04-25T09:57:09.997380+0000","last_undegraded":"2024-04-25T09:57:09.997380+0000","last_fullsized":"2024-04-25T09:57:09.997380+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.011448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977753+0000","last_change":"2024-04-25T09:57:09.977753+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977753+0000","last_undegraded":"2024-04-25T09:57:09.977753+0000","last_fullsized":"2024-04-25T09:57:09.977753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.996932+0000","last_change":"2024-04-25T09:57:09.994224+0000","last_active":"2024-04-25T09:57:09.996932+0000","last_peered":"2024-04-25T09:57:09.996932+0000","last_clean":"2024-04-25T09:57:09.996932+0000","last_became_active":"2024-04-25T09:57:09.993902+0000","last_became_peered":"2024-04-25T09:57:09.993902+0000","last_unstale":"2024-04-25T09:57:09.996932+0000","last_undegraded":"2024-04-25T09:57:09.996932+0000","last_fullsized":"2024-04-25T09:57:09.996932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983273+0000","last_change":"2024-04-25T09:57:09.983273+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983273+0000","last_undegraded":"2024-04-25T09:57:09.983273+0000","last_fullsized":"2024-04-25T09:57:09.983273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989862+0000","last_change":"2024-04-25T09:57:09.988098+0000","last_active":"2024-04-25T09:57:09.989862+0000","last_peered":"2024-04-25T09:57:09.989862+0000","last_clean":"2024-04-25T09:57:09.989862+0000","last_became_active":"2024-04-25T09:57:09.987672+0000","last_became_peered":"2024-04-25T09:57:09.987672+0000","last_unstale":"2024-04-25T09:57:09.989862+0000","last_undegraded":"2024-04-25T09:57:09.989862+0000","last_fullsized":"2024-04-25T09:57:09.989862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981392+0000","last_change":"2024-04-25T09:57:09.981392+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981392+0000","last_undegraded":"2024-04-25T09:57:09.981392+0000","last_fullsized":"2024-04-25T09:57:09.981392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987441+0000","last_change":"2024-04-25T09:57:09.985927+0000","last_active":"2024-04-25T09:57:09.987441+0000","last_peered":"2024-04-25T09:57:09.987441+0000","last_clean":"2024-04-25T09:57:09.987441+0000","last_became_active":"2024-04-25T09:57:09.985657+0000","last_became_peered":"2024-04-25T09:57:09.985657+0000","last_unstale":"2024-04-25T09:57:09.987441+0000","last_undegraded":"2024-04-25T09:57:09.987441+0000","last_fullsized":"2024-04-25T09:57:09.987441+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981050+0000","last_change":"2024-04-25T09:57:09.981050+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981050+0000","last_undegraded":"2024-04-25T09:57:09.981050+0000","last_fullsized":"2024-04-25T09:57:09.981050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991567+0000","last_change":"2024-04-25T09:57:09.990770+0000","last_active":"2024-04-25T09:57:09.991567+0000","last_peered":"2024-04-25T09:57:09.991567+0000","last_clean":"2024-04-25T09:57:09.991567+0000","last_became_active":"2024-04-25T09:57:09.990495+0000","last_became_peered":"2024-04-25T09:57:09.990495+0000","last_unstale":"2024-04-25T09:57:09.991567+0000","last_undegraded":"2024-04-25T09:57:09.991567+0000","last_fullsized":"2024-04-25T09:57:09.991567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.981492+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:09.981492+0000","last_peered":"2024-04-25T09:57:09.981492+0000","last_clean":"2024-04-25T09:57:09.981492+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:09.981492+0000","last_undegraded":"2024-04-25T09:57:09.981492+0000","last_fullsized":"2024-04-25T09:57:09.981492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":866229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1224704,"data_stored":872896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981364+0000', 'last_change': '2024-04-25T09:57:09.981364+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981364+0000', 'last_undegraded': '2024-04-25T09:57:09.981364+0000', 'last_fullsized': '2024-04-25T09:57:09.981364+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980666+0000', 'last_change': '2024-04-25T09:57:09.980666+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980666+0000', 'last_undegraded': '2024-04-25T09:57:09.980666+0000', 'last_fullsized': '2024-04-25T09:57:09.980666+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.987160+0000', 'last_change': '2024-04-25T09:57:09.987160+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.987160+0000', 'last_undegraded': '2024-04-25T09:57:09.987160+0000', 'last_fullsized': '2024-04-25T09:57:09.987160+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-25T09:57:14.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980352+0000', 'last_change': '2024-04-25T09:57:09.980352+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980352+0000', 'last_undegraded': '2024-04-25T09:57:09.980352+0000', 'last_fullsized': '2024-04-25T09:57:09.980352+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.976437+0000', 'last_change': '2024-04-25T09:57:09.976437+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.976437+0000', 'last_undegraded': '2024-04-25T09:57:09.976437+0000', 'last_fullsized': '2024-04-25T09:57:09.976437+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982151+0000', 'last_change': '2024-04-25T09:57:09.982151+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982151+0000', 'last_undegraded': '2024-04-25T09:57:09.982151+0000', 'last_fullsized': '2024-04-25T09:57:09.982151+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.986707+0000', 'last_change': '2024-04-25T09:57:09.986707+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.986707+0000', 'last_undegraded': '2024-04-25T09:57:09.986707+0000', 'last_fullsized': '2024-04-25T09:57:09.986707+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.987280+0000', 'last_change': '2024-04-25T09:57:09.987280+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.987280+0000', 'last_undegraded': '2024-04-25T09:57:09.987280+0000', 'last_fullsized': '2024-04-25T09:57:09.987280+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982972+0000', 'last_change': '2024-04-25T09:57:09.982972+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982972+0000', 'last_undegraded': '2024-04-25T09:57:09.982972+0000', 'last_fullsized': '2024-04-25T09:57:09.982972+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-25T09:57:14.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.983088+0000', 'last_change': '2024-04-25T09:57:09.983088+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.983088+0000', 'last_undegraded': '2024-04-25T09:57:09.983088+0000', 'last_fullsized': '2024-04-25T09:57:09.983088+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977943+0000', 'last_change': '2024-04-25T09:57:09.977943+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977943+0000', 'last_undegraded': '2024-04-25T09:57:09.977943+0000', 'last_fullsized': '2024-04-25T09:57:09.977943+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.979313+0000', 'last_change': '2024-04-25T09:57:09.979313+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.979313+0000', 'last_undegraded': '2024-04-25T09:57:09.979313+0000', 'last_fullsized': '2024-04-25T09:57:09.979313+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.983403+0000', 'last_change': '2024-04-25T09:57:09.983403+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.983403+0000', 'last_undegraded': '2024-04-25T09:57:09.983403+0000', 'last_fullsized': '2024-04-25T09:57:09.983403+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.986278+0000', 'last_change': '2024-04-25T09:57:09.986278+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.986278+0000', 'last_undegraded': '2024-04-25T09:57:09.986278+0000', 'last_fullsized': '2024-04-25T09:57:09.986278+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.979368+0000', 'last_change': '2024-04-25T09:57:09.979368+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.979368+0000', 'last_undegraded': '2024-04-25T09:57:09.979368+0000', 'last_fullsized': '2024-04-25T09:57:09.979368+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980509+0000', 'last_change': '2024-04-25T09:57:09.980509+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980509+0000', 'last_undegraded': '2024-04-25T09:57:09.980509+0000', 'last_fullsized': '2024-04-25T09:57:09.980509+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.987736+0000', 'last_change': '2024-04-25T09:57:09.987736+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.987736+0000', 'last_undegraded': '2024-04-25T09:57:09.987736+0000', 'last_fullsized': '2024-04-25T09:57:09.987736+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.978066+0000', 'last_change': '2024-04-25T09:57:09.978066+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.978066+0000', 'last_undegraded': '2024-04-25T09:57:09.978066+0000', 'last_fullsized': '2024-04-25T09:57:09.978066+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.985583+0000', 'last_change': '2024-04-25T09:57:09.985583+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.985583+0000', 'last_undegraded': '2024-04-25T09:57:09.985583+0000', 'last_fullsized': '2024-04-25T09:57:09.985583+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982992+0000', 'last_change': '2024-04-25T09:57:09.982992+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982992+0000', 'last_undegraded': '2024-04-25T09:57:09.982992+0000', 'last_fullsized': '2024-04-25T09:57:09.982992+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-25T09:57:14.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981564+0000', 'last_change': '2024-04-25T09:57:09.981564+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981564+0000', 'last_undegraded': '2024-04-25T09:57:09.981564+0000', 'last_fullsized': '2024-04-25T09:57:09.981564+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980454+0000', 'last_change': '2024-04-25T09:57:09.980454+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980454+0000', 'last_undegraded': '2024-04-25T09:57:09.980454+0000', 'last_fullsized': '2024-04-25T09:57:09.980454+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977821+0000', 'last_change': '2024-04-25T09:57:09.977821+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977821+0000', 'last_undegraded': '2024-04-25T09:57:09.977821+0000', 'last_fullsized': '2024-04-25T09:57:09.977821+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.985146+0000', 'last_change': '2024-04-25T09:57:09.985146+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.985146+0000', 'last_undegraded': '2024-04-25T09:57:09.985146+0000', 'last_fullsized': '2024-04-25T09:57:09.985146+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.985837+0000', 'last_change': '2024-04-25T09:57:09.985837+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.985837+0000', 'last_undegraded': '2024-04-25T09:57:09.985837+0000', 'last_fullsized': '2024-04-25T09:57:09.985837+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-25T09:57:14.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980855+0000', 'last_change': '2024-04-25T09:57:09.980855+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980855+0000', 'last_undegraded': '2024-04-25T09:57:09.980855+0000', 'last_fullsized': '2024-04-25T09:57:09.980855+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980652+0000', 'last_change': '2024-04-25T09:57:09.980652+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980652+0000', 'last_undegraded': '2024-04-25T09:57:09.980652+0000', 'last_fullsized': '2024-04-25T09:57:09.980652+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981345+0000', 'last_change': '2024-04-25T09:57:09.981345+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981345+0000', 'last_undegraded': '2024-04-25T09:57:09.981345+0000', 'last_fullsized': '2024-04-25T09:57:09.981345+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.984479+0000', 'last_change': '2024-04-25T09:57:09.984479+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.984479+0000', 'last_undegraded': '2024-04-25T09:57:09.984479+0000', 'last_fullsized': '2024-04-25T09:57:09.984479+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.979739+0000', 'last_change': '2024-04-25T09:57:09.979739+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.979739+0000', 'last_undegraded': '2024-04-25T09:57:09.979739+0000', 'last_fullsized': '2024-04-25T09:57:09.979739+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.984343+0000', 'last_change': '2024-04-25T09:57:09.984343+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.984343+0000', 'last_undegraded': '2024-04-25T09:57:09.984343+0000', 'last_fullsized': '2024-04-25T09:57:09.984343+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-25T09:57:14.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.978355+0000', 'last_change': '2024-04-25T09:57:09.978355+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.978355+0000', 'last_undegraded': '2024-04-25T09:57:09.978355+0000', 'last_fullsized': '2024-04-25T09:57:09.978355+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980990+0000', 'last_change': '2024-04-25T09:57:09.980990+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980990+0000', 'last_undegraded': '2024-04-25T09:57:09.980990+0000', 'last_fullsized': '2024-04-25T09:57:09.980990+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977814+0000', 'last_change': '2024-04-25T09:57:09.977814+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977814+0000', 'last_undegraded': '2024-04-25T09:57:09.977814+0000', 'last_fullsized': '2024-04-25T09:57:09.977814+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980833+0000', 'last_change': '2024-04-25T09:57:09.980833+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980833+0000', 'last_undegraded': '2024-04-25T09:57:09.980833+0000', 'last_fullsized': '2024-04-25T09:57:09.980833+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981254+0000', 'last_change': '2024-04-25T09:57:09.981254+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981254+0000', 'last_undegraded': '2024-04-25T09:57:09.981254+0000', 'last_fullsized': '2024-04-25T09:57:09.981254+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.975566+0000', 'last_change': '2024-04-25T09:57:09.975566+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.975566+0000', 'last_undegraded': '2024-04-25T09:57:09.975566+0000', 'last_fullsized': '2024-04-25T09:57:09.975566+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-25T09:57:14.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982554+0000', 'last_change': '2024-04-25T09:57:09.982554+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982554+0000', 'last_undegraded': '2024-04-25T09:57:09.982554+0000', 'last_fullsized': '2024-04-25T09:57:09.982554+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.979321+0000', 'last_change': '2024-04-25T09:57:09.979321+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.979321+0000', 'last_undegraded': '2024-04-25T09:57:09.979321+0000', 'last_fullsized': '2024-04-25T09:57:09.979321+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.978986+0000', 'last_change': '2024-04-25T09:57:09.978986+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.978986+0000', 'last_undegraded': '2024-04-25T09:57:09.978986+0000', 'last_fullsized': '2024-04-25T09:57:09.978986+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.975670+0000', 'last_change': '2024-04-25T09:57:09.975670+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.975670+0000', 'last_undegraded': '2024-04-25T09:57:09.975670+0000', 'last_fullsized': '2024-04-25T09:57:09.975670+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.978288+0000', 'last_change': '2024-04-25T09:57:09.978288+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.978288+0000', 'last_undegraded': '2024-04-25T09:57:09.978288+0000', 'last_fullsized': '2024-04-25T09:57:09.978288+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-25T09:57:14.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.983313+0000', 'last_change': '2024-04-25T09:57:09.983313+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.983313+0000', 'last_undegraded': '2024-04-25T09:57:09.983313+0000', 'last_fullsized': '2024-04-25T09:57:09.983313+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.976028+0000', 'last_change': '2024-04-25T09:57:09.976028+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.976028+0000', 'last_undegraded': '2024-04-25T09:57:09.976028+0000', 'last_fullsized': '2024-04-25T09:57:09.976028+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.978626+0000', 'last_change': '2024-04-25T09:57:09.978626+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.978626+0000', 'last_undegraded': '2024-04-25T09:57:09.978626+0000', 'last_fullsized': '2024-04-25T09:57:09.978626+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.983786+0000', 'last_change': '2024-04-25T09:57:09.983786+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.983786+0000', 'last_undegraded': '2024-04-25T09:57:09.983786+0000', 'last_fullsized': '2024-04-25T09:57:09.983786+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.979732+0000', 'last_change': '2024-04-25T09:57:09.979732+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.979732+0000', 'last_undegraded': '2024-04-25T09:57:09.979732+0000', 'last_fullsized': '2024-04-25T09:57:09.979732+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-25T09:57:14.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.984631+0000', 'last_change': '2024-04-25T09:57:09.984631+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.984631+0000', 'last_undegraded': '2024-04-25T09:57:09.984631+0000', 'last_fullsized': '2024-04-25T09:57:09.984631+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.978354+0000', 'last_change': '2024-04-25T09:57:09.978354+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.978354+0000', 'last_undegraded': '2024-04-25T09:57:09.978354+0000', 'last_fullsized': '2024-04-25T09:57:09.978354+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982863+0000', 'last_change': '2024-04-25T09:57:09.982863+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982863+0000', 'last_undegraded': '2024-04-25T09:57:09.982863+0000', 'last_fullsized': '2024-04-25T09:57:09.982863+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.975678+0000', 'last_change': '2024-04-25T09:57:09.975678+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.975678+0000', 'last_undegraded': '2024-04-25T09:57:09.975678+0000', 'last_fullsized': '2024-04-25T09:57:09.975678+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-25T09:57:14.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982962+0000', 'last_change': '2024-04-25T09:57:09.982962+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982962+0000', 'last_undegraded': '2024-04-25T09:57:09.982962+0000', 'last_fullsized': '2024-04-25T09:57:09.982962+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977199+0000', 'last_change': '2024-04-25T09:57:09.977199+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977199+0000', 'last_undegraded': '2024-04-25T09:57:09.977199+0000', 'last_fullsized': '2024-04-25T09:57:09.977199+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980632+0000', 'last_change': '2024-04-25T09:57:09.980632+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980632+0000', 'last_undegraded': '2024-04-25T09:57:09.980632+0000', 'last_fullsized': '2024-04-25T09:57:09.980632+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982756+0000', 'last_change': '2024-04-25T09:57:09.982756+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982756+0000', 'last_undegraded': '2024-04-25T09:57:09.982756+0000', 'last_fullsized': '2024-04-25T09:57:09.982756+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.980965+0000', 'last_change': '2024-04-25T09:57:09.980965+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.980965+0000', 'last_undegraded': '2024-04-25T09:57:09.980965+0000', 'last_fullsized': '2024-04-25T09:57:09.980965+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-25T09:57:14.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977222+0000', 'last_change': '2024-04-25T09:57:09.977222+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977222+0000', 'last_undegraded': '2024-04-25T09:57:09.977222+0000', 'last_fullsized': '2024-04-25T09:57:09.977222+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981755+0000', 'last_change': '2024-04-25T09:57:09.981755+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981755+0000', 'last_undegraded': '2024-04-25T09:57:09.981755+0000', 'last_fullsized': '2024-04-25T09:57:09.981755+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977462+0000', 'last_change': '2024-04-25T09:57:09.977462+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977462+0000', 'last_undegraded': '2024-04-25T09:57:09.977462+0000', 'last_fullsized': '2024-04-25T09:57:09.977462+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.982765+0000', 'last_change': '2024-04-25T09:57:09.982765+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.982765+0000', 'last_undegraded': '2024-04-25T09:57:09.982765+0000', 'last_fullsized': '2024-04-25T09:57:09.982765+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.977753+0000', 'last_change': '2024-04-25T09:57:09.977753+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.977753+0000', 'last_undegraded': '2024-04-25T09:57:09.977753+0000', 'last_fullsized': '2024-04-25T09:57:09.977753+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T09:57:14.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.983273+0000', 'last_change': '2024-04-25T09:57:09.983273+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.983273+0000', 'last_undegraded': '2024-04-25T09:57:09.983273+0000', 'last_fullsized': '2024-04-25T09:57:09.983273+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:57:14.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-25T09:57:14.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981392+0000', 'last_change': '2024-04-25T09:57:09.981392+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981392+0000', 'last_undegraded': '2024-04-25T09:57:09.981392+0000', 'last_fullsized': '2024-04-25T09:57:09.981392+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:57:14.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-25T09:57:14.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:57:09.981050+0000', 'last_change': '2024-04-25T09:57:09.981050+0000', 'last_active': '2024-04-25T09:57:09.966917+0000', 'last_peered': '2024-04-25T09:57:09.966917+0000', 'last_clean': '2024-04-25T09:57:09.966917+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:57:09.981050+0000', 'last_undegraded': '2024-04-25T09:57:09.981050+0000', 'last_fullsized': '2024-04-25T09:57:09.981050+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'last_clean_scrub_stamp': '2024-04-25T09:57:09.966917+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:57:14.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:57:14.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.739+0000 7fa9d023f640 1 Processor -- start 2024-04-25T09:57:14.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.739+0000 7fa9d023f640 1 -- start start 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9d023f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9c815dbe0 0x7fa9c80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9d023f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9c80b2cb0 0x7fa9c80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9d023f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b92a0 0x7fa9c80b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9d023f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c815dfc0 con 0x7fa9c80b2cb0 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9d023f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c80d1a00 con 0x7fa9c815dbe0 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9d023f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80b97a0 con 0x7fa9c80b92a0 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9cd7da640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9c80b2cb0 0x7fa9c80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9cd7da640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa9c80b2cb0 0x7fa9c80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43786/0 (socket says 172.21.15.148:43786) 2024-04-25T09:57:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.740+0000 7fa9cd7da640 1 -- 172.21.15.148:0/1847338746 learned_addr learned my addr 172.21.15.148:0/1847338746 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9ce7dc640 1 --2- 172.21.15.148:0/1847338746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b92a0 0x7fa9c80b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cd7da640 1 -- 172.21.15.148:0/1847338746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b92a0 msgr2=0x7fa9c80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cdfdb640 1 --2- 172.21.15.148:0/1847338746 >> v2:172.21.15.188:3300/0 conn(0x7fa9c815dbe0 0x7fa9c80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cd7da640 1 --2- 172.21.15.148:0/1847338746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b92a0 0x7fa9c80b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cd7da640 1 -- 172.21.15.148:0/1847338746 >> v2:172.21.15.188:3300/0 conn(0x7fa9c815dbe0 msgr2=0x7fa9c80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cd7da640 1 --2- 172.21.15.148:0/1847338746 >> v2:172.21.15.188:3300/0 conn(0x7fa9c815dbe0 0x7fa9c80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cd7da640 1 -- 172.21.15.148:0/1847338746 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c80d1c50 con 0x7fa9c80b2cb0 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.741+0000 7fa9cd7da640 1 --2- 172.21.15.148:0/1847338746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c80b2cb0 0x7fa9c80d0e50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa9c806c140 tx=0x7fa9b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ba2597530dca72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9ccfd9640 1 -- 172.21.15.148:0/1847338746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8037070 con 0x7fa9c80b2cb0 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9ccfd9640 1 -- 172.21.15.148:0/1847338746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b8032440 con 0x7fa9c80b2cb0 2024-04-25T09:57:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9ccfd9640 1 -- 172.21.15.148:0/1847338746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8032810 con 0x7fa9c80b2cb0 2024-04-25T09:57:14.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 -- 172.21.15.148:0/1847338746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c80b2cb0 msgr2=0x7fa9c80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 --2- 172.21.15.148:0/1847338746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c80b2cb0 0x7fa9c80d0e50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa9c806c140 tx=0x7fa9b802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 -- 172.21.15.148:0/1847338746 shutdown_connections 2024-04-25T09:57:14.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 --2- 172.21.15.148:0/1847338746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b92a0 0x7fa9c80b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 --2- 172.21.15.148:0/1847338746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c80b2cb0 0x7fa9c80d0e50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 --2- 172.21.15.148:0/1847338746 >> v2:172.21.15.188:3300/0 conn(0x7fa9c815dbe0 0x7fa9c80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 -- 172.21.15.148:0/1847338746 >> 172.21.15.148:0/1847338746 conn(0x7fa9c8061a20 msgr2=0x7fa9c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 -- 172.21.15.148:0/1847338746 shutdown_connections 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.742+0000 7fa9d023f640 1 -- 172.21.15.148:0/1847338746 wait complete. 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.743+0000 7fa9d023f640 1 Processor -- start 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.743+0000 7fa9d023f640 1 -- start start 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9d023f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b2cb0 0x7fa9c8160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9d023f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9c80b92a0 0x7fa9c80c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9d023f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 0x7fa9c80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9d023f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80b6ab0 con 0x7fa9c815dbe0 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9d023f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80d2210 con 0x7fa9c80b92a0 2024-04-25T09:57:14.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9d023f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c80b6c30 con 0x7fa9c80b2cb0 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 0x7fa9c80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 0x7fa9c80b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43798/0 (socket says 172.21.15.148:43798) 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 -- 172.21.15.148:0/3905511331 learned_addr learned my addr 172.21.15.148:0/3905511331 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9cdfdb640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b2cb0 0x7fa9c8160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9cd7da640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9c80b92a0 0x7fa9c80c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 -- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b2cb0 msgr2=0x7fa9c8160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b2cb0 0x7fa9c8160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 -- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9c80b92a0 msgr2=0x7fa9c80c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.744+0000 7fa9ce7dc640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9c80b92a0 0x7fa9c80c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9ce7dc640 1 -- 172.21.15.148:0/3905511331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c8160aa0 con 0x7fa9c815dbe0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9ce7dc640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 0x7fa9c80b5060 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c5c0 tx=0x7fa9c000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0015070 con 0x7fa9c815dbe0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c0004220 con 0x7fa9c815dbe0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0003610 con 0x7fa9c815dbe0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c815ed90 con 0x7fa9c815dbe0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.745+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c815f350 con 0x7fa9c815dbe0 2024-04-25T09:57:14.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.746+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c8073990 con 0x7fa9c815dbe0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.746+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9c00037b0 con 0x7fa9c815dbe0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.747+0000 7fa9be7fc640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9ac06b4d0 0x7fa9ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.747+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9c008a060 con 0x7fa9c815dbe0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.747+0000 7fa9cdfdb640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9ac06b4d0 0x7fa9ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.747+0000 7fa9cdfdb640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9ac06b4d0 0x7fa9ac06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4007f60 tx=0x7fa9c4007710 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.751+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9c0058790 con 0x7fa9c815dbe0 2024-04-25T09:57:14.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.986+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa9c815ced0 con 0x7fa9c815dbe0 2024-04-25T09:57:14.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.987+0000 7fa9be7fc640 1 -- 172.21.15.148:0/3905511331 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1195 (secure 0 0 0) 0x7fa9c005c830 con 0x7fa9c815dbe0 2024-04-25T09:57:14.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:14.990 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":196,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":282882048,"bytes_avail":772811231232,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5862}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T09:56:35.046115+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9ac06b4d0 msgr2=0x7fa9ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9ac06b4d0 0x7fa9ac06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4007f60 tx=0x7fa9c4007710 comp rx=0 tx=0).stop 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 msgr2=0x7fa9c80b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 0x7fa9c80b5060 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c5c0 tx=0x7fa9c000ca80 comp rx=0 tx=0).stop 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 shutdown_connections 2024-04-25T09:57:14.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9ac06b4d0 0x7fa9ac06d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9c815dbe0 0x7fa9c80b5060 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9c80b92a0 0x7fa9c80c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 --2- 172.21.15.148:0/3905511331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9c80b2cb0 0x7fa9c8160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:14.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 >> 172.21.15.148:0/3905511331 conn(0x7fa9c8061a20 msgr2=0x7fa9c80b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:14.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 shutdown_connections 2024-04-25T09:57:14.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:14.989+0000 7fa9d023f640 1 -- 172.21.15.148:0/3905511331 wait complete. 2024-04-25T09:57:15.003 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:57:15.003 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:15.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.098+0000 7f19d97c4640 1 Processor -- start 2024-04-25T09:57:15.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.098+0000 7f19d97c4640 1 -- start start 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d97c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19d40afa00 0x7f19d406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d97c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19d406b120 0x7f19d40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d97c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d40aef60 con 0x7f19d4145310 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d97c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40b5590 con 0x7f19d406b120 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d97c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40af200 con 0x7f19d40afa00 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19d406b120 0x7f19d40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19d406b120 0x7f19d40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43806/0 (socket says 172.21.15.148:43806) 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 -- 172.21.15.148:0/4006371012 learned_addr learned my addr 172.21.15.148:0/4006371012 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 -- 172.21.15.148:0/4006371012 >> v2:172.21.15.188:3300/0 conn(0x7f19d40afa00 msgr2=0x7f19d406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 --2- 172.21.15.148:0/4006371012 >> v2:172.21.15.188:3300/0 conn(0x7f19d40afa00 0x7f19d406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 -- 172.21.15.148:0/4006371012 >> v1:172.21.15.188:6789/0 conn(0x7f19d4145310 legacy=0x7f19d415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:15.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.099+0000 7f19d37fe640 1 -- 172.21.15.148:0/4006371012 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40af4e0 con 0x7f19d406b120 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.100+0000 7f19d37fe640 1 --2- 172.21.15.148:0/4006371012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d40b3590 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b32f0 tx=0x7f19cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5491a72afe55e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.100+0000 7f19d1ffb640 1 -- 172.21.15.148:0/4006371012 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc036070 con 0x7f19d406b120 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.100+0000 7f19d1ffb640 1 -- 172.21.15.148:0/4006371012 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc031600 con 0x7f19d406b120 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.100+0000 7f19d1ffb640 1 -- 172.21.15.148:0/4006371012 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0319d0 con 0x7f19d406b120 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 -- 172.21.15.148:0/4006371012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 msgr2=0x7f19d40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 --2- 172.21.15.148:0/4006371012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d40b3590 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b32f0 tx=0x7f19cc02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 -- 172.21.15.148:0/4006371012 shutdown_connections 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 --2- 172.21.15.148:0/4006371012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d40b3590 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 --2- 172.21.15.148:0/4006371012 >> v2:172.21.15.188:3300/0 conn(0x7f19d40afa00 0x7f19d406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 -- 172.21.15.148:0/4006371012 >> 172.21.15.148:0/4006371012 conn(0x7f19d40b5af0 msgr2=0x7f19d40b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 -- 172.21.15.148:0/4006371012 shutdown_connections 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.101+0000 7f19d97c4640 1 -- 172.21.15.148:0/4006371012 wait complete. 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.102+0000 7f19d97c4640 1 Processor -- start 2024-04-25T09:57:15.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.102+0000 7f19d97c4640 1 -- start start 2024-04-25T09:57:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d97c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d41f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d97c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19d40afa00 0x7f19d4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d97c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19d4145310 0x7f19d4143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d97c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40b51f0 con 0x7f19d406b120 2024-04-25T09:57:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d97c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40ad7e0 con 0x7f19d40afa00 2024-04-25T09:57:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d97c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40b5370 con 0x7f19d4145310 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d41f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d41f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43810/0 (socket says 172.21.15.148:43810) 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 -- 172.21.15.148:0/257543640 learned_addr learned my addr 172.21.15.148:0/257543640 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d37fe640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19d4145310 0x7f19d4143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d27fc640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19d40afa00 0x7f19d4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 -- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19d4145310 msgr2=0x7f19d4143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19d4145310 0x7f19d4143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 -- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19d40afa00 msgr2=0x7f19d4143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19d40afa00 0x7f19d4143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 -- 172.21.15.148:0/257543640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40ce550 con 0x7f19d406b120 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d37fe640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19d4145310 0x7f19d4143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d27fc640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19d40afa00 0x7f19d4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.103+0000 7f19d2ffd640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d41f1310 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c440 tx=0x7f19c400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.104+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4017070 con 0x7f19d406b120 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.104+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c40035f0 con 0x7f19d406b120 2024-04-25T09:57:15.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.104+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d40ce830 con 0x7f19d406b120 2024-04-25T09:57:15.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.104+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c40039e0 con 0x7f19d406b120 2024-04-25T09:57:15.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.104+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d41f1850 con 0x7f19d406b120 2024-04-25T09:57:15.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.105+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4073990 con 0x7f19d406b120 2024-04-25T09:57:15.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.105+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19c401a070 con 0x7f19d406b120 2024-04-25T09:57:15.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.105+0000 7f19bb7fe640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19c006b4d0 0x7f19c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:15.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.106+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f19c4089680 con 0x7f19d406b120 2024-04-25T09:57:15.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.106+0000 7f19d27fc640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19c006b4d0 0x7f19c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:15.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.106+0000 7f19d27fc640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19c006b4d0 0x7f19c006d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b3920 tx=0x7f19c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:15.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.110+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19c4058400 con 0x7f19d406b120 2024-04-25T09:57:15.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.274+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19d40d0490 con 0x7f19c006b4d0 2024-04-25T09:57:15.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.280+0000 7f19bb7fe640 1 -- 172.21.15.148:0/257543640 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246636 (secure 0 0 0) 0x7f19d40d0490 con 0x7f19c006b4d0 2024-04-25T09:57:15.286 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:15.287 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:57:15.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19c006b4d0 msgr2=0x7f19c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19c006b4d0 0x7f19c006d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b3920 tx=0x7f19c8008040 comp rx=0 tx=0).stop 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 msgr2=0x7f19d41f1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d41f1310 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c440 tx=0x7f19c400c900 comp rx=0 tx=0).stop 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 shutdown_connections 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19c006b4d0 0x7f19c006d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19d4145310 0x7f19d4143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19d40afa00 0x7f19d4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 --2- 172.21.15.148:0/257543640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19d406b120 0x7f19d41f1310 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 >> 172.21.15.148:0/257543640 conn(0x7f19d40b5af0 msgr2=0x7f19d40b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 shutdown_connections 2024-04-25T09:57:15.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:15.283+0000 7f19d97c4640 1 -- 172.21.15.148:0/257543640 wait complete. 2024-04-25T09:57:15.297 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-04-25T09:57:15.054983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276252,"kb_used_data":6004,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754698468,"statfs":{"total":773094113280,"available":772811231232,"internally_reserved":0,"allocated":6148096,"data_stored":3394291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":276614704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000850"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981364+0000","last_change":"2024-04-25T09:57:09.981364+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981364+0000","last_undegraded":"2024-04-25T09:57:09.981364+0000","last_fullsized":"2024-04-25T09:57:09.981364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980666+0000","last_change":"2024-04-25T09:57:09.980666+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980666+0000","last_undegraded":"2024-04-25T09:57:09.980666+0000","last_fullsized":"2024-04-25T09:57:09.980666+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987160+0000","last_change":"2024-04-25T09:57:09.987160+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987160+0000","last_undegraded":"2024-04-25T09:57:09.987160+0000","last_fullsized":"2024-04-25T09:57:09.987160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980352+0000","last_change":"2024-04-25T09:57:09.980352+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980352+0000","last_undegraded":"2024-04-25T09:57:09.980352+0000","last_fullsized":"2024-04-25T09:57:09.980352+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976437+0000","last_change":"2024-04-25T09:57:09.976437+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976437+0000","last_undegraded":"2024-04-25T09:57:09.976437+0000","last_fullsized":"2024-04-25T09:57:09.976437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982151+0000","last_change":"2024-04-25T09:57:09.982151+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982151+0000","last_undegraded":"2024-04-25T09:57:09.982151+0000","last_fullsized":"2024-04-25T09:57:09.982151+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986707+0000","last_change":"2024-04-25T09:57:09.986707+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986707+0000","last_undegraded":"2024-04-25T09:57:09.986707+0000","last_fullsized":"2024-04-25T09:57:09.986707+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987280+0000","last_change":"2024-04-25T09:57:09.987280+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987280+0000","last_undegraded":"2024-04-25T09:57:09.987280+0000","last_fullsized":"2024-04-25T09:57:09.987280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982972+0000","last_change":"2024-04-25T09:57:09.982972+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982972+0000","last_undegraded":"2024-04-25T09:57:09.982972+0000","last_fullsized":"2024-04-25T09:57:09.982972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983088+0000","last_change":"2024-04-25T09:57:09.983088+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983088+0000","last_undegraded":"2024-04-25T09:57:09.983088+0000","last_fullsized":"2024-04-25T09:57:09.983088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977943+0000","last_change":"2024-04-25T09:57:09.977943+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977943+0000","last_undegraded":"2024-04-25T09:57:09.977943+0000","last_fullsized":"2024-04-25T09:57:09.977943+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979313+0000","last_change":"2024-04-25T09:57:09.979313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979313+0000","last_undegraded":"2024-04-25T09:57:09.979313+0000","last_fullsized":"2024-04-25T09:57:09.979313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983403+0000","last_change":"2024-04-25T09:57:09.983403+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983403+0000","last_undegraded":"2024-04-25T09:57:09.983403+0000","last_fullsized":"2024-04-25T09:57:09.983403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.986278+0000","last_change":"2024-04-25T09:57:09.986278+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.986278+0000","last_undegraded":"2024-04-25T09:57:09.986278+0000","last_fullsized":"2024-04-25T09:57:09.986278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979368+0000","last_change":"2024-04-25T09:57:09.979368+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979368+0000","last_undegraded":"2024-04-25T09:57:09.979368+0000","last_fullsized":"2024-04-25T09:57:09.979368+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980509+0000","last_change":"2024-04-25T09:57:09.980509+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980509+0000","last_undegraded":"2024-04-25T09:57:09.980509+0000","last_fullsized":"2024-04-25T09:57:09.980509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.987736+0000","last_change":"2024-04-25T09:57:09.987736+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.987736+0000","last_undegraded":"2024-04-25T09:57:09.987736+0000","last_fullsized":"2024-04-25T09:57:09.987736+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978066+0000","last_change":"2024-04-25T09:57:09.978066+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978066+0000","last_undegraded":"2024-04-25T09:57:09.978066+0000","last_fullsized":"2024-04-25T09:57:09.978066+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985583+0000","last_change":"2024-04-25T09:57:09.985583+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985583+0000","last_undegraded":"2024-04-25T09:57:09.985583+0000","last_fullsized":"2024-04-25T09:57:09.985583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982992+0000","last_change":"2024-04-25T09:57:09.982992+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982992+0000","last_undegraded":"2024-04-25T09:57:09.982992+0000","last_fullsized":"2024-04-25T09:57:09.982992+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981564+0000","last_change":"2024-04-25T09:57:09.981564+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981564+0000","last_undegraded":"2024-04-25T09:57:09.981564+0000","last_fullsized":"2024-04-25T09:57:09.981564+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980454+0000","last_change":"2024-04-25T09:57:09.980454+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980454+0000","last_undegraded":"2024-04-25T09:57:09.980454+0000","last_fullsized":"2024-04-25T09:57:09.980454+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977821+0000","last_change":"2024-04-25T09:57:09.977821+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977821+0000","last_undegraded":"2024-04-25T09:57:09.977821+0000","last_fullsized":"2024-04-25T09:57:09.977821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985146+0000","last_change":"2024-04-25T09:57:09.985146+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985146+0000","last_undegraded":"2024-04-25T09:57:09.985146+0000","last_fullsized":"2024-04-25T09:57:09.985146+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.985837+0000","last_change":"2024-04-25T09:57:09.985837+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.985837+0000","last_undegraded":"2024-04-25T09:57:09.985837+0000","last_fullsized":"2024-04-25T09:57:09.985837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980855+0000","last_change":"2024-04-25T09:57:09.980855+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980855+0000","last_undegraded":"2024-04-25T09:57:09.980855+0000","last_fullsized":"2024-04-25T09:57:09.980855+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980652+0000","last_change":"2024-04-25T09:57:09.980652+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980652+0000","last_undegraded":"2024-04-25T09:57:09.980652+0000","last_fullsized":"2024-04-25T09:57:09.980652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981345+0000","last_change":"2024-04-25T09:57:09.981345+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981345+0000","last_undegraded":"2024-04-25T09:57:09.981345+0000","last_fullsized":"2024-04-25T09:57:09.981345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984479+0000","last_change":"2024-04-25T09:57:09.984479+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984479+0000","last_undegraded":"2024-04-25T09:57:09.984479+0000","last_fullsized":"2024-04-25T09:57:09.984479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979739+0000","last_change":"2024-04-25T09:57:09.979739+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979739+0000","last_undegraded":"2024-04-25T09:57:09.979739+0000","last_fullsized":"2024-04-25T09:57:09.979739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984343+0000","last_change":"2024-04-25T09:57:09.984343+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984343+0000","last_undegraded":"2024-04-25T09:57:09.984343+0000","last_fullsized":"2024-04-25T09:57:09.984343+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978355+0000","last_change":"2024-04-25T09:57:09.978355+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978355+0000","last_undegraded":"2024-04-25T09:57:09.978355+0000","last_fullsized":"2024-04-25T09:57:09.978355+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980990+0000","last_change":"2024-04-25T09:57:09.980990+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980990+0000","last_undegraded":"2024-04-25T09:57:09.980990+0000","last_fullsized":"2024-04-25T09:57:09.980990+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989713+0000","last_change":"2024-04-25T09:57:09.986934+0000","last_active":"2024-04-25T09:57:09.989713+0000","last_peered":"2024-04-25T09:57:09.989713+0000","last_clean":"2024-04-25T09:57:09.989713+0000","last_became_active":"2024-04-25T09:57:09.986610+0000","last_became_peered":"2024-04-25T09:57:09.986610+0000","last_unstale":"2024-04-25T09:57:09.989713+0000","last_undegraded":"2024-04-25T09:57:09.989713+0000","last_fullsized":"2024-04-25T09:57:09.989713+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:16.052402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977814+0000","last_change":"2024-04-25T09:57:09.977814+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977814+0000","last_undegraded":"2024-04-25T09:57:09.977814+0000","last_fullsized":"2024-04-25T09:57:09.977814+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997056+0000","last_change":"2024-04-25T09:57:09.996107+0000","last_active":"2024-04-25T09:57:09.997056+0000","last_peered":"2024-04-25T09:57:09.997056+0000","last_clean":"2024-04-25T09:57:09.997056+0000","last_became_active":"2024-04-25T09:57:09.995840+0000","last_became_peered":"2024-04-25T09:57:09.995840+0000","last_unstale":"2024-04-25T09:57:09.997056+0000","last_undegraded":"2024-04-25T09:57:09.997056+0000","last_fullsized":"2024-04-25T09:57:09.997056+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980833+0000","last_change":"2024-04-25T09:57:09.980833+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980833+0000","last_undegraded":"2024-04-25T09:57:09.980833+0000","last_fullsized":"2024-04-25T09:57:09.980833+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.993934+0000","last_change":"2024-04-25T09:57:09.988639+0000","last_active":"2024-04-25T09:57:09.993934+0000","last_peered":"2024-04-25T09:57:09.993934+0000","last_clean":"2024-04-25T09:57:09.993934+0000","last_became_active":"2024-04-25T09:57:09.988362+0000","last_became_peered":"2024-04-25T09:57:09.988362+0000","last_unstale":"2024-04-25T09:57:09.993934+0000","last_undegraded":"2024-04-25T09:57:09.993934+0000","last_fullsized":"2024-04-25T09:57:09.993934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.037390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981254+0000","last_change":"2024-04-25T09:57:09.981254+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981254+0000","last_undegraded":"2024-04-25T09:57:09.981254+0000","last_fullsized":"2024-04-25T09:57:09.981254+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986629+0000","last_change":"2024-04-25T09:57:09.981554+0000","last_active":"2024-04-25T09:57:09.986629+0000","last_peered":"2024-04-25T09:57:09.986629+0000","last_clean":"2024-04-25T09:57:09.986629+0000","last_became_active":"2024-04-25T09:57:09.981290+0000","last_became_peered":"2024-04-25T09:57:09.981290+0000","last_unstale":"2024-04-25T09:57:09.986629+0000","last_undegraded":"2024-04-25T09:57:09.986629+0000","last_fullsized":"2024-04-25T09:57:09.986629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:02.596300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975566+0000","last_change":"2024-04-25T09:57:09.975566+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975566+0000","last_undegraded":"2024-04-25T09:57:09.975566+0000","last_fullsized":"2024-04-25T09:57:09.975566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997140+0000","last_change":"2024-04-25T09:57:09.995396+0000","last_active":"2024-04-25T09:57:09.997140+0000","last_peered":"2024-04-25T09:57:09.997140+0000","last_clean":"2024-04-25T09:57:09.997140+0000","last_became_active":"2024-04-25T09:57:09.995117+0000","last_became_peered":"2024-04-25T09:57:09.995117+0000","last_unstale":"2024-04-25T09:57:09.997140+0000","last_undegraded":"2024-04-25T09:57:09.997140+0000","last_fullsized":"2024-04-25T09:57:09.997140+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:27.807847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982554+0000","last_change":"2024-04-25T09:57:09.982554+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982554+0000","last_undegraded":"2024-04-25T09:57:09.982554+0000","last_fullsized":"2024-04-25T09:57:09.982554+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987303+0000","last_change":"2024-04-25T09:57:09.985140+0000","last_active":"2024-04-25T09:57:09.987303+0000","last_peered":"2024-04-25T09:57:09.987303+0000","last_clean":"2024-04-25T09:57:09.987303+0000","last_became_active":"2024-04-25T09:57:09.983854+0000","last_became_peered":"2024-04-25T09:57:09.983854+0000","last_unstale":"2024-04-25T09:57:09.987303+0000","last_undegraded":"2024-04-25T09:57:09.987303+0000","last_fullsized":"2024-04-25T09:57:09.987303+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979321+0000","last_change":"2024-04-25T09:57:09.979321+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979321+0000","last_undegraded":"2024-04-25T09:57:09.979321+0000","last_fullsized":"2024-04-25T09:57:09.979321+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987086+0000","last_change":"2024-04-25T09:57:09.985009+0000","last_active":"2024-04-25T09:57:09.987086+0000","last_peered":"2024-04-25T09:57:09.987086+0000","last_clean":"2024-04-25T09:57:09.987086+0000","last_became_active":"2024-04-25T09:57:09.984733+0000","last_became_peered":"2024-04-25T09:57:09.984733+0000","last_unstale":"2024-04-25T09:57:09.987086+0000","last_undegraded":"2024-04-25T09:57:09.987086+0000","last_fullsized":"2024-04-25T09:57:09.987086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.282213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978986+0000","last_change":"2024-04-25T09:57:09.978986+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978986+0000","last_undegraded":"2024-04-25T09:57:09.978986+0000","last_fullsized":"2024-04-25T09:57:09.978986+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991792+0000","last_change":"2024-04-25T09:57:09.988494+0000","last_active":"2024-04-25T09:57:09.991792+0000","last_peered":"2024-04-25T09:57:09.991792+0000","last_clean":"2024-04-25T09:57:09.991792+0000","last_became_active":"2024-04-25T09:57:09.988229+0000","last_became_peered":"2024-04-25T09:57:09.988229+0000","last_unstale":"2024-04-25T09:57:09.991792+0000","last_undegraded":"2024-04-25T09:57:09.991792+0000","last_fullsized":"2024-04-25T09:57:09.991792+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975670+0000","last_change":"2024-04-25T09:57:09.975670+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975670+0000","last_undegraded":"2024-04-25T09:57:09.975670+0000","last_fullsized":"2024-04-25T09:57:09.975670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990292+0000","last_change":"2024-04-25T09:57:09.988417+0000","last_active":"2024-04-25T09:57:09.990292+0000","last_peered":"2024-04-25T09:57:09.990292+0000","last_clean":"2024-04-25T09:57:09.990292+0000","last_became_active":"2024-04-25T09:57:09.987870+0000","last_became_peered":"2024-04-25T09:57:09.987870+0000","last_unstale":"2024-04-25T09:57:09.990292+0000","last_undegraded":"2024-04-25T09:57:09.990292+0000","last_fullsized":"2024-04-25T09:57:09.990292+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.029172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978288+0000","last_change":"2024-04-25T09:57:09.978288+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978288+0000","last_undegraded":"2024-04-25T09:57:09.978288+0000","last_fullsized":"2024-04-25T09:57:09.978288+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985102+0000","last_change":"2024-04-25T09:57:09.981400+0000","last_active":"2024-04-25T09:57:09.985102+0000","last_peered":"2024-04-25T09:57:09.985102+0000","last_clean":"2024-04-25T09:57:09.985102+0000","last_became_active":"2024-04-25T09:57:09.981081+0000","last_became_peered":"2024-04-25T09:57:09.981081+0000","last_unstale":"2024-04-25T09:57:09.985102+0000","last_undegraded":"2024-04-25T09:57:09.985102+0000","last_fullsized":"2024-04-25T09:57:09.985102+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983313+0000","last_change":"2024-04-25T09:57:09.983313+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983313+0000","last_undegraded":"2024-04-25T09:57:09.983313+0000","last_fullsized":"2024-04-25T09:57:09.983313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992963+0000","last_change":"2024-04-25T09:57:09.991460+0000","last_active":"2024-04-25T09:57:09.992963+0000","last_peered":"2024-04-25T09:57:09.992963+0000","last_clean":"2024-04-25T09:57:09.992963+0000","last_became_active":"2024-04-25T09:57:09.991137+0000","last_became_peered":"2024-04-25T09:57:09.991137+0000","last_unstale":"2024-04-25T09:57:09.992963+0000","last_undegraded":"2024-04-25T09:57:09.992963+0000","last_fullsized":"2024-04-25T09:57:09.992963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.780147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.976028+0000","last_change":"2024-04-25T09:57:09.976028+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.976028+0000","last_undegraded":"2024-04-25T09:57:09.976028+0000","last_fullsized":"2024-04-25T09:57:09.976028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997454+0000","last_change":"2024-04-25T09:57:09.996022+0000","last_active":"2024-04-25T09:57:09.997454+0000","last_peered":"2024-04-25T09:57:09.997454+0000","last_clean":"2024-04-25T09:57:09.997454+0000","last_became_active":"2024-04-25T09:57:09.995252+0000","last_became_peered":"2024-04-25T09:57:09.995252+0000","last_unstale":"2024-04-25T09:57:09.997454+0000","last_undegraded":"2024-04-25T09:57:09.997454+0000","last_fullsized":"2024-04-25T09:57:09.997454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:03.771862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978626+0000","last_change":"2024-04-25T09:57:09.978626+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978626+0000","last_undegraded":"2024-04-25T09:57:09.978626+0000","last_fullsized":"2024-04-25T09:57:09.978626+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991523+0000","last_change":"2024-04-25T09:57:09.988947+0000","last_active":"2024-04-25T09:57:09.991523+0000","last_peered":"2024-04-25T09:57:09.991523+0000","last_clean":"2024-04-25T09:57:09.991523+0000","last_became_active":"2024-04-25T09:57:09.988669+0000","last_became_peered":"2024-04-25T09:57:09.988669+0000","last_unstale":"2024-04-25T09:57:09.991523+0000","last_undegraded":"2024-04-25T09:57:09.991523+0000","last_fullsized":"2024-04-25T09:57:09.991523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983786+0000","last_change":"2024-04-25T09:57:09.983786+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983786+0000","last_undegraded":"2024-04-25T09:57:09.983786+0000","last_fullsized":"2024-04-25T09:57:09.983786+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.985528+0000","last_change":"2024-04-25T09:57:09.983965+0000","last_active":"2024-04-25T09:57:09.985528+0000","last_peered":"2024-04-25T09:57:09.985528+0000","last_clean":"2024-04-25T09:57:09.985528+0000","last_became_active":"2024-04-25T09:57:09.983606+0000","last_became_peered":"2024-04-25T09:57:09.983606+0000","last_unstale":"2024-04-25T09:57:09.985528+0000","last_undegraded":"2024-04-25T09:57:09.985528+0000","last_fullsized":"2024-04-25T09:57:09.985528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:27.673019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.979732+0000","last_change":"2024-04-25T09:57:09.979732+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.979732+0000","last_undegraded":"2024-04-25T09:57:09.979732+0000","last_fullsized":"2024-04-25T09:57:09.979732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989924+0000","last_change":"2024-04-25T09:57:09.986995+0000","last_active":"2024-04-25T09:57:09.989924+0000","last_peered":"2024-04-25T09:57:09.989924+0000","last_clean":"2024-04-25T09:57:09.989924+0000","last_became_active":"2024-04-25T09:57:09.986584+0000","last_became_peered":"2024-04-25T09:57:09.986584+0000","last_unstale":"2024-04-25T09:57:09.989924+0000","last_undegraded":"2024-04-25T09:57:09.989924+0000","last_fullsized":"2024-04-25T09:57:09.989924+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:46.383736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.984631+0000","last_change":"2024-04-25T09:57:09.984631+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.984631+0000","last_undegraded":"2024-04-25T09:57:09.984631+0000","last_fullsized":"2024-04-25T09:57:09.984631+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997402+0000","last_change":"2024-04-25T09:57:09.995540+0000","last_active":"2024-04-25T09:57:09.997402+0000","last_peered":"2024-04-25T09:57:09.997402+0000","last_clean":"2024-04-25T09:57:09.997402+0000","last_became_active":"2024-04-25T09:57:09.995243+0000","last_became_peered":"2024-04-25T09:57:09.995243+0000","last_unstale":"2024-04-25T09:57:09.997402+0000","last_undegraded":"2024-04-25T09:57:09.997402+0000","last_fullsized":"2024-04-25T09:57:09.997402+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:32.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.978354+0000","last_change":"2024-04-25T09:57:09.978354+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.978354+0000","last_undegraded":"2024-04-25T09:57:09.978354+0000","last_fullsized":"2024-04-25T09:57:09.978354+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997145+0000","last_change":"2024-04-25T09:57:09.995365+0000","last_active":"2024-04-25T09:57:09.997145+0000","last_peered":"2024-04-25T09:57:09.997145+0000","last_clean":"2024-04-25T09:57:09.997145+0000","last_became_active":"2024-04-25T09:57:09.995083+0000","last_became_peered":"2024-04-25T09:57:09.995083+0000","last_unstale":"2024-04-25T09:57:09.997145+0000","last_undegraded":"2024-04-25T09:57:09.997145+0000","last_fullsized":"2024-04-25T09:57:09.997145+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:57.260934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982863+0000","last_change":"2024-04-25T09:57:09.982863+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982863+0000","last_undegraded":"2024-04-25T09:57:09.982863+0000","last_fullsized":"2024-04-25T09:57:09.982863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986423+0000","last_change":"2024-04-25T09:57:09.984256+0000","last_active":"2024-04-25T09:57:09.986423+0000","last_peered":"2024-04-25T09:57:09.986423+0000","last_clean":"2024-04-25T09:57:09.986423+0000","last_became_active":"2024-04-25T09:57:09.983972+0000","last_became_peered":"2024-04-25T09:57:09.983972+0000","last_unstale":"2024-04-25T09:57:09.986423+0000","last_undegraded":"2024-04-25T09:57:09.986423+0000","last_fullsized":"2024-04-25T09:57:09.986423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:23.222733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.975678+0000","last_change":"2024-04-25T09:57:09.975678+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.975678+0000","last_undegraded":"2024-04-25T09:57:09.975678+0000","last_fullsized":"2024-04-25T09:57:09.975678+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992827+0000","last_change":"2024-04-25T09:57:09.991436+0000","last_active":"2024-04-25T09:57:09.992827+0000","last_peered":"2024-04-25T09:57:09.992827+0000","last_clean":"2024-04-25T09:57:09.992827+0000","last_became_active":"2024-04-25T09:57:09.991138+0000","last_became_peered":"2024-04-25T09:57:09.991138+0000","last_unstale":"2024-04-25T09:57:09.992827+0000","last_undegraded":"2024-04-25T09:57:09.992827+0000","last_fullsized":"2024-04-25T09:57:09.992827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:31.815985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982962+0000","last_change":"2024-04-25T09:57:09.982962+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982962+0000","last_undegraded":"2024-04-25T09:57:09.982962+0000","last_fullsized":"2024-04-25T09:57:09.982962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.990260+0000","last_change":"2024-04-25T09:57:09.988005+0000","last_active":"2024-04-25T09:57:09.990260+0000","last_peered":"2024-04-25T09:57:09.990260+0000","last_clean":"2024-04-25T09:57:09.990260+0000","last_became_active":"2024-04-25T09:57:09.987706+0000","last_became_peered":"2024-04-25T09:57:09.987706+0000","last_unstale":"2024-04-25T09:57:09.990260+0000","last_undegraded":"2024-04-25T09:57:09.990260+0000","last_fullsized":"2024-04-25T09:57:09.990260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:48.802471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977199+0000","last_change":"2024-04-25T09:57:09.977199+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977199+0000","last_undegraded":"2024-04-25T09:57:09.977199+0000","last_fullsized":"2024-04-25T09:57:09.977199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994711+0000","last_change":"2024-04-25T09:57:09.990696+0000","last_active":"2024-04-25T09:57:09.994711+0000","last_peered":"2024-04-25T09:57:09.994711+0000","last_clean":"2024-04-25T09:57:09.994711+0000","last_became_active":"2024-04-25T09:57:09.990433+0000","last_became_peered":"2024-04-25T09:57:09.990433+0000","last_unstale":"2024-04-25T09:57:09.994711+0000","last_undegraded":"2024-04-25T09:57:09.994711+0000","last_fullsized":"2024-04-25T09:57:09.994711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980632+0000","last_change":"2024-04-25T09:57:09.980632+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980632+0000","last_undegraded":"2024-04-25T09:57:09.980632+0000","last_fullsized":"2024-04-25T09:57:09.980632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997423+0000","last_change":"2024-04-25T09:57:09.995784+0000","last_active":"2024-04-25T09:57:09.997423+0000","last_peered":"2024-04-25T09:57:09.997423+0000","last_clean":"2024-04-25T09:57:09.997423+0000","last_became_active":"2024-04-25T09:57:09.995393+0000","last_became_peered":"2024-04-25T09:57:09.995393+0000","last_unstale":"2024-04-25T09:57:09.997423+0000","last_undegraded":"2024-04-25T09:57:09.997423+0000","last_fullsized":"2024-04-25T09:57:09.997423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:44.157462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982756+0000","last_change":"2024-04-25T09:57:09.982756+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982756+0000","last_undegraded":"2024-04-25T09:57:09.982756+0000","last_fullsized":"2024-04-25T09:57:09.982756+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994920+0000","last_change":"2024-04-25T09:57:09.993056+0000","last_active":"2024-04-25T09:57:09.994920+0000","last_peered":"2024-04-25T09:57:09.994920+0000","last_clean":"2024-04-25T09:57:09.994920+0000","last_became_active":"2024-04-25T09:57:09.992768+0000","last_became_peered":"2024-04-25T09:57:09.992768+0000","last_unstale":"2024-04-25T09:57:09.994920+0000","last_undegraded":"2024-04-25T09:57:09.994920+0000","last_fullsized":"2024-04-25T09:57:09.994920+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.980965+0000","last_change":"2024-04-25T09:57:09.980965+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.980965+0000","last_undegraded":"2024-04-25T09:57:09.980965+0000","last_fullsized":"2024-04-25T09:57:09.980965+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.994590+0000","last_change":"2024-04-25T09:57:09.987312+0000","last_active":"2024-04-25T09:57:09.994590+0000","last_peered":"2024-04-25T09:57:09.994590+0000","last_clean":"2024-04-25T09:57:09.994590+0000","last_became_active":"2024-04-25T09:57:09.987064+0000","last_became_peered":"2024-04-25T09:57:09.987064+0000","last_unstale":"2024-04-25T09:57:09.994590+0000","last_undegraded":"2024-04-25T09:57:09.994590+0000","last_fullsized":"2024-04-25T09:57:09.994590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:46.889960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977222+0000","last_change":"2024-04-25T09:57:09.977222+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977222+0000","last_undegraded":"2024-04-25T09:57:09.977222+0000","last_fullsized":"2024-04-25T09:57:09.977222+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.986936+0000","last_change":"2024-04-25T09:57:09.985025+0000","last_active":"2024-04-25T09:57:09.986936+0000","last_peered":"2024-04-25T09:57:09.986936+0000","last_clean":"2024-04-25T09:57:09.986936+0000","last_became_active":"2024-04-25T09:57:09.984357+0000","last_became_peered":"2024-04-25T09:57:09.984357+0000","last_unstale":"2024-04-25T09:57:09.986936+0000","last_undegraded":"2024-04-25T09:57:09.986936+0000","last_fullsized":"2024-04-25T09:57:09.986936+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:11.720778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981755+0000","last_change":"2024-04-25T09:57:09.981755+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981755+0000","last_undegraded":"2024-04-25T09:57:09.981755+0000","last_fullsized":"2024-04-25T09:57:09.981755+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.992985+0000","last_change":"2024-04-25T09:57:09.991923+0000","last_active":"2024-04-25T09:57:09.992985+0000","last_peered":"2024-04-25T09:57:09.992985+0000","last_clean":"2024-04-25T09:57:09.992985+0000","last_became_active":"2024-04-25T09:57:09.991642+0000","last_became_peered":"2024-04-25T09:57:09.991642+0000","last_unstale":"2024-04-25T09:57:09.992985+0000","last_undegraded":"2024-04-25T09:57:09.992985+0000","last_fullsized":"2024-04-25T09:57:09.992985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.657727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977462+0000","last_change":"2024-04-25T09:57:09.977462+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977462+0000","last_undegraded":"2024-04-25T09:57:09.977462+0000","last_fullsized":"2024-04-25T09:57:09.977462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.984066+0000","last_change":"2024-04-25T09:57:09.982146+0000","last_active":"2024-04-25T09:57:09.984066+0000","last_peered":"2024-04-25T09:57:09.984066+0000","last_clean":"2024-04-25T09:57:09.984066+0000","last_became_active":"2024-04-25T09:57:09.981876+0000","last_became_peered":"2024-04-25T09:57:09.981876+0000","last_unstale":"2024-04-25T09:57:09.984066+0000","last_undegraded":"2024-04-25T09:57:09.984066+0000","last_fullsized":"2024-04-25T09:57:09.984066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.982765+0000","last_change":"2024-04-25T09:57:09.982765+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.982765+0000","last_undegraded":"2024-04-25T09:57:09.982765+0000","last_fullsized":"2024-04-25T09:57:09.982765+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.997380+0000","last_change":"2024-04-25T09:57:09.995462+0000","last_active":"2024-04-25T09:57:09.997380+0000","last_peered":"2024-04-25T09:57:09.997380+0000","last_clean":"2024-04-25T09:57:09.997380+0000","last_became_active":"2024-04-25T09:57:09.995179+0000","last_became_peered":"2024-04-25T09:57:09.995179+0000","last_unstale":"2024-04-25T09:57:09.997380+0000","last_undegraded":"2024-04-25T09:57:09.997380+0000","last_fullsized":"2024-04-25T09:57:09.997380+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.011448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.977753+0000","last_change":"2024-04-25T09:57:09.977753+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.977753+0000","last_undegraded":"2024-04-25T09:57:09.977753+0000","last_fullsized":"2024-04-25T09:57:09.977753+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.996932+0000","last_change":"2024-04-25T09:57:09.994224+0000","last_active":"2024-04-25T09:57:09.996932+0000","last_peered":"2024-04-25T09:57:09.996932+0000","last_clean":"2024-04-25T09:57:09.996932+0000","last_became_active":"2024-04-25T09:57:09.993902+0000","last_became_peered":"2024-04-25T09:57:09.993902+0000","last_unstale":"2024-04-25T09:57:09.996932+0000","last_undegraded":"2024-04-25T09:57:09.996932+0000","last_fullsized":"2024-04-25T09:57:09.996932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.983273+0000","last_change":"2024-04-25T09:57:09.983273+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.983273+0000","last_undegraded":"2024-04-25T09:57:09.983273+0000","last_fullsized":"2024-04-25T09:57:09.983273+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.989862+0000","last_change":"2024-04-25T09:57:09.988098+0000","last_active":"2024-04-25T09:57:09.989862+0000","last_peered":"2024-04-25T09:57:09.989862+0000","last_clean":"2024-04-25T09:57:09.989862+0000","last_became_active":"2024-04-25T09:57:09.987672+0000","last_became_peered":"2024-04-25T09:57:09.987672+0000","last_unstale":"2024-04-25T09:57:09.989862+0000","last_undegraded":"2024-04-25T09:57:09.989862+0000","last_fullsized":"2024-04-25T09:57:09.989862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981392+0000","last_change":"2024-04-25T09:57:09.981392+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981392+0000","last_undegraded":"2024-04-25T09:57:09.981392+0000","last_fullsized":"2024-04-25T09:57:09.981392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.987441+0000","last_change":"2024-04-25T09:57:09.985927+0000","last_active":"2024-04-25T09:57:09.987441+0000","last_peered":"2024-04-25T09:57:09.987441+0000","last_clean":"2024-04-25T09:57:09.987441+0000","last_became_active":"2024-04-25T09:57:09.985657+0000","last_became_peered":"2024-04-25T09:57:09.985657+0000","last_unstale":"2024-04-25T09:57:09.987441+0000","last_undegraded":"2024-04-25T09:57:09.987441+0000","last_fullsized":"2024-04-25T09:57:09.987441+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T09:57:09.981050+0000","last_change":"2024-04-25T09:57:09.981050+0000","last_active":"2024-04-25T09:57:09.966917+0000","last_peered":"2024-04-25T09:57:09.966917+0000","last_clean":"2024-04-25T09:57:09.966917+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:57:09.981050+0000","last_undegraded":"2024-04-25T09:57:09.981050+0000","last_fullsized":"2024-04-25T09:57:09.981050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.991567+0000","last_change":"2024-04-25T09:57:09.990770+0000","last_active":"2024-04-25T09:57:09.991567+0000","last_peered":"2024-04-25T09:57:09.991567+0000","last_clean":"2024-04-25T09:57:09.991567+0000","last_became_active":"2024-04-25T09:57:09.990495+0000","last_became_peered":"2024-04-25T09:57:09.990495+0000","last_unstale":"2024-04-25T09:57:09.991567+0000","last_undegraded":"2024-04-25T09:57:09.991567+0000","last_fullsized":"2024-04-25T09:57:09.991567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T09:57:09.981492+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:09.981492+0000","last_peered":"2024-04-25T09:57:09.981492+0000","last_clean":"2024-04-25T09:57:09.981492+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:09.981492+0000","last_undegraded":"2024-04-25T09:57:09.981492+0000","last_fullsized":"2024-04-25T09:57:09.981492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":866229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":618496,"data_stored":275861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1224704,"data_stored":872896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:16.233 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T09:57:16.229+0000 7f488b419640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:57:18.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:57:18.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.407+0000 7f8b20e15640 1 Processor -- start 2024-04-25T09:57:18.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.407+0000 7f8b20e15640 1 -- start start 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b20e15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b20e15640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b1c0cc320 0x7f8b1c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b20e15640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c0d0480 con 0x7f8b1c0b2d40 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b20e15640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c15c1b0 con 0x7f8b1c0cc320 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b20e15640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15c490 con 0x7f8b1c1597d0 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b1ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b1c0cc320 0x7f8b1c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b1ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b1c0cc320 0x7f8b1c06b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43820/0 (socket says 172.21.15.148:43820) 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b19d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8b1c0b2d40 0x7f8b1c0ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38988/0 (socket says 172.21.15.148:38988) 2024-04-25T09:57:18.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b1ad9d640 1 -- 172.21.15.148:0/2991740938 learned_addr learned my addr 172.21.15.148:0/2991740938 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:18.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.408+0000 7f8b1a59c640 1 --2- 172.21.15.148:0/2991740938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1ad9d640 1 -- 172.21.15.148:0/2991740938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 msgr2=0x7f8b1c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1ad9d640 1 --2- 172.21.15.148:0/2991740938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1ad9d640 1 -- 172.21.15.148:0/2991740938 >> v1:172.21.15.148:6789/0 conn(0x7f8b1c0b2d40 legacy=0x7f8b1c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1959a640 1 -- 172.21.15.148:0/2991740938 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649538715 0 0) 0x7f8b1c0d0480 con 0x7f8b1c0b2d40 2024-04-25T09:57:18.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1ad9d640 1 -- 172.21.15.148:0/2991740938 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c15c770 con 0x7f8b1c0cc320 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1ad9d640 1 --2- 172.21.15.148:0/2991740938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06b120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009870 tx=0x7f8b1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b48c461ffc2b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1959a640 1 -- 172.21.15.148:0/2991740938 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14037070 con 0x7f8b1c0cc320 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1959a640 1 -- 172.21.15.148:0/2991740938 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14032440 con 0x7f8b1c0cc320 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.409+0000 7f8b1959a640 1 -- 172.21.15.148:0/2991740938 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14032810 con 0x7f8b1c0cc320 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 -- 172.21.15.148:0/2991740938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 msgr2=0x7f8b1c06b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2991740938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06b120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009870 tx=0x7f8b1402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:18.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 -- 172.21.15.148:0/2991740938 shutdown_connections 2024-04-25T09:57:18.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2991740938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06b120 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2991740938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 -- 172.21.15.148:0/2991740938 >> 172.21.15.148:0/2991740938 conn(0x7f8b1c0b2220 msgr2=0x7f8b1c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 -- 172.21.15.148:0/2991740938 shutdown_connections 2024-04-25T09:57:18.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.410+0000 7f8b20e15640 1 -- 172.21.15.148:0/2991740938 wait complete. 2024-04-25T09:57:18.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.411+0000 7f8b20e15640 1 Processor -- start 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 -- start start 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c0b2d40 0x7f8b1c0c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c06b920 con 0x7f8b1c0cc320 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c06b7a0 con 0x7f8b1c0b2d40 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b20e15640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0cf570 con 0x7f8b1c1597d0 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43836/0 (socket says 172.21.15.148:43836) 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b19d9b640 1 -- 172.21.15.148:0/2249448529 learned_addr learned my addr 172.21.15.148:0/2249448529 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b1ad9d640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.412+0000 7f8b1a59c640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c0b2d40 0x7f8b1c0c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b19d9b640 1 -- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 msgr2=0x7f8b1c0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b19d9b640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c0ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b19d9b640 1 -- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c0b2d40 msgr2=0x7f8b1c0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b19d9b640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c0b2d40 0x7f8b1c0c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b19d9b640 1 -- 172.21.15.148:0/2249448529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c070410 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b1a59c640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c0b2d40 0x7f8b1c0c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b19d9b640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06fed0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400c4b0 tx=0x7f8b0400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04018070 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b04007500 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c0c4530 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04004a20 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.413+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c0c4af0 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.414+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c073990 con 0x7f8b1c0cc320 2024-04-25T09:57:18.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.415+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b0401b070 con 0x7f8b1c0cc320 2024-04-25T09:57:18.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.415+0000 7f8b12ffd640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8afc06b4d0 0x7f8afc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.415+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8b04089630 con 0x7f8b1c0cc320 2024-04-25T09:57:18.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.415+0000 7f8b1a59c640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8afc06b4d0 0x7f8afc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.415+0000 7f8b1a59c640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8afc06b4d0 0x7f8afc06d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009a80 tx=0x7f8b0c0039d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.419+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b04057d00 con 0x7f8b1c0cc320 2024-04-25T09:57:18.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.586+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b1c0cd490 con 0x7f8afc06b4d0 2024-04-25T09:57:18.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.592+0000 7f8b12ffd640 1 -- 172.21.15.148:0/2249448529 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251957 (secure 0 0 0) 0x7f8b1c0cd490 con 0x7f8afc06b4d0 2024-04-25T09:57:18.598 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:18.599 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8afc06b4d0 msgr2=0x7f8afc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8afc06b4d0 0x7f8afc06d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009a80 tx=0x7f8b0c0039d0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 msgr2=0x7f8b1c06fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06fed0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400c4b0 tx=0x7f8b0400c970 comp rx=0 tx=0).stop 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 shutdown_connections 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8afc06b4d0 0x7f8afc06d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c1597d0 0x7f8b1c0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b1c0cc320 0x7f8b1c06fed0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 --2- 172.21.15.148:0/2249448529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c0b2d40 0x7f8b1c0c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 >> 172.21.15.148:0/2249448529 conn(0x7f8b1c0b2220 msgr2=0x7f8b1c0cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.595+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 shutdown_connections 2024-04-25T09:57:18.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.596+0000 7f8b20e15640 1 -- 172.21.15.148:0/2249448529 wait complete. 2024-04-25T09:57:18.610 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T09:57:17.055265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":280768,"kb_used_data":6344,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754693952,"statfs":{"total":773094113280,"available":772806606848,"internally_reserved":0,"allocated":6496256,"data_stored":3556652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.082304"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358165+0000","last_change":"2024-04-25T09:57:10.998249+0000","last_active":"2024-04-25T09:57:12.358165+0000","last_peered":"2024-04-25T09:57:12.358165+0000","last_clean":"2024-04-25T09:57:12.358165+0000","last_became_active":"2024-04-25T09:57:10.995125+0000","last_became_peered":"2024-04-25T09:57:10.995125+0000","last_unstale":"2024-04-25T09:57:12.358165+0000","last_undegraded":"2024-04-25T09:57:12.358165+0000","last_fullsized":"2024-04-25T09:57:12.358165+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:33.712843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994329+0000","last_change":"2024-04-25T09:57:10.994243+0000","last_active":"2024-04-25T09:57:11.994329+0000","last_peered":"2024-04-25T09:57:11.994329+0000","last_clean":"2024-04-25T09:57:11.994329+0000","last_became_active":"2024-04-25T09:57:10.993950+0000","last_became_peered":"2024-04-25T09:57:10.993950+0000","last_unstale":"2024-04-25T09:57:11.994329+0000","last_undegraded":"2024-04-25T09:57:11.994329+0000","last_fullsized":"2024-04-25T09:57:11.994329+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:08.008263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979303+0000","last_change":"2024-04-25T09:57:11.010242+0000","last_active":"2024-04-25T09:57:12.979303+0000","last_peered":"2024-04-25T09:57:12.979303+0000","last_clean":"2024-04-25T09:57:12.979303+0000","last_became_active":"2024-04-25T09:57:11.009959+0000","last_became_peered":"2024-04-25T09:57:11.009959+0000","last_unstale":"2024-04-25T09:57:12.979303+0000","last_undegraded":"2024-04-25T09:57:12.979303+0000","last_fullsized":"2024-04-25T09:57:12.979303+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:32.921691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993778+0000","last_change":"2024-04-25T09:57:10.996447+0000","last_active":"2024-04-25T09:57:11.993778+0000","last_peered":"2024-04-25T09:57:11.993778+0000","last_clean":"2024-04-25T09:57:11.993778+0000","last_became_active":"2024-04-25T09:57:10.996181+0000","last_became_peered":"2024-04-25T09:57:10.996181+0000","last_unstale":"2024-04-25T09:57:11.993778+0000","last_undegraded":"2024-04-25T09:57:11.993778+0000","last_fullsized":"2024-04-25T09:57:11.993778+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:44.620523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.995227+0000","last_change":"2024-04-25T09:57:10.984754+0000","last_active":"2024-04-25T09:57:11.995227+0000","last_peered":"2024-04-25T09:57:11.995227+0000","last_clean":"2024-04-25T09:57:11.995227+0000","last_became_active":"2024-04-25T09:57:10.984425+0000","last_became_peered":"2024-04-25T09:57:10.984425+0000","last_unstale":"2024-04-25T09:57:11.995227+0000","last_undegraded":"2024-04-25T09:57:11.995227+0000","last_fullsized":"2024-04-25T09:57:11.995227+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:14.665259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.357969+0000","last_change":"2024-04-25T09:57:10.993960+0000","last_active":"2024-04-25T09:57:12.357969+0000","last_peered":"2024-04-25T09:57:12.357969+0000","last_clean":"2024-04-25T09:57:12.357969+0000","last_became_active":"2024-04-25T09:57:10.992580+0000","last_became_peered":"2024-04-25T09:57:10.992580+0000","last_unstale":"2024-04-25T09:57:12.357969+0000","last_undegraded":"2024-04-25T09:57:12.357969+0000","last_fullsized":"2024-04-25T09:57:12.357969+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:13.412091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979085+0000","last_change":"2024-04-25T09:57:11.009451+0000","last_active":"2024-04-25T09:57:12.979085+0000","last_peered":"2024-04-25T09:57:12.979085+0000","last_clean":"2024-04-25T09:57:12.979085+0000","last_became_active":"2024-04-25T09:57:11.009196+0000","last_became_peered":"2024-04-25T09:57:11.009196+0000","last_unstale":"2024-04-25T09:57:12.979085+0000","last_undegraded":"2024-04-25T09:57:12.979085+0000","last_fullsized":"2024-04-25T09:57:12.979085+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:19.777591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500551+0000","last_change":"2024-04-25T09:57:11.005863+0000","last_active":"2024-04-25T09:57:12.500551+0000","last_peered":"2024-04-25T09:57:12.500551+0000","last_clean":"2024-04-25T09:57:12.500551+0000","last_became_active":"2024-04-25T09:57:11.005587+0000","last_became_peered":"2024-04-25T09:57:11.005587+0000","last_unstale":"2024-04-25T09:57:12.500551+0000","last_undegraded":"2024-04-25T09:57:12.500551+0000","last_fullsized":"2024-04-25T09:57:12.500551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:57.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500947+0000","last_change":"2024-04-25T09:57:11.013426+0000","last_active":"2024-04-25T09:57:12.500947+0000","last_peered":"2024-04-25T09:57:12.500947+0000","last_clean":"2024-04-25T09:57:12.500947+0000","last_became_active":"2024-04-25T09:57:11.013115+0000","last_became_peered":"2024-04-25T09:57:11.013115+0000","last_unstale":"2024-04-25T09:57:12.500947+0000","last_undegraded":"2024-04-25T09:57:12.500947+0000","last_fullsized":"2024-04-25T09:57:12.500947+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:31.118303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358190+0000","last_change":"2024-04-25T09:57:10.993186+0000","last_active":"2024-04-25T09:57:12.358190+0000","last_peered":"2024-04-25T09:57:12.358190+0000","last_clean":"2024-04-25T09:57:12.358190+0000","last_became_active":"2024-04-25T09:57:10.992901+0000","last_became_peered":"2024-04-25T09:57:10.992901+0000","last_unstale":"2024-04-25T09:57:12.358190+0000","last_undegraded":"2024-04-25T09:57:12.358190+0000","last_fullsized":"2024-04-25T09:57:12.358190+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:45.164015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.756533+0000","last_change":"2024-04-25T09:57:10.986150+0000","last_active":"2024-04-25T09:57:12.756533+0000","last_peered":"2024-04-25T09:57:12.756533+0000","last_clean":"2024-04-25T09:57:12.756533+0000","last_became_active":"2024-04-25T09:57:10.985891+0000","last_became_peered":"2024-04-25T09:57:10.985891+0000","last_unstale":"2024-04-25T09:57:12.756533+0000","last_undegraded":"2024-04-25T09:57:12.756533+0000","last_fullsized":"2024-04-25T09:57:12.756533+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:05.939217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500902+0000","last_change":"2024-04-25T09:57:10.994385+0000","last_active":"2024-04-25T09:57:12.500902+0000","last_peered":"2024-04-25T09:57:12.500902+0000","last_clean":"2024-04-25T09:57:12.500902+0000","last_became_active":"2024-04-25T09:57:10.992372+0000","last_became_peered":"2024-04-25T09:57:10.992372+0000","last_unstale":"2024-04-25T09:57:12.500902+0000","last_undegraded":"2024-04-25T09:57:12.500902+0000","last_fullsized":"2024-04-25T09:57:12.500902+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:09.762236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.357975+0000","last_change":"2024-04-25T09:57:10.995035+0000","last_active":"2024-04-25T09:57:12.357975+0000","last_peered":"2024-04-25T09:57:12.357975+0000","last_clean":"2024-04-25T09:57:12.357975+0000","last_became_active":"2024-04-25T09:57:10.993124+0000","last_became_peered":"2024-04-25T09:57:10.993124+0000","last_unstale":"2024-04-25T09:57:12.357975+0000","last_undegraded":"2024-04-25T09:57:12.357975+0000","last_fullsized":"2024-04-25T09:57:12.357975+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:03.602853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979118+0000","last_change":"2024-04-25T09:57:11.012597+0000","last_active":"2024-04-25T09:57:12.979118+0000","last_peered":"2024-04-25T09:57:12.979118+0000","last_clean":"2024-04-25T09:57:12.979118+0000","last_became_active":"2024-04-25T09:57:11.012097+0000","last_became_peered":"2024-04-25T09:57:11.012097+0000","last_unstale":"2024-04-25T09:57:12.979118+0000","last_undegraded":"2024-04-25T09:57:12.979118+0000","last_fullsized":"2024-04-25T09:57:12.979118+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:08.008263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993534+0000","last_change":"2024-04-25T09:57:10.992098+0000","last_active":"2024-04-25T09:57:11.993534+0000","last_peered":"2024-04-25T09:57:11.993534+0000","last_clean":"2024-04-25T09:57:11.993534+0000","last_became_active":"2024-04-25T09:57:10.991823+0000","last_became_peered":"2024-04-25T09:57:10.991823+0000","last_unstale":"2024-04-25T09:57:11.993534+0000","last_undegraded":"2024-04-25T09:57:11.993534+0000","last_fullsized":"2024-04-25T09:57:11.993534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:52.256074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.357909+0000","last_change":"2024-04-25T09:57:10.985582+0000","last_active":"2024-04-25T09:57:12.357909+0000","last_peered":"2024-04-25T09:57:12.357909+0000","last_clean":"2024-04-25T09:57:12.357909+0000","last_became_active":"2024-04-25T09:57:10.985308+0000","last_became_peered":"2024-04-25T09:57:10.985308+0000","last_unstale":"2024-04-25T09:57:12.357909+0000","last_undegraded":"2024-04-25T09:57:12.357909+0000","last_fullsized":"2024-04-25T09:57:12.357909+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979715+0000","last_change":"2024-04-25T09:57:11.013259+0000","last_active":"2024-04-25T09:57:12.979715+0000","last_peered":"2024-04-25T09:57:12.979715+0000","last_clean":"2024-04-25T09:57:12.979715+0000","last_became_active":"2024-04-25T09:57:11.012633+0000","last_became_peered":"2024-04-25T09:57:11.012633+0000","last_unstale":"2024-04-25T09:57:12.979715+0000","last_undegraded":"2024-04-25T09:57:12.979715+0000","last_fullsized":"2024-04-25T09:57:12.979715+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:57.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994106+0000","last_change":"2024-04-25T09:57:10.993982+0000","last_active":"2024-04-25T09:57:11.994106+0000","last_peered":"2024-04-25T09:57:11.994106+0000","last_clean":"2024-04-25T09:57:11.994106+0000","last_became_active":"2024-04-25T09:57:10.993684+0000","last_became_peered":"2024-04-25T09:57:10.993684+0000","last_unstale":"2024-04-25T09:57:11.994106+0000","last_undegraded":"2024-04-25T09:57:11.994106+0000","last_fullsized":"2024-04-25T09:57:11.994106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:18.333100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979348+0000","last_change":"2024-04-25T09:57:11.012226+0000","last_active":"2024-04-25T09:57:12.979348+0000","last_peered":"2024-04-25T09:57:12.979348+0000","last_clean":"2024-04-25T09:57:12.979348+0000","last_became_active":"2024-04-25T09:57:11.011939+0000","last_became_peered":"2024-04-25T09:57:11.011939+0000","last_unstale":"2024-04-25T09:57:12.979348+0000","last_undegraded":"2024-04-25T09:57:12.979348+0000","last_fullsized":"2024-04-25T09:57:12.979348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.640272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384499+0000","last_change":"2024-04-25T09:57:10.990887+0000","last_active":"2024-04-25T09:57:13.384499+0000","last_peered":"2024-04-25T09:57:13.384499+0000","last_clean":"2024-04-25T09:57:13.384499+0000","last_became_active":"2024-04-25T09:57:10.990321+0000","last_became_peered":"2024-04-25T09:57:10.990321+0000","last_unstale":"2024-04-25T09:57:13.384499+0000","last_undegraded":"2024-04-25T09:57:13.384499+0000","last_fullsized":"2024-04-25T09:57:13.384499+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384448+0000","last_change":"2024-04-25T09:57:10.990656+0000","last_active":"2024-04-25T09:57:13.384448+0000","last_peered":"2024-04-25T09:57:13.384448+0000","last_clean":"2024-04-25T09:57:13.384448+0000","last_became_active":"2024-04-25T09:57:10.990377+0000","last_became_peered":"2024-04-25T09:57:10.990377+0000","last_unstale":"2024-04-25T09:57:13.384448+0000","last_undegraded":"2024-04-25T09:57:13.384448+0000","last_fullsized":"2024-04-25T09:57:13.384448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:30.226423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500588+0000","last_change":"2024-04-25T09:57:11.005425+0000","last_active":"2024-04-25T09:57:12.500588+0000","last_peered":"2024-04-25T09:57:12.500588+0000","last_clean":"2024-04-25T09:57:12.500588+0000","last_became_active":"2024-04-25T09:57:11.005164+0000","last_became_peered":"2024-04-25T09:57:11.005164+0000","last_unstale":"2024-04-25T09:57:12.500588+0000","last_undegraded":"2024-04-25T09:57:12.500588+0000","last_fullsized":"2024-04-25T09:57:12.500588+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:46.057576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.756370+0000","last_change":"2024-04-25T09:57:10.987612+0000","last_active":"2024-04-25T09:57:12.756370+0000","last_peered":"2024-04-25T09:57:12.756370+0000","last_clean":"2024-04-25T09:57:12.756370+0000","last_became_active":"2024-04-25T09:57:10.987339+0000","last_became_peered":"2024-04-25T09:57:10.987339+0000","last_unstale":"2024-04-25T09:57:12.756370+0000","last_undegraded":"2024-04-25T09:57:12.756370+0000","last_fullsized":"2024-04-25T09:57:12.756370+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:28.735135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979026+0000","last_change":"2024-04-25T09:57:11.009659+0000","last_active":"2024-04-25T09:57:12.979026+0000","last_peered":"2024-04-25T09:57:12.979026+0000","last_clean":"2024-04-25T09:57:12.979026+0000","last_became_active":"2024-04-25T09:57:11.009386+0000","last_became_peered":"2024-04-25T09:57:11.009386+0000","last_unstale":"2024-04-25T09:57:12.979026+0000","last_undegraded":"2024-04-25T09:57:12.979026+0000","last_fullsized":"2024-04-25T09:57:12.979026+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:18.333100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979829+0000","last_change":"2024-04-25T09:57:11.013716+0000","last_active":"2024-04-25T09:57:12.979829+0000","last_peered":"2024-04-25T09:57:12.979829+0000","last_clean":"2024-04-25T09:57:12.979829+0000","last_became_active":"2024-04-25T09:57:11.012974+0000","last_became_peered":"2024-04-25T09:57:11.012974+0000","last_unstale":"2024-04-25T09:57:12.979829+0000","last_undegraded":"2024-04-25T09:57:12.979829+0000","last_fullsized":"2024-04-25T09:57:12.979829+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:46.057576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500828+0000","last_change":"2024-04-25T09:57:11.003197+0000","last_active":"2024-04-25T09:57:12.500828+0000","last_peered":"2024-04-25T09:57:12.500828+0000","last_clean":"2024-04-25T09:57:12.500828+0000","last_became_active":"2024-04-25T09:57:10.999052+0000","last_became_peered":"2024-04-25T09:57:10.999052+0000","last_unstale":"2024-04-25T09:57:12.500828+0000","last_undegraded":"2024-04-25T09:57:12.500828+0000","last_fullsized":"2024-04-25T09:57:12.500828+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:37.482367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358236+0000","last_change":"2024-04-25T09:57:11.003400+0000","last_active":"2024-04-25T09:57:12.358236+0000","last_peered":"2024-04-25T09:57:12.358236+0000","last_clean":"2024-04-25T09:57:12.358236+0000","last_became_active":"2024-04-25T09:57:11.002825+0000","last_became_peered":"2024-04-25T09:57:11.002825+0000","last_unstale":"2024-04-25T09:57:12.358236+0000","last_undegraded":"2024-04-25T09:57:12.358236+0000","last_fullsized":"2024-04-25T09:57:12.358236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:08.034672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384597+0000","last_change":"2024-04-25T09:57:10.993930+0000","last_active":"2024-04-25T09:57:13.384597+0000","last_peered":"2024-04-25T09:57:13.384597+0000","last_clean":"2024-04-25T09:57:13.384597+0000","last_became_active":"2024-04-25T09:57:10.993620+0000","last_became_peered":"2024-04-25T09:57:10.993620+0000","last_unstale":"2024-04-25T09:57:13.384597+0000","last_undegraded":"2024-04-25T09:57:13.384597+0000","last_fullsized":"2024-04-25T09:57:13.384597+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:13.412091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979235+0000","last_change":"2024-04-25T09:57:11.012414+0000","last_active":"2024-04-25T09:57:12.979235+0000","last_peered":"2024-04-25T09:57:12.979235+0000","last_clean":"2024-04-25T09:57:12.979235+0000","last_became_active":"2024-04-25T09:57:11.012084+0000","last_became_peered":"2024-04-25T09:57:11.012084+0000","last_unstale":"2024-04-25T09:57:12.979235+0000","last_undegraded":"2024-04-25T09:57:12.979235+0000","last_fullsized":"2024-04-25T09:57:12.979235+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:23.270042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993460+0000","last_change":"2024-04-25T09:57:10.994770+0000","last_active":"2024-04-25T09:57:11.993460+0000","last_peered":"2024-04-25T09:57:11.993460+0000","last_clean":"2024-04-25T09:57:11.993460+0000","last_became_active":"2024-04-25T09:57:10.994495+0000","last_became_peered":"2024-04-25T09:57:10.994495+0000","last_unstale":"2024-04-25T09:57:11.993460+0000","last_undegraded":"2024-04-25T09:57:11.993460+0000","last_fullsized":"2024-04-25T09:57:11.993460+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:08.034672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979173+0000","last_change":"2024-04-25T09:57:11.009646+0000","last_active":"2024-04-25T09:57:12.979173+0000","last_peered":"2024-04-25T09:57:12.979173+0000","last_clean":"2024-04-25T09:57:12.979173+0000","last_became_active":"2024-04-25T09:57:11.009344+0000","last_became_peered":"2024-04-25T09:57:11.009344+0000","last_unstale":"2024-04-25T09:57:12.979173+0000","last_undegraded":"2024-04-25T09:57:12.979173+0000","last_fullsized":"2024-04-25T09:57:12.979173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.339723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390050+0000","last_change":"2024-04-25T09:57:10.991116+0000","last_active":"2024-04-25T09:57:13.390050+0000","last_peered":"2024-04-25T09:57:13.390050+0000","last_clean":"2024-04-25T09:57:13.390050+0000","last_became_active":"2024-04-25T09:57:10.990831+0000","last_became_peered":"2024-04-25T09:57:10.990831+0000","last_unstale":"2024-04-25T09:57:13.390050+0000","last_undegraded":"2024-04-25T09:57:13.390050+0000","last_fullsized":"2024-04-25T09:57:13.390050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:23.270042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993194+0000","last_change":"2024-04-25T09:57:10.985079+0000","last_active":"2024-04-25T09:57:11.993194+0000","last_peered":"2024-04-25T09:57:11.993194+0000","last_clean":"2024-04-25T09:57:11.993194+0000","last_became_active":"2024-04-25T09:57:10.984814+0000","last_became_peered":"2024-04-25T09:57:10.984814+0000","last_unstale":"2024-04-25T09:57:11.993194+0000","last_undegraded":"2024-04-25T09:57:11.993194+0000","last_fullsized":"2024-04-25T09:57:11.993194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:04.778416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384602+0000","last_change":"2024-04-25T09:57:09.986934+0000","last_active":"2024-04-25T09:57:13.384602+0000","last_peered":"2024-04-25T09:57:13.384602+0000","last_clean":"2024-04-25T09:57:13.384602+0000","last_became_active":"2024-04-25T09:57:09.986610+0000","last_became_peered":"2024-04-25T09:57:09.986610+0000","last_unstale":"2024-04-25T09:57:13.384602+0000","last_undegraded":"2024-04-25T09:57:13.384602+0000","last_fullsized":"2024-04-25T09:57:13.384602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:35.037390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390193+0000","last_change":"2024-04-25T09:57:10.984909+0000","last_active":"2024-04-25T09:57:13.390193+0000","last_peered":"2024-04-25T09:57:13.390193+0000","last_clean":"2024-04-25T09:57:13.390193+0000","last_became_active":"2024-04-25T09:57:10.984580+0000","last_became_peered":"2024-04-25T09:57:10.984580+0000","last_unstale":"2024-04-25T09:57:13.390193+0000","last_undegraded":"2024-04-25T09:57:13.390193+0000","last_fullsized":"2024-04-25T09:57:13.390193+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.640272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358263+0000","last_change":"2024-04-25T09:57:09.996107+0000","last_active":"2024-04-25T09:57:12.358263+0000","last_peered":"2024-04-25T09:57:12.358263+0000","last_clean":"2024-04-25T09:57:12.358263+0000","last_became_active":"2024-04-25T09:57:09.995840+0000","last_became_peered":"2024-04-25T09:57:09.995840+0000","last_unstale":"2024-04-25T09:57:12.358263+0000","last_undegraded":"2024-04-25T09:57:12.358263+0000","last_fullsized":"2024-04-25T09:57:12.358263+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384327+0000","last_change":"2024-04-25T09:57:10.994392+0000","last_active":"2024-04-25T09:57:13.384327+0000","last_peered":"2024-04-25T09:57:13.384327+0000","last_clean":"2024-04-25T09:57:13.384327+0000","last_became_active":"2024-04-25T09:57:10.994016+0000","last_became_peered":"2024-04-25T09:57:10.994016+0000","last_unstale":"2024-04-25T09:57:13.384327+0000","last_undegraded":"2024-04-25T09:57:13.384327+0000","last_fullsized":"2024-04-25T09:57:13.384327+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:55.408299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358213+0000","last_change":"2024-04-25T09:57:09.988639+0000","last_active":"2024-04-25T09:57:12.358213+0000","last_peered":"2024-04-25T09:57:12.358213+0000","last_clean":"2024-04-25T09:57:12.358213+0000","last_became_active":"2024-04-25T09:57:09.988362+0000","last_became_peered":"2024-04-25T09:57:09.988362+0000","last_unstale":"2024-04-25T09:57:12.358213+0000","last_undegraded":"2024-04-25T09:57:12.358213+0000","last_fullsized":"2024-04-25T09:57:12.358213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500669+0000","last_change":"2024-04-25T09:57:11.000555+0000","last_active":"2024-04-25T09:57:12.500669+0000","last_peered":"2024-04-25T09:57:12.500669+0000","last_clean":"2024-04-25T09:57:12.500669+0000","last_became_active":"2024-04-25T09:57:11.000292+0000","last_became_peered":"2024-04-25T09:57:11.000292+0000","last_unstale":"2024-04-25T09:57:12.500669+0000","last_undegraded":"2024-04-25T09:57:12.500669+0000","last_fullsized":"2024-04-25T09:57:12.500669+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:32.784026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390406+0000","last_change":"2024-04-25T09:57:09.981554+0000","last_active":"2024-04-25T09:57:13.390406+0000","last_peered":"2024-04-25T09:57:13.390406+0000","last_clean":"2024-04-25T09:57:13.390406+0000","last_became_active":"2024-04-25T09:57:09.981290+0000","last_became_peered":"2024-04-25T09:57:09.981290+0000","last_unstale":"2024-04-25T09:57:13.390406+0000","last_undegraded":"2024-04-25T09:57:13.390406+0000","last_fullsized":"2024-04-25T09:57:13.390406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.011448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.995397+0000","last_change":"2024-04-25T09:57:10.997556+0000","last_active":"2024-04-25T09:57:11.995397+0000","last_peered":"2024-04-25T09:57:11.995397+0000","last_clean":"2024-04-25T09:57:11.995397+0000","last_became_active":"2024-04-25T09:57:10.997199+0000","last_became_peered":"2024-04-25T09:57:10.997199+0000","last_unstale":"2024-04-25T09:57:11.995397+0000","last_undegraded":"2024-04-25T09:57:11.995397+0000","last_fullsized":"2024-04-25T09:57:11.995397+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:27.273877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979125+0000","last_change":"2024-04-25T09:57:09.995396+0000","last_active":"2024-04-25T09:57:12.979125+0000","last_peered":"2024-04-25T09:57:12.979125+0000","last_clean":"2024-04-25T09:57:12.979125+0000","last_became_active":"2024-04-25T09:57:09.995117+0000","last_became_peered":"2024-04-25T09:57:09.995117+0000","last_unstale":"2024-04-25T09:57:12.979125+0000","last_undegraded":"2024-04-25T09:57:12.979125+0000","last_fullsized":"2024-04-25T09:57:12.979125+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.780147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993896+0000","last_change":"2024-04-25T09:57:11.001362+0000","last_active":"2024-04-25T09:57:11.993896+0000","last_peered":"2024-04-25T09:57:11.993896+0000","last_clean":"2024-04-25T09:57:11.993896+0000","last_became_active":"2024-04-25T09:57:11.001141+0000","last_became_peered":"2024-04-25T09:57:11.001141+0000","last_unstale":"2024-04-25T09:57:11.993896+0000","last_undegraded":"2024-04-25T09:57:11.993896+0000","last_fullsized":"2024-04-25T09:57:11.993896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:30.890293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.995454+0000","last_change":"2024-04-25T09:57:09.985140+0000","last_active":"2024-04-25T09:57:11.995454+0000","last_peered":"2024-04-25T09:57:11.995454+0000","last_clean":"2024-04-25T09:57:11.995454+0000","last_became_active":"2024-04-25T09:57:09.983854+0000","last_became_peered":"2024-04-25T09:57:09.983854+0000","last_unstale":"2024-04-25T09:57:11.995454+0000","last_undegraded":"2024-04-25T09:57:11.995454+0000","last_fullsized":"2024-04-25T09:57:11.995454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.029172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500979+0000","last_change":"2024-04-25T09:57:11.002924+0000","last_active":"2024-04-25T09:57:12.500979+0000","last_peered":"2024-04-25T09:57:12.500979+0000","last_clean":"2024-04-25T09:57:12.500979+0000","last_became_active":"2024-04-25T09:57:10.998715+0000","last_became_peered":"2024-04-25T09:57:10.998715+0000","last_unstale":"2024-04-25T09:57:12.500979+0000","last_undegraded":"2024-04-25T09:57:12.500979+0000","last_fullsized":"2024-04-25T09:57:12.500979+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:16.568291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384374+0000","last_change":"2024-04-25T09:57:09.985009+0000","last_active":"2024-04-25T09:57:13.384374+0000","last_peered":"2024-04-25T09:57:13.384374+0000","last_clean":"2024-04-25T09:57:13.384374+0000","last_became_active":"2024-04-25T09:57:09.984733+0000","last_became_peered":"2024-04-25T09:57:09.984733+0000","last_unstale":"2024-04-25T09:57:13.384374+0000","last_undegraded":"2024-04-25T09:57:13.384374+0000","last_fullsized":"2024-04-25T09:57:13.384374+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994111+0000","last_change":"2024-04-25T09:57:11.013128+0000","last_active":"2024-04-25T09:57:11.994111+0000","last_peered":"2024-04-25T09:57:11.994111+0000","last_clean":"2024-04-25T09:57:11.994111+0000","last_became_active":"2024-04-25T09:57:11.012857+0000","last_became_peered":"2024-04-25T09:57:11.012857+0000","last_unstale":"2024-04-25T09:57:11.994111+0000","last_undegraded":"2024-04-25T09:57:11.994111+0000","last_fullsized":"2024-04-25T09:57:11.994111+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:19.777591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979046+0000","last_change":"2024-04-25T09:57:09.988494+0000","last_active":"2024-04-25T09:57:12.979046+0000","last_peered":"2024-04-25T09:57:12.979046+0000","last_clean":"2024-04-25T09:57:12.979046+0000","last_became_active":"2024-04-25T09:57:09.988229+0000","last_became_peered":"2024-04-25T09:57:09.988229+0000","last_unstale":"2024-04-25T09:57:12.979046+0000","last_undegraded":"2024-04-25T09:57:12.979046+0000","last_fullsized":"2024-04-25T09:57:12.979046+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:26.975512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994987+0000","last_change":"2024-04-25T09:57:10.998201+0000","last_active":"2024-04-25T09:57:11.994987+0000","last_peered":"2024-04-25T09:57:11.994987+0000","last_clean":"2024-04-25T09:57:11.994987+0000","last_became_active":"2024-04-25T09:57:10.997887+0000","last_became_peered":"2024-04-25T09:57:10.997887+0000","last_unstale":"2024-04-25T09:57:11.994987+0000","last_undegraded":"2024-04-25T09:57:11.994987+0000","last_fullsized":"2024-04-25T09:57:11.994987+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:28.582303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994200+0000","last_change":"2024-04-25T09:57:09.988417+0000","last_active":"2024-04-25T09:57:11.994200+0000","last_peered":"2024-04-25T09:57:11.994200+0000","last_clean":"2024-04-25T09:57:11.994200+0000","last_became_active":"2024-04-25T09:57:09.987870+0000","last_became_peered":"2024-04-25T09:57:09.987870+0000","last_unstale":"2024-04-25T09:57:11.994200+0000","last_undegraded":"2024-04-25T09:57:11.994200+0000","last_fullsized":"2024-04-25T09:57:11.994200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:09.358541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390032+0000","last_change":"2024-04-25T09:57:10.983110+0000","last_active":"2024-04-25T09:57:13.390032+0000","last_peered":"2024-04-25T09:57:13.390032+0000","last_clean":"2024-04-25T09:57:13.390032+0000","last_became_active":"2024-04-25T09:57:10.982844+0000","last_became_peered":"2024-04-25T09:57:10.982844+0000","last_unstale":"2024-04-25T09:57:13.390032+0000","last_undegraded":"2024-04-25T09:57:13.390032+0000","last_fullsized":"2024-04-25T09:57:13.390032+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.339723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.995508+0000","last_change":"2024-04-25T09:57:09.981400+0000","last_active":"2024-04-25T09:57:11.995508+0000","last_peered":"2024-04-25T09:57:11.995508+0000","last_clean":"2024-04-25T09:57:11.995508+0000","last_became_active":"2024-04-25T09:57:09.981081+0000","last_became_peered":"2024-04-25T09:57:09.981081+0000","last_unstale":"2024-04-25T09:57:11.995508+0000","last_undegraded":"2024-04-25T09:57:11.995508+0000","last_fullsized":"2024-04-25T09:57:11.995508+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:46.383736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979437+0000","last_change":"2024-04-25T09:57:11.000734+0000","last_active":"2024-04-25T09:57:12.979437+0000","last_peered":"2024-04-25T09:57:12.979437+0000","last_clean":"2024-04-25T09:57:12.979437+0000","last_became_active":"2024-04-25T09:57:11.000463+0000","last_became_peered":"2024-04-25T09:57:11.000463+0000","last_unstale":"2024-04-25T09:57:12.979437+0000","last_undegraded":"2024-04-25T09:57:12.979437+0000","last_fullsized":"2024-04-25T09:57:12.979437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:17.709060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994117+0000","last_change":"2024-04-25T09:57:09.991460+0000","last_active":"2024-04-25T09:57:11.994117+0000","last_peered":"2024-04-25T09:57:11.994117+0000","last_clean":"2024-04-25T09:57:11.994117+0000","last_became_active":"2024-04-25T09:57:09.991137+0000","last_became_peered":"2024-04-25T09:57:09.991137+0000","last_unstale":"2024-04-25T09:57:11.994117+0000","last_undegraded":"2024-04-25T09:57:11.994117+0000","last_fullsized":"2024-04-25T09:57:11.994117+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:27.575750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.995274+0000","last_change":"2024-04-25T09:57:10.982804+0000","last_active":"2024-04-25T09:57:11.995274+0000","last_peered":"2024-04-25T09:57:11.995274+0000","last_clean":"2024-04-25T09:57:11.995274+0000","last_became_active":"2024-04-25T09:57:10.982542+0000","last_became_peered":"2024-04-25T09:57:10.982542+0000","last_unstale":"2024-04-25T09:57:11.995274+0000","last_undegraded":"2024-04-25T09:57:11.995274+0000","last_fullsized":"2024-04-25T09:57:11.995274+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:30.226423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":20,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500902+0000","last_change":"2024-04-25T09:57:09.996022+0000","last_active":"2024-04-25T09:57:12.500902+0000","last_peered":"2024-04-25T09:57:12.500902+0000","last_clean":"2024-04-25T09:57:12.500902+0000","last_became_active":"2024-04-25T09:57:09.995252+0000","last_became_peered":"2024-04-25T09:57:09.995252+0000","last_unstale":"2024-04-25T09:57:12.500902+0000","last_undegraded":"2024-04-25T09:57:12.500902+0000","last_fullsized":"2024-04-25T09:57:12.500902+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:43.613970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390197+0000","last_change":"2024-04-25T09:57:10.984175+0000","last_active":"2024-04-25T09:57:13.390197+0000","last_peered":"2024-04-25T09:57:13.390197+0000","last_clean":"2024-04-25T09:57:13.390197+0000","last_became_active":"2024-04-25T09:57:10.983900+0000","last_became_peered":"2024-04-25T09:57:10.983900+0000","last_unstale":"2024-04-25T09:57:13.390197+0000","last_undegraded":"2024-04-25T09:57:13.390197+0000","last_fullsized":"2024-04-25T09:57:13.390197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:42.117546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979326+0000","last_change":"2024-04-25T09:57:09.988947+0000","last_active":"2024-04-25T09:57:12.979326+0000","last_peered":"2024-04-25T09:57:12.979326+0000","last_clean":"2024-04-25T09:57:12.979326+0000","last_became_active":"2024-04-25T09:57:09.988669+0000","last_became_peered":"2024-04-25T09:57:09.988669+0000","last_unstale":"2024-04-25T09:57:12.979326+0000","last_undegraded":"2024-04-25T09:57:12.979326+0000","last_fullsized":"2024-04-25T09:57:12.979326+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.657727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979567+0000","last_change":"2024-04-25T09:57:11.007558+0000","last_active":"2024-04-25T09:57:12.979567+0000","last_peered":"2024-04-25T09:57:12.979567+0000","last_clean":"2024-04-25T09:57:12.979567+0000","last_became_active":"2024-04-25T09:57:11.007273+0000","last_became_peered":"2024-04-25T09:57:11.007273+0000","last_unstale":"2024-04-25T09:57:12.979567+0000","last_undegraded":"2024-04-25T09:57:12.979567+0000","last_fullsized":"2024-04-25T09:57:12.979567+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:28.042407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384311+0000","last_change":"2024-04-25T09:57:09.983965+0000","last_active":"2024-04-25T09:57:13.384311+0000","last_peered":"2024-04-25T09:57:13.384311+0000","last_clean":"2024-04-25T09:57:13.384311+0000","last_became_active":"2024-04-25T09:57:09.983606+0000","last_became_peered":"2024-04-25T09:57:09.983606+0000","last_unstale":"2024-04-25T09:57:13.384311+0000","last_undegraded":"2024-04-25T09:57:13.384311+0000","last_fullsized":"2024-04-25T09:57:13.384311+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:46.078334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500866+0000","last_change":"2024-04-25T09:57:11.003033+0000","last_active":"2024-04-25T09:57:12.500866+0000","last_peered":"2024-04-25T09:57:12.500866+0000","last_clean":"2024-04-25T09:57:12.500866+0000","last_became_active":"2024-04-25T09:57:10.998361+0000","last_became_peered":"2024-04-25T09:57:10.998361+0000","last_unstale":"2024-04-25T09:57:12.500866+0000","last_undegraded":"2024-04-25T09:57:12.500866+0000","last_fullsized":"2024-04-25T09:57:12.500866+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:00.083828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993293+0000","last_change":"2024-04-25T09:57:09.986995+0000","last_active":"2024-04-25T09:57:11.993293+0000","last_peered":"2024-04-25T09:57:11.993293+0000","last_clean":"2024-04-25T09:57:11.993293+0000","last_became_active":"2024-04-25T09:57:09.986584+0000","last_became_peered":"2024-04-25T09:57:09.986584+0000","last_unstale":"2024-04-25T09:57:11.993293+0000","last_undegraded":"2024-04-25T09:57:11.993293+0000","last_fullsized":"2024-04-25T09:57:11.993293+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:26.975512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979628+0000","last_change":"2024-04-25T09:57:11.013060+0000","last_active":"2024-04-25T09:57:12.979628+0000","last_peered":"2024-04-25T09:57:12.979628+0000","last_clean":"2024-04-25T09:57:12.979628+0000","last_became_active":"2024-04-25T09:57:11.012558+0000","last_became_peered":"2024-04-25T09:57:11.012558+0000","last_unstale":"2024-04-25T09:57:12.979628+0000","last_undegraded":"2024-04-25T09:57:12.979628+0000","last_fullsized":"2024-04-25T09:57:12.979628+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:14.168640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":22,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500689+0000","last_change":"2024-04-25T09:57:09.995540+0000","last_active":"2024-04-25T09:57:12.500689+0000","last_peered":"2024-04-25T09:57:12.500689+0000","last_clean":"2024-04-25T09:57:12.500689+0000","last_became_active":"2024-04-25T09:57:09.995243+0000","last_became_peered":"2024-04-25T09:57:09.995243+0000","last_unstale":"2024-04-25T09:57:12.500689+0000","last_undegraded":"2024-04-25T09:57:12.500689+0000","last_fullsized":"2024-04-25T09:57:12.500689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:29.883740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500700+0000","last_change":"2024-04-25T09:57:10.998500+0000","last_active":"2024-04-25T09:57:12.500700+0000","last_peered":"2024-04-25T09:57:12.500700+0000","last_clean":"2024-04-25T09:57:12.500700+0000","last_became_active":"2024-04-25T09:57:10.997919+0000","last_became_peered":"2024-04-25T09:57:10.997919+0000","last_unstale":"2024-04-25T09:57:12.500700+0000","last_undegraded":"2024-04-25T09:57:12.500700+0000","last_fullsized":"2024-04-25T09:57:12.500700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:42.776889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979806+0000","last_change":"2024-04-25T09:57:09.995365+0000","last_active":"2024-04-25T09:57:12.979806+0000","last_peered":"2024-04-25T09:57:12.979806+0000","last_clean":"2024-04-25T09:57:12.979806+0000","last_became_active":"2024-04-25T09:57:09.995083+0000","last_became_peered":"2024-04-25T09:57:09.995083+0000","last_unstale":"2024-04-25T09:57:12.979806+0000","last_undegraded":"2024-04-25T09:57:12.979806+0000","last_fullsized":"2024-04-25T09:57:12.979806+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:05.419943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.357910+0000","last_change":"2024-04-25T09:57:11.003892+0000","last_active":"2024-04-25T09:57:12.357910+0000","last_peered":"2024-04-25T09:57:12.357910+0000","last_clean":"2024-04-25T09:57:12.357910+0000","last_became_active":"2024-04-25T09:57:11.002613+0000","last_became_peered":"2024-04-25T09:57:11.002613+0000","last_unstale":"2024-04-25T09:57:12.357910+0000","last_undegraded":"2024-04-25T09:57:12.357910+0000","last_fullsized":"2024-04-25T09:57:12.357910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:14.665259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.756527+0000","last_change":"2024-04-25T09:57:09.984256+0000","last_active":"2024-04-25T09:57:12.756527+0000","last_peered":"2024-04-25T09:57:12.756527+0000","last_clean":"2024-04-25T09:57:12.756527+0000","last_became_active":"2024-04-25T09:57:09.983972+0000","last_became_peered":"2024-04-25T09:57:09.983972+0000","last_unstale":"2024-04-25T09:57:12.756527+0000","last_undegraded":"2024-04-25T09:57:12.756527+0000","last_fullsized":"2024-04-25T09:57:12.756527+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:56.755842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.995104+0000","last_change":"2024-04-25T09:57:10.997756+0000","last_active":"2024-04-25T09:57:11.995104+0000","last_peered":"2024-04-25T09:57:11.995104+0000","last_clean":"2024-04-25T09:57:11.995104+0000","last_became_active":"2024-04-25T09:57:10.997305+0000","last_became_peered":"2024-04-25T09:57:10.997305+0000","last_unstale":"2024-04-25T09:57:11.995104+0000","last_undegraded":"2024-04-25T09:57:11.995104+0000","last_fullsized":"2024-04-25T09:57:11.995104+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:55.408299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384595+0000","last_change":"2024-04-25T09:57:09.991436+0000","last_active":"2024-04-25T09:57:13.384595+0000","last_peered":"2024-04-25T09:57:13.384595+0000","last_clean":"2024-04-25T09:57:13.384595+0000","last_became_active":"2024-04-25T09:57:09.991138+0000","last_became_peered":"2024-04-25T09:57:09.991138+0000","last_unstale":"2024-04-25T09:57:13.384595+0000","last_undegraded":"2024-04-25T09:57:13.384595+0000","last_fullsized":"2024-04-25T09:57:13.384595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:33.873812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358022+0000","last_change":"2024-04-25T09:57:10.994704+0000","last_active":"2024-04-25T09:57:12.358022+0000","last_peered":"2024-04-25T09:57:12.358022+0000","last_clean":"2024-04-25T09:57:12.358022+0000","last_became_active":"2024-04-25T09:57:10.994444+0000","last_became_peered":"2024-04-25T09:57:10.994444+0000","last_unstale":"2024-04-25T09:57:12.358022+0000","last_undegraded":"2024-04-25T09:57:12.358022+0000","last_fullsized":"2024-04-25T09:57:12.358022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:02.018001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384634+0000","last_change":"2024-04-25T09:57:09.988005+0000","last_active":"2024-04-25T09:57:13.384634+0000","last_peered":"2024-04-25T09:57:13.384634+0000","last_clean":"2024-04-25T09:57:13.384634+0000","last_became_active":"2024-04-25T09:57:09.987706+0000","last_became_peered":"2024-04-25T09:57:09.987706+0000","last_unstale":"2024-04-25T09:57:13.384634+0000","last_undegraded":"2024-04-25T09:57:13.384634+0000","last_fullsized":"2024-04-25T09:57:13.384634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:44.117451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390209+0000","last_change":"2024-04-25T09:57:10.982447+0000","last_active":"2024-04-25T09:57:13.390209+0000","last_peered":"2024-04-25T09:57:13.390209+0000","last_clean":"2024-04-25T09:57:13.390209+0000","last_became_active":"2024-04-25T09:57:10.982155+0000","last_became_peered":"2024-04-25T09:57:10.982155+0000","last_unstale":"2024-04-25T09:57:13.390209+0000","last_undegraded":"2024-04-25T09:57:13.390209+0000","last_fullsized":"2024-04-25T09:57:13.390209+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:10.345044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358091+0000","last_change":"2024-04-25T09:57:09.990696+0000","last_active":"2024-04-25T09:57:12.358091+0000","last_peered":"2024-04-25T09:57:12.358091+0000","last_clean":"2024-04-25T09:57:12.358091+0000","last_became_active":"2024-04-25T09:57:09.990433+0000","last_became_peered":"2024-04-25T09:57:09.990433+0000","last_unstale":"2024-04-25T09:57:12.358091+0000","last_undegraded":"2024-04-25T09:57:12.358091+0000","last_fullsized":"2024-04-25T09:57:12.358091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:12.966966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384474+0000","last_change":"2024-04-25T09:57:10.991309+0000","last_active":"2024-04-25T09:57:13.384474+0000","last_peered":"2024-04-25T09:57:13.384474+0000","last_clean":"2024-04-25T09:57:13.384474+0000","last_became_active":"2024-04-25T09:57:10.991025+0000","last_became_peered":"2024-04-25T09:57:10.991025+0000","last_unstale":"2024-04-25T09:57:13.384474+0000","last_undegraded":"2024-04-25T09:57:13.384474+0000","last_fullsized":"2024-04-25T09:57:13.384474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:14.665259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500543+0000","last_change":"2024-04-25T09:57:09.995784+0000","last_active":"2024-04-25T09:57:12.500543+0000","last_peered":"2024-04-25T09:57:12.500543+0000","last_clean":"2024-04-25T09:57:12.500543+0000","last_became_active":"2024-04-25T09:57:09.995393+0000","last_became_peered":"2024-04-25T09:57:09.995393+0000","last_unstale":"2024-04-25T09:57:12.500543+0000","last_undegraded":"2024-04-25T09:57:12.500543+0000","last_fullsized":"2024-04-25T09:57:12.500543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:07.028119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979410+0000","last_change":"2024-04-25T09:57:11.010849+0000","last_active":"2024-04-25T09:57:12.979410+0000","last_peered":"2024-04-25T09:57:12.979410+0000","last_clean":"2024-04-25T09:57:12.979410+0000","last_became_active":"2024-04-25T09:57:11.009479+0000","last_became_peered":"2024-04-25T09:57:11.009479+0000","last_unstale":"2024-04-25T09:57:12.979410+0000","last_undegraded":"2024-04-25T09:57:12.979410+0000","last_fullsized":"2024-04-25T09:57:12.979410+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:10.345044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979194+0000","last_change":"2024-04-25T09:57:09.993056+0000","last_active":"2024-04-25T09:57:12.979194+0000","last_peered":"2024-04-25T09:57:12.979194+0000","last_clean":"2024-04-25T09:57:12.979194+0000","last_became_active":"2024-04-25T09:57:09.992768+0000","last_became_peered":"2024-04-25T09:57:09.992768+0000","last_unstale":"2024-04-25T09:57:12.979194+0000","last_undegraded":"2024-04-25T09:57:12.979194+0000","last_fullsized":"2024-04-25T09:57:12.979194+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:46.383736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358202+0000","last_change":"2024-04-25T09:57:11.003476+0000","last_active":"2024-04-25T09:57:12.358202+0000","last_peered":"2024-04-25T09:57:12.358202+0000","last_clean":"2024-04-25T09:57:12.358202+0000","last_became_active":"2024-04-25T09:57:11.002410+0000","last_became_peered":"2024-04-25T09:57:11.002410+0000","last_unstale":"2024-04-25T09:57:12.358202+0000","last_undegraded":"2024-04-25T09:57:12.358202+0000","last_fullsized":"2024-04-25T09:57:12.358202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:04.778416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.756266+0000","last_change":"2024-04-25T09:57:09.987312+0000","last_active":"2024-04-25T09:57:12.756266+0000","last_peered":"2024-04-25T09:57:12.756266+0000","last_clean":"2024-04-25T09:57:12.756266+0000","last_became_active":"2024-04-25T09:57:09.987064+0000","last_became_peered":"2024-04-25T09:57:09.987064+0000","last_unstale":"2024-04-25T09:57:12.756266+0000","last_undegraded":"2024-04-25T09:57:12.756266+0000","last_fullsized":"2024-04-25T09:57:12.756266+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:37.295748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390112+0000","last_change":"2024-04-25T09:57:10.990572+0000","last_active":"2024-04-25T09:57:13.390112+0000","last_peered":"2024-04-25T09:57:13.390112+0000","last_clean":"2024-04-25T09:57:13.390112+0000","last_became_active":"2024-04-25T09:57:10.990112+0000","last_became_peered":"2024-04-25T09:57:10.990112+0000","last_unstale":"2024-04-25T09:57:13.390112+0000","last_undegraded":"2024-04-25T09:57:13.390112+0000","last_fullsized":"2024-04-25T09:57:13.390112+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:32.921691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384659+0000","last_change":"2024-04-25T09:57:09.985025+0000","last_active":"2024-04-25T09:57:13.384659+0000","last_peered":"2024-04-25T09:57:13.384659+0000","last_clean":"2024-04-25T09:57:13.384659+0000","last_became_active":"2024-04-25T09:57:09.984357+0000","last_became_peered":"2024-04-25T09:57:09.984357+0000","last_unstale":"2024-04-25T09:57:13.384659+0000","last_undegraded":"2024-04-25T09:57:13.384659+0000","last_fullsized":"2024-04-25T09:57:13.384659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:12.452815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.358014+0000","last_change":"2024-04-25T09:57:11.000296+0000","last_active":"2024-04-25T09:57:12.358014+0000","last_peered":"2024-04-25T09:57:12.358014+0000","last_clean":"2024-04-25T09:57:12.358014+0000","last_became_active":"2024-04-25T09:57:10.999972+0000","last_became_peered":"2024-04-25T09:57:10.999972+0000","last_unstale":"2024-04-25T09:57:12.358014+0000","last_undegraded":"2024-04-25T09:57:12.358014+0000","last_fullsized":"2024-04-25T09:57:12.358014+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:26.451380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994021+0000","last_change":"2024-04-25T09:57:09.991923+0000","last_active":"2024-04-25T09:57:11.994021+0000","last_peered":"2024-04-25T09:57:11.994021+0000","last_clean":"2024-04-25T09:57:11.994021+0000","last_became_active":"2024-04-25T09:57:09.991642+0000","last_became_peered":"2024-04-25T09:57:09.991642+0000","last_unstale":"2024-04-25T09:57:11.994021+0000","last_undegraded":"2024-04-25T09:57:11.994021+0000","last_fullsized":"2024-04-25T09:57:11.994021+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:54.401746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500701+0000","last_change":"2024-04-25T09:57:10.997738+0000","last_active":"2024-04-25T09:57:12.500701+0000","last_peered":"2024-04-25T09:57:12.500701+0000","last_clean":"2024-04-25T09:57:12.500701+0000","last_became_active":"2024-04-25T09:57:10.997440+0000","last_became_peered":"2024-04-25T09:57:10.997440+0000","last_unstale":"2024-04-25T09:57:12.500701+0000","last_undegraded":"2024-04-25T09:57:12.500701+0000","last_fullsized":"2024-04-25T09:57:12.500701+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:06.317062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979385+0000","last_change":"2024-04-25T09:57:09.982146+0000","last_active":"2024-04-25T09:57:12.979385+0000","last_peered":"2024-04-25T09:57:12.979385+0000","last_clean":"2024-04-25T09:57:12.979385+0000","last_became_active":"2024-04-25T09:57:09.981876+0000","last_became_peered":"2024-04-25T09:57:09.981876+0000","last_unstale":"2024-04-25T09:57:12.979385+0000","last_undegraded":"2024-04-25T09:57:12.979385+0000","last_fullsized":"2024-04-25T09:57:12.979385+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.029172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979401+0000","last_change":"2024-04-25T09:57:11.008187+0000","last_active":"2024-04-25T09:57:12.979401+0000","last_peered":"2024-04-25T09:57:12.979401+0000","last_clean":"2024-04-25T09:57:12.979401+0000","last_became_active":"2024-04-25T09:57:11.007896+0000","last_became_peered":"2024-04-25T09:57:11.007896+0000","last_unstale":"2024-04-25T09:57:12.979401+0000","last_undegraded":"2024-04-25T09:57:12.979401+0000","last_fullsized":"2024-04-25T09:57:12.979401+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:42.117546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500761+0000","last_change":"2024-04-25T09:57:09.995462+0000","last_active":"2024-04-25T09:57:12.500761+0000","last_peered":"2024-04-25T09:57:12.500761+0000","last_clean":"2024-04-25T09:57:12.500761+0000","last_became_active":"2024-04-25T09:57:09.995179+0000","last_became_peered":"2024-04-25T09:57:09.995179+0000","last_unstale":"2024-04-25T09:57:12.500761+0000","last_undegraded":"2024-04-25T09:57:12.500761+0000","last_fullsized":"2024-04-25T09:57:12.500761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:17.326547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390309+0000","last_change":"2024-04-25T09:57:10.991545+0000","last_active":"2024-04-25T09:57:13.390309+0000","last_peered":"2024-04-25T09:57:13.390309+0000","last_clean":"2024-04-25T09:57:13.390309+0000","last_became_active":"2024-04-25T09:57:10.987714+0000","last_became_peered":"2024-04-25T09:57:10.987714+0000","last_unstale":"2024-04-25T09:57:13.390309+0000","last_undegraded":"2024-04-25T09:57:13.390309+0000","last_fullsized":"2024-04-25T09:57:13.390309+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:17.709060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.500641+0000","last_change":"2024-04-25T09:57:09.994224+0000","last_active":"2024-04-25T09:57:12.500641+0000","last_peered":"2024-04-25T09:57:12.500641+0000","last_clean":"2024-04-25T09:57:12.500641+0000","last_became_active":"2024-04-25T09:57:09.993902+0000","last_became_peered":"2024-04-25T09:57:09.993902+0000","last_unstale":"2024-04-25T09:57:12.500641+0000","last_undegraded":"2024-04-25T09:57:12.500641+0000","last_fullsized":"2024-04-25T09:57:12.500641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:51.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979443+0000","last_change":"2024-04-25T09:57:11.010053+0000","last_active":"2024-04-25T09:57:12.979443+0000","last_peered":"2024-04-25T09:57:12.979443+0000","last_clean":"2024-04-25T09:57:12.979443+0000","last_became_active":"2024-04-25T09:57:11.009221+0000","last_became_peered":"2024-04-25T09:57:11.009221+0000","last_unstale":"2024-04-25T09:57:12.979443+0000","last_undegraded":"2024-04-25T09:57:12.979443+0000","last_fullsized":"2024-04-25T09:57:12.979443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:56.937329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.994963+0000","last_change":"2024-04-25T09:57:09.988098+0000","last_active":"2024-04-25T09:57:11.994963+0000","last_peered":"2024-04-25T09:57:11.994963+0000","last_clean":"2024-04-25T09:57:11.994963+0000","last_became_active":"2024-04-25T09:57:09.987672+0000","last_became_peered":"2024-04-25T09:57:09.987672+0000","last_unstale":"2024-04-25T09:57:11.994963+0000","last_undegraded":"2024-04-25T09:57:11.994963+0000","last_fullsized":"2024-04-25T09:57:11.994963+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:27.807847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.501073+0000","last_change":"2024-04-25T09:57:11.003026+0000","last_active":"2024-04-25T09:57:12.501073+0000","last_peered":"2024-04-25T09:57:12.501073+0000","last_clean":"2024-04-25T09:57:12.501073+0000","last_became_active":"2024-04-25T09:57:11.002677+0000","last_became_peered":"2024-04-25T09:57:11.002677+0000","last_unstale":"2024-04-25T09:57:12.501073+0000","last_undegraded":"2024-04-25T09:57:12.501073+0000","last_fullsized":"2024-04-25T09:57:12.501073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:01.813300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.390222+0000","last_change":"2024-04-25T09:57:09.985927+0000","last_active":"2024-04-25T09:57:13.390222+0000","last_peered":"2024-04-25T09:57:13.390222+0000","last_clean":"2024-04-25T09:57:13.390222+0000","last_became_active":"2024-04-25T09:57:09.985657+0000","last_became_peered":"2024-04-25T09:57:09.985657+0000","last_unstale":"2024-04-25T09:57:13.390222+0000","last_undegraded":"2024-04-25T09:57:13.390222+0000","last_fullsized":"2024-04-25T09:57:13.390222+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:32.706290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:13.384231+0000","last_change":"2024-04-25T09:57:10.992821+0000","last_active":"2024-04-25T09:57:13.384231+0000","last_peered":"2024-04-25T09:57:13.384231+0000","last_clean":"2024-04-25T09:57:13.384231+0000","last_became_active":"2024-04-25T09:57:10.992495+0000","last_became_peered":"2024-04-25T09:57:10.992495+0000","last_unstale":"2024-04-25T09:57:13.384231+0000","last_undegraded":"2024-04-25T09:57:13.384231+0000","last_fullsized":"2024-04-25T09:57:13.384231+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:09.966917+0000","last_clean_scrub_stamp":"2024-04-25T09:57:09.966917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:28.582303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:12.979573+0000","last_change":"2024-04-25T09:57:09.990770+0000","last_active":"2024-04-25T09:57:12.979573+0000","last_peered":"2024-04-25T09:57:12.979573+0000","last_clean":"2024-04-25T09:57:12.979573+0000","last_became_active":"2024-04-25T09:57:09.990495+0000","last_became_peered":"2024-04-25T09:57:09.990495+0000","last_unstale":"2024-04-25T09:57:12.979573+0000","last_undegraded":"2024-04-25T09:57:12.979573+0000","last_fullsized":"2024-04-25T09:57:12.979573+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:57:08.960364+0000","last_clean_scrub_stamp":"2024-04-25T09:57:08.960364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:09.358541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":236,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T09:57:11.993205+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:11.993205+0000","last_peered":"2024-04-25T09:57:11.993205+0000","last_clean":"2024-04-25T09:57:11.993205+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:11.993205+0000","last_undegraded":"2024-04-25T09:57:11.993205+0000","last_fullsized":"2024-04-25T09:57:11.993205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":886879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35988,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335852,"statfs":{"total":96636764160,"available":96599912448,"internally_reserved":0,"allocated":663552,"data_stored":297039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":655360,"data_stored":296387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35992,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335848,"statfs":{"total":96636764160,"available":96599908352,"internally_reserved":0,"allocated":659456,"data_stored":296969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35996,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335844,"statfs":{"total":96636764160,"available":96599904256,"internally_reserved":0,"allocated":671744,"data_stored":298021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35996,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335844,"statfs":{"total":96636764160,"available":96599904256,"internally_reserved":0,"allocated":667648,"data_stored":297999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35992,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335848,"statfs":{"total":96636764160,"available":96599908352,"internally_reserved":0,"allocated":663552,"data_stored":296547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36568,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335272,"statfs":{"total":96636764160,"available":96599318528,"internally_reserved":0,"allocated":1257472,"data_stored":886811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:57:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:57:18.614 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:57:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 Processor -- start 2024-04-25T09:57:18.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 -- start start 2024-04-25T09:57:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc1b415dbd0 0x7fc1b40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc1b406bbf0 0x7fc1b406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40cb0e0 con 0x7fc1b406ff30 2024-04-25T09:57:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b406c5a0 con 0x7fc1b415dbd0 2024-04-25T09:57:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.752+0000 7fc1ba9b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b40cb3c0 con 0x7fc1b406bbf0 2024-04-25T09:57:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc1b415dbd0 0x7fc1b40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc1b415dbd0 0x7fc1b40caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43838/0 (socket says 172.21.15.148:43838) 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 -- 172.21.15.148:0/2266316880 learned_addr learned my addr 172.21.15.148:0/2266316880 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b8f4d640 1 --2- 172.21.15.148:0/2266316880 >> v2:172.21.15.188:3300/0 conn(0x7fc1b406bbf0 0x7fc1b406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 -- 172.21.15.148:0/2266316880 >> v2:172.21.15.188:3300/0 conn(0x7fc1b406bbf0 msgr2=0x7fc1b406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 --2- 172.21.15.148:0/2266316880 >> v2:172.21.15.188:3300/0 conn(0x7fc1b406bbf0 0x7fc1b406c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 -- 172.21.15.148:0/2266316880 >> v1:172.21.15.188:6789/0 conn(0x7fc1b406ff30 legacy=0x7fc1b40cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.753+0000 7fc1b3fff640 1 -- 172.21.15.148:0/2266316880 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40cb6a0 con 0x7fc1b415dbd0 2024-04-25T09:57:18.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1b3fff640 1 --2- 172.21.15.148:0/2266316880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b40caa60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4071a80 tx=0x7fc1a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511cee93c0740c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1b2ffd640 1 -- 172.21.15.148:0/2266316880 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4037070 con 0x7fc1b415dbd0 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1b2ffd640 1 -- 172.21.15.148:0/2266316880 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4032440 con 0x7fc1b415dbd0 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1b2ffd640 1 -- 172.21.15.148:0/2266316880 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4032810 con 0x7fc1b415dbd0 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2266316880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 msgr2=0x7fc1b40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2266316880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b40caa60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4071a80 tx=0x7fc1a402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2266316880 shutdown_connections 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2266316880 >> v2:172.21.15.188:3300/0 conn(0x7fc1b406bbf0 0x7fc1b406c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2266316880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b40caa60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2266316880 >> 172.21.15.148:0/2266316880 conn(0x7fc1b4071af0 msgr2=0x7fc1b4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2266316880 shutdown_connections 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.754+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2266316880 wait complete. 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.755+0000 7fc1ba9b0640 1 Processor -- start 2024-04-25T09:57:18.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 -- start start 2024-04-25T09:57:18.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b406bbf0 0x7fc1b40b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b406ff30 0x7fc1b4160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b415dfb0 con 0x7fc1b415dbd0 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40cbc60 con 0x7fc1b406bbf0 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1ba9b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b415e130 con 0x7fc1b406ff30 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1b8f4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1b8f4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b414b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43850/0 (socket says 172.21.15.148:43850) 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.756+0000 7fc1b8f4d640 1 -- 172.21.15.148:0/2561158040 learned_addr learned my addr 172.21.15.148:0/2561158040 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b3fff640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b406bbf0 0x7fc1b40b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b37fe640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b406ff30 0x7fc1b4160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b8f4d640 1 -- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b406ff30 msgr2=0x7fc1b4160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b8f4d640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b406ff30 0x7fc1b4160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b8f4d640 1 -- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b406bbf0 msgr2=0x7fc1b40b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b8f4d640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b406bbf0 0x7fc1b40b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b8f4d640 1 -- 172.21.15.148:0/2561158040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40b3290 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b8f4d640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b414b7f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00efc0 tx=0x7fc1ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac018020 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac014480 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b40b3570 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac014870 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.757+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b40b3b30 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.758+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b4073a00 con 0x7fc1b415dbd0 2024-04-25T09:57:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.758+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1ac005d10 con 0x7fc1b415dbd0 2024-04-25T09:57:18.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.759+0000 7fc1b0ff9640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc1a006b4d0 0x7fc1a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:18.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.759+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc1ac08b210 con 0x7fc1b415dbd0 2024-04-25T09:57:18.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.759+0000 7fc1b3fff640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc1a006b4d0 0x7fc1a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:18.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.759+0000 7fc1b3fff640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc1a006b4d0 0x7fc1a006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4034000 tx=0x7fc1a402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:18.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:18.763+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1ac022030 con 0x7fc1b415dbd0 2024-04-25T09:57:19.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.000+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc1b40af4f0 con 0x7fc1b415dbd0 2024-04-25T09:57:19.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.002+0000 7fc1b0ff9640 1 -- 172.21.15.148:0/2561158040 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc1ac010040 con 0x7fc1b415dbd0 2024-04-25T09:57:19.005 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T09:57:19.005 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T09:57:19.006 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T09:57:19.006 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T09:57:19.006 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:19.006 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc1a006b4d0 msgr2=0x7fc1a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc1a006b4d0 0x7fc1a006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4034000 tx=0x7fc1a402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 msgr2=0x7fc1b414b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b414b7f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00efc0 tx=0x7fc1ac00c490 comp rx=0 tx=0).stop 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 shutdown_connections 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc1a006b4d0 0x7fc1a006d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1b415dbd0 0x7fc1b414b7f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1b406ff30 0x7fc1b4160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 --2- 172.21.15.148:0/2561158040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1b406bbf0 0x7fc1b40b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 >> 172.21.15.148:0/2561158040 conn(0x7fc1b4071af0 msgr2=0x7fc1b406db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 shutdown_connections 2024-04-25T09:57:19.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.004+0000 7fc1ba9b0640 1 -- 172.21.15.148:0/2561158040 wait complete. 2024-04-25T09:57:19.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:19.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.114+0000 7fbd2a91d640 1 Processor -- start 2024-04-25T09:57:19.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.114+0000 7fbd2a91d640 1 -- start start 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd2a91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 0x7fbd240d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd2a91d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240b97a0 con 0x7fbd240b2cb0 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd2a91d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240d1c50 con 0x7fbd2415dbe0 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd2a91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2415dfc0 con 0x7fbd240d1020 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd237fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbd240b2cb0 0x7fbd240d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39004/0 (socket says 172.21.15.148:39004) 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd237fe640 1 -- 172.21.15.148:0/9849402 learned_addr learned my addr 172.21.15.148:0/9849402 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd28eba640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 0x7fbd240d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974937477 0 0) 0x7fbd240b97a0 con 0x7fbd240b2cb0 2024-04-25T09:57:19.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd0c003520 con 0x7fbd240b2cb0 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3536999834 0 0) 0x7fbd0c003520 con 0x7fbd240b2cb0 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 msgr2=0x7fbd240d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.115+0000 7fbd22ffd640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 0x7fbd240d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 >> v1:172.21.15.188:6789/0 conn(0x7fbd2415dbe0 legacy=0x7fbd240d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240d1f30 con 0x7fbd240b2cb0 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbd18001670 con 0x7fbd240b2cb0 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 >> v1:172.21.15.148:6789/0 conn(0x7fbd240b2cb0 legacy=0x7fbd240d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 0x7fbd0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd0c004d40 0x7fbd0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00baa0 con 0x7fbd0c0039e0 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00bc50 con 0x7fbd0c004d40 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c00be50 con 0x7fbd240d1020 2024-04-25T09:57:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd23fff640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd237fe640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 0x7fbd0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd28eba640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd0c004d40 0x7fbd0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd23fff640 1 -- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 msgr2=0x7fbd0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd23fff640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd240d1020 0x7fbd0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd23fff640 1 -- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd0c004d40 msgr2=0x7fbd0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd23fff640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd0c004d40 0x7fbd0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd23fff640 1 -- 172.21.15.148:0/9849402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c003520 con 0x7fbd0c0039e0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.116+0000 7fbd28eba640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd0c004d40 0x7fbd0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd23fff640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd0c003e00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbd14001720 tx=0x7fbd0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdca76489710ae09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14001370 con 0x7fbd0c0039e0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd140052d0 con 0x7fbd0c0039e0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd22ffd640 1 -- 172.21.15.148:0/9849402 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1402abc0 con 0x7fbd0c0039e0 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd2a91d640 1 -- 172.21.15.148:0/9849402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 msgr2=0x7fbd0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/9849402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd0c003e00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbd14001720 tx=0x7fbd0c005830 comp rx=0 tx=0).stop 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd28eba640 1 -- 172.21.15.148:0/9849402 reap_dead start 2024-04-25T09:57:19.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd2a91d640 1 -- 172.21.15.148:0/9849402 shutdown_connections 2024-04-25T09:57:19.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd2a91d640 1 -- 172.21.15.148:0/9849402 >> 172.21.15.148:0/9849402 conn(0x7fbd24061a20 msgr2=0x7fbd24061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd2a91d640 1 -- 172.21.15.148:0/9849402 shutdown_connections 2024-04-25T09:57:19.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.117+0000 7fbd2a91d640 1 -- 172.21.15.148:0/9849402 wait complete. 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.119+0000 7fbd2a91d640 1 Processor -- start 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.119+0000 7fbd2a91d640 1 -- start start 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.119+0000 7fbd2a91d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd241f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.119+0000 7fbd2a91d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24153d70 0x7fbd241f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd2a91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd241f7080 0x7fbd241527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd2a91d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240b92a0 con 0x7fbd0c0039e0 2024-04-25T09:57:19.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd2a91d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240d1a00 con 0x7fbd24153d70 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd2a91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240b9420 con 0x7fbd241f7080 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd241f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd241f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43864/0 (socket says 172.21.15.148:43864) 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 -- 172.21.15.148:0/129767633 learned_addr learned my addr 172.21.15.148:0/129767633 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 -- 172.21.15.148:0/129767633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd241f7080 msgr2=0x7fbd241527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd241f7080 0x7fbd241527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 -- 172.21.15.148:0/129767633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24153d70 msgr2=0x7fbd241f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24153d70 0x7fbd241f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 -- 172.21.15.148:0/129767633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd241f3550 con 0x7fbd0c0039e0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd23fff640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd241f3fd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbd14008fd0 tx=0x7fbd14001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14042070 con 0x7fbd0c0039e0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1402be50 con 0x7fbd0c0039e0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd241f4510 con 0x7fbd0c0039e0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14037040 con 0x7fbd0c0039e0 2024-04-25T09:57:19.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.120+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd241f78c0 con 0x7fbd0c0039e0 2024-04-25T09:57:19.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.121+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd241f7a90 con 0x7fbd0c0039e0 2024-04-25T09:57:19.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.122+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd14045070 con 0x7fbd0c0039e0 2024-04-25T09:57:19.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.122+0000 7fbd20ff9640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd1006b4d0 0x7fbd1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.122+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbd140a9b50 con 0x7fbd0c0039e0 2024-04-25T09:57:19.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.122+0000 7fbd237fe640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd1006b4d0 0x7fbd1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.122+0000 7fbd237fe640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd1006b4d0 0x7fbd1006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbd18001ab0 tx=0x7fbd18001b40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.126+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd241f7a90 con 0x7fbd0c0039e0 2024-04-25T09:57:19.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.331+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd24158c50 con 0x7fbd0c0039e0 2024-04-25T09:57:19.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.332+0000 7fbd20ff9640 1 -- 172.21.15.148:0/129767633 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fbd1407e020 con 0x7fbd0c0039e0 2024-04-25T09:57:19.336 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:19.336 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:19.336 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:19.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd1006b4d0 msgr2=0x7fbd1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd1006b4d0 0x7fbd1006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbd18001ab0 tx=0x7fbd18001b40 comp rx=0 tx=0).stop 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 msgr2=0x7fbd241f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd241f3fd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbd14008fd0 tx=0x7fbd14001120 comp rx=0 tx=0).stop 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 shutdown_connections 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd1006b4d0 0x7fbd1006d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd241f7080 0x7fbd241527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24153d70 0x7fbd241f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 --2- 172.21.15.148:0/129767633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd0c0039e0 0x7fbd241f3fd0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 >> 172.21.15.148:0/129767633 conn(0x7fbd24061a20 msgr2=0x7fbd2406c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 shutdown_connections 2024-04-25T09:57:19.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.334+0000 7fbd2a91d640 1 -- 172.21.15.148:0/129767633 wait complete. 2024-04-25T09:57:19.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:57:10.325425+0000', 'modified': '2024-04-25T09:57:12.322346+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5862}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5862': {'gid': 5862, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/3309890912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 3309890912}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3309890912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:57:19.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:57:19.751 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:57:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 Processor -- start 2024-04-25T09:57:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 -- start start 2024-04-25T09:57:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a3815dac0 0x7f4a380d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a380d1f30 0x7f4a380d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380d2420 con 0x7f4a380d34c0 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380d3030 con 0x7f4a3815dac0 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.854+0000 7f4a3d01b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380d2700 con 0x7f4a380d1f30 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a3815dac0 0x7f4a380d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4a3815dac0 0x7f4a380d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43872/0 (socket says 172.21.15.148:43872) 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 -- 172.21.15.148:0/1268792314 learned_addr learned my addr 172.21.15.148:0/1268792314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 -- 172.21.15.148:0/1268792314 >> v2:172.21.15.188:3300/0 conn(0x7f4a380d1f30 msgr2=0x7f4a380d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/1268792314 >> v2:172.21.15.188:3300/0 conn(0x7f4a380d1f30 0x7f4a380d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1268792314 >> v2:172.21.15.188:3300/0 conn(0x7f4a380d1f30 0x7f4a380d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 -- 172.21.15.148:0/1268792314 >> v1:172.21.15.188:6789/0 conn(0x7f4a380d34c0 legacy=0x7f4a380b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.855+0000 7f4a36d9d640 1 -- 172.21.15.148:0/1268792314 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380d29e0 con 0x7f4a3815dac0 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1268792314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815dac0 0x7f4a380d19f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4a38161590 tx=0x7f4a2802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c24521e792cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3659c640 1 -- 172.21.15.148:0/1268792314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28037070 con 0x7f4a3815dac0 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3659c640 1 -- 172.21.15.148:0/1268792314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28032440 con 0x7f4a3815dac0 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3659c640 1 -- 172.21.15.148:0/1268792314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28032810 con 0x7f4a3815dac0 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1268792314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815dac0 msgr2=0x7f4a380d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1268792314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815dac0 0x7f4a380d19f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4a38161590 tx=0x7f4a2802a2c0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1268792314 shutdown_connections 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1268792314 >> v2:172.21.15.188:3300/0 conn(0x7f4a380d1f30 0x7f4a380d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1268792314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3815dac0 0x7f4a380d19f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1268792314 >> 172.21.15.148:0/1268792314 conn(0x7f4a38061c30 msgr2=0x7f4a3806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1268792314 shutdown_connections 2024-04-25T09:57:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.856+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1268792314 wait complete. 2024-04-25T09:57:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 Processor -- start 2024-04-25T09:57:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 -- start start 2024-04-25T09:57:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a380d1f30 0x7f4a380b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 0x7f4a380cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a3815dac0 0x7f4a38151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b97d0 con 0x7f4a380d34c0 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815dea0 con 0x7f4a380d1f30 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.858+0000 7f4a3d01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b9950 con 0x7f4a3815dac0 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 0x7f4a380cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 0x7f4a380cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43876/0 (socket says 172.21.15.148:43876) 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 -- 172.21.15.148:0/1422704800 learned_addr learned my addr 172.21.15.148:0/1422704800 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a3759e640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a3815dac0 0x7f4a38151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a380d1f30 0x7f4a380b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 -- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a3815dac0 msgr2=0x7f4a38151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a3815dac0 0x7f4a38151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 -- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a380d1f30 msgr2=0x7f4a380b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a380d1f30 0x7f4a380b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 -- 172.21.15.148:0/1422704800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a38160740 con 0x7f4a380d34c0 2024-04-25T09:57:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 0x7f4a380cbc90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c7f0 tx=0x7f4a2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2f7fe640 1 -- 172.21.15.148:0/1422704800 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20015070 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2f7fe640 1 -- 172.21.15.148:0/1422704800 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a20007820 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a2f7fe640 1 -- 172.21.15.148:0/1422704800 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20007c10 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a38160a20 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.859+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3815e3a0 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.861+0000 7f4a2f7fe640 1 -- 172.21.15.148:0/1422704800 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a20018070 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.861+0000 7f4a2f7fe640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a1806b4d0 0x7f4a1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.861+0000 7f4a2f7fe640 1 -- 172.21.15.148:0/1422704800 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a2008a440 con 0x7f4a380d34c0 2024-04-25T09:57:19.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.861+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a1806b4d0 0x7f4a1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.862+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a1806b4d0 0x7f4a1806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034000 tx=0x7f4a2802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.862+0000 7f4a3fdf1740 1 Processor -- start 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 -- start start 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 0x560b5cb43610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb60ad0 0x560b5cb61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb18f10 0x560b5cb5f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ca7ba10 con 0x560b5cb43230 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ca24b30 con 0x560b5cb60ad0 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b5ca77c50 con 0x560b5cb18f10 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a36d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 0x560b5cb43610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a36d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 0x560b5cb43610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43880/0 (socket says 172.21.15.148:43880) 2024-04-25T09:57:19.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.863+0000 7f4a36d9d640 1 -- 172.21.15.148:0/3016182203 learned_addr learned my addr 172.21.15.148:0/3016182203 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a3759e640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb18f10 0x560b5cb5f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb60ad0 0x560b5cb61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a36d9d640 1 -- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb18f10 msgr2=0x560b5cb5f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb18f10 0x560b5cb5f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a36d9d640 1 -- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb60ad0 msgr2=0x560b5cb61020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb60ad0 0x560b5cb61020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a36d9d640 1 -- 172.21.15.148:0/3016182203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b5ca39e10 con 0x560b5cb43230 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a3759e640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb18f10 0x560b5cb5f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb60ad0 0x560b5cb61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 0x560b5cb43610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4a28035040 tx=0x7f4a2802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f7b3c6249b53c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a2cff9640 1 -- 172.21.15.148:0/3016182203 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2804d070 con 0x560b5cb43230 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a2cff9640 1 -- 172.21.15.148:0/3016182203 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28032e20 con 0x560b5cb43230 2024-04-25T09:57:19.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.864+0000 7f4a2cff9640 1 -- 172.21.15.148:0/3016182203 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28037040 con 0x560b5cb43230 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/3016182203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 msgr2=0x560b5cb43610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 0x560b5cb43610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4a28035040 tx=0x7f4a2802c000 comp rx=0 tx=0).stop 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/3016182203 shutdown_connections 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb18f10 0x560b5cb5f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb60ad0 0x560b5cb61020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/3016182203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb43230 0x560b5cb43610 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/3016182203 >> 172.21.15.148:0/3016182203 conn(0x560b5cb15840 msgr2=0x560b5cb5e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/3016182203 shutdown_connections 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/3016182203 wait complete. 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 -- start start 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.865+0000 7f4a3fdf1740 1 Processor -- start 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb37520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb5ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb5a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5cb19cb0 con 0x560b5cb60ad0 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5ca380c0 con 0x560b5cb5a1e0 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b5cb13770 con 0x560b5cb43230 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3fdf1740 10 client.0 _ll_register_callbacks cb 0x560b5ca2bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb5ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb5ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43888/0 (socket says 172.21.15.148:43888) 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 -- 172.21.15.148:0/4216843931 learned_addr learned my addr 172.21.15.148:0/4216843931 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a3759e640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb5a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb37520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 -- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 msgr2=0x560b5cb37520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb37520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 -- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 msgr2=0x560b5cb5a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb5a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a2ffff640 1 -- 172.21.15.148:0/4216843931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b5c7902d0 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.866+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb37520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb5ce60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4a20058a50 tx=0x7f4a20058b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214e8a5d51823b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a15ffb640 1 -- 172.21.15.148:0/4216843931 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2001f040 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a15ffb640 1 -- 172.21.15.148:0/4216843931 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a20056dc0 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a28032e20 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a15ffb640 1 -- 172.21.15.148:0/4216843931 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a2008a0b0 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 10 client.6033 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560b5c85a6b0 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a15ffb640 1 -- 172.21.15.148:0/4216843931 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28032e20 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a15ffb640 1 -- 172.21.15.148:0/4216843931 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a2005db30 con 0x560b5cb60ad0 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 10 client.6033 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 10 client.6033 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 20 client.6033 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb71ca0 0x560b5cb72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 4 client.6033 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b5cada920 con 0x560b5cb71ca0 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.867+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb71ca0 0x560b5cb72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb71ca0 0x560b5cb72080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c548af553be9e9 server_cookie=461efe85a135319d in_seq=0 out_seq=0 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a15ffb640 10 client.6033 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a15ffb640 1 -- 172.21.15.148:0/4216843931 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b5cada920 con 0x560b5cb71ca0 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a15ffb640 10 client.6033 handle_command_reply: tid=0 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 client.6033 shutdown 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 msgr2=0x560b5cb5ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb5ce60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4a20058a50 tx=0x7f4a20058b80 comp rx=0 tx=0).stop 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 shutdown_connections 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb71ca0 0x560b5cb72080 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb5a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb5ce60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a15ffb640 0 client.6033 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/4216843931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb37520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 >> 172.21.15.148:0/4216843931 conn(0x560b5cb18f10 msgr2=0x560b5cb61e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 shutdown_connections 2024-04-25T09:57:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.868+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/4216843931 wait complete. 2024-04-25T09:57:19.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.869+0000 7f4a3fdf1740 20 client.6033 trim_cache size 0 max 16384 2024-04-25T09:57:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.884+0000 7f4a3fdf1740 1 Processor -- start 2024-04-25T09:57:19.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.884+0000 7f4a3fdf1740 1 -- start start 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb43230 0x560b5cb58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a382003a0 con 0x560b5cb60ad0 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381ff5b0 con 0x560b5cb43230 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381f40e0 con 0x560b5cb5a1e0 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43898/0 (socket says 172.21.15.148:43898) 2024-04-25T09:57:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 -- 172.21.15.148:0/1707991743 learned_addr learned my addr 172.21.15.148:0/1707991743 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 -- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb5a1e0 msgr2=0x560b5cb192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb43230 0x560b5cb58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 -- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb43230 msgr2=0x560b5cb58ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb43230 0x560b5cb58ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 -- 172.21.15.148:0/1707991743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b5cb56370 con 0x560b5cb60ad0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb43230 0x560b5cb58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a3759e640 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4a30005520 tx=0x7f4a30011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0a5404f7fe4cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a17fff640 1 -- 172.21.15.148:0/1707991743 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30018070 con 0x560b5cb60ad0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a17fff640 1 -- 172.21.15.148:0/1707991743 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000c070 con 0x560b5cb60ad0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.885+0000 7f4a17fff640 1 -- 172.21.15.148:0/1707991743 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30014850 con 0x560b5cb60ad0 2024-04-25T09:57:19.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/1707991743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 msgr2=0x560b5cb19830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4a30005520 tx=0x7f4a30011680 comp rx=0 tx=0).stop 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/1707991743 shutdown_connections 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/1707991743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb43230 0x560b5cb58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/1707991743 >> 172.21.15.148:0/1707991743 conn(0x560b5cb18f10 msgr2=0x560b5cb37330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/1707991743 shutdown_connections 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/1707991743 wait complete. 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 -- start start 2024-04-25T09:57:19.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.886+0000 7f4a3fdf1740 1 Processor -- start 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb70cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b5cb37a60 con 0x560b5cb60ad0 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a381ff150 con 0x560b5cb5a1e0 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b5cb55470 con 0x560b5cb43230 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3fdf1740 10 client.0 _ll_register_callbacks cb 0x560b5cb13c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43906/0 (socket says 172.21.15.148:43906) 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a3759e640 1 -- 172.21.15.148:0/243166764 learned_addr learned my addr 172.21.15.148:0/243166764 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb70cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.887+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3759e640 1 -- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 msgr2=0x560b5cb70cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3759e640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb70cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3759e640 1 -- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 msgr2=0x560b5cb192f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3759e640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3759e640 1 -- 172.21.15.148:0/243166764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b5cb602f0 con 0x560b5cb60ad0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a2ffff640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3759e640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4a300116c0 tx=0x7f4a30003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447c5230e9ffe05d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a15ffb640 1 -- 172.21.15.148:0/243166764 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3002d060 con 0x560b5cb60ad0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a15ffb640 1 -- 172.21.15.148:0/243166764 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a3001ec40 con 0x560b5cb60ad0 2024-04-25T09:57:19.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a3000c070 con 0x560b5cb60ad0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a15ffb640 1 -- 172.21.15.148:0/243166764 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a30033050 con 0x560b5cb60ad0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 10 client.6039 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560b5cb12760 con 0x560b5cb60ad0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a15ffb640 1 -- 172.21.15.148:0/243166764 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000c070 con 0x560b5cb60ad0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a15ffb640 1 -- 172.21.15.148:0/243166764 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4a30036020 con 0x560b5cb60ad0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 10 client.6039 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 10 client.6039 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 20 client.6039 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb173e0 0x560b5cb45f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 4 client.6039 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560b5ca88f50 con 0x560b5cb173e0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.888+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb173e0 0x560b5cb45f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a15ffb640 10 client.6039 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:19.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a36d9d640 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb173e0 0x560b5cb45f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4957c6181dbedcc4 server_cookie=7b2128be8f71091c in_seq=0 out_seq=0 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a15ffb640 1 -- 172.21.15.148:0/243166764 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560b5ca88f50 con 0x560b5cb173e0 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a15ffb640 10 client.6039 handle_command_reply: tid=0 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 client.6039 shutdown 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 msgr2=0x560b5cb19830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4a300116c0 tx=0x7f4a30003a70 comp rx=0 tx=0).stop 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 shutdown_connections 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5cb60ad0 0x560b5cb19830 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x560b5cb173e0 0x560b5cb45f00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560b5cb5a1e0 0x560b5cb192f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 --2- 172.21.15.148:0/243166764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560b5cb43230 0x560b5cb70cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a15ffb640 0 client.6039 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 >> 172.21.15.148:0/243166764 conn(0x560b5cb18f10 msgr2=0x560b5cb18690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 shutdown_connections 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 1 -- 172.21.15.148:0/243166764 wait complete. 2024-04-25T09:57:19.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.889+0000 7f4a3fdf1740 20 client.6039 trim_cache size 0 max 16384 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 5862, 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 8.9155253319999996, 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 23, 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 55, 2024-04-25T09:57:19.901 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 55, 2024-04-25T09:57:19.902 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 14.001822998 2024-04-25T09:57:19.902 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:19.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a1806b4d0 msgr2=0x7f4a1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a1806b4d0 0x7f4a1806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034000 tx=0x7f4a2802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 msgr2=0x7f4a380cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 0x7f4a380cbc90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000c7f0 tx=0x7f4a2000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 shutdown_connections 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a1806b4d0 0x7f4a1806d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a3815dac0 0x7f4a38151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a380d34c0 0x7f4a380cbc90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 --2- 172.21.15.148:0/1422704800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a380d1f30 0x7f4a380b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 >> 172.21.15.148:0/1422704800 conn(0x7f4a38061c30 msgr2=0x7f4a380b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 shutdown_connections 2024-04-25T09:57:19.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.890+0000 7f4a3d01b640 1 -- 172.21.15.148:0/1422704800 wait complete. 2024-04-25T09:57:19.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:57:20.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.999+0000 7fbecf73b640 1 Processor -- start 2024-04-25T09:57:20.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:19.999+0000 7fbecf73b640 1 -- start start 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecf73b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80ce330 0x7fbec806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecf73b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbec80700a0 0x7fbec815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecf73b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec81626e0 con 0x7fbec806abe0 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecf73b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80cd270 con 0x7fbec80700a0 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecf73b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80cd550 con 0x7fbec80ce330 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecdcd8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbec80700a0 0x7fbec815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecdcd8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbec80700a0 0x7fbec815d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43920/0 (socket says 172.21.15.148:43920) 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.000+0000 7fbecdcd8640 1 -- 172.21.15.148:0/3673629370 learned_addr learned my addr 172.21.15.148:0/3673629370 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:20.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbecccd6640 1 --2- 172.21.15.148:0/3673629370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80ce330 0x7fbec806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbecdcd8640 1 -- 172.21.15.148:0/3673629370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80ce330 msgr2=0x7fbec806fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbecdcd8640 1 --2- 172.21.15.148:0/3673629370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80ce330 0x7fbec806fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbecdcd8640 1 -- 172.21.15.148:0/3673629370 >> v1:172.21.15.188:6789/0 conn(0x7fbec806abe0 legacy=0x7fbec806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:20.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbecdcd8640 1 -- 172.21.15.148:0/3673629370 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80cd830 con 0x7fbec80700a0 2024-04-25T09:57:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbecdcd8640 1 --2- 172.21.15.148:0/3673629370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec80700a0 0x7fbec815d370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009870 tx=0x7fbec002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26aa4fd2541433a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbebffff640 1 -- 172.21.15.148:0/3673629370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0037070 con 0x7fbec80700a0 2024-04-25T09:57:20.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbebffff640 1 -- 172.21.15.148:0/3673629370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0032440 con 0x7fbec80700a0 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.001+0000 7fbebffff640 1 -- 172.21.15.148:0/3673629370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0032810 con 0x7fbec80700a0 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 -- 172.21.15.148:0/3673629370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec80700a0 msgr2=0x7fbec815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 --2- 172.21.15.148:0/3673629370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec80700a0 0x7fbec815d370 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009870 tx=0x7fbec002a180 comp rx=0 tx=0).stop 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 -- 172.21.15.148:0/3673629370 shutdown_connections 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 --2- 172.21.15.148:0/3673629370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec80700a0 0x7fbec815d370 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 --2- 172.21.15.148:0/3673629370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80ce330 0x7fbec806fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 -- 172.21.15.148:0/3673629370 >> 172.21.15.148:0/3673629370 conn(0x7fbec80b2220 msgr2=0x7fbec80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 -- 172.21.15.148:0/3673629370 shutdown_connections 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.002+0000 7fbecf73b640 1 -- 172.21.15.148:0/3673629370 wait complete. 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.003+0000 7fbecf73b640 1 Processor -- start 2024-04-25T09:57:20.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.003+0000 7fbecf73b640 1 -- start start 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecf73b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 0x7fbec81f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecf73b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80700a0 0x7fbec81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecf73b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec80ce330 0x7fbec80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecf73b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec815d8b0 con 0x7fbec806abe0 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecf73b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8161780 con 0x7fbec80ce330 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecf73b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec815da30 con 0x7fbec80700a0 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 0x7fbec81f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 0x7fbec81f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43926/0 (socket says 172.21.15.148:43926) 2024-04-25T09:57:20.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 -- 172.21.15.148:0/725636442 learned_addr learned my addr 172.21.15.148:0/725636442 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecccd6640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80700a0 0x7fbec81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecdcd8640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec80ce330 0x7fbec80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 -- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80700a0 msgr2=0x7fbec81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80700a0 0x7fbec81f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 -- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec80ce330 msgr2=0x7fbec80b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec80ce330 0x7fbec80b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecd4d7640 1 -- 172.21.15.148:0/725636442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec81f5170 con 0x7fbec806abe0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.004+0000 7fbecdcd8640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec80ce330 0x7fbec80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.005+0000 7fbecd4d7640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 0x7fbec81f8690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c7c0 tx=0x7fbec400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.005+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4016020 con 0x7fbec806abe0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.005+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec400aa50 con 0x7fbec806abe0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.005+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4012420 con 0x7fbec806abe0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.005+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec81f5450 con 0x7fbec806abe0 2024-04-25T09:57:20.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.005+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec81f8bd0 con 0x7fbec806abe0 2024-04-25T09:57:20.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.006+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec80c7530 con 0x7fbec806abe0 2024-04-25T09:57:20.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.006+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbec4019020 con 0x7fbec806abe0 2024-04-25T09:57:20.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.006+0000 7fbebdffb640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbeac06b4d0 0x7fbeac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:20.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.007+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbec408a0d0 con 0x7fbec806abe0 2024-04-25T09:57:20.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.007+0000 7fbecccd6640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbeac06b4d0 0x7fbeac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:20.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.007+0000 7fbecccd6640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbeac06b4d0 0x7fbeac06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8006fd0 tx=0x7fbeb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:20.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.011+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbec4020070 con 0x7fbec806abe0 2024-04-25T09:57:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.217+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbec80b2ff0 con 0x7fbec806abe0 2024-04-25T09:57:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.218+0000 7fbebdffb640 1 -- 172.21.15.148:0/725636442 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbec4058e50 con 0x7fbec806abe0 2024-04-25T09:57:20.221 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:57:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbeac06b4d0 msgr2=0x7fbeac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbeac06b4d0 0x7fbeac06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8006fd0 tx=0x7fbeb8008040 comp rx=0 tx=0).stop 2024-04-25T09:57:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 msgr2=0x7fbec81f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 0x7fbec81f8690 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c7c0 tx=0x7fbec400cc80 comp rx=0 tx=0).stop 2024-04-25T09:57:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 shutdown_connections 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbeac06b4d0 0x7fbeac06d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.219+0000 7fbecf73b640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbec80ce330 0x7fbec80b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.220+0000 7fbecf73b640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbec80700a0 0x7fbec81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.220+0000 7fbecf73b640 1 --2- 172.21.15.148:0/725636442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbec806abe0 0x7fbec81f8690 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.220+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 >> 172.21.15.148:0/725636442 conn(0x7fbec80b2220 msgr2=0x7fbec815fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.220+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 shutdown_connections 2024-04-25T09:57:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:20.220+0000 7fbecf73b640 1 -- 172.21.15.148:0/725636442 wait complete. 2024-04-25T09:57:20.233 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:57:20.233 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:57:20.233 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:57:20.234 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:57:20.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:20.234 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:57:20.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:57:20.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:57:20.234 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:20.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.234 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:57:20.249 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:57:20.249 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:57:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:57:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:57:20.250 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42595sec preferred_lft 42595sec 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:57:20.251 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T09:57:20.252 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:57:20.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.253 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:57:20.312 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:57:20.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.314 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:57:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:57:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:57:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:57:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:57:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:57:20.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.371 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:57:20.371 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:57:20.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:57:20.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:57:20.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:57:20.371 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:57:20.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.454 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:57:20.508 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:57:20.509 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:57:20.509 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:20.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:57:20.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:20.573 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:20.627 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:57:20.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:57:20.628 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:20.683 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:57:20.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.683 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:57:20.765 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:57:20.765 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:57:20.765 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:57:20.773 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:57:20.774 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:57:20.774 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:57:20.774 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:57:20.774 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:57:20.779 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:57:20.779 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:57:20.779 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:57:20.779 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:57:20.780 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:57:20.781 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:57:20.781 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:57:20.782 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:57:20.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.782 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:20.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.847 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:57:20.903 INFO:teuthology.orchestra.run.smithi148.stdout:client5123 2024-04-25T09:57:20.903 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:20.903 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:20.958 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:20.959 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:57:20.959 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:57:20.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:20.959 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> import json 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> import sys 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2024-04-25T09:57:21.017 DEBUG:teuthology.orchestra.run.smithi148:> sys.exit(e.errno) 2024-04-25T09:57:21.018 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:57:21.018 DEBUG:teuthology.orchestra.run.smithi148:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:57:21.018 DEBUG:teuthology.orchestra.run.smithi148:> print(json.dumps( 2024-04-25T09:57:21.018 DEBUG:teuthology.orchestra.run.smithi148:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:57:21.018 DEBUG:teuthology.orchestra.run.smithi148:> indent=2)) 2024-04-25T09:57:21.018 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:57:21.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:21 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T09:57:21.135 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T09:57:21.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:21.136 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T09:57:21.191 INFO:teuthology.orchestra.run.smithi148.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T09:57:21.336 INFO:teuthology.orchestra.run.smithi148.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T09:57:23.780 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:23.782 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:23.871 INFO:teuthology.orchestra.run.smithi148.stderr:Updating files: 57% (805/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T09:57:23.876 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T09:57:23.876 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:57:23.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:24.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.014+0000 7fbbde01b640 1 Processor -- start 2024-04-25T09:57:24.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.014+0000 7fbbde01b640 1 -- start start 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbde01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd80cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbde01b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbbd8099a40 0x7fbbd8099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbde01b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd80cc3c0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbde01b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd80cccb0 con 0x7fbbd8099a40 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbde01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd815f400 con 0x7fbbd815f020 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbbd80b0b90 0x7fbbd80b10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47200/0 (socket says 172.21.15.148:47200) 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd6ffd640 1 -- 172.21.15.148:0/1159461765 learned_addr learned my addr 172.21.15.148:0/1159461765 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd7fff640 1 --2- 172.21.15.148:0/1159461765 >> v2:172.21.15.188:3300/0 conn(0x7fbbd8099a40 0x7fbbd8099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/1159461765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd80cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd67fc640 1 -- 172.21.15.148:0/1159461765 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331115012 0 0) 0x7fbbd80cc3c0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd67fc640 1 -- 172.21.15.148:0/1159461765 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc0003520 con 0x7fbbd80b0b90 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd77fe640 1 -- 172.21.15.148:0/1159461765 >> v2:172.21.15.188:3300/0 conn(0x7fbbd8099a40 msgr2=0x7fbbd8099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/1159461765 >> v2:172.21.15.188:3300/0 conn(0x7fbbd8099a40 0x7fbbd8099ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd77fe640 1 -- 172.21.15.148:0/1159461765 >> v1:172.21.15.148:6789/0 conn(0x7fbbd80b0b90 legacy=0x7fbbd80b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.015+0000 7fbbd67fc640 1 -- 172.21.15.148:0/1159461765 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2781325406 0 0) 0x7fbbc0003520 con 0x7fbbd80b0b90 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.016+0000 7fbbd77fe640 1 -- 172.21.15.148:0/1159461765 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80d0550 con 0x7fbbd815f020 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.016+0000 7fbbd7fff640 1 --2- 172.21.15.148:0/1159461765 >> v2:172.21.15.188:3300/0 conn(0x7fbbd8099a40 0x7fbbd8099ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.016+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/1159461765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd80cfe00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80097a0 tx=0x7fbbc802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc1cb26b0fc1996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbd67fc640 1 -- 172.21.15.148:0/1159461765 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8037070 con 0x7fbbd815f020 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbd67fc640 1 -- 172.21.15.148:0/1159461765 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc802aba0 con 0x7fbbd815f020 2024-04-25T09:57:24.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbd67fc640 1 -- 172.21.15.148:0/1159461765 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc001670 con 0x7fbbd815f020 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 -- 172.21.15.148:0/1159461765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 msgr2=0x7fbbd80cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 --2- 172.21.15.148:0/1159461765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd80cfe00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80097a0 tx=0x7fbbc802a110 comp rx=0 tx=0).stop 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 -- 172.21.15.148:0/1159461765 shutdown_connections 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 --2- 172.21.15.148:0/1159461765 >> v2:172.21.15.188:3300/0 conn(0x7fbbd8099a40 0x7fbbd8099ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 --2- 172.21.15.148:0/1159461765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd80cfe00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 -- 172.21.15.148:0/1159461765 >> 172.21.15.148:0/1159461765 conn(0x7fbbd80b3310 msgr2=0x7fbbd80b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 -- 172.21.15.148:0/1159461765 shutdown_connections 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.017+0000 7fbbde01b640 1 -- 172.21.15.148:0/1159461765 wait complete. 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.018+0000 7fbbde01b640 1 Processor -- start 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.018+0000 7fbbde01b640 1 -- start start 2024-04-25T09:57:24.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbde01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd8099a40 0x7fbbd81f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbde01b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 0x7fbbd80c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbde01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd8156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbde01b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cc940 con 0x7fbbd80b0b90 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbde01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cc660 con 0x7fbbd8099a40 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbde01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd80cd140 con 0x7fbbd815f020 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbd6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 0x7fbbd80c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbd6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 0x7fbbd80c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42734/0 (socket says 172.21.15.148:42734) 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbd6ffd640 1 -- 172.21.15.148:0/438061667 learned_addr learned my addr 172.21.15.148:0/438061667 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd8099a40 0x7fbbd81f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.019+0000 7fbbd7fff640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd8156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd6ffd640 1 -- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 msgr2=0x7fbbd8156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd6ffd640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd8156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd6ffd640 1 -- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd8099a40 msgr2=0x7fbbd81f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd6ffd640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd8099a40 0x7fbbd81f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd6ffd640 1 -- 172.21.15.148:0/438061667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd81f5ca0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd8099a40 0x7fbbd81f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbd6ffd640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 0x7fbbd80c7c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c7b0 tx=0x7fbbcc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc017070 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc005ad0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd81f5400 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc007070 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.020+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd81f59c0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.021+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd8073990 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.021+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbcc01a070 con 0x7fbbd80b0b90 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.022+0000 7fbbbffff640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbc406b500 0x7fbbc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.022+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbbcc089830 con 0x7fbbd80b0b90 2024-04-25T09:57:24.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.022+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbc406b500 0x7fbbc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.022+0000 7fbbd77fe640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbc406b500 0x7fbbc406d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.026+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbcc0585b0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.233+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbd80b77a0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.234+0000 7fbbbffff640 1 -- 172.21.15.148:0/438061667 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fbbcc057eb0 con 0x7fbbd80b0b90 2024-04-25T09:57:24.237 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:24.238 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:24.238 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbc406b500 msgr2=0x7fbbc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbc406b500 0x7fbbc406d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 msgr2=0x7fbbd80c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 0x7fbbd80c7c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c7b0 tx=0x7fbbcc00cc70 comp rx=0 tx=0).stop 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 shutdown_connections 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbc406b500 0x7fbbc406d9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd815f020 0x7fbbd8156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd80b0b90 0x7fbbd80c7c60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 --2- 172.21.15.148:0/438061667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd8099a40 0x7fbbd81f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 >> 172.21.15.148:0/438061667 conn(0x7fbbd80b3310 msgr2=0x7fbbd81595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 shutdown_connections 2024-04-25T09:57:24.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.236+0000 7fbbde01b640 1 -- 172.21.15.148:0/438061667 wait complete. 2024-04-25T09:57:24.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:57:24.352 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:24.352 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": -2 2024-04-25T09:57:24.352 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:24.360 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:57:24.360 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:57:24.361 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:24.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:57:24.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.457+0000 7f56f71e0640 1 Processor -- start 2024-04-25T09:57:24.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.458+0000 7f56f71e0640 1 -- start start 2024-04-25T09:57:24.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.458+0000 7f56f71e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f015a9f0 0x7f56f00c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.458+0000 7f56f71e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f56f00cd6f0 0x7f56f00cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.458+0000 7f56f71e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f00c6940 con 0x7f56f00cdbf0 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.458+0000 7f56f71e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00b0020 con 0x7f56f00cd6f0 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.458+0000 7f56f71e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cb5a0 con 0x7f56f015a9f0 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f577d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f56f00cd6f0 0x7f56f00cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56effff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f56f00cdbf0 0x7f56f00ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53018/0 (socket says 172.21.15.148:53018) 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56effff640 1 -- 172.21.15.148:0/1730407716 learned_addr learned my addr 172.21.15.148:0/1730407716 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f015a9f0 0x7f56f00c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f577d640 1 -- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f015a9f0 msgr2=0x7f56f00c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f577d640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f015a9f0 0x7f56f00c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f577d640 1 -- 172.21.15.148:0/1730407716 >> v1:172.21.15.188:6789/0 conn(0x7f56f00cdbf0 legacy=0x7f56f00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f577d640 1 -- 172.21.15.148:0/1730407716 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f015add0 con 0x7f56f00cd6f0 2024-04-25T09:57:24.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.459+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f015a9f0 0x7f56f00c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.460+0000 7f56f577d640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f00cd6f0 0x7f56f00cba30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009d90 tx=0x7f56e802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7eade34d99d2ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.460+0000 7f56ef7fe640 1 -- 172.21.15.148:0/1730407716 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8007740 con 0x7f56f00cd6f0 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.460+0000 7f56ef7fe640 1 -- 172.21.15.148:0/1730407716 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e80078e0 con 0x7f56f00cd6f0 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.460+0000 7f56ef7fe640 1 -- 172.21.15.148:0/1730407716 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8007cb0 con 0x7f56f00cd6f0 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 -- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f00cd6f0 msgr2=0x7f56f00cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f00cd6f0 0x7f56f00cba30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009d90 tx=0x7f56e802a660 comp rx=0 tx=0).stop 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 -- 172.21.15.148:0/1730407716 shutdown_connections 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f00cd6f0 0x7f56f00cba30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 --2- 172.21.15.148:0/1730407716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f015a9f0 0x7f56f00c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 -- 172.21.15.148:0/1730407716 >> 172.21.15.148:0/1730407716 conn(0x7f56f00b2390 msgr2=0x7f56f00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 -- 172.21.15.148:0/1730407716 shutdown_connections 2024-04-25T09:57:24.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.461+0000 7f56f71e0640 1 -- 172.21.15.148:0/1730407716 wait complete. 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.462+0000 7f56f71e0640 1 Processor -- start 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.462+0000 7f56f71e0640 1 -- start start 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f71e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 0x7f56f014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f71e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f00cdbf0 0x7f56f0165b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f71e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f015a9f0 0x7f56f00aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f71e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cb230 con 0x7f56f00cd6f0 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f71e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00caf50 con 0x7f56f015a9f0 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f71e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00ce070 con 0x7f56f00cdbf0 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 0x7f56f014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 0x7f56f014d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42736/0 (socket says 172.21.15.148:42736) 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2621859216 learned_addr learned my addr 172.21.15.148:0/2621859216 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56effff640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f00cdbf0 0x7f56f0165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f577d640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f015a9f0 0x7f56f00aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f00cdbf0 msgr2=0x7f56f0165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f00cdbf0 0x7f56f0165b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f015a9f0 msgr2=0x7f56f00aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f015a9f0 0x7f56f00aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2621859216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0163d50 con 0x7f56f00cd6f0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f577d640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f015a9f0 0x7f56f00aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.463+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 0x7f56f014d720 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f56e000e990 tx=0x7f56e000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.464+0000 7f56ed7fa640 1 -- 172.21.15.148:0/2621859216 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0017070 con 0x7f56f00cd6f0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.464+0000 7f56ed7fa640 1 -- 172.21.15.148:0/2621859216 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000c9e0 con 0x7f56f00cd6f0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.464+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f0164030 con 0x7f56f00cd6f0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.464+0000 7f56ed7fa640 1 -- 172.21.15.148:0/2621859216 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000cdb0 con 0x7f56f00cd6f0 2024-04-25T09:57:24.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.464+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f01645f0 con 0x7f56f00cd6f0 2024-04-25T09:57:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.465+0000 7f56ed7fa640 1 -- 172.21.15.148:0/2621859216 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56e001a070 con 0x7f56f00cd6f0 2024-04-25T09:57:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.466+0000 7f56ed7fa640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56dc06b4d0 0x7f56dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.466+0000 7f56effff640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56dc06b4d0 0x7f56dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.466+0000 7f56ed7fa640 1 -- 172.21.15.148:0/2621859216 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e008a560 con 0x7f56f00cd6f0 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.466+0000 7f56effff640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56dc06b4d0 0x7f56dc06d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56f00af8f0 tx=0x7f56e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 Processor -- start 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 -- start start 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdca9ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab9f50 0x55efdcaba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 0x55efdcab8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efdc9d4c60 con 0x55efdca72390 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efdc97dee0 con 0x55efdca9c6b0 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.467+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efdc9d0d00 con 0x55efdcab9f50 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 0x55efdcab8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 0x55efdcab8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42742/0 (socket says 172.21.15.148:42742) 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 -- 172.21.15.148:0/393840653 learned_addr learned my addr 172.21.15.148:0/393840653 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdca9ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56effff640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab9f50 0x55efdcaba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 -- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab9f50 msgr2=0x55efdcaba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab9f50 0x55efdcaba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 -- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 msgr2=0x55efdca9ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdca9ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 -- 172.21.15.148:0/393840653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efdc7b59f0 con 0x55efdca72390 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56effff640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab9f50 0x55efdcaba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdca9ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56f577d640 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 0x55efdcab8610 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009eb0 tx=0x7f56e802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8263160ee4f01cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56d67fc640 1 -- 172.21.15.148:0/393840653 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8007740 con 0x55efdca72390 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56d67fc640 1 -- 172.21.15.148:0/393840653 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e80078e0 con 0x55efdca72390 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.468+0000 7f56d67fc640 1 -- 172.21.15.148:0/393840653 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8007cb0 con 0x55efdca72390 2024-04-25T09:57:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 -- 172.21.15.148:0/393840653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 msgr2=0x55efdcab8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 0x55efdcab8610 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009eb0 tx=0x7f56e802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 -- 172.21.15.148:0/393840653 shutdown_connections 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca72390 0x55efdcab8610 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab9f50 0x55efdcaba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/393840653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdca9ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 -- 172.21.15.148:0/393840653 >> 172.21.15.148:0/393840653 conn(0x55efdca6ecc0 msgr2=0x55efdcab74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 -- 172.21.15.148:0/393840653 shutdown_connections 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 -- 172.21.15.148:0/393840653 wait complete. 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.469+0000 7f56f9fc0740 1 -- start start 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 Processor -- start 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdcacb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efdca73130 con 0x55efdcab9f50 2024-04-25T09:57:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efdc991470 con 0x55efdca9c6b0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efdca6cbf0 con 0x55efdcab0120 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56f9fc0740 10 client.0 _ll_register_callbacks cb 0x55efdc95d7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdcacb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdcacb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42744/0 (socket says 172.21.15.148:42744) 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.470+0000 7f56effff640 1 -- 172.21.15.148:0/508419579 learned_addr learned my addr 172.21.15.148:0/508419579 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f577d640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56effff640 1 -- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 msgr2=0x55efdca9e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56effff640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56effff640 1 -- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 msgr2=0x55efdcab60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56effff640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56effff640 1 -- 172.21.15.148:0/508419579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efdc6a92e0 con 0x55efdcab9f50 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56effff640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdcacb5c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f56e40073b0 tx=0x7f56e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea451ee696cea491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56cf7fe640 1 -- 172.21.15.148:0/508419579 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e400ec10 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56cf7fe640 1 -- 172.21.15.148:0/508419579 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e400edb0 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e80078e0 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56cf7fe640 1 -- 172.21.15.148:0/508419579 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e4022430 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f9fc0740 10 client.6060 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55efdc816180 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56cf7fe640 1 -- 172.21.15.148:0/508419579 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e80078e0 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56cf7fe640 1 -- 172.21.15.148:0/508419579 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56e40225d0 con 0x55efdcab9f50 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f9fc0740 10 client.6060 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f9fc0740 10 client.6060 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.471+0000 7f56f9fc0740 20 client.6060 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca9e900 0x55efdca6bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56f9fc0740 4 client.6060 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efdc9e21a0 con 0x55efdca9e900 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca9e900 0x55efdca6bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56cf7fe640 10 client.6060 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca9e900 0x55efdca6bdf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e7ed5719817a9d server_cookie=c2d06fdc7981838f in_seq=0 out_seq=0 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56cf7fe640 1 -- 172.21.15.148:0/508419579 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efdc9e21a0 con 0x55efdca9e900 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56cf7fe640 10 client.6060 handle_command_reply: tid=0 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.472+0000 7f56f9fc0740 1 client.6060 shutdown 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 msgr2=0x55efdcacb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdcacb5c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f56e40073b0 tx=0x7f56e4000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 shutdown_connections 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdcacb5c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca9e900 0x55efdca6bdf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/508419579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56cf7fe640 0 client.6060 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 >> 172.21.15.148:0/508419579 conn(0x55efdca72390 msgr2=0x55efdcab6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 shutdown_connections 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 1 -- 172.21.15.148:0/508419579 wait complete. 2024-04-25T09:57:24.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.473+0000 7f56f9fc0740 20 client.6060 trim_cache size 0 max 16384 2024-04-25T09:57:24.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 Processor -- start 2024-04-25T09:57:24.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 -- start start 2024-04-25T09:57:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f02047e0 con 0x55efdcab9f50 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f02039f0 con 0x55efdca9c6b0 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00b59e0 con 0x55efdcab0120 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.488+0000 7f56f577d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42750/0 (socket says 172.21.15.148:42750) 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 -- 172.21.15.148:0/2720703341 learned_addr learned my addr 172.21.15.148:0/2720703341 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 -- 172.21.15.148:0/2720703341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 msgr2=0x55efdca9ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 -- 172.21.15.148:0/2720703341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 msgr2=0x55efdcab12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 -- 172.21.15.148:0/2720703341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efdca6c160 con 0x55efdcab9f50 2024-04-25T09:57:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56f577d640 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdca72770 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f56e8007300 tx=0x7f56e8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572d3a62d7f4e9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56d5ffb640 1 -- 172.21.15.148:0/2720703341 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8003a90 con 0x55efdcab9f50 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56d5ffb640 1 -- 172.21.15.148:0/2720703341 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8003c30 con 0x55efdcab9f50 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.489+0000 7f56d5ffb640 1 -- 172.21.15.148:0/2720703341 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004040 con 0x55efdcab9f50 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2720703341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 msgr2=0x55efdca72770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdca72770 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f56e8007300 tx=0x7f56e8030000 comp rx=0 tx=0).stop 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2720703341 shutdown_connections 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdca9ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2720703341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdca9c6b0 0x55efdcab12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2720703341 >> 172.21.15.148:0/2720703341 conn(0x55efdca72390 msgr2=0x55efdca6d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2720703341 shutdown_connections 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2720703341 wait complete. 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 -- start start 2024-04-25T09:57:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.490+0000 7f56f9fc0740 1 Processor -- start 2024-04-25T09:57:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 0x55efdca73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdcb216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0203490 con 0x55efdca9c6b0 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efdca9e710 con 0x55efdcab9f50 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efdcab5020 con 0x55efdcab0120 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f9fc0740 10 client.0 _ll_register_callbacks cb 0x55efdca6d0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 0x55efdca73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 0x55efdca73760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42754/0 (socket says 172.21.15.148:42754) 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2184753518 learned_addr learned my addr 172.21.15.148:0/2184753518 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f577d640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56effff640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdcb216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 msgr2=0x55efdcb216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdcb216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdcab9f50 msgr2=0x55efdca72770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.491+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f4f7c640 1 -- 172.21.15.148:0/2184753518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efdc6823a0 con 0x55efdca9c6b0 2024-04-25T09:57:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f4f7c640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 0x55efdca73760 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f56e0054ff0 tx=0x7f56e0058d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60970f3738753638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56cf7fe640 1 -- 172.21.15.148:0/2184753518 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0021070 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56cf7fe640 1 -- 172.21.15.148:0/2184753518 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e0056ee0 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e8003c30 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56cf7fe640 1 -- 172.21.15.148:0/2184753518 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e0055250 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 10 client.6066 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55efdcab83e0 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56cf7fe640 1 -- 172.21.15.148:0/2184753518 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8003c30 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56cf7fe640 1 -- 172.21.15.148:0/2184753518 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56e0083070 con 0x55efdca9c6b0 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 10 client.6066 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 10 client.6066 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 20 client.6066 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca72cb0 0x55efdcaae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 4 client.6066 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:57:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55efdc9dc4f0 con 0x55efdca72cb0 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.492+0000 7f56effff640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca72cb0 0x55efdcaae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56cf7fe640 10 client.6066 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56effff640 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca72cb0 0x55efdcaae530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d115c86415da7fdd server_cookie=4b40b91f5f71f565 in_seq=0 out_seq=0 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56cf7fe640 1 -- 172.21.15.148:0/2184753518 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55efdc9dc4f0 con 0x55efdca72cb0 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56cf7fe640 10 client.6066 handle_command_reply: tid=0 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 client.6066 shutdown 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 msgr2=0x55efdca73760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 0x55efdca73760 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f56e0054ff0 tx=0x7f56e0058d30 comp rx=0 tx=0).stop 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 shutdown_connections 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55efdca72cb0 0x55efdcaae530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efdcab9f50 0x55efdca72770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efdcab0120 0x55efdcb216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 --2- 172.21.15.148:0/2184753518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efdca9c6b0 0x55efdca73760 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 >> 172.21.15.148:0/2184753518 conn(0x55efdca72390 msgr2=0x55efdcab12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56cf7fe640 0 client.6066 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 shutdown_connections 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 1 -- 172.21.15.148:0/2184753518 wait complete. 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.493+0000 7f56f9fc0740 20 client.6066 trim_cache size 0 max 16384 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": -2 2024-04-25T09:57:24.503 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:24.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56dc06b4d0 msgr2=0x7f56dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56dc06b4d0 0x7f56dc06d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56f00af8f0 tx=0x7f56e4009040 comp rx=0 tx=0).stop 2024-04-25T09:57:24.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 msgr2=0x7f56f014d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 0x7f56f014d720 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f56e000e990 tx=0x7f56e000ee50 comp rx=0 tx=0).stop 2024-04-25T09:57:24.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 shutdown_connections 2024-04-25T09:57:24.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56dc06b4d0 0x7f56dc06d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f015a9f0 0x7f56f00aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56f00cdbf0 0x7f56f0165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 --2- 172.21.15.148:0/2621859216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f00cd6f0 0x7f56f014d720 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 >> 172.21.15.148:0/2621859216 conn(0x7f56f00b2390 msgr2=0x7f56f00cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 shutdown_connections 2024-04-25T09:57:24.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.494+0000 7f56f71e0640 1 -- 172.21.15.148:0/2621859216 wait complete. 2024-04-25T09:57:24.508 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:57:24.508 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:57:24.508 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:57:24.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:24.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.601+0000 7f3df43f5640 1 Processor -- start 2024-04-25T09:57:24.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.601+0000 7f3df43f5640 1 -- start start 2024-04-25T09:57:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df43f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3dec144750 0x7f3dec06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df43f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec144b30 con 0x7f3dec06eb10 2024-04-25T09:57:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df43f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec0712d0 con 0x7f3dec0cf030 2024-04-25T09:57:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df43f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dec0715b0 con 0x7f3dec144750 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df1990640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3dec06eb10 0x7f3dec0ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47208/0 (socket says 172.21.15.148:47208) 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df1990640 1 -- 172.21.15.148:0/2171113837 learned_addr learned my addr 172.21.15.148:0/2171113837 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.602+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999082279 0 0) 0x7f3dec144b30 con 0x7f3dec06eb10 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2191640 1 --2- 172.21.15.148:0/2171113837 >> v2:172.21.15.188:3300/0 conn(0x7f3dec144750 0x7f3dec06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd8003520 con 0x7f3dec06eb10 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495442343 0 0) 0x7f3dd8003520 con 0x7f3dec06eb10 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 >> v2:172.21.15.188:3300/0 conn(0x7f3dec144750 msgr2=0x7f3dec06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 --2- 172.21.15.148:0/2171113837 >> v2:172.21.15.188:3300/0 conn(0x7f3dec144750 0x7f3dec06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 >> v1:172.21.15.188:6789/0 conn(0x7f3dec0cf030 legacy=0x7f3dec06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec071890 con 0x7f3dec06eb10 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3ddc001670 con 0x7f3dec06eb10 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 >> v1:172.21.15.148:6789/0 conn(0x7f3dec06eb10 legacy=0x7f3dec0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dd8004800 0x7f3dd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80051c0 0x7f3dd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0715b0 con 0x7f3dd80051c0 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd800c060 con 0x7f3dd80039b0 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd800c260 con 0x7f3dd8004800 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2992640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80051c0 0x7f3dd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2191640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df1990640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dd8004800 0x7f3dd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2992640 1 -- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dd8004800 msgr2=0x7f3dd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2992640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dd8004800 0x7f3dd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2992640 1 -- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80039b0 msgr2=0x7f3dd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2992640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.603+0000 7f3df2992640 1 -- 172.21.15.148:0/2171113837 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8003520 con 0x7f3dd80051c0 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.604+0000 7f3df2992640 1 -- 172.21.15.148:0/2171113837 reap_dead start 2024-04-25T09:57:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.604+0000 7f3df2992640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80051c0 0x7f3dd800b9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3de4001750 tx=0x7f3dd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d844bfcfb2dc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.604+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4001370 con 0x7f3dd80051c0 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.604+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4005310 con 0x7f3dd80051c0 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.604+0000 7f3df118f640 1 -- 172.21.15.148:0/2171113837 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de402abe0 con 0x7f3dd80051c0 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 -- 172.21.15.148:0/2171113837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80051c0 msgr2=0x7f3dd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80051c0 0x7f3dd800b9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3de4001750 tx=0x7f3dd8005830 comp rx=0 tx=0).stop 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 -- 172.21.15.148:0/2171113837 shutdown_connections 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 --2- 172.21.15.148:0/2171113837 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80051c0 0x7f3dd800b9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 -- 172.21.15.148:0/2171113837 >> 172.21.15.148:0/2171113837 conn(0x7f3dec06abe0 msgr2=0x7f3dec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 -- 172.21.15.148:0/2171113837 shutdown_connections 2024-04-25T09:57:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.605+0000 7f3df43f5640 1 -- 172.21.15.148:0/2171113837 wait complete. 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.606+0000 7f3df43f5640 1 Processor -- start 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.606+0000 7f3df43f5640 1 -- start start 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.606+0000 7f3df43f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80051c0 0x7f3dec141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.606+0000 7f3df43f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 0x7f3dec13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.606+0000 7f3df43f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec13cbd0 0x7f3dec13d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df43f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1455d0 con 0x7f3dec143210 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df43f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec145450 con 0x7f3dd80051c0 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df43f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec15c000 con 0x7f3dec13cbd0 2024-04-25T09:57:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 0x7f3dec13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 0x7f3dec13f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42764/0 (socket says 172.21.15.148:42764) 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 -- 172.21.15.148:0/3758107856 learned_addr learned my addr 172.21.15.148:0/3758107856 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df2992640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec13cbd0 0x7f3dec13d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df2191640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80051c0 0x7f3dec141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 -- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec13cbd0 msgr2=0x7f3dec13d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec13cbd0 0x7f3dec13d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 -- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80051c0 msgr2=0x7f3dec141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80051c0 0x7f3dec141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 -- 172.21.15.148:0/3758107856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec13d5c0 con 0x7f3dec143210 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df2191640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80051c0 0x7f3dec141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df1990640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 0x7f3dec13f5f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0072c0 tx=0x7f3ddc007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00f8e0 con 0x7f3dec143210 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc018070 con 0x7f3dec143210 2024-04-25T09:57:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00fce0 con 0x7f3dec143210 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec141d80 con 0x7f3dec143210 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.607+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec142340 con 0x7f3dec143210 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.608+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec073990 con 0x7f3dec143210 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.609+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ddc01d460 con 0x7f3dec143210 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.609+0000 7f3de2ffd640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd006b4d0 0x7f3dd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.609+0000 7f3df2191640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd006b4d0 0x7f3dd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.609+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ddc08a3d0 con 0x7f3dec143210 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.610+0000 7f3df2191640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd006b4d0 0x7f3dd006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3dec144010 tx=0x7f3de800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:24.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.614+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ddc059150 con 0x7f3dec143210 2024-04-25T09:57:24.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.818+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dec0b02d0 con 0x7f3dec143210 2024-04-25T09:57:24.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.818+0000 7f3de2ffd640 1 -- 172.21.15.148:0/3758107856 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f3ddc058a50 con 0x7f3dec143210 2024-04-25T09:57:24.822 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:24.822 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:24.823 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:24.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.820+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd006b4d0 msgr2=0x7f3dd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.820+0000 7f3df43f5640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd006b4d0 0x7f3dd006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3dec144010 tx=0x7f3de800b040 comp rx=0 tx=0).stop 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.820+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 msgr2=0x7f3dec13f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.820+0000 7f3df43f5640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 0x7f3dec13f5f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0072c0 tx=0x7f3ddc007780 comp rx=0 tx=0).stop 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 shutdown_connections 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dd006b4d0 0x7f3dd006d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec13cbd0 0x7f3dec13d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec143210 0x7f3dec13f5f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 --2- 172.21.15.148:0/3758107856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dd80051c0 0x7f3dec141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 >> 172.21.15.148:0/3758107856 conn(0x7f3dec06abe0 msgr2=0x7f3dec0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 shutdown_connections 2024-04-25T09:57:24.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:24.821+0000 7f3df43f5640 1 -- 172.21.15.148:0/3758107856 wait complete. 2024-04-25T09:57:24.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:57:25.023 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:25.023 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:25.023 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:25.031 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:25.032 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:57:25.032 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:25.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:57:25.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.131+0000 7f85b644a640 1 Processor -- start 2024-04-25T09:57:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.131+0000 7f85b644a640 1 -- start start 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b644a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f85b00d0ab0 0x7f85b0168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b644a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00d2420 con 0x7f85b015dac0 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b644a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00d2700 con 0x7f85b00d34c0 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b644a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00d3030 con 0x7f85b00d0ab0 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b49e7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f85b00d0ab0 0x7f85b0168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85affff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f85b015dac0 0x7f85b00d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47216/0 (socket says 172.21.15.148:47216) 2024-04-25T09:57:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85affff640 1 -- 172.21.15.148:0/3733782885 learned_addr learned my addr 172.21.15.148:0/3733782885 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85aeffd640 1 -- 172.21.15.148:0/3733782885 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358375697 0 0) 0x7f85b00d2420 con 0x7f85b015dac0 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85aeffd640 1 -- 172.21.15.148:0/3733782885 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8598003520 con 0x7f85b015dac0 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b49e7640 1 -- 172.21.15.148:0/3733782885 >> v1:172.21.15.188:6789/0 conn(0x7f85b00d34c0 legacy=0x7f85b00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b49e7640 1 -- 172.21.15.148:0/3733782885 >> v1:172.21.15.148:6789/0 conn(0x7f85b015dac0 legacy=0x7f85b00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.132+0000 7f85b49e7640 1 -- 172.21.15.148:0/3733782885 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00d29e0 con 0x7f85b00d0ab0 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.133+0000 7f85b49e7640 1 --2- 172.21.15.148:0/3733782885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b00d0ab0 0x7f85b0168670 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f85b00d31b0 tx=0x7f85a802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9064470b0d96d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.133+0000 7f85aeffd640 1 -- 172.21.15.148:0/3733782885 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8037070 con 0x7f85b00d0ab0 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.133+0000 7f85aeffd640 1 -- 172.21.15.148:0/3733782885 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a8007740 con 0x7f85b00d0ab0 2024-04-25T09:57:25.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.133+0000 7f85aeffd640 1 -- 172.21.15.148:0/3733782885 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8007b10 con 0x7f85b00d0ab0 2024-04-25T09:57:25.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 -- 172.21.15.148:0/3733782885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b00d0ab0 msgr2=0x7f85b0168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 --2- 172.21.15.148:0/3733782885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b00d0ab0 0x7f85b0168670 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f85b00d31b0 tx=0x7f85a802a780 comp rx=0 tx=0).stop 2024-04-25T09:57:25.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 -- 172.21.15.148:0/3733782885 shutdown_connections 2024-04-25T09:57:25.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 --2- 172.21.15.148:0/3733782885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b00d0ab0 0x7f85b0168670 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 -- 172.21.15.148:0/3733782885 >> 172.21.15.148:0/3733782885 conn(0x7f85b0061c30 msgr2=0x7f85b006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 -- 172.21.15.148:0/3733782885 shutdown_connections 2024-04-25T09:57:25.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.134+0000 7f85b644a640 1 -- 172.21.15.148:0/3733782885 wait complete. 2024-04-25T09:57:25.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.135+0000 7f85b644a640 1 Processor -- start 2024-04-25T09:57:25.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.135+0000 7f85b644a640 1 -- start start 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b644a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b00d0ab0 0x7f85b00b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b644a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d34c0 0x7f85b00b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b644a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 0x7f85b01469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b644a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b015dea0 con 0x7f85b015dac0 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b644a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00d2100 con 0x7f85b00d34c0 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b644a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00b97d0 con 0x7f85b00d0ab0 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 0x7f85b01469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 0x7f85b01469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42780/0 (socket says 172.21.15.148:42780) 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 -- 172.21.15.148:0/1057948940 learned_addr learned my addr 172.21.15.148:0/1057948940 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85af7fe640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d34c0 0x7f85b00b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85affff640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b00d0ab0 0x7f85b00b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 -- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b00d0ab0 msgr2=0x7f85b00b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b00d0ab0 0x7f85b00b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 -- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d34c0 msgr2=0x7f85b00b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d34c0 0x7f85b00b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 -- 172.21.15.148:0/1057948940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b01f73d0 con 0x7f85b015dac0 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85af7fe640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d34c0 0x7f85b00b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85b49e7640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 0x7f85b01469e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e80 tx=0x7f85a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85acff9640 1 -- 172.21.15.148:0/1057948940 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8042020 con 0x7f85b015dac0 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85acff9640 1 -- 172.21.15.148:0/1057948940 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a80327f0 con 0x7f85b015dac0 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.136+0000 7f85acff9640 1 -- 172.21.15.148:0/1057948940 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8037040 con 0x7f85b015dac0 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.137+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b01f7670 con 0x7f85b015dac0 2024-04-25T09:57:25.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.137+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b01f7c30 con 0x7f85b015dac0 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.138+0000 7f85acff9640 1 -- 172.21.15.148:0/1057948940 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85a8045020 con 0x7f85b015dac0 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.138+0000 7f85acff9640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f859c06b4d0 0x7f859c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.138+0000 7f85acff9640 1 -- 172.21.15.148:0/1057948940 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85a80ab3f0 con 0x7f85b015dac0 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.138+0000 7f85affff640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f859c06b4d0 0x7f859c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.139+0000 7f85affff640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f859c06b4d0 0x7f859c06d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85a00011e0 tx=0x7f85a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.139+0000 7f85b9228740 1 Processor -- start 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.139+0000 7f85b9228740 1 -- start start 2024-04-25T09:57:25.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b9228740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291689d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x56291689b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168c59d0 0x5629168c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b9228740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629167ccfe0 con 0x56291689af10 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629167fed20 con 0x5629168c59d0 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629167a7fa0 con 0x5629168d75e0 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291689d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291689d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42796/0 (socket says 172.21.15.148:42796) 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 -- 172.21.15.148:0/4028511074 learned_addr learned my addr 172.21.15.148:0/4028511074 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 -- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 msgr2=0x56291689b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x56291689b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 -- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168c59d0 msgr2=0x5629168c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85b49e7640 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168c59d0 0x5629168c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168c59d0 0x5629168c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 -- 172.21.15.148:0/4028511074 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629165dfb00 con 0x56291689af10 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.140+0000 7f85affff640 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291689d070 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f85a0005cd0 tx=0x7f85a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8f931b4dc9000b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f8595ffb640 1 -- 172.21.15.148:0/4028511074 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0007560 con 0x56291689af10 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f8595ffb640 1 -- 172.21.15.148:0/4028511074 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0007700 con 0x56291689af10 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f8595ffb640 1 -- 172.21.15.148:0/4028511074 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0014bb0 con 0x56291689af10 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 -- 172.21.15.148:0/4028511074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 msgr2=0x56291689d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291689d070 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f85a0005cd0 tx=0x7f85a0010000 comp rx=0 tx=0).stop 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 -- 172.21.15.148:0/4028511074 shutdown_connections 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168c59d0 0x5629168c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x56291689b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 --2- 172.21.15.148:0/4028511074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291689d070 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 -- 172.21.15.148:0/4028511074 >> 172.21.15.148:0/4028511074 conn(0x562916863100 msgr2=0x5629168e09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 -- 172.21.15.148:0/4028511074 shutdown_connections 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.141+0000 7f85b9228740 1 -- 172.21.15.148:0/4028511074 wait complete. 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.142+0000 7f85b9228740 1 -- start start 2024-04-25T09:57:25.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.142+0000 7f85b9228740 1 Processor -- start 2024-04-25T09:57:25.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56291689af10 0x5629168c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291679d750 con 0x5629168c59d0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629167fadc0 con 0x5629168d75e0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629168dbc60 con 0x56291689af10 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 10 client.0 _ll_register_callbacks cb 0x5629167b5ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42806/0 (socket says 172.21.15.148:42806) 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 -- 172.21.15.148:0/4276041034 learned_addr learned my addr 172.21.15.148:0/4276041034 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b49e7640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85affff640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56291689af10 0x5629168c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 -- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56291689af10 msgr2=0x5629168c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56291689af10 0x5629168c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 -- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 msgr2=0x5629168dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 -- 172.21.15.148:0/4276041034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629164d33f0 con 0x5629168c59d0 2024-04-25T09:57:25.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85af7fe640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168c8c60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001d40 tx=0x7f85a400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde745a67d81c9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f858effd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f858effd640 1 -- 172.21.15.148:0/4276041034 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4018060 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f858effd640 1 -- 172.21.15.148:0/4276041034 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85a40052d0 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.143+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a0007700 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f858effd640 1 -- 172.21.15.148:0/4276041034 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a401e050 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 10 client.6087 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x562916640290 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f858effd640 1 -- 172.21.15.148:0/4276041034 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0007700 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f858effd640 1 -- 172.21.15.148:0/4276041034 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85a4021020 con 0x5629168c59d0 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 10 client.6087 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 10 client.6087 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 20 client.6087 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168f3ec0 0x5629168f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 4 client.6087 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:25.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56291680c260 con 0x5629168f3ec0 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85affff640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168f3ec0 0x5629168f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f85affff640 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168f3ec0 0x5629168f42a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0156546b90611de server_cookie=a60e766166c4928f in_seq=0 out_seq=0 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.144+0000 7f858effd640 10 client.6087 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f858effd640 1 -- 172.21.15.148:0/4276041034 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56291680c260 con 0x5629168f3ec0 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f858effd640 10 client.6087 handle_command_reply: tid=0 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 client.6087 shutdown 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 msgr2=0x5629168c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168c8c60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001d40 tx=0x7f85a400d520 comp rx=0 tx=0).stop 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 shutdown_connections 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168f3ec0 0x5629168f42a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168c8c60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f858effd640 0 client.6087 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 --2- 172.21.15.148:0/4276041034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56291689af10 0x5629168c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 >> 172.21.15.148:0/4276041034 conn(0x562916863100 msgr2=0x5629168e1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 shutdown_connections 2024-04-25T09:57:25.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 1 -- 172.21.15.148:0/4276041034 wait complete. 2024-04-25T09:57:25.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.145+0000 7f85b9228740 20 client.6087 trim_cache size 0 max 16384 2024-04-25T09:57:25.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.160+0000 7f85b9228740 1 Processor -- start 2024-04-25T09:57:25.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.160+0000 7f85b9228740 1 -- start start 2024-04-25T09:57:25.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56291689af10 0x5629168de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b9228740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168f3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x5629168c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b9228740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0200560 con 0x5629168c59d0 2024-04-25T09:57:25.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b01ff790 con 0x56291689af10 2024-04-25T09:57:25.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0155910 con 0x5629168d75e0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42820/0 (socket says 172.21.15.148:42820) 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 -- 172.21.15.148:0/474550421 learned_addr learned my addr 172.21.15.148:0/474550421 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85affff640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56291689af10 0x5629168de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b49e7640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x5629168c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 -- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 msgr2=0x5629168c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x5629168c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 -- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56291689af10 msgr2=0x5629168de6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56291689af10 0x5629168de6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 -- 172.21.15.148:0/474550421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629164ac4b0 con 0x5629168c59d0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85affff640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56291689af10 0x5629168de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85b49e7640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x5629168c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.161+0000 7f85af7fe640 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168f3370 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009010 tx=0x7f85a400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29df254d4abfdeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85957fa640 1 -- 172.21.15.148:0/474550421 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4018070 con 0x5629168c59d0 2024-04-25T09:57:25.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85957fa640 1 -- 172.21.15.148:0/474550421 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400da20 con 0x5629168c59d0 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85957fa640 1 -- 172.21.15.148:0/474550421 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4003790 con 0x5629168c59d0 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 -- 172.21.15.148:0/474550421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 msgr2=0x5629168f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168f3370 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009010 tx=0x7f85a400b000 comp rx=0 tx=0).stop 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 -- 172.21.15.148:0/474550421 shutdown_connections 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168d75e0 0x5629168c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5629168c59d0 0x5629168f3370 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 --2- 172.21.15.148:0/474550421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56291689af10 0x5629168de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 -- 172.21.15.148:0/474550421 >> 172.21.15.148:0/474550421 conn(0x562916863100 msgr2=0x562916891820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 -- 172.21.15.148:0/474550421 shutdown_connections 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.162+0000 7f85b9228740 1 -- 172.21.15.148:0/474550421 wait complete. 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.163+0000 7f85b9228740 1 -- start start 2024-04-25T09:57:25.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.163+0000 7f85b9228740 1 Processor -- start 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291694a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168c59d0 0x5629168dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629168e1800 con 0x56291689af10 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0155f10 con 0x5629168d75e0 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629168fa2d0 con 0x5629168c59d0 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 10 client.0 _ll_register_callbacks cb 0x7f85a0007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:25.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291694a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291694a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42822/0 (socket says 172.21.15.148:42822) 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 -- 172.21.15.148:0/4055188 learned_addr learned my addr 172.21.15.148:0/4055188 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b49e7640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85af7fe640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168c59d0 0x5629168dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 -- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168c59d0 msgr2=0x5629168dc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168c59d0 0x5629168dc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 -- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 msgr2=0x5629168dce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 -- 172.21.15.148:0/4055188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56291689cc90 con 0x56291689af10 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85affff640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291694a7c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f85a0005c70 tx=0x7f85a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d28987533e9168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f858effd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f858effd640 1 -- 172.21.15.148:0/4055188 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0033060 con 0x56291689af10 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f858effd640 1 -- 172.21.15.148:0/4055188 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85a00149b0 con 0x56291689af10 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a400da20 con 0x56291689af10 2024-04-25T09:57:25.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f858effd640 1 -- 172.21.15.148:0/4055188 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a0039050 con 0x56291689af10 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 10 client.6093 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.164+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5629168f2d50 con 0x56291689af10 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f858effd640 1 -- 172.21.15.148:0/4055188 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400da20 con 0x56291689af10 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f858effd640 1 -- 172.21.15.148:0/4055188 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85a003c020 con 0x56291689af10 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85b9228740 10 client.6093 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85b9228740 10 client.6093 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85b9228740 20 client.6093 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85b9228740 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168d6b40 0x5629168df4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85b9228740 4 client.6093 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x5629168065b0 con 0x5629168d6b40 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85af7fe640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168d6b40 0x5629168df4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f858effd640 10 client.6093 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.165+0000 7f85af7fe640 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168d6b40 0x5629168df4e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed9322d0f77318b server_cookie=7fc780c706931b7c in_seq=0 out_seq=0 2024-04-25T09:57:25.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f858effd640 1 -- 172.21.15.148:0/4055188 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5629168065b0 con 0x5629168d6b40 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f858effd640 10 client.6093 handle_command_reply: tid=0 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 client.6093 shutdown 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 msgr2=0x56291694a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291694a7c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f85a0005c70 tx=0x7f85a0000ed0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 shutdown_connections 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5629168d6b40 0x5629168df4e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629168d75e0 0x5629168dce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5629168c59d0 0x5629168dc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f858effd640 0 client.6093 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 --2- 172.21.15.148:0/4055188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291689af10 0x56291694a7c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 >> 172.21.15.148:0/4055188 conn(0x562916863100 msgr2=0x5629168de5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 shutdown_connections 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 1 -- 172.21.15.148:0/4055188 wait complete. 2024-04-25T09:57:25.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.166+0000 7f85b9228740 20 client.6093 trim_cache size 0 max 16384 2024-04-25T09:57:25.176 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:25.176 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:25.176 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "2a70b696-965f-4d73-a267-921ffde3fc84", 2024-04-25T09:57:25.176 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:25.176 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f859c06b4d0 msgr2=0x7f859c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f859c06b4d0 0x7f859c06d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85a00011e0 tx=0x7f85a0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 msgr2=0x7f85b01469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 0x7f85b01469e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e80 tx=0x7f85a8004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 shutdown_connections 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f859c06b4d0 0x7f859c06d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85b015dac0 0x7f85b01469e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d34c0 0x7f85b00b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 --2- 172.21.15.148:0/1057948940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b00d0ab0 0x7f85b00b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 >> 172.21.15.148:0/1057948940 conn(0x7f85b0061c30 msgr2=0x7f85b00c82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 shutdown_connections 2024-04-25T09:57:25.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.167+0000 7f85b644a640 1 -- 172.21.15.148:0/1057948940 wait complete. 2024-04-25T09:57:25.180 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '2a70b696-965f-4d73-a267-921ffde3fc84', 'mode': 'asynchronous'}' 2024-04-25T09:57:25.181 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:57:25.181 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:57:25.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:25.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.276+0000 7efe3d04a640 1 Processor -- start 2024-04-25T09:57:25.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.276+0000 7efe3d04a640 1 -- start start 2024-04-25T09:57:25.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.276+0000 7efe3d04a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe381625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.276+0000 7efe3d04a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380cce50 con 0x7efe3815dc90 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.276+0000 7efe3d04a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38161af0 con 0x7efe380cc8e0 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.276+0000 7efe3d04a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe38161dd0 con 0x7efe380d1f20 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe36d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe3815dc90 0x7efe3815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47228/0 (socket says 172.21.15.148:47228) 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe36d9d640 1 -- 172.21.15.148:0/4225147287 learned_addr learned my addr 172.21.15.148:0/4225147287 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe3759e640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe381625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461025949 0 0) 0x7efe380cce50 con 0x7efe3815dc90 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe20003520 con 0x7efe3815dc90 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032621820 0 0) 0x7efe38161af0 con 0x7efe380cc8e0 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe380cce50 con 0x7efe380cc8e0 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1120132755 0 0) 0x7efe20003520 con 0x7efe3815dc90 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 msgr2=0x7efe381625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe381625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 >> v1:172.21.15.188:6789/0 conn(0x7efe380cc8e0 legacy=0x7efe380d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe381620b0 con 0x7efe3815dc90 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7efe28001670 con 0x7efe3815dc90 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe3759e640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe381625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 >> v1:172.21.15.148:6789/0 conn(0x7efe3815dc90 legacy=0x7efe3815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe20003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe200040b0 0x7efe20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe20004d40 0x7efe200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe38161dd0 con 0x7efe20004d40 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2000bb00 con 0x7efe200040b0 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2000bd00 con 0x7efe380d1f20 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe3759e640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe20004d40 0x7efe200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe36d9d640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe20003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.277+0000 7efe3659c640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe200040b0 0x7efe20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe3759e640 1 -- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 msgr2=0x7efe20003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe3759e640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe380d1f20 0x7efe20003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe3759e640 1 -- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe200040b0 msgr2=0x7efe20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe3759e640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe200040b0 0x7efe20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe3759e640 1 -- 172.21.15.148:0/4225147287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20003520 con 0x7efe20004d40 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe3759e640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe20004d40 0x7efe200051c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e320 tx=0x7efe20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51d19db9ea78281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30009dc0 con 0x7efe20004d40 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30003b90 con 0x7efe20004d40 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.278+0000 7efe35d9b640 1 -- 172.21.15.148:0/4225147287 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30004030 con 0x7efe20004d40 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3d04a640 1 -- 172.21.15.148:0/4225147287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe20004d40 msgr2=0x7efe200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3d04a640 1 --2- 172.21.15.148:0/4225147287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe20004d40 0x7efe200051c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e320 tx=0x7efe20005830 comp rx=0 tx=0).stop 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3759e640 1 -- 172.21.15.148:0/4225147287 reap_dead start 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3d04a640 1 -- 172.21.15.148:0/4225147287 shutdown_connections 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3d04a640 1 -- 172.21.15.148:0/4225147287 >> 172.21.15.148:0/4225147287 conn(0x7efe3806f590 msgr2=0x7efe3806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3d04a640 1 -- 172.21.15.148:0/4225147287 shutdown_connections 2024-04-25T09:57:25.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.279+0000 7efe3d04a640 1 -- 172.21.15.148:0/4225147287 wait complete. 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.280+0000 7efe3d04a640 1 Processor -- start 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.280+0000 7efe3d04a640 1 -- start start 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3d04a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 0x7efe381e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3d04a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe3814ff90 0x7efe381f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3d04a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe380ca0b0 0x7efe381f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3d04a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3815e1f0 con 0x7efe3814fbb0 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3d04a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3815e070 con 0x7efe380ca0b0 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3d04a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe381625d0 con 0x7efe3814ff90 2024-04-25T09:57:25.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe36d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 0x7efe381e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe36d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 0x7efe381e8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42848/0 (socket says 172.21.15.148:42848) 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe36d9d640 1 -- 172.21.15.148:0/102739185 learned_addr learned my addr 172.21.15.148:0/102739185 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3759e640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe380ca0b0 0x7efe381f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.281+0000 7efe3659c640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe3814ff90 0x7efe381f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe36d9d640 1 -- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe3814ff90 msgr2=0x7efe381f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe36d9d640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe3814ff90 0x7efe381f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe36d9d640 1 -- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe380ca0b0 msgr2=0x7efe381f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe36d9d640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe380ca0b0 0x7efe381f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe36d9d640 1 -- 172.21.15.148:0/102739185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe381e8790 con 0x7efe3814fbb0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe36d9d640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 0x7efe381e8250 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efe28004f30 tx=0x7efe280074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800f750 con 0x7efe3814fbb0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28018070 con 0x7efe3814fbb0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe380ca670 con 0x7efe3814fbb0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800fb50 con 0x7efe3814fbb0 2024-04-25T09:57:25.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.282+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe380cac30 con 0x7efe3814fbb0 2024-04-25T09:57:25.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.283+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3806fed0 con 0x7efe3814fbb0 2024-04-25T09:57:25.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.283+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe2800fcf0 con 0x7efe3814fbb0 2024-04-25T09:57:25.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.284+0000 7efe1f7fe640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe2406b4d0 0x7efe2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.284+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe2808a3c0 con 0x7efe3814fbb0 2024-04-25T09:57:25.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.284+0000 7efe3659c640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe2406b4d0 0x7efe2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.284+0000 7efe3659c640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe2406b4d0 0x7efe2406d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe381e7c10 tx=0x7efe2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.288+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe3806fed0 con 0x7efe3814fbb0 2024-04-25T09:57:25.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.492+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe380cf6b0 con 0x7efe3814fbb0 2024-04-25T09:57:25.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.493+0000 7efe1f7fe640 1 -- 172.21.15.148:0/102739185 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7efe28059020 con 0x7efe3814fbb0 2024-04-25T09:57:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:25.496 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:25.497 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:25.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe2406b4d0 msgr2=0x7efe2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe2406b4d0 0x7efe2406d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe381e7c10 tx=0x7efe2c009040 comp rx=0 tx=0).stop 2024-04-25T09:57:25.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 msgr2=0x7efe381e8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 0x7efe381e8250 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7efe28004f30 tx=0x7efe280074f0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 shutdown_connections 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe2406b4d0 0x7efe2406d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe380ca0b0 0x7efe381f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe3814ff90 0x7efe381f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 --2- 172.21.15.148:0/102739185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe3814fbb0 0x7efe381e8250 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 >> 172.21.15.148:0/102739185 conn(0x7efe3806f590 msgr2=0x7efe38160030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 shutdown_connections 2024-04-25T09:57:25.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.495+0000 7efe3d04a640 1 -- 172.21.15.148:0/102739185 wait complete. 2024-04-25T09:57:25.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:57:25.613 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:25.614 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:25.614 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:25.622 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:25.623 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:57:25.623 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:25.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:57:25.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.721+0000 7f851ad48640 1 Processor -- start 2024-04-25T09:57:25.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.721+0000 7f851ad48640 1 -- start start 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f851ad48640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f851ad48640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f851406c9e0 0x7f85140b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f851ad48640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851406cf50 con 0x7f85140b5620 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f851ad48640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140b5fc0 con 0x7f851406c9e0 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f851ad48640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140ce530 con 0x7f851415b7d0 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f851406c9e0 0x7f85140b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f851406c9e0 0x7f85140b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42858/0 (socket says 172.21.15.148:42858) 2024-04-25T09:57:25.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 -- 172.21.15.148:0/2391965139 learned_addr learned my addr 172.21.15.148:0/2391965139 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2391965139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 -- 172.21.15.148:0/2391965139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 msgr2=0x7f85140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 --2- 172.21.15.148:0/2391965139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 -- 172.21.15.148:0/2391965139 >> v1:172.21.15.188:6789/0 conn(0x7f85140b5620 legacy=0x7f851406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.722+0000 7f85192e5640 1 -- 172.21.15.148:0/2391965139 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140ce810 con 0x7f851406c9e0 2024-04-25T09:57:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.723+0000 7f85192e5640 1 --2- 172.21.15.148:0/2391965139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140b5a80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f850c009e50 tx=0x7f850c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf7c421defd7cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.723+0000 7f85137fe640 1 -- 172.21.15.148:0/2391965139 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c036070 con 0x7f851406c9e0 2024-04-25T09:57:25.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.723+0000 7f85137fe640 1 -- 172.21.15.148:0/2391965139 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c031600 con 0x7f851406c9e0 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.723+0000 7f85137fe640 1 -- 172.21.15.148:0/2391965139 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0319d0 con 0x7f851406c9e0 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 -- 172.21.15.148:0/2391965139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 msgr2=0x7f85140b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 --2- 172.21.15.148:0/2391965139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140b5a80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f850c009e50 tx=0x7f850c02a710 comp rx=0 tx=0).stop 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 -- 172.21.15.148:0/2391965139 shutdown_connections 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 --2- 172.21.15.148:0/2391965139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140b5a80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 --2- 172.21.15.148:0/2391965139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 -- 172.21.15.148:0/2391965139 >> 172.21.15.148:0/2391965139 conn(0x7f85140d0800 msgr2=0x7f85140d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 -- 172.21.15.148:0/2391965139 shutdown_connections 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.724+0000 7f851ad48640 1 -- 172.21.15.148:0/2391965139 wait complete. 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.725+0000 7f851ad48640 1 Processor -- start 2024-04-25T09:57:25.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.725+0000 7f851ad48640 1 -- start start 2024-04-25T09:57:25.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f851ad48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f851ad48640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85140b5620 0x7f85140cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f851ad48640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f851ad48640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140cf550 con 0x7f851406c9e0 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f851ad48640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851406d230 con 0x7f85140b5620 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f851ad48640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140cf6d0 con 0x7f851415b7d0 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42862/0 (socket says 172.21.15.148:42862) 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 -- 172.21.15.148:0/2719930796 learned_addr learned my addr 172.21.15.148:0/2719930796 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8513fff640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85140b5620 0x7f85140cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f85192e5640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 -- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 msgr2=0x7f85140cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 -- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85140b5620 msgr2=0x7f85140cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85140b5620 0x7f85140cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8518ae4640 1 -- 172.21.15.148:0/2719930796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140cb1b0 con 0x7f851406c9e0 2024-04-25T09:57:25.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.726+0000 7f8513fff640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85140b5620 0x7f85140cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.727+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140c9f10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f850400e8a0 tx=0x7f850400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.727+0000 7f85117fa640 1 -- 172.21.15.148:0/2719930796 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504017070 con 0x7f851406c9e0 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.727+0000 7f85117fa640 1 -- 172.21.15.148:0/2719930796 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850400c940 con 0x7f851406c9e0 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.727+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85140cd3b0 con 0x7f851406c9e0 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.727+0000 7f85117fa640 1 -- 172.21.15.148:0/2719930796 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850400cd30 con 0x7f851406c9e0 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.727+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85140cd970 con 0x7f851406c9e0 2024-04-25T09:57:25.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.729+0000 7f85117fa640 1 -- 172.21.15.148:0/2719930796 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f850401a070 con 0x7f851406c9e0 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.729+0000 7f85117fa640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f850006b4d0 0x7f850006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.729+0000 7f85117fa640 1 -- 172.21.15.148:0/2719930796 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f850408a220 con 0x7f851406c9e0 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.729+0000 7f8513fff640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f850006b4d0 0x7f850006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.729+0000 7f8513fff640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f850006b4d0 0x7f850006d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85140b9230 tx=0x7f8508009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.730+0000 7f851da45740 1 Processor -- start 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.730+0000 7f851da45740 1 -- start start 2024-04-25T09:57:25.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b4429f60 0x5594b44281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f851da45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b442a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b4427590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f851da45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b4346e60 con 0x5594b43e4130 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b42f0120 con 0x5594b4429f60 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594b4342f00 con 0x5594b440aaf0 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b442a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b442a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42876/0 (socket says 172.21.15.148:42876) 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 -- 172.21.15.148:0/1828690454 learned_addr learned my addr 172.21.15.148:0/1828690454 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 -- 172.21.15.148:0/1828690454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 msgr2=0x5594b4427590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b4427590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 -- 172.21.15.148:0/1828690454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b4429f60 msgr2=0x5594b44281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b4429f60 0x5594b44281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 -- 172.21.15.148:0/1828690454 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594b4127f00 con 0x5594b43e4130 2024-04-25T09:57:25.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f8513fff640 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b442a340 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8508005120 tx=0x7f8508011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ab14addd0a5da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f84fa7fc640 1 -- 172.21.15.148:0/1828690454 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85080035f0 con 0x5594b43e4130 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f84fa7fc640 1 -- 172.21.15.148:0/1828690454 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508003790 con 0x5594b43e4130 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.731+0000 7f84fa7fc640 1 -- 172.21.15.148:0/1828690454 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508003b80 con 0x5594b43e4130 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 -- 172.21.15.148:0/1828690454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 msgr2=0x5594b442a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b442a340 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8508005120 tx=0x7f8508011000 comp rx=0 tx=0).stop 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 -- 172.21.15.148:0/1828690454 shutdown_connections 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b4427590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b442a340 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 --2- 172.21.15.148:0/1828690454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b4429f60 0x5594b44281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 -- 172.21.15.148:0/1828690454 >> 172.21.15.148:0/1828690454 conn(0x5594b43aaf70 msgr2=0x5594b43e1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 -- 172.21.15.148:0/1828690454 shutdown_connections 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 -- 172.21.15.148:0/1828690454 wait complete. 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.732+0000 7f851da45740 1 -- start start 2024-04-25T09:57:25.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 Processor -- start 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b4402080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 0x5594b4424530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b4429760 con 0x5594b440aaf0 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b42e5cd0 con 0x5594b43e4130 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594b4429480 con 0x5594b4429f60 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f851da45740 10 client.0 _ll_register_callbacks cb 0x5594b405b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.733+0000 7f8513fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 0x5594b4424530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 0x5594b4424530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42888/0 (socket says 172.21.15.148:42888) 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 -- 172.21.15.148:0/3007366896 learned_addr learned my addr 172.21.15.148:0/3007366896 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8518ae4640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b4402080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f85192e5640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 -- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 msgr2=0x5594b43e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 -- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 msgr2=0x5594b4402080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b4402080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 -- 172.21.15.148:0/3007366896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594b401b7f0 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f8513fff640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 0x5594b4424530 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f850800e530 tx=0x7f8508000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830fc7e875de14b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f84f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f84f37fe640 1 -- 172.21.15.148:0/3007366896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850802f060 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f84f37fe640 1 -- 172.21.15.148:0/3007366896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8508021660 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8508003790 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f84f37fe640 1 -- 172.21.15.148:0/3007366896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8508035050 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f851da45740 10 client.6111 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5594b4188690 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f84f37fe640 1 -- 172.21.15.148:0/3007366896 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508003790 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.734+0000 7f84f37fe640 1 -- 172.21.15.148:0/3007366896 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8508038020 con 0x5594b440aaf0 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 10 client.6111 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 10 client.6111 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:25.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 20 client.6111 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4425a00 0x5594b43e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 4 client.6111 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594b43543a0 con 0x5594b4425a00 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f8518ae4640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4425a00 0x5594b43e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f84f37fe640 10 client.6111 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f8518ae4640 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4425a00 0x5594b43e2d60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3d5f18316b0cc server_cookie=4b96df968e07f923 in_seq=0 out_seq=0 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f84f37fe640 1 -- 172.21.15.148:0/3007366896 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594b43543a0 con 0x5594b4425a00 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f84f37fe640 10 client.6111 handle_command_reply: tid=0 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.735+0000 7f851da45740 1 client.6111 shutdown 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 msgr2=0x5594b4424530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 0x5594b4424530 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f850800e530 tx=0x7f8508000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 shutdown_connections 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b440aaf0 0x5594b4424530 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4425a00 0x5594b43e2d60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 --2- 172.21.15.148:0/3007366896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b4402080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f84f37fe640 0 client.6111 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 >> 172.21.15.148:0/3007366896 conn(0x5594b43aaf70 msgr2=0x5594b442bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 shutdown_connections 2024-04-25T09:57:25.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 1 -- 172.21.15.148:0/3007366896 wait complete. 2024-04-25T09:57:25.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.736+0000 7f851da45740 20 client.6111 trim_cache size 0 max 16384 2024-04-25T09:57:25.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 Processor -- start 2024-04-25T09:57:25.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 -- start start 2024-04-25T09:57:25.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b440cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b440aaf0 0x5594b4409170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514200480 con 0x5594b43e4130 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141ff620 con 0x5594b440aaf0 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c8090 con 0x5594b4429f60 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.751+0000 7f8518ae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b440cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b440cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42900/0 (socket says 172.21.15.148:42900) 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 -- 172.21.15.148:0/2836980150 learned_addr learned my addr 172.21.15.148:0/2836980150 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 -- 172.21.15.148:0/2836980150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 msgr2=0x5594b43e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 -- 172.21.15.148:0/2836980150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b440aaf0 msgr2=0x5594b4409170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b440aaf0 0x5594b4409170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 -- 172.21.15.148:0/2836980150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594b43e2190 con 0x5594b43e4130 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f8518ae4640 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b440cda0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f850400c520 tx=0x7f8504000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97635bf79af42df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f84f9ffb640 1 -- 172.21.15.148:0/2836980150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504021040 con 0x5594b43e4130 2024-04-25T09:57:25.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f84f9ffb640 1 -- 172.21.15.148:0/2836980150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850405c960 con 0x5594b43e4130 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f84f9ffb640 1 -- 172.21.15.148:0/2836980150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504060f20 con 0x5594b43e4130 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f851da45740 1 -- 172.21.15.148:0/2836980150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 msgr2=0x5594b440cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f851da45740 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b440cda0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f850400c520 tx=0x7f8504000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f851da45740 1 -- 172.21.15.148:0/2836980150 shutdown_connections 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.752+0000 7f851da45740 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b4429f60 0x5594b43e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b440aaf0 0x5594b4409170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 --2- 172.21.15.148:0/2836980150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b43e4130 0x5594b440cda0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 -- 172.21.15.148:0/2836980150 >> 172.21.15.148:0/2836980150 conn(0x5594b43aaf70 msgr2=0x5594b43e2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 -- 172.21.15.148:0/2836980150 shutdown_connections 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 -- 172.21.15.148:0/2836980150 wait complete. 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 -- start start 2024-04-25T09:57:25.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.753+0000 7f851da45740 1 Processor -- start 2024-04-25T09:57:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b444bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b43dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 0x5594b43e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c8690 con 0x5594b4429f60 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b44057c0 con 0x5594b43e4130 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594b43dd890 con 0x5594b440aaf0 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f851da45740 10 client.0 _ll_register_callbacks cb 0x7f8508001440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 0x5594b43e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 0x5594b43e3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42912/0 (socket says 172.21.15.148:42912) 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 -- 172.21.15.148:0/74834927 learned_addr learned my addr 172.21.15.148:0/74834927 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f8518ae4640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b444bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f8513fff640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b43dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 -- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 msgr2=0x5594b43dd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b43dd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 -- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 msgr2=0x5594b444bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b444bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.754+0000 7f85192e5640 1 -- 172.21.15.148:0/74834927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594b3ff48b0 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f85192e5640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 0x5594b43e3260 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f850c009e50 tx=0x7f850c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e048058dafacbb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f84f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f84f37fe640 1 -- 172.21.15.148:0/74834927 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c043070 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f84f37fe640 1 -- 172.21.15.148:0/74834927 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f850c03d930 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f850405c960 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f84f37fe640 1 -- 172.21.15.148:0/74834927 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f850c04e050 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 10 client.6117 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5594b43dd5d0 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f84f37fe640 1 -- 172.21.15.148:0/74834927 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850405c960 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f84f37fe640 1 -- 172.21.15.148:0/74834927 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f850c051020 con 0x5594b4429f60 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 10 client.6117 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 10 client.6117 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 20 client.6117 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:25.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4478a30 0x5594b43e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 4 client.6117 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x5594b434e6f0 con 0x5594b4478a30 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.755+0000 7f8518ae4640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4478a30 0x5594b43e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.756+0000 7f84f37fe640 10 client.6117 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.756+0000 7f8518ae4640 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4478a30 0x5594b43e37a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d26aab0f6abdb4 server_cookie=703f1dbabe76c2ca in_seq=0 out_seq=0 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.756+0000 7f84f37fe640 1 -- 172.21.15.148:0/74834927 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5594b434e6f0 con 0x5594b4478a30 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.756+0000 7f84f37fe640 10 client.6117 handle_command_reply: tid=0 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.756+0000 7f851da45740 1 client.6117 shutdown 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 msgr2=0x5594b43e3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 0x5594b43e3260 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f850c009e50 tx=0x7f850c009c20 comp rx=0 tx=0).stop 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 shutdown_connections 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594b4429f60 0x5594b43e3260 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594b440aaf0 0x5594b43dd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5594b4478a30 0x5594b43e37a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 --2- 172.21.15.148:0/74834927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594b43e4130 0x5594b444bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f84f37fe640 0 client.6117 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 >> 172.21.15.148:0/74834927 conn(0x5594b43aaf70 msgr2=0x5594b4428470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 shutdown_connections 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 1 -- 172.21.15.148:0/74834927 wait complete. 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851da45740 20 client.6117 trim_cache size 0 max 16384 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "02f54114-7426-4e9f-9c72-573d1041069f", 2024-04-25T09:57:25.766 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:25.767 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f850006b4d0 msgr2=0x7f850006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f850006b4d0 0x7f850006d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85140b9230 tx=0x7f8508009040 comp rx=0 tx=0).stop 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 msgr2=0x7f85140c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140c9f10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f850400e8a0 tx=0x7f850400ed60 comp rx=0 tx=0).stop 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 shutdown_connections 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f850006b4d0 0x7f850006d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f851415b7d0 0x7f85140cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85140b5620 0x7f85140cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 --2- 172.21.15.148:0/2719930796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f851406c9e0 0x7f85140c9f10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.757+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 >> 172.21.15.148:0/2719930796 conn(0x7f85140d0800 msgr2=0x7f8514158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.758+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 shutdown_connections 2024-04-25T09:57:25.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.758+0000 7f851ad48640 1 -- 172.21.15.148:0/2719930796 wait complete. 2024-04-25T09:57:25.771 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '02f54114-7426-4e9f-9c72-573d1041069f', 'mode': 'asynchronous'}' 2024-04-25T09:57:25.771 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T09:57:25.771 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:57:25.772 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:25.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.865+0000 7f3c4e4b3640 1 Processor -- start 2024-04-25T09:57:25.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.865+0000 7f3c4e4b3640 1 -- start start 2024-04-25T09:57:25.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4e4b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3c4815dc90 0x7f3c481625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4e4b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c480cc8e0 0x7f3c4815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4e4b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4e4b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c480d21c0 con 0x7f3c4815dc90 2024-04-25T09:57:25.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4e4b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c4815f390 con 0x7f3c480cc8e0 2024-04-25T09:57:25.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4e4b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480cce50 con 0x7f3c480d1810 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3c4815dc90 0x7f3c481625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3c4815dc90 0x7f3c481625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42920/0 (socket says 172.21.15.148:42920) 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 -- 172.21.15.148:0/505452226 learned_addr learned my addr 172.21.15.148:0/505452226 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c477fe640 1 --2- 172.21.15.148:0/505452226 >> v2:172.21.15.188:3300/0 conn(0x7f3c480cc8e0 0x7f3c4815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c4ca50640 1 --2- 172.21.15.148:0/505452226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 -- 172.21.15.148:0/505452226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 msgr2=0x7f3c480d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 --2- 172.21.15.148:0/505452226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 -- 172.21.15.148:0/505452226 >> v2:172.21.15.188:3300/0 conn(0x7f3c480cc8e0 msgr2=0x7f3c4815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 --2- 172.21.15.148:0/505452226 >> v2:172.21.15.188:3300/0 conn(0x7f3c480cc8e0 0x7f3c4815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.866+0000 7f3c47fff640 1 -- 172.21.15.148:0/505452226 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48161af0 con 0x7f3c4815dc90 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.867+0000 7f3c47fff640 1 --2- 172.21.15.148:0/505452226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c4815dc90 0x7f3c481625d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3c480c88a0 tx=0x7f3c3802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75641fb5ab2d0d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.867+0000 7f3c46ffd640 1 -- 172.21.15.148:0/505452226 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38037070 con 0x7f3c4815dc90 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.867+0000 7f3c46ffd640 1 -- 172.21.15.148:0/505452226 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38032440 con 0x7f3c4815dc90 2024-04-25T09:57:25.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.867+0000 7f3c46ffd640 1 -- 172.21.15.148:0/505452226 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38032810 con 0x7f3c4815dc90 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/505452226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c4815dc90 msgr2=0x7f3c481625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/505452226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c4815dc90 0x7f3c481625d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3c480c88a0 tx=0x7f3c3802a140 comp rx=0 tx=0).stop 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/505452226 shutdown_connections 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/505452226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/505452226 >> v2:172.21.15.188:3300/0 conn(0x7f3c480cc8e0 0x7f3c4815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/505452226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c4815dc90 0x7f3c481625d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/505452226 >> 172.21.15.148:0/505452226 conn(0x7f3c4806f590 msgr2=0x7f3c4806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/505452226 shutdown_connections 2024-04-25T09:57:25.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.868+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/505452226 wait complete. 2024-04-25T09:57:25.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.869+0000 7f3c4e4b3640 1 Processor -- start 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.869+0000 7f3c4e4b3640 1 -- start start 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 0x7f3c48155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c4815dc90 0x7f3c481f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480cd130 con 0x7f3c480cc8e0 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c481620b0 con 0x7f3c4815dc90 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4815e070 con 0x7f3c480d1810 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 0x7f3c48155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 0x7f3c48155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42934/0 (socket says 172.21.15.148:42934) 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 -- 172.21.15.148:0/2846106109 learned_addr learned my addr 172.21.15.148:0/2846106109 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 -- 172.21.15.148:0/2846106109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 msgr2=0x7f3c480cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 -- 172.21.15.148:0/2846106109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c4815dc90 msgr2=0x7f3c481f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:25.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c4815dc90 0x7f3c481f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 -- 172.21.15.148:0/2846106109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c481e9700 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c47fff640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 0x7f3c48155ac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38041020 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c380306e0 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c481e99e0 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38037040 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.870+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c480c89f0 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.871+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c48073990 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.872+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c38044020 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.872+0000 7f3c44ff9640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c3406b4d0 0x7f3c3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.872+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3c380aabe0 con 0x7f3c480cc8e0 2024-04-25T09:57:25.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.872+0000 7f3c477fe640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c3406b4d0 0x7f3c3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:25.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.873+0000 7f3c477fe640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c3406b4d0 0x7f3c3406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006fd0 tx=0x7f3c3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:25.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:25.876+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c3804b030 con 0x7f3c480cc8e0 2024-04-25T09:57:26.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.086+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c480cf6b0 con 0x7f3c480cc8e0 2024-04-25T09:57:26.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.087+0000 7f3c44ff9640 1 -- 172.21.15.148:0/2846106109 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f3c380798d0 con 0x7f3c480cc8e0 2024-04-25T09:57:26.090 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:26.090 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:26.091 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:26.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c3406b4d0 msgr2=0x7f3c3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c3406b4d0 0x7f3c3406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006fd0 tx=0x7f3c3c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 msgr2=0x7f3c48155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 0x7f3c48155ac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c380046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 shutdown_connections 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c3406b4d0 0x7f3c3406d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c4815dc90 0x7f3c481f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c480d1810 0x7f3c480cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 --2- 172.21.15.148:0/2846106109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c480cc8e0 0x7f3c48155ac0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 >> 172.21.15.148:0/2846106109 conn(0x7f3c4806f590 msgr2=0x7f3c480cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 shutdown_connections 2024-04-25T09:57:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.089+0000 7f3c4e4b3640 1 -- 172.21.15.148:0/2846106109 wait complete. 2024-04-25T09:57:26.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T09:57:26.207 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:26.207 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:26.207 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:26.216 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:26.216 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:57:26.216 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:26.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:57:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.313+0000 7fd7af7e0640 1 Processor -- start 2024-04-25T09:57:26.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.313+0000 7fd7af7e0640 1 -- start start 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7af7e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7a815dac0 0x7fd7a80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7af7e0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a80d1f30 0x7fd7a80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7af7e0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a80d2420 con 0x7fd7a80d34c0 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7af7e0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a80d3030 con 0x7fd7a80d1f30 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7af7e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a80d2700 con 0x7fd7a815dac0 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7acd7b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7a80d1f30 0x7fd7a80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7add7d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd7a80d34c0 0x7fd7a80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47242/0 (socket says 172.21.15.148:47242) 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7add7d640 1 -- 172.21.15.148:0/2244128794 learned_addr learned my addr 172.21.15.148:0/2244128794 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/2244128794 >> v2:172.21.15.188:3300/0 conn(0x7fd7a815dac0 0x7fd7a80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7acd7b640 1 -- 172.21.15.148:0/2244128794 >> v2:172.21.15.188:3300/0 conn(0x7fd7a815dac0 msgr2=0x7fd7a80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/2244128794 >> v2:172.21.15.188:3300/0 conn(0x7fd7a815dac0 0x7fd7a80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7acd7b640 1 -- 172.21.15.148:0/2244128794 >> v1:172.21.15.148:6789/0 conn(0x7fd7a80d34c0 legacy=0x7fd7a80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7acd7b640 1 -- 172.21.15.148:0/2244128794 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a80d29e0 con 0x7fd7a80d1f30 2024-04-25T09:57:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd79ffff640 1 -- 172.21.15.148:0/2244128794 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374415065 0 0) 0x7fd7a80d2420 con 0x7fd7a80d34c0 2024-04-25T09:57:26.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.314+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/2244128794 >> v2:172.21.15.188:3300/0 conn(0x7fd7a815dac0 0x7fd7a80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.315+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/2244128794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80d03a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80b6520 tx=0x7fd79802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71670575ff94b2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.315+0000 7fd79ffff640 1 -- 172.21.15.148:0/2244128794 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798037070 con 0x7fd7a80d1f30 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.315+0000 7fd79ffff640 1 -- 172.21.15.148:0/2244128794 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798032440 con 0x7fd7a80d1f30 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.315+0000 7fd79ffff640 1 -- 172.21.15.148:0/2244128794 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798032810 con 0x7fd7a80d1f30 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2244128794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 msgr2=0x7fd7a80d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2244128794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80d03a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80b6520 tx=0x7fd79802a230 comp rx=0 tx=0).stop 2024-04-25T09:57:26.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2244128794 shutdown_connections 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2244128794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80d03a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2244128794 >> v2:172.21.15.188:3300/0 conn(0x7fd7a815dac0 0x7fd7a80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2244128794 >> 172.21.15.148:0/2244128794 conn(0x7fd7a8061c30 msgr2=0x7fd7a806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2244128794 shutdown_connections 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.316+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2244128794 wait complete. 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.317+0000 7fd7af7e0640 1 Processor -- start 2024-04-25T09:57:26.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.317+0000 7fd7af7e0640 1 -- start start 2024-04-25T09:57:26.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a80d34c0 0x7fd7a81f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a815dac0 0x7fd7a80ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80b97d0 con 0x7fd7a80d1f30 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a815dea0 con 0x7fd7a80d34c0 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80b9950 con 0x7fd7a815dac0 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42956/0 (socket says 172.21.15.148:42956) 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 -- 172.21.15.148:0/2369622618 learned_addr learned my addr 172.21.15.148:0/2369622618 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a80d34c0 0x7fd7a81f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7add7d640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a815dac0 0x7fd7a80ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 -- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a815dac0 msgr2=0x7fd7a80ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a815dac0 0x7fd7a80ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 -- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a80d34c0 msgr2=0x7fd7a81f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a80d34c0 0x7fd7a81f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 -- 172.21.15.148:0/2369622618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a81f3e20 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7add7d640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a815dac0 0x7fd7a80ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80caaa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c820 tx=0x7fd7a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd79dffb640 1 -- 172.21.15.148:0/2369622618 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4015070 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd79dffb640 1 -- 172.21.15.148:0/2369622618 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a4007a30 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd79dffb640 1 -- 172.21.15.148:0/2369622618 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4007de0 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.318+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a81f4100 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.319+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a80cafe0 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.320+0000 7fd79dffb640 1 -- 172.21.15.148:0/2369622618 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7a4018070 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.320+0000 7fd79dffb640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd78c06b4d0 0x7fd78c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.320+0000 7fd79dffb640 1 -- 172.21.15.148:0/2369622618 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7a408a2c0 con 0x7fd7a80d1f30 2024-04-25T09:57:26.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.320+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd78c06b4d0 0x7fd78c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.321+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd78c06b4d0 0x7fd78c06d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd798034000 tx=0x7fd79802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.321+0000 7fd7b25bf740 1 Processor -- start 2024-04-25T09:57:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 -- start start 2024-04-25T09:57:26.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791df3c0 0x55b1791e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791b1c40 0x55b1791b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b17911aa50 con 0x55b1791defe0 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1790c3d40 con 0x55b1791b1c40 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b179116af0 con 0x55b1791df3c0 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7ad57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7ad57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791e17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42968/0 (socket says 172.21.15.148:42968) 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7ad57c640 1 -- 172.21.15.148:0/3165747338 learned_addr learned my addr 172.21.15.148:0/3165747338 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7add7d640 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791b1c40 0x55b1791b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.322+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791df3c0 0x55b1791e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7ad57c640 1 -- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791df3c0 msgr2=0x55b1791e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791df3c0 0x55b1791e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7ad57c640 1 -- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791b1c40 msgr2=0x55b1791b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791b1c40 0x55b1791b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7ad57c640 1 -- 172.21.15.148:0/3165747338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b178efb690 con 0x55b1791defe0 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7add7d640 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791b1c40 0x55b1791b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791e17f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4058840 tx=0x7fd7a4058970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b302e90a3c31d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd78affd640 1 -- 172.21.15.148:0/3165747338 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401f070 con 0x55b1791defe0 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd78affd640 1 -- 172.21.15.148:0/3165747338 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400ce10 con 0x55b1791defe0 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd78affd640 1 -- 172.21.15.148:0/3165747338 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4056bb0 con 0x55b1791defe0 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3165747338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 msgr2=0x55b1791e17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791e17f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4058840 tx=0x7fd7a4058970 comp rx=0 tx=0).stop 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3165747338 shutdown_connections 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791b1c40 0x55b1791b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791df3c0 0x55b1791e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3165747338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791e17f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.323+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3165747338 >> 172.21.15.148:0/3165747338 conn(0x55b1791b3eb0 msgr2=0x55b1791fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.324+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3165747338 shutdown_connections 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.324+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3165747338 wait complete. 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.324+0000 7fd7b25bf740 1 -- start start 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.324+0000 7fd7b25bf740 1 Processor -- start 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791d5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1791b7060 con 0x55b1791defe0 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1790b9340 con 0x55b1791df3c0 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1791b7340 con 0x55b1791d9320 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7b25bf740 10 client.0 _ll_register_callbacks cb 0x55b1790d6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791d5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42984/0 (socket says 172.21.15.148:42984) 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 -- 172.21.15.148:0/489354833 learned_addr learned my addr 172.21.15.148:0/489354833 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7add7d640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 -- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 msgr2=0x55b1791d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 -- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 msgr2=0x55b1791d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.325+0000 7fd7ad57c640 1 -- 172.21.15.148:0/489354833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b178f5be20 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791d5df0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40585b0 tx=0x7fd7a401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb48cb58e3230c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd783fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd783fff640 1 -- 172.21.15.148:0/489354833 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4087070 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd783fff640 1 -- 172.21.15.148:0/489354833 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7a4059be0 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a400ce10 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd783fff640 1 -- 172.21.15.148:0/489354833 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a4059e00 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 10 client.6138 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55b178deef80 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd783fff640 1 -- 172.21.15.148:0/489354833 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a400ce10 con 0x55b1791defe0 2024-04-25T09:57:26.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd783fff640 1 -- 172.21.15.148:0/489354833 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7a4077300 con 0x55b1791defe0 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 10 client.6138 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 20 client.6138 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b1791fd920 0x55b1791fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 4 client.6138 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b179127f90 con 0x55b1791fd920 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.326+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b1791fd920 0x55b1791fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd783fff640 10 client.6138 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b1791fd920 0x55b1791fdd00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acc7efda4196f09 server_cookie=c466994cc61dcf8f in_seq=0 out_seq=0 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd783fff640 1 -- 172.21.15.148:0/489354833 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b179127f90 con 0x55b1791fd920 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd783fff640 10 client.6138 handle_command_reply: tid=0 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 client.6138 shutdown 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 msgr2=0x55b1791d5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791d5df0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40585b0 tx=0x7fd7a401d000 comp rx=0 tx=0).stop 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 shutdown_connections 2024-04-25T09:57:26.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b1791fd920 0x55b1791fdd00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/489354833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791d5df0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd783fff640 0 client.6138 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 >> 172.21.15.148:0/489354833 conn(0x55b1791b1c40 msgr2=0x55b1791b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 shutdown_connections 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 1 -- 172.21.15.148:0/489354833 wait complete. 2024-04-25T09:57:26.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.327+0000 7fd7b25bf740 20 client.6138 trim_cache size 0 max 16384 2024-04-25T09:57:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.342+0000 7fd7b25bf740 1 Processor -- start 2024-04-25T09:57:26.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.342+0000 7fd7b25bf740 1 -- start start 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a82001d0 con 0x55b1791defe0 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a81ff3d0 con 0x55b1791df3c0 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a81f74d0 con 0x55b1791d9320 2024-04-25T09:57:26.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42988/0 (socket says 172.21.15.148:42988) 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 -- 172.21.15.148:0/1600843919 learned_addr learned my addr 172.21.15.148:0/1600843919 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 -- 172.21.15.148:0/1600843919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 msgr2=0x55b1791d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 -- 172.21.15.148:0/1600843919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 msgr2=0x55b1791b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.343+0000 7fd7acd7b640 1 -- 172.21.15.148:0/1600843919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1791b3ac0 con 0x55b1791defe0 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791ff570 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd798035040 tx=0x7fd79802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336293c64c1e02e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd78a7fc640 1 -- 172.21.15.148:0/1600843919 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798052070 con 0x55b1791defe0 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd78a7fc640 1 -- 172.21.15.148:0/1600843919 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79804e3b0 con 0x55b1791defe0 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd78a7fc640 1 -- 172.21.15.148:0/1600843919 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798037040 con 0x55b1791defe0 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 -- 172.21.15.148:0/1600843919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 msgr2=0x55b1791ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791ff570 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd798035040 tx=0x7fd79802c000 comp rx=0 tx=0).stop 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 -- 172.21.15.148:0/1600843919 shutdown_connections 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791df3c0 0x55b1791b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791defe0 0x55b1791ff570 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/1600843919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791d9320 0x55b1791d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 -- 172.21.15.148:0/1600843919 >> 172.21.15.148:0/1600843919 conn(0x55b1791b1c40 msgr2=0x55b1791f96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 -- 172.21.15.148:0/1600843919 shutdown_connections 2024-04-25T09:57:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.344+0000 7fd7b25bf740 1 -- 172.21.15.148:0/1600843919 wait complete. 2024-04-25T09:57:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.345+0000 7fd7b25bf740 1 -- start start 2024-04-25T09:57:26.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.345+0000 7fd7b25bf740 1 Processor -- start 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791d9320 0x55b1791d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791defe0 0x55b1791b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 0x55b1791b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a81fef00 con 0x55b1791df3c0 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1792683e0 con 0x55b1791d9320 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1791f2d30 con 0x55b1791defe0 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7b25bf740 10 client.0 _ll_register_callbacks cb 0x7fd7a4058fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 0x55b1791b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 0x55b1791b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43002/0 (socket says 172.21.15.148:43002) 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 -- 172.21.15.148:0/3854645257 learned_addr learned my addr 172.21.15.148:0/3854645257 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791d9320 0x55b1791d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7acd7b640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791defe0 0x55b1791b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 -- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791defe0 msgr2=0x55b1791b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791defe0 0x55b1791b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 -- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791d9320 msgr2=0x55b1791d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791d9320 0x55b1791d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.346+0000 7fd7add7d640 1 -- 172.21.15.148:0/3854645257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1791fd3a0 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7add7d640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 0x55b1791b6960 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00016d0 tx=0x7fd7a0001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb616a6983cd31d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 1 -- 172.21.15.148:0/3854645257 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0014730 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 1 -- 172.21.15.148:0/3854645257 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7a00148d0 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd79804e3b0 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 1 -- 172.21.15.148:0/3854645257 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a0022050 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 10 client.6144 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55b1791d6ea0 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 1 -- 172.21.15.148:0/3854645257 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79804e3b0 con 0x55b1791df3c0 2024-04-25T09:57:26.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 1 -- 172.21.15.148:0/3854645257 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd7a0025020 con 0x55b1791df3c0 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 10 client.6144 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 10 client.6144 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 20 client.6144 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b179261830 0x55b179267570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 4 client.6144 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55b1791222e0 con 0x55b179261830 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b179261830 0x55b179267570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd783fff640 10 client.6144 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.347+0000 7fd7ad57c640 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b179261830 0x55b179267570 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1988ff247e7403 server_cookie=2bc6a922cffa3bac in_seq=0 out_seq=0 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd783fff640 1 -- 172.21.15.148:0/3854645257 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b1791222e0 con 0x55b179261830 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd783fff640 10 client.6144 handle_command_reply: tid=0 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 client.6144 shutdown 2024-04-25T09:57:26.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 msgr2=0x55b1791b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 0x55b1791b6960 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00016d0 tx=0x7fd7a0001b90 comp rx=0 tx=0).stop 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 shutdown_connections 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1791df3c0 0x55b1791b6960 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1791defe0 0x55b1791b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b179261830 0x55b179267570 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 --2- 172.21.15.148:0/3854645257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1791d9320 0x55b1791d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 >> 172.21.15.148:0/3854645257 conn(0x55b1791b1c40 msgr2=0x55b1791d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd783fff640 0 client.6144 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 shutdown_connections 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 1 -- 172.21.15.148:0/3854645257 wait complete. 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.348+0000 7fd7b25bf740 20 client.6144 trim_cache size 0 max 16384 2024-04-25T09:57:26.357 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:26.358 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:26.358 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "e0fc9fd7-9ca9-4c5e-8d52-0f1767488e6f", 2024-04-25T09:57:26.358 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:26.358 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd78c06b4d0 msgr2=0x7fd78c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd78c06b4d0 0x7fd78c06d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd798034000 tx=0x7fd79802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 msgr2=0x7fd7a80caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80caaa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c820 tx=0x7fd7a400cce0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 shutdown_connections 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd78c06b4d0 0x7fd78c06d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a815dac0 0x7fd7a80ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a80d34c0 0x7fd7a81f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 --2- 172.21.15.148:0/2369622618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7a80d1f30 0x7fd7a80caaa0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 >> 172.21.15.148:0/2369622618 conn(0x7fd7a8061c30 msgr2=0x7fd7a80c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 shutdown_connections 2024-04-25T09:57:26.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.349+0000 7fd7af7e0640 1 -- 172.21.15.148:0/2369622618 wait complete. 2024-04-25T09:57:26.363 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'e0fc9fd7-9ca9-4c5e-8d52-0f1767488e6f', 'mode': 'asynchronous'}' 2024-04-25T09:57:26.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.364 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T09:57:26.379 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T09:57:26.379 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:26.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.513+0000 7f4685a9e640 1 Processor -- start 2024-04-25T09:57:26.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.513+0000 7f4685a9e640 1 -- start start 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f4685a9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f4685a9e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46800d0ad0 0x7f46800b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f4685a9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46800d1df0 con 0x7f46800b9770 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f4685a9e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800d20d0 con 0x7f46800d0ad0 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f4685a9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d23b0 con 0x7f468015dd30 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f467f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f467ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46800d0ad0 0x7f46800b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f467effd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f46800b9770 0x7f46800d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53058/0 (socket says 172.21.15.148:53058) 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f467f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:35992/0 (socket says 172.21.15.148:35992) 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.514+0000 7f467effd640 1 -- 172.21.15.148:0/1470518852 learned_addr learned my addr 172.21.15.148:0/1470518852 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:26.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467ffff640 1 -- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 msgr2=0x7f46800b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467ffff640 1 --2- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467ffff640 1 -- 172.21.15.148:0/1470518852 >> v1:172.21.15.188:6789/0 conn(0x7f46800b9770 legacy=0x7f46800d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467ffff640 1 -- 172.21.15.148:0/1470518852 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800d2690 con 0x7f46800d0ad0 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467f7fe640 1 --2- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467ffff640 1 --2- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800d0ad0 0x7f46800b6a40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d90 tx=0x7f467802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bebe29127220bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.515+0000 7f467e7fc640 1 -- 172.21.15.148:0/1470518852 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678031600 con 0x7f46800d0ad0 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f467e7fc640 1 -- 172.21.15.148:0/1470518852 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46780317a0 con 0x7f46800d0ad0 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f467e7fc640 1 -- 172.21.15.148:0/1470518852 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678031b70 con 0x7f46800d0ad0 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 -- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800d0ad0 msgr2=0x7f46800b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 --2- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800d0ad0 0x7f46800b6a40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4678009d90 tx=0x7f467802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 -- 172.21.15.148:0/1470518852 shutdown_connections 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 --2- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800d0ad0 0x7f46800b6a40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 --2- 172.21.15.148:0/1470518852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f46800b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 -- 172.21.15.148:0/1470518852 >> 172.21.15.148:0/1470518852 conn(0x7f4680061c30 msgr2=0x7f468006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 -- 172.21.15.148:0/1470518852 shutdown_connections 2024-04-25T09:57:26.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.516+0000 7f4685a9e640 1 -- 172.21.15.148:0/1470518852 wait complete. 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.517+0000 7f4685a9e640 1 Processor -- start 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.517+0000 7f4685a9e640 1 -- start start 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f4685a9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800b9770 0x7f46801f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f4685a9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 0x7f46801593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f4685a9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f468015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f4685a9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d1b90 con 0x7f46800d0ad0 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f4685a9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d1a10 con 0x7f46800b9770 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f4685a9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800d2e90 con 0x7f468015dd30 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f467effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 0x7f46801593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f467effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 0x7f46801593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43004/0 (socket says 172.21.15.148:43004) 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f467effd640 1 -- 172.21.15.148:0/3264149858 learned_addr learned my addr 172.21.15.148:0/3264149858 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f467ffff640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f468015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.518+0000 7f467f7fe640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800b9770 0x7f46801f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467effd640 1 -- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 msgr2=0x7f468015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467effd640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f468015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467effd640 1 -- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800b9770 msgr2=0x7f46801f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467effd640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800b9770 0x7f46801f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467effd640 1 -- 172.21.15.148:0/3264149858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680159da0 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467f7fe640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800b9770 0x7f46801f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f467effd640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 0x7f46801593e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f467400c460 tx=0x7f467400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674017070 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4674005ce0 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4680159ac0 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740072b0 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.519+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46800b3560 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.520+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680073990 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.520+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f467401a070 con 0x7f46800d0ad0 2024-04-25T09:57:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.521+0000 7f4667fff640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b4d0 0x7f466c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.521+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46740895b0 con 0x7f46800d0ad0 2024-04-25T09:57:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.521+0000 7f467f7fe640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b4d0 0x7f466c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.521+0000 7f467f7fe640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b4d0 0x7f466c06d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4670009af0 tx=0x7f4670008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.525+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4674057c50 con 0x7f46800d0ad0 2024-04-25T09:57:26.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.737+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f468015d460 con 0x7f46800d0ad0 2024-04-25T09:57:26.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.737+0000 7f4667fff640 1 -- 172.21.15.148:0/3264149858 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f467405bcf0 con 0x7f46800d0ad0 2024-04-25T09:57:26.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:26.741 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:26.741 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b4d0 msgr2=0x7f466c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b4d0 0x7f466c06d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4670009af0 tx=0x7f4670008040 comp rx=0 tx=0).stop 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 msgr2=0x7f46801593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 0x7f46801593e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f467400c460 tx=0x7f467400c920 comp rx=0 tx=0).stop 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 shutdown_connections 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b4d0 0x7f466c06d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f468015dd30 0x7f468015feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46800d0ad0 0x7f46801593e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 --2- 172.21.15.148:0/3264149858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46800b9770 0x7f46801f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:26.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.739+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 >> 172.21.15.148:0/3264149858 conn(0x7f4680061c30 msgr2=0x7f46800b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:26.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.740+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 shutdown_connections 2024-04-25T09:57:26.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:26.740+0000 7f4685a9e640 1 -- 172.21.15.148:0/3264149858 wait complete. 2024-04-25T09:57:26.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T09:57:26.859 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:26.860 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:26.860 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:26.868 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:26.869 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:26.869 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:26.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:26.884 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:57:26.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T09:57:26.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T09:57:26.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793024 s, 132 MB/s 2024-04-25T09:57:26.948 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:57:26.948 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:27.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.084+0000 7ff22a109640 1 Processor -- start 2024-04-25T09:57:27.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.084+0000 7ff22a109640 1 -- start start 2024-04-25T09:57:27.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff22a109640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240cfec0 0x7ff2240cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff22a109640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff2240ce3c0 0x7ff22406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff22a109640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff224071480 con 0x7ff224154ee0 2024-04-25T09:57:27.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff22a109640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff224072090 con 0x7ff2240ce3c0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff22a109640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff224071230 con 0x7ff2240cfec0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff2240ce3c0 0x7ff22406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff2240ce3c0 0x7ff22406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43014/0 (socket says 172.21.15.148:43014) 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 -- 172.21.15.148:0/3541044267 learned_addr learned my addr 172.21.15.148:0/3541044267 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff222ffd640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240cfec0 0x7ff2240cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 -- 172.21.15.148:0/3541044267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240cfec0 msgr2=0x7ff2240cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240cfec0 0x7ff2240cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 -- 172.21.15.148:0/3541044267 >> v1:172.21.15.188:6789/0 conn(0x7ff224154ee0 legacy=0x7ff2240cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff223fff640 1 -- 172.21.15.148:0/3541044267 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224071760 con 0x7ff2240ce3c0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.085+0000 7ff222ffd640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240cfec0 0x7ff2240cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.086+0000 7ff223fff640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2240ce3c0 0x7ff22406d6f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d150 tx=0x7ff21c02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e93fdd4f1d9964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.086+0000 7ff2227fc640 1 -- 172.21.15.148:0/3541044267 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c009600 con 0x7ff2240ce3c0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.086+0000 7ff2227fc640 1 -- 172.21.15.148:0/3541044267 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c037070 con 0x7ff2240ce3c0 2024-04-25T09:57:27.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.086+0000 7ff2227fc640 1 -- 172.21.15.148:0/3541044267 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0099c0 con 0x7ff2240ce3c0 2024-04-25T09:57:27.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 -- 172.21.15.148:0/3541044267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2240ce3c0 msgr2=0x7ff22406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2240ce3c0 0x7ff22406d6f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d150 tx=0x7ff21c02c930 comp rx=0 tx=0).stop 2024-04-25T09:57:27.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 -- 172.21.15.148:0/3541044267 shutdown_connections 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2240ce3c0 0x7ff22406d6f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 --2- 172.21.15.148:0/3541044267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240cfec0 0x7ff2240cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 -- 172.21.15.148:0/3541044267 >> 172.21.15.148:0/3541044267 conn(0x7ff22406abe0 msgr2=0x7ff22406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 -- 172.21.15.148:0/3541044267 shutdown_connections 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.087+0000 7ff22a109640 1 -- 172.21.15.148:0/3541044267 wait complete. 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.088+0000 7ff22a109640 1 Processor -- start 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.088+0000 7ff22a109640 1 -- start start 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240ce3c0 0x7ff2241f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2240cfec0 0x7ff2241571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 0x7ff2240b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff224071d20 con 0x7ff224154ee0 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240ce800 con 0x7ff2240cfec0 2024-04-25T09:57:27.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241552c0 con 0x7ff2240ce3c0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 0x7ff2240b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 0x7ff2240b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43028/0 (socket says 172.21.15.148:43028) 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 -- 172.21.15.148:0/1492363174 learned_addr learned my addr 172.21.15.148:0/1492363174 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff222ffd640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2240cfec0 0x7ff2241571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff2237fe640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240ce3c0 0x7ff2241f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 -- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240ce3c0 msgr2=0x7ff2241f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240ce3c0 0x7ff2241f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 -- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2240cfec0 msgr2=0x7ff2241571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2240cfec0 0x7ff2241571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 -- 172.21.15.148:0/1492363174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224153cd0 con 0x7ff224154ee0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff223fff640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 0x7ff2240b1110 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d150 tx=0x7ff21c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c009dc0 con 0x7ff224154ee0 2024-04-25T09:57:27.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c047480 con 0x7ff224154ee0 2024-04-25T09:57:27.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff224157ad0 con 0x7ff224154ee0 2024-04-25T09:57:27.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c047870 con 0x7ff224154ee0 2024-04-25T09:57:27.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.089+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff224158090 con 0x7ff224154ee0 2024-04-25T09:57:27.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.090+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff224073990 con 0x7ff224154ee0 2024-04-25T09:57:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.091+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff21c032420 con 0x7ff224154ee0 2024-04-25T09:57:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.091+0000 7ff20bfff640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff21006b500 0x7ff21006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.091+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff21c0ad580 con 0x7ff224154ee0 2024-04-25T09:57:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.091+0000 7ff2237fe640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff21006b500 0x7ff21006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.092+0000 7ff2237fe640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff21006b500 0x7ff21006d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff218007fd0 tx=0x7ff218009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.095+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff224153cd0 con 0x7ff224154ee0 2024-04-25T09:57:27.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.305+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff22406e9a0 con 0x7ff224154ee0 2024-04-25T09:57:27.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.306+0000 7ff20bfff640 1 -- 172.21.15.148:0/1492363174 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4950 (secure 0 0 0) 0x7ff21c07c300 con 0x7ff224154ee0 2024-04-25T09:57:27.310 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:27.310 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:12.322346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:27.310 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 23 2024-04-25T09:57:27.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff21006b500 msgr2=0x7ff21006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff21006b500 0x7ff21006d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff218007fd0 tx=0x7ff218009040 comp rx=0 tx=0).stop 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 msgr2=0x7ff2240b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 0x7ff2240b1110 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00d150 tx=0x7ff21c004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 shutdown_connections 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff21006b500 0x7ff21006d9c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff224154ee0 0x7ff2240b1110 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2240cfec0 0x7ff2241571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 --2- 172.21.15.148:0/1492363174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2240ce3c0 0x7ff2241f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 >> 172.21.15.148:0/1492363174 conn(0x7ff22406abe0 msgr2=0x7ff22415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 shutdown_connections 2024-04-25T09:57:27.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.308+0000 7ff22a109640 1 -- 172.21.15.148:0/1492363174 wait complete. 2024-04-25T09:57:27.323 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:57:27.429 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:27.429 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:27.429 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:27.438 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:27.439 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:57:27.439 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:57:27.439 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T09:57:27.439 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:57:27.439 DEBUG:teuthology.orchestra.run.smithi148:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T09:57:27.439 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:57:27.504 INFO:teuthology.orchestra.run.smithi148.stdout:1099511630186 2024-04-25T09:57:27.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:57:27.513 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:27.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:57:27.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.649+0000 7fcb7ed10640 1 Processor -- start 2024-04-25T09:57:27.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.649+0000 7fcb7ed10640 1 -- start start 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7ed10640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb780c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7ed10640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb780cd6f0 0x7fcb780cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7ed10640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb780c6940 con 0x7fcb780cdbf0 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7ed10640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb780b0020 con 0x7fcb780cd6f0 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7ed10640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780cb5a0 con 0x7fcb7815a9f0 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7d2ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb780cd6f0 0x7fcb780cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7d2ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb780cd6f0 0x7fcb780cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43038/0 (socket says 172.21.15.148:43038) 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.650+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/4077626419 learned_addr learned my addr 172.21.15.148:0/4077626419 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.651+0000 7fcb7caac640 1 --2- 172.21.15.148:0/4077626419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb780c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.651+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/4077626419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 msgr2=0x7fcb780c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.651+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/4077626419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb780c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.651+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/4077626419 >> v1:172.21.15.188:6789/0 conn(0x7fcb780cdbf0 legacy=0x7fcb780ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.651+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/4077626419 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7815add0 con 0x7fcb780cd6f0 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.651+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/4077626419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb780cba30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d150 tx=0x7fcb7002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d245da0586c53e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb777fe640 1 -- 172.21.15.148:0/4077626419 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70009600 con 0x7fcb780cd6f0 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb777fe640 1 -- 172.21.15.148:0/4077626419 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70037070 con 0x7fcb780cd6f0 2024-04-25T09:57:27.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb777fe640 1 -- 172.21.15.148:0/4077626419 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700099c0 con 0x7fcb780cd6f0 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 -- 172.21.15.148:0/4077626419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 msgr2=0x7fcb780cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/4077626419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb780cba30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d150 tx=0x7fcb7002c930 comp rx=0 tx=0).stop 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 -- 172.21.15.148:0/4077626419 shutdown_connections 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/4077626419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb780cba30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/4077626419 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb780c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 -- 172.21.15.148:0/4077626419 >> 172.21.15.148:0/4077626419 conn(0x7fcb780b2390 msgr2=0x7fcb780b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 -- 172.21.15.148:0/4077626419 shutdown_connections 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.652+0000 7fcb7ed10640 1 -- 172.21.15.148:0/4077626419 wait complete. 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.653+0000 7fcb7ed10640 1 Processor -- start 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.653+0000 7fcb7ed10640 1 -- start start 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7ed10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb781f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7ed10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb780cdbf0 0x7fcb7806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7ed10640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb7806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7ed10640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780cb230 con 0x7fcb780cd6f0 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7ed10640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780caf50 con 0x7fcb780cdbf0 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7ed10640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780ce070 con 0x7fcb7815a9f0 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7caac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb781f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7caac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb781f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43050/0 (socket says 172.21.15.148:43050) 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb7caac640 1 -- 172.21.15.148:0/3172817414 learned_addr learned my addr 172.21.15.148:0/3172817414 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.654+0000 7fcb77fff640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb780cdbf0 0x7fcb7806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb7806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7caac640 1 -- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 msgr2=0x7fcb7806eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb7806eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7caac640 1 -- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb780cdbf0 msgr2=0x7fcb7806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb780cdbf0 0x7fcb7806e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7caac640 1 -- 172.21.15.148:0/3172817414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780c9790 con 0x7fcb780cd6f0 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb77fff640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb780cdbf0 0x7fcb7806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb781f31c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800d5f0 tx=0x7fcb6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb757fa640 1 -- 172.21.15.148:0/3172817414 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680125f0 con 0x7fcb780cd6f0 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb757fa640 1 -- 172.21.15.148:0/3172817414 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68019070 con 0x7fcb780cd6f0 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb757fa640 1 -- 172.21.15.148:0/3172817414 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680129d0 con 0x7fcb780cd6f0 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb78154180 con 0x7fcb780cd6f0 2024-04-25T09:57:27.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.655+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb78154740 con 0x7fcb780cd6f0 2024-04-25T09:57:27.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.657+0000 7fcb757fa640 1 -- 172.21.15.148:0/3172817414 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb68012b70 con 0x7fcb780cd6f0 2024-04-25T09:57:27.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.657+0000 7fcb757fa640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb6406b4d0 0x7fcb6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.657+0000 7fcb757fa640 1 -- 172.21.15.148:0/3172817414 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb6808a7b0 con 0x7fcb780cd6f0 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.657+0000 7fcb77fff640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb6406b4d0 0x7fcb6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.658+0000 7fcb77fff640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb6406b4d0 0x7fcb6406d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007fd0 tx=0x7fcb6c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.658+0000 7fcb81a10740 1 Processor -- start 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.658+0000 7fcb81a10740 1 -- start start 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce10267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 0x555ce0ff9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb81a10740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0f5fe00 con 0x555ce0ff6700 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0f08d20 con 0x555ce1023e80 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ce0f5bea0 con 0x555ce1023aa0 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 0x555ce0ff9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 0x555ce0ff9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43052/0 (socket says 172.21.15.148:43052) 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/1476998977 learned_addr learned my addr 172.21.15.148:0/1476998977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7caac640 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb77fff640 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce10267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 msgr2=0x555ce10262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 msgr2=0x555ce10267f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce10267f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/1476998977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ce0f2adf0 con 0x555ce0ff6700 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.659+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 0x555ce0ff9f10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d150 tx=0x7fcb70031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f288d38d1fd218ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb5e7fc640 1 -- 172.21.15.148:0/1476998977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70032420 con 0x555ce0ff6700 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb5e7fc640 1 -- 172.21.15.148:0/1476998977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70037040 con 0x555ce0ff6700 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb5e7fc640 1 -- 172.21.15.148:0/1476998977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700327e0 con 0x555ce0ff6700 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 -- 172.21.15.148:0/1476998977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 msgr2=0x555ce0ff9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 0x555ce0ff9f10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d150 tx=0x7fcb70031000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 -- 172.21.15.148:0/1476998977 shutdown_connections 2024-04-25T09:57:27.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce0ff6700 0x555ce0ff9f10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce10267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 --2- 172.21.15.148:0/1476998977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 -- 172.21.15.148:0/1476998977 >> 172.21.15.148:0/1476998977 conn(0x555ce0ff8970 msgr2=0x555ce10412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 -- 172.21.15.148:0/1476998977 shutdown_connections 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.660+0000 7fcb81a10740 1 -- 172.21.15.148:0/1476998977 wait complete. 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.661+0000 7fcb81a10740 1 -- start start 2024-04-25T09:57:27.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.661+0000 7fcb81a10740 1 Processor -- start 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1044750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023e80 0x555ce1029920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1028810 0x555ce101c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0ffbb20 con 0x555ce1023aa0 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce0efe590 con 0x555ce1028810 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ce0ffbe00 con 0x555ce1023e80 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 10 client.0 _ll_register_callbacks cb 0x555ce0f0ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1044750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1044750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43064/0 (socket says 172.21.15.148:43064) 2024-04-25T09:57:27.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 -- 172.21.15.148:0/160954511 learned_addr learned my addr 172.21.15.148:0/160954511 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb77fff640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023e80 0x555ce1029920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1028810 0x555ce101c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 -- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023e80 msgr2=0x555ce1029920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023e80 0x555ce1029920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 -- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1028810 msgr2=0x555ce101c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1028810 0x555ce101c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 -- 172.21.15.148:0/160954511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ce0c743f0 con 0x555ce1023aa0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1028810 0x555ce101c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb7caac640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1044750 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcb68055200 tx=0x7fcb68058da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139736840a9ba3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb577fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb577fe640 1 -- 172.21.15.148:0/160954511 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68087060 con 0x555ce1023aa0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb577fe640 1 -- 172.21.15.148:0/160954511 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb6805f0c0 con 0x555ce1023aa0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb70037040 con 0x555ce1023aa0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb577fe640 1 -- 172.21.15.148:0/160954511 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb68070030 con 0x555ce1023aa0 2024-04-25T09:57:27.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 10 client.6165 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.662+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x555ce0d3e7d0 con 0x555ce1023aa0 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb577fe640 1 -- 172.21.15.148:0/160954511 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb70037040 con 0x555ce1023aa0 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb577fe640 1 -- 172.21.15.148:0/160954511 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb68013070 con 0x555ce1023aa0 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb81a10740 10 client.6165 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb81a10740 10 client.6165 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb81a10740 20 client.6165 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb81a10740 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce10428b0 0x555ce1042ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb81a10740 4 client.6165 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ce0f6d340 con 0x555ce10428b0 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb77fff640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce10428b0 0x555ce1042ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb577fe640 10 client.6165 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.663+0000 7fcb77fff640 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce10428b0 0x555ce1042ce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ee9c2b52e62bce server_cookie=ae7d89102d6f8ecc in_seq=0 out_seq=0 2024-04-25T09:57:27.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb577fe640 1 -- 172.21.15.148:0/160954511 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ce0f6d340 con 0x555ce10428b0 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb577fe640 10 client.6165 handle_command_reply: tid=0 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 client.6165 shutdown 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 msgr2=0x555ce1044750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1044750 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcb68055200 tx=0x7fcb68058da0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 shutdown_connections 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce10428b0 0x555ce1042ce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1028810 0x555ce101c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023e80 0x555ce1029920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 --2- 172.21.15.148:0/160954511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1044750 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb577fe640 0 client.6165 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 >> 172.21.15.148:0/160954511 conn(0x555ce0ff6700 msgr2=0x555ce1022e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 shutdown_connections 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 1 -- 172.21.15.148:0/160954511 wait complete. 2024-04-25T09:57:27.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.664+0000 7fcb81a10740 20 client.6165 trim_cache size 0 max 16384 2024-04-25T09:57:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 Processor -- start 2024-04-25T09:57:27.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 -- start start 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1027180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce1026270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1028810 0x555ce0ff2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb78200410 con 0x555ce1023aa0 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb781ff570 con 0x555ce1023e80 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb78155980 con 0x555ce1028810 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb7caac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1027180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb7caac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1027180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43076/0 (socket says 172.21.15.148:43076) 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.676+0000 7fcb7caac640 1 -- 172.21.15.148:0/783557534 learned_addr learned my addr 172.21.15.148:0/783557534 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb77fff640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce1026270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7caac640 1 -- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1028810 msgr2=0x555ce0ff2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1028810 0x555ce0ff2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7caac640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1028810 0x555ce0ff2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7caac640 1 -- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 msgr2=0x555ce1026270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7caac640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce1026270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7caac640 1 -- 172.21.15.148:0/783557534 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ce103e760 con 0x555ce1023aa0 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb77fff640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce1026270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1028810 0x555ce0ff2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb7caac640 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1027180 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcb68019fd0 tx=0x7fcb6805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c5e6a4903c37a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb5dffb640 1 -- 172.21.15.148:0/783557534 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68055040 con 0x555ce1023aa0 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb5dffb640 1 -- 172.21.15.148:0/783557534 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6807b020 con 0x555ce1023aa0 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb5dffb640 1 -- 172.21.15.148:0/783557534 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680570a0 con 0x555ce1023aa0 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 -- 172.21.15.148:0/783557534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 msgr2=0x555ce1027180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1027180 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcb68019fd0 tx=0x7fcb6805c000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 -- 172.21.15.148:0/783557534 shutdown_connections 2024-04-25T09:57:27.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1028810 0x555ce0ff2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce1026270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 --2- 172.21.15.148:0/783557534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1023aa0 0x555ce1027180 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.677+0000 7fcb81a10740 1 -- 172.21.15.148:0/783557534 >> 172.21.15.148:0/783557534 conn(0x555ce0ff6700 msgr2=0x555ce1044560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.678+0000 7fcb81a10740 1 -- 172.21.15.148:0/783557534 shutdown_connections 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.678+0000 7fcb81a10740 1 -- 172.21.15.148:0/783557534 wait complete. 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.678+0000 7fcb81a10740 1 -- start start 2024-04-25T09:57:27.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.678+0000 7fcb81a10740 1 Processor -- start 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce0ff57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 0x555ce1028cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce101b2e0 con 0x555ce1028810 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb78155f80 con 0x555ce1023e80 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ce0ee5d70 con 0x555ce1023aa0 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb81a10740 10 client.0 _ll_register_callbacks cb 0x7fcb7000d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 0x555ce1028cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 0x555ce1028cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43092/0 (socket says 172.21.15.148:43092) 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/3585438146 learned_addr learned my addr 172.21.15.148:0/3585438146 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb77fff640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce0ff57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 msgr2=0x555ce10abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 msgr2=0x555ce0ff57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce0ff57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 -- 172.21.15.148:0/3585438146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ce1065210 con 0x555ce1028810 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb77fff640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce0ff57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb7d2ad640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 0x555ce1028cc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d240 tx=0x7fcb70004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e47bb74d5d09b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.679+0000 7fcb577fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb577fe640 1 -- 172.21.15.148:0/3585438146 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70003ba0 con 0x555ce1028810 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb577fe640 1 -- 172.21.15.148:0/3585438146 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb70003d40 con 0x555ce1028810 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb6807b020 con 0x555ce1028810 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb577fe640 1 -- 172.21.15.148:0/3585438146 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb70058050 con 0x555ce1028810 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 10 client.6171 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x555ce0ff5d10 con 0x555ce1028810 2024-04-25T09:57:27.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb577fe640 1 -- 172.21.15.148:0/3585438146 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6807b020 con 0x555ce1028810 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb577fe640 1 -- 172.21.15.148:0/3585438146 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb7005b020 con 0x555ce1028810 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 10 client.6171 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 10 client.6171 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 20 client.6171 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce1029200 0x555ce1022ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 4 client.6171 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ce1045490 con 0x555ce1029200 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce1029200 0x555ce1022ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb577fe640 10 client.6171 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.680+0000 7fcb7caac640 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce1029200 0x555ce1022ce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8d62df2478bc23 server_cookie=81a72bda05c58b8d in_seq=0 out_seq=0 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb577fe640 1 -- 172.21.15.148:0/3585438146 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ce1045490 con 0x555ce1029200 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb577fe640 10 client.6171 handle_command_reply: tid=0 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 client.6171 shutdown 2024-04-25T09:57:27.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 msgr2=0x555ce1028cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 0x555ce1028cc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcb7000d240 tx=0x7fcb70004ab0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 shutdown_connections 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ce1028810 0x555ce1028cc0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x555ce1029200 0x555ce1022ce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ce1023e80 0x555ce0ff57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 --2- 172.21.15.148:0/3585438146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555ce1023aa0 0x555ce10abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb577fe640 0 client.6171 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 >> 172.21.15.148:0/3585438146 conn(0x555ce0ff6700 msgr2=0x555ce0ff1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 shutdown_connections 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 1 -- 172.21.15.148:0/3585438146 wait complete. 2024-04-25T09:57:27.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb81a10740 20 client.6171 trim_cache size 0 max 16384 2024-04-25T09:57:27.692 INFO:teuthology.orchestra.run.smithi148.stdout:[] 2024-04-25T09:57:27.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb6406b4d0 msgr2=0x7fcb6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb6406b4d0 0x7fcb6406d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c007fd0 tx=0x7fcb6c009040 comp rx=0 tx=0).stop 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 msgr2=0x7fcb781f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.681+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb781f31c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800d5f0 tx=0x7fcb6800dab0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 shutdown_connections 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb6406b4d0 0x7fcb6406d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb7815a9f0 0x7fcb7806eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb780cdbf0 0x7fcb7806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 --2- 172.21.15.148:0/3172817414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb780cd6f0 0x7fcb781f31c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 >> 172.21.15.148:0/3172817414 conn(0x7fcb780b2390 msgr2=0x7fcb7806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 shutdown_connections 2024-04-25T09:57:27.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.682+0000 7fcb7ed10640 1 -- 172.21.15.148:0/3172817414 wait complete. 2024-04-25T09:57:27.695 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096a.00000000 parent 2024-04-25T09:57:27.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 Processor -- start 2024-04-25T09:57:27.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 -- start start 2024-04-25T09:57:27.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fa82d40 0x56052fa3aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 0x56052fb020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fb0d010 0x56052fb0da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052fa671e0 con 0x56052fb0cc30 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052f9dfdf0 con 0x56052fb0d010 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.734+0000 7f6fc738af40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56052fa67480 con 0x56052fa82d40 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 0x56052fb020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 0x56052fb020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43120/0 (socket says 172.21.15.148:43120) 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 -- 172.21.15.148:0/1180401294 learned_addr learned my addr 172.21.15.148:0/1180401294 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 -- 172.21.15.148:0/1180401294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fa82d40 msgr2=0x56052fa3aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fa82d40 0x56052fa3aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 -- 172.21.15.148:0/1180401294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fb0d010 msgr2=0x56052fb0da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fb0d010 0x56052fb0da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 -- 172.21.15.148:0/1180401294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56052f965c50 con 0x56052fb0cc30 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbffff640 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 0x56052fb020d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000a470 tx=0x7f6fb000a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763b3b94fc9e3997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbeffd640 1 -- 172.21.15.148:0/1180401294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000b930 con 0x56052fb0cc30 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbeffd640 1 -- 172.21.15.148:0/1180401294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb000f070 con 0x56052fb0cc30 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fbeffd640 1 -- 172.21.15.148:0/1180401294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000bd30 con 0x56052fb0cc30 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fc738af40 1 -- 172.21.15.148:0/1180401294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 msgr2=0x56052fb020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.735+0000 7f6fc738af40 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 0x56052fb020d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000a470 tx=0x7f6fb000a930 comp rx=0 tx=0).stop 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 -- 172.21.15.148:0/1180401294 shutdown_connections 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fb0d010 0x56052fb0da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0cc30 0x56052fb020d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 --2- 172.21.15.148:0/1180401294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fa82d40 0x56052fa3aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 -- 172.21.15.148:0/1180401294 >> 172.21.15.148:0/1180401294 conn(0x56052fa379b0 msgr2=0x56052fa38620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 -- 172.21.15.148:0/1180401294 shutdown_connections 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 -- 172.21.15.148:0/1180401294 wait complete. 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 Processor -- start 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.736+0000 7f6fc738af40 1 -- start start 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc738af40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fa82d40 0x56052fb0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc738af40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fb0cc30 0x56052fbaa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc738af40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0d010 0x56052fb08ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc738af40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052fb08b30 con 0x56052fb0d010 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc738af40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052fb030f0 con 0x56052fa82d40 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc738af40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56052fb03390 con 0x56052fb0cc30 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0d010 0x56052fb08ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0d010 0x56052fb08ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43132/0 (socket says 172.21.15.148:43132) 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 -- 172.21.15.148:0/3304273414 learned_addr learned my addr 172.21.15.148:0/3304273414 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fbffff640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fb0cc30 0x56052fbaa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc4e8b640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fa82d40 0x56052fb0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 -- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fb0cc30 msgr2=0x56052fbaa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fb0cc30 0x56052fbaa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 -- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fa82d40 msgr2=0x56052fb0ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fa82d40 0x56052fb0ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 -- 172.21.15.148:0/3304273414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56052fb0b180 con 0x56052fb0d010 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc4e8b640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56052fa82d40 0x56052fb0ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fbffff640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56052fb0cc30 0x56052fbaa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fc568c640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0d010 0x56052fb08ea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0002170 tx=0x7f6fc000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.737+0000 7f6fbdffb640 1 -- 172.21.15.148:0/3304273414 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00062d0 con 0x56052fb0d010 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.738+0000 7f6fbdffb640 1 -- 172.21.15.148:0/3304273414 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc0003450 con 0x56052fb0d010 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.738+0000 7f6fbdffb640 1 -- 172.21.15.148:0/3304273414 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0003880 con 0x56052fb0d010 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.738+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56052fbaa6e0 con 0x56052fb0d010 2024-04-25T09:57:27.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.738+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56052fa7ca70 con 0x56052fb0d010 2024-04-25T09:57:27.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.739+0000 7f6fbdffb640 1 -- 172.21.15.148:0/3304273414 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fc0010070 con 0x56052fb0d010 2024-04-25T09:57:27.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.739+0000 7f6fbdffb640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fa806a050 0x7f6fa806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.739+0000 7f6fc4e8b640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fa806a050 0x7f6fa806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.739+0000 7f6fbdffb640 1 -- 172.21.15.148:0/3304273414 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fc0088b00 con 0x56052fb0d010 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.740+0000 7f6fc738af40 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x56052fb093e0 0x56052fb025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.740+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 --> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] -- osd_op(unknown.0.0:1 8.11 8:88e5eea5:::1000000096a.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v9 -- 0x56052fa7ec50 con 0x56052fb093e0 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.740+0000 7f6fc4e8b640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fa806a050 0x7f6fa806c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56052fa24820 tx=0x7f6fb804e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.740+0000 7f6fbffff640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x56052fb093e0 0x56052fb025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.740+0000 7f6fbffff640 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x56052fb093e0 0x56052fb025e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.741+0000 7f6fbffff640 1 -- 172.21.15.148:0/3304273414 <== osd.5 v2:172.21.15.188:6808/3945194105 1 ==== osd_op_reply(1 1000000096a.00000000 [rmxattr] v55'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6fb000cb60 con 0x56052fb093e0 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.741+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x56052fb093e0 msgr2=0x56052fb025e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.741+0000 7f6fc738af40 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x56052fb093e0 0x56052fb025e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fa806a050 msgr2=0x7f6fa806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fa806a050 0x7f6fa806c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56052fa24820 tx=0x7f6fb804e000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0d010 msgr2=0x56052fb08ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 --2- 172.21.15.148:0/3304273414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56052fb0d010 0x56052fb08ea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0002170 tx=0x7f6fc000b4b0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc568c640 1 -- 172.21.15.148:0/3304273414 reap_dead start 2024-04-25T09:57:27.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 shutdown_connections 2024-04-25T09:57:27.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 >> 172.21.15.148:0/3304273414 conn(0x56052fa379b0 msgr2=0x56052fb015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 shutdown_connections 2024-04-25T09:57:27.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.742+0000 7f6fc738af40 1 -- 172.21.15.148:0/3304273414 wait complete. 2024-04-25T09:57:27.749 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T09:57:27.749 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:27.749 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T09:57:27.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.885+0000 7f777fb3b640 1 Processor -- start 2024-04-25T09:57:27.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.885+0000 7f777fb3b640 1 -- start start 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777fb3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7778155f80 0x7f777806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777fb3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77780ce900 0x7f77780b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777fb3b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77780b4d80 con 0x7f777806fa00 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777fb3b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77780b5390 con 0x7f7778155f80 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777fb3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77780d07f0 con 0x7f77780ce900 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777d8d7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7778155f80 0x7f777806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777d8d7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7778155f80 0x7f777806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43138/0 (socket says 172.21.15.148:43138) 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777d0d6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f777806fa00 0x7f77780ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47270/0 (socket says 172.21.15.148:47270) 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777d8d7640 1 -- 172.21.15.148:0/2220578402 learned_addr learned my addr 172.21.15.148:0/2220578402 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.886+0000 7f777e0d8640 1 --2- 172.21.15.148:0/2220578402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77780ce900 0x7f77780b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777d8d7640 1 -- 172.21.15.148:0/2220578402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77780ce900 msgr2=0x7f77780b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777d8d7640 1 --2- 172.21.15.148:0/2220578402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77780ce900 0x7f77780b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777d8d7640 1 -- 172.21.15.148:0/2220578402 >> v1:172.21.15.148:6789/0 conn(0x7f777806fa00 legacy=0x7f77780ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777c8d5640 1 -- 172.21.15.148:0/2220578402 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161514132 0 0) 0x7f77780b4d80 con 0x7f777806fa00 2024-04-25T09:57:27.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777d8d7640 1 -- 172.21.15.148:0/2220578402 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77780cef70 con 0x7f7778155f80 2024-04-25T09:57:27.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777d8d7640 1 --2- 172.21.15.148:0/2220578402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777806f4c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f77780b5510 tx=0x7f777402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af2e43834ff4f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777c8d5640 1 -- 172.21.15.148:0/2220578402 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774036070 con 0x7f7778155f80 2024-04-25T09:57:27.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777c8d5640 1 -- 172.21.15.148:0/2220578402 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774031600 con 0x7f7778155f80 2024-04-25T09:57:27.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.887+0000 7f777c8d5640 1 -- 172.21.15.148:0/2220578402 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740319d0 con 0x7f7778155f80 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 -- 172.21.15.148:0/2220578402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 msgr2=0x7f777806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 --2- 172.21.15.148:0/2220578402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777806f4c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f77780b5510 tx=0x7f777402a710 comp rx=0 tx=0).stop 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 -- 172.21.15.148:0/2220578402 shutdown_connections 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 --2- 172.21.15.148:0/2220578402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77780ce900 0x7f77780b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 --2- 172.21.15.148:0/2220578402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777806f4c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 -- 172.21.15.148:0/2220578402 >> 172.21.15.148:0/2220578402 conn(0x7f7778061600 msgr2=0x7f77780619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 -- 172.21.15.148:0/2220578402 shutdown_connections 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.888+0000 7f777fb3b640 1 -- 172.21.15.148:0/2220578402 wait complete. 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.889+0000 7f777fb3b640 1 Processor -- start 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.889+0000 7f777fb3b640 1 -- start start 2024-04-25T09:57:27.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777fb3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777806fa00 0x7f777813ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777fb3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77780ce900 0x7f777814d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777fb3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777815c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777fb3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77780cf250 con 0x7f7778155f80 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777fb3b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778156360 con 0x7f77780ce900 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777fb3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77780cf3d0 con 0x7f777806fa00 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777815c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43150/0 (socket says 172.21.15.148:43150) 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 -- 172.21.15.148:0/1220763257 learned_addr learned my addr 172.21.15.148:0/1220763257 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777806fa00 0x7f777813ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777d0d6640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77780ce900 0x7f777814d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 -- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777806fa00 msgr2=0x7f777813ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777806fa00 0x7f777813ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 -- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77780ce900 msgr2=0x7f777814d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77780ce900 0x7f777814d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 -- 172.21.15.148:0/1220763257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777813cf70 con 0x7f7778155f80 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777806fa00 0x7f777813ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777d0d6640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77780ce900 0x7f777814d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.890+0000 7f777e0d8640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777815c470 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f777000c7f0 tx=0x7f777000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.891+0000 7f776e7fc640 1 -- 172.21.15.148:0/1220763257 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770015070 con 0x7f7778155f80 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.891+0000 7f776e7fc640 1 -- 172.21.15.148:0/1220763257 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770004220 con 0x7f7778155f80 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.891+0000 7f776e7fc640 1 -- 172.21.15.148:0/1220763257 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770007620 con 0x7f7778155f80 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.891+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7778150850 con 0x7f7778155f80 2024-04-25T09:57:27.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.891+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77781510f0 con 0x7f7778155f80 2024-04-25T09:57:27.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.892+0000 7f776e7fc640 1 -- 172.21.15.148:0/1220763257 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77700079d0 con 0x7f7778155f80 2024-04-25T09:57:27.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.892+0000 7f776e7fc640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f775c06b4d0 0x7f775c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.893+0000 7f776e7fc640 1 -- 172.21.15.148:0/1220763257 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777008a470 con 0x7f7778155f80 2024-04-25T09:57:27.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.893+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f775c06b4d0 0x7f775c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.893+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f775c06b4d0 0x7f775c06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7774033000 tx=0x7f7774009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 Processor -- start 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 -- start start 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x5642820957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x56428207cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x5642820527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564281fb5ee0 con 0x564282075360 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564281f5ec50 con 0x56428207d140 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564281fb1f80 con 0x564282054360 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f777d8d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x5642820957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f777d8d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x5642820957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43164/0 (socket says 172.21.15.148:43164) 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.894+0000 7f777d8d7640 1 -- 172.21.15.148:0/2974713010 learned_addr learned my addr 172.21.15.148:0/2974713010 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777d8d7640 1 -- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 msgr2=0x56428207cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777d8d7640 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x56428207cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777d8d7640 1 -- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 msgr2=0x5642820527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777e0d8640 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x5642820527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777d8d7640 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x5642820527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777d8d7640 1 -- 172.21.15.148:0/2974713010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564281d96c00 con 0x564282075360 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777e0d8640 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x5642820527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f777d8d7640 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x5642820957d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7774009f10 tx=0x7f777402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d8453b7df3c775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f775b7fe640 1 -- 172.21.15.148:0/2974713010 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774036040 con 0x564282075360 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f775b7fe640 1 -- 172.21.15.148:0/2974713010 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777404b070 con 0x564282075360 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.895+0000 7f775b7fe640 1 -- 172.21.15.148:0/2974713010 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777403cd40 con 0x564282075360 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 -- 172.21.15.148:0/2974713010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 msgr2=0x5642820957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x5642820957d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7774009f10 tx=0x7f777402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 -- 172.21.15.148:0/2974713010 shutdown_connections 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x5642820527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x56428207cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 --2- 172.21.15.148:0/2974713010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x5642820957d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 -- 172.21.15.148:0/2974713010 >> 172.21.15.148:0/2974713010 conn(0x564282019810 msgr2=0x564282097c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 -- 172.21.15.148:0/2974713010 shutdown_connections 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 -- 172.21.15.148:0/2974713010 wait complete. 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 -- start start 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.896+0000 7f778283f740 1 Processor -- start 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x564282091420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x56428207c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x56428207c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282052330 con 0x564282075360 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564281f54340 con 0x56428207d140 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56428204d1a0 con 0x564282054360 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f778283f740 10 client.0 _ll_register_callbacks cb 0x564281cc9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:27.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f777d0d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x56428207c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f777d0d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x56428207c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43178/0 (socket says 172.21.15.148:43178) 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f777d0d6640 1 -- 172.21.15.148:0/1820199527 learned_addr learned my addr 172.21.15.148:0/1820199527 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.897+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x564282091420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777e0d8640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x56428207c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d0d6640 1 -- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 msgr2=0x564282091420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d0d6640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x564282091420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d0d6640 1 -- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 msgr2=0x56428207c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d0d6640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x56428207c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d0d6640 1 -- 172.21.15.148:0/1820199527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564281c8a4f0 con 0x564282075360 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x564282091420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777e0d8640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x56428207c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f777d0d6640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x56428207c480 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7778155a90 tx=0x7f776800e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f906ecc28731f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f7758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:27.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f7758ff9640 1 -- 172.21.15.148:0/1820199527 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768019060 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f7758ff9640 1 -- 172.21.15.148:0/1820199527 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f77680070d0 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f777404b070 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f7758ff9640 1 -- 172.21.15.148:0/1820199527 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f776801f050 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 10 client.6198 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x564281df7390 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f7758ff9640 1 -- 172.21.15.148:0/1820199527 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777404b070 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f7758ff9640 1 -- 172.21.15.148:0/1820199527 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7768022020 con 0x564282075360 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 10 client.6198 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 10 client.6198 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 20 client.6198 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428208f160 0x56428208f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 4 client.6198 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.898+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564281fc3420 con 0x56428208f160 2024-04-25T09:57:27.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428208f160 0x56428208f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f777d8d7640 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428208f160 0x56428208f560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abffc36b990d30e server_cookie=57aedbcfd30b6e8 in_seq=0 out_seq=0 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f7758ff9640 10 client.6198 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f7758ff9640 1 -- 172.21.15.148:0/1820199527 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564281fc3420 con 0x56428208f160 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f7758ff9640 10 client.6198 handle_command_reply: tid=0 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 client.6198 shutdown 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 msgr2=0x56428207c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x56428207c480 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7778155a90 tx=0x7f776800e5f0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 shutdown_connections 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428208f160 0x56428208f560 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x56428207c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282075360 0x56428207c480 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f7758ff9640 0 client.6198 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 --2- 172.21.15.148:0/1820199527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282054360 0x564282091420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.899+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 >> 172.21.15.148:0/1820199527 conn(0x564282019810 msgr2=0x564282097fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.900+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 shutdown_connections 2024-04-25T09:57:27.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.900+0000 7f778283f740 1 -- 172.21.15.148:0/1820199527 wait complete. 2024-04-25T09:57:27.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.900+0000 7f778283f740 20 client.6198 trim_cache size 0 max 16384 2024-04-25T09:57:27.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 Processor -- start 2024-04-25T09:57:27.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 -- start start 2024-04-25T09:57:27.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564282054360 0x564282090ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428207c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 0x56428207c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77781fe040 con 0x56428207d140 2024-04-25T09:57:27.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77781fd240 con 0x564282054360 2024-04-25T09:57:27.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777813a8a0 con 0x564282075360 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.915+0000 7f777e0d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 0x56428207c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 0x56428207c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43192/0 (socket says 172.21.15.148:43192) 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 -- 172.21.15.148:0/3846714706 learned_addr learned my addr 172.21.15.148:0/3846714706 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777d8d7640 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564282054360 0x564282090ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777d0d6640 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428207c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 -- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 msgr2=0x56428207c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428207c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 -- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564282054360 msgr2=0x564282090ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564282054360 0x564282090ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 -- 172.21.15.148:0/3846714706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564281c63670 con 0x56428207d140 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777d0d6640 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428207c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f777e0d8640 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 0x56428207c900 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f77700887b0 tx=0x7f7770058640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cdae156389ba1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f775affd640 1 -- 172.21.15.148:0/3846714706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770083070 con 0x56428207d140 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f775affd640 1 -- 172.21.15.148:0/3846714706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770056cd0 con 0x56428207d140 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.916+0000 7f775affd640 1 -- 172.21.15.148:0/3846714706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777001e050 con 0x56428207d140 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 -- 172.21.15.148:0/3846714706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 msgr2=0x56428207c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 0x56428207c900 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f77700887b0 tx=0x7f7770058640 comp rx=0 tx=0).stop 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 -- 172.21.15.148:0/3846714706 shutdown_connections 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56428207d140 0x56428207c900 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428207c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 --2- 172.21.15.148:0/3846714706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564282054360 0x564282090ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 -- 172.21.15.148:0/3846714706 >> 172.21.15.148:0/3846714706 conn(0x564282019810 msgr2=0x56428207d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 -- 172.21.15.148:0/3846714706 shutdown_connections 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 -- 172.21.15.148:0/3846714706 wait complete. 2024-04-25T09:57:27.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 -- start start 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.917+0000 7f778283f740 1 Processor -- start 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 0x56428207f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428208e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x564282101740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564282090210 con 0x564282054360 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777813aea0 con 0x56428207d140 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564282080680 con 0x564282075360 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f778283f740 10 client.0 _ll_register_callbacks cb 0x7f7774034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 0x56428207f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 0x56428207f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43208/0 (socket says 172.21.15.148:43208) 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 -- 172.21.15.148:0/3714754505 learned_addr learned my addr 172.21.15.148:0/3714754505 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777e0d8640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x564282101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d0d6640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428208e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 -- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 msgr2=0x56428208e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428208e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 -- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 msgr2=0x564282101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x564282101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d8d7640 1 -- 172.21.15.148:0/3714754505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642820801b0 con 0x564282054360 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777e0d8640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x564282101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.918+0000 7f777d0d6640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428208e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f777d8d7640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 0x56428207f100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7774000f80 tx=0x7f777402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ff95e1f396bc13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f7758ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f7758ff9640 1 -- 172.21.15.148:0/3714754505 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774036040 con 0x564282054360 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f7758ff9640 1 -- 172.21.15.148:0/3714754505 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7774048870 con 0x564282054360 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7770056cd0 con 0x564282054360 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f7758ff9640 1 -- 172.21.15.148:0/3714754505 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7774048a70 con 0x564282054360 2024-04-25T09:57:27.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 10 client.6204 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56428209b0d0 con 0x564282054360 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f7758ff9640 1 -- 172.21.15.148:0/3714754505 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770056cd0 con 0x564282054360 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f7758ff9640 1 -- 172.21.15.148:0/3714754505 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7774008050 con 0x564282054360 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 10 client.6204 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 10 client.6204 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 20 client.6204 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428204c160 0x56428204c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 4 client.6204 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x564281fbd770 con 0x56428204c160 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.919+0000 7f777d0d6640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428204c160 0x56428204c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f7758ff9640 10 client.6204 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f777d0d6640 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428204c160 0x56428204c570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e921036b484e7f32 server_cookie=43896713d5781202 in_seq=0 out_seq=0 2024-04-25T09:57:27.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f7758ff9640 1 -- 172.21.15.148:0/3714754505 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564281fbd770 con 0x56428204c160 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f7758ff9640 10 client.6204 handle_command_reply: tid=0 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 client.6204 shutdown 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 msgr2=0x56428207f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 0x56428207f100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7774000f80 tx=0x7f777402c000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 shutdown_connections 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56428204c160 0x56428204c570 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56428207d140 0x564282101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f7758ff9640 0 client.6204 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564282075360 0x56428208e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 --2- 172.21.15.148:0/3714754505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564282054360 0x56428207f100 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 >> 172.21.15.148:0/3714754505 conn(0x564282019810 msgr2=0x564282090ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 shutdown_connections 2024-04-25T09:57:27.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 1 -- 172.21.15.148:0/3714754505 wait complete. 2024-04-25T09:57:27.932 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f4887411640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096a [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v450 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5123=0-4194304@1} caps={5123=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5123 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5625f01f9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096a:[<0x10000000001/new_file_0 v450>,<0x10000000000/ceph-qa-suite v38>,<0x1/scrub_test_path v9>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:57:27.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.920+0000 7f778283f740 20 client.6204 trim_cache size 0 max 16384 2024-04-25T09:57:27.932 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:27.932 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:27.932 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "9f4e8eb9-9ea0-4b91-ad20-524707db4427", 2024-04-25T09:57:27.933 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:27.933 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:27.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f775c06b4d0 msgr2=0x7f775c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f775c06b4d0 0x7f775c06d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7774033000 tx=0x7f7774009000 comp rx=0 tx=0).stop 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 msgr2=0x7f777815c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777815c470 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f777000c7f0 tx=0x7f777000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 shutdown_connections 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f775c06b4d0 0x7f775c06d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7778155f80 0x7f777815c470 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77780ce900 0x7f777814d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 --2- 172.21.15.148:0/1220763257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777806fa00 0x7f777813ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 >> 172.21.15.148:0/1220763257 conn(0x7f7778061600 msgr2=0x7f77780d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 shutdown_connections 2024-04-25T09:57:27.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:27.921+0000 7f777fb3b640 1 -- 172.21.15.148:0/1220763257 wait complete. 2024-04-25T09:57:27.935 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '9f4e8eb9-9ea0-4b91-ad20-524707db4427', 'mode': 'asynchronous'}' 2024-04-25T09:57:27.935 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:57:27.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:57:28.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.032+0000 7f52cc8b3640 1 Processor -- start 2024-04-25T09:57:28.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.033+0000 7f52cc8b3640 1 -- start start 2024-04-25T09:57:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.033+0000 7f52cc8b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52c80d0a00 0x7f52c8162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.033+0000 7f52cc8b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52c80cd710 0x7f52c80caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52cc8b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c815e3a0 con 0x7f52c80d0de0 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52cc8b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c80cdc70 con 0x7f52c80d0a00 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52cc8b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c80cdf50 con 0x7f52c80cd710 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52c80d0a00 0x7f52c8162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c5d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f52c80d0de0 0x7f52c80cd000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47276/0 (socket says 172.21.15.148:47276) 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c5d9b640 1 -- 172.21.15.148:0/1747827155 learned_addr learned my addr 172.21.15.148:0/1747827155 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c659c640 1 -- 172.21.15.148:0/1747827155 >> v2:172.21.15.188:3300/0 conn(0x7f52c80cd710 msgr2=0x7f52c80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c659c640 1 --2- 172.21.15.148:0/1747827155 >> v2:172.21.15.188:3300/0 conn(0x7f52c80cd710 0x7f52c80caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c659c640 1 -- 172.21.15.148:0/1747827155 >> v1:172.21.15.148:6789/0 conn(0x7f52c80d0de0 legacy=0x7f52c80cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c659c640 1 -- 172.21.15.148:0/1747827155 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c8161660 con 0x7f52c80d0a00 2024-04-25T09:57:28.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.034+0000 7f52c559a640 1 -- 172.21.15.148:0/1747827155 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784436789 0 0) 0x7f52c815e3a0 con 0x7f52c80d0de0 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52c659c640 1 --2- 172.21.15.148:0/1747827155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c8162140 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f52c815f1c0 tx=0x7f52b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc93c3e43ff7ae3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52c559a640 1 -- 172.21.15.148:0/1747827155 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8037070 con 0x7f52c80d0a00 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52c559a640 1 -- 172.21.15.148:0/1747827155 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8032440 con 0x7f52c80d0a00 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52c559a640 1 -- 172.21.15.148:0/1747827155 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b802ae60 con 0x7f52c80d0a00 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 -- 172.21.15.148:0/1747827155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 msgr2=0x7f52c8162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/1747827155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c8162140 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f52c815f1c0 tx=0x7f52b802a110 comp rx=0 tx=0).stop 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 -- 172.21.15.148:0/1747827155 shutdown_connections 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/1747827155 >> v2:172.21.15.188:3300/0 conn(0x7f52c80cd710 0x7f52c80caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/1747827155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c8162140 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 -- 172.21.15.148:0/1747827155 >> 172.21.15.148:0/1747827155 conn(0x7f52c80d17c0 msgr2=0x7f52c80d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 -- 172.21.15.148:0/1747827155 shutdown_connections 2024-04-25T09:57:28.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.035+0000 7f52cc8b3640 1 -- 172.21.15.148:0/1747827155 wait complete. 2024-04-25T09:57:28.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 Processor -- start 2024-04-25T09:57:28.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 -- start start 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52c80cd710 0x7f52c80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c80b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52c80d0de0 0x7f52c815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80d1260 con 0x7f52c80d0a00 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c8161c20 con 0x7f52c80cd710 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.037+0000 7f52cc8b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c81475d0 con 0x7f52c80d0de0 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c80b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c80b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43236/0 (socket says 172.21.15.148:43236) 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 -- 172.21.15.148:0/83644627 learned_addr learned my addr 172.21.15.148:0/83644627 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 -- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52c80d0de0 msgr2=0x7f52c815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c659c640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52c80cd710 0x7f52c80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52c80d0de0 0x7f52c815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 -- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52c80cd710 msgr2=0x7f52c80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52c80cd710 0x7f52c80b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 -- 172.21.15.148:0/83644627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c80b6000 con 0x7f52c80d0a00 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c659c640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52c80cd710 0x7f52c80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c80b70a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c430 tx=0x7f52bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52b6ffd640 1 -- 172.21.15.148:0/83644627 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0074e0 con 0x7f52c80d0a00 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52b6ffd640 1 -- 172.21.15.148:0/83644627 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc0047e0 con 0x7f52c80d0a00 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c80b62e0 con 0x7f52c80d0a00 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52b6ffd640 1 -- 172.21.15.148:0/83644627 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004bd0 con 0x7f52c80d0a00 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.038+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c80b68a0 con 0x7f52c80d0a00 2024-04-25T09:57:28.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.040+0000 7f52b6ffd640 1 -- 172.21.15.148:0/83644627 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52bc011070 con 0x7f52c80d0a00 2024-04-25T09:57:28.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.040+0000 7f52b6ffd640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52a806b4d0 0x7f52a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.040+0000 7f52b6ffd640 1 -- 172.21.15.148:0/83644627 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52bc089fd0 con 0x7f52c80d0a00 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.040+0000 7f52c659c640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52a806b4d0 0x7f52a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.041+0000 7f52c659c640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52a806b4d0 0x7f52a806d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52b8034000 tx=0x7f52b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.041+0000 7f52cf5b1740 1 Processor -- start 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.041+0000 7f52cf5b1740 1 -- start start 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 0x55670a169320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166ef0 0x55670a169860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a139770 0x55670a13cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670a0a2da0 con 0x55670a166b10 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670a04bb80 con 0x55670a166ef0 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55670a09ee40 con 0x55670a139770 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 0x55670a169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 0x55670a169320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43248/0 (socket says 172.21.15.148:43248) 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 -- 172.21.15.148:0/225909464 learned_addr learned my addr 172.21.15.148:0/225909464 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166ef0 0x55670a169860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c6d9d640 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a139770 0x55670a13cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 -- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a139770 msgr2=0x55670a13cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a139770 0x55670a13cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 -- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166ef0 msgr2=0x55670a169860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166ef0 0x55670a169860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.042+0000 7f52c659c640 1 -- 172.21.15.148:0/225909464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55670a055f10 con 0x55670a166b10 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52c659c640 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 0x55670a169320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009860 tx=0x7f52b802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6886bf78aaefb5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52a7fff640 1 -- 172.21.15.148:0/225909464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b804e070 con 0x55670a166b10 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52a7fff640 1 -- 172.21.15.148:0/225909464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8049470 con 0x55670a166b10 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52a7fff640 1 -- 172.21.15.148:0/225909464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8037040 con 0x55670a166b10 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 -- 172.21.15.148:0/225909464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 msgr2=0x55670a169320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 0x55670a169320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009860 tx=0x7f52b802c000 comp rx=0 tx=0).stop 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 -- 172.21.15.148:0/225909464 shutdown_connections 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a139770 0x55670a13cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166ef0 0x55670a169860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/225909464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166b10 0x55670a169320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.043+0000 7f52cf5b1740 1 -- 172.21.15.148:0/225909464 >> 172.21.15.148:0/225909464 conn(0x55670a13b9e0 msgr2=0x55670a184320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.044+0000 7f52cf5b1740 1 -- 172.21.15.148:0/225909464 shutdown_connections 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.044+0000 7f52cf5b1740 1 -- 172.21.15.148:0/225909464 wait complete. 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.044+0000 7f52cf5b1740 1 -- start start 2024-04-25T09:57:28.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.044+0000 7f52cf5b1740 1 Processor -- start 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a185450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166ef0 0x55670a186dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 0x55670a168c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670a13eb90 con 0x55670a181510 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670a0414b0 con 0x55670a166b10 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55670a13ee70 con 0x55670a166ef0 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52cf5b1740 10 client.0 _ll_register_callbacks cb 0x55670a081180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 0x55670a168c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 0x55670a168c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43258/0 (socket says 172.21.15.148:43258) 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 -- 172.21.15.148:0/1667680050 learned_addr learned my addr 172.21.15.148:0/1667680050 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c659c640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a185450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166ef0 0x55670a186dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 -- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166ef0 msgr2=0x55670a186dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166ef0 0x55670a186dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 -- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 msgr2=0x55670a185450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a185450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.045+0000 7f52c6d9d640 1 -- 172.21.15.148:0/1667680050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556709e817d0 con 0x55670a181510 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52c6d9d640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 0x55670a168c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f52c000ca70 tx=0x7f52c000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1fd0d909152e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52a57fa640 1 -- 172.21.15.148:0/1667680050 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c001a060 con 0x55670a181510 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52a57fa640 1 -- 172.21.15.148:0/1667680050 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52c0007950 con 0x55670a181510 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52b8049470 con 0x55670a181510 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52a57fa640 1 -- 172.21.15.148:0/1667680050 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52c0020050 con 0x55670a181510 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 10 client.6219 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:28.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x556709db73f0 con 0x55670a181510 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52a57fa640 1 -- 172.21.15.148:0/1667680050 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8049470 con 0x55670a181510 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52a57fa640 1 -- 172.21.15.148:0/1667680050 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52c0023020 con 0x55670a181510 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 10 client.6219 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 10 client.6219 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 20 client.6219 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a17ee20 0x55670a17f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 4 client.6219 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55670a0b02e0 con 0x55670a17ee20 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.046+0000 7f52c659c640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a17ee20 0x55670a17f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52a57fa640 10 client.6219 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52c659c640 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a17ee20 0x55670a17f200 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5469e0ea14b41ed3 server_cookie=4214a2e3a95412f in_seq=0 out_seq=0 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52a57fa640 1 -- 172.21.15.148:0/1667680050 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55670a0b02e0 con 0x55670a17ee20 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52a57fa640 10 client.6219 handle_command_reply: tid=0 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 client.6219 shutdown 2024-04-25T09:57:28.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 msgr2=0x55670a168c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 0x55670a168c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f52c000ca70 tx=0x7f52c000cf30 comp rx=0 tx=0).stop 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 shutdown_connections 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a181510 0x55670a168c80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166ef0 0x55670a186dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a17ee20 0x55670a17f200 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/1667680050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a185450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52a57fa640 0 client.6219 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 >> 172.21.15.148:0/1667680050 conn(0x55670a139770 msgr2=0x55670a13e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 shutdown_connections 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.047+0000 7f52cf5b1740 1 -- 172.21.15.148:0/1667680050 wait complete. 2024-04-25T09:57:28.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.048+0000 7f52cf5b1740 20 client.6219 trim_cache size 0 max 16384 2024-04-25T09:57:28.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.062+0000 7f52cf5b1740 1 Processor -- start 2024-04-25T09:57:28.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.062+0000 7f52cf5b1740 1 -- start start 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166b10 0x55670a140600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a181510 0x55670a13d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c8200340 con 0x55670a166ef0 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80b5380 con 0x55670a181510 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80b4780 con 0x55670a166b10 2024-04-25T09:57:28.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a13eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43272/0 (socket says 172.21.15.148:43272) 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2644976488 learned_addr learned my addr 172.21.15.148:0/2644976488 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c659c640 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166b10 0x55670a140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c6d9d640 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a181510 0x55670a13d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166b10 msgr2=0x55670a140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166b10 0x55670a140600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a181510 msgr2=0x55670a13d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a181510 0x55670a13d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.063+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2644976488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556709d770f0 con 0x55670a166ef0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a13eda0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f52bc001120 tx=0x7f52bc058df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f90cd615919e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52a77fe640 1 -- 172.21.15.148:0/2644976488 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0568c0 con 0x55670a166ef0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52a77fe640 1 -- 172.21.15.148:0/2644976488 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc070030 con 0x55670a166ef0 2024-04-25T09:57:28.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52a77fe640 1 -- 172.21.15.148:0/2644976488 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc01ad90 con 0x55670a166ef0 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2644976488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 msgr2=0x55670a13eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a13eda0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f52bc001120 tx=0x7f52bc058df0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2644976488 shutdown_connections 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a181510 0x55670a13d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a13eda0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2644976488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a166b10 0x55670a140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2644976488 >> 172.21.15.148:0/2644976488 conn(0x55670a139770 msgr2=0x55670a194c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2644976488 shutdown_connections 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.064+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2644976488 wait complete. 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.065+0000 7f52cf5b1740 1 -- start start 2024-04-25T09:57:28.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.065+0000 7f52cf5b1740 1 Processor -- start 2024-04-25T09:57:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a1ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a138f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a181510 0x55670a1672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55670a15dd70 con 0x55670a166ef0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80b4d80 con 0x55670a166b10 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55670a13f530 con 0x55670a181510 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 10 client.0 _ll_register_callbacks cb 0x7f52b8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43288/0 (socket says 172.21.15.148:43288) 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2462544132 learned_addr learned my addr 172.21.15.148:0/2462544132 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c659c640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a1ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c6d9d640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a181510 0x55670a1672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a181510 msgr2=0x55670a1672d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a181510 0x55670a1672d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 msgr2=0x55670a1ee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a1ee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 -- 172.21.15.148:0/2462544132 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55670a164c10 con 0x55670a166ef0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52c5d9b640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a138f10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00cdf0 tx=0x7f52bc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29b5f43c2ac6c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52a57fa640 1 -- 172.21.15.148:0/2462544132 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc056980 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52a57fa640 1 -- 172.21.15.148:0/2462544132 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52bc057a10 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.066+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52bc070030 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52a57fa640 1 -- 172.21.15.148:0/2462544132 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52bc057c50 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 10 client.6225 fetch_fsmap learned FSMap version 23 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55670a13d820 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52a57fa640 1 -- 172.21.15.148:0/2462544132 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc070030 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52a57fa640 1 -- 172.21.15.148:0/2462544132 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52bc054310 con 0x55670a166ef0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 10 client.6225 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 10 client.6225 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 20 client.6225 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a167810 0x55670a17a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 4 client.6225 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55670a0aa630 con 0x55670a167810 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52c659c640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a167810 0x55670a17a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:28.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52a57fa640 10 client.6225 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52c659c640 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a167810 0x55670a17a250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5fa544d93ad500 server_cookie=586b843693bd7c8c in_seq=0 out_seq=0 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52a57fa640 1 -- 172.21.15.148:0/2462544132 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+455 (crc 0 0 0) 0x55670a0aa630 con 0x55670a167810 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52a57fa640 10 client.6225 handle_command_reply: tid=0 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.067+0000 7f52cf5b1740 1 client.6225 shutdown 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 msgr2=0x55670a138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a138f10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00cdf0 tx=0x7f52bc000ed0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 shutdown_connections 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55670a181510 0x55670a1672d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55670a166ef0 0x55670a138f10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55670a167810 0x55670a17a250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 --2- 172.21.15.148:0/2462544132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55670a166b10 0x55670a1ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 >> 172.21.15.148:0/2462544132 conn(0x55670a139770 msgr2=0x55670a13e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52a57fa640 0 client.6225 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 shutdown_connections 2024-04-25T09:57:28.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 1 -- 172.21.15.148:0/2462544132 wait complete. 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cf5b1740 20 client.6225 trim_cache size 0 max 16384 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "9f4e8eb9-9ea0-4b91-ad20-524707db4427": { 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "9f4e8eb9-9ea0-4b91-ad20-524707db4427", 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "" 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:28.078 INFO:teuthology.orchestra.run.smithi148.stdout: "e0fc9fd7-9ca9-4c5e-8d52-0f1767488e6f": { 2024-04-25T09:57:28.079 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-04-25T09:57:28.079 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "e0fc9fd7-9ca9-4c5e-8d52-0f1767488e6f", 2024-04-25T09:57:28.079 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "" 2024-04-25T09:57:28.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:28.079 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:28.079 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52a806b4d0 msgr2=0x7f52a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52a806b4d0 0x7f52a806d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52b8034000 tx=0x7f52b802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 msgr2=0x7f52c80b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c80b70a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c430 tx=0x7f52bc00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 shutdown_connections 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52a806b4d0 0x7f52a806d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52c80d0de0 0x7f52c815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52c80d0a00 0x7f52c80b70a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 --2- 172.21.15.148:0/83644627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52c80cd710 0x7f52c80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:28.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 >> 172.21.15.148:0/83644627 conn(0x7f52c80d17c0 msgr2=0x7f52c80cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:28.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 shutdown_connections 2024-04-25T09:57:28.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:28.068+0000 7f52cc8b3640 1 -- 172.21.15.148:0/83644627 wait complete. 2024-04-25T09:57:31.143 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:57:31.140+0000 7fb15b777640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:57:33.083 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:57:33.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:57:33.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.191+0000 7fb5f53c8640 1 Processor -- start 2024-04-25T09:57:33.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.191+0000 7fb5f53c8640 1 -- start start 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5f53c8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5f015dc20 0x7fb5f00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5f53c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5f53c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f015e000 con 0x7fb5f0161710 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5f53c8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f00d1de0 con 0x7fb5f015dc20 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5f53c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0161c60 con 0x7fb5f00d32f0 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5ee7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb5f0161710 0x7fb5f006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47288/0 (socket says 172.21.15.148:47288) 2024-04-25T09:57:33.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/666841086 learned_addr learned my addr 172.21.15.148:0/666841086 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5edffb640 1 -- 172.21.15.148:0/666841086 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809499075 0 0) 0x7fb5f015e000 con 0x7fb5f0161710 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/666841086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/666841086 >> v2:172.21.15.188:3300/0 conn(0x7fb5f015dc20 0x7fb5f00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.192+0000 7fb5edffb640 1 -- 172.21.15.148:0/666841086 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d8003520 con 0x7fb5f0161710 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.193+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/666841086 >> v2:172.21.15.188:3300/0 conn(0x7fb5f015dc20 msgr2=0x7fb5f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.193+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/666841086 >> v2:172.21.15.188:3300/0 conn(0x7fb5f015dc20 0x7fb5f00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.193+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/666841086 >> v1:172.21.15.148:6789/0 conn(0x7fb5f0161710 legacy=0x7fb5f006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.193+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/666841086 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0161f40 con 0x7fb5f00d32f0 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.193+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/666841086 >> v2:172.21.15.188:3300/0 conn(0x7fb5f015dc20 0x7fb5f00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:33.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.194+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/666841086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f00cebb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009d90 tx=0x7fb5e802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad92e11c55da61e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.196+0000 7fb5edffb640 1 -- 172.21.15.148:0/666841086 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007740 con 0x7fb5f00d32f0 2024-04-25T09:57:33.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.196+0000 7fb5edffb640 1 -- 172.21.15.148:0/666841086 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80078e0 con 0x7fb5f00d32f0 2024-04-25T09:57:33.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.196+0000 7fb5edffb640 1 -- 172.21.15.148:0/666841086 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007cb0 con 0x7fb5f00d32f0 2024-04-25T09:57:33.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 -- 172.21.15.148:0/666841086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 msgr2=0x7fb5f00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/666841086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f00cebb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009d90 tx=0x7fb5e802a660 comp rx=0 tx=0).stop 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 -- 172.21.15.148:0/666841086 shutdown_connections 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/666841086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f00cebb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/666841086 >> v2:172.21.15.188:3300/0 conn(0x7fb5f015dc20 0x7fb5f00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 -- 172.21.15.148:0/666841086 >> 172.21.15.148:0/666841086 conn(0x7fb5f00d0420 msgr2=0x7fb5f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 -- 172.21.15.148:0/666841086 shutdown_connections 2024-04-25T09:57:33.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.197+0000 7fb5f53c8640 1 -- 172.21.15.148:0/666841086 wait complete. 2024-04-25T09:57:33.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.198+0000 7fb5f53c8640 1 Processor -- start 2024-04-25T09:57:33.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.198+0000 7fb5f53c8640 1 -- start start 2024-04-25T09:57:33.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5f53c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5f53c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f015dc20 0x7fb5f00aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5f53c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 0x7fb5f00ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5f53c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b5e50 con 0x7fb5f0161710 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5f53c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0162500 con 0x7fb5f015dc20 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5f53c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b5fd0 con 0x7fb5f00d32f0 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 0x7fb5f00ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 0x7fb5f00ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43290/0 (socket says 172.21.15.148:43290) 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/3980314524 learned_addr learned my addr 172.21.15.148:0/3980314524 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f015dc20 0x7fb5f00aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 msgr2=0x7fb5f015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f015dc20 msgr2=0x7fb5f00aac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.199+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f015dc20 0x7fb5f00aac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/3980314524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00ccb70 con 0x7fb5f0161710 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 0x7fb5f00ab140 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009eb0 tx=0x7fb5e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5d77fe640 1 -- 172.21.15.148:0/3980314524 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e803f070 con 0x7fb5f0161710 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5d77fe640 1 -- 172.21.15.148:0/3980314524 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8031bc0 con 0x7fb5f0161710 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f00cce50 con 0x7fb5f0161710 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5d77fe640 1 -- 172.21.15.148:0/3980314524 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8009450 con 0x7fb5f0161710 2024-04-25T09:57:33.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.200+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f00cd410 con 0x7fb5f0161710 2024-04-25T09:57:33.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.201+0000 7fb5d77fe640 1 -- 172.21.15.148:0/3980314524 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5e8007790 con 0x7fb5f0161710 2024-04-25T09:57:33.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.202+0000 7fb5d77fe640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5dc06b4d0 0x7fb5dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.202+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5dc06b4d0 0x7fb5dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.202+0000 7fb5d77fe640 1 -- 172.21.15.148:0/3980314524 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5e80ab7f0 con 0x7fb5f0161710 2024-04-25T09:57:33.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.202+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5dc06b4d0 0x7fb5dc06d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00cd970 tx=0x7fb5e0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.203+0000 7fb5f81a7740 1 Processor -- start 2024-04-25T09:57:33.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.203+0000 7fb5f81a7740 1 -- start start 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6c0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe690770 0x5560fe693f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fe5f9da0 con 0x5560fe6bdef0 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fe5a2b80 con 0x5560fe6bdb10 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560fe5f5e40 con 0x5560fe690770 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6c0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43298/0 (socket says 172.21.15.148:43298) 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/3968445312 learned_addr learned my addr 172.21.15.148:0/3968445312 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6c0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe690770 0x5560fe693f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe690770 msgr2=0x5560fe693f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe690770 0x5560fe693f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 msgr2=0x5560fe6c0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6c0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/3968445312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560fe5acf10 con 0x5560fe6bdef0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.204+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6c0860 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4001800 tx=0x7fb5e4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdd8d5b97620464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5d4ff9640 1 -- 172.21.15.148:0/3968445312 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005ac0 con 0x5560fe6bdef0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5d4ff9640 1 -- 172.21.15.148:0/3968445312 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4018070 con 0x5560fe6bdef0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5d4ff9640 1 -- 172.21.15.148:0/3968445312 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40070d0 con 0x5560fe6bdef0 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5f81a7740 1 -- 172.21.15.148:0/3968445312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 msgr2=0x5560fe6c0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6c0860 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4001800 tx=0x7fb5e4000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5f81a7740 1 -- 172.21.15.148:0/3968445312 shutdown_connections 2024-04-25T09:57:33.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.205+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe690770 0x5560fe693f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6c0860 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/3968445312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6c0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 -- 172.21.15.148:0/3968445312 >> 172.21.15.148:0/3968445312 conn(0x5560fe6929e0 msgr2=0x5560fe6db320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 -- 172.21.15.148:0/3968445312 shutdown_connections 2024-04-25T09:57:33.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 -- 172.21.15.148:0/3968445312 wait complete. 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 -- start start 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.206+0000 7fb5f81a7740 1 Processor -- start 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe6d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fe695b90 con 0x5560fe6bdef0 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fe5984b0 con 0x5560fe6bdb10 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560fe695e70 con 0x5560fe6b50e0 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5f81a7740 10 client.0 _ll_register_callbacks cb 0x5560fe5d8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43312/0 (socket says 172.21.15.148:43312) 2024-04-25T09:57:33.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/2787201655 learned_addr learned my addr 172.21.15.148:0/2787201655 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe6d77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 msgr2=0x5560fe6d77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe6d77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 msgr2=0x5560fe6dd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6dd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.207+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/2787201655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560fe3d87d0 con 0x5560fe6bdef0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b67d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4014000 tx=0x7fb5e40010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631772fc249fdebb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5cdffb640 1 -- 172.21.15.148:0/2787201655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4001420 con 0x5560fe6bdef0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5cdffb640 1 -- 172.21.15.148:0/2787201655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5e400f750 con 0x5560fe6bdef0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5e4018070 con 0x5560fe6bdef0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5cdffb640 1 -- 172.21.15.148:0/2787201655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e402f050 con 0x5560fe6bdef0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 10 client.6237 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5560fe30e3f0 con 0x5560fe6bdef0 2024-04-25T09:57:33.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5cdffb640 1 -- 172.21.15.148:0/2787201655 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4018070 con 0x5560fe6bdef0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5cdffb640 1 -- 172.21.15.148:0/2787201655 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb5e4032020 con 0x5560fe6bdef0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 10 client.6237 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 10 client.6237 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 20 client.6237 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6d6ad0 0x5560fe6f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 4 client.6237 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560fe6072e0 con 0x5560fe6d6ad0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.208+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6d6ad0 0x5560fe6f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5cdffb640 10 client.6237 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6d6ad0 0x5560fe6f8670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77dedcddb1cf46 server_cookie=659bd20a1b87855c in_seq=0 out_seq=0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5cdffb640 1 -- 172.21.15.148:0/2787201655 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5560fe6072e0 con 0x5560fe6d6ad0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5cdffb640 10 client.6237 handle_command_reply: tid=0 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 client.6237 shutdown 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 msgr2=0x5560fe6b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b67d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4014000 tx=0x7fb5e40010a0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 shutdown_connections 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe6d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b67d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6d6ad0 0x5560fe6f8670 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/2787201655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6dd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5cdffb640 0 client.6237 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 >> 172.21.15.148:0/2787201655 conn(0x5560fe690770 msgr2=0x5560fe6bcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 shutdown_connections 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.209+0000 7fb5f81a7740 1 -- 172.21.15.148:0/2787201655 wait complete. 2024-04-25T09:57:33.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.210+0000 7fb5f81a7740 20 client.6237 trim_cache size 0 max 16384 2024-04-25T09:57:33.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.224+0000 7fb5f81a7740 1 Processor -- start 2024-04-25T09:57:33.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.224+0000 7fb5f81a7740 1 -- start start 2024-04-25T09:57:33.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6b50e0 0x5560fe6db320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 0x5560fe6db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6bdef0 0x5560fe6b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fe6905f0 con 0x5560fe6bdb10 2024-04-25T09:57:33.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fe6b62a0 con 0x5560fe6b50e0 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f0200480 con 0x5560fe6bdef0 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 0x5560fe6db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 0x5560fe6db860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43314/0 (socket says 172.21.15.148:43314) 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/4231252177 learned_addr learned my addr 172.21.15.148:0/4231252177 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6b50e0 0x5560fe6db320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6bdef0 0x5560fe6b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6bdef0 msgr2=0x5560fe6b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6bdef0 0x5560fe6b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6b50e0 msgr2=0x5560fe6db320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6b50e0 0x5560fe6db320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ee7fc640 1 -- 172.21.15.148:0/4231252177 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560fe2ce0f0 con 0x5560fe6bdb10 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6b50e0 0x5560fe6db320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.225+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6bdef0 0x5560fe6b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 0x5560fe6db860 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4000c00 tx=0x7fb5e4021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe7ff6c30c8337f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5cffff640 1 -- 172.21.15.148:0/4231252177 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4038070 con 0x5560fe6bdb10 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5cffff640 1 -- 172.21.15.148:0/4231252177 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4018070 con 0x5560fe6bdb10 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5cffff640 1 -- 172.21.15.148:0/4231252177 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400fac0 con 0x5560fe6bdb10 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 -- 172.21.15.148:0/4231252177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 msgr2=0x5560fe6db860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 0x5560fe6db860 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4000c00 tx=0x7fb5e4021070 comp rx=0 tx=0).stop 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 -- 172.21.15.148:0/4231252177 shutdown_connections 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6bdef0 0x5560fe6b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdb10 0x5560fe6db860 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/4231252177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6b50e0 0x5560fe6db320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 -- 172.21.15.148:0/4231252177 >> 172.21.15.148:0/4231252177 conn(0x5560fe690770 msgr2=0x5560fe6bcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 -- 172.21.15.148:0/4231252177 shutdown_connections 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.226+0000 7fb5f81a7740 1 -- 172.21.15.148:0/4231252177 wait complete. 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.227+0000 7fb5f81a7740 1 -- start start 2024-04-25T09:57:33.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.227+0000 7fb5f81a7740 1 Processor -- start 2024-04-25T09:57:33.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe744cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01fea50 con 0x5560fe6bdef0 2024-04-25T09:57:33.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01ff620 con 0x5560fe6bdb10 2024-04-25T09:57:33.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01ff020 con 0x5560fe6b50e0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 10 client.0 _ll_register_callbacks cb 0x7fb5e4001210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43316/0 (socket says 172.21.15.148:43316) 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/957086613 learned_addr learned my addr 172.21.15.148:0/957086613 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe744cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ee7fc640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 msgr2=0x5560fe744cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe744cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 msgr2=0x5560fe6d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 -- 172.21.15.148:0/957086613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560fe6d1b90 con 0x5560fe6bdef0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe744cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5ef7fe640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b3880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8082050 tx=0x7fb5e8080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ce63399271846e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5cdffb640 1 -- 172.21.15.148:0/957086613 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80802b0 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5cdffb640 1 -- 172.21.15.148:0/957086613 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5e807a180 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5e4018070 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5cdffb640 1 -- 172.21.15.148:0/957086613 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5e8082580 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 10 client.6243 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.228+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5560fe6b4190 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5cdffb640 1 -- 172.21.15.148:0/957086613 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4018070 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5cdffb640 1 -- 172.21.15.148:0/957086613 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb5e80820f0 con 0x5560fe6bdef0 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 10 client.6243 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 10 client.6243 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 20 client.6243 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6b6910 0x5560fe690c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 4 client.6243 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:57:33.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5560fe601630 con 0x5560fe6b6910 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6b6910 0x5560fe690c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5cdffb640 10 client.6243 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5eeffd640 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6b6910 0x5560fe690c50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d198286472aa8208 server_cookie=b5eb9e1928054605 in_seq=0 out_seq=0 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5cdffb640 1 -- 172.21.15.148:0/957086613 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5560fe601630 con 0x5560fe6b6910 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5cdffb640 10 client.6243 handle_command_reply: tid=0 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 client.6243 shutdown 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 msgr2=0x5560fe6b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b3880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8082050 tx=0x7fb5e8080040 comp rx=0 tx=0).stop 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 shutdown_connections 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560fe6bdef0 0x5560fe6b3880 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5560fe6b6910 0x5560fe690c50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560fe6bdb10 0x5560fe6d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 --2- 172.21.15.148:0/957086613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560fe6b50e0 0x5560fe744cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5cdffb640 0 client.6243 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.229+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 >> 172.21.15.148:0/957086613 conn(0x5560fe690770 msgr2=0x5560fe6db320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 shutdown_connections 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f81a7740 1 -- 172.21.15.148:0/957086613 wait complete. 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f81a7740 20 client.6243 trim_cache size 0 max 16384 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T09:57:33.240 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:33.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5dc06b4d0 msgr2=0x7fb5dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5dc06b4d0 0x7fb5dc06d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00cd970 tx=0x7fb5e0009190 comp rx=0 tx=0).stop 2024-04-25T09:57:33.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 msgr2=0x7fb5f00ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 0x7fb5f00ab140 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009eb0 tx=0x7fb5e8004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 shutdown_connections 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb5dc06b4d0 0x7fb5dc06d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5f0161710 0x7fb5f00ab140 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5f015dc20 0x7fb5f00aac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 --2- 172.21.15.148:0/3980314524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5f00d32f0 0x7fb5f015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 >> 172.21.15.148:0/3980314524 conn(0x7fb5f00d0420 msgr2=0x7fb5f00b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 shutdown_connections 2024-04-25T09:57:33.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.230+0000 7fb5f53c8640 1 -- 172.21.15.148:0/3980314524 wait complete. 2024-04-25T09:57:33.244 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:57:33.244 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:33.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:57:33.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.340+0000 7f9e023f4640 1 Processor -- start 2024-04-25T09:57:33.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.340+0000 7f9e023f4640 1 -- start start 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e023f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dfc15b7d0 0x7f9dfc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e023f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dfc0b5620 0x7f9dfc0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e023f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc06c2d0 0x7f9dfc06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e023f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc06cc80 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e023f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc06cf50 con 0x7f9dfc0b5620 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e023f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0ce530 con 0x7f9dfc06c2d0 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dfc15b7d0 0x7f9dfc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dfc15b7d0 0x7f9dfc0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43320/0 (socket says 172.21.15.148:43320) 2024-04-25T09:57:33.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2074462176 learned_addr learned my addr 172.21.15.148:0/2074462176 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9e00991640 1 --2- 172.21.15.148:0/2074462176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc06c2d0 0x7f9dfc06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/2074462176 >> v2:172.21.15.188:3300/0 conn(0x7f9dfc0b5620 0x7f9dfc0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2074462176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc06c2d0 msgr2=0x7f9dfc06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2074462176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc06c2d0 0x7f9dfc06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2074462176 >> v2:172.21.15.188:3300/0 conn(0x7f9dfc0b5620 msgr2=0x7f9dfc0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2074462176 >> v2:172.21.15.188:3300/0 conn(0x7f9dfc0b5620 0x7f9dfc0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.341+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2074462176 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0ce810 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2074462176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0cf010 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0ce180 tx=0x7f9dec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45501d3088198f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9dfaffd640 1 -- 172.21.15.148:0/2074462176 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec037070 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9dfaffd640 1 -- 172.21.15.148:0/2074462176 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec032440 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9dfaffd640 1 -- 172.21.15.148:0/2074462176 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec032810 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 -- 172.21.15.148:0/2074462176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 msgr2=0x7f9dfc0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2074462176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0cf010 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0ce180 tx=0x7f9dec02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 -- 172.21.15.148:0/2074462176 shutdown_connections 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2074462176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc06c2d0 0x7f9dfc06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2074462176 >> v2:172.21.15.188:3300/0 conn(0x7f9dfc0b5620 0x7f9dfc0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2074462176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0cf010 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 -- 172.21.15.148:0/2074462176 >> 172.21.15.148:0/2074462176 conn(0x7f9dfc0d0800 msgr2=0x7f9dfc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 -- 172.21.15.148:0/2074462176 shutdown_connections 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.342+0000 7f9e023f4640 1 -- 172.21.15.148:0/2074462176 wait complete. 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 Processor -- start 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 -- start start 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dfc06c2d0 0x7f9dfc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc0b5620 0x7f9dfc0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cf550 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc06d230 con 0x7f9dfc06c2d0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.344+0000 7f9e023f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cf6d0 con 0x7f9dfc0b5620 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43326/0 (socket says 172.21.15.148:43326) 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 -- 172.21.15.148:0/2934920988 learned_addr learned my addr 172.21.15.148:0/2934920988 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc0b5620 0x7f9dfc0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dfc06c2d0 0x7f9dfc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 -- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc0b5620 msgr2=0x7f9dfc0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc0b5620 0x7f9dfc0c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 -- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dfc06c2d0 msgr2=0x7f9dfc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dfc06c2d0 0x7f9dfc155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 -- 172.21.15.148:0/2934920988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0cbc40 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e00991640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0c9560 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007d40 tx=0x7f9df400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9df8ff9640 1 -- 172.21.15.148:0/2934920988 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4014420 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9df8ff9640 1 -- 172.21.15.148:0/2934920988 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4019070 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc0cbf20 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9df8ff9640 1 -- 172.21.15.148:0/2934920988 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4014820 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.345+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc0cc4e0 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.347+0000 7f9df8ff9640 1 -- 172.21.15.148:0/2934920988 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9df40149c0 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.347+0000 7f9df8ff9640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9de806b4d0 0x7f9de806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.347+0000 7f9df8ff9640 1 -- 172.21.15.148:0/2934920988 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9df408aaf0 con 0x7f9dfc15b7d0 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.347+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9de806b4d0 0x7f9de806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.347+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9de806b4d0 0x7f9de806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9dec034000 tx=0x7f9dec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.348+0000 7f9e051ce740 1 Processor -- start 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.348+0000 7f9e051ce740 1 -- start start 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbdaa0 0x55653fdc02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 0x55653fdc07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fd90700 0x55653fd93f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e051ce740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653fcf9e00 con 0x55653fdbde80 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653fca2d20 con 0x55653fd90700 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55653fcf5ea0 con 0x55653fdbdaa0 2024-04-25T09:57:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 0x55653fdc07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 0x55653fdc07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43334/0 (socket says 172.21.15.148:43334) 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 -- 172.21.15.148:0/3440185726 learned_addr learned my addr 172.21.15.148:0/3440185726 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e00991640 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fd90700 0x55653fd93f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 -- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbdaa0 msgr2=0x55653fdc02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbdaa0 0x55653fdc02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 -- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fd90700 msgr2=0x55653fd93f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fd90700 0x55653fd93f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 -- 172.21.15.148:0/3440185726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653fcc4df0 con 0x55653fdbde80 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9e00991640 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fd90700 0x55653fd93f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 0x55653fdc07f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006fd0 tx=0x7f9df000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79e627a202faecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.349+0000 7f9de1ffb640 1 -- 172.21.15.148:0/3440185726 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0018030 con 0x55653fdbde80 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9de1ffb640 1 -- 172.21.15.148:0/3440185726 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0004020 con 0x55653fdbde80 2024-04-25T09:57:33.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9de1ffb640 1 -- 172.21.15.148:0/3440185726 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00043d0 con 0x55653fdbde80 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 -- 172.21.15.148:0/3440185726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 msgr2=0x55653fdc07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 0x55653fdc07f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9df0006fd0 tx=0x7f9df000bd00 comp rx=0 tx=0).stop 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 -- 172.21.15.148:0/3440185726 shutdown_connections 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fd90700 0x55653fd93f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbde80 0x55653fdc07f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 --2- 172.21.15.148:0/3440185726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbdaa0 0x55653fdc02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 -- 172.21.15.148:0/3440185726 >> 172.21.15.148:0/3440185726 conn(0x55653fd92970 msgr2=0x55653fddb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 -- 172.21.15.148:0/3440185726 shutdown_connections 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.350+0000 7f9e051ce740 1 -- 172.21.15.148:0/3440185726 wait complete. 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.351+0000 7f9e051ce740 1 -- start start 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.351+0000 7f9e051ce740 1 Processor -- start 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fdb38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fdc0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653fd95b20 con 0x55653fdbdaa0 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653fc98590 con 0x55653fdbfe00 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55653fd95e00 con 0x55653fdbde80 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 10 client.0 _ll_register_callbacks cb 0x55653fca4e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fdb38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fdb38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43348/0 (socket says 172.21.15.148:43348) 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1941229937 learned_addr learned my addr 172.21.15.148:0/1941229937 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e00991640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fdc0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 msgr2=0x55653fdb4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 msgr2=0x55653fdc0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fdc0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1941229937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653fa0e3f0 con 0x55653fdbdaa0 2024-04-25T09:57:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e00991640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fdc0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fdb38f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009930 tx=0x7f9dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d00d0c50d39eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1941229937 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec04d060 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1941229937 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9dec03c520 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.352+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9df0004020 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1941229937 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dec053050 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 10 client.6255 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55653fad87d0 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1941229937 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0004020 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1941229937 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9dec056020 con 0x55653fdbdaa0 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 10 client.6255 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 10 client.6255 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 20 client.6255 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:33.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fddc8b0 0x55653fddcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 4 client.6255 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55653fd07340 con 0x55653fddc8b0 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fddc8b0 0x55653fddcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9ddaffd640 10 client.6255 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.353+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fddc8b0 0x55653fddcd00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e6b1216d56d28 server_cookie=6825dab8f9df978d in_seq=0 out_seq=0 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1941229937 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55653fd07340 con 0x55653fddc8b0 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9ddaffd640 10 client.6255 handle_command_reply: tid=0 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 client.6255 shutdown 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 msgr2=0x55653fdb38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fdb38f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009930 tx=0x7f9dec0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 shutdown_connections 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fddc8b0 0x55653fddcd00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fdc0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9ddaffd640 0 client.6255 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1941229937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fdb38f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 >> 172.21.15.148:0/1941229937 conn(0x55653fd90700 msgr2=0x55653fd952c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 shutdown_connections 2024-04-25T09:57:33.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 1 -- 172.21.15.148:0/1941229937 wait complete. 2024-04-25T09:57:33.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.354+0000 7f9e051ce740 20 client.6255 trim_cache size 0 max 16384 2024-04-25T09:57:33.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 Processor -- start 2024-04-25T09:57:33.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 -- start start 2024-04-25T09:57:33.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fd912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fd93f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc200330 con 0x55653fdbdaa0 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc1ff4b0 con 0x55653fdbfe00 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc1577c0 con 0x55653fdbde80 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fd912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fd912c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43352/0 (socket says 172.21.15.148:43352) 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.366+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2579302150 learned_addr learned my addr 172.21.15.148:0/2579302150 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2579302150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 msgr2=0x55653fdb44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2579302150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 msgr2=0x55653fd93f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fd93f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9dfbfff640 1 -- 172.21.15.148:0/2579302150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653fdc05f0 con 0x55653fdbdaa0 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fd912c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9dec02a7d0 tx=0x7f9dec042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e81d19affd11b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9de17fa640 1 -- 172.21.15.148:0/2579302150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec04d070 con 0x55653fdbdaa0 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9de17fa640 1 -- 172.21.15.148:0/2579302150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec032440 con 0x55653fdbdaa0 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9de17fa640 1 -- 172.21.15.148:0/2579302150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec037040 con 0x55653fdbdaa0 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9e051ce740 1 -- 172.21.15.148:0/2579302150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 msgr2=0x55653fd912c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.367+0000 7f9e051ce740 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fd912c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9dec02a7d0 tx=0x7f9dec042000 comp rx=0 tx=0).stop 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 -- 172.21.15.148:0/2579302150 shutdown_connections 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbfe00 0x55653fd93f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbde80 0x55653fdb44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 --2- 172.21.15.148:0/2579302150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fd912c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 -- 172.21.15.148:0/2579302150 >> 172.21.15.148:0/2579302150 conn(0x55653fd90700 msgr2=0x55653fdbb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 -- 172.21.15.148:0/2579302150 shutdown_connections 2024-04-25T09:57:33.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 -- 172.21.15.148:0/2579302150 wait complete. 2024-04-25T09:57:33.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 -- start start 2024-04-25T09:57:33.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.368+0000 7f9e051ce740 1 Processor -- start 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fe456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbde80 0x55653fdbc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbfe00 0x55653fdbe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653fcff0f0 con 0x55653fdbdaa0 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc1feeb0 con 0x55653fdbde80 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55653fd8c090 con 0x55653fdbfe00 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e051ce740 10 client.0 _ll_register_callbacks cb 0x7f9df000b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fe456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fe456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43356/0 (socket says 172.21.15.148:43356) 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1096542547 learned_addr learned my addr 172.21.15.148:0/1096542547 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9e00991640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbfe00 0x55653fdbe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbfe00 msgr2=0x55653fdbe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbde80 0x55653fdbc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbfe00 0x55653fdbe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbde80 msgr2=0x55653fdbc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbde80 0x55653fdbc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.369+0000 7f9dfbfff640 1 -- 172.21.15.148:0/1096542547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653fdc3090 con 0x55653fdbdaa0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9dfbfff640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fe456b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009770 tx=0x7f9dec008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1420975e7f08f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1096542547 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec037040 con 0x55653fdbdaa0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1096542547 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9dec043040 con 0x55653fdbdaa0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dec032440 con 0x55653fdbdaa0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1096542547 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dec04d040 con 0x55653fdbdaa0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 10 client.6261 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55653fdbe4d0 con 0x55653fdbdaa0 2024-04-25T09:57:33.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1096542547 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec032440 con 0x55653fdbdaa0 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1096542547 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9dec060070 con 0x55653fdbdaa0 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 10 client.6261 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 20 client.6261 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fd95620 0x55653fd93f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 4 client.6261 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55653fd01690 con 0x55653fd95620 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.370+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fd95620 0x55653fd93f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9ddaffd640 10 client.6261 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9dfb7fe640 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fd95620 0x55653fd93f10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c4b93a97a9e318 server_cookie=d5f52bb1ce3e2369 in_seq=0 out_seq=0 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9ddaffd640 1 -- 172.21.15.148:0/1096542547 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+167 (crc 0 0 0) 0x55653fd01690 con 0x55653fd95620 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9ddaffd640 10 client.6261 handle_command_reply: tid=0 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 client.6261 shutdown 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 msgr2=0x55653fe456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fe456b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009770 tx=0x7f9dec008030 comp rx=0 tx=0).stop 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 shutdown_connections 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55653fdbfe00 0x55653fdbe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55653fd95620 0x55653fd93f10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55653fdbde80 0x55653fdbc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9ddaffd640 0 client.6261 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 --2- 172.21.15.148:0/1096542547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55653fdbdaa0 0x55653fe456b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 >> 172.21.15.148:0/1096542547 conn(0x55653fd90700 msgr2=0x55653fdbb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 shutdown_connections 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 1 -- 172.21.15.148:0/1096542547 wait complete. 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.371+0000 7f9e051ce740 20 client.6261 trim_cache size 0 max 16384 2024-04-25T09:57:33.381 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "damage_type": "backtrace", 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 734530555, 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 1099511630186, 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:33.382 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9de806b4d0 msgr2=0x7f9de806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9de806b4d0 0x7f9de806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9dec034000 tx=0x7f9dec02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 msgr2=0x7f9dfc0c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0c9560 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007d40 tx=0x7f9df400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 shutdown_connections 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9de806b4d0 0x7f9de806d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dfc15b7d0 0x7f9dfc0c9560 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dfc0b5620 0x7f9dfc0c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 --2- 172.21.15.148:0/2934920988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dfc06c2d0 0x7f9dfc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 >> 172.21.15.148:0/2934920988 conn(0x7f9dfc0d0800 msgr2=0x7f9dfc15eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 shutdown_connections 2024-04-25T09:57:33.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.372+0000 7f9e023f4640 1 -- 172.21.15.148:0/2934920988 wait complete. 2024-04-25T09:57:33.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.479+0000 7fe6b69f9640 1 Processor -- start 2024-04-25T09:57:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.479+0000 7fe6b69f9640 1 -- start start 2024-04-25T09:57:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6b69f9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6b00cfec0 0x7fe6b00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6b69f9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b0071480 con 0x7fe6b00ce3c0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6b69f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b0071760 con 0x7fe6b0154ee0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6b69f9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b00ce830 con 0x7fe6b00cfec0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6b4f96640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe6b00ce3c0 0x7fe6b006d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37478/0 (socket says 172.21.15.148:37478) 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6b4f96640 1 -- 172.21.15.148:0/4166188801 learned_addr learned my addr 172.21.15.148:0/4166188801 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.480+0000 7fe6af7fe640 1 --2- 172.21.15.148:0/4166188801 >> v2:172.21.15.188:3300/0 conn(0x7fe6b00cfec0 0x7fe6b00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388551779 0 0) 0x7fe6b0071480 con 0x7fe6b00ce3c0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe698003520 con 0x7fe6b00ce3c0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3554748116 0 0) 0x7fe698003520 con 0x7fe6b00ce3c0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 >> v2:172.21.15.188:3300/0 conn(0x7fe6b00cfec0 msgr2=0x7fe6b00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 --2- 172.21.15.148:0/4166188801 >> v2:172.21.15.188:3300/0 conn(0x7fe6b00cfec0 0x7fe6b00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 >> v1:172.21.15.188:6789/0 conn(0x7fe6b0154ee0 legacy=0x7fe6b00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0071a40 con 0x7fe6b00ce3c0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe6a80019f0 con 0x7fe6b00ce3c0 2024-04-25T09:57:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 >> v1:172.21.15.148:6789/0 conn(0x7fe6b00ce3c0 legacy=0x7fe6b006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6980039b0 0x7fe698003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe698004800 0x7fe698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6980051c0 0x7fe69800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00ce830 con 0x7fe698004800 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69800c060 con 0x7fe6980039b0 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69800e470 con 0x7fe6980051c0 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6af7fe640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe698004800 0x7fe698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6affff640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6980039b0 0x7fe698003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6b4f96640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6980051c0 0x7fe69800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6af7fe640 1 -- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6980051c0 msgr2=0x7fe69800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6af7fe640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6980051c0 0x7fe69800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6af7fe640 1 -- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6980039b0 msgr2=0x7fe698003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6af7fe640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6980039b0 0x7fe698003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6b4f96640 1 -- 172.21.15.148:0/4166188801 reap_dead start 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.481+0000 7fe6af7fe640 1 -- 172.21.15.148:0/4166188801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698003520 con 0x7fe698004800 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.482+0000 7fe6af7fe640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe698004800 0x7fe698004c80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009ac0 tx=0x7fe698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8335a38262e9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.482+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4036070 con 0x7fe698004800 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.482+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4031400 con 0x7fe698004800 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.482+0000 7fe6aeffd640 1 -- 172.21.15.148:0/4166188801 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4009690 con 0x7fe698004800 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 -- 172.21.15.148:0/4166188801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe698004800 msgr2=0x7fe698004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe698004800 0x7fe698004c80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009ac0 tx=0x7fe698005830 comp rx=0 tx=0).stop 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 -- 172.21.15.148:0/4166188801 shutdown_connections 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/4166188801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe698004800 0x7fe698004c80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 -- 172.21.15.148:0/4166188801 >> 172.21.15.148:0/4166188801 conn(0x7fe6b006abe0 msgr2=0x7fe6b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 -- 172.21.15.148:0/4166188801 shutdown_connections 2024-04-25T09:57:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.483+0000 7fe6b69f9640 1 -- 172.21.15.148:0/4166188801 wait complete. 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.484+0000 7fe6b69f9640 1 Processor -- start 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.484+0000 7fe6b69f9640 1 -- start start 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b69f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe698004800 0x7fe6b013ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b69f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b0150440 0x7fe6b014eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b69f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 0x7fe6b015c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b69f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0071230 con 0x7fe6b013b970 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b69f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b01552c0 con 0x7fe698004800 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b69f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0155440 con 0x7fe6b0150440 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 0x7fe6b015c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 0x7fe6b015c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40452/0 (socket says 172.21.15.148:40452) 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 -- 172.21.15.148:0/2657898220 learned_addr learned my addr 172.21.15.148:0/2657898220 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6af7fe640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b0150440 0x7fe6b014eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6affff640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe698004800 0x7fe6b013ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 -- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b0150440 msgr2=0x7fe6b014eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b0150440 0x7fe6b014eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 -- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe698004800 msgr2=0x7fe6b013ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe698004800 0x7fe6b013ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 -- 172.21.15.148:0/2657898220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b014f3a0 con 0x7fe6b013b970 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.485+0000 7fe6b4f96640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 0x7fe6b015c4e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00cd2d0 tx=0x7fe6a8007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.486+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800ea20 con 0x7fe6b013b970 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.486+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8018070 con 0x7fe6b013b970 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.486+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b014f680 con 0x7fe6b013b970 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.486+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800ee20 con 0x7fe6b013b970 2024-04-25T09:57:33.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.486+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b014fc40 con 0x7fe6b013b970 2024-04-25T09:57:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.487+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0073990 con 0x7fe6b013b970 2024-04-25T09:57:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.487+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6a801d460 con 0x7fe6b013b970 2024-04-25T09:57:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.487+0000 7fe6acff9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe69c06b500 0x7fe69c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.488+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6a808a3f0 con 0x7fe6b013b970 2024-04-25T09:57:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.488+0000 7fe6affff640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe69c06b500 0x7fe69c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.488+0000 7fe6affff640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe69c06b500 0x7fe69c06d9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.492+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6a8058a00 con 0x7fe6b013b970 2024-04-25T09:57:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.695+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6b006d360 con 0x7fe6b013b970 2024-04-25T09:57:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.695+0000 7fe6acff9640 1 -- 172.21.15.148:0/2657898220 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4950 (secure 0 0 0) 0x7fe6a805caa0 con 0x7fe6b013b970 2024-04-25T09:57:33.699 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:33.699 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:31.018031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2024-04-25T09:57:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.697+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe69c06b500 msgr2=0x7fe69c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.697+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe69c06b500 0x7fe69c06d9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.697+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 msgr2=0x7fe6b015c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.697+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 0x7fe6b015c4e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00cd2d0 tx=0x7fe6a8007bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 shutdown_connections 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe69c06b500 0x7fe69c06d9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6b013b970 0x7fe6b015c4e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b0150440 0x7fe6b014eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 --2- 172.21.15.148:0/2657898220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe698004800 0x7fe6b013ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 >> 172.21.15.148:0/2657898220 conn(0x7fe6b006abe0 msgr2=0x7fe6b00b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 shutdown_connections 2024-04-25T09:57:33.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.698+0000 7fe6b69f9640 1 -- 172.21.15.148:0/2657898220 wait complete. 2024-04-25T09:57:33.712 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 734530555 2024-04-25T09:57:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.808+0000 7f30bd838640 1 Processor -- start 2024-04-25T09:57:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.808+0000 7f30bd838640 1 -- start start 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30bd838640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30b815dc20 0x7f30b80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30bd838640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b8161710 0x7f30b80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30bd838640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b815e000 con 0x7f30b80b9330 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30bd838640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80d1de0 con 0x7f30b815dc20 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30bd838640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8161c60 con 0x7f30b8161710 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30b6ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30b815dc20 0x7f30b80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b8161710 0x7f30b80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30b6ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30b815dc20 0x7f30b80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:46824/0 (socket says 172.21.15.148:46824) 2024-04-25T09:57:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30b77fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f30b80b9330 0x7f30b806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56346/0 (socket says 172.21.15.148:56346) 2024-04-25T09:57:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.809+0000 7f30b6ffd640 1 -- 172.21.15.148:0/1264066742 learned_addr learned my addr 172.21.15.148:0/1264066742 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.810+0000 7f30b67fc640 1 -- 172.21.15.148:0/1264066742 >> v2:172.21.15.188:3300/0 conn(0x7f30b815dc20 msgr2=0x7f30b80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.810+0000 7f30b67fc640 1 --2- 172.21.15.148:0/1264066742 >> v2:172.21.15.188:3300/0 conn(0x7f30b815dc20 0x7f30b80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.810+0000 7f30b67fc640 1 -- 172.21.15.148:0/1264066742 >> v1:172.21.15.188:6789/0 conn(0x7f30b80b9330 legacy=0x7f30b806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.810+0000 7f30b67fc640 1 -- 172.21.15.148:0/1264066742 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8161f40 con 0x7f30b8161710 2024-04-25T09:57:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.810+0000 7f30b67fc640 1 --2- 172.21.15.148:0/1264066742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b8161710 0x7f30b80cebb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097a0 tx=0x7f30ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31bc9b4b63fd3ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30b5ffb640 1 -- 172.21.15.148:0/1264066742 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac037070 con 0x7f30b8161710 2024-04-25T09:57:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30b5ffb640 1 -- 172.21.15.148:0/1264066742 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac032440 con 0x7f30b8161710 2024-04-25T09:57:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30b5ffb640 1 -- 172.21.15.148:0/1264066742 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac032810 con 0x7f30b8161710 2024-04-25T09:57:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 -- 172.21.15.148:0/1264066742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b8161710 msgr2=0x7f30b80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 --2- 172.21.15.148:0/1264066742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b8161710 0x7f30b80cebb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097a0 tx=0x7f30ac02a180 comp rx=0 tx=0).stop 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 -- 172.21.15.148:0/1264066742 shutdown_connections 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 --2- 172.21.15.148:0/1264066742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b8161710 0x7f30b80cebb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 --2- 172.21.15.148:0/1264066742 >> v2:172.21.15.188:3300/0 conn(0x7f30b815dc20 0x7f30b80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 -- 172.21.15.148:0/1264066742 >> 172.21.15.148:0/1264066742 conn(0x7f30b80d0420 msgr2=0x7f30b80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 -- 172.21.15.148:0/1264066742 shutdown_connections 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.811+0000 7f30bd838640 1 -- 172.21.15.148:0/1264066742 wait complete. 2024-04-25T09:57:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.812+0000 7f30bd838640 1 Processor -- start 2024-04-25T09:57:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.812+0000 7f30bd838640 1 -- start start 2024-04-25T09:57:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30bd838640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b80b9330 0x7f30b8152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30bd838640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 0x7f30b80acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30bd838640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8161710 0x7f30b80ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30bd838640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b5e50 con 0x7f30b815dc20 2024-04-25T09:57:33.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30bd838640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8162500 con 0x7f30b8161710 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30bd838640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b5fd0 con 0x7f30b80b9330 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 0x7f30b80acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 0x7f30b80acd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40454/0 (socket says 172.21.15.148:40454) 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30b67fc640 1 -- 172.21.15.148:0/615171242 learned_addr learned my addr 172.21.15.148:0/615171242 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30b77fe640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8161710 0x7f30b80ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.813+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b80b9330 0x7f30b8152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30b67fc640 1 -- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b80b9330 msgr2=0x7f30b8152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30b67fc640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b80b9330 0x7f30b8152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30b67fc640 1 -- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8161710 msgr2=0x7f30b80ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30b67fc640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8161710 0x7f30b80ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30b67fc640 1 -- 172.21.15.148:0/615171242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b815d490 con 0x7f30b815dc20 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30b67fc640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 0x7f30b80acd20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f30ac035040 tx=0x7f30ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f309ffff640 1 -- 172.21.15.148:0/615171242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac037040 con 0x7f30b815dc20 2024-04-25T09:57:33.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f309ffff640 1 -- 172.21.15.148:0/615171242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac030500 con 0x7f30b815dc20 2024-04-25T09:57:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b80aae30 con 0x7f30b815dc20 2024-04-25T09:57:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f309ffff640 1 -- 172.21.15.148:0/615171242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac030910 con 0x7f30b815dc20 2024-04-25T09:57:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.814+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b80ab3f0 con 0x7f30b815dc20 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.815+0000 7f309ffff640 1 -- 172.21.15.148:0/615171242 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30ac044020 con 0x7f30b815dc20 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.816+0000 7f309ffff640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30a406b4d0 0x7f30a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.816+0000 7f309ffff640 1 -- 172.21.15.148:0/615171242 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30ac0aa430 con 0x7f30b815dc20 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.816+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30a406b4d0 0x7f30a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.816+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30a406b4d0 0x7f30a406d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f30a8011340 tx=0x7f30a800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.817+0000 7f30c061c740 1 Processor -- start 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.817+0000 7f30c061c740 1 -- start start 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a761530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30c061c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a790db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7912f0 0x56190a766100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30c061c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a6ca010 con 0x56190a791a30 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a669180 con 0x56190a7912f0 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190a6c60b0 con 0x56190a7ae910 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a790db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a790db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40466/0 (socket says 172.21.15.148:40466) 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 -- 172.21.15.148:0/215321935 learned_addr learned my addr 172.21.15.148:0/215321935 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a761530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b77fe640 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7912f0 0x56190a766100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 -- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 msgr2=0x56190a761530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a761530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 -- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7912f0 msgr2=0x56190a766100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7912f0 0x56190a766100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 -- 172.21.15.148:0/215321935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56190a4aa900 con 0x56190a791a30 2024-04-25T09:57:33.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b77fe640 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7912f0 0x56190a766100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f30b67fc640 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a790db0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0a87a0 tx=0x7f30ac078580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94579c90367396fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f309d7fa640 1 -- 172.21.15.148:0/215321935 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0811c0 con 0x56190a791a30 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f309d7fa640 1 -- 172.21.15.148:0/215321935 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac076e60 con 0x56190a791a30 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.818+0000 7f309d7fa640 1 -- 172.21.15.148:0/215321935 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0aa0b0 con 0x56190a791a30 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 -- 172.21.15.148:0/215321935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 msgr2=0x56190a790db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a790db0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0a87a0 tx=0x7f30ac078580 comp rx=0 tx=0).stop 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 -- 172.21.15.148:0/215321935 shutdown_connections 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7912f0 0x56190a766100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a790db0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 --2- 172.21.15.148:0/215321935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a761530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 -- 172.21.15.148:0/215321935 >> 172.21.15.148:0/215321935 conn(0x56190a7637a0 msgr2=0x56190a7abdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 -- 172.21.15.148:0/215321935 shutdown_connections 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.819+0000 7f30c061c740 1 -- 172.21.15.148:0/215321935 wait complete. 2024-04-25T09:57:33.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.820+0000 7f30c061c740 1 -- start start 2024-04-25T09:57:33.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.820+0000 7f30c061c740 1 Processor -- start 2024-04-25T09:57:33.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a781d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a7a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a7ace00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a6ca010 con 0x56190a791a30 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a666c50 con 0x56190a7a3d00 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190a7656c0 con 0x56190a7ae910 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 10 client.0 _ll_register_callbacks cb 0x56190a66e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a781d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a781d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40468/0 (socket says 172.21.15.148:40468) 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 -- 172.21.15.148:0/1202853616 learned_addr learned my addr 172.21.15.148:0/1202853616 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b67fc640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a7a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b77fe640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a7ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 -- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 msgr2=0x56190a7a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a7a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 -- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 msgr2=0x56190a7ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a7ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 -- 172.21.15.148:0/1202853616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56190a39e1f0 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b77fe640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a7ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a781d80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f30a8017f40 tx=0x7f30a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71886c2b20cb0221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30967fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30967fc640 1 -- 172.21.15.148:0/1202853616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8026030 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30967fc640 1 -- 172.21.15.148:0/1202853616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30a8022620 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30ac076e60 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30967fc640 1 -- 172.21.15.148:0/1202853616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30a8032050 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 10 client.6276 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.821+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56190a50b090 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30967fc640 1 -- 172.21.15.148:0/1202853616 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac076e60 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30967fc640 1 -- 172.21.15.148:0/1202853616 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f30a8035020 con 0x56190a791a30 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30c061c740 10 client.6276 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30c061c740 10 client.6276 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30c061c740 20 client.6276 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:33.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30c061c740 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a77f570 0x56190a7924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30c061c740 4 client.6276 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56190a6d7550 con 0x56190a77f570 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30b67fc640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a77f570 0x56190a7924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30967fc640 10 client.6276 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.822+0000 7f30b67fc640 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a77f570 0x56190a7924f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7086dc5b89c0409e server_cookie=57ace85752e15053 in_seq=0 out_seq=0 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30967fc640 1 -- 172.21.15.148:0/1202853616 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56190a6d7550 con 0x56190a77f570 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30967fc640 10 client.6276 handle_command_reply: tid=0 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 client.6276 shutdown 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 msgr2=0x56190a781d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a781d80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f30a8017f40 tx=0x7f30a8004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 shutdown_connections 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a77f570 0x56190a7924f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a7ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a7ae910 0x56190a7a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 --2- 172.21.15.148:0/1202853616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a791a30 0x56190a781d80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30967fc640 0 client.6276 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 >> 172.21.15.148:0/1202853616 conn(0x56190a7912f0 msgr2=0x56190a760c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 shutdown_connections 2024-04-25T09:57:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 1 -- 172.21.15.148:0/1202853616 wait complete. 2024-04-25T09:57:33.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.823+0000 7f30c061c740 20 client.6276 trim_cache size 0 max 16384 2024-04-25T09:57:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.834+0000 7f30c061c740 1 Processor -- start 2024-04-25T09:57:33.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 -- start start 2024-04-25T09:57:33.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a78d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a78c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 0x56190a766ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b81ff830 con 0x56190a7ae910 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80cdce0 con 0x56190a7a3d00 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.835+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b81ff260 con 0x56190a791a30 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 0x56190a766ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 0x56190a766ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40476/0 (socket says 172.21.15.148:40476) 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 -- 172.21.15.148:0/533159181 learned_addr learned my addr 172.21.15.148:0/533159181 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b67fc640 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a78c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a78d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 -- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 msgr2=0x56190a78d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a78d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 -- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 msgr2=0x56190a78c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a78c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 -- 172.21.15.148:0/533159181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56190a764650 con 0x56190a7ae910 2024-04-25T09:57:33.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a78d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f30b77fe640 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 0x56190a766ac0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30b0004ed0 tx=0x7f30b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5674a53aae6ed7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f309cff9640 1 -- 172.21.15.148:0/533159181 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b001a020 con 0x56190a7ae910 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f309cff9640 1 -- 172.21.15.148:0/533159181 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b00138b0 con 0x56190a7ae910 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.836+0000 7f309cff9640 1 -- 172.21.15.148:0/533159181 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0013ca0 con 0x56190a7ae910 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 -- 172.21.15.148:0/533159181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 msgr2=0x56190a766ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 0x56190a766ac0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f30b0004ed0 tx=0x7f30b0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 -- 172.21.15.148:0/533159181 shutdown_connections 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7ae910 0x56190a766ac0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7a3d00 0x56190a78c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 --2- 172.21.15.148:0/533159181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a78d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 -- 172.21.15.148:0/533159181 >> 172.21.15.148:0/533159181 conn(0x56190a7912f0 msgr2=0x56190a781b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 -- 172.21.15.148:0/533159181 shutdown_connections 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 -- 172.21.15.148:0/533159181 wait complete. 2024-04-25T09:57:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 -- start start 2024-04-25T09:57:33.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.837+0000 7f30c061c740 1 Processor -- start 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a814510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 0x56190a7ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7ae910 0x56190a7ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a780070 con 0x56190a7a3d00 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a7946d0 con 0x56190a7ae910 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190a77f9a0 con 0x56190a791a30 2024-04-25T09:57:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30c061c740 10 client.0 _ll_register_callbacks cb 0x7f30ac009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 0x56190a7ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 0x56190a7ab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40478/0 (socket says 172.21.15.148:40478) 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 -- 172.21.15.148:0/2544106311 learned_addr learned my addr 172.21.15.148:0/2544106311 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a814510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b77fe640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7ae910 0x56190a7ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 -- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 msgr2=0x56190a814510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a814510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 -- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7ae910 msgr2=0x56190a7ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7ae910 0x56190a7ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b67fc640 1 -- 172.21.15.148:0/2544106311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56190a377370 con 0x56190a7a3d00 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.838+0000 7f30b77fe640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7ae910 0x56190a7ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30b67fc640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 0x56190a7ab2d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f30ac078860 tx=0x7f30ac0aa040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdbbc45dedf9fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:33.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 1 -- 172.21.15.148:0/2544106311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0751d0 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 1 -- 172.21.15.148:0/2544106311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30ac0aa0b0 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30b00138b0 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 1 -- 172.21.15.148:0/2544106311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30ac075370 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 10 client.6282 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56190a7a2e70 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 1 -- 172.21.15.148:0/2544106311 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b00138b0 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 1 -- 172.21.15.148:0/2544106311 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f30ac03a070 con 0x56190a7a3d00 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 10 client.6282 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 10 client.6282 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 20 client.6282 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a81a9c0 0x56190a81ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 4 client.6282 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 734530555}] 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "damage rm", "damage_id": 734530555}) v1 -- 0x56190a7a4b80 con 0x56190a81a9c0 2024-04-25T09:57:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a81a9c0 0x56190a81ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30967fc640 10 client.6282 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.839+0000 7f30b6ffd640 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a81a9c0 0x56190a81ada0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbef3e3be0041305 server_cookie=67e713b747e40ab8 in_seq=0 out_seq=0 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30967fc640 1 -- 172.21.15.148:0/2544106311 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x56190a7a4b80 con 0x56190a81a9c0 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30967fc640 10 client.6282 handle_command_reply: tid=0 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 client.6282 shutdown 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 msgr2=0x56190a7ab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 0x56190a7ab2d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f30ac078860 tx=0x7f30ac0aa040 comp rx=0 tx=0).stop 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 shutdown_connections 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56190a81a9c0 0x56190a81ada0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56190a7ae910 0x56190a7ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56190a7a3d00 0x56190a7ab2d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30967fc640 0 client.6282 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 --2- 172.21.15.148:0/2544106311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56190a791a30 0x56190a814510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 >> 172.21.15.148:0/2544106311 conn(0x56190a7912f0 msgr2=0x56190a766ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 shutdown_connections 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 1 -- 172.21.15.148:0/2544106311 wait complete. 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30c061c740 20 client.6282 trim_cache size 0 max 16384 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30a406b4d0 msgr2=0x7f30a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30a406b4d0 0x7f30a406d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f30a8011340 tx=0x7f30a800b040 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 msgr2=0x7f30b80acd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 0x7f30b80acd20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f30ac035040 tx=0x7f30ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 shutdown_connections 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30a406b4d0 0x7f30a406d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8161710 0x7f30b80ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30b815dc20 0x7f30b80acd20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 --2- 172.21.15.148:0/615171242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30b80b9330 0x7f30b8152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.840+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 >> 172.21.15.148:0/615171242 conn(0x7f30b80d0420 msgr2=0x7f30b80b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.841+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 shutdown_connections 2024-04-25T09:57:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.841+0000 7f30bd838640 1 -- 172.21.15.148:0/615171242 wait complete. 2024-04-25T09:57:33.854 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:57:33.854 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:33.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.948+0000 7f99b854c640 1 Processor -- start 2024-04-25T09:57:33.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.948+0000 7f99b854c640 1 -- start start 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b854c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99b015dbe0 0x7f99b00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b854c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99b00b2cb0 0x7f99b00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b854c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b00b97a0 con 0x7f99b00b92a0 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b854c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b015dfc0 con 0x7f99b015dbe0 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b854c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b00d1a00 con 0x7f99b00b2cb0 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b62e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99b015dbe0 0x7f99b00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b62e8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99b015dbe0 0x7f99b00d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40490/0 (socket says 172.21.15.148:40490) 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.949+0000 7f99b62e8640 1 -- 172.21.15.148:0/4242974063 learned_addr learned my addr 172.21.15.148:0/4242974063 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b5ae7640 1 --2- 172.21.15.148:0/4242974063 >> v2:172.21.15.188:3300/0 conn(0x7f99b00b2cb0 0x7f99b00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b62e8640 1 -- 172.21.15.148:0/4242974063 >> v2:172.21.15.188:3300/0 conn(0x7f99b00b2cb0 msgr2=0x7f99b00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b62e8640 1 --2- 172.21.15.148:0/4242974063 >> v2:172.21.15.188:3300/0 conn(0x7f99b00b2cb0 0x7f99b00d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b62e8640 1 -- 172.21.15.148:0/4242974063 >> v1:172.21.15.148:6789/0 conn(0x7f99b00b92a0 legacy=0x7f99b00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b62e8640 1 -- 172.21.15.148:0/4242974063 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b00d1c50 con 0x7f99b015dbe0 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b52e6640 1 -- 172.21.15.148:0/4242974063 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598441177 0 0) 0x7f99b00b97a0 con 0x7f99b00b92a0 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.950+0000 7f99b62e8640 1 --2- 172.21.15.148:0/4242974063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00d0910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f99b006cff0 tx=0x7f99a802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bdddd20f6865fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b52e6640 1 -- 172.21.15.148:0/4242974063 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8037070 con 0x7f99b015dbe0 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b52e6640 1 -- 172.21.15.148:0/4242974063 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a8032440 con 0x7f99b015dbe0 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b52e6640 1 -- 172.21.15.148:0/4242974063 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8032810 con 0x7f99b015dbe0 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 -- 172.21.15.148:0/4242974063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 msgr2=0x7f99b00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 --2- 172.21.15.148:0/4242974063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00d0910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f99b006cff0 tx=0x7f99a802a200 comp rx=0 tx=0).stop 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 -- 172.21.15.148:0/4242974063 shutdown_connections 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 --2- 172.21.15.148:0/4242974063 >> v2:172.21.15.188:3300/0 conn(0x7f99b00b2cb0 0x7f99b00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 --2- 172.21.15.148:0/4242974063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00d0910 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 -- 172.21.15.148:0/4242974063 >> 172.21.15.148:0/4242974063 conn(0x7f99b0061a20 msgr2=0x7f99b0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 -- 172.21.15.148:0/4242974063 shutdown_connections 2024-04-25T09:57:33.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.951+0000 7f99b854c640 1 -- 172.21.15.148:0/4242974063 wait complete. 2024-04-25T09:57:33.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.952+0000 7f99b854c640 1 Processor -- start 2024-04-25T09:57:33.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.952+0000 7f99b854c640 1 -- start start 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b854c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b00b2cb0 0x7f99b01f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b854c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b00b92a0 0x7f99b0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b854c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b854c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00b6ab0 con 0x7f99b015dbe0 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b854c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00d2210 con 0x7f99b00b92a0 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b854c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00b6c30 con 0x7f99b00b2cb0 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40494/0 (socket says 172.21.15.148:40494) 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 -- 172.21.15.148:0/3919003680 learned_addr learned my addr 172.21.15.148:0/3919003680 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b5ae7640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b00b92a0 0x7f99b0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b62e8640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b00b2cb0 0x7f99b01f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 -- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b00b2cb0 msgr2=0x7f99b01f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b00b2cb0 0x7f99b01f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 -- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b00b92a0 msgr2=0x7f99b0151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b00b92a0 0x7f99b0151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b6ae9640 1 -- 172.21.15.148:0/3919003680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b01f4910 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.953+0000 7f99b5ae7640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b00b92a0 0x7f99b0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.954+0000 7f99b6ae9640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00b32e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00c730 tx=0x7f99ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.954+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac003da0 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.954+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac00ce40 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.954+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0052b0 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.954+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b01f4bb0 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.954+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b01f5170 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.955+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b0073990 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.955+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99ac005450 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.955+0000 7f99a6ffd640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f999406b4d0 0x7f999406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.956+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99ac08a290 con 0x7f99b015dbe0 2024-04-25T09:57:33.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.956+0000 7f99b62e8640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f999406b4d0 0x7f999406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:33.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.956+0000 7f99b62e8640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f999406b4d0 0x7f999406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f99a8034000 tx=0x7f99a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:33.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:33.960+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99ac058f80 con 0x7f99b015dbe0 2024-04-25T09:57:34.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.170+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99b01523d0 con 0x7f99b015dbe0 2024-04-25T09:57:34.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.171+0000 7f99a6ffd640 1 -- 172.21.15.148:0/3919003680 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f99ac058880 con 0x7f99b015dbe0 2024-04-25T09:57:34.174 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:34.174 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:31.018031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:34.174 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2024-04-25T09:57:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.172+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f999406b4d0 msgr2=0x7f999406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.172+0000 7f99b854c640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f999406b4d0 0x7f999406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f99a8034000 tx=0x7f99a802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:34.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.172+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 msgr2=0x7f99b00b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.172+0000 7f99b854c640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00b32e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00c730 tx=0x7f99ac00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 shutdown_connections 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f999406b4d0 0x7f999406d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99b015dbe0 0x7f99b00b32e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b00b92a0 0x7f99b0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 --2- 172.21.15.148:0/3919003680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b00b2cb0 0x7f99b01f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 >> 172.21.15.148:0/3919003680 conn(0x7f99b0061a20 msgr2=0x7f99b01609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 shutdown_connections 2024-04-25T09:57:34.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.173+0000 7f99b854c640 1 -- 172.21.15.148:0/3919003680 wait complete. 2024-04-25T09:57:34.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T09:57:34.290 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:34.290 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:34.290 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:34.299 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:34.299 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:34.299 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T09:57:34.299 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T09:57:34.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:34.300 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T09:57:34.314 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> import json 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> import sys 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> sys.exit(e.errno) 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> print(json.dumps( 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T09:57:34.315 DEBUG:teuthology.orchestra.run.smithi148:> indent=2)) 2024-04-25T09:57:34.316 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_mode": 16877, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_ino": 1099511627777, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_dev": 43, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_nlink": 16, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_uid": 1000, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_gid": 1316, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_size": 18, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_atime": 1714039041.1876004, 2024-04-25T09:57:34.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_mtime": 1714039046.935459, 2024-04-25T09:57:34.423 INFO:teuthology.orchestra.run.smithi148.stdout: "st_ctime": 1714039046.935459 2024-04-25T09:57:34.423 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:34.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:57:34.430 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T09:57:34.431 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T09:57:34.431 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:34.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.565+0000 7f79ddeee640 1 Processor -- start 2024-04-25T09:57:34.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.565+0000 7f79ddeee640 1 -- start start 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79ddeee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d815dc00 0x7f79d80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79ddeee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79d80ceb20 0x7f79d80d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79ddeee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f79d80d1c00 0x7f79d80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79ddeee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80d2060 con 0x7f79d80ceb20 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79ddeee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d815dfe0 con 0x7f79d80d1c00 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79ddeee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80d00c0 con 0x7f79d815dc00 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79d80ceb20 0x7f79d80d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79d80ceb20 0x7f79d80d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40506/0 (socket says 172.21.15.148:40506) 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 -- 172.21.15.148:0/77645600 learned_addr learned my addr 172.21.15.148:0/77645600 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 -- 172.21.15.148:0/77645600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d815dc00 msgr2=0x7f79d80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/77645600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d815dc00 0x7f79d80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 -- 172.21.15.148:0/77645600 >> v2:172.21.15.188:3300/0 conn(0x7f79d80d1c00 msgr2=0x7f79d80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:34.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/77645600 >> v2:172.21.15.188:3300/0 conn(0x7f79d80d1c00 0x7f79d80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.566+0000 7f79d6ffd640 1 -- 172.21.15.148:0/77645600 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80d03a0 con 0x7f79d80ceb20 2024-04-25T09:57:34.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.567+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/77645600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d80d16c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f79d80b7a10 tx=0x7f79cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13006306b0c9cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.567+0000 7f79d67fc640 1 -- 172.21.15.148:0/77645600 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc037070 con 0x7f79d80ceb20 2024-04-25T09:57:34.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.567+0000 7f79d67fc640 1 -- 172.21.15.148:0/77645600 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc032440 con 0x7f79d80ceb20 2024-04-25T09:57:34.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.567+0000 7f79d67fc640 1 -- 172.21.15.148:0/77645600 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc032810 con 0x7f79d80ceb20 2024-04-25T09:57:34.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 -- 172.21.15.148:0/77645600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 msgr2=0x7f79d80d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 --2- 172.21.15.148:0/77645600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d80d16c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f79d80b7a10 tx=0x7f79cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 -- 172.21.15.148:0/77645600 shutdown_connections 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 --2- 172.21.15.148:0/77645600 >> v2:172.21.15.188:3300/0 conn(0x7f79d80d1c00 0x7f79d80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 --2- 172.21.15.148:0/77645600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d80d16c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 --2- 172.21.15.148:0/77645600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d815dc00 0x7f79d80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 -- 172.21.15.148:0/77645600 >> 172.21.15.148:0/77645600 conn(0x7f79d806f5d0 msgr2=0x7f79d806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 -- 172.21.15.148:0/77645600 shutdown_connections 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.568+0000 7f79ddeee640 1 -- 172.21.15.148:0/77645600 wait complete. 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.569+0000 7f79ddeee640 1 Processor -- start 2024-04-25T09:57:34.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.569+0000 7f79ddeee640 1 -- start start 2024-04-25T09:57:34.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d8151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d80d1c00 0x7f79d81e87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d815dc00 0x7f79d81e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cc130 con 0x7f79d80ceb20 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cc720 con 0x7f79d815dc00 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d80cfd90 con 0x7f79d80d1c00 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d8151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d8151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40516/0 (socket says 172.21.15.148:40516) 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 -- 172.21.15.148:0/3875303418 learned_addr learned my addr 172.21.15.148:0/3875303418 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d7fff640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d815dc00 0x7f79d81e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d80d1c00 0x7f79d81e87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 -- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d80d1c00 msgr2=0x7f79d81e87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d80d1c00 0x7f79d81e87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 -- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d815dc00 msgr2=0x7f79d81e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d815dc00 0x7f79d81e7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 -- 172.21.15.148:0/3875303418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d81e9180 con 0x7f79d80ceb20 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d7fff640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d815dc00 0x7f79d81e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d80d1c00 0x7f79d81e87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79d77fe640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d8151010 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f79c800b4a0 tx=0x7f79c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8014070 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c8010610 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d81e9460 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.570+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8010a00 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.571+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d81e9a20 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.572+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d8073990 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.572+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79c8017070 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.572+0000 7f79bffff640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79c406b4d0 0x7f79c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.572+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79c8089d40 con 0x7f79d80ceb20 2024-04-25T09:57:34.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.572+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79c406b4d0 0x7f79c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.573+0000 7f79d6ffd640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79c406b4d0 0x7f79c406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f79cc034000 tx=0x7f79cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:34.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.577+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79c8058ac0 con 0x7f79d80ceb20 2024-04-25T09:57:34.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.784+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79d815afb0 con 0x7f79d80ceb20 2024-04-25T09:57:34.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.785+0000 7f79bffff640 1 -- 172.21.15.148:0/3875303418 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f79c80583c0 con 0x7f79d80ceb20 2024-04-25T09:57:34.788 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:34.788 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:31.018031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:34.789 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2024-04-25T09:57:34.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79c406b4d0 msgr2=0x7f79c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79c406b4d0 0x7f79c406d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f79cc034000 tx=0x7f79cc02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 msgr2=0x7f79d8151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d8151010 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f79c800b4a0 tx=0x7f79c800b960 comp rx=0 tx=0).stop 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 shutdown_connections 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79c406b4d0 0x7f79c406d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d815dc00 0x7f79d81e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d80d1c00 0x7f79d81e87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 --2- 172.21.15.148:0/3875303418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79d80ceb20 0x7f79d8151010 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 >> 172.21.15.148:0/3875303418 conn(0x7f79d806f5d0 msgr2=0x7f79d80c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 shutdown_connections 2024-04-25T09:57:34.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:34.787+0000 7f79ddeee640 1 -- 172.21.15.148:0/3875303418 wait complete. 2024-04-25T09:57:34.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T09:57:34.904 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:34.904 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": -2 2024-04-25T09:57:34.904 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:34.912 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T09:57:34.913 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T09:57:34.913 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:34.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T09:57:35.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.013+0000 7f29877f9640 1 Processor -- start 2024-04-25T09:57:35.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.013+0000 7f29877f9640 1 -- start start 2024-04-25T09:57:35.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f29877f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29800ced80 0x7f29800d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f29877f9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298015e000 con 0x7f298015dc20 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f29877f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29800d1de0 con 0x7f2980161710 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f29877f9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2980161c60 con 0x7f29800ced80 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2984d94640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29800ced80 0x7f29800d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2985595640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f298015dc20 0x7f29800ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37482/0 (socket says 172.21.15.148:37482) 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2984d94640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f29800ced80 0x7f29800d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40526/0 (socket says 172.21.15.148:40526) 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2985595640 1 -- 172.21.15.148:0/2206465092 learned_addr learned my addr 172.21.15.148:0/2206465092 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2984d94640 1 -- 172.21.15.148:0/2206465092 >> v1:172.21.15.188:6789/0 conn(0x7f2980161710 legacy=0x7f29800d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2984d94640 1 -- 172.21.15.148:0/2206465092 >> v1:172.21.15.148:6789/0 conn(0x7f298015dc20 legacy=0x7f29800ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2977fff640 1 -- 172.21.15.148:0/2206465092 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855427140 0 0) 0x7f298015e000 con 0x7f298015dc20 2024-04-25T09:57:35.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.014+0000 7f2984d94640 1 -- 172.21.15.148:0/2206465092 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980161f40 con 0x7f29800ced80 2024-04-25T09:57:35.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.015+0000 7f2984d94640 1 --2- 172.21.15.148:0/2206465092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29800ced80 0x7f29800d22e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f29700097a0 tx=0x7f297002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c220feb4ffedd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.015+0000 7f2977fff640 1 -- 172.21.15.148:0/2206465092 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970037070 con 0x7f29800ced80 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.015+0000 7f2977fff640 1 -- 172.21.15.148:0/2206465092 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970032440 con 0x7f29800ced80 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.015+0000 7f2977fff640 1 -- 172.21.15.148:0/2206465092 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970032810 con 0x7f29800ced80 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.015+0000 7f29877f9640 1 -- 172.21.15.148:0/2206465092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29800ced80 msgr2=0x7f29800d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.015+0000 7f29877f9640 1 --2- 172.21.15.148:0/2206465092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29800ced80 0x7f29800d22e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f29700097a0 tx=0x7f297002a230 comp rx=0 tx=0).stop 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.016+0000 7f29877f9640 1 -- 172.21.15.148:0/2206465092 shutdown_connections 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.016+0000 7f29877f9640 1 --2- 172.21.15.148:0/2206465092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29800ced80 0x7f29800d22e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.016+0000 7f29877f9640 1 -- 172.21.15.148:0/2206465092 >> 172.21.15.148:0/2206465092 conn(0x7f29800d0420 msgr2=0x7f29800d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.016+0000 7f29877f9640 1 -- 172.21.15.148:0/2206465092 shutdown_connections 2024-04-25T09:57:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.016+0000 7f29877f9640 1 -- 172.21.15.148:0/2206465092 wait complete. 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 Processor -- start 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 -- start start 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29800ced80 0x7f29800c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 0x7f29800b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2980161710 0x7f29801f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b5e50 con 0x7f298015dc20 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2980162500 con 0x7f29800ced80 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.017+0000 7f29877f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b5fd0 con 0x7f2980161710 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 0x7f29800b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 0x7f29800b8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40528/0 (socket says 172.21.15.148:40528) 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 -- 172.21.15.148:0/1568376777 learned_addr learned my addr 172.21.15.148:0/1568376777 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2985595640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29800ced80 0x7f29800c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 -- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2980161710 msgr2=0x7f29801f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2985d96640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2980161710 0x7f29801f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2980161710 0x7f29801f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 -- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29800ced80 msgr2=0x7f29800c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29800ced80 0x7f29800c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 -- 172.21.15.148:0/1568376777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29801f7590 con 0x7f298015dc20 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2985595640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29800ced80 0x7f29800c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2985d96640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2980161710 0x7f29801f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2984d94640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 0x7f29800b8690 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f29700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2975ffb640 1 -- 172.21.15.148:0/1568376777 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970037040 con 0x7f298015dc20 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2975ffb640 1 -- 172.21.15.148:0/1568376777 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970042070 con 0x7f298015dc20 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f2975ffb640 1 -- 172.21.15.148:0/1568376777 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970030a70 con 0x7f298015dc20 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29800cd850 con 0x7f298015dc20 2024-04-25T09:57:35.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.018+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29800cd520 con 0x7f298015dc20 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.020+0000 7f2975ffb640 1 -- 172.21.15.148:0/1568376777 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2970030c10 con 0x7f298015dc20 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.020+0000 7f2975ffb640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296406b4d0 0x7f296406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.021+0000 7f2975ffb640 1 -- 172.21.15.148:0/1568376777 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29700aae10 con 0x7f298015dc20 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.021+0000 7f2985595640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296406b4d0 0x7f296406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.021+0000 7f2985595640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296406b4d0 0x7f296406d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f297c000f60 tx=0x7f297c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.021+0000 7f298a5d9740 1 Processor -- start 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 -- start start 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf62a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf7ff50 0x55b6fdf804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 0x55b6fdf7e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fde9ac60 con 0x55b6fdf38390 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fde43ee0 con 0x55b6fdf626b0 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6fde96d00 con 0x55b6fdf7ff50 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f2985d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 0x55b6fdf7e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f2985d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 0x55b6fdf7e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40532/0 (socket says 172.21.15.148:40532) 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.022+0000 7f2985d96640 1 -- 172.21.15.148:0/1646955815 learned_addr learned my addr 172.21.15.148:0/1646955815 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985d96640 1 -- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf7ff50 msgr2=0x55b6fdf804a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2984d94640 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf7ff50 0x55b6fdf804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985595640 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf62a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985d96640 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf7ff50 0x55b6fdf804a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985d96640 1 -- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 msgr2=0x55b6fdf62a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985d96640 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf62a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985d96640 1 -- 172.21.15.148:0/1646955815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fdc7b9f0 con 0x55b6fdf38390 2024-04-25T09:57:35.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2985d96640 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 0x55b6fdf7e610 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2978001e40 tx=0x7f297800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bfb99411652659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2962ffd640 1 -- 172.21.15.148:0/1646955815 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978018070 con 0x55b6fdf38390 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2962ffd640 1 -- 172.21.15.148:0/1646955815 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800de30 con 0x55b6fdf38390 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f2962ffd640 1 -- 172.21.15.148:0/1646955815 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978003b90 con 0x55b6fdf38390 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f298a5d9740 1 -- 172.21.15.148:0/1646955815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 msgr2=0x55b6fdf7e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.023+0000 7f298a5d9740 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 0x55b6fdf7e610 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2978001e40 tx=0x7f297800d520 comp rx=0 tx=0).stop 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 -- 172.21.15.148:0/1646955815 shutdown_connections 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf38390 0x55b6fdf7e610 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf7ff50 0x55b6fdf804a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 --2- 172.21.15.148:0/1646955815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf62a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 -- 172.21.15.148:0/1646955815 >> 172.21.15.148:0/1646955815 conn(0x55b6fdf34cc0 msgr2=0x55b6fdf7d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 -- 172.21.15.148:0/1646955815 shutdown_connections 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 -- 172.21.15.148:0/1646955815 wait complete. 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 -- start start 2024-04-25T09:57:35.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.024+0000 7f298a5d9740 1 Processor -- start 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf5ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf75860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf77130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fdf39130 con 0x55b6fdf643c0 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fde57470 con 0x55b6fdf7ff50 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6fdf32bf0 con 0x55b6fdf626b0 2024-04-25T09:57:35.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f298a5d9740 10 client.0 _ll_register_callbacks cb 0x55b6fde237d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f2985d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf77130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f2985d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf77130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40536/0 (socket says 172.21.15.148:40536) 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f2985d96640 1 -- 172.21.15.148:0/979321110 learned_addr learned my addr 172.21.15.148:0/979321110 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f2985595640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf5ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.025+0000 7f2984d94640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf75860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985d96640 1 -- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 msgr2=0x55b6fdf5ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985d96640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf5ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985d96640 1 -- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 msgr2=0x55b6fdf75860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985d96640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf75860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985d96640 1 -- 172.21.15.148:0/979321110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6fdb6f2e0 con 0x55b6fdf643c0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985595640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf5ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985d96640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf77130 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2978009010 tx=0x7f2978001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bfff8d2b4767d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f295bfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f295bfff640 1 -- 172.21.15.148:0/979321110 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978018040 con 0x55b6fdf643c0 2024-04-25T09:57:35.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f295bfff640 1 -- 172.21.15.148:0/979321110 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2978003b70 con 0x55b6fdf643c0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f297800de30 con 0x55b6fdf643c0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f295bfff640 1 -- 172.21.15.148:0/979321110 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297802a050 con 0x55b6fdf643c0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 10 client.6312 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b6fdcdc180 con 0x55b6fdf643c0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f295bfff640 1 -- 172.21.15.148:0/979321110 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800de30 con 0x55b6fdf643c0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f295bfff640 1 -- 172.21.15.148:0/979321110 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f297802d020 con 0x55b6fdf643c0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 10 client.6312 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 10 client.6312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 20 client.6312 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf64890 0x55b6fdf90d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 4 client.6312 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6fdea81a0 con 0x55b6fdf64890 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.026+0000 7f2985595640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf64890 0x55b6fdf90d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f295bfff640 10 client.6312 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f2985595640 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf64890 0x55b6fdf90d70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2ffe835f1f38e1 server_cookie=4c3d41b362552d30 in_seq=0 out_seq=0 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f295bfff640 1 -- 172.21.15.148:0/979321110 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6fdea81a0 con 0x55b6fdf64890 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f295bfff640 10 client.6312 handle_command_reply: tid=0 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 client.6312 shutdown 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 msgr2=0x55b6fdf77130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf77130 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2978009010 tx=0x7f2978001250 comp rx=0 tx=0).stop 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 shutdown_connections 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf77130 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf64890 0x55b6fdf90d70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf75860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 --2- 172.21.15.148:0/979321110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf5ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f295bfff640 0 client.6312 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.027+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 >> 172.21.15.148:0/979321110 conn(0x55b6fdf38390 msgr2=0x55b6fdf80a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.028+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 shutdown_connections 2024-04-25T09:57:35.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.028+0000 7f298a5d9740 1 -- 172.21.15.148:0/979321110 wait complete. 2024-04-25T09:57:35.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.028+0000 7f298a5d9740 20 client.6312 trim_cache size 0 max 16384 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.043+0000 7f298a5d9740 1 Processor -- start 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.043+0000 7f298a5d9740 1 -- start start 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf643c0 0x55b6fdf66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f298a5d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29802002e0 con 0x55b6fdf7ff50 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801ff4a0 con 0x55b6fdf626b0 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800c7d30 con 0x55b6fdf643c0 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf8b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40548/0 (socket says 172.21.15.148:40548) 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 -- 172.21.15.148:0/163039033 learned_addr learned my addr 172.21.15.148:0/163039033 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 -- 172.21.15.148:0/163039033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf643c0 msgr2=0x55b6fdf66150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf643c0 0x55b6fdf66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 -- 172.21.15.148:0/163039033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 msgr2=0x55b6fdf31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 -- 172.21.15.148:0/163039033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6fdb483a0 con 0x55b6fdf7ff50 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f2985d96640 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf8b810 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2978007e10 tx=0x7f297801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9125919bae6109e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f29627fc640 1 -- 172.21.15.148:0/163039033 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978018040 con 0x55b6fdf7ff50 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f29627fc640 1 -- 172.21.15.148:0/163039033 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978022070 con 0x55b6fdf7ff50 2024-04-25T09:57:35.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.044+0000 7f29627fc640 1 -- 172.21.15.148:0/163039033 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978003b70 con 0x55b6fdf7ff50 2024-04-25T09:57:35.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 -- 172.21.15.148:0/163039033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 msgr2=0x55b6fdf8b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf8b810 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2978007e10 tx=0x7f297801c000 comp rx=0 tx=0).stop 2024-04-25T09:57:35.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 -- 172.21.15.148:0/163039033 shutdown_connections 2024-04-25T09:57:35.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf8b810 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf643c0 0x55b6fdf66150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 --2- 172.21.15.148:0/163039033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf626b0 0x55b6fdf31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 -- 172.21.15.148:0/163039033 >> 172.21.15.148:0/163039033 conn(0x55b6fdf38390 msgr2=0x55b6fdf38820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 -- 172.21.15.148:0/163039033 shutdown_connections 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 -- 172.21.15.148:0/163039033 wait complete. 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.045+0000 7f298a5d9740 1 -- start start 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 Processor -- start 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf7e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf90b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6fdf753e0 con 0x55b6fdf643c0 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29801fef00 con 0x55b6fdf7ff50 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b6fdf8cc60 con 0x55b6fdf626b0 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f298a5d9740 10 client.0 _ll_register_callbacks cb 0x55b6fdf330d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.046+0000 7f2984d94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf7e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf7e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40562/0 (socket says 172.21.15.148:40562) 2024-04-25T09:57:35.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 -- 172.21.15.148:0/654394826 learned_addr learned my addr 172.21.15.148:0/654394826 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 -- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 msgr2=0x55b6fdf36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2985595640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2985d96640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf90b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf36260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 -- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 msgr2=0x55b6fdf90b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf90b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 -- 172.21.15.148:0/654394826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6fdf73af0 con 0x55b6fdf643c0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2985595640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2985d96640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf90b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f2984d94640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf7e220 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2970079190 tx=0x7f2970078f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91929cd09aa266a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f295bfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f295bfff640 1 -- 172.21.15.148:0/654394826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970091020 con 0x55b6fdf643c0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f295bfff640 1 -- 172.21.15.148:0/654394826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29700798f0 con 0x55b6fdf643c0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2978022070 con 0x55b6fdf643c0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f295bfff640 1 -- 172.21.15.148:0/654394826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297007d5e0 con 0x55b6fdf643c0 2024-04-25T09:57:35.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 10 client.6318 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b6fdf80390 con 0x55b6fdf643c0 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f295bfff640 1 -- 172.21.15.148:0/654394826 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2978022070 con 0x55b6fdf643c0 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f295bfff640 1 -- 172.21.15.148:0/654394826 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f297003a030 con 0x55b6fdf643c0 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 10 client.6318 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 10 client.6318 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 20 client.6318 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.047+0000 7f298a5d9740 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf72f30 0x55b6fdf91040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 4 client.6318 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55b6fdea24f0 con 0x55b6fdf72f30 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f2985595640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf72f30 0x55b6fdf91040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f295bfff640 10 client.6318 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f2985595640 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf72f30 0x55b6fdf91040 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e10c97c480df24 server_cookie=736fc4508fc5af6c in_seq=0 out_seq=0 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f295bfff640 1 -- 172.21.15.148:0/654394826 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55b6fdea24f0 con 0x55b6fdf72f30 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f295bfff640 10 client.6318 handle_command_reply: tid=0 2024-04-25T09:57:35.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 client.6318 shutdown 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 msgr2=0x55b6fdf7e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf7e220 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2970079190 tx=0x7f2970078f20 comp rx=0 tx=0).stop 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 shutdown_connections 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55b6fdf72f30 0x55b6fdf91040 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6fdf7ff50 0x55b6fdf90b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6fdf643c0 0x55b6fdf7e220 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 --2- 172.21.15.148:0/654394826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b6fdf626b0 0x55b6fdf36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f295bfff640 0 client.6318 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.048+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 >> 172.21.15.148:0/654394826 conn(0x55b6fdf38390 msgr2=0x55b6fdf38820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 shutdown_connections 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f298a5d9740 1 -- 172.21.15.148:0/654394826 wait complete. 2024-04-25T09:57:35.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f298a5d9740 20 client.6318 trim_cache size 0 max 16384 2024-04-25T09:57:35.059 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:35.059 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": -2 2024-04-25T09:57:35.059 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296406b4d0 msgr2=0x7f296406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296406b4d0 0x7f296406d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f297c000f60 tx=0x7f297c009040 comp rx=0 tx=0).stop 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 msgr2=0x7f29800b8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 0x7f29800b8690 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f29700046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 shutdown_connections 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f296406b4d0 0x7f296406d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2980161710 0x7f29801f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f298015dc20 0x7f29800b8690 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 --2- 172.21.15.148:0/1568376777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29800ced80 0x7f29800c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 >> 172.21.15.148:0/1568376777 conn(0x7f29800d0420 msgr2=0x7f298006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 shutdown_connections 2024-04-25T09:57:35.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.049+0000 7f29877f9640 1 -- 172.21.15.148:0/1568376777 wait complete. 2024-04-25T09:57:35.063 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T09:57:35.063 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T09:57:35.063 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:35.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T09:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.160+0000 7fb8fb36a640 1 Processor -- start 2024-04-25T09:57:35.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8fb36a640 1 -- start start 2024-04-25T09:57:35.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8fb36a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8fb36a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8f41616f0 0x7fb8f4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8fb36a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f40c6960 con 0x7fb8f40ce0a0 2024-04-25T09:57:35.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8fb36a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f41614d0 con 0x7fb8f41616f0 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8fb36a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f4162260 con 0x7fb8f41610f0 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.161+0000 7fb8f8905640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8f41616f0 0x7fb8f4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f9907640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb8f40ce0a0 0x7fb8f40d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37488/0 (socket says 172.21.15.148:37488) 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f8905640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8f41616f0 0x7fb8f4161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40578/0 (socket says 172.21.15.148:40578) 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f9907640 1 -- 172.21.15.148:0/3442859865 learned_addr learned my addr 172.21.15.148:0/3442859865 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f8905640 1 -- 172.21.15.148:0/3442859865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 msgr2=0x7fb8f406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f8905640 1 -- 172.21.15.148:0/3442859865 >> v1:172.21.15.148:6789/0 conn(0x7fb8f40ce0a0 legacy=0x7fb8f40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f8905640 1 -- 172.21.15.148:0/3442859865 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f4162540 con 0x7fb8f41616f0 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8ebfff640 1 -- 172.21.15.148:0/3442859865 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901067333 0 0) 0x7fb8f40c6960 con 0x7fb8f40ce0a0 2024-04-25T09:57:35.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.162+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f4161b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00097a0 tx=0x7fb8f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7601f4ef1d1e73f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8ebfff640 1 -- 172.21.15.148:0/3442859865 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0037070 con 0x7fb8f41616f0 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8ebfff640 1 -- 172.21.15.148:0/3442859865 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f0032440 con 0x7fb8f41616f0 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8ebfff640 1 -- 172.21.15.148:0/3442859865 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0032810 con 0x7fb8f41616f0 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 -- 172.21.15.148:0/3442859865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 msgr2=0x7fb8f4161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f4161b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00097a0 tx=0x7fb8f002a180 comp rx=0 tx=0).stop 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 -- 172.21.15.148:0/3442859865 shutdown_connections 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f4161b50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/3442859865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 -- 172.21.15.148:0/3442859865 >> 172.21.15.148:0/3442859865 conn(0x7fb8f406f500 msgr2=0x7fb8f406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 -- 172.21.15.148:0/3442859865 shutdown_connections 2024-04-25T09:57:35.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.163+0000 7fb8fb36a640 1 -- 172.21.15.148:0/3442859865 wait complete. 2024-04-25T09:57:35.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.164+0000 7fb8fb36a640 1 Processor -- start 2024-04-25T09:57:35.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.164+0000 7fb8fb36a640 1 -- start start 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8fb36a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f40ce0a0 0x7fb8f415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8fb36a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f40b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8fb36a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f41530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8fb36a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40cf690 con 0x7fb8f41616f0 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8fb36a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40cfce0 con 0x7fb8f40ce0a0 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8fb36a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f406c700 con 0x7fb8f41610f0 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8f9907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f41530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8f9907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f41530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40590/0 (socket says 172.21.15.148:40590) 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8f9907640 1 -- 172.21.15.148:0/1118476107 learned_addr learned my addr 172.21.15.148:0/1118476107 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8f9106640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f40ce0a0 0x7fb8f415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.165+0000 7fb8f8905640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f40b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9907640 1 -- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 msgr2=0x7fb8f40b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9907640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f40b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9907640 1 -- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f40ce0a0 msgr2=0x7fb8f415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9907640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f40ce0a0 0x7fb8f415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9907640 1 -- 172.21.15.148:0/1118476107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f415b200 con 0x7fb8f41616f0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9106640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f40ce0a0 0x7fb8f415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f8905640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f40b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8f9907640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f41530a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c730 tx=0x7fb8ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/1118476107 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec018070 con 0x7fb8f41616f0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/1118476107 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec00ce40 con 0x7fb8f41616f0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f415b4e0 con 0x7fb8f41616f0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/1118476107 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00a860 con 0x7fb8f41616f0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.166+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f415f290 con 0x7fb8f41616f0 2024-04-25T09:57:35.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.168+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/1118476107 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8ec01b070 con 0x7fb8f41616f0 2024-04-25T09:57:35.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.168+0000 7fb8e9ffb640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8d806b4d0 0x7fb8d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.168+0000 7fb8e9ffb640 1 -- 172.21.15.148:0/1118476107 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8ec089990 con 0x7fb8f41616f0 2024-04-25T09:57:35.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.168+0000 7fb8f9106640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8d806b4d0 0x7fb8d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.168+0000 7fb8f9106640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8d806b4d0 0x7fb8d806d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb8f415e800 tx=0x7fb8e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.169+0000 7fb8fe073740 1 Processor -- start 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.169+0000 7fb8fe073740 1 -- start start 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8fe073740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba414d4fe0 con 0x55ba415a2f10 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba41506d20 con 0x55ba415cd9d0 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba414affa0 con 0x55ba415df5e0 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40592/0 (socket says 172.21.15.148:40592) 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 -- 172.21.15.148:0/3575022635 learned_addr learned my addr 172.21.15.148:0/3575022635 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 -- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 msgr2=0x55ba415a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9907640 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 -- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 msgr2=0x55ba415cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 -- 172.21.15.148:0/3575022635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba412e7b00 con 0x55ba415a2f10 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415a5070 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400fae0 tx=0x7fb8e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d10e5646ca36b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8deffd640 1 -- 172.21.15.148:0/3575022635 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e401b050 con 0x55ba415a2f10 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8deffd640 1 -- 172.21.15.148:0/3575022635 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e4025070 con 0x55ba415a2f10 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.170+0000 7fb8deffd640 1 -- 172.21.15.148:0/3575022635 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4020440 con 0x55ba415a2f10 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 -- 172.21.15.148:0/3575022635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 msgr2=0x55ba415a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415a5070 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400fae0 tx=0x7fb8e4012000 comp rx=0 tx=0).stop 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 -- 172.21.15.148:0/3575022635 shutdown_connections 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3575022635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415a5070 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 -- 172.21.15.148:0/3575022635 >> 172.21.15.148:0/3575022635 conn(0x55ba4156b100 msgr2=0x55ba415e89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 -- 172.21.15.148:0/3575022635 shutdown_connections 2024-04-25T09:57:35.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.171+0000 7fb8fe073740 1 -- 172.21.15.148:0/3575022635 wait complete. 2024-04-25T09:57:35.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.172+0000 7fb8fe073740 1 -- start start 2024-04-25T09:57:35.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.172+0000 7fb8fe073740 1 Processor -- start 2024-04-25T09:57:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415cd9d0 0x55ba415bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415df5e0 0x55ba415e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba414a5750 con 0x55ba415a2f10 2024-04-25T09:57:35.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba41502dc0 con 0x55ba415df5e0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba415bd850 con 0x55ba415cd9d0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 10 client.0 _ll_register_callbacks cb 0x55ba414bdee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40594/0 (socket says 172.21.15.148:40594) 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 -- 172.21.15.148:0/3665530630 learned_addr learned my addr 172.21.15.148:0/3665530630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9907640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415df5e0 0x55ba415e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415cd9d0 0x55ba415bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 -- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415cd9d0 msgr2=0x55ba415bcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415cd9d0 0x55ba415bcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 -- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415df5e0 msgr2=0x55ba415e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415df5e0 0x55ba415e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 -- 172.21.15.148:0/3665530630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba411db3f0 con 0x55ba415a2f10 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415cb0d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400a410 tx=0x7fb8e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2cb2dbc2735dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3665530630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4020ab0 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3665530630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8e4035c70 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8e4025070 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.173+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3665530630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e401b020 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 10 client.6333 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ba41348290 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3665530630 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e4025070 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3665530630 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8e4020c50 con 0x55ba415a2f10 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 10 client.6333 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 10 client.6333 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 20 client.6333 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba415cf9c0 0x55ba415bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 4 client.6333 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba41514260 con 0x55ba415cf9c0 2024-04-25T09:57:35.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba415cf9c0 0x55ba415bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8d3fff640 10 client.6333 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.174+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba415cf9c0 0x55ba415bc0d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e69118eee7807c server_cookie=8033a7427d876e1f in_seq=0 out_seq=0 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3665530630 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba41514260 con 0x55ba415cf9c0 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8d3fff640 10 client.6333 handle_command_reply: tid=0 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 client.6333 shutdown 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 msgr2=0x55ba415cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415cb0d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400a410 tx=0x7fb8e4004730 comp rx=0 tx=0).stop 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 shutdown_connections 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba415cf9c0 0x55ba415bc0d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415df5e0 0x55ba415e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415cd9d0 0x55ba415bcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8d3fff640 0 client.6333 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3665530630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415a2f10 0x55ba415cb0d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 >> 172.21.15.148:0/3665530630 conn(0x55ba4156b100 msgr2=0x55ba415cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 shutdown_connections 2024-04-25T09:57:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 1 -- 172.21.15.148:0/3665530630 wait complete. 2024-04-25T09:57:35.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.175+0000 7fb8fe073740 20 client.6333 trim_cache size 0 max 16384 2024-04-25T09:57:35.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.190+0000 7fb8fe073740 1 Processor -- start 2024-04-25T09:57:35.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.190+0000 7fb8fe073740 1 -- start start 2024-04-25T09:57:35.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415a2f10 0x55ba415bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 0x55ba415e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8fe073740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f42003e0 con 0x55ba415cd9d0 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41ff5d0 con 0x55ba415a2f10 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415ffe0 con 0x55ba415df5e0 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 0x55ba415e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 0x55ba415e4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40606/0 (socket says 172.21.15.148:40606) 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 -- 172.21.15.148:0/3451495792 learned_addr learned my addr 172.21.15.148:0/3451495792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 -- 172.21.15.148:0/3451495792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 msgr2=0x55ba415d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 -- 172.21.15.148:0/3451495792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415a2f10 msgr2=0x55ba415bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415a2f10 0x55ba415bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 -- 172.21.15.148:0/3451495792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba411b44b0 con 0x55ba415cd9d0 2024-04-25T09:57:35.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8f8905640 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 0x55ba415e4cd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000c00 tx=0x7fb8f002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0b1ae133fb725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8de7fc640 1 -- 172.21.15.148:0/3451495792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0037040 con 0x55ba415cd9d0 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8de7fc640 1 -- 172.21.15.148:0/3451495792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f0045d80 con 0x55ba415cd9d0 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.191+0000 7fb8de7fc640 1 -- 172.21.15.148:0/3451495792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0004300 con 0x55ba415cd9d0 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 -- 172.21.15.148:0/3451495792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 msgr2=0x55ba415e4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 0x55ba415e4cd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0000c00 tx=0x7fb8f002c000 comp rx=0 tx=0).stop 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 -- 172.21.15.148:0/3451495792 shutdown_connections 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415df5e0 0x55ba415d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415cd9d0 0x55ba415e4cd0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3451495792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415a2f10 0x55ba415bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 -- 172.21.15.148:0/3451495792 >> 172.21.15.148:0/3451495792 conn(0x55ba4156b100 msgr2=0x55ba4159d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 -- 172.21.15.148:0/3451495792 shutdown_connections 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 -- 172.21.15.148:0/3451495792 wait complete. 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.192+0000 7fb8fe073740 1 -- start start 2024-04-25T09:57:35.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 Processor -- start 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415a2f10 0x55ba415ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 0x55ba41652dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba415e6530 con 0x55ba415df5e0 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f41605e0 con 0x55ba415cd9d0 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba415eb5c0 con 0x55ba415a2f10 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.193+0000 7fb8fe073740 10 client.0 _ll_register_callbacks cb 0x7fb8e400a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 0x55ba41652dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 0x55ba41652dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40608/0 (socket says 172.21.15.148:40608) 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 -- 172.21.15.148:0/3655329465 learned_addr learned my addr 172.21.15.148:0/3655329465 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 -- 172.21.15.148:0/3655329465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415a2f10 msgr2=0x55ba415ded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415a2f10 0x55ba415ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 -- 172.21.15.148:0/3655329465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 msgr2=0x55ba415bc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 -- 172.21.15.148:0/3655329465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba415d1710 con 0x55ba415df5e0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8f9907640 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 0x55ba41652dd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec060160 tx=0x7fb8ec058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f36bd3f68d079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3655329465 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec086060 con 0x55ba415df5e0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3655329465 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8ec087a40 con 0x55ba415df5e0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8f0045d80 con 0x55ba415df5e0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3655329465 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8ec074050 con 0x55ba415df5e0 2024-04-25T09:57:35.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8fe073740 10 client.6339 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ba415bbeb0 con 0x55ba415df5e0 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3655329465 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec076e50 con 0x55ba415df5e0 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3655329465 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8ec05e060 con 0x55ba415df5e0 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8fe073740 10 client.6339 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.194+0000 7fb8fe073740 10 client.6339 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8fe073740 20 client.6339 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba41653310 0x55ba415a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8fe073740 4 client.6339 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55ba4150e5b0 con 0x55ba41653310 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba41653310 0x55ba415a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8d3fff640 10 client.6339 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8f9106640 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba41653310 0x55ba415a15d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3a12b78a5ea55b server_cookie=a3b8b051559f1f8a in_seq=0 out_seq=0 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8d3fff640 1 -- 172.21.15.148:0/3655329465 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ba4150e5b0 con 0x55ba41653310 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8d3fff640 10 client.6339 handle_command_reply: tid=0 2024-04-25T09:57:35.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.195+0000 7fb8fe073740 1 client.6339 shutdown 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 msgr2=0x55ba41652dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 0x55ba41652dd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec060160 tx=0x7fb8ec058040 comp rx=0 tx=0).stop 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 shutdown_connections 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba415df5e0 0x55ba41652dd0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ba41653310 0x55ba415a15d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ba415cd9d0 0x55ba415bc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 --2- 172.21.15.148:0/3655329465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ba415a2f10 0x55ba415ded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8d3fff640 0 client.6339 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 >> 172.21.15.148:0/3655329465 conn(0x55ba4156b100 msgr2=0x55ba415e4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 shutdown_connections 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 1 -- 172.21.15.148:0/3655329465 wait complete. 2024-04-25T09:57:35.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.196+0000 7fb8fe073740 20 client.6339 trim_cache size 0 max 16384 2024-04-25T09:57:35.205 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:35.205 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:35.205 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "a0e7e637-acba-423d-9937-e5c2a14c7b12", 2024-04-25T09:57:35.205 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:35.205 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8d806b4d0 msgr2=0x7fb8d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8d806b4d0 0x7fb8d806d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb8f415e800 tx=0x7fb8e4008040 comp rx=0 tx=0).stop 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 msgr2=0x7fb8f41530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f41530a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c730 tx=0x7fb8ec00cbf0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 shutdown_connections 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8d806b4d0 0x7fb8d806d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8f41616f0 0x7fb8f41530a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f41610f0 0x7fb8f40b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 --2- 172.21.15.148:0/1118476107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f40ce0a0 0x7fb8f415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 >> 172.21.15.148:0/1118476107 conn(0x7fb8f406f500 msgr2=0x7fb8f4070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 shutdown_connections 2024-04-25T09:57:35.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.197+0000 7fb8fb36a640 1 -- 172.21.15.148:0/1118476107 wait complete. 2024-04-25T09:57:35.211 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'a0e7e637-acba-423d-9937-e5c2a14c7b12', 'mode': 'asynchronous'}' 2024-04-25T09:57:35.211 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T09:57:35.211 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:35.211 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.308+0000 7f145fc5a640 1 Processor -- start 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.308+0000 7f145fc5a640 1 -- start start 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145fc5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f145816a720 0x7f1458071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145fc5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f145806ee00 0x7f14580cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145fc5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145816e5a0 con 0x7f1458072050 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145fc5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14580724b0 con 0x7f145806ee00 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145fc5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145816f170 con 0x7f145816a720 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145e1f7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f145806ee00 0x7f14580cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145d1f5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1458072050 0x7f145806e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37496/0 (socket says 172.21.15.148:37496) 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145d1f5640 1 -- 172.21.15.148:0/3960396121 learned_addr learned my addr 172.21.15.148:0/3960396121 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3960396121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f145816a720 0x7f1458071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.309+0000 7f145c9f4640 1 -- 172.21.15.148:0/3960396121 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384541430 0 0) 0x7f145816e5a0 con 0x7f1458072050 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.347+0000 7f145c9f4640 1 -- 172.21.15.148:0/3960396121 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1444003500 con 0x7f1458072050 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.347+0000 7f145e1f7640 1 -- 172.21.15.148:0/3960396121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f145816a720 msgr2=0x7f1458071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.347+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3960396121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f145816a720 0x7f1458071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.347+0000 7f145c9f4640 1 -- 172.21.15.148:0/3960396121 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505698123 0 0) 0x7f1444003500 con 0x7f1458072050 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.347+0000 7f145e1f7640 1 -- 172.21.15.148:0/3960396121 >> v1:172.21.15.148:6789/0 conn(0x7f1458072050 legacy=0x7f145806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.347+0000 7f145e1f7640 1 -- 172.21.15.148:0/3960396121 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145816e840 con 0x7f145806ee00 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.348+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3960396121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145806ee00 0x7f14580cecc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1450013a60 tx=0x7f14500331f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2595c2187f9d4e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.348+0000 7f145c9f4640 1 -- 172.21.15.148:0/3960396121 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454001670 con 0x7f145806ee00 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.348+0000 7f145c9f4640 1 -- 172.21.15.148:0/3960396121 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450033c80 con 0x7f145806ee00 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.348+0000 7f145c9f4640 1 -- 172.21.15.148:0/3960396121 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000a2f0 con 0x7f145806ee00 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 -- 172.21.15.148:0/3960396121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145806ee00 msgr2=0x7f14580cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 --2- 172.21.15.148:0/3960396121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145806ee00 0x7f14580cecc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1450013a60 tx=0x7f14500331f0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 -- 172.21.15.148:0/3960396121 shutdown_connections 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 --2- 172.21.15.148:0/3960396121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145806ee00 0x7f14580cecc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 --2- 172.21.15.148:0/3960396121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f145816a720 0x7f1458071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 -- 172.21.15.148:0/3960396121 >> 172.21.15.148:0/3960396121 conn(0x7f145806abe0 msgr2=0x7f145806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 -- 172.21.15.148:0/3960396121 shutdown_connections 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.349+0000 7f145fc5a640 1 -- 172.21.15.148:0/3960396121 wait complete. 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.350+0000 7f145fc5a640 1 Processor -- start 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.350+0000 7f145fc5a640 1 -- start start 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f145806ee00 0x7f14580c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1458072050 0x7f1458164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 0x7f14580ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145816ab00 con 0x7f145816a720 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145816ee00 con 0x7f145806ee00 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145816ac80 con 0x7f1458072050 2024-04-25T09:57:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 0x7f14580ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 0x7f14580ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40632/0 (socket says 172.21.15.148:40632) 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 -- 172.21.15.148:0/2817438657 learned_addr learned my addr 172.21.15.148:0/2817438657 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 -- 172.21.15.148:0/2817438657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1458072050 msgr2=0x7f1458164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1458072050 0x7f1458164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 -- 172.21.15.148:0/2817438657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f145806ee00 msgr2=0x7f14580c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f145806ee00 0x7f14580c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 -- 172.21.15.148:0/2817438657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580a8f50 con 0x7f145816a720 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145e1f7640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 0x7f14580ca7c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1450013a60 tx=0x7f1450004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f144e7fc640 1 -- 172.21.15.148:0/2817438657 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145004b020 con 0x7f145816a720 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f144e7fc640 1 -- 172.21.15.148:0/2817438657 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450047880 con 0x7f145816a720 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14580c8950 con 0x7f145816a720 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f144e7fc640 1 -- 172.21.15.148:0/2817438657 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450047c30 con 0x7f145816a720 2024-04-25T09:57:35.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.351+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14580c8f10 con 0x7f145816a720 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.353+0000 7f144e7fc640 1 -- 172.21.15.148:0/2817438657 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f145004e020 con 0x7f145816a720 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.354+0000 7f144e7fc640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f143c06b500 0x7f143c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.354+0000 7f144e7fc640 1 -- 172.21.15.148:0/2817438657 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14500b4320 con 0x7f145816a720 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.354+0000 7f145d9f6640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f143c06b500 0x7f143c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.354+0000 7f145d9f6640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f143c06b500 0x7f143c06d9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f144800dbe0 tx=0x7f144800f040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 Processor -- start 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 -- start start 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c62590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd2b81e60 con 0x562dd2c45af0 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd2b2b120 con 0x562dd2c1f130 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd2b7df00 con 0x562dd2c64f60 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.355+0000 7f145e1f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c62590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c62590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40642/0 (socket says 172.21.15.148:40642) 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 -- 172.21.15.148:0/3155430981 learned_addr learned my addr 172.21.15.148:0/3155430981 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 -- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 msgr2=0x562dd2c631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 -- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 msgr2=0x562dd2c65340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c65340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 -- 172.21.15.148:0/3155430981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dd2962f00 con 0x562dd2c45af0 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c62590 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f14500075a0 tx=0x7f14500830a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9101bf7fd24f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f143b7fe640 1 -- 172.21.15.148:0/3155430981 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450086af0 con 0x562dd2c45af0 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f143b7fe640 1 -- 172.21.15.148:0/3155430981 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145008b0b0 con 0x562dd2c45af0 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.356+0000 7f143b7fe640 1 -- 172.21.15.148:0/3155430981 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450080ca0 con 0x562dd2c45af0 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 -- 172.21.15.148:0/3155430981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 msgr2=0x562dd2c62590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c62590 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f14500075a0 tx=0x7f14500830a0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 -- 172.21.15.148:0/3155430981 shutdown_connections 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c62590 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 --2- 172.21.15.148:0/3155430981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 -- 172.21.15.148:0/3155430981 >> 172.21.15.148:0/3155430981 conn(0x562dd2be5f70 msgr2=0x562dd2c1c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 -- 172.21.15.148:0/3155430981 shutdown_connections 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 -- 172.21.15.148:0/3155430981 wait complete. 2024-04-25T09:57:35.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 -- start start 2024-04-25T09:57:35.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.357+0000 7f1462a34740 1 Processor -- start 2024-04-25T09:57:35.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd2c64760 con 0x562dd2c45af0 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd2b20cd0 con 0x562dd2c1f130 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd2c64480 con 0x562dd2c64f60 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f1462a34740 10 client.0 _ll_register_callbacks cb 0x562dd28962e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f145d1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f145d1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40644/0 (socket says 172.21.15.148:40644) 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.358+0000 7f145d1f5640 1 -- 172.21.15.148:0/3856620364 learned_addr learned my addr 172.21.15.148:0/3856620364 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c3f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d1f5640 1 -- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 msgr2=0x562dd2c1e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c1e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d1f5640 1 -- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 msgr2=0x562dd2c3f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c3f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d1f5640 1 -- 172.21.15.148:0/3856620364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dd28567f0 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c488e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f145400d4b0 tx=0x7f145400d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc9578b58006524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1438ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1438ff9640 1 -- 172.21.15.148:0/3856620364 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454018060 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1438ff9640 1 -- 172.21.15.148:0/3856620364 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1454005ce0 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f145008b0b0 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1438ff9640 1 -- 172.21.15.148:0/3856620364 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14540070d0 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1462a34740 10 client.6354 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x562dd29c3690 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1438ff9640 1 -- 172.21.15.148:0/3856620364 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145008b0b0 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1438ff9640 1 -- 172.21.15.148:0/3856620364 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1454021080 con 0x562dd2c45af0 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1462a34740 10 client.6354 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.359+0000 7f1462a34740 10 client.6354 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1462a34740 20 client.6354 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1462a34740 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c1ce10 0x562dd2c1d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1462a34740 4 client.6354 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dd2b8f3a0 con 0x562dd2c1ce10 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c1ce10 0x562dd2c1d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1438ff9640 10 client.6354 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c1ce10 0x562dd2c1d210 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8621ce6013253d server_cookie=67f076f61863b66 in_seq=0 out_seq=0 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1438ff9640 1 -- 172.21.15.148:0/3856620364 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562dd2b8f3a0 con 0x562dd2c1ce10 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1438ff9640 10 client.6354 handle_command_reply: tid=0 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.360+0000 7f1462a34740 1 client.6354 shutdown 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 msgr2=0x562dd2c488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c488e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f145400d4b0 tx=0x7f145400d970 comp rx=0 tx=0).stop 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 shutdown_connections 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c1e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c488e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c1ce10 0x562dd2c1d210 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 --2- 172.21.15.148:0/3856620364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c3f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1438ff9640 0 client.6354 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 >> 172.21.15.148:0/3856620364 conn(0x562dd2be5f70 msgr2=0x562dd2c634f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 shutdown_connections 2024-04-25T09:57:35.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 1 -- 172.21.15.148:0/3856620364 wait complete. 2024-04-25T09:57:35.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.361+0000 7f1462a34740 20 client.6354 trim_cache size 0 max 16384 2024-04-25T09:57:35.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.376+0000 7f1462a34740 1 Processor -- start 2024-04-25T09:57:35.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.376+0000 7f1462a34740 1 -- start start 2024-04-25T09:57:35.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f1462a34740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c42eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f1462a34740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd2c7db80 con 0x562dd2c45af0 2024-04-25T09:57:35.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145820cda0 con 0x562dd2c1f130 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145820bf60 con 0x562dd2c64f60 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c42970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40656/0 (socket says 172.21.15.148:40656) 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 -- 172.21.15.148:0/3129593117 learned_addr learned my addr 172.21.15.148:0/3129593117 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 -- 172.21.15.148:0/3129593117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 msgr2=0x562dd2c42eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c42eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 -- 172.21.15.148:0/3129593117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 msgr2=0x562dd2c450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 -- 172.21.15.148:0/3129593117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dd2c656b0 con 0x562dd2c45af0 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c42970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1454016040 tx=0x7f145400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9ba50288710d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f143affd640 1 -- 172.21.15.148:0/3129593117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145402d070 con 0x562dd2c45af0 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f143affd640 1 -- 172.21.15.148:0/3129593117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454007270 con 0x562dd2c45af0 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.377+0000 7f143affd640 1 -- 172.21.15.148:0/3129593117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454007640 con 0x562dd2c45af0 2024-04-25T09:57:35.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 -- 172.21.15.148:0/3129593117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 msgr2=0x562dd2c42970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c42970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1454016040 tx=0x7f145400b000 comp rx=0 tx=0).stop 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 -- 172.21.15.148:0/3129593117 shutdown_connections 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c42eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c42970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 --2- 172.21.15.148:0/3129593117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2c450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 -- 172.21.15.148:0/3129593117 >> 172.21.15.148:0/3129593117 conn(0x562dd2be5f70 msgr2=0x562dd2c14440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 -- 172.21.15.148:0/3129593117 shutdown_connections 2024-04-25T09:57:35.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 -- 172.21.15.148:0/3129593117 wait complete. 2024-04-25T09:57:35.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 -- start start 2024-04-25T09:57:35.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.378+0000 7f1462a34740 1 Processor -- start 2024-04-25T09:57:35.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2cd0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145816c9f0 con 0x562dd2c45af0 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145816c3f0 con 0x562dd2c1f130 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd2c832e0 con 0x562dd2c64f60 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f1462a34740 10 client.0 _ll_register_callbacks cb 0x7f145007ec40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f145d1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.379+0000 7f145d1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c67120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40670/0 (socket says 172.21.15.148:40670) 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 -- 172.21.15.148:0/3089926229 learned_addr learned my addr 172.21.15.148:0/3089926229 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145e1f7640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2cd0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 -- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 msgr2=0x562dd2c825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 -- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 msgr2=0x562dd2cd0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2cd0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 -- 172.21.15.148:0/3089926229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dd282f8b0 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f145d1f5640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c67120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1454001350 tx=0x7f1454000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83f3257b90ab1af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1438ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1438ff9640 1 -- 172.21.15.148:0/3089926229 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454018030 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1454007270 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1438ff9640 1 -- 172.21.15.148:0/3089926229 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1454013670 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1438ff9640 1 -- 172.21.15.148:0/3089926229 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145403e050 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1462a34740 10 client.6360 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x562dd2c801a0 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.380+0000 7f1438ff9640 1 -- 172.21.15.148:0/3089926229 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454007270 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1438ff9640 1 -- 172.21.15.148:0/3089926229 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1454041020 con 0x562dd2c45af0 2024-04-25T09:57:35.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1462a34740 10 client.6360 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1462a34740 10 client.6360 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1462a34740 20 client.6360 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1462a34740 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c82b00 0x562dd2c4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1462a34740 4 client.6360 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x562dd2c4a4f0 con 0x562dd2c82b00 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c82b00 0x562dd2c4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1438ff9640 10 client.6360 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f145d9f6640 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c82b00 0x562dd2c4b090 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32267fc5b283b654 server_cookie=40494f7718dc25f0 in_seq=0 out_seq=0 2024-04-25T09:57:35.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.381+0000 7f1438ff9640 1 -- 172.21.15.148:0/3089926229 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562dd2c4a4f0 con 0x562dd2c82b00 2024-04-25T09:57:35.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1438ff9640 10 client.6360 handle_command_reply: tid=0 2024-04-25T09:57:35.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 client.6360 shutdown 2024-04-25T09:57:35.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 msgr2=0x562dd2c67120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c67120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1454001350 tx=0x7f1454000ed0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 shutdown_connections 2024-04-25T09:57:35.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dd2c64f60 0x562dd2c825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562dd2c45af0 0x562dd2c67120 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x562dd2c82b00 0x562dd2c4b090 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 --2- 172.21.15.148:0/3089926229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dd2c1f130 0x562dd2cd0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 >> 172.21.15.148:0/3089926229 conn(0x562dd2be5f70 msgr2=0x562dd2c3f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1438ff9640 0 client.6360 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 shutdown_connections 2024-04-25T09:57:35.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 1 -- 172.21.15.148:0/3089926229 wait complete. 2024-04-25T09:57:35.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f1462a34740 20 client.6360 trim_cache size 0 max 16384 2024-04-25T09:57:35.394 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "7b487ab3-ea2d-40f5-87eb-57db52503a43", 2024-04-25T09:57:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:35.394 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f143c06b500 msgr2=0x7f143c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f143c06b500 0x7f143c06d9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f144800dbe0 tx=0x7f144800f040 comp rx=0 tx=0).stop 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 msgr2=0x7f14580ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 0x7f14580ca7c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1450013a60 tx=0x7f1450004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 shutdown_connections 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f143c06b500 0x7f143c06d9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145816a720 0x7f14580ca7c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.382+0000 7f145fc5a640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1458072050 0x7f1458164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.383+0000 7f145fc5a640 1 --2- 172.21.15.148:0/2817438657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f145806ee00 0x7f14580c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.383+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 >> 172.21.15.148:0/2817438657 conn(0x7f145806abe0 msgr2=0x7f145816cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.383+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 shutdown_connections 2024-04-25T09:57:35.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.383+0000 7f145fc5a640 1 -- 172.21.15.148:0/2817438657 wait complete. 2024-04-25T09:57:35.397 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '7b487ab3-ea2d-40f5-87eb-57db52503a43', 'mode': 'asynchronous'}' 2024-04-25T09:57:35.397 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T09:57:35.397 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:35.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T09:57:35.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.495+0000 7fac2604a640 1 Processor -- start 2024-04-25T09:57:35.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.495+0000 7fac2604a640 1 -- start start 2024-04-25T09:57:35.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.495+0000 7fac2604a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac2015b7d0 0x7fac200cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.495+0000 7fac2604a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2006c9e0 0x7fac200b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac2604a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2006cf50 con 0x7fac200b5620 2024-04-25T09:57:35.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac2604a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac200ce530 con 0x7fac2015b7d0 2024-04-25T09:57:35.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac2604a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200ce810 con 0x7fac2006c9e0 2024-04-25T09:57:35.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac17fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fac200b5620 0x7fac2006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37504/0 (socket says 172.21.15.148:37504) 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac17fff640 1 -- 172.21.15.148:0/1261663802 learned_addr learned my addr 172.21.15.148:0/1261663802 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/1261663802 >> v2:172.21.15.188:3300/0 conn(0x7fac2015b7d0 0x7fac200cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1ffff640 1 --2- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2006c9e0 0x7fac200b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1effd640 1 -- 172.21.15.148:0/1261663802 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807232351 0 0) 0x7fac2006cf50 con 0x7fac200b5620 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1effd640 1 -- 172.21.15.148:0/1261663802 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac0c003520 con 0x7fac200b5620 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1f7fe640 1 -- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2006c9e0 msgr2=0x7fac200b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2006c9e0 0x7fac200b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1f7fe640 1 -- 172.21.15.148:0/1261663802 >> v1:172.21.15.148:6789/0 conn(0x7fac200b5620 legacy=0x7fac2006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.496+0000 7fac1f7fe640 1 -- 172.21.15.148:0/1261663802 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200ceaf0 con 0x7fac2015b7d0 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.497+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2015b7d0 0x7fac200cf010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac100097a0 tx=0x7fac1002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c9a5fac63bd1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.497+0000 7fac1effd640 1 -- 172.21.15.148:0/1261663802 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10037070 con 0x7fac2015b7d0 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.497+0000 7fac1effd640 1 -- 172.21.15.148:0/1261663802 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10032440 con 0x7fac2015b7d0 2024-04-25T09:57:35.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.497+0000 7fac1effd640 1 -- 172.21.15.148:0/1261663802 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10032810 con 0x7fac2015b7d0 2024-04-25T09:57:35.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 -- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2015b7d0 msgr2=0x7fac200cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 --2- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2015b7d0 0x7fac200cf010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac100097a0 tx=0x7fac1002a180 comp rx=0 tx=0).stop 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 -- 172.21.15.148:0/1261663802 shutdown_connections 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 --2- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2006c9e0 0x7fac200b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 --2- 172.21.15.148:0/1261663802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2015b7d0 0x7fac200cf010 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 -- 172.21.15.148:0/1261663802 >> 172.21.15.148:0/1261663802 conn(0x7fac200d0800 msgr2=0x7fac200d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 -- 172.21.15.148:0/1261663802 shutdown_connections 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.498+0000 7fac2604a640 1 -- 172.21.15.148:0/1261663802 wait complete. 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.499+0000 7fac2604a640 1 Processor -- start 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.499+0000 7fac2604a640 1 -- start start 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2006c9e0 0x7fac201571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac200b5620 0x7fac200b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 0x7fac200b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200cf6d0 con 0x7fac2015b7d0 2024-04-25T09:57:35.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200cf550 con 0x7fac2006c9e0 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2015bbb0 con 0x7fac200b5620 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 0x7fac200b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 0x7fac200b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40680/0 (socket says 172.21.15.148:40680) 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 -- 172.21.15.148:0/2888804981 learned_addr learned my addr 172.21.15.148:0/2888804981 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac17fff640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac200b5620 0x7fac200b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 -- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac200b5620 msgr2=0x7fac200b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2006c9e0 0x7fac201571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac200b5620 0x7fac200b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 -- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2006c9e0 msgr2=0x7fac201571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2006c9e0 0x7fac201571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 -- 172.21.15.148:0/2888804981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200c76a0 con 0x7fac2015b7d0 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac17fff640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac200b5620 0x7fac200b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2006c9e0 0x7fac201571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1ffff640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 0x7fac200b8590 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fac18009d20 tx=0x7fac1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1cff9640 1 -- 172.21.15.148:0/2888804981 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18018020 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1cff9640 1 -- 172.21.15.148:0/2888804981 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18009b10 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac200c7980 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac1cff9640 1 -- 172.21.15.148:0/2888804981 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180146c0 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.500+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac200c7f40 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.502+0000 7fac1cff9640 1 -- 172.21.15.148:0/2888804981 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac1801b020 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.502+0000 7fac1cff9640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac0006b500 0x7fac0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.502+0000 7fac1cff9640 1 -- 172.21.15.148:0/2888804981 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac1808aea0 con 0x7fac2015b7d0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.502+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac0006b500 0x7fac0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.503+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac0006b500 0x7fac0006d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fac10034000 tx=0x7fac1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.503+0000 7fac28e29740 1 Processor -- start 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.503+0000 7fac28e29740 1 -- start start 2024-04-25T09:57:35.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dae57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac28e29740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dab5c40 0x56081dab9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac28e29740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081da1ea50 con 0x56081dae33c0 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081d9c7d40 con 0x56081dae2fe0 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56081da1aaf0 con 0x56081dab5c40 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40696/0 (socket says 172.21.15.148:40696) 2024-04-25T09:57:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 -- 172.21.15.148:0/2317573956 learned_addr learned my addr 172.21.15.148:0/2317573956 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dae57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac1ffff640 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dab5c40 0x56081dab9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 -- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dab5c40 msgr2=0x56081dab9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dab5c40 0x56081dab9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 -- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 msgr2=0x56081dae57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dae57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 -- 172.21.15.148:0/2317573956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56081d7ff690 con 0x56081dae33c0 2024-04-25T09:57:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.504+0000 7fac17fff640 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae5d30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fac08001740 tx=0x7fac08001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3585779cc50208b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac157fa640 1 -- 172.21.15.148:0/2317573956 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08018070 con 0x56081dae33c0 2024-04-25T09:57:35.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac157fa640 1 -- 172.21.15.148:0/2317573956 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac08005ce0 con 0x56081dae33c0 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac157fa640 1 -- 172.21.15.148:0/2317573956 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08007310 con 0x56081dae33c0 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 -- 172.21.15.148:0/2317573956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 msgr2=0x56081dae5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae5d30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fac08001740 tx=0x7fac08001c00 comp rx=0 tx=0).stop 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 -- 172.21.15.148:0/2317573956 shutdown_connections 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dab5c40 0x56081dab9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae5d30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 --2- 172.21.15.148:0/2317573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dae57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 -- 172.21.15.148:0/2317573956 >> 172.21.15.148:0/2317573956 conn(0x56081dab7eb0 msgr2=0x56081db007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 -- 172.21.15.148:0/2317573956 shutdown_connections 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.505+0000 7fac28e29740 1 -- 172.21.15.148:0/2317573956 wait complete. 2024-04-25T09:57:35.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.506+0000 7fac28e29740 1 -- start start 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.506+0000 7fac28e29740 1 Processor -- start 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081db02fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dafd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daf82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081dabb060 con 0x56081dae33c0 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081d9bd340 con 0x56081dae2fe0 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56081dabb340 con 0x56081dad9c90 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 10 client.0 _ll_register_callbacks cb 0x56081d9dacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dafd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dafd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40706/0 (socket says 172.21.15.148:40706) 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 -- 172.21.15.148:0/2530046140 learned_addr learned my addr 172.21.15.148:0/2530046140 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac1ffff640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daf82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081db02fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 -- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 msgr2=0x56081daf82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daf82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 -- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 msgr2=0x56081db02fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081db02fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 -- 172.21.15.148:0/2530046140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56081d85fe20 con 0x56081dae33c0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081db02fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac17fff640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dafd0a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fac08001c40 tx=0x7fac08000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c725b73de09638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fabfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fabfe7fc640 1 -- 172.21.15.148:0/2530046140 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08023020 con 0x56081dae33c0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fabfe7fc640 1 -- 172.21.15.148:0/2530046140 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac080012f0 con 0x56081dae33c0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.507+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac08005ce0 con 0x56081dae33c0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fabfe7fc640 1 -- 172.21.15.148:0/2530046140 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0802d050 con 0x56081dae33c0 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 10 client.6372 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56081d6f2f80 con 0x56081dae33c0 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fabfe7fc640 1 -- 172.21.15.148:0/2530046140 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac08005ce0 con 0x56081dae33c0 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fabfe7fc640 1 -- 172.21.15.148:0/2530046140 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac08030020 con 0x56081dae33c0 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 10 client.6372 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 10 client.6372 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 20 client.6372 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081db03c30 0x56081db04030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 4 client.6372 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56081da2bf90 con 0x56081db03c30 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081db03c30 0x56081db04030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081db03c30 0x56081db04030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfdf974ba1111619 server_cookie=df9be4c469242428 in_seq=0 out_seq=0 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.508+0000 7fabfe7fc640 10 client.6372 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fabfe7fc640 1 -- 172.21.15.148:0/2530046140 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56081da2bf90 con 0x56081db03c30 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fabfe7fc640 10 client.6372 handle_command_reply: tid=0 2024-04-25T09:57:35.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 client.6372 shutdown 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 msgr2=0x56081dafd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dafd0a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fac08001c40 tx=0x7fac08000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 shutdown_connections 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daf82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dafd0a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081db03c30 0x56081db04030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 --2- 172.21.15.148:0/2530046140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081db02fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fabfe7fc640 0 client.6372 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 >> 172.21.15.148:0/2530046140 conn(0x56081dab5c40 msgr2=0x56081dab6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 shutdown_connections 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 1 -- 172.21.15.148:0/2530046140 wait complete. 2024-04-25T09:57:35.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.509+0000 7fac28e29740 20 client.6372 trim_cache size 0 max 16384 2024-04-25T09:57:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.524+0000 7fac28e29740 1 Processor -- start 2024-04-25T09:57:35.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.524+0000 7fac28e29740 1 -- start start 2024-04-25T09:57:35.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daba510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dab6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac28e29740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac28e29740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20200150 con 0x56081dae33c0 2024-04-25T09:57:35.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201ff2f0 con 0x56081dae2fe0 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac201fe770 con 0x56081dad9c90 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40710/0 (socket says 172.21.15.148:40710) 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 -- 172.21.15.148:0/1012198382 learned_addr learned my addr 172.21.15.148:0/1012198382 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 -- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 msgr2=0x56081daba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daba510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac17fff640 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dab6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 -- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 msgr2=0x56081dab6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dab6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac1ffff640 1 -- 172.21.15.148:0/1012198382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56081dafc090 con 0x56081dae33c0 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.525+0000 7fac17fff640 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dab6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac1ffff640 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae57b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fac180594b0 tx=0x7fac18059420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c45edb67503387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac14ff9640 1 -- 172.21.15.148:0/1012198382 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18087070 con 0x56081dae33c0 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac14ff9640 1 -- 172.21.15.148:0/1012198382 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18055b90 con 0x56081dae33c0 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac14ff9640 1 -- 172.21.15.148:0/1012198382 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180619a0 con 0x56081dae33c0 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 -- 172.21.15.148:0/1012198382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 msgr2=0x56081dae57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae57b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fac180594b0 tx=0x7fac18059420 comp rx=0 tx=0).stop 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 -- 172.21.15.148:0/1012198382 shutdown_connections 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae33c0 0x56081dae57b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dae2fe0 0x56081dab6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 --2- 172.21.15.148:0/1012198382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dad9c90 0x56081daba510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 -- 172.21.15.148:0/1012198382 >> 172.21.15.148:0/1012198382 conn(0x56081dab5c40 msgr2=0x56081dafc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 -- 172.21.15.148:0/1012198382 shutdown_connections 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.526+0000 7fac28e29740 1 -- 172.21.15.148:0/1012198382 wait complete. 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.527+0000 7fac28e29740 1 -- start start 2024-04-25T09:57:35.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.527+0000 7fac28e29740 1 Processor -- start 2024-04-25T09:57:35.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dad9c90 0x56081dae57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 0x56081db6bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dae33c0 0x56081daf6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201fecf0 con 0x56081dae2fe0 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081dad9aa0 con 0x56081dad9c90 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56081daf6c70 con 0x56081dae33c0 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 10 client.0 _ll_register_callbacks cb 0x7fac08016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 0x56081db6bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 0x56081db6bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40718/0 (socket says 172.21.15.148:40718) 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 -- 172.21.15.148:0/3819030615 learned_addr learned my addr 172.21.15.148:0/3819030615 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac1ffff640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dae33c0 0x56081daf6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dad9c90 0x56081dae57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 -- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dae33c0 msgr2=0x56081daf6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dae33c0 0x56081daf6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 -- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dad9c90 msgr2=0x56081dae57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dad9c90 0x56081dae57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 -- 172.21.15.148:0/3819030615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56081dab8ce0 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac17fff640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 0x56081db6bb00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fac08000ea0 tx=0x7fac08009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e217f7879eb3b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fabfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fabfe7fc640 1 -- 172.21.15.148:0/3819030615 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08023070 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac18055b90 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.528+0000 7fabfe7fc640 1 -- 172.21.15.148:0/3819030615 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac0800dc40 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fabfe7fc640 1 -- 172.21.15.148:0/3819030615 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0803d050 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 10 client.6378 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56081dae23d0 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fabfe7fc640 1 -- 172.21.15.148:0/3819030615 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18055b90 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fabfe7fc640 1 -- 172.21.15.148:0/3819030615 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac08040020 con 0x56081dae2fe0 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 10 client.6378 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 10 client.6378 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:35.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 20 client.6378 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081daf9e00 0x56081dab9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 4 client.6378 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x56081daf7e50 con 0x56081daf9e00 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081daf9e00 0x56081dab9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fabfe7fc640 10 client.6378 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.529+0000 7fac1f7fe640 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081daf9e00 0x56081dab9bf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7241a11e1c6caa1d server_cookie=ddc9b3ae5345fed in_seq=0 out_seq=0 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fabfe7fc640 1 -- 172.21.15.148:0/3819030615 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56081daf7e50 con 0x56081daf9e00 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fabfe7fc640 10 client.6378 handle_command_reply: tid=0 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 client.6378 shutdown 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 msgr2=0x56081db6bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 0x56081db6bb00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fac08000ea0 tx=0x7fac08009040 comp rx=0 tx=0).stop 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 shutdown_connections 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56081dae33c0 0x56081daf6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56081dae2fe0 0x56081db6bb00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x56081daf9e00 0x56081dab9bf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 --2- 172.21.15.148:0/3819030615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56081dad9c90 0x56081dae57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fabfe7fc640 0 client.6378 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 >> 172.21.15.148:0/3819030615 conn(0x56081dab5c40 msgr2=0x56081dab6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 shutdown_connections 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 1 -- 172.21.15.148:0/3819030615 wait complete. 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.530+0000 7fac28e29740 20 client.6378 trim_cache size 0 max 16384 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "88b7983e-fde0-4eed-80cf-f2990e76eb1a", 2024-04-25T09:57:35.540 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:35.541 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:35.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac0006b500 msgr2=0x7fac0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac0006b500 0x7fac0006d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fac10034000 tx=0x7fac1002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:35.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 msgr2=0x7fac200b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 0x7fac200b8590 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fac18009d20 tx=0x7fac1800c490 comp rx=0 tx=0).stop 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 shutdown_connections 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac0006b500 0x7fac0006d9c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac2015b7d0 0x7fac200b8590 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac200b5620 0x7fac200b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 --2- 172.21.15.148:0/2888804981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2006c9e0 0x7fac201571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 >> 172.21.15.148:0/2888804981 conn(0x7fac200d0800 msgr2=0x7fac2015afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 shutdown_connections 2024-04-25T09:57:35.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.531+0000 7fac2604a640 1 -- 172.21.15.148:0/2888804981 wait complete. 2024-04-25T09:57:35.545 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '88b7983e-fde0-4eed-80cf-f2990e76eb1a', 'mode': 'asynchronous'}' 2024-04-25T09:57:35.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:35.545 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T09:57:35.562 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T09:57:35.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:35.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.696+0000 7f143cb7e640 1 Processor -- start 2024-04-25T09:57:35.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.697+0000 7f143cb7e640 1 -- start start 2024-04-25T09:57:35.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.697+0000 7f143cb7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14380d1dd0 0x7f14380ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.697+0000 7f143cb7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143815dfe0 con 0x7f143815dc00 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.697+0000 7f143cb7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380d00c0 con 0x7f14380c6df0 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.697+0000 7f143cb7e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14380d03a0 con 0x7f14380d1dd0 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f143815dc00 0x7f14380ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37506/0 (socket says 172.21.15.148:37506) 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143659c640 1 -- 172.21.15.148:0/968642610 learned_addr learned my addr 172.21.15.148:0/968642610 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f1436d9d640 1 --2- 172.21.15.148:0/968642610 >> v2:172.21.15.188:3300/0 conn(0x7f14380d1dd0 0x7f14380ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994970353 0 0) 0x7f142c001110 con 0x7f143815dc00 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1420003520 con 0x7f143815dc00 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133351177 0 0) 0x7f14380d00c0 con 0x7f14380c6df0 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f142c001110 con 0x7f14380c6df0 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583877098 0 0) 0x7f1420003520 con 0x7f143815dc00 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 >> v2:172.21.15.188:3300/0 conn(0x7f14380d1dd0 msgr2=0x7f14380ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 --2- 172.21.15.148:0/968642610 >> v2:172.21.15.188:3300/0 conn(0x7f14380d1dd0 0x7f14380ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 >> v1:172.21.15.188:6789/0 conn(0x7f14380c6df0 legacy=0x7f14380d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380cc440 con 0x7f143815dc00 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f142c001840 con 0x7f143815dc00 2024-04-25T09:57:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f1436d9d640 1 --2- 172.21.15.148:0/968642610 >> v2:172.21.15.188:3300/0 conn(0x7f14380d1dd0 0x7f14380ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 >> v1:172.21.15.148:6789/0 conn(0x7f143815dc00 legacy=0x7f14380ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f1420003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14200051c0 0x7f142000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380d03a0 con 0x7f1420003b40 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142000c040 con 0x7f14200051c0 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142000c240 con 0x7f1420004800 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f143659c640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f1420003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f1436d9d640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14200051c0 0x7f142000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.698+0000 7f1435d9b640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143659c640 1 -- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1420004800 msgr2=0x7f1420004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143659c640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143659c640 1 -- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14200051c0 msgr2=0x7f142000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143659c640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14200051c0 0x7f142000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f1436d9d640 1 -- 172.21.15.148:0/968642610 reap_dead start 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143659c640 1 -- 172.21.15.148:0/968642610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420003520 con 0x7f1420003b40 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f1435d9b640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1420004800 0x7f1420004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143659c640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f1420003f40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c490 tx=0x7f1420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cd763652e4d3da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c00c0e0 con 0x7f1420003b40 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c034070 con 0x7f1420003b40 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.699+0000 7f143559a640 1 -- 172.21.15.148:0/968642610 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c005d70 con 0x7f1420003b40 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 -- 172.21.15.148:0/968642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 msgr2=0x7f1420003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f1420003f40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c490 tx=0x7f1420005830 comp rx=0 tx=0).stop 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 -- 172.21.15.148:0/968642610 shutdown_connections 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 --2- 172.21.15.148:0/968642610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f1420003f40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 -- 172.21.15.148:0/968642610 >> 172.21.15.148:0/968642610 conn(0x7f143806f5d0 msgr2=0x7f143806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 -- 172.21.15.148:0/968642610 shutdown_connections 2024-04-25T09:57:35.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.700+0000 7f143cb7e640 1 -- 172.21.15.148:0/968642610 wait complete. 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.701+0000 7f143cb7e640 1 Processor -- start 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.701+0000 7f143cb7e640 1 -- start start 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143cb7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f14380cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143cb7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143815ca50 0x7f143815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143cb7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380b13c0 0x7f14380b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143cb7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380cfd90 con 0x7f1420003b40 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143cb7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380cc130 con 0x7f14380b13c0 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143cb7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380cdd70 con 0x7f143815ca50 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f14380cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f14380cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40744/0 (socket says 172.21.15.148:40744) 2024-04-25T09:57:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 -- 172.21.15.148:0/3528017790 learned_addr learned my addr 172.21.15.148:0/3528017790 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f1435d9b640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143815ca50 0x7f143815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f1436d9d640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380b13c0 0x7f14380b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 -- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143815ca50 msgr2=0x7f143815ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143815ca50 0x7f143815ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 -- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380b13c0 msgr2=0x7f14380b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380b13c0 0x7f14380b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.702+0000 7f143659c640 1 -- 172.21.15.148:0/3528017790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438159df0 con 0x7f1420003b40 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.703+0000 7f143659c640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f14380cb290 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f142c031040 tx=0x7f142c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.703+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c007480 con 0x7f1420003b40 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.703+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c03f030 con 0x7f1420003b40 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.703+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143815a0d0 con 0x7f1420003b40 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.703+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c007840 con 0x7f1420003b40 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.703+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143815a690 con 0x7f1420003b40 2024-04-25T09:57:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.704+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438073990 con 0x7f1420003b40 2024-04-25T09:57:35.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.704+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f142c02e070 con 0x7f1420003b40 2024-04-25T09:57:35.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.705+0000 7f1426ffd640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f141806b4d0 0x7f141806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:35.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.705+0000 7f1435d9b640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f141806b4d0 0x7f141806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:35.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.705+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f142c0aa9c0 con 0x7f1420003b40 2024-04-25T09:57:35.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.705+0000 7f1435d9b640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f141806b4d0 0x7f141806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1428007fd0 tx=0x7f1428009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:35.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.709+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f142c0ace20 con 0x7f1420003b40 2024-04-25T09:57:35.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.916+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14380c66c0 con 0x7f1420003b40 2024-04-25T09:57:35.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.916+0000 7f1426ffd640 1 -- 172.21.15.148:0/3528017790 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f142c049030 con 0x7f1420003b40 2024-04-25T09:57:35.920 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:35.920 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:31.018031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:35.920 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f141806b4d0 msgr2=0x7f141806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f141806b4d0 0x7f141806d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1428007fd0 tx=0x7f1428009040 comp rx=0 tx=0).stop 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 msgr2=0x7f14380cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f14380cb290 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f142c031040 tx=0x7f142c001b90 comp rx=0 tx=0).stop 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 shutdown_connections 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f141806b4d0 0x7f141806d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14380b13c0 0x7f14380b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143815ca50 0x7f143815ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 --2- 172.21.15.148:0/3528017790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1420003b40 0x7f14380cb290 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:35.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.918+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 >> 172.21.15.148:0/3528017790 conn(0x7f143806f5d0 msgr2=0x7f143806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:35.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.919+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 shutdown_connections 2024-04-25T09:57:35.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:35.919+0000 7f143cb7e640 1 -- 172.21.15.148:0/3528017790 wait complete. 2024-04-25T09:57:35.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T09:57:36.038 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:36.038 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:36.038 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:36.046 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:36.047 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:36.047 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:36.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:36.062 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T09:57:36.125 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T09:57:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T09:57:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826244 s, 127 MB/s 2024-04-25T09:57:36.126 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:57:36.127 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:36.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.261+0000 7f41be814640 1 Processor -- start 2024-04-25T09:57:36.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.261+0000 7f41be814640 1 -- start start 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be814640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f41b80d06c0 0x7f41b8168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be814640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b80d33f0 con 0x7f41b80b93c0 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be814640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b8161df0 con 0x7f41b815db90 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be814640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b80d1f50 con 0x7f41b80d06c0 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be013640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f41b80d06c0 0x7f41b8168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be013640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f41b80d06c0 0x7f41b8168930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40748/0 (socket says 172.21.15.148:40748) 2024-04-25T09:57:36.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41bd011640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f41b80b93c0 0x7f41b80cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37512/0 (socket says 172.21.15.148:37512) 2024-04-25T09:57:36.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.262+0000 7f41be013640 1 -- 172.21.15.148:0/4134809638 learned_addr learned my addr 172.21.15.148:0/4134809638 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41be013640 1 -- 172.21.15.148:0/4134809638 >> v1:172.21.15.188:6789/0 conn(0x7f41b815db90 legacy=0x7f41b81615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41be013640 1 -- 172.21.15.148:0/4134809638 >> v1:172.21.15.148:6789/0 conn(0x7f41b80b93c0 legacy=0x7f41b80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41be013640 1 -- 172.21.15.148:0/4134809638 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b81620d0 con 0x7f41b80d06c0 2024-04-25T09:57:36.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41bc810640 1 -- 172.21.15.148:0/4134809638 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929878939 0 0) 0x7f41b80d33f0 con 0x7f41b80b93c0 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41be013640 1 --2- 172.21.15.148:0/4134809638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8168930 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009e50 tx=0x7f41b002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd1f7df1cc5f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41bc810640 1 -- 172.21.15.148:0/4134809638 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0036070 con 0x7f41b80d06c0 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.263+0000 7f41bc810640 1 -- 172.21.15.148:0/4134809638 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0031600 con 0x7f41b80d06c0 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41bc810640 1 -- 172.21.15.148:0/4134809638 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b00319d0 con 0x7f41b80d06c0 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 -- 172.21.15.148:0/4134809638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 msgr2=0x7f41b8168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 --2- 172.21.15.148:0/4134809638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8168930 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009e50 tx=0x7f41b002a760 comp rx=0 tx=0).stop 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 -- 172.21.15.148:0/4134809638 shutdown_connections 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 --2- 172.21.15.148:0/4134809638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8168930 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 -- 172.21.15.148:0/4134809638 >> 172.21.15.148:0/4134809638 conn(0x7f41b80b52b0 msgr2=0x7f41b80b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 -- 172.21.15.148:0/4134809638 shutdown_connections 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.264+0000 7f41be814640 1 -- 172.21.15.148:0/4134809638 wait complete. 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.265+0000 7f41be814640 1 Processor -- start 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.265+0000 7f41be814640 1 -- start start 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be814640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b80b93c0 0x7f41b815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be814640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be814640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b815db90 0x7f41b815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be814640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80b68d0 con 0x7f41b80d06c0 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be814640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80d1be0 con 0x7f41b815db90 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be814640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b80b6a50 con 0x7f41b80b93c0 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40754/0 (socket says 172.21.15.148:40754) 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 -- 172.21.15.148:0/3155303604 learned_addr learned my addr 172.21.15.148:0/3155303604 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be013640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b815db90 0x7f41b815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 -- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b80b93c0 msgr2=0x7f41b815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd812640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b80b93c0 0x7f41b815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b80b93c0 0x7f41b815f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 -- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b815db90 msgr2=0x7f41b815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b815db90 0x7f41b815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41bd011640 1 -- 172.21.15.148:0/3155303604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b81f38e0 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.266+0000 7f41be013640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b815db90 0x7f41b815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.267+0000 7f41bd011640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8151b90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c4d0 tx=0x7f41a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.267+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8018070 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.267+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a8007500 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.267+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b81f3bc0 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.267+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8004a20 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.267+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b81f4180 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.268+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b8073990 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.268+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41a801b070 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.269+0000 7f41ae7fc640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f419c06b4d0 0x7f419c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.269+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f41a80896d0 con 0x7f41b80d06c0 2024-04-25T09:57:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.269+0000 7f41bd812640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f419c06b4d0 0x7f419c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.269+0000 7f41bd812640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f419c06b4d0 0x7f419c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f41b400a730 tx=0x7f41b4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.273+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41a80583c0 con 0x7f41b80d06c0 2024-04-25T09:57:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.478+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41b80d14b0 con 0x7f41b80d06c0 2024-04-25T09:57:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.479+0000 7f41ae7fc640 1 -- 172.21.15.148:0/3155303604 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4950 (secure 0 0 0) 0x7f41a8057cc0 con 0x7f41b80d06c0 2024-04-25T09:57:36.483 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:36.483 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:31.018031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 24 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f419c06b4d0 msgr2=0x7f419c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f419c06b4d0 0x7f419c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f41b400a730 tx=0x7f41b4008040 comp rx=0 tx=0).stop 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 msgr2=0x7f41b8151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8151b90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c4d0 tx=0x7f41a800c990 comp rx=0 tx=0).stop 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 shutdown_connections 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f419c06b4d0 0x7f419c06d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b815db90 0x7f41b815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b80d06c0 0x7f41b8151b90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 --2- 172.21.15.148:0/3155303604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b80b93c0 0x7f41b815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 >> 172.21.15.148:0/3155303604 conn(0x7f41b80b52b0 msgr2=0x7f41b8061600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 shutdown_connections 2024-04-25T09:57:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.481+0000 7f41be814640 1 -- 172.21.15.148:0/3155303604 wait complete. 2024-04-25T09:57:36.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:57:36.602 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:36.602 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:36.602 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:36.611 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:36.611 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:57:36.611 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:57:36.611 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T09:57:36.611 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:57:36.611 DEBUG:teuthology.orchestra.run.smithi148:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T09:57:36.611 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:57:36.676 INFO:teuthology.orchestra.run.smithi148.stdout:1099511630188 2024-04-25T09:57:36.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:57:36.686 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:36.686 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:57:36.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.823+0000 7fa310d1d640 1 Processor -- start 2024-04-25T09:57:36.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.823+0000 7fa310d1d640 1 -- start start 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa310d1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa30c0c6010 0x7fa30c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa310d1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa310d1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c0c6940 con 0x7fa30c15a9f0 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa310d1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0b0020 con 0x7fa30c0c6010 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa310d1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0cb5a0 con 0x7fa30c0cd520 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa30c0c6010 0x7fa30c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa30c0c6010 0x7fa30c0ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40766/0 (socket says 172.21.15.148:40766) 2024-04-25T09:57:36.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa30a59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa30c15a9f0 0x7fa30c0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37524/0 (socket says 172.21.15.148:37524) 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 -- 172.21.15.148:0/291464355 learned_addr learned my addr 172.21.15.148:0/291464355 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 -- 172.21.15.148:0/291464355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 msgr2=0x7fa30c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 -- 172.21.15.148:0/291464355 >> v1:172.21.15.148:6789/0 conn(0x7fa30c15a9f0 legacy=0x7fa30c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa309d9b640 1 -- 172.21.15.148:0/291464355 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c15add0 con 0x7fa30c0c6010 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa30959a640 1 -- 172.21.15.148:0/291464355 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288487905 0 0) 0x7fa304001490 con 0x7fa30c15a9f0 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.824+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.825+0000 7fa309d9b640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c0c6010 0x7fa30c0ccfe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009870 tx=0x7fa2f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc838fdd10a96136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.825+0000 7fa30959a640 1 -- 172.21.15.148:0/291464355 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4037070 con 0x7fa30c0c6010 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.825+0000 7fa30959a640 1 -- 172.21.15.148:0/291464355 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f4032440 con 0x7fa30c0c6010 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.825+0000 7fa30959a640 1 -- 172.21.15.148:0/291464355 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4032810 con 0x7fa30c0c6010 2024-04-25T09:57:36.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.825+0000 7fa310d1d640 1 -- 172.21.15.148:0/291464355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c0c6010 msgr2=0x7fa30c0ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.825+0000 7fa310d1d640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c0c6010 0x7fa30c0ccfe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009870 tx=0x7fa2f402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.826+0000 7fa310d1d640 1 -- 172.21.15.148:0/291464355 shutdown_connections 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.826+0000 7fa310d1d640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.826+0000 7fa310d1d640 1 --2- 172.21.15.148:0/291464355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c0c6010 0x7fa30c0ccfe0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.826+0000 7fa310d1d640 1 -- 172.21.15.148:0/291464355 >> 172.21.15.148:0/291464355 conn(0x7fa30c0b2390 msgr2=0x7fa30c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.826+0000 7fa310d1d640 1 -- 172.21.15.148:0/291464355 shutdown_connections 2024-04-25T09:57:36.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.826+0000 7fa310d1d640 1 -- 172.21.15.148:0/291464355 wait complete. 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.827+0000 7fa310d1d640 1 Processor -- start 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.827+0000 7fa310d1d640 1 -- start start 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa30c0c6010 0x7fa30c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 0x7fa30c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0cb230 con 0x7fa30c15a9f0 2024-04-25T09:57:36.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0caf50 con 0x7fa30c0c6010 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0cbf70 con 0x7fa30c0cd520 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 0x7fa30c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 0x7fa30c0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40770/0 (socket says 172.21.15.148:40770) 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2817690370 learned_addr learned my addr 172.21.15.148:0/2817690370 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa309d9b640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa30c0c6010 0x7fa30c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 msgr2=0x7fa30c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa30c0c6010 msgr2=0x7fa30c14f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa30c0c6010 0x7fa30c14f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2817690370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0b4790 con 0x7fa30c15a9f0 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 0x7fa30c0b5890 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00e7e0 tx=0x7fa2fc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa302ffd640 1 -- 172.21.15.148:0/2817690370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00f040 con 0x7fa30c15a9f0 2024-04-25T09:57:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa302ffd640 1 -- 172.21.15.148:0/2817690370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc004030 con 0x7fa30c15a9f0 2024-04-25T09:57:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c0b4a70 con 0x7fa30c15a9f0 2024-04-25T09:57:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa302ffd640 1 -- 172.21.15.148:0/2817690370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0043e0 con 0x7fa30c15a9f0 2024-04-25T09:57:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.828+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c0b4380 con 0x7fa30c15a9f0 2024-04-25T09:57:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.830+0000 7fa302ffd640 1 -- 172.21.15.148:0/2817690370 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2fc00c6d0 con 0x7fa30c15a9f0 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.830+0000 7fa302ffd640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2ec06b4d0 0x7fa2ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.831+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2ec06b4d0 0x7fa2ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.831+0000 7fa302ffd640 1 -- 172.21.15.148:0/2817690370 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2fc08a030 con 0x7fa30c15a9f0 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.831+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2ec06b4d0 0x7fa2ec06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa3040012e0 tx=0x7fa304005eb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.831+0000 7fa313a1b740 1 Processor -- start 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 -- start start 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff0f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffefedf700 0x55ffefee2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffefe48e00 con 0x55ffeff0caa0 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffefdf1d20 con 0x55ffefedf700 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffefe44ea0 con 0x55ffeff0ce80 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa30a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.832+0000 7fa30a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff0f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40780/0 (socket says 172.21.15.148:40780) 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 -- 172.21.15.148:0/1704223215 learned_addr learned my addr 172.21.15.148:0/1704223215 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffefedf700 0x55ffefee2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa309d9b640 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff0f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 -- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 msgr2=0x55ffeff0f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff0f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 -- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffefedf700 msgr2=0x55ffefee2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffefedf700 0x55ffefee2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 -- 172.21.15.148:0/1704223215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffefe13df0 con 0x55ffeff0caa0 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffefedf700 0x55ffefee2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:36.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa30a59c640 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff0f2b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa304001c40 tx=0x7fa304012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf69c3fd141c01d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa2ebfff640 1 -- 172.21.15.148:0/1704223215 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3040159c0 con 0x55ffeff0caa0 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa2ebfff640 1 -- 172.21.15.148:0/1704223215 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa304015b60 con 0x55ffeff0caa0 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.833+0000 7fa2ebfff640 1 -- 172.21.15.148:0/1704223215 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304021450 con 0x55ffeff0caa0 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 -- 172.21.15.148:0/1704223215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 msgr2=0x55ffeff0f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff0f2b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa304001c40 tx=0x7fa304012000 comp rx=0 tx=0).stop 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 -- 172.21.15.148:0/1704223215 shutdown_connections 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffefedf700 0x55ffefee2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff0f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 --2- 172.21.15.148:0/1704223215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff0f2b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 -- 172.21.15.148:0/1704223215 >> 172.21.15.148:0/1704223215 conn(0x55ffefee1970 msgr2=0x55ffeff2a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 -- 172.21.15.148:0/1704223215 shutdown_connections 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 -- 172.21.15.148:0/1704223215 wait complete. 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 -- start start 2024-04-25T09:57:36.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.834+0000 7fa313a1b740 1 Processor -- start 2024-04-25T09:57:36.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0caa0 0x55ffefee4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff10cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffeff11200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffefee4b20 con 0x55ffeff2f360 2024-04-25T09:57:36.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffefde7590 con 0x55ffeff0caa0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffefee4e00 con 0x55ffeff0ce80 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa313a1b740 10 client.0 _ll_register_callbacks cb 0x55ffefdf3e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa30ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffeff11200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa30ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffeff11200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40794/0 (socket says 172.21.15.148:40794) 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.835+0000 7fa30ad9d640 1 -- 172.21.15.148:0/4189997736 learned_addr learned my addr 172.21.15.148:0/4189997736 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30a59c640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0caa0 0x55ffefee4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa309d9b640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff10cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30ad9d640 1 -- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 msgr2=0x55ffeff10cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff10cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30ad9d640 1 -- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0caa0 msgr2=0x55ffefee4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0caa0 0x55ffefee4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30ad9d640 1 -- 172.21.15.148:0/4189997736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffefb5d3f0 con 0x55ffeff2f360 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffeff11200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc060890 tx=0x7fa2fc058ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242d50d6a577c9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa2e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:36.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa2e97fa640 1 -- 172.21.15.148:0/4189997736 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc020030 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa2e97fa640 1 -- 172.21.15.148:0/4189997736 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2fc057b00 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa304015b60 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa2e97fa640 1 -- 172.21.15.148:0/4189997736 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2fc088050 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 10 client.6408 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ffefc277d0 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa2e97fa640 1 -- 172.21.15.148:0/4189997736 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa304015b60 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa2e97fa640 1 -- 172.21.15.148:0/4189997736 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2fc012050 con 0x55ffeff2f360 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 10 client.6408 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 10 client.6408 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 20 client.6408 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff02d80 0x55ffeff2b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 4 client.6408 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffefe56340 con 0x55ffeff02d80 2024-04-25T09:57:36.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.836+0000 7fa30a59c640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff02d80 0x55ffeff2b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa30a59c640 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff02d80 0x55ffeff2b3e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2db9c45f47570d4 server_cookie=a4d67f9969506d7 in_seq=0 out_seq=0 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa2e97fa640 10 client.6408 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa2e97fa640 1 -- 172.21.15.148:0/4189997736 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffefe56340 con 0x55ffeff02d80 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa2e97fa640 10 client.6408 handle_command_reply: tid=0 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 client.6408 shutdown 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 msgr2=0x55ffeff11200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffeff11200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc060890 tx=0x7fa2fc058ef0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 shutdown_connections 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffeff11200 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff10cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff02d80 0x55ffeff2b3e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 --2- 172.21.15.148:0/4189997736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0caa0 0x55ffefee4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa2e97fa640 0 client.6408 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.837+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 >> 172.21.15.148:0/4189997736 conn(0x55ffefedf700 msgr2=0x55ffeff0b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.838+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 shutdown_connections 2024-04-25T09:57:36.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.838+0000 7fa313a1b740 1 -- 172.21.15.148:0/4189997736 wait complete. 2024-04-25T09:57:36.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.838+0000 7fa313a1b740 20 client.6408 trim_cache size 0 max 16384 2024-04-25T09:57:36.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.849+0000 7fa313a1b740 1 Processor -- start 2024-04-25T09:57:36.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.849+0000 7fa313a1b740 1 -- start start 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0caa0 0x55ffeff110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0ce80 0x55ffefee00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffefee0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa313a1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1fe2e0 con 0x55ffeff2f360 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1fd460 con 0x55ffeff0ce80 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0c8fc0 con 0x55ffeff0caa0 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffefee0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffefee0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40802/0 (socket says 172.21.15.148:40802) 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2233219381 learned_addr learned my addr 172.21.15.148:0/2233219381 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa309d9b640 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0ce80 0x55ffefee00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0caa0 0x55ffeff110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0caa0 msgr2=0x55ffeff110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0caa0 0x55ffeff110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0ce80 msgr2=0x55ffefee00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0ce80 0x55ffefee00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 -- 172.21.15.148:0/2233219381 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffeff11600 con 0x55ffeff2f360 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.850+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffefee0620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c4c0 tx=0x7fa2fc05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593784fdb31bcd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa2eb7fe640 1 -- 172.21.15.148:0/2233219381 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc020040 con 0x55ffeff2f360 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa2eb7fe640 1 -- 172.21.15.148:0/2233219381 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc057b00 con 0x55ffeff2f360 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa2eb7fe640 1 -- 172.21.15.148:0/2233219381 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc054620 con 0x55ffeff2f360 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 -- 172.21.15.148:0/2233219381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 msgr2=0x55ffefee0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffefee0620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c4c0 tx=0x7fa2fc05b000 comp rx=0 tx=0).stop 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 -- 172.21.15.148:0/2233219381 shutdown_connections 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff2f360 0x55ffefee0620 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff0ce80 0x55ffefee00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2233219381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0caa0 0x55ffeff110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 -- 172.21.15.148:0/2233219381 >> 172.21.15.148:0/2233219381 conn(0x55ffefedf700 msgr2=0x55ffefedadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 -- 172.21.15.148:0/2233219381 shutdown_connections 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.851+0000 7fa313a1b740 1 -- 172.21.15.148:0/2233219381 wait complete. 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.852+0000 7fa313a1b740 1 -- start start 2024-04-25T09:57:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.852+0000 7fa313a1b740 1 Processor -- start 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff11c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff2f360 0x55ffeff0bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1fce60 con 0x55ffeff0caa0 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffeff20c30 con 0x55ffeff2f360 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffefee02b0 con 0x55ffeff0ce80 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 10 client.0 _ll_register_callbacks cb 0x7fa3040071f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:36.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40818/0 (socket says 172.21.15.148:40818) 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 -- 172.21.15.148:0/2865222416 learned_addr learned my addr 172.21.15.148:0/2865222416 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa309d9b640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff11c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30ad9d640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff2f360 0x55ffeff0bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 -- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 msgr2=0x55ffeff11c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff11c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 -- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff2f360 msgr2=0x55ffeff0bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff2f360 0x55ffeff0bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 -- 172.21.15.148:0/2865222416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffeff2ab40 con 0x55ffeff0caa0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa30a59c640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff204e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa304014000 tx=0x7fa304011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb21d09e659436c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa2e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa2e97fa640 1 -- 172.21.15.148:0/2865222416 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304031020 con 0x55ffeff0caa0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa2e97fa640 1 -- 172.21.15.148:0/2865222416 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa30402d380 con 0x55ffeff0caa0 2024-04-25T09:57:36.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2fc057b00 con 0x55ffeff0caa0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa2e97fa640 1 -- 172.21.15.148:0/2865222416 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa30403b050 con 0x55ffeff0caa0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 10 client.6414 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.853+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ffeff9db40 con 0x55ffeff0caa0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa2e97fa640 1 -- 172.21.15.148:0/2865222416 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc057b00 con 0x55ffeff0caa0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa2e97fa640 1 -- 172.21.15.148:0/2865222416 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa304004b60 con 0x55ffeff0caa0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 10 client.6414 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 10 client.6414 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 20 client.6414 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff12190 0x55ffeff0c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 4 client.6414 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffefe50690 con 0x55ffeff12190 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa309d9b640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff12190 0x55ffeff0c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa2e97fa640 10 client.6414 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa309d9b640 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff12190 0x55ffeff0c160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b9bec18c517fa3 server_cookie=18f7f179183d133d in_seq=0 out_seq=0 2024-04-25T09:57:36.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa2e97fa640 1 -- 172.21.15.148:0/2865222416 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffefe50690 con 0x55ffeff12190 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa2e97fa640 10 client.6414 handle_command_reply: tid=0 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 client.6414 shutdown 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 msgr2=0x55ffeff204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff204e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa304014000 tx=0x7fa304011000 comp rx=0 tx=0).stop 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 shutdown_connections 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ffeff12190 0x55ffeff0c160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ffeff2f360 0x55ffeff0bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ffeff0ce80 0x55ffeff11c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 --2- 172.21.15.148:0/2865222416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffeff0caa0 0x55ffeff204e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa2e97fa640 0 client.6414 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.854+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 >> 172.21.15.148:0/2865222416 conn(0x55ffefedf700 msgr2=0x55ffeff31c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 shutdown_connections 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa313a1b740 1 -- 172.21.15.148:0/2865222416 wait complete. 2024-04-25T09:57:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa313a1b740 20 client.6414 trim_cache size 0 max 16384 2024-04-25T09:57:36.865 INFO:teuthology.orchestra.run.smithi148.stdout:[] 2024-04-25T09:57:36.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2ec06b4d0 msgr2=0x7fa2ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2ec06b4d0 0x7fa2ec06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa3040012e0 tx=0x7fa304005eb0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 msgr2=0x7fa30c0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 0x7fa30c0b5890 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00e7e0 tx=0x7fa2fc00eca0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 shutdown_connections 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2ec06b4d0 0x7fa2ec06d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa30c15a9f0 0x7fa30c0b5890 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa30c0cd520 0x7fa30c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 --2- 172.21.15.148:0/2817690370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa30c0c6010 0x7fa30c14f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 >> 172.21.15.148:0/2817690370 conn(0x7fa30c0b2390 msgr2=0x7fa30c0af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 shutdown_connections 2024-04-25T09:57:36.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.855+0000 7fa310d1d640 1 -- 172.21.15.148:0/2817690370 wait complete. 2024-04-25T09:57:36.869 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096c.00000000 parent 2024-04-25T09:57:36.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 Processor -- start 2024-04-25T09:57:36.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 -- start start 2024-04-25T09:57:36.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593f9fb9de0 0x5593fa08c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593f9fe92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 0x5593f9fe68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa004f30 con 0x5593fa091060 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f9f62df0 con 0x5593f9fb9de0 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4cd1728f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593fa0051d0 con 0x5593fa004a80 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4ccfa2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 0x5593f9fe68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4ccfa2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 0x5593f9fe68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40838/0 (socket says 172.21.15.148:40838) 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.908+0000 7f4ccfa2c640 1 -- 172.21.15.148:0/1165673956 learned_addr learned my addr 172.21.15.148:0/1165673956 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccfa2c640 1 -- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 msgr2=0x5593f9fe92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593f9fe92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccf22b640 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593f9fb9de0 0x5593fa08c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccfa2c640 1 -- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593f9fb9de0 msgr2=0x5593fa08c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593f9fb9de0 0x5593fa08c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccfa2c640 1 -- 172.21.15.148:0/1165673956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f9ee8c50 con 0x5593fa091060 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccf22b640 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593f9fb9de0 0x5593fa08c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 0x5593f9fe68e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8001600 tx=0x7f4cc8001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f177dd5e22f0e0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccda28640 1 -- 172.21.15.148:0/1165673956 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8003650 con 0x5593fa091060 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccda28640 1 -- 172.21.15.148:0/1165673956 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80037f0 con 0x5593fa091060 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4ccda28640 1 -- 172.21.15.148:0/1165673956 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8003c00 con 0x5593fa091060 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1165673956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 msgr2=0x5593f9fe68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 0x5593f9fe68e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8001600 tx=0x7f4cc8001ac0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1165673956 shutdown_connections 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593fa091060 0x5593f9fe68e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593f9fe92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1165673956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593f9fb9de0 0x5593fa08c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1165673956 >> 172.21.15.148:0/1165673956 conn(0x5593f9fba9b0 msgr2=0x5593f9fe5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.909+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1165673956 shutdown_connections 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.910+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1165673956 wait complete. 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.910+0000 7f4cd1728f40 1 Processor -- start 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.910+0000 7f4cd1728f40 1 -- start start 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593f9fb9de0 0x5593fa087280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593fa11dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593fa091060 0x5593fa11e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa08e030 con 0x5593f9fb9de0 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fa08e2d0 con 0x5593fa091060 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593f9fe7aa0 con 0x5593fa004a80 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593f9fb9de0 0x5593fa087280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593f9fb9de0 0x5593fa087280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40842/0 (socket says 172.21.15.148:40842) 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 -- 172.21.15.148:0/1784393597 learned_addr learned my addr 172.21.15.148:0/1784393597 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccea2a640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593fa11dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593fa091060 0x5593fa11e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 -- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 msgr2=0x5593fa11dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593fa11dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 -- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593fa091060 msgr2=0x5593fa11e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593fa091060 0x5593fa11e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 -- 172.21.15.148:0/1784393597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fa11d6f0 con 0x5593f9fb9de0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccea2a640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593fa004a80 0x5593fa11dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593fa091060 0x5593fa11e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccf22b640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593f9fb9de0 0x5593fa087280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000bf50 tx=0x7f4cc00074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccca26640 1 -- 172.21.15.148:0/1784393597 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0019070 con 0x5593f9fb9de0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccca26640 1 -- 172.21.15.148:0/1784393597 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc0007c60 con 0x5593f9fb9de0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5593fa08d2b0 con 0x5593f9fb9de0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.911+0000 7f4ccca26640 1 -- 172.21.15.148:0/1784393597 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0014630 con 0x5593f9fb9de0 2024-04-25T09:57:36.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.912+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593fa0877c0 con 0x5593f9fb9de0 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.913+0000 7f4ccca26640 1 -- 172.21.15.148:0/1784393597 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cc001c070 con 0x5593f9fb9de0 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.913+0000 7f4ccca26640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cbc069eb0 0x7f4cbc06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.913+0000 7f4ccca26640 1 -- 172.21.15.148:0/1784393597 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cc00881d0 con 0x5593f9fb9de0 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.913+0000 7f4ccea2a640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cbc069eb0 0x7f4cbc06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.913+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5593fa08d910 0x5593fa0896b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.913+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- osd_op(unknown.0.0:1 8.2b 8:d716f75e:::1000000096c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v9 -- 0x5593fa089bf0 con 0x5593fa08d910 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.914+0000 7f4ccea2a640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cbc069eb0 0x7f4cbc06c370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4056000 tx=0x7f4cc404e070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.914+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5593fa08d910 0x5593fa0896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.914+0000 7f4ccfa2c640 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5593fa08d910 0x5593fa0896b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4ccfa2c640 1 -- 172.21.15.148:0/1784393597 <== osd.6 v2:172.21.15.188:6816/1362851547 1 ==== osd_op_reply(1 1000000096c.00000000 [rmxattr] v55'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4cc800f030 con 0x5593fa08d910 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5593fa08d910 msgr2=0x5593fa0896b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5593fa08d910 0x5593fa0896b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cbc069eb0 msgr2=0x7f4cbc06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cbc069eb0 0x7f4cbc06c370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4056000 tx=0x7f4cc404e070 comp rx=0 tx=0).stop 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593f9fb9de0 msgr2=0x5593fa087280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 --2- 172.21.15.148:0/1784393597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593f9fb9de0 0x5593fa087280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000bf50 tx=0x7f4cc00074b0 comp rx=0 tx=0).stop 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4ccfa2c640 1 -- 172.21.15.148:0/1784393597 reap_dead start 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 shutdown_connections 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 >> 172.21.15.148:0/1784393597 conn(0x5593f9fba9b0 msgr2=0x5593f9fe5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 shutdown_connections 2024-04-25T09:57:36.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:36.915+0000 7f4cd1728f40 1 -- 172.21.15.148:0/1784393597 wait complete. 2024-04-25T09:57:36.923 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T09:57:36.924 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:36.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T09:57:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 Processor -- start 2024-04-25T09:57:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 -- start start 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67bc15ba20 0x7f67bc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67bc0af260 0x7f67bc06c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc072aa0 con 0x7f67bc072610 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc06f300 con 0x7f67bc0af260 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67c1570640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc072d80 con 0x7f67bc15ba20 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.060+0000 7f67bb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67bc0af260 0x7f67bc06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67bc0af260 0x7f67bc06c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40854/0 (socket says 172.21.15.148:40854) 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3503420672 learned_addr learned my addr 172.21.15.148:0/3503420672 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3503420672 >> v2:172.21.15.188:3300/0 conn(0x7f67bc15ba20 msgr2=0x7f67bc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67baffd640 1 --2- 172.21.15.148:0/3503420672 >> v2:172.21.15.188:3300/0 conn(0x7f67bc15ba20 0x7f67bc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/3503420672 >> v2:172.21.15.188:3300/0 conn(0x7f67bc15ba20 0x7f67bc06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3503420672 >> v1:172.21.15.188:6789/0 conn(0x7f67bc072610 legacy=0x7f67bc0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3503420672 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc06f530 con 0x7f67bc0af260 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67baffd640 1 --2- 172.21.15.148:0/3503420672 >> v2:172.21.15.188:3300/0 conn(0x7f67bc15ba20 0x7f67bc06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.061+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/3503420672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc0af260 0x7f67bc06c130 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009870 tx=0x7f67b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8016264d5c02c82f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67b9ffb640 1 -- 172.21.15.148:0/3503420672 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4037070 con 0x7f67bc0af260 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67b9ffb640 1 -- 172.21.15.148:0/3503420672 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4032440 con 0x7f67bc0af260 2024-04-25T09:57:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67b9ffb640 1 -- 172.21.15.148:0/3503420672 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4032810 con 0x7f67bc0af260 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 -- 172.21.15.148:0/3503420672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc0af260 msgr2=0x7f67bc06c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 --2- 172.21.15.148:0/3503420672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc0af260 0x7f67bc06c130 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009870 tx=0x7f67b402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 -- 172.21.15.148:0/3503420672 shutdown_connections 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 --2- 172.21.15.148:0/3503420672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc0af260 0x7f67bc06c130 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 --2- 172.21.15.148:0/3503420672 >> v2:172.21.15.188:3300/0 conn(0x7f67bc15ba20 0x7f67bc06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 -- 172.21.15.148:0/3503420672 >> 172.21.15.148:0/3503420672 conn(0x7f67bc071af0 msgr2=0x7f67bc071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 -- 172.21.15.148:0/3503420672 shutdown_connections 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.062+0000 7f67c1570640 1 -- 172.21.15.148:0/3503420672 wait complete. 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 Processor -- start 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 -- start start 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67bc0af260 0x7f67bc1f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 0x7f67bc1f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67bc0c8380 0x7f67bc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc070010 con 0x7f67bc15ba20 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc06faf0 con 0x7f67bc0af260 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.064+0000 7f67c1570640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc15ca80 con 0x7f67bc0c8380 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 0x7f67bc1f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 0x7f67bc1f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40870/0 (socket says 172.21.15.148:40870) 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 -- 172.21.15.148:0/2352024181 learned_addr learned my addr 172.21.15.148:0/2352024181 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67baffd640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67bc0af260 0x7f67bc1f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67bc0c8380 0x7f67bc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 -- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67bc0c8380 msgr2=0x7f67bc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67bc0c8380 0x7f67bc0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 -- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67bc0af260 msgr2=0x7f67bc1f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67bc0af260 0x7f67bc1f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 -- 172.21.15.148:0/2352024181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc15c150 con 0x7f67bc15ba20 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67baffd640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67bc0af260 0x7f67bc1f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 0x7f67bc1f27d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c3f0 tx=0x7f67b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67a37fe640 1 -- 172.21.15.148:0/2352024181 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0007020 con 0x7f67bc15ba20 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67a37fe640 1 -- 172.21.15.148:0/2352024181 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b00071c0 con 0x7f67bc15ba20 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc15c430 con 0x7f67bc15ba20 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67a37fe640 1 -- 172.21.15.148:0/2352024181 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0007590 con 0x7f67bc15ba20 2024-04-25T09:57:37.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.065+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc15cf10 con 0x7f67bc15ba20 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.067+0000 7f67a37fe640 1 -- 172.21.15.148:0/2352024181 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67b0019020 con 0x7f67bc15ba20 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.067+0000 7f67a37fe640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67a806b4d0 0x7f67a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.067+0000 7f67a37fe640 1 -- 172.21.15.148:0/2352024181 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67b0089af0 con 0x7f67bc15ba20 2024-04-25T09:57:37.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.067+0000 7f67baffd640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67a806b4d0 0x7f67a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.067+0000 7f67baffd640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67a806b4d0 0x7f67a806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007f40 tx=0x7f67ac007490 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.068+0000 7f67c427b740 1 Processor -- start 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.068+0000 7f67c427b740 1 -- start start 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67c427b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cf2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19cf6360 0x55ca19cf2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67c427b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca19c57ee0 con 0x55ca19d1ec00 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca19c00c50 con 0x55ca19cf6360 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca19c53f80 con 0x55ca19d1c390 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d1efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40884/0 (socket says 172.21.15.148:40884) 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 -- 172.21.15.148:0/2536031964 learned_addr learned my addr 172.21.15.148:0/2536031964 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19cf6360 0x55ca19cf2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cf2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 -- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 msgr2=0x55ca19cf2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cf2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 -- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19cf6360 msgr2=0x55ca19cf2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19cf6360 0x55ca19cf2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 -- 172.21.15.148:0/2536031964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca19a38c00 con 0x55ca19d1ec00 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19cf6360 0x55ca19cf2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67baffd640 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d1efe0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00f770 tx=0x7f67ac010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3119807041839c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.069+0000 7f67a0ff9640 1 -- 172.21.15.148:0/2536031964 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac020070 con 0x55ca19d1ec00 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67a0ff9640 1 -- 172.21.15.148:0/2536031964 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00ea30 con 0x55ca19d1ec00 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67a0ff9640 1 -- 172.21.15.148:0/2536031964 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00ee00 con 0x55ca19d1ec00 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 -- 172.21.15.148:0/2536031964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 msgr2=0x55ca19d1efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d1efe0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00f770 tx=0x7f67ac010000 comp rx=0 tx=0).stop 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 -- 172.21.15.148:0/2536031964 shutdown_connections 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19cf6360 0x55ca19cf2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cf2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 --2- 172.21.15.148:0/2536031964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d1efe0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 -- 172.21.15.148:0/2536031964 >> 172.21.15.148:0/2536031964 conn(0x55ca19cbb810 msgr2=0x55ca19d39c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 -- 172.21.15.148:0/2536031964 shutdown_connections 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.070+0000 7f67c427b740 1 -- 172.21.15.148:0/2536031964 wait complete. 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.071+0000 7f67c427b740 1 -- start start 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.071+0000 7f67c427b740 1 Processor -- start 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d391b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1ec00 0x55ca19d20cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca19cef0d0 con 0x55ca19cf6360 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca19bf6340 con 0x55ca19d1c390 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca19ceee30 con 0x55ca19d1ec00 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67c427b740 10 client.0 _ll_register_callbacks cb 0x55ca1996bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40888/0 (socket says 172.21.15.148:40888) 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 -- 172.21.15.148:0/3648724370 learned_addr learned my addr 172.21.15.148:0/3648724370 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1ec00 0x55ca19d20cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 -- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1ec00 msgr2=0x55ca19d20cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1ec00 0x55ca19d20cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 -- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 msgr2=0x55ca19d207b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d207b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 -- 172.21.15.148:0/3648724370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca1992c4f0 con 0x55ca19cf6360 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f67baffd640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d391b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007cb0 tx=0x7f67ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2659e317172fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f6799ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f6799ffb640 1 -- 172.21.15.148:0/3648724370 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac020040 con 0x55ca19cf6360 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.072+0000 7f6799ffb640 1 -- 172.21.15.148:0/3648724370 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67ac00e830 con 0x55ca19cf6360 2024-04-25T09:57:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67ac00ea30 con 0x55ca19cf6360 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f6799ffb640 1 -- 172.21.15.148:0/3648724370 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67ac025660 con 0x55ca19cf6360 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 10 client.6438 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ca19a99390 con 0x55ca19cf6360 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f6799ffb640 1 -- 172.21.15.148:0/3648724370 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00ea30 con 0x55ca19cf6360 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f6799ffb640 1 -- 172.21.15.148:0/3648724370 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67ac038050 con 0x55ca19cf6360 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 10 client.6438 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 10 client.6438 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 20 client.6438 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d3c630 0x55ca19d4df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 4 client.6438 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca19c65420 con 0x55ca19d3c630 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d3c630 0x55ca19d4df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f6799ffb640 10 client.6438 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.073+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d3c630 0x55ca19d4df70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49187af4a831fb4 server_cookie=81219b47062e8786 in_seq=0 out_seq=0 2024-04-25T09:57:37.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f6799ffb640 1 -- 172.21.15.148:0/3648724370 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca19c65420 con 0x55ca19d3c630 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f6799ffb640 10 client.6438 handle_command_reply: tid=0 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 client.6438 shutdown 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 msgr2=0x55ca19d391b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d391b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007cb0 tx=0x7f67ac004ab0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 shutdown_connections 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1ec00 0x55ca19d20cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d3c630 0x55ca19d4df70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 --2- 172.21.15.148:0/3648724370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d391b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f6799ffb640 0 client.6438 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 >> 172.21.15.148:0/3648724370 conn(0x55ca19cbb810 msgr2=0x55ca19d3d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 shutdown_connections 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 1 -- 172.21.15.148:0/3648724370 wait complete. 2024-04-25T09:57:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.074+0000 7f67c427b740 20 client.6438 trim_cache size 0 max 16384 2024-04-25T09:57:37.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.089+0000 7f67c427b740 1 Processor -- start 2024-04-25T09:57:37.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.089+0000 7f67c427b740 1 -- start start 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67c427b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d12820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1ec00 0x55ca19d9f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67c427b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0cfa50 con 0x55ca19cf6360 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0cec30 con 0x55ca19d1ec00 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc147ed0 con 0x55ca19d1c390 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d12820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d12820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40896/0 (socket says 172.21.15.148:40896) 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 -- 172.21.15.148:0/321858523 learned_addr learned my addr 172.21.15.148:0/321858523 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 -- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 msgr2=0x55ca19cea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 -- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1ec00 msgr2=0x55ca19d9f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1ec00 0x55ca19d9f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1ec00 0x55ca19d9f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 -- 172.21.15.148:0/321858523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca19905670 con 0x55ca19cf6360 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1ec00 0x55ca19d9f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f67baffd640 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d12820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f67ac025a80 tx=0x7f67ac02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15873cc46d307609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f679bfff640 1 -- 172.21.15.148:0/321858523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac03f030 con 0x55ca19cf6360 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f679bfff640 1 -- 172.21.15.148:0/321858523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00e830 con 0x55ca19cf6360 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.090+0000 7f679bfff640 1 -- 172.21.15.148:0/321858523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac020070 con 0x55ca19cf6360 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 -- 172.21.15.148:0/321858523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 msgr2=0x55ca19d12820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d12820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f67ac025a80 tx=0x7f67ac02b000 comp rx=0 tx=0).stop 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 -- 172.21.15.148:0/321858523 shutdown_connections 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1ec00 0x55ca19d9f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19d1c390 0x55ca19cea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 --2- 172.21.15.148:0/321858523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19cf6360 0x55ca19d12820 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 -- 172.21.15.148:0/321858523 >> 172.21.15.148:0/321858523 conn(0x55ca19cbb810 msgr2=0x55ca19cf4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 -- 172.21.15.148:0/321858523 shutdown_connections 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.091+0000 7f67c427b740 1 -- 172.21.15.148:0/321858523 wait complete. 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 -- start start 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 Processor -- start 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19cf6360 0x55ca19d3db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca19d221e0 con 0x55ca19d1ec00 2024-04-25T09:57:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc1f5d00 con 0x55ca19d1c390 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca19d1be00 con 0x55ca19cf6360 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67c427b740 10 client.0 _ll_register_callbacks cb 0x7f67ac013070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.092+0000 7f67bb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d36190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40902/0 (socket says 172.21.15.148:40902) 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3090230220 learned_addr learned my addr 172.21.15.148:0/3090230220 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67ba7fc640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67baffd640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19cf6360 0x55ca19d3db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19cf6360 msgr2=0x55ca19d3db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19cf6360 0x55ca19d3db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 msgr2=0x55ca19d35c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d35c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 -- 172.21.15.148:0/3090230220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca19d571b0 con 0x55ca19d1ec00 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67bb7fe640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d36190 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f67b4035040 tx=0x7f67b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5870a2f533f41cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f6799ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f6799ffb640 1 -- 172.21.15.148:0/3090230220 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4042070 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f6799ffb640 1 -- 172.21.15.148:0/3090230220 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67b4047c40 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67ac00e830 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f6799ffb640 1 -- 172.21.15.148:0/3090230220 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b4037040 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 10 client.6444 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ca19da0660 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f6799ffb640 1 -- 172.21.15.148:0/3090230220 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00e830 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f6799ffb640 1 -- 172.21.15.148:0/3090230220 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67b404e050 con 0x55ca19d1ec00 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 10 client.6444 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 10 client.6444 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 20 client.6444 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d9c4b0 0x55ca19d1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 4 client.6444 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T09:57:37.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.093+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55ca19c5f770 con 0x55ca19d9c4b0 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67baffd640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d9c4b0 0x55ca19d1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f6799ffb640 10 client.6444 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67baffd640 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d9c4b0 0x55ca19d1aa10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8bbf8509669c373 server_cookie=e1219fccc8818fad in_seq=0 out_seq=0 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f6799ffb640 1 -- 172.21.15.148:0/3090230220 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ca19c5f770 con 0x55ca19d9c4b0 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f6799ffb640 10 client.6444 handle_command_reply: tid=0 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 client.6444 shutdown 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 msgr2=0x55ca19d36190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d36190 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f67b4035040 tx=0x7f67b4004730 comp rx=0 tx=0).stop 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 shutdown_connections 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca19d1ec00 0x55ca19d36190 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55ca19d9c4b0 0x55ca19d1aa10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f6799ffb640 0 client.6444 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca19d1c390 0x55ca19d35c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 --2- 172.21.15.148:0/3090230220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca19cf6360 0x55ca19d3db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 >> 172.21.15.148:0/3090230220 conn(0x55ca19cbb810 msgr2=0x55ca19cf4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.094+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 shutdown_connections 2024-04-25T09:57:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c427b740 1 -- 172.21.15.148:0/3090230220 wait complete. 2024-04-25T09:57:37.108 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f4887411640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096c [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v473 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5123=0-4194304@1} caps={5123=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5123 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5625f01fb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096c:[<0x10000000001/new_file_1 v473>,<0x10000000000/ceph-qa-suite v53>,<0x1/scrub_test_path v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T09:57:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c427b740 20 client.6444 trim_cache size 0 max 16384 2024-04-25T09:57:37.108 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:37.109 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:57:37.109 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "3a66ba86-7d8d-402f-a7fc-de76dd74607a", 2024-04-25T09:57:37.109 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:57:37.109 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67a806b4d0 msgr2=0x7f67a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67a806b4d0 0x7f67a806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67ac007f40 tx=0x7f67ac007490 comp rx=0 tx=0).stop 2024-04-25T09:57:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 msgr2=0x7f67bc1f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 0x7f67bc1f27d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c3f0 tx=0x7f67b000c8b0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 shutdown_connections 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67a806b4d0 0x7f67a806d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67bc0c8380 0x7f67bc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67bc15ba20 0x7f67bc1f27d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 --2- 172.21.15.148:0/2352024181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67bc0af260 0x7f67bc1f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 >> 172.21.15.148:0/2352024181 conn(0x7f67bc071af0 msgr2=0x7f67bc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 shutdown_connections 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.095+0000 7f67c1570640 1 -- 172.21.15.148:0/2352024181 wait complete. 2024-04-25T09:57:37.111 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '3a66ba86-7d8d-402f-a7fc-de76dd74607a', 'mode': 'asynchronous'}' 2024-04-25T09:57:37.111 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:57:37.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:57:37.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.207+0000 7f30508e2640 1 Processor -- start 2024-04-25T09:57:37.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.207+0000 7f30508e2640 1 -- start start 2024-04-25T09:57:37.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f30508e2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 0x7f304c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f30508e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f30508e2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c0cb190 con 0x7f304c0ccdd0 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f30508e2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c0cb470 con 0x7f304c15bb20 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f30508e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0cb750 con 0x7f304c0d00e0 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304a59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 0x7f304c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304a59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 0x7f304c0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:47246/0 (socket says 172.21.15.148:47246) 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f3049d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f304c0ccdd0 0x7f304c0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56388/0 (socket says 172.21.15.148:56388) 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304a59c640 1 -- 172.21.15.148:0/673173018 learned_addr learned my addr 172.21.15.148:0/673173018 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304ad9d640 1 -- 172.21.15.148:0/673173018 >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 msgr2=0x7f304c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304ad9d640 1 --2- 172.21.15.148:0/673173018 >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 0x7f304c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304ad9d640 1 -- 172.21.15.148:0/673173018 >> v1:172.21.15.188:6789/0 conn(0x7f304c0ccdd0 legacy=0x7f304c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304ad9d640 1 -- 172.21.15.148:0/673173018 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0cba30 con 0x7f304c0d00e0 2024-04-25T09:57:37.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.208+0000 7f304a59c640 1 --2- 172.21.15.148:0/673173018 >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 0x7f304c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:37.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.209+0000 7f304ad9d640 1 --2- 172.21.15.148:0/673173018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0ad600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3044009d90 tx=0x7f304402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59c69ea19697e1de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.209+0000 7f304959a640 1 -- 172.21.15.148:0/673173018 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044031600 con 0x7f304c0d00e0 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.209+0000 7f304959a640 1 -- 172.21.15.148:0/673173018 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30440317a0 con 0x7f304c0d00e0 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.209+0000 7f304959a640 1 -- 172.21.15.148:0/673173018 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044031b70 con 0x7f304c0d00e0 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 -- 172.21.15.148:0/673173018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 msgr2=0x7f304c0ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 --2- 172.21.15.148:0/673173018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0ad600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3044009d90 tx=0x7f304402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 -- 172.21.15.148:0/673173018 shutdown_connections 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 --2- 172.21.15.148:0/673173018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0ad600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 --2- 172.21.15.148:0/673173018 >> v2:172.21.15.188:3300/0 conn(0x7f304c15bb20 0x7f304c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 -- 172.21.15.148:0/673173018 >> 172.21.15.148:0/673173018 conn(0x7f304c0b2390 msgr2=0x7f304c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 -- 172.21.15.148:0/673173018 shutdown_connections 2024-04-25T09:57:37.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.210+0000 7f30508e2640 1 -- 172.21.15.148:0/673173018 wait complete. 2024-04-25T09:57:37.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.211+0000 7f30508e2640 1 Processor -- start 2024-04-25T09:57:37.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.211+0000 7f30508e2640 1 -- start start 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f30508e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f304c0ccdd0 0x7f304c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f30508e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f30508e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 0x7f304c0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f30508e2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15bf00 con 0x7f304c15bb20 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f30508e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0cc770 con 0x7f304c0ccdd0 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f30508e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15c080 con 0x7f304c0d00e0 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 0x7f304c0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 0x7f304c0b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40916/0 (socket says 172.21.15.148:40916) 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 -- 172.21.15.148:0/1940614496 learned_addr learned my addr 172.21.15.148:0/1940614496 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304a59c640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f304c0ccdd0 0x7f304c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f3049d9b640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 -- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 msgr2=0x7f304c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 -- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f304c0ccdd0 msgr2=0x7f304c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f304c0ccdd0 0x7f304c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304ad9d640 1 -- 172.21.15.148:0/1940614496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0b5bc0 con 0x7f304c15bb20 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.212+0000 7f304a59c640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f304c0ccdd0 0x7f304c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.213+0000 7f304ad9d640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 0x7f304c0b0510 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3044009eb0 tx=0x7f3044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.213+0000 7f303affd640 1 -- 172.21.15.148:0/1940614496 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044040070 con 0x7f304c15bb20 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.213+0000 7f303affd640 1 -- 172.21.15.148:0/1940614496 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044030b60 con 0x7f304c15bb20 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.213+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c0ca680 con 0x7f304c15bb20 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.213+0000 7f303affd640 1 -- 172.21.15.148:0/1940614496 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044009450 con 0x7f304c15bb20 2024-04-25T09:57:37.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.213+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c0b6160 con 0x7f304c15bb20 2024-04-25T09:57:37.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.214+0000 7f303affd640 1 -- 172.21.15.148:0/1940614496 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3044039040 con 0x7f304c15bb20 2024-04-25T09:57:37.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.215+0000 7f303affd640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f302c06b4d0 0x7f302c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.215+0000 7f303affd640 1 -- 172.21.15.148:0/1940614496 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30440ab7f0 con 0x7f304c15bb20 2024-04-25T09:57:37.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.215+0000 7f304a59c640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f302c06b4d0 0x7f302c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.215+0000 7f304a59c640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f302c06b4d0 0x7f302c06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f303c009630 tx=0x7f303c009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 Processor -- start 2024-04-25T09:57:37.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 -- start start 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0ef0 0x55aafcbf3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 0x55aafcbc6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafcb2cda0 con 0x55aafcbc3770 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafcad5b80 con 0x55aafcbf0b10 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.216+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aafcb28e40 con 0x55aafcbf0ef0 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 0x55aafcbc6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 0x55aafcbc6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40918/0 (socket says 172.21.15.148:40918) 2024-04-25T09:57:37.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 -- 172.21.15.148:0/3722996680 learned_addr learned my addr 172.21.15.148:0/3722996680 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304a59c640 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f3049d9b640 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0ef0 0x55aafcbf3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 -- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0ef0 msgr2=0x55aafcbf3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0ef0 0x55aafcbf3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 -- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 msgr2=0x55aafcbf3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 -- 172.21.15.148:0/3722996680 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aafcadff10 con 0x55aafcbc3770 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304a59c640 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f304ad9d640 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 0x55aafcbc6f80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3044076380 tx=0x7f30440799b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a0c124b97bbff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f302bfff640 1 -- 172.21.15.148:0/3722996680 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044091020 con 0x55aafcbc3770 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f302bfff640 1 -- 172.21.15.148:0/3722996680 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044031700 con 0x55aafcbc3770 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.217+0000 7f302bfff640 1 -- 172.21.15.148:0/3722996680 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044076590 con 0x55aafcbc3770 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 -- 172.21.15.148:0/3722996680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 msgr2=0x55aafcbc6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 0x55aafcbc6f80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3044076380 tx=0x7f30440799b0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 -- 172.21.15.148:0/3722996680 shutdown_connections 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbc3770 0x55aafcbc6f80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0ef0 0x55aafcbf3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 --2- 172.21.15.148:0/3722996680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 -- 172.21.15.148:0/3722996680 >> 172.21.15.148:0/3722996680 conn(0x55aafcbc59e0 msgr2=0x55aafcc0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 -- 172.21.15.148:0/3722996680 shutdown_connections 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 -- 172.21.15.148:0/3722996680 wait complete. 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.218+0000 7f30535e1740 1 -- start start 2024-04-25T09:57:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.219+0000 7f30535e1740 1 Processor -- start 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.219+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0b10 0x55aafcbe5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcc0f450 0x55aafcbef190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafcb2cda0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafcacb4b0 con 0x55aafcc0f450 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aafcbc8b90 con 0x55aafcbf0b10 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 10 client.0 _ll_register_callbacks cb 0x55aafcb0b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40934/0 (socket says 172.21.15.148:40934) 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 -- 172.21.15.148:0/1393918111 learned_addr learned my addr 172.21.15.148:0/1393918111 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f304a59c640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0b10 0x55aafcbe5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f304ad9d640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcc0f450 0x55aafcbef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 -- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0b10 msgr2=0x55aafcbe5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0b10 0x55aafcbe5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 -- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcc0f450 msgr2=0x55aafcbef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcc0f450 0x55aafcbef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 -- 172.21.15.148:0/1393918111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aafc90b7d0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f3049d9b640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf5610 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3040007fd0 tx=0x7f3040007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a013f67a7cee864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30297fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30297fa640 1 -- 172.21.15.148:0/1393918111 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040001b40 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30297fa640 1 -- 172.21.15.148:0/1393918111 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3040001ce0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3044031700 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30297fa640 1 -- 172.21.15.148:0/1393918111 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f304001f460 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 10 client.6453 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.220+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55aafc8413f0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30297fa640 1 -- 172.21.15.148:0/1393918111 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044031700 con 0x55aafcbf0ef0 2024-04-25T09:57:37.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30297fa640 1 -- 172.21.15.148:0/1393918111 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3040014400 con 0x55aafcbf0ef0 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30535e1740 10 client.6453 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30535e1740 10 client.6453 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30535e1740 20 client.6453 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30535e1740 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcbe6fa0 0x55aafcbe7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30535e1740 4 client.6453 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aafcb3a2e0 con 0x55aafcbe6fa0 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f304a59c640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcbe6fa0 0x55aafcbe7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f30297fa640 10 client.6453 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.221+0000 7f304a59c640 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcbe6fa0 0x55aafcbe7380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84fa0778624534a server_cookie=961bd6f1a30c1bd3 in_seq=0 out_seq=0 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30297fa640 1 -- 172.21.15.148:0/1393918111 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aafcb3a2e0 con 0x55aafcbe6fa0 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30297fa640 10 client.6453 handle_command_reply: tid=0 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 client.6453 shutdown 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 msgr2=0x55aafcbf5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf5610 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3040007fd0 tx=0x7f3040007720 comp rx=0 tx=0).stop 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 shutdown_connections 2024-04-25T09:57:37.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcbe6fa0 0x55aafcbe7380 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcc0f450 0x55aafcbef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf5610 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 --2- 172.21.15.148:0/1393918111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcbf0b10 0x55aafcbe5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30297fa640 0 client.6453 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 >> 172.21.15.148:0/1393918111 conn(0x55aafcbc3770 msgr2=0x55aafcbefd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 shutdown_connections 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 1 -- 172.21.15.148:0/1393918111 wait complete. 2024-04-25T09:57:37.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.222+0000 7f30535e1740 20 client.6453 trim_cache size 0 max 16384 2024-04-25T09:57:37.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.237+0000 7f30535e1740 1 Processor -- start 2024-04-25T09:57:37.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.237+0000 7f30535e1740 1 -- start start 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f30535e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcc15e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f30535e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1fe3e0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1fd5b0 con 0x55aafcbf0b10 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0c9db0 con 0x55aafcc0f450 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40944/0 (socket says 172.21.15.148:40944) 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 -- 172.21.15.148:0/4003846785 learned_addr learned my addr 172.21.15.148:0/4003846785 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f304a59c640 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f304ad9d640 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcc15e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 -- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 msgr2=0x55aafcc15e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcc15e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 -- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 msgr2=0x55aafcbf6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 -- 172.21.15.148:0/4003846785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aafcbe76f0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f3049d9b640 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf7450 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3040007760 tx=0x7f304000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111d3103e2858a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f302b7fe640 1 -- 172.21.15.148:0/4003846785 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304002e070 con 0x55aafcbf0ef0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f302b7fe640 1 -- 172.21.15.148:0/4003846785 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040019070 con 0x55aafcbf0ef0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.238+0000 7f302b7fe640 1 -- 172.21.15.148:0/4003846785 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040014400 con 0x55aafcbf0ef0 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 -- 172.21.15.148:0/4003846785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 msgr2=0x55aafcbf7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf7450 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3040007760 tx=0x7f304000e000 comp rx=0 tx=0).stop 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 -- 172.21.15.148:0/4003846785 shutdown_connections 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcc15e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbf7450 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 --2- 172.21.15.148:0/4003846785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcbf6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 -- 172.21.15.148:0/4003846785 >> 172.21.15.148:0/4003846785 conn(0x55aafcbc3770 msgr2=0x55aafcbe51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 -- 172.21.15.148:0/4003846785 shutdown_connections 2024-04-25T09:57:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 -- 172.21.15.148:0/4003846785 wait complete. 2024-04-25T09:57:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.239+0000 7f30535e1740 1 -- start start 2024-04-25T09:57:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 Processor -- start 2024-04-25T09:57:37.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcc75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcbefa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafcbe73f0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0ca3b0 con 0x55aafcbf0b10 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aafcbf3bf0 con 0x55aafcc0f450 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f30535e1740 10 client.0 _ll_register_callbacks cb 0x7f3044009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.240+0000 7f3049d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbea400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40946/0 (socket says 172.21.15.148:40946) 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 -- 172.21.15.148:0/3014868529 learned_addr learned my addr 172.21.15.148:0/3014868529 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f304a59c640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcc75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f304ad9d640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcbefa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 -- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 msgr2=0x55aafcbefa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcbefa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 -- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 msgr2=0x55aafcc75c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcc75c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 -- 172.21.15.148:0/3014868529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aafc8010f0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f304a59c640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcc75c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f3049d9b640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbea400 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f304002c040 tx=0x7f3040013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e954876eb3c072ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30297fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30297fa640 1 -- 172.21.15.148:0/3014868529 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040037060 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30297fa640 1 -- 172.21.15.148:0/3014868529 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f304001f600 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3040019070 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30297fa640 1 -- 172.21.15.148:0/3014868529 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f304003d050 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 10 client.6459 fetch_fsmap learned FSMap version 24 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55aafcc785e0 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30297fa640 1 -- 172.21.15.148:0/3014868529 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040019070 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30297fa640 1 -- 172.21.15.148:0/3014868529 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3040040020 con 0x55aafcbf0ef0 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 10 client.6459 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 10 client.6459 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 20 client.6459 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcc797f0 0x55aafcbeffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:37.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 4 client.6459 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.241+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aafcbf6cc0 con 0x55aafcc797f0 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f304a59c640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcc797f0 0x55aafcbeffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30297fa640 10 client.6459 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f304a59c640 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcc797f0 0x55aafcbeffa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1ed5531416f5b0 server_cookie=e9aa63723c27d6c5 in_seq=0 out_seq=0 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30297fa640 1 -- 172.21.15.148:0/3014868529 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x55aafcbf6cc0 con 0x55aafcc797f0 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30297fa640 10 client.6459 handle_command_reply: tid=0 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 client.6459 shutdown 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 msgr2=0x55aafcbea400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbea400 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f304002c040 tx=0x7f3040013000 comp rx=0 tx=0).stop 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 shutdown_connections 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aafcc0f450 0x55aafcbefa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aafcbf0ef0 0x55aafcbea400 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55aafcc797f0 0x55aafcbeffa0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 --2- 172.21.15.148:0/3014868529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aafcbf0b10 0x55aafcc75c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30297fa640 0 client.6459 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 >> 172.21.15.148:0/3014868529 conn(0x55aafcbc3770 msgr2=0x55aafcbf32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 shutdown_connections 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.242+0000 7f30535e1740 1 -- 172.21.15.148:0/3014868529 wait complete. 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30535e1740 20 client.6459 trim_cache size 0 max 16384 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: "3a66ba86-7d8d-402f-a7fc-de76dd74607a": { 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "3a66ba86-7d8d-402f-a7fc-de76dd74607a", 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "" 2024-04-25T09:57:37.252 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:37.253 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:37.253 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:37.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f302c06b4d0 msgr2=0x7f302c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f302c06b4d0 0x7f302c06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f303c009630 tx=0x7f303c009240 comp rx=0 tx=0).stop 2024-04-25T09:57:37.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 msgr2=0x7f304c0b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 0x7f304c0b0510 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3044009eb0 tx=0x7f3044004a40 comp rx=0 tx=0).stop 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 shutdown_connections 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f302c06b4d0 0x7f302c06d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c15bb20 0x7f304c0b0510 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f304c0d00e0 0x7f304c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 --2- 172.21.15.148:0/1940614496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f304c0ccdd0 0x7f304c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 >> 172.21.15.148:0/1940614496 conn(0x7f304c0b2390 msgr2=0x7f304c15a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 shutdown_connections 2024-04-25T09:57:37.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:37.243+0000 7f30508e2640 1 -- 172.21.15.148:0/1940614496 wait complete. 2024-04-25T09:57:42.258 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:57:42.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T09:57:42.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.367+0000 7f974005d640 1 Processor -- start 2024-04-25T09:57:42.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.367+0000 7f974005d640 1 -- start start 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f974005d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f97380b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f974005d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f973806e8a0 0x7f97380b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f974005d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380cccc0 con 0x7f9738144d50 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f974005d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97380b2b20 con 0x7f973806e8a0 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f974005d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973806d400 con 0x7f97381446d0 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f973d5f8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9738144d50 0x7f97380cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37528/0 (socket says 172.21.15.148:37528) 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.368+0000 7f973d5f8640 1 -- 172.21.15.148:0/1778290884 learned_addr learned my addr 172.21.15.148:0/1778290884 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973e5fa640 1 --2- 172.21.15.148:0/1778290884 >> v2:172.21.15.188:3300/0 conn(0x7f973806e8a0 0x7f97380b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973ddf9640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f97380b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905869261 0 0) 0x7f97380cccc0 con 0x7f9738144d50 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724003520 con 0x7f9738144d50 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4161745167 0 0) 0x7f9724003520 con 0x7f9738144d50 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 msgr2=0x7f97380b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f97380b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 >> v2:172.21.15.188:3300/0 conn(0x7f973806e8a0 msgr2=0x7f97380b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 --2- 172.21.15.148:0/1778290884 >> v2:172.21.15.188:3300/0 conn(0x7f973806e8a0 0x7f97380b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380cc140 con 0x7f9738144d50 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973e5fa640 1 --2- 172.21.15.148:0/1778290884 >> v2:172.21.15.188:3300/0 conn(0x7f973806e8a0 0x7f97380b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9728001670 con 0x7f9738144d50 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973ddf9640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f97380b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 >> v1:172.21.15.148:6789/0 conn(0x7f9738144d50 legacy=0x7f97380cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97240039b0 0x7f9724003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f9724004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724004d40 0x7f97240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380b2b20 con 0x7f97240039b0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973806d400 con 0x7f9724004d40 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972400b9c0 con 0x7f97381446d0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973ddf9640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97240039b0 0x7f9724003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973d5f8640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f9724004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.369+0000 7f973e5fa640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724004d40 0x7f97240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973ddf9640 1 -- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 msgr2=0x7f9724004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973ddf9640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97381446d0 0x7f9724004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973ddf9640 1 -- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724004d40 msgr2=0x7f97240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973ddf9640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724004d40 0x7f97240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973ddf9640 1 -- 172.21.15.148:0/1778290884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9724003520 con 0x7f97240039b0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973e5fa640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9724004d40 0x7f97240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973ddf9640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97240039b0 0x7f9724003dd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9734009c90 tx=0x7f9724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a505c8f3ab4805 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734036070 con 0x7f97240039b0 2024-04-25T09:57:42.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734031400 con 0x7f97240039b0 2024-04-25T09:57:42.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.370+0000 7f973cdf7640 1 -- 172.21.15.148:0/1778290884 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734009860 con 0x7f97240039b0 2024-04-25T09:57:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f974005d640 1 -- 172.21.15.148:0/1778290884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97240039b0 msgr2=0x7f9724003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f974005d640 1 --2- 172.21.15.148:0/1778290884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97240039b0 0x7f9724003dd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9734009c90 tx=0x7f9724005830 comp rx=0 tx=0).stop 2024-04-25T09:57:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f973e5fa640 1 -- 172.21.15.148:0/1778290884 reap_dead start 2024-04-25T09:57:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f974005d640 1 -- 172.21.15.148:0/1778290884 shutdown_connections 2024-04-25T09:57:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f974005d640 1 -- 172.21.15.148:0/1778290884 >> 172.21.15.148:0/1778290884 conn(0x7f973806abe0 msgr2=0x7f973806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f974005d640 1 -- 172.21.15.148:0/1778290884 shutdown_connections 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.371+0000 7f974005d640 1 -- 172.21.15.148:0/1778290884 wait complete. 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.372+0000 7f974005d640 1 Processor -- start 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.372+0000 7f974005d640 1 -- start start 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f974005d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240039b0 0x7f97380add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f974005d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97381565d0 0x7f97381f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f974005d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 0x7f9738142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f974005d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380cc700 con 0x7f9738142a50 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f974005d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380b49a0 con 0x7f97381565d0 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f974005d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380cc9e0 con 0x7f97240039b0 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 0x7f9738142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 0x7f9738142ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40958/0 (socket says 172.21.15.148:40958) 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 -- 172.21.15.148:0/3973059098 learned_addr learned my addr 172.21.15.148:0/3973059098 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973ddf9640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240039b0 0x7f97380add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973d5f8640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97381565d0 0x7f97381f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 -- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240039b0 msgr2=0x7f97380add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240039b0 0x7f97380add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 -- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97381565d0 msgr2=0x7f97381f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97381565d0 0x7f97381f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973e5fa640 1 -- 172.21.15.148:0/3973059098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97381581a0 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973ddf9640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240039b0 0x7f97380add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.373+0000 7f973d5f8640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97381565d0 0x7f97381f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.374+0000 7f973e5fa640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 0x7f9738142ed0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f30 tx=0x7f97300115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.374+0000 7f972e7fc640 1 -- 172.21.15.148:0/3973059098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730011ea0 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.374+0000 7f972e7fc640 1 -- 172.21.15.148:0/3973059098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973001b070 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.374+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9738156a50 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.374+0000 7f972e7fc640 1 -- 172.21.15.148:0/3973059098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730003990 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.374+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97380ce220 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.375+0000 7f972e7fc640 1 -- 172.21.15.148:0/3973059098 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9730003b30 con 0x7f9738142a50 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.376+0000 7f972e7fc640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f971c06b500 0x7f971c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.376+0000 7f972e7fc640 1 -- 172.21.15.148:0/3973059098 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f973008ab70 con 0x7f9738142a50 2024-04-25T09:57:42.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.376+0000 7f973ddf9640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f971c06b500 0x7f971c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.376+0000 7f973ddf9640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f971c06b500 0x7f971c06d9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9734033000 tx=0x7f9734008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 Processor -- start 2024-04-25T09:57:42.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 -- start start 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384450320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x558384450860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 0x558384423f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384389da0 con 0x558384420770 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384332b80 con 0x55838444def0 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.377+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558384385e40 con 0x55838444db10 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 0x558384423f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 0x558384423f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40970/0 (socket says 172.21.15.148:40970) 2024-04-25T09:57:42.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 -- 172.21.15.148:0/2554047191 learned_addr learned my addr 172.21.15.148:0/2554047191 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 -- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 msgr2=0x558384450320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384450320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 -- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 msgr2=0x558384450860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973d5f8640 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x558384450860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x558384450860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 -- 172.21.15.148:0/2554047191 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55838433cf10 con 0x558384420770 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f973e5fa640 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 0x558384423f80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9730088fa0 tx=0x7f9730003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edf8af370b97844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f971b7fe640 1 -- 172.21.15.148:0/2554047191 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300591f0 con 0x558384420770 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f971b7fe640 1 -- 172.21.15.148:0/2554047191 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973005d340 con 0x558384420770 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.378+0000 7f971b7fe640 1 -- 172.21.15.148:0/2554047191 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730061900 con 0x558384420770 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 -- 172.21.15.148:0/2554047191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 msgr2=0x558384423f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 0x558384423f80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9730088fa0 tx=0x7f9730003e20 comp rx=0 tx=0).stop 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 -- 172.21.15.148:0/2554047191 shutdown_connections 2024-04-25T09:57:42.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558384420770 0x558384423f80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x558384450860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 --2- 172.21.15.148:0/2554047191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384450320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 -- 172.21.15.148:0/2554047191 >> 172.21.15.148:0/2554047191 conn(0x5583844229e0 msgr2=0x55838446b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 -- 172.21.15.148:0/2554047191 shutdown_connections 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 -- 172.21.15.148:0/2554047191 wait complete. 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 -- start start 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.379+0000 7f9742d66740 1 Processor -- start 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444db10 0x558384452a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444def0 0x5583844542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x55838446a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558384425b90 con 0x55838447ae80 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583843284b0 con 0x55838444db10 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558384425e70 con 0x55838444def0 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.380+0000 7f9742d66740 10 client.0 _ll_register_callbacks cb 0x558384368180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x55838446a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x55838446a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40980/0 (socket says 172.21.15.148:40980) 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 -- 172.21.15.148:0/789495124 learned_addr learned my addr 172.21.15.148:0/789495124 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973ddf9640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444db10 0x558384452a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973d5f8640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444def0 0x5583844542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 -- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444def0 msgr2=0x5583844542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444def0 0x5583844542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 -- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444db10 msgr2=0x558384452a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444db10 0x558384452a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 -- 172.21.15.148:0/789495124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583841687d0 con 0x55838447ae80 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f973e5fa640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x55838446a320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9730003fd0 tx=0x7f973000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49133765f50f81a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9718ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9718ff9640 1 -- 172.21.15.148:0/789495124 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730061670 con 0x55838447ae80 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9718ff9640 1 -- 172.21.15.148:0/789495124 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9730058640 con 0x55838447ae80 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f973005d340 con 0x55838447ae80 2024-04-25T09:57:42.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9718ff9640 1 -- 172.21.15.148:0/789495124 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f973007f050 con 0x55838447ae80 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9742d66740 10 client.6471 fetch_fsmap learned FSMap version 25 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55838409e3f0 con 0x55838447ae80 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.381+0000 7f9718ff9640 1 -- 172.21.15.148:0/789495124 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973005d340 con 0x55838447ae80 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9718ff9640 1 -- 172.21.15.148:0/789495124 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9730082020 con 0x55838447ae80 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9742d66740 10 client.6471 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9742d66740 10 client.6471 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9742d66740 20 client.6471 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9742d66740 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838447b300 0x5583844444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9742d66740 4 client.6471 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583843972e0 con 0x55838447b300 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f973ddf9640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838447b300 0x5583844444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9718ff9640 10 client.6471 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f973ddf9640 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838447b300 0x5583844444d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae63380cae5f592e server_cookie=673e3206ab957b86 in_seq=0 out_seq=0 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9718ff9640 1 -- 172.21.15.148:0/789495124 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583843972e0 con 0x55838447b300 2024-04-25T09:57:42.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.382+0000 7f9718ff9640 10 client.6471 handle_command_reply: tid=0 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 client.6471 shutdown 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 msgr2=0x55838446a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x55838446a320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9730003fd0 tx=0x7f973000e000 comp rx=0 tx=0).stop 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 shutdown_connections 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x55838446a320 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444def0 0x5583844542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838447b300 0x5583844444d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 --2- 172.21.15.148:0/789495124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444db10 0x558384452a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9718ff9640 0 client.6471 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 >> 172.21.15.148:0/789495124 conn(0x558384420770 msgr2=0x558384450a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 shutdown_connections 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 1 -- 172.21.15.148:0/789495124 wait complete. 2024-04-25T09:57:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.383+0000 7f9742d66740 20 client.6471 trim_cache size 0 max 16384 2024-04-25T09:57:42.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.398+0000 7f9742d66740 1 Processor -- start 2024-04-25T09:57:42.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.398+0000 7f9742d66740 1 -- start start 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x5583844260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x55838441bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f9742d66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384452c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f9742d66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9738200ba0 con 0x55838447ae80 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97381ffd60 con 0x55838444def0 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97381ff160 con 0x55838444db10 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384452c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384452c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40986/0 (socket says 172.21.15.148:40986) 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 -- 172.21.15.148:0/541574879 learned_addr learned my addr 172.21.15.148:0/541574879 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973ddf9640 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x5583844260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973d5f8640 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x55838441bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 -- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 msgr2=0x5583844260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x5583844260b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 -- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 msgr2=0x55838441bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x55838441bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 -- 172.21.15.148:0/541574879 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55838446d9e0 con 0x55838447ae80 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973d5f8640 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x55838441bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f973e5fa640 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384452c70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f97300574a0 tx=0x7f9730072040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aba64409b2e9adc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f971affd640 1 -- 172.21.15.148:0/541574879 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973005d340 con 0x55838447ae80 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f971affd640 1 -- 172.21.15.148:0/541574879 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730065070 con 0x55838447ae80 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.399+0000 7f971affd640 1 -- 172.21.15.148:0/541574879 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730061470 con 0x55838447ae80 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 -- 172.21.15.148:0/541574879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 msgr2=0x558384452c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384452c70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f97300574a0 tx=0x7f9730072040 comp rx=0 tx=0).stop 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 -- 172.21.15.148:0/541574879 shutdown_connections 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384452c70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x55838441bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 --2- 172.21.15.148:0/541574879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x5583844260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 -- 172.21.15.148:0/541574879 >> 172.21.15.148:0/541574879 conn(0x558384420770 msgr2=0x5583844267a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 -- 172.21.15.148:0/541574879 shutdown_connections 2024-04-25T09:57:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 -- 172.21.15.148:0/541574879 wait complete. 2024-04-25T09:57:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 -- start start 2024-04-25T09:57:42.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.400+0000 7f9742d66740 1 Processor -- start 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384468ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x5583844444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384444a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583844535b0 con 0x55838447ae80 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97381ff760 con 0x55838444def0 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583843972e0 con 0x55838444db10 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f9742d66740 10 client.0 _ll_register_callbacks cb 0x7f9730058dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384444a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f973e5fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384444a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41002/0 (socket says 172.21.15.148:41002) 2024-04-25T09:57:42.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.401+0000 7f973e5fa640 1 -- 172.21.15.148:0/919811263 learned_addr learned my addr 172.21.15.148:0/919811263 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973ddf9640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384468ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973d5f8640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x5583844444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973e5fa640 1 -- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 msgr2=0x558384468ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973e5fa640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384468ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973e5fa640 1 -- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 msgr2=0x5583844444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973e5fa640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x5583844444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973e5fa640 1 -- 172.21.15.148:0/919811263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55838405e0f0 con 0x55838447ae80 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973d5f8640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x5583844444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f973e5fa640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384444a10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9730055b70 tx=0x7f9730055510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4325560184037e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9718ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9718ff9640 1 -- 172.21.15.148:0/919811263 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730061d50 con 0x55838447ae80 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9718ff9640 1 -- 172.21.15.148:0/919811263 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9730058640 con 0x55838447ae80 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9730065070 con 0x55838447ae80 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9718ff9640 1 -- 172.21.15.148:0/919811263 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f973005d340 con 0x55838447ae80 2024-04-25T09:57:42.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 10 client.6477 fetch_fsmap learned FSMap version 25 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55838405e0f0 con 0x55838447ae80 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9718ff9640 1 -- 172.21.15.148:0/919811263 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730065070 con 0x55838447ae80 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9718ff9640 1 -- 172.21.15.148:0/919811263 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9730082040 con 0x55838447ae80 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 10 client.6477 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 10 client.6477 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 20 client.6477 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838446c450 0x558384420b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 4 client.6477 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.402+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558384391630 con 0x55838446c450 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f973ddf9640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838446c450 0x558384420b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9718ff9640 10 client.6477 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f973ddf9640 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838446c450 0x558384420b90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8f69a2fa86b5e1 server_cookie=91439b9116288b76 in_seq=0 out_seq=0 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9718ff9640 1 -- 172.21.15.148:0/919811263 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558384391630 con 0x55838446c450 2024-04-25T09:57:42.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9718ff9640 10 client.6477 handle_command_reply: tid=0 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 client.6477 shutdown 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 msgr2=0x558384444a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384444a10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9730055b70 tx=0x7f9730055510 comp rx=0 tx=0).stop 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 shutdown_connections 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838447ae80 0x558384444a10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55838446c450 0x558384420b90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55838444def0 0x5583844444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 --2- 172.21.15.148:0/919811263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55838444db10 0x558384468ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9718ff9640 0 client.6477 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 >> 172.21.15.148:0/919811263 conn(0x558384420770 msgr2=0x558384425340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 shutdown_connections 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 1 -- 172.21.15.148:0/919811263 wait complete. 2024-04-25T09:57:42.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.403+0000 7f9742d66740 20 client.6477 trim_cache size 0 max 16384 2024-04-25T09:57:42.414 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T09:57:42.414 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T09:57:42.414 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f971c06b500 msgr2=0x7f971c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f971c06b500 0x7f971c06d9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9734033000 tx=0x7f9734008030 comp rx=0 tx=0).stop 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 msgr2=0x7f9738142ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 0x7f9738142ed0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f30 tx=0x7f97300115a0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 shutdown_connections 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f971c06b500 0x7f971c06d9c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9738142a50 0x7f9738142ed0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97381565d0 0x7f97381f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 --2- 172.21.15.148:0/3973059098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97240039b0 0x7f97380add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 >> 172.21.15.148:0/3973059098 conn(0x7f973806abe0 msgr2=0x7f97380af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 shutdown_connections 2024-04-25T09:57:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.404+0000 7f974005d640 1 -- 172.21.15.148:0/3973059098 wait complete. 2024-04-25T09:57:42.418 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:57:42.418 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:57:42.418 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T09:57:42.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.514+0000 7f88d3fff640 1 Processor -- start 2024-04-25T09:57:42.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.514+0000 7f88d3fff640 1 -- start start 2024-04-25T09:57:42.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88d406bff0 0x7f88d40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d3fff640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d415e110 con 0x7f88d415dd30 2024-04-25T09:57:42.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d3fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d40b2fd0 con 0x7f88d40b5970 2024-04-25T09:57:42.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d3fff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d406d000 con 0x7f88d406bff0 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f88d415dd30 0x7f88d40cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37532/0 (socket says 172.21.15.148:37532) 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d2ffd640 1 -- 172.21.15.148:0/1528997059 learned_addr learned my addr 172.21.15.148:0/1528997059 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217677516 0 0) 0x7f88d415e110 con 0x7f88d415dd30 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1528997059 >> v2:172.21.15.188:3300/0 conn(0x7f88d406bff0 0x7f88d40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88bc003520 con 0x7f88d415dd30 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089552662 0 0) 0x7f88bc003520 con 0x7f88d415dd30 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 >> v2:172.21.15.188:3300/0 conn(0x7f88d406bff0 msgr2=0x7f88d40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 --2- 172.21.15.148:0/1528997059 >> v2:172.21.15.188:3300/0 conn(0x7f88d406bff0 0x7f88d40b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 >> v1:172.21.15.188:6789/0 conn(0x7f88d40b5970 legacy=0x7f88d40d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d406c480 con 0x7f88d415dd30 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f88c4001670 con 0x7f88d415dd30 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 >> v1:172.21.15.148:6789/0 conn(0x7f88d415dd30 legacy=0x7f88d40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.515+0000 7f88d1ffb640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88bc0039b0 0x7f88bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d1ffb640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88bc004800 0x7f88bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d1ffb640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d406d000 con 0x7f88bc0051c0 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00c060 con 0x7f88bc0039b0 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc00c260 con 0x7f88bc004800 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d37fe640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88bc0039b0 0x7f88bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88bc004800 0x7f88bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d37fe640 1 -- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88bc004800 msgr2=0x7f88bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d37fe640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88bc004800 0x7f88bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d37fe640 1 -- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88bc0039b0 msgr2=0x7f88bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d37fe640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88bc0039b0 0x7f88bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.516+0000 7f88d37fe640 1 -- 172.21.15.148:0/1528997059 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc003520 con 0x7f88bc0051c0 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d37fe640 1 -- 172.21.15.148:0/1528997059 reap_dead start 2024-04-25T09:57:42.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d37fe640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88bc00b9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f88cc001e50 tx=0x7f88bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aab62b7a90f70ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc001a70 con 0x7f88bc0051c0 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc003970 con 0x7f88bc0051c0 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d1ffb640 1 -- 172.21.15.148:0/1528997059 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc003d60 con 0x7f88bc0051c0 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 -- 172.21.15.148:0/1528997059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 msgr2=0x7f88bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88bc00b9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f88cc001e50 tx=0x7f88bc005830 comp rx=0 tx=0).stop 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 -- 172.21.15.148:0/1528997059 shutdown_connections 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1528997059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88bc00b9c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 -- 172.21.15.148:0/1528997059 >> 172.21.15.148:0/1528997059 conn(0x7f88d40d0610 msgr2=0x7f88d40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 -- 172.21.15.148:0/1528997059 shutdown_connections 2024-04-25T09:57:42.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.517+0000 7f88d3fff640 1 -- 172.21.15.148:0/1528997059 wait complete. 2024-04-25T09:57:42.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.519+0000 7f88d3fff640 1 Processor -- start 2024-04-25T09:57:42.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.519+0000 7f88d3fff640 1 -- start start 2024-04-25T09:57:42.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.519+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88d40ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.519+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d40d2730 0x7f88d40b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d41f3770 0x7f88d415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d3fff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d406cd20 con 0x7f88bc0051c0 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d3fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d406ca40 con 0x7f88d40d2730 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d3fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40b5430 con 0x7f88d41f3770 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88d40ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88d40ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41016/0 (socket says 172.21.15.148:41016) 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 -- 172.21.15.148:0/1766259599 learned_addr learned my addr 172.21.15.148:0/1766259599 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d40d2730 0x7f88d40b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d37fe640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d41f3770 0x7f88d415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 -- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d41f3770 msgr2=0x7f88d415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d41f3770 0x7f88d415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 -- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d40d2730 msgr2=0x7f88d40b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d40d2730 0x7f88d40b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 -- 172.21.15.148:0/1766259599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40d2530 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88d40ccc50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f88c4007290 tx=0x7f88c4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88bb7fe640 1 -- 172.21.15.148:0/1766259599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c400f960 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88bb7fe640 1 -- 172.21.15.148:0/1766259599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4018070 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d40cd330 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88bb7fe640 1 -- 172.21.15.148:0/1766259599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c400fd60 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.520+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d40cd8f0 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.522+0000 7f88bb7fe640 1 -- 172.21.15.148:0/1766259599 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88c401d460 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.522+0000 7f88bb7fe640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88c006b4d0 0x7f88c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.522+0000 7f88bb7fe640 1 -- 172.21.15.148:0/1766259599 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f88c408a400 con 0x7f88bc0051c0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.522+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88c006b4d0 0x7f88c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.523+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88c006b4d0 0x7f88c006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88d40d2230 tx=0x7f88c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.523+0000 7f88dc1de740 1 Processor -- start 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.523+0000 7f88dc1de740 1 -- start start 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 0x5624eb81b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb819090 0x5624eb81ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb7eb910 0x5624eb7ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88dc1de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624eb754e00 con 0x5624eb818cb0 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624eb6fdd20 con 0x5624eb7eb910 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624eb750ea0 con 0x5624eb819090 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 0x5624eb81b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 0x5624eb81b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41032/0 (socket says 172.21.15.148:41032) 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 -- 172.21.15.148:0/2466674362 learned_addr learned my addr 172.21.15.148:0/2466674362 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d37fe640 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb7eb910 0x5624eb7ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d27fc640 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb819090 0x5624eb81ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 -- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb819090 msgr2=0x5624eb81ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb819090 0x5624eb81ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 -- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb7eb910 msgr2=0x5624eb7ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb7eb910 0x5624eb7ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.524+0000 7f88d2ffd640 1 -- 172.21.15.148:0/2466674362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624eb71fdf0 con 0x5624eb818cb0 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 0x5624eb81b4c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f88c4001970 tx=0x7f88c4058350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5720e04d02763c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88b8ff9640 1 -- 172.21.15.148:0/2466674362 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c405cb40 con 0x5624eb818cb0 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88b8ff9640 1 -- 172.21.15.148:0/2466674362 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4061100 con 0x5624eb818cb0 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88b8ff9640 1 -- 172.21.15.148:0/2466674362 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4055060 con 0x5624eb818cb0 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 -- 172.21.15.148:0/2466674362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 msgr2=0x5624eb81b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 0x5624eb81b4c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f88c4001970 tx=0x7f88c4058350 comp rx=0 tx=0).stop 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 -- 172.21.15.148:0/2466674362 shutdown_connections 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb7eb910 0x5624eb7ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb819090 0x5624eb81ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 --2- 172.21.15.148:0/2466674362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb818cb0 0x5624eb81b4c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.525+0000 7f88dc1de740 1 -- 172.21.15.148:0/2466674362 >> 172.21.15.148:0/2466674362 conn(0x5624eb7edb80 msgr2=0x5624eb8364c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.526+0000 7f88dc1de740 1 -- 172.21.15.148:0/2466674362 shutdown_connections 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.526+0000 7f88dc1de740 1 -- 172.21.15.148:0/2466674362 wait complete. 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.526+0000 7f88dc1de740 1 -- start start 2024-04-25T09:57:42.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.526+0000 7f88dc1de740 1 Processor -- start 2024-04-25T09:57:42.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb80f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb8385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb838b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624eb7f0d30 con 0x5624eb819090 2024-04-25T09:57:42.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624eb6f3590 con 0x5624eb818cb0 2024-04-25T09:57:42.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624eb7f1010 con 0x5624eb846ed0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88dc1de740 10 client.0 _ll_register_callbacks cb 0x5624eb6ffe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb8385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb8385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41036/0 (socket says 172.21.15.148:41036) 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 -- 172.21.15.148:0/1535790528 learned_addr learned my addr 172.21.15.148:0/1535790528 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb80f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d37fe640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb838b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 -- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 msgr2=0x5624eb838b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb838b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 -- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 msgr2=0x5624eb80f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb80f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 -- 172.21.15.148:0/1535790528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624eb4693f0 con 0x5624eb819090 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88d27fc640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb8385c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f88c8004610 tx=0x7f88c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32883aabd167b0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.527+0000 7f88b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:42.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88b1ffb640 1 -- 172.21.15.148:0/1535790528 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8029020 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88b1ffb640 1 -- 172.21.15.148:0/1535790528 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f88c8025650 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88c4061100 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88b1ffb640 1 -- 172.21.15.148:0/1535790528 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c8033050 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 10 client.6489 fetch_fsmap learned FSMap version 25 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5624eb5337d0 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88b1ffb640 1 -- 172.21.15.148:0/1535790528 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4061100 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88b1ffb640 1 -- 172.21.15.148:0/1535790528 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f88c8036020 con 0x5624eb819090 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 10 client.6489 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 10 client.6489 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 20 client.6489 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb81c390 0x5624eb81c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 4 client.6489 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624eb762340 con 0x5624eb81c390 2024-04-25T09:57:42.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb81c390 0x5624eb81c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88b1ffb640 10 client.6489 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.528+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb81c390 0x5624eb81c7b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ff4b8fdd78f01b server_cookie=df5cffad27bb2642 in_seq=0 out_seq=0 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88b1ffb640 1 -- 172.21.15.148:0/1535790528 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624eb762340 con 0x5624eb81c390 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88b1ffb640 10 client.6489 handle_command_reply: tid=0 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 client.6489 shutdown 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 msgr2=0x5624eb8385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb8385c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f88c8004610 tx=0x7f88c8004730 comp rx=0 tx=0).stop 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 shutdown_connections 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb838b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb8385c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb81c390 0x5624eb81c7b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 --2- 172.21.15.148:0/1535790528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb80f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 >> 172.21.15.148:0/1535790528 conn(0x5624eb7eb910 msgr2=0x5624eb816b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88b1ffb640 0 client.6489 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 shutdown_connections 2024-04-25T09:57:42.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 1 -- 172.21.15.148:0/1535790528 wait complete. 2024-04-25T09:57:42.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.529+0000 7f88dc1de740 20 client.6489 trim_cache size 0 max 16384 2024-04-25T09:57:42.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.541+0000 7f88dc1de740 1 Processor -- start 2024-04-25T09:57:42.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.541+0000 7f88dc1de740 1 -- start start 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb836c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb816520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb816a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88dc1de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d42009a0 con 0x5624eb819090 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d41ffbb0 con 0x5624eb818cb0 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d41ff050 con 0x5624eb846ed0 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb816520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb816520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41040/0 (socket says 172.21.15.148:41040) 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 -- 172.21.15.148:0/4199390514 learned_addr learned my addr 172.21.15.148:0/4199390514 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 -- 172.21.15.148:0/4199390514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 msgr2=0x5624eb816a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb816a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 -- 172.21.15.148:0/4199390514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 msgr2=0x5624eb836c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb836c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 -- 172.21.15.148:0/4199390514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624eb8364c0 con 0x5624eb819090 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88d27fc640 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb816520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009850 tx=0x7f88c8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5383aabd046f6de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88b3fff640 1 -- 172.21.15.148:0/4199390514 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8029070 con 0x5624eb819090 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88b3fff640 1 -- 172.21.15.148:0/4199390514 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c803c070 con 0x5624eb819090 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.542+0000 7f88b3fff640 1 -- 172.21.15.148:0/4199390514 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8025d50 con 0x5624eb819090 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 -- 172.21.15.148:0/4199390514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 msgr2=0x5624eb816520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb816520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009850 tx=0x7f88c8025000 comp rx=0 tx=0).stop 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 -- 172.21.15.148:0/4199390514 shutdown_connections 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb846ed0 0x5624eb816a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb816520 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 --2- 172.21.15.148:0/4199390514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb818cb0 0x5624eb836c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 -- 172.21.15.148:0/4199390514 >> 172.21.15.148:0/4199390514 conn(0x5624eb7eb910 msgr2=0x5624eb80ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 -- 172.21.15.148:0/4199390514 shutdown_connections 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 -- 172.21.15.148:0/4199390514 wait complete. 2024-04-25T09:57:42.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 -- start start 2024-04-25T09:57:42.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.543+0000 7f88dc1de740 1 Processor -- start 2024-04-25T09:57:42.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb818cb0 0x5624eb898780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb83e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb846ed0 0x5624eb83e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d41ff5b0 con 0x5624eb819090 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624eb7886a0 con 0x5624eb846ed0 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624eb817970 con 0x5624eb818cb0 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88dc1de740 10 client.0 _ll_register_callbacks cb 0x7f88c4004ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb83e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb83e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41046/0 (socket says 172.21.15.148:41046) 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 -- 172.21.15.148:0/615194637 learned_addr learned my addr 172.21.15.148:0/615194637 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d37fe640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb846ed0 0x5624eb83e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb818cb0 0x5624eb898780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 -- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb818cb0 msgr2=0x5624eb898780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb818cb0 0x5624eb898780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 -- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb846ed0 msgr2=0x5624eb83e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb846ed0 0x5624eb83e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d27fc640 1 -- 172.21.15.148:0/615194637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624eb7ec260 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.544+0000 7f88d37fe640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb846ed0 0x5624eb83e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88d27fc640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb83e1e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f88c8014720 tx=0x7f88c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95075177d2d05919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88b1ffb640 1 -- 172.21.15.148:0/615194637 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8029040 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88b1ffb640 1 -- 172.21.15.148:0/615194637 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f88c8005da0 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88c803c070 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88b1ffb640 1 -- 172.21.15.148:0/615194637 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c8051050 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 10 client.6495 fetch_fsmap learned FSMap version 25 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5624eb817770 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88b1ffb640 1 -- 172.21.15.148:0/615194637 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c803c070 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88b1ffb640 1 -- 172.21.15.148:0/615194637 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f88c8054020 con 0x5624eb819090 2024-04-25T09:57:42.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 10 client.6495 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 10 client.6495 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 20 client.6495 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb7e7270 0x5624eb8a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 4 client.6495 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624eb75c690 con 0x5624eb7e7270 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.545+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb7e7270 0x5624eb8a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88b1ffb640 10 client.6495 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d2ffd640 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb7e7270 0x5624eb8a2060 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4139ae4d5edf12c2 server_cookie=b30070a577c80524 in_seq=0 out_seq=0 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88b1ffb640 1 -- 172.21.15.148:0/615194637 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x5624eb75c690 con 0x5624eb7e7270 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88b1ffb640 10 client.6495 handle_command_reply: tid=0 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 client.6495 shutdown 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 msgr2=0x5624eb83e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb83e1e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f88c8014720 tx=0x7f88c8018000 comp rx=0 tx=0).stop 2024-04-25T09:57:42.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 shutdown_connections 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x5624eb7e7270 0x5624eb8a2060 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5624eb846ed0 0x5624eb83e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624eb819090 0x5624eb83e1e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88b1ffb640 0 client.6495 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 --2- 172.21.15.148:0/615194637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5624eb818cb0 0x5624eb898780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 >> 172.21.15.148:0/615194637 conn(0x5624eb7eb910 msgr2=0x5624eb816520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 shutdown_connections 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 1 -- 172.21.15.148:0/615194637 wait complete. 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88dc1de740 20 client.6495 trim_cache size 0 max 16384 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:57:42.556 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:42.557 INFO:teuthology.orchestra.run.smithi148.stdout: "damage_type": "backtrace", 2024-04-25T09:57:42.557 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 3030538701, 2024-04-25T09:57:42.557 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 1099511630188, 2024-04-25T09:57:42.557 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T09:57:42.557 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:42.557 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88c006b4d0 msgr2=0x7f88c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88c006b4d0 0x7f88c006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f88d40d2230 tx=0x7f88c8008040 comp rx=0 tx=0).stop 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 msgr2=0x7f88d40ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88d40ccc50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f88c4007290 tx=0x7f88c4007750 comp rx=0 tx=0).stop 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 shutdown_connections 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88c006b4d0 0x7f88c006d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d41f3770 0x7f88d415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d40d2730 0x7f88d40b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 --2- 172.21.15.148:0/1766259599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88bc0051c0 0x7f88d40ccc50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 >> 172.21.15.148:0/1766259599 conn(0x7f88d40d0610 msgr2=0x7f88d40b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.546+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 shutdown_connections 2024-04-25T09:57:42.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.547+0000 7f88d3fff640 1 -- 172.21.15.148:0/1766259599 wait complete. 2024-04-25T09:57:42.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.654+0000 7f0e08529640 1 Processor -- start 2024-04-25T09:57:42.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.654+0000 7f0e08529640 1 -- start start 2024-04-25T09:57:42.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.655+0000 7f0e08529640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e0015dbe0 0x7f0e000d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.655+0000 7f0e08529640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e000b2cb0 0x7f0e000d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.655+0000 7f0e08529640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e000b97a0 con 0x7f0e000b92a0 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.655+0000 7f0e08529640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0015dfc0 con 0x7f0e0015dbe0 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.655+0000 7f0e08529640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e000d1a00 con 0x7f0e000b2cb0 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e0015dbe0 0x7f0e000d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e0015dbe0 0x7f0e000d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41056/0 (socket says 172.21.15.148:41056) 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 -- 172.21.15.148:0/2901954972 learned_addr learned my addr 172.21.15.148:0/2901954972 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e05ac4640 1 --2- 172.21.15.148:0/2901954972 >> v2:172.21.15.188:3300/0 conn(0x7f0e000b2cb0 0x7f0e000d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 -- 172.21.15.148:0/2901954972 >> v2:172.21.15.188:3300/0 conn(0x7f0e000b2cb0 msgr2=0x7f0e000d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 --2- 172.21.15.148:0/2901954972 >> v2:172.21.15.188:3300/0 conn(0x7f0e000b2cb0 0x7f0e000d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 -- 172.21.15.148:0/2901954972 >> v1:172.21.15.188:6789/0 conn(0x7f0e000b92a0 legacy=0x7f0e000b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.656+0000 7f0e062c5640 1 -- 172.21.15.148:0/2901954972 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000d1c50 con 0x7f0e0015dbe0 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e062c5640 1 --2- 172.21.15.148:0/2901954972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e0015dbe0 0x7f0e000d0910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0e0006cff0 tx=0x7f0dfc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8efdfe6d500c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e052c3640 1 -- 172.21.15.148:0/2901954972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc037070 con 0x7f0e0015dbe0 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e052c3640 1 -- 172.21.15.148:0/2901954972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc032440 con 0x7f0e0015dbe0 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e052c3640 1 -- 172.21.15.148:0/2901954972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc032810 con 0x7f0e0015dbe0 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 -- 172.21.15.148:0/2901954972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e0015dbe0 msgr2=0x7f0e000d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 --2- 172.21.15.148:0/2901954972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e0015dbe0 0x7f0e000d0910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0e0006cff0 tx=0x7f0dfc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 -- 172.21.15.148:0/2901954972 shutdown_connections 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 --2- 172.21.15.148:0/2901954972 >> v2:172.21.15.188:3300/0 conn(0x7f0e000b2cb0 0x7f0e000d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 --2- 172.21.15.148:0/2901954972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e0015dbe0 0x7f0e000d0910 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 -- 172.21.15.148:0/2901954972 >> 172.21.15.148:0/2901954972 conn(0x7f0e00061a20 msgr2=0x7f0e00061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 -- 172.21.15.148:0/2901954972 shutdown_connections 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.657+0000 7f0e08529640 1 -- 172.21.15.148:0/2901954972 wait complete. 2024-04-25T09:57:42.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.658+0000 7f0e08529640 1 Processor -- start 2024-04-25T09:57:42.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.658+0000 7f0e08529640 1 -- start start 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e08529640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e000b2cb0 0x7f0e000c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e08529640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 0x7f0e000cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e08529640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e0015dbe0 0x7f0e000b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e08529640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000b6ab0 con 0x7f0e000b92a0 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e08529640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000d2210 con 0x7f0e0015dbe0 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e08529640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000b6c30 con 0x7f0e000b2cb0 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e05ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 0x7f0e000cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e05ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 0x7f0e000cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41072/0 (socket says 172.21.15.148:41072) 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.659+0000 7f0e05ac4640 1 -- 172.21.15.148:0/1221265474 learned_addr learned my addr 172.21.15.148:0/1221265474 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e06ac6640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e0015dbe0 0x7f0e000b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e05ac4640 1 -- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e000b2cb0 msgr2=0x7f0e000c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e062c5640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e000b2cb0 0x7f0e000c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e05ac4640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e000b2cb0 0x7f0e000c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e05ac4640 1 -- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e0015dbe0 msgr2=0x7f0e000b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e05ac4640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e0015dbe0 0x7f0e000b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e05ac4640 1 -- 172.21.15.148:0/1221265474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000cb040 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e05ac4640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 0x7f0e000cab00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0df000d7d0 tx=0x7f0df000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000b6c0 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df0019070 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e000b4a90 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000bac0 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.660+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e000b5450 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.661+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e00073990 con 0x7f0e000b92a0 2024-04-25T09:57:42.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.661+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0df000bc60 con 0x7f0e000b92a0 2024-04-25T09:57:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.662+0000 7f0df6ffd640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0de406b500 0x7f0de406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.662+0000 7f0e062c5640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0de406b500 0x7f0de406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.662+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0df008a7f0 con 0x7f0e000b92a0 2024-04-25T09:57:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.662+0000 7f0e062c5640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0de406b500 0x7f0de406d9c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc034000 tx=0x7f0dfc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.666+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0df001c360 con 0x7f0e000b92a0 2024-04-25T09:57:42.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.870+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e0006c770 con 0x7f0e000b92a0 2024-04-25T09:57:42.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.870+0000 7f0df6ffd640 1 -- 172.21.15.148:0/1221265474 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f0df00594e0 con 0x7f0e000b92a0 2024-04-25T09:57:42.874 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:42.874 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:39.018466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:42.875 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 25 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0de406b500 msgr2=0x7f0de406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0de406b500 0x7f0de406d9c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc034000 tx=0x7f0dfc02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 msgr2=0x7f0e000cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 0x7f0e000cab00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0df000d7d0 tx=0x7f0df000dc90 comp rx=0 tx=0).stop 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 shutdown_connections 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0de406b500 0x7f0de406d9c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e0015dbe0 0x7f0e000b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e000b92a0 0x7f0e000cab00 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 --2- 172.21.15.148:0/1221265474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e000b2cb0 0x7f0e000c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.872+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 >> 172.21.15.148:0/1221265474 conn(0x7f0e00061a20 msgr2=0x7f0e0015c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.873+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 shutdown_connections 2024-04-25T09:57:42.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.873+0000 7f0e08529640 1 -- 172.21.15.148:0/1221265474 wait complete. 2024-04-25T09:57:42.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3030538701 2024-04-25T09:57:42.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.983+0000 7fa00c8e1640 1 Processor -- start 2024-04-25T09:57:42.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.984+0000 7fa00c8e1640 1 -- start start 2024-04-25T09:57:42.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.984+0000 7fa00c8e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 0x7fa00806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.984+0000 7fa00c8e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa00815deb0 0x7fa00815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.984+0000 7fa00c8e1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0080d1770 con 0x7fa0080711a0 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.984+0000 7fa00c8e1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0080b0eb0 con 0x7fa00815deb0 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.984+0000 7fa00c8e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081622b0 con 0x7fa008070dc0 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa00659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 0x7fa00806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa006d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa00815deb0 0x7fa00815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa00659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 0x7fa00806ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:45366/0 (socket says 172.21.15.148:45366) 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa006d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa00815deb0 0x7fa00815e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:47462/0 (socket says 172.21.15.148:47462) 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa00659c640 1 -- 172.21.15.148:0/2578452746 learned_addr learned my addr 172.21.15.148:0/2578452746 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa006d9d640 1 -- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 msgr2=0x7fa00806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa006d9d640 1 --2- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 0x7fa00806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa006d9d640 1 -- 172.21.15.148:0/2578452746 >> v1:172.21.15.188:6789/0 conn(0x7fa0080711a0 legacy=0x7fa0080b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa006d9d640 1 -- 172.21.15.148:0/2578452746 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008162740 con 0x7fa00815deb0 2024-04-25T09:57:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.985+0000 7fa00659c640 1 --2- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 0x7fa00806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.986+0000 7fa006d9d640 1 --2- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00815deb0 0x7fa00815e340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa000009d90 tx=0x7fa00002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74f2a755f9fe438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.986+0000 7fa00559a640 1 -- 172.21.15.148:0/2578452746 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000007740 con 0x7fa00815deb0 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.986+0000 7fa00559a640 1 -- 172.21.15.148:0/2578452746 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0000078e0 con 0x7fa00815deb0 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.986+0000 7fa00559a640 1 -- 172.21.15.148:0/2578452746 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000007cb0 con 0x7fa00815deb0 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.986+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00815deb0 msgr2=0x7fa00815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.986+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00815deb0 0x7fa00815e340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa000009d90 tx=0x7fa00002a530 comp rx=0 tx=0).stop 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.987+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2578452746 shutdown_connections 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.987+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00815deb0 0x7fa00815e340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.987+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2578452746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008070dc0 0x7fa00806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.987+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2578452746 >> 172.21.15.148:0/2578452746 conn(0x7fa0080b3530 msgr2=0x7fa0080b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:42.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.987+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2578452746 shutdown_connections 2024-04-25T09:57:42.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.987+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2578452746 wait complete. 2024-04-25T09:57:42.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.988+0000 7fa00c8e1640 1 Processor -- start 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.988+0000 7fa00c8e1640 1 -- start start 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00c8e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa008070dc0 0x7fa008159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00c8e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0080711a0 0x7fa008155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00c8e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 0x7fa0080cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00c8e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa008161c60 con 0x7fa00815deb0 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00c8e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080b10f0 con 0x7fa008070dc0 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00c8e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa008161f40 con 0x7fa0080711a0 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa006d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 0x7fa0080cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa008070dc0 0x7fa008159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa006d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 0x7fa0080cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41076/0 (socket says 172.21.15.148:41076) 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa006d9d640 1 -- 172.21.15.148:0/2995283414 learned_addr learned my addr 172.21.15.148:0/2995283414 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa005d9b640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0080711a0 0x7fa008155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 -- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0080711a0 msgr2=0x7fa008155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0080711a0 0x7fa008155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 -- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 msgr2=0x7fa0080cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 0x7fa0080cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 -- 172.21.15.148:0/2995283414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008157f10 con 0x7fa008070dc0 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa006d9d640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 0x7fa0080cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.989+0000 7fa00659c640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa008070dc0 0x7fa008159a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e930 tx=0x7f9ff000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.990+0000 7f9ffeffd640 1 -- 172.21.15.148:0/2995283414 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000ca00 con 0x7fa008070dc0 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.990+0000 7f9ffeffd640 1 -- 172.21.15.148:0/2995283414 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff000cba0 con 0x7fa008070dc0 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.990+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0081581f0 con 0x7fa008070dc0 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.990+0000 7f9ffeffd640 1 -- 172.21.15.148:0/2995283414 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004030 con 0x7fa008070dc0 2024-04-25T09:57:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.990+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081587b0 con 0x7fa008070dc0 2024-04-25T09:57:42.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.991+0000 7f9ffeffd640 1 -- 172.21.15.148:0/2995283414 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ff00041d0 con 0x7fa008070dc0 2024-04-25T09:57:42.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.992+0000 7f9ffeffd640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9fe806b4d0 0x7f9fe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.992+0000 7f9ffeffd640 1 -- 172.21.15.148:0/2995283414 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ff008a6d0 con 0x7fa008070dc0 2024-04-25T09:57:42.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.992+0000 7fa005d9b640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9fe806b4d0 0x7f9fe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:42.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.993+0000 7fa005d9b640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9fe806b4d0 0x7f9fe806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa00815dbe0 tx=0x7f9ff8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:42.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.993+0000 7fa00f5e7740 1 Processor -- start 2024-04-25T09:57:42.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.993+0000 7fa00f5e7740 1 -- start start 2024-04-25T09:57:42.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f1fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 0x55a17f1ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:42.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a17f133ac0 con 0x55a17f1cabe0 2024-04-25T09:57:42.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a17f0dcdb0 con 0x55a17f1f7f80 2024-04-25T09:57:42.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a17f12fb60 con 0x55a17f1f8360 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 0x55a17f1ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa00659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f1fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 0x55a17f1ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41082/0 (socket says 172.21.15.148:41082) 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 -- 172.21.15.148:0/4063599333 learned_addr learned my addr 172.21.15.148:0/4063599333 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa005d9b640 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 -- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 msgr2=0x55a17f1facd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1facd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 -- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 msgr2=0x55a17f1fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f1fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 -- 172.21.15.148:0/4063599333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a17ef14710 con 0x55a17f1cabe0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7fa006d9d640 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 0x55a17f1ce3f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa000009e80 tx=0x7fa00002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f618594f99e730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.994+0000 7f9fe7fff640 1 -- 172.21.15.148:0/4063599333 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000007800 con 0x55a17f1cabe0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7f9fe7fff640 1 -- 172.21.15.148:0/4063599333 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000044430 con 0x55a17f1cabe0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7f9fe7fff640 1 -- 172.21.15.148:0/4063599333 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0000447e0 con 0x55a17f1cabe0 2024-04-25T09:57:43.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 -- 172.21.15.148:0/4063599333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 msgr2=0x55a17f1ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 0x55a17f1ce3f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa000009e80 tx=0x7fa00002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 -- 172.21.15.148:0/4063599333 shutdown_connections 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1cabe0 0x55a17f1ce3f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1facd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/4063599333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f1fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 -- 172.21.15.148:0/4063599333 >> 172.21.15.148:0/4063599333 conn(0x55a17f1cce50 msgr2=0x55a17f215790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 -- 172.21.15.148:0/4063599333 shutdown_connections 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.995+0000 7fa00f5e7740 1 -- 172.21.15.148:0/4063599333 wait complete. 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.996+0000 7fa00f5e7740 1 -- start start 2024-04-25T09:57:43.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.996+0000 7fa00f5e7740 1 Processor -- start 2024-04-25T09:57:43.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f8360 0x55a17f2186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f1cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a17f1d0000 con 0x55a17f1f7f80 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a17f0d24e0 con 0x55a17f1f8360 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a17f1d02e0 con 0x55a17f1cb890 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00f5e7740 10 client.0 _ll_register_callbacks cb 0x55a17ee47af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41092/0 (socket says 172.21.15.148:41092) 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 -- 172.21.15.148:0/3756711656 learned_addr learned my addr 172.21.15.148:0/3756711656 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa005d9b640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f8360 0x55a17f2186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa006d9d640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f1cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 -- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 msgr2=0x55a17f1cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f1cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 -- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f8360 msgr2=0x55a17f2186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f8360 0x55a17f2186b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.997+0000 7fa00659c640 1 -- 172.21.15.148:0/3756711656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a17ef74ea0 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00659c640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fd6f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00889d0 tx=0x7f9ff005f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7120001d9350dd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7f9fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7f9fe57fa640 1 -- 172.21.15.148:0/3756711656 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0070020 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7f9fe57fa640 1 -- 172.21.15.148:0/3756711656 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ff006cbe0 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa000044430 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7f9fe57fa640 1 -- 172.21.15.148:0/3756711656 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff0020070 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 10 client.6510 fetch_fsmap learned FSMap version 25 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a17ee08000 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7f9fe57fa640 1 -- 172.21.15.148:0/3756711656 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000044430 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7f9fe57fa640 1 -- 172.21.15.148:0/3756711656 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9ff0058120 con 0x55a17f1f7f80 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 10 client.6510 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 10 client.6510 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 20 client.6510 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1ee680 0x55a17f1eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 4 client.6510 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a17f141000 con 0x55a17f1ee680 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.998+0000 7fa005d9b640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1ee680 0x55a17f1eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7f9fe57fa640 10 client.6510 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7fa005d9b640 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1ee680 0x55a17f1eea60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c276ade2f16d97a server_cookie=2c91a337e89f8734 in_seq=0 out_seq=0 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7f9fe57fa640 1 -- 172.21.15.148:0/3756711656 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a17f141000 con 0x55a17f1ee680 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7f9fe57fa640 10 client.6510 handle_command_reply: tid=0 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7fa00f5e7740 1 client.6510 shutdown 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 msgr2=0x55a17f1fd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:42.999+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fd6f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00889d0 tx=0x7f9ff005f140 comp rx=0 tx=0).stop 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 shutdown_connections 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f1cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1ee680 0x55a17f1eea60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f8360 0x55a17f2186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/3756711656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fd6f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 >> 172.21.15.148:0/3756711656 conn(0x55a17f1cabe0 msgr2=0x55a17f1f7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7f9fe57fa640 0 client.6510 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:43.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 shutdown_connections 2024-04-25T09:57:43.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 1 -- 172.21.15.148:0/3756711656 wait complete. 2024-04-25T09:57:43.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.000+0000 7fa00f5e7740 20 client.6510 trim_cache size 0 max 16384 2024-04-25T09:57:43.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.012+0000 7fa00f5e7740 1 Processor -- start 2024-04-25T09:57:43.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.012+0000 7fa00f5e7740 1 -- start start 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f218f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f219460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 0x55a17f1fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080c7fb0 con 0x55a17f1f8360 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa008160820 con 0x55a17f1f7f80 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0080c79b0 con 0x55a17f1cb890 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 0x55a17f1fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 0x55a17f1fc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41098/0 (socket says 172.21.15.148:41098) 2024-04-25T09:57:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 -- 172.21.15.148:0/436573864 learned_addr learned my addr 172.21.15.148:0/436573864 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa005d9b640 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f219460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa00659c640 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f218f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 -- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 msgr2=0x55a17f218f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f218f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 -- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 msgr2=0x55a17f219460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.013+0000 7fa006d9d640 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f219460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa006d9d640 1 -- 172.21.15.148:0/436573864 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a17f1c9c30 con 0x55a17f1f8360 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa006d9d640 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 0x55a17f1fc670 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa00003d000 tx=0x7fa000030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c3c1d22ac528ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7f9fe77fe640 1 -- 172.21.15.148:0/436573864 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000003990 con 0x55a17f1f8360 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7f9fe77fe640 1 -- 172.21.15.148:0/436573864 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000003b30 con 0x55a17f1f8360 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7f9fe77fe640 1 -- 172.21.15.148:0/436573864 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000004040 con 0x55a17f1f8360 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 -- 172.21.15.148:0/436573864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 msgr2=0x55a17f1fc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 0x55a17f1fc670 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa00003d000 tx=0x7fa000030000 comp rx=0 tx=0).stop 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 -- 172.21.15.148:0/436573864 shutdown_connections 2024-04-25T09:57:43.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f8360 0x55a17f1fc670 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1f7f80 0x55a17f219460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/436573864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1cb890 0x55a17f218f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 -- 172.21.15.148:0/436573864 >> 172.21.15.148:0/436573864 conn(0x55a17f1cabe0 msgr2=0x55a17f1c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.014+0000 7fa00f5e7740 1 -- 172.21.15.148:0/436573864 shutdown_connections 2024-04-25T09:57:43.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.015+0000 7fa00f5e7740 1 -- 172.21.15.148:0/436573864 wait complete. 2024-04-25T09:57:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.015+0000 7fa00f5e7740 1 -- start start 2024-04-25T09:57:43.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.015+0000 7fa00f5e7740 1 Processor -- start 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1cb890 0x55a17f1fd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a17f1ef680 con 0x55a17f1f7f80 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a17f229910 con 0x55a17f1cb890 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a17f216d90 con 0x55a17f1f8360 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00f5e7740 10 client.0 _ll_register_callbacks cb 0x7fa000009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41102/0 (socket says 172.21.15.148:41102) 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 -- 172.21.15.148:0/2127930285 learned_addr learned my addr 172.21.15.148:0/2127930285 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa00659c640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1cb890 0x55a17f1fd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa006d9d640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 -- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 msgr2=0x55a17f1f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 -- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1cb890 msgr2=0x55a17f1fd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1cb890 0x55a17f1fd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.016+0000 7fa005d9b640 1 -- 172.21.15.148:0/2127930285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a17f2197a0 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa005d9b640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fda40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa0080ce3e0 tx=0x7f9ff8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7e2ea7385aa9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 1 -- 172.21.15.148:0/2127930285 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff802f8c0 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 1 -- 172.21.15.148:0/2127930285 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ff803bc40 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa000003b30 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 1 -- 172.21.15.148:0/2127930285 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff803be40 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 10 client.6516 fetch_fsmap learned FSMap version 25 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55a17f2194c0 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 1 -- 172.21.15.148:0/2127930285 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000003b30 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 1 -- 172.21.15.148:0/2127930285 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9ff8042050 con 0x55a17f1f7f80 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 10 client.6516 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T09:57:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 10 client.6516 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 20 client.6516 populate_metadata read hostname 'smithi148' 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1f2680 0x55a17f1fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 4 client.6516 mds_command: new command op to 5862 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3030538701}] 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 --> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3030538701}) v1 -- 0x55a17f13b350 con 0x55a17f1f2680 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7fa00659c640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1f2680 0x55a17f1fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.017+0000 7f9fe57fa640 10 client.6516 ms_handle_connect on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00659c640 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1f2680 0x55a17f1fb660 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269e54fa2f41292d server_cookie=374a504d02935945 in_seq=0 out_seq=0 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7f9fe57fa640 1 -- 172.21.15.148:0/2127930285 <== mds.0 v2:172.21.15.148:6835/3309890912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55a17f13b350 con 0x55a17f1f2680 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7f9fe57fa640 10 client.6516 handle_command_reply: tid=0 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 client.6516 shutdown 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 msgr2=0x55a17f1fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fda40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa0080ce3e0 tx=0x7f9ff8000bc0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 shutdown_connections 2024-04-25T09:57:43.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a17f1f8360 0x55a17f1f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a17f1f7f80 0x55a17f1fda40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.148:6835/3309890912,v1:172.21.15.148:6837/3309890912] conn(0x55a17f1f2680 0x55a17f1fb660 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 --2- 172.21.15.148:0/2127930285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a17f1cb890 0x55a17f1fd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7f9fe57fa640 0 client.6516 ms_handle_reset on v2:172.21.15.148:6835/3309890912 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 >> 172.21.15.148:0/2127930285 conn(0x55a17f1cabe0 msgr2=0x55a17f1d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 shutdown_connections 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 1 -- 172.21.15.148:0/2127930285 wait complete. 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00f5e7740 20 client.6516 trim_cache size 0 max 16384 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9fe806b4d0 msgr2=0x7f9fe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9fe806b4d0 0x7f9fe806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa00815dbe0 tx=0x7f9ff8009040 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa008070dc0 msgr2=0x7fa008159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.018+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa008070dc0 0x7fa008159a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000e930 tx=0x7f9ff000edf0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 shutdown_connections 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9fe806b4d0 0x7f9fe806d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa00815deb0 0x7fa0080cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0080711a0 0x7fa008155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 --2- 172.21.15.148:0/2995283414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa008070dc0 0x7fa008159a20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 >> 172.21.15.148:0/2995283414 conn(0x7fa0080b3530 msgr2=0x7fa0080cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 shutdown_connections 2024-04-25T09:57:43.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.019+0000 7fa00c8e1640 1 -- 172.21.15.148:0/2995283414 wait complete. 2024-04-25T09:57:43.033 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T09:57:43.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.126+0000 7fcc33363640 1 Processor -- start 2024-04-25T09:57:43.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.127+0000 7fcc33363640 1 -- start start 2024-04-25T09:57:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.127+0000 7fcc33363640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc2c15b9c0 0x7fcc2c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.127+0000 7fcc33363640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcc2c06abe0 0x7fcc2c06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.127+0000 7fcc33363640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2c073150 con 0x7fcc2c072620 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.127+0000 7fcc33363640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cfa60 con 0x7fcc2c15b9c0 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.127+0000 7fcc33363640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cfd40 con 0x7fcc2c06abe0 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc310ff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc2c15b9c0 0x7fcc2c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc31900640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcc2c072620 0x7fcc2c072a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37538/0 (socket says 172.21.15.148:37538) 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc31900640 1 -- 172.21.15.148:0/3015902838 learned_addr learned my addr 172.21.15.148:0/3015902838 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc308fe640 1 --2- 172.21.15.148:0/3015902838 >> v2:172.21.15.188:3300/0 conn(0x7fcc2c06abe0 0x7fcc2c06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc310ff640 1 -- 172.21.15.148:0/3015902838 >> v2:172.21.15.188:3300/0 conn(0x7fcc2c06abe0 msgr2=0x7fcc2c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc310ff640 1 --2- 172.21.15.148:0/3015902838 >> v2:172.21.15.188:3300/0 conn(0x7fcc2c06abe0 0x7fcc2c06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc310ff640 1 -- 172.21.15.148:0/3015902838 >> v1:172.21.15.148:6789/0 conn(0x7fcc2c072620 legacy=0x7fcc2c072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc23fff640 1 -- 172.21.15.148:0/3015902838 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416871504 0 0) 0x7fcc2c073150 con 0x7fcc2c072620 2024-04-25T09:57:43.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.128+0000 7fcc310ff640 1 -- 172.21.15.148:0/3015902838 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0d0020 con 0x7fcc2c15b9c0 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc310ff640 1 --2- 172.21.15.148:0/3015902838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c15b9c0 0x7fcc2c15bda0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0b5980 tx=0x7fcc1c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee705e296cae3b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc23fff640 1 -- 172.21.15.148:0/3015902838 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c037070 con 0x7fcc2c15b9c0 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc23fff640 1 -- 172.21.15.148:0/3015902838 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c032440 con 0x7fcc2c15b9c0 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc23fff640 1 -- 172.21.15.148:0/3015902838 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c032810 con 0x7fcc2c15b9c0 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 -- 172.21.15.148:0/3015902838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c15b9c0 msgr2=0x7fcc2c15bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 --2- 172.21.15.148:0/3015902838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c15b9c0 0x7fcc2c15bda0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0b5980 tx=0x7fcc1c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 -- 172.21.15.148:0/3015902838 shutdown_connections 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 --2- 172.21.15.148:0/3015902838 >> v2:172.21.15.188:3300/0 conn(0x7fcc2c06abe0 0x7fcc2c06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 --2- 172.21.15.148:0/3015902838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c15b9c0 0x7fcc2c15bda0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 -- 172.21.15.148:0/3015902838 >> 172.21.15.148:0/3015902838 conn(0x7fcc2c0b2370 msgr2=0x7fcc2c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 -- 172.21.15.148:0/3015902838 shutdown_connections 2024-04-25T09:57:43.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.129+0000 7fcc33363640 1 -- 172.21.15.148:0/3015902838 wait complete. 2024-04-25T09:57:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.130+0000 7fcc33363640 1 Processor -- start 2024-04-25T09:57:43.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 -- start start 2024-04-25T09:57:43.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc2c06abe0 0x7fcc2c1f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 0x7fcc2c151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc2c15b9c0 0x7fcc2c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c15c2e0 con 0x7fcc2c072620 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0cf710 con 0x7fcc2c15b9c0 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc33363640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0d0820 con 0x7fcc2c06abe0 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc308fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 0x7fcc2c151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc308fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 0x7fcc2c151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41122/0 (socket says 172.21.15.148:41122) 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.131+0000 7fcc308fe640 1 -- 172.21.15.148:0/672747298 learned_addr learned my addr 172.21.15.148:0/672747298 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc308fe640 1 -- 172.21.15.148:0/672747298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc2c06abe0 msgr2=0x7fcc2c1f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc308fe640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc2c06abe0 0x7fcc2c1f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc308fe640 1 -- 172.21.15.148:0/672747298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc2c15b9c0 msgr2=0x7fcc2c15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc308fe640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc2c15b9c0 0x7fcc2c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc308fe640 1 -- 172.21.15.148:0/672747298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c151de0 con 0x7fcc2c072620 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc308fe640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 0x7fcc2c151360 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800d800 tx=0x7fcc2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2800b6c0 con 0x7fcc2c072620 2024-04-25T09:57:43.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc28019070 con 0x7fcc2c072620 2024-04-25T09:57:43.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c1520c0 con 0x7fcc2c072620 2024-04-25T09:57:43.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2800ba80 con 0x7fcc2c072620 2024-04-25T09:57:43.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.132+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c152680 con 0x7fcc2c072620 2024-04-25T09:57:43.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.133+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c073990 con 0x7fcc2c072620 2024-04-25T09:57:43.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.134+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc2800bc20 con 0x7fcc2c072620 2024-04-25T09:57:43.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.134+0000 7fcc21ffb640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc1006b500 0x7fcc1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.134+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc2808a7f0 con 0x7fcc2c072620 2024-04-25T09:57:43.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.134+0000 7fcc310ff640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc1006b500 0x7fcc1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.135+0000 7fcc310ff640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc1006b500 0x7fcc1006d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c034000 tx=0x7fcc1c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.138+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc2801c360 con 0x7fcc2c072620 2024-04-25T09:57:43.216 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:57:43.213+0000 7fb15b777640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T09:57:43.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.342+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc2c155980 con 0x7fcc2c072620 2024-04-25T09:57:43.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.343+0000 7fcc21ffb640 1 -- 172.21.15.148:0/672747298 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7fcc280594e0 con 0x7fcc2c072620 2024-04-25T09:57:43.346 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:43.347 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:43.019532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:43.347 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 26 2024-04-25T09:57:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc1006b500 msgr2=0x7fcc1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc1006b500 0x7fcc1006d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c034000 tx=0x7fcc1c02c000 comp rx=0 tx=0).stop 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 msgr2=0x7fcc2c151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 0x7fcc2c151360 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800d800 tx=0x7fcc2800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 shutdown_connections 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc1006b500 0x7fcc1006d9c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc2c15b9c0 0x7fcc2c15d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc2c072620 0x7fcc2c151360 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 --2- 172.21.15.148:0/672747298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc2c06abe0 0x7fcc2c1f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 >> 172.21.15.148:0/672747298 conn(0x7fcc2c0b2370 msgr2=0x7fcc2c0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 shutdown_connections 2024-04-25T09:57:43.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.345+0000 7fcc33363640 1 -- 172.21.15.148:0/672747298 wait complete. 2024-04-25T09:57:43.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T09:57:43.462 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:43.463 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T09:57:43.463 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:43.471 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T09:57:43.472 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:43.472 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:43.510 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:43.556 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:43.557 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:43.557 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:43.572 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:43.572 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:43.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:43.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.761+0000 7f44ce6b3640 1 Processor -- start 2024-04-25T09:57:43.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.761+0000 7f44ce6b3640 1 -- start start 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.761+0000 7f44ce6b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44c815dc00 0x7f44c80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44ce6b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c815dfe0 con 0x7f44c80ceb20 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44ce6b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c80d00c0 con 0x7f44c80d1dd0 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44ce6b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c80d03a0 con 0x7f44c815dc00 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f44c80ceb20 0x7f44c80d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35602/0 (socket says 172.21.15.148:35602) 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c77fe640 1 -- 172.21.15.148:0/4046245902 learned_addr learned my addr 172.21.15.148:0/4046245902 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130387651 0 0) 0x7f44bc001110 con 0x7f44c80ceb20 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b0003520 con 0x7f44c80ceb20 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c7fff640 1 --2- 172.21.15.148:0/4046245902 >> v2:172.21.15.188:3300/0 conn(0x7f44c815dc00 0x7f44c80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735618093 0 0) 0x7f44b0003520 con 0x7f44c80ceb20 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 >> v2:172.21.15.188:3300/0 conn(0x7f44c815dc00 msgr2=0x7f44c80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 --2- 172.21.15.148:0/4046245902 >> v2:172.21.15.188:3300/0 conn(0x7f44c815dc00 0x7f44c80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 >> v1:172.21.15.188:6789/0 conn(0x7f44c80d1dd0 legacy=0x7f44c80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c80cc440 con 0x7f44c80ceb20 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f44bc001840 con 0x7f44c80ceb20 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 >> v1:172.21.15.148:6789/0 conn(0x7f44c80ceb20 legacy=0x7f44c80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b00039b0 0x7f44b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b0004800 0x7f44b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80d03a0 con 0x7f44b00051c0 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000c060 con 0x7f44b0004800 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000c260 con 0x7f44b00039b0 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44ccc50640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.762+0000 7f44c7fff640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b00039b0 0x7f44b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44c77fe640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b0004800 0x7f44b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 -- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b00039b0 msgr2=0x7f44b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44b00039b0 0x7f44b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 -- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b0004800 msgr2=0x7f44b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44b0004800 0x7f44b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 -- 172.21.15.148:0/4046245902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0003520 con 0x7f44b00051c0 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 -- 172.21.15.148:0/4046245902 reap_dead start 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.763+0000 7f44ccc50640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44b000b9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44c80d21e0 tx=0x7f44b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d06013ad691e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0001a70 con 0x7f44b00051c0 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c002ae10 con 0x7f44b00051c0 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44c6ffd640 1 -- 172.21.15.148:0/4046245902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0003bb0 con 0x7f44b00051c0 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 -- 172.21.15.148:0/4046245902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 msgr2=0x7f44b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44b000b9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44c80d21e0 tx=0x7f44b0005830 comp rx=0 tx=0).stop 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 -- 172.21.15.148:0/4046245902 shutdown_connections 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/4046245902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44b000b9c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 -- 172.21.15.148:0/4046245902 >> 172.21.15.148:0/4046245902 conn(0x7f44c806f5d0 msgr2=0x7f44c806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 -- 172.21.15.148:0/4046245902 shutdown_connections 2024-04-25T09:57:43.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.764+0000 7f44ce6b3640 1 -- 172.21.15.148:0/4046245902 wait complete. 2024-04-25T09:57:43.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.765+0000 7f44ce6b3640 1 Processor -- start 2024-04-25T09:57:43.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.765+0000 7f44ce6b3640 1 -- start start 2024-04-25T09:57:43.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44ce6b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44c8144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44ce6b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c80b6ec0 0x7f44c8141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44ce6b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c80b44a0 0x7f44c80b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44ce6b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80cfd90 con 0x7f44b00051c0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44ce6b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80cc130 con 0x7f44c80b6ec0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44ce6b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80cdd70 con 0x7f44c80b44a0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44c8144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44c8144220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55342/0 (socket says 172.21.15.148:55342) 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 -- 172.21.15.148:0/893783272 learned_addr learned my addr 172.21.15.148:0/893783272 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 -- 172.21.15.148:0/893783272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c80b44a0 msgr2=0x7f44c80b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c80b44a0 0x7f44c80b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 -- 172.21.15.148:0/893783272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c80b6ec0 msgr2=0x7f44c8141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c80b6ec0 0x7f44c8141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.766+0000 7f44c7fff640 1 -- 172.21.15.148:0/893783272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c80c99e0 con 0x7f44b00051c0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.767+0000 7f44c7fff640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44c8144220 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f44b8011910 tx=0x7f44b8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.767+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b800fa90 con 0x7f44b00051c0 2024-04-25T09:57:43.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.767+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b800fc30 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.767+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c8141ea0 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.767+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8014440 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.767+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c8144ae0 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.768+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c8073990 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.768+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44b8012040 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.769+0000 7f44c4ff9640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44b406b4d0 0x7f44b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.769+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44b808a8a0 con 0x7f44b00051c0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.769+0000 7f44c77fe640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44b406b4d0 0x7f44b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:43.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.769+0000 7f44c77fe640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44b406b4d0 0x7f44b406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0012b0 tx=0x7f44bc000ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:43.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.773+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44b8019030 con 0x7f44b00051c0 2024-04-25T09:57:43.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.944+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44c80c66c0 con 0x7f44b00051c0 2024-04-25T09:57:43.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.945+0000 7f44c4ff9640 1 -- 172.21.15.148:0/893783272 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f44b8059620 con 0x7f44b00051c0 2024-04-25T09:57:43.949 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:43.949 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:11.985711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:43.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44b406b4d0 msgr2=0x7f44b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44b406b4d0 0x7f44b406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0012b0 tx=0x7f44bc000ed0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 msgr2=0x7f44c8144220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:43.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44c8144220 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f44b8011910 tx=0x7f44b8011dd0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 shutdown_connections 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44b406b4d0 0x7f44b406d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c80b44a0 0x7f44c80b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c80b6ec0 0x7f44c8141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.947+0000 7f44ce6b3640 1 --2- 172.21.15.148:0/893783272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44b00051c0 0x7f44c8144220 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.948+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 >> 172.21.15.148:0/893783272 conn(0x7f44c806f5d0 msgr2=0x7f44c80b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.948+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 shutdown_connections 2024-04-25T09:57:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:43.948+0000 7f44ce6b3640 1 -- 172.21.15.148:0/893783272 wait complete. 2024-04-25T09:57:43.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:08.515642+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:09.289387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:43.963 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:44.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.060+0000 7f171fb6a640 1 Processor -- start 2024-04-25T09:57:44.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.060+0000 7f171fb6a640 1 -- start start 2024-04-25T09:57:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171fb6a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1718070dc0 0x7f171806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171fb6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f171815deb0 0x7f171815e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171fb6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17180d1770 con 0x7f17180711a0 2024-04-25T09:57:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171fb6a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17180b0eb0 con 0x7f1718070dc0 2024-04-25T09:57:44.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171fb6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17181622b0 con 0x7f171815deb0 2024-04-25T09:57:44.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d906640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1718070dc0 0x7f171806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d105640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f17180711a0 0x7f17180b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35612/0 (socket says 172.21.15.148:35612) 2024-04-25T09:57:44.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d105640 1 -- 172.21.15.148:0/3800863171 learned_addr learned my addr 172.21.15.148:0/3800863171 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:44.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171e107640 1 --2- 172.21.15.148:0/3800863171 >> v2:172.21.15.188:3300/0 conn(0x7f171815deb0 0x7f171815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d906640 1 -- 172.21.15.148:0/3800863171 >> v2:172.21.15.188:3300/0 conn(0x7f171815deb0 msgr2=0x7f171815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:44.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d906640 1 --2- 172.21.15.148:0/3800863171 >> v2:172.21.15.188:3300/0 conn(0x7f171815deb0 0x7f171815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d906640 1 -- 172.21.15.148:0/3800863171 >> v1:172.21.15.148:6789/0 conn(0x7f17180711a0 legacy=0x7f17180b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171d906640 1 -- 172.21.15.148:0/3800863171 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718162740 con 0x7f1718070dc0 2024-04-25T09:57:44.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.061+0000 7f171c904640 1 -- 172.21.15.148:0/3800863171 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301564526 0 0) 0x7f17180d1770 con 0x7f17180711a0 2024-04-25T09:57:44.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.062+0000 7f171d906640 1 --2- 172.21.15.148:0/3800863171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f171806ccd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1718070570 tx=0x7f171402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aea6c08dd473e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.062+0000 7f171c904640 1 -- 172.21.15.148:0/3800863171 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714037070 con 0x7f1718070dc0 2024-04-25T09:57:44.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.062+0000 7f171c904640 1 -- 172.21.15.148:0/3800863171 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171402add0 con 0x7f1718070dc0 2024-04-25T09:57:44.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.062+0000 7f171c904640 1 -- 172.21.15.148:0/3800863171 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714032660 con 0x7f1718070dc0 2024-04-25T09:57:44.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 -- 172.21.15.148:0/3800863171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 msgr2=0x7f171806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 --2- 172.21.15.148:0/3800863171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f171806ccd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1718070570 tx=0x7f171402a080 comp rx=0 tx=0).stop 2024-04-25T09:57:44.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 -- 172.21.15.148:0/3800863171 shutdown_connections 2024-04-25T09:57:44.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 --2- 172.21.15.148:0/3800863171 >> v2:172.21.15.188:3300/0 conn(0x7f171815deb0 0x7f171815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 --2- 172.21.15.148:0/3800863171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f171806ccd0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 -- 172.21.15.148:0/3800863171 >> 172.21.15.148:0/3800863171 conn(0x7f17180b3530 msgr2=0x7f17180b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 -- 172.21.15.148:0/3800863171 shutdown_connections 2024-04-25T09:57:44.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.063+0000 7f171fb6a640 1 -- 172.21.15.148:0/3800863171 wait complete. 2024-04-25T09:57:44.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.064+0000 7f171fb6a640 1 Processor -- start 2024-04-25T09:57:44.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.064+0000 7f171fb6a640 1 -- start start 2024-04-25T09:57:44.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f17180cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17180711a0 0x7f171814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171815deb0 0x7f17180b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718161c60 con 0x7f1718070dc0 2024-04-25T09:57:44.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17180b10f0 con 0x7f17180711a0 2024-04-25T09:57:44.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1718161f40 con 0x7f171815deb0 2024-04-25T09:57:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f17180cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f17180cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55354/0 (socket says 172.21.15.148:55354) 2024-04-25T09:57:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 -- 172.21.15.148:0/1403280530 learned_addr learned my addr 172.21.15.148:0/1403280530 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171e107640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171815deb0 0x7f17180b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d105640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17180711a0 0x7f171814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 -- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171815deb0 msgr2=0x7f17180b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171815deb0 0x7f17180b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 -- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17180711a0 msgr2=0x7f171814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17180711a0 0x7f171814f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 -- 172.21.15.148:0/1403280530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180cb0c0 con 0x7f1718070dc0 2024-04-25T09:57:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171d906640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f17180cc160 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1714000c00 tx=0x7f17140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714037040 con 0x7f1718070dc0 2024-04-25T09:57:44.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714042070 con 0x7f1718070dc0 2024-04-25T09:57:44.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17180cb3a0 con 0x7f1718070dc0 2024-04-25T09:57:44.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171403b890 con 0x7f1718070dc0 2024-04-25T09:57:44.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.065+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17180cb960 con 0x7f1718070dc0 2024-04-25T09:57:44.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.066+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1718073a00 con 0x7f1718070dc0 2024-04-25T09:57:44.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.067+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f171402ac50 con 0x7f1718070dc0 2024-04-25T09:57:44.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.067+0000 7f170e7fc640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16fc06b4d0 0x7f16fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.067+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17140aa9d0 con 0x7f1718070dc0 2024-04-25T09:57:44.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.067+0000 7f171d105640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16fc06b4d0 0x7f16fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.068+0000 7f171d105640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16fc06b4d0 0x7f16fc06d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17181f6fa0 tx=0x7f1708007b90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.072+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1714030e20 con 0x7f1718070dc0 2024-04-25T09:57:44.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.239+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17180caea0 con 0x7f1718070dc0 2024-04-25T09:57:44.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.239+0000 7f170e7fc640 1 -- 172.21.15.148:0/1403280530 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17140796c0 con 0x7f1718070dc0 2024-04-25T09:57:44.246 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16fc06b4d0 msgr2=0x7f16fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16fc06b4d0 0x7f16fc06d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f17181f6fa0 tx=0x7f1708007b90 comp rx=0 tx=0).stop 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 msgr2=0x7f17180cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f17180cc160 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1714000c00 tx=0x7f17140046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 shutdown_connections 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16fc06b4d0 0x7f16fc06d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f171815deb0 0x7f17180b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17180711a0 0x7f171814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 --2- 172.21.15.148:0/1403280530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1718070dc0 0x7f17180cc160 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 >> 172.21.15.148:0/1403280530 conn(0x7f17180b3530 msgr2=0x7f17180b5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 shutdown_connections 2024-04-25T09:57:44.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.241+0000 7f171fb6a640 1 -- 172.21.15.148:0/1403280530 wait complete. 2024-04-25T09:57:44.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:44.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.353+0000 7f1fc9522640 1 Processor -- start 2024-04-25T09:57:44.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.353+0000 7f1fc9522640 1 -- start start 2024-04-25T09:57:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc9522640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1fc415dc20 0x7f1fc40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc9522640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1fc4161710 0x7f1fc40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc9522640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc415e000 con 0x7f1fc40b9330 2024-04-25T09:57:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc9522640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc40d1de0 con 0x7f1fc415dc20 2024-04-25T09:57:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc9522640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc4161c60 con 0x7f1fc4161710 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1fc415dc20 0x7f1fc40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1fc415dc20 0x7f1fc40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55364/0 (socket says 172.21.15.148:55364) 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/727146479 learned_addr learned my addr 172.21.15.148:0/727146479 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fbbfff640 1 --2- 172.21.15.148:0/727146479 >> v2:172.21.15.188:3300/0 conn(0x7f1fc4161710 0x7f1fc40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/727146479 >> v2:172.21.15.188:3300/0 conn(0x7f1fc4161710 msgr2=0x7f1fc40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 --2- 172.21.15.148:0/727146479 >> v2:172.21.15.188:3300/0 conn(0x7f1fc4161710 0x7f1fc40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/727146479 >> v1:172.21.15.188:6789/0 conn(0x7f1fc40b9330 legacy=0x7f1fc406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/727146479 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4161f40 con 0x7f1fc415dc20 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.354+0000 7f1fbbfff640 1 --2- 172.21.15.148:0/727146479 >> v2:172.21.15.188:3300/0 conn(0x7f1fc4161710 0x7f1fc40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.355+0000 7f1fc2ffd640 1 --2- 172.21.15.148:0/727146479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc415dc20 0x7f1fc40ce670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1fc406da90 tx=0x7f1fb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb1982ba0369cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.355+0000 7f1fc27fc640 1 -- 172.21.15.148:0/727146479 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4037070 con 0x7f1fc415dc20 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.355+0000 7f1fc27fc640 1 -- 172.21.15.148:0/727146479 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4032440 con 0x7f1fc415dc20 2024-04-25T09:57:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.355+0000 7f1fc27fc640 1 -- 172.21.15.148:0/727146479 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4032810 con 0x7f1fc415dc20 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 -- 172.21.15.148:0/727146479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc415dc20 msgr2=0x7f1fc40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 --2- 172.21.15.148:0/727146479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc415dc20 0x7f1fc40ce670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1fc406da90 tx=0x7f1fb402a270 comp rx=0 tx=0).stop 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 -- 172.21.15.148:0/727146479 shutdown_connections 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 --2- 172.21.15.148:0/727146479 >> v2:172.21.15.188:3300/0 conn(0x7f1fc4161710 0x7f1fc40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 --2- 172.21.15.148:0/727146479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc415dc20 0x7f1fc40ce670 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 -- 172.21.15.148:0/727146479 >> 172.21.15.148:0/727146479 conn(0x7f1fc40d0420 msgr2=0x7f1fc40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 -- 172.21.15.148:0/727146479 shutdown_connections 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.356+0000 7f1fc9522640 1 -- 172.21.15.148:0/727146479 wait complete. 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 Processor -- start 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 -- start start 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 0x7f1fc41f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc415dc20 0x7f1fc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc4161710 0x7f1fc41f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40b5e50 con 0x7f1fc40b9330 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4162500 con 0x7f1fc415dc20 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.357+0000 7f1fc9522640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc40b5fd0 con 0x7f1fc4161710 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 0x7f1fc41f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 0x7f1fc41f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55370/0 (socket says 172.21.15.148:55370) 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/2081565470 learned_addr learned my addr 172.21.15.148:0/2081565470 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fbbfff640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc415dc20 0x7f1fc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc37fe640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc4161710 0x7f1fc41f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc4161710 msgr2=0x7f1fc41f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc4161710 0x7f1fc41f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc415dc20 msgr2=0x7f1fc40b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc415dc20 0x7f1fc40b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 -- 172.21.15.148:0/2081565470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc41f4bb0 con 0x7f1fc40b9330 2024-04-25T09:57:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fbbfff640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc415dc20 0x7f1fc40b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc2ffd640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 0x7f1fc41f4670 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1fb402a2b0 tx=0x7f1fb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4041020 con 0x7f1fc40b9330 2024-04-25T09:57:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb40306c0 con 0x7f1fc40b9330 2024-04-25T09:57:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4037040 con 0x7f1fc40b9330 2024-04-25T09:57:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc40aa720 con 0x7f1fc40b9330 2024-04-25T09:57:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.358+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc41f5130 con 0x7f1fc40b9330 2024-04-25T09:57:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.359+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc40ce340 con 0x7f1fc40b9330 2024-04-25T09:57:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.360+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fb4044020 con 0x7f1fc40b9330 2024-04-25T09:57:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.360+0000 7f1fbb7fe640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fa406b4d0 0x7f1fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.360+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fb40aaba0 con 0x7f1fc40b9330 2024-04-25T09:57:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.360+0000 7f1fbbfff640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fa406b4d0 0x7f1fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.360+0000 7f1fbbfff640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fa406b4d0 0x7f1fa406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40cf480 tx=0x7f1fac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.364+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fc40ce340 con 0x7f1fc40b9330 2024-04-25T09:57:44.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.533+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1fc40cfc20 con 0x7f1fc40b9330 2024-04-25T09:57:44.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.533+0000 7f1fbb7fe640 1 -- 172.21.15.148:0/2081565470 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1fc40cfc20 con 0x7f1fc40b9330 2024-04-25T09:57:44.537 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:57:44.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fa406b4d0 msgr2=0x7f1fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fa406b4d0 0x7f1fa406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40cf480 tx=0x7f1fac008040 comp rx=0 tx=0).stop 2024-04-25T09:57:44.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 msgr2=0x7f1fc41f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 0x7f1fc41f4670 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1fb402a2b0 tx=0x7f1fb40046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 shutdown_connections 2024-04-25T09:57:44.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fa406b4d0 0x7f1fa406d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc4161710 0x7f1fc41f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc415dc20 0x7f1fc40b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 --2- 172.21.15.148:0/2081565470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc40b9330 0x7f1fc41f4670 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 >> 172.21.15.148:0/2081565470 conn(0x7f1fc40d0420 msgr2=0x7f1fc40b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 shutdown_connections 2024-04-25T09:57:44.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.535+0000 7f1fc9522640 1 -- 172.21.15.148:0/2081565470 wait complete. 2024-04-25T09:57:44.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.648+0000 7fcc7cd90640 1 Processor -- start 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.648+0000 7fcc7cd90640 1 -- start start 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc7cd90640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc780ced80 0x7fcc780d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc7cd90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcc78161710 0x7fcc780d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc7cd90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7815e000 con 0x7fcc7815dc20 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc7cd90640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc780d1de0 con 0x7fcc780ced80 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc7cd90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc78161c60 con 0x7fcc78161710 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc75d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc780ced80 0x7fcc780d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc75d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc780ced80 0x7fcc780d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55372/0 (socket says 172.21.15.148:55372) 2024-04-25T09:57:44.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.649+0000 7fcc75d9b640 1 -- 172.21.15.148:0/2699287446 learned_addr learned my addr 172.21.15.148:0/2699287446 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.650+0000 7fcc76d9d640 1 --2- 172.21.15.148:0/2699287446 >> v2:172.21.15.188:3300/0 conn(0x7fcc78161710 0x7fcc780d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.650+0000 7fcc75d9b640 1 -- 172.21.15.148:0/2699287446 >> v2:172.21.15.188:3300/0 conn(0x7fcc78161710 msgr2=0x7fcc780d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.650+0000 7fcc75d9b640 1 --2- 172.21.15.148:0/2699287446 >> v2:172.21.15.188:3300/0 conn(0x7fcc78161710 0x7fcc780d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.650+0000 7fcc75d9b640 1 -- 172.21.15.148:0/2699287446 >> v1:172.21.15.188:6789/0 conn(0x7fcc7815dc20 legacy=0x7fcc780ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.650+0000 7fcc75d9b640 1 -- 172.21.15.148:0/2699287446 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78161f40 con 0x7fcc780ced80 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.650+0000 7fcc75d9b640 1 --2- 172.21.15.148:0/2699287446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc780ced80 0x7fcc780d22e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcc780b6570 tx=0x7fcc6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b66b2533929719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7559a640 1 -- 172.21.15.148:0/2699287446 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c037070 con 0x7fcc780ced80 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7559a640 1 -- 172.21.15.148:0/2699287446 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c032440 con 0x7fcc780ced80 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7559a640 1 -- 172.21.15.148:0/2699287446 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c032810 con 0x7fcc780ced80 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 -- 172.21.15.148:0/2699287446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc780ced80 msgr2=0x7fcc780d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/2699287446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc780ced80 0x7fcc780d22e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcc780b6570 tx=0x7fcc6c02a270 comp rx=0 tx=0).stop 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 -- 172.21.15.148:0/2699287446 shutdown_connections 2024-04-25T09:57:44.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/2699287446 >> v2:172.21.15.188:3300/0 conn(0x7fcc78161710 0x7fcc780d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/2699287446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc780ced80 0x7fcc780d22e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 -- 172.21.15.148:0/2699287446 >> 172.21.15.148:0/2699287446 conn(0x7fcc780d0420 msgr2=0x7fcc780d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 -- 172.21.15.148:0/2699287446 shutdown_connections 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.651+0000 7fcc7cd90640 1 -- 172.21.15.148:0/2699287446 wait complete. 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.652+0000 7fcc7cd90640 1 Processor -- start 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 -- start start 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc780ced80 0x7fcc78148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 0x7fcc780c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc78161710 0x7fcc781587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc780b5e50 con 0x7fcc7815dc20 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78162500 con 0x7fcc780ced80 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7cd90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc780b5fd0 con 0x7fcc78161710 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc75d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 0x7fcc780c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc75d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 0x7fcc780c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55376/0 (socket says 172.21.15.148:55376) 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc75d9b640 1 -- 172.21.15.148:0/1714135242 learned_addr learned my addr 172.21.15.148:0/1714135242 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc76d9d640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc78161710 0x7fcc781587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.653+0000 7fcc7659c640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc780ced80 0x7fcc78148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc75d9b640 1 -- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc78161710 msgr2=0x7fcc781587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc75d9b640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc78161710 0x7fcc781587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc75d9b640 1 -- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc780ced80 msgr2=0x7fcc78148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc75d9b640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc780ced80 0x7fcc78148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc75d9b640 1 -- 172.21.15.148:0/1714135242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7815a650 con 0x7fcc7815dc20 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc7659c640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc780ced80 0x7fcc78148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc76d9d640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc78161710 0x7fcc781587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc75d9b640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 0x7fcc780c7300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c000c00 tx=0x7fcc6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c041020 con 0x7fcc7815dc20 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0306c0 con 0x7fcc7815dc20 2024-04-25T09:57:44.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c037040 con 0x7fcc7815dc20 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7815a930 con 0x7fcc7815dc20 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.654+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7815aef0 con 0x7fcc7815dc20 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.655+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc78073a00 con 0x7fcc7815dc20 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.655+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc6c044020 con 0x7fcc7815dc20 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.656+0000 7fcc66ffd640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.656+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc6c0aaba0 con 0x7fcc7815dc20 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.656+0000 7fcc7659c640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.656+0000 7fcc7659c640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001d60 tx=0x7fcc68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.660+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc6c04b030 con 0x7fcc7815dc20 2024-04-25T09:57:44.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.830+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc780d16b0 con 0x7fcc7815dc20 2024-04-25T09:57:44.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.831+0000 7fcc66ffd640 1 -- 172.21.15.148:0/1714135242 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc6c079880 con 0x7fcc7815dc20 2024-04-25T09:57:44.834 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 msgr2=0x7fcc5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcc68001d60 tx=0x7fcc68009040 comp rx=0 tx=0).stop 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 msgr2=0x7fcc780c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 0x7fcc780c7300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c000c00 tx=0x7fcc6c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 shutdown_connections 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc78161710 0x7fcc781587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc7815dc20 0x7fcc780c7300 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 --2- 172.21.15.148:0/1714135242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc780ced80 0x7fcc78148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 >> 172.21.15.148:0/1714135242 conn(0x7fcc780d0420 msgr2=0x7fcc780b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 shutdown_connections 2024-04-25T09:57:44.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.833+0000 7fcc7cd90640 1 -- 172.21.15.148:0/1714135242 wait complete. 2024-04-25T09:57:44.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:57:44.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.941+0000 7f1c0945a640 1 Processor -- start 2024-04-25T09:57:44.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.941+0000 7f1c0945a640 1 -- start start 2024-04-25T09:57:44.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c0945a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161240 0x7f1c040b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c0945a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c04161890 0x7f1c040ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c0945a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c04161620 con 0x7f1c040b54a0 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c0945a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040b7120 con 0x7f1c04161890 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c0945a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04161e00 con 0x7f1c04161240 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c027fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1c04161890 0x7f1c040ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c037fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1c040b54a0 0x7f1c040d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35614/0 (socket says 172.21.15.148:35614) 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.942+0000 7f1c037fe640 1 -- 172.21.15.148:0/2990671703 learned_addr learned my addr 172.21.15.148:0/2990671703 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c02ffd640 1 --2- 172.21.15.148:0/2990671703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161240 0x7f1c040b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c027fc640 1 -- 172.21.15.148:0/2990671703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161240 msgr2=0x7f1c040b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c027fc640 1 --2- 172.21.15.148:0/2990671703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161240 0x7f1c040b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c027fc640 1 -- 172.21.15.148:0/2990671703 >> v1:172.21.15.148:6789/0 conn(0x7f1c040b54a0 legacy=0x7f1c040d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c027fc640 1 -- 172.21.15.148:0/2990671703 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c041620e0 con 0x7f1c04161890 2024-04-25T09:57:44.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c027fc640 1 --2- 172.21.15.148:0/2990671703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161890 0x7f1c040ceac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097a0 tx=0x7f1bf802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a250b810398b2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.943+0000 7f1c01ffb640 1 -- 172.21.15.148:0/2990671703 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8037070 con 0x7f1c04161890 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c01ffb640 1 -- 172.21.15.148:0/2990671703 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8032440 con 0x7f1c04161890 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c01ffb640 1 -- 172.21.15.148:0/2990671703 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8032810 con 0x7f1c04161890 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 -- 172.21.15.148:0/2990671703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161890 msgr2=0x7f1c040ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 --2- 172.21.15.148:0/2990671703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161890 0x7f1c040ceac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097a0 tx=0x7f1bf802a180 comp rx=0 tx=0).stop 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 -- 172.21.15.148:0/2990671703 shutdown_connections 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 --2- 172.21.15.148:0/2990671703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161890 0x7f1c040ceac0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 --2- 172.21.15.148:0/2990671703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161240 0x7f1c040b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 -- 172.21.15.148:0/2990671703 >> 172.21.15.148:0/2990671703 conn(0x7f1c040d04a0 msgr2=0x7f1c040d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 -- 172.21.15.148:0/2990671703 shutdown_connections 2024-04-25T09:57:44.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.944+0000 7f1c0945a640 1 -- 172.21.15.148:0/2990671703 wait complete. 2024-04-25T09:57:44.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.945+0000 7f1c0945a640 1 Processor -- start 2024-04-25T09:57:44.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.945+0000 7f1c0945a640 1 -- start start 2024-04-25T09:57:44.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c0945a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c040b54a0 0x7f1c04150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c0945a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 0x7f1c041f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c0945a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161890 0x7f1c0415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c0945a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040d15f0 con 0x7f1c04161240 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c0945a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041626a0 con 0x7f1c040b54a0 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c0945a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040cf000 con 0x7f1c04161890 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c027fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 0x7f1c041f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c027fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 0x7f1c041f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55390/0 (socket says 172.21.15.148:55390) 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.946+0000 7f1c027fc640 1 -- 172.21.15.148:0/3559711539 learned_addr learned my addr 172.21.15.148:0/3559711539 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c027fc640 1 -- 172.21.15.148:0/3559711539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161890 msgr2=0x7f1c0415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c027fc640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161890 0x7f1c0415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c027fc640 1 -- 172.21.15.148:0/3559711539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c040b54a0 msgr2=0x7f1c04150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c027fc640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c040b54a0 0x7f1c04150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:44.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c027fc640 1 -- 172.21.15.148:0/3559711539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040c8c40 con 0x7f1c04161240 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c027fc640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 0x7f1c041f3d30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80098c0 tx=0x7f1bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8041070 con 0x7f1c04161240 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf803c930 con 0x7f1c04161240 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c041f4270 con 0x7f1c04161240 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8037040 con 0x7f1c04161240 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.947+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c040c8eb0 con 0x7f1c04161240 2024-04-25T09:57:44.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.948+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c04073a00 con 0x7f1c04161240 2024-04-25T09:57:44.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.948+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bf8044070 con 0x7f1c04161240 2024-04-25T09:57:44.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.949+0000 7f1beb7fe640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1bf006b4d0 0x7f1bf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:44.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.949+0000 7f1c02ffd640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1bf006b4d0 0x7f1bf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:44.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.949+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1bf80aaff0 con 0x7f1c04161240 2024-04-25T09:57:44.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.949+0000 7f1c02ffd640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1bf006b4d0 0x7f1bf006d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40070b0 tx=0x7f1bf4007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:44.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:44.953+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bf8079e90 con 0x7f1c04161240 2024-04-25T09:57:45.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.138+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1c040b5170 con 0x7f1c04161240 2024-04-25T09:57:45.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.138+0000 7f1beb7fe640 1 -- 172.21.15.148:0/3559711539 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1bf8079790 con 0x7f1c04161240 2024-04-25T09:57:45.142 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1bf006b4d0 msgr2=0x7f1bf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1bf006b4d0 0x7f1bf006d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40070b0 tx=0x7f1bf4007040 comp rx=0 tx=0).stop 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 msgr2=0x7f1c041f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 0x7f1c041f3d30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80098c0 tx=0x7f1bf80046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 shutdown_connections 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1bf006b4d0 0x7f1bf006d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c04161890 0x7f1c0415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c04161240 0x7f1c041f3d30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 --2- 172.21.15.148:0/3559711539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c040b54a0 0x7f1c04150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 >> 172.21.15.148:0/3559711539 conn(0x7f1c040d04a0 msgr2=0x7f1c040d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:45.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 shutdown_connections 2024-04-25T09:57:45.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:45.140+0000 7f1c0945a640 1 -- 172.21.15.148:0/3559711539 wait complete. 2024-04-25T09:57:45.154 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:57:45.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:45.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:57:46.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:57:46.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:57:46.516 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:57:46.531 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:57:46.531 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:46.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:46.549 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:46.549 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:46.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:46.587 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:57:46.587 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:46.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:46.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.720+0000 7f0926874640 1 Processor -- start 2024-04-25T09:57:46.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.720+0000 7f0926874640 1 -- start start 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0926874640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0920154ed0 0x7f09200aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0926874640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f092006c1d0 0x7f092006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0926874640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092015ec60 con 0x7f0920071ac0 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0926874640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092015ef40 con 0x7f0920154ed0 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0926874640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092006db90 con 0x7f092006c1d0 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0920154ed0 0x7f09200aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0920071ac0 0x7f09200ad410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35624/0 (socket says 172.21.15.148:35624) 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091f7fe640 1 -- 172.21.15.148:0/679523336 learned_addr learned my addr 172.21.15.148:0/679523336 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:46.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0924e11640 1 --2- 172.21.15.148:0/679523336 >> v2:172.21.15.188:3300/0 conn(0x7f092006c1d0 0x7f092006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091ffff640 1 -- 172.21.15.148:0/679523336 >> v2:172.21.15.188:3300/0 conn(0x7f092006c1d0 msgr2=0x7f092006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091ffff640 1 --2- 172.21.15.148:0/679523336 >> v2:172.21.15.188:3300/0 conn(0x7f092006c1d0 0x7f092006afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091ffff640 1 -- 172.21.15.148:0/679523336 >> v1:172.21.15.148:6789/0 conn(0x7f0920071ac0 legacy=0x7f09200ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091ffff640 1 -- 172.21.15.148:0/679523336 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092006de70 con 0x7f0920154ed0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f091effd640 1 -- 172.21.15.148:0/679523336 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261895867 0 0) 0x7f092015ec60 con 0x7f0920071ac0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.721+0000 7f0924e11640 1 --2- 172.21.15.148:0/679523336 >> v2:172.21.15.188:3300/0 conn(0x7f092006c1d0 0x7f092006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f091ffff640 1 --2- 172.21.15.148:0/679523336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0920154ed0 0x7f09200aced0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f09200cc3d0 tx=0x7f091002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d833e7d50eafac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f091effd640 1 -- 172.21.15.148:0/679523336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910037070 con 0x7f0920154ed0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f091effd640 1 -- 172.21.15.148:0/679523336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0910032440 con 0x7f0920154ed0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f091effd640 1 -- 172.21.15.148:0/679523336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091002ae60 con 0x7f0920154ed0 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f0926874640 1 -- 172.21.15.148:0/679523336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0920154ed0 msgr2=0x7f09200aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f0926874640 1 --2- 172.21.15.148:0/679523336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0920154ed0 0x7f09200aced0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f09200cc3d0 tx=0x7f091002a110 comp rx=0 tx=0).stop 2024-04-25T09:57:46.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f0926874640 1 -- 172.21.15.148:0/679523336 shutdown_connections 2024-04-25T09:57:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f0926874640 1 --2- 172.21.15.148:0/679523336 >> v2:172.21.15.188:3300/0 conn(0x7f092006c1d0 0x7f092006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f0926874640 1 --2- 172.21.15.148:0/679523336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0920154ed0 0x7f09200aced0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.722+0000 7f0926874640 1 -- 172.21.15.148:0/679523336 >> 172.21.15.148:0/679523336 conn(0x7f09200af670 msgr2=0x7f09200b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.723+0000 7f0926874640 1 -- 172.21.15.148:0/679523336 shutdown_connections 2024-04-25T09:57:46.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.723+0000 7f0926874640 1 -- 172.21.15.148:0/679523336 wait complete. 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 Processor -- start 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 -- start start 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0920071ac0 0x7f092014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0920154ed0 0x7f09200ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 0x7f09200b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092006b660 con 0x7f09200cd4a0 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.724+0000 7f0926874640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092006b4e0 con 0x7f0920071ac0 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0926874640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092006b7e0 con 0x7f0920154ed0 2024-04-25T09:57:46.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 0x7f09200b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 0x7f09200b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55414/0 (socket says 172.21.15.148:55414) 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 -- 172.21.15.148:0/3133816899 learned_addr learned my addr 172.21.15.148:0/3133816899 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f091f7fe640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0920154ed0 0x7f09200ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f091ffff640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0920071ac0 0x7f092014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 -- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0920154ed0 msgr2=0x7f09200ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0920154ed0 0x7f09200ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 -- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0920071ac0 msgr2=0x7f092014e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0920071ac0 0x7f092014e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 -- 172.21.15.148:0/3133816899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092014eed0 con 0x7f09200cd4a0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f091f7fe640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0920154ed0 0x7f09200ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0924e11640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 0x7f09200b0ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f091800ce70 tx=0x7f091800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091801a070 con 0x7f09200cd4a0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091800d840 con 0x7f09200cd4a0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09200b16c0 con 0x7f09200cd4a0 2024-04-25T09:57:46.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800dc30 con 0x7f09200cd4a0 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.725+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0920156770 con 0x7f09200cd4a0 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.726+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09200b2b10 con 0x7f09200cd4a0 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.727+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f091801d070 con 0x7f09200cd4a0 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.727+0000 7f091cff9640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f090c06b4d0 0x7f090c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.727+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f091808ab30 con 0x7f09200cd4a0 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.727+0000 7f091ffff640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f090c06b4d0 0x7f090c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:46.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.728+0000 7f091ffff640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f090c06b4d0 0x7f090c06d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0910034000 tx=0x7f091002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:46.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.732+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f092014eed0 con 0x7f09200cd4a0 2024-04-25T09:57:46.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.939+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0920073990 con 0x7f09200cd4a0 2024-04-25T09:57:46.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.939+0000 7f091cff9640 1 -- 172.21.15.148:0/3133816899 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f09180598b0 con 0x7f09200cd4a0 2024-04-25T09:57:46.944 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:46.944 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:43.019532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:46.944 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 26 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f090c06b4d0 msgr2=0x7f090c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f090c06b4d0 0x7f090c06d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0910034000 tx=0x7f091002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 msgr2=0x7f09200b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 0x7f09200b0ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f091800ce70 tx=0x7f091800a710 comp rx=0 tx=0).stop 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 shutdown_connections 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f090c06b4d0 0x7f090c06d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09200cd4a0 0x7f09200b0ae0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0920154ed0 0x7f09200ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 --2- 172.21.15.148:0/3133816899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0920071ac0 0x7f092014e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:46.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 >> 172.21.15.148:0/3133816899 conn(0x7f092006c1d0 msgr2=0x7f09200aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:46.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 shutdown_connections 2024-04-25T09:57:46.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:46.941+0000 7f0926874640 1 -- 172.21.15.148:0/3133816899 wait complete. 2024-04-25T09:57:46.956 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:47.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.051+0000 7f7536190640 1 Processor -- start 2024-04-25T09:57:47.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.051+0000 7f7536190640 1 -- start start 2024-04-25T09:57:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f7536190640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7530154ed0 0x7f75300aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f7536190640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753015ec60 con 0x7f7530071ac0 2024-04-25T09:57:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f7536190640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753015ef40 con 0x7f753006c1d0 2024-04-25T09:57:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f7536190640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753006db90 con 0x7f7530154ed0 2024-04-25T09:57:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7530154ed0 0x7f75300aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7530071ac0 0x7f75300ad410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35634/0 (socket says 172.21.15.148:35634) 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752effd640 1 -- 172.21.15.148:0/3594424394 learned_addr learned my addr 172.21.15.148:0/3594424394 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752f7fe640 1 -- 172.21.15.148:0/3594424394 >> v1:172.21.15.188:6789/0 conn(0x7f753006c1d0 legacy=0x7f753006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752f7fe640 1 -- 172.21.15.148:0/3594424394 >> v1:172.21.15.148:6789/0 conn(0x7f7530071ac0 legacy=0x7f75300ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752e7fc640 1 -- 172.21.15.148:0/3594424394 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39927338 0 0) 0x7f753015ec60 con 0x7f7530071ac0 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.052+0000 7f752f7fe640 1 -- 172.21.15.148:0/3594424394 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753006de70 con 0x7f7530154ed0 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f752f7fe640 1 --2- 172.21.15.148:0/3594424394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75300aced0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f75200097a0 tx=0x7f752002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfdc9776977c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f752e7fc640 1 -- 172.21.15.148:0/3594424394 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520037070 con 0x7f7530154ed0 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f752e7fc640 1 -- 172.21.15.148:0/3594424394 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752002ae50 con 0x7f7530154ed0 2024-04-25T09:57:47.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f752e7fc640 1 -- 172.21.15.148:0/3594424394 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520032660 con 0x7f7530154ed0 2024-04-25T09:57:47.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f7536190640 1 -- 172.21.15.148:0/3594424394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 msgr2=0x7f75300aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f7536190640 1 --2- 172.21.15.148:0/3594424394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75300aced0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f75200097a0 tx=0x7f752002a140 comp rx=0 tx=0).stop 2024-04-25T09:57:47.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f7536190640 1 -- 172.21.15.148:0/3594424394 shutdown_connections 2024-04-25T09:57:47.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f7536190640 1 --2- 172.21.15.148:0/3594424394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75300aced0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.053+0000 7f7536190640 1 -- 172.21.15.148:0/3594424394 >> 172.21.15.148:0/3594424394 conn(0x7f75300af670 msgr2=0x7f75300b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.054+0000 7f7536190640 1 -- 172.21.15.148:0/3594424394 shutdown_connections 2024-04-25T09:57:47.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.054+0000 7f7536190640 1 -- 172.21.15.148:0/3594424394 wait complete. 2024-04-25T09:57:47.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 Processor -- start 2024-04-25T09:57:47.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 -- start start 2024-04-25T09:57:47.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7530071ac0 0x7f75300ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75301f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75301f77c0 0x7f75301f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753006b830 con 0x7f7530154ed0 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753006b6b0 con 0x7f75301f77c0 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.055+0000 7f7536190640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753006b9b0 con 0x7f7530071ac0 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75301f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75301f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55436/0 (socket says 172.21.15.148:55436) 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 -- 172.21.15.148:0/1326824644 learned_addr learned my addr 172.21.15.148:0/1326824644 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752f7fe640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7530071ac0 0x7f75300ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 -- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7530071ac0 msgr2=0x7f75300ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752ffff640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75301f77c0 0x7f75301f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7530071ac0 0x7f75300ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 -- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75301f77c0 msgr2=0x7f75301f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75301f77c0 0x7f75301f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 -- 172.21.15.148:0/1326824644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300b06c0 con 0x7f7530154ed0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f752effd640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75301f7280 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f752400c4b0 tx=0x7f752400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524018070 con 0x7f7530154ed0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7524007500 con 0x7f7530154ed0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75300b09a0 con 0x7f7530154ed0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524004a20 con 0x7f7530154ed0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.056+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75300b0f60 con 0x7f7530154ed0 2024-04-25T09:57:47.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.057+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75300b3c90 con 0x7f7530154ed0 2024-04-25T09:57:47.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.058+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f752401b070 con 0x7f7530154ed0 2024-04-25T09:57:47.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.058+0000 7f7517fff640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751c06b4d0 0x7f751c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.058+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75240896f0 con 0x7f7530154ed0 2024-04-25T09:57:47.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.058+0000 7f752f7fe640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751c06b4d0 0x7f751c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.058+0000 7f752f7fe640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751c06b4d0 0x7f751c06d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7520034000 tx=0x7f752002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.062+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75240584a0 con 0x7f7530154ed0 2024-04-25T09:57:47.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.235+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f753014feb0 con 0x7f7530154ed0 2024-04-25T09:57:47.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.236+0000 7f7517fff640 1 -- 172.21.15.148:0/1326824644 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f7524057da0 con 0x7f7530154ed0 2024-04-25T09:57:47.241 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:47.241 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:11.985711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751c06b4d0 msgr2=0x7f751c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751c06b4d0 0x7f751c06d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7520034000 tx=0x7f752002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 msgr2=0x7f75301f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75301f7280 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f752400c4b0 tx=0x7f752400c970 comp rx=0 tx=0).stop 2024-04-25T09:57:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 shutdown_connections 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751c06b4d0 0x7f751c06d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75301f77c0 0x7f75301f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7530154ed0 0x7f75301f7280 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 --2- 172.21.15.148:0/1326824644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7530071ac0 0x7f75300ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 >> 172.21.15.148:0/1326824644 conn(0x7f753006c1d0 msgr2=0x7f75300b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 shutdown_connections 2024-04-25T09:57:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.238+0000 7f7536190640 1 -- 172.21.15.148:0/1326824644 wait complete. 2024-04-25T09:57:47.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:57:08.515642+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:57:09.289387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:47.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:47.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.350+0000 7f67b04f5640 1 Processor -- start 2024-04-25T09:57:47.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.350+0000 7f67b04f5640 1 -- start start 2024-04-25T09:57:47.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67b04f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67a80ce900 0x7f67a8166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67b04f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a80ced00 con 0x7f67a806fbd0 2024-04-25T09:57:47.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67b04f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a80b4d80 con 0x7f67a8155f80 2024-04-25T09:57:47.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67b04f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a80b5390 con 0x7f67a80ce900 2024-04-25T09:57:47.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67ada90640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f67a806fbd0 0x7f67a80ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35638/0 (socket says 172.21.15.148:35638) 2024-04-25T09:57:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67ada90640 1 -- 172.21.15.148:0/2839096724 learned_addr learned my addr 172.21.15.148:0/2839096724 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.351+0000 7f67aea92640 1 --2- 172.21.15.148:0/2839096724 >> v2:172.21.15.148:3300/0 conn(0x7f67a80ce900 0x7f67a8166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67aea92640 1 -- 172.21.15.148:0/2839096724 >> v1:172.21.15.188:6789/0 conn(0x7f67a8155f80 legacy=0x7f67a806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67aea92640 1 -- 172.21.15.148:0/2839096724 >> v1:172.21.15.148:6789/0 conn(0x7f67a806fbd0 legacy=0x7f67a80ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67aea92640 1 -- 172.21.15.148:0/2839096724 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a80cef70 con 0x7f67a80ce900 2024-04-25T09:57:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67ad28f640 1 -- 172.21.15.148:0/2839096724 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488825725 0 0) 0x7f67a80ced00 con 0x7f67a806fbd0 2024-04-25T09:57:47.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67aea92640 1 --2- 172.21.15.148:0/2839096724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a80ce900 0x7f67a8166870 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f67a80b5510 tx=0x7f67a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69100d969599a504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67ad28f640 1 -- 172.21.15.148:0/2839096724 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0037070 con 0x7f67a80ce900 2024-04-25T09:57:47.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67ad28f640 1 -- 172.21.15.148:0/2839096724 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0032440 con 0x7f67a80ce900 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.352+0000 7f67ad28f640 1 -- 172.21.15.148:0/2839096724 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0032810 con 0x7f67a80ce900 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 -- 172.21.15.148:0/2839096724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a80ce900 msgr2=0x7f67a8166870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2839096724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a80ce900 0x7f67a8166870 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f67a80b5510 tx=0x7f67a002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 -- 172.21.15.148:0/2839096724 shutdown_connections 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2839096724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a80ce900 0x7f67a8166870 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 -- 172.21.15.148:0/2839096724 >> 172.21.15.148:0/2839096724 conn(0x7f67a8061600 msgr2=0x7f67a80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 -- 172.21.15.148:0/2839096724 shutdown_connections 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.353+0000 7f67b04f5640 1 -- 172.21.15.148:0/2839096724 wait complete. 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.354+0000 7f67b04f5640 1 Processor -- start 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.354+0000 7f67b04f5640 1 -- start start 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67b04f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 0x7f67a81f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67b04f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a80ce900 0x7f67a81f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67b04f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a8155f80 0x7f67a814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67b04f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8156360 con 0x7f67a806fbd0 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67b04f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80d07f0 con 0x7f67a8155f80 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67b04f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80cf250 con 0x7f67a80ce900 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 0x7f67a81f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 0x7f67a81f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55450/0 (socket says 172.21.15.148:55450) 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 -- 172.21.15.148:0/2941349685 learned_addr learned my addr 172.21.15.148:0/2941349685 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ada90640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a80ce900 0x7f67a81f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67aea92640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a8155f80 0x7f67a814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 -- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a80ce900 msgr2=0x7f67a81f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a80ce900 0x7f67a81f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 -- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a8155f80 msgr2=0x7f67a814a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a8155f80 0x7f67a814a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ae291640 1 -- 172.21.15.148:0/2941349685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a81f7c80 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67ada90640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a80ce900 0x7f67a81f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.355+0000 7f67aea92640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a8155f80 0x7f67a814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.356+0000 7f67ae291640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 0x7f67a81f7550 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f67a400ba50 tx=0x7f67a400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.356+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4019070 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.356+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a400cc20 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.356+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a81f7f60 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.356+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4014680 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.356+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a814bbb0 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.357+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a8073a00 con 0x7f67a806fbd0 2024-04-25T09:57:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.357+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67a401c070 con 0x7f67a806fbd0 2024-04-25T09:57:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.358+0000 7f679effd640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f678c06b4d0 0x7f678c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.358+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67a408a0a0 con 0x7f67a806fbd0 2024-04-25T09:57:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.358+0000 7f67ada90640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f678c06b4d0 0x7f678c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.358+0000 7f67ada90640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f678c06b4d0 0x7f678c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67a814dc50 tx=0x7f6798009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.362+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67a4058740 con 0x7f67a806fbd0 2024-04-25T09:57:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.531+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67a80b1030 con 0x7f67a806fbd0 2024-04-25T09:57:47.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.531+0000 7f679effd640 1 -- 172.21.15.148:0/2941349685 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67a405c7e0 con 0x7f67a806fbd0 2024-04-25T09:57:47.536 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:57:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f678c06b4d0 msgr2=0x7f678c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f678c06b4d0 0x7f678c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f67a814dc50 tx=0x7f6798009040 comp rx=0 tx=0).stop 2024-04-25T09:57:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 msgr2=0x7f67a81f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 0x7f67a81f7550 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f67a400ba50 tx=0x7f67a400bf10 comp rx=0 tx=0).stop 2024-04-25T09:57:47.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 shutdown_connections 2024-04-25T09:57:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f678c06b4d0 0x7f678c06d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a8155f80 0x7f67a814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67a80ce900 0x7f67a81f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 --2- 172.21.15.148:0/2941349685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a806fbd0 0x7f67a81f7550 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 >> 172.21.15.148:0/2941349685 conn(0x7f67a8061600 msgr2=0x7f67a80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 shutdown_connections 2024-04-25T09:57:47.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.533+0000 7f67b04f5640 1 -- 172.21.15.148:0/2941349685 wait complete. 2024-04-25T09:57:47.549 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:57:47.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.645+0000 7f942319e640 1 Processor -- start 2024-04-25T09:57:47.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.645+0000 7f942319e640 1 -- start start 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.645+0000 7f942319e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f941c0ca5a0 0x7f941c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.645+0000 7f942319e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c0cc2e0 con 0x7f941c06c300 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.645+0000 7f942319e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c0cc5c0 con 0x7f941c15aa10 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.645+0000 7f942319e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c0cfa70 con 0x7f941c0ca5a0 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f942173b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f941c06c300 0x7f941c0acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35654/0 (socket says 172.21.15.148:35654) 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f942173b640 1 -- 172.21.15.148:0/2323341242 learned_addr learned my addr 172.21.15.148:0/2323341242 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941bfff640 1 --2- 172.21.15.148:0/2323341242 >> v2:172.21.15.188:3300/0 conn(0x7f941c0ca5a0 0x7f941c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036928592 0 0) 0x7f941c0cc2e0 con 0x7f941c06c300 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9404003420 con 0x7f941c06c300 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2410176530 0 0) 0x7f9404003420 con 0x7f941c06c300 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 >> v2:172.21.15.188:3300/0 conn(0x7f941c0ca5a0 msgr2=0x7f941c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 --2- 172.21.15.148:0/2323341242 >> v2:172.21.15.188:3300/0 conn(0x7f941c0ca5a0 0x7f941c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 >> v1:172.21.15.188:6789/0 conn(0x7f941c15aa10 legacy=0x7f941c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0cf420 con 0x7f941c06c300 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f94140019f0 con 0x7f941c06c300 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 >> v1:172.21.15.148:6789/0 conn(0x7f941c06c300 legacy=0x7f941c0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f9404004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94040036a0 0x7f9404003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f940400db60 0x7f940400ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0cfa70 con 0x7f9404004a90 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9404010540 con 0x7f94040036a0 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9404010740 con 0x7f940400db60 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f9420f3a640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f9404004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f941bfff640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94040036a0 0x7f9404003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.646+0000 7f942173b640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f940400db60 0x7f940400ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f9420f3a640 1 -- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f940400db60 msgr2=0x7f940400ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f9420f3a640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f940400db60 0x7f940400ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f9420f3a640 1 -- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94040036a0 msgr2=0x7f9404003b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f9420f3a640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94040036a0 0x7f9404003b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f9420f3a640 1 -- 172.21.15.148:0/2323341242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404003420 con 0x7f9404004a90 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f942173b640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f940400db60 0x7f940400ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f942173b640 1 -- 172.21.15.148:0/2323341242 reap_dead start 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f9420f3a640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f9404004ed0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f940c001750 tx=0x7f9404004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc8794e0808b8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c001370 con 0x7f9404004a90 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c0052d0 con 0x7f9404004a90 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.647+0000 7f941b7fe640 1 -- 172.21.15.148:0/2323341242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c02abc0 con 0x7f9404004a90 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 -- 172.21.15.148:0/2323341242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 msgr2=0x7f9404004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f9404004ed0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f940c001750 tx=0x7f9404004720 comp rx=0 tx=0).stop 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 -- 172.21.15.148:0/2323341242 shutdown_connections 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 --2- 172.21.15.148:0/2323341242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f9404004ed0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 -- 172.21.15.148:0/2323341242 >> 172.21.15.148:0/2323341242 conn(0x7f941c071af0 msgr2=0x7f941c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 -- 172.21.15.148:0/2323341242 shutdown_connections 2024-04-25T09:57:47.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.648+0000 7f942319e640 1 -- 172.21.15.148:0/2323341242 wait complete. 2024-04-25T09:57:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.649+0000 7f942319e640 1 Processor -- start 2024-04-25T09:57:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.649+0000 7f942319e640 1 -- start start 2024-04-25T09:57:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942319e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f941c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942319e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f941c1efa50 0x7f941c09ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942319e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f941c09b280 0x7f941c0c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942319e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0cbff0 con 0x7f9404004a90 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942319e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0cbe70 con 0x7f941c09b280 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942319e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941c0c6830 con 0x7f941c1efa50 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f941c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f941c14faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55456/0 (socket says 172.21.15.148:55456) 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 -- 172.21.15.148:0/3094913523 learned_addr learned my addr 172.21.15.148:0/3094913523 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f942173b640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f941c09b280 0x7f941c0c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f941bfff640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f941c1efa50 0x7f941c09ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 -- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f941c1efa50 msgr2=0x7f941c09ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f941c1efa50 0x7f941c09ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 -- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f941c09b280 msgr2=0x7f941c0c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f941c09b280 0x7f941c0c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.650+0000 7f9420f3a640 1 -- 172.21.15.148:0/3094913523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c150030 con 0x7f9404004a90 2024-04-25T09:57:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.651+0000 7f9420f3a640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f941c14faf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f940c02a570 tx=0x7f940c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.651+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c03f030 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.651+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c005480 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.651+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c0c70c0 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.651+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c037040 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.651+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c0c7360 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.652+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c073a00 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.652+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f940c042070 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.653+0000 7f94197fa640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940806b4d0 0x7f940806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.653+0000 7f941bfff640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940806b4d0 0x7f940806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.653+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f940c0a98e0 con 0x7f9404004a90 2024-04-25T09:57:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.653+0000 7f941bfff640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940806b4d0 0x7f940806d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f941c0cb3c0 tx=0x7f9410008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.657+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f941c150030 con 0x7f9404004a90 2024-04-25T09:57:47.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.828+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f941c0b2b90 con 0x7f9404004a90 2024-04-25T09:57:47.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.828+0000 7f94197fa640 1 -- 172.21.15.148:0/3094913523 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f940c078540 con 0x7f9404004a90 2024-04-25T09:57:47.833 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:57:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940806b4d0 msgr2=0x7f940806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940806b4d0 0x7f940806d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f941c0cb3c0 tx=0x7f9410008040 comp rx=0 tx=0).stop 2024-04-25T09:57:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 msgr2=0x7f941c14faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f941c14faf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f940c02a570 tx=0x7f940c001120 comp rx=0 tx=0).stop 2024-04-25T09:57:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 shutdown_connections 2024-04-25T09:57:47.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940806b4d0 0x7f940806d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f941c09b280 0x7f941c0c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f941c1efa50 0x7f941c09ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 --2- 172.21.15.148:0/3094913523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9404004a90 0x7f941c14faf0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 >> 172.21.15.148:0/3094913523 conn(0x7f941c071af0 msgr2=0x7f941c06fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 shutdown_connections 2024-04-25T09:57:47.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.830+0000 7f942319e640 1 -- 172.21.15.148:0/3094913523 wait complete. 2024-04-25T09:57:47.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:57:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.940+0000 7f269193b640 1 Processor -- start 2024-04-25T09:57:47.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.940+0000 7f269193b640 1 -- start start 2024-04-25T09:57:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f269193b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f268c15bb50 0x7f268c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f269193b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0c70d0 0x7f268c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f269193b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f268c0cfb40 0x7f268c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f269193b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0c8680 con 0x7f268c15bb50 2024-04-25T09:57:47.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f269193b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0c75e0 con 0x7f268c0cfb40 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f269193b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0c78c0 con 0x7f268c0c70d0 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f268affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f268c15bb50 0x7f268c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f268affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f268c15bb50 0x7f268c0c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55462/0 (socket says 172.21.15.148:55462) 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f268affd640 1 -- 172.21.15.148:0/3915527317 learned_addr learned my addr 172.21.15.148:0/3915527317 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.941+0000 7f268a7fc640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0c70d0 0x7f268c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268affd640 1 -- 172.21.15.148:0/3915527317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0c70d0 msgr2=0x7f268c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268b7fe640 1 --2- 172.21.15.148:0/3915527317 >> v2:172.21.15.188:3300/0 conn(0x7f268c0cfb40 0x7f268c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268affd640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0c70d0 0x7f268c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268affd640 1 -- 172.21.15.148:0/3915527317 >> v2:172.21.15.188:3300/0 conn(0x7f268c0cfb40 msgr2=0x7f268c070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268affd640 1 --2- 172.21.15.148:0/3915527317 >> v2:172.21.15.188:3300/0 conn(0x7f268c0cfb40 0x7f268c070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268affd640 1 -- 172.21.15.148:0/3915527317 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0c7ba0 con 0x7f268c15bb50 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268a7fc640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0c70d0 0x7f268c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f268affd640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c0c6b90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f268c0b33e0 tx=0x7f267c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3b755ae23ed00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f2689ffb640 1 -- 172.21.15.148:0/3915527317 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c036070 con 0x7f268c15bb50 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f2689ffb640 1 -- 172.21.15.148:0/3915527317 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c031600 con 0x7f268c15bb50 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.942+0000 7f2689ffb640 1 -- 172.21.15.148:0/3915527317 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0319d0 con 0x7f268c15bb50 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 -- 172.21.15.148:0/3915527317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 msgr2=0x7f268c0c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c0c6b90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f268c0b33e0 tx=0x7f267c02a730 comp rx=0 tx=0).stop 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 -- 172.21.15.148:0/3915527317 shutdown_connections 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 --2- 172.21.15.148:0/3915527317 >> v2:172.21.15.188:3300/0 conn(0x7f268c0cfb40 0x7f268c070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0c70d0 0x7f268c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 --2- 172.21.15.148:0/3915527317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c0c6b90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 -- 172.21.15.148:0/3915527317 >> 172.21.15.148:0/3915527317 conn(0x7f268c06f500 msgr2=0x7f268c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 -- 172.21.15.148:0/3915527317 shutdown_connections 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.943+0000 7f269193b640 1 -- 172.21.15.148:0/3915527317 wait complete. 2024-04-25T09:57:47.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.944+0000 7f269193b640 1 Processor -- start 2024-04-25T09:57:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.944+0000 7f269193b640 1 -- start start 2024-04-25T09:57:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f269193b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268c0c70d0 0x7f268c157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f269193b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0cfb40 0x7f268c142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f269193b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f269193b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c15f6d0 con 0x7f268c15bb50 2024-04-25T09:57:47.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f269193b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0c8160 con 0x7f268c0c70d0 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f269193b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c15f850 con 0x7f268c0cfb40 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c142640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55476/0 (socket says 172.21.15.148:55476) 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.945+0000 7f268b7fe640 1 -- 172.21.15.148:0/3347720905 learned_addr learned my addr 172.21.15.148:0/3347720905 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268a7fc640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0cfb40 0x7f268c142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268b7fe640 1 -- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0cfb40 msgr2=0x7f268c142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268b7fe640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0cfb40 0x7f268c142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268affd640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268c0c70d0 0x7f268c157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268b7fe640 1 -- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268c0c70d0 msgr2=0x7f268c157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268b7fe640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268c0c70d0 0x7f268c157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268b7fe640 1 -- 172.21.15.148:0/3347720905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0ca3f0 con 0x7f268c15bb50 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f268b7fe640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c142640 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f268400c7f0 tx=0x7f268400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684015070 con 0x7f268c15bb50 2024-04-25T09:57:47.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684004220 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c0ca6d0 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684007620 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.946+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c0cac90 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.947+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c073a00 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.947+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26840079d0 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.948+0000 7f2673fff640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.948+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f268408a3e0 con 0x7f268c15bb50 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.948+0000 7f268affd640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:47.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.948+0000 7f268affd640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f267806b4d0 0x7f267806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f267c033000 tx=0x7f267c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:47.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:47.952+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26840590d0 con 0x7f268c15bb50 2024-04-25T09:57:48.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.121+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f268c0b0e20 con 0x7f268c15bb50 2024-04-25T09:57:48.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.121+0000 7f2673fff640 1 -- 172.21.15.148:0/3347720905 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26840589d0 con 0x7f268c15bb50 2024-04-25T09:57:48.126 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:57:48.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f267806b4d0 msgr2=0x7f267806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f267806b4d0 0x7f267806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f267c033000 tx=0x7f267c009000 comp rx=0 tx=0).stop 2024-04-25T09:57:48.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 msgr2=0x7f268c142640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c142640 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f268400c7f0 tx=0x7f268400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 shutdown_connections 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f267806b4d0 0x7f267806d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f268c15bb50 0x7f268c142640 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f268c0cfb40 0x7f268c142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 --2- 172.21.15.148:0/3347720905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268c0c70d0 0x7f268c157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.123+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 >> 172.21.15.148:0/3347720905 conn(0x7f268c06f500 msgr2=0x7f268c070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.124+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 shutdown_connections 2024-04-25T09:57:48.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.124+0000 7f269193b640 1 -- 172.21.15.148:0/3347720905 wait complete. 2024-04-25T09:57:48.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:48.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.232+0000 7ff4f78ee640 1 Processor -- start 2024-04-25T09:57:48.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.232+0000 7ff4f78ee640 1 -- start start 2024-04-25T09:57:48.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f78ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4f00b5c30 0x7ff4f0168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f78ee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f00ceee0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f78ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f00d1d10 con 0x7ff4f015db70 2024-04-25T09:57:48.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f78ee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f00d3800 con 0x7ff4f00b5c30 2024-04-25T09:57:48.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f5e8b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff4f00b5c30 0x7ff4f0168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f4e89640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff4f00b6cb0 0x7ff4f00b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35660/0 (socket says 172.21.15.148:35660) 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f4e89640 1 -- 172.21.15.148:0/2781231910 learned_addr learned my addr 172.21.15.148:0/2781231910 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f5e8b640 1 -- 172.21.15.148:0/2781231910 >> v1:172.21.15.188:6789/0 conn(0x7ff4f015db70 legacy=0x7ff4f0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f5e8b640 1 -- 172.21.15.148:0/2781231910 >> v1:172.21.15.148:6789/0 conn(0x7ff4f00b6cb0 legacy=0x7ff4f00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4f5e8b640 1 -- 172.21.15.148:0/2781231910 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00d1ff0 con 0x7ff4f00b5c30 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.233+0000 7ff4e3fff640 1 -- 172.21.15.148:0/2781231910 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380256858 0 0) 0x7ff4f00ceee0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4f5e8b640 1 --2- 172.21.15.148:0/2781231910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b5c30 0x7ff4f0168ad0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00d3980 tx=0x7ff4e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea043b3ccf8cd72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4e3fff640 1 -- 172.21.15.148:0/2781231910 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8007740 con 0x7ff4f00b5c30 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4e3fff640 1 -- 172.21.15.148:0/2781231910 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e80078e0 con 0x7ff4f00b5c30 2024-04-25T09:57:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4e3fff640 1 -- 172.21.15.148:0/2781231910 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8007cb0 con 0x7ff4f00b5c30 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4f78ee640 1 -- 172.21.15.148:0/2781231910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b5c30 msgr2=0x7ff4f0168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/2781231910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b5c30 0x7ff4f0168ad0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00d3980 tx=0x7ff4e802a6f0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4f78ee640 1 -- 172.21.15.148:0/2781231910 shutdown_connections 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/2781231910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b5c30 0x7ff4f0168ad0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.234+0000 7ff4f78ee640 1 -- 172.21.15.148:0/2781231910 >> 172.21.15.148:0/2781231910 conn(0x7ff4f00d0520 msgr2=0x7ff4f00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.235+0000 7ff4f78ee640 1 -- 172.21.15.148:0/2781231910 shutdown_connections 2024-04-25T09:57:48.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.235+0000 7ff4f78ee640 1 -- 172.21.15.148:0/2781231910 wait complete. 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 Processor -- start 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 -- start start 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4f00b5c30 0x7ff4f00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 0x7ff4f015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4f015db70 0x7ff4f0158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00ce530 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f015df50 con 0x7ff4f015db70 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.236+0000 7ff4f78ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00cf1c0 con 0x7ff4f00b5c30 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 0x7ff4f015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 0x7ff4f015e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55486/0 (socket says 172.21.15.148:55486) 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 -- 172.21.15.148:0/258938906 learned_addr learned my addr 172.21.15.148:0/258938906 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:48.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f568a640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4f00b5c30 0x7ff4f00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f5e8b640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4f015db70 0x7ff4f0158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 -- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4f00b5c30 msgr2=0x7ff4f00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4f00b5c30 0x7ff4f00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 -- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4f015db70 msgr2=0x7ff4f0158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4f015db70 0x7ff4f0158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 -- 172.21.15.148:0/258938906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f01572d0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f568a640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4f00b5c30 0x7ff4f00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f4e89640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 0x7ff4f015e160 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c490 tx=0x7ff4ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec018070 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec0074e0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f00cd2c0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec004a20 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.237+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f00cd880 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.238+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f006d100 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.239+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4ec01b070 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.239+0000 7ff4e1ffb640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff4d406b4d0 0x7ff4d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.239+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4ec0895e0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.239+0000 7ff4f568a640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff4d406b4d0 0x7ff4d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.240+0000 7ff4f568a640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff4d406b4d0 0x7ff4d406d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4007fd0 tx=0x7ff4e4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.243+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4f006d100 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.449+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4f00d1820 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.449+0000 7ff4e1ffb640 1 -- 172.21.15.148:0/258938906 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4951 (secure 0 0 0) 0x7ff4ec0582d0 con 0x7ff4f00b6cb0 2024-04-25T09:57:48.454 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:48.454 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:47.019261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:48.454 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 27 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff4d406b4d0 msgr2=0x7ff4d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff4d406b4d0 0x7ff4d406d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4007fd0 tx=0x7ff4e4009040 comp rx=0 tx=0).stop 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 msgr2=0x7ff4f015e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 0x7ff4f015e160 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c490 tx=0x7ff4ec00c950 comp rx=0 tx=0).stop 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 shutdown_connections 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff4d406b4d0 0x7ff4d406d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4f015db70 0x7ff4f0158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f00b6cb0 0x7ff4f015e160 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 --2- 172.21.15.148:0/258938906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4f00b5c30 0x7ff4f00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 >> 172.21.15.148:0/258938906 conn(0x7ff4f00d0520 msgr2=0x7ff4f00b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:48.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 shutdown_connections 2024-04-25T09:57:48.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.451+0000 7ff4f78ee640 1 -- 172.21.15.148:0/258938906 wait complete. 2024-04-25T09:57:48.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:48.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.558+0000 7f21df89e640 1 Processor -- start 2024-04-25T09:57:48.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.559+0000 7f21df89e640 1 -- start start 2024-04-25T09:57:48.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.559+0000 7f21df89e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 0x7f21d806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.559+0000 7f21df89e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8161960 con 0x7f21d806b580 2024-04-25T09:57:48.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.559+0000 7f21df89e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8161c40 con 0x7f21d80d0170 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.559+0000 7f21df89e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cbd50 con 0x7f21d806b1a0 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dce39640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f21d806b580 0x7f21d80cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35666/0 (socket says 172.21.15.148:35666) 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dce39640 1 -- 172.21.15.148:0/2528880753 learned_addr learned my addr 172.21.15.148:0/2528880753 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309689164 0 0) 0x7f21d8161960 con 0x7f21d806b580 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c4003520 con 0x7f21d806b580 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dd63a640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 0x7f21d806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325422582 0 0) 0x7f21c4003520 con 0x7f21d806b580 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 msgr2=0x7f21d806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 0x7f21d806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 >> v1:172.21.15.188:6789/0 conn(0x7f21d80d0170 legacy=0x7f21d815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8161f20 con 0x7f21d806b580 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f21d4001670 con 0x7f21d806b580 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 >> v1:172.21.15.148:6789/0 conn(0x7f21d806b580 legacy=0x7f21d80cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c40039e0 0x7f21c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 0x7f21c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21c4004d40 0x7f21c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cbd50 con 0x7f21c40039e0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c400baf0 con 0x7f21c4004d40 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c400bcf0 con 0x7f21d806b1a0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dd63a640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c40039e0 0x7f21c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dce39640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 0x7f21c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dde3b640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21c4004d40 0x7f21c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dd63a640 1 -- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 msgr2=0x7f21c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dd63a640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d806b1a0 0x7f21c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.560+0000 7f21dd63a640 1 -- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21c4004d40 msgr2=0x7f21c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21dd63a640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21c4004d40 0x7f21c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21dd63a640 1 -- 172.21.15.148:0/2528880753 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4003520 con 0x7f21c40039e0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21dde3b640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21c4004d40 0x7f21c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21dd63a640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c40039e0 0x7f21c4003e00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009ac0 tx=0x7f21c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6e8801858f7a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc036070 con 0x7f21c40039e0 2024-04-25T09:57:48.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc031400 con 0x7f21c40039e0 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.561+0000 7f21cbfff640 1 -- 172.21.15.148:0/2528880753 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc009690 con 0x7f21c40039e0 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21df89e640 1 -- 172.21.15.148:0/2528880753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c40039e0 msgr2=0x7f21c4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21df89e640 1 --2- 172.21.15.148:0/2528880753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21c40039e0 0x7f21c4003e00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009ac0 tx=0x7f21c4005830 comp rx=0 tx=0).stop 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21dde3b640 1 -- 172.21.15.148:0/2528880753 reap_dead start 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21df89e640 1 -- 172.21.15.148:0/2528880753 shutdown_connections 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21df89e640 1 -- 172.21.15.148:0/2528880753 >> 172.21.15.148:0/2528880753 conn(0x7f21d80b2220 msgr2=0x7f21d80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21df89e640 1 -- 172.21.15.148:0/2528880753 shutdown_connections 2024-04-25T09:57:48.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.562+0000 7f21df89e640 1 -- 172.21.15.148:0/2528880753 wait complete. 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.563+0000 7f21df89e640 1 Processor -- start 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.563+0000 7f21df89e640 1 -- start start 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21df89e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21c40039e0 0x7f21d80cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21df89e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d815aa00 0x7f21d80c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21df89e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 0x7f21d80c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21df89e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8070330 con 0x7f21d815aea0 2024-04-25T09:57:48.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21df89e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8162720 con 0x7f21d815aa00 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21df89e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d806f9c0 con 0x7f21c40039e0 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 0x7f21d80c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 0x7f21d80c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55498/0 (socket says 172.21.15.148:55498) 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 -- 172.21.15.148:0/206414790 learned_addr learned my addr 172.21.15.148:0/206414790 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dd63a640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21c40039e0 0x7f21d80cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dce39640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d815aa00 0x7f21d80c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 -- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21c40039e0 msgr2=0x7f21d80cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21c40039e0 0x7f21d80cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 -- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d815aa00 msgr2=0x7f21d80c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d815aa00 0x7f21d80c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 -- 172.21.15.148:0/206414790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80b4c70 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.564+0000 7f21dde3b640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 0x7f21d80c69d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f21d80c7750 tx=0x7f21d000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.565+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0006b50 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.565+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d0006d10 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.565+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80b4f30 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.565+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0003970 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.565+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d80b54f0 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.566+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d80b26d0 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.566+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21d001a020 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.566+0000 7f21c9ffb640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.567+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21d0089a70 con 0x7f21d815aea0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.567+0000 7f21dd63a640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.567+0000 7f21dd63a640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21cc033000 tx=0x7f21cc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.571+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21d0058760 con 0x7f21d815aea0 2024-04-25T09:57:48.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.743+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21d806d370 con 0x7f21d815aea0 2024-04-25T09:57:48.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.744+0000 7f21c9ffb640 1 -- 172.21.15.148:0/206414790 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f21d0058060 con 0x7f21d815aea0 2024-04-25T09:57:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:48.749 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:11.985711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21bc06b4d0 msgr2=0x7f21bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f21cc033000 tx=0x7f21cc008030 comp rx=0 tx=0).stop 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 msgr2=0x7f21d80c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 0x7f21d80c69d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f21d80c7750 tx=0x7f21d000eb90 comp rx=0 tx=0).stop 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 shutdown_connections 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d815aea0 0x7f21d80c69d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d815aa00 0x7f21d80c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 --2- 172.21.15.148:0/206414790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21c40039e0 0x7f21d80cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 >> 172.21.15.148:0/206414790 conn(0x7f21d80b2220 msgr2=0x7f21d80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:48.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 shutdown_connections 2024-04-25T09:57:48.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.746+0000 7f21df89e640 1 -- 172.21.15.148:0/206414790 wait complete. 2024-04-25T09:57:48.761 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:57:48.761 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.853+0000 7f9d6a64d640 1 Processor -- start 2024-04-25T09:57:48.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 -- start start 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d6415bd30 0x7f9d640cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d640cfe40 0x7f9d640d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d640d1930 0x7f9d640d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d640d0350 con 0x7f9d640d1930 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d640d0630 con 0x7f9d6415bd30 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.854+0000 7f9d6a64d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d640d0910 con 0x7f9d640cfe40 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d640d1930 0x7f9d640d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d640d1930 0x7f9d640d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55506/0 (socket says 172.21.15.148:55506) 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 -- 172.21.15.148:0/1641863150 learned_addr learned my addr 172.21.15.148:0/1641863150 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d637fe640 1 --2- 172.21.15.148:0/1641863150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d640cfe40 0x7f9d640d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d63fff640 1 --2- 172.21.15.148:0/1641863150 >> v2:172.21.15.188:3300/0 conn(0x7f9d6415bd30 0x7f9d640cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 -- 172.21.15.148:0/1641863150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d640cfe40 msgr2=0x7f9d640d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 --2- 172.21.15.148:0/1641863150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d640cfe40 0x7f9d640d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 -- 172.21.15.148:0/1641863150 >> v2:172.21.15.188:3300/0 conn(0x7f9d6415bd30 msgr2=0x7f9d640cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 --2- 172.21.15.148:0/1641863150 >> v2:172.21.15.188:3300/0 conn(0x7f9d6415bd30 0x7f9d640cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.855+0000 7f9d68bea640 1 -- 172.21.15.148:0/1641863150 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640d0bf0 con 0x7f9d640d1930 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d68bea640 1 --2- 172.21.15.148:0/1641863150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1930 0x7f9d640d1d40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9d640702b0 tx=0x7f9d5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373757b8c7e3c26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d62ffd640 1 -- 172.21.15.148:0/1641863150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c036070 con 0x7f9d640d1930 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d62ffd640 1 -- 172.21.15.148:0/1641863150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c031600 con 0x7f9d640d1930 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d62ffd640 1 -- 172.21.15.148:0/1641863150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0319d0 con 0x7f9d640d1930 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 -- 172.21.15.148:0/1641863150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1930 msgr2=0x7f9d640d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/1641863150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1930 0x7f9d640d1d40 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9d640702b0 tx=0x7f9d5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 -- 172.21.15.148:0/1641863150 shutdown_connections 2024-04-25T09:57:48.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/1641863150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1930 0x7f9d640d1d40 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/1641863150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d640cfe40 0x7f9d640d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/1641863150 >> v2:172.21.15.188:3300/0 conn(0x7f9d6415bd30 0x7f9d640cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 -- 172.21.15.148:0/1641863150 >> 172.21.15.148:0/1641863150 conn(0x7f9d6406f590 msgr2=0x7f9d6406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:48.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 -- 172.21.15.148:0/1641863150 shutdown_connections 2024-04-25T09:57:48.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.856+0000 7f9d6a64d640 1 -- 172.21.15.148:0/1641863150 wait complete. 2024-04-25T09:57:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.857+0000 7f9d6a64d640 1 Processor -- start 2024-04-25T09:57:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 -- start start 2024-04-25T09:57:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 0x7f9d640afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640d1930 0x7f9d640ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415bd30 0x7f9d640a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d640b73b0 con 0x7f9d640cfe40 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6415c110 con 0x7f9d640d1930 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d6a64d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d640b7530 con 0x7f9d6415bd30 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d63fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 0x7f9d640afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d63fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 0x7f9d640afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55522/0 (socket says 172.21.15.148:55522) 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.858+0000 7f9d63fff640 1 -- 172.21.15.148:0/261611476 learned_addr learned my addr 172.21.15.148:0/261611476 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d63fff640 1 -- 172.21.15.148:0/261611476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415bd30 msgr2=0x7f9d640a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d63fff640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415bd30 0x7f9d640a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d63fff640 1 -- 172.21.15.148:0/261611476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640d1930 msgr2=0x7f9d640ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:48.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d63fff640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640d1930 0x7f9d640ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d63fff640 1 -- 172.21.15.148:0/261611476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640cb820 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d63fff640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 0x7f9d640afdd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9b0 tx=0x7f9d5400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54017070 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5400c9a0 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d640cb260 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400cd90 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.859+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6414b810 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.860+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6414bc50 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.860+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d5401a070 con 0x7f9d640cfe40 2024-04-25T09:57:48.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.861+0000 7f9d60ff9640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d5006b4d0 0x7f9d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.861+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d5408a6e0 con 0x7f9d640cfe40 2024-04-25T09:57:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.861+0000 7f9d637fe640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d5006b4d0 0x7f9d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.861+0000 7f9d637fe640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d5006b4d0 0x7f9d5006d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d6415f5a0 tx=0x7f9d58009460 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:48.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:48.865+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d54059460 con 0x7f9d640cfe40 2024-04-25T09:57:49.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.074+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d6415f5d0 con 0x7f9d640cfe40 2024-04-25T09:57:49.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.074+0000 7f9d60ff9640 1 -- 172.21.15.148:0/261611476 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f9d54058d60 con 0x7f9d640cfe40 2024-04-25T09:57:49.079 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:49.079 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:47.019261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:49.079 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 27 2024-04-25T09:57:49.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d5006b4d0 msgr2=0x7f9d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d5006b4d0 0x7f9d5006d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d6415f5a0 tx=0x7f9d58009460 comp rx=0 tx=0).stop 2024-04-25T09:57:49.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 msgr2=0x7f9d640afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 0x7f9d640afdd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e9b0 tx=0x7f9d5400ee70 comp rx=0 tx=0).stop 2024-04-25T09:57:49.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 shutdown_connections 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d5006b4d0 0x7f9d5006d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415bd30 0x7f9d640a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640d1930 0x7f9d640ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 --2- 172.21.15.148:0/261611476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cfe40 0x7f9d640afdd0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.076+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 >> 172.21.15.148:0/261611476 conn(0x7f9d6406f590 msgr2=0x7f9d640b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.077+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 shutdown_connections 2024-04-25T09:57:49.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.077+0000 7f9d6a64d640 1 -- 172.21.15.148:0/261611476 wait complete. 2024-04-25T09:57:49.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.185+0000 7f25b4015640 1 Processor -- start 2024-04-25T09:57:49.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.185+0000 7f25b4015640 1 -- start start 2024-04-25T09:57:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b4015640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25ac15db70 0x7f25ac162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b4015640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25ac0b6cb0 0x7f25ac0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b4015640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac162690 con 0x7f25ac0b5a60 2024-04-25T09:57:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b4015640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac0d3800 con 0x7f25ac15db70 2024-04-25T09:57:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b4015640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac0ceee0 con 0x7f25ac0b6cb0 2024-04-25T09:57:49.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25ac15db70 0x7f25ac162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f25ac15db70 0x7f25ac162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55530/0 (socket says 172.21.15.148:55530) 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 -- 172.21.15.148:0/3969259093 learned_addr learned my addr 172.21.15.148:0/3969259093 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b15b0640 1 --2- 172.21.15.148:0/3969259093 >> v2:172.21.15.188:3300/0 conn(0x7f25ac0b6cb0 0x7f25ac0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 -- 172.21.15.148:0/3969259093 >> v2:172.21.15.188:3300/0 conn(0x7f25ac0b6cb0 msgr2=0x7f25ac0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 --2- 172.21.15.148:0/3969259093 >> v2:172.21.15.188:3300/0 conn(0x7f25ac0b6cb0 0x7f25ac0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 -- 172.21.15.148:0/3969259093 >> v1:172.21.15.188:6789/0 conn(0x7f25ac0b5a60 legacy=0x7f25ac06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.186+0000 7f25b1db1640 1 -- 172.21.15.148:0/3969259093 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0d1d10 con 0x7f25ac15db70 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b1db1640 1 --2- 172.21.15.148:0/3969259093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac162150 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0d3980 tx=0x7f259c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27011c53d7d195f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b0daf640 1 -- 172.21.15.148:0/3969259093 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c037070 con 0x7f25ac15db70 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b0daf640 1 -- 172.21.15.148:0/3969259093 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c032440 con 0x7f25ac15db70 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b0daf640 1 -- 172.21.15.148:0/3969259093 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c032810 con 0x7f25ac15db70 2024-04-25T09:57:49.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 -- 172.21.15.148:0/3969259093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 msgr2=0x7f25ac162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 --2- 172.21.15.148:0/3969259093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac162150 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0d3980 tx=0x7f259c02a180 comp rx=0 tx=0).stop 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 -- 172.21.15.148:0/3969259093 shutdown_connections 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 --2- 172.21.15.148:0/3969259093 >> v2:172.21.15.188:3300/0 conn(0x7f25ac0b6cb0 0x7f25ac0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 --2- 172.21.15.148:0/3969259093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac162150 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 -- 172.21.15.148:0/3969259093 >> 172.21.15.148:0/3969259093 conn(0x7f25ac0d0520 msgr2=0x7f25ac0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 -- 172.21.15.148:0/3969259093 shutdown_connections 2024-04-25T09:57:49.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.187+0000 7f25b4015640 1 -- 172.21.15.148:0/3969259093 wait complete. 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 Processor -- start 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 -- start start 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25ac0b5a60 0x7f25ac15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25ac0b6cb0 0x7f25ac152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac15df50 con 0x7f25ac15db70 2024-04-25T09:57:49.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0d22d0 con 0x7f25ac0b6cb0 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b4015640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0ce530 con 0x7f25ac0b5a60 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b25b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b25b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55542/0 (socket says 172.21.15.148:55542) 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.189+0000 7f25b25b2640 1 -- 172.21.15.148:0/1711518225 learned_addr learned my addr 172.21.15.148:0/1711518225 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b15b0640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25ac0b6cb0 0x7f25ac152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b1db1640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25ac0b5a60 0x7f25ac15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b25b2640 1 -- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25ac0b5a60 msgr2=0x7f25ac15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b25b2640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25ac0b5a60 0x7f25ac15c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b25b2640 1 -- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25ac0b6cb0 msgr2=0x7f25ac152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b25b2640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25ac0b6cb0 0x7f25ac152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b25b2640 1 -- 172.21.15.148:0/1711518225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac161130 con 0x7f25ac15db70 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b25b2640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac152ea0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c780 tx=0x7f25a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4004b60 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a4004d00 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400aa90 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac153c90 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.190+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac154250 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.191+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac073990 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.192+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25a400ac30 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.192+0000 7f25a27fc640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f259006b4d0 0x7f259006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.192+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f25a408a5f0 con 0x7f25ac15db70 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.192+0000 7f25b1db1640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f259006b4d0 0x7f259006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.192+0000 7f25b1db1640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f259006b4d0 0x7f259006d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f259c034000 tx=0x7f259c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.196+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25a4020070 con 0x7f25ac15db70 2024-04-25T09:57:49.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.403+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25ac0d1820 con 0x7f25ac15db70 2024-04-25T09:57:49.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.404+0000 7f25a27fc640 1 -- 172.21.15.148:0/1711518225 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4951 (secure 0 0 0) 0x7f25a40592e0 con 0x7f25ac15db70 2024-04-25T09:57:49.409 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:49.409 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:57:10.325425+0000","modified":"2024-04-25T09:57:47.019261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/3309890912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3309890912},{"type":"v1","addr":"172.21.15.148:6837","nonce":3309890912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T09:57:49.409 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 27 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f259006b4d0 msgr2=0x7f259006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f259006b4d0 0x7f259006d990 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f259c034000 tx=0x7f259c02e000 comp rx=0 tx=0).stop 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 msgr2=0x7f25ac152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac152ea0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f25a400c780 tx=0x7f25a400cc40 comp rx=0 tx=0).stop 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 shutdown_connections 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f259006b4d0 0x7f259006d990 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25ac15db70 0x7f25ac152ea0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25ac0b6cb0 0x7f25ac152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 --2- 172.21.15.148:0/1711518225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25ac0b5a60 0x7f25ac15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 >> 172.21.15.148:0/1711518225 conn(0x7f25ac0d0520 msgr2=0x7f25ac06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:49.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 shutdown_connections 2024-04-25T09:57:49.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.406+0000 7f25b4015640 1 -- 172.21.15.148:0/1711518225 wait complete. 2024-04-25T09:57:49.422 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.514+0000 7f0351934640 1 Processor -- start 2024-04-25T09:57:49.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.514+0000 7f0351934640 1 -- start start 2024-04-25T09:57:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f0351934640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 0x7f034c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f0351934640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c0b6220 con 0x7f034c0b2cb0 2024-04-25T09:57:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f0351934640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c0b97a0 con 0x7f034c0b92a0 2024-04-25T09:57:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f0351934640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c15dfc0 con 0x7f034c15dbe0 2024-04-25T09:57:49.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f0343fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f034c0b2cb0 0x7f034c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35682/0 (socket says 172.21.15.148:35682) 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f0343fff640 1 -- 172.21.15.148:0/3518675536 learned_addr learned my addr 172.21.15.148:0/3518675536 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034affd640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 0x7f034c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827096468 0 0) 0x7f034c0b6220 con 0x7f034c0b2cb0 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0338003520 con 0x7f034c0b2cb0 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192391708 0 0) 0x7f0338003520 con 0x7f034c0b2cb0 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 msgr2=0x7f034c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 0x7f034c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 >> v1:172.21.15.188:6789/0 conn(0x7f034c0b92a0 legacy=0x7f034c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.515+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0d1c50 con 0x7f034c0b2cb0 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f0334001670 con 0x7f034c0b2cb0 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 >> v1:172.21.15.148:6789/0 conn(0x7f034c0b2cb0 legacy=0x7f034c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03380039e0 0x7f0338003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0338004800 0x7f0338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 0x7f03380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033800ba70 con 0x7f0338004800 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033800bbf0 con 0x7f03380039e0 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033800bdf0 con 0x7f034c15dbe0 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f0343fff640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0338004800 0x7f0338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034affd640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03380039e0 0x7f0338003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034b7fe640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 0x7f03380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f0343fff640 1 -- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 msgr2=0x7f03380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f0343fff640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c15dbe0 0x7f03380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f0343fff640 1 -- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03380039e0 msgr2=0x7f0338003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f0343fff640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03380039e0 0x7f0338003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f0343fff640 1 -- 172.21.15.148:0/3518675536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338003520 con 0x7f0338004800 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.516+0000 7f034affd640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03380039e0 0x7f0338003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f0343fff640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0338004800 0x7f0338004c80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f033400c210 tx=0x7f0338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb342249cac26e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334007d10 con 0x7f0338004800 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334005ad0 con 0x7f0338004800 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f034a7fc640 1 -- 172.21.15.148:0/3518675536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334007070 con 0x7f0338004800 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f0351934640 1 -- 172.21.15.148:0/3518675536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0338004800 msgr2=0x7f0338004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f0351934640 1 --2- 172.21.15.148:0/3518675536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0338004800 0x7f0338004c80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f033400c210 tx=0x7f0338005830 comp rx=0 tx=0).stop 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f034b7fe640 1 -- 172.21.15.148:0/3518675536 reap_dead start 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f0351934640 1 -- 172.21.15.148:0/3518675536 shutdown_connections 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.517+0000 7f0351934640 1 -- 172.21.15.148:0/3518675536 >> 172.21.15.148:0/3518675536 conn(0x7f034c061a20 msgr2=0x7f034c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.518+0000 7f0351934640 1 -- 172.21.15.148:0/3518675536 shutdown_connections 2024-04-25T09:57:49.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.518+0000 7f0351934640 1 -- 172.21.15.148:0/3518675536 wait complete. 2024-04-25T09:57:49.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 Processor -- start 2024-04-25T09:57:49.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 -- start start 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0338004800 0x7f034c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c160540 0x7f034c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 0x7f034c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0d2210 con 0x7f034c15dfc0 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0d1a00 con 0x7f0338004800 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f0351934640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0b6ab0 con 0x7f034c160540 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f034b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 0x7f034c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f034b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 0x7f034c15e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55560/0 (socket says 172.21.15.148:55560) 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.519+0000 7f034b7fe640 1 -- 172.21.15.148:0/73484021 learned_addr learned my addr 172.21.15.148:0/73484021 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:49.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034b7fe640 1 -- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c160540 msgr2=0x7f034c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f0343fff640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c160540 0x7f034c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034affd640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0338004800 0x7f034c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034b7fe640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c160540 0x7f034c1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034b7fe640 1 -- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0338004800 msgr2=0x7f034c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034b7fe640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0338004800 0x7f034c154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034b7fe640 1 -- 172.21.15.148:0/73484021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c160f00 con 0x7f034c15dfc0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f034b7fe640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 0x7f034c15e470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f034400f800 tx=0x7f034400fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c009690 con 0x7f034c15dfc0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0344018070 con 0x7f034c15dfc0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c154560 con 0x7f034c15dfc0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344003da0 con 0x7f034c15dfc0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.520+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c15ec60 con 0x7f034c15dfc0 2024-04-25T09:57:49.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.521+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c073990 con 0x7f034c15dfc0 2024-04-25T09:57:49.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.521+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03440050d0 con 0x7f034c15dfc0 2024-04-25T09:57:49.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.522+0000 7f0348ff9640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032c06b500 0x7f032c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.522+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0344089a20 con 0x7f034c15dfc0 2024-04-25T09:57:49.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.522+0000 7f034affd640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032c06b500 0x7f032c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.522+0000 7f034affd640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032c06b500 0x7f032c06d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f034c15cf40 tx=0x7f033c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.526+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0344021020 con 0x7f034c15dfc0 2024-04-25T09:57:49.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.698+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f034c061fa0 con 0x7f034c15dfc0 2024-04-25T09:57:49.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.699+0000 7f0348ff9640 1 -- 172.21.15.148:0/73484021 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16990 (secure 0 0 0) 0x7f034c154560 con 0x7f034c15dfc0 2024-04-25T09:57:49.704 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:49.704 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:11.985711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:57:08.515642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T09:57:09.289387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032c06b500 msgr2=0x7f032c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032c06b500 0x7f032c06d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f034c15cf40 tx=0x7f033c008040 comp rx=0 tx=0).stop 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 msgr2=0x7f034c15e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 0x7f034c15e470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f034400f800 tx=0x7f034400fcc0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 shutdown_connections 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032c06b500 0x7f032c06d9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f034c15dfc0 0x7f034c15e470 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f034c160540 0x7f034c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 --2- 172.21.15.148:0/73484021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0338004800 0x7f034c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 >> 172.21.15.148:0/73484021 conn(0x7f034c061a20 msgr2=0x7f034c0d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 shutdown_connections 2024-04-25T09:57:49.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.702+0000 7f0351934640 1 -- 172.21.15.148:0/73484021 wait complete. 2024-04-25T09:57:49.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:57:49.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.812+0000 7fea2df89640 1 Processor -- start 2024-04-25T09:57:49.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.812+0000 7fea2df89640 1 -- start start 2024-04-25T09:57:49.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.812+0000 7fea2df89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 0x7fea280b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea2df89640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fea280b54a0 0x7fea280b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea2df89640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea280cf1d0 con 0x7fea28161240 2024-04-25T09:57:49.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea2df89640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea280b7120 con 0x7fea280b54a0 2024-04-25T09:57:49.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea2df89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea28161620 con 0x7fea28161890 2024-04-25T09:57:49.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea277fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fea28161240 0x7fea280ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35686/0 (socket says 172.21.15.148:35686) 2024-04-25T09:57:49.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea277fe640 1 -- 172.21.15.148:0/4184656763 learned_addr learned my addr 172.21.15.148:0/4184656763 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:49.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea1ffff640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 0x7fea280b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104115541 0 0) 0x7fea280cf1d0 con 0x7fea28161240 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea27fff640 1 --2- 172.21.15.148:0/4184656763 >> v2:172.21.15.188:3300/0 conn(0x7fea280b54a0 0x7fea280b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea14003520 con 0x7fea28161240 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287022664 0 0) 0x7fea14003520 con 0x7fea28161240 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 msgr2=0x7fea280b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 0x7fea280b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 >> v2:172.21.15.188:3300/0 conn(0x7fea280b54a0 msgr2=0x7fea280b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 --2- 172.21.15.148:0/4184656763 >> v2:172.21.15.188:3300/0 conn(0x7fea280b54a0 0x7fea280b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea28161e00 con 0x7fea28161240 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fea18001670 con 0x7fea28161240 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 >> v1:172.21.15.148:6789/0 conn(0x7fea28161240 legacy=0x7fea280ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 0x7fea140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea14004d40 0x7fea140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28161620 con 0x7fea14003f30 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1400b9c0 con 0x7fea14004d40 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1400bb40 con 0x7fea28161890 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.813+0000 7fea1ffff640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea277fe640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 0x7fea140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea27fff640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea14004d40 0x7fea140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea1ffff640 1 -- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 msgr2=0x7fea140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea1ffff640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28161890 0x7fea140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea1ffff640 1 -- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea14004d40 msgr2=0x7fea140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea1ffff640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea14004d40 0x7fea140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea1ffff640 1 -- 172.21.15.148:0/4184656763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea14003520 con 0x7fea14003f30 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.814+0000 7fea1ffff640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea14004800 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ac0 tx=0x7fea14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad55d32b05fcffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10036070 con 0x7fea14003f30 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea10031400 con 0x7fea14003f30 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea26ffd640 1 -- 172.21.15.148:0/4184656763 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10009690 con 0x7fea14003f30 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea2df89640 1 -- 172.21.15.148:0/4184656763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 msgr2=0x7fea14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea2df89640 1 --2- 172.21.15.148:0/4184656763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea14004800 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ac0 tx=0x7fea14005830 comp rx=0 tx=0).stop 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea27fff640 1 -- 172.21.15.148:0/4184656763 reap_dead start 2024-04-25T09:57:49.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea2df89640 1 -- 172.21.15.148:0/4184656763 shutdown_connections 2024-04-25T09:57:49.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea2df89640 1 -- 172.21.15.148:0/4184656763 >> 172.21.15.148:0/4184656763 conn(0x7fea280d04a0 msgr2=0x7fea280d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:49.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea2df89640 1 -- 172.21.15.148:0/4184656763 shutdown_connections 2024-04-25T09:57:49.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.815+0000 7fea2df89640 1 -- 172.21.15.148:0/4184656763 wait complete. 2024-04-25T09:57:49.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.816+0000 7fea2df89640 1 Processor -- start 2024-04-25T09:57:49.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.816+0000 7fea2df89640 1 -- start start 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea2df89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea280ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea2df89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea280b83a0 0x7fea280ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea2df89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28151a90 0x7fea28151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea2df89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea281626a0 con 0x7fea14003f30 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea2df89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea281623c0 con 0x7fea280b83a0 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea2df89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea280d15f0 con 0x7fea28151a90 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea280ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea280ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55586/0 (socket says 172.21.15.148:55586) 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 -- 172.21.15.148:0/3468134414 learned_addr learned my addr 172.21.15.148:0/3468134414 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:49.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea1ffff640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea280b83a0 0x7fea280ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea27fff640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28151a90 0x7fea28151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 -- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28151a90 msgr2=0x7fea28151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28151a90 0x7fea28151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 -- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea280b83a0 msgr2=0x7fea280ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.817+0000 7fea277fe640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea280b83a0 0x7fea280ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea277fe640 1 -- 172.21.15.148:0/3468134414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea28152450 con 0x7fea14003f30 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea27fff640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28151a90 0x7fea28151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea277fe640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea280ca320 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fea180073a0 tx=0x7fea18007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1801a070 con 0x7fea14003f30 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1800f9c0 con 0x7fea14003f30 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea280c8fb0 con 0x7fea14003f30 2024-04-25T09:57:49.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800fd90 con 0x7fea14003f30 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.818+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea280c9570 con 0x7fea14003f30 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.819+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea28073a00 con 0x7fea14003f30 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.819+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea1801d360 con 0x7fea14003f30 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.820+0000 7fea24ff9640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea0806b4d0 0x7fea0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.820+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fea18089d00 con 0x7fea14003f30 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.820+0000 7fea1ffff640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea0806b4d0 0x7fea0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.820+0000 7fea1ffff640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea0806b4d0 0x7fea0806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fea10033000 tx=0x7fea10008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:49.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:49.824+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea28152450 con 0x7fea14003f30 2024-04-25T09:57:50.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.034+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fea280d2c80 con 0x7fea14003f30 2024-04-25T09:57:50.039 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:57:50.035+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:57:50.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.036+0000 7fea24ff9640 1 -- 172.21.15.148:0/3468134414 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fea180583b0 con 0x7fea14003f30 2024-04-25T09:57:50.041 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:57:50.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.039+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea0806b4d0 msgr2=0x7fea0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:50.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.039+0000 7fea2df89640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea0806b4d0 0x7fea0806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fea10033000 tx=0x7fea10008030 comp rx=0 tx=0).stop 2024-04-25T09:57:50.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.039+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 msgr2=0x7fea280ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.039+0000 7fea2df89640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea280ca320 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fea180073a0 tx=0x7fea18007860 comp rx=0 tx=0).stop 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 shutdown_connections 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea0806b4d0 0x7fea0806d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea28151a90 0x7fea28151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea280b83a0 0x7fea280ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 --2- 172.21.15.148:0/3468134414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea14003f30 0x7fea280ca320 secure :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fea180073a0 tx=0x7fea18007860 comp rx=0 tx=0).stop 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 >> 172.21.15.148:0/3468134414 conn(0x7fea280d04a0 msgr2=0x7fea280d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 shutdown_connections 2024-04-25T09:57:50.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.040+0000 7fea2df89640 1 -- 172.21.15.148:0/3468134414 wait complete. 2024-04-25T09:57:50.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:57:50.082 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:57:50.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.153+0000 7fbbc2f0a640 1 Processor -- start 2024-04-25T09:57:50.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.153+0000 7fbbc2f0a640 1 -- start start 2024-04-25T09:57:50.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbc2f0a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbbc06b2f0 0x7fbbbc0aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbc2f0a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbbc15cde0 con 0x7fbbbc15c970 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbc2f0a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbbc0af540 con 0x7fbbbc1454a0 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbc2f0a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbbc0af820 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbbc06b2f0 0x7fbbbc0aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbbc06b2f0 0x7fbbbc0aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55610/0 (socket says 172.21.15.148:55610) 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbc14a7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbbbc15c970 0x7fbbbc15d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35704/0 (socket says 172.21.15.148:35704) 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb3fff640 1 -- 172.21.15.148:0/318651829 learned_addr learned my addr 172.21.15.148:0/318651829 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb3fff640 1 -- 172.21.15.148:0/318651829 >> v1:172.21.15.188:6789/0 conn(0x7fbbbc1454a0 legacy=0x7fbbbc06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb3fff640 1 -- 172.21.15.148:0/318651829 >> v1:172.21.15.148:6789/0 conn(0x7fbbbc15c970 legacy=0x7fbbbc15d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb3fff640 1 -- 172.21.15.148:0/318651829 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0afb00 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.154+0000 7fbbb37fe640 1 -- 172.21.15.148:0/318651829 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003080551 0 0) 0x7fbbbc15cde0 con 0x7fbbbc15c970 2024-04-25T09:57:50.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.155+0000 7fbbb3fff640 1 --2- 172.21.15.148:0/318651829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc0aee60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009870 tx=0x7fbbac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bed8b7c81341b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.155+0000 7fbbb37fe640 1 -- 172.21.15.148:0/318651829 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac037070 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.155+0000 7fbbb37fe640 1 -- 172.21.15.148:0/318651829 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac032440 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.155+0000 7fbbb37fe640 1 -- 172.21.15.148:0/318651829 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac032810 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/318651829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 msgr2=0x7fbbbc0aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/318651829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc0aee60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009870 tx=0x7fbbac02a2a0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/318651829 shutdown_connections 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/318651829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc0aee60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/318651829 >> 172.21.15.148:0/318651829 conn(0x7fbbbc0b5af0 msgr2=0x7fbbbc0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/318651829 shutdown_connections 2024-04-25T09:57:50.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.156+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/318651829 wait complete. 2024-04-25T09:57:50.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.157+0000 7fbbc2f0a640 1 Processor -- start 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.157+0000 7fbbc2f0a640 1 -- start start 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc15b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbbc1454a0 0x7fbbbc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbbc15c970 0x7fbbbc0b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc15dd20 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc15dba0 con 0x7fbbbc15c970 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbbc15dea0 con 0x7fbbbc1454a0 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc15b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55612/0 (socket says 172.21.15.148:55612) 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 -- 172.21.15.148:0/1645243021 learned_addr learned my addr 172.21.15.148:0/1645243021 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 -- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbbc1454a0 msgr2=0x7fbbbc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbbc1454a0 0x7fbbbc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 -- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbbc15c970 msgr2=0x7fbbbc0b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc14a7640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbbc15c970 0x7fbbbc0b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbbc15c970 0x7fbbbc0b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 -- 172.21.15.148:0/1645243021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0d11b0 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc0ca6640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc15b470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c410 tx=0x7fbbb800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8017070 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb80035f0 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8003a20 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbbc0b2de0 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.158+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbbc0b33a0 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.159+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbbc06cc80 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.160+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbb801a070 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.160+0000 7fbbb17fa640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbba806b4d0 0x7fbba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.160+0000 7fbbb3fff640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbba806b4d0 0x7fbba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.160+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbbb8089810 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.161+0000 7fbbb3fff640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbba806b4d0 0x7fbba806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbbac034000 tx=0x7fbbac02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:50.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.164+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbb8058540 con 0x7fbbbc06b2f0 2024-04-25T09:57:50.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:50.371+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbbbc073a00 con 0x7fbbbc06b2f0 2024-04-25T09:57:51.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.038+0000 7fbbb17fa640 1 -- 172.21.15.148:0/1645243021 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbbb80010c0 con 0x7fbbbc06b2f0 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbba806b4d0 msgr2=0x7fbba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbba806b4d0 0x7fbba806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbbac034000 tx=0x7fbbac02c000 comp rx=0 tx=0).stop 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 msgr2=0x7fbbbc15b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc15b470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800c410 tx=0x7fbbb800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 shutdown_connections 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbba806b4d0 0x7fbba806d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbbc15c970 0x7fbbbc0b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbbc1454a0 0x7fbbbc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 --2- 172.21.15.148:0/1645243021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbbc06b2f0 0x7fbbbc15b470 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 >> 172.21.15.148:0/1645243021 conn(0x7fbbbc0b5af0 msgr2=0x7fbbbc15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 shutdown_connections 2024-04-25T09:57:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.040+0000 7fbbc2f0a640 1 -- 172.21.15.148:0/1645243021 wait complete. 2024-04-25T09:57:51.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:57:51.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.199+0000 7f7d706b6640 1 Processor -- start 2024-04-25T09:57:51.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.199+0000 7f7d706b6640 1 -- start start 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d706b6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d68070dc0 0x7f7d6806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d706b6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d6815deb0 0x7f7d6815e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d706b6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680d1770 con 0x7f7d680711a0 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d706b6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d680b0eb0 con 0x7f7d6815deb0 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d706b6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d681622b0 con 0x7f7d68070dc0 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d6ec53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d6815deb0 0x7f7d6815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d6ec53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d6815deb0 0x7f7d6815e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55618/0 (socket says 172.21.15.148:55618) 2024-04-25T09:57:51.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.200+0000 7f7d6ec53640 1 -- 172.21.15.148:0/2531570672 learned_addr learned my addr 172.21.15.148:0/2531570672 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6ec53640 1 -- 172.21.15.148:0/2531570672 >> v2:172.21.15.188:3300/0 conn(0x7f7d68070dc0 msgr2=0x7f7d6806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6ec53640 1 --2- 172.21.15.148:0/2531570672 >> v2:172.21.15.188:3300/0 conn(0x7f7d68070dc0 0x7f7d6806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6ec53640 1 -- 172.21.15.148:0/2531570672 >> v1:172.21.15.188:6789/0 conn(0x7f7d680711a0 legacy=0x7f7d680b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6ec53640 1 -- 172.21.15.148:0/2531570672 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68162740 con 0x7f7d6815deb0 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6ec53640 1 --2- 172.21.15.148:0/2531570672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6815deb0 0x7f7d6815e300 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7d68070a00 tx=0x7f7d6002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70ca62f62b627c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6d450640 1 -- 172.21.15.148:0/2531570672 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60036070 con 0x7f7d6815deb0 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6d450640 1 -- 172.21.15.148:0/2531570672 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60031600 con 0x7f7d6815deb0 2024-04-25T09:57:51.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.201+0000 7f7d6d450640 1 -- 172.21.15.148:0/2531570672 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600319d0 con 0x7f7d6815deb0 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 -- 172.21.15.148:0/2531570672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6815deb0 msgr2=0x7f7d6815e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2531570672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6815deb0 0x7f7d6815e300 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7d68070a00 tx=0x7f7d6002a5d0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 -- 172.21.15.148:0/2531570672 shutdown_connections 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2531570672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d6815deb0 0x7f7d6815e300 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2531570672 >> v2:172.21.15.188:3300/0 conn(0x7f7d68070dc0 0x7f7d6806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 -- 172.21.15.148:0/2531570672 >> 172.21.15.148:0/2531570672 conn(0x7f7d680b3530 msgr2=0x7f7d680b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 -- 172.21.15.148:0/2531570672 shutdown_connections 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.202+0000 7f7d706b6640 1 -- 172.21.15.148:0/2531570672 wait complete. 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.203+0000 7f7d706b6640 1 Processor -- start 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.203+0000 7f7d706b6640 1 -- start start 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d706b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d68070dc0 0x7f7d68148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:51.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d706b6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 0x7f7d6815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d706b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d6815deb0 0x7f7d6814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d706b6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68161c60 con 0x7f7d680711a0 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d706b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680b10f0 con 0x7f7d68070dc0 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d706b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d68161f40 con 0x7f7d6815deb0 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 0x7f7d6815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 0x7f7d6815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55620/0 (socket says 172.21.15.148:55620) 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 -- 172.21.15.148:0/2167328153 learned_addr learned my addr 172.21.15.148:0/2167328153 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6ec53640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d6815deb0 0x7f7d6814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6e452640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d68070dc0 0x7f7d68148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 -- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d6815deb0 msgr2=0x7f7d6814b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d6815deb0 0x7f7d6814b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 -- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d68070dc0 msgr2=0x7f7d68148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d68070dc0 0x7f7d68148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:51.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6dc51640 1 -- 172.21.15.148:0/2167328153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68149030 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6ec53640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d6815deb0 0x7f7d6814b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.204+0000 7f7d6e452640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d68070dc0 0x7f7d68148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.205+0000 7f7d6dc51640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 0x7f7d6815e450 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c410 tx=0x7f7d5800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.205+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58017070 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.205+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d580035f0 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.205+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003a20 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.205+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d681f3d20 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.205+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d681f42e0 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.206+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68073a00 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.206+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d5801a070 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.206+0000 7f7d5effd640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d4c06b4d0 0x7f7d4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.207+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7d58089830 con 0x7f7d680711a0 2024-04-25T09:57:51.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.207+0000 7f7d6e452640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d4c06b4d0 0x7f7d4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:51.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.207+0000 7f7d6e452640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d4c06b4d0 0x7f7d4c06d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7d64006fd0 tx=0x7f7d64008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:51.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.212+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d58058560 con 0x7f7d680711a0 2024-04-25T09:57:51.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:51.382+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7d680b6860 con 0x7f7d680711a0 2024-04-25T09:57:52.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.260+0000 7f7d5effd640 1 -- 172.21.15.148:0/2167328153 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7d580010c0 con 0x7f7d680711a0 2024-04-25T09:57:52.264 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d4c06b4d0 msgr2=0x7f7d4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d4c06b4d0 0x7f7d4c06d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7d64006fd0 tx=0x7f7d64008040 comp rx=0 tx=0).stop 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 msgr2=0x7f7d6815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 0x7f7d6815e450 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c410 tx=0x7f7d5800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 shutdown_connections 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d4c06b4d0 0x7f7d4c06d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d6815deb0 0x7f7d6814b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680711a0 0x7f7d6815e450 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 --2- 172.21.15.148:0/2167328153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d68070dc0 0x7f7d68148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 >> 172.21.15.148:0/2167328153 conn(0x7f7d680b3530 msgr2=0x7f7d680ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 shutdown_connections 2024-04-25T09:57:52.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.263+0000 7f7d706b6640 1 -- 172.21.15.148:0/2167328153 wait complete. 2024-04-25T09:57:52.282 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:57:52.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.386+0000 7fb7de4ac640 1 Processor -- start 2024-04-25T09:57:52.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.386+0000 7fb7de4ac640 1 -- start start 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.386+0000 7fb7de4ac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7d815b710 0x7fb7d80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.386+0000 7fb7de4ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d815e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7de4ac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb7d815ee60 0x7fb7d80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7de4ac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d815e490 con 0x7fb7d815b710 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7de4ac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d80cfd80 con 0x7fb7d815ee60 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7de4ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80d0060 con 0x7fb7d806ec50 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7d815b710 0x7fb7d80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb7d815b710 0x7fb7d80cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55626/0 (socket says 172.21.15.148:55626) 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 -- 172.21.15.148:0/1166847023 learned_addr learned my addr 172.21.15.148:0/1166847023 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d77fe640 1 --2- 172.21.15.148:0/1166847023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7dca49640 1 --2- 172.21.15.148:0/1166847023 >> v2:172.21.15.188:3300/0 conn(0x7fb7d815ee60 0x7fb7d80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 -- 172.21.15.148:0/1166847023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 msgr2=0x7fb7d815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:52.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/1166847023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d815e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 -- 172.21.15.148:0/1166847023 >> v2:172.21.15.188:3300/0 conn(0x7fb7d815ee60 msgr2=0x7fb7d80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:52.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/1166847023 >> v2:172.21.15.188:3300/0 conn(0x7fb7d815ee60 0x7fb7d80ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.387+0000 7fb7d7fff640 1 -- 172.21.15.148:0/1166847023 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80d0340 con 0x7fb7d815b710 2024-04-25T09:57:52.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/1166847023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d80cf640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80ccf60 tx=0x7fb7c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568d8bb5f369e3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7d6ffd640 1 -- 172.21.15.148:0/1166847023 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8037070 con 0x7fb7d815b710 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7d6ffd640 1 -- 172.21.15.148:0/1166847023 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8032440 con 0x7fb7d815b710 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7d6ffd640 1 -- 172.21.15.148:0/1166847023 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8032810 con 0x7fb7d815b710 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7de4ac640 1 -- 172.21.15.148:0/1166847023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 msgr2=0x7fb7d80cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/1166847023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d80cf640 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80ccf60 tx=0x7fb7c802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.388+0000 7fb7de4ac640 1 -- 172.21.15.148:0/1166847023 shutdown_connections 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.389+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/1166847023 >> v2:172.21.15.188:3300/0 conn(0x7fb7d815ee60 0x7fb7d80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.389+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/1166847023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.389+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/1166847023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d80cf640 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.389+0000 7fb7de4ac640 1 -- 172.21.15.148:0/1166847023 >> 172.21.15.148:0/1166847023 conn(0x7fb7d806abe0 msgr2=0x7fb7d806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:52.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.389+0000 7fb7de4ac640 1 -- 172.21.15.148:0/1166847023 shutdown_connections 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.389+0000 7fb7de4ac640 1 -- 172.21.15.148:0/1166847023 wait complete. 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.390+0000 7fb7de4ac640 1 Processor -- start 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.390+0000 7fb7de4ac640 1 -- start start 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d8155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d81595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d815ee60 0x7fb7d8159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d815bc70 con 0x7fb7d815b710 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d815baf0 con 0x7fb7d815ee60 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d80acdf0 con 0x7fb7d806ec50 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d81595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d81595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55642/0 (socket says 172.21.15.148:55642) 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 -- 172.21.15.148:0/3433077972 learned_addr learned my addr 172.21.15.148:0/3433077972 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:52.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7dca49640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d815ee60 0x7fb7d8159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d8155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 -- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 msgr2=0x7fb7d8155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d8155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 -- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d815ee60 msgr2=0x7fb7d8159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d815ee60 0x7fb7d8159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 -- 172.21.15.148:0/3433077972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d81f38e0 con 0x7fb7d815b710 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d8155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d77fe640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d81595e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00e8c0 tx=0x7fb7cc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc017070 con 0x7fb7d815b710 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc00c940 con 0x7fb7d815b710 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d815a150 con 0x7fb7d815b710 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00cd10 con 0x7fb7d815b710 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.391+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d815a710 con 0x7fb7d815b710 2024-04-25T09:57:52.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.392+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d815e900 con 0x7fb7d815b710 2024-04-25T09:57:52.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.393+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7cc01a070 con 0x7fb7d815b710 2024-04-25T09:57:52.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.393+0000 7fb7d4ff9640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7c406b4d0 0x7fb7c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:52.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.393+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fb7cc08a080 con 0x7fb7d815b710 2024-04-25T09:57:52.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.393+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7c406b4d0 0x7fb7c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:52.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.394+0000 7fb7d7fff640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7c406b4d0 0x7fb7c406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8034000 tx=0x7fb7c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:52.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.397+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7cc058910 con 0x7fb7d815b710 2024-04-25T09:57:52.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:52.577+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7d80aca30 con 0x7fb7d815b710 2024-04-25T09:57:53.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.267+0000 7fb7d4ff9640 1 -- 172.21.15.148:0/3433077972 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb7cc05c9b0 con 0x7fb7d815b710 2024-04-25T09:57:53.271 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T09:57:53.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7c406b4d0 msgr2=0x7fb7c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7c406b4d0 0x7fb7c406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8034000 tx=0x7fb7c802e000 comp rx=0 tx=0).stop 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 msgr2=0x7fb7d81595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d81595e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00e8c0 tx=0x7fb7cc00ed80 comp rx=0 tx=0).stop 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 shutdown_connections 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7c406b4d0 0x7fb7c406d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d815ee60 0x7fb7d8159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7d815b710 0x7fb7d81595e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 --2- 172.21.15.148:0/3433077972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d806ec50 0x7fb7d8155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 >> 172.21.15.148:0/3433077972 conn(0x7fb7d806abe0 msgr2=0x7fb7d815c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:53.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 shutdown_connections 2024-04-25T09:57:53.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.270+0000 7fb7de4ac640 1 -- 172.21.15.148:0/3433077972 wait complete. 2024-04-25T09:57:53.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:57:53.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.385+0000 7f15d5389640 1 Processor -- start 2024-04-25T09:57:53.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.385+0000 7f15d5389640 1 -- start start 2024-04-25T09:57:53.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.385+0000 7f15d5389640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15d0161710 0x7f15d00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.385+0000 7f15d5389640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d015e000 con 0x7f15d00ced80 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.385+0000 7f15d5389640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00d1de0 con 0x7f15d015dc20 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.385+0000 7f15d5389640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d0161c60 con 0x7f15d0161710 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f15d00ced80 0x7f15d00d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35716/0 (socket says 172.21.15.148:35716) 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 -- 172.21.15.148:0/1683850889 learned_addr learned my addr 172.21.15.148:0/1683850889 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909690695 0 0) 0x7f15d015e000 con 0x7f15d00ced80 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cf7fe640 1 --2- 172.21.15.148:0/1683850889 >> v2:172.21.15.188:3300/0 conn(0x7f15d0161710 0x7f15d00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b8003520 con 0x7f15d00ced80 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065315734 0 0) 0x7f15b8003520 con 0x7f15d00ced80 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 >> v2:172.21.15.188:3300/0 conn(0x7f15d0161710 msgr2=0x7f15d00d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 --2- 172.21.15.148:0/1683850889 >> v2:172.21.15.188:3300/0 conn(0x7f15d0161710 0x7f15d00d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 >> v1:172.21.15.188:6789/0 conn(0x7f15d015dc20 legacy=0x7f15d00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0161f40 con 0x7f15d00ced80 2024-04-25T09:57:53.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f15c4001670 con 0x7f15d00ced80 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 >> v1:172.21.15.148:6789/0 conn(0x7f15d00ced80 legacy=0x7f15d00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80039b0 0x7f15b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b8004800 0x7f15b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b80051c0 0x7f15b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0161c60 con 0x7f15b8004800 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b800c060 con 0x7f15b80051c0 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b800c260 con 0x7f15b80039b0 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b8004800 0x7f15b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cf7fe640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b80051c0 0x7f15b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ceffd640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80039b0 0x7f15b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 -- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80039b0 msgr2=0x7f15b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80039b0 0x7f15b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 -- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b80051c0 msgr2=0x7f15b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b80051c0 0x7f15b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15ce7fc640 1 -- 172.21.15.148:0/1683850889 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8003520 con 0x7f15b8004800 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cf7fe640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b80051c0 0x7f15b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.386+0000 7f15cf7fe640 1 -- 172.21.15.148:0/1683850889 reap_dead start 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.387+0000 7f15ce7fc640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b8004800 0x7f15b8004c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c210 tx=0x7f15b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e4dfc6e8af51dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.387+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4007d10 con 0x7f15b8004800 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.387+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c4005ad0 con 0x7f15b8004800 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.387+0000 7f15cdffb640 1 -- 172.21.15.148:0/1683850889 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4007070 con 0x7f15b8004800 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 -- 172.21.15.148:0/1683850889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b8004800 msgr2=0x7f15b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b8004800 0x7f15b8004c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c210 tx=0x7f15b8005830 comp rx=0 tx=0).stop 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 -- 172.21.15.148:0/1683850889 shutdown_connections 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 --2- 172.21.15.148:0/1683850889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b8004800 0x7f15b8004c80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 -- 172.21.15.148:0/1683850889 >> 172.21.15.148:0/1683850889 conn(0x7f15d00d0420 msgr2=0x7f15d00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 -- 172.21.15.148:0/1683850889 shutdown_connections 2024-04-25T09:57:53.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.388+0000 7f15d5389640 1 -- 172.21.15.148:0/1683850889 wait complete. 2024-04-25T09:57:53.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.389+0000 7f15d5389640 1 Processor -- start 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.389+0000 7f15d5389640 1 -- start start 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15d5389640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b8004800 0x7f15d00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15d5389640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d015cd00 0x7f15d00b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15d5389640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 0x7f15d014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15d5389640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b5e50 con 0x7f15d00b8790 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15d5389640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0162500 con 0x7f15b8004800 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15d5389640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b5fd0 con 0x7f15d015cd00 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 0x7f15d014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 0x7f15d014ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55658/0 (socket says 172.21.15.148:55658) 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 -- 172.21.15.148:0/2188034213 learned_addr learned my addr 172.21.15.148:0/2188034213 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15ceffd640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b8004800 0x7f15d00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15ce7fc640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d015cd00 0x7f15d00b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 -- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d015cd00 msgr2=0x7f15d00b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d015cd00 0x7f15d00b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 -- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b8004800 msgr2=0x7f15d00cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b8004800 0x7f15d00cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15cf7fe640 1 -- 172.21.15.148:0/2188034213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00cc3c0 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15ceffd640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b8004800 0x7f15d00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.390+0000 7f15ce7fc640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d015cd00 0x7f15d00b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.391+0000 7f15cf7fe640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 0x7f15d014ec50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f15d00b6880 tx=0x7f15c8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.391+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c801a030 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.391+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8010bc0 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.391+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d00cc6a0 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.391+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8003990 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.391+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d014cbc0 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.392+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d00c75d0 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.392+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15c8005d10 con 0x7f15d00b8790 2024-04-25T09:57:53.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.392+0000 7f15b77fe640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15bc06b500 0x7f15bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.393+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f15c808a680 con 0x7f15d00b8790 2024-04-25T09:57:53.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.393+0000 7f15ceffd640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15bc06b500 0x7f15bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.393+0000 7f15ceffd640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15bc06b500 0x7f15bc06d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000c00 tx=0x7f15c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:53.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.397+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15c8025020 con 0x7f15d00b8790 2024-04-25T09:57:53.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.565+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f15d00b25a0 con 0x7f15d00b8790 2024-04-25T09:57:53.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.571+0000 7f15b77fe640 1 -- 172.21.15.148:0/2188034213 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f15c8059670 con 0x7f15d00b8790 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15bc06b500 msgr2=0x7f15bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15bc06b500 0x7f15bc06d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f15c0000c00 tx=0x7f15c0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 msgr2=0x7f15d014ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 0x7f15d014ec50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f15d00b6880 tx=0x7f15c8010590 comp rx=0 tx=0).stop 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 shutdown_connections 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15bc06b500 0x7f15bc06d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d00b8790 0x7f15d014ec50 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15d015cd00 0x7f15d00b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 --2- 172.21.15.148:0/2188034213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b8004800 0x7f15d00cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 >> 172.21.15.148:0/2188034213 conn(0x7f15d00d0420 msgr2=0x7f15d0160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 shutdown_connections 2024-04-25T09:57:53.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.573+0000 7f15d5389640 1 -- 172.21.15.148:0/2188034213 wait complete. 2024-04-25T09:57:53.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T09:57:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.680+0000 7fe7b631b640 1 Processor -- start 2024-04-25T09:57:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.680+0000 7fe7b631b640 1 -- start start 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7b631b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7b00b6cb0 0x7fe7b00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7b631b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7b631b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b00ceee0 con 0x7fe7b015db70 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7b631b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b00d1d10 con 0x7fe7b00b6cb0 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7b631b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00d1ff0 con 0x7fe7b00b5a60 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7af7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7b00b6cb0 0x7fe7b00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7affff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe7b015db70 0x7fe7b0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46600/0 (socket says 172.21.15.148:46600) 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7affff640 1 -- 172.21.15.148:0/2839845367 learned_addr learned my addr 172.21.15.148:0/2839845367 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7af7fe640 1 -- 172.21.15.148:0/2839845367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 msgr2=0x7fe7b00d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7b48b8640 1 --2- 172.21.15.148:0/2839845367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7af7fe640 1 --2- 172.21.15.148:0/2839845367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7af7fe640 1 -- 172.21.15.148:0/2839845367 >> v1:172.21.15.148:6789/0 conn(0x7fe7b015db70 legacy=0x7fe7b0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7af7fe640 1 -- 172.21.15.148:0/2839845367 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00d22d0 con 0x7fe7b00b6cb0 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.681+0000 7fe7aeffd640 1 -- 172.21.15.148:0/2839845367 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244922580 0 0) 0x7fe7b00ceee0 con 0x7fe7b015db70 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.682+0000 7fe7af7fe640 1 --2- 172.21.15.148:0/2839845367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00b5520 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40097a0 tx=0x7fe7a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283b35e1edb0e5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.682+0000 7fe7aeffd640 1 -- 172.21.15.148:0/2839845367 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4037070 con 0x7fe7b00b6cb0 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.682+0000 7fe7aeffd640 1 -- 172.21.15.148:0/2839845367 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a4032440 con 0x7fe7b00b6cb0 2024-04-25T09:57:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.682+0000 7fe7aeffd640 1 -- 172.21.15.148:0/2839845367 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4032810 con 0x7fe7b00b6cb0 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 -- 172.21.15.148:0/2839845367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b00b6cb0 msgr2=0x7fe7b00b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2839845367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00b5520 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40097a0 tx=0x7fe7a402a180 comp rx=0 tx=0).stop 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 -- 172.21.15.148:0/2839845367 shutdown_connections 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2839845367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2839845367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00b5520 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 -- 172.21.15.148:0/2839845367 >> 172.21.15.148:0/2839845367 conn(0x7fe7b00d0520 msgr2=0x7fe7b00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 -- 172.21.15.148:0/2839845367 shutdown_connections 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.683+0000 7fe7b631b640 1 -- 172.21.15.148:0/2839845367 wait complete. 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.684+0000 7fe7b631b640 1 Processor -- start 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.684+0000 7fe7b631b640 1 -- start start 2024-04-25T09:57:53.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b631b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b631b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b631b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 0x7fe7b0160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b631b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00cf1c0 con 0x7fe7b015db70 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b631b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00ce530 con 0x7fe7b00b6cb0 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b631b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b00cf340 con 0x7fe7b00b5a60 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 0x7fe7b0160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 0x7fe7b0160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39200/0 (socket says 172.21.15.148:39200) 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 -- 172.21.15.148:0/2697317035 learned_addr learned my addr 172.21.15.148:0/2697317035 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7affff640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7af7fe640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 -- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 msgr2=0x7fe7b00ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 -- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b00b6cb0 msgr2=0x7fe7b00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:53.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 -- 172.21.15.148:0/2697317035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00cbc40 con 0x7fe7b015db70 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.685+0000 7fe7b48b8640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 0x7fe7b0160c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800cd70 tx=0x7fe7a800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.686+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8003b90 con 0x7fe7b015db70 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.686+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a8003d30 con 0x7fe7b015db70 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.686+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b00cbf20 con 0x7fe7b015db70 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.686+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8004240 con 0x7fe7b015db70 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.686+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b00cc4e0 con 0x7fe7b015db70 2024-04-25T09:57:53.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.687+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b00cef80 con 0x7fe7b015db70 2024-04-25T09:57:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.687+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7a80043e0 con 0x7fe7b015db70 2024-04-25T09:57:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.687+0000 7fe7acff9640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe79c06b4d0 0x7fe79c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.687+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe7a808ab80 con 0x7fe7b015db70 2024-04-25T09:57:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.687+0000 7fe7affff640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe79c06b4d0 0x7fe79c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:53.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.688+0000 7fe7affff640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe79c06b4d0 0x7fe79c06d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000f60 tx=0x7fe7a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:53.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.691+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7b00cef80 con 0x7fe7b015db70 2024-04-25T09:57:53.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:53.852 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:57:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:53.931+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fe7b00d1820 con 0x7fe7b015db70 2024-04-25T09:57:54.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.270+0000 7fe7acff9640 1 -- 172.21.15.148:0/2697317035 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v191) v1 ==== 143+0+0 (secure 0 0 0) 0x7fe7a801d340 con 0x7fe7b015db70 2024-04-25T09:57:54.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.272+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe79c06b4d0 msgr2=0x7fe79c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.272+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe79c06b4d0 0x7fe79c06d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000f60 tx=0x7fe7a0009040 comp rx=0 tx=0).stop 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.272+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 msgr2=0x7fe7b0160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.272+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 0x7fe7b0160c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800cd70 tx=0x7fe7a800a660 comp rx=0 tx=0).stop 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 shutdown_connections 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe79c06b4d0 0x7fe79c06d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7b015db70 0x7fe7b0160c80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b00b6cb0 0x7fe7b00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 --2- 172.21.15.148:0/2697317035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b00b5a60 0x7fe7b00ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 >> 172.21.15.148:0/2697317035 conn(0x7fe7b00d0520 msgr2=0x7fe7b00b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 shutdown_connections 2024-04-25T09:57:54.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.273+0000 7fe7b631b640 1 -- 172.21.15.148:0/2697317035 wait complete. 2024-04-25T09:57:54.288 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:57:54.289 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:57:54.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.423+0000 7fb82c1e0640 1 Processor -- start 2024-04-25T09:57:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.423+0000 7fb82c1e0640 1 -- start start 2024-04-25T09:57:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82c1e0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8240ce330 0x7fb82406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82c1e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8240700a0 0x7fb82415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82c1e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8241626e0 con 0x7fb82406abe0 2024-04-25T09:57:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82c1e0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8240cd270 con 0x7fb8240ce330 2024-04-25T09:57:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82c1e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240cd550 con 0x7fb8240700a0 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82977b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8240ce330 0x7fb82406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82977b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb8240ce330 0x7fb82406fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39204/0 (socket says 172.21.15.148:39204) 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.424+0000 7fb82977b640 1 -- 172.21.15.148:0/1302500519 learned_addr learned my addr 172.21.15.148:0/1302500519 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.425+0000 7fb82a77d640 1 --2- 172.21.15.148:0/1302500519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8240700a0 0x7fb82415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.425+0000 7fb82977b640 1 -- 172.21.15.148:0/1302500519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8240700a0 msgr2=0x7fb82415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.425+0000 7fb82977b640 1 --2- 172.21.15.148:0/1302500519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8240700a0 0x7fb82415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.425+0000 7fb82977b640 1 -- 172.21.15.148:0/1302500519 >> v1:172.21.15.188:6789/0 conn(0x7fb82406abe0 legacy=0x7fb82406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.425+0000 7fb82977b640 1 -- 172.21.15.148:0/1302500519 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8240cd830 con 0x7fb8240ce330 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.425+0000 7fb82977b640 1 --2- 172.21.15.148:0/1302500519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240ce330 0x7fb82406fb60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb814009870 tx=0x7fb81402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f651090cab63e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb828f7a640 1 -- 172.21.15.148:0/1302500519 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814037070 con 0x7fb8240ce330 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb828f7a640 1 -- 172.21.15.148:0/1302500519 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814032440 con 0x7fb8240ce330 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb828f7a640 1 -- 172.21.15.148:0/1302500519 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814032810 con 0x7fb8240ce330 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 -- 172.21.15.148:0/1302500519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240ce330 msgr2=0x7fb82406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/1302500519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240ce330 0x7fb82406fb60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb814009870 tx=0x7fb81402a180 comp rx=0 tx=0).stop 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 -- 172.21.15.148:0/1302500519 shutdown_connections 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/1302500519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240ce330 0x7fb82406fb60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/1302500519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8240700a0 0x7fb82415d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 -- 172.21.15.148:0/1302500519 >> 172.21.15.148:0/1302500519 conn(0x7fb8240b2220 msgr2=0x7fb8240b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 -- 172.21.15.148:0/1302500519 shutdown_connections 2024-04-25T09:57:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.426+0000 7fb82c1e0640 1 -- 172.21.15.148:0/1302500519 wait complete. 2024-04-25T09:57:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.427+0000 7fb82c1e0640 1 Processor -- start 2024-04-25T09:57:54.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.427+0000 7fb82c1e0640 1 -- start start 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82c1e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb82406abe0 0x7fb82409a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82c1e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 0x7fb8240b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82c1e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8240ce330 0x7fb8241f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82c1e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82415d8b0 con 0x7fb8240700a0 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82c1e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb824161780 con 0x7fb8240ce330 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82c1e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82415da30 con 0x7fb82406abe0 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82977b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 0x7fb8240b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82977b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 0x7fb8240b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39220/0 (socket says 172.21.15.148:39220) 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82977b640 1 -- 172.21.15.148:0/3355631483 learned_addr learned my addr 172.21.15.148:0/3355631483 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb829f7c640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb82406abe0 0x7fb82409a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.428+0000 7fb82a77d640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8240ce330 0x7fb8241f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82977b640 1 -- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb82406abe0 msgr2=0x7fb82409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82977b640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb82406abe0 0x7fb82409a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82977b640 1 -- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8240ce330 msgr2=0x7fb8241f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82977b640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8240ce330 0x7fb8241f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82977b640 1 -- 172.21.15.148:0/3355631483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82409b720 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb829f7c640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb82406abe0 0x7fb82409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82a77d640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8240ce330 0x7fb8241f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82977b640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 0x7fb8240b5730 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb81402a1c0 tx=0x7fb8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814037040 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814042070 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814030bd0 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82409b160 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.429+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240c8270 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.430+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb824073990 con 0x7fb8240700a0 2024-04-25T09:57:54.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.430+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb814047460 con 0x7fb8240700a0 2024-04-25T09:57:54.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.431+0000 7fb81a7fc640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb80806b500 0x7fb80806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.431+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb8140aad10 con 0x7fb8240700a0 2024-04-25T09:57:54.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.431+0000 7fb829f7c640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb80806b500 0x7fb80806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.431+0000 7fb829f7c640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb80806b500 0x7fb80806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb820003eb0 tx=0x7fb820009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:54.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.435+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb814079d00 con 0x7fb8240700a0 2024-04-25T09:57:54.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.607+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8240b2ff0 con 0x7fb8240700a0 2024-04-25T09:57:54.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.607+0000 7fb81a7fc640 1 -- 172.21.15.148:0/3355631483 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13343 (secure 0 0 0) 0x7fb814079600 con 0x7fb8240700a0 2024-04-25T09:57:54.611 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:54.611 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":58,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:53.261845+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/3309890912":"2024-04-26T09:57:50.036553+0000","172.21.15.148:6835/3309890912":"2024-04-26T09:57:50.036553+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb80806b500 msgr2=0x7fb80806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb80806b500 0x7fb80806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb820003eb0 tx=0x7fb820009040 comp rx=0 tx=0).stop 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 msgr2=0x7fb8240b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 0x7fb8240b5730 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb81402a1c0 tx=0x7fb8140046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 shutdown_connections 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb80806b500 0x7fb80806d9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8240ce330 0x7fb8241f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8240700a0 0x7fb8240b5730 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 --2- 172.21.15.148:0/3355631483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb82406abe0 0x7fb82409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 >> 172.21.15.148:0/3355631483 conn(0x7fb8240b2220 msgr2=0x7fb82415fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:54.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 shutdown_connections 2024-04-25T09:57:54.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.610+0000 7fb82c1e0640 1 -- 172.21.15.148:0/3355631483 wait complete. 2024-04-25T09:57:54.625 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:54.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:54.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.738+0000 7f4ed02f5640 1 Processor -- start 2024-04-25T09:57:54.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.738+0000 7f4ed02f5640 1 -- start start 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ed02f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ec815dc20 0x7f4ec80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ed02f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d32f0 0x7f4ec80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ed02f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec815e000 con 0x7f4ec8161710 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ed02f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec80d1de0 con 0x7f4ec815dc20 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ed02f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8161c60 con 0x7f4ec80d32f0 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ece091640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4ec815dc20 0x7f4ec80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ecd890640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4ec8161710 0x7f4ec806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46610/0 (socket says 172.21.15.148:46610) 2024-04-25T09:57:54.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ecd890640 1 -- 172.21.15.148:0/1234811834 learned_addr learned my addr 172.21.15.148:0/1234811834 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:54.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ece091640 1 -- 172.21.15.148:0/1234811834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d32f0 msgr2=0x7f4ec80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:54.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ece091640 1 --2- 172.21.15.148:0/1234811834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d32f0 0x7f4ec80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ece091640 1 -- 172.21.15.148:0/1234811834 >> v1:172.21.15.148:6789/0 conn(0x7f4ec8161710 legacy=0x7f4ec806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ece091640 1 -- 172.21.15.148:0/1234811834 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8161f40 con 0x7f4ec815dc20 2024-04-25T09:57:54.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.739+0000 7f4ecd08f640 1 -- 172.21.15.148:0/1234811834 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980345635 0 0) 0x7f4ec815e000 con 0x7f4ec8161710 2024-04-25T09:57:54.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.740+0000 7f4ece091640 1 --2- 172.21.15.148:0/1234811834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec80ce670 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00097a0 tx=0x7f4ec002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f151dcb1a59c697a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:54.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.740+0000 7f4ecd08f640 1 -- 172.21.15.148:0/1234811834 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0037070 con 0x7f4ec815dc20 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.740+0000 7f4ecd08f640 1 -- 172.21.15.148:0/1234811834 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec0032440 con 0x7f4ec815dc20 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.740+0000 7f4ecd08f640 1 -- 172.21.15.148:0/1234811834 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0032810 con 0x7f4ec815dc20 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 -- 172.21.15.148:0/1234811834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 msgr2=0x7f4ec80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/1234811834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec80ce670 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00097a0 tx=0x7f4ec002a180 comp rx=0 tx=0).stop 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 -- 172.21.15.148:0/1234811834 shutdown_connections 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/1234811834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec80d32f0 0x7f4ec80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/1234811834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec80ce670 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 -- 172.21.15.148:0/1234811834 >> 172.21.15.148:0/1234811834 conn(0x7f4ec80d0420 msgr2=0x7f4ec80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 -- 172.21.15.148:0/1234811834 shutdown_connections 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.741+0000 7f4ed02f5640 1 -- 172.21.15.148:0/1234811834 wait complete. 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.742+0000 7f4ed02f5640 1 Processor -- start 2024-04-25T09:57:54.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.742+0000 7f4ed02f5640 1 -- start start 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec80d32f0 0x7f4ec815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec81f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec8161710 0x7f4ec81e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80b5e50 con 0x7f4ec815dc20 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8162500 con 0x7f4ec80d32f0 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80b5fd0 con 0x7f4ec8161710 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec81f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec81f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39230/0 (socket says 172.21.15.148:39230) 2024-04-25T09:57:54.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 -- 172.21.15.148:0/4035899146 learned_addr learned my addr 172.21.15.148:0/4035899146 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ece892640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec8161710 0x7f4ec81e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ece091640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec80d32f0 0x7f4ec815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 -- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec8161710 msgr2=0x7f4ec81e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec8161710 0x7f4ec81e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 -- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec80d32f0 msgr2=0x7f4ec815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec80d32f0 0x7f4ec815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 -- 172.21.15.148:0/4035899146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80cac80 con 0x7f4ec815dc20 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ecd890640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec81f3e20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c460 tx=0x7f4ebc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc018070 con 0x7f4ec815dc20 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc007500 con 0x7f4ec815dc20 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec80caf60 con 0x7f4ec815dc20 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc004a20 con 0x7f4ec815dc20 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.743+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec81f4360 con 0x7f4ec815dc20 2024-04-25T09:57:54.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.744+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec8161450 con 0x7f4ec815dc20 2024-04-25T09:57:54.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.745+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ebc01b070 con 0x7f4ec815dc20 2024-04-25T09:57:54.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.745+0000 7f4ec6ffd640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4eb406b4d0 0x7f4eb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:54.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.745+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4ebc089360 con 0x7f4ec815dc20 2024-04-25T09:57:54.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.745+0000 7f4ece091640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4eb406b4d0 0x7f4eb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:54.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.746+0000 7f4ece091640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4eb406b4d0 0x7f4eb406d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0034000 tx=0x7f4ec002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:54.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.750+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ec8161450 con 0x7f4ec815dc20 2024-04-25T09:57:54.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.921+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ec8068370 con 0x7f4ec815dc20 2024-04-25T09:57:54.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.921+0000 7f4ec6ffd640 1 -- 172.21.15.148:0/4035899146 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ec8068370 con 0x7f4ec815dc20 2024-04-25T09:57:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4eb406b4d0 msgr2=0x7f4eb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4eb406b4d0 0x7f4eb406d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0034000 tx=0x7f4ec002e000 comp rx=0 tx=0).stop 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 msgr2=0x7f4ec81f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec81f3e20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c460 tx=0x7f4ebc00c920 comp rx=0 tx=0).stop 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 shutdown_connections 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4eb406b4d0 0x7f4eb406d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ec8161710 0x7f4ec81e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ec815dc20 0x7f4ec81f3e20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 --2- 172.21.15.148:0/4035899146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ec80d32f0 0x7f4ec815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 >> 172.21.15.148:0/4035899146 conn(0x7f4ec80d0420 msgr2=0x7f4ec80b8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 shutdown_connections 2024-04-25T09:57:54.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:54.923+0000 7f4ed02f5640 1 -- 172.21.15.148:0/4035899146 wait complete. 2024-04-25T09:57:54.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:57:55.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 Processor -- start 2024-04-25T09:57:55.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 -- start start 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca9c15dbe0 0x7fca9c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c0b92a0 0x7fca9c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c0b97a0 con 0x7fca9c0b2cb0 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c15dfc0 con 0x7fca9c15dbe0 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.031+0000 7fcaa1488640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0d1a00 con 0x7fca9c0b92a0 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca9c15dbe0 0x7fca9c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9a7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fca9c0b2cb0 0x7fca9c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46612/0 (socket says 172.21.15.148:46612) 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca9c15dbe0 0x7fca9c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39244/0 (socket says 172.21.15.148:39244) 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9a7fc640 1 -- 172.21.15.148:0/1777735016 learned_addr learned my addr 172.21.15.148:0/1777735016 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9b7fe640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c0b92a0 0x7fca9c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9affd640 1 -- 172.21.15.148:0/1777735016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c0b92a0 msgr2=0x7fca9c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9affd640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c0b92a0 0x7fca9c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:55.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9affd640 1 -- 172.21.15.148:0/1777735016 >> v1:172.21.15.148:6789/0 conn(0x7fca9c0b2cb0 legacy=0x7fca9c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca99ffb640 1 -- 172.21.15.148:0/1777735016 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308524084 0 0) 0x7fca9c0b97a0 con 0x7fca9c0b2cb0 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9affd640 1 -- 172.21.15.148:0/1777735016 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c0d1c50 con 0x7fca9c15dbe0 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.032+0000 7fca9b7fe640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c0b92a0 0x7fca9c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fca9affd640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c15dbe0 0x7fca9c0d0910 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097a0 tx=0x7fca8c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a991b5a69604a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fca99ffb640 1 -- 172.21.15.148:0/1777735016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c037070 con 0x7fca9c15dbe0 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fca99ffb640 1 -- 172.21.15.148:0/1777735016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c032440 con 0x7fca9c15dbe0 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fca99ffb640 1 -- 172.21.15.148:0/1777735016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c032810 con 0x7fca9c15dbe0 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 -- 172.21.15.148:0/1777735016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c15dbe0 msgr2=0x7fca9c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c15dbe0 0x7fca9c0d0910 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0097a0 tx=0x7fca8c02a180 comp rx=0 tx=0).stop 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 -- 172.21.15.148:0/1777735016 shutdown_connections 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c0b92a0 0x7fca9c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 --2- 172.21.15.148:0/1777735016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c15dbe0 0x7fca9c0d0910 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:55.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 -- 172.21.15.148:0/1777735016 >> 172.21.15.148:0/1777735016 conn(0x7fca9c061a20 msgr2=0x7fca9c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 -- 172.21.15.148:0/1777735016 shutdown_connections 2024-04-25T09:57:55.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.033+0000 7fcaa1488640 1 -- 172.21.15.148:0/1777735016 wait complete. 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 Processor -- start 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 -- start start 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9c0b2cb0 0x7fca9c0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 0x7fca9c152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c15dbe0 0x7fca9c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0b6ab0 con 0x7fca9c0b92a0 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0d2210 con 0x7fca9c0b2cb0 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fcaa1488640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0b6c30 con 0x7fca9c15dbe0 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 0x7fca9c152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fca9a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 0x7fca9c152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39250/0 (socket says 172.21.15.148:39250) 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.035+0000 7fca9a7fc640 1 -- 172.21.15.148:0/557624141 learned_addr learned my addr 172.21.15.148:0/557624141 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9b7fe640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c15dbe0 0x7fca9c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:55.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9a7fc640 1 -- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c15dbe0 msgr2=0x7fca9c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9a7fc640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c15dbe0 0x7fca9c0cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9a7fc640 1 -- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9c0b2cb0 msgr2=0x7fca9c0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9a7fc640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9c0b2cb0 0x7fca9c0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9a7fc640 1 -- 172.21.15.148:0/557624141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c0b6030 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca9a7fc640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 0x7fca9c152000 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c460 tx=0x7fca9000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90018070 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90007520 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c0b4f90 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90004a20 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.036+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c0b5550 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.037+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c073990 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.037+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca9001b070 con 0x7fca9c0b92a0 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.038+0000 7fca837fe640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca8806b4d0 0x7fca8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:55.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.038+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fca90089480 con 0x7fca9c0b92a0 2024-04-25T09:57:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.038+0000 7fca9affd640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca8806b4d0 0x7fca8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.038+0000 7fca9affd640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca8806b4d0 0x7fca8806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca8c034000 tx=0x7fca8c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:55.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.042+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca90058470 con 0x7fca9c0b92a0 2024-04-25T09:57:55.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:55.278+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fca9c0d32e0 con 0x7fca9c0b92a0 2024-04-25T09:57:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.276+0000 7fca837fe640 1 -- 172.21.15.148:0/557624141 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v193) v1 ==== 146+0+0 (secure 0 0 0) 0x7fca90057d70 con 0x7fca9c0b92a0 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca8806b4d0 msgr2=0x7fca8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca8806b4d0 0x7fca8806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca8c034000 tx=0x7fca8c02c000 comp rx=0 tx=0).stop 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 msgr2=0x7fca9c152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 0x7fca9c152000 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c460 tx=0x7fca9000c920 comp rx=0 tx=0).stop 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 shutdown_connections 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca8806b4d0 0x7fca8806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9c15dbe0 0x7fca9c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca9c0b92a0 0x7fca9c152000 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 --2- 172.21.15.148:0/557624141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9c0b2cb0 0x7fca9c0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 >> 172.21.15.148:0/557624141 conn(0x7fca9c061a20 msgr2=0x7fca9c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 shutdown_connections 2024-04-25T09:57:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.279+0000 7fcaa1488640 1 -- 172.21.15.148:0/557624141 wait complete. 2024-04-25T09:57:56.296 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:57:56.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.398+0000 7f859fa5d640 1 Processor -- start 2024-04-25T09:57:56.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.398+0000 7f859fa5d640 1 -- start start 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859fa5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f859806fbd0 0x7f85980ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859fa5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85980ce730 0x7f85980b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859fa5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85980b4d80 con 0x7f8598155f80 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859fa5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85980b5390 con 0x7f85980ce730 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859fa5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85980d07f0 con 0x7f859806fbd0 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859dffa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85980ce730 0x7f85980b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859cff8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f859806fbd0 0x7f85980ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859d7f9640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8598155f80 0x7f859806f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:37926/0 (socket says 172.21.15.148:37926) 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859d7f9640 1 -- 172.21.15.148:0/3034227938 learned_addr learned my addr 172.21.15.148:0/3034227938 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859dffa640 1 -- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f859806fbd0 msgr2=0x7f85980ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859dffa640 1 --2- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f859806fbd0 0x7f85980ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859dffa640 1 -- 172.21.15.148:0/3034227938 >> v1:172.21.15.188:6789/0 conn(0x7f8598155f80 legacy=0x7f859806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859dffa640 1 -- 172.21.15.148:0/3034227938 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85980cef70 con 0x7f85980ce730 2024-04-25T09:57:56.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.399+0000 7f859cff8640 1 --2- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f859806fbd0 0x7f85980ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.400+0000 7f859dffa640 1 --2- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85980ce730 0x7f85980b5820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85980b4f50 tx=0x7f858802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c848cc5925a6224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.400+0000 7f8597fff640 1 -- 172.21.15.148:0/3034227938 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588037070 con 0x7f85980ce730 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.400+0000 7f8597fff640 1 -- 172.21.15.148:0/3034227938 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588032440 con 0x7f85980ce730 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.400+0000 7f8597fff640 1 -- 172.21.15.148:0/3034227938 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588032810 con 0x7f85980ce730 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 -- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85980ce730 msgr2=0x7f85980b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 --2- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85980ce730 0x7f85980b5820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85980b4f50 tx=0x7f858802a180 comp rx=0 tx=0).stop 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 -- 172.21.15.148:0/3034227938 shutdown_connections 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 --2- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85980ce730 0x7f85980b5820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 --2- 172.21.15.148:0/3034227938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f859806fbd0 0x7f85980ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 -- 172.21.15.148:0/3034227938 >> 172.21.15.148:0/3034227938 conn(0x7f8598061600 msgr2=0x7f85980619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 -- 172.21.15.148:0/3034227938 shutdown_connections 2024-04-25T09:57:56.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.401+0000 7f859fa5d640 1 -- 172.21.15.148:0/3034227938 wait complete. 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.402+0000 7f859fa5d640 1 Processor -- start 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.402+0000 7f859fa5d640 1 -- start start 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859fa5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859806fbd0 0x7f85981f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859fa5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85980ce730 0x7f8598151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859fa5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 0x7f85981515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859fa5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85980cf250 con 0x7f8598155f80 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859fa5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598156360 con 0x7f859806fbd0 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859fa5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85980cf3d0 con 0x7f85980ce730 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 0x7f85981515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 0x7f85981515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39254/0 (socket says 172.21.15.148:39254) 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 -- 172.21.15.148:0/894263845 learned_addr learned my addr 172.21.15.148:0/894263845 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859d7f9640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859806fbd0 0x7f85981f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859cff8640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85980ce730 0x7f8598151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 -- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85980ce730 msgr2=0x7f8598151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85980ce730 0x7f8598151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 -- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859806fbd0 msgr2=0x7f85981f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859806fbd0 0x7f85981f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.403+0000 7f859dffa640 1 -- 172.21.15.148:0/894263845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859814c740 con 0x7f8598155f80 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.404+0000 7f859dffa640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 0x7f85981515b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8588035040 tx=0x7f85880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.404+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588037040 con 0x7f8598155f80 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.404+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588042070 con 0x7f8598155f80 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.404+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880308a0 con 0x7f8598155f80 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.404+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859814ca20 con 0x7f8598155f80 2024-04-25T09:57:56.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.404+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859814cfe0 con 0x7f8598155f80 2024-04-25T09:57:56.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.405+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8598073a00 con 0x7f8598155f80 2024-04-25T09:57:56.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.405+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8588030a40 con 0x7f8598155f80 2024-04-25T09:57:56.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.406+0000 7f8595ffb640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f858406b4d0 0x7f858406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.406+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f85880aa7c0 con 0x7f8598155f80 2024-04-25T09:57:56.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.406+0000 7f859d7f9640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f858406b4d0 0x7f858406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.406+0000 7f859d7f9640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f858406b4d0 0x7f858406d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f85900012e0 tx=0x7f8590001990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.410+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85880791f0 con 0x7f8598155f80 2024-04-25T09:57:56.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.579+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f85981f2260 con 0x7f8598155f80 2024-04-25T09:57:56.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.583+0000 7f8595ffb640 1 -- 172.21.15.148:0/894263845 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f858807f070 con 0x7f8598155f80 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f858406b4d0 msgr2=0x7f858406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f858406b4d0 0x7f858406d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f85900012e0 tx=0x7f8590001990 comp rx=0 tx=0).stop 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 msgr2=0x7f85981515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 0x7f85981515b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8588035040 tx=0x7f85880046c0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 shutdown_connections 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f858406b4d0 0x7f858406d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8598155f80 0x7f85981515b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85980ce730 0x7f8598151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 --2- 172.21.15.148:0/894263845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859806fbd0 0x7f85981f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 >> 172.21.15.148:0/894263845 conn(0x7f8598061600 msgr2=0x7f859806f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 shutdown_connections 2024-04-25T09:57:56.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.585+0000 7f859fa5d640 1 -- 172.21.15.148:0/894263845 wait complete. 2024-04-25T09:57:56.599 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:57:56.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:56.654 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:57:56.702 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:57:56.702 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:56.702 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:56.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:56.717 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:57:56.718 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:56.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:56.735 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:57:56.735 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:56.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:57:56.772 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:57:56.772 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:57:56.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:57:56.792 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:57:56.792 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:57:56.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.907+0000 7f6aa3f1d640 1 Processor -- start 2024-04-25T09:57:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.907+0000 7f6aa3f1d640 1 -- start start 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa3f1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a9c0d1f20 0x7f6a9c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa3f1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c0cce50 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa3f1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c161af0 con 0x7f6a9c15dc90 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa3f1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a9c161dd0 con 0x7f6a9c0d1f20 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa24ba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a9c0d1f20 0x7f6a9c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa14b8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6a9c0cc8e0 0x7f6a9c0d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46620/0 (socket says 172.21.15.148:46620) 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa24ba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a9c0d1f20 0x7f6a9c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39268/0 (socket says 172.21.15.148:39268) 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa14b8640 1 -- 172.21.15.148:0/900987481 learned_addr learned my addr 172.21.15.148:0/900987481 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa24ba640 1 -- 172.21.15.148:0/900987481 >> v1:172.21.15.188:6789/0 conn(0x7f6a9c15dc90 legacy=0x7f6a9c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa24ba640 1 -- 172.21.15.148:0/900987481 >> v1:172.21.15.148:6789/0 conn(0x7f6a9c0cc8e0 legacy=0x7f6a9c0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa24ba640 1 -- 172.21.15.148:0/900987481 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c1620b0 con 0x7f6a9c0d1f20 2024-04-25T09:57:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.908+0000 7f6aa0cb7640 1 -- 172.21.15.148:0/900987481 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799291993 0 0) 0x7f6a9c0cce50 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.909+0000 7f6aa24ba640 1 --2- 172.21.15.148:0/900987481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c1625d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009d90 tx=0x7f6a9402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aca5838e9a46143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.909+0000 7f6aa0cb7640 1 -- 172.21.15.148:0/900987481 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94031600 con 0x7f6a9c0d1f20 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.909+0000 7f6aa0cb7640 1 -- 172.21.15.148:0/900987481 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a940317a0 con 0x7f6a9c0d1f20 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.909+0000 7f6aa0cb7640 1 -- 172.21.15.148:0/900987481 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94031b70 con 0x7f6a9c0d1f20 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/900987481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0d1f20 msgr2=0x7f6a9c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/900987481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c1625d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009d90 tx=0x7f6a9402a610 comp rx=0 tx=0).stop 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/900987481 shutdown_connections 2024-04-25T09:57:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/900987481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c1625d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/900987481 >> 172.21.15.148:0/900987481 conn(0x7f6a9c06f590 msgr2=0x7f6a9c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/900987481 shutdown_connections 2024-04-25T09:57:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.910+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/900987481 wait complete. 2024-04-25T09:57:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.911+0000 7f6aa3f1d640 1 Processor -- start 2024-04-25T09:57:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.911+0000 7f6aa3f1d640 1 -- start start 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.911+0000 7f6aa3f1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 0x7f6a9c1f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c15dc90 0x7f6a9c15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c15e1f0 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c15e070 con 0x7f6a9c0d1f20 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cfde0 con 0x7f6a9c15dc90 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 0x7f6a9c1f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 0x7f6a9c1f7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39274/0 (socket says 172.21.15.148:39274) 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 -- 172.21.15.148:0/3308448848 learned_addr learned my addr 172.21.15.148:0/3308448848 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa14b8640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa24ba640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c15dc90 0x7f6a9c15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 -- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c15dc90 msgr2=0x7f6a9c15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c15dc90 0x7f6a9c15c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 -- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c0d1f20 msgr2=0x7f6a9c0cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c0cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 -- 172.21.15.148:0/3308448848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0cb320 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa1cb9640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 0x7f6a9c1f7f50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b410 tx=0x7f6a9800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98015020 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98011830 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c0c89f0 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011c20 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.912+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c0c8fb0 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.913+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c073990 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.914+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a98018020 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.914+0000 7f6a927fc640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a8006b4d0 0x7f6a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:56.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.914+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a98089fe0 con 0x7f6a9c0cc8e0 2024-04-25T09:57:56.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.914+0000 7f6aa14b8640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a8006b4d0 0x7f6a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:56.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.914+0000 7f6aa14b8640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a8006b4d0 0x7f6a8006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0b3260 tx=0x7f6a8c007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:56.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:56.918+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a9801f020 con 0x7f6a9c0cc8e0 2024-04-25T09:57:57.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.125+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a9c0cf6b0 con 0x7f6a9c0cc8e0 2024-04-25T09:57:57.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.126+0000 7f6a927fc640 1 -- 172.21.15.148:0/3308448848 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f6a980172c0 con 0x7f6a9c0cc8e0 2024-04-25T09:57:57.130 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:57.130 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4068832564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4068832564},{"type":"v1","addr":"172.21.15.188:6834","nonce":4068832564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5042,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2988726277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2988726277},{"type":"v1","addr":"172.21.15.188:6835","nonce":2988726277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2051204748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2051204748},{"type":"v1","addr":"172.21.15.148:6836","nonce":2051204748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6648,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1061633518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1061633518},{"type":"v1","addr":"172.21.15.148:6837","nonce":1061633518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-04-25T09:57:57.130 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 29 2024-04-25T09:57:57.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.127+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a8006b4d0 msgr2=0x7f6a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.127+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a8006b4d0 0x7f6a8006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0b3260 tx=0x7f6a8c007b50 comp rx=0 tx=0).stop 2024-04-25T09:57:57.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.127+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 msgr2=0x7f6a9c1f7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.127+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 0x7f6a9c1f7f50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b410 tx=0x7f6a9800b8d0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 shutdown_connections 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a8006b4d0 0x7f6a8006d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c15dc90 0x7f6a9c15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c0d1f20 0x7f6a9c0cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 --2- 172.21.15.148:0/3308448848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a9c0cc8e0 0x7f6a9c1f7f50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 >> 172.21.15.148:0/3308448848 conn(0x7f6a9c06f590 msgr2=0x7f6a9c06f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 shutdown_connections 2024-04-25T09:57:57.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.128+0000 7f6aa3f1d640 1 -- 172.21.15.148:0/3308448848 wait complete. 2024-04-25T09:57:57.142 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:57:57.142 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:57:57.143 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:57:57.143 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:57.143 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:57:57.143 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:57:57.143 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:57:57.143 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:57.144 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:57:57.144 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:57:57.144 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:57:57.144 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:57.144 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:57:57.144 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:57:57.144 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:57:57.144 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:57:57.154 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:57:57.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:57:57.195 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:57:57.195 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:57:57.196 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:57:57.196 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:57:57.197 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:57:57.197 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:57:57.237 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:57:57.238 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:57.239 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:57:57.239 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:57.240 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:57:57.240 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:57.241 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:57:57.241 DEBUG:teuthology.parallel:result is None 2024-04-25T09:57:57.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:57:57.311 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:57:57.311 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:57:57.321 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T09:57:57.321 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:57:57.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.340+0000 7f022119b640 1 Processor -- start 2024-04-25T09:57:57.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.340+0000 7f022119b640 1 -- start start 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f022119b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f021c0c6010 0x7f021c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f022119b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c0c6940 con 0x7f021c15a9f0 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f022119b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c15add0 con 0x7f021c0cd520 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f022119b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c0b0020 con 0x7f021c0c6010 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f021ad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f021c15a9f0 0x7f021c0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46628/0 (socket says 172.21.15.148:46628) 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f021ad9d640 1 -- 172.21.15.148:0/1538098373 learned_addr learned my addr 172.21.15.148:0/1538098373 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.341+0000 7f021a59c640 1 --2- 172.21.15.148:0/1538098373 >> v2:172.21.15.188:3300/0 conn(0x7f021c0c6010 0x7f021c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184670344 0 0) 0x7f020c001110 con 0x7f021c15a9f0 2024-04-25T09:57:57.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0204003520 con 0x7f021c15a9f0 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549072612 0 0) 0x7f021c15add0 con 0x7f021c0cd520 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f020c001110 con 0x7f021c0cd520 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3838190055 0 0) 0x7f0204003520 con 0x7f021c15a9f0 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f021a59c640 1 -- 172.21.15.148:0/1538098373 >> v1:172.21.15.188:6789/0 conn(0x7f021c0cd520 legacy=0x7f021c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f021a59c640 1 -- 172.21.15.148:0/1538098373 >> v1:172.21.15.148:6789/0 conn(0x7f021c15a9f0 legacy=0x7f021c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.342+0000 7f021a59c640 1 -- 172.21.15.148:0/1538098373 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0c6660 con 0x7f021c0c6010 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f021a59c640 1 --2- 172.21.15.148:0/1538098373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c0ccfe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f02100097a0 tx=0x7f021002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4db2366ca78656b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210037070 con 0x7f021c0c6010 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021002ac50 con 0x7f021c0c6010 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f0219d9b640 1 -- 172.21.15.148:0/1538098373 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c001840 con 0x7f021c0c6010 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 -- 172.21.15.148:0/1538098373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 msgr2=0x7f021c0ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 --2- 172.21.15.148:0/1538098373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c0ccfe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f02100097a0 tx=0x7f021002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 -- 172.21.15.148:0/1538098373 shutdown_connections 2024-04-25T09:57:57.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 --2- 172.21.15.148:0/1538098373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c0ccfe0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 -- 172.21.15.148:0/1538098373 >> 172.21.15.148:0/1538098373 conn(0x7f021c0b2390 msgr2=0x7f021c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:57.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 -- 172.21.15.148:0/1538098373 shutdown_connections 2024-04-25T09:57:57.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.343+0000 7f022119b640 1 -- 172.21.15.148:0/1538098373 wait complete. 2024-04-25T09:57:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.345+0000 7f022119b640 1 Processor -- start 2024-04-25T09:57:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.345+0000 7f022119b640 1 -- start start 2024-04-25T09:57:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f021c0cd520 0x7f021c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 0x7f021c152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0cb230 con 0x7f021c15a9f0 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0cb5a0 con 0x7f021c0c6010 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0cc140 con 0x7f021c0cd520 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 0x7f021c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 0x7f021c152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39308/0 (socket says 172.21.15.148:39308) 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 -- 172.21.15.148:0/1976318010 learned_addr learned my addr 172.21.15.148:0/1976318010 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021a59c640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f021c0cd520 0x7f021c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021ad9d640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 -- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f021c0cd520 msgr2=0x7f021c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f021c0cd520 0x7f021c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 -- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 msgr2=0x7f021c159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 -- 172.21.15.148:0/1976318010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0b6330 con 0x7f021c15a9f0 2024-04-25T09:57:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f021b59e640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 0x7f021c152300 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f021400c7a0 tx=0x7f021400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214003b70 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214003d30 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c0b3bd0 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.346+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400aa90 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.347+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c0b4190 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.347+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c06cc80 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.348+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f021400ac30 con 0x7f021c15a9f0 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.348+0000 7f02037fe640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.349+0000 7f021ad9d640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:57.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.349+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0214089e80 con 0x7f021c15a9f0 2024-04-25T09:57:57.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.349+0000 7f021ad9d640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f020806b4d0 0x7f020806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f020c001b90 tx=0x7f020c001b20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:57.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.353+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f021c0b6330 con 0x7f021c15a9f0 2024-04-25T09:57:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:57.535+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f021c0739e0 con 0x7f021c15a9f0 2024-04-25T09:57:58.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.604+0000 7f02037fe640 1 -- 172.21.15.148:0/1976318010 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0214058820 con 0x7f021c15a9f0 2024-04-25T09:57:58.608 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f020806b4d0 msgr2=0x7f020806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f020806b4d0 0x7f020806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f020c001b90 tx=0x7f020c001b20 comp rx=0 tx=0).stop 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 msgr2=0x7f021c152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 0x7f021c152300 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f021400c7a0 tx=0x7f021400cc60 comp rx=0 tx=0).stop 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 shutdown_connections 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c15a9f0 0x7f021c152300 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f021c0cd520 0x7f021c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 --2- 172.21.15.148:0/1976318010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f021c0c6010 0x7f021c159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 >> 172.21.15.148:0/1976318010 conn(0x7f021c0b2390 msgr2=0x7f021c0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:58.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 shutdown_connections 2024-04-25T09:57:58.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.608+0000 7f022119b640 1 -- 172.21.15.148:0/1976318010 wait complete. 2024-04-25T09:57:58.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:57:58.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.724+0000 7efc2c59e640 1 Processor -- start 2024-04-25T09:57:58.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.724+0000 7efc2c59e640 1 -- start start 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc2c59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efc2415a790 0x7efc240d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc2c59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc2406fea0 0x7efc240d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc2c59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc24072300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc2c59e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc240b3710 con 0x7efc2406fea0 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc2c59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc240cf0e0 con 0x7efc2415a790 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc2c59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc240cf3c0 con 0x7efc240af9c0 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.725+0000 7efc29b39640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc2406fea0 0x7efc240d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc2406fea0 0x7efc240d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39312/0 (socket says 172.21.15.148:39312) 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 -- 172.21.15.148:0/1390716071 learned_addr learned my addr 172.21.15.148:0/1390716071 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc2a33a640 1 --2- 172.21.15.148:0/1390716071 >> v2:172.21.15.188:3300/0 conn(0x7efc2415a790 0x7efc240d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc2ab3b640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc24072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 -- 172.21.15.148:0/1390716071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 msgr2=0x7efc24072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc24072300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 -- 172.21.15.148:0/1390716071 >> v2:172.21.15.188:3300/0 conn(0x7efc2415a790 msgr2=0x7efc240d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 --2- 172.21.15.148:0/1390716071 >> v2:172.21.15.188:3300/0 conn(0x7efc2415a790 0x7efc240d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc29b39640 1 -- 172.21.15.148:0/1390716071 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc240cf6a0 con 0x7efc2406fea0 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.726+0000 7efc2ab3b640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc24072300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc29b39640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc240d08d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7efc240b2e90 tx=0x7efc1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385d2ab1f7c82a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc29338640 1 -- 172.21.15.148:0/1390716071 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14037070 con 0x7efc2406fea0 2024-04-25T09:57:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc29338640 1 -- 172.21.15.148:0/1390716071 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14032440 con 0x7efc2406fea0 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc29338640 1 -- 172.21.15.148:0/1390716071 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14032810 con 0x7efc2406fea0 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 -- 172.21.15.148:0/1390716071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 msgr2=0x7efc240d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc240d08d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7efc240b2e90 tx=0x7efc1402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 -- 172.21.15.148:0/1390716071 shutdown_connections 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc24072300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 --2- 172.21.15.148:0/1390716071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc240d08d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 --2- 172.21.15.148:0/1390716071 >> v2:172.21.15.188:3300/0 conn(0x7efc2415a790 0x7efc240d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 -- 172.21.15.148:0/1390716071 >> 172.21.15.148:0/1390716071 conn(0x7efc240b2370 msgr2=0x7efc240b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 -- 172.21.15.148:0/1390716071 shutdown_connections 2024-04-25T09:57:58.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.727+0000 7efc2c59e640 1 -- 172.21.15.148:0/1390716071 wait complete. 2024-04-25T09:57:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 Processor -- start 2024-04-25T09:57:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 -- start start 2024-04-25T09:57:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc2415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc241eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415a790 0x7efc24158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2415ab70 con 0x7efc2406fea0 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240cf980 con 0x7efc2415a790 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2c59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2415acf0 con 0x7efc240af9c0 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2a33a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc2415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2a33a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc2415bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39316/0 (socket says 172.21.15.148:39316) 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.729+0000 7efc2a33a640 1 -- 172.21.15.148:0/711442437 learned_addr learned my addr 172.21.15.148:0/711442437 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2ab3b640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415a790 0x7efc24158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc29b39640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc241eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2a33a640 1 -- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 msgr2=0x7efc241eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2a33a640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc241eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2a33a640 1 -- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415a790 msgr2=0x7efc24158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2a33a640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415a790 0x7efc24158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2a33a640 1 -- 172.21.15.148:0/711442437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc241f3980 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2ab3b640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415a790 0x7efc24158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2a33a640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc2415bb60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e870 tx=0x7efc2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20017070 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2000c940 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2415b080 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2000cd30 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.730+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc241ef3f0 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.731+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24068700 con 0x7efc2406fea0 2024-04-25T09:57:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.731+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc2001a070 con 0x7efc2406fea0 2024-04-25T09:57:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.732+0000 7efc1affd640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0806b4d0 0x7efc0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.732+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efc2008a230 con 0x7efc2406fea0 2024-04-25T09:57:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.732+0000 7efc29b39640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0806b4d0 0x7efc0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.732+0000 7efc29b39640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0806b4d0 0x7efc0806d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc1402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:58.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.736+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc24068700 con 0x7efc2406fea0 2024-04-25T09:57:58.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.968+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efc2415ae70 con 0x7efc2406fea0 2024-04-25T09:57:58.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.969+0000 7efc1affd640 1 -- 172.21.15.148:0/711442437 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efc20059300 con 0x7efc2406fea0 2024-04-25T09:57:58.972 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:58.972 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.973 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.974 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:58.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T09:57:58.978 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.979 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T09:57:58.980 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.981 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.982 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:57:58.983 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T09:57:58.984 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:57:58.985 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T09:57:58.988 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.970+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0806b4d0 msgr2=0x7efc0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.970+0000 7efc2c59e640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0806b4d0 0x7efc0806d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc1402e000 comp rx=0 tx=0).stop 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.970+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 msgr2=0x7efc2415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.970+0000 7efc2c59e640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc2415bb60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e870 tx=0x7efc2000ed30 comp rx=0 tx=0).stop 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 shutdown_connections 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0806b4d0 0x7efc0806d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415a790 0x7efc24158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240af9c0 0x7efc241eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 --2- 172.21.15.148:0/711442437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc2406fea0 0x7efc2415bb60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:58.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 >> 172.21.15.148:0/711442437 conn(0x7efc240b2370 msgr2=0x7efc24156170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:58.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 shutdown_connections 2024-04-25T09:57:58.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:58.971+0000 7efc2c59e640 1 -- 172.21.15.148:0/711442437 wait complete. 2024-04-25T09:57:58.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:57:59.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.086+0000 7fb1cac5d640 1 Processor -- start 2024-04-25T09:57:59.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.086+0000 7fb1cac5d640 1 -- start start 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1cac5d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1c40cc8e0 0x7fb1c41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1cac5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c40d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1cac5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40cce50 con 0x7fb1c415dc90 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1cac5d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c40d21c0 con 0x7fb1c40cc8e0 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1cac5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4161af0 con 0x7fb1c40d1810 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb1c40cc8e0 0x7fb1c41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c89f9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb1c415dc90 0x7fb1c415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46644/0 (socket says 172.21.15.148:46644) 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c89f9640 1 -- 172.21.15.148:0/1720587764 learned_addr learned my addr 172.21.15.148:0/1720587764 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c3fff640 1 -- 172.21.15.148:0/1720587764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 msgr2=0x7fb1c40d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c3fff640 1 --2- 172.21.15.148:0/1720587764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c40d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c3fff640 1 -- 172.21.15.148:0/1720587764 >> v1:172.21.15.148:6789/0 conn(0x7fb1c415dc90 legacy=0x7fb1c415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c3fff640 1 -- 172.21.15.148:0/1720587764 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4161dd0 con 0x7fb1c40cc8e0 2024-04-25T09:57:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.087+0000 7fb1c37fe640 1 -- 172.21.15.148:0/1720587764 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192314330 0 0) 0x7fb1c40cce50 con 0x7fb1c415dc90 2024-04-25T09:57:59.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.088+0000 7fb1c3fff640 1 --2- 172.21.15.148:0/1720587764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c41625d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80097a0 tx=0x7fb1b802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6513382c0f94d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.088+0000 7fb1c37fe640 1 -- 172.21.15.148:0/1720587764 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8037070 con 0x7fb1c40cc8e0 2024-04-25T09:57:59.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.088+0000 7fb1c37fe640 1 -- 172.21.15.148:0/1720587764 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b802ae50 con 0x7fb1c40cc8e0 2024-04-25T09:57:59.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.088+0000 7fb1c37fe640 1 -- 172.21.15.148:0/1720587764 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8032660 con 0x7fb1c40cc8e0 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 -- 172.21.15.148:0/1720587764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c40cc8e0 msgr2=0x7fb1c41625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/1720587764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c41625d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80097a0 tx=0x7fb1b802a100 comp rx=0 tx=0).stop 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 -- 172.21.15.148:0/1720587764 shutdown_connections 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/1720587764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c40d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/1720587764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c41625d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 -- 172.21.15.148:0/1720587764 >> 172.21.15.148:0/1720587764 conn(0x7fb1c406f590 msgr2=0x7fb1c406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 -- 172.21.15.148:0/1720587764 shutdown_connections 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.089+0000 7fb1cac5d640 1 -- 172.21.15.148:0/1720587764 wait complete. 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.090+0000 7fb1cac5d640 1 Processor -- start 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.090+0000 7fb1cac5d640 1 -- start start 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1cac5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c40ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1cac5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c41f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1cac5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 0x7fb1c415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1cac5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415e070 con 0x7fb1c415dc90 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1cac5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40cd130 con 0x7fb1c40cc8e0 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1cac5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415e1f0 con 0x7fb1c40d1810 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 0x7fb1c415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 0x7fb1c415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39330/0 (socket says 172.21.15.148:39330) 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 -- 172.21.15.148:0/3109496116 learned_addr learned my addr 172.21.15.148:0/3109496116 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c3fff640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c41f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c89f9640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c40ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 -- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 msgr2=0x7fb1c41f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c41f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 -- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1c40cc8e0 msgr2=0x7fb1c40ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c40ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c91fa640 1 -- 172.21.15.148:0/3109496116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c41f4450 con 0x7fb1c415dc90 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c89f9640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c40ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.091+0000 7fb1c3fff640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c41f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.092+0000 7fb1c91fa640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 0x7fb1c415c6a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00b9e0 tx=0x7fb1bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.092+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc019070 con 0x7fb1c415dc90 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.092+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc00cac0 con 0x7fb1c415dc90 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.092+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c41f4730 con 0x7fb1c415dc90 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.092+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc014440 con 0x7fb1c415dc90 2024-04-25T09:57:59.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.092+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c41f4fd0 con 0x7fb1c415dc90 2024-04-25T09:57:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.093+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c41f4900 con 0x7fb1c415dc90 2024-04-25T09:57:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.093+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1bc01c070 con 0x7fb1c415dc90 2024-04-25T09:57:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.093+0000 7fb1c17fa640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.094+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb1bc089c80 con 0x7fb1c415dc90 2024-04-25T09:57:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.094+0000 7fb1c89f9640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.094+0000 7fb1c89f9640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb1b006b4d0 0x7fb1b006d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4000f60 tx=0x7fb1b4007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.098+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1bc058d50 con 0x7fb1c415dc90 2024-04-25T09:57:59.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.271+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c4073990 con 0x7fb1c415dc90 2024-04-25T09:57:59.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.272+0000 7fb1c17fa640 1 -- 172.21.15.148:0/3109496116 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13212 (secure 0 0 0) 0x7fb1bc058650 con 0x7fb1c415dc90 2024-04-25T09:57:59.276 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:59.276 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":63,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:57:58.603421+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb1b006b4d0 msgr2=0x7fb1b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb1b006b4d0 0x7fb1b006d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4000f60 tx=0x7fb1b4007c10 comp rx=0 tx=0).stop 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 msgr2=0x7fb1c415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 0x7fb1c415c6a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00b9e0 tx=0x7fb1bc00bea0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 shutdown_connections 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1c415dc90 0x7fb1c415c6a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1c40d1810 0x7fb1c41f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 --2- 172.21.15.148:0/3109496116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1c40cc8e0 0x7fb1c40ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 >> 172.21.15.148:0/3109496116 conn(0x7fb1c406f590 msgr2=0x7fb1c4160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 shutdown_connections 2024-04-25T09:57:59.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.274+0000 7fb1cac5d640 1 -- 172.21.15.148:0/3109496116 wait complete. 2024-04-25T09:57:59.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:57:59.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:57:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.385+0000 7fb0cf7a5640 1 Processor -- start 2024-04-25T09:57:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.385+0000 7fb0cf7a5640 1 -- start start 2024-04-25T09:57:59.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cf7a5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb0c80b5570 0x7fb0c806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cf7a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c80d1d50 con 0x7fb0c8158230 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cf7a5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c80d2020 con 0x7fb0c806c330 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cf7a5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c8158610 con 0x7fb0c80b5570 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cd541640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb0c8158230 0x7fb0c80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46652/0 (socket says 172.21.15.148:46652) 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cd541640 1 -- 172.21.15.148:0/2138681157 learned_addr learned my addr 172.21.15.148:0/2138681157 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0cdd42640 1 --2- 172.21.15.148:0/2138681157 >> v2:172.21.15.188:3300/0 conn(0x7fb0c80b5570 0x7fb0c806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.386+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945742983 0 0) 0x7fb0c80d1d50 con 0x7fb0c8158230 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0b4003520 con 0x7fb0c8158230 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047248411 0 0) 0x7fb0b4003520 con 0x7fb0c8158230 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 >> v2:172.21.15.188:3300/0 conn(0x7fb0c80b5570 msgr2=0x7fb0c806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 --2- 172.21.15.148:0/2138681157 >> v2:172.21.15.188:3300/0 conn(0x7fb0c80b5570 0x7fb0c806f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 >> v1:172.21.15.188:6789/0 conn(0x7fb0c806c330 legacy=0x7fb0c80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80d2300 con 0x7fb0c8158230 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb0c40019f0 con 0x7fb0c8158230 2024-04-25T09:57:59.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 >> v1:172.21.15.148:6789/0 conn(0x7fb0c8158230 legacy=0x7fb0c80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b40039b0 0x7fb0b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40043b0 0x7fb0b4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b4005080 0x7fb0b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8158610 con 0x7fb0b40039b0 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b400bfe0 con 0x7fb0b4005080 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b400c1e0 con 0x7fb0b40043b0 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cd541640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b40039b0 0x7fb0b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cdd42640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b4005080 0x7fb0b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0ccd40640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40043b0 0x7fb0b4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cd541640 1 -- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40043b0 msgr2=0x7fb0b4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cd541640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40043b0 0x7fb0b4004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cd541640 1 -- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b4005080 msgr2=0x7fb0b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cd541640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b4005080 0x7fb0b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cdd42640 1 -- 172.21.15.148:0/2138681157 reap_dead start 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0cd541640 1 -- 172.21.15.148:0/2138681157 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4003520 con 0x7fb0b40039b0 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.387+0000 7fb0ccd40640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40043b0 0x7fb0b4004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.388+0000 7fb0cd541640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b40039b0 0x7fb0b4003d90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400a860 tx=0x7fb0b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bf1b904c5af841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.388+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400a480 con 0x7fb0b40039b0 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.388+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c402f070 con 0x7fb0b40039b0 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.388+0000 7fb0bffff640 1 -- 172.21.15.148:0/2138681157 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c402bcb0 con 0x7fb0b40039b0 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2138681157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b40039b0 msgr2=0x7fb0b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b40039b0 0x7fb0b4003d90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400a860 tx=0x7fb0b4005830 comp rx=0 tx=0).stop 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2138681157 shutdown_connections 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2138681157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b40039b0 0x7fb0b4003d90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2138681157 >> 172.21.15.148:0/2138681157 conn(0x7fb0c8061600 msgr2=0x7fb0c80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2138681157 shutdown_connections 2024-04-25T09:57:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.389+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2138681157 wait complete. 2024-04-25T09:57:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.390+0000 7fb0cf7a5640 1 Processor -- start 2024-04-25T09:57:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.390+0000 7fb0cf7a5640 1 -- start start 2024-04-25T09:57:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40039b0 0x7fb0c8146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c80cfa50 0x7fb0c80cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 0x7fb0c81f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c80d25e0 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c806bea0 con 0x7fb0c80cfa50 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c815e320 con 0x7fb0b40039b0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 0x7fb0c81f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 0x7fb0c81f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39350/0 (socket says 172.21.15.148:39350) 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 -- 172.21.15.148:0/2424416563 learned_addr learned my addr 172.21.15.148:0/2424416563 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 -- 172.21.15.148:0/2424416563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40039b0 msgr2=0x7fb0c8146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40039b0 0x7fb0c8146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 -- 172.21.15.148:0/2424416563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c80cfa50 msgr2=0x7fb0c80cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c80cfa50 0x7fb0c80cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 -- 172.21.15.148:0/2424416563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c8148d10 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cdd42640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 0x7fb0c81f7420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0011860 tx=0x7fb0c0011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0019020 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c0004240 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c8148f90 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000f7a0 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.391+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c8149550 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.392+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c806de50 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.393+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0c001c020 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.393+0000 7fb0bdffb640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.393+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb0c008a380 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.394+0000 7fb0cd541640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.394+0000 7fb0cd541640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4033000 tx=0x7fb0c402d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.398+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0c0023020 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.565+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0c8073a00 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.566+0000 7fb0bdffb640 1 -- 172.21.15.148:0/2424416563 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0c8148d10 con 0x7fb0c81f6fa0 2024-04-25T09:57:59.569 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb0ac06b4d0 msgr2=0x7fb0ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4033000 tx=0x7fb0c402d000 comp rx=0 tx=0).stop 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 msgr2=0x7fb0c81f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 0x7fb0c81f7420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0011860 tx=0x7fb0c0011d20 comp rx=0 tx=0).stop 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 shutdown_connections 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb0ac06b4d0 0x7fb0ac06d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0c81f6fa0 0x7fb0c81f7420 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c80cfa50 0x7fb0c80cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 --2- 172.21.15.148:0/2424416563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b40039b0 0x7fb0c8146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 >> 172.21.15.148:0/2424416563 conn(0x7fb0c8061600 msgr2=0x7fb0c815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 shutdown_connections 2024-04-25T09:57:59.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.568+0000 7fb0cf7a5640 1 -- 172.21.15.148:0/2424416563 wait complete. 2024-04-25T09:57:59.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:57:59.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 Processor -- start 2024-04-25T09:57:59.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 -- start start 2024-04-25T09:57:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd66815dc90 0x7fd6681625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6680cc8e0 0x7fd66815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd6680d1810 0x7fd6680d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680d2180 con 0x7fd6680d1810 2024-04-25T09:57:59.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66815f390 con 0x7fd6680cc8e0 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.678+0000 7fd66cd7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680cce50 con 0x7fd66815dc90 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd6680d1810 0x7fd6680d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd6680d1810 0x7fd6680d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39358/0 (socket says 172.21.15.148:39358) 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 -- 172.21.15.148:0/3656881521 learned_addr learned my addr 172.21.15.148:0/3656881521 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd66659c640 1 --2- 172.21.15.148:0/3656881521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd66815dc90 0x7fd6681625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd65ffff640 1 --2- 172.21.15.148:0/3656881521 >> v2:172.21.15.188:3300/0 conn(0x7fd6680cc8e0 0x7fd66815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 -- 172.21.15.148:0/3656881521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd66815dc90 msgr2=0x7fd6681625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 --2- 172.21.15.148:0/3656881521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd66815dc90 0x7fd6681625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 -- 172.21.15.148:0/3656881521 >> v2:172.21.15.188:3300/0 conn(0x7fd6680cc8e0 msgr2=0x7fd66815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 --2- 172.21.15.148:0/3656881521 >> v2:172.21.15.188:3300/0 conn(0x7fd6680cc8e0 0x7fd66815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.679+0000 7fd666d9d640 1 -- 172.21.15.148:0/3656881521 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668161af0 con 0x7fd6680d1810 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd666d9d640 1 --2- 172.21.15.148:0/3656881521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd6680d1c40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cf390 tx=0x7fd66002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab83b283f899fed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd665d9b640 1 -- 172.21.15.148:0/3656881521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660036070 con 0x7fd6680d1810 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd665d9b640 1 -- 172.21.15.148:0/3656881521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660031600 con 0x7fd6680d1810 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd665d9b640 1 -- 172.21.15.148:0/3656881521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600319d0 con 0x7fd6680d1810 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 -- 172.21.15.148:0/3656881521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 msgr2=0x7fd6680d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/3656881521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd6680d1c40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cf390 tx=0x7fd66002a610 comp rx=0 tx=0).stop 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 -- 172.21.15.148:0/3656881521 shutdown_connections 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/3656881521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd6680d1c40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/3656881521 >> v2:172.21.15.188:3300/0 conn(0x7fd6680cc8e0 0x7fd66815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/3656881521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd66815dc90 0x7fd6681625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 -- 172.21.15.148:0/3656881521 >> 172.21.15.148:0/3656881521 conn(0x7fd66806f590 msgr2=0x7fd66806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 -- 172.21.15.148:0/3656881521 shutdown_connections 2024-04-25T09:57:59.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.680+0000 7fd66cd7e640 1 -- 172.21.15.148:0/3656881521 wait complete. 2024-04-25T09:57:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 Processor -- start 2024-04-25T09:57:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 -- start start 2024-04-25T09:57:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6680cc8e0 0x7fd6681f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd66814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd66815dc90 0x7fd6680b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680cd130 con 0x7fd6680d1810 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681620b0 con 0x7fd66815dc90 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd66cd7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66815e070 con 0x7fd6680cc8e0 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd66814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd65ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd66814edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39374/0 (socket says 172.21.15.148:39374) 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.682+0000 7fd65ffff640 1 -- 172.21.15.148:0/1320768833 learned_addr learned my addr 172.21.15.148:0/1320768833 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd666d9d640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd66815dc90 0x7fd6680b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd66659c640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6680cc8e0 0x7fd6681f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65ffff640 1 -- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6680cc8e0 msgr2=0x7fd6681f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65ffff640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6680cc8e0 0x7fd6681f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65ffff640 1 -- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd66815dc90 msgr2=0x7fd6680b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65ffff640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd66815dc90 0x7fd6680b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65ffff640 1 -- 172.21.15.148:0/1320768833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680cc020 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd66659c640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6680cc8e0 0x7fd6681f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65ffff640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd66814edf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd65000d830 tx=0x7fd65000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650018020 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd650009640 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6681f4f60 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650009a10 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.683+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6681f5520 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.684+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668073990 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.685+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd65001b020 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.685+0000 7fd65effd640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.685+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd65008a1c0 con 0x7fd6680d1810 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.685+0000 7fd66659c640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.685+0000 7fd66659c640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd64806b4d0 0x7fd64806d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd658006fd0 tx=0x7fd658008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.689+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd650022030 con 0x7fd6680d1810 2024-04-25T09:57:59.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.860+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd6680b1890 con 0x7fd6680d1810 2024-04-25T09:57:59.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.861+0000 7fd65effd640 1 -- 172.21.15.148:0/1320768833 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd650059200 con 0x7fd6680d1810 2024-04-25T09:57:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:57:59.865 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:57:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:57:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T09:57:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T09:57:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:57:59.865 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd64806b4d0 msgr2=0x7fd64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd64806b4d0 0x7fd64806d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd658006fd0 tx=0x7fd658008040 comp rx=0 tx=0).stop 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 msgr2=0x7fd66814edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd66814edf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd65000d830 tx=0x7fd65000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 shutdown_connections 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd64806b4d0 0x7fd64806d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd66815dc90 0x7fd6680b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6680d1810 0x7fd66814edf0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 --2- 172.21.15.148:0/1320768833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6680cc8e0 0x7fd6681f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 >> 172.21.15.148:0/1320768833 conn(0x7fd66806f590 msgr2=0x7fd66806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 shutdown_connections 2024-04-25T09:57:59.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.863+0000 7fd66cd7e640 1 -- 172.21.15.148:0/1320768833 wait complete. 2024-04-25T09:57:59.877 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:57:59.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:57:59.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.975+0000 7fb812389640 1 Processor -- start 2024-04-25T09:57:59.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.975+0000 7fb812389640 1 -- start start 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb812389640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 0x7fb80c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb812389640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb80c15deb0 0x7fb80c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb812389640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80c0d1770 con 0x7fb80c070dc0 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb812389640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c1622b0 con 0x7fb80c15deb0 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb812389640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b10f0 con 0x7fb80c06ccd0 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb80c070dc0 0x7fb80c0711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46664/0 (socket says 172.21.15.148:46664) 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80bfff640 1 -- 172.21.15.148:0/1988237046 learned_addr learned my addr 172.21.15.148:0/1988237046 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846245281 0 0) 0x7fb80c0d1770 con 0x7fb80c070dc0 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7f4003520 con 0x7fb80c070dc0 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80b7fe640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 0x7fb80c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb810926640 1 --2- 172.21.15.148:0/1988237046 >> v2:172.21.15.188:3300/0 conn(0x7fb80c15deb0 0x7fb80c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 859980724 0 0) 0x7fb7f4003520 con 0x7fb80c070dc0 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 msgr2=0x7fb80c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 0x7fb80c0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 >> v2:172.21.15.188:3300/0 conn(0x7fb80c15deb0 msgr2=0x7fb80c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 --2- 172.21.15.148:0/1988237046 >> v2:172.21.15.188:3300/0 conn(0x7fb80c15deb0 0x7fb80c15e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c162740 con 0x7fb80c070dc0 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb7fc001670 con 0x7fb80c070dc0 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 >> v1:172.21.15.148:6789/0 conn(0x7fb80c070dc0 legacy=0x7fb80c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7f40039b0 0x7fb7f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f4004800 0x7fb7f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.976+0000 7fb80affd640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 0x7fb7f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1622b0 con 0x7fb7f40039b0 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b10f0 con 0x7fb7f4004800 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f400ba40 con 0x7fb80c06ccd0 2024-04-25T09:57:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7f40039b0 0x7fb7f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb810926640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 0x7fb7f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80b7fe640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f4004800 0x7fb7f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 -- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 msgr2=0x7fb7f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c06ccd0 0x7fb7f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 -- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f4004800 msgr2=0x7fb7f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f4004800 0x7fb7f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 -- 172.21.15.148:0/1988237046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4003520 con 0x7fb7f40039b0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80b7fe640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f4004800 0x7fb7f4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.977+0000 7fb80bfff640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7f40039b0 0x7fb7f4003dd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc007fd0 tx=0x7fb7f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8c0300bc03d634 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc007d50 con 0x7fb7f40039b0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc005ad0 con 0x7fb7f40039b0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb80affd640 1 -- 172.21.15.148:0/1988237046 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc007070 con 0x7fb7f40039b0 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb812389640 1 -- 172.21.15.148:0/1988237046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7f40039b0 msgr2=0x7fb7f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb812389640 1 --2- 172.21.15.148:0/1988237046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7f40039b0 0x7fb7f4003dd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc007fd0 tx=0x7fb7f4005830 comp rx=0 tx=0).stop 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb810926640 1 -- 172.21.15.148:0/1988237046 reap_dead start 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb812389640 1 -- 172.21.15.148:0/1988237046 shutdown_connections 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb812389640 1 -- 172.21.15.148:0/1988237046 >> 172.21.15.148:0/1988237046 conn(0x7fb80c0b3530 msgr2=0x7fb80c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb812389640 1 -- 172.21.15.148:0/1988237046 shutdown_connections 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.978+0000 7fb812389640 1 -- 172.21.15.148:0/1988237046 wait complete. 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.979+0000 7fb812389640 1 Processor -- start 2024-04-25T09:57:59.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 -- start start 2024-04-25T09:57:59.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40039b0 0x7fb80c157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 0x7fb80c0d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c0d18f0 0x7fb80c155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c161f40 con 0x7fb80c151810 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c161c60 con 0x7fb7f40039b0 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb812389640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c0b5840 con 0x7fb80c0d18f0 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb80b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 0x7fb80c0d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb80b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 0x7fb80c0d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39384/0 (socket says 172.21.15.148:39384) 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb80b7fe640 1 -- 172.21.15.148:0/974300121 learned_addr learned my addr 172.21.15.148:0/974300121 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb810926640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c0d18f0 0x7fb80c155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.980+0000 7fb80bfff640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40039b0 0x7fb80c157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb80b7fe640 1 -- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c0d18f0 msgr2=0x7fb80c155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb80b7fe640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c0d18f0 0x7fb80c155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb80b7fe640 1 -- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40039b0 msgr2=0x7fb80c157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb80b7fe640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40039b0 0x7fb80c157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb80b7fe640 1 -- 172.21.15.148:0/974300121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c156280 con 0x7fb80c151810 2024-04-25T09:57:59.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb80b7fe640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 0x7fb80c0d13b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb800011570 tx=0x7fb800011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800015870 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800015a10 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800015dc0 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c156560 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.981+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c156b20 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.982+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c073a00 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.982+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb80001d400 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.983+0000 7fb808ff9640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7f806b500 0x7fb7f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.983+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb80008a6c0 con 0x7fb80c151810 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.983+0000 7fb80bfff640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7f806b500 0x7fb7f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:57:59.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.983+0000 7fb80bfff640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7f806b500 0x7fb7f806d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc033000 tx=0x7fb7fc009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:57:59.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:57:59.987+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb800019030 con 0x7fb80c151810 2024-04-25T09:58:00.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.158+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb80c15bda0 con 0x7fb80c151810 2024-04-25T09:58:00.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.607+0000 7fb808ff9640 1 -- 172.21.15.148:0/974300121 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb8000590b0 con 0x7fb80c151810 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7f806b500 msgr2=0x7fb7f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7f806b500 0x7fb7f806d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc033000 tx=0x7fb7fc009030 comp rx=0 tx=0).stop 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 msgr2=0x7fb80c0d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 0x7fb80c0d13b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb800011570 tx=0x7fb800011a30 comp rx=0 tx=0).stop 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 shutdown_connections 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb7f806b500 0x7fb7f806d9c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb80c0d18f0 0x7fb80c155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb80c151810 0x7fb80c0d13b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 --2- 172.21.15.148:0/974300121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40039b0 0x7fb80c157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.610+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 >> 172.21.15.148:0/974300121 conn(0x7fb80c0b3530 msgr2=0x7fb80c15c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.611+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 shutdown_connections 2024-04-25T09:58:00.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.611+0000 7fb812389640 1 -- 172.21.15.148:0/974300121 wait complete. 2024-04-25T09:58:00.630 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:58:00.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.731+0000 7f54ba330640 1 Processor -- start 2024-04-25T09:58:00.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.731+0000 7f54ba330640 1 -- start start 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54ba330640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f54b415dc20 0x7f54b40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54ba330640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b406cda0 con 0x7f54b40d32f0 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54ba330640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b415e000 con 0x7f54b4161710 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54ba330640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b40d1de0 con 0x7f54b415dc20 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54b88cd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f54b40d32f0 0x7f54b406c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46666/0 (socket says 172.21.15.148:46666) 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54b88cd640 1 -- 172.21.15.148:0/1728863553 learned_addr learned my addr 172.21.15.148:0/1728863553 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54b3fff640 1 --2- 172.21.15.148:0/1728863553 >> v2:172.21.15.188:3300/0 conn(0x7f54b415dc20 0x7f54b40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781051521 0 0) 0x7f54b406cda0 con 0x7f54b40d32f0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.732+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f549c003520 con 0x7f54b40d32f0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 324833116 0 0) 0x7f549c003520 con 0x7f54b40d32f0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 >> v2:172.21.15.188:3300/0 conn(0x7f54b415dc20 msgr2=0x7f54b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 --2- 172.21.15.148:0/1728863553 >> v2:172.21.15.188:3300/0 conn(0x7f54b415dc20 0x7f54b40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 >> v1:172.21.15.188:6789/0 conn(0x7f54b4161710 legacy=0x7f54b406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4161c60 con 0x7f54b40d32f0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f54ac0019f0 con 0x7f54b40d32f0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 >> v1:172.21.15.148:6789/0 conn(0x7f54b40d32f0 legacy=0x7f54b406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c0039b0 0x7f549c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c004800 0x7f549c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0051c0 0x7f549c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40d1de0 con 0x7f549c0039b0 2024-04-25T09:58:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00c060 con 0x7f549c0051c0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00c260 con 0x7f549c004800 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b3fff640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c0039b0 0x7f549c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b88cd640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0051c0 0x7f549c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b37fe640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c004800 0x7f549c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b3fff640 1 -- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c004800 msgr2=0x7f549c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b3fff640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c004800 0x7f549c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b3fff640 1 -- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0051c0 msgr2=0x7f549c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b3fff640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0051c0 0x7f549c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b88cd640 1 -- 172.21.15.148:0/1728863553 reap_dead start 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b3fff640 1 -- 172.21.15.148:0/1728863553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c003520 con 0x7f549c0039b0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.733+0000 7f54b37fe640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c004800 0x7f549c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.734+0000 7f54b3fff640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c0039b0 0x7f549c003dd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009ac0 tx=0x7f549c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd3dd9c1da5870a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.734+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4036070 con 0x7f549c0039b0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.734+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a4031400 con 0x7f549c0039b0 2024-04-25T09:58:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.734+0000 7f54b2ffd640 1 -- 172.21.15.148:0/1728863553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a4009690 con 0x7f549c0039b0 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 -- 172.21.15.148:0/1728863553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c0039b0 msgr2=0x7f549c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c0039b0 0x7f549c003dd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009ac0 tx=0x7f549c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 -- 172.21.15.148:0/1728863553 shutdown_connections 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 --2- 172.21.15.148:0/1728863553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c0039b0 0x7f549c003dd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 -- 172.21.15.148:0/1728863553 >> 172.21.15.148:0/1728863553 conn(0x7f54b40d0420 msgr2=0x7f54b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 -- 172.21.15.148:0/1728863553 shutdown_connections 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.735+0000 7f54ba330640 1 -- 172.21.15.148:0/1728863553 wait complete. 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.736+0000 7f54ba330640 1 Processor -- start 2024-04-25T09:58:00.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.736+0000 7f54ba330640 1 -- start start 2024-04-25T09:58:00.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0039b0 0x7f54b4152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b41e7f00 0x7f54b41e8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 0x7f54b415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4162500 con 0x7f54b415a930 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b4162220 con 0x7f549c0039b0 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b40b5e50 con 0x7f54b41e7f00 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 0x7f54b415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 0x7f54b415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39398/0 (socket says 172.21.15.148:39398) 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 -- 172.21.15.148:0/3043715324 learned_addr learned my addr 172.21.15.148:0/3043715324 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b3fff640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0039b0 0x7f54b4152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b37fe640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b41e7f00 0x7f54b41e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 -- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b41e7f00 msgr2=0x7f54b41e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b41e7f00 0x7f54b41e8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 -- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0039b0 msgr2=0x7f54b4152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0039b0 0x7f54b4152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 -- 172.21.15.148:0/3043715324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b41e8a70 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b88cd640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 0x7f54b415adb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f54b40b6570 tx=0x7f54ac007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00ab00 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ac018070 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b41e8d50 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0133f0 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.737+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b41e9310 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.738+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b4073a00 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.739+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54ac013590 con 0x7f54b415a930 2024-04-25T09:58:00.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.739+0000 7f54b0ff9640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a006b4d0 0x7f54a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:00.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.739+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f54ac08a470 con 0x7f54b415a930 2024-04-25T09:58:00.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.739+0000 7f54b3fff640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a006b4d0 0x7f54a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:00.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.740+0000 7f54b3fff640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a006b4d0 0x7f54a006d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f54a4033000 tx=0x7f54a4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:00.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.744+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54ac058cf0 con 0x7f54b415a930 2024-04-25T09:58:00.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:00.915+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f54b40d16b0 con 0x7f54b415a930 2024-04-25T09:58:01.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.612+0000 7f54b0ff9640 1 -- 172.21.15.148:0/3043715324 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7f54ac05cd90 con 0x7f54b415a930 2024-04-25T09:58:01.616 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T09:58:01.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a006b4d0 msgr2=0x7f54a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a006b4d0 0x7f54a006d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f54a4033000 tx=0x7f54a4008030 comp rx=0 tx=0).stop 2024-04-25T09:58:01.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 msgr2=0x7f54b415adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 0x7f54b415adb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f54b40b6570 tx=0x7f54ac007a90 comp rx=0 tx=0).stop 2024-04-25T09:58:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 shutdown_connections 2024-04-25T09:58:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a006b4d0 0x7f54a006d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b415a930 0x7f54b415adb0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b41e7f00 0x7f54b41e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 --2- 172.21.15.148:0/3043715324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c0039b0 0x7f54b4152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 >> 172.21.15.148:0/3043715324 conn(0x7f54b40d0420 msgr2=0x7f54b40b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:01.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 shutdown_connections 2024-04-25T09:58:01.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.615+0000 7f54ba330640 1 -- 172.21.15.148:0/3043715324 wait complete. 2024-04-25T09:58:01.645 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:58:01.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.745+0000 7fabdb0b3640 1 Processor -- start 2024-04-25T09:58:01.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.745+0000 7fabdb0b3640 1 -- start start 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabdb0b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabd4070dc0 0x7fabd406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabdb0b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd40d1770 con 0x7fabd40711a0 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabdb0b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd4162740 con 0x7fabd415deb0 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabdb0b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40b0eb0 con 0x7fabd4070dc0 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fabd40711a0 0x7fabd40b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46672/0 (socket says 172.21.15.148:46672) 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd3fff640 1 -- 172.21.15.148:0/2225183578 learned_addr learned my addr 172.21.15.148:0/2225183578 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd8e4f640 1 --2- 172.21.15.148:0/2225183578 >> v2:172.21.15.188:3300/0 conn(0x7fabd4070dc0 0x7fabd406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686940015 0 0) 0x7fabd40d1770 con 0x7fabd40711a0 2024-04-25T09:58:01.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc4003520 con 0x7fabd40711a0 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067283620 0 0) 0x7fabc4003520 con 0x7fabd40711a0 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 >> v2:172.21.15.188:3300/0 conn(0x7fabd4070dc0 msgr2=0x7fabd406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 --2- 172.21.15.148:0/2225183578 >> v2:172.21.15.188:3300/0 conn(0x7fabd4070dc0 0x7fabd406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 >> v1:172.21.15.188:6789/0 conn(0x7fabd415deb0 legacy=0x7fabd415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4161980 con 0x7fabd40711a0 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fabc0001670 con 0x7fabd40711a0 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 >> v1:172.21.15.148:6789/0 conn(0x7fabd40711a0 legacy=0x7fabd40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabc40039b0 0x7fabc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc4004800 0x7fabc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40b0eb0 con 0x7fabc40052e0 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc400c1b0 con 0x7fabc4004800 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc400c3b0 con 0x7fabc40039b0 2024-04-25T09:58:01.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.746+0000 7fabd9650640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd3fff640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc4004800 0x7fabc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd8e4f640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabc40039b0 0x7fabc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 -- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabc40039b0 msgr2=0x7fabc4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabc40039b0 0x7fabc4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 -- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc4004800 msgr2=0x7fabc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabc4004800 0x7fabc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 -- 172.21.15.148:0/2225183578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4003520 con 0x7fabc40052e0 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd8e4f640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabc40039b0 0x7fabc4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 -- 172.21.15.148:0/2225183578 reap_dead start 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd9650640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabc400b9c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fabcc001e20 tx=0x7fabc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4b1d5d87bf0de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.747+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc001a70 con 0x7fabc40052e0 2024-04-25T09:58:01.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc02ae40 con 0x7fabc40052e0 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabd37fe640 1 -- 172.21.15.148:0/2225183578 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003b90 con 0x7fabc40052e0 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 -- 172.21.15.148:0/2225183578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 msgr2=0x7fabc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabc400b9c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fabcc001e20 tx=0x7fabc4005830 comp rx=0 tx=0).stop 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 -- 172.21.15.148:0/2225183578 shutdown_connections 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/2225183578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabc400b9c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 -- 172.21.15.148:0/2225183578 >> 172.21.15.148:0/2225183578 conn(0x7fabd40b3530 msgr2=0x7fabd40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 -- 172.21.15.148:0/2225183578 shutdown_connections 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.748+0000 7fabdb0b3640 1 -- 172.21.15.148:0/2225183578 wait complete. 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.749+0000 7fabdb0b3640 1 Processor -- start 2024-04-25T09:58:01.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.749+0000 7fabdb0b3640 1 -- start start 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabdb0b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabd415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabdb0b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabd40cc220 0x7fabd415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabdb0b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabd41557a0 0x7fabd415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabdb0b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd40b10f0 con 0x7fabc40052e0 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabdb0b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd41622b0 con 0x7fabd41557a0 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabdb0b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd4161f40 con 0x7fabd40cc220 2024-04-25T09:58:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabd415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabd415ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39406/0 (socket says 172.21.15.148:39406) 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 -- 172.21.15.148:0/184959139 learned_addr learned my addr 172.21.15.148:0/184959139 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd3fff640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabd40cc220 0x7fabd415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd9650640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabd41557a0 0x7fabd415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 -- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabd40cc220 msgr2=0x7fabd415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabd40cc220 0x7fabd415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 -- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabd41557a0 msgr2=0x7fabd415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabd41557a0 0x7fabd415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd8e4f640 1 -- 172.21.15.148:0/184959139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40cc6c0 con 0x7fabc40052e0 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.750+0000 7fabd9650640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabd41557a0 0x7fabd415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.751+0000 7fabd8e4f640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabd415ecd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fabc8011570 tx=0x7fabc8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.751+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8015870 con 0x7fabc40052e0 2024-04-25T09:58:01.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.751+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc8015a10 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.751+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd40cc9a0 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.751+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8015dc0 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.751+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd40ccf60 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.752+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd40cd160 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.752+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabc800f460 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.753+0000 7fabd17fa640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fabb806b500 0x7fabb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.753+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fabc808ab20 con 0x7fabc40052e0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.753+0000 7fabd3fff640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fabb806b500 0x7fabb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.753+0000 7fabd3fff640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fabb806b500 0x7fabb806d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fabc00010e0 tx=0x7fabc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:01.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.757+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fabc8019030 con 0x7fabc40052e0 2024-04-25T09:58:01.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:01.964+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fabd4160c40 con 0x7fabc40052e0 2024-04-25T09:58:02.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.614+0000 7fabd17fa640 1 -- 172.21.15.148:0/184959139 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) v1 ==== 173+0+0 (secure 0 0 0) 0x7fabc8059900 con 0x7fabc40052e0 2024-04-25T09:58:02.618 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fabb806b500 msgr2=0x7fabb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fabb806b500 0x7fabb806d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fabc00010e0 tx=0x7fabc0009040 comp rx=0 tx=0).stop 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 msgr2=0x7fabd415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabd415ecd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fabc8011570 tx=0x7fabc8011a30 comp rx=0 tx=0).stop 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 shutdown_connections 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fabb806b500 0x7fabb806d9c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabd41557a0 0x7fabd415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabd40cc220 0x7fabd415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 --2- 172.21.15.148:0/184959139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabc40052e0 0x7fabd415ecd0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 >> 172.21.15.148:0/184959139 conn(0x7fabd40b3530 msgr2=0x7fabd40b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:02.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 shutdown_connections 2024-04-25T09:58:02.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.617+0000 7fabdb0b3640 1 -- 172.21.15.148:0/184959139 wait complete. 2024-04-25T09:58:02.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:02.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.747+0000 7f9f4699e640 1 Processor -- start 2024-04-25T09:58:02.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.747+0000 7f9f4699e640 1 -- start start 2024-04-25T09:58:02.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f4699e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f4015dc00 0x7f9f400c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f4699e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f400d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f4699e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f400d1c00 0x7f9f400ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f4699e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400d2060 con 0x7f9f400d1c00 2024-04-25T09:58:02.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f4699e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4015dfe0 con 0x7f9f4015dc00 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f4699e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400d00c0 con 0x7f9f400ceb20 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f400d1c00 0x7f9f400ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f400d1c00 0x7f9f400ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39410/0 (socket says 172.21.15.148:39410) 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 -- 172.21.15.148:0/1577584504 learned_addr learned my addr 172.21.15.148:0/1577584504 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f3ffff640 1 --2- 172.21.15.148:0/1577584504 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015dc00 0x7f9f400c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f3f7fe640 1 --2- 172.21.15.148:0/1577584504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f400d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 -- 172.21.15.148:0/1577584504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 msgr2=0x7f9f400d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 --2- 172.21.15.148:0/1577584504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f400d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 -- 172.21.15.148:0/1577584504 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015dc00 msgr2=0x7f9f400c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 --2- 172.21.15.148:0/1577584504 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015dc00 0x7f9f400c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.748+0000 7f9f44f3b640 1 -- 172.21.15.148:0/1577584504 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400d03a0 con 0x7f9f400d1c00 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.749+0000 7f9f44f3b640 1 --2- 172.21.15.148:0/1577584504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f400ccc40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9f4006efe0 tx=0x7f9f3802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bc4ea78eb231bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.749+0000 7f9f3effd640 1 -- 172.21.15.148:0/1577584504 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38036070 con 0x7f9f400d1c00 2024-04-25T09:58:02.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.749+0000 7f9f3effd640 1 -- 172.21.15.148:0/1577584504 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38031600 con 0x7f9f400d1c00 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.749+0000 7f9f3effd640 1 -- 172.21.15.148:0/1577584504 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380319d0 con 0x7f9f400d1c00 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 -- 172.21.15.148:0/1577584504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 msgr2=0x7f9f400ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 --2- 172.21.15.148:0/1577584504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f400ccc40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9f4006efe0 tx=0x7f9f3802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 -- 172.21.15.148:0/1577584504 shutdown_connections 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 --2- 172.21.15.148:0/1577584504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f400ccc40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 --2- 172.21.15.148:0/1577584504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f400d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 --2- 172.21.15.148:0/1577584504 >> v2:172.21.15.188:3300/0 conn(0x7f9f4015dc00 0x7f9f400c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 -- 172.21.15.148:0/1577584504 >> 172.21.15.148:0/1577584504 conn(0x7f9f4006f5d0 msgr2=0x7f9f4006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 -- 172.21.15.148:0/1577584504 shutdown_connections 2024-04-25T09:58:02.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.750+0000 7f9f4699e640 1 -- 172.21.15.148:0/1577584504 wait complete. 2024-04-25T09:58:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.751+0000 7f9f4699e640 1 Processor -- start 2024-04-25T09:58:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.751+0000 7f9f4699e640 1 -- start start 2024-04-25T09:58:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f401f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f40142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015dc00 0x7f9f40144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400cc130 con 0x7f9f400d1c00 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400cc720 con 0x7f9f4015dc00 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400cfd90 con 0x7f9f400ceb20 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f40142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f40142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39418/0 (socket says 172.21.15.148:39418) 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 -- 172.21.15.148:0/4034603079 learned_addr learned my addr 172.21.15.148:0/4034603079 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 -- 172.21.15.148:0/4034603079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 msgr2=0x7f9f401f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f401f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 -- 172.21.15.148:0/4034603079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015dc00 msgr2=0x7f9f40144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015dc00 0x7f9f40144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 -- 172.21.15.148:0/4034603079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40145810 con 0x7f9f400d1c00 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3f7fe640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f40142470 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e890 tx=0x7f9f2800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28017070 con 0x7f9f400d1c00 2024-04-25T09:58:02.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2800c940 con 0x7f9f400d1c00 2024-04-25T09:58:02.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800cd10 con 0x7f9f400d1c00 2024-04-25T09:58:02.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f40145af0 con 0x7f9f400d1c00 2024-04-25T09:58:02.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.752+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f401f45e0 con 0x7f9f400d1c00 2024-04-25T09:58:02.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.753+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f401f36b0 con 0x7f9f400d1c00 2024-04-25T09:58:02.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.754+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f2801a070 con 0x7f9f400d1c00 2024-04-25T09:58:02.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.754+0000 7f9f3cff9640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b4d0 0x7f9f2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:02.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.754+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f2808a6e0 con 0x7f9f400d1c00 2024-04-25T09:58:02.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.754+0000 7f9f3ffff640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b4d0 0x7f9f2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:02.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.755+0000 7f9f3ffff640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b4d0 0x7f9f2006d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006a90 tx=0x7f9f30006a20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:02.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.759+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f280593d0 con 0x7f9f400d1c00 2024-04-25T09:58:02.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.934+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f400c66c0 con 0x7f9f400d1c00 2024-04-25T09:58:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.935+0000 7f9f3cff9640 1 -- 172.21.15.148:0/4034603079 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f9f28058cd0 con 0x7f9f400d1c00 2024-04-25T09:58:02.939 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:02.939 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":66,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:01.965450+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b4d0 msgr2=0x7f9f2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b4d0 0x7f9f2006d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006a90 tx=0x7f9f30006a20 comp rx=0 tx=0).stop 2024-04-25T09:58:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 msgr2=0x7f9f40142470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:02.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f40142470 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e890 tx=0x7f9f2800ed50 comp rx=0 tx=0).stop 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 shutdown_connections 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b4d0 0x7f9f2006d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f4015dc00 0x7f9f40144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f400d1c00 0x7f9f40142470 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 --2- 172.21.15.148:0/4034603079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f400ceb20 0x7f9f401f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.937+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 >> 172.21.15.148:0/4034603079 conn(0x7f9f4006f5d0 msgr2=0x7f9f400cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.938+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 shutdown_connections 2024-04-25T09:58:02.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:02.938+0000 7f9f4699e640 1 -- 172.21.15.148:0/4034603079 wait complete. 2024-04-25T09:58:02.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.046+0000 7efe1c483640 1 Processor -- start 2024-04-25T09:58:03.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.046+0000 7efe1c483640 1 -- start start 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1c483640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe140d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1c483640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe140d1c00 0x7efe140ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1c483640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1415dfe0 con 0x7efe1415dc00 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1c483640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe140d2060 con 0x7efe140d1c00 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1c483640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140d00c0 con 0x7efe140c6df0 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1a21f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe1415dc00 0x7efe140ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46684/0 (socket says 172.21.15.148:46684) 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1a21f640 1 -- 172.21.15.148:0/1197676916 learned_addr learned my addr 172.21.15.148:0/1197676916 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1aa20640 1 --2- 172.21.15.148:0/1197676916 >> v2:172.21.15.188:3300/0 conn(0x7efe140d1c00 0x7efe140ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293756225 0 0) 0x7efe10001110 con 0x7efe1415dc00 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe19a1e640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe140d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe00003520 con 0x7efe1415dc00 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380778152 0 0) 0x7efe00003520 con 0x7efe1415dc00 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 msgr2=0x7efe140d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe140d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 >> v2:172.21.15.188:3300/0 conn(0x7efe140d1c00 msgr2=0x7efe140ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 --2- 172.21.15.148:0/1197676916 >> v2:172.21.15.188:3300/0 conn(0x7efe140d1c00 0x7efe140ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140d03a0 con 0x7efe1415dc00 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7efe10001840 con 0x7efe1415dc00 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 >> v1:172.21.15.148:6789/0 conn(0x7efe1415dc00 legacy=0x7efe140ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00004d40 0x7efe000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140d2060 con 0x7efe000039b0 2024-04-25T09:58:03.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140d00c0 con 0x7efe00004d40 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.047+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0000b9c0 con 0x7efe140c6df0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1aa20640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00004d40 0x7efe000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe19a1e640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 -- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 msgr2=0x7efe00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 -- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00004d40 msgr2=0x7efe000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00004d40 0x7efe000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 -- 172.21.15.148:0/1197676916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe00003520 con 0x7efe000039b0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe19a1e640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140c6df0 0x7efe00004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.048+0000 7efe1a21f640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe00003dd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c490 tx=0x7efe00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed03b6a169b2b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10032020 con 0x7efe000039b0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe10005ad0 con 0x7efe000039b0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1921d640 1 -- 172.21.15.148:0/1197676916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000c0e0 con 0x7efe000039b0 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1c483640 1 -- 172.21.15.148:0/1197676916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 msgr2=0x7efe00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1c483640 1 --2- 172.21.15.148:0/1197676916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe00003dd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c490 tx=0x7efe00005830 comp rx=0 tx=0).stop 2024-04-25T09:58:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1aa20640 1 -- 172.21.15.148:0/1197676916 reap_dead start 2024-04-25T09:58:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1c483640 1 -- 172.21.15.148:0/1197676916 shutdown_connections 2024-04-25T09:58:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1c483640 1 -- 172.21.15.148:0/1197676916 >> 172.21.15.148:0/1197676916 conn(0x7efe1406f5d0 msgr2=0x7efe1406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1c483640 1 -- 172.21.15.148:0/1197676916 shutdown_connections 2024-04-25T09:58:03.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.049+0000 7efe1c483640 1 -- 172.21.15.148:0/1197676916 wait complete. 2024-04-25T09:58:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.050+0000 7efe1c483640 1 Processor -- start 2024-04-25T09:58:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 -- start start 2024-04-25T09:58:03.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe140cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1415c820 0x7efe14150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140ca8b0 0x7efe1415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cc130 con 0x7efe000039b0 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cc720 con 0x7efe1415c820 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1c483640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cfd90 con 0x7efe140ca8b0 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1a21f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe140cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1a21f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe140cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39448/0 (socket says 172.21.15.148:39448) 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.051+0000 7efe1a21f640 1 -- 172.21.15.148:0/3052130977 learned_addr learned my addr 172.21.15.148:0/3052130977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe19a1e640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1415c820 0x7efe14150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1aa20640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140ca8b0 0x7efe1415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1a21f640 1 -- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140ca8b0 msgr2=0x7efe1415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1a21f640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140ca8b0 0x7efe1415b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1a21f640 1 -- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1415c820 msgr2=0x7efe14150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1a21f640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1415c820 0x7efe14150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1a21f640 1 -- 172.21.15.148:0/3052130977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140b4fe0 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe19a1e640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1415c820 0x7efe14150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1aa20640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140ca8b0 0x7efe1415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1a21f640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe140cb320 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efe10031040 tx=0x7efe10000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10004d30 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe10007070 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe140b52c0 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10007420 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.052+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe140ca0b0 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.053+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe14073990 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.053+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe10035020 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.054+0000 7efe0affd640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdf806b500 0x7efdf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.054+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe100aa4c0 con 0x7efe000039b0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.054+0000 7efe19a1e640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdf806b500 0x7efdf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.054+0000 7efe19a1e640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdf806b500 0x7efdf806d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe04006f80 tx=0x7efe04008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.058+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe10047030 con 0x7efe000039b0 2024-04-25T09:58:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.228+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe140c66c0 con 0x7efe000039b0 2024-04-25T09:58:03.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.230+0000 7efe0affd640 1 -- 172.21.15.148:0/3052130977 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16945 (secure 0 0 0) 0x7efe1003e070 con 0x7efe000039b0 2024-04-25T09:58:03.233 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:03.233 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":66,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:01.965450+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.231+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdf806b500 msgr2=0x7efdf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.231+0000 7efe1c483640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdf806b500 0x7efdf806d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efe04006f80 tx=0x7efe04008040 comp rx=0 tx=0).stop 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.231+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 msgr2=0x7efe140cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.231+0000 7efe1c483640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe140cb320 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efe10031040 tx=0x7efe10000fe0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 shutdown_connections 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdf806b500 0x7efdf806d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe140ca8b0 0x7efe1415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1415c820 0x7efe14150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 --2- 172.21.15.148:0/3052130977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe000039b0 0x7efe140cb320 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 >> 172.21.15.148:0/3052130977 conn(0x7efe1406f5d0 msgr2=0x7efe140ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 shutdown_connections 2024-04-25T09:58:03.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.232+0000 7efe1c483640 1 -- 172.21.15.148:0/3052130977 wait complete. 2024-04-25T09:58:03.247 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:58:03.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.345+0000 7f4437930640 1 Processor -- start 2024-04-25T09:58:03.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.345+0000 7f4437930640 1 -- start start 2024-04-25T09:58:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f4437930640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f443015ba80 0x7f44300cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f4437930640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44300cf250 0x7f44300b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f4437930640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443015ce20 con 0x7f443015c990 2024-04-25T09:58:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f4437930640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4430070130 con 0x7f443015ba80 2024-04-25T09:58:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f4437930640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443006fe50 con 0x7f44300cf250 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f443015ba80 0x7f44300cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f443015ba80 0x7f44300cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39460/0 (socket says 172.21.15.148:39460) 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 -- 172.21.15.148:0/3880929422 learned_addr learned my addr 172.21.15.148:0/3880929422 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f4435ecd640 1 --2- 172.21.15.148:0/3880929422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44300cf250 0x7f44300b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 -- 172.21.15.148:0/3880929422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44300cf250 msgr2=0x7f44300b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 --2- 172.21.15.148:0/3880929422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44300cf250 0x7f44300b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 -- 172.21.15.148:0/3880929422 >> v1:172.21.15.188:6789/0 conn(0x7f443015c990 legacy=0x7f44300ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.346+0000 7f44356cc640 1 -- 172.21.15.148:0/3880929422 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300cc8b0 con 0x7f443015ba80 2024-04-25T09:58:03.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.347+0000 7f44356cc640 1 --2- 172.21.15.148:0/3880929422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015ba80 0x7f44300cd390 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f4420009870 tx=0x7f442002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6177ffe8988711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.347+0000 7f442ffff640 1 -- 172.21.15.148:0/3880929422 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420037070 con 0x7f443015ba80 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.347+0000 7f442ffff640 1 -- 172.21.15.148:0/3880929422 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420032440 con 0x7f443015ba80 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.347+0000 7f442ffff640 1 -- 172.21.15.148:0/3880929422 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420032810 con 0x7f443015ba80 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 -- 172.21.15.148:0/3880929422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015ba80 msgr2=0x7f44300cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 --2- 172.21.15.148:0/3880929422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015ba80 0x7f44300cd390 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f4420009870 tx=0x7f442002a210 comp rx=0 tx=0).stop 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 -- 172.21.15.148:0/3880929422 shutdown_connections 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 --2- 172.21.15.148:0/3880929422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44300cf250 0x7f44300b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 --2- 172.21.15.148:0/3880929422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015ba80 0x7f44300cd390 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 -- 172.21.15.148:0/3880929422 >> 172.21.15.148:0/3880929422 conn(0x7f443006abe0 msgr2=0x7f443006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 -- 172.21.15.148:0/3880929422 shutdown_connections 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.348+0000 7f4437930640 1 -- 172.21.15.148:0/3880929422 wait complete. 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.349+0000 7f4437930640 1 Processor -- start 2024-04-25T09:58:03.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.349+0000 7f4437930640 1 -- start start 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44300cf250 0x7f443014d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443015ba80 0x7f44300adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 0x7f4430156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443006fb00 con 0x7f443015c990 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300cce70 con 0x7f44300cf250 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443006b850 con 0x7f443015ba80 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 0x7f4430156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 0x7f4430156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39474/0 (socket says 172.21.15.148:39474) 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 -- 172.21.15.148:0/2179342706 learned_addr learned my addr 172.21.15.148:0/2179342706 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f44356cc640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44300cf250 0x7f443014d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 -- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443015ba80 msgr2=0x7f44300adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443015ba80 0x7f44300adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 -- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44300cf250 msgr2=0x7f443014d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44300cf250 0x7f443014d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 -- 172.21.15.148:0/2179342706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430152a60 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4435ecd640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 0x7f4430156610 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f442800cd70 tx=0x7f442800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428003b90 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4428003d30 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4430152d40 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.350+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428004240 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.351+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4430153300 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.352+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44300ce440 con 0x7f443015c990 2024-04-25T09:58:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.352+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44280043e0 con 0x7f443015c990 2024-04-25T09:58:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.352+0000 7f442dffb640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f441c06b500 0x7f441c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.352+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f442808aee0 con 0x7f443015c990 2024-04-25T09:58:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.352+0000 7f44356cc640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f441c06b500 0x7f441c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.353+0000 7f44356cc640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f441c06b500 0x7f441c06d9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4420034000 tx=0x7f442002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.357+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44300ce440 con 0x7f443015c990 2024-04-25T09:58:03.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.562+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f443006afc0 con 0x7f443015c990 2024-04-25T09:58:03.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.632+0000 7f442dffb640 1 -- 172.21.15.148:0/2179342706 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f443006afc0 con 0x7f443015c990 2024-04-25T09:58:03.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f441c06b500 msgr2=0x7f441c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f441c06b500 0x7f441c06d9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4420034000 tx=0x7f442002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:03.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 msgr2=0x7f4430156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 0x7f4430156610 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f442800cd70 tx=0x7f442800a660 comp rx=0 tx=0).stop 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 shutdown_connections 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f441c06b500 0x7f441c06d9c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f443015c990 0x7f4430156610 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f443015ba80 0x7f44300adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 --2- 172.21.15.148:0/2179342706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44300cf250 0x7f443014d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 >> 172.21.15.148:0/2179342706 conn(0x7f443006abe0 msgr2=0x7f443015e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.634+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 shutdown_connections 2024-04-25T09:58:03.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.635+0000 7f4437930640 1 -- 172.21.15.148:0/2179342706 wait complete. 2024-04-25T09:58:03.650 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:58:03.650 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:58:03.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.747+0000 7fb3b84d1640 1 Processor -- start 2024-04-25T09:58:03.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.747+0000 7fb3b84d1640 1 -- start start 2024-04-25T09:58:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.748+0000 7fb3b84d1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3b015de80 0x7fb3b00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.748+0000 7fb3b84d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00afc80 con 0x7fb3b00cefb0 2024-04-25T09:58:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.748+0000 7fb3b84d1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00ce4d0 con 0x7fb3b00ccfe0 2024-04-25T09:58:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.748+0000 7fb3b84d1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00ce7b0 con 0x7fb3b015de80 2024-04-25T09:58:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.748+0000 7fb3b5a6c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb3b00cefb0 0x7fb3b00cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41450/0 (socket says 172.21.15.148:41450) 2024-04-25T09:58:03.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.748+0000 7fb3b5a6c640 1 -- 172.21.15.148:0/2851556925 learned_addr learned my addr 172.21.15.148:0/2851556925 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 --2- 172.21.15.148:0/2851556925 >> v2:172.21.15.188:3300/0 conn(0x7fb3b015de80 0x7fb3b00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464052264 0 0) 0x7fb3b00afc80 con 0x7fb3b00cefb0 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb39c003520 con 0x7fb3b00cefb0 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 267805618 0 0) 0x7fb39c003520 con 0x7fb3b00cefb0 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 >> v2:172.21.15.188:3300/0 conn(0x7fb3b015de80 msgr2=0x7fb3b00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 --2- 172.21.15.148:0/2851556925 >> v2:172.21.15.188:3300/0 conn(0x7fb3b015de80 0x7fb3b00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 >> v1:172.21.15.188:6789/0 conn(0x7fb3b00ccfe0 legacy=0x7fb3b00cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00cea90 con 0x7fb3b00cefb0 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb3a0001670 con 0x7fb3b00cefb0 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 >> v1:172.21.15.148:6789/0 conn(0x7fb3b00cefb0 legacy=0x7fb3b00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb39c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb39c004800 0x7fb39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb39c0051c0 0x7fb39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00ce7b0 con 0x7fb39c0039b0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c00c060 con 0x7fb39c004800 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c00c260 con 0x7fb39c0051c0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb39c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b5a6c640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb39c004800 0x7fb39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b6a6e640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb39c0051c0 0x7fb39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 -- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb39c0051c0 msgr2=0x7fb39c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb39c0051c0 0x7fb39c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 -- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb39c004800 msgr2=0x7fb39c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb39c004800 0x7fb39c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b626d640 1 -- 172.21.15.148:0/2851556925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c003520 con 0x7fb39c0039b0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b6a6e640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb39c0051c0 0x7fb39c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.749+0000 7fb3b6a6e640 1 -- 172.21.15.148:0/2851556925 reap_dead start 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.750+0000 7fb3b626d640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb39c003dd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009ac0 tx=0x7fb39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d7ea7b636a130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.750+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac036070 con 0x7fb39c0039b0 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.750+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac031400 con 0x7fb39c0039b0 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.750+0000 7fb3b526b640 1 -- 172.21.15.148:0/2851556925 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac009690 con 0x7fb39c0039b0 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 -- 172.21.15.148:0/2851556925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 msgr2=0x7fb39c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb39c003dd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009ac0 tx=0x7fb39c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 -- 172.21.15.148:0/2851556925 shutdown_connections 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/2851556925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb39c003dd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 -- 172.21.15.148:0/2851556925 >> 172.21.15.148:0/2851556925 conn(0x7fb3b00b2390 msgr2=0x7fb3b00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 -- 172.21.15.148:0/2851556925 shutdown_connections 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.751+0000 7fb3b84d1640 1 -- 172.21.15.148:0/2851556925 wait complete. 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.752+0000 7fb3b84d1640 1 Processor -- start 2024-04-25T09:58:03.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.752+0000 7fb3b84d1640 1 -- start start 2024-04-25T09:58:03.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb3b01f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b00c9940 0x7fb3b00c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b00c7830 0x7fb3b00c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00b3030 con 0x7fb39c0039b0 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00b2eb0 con 0x7fb3b00c7830 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00c5600 con 0x7fb3b00c9940 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb3b01f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb3b01f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34524/0 (socket says 172.21.15.148:34524) 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 -- 172.21.15.148:0/1156358250 learned_addr learned my addr 172.21.15.148:0/1156358250 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b5a6c640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b00c9940 0x7fb3b00c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b6a6e640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b00c7830 0x7fb3b00c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 -- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b00c9940 msgr2=0x7fb3b00c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b00c9940 0x7fb3b00c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 -- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b00c7830 msgr2=0x7fb3b00c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b00c7830 0x7fb3b00c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 -- 172.21.15.148:0/1156358250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b015b430 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b626d640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb3b01f6c20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac034040 tx=0x7fb3ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac03f020 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac037040 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac036070 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b015b710 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.753+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b015bcd0 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.754+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b0073a00 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.755+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3ac042020 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.755+0000 7fb3a6ffd640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb39406b4d0 0x7fb39406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.755+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3ac0aa2f0 con 0x7fb39c0039b0 2024-04-25T09:58:03.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.755+0000 7fb3b5a6c640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb39406b4d0 0x7fb39406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:03.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.756+0000 7fb3b5a6c640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb39406b4d0 0x7fb39406d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0001eb0 tx=0x7fb3a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:03.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.759+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3ac049030 con 0x7fb39c0039b0 2024-04-25T09:58:03.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:03.968+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3b01f7160 con 0x7fb39c0039b0 2024-04-25T09:58:04.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.642+0000 7fb3a6ffd640 1 -- 172.21.15.148:0/1156358250 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3ac079130 con 0x7fb39c0039b0 2024-04-25T09:58:04.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb39406b4d0 msgr2=0x7fb39406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb39406b4d0 0x7fb39406d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0001eb0 tx=0x7fb3a0009040 comp rx=0 tx=0).stop 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 msgr2=0x7fb3b01f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb3b01f6c20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac034040 tx=0x7fb3ac004730 comp rx=0 tx=0).stop 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 shutdown_connections 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb39406b4d0 0x7fb39406d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b00c7830 0x7fb3b00c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b00c9940 0x7fb3b00c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 --2- 172.21.15.148:0/1156358250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb39c0039b0 0x7fb3b01f6c20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 >> 172.21.15.148:0/1156358250 conn(0x7fb3b00b2390 msgr2=0x7fb3b006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 shutdown_connections 2024-04-25T09:58:04.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.645+0000 7fb3b84d1640 1 -- 172.21.15.148:0/1156358250 wait complete. 2024-04-25T09:58:04.662 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:58:04.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:04.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.761+0000 7f1b75c5d640 1 Processor -- start 2024-04-25T09:58:04.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.761+0000 7f1b75c5d640 1 -- start start 2024-04-25T09:58:04.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b75c5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b700c6df0 0x7f1b700d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b75c5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7015dfe0 con 0x7f1b700d1c00 2024-04-25T09:58:04.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b75c5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b700d00c0 con 0x7f1b7015dc00 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b75c5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b700d2070 con 0x7f1b700c6df0 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1b700d1c00 0x7f1b700ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41466/0 (socket says 172.21.15.148:41466) 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6ffff640 1 -- 172.21.15.148:0/835075693 learned_addr learned my addr 172.21.15.148:0/835075693 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6effd640 1 --2- 172.21.15.148:0/835075693 >> v2:172.21.15.188:3300/0 conn(0x7f1b700c6df0 0x7f1b700d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981867319 0 0) 0x7f1b68001490 con 0x7f1b700d1c00 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b58003520 con 0x7f1b700d1c00 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977288683 0 0) 0x7f1b58003520 con 0x7f1b700d1c00 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 >> v2:172.21.15.188:3300/0 conn(0x7f1b700c6df0 msgr2=0x7f1b700d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 --2- 172.21.15.148:0/835075693 >> v2:172.21.15.188:3300/0 conn(0x7f1b700c6df0 0x7f1b700d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 >> v1:172.21.15.188:6789/0 conn(0x7f1b7015dc00 legacy=0x7f1b700ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b700d03a0 con 0x7f1b700d1c00 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f1b68001bc0 con 0x7f1b700d1c00 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 >> v1:172.21.15.148:6789/0 conn(0x7f1b700d1c00 legacy=0x7f1b700ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580039b0 0x7f1b58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b58004800 0x7f1b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b580051c0 0x7f1b5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700d2070 con 0x7f1b580051c0 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5800c060 con 0x7f1b58004800 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.762+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5800c260 con 0x7f1b580039b0 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b580051c0 0x7f1b5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6effd640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b58004800 0x7f1b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6f7fe640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580039b0 0x7f1b58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 -- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580039b0 msgr2=0x7f1b58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580039b0 0x7f1b58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 -- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b58004800 msgr2=0x7f1b58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b58004800 0x7f1b58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 -- 172.21.15.148:0/835075693 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58003520 con 0x7f1b580051c0 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6f7fe640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580039b0 0x7f1b58003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 -- 172.21.15.148:0/835075693 reap_dead start 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.763+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b580051c0 0x7f1b5800b9c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800aa50 tx=0x7f1b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfe2e1cd314878d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800a670 con 0x7f1b580051c0 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6802f070 con 0x7f1b580051c0 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b6e7fc640 1 -- 172.21.15.148:0/835075693 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68034470 con 0x7f1b580051c0 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 -- 172.21.15.148:0/835075693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b580051c0 msgr2=0x7f1b5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b580051c0 0x7f1b5800b9c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800aa50 tx=0x7f1b58005830 comp rx=0 tx=0).stop 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 -- 172.21.15.148:0/835075693 shutdown_connections 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/835075693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b580051c0 0x7f1b5800b9c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 -- 172.21.15.148:0/835075693 >> 172.21.15.148:0/835075693 conn(0x7f1b7006f5d0 msgr2=0x7f1b7006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:04.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 -- 172.21.15.148:0/835075693 shutdown_connections 2024-04-25T09:58:04.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.764+0000 7f1b75c5d640 1 -- 172.21.15.148:0/835075693 wait complete. 2024-04-25T09:58:04.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.765+0000 7f1b75c5d640 1 Processor -- start 2024-04-25T09:58:04.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 -- start start 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580051c0 0x7f1b70154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b701e99b0 0x7f1b701f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 0x7f1b700cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700cc130 con 0x7f1b701f7500 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700cc720 con 0x7f1b701e99b0 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b75c5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b700cfd90 con 0x7f1b580051c0 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b6ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 0x7f1b700cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b6ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 0x7f1b700cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34550/0 (socket says 172.21.15.148:34550) 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.766+0000 7f1b6ffff640 1 -- 172.21.15.148:0/3243671899 learned_addr learned my addr 172.21.15.148:0/3243671899 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6effd640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b701e99b0 0x7f1b701f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6f7fe640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580051c0 0x7f1b70154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6ffff640 1 -- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580051c0 msgr2=0x7f1b70154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580051c0 0x7f1b70154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6ffff640 1 -- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b701e99b0 msgr2=0x7f1b701f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b701e99b0 0x7f1b701f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6ffff640 1 -- 172.21.15.148:0/3243671899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b701f3fc0 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6effd640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b701e99b0 0x7f1b701f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b6ffff640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 0x7f1b700cad00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800ab40 tx=0x7f1b68001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800a670 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68001490 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68034670 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b701f42a0 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.767+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b701f6fe0 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.768+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b701f71e0 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.768+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b68034810 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.769+0000 7f1b57fff640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b5c06b500 0x7f1b5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.769+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b680aa220 con 0x7f1b701f7500 2024-04-25T09:58:04.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.769+0000 7f1b6f7fe640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b5c06b500 0x7f1b5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:04.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.769+0000 7f1b6f7fe640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b5c06b500 0x7f1b5c06d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b60000c00 tx=0x7f1b60009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:04.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.773+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b701f71e0 con 0x7f1b701f7500 2024-04-25T09:58:04.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.982+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b700c66c0 con 0x7f1b701f7500 2024-04-25T09:58:04.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.982+0000 7f1b57fff640 1 -- 172.21.15.148:0/3243671899 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f1b7015dfe0 con 0x7f1b701f7500 2024-04-25T09:58:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:04.986 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3775040580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3775040580},{"type":"v1","addr":"172.21.15.148:6837","nonce":3775040580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/733820966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":733820966},{"type":"v1","addr":"172.21.15.188:6834","nonce":733820966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2576966127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2576966127},{"type":"v1","addr":"172.21.15.188:6835","nonce":2576966127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:03.969534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6729},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6729":{"gid":6729,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/2397025400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2397025400},{"type":"v1","addr":"172.21.15.148:6836","nonce":2397025400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:04.986 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 33 2024-04-25T09:58:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b5c06b500 msgr2=0x7f1b5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b5c06b500 0x7f1b5c06d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b60000c00 tx=0x7f1b60009040 comp rx=0 tx=0).stop 2024-04-25T09:58:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 msgr2=0x7f1b700cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 0x7f1b700cad00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800ab40 tx=0x7f1b68001f90 comp rx=0 tx=0).stop 2024-04-25T09:58:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 shutdown_connections 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b5c06b500 0x7f1b5c06d9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b701f7500 0x7f1b700cad00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b701e99b0 0x7f1b701f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 --2- 172.21.15.148:0/3243671899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580051c0 0x7f1b70154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.984+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 >> 172.21.15.148:0/3243671899 conn(0x7f1b7006f5d0 msgr2=0x7f1b7006ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.985+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 shutdown_connections 2024-04-25T09:58:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:04.985+0000 7f1b75c5d640 1 -- 172.21.15.148:0/3243671899 wait complete. 2024-04-25T09:58:04.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.094+0000 7f90d4534640 1 Processor -- start 2024-04-25T09:58:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.094+0000 7f90d4534640 1 -- start start 2024-04-25T09:58:05.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d4534640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d4534640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90cc0d0900 0x7f90cc0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d4534640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0b9940 con 0x7f90cc15dd30 2024-04-25T09:58:05.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d4534640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc0d0d70 con 0x7f90cc0d0900 2024-04-25T09:58:05.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d4534640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0d1df0 con 0x7f90cc0d2e90 2024-04-25T09:58:05.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d2ad1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90cc0d0900 0x7f90cc0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d2ad1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90cc0d0900 0x7f90cc0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34556/0 (socket says 172.21.15.148:34556) 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d2ad1640 1 -- 172.21.15.148:0/804814528 learned_addr learned my addr 172.21.15.148:0/804814528 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.095+0000 7f90d1acf640 1 --2- 172.21.15.148:0/804814528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d2ad1640 1 -- 172.21.15.148:0/804814528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 msgr2=0x7f90cc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d2ad1640 1 --2- 172.21.15.148:0/804814528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d2ad1640 1 -- 172.21.15.148:0/804814528 >> v1:172.21.15.188:6789/0 conn(0x7f90cc15dd30 legacy=0x7f90cc0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d2ad1640 1 -- 172.21.15.148:0/804814528 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0d20d0 con 0x7f90cc0d0900 2024-04-25T09:58:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d2ad1640 1 --2- 172.21.15.148:0/804814528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b6a40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009e50 tx=0x7f90c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215f276656ab3b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d12ce640 1 -- 172.21.15.148:0/804814528 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4036070 con 0x7f90cc0d0900 2024-04-25T09:58:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d12ce640 1 -- 172.21.15.148:0/804814528 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4031600 con 0x7f90cc0d0900 2024-04-25T09:58:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.096+0000 7f90d12ce640 1 -- 172.21.15.148:0/804814528 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40319d0 con 0x7f90cc0d0900 2024-04-25T09:58:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 -- 172.21.15.148:0/804814528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0d0900 msgr2=0x7f90cc0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 --2- 172.21.15.148:0/804814528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b6a40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009e50 tx=0x7f90c402a680 comp rx=0 tx=0).stop 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 -- 172.21.15.148:0/804814528 shutdown_connections 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 --2- 172.21.15.148:0/804814528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b6a40 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 --2- 172.21.15.148:0/804814528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 -- 172.21.15.148:0/804814528 >> 172.21.15.148:0/804814528 conn(0x7f90cc061c30 msgr2=0x7f90cc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 -- 172.21.15.148:0/804814528 shutdown_connections 2024-04-25T09:58:05.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.097+0000 7f90d4534640 1 -- 172.21.15.148:0/804814528 wait complete. 2024-04-25T09:58:05.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.098+0000 7f90d4534640 1 Processor -- start 2024-04-25T09:58:05.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.098+0000 7f90d4534640 1 -- start start 2024-04-25T09:58:05.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d4534640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d4534640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc15f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d4534640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 0x7f90cc15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d4534640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0d2970 con 0x7f90cc15dd30 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d4534640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0d2690 con 0x7f90cc0d0900 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d4534640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0d1a10 con 0x7f90cc0d2e90 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 0x7f90cc15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 0x7f90cc15fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34562/0 (socket says 172.21.15.148:34562) 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 -- 172.21.15.148:0/2631683126 learned_addr learned my addr 172.21.15.148:0/2631683126 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d22d0640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d1acf640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 -- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 msgr2=0x7f90cc15f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc15f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 -- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0d0900 msgr2=0x7f90cc0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.099+0000 7f90d2ad1640 1 -- 172.21.15.148:0/2631683126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc147260 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.100+0000 7f90d2ad1640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 0x7f90cc15fc40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009f70 tx=0x7f90c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.100+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c403f020 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.100+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4037040 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.100+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc0b61b0 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.100+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4036070 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.100+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc1442a0 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.101+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc073990 con 0x7f90cc15dd30 2024-04-25T09:58:05.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.101+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90c4042020 con 0x7f90cc15dd30 2024-04-25T09:58:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.102+0000 7f90c2ffd640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.102+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90c40aa660 con 0x7f90cc15dd30 2024-04-25T09:58:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.102+0000 7f90d22d0640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.102+0000 7f90d22d0640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b006b4d0 0x7f90b006d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90c80016e0 tx=0x7f90c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.106+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90c4049030 con 0x7f90cc15dd30 2024-04-25T09:58:05.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.278+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90cc15d1a0 con 0x7f90cc15dd30 2024-04-25T09:58:05.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.279+0000 7f90c2ffd640 1 -- 172.21.15.148:0/2631683126 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f90c40a8e20 con 0x7f90cc15dd30 2024-04-25T09:58:05.283 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:05.283 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:03.627039+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b006b4d0 msgr2=0x7f90b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b006b4d0 0x7f90b006d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f90c80016e0 tx=0x7f90c8009040 comp rx=0 tx=0).stop 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 msgr2=0x7f90cc15fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 0x7f90cc15fc40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009f70 tx=0x7f90c4004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 shutdown_connections 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b006b4d0 0x7f90b006d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc15dd30 0x7f90cc15fc40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc0d2e90 0x7f90cc15f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 --2- 172.21.15.148:0/2631683126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0d0900 0x7f90cc0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 >> 172.21.15.148:0/2631683126 conn(0x7f90cc061c30 msgr2=0x7f90cc0c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:05.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 shutdown_connections 2024-04-25T09:58:05.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.281+0000 7f90d4534640 1 -- 172.21.15.148:0/2631683126 wait complete. 2024-04-25T09:58:05.296 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:58:05.296 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.390+0000 7f7048fb0640 1 Processor -- start 2024-04-25T09:58:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7048fb0640 1 -- start start 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7048fb0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70441625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7048fb0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70440d1810 0x7f70440d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7048fb0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70440cce50 con 0x7f704415dc90 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7048fb0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7044161af0 con 0x7f70440d1810 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7048fb0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044161dd0 con 0x7f70440cc8e0 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f7042d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70440d1810 0x7f70440d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f704259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f704415dc90 0x7f704415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41482/0 (socket says 172.21.15.148:41482) 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.391+0000 7f704259c640 1 -- 172.21.15.148:0/3018315702 learned_addr learned my addr 172.21.15.148:0/3018315702 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f703bfff640 1 --2- 172.21.15.148:0/3018315702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70441625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f7042d9d640 1 -- 172.21.15.148:0/3018315702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 msgr2=0x7f70441625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f7042d9d640 1 --2- 172.21.15.148:0/3018315702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70441625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f7042d9d640 1 -- 172.21.15.148:0/3018315702 >> v1:172.21.15.148:6789/0 conn(0x7f704415dc90 legacy=0x7f704415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f7042d9d640 1 -- 172.21.15.148:0/3018315702 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70441620b0 con 0x7f70440d1810 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f7041d9b640 1 -- 172.21.15.148:0/3018315702 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697055582 0 0) 0x7f70440cce50 con 0x7f704415dc90 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.392+0000 7f7042d9d640 1 --2- 172.21.15.148:0/3018315702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70440d1810 0x7f70440d1ca0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f703c009d90 tx=0x7f703c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c8d3902853a1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7041d9b640 1 -- 172.21.15.148:0/3018315702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c031600 con 0x7f70440d1810 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7041d9b640 1 -- 172.21.15.148:0/3018315702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c0317a0 con 0x7f70440d1810 2024-04-25T09:58:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7041d9b640 1 -- 172.21.15.148:0/3018315702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c031b70 con 0x7f70440d1810 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 -- 172.21.15.148:0/3018315702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70440d1810 msgr2=0x7f70440d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 --2- 172.21.15.148:0/3018315702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70440d1810 0x7f70440d1ca0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f703c009d90 tx=0x7f703c02a5d0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 -- 172.21.15.148:0/3018315702 shutdown_connections 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 --2- 172.21.15.148:0/3018315702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70440d1810 0x7f70440d1ca0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 --2- 172.21.15.148:0/3018315702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70441625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 -- 172.21.15.148:0/3018315702 >> 172.21.15.148:0/3018315702 conn(0x7f704406f590 msgr2=0x7f704406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:05.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 -- 172.21.15.148:0/3018315702 shutdown_connections 2024-04-25T09:58:05.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.393+0000 7f7048fb0640 1 -- 172.21.15.148:0/3018315702 wait complete. 2024-04-25T09:58:05.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.394+0000 7f7048fb0640 1 Processor -- start 2024-04-25T09:58:05.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 -- start start 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70440c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70440d1810 0x7f70440cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 0x7f70440ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704415e1f0 con 0x7f704415dc90 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704415e070 con 0x7f70440d1810 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7048fb0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70440cfde0 con 0x7f70440cc8e0 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 0x7f70440ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 0x7f70440ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34578/0 (socket says 172.21.15.148:34578) 2024-04-25T09:58:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f7042d9d640 1 -- 172.21.15.148:0/4166085371 learned_addr learned my addr 172.21.15.148:0/4166085371 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f704259c640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70440c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.395+0000 7f703bfff640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70440d1810 0x7f70440cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7042d9d640 1 -- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 msgr2=0x7f70440c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7042d9d640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70440c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7042d9d640 1 -- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70440d1810 msgr2=0x7f70440cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7042d9d640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70440d1810 0x7f70440cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7042d9d640 1 -- 172.21.15.148:0/4166085371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704414d890 con 0x7f704415dc90 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f704259c640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70440c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f703bfff640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70440d1810 0x7f70440cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7042d9d640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 0x7f70440ce440 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f703c009eb0 tx=0x7f703c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c040070 con 0x7f704415dc90 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c030b60 con 0x7f704415dc90 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70440af7e0 con 0x7f704415dc90 2024-04-25T09:58:05.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c009450 con 0x7f704415dc90 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.396+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70440afda0 con 0x7f704415dc90 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.397+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7044073990 con 0x7f704415dc90 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.397+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f703c039040 con 0x7f704415dc90 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.398+0000 7f703affd640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f702406b4d0 0x7f702406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.398+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f703c0ab9a0 con 0x7f704415dc90 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.398+0000 7f704259c640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f702406b4d0 0x7f702406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.398+0000 7f704259c640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f702406b4d0 0x7f702406d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7034000f60 tx=0x7f7034007ca0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.402+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f703c07a720 con 0x7f704415dc90 2024-04-25T09:58:05.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.568+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f704406c8c0 con 0x7f702406b4d0 2024-04-25T09:58:05.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.574+0000 7f703affd640 1 -- 172.21.15.148:0/4166085371 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237227 (secure 0 0 0) 0x7f704406c8c0 con 0x7f702406b4d0 2024-04-25T09:58:05.580 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:05.581 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f702406b4d0 msgr2=0x7f702406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f702406b4d0 0x7f702406d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7034000f60 tx=0x7f7034007ca0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 msgr2=0x7f70440ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 0x7f70440ce440 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f703c009eb0 tx=0x7f703c004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 shutdown_connections 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f702406b4d0 0x7f702406d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f704415dc90 0x7f70440ce440 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70440d1810 0x7f70440cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 --2- 172.21.15.148:0/4166085371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70440cc8e0 0x7f70440c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 >> 172.21.15.148:0/4166085371 conn(0x7f704406f590 msgr2=0x7f70440b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 shutdown_connections 2024-04-25T09:58:05.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.577+0000 7f7048fb0640 1 -- 172.21.15.148:0/4166085371 wait complete. 2024-04-25T09:58:05.592 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-25T09:58:05.064741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":289832,"kb_used_data":7112,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754684888,"statfs":{"total":773094113280,"available":772797325312,"internally_reserved":0,"allocated":7282688,"data_stored":3923704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.455666"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T09:57:59.216712+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:59.216712+0000","last_peered":"2024-04-25T09:57:59.216712+0000","last_clean":"2024-04-25T09:57:59.216712+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:59.216712+0000","last_undegraded":"2024-04-25T09:57:59.216712+0000","last_fullsized":"2024-04-25T09:57:59.216712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:05.596 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:05.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.727+0000 7eff3b458640 1 Processor -- start 2024-04-25T09:58:05.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.728+0000 7eff3b458640 1 -- start start 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.728+0000 7eff3b458640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 0x7eff340ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.728+0000 7eff3b458640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff340d0b10 0x7eff340d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.728+0000 7eff3b458640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340cec00 con 0x7eff340b5f00 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.728+0000 7eff3b458640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff340ce930 con 0x7eff340d0b10 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.728+0000 7eff3b458640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b6450 con 0x7eff3415db60 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff399f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff340d0b10 0x7eff340d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff391f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 0x7eff340ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff389f3640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7eff340b5f00 0x7eff340d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56930/0 (socket says 172.21.15.148:56930) 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff399f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff340d0b10 0x7eff340d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:49812/0 (socket says 172.21.15.148:49812) 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff389f3640 1 -- 172.21.15.148:0/4261293638 learned_addr learned my addr 172.21.15.148:0/4261293638 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff391f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 0x7eff340ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:51412/0 (socket says 172.21.15.148:51412) 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff399f5640 1 -- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 msgr2=0x7eff340ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff399f5640 1 --2- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 0x7eff340ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff399f5640 1 -- 172.21.15.148:0/4261293638 >> v1:172.21.15.188:6789/0 conn(0x7eff340b5f00 legacy=0x7eff340d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff399f5640 1 -- 172.21.15.148:0/4261293638 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340ceee0 con 0x7eff340d0b10 2024-04-25T09:58:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.729+0000 7eff391f4640 1 --2- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 0x7eff340ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.730+0000 7eff399f5640 1 --2- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff340d2270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff240097a0 tx=0x7eff2402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd6f5c69fe94761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.730+0000 7eff33fff640 1 -- 172.21.15.148:0/4261293638 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24037070 con 0x7eff340d0b10 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.730+0000 7eff33fff640 1 -- 172.21.15.148:0/4261293638 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2402ae50 con 0x7eff340d0b10 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.730+0000 7eff33fff640 1 -- 172.21.15.148:0/4261293638 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24032660 con 0x7eff340d0b10 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 -- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 msgr2=0x7eff340d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 --2- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff340d2270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff240097a0 tx=0x7eff2402a100 comp rx=0 tx=0).stop 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 -- 172.21.15.148:0/4261293638 shutdown_connections 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 --2- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff340d2270 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 --2- 172.21.15.148:0/4261293638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff3415db60 0x7eff340ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 -- 172.21.15.148:0/4261293638 >> 172.21.15.148:0/4261293638 conn(0x7eff34061600 msgr2=0x7eff34061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 -- 172.21.15.148:0/4261293638 shutdown_connections 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.731+0000 7eff3b458640 1 -- 172.21.15.148:0/4261293638 wait complete. 2024-04-25T09:58:05.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.732+0000 7eff3b458640 1 Processor -- start 2024-04-25T09:58:05.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.732+0000 7eff3b458640 1 -- start start 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff3b458640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff340b5f00 0x7eff340b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff3b458640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff34161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff3b458640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 0x7eff34161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff3b458640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3415df40 con 0x7eff3415db60 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff3b458640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340d1d50 con 0x7eff340d0b10 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff3b458640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3415e0c0 con 0x7eff340b5f00 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 0x7eff34161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 0x7eff34161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34590/0 (socket says 172.21.15.148:34590) 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 -- 172.21.15.148:0/3414373634 learned_addr learned my addr 172.21.15.148:0/3414373634 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff391f4640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff340b5f00 0x7eff340b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff389f3640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff34161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 -- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff340b5f00 msgr2=0x7eff340b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff340b5f00 0x7eff340b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 -- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 msgr2=0x7eff34161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff34161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 -- 172.21.15.148:0/3414373634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff341f46d0 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.733+0000 7eff399f5640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 0x7eff34161e40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7eff24035040 tx=0x7eff240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.734+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24041070 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.734+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2403c460 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.734+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff34162550 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.734+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24037040 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.734+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff340cc1f0 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.735+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff34073270 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.735+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff24044070 con 0x7eff3415db60 2024-04-25T09:58:05.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.736+0000 7eff31ffb640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.736+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff240aa880 con 0x7eff3415db60 2024-04-25T09:58:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.736+0000 7eff391f4640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.736+0000 7eff391f4640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7eff2006b4d0 0x7eff2006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009bd0 tx=0x7eff2c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:05.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.740+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff24078f20 con 0x7eff3415db60 2024-04-25T09:58:05.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.907+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff34073990 con 0x7eff2006b4d0 2024-04-25T09:58:05.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.913+0000 7eff31ffb640 1 -- 172.21.15.148:0/3414373634 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237227 (secure 0 0 0) 0x7eff34073990 con 0x7eff2006b4d0 2024-04-25T09:58:05.919 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:05.920 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:05.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.915+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7eff2006b4d0 msgr2=0x7eff2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.915+0000 7eff3b458640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7eff2006b4d0 0x7eff2006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009bd0 tx=0x7eff2c00b040 comp rx=0 tx=0).stop 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 msgr2=0x7eff34161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 0x7eff34161e40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7eff24035040 tx=0x7eff240046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 shutdown_connections 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff3415db60 0x7eff34161e40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff340d0b10 0x7eff34161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 --2- 172.21.15.148:0/3414373634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff340b5f00 0x7eff340b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 >> 172.21.15.148:0/3414373634 conn(0x7eff34061600 msgr2=0x7eff34061bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 shutdown_connections 2024-04-25T09:58:05.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:05.916+0000 7eff3b458640 1 -- 172.21.15.148:0/3414373634 wait complete. 2024-04-25T09:58:05.930 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-25T09:58:05.064741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":289832,"kb_used_data":7112,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754684888,"statfs":{"total":773094113280,"available":772797325312,"internally_reserved":0,"allocated":7282688,"data_stored":3923704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.455666"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T09:57:59.216712+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:59.216712+0000","last_peered":"2024-04-25T09:57:59.216712+0000","last_clean":"2024-04-25T09:57:59.216712+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:59.216712+0000","last_undegraded":"2024-04-25T09:57:59.216712+0000","last_fullsized":"2024-04-25T09:57:59.216712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:05.934 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:06.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.064+0000 7faf1a163640 1 Processor -- start 2024-04-25T09:58:06.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.064+0000 7faf1a163640 1 -- start start 2024-04-25T09:58:06.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf1a163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf140cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf1a163640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf140ce590 0x7faf1406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf1a163640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf14071480 con 0x7faf140cfcf0 2024-04-25T09:58:06.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf1a163640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf14072090 con 0x7faf140ce590 2024-04-25T09:58:06.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf1a163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140d0150 con 0x7faf14154ee0 2024-04-25T09:58:06.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf13fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faf140ce590 0x7faf1406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf12ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7faf140cfcf0 0x7faf140cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41490/0 (socket says 172.21.15.148:41490) 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf12ffd640 1 -- 172.21.15.148:0/2151325922 learned_addr learned my addr 172.21.15.148:0/2151325922 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf137fe640 1 --2- 172.21.15.148:0/2151325922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf140cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf13fff640 1 -- 172.21.15.148:0/2151325922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 msgr2=0x7faf140cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf13fff640 1 --2- 172.21.15.148:0/2151325922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf140cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf13fff640 1 -- 172.21.15.148:0/2151325922 >> v1:172.21.15.148:6789/0 conn(0x7faf140cfcf0 legacy=0x7faf140cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf127fc640 1 -- 172.21.15.148:0/2151325922 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169575217 0 0) 0x7faf14071480 con 0x7faf140cfcf0 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.065+0000 7faf13fff640 1 -- 172.21.15.148:0/2151325922 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf14071760 con 0x7faf140ce590 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf13fff640 1 --2- 172.21.15.148:0/2151325922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140ce590 0x7faf1406d6f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009d90 tx=0x7faf0c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d751297ec71125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf127fc640 1 -- 172.21.15.148:0/2151325922 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007740 con 0x7faf140ce590 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf127fc640 1 -- 172.21.15.148:0/2151325922 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c0078e0 con 0x7faf140ce590 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf127fc640 1 -- 172.21.15.148:0/2151325922 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007cb0 con 0x7faf140ce590 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 -- 172.21.15.148:0/2151325922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140ce590 msgr2=0x7faf1406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 --2- 172.21.15.148:0/2151325922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140ce590 0x7faf1406d6f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009d90 tx=0x7faf0c02a660 comp rx=0 tx=0).stop 2024-04-25T09:58:06.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 -- 172.21.15.148:0/2151325922 shutdown_connections 2024-04-25T09:58:06.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 --2- 172.21.15.148:0/2151325922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140ce590 0x7faf1406d6f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 --2- 172.21.15.148:0/2151325922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf140cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 -- 172.21.15.148:0/2151325922 >> 172.21.15.148:0/2151325922 conn(0x7faf1406abe0 msgr2=0x7faf1406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:06.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 -- 172.21.15.148:0/2151325922 shutdown_connections 2024-04-25T09:58:06.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.066+0000 7faf1a163640 1 -- 172.21.15.148:0/2151325922 wait complete. 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 Processor -- start 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 -- start start 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf140ce590 0x7faf1415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 0x7faf14157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf14158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14071d20 con 0x7faf140cfcf0 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14071230 con 0x7faf140ce590 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf1a163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf141552c0 con 0x7faf14154ee0 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.068+0000 7faf12ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 0x7faf14157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 0x7faf14157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34606/0 (socket says 172.21.15.148:34606) 2024-04-25T09:58:06.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 -- 172.21.15.148:0/274187228 learned_addr learned my addr 172.21.15.148:0/274187228 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 -- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 msgr2=0x7faf14158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf13fff640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf14158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf137fe640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf140ce590 0x7faf1415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf14158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 -- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf140ce590 msgr2=0x7faf1415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf140ce590 0x7faf1415a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 -- 172.21.15.148:0/274187228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf14159380 con 0x7faf140cfcf0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf12ffd640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 0x7faf14157bc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c500 tx=0x7faf0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04018070 con 0x7faf140cfcf0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf04007580 con 0x7faf140cfcf0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04004a20 con 0x7faf140cfcf0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf14159620 con 0x7faf140cfcf0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.069+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf14159ec0 con 0x7faf140cfcf0 2024-04-25T09:58:06.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.070+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf14073990 con 0x7faf140cfcf0 2024-04-25T09:58:06.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.071+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf0401b070 con 0x7faf140cfcf0 2024-04-25T09:58:06.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.071+0000 7faefbfff640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faf0006b4d0 0x7faf0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.071+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faf040896a0 con 0x7faf140cfcf0 2024-04-25T09:58:06.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.071+0000 7faf137fe640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faf0006b4d0 0x7faf0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.071+0000 7faf137fe640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faf0006b4d0 0x7faf0006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf08006fd0 tx=0x7faf08008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.075+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf04058390 con 0x7faf140cfcf0 2024-04-25T09:58:06.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.243+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf1406d360 con 0x7faf0006b4d0 2024-04-25T09:58:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.249+0000 7faefbfff640 1 -- 172.21.15.148:0/274187228 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237227 (secure 0 0 0) 0x7faf1406d360 con 0x7faf0006b4d0 2024-04-25T09:58:06.255 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:06.256 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:06.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.251+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faf0006b4d0 msgr2=0x7faf0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faf0006b4d0 0x7faf0006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf08006fd0 tx=0x7faf08008040 comp rx=0 tx=0).stop 2024-04-25T09:58:06.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 msgr2=0x7faf14157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 0x7faf14157bc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c500 tx=0x7faf0400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 shutdown_connections 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faf0006b4d0 0x7faf0006d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf14154ee0 0x7faf14158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf140cfcf0 0x7faf14157bc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 --2- 172.21.15.148:0/274187228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf140ce590 0x7faf1415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 >> 172.21.15.148:0/274187228 conn(0x7faf1406abe0 msgr2=0x7faf1415d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 shutdown_connections 2024-04-25T09:58:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.252+0000 7faf1a163640 1 -- 172.21.15.148:0/274187228 wait complete. 2024-04-25T09:58:06.266 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-25T09:58:05.064741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":289832,"kb_used_data":7112,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754684888,"statfs":{"total":773094113280,"available":772797325312,"internally_reserved":0,"allocated":7282688,"data_stored":3923704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.455666"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T09:57:59.216712+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:59.216712+0000","last_peered":"2024-04-25T09:57:59.216712+0000","last_clean":"2024-04-25T09:57:59.216712+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:59.216712+0000","last_undegraded":"2024-04-25T09:57:59.216712+0000","last_fullsized":"2024-04-25T09:57:59.216712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:06.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-04-25T09:58:06.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-04-25T09:58:06.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-04-25T09:58:06.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-25T09:58:06.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-04-25T09:58:06.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-25T09:58:06.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-25T09:58:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-04-25T09:58:06.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-04-25T09:58:06.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-04-25T09:58:06.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-25T09:58:06.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-04-25T09:58:06.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-04-25T09:58:06.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-04-25T09:58:06.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:01.608761+0000', 'last_change': '2024-04-25T09:58:01.608761+0000', 'last_active': '2024-04-25T09:58:01.608761+0000', 'last_peered': '2024-04-25T09:58:01.608761+0000', 'last_clean': '2024-04-25T09:58:01.608761+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:01.608761+0000', 'last_undegraded': '2024-04-25T09:58:01.608761+0000', 'last_fullsized': '2024-04-25T09:58:01.608761+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:01.608761+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-25T09:58:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T09:58:00.606483+0000', 'last_change': '2024-04-25T09:58:00.606483+0000', 'last_active': '2024-04-25T09:58:00.606483+0000', 'last_peered': '2024-04-25T09:58:00.606483+0000', 'last_clean': '2024-04-25T09:58:00.606483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:58:00.606483+0000', 'last_undegraded': '2024-04-25T09:58:00.606483+0000', 'last_fullsized': '2024-04-25T09:58:00.606483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'last_clean_scrub_stamp': '2024-04-25T09:58:00.606483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T09:58:06.288 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:58:06.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.400+0000 7fab2117e640 1 Processor -- start 2024-04-25T09:58:06.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.400+0000 7fab2117e640 1 -- start start 2024-04-25T09:58:06.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab2117e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c0ce330 0x7fab1c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab2117e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0700a0 0x7fab1c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab2117e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c1626e0 con 0x7fab1c06abe0 2024-04-25T09:58:06.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab2117e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c0cd270 con 0x7fab1c0700a0 2024-04-25T09:58:06.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab2117e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cd550 con 0x7fab1c0ce330 2024-04-25T09:58:06.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0700a0 0x7fab1c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0700a0 0x7fab1c15d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34608/0 (socket says 172.21.15.148:34608) 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 -- 172.21.15.148:0/431692477 learned_addr learned my addr 172.21.15.148:0/431692477 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 -- 172.21.15.148:0/431692477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c0ce330 msgr2=0x7fab1c06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 --2- 172.21.15.148:0/431692477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c0ce330 0x7fab1c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 -- 172.21.15.148:0/431692477 >> v1:172.21.15.188:6789/0 conn(0x7fab1c06abe0 legacy=0x7fab1c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.401+0000 7fab1b59e640 1 -- 172.21.15.148:0/431692477 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0cd830 con 0x7fab1c0700a0 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab1b59e640 1 --2- 172.21.15.148:0/431692477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0700a0 0x7fab1c15d370 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e50 tx=0x7fab1402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30340eba92caf82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab1a59c640 1 -- 172.21.15.148:0/431692477 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14036070 con 0x7fab1c0700a0 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab1a59c640 1 -- 172.21.15.148:0/431692477 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab14031600 con 0x7fab1c0700a0 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab1a59c640 1 -- 172.21.15.148:0/431692477 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140319d0 con 0x7fab1c0700a0 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab2117e640 1 -- 172.21.15.148:0/431692477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0700a0 msgr2=0x7fab1c15d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab2117e640 1 --2- 172.21.15.148:0/431692477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0700a0 0x7fab1c15d370 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e50 tx=0x7fab1402a630 comp rx=0 tx=0).stop 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab2117e640 1 -- 172.21.15.148:0/431692477 shutdown_connections 2024-04-25T09:58:06.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab2117e640 1 --2- 172.21.15.148:0/431692477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0700a0 0x7fab1c15d370 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.402+0000 7fab2117e640 1 --2- 172.21.15.148:0/431692477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c0ce330 0x7fab1c06fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.403+0000 7fab2117e640 1 -- 172.21.15.148:0/431692477 >> 172.21.15.148:0/431692477 conn(0x7fab1c0b2220 msgr2=0x7fab1c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:06.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.403+0000 7fab2117e640 1 -- 172.21.15.148:0/431692477 shutdown_connections 2024-04-25T09:58:06.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.403+0000 7fab2117e640 1 -- 172.21.15.148:0/431692477 wait complete. 2024-04-25T09:58:06.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.404+0000 7fab2117e640 1 Processor -- start 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.404+0000 7fab2117e640 1 -- start start 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.404+0000 7fab2117e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c06abe0 0x7fab1c1f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.404+0000 7fab2117e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0700a0 0x7fab1c1f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab2117e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 0x7fab1c0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab2117e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c15d8b0 con 0x7fab1c0ce330 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab2117e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c161780 con 0x7fab1c0700a0 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab2117e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c15da30 con 0x7fab1c06abe0 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 0x7fab1c0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 0x7fab1c0ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34614/0 (socket says 172.21.15.148:34614) 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 -- 172.21.15.148:0/828188036 learned_addr learned my addr 172.21.15.148:0/828188036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 -- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c06abe0 msgr2=0x7fab1c1f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c06abe0 0x7fab1c1f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 -- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0700a0 msgr2=0x7fab1c1f6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab13fff640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0700a0 0x7fab1c1f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0700a0 0x7fab1c1f6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 -- 172.21.15.148:0/828188036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c1f4ef0 con 0x7fab1c0ce330 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab13fff640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0700a0 0x7fab1c1f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab1b59e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 0x7fab1c0ade80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e50 tx=0x7fab14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14041020 con 0x7fab1c0ce330 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab14007780 con 0x7fab1c0ce330 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c160790 con 0x7fab1c0ce330 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14036040 con 0x7fab1c0ce330 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.405+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c160d50 con 0x7fab1c0ce330 2024-04-25T09:58:06.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.406+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c0afe50 con 0x7fab1c0ce330 2024-04-25T09:58:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.407+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab14044020 con 0x7fab1c0ce330 2024-04-25T09:58:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.407+0000 7fab137fe640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faafc06b4d0 0x7faafc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.407+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab140ab130 con 0x7fab1c0ce330 2024-04-25T09:58:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.407+0000 7fab1ad9d640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faafc06b4d0 0x7faafc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.408+0000 7fab1ad9d640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faafc06b4d0 0x7faafc06d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0cce90 tx=0x7fab0c0079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.411+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab1404b030 con 0x7fab1c0ce330 2024-04-25T09:58:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.648+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fab1c06b340 con 0x7fab1c0ce330 2024-04-25T09:58:06.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.651+0000 7fab137fe640 1 -- 172.21.15.148:0/828188036 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1184 (secure 0 0 0) 0x7fab14079e20 con 0x7fab1c0ce330 2024-04-25T09:58:06.655 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:06.655 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":247,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":296787968,"bytes_avail":772797325312,"bytes_total":773094113280,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":33,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6729}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T09:56:35.046115+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faafc06b4d0 msgr2=0x7faafc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faafc06b4d0 0x7faafc06d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0cce90 tx=0x7fab0c0079b0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 msgr2=0x7fab1c0ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 0x7fab1c0ade80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e50 tx=0x7fab14004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 shutdown_connections 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faafc06b4d0 0x7faafc06d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0ce330 0x7fab1c0ade80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0700a0 0x7fab1c1f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 --2- 172.21.15.148:0/828188036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c06abe0 0x7fab1c1f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 >> 172.21.15.148:0/828188036 conn(0x7fab1c0b2220 msgr2=0x7fab1c15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 shutdown_connections 2024-04-25T09:58:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.653+0000 7fab2117e640 1 -- 172.21.15.148:0/828188036 wait complete. 2024-04-25T09:58:06.668 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:58:06.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:06.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.762+0000 7ff94e334640 1 Processor -- start 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.762+0000 7ff94e334640 1 -- start start 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.763+0000 7ff94e334640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b6cb0 0x7ff9480b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.763+0000 7ff94e334640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 0x7ff9480d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.763+0000 7ff94e334640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9480ceee0 con 0x7ff94815db70 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.763+0000 7ff94e334640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9480d1d10 con 0x7ff9480b5a60 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.763+0000 7ff94e334640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480d1ff0 con 0x7ff9480b6cb0 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.763+0000 7ff94c8d1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 0x7ff9480d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff947fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff94815db70 0x7ff948162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56952/0 (socket says 172.21.15.148:56952) 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff947fff640 1 -- 172.21.15.148:0/2988389582 learned_addr learned my addr 172.21.15.148:0/2988389582 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff94c8d1640 1 --2- 172.21.15.148:0/2988389582 >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 0x7ff9480d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:49854/0 (socket says 172.21.15.148:49854) 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff9477fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b6cb0 0x7ff9480b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff9477fe640 1 -- 172.21.15.148:0/2988389582 >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 msgr2=0x7ff9480d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff9477fe640 1 --2- 172.21.15.148:0/2988389582 >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 0x7ff9480d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff9477fe640 1 -- 172.21.15.148:0/2988389582 >> v1:172.21.15.188:6789/0 conn(0x7ff94815db70 legacy=0x7ff948162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff9477fe640 1 -- 172.21.15.148:0/2988389582 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480d22d0 con 0x7ff9480b6cb0 2024-04-25T09:58:06.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.764+0000 7ff94c8d1640 1 --2- 172.21.15.148:0/2988389582 >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 0x7ff9480d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:06.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff9477fe640 1 --2- 172.21.15.148:0/2988389582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b6cb0 0x7ff9480b5520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097a0 tx=0x7ff93c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fed00a0e1efd0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff946ffd640 1 -- 172.21.15.148:0/2988389582 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c037070 con 0x7ff9480b6cb0 2024-04-25T09:58:06.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff946ffd640 1 -- 172.21.15.148:0/2988389582 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c032440 con 0x7ff9480b6cb0 2024-04-25T09:58:06.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff946ffd640 1 -- 172.21.15.148:0/2988389582 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c032810 con 0x7ff9480b6cb0 2024-04-25T09:58:06.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff94e334640 1 -- 172.21.15.148:0/2988389582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b6cb0 msgr2=0x7ff9480b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff94e334640 1 --2- 172.21.15.148:0/2988389582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b6cb0 0x7ff9480b5520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097a0 tx=0x7ff93c02a180 comp rx=0 tx=0).stop 2024-04-25T09:58:06.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff94e334640 1 -- 172.21.15.148:0/2988389582 shutdown_connections 2024-04-25T09:58:06.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff94e334640 1 --2- 172.21.15.148:0/2988389582 >> v2:172.21.15.188:3300/0 conn(0x7ff9480b5a60 0x7ff9480d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff94e334640 1 --2- 172.21.15.148:0/2988389582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b6cb0 0x7ff9480b5520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.765+0000 7ff94e334640 1 -- 172.21.15.148:0/2988389582 >> 172.21.15.148:0/2988389582 conn(0x7ff9480d0520 msgr2=0x7ff9480d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.766+0000 7ff94e334640 1 -- 172.21.15.148:0/2988389582 shutdown_connections 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.766+0000 7ff94e334640 1 -- 172.21.15.148:0/2988389582 wait complete. 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.767+0000 7ff94e334640 1 Processor -- start 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.767+0000 7ff94e334640 1 -- start start 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b5a60 0x7ff948160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9480b6cb0 0x7ff948157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 0x7ff948158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480cf1c0 con 0x7ff94815db70 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480ce530 con 0x7ff9480b6cb0 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480cf340 con 0x7ff9480b5a60 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 0x7ff948158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 0x7ff948158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34620/0 (socket says 172.21.15.148:34620) 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 -- 172.21.15.148:0/3521297925 learned_addr learned my addr 172.21.15.148:0/3521297925 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:06.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff9477fe640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9480b6cb0 0x7ff948157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff947fff640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b5a60 0x7ff948160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 -- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b5a60 msgr2=0x7ff948160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b5a60 0x7ff948160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 -- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9480b6cb0 msgr2=0x7ff948157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9480b6cb0 0x7ff948157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 -- 172.21.15.148:0/3521297925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948159760 con 0x7ff94815db70 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94c8d1640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 0x7ff948158120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff940009f00 tx=0x7ff94000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940018030 con 0x7ff94815db70 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94000ce50 con 0x7ff94815db70 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940003bb0 con 0x7ff94815db70 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff948159a20 con 0x7ff94815db70 2024-04-25T09:58:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.768+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff948159fe0 con 0x7ff94815db70 2024-04-25T09:58:06.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.769+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff948073990 con 0x7ff94815db70 2024-04-25T09:58:06.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.770+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff940005d10 con 0x7ff94815db70 2024-04-25T09:58:06.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.770+0000 7ff944ff9640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff93406b4d0 0x7ff93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:06.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.770+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff94008ac80 con 0x7ff94815db70 2024-04-25T09:58:06.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.770+0000 7ff947fff640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff93406b4d0 0x7ff93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:06.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.770+0000 7ff947fff640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff93406b4d0 0x7ff93406d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff938007fd0 tx=0x7ff938007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:06.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.774+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff940022070 con 0x7ff94815db70 2024-04-25T09:58:06.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.941+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9480d1820 con 0x7ff93406b4d0 2024-04-25T09:58:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.946+0000 7ff944ff9640 1 -- 172.21.15.148:0/3521297925 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237227 (secure 0 0 0) 0x7ff9480d1820 con 0x7ff93406b4d0 2024-04-25T09:58:06.953 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:06.954 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff93406b4d0 msgr2=0x7ff93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff93406b4d0 0x7ff93406d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff938007fd0 tx=0x7ff938007e80 comp rx=0 tx=0).stop 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 msgr2=0x7ff948158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 0x7ff948158120 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff940009f00 tx=0x7ff94000c490 comp rx=0 tx=0).stop 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 shutdown_connections 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff93406b4d0 0x7ff93406d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff94815db70 0x7ff948158120 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9480b6cb0 0x7ff948157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 --2- 172.21.15.148:0/3521297925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9480b5a60 0x7ff948160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.949+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 >> 172.21.15.148:0/3521297925 conn(0x7ff9480d0520 msgr2=0x7ff948157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.950+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 shutdown_connections 2024-04-25T09:58:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:06.950+0000 7ff94e334640 1 -- 172.21.15.148:0/3521297925 wait complete. 2024-04-25T09:58:06.964 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-25T09:58:05.064741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":289832,"kb_used_data":7112,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754684888,"statfs":{"total":773094113280,"available":772797325312,"internally_reserved":0,"allocated":7282688,"data_stored":3923704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.455666"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:01.608761+0000","last_change":"2024-04-25T09:58:01.608761+0000","last_active":"2024-04-25T09:58:01.608761+0000","last_peered":"2024-04-25T09:58:01.608761+0000","last_clean":"2024-04-25T09:58:01.608761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:01.608761+0000","last_undegraded":"2024-04-25T09:58:01.608761+0000","last_fullsized":"2024-04-25T09:58:01.608761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T09:58:00.606483+0000","last_change":"2024-04-25T09:58:00.606483+0000","last_active":"2024-04-25T09:58:00.606483+0000","last_peered":"2024-04-25T09:58:00.606483+0000","last_clean":"2024-04-25T09:58:00.606483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:58:00.606483+0000","last_undegraded":"2024-04-25T09:58:00.606483+0000","last_fullsized":"2024-04-25T09:58:00.606483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T09:57:59.216712+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:57:59.216712+0000","last_peered":"2024-04-25T09:57:59.216712+0000","last_clean":"2024-04-25T09:57:59.216712+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:57:59.216712+0000","last_undegraded":"2024-04-25T09:57:59.216712+0000","last_fullsized":"2024-04-25T09:57:59.216712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36084,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":761856,"data_stored":342871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1355776,"data_stored":933239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:08.296 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T09:58:08.292+0000 7f03714e0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:58:09.974 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:10.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.079+0000 7f57d97b7640 1 Processor -- start 2024-04-25T09:58:10.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.079+0000 7f57d97b7640 1 -- start start 2024-04-25T09:58:10.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d97b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57d415dbe0 0x7f57d40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d97b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d40b92a0 0x7f57d40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d97b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40b97a0 con 0x7f57d40b2cb0 2024-04-25T09:58:10.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d97b7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d415dfc0 con 0x7f57d415dbe0 2024-04-25T09:58:10.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d97b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40d1a00 con 0x7f57d40b92a0 2024-04-25T09:58:10.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57cbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f57d40b2cb0 0x7f57d40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41502/0 (socket says 172.21.15.148:41502) 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57cbfff640 1 -- 172.21.15.148:0/1325130403 learned_addr learned my addr 172.21.15.148:0/1325130403 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57d415dbe0 0x7f57d40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d37fe640 1 --2- 172.21.15.148:0/1325130403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d40b92a0 0x7f57d40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1325130403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d40b92a0 msgr2=0x7f57d40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d2ffd640 1 --2- 172.21.15.148:0/1325130403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d40b92a0 0x7f57d40d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1325130403 >> v1:172.21.15.148:6789/0 conn(0x7f57d40b2cb0 legacy=0x7f57d40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d27fc640 1 -- 172.21.15.148:0/1325130403 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155893911 0 0) 0x7f57d40b97a0 con 0x7f57d40b2cb0 2024-04-25T09:58:10.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.080+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1325130403 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40d1c50 con 0x7f57d415dbe0 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.081+0000 7f57d2ffd640 1 --2- 172.21.15.148:0/1325130403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d415dbe0 0x7f57d40d0910 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097a0 tx=0x7f57c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808d741b0b4fb036 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.081+0000 7f57d27fc640 1 -- 172.21.15.148:0/1325130403 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4037070 con 0x7f57d415dbe0 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.081+0000 7f57d27fc640 1 -- 172.21.15.148:0/1325130403 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c4032440 con 0x7f57d415dbe0 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.081+0000 7f57d27fc640 1 -- 172.21.15.148:0/1325130403 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4032810 con 0x7f57d415dbe0 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 -- 172.21.15.148:0/1325130403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d415dbe0 msgr2=0x7f57d40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1325130403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d415dbe0 0x7f57d40d0910 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f57c40097a0 tx=0x7f57c402a180 comp rx=0 tx=0).stop 2024-04-25T09:58:10.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 -- 172.21.15.148:0/1325130403 shutdown_connections 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1325130403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d40b92a0 0x7f57d40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1325130403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d415dbe0 0x7f57d40d0910 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 -- 172.21.15.148:0/1325130403 >> 172.21.15.148:0/1325130403 conn(0x7f57d4061a20 msgr2=0x7f57d4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 -- 172.21.15.148:0/1325130403 shutdown_connections 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.082+0000 7f57d97b7640 1 -- 172.21.15.148:0/1325130403 wait complete. 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.083+0000 7f57d97b7640 1 Processor -- start 2024-04-25T09:58:10.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.083+0000 7f57d97b7640 1 -- start start 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d97b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 0x7f57d414e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d97b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57d40b92a0 0x7f57d40cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d97b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d415dbe0 0x7f57d415f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d97b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b6ab0 con 0x7f57d40b2cb0 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d97b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40d2210 con 0x7f57d40b92a0 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d97b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b6c30 con 0x7f57d415dbe0 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 0x7f57d414e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 0x7f57d414e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34644/0 (socket says 172.21.15.148:34644) 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1187979373 learned_addr learned my addr 172.21.15.148:0/1187979373 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57cbfff640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57d40b92a0 0x7f57d40cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d37fe640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d415dbe0 0x7f57d415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d415dbe0 msgr2=0x7f57d415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d415dbe0 0x7f57d415f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57d40b92a0 msgr2=0x7f57d40cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57d40b92a0 0x7f57d40cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 -- 172.21.15.148:0/1187979373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40cc9f0 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57cbfff640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57d40b92a0 0x7f57d40cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.084+0000 7f57d2ffd640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 0x7f57d414e860 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000c00 tx=0x7f57c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.085+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4037040 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.085+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c4042070 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.085+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d40cccd0 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.085+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4030a70 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.085+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d40cd290 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.086+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d4073990 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.086+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57c4030c10 con 0x7f57d40b2cb0 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.086+0000 7f57cb7fe640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f57b406b4d0 0x7f57b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.086+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57c40aad80 con 0x7f57d40b2cb0 2024-04-25T09:58:10.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.087+0000 7f57cbfff640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f57b406b4d0 0x7f57b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.087+0000 7f57cbfff640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f57b406b4d0 0x7f57b406d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57d40ca3b0 tx=0x7f57bc007c10 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.091+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57c4045360 con 0x7f57d40b2cb0 2024-04-25T09:58:10.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.259+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57d40d4480 con 0x7f57b406b4d0 2024-04-25T09:58:10.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.265+0000 7f57cb7fe640 1 -- 172.21.15.148:0/1187979373 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252019 (secure 0 0 0) 0x7f57d40d4480 con 0x7f57b406b4d0 2024-04-25T09:58:10.271 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:10.272 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.267+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f57b406b4d0 msgr2=0x7f57b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.267+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f57b406b4d0 0x7f57b406d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57d40ca3b0 tx=0x7f57bc007c10 comp rx=0 tx=0).stop 2024-04-25T09:58:10.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 msgr2=0x7f57d414e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 0x7f57d414e860 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000c00 tx=0x7f57c40046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 shutdown_connections 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f57b406b4d0 0x7f57b406d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57d415dbe0 0x7f57d415f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57d40b92a0 0x7f57d40cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 --2- 172.21.15.148:0/1187979373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57d40b2cb0 0x7f57d414e860 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 >> 172.21.15.148:0/1187979373 conn(0x7f57d4061a20 msgr2=0x7f57d415b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 shutdown_connections 2024-04-25T09:58:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.268+0000 7f57d97b7640 1 -- 172.21.15.148:0/1187979373 wait complete. 2024-04-25T09:58:10.282 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-04-25T09:58:09.065740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290296,"kb_used_data":7548,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754684424,"statfs":{"total":773094113280,"available":772796850176,"internally_reserved":0,"allocated":7729152,"data_stored":4135648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.451845"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639271+0000","last_change":"2024-04-25T09:58:02.633388+0000","last_active":"2024-04-25T09:58:03.639271+0000","last_peered":"2024-04-25T09:58:03.639271+0000","last_clean":"2024-04-25T09:58:03.639271+0000","last_became_active":"2024-04-25T09:58:02.630538+0000","last_became_peered":"2024-04-25T09:58:02.630538+0000","last_unstale":"2024-04-25T09:58:03.639271+0000","last_undegraded":"2024-04-25T09:58:03.639271+0000","last_fullsized":"2024-04-25T09:58:03.639271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:03.538551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639138+0000","last_change":"2024-04-25T09:58:02.630115+0000","last_active":"2024-04-25T09:58:03.639138+0000","last_peered":"2024-04-25T09:58:03.639138+0000","last_clean":"2024-04-25T09:58:03.639138+0000","last_became_active":"2024-04-25T09:58:02.629845+0000","last_became_peered":"2024-04-25T09:58:02.629845+0000","last_unstale":"2024-04-25T09:58:03.639138+0000","last_undegraded":"2024-04-25T09:58:03.639138+0000","last_fullsized":"2024-04-25T09:58:03.639138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:34.062553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417206+0000","last_change":"2024-04-25T09:58:02.640691+0000","last_active":"2024-04-25T09:58:04.417206+0000","last_peered":"2024-04-25T09:58:04.417206+0000","last_clean":"2024-04-25T09:58:04.417206+0000","last_became_active":"2024-04-25T09:58:02.636656+0000","last_became_peered":"2024-04-25T09:58:02.636656+0000","last_unstale":"2024-04-25T09:58:04.417206+0000","last_undegraded":"2024-04-25T09:58:04.417206+0000","last_fullsized":"2024-04-25T09:58:04.417206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:44.946943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636332+0000","last_change":"2024-04-25T09:58:02.635932+0000","last_active":"2024-04-25T09:58:03.636332+0000","last_peered":"2024-04-25T09:58:03.636332+0000","last_clean":"2024-04-25T09:58:03.636332+0000","last_became_active":"2024-04-25T09:58:02.635644+0000","last_became_peered":"2024-04-25T09:58:02.635644+0000","last_unstale":"2024-04-25T09:58:03.636332+0000","last_undegraded":"2024-04-25T09:58:03.636332+0000","last_fullsized":"2024-04-25T09:58:03.636332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:44.573829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416356+0000","last_change":"2024-04-25T09:58:02.633603+0000","last_active":"2024-04-25T09:58:04.416356+0000","last_peered":"2024-04-25T09:58:04.416356+0000","last_clean":"2024-04-25T09:58:04.416356+0000","last_became_active":"2024-04-25T09:58:02.633220+0000","last_became_peered":"2024-04-25T09:58:02.633220+0000","last_unstale":"2024-04-25T09:58:04.416356+0000","last_undegraded":"2024-04-25T09:58:04.416356+0000","last_fullsized":"2024-04-25T09:58:04.416356+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:22.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416599+0000","last_change":"2024-04-25T09:58:02.634629+0000","last_active":"2024-04-25T09:58:04.416599+0000","last_peered":"2024-04-25T09:58:04.416599+0000","last_clean":"2024-04-25T09:58:04.416599+0000","last_became_active":"2024-04-25T09:58:02.634201+0000","last_became_peered":"2024-04-25T09:58:02.634201+0000","last_unstale":"2024-04-25T09:58:04.416599+0000","last_undegraded":"2024-04-25T09:58:04.416599+0000","last_fullsized":"2024-04-25T09:58:04.416599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:49.385120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416385+0000","last_change":"2024-04-25T09:58:02.632286+0000","last_active":"2024-04-25T09:58:04.416385+0000","last_peered":"2024-04-25T09:58:04.416385+0000","last_clean":"2024-04-25T09:58:04.416385+0000","last_became_active":"2024-04-25T09:58:02.632016+0000","last_became_peered":"2024-04-25T09:58:02.632016+0000","last_unstale":"2024-04-25T09:58:04.416385+0000","last_undegraded":"2024-04-25T09:58:04.416385+0000","last_fullsized":"2024-04-25T09:58:04.416385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:49.909331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417841+0000","last_change":"2024-04-25T09:58:02.633670+0000","last_active":"2024-04-25T09:58:04.417841+0000","last_peered":"2024-04-25T09:58:04.417841+0000","last_clean":"2024-04-25T09:58:04.417841+0000","last_became_active":"2024-04-25T09:58:02.633361+0000","last_became_peered":"2024-04-25T09:58:02.633361+0000","last_unstale":"2024-04-25T09:58:04.417841+0000","last_undegraded":"2024-04-25T09:58:04.417841+0000","last_fullsized":"2024-04-25T09:58:04.417841+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:09.029428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417371+0000","last_change":"2024-04-25T09:58:02.637028+0000","last_active":"2024-04-25T09:58:04.417371+0000","last_peered":"2024-04-25T09:58:04.417371+0000","last_clean":"2024-04-25T09:58:04.417371+0000","last_became_active":"2024-04-25T09:58:02.636241+0000","last_became_peered":"2024-04-25T09:58:02.636241+0000","last_unstale":"2024-04-25T09:58:04.417371+0000","last_undegraded":"2024-04-25T09:58:04.417371+0000","last_fullsized":"2024-04-25T09:58:04.417371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:59.052561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417867+0000","last_change":"2024-04-25T09:58:02.639677+0000","last_active":"2024-04-25T09:58:04.417867+0000","last_peered":"2024-04-25T09:58:04.417867+0000","last_clean":"2024-04-25T09:58:04.417867+0000","last_became_active":"2024-04-25T09:58:02.638166+0000","last_became_peered":"2024-04-25T09:58:02.638166+0000","last_unstale":"2024-04-25T09:58:04.417867+0000","last_undegraded":"2024-04-25T09:58:04.417867+0000","last_fullsized":"2024-04-25T09:58:04.417867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:25.333610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636314+0000","last_change":"2024-04-25T09:58:02.640546+0000","last_active":"2024-04-25T09:58:03.636314+0000","last_peered":"2024-04-25T09:58:03.636314+0000","last_clean":"2024-04-25T09:58:03.636314+0000","last_became_active":"2024-04-25T09:58:02.640241+0000","last_became_peered":"2024-04-25T09:58:02.640241+0000","last_unstale":"2024-04-25T09:58:03.636314+0000","last_undegraded":"2024-04-25T09:58:03.636314+0000","last_fullsized":"2024-04-25T09:58:03.636314+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:43.407254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417609+0000","last_change":"2024-04-25T09:58:02.631937+0000","last_active":"2024-04-25T09:58:04.417609+0000","last_peered":"2024-04-25T09:58:04.417609+0000","last_clean":"2024-04-25T09:58:04.417609+0000","last_became_active":"2024-04-25T09:58:02.631564+0000","last_became_peered":"2024-04-25T09:58:02.631564+0000","last_unstale":"2024-04-25T09:58:04.417609+0000","last_undegraded":"2024-04-25T09:58:04.417609+0000","last_fullsized":"2024-04-25T09:58:04.417609+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:10.536506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416306+0000","last_change":"2024-04-25T09:58:02.635170+0000","last_active":"2024-04-25T09:58:04.416306+0000","last_peered":"2024-04-25T09:58:04.416306+0000","last_clean":"2024-04-25T09:58:04.416306+0000","last_became_active":"2024-04-25T09:58:02.634905+0000","last_became_peered":"2024-04-25T09:58:02.634905+0000","last_unstale":"2024-04-25T09:58:04.416306+0000","last_undegraded":"2024-04-25T09:58:04.416306+0000","last_fullsized":"2024-04-25T09:58:04.416306+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:21.869689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417839+0000","last_change":"2024-04-25T09:58:02.637282+0000","last_active":"2024-04-25T09:58:04.417839+0000","last_peered":"2024-04-25T09:58:04.417839+0000","last_clean":"2024-04-25T09:58:04.417839+0000","last_became_active":"2024-04-25T09:58:02.636767+0000","last_became_peered":"2024-04-25T09:58:02.636767+0000","last_unstale":"2024-04-25T09:58:04.417839+0000","last_undegraded":"2024-04-25T09:58:04.417839+0000","last_fullsized":"2024-04-25T09:58:04.417839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:21.869689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417590+0000","last_change":"2024-04-25T09:58:02.637878+0000","last_active":"2024-04-25T09:58:04.417590+0000","last_peered":"2024-04-25T09:58:04.417590+0000","last_clean":"2024-04-25T09:58:04.417590+0000","last_became_active":"2024-04-25T09:58:02.637115+0000","last_became_peered":"2024-04-25T09:58:02.637115+0000","last_unstale":"2024-04-25T09:58:04.417590+0000","last_undegraded":"2024-04-25T09:58:04.417590+0000","last_fullsized":"2024-04-25T09:58:04.417590+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:52.523658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417138+0000","last_change":"2024-04-25T09:58:02.628076+0000","last_active":"2024-04-25T09:58:04.417138+0000","last_peered":"2024-04-25T09:58:04.417138+0000","last_clean":"2024-04-25T09:58:04.417138+0000","last_became_active":"2024-04-25T09:58:02.627768+0000","last_became_peered":"2024-04-25T09:58:02.627768+0000","last_unstale":"2024-04-25T09:58:04.417138+0000","last_undegraded":"2024-04-25T09:58:04.417138+0000","last_fullsized":"2024-04-25T09:58:04.417138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:08.677569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639272+0000","last_change":"2024-04-25T09:58:02.642730+0000","last_active":"2024-04-25T09:58:03.639272+0000","last_peered":"2024-04-25T09:58:03.639272+0000","last_clean":"2024-04-25T09:58:03.639272+0000","last_became_active":"2024-04-25T09:58:02.641957+0000","last_became_peered":"2024-04-25T09:58:02.641957+0000","last_unstale":"2024-04-25T09:58:03.639272+0000","last_undegraded":"2024-04-25T09:58:03.639272+0000","last_fullsized":"2024-04-25T09:58:03.639272+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:32.730922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636059+0000","last_change":"2024-04-25T09:58:02.636020+0000","last_active":"2024-04-25T09:58:03.636059+0000","last_peered":"2024-04-25T09:58:03.636059+0000","last_clean":"2024-04-25T09:58:03.636059+0000","last_became_active":"2024-04-25T09:58:02.635461+0000","last_became_peered":"2024-04-25T09:58:02.635461+0000","last_unstale":"2024-04-25T09:58:03.636059+0000","last_undegraded":"2024-04-25T09:58:03.636059+0000","last_fullsized":"2024-04-25T09:58:03.636059+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:08.210135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416823+0000","last_change":"2024-04-25T09:58:02.629798+0000","last_active":"2024-04-25T09:58:04.416823+0000","last_peered":"2024-04-25T09:58:04.416823+0000","last_clean":"2024-04-25T09:58:04.416823+0000","last_became_active":"2024-04-25T09:58:02.629408+0000","last_became_peered":"2024-04-25T09:58:02.629408+0000","last_unstale":"2024-04-25T09:58:04.416823+0000","last_undegraded":"2024-04-25T09:58:04.416823+0000","last_fullsized":"2024-04-25T09:58:04.416823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:48.266642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416583+0000","last_change":"2024-04-25T09:58:02.636801+0000","last_active":"2024-04-25T09:58:04.416583+0000","last_peered":"2024-04-25T09:58:04.416583+0000","last_clean":"2024-04-25T09:58:04.416583+0000","last_became_active":"2024-04-25T09:58:02.636523+0000","last_became_peered":"2024-04-25T09:58:02.636523+0000","last_unstale":"2024-04-25T09:58:04.416583+0000","last_undegraded":"2024-04-25T09:58:04.416583+0000","last_fullsized":"2024-04-25T09:58:04.416583+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:16.998736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416479+0000","last_change":"2024-04-25T09:58:02.638669+0000","last_active":"2024-04-25T09:58:04.416479+0000","last_peered":"2024-04-25T09:58:04.416479+0000","last_clean":"2024-04-25T09:58:04.416479+0000","last_became_active":"2024-04-25T09:58:02.638363+0000","last_became_peered":"2024-04-25T09:58:02.638363+0000","last_unstale":"2024-04-25T09:58:04.416479+0000","last_undegraded":"2024-04-25T09:58:04.416479+0000","last_fullsized":"2024-04-25T09:58:04.416479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:17.387746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416426+0000","last_change":"2024-04-25T09:58:02.638223+0000","last_active":"2024-04-25T09:58:04.416426+0000","last_peered":"2024-04-25T09:58:04.416426+0000","last_clean":"2024-04-25T09:58:04.416426+0000","last_became_active":"2024-04-25T09:58:02.637969+0000","last_became_peered":"2024-04-25T09:58:02.637969+0000","last_unstale":"2024-04-25T09:58:04.416426+0000","last_undegraded":"2024-04-25T09:58:04.416426+0000","last_fullsized":"2024-04-25T09:58:04.416426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.398866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416209+0000","last_change":"2024-04-25T09:58:02.636648+0000","last_active":"2024-04-25T09:58:04.416209+0000","last_peered":"2024-04-25T09:58:04.416209+0000","last_clean":"2024-04-25T09:58:04.416209+0000","last_became_active":"2024-04-25T09:58:02.636366+0000","last_became_peered":"2024-04-25T09:58:02.636366+0000","last_unstale":"2024-04-25T09:58:04.416209+0000","last_undegraded":"2024-04-25T09:58:04.416209+0000","last_fullsized":"2024-04-25T09:58:04.416209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:55.687289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416719+0000","last_change":"2024-04-25T09:58:02.645710+0000","last_active":"2024-04-25T09:58:04.416719+0000","last_peered":"2024-04-25T09:58:04.416719+0000","last_clean":"2024-04-25T09:58:04.416719+0000","last_became_active":"2024-04-25T09:58:02.645440+0000","last_became_peered":"2024-04-25T09:58:02.645440+0000","last_unstale":"2024-04-25T09:58:04.416719+0000","last_undegraded":"2024-04-25T09:58:04.416719+0000","last_fullsized":"2024-04-25T09:58:04.416719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:26.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417680+0000","last_change":"2024-04-25T09:58:02.632192+0000","last_active":"2024-04-25T09:58:04.417680+0000","last_peered":"2024-04-25T09:58:04.417680+0000","last_clean":"2024-04-25T09:58:04.417680+0000","last_became_active":"2024-04-25T09:58:02.631215+0000","last_became_peered":"2024-04-25T09:58:02.631215+0000","last_unstale":"2024-04-25T09:58:04.417680+0000","last_undegraded":"2024-04-25T09:58:04.417680+0000","last_fullsized":"2024-04-25T09:58:04.417680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:22.051916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417185+0000","last_change":"2024-04-25T09:58:02.631777+0000","last_active":"2024-04-25T09:58:04.417185+0000","last_peered":"2024-04-25T09:58:04.417185+0000","last_clean":"2024-04-25T09:58:04.417185+0000","last_became_active":"2024-04-25T09:58:02.631455+0000","last_became_peered":"2024-04-25T09:58:02.631455+0000","last_unstale":"2024-04-25T09:58:04.417185+0000","last_undegraded":"2024-04-25T09:58:04.417185+0000","last_fullsized":"2024-04-25T09:58:04.417185+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:15.450522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636032+0000","last_change":"2024-04-25T09:58:02.630502+0000","last_active":"2024-04-25T09:58:03.636032+0000","last_peered":"2024-04-25T09:58:03.636032+0000","last_clean":"2024-04-25T09:58:03.636032+0000","last_became_active":"2024-04-25T09:58:02.629942+0000","last_became_peered":"2024-04-25T09:58:02.629942+0000","last_unstale":"2024-04-25T09:58:03.636032+0000","last_undegraded":"2024-04-25T09:58:03.636032+0000","last_fullsized":"2024-04-25T09:58:03.636032+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:22.760147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639066+0000","last_change":"2024-04-25T09:58:02.630212+0000","last_active":"2024-04-25T09:58:03.639066+0000","last_peered":"2024-04-25T09:58:03.639066+0000","last_clean":"2024-04-25T09:58:03.639066+0000","last_became_active":"2024-04-25T09:58:02.629938+0000","last_became_peered":"2024-04-25T09:58:02.629938+0000","last_unstale":"2024-04-25T09:58:03.639066+0000","last_undegraded":"2024-04-25T09:58:03.639066+0000","last_fullsized":"2024-04-25T09:58:03.639066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:22.920228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416814+0000","last_change":"2024-04-25T09:58:02.636592+0000","last_active":"2024-04-25T09:58:04.416814+0000","last_peered":"2024-04-25T09:58:04.416814+0000","last_clean":"2024-04-25T09:58:04.416814+0000","last_became_active":"2024-04-25T09:58:02.636133+0000","last_became_peered":"2024-04-25T09:58:02.636133+0000","last_unstale":"2024-04-25T09:58:04.416814+0000","last_undegraded":"2024-04-25T09:58:04.416814+0000","last_fullsized":"2024-04-25T09:58:04.416814+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:54.850858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417226+0000","last_change":"2024-04-25T09:58:02.632712+0000","last_active":"2024-04-25T09:58:04.417226+0000","last_peered":"2024-04-25T09:58:04.417226+0000","last_clean":"2024-04-25T09:58:04.417226+0000","last_became_active":"2024-04-25T09:58:02.632405+0000","last_became_peered":"2024-04-25T09:58:02.632405+0000","last_unstale":"2024-04-25T09:58:04.417226+0000","last_undegraded":"2024-04-25T09:58:04.417226+0000","last_fullsized":"2024-04-25T09:58:04.417226+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:17.183045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.635758+0000","last_change":"2024-04-25T09:58:02.630353+0000","last_active":"2024-04-25T09:58:03.635758+0000","last_peered":"2024-04-25T09:58:03.635758+0000","last_clean":"2024-04-25T09:58:03.635758+0000","last_became_active":"2024-04-25T09:58:02.629949+0000","last_became_peered":"2024-04-25T09:58:02.629949+0000","last_unstale":"2024-04-25T09:58:03.635758+0000","last_undegraded":"2024-04-25T09:58:03.635758+0000","last_fullsized":"2024-04-25T09:58:03.635758+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:51.725672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416281+0000","last_change":"2024-04-25T09:58:02.635138+0000","last_active":"2024-04-25T09:58:04.416281+0000","last_peered":"2024-04-25T09:58:04.416281+0000","last_clean":"2024-04-25T09:58:04.416281+0000","last_became_active":"2024-04-25T09:58:02.634816+0000","last_became_peered":"2024-04-25T09:58:02.634816+0000","last_unstale":"2024-04-25T09:58:04.416281+0000","last_undegraded":"2024-04-25T09:58:04.416281+0000","last_fullsized":"2024-04-25T09:58:04.416281+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:25.333610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416939+0000","last_change":"2024-04-25T09:58:02.630590+0000","last_active":"2024-04-25T09:58:04.416939+0000","last_peered":"2024-04-25T09:58:04.416939+0000","last_clean":"2024-04-25T09:58:04.416939+0000","last_became_active":"2024-04-25T09:58:02.630319+0000","last_became_peered":"2024-04-25T09:58:02.630319+0000","last_unstale":"2024-04-25T09:58:04.416939+0000","last_undegraded":"2024-04-25T09:58:04.416939+0000","last_fullsized":"2024-04-25T09:58:04.416939+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:08.210135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639373+0000","last_change":"2024-04-25T09:58:01.634003+0000","last_active":"2024-04-25T09:58:03.639373+0000","last_peered":"2024-04-25T09:58:03.639373+0000","last_clean":"2024-04-25T09:58:03.639373+0000","last_became_active":"2024-04-25T09:58:01.633775+0000","last_became_peered":"2024-04-25T09:58:01.633775+0000","last_unstale":"2024-04-25T09:58:03.639373+0000","last_undegraded":"2024-04-25T09:58:03.639373+0000","last_fullsized":"2024-04-25T09:58:03.639373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:28.121933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636328+0000","last_change":"2024-04-25T09:58:02.635769+0000","last_active":"2024-04-25T09:58:03.636328+0000","last_peered":"2024-04-25T09:58:03.636328+0000","last_clean":"2024-04-25T09:58:03.636328+0000","last_became_active":"2024-04-25T09:58:02.635475+0000","last_became_peered":"2024-04-25T09:58:02.635475+0000","last_unstale":"2024-04-25T09:58:03.636328+0000","last_undegraded":"2024-04-25T09:58:03.636328+0000","last_fullsized":"2024-04-25T09:58:03.636328+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:48.266642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417138+0000","last_change":"2024-04-25T09:58:01.632332+0000","last_active":"2024-04-25T09:58:04.417138+0000","last_peered":"2024-04-25T09:58:04.417138+0000","last_clean":"2024-04-25T09:58:04.417138+0000","last_became_active":"2024-04-25T09:58:01.632054+0000","last_became_peered":"2024-04-25T09:58:01.632054+0000","last_unstale":"2024-04-25T09:58:04.417138+0000","last_undegraded":"2024-04-25T09:58:04.417138+0000","last_fullsized":"2024-04-25T09:58:04.417138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:17.182378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417280+0000","last_change":"2024-04-25T09:58:02.638414+0000","last_active":"2024-04-25T09:58:04.417280+0000","last_peered":"2024-04-25T09:58:04.417280+0000","last_clean":"2024-04-25T09:58:04.417280+0000","last_became_active":"2024-04-25T09:58:02.638151+0000","last_became_peered":"2024-04-25T09:58:02.638151+0000","last_unstale":"2024-04-25T09:58:04.417280+0000","last_undegraded":"2024-04-25T09:58:04.417280+0000","last_fullsized":"2024-04-25T09:58:04.417280+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:17.183045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417126+0000","last_change":"2024-04-25T09:58:01.631523+0000","last_active":"2024-04-25T09:58:04.417126+0000","last_peered":"2024-04-25T09:58:04.417126+0000","last_clean":"2024-04-25T09:58:04.417126+0000","last_became_active":"2024-04-25T09:58:01.631180+0000","last_became_peered":"2024-04-25T09:58:01.631180+0000","last_unstale":"2024-04-25T09:58:04.417126+0000","last_undegraded":"2024-04-25T09:58:04.417126+0000","last_fullsized":"2024-04-25T09:58:04.417126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:45.031770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417072+0000","last_change":"2024-04-25T09:58:02.637675+0000","last_active":"2024-04-25T09:58:04.417072+0000","last_peered":"2024-04-25T09:58:04.417072+0000","last_clean":"2024-04-25T09:58:04.417072+0000","last_became_active":"2024-04-25T09:58:02.637408+0000","last_became_peered":"2024-04-25T09:58:02.637408+0000","last_unstale":"2024-04-25T09:58:04.417072+0000","last_undegraded":"2024-04-25T09:58:04.417072+0000","last_fullsized":"2024-04-25T09:58:04.417072+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:05.155089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416798+0000","last_change":"2024-04-25T09:58:01.628004+0000","last_active":"2024-04-25T09:58:04.416798+0000","last_peered":"2024-04-25T09:58:04.416798+0000","last_clean":"2024-04-25T09:58:04.416798+0000","last_became_active":"2024-04-25T09:58:01.627143+0000","last_became_peered":"2024-04-25T09:58:01.627143+0000","last_unstale":"2024-04-25T09:58:04.416798+0000","last_undegraded":"2024-04-25T09:58:04.416798+0000","last_fullsized":"2024-04-25T09:58:04.416798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:46.106436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417162+0000","last_change":"2024-04-25T09:58:02.631773+0000","last_active":"2024-04-25T09:58:04.417162+0000","last_peered":"2024-04-25T09:58:04.417162+0000","last_clean":"2024-04-25T09:58:04.417162+0000","last_became_active":"2024-04-25T09:58:02.630042+0000","last_became_peered":"2024-04-25T09:58:02.630042+0000","last_unstale":"2024-04-25T09:58:04.417162+0000","last_undegraded":"2024-04-25T09:58:04.417162+0000","last_fullsized":"2024-04-25T09:58:04.417162+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:16.428544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416594+0000","last_change":"2024-04-25T09:58:01.631301+0000","last_active":"2024-04-25T09:58:04.416594+0000","last_peered":"2024-04-25T09:58:04.416594+0000","last_clean":"2024-04-25T09:58:04.416594+0000","last_became_active":"2024-04-25T09:58:01.631045+0000","last_became_peered":"2024-04-25T09:58:01.631045+0000","last_unstale":"2024-04-25T09:58:04.416594+0000","last_undegraded":"2024-04-25T09:58:04.416594+0000","last_fullsized":"2024-04-25T09:58:04.416594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:20.867411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.638888+0000","last_change":"2024-04-25T09:58:02.640624+0000","last_active":"2024-04-25T09:58:03.638888+0000","last_peered":"2024-04-25T09:58:03.638888+0000","last_clean":"2024-04-25T09:58:03.638888+0000","last_became_active":"2024-04-25T09:58:02.640364+0000","last_became_peered":"2024-04-25T09:58:02.640364+0000","last_unstale":"2024-04-25T09:58:03.638888+0000","last_undegraded":"2024-04-25T09:58:03.638888+0000","last_fullsized":"2024-04-25T09:58:03.638888+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:44.573829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416789+0000","last_change":"2024-04-25T09:58:01.625471+0000","last_active":"2024-04-25T09:58:04.416789+0000","last_peered":"2024-04-25T09:58:04.416789+0000","last_clean":"2024-04-25T09:58:04.416789+0000","last_became_active":"2024-04-25T09:58:01.625198+0000","last_became_peered":"2024-04-25T09:58:01.625198+0000","last_unstale":"2024-04-25T09:58:04.416789+0000","last_undegraded":"2024-04-25T09:58:04.416789+0000","last_fullsized":"2024-04-25T09:58:04.416789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:48.382841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416739+0000","last_change":"2024-04-25T09:58:02.631924+0000","last_active":"2024-04-25T09:58:04.416739+0000","last_peered":"2024-04-25T09:58:04.416739+0000","last_clean":"2024-04-25T09:58:04.416739+0000","last_became_active":"2024-04-25T09:58:02.630684+0000","last_became_peered":"2024-04-25T09:58:02.630684+0000","last_unstale":"2024-04-25T09:58:04.416739+0000","last_undegraded":"2024-04-25T09:58:04.416739+0000","last_fullsized":"2024-04-25T09:58:04.416739+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:43.407254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416227+0000","last_change":"2024-04-25T09:58:01.631164+0000","last_active":"2024-04-25T09:58:04.416227+0000","last_peered":"2024-04-25T09:58:04.416227+0000","last_clean":"2024-04-25T09:58:04.416227+0000","last_became_active":"2024-04-25T09:58:01.630883+0000","last_became_peered":"2024-04-25T09:58:01.630883+0000","last_unstale":"2024-04-25T09:58:04.416227+0000","last_undegraded":"2024-04-25T09:58:04.416227+0000","last_fullsized":"2024-04-25T09:58:04.416227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:24.331332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416496+0000","last_change":"2024-04-25T09:58:02.637788+0000","last_active":"2024-04-25T09:58:04.416496+0000","last_peered":"2024-04-25T09:58:04.416496+0000","last_clean":"2024-04-25T09:58:04.416496+0000","last_became_active":"2024-04-25T09:58:02.637514+0000","last_became_peered":"2024-04-25T09:58:02.637514+0000","last_unstale":"2024-04-25T09:58:04.416496+0000","last_undegraded":"2024-04-25T09:58:04.416496+0000","last_fullsized":"2024-04-25T09:58:04.416496+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:27.937998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416222+0000","last_change":"2024-04-25T09:58:01.632019+0000","last_active":"2024-04-25T09:58:04.416222+0000","last_peered":"2024-04-25T09:58:04.416222+0000","last_clean":"2024-04-25T09:58:04.416222+0000","last_became_active":"2024-04-25T09:58:01.631743+0000","last_became_peered":"2024-04-25T09:58:01.631743+0000","last_unstale":"2024-04-25T09:58:04.416222+0000","last_undegraded":"2024-04-25T09:58:04.416222+0000","last_fullsized":"2024-04-25T09:58:04.416222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:01.279838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639068+0000","last_change":"2024-04-25T09:58:02.630437+0000","last_active":"2024-04-25T09:58:03.639068+0000","last_peered":"2024-04-25T09:58:03.639068+0000","last_clean":"2024-04-25T09:58:03.639068+0000","last_became_active":"2024-04-25T09:58:02.629477+0000","last_became_peered":"2024-04-25T09:58:02.629477+0000","last_unstale":"2024-04-25T09:58:03.639068+0000","last_undegraded":"2024-04-25T09:58:03.639068+0000","last_fullsized":"2024-04-25T09:58:03.639068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:05.497205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417477+0000","last_change":"2024-04-25T09:58:01.635280+0000","last_active":"2024-04-25T09:58:04.417477+0000","last_peered":"2024-04-25T09:58:04.417477+0000","last_clean":"2024-04-25T09:58:04.417477+0000","last_became_active":"2024-04-25T09:58:01.635034+0000","last_became_peered":"2024-04-25T09:58:01.635034+0000","last_unstale":"2024-04-25T09:58:04.417477+0000","last_undegraded":"2024-04-25T09:58:04.417477+0000","last_fullsized":"2024-04-25T09:58:04.417477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:47.264364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416711+0000","last_change":"2024-04-25T09:58:02.637429+0000","last_active":"2024-04-25T09:58:04.416711+0000","last_peered":"2024-04-25T09:58:04.416711+0000","last_clean":"2024-04-25T09:58:04.416711+0000","last_became_active":"2024-04-25T09:58:02.637157+0000","last_became_peered":"2024-04-25T09:58:02.637157+0000","last_unstale":"2024-04-25T09:58:04.416711+0000","last_undegraded":"2024-04-25T09:58:04.416711+0000","last_fullsized":"2024-04-25T09:58:04.416711+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:42.287310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636265+0000","last_change":"2024-04-25T09:58:01.629933+0000","last_active":"2024-04-25T09:58:03.636265+0000","last_peered":"2024-04-25T09:58:03.636265+0000","last_clean":"2024-04-25T09:58:03.636265+0000","last_became_active":"2024-04-25T09:58:01.629617+0000","last_became_peered":"2024-04-25T09:58:01.629617+0000","last_unstale":"2024-04-25T09:58:03.636265+0000","last_undegraded":"2024-04-25T09:58:03.636265+0000","last_fullsized":"2024-04-25T09:58:03.636265+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:32.757111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416905+0000","last_change":"2024-04-25T09:58:02.628651+0000","last_active":"2024-04-25T09:58:04.416905+0000","last_peered":"2024-04-25T09:58:04.416905+0000","last_clean":"2024-04-25T09:58:04.416905+0000","last_became_active":"2024-04-25T09:58:02.628379+0000","last_became_peered":"2024-04-25T09:58:02.628379+0000","last_unstale":"2024-04-25T09:58:04.416905+0000","last_undegraded":"2024-04-25T09:58:04.416905+0000","last_fullsized":"2024-04-25T09:58:04.416905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:44.573829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417330+0000","last_change":"2024-04-25T09:58:01.638189+0000","last_active":"2024-04-25T09:58:04.417330+0000","last_peered":"2024-04-25T09:58:04.417330+0000","last_clean":"2024-04-25T09:58:04.417330+0000","last_became_active":"2024-04-25T09:58:01.637987+0000","last_became_peered":"2024-04-25T09:58:01.637987+0000","last_unstale":"2024-04-25T09:58:04.417330+0000","last_undegraded":"2024-04-25T09:58:04.417330+0000","last_fullsized":"2024-04-25T09:58:04.417330+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:43.571550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636206+0000","last_change":"2024-04-25T09:58:02.639690+0000","last_active":"2024-04-25T09:58:03.636206+0000","last_peered":"2024-04-25T09:58:03.636206+0000","last_clean":"2024-04-25T09:58:03.636206+0000","last_became_active":"2024-04-25T09:58:02.639411+0000","last_became_peered":"2024-04-25T09:58:02.639411+0000","last_unstale":"2024-04-25T09:58:03.636206+0000","last_undegraded":"2024-04-25T09:58:03.636206+0000","last_fullsized":"2024-04-25T09:58:03.636206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:53.455144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639179+0000","last_change":"2024-04-25T09:58:01.635429+0000","last_active":"2024-04-25T09:58:03.639179+0000","last_peered":"2024-04-25T09:58:03.639179+0000","last_clean":"2024-04-25T09:58:03.639179+0000","last_became_active":"2024-04-25T09:58:01.635077+0000","last_became_peered":"2024-04-25T09:58:01.635077+0000","last_unstale":"2024-04-25T09:58:03.639179+0000","last_undegraded":"2024-04-25T09:58:03.639179+0000","last_fullsized":"2024-04-25T09:58:03.639179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:33.416454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.635707+0000","last_change":"2024-04-25T09:58:02.636789+0000","last_active":"2024-04-25T09:58:03.635707+0000","last_peered":"2024-04-25T09:58:03.635707+0000","last_clean":"2024-04-25T09:58:03.635707+0000","last_became_active":"2024-04-25T09:58:02.636472+0000","last_became_peered":"2024-04-25T09:58:02.636472+0000","last_unstale":"2024-04-25T09:58:03.635707+0000","last_undegraded":"2024-04-25T09:58:03.635707+0000","last_fullsized":"2024-04-25T09:58:03.635707+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:29.124211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416931+0000","last_change":"2024-04-25T09:58:01.633445+0000","last_active":"2024-04-25T09:58:04.416931+0000","last_peered":"2024-04-25T09:58:04.416931+0000","last_clean":"2024-04-25T09:58:04.416931+0000","last_became_active":"2024-04-25T09:58:01.633154+0000","last_became_peered":"2024-04-25T09:58:01.633154+0000","last_unstale":"2024-04-25T09:58:04.416931+0000","last_undegraded":"2024-04-25T09:58:04.416931+0000","last_fullsized":"2024-04-25T09:58:04.416931+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:45.503795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416216+0000","last_change":"2024-04-25T09:58:02.639470+0000","last_active":"2024-04-25T09:58:04.416216+0000","last_peered":"2024-04-25T09:58:04.416216+0000","last_clean":"2024-04-25T09:58:04.416216+0000","last_became_active":"2024-04-25T09:58:02.639169+0000","last_became_peered":"2024-04-25T09:58:02.639169+0000","last_unstale":"2024-04-25T09:58:04.416216+0000","last_undegraded":"2024-04-25T09:58:04.416216+0000","last_fullsized":"2024-04-25T09:58:04.416216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:21.037602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417342+0000","last_change":"2024-04-25T09:58:01.626067+0000","last_active":"2024-04-25T09:58:04.417342+0000","last_peered":"2024-04-25T09:58:04.417342+0000","last_clean":"2024-04-25T09:58:04.417342+0000","last_became_active":"2024-04-25T09:58:01.625629+0000","last_became_peered":"2024-04-25T09:58:01.625629+0000","last_unstale":"2024-04-25T09:58:04.417342+0000","last_undegraded":"2024-04-25T09:58:04.417342+0000","last_fullsized":"2024-04-25T09:58:04.417342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:04.494927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416917+0000","last_change":"2024-04-25T09:58:02.630968+0000","last_active":"2024-04-25T09:58:04.416917+0000","last_peered":"2024-04-25T09:58:04.416917+0000","last_clean":"2024-04-25T09:58:04.416917+0000","last_became_active":"2024-04-25T09:58:02.630106+0000","last_became_peered":"2024-04-25T09:58:02.630106+0000","last_unstale":"2024-04-25T09:58:04.416917+0000","last_undegraded":"2024-04-25T09:58:04.416917+0000","last_fullsized":"2024-04-25T09:58:04.416917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:25.696390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417844+0000","last_change":"2024-04-25T09:58:01.631726+0000","last_active":"2024-04-25T09:58:04.417844+0000","last_peered":"2024-04-25T09:58:04.417844+0000","last_clean":"2024-04-25T09:58:04.417844+0000","last_became_active":"2024-04-25T09:58:01.631427+0000","last_became_peered":"2024-04-25T09:58:01.631427+0000","last_unstale":"2024-04-25T09:58:04.417844+0000","last_undegraded":"2024-04-25T09:58:04.417844+0000","last_fullsized":"2024-04-25T09:58:04.417844+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.452865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639477+0000","last_change":"2024-04-25T09:58:02.639067+0000","last_active":"2024-04-25T09:58:03.639477+0000","last_peered":"2024-04-25T09:58:03.639477+0000","last_clean":"2024-04-25T09:58:03.639477+0000","last_became_active":"2024-04-25T09:58:02.638786+0000","last_became_peered":"2024-04-25T09:58:02.638786+0000","last_unstale":"2024-04-25T09:58:03.639477+0000","last_undegraded":"2024-04-25T09:58:03.639477+0000","last_fullsized":"2024-04-25T09:58:03.639477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:46.435359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.635735+0000","last_change":"2024-04-25T09:58:01.631264+0000","last_active":"2024-04-25T09:58:03.635735+0000","last_peered":"2024-04-25T09:58:03.635735+0000","last_clean":"2024-04-25T09:58:03.635735+0000","last_became_active":"2024-04-25T09:58:01.630967+0000","last_became_peered":"2024-04-25T09:58:01.630967+0000","last_unstale":"2024-04-25T09:58:03.635735+0000","last_undegraded":"2024-04-25T09:58:03.635735+0000","last_fullsized":"2024-04-25T09:58:03.635735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:01.279838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417671+0000","last_change":"2024-04-25T09:58:02.636845+0000","last_active":"2024-04-25T09:58:04.417671+0000","last_peered":"2024-04-25T09:58:04.417671+0000","last_clean":"2024-04-25T09:58:04.417671+0000","last_became_active":"2024-04-25T09:58:02.636215+0000","last_became_peered":"2024-04-25T09:58:02.636215+0000","last_unstale":"2024-04-25T09:58:04.417671+0000","last_undegraded":"2024-04-25T09:58:04.417671+0000","last_fullsized":"2024-04-25T09:58:04.417671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:21.037602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417330+0000","last_change":"2024-04-25T09:58:01.628504+0000","last_active":"2024-04-25T09:58:04.417330+0000","last_peered":"2024-04-25T09:58:04.417330+0000","last_clean":"2024-04-25T09:58:04.417330+0000","last_became_active":"2024-04-25T09:58:01.628077+0000","last_became_peered":"2024-04-25T09:58:01.628077+0000","last_unstale":"2024-04-25T09:58:04.417330+0000","last_undegraded":"2024-04-25T09:58:04.417330+0000","last_fullsized":"2024-04-25T09:58:04.417330+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:31.728644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416961+0000","last_change":"2024-04-25T09:58:02.632969+0000","last_active":"2024-04-25T09:58:04.416961+0000","last_peered":"2024-04-25T09:58:04.416961+0000","last_clean":"2024-04-25T09:58:04.416961+0000","last_became_active":"2024-04-25T09:58:02.632650+0000","last_became_peered":"2024-04-25T09:58:02.632650+0000","last_unstale":"2024-04-25T09:58:04.416961+0000","last_undegraded":"2024-04-25T09:58:04.416961+0000","last_fullsized":"2024-04-25T09:58:04.416961+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:22.920228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417769+0000","last_change":"2024-04-25T09:58:01.634057+0000","last_active":"2024-04-25T09:58:04.417769+0000","last_peered":"2024-04-25T09:58:04.417769+0000","last_clean":"2024-04-25T09:58:04.417769+0000","last_became_active":"2024-04-25T09:58:01.633777+0000","last_became_peered":"2024-04-25T09:58:01.633777+0000","last_unstale":"2024-04-25T09:58:04.417769+0000","last_undegraded":"2024-04-25T09:58:04.417769+0000","last_fullsized":"2024-04-25T09:58:04.417769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:24.694112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.636212+0000","last_change":"2024-04-25T09:58:02.636242+0000","last_active":"2024-04-25T09:58:03.636212+0000","last_peered":"2024-04-25T09:58:03.636212+0000","last_clean":"2024-04-25T09:58:03.636212+0000","last_became_active":"2024-04-25T09:58:02.635842+0000","last_became_peered":"2024-04-25T09:58:02.635842+0000","last_unstale":"2024-04-25T09:58:03.636212+0000","last_undegraded":"2024-04-25T09:58:03.636212+0000","last_fullsized":"2024-04-25T09:58:03.636212+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.506073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417447+0000","last_change":"2024-04-25T09:58:01.635908+0000","last_active":"2024-04-25T09:58:04.417447+0000","last_peered":"2024-04-25T09:58:04.417447+0000","last_clean":"2024-04-25T09:58:04.417447+0000","last_became_active":"2024-04-25T09:58:01.635720+0000","last_became_peered":"2024-04-25T09:58:01.635720+0000","last_unstale":"2024-04-25T09:58:04.417447+0000","last_undegraded":"2024-04-25T09:58:04.417447+0000","last_fullsized":"2024-04-25T09:58:04.417447+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:21.917950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416671+0000","last_change":"2024-04-25T09:58:02.639325+0000","last_active":"2024-04-25T09:58:04.416671+0000","last_peered":"2024-04-25T09:58:04.416671+0000","last_clean":"2024-04-25T09:58:04.416671+0000","last_became_active":"2024-04-25T09:58:02.639040+0000","last_became_peered":"2024-04-25T09:58:02.639040+0000","last_unstale":"2024-04-25T09:58:04.416671+0000","last_undegraded":"2024-04-25T09:58:04.416671+0000","last_fullsized":"2024-04-25T09:58:04.416671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:46.506073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416684+0000","last_change":"2024-04-25T09:58:01.626255+0000","last_active":"2024-04-25T09:58:04.416684+0000","last_peered":"2024-04-25T09:58:04.416684+0000","last_clean":"2024-04-25T09:58:04.416684+0000","last_became_active":"2024-04-25T09:58:01.625985+0000","last_became_peered":"2024-04-25T09:58:01.625985+0000","last_unstale":"2024-04-25T09:58:04.416684+0000","last_undegraded":"2024-04-25T09:58:04.416684+0000","last_fullsized":"2024-04-25T09:58:04.416684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:16.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416435+0000","last_change":"2024-04-25T09:58:02.634314+0000","last_active":"2024-04-25T09:58:04.416435+0000","last_peered":"2024-04-25T09:58:04.416435+0000","last_clean":"2024-04-25T09:58:04.416435+0000","last_became_active":"2024-04-25T09:58:02.633913+0000","last_became_peered":"2024-04-25T09:58:02.633913+0000","last_unstale":"2024-04-25T09:58:04.416435+0000","last_undegraded":"2024-04-25T09:58:04.416435+0000","last_fullsized":"2024-04-25T09:58:04.416435+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:17.387746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416792+0000","last_change":"2024-04-25T09:58:01.629363+0000","last_active":"2024-04-25T09:58:04.416792+0000","last_peered":"2024-04-25T09:58:04.416792+0000","last_clean":"2024-04-25T09:58:04.416792+0000","last_became_active":"2024-04-25T09:58:01.629077+0000","last_became_peered":"2024-04-25T09:58:01.629077+0000","last_unstale":"2024-04-25T09:58:04.416792+0000","last_undegraded":"2024-04-25T09:58:04.416792+0000","last_fullsized":"2024-04-25T09:58:04.416792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:25.869174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416409+0000","last_change":"2024-04-25T09:58:02.631872+0000","last_active":"2024-04-25T09:58:04.416409+0000","last_peered":"2024-04-25T09:58:04.416409+0000","last_clean":"2024-04-25T09:58:04.416409+0000","last_became_active":"2024-04-25T09:58:02.631535+0000","last_became_peered":"2024-04-25T09:58:02.631535+0000","last_unstale":"2024-04-25T09:58:04.416409+0000","last_undegraded":"2024-04-25T09:58:04.416409+0000","last_fullsized":"2024-04-25T09:58:04.416409+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:25.333610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417442+0000","last_change":"2024-04-25T09:58:01.625784+0000","last_active":"2024-04-25T09:58:04.417442+0000","last_peered":"2024-04-25T09:58:04.417442+0000","last_clean":"2024-04-25T09:58:04.417442+0000","last_became_active":"2024-04-25T09:58:01.625256+0000","last_became_peered":"2024-04-25T09:58:01.625256+0000","last_unstale":"2024-04-25T09:58:04.417442+0000","last_undegraded":"2024-04-25T09:58:04.417442+0000","last_fullsized":"2024-04-25T09:58:04.417442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:33.060275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417088+0000","last_change":"2024-04-25T09:58:02.631786+0000","last_active":"2024-04-25T09:58:04.417088+0000","last_peered":"2024-04-25T09:58:04.417088+0000","last_clean":"2024-04-25T09:58:04.417088+0000","last_became_active":"2024-04-25T09:58:02.631094+0000","last_became_peered":"2024-04-25T09:58:02.631094+0000","last_unstale":"2024-04-25T09:58:04.417088+0000","last_undegraded":"2024-04-25T09:58:04.417088+0000","last_fullsized":"2024-04-25T09:58:04.417088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:34.062553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416905+0000","last_change":"2024-04-25T09:58:01.631198+0000","last_active":"2024-04-25T09:58:04.416905+0000","last_peered":"2024-04-25T09:58:04.416905+0000","last_clean":"2024-04-25T09:58:04.416905+0000","last_became_active":"2024-04-25T09:58:01.630948+0000","last_became_peered":"2024-04-25T09:58:01.630948+0000","last_unstale":"2024-04-25T09:58:04.416905+0000","last_undegraded":"2024-04-25T09:58:04.416905+0000","last_fullsized":"2024-04-25T09:58:04.416905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:53.848580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416689+0000","last_change":"2024-04-25T09:58:02.628459+0000","last_active":"2024-04-25T09:58:04.416689+0000","last_peered":"2024-04-25T09:58:04.416689+0000","last_clean":"2024-04-25T09:58:04.416689+0000","last_became_active":"2024-04-25T09:58:02.628184+0000","last_became_peered":"2024-04-25T09:58:02.628184+0000","last_unstale":"2024-04-25T09:58:04.416689+0000","last_undegraded":"2024-04-25T09:58:04.416689+0000","last_fullsized":"2024-04-25T09:58:04.416689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:15.450522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417536+0000","last_change":"2024-04-25T09:58:01.631535+0000","last_active":"2024-04-25T09:58:04.417536+0000","last_peered":"2024-04-25T09:58:04.417536+0000","last_clean":"2024-04-25T09:58:04.417536+0000","last_became_active":"2024-04-25T09:58:01.631258+0000","last_became_peered":"2024-04-25T09:58:01.631258+0000","last_unstale":"2024-04-25T09:58:04.417536+0000","last_undegraded":"2024-04-25T09:58:04.417536+0000","last_fullsized":"2024-04-25T09:58:04.417536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:02.536273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416805+0000","last_change":"2024-04-25T09:58:02.631432+0000","last_active":"2024-04-25T09:58:04.416805+0000","last_peered":"2024-04-25T09:58:04.416805+0000","last_clean":"2024-04-25T09:58:04.416805+0000","last_became_active":"2024-04-25T09:58:02.629913+0000","last_became_peered":"2024-04-25T09:58:02.629913+0000","last_unstale":"2024-04-25T09:58:04.416805+0000","last_undegraded":"2024-04-25T09:58:04.416805+0000","last_fullsized":"2024-04-25T09:58:04.416805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:39.032133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.620794+0000","last_change":"2024-04-25T09:58:01.627182+0000","last_active":"2024-04-25T09:58:04.620794+0000","last_peered":"2024-04-25T09:58:04.620794+0000","last_clean":"2024-04-25T09:58:04.620794+0000","last_became_active":"2024-04-25T09:58:01.626925+0000","last_became_peered":"2024-04-25T09:58:01.626925+0000","last_unstale":"2024-04-25T09:58:04.620794+0000","last_undegraded":"2024-04-25T09:58:04.620794+0000","last_fullsized":"2024-04-25T09:58:04.620794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:21.917950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416834+0000","last_change":"2024-04-25T09:58:02.639356+0000","last_active":"2024-04-25T09:58:04.416834+0000","last_peered":"2024-04-25T09:58:04.416834+0000","last_clean":"2024-04-25T09:58:04.416834+0000","last_became_active":"2024-04-25T09:58:02.639073+0000","last_became_peered":"2024-04-25T09:58:02.639073+0000","last_unstale":"2024-04-25T09:58:04.416834+0000","last_undegraded":"2024-04-25T09:58:04.416834+0000","last_fullsized":"2024-04-25T09:58:04.416834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:49.385120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417226+0000","last_change":"2024-04-25T09:58:01.630640+0000","last_active":"2024-04-25T09:58:04.417226+0000","last_peered":"2024-04-25T09:58:04.417226+0000","last_clean":"2024-04-25T09:58:04.417226+0000","last_became_active":"2024-04-25T09:58:01.630203+0000","last_became_peered":"2024-04-25T09:58:01.630203+0000","last_unstale":"2024-04-25T09:58:04.417226+0000","last_undegraded":"2024-04-25T09:58:04.417226+0000","last_fullsized":"2024-04-25T09:58:04.417226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:25.491047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417032+0000","last_change":"2024-04-25T09:58:02.639424+0000","last_active":"2024-04-25T09:58:04.417032+0000","last_peered":"2024-04-25T09:58:04.417032+0000","last_clean":"2024-04-25T09:58:04.417032+0000","last_became_active":"2024-04-25T09:58:02.639142+0000","last_became_peered":"2024-04-25T09:58:02.639142+0000","last_unstale":"2024-04-25T09:58:04.417032+0000","last_undegraded":"2024-04-25T09:58:04.417032+0000","last_fullsized":"2024-04-25T09:58:04.417032+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:05.497205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.639184+0000","last_change":"2024-04-25T09:58:01.634594+0000","last_active":"2024-04-25T09:58:03.639184+0000","last_peered":"2024-04-25T09:58:03.639184+0000","last_clean":"2024-04-25T09:58:03.639184+0000","last_became_active":"2024-04-25T09:58:01.634333+0000","last_became_peered":"2024-04-25T09:58:01.634333+0000","last_unstale":"2024-04-25T09:58:03.639184+0000","last_undegraded":"2024-04-25T09:58:03.639184+0000","last_fullsized":"2024-04-25T09:58:03.639184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.956627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416128+0000","last_change":"2024-04-25T09:58:02.629907+0000","last_active":"2024-04-25T09:58:04.416128+0000","last_peered":"2024-04-25T09:58:04.416128+0000","last_clean":"2024-04-25T09:58:04.416128+0000","last_became_active":"2024-04-25T09:58:02.629610+0000","last_became_peered":"2024-04-25T09:58:02.629610+0000","last_unstale":"2024-04-25T09:58:04.416128+0000","last_undegraded":"2024-04-25T09:58:04.416128+0000","last_fullsized":"2024-04-25T09:58:04.416128+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:01.404079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.417033+0000","last_change":"2024-04-25T09:58:01.633427+0000","last_active":"2024-04-25T09:58:04.417033+0000","last_peered":"2024-04-25T09:58:04.417033+0000","last_clean":"2024-04-25T09:58:04.417033+0000","last_became_active":"2024-04-25T09:58:01.633131+0000","last_became_peered":"2024-04-25T09:58:01.633131+0000","last_unstale":"2024-04-25T09:58:04.417033+0000","last_undegraded":"2024-04-25T09:58:04.417033+0000","last_fullsized":"2024-04-25T09:58:04.417033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:42.404975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.635963+0000","last_change":"2024-04-25T09:58:02.637618+0000","last_active":"2024-04-25T09:58:03.635963+0000","last_peered":"2024-04-25T09:58:03.635963+0000","last_clean":"2024-04-25T09:58:03.635963+0000","last_became_active":"2024-04-25T09:58:02.637354+0000","last_became_peered":"2024-04-25T09:58:02.637354+0000","last_unstale":"2024-04-25T09:58:03.635963+0000","last_undegraded":"2024-04-25T09:58:03.635963+0000","last_fullsized":"2024-04-25T09:58:03.635963+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:01.404079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416553+0000","last_change":"2024-04-25T09:58:01.629306+0000","last_active":"2024-04-25T09:58:04.416553+0000","last_peered":"2024-04-25T09:58:04.416553+0000","last_clean":"2024-04-25T09:58:04.416553+0000","last_became_active":"2024-04-25T09:58:01.629038+0000","last_became_peered":"2024-04-25T09:58:01.629038+0000","last_unstale":"2024-04-25T09:58:04.416553+0000","last_undegraded":"2024-04-25T09:58:04.416553+0000","last_fullsized":"2024-04-25T09:58:04.416553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:14.448243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.638897+0000","last_change":"2024-04-25T09:58:02.639257+0000","last_active":"2024-04-25T09:58:03.638897+0000","last_peered":"2024-04-25T09:58:03.638897+0000","last_clean":"2024-04-25T09:58:03.638897+0000","last_became_active":"2024-04-25T09:58:02.638949+0000","last_became_peered":"2024-04-25T09:58:02.638949+0000","last_unstale":"2024-04-25T09:58:03.638897+0000","last_undegraded":"2024-04-25T09:58:03.638897+0000","last_fullsized":"2024-04-25T09:58:03.638897+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:25.696390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416335+0000","last_change":"2024-04-25T09:58:01.630516+0000","last_active":"2024-04-25T09:58:04.416335+0000","last_peered":"2024-04-25T09:58:04.416335+0000","last_clean":"2024-04-25T09:58:04.416335+0000","last_became_active":"2024-04-25T09:58:01.630199+0000","last_became_peered":"2024-04-25T09:58:01.630199+0000","last_unstale":"2024-04-25T09:58:04.416335+0000","last_undegraded":"2024-04-25T09:58:04.416335+0000","last_fullsized":"2024-04-25T09:58:04.416335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:16.385468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416753+0000","last_change":"2024-04-25T09:58:02.628201+0000","last_active":"2024-04-25T09:58:04.416753+0000","last_peered":"2024-04-25T09:58:04.416753+0000","last_clean":"2024-04-25T09:58:04.416753+0000","last_became_active":"2024-04-25T09:58:02.627935+0000","last_became_peered":"2024-04-25T09:58:02.627935+0000","last_unstale":"2024-04-25T09:58:04.416753+0000","last_undegraded":"2024-04-25T09:58:04.416753+0000","last_fullsized":"2024-04-25T09:58:04.416753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:20.456244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:03.638813+0000","last_change":"2024-04-25T09:58:01.628453+0000","last_active":"2024-04-25T09:58:03.638813+0000","last_peered":"2024-04-25T09:58:03.638813+0000","last_clean":"2024-04-25T09:58:03.638813+0000","last_became_active":"2024-04-25T09:58:01.628191+0000","last_became_peered":"2024-04-25T09:58:01.628191+0000","last_unstale":"2024-04-25T09:58:03.638813+0000","last_undegraded":"2024-04-25T09:58:03.638813+0000","last_fullsized":"2024-04-25T09:58:03.638813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:23.423591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416528+0000","last_change":"2024-04-25T09:58:02.631467+0000","last_active":"2024-04-25T09:58:04.416528+0000","last_peered":"2024-04-25T09:58:04.416528+0000","last_clean":"2024-04-25T09:58:04.416528+0000","last_became_active":"2024-04-25T09:58:02.631171+0000","last_became_peered":"2024-04-25T09:58:02.631171+0000","last_unstale":"2024-04-25T09:58:04.416528+0000","last_undegraded":"2024-04-25T09:58:04.416528+0000","last_fullsized":"2024-04-25T09:58:04.416528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:01.608761+0000","last_clean_scrub_stamp":"2024-04-25T09:58:01.608761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:53.455144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416125+0000","last_change":"2024-04-25T09:58:01.630581+0000","last_active":"2024-04-25T09:58:04.416125+0000","last_peered":"2024-04-25T09:58:04.416125+0000","last_clean":"2024-04-25T09:58:04.416125+0000","last_became_active":"2024-04-25T09:58:01.630229+0000","last_became_peered":"2024-04-25T09:58:01.630229+0000","last_unstale":"2024-04-25T09:58:04.416125+0000","last_undegraded":"2024-04-25T09:58:04.416125+0000","last_fullsized":"2024-04-25T09:58:04.416125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:00.606483+0000","last_clean_scrub_stamp":"2024-04-25T09:58:00.606483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:20.035323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":256,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T09:58:04.416847+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:58:04.416847+0000","last_peered":"2024-04-25T09:58:04.416847+0000","last_clean":"2024-04-25T09:58:04.416847+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:58:04.416847+0000","last_undegraded":"2024-04-25T09:58:04.416847+0000","last_fullsized":"2024-04-25T09:58:04.416847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":1421312,"data_stored":960742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":815104,"data_stored":369632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":815104,"data_stored":369620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36152,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335688,"statfs":{"total":96636764160,"available":96599744512,"internally_reserved":0,"allocated":823296,"data_stored":371210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":811008,"data_stored":369598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36120,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335720,"statfs":{"total":96636764160,"available":96599777280,"internally_reserved":0,"allocated":798720,"data_stored":362906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":839680,"data_stored":371974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1404928,"data_stored":959966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:10.286 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:58:10.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:58:10.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.423+0000 7f5710cac640 1 Processor -- start 2024-04-25T09:58:10.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.423+0000 7f5710cac640 1 -- start start 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f5710cac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f570c15dac0 0x7f570c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f5710cac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f570c0d0ab0 0x7f570c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f5710cac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570c0d2420 con 0x7f570c0d1f30 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f5710cac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c0d3030 con 0x7f570c0d0ab0 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f5710cac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c0d2700 con 0x7f570c15dac0 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f570c0d0ab0 0x7f570c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f570c0d0ab0 0x7f570c0b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34660/0 (socket says 172.21.15.148:34660) 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 -- 172.21.15.148:0/421495630 learned_addr learned my addr 172.21.15.148:0/421495630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570a59c640 1 --2- 172.21.15.148:0/421495630 >> v2:172.21.15.188:3300/0 conn(0x7f570c15dac0 0x7f570c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 -- 172.21.15.148:0/421495630 >> v2:172.21.15.188:3300/0 conn(0x7f570c15dac0 msgr2=0x7f570c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 --2- 172.21.15.148:0/421495630 >> v2:172.21.15.188:3300/0 conn(0x7f570c15dac0 0x7f570c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 -- 172.21.15.148:0/421495630 >> v1:172.21.15.188:6789/0 conn(0x7f570c0d1f30 legacy=0x7f570c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:10.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.424+0000 7f570ad9d640 1 -- 172.21.15.148:0/421495630 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0d29e0 con 0x7f570c0d0ab0 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.425+0000 7f570ad9d640 1 --2- 172.21.15.148:0/421495630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0d0ab0 0x7f570c0b6990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f570c0b6520 tx=0x7f570402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e68813d2458e9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.425+0000 7f5709d9b640 1 -- 172.21.15.148:0/421495630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704036070 con 0x7f570c0d0ab0 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.425+0000 7f5709d9b640 1 -- 172.21.15.148:0/421495630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704031600 con 0x7f570c0d0ab0 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.425+0000 7f5709d9b640 1 -- 172.21.15.148:0/421495630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040319d0 con 0x7f570c0d0ab0 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.425+0000 7f5710cac640 1 -- 172.21.15.148:0/421495630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0d0ab0 msgr2=0x7f570c0b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.425+0000 7f5710cac640 1 --2- 172.21.15.148:0/421495630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0d0ab0 0x7f570c0b6990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f570c0b6520 tx=0x7f570402a780 comp rx=0 tx=0).stop 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.426+0000 7f5710cac640 1 -- 172.21.15.148:0/421495630 shutdown_connections 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.426+0000 7f5710cac640 1 --2- 172.21.15.148:0/421495630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0d0ab0 0x7f570c0b6990 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.426+0000 7f5710cac640 1 --2- 172.21.15.148:0/421495630 >> v2:172.21.15.188:3300/0 conn(0x7f570c15dac0 0x7f570c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.426+0000 7f5710cac640 1 -- 172.21.15.148:0/421495630 >> 172.21.15.148:0/421495630 conn(0x7f570c061c30 msgr2=0x7f570c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.426+0000 7f5710cac640 1 -- 172.21.15.148:0/421495630 shutdown_connections 2024-04-25T09:58:10.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.426+0000 7f5710cac640 1 -- 172.21.15.148:0/421495630 wait complete. 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.427+0000 7f5710cac640 1 Processor -- start 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.427+0000 7f5710cac640 1 -- start start 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c0d0ab0 0x7f570c1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0d1f30 0x7f570c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 0x7f570c1f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0b97d0 con 0x7f570c15dac0 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c15dea0 con 0x7f570c0d0ab0 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c0b9950 con 0x7f570c0d1f30 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 0x7f570c1f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 0x7f570c1f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34662/0 (socket says 172.21.15.148:34662) 2024-04-25T09:58:10.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 -- 172.21.15.148:0/1667127162 learned_addr learned my addr 172.21.15.148:0/1667127162 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 -- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0d1f30 msgr2=0x7f570c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0d1f30 0x7f570c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 -- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c0d0ab0 msgr2=0x7f570c1f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570a59c640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c0d0ab0 0x7f570c1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c0d0ab0 0x7f570c1f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 -- 172.21.15.148:0/1667127162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c1532e0 con 0x7f570c15dac0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f570ad9d640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 0x7f570c1f6f10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f570402ad70 tx=0x7f5704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704036050 con 0x7f570c15dac0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704042070 con 0x7f570c15dac0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c0cd410 con 0x7f570c15dac0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704009510 con 0x7f570c15dac0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.428+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c1f7d60 con 0x7f570c15dac0 2024-04-25T09:58:10.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.429+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c073a00 con 0x7f570c15dac0 2024-04-25T09:58:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.430+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5704007bb0 con 0x7f570c15dac0 2024-04-25T09:58:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.430+0000 7f5702ffd640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.430+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57040ab1b0 con 0x7f570c15dac0 2024-04-25T09:58:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.430+0000 7f570a59c640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.431+0000 7f570a59c640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56ec06b4d0 0x7f56ec06d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006fd0 tx=0x7f56fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.434+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5704079f30 con 0x7f570c15dac0 2024-04-25T09:58:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.671+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f570c0ca130 con 0x7f570c15dac0 2024-04-25T09:58:10.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.673+0000 7f5702ffd640 1 -- 172.21.15.148:0/1667127162 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5704079830 con 0x7f570c15dac0 2024-04-25T09:58:10.677 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T09:58:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T09:58:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T09:58:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T09:58:10.677 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:58:10.678 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56ec06b4d0 msgr2=0x7f56ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56ec06b4d0 0x7f56ec06d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006fd0 tx=0x7f56fc008040 comp rx=0 tx=0).stop 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 msgr2=0x7f570c1f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 0x7f570c1f6f10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f570402ad70 tx=0x7f5704004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 shutdown_connections 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c15dac0 0x7f570c1f6f10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0d1f30 0x7f570c0cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 --2- 172.21.15.148:0/1667127162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c0d0ab0 0x7f570c1f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 >> 172.21.15.148:0/1667127162 conn(0x7f570c061c30 msgr2=0x7f570c0b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 shutdown_connections 2024-04-25T09:58:10.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.676+0000 7f5710cac640 1 -- 172.21.15.148:0/1667127162 wait complete. 2024-04-25T09:58:10.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:10.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 Processor -- start 2024-04-25T09:58:10.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 -- start start 2024-04-25T09:58:10.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d10154ed0 0x7f0d100aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 0x7f0d1006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1015ec60 con 0x7f0d10071ac0 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1015ef40 con 0x7f0d10154ed0 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.784+0000 7f0d16ec4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1006db90 con 0x7f0d1006c1d0 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d14c60640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d10154ed0 0x7f0d100aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 0x7f0d1006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 0x7f0d1006afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:51518/0 (socket says 172.21.15.148:51518) 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 -- 172.21.15.148:0/4037870234 learned_addr learned my addr 172.21.15.148:0/4037870234 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 -- 172.21.15.148:0/4037870234 >> v2:172.21.15.188:3300/0 conn(0x7f0d10154ed0 msgr2=0x7f0d100aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 --2- 172.21.15.148:0/4037870234 >> v2:172.21.15.188:3300/0 conn(0x7f0d10154ed0 0x7f0d100aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 -- 172.21.15.148:0/4037870234 >> v1:172.21.15.188:6789/0 conn(0x7f0d10071ac0 legacy=0x7f0d100ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d15461640 1 -- 172.21.15.148:0/4037870234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1006de70 con 0x7f0d1006c1d0 2024-04-25T09:58:10.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.785+0000 7f0d14c60640 1 --2- 172.21.15.148:0/4037870234 >> v2:172.21.15.188:3300/0 conn(0x7f0d10154ed0 0x7f0d100aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.786+0000 7f0d15461640 1 --2- 172.21.15.148:0/4037870234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 0x7f0d1006afa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009e50 tx=0x7f0d0802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbcb27487dd6e666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.786+0000 7f0d0f7fe640 1 -- 172.21.15.148:0/4037870234 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08036070 con 0x7f0d1006c1d0 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.786+0000 7f0d0f7fe640 1 -- 172.21.15.148:0/4037870234 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d08031600 con 0x7f0d1006c1d0 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.786+0000 7f0d0f7fe640 1 -- 172.21.15.148:0/4037870234 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080319d0 con 0x7f0d1006c1d0 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4037870234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 msgr2=0x7f0d1006afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4037870234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 0x7f0d1006afa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009e50 tx=0x7f0d0802a710 comp rx=0 tx=0).stop 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4037870234 shutdown_connections 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4037870234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d1006c1d0 0x7f0d1006afa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4037870234 >> v2:172.21.15.188:3300/0 conn(0x7f0d10154ed0 0x7f0d100aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4037870234 >> 172.21.15.148:0/4037870234 conn(0x7f0d100af670 msgr2=0x7f0d100b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4037870234 shutdown_connections 2024-04-25T09:58:10.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.787+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4037870234 wait complete. 2024-04-25T09:58:10.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.788+0000 7f0d16ec4640 1 Processor -- start 2024-04-25T09:58:10.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.788+0000 7f0d16ec4640 1 -- start start 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d16ec4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d10071ac0 0x7f0d1014c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d16ec4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 0x7f0d10155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d16ec4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d1014cc70 0x7f0d10157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d16ec4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1006b660 con 0x7f0d10154ed0 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d16ec4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1006b4e0 con 0x7f0d1014cc70 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d16ec4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1006b7e0 con 0x7f0d10071ac0 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d0ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 0x7f0d10155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d0ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 0x7f0d10155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34674/0 (socket says 172.21.15.148:34674) 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d0ffff640 1 -- 172.21.15.148:0/4029727830 learned_addr learned my addr 172.21.15.148:0/4029727830 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d14c60640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d10071ac0 0x7f0d1014c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.789+0000 7f0d15461640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d1014cc70 0x7f0d10157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0ffff640 1 -- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d10071ac0 msgr2=0x7f0d1014c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0ffff640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d10071ac0 0x7f0d1014c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0ffff640 1 -- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d1014cc70 msgr2=0x7f0d10157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0ffff640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d1014cc70 0x7f0d10157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0ffff640 1 -- 172.21.15.148:0/4029727830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d101565d0 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0ffff640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 0x7f0d10155d10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00c470 tx=0x7f0cfc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0052d0 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc005470 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc007260 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d10156dd0 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.790+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d10157670 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.791+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d101571c0 con 0x7f0d10154ed0 2024-04-25T09:58:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.791+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cfc007400 con 0x7f0d10154ed0 2024-04-25T09:58:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.792+0000 7f0d0d7fa640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.792+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0cfc0897f0 con 0x7f0d10154ed0 2024-04-25T09:58:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.792+0000 7f0d14c60640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.792+0000 7f0d14c60640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0cf406b4d0 0x7f0cf406d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009560 tx=0x7f0d04009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:10.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:10.796+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0cfc057e90 con 0x7f0d10154ed0 2024-04-25T09:58:11.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.004+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d100b2b10 con 0x7f0d10154ed0 2024-04-25T09:58:11.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.004+0000 7f0d0d7fa640 1 -- 172.21.15.148:0/4029727830 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f0cfc05bf30 con 0x7f0d10154ed0 2024-04-25T09:58:11.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:11.008 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3775040580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3775040580},{"type":"v1","addr":"172.21.15.148:6837","nonce":3775040580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/733820966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":733820966},{"type":"v1","addr":"172.21.15.188:6834","nonce":733820966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2576966127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2576966127},{"type":"v1","addr":"172.21.15.188:6835","nonce":2576966127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:03.969534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6729},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6729":{"gid":6729,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/2397025400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2397025400},{"type":"v1","addr":"172.21.15.148:6836","nonce":2397025400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:11.009 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 33 2024-04-25T09:58:11.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.006+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0cf406b4d0 msgr2=0x7f0cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.006+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0cf406b4d0 0x7f0cf406d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009560 tx=0x7f0d04009190 comp rx=0 tx=0).stop 2024-04-25T09:58:11.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.006+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 msgr2=0x7f0d10155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.006+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 0x7f0d10155d10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00c470 tx=0x7f0cfc001e70 comp rx=0 tx=0).stop 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 shutdown_connections 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d1014cc70 0x7f0d10157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d10154ed0 0x7f0d10155d10 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 --2- 172.21.15.148:0/4029727830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d10071ac0 0x7f0d1014c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 >> 172.21.15.148:0/4029727830 conn(0x7f0d1006c1d0 msgr2=0x7f0d1015f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 shutdown_connections 2024-04-25T09:58:11.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.007+0000 7f0d16ec4640 1 -- 172.21.15.148:0/4029727830 wait complete. 2024-04-25T09:58:11.424 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:01.965750+0000', 'modified': '2024-04-25T09:58:03.969534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6729}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6729': {'gid': 6729, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6836/2397025400', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 2397025400}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 2397025400}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:58:11.424 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:58:11.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:58:11.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.521+0000 7fc305f7e640 1 Processor -- start 2024-04-25T09:58:11.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.521+0000 7fc305f7e640 1 -- start start 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.521+0000 7fc305f7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3000b5570 0x7fc30006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.521+0000 7fc305f7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000d1d50 con 0x7fc30006c330 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.521+0000 7fc305f7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000d2020 con 0x7fc300158230 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.521+0000 7fc305f7e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc300158610 con 0x7fc3000b5570 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2feffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc30006c330 0x7fc3000b4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41510/0 (socket says 172.21.15.148:41510) 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2feffd640 1 -- 172.21.15.148:0/328344893 learned_addr learned my addr 172.21.15.148:0/328344893 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fffff640 1 --2- 172.21.15.148:0/328344893 >> v2:172.21.15.188:3300/0 conn(0x7fc3000b5570 0x7fc30006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350502607 0 0) 0x7fc3000d1d50 con 0x7fc30006c330 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2e8003520 con 0x7fc30006c330 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34831051 0 0) 0x7fc2e8003520 con 0x7fc30006c330 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 >> v2:172.21.15.188:3300/0 conn(0x7fc3000b5570 msgr2=0x7fc30006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 --2- 172.21.15.148:0/328344893 >> v2:172.21.15.188:3300/0 conn(0x7fc3000b5570 0x7fc30006f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 >> v1:172.21.15.188:6789/0 conn(0x7fc300158230 legacy=0x7fc3000d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000d2300 con 0x7fc30006c330 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc2f40017f0 con 0x7fc30006c330 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 >> v1:172.21.15.148:6789/0 conn(0x7fc30006c330 legacy=0x7fc3000b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80039b0 0x7fc2e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e8004800 0x7fc2e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80051c0 0x7fc2e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300158610 con 0x7fc2e80051c0 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e800c060 con 0x7fc2e80039b0 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e800c260 con 0x7fc2e8004800 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2fffff640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80051c0 0x7fc2e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80039b0 0x7fc2e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.522+0000 7fc2feffd640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e8004800 0x7fc2e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 -- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e8004800 msgr2=0x7fc2e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e8004800 0x7fc2e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 -- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80039b0 msgr2=0x7fc2e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80039b0 0x7fc2e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 -- 172.21.15.148:0/328344893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e8003520 con 0x7fc2e80051c0 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 -- 172.21.15.148:0/328344893 reap_dead start 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fffff640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80051c0 0x7fc2e800b9c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009fc0 tx=0x7fc2e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe11a450786af23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8035020 con 0x7fc2e80051c0 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8003bd0 con 0x7fc2e80051c0 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.523+0000 7fc2fe7fc640 1 -- 172.21.15.148:0/328344893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8009c10 con 0x7fc2e80051c0 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 -- 172.21.15.148:0/328344893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80051c0 msgr2=0x7fc2e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80051c0 0x7fc2e800b9c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009fc0 tx=0x7fc2e8005830 comp rx=0 tx=0).stop 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 -- 172.21.15.148:0/328344893 shutdown_connections 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 --2- 172.21.15.148:0/328344893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80051c0 0x7fc2e800b9c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 -- 172.21.15.148:0/328344893 >> 172.21.15.148:0/328344893 conn(0x7fc300061600 msgr2=0x7fc3000619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 -- 172.21.15.148:0/328344893 shutdown_connections 2024-04-25T09:58:11.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.524+0000 7fc305f7e640 1 -- 172.21.15.148:0/328344893 wait complete. 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.525+0000 7fc305f7e640 1 Processor -- start 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.525+0000 7fc305f7e640 1 -- start start 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc305f7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80051c0 0x7fc3001f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc305f7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 0x7fc3001e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc305f7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc300148060 0x7fc3001484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc305f7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000d25e0 con 0x7fc3001e4970 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc305f7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30006bea0 con 0x7fc2e80051c0 2024-04-25T09:58:11.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc305f7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30015e320 con 0x7fc300148060 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 0x7fc3001e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 0x7fc3001e4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34690/0 (socket says 172.21.15.148:34690) 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 -- 172.21.15.148:0/4055162314 learned_addr learned my addr 172.21.15.148:0/4055162314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2fffff640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc300148060 0x7fc3001484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80051c0 0x7fc3001f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 -- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc300148060 msgr2=0x7fc3001484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc300148060 0x7fc3001484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 -- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80051c0 msgr2=0x7fc3001f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80051c0 0x7fc3001f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2feffd640 1 -- 172.21.15.148:0/4055162314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300148e00 con 0x7fc3001e4970 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.526+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80051c0 0x7fc3001f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.527+0000 7fc2feffd640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 0x7fc3001e4dd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400f7f0 tx=0x7fc2f400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.527+0000 7fc2e7fff640 1 -- 172.21.15.148:0/4055162314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400fea0 con 0x7fc3001e4970 2024-04-25T09:58:11.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.527+0000 7fc2e7fff640 1 -- 172.21.15.148:0/4055162314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4018070 con 0x7fc3001e4970 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.527+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3001490e0 con 0x7fc3001e4970 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.527+0000 7fc2e7fff640 1 -- 172.21.15.148:0/4055162314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4007520 con 0x7fc3001e4970 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.527+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3001496a0 con 0x7fc3001e4970 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.528+0000 7fc2e7fff640 1 -- 172.21.15.148:0/4055162314 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2f40057a0 con 0x7fc3001e4970 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.529+0000 7fc2e7fff640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2ec06b500 0x7fc2ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.529+0000 7fc2e7fff640 1 -- 172.21.15.148:0/4055162314 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2f408a390 con 0x7fc3001e4970 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.529+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2ec06b500 0x7fc2ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.529+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2ec06b500 0x7fc2ec06d9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0005580 tx=0x7fc2f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.530+0000 7fc308c82740 1 Processor -- start 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.530+0000 7fc308c82740 1 -- start start 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc308c82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e3085610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e30a2ad0 0x5593e30a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e305af10 0x5593e30a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc308c82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593e2fbda10 con 0x5593e3085230 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593e2f66b30 con 0x5593e305af10 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593e2fb9c50 con 0x5593e30a2ad0 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e3085610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e3085610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34700/0 (socket says 172.21.15.148:34700) 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/3715200091 learned_addr learned my addr 172.21.15.148:0/3715200091 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e30a2ad0 msgr2=0x5593e30a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2fffff640 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e305af10 0x5593e30a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2feffd640 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e30a2ad0 0x5593e30a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e30a2ad0 0x5593e30a3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e305af10 msgr2=0x5593e30a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e305af10 0x5593e30a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/3715200091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593e2f7be10 con 0x5593e3085230 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e3085610 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0001650 tx=0x7fc2f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2600a81bd23754ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2e57fa640 1 -- 172.21.15.148:0/3715200091 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x5593e3085230 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2e57fa640 1 -- 172.21.15.148:0/3715200091 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00147c0 con 0x5593e3085230 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.531+0000 7fc2e57fa640 1 -- 172.21.15.148:0/3715200091 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0014b70 con 0x5593e3085230 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 -- 172.21.15.148:0/3715200091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 msgr2=0x5593e3085610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e3085610 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0001650 tx=0x7fc2f0013000 comp rx=0 tx=0).stop 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 -- 172.21.15.148:0/3715200091 shutdown_connections 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e305af10 0x5593e30a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e30a2ad0 0x5593e30a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 --2- 172.21.15.148:0/3715200091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e3085610 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 -- 172.21.15.148:0/3715200091 >> 172.21.15.148:0/3715200091 conn(0x5593e3057840 msgr2=0x5593e30a0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 -- 172.21.15.148:0/3715200091 shutdown_connections 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.532+0000 7fc308c82740 1 -- 172.21.15.148:0/3715200091 wait complete. 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.533+0000 7fc308c82740 1 -- start start 2024-04-25T09:58:11.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.533+0000 7fc308c82740 1 Processor -- start 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.533+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3085230 0x5593e3080b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.533+0000 7fc308c82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 0x5593e30b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.533+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e3097f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593e305bcb0 con 0x5593e30a2ad0 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593e2f7a0c0 con 0x5593e3097b00 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593e3055770 con 0x5593e3085230 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 10 client.0 _ll_register_callbacks cb 0x5593e2f6dec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 0x5593e30b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 0x5593e30b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34716/0 (socket says 172.21.15.148:34716) 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 -- 172.21.15.148:0/1576745189 learned_addr learned my addr 172.21.15.148:0/1576745189 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3085230 0x5593e3080b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2fffff640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e3097f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 -- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3085230 msgr2=0x5593e3080b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3085230 0x5593e3080b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 -- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 msgr2=0x5593e3097f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e3097f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 -- 172.21.15.148:0/1576745189 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593e2cd22d0 con 0x5593e30a2ad0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3085230 0x5593e3080b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2feffd640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 0x5593e30b0490 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4058680 tx=0x7fc2f4058a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a9e698d21db0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2de7fc640 1 -- 172.21.15.148:0/1576745189 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4088b50 con 0x5593e30a2ad0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2de7fc640 1 -- 172.21.15.148:0/1576745189 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2f4088cf0 con 0x5593e30a2ad0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2f00147c0 con 0x5593e30a2ad0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2de7fc640 1 -- 172.21.15.148:0/1576745189 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2f4057dd0 con 0x5593e30a2ad0 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 10 client.6894 fetch_fsmap learned FSMap version 33 2024-04-25T09:58:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5593e2d9c6b0 con 0x5593e30a2ad0 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.534+0000 7fc2de7fc640 1 -- 172.21.15.148:0/1576745189 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00147c0 con 0x5593e30a2ad0 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc2de7fc640 1 -- 172.21.15.148:0/1576745189 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc2f4057f70 con 0x5593e30a2ad0 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc308c82740 10 client.6894 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc308c82740 10 client.6894 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc308c82740 20 client.6894 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc308c82740 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e3088c60 0x5593e3089040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc308c82740 4 client.6894 mds_command: new command op to 6729 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 --> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593e301c920 con 0x5593e3088c60 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e3088c60 0x5593e3089040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc2de7fc640 10 client.6894 ms_handle_connect on v2:172.21.15.148:6834/2397025400 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.535+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e3088c60 0x5593e3089040 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20982c7ecac23ccc server_cookie=31ca0eb6c9f09c45 in_seq=0 out_seq=0 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc2de7fc640 1 -- 172.21.15.148:0/1576745189 <== mds.0 v2:172.21.15.148:6834/2397025400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593e301c920 con 0x5593e3088c60 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc2de7fc640 10 client.6894 handle_command_reply: tid=0 2024-04-25T09:58:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 client.6894 shutdown 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 msgr2=0x5593e30b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 0x5593e30b0490 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4058680 tx=0x7fc2f4058a40 comp rx=0 tx=0).stop 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 shutdown_connections 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e3088c60 0x5593e3089040 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc2de7fc640 0 client.6894 ms_handle_reset on v2:172.21.15.148:6834/2397025400 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e3097f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e30a2ad0 0x5593e30b0490 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 --2- 172.21.15.148:0/1576745189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3085230 0x5593e3080b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 >> 172.21.15.148:0/1576745189 conn(0x5593e305af10 msgr2=0x5593e30a3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 shutdown_connections 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 1 -- 172.21.15.148:0/1576745189 wait complete. 2024-04-25T09:58:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.536+0000 7fc308c82740 20 client.6894 trim_cache size 0 max 16384 2024-04-25T09:58:11.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 Processor -- start 2024-04-25T09:58:11.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 -- start start 2024-04-25T09:58:11.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3085230 0x5593e305a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e30a1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 0x5593e3097080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001ffce0 con 0x5593e3097b00 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000ccd80 con 0x5593e3085230 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.552+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001ff6e0 con 0x5593e3088c60 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 0x5593e3097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 0x5593e3097080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34728/0 (socket says 172.21.15.148:34728) 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 -- 172.21.15.148:0/471874592 learned_addr learned my addr 172.21.15.148:0/471874592 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3085230 0x5593e305a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2feffd640 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e30a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 -- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 msgr2=0x5593e30a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e30a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 -- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3085230 msgr2=0x5593e305a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3085230 0x5593e305a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 -- 172.21.15.148:0/471874592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593e309a6d0 con 0x5593e3097b00 2024-04-25T09:58:11.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2fffff640 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 0x5593e3097080 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc2f802af10 tx=0x7fc2f802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108511835e6dad1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2e4ff9640 1 -- 172.21.15.148:0/471874592 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8035070 con 0x5593e3097b00 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2e4ff9640 1 -- 172.21.15.148:0/471874592 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8003c70 con 0x5593e3097b00 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.553+0000 7fc2e4ff9640 1 -- 172.21.15.148:0/471874592 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f802d070 con 0x5593e3097b00 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 -- 172.21.15.148:0/471874592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 msgr2=0x5593e3097080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 0x5593e3097080 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc2f802af10 tx=0x7fc2f802d000 comp rx=0 tx=0).stop 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 -- 172.21.15.148:0/471874592 shutdown_connections 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3097b00 0x5593e3097080 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e30a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 --2- 172.21.15.148:0/471874592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3085230 0x5593e305a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 -- 172.21.15.148:0/471874592 >> 172.21.15.148:0/471874592 conn(0x5593e305af10 msgr2=0x5593e3089040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 -- 172.21.15.148:0/471874592 shutdown_connections 2024-04-25T09:58:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 -- 172.21.15.148:0/471874592 wait complete. 2024-04-25T09:58:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.554+0000 7fc308c82740 1 -- start start 2024-04-25T09:58:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 Processor -- start 2024-04-25T09:58:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e310a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e305b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e305b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593e3086700 con 0x5593e3085230 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593e3086a70 con 0x5593e3097b00 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593e3098e70 con 0x5593e3088c60 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc308c82740 10 client.0 _ll_register_callbacks cb 0x5593e3055c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc2ff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e310a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc2ff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e310a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34736/0 (socket says 172.21.15.148:34736) 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.555+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/2890047506 learned_addr learned my addr 172.21.15.148:0/2890047506 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/2890047506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 msgr2=0x5593e305b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e305b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/2890047506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 msgr2=0x5593e305b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e305b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2ff7fe640 1 -- 172.21.15.148:0/2890047506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593e310df00 con 0x5593e3085230 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2ff7fe640 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e310a270 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009010 tx=0x7fc2f0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e75f2f48972539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2de7fc640 1 -- 172.21.15.148:0/2890047506 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0027040 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2de7fc640 1 -- 172.21.15.148:0/2890047506 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2f0014c30 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2f8003c70 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2de7fc640 1 -- 172.21.15.148:0/2890047506 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2f0041050 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 10 client.6900 fetch_fsmap learned FSMap version 33 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5593e3054920 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2de7fc640 1 -- 172.21.15.148:0/2890047506 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8003c70 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2de7fc640 1 -- 172.21.15.148:0/2890047506 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc2f0044020 con 0x5593e3085230 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 10 client.6900 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 10 client.6900 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 20 client.6900 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e30a2ad0 0x5593e310b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 4 client.6900 mds_command: new command op to 6729 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:58:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 --> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5593e2fcaf50 con 0x5593e30a2ad0 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.556+0000 7fc2feffd640 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e30a2ad0 0x5593e310b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc2de7fc640 10 client.6900 ms_handle_connect on v2:172.21.15.148:6834/2397025400 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc2feffd640 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e30a2ad0 0x5593e310b3b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31daace84fb1ab25 server_cookie=44dfaffadfb75b27 in_seq=0 out_seq=0 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc2de7fc640 1 -- 172.21.15.148:0/2890047506 <== mds.0 v2:172.21.15.148:6834/2397025400 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5593e2fcaf50 con 0x5593e30a2ad0 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc2de7fc640 10 client.6900 handle_command_reply: tid=0 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 client.6900 shutdown 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 msgr2=0x5593e310a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e310a270 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009010 tx=0x7fc2f0001120 comp rx=0 tx=0).stop 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 shutdown_connections 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:6834/2397025400,v1:172.21.15.148:6836/2397025400] conn(0x5593e30a2ad0 0x5593e310b3b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e3097b00 0x5593e305b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5593e3088c60 0x5593e305b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc2de7fc640 0 client.6900 ms_handle_reset on v2:172.21.15.148:6834/2397025400 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 --2- 172.21.15.148:0/2890047506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593e3085230 0x5593e310a270 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 >> 172.21.15.148:0/2890047506 conn(0x5593e305af10 msgr2=0x5593e3096e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 shutdown_connections 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 1 -- 172.21.15.148:0/2890047506 wait complete. 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.557+0000 7fc308c82740 20 client.6900 trim_cache size 0 max 16384 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 6729, 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:58:11.567 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 8.9424481050000004, 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 33, 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 67, 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 67, 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 14.247546494 2024-04-25T09:58:11.568 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2ec06b500 msgr2=0x7fc2ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2ec06b500 0x7fc2ec06d9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0005580 tx=0x7fc2f0009040 comp rx=0 tx=0).stop 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 msgr2=0x7fc3001e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 0x7fc3001e4dd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400f7f0 tx=0x7fc2f400fcb0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 shutdown_connections 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2ec06b500 0x7fc2ec06d9c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc300148060 0x7fc3001484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3001e4970 0x7fc3001e4dd0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 --2- 172.21.15.148:0/4055162314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2e80051c0 0x7fc3001f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 >> 172.21.15.148:0/4055162314 conn(0x7fc300061600 msgr2=0x7fc3000b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 shutdown_connections 2024-04-25T09:58:11.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.558+0000 7fc305f7e640 1 -- 172.21.15.148:0/4055162314 wait complete. 2024-04-25T09:58:11.572 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:58:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.666+0000 7fb532b9e640 1 Processor -- start 2024-04-25T09:58:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.666+0000 7fb532b9e640 1 -- start start 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.667+0000 7fb532b9e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb52c06bd00 0x7fb52c168910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.667+0000 7fb532b9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0b49c0 con 0x7fb52c15db70 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.667+0000 7fb532b9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0b4c60 con 0x7fb52c0cf650 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.667+0000 7fb532b9e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c15df50 con 0x7fb52c06bd00 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.667+0000 7fb531b9c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb52c15db70 0x7fb52c0b2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41512/0 (socket says 172.21.15.148:41512) 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.667+0000 7fb531b9c640 1 -- 172.21.15.148:0/1938969121 learned_addr learned my addr 172.21.15.148:0/1938969121 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155631633 0 0) 0x7fb52c0b49c0 con 0x7fb52c15db70 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb518003520 con 0x7fb52c15db70 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb53239d640 1 --2- 172.21.15.148:0/1938969121 >> v2:172.21.15.188:3300/0 conn(0x7fb52c06bd00 0x7fb52c168910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1791789807 0 0) 0x7fb518003520 con 0x7fb52c15db70 2024-04-25T09:58:11.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 >> v2:172.21.15.188:3300/0 conn(0x7fb52c06bd00 msgr2=0x7fb52c168910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 --2- 172.21.15.148:0/1938969121 >> v2:172.21.15.188:3300/0 conn(0x7fb52c06bd00 0x7fb52c168910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 >> v1:172.21.15.188:6789/0 conn(0x7fb52c0cf650 legacy=0x7fb52c06b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0b4f40 con 0x7fb52c15db70 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb5240019f0 con 0x7fb52c15db70 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 >> v1:172.21.15.148:6789/0 conn(0x7fb52c15db70 legacy=0x7fb52c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb518004800 0x7fb518004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180051c0 0x7fb51800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51800c040 con 0x7fb5180039b0 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51800c1f0 con 0x7fb518004800 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51800c3f0 con 0x7fb5180051c0 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb531b9c640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb53139b640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb518004800 0x7fb518004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb53239d640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180051c0 0x7fb51800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb531b9c640 1 -- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180051c0 msgr2=0x7fb51800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb531b9c640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180051c0 0x7fb51800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb531b9c640 1 -- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb518004800 msgr2=0x7fb518004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb531b9c640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb518004800 0x7fb518004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb531b9c640 1 -- 172.21.15.148:0/1938969121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518003520 con 0x7fb5180039b0 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.668+0000 7fb53239d640 1 -- 172.21.15.148:0/1938969121 reap_dead start 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.669+0000 7fb531b9c640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb52400a840 tx=0x7fb518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4522a3e9b5e52b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.669+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400a460 con 0x7fb5180039b0 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.669+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52402f070 con 0x7fb5180039b0 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.669+0000 7fb530b9a640 1 -- 172.21.15.148:0/1938969121 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52402bc70 con 0x7fb5180039b0 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 -- 172.21.15.148:0/1938969121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180039b0 msgr2=0x7fb518003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb52400a840 tx=0x7fb518005830 comp rx=0 tx=0).stop 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 -- 172.21.15.148:0/1938969121 shutdown_connections 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 --2- 172.21.15.148:0/1938969121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5180039b0 0x7fb518003dd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 -- 172.21.15.148:0/1938969121 >> 172.21.15.148:0/1938969121 conn(0x7fb52c0cc7f0 msgr2=0x7fb52c0ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 -- 172.21.15.148:0/1938969121 shutdown_connections 2024-04-25T09:58:11.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.670+0000 7fb532b9e640 1 -- 172.21.15.148:0/1938969121 wait complete. 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.671+0000 7fb532b9e640 1 Processor -- start 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.671+0000 7fb532b9e640 1 -- start start 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180039b0 0x7fb52c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 0x7fb52c159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb52c1598a0 0x7fb52c14e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0cfab0 con 0x7fb52c06e470 2024-04-25T09:58:11.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0cf130 con 0x7fb52c1598a0 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c06c170 con 0x7fb5180039b0 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 0x7fb52c159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 0x7fb52c159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34740/0 (socket says 172.21.15.148:34740) 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 -- 172.21.15.148:0/3958611540 learned_addr learned my addr 172.21.15.148:0/3958611540 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 -- 172.21.15.148:0/3958611540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180039b0 msgr2=0x7fb52c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180039b0 0x7fb52c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 -- 172.21.15.148:0/3958611540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb52c1598a0 msgr2=0x7fb52c14e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb52c1598a0 0x7fb52c14e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 -- 172.21.15.148:0/3958611540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c06f2a0 con 0x7fb52c06e470 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb53139b640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 0x7fb52c159360 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00f620 tx=0x7fb51c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0052e0 con 0x7fb52c06e470 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c018070 con 0x7fb52c06e470 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c06f580 con 0x7fb52c06e470 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.672+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c003810 con 0x7fb52c06e470 2024-04-25T09:58:11.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.673+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c06fb40 con 0x7fb52c06e470 2024-04-25T09:58:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.673+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c073990 con 0x7fb52c06e470 2024-04-25T09:58:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.674+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb51c0039b0 con 0x7fb52c06e470 2024-04-25T09:58:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.674+0000 7fb5227fc640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb51006b4d0 0x7fb51006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.674+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb51c08a030 con 0x7fb52c06e470 2024-04-25T09:58:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.674+0000 7fb531b9c640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb51006b4d0 0x7fb51006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:11.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.675+0000 7fb531b9c640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb51006b4d0 0x7fb51006d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb524033000 tx=0x7fb52402d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:11.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.678+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb51c058db0 con 0x7fb52c06e470 2024-04-25T09:58:11.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.888+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb52c0cd5f0 con 0x7fb52c06e470 2024-04-25T09:58:11.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.889+0000 7fb5227fc640 1 -- 172.21.15.148:0/3958611540 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb51c0586b0 con 0x7fb52c06e470 2024-04-25T09:58:11.892 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:58:11.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb51006b4d0 msgr2=0x7fb51006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb51006b4d0 0x7fb51006d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb524033000 tx=0x7fb52402d000 comp rx=0 tx=0).stop 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 msgr2=0x7fb52c159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 0x7fb52c159360 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00f620 tx=0x7fb51c00fae0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 shutdown_connections 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb51006b4d0 0x7fb51006d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb52c1598a0 0x7fb52c14e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb52c06e470 0x7fb52c159360 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 --2- 172.21.15.148:0/3958611540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5180039b0 0x7fb52c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 >> 172.21.15.148:0/3958611540 conn(0x7fb52c0cc7f0 msgr2=0x7fb52c0af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 shutdown_connections 2024-04-25T09:58:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:11.891+0000 7fb532b9e640 1 -- 172.21.15.148:0/3958611540 wait complete. 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:58:11.905 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:58:11.906 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:11.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:11.906 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:58:11.921 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:58:11.921 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:58:11.921 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:58:11.921 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.921 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:58:11.922 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42544sec preferred_lft 42544sec 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:11.923 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:58:11.924 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T09:58:11.924 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:11.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:11.924 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:58:11.983 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:58:11.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:11.985 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:58:12.041 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:58:12.041 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:58:12.041 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:58:12.042 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:58:12.042 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:58:12.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.042 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:58:12.042 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:58:12.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:58:12.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:58:12.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:58:12.043 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:58:12.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.125 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:58:12.180 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:58:12.180 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:58:12.180 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:12.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:58:12.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:12.244 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:12.299 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:12.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:12.299 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:12.353 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:58:12.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.354 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:12.435 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:12.435 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:12.435 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:12.444 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:12.444 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:12.444 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:12.444 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:58:12.444 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:58:12.449 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:58:12.450 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:58:12.450 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:58:12.450 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:58:12.450 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:58:12.450 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:58:12.451 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:58:12.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:12.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.516 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:58:12.576 INFO:teuthology.orchestra.run.smithi148.stdout:client6028 2024-04-25T09:58:12.576 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:12.576 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:12.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.632 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T09:58:12.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:12.692 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T09:58:12.748 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:58:12.748 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:58:12.749 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T09:58:12.749 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:58:12.749 DEBUG:teuthology.orchestra.run.smithi148:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T09:58:12.749 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:58:12.852 INFO:teuthology.orchestra.run.smithi148.stdout:1099511627776 2024-04-25T09:58:12.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:58:12.861 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:12.861 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:12.917 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:12.917 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:12.973 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:13.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.107+0000 7f4195909640 1 Processor -- start 2024-04-25T09:58:13.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.107+0000 7f4195909640 1 -- start start 2024-04-25T09:58:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4195909640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f41900b2cb0 0x7f41900d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4195909640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41900b97a0 con 0x7f41900d1020 2024-04-25T09:58:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4195909640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41900d1c50 con 0x7f419015dbe0 2024-04-25T09:58:13.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4195909640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419015dfc0 con 0x7f41900b2cb0 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f418f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f41900d1020 0x7f41900d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41528/0 (socket says 172.21.15.148:41528) 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f418f7fe640 1 -- 172.21.15.148:0/3745790030 learned_addr learned my addr 172.21.15.148:0/3745790030 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4187fff640 1 --2- 172.21.15.148:0/3745790030 >> v2:172.21.15.148:3300/0 conn(0x7f41900b2cb0 0x7f41900d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f418e7fc640 1 -- 172.21.15.148:0/3745790030 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287632165 0 0) 0x7f41900b97a0 con 0x7f41900d1020 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f418e7fc640 1 -- 172.21.15.148:0/3745790030 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f417c003520 con 0x7f41900d1020 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4187fff640 1 -- 172.21.15.148:0/3745790030 >> v1:172.21.15.188:6789/0 conn(0x7f419015dbe0 legacy=0x7f41900d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4187fff640 1 -- 172.21.15.148:0/3745790030 >> v1:172.21.15.148:6789/0 conn(0x7f41900d1020 legacy=0x7f41900d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.108+0000 7f4187fff640 1 -- 172.21.15.148:0/3745790030 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41900d1f30 con 0x7f41900b2cb0 2024-04-25T09:58:13.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.109+0000 7f4187fff640 1 --2- 172.21.15.148:0/3745790030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f41900d2730 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f41780097a0 tx=0x7f417802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbda1f110af60f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.109+0000 7f418e7fc640 1 -- 172.21.15.148:0/3745790030 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178037070 con 0x7f41900b2cb0 2024-04-25T09:58:13.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.109+0000 7f418e7fc640 1 -- 172.21.15.148:0/3745790030 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4178032440 con 0x7f41900b2cb0 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.109+0000 7f418e7fc640 1 -- 172.21.15.148:0/3745790030 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178032810 con 0x7f41900b2cb0 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 -- 172.21.15.148:0/3745790030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 msgr2=0x7f41900d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 --2- 172.21.15.148:0/3745790030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f41900d2730 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f41780097a0 tx=0x7f417802a230 comp rx=0 tx=0).stop 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 -- 172.21.15.148:0/3745790030 shutdown_connections 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 --2- 172.21.15.148:0/3745790030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f41900d2730 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 -- 172.21.15.148:0/3745790030 >> 172.21.15.148:0/3745790030 conn(0x7f4190061a20 msgr2=0x7f4190061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 -- 172.21.15.148:0/3745790030 shutdown_connections 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.110+0000 7f4195909640 1 -- 172.21.15.148:0/3745790030 wait complete. 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.111+0000 7f4195909640 1 Processor -- start 2024-04-25T09:58:13.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.111+0000 7f4195909640 1 -- start start 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f419014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41900d1020 0x7f41900b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419015dbe0 0x7f4190153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900b92a0 con 0x7f41900b2cb0 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41900d1a00 con 0x7f419015dbe0 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41900b9420 con 0x7f41900d1020 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f419014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f419014fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34754/0 (socket says 172.21.15.148:34754) 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 -- 172.21.15.148:0/1964128644 learned_addr learned my addr 172.21.15.148:0/1964128644 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 -- 172.21.15.148:0/1964128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41900d1020 msgr2=0x7f41900b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41900d1020 0x7f41900b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 -- 172.21.15.148:0/1964128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419015dbe0 msgr2=0x7f4190153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419015dbe0 0x7f4190153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 -- 172.21.15.148:0/1964128644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190156090 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418effd640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f419014fbb0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f418000c3c0 tx=0x7f418000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41800052d0 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4180005470 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180007260 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190156330 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.112+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41901568f0 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.113+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4190073990 con 0x7f41900b2cb0 2024-04-25T09:58:13.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.114+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4180007400 con 0x7f41900b2cb0 2024-04-25T09:58:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.114+0000 7f418cff9640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f417006b4d0 0x7f417006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.114+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4180089880 con 0x7f41900b2cb0 2024-04-25T09:58:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.114+0000 7f4187fff640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f417006b4d0 0x7f417006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.115+0000 7f4187fff640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f417006b4d0 0x7f417006d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4178034000 tx=0x7f417802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.118+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4180058570 con 0x7f41900b2cb0 2024-04-25T09:58:13.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.292+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41900b87d0 con 0x7f41900b2cb0 2024-04-25T09:58:13.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.293+0000 7f418cff9640 1 -- 172.21.15.148:0/1964128644 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f4180057e70 con 0x7f41900b2cb0 2024-04-25T09:58:13.297 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:13.297 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:03.627039+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:13.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.295+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f417006b4d0 msgr2=0x7f417006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.295+0000 7f4195909640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f417006b4d0 0x7f417006d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4178034000 tx=0x7f417802c000 comp rx=0 tx=0).stop 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.295+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 msgr2=0x7f419014fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.295+0000 7f4195909640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f419014fbb0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f418000c3c0 tx=0x7f418000c880 comp rx=0 tx=0).stop 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 shutdown_connections 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f417006b4d0 0x7f417006d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419015dbe0 0x7f4190153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41900d1020 0x7f41900b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 --2- 172.21.15.148:0/1964128644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41900b2cb0 0x7f419014fbb0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 >> 172.21.15.148:0/1964128644 conn(0x7f4190061a20 msgr2=0x7f4190158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 shutdown_connections 2024-04-25T09:58:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.296+0000 7f4195909640 1 -- 172.21.15.148:0/1964128644 wait complete. 2024-04-25T09:58:13.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:00.177731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:00.933836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:13.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.407+0000 7f9c38c1f640 1 Processor -- start 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c38c1f640 1 -- start start 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c38c1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c38c1f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c34161710 0x7f9c340cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c38c1f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3415e000 con 0x7f9c340b9330 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c38c1f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c340d1de0 con 0x7f9c34161710 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c38c1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34161c60 con 0x7f9c3415dc20 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c32d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9c340b9330 0x7f9c3406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41542/0 (socket says 172.21.15.148:41542) 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.408+0000 7f9c32d9d640 1 -- 172.21.15.148:0/2697491402 learned_addr learned my addr 172.21.15.148:0/2697491402 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c3259c640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c2bfff640 1 --2- 172.21.15.148:0/2697491402 >> v2:172.21.15.188:3300/0 conn(0x7f9c34161710 0x7f9c340cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c31d9b640 1 -- 172.21.15.148:0/2697491402 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799212597 0 0) 0x7f9c3415e000 con 0x7f9c340b9330 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c31d9b640 1 -- 172.21.15.148:0/2697491402 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c20003520 con 0x7f9c340b9330 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c31d9b640 1 -- 172.21.15.148:0/2697491402 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3109571666 0 0) 0x7f9c20003520 con 0x7f9c340b9330 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c2bfff640 1 -- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 msgr2=0x7f9c340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c2bfff640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c2bfff640 1 -- 172.21.15.148:0/2697491402 >> v1:172.21.15.148:6789/0 conn(0x7f9c340b9330 legacy=0x7f9c3406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c2bfff640 1 -- 172.21.15.148:0/2697491402 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34161f40 con 0x7f9c34161710 2024-04-25T09:58:13.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.409+0000 7f9c3259c640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.410+0000 7f9c2bfff640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c34161710 0x7f9c340cebb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0097a0 tx=0x7f9c1c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5896f58cdb4b833a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.410+0000 7f9c31d9b640 1 -- 172.21.15.148:0/2697491402 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c037070 con 0x7f9c34161710 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.410+0000 7f9c31d9b640 1 -- 172.21.15.148:0/2697491402 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c02aba0 con 0x7f9c34161710 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.410+0000 7f9c31d9b640 1 -- 172.21.15.148:0/2697491402 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0019f0 con 0x7f9c34161710 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c34161710 msgr2=0x7f9c340cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c34161710 0x7f9c340cebb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0097a0 tx=0x7f9c1c02a110 comp rx=0 tx=0).stop 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2697491402 shutdown_connections 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c34161710 0x7f9c340cebb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2697491402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2697491402 >> 172.21.15.148:0/2697491402 conn(0x7f9c340d0420 msgr2=0x7f9c340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2697491402 shutdown_connections 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.411+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2697491402 wait complete. 2024-04-25T09:58:13.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.412+0000 7f9c38c1f640 1 Processor -- start 2024-04-25T09:58:13.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.412+0000 7f9c38c1f640 1 -- start start 2024-04-25T09:58:13.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c340b9330 0x7f9c34151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 0x7f9c340ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b5e50 con 0x7f9c34161710 2024-04-25T09:58:13.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34162500 con 0x7f9c340b9330 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340b5fd0 con 0x7f9c3415dc20 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 0x7f9c340ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 0x7f9c340ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34758/0 (socket says 172.21.15.148:34758) 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 -- 172.21.15.148:0/2943102022 learned_addr learned my addr 172.21.15.148:0/2943102022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c2bfff640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c3259c640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c340b9330 0x7f9c34151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 -- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 msgr2=0x7f9c340aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 -- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c340b9330 msgr2=0x7f9c34151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c340b9330 0x7f9c34151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 -- 172.21.15.148:0/2943102022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340aacf0 con 0x7f9c34161710 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c32d9d640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 0x7f9c340ccac0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00cb60 tx=0x7f9c2c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00ad40 con 0x7f9c34161710 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c012400 con 0x7f9c34161710 2024-04-25T09:58:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c340aafd0 con 0x7f9c34161710 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.413+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0127d0 con 0x7f9c34161710 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.414+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340ab870 con 0x7f9c34161710 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.415+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c340ab3c0 con 0x7f9c34161710 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.415+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c2c012970 con 0x7f9c34161710 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.415+0000 7f9c2affd640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c1406b4d0 0x7f9c1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.416+0000 7f9c3259c640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c1406b4d0 0x7f9c1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.416+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c2c08a980 con 0x7f9c34161710 2024-04-25T09:58:13.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.416+0000 7f9c3259c640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c1406b4d0 0x7f9c1406d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009560 tx=0x7f9c24009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.420+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c2c059020 con 0x7f9c34161710 2024-04-25T09:58:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.589+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c34073a00 con 0x7f9c34161710 2024-04-25T09:58:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.590+0000 7f9c2affd640 1 -- 172.21.15.148:0/2943102022 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c2c05d0c0 con 0x7f9c34161710 2024-04-25T09:58:13.593 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.591+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c1406b4d0 msgr2=0x7f9c1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.591+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c1406b4d0 0x7f9c1406d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009560 tx=0x7f9c24009190 comp rx=0 tx=0).stop 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.591+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 msgr2=0x7f9c340ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.591+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 0x7f9c340ccac0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00cb60 tx=0x7f9c2c00a430 comp rx=0 tx=0).stop 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 shutdown_connections 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c1406b4d0 0x7f9c1406d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c34161710 0x7f9c340ccac0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c3415dc20 0x7f9c340aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 --2- 172.21.15.148:0/2943102022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c340b9330 0x7f9c34151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 >> 172.21.15.148:0/2943102022 conn(0x7f9c340d0420 msgr2=0x7f9c340d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 shutdown_connections 2024-04-25T09:58:13.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.592+0000 7f9c38c1f640 1 -- 172.21.15.148:0/2943102022 wait complete. 2024-04-25T09:58:13.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:13.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.703+0000 7fe2e3438640 1 Processor -- start 2024-04-25T09:58:13.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.703+0000 7fe2e3438640 1 -- start start 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.704+0000 7fe2e3438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2dc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.704+0000 7fe2e3438640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0cc5e0 con 0x7fe2dc09a090 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.704+0000 7fe2e3438640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0b2ee0 con 0x7fe2dc099cb0 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.704+0000 7fe2e3438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cc3b0 con 0x7fe2dc0cfd30 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.704+0000 7fe2e19d5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe2dc09a090 0x7fe2dc0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56056/0 (socket says 172.21.15.148:56056) 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.704+0000 7fe2e19d5640 1 -- 172.21.15.148:0/3385396905 learned_addr learned my addr 172.21.15.148:0/3385396905 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821267950 0 0) 0x7fe2dc0cc5e0 con 0x7fe2dc09a090 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c8003520 con 0x7fe2dc09a090 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2dc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149084058 0 0) 0x7fe2c8003520 con 0x7fe2dc09a090 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 msgr2=0x7fe2dc15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2dc15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 >> v1:172.21.15.188:6789/0 conn(0x7fe2dc099cb0 legacy=0x7fe2dc0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0d0210 con 0x7fe2dc09a090 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe2d4001670 con 0x7fe2dc09a090 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 >> v1:172.21.15.148:6789/0 conn(0x7fe2dc09a090 legacy=0x7fe2dc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8003f60 0x7fe2c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2c8004d40 0x7fe2c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c800bb50 con 0x7fe2c8003f60 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c800bcd0 con 0x7fe2c8004d40 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c800bed0 con 0x7fe2dc0cfd30 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8003f60 0x7fe2c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e11d4640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e19d5640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2c8004d40 0x7fe2c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 -- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 msgr2=0x7fe2c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2c80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 -- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2c8004d40 msgr2=0x7fe2c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2c8004d40 0x7fe2c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e09d3640 1 -- 172.21.15.148:0/3385396905 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c8003520 con 0x7fe2c8003f60 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e19d5640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2c8004d40 0x7fe2c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.705+0000 7fe2e11d4640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2dc0cfd30 0x7fe2c80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.706+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8003f60 0x7fe2c8004800 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009ac0 tx=0x7fe2c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd1d42f923d4ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.706+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc036070 con 0x7fe2c8003f60 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.706+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc031400 con 0x7fe2c8003f60 2024-04-25T09:58:13.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.706+0000 7fe2d3fff640 1 -- 172.21.15.148:0/3385396905 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc009690 con 0x7fe2c8003f60 2024-04-25T09:58:13.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e3438640 1 -- 172.21.15.148:0/3385396905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8003f60 msgr2=0x7fe2c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e3438640 1 --2- 172.21.15.148:0/3385396905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c8003f60 0x7fe2c8004800 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009ac0 tx=0x7fe2c8005830 comp rx=0 tx=0).stop 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e19d5640 1 -- 172.21.15.148:0/3385396905 reap_dead start 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e3438640 1 -- 172.21.15.148:0/3385396905 shutdown_connections 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e3438640 1 -- 172.21.15.148:0/3385396905 >> 172.21.15.148:0/3385396905 conn(0x7fe2dc0b3290 msgr2=0x7fe2dc0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e3438640 1 -- 172.21.15.148:0/3385396905 shutdown_connections 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.707+0000 7fe2e3438640 1 -- 172.21.15.148:0/3385396905 wait complete. 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.708+0000 7fe2e3438640 1 Processor -- start 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.708+0000 7fe2e3438640 1 -- start start 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2c8003f60 0x7fe2dc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 0x7fe2dc1f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc1f1d20 0x7fe2dc15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0d07d0 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cc880 con 0x7fe2dc1f1d20 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c52f0 con 0x7fe2c8003f60 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 0x7fe2dc1f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 0x7fe2dc1f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33226/0 (socket says 172.21.15.148:33226) 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 -- 172.21.15.148:0/248572771 learned_addr learned my addr 172.21.15.148:0/248572771 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e11d4640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2c8003f60 0x7fe2dc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 -- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2c8003f60 msgr2=0x7fe2dc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2c8003f60 0x7fe2dc158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e19d5640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc1f1d20 0x7fe2dc15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 -- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc1f1d20 msgr2=0x7fe2dc15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc1f1d20 0x7fe2dc15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 -- 172.21.15.148:0/248572771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc159460 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e11d4640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2c8003f60 0x7fe2dc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e19d5640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc1f1d20 0x7fe2dc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e09d3640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 0x7fe2dc1f17e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc034040 tx=0x7fe2cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc040020 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc003bc0 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc036040 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.709+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc1f2210 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.710+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc1f2b00 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.711+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc1f2730 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.711+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2cc043020 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.711+0000 7fe2d1ffb640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.711+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe2cc0aa7e0 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.712+0000 7fe2e11d4640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:13.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.712+0000 7fe2e11d4640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe2c006b500 0x7fe2c006d9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8007fd0 tx=0x7fe2d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:13.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.716+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2dc1f2730 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.884+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2dc0cbff0 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.884+0000 7fe2d1ffb640 1 -- 172.21.15.148:0/248572771 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2dc0cbff0 con 0x7fe2dc0c97a0 2024-04-25T09:58:13.888 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:58:13.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe2c006b500 msgr2=0x7fe2c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe2c006b500 0x7fe2c006d9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8007fd0 tx=0x7fe2d8009040 comp rx=0 tx=0).stop 2024-04-25T09:58:13.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 msgr2=0x7fe2dc1f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 0x7fe2dc1f17e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc034040 tx=0x7fe2cc004730 comp rx=0 tx=0).stop 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 shutdown_connections 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc1f1d20 0x7fe2dc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2dc0c97a0 0x7fe2dc1f17e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 --2- 172.21.15.148:0/248572771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2c8003f60 0x7fe2dc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.886+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 >> 172.21.15.148:0/248572771 conn(0x7fe2dc0b3290 msgr2=0x7fe2dc15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.887+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 shutdown_connections 2024-04-25T09:58:13.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.887+0000 7fe2e3438640 1 -- 172.21.15.148:0/248572771 wait complete. 2024-04-25T09:58:13.901 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.998+0000 7fc2e2dce640 1 Processor -- start 2024-04-25T09:58:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.998+0000 7fc2e2dce640 1 -- start start 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.999+0000 7fc2e2dce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2dc0b38d0 0x7fc2dc0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.999+0000 7fc2e2dce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc15e020 con 0x7fc2dc06bd20 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.999+0000 7fc2e2dce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc06ac80 con 0x7fc2dc06e360 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.999+0000 7fc2e2dce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc06b890 con 0x7fc2dc0b38d0 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.999+0000 7fc2e136b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc2dc06bd20 0x7fc2dc0acea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56072/0 (socket says 172.21.15.148:56072) 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:13.999+0000 7fc2e136b640 1 -- 172.21.15.148:0/1724136227 learned_addr learned my addr 172.21.15.148:0/1724136227 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184592092 0 0) 0x7fc2dc15e020 con 0x7fc2dc06bd20 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2cc003520 con 0x7fc2dc06bd20 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/1724136227 >> v2:172.21.15.188:3300/0 conn(0x7fc2dc0b38d0 0x7fc2dc0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951831595 0 0) 0x7fc2cc003520 con 0x7fc2dc06bd20 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 >> v2:172.21.15.188:3300/0 conn(0x7fc2dc0b38d0 msgr2=0x7fc2dc0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 --2- 172.21.15.148:0/1724136227 >> v2:172.21.15.188:3300/0 conn(0x7fc2dc0b38d0 0x7fc2dc0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 >> v1:172.21.15.188:6789/0 conn(0x7fc2dc06e360 legacy=0x7fc2dc06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc06af60 con 0x7fc2dc06bd20 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc2d40019f0 con 0x7fc2dc06bd20 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/1724136227 >> v2:172.21.15.188:3300/0 conn(0x7fc2dc0b38d0 0x7fc2dc0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 >> v1:172.21.15.148:6789/0 conn(0x7fc2dc06bd20 legacy=0x7fc2dc0acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc0039b0 0x7fc2cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc0051c0 0x7fc2cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc06b890 con 0x7fc2cc004800 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc00c0f0 con 0x7fc2cc0051c0 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc00c2f0 con 0x7fc2cc0039b0 2024-04-25T09:58:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2e0b6a640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc0039b0 0x7fc2cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2e136b640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc0051c0 0x7fc2cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 -- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc0039b0 msgr2=0x7fc2cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc0039b0 0x7fc2cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 -- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc0051c0 msgr2=0x7fc2cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc0051c0 0x7fc2cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2dbfff640 1 -- 172.21.15.148:0/1724136227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc003520 con 0x7fc2cc004800 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.000+0000 7fc2e136b640 1 -- 172.21.15.148:0/1724136227 reap_dead start 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.001+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2cc004c80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80098a0 tx=0x7fc2cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a25df883071230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.001+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800f040 con 0x7fc2cc004800 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.001+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c802ace0 con 0x7fc2cc004800 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.001+0000 7fc2db7fe640 1 -- 172.21.15.148:0/1724136227 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8004250 con 0x7fc2cc004800 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 -- 172.21.15.148:0/1724136227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 msgr2=0x7fc2cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2cc004c80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80098a0 tx=0x7fc2cc005830 comp rx=0 tx=0).stop 2024-04-25T09:58:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 -- 172.21.15.148:0/1724136227 shutdown_connections 2024-04-25T09:58:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/1724136227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2cc004c80 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 -- 172.21.15.148:0/1724136227 >> 172.21.15.148:0/1724136227 conn(0x7fc2dc0cfec0 msgr2=0x7fc2dc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 -- 172.21.15.148:0/1724136227 shutdown_connections 2024-04-25T09:58:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.002+0000 7fc2e2dce640 1 -- 172.21.15.148:0/1724136227 wait complete. 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.003+0000 7fc2e2dce640 1 Processor -- start 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.003+0000 7fc2e2dce640 1 -- start start 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2dc14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2dc1470d0 0x7fc2dc147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2dc145160 0x7fc2dc1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0b2fd0 con 0x7fc2cc004800 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc06b520 con 0x7fc2dc145160 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0b3150 con 0x7fc2dc1470d0 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2dc14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2dc14ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33240/0 (socket says 172.21.15.148:33240) 2024-04-25T09:58:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 -- 172.21.15.148:0/2096768923 learned_addr learned my addr 172.21.15.148:0/2096768923 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2dc1470d0 0x7fc2dc147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e136b640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2dc145160 0x7fc2dc1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 -- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2dc1470d0 msgr2=0x7fc2dc147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2dc1470d0 0x7fc2dc147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 -- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2dc145160 msgr2=0x7fc2dc1455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2dc145160 0x7fc2dc1455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 -- 172.21.15.148:0/2096768923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc145f00 con 0x7fc2cc004800 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e136b640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2dc145160 0x7fc2dc1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e0b6a640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2dc14ae80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e4f0 tx=0x7fc2d000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00052e0 con 0x7fc2cc004800 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0018070 con 0x7fc2cc004800 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc1461e0 con 0x7fc2cc004800 2024-04-25T09:58:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.004+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0003830 con 0x7fc2cc004800 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.005+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc1467a0 con 0x7fc2cc004800 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.006+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc073a00 con 0x7fc2cc004800 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.006+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2d00039d0 con 0x7fc2cc004800 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.006+0000 7fc2d97fa640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2c006b500 0x7fc2c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.006+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2d008a030 con 0x7fc2cc004800 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.006+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2c006b500 0x7fc2c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.007+0000 7fc2dbfff640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2c006b500 0x7fc2c006d9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8033000 tx=0x7fc2c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.011+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2d0058c90 con 0x7fc2cc004800 2024-04-25T09:58:14.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.181+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2dc15e020 con 0x7fc2cc004800 2024-04-25T09:58:14.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.181+0000 7fc2d97fa640 1 -- 172.21.15.148:0/2096768923 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2d0058590 con 0x7fc2cc004800 2024-04-25T09:58:14.185 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2c006b500 msgr2=0x7fc2c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2c006b500 0x7fc2c006d9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8033000 tx=0x7fc2c802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 msgr2=0x7fc2dc14ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2dc14ae80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000e4f0 tx=0x7fc2d000e9b0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 shutdown_connections 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2c006b500 0x7fc2c006d9c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2dc145160 0x7fc2dc1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2dc1470d0 0x7fc2dc147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 --2- 172.21.15.148:0/2096768923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2cc004800 0x7fc2dc14ae80 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 >> 172.21.15.148:0/2096768923 conn(0x7fc2dc0cfec0 msgr2=0x7fc2dc15a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:14.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 shutdown_connections 2024-04-25T09:58:14.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.183+0000 7fc2e2dce640 1 -- 172.21.15.148:0/2096768923 wait complete. 2024-04-25T09:58:14.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:58:14.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.293+0000 7f4f1a088640 1 Processor -- start 2024-04-25T09:58:14.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.293+0000 7f4f1a088640 1 -- start start 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f1a088640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 0x7f4f140aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f1a088640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 0x7f4f1406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f1a088640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f140af540 con 0x7f4f1406b120 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f1a088640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f140af820 con 0x7f4f1415c970 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f1a088640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140afb00 con 0x7f4f141454a0 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f12ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 0x7f4f1406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f137fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 0x7f4f140aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f12ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 0x7f4f1406abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:41958/0 (socket says 172.21.15.148:41958) 2024-04-25T09:58:14.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f137fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 0x7f4f140aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:42244/0 (socket says 172.21.15.148:42244) 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f12ffd640 1 -- 172.21.15.148:0/2144118372 learned_addr learned my addr 172.21.15.148:0/2144118372 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.294+0000 7f4f13fff640 1 --1- 172.21.15.148:0/2144118372 >> v1:172.21.15.188:6789/0 conn(0x7f4f1406b120 0x7f4f1415f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60896/0 (socket says 172.21.15.148:60896) 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.295+0000 7f4f137fe640 1 -- 172.21.15.148:0/2144118372 >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 msgr2=0x7f4f1406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.295+0000 7f4f137fe640 1 --2- 172.21.15.148:0/2144118372 >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 0x7f4f1406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.295+0000 7f4f137fe640 1 -- 172.21.15.148:0/2144118372 >> v1:172.21.15.188:6789/0 conn(0x7f4f1406b120 legacy=0x7f4f1415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.295+0000 7f4f137fe640 1 -- 172.21.15.148:0/2144118372 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140afde0 con 0x7f4f141454a0 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.295+0000 7f4f12ffd640 1 --2- 172.21.15.148:0/2144118372 >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 0x7f4f1406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.295+0000 7f4f137fe640 1 --2- 172.21.15.148:0/2144118372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 0x7f4f140aee60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400dbc0 tx=0x7f4f0402e480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ebf6b93176256cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f127fc640 1 -- 172.21.15.148:0/2144118372 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0403a070 con 0x7f4f141454a0 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f127fc640 1 -- 172.21.15.148:0/2144118372 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0400b650 con 0x7f4f141454a0 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f127fc640 1 -- 172.21.15.148:0/2144118372 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400ba20 con 0x7f4f141454a0 2024-04-25T09:58:14.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 -- 172.21.15.148:0/2144118372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 msgr2=0x7f4f140aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 --2- 172.21.15.148:0/2144118372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 0x7f4f140aee60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400dbc0 tx=0x7f4f0402e480 comp rx=0 tx=0).stop 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 -- 172.21.15.148:0/2144118372 shutdown_connections 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 --2- 172.21.15.148:0/2144118372 >> v2:172.21.15.188:3300/0 conn(0x7f4f1415c970 0x7f4f1406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 --2- 172.21.15.148:0/2144118372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f141454a0 0x7f4f140aee60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 -- 172.21.15.148:0/2144118372 >> 172.21.15.148:0/2144118372 conn(0x7f4f140b5af0 msgr2=0x7f4f140b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 -- 172.21.15.148:0/2144118372 shutdown_connections 2024-04-25T09:58:14.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.296+0000 7f4f1a088640 1 -- 172.21.15.148:0/2144118372 wait complete. 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.297+0000 7f4f1a088640 1 Processor -- start 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.297+0000 7f4f1a088640 1 -- start start 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f1a088640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1406b120 0x7f4f1415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f1a088640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 0x7f4f1409d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f1a088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1415c970 0x7f4f1409d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f1a088640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1415e620 con 0x7f4f141454a0 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f1a088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1415e4a0 con 0x7f4f1415c970 2024-04-25T09:58:14.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f1a088640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1415e7a0 con 0x7f4f1406b120 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f12ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 0x7f4f1409d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f12ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 0x7f4f1409d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33246/0 (socket says 172.21.15.148:33246) 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f12ffd640 1 -- 172.21.15.148:0/4268535144 learned_addr learned my addr 172.21.15.148:0/4268535144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f137fe640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1406b120 0x7f4f1415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.298+0000 7f4f13fff640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1415c970 0x7f4f1409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f12ffd640 1 -- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1406b120 msgr2=0x7f4f1415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f12ffd640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1406b120 0x7f4f1415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f12ffd640 1 -- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1415c970 msgr2=0x7f4f1409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f12ffd640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1415c970 0x7f4f1409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f12ffd640 1 -- 172.21.15.148:0/4268535144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f141430f0 con 0x7f4f141454a0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f12ffd640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 0x7f4f1409d1d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9b0 tx=0x7f4f0800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08017070 con 0x7f4f141454a0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0800c9a0 con 0x7f4f141454a0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1409dd20 con 0x7f4f141454a0 2024-04-25T09:58:14.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800cd50 con 0x7f4f141454a0 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.299+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1409e2e0 con 0x7f4f141454a0 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.300+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1406cc80 con 0x7f4f141454a0 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.300+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f0801a070 con 0x7f4f141454a0 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.301+0000 7f4efbfff640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f0006b4d0 0x7f4f0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.301+0000 7f4f137fe640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f0006b4d0 0x7f4f0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.301+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f0808a100 con 0x7f4f141454a0 2024-04-25T09:58:14.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.301+0000 7f4f137fe640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f0006b4d0 0x7f4f0006d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4f04037000 tx=0x7f4f0400d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.305+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f08058830 con 0x7f4f141454a0 2024-04-25T09:58:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.488+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4f14073a00 con 0x7f4f141454a0 2024-04-25T09:58:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.488+0000 7f4efbfff640 1 -- 172.21.15.148:0/4268535144 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4f0805c8d0 con 0x7f4f141454a0 2024-04-25T09:58:14.492 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T09:58:14.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f0006b4d0 msgr2=0x7f4f0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f0006b4d0 0x7f4f0006d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4f04037000 tx=0x7f4f0400d000 comp rx=0 tx=0).stop 2024-04-25T09:58:14.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 msgr2=0x7f4f1409d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 0x7f4f1409d1d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9b0 tx=0x7f4f0800ee70 comp rx=0 tx=0).stop 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 shutdown_connections 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f0006b4d0 0x7f4f0006d990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f1415c970 0x7f4f1409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f141454a0 0x7f4f1409d1d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 --2- 172.21.15.148:0/4268535144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f1406b120 0x7f4f1415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 >> 172.21.15.148:0/4268535144 conn(0x7f4f140b5af0 msgr2=0x7f4f1415bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.490+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 shutdown_connections 2024-04-25T09:58:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.491+0000 7f4f1a088640 1 -- 172.21.15.148:0/4268535144 wait complete. 2024-04-25T09:58:14.505 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:58:14.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:14.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:14.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:58:14.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:14.544 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:58:14.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.732+0000 7f3f5a1fe640 1 Processor -- start 2024-04-25T09:58:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.732+0000 7f3f5a1fe640 1 -- start start 2024-04-25T09:58:14.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.733+0000 7f3f5a1fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3f5406abe0 0x7f3f5406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.733+0000 7f3f5a1fe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f54073150 con 0x7f3f5415b9c0 2024-04-25T09:58:14.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.733+0000 7f3f5a1fe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540cfa60 con 0x7f3f54072620 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.733+0000 7f3f5a1fe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540cfd40 con 0x7f3f5406abe0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.733+0000 7f3f537fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3f5415b9c0 0x7f3f5415bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56080/0 (socket says 172.21.15.148:56080) 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.733+0000 7f3f537fe640 1 -- 172.21.15.148:0/686721093 learned_addr learned my addr 172.21.15.148:0/686721093 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/686721093 >> v2:172.21.15.188:3300/0 conn(0x7f3f5406abe0 0x7f3f5406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394120803 0 0) 0x7f3f54073150 con 0x7f3f5415b9c0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f40003520 con 0x7f3f5415b9c0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102750251 0 0) 0x7f3f540cfa60 con 0x7f3f54072620 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f54073150 con 0x7f3f54072620 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2625352889 0 0) 0x7f3f40003520 con 0x7f3f5415b9c0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 >> v2:172.21.15.188:3300/0 conn(0x7f3f5406abe0 msgr2=0x7f3f5406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 --2- 172.21.15.148:0/686721093 >> v2:172.21.15.188:3300/0 conn(0x7f3f5406abe0 0x7f3f5406b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 >> v1:172.21.15.188:6789/0 conn(0x7f3f54072620 legacy=0x7f3f54072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540d0020 con 0x7f3f5415b9c0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3f44001970 con 0x7f3f5415b9c0 2024-04-25T09:58:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 >> v1:172.21.15.148:6789/0 conn(0x7f3f5415b9c0 legacy=0x7f3f5415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f40003b90 0x7f3f40003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f400051c0 0x7f3f4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cfd40 con 0x7f3f40004800 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4000c090 con 0x7f3f40003b90 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4000c290 con 0x7f3f400051c0 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f537fe640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f40003b90 0x7f3f40003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f53fff640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f400051c0 0x7f3f4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 -- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f400051c0 msgr2=0x7f3f4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f400051c0 0x7f3f4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 -- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f40003b90 msgr2=0x7f3f40003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f40003b90 0x7f3f40003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f52ffd640 1 -- 172.21.15.148:0/686721093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40003520 con 0x7f3f40004800 2024-04-25T09:58:14.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f53fff640 1 -- 172.21.15.148:0/686721093 reap_dead start 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.734+0000 7f3f537fe640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f40003b90 0x7f3f40003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.735+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f40004c80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009ac0 tx=0x7f3f40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951afe2769de1268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.735+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c036070 con 0x7f3f40004800 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.735+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c031400 con 0x7f3f40004800 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.735+0000 7f3f527fc640 1 -- 172.21.15.148:0/686721093 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c009690 con 0x7f3f40004800 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/686721093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 msgr2=0x7f3f40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f40004c80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009ac0 tx=0x7f3f40005830 comp rx=0 tx=0).stop 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/686721093 shutdown_connections 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/686721093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f40004c80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/686721093 >> 172.21.15.148:0/686721093 conn(0x7f3f540b2370 msgr2=0x7f3f540b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/686721093 shutdown_connections 2024-04-25T09:58:14.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.736+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/686721093 wait complete. 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.737+0000 7f3f5a1fe640 1 Processor -- start 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.737+0000 7f3f5a1fe640 1 -- start start 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f5a1fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f541f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f5a1fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f541503f0 0x7f3f540ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f5a1fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f54150870 0x7f3f540cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f5a1fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5406b070 con 0x7f3f40004800 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f5a1fe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cf710 con 0x7f3f541503f0 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f5a1fe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5406b1f0 con 0x7f3f54150870 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f541f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f541f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33258/0 (socket says 172.21.15.148:33258) 2024-04-25T09:58:14.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 -- 172.21.15.148:0/859660350 learned_addr learned my addr 172.21.15.148:0/859660350 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f541503f0 0x7f3f540ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f53fff640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f54150870 0x7f3f540cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 -- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f54150870 msgr2=0x7f3f540cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f54150870 0x7f3f540cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 -- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f541503f0 msgr2=0x7f3f540ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f541503f0 0x7f3f540ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 -- 172.21.15.148:0/859660350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f541f4640 con 0x7f3f40004800 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f537fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f541f4100 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3f44007a40 tx=0x7f3f4400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.738+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4400edf0 con 0x7f3f40004800 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.739+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44018070 con 0x7f3f40004800 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.739+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f541510f0 con 0x7f3f40004800 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.739+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44013620 con 0x7f3f40004800 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.739+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541516b0 con 0x7f3f40004800 2024-04-25T09:58:14.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.740+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f54073990 con 0x7f3f40004800 2024-04-25T09:58:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.740+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f440137c0 con 0x7f3f40004800 2024-04-25T09:58:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.741+0000 7f3f4bfff640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3406b4d0 0x7f3f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.741+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3406b4d0 0x7f3f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.741+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f4408a6d0 con 0x7f3f40004800 2024-04-25T09:58:14.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.741+0000 7f3f52ffd640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3406b4d0 0x7f3f3406d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c033000 tx=0x7f3f3c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:14.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.745+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f44058e00 con 0x7f3f40004800 2024-04-25T09:58:14.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.952+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f5406ed50 con 0x7f3f40004800 2024-04-25T09:58:14.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.952+0000 7f3f4bfff640 1 -- 172.21.15.148:0/859660350 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f3f4405cea0 con 0x7f3f40004800 2024-04-25T09:58:14.956 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:14.956 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3775040580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3775040580},{"type":"v1","addr":"172.21.15.148:6837","nonce":3775040580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5887,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/733820966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":733820966},{"type":"v1","addr":"172.21.15.188:6834","nonce":733820966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5890,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2576966127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2576966127},{"type":"v1","addr":"172.21.15.188:6835","nonce":2576966127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:03.969534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6729},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6729":{"gid":6729,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6836/2397025400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2397025400},{"type":"v1","addr":"172.21.15.148:6836","nonce":2397025400}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:14.956 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 33 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3406b4d0 msgr2=0x7f3f3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3406b4d0 0x7f3f3406d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c033000 tx=0x7f3f3c008030 comp rx=0 tx=0).stop 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 msgr2=0x7f3f541f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f541f4100 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3f44007a40 tx=0x7f3f4400e530 comp rx=0 tx=0).stop 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 shutdown_connections 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3406b4d0 0x7f3f3406d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f54150870 0x7f3f540cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f541503f0 0x7f3f540ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 --2- 172.21.15.148:0/859660350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f40004800 0x7f3f541f4100 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.954+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 >> 172.21.15.148:0/859660350 conn(0x7f3f540b2370 msgr2=0x7f3f54072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.955+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 shutdown_connections 2024-04-25T09:58:14.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:14.955+0000 7f3f5a1fe640 1 -- 172.21.15.148:0/859660350 wait complete. 2024-04-25T09:58:14.969 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T09:58:15.092 INFO:teuthology.orchestra.run.smithi148.stdout:{"message":"","return_code":0} 2024-04-25T09:58:15.093 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:58:15.093 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:58:15.094 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:15.094 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:58:15.094 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:15.095 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:58:15.095 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:15.095 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:58:15.095 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:15.104 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:58:15.105 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.105 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:58:15.105 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.107 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:58:15.107 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.117 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:58:15.117 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.117 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T09:58:15.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 Processor -- start 2024-04-25T09:58:15.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 -- start start 2024-04-25T09:58:15.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 0x5574f4e1cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574f4e1d8e0 0x5574f4d5d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 0x5574f4d5d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f4d01df0 con 0x7f859c0030f0 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f4d05d50 con 0x5574f4e1d8e0 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b561ef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574f4da3f40 con 0x5574f4e2a710 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b24ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 0x5574f4e1cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b24ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 0x5574f4e1cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33276/0 (socket says 172.21.15.148:33276) 2024-04-25T09:58:15.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.184+0000 7f85b24ad640 1 -- 172.21.15.148:0/1727870352 learned_addr learned my addr 172.21.15.148:0/1727870352 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b24ad640 1 -- 172.21.15.148:0/1727870352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 msgr2=0x5574f4d5d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b24ad640 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 0x5574f4d5d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b24ad640 1 -- 172.21.15.148:0/1727870352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574f4e1d8e0 msgr2=0x5574f4d5d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b24ad640 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574f4e1d8e0 0x5574f4d5d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b24ad640 1 -- 172.21.15.148:0/1727870352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f4e1d540 con 0x7f859c0030f0 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b24ad640 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 0x5574f4e1cd30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f85a4006090 tx=0x7f85a4006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5d378d046248e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b0caa640 1 -- 172.21.15.148:0/1727870352 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4018070 con 0x7f859c0030f0 2024-04-25T09:58:15.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b0caa640 1 -- 172.21.15.148:0/1727870352 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a40037e0 con 0x7f859c0030f0 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b0caa640 1 -- 172.21.15.148:0/1727870352 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4003bb0 con 0x7f859c0030f0 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 -- 172.21.15.148:0/1727870352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 msgr2=0x5574f4e1cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 0x5574f4e1cd30 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f85a4006090 tx=0x7f85a4006810 comp rx=0 tx=0).stop 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 -- 172.21.15.148:0/1727870352 shutdown_connections 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 0x5574f4d5d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5574f4e1d8e0 0x5574f4d5d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 --2- 172.21.15.148:0/1727870352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f859c0030f0 0x5574f4e1cd30 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 -- 172.21.15.148:0/1727870352 >> 172.21.15.148:0/1727870352 conn(0x5574f4d599b0 msgr2=0x5574f4d5a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 -- 172.21.15.148:0/1727870352 shutdown_connections 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.185+0000 7f85b561ef40 1 -- 172.21.15.148:0/1727870352 wait complete. 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.186+0000 7f85b561ef40 1 Processor -- start 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.186+0000 7f85b561ef40 1 -- start start 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574f4e1d8e0 0x5574f4e18df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 0x5574f4e1b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859c0030f0 0x5574f4e31400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f4d5af90 con 0x5574f4e1d8e0 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f4d5ad30 con 0x7f859c0030f0 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574f4e2c340 con 0x5574f4e2a710 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574f4e1d8e0 0x5574f4e18df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574f4e1d8e0 0x5574f4e18df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33280/0 (socket says 172.21.15.148:33280) 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 -- 172.21.15.148:0/3126158451 learned_addr learned my addr 172.21.15.148:0/3126158451 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b1cac640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 0x5574f4e1b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b2cae640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859c0030f0 0x5574f4e31400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 -- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 msgr2=0x5574f4e1b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5574f4e2a710 0x5574f4e1b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 -- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859c0030f0 msgr2=0x5574f4e31400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f859c0030f0 0x5574f4e31400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 -- 172.21.15.148:0/3126158451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f4e1bd40 con 0x5574f4e1d8e0 2024-04-25T09:58:15.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b24ad640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574f4e1d8e0 0x5574f4e18df0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f85a40015a0 tx=0x7f85a4001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85a37fe640 1 -- 172.21.15.148:0/3126158451 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4018070 con 0x5574f4e1d8e0 2024-04-25T09:58:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85a37fe640 1 -- 172.21.15.148:0/3126158451 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a4023070 con 0x5574f4e1d8e0 2024-04-25T09:58:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5574f4e1c020 con 0x5574f4e1d8e0 2024-04-25T09:58:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85a37fe640 1 -- 172.21.15.148:0/3126158451 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a401e660 con 0x5574f4e1d8e0 2024-04-25T09:58:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.187+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574f4e1c5e0 con 0x5574f4e1d8e0 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.189+0000 7f85a37fe640 1 -- 172.21.15.148:0/3126158451 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85a4001dd0 con 0x5574f4e1d8e0 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.189+0000 7f85a37fe640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8594069ee0 0x7f859406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.189+0000 7f85a37fe640 1 -- 172.21.15.148:0/3126158451 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85a408aa20 con 0x5574f4e1d8e0 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.189+0000 7f85b1cac640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8594069ee0 0x7f859406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.189+0000 7f85b561ef40 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x5574f4ecc260 0x5574f4ecc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.189+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 --> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v9 -- 0x5574f4eccba0 con 0x5574f4ecc260 2024-04-25T09:58:15.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.190+0000 7f85b1cac640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8594069ee0 0x7f859406c3a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009810 tx=0x7f85a8051000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.190+0000 7f85b2cae640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x5574f4ecc260 0x5574f4ecc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.190+0000 7f85b2cae640 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x5574f4ecc260 0x5574f4ecc660 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b2cae640 1 -- 172.21.15.148:0/3126158451 <== osd.4 v2:172.21.15.188:6800/3303846211 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v67'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f85ac00e040 con 0x5574f4ecc260 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x5574f4ecc260 msgr2=0x5574f4ecc660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b561ef40 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x5574f4ecc260 0x5574f4ecc660 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8594069ee0 msgr2=0x7f859406c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b561ef40 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8594069ee0 0x7f859406c3a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009810 tx=0x7f85a8051000 comp rx=0 tx=0).stop 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574f4e1d8e0 msgr2=0x5574f4e18df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b561ef40 1 --2- 172.21.15.148:0/3126158451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574f4e1d8e0 0x5574f4e18df0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f85a40015a0 tx=0x7f85a4001d60 comp rx=0 tx=0).stop 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.191+0000 7f85b2cae640 1 -- 172.21.15.148:0/3126158451 reap_dead start 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.192+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 shutdown_connections 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.192+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 >> 172.21.15.148:0/3126158451 conn(0x5574f4d599b0 msgr2=0x5574f4d86730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.192+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 shutdown_connections 2024-04-25T09:58:15.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.192+0000 7f85b561ef40 1 -- 172.21.15.148:0/3126158451 wait complete. 2024-04-25T09:58:15.200 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T09:58:15.200 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T09:58:15.200 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T09:58:15.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T09:58:15.201 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T09:58:15.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T09:58:15.201 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T09:58:15.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T09:58:15.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.338+0000 7f90d2238640 1 Processor -- start 2024-04-25T09:58:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f55d1aee640 1 Processor -- start 2024-04-25T09:58:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7fc73ec7d640 1 Processor -- start 2024-04-25T09:58:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7fc73ec7d640 1 -- start start 2024-04-25T09:58:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f90d2238640 1 -- start start 2024-04-25T09:58:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f55d1aee640 1 -- start start 2024-04-25T09:58:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f3625607640 1 Processor -- start 2024-04-25T09:58:15.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f3625607640 1 -- start start 2024-04-25T09:58:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f90d2238640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90cc0cd8a0 0x7f90cc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f90d2238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f90d2238640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc15e6e0 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f90d2238640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc151560 con 0x7f90cc0cd8a0 2024-04-25T09:58:15.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f90d2238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15e9c0 con 0x7f90cc150b90 2024-04-25T09:58:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.339+0000 7f55d1aee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55d1aee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55cc0cfd30 0x7f55cc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55d1aee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55cc0cc5e0 con 0x7f55cc09a090 2024-04-25T09:58:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55d1aee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc0cc3b0 con 0x7f55cc0cfd30 2024-04-25T09:58:15.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55d1aee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d0210 con 0x7f55cc099cb0 2024-04-25T09:58:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90cc0cd8a0 0x7f90cc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f90cc0cd8a0 0x7f90cc15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33288/0 (socket says 172.21.15.148:33288) 2024-04-25T09:58:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 -- 172.21.15.148:0/3365159935 learned_addr learned my addr 172.21.15.148:0/3365159935 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55cc0cfd30 0x7f55cc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55cc0cfd30 0x7f55cc15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33304/0 (socket says 172.21.15.148:33304) 2024-04-25T09:58:15.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 -- 172.21.15.148:0/1501652501 learned_addr learned my addr 172.21.15.148:0/1501652501 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90cbfff640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55cbfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f55cc09a090 0x7f55cc15f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60938/0 (socket says 172.21.15.148:60938) 2024-04-25T09:58:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55cb7fe640 1 --2- 172.21.15.148:0/1501652501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ec7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc7380cc4f0 0x7fc7380cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90ca7fc640 1 -- 172.21.15.148:0/3365159935 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900048252 0 0) 0x7f90cc15e6e0 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 -- 172.21.15.148:0/3365159935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 msgr2=0x7f90cc151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 -- 172.21.15.148:0/3365159935 >> v1:172.21.15.188:6789/0 conn(0x7f90cc0cd4c0 legacy=0x7f90cc15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 -- 172.21.15.148:0/1501652501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 msgr2=0x7f55cc0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 --2- 172.21.15.148:0/1501652501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc0b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90caffd640 1 -- 172.21.15.148:0/3365159935 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc15a9c0 con 0x7f90cc0cd8a0 2024-04-25T09:58:15.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55ca7fc640 1 -- 172.21.15.148:0/1501652501 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593654354 0 0) 0x7f55cc0cc5e0 con 0x7f55cc09a090 2024-04-25T09:58:15.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 -- 172.21.15.148:0/1501652501 >> v1:172.21.15.188:6789/0 conn(0x7f55cc09a090 legacy=0x7f55cc15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f3625607640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f362015de80 0x7f36200cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f55caffd640 1 -- 172.21.15.148:0/1501652501 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0d04f0 con 0x7f55cc0cfd30 2024-04-25T09:58:15.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ec7d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc73815b980 0x7fc7380cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ec7d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc738072280 con 0x7fc7380cdb20 2024-04-25T09:58:15.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ec7d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc738071270 con 0x7fc73815b980 2024-04-25T09:58:15.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ec7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73815bd80 con 0x7fc7380cc4f0 2024-04-25T09:58:15.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f90cbfff640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f3625607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f3625607640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36200afc80 con 0x7f36200cefb0 2024-04-25T09:58:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f3625607640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36200ce4d0 con 0x7f362015de80 2024-04-25T09:58:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7f3625607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ce7b0 con 0x7f36200ccfe0 2024-04-25T09:58:15.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ca19640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc7380cc4f0 0x7fc7380cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ca19640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc7380cc4f0 0x7fc7380cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:42018/0 (socket says 172.21.15.148:42018) 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc73ca19640 1 -- 172.21.15.148:0/4277004658 learned_addr learned my addr 172.21.15.148:0/4277004658 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.340+0000 7fc737fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc7380cdb20 0x7fc73815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60940/0 (socket says 172.21.15.148:60940) 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc73d21a640 1 --2- 172.21.15.148:0/4277004658 >> v2:172.21.15.148:3300/0 conn(0x7fc73815b980 0x7fc7380cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361e7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f36200cefb0 0x7f36200cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56100/0 (socket says 172.21.15.148:56100) 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f362015de80 0x7f36200cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361e7fc640 1 -- 172.21.15.148:0/4020271922 learned_addr learned my addr 172.21.15.148:0/4020271922 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361f7fe640 1 --2- 172.21.15.148:0/4020271922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc7377fe640 1 -- 172.21.15.148:0/4277004658 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590288196 0 0) 0x7fc738072280 con 0x7fc7380cdb20 2024-04-25T09:58:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc7377fe640 1 -- 172.21.15.148:0/4277004658 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc728003520 con 0x7fc7380cdb20 2024-04-25T09:58:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc73ca19640 1 -- 172.21.15.148:0/4277004658 >> v2:172.21.15.148:3300/0 conn(0x7fc73815b980 msgr2=0x7fc7380cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc73ca19640 1 --2- 172.21.15.148:0/4277004658 >> v2:172.21.15.148:3300/0 conn(0x7fc73815b980 0x7fc7380cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc73ca19640 1 -- 172.21.15.148:0/4277004658 >> v1:172.21.15.188:6789/0 conn(0x7fc7380cdb20 legacy=0x7fc73815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7fc73ca19640 1 -- 172.21.15.148:0/4277004658 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380714c0 con 0x7fc7380cc4f0 2024-04-25T09:58:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361f7fe640 1 -- 172.21.15.148:0/4020271922 >> v2:172.21.15.148:3300/0 conn(0x7f362015de80 msgr2=0x7f36200cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361f7fe640 1 --2- 172.21.15.148:0/4020271922 >> v2:172.21.15.148:3300/0 conn(0x7f362015de80 0x7f36200cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361f7fe640 1 -- 172.21.15.148:0/4020271922 >> v1:172.21.15.148:6789/0 conn(0x7f36200cefb0 legacy=0x7f36200cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f361f7fe640 1 -- 172.21.15.148:0/4020271922 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200cea90 con 0x7f36200ccfe0 2024-04-25T09:58:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f90caffd640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc15b1c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f90c000bb00 tx=0x7f90c002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5cd445e777ee47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f55caffd640 1 --2- 172.21.15.148:0/1501652501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc15ecc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85085acb3ae273ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f90ca7fc640 1 -- 172.21.15.148:0/3365159935 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c003a070 con 0x7f90cc0cd8a0 2024-04-25T09:58:15.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.341+0000 7f90ca7fc640 1 -- 172.21.15.148:0/3365159935 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c0032480 con 0x7f90cc0cd8a0 2024-04-25T09:58:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55ca7fc640 1 -- 172.21.15.148:0/1501652501 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0037070 con 0x7f55cc0cfd30 2024-04-25T09:58:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90ca7fc640 1 -- 172.21.15.148:0/3365159935 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c003a070 con 0x7f90cc0cd8a0 2024-04-25T09:58:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55ca7fc640 1 -- 172.21.15.148:0/1501652501 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c0032440 con 0x7f55cc0cfd30 2024-04-25T09:58:15.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55ca7fc640 1 -- 172.21.15.148:0/1501652501 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0032810 con 0x7f55cc0cfd30 2024-04-25T09:58:15.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7fc73ca19640 1 --2- 172.21.15.148:0/4277004658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7380cc8d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7380cd290 tx=0x7fc72c02c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63731edea8147612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f361f7fe640 1 --2- 172.21.15.148:0/4020271922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200cd450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3618009d90 tx=0x7f361802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b20a606c465a28ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7fc7377fe640 1 -- 172.21.15.148:0/4277004658 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c036020 con 0x7fc7380cc4f0 2024-04-25T09:58:15.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7fc7377fe640 1 -- 172.21.15.148:0/4277004658 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c032640 con 0x7fc7380cc4f0 2024-04-25T09:58:15.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7fc7377fe640 1 -- 172.21.15.148:0/4277004658 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c032a10 con 0x7fc7380cc4f0 2024-04-25T09:58:15.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 -- 172.21.15.148:0/3365159935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 msgr2=0x7f90cc15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f361dffb640 1 -- 172.21.15.148:0/4020271922 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618007740 con 0x7f36200ccfe0 2024-04-25T09:58:15.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc15b1c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f90c000bb00 tx=0x7f90c002c6e0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f361dffb640 1 -- 172.21.15.148:0/4020271922 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36180078e0 con 0x7f36200ccfe0 2024-04-25T09:58:15.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 -- 172.21.15.148:0/1501652501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 msgr2=0x7f55cc15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 --2- 172.21.15.148:0/1501652501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc15ecc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009870 tx=0x7f55c002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f361dffb640 1 -- 172.21.15.148:0/4020271922 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618007cb0 con 0x7f36200ccfe0 2024-04-25T09:58:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 -- 172.21.15.148:0/3365159935 shutdown_connections 2024-04-25T09:58:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 -- 172.21.15.148:0/1501652501 shutdown_connections 2024-04-25T09:58:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 --2- 172.21.15.148:0/1501652501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc15ecc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 --2- 172.21.15.148:0/1501652501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc0b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 -- 172.21.15.148:0/1501652501 >> 172.21.15.148:0/1501652501 conn(0x7f55cc0b3290 msgr2=0x7f55cc0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc15b1c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 --2- 172.21.15.148:0/3365159935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 -- 172.21.15.148:0/3365159935 >> 172.21.15.148:0/3365159935 conn(0x7f90cc06abe0 msgr2=0x7f90cc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f55d1aee640 1 -- 172.21.15.148:0/1501652501 shutdown_connections 2024-04-25T09:58:15.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.342+0000 7f90d2238640 1 -- 172.21.15.148:0/3365159935 shutdown_connections 2024-04-25T09:58:15.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f55d1aee640 1 -- 172.21.15.148:0/1501652501 wait complete. 2024-04-25T09:58:15.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f90d2238640 1 -- 172.21.15.148:0/3365159935 wait complete. 2024-04-25T09:58:15.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 -- 172.21.15.148:0/4020271922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 msgr2=0x7f36200cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 --2- 172.21.15.148:0/4020271922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200cd450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3618009d90 tx=0x7f361802a660 comp rx=0 tx=0).stop 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 -- 172.21.15.148:0/4020271922 shutdown_connections 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 --2- 172.21.15.148:0/4020271922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200cd450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 --2- 172.21.15.148:0/4020271922 >> v2:172.21.15.148:3300/0 conn(0x7f362015de80 0x7f36200cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 -- 172.21.15.148:0/4020271922 >> 172.21.15.148:0/4020271922 conn(0x7f36200b2390 msgr2=0x7f36200b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 -- 172.21.15.148:0/4277004658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 msgr2=0x7fc7380cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/4277004658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7380cc8d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7380cd290 tx=0x7fc72c02c940 comp rx=0 tx=0).stop 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 -- 172.21.15.148:0/4277004658 shutdown_connections 2024-04-25T09:58:15.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 -- 172.21.15.148:0/4020271922 shutdown_connections 2024-04-25T09:58:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/4277004658 >> v2:172.21.15.148:3300/0 conn(0x7fc73815b980 0x7fc7380cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/4277004658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7380cc8d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 -- 172.21.15.148:0/4277004658 >> 172.21.15.148:0/4277004658 conn(0x7fc73806abe0 msgr2=0x7fc73806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 -- 172.21.15.148:0/4277004658 shutdown_connections 2024-04-25T09:58:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7fc73ec7d640 1 -- 172.21.15.148:0/4277004658 wait complete. 2024-04-25T09:58:15.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.343+0000 7f3625607640 1 -- 172.21.15.148:0/4020271922 wait complete. 2024-04-25T09:58:15.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.344+0000 7f90d2238640 1 Processor -- start 2024-04-25T09:58:15.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.344+0000 7f55d1aee640 1 Processor -- start 2024-04-25T09:58:15.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.344+0000 7f90d2238640 1 -- start start 2024-04-25T09:58:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.344+0000 7f55d1aee640 1 -- start start 2024-04-25T09:58:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.344+0000 7fc73ec7d640 1 Processor -- start 2024-04-25T09:58:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.344+0000 7fc73ec7d640 1 -- start start 2024-04-25T09:58:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f3625607640 1 Processor -- start 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90d2238640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 0x7f90cc149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55d1aee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55d1aee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc09a090 0x7f55cc0b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55d1aee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55d1aee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0c52f0 con 0x7f55cc0cfd30 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55d1aee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0d07d0 con 0x7f55cc09a090 2024-04-25T09:58:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55d1aee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0c5470 con 0x7f55cc099cb0 2024-04-25T09:58:15.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f3625607640 1 -- start start 2024-04-25T09:58:15.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90d2238640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc14b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90d2238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90d2238640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15e370 con 0x7f90cc0cd8a0 2024-04-25T09:58:15.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90d2238640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15a670 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90d2238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc06eca0 con 0x7f90cc150b90 2024-04-25T09:58:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55cb7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:42336/0 (socket says 172.21.15.148:42336) 2024-04-25T09:58:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55cb7fe640 1 -- 172.21.15.148:0/664539947 learned_addr learned my addr 172.21.15.148:0/664539947 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f55caffd640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc09a090 0x7f55cc0b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 0x7f90cc149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90cb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 0x7f90cc149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:42022/0 (socket says 172.21.15.148:42022) 2024-04-25T09:58:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7fc73ec7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7381f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90cb7fe640 1 -- 172.21.15.148:0/1265753156 learned_addr learned my addr 172.21.15.148:0/1265753156 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7fc73ec7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 0x7fc738150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90cbfff640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7f90caffd640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc14b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7fc73ec7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815b980 0x7fc73815d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7fc73ec7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738071d60 con 0x7fc73815b980 2024-04-25T09:58:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7fc73ec7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738071a80 con 0x7fc7380cc4f0 2024-04-25T09:58:15.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.345+0000 7fc73ec7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380ae500 con 0x7fc7380cdb20 2024-04-25T09:58:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cbfff640 1 -- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 msgr2=0x7f55cc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cbfff640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cbfff640 1 -- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc09a090 msgr2=0x7f55cc0b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cbfff640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc09a090 0x7f55cc0b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cbfff640 1 -- 172.21.15.148:0/664539947 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc15e0e0 con 0x7f55cc0cfd30 2024-04-25T09:58:15.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cb7fe640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:15.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 0x7fc738150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 0x7fc738150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:42350/0 (socket says 172.21.15.148:42350) 2024-04-25T09:58:15.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 -- 172.21.15.148:0/2799605855 learned_addr learned my addr 172.21.15.148:0/2799605855 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc73d21a640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815b980 0x7fc73815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc73ca19640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7381f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90cb7fe640 1 -- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 msgr2=0x7f90cc14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90cb7fe640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90cb7fe640 1 -- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 msgr2=0x7f90cc14b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90cb7fe640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc14b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90cb7fe640 1 -- 172.21.15.148:0/1265753156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc14bf20 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55cbfff640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc0b5cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c6d0 tx=0x7f55c400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 -- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 msgr2=0x7fc7381f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7381f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 -- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815b980 msgr2=0x7fc73815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f3625607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f3625607640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 0x7f36200c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815b980 0x7fc73815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f3625607640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 0x7f36200b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 -- 172.21.15.148:0/2799605855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738151110 con 0x7fc7380cdb20 2024-04-25T09:58:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f3625607640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200cfb50 con 0x7f36200cefb0 2024-04-25T09:58:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f3625607640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b2eb0 con 0x7f362015de80 2024-04-25T09:58:15.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f3625607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b05e0 con 0x7f36200ccfe0 2024-04-25T09:58:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90cb7fe640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 0x7f90cc149460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c410 tx=0x7f90bc00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f361f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 0x7f36200b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f361effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4018070 con 0x7f55cc0cfd30 2024-04-25T09:58:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f361e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 0x7f36200c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f361f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 0x7f36200b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:42042/0 (socket says 172.21.15.148:42042) 2024-04-25T09:58:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f361e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 0x7f36200c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33374/0 (socket says 172.21.15.148:33374) 2024-04-25T09:58:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc737fff640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 0x7fc738150bd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c500 tx=0x7fc72400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc0b46d0 con 0x7f55cc0cfd30 2024-04-25T09:58:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f361f7fe640 1 -- 172.21.15.148:0/812612686 learned_addr learned my addr 172.21.15.148:0/812612686 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c4004d60 con 0x7f55cc0cfd30 2024-04-25T09:58:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400a880 con 0x7f55cc0cfd30 2024-04-25T09:58:15.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc017070 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc0035f0 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc003a40 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc0b4c90 con 0x7f55cc0cfd30 2024-04-25T09:58:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc14c200 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.346+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724018070 con 0x7fc7380cdb20 2024-04-25T09:58:15.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc724007500 con 0x7fc7380cdb20 2024-04-25T09:58:15.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724004a20 con 0x7fc7380cdb20 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361f7fe640 1 -- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 msgr2=0x7f36200c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361f7fe640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361f7fe640 1 -- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 msgr2=0x7f36200c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361f7fe640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 0x7f36200c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361f7fe640 1 -- 172.21.15.148:0/812612686 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36201577a0 con 0x7f362015de80 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361effd640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361e7fc640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 0x7f36200c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f361f7fe640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 0x7f36200b0a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3618009eb0 tx=0x7f3618004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73815db70 con 0x7fc7380cdb20 2024-04-25T09:58:15.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361803f070 con 0x7f362015de80 2024-04-25T09:58:15.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618031c60 con 0x7f362015de80 2024-04-25T09:58:15.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7381f3ed0 con 0x7fc7380cdb20 2024-04-25T09:58:15.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618009450 con 0x7f362015de80 2024-04-25T09:58:15.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3620156f00 con 0x7f362015de80 2024-04-25T09:58:15.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.347+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36201574c0 con 0x7f362015de80 2024-04-25T09:58:15.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.348+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc14c7c0 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.348+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc073a00 con 0x7f55cc0cfd30 2024-04-25T09:58:15.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.348+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55c401b070 con 0x7f55cc0cfd30 2024-04-25T09:58:15.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7f55b3fff640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55b806b500 0x7f55b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55c40897e0 con 0x7f55cc0cfd30 2024-04-25T09:58:15.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738073a00 con 0x7fc7380cdb20 2024-04-25T09:58:15.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3620073a00 con 0x7f362015de80 2024-04-25T09:58:15.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc073a00 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7f55cb7fe640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55b806b500 0x7f55b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.349+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc72401b070 con 0x7fc7380cdb20 2024-04-25T09:58:15.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f361803b410 con 0x7f362015de80 2024-04-25T09:58:15.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7f55cb7fe640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55b806b500 0x7f55b806d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0cee10 tx=0x7f55bc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7fc7357fa640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc71c06b5a0 0x7fc71c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7f36077fe640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f360c06b4d0 0x7f360c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36180ab290 con 0x7f362015de80 2024-04-25T09:58:15.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc724089780 con 0x7fc7380cdb20 2024-04-25T09:58:15.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7fc73ca19640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc71c06b5a0 0x7fc71c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7f361effd640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f360c06b4d0 0x7f360c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90bc01a070 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.350+0000 7fc73ca19640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc71c06b5a0 0x7fc71c06da60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc72c037000 tx=0x7fc72c032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.351+0000 7f361effd640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f360c06b4d0 0x7f360c06d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3610009f30 tx=0x7f3610009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.351+0000 7f90b3fff640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b806b5a0 0x7f90b806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.351+0000 7f90caffd640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b806b5a0 0x7f90b806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.351+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90bc089550 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.351+0000 7f90caffd640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b806b5a0 0x7f90b806da60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90c0037000 tx=0x7f90c0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.354+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55c4057e70 con 0x7f55cc0cfd30 2024-04-25T09:58:15.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.355+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc724057f50 con 0x7fc7380cdb20 2024-04-25T09:58:15.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.355+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3618079890 con 0x7f362015de80 2024-04-25T09:58:15.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.359+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90bc057bd0 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.608+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f36200d0110 con 0x7f362015de80 2024-04-25T09:58:15.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.608+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f90cc0ccb10 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.622+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fc73806d3a0 con 0x7fc7380cdb20 2024-04-25T09:58:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.624+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f55cc0c7560 con 0x7f55cc0cfd30 2024-04-25T09:58:15.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.699+0000 7f36077fe640 1 -- 172.21.15.148:0/812612686 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5890 v34) v1 ==== 95+0+0 (secure 0 0 0) 0x7f361807d930 con 0x7f362015de80 2024-04-25T09:58:15.703 INFO:teuthology.orchestra.run.smithi148.stderr:failed mds gid 5890 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f360c06b4d0 msgr2=0x7f360c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f360c06b4d0 0x7f360c06d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3610009f30 tx=0x7f3610009190 comp rx=0 tx=0).stop 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 msgr2=0x7f36200b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 0x7f36200b0a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3618009eb0 tx=0x7f3618004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 shutdown_connections 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f360c06b4d0 0x7f360c06d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362015de80 0x7f36200b0a70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36200cefb0 0x7f36200c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 --2- 172.21.15.148:0/812612686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36200ccfe0 0x7f36200c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 >> 172.21.15.148:0/812612686 conn(0x7f36200b2390 msgr2=0x7f36200cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 shutdown_connections 2024-04-25T09:58:15.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.702+0000 7f3625607640 1 -- 172.21.15.148:0/812612686 wait complete. 2024-04-25T09:58:15.718 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:58:15.719 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:58:15.720 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:58:15.717+0000 7fb15676d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:58:15.721 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:58:15.721 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.722+0000 7f55b3fff640 1 -- 172.21.15.148:0/664539947 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6729 v35) v1 ==== 95+0+0 (secure 0 0 0) 0x7f55c405bf10 con 0x7f55cc0cfd30 2024-04-25T09:58:15.725 INFO:teuthology.orchestra.run.smithi148.stderr:failed mds gid 6729 2024-04-25T09:58:15.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55b806b500 msgr2=0x7f55b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55b806b500 0x7f55b806d9c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0cee10 tx=0x7f55bc009190 comp rx=0 tx=0).stop 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 msgr2=0x7f55cc0b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc0b5cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f55c400c6d0 tx=0x7f55c400cb90 comp rx=0 tx=0).stop 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 shutdown_connections 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55b806b500 0x7f55b806d9c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55cc0cfd30 0x7f55cc0b5cb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55cc09a090 0x7f55cc0b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 --2- 172.21.15.148:0/664539947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55cc099cb0 0x7f55cc06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 >> 172.21.15.148:0/664539947 conn(0x7f55cc0b3290 msgr2=0x7f55cc0c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 shutdown_connections 2024-04-25T09:58:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.724+0000 7f55d1aee640 1 -- 172.21.15.148:0/664539947 wait complete. 2024-04-25T09:58:15.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.725+0000 7f90b3fff640 1 -- 172.21.15.148:0/1265753156 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5887 v35) v1 ==== 95+0+0 (secure 0 0 0) 0x7f90bc05bc70 con 0x7f90cc0cd4c0 2024-04-25T09:58:15.729 INFO:teuthology.orchestra.run.smithi148.stderr:failed mds gid 5887 2024-04-25T09:58:15.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.726+0000 7fc7357fa640 1 -- 172.21.15.148:0/2799605855 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 5474 v35) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc72405bff0 con 0x7fc7380cdb20 2024-04-25T09:58:15.729 INFO:teuthology.orchestra.run.smithi148.stderr:failed mds gid 5474 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b806b5a0 msgr2=0x7f90b806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b806b5a0 0x7f90b806da60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f90c0037000 tx=0x7f90c0030000 comp rx=0 tx=0).stop 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 msgr2=0x7f90cc149460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 0x7f90cc149460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c410 tx=0x7f90bc00c810 comp rx=0 tx=0).stop 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 shutdown_connections 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90b806b5a0 0x7f90b806da60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90cc150b90 0x7f90cc14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90cc0cd8a0 0x7f90cc14b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 --2- 172.21.15.148:0/1265753156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90cc0cd4c0 0x7f90cc149460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 >> 172.21.15.148:0/1265753156 conn(0x7f90cc06abe0 msgr2=0x7f90cc15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 shutdown_connections 2024-04-25T09:58:15.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.727+0000 7f90d2238640 1 -- 172.21.15.148:0/1265753156 wait complete. 2024-04-25T09:58:15.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.728+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc71c06b5a0 msgr2=0x7fc71c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.728+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc71c06b5a0 0x7fc71c06da60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc72c037000 tx=0x7fc72c032000 comp rx=0 tx=0).stop 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.728+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 msgr2=0x7fc738150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.728+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 0x7fc738150bd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c500 tx=0x7fc72400c9c0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 shutdown_connections 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc71c06b5a0 0x7fc71c06da60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815b980 0x7fc73815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7380cdb20 0x7fc738150bd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 --2- 172.21.15.148:0/2799605855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7380cc4f0 0x7fc7381f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 >> 172.21.15.148:0/2799605855 conn(0x7fc73806abe0 msgr2=0x7fc73806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 shutdown_connections 2024-04-25T09:58:15.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.729+0000 7fc73ec7d640 1 -- 172.21.15.148:0/2799605855 wait complete. 2024-04-25T09:58:15.742 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:58:15.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:58:15.749 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:58:15.749 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:58:15.750 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:58:15.750 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:58:15.751 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:58:15.751 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.753 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:58:15.753 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.793 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:58:15.793 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:15.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:15.800 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T09:58:15.822 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:58:15.826 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:58:15.869 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:58:15.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.889+0000 7f9a3ae9e640 1 Processor -- start 2024-04-25T09:58:15.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3ae9e640 1 -- start start 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3ae9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3415dbe0 0x7f9a340d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3ae9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a340b92a0 0x7f9a340d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3ae9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a340b97a0 con 0x7f9a340b2cb0 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3ae9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3415dfc0 con 0x7f9a340b92a0 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3ae9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340d1a00 con 0x7f9a3415dbe0 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.890+0000 7f9a3943b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a340b92a0 0x7f9a340d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9a340b92a0 0x7f9a340d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33394/0 (socket says 172.21.15.148:33394) 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 -- 172.21.15.148:0/2168440293 learned_addr learned my addr 172.21.15.148:0/2168440293 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a38c3a640 1 --2- 172.21.15.148:0/2168440293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3415dbe0 0x7f9a340d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 -- 172.21.15.148:0/2168440293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3415dbe0 msgr2=0x7f9a340d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 --2- 172.21.15.148:0/2168440293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3415dbe0 0x7f9a340d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 -- 172.21.15.148:0/2168440293 >> v1:172.21.15.188:6789/0 conn(0x7f9a340b2cb0 legacy=0x7f9a340b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 -- 172.21.15.148:0/2168440293 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340d1c50 con 0x7f9a340b92a0 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.891+0000 7f9a3943b640 1 --2- 172.21.15.148:0/2168440293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340b92a0 0x7f9a340d0e50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009e50 tx=0x7f9a2c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34710d9402a161df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a337fe640 1 -- 172.21.15.148:0/2168440293 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c036070 con 0x7f9a340b92a0 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a337fe640 1 -- 172.21.15.148:0/2168440293 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c031600 con 0x7f9a340b92a0 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a337fe640 1 -- 172.21.15.148:0/2168440293 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0319d0 con 0x7f9a340b92a0 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/2168440293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340b92a0 msgr2=0x7f9a340d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/2168440293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340b92a0 0x7f9a340d0e50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009e50 tx=0x7f9a2c02a6c0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/2168440293 shutdown_connections 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/2168440293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a340b92a0 0x7f9a340d0e50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/2168440293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a3415dbe0 0x7f9a340d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/2168440293 >> 172.21.15.148:0/2168440293 conn(0x7f9a34061a20 msgr2=0x7f9a34061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:15.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/2168440293 shutdown_connections 2024-04-25T09:58:15.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.892+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/2168440293 wait complete. 2024-04-25T09:58:15.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.893+0000 7f9a3ae9e640 1 Processor -- start 2024-04-25T09:58:15.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.893+0000 7f9a3ae9e640 1 -- start start 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3ae9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340b2cb0 0x7f9a340b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3ae9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b92a0 0x7f9a340cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3ae9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a3415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3ae9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340b6ab0 con 0x7f9a3415dbe0 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3ae9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340d2210 con 0x7f9a340b92a0 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3ae9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340b6c30 con 0x7f9a340b2cb0 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3943b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a3415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3943b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a3415f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33410/0 (socket says 172.21.15.148:33410) 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.894+0000 7f9a3943b640 1 -- 172.21.15.148:0/1448440669 learned_addr learned my addr 172.21.15.148:0/1448440669 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a38c3a640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340b2cb0 0x7f9a340b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a33fff640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b92a0 0x7f9a340cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3943b640 1 -- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340b2cb0 msgr2=0x7f9a340b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3943b640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340b2cb0 0x7f9a340b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3943b640 1 -- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b92a0 msgr2=0x7f9a340cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3943b640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b92a0 0x7f9a340cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3943b640 1 -- 172.21.15.148:0/1448440669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340cd9b0 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a38c3a640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340b2cb0 0x7f9a340b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3943b640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a3415f1a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009e50 tx=0x7f9a2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c036050 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c042070 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c009510 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a340cdc90 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.895+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a340b3940 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.896+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a34073990 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.897+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a2c007bd0 con 0x7f9a3415dbe0 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.897+0000 7f9a317fa640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a1806b500 0x7f9a1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:15.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.897+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9a2c0ab2d0 con 0x7f9a3415dbe0 2024-04-25T09:58:15.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.897+0000 7f9a38c3a640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a1806b500 0x7f9a1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:15.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.897+0000 7f9a38c3a640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a1806b500 0x7f9a1806d9c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006fd0 tx=0x7f9a28008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:15.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:15.901+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a2c079f70 con 0x7f9a3415dbe0 2024-04-25T09:58:16.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.107+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a340d2b60 con 0x7f9a3415dbe0 2024-04-25T09:58:16.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.108+0000 7f9a317fa640 1 -- 172.21.15.148:0/1448440669 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+1768 (secure 0 0 0) 0x7f9a2c079870 con 0x7f9a3415dbe0 2024-04-25T09:58:16.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:16.112 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:15.718248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:16.112 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 35 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a1806b500 msgr2=0x7f9a1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a1806b500 0x7f9a1806d9c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006fd0 tx=0x7f9a28008040 comp rx=0 tx=0).stop 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 msgr2=0x7f9a3415f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a3415f1a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009e50 tx=0x7f9a2c004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 shutdown_connections 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a1806b500 0x7f9a1806d9c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a3415dbe0 0x7f9a3415f1a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a340b92a0 0x7f9a340cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 --2- 172.21.15.148:0/1448440669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a340b2cb0 0x7f9a340b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 >> 172.21.15.148:0/1448440669 conn(0x7f9a34061a20 msgr2=0x7f9a340b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 shutdown_connections 2024-04-25T09:58:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:16.110+0000 7f9a3ae9e640 1 -- 172.21.15.148:0/1448440669 wait complete. 2024-04-25T09:58:16.527 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:01.965750+0000', 'modified': '2024-04-25T09:58:15.718248+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:58:16.527 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-25T09:58:17.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:17.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.635+0000 7ff17495a640 1 Processor -- start 2024-04-25T09:58:17.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.635+0000 7ff17495a640 1 -- start start 2024-04-25T09:58:17.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.635+0000 7ff17495a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff17015dbe0 0x7ff1700d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.635+0000 7ff17495a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1700b6220 con 0x7ff1700b92a0 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.635+0000 7ff17495a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1700b97a0 con 0x7ff1700b2cb0 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.635+0000 7ff17495a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17015dfc0 con 0x7ff17015dbe0 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16ed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff1700b92a0 0x7ff1700b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56132/0 (socket says 172.21.15.148:56132) 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16ed9d640 1 -- 172.21.15.148:0/1035062203 learned_addr learned my addr 172.21.15.148:0/1035062203 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16e59c640 1 --2- 172.21.15.148:0/1035062203 >> v2:172.21.15.188:3300/0 conn(0x7ff17015dbe0 0x7ff1700d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16dd9b640 1 -- 172.21.15.148:0/1035062203 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811149178 0 0) 0x7ff1700b6220 con 0x7ff1700b92a0 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16dd9b640 1 -- 172.21.15.148:0/1035062203 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff15c003520 con 0x7ff1700b92a0 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16e59c640 1 -- 172.21.15.148:0/1035062203 >> v1:172.21.15.188:6789/0 conn(0x7ff1700b2cb0 legacy=0x7ff1700b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16e59c640 1 -- 172.21.15.148:0/1035062203 >> v1:172.21.15.148:6789/0 conn(0x7ff1700b92a0 legacy=0x7ff1700b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.636+0000 7ff16e59c640 1 -- 172.21.15.148:0/1035062203 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1700d1c50 con 0x7ff17015dbe0 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.637+0000 7ff16e59c640 1 --2- 172.21.15.148:0/1035062203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff17015dbe0 0x7ff1700d0910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1600097a0 tx=0x7ff16002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fda62dc75bfbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.637+0000 7ff16dd9b640 1 -- 172.21.15.148:0/1035062203 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160037070 con 0x7ff17015dbe0 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.637+0000 7ff16dd9b640 1 -- 172.21.15.148:0/1035062203 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff160032440 con 0x7ff17015dbe0 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.637+0000 7ff16dd9b640 1 -- 172.21.15.148:0/1035062203 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160032810 con 0x7ff17015dbe0 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 -- 172.21.15.148:0/1035062203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff17015dbe0 msgr2=0x7ff1700d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 --2- 172.21.15.148:0/1035062203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff17015dbe0 0x7ff1700d0910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1600097a0 tx=0x7ff16002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 -- 172.21.15.148:0/1035062203 shutdown_connections 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 --2- 172.21.15.148:0/1035062203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff17015dbe0 0x7ff1700d0910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 -- 172.21.15.148:0/1035062203 >> 172.21.15.148:0/1035062203 conn(0x7ff170061a20 msgr2=0x7ff170061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 -- 172.21.15.148:0/1035062203 shutdown_connections 2024-04-25T09:58:17.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.638+0000 7ff17495a640 1 -- 172.21.15.148:0/1035062203 wait complete. 2024-04-25T09:58:17.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.639+0000 7ff17495a640 1 Processor -- start 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.639+0000 7ff17495a640 1 -- start start 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff17495a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 0x7ff170147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff17495a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1700b92a0 0x7ff1700b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff17495a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff17015dbe0 0x7ff1700b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff17495a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1700d2210 con 0x7ff1700b2cb0 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff17495a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1700d1a00 con 0x7ff1700b92a0 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff17495a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1700b6ab0 con 0x7ff17015dbe0 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 0x7ff170147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 0x7ff170147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33414/0 (socket says 172.21.15.148:33414) 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 -- 172.21.15.148:0/2266961179 learned_addr learned my addr 172.21.15.148:0/2266961179 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff167fff640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1700b92a0 0x7ff1700b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 -- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff17015dbe0 msgr2=0x7ff1700b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16ed9d640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff17015dbe0 0x7ff1700b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff17015dbe0 0x7ff1700b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 -- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1700b92a0 msgr2=0x7ff1700b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1700b92a0 0x7ff1700b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 -- 172.21.15.148:0/2266961179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1700cb760 con 0x7ff1700b2cb0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff167fff640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1700b92a0 0x7ff1700b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16ed9d640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff17015dbe0 0x7ff1700b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.640+0000 7ff16e59c640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 0x7ff170147b10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff160000c00 tx=0x7ff1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.641+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160037040 con 0x7ff1700b2cb0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.641+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff160042070 con 0x7ff1700b2cb0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.641+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1700cba40 con 0x7ff1700b2cb0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.641+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160030940 con 0x7ff1700b2cb0 2024-04-25T09:58:17.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.641+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1700cc000 con 0x7ff1700b2cb0 2024-04-25T09:58:17.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.642+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff170073990 con 0x7ff1700b2cb0 2024-04-25T09:58:17.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.642+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff160030ae0 con 0x7ff1700b2cb0 2024-04-25T09:58:17.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.642+0000 7ff166ffd640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff15006b4d0 0x7ff15006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:17.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.643+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff1600aacd0 con 0x7ff1700b2cb0 2024-04-25T09:58:17.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.643+0000 7ff167fff640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff15006b4d0 0x7ff15006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:17.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.643+0000 7ff167fff640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff15006b4d0 0x7ff15006d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff1701525f0 tx=0x7ff158009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:17.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.647+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff160045360 con 0x7ff1700b2cb0 2024-04-25T09:58:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.855+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1700ca2e0 con 0x7ff1700b2cb0 2024-04-25T09:58:17.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.855+0000 7ff166ffd640 1 -- 172.21.15.148:0/2266961179 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4952 (secure 0 0 0) 0x7ff160079970 con 0x7ff1700b2cb0 2024-04-25T09:58:17.859 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:17.859 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:16.739409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:17.859 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 38 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff15006b4d0 msgr2=0x7ff15006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff15006b4d0 0x7ff15006d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff1701525f0 tx=0x7ff158009040 comp rx=0 tx=0).stop 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 msgr2=0x7ff170147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 0x7ff170147b10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff160000c00 tx=0x7ff1600046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 shutdown_connections 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff15006b4d0 0x7ff15006d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff17015dbe0 0x7ff1700b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1700b92a0 0x7ff1700b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 --2- 172.21.15.148:0/2266961179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1700b2cb0 0x7ff170147b10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 >> 172.21.15.148:0/2266961179 conn(0x7ff170061a20 msgr2=0x7ff1700b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 shutdown_connections 2024-04-25T09:58:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:17.857+0000 7ff17495a640 1 -- 172.21.15.148:0/2266961179 wait complete. 2024-04-25T09:58:18.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:01.965750+0000', 'modified': '2024-04-25T09:58:16.739409+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.148:6835/328449171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 328449171}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 328449171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:58:18.274 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6115:up:reconnect 2024-04-25T09:58:19.275 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:19.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.380+0000 7f10e4b7e640 1 Processor -- start 2024-04-25T09:58:19.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.380+0000 7f10e4b7e640 1 -- start start 2024-04-25T09:58:19.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10e4b7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10e0071ac0 0x7f10e00ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10e4b7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e015ec60 con 0x7f10e006c1d0 2024-04-25T09:58:19.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10e4b7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e015ef40 con 0x7f10e0154ed0 2024-04-25T09:58:19.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10e4b7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e006db90 con 0x7f10e0071ac0 2024-04-25T09:58:19.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10ddd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10e0071ac0 0x7f10e00ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10ded9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f10e006c1d0 0x7f10e006afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56142/0 (socket says 172.21.15.148:56142) 2024-04-25T09:58:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.381+0000 7f10ded9d640 1 -- 172.21.15.148:0/2538902787 learned_addr learned my addr 172.21.15.148:0/2538902787 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.382+0000 7f10ddd9b640 1 -- 172.21.15.148:0/2538902787 >> v1:172.21.15.188:6789/0 conn(0x7f10e0154ed0 legacy=0x7f10e00aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.382+0000 7f10ddd9b640 1 -- 172.21.15.148:0/2538902787 >> v1:172.21.15.148:6789/0 conn(0x7f10e006c1d0 legacy=0x7f10e006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.382+0000 7f10ddd9b640 1 -- 172.21.15.148:0/2538902787 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e006de70 con 0x7f10e0071ac0 2024-04-25T09:58:19.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.382+0000 7f10dd59a640 1 -- 172.21.15.148:0/2538902787 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269276618 0 0) 0x7f10e015ec60 con 0x7f10e006c1d0 2024-04-25T09:58:19.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.382+0000 7f10ddd9b640 1 --2- 172.21.15.148:0/2538902787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0071ac0 0x7f10e00ad5e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f10d40097a0 tx=0x7f10d402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f44754111ccd859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.382+0000 7f10dd59a640 1 -- 172.21.15.148:0/2538902787 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4037070 con 0x7f10e0071ac0 2024-04-25T09:58:19.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10dd59a640 1 -- 172.21.15.148:0/2538902787 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d402ae50 con 0x7f10e0071ac0 2024-04-25T09:58:19.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10dd59a640 1 -- 172.21.15.148:0/2538902787 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4032660 con 0x7f10e0071ac0 2024-04-25T09:58:19.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 -- 172.21.15.148:0/2538902787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0071ac0 msgr2=0x7f10e00ad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/2538902787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0071ac0 0x7f10e00ad5e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f10d40097a0 tx=0x7f10d402a140 comp rx=0 tx=0).stop 2024-04-25T09:58:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 -- 172.21.15.148:0/2538902787 shutdown_connections 2024-04-25T09:58:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/2538902787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0071ac0 0x7f10e00ad5e0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 -- 172.21.15.148:0/2538902787 >> 172.21.15.148:0/2538902787 conn(0x7f10e00af670 msgr2=0x7f10e00b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 -- 172.21.15.148:0/2538902787 shutdown_connections 2024-04-25T09:58:19.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.383+0000 7f10e4b7e640 1 -- 172.21.15.148:0/2538902787 wait complete. 2024-04-25T09:58:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 Processor -- start 2024-04-25T09:58:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 -- start start 2024-04-25T09:58:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10e0071ac0 0x7f10e00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 0x7f10e0142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e00afa60 0x7f10e00ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e006b830 con 0x7f10e0154ed0 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e006b6b0 con 0x7f10e00afa60 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10e4b7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e006b9b0 con 0x7f10e0071ac0 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10ddd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 0x7f10e0142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10ddd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 0x7f10e0142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33432/0 (socket says 172.21.15.148:33432) 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.385+0000 7f10ddd9b640 1 -- 172.21.15.148:0/3896669875 learned_addr learned my addr 172.21.15.148:0/3896669875 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ded9d640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e00afa60 0x7f10e00ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10de59c640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10e0071ac0 0x7f10e00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ddd9b640 1 -- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10e0071ac0 msgr2=0x7f10e00b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ddd9b640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10e0071ac0 0x7f10e00b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ddd9b640 1 -- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e00afa60 msgr2=0x7f10e00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ddd9b640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e00afa60 0x7f10e00ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ddd9b640 1 -- 172.21.15.148:0/3896669875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00b0e90 con 0x7f10e0154ed0 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ded9d640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e00afa60 0x7f10e00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10de59c640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10e0071ac0 0x7f10e00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ddd9b640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 0x7f10e0142630 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f10d40097a0 tx=0x7f10d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4037040 con 0x7f10e0154ed0 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d4004030 con 0x7f10e0154ed0 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e00b1160 con 0x7f10e0154ed0 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4006e10 con 0x7f10e0154ed0 2024-04-25T09:58:19.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.386+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e00b1720 con 0x7f10e0154ed0 2024-04-25T09:58:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.387+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e00b2b10 con 0x7f10e0154ed0 2024-04-25T09:58:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.387+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10d403d650 con 0x7f10e0154ed0 2024-04-25T09:58:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.388+0000 7f10ceffd640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10c006b4d0 0x7f10c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.388+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f10d40aac30 con 0x7f10e0154ed0 2024-04-25T09:58:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.388+0000 7f10de59c640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10c006b4d0 0x7f10c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:19.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.388+0000 7f10de59c640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10c006b4d0 0x7f10c006d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10d000a830 tx=0x7f10d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:19.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.392+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10d4042020 con 0x7f10e0154ed0 2024-04-25T09:58:19.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.600+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10e014fda0 con 0x7f10e0154ed0 2024-04-25T09:58:19.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.600+0000 7f10ceffd640 1 -- 172.21.15.148:0/3896669875 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f10d40798d0 con 0x7f10e0154ed0 2024-04-25T09:58:19.604 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:19.604 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:17.740144+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:19.605 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 39 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10c006b4d0 msgr2=0x7f10c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10c006b4d0 0x7f10c006d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10d000a830 tx=0x7f10d0008040 comp rx=0 tx=0).stop 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 msgr2=0x7f10e0142630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 0x7f10e0142630 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f10d40097a0 tx=0x7f10d40046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 shutdown_connections 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10c006b4d0 0x7f10c006d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e00afa60 0x7f10e00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e0154ed0 0x7f10e0142630 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 --2- 172.21.15.148:0/3896669875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10e0071ac0 0x7f10e00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.602+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 >> 172.21.15.148:0/3896669875 conn(0x7f10e006c1d0 msgr2=0x7f10e0160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.603+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 shutdown_connections 2024-04-25T09:58:19.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:19.603+0000 7f10e4b7e640 1 -- 172.21.15.148:0/3896669875 wait complete. 2024-04-25T09:58:20.020 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:01.965750+0000', 'modified': '2024-04-25T09:58:17.740144+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.148:6835/328449171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 328449171}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 328449171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:58:20.020 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6115:up:rejoin 2024-04-25T09:58:21.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:21.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.127+0000 7fda4aaf5640 1 Processor -- start 2024-04-25T09:58:21.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.127+0000 7fda4aaf5640 1 -- start start 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda4aaf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda440d2e90 0x7fda440d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda4aaf5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440b9940 con 0x7fda440d0900 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda4aaf5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440d1df0 con 0x7fda4415dd30 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda4aaf5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440d0d40 con 0x7fda440d2e90 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda43fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda440d2e90 0x7fda440d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda49092640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fda440d0900 0x7fda440b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56156/0 (socket says 172.21.15.148:56156) 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda49092640 1 -- 172.21.15.148:0/3106621885 learned_addr learned my addr 172.21.15.148:0/3106621885 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda43fff640 1 -- 172.21.15.148:0/3106621885 >> v1:172.21.15.188:6789/0 conn(0x7fda4415dd30 legacy=0x7fda440b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda43fff640 1 -- 172.21.15.148:0/3106621885 >> v1:172.21.15.148:6789/0 conn(0x7fda440d0900 legacy=0x7fda440b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda437fe640 1 -- 172.21.15.148:0/3106621885 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883560017 0 0) 0x7fda440b9940 con 0x7fda440d0900 2024-04-25T09:58:21.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.128+0000 7fda43fff640 1 -- 172.21.15.148:0/3106621885 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440d20d0 con 0x7fda440d2e90 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.129+0000 7fda43fff640 1 --2- 172.21.15.148:0/3106621885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda440d03c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fda30009790 tx=0x7fda3002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6c3841d6136125 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.129+0000 7fda437fe640 1 -- 172.21.15.148:0/3106621885 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30037070 con 0x7fda440d2e90 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.129+0000 7fda437fe640 1 -- 172.21.15.148:0/3106621885 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3002ae50 con 0x7fda440d2e90 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.129+0000 7fda437fe640 1 -- 172.21.15.148:0/3106621885 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30032660 con 0x7fda440d2e90 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 -- 172.21.15.148:0/3106621885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 msgr2=0x7fda440d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/3106621885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda440d03c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fda30009790 tx=0x7fda3002a100 comp rx=0 tx=0).stop 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 -- 172.21.15.148:0/3106621885 shutdown_connections 2024-04-25T09:58:21.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/3106621885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda440d03c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 -- 172.21.15.148:0/3106621885 >> 172.21.15.148:0/3106621885 conn(0x7fda44061c30 msgr2=0x7fda4406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 -- 172.21.15.148:0/3106621885 shutdown_connections 2024-04-25T09:58:21.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.130+0000 7fda4aaf5640 1 -- 172.21.15.148:0/3106621885 wait complete. 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.131+0000 7fda4aaf5640 1 Processor -- start 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.131+0000 7fda4aaf5640 1 -- start start 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda4aaf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda440d0900 0x7fda441f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda4aaf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda441f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda4aaf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda4415dd30 0x7fda440b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda4aaf5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440d2970 con 0x7fda440d2e90 2024-04-25T09:58:21.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda4aaf5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440d2690 con 0x7fda4415dd30 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda4aaf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440d1a10 con 0x7fda440d0900 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda441f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda441f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33442/0 (socket says 172.21.15.148:33442) 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 -- 172.21.15.148:0/2333778924 learned_addr learned my addr 172.21.15.148:0/2333778924 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 -- 172.21.15.148:0/2333778924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda440d0900 msgr2=0x7fda441f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda440d0900 0x7fda441f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 -- 172.21.15.148:0/2333778924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda4415dd30 msgr2=0x7fda440b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda4415dd30 0x7fda440b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 -- 172.21.15.148:0/2333778924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda441f42a0 con 0x7fda440d2e90 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.132+0000 7fda43fff640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda441f3d60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fda30035040 tx=0x7fda300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.133+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30041070 con 0x7fda440d2e90 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.133+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3003c670 con 0x7fda440d2e90 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.133+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30037040 con 0x7fda440d2e90 2024-04-25T09:58:21.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.133+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440b4ee0 con 0x7fda440d2e90 2024-04-25T09:58:21.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.133+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440b5490 con 0x7fda440d2e90 2024-04-25T09:58:21.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.134+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44073990 con 0x7fda440d2e90 2024-04-25T09:58:21.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.134+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda30044070 con 0x7fda440d2e90 2024-04-25T09:58:21.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.134+0000 7fda417fa640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda2806b500 0x7fda2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.134+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fda300aac00 con 0x7fda440d2e90 2024-04-25T09:58:21.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.134+0000 7fda48891640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda2806b500 0x7fda2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.135+0000 7fda48891640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda2806b500 0x7fda2806d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fda38000f60 tx=0x7fda380079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.138+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda300b1080 con 0x7fda440d2e90 2024-04-25T09:58:21.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.346+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda4406b210 con 0x7fda440d2e90 2024-04-25T09:58:21.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.347+0000 7fda417fa640 1 -- 172.21.15.148:0/2333778924 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fda300798a0 con 0x7fda440d2e90 2024-04-25T09:58:21.350 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:21.351 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:18.742068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:21.351 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2024-04-25T09:58:21.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.348+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda2806b500 msgr2=0x7fda2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.348+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda2806b500 0x7fda2806d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fda38000f60 tx=0x7fda380079b0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.348+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 msgr2=0x7fda441f3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.348+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda441f3d60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fda30035040 tx=0x7fda300046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 shutdown_connections 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda2806b500 0x7fda2806d9c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda4415dd30 0x7fda440b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda440d2e90 0x7fda441f3d60 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 --2- 172.21.15.148:0/2333778924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda440d0900 0x7fda441f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 >> 172.21.15.148:0/2333778924 conn(0x7fda44061c30 msgr2=0x7fda440c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 shutdown_connections 2024-04-25T09:58:21.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.349+0000 7fda4aaf5640 1 -- 172.21.15.148:0/2333778924 wait complete. 2024-04-25T09:58:21.735 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T09:58:21.732+0000 7f29ae978640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:58:21.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:01.965750+0000', 'modified': '2024-04-25T09:58:18.742068+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.148:6835/328449171', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 328449171}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 328449171}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:58:21.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:58:21.766 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:58:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.864+0000 7f80992f5640 1 Processor -- start 2024-04-25T09:58:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.864+0000 7f80992f5640 1 -- start start 2024-04-25T09:58:21.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80992f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f80940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80992f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8094161710 0x7f80940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80992f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809415e000 con 0x7f80940b9330 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80992f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80940d1de0 con 0x7f8094161710 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80992f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8094161c60 con 0x7f809415dc20 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8094161710 0x7f80940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80937fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f80940b9330 0x7f809406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56168/0 (socket says 172.21.15.148:56168) 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80937fe640 1 -- 172.21.15.148:0/3834164870 learned_addr learned my addr 172.21.15.148:0/3834164870 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f8092ffd640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f80940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80927fc640 1 -- 172.21.15.148:0/3834164870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 msgr2=0x7f80940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80927fc640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f80940ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80927fc640 1 -- 172.21.15.148:0/3834164870 >> v1:172.21.15.148:6789/0 conn(0x7f80940b9330 legacy=0x7f809406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f8091ffb640 1 -- 172.21.15.148:0/3834164870 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045835974 0 0) 0x7f809415e000 con 0x7f80940b9330 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f80927fc640 1 -- 172.21.15.148:0/3834164870 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094161f40 con 0x7f8094161710 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.865+0000 7f8092ffd640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f80940ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.866+0000 7f80927fc640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8094161710 0x7f80940cebb0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f80880097a0 tx=0x7f808802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e8e9499cb4c23d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.866+0000 7f8091ffb640 1 -- 172.21.15.148:0/3834164870 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088037070 con 0x7f8094161710 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.866+0000 7f8091ffb640 1 -- 172.21.15.148:0/3834164870 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088032440 con 0x7f8094161710 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.866+0000 7f8091ffb640 1 -- 172.21.15.148:0/3834164870 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088032810 con 0x7f8094161710 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 -- 172.21.15.148:0/3834164870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8094161710 msgr2=0x7f80940cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8094161710 0x7f80940cebb0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f80880097a0 tx=0x7f808802a180 comp rx=0 tx=0).stop 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 -- 172.21.15.148:0/3834164870 shutdown_connections 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8094161710 0x7f80940cebb0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 --2- 172.21.15.148:0/3834164870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f80940ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 -- 172.21.15.148:0/3834164870 >> 172.21.15.148:0/3834164870 conn(0x7f80940d0420 msgr2=0x7f80940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 -- 172.21.15.148:0/3834164870 shutdown_connections 2024-04-25T09:58:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.867+0000 7f80992f5640 1 -- 172.21.15.148:0/3834164870 wait complete. 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.868+0000 7f80992f5640 1 Processor -- start 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.868+0000 7f80992f5640 1 -- start start 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 0x7f8094157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f8094154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8094161710 0x7f809415ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b5e50 con 0x7f80940b9330 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8094162500 con 0x7f8094161710 2024-04-25T09:58:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b5fd0 con 0x7f809415dc20 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 0x7f8094157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 0x7f8094157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33460/0 (socket says 172.21.15.148:33460) 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 -- 172.21.15.148:0/3508063038 learned_addr learned my addr 172.21.15.148:0/3508063038 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80937fe640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8094161710 0x7f809415ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80927fc640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f8094154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 -- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 msgr2=0x7f8094154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f8094154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 -- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8094161710 msgr2=0x7f809415ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8094161710 0x7f809415ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 -- 172.21.15.148:0/3508063038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094154c80 con 0x7f80940b9330 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80937fe640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8094161710 0x7f809415ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80927fc640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f8094154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f8092ffd640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 0x7f8094157390 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f808400c7f0 tx=0x7f808400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f807b7fe640 1 -- 172.21.15.148:0/3508063038 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084015070 con 0x7f80940b9330 2024-04-25T09:58:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f807b7fe640 1 -- 172.21.15.148:0/3508063038 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084007820 con 0x7f80940b9330 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80941562f0 con 0x7f80940b9330 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f807b7fe640 1 -- 172.21.15.148:0/3508063038 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084007bd0 con 0x7f80940b9330 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.869+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941568b0 con 0x7f80940b9330 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.871+0000 7f807b7fe640 1 -- 172.21.15.148:0/3508063038 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8084018070 con 0x7f80940b9330 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.871+0000 7f807b7fe640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f808006b500 0x7f808006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.871+0000 7f807b7fe640 1 -- 172.21.15.148:0/3508063038 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f808408a400 con 0x7f80940b9330 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.871+0000 7f80927fc640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f808006b500 0x7f808006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.872+0000 7f80927fc640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f808006b500 0x7f808006d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8088034000 tx=0x7f808802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.872+0000 7f809bffb740 1 Processor -- start 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.872+0000 7f809bffb740 1 -- start start 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845db7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f809bffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845dd5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845d8cf10 0x562845dd3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f809bffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845cefa10 con 0x562845dd4ad0 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845c98b30 con 0x562845db7230 2024-04-25T09:58:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562845cebc50 con 0x562845d8cf10 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845dd5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845dd5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33470/0 (socket says 172.21.15.148:33470) 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 -- 172.21.15.148:0/2275500172 learned_addr learned my addr 172.21.15.148:0/2275500172 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80937fe640 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845d8cf10 0x562845dd3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845db7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 -- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845d8cf10 msgr2=0x562845dd3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845d8cf10 0x562845dd3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 -- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 msgr2=0x562845db7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845db7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 -- 172.21.15.148:0/2275500172 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562845cade10 con 0x562845dd4ad0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.873+0000 7f80927fc640 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845dd5020 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8088035040 tx=0x7f808802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd161bba38b8fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f8078ff9640 1 -- 172.21.15.148:0/2275500172 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808804d070 con 0x562845dd4ad0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f8078ff9640 1 -- 172.21.15.148:0/2275500172 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088048470 con 0x562845dd4ad0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f8078ff9640 1 -- 172.21.15.148:0/2275500172 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088037040 con 0x562845dd4ad0 2024-04-25T09:58:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 -- 172.21.15.148:0/2275500172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 msgr2=0x562845dd5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845dd5020 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8088035040 tx=0x7f808802c000 comp rx=0 tx=0).stop 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 -- 172.21.15.148:0/2275500172 shutdown_connections 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845d8cf10 0x562845dd3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845dd5020 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 --2- 172.21.15.148:0/2275500172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845db7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 -- 172.21.15.148:0/2275500172 >> 172.21.15.148:0/2275500172 conn(0x562845d89840 msgr2=0x562845dd2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 -- 172.21.15.148:0/2275500172 shutdown_connections 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.874+0000 7f809bffb740 1 -- 172.21.15.148:0/2275500172 wait complete. 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.875+0000 7f809bffb740 1 -- start start 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.875+0000 7f809bffb740 1 Processor -- start 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845dca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845db5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845d8dcb0 con 0x562845dd4ad0 2024-04-25T09:58:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845cac0c0 con 0x562845db7230 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562845d87770 con 0x562845db5d60 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 10 client.0 _ll_register_callbacks cb 0x562845c9fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845db5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845db5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33476/0 (socket says 172.21.15.148:33476) 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 -- 172.21.15.148:0/2330624487 learned_addr learned my addr 172.21.15.148:0/2330624487 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80937fe640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845dca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 -- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845db5d60 msgr2=0x562845dba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 -- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 msgr2=0x562845dca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845dca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 -- 172.21.15.148:0/2330624487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562845a042d0 con 0x562845dd4ad0 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845dca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:21.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f80927fc640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845db5820 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f808804b040 tx=0x7f8088004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65868c9d3312528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f8071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f8071ffb640 1 -- 172.21.15.148:0/2330624487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808804d030 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f8071ffb640 1 -- 172.21.15.148:0/2330624487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8088054ca0 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.876+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8088048470 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f8071ffb640 1 -- 172.21.15.148:0/2330624487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8088037070 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 10 client.7050 fetch_fsmap learned FSMap version 40 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x562845ace6b0 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f8071ffb640 1 -- 172.21.15.148:0/2330624487 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8088048470 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f8071ffb640 1 -- 172.21.15.148:0/2330624487 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8088048830 con 0x562845dd4ad0 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 10 client.7050 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 10 client.7050 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 20 client.7050 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845de3520 0x562845de3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 4 client.7050 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:21.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 --> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562845d4e920 con 0x562845de3520 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845de3520 0x562845de3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845de3520 0x562845de3900 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda97f0aef052019 server_cookie=4a152d92407234ca in_seq=0 out_seq=0 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.877+0000 7f8071ffb640 10 client.7050 ms_handle_connect on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f8071ffb640 1 -- 172.21.15.148:0/2330624487 <== mds.0 v2:172.21.15.148:6834/328449171 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562845d4e920 con 0x562845de3520 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f8071ffb640 10 client.7050 handle_command_reply: tid=0 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 client.7050 shutdown 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 msgr2=0x562845db5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845db5820 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f808804b040 tx=0x7f8088004730 comp rx=0 tx=0).stop 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 shutdown_connections 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845de3520 0x562845de3900 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845dd4ad0 0x562845db5820 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f8071ffb640 0 client.7050 ms_handle_reset on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 --2- 172.21.15.148:0/2330624487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845dca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 >> 172.21.15.148:0/2330624487 conn(0x562845d8cf10 msgr2=0x562845dd55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 shutdown_connections 2024-04-25T09:58:21.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 1 -- 172.21.15.148:0/2330624487 wait complete. 2024-04-25T09:58:21.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.878+0000 7f809bffb740 20 client.7050 trim_cache size 0 max 16384 2024-04-25T09:58:21.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 Processor -- start 2024-04-25T09:58:21.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 -- start start 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845d8c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845dd5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941ff4f0 con 0x562845db5d60 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941f4210 con 0x562845db7230 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.894+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80941ff110 con 0x562845dd4ad0 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33482/0 (socket says 172.21.15.148:33482) 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 -- 172.21.15.148:0/4134770985 learned_addr learned my addr 172.21.15.148:0/4134770985 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 -- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 msgr2=0x562845dd5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f80927fc640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845d8c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f80937fe640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845dd5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845dd5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 -- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 msgr2=0x562845d8c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845d8c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 -- 172.21.15.148:0/4134770985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562845db3fa0 con 0x562845db5d60 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f80937fe640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845dd5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f80927fc640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845d8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8092ffd640 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 0x562845dba410 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f80840589c0 tx=0x7f808408a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0c1fd86dcdc20d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8073fff640 1 -- 172.21.15.148:0/4134770985 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808401f040 con 0x562845db5d60 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8073fff640 1 -- 172.21.15.148:0/4134770985 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084056d30 con 0x562845db5d60 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.895+0000 7f8073fff640 1 -- 172.21.15.148:0/4134770985 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808408a0b0 con 0x562845db5d60 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 -- 172.21.15.148:0/4134770985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 msgr2=0x562845dba410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 0x562845dba410 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f80840589c0 tx=0x7f808408a040 comp rx=0 tx=0).stop 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 -- 172.21.15.148:0/4134770985 shutdown_connections 2024-04-25T09:58:21.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845dd5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db7230 0x562845d8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 --2- 172.21.15.148:0/4134770985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db5d60 0x562845dba410 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 -- 172.21.15.148:0/4134770985 >> 172.21.15.148:0/4134770985 conn(0x562845d8cf10 msgr2=0x562845db65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 -- 172.21.15.148:0/4134770985 shutdown_connections 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 -- 172.21.15.148:0/4134770985 wait complete. 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 -- start start 2024-04-25T09:58:21.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.896+0000 7f809bffb740 1 Processor -- start 2024-04-25T09:58:21.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db5d60 0x562845e33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 0x562845db4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845db2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845de3ff0 con 0x562845db7230 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562845d8dcb0 con 0x562845db5d60 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562845dbae20 con 0x562845dd4ad0 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f809bffb740 10 client.0 _ll_register_callbacks cb 0x562845d87c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 0x562845db4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 0x562845db4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33494/0 (socket says 172.21.15.148:33494) 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 -- 172.21.15.148:0/2124978162 learned_addr learned my addr 172.21.15.148:0/2124978162 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db5d60 0x562845e33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80937fe640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845db2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 -- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 msgr2=0x562845db2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845db2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 -- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db5d60 msgr2=0x562845e33740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db5d60 0x562845e33740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 -- 172.21.15.148:0/2124978162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562845dd5ba0 con 0x562845db7230 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f80927fc640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 0x562845db4a90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f808802a1c0 tx=0x7f8088008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99bd47a84c97a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:21.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.897+0000 7f8071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8071ffb640 1 -- 172.21.15.148:0/2124978162 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8088061020 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8071ffb640 1 -- 172.21.15.148:0/2124978162 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f80880558c0 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8084056d30 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8071ffb640 1 -- 172.21.15.148:0/2124978162 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f808804d040 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 10 client.7056 fetch_fsmap learned FSMap version 40 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x562845db4560 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8071ffb640 1 -- 172.21.15.148:0/2124978162 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084056d30 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8071ffb640 1 -- 172.21.15.148:0/2124978162 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8088055ac0 con 0x562845db7230 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 10 client.7056 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 10 client.7056 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 20 client.7056 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845db27d0 0x562845d864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 4 client.7056 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 --> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562845dcbb10 con 0x562845db27d0 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845db27d0 0x562845d864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8071ffb640 10 client.7056 ms_handle_connect on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.898+0000 7f8092ffd640 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845db27d0 0x562845d864a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7f2031203c24dc server_cookie=d35942edc5108c14 in_seq=0 out_seq=0 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f8071ffb640 1 -- 172.21.15.148:0/2124978162 <== mds.0 v2:172.21.15.148:6834/328449171 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562845dcbb10 con 0x562845db27d0 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f8071ffb640 10 client.7056 handle_command_reply: tid=0 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 client.7056 shutdown 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 msgr2=0x562845db4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 0x562845db4a90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f808802a1c0 tx=0x7f8088008030 comp rx=0 tx=0).stop 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 shutdown_connections 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x562845db27d0 0x562845d864a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562845dd4ad0 0x562845db2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f8071ffb640 0 client.7056 ms_handle_reset on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562845db7230 0x562845db4a90 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 --2- 172.21.15.148:0/2124978162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562845db5d60 0x562845e33740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 >> 172.21.15.148:0/2124978162 conn(0x562845d8cf10 msgr2=0x562845db65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 shutdown_connections 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 1 -- 172.21.15.148:0/2124978162 wait complete. 2024-04-25T09:58:21.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f809bffb740 20 client.7056 trim_cache size 0 max 16384 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 6115, 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 5.167875703, 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 40, 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 71, 2024-04-25T09:58:21.909 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 71, 2024-04-25T09:58:21.910 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 6.0782779600000003 2024-04-25T09:58:21.910 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:58:21.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f808006b500 msgr2=0x7f808006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f808006b500 0x7f808006d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8088034000 tx=0x7f808802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 msgr2=0x7f8094157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 0x7f8094157390 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f808400c7f0 tx=0x7f808400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 shutdown_connections 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f808006b500 0x7f808006d9c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8094161710 0x7f809415ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f809415dc20 0x7f8094154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 --2- 172.21.15.148:0/3508063038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b9330 0x7f8094157390 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 >> 172.21.15.148:0/3508063038 conn(0x7f80940d0420 msgr2=0x7f80940d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 shutdown_connections 2024-04-25T09:58:21.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:21.899+0000 7f80992f5640 1 -- 172.21.15.148:0/3508063038 wait complete. 2024-04-25T09:58:21.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:58:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.008+0000 7ff1aa3d5640 1 Processor -- start 2024-04-25T09:58:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1aa3d5640 1 -- start start 2024-04-25T09:58:22.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1aa3d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff1a406c1d0 0x7ff1a4168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1aa3d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a40adcf0 con 0x7ff1a4154ed0 2024-04-25T09:58:22.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1aa3d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a415ec60 con 0x7ff1a4071ac0 2024-04-25T09:58:22.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1aa3d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a415ef40 con 0x7ff1a406c1d0 2024-04-25T09:58:22.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1a3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff1a4154ed0 0x7ff1a40aced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56174/0 (socket says 172.21.15.148:56174) 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.009+0000 7ff1a3fff640 1 -- 172.21.15.148:0/3211747597 learned_addr learned my addr 172.21.15.148:0/3211747597 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3211747597 >> v2:172.21.15.188:3300/0 conn(0x7ff1a406c1d0 0x7ff1a4168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428532721 0 0) 0x7ff1a40adcf0 con 0x7ff1a4154ed0 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff18c003520 con 0x7ff1a4154ed0 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1192366216 0 0) 0x7ff18c003520 con 0x7ff1a4154ed0 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 >> v2:172.21.15.188:3300/0 conn(0x7ff1a406c1d0 msgr2=0x7ff1a4168930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 --2- 172.21.15.148:0/3211747597 >> v2:172.21.15.188:3300/0 conn(0x7ff1a406c1d0 0x7ff1a4168930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 >> v1:172.21.15.188:6789/0 conn(0x7ff1a4071ac0 legacy=0x7ff1a40ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a406db90 con 0x7ff1a4154ed0 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff194001670 con 0x7ff1a4154ed0 2024-04-25T09:58:22.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 >> v1:172.21.15.148:6789/0 conn(0x7ff1a4154ed0 legacy=0x7ff1a40aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff18c0039b0 0x7ff18c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff18c004800 0x7ff18c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0051c0 0x7ff18c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a415ef40 con 0x7ff18c0051c0 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c00c160 con 0x7ff18c004800 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c00c360 con 0x7ff18c0039b0 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0051c0 0x7ff18c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a3fff640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff18c0039b0 0x7ff18c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff18c004800 0x7ff18c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 -- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff18c0039b0 msgr2=0x7ff18c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff18c0039b0 0x7ff18c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 -- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff18c004800 msgr2=0x7ff18c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:22.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff18c004800 0x7ff18c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a8972640 1 -- 172.21.15.148:0/3211747597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c003520 con 0x7ff18c0051c0 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.010+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff18c004800 0x7ff18c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.011+0000 7ff1a8972640 1 -- 172.21.15.148:0/3211747597 reap_dead start 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.011+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0051c0 0x7ff18c00b9c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009fc0 tx=0x7ff18c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bacf5c8c1cfe99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.011+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c035020 con 0x7ff18c0051c0 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.011+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c003bd0 con 0x7ff18c0051c0 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.011+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3211747597 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c009c10 con 0x7ff18c0051c0 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3211747597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0051c0 msgr2=0x7ff18c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0051c0 0x7ff18c00b9c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009fc0 tx=0x7ff18c005830 comp rx=0 tx=0).stop 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3211747597 shutdown_connections 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3211747597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff18c0051c0 0x7ff18c00b9c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3211747597 >> 172.21.15.148:0/3211747597 conn(0x7ff1a40af670 msgr2=0x7ff1a40b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3211747597 shutdown_connections 2024-04-25T09:58:22.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.012+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3211747597 wait complete. 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.013+0000 7ff1aa3d5640 1 Processor -- start 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.013+0000 7ff1aa3d5640 1 -- start start 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1aa3d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a414ed40 0x7ff1a40b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1aa3d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 0x7ff1a41f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1aa3d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a41f3e90 0x7ff1a4150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1aa3d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40ad050 con 0x7ff1a40b1470 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1aa3d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40aced0 con 0x7ff1a414ed40 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1aa3d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40ad1d0 con 0x7ff1a41f3e90 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 0x7ff1a41f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 0x7ff1a41f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33514/0 (socket says 172.21.15.148:33514) 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3330900641 learned_addr learned my addr 172.21.15.148:0/3330900641 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a3fff640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a414ed40 0x7ff1a40b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a41f3e90 0x7ff1a4150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a41f3e90 msgr2=0x7ff1a4150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a41f3e90 0x7ff1a4150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a414ed40 msgr2=0x7ff1a40b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a414ed40 0x7ff1a40b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3330900641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a41525a0 con 0x7ff1a40b1470 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a8972640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a41f3e90 0x7ff1a4150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.014+0000 7ff1a3fff640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a414ed40 0x7ff1a40b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.015+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 0x7ff1a41f3950 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff198007810 tx=0x7ff19800e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.015+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198019070 con 0x7ff1a40b1470 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.015+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980052e0 con 0x7ff1a40b1470 2024-04-25T09:58:22.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.015+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a4152880 con 0x7ff1a40b1470 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.015+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198005490 con 0x7ff1a40b1470 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.015+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a4152e40 con 0x7ff1a40b1470 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.016+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a40b2b10 con 0x7ff1a40b1470 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.016+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff19801c070 con 0x7ff1a40b1470 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.016+0000 7ff1a0ff9640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff19006b500 0x7ff19006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.017+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff1980898b0 con 0x7ff1a40b1470 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.017+0000 7ff1a3fff640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff19006b500 0x7ff19006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.017+0000 7ff1a3fff640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff19006b500 0x7ff19006d9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff194001050 tx=0x7ff194009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:22.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.021+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff198057f00 con 0x7ff1a40b1470 2024-04-25T09:58:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.229+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1a4073990 con 0x7ff1a40b1470 2024-04-25T09:58:22.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.230+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/3330900641 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff19805bfa0 con 0x7ff1a40b1470 2024-04-25T09:58:22.233 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:58:22.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.231+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff19006b500 msgr2=0x7ff19006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.231+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff19006b500 0x7ff19006d9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff194001050 tx=0x7ff194009040 comp rx=0 tx=0).stop 2024-04-25T09:58:22.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.231+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 msgr2=0x7ff1a41f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.231+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 0x7ff1a41f3950 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff198007810 tx=0x7ff19800e910 comp rx=0 tx=0).stop 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 shutdown_connections 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff19006b500 0x7ff19006d9c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a41f3e90 0x7ff1a4150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b1470 0x7ff1a41f3950 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 --2- 172.21.15.148:0/3330900641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a414ed40 0x7ff1a40b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 >> 172.21.15.148:0/3330900641 conn(0x7ff18c0051c0 msgr2=0x7ff1a415fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 shutdown_connections 2024-04-25T09:58:22.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:22.232+0000 7ff1aa3d5640 1 -- 172.21.15.148:0/3330900641 wait complete. 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:58:22.246 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:58:22.247 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:58:22.247 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:22.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.247 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:58:22.262 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:58:22.262 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:58:22.262 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:58:22.262 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.262 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:58:22.262 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:58:22.263 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T09:58:22.264 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:22.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.265 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:58:22.324 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:58:22.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.326 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:58:22.382 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:58:22.382 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:58:22.382 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:58:22.382 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:58:22.382 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:58:22.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.383 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:58:22.383 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:58:22.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:58:22.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:58:22.383 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:58:22.383 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:58:22.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.467 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:58:22.482 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:58:22.483 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:58:22.483 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:22.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.483 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:58:22.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:22.547 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:22.602 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:22.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:22.602 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:22.656 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:58:22.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:22.737 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:22.738 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:22.738 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:22.746 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:22.746 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:22.746 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:22.746 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:58:22.746 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:58:22.754 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:22.754 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:22.754 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:22.754 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:58:22.755 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:58:22.756 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:58:22.756 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:58:22.756 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:58:22.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.757 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:22.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.819 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:58:22.875 INFO:teuthology.orchestra.run.smithi148.stdout:client5654 2024-04-25T09:58:22.875 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:22.875 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:22.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:22.931 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:58:22.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:58:22.987 INFO:teuthology.orchestra.run.smithi148.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T09:58:22.987 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T09:58:22.988 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:58:22.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T09:58:23.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.125+0000 7f2c35f1d640 1 Processor -- start 2024-04-25T09:58:23.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.125+0000 7f2c35f1d640 1 -- start start 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c35f1d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2c30155f80 0x7f2c3006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c35f1d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c300ce900 0x7f2c300b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c35f1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300b4d80 con 0x7f2c3006fa00 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c35f1d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300ced30 con 0x7f2c30155f80 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c35f1d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300b5390 con 0x7f2c300ce900 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2c30155f80 0x7f2c3006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2c3006fa00 0x7f2c300ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56190/0 (socket says 172.21.15.148:56190) 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2effd640 1 -- 172.21.15.148:0/3040955782 learned_addr learned my addr 172.21.15.148:0/3040955782 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/3040955782 >> v2:172.21.15.188:3300/0 conn(0x7f2c300ce900 msgr2=0x7f2c300b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/3040955782 >> v2:172.21.15.188:3300/0 conn(0x7f2c300ce900 0x7f2c300b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/3040955782 >> v1:172.21.15.148:6789/0 conn(0x7f2c3006fa00 legacy=0x7f2c300ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/3040955782 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300cef70 con 0x7f2c30155f80 2024-04-25T09:58:23.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.126+0000 7f2c2e7fc640 1 -- 172.21.15.148:0/3040955782 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228598400 0 0) 0x7f2c300b4d80 con 0x7f2c3006fa00 2024-04-25T09:58:23.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.127+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/3040955782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c30155f80 0x7f2c3006f4c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2c300d2860 tx=0x7f2c2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe0e1468a4a3ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.127+0000 7f2c2e7fc640 1 -- 172.21.15.148:0/3040955782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20037070 con 0x7f2c30155f80 2024-04-25T09:58:23.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.127+0000 7f2c2e7fc640 1 -- 172.21.15.148:0/3040955782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20032440 con 0x7f2c30155f80 2024-04-25T09:58:23.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.127+0000 7f2c2e7fc640 1 -- 172.21.15.148:0/3040955782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20032810 con 0x7f2c30155f80 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 -- 172.21.15.148:0/3040955782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c30155f80 msgr2=0x7f2c3006f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/3040955782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c30155f80 0x7f2c3006f4c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2c300d2860 tx=0x7f2c2002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 -- 172.21.15.148:0/3040955782 shutdown_connections 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/3040955782 >> v2:172.21.15.188:3300/0 conn(0x7f2c300ce900 0x7f2c300b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/3040955782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c30155f80 0x7f2c3006f4c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 -- 172.21.15.148:0/3040955782 >> 172.21.15.148:0/3040955782 conn(0x7f2c30061600 msgr2=0x7f2c300619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 -- 172.21.15.148:0/3040955782 shutdown_connections 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.128+0000 7f2c35f1d640 1 -- 172.21.15.148:0/3040955782 wait complete. 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.129+0000 7f2c35f1d640 1 Processor -- start 2024-04-25T09:58:23.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.129+0000 7f2c35f1d640 1 -- start start 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c35f1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 0x7f2c30149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c35f1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c300ce900 0x7f2c3014ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c35f1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c30155f80 0x7f2c3015a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c35f1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30156360 con 0x7f2c3006fa00 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c35f1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300d07f0 con 0x7f2c30155f80 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c35f1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300cf250 con 0x7f2c300ce900 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 0x7f2c30149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 0x7f2c30149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33538/0 (socket says 172.21.15.148:33538) 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/798209708 learned_addr learned my addr 172.21.15.148:0/798209708 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/798209708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c300ce900 msgr2=0x7f2c3014ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c300ce900 0x7f2c3014ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/798209708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c30155f80 msgr2=0x7f2c3015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c30155f80 0x7f2c3015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/798209708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3006dd30 con 0x7f2c3006fa00 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 0x7f2c30149c00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.130+0000 7f2c17fff640 1 -- 172.21.15.148:0/798209708 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20041070 con 0x7f2c3006fa00 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.131+0000 7f2c17fff640 1 -- 172.21.15.148:0/798209708 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2003c950 con 0x7f2c3006fa00 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.131+0000 7f2c17fff640 1 -- 172.21.15.148:0/798209708 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20037040 con 0x7f2c3006fa00 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.131+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3014a370 con 0x7f2c3006fa00 2024-04-25T09:58:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.131+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3014dbb0 con 0x7f2c3006fa00 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.132+0000 7f2c17fff640 1 -- 172.21.15.148:0/798209708 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c20044070 con 0x7f2c3006fa00 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.132+0000 7f2c17fff640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.132+0000 7f2c17fff640 1 -- 172.21.15.148:0/798209708 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2c200aaf40 con 0x7f2c3006fa00 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.132+0000 7f2c2effd640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.133+0000 7f2c2effd640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007fd0 tx=0x7f2c240077c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.133+0000 7f2c38c1b740 1 Processor -- start 2024-04-25T09:58:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.133+0000 7f2c38c1b740 1 -- start start 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34080f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a34083f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083f700 0x55a340814510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a340777f40 con 0x55a34085cd20 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3407170d0 con 0x55a34083f700 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a340773fe0 con 0x55a34083fe40 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34080f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34080f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33552/0 (socket says 172.21.15.148:33552) 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/1216340171 learned_addr learned my addr 172.21.15.148:0/1216340171 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/1216340171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 msgr2=0x55a34083f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a34083f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/1216340171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083f700 msgr2=0x55a340814510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.134+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083f700 0x55a340814510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c2f7fe640 1 -- 172.21.15.148:0/1216340171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a340558af0 con 0x55a34085cd20 2024-04-25T09:58:23.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34080f6b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2c20081870 tx=0x7f2c200790b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61da003ad723ad06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c157fa640 1 -- 172.21.15.148:0/1216340171 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2004b040 con 0x55a34085cd20 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c157fa640 1 -- 172.21.15.148:0/1216340171 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2007d7e0 con 0x55a34085cd20 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c157fa640 1 -- 172.21.15.148:0/1216340171 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20081da0 con 0x55a34085cd20 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c38c1b740 1 -- 172.21.15.148:0/1216340171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 msgr2=0x55a34080f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.135+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34080f6b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2c20081870 tx=0x7f2c200790b0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 -- 172.21.15.148:0/1216340171 shutdown_connections 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083f700 0x55a340814510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a34083f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/1216340171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34080f6b0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 -- 172.21.15.148:0/1216340171 >> 172.21.15.148:0/1216340171 conn(0x55a340811920 msgr2=0x55a34085a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 -- 172.21.15.148:0/1216340171 shutdown_connections 2024-04-25T09:58:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 -- 172.21.15.148:0/1216340171 wait complete. 2024-04-25T09:58:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 -- start start 2024-04-25T09:58:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.136+0000 7f2c38c1b740 1 Processor -- start 2024-04-25T09:58:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a3408525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a34083aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a3408409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a340777f40 con 0x55a340840560 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a340714ba0 con 0x55a34085cd20 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a340813ad0 con 0x55a34083fe40 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c38c1b740 10 client.0 _ll_register_callbacks cb 0x55a340714570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a3408409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a3408409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33556/0 (socket says 172.21.15.148:33556) 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 -- 172.21.15.148:0/491220350 learned_addr learned my addr 172.21.15.148:0/491220350 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a3408525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2effd640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a34083aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 -- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 msgr2=0x55a3408525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a3408525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 -- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 msgr2=0x55a34083aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a34083aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.137+0000 7f2c2ffff640 1 -- 172.21.15.148:0/491220350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a34044c3e0 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c2ffff640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a3408409e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b820 tx=0x7f2c2800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11e749a287c3b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/491220350 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28019060 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/491220350 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2c2800c8e0 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c2007d7e0 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/491220350 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2801f050 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 10 client.7077 fetch_fsmap learned FSMap version 40 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55a3405b9280 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/491220350 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2007d7e0 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/491220350 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c28022020 con 0x55a340840560 2024-04-25T09:58:23.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 10 client.7077 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 10 client.7077 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 20 client.7077 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a340841de0 0x55a34086dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 4 client.7077 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 --> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a340785480 con 0x55a340841de0 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.138+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a340841de0 0x55a34086dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a340841de0 0x55a34086dc80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74cf14fdbbd80ee server_cookie=4edf0954ee408912 in_seq=0 out_seq=0 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c0e7fc640 10 client.7077 ms_handle_connect on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/491220350 <== mds.0 v2:172.21.15.148:6834/328449171 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a340785480 con 0x55a340841de0 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c0e7fc640 10 client.7077 handle_command_reply: tid=0 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 client.7077 shutdown 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 msgr2=0x55a3408409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a3408409e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800b820 tx=0x7f2c2800bce0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 shutdown_connections 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a340841de0 0x55a34086dc80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a3408409e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c0e7fc640 0 client.7077 ms_handle_reset on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a34083aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/491220350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a3408525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 >> 172.21.15.148:0/491220350 conn(0x55a34083f700 msgr2=0x55a3408130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 shutdown_connections 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 1 -- 172.21.15.148:0/491220350 wait complete. 2024-04-25T09:58:23.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.139+0000 7f2c38c1b740 20 client.7077 trim_cache size 0 max 16384 2024-04-25T09:58:23.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.154+0000 7f2c38c1b740 1 Processor -- start 2024-04-25T09:58:23.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.154+0000 7f2c38c1b740 1 -- start start 2024-04-25T09:58:23.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a34086cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a340851230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a340859fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30200520 con 0x55a340840560 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301ff690 con 0x55a34085cd20 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3006d170 con 0x55a34083fe40 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c2effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a340851230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c2effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a340851230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33562/0 (socket says 172.21.15.148:33562) 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.155+0000 7f2c2effd640 1 -- 172.21.15.148:0/160850108 learned_addr learned my addr 172.21.15.148:0/160850108 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c2effd640 1 -- 172.21.15.148:0/160850108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 msgr2=0x55a34086cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c2effd640 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a34086cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c2effd640 1 -- 172.21.15.148:0/160850108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 msgr2=0x55a340859fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c2effd640 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a340859fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c2effd640 1 -- 172.21.15.148:0/160850108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a340425560 con 0x55a340840560 2024-04-25T09:58:23.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c2effd640 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a340851230 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007670 tx=0x7f2c24001050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c193f4c9ab0b35cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c14ff9640 1 -- 172.21.15.148:0/160850108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24015910 con 0x55a340840560 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c14ff9640 1 -- 172.21.15.148:0/160850108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24015ab0 con 0x55a340840560 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c14ff9640 1 -- 172.21.15.148:0/160850108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24015e60 con 0x55a340840560 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c38c1b740 1 -- 172.21.15.148:0/160850108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 msgr2=0x55a340851230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.156+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a340851230 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007670 tx=0x7f2c24001050 comp rx=0 tx=0).stop 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 -- 172.21.15.148:0/160850108 shutdown_connections 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34085cd20 0x55a340859fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a340840560 0x55a340851230 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/160850108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a34083fe40 0x55a34086cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 -- 172.21.15.148:0/160850108 >> 172.21.15.148:0/160850108 conn(0x55a34083f700 msgr2=0x55a34080e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 -- 172.21.15.148:0/160850108 shutdown_connections 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 -- 172.21.15.148:0/160850108 wait complete. 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 -- start start 2024-04-25T09:58:23.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.157+0000 7f2c38c1b740 1 Processor -- start 2024-04-25T09:58:23.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083fe40 0x55a340873f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a340840560 0x55a34086ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34086cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a340852970 con 0x55a34085cd20 2024-04-25T09:58:23.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300b1d90 con 0x55a34083fe40 2024-04-25T09:58:23.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a340867760 con 0x55a340840560 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c38c1b740 10 client.0 _ll_register_callbacks cb 0x7f2c280094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34086cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34086cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33578/0 (socket says 172.21.15.148:33578) 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 -- 172.21.15.148:0/308897966 learned_addr learned my addr 172.21.15.148:0/308897966 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2effd640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a340840560 0x55a34086ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083fe40 0x55a340873f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 -- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a340840560 msgr2=0x55a34086ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a340840560 0x55a34086ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 -- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083fe40 msgr2=0x55a340873f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083fe40 0x55a340873f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 -- 172.21.15.148:0/308897966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3408749f0 con 0x55a34085cd20 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083fe40 0x55a340873f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.158+0000 7f2c2ffff640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34086cf80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800bd20 tx=0x7f2c28004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed21a278c20866e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:23.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/308897966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28027070 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/308897966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2c2800b440 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c24015ab0 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/308897966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c28032050 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 10 client.7083 fetch_fsmap learned FSMap version 40 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55a34085c720 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/308897966 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24015ab0 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/308897966 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c28035020 con 0x55a34085cd20 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 10 client.7083 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 10 client.7083 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 20 client.7083 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a3408c61c0 0x55a3408668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 4 client.7083 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 --> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55a34083c780 con 0x55a3408c61c0 2024-04-25T09:58:23.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a3408c61c0 0x55a3408668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c0e7fc640 10 client.7083 ms_handle_connect on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.159+0000 7f2c2f7fe640 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a3408c61c0 0x55a3408668d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b51a1ce0b8714d server_cookie=7e1ed31dbaf32a49 in_seq=0 out_seq=0 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c0e7fc640 1 -- 172.21.15.148:0/308897966 <== mds.0 v2:172.21.15.148:6834/328449171 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a34083c780 con 0x55a3408c61c0 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c0e7fc640 10 client.7083 handle_command_reply: tid=0 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 client.7083 shutdown 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 msgr2=0x55a34086cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34086cf80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800bd20 tx=0x7f2c28004b50 comp rx=0 tx=0).stop 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 shutdown_connections 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:6834/328449171,v1:172.21.15.148:6835/328449171] conn(0x55a3408c61c0 0x55a3408668d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a34085cd20 0x55a34086cf80 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a340840560 0x55a34086ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c0e7fc640 0 client.7083 ms_handle_reset on v2:172.21.15.148:6834/328449171 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 --2- 172.21.15.148:0/308897966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a34083fe40 0x55a340873f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 >> 172.21.15.148:0/308897966 conn(0x55a34083f700 msgr2=0x55a340815410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 shutdown_connections 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 1 -- 172.21.15.148:0/308897966 wait complete. 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.160+0000 7f2c38c1b740 20 client.7083 trim_cache size 0 max 16384 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "2b521cf0-654e-4ae0-9768-3e7e48186ee8", 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:58:23.170 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:58:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c1c06b500 msgr2=0x7f2c1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007fd0 tx=0x7f2c240077c0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 msgr2=0x7f2c30149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:23.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 0x7f2c30149c00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c200046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 shutdown_connections 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c1c06b500 0x7f2c1c06d9c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c30155f80 0x7f2c3015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c300ce900 0x7f2c3014ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 --2- 172.21.15.148:0/798209708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c3006fa00 0x7f2c30149c00 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 >> 172.21.15.148:0/798209708 conn(0x7f2c30061600 msgr2=0x7f2c30154640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 shutdown_connections 2024-04-25T09:58:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:23.161+0000 7f2c35f1d640 1 -- 172.21.15.148:0/798209708 wait complete. 2024-04-25T09:58:23.175 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '2b521cf0-654e-4ae0-9768-3e7e48186ee8', 'mode': 'asynchronous'}' 2024-04-25T09:58:24.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:24.023 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:33.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:33.176 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T09:58:33.197 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:58:33.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:33.288 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:33.331 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:58:33.331 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:33.331 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:33.349 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:33.349 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:33.407 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:33.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.549+0000 7fb9931e7640 1 Processor -- start 2024-04-25T09:58:33.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.549+0000 7fb9931e7640 1 -- start start 2024-04-25T09:58:33.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb9931e7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb98c06c1d0 0x7fb98c168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb9931e7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c0adcf0 con 0x7fb98c071ac0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb9931e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c15ec60 con 0x7fb98c154ed0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb9931e7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c15ef40 con 0x7fb98c06c1d0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb991784640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb98c06c1d0 0x7fb98c168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb98bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb98c071ac0 0x7fb98c0ad5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54156/0 (socket says 172.21.15.148:54156) 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb98bfff640 1 -- 172.21.15.148:0/672137169 learned_addr learned my addr 172.21.15.148:0/672137169 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb991784640 1 -- 172.21.15.148:0/672137169 >> v1:172.21.15.188:6789/0 conn(0x7fb98c154ed0 legacy=0x7fb98c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb991784640 1 -- 172.21.15.148:0/672137169 >> v1:172.21.15.148:6789/0 conn(0x7fb98c071ac0 legacy=0x7fb98c0ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb98b7fe640 1 -- 172.21.15.148:0/672137169 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118611603 0 0) 0x7fb98c0adcf0 con 0x7fb98c071ac0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.550+0000 7fb991784640 1 -- 172.21.15.148:0/672137169 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c06db90 con 0x7fb98c06c1d0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.551+0000 7fb991784640 1 --2- 172.21.15.148:0/672137169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c06c1d0 0x7fb98c168930 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb984009d90 tx=0x7fb98402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10b35e6c9d53a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.551+0000 7fb98b7fe640 1 -- 172.21.15.148:0/672137169 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984007740 con 0x7fb98c06c1d0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.551+0000 7fb98b7fe640 1 -- 172.21.15.148:0/672137169 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9840078e0 con 0x7fb98c06c1d0 2024-04-25T09:58:33.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.551+0000 7fb98b7fe640 1 -- 172.21.15.148:0/672137169 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984007cb0 con 0x7fb98c06c1d0 2024-04-25T09:58:33.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 -- 172.21.15.148:0/672137169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c06c1d0 msgr2=0x7fb98c168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 --2- 172.21.15.148:0/672137169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c06c1d0 0x7fb98c168930 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb984009d90 tx=0x7fb98402a660 comp rx=0 tx=0).stop 2024-04-25T09:58:33.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 -- 172.21.15.148:0/672137169 shutdown_connections 2024-04-25T09:58:33.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 --2- 172.21.15.148:0/672137169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c06c1d0 0x7fb98c168930 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 -- 172.21.15.148:0/672137169 >> 172.21.15.148:0/672137169 conn(0x7fb98c0af670 msgr2=0x7fb98c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:33.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 -- 172.21.15.148:0/672137169 shutdown_connections 2024-04-25T09:58:33.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.552+0000 7fb9931e7640 1 -- 172.21.15.148:0/672137169 wait complete. 2024-04-25T09:58:33.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.553+0000 7fb9931e7640 1 Processor -- start 2024-04-25T09:58:33.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.553+0000 7fb9931e7640 1 -- start start 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb9931e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c071ac0 0x7fb98c144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb9931e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb98c154ed0 0x7fb98c0ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb9931e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 0x7fb98c14c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb9931e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c168ff0 con 0x7fb98c0cd510 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb9931e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c168e70 con 0x7fb98c071ac0 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb9931e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c169170 con 0x7fb98c154ed0 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 0x7fb98c14c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 0x7fb98c14c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48502/0 (socket says 172.21.15.148:48502) 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 -- 172.21.15.148:0/2925933449 learned_addr learned my addr 172.21.15.148:0/2925933449 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb990f83640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c071ac0 0x7fb98c144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb98bfff640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb98c154ed0 0x7fb98c0ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 -- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb98c154ed0 msgr2=0x7fb98c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb98c154ed0 0x7fb98c0ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 -- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c071ac0 msgr2=0x7fb98c144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c071ac0 0x7fb98c144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 -- 172.21.15.148:0/2925933449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c155fc0 con 0x7fb98c0cd510 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb98bfff640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb98c154ed0 0x7fb98c0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb990f83640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c071ac0 0x7fb98c144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.554+0000 7fb991784640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 0x7fb98c14c150 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb984009e80 tx=0x7fb984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.555+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984040070 con 0x7fb98c0cd510 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.555+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984031b70 con 0x7fb98c0cd510 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.555+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984045460 con 0x7fb98c0cd510 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.555+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c1562a0 con 0x7fb98c0cd510 2024-04-25T09:58:33.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.555+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c156ac0 con 0x7fb98c0cd510 2024-04-25T09:58:33.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.556+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c073990 con 0x7fb98c0cd510 2024-04-25T09:58:33.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.556+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb984039040 con 0x7fb98c0cd510 2024-04-25T09:58:33.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.556+0000 7fb9897fa640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb97006b4d0 0x7fb97006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.557+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb9840abd30 con 0x7fb98c0cd510 2024-04-25T09:58:33.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.557+0000 7fb990f83640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb97006b4d0 0x7fb97006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.557+0000 7fb990f83640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb97006b4d0 0x7fb97006d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb98c15d9c0 tx=0x7fb980009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:33.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.561+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb98403ae60 con 0x7fb98c0cd510 2024-04-25T09:58:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.733+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb98c160f30 con 0x7fb98c0cd510 2024-04-25T09:58:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.735+0000 7fb9897fa640 1 -- 172.21.15.148:0/2925933449 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17063 (secure 0 0 0) 0x7fb98407a9d0 con 0x7fb98c0cd510 2024-04-25T09:58:33.739 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:33.739 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:15.718258+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/2397025400":"2024-04-26T09:58:15.718240+0000","172.21.15.148:6834/2397025400":"2024-04-26T09:58:15.718240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb97006b4d0 msgr2=0x7fb97006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb97006b4d0 0x7fb97006d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb98c15d9c0 tx=0x7fb980009040 comp rx=0 tx=0).stop 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 msgr2=0x7fb98c14c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 0x7fb98c14c150 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb984009e80 tx=0x7fb984004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 shutdown_connections 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb97006b4d0 0x7fb97006d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb98c0cd510 0x7fb98c14c150 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb98c154ed0 0x7fb98c0ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 --2- 172.21.15.148:0/2925933449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb98c071ac0 0x7fb98c144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 >> 172.21.15.148:0/2925933449 conn(0x7fb98c06c1d0 msgr2=0x7fb98c15f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 shutdown_connections 2024-04-25T09:58:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.737+0000 7fb9931e7640 1 -- 172.21.15.148:0/2925933449 wait complete. 2024-04-25T09:58:33.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:00.177731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:00.933836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:33.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.850+0000 7f6371b7e640 1 Processor -- start 2024-04-25T09:58:33.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.850+0000 7f6371b7e640 1 -- start start 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f6371b7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 0x7f636c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f6371b7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f636c161710 0x7f636c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f6371b7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636c15e000 con 0x7f636c0b9330 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f6371b7e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636c0d1de0 con 0x7f636c161710 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f6371b7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c161c60 con 0x7f636c15dc20 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f636bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f636c0b9330 0x7f636c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54168/0 (socket says 172.21.15.148:54168) 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.851+0000 7f636bfff640 1 -- 172.21.15.148:0/1665552976 learned_addr learned my addr 172.21.15.148:0/1665552976 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f6362ffd640 1 --2- 172.21.15.148:0/1665552976 >> v2:172.21.15.188:3300/0 conn(0x7f636c161710 0x7f636c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 0x7f636c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67561587 0 0) 0x7f636c15e000 con 0x7f636c0b9330 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6358003520 con 0x7f636c0b9330 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900506972 0 0) 0x7f6358003520 con 0x7f636c0b9330 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 msgr2=0x7f636c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 0x7f636c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 >> v2:172.21.15.188:3300/0 conn(0x7f636c161710 msgr2=0x7f636c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 --2- 172.21.15.148:0/1665552976 >> v2:172.21.15.188:3300/0 conn(0x7f636c161710 0x7f636c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c161f40 con 0x7f636c0b9330 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f63640019f0 con 0x7f636c0b9330 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 >> v1:172.21.15.148:6789/0 conn(0x7f636c0b9330 legacy=0x7f636c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f6358003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 0x7f6358004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6358004d40 0x7f63580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0d1de0 con 0x7f63580039b0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c161c60 con 0x7f6358004d40 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635800b9c0 con 0x7f636c15dc20 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f6358003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636bfff640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6358004d40 0x7f63580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f6362ffd640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 0x7f6358004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 -- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 msgr2=0x7f6358004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c15dc20 0x7f6358004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 -- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6358004d40 msgr2=0x7f63580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6358004d40 0x7f63580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636b7fe640 1 -- 172.21.15.148:0/1665552976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358003520 con 0x7f63580039b0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.852+0000 7f636bfff640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6358004d40 0x7f63580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.853+0000 7f636b7fe640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f6358003dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f635c009ac0 tx=0x7f6358005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d20cb7cd3cebde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.853+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c036070 con 0x7f63580039b0 2024-04-25T09:58:33.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.853+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c031400 con 0x7f63580039b0 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.853+0000 7f636affd640 1 -- 172.21.15.148:0/1665552976 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c009690 con 0x7f63580039b0 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f6371b7e640 1 -- 172.21.15.148:0/1665552976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 msgr2=0x7f6358003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f6371b7e640 1 --2- 172.21.15.148:0/1665552976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f6358003dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f635c009ac0 tx=0x7f6358005830 comp rx=0 tx=0).stop 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f636bfff640 1 -- 172.21.15.148:0/1665552976 reap_dead start 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f6371b7e640 1 -- 172.21.15.148:0/1665552976 shutdown_connections 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f6371b7e640 1 -- 172.21.15.148:0/1665552976 >> 172.21.15.148:0/1665552976 conn(0x7f636c0d0420 msgr2=0x7f636c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f6371b7e640 1 -- 172.21.15.148:0/1665552976 shutdown_connections 2024-04-25T09:58:33.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.854+0000 7f6371b7e640 1 -- 172.21.15.148:0/1665552976 wait complete. 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.855+0000 7f6371b7e640 1 Processor -- start 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.855+0000 7f6371b7e640 1 -- start start 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6371b7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f636c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6371b7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15a210 0x7f636c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6371b7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c146af0 0x7f636c146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6371b7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0b5e50 con 0x7f63580039b0 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6371b7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c162500 con 0x7f636c15a210 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6371b7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c0b5fd0 con 0x7f636c146af0 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f636c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f636c0c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48514/0 (socket says 172.21.15.148:48514) 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 -- 172.21.15.148:0/2808417101 learned_addr learned my addr 172.21.15.148:0/2808417101 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:33.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f6362ffd640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15a210 0x7f636c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636bfff640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c146af0 0x7f636c146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 -- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c146af0 msgr2=0x7f636c146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c146af0 0x7f636c146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 -- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15a210 msgr2=0x7f636c15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15a210 0x7f636c15a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 -- 172.21.15.148:0/2808417101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c0ca2f0 con 0x7f63580039b0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636bfff640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c146af0 0x7f636c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.856+0000 7f636b7fe640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f636c0c9db0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f635c009ac0 tx=0x7f635c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.857+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c040020 con 0x7f63580039b0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.857+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c003bc0 con 0x7f63580039b0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.857+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c149790 con 0x7f63580039b0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.857+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c036040 con 0x7f63580039b0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.857+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c159730 con 0x7f63580039b0 2024-04-25T09:58:33.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.858+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c073a00 con 0x7f63580039b0 2024-04-25T09:58:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.858+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f635c043020 con 0x7f63580039b0 2024-04-25T09:58:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.858+0000 7f6368ff9640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b500 0x7f634c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.858+0000 7f6362ffd640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b500 0x7f634c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.858+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f635c0aa380 con 0x7f63580039b0 2024-04-25T09:58:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.859+0000 7f6362ffd640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b500 0x7f634c06d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f636c0b36e0 tx=0x7f6354009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:33.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:33.863+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f635c04a070 con 0x7f63580039b0 2024-04-25T09:58:34.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.031+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f636c0d16b0 con 0x7f63580039b0 2024-04-25T09:58:34.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.031+0000 7f6368ff9640 1 -- 172.21.15.148:0/2808417101 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f635c0044e0 con 0x7f63580039b0 2024-04-25T09:58:34.035 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:58:34.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.033+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b500 msgr2=0x7f634c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.033+0000 7f6371b7e640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b500 0x7f634c06d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f636c0b36e0 tx=0x7f6354009190 comp rx=0 tx=0).stop 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.033+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 msgr2=0x7f636c0c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.033+0000 7f6371b7e640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f636c0c9db0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f635c009ac0 tx=0x7f635c004730 comp rx=0 tx=0).stop 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 shutdown_connections 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b500 0x7f634c06d9c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c146af0 0x7f636c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c15a210 0x7f636c15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 --2- 172.21.15.148:0/2808417101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63580039b0 0x7f636c0c9db0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 >> 172.21.15.148:0/2808417101 conn(0x7f636c0d0420 msgr2=0x7f636c0b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 shutdown_connections 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.034+0000 7f6371b7e640 1 -- 172.21.15.148:0/2808417101 wait complete. 2024-04-25T09:58:34.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:34.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.145+0000 7f2799341640 1 Processor -- start 2024-04-25T09:58:34.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.145+0000 7f2799341640 1 -- start start 2024-04-25T09:58:34.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f2799341640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f27940ced80 0x7f27940d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f2799341640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279415e000 con 0x7f2794161710 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f2799341640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27940d1de0 con 0x7f279415dc20 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f2799341640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2794161c60 con 0x7f27940ced80 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f27927fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f27940ced80 0x7f27940d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f27937fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2794161710 0x7f27940d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54180/0 (socket says 172.21.15.148:54180) 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f27937fe640 1 -- 172.21.15.148:0/2337350807 learned_addr learned my addr 172.21.15.148:0/2337350807 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f27927fc640 1 -- 172.21.15.148:0/2337350807 >> v1:172.21.15.188:6789/0 conn(0x7f279415dc20 legacy=0x7f27940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f2791ffb640 1 -- 172.21.15.148:0/2337350807 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326347682 0 0) 0x7f279415e000 con 0x7f2794161710 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f27927fc640 1 -- 172.21.15.148:0/2337350807 >> v1:172.21.15.148:6789/0 conn(0x7f2794161710 legacy=0x7f27940d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.146+0000 7f27927fc640 1 -- 172.21.15.148:0/2337350807 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794161f40 con 0x7f27940ced80 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f27927fc640 1 --2- 172.21.15.148:0/2337350807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f27940d22e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f27880097a0 tx=0x7f278802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1852b185d8bf3d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2791ffb640 1 -- 172.21.15.148:0/2337350807 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788037070 con 0x7f27940ced80 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2791ffb640 1 -- 172.21.15.148:0/2337350807 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2788032440 con 0x7f27940ced80 2024-04-25T09:58:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2791ffb640 1 -- 172.21.15.148:0/2337350807 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788032810 con 0x7f27940ced80 2024-04-25T09:58:34.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 -- 172.21.15.148:0/2337350807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 msgr2=0x7f27940d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 --2- 172.21.15.148:0/2337350807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f27940d22e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f27880097a0 tx=0x7f278802a230 comp rx=0 tx=0).stop 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 -- 172.21.15.148:0/2337350807 shutdown_connections 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 --2- 172.21.15.148:0/2337350807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f27940d22e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 -- 172.21.15.148:0/2337350807 >> 172.21.15.148:0/2337350807 conn(0x7f27940d0420 msgr2=0x7f27940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 -- 172.21.15.148:0/2337350807 shutdown_connections 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.147+0000 7f2799341640 1 -- 172.21.15.148:0/2337350807 wait complete. 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 Processor -- start 2024-04-25T09:58:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 -- start start 2024-04-25T09:58:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f2794154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279415dc20 0x7f27940c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2794161710 0x7f279415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940b5e50 con 0x7f27940ced80 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794162500 con 0x7f279415dc20 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.149+0000 7f2799341640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27940b5fd0 con 0x7f2794161710 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f2794154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f2794154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48522/0 (socket says 172.21.15.148:48522) 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 -- 172.21.15.148:0/3392691741 learned_addr learned my addr 172.21.15.148:0/3392691741 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 -- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2794161710 msgr2=0x7f279415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2794161710 0x7f279415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 -- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279415dc20 msgr2=0x7f27940c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f27927fc640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279415dc20 0x7f27940c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279415dc20 0x7f27940c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 -- 172.21.15.148:0/3392691741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940c7c00 con 0x7f27940ced80 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2792ffd640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f2794154400 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f278400c450 tx=0x7f278400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27840052d0 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784005470 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27940c7ee0 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784012870 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.150+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27940c8710 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.151+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2794073a00 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.152+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2784012a10 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.152+0000 7f277b7fe640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f278006b500 0x7f278006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.152+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2784089f30 con 0x7f27940ced80 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.152+0000 7f27927fc640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f278006b500 0x7f278006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.152+0000 7f27927fc640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f278006b500 0x7f278006d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2788034000 tx=0x7f278802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.156+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2784058c60 con 0x7f27940ced80 2024-04-25T09:58:34.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.324+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f279415b590 con 0x7f27940ced80 2024-04-25T09:58:34.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.324+0000 7f277b7fe640 1 -- 172.21.15.148:0/3392691741 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2784058560 con 0x7f27940ced80 2024-04-25T09:58:34.328 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:58:34.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f278006b500 msgr2=0x7f278006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f278006b500 0x7f278006d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2788034000 tx=0x7f278802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:34.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 msgr2=0x7f2794154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f2794154400 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f278400c450 tx=0x7f278400c910 comp rx=0 tx=0).stop 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 shutdown_connections 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f278006b500 0x7f278006d9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2794161710 0x7f279415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279415dc20 0x7f27940c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 --2- 172.21.15.148:0/3392691741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27940ced80 0x7f2794154400 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 >> 172.21.15.148:0/3392691741 conn(0x7f27940d0420 msgr2=0x7f27940d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 shutdown_connections 2024-04-25T09:58:34.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.326+0000 7f2799341640 1 -- 172.21.15.148:0/3392691741 wait complete. 2024-04-25T09:58:34.341 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.437+0000 7f5c1d03f640 1 Processor -- start 2024-04-25T09:58:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.438+0000 7f5c1d03f640 1 -- start start 2024-04-25T09:58:34.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.438+0000 7f5c1d03f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c180cf600 0x7f5c180cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.438+0000 7f5c1d03f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c180c75e0 con 0x7f5c1815bb50 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.438+0000 7f5c1d03f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c180c78c0 con 0x7f5c180cbf20 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.438+0000 7f5c1d03f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c180c8680 con 0x7f5c180cf600 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.438+0000 7f5c0ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c180cf600 0x7f5c180cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c16d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5c1815bb50 0x7f5c180c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54190/0 (socket says 172.21.15.148:54190) 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c0ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5c180cf600 0x7f5c180cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48530/0 (socket says 172.21.15.148:48530) 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c16d9d640 1 -- 172.21.15.148:0/3726760149 learned_addr learned my addr 172.21.15.148:0/3726760149 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c0ffff640 1 -- 172.21.15.148:0/3726760149 >> v1:172.21.15.188:6789/0 conn(0x7f5c180cbf20 legacy=0x7f5c180cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c0ffff640 1 -- 172.21.15.148:0/3726760149 >> v1:172.21.15.148:6789/0 conn(0x7f5c1815bb50 legacy=0x7f5c180c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c0ffff640 1 -- 172.21.15.148:0/3726760149 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180c7ba0 con 0x7f5c180cf600 2024-04-25T09:58:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.439+0000 7f5c0ffff640 1 --2- 172.21.15.148:0/3726760149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c180cf600 0x7f5c180cfa40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097a0 tx=0x7f5c0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd73fe931ed0e741 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1659c640 1 -- 172.21.15.148:0/3726760149 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00037070 con 0x7f5c180cf600 2024-04-25T09:58:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1659c640 1 -- 172.21.15.148:0/3726760149 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00032440 con 0x7f5c180cf600 2024-04-25T09:58:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1659c640 1 -- 172.21.15.148:0/3726760149 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00032810 con 0x7f5c180cf600 2024-04-25T09:58:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 -- 172.21.15.148:0/3726760149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c180cf600 msgr2=0x7f5c180cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/3726760149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c180cf600 0x7f5c180cfa40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5c000097a0 tx=0x7f5c0002a270 comp rx=0 tx=0).stop 2024-04-25T09:58:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 -- 172.21.15.148:0/3726760149 shutdown_connections 2024-04-25T09:58:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/3726760149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c180cf600 0x7f5c180cfa40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 -- 172.21.15.148:0/3726760149 >> 172.21.15.148:0/3726760149 conn(0x7f5c1806f500 msgr2=0x7f5c1806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 -- 172.21.15.148:0/3726760149 shutdown_connections 2024-04-25T09:58:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.440+0000 7f5c1d03f640 1 -- 172.21.15.148:0/3726760149 wait complete. 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 Processor -- start 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 -- start start 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c180cbf20 0x7f5c1815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c180cf600 0x7f5c181f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 0x7f5c1815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180c8160 con 0x7f5c1815bb50 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180cff80 con 0x7f5c180cf600 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1d03f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180c72a0 con 0x7f5c180cbf20 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 0x7f5c1815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 0x7f5c1815d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48536/0 (socket says 172.21.15.148:48536) 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.442+0000 7f5c1759e640 1 -- 172.21.15.148:0/2620047156 learned_addr learned my addr 172.21.15.148:0/2620047156 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c0ffff640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c180cf600 0x7f5c181f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c16d9d640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c180cbf20 0x7f5c1815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1759e640 1 -- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c180cbf20 msgr2=0x7f5c1815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1759e640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c180cbf20 0x7f5c1815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1759e640 1 -- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c180cf600 msgr2=0x7f5c181f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1759e640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c180cf600 0x7f5c181f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1759e640 1 -- 172.21.15.148:0/2620047156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c180c89a0 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1759e640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 0x7f5c1815d920 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c780 tx=0x7f5c1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10004b60 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c10004d00 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1806d510 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1000aa90 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.443+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c180c8f20 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.444+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c18073a00 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.444+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c1000ac30 con 0x7f5c1815bb50 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.445+0000 7f5c0f7fe640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bf806b4d0 0x7f5bf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.445+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5c1008a610 con 0x7f5c1815bb50 2024-04-25T09:58:34.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.445+0000 7f5c16d9d640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bf806b4d0 0x7f5bf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.445+0000 7f5c16d9d640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bf806b4d0 0x7f5bf806d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000f60 tx=0x7f5c08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.449+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c10020070 con 0x7f5c1815bb50 2024-04-25T09:58:34.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.618+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c18070030 con 0x7f5c1815bb50 2024-04-25T09:58:34.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.618+0000 7f5c0f7fe640 1 -- 172.21.15.148:0/2620047156 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c100592b0 con 0x7f5c1815bb50 2024-04-25T09:58:34.622 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bf806b4d0 msgr2=0x7f5bf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bf806b4d0 0x7f5bf806d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c08000f60 tx=0x7f5c08009040 comp rx=0 tx=0).stop 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 msgr2=0x7f5c1815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 0x7f5c1815d920 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c780 tx=0x7f5c1000cc40 comp rx=0 tx=0).stop 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 shutdown_connections 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bf806b4d0 0x7f5bf806d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c1815bb50 0x7f5c1815d920 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c180cf600 0x7f5c181f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 --2- 172.21.15.148:0/2620047156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c180cbf20 0x7f5c1815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 >> 172.21.15.148:0/2620047156 conn(0x7f5c1806f500 msgr2=0x7f5c180b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 shutdown_connections 2024-04-25T09:58:34.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.620+0000 7f5c1d03f640 1 -- 172.21.15.148:0/2620047156 wait complete. 2024-04-25T09:58:34.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:58:34.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.729+0000 7f1f3c276640 1 Processor -- start 2024-04-25T09:58:34.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.729+0000 7f1f3c276640 1 -- start start 2024-04-25T09:58:34.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f3c276640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f3406c330 0x7f1f340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f3c276640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f340b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f3c276640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f340d1d50 con 0x7f1f34158230 2024-04-25T09:58:34.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f3c276640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f34158610 con 0x7f1f3406c330 2024-04-25T09:58:34.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f3c276640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3406bea0 con 0x7f1f3406c7e0 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f3406c330 0x7f1f340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1f3406c330 0x7f1f340b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48546/0 (socket says 172.21.15.148:48546) 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 -- 172.21.15.148:0/3484781180 learned_addr learned my addr 172.21.15.148:0/3484781180 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f3a813640 1 --2- 172.21.15.148:0/3484781180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f340b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 -- 172.21.15.148:0/3484781180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 msgr2=0x7f1f340b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 --2- 172.21.15.148:0/3484781180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f340b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 -- 172.21.15.148:0/3484781180 >> v1:172.21.15.188:6789/0 conn(0x7f1f34158230 legacy=0x7f1f340d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.730+0000 7f1f39811640 1 -- 172.21.15.148:0/3484781180 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340d2020 con 0x7f1f3406c330 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.731+0000 7f1f39811640 1 --2- 172.21.15.148:0/3484781180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f340b4e60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009850 tx=0x7f1f2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6151bba6f4d3b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.731+0000 7f1f39010640 1 -- 172.21.15.148:0/3484781180 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24004030 con 0x7f1f3406c330 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.731+0000 7f1f39010640 1 -- 172.21.15.148:0/3484781180 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f240041d0 con 0x7f1f3406c330 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.731+0000 7f1f39010640 1 -- 172.21.15.148:0/3484781180 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24003610 con 0x7f1f3406c330 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.731+0000 7f1f3c276640 1 -- 172.21.15.148:0/3484781180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 msgr2=0x7f1f340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.731+0000 7f1f3c276640 1 --2- 172.21.15.148:0/3484781180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f340b4e60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009850 tx=0x7f1f2402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.732+0000 7f1f3c276640 1 -- 172.21.15.148:0/3484781180 shutdown_connections 2024-04-25T09:58:34.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.732+0000 7f1f3c276640 1 --2- 172.21.15.148:0/3484781180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f340b4e60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.732+0000 7f1f3c276640 1 --2- 172.21.15.148:0/3484781180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f340b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.732+0000 7f1f3c276640 1 -- 172.21.15.148:0/3484781180 >> 172.21.15.148:0/3484781180 conn(0x7f1f34061600 msgr2=0x7f1f340619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.732+0000 7f1f3c276640 1 -- 172.21.15.148:0/3484781180 shutdown_connections 2024-04-25T09:58:34.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.732+0000 7f1f3c276640 1 -- 172.21.15.148:0/3484781180 wait complete. 2024-04-25T09:58:34.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.733+0000 7f1f3c276640 1 Processor -- start 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.733+0000 7f1f3c276640 1 -- start start 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.733+0000 7f1f3c276640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f341e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f341430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f34158230 0x7f1f341e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340d25e0 con 0x7f1f3406c330 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3406bb30 con 0x7f1f34158230 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3415e320 con 0x7f1f3406c7e0 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f341e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f341e6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48560/0 (socket says 172.21.15.148:48560) 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 -- 172.21.15.148:0/4255087536 learned_addr learned my addr 172.21.15.148:0/4255087536 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 -- 172.21.15.148:0/4255087536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 msgr2=0x7f1f341430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f341430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 -- 172.21.15.148:0/4255087536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f34158230 msgr2=0x7f1f341e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f34158230 0x7f1f341e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 -- 172.21.15.148:0/4255087536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f341e5ab0 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3a012640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f341e6b50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c410 tx=0x7f1f3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30017070 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f300035f0 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30003a20 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f341e5d90 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.734+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f341e6350 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.735+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3406de50 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.736+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f3001a070 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.736+0000 7f1f2affd640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1f1806b4d0 0x7f1f1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.736+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1f300898d0 con 0x7f1f3406c330 2024-04-25T09:58:34.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.736+0000 7f1f39811640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1f1806b4d0 0x7f1f1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:34.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.736+0000 7f1f39811640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1f1806b4d0 0x7f1f1806d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f24033000 tx=0x7f1f2402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:34.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.740+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f30058570 con 0x7f1f3406c330 2024-04-25T09:58:34.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.923+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1f340b0d10 con 0x7f1f3406c330 2024-04-25T09:58:34.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.923+0000 7f1f2affd640 1 -- 172.21.15.148:0/4255087536 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+124 (secure 0 0 0) 0x7f1f30057e70 con 0x7f1f3406c330 2024-04-25T09:58:34.927 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148:6836/2397025400 2024-04-26T09:58:15.718240+0000 2024-04-25T09:58:34.927 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148:6834/2397025400 2024-04-26T09:58:15.718240+0000 2024-04-25T09:58:34.927 INFO:teuthology.orchestra.run.smithi148.stderr:listed 2 entries 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1f1806b4d0 msgr2=0x7f1f1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1f1806b4d0 0x7f1f1806d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1f24033000 tx=0x7f1f2402c000 comp rx=0 tx=0).stop 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 msgr2=0x7f1f341e6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f341e6b50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c410 tx=0x7f1f3000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 shutdown_connections 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1f1806b4d0 0x7f1f1806d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f34158230 0x7f1f341e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f3406c7e0 0x7f1f341430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 --2- 172.21.15.148:0/4255087536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3406c330 0x7f1f341e6b50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:34.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 >> 172.21.15.148:0/4255087536 conn(0x7f1f34061600 msgr2=0x7f1f3415d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:34.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 shutdown_connections 2024-04-25T09:58:34.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:34.925+0000 7f1f3c276640 1 -- 172.21.15.148:0/4255087536 wait complete. 2024-04-25T09:58:34.940 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T09:58:34.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:34.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:34.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T09:58:34.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:34.968 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:58:35.024 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:58:35.024 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:35.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:35.043 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:35.043 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:35.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:35.079 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:58:35.079 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:35.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:35.101 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:35.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.213+0000 7f9d6b231640 1 Processor -- start 2024-04-25T09:58:35.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.213+0000 7f9d6b231640 1 -- start start 2024-04-25T09:58:35.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d6b231640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d6b231640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d640cc8e0 0x7f9d6415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d6b231640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d640d1810 0x7f9d640d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d6b231640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d640d2180 con 0x7f9d640cc8e0 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d6b231640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d6415f390 con 0x7f9d640d1810 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d6b231640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d640cce50 con 0x7f9d6415dc90 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d640cc8e0 0x7f9d6415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9d640cc8e0 0x7f9d6415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48566/0 (socket says 172.21.15.148:48566) 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 -- 172.21.15.148:0/2086684687 learned_addr learned my addr 172.21.15.148:0/2086684687 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d68fcd640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d697ce640 1 --2- 172.21.15.148:0/2086684687 >> v2:172.21.15.188:3300/0 conn(0x7f9d640d1810 0x7f9d640d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 -- 172.21.15.148:0/2086684687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 msgr2=0x7f9d641625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 -- 172.21.15.148:0/2086684687 >> v2:172.21.15.188:3300/0 conn(0x7f9d640d1810 msgr2=0x7f9d640d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 --2- 172.21.15.148:0/2086684687 >> v2:172.21.15.188:3300/0 conn(0x7f9d640d1810 0x7f9d640d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d63fff640 1 -- 172.21.15.148:0/2086684687 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64161af0 con 0x7f9d640cc8e0 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.214+0000 7f9d68fcd640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.215+0000 7f9d63fff640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cc8e0 0x7f9d6415ee50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9d640ccd60 tx=0x7f9d5002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461b1771a0d529d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.215+0000 7f9d637fe640 1 -- 172.21.15.148:0/2086684687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50037070 con 0x7f9d640cc8e0 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.215+0000 7f9d637fe640 1 -- 172.21.15.148:0/2086684687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50032440 con 0x7f9d640cc8e0 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.215+0000 7f9d637fe640 1 -- 172.21.15.148:0/2086684687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50032810 con 0x7f9d640cc8e0 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 -- 172.21.15.148:0/2086684687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cc8e0 msgr2=0x7f9d6415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cc8e0 0x7f9d6415ee50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9d640ccd60 tx=0x7f9d5002a140 comp rx=0 tx=0).stop 2024-04-25T09:58:35.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 -- 172.21.15.148:0/2086684687 shutdown_connections 2024-04-25T09:58:35.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 --2- 172.21.15.148:0/2086684687 >> v2:172.21.15.188:3300/0 conn(0x7f9d640d1810 0x7f9d640d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640cc8e0 0x7f9d6415ee50 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 --2- 172.21.15.148:0/2086684687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 -- 172.21.15.148:0/2086684687 >> 172.21.15.148:0/2086684687 conn(0x7f9d6406f590 msgr2=0x7f9d6406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:35.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 -- 172.21.15.148:0/2086684687 shutdown_connections 2024-04-25T09:58:35.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.216+0000 7f9d6b231640 1 -- 172.21.15.148:0/2086684687 wait complete. 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.217+0000 7f9d6b231640 1 Processor -- start 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.217+0000 7f9d6b231640 1 -- start start 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640cc8e0 0x7f9d640cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 0x7f9d640c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d640cd130 con 0x7f9d640d1810 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d641620b0 con 0x7f9d640cc8e0 2024-04-25T09:58:35.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6415e070 con 0x7f9d6415dc90 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 0x7f9d640c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 0x7f9d640c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48580/0 (socket says 172.21.15.148:48580) 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 -- 172.21.15.148:0/4093097893 learned_addr learned my addr 172.21.15.148:0/4093097893 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 -- 172.21.15.148:0/4093097893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 msgr2=0x7f9d641f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 -- 172.21.15.148:0/4093097893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640cc8e0 msgr2=0x7f9d640cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640cc8e0 0x7f9d640cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 -- 172.21.15.148:0/4093097893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d640c98a0 con 0x7f9d640d1810 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d63fff640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 0x7f9d640c9360 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9d5002a180 tx=0x7f9d500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50037040 con 0x7f9d640d1810 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50042070 con 0x7f9d640d1810 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50030bb0 con 0x7f9d640d1810 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d641f4e20 con 0x7f9d640d1810 2024-04-25T09:58:35.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.218+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d640c9e00 con 0x7f9d640d1810 2024-04-25T09:58:35.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.219+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d64073990 con 0x7f9d640d1810 2024-04-25T09:58:35.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.220+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d50047460 con 0x7f9d640d1810 2024-04-25T09:58:35.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.220+0000 7f9d617fa640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d4806b4d0 0x7f9d4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.220+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9d500aaff0 con 0x7f9d640d1810 2024-04-25T09:58:35.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.220+0000 7f9d68fcd640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d4806b4d0 0x7f9d4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.221+0000 7f9d68fcd640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d4806b4d0 0x7f9d4806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9d58006fd0 tx=0x7f9d58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.224+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d640c98a0 con 0x7f9d640d1810 2024-04-25T09:58:35.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.431+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d640cf6b0 con 0x7f9d640d1810 2024-04-25T09:58:35.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.431+0000 7f9d617fa640 1 -- 172.21.15.148:0/4093097893 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f9d50045360 con 0x7f9d640d1810 2024-04-25T09:58:35.435 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:35.435 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:18.742068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:35.435 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2024-04-25T09:58:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d4806b4d0 msgr2=0x7f9d4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d4806b4d0 0x7f9d4806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9d58006fd0 tx=0x7f9d58008040 comp rx=0 tx=0).stop 2024-04-25T09:58:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 msgr2=0x7f9d640c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 0x7f9d640c9360 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9d5002a180 tx=0x7f9d500046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 shutdown_connections 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d4806b4d0 0x7f9d4806d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d6415dc90 0x7f9d641f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d640d1810 0x7f9d640c9360 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 --2- 172.21.15.148:0/4093097893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d640cc8e0 0x7f9d640cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 >> 172.21.15.148:0/4093097893 conn(0x7f9d6406f590 msgr2=0x7f9d640ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 shutdown_connections 2024-04-25T09:58:35.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.433+0000 7f9d6b231640 1 -- 172.21.15.148:0/4093097893 wait complete. 2024-04-25T09:58:35.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.542+0000 7f51e745d640 1 Processor -- start 2024-04-25T09:58:35.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.542+0000 7f51e745d640 1 -- start start 2024-04-25T09:58:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e745d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 0x7f51e00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e745d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51e00d1b00 0x7f51e00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e745d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e00d0350 con 0x7f51e00cfe40 2024-04-25T09:58:35.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e745d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00d0630 con 0x7f51e00d1b00 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e745d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00d0910 con 0x7f51e015bd30 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51e00d1b00 0x7f51e00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e51f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 0x7f51e00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51e00d1b00 0x7f51e00b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:35888/0 (socket says 172.21.15.148:35888) 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e51f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 0x7f51e00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55612/0 (socket says 172.21.15.148:55612) 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 -- 172.21.15.148:0/1757813796 learned_addr learned my addr 172.21.15.148:0/1757813796 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e49f8640 1 --1- 172.21.15.148:0/1757813796 >> v1:172.21.15.188:6789/0 conn(0x7f51e00cfe40 0x7f51e00d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:59298/0 (socket says 172.21.15.148:59298) 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 -- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 msgr2=0x7f51e00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 --2- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 0x7f51e00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 -- 172.21.15.148:0/1757813796 >> v1:172.21.15.188:6789/0 conn(0x7f51e00cfe40 legacy=0x7f51e00d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e59fa640 1 -- 172.21.15.148:0/1757813796 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00d0bf0 con 0x7f51e00d1b00 2024-04-25T09:58:35.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.543+0000 7f51e51f9640 1 --2- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 0x7f51e00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.544+0000 7f51e59fa640 1 --2- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00d1b00 0x7f51e00b4a30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009e50 tx=0x7f51d802a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6ee173b36da48e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.544+0000 7f51d7fff640 1 -- 172.21.15.148:0/1757813796 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8036070 con 0x7f51e00d1b00 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.544+0000 7f51d7fff640 1 -- 172.21.15.148:0/1757813796 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8031600 con 0x7f51e00d1b00 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.544+0000 7f51d7fff640 1 -- 172.21.15.148:0/1757813796 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80319d0 con 0x7f51e00d1b00 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 -- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00d1b00 msgr2=0x7f51e00b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 --2- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00d1b00 0x7f51e00b4a30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009e50 tx=0x7f51d802a7a0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 -- 172.21.15.148:0/1757813796 shutdown_connections 2024-04-25T09:58:35.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 --2- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00d1b00 0x7f51e00b4a30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 --2- 172.21.15.148:0/1757813796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e015bd30 0x7f51e00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 -- 172.21.15.148:0/1757813796 >> 172.21.15.148:0/1757813796 conn(0x7f51e006f590 msgr2=0x7f51e006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 -- 172.21.15.148:0/1757813796 shutdown_connections 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.545+0000 7f51e745d640 1 -- 172.21.15.148:0/1757813796 wait complete. 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.546+0000 7f51e745d640 1 Processor -- start 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.546+0000 7f51e745d640 1 -- start start 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e745d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00cfe40 0x7f51e00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e745d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00d1b00 0x7f51e00c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e745d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 0x7f51e01f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e745d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b73b0 con 0x7f51e015bd30 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e745d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e015c110 con 0x7f51e00cfe40 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e745d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00b7530 con 0x7f51e00d1b00 2024-04-25T09:58:35.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 0x7f51e01f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 0x7f51e01f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48588/0 (socket says 172.21.15.148:48588) 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 -- 172.21.15.148:0/2425370584 learned_addr learned my addr 172.21.15.148:0/2425370584 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e51f9640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00cfe40 0x7f51e00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e49f8640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00d1b00 0x7f51e00c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 -- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00d1b00 msgr2=0x7f51e00c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00d1b00 0x7f51e00c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 -- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00cfe40 msgr2=0x7f51e00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00cfe40 0x7f51e00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 -- 172.21.15.148:0/2425370584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00ca430 con 0x7f51e015bd30 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e51f9640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00cfe40 0x7f51e00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.547+0000 7f51e59fa640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 0x7f51e01f1460 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f51d8034040 tx=0x7f51d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.548+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8036050 con 0x7f51e015bd30 2024-04-25T09:58:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.548+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8042070 con 0x7f51e015bd30 2024-04-25T09:58:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.548+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8009510 con 0x7f51e015bd30 2024-04-25T09:58:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.548+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e00c9b90 con 0x7f51e015bd30 2024-04-25T09:58:35.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.548+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e00ca150 con 0x7f51e015bd30 2024-04-25T09:58:35.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.549+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0073990 con 0x7f51e015bd30 2024-04-25T09:58:35.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.549+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51d8007bf0 con 0x7f51e015bd30 2024-04-25T09:58:35.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.550+0000 7f51d5ffb640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f51c406b500 0x7f51c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.550+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f51d80aa940 con 0x7f51e015bd30 2024-04-25T09:58:35.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.550+0000 7f51e51f9640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f51c406b500 0x7f51c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.550+0000 7f51e51f9640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f51c406b500 0x7f51c406d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009560 tx=0x7f51dc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.554+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51d804b020 con 0x7f51e015bd30 2024-04-25T09:58:35.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.724+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51e00b4210 con 0x7f51e015bd30 2024-04-25T09:58:35.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.725+0000 7f51d5ffb640 1 -- 172.21.15.148:0/2425370584 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17063 (secure 0 0 0) 0x7f51d8078fa0 con 0x7f51e015bd30 2024-04-25T09:58:35.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:35.729 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:15.718258+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/2397025400":"2024-04-26T09:58:15.718240+0000","172.21.15.148:6834/2397025400":"2024-04-26T09:58:15.718240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:35.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f51c406b500 msgr2=0x7f51c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f51c406b500 0x7f51c406d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009560 tx=0x7f51dc009190 comp rx=0 tx=0).stop 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 msgr2=0x7f51e01f1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 0x7f51e01f1460 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f51d8034040 tx=0x7f51d8004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 shutdown_connections 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f51c406b500 0x7f51c406d9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51e015bd30 0x7f51e01f1460 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51e00d1b00 0x7f51e00c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 --2- 172.21.15.148:0/2425370584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51e00cfe40 0x7f51e00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.727+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 >> 172.21.15.148:0/2425370584 conn(0x7f51e006f590 msgr2=0x7f51e00b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.728+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 shutdown_connections 2024-04-25T09:58:35.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.728+0000 7f51e745d640 1 -- 172.21.15.148:0/2425370584 wait complete. 2024-04-25T09:58:35.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:00.177731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:00.933836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:35.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.842+0000 7f7e8d33b640 1 Processor -- start 2024-04-25T09:58:35.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.842+0000 7f7e8d33b640 1 -- start start 2024-04-25T09:58:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e8d33b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 0x7f7e88161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e8d33b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8806c8d0 con 0x7f7e880ce0a0 2024-04-25T09:58:35.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e8d33b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e880c6960 con 0x7f7e881610f0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e8d33b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e881614d0 con 0x7f7e881616f0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e877fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7e880ce0a0 0x7f7e880d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54198/0 (socket says 172.21.15.148:54198) 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e877fe640 1 -- 172.21.15.148:0/2915574275 learned_addr learned my addr 172.21.15.148:0/2915574275 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 0x7f7e88161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758678059 0 0) 0x7f7e8806c8d0 con 0x7f7e880ce0a0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e74003520 con 0x7f7e880ce0a0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837497848 0 0) 0x7f7e74003520 con 0x7f7e880ce0a0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 msgr2=0x7f7e88161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 0x7f7e88161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 >> v1:172.21.15.188:6789/0 conn(0x7f7e881610f0 legacy=0x7f7e8806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e88162260 con 0x7f7e880ce0a0 2024-04-25T09:58:35.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7e800019f0 con 0x7f7e880ce0a0 2024-04-25T09:58:35.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 >> v1:172.21.15.148:6789/0 conn(0x7f7e880ce0a0 legacy=0x7f7e880d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e740039e0 0x7f7e74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e74004800 0x7f7e74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 0x7f7e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e881614d0 con 0x7f7e74004800 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7400ba70 con 0x7f7e740039e0 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.843+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7400bc70 con 0x7f7e881616f0 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e74004800 0x7f7e74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e86ffd640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e740039e0 0x7f7e74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e877fe640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 0x7f7e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 -- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 msgr2=0x7f7e740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e881616f0 0x7f7e740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 -- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e740039e0 msgr2=0x7f7e74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:35.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e740039e0 0x7f7e74003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 -- 172.21.15.148:0/2915574275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e74003520 con 0x7f7e74004800 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e86ffd640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e740039e0 0x7f7e74003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.844+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e74004800 0x7f7e74004c80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009a90 tx=0x7f7e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc432aec0aa0a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70036070 con 0x7f7e74004800 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e70031400 con 0x7f7e74004800 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e867fc640 1 -- 172.21.15.148:0/2915574275 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70009690 con 0x7f7e74004800 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e8d33b640 1 -- 172.21.15.148:0/2915574275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e74004800 msgr2=0x7f7e74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/2915574275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e74004800 0x7f7e74004c80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009a90 tx=0x7f7e74005830 comp rx=0 tx=0).stop 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e877fe640 1 -- 172.21.15.148:0/2915574275 reap_dead start 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e8d33b640 1 -- 172.21.15.148:0/2915574275 shutdown_connections 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e8d33b640 1 -- 172.21.15.148:0/2915574275 >> 172.21.15.148:0/2915574275 conn(0x7f7e8806f500 msgr2=0x7f7e8806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e8d33b640 1 -- 172.21.15.148:0/2915574275 shutdown_connections 2024-04-25T09:58:35.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.845+0000 7f7e8d33b640 1 -- 172.21.15.148:0/2915574275 wait complete. 2024-04-25T09:58:35.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.846+0000 7f7e8d33b640 1 Processor -- start 2024-04-25T09:58:35.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.846+0000 7f7e8d33b640 1 -- start start 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e8d33b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e74004800 0x7f7e8815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e8d33b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 0x7f7e88160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e8d33b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e88156bd0 0x7f7e880b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e8d33b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880cfce0 con 0x7f7e8814c4f0 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e8d33b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880cfa00 con 0x7f7e74004800 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e8d33b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880cf690 con 0x7f7e88156bd0 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e7ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 0x7f7e88160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e7ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 0x7f7e88160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48594/0 (socket says 172.21.15.148:48594) 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e7ffff640 1 -- 172.21.15.148:0/4008172788 learned_addr learned my addr 172.21.15.148:0/4008172788 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e86ffd640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e74004800 0x7f7e8815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.847+0000 7f7e877fe640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e88156bd0 0x7f7e880b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7ffff640 1 -- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e88156bd0 msgr2=0x7f7e880b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e88156bd0 0x7f7e880b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7ffff640 1 -- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e74004800 msgr2=0x7f7e8815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:35.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e74004800 0x7f7e8815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7ffff640 1 -- 172.21.15.148:0/4008172788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e881608b0 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e877fe640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e88156bd0 0x7f7e880b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7ffff640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 0x7f7e88160370 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009a90 tx=0x7f7e70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70040020 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e70003bc0 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8815d540 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70036040 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.848+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8815db00 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.849+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e88067be0 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.849+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e70043020 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.850+0000 7f7e7f7fe640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e6806b500 0x7f7e6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.850+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7e700aa760 con 0x7f7e8814c4f0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.850+0000 7f7e86ffd640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e6806b500 0x7f7e6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.850+0000 7f7e86ffd640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e6806b500 0x7f7e6806d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001750 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:35.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:35.854+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e88067be0 con 0x7f7e8814c4f0 2024-04-25T09:58:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.025+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e88073a00 con 0x7f7e8814c4f0 2024-04-25T09:58:36.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.026+0000 7f7e7f7fe640 1 -- 172.21.15.148:0/4008172788 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e7004a030 con 0x7f7e8814c4f0 2024-04-25T09:58:36.029 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e6806b500 msgr2=0x7f7e6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e6806b500 0x7f7e6806d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001750 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 msgr2=0x7f7e88160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 0x7f7e88160370 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009a90 tx=0x7f7e70004730 comp rx=0 tx=0).stop 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 shutdown_connections 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e6806b500 0x7f7e6806d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e88156bd0 0x7f7e880b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8814c4f0 0x7f7e88160370 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 --2- 172.21.15.148:0/4008172788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e74004800 0x7f7e8815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 >> 172.21.15.148:0/4008172788 conn(0x7f7e8806f500 msgr2=0x7f7e880b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 shutdown_connections 2024-04-25T09:58:36.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.028+0000 7f7e8d33b640 1 -- 172.21.15.148:0/4008172788 wait complete. 2024-04-25T09:58:36.042 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:58:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 Processor -- start 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 -- start start 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ce415dc20 0x7f3ce40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ce4161710 0x7f3ce40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce415e000 con 0x7f3ce40b9330 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce40d1de0 con 0x7f3ce415dc20 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.139+0000 7f3ce92dd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce4161c60 con 0x7f3ce4161710 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ce415dc20 0x7f3ce40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ce415dc20 0x7f3ce40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48596/0 (socket says 172.21.15.148:48596) 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/4282902771 learned_addr learned my addr 172.21.15.148:0/4282902771 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3cda7fc640 1 --2- 172.21.15.148:0/4282902771 >> v2:172.21.15.188:3300/0 conn(0x7f3ce4161710 0x7f3ce40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/4282902771 >> v2:172.21.15.188:3300/0 conn(0x7f3ce4161710 msgr2=0x7f3ce40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 --2- 172.21.15.148:0/4282902771 >> v2:172.21.15.188:3300/0 conn(0x7f3ce4161710 0x7f3ce40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/4282902771 >> v1:172.21.15.188:6789/0 conn(0x7f3ce40b9330 legacy=0x7f3ce406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/4282902771 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce4161f40 con 0x7f3ce415dc20 2024-04-25T09:58:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.140+0000 7f3cda7fc640 1 --2- 172.21.15.148:0/4282902771 >> v2:172.21.15.188:3300/0 conn(0x7f3ce4161710 0x7f3ce40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce2ffd640 1 --2- 172.21.15.148:0/4282902771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce415dc20 0x7f3ce40ce670 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3ce406da90 tx=0x7f3cd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab3d9d4c0786765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce27fc640 1 -- 172.21.15.148:0/4282902771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037070 con 0x7f3ce415dc20 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce27fc640 1 -- 172.21.15.148:0/4282902771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4032440 con 0x7f3ce415dc20 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce27fc640 1 -- 172.21.15.148:0/4282902771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4032810 con 0x7f3ce415dc20 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 -- 172.21.15.148:0/4282902771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce415dc20 msgr2=0x7f3ce40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/4282902771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce415dc20 0x7f3ce40ce670 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3ce406da90 tx=0x7f3cd402a230 comp rx=0 tx=0).stop 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 -- 172.21.15.148:0/4282902771 shutdown_connections 2024-04-25T09:58:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/4282902771 >> v2:172.21.15.188:3300/0 conn(0x7f3ce4161710 0x7f3ce40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/4282902771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce415dc20 0x7f3ce40ce670 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 -- 172.21.15.148:0/4282902771 >> 172.21.15.148:0/4282902771 conn(0x7f3ce40d0420 msgr2=0x7f3ce40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 -- 172.21.15.148:0/4282902771 shutdown_connections 2024-04-25T09:58:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.141+0000 7f3ce92dd640 1 -- 172.21.15.148:0/4282902771 wait complete. 2024-04-25T09:58:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.142+0000 7f3ce92dd640 1 Processor -- start 2024-04-25T09:58:36.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 -- start start 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 0x7f3ce41f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ce415dc20 0x7f3ce41e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ce4161710 0x7f3ce414bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40b5e50 con 0x7f3ce40b9330 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4162500 con 0x7f3ce415dc20 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce92dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40b5fd0 con 0x7f3ce4161710 2024-04-25T09:58:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 0x7f3ce41f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 0x7f3ce41f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48600/0 (socket says 172.21.15.148:48600) 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.143+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/405948122 learned_addr learned my addr 172.21.15.148:0/405948122 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/405948122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ce4161710 msgr2=0x7f3ce414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce2ffd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ce4161710 0x7f3ce414bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/405948122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ce415dc20 msgr2=0x7f3ce41e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce2ffd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ce415dc20 0x7f3ce41e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce2ffd640 1 -- 172.21.15.148:0/405948122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce41f3290 con 0x7f3ce40b9330 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce2ffd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 0x7f3ce41f2d50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3cd402a270 tx=0x7f3cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037040 con 0x7f3ce40b9330 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4042070 con 0x7f3ce40b9330 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce41e88a0 con 0x7f3ce40b9330 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4030bb0 con 0x7f3ce40b9330 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.144+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce41e8e60 con 0x7f3ce40b9330 2024-04-25T09:58:36.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.145+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce4073a00 con 0x7f3ce40b9330 2024-04-25T09:58:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.145+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cd4047460 con 0x7f3ce40b9330 2024-04-25T09:58:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.146+0000 7f3cdbfff640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cc406b4d0 0x7f3cc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.146+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3cd40ab090 con 0x7f3ce40b9330 2024-04-25T09:58:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.146+0000 7f3cda7fc640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cc406b4d0 0x7f3cc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.146+0000 7f3cda7fc640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cc406b4d0 0x7f3cc406d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006fd0 tx=0x7f3ccc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.150+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cd4079d30 con 0x7f3ce40b9330 2024-04-25T09:58:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.317+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ce40d16b0 con 0x7f3ce40b9330 2024-04-25T09:58:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.318+0000 7f3cdbfff640 1 -- 172.21.15.148:0/405948122 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cd4079630 con 0x7f3ce40b9330 2024-04-25T09:58:36.322 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cc406b4d0 msgr2=0x7f3cc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cc406b4d0 0x7f3cc406d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006fd0 tx=0x7f3ccc008040 comp rx=0 tx=0).stop 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 msgr2=0x7f3ce41f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 0x7f3ce41f2d50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3cd402a270 tx=0x7f3cd40046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 shutdown_connections 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cc406b4d0 0x7f3cc406d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ce4161710 0x7f3ce414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ce415dc20 0x7f3ce41e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 --2- 172.21.15.148:0/405948122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40b9330 0x7f3ce41f2d50 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 >> 172.21.15.148:0/405948122 conn(0x7f3ce40d0420 msgr2=0x7f3ce40cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 shutdown_connections 2024-04-25T09:58:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.320+0000 7f3ce92dd640 1 -- 172.21.15.148:0/405948122 wait complete. 2024-04-25T09:58:36.334 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:58:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 Processor -- start 2024-04-25T09:58:36.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 -- start start 2024-04-25T09:58:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f025c0cfd10 0x7f025c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c0c75e0 con 0x7f025c0c70d0 2024-04-25T09:58:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c0c8680 con 0x7f025c0cfd10 2024-04-25T09:58:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.431+0000 7f02611ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0c78c0 con 0x7f025c15bb50 2024-04-25T09:58:36.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025b59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f025c0cfd10 0x7f025c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f0253fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f025c0c70d0 0x7f025c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54200/0 (socket says 172.21.15.148:54200) 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f0253fff640 1 -- 172.21.15.148:0/2777257283 learned_addr learned my addr 172.21.15.148:0/2777257283 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025ad9d640 1 --2- 172.21.15.148:0/2777257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025b59e640 1 -- 172.21.15.148:0/2777257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 msgr2=0x7f025c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025b59e640 1 --2- 172.21.15.148:0/2777257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025b59e640 1 -- 172.21.15.148:0/2777257283 >> v1:172.21.15.148:6789/0 conn(0x7f025c0c70d0 legacy=0x7f025c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025b59e640 1 -- 172.21.15.148:0/2777257283 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0c7ba0 con 0x7f025c0cfd10 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.432+0000 7f025a59c640 1 -- 172.21.15.148:0/2777257283 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514641497 0 0) 0x7f025c0c75e0 con 0x7f025c0c70d0 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f025b59e640 1 --2- 172.21.15.148:0/2777257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0cfd10 0x7f025c0cbf20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0254009d90 tx=0x7f025402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de1c76dcc51a229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f025a59c640 1 -- 172.21.15.148:0/2777257283 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254007740 con 0x7f025c0cfd10 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f025a59c640 1 -- 172.21.15.148:0/2777257283 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02540078e0 con 0x7f025c0cfd10 2024-04-25T09:58:36.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f025a59c640 1 -- 172.21.15.148:0/2777257283 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254007cb0 con 0x7f025c0cfd10 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 -- 172.21.15.148:0/2777257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0cfd10 msgr2=0x7f025c0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 --2- 172.21.15.148:0/2777257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0cfd10 0x7f025c0cbf20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0254009d90 tx=0x7f025402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 -- 172.21.15.148:0/2777257283 shutdown_connections 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 --2- 172.21.15.148:0/2777257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0cfd10 0x7f025c0cbf20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 --2- 172.21.15.148:0/2777257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 -- 172.21.15.148:0/2777257283 >> 172.21.15.148:0/2777257283 conn(0x7f025c06f500 msgr2=0x7f025c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 -- 172.21.15.148:0/2777257283 shutdown_connections 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.433+0000 7f02611ce640 1 -- 172.21.15.148:0/2777257283 wait complete. 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 Processor -- start 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 -- start start 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 0x7f025c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c0cfd10 0x7f025c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15f6d0 con 0x7f025c0c70d0 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0c8160 con 0x7f025c0cfd10 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f02611ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15f850 con 0x7f025c15bb50 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 0x7f025c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 0x7f025c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48606/0 (socket says 172.21.15.148:48606) 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.435+0000 7f025ad9d640 1 -- 172.21.15.148:0/1349708373 learned_addr learned my addr 172.21.15.148:0/1349708373 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f025ad9d640 1 -- 172.21.15.148:0/1349708373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 msgr2=0x7f025c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f025ad9d640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f025ad9d640 1 -- 172.21.15.148:0/1349708373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c0cfd10 msgr2=0x7f025c1526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f025ad9d640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c0cfd10 0x7f025c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f025ad9d640 1 -- 172.21.15.148:0/1349708373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0b5310 con 0x7f025c0c70d0 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f025ad9d640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 0x7f025c156680 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f024c00e870 tx=0x7f024c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c017070 con 0x7f025c0c70d0 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c00c940 con 0x7f025c0c70d0 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c0ca820 con 0x7f025c0c70d0 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c00cd30 con 0x7f025c0c70d0 2024-04-25T09:58:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.436+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c0cade0 con 0x7f025c0c70d0 2024-04-25T09:58:36.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.437+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c0d0e50 con 0x7f025c0c70d0 2024-04-25T09:58:36.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.438+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f024c01a070 con 0x7f025c0c70d0 2024-04-25T09:58:36.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.438+0000 7f02537fe640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f023c06b4d0 0x7f023c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.438+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f024c08a210 con 0x7f025c0c70d0 2024-04-25T09:58:36.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.438+0000 7f0253fff640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f023c06b4d0 0x7f023c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.439+0000 7f0253fff640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f023c06b4d0 0x7f023c06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0244007fd0 tx=0x7f0244007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.442+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f025c0d0e50 con 0x7f025c0c70d0 2024-04-25T09:58:36.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.612+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f025c0b2af0 con 0x7f025c0c70d0 2024-04-25T09:58:36.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.613+0000 7f02537fe640 1 -- 172.21.15.148:0/1349708373 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f025c0b5310 con 0x7f025c0c70d0 2024-04-25T09:58:36.617 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.614+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f023c06b4d0 msgr2=0x7f023c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.614+0000 7f02611ce640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f023c06b4d0 0x7f023c06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0244007fd0 tx=0x7f0244007920 comp rx=0 tx=0).stop 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.614+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 msgr2=0x7f025c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.614+0000 7f02611ce640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 0x7f025c156680 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f024c00e870 tx=0x7f024c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 shutdown_connections 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f023c06b4d0 0x7f023c06d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f025c15bb50 0x7f025c0c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f025c0cfd10 0x7f025c1526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 --2- 172.21.15.148:0/1349708373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f025c0c70d0 0x7f025c156680 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 >> 172.21.15.148:0/1349708373 conn(0x7f025c06f500 msgr2=0x7f025c0cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 shutdown_connections 2024-04-25T09:58:36.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.615+0000 7f02611ce640 1 -- 172.21.15.148:0/1349708373 wait complete. 2024-04-25T09:58:36.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.722+0000 7fd9e5e6b640 1 Processor -- start 2024-04-25T09:58:36.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.722+0000 7fd9e5e6b640 1 -- start start 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9e5e6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9e00b0b90 0x7fd9e00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9e5e6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00cc3c0 con 0x7fd9e0099a40 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9e5e6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e015f400 con 0x7fd9e015f020 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9e5e6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00cccb0 con 0x7fd9e00b0b90 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9dffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9e0099a40 0x7fd9e00cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54210/0 (socket says 172.21.15.148:54210) 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9dffff640 1 -- 172.21.15.148:0/2371926061 learned_addr learned my addr 172.21.15.148:0/2371926061 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.723+0000 7fd9d6ffd640 1 --2- 172.21.15.148:0/2371926061 >> v2:172.21.15.188:3300/0 conn(0x7fd9e00b0b90 0x7fd9e00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.724+0000 7fd9deffd640 1 -- 172.21.15.148:0/2371926061 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739133900 0 0) 0x7fd9e00cc3c0 con 0x7fd9e0099a40 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.724+0000 7fd9deffd640 1 -- 172.21.15.148:0/2371926061 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9cc003520 con 0x7fd9e0099a40 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.724+0000 7fd9d6ffd640 1 -- 172.21.15.148:0/2371926061 >> v1:172.21.15.188:6789/0 conn(0x7fd9e015f020 legacy=0x7fd9e00b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.724+0000 7fd9deffd640 1 -- 172.21.15.148:0/2371926061 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624057087 0 0) 0x7fd9e015f400 con 0x7fd9e015f020 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.724+0000 7fd9d6ffd640 1 -- 172.21.15.148:0/2371926061 >> v1:172.21.15.148:6789/0 conn(0x7fd9e0099a40 legacy=0x7fd9e00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.724+0000 7fd9d6ffd640 1 -- 172.21.15.148:0/2371926061 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00d0550 con 0x7fd9e00b0b90 2024-04-25T09:58:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9d6ffd640 1 --2- 172.21.15.148:0/2371926061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e00cfe00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72cbcd55f8fe1adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9deffd640 1 -- 172.21.15.148:0/2371926061 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8037070 con 0x7fd9e00b0b90 2024-04-25T09:58:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9deffd640 1 -- 172.21.15.148:0/2371926061 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c802ac30 con 0x7fd9e00b0b90 2024-04-25T09:58:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9deffd640 1 -- 172.21.15.148:0/2371926061 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80019f0 con 0x7fd9e00b0b90 2024-04-25T09:58:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/2371926061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 msgr2=0x7fd9e00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/2371926061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e00cfe00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/2371926061 shutdown_connections 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/2371926061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e00cfe00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/2371926061 >> 172.21.15.148:0/2371926061 conn(0x7fd9e00b3310 msgr2=0x7fd9e00b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/2371926061 shutdown_connections 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.725+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/2371926061 wait complete. 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 Processor -- start 2024-04-25T09:58:36.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 -- start start 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 0x7fd9e014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e014f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9e015f020 0x7fd9e00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cc940 con 0x7fd9e0099a40 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cc660 con 0x7fd9e00b0b90 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.727+0000 7fd9e5e6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cd850 con 0x7fd9e015f020 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 0x7fd9e014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 0x7fd9e014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48614/0 (socket says 172.21.15.148:48614) 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 -- 172.21.15.148:0/452305596 learned_addr learned my addr 172.21.15.148:0/452305596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9dffff640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9e015f020 0x7fd9e00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9d6ffd640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e014f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 -- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9e015f020 msgr2=0x7fd9e00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9e015f020 0x7fd9e00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 -- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 msgr2=0x7fd9e014f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e014f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 -- 172.21.15.148:0/452305596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e006dad0 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9dffff640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9e015f020 0x7fd9e00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9df7fe640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 0x7fd9e014fc70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c410 tx=0x7fd9d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0017070 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d00035f0 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00039e0 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00af4c0 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.728+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00afa80 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.729+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e00af2f0 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.730+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9d001a070 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.730+0000 7fd9dcff9640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9c006b4d0 0x7fd9c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.730+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd9d0089830 con 0x7fd9e0099a40 2024-04-25T09:58:36.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.730+0000 7fd9d6ffd640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9c006b4d0 0x7fd9c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:36.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.731+0000 7fd9d6ffd640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9c006b4d0 0x7fd9c006d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8034000 tx=0x7fd9c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:36.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.734+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9d00584d0 con 0x7fd9e0099a40 2024-04-25T09:58:36.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.944+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9e0073990 con 0x7fd9e0099a40 2024-04-25T09:58:36.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.944+0000 7fd9dcff9640 1 -- 172.21.15.148:0/452305596 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fd9d0057dd0 con 0x7fd9e0099a40 2024-04-25T09:58:36.948 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:36.948 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:18.742068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:36.948 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9c006b4d0 msgr2=0x7fd9c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9c006b4d0 0x7fd9c006d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8034000 tx=0x7fd9c802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 msgr2=0x7fd9e014fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 0x7fd9e014fc70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c410 tx=0x7fd9d000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 shutdown_connections 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9c006b4d0 0x7fd9c006d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9e015f020 0x7fd9e00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e00b0b90 0x7fd9e014f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 --2- 172.21.15.148:0/452305596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9e0099a40 0x7fd9e014fc70 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 >> 172.21.15.148:0/452305596 conn(0x7fd9e00b3310 msgr2=0x7fd9e015f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 shutdown_connections 2024-04-25T09:58:36.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:36.947+0000 7fd9e5e6b640 1 -- 172.21.15.148:0/452305596 wait complete. 2024-04-25T09:58:36.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:37.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.055+0000 7f496a4d8640 1 Processor -- start 2024-04-25T09:58:37.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.055+0000 7f496a4d8640 1 -- start start 2024-04-25T09:58:37.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f496a4d8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f496415db70 0x7f4964162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f496a4d8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f49640d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f496a4d8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4964162690 con 0x7f49640b6cb0 2024-04-25T09:58:37.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f496a4d8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49640ceee0 con 0x7f496415db70 2024-04-25T09:58:37.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f496a4d8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640d1d10 con 0x7f49640b5c30 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4963fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f496415db70 0x7f4964162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f49637fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49640b6cb0 0x7f49640b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54220/0 (socket says 172.21.15.148:54220) 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4963fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f496415db70 0x7f4964162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48630/0 (socket says 172.21.15.148:48630) 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f49637fe640 1 -- 172.21.15.148:0/2869183967 learned_addr learned my addr 172.21.15.148:0/2869183967 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4968a75640 1 --2- 172.21.15.148:0/2869183967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f49640d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4963fff640 1 -- 172.21.15.148:0/2869183967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 msgr2=0x7f49640d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4963fff640 1 --2- 172.21.15.148:0/2869183967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f49640d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4963fff640 1 -- 172.21.15.148:0/2869183967 >> v1:172.21.15.148:6789/0 conn(0x7f49640b6cb0 legacy=0x7f49640b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4962ffd640 1 -- 172.21.15.148:0/2869183967 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403280920 0 0) 0x7f4964162690 con 0x7f49640b6cb0 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.056+0000 7f4963fff640 1 -- 172.21.15.148:0/2869183967 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640d1ff0 con 0x7f496415db70 2024-04-25T09:58:37.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.057+0000 7f4963fff640 1 --2- 172.21.15.148:0/2869183967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f496415db70 0x7f4964162150 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f49540097a0 tx=0x7f495402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb9f1d0a8815ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.057+0000 7f4962ffd640 1 -- 172.21.15.148:0/2869183967 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954037070 con 0x7f496415db70 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.057+0000 7f4962ffd640 1 -- 172.21.15.148:0/2869183967 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4954032440 con 0x7f496415db70 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.057+0000 7f4962ffd640 1 -- 172.21.15.148:0/2869183967 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954032810 con 0x7f496415db70 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 -- 172.21.15.148:0/2869183967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f496415db70 msgr2=0x7f4964162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2869183967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f496415db70 0x7f4964162150 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f49540097a0 tx=0x7f495402a180 comp rx=0 tx=0).stop 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 -- 172.21.15.148:0/2869183967 shutdown_connections 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2869183967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f49640d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2869183967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f496415db70 0x7f4964162150 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 -- 172.21.15.148:0/2869183967 >> 172.21.15.148:0/2869183967 conn(0x7f49640d0520 msgr2=0x7f49640d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 -- 172.21.15.148:0/2869183967 shutdown_connections 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.058+0000 7f496a4d8640 1 -- 172.21.15.148:0/2869183967 wait complete. 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.059+0000 7f496a4d8640 1 Processor -- start 2024-04-25T09:58:37.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.059+0000 7f496a4d8640 1 -- start start 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f496414bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 0x7f49640cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f496415db70 0x7f49640cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49640ce530 con 0x7f49640b6cb0 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496415df50 con 0x7f496415db70 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640cf1c0 con 0x7f49640b5c30 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 0x7f49640cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 0x7f49640cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48642/0 (socket says 172.21.15.148:48642) 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 -- 172.21.15.148:0/2766445670 learned_addr learned my addr 172.21.15.148:0/2766445670 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f4963fff640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f496414bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 -- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 msgr2=0x7f496414bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f4968a75640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f496415db70 0x7f49640cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f496414bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 -- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f496415db70 msgr2=0x7f49640cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f496415db70 0x7f49640cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 -- 172.21.15.148:0/2766445670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496415fce0 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f4963fff640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f496414bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f49637fe640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 0x7f49640cd070 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f495800c4b0 tx=0x7f495800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958018070 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4958007500 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958004a20 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496415ffc0 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.060+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49640cae50 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.062+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4964073990 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.062+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f495801b070 con 0x7f49640b6cb0 2024-04-25T09:58:37.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.062+0000 7f4960ff9640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495006b4d0 0x7f495006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.062+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4958089680 con 0x7f49640b6cb0 2024-04-25T09:58:37.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.062+0000 7f4963fff640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495006b4d0 0x7f495006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.063+0000 7f4963fff640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495006b4d0 0x7f495006d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4954034000 tx=0x7f495402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.066+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4958058470 con 0x7f49640b6cb0 2024-04-25T09:58:37.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.238+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49640d1820 con 0x7f49640b6cb0 2024-04-25T09:58:37.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.239+0000 7f4960ff9640 1 -- 172.21.15.148:0/2766445670 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17063 (secure 0 0 0) 0x7f4958057d70 con 0x7f49640b6cb0 2024-04-25T09:58:37.243 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:37.243 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:15.718258+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/2397025400":"2024-04-26T09:58:15.718240+0000","172.21.15.148:6834/2397025400":"2024-04-26T09:58:15.718240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:37.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495006b4d0 msgr2=0x7f495006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495006b4d0 0x7f495006d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4954034000 tx=0x7f495402e000 comp rx=0 tx=0).stop 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 msgr2=0x7f49640cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 0x7f49640cd070 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f495800c4b0 tx=0x7f495800c970 comp rx=0 tx=0).stop 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 shutdown_connections 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495006b4d0 0x7f495006d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f496415db70 0x7f49640cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49640b6cb0 0x7f49640cd070 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 --2- 172.21.15.148:0/2766445670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49640b5c30 0x7f496414bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 >> 172.21.15.148:0/2766445670 conn(0x7f49640d0520 msgr2=0x7f4964162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 shutdown_connections 2024-04-25T09:58:37.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.241+0000 7f496a4d8640 1 -- 172.21.15.148:0/2766445670 wait complete. 2024-04-25T09:58:37.256 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:58:37.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:37.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 Processor -- start 2024-04-25T09:58:37.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 -- start start 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdeb815dc00 0x7fdeb80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb815dfe0 con 0x7fdeb80ceb20 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb80d00c0 con 0x7fdeb815dc00 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.349+0000 7fdebea76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80d03a0 con 0x7fdeb80d1dd0 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebc812640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdeb815dc00 0x7fdeb80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdeb7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdeb80ceb20 0x7fdeb80d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54236/0 (socket says 172.21.15.148:54236) 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdeb7fff640 1 -- 172.21.15.148:0/3742656626 learned_addr learned my addr 172.21.15.148:0/3742656626 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebd013640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebc812640 1 -- 172.21.15.148:0/3742656626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 msgr2=0x7fdeb80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebc812640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb80ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebc812640 1 -- 172.21.15.148:0/3742656626 >> v1:172.21.15.148:6789/0 conn(0x7fdeb80ceb20 legacy=0x7fdeb80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebc812640 1 -- 172.21.15.148:0/3742656626 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80cc440 con 0x7fdeb815dc00 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdeb77fe640 1 -- 172.21.15.148:0/3742656626 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892522426 0 0) 0x7fdea4001110 con 0x7fdeb80ceb20 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.350+0000 7fdebd013640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebc812640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb80c6df0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0097a0 tx=0x7fdeac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872a897b56a804be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdeb77fe640 1 -- 172.21.15.148:0/3742656626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac037070 con 0x7fdeb815dc00 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdeb77fe640 1 -- 172.21.15.148:0/3742656626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac032440 con 0x7fdeb815dc00 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdeb77fe640 1 -- 172.21.15.148:0/3742656626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac032810 con 0x7fdeb815dc00 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 -- 172.21.15.148:0/3742656626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 msgr2=0x7fdeb80c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb80c6df0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0097a0 tx=0x7fdeac02a180 comp rx=0 tx=0).stop 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 -- 172.21.15.148:0/3742656626 shutdown_connections 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 --2- 172.21.15.148:0/3742656626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb80c6df0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 -- 172.21.15.148:0/3742656626 >> 172.21.15.148:0/3742656626 conn(0x7fdeb806f5d0 msgr2=0x7fdeb806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:37.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 -- 172.21.15.148:0/3742656626 shutdown_connections 2024-04-25T09:58:37.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.351+0000 7fdebea76640 1 -- 172.21.15.148:0/3742656626 wait complete. 2024-04-25T09:58:37.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.352+0000 7fdebea76640 1 Processor -- start 2024-04-25T09:58:37.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.352+0000 7fdebea76640 1 -- start start 2024-04-25T09:58:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebea76640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb80ceb20 0x7fdeb815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebea76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebea76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebea76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80cfd90 con 0x7fdeb815dc00 2024-04-25T09:58:37.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebea76640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80cc130 con 0x7fdeb80ceb20 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebea76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80cdd70 con 0x7fdeb80d1dd0 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebd013640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebd013640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb815f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48666/0 (socket says 172.21.15.148:48666) 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.353+0000 7fdebd013640 1 -- 172.21.15.148:0/3193930975 learned_addr learned my addr 172.21.15.148:0/3193930975 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebc812640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb80ceb20 0x7fdeb815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdeb7fff640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebd013640 1 -- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 msgr2=0x7fdeb815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebd013640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb815c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebd013640 1 -- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb80ceb20 msgr2=0x7fdeb815c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebd013640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb80ceb20 0x7fdeb815c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebd013640 1 -- 172.21.15.148:0/3193930975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80b7080 con 0x7fdeb815dc00 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebd013640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb815f7a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000ce80 tx=0x7fdeb000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb001a070 con 0x7fdeb815dc00 2024-04-25T09:58:37.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb000d880 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000dc70 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb80b5fe0 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.354+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb80b65a0 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.355+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb8073990 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.355+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdeb001d070 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.356+0000 7fdeb57fa640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde9c06b4d0 0x7fde9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.356+0000 7fdebc812640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde9c06b4d0 0x7fde9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.356+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdeb008abb0 con 0x7fdeb815dc00 2024-04-25T09:58:37.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.356+0000 7fdebc812640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde9c06b4d0 0x7fde9c06d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdeac034000 tx=0x7fdeac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.360+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdeb0059160 con 0x7fdeb815dc00 2024-04-25T09:58:37.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.564+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdeb80c66c0 con 0x7fdeb815dc00 2024-04-25T09:58:37.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.565+0000 7fdeb57fa640 1 -- 172.21.15.148:0/3193930975 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fdeb005d200 con 0x7fdeb815dc00 2024-04-25T09:58:37.569 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:37.569 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:18.742068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:37.569 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde9c06b4d0 msgr2=0x7fde9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde9c06b4d0 0x7fde9c06d990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdeac034000 tx=0x7fdeac02e000 comp rx=0 tx=0).stop 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 msgr2=0x7fdeb815f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb815f7a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000ce80 tx=0x7fdeb000a710 comp rx=0 tx=0).stop 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 shutdown_connections 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde9c06b4d0 0x7fde9c06d990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb815dc00 0x7fdeb815f7a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdeb80d1dd0 0x7fdeb815c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 --2- 172.21.15.148:0/3193930975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdeb80ceb20 0x7fdeb815c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 >> 172.21.15.148:0/3193930975 conn(0x7fdeb806f5d0 msgr2=0x7fdeb80ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 shutdown_connections 2024-04-25T09:58:37.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.567+0000 7fdebea76640 1 -- 172.21.15.148:0/3193930975 wait complete. 2024-04-25T09:58:37.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.675+0000 7f6d64110640 1 Processor -- start 2024-04-25T09:58:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.675+0000 7f6d64110640 1 -- start start 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d64110640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0d0ad0 0x7f6d5c168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d64110640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c0b9940 con 0x7f6d5c0d2e90 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d64110640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c0d1df0 con 0x7f6d5c15dd30 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d64110640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c0b7a50 con 0x7f6d5c0d0ad0 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d616ab640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6d5c0d2e90 0x7f6d5c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54250/0 (socket says 172.21.15.148:54250) 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d616ab640 1 -- 172.21.15.148:0/2147566536 learned_addr learned my addr 172.21.15.148:0/2147566536 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d626ad640 1 --2- 172.21.15.148:0/2147566536 >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0d0ad0 0x7f6d5c168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623794419 0 0) 0x7f6d5c0b9940 con 0x7f6d5c0d2e90 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.676+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d48003520 con 0x7f6d5c0d2e90 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4068859410 0 0) 0x7f6d48003520 con 0x7f6d5c0d2e90 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0d0ad0 msgr2=0x7f6d5c168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 --2- 172.21.15.148:0/2147566536 >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0d0ad0 0x7f6d5c168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 >> v1:172.21.15.188:6789/0 conn(0x7f6d5c15dd30 legacy=0x7f6d5c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0d20d0 con 0x7f6d5c0d2e90 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6d4c0017f0 con 0x7f6d5c0d2e90 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 >> v1:172.21.15.148:6789/0 conn(0x7f6d5c0d2e90 legacy=0x7f6d5c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d480039b0 0x7f6d48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48004800 0x7f6d48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480052e0 0x7f6d4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0b7a50 con 0x7f6d480052e0 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4800c1b0 con 0x7f6d48004800 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4800c3b0 con 0x7f6d480039b0 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d626ad640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480052e0 0x7f6d4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d616ab640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48004800 0x7f6d48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d61eac640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d480039b0 0x7f6d48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d626ad640 1 -- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d480039b0 msgr2=0x7f6d48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d626ad640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d480039b0 0x7f6d48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d626ad640 1 -- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48004800 msgr2=0x7f6d48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d626ad640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48004800 0x7f6d48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d626ad640 1 -- 172.21.15.148:0/2147566536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d48003520 con 0x7f6d480052e0 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d616ab640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48004800 0x7f6d48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.677+0000 7f6d61eac640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d480039b0 0x7f6d48003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d626ad640 1 -- 172.21.15.148:0/2147566536 reap_dead start 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d626ad640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480052e0 0x7f6d4800b9c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009f90 tx=0x7f6d48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37efb249d3baf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54035020 con 0x7f6d480052e0 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d54003b90 con 0x7f6d480052e0 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d60eaa640 1 -- 172.21.15.148:0/2147566536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54009c10 con 0x7f6d480052e0 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d64110640 1 -- 172.21.15.148:0/2147566536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480052e0 msgr2=0x7f6d4800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.678+0000 7f6d64110640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480052e0 0x7f6d4800b9c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009f90 tx=0x7f6d48005830 comp rx=0 tx=0).stop 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.679+0000 7f6d64110640 1 -- 172.21.15.148:0/2147566536 shutdown_connections 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.679+0000 7f6d64110640 1 --2- 172.21.15.148:0/2147566536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d480052e0 0x7f6d4800b9c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.679+0000 7f6d64110640 1 -- 172.21.15.148:0/2147566536 >> 172.21.15.148:0/2147566536 conn(0x7f6d5c061c30 msgr2=0x7f6d5c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:37.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.679+0000 7f6d64110640 1 -- 172.21.15.148:0/2147566536 shutdown_connections 2024-04-25T09:58:37.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.679+0000 7f6d64110640 1 -- 172.21.15.148:0/2147566536 wait complete. 2024-04-25T09:58:37.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 Processor -- start 2024-04-25T09:58:37.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 -- start start 2024-04-25T09:58:37.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d480052e0 0x7f6d5c0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 0x7f6d5c1e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c14e7d0 0x7f6d5c14ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0d2970 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0d2690 con 0x7f6d480052e0 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.680+0000 7f6d64110640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0d1a10 con 0x7f6d5c14e7d0 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 0x7f6d5c1e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 0x7f6d5c1e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48686/0 (socket says 172.21.15.148:48686) 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 -- 172.21.15.148:0/326162165 learned_addr learned my addr 172.21.15.148:0/326162165 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d61eac640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d480052e0 0x7f6d5c0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d626ad640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c14e7d0 0x7f6d5c14ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 -- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c14e7d0 msgr2=0x7f6d5c14ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c14e7d0 0x7f6d5c14ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 -- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d480052e0 msgr2=0x7f6d5c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d480052e0 0x7f6d5c0ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 -- 172.21.15.148:0/326162165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c1e9c10 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d616ab640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 0x7f6d5c1e90c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e580 tx=0x7f6d4c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0072e0 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c018070 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c1e9930 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0057a0 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.681+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c0cbb20 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.682+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c073990 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.683+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d4c005940 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.683+0000 7f6d527fc640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d4006b4d0 0x7f6d4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.683+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6d4c08a150 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.683+0000 7f6d61eac640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d4006b4d0 0x7f6d4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:37.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.684+0000 7f6d61eac640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d4006b4d0 0x7f6d4006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6d58001530 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:37.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.687+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d4c058e00 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.896+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d5c0b85d0 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.896+0000 7f6d527fc640 1 -- 172.21.15.148:0/326162165 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f6d4c058700 con 0x7f6d5c1e8c40 2024-04-25T09:58:37.900 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:37.900 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:01.965750+0000","modified":"2024-04-25T09:58:18.742068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6835/328449171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":328449171},{"type":"v1","addr":"172.21.15.148:6835","nonce":328449171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T09:58:37.900 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 40 2024-04-25T09:58:37.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d4006b4d0 msgr2=0x7f6d4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d4006b4d0 0x7f6d4006d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6d58001530 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2024-04-25T09:58:37.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 msgr2=0x7f6d5c1e90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:37.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 0x7f6d5c1e90c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e580 tx=0x7f6d4c00ea40 comp rx=0 tx=0).stop 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 shutdown_connections 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d4006b4d0 0x7f6d4006d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c14e7d0 0x7f6d5c14ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d5c1e8c40 0x7f6d5c1e90c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 --2- 172.21.15.148:0/326162165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d480052e0 0x7f6d5c0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 >> 172.21.15.148:0/326162165 conn(0x7f6d5c061c30 msgr2=0x7f6d5c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 shutdown_connections 2024-04-25T09:58:37.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:37.898+0000 7f6d64110640 1 -- 172.21.15.148:0/326162165 wait complete. 2024-04-25T09:58:37.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:38.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.006+0000 7f085aeb3640 1 Processor -- start 2024-04-25T09:58:38.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f085aeb3640 1 -- start start 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f085aeb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 0x7f08540d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f085aeb3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08540b97a0 con 0x7f085415dbe0 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f085aeb3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08540d1c50 con 0x7f08540d1020 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f085aeb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085415dfc0 con 0x7f08540b2cb0 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f0858c4f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f085415dbe0 0x7f08540d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54252/0 (socket says 172.21.15.148:54252) 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.007+0000 7f0858c4f640 1 -- 172.21.15.148:0/1621504980 learned_addr learned my addr 172.21.15.148:0/1621504980 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87188499 0 0) 0x7f08540b97a0 con 0x7f085415dbe0 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0844003520 con 0x7f085415dbe0 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 0x7f08540d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970677773 0 0) 0x7f0844003520 con 0x7f085415dbe0 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 msgr2=0x7f08540d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 0x7f08540d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 >> v1:172.21.15.188:6789/0 conn(0x7f08540d1020 legacy=0x7f08540d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08540d1f30 con 0x7f085415dbe0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f0848001670 con 0x7f085415dbe0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 >> v1:172.21.15.148:6789/0 conn(0x7f085415dbe0 legacy=0x7f08540d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08440039e0 0x7f0844003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f0844004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 0x7f0844005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084400bac0 con 0x7f0844004800 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084400bc70 con 0x7f08440039e0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084400be70 con 0x7f08540b2cb0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f0844004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0858c4f640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08440039e0 0x7f0844003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0859450640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 0x7f0844005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 -- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 msgr2=0x7f0844005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08540b2cb0 0x7f0844005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 -- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08440039e0 msgr2=0x7f0844003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08440039e0 0x7f0844003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.008+0000 7f0853fff640 1 -- 172.21.15.148:0/1621504980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844003520 con 0x7f0844004800 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.009+0000 7f0853fff640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f0844004c80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f084000ba30 tx=0x7f0844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7887adec453a0fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.009+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840030070 con 0x7f0844004800 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.009+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084002ade0 con 0x7f0844004800 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.009+0000 7f08537fe640 1 -- 172.21.15.148:0/1621504980 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084000b600 con 0x7f0844004800 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f085aeb3640 1 -- 172.21.15.148:0/1621504980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 msgr2=0x7f0844004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f085aeb3640 1 --2- 172.21.15.148:0/1621504980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f0844004c80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f084000ba30 tx=0x7f0844005830 comp rx=0 tx=0).stop 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f0859450640 1 -- 172.21.15.148:0/1621504980 reap_dead start 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f085aeb3640 1 -- 172.21.15.148:0/1621504980 shutdown_connections 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f085aeb3640 1 -- 172.21.15.148:0/1621504980 >> 172.21.15.148:0/1621504980 conn(0x7f0854061a20 msgr2=0x7f0854061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:38.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f085aeb3640 1 -- 172.21.15.148:0/1621504980 shutdown_connections 2024-04-25T09:58:38.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.010+0000 7f085aeb3640 1 -- 172.21.15.148:0/1621504980 wait complete. 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.011+0000 7f085aeb3640 1 Processor -- start 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.011+0000 7f085aeb3640 1 -- start start 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f085aeb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f08541f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f085aeb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08541e60a0 0x7f08540cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f085aeb3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08541e6520 0x7f08540ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f085aeb3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08540b92a0 con 0x7f0844004800 2024-04-25T09:58:38.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f085aeb3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08540d1a00 con 0x7f08541e6520 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f085aeb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08540b9420 con 0x7f08541e60a0 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f08541f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f08541f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48706/0 (socket says 172.21.15.148:48706) 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 -- 172.21.15.148:0/4113084124 learned_addr learned my addr 172.21.15.148:0/4113084124 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0859450640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08541e6520 0x7f08540ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0853fff640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08541e60a0 0x7f08540cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 -- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08541e60a0 msgr2=0x7f08540cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08541e60a0 0x7f08540cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 -- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08541e6520 msgr2=0x7f08540ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08541e6520 0x7f08540ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 -- 172.21.15.148:0/4113084124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08541f6ca0 con 0x7f0844004800 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0859450640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08541e6520 0x7f08540ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f0858c4f640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f08541f6760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0848007310 tx=0x7f08480077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.012+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084800f960 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.013+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0848018070 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.013+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084800fd20 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.013+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08541f4cb0 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.013+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08541e6dd0 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.013+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0854073990 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.014+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f084801d460 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.014+0000 7f08517fa640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f083806b4d0 0x7f083806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.014+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f084808a4e0 con 0x7f0844004800 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.014+0000 7f0853fff640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f083806b4d0 0x7f083806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.015+0000 7f0853fff640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f083806b4d0 0x7f083806d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f084000ba00 tx=0x7f0840030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.019+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0848059180 con 0x7f0844004800 2024-04-25T09:58:38.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.191+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08540b28d0 con 0x7f0844004800 2024-04-25T09:58:38.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.192+0000 7f08517fa640 1 -- 172.21.15.148:0/4113084124 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17063 (secure 0 0 0) 0x7f0848058a80 con 0x7f0844004800 2024-04-25T09:58:38.196 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:38.196 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":71,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:15.718258+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:00.177731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:00.933836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/2397025400":"2024-04-26T09:58:15.718240+0000","172.21.15.148:6834/2397025400":"2024-04-26T09:58:15.718240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f083806b4d0 msgr2=0x7f083806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f083806b4d0 0x7f083806d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f084000ba00 tx=0x7f0840030000 comp rx=0 tx=0).stop 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 msgr2=0x7f08541f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f08541f6760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0848007310 tx=0x7f08480077d0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 shutdown_connections 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f083806b4d0 0x7f083806d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08541e6520 0x7f08540ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08541e60a0 0x7f08540cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 --2- 172.21.15.148:0/4113084124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0844004800 0x7f08541f6760 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 >> 172.21.15.148:0/4113084124 conn(0x7f0854061a20 msgr2=0x7f08540b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:38.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 shutdown_connections 2024-04-25T09:58:38.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.194+0000 7f085aeb3640 1 -- 172.21.15.148:0/4113084124 wait complete. 2024-04-25T09:58:38.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:58:38.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 Processor -- start 2024-04-25T09:58:38.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 -- start start 2024-04-25T09:58:38.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd27c144d20 0x7fd27c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0714d0 0x7fd27c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27c0ceba0 con 0x7fd27c1454b0 2024-04-25T09:58:38.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27c0b4830 con 0x7fd27c144d20 2024-04-25T09:58:38.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.303+0000 7fd283163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0cee80 con 0x7fd27c0714d0 2024-04-25T09:58:38.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd27c144d20 0x7fd27c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd27c144d20 0x7fd27c0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48710/0 (socket says 172.21.15.148:48710) 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 -- 172.21.15.148:0/3994662984 learned_addr learned my addr 172.21.15.148:0/3994662984 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd27bfff640 1 --1- 172.21.15.148:0/3994662984 >> v1:172.21.15.148:6789/0 conn(0x7fd27c1454b0 0x7fd27c070dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54268/0 (socket says 172.21.15.148:54268) 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd281700640 1 --2- 172.21.15.148:0/3994662984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0714d0 0x7fd27c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 -- 172.21.15.148:0/3994662984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0714d0 msgr2=0x7fd27c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 --2- 172.21.15.148:0/3994662984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0714d0 0x7fd27c0b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 -- 172.21.15.148:0/3994662984 >> v1:172.21.15.148:6789/0 conn(0x7fd27c1454b0 legacy=0x7fd27c070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd280eff640 1 -- 172.21.15.148:0/3994662984 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c0cf160 con 0x7fd27c144d20 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.304+0000 7fd27b7fe640 1 -- 172.21.15.148:0/3994662984 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905384984 0 0) 0x7fd27c0ceba0 con 0x7fd27c1454b0 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd280eff640 1 --2- 172.21.15.148:0/3994662984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c144d20 0x7fd27c0ce350 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd270009870 tx=0x7fd27002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63edcd3892c2ea94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd27b7fe640 1 -- 172.21.15.148:0/3994662984 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270037070 con 0x7fd27c144d20 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd27b7fe640 1 -- 172.21.15.148:0/3994662984 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd270032440 con 0x7fd27c144d20 2024-04-25T09:58:38.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd27b7fe640 1 -- 172.21.15.148:0/3994662984 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270032810 con 0x7fd27c144d20 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 -- 172.21.15.148:0/3994662984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c144d20 msgr2=0x7fd27c0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 --2- 172.21.15.148:0/3994662984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c144d20 0x7fd27c0ce350 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd270009870 tx=0x7fd27002a270 comp rx=0 tx=0).stop 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 -- 172.21.15.148:0/3994662984 shutdown_connections 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 --2- 172.21.15.148:0/3994662984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0714d0 0x7fd27c0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 --2- 172.21.15.148:0/3994662984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c144d20 0x7fd27c0ce350 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 -- 172.21.15.148:0/3994662984 >> 172.21.15.148:0/3994662984 conn(0x7fd27c0b3530 msgr2=0x7fd27c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 -- 172.21.15.148:0/3994662984 shutdown_connections 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.305+0000 7fd283163640 1 -- 172.21.15.148:0/3994662984 wait complete. 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.306+0000 7fd283163640 1 Processor -- start 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.306+0000 7fd283163640 1 -- start start 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd283163640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 0x7fd27c1e1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd283163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c144d20 0x7fd27c1e2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd283163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c1454b0 0x7fd27c15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd283163640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c145100 con 0x7fd27c0714d0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd283163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0ce890 con 0x7fd27c1454b0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd283163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c145280 con 0x7fd27c144d20 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd280eff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 0x7fd27c1e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd280eff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 0x7fd27c1e1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48712/0 (socket says 172.21.15.148:48712) 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.307+0000 7fd280eff640 1 -- 172.21.15.148:0/3858288539 learned_addr learned my addr 172.21.15.148:0/3858288539 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd280eff640 1 -- 172.21.15.148:0/3858288539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c144d20 msgr2=0x7fd27c1e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd280eff640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c144d20 0x7fd27c1e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd280eff640 1 -- 172.21.15.148:0/3858288539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c1454b0 msgr2=0x7fd27c15b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd280eff640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c1454b0 0x7fd27c15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd280eff640 1 -- 172.21.15.148:0/3858288539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c1f6240 con 0x7fd27c0714d0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd280eff640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 0x7fd27c1e1950 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd27002a2b0 tx=0x7fd2700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270041020 con 0x7fd27c0714d0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2700306c0 con 0x7fd27c0714d0 2024-04-25T09:58:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270037040 con 0x7fd27c0714d0 2024-04-25T09:58:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c1f5f60 con 0x7fd27c0714d0 2024-04-25T09:58:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.308+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1e1150 con 0x7fd27c0714d0 2024-04-25T09:58:38.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.309+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c0b3320 con 0x7fd27c0714d0 2024-04-25T09:58:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.309+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd270044020 con 0x7fd27c0714d0 2024-04-25T09:58:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.310+0000 7fd2797fa640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd26006b4d0 0x7fd26006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.310+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd2700aa7d0 con 0x7fd27c0714d0 2024-04-25T09:58:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.310+0000 7fd27bfff640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd26006b4d0 0x7fd26006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.310+0000 7fd27bfff640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd26006b4d0 0x7fd26006d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd268007fd0 tx=0x7fd268007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.314+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd27c0b3320 con 0x7fd27c0714d0 2024-04-25T09:58:38.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.519+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd27c073a00 con 0x7fd27c0714d0 2024-04-25T09:58:38.523 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T09:58:38.520+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:58:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.521+0000 7fd2797fa640 1 -- 172.21.15.148:0/3858288539 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd27004b070 con 0x7fd27c0714d0 2024-04-25T09:58:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:58:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd26006b4d0 msgr2=0x7fd26006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd26006b4d0 0x7fd26006d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd268007fd0 tx=0x7fd268007920 comp rx=0 tx=0).stop 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 msgr2=0x7fd27c1e1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 0x7fd27c1e1950 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd27002a2b0 tx=0x7fd2700046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 shutdown_connections 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd26006b4d0 0x7fd26006d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c1454b0 0x7fd27c15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c144d20 0x7fd27c1e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 --2- 172.21.15.148:0/3858288539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0714d0 0x7fd27c1e1950 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.523+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 >> 172.21.15.148:0/3858288539 conn(0x7fd27c0b3530 msgr2=0x7fd27c0b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.524+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 shutdown_connections 2024-04-25T09:58:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.524+0000 7fd283163640 1 -- 172.21.15.148:0/3858288539 wait complete. 2024-04-25T09:58:38.530 INFO:tasks.ceph.mds.a.smithi148.stderr: -2370> 2024-04-25T09:58:21.732+0000 7f29ae978640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:58:38.540 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:58:38.558 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:58:38.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.636+0000 7fa2729e0640 1 Processor -- start 2024-04-25T09:58:38.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.637+0000 7fa2729e0640 1 -- start start 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.637+0000 7fa2729e0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa26c15bb50 0x7fa26c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.637+0000 7fa2729e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa26c0c70d0 0x7fa26c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.637+0000 7fa2729e0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c0c75e0 con 0x7fa26c0cfb40 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.637+0000 7fa2729e0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0c8680 con 0x7fa26c15bb50 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.637+0000 7fa2729e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0c78c0 con 0x7fa26c0c70d0 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa26bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa26c15bb50 0x7fa26c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa270f7d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa26c0cfb40 0x7fa26c070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54278/0 (socket says 172.21.15.148:54278) 2024-04-25T09:58:38.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa270f7d640 1 -- 172.21.15.148:0/912871 learned_addr learned my addr 172.21.15.148:0/912871 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:38.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa26bfff640 1 -- 172.21.15.148:0/912871 >> v2:172.21.15.188:3300/0 conn(0x7fa26c0c70d0 msgr2=0x7fa26c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:38.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa26bfff640 1 --2- 172.21.15.148:0/912871 >> v2:172.21.15.188:3300/0 conn(0x7fa26c0c70d0 0x7fa26c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa26bfff640 1 -- 172.21.15.148:0/912871 >> v1:172.21.15.148:6789/0 conn(0x7fa26c0cfb40 legacy=0x7fa26c070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa26bfff640 1 -- 172.21.15.148:0/912871 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c0c7ba0 con 0x7fa26c15bb50 2024-04-25T09:58:38.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.638+0000 7fa26affd640 1 -- 172.21.15.148:0/912871 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304589882 0 0) 0x7fa26c0c75e0 con 0x7fa26c0cfb40 2024-04-25T09:58:38.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa26bfff640 1 --2- 172.21.15.148:0/912871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15bb50 0x7fa26c0c6b90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa26c06f220 tx=0x7fa26002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dada5ac3ff93f9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa26affd640 1 -- 172.21.15.148:0/912871 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260037070 con 0x7fa26c15bb50 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa26affd640 1 -- 172.21.15.148:0/912871 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa260032440 con 0x7fa26c15bb50 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa26affd640 1 -- 172.21.15.148:0/912871 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260032810 con 0x7fa26c15bb50 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 -- 172.21.15.148:0/912871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15bb50 msgr2=0x7fa26c0c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 --2- 172.21.15.148:0/912871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15bb50 0x7fa26c0c6b90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa26c06f220 tx=0x7fa26002a230 comp rx=0 tx=0).stop 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 -- 172.21.15.148:0/912871 shutdown_connections 2024-04-25T09:58:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 --2- 172.21.15.148:0/912871 >> v2:172.21.15.188:3300/0 conn(0x7fa26c0c70d0 0x7fa26c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 --2- 172.21.15.148:0/912871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c15bb50 0x7fa26c0c6b90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 -- 172.21.15.148:0/912871 >> 172.21.15.148:0/912871 conn(0x7fa26c06f500 msgr2=0x7fa26c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:38.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 -- 172.21.15.148:0/912871 shutdown_connections 2024-04-25T09:58:38.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.639+0000 7fa2729e0640 1 -- 172.21.15.148:0/912871 wait complete. 2024-04-25T09:58:38.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.640+0000 7fa2729e0640 1 Processor -- start 2024-04-25T09:58:38.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 -- start start 2024-04-25T09:58:38.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 0x7fa26c1f1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26c0cfb40 0x7fa26c15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa26c15bb50 0x7fa26c146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c15f6d0 con 0x7fa26c0c70d0 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0c8160 con 0x7fa26c0cfb40 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa2729e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c15f850 con 0x7fa26c15bb50 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 0x7fa26c1f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa26bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 0x7fa26c1f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48756/0 (socket says 172.21.15.148:48756) 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.641+0000 7fa26bfff640 1 -- 172.21.15.148:0/2637145240 learned_addr learned my addr 172.21.15.148:0/2637145240 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa26bfff640 1 -- 172.21.15.148:0/2637145240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa26c15bb50 msgr2=0x7fa26c146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa26bfff640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa26c15bb50 0x7fa26c146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa26bfff640 1 -- 172.21.15.148:0/2637145240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26c0cfb40 msgr2=0x7fa26c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa26bfff640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26c0cfb40 0x7fa26c15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa26bfff640 1 -- 172.21.15.148:0/2637145240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c153250 con 0x7fa26c0c70d0 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa26bfff640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 0x7fa26c1f1890 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa260000c00 tx=0x7fa2600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260041070 con 0x7fa26c0c70d0 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26003c950 con 0x7fa26c0c70d0 2024-04-25T09:58:38.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c0c9440 con 0x7fa26c0c70d0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260037040 con 0x7fa26c0c70d0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.642+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c146740 con 0x7fa26c0c70d0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.643+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c073a00 con 0x7fa26c0c70d0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.643+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa260044070 con 0x7fa26c0c70d0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.644+0000 7fa268ff9640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa25806b4d0 0x7fa25806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.644+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa2600ab290 con 0x7fa26c0c70d0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.644+0000 7fa26b7fe640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa25806b4d0 0x7fa25806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:38.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.644+0000 7fa26b7fe640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa25806b4d0 0x7fa25806d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa26c153ae0 tx=0x7fa25c006c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:38.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.648+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa260079ff0 con 0x7fa26c0c70d0 2024-04-25T09:58:38.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:38.860+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa26c0ce130 con 0x7fa26c0c70d0 2024-04-25T09:58:39.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.523+0000 7fa268ff9640 1 -- 172.21.15.148:0/2637145240 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa2600798f0 con 0x7fa26c0c70d0 2024-04-25T09:58:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa25806b4d0 msgr2=0x7fa25806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa25806b4d0 0x7fa25806d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa26c153ae0 tx=0x7fa25c006c90 comp rx=0 tx=0).stop 2024-04-25T09:58:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 msgr2=0x7fa26c1f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 0x7fa26c1f1890 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa260000c00 tx=0x7fa2600046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 shutdown_connections 2024-04-25T09:58:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa25806b4d0 0x7fa25806d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa26c15bb50 0x7fa26c146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa26c0cfb40 0x7fa26c15ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 --2- 172.21.15.148:0/2637145240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa26c0c70d0 0x7fa26c1f1890 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 >> 172.21.15.148:0/2637145240 conn(0x7fa26c06f500 msgr2=0x7fa26c0ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 shutdown_connections 2024-04-25T09:58:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.526+0000 7fa2729e0640 1 -- 172.21.15.148:0/2637145240 wait complete. 2024-04-25T09:58:39.542 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:58:39.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.681+0000 7f64eb35c640 1 Processor -- start 2024-04-25T09:58:39.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.681+0000 7f64eb35c640 1 -- start start 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64eb35c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64e415dc20 0x7f64e40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64eb35c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f64e4161710 0x7f64e40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64eb35c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e415e000 con 0x7f64e40b9330 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64eb35c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e40d1de0 con 0x7f64e415dc20 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64eb35c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e4161c60 con 0x7f64e4161710 2024-04-25T09:58:39.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64e90f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64e415dc20 0x7f64e40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64e98f9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f64e40b9330 0x7f64e406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54292/0 (socket says 172.21.15.148:54292) 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.682+0000 7f64e98f9640 1 -- 172.21.15.148:0/2934219924 learned_addr learned my addr 172.21.15.148:0/2934219924 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.683+0000 7f64e90f8640 1 -- 172.21.15.148:0/2934219924 >> v2:172.21.15.188:3300/0 conn(0x7f64e4161710 msgr2=0x7f64e40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.683+0000 7f64e90f8640 1 --2- 172.21.15.148:0/2934219924 >> v2:172.21.15.188:3300/0 conn(0x7f64e4161710 0x7f64e40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.683+0000 7f64e90f8640 1 -- 172.21.15.148:0/2934219924 >> v1:172.21.15.148:6789/0 conn(0x7f64e40b9330 legacy=0x7f64e406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.683+0000 7f64dbfff640 1 -- 172.21.15.148:0/2934219924 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299916396 0 0) 0x7f64e415e000 con 0x7f64e40b9330 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.683+0000 7f64e90f8640 1 -- 172.21.15.148:0/2934219924 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4161f40 con 0x7f64e415dc20 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.683+0000 7f64e90f8640 1 --2- 172.21.15.148:0/2934219924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e415dc20 0x7f64e40ce670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f64e406da90 tx=0x7f64e002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c9375f199696f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64dbfff640 1 -- 172.21.15.148:0/2934219924 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0007740 con 0x7f64e415dc20 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64dbfff640 1 -- 172.21.15.148:0/2934219924 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e00078e0 con 0x7f64e415dc20 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64dbfff640 1 -- 172.21.15.148:0/2934219924 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0007cb0 con 0x7f64e415dc20 2024-04-25T09:58:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 -- 172.21.15.148:0/2934219924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e415dc20 msgr2=0x7f64e40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 --2- 172.21.15.148:0/2934219924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e415dc20 0x7f64e40ce670 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f64e406da90 tx=0x7f64e002a680 comp rx=0 tx=0).stop 2024-04-25T09:58:39.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 -- 172.21.15.148:0/2934219924 shutdown_connections 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 --2- 172.21.15.148:0/2934219924 >> v2:172.21.15.188:3300/0 conn(0x7f64e4161710 0x7f64e40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 --2- 172.21.15.148:0/2934219924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e415dc20 0x7f64e40ce670 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 -- 172.21.15.148:0/2934219924 >> 172.21.15.148:0/2934219924 conn(0x7f64e40d0420 msgr2=0x7f64e40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 -- 172.21.15.148:0/2934219924 shutdown_connections 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.684+0000 7f64eb35c640 1 -- 172.21.15.148:0/2934219924 wait complete. 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.685+0000 7f64eb35c640 1 Processor -- start 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.685+0000 7f64eb35c640 1 -- start start 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64eb35c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 0x7f64e41496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64eb35c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64e415dc20 0x7f64e4159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64eb35c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64e4161710 0x7f64e4157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64eb35c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e40b5e50 con 0x7f64e40b9330 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64eb35c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e4162500 con 0x7f64e415dc20 2024-04-25T09:58:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64eb35c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e40b5fd0 con 0x7f64e4161710 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 0x7f64e41496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 0x7f64e41496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48786/0 (socket says 172.21.15.148:48786) 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 -- 172.21.15.148:0/1618015897 learned_addr learned my addr 172.21.15.148:0/1618015897 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e88f7640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64e415dc20 0x7f64e4159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e98f9640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64e4161710 0x7f64e4157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 -- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64e4161710 msgr2=0x7f64e4157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64e4161710 0x7f64e4157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 -- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64e415dc20 msgr2=0x7f64e4159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64e415dc20 0x7f64e4159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e90f8640 1 -- 172.21.15.148:0/1618015897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e415a080 con 0x7f64e40b9330 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.686+0000 7f64e88f7640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64e415dc20 0x7f64e4159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.687+0000 7f64e90f8640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 0x7f64e41496c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009e80 tx=0x7f64e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.687+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0040070 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.687+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0031d00 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.687+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0045680 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.687+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e415a360 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.687+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e415a920 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.688+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e40cf690 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.688+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64e0039040 con 0x7f64e40b9330 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.689+0000 7f64d9ffb640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64c806b4d0 0x7f64c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.689+0000 7f64e88f7640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64c806b4d0 0x7f64c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:39.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.689+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f64e00abe00 con 0x7f64e40b9330 2024-04-25T09:58:39.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.689+0000 7f64e88f7640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64c806b4d0 0x7f64c806d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64e41610c0 tx=0x7f64d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:39.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.693+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64e40cf690 con 0x7f64e40b9330 2024-04-25T09:58:39.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:39.871+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f64e4073a00 con 0x7f64e40b9330 2024-04-25T09:58:40.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.798+0000 7f64d9ffb640 1 -- 172.21.15.148:0/1618015897 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f64e007aa40 con 0x7f64e40b9330 2024-04-25T09:58:40.802 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64c806b4d0 msgr2=0x7f64c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64c806b4d0 0x7f64c806d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64e41610c0 tx=0x7f64d4008040 comp rx=0 tx=0).stop 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 msgr2=0x7f64e41496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 0x7f64e41496c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009e80 tx=0x7f64e0004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 shutdown_connections 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64c806b4d0 0x7f64c806d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64e4161710 0x7f64e4157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64e415dc20 0x7f64e4159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 --2- 172.21.15.148:0/1618015897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e40b9330 0x7f64e41496c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 >> 172.21.15.148:0/1618015897 conn(0x7f64e40d0420 msgr2=0x7f64e40b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 shutdown_connections 2024-04-25T09:58:40.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.801+0000 7f64eb35c640 1 -- 172.21.15.148:0/1618015897 wait complete. 2024-04-25T09:58:40.818 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:58:40.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.920+0000 7f01c3309640 1 Processor -- start 2024-04-25T09:58:40.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c3309640 1 -- start start 2024-04-25T09:58:40.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c3309640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dc20 0x7f01bc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c3309640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01bc0d32f0 0x7f01bc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c3309640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc15e000 con 0x7f01bc161710 2024-04-25T09:58:40.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c3309640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc0d1de0 con 0x7f01bc0d32f0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c3309640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc161c60 con 0x7f01bc15dc20 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.921+0000 7f01c18a6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01bc0d32f0 0x7f01bc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01bc0d32f0 0x7f01bc0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48788/0 (socket says 172.21.15.148:48788) 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 -- 172.21.15.148:0/1687425948 learned_addr learned my addr 172.21.15.148:0/1687425948 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 -- 172.21.15.148:0/1687425948 >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dc20 msgr2=0x7f01bc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c10a5640 1 --2- 172.21.15.148:0/1687425948 >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dc20 0x7f01bc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 --2- 172.21.15.148:0/1687425948 >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dc20 0x7f01bc0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 -- 172.21.15.148:0/1687425948 >> v1:172.21.15.188:6789/0 conn(0x7f01bc161710 legacy=0x7f01bc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 -- 172.21.15.148:0/1687425948 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc161f40 con 0x7f01bc0d32f0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.922+0000 7f01c18a6640 1 --2- 172.21.15.148:0/1687425948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d32f0 0x7f01bc0cebb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0d1520 tx=0x7f01b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90317bc800a77f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01b3fff640 1 -- 172.21.15.148:0/1687425948 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4037070 con 0x7f01bc0d32f0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01b3fff640 1 -- 172.21.15.148:0/1687425948 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4032440 con 0x7f01bc0d32f0 2024-04-25T09:58:40.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01b3fff640 1 -- 172.21.15.148:0/1687425948 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4032810 con 0x7f01bc0d32f0 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 -- 172.21.15.148:0/1687425948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d32f0 msgr2=0x7f01bc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 --2- 172.21.15.148:0/1687425948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d32f0 0x7f01bc0cebb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0d1520 tx=0x7f01b402a230 comp rx=0 tx=0).stop 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 -- 172.21.15.148:0/1687425948 shutdown_connections 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 --2- 172.21.15.148:0/1687425948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d32f0 0x7f01bc0cebb0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 --2- 172.21.15.148:0/1687425948 >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dc20 0x7f01bc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 -- 172.21.15.148:0/1687425948 >> 172.21.15.148:0/1687425948 conn(0x7f01bc0d0420 msgr2=0x7f01bc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 -- 172.21.15.148:0/1687425948 shutdown_connections 2024-04-25T09:58:40.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.923+0000 7f01c3309640 1 -- 172.21.15.148:0/1687425948 wait complete. 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.924+0000 7f01c3309640 1 Processor -- start 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 -- start start 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc0d32f0 0x7f01bc158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc15dc20 0x7f01bc15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 0x7f01bc0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0b5e50 con 0x7f01bc161710 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc162500 con 0x7f01bc0d32f0 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c3309640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0b5fd0 con 0x7f01bc15dc20 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c18a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 0x7f01bc0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c18a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 0x7f01bc0cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48798/0 (socket says 172.21.15.148:48798) 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.925+0000 7f01c18a6640 1 -- 172.21.15.148:0/1256637541 learned_addr learned my addr 172.21.15.148:0/1256637541 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c18a6640 1 -- 172.21.15.148:0/1256637541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc15dc20 msgr2=0x7f01bc15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c18a6640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc15dc20 0x7f01bc15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c18a6640 1 -- 172.21.15.148:0/1256637541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc0d32f0 msgr2=0x7f01bc158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c18a6640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc0d32f0 0x7f01bc158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c18a6640 1 -- 172.21.15.148:0/1256637541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc15f6a0 con 0x7f01bc161710 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c18a6640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 0x7f01bc0cc7c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f01b40090b0 tx=0x7f01b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4041020 con 0x7f01bc161710 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4030760 con 0x7f01bc161710 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4037040 con 0x7f01bc161710 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc15f980 con 0x7f01bc161710 2024-04-25T09:58:40.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.926+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc15a300 con 0x7f01bc161710 2024-04-25T09:58:40.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.927+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc073a00 con 0x7f01bc161710 2024-04-25T09:58:40.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.928+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01b4044020 con 0x7f01bc161710 2024-04-25T09:58:40.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.928+0000 7f01b1ffb640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a006b500 0x7f01a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:40.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.928+0000 7f01c10a5640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a006b500 0x7f01a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:40.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.928+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f01b40aaaf0 con 0x7f01bc161710 2024-04-25T09:58:40.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.928+0000 7f01c10a5640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a006b500 0x7f01a006d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f01b8007dc0 tx=0x7f01b80072f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:40.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:40.932+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01b404b030 con 0x7f01bc161710 2024-04-25T09:58:41.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.109+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01bc0d16b0 con 0x7f01bc161710 2024-04-25T09:58:41.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.803+0000 7f01b1ffb640 1 -- 172.21.15.148:0/1256637541 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f01b40798f0 con 0x7f01bc161710 2024-04-25T09:58:41.807 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a006b500 msgr2=0x7f01a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a006b500 0x7f01a006d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f01b8007dc0 tx=0x7f01b80072f0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 msgr2=0x7f01bc0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 0x7f01bc0cc7c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f01b40090b0 tx=0x7f01b40046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 shutdown_connections 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a006b500 0x7f01a006d9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc161710 0x7f01bc0cc7c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc15dc20 0x7f01bc15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 --2- 172.21.15.148:0/1256637541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc0d32f0 0x7f01bc158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 >> 172.21.15.148:0/1256637541 conn(0x7f01bc0d0420 msgr2=0x7f01bc0b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 shutdown_connections 2024-04-25T09:58:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.806+0000 7f01c3309640 1 -- 172.21.15.148:0/1256637541 wait complete. 2024-04-25T09:58:41.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:58:41.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.924+0000 7f5bf217e640 1 Processor -- start 2024-04-25T09:58:41.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.924+0000 7f5bf217e640 1 -- start start 2024-04-25T09:58:41.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bf217e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec0cc4f0 0x7f5bec0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bf217e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5bec0cdb20 0x7f5bec0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bf217e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec072280 con 0x7f5bec15b270 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bf217e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec0ce4c0 con 0x7f5bec0cdb20 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bf217e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec071270 con 0x7f5bec0cc4f0 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5beaffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5bec0cdb20 0x7f5bec0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bebfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5bec15b270 0x7f5bec15b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54302/0 (socket says 172.21.15.148:54302) 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bebfff640 1 -- 172.21.15.148:0/2651713023 learned_addr learned my addr 172.21.15.148:0/2651713023 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5beaffd640 1 -- 172.21.15.148:0/2651713023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec0cc4f0 msgr2=0x7f5bec0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5beaffd640 1 --2- 172.21.15.148:0/2651713023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec0cc4f0 0x7f5bec0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5beaffd640 1 -- 172.21.15.148:0/2651713023 >> v1:172.21.15.148:6789/0 conn(0x7f5bec15b270 legacy=0x7f5bec15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5bea7fc640 1 -- 172.21.15.148:0/2651713023 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025610123 0 0) 0x7f5bec072280 con 0x7f5bec15b270 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.925+0000 7f5beaffd640 1 -- 172.21.15.148:0/2651713023 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec0714c0 con 0x7f5bec0cdb20 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.926+0000 7f5beaffd640 1 --2- 172.21.15.148:0/2651713023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec0cdf80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5be00097a0 tx=0x7f5be002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5199de7d550b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.926+0000 7f5bea7fc640 1 -- 172.21.15.148:0/2651713023 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0037070 con 0x7f5bec0cdb20 2024-04-25T09:58:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.926+0000 7f5bea7fc640 1 -- 172.21.15.148:0/2651713023 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be0032440 con 0x7f5bec0cdb20 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.926+0000 7f5bea7fc640 1 -- 172.21.15.148:0/2651713023 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0032810 con 0x7f5bec0cdb20 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 -- 172.21.15.148:0/2651713023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 msgr2=0x7f5bec0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 --2- 172.21.15.148:0/2651713023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec0cdf80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5be00097a0 tx=0x7f5be002a180 comp rx=0 tx=0).stop 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 -- 172.21.15.148:0/2651713023 shutdown_connections 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 --2- 172.21.15.148:0/2651713023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec0cdf80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 --2- 172.21.15.148:0/2651713023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec0cc4f0 0x7f5bec0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 -- 172.21.15.148:0/2651713023 >> 172.21.15.148:0/2651713023 conn(0x7f5bec06abe0 msgr2=0x7f5bec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 -- 172.21.15.148:0/2651713023 shutdown_connections 2024-04-25T09:58:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.927+0000 7f5bf217e640 1 -- 172.21.15.148:0/2651713023 wait complete. 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.928+0000 7f5bf217e640 1 Processor -- start 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.928+0000 7f5bf217e640 1 -- start start 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bec0cc4f0 0x7f5bec157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec15b270 0x7f5bec15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec071a80 con 0x7f5bec0cdb20 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec15bd90 con 0x7f5bec0cc4f0 2024-04-25T09:58:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec071d60 con 0x7f5bec15b270 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec14da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48820/0 (socket says 172.21.15.148:48820) 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 -- 172.21.15.148:0/1816480376 learned_addr learned my addr 172.21.15.148:0/1816480376 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bebfff640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec15b270 0x7f5bec15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 -- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec15b270 msgr2=0x7f5bec15cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beb7fe640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bec0cc4f0 0x7f5bec157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec15b270 0x7f5bec15cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 -- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bec0cc4f0 msgr2=0x7f5bec157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bec0cc4f0 0x7f5bec157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 -- 172.21.15.148:0/1816480376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec158330 con 0x7f5bec0cdb20 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bebfff640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec15b270 0x7f5bec15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beb7fe640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bec0cc4f0 0x7f5bec157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5beaffd640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec14da00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f5be00098c0 tx=0x7f5be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0041070 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be003c870 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0037040 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec14e340 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.929+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec1575f0 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.930+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec157130 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.931+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5be0044070 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.931+0000 7f5bd3fff640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bd806b4d0 0x7f5bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.931+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5be00aaba0 con 0x7f5bec0cdb20 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.931+0000 7f5beb7fe640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bd806b4d0 0x7f5bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.932+0000 7f5beb7fe640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bd806b4d0 0x7f5bd806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00a870 tx=0x7f5bdc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:41.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:41.935+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bec157130 con 0x7f5bec0cdb20 2024-04-25T09:58:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.109+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5bec073a00 con 0x7f5bec0cdb20 2024-04-25T09:58:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.113+0000 7f5bd3fff640 1 -- 172.21.15.148:0/1816480376 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5be0079cf0 con 0x7f5bec0cdb20 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bd806b4d0 msgr2=0x7f5bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bd806b4d0 0x7f5bd806d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00a870 tx=0x7f5bdc008040 comp rx=0 tx=0).stop 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 msgr2=0x7f5bec14da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec14da00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f5be00098c0 tx=0x7f5be00046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 shutdown_connections 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5bd806b4d0 0x7f5bd806d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bec15b270 0x7f5bec15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bec0cdb20 0x7f5bec14da00 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 --2- 172.21.15.148:0/1816480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bec0cc4f0 0x7f5bec157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 >> 172.21.15.148:0/1816480376 conn(0x7f5bec06abe0 msgr2=0x7f5bec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 shutdown_connections 2024-04-25T09:58:42.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.115+0000 7f5bf217e640 1 -- 172.21.15.148:0/1816480376 wait complete. 2024-04-25T09:58:42.130 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T09:58:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.224+0000 7fa02ba38640 1 Processor -- start 2024-04-25T09:58:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.224+0000 7fa02ba38640 1 -- start start 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.225+0000 7fa02ba38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 0x7fa024168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.225+0000 7fa02ba38640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240ceee0 con 0x7fa0240b6cb0 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.225+0000 7fa02ba38640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240d1d10 con 0x7fa02415db70 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.225+0000 7fa02ba38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240d3800 con 0x7fa0240b5c30 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.225+0000 7fa028fd3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa0240b6cb0 0x7fa0240b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:54314/0 (socket says 172.21.15.148:54314) 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.225+0000 7fa028fd3640 1 -- 172.21.15.148:0/3465501340 learned_addr learned my addr 172.21.15.148:0/3465501340 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa029fd5640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 0x7fa024168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577307990 0 0) 0x7fa0240ceee0 con 0x7fa0240b6cb0 2024-04-25T09:58:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa010003520 con 0x7fa0240b6cb0 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775086964 0 0) 0x7fa010003520 con 0x7fa0240b6cb0 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 msgr2=0x7fa024168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 0x7fa024168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 >> v1:172.21.15.188:6789/0 conn(0x7fa02415db70 legacy=0x7fa024162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240d1ff0 con 0x7fa0240b6cb0 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa014001670 con 0x7fa0240b6cb0 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 >> v1:172.21.15.148:6789/0 conn(0x7fa0240b6cb0 legacy=0x7fa0240b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0100039e0 0x7fa010003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010004800 0x7fa010004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 0x7fa0100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240d3800 con 0x7fa010004800 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01000baa0 con 0x7fa0100039e0 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01000bca0 con 0x7fa0240b5c30 2024-04-25T09:58:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa028fd3640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010004800 0x7fa010004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa0297d4640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0100039e0 0x7fa010003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa029fd5640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 0x7fa0100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa028fd3640 1 -- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 msgr2=0x7fa0100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa028fd3640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0240b5c30 0x7fa0100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa028fd3640 1 -- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0100039e0 msgr2=0x7fa010003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa028fd3640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0100039e0 0x7fa010003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.226+0000 7fa028fd3640 1 -- 172.21.15.148:0/3465501340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010003520 con 0x7fa010004800 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.227+0000 7fa028fd3640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010004800 0x7fa010004c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa0240b7090 tx=0x7fa010005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf6b8e015ef2064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.227+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014007cf0 con 0x7fa010004800 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.227+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014005af0 con 0x7fa010004800 2024-04-25T09:58:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.227+0000 7fa01bfff640 1 -- 172.21.15.148:0/3465501340 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014007070 con 0x7fa010004800 2024-04-25T09:58:42.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa02ba38640 1 -- 172.21.15.148:0/3465501340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010004800 msgr2=0x7fa010004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa02ba38640 1 --2- 172.21.15.148:0/3465501340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010004800 0x7fa010004c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa0240b7090 tx=0x7fa010005830 comp rx=0 tx=0).stop 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa029fd5640 1 -- 172.21.15.148:0/3465501340 reap_dead start 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa02ba38640 1 -- 172.21.15.148:0/3465501340 shutdown_connections 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa02ba38640 1 -- 172.21.15.148:0/3465501340 >> 172.21.15.148:0/3465501340 conn(0x7fa0240d0520 msgr2=0x7fa0240d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa02ba38640 1 -- 172.21.15.148:0/3465501340 shutdown_connections 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.228+0000 7fa02ba38640 1 -- 172.21.15.148:0/3465501340 wait complete. 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.229+0000 7fa02ba38640 1 Processor -- start 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.229+0000 7fa02ba38640 1 -- start start 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa010004800 0x7fa024151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024155a90 0x7fa024153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 0x7fa02415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240ce530 con 0x7fa02415c1f0 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02415df50 con 0x7fa024155a90 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240cf1c0 con 0x7fa010004800 2024-04-25T09:58:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 0x7fa02415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 0x7fa02415c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48838/0 (socket says 172.21.15.148:48838) 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 -- 172.21.15.148:0/620154913 learned_addr learned my addr 172.21.15.148:0/620154913 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa028fd3640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024155a90 0x7fa024153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 -- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa010004800 msgr2=0x7fa024151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa010004800 0x7fa024151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 -- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024155a90 msgr2=0x7fa024153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024155a90 0x7fa024153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 -- 172.21.15.148:0/620154913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0241524d0 con 0x7fa02415c1f0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa029fd5640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 0x7fa02415c680 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa0240b6690 tx=0x7fa01c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c003610 con 0x7fa02415c1f0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c01b070 con 0x7fa02415c1f0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c003a10 con 0x7fa02415c1f0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.230+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa024153a80 con 0x7fa02415c1f0 2024-04-25T09:58:42.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.231+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024154040 con 0x7fa02415c1f0 2024-04-25T09:58:42.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.232+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa024073990 con 0x7fa02415c1f0 2024-04-25T09:58:42.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.232+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa01c003bb0 con 0x7fa02415c1f0 2024-04-25T09:58:42.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.232+0000 7fa019ffb640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa00806b4d0 0x7fa00806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.232+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa01c089ed0 con 0x7fa02415c1f0 2024-04-25T09:58:42.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.232+0000 7fa0297d4640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa00806b4d0 0x7fa00806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.233+0000 7fa0297d4640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa00806b4d0 0x7fa00806d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa02415ceb0 tx=0x7fa020009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.237+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa01c024020 con 0x7fa02415c1f0 2024-04-25T09:58:42.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.479+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fa0240d1820 con 0x7fa02415c1f0 2024-04-25T09:58:42.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.804+0000 7fa019ffb640 1 -- 172.21.15.148:0/620154913 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v233) v1 ==== 143+0+0 (secure 0 0 0) 0x7fa0241524d0 con 0x7fa02415c1f0 2024-04-25T09:58:42.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa00806b4d0 msgr2=0x7fa00806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa00806b4d0 0x7fa00806d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa02415ceb0 tx=0x7fa020009040 comp rx=0 tx=0).stop 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 msgr2=0x7fa02415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 0x7fa02415c680 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa0240b6690 tx=0x7fa01c011920 comp rx=0 tx=0).stop 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 shutdown_connections 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa00806b4d0 0x7fa00806d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa02415c1f0 0x7fa02415c680 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa024155a90 0x7fa024153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 --2- 172.21.15.148:0/620154913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa010004800 0x7fa024151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 >> 172.21.15.148:0/620154913 conn(0x7fa0240d0520 msgr2=0x7fa024158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 shutdown_connections 2024-04-25T09:58:42.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.807+0000 7fa02ba38640 1 -- 172.21.15.148:0/620154913 wait complete. 2024-04-25T09:58:42.822 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T09:58:42.822 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T09:58:42.822 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T09:58:42.823 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:42.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.917+0000 7f954e570640 1 Processor -- start 2024-04-25T09:58:42.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.917+0000 7f954e570640 1 -- start start 2024-04-25T09:58:42.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f954e570640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f954815bd30 0x7f95480cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f954e570640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95480d1b00 0x7f95480b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f954e570640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480d0350 con 0x7f95480cfe40 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f954e570640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480d0630 con 0x7f954815bd30 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f954e570640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480d0910 con 0x7f95480d1b00 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f954815bd30 0x7f95480cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f954815bd30 0x7f95480cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48842/0 (socket says 172.21.15.148:48842) 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 -- 172.21.15.148:0/3627030846 learned_addr learned my addr 172.21.15.148:0/3627030846 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f954cb0d640 1 --2- 172.21.15.148:0/3627030846 >> v2:172.21.15.188:3300/0 conn(0x7f95480d1b00 0x7f95480b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 -- 172.21.15.148:0/3627030846 >> v2:172.21.15.188:3300/0 conn(0x7f95480d1b00 msgr2=0x7f95480b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 --2- 172.21.15.148:0/3627030846 >> v2:172.21.15.188:3300/0 conn(0x7f95480d1b00 0x7f95480b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 -- 172.21.15.148:0/3627030846 >> v1:172.21.15.188:6789/0 conn(0x7f95480cfe40 legacy=0x7f95480d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.918+0000 7f9547fff640 1 -- 172.21.15.148:0/3627030846 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480d0bf0 con 0x7f954815bd30 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.919+0000 7f9547fff640 1 --2- 172.21.15.148:0/3627030846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f954815bd30 0x7f95480cf900 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f95480b1390 tx=0x7f953802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896ec182670c0fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.919+0000 7f9546ffd640 1 -- 172.21.15.148:0/3627030846 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538037070 con 0x7f954815bd30 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.919+0000 7f9546ffd640 1 -- 172.21.15.148:0/3627030846 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953802ae50 con 0x7f954815bd30 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.919+0000 7f9546ffd640 1 -- 172.21.15.148:0/3627030846 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538032860 con 0x7f954815bd30 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.919+0000 7f954e570640 1 -- 172.21.15.148:0/3627030846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f954815bd30 msgr2=0x7f95480cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.919+0000 7f954e570640 1 --2- 172.21.15.148:0/3627030846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f954815bd30 0x7f95480cf900 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f95480b1390 tx=0x7f953802a300 comp rx=0 tx=0).stop 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.920+0000 7f954e570640 1 -- 172.21.15.148:0/3627030846 shutdown_connections 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.920+0000 7f954e570640 1 --2- 172.21.15.148:0/3627030846 >> v2:172.21.15.188:3300/0 conn(0x7f95480d1b00 0x7f95480b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.920+0000 7f954e570640 1 --2- 172.21.15.148:0/3627030846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f954815bd30 0x7f95480cf900 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.920+0000 7f954e570640 1 -- 172.21.15.148:0/3627030846 >> 172.21.15.148:0/3627030846 conn(0x7f954806f590 msgr2=0x7f954806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.920+0000 7f954e570640 1 -- 172.21.15.148:0/3627030846 shutdown_connections 2024-04-25T09:58:42.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.920+0000 7f954e570640 1 -- 172.21.15.148:0/3627030846 wait complete. 2024-04-25T09:58:42.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.921+0000 7f954e570640 1 Processor -- start 2024-04-25T09:58:42.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.921+0000 7f954e570640 1 -- start start 2024-04-25T09:58:42.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95480cfe40 0x7f95480c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 0x7f95481f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954815bd30 0x7f95480a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95480b73b0 con 0x7f95480d1b00 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954815c110 con 0x7f95480cfe40 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480b7530 con 0x7f954815bd30 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 0x7f95481f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 0x7f95481f1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48856/0 (socket says 172.21.15.148:48856) 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 -- 172.21.15.148:0/3203923840 learned_addr learned my addr 172.21.15.148:0/3203923840 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f9547fff640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95480cfe40 0x7f95480c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954cb0d640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954815bd30 0x7f95480a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 -- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954815bd30 msgr2=0x7f95480a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954815bd30 0x7f95480a8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 -- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95480cfe40 msgr2=0x7f95480c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95480cfe40 0x7f95480c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 -- 172.21.15.148:0/3203923840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954815d370 con 0x7f95480d1b00 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f95477fe640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 0x7f95481f1720 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f953c00c470 tx=0x7f953c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0052d0 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c005470 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95481f1f70 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c007260 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.922+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481f2530 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.923+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548073990 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.924+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f953c007400 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.924+0000 7f9544ff9640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f953406b4d0 0x7f953406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.924+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f953c0895f0 con 0x7f95480d1b00 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.924+0000 7f9547fff640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f953406b4d0 0x7f953406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:42.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.925+0000 7f9547fff640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f953406b4d0 0x7f953406d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9538034000 tx=0x7f953802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:42.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:42.928+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f953c058590 con 0x7f95480d1b00 2024-04-25T09:58:43.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.100+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9548074200 con 0x7f95480d1b00 2024-04-25T09:58:43.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.101+0000 7f9544ff9640 1 -- 172.21.15.148:0/3203923840 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13474 (secure 0 0 0) 0x7f953c057e90 con 0x7f95480d1b00 2024-04-25T09:58:43.105 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:43.105 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:41.799504+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/328449171":"2024-04-26T09:58:38.520785+0000","172.21.15.148:6834/328449171":"2024-04-26T09:58:38.520785+0000","172.21.15.148:6836/2397025400":"2024-04-26T09:58:15.718240+0000","172.21.15.148:6834/2397025400":"2024-04-26T09:58:15.718240+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:43.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f953406b4d0 msgr2=0x7f953406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f953406b4d0 0x7f953406d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9538034000 tx=0x7f953802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:43.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 msgr2=0x7f95481f1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 0x7f95481f1720 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f953c00c470 tx=0x7f953c001e70 comp rx=0 tx=0).stop 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 shutdown_connections 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f953406b4d0 0x7f953406d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f954815bd30 0x7f95480a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95480d1b00 0x7f95481f1720 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 --2- 172.21.15.148:0/3203923840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95480cfe40 0x7f95480c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 >> 172.21.15.148:0/3203923840 conn(0x7f954806f590 msgr2=0x7f95480b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.103+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 shutdown_connections 2024-04-25T09:58:43.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.104+0000 7f954e570640 1 -- 172.21.15.148:0/3203923840 wait complete. 2024-04-25T09:58:43.118 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:43.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.215+0000 7f3494807640 1 Processor -- start 2024-04-25T09:58:43.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.215+0000 7f3494807640 1 -- start start 2024-04-25T09:58:43.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3494807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3494807640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34900c70d0 0x7f34900cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3494807640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34900c75e0 con 0x7f34900cfb40 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3494807640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34900c8680 con 0x7f34900c70d0 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3494807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34900c78c0 con 0x7f349015bb50 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3487fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34900c70d0 0x7f34900cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3487fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34900c70d0 0x7f34900cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48862/0 (socket says 172.21.15.148:48862) 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.216+0000 7f3487fff640 1 -- 172.21.15.148:0/3401082347 learned_addr learned my addr 172.21.15.148:0/3401082347 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.217+0000 7f348e59c640 1 --2- 172.21.15.148:0/3401082347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.217+0000 7f3487fff640 1 -- 172.21.15.148:0/3401082347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 msgr2=0x7f34900c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.217+0000 7f3487fff640 1 --2- 172.21.15.148:0/3401082347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.217+0000 7f3487fff640 1 -- 172.21.15.148:0/3401082347 >> v1:172.21.15.188:6789/0 conn(0x7f34900cfb40 legacy=0x7f3490070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.217+0000 7f3487fff640 1 -- 172.21.15.148:0/3401082347 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900c7ba0 con 0x7f34900c70d0 2024-04-25T09:58:43.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.217+0000 7f3487fff640 1 --2- 172.21.15.148:0/3401082347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900cf600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3478009870 tx=0x7f347802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423a8b809a505f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f348dd9b640 1 -- 172.21.15.148:0/3401082347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478037070 con 0x7f34900c70d0 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f348dd9b640 1 -- 172.21.15.148:0/3401082347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478032440 con 0x7f34900c70d0 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f348dd9b640 1 -- 172.21.15.148:0/3401082347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478032810 con 0x7f34900c70d0 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 -- 172.21.15.148:0/3401082347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 msgr2=0x7f34900cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 --2- 172.21.15.148:0/3401082347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900cf600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3478009870 tx=0x7f347802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 -- 172.21.15.148:0/3401082347 shutdown_connections 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 --2- 172.21.15.148:0/3401082347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900cf600 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 --2- 172.21.15.148:0/3401082347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 -- 172.21.15.148:0/3401082347 >> 172.21.15.148:0/3401082347 conn(0x7f349006f500 msgr2=0x7f349006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 -- 172.21.15.148:0/3401082347 shutdown_connections 2024-04-25T09:58:43.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.218+0000 7f3494807640 1 -- 172.21.15.148:0/3401082347 wait complete. 2024-04-25T09:58:43.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.219+0000 7f3494807640 1 Processor -- start 2024-04-25T09:58:43.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.219+0000 7f3494807640 1 -- start start 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f3494807640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f3494807640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34900cfb40 0x7f34900c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f3494807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f3494807640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349015f6d0 con 0x7f34900c70d0 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f3494807640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34900c8160 con 0x7f34900cfb40 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f3494807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349015f850 con 0x7f349015bb50 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f348e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f348e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48864/0 (socket says 172.21.15.148:48864) 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f348e59c640 1 -- 172.21.15.148:0/2754566473 learned_addr learned my addr 172.21.15.148:0/2754566473 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.220+0000 7f348ed9d640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f348e59c640 1 -- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 msgr2=0x7f34900b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f3487fff640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34900cfb40 0x7f34900c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f348e59c640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f348e59c640 1 -- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34900cfb40 msgr2=0x7f34900c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f348e59c640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34900cfb40 0x7f34900c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f348e59c640 1 -- 172.21.15.148:0/2754566473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900ca2c0 con 0x7f34900c70d0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f348e59c640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900b6f80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f348000c7f0 tx=0x7f348000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480015070 con 0x7f34900c70d0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480004220 con 0x7f34900c70d0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34900ca560 con 0x7f34900c70d0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480007620 con 0x7f34900c70d0 2024-04-25T09:58:43.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.221+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34900cab20 con 0x7f34900c70d0 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.222+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34900cf960 con 0x7f34900c70d0 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.222+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34800077c0 con 0x7f34900c70d0 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.222+0000 7f3486ffd640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f347006b500 0x7f347006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.223+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3480089ea0 con 0x7f34900c70d0 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.223+0000 7f3487fff640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f347006b500 0x7f347006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.223+0000 7f3487fff640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f347006b500 0x7f347006d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3478034000 tx=0x7f347802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.227+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34900cf960 con 0x7f34900c70d0 2024-04-25T09:58:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.396+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3490073a00 con 0x7f34900c70d0 2024-04-25T09:58:43.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.396+0000 7f3486ffd640 1 -- 172.21.15.148:0/2754566473 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3480058e40 con 0x7f34900c70d0 2024-04-25T09:58:43.400 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f347006b500 msgr2=0x7f347006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f347006b500 0x7f347006d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3478034000 tx=0x7f347802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 msgr2=0x7f34900b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900b6f80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f348000c7f0 tx=0x7f348000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 shutdown_connections 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f347006b500 0x7f347006d9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f349015bb50 0x7f34900b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34900cfb40 0x7f34900c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 --2- 172.21.15.148:0/2754566473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34900c70d0 0x7f34900b6f80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 >> 172.21.15.148:0/2754566473 conn(0x7f349006f500 msgr2=0x7f34900b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 shutdown_connections 2024-04-25T09:58:43.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.398+0000 7f3494807640 1 -- 172.21.15.148:0/2754566473 wait complete. 2024-04-25T09:58:43.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:58:43.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.506+0000 7fdfeeeac640 1 Processor -- start 2024-04-25T09:58:43.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.506+0000 7fdfeeeac640 1 -- start start 2024-04-25T09:58:43.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfeeeac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfeeeac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdfe806b5f0 0x7fdfe80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfeeeac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfe80b2d40 0x7fdfe80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfeeeac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe815f710 con 0x7fdfe80b2d40 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfeeeac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe815f9b0 con 0x7fdfe806b5f0 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfeeeac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe815fc90 con 0x7fdfe815b980 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfe80b2d40 0x7fdfe80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdfe80b2d40 0x7fdfe80cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55752/0 (socket says 172.21.15.148:55752) 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 -- 172.21.15.148:0/1599609076 learned_addr learned my addr 172.21.15.148:0/1599609076 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 -- 172.21.15.148:0/1599609076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 msgr2=0x7fdfe80cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 --2- 172.21.15.148:0/1599609076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 -- 172.21.15.148:0/1599609076 >> v2:172.21.15.188:3300/0 conn(0x7fdfe806b5f0 msgr2=0x7fdfe80cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 --2- 172.21.15.148:0/1599609076 >> v2:172.21.15.188:3300/0 conn(0x7fdfe806b5f0 0x7fdfe80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.507+0000 7fdfed449640 1 -- 172.21.15.148:0/1599609076 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe815ff70 con 0x7fdfe80b2d40 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.508+0000 7fdfed449640 1 --2- 172.21.15.148:0/1599609076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80cfde0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8070dc0 tx=0x7fdfe002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8ae8a1ff34818f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.508+0000 7fdfe77fe640 1 -- 172.21.15.148:0/1599609076 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0036070 con 0x7fdfe80b2d40 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.508+0000 7fdfe77fe640 1 -- 172.21.15.148:0/1599609076 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0031600 con 0x7fdfe80b2d40 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.508+0000 7fdfe77fe640 1 -- 172.21.15.148:0/1599609076 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00319d0 con 0x7fdfe80b2d40 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 -- 172.21.15.148:0/1599609076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 msgr2=0x7fdfe80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/1599609076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80cfde0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8070dc0 tx=0x7fdfe002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 -- 172.21.15.148:0/1599609076 shutdown_connections 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/1599609076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80cfde0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/1599609076 >> v2:172.21.15.188:3300/0 conn(0x7fdfe806b5f0 0x7fdfe80cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/1599609076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 -- 172.21.15.148:0/1599609076 >> 172.21.15.148:0/1599609076 conn(0x7fdfe80b2220 msgr2=0x7fdfe80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 -- 172.21.15.148:0/1599609076 shutdown_connections 2024-04-25T09:58:43.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.509+0000 7fdfeeeac640 1 -- 172.21.15.148:0/1599609076 wait complete. 2024-04-25T09:58:43.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.510+0000 7fdfeeeac640 1 Processor -- start 2024-04-25T09:58:43.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.510+0000 7fdfeeeac640 1 -- start start 2024-04-25T09:58:43.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfeeeac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe806b5f0 0x7fdfe81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfeeeac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfeeeac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfeeeac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80cf550 con 0x7fdfe80b2d40 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfeeeac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe815bd60 con 0x7fdfe806b5f0 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfeeeac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80ce2c0 con 0x7fdfe815b980 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55756/0 (socket says 172.21.15.148:55756) 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 -- 172.21.15.148:0/3176241676 learned_addr learned my addr 172.21.15.148:0/3176241676 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfecc48640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe806b5f0 0x7fdfe81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfed449640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 -- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 msgr2=0x7fdfe80b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 -- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe806b5f0 msgr2=0x7fdfe81587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe806b5f0 0x7fdfe81587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 -- 172.21.15.148:0/3176241676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe81f4f50 con 0x7fdfe80b2d40 2024-04-25T09:58:43.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfed449640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.511+0000 7fdfe7fff640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80b4710 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b450 tx=0x7fdfd000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.512+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0015020 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.512+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd0011890 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.512+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe80c6390 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.512+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0011c80 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.512+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe80c6950 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.513+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe8073990 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.513+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfd0018020 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.513+0000 7fdfe57fa640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdfc806b4d0 0x7fdfc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.513+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdfd008a180 con 0x7fdfe80b2d40 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.513+0000 7fdfecc48640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdfc806b4d0 0x7fdfc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.514+0000 7fdfecc48640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdfc806b4d0 0x7fdfc806d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80c82e0 tx=0x7fdfd8009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.518+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfd001f020 con 0x7fdfe80b2d40 2024-04-25T09:58:43.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.751+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fdfe80b6150 con 0x7fdfe80b2d40 2024-04-25T09:58:43.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.809+0000 7fdfe57fa640 1 -- 172.21.15.148:0/3176241676 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v234) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdfd000d040 con 0x7fdfe80b2d40 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdfc806b4d0 msgr2=0x7fdfc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdfc806b4d0 0x7fdfc806d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80c82e0 tx=0x7fdfd8009060 comp rx=0 tx=0).stop 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 msgr2=0x7fdfe80b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80b4710 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b450 tx=0x7fdfd000b910 comp rx=0 tx=0).stop 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 shutdown_connections 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdfc806b4d0 0x7fdfc806d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe815b980 0x7fdfe80b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe80b2d40 0x7fdfe80b4710 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 --2- 172.21.15.148:0/3176241676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe806b5f0 0x7fdfe81587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 >> 172.21.15.148:0/3176241676 conn(0x7fdfe80b2220 msgr2=0x7fdfe80ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 shutdown_connections 2024-04-25T09:58:43.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.811+0000 7fdfeeeac640 1 -- 172.21.15.148:0/3176241676 wait complete. 2024-04-25T09:58:43.826 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:58:43.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.922+0000 7f4c326b3640 1 Processor -- start 2024-04-25T09:58:43.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c326b3640 1 -- start start 2024-04-25T09:58:43.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c326b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c2c165520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c326b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c0c6940 con 0x7f4c2c15a9f0 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c326b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c15add0 con 0x7f4c2c0c6010 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c326b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0b0020 con 0x7f4c2c0cd6f0 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c2bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4c2c15a9f0 0x7f4c2c0cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39566/0 (socket says 172.21.15.148:39566) 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.923+0000 7f4c2bfff640 1 -- 172.21.15.148:0/2330550442 learned_addr learned my addr 172.21.15.148:0/2330550442 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c2c165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070812318 0 0) 0x7f4c1c001110 con 0x7f4c2c15a9f0 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c14003520 con 0x7f4c2c15a9f0 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408659454 0 0) 0x7f4c14003520 con 0x7f4c2c15a9f0 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 msgr2=0x7f4c2c165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c2c165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 >> v1:172.21.15.188:6789/0 conn(0x7f4c2c0c6010 legacy=0x7f4c2c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0c6660 con 0x7f4c2c15a9f0 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4c1c001840 con 0x7f4c2c15a9f0 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 >> v1:172.21.15.148:6789/0 conn(0x7f4c2c15a9f0 legacy=0x7f4c2c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14003f60 0x7f4c14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14004d40 0x7f4c140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1400ba70 con 0x7f4c14004d40 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1400bc20 con 0x7f4c14003f60 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1400e030 con 0x7f4c2c0cd6f0 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14004d40 0x7f4c140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2bfff640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2b7fe640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14003f60 0x7f4c14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 -- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 msgr2=0x7f4c140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 -- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14003f60 msgr2=0x7f4c14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14003f60 0x7f4c14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c30c50640 1 -- 172.21.15.148:0/2330550442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14003520 con 0x7f4c14004d40 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2bfff640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0cd6f0 0x7f4c140039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.924+0000 7f4c2b7fe640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14003f60 0x7f4c14004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.925+0000 7f4c30c50640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14004d40 0x7f4c140051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009fc0 tx=0x7f4c14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f387800ebf7551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.925+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24009c10 con 0x7f4c14004d40 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.925+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24036070 con 0x7f4c14004d40 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.925+0000 7f4c2affd640 1 -- 172.21.15.148:0/2330550442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24031880 con 0x7f4c14004d40 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c326b3640 1 -- 172.21.15.148:0/2330550442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14004d40 msgr2=0x7f4c140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c326b3640 1 --2- 172.21.15.148:0/2330550442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c14004d40 0x7f4c140051c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009fc0 tx=0x7f4c14005830 comp rx=0 tx=0).stop 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c30c50640 1 -- 172.21.15.148:0/2330550442 reap_dead start 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c326b3640 1 -- 172.21.15.148:0/2330550442 shutdown_connections 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c326b3640 1 -- 172.21.15.148:0/2330550442 >> 172.21.15.148:0/2330550442 conn(0x7f4c2c0b2390 msgr2=0x7f4c2c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:43.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c326b3640 1 -- 172.21.15.148:0/2330550442 shutdown_connections 2024-04-25T09:58:43.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.926+0000 7f4c326b3640 1 -- 172.21.15.148:0/2330550442 wait complete. 2024-04-25T09:58:43.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.927+0000 7f4c326b3640 1 Processor -- start 2024-04-25T09:58:43.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.927+0000 7f4c326b3640 1 -- start start 2024-04-25T09:58:43.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c326b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c1532a0 0x7f4c2c155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c326b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 0x7f4c2c0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c326b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b1a60 0x7f4c2c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c326b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0cb230 con 0x7f4c2c153680 2024-04-25T09:58:43.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c326b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0cb5a0 con 0x7f4c2c1532a0 2024-04-25T09:58:43.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c326b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0cca40 con 0x7f4c2c0b1a60 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 0x7f4c2c0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 0x7f4c2c0b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55760/0 (socket says 172.21.15.148:55760) 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 -- 172.21.15.148:0/3856342589 learned_addr learned my addr 172.21.15.148:0/3856342589 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2bfff640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c1532a0 0x7f4c2c155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c30c50640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b1a60 0x7f4c2c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 -- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b1a60 msgr2=0x7f4c2c0ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b1a60 0x7f4c2c0ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 -- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c1532a0 msgr2=0x7f4c2c155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c1532a0 0x7f4c2c155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.928+0000 7f4c2b7fe640 1 -- 172.21.15.148:0/3856342589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c1542a0 con 0x7f4c2c153680 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.929+0000 7f4c2b7fe640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 0x7f4c2c0b1520 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e500 tx=0x7f4c2000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.929+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200052e0 con 0x7f4c2c153680 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.929+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20018070 con 0x7f4c2c153680 2024-04-25T09:58:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.929+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c154560 con 0x7f4c2c153680 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.929+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20003810 con 0x7f4c2c153680 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.929+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c0aeec0 con 0x7f4c2c153680 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.930+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c06cc80 con 0x7f4c2c153680 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.930+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c200039b0 con 0x7f4c2c153680 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.931+0000 7f4c28ff9640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c1806b500 0x7f4c1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.931+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4c20089bc0 con 0x7f4c2c153680 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.931+0000 7f4c2bfff640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c1806b500 0x7f4c1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:43.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.931+0000 7f4c2bfff640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c1806b500 0x7f4c1806d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c004840 tx=0x7f4c1c001190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:43.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:43.935+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c20058520 con 0x7f4c2c153680 2024-04-25T09:58:44.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.107+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4c2c0b1e70 con 0x7f4c2c153680 2024-04-25T09:58:44.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.110+0000 7f4c28ff9640 1 -- 172.21.15.148:0/3856342589 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c2c0b1e70 con 0x7f4c2c153680 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c1806b500 msgr2=0x7f4c1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c1806b500 0x7f4c1806d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c004840 tx=0x7f4c1c001190 comp rx=0 tx=0).stop 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 msgr2=0x7f4c2c0b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 0x7f4c2c0b1520 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000e500 tx=0x7f4c2000e9c0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 shutdown_connections 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c1806b500 0x7f4c1806d9c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c2c0b1a60 0x7f4c2c0ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c2c153680 0x7f4c2c0b1520 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 --2- 172.21.15.148:0/3856342589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c2c1532a0 0x7f4c2c155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 >> 172.21.15.148:0/3856342589 conn(0x7f4c2c0b2390 msgr2=0x7f4c2c06ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 shutdown_connections 2024-04-25T09:58:44.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.112+0000 7f4c326b3640 1 -- 172.21.15.148:0/3856342589 wait complete. 2024-04-25T09:58:44.127 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:58:44.127 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:44.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:58:44.206 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:58:44.207 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:44.207 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:44.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:44.222 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:58:44.222 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:44.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:44.264 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:44.264 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:44.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:58:44.279 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T09:58:44.279 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:44.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:58:44.320 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:58:44.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:44.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.413+0000 7f15b6b22640 1 Processor -- start 2024-04-25T09:58:44.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 -- start start 2024-04-25T09:58:44.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15b0144750 0x7f15b006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15b006eb10 0x7f15b00ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b00cee60 0x7f15b015f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0144b30 con 0x7f15b0144750 2024-04-25T09:58:44.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b00712d0 con 0x7f15b006eb10 2024-04-25T09:58:44.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.414+0000 7f15b6b22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00715b0 con 0x7f15b00cee60 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15b0144750 0x7f15b006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15b0144750 0x7f15b006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55766/0 (socket says 172.21.15.148:55766) 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 -- 172.21.15.148:0/4048921921 learned_addr learned my addr 172.21.15.148:0/4048921921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b50bf640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b00cee60 0x7f15b015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15affff640 1 --2- 172.21.15.148:0/4048921921 >> v2:172.21.15.188:3300/0 conn(0x7f15b006eb10 0x7f15b00ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 -- 172.21.15.148:0/4048921921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b00cee60 msgr2=0x7f15b015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b00cee60 0x7f15b015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 -- 172.21.15.148:0/4048921921 >> v2:172.21.15.188:3300/0 conn(0x7f15b006eb10 msgr2=0x7f15b00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 --2- 172.21.15.148:0/4048921921 >> v2:172.21.15.188:3300/0 conn(0x7f15b006eb10 0x7f15b00ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b48be640 1 -- 172.21.15.148:0/4048921921 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0071890 con 0x7f15b0144750 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15b50bf640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b00cee60 0x7f15b015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.415+0000 7f15affff640 1 --2- 172.21.15.148:0/4048921921 >> v2:172.21.15.188:3300/0 conn(0x7f15b006eb10 0x7f15b00ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b48be640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b0144750 0x7f15b006e5d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f15b01606b0 tx=0x7f15a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5202b45b1aec45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15af7fe640 1 -- 172.21.15.148:0/4048921921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4037070 con 0x7f15b0144750 2024-04-25T09:58:44.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15af7fe640 1 -- 172.21.15.148:0/4048921921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4032440 con 0x7f15b0144750 2024-04-25T09:58:44.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15af7fe640 1 -- 172.21.15.148:0/4048921921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4032810 con 0x7f15b0144750 2024-04-25T09:58:44.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 -- 172.21.15.148:0/4048921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b0144750 msgr2=0x7f15b006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b0144750 0x7f15b006e5d0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f15b01606b0 tx=0x7f15a402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 -- 172.21.15.148:0/4048921921 shutdown_connections 2024-04-25T09:58:44.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b00cee60 0x7f15b015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 --2- 172.21.15.148:0/4048921921 >> v2:172.21.15.188:3300/0 conn(0x7f15b006eb10 0x7f15b00ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 --2- 172.21.15.148:0/4048921921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b0144750 0x7f15b006e5d0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 -- 172.21.15.148:0/4048921921 >> 172.21.15.148:0/4048921921 conn(0x7f15b006abe0 msgr2=0x7f15b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 -- 172.21.15.148:0/4048921921 shutdown_connections 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.416+0000 7f15b6b22640 1 -- 172.21.15.148:0/4048921921 wait complete. 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.417+0000 7f15b6b22640 1 Processor -- start 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.417+0000 7f15b6b22640 1 -- start start 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b6b22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 0x7f15b0143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b6b22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b00cee60 0x7f15b01415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b6b22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b0144750 0x7f15b0141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b6b22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0145450 con 0x7f15b006eb10 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b6b22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b006f020 con 0x7f15b00cee60 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b6b22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b01455d0 con 0x7f15b0144750 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b48be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 0x7f15b0143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b48be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 0x7f15b0143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55770/0 (socket says 172.21.15.148:55770) 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.418+0000 7f15b48be640 1 -- 172.21.15.148:0/747494664 learned_addr learned my addr 172.21.15.148:0/747494664 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:44.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15affff640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b00cee60 0x7f15b01415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b50bf640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b0144750 0x7f15b0141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b48be640 1 -- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b0144750 msgr2=0x7f15b0141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b48be640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b0144750 0x7f15b0141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b48be640 1 -- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b00cee60 msgr2=0x7f15b01415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b48be640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b00cee60 0x7f15b01415b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b48be640 1 -- 172.21.15.148:0/747494664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b013cce0 con 0x7f15b006eb10 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15affff640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b00cee60 0x7f15b01415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b50bf640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b0144750 0x7f15b0141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b48be640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 0x7f15b0143760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f15a4000c00 tx=0x7f15a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4037040 con 0x7f15b006eb10 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4042070 con 0x7f15b006eb10 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4030bb0 con 0x7f15b006eb10 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b0142120 con 0x7f15b006eb10 2024-04-25T09:58:44.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.419+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b01426e0 con 0x7f15b006eb10 2024-04-25T09:58:44.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.420+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b0073990 con 0x7f15b006eb10 2024-04-25T09:58:44.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.421+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15a4047460 con 0x7f15b006eb10 2024-04-25T09:58:44.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.421+0000 7f15ad7fa640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f159406b4d0 0x7f159406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.421+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f15a40aad10 con 0x7f15b006eb10 2024-04-25T09:58:44.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.421+0000 7f15affff640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f159406b4d0 0x7f159406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.422+0000 7f15affff640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f159406b4d0 0x7f159406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f15b013ee40 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.425+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15a4045360 con 0x7f15b006eb10 2024-04-25T09:58:44.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.635+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15b00d0510 con 0x7f15b006eb10 2024-04-25T09:58:44.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.635+0000 7f15ad7fa640 1 -- 172.21.15.148:0/747494664 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f15a4079cb0 con 0x7f15b006eb10 2024-04-25T09:58:44.639 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:44.639 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2918106857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2918106857},{"type":"v1","addr":"172.21.15.188:6833","nonce":2918106857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5627,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3031206696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3031206696},{"type":"v1","addr":"172.21.15.188:6835","nonce":3031206696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5633,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3273375203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3273375203},{"type":"v1","addr":"172.21.15.148:6837","nonce":3273375203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7203,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/494402590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":494402590},{"type":"v1","addr":"172.21.15.148:6835","nonce":494402590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[]} 2024-04-25T09:58:44.640 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 42 2024-04-25T09:58:44.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.637+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f159406b4d0 msgr2=0x7f159406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.637+0000 7f15b6b22640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f159406b4d0 0x7f159406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f15b013ee40 tx=0x7f159c008040 comp rx=0 tx=0).stop 2024-04-25T09:58:44.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.637+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 msgr2=0x7f15b0143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.637+0000 7f15b6b22640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 0x7f15b0143760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f15a4000c00 tx=0x7f15a40046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 shutdown_connections 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f159406b4d0 0x7f159406d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b0144750 0x7f15b0141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b00cee60 0x7f15b01415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 --2- 172.21.15.148:0/747494664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b006eb10 0x7f15b0143760 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 >> 172.21.15.148:0/747494664 conn(0x7f15b006abe0 msgr2=0x7f15b013df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 shutdown_connections 2024-04-25T09:58:44.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.638+0000 7f15b6b22640 1 -- 172.21.15.148:0/747494664 wait complete. 2024-04-25T09:58:44.653 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:58:44.653 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:58:44.654 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:58:44.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:44.657 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:58:44.658 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:58:44.658 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:58:44.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:44.658 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:58:44.659 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:58:44.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:58:44.659 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:44.659 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:58:44.660 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:58:44.660 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:58:44.660 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:58:44.664 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:58:44.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:58:44.668 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:58:44.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:58:44.671 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:58:44.672 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:58:44.672 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:58:44.673 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:58:44.710 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:58:44.711 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:44.712 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:58:44.712 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:44.714 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:58:44.715 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:44.715 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:58:44.715 DEBUG:teuthology.parallel:result is None 2024-04-25T09:58:44.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:58:44.784 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T09:58:44.784 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T09:58:44.794 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T09:58:44.795 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T09:58:44.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.816+0000 7fa72af28640 1 Processor -- start 2024-04-25T09:58:44.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.816+0000 7fa72af28640 1 -- start start 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa72af28640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa72415a9f0 0x7fa7240c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa72af28640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7240cdbf0 0x7fa7240ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa72af28640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7240c6940 con 0x7fa7240cd520 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa72af28640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7240b0020 con 0x7fa7240cdbf0 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa72af28640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7240cb5a0 con 0x7fa72415a9f0 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa71bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7240cdbf0 0x7fa7240ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa71bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa7240cdbf0 0x7fa7240ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55800/0 (socket says 172.21.15.148:55800) 2024-04-25T09:58:44.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa71bfff640 1 -- 172.21.15.148:0/1564823763 learned_addr learned my addr 172.21.15.148:0/1564823763 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.817+0000 7fa728cc4640 1 --2- 172.21.15.148:0/1564823763 >> v2:172.21.15.188:3300/0 conn(0x7fa72415a9f0 0x7fa7240c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.818+0000 7fa71bfff640 1 -- 172.21.15.148:0/1564823763 >> v2:172.21.15.188:3300/0 conn(0x7fa72415a9f0 msgr2=0x7fa7240c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.818+0000 7fa71bfff640 1 --2- 172.21.15.148:0/1564823763 >> v2:172.21.15.188:3300/0 conn(0x7fa72415a9f0 0x7fa7240c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.818+0000 7fa71bfff640 1 -- 172.21.15.148:0/1564823763 >> v1:172.21.15.188:6789/0 conn(0x7fa7240cd520 legacy=0x7fa7240ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.818+0000 7fa71bfff640 1 -- 172.21.15.148:0/1564823763 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72415add0 con 0x7fa7240cdbf0 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.818+0000 7fa71bfff640 1 --2- 172.21.15.148:0/1564823763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cdbf0 0x7fa7240ccfe0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa7240c77e0 tx=0x7fa71402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1675611c0e9fbec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.820+0000 7fa71b7fe640 1 -- 172.21.15.148:0/1564823763 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714036020 con 0x7fa7240cdbf0 2024-04-25T09:58:44.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.820+0000 7fa71b7fe640 1 -- 172.21.15.148:0/1564823763 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa714032640 con 0x7fa7240cdbf0 2024-04-25T09:58:44.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.820+0000 7fa71b7fe640 1 -- 172.21.15.148:0/1564823763 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714032a10 con 0x7fa7240cdbf0 2024-04-25T09:58:44.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 -- 172.21.15.148:0/1564823763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cdbf0 msgr2=0x7fa7240ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 --2- 172.21.15.148:0/1564823763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cdbf0 0x7fa7240ccfe0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa7240c77e0 tx=0x7fa71402c940 comp rx=0 tx=0).stop 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 -- 172.21.15.148:0/1564823763 shutdown_connections 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 --2- 172.21.15.148:0/1564823763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cdbf0 0x7fa7240ccfe0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 --2- 172.21.15.148:0/1564823763 >> v2:172.21.15.188:3300/0 conn(0x7fa72415a9f0 0x7fa7240c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 -- 172.21.15.148:0/1564823763 >> 172.21.15.148:0/1564823763 conn(0x7fa7240b2390 msgr2=0x7fa7240b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 -- 172.21.15.148:0/1564823763 shutdown_connections 2024-04-25T09:58:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.821+0000 7fa72af28640 1 -- 172.21.15.148:0/1564823763 wait complete. 2024-04-25T09:58:44.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.822+0000 7fa72af28640 1 Processor -- start 2024-04-25T09:58:44.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.822+0000 7fa72af28640 1 -- start start 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa72af28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 0x7fa72414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa72af28640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7240cdbf0 0x7fa7241e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa72af28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72415a9f0 0x7fa7240cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa72af28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240cb230 con 0x7fa7240cd520 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa72af28640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240caf50 con 0x7fa72415a9f0 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa72af28640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240ce070 con 0x7fa7240cdbf0 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa728cc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 0x7fa72414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa728cc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 0x7fa72414c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55812/0 (socket says 172.21.15.148:55812) 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa728cc4640 1 -- 172.21.15.148:0/2544533088 learned_addr learned my addr 172.21.15.148:0/2544533088 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:44.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa71bfff640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7240cdbf0 0x7fa7241e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.823+0000 7fa7294c5640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72415a9f0 0x7fa7240cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa728cc4640 1 -- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7240cdbf0 msgr2=0x7fa7241e4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa728cc4640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7240cdbf0 0x7fa7241e4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa728cc4640 1 -- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72415a9f0 msgr2=0x7fa7240cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa728cc4640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72415a9f0 0x7fa7240cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa728cc4640 1 -- 172.21.15.148:0/2544533088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724157c80 con 0x7fa7240cd520 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa728cc4640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 0x7fa72414c380 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa72000e870 tx=0x7fa72000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720017070 con 0x7fa7240cd520 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72000c940 con 0x7fa7240cd520 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7240c9810 con 0x7fa7240cd520 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000cd30 con 0x7fa7240cd520 2024-04-25T09:58:44.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.824+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7240c9dd0 con 0x7fa7240cd520 2024-04-25T09:58:44.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.825+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72406cc80 con 0x7fa7240cd520 2024-04-25T09:58:44.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.825+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa72001a070 con 0x7fa7240cd520 2024-04-25T09:58:44.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.826+0000 7fa7197fa640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:44.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.826+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa72008a250 con 0x7fa7240cd520 2024-04-25T09:58:44.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.826+0000 7fa71bfff640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:44.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.826+0000 7fa71bfff640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa714037000 tx=0x7fa714008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:44.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:44.830+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa720058ba0 con 0x7fa7240cd520 2024-04-25T09:58:45.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.016+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa724073a00 con 0x7fa7240cd520 2024-04-25T09:58:45.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.830+0000 7fa7197fa640 1 -- 172.21.15.148:0/2544533088 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa72005cc40 con 0x7fa7240cd520 2024-04-25T09:58:45.834 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T09:58:45.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.832+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 msgr2=0x7fa71006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.832+0000 7fa72af28640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa714037000 tx=0x7fa714008040 comp rx=0 tx=0).stop 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.832+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 msgr2=0x7fa72414c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.832+0000 7fa72af28640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 0x7fa72414c380 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa72000e870 tx=0x7fa72000ed30 comp rx=0 tx=0).stop 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 shutdown_connections 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72415a9f0 0x7fa7240cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7240cdbf0 0x7fa7241e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 --2- 172.21.15.148:0/2544533088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240cd520 0x7fa72414c380 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 >> 172.21.15.148:0/2544533088 conn(0x7fa7240b2390 msgr2=0x7fa72406e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 shutdown_connections 2024-04-25T09:58:45.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.833+0000 7fa72af28640 1 -- 172.21.15.148:0/2544533088 wait complete. 2024-04-25T09:58:45.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:58:45.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.946+0000 7f378797e640 1 Processor -- start 2024-04-25T09:58:45.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.946+0000 7f378797e640 1 -- start start 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378797e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f378015b9c0 0x7f378015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378797e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f378006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378797e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3780072620 0x7f3780072a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378797e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378006b580 con 0x7f378015b9c0 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378797e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37800cfa60 con 0x7f3780072620 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378797e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37800cfd40 con 0x7f378006abe0 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378571a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f378015b9c0 0x7f378015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378571a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f378015b9c0 0x7f378015bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55826/0 (socket says 172.21.15.148:55826) 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f378571a640 1 -- 172.21.15.148:0/551502391 learned_addr learned my addr 172.21.15.148:0/551502391 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f3785f1b640 1 --2- 172.21.15.148:0/551502391 >> v2:172.21.15.188:3300/0 conn(0x7f3780072620 0x7f3780072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.947+0000 7f3784f19640 1 --2- 172.21.15.148:0/551502391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f378006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f378571a640 1 -- 172.21.15.148:0/551502391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 msgr2=0x7f378006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f378571a640 1 --2- 172.21.15.148:0/551502391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f378006b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f378571a640 1 -- 172.21.15.148:0/551502391 >> v2:172.21.15.188:3300/0 conn(0x7f3780072620 msgr2=0x7f3780072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f378571a640 1 --2- 172.21.15.148:0/551502391 >> v2:172.21.15.188:3300/0 conn(0x7f3780072620 0x7f3780072a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f378571a640 1 -- 172.21.15.148:0/551502391 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800d0020 con 0x7f378015b9c0 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f378571a640 1 --2- 172.21.15.148:0/551502391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378015b9c0 0x7f378015bda0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f37800b3390 tx=0x7f377c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10ac24dce8d2967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f3777fff640 1 -- 172.21.15.148:0/551502391 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c036070 con 0x7f378015b9c0 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f3777fff640 1 -- 172.21.15.148:0/551502391 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c031600 con 0x7f378015b9c0 2024-04-25T09:58:45.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.948+0000 7f3777fff640 1 -- 172.21.15.148:0/551502391 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c0319d0 con 0x7f378015b9c0 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 -- 172.21.15.148:0/551502391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378015b9c0 msgr2=0x7f378015bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 --2- 172.21.15.148:0/551502391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378015b9c0 0x7f378015bda0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f37800b3390 tx=0x7f377c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 -- 172.21.15.148:0/551502391 shutdown_connections 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 --2- 172.21.15.148:0/551502391 >> v2:172.21.15.188:3300/0 conn(0x7f3780072620 0x7f3780072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 --2- 172.21.15.148:0/551502391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f378006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 --2- 172.21.15.148:0/551502391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378015b9c0 0x7f378015bda0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 -- 172.21.15.148:0/551502391 >> 172.21.15.148:0/551502391 conn(0x7f37800b2370 msgr2=0x7f37800b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 -- 172.21.15.148:0/551502391 shutdown_connections 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.949+0000 7f378797e640 1 -- 172.21.15.148:0/551502391 wait complete. 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.950+0000 7f378797e640 1 Processor -- start 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.950+0000 7f378797e640 1 -- start start 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378797e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f37800b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378797e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 0x7f37800cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378797e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f378015b9c0 0x7f37800aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378797e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378015c2e0 con 0x7f3780072620 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378797e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37800cf710 con 0x7f378015b9c0 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378797e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3780072fd0 con 0x7f378006abe0 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 0x7f37800cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 0x7f37800cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55828/0 (socket says 172.21.15.148:55828) 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 -- 172.21.15.148:0/2715303426 learned_addr learned my addr 172.21.15.148:0/2715303426 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:45.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f378571a640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f37800b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3785f1b640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f378015b9c0 0x7f37800aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 -- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 msgr2=0x7f37800b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f37800b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 -- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f378015b9c0 msgr2=0x7f37800aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f378015b9c0 0x7f37800aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.951+0000 7f3784f19640 1 -- 172.21.15.148:0/2715303426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800b4c20 con 0x7f3780072620 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.952+0000 7f3784f19640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 0x7f37800cb0d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f377000d800 tx=0x7f377000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.952+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770018030 con 0x7f3780072620 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.952+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37700096a0 con 0x7f3780072620 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.952+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37800b4f00 con 0x7f3780072620 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.952+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770009870 con 0x7f3780072620 2024-04-25T09:58:45.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.952+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37800b54c0 con 0x7f3780072620 2024-04-25T09:58:45.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.953+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37801f38e0 con 0x7f3780072620 2024-04-25T09:58:45.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.953+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f377001b070 con 0x7f3780072620 2024-04-25T09:58:45.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.954+0000 7f3775ffb640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f376406b500 0x7f376406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:45.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.954+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f377008a120 con 0x7f3780072620 2024-04-25T09:58:45.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.954+0000 7f378571a640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f376406b500 0x7f376406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:45.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.954+0000 7f378571a640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f376406b500 0x7f376406d9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f377c033000 tx=0x7f377c009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:45.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:45.958+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37801f38e0 con 0x7f3780072620 2024-04-25T09:58:46.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.193+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3780073990 con 0x7f3780072620 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.193+0000 7f3775ffb640 1 -- 172.21.15.148:0/2715303426 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3770058b10 con 0x7f3780072620 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.197 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:58:46.198 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:58:46.198 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.198 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.198 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.198 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.198 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.199 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.200 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.201 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.202 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.203 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.204 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.205 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T09:58:46.206 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.207 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T09:58:46.208 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T09:58:46.209 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T09:58:46.210 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T09:58:46.211 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.212 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:46.212 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T09:58:46.212 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:58:46.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f376406b500 msgr2=0x7f376406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f376406b500 0x7f376406d9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f377c033000 tx=0x7f377c009000 comp rx=0 tx=0).stop 2024-04-25T09:58:46.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 msgr2=0x7f37800cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 0x7f37800cb0d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f377000d800 tx=0x7f377000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 shutdown_connections 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f376406b500 0x7f376406d9c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f378015b9c0 0x7f37800aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3780072620 0x7f37800cb0d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 --2- 172.21.15.148:0/2715303426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f378006abe0 0x7f37800b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 >> 172.21.15.148:0/2715303426 conn(0x7f37800b2370 msgr2=0x7f37800d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 shutdown_connections 2024-04-25T09:58:46.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.195+0000 7f378797e640 1 -- 172.21.15.148:0/2715303426 wait complete. 2024-04-25T09:58:46.215 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:46.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.308+0000 7f7ba4bb7640 1 Processor -- start 2024-04-25T09:58:46.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.308+0000 7f7ba4bb7640 1 -- start start 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7ba4bb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba00cfec0 0x7f7ba00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7ba4bb7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ba00ce3c0 0x7f7ba006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7ba4bb7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba0071480 con 0x7f7ba0154ee0 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7ba4bb7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba0072090 con 0x7f7ba00ce3c0 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7ba4bb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0071230 con 0x7f7ba00cfec0 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ba00ce3c0 0x7f7ba006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ba00ce3c0 0x7f7ba006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55844/0 (socket says 172.21.15.148:55844) 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 -- 172.21.15.148:0/3613605326 learned_addr learned my addr 172.21.15.148:0/3613605326 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9dd9b640 1 --2- 172.21.15.148:0/3613605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba00cfec0 0x7f7ba00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 -- 172.21.15.148:0/3613605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba00cfec0 msgr2=0x7f7ba00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 --2- 172.21.15.148:0/3613605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba00cfec0 0x7f7ba00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 -- 172.21.15.148:0/3613605326 >> v1:172.21.15.188:6789/0 conn(0x7f7ba0154ee0 legacy=0x7f7ba00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.309+0000 7f7b9ed9d640 1 -- 172.21.15.148:0/3613605326 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0071760 con 0x7f7ba00ce3c0 2024-04-25T09:58:46.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.310+0000 7f7b9ed9d640 1 --2- 172.21.15.148:0/3613605326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba006d6f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000bb10 tx=0x7f7b9002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed87b1851914253a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.310+0000 7f7b9d59a640 1 -- 172.21.15.148:0/3613605326 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90036020 con 0x7f7ba00ce3c0 2024-04-25T09:58:46.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.310+0000 7f7b9d59a640 1 -- 172.21.15.148:0/3613605326 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90032820 con 0x7f7ba00ce3c0 2024-04-25T09:58:46.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.310+0000 7f7b9d59a640 1 -- 172.21.15.148:0/3613605326 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90032bf0 con 0x7f7ba00ce3c0 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/3613605326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00ce3c0 msgr2=0x7f7ba006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/3613605326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba006d6f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000bb10 tx=0x7f7b9002c920 comp rx=0 tx=0).stop 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/3613605326 shutdown_connections 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/3613605326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba006d6f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/3613605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba00cfec0 0x7f7ba00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/3613605326 >> 172.21.15.148:0/3613605326 conn(0x7f7ba006abe0 msgr2=0x7f7ba006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/3613605326 shutdown_connections 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.311+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/3613605326 wait complete. 2024-04-25T09:58:46.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.312+0000 7f7ba4bb7640 1 Processor -- start 2024-04-25T09:58:46.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.312+0000 7f7ba4bb7640 1 -- start start 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7ba4bb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba00add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7ba4bb7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 0x7f7ba0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7ba4bb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba0154ee0 0x7f7ba0153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7ba4bb7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0071d20 con 0x7f7ba00cfec0 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7ba4bb7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00ce800 con 0x7f7ba00ce3c0 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7ba4bb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01552c0 con 0x7f7ba0154ee0 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 0x7f7ba0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 0x7f7ba0159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55852/0 (socket says 172.21.15.148:55852) 2024-04-25T09:58:46.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 -- 172.21.15.148:0/700637026 learned_addr learned my addr 172.21.15.148:0/700637026 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9ed9d640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba0154ee0 0x7f7ba0153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9e59c640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba00add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 -- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba0154ee0 msgr2=0x7f7ba0153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba0154ee0 0x7f7ba0153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 -- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba00ce3c0 msgr2=0x7f7ba00add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba00add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 -- 172.21.15.148:0/700637026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba01f7240 con 0x7f7ba00cfec0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9ed9d640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba0154ee0 0x7f7ba0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9e59c640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba00add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.313+0000 7f7b9dd9b640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 0x7f7ba0159c90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400d800 tx=0x7f7b9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.314+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400b6c0 con 0x7f7ba00cfec0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.314+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94019070 con 0x7f7ba00cfec0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.314+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba01f7520 con 0x7f7ba00cfec0 2024-04-25T09:58:46.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.314+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400ba80 con 0x7f7ba00cfec0 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.314+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01f7ae0 con 0x7f7ba00cfec0 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.315+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba0153f50 con 0x7f7ba00cfec0 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.315+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b9400bc20 con 0x7f7ba00cfec0 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.315+0000 7f7b8effd640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b8006b4d0 0x7f7b8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.316+0000 7f7b9e59c640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b8006b4d0 0x7f7b8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.316+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7b9408a460 con 0x7f7ba00cfec0 2024-04-25T09:58:46.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.316+0000 7f7b9e59c640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b8006b4d0 0x7f7b8006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7b98001cd0 tx=0x7f7b98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.320+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b94059530 con 0x7f7ba00cfec0 2024-04-25T09:58:46.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.493+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ba0073990 con 0x7f7ba00cfec0 2024-04-25T09:58:46.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.494+0000 7f7b8effd640 1 -- 172.21.15.148:0/700637026 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13213 (secure 0 0 0) 0x7f7b94058e30 con 0x7f7ba00cfec0 2024-04-25T09:58:46.497 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:46.497 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":79,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:45.822924+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b8006b4d0 msgr2=0x7f7b8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b8006b4d0 0x7f7b8006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7b98001cd0 tx=0x7f7b98009040 comp rx=0 tx=0).stop 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 msgr2=0x7f7ba0159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 0x7f7ba0159c90 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400d800 tx=0x7f7b9400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 shutdown_connections 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b8006b4d0 0x7f7b8006d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ba0154ee0 0x7f7ba0153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba00cfec0 0x7f7ba0159c90 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 --2- 172.21.15.148:0/700637026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba00ce3c0 0x7f7ba00add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 >> 172.21.15.148:0/700637026 conn(0x7f7ba006abe0 msgr2=0x7f7ba015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 shutdown_connections 2024-04-25T09:58:46.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.496+0000 7f7ba4bb7640 1 -- 172.21.15.148:0/700637026 wait complete. 2024-04-25T09:58:46.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:58:46.511 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:58:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.607+0000 7f4a7e06f640 1 Processor -- start 2024-04-25T09:58:46.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.607+0000 7f4a7e06f640 1 -- start start 2024-04-25T09:58:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a7e06f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 0x7f4a780d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a7e06f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a780d1b80 0x7f4a7806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a7e06f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a780d2020 con 0x7f4a7806c870 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a7e06f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a78158610 con 0x7f4a780d1b80 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a7e06f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7806bea0 con 0x7f4a78158230 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 0x7f4a780d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a76ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a780d1b80 0x7f4a7806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 0x7f4a780d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55224/0 (socket says 172.21.15.148:55224) 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a777fe640 1 -- 172.21.15.148:0/3834487792 learned_addr learned my addr 172.21.15.148:0/3834487792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.608+0000 7f4a77fff640 1 --1- 172.21.15.148:0/3834487792 >> v1:172.21.15.188:6789/0 conn(0x7f4a7806c870 0x7f4a780b4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:52444/0 (socket says 172.21.15.148:52444) 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.609+0000 7f4a777fe640 1 -- 172.21.15.148:0/3834487792 >> v2:172.21.15.188:3300/0 conn(0x7f4a780d1b80 msgr2=0x7f4a7806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.609+0000 7f4a777fe640 1 --2- 172.21.15.148:0/3834487792 >> v2:172.21.15.188:3300/0 conn(0x7f4a780d1b80 0x7f4a7806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.609+0000 7f4a777fe640 1 -- 172.21.15.148:0/3834487792 >> v1:172.21.15.188:6789/0 conn(0x7f4a7806c870 legacy=0x7f4a780b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.609+0000 7f4a777fe640 1 -- 172.21.15.148:0/3834487792 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780d2300 con 0x7f4a78158230 2024-04-25T09:58:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a777fe640 1 --2- 172.21.15.148:0/3834487792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 0x7f4a780d1640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009870 tx=0x7f4a6802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcf7004f84aab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a767fc640 1 -- 172.21.15.148:0/3834487792 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68037070 con 0x7f4a78158230 2024-04-25T09:58:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a767fc640 1 -- 172.21.15.148:0/3834487792 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68032440 con 0x7f4a78158230 2024-04-25T09:58:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a767fc640 1 -- 172.21.15.148:0/3834487792 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68032810 con 0x7f4a78158230 2024-04-25T09:58:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 -- 172.21.15.148:0/3834487792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 msgr2=0x7f4a780d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/3834487792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 0x7f4a780d1640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009870 tx=0x7f4a6802a270 comp rx=0 tx=0).stop 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 -- 172.21.15.148:0/3834487792 shutdown_connections 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/3834487792 >> v2:172.21.15.188:3300/0 conn(0x7f4a780d1b80 0x7f4a7806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/3834487792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a78158230 0x7f4a780d1640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 -- 172.21.15.148:0/3834487792 >> 172.21.15.148:0/3834487792 conn(0x7f4a78061600 msgr2=0x7f4a780619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 -- 172.21.15.148:0/3834487792 shutdown_connections 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.610+0000 7f4a7e06f640 1 -- 172.21.15.148:0/3834487792 wait complete. 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.611+0000 7f4a7e06f640 1 Processor -- start 2024-04-25T09:58:46.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 -- start start 2024-04-25T09:58:46.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a7806c870 0x7f4a78149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780d1b80 0x7f4a78158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 0x7f4a781568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7815e320 con 0x7f4a78158230 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780d25e0 con 0x7f4a7806c870 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a7e06f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7815e4a0 con 0x7f4a780d1b80 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.612+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 0x7f4a781568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 0x7f4a781568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55858/0 (socket says 172.21.15.148:55858) 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 -- 172.21.15.148:0/1083749673 learned_addr learned my addr 172.21.15.148:0/1083749673 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a777fe640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a7806c870 0x7f4a78149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a76ffd640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780d1b80 0x7f4a78158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 -- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780d1b80 msgr2=0x7f4a78158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780d1b80 0x7f4a78158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 -- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a7806c870 msgr2=0x7f4a78149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a7806c870 0x7f4a78149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 -- 172.21.15.148:0/1083749673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78158cd0 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a77fff640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 0x7f4a781568e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c780 tx=0x7f4a7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70017070 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7000a640 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7815b0b0 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000aa30 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.613+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7815b670 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.614+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7806de50 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.615+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a7001a070 con 0x7f4a78158230 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.615+0000 7f4a5ffff640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6406b4d0 0x7f4a6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.615+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4a700894c0 con 0x7f4a78158230 2024-04-25T09:58:46.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.615+0000 7f4a777fe640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6406b4d0 0x7f4a6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.615+0000 7f4a777fe640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6406b4d0 0x7f4a6406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a68034000 tx=0x7f4a6802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.619+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a78158cd0 con 0x7f4a78158230 2024-04-25T09:58:46.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.789+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a78073a00 con 0x7f4a78158230 2024-04-25T09:58:46.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.790+0000 7f4a5ffff640 1 -- 172.21.15.148:0/1083749673 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a70057f40 con 0x7f4a78158230 2024-04-25T09:58:46.793 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6406b4d0 msgr2=0x7f4a6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6406b4d0 0x7f4a6406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a68034000 tx=0x7f4a6802e000 comp rx=0 tx=0).stop 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 msgr2=0x7f4a781568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 0x7f4a781568e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c780 tx=0x7f4a7000cc40 comp rx=0 tx=0).stop 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 shutdown_connections 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a6406b4d0 0x7f4a6406d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a78158230 0x7f4a781568e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780d1b80 0x7f4a78158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 --2- 172.21.15.148:0/1083749673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a7806c870 0x7f4a78149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 >> 172.21.15.148:0/1083749673 conn(0x7f4a78061600 msgr2=0x7f4a780ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 shutdown_connections 2024-04-25T09:58:46.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.792+0000 7f4a7e06f640 1 -- 172.21.15.148:0/1083749673 wait complete. 2024-04-25T09:58:46.807 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:58:46.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.901+0000 7f85867d5640 1 Processor -- start 2024-04-25T09:58:46.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.901+0000 7f85867d5640 1 -- start start 2024-04-25T09:58:46.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f85867d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f858015b980 0x7f85800cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f85867d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 0x7f85800cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f85867d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858015f710 con 0x7f85800b2d40 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f85867d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858015f9b0 con 0x7f858015b980 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f85867d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858015fc90 con 0x7f858006b5f0 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 0x7f85800cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f857ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f858015b980 0x7f85800cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 0x7f85800cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55246/0 (socket says 172.21.15.148:55246) 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.902+0000 7f857f7fe640 1 -- 172.21.15.148:0/4199976237 learned_addr learned my addr 172.21.15.148:0/4199976237 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.903+0000 7f857f7fe640 1 -- 172.21.15.148:0/4199976237 >> v2:172.21.15.188:3300/0 conn(0x7f858015b980 msgr2=0x7f85800cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.903+0000 7f857f7fe640 1 --2- 172.21.15.148:0/4199976237 >> v2:172.21.15.188:3300/0 conn(0x7f858015b980 0x7f85800cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.903+0000 7f857f7fe640 1 -- 172.21.15.148:0/4199976237 >> v1:172.21.15.188:6789/0 conn(0x7f85800b2d40 legacy=0x7f85800ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.903+0000 7f857f7fe640 1 -- 172.21.15.148:0/4199976237 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858015ff70 con 0x7f858006b5f0 2024-04-25T09:58:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.903+0000 7f857f7fe640 1 --2- 172.21.15.148:0/4199976237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 0x7f85800cf8a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8574009870 tx=0x7f857402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e32e82527846719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f857effd640 1 -- 172.21.15.148:0/4199976237 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574037070 con 0x7f858006b5f0 2024-04-25T09:58:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f857effd640 1 -- 172.21.15.148:0/4199976237 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8574032440 con 0x7f858006b5f0 2024-04-25T09:58:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f857effd640 1 -- 172.21.15.148:0/4199976237 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574032810 con 0x7f858006b5f0 2024-04-25T09:58:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 -- 172.21.15.148:0/4199976237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 msgr2=0x7f85800cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 --2- 172.21.15.148:0/4199976237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 0x7f85800cf8a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8574009870 tx=0x7f857402a270 comp rx=0 tx=0).stop 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 -- 172.21.15.148:0/4199976237 shutdown_connections 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 --2- 172.21.15.148:0/4199976237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006b5f0 0x7f85800cf8a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 --2- 172.21.15.148:0/4199976237 >> v2:172.21.15.188:3300/0 conn(0x7f858015b980 0x7f85800cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 -- 172.21.15.148:0/4199976237 >> 172.21.15.148:0/4199976237 conn(0x7f85800b2220 msgr2=0x7f85800b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 -- 172.21.15.148:0/4199976237 shutdown_connections 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.904+0000 7f85867d5640 1 -- 172.21.15.148:0/4199976237 wait complete. 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.905+0000 7f85867d5640 1 Processor -- start 2024-04-25T09:58:46.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 -- start start 2024-04-25T09:58:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858006b5f0 0x7f858015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 0x7f85800c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015b980 0x7f85800ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800cf550 con 0x7f85800b2d40 2024-04-25T09:58:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858015bd60 con 0x7f858006b5f0 2024-04-25T09:58:46.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f85867d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85800cfde0 con 0x7f858015b980 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 0x7f85800c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 0x7f85800c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55862/0 (socket says 172.21.15.148:55862) 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.906+0000 7f857f7fe640 1 -- 172.21.15.148:0/1605130105 learned_addr learned my addr 172.21.15.148:0/1605130105 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857ffff640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858006b5f0 0x7f858015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f8584d72640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015b980 0x7f85800ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857f7fe640 1 -- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015b980 msgr2=0x7f85800ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857f7fe640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015b980 0x7f85800ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857f7fe640 1 -- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858006b5f0 msgr2=0x7f858015a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:46.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857f7fe640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858006b5f0 0x7f858015a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857f7fe640 1 -- 172.21.15.148:0/1605130105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800c81d0 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857ffff640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858006b5f0 0x7f858015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857f7fe640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 0x7f85800c9df0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8574000c00 tx=0x7f85740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574037040 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8574042070 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85800c84b0 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574030860 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.907+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85800c8a70 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.908+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8580073990 con 0x7f85800b2d40 2024-04-25T09:58:46.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.909+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8574030a00 con 0x7f85800b2d40 2024-04-25T09:58:46.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.909+0000 7f857cff9640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856c06b4d0 0x7f856c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:46.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.909+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f85740aa860 con 0x7f85800b2d40 2024-04-25T09:58:46.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.909+0000 7f857ffff640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856c06b4d0 0x7f856c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:46.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.910+0000 7f857ffff640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856c06b4d0 0x7f856c06d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8570009af0 tx=0x7f8570008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:46.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:46.913+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8574045360 con 0x7f85800b2d40 2024-04-25T09:58:47.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.086+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f85800b6670 con 0x7f85800b2d40 2024-04-25T09:58:47.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.086+0000 7f857cff9640 1 -- 172.21.15.148:0/1605130105 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8574079930 con 0x7f85800b2d40 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:58:47.091 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T09:58:47.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856c06b4d0 msgr2=0x7f856c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856c06b4d0 0x7f856c06d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8570009af0 tx=0x7f8570008040 comp rx=0 tx=0).stop 2024-04-25T09:58:47.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 msgr2=0x7f85800c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 0x7f85800c9df0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8574000c00 tx=0x7f85740046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 shutdown_connections 2024-04-25T09:58:47.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856c06b4d0 0x7f856c06d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858015b980 0x7f85800ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800b2d40 0x7f85800c9df0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 --2- 172.21.15.148:0/1605130105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858006b5f0 0x7f858015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 >> 172.21.15.148:0/1605130105 conn(0x7f85800b2220 msgr2=0x7f85800b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:47.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 shutdown_connections 2024-04-25T09:58:47.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.088+0000 7f85867d5640 1 -- 172.21.15.148:0/1605130105 wait complete. 2024-04-25T09:58:47.103 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:58:47.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:58:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.201+0000 7f391a3f4640 1 Processor -- start 2024-04-25T09:58:47.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.201+0000 7f391a3f4640 1 -- start start 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f391a3f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3914070dc0 0x7f391406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f391a3f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f391415deb0 0x7f391415e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f391a3f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140d1770 con 0x7f39140711a0 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f391a3f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39140b0eb0 con 0x7f3914070dc0 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f391a3f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39141622b0 con 0x7f391415deb0 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3913fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3914070dc0 0x7f391406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f39137fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f39140711a0 0x7f39140b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39582/0 (socket says 172.21.15.148:39582) 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f39137fe640 1 -- 172.21.15.148:0/2251256357 learned_addr learned my addr 172.21.15.148:0/2251256357 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3918991640 1 --2- 172.21.15.148:0/2251256357 >> v2:172.21.15.188:3300/0 conn(0x7f391415deb0 0x7f391415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3913fff640 1 -- 172.21.15.148:0/2251256357 >> v2:172.21.15.188:3300/0 conn(0x7f391415deb0 msgr2=0x7f391415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3913fff640 1 --2- 172.21.15.148:0/2251256357 >> v2:172.21.15.188:3300/0 conn(0x7f391415deb0 0x7f391415e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3913fff640 1 -- 172.21.15.148:0/2251256357 >> v1:172.21.15.148:6789/0 conn(0x7f39140711a0 legacy=0x7f39140b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3913fff640 1 -- 172.21.15.148:0/2251256357 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914162740 con 0x7f3914070dc0 2024-04-25T09:58:47.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.202+0000 7f3912ffd640 1 -- 172.21.15.148:0/2251256357 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409765365 0 0) 0x7f39140d1770 con 0x7f39140711a0 2024-04-25T09:58:47.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.203+0000 7f3913fff640 1 --2- 172.21.15.148:0/2251256357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3914070dc0 0x7f391406ccd0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3914070570 tx=0x7f390402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ce026942c96a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.203+0000 7f3912ffd640 1 -- 172.21.15.148:0/2251256357 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904037070 con 0x7f3914070dc0 2024-04-25T09:58:47.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.203+0000 7f3912ffd640 1 -- 172.21.15.148:0/2251256357 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390402add0 con 0x7f3914070dc0 2024-04-25T09:58:47.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.203+0000 7f3912ffd640 1 -- 172.21.15.148:0/2251256357 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904032660 con 0x7f3914070dc0 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 -- 172.21.15.148:0/2251256357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3914070dc0 msgr2=0x7f391406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 --2- 172.21.15.148:0/2251256357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3914070dc0 0x7f391406ccd0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3914070570 tx=0x7f390402a080 comp rx=0 tx=0).stop 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 -- 172.21.15.148:0/2251256357 shutdown_connections 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 --2- 172.21.15.148:0/2251256357 >> v2:172.21.15.188:3300/0 conn(0x7f391415deb0 0x7f391415e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 --2- 172.21.15.148:0/2251256357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3914070dc0 0x7f391406ccd0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 -- 172.21.15.148:0/2251256357 >> 172.21.15.148:0/2251256357 conn(0x7f39140b3530 msgr2=0x7f39140b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:47.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 -- 172.21.15.148:0/2251256357 shutdown_connections 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.204+0000 7f391a3f4640 1 -- 172.21.15.148:0/2251256357 wait complete. 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.205+0000 7f391a3f4640 1 Processor -- start 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.205+0000 7f391a3f4640 1 -- start start 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3914070dc0 0x7f39141f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140711a0 0x7f3914160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 0x7f3914160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914161c60 con 0x7f391415deb0 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39140b10f0 con 0x7f39140711a0 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3914161f40 con 0x7f3914070dc0 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 0x7f3914160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 0x7f3914160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55876/0 (socket says 172.21.15.148:55876) 2024-04-25T09:58:47.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 -- 172.21.15.148:0/1375510338 learned_addr learned my addr 172.21.15.148:0/1375510338 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 -- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3914070dc0 msgr2=0x7f39141f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3913fff640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3914070dc0 0x7f39141f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f39137fe640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140711a0 0x7f3914160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3914070dc0 0x7f39141f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 -- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140711a0 msgr2=0x7f3914160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140711a0 0x7f3914160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 -- 172.21.15.148:0/1375510338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39141f4620 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3918991640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 0x7f3914160890 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f390c007e90 tx=0x7f390c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c014420 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c019070 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c014800 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.206+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391415ec90 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.207+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391415f250 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.208+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140cd510 con 0x7f391415deb0 2024-04-25T09:58:47.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.208+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f390c0149a0 con 0x7f391415deb0 2024-04-25T09:58:47.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.208+0000 7f3910ff9640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.208+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f390c08a7e0 con 0x7f391415deb0 2024-04-25T09:58:47.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.208+0000 7f3913fff640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.209+0000 7f3913fff640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f390006b4d0 0x7f390006d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3904034000 tx=0x7f390402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.212+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39141f4620 con 0x7f391415deb0 2024-04-25T09:58:47.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.382+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3914153280 con 0x7f391415deb0 2024-04-25T09:58:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.838+0000 7f3910ff9640 1 -- 172.21.15.148:0/1375510338 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f390c0598b0 con 0x7f391415deb0 2024-04-25T09:58:47.842 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T09:58:47.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f390006b4d0 msgr2=0x7f390006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f390006b4d0 0x7f390006d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3904034000 tx=0x7f390402e000 comp rx=0 tx=0).stop 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 msgr2=0x7f3914160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 0x7f3914160890 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f390c007e90 tx=0x7f390c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 shutdown_connections 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f390006b4d0 0x7f390006d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f391415deb0 0x7f3914160890 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140711a0 0x7f3914160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 --2- 172.21.15.148:0/1375510338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3914070dc0 0x7f39141f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 >> 172.21.15.148:0/1375510338 conn(0x7f39140b3530 msgr2=0x7f391415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 shutdown_connections 2024-04-25T09:58:47.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.841+0000 7f391a3f4640 1 -- 172.21.15.148:0/1375510338 wait complete. 2024-04-25T09:58:47.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.959+0000 7f1ef86ea640 1 Processor -- start 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.959+0000 7f1ef86ea640 1 -- start start 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.960+0000 7f1ef86ea640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1ef015de80 0x7f1ef00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.960+0000 7f1ef86ea640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ef00cefb0 0x7f1ef00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.960+0000 7f1ef86ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef00afc80 con 0x7f1ef00cf980 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.960+0000 7f1ef86ea640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00ce4d0 con 0x7f1ef015de80 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.960+0000 7f1ef86ea640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef00ce7b0 con 0x7f1ef00cefb0 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1ef015de80 0x7f1ef00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1ef015de80 0x7f1ef00cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55890/0 (socket says 172.21.15.148:55890) 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6c87640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1ef00cf980 0x7f1ef006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39584/0 (socket says 172.21.15.148:39584) 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 -- 172.21.15.148:0/3070180056 learned_addr learned my addr 172.21.15.148:0/3070180056 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:47.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef5c85640 1 --2- 172.21.15.148:0/3070180056 >> v2:172.21.15.188:3300/0 conn(0x7f1ef00cefb0 0x7f1ef00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 -- 172.21.15.148:0/3070180056 >> v2:172.21.15.188:3300/0 conn(0x7f1ef00cefb0 msgr2=0x7f1ef00cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 --2- 172.21.15.148:0/3070180056 >> v2:172.21.15.188:3300/0 conn(0x7f1ef00cefb0 0x7f1ef00cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 -- 172.21.15.148:0/3070180056 >> v1:172.21.15.148:6789/0 conn(0x7f1ef00cf980 legacy=0x7f1ef006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.961+0000 7f1ef6486640 1 -- 172.21.15.148:0/3070180056 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00cea90 con 0x7f1ef015de80 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef6486640 1 --2- 172.21.15.148:0/3070180056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef015de80 0x7f1ef00cadb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0062500 tx=0x7f1eec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2960de5707c777e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef5484640 1 -- 172.21.15.148:0/3070180056 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec036070 con 0x7f1ef015de80 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef5484640 1 -- 172.21.15.148:0/3070180056 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec031600 con 0x7f1ef015de80 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef5484640 1 -- 172.21.15.148:0/3070180056 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0319d0 con 0x7f1ef015de80 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 -- 172.21.15.148:0/3070180056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef015de80 msgr2=0x7f1ef00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/3070180056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef015de80 0x7f1ef00cadb0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0062500 tx=0x7f1eec02a730 comp rx=0 tx=0).stop 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 -- 172.21.15.148:0/3070180056 shutdown_connections 2024-04-25T09:58:47.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/3070180056 >> v2:172.21.15.188:3300/0 conn(0x7f1ef00cefb0 0x7f1ef00cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/3070180056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef015de80 0x7f1ef00cadb0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 -- 172.21.15.148:0/3070180056 >> 172.21.15.148:0/3070180056 conn(0x7f1ef00b2390 msgr2=0x7f1ef00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 -- 172.21.15.148:0/3070180056 shutdown_connections 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.962+0000 7f1ef86ea640 1 -- 172.21.15.148:0/3070180056 wait complete. 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 Processor -- start 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 -- start start 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 0x7f1ef015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef00cf980 0x7f1ef015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef015de80 0x7f1ef015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b3030 con 0x7f1ef00cefb0 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b2eb0 con 0x7f1ef015de80 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef86ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00c5600 con 0x7f1ef00cf980 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef6486640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 0x7f1ef015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef6486640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 0x7f1ef015c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55900/0 (socket says 172.21.15.148:55900) 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.964+0000 7f1ef6486640 1 -- 172.21.15.148:0/260721495 learned_addr learned my addr 172.21.15.148:0/260721495 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef6486640 1 -- 172.21.15.148:0/260721495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef00cf980 msgr2=0x7f1ef015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef6486640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef00cf980 0x7f1ef015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef6486640 1 -- 172.21.15.148:0/260721495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef015de80 msgr2=0x7f1ef015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef6486640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef015de80 0x7f1ef015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef6486640 1 -- 172.21.15.148:0/260721495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00cb490 con 0x7f1ef00cefb0 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef6486640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 0x7f1ef015c400 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1eec034040 tx=0x7f1eec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec036050 con 0x7f1ef00cefb0 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec042070 con 0x7f1ef00cefb0 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00caed0 con 0x7f1ef00cefb0 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec009510 con 0x7f1ef00cefb0 2024-04-25T09:58:47.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.965+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef00c7c00 con 0x7f1ef00cefb0 2024-04-25T09:58:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.966+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef0073a00 con 0x7f1ef00cefb0 2024-04-25T09:58:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.967+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1eec007c50 con 0x7f1ef00cefb0 2024-04-25T09:58:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.967+0000 7f1ee6ffd640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ed406b4d0 0x7f1ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.967+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1eec0ab0f0 con 0x7f1ef00cefb0 2024-04-25T09:58:47.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.967+0000 7f1ef5c85640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ed406b4d0 0x7f1ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:47.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.967+0000 7f1ef5c85640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ed406b4d0 0x7f1ed406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006fd0 tx=0x7f1ee0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:47.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:47.972+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ef00cb490 con 0x7f1ef00cefb0 2024-04-25T09:58:48.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.148+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1ef006fa80 con 0x7f1ef00cefb0 2024-04-25T09:58:48.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.841+0000 7f1ee6ffd640 1 -- 172.21.15.148:0/260721495 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1eec045360 con 0x7f1ef00cefb0 2024-04-25T09:58:48.845 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T09:58:48.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ed406b4d0 msgr2=0x7f1ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ed406b4d0 0x7f1ed406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006fd0 tx=0x7f1ee0008040 comp rx=0 tx=0).stop 2024-04-25T09:58:48.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 msgr2=0x7f1ef015c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 0x7f1ef015c400 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1eec034040 tx=0x7f1eec004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:48.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 shutdown_connections 2024-04-25T09:58:48.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ed406b4d0 0x7f1ed406d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ef015de80 0x7f1ef015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ef00cf980 0x7f1ef015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 --2- 172.21.15.148:0/260721495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ef00cefb0 0x7f1ef015c400 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 >> 172.21.15.148:0/260721495 conn(0x7f1ef00b2390 msgr2=0x7f1ef00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:48.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 shutdown_connections 2024-04-25T09:58:48.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.844+0000 7f1ef86ea640 1 -- 172.21.15.148:0/260721495 wait complete. 2024-04-25T09:58:48.874 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:58:48.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.973+0000 7f1313476640 1 Processor -- start 2024-04-25T09:58:48.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.973+0000 7f1313476640 1 -- start start 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1313476640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1313476640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f130c0cdb20 0x7f130c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1313476640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c072280 con 0x7f130c15b270 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1313476640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0ce4c0 con 0x7f130c0cdb20 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1313476640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c071270 con 0x7f130c0cc4f0 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1310a11640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f130c0cdb20 0x7f130c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1311a13640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f130c15b270 0x7f130c15b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39586/0 (socket says 172.21.15.148:39586) 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1311a13640 1 -- 172.21.15.148:0/3803371194 learned_addr learned my addr 172.21.15.148:0/3803371194 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1311212640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1310a11640 1 -- 172.21.15.148:0/3803371194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 msgr2=0x7f130c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1303fff640 1 -- 172.21.15.148:0/3803371194 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507666654 0 0) 0x7f130c072280 con 0x7f130c15b270 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1310a11640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1310a11640 1 -- 172.21.15.148:0/3803371194 >> v1:172.21.15.148:6789/0 conn(0x7f130c15b270 legacy=0x7f130c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1310a11640 1 -- 172.21.15.148:0/3803371194 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0714c0 con 0x7f130c0cdb20 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.974+0000 7f1311212640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1310a11640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c0cdb20 0x7f130c0cdf80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097a0 tx=0x7f12fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3630cbb4a258fa8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1303fff640 1 -- 172.21.15.148:0/3803371194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc037070 con 0x7f130c0cdb20 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1303fff640 1 -- 172.21.15.148:0/3803371194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc032440 con 0x7f130c0cdb20 2024-04-25T09:58:48.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1303fff640 1 -- 172.21.15.148:0/3803371194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc032810 con 0x7f130c0cdb20 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1313476640 1 -- 172.21.15.148:0/3803371194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c0cdb20 msgr2=0x7f130c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1313476640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c0cdb20 0x7f130c0cdf80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0097a0 tx=0x7f12fc02a180 comp rx=0 tx=0).stop 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1313476640 1 -- 172.21.15.148:0/3803371194 shutdown_connections 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1313476640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c0cdb20 0x7f130c0cdf80 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.975+0000 7f1313476640 1 --2- 172.21.15.148:0/3803371194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.976+0000 7f1313476640 1 -- 172.21.15.148:0/3803371194 >> 172.21.15.148:0/3803371194 conn(0x7f130c06abe0 msgr2=0x7f130c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.976+0000 7f1313476640 1 -- 172.21.15.148:0/3803371194 shutdown_connections 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.976+0000 7f1313476640 1 -- 172.21.15.148:0/3803371194 wait complete. 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 Processor -- start 2024-04-25T09:58:48.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 -- start start 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c0cdb20 0x7f130c1f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 0x7f130c1f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c071a80 con 0x7f130c15b270 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c15bd90 con 0x7f130c0cdb20 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1313476640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c071d60 con 0x7f130c0cc4f0 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.977+0000 7f1311a13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 0x7f130c1f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 0x7f130c1f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55918/0 (socket says 172.21.15.148:55918) 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 -- 172.21.15.148:0/2438678986 learned_addr learned my addr 172.21.15.148:0/2438678986 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1310a11640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c0cdb20 0x7f130c1f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311212640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 -- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 msgr2=0x7f130c148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 -- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c0cdb20 msgr2=0x7f130c1f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c0cdb20 0x7f130c1f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 -- 172.21.15.148:0/2438678986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c156140 con 0x7f130c15b270 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311212640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1311a13640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 0x7f130c1f7680 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f130400c7a0 tx=0x7f130400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304018070 con 0x7f130c15b270 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400a640 con 0x7f130c15b270 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400aab0 con 0x7f130c15b270 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c156420 con 0x7f130c15b270 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.978+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c1569e0 con 0x7f130c15b270 2024-04-25T09:58:48.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.979+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c073a00 con 0x7f130c15b270 2024-04-25T09:58:48.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.980+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f130401b070 con 0x7f130c15b270 2024-04-25T09:58:48.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.980+0000 7f1301ffb640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f12f006b4d0 0x7f12f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:48.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.980+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f1304089940 con 0x7f130c15b270 2024-04-25T09:58:48.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.980+0000 7f1311212640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f12f006b4d0 0x7f12f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:48.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.980+0000 7f1311212640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f12f006b4d0 0x7f12f006d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f130800a990 tx=0x7f1308008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:48.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:48.984+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1304057fa0 con 0x7f130c15b270 2024-04-25T09:58:49.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.191+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f130c06d3a0 con 0x7f130c15b270 2024-04-25T09:58:49.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.867+0000 7f1301ffb640 1 -- 172.21.15.148:0/2438678986 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f130405c040 con 0x7f130c15b270 2024-04-25T09:58:49.875 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f12f006b4d0 msgr2=0x7f12f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f12f006b4d0 0x7f12f006d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f130800a990 tx=0x7f1308008040 comp rx=0 tx=0).stop 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 msgr2=0x7f130c1f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 0x7f130c1f7680 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f130400c7a0 tx=0x7f130400cc60 comp rx=0 tx=0).stop 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 shutdown_connections 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f12f006b4d0 0x7f12f006d990 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f130800a990 tx=0x7f1308008040 comp rx=0 tx=0).stop 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f130c15b270 0x7f130c1f7680 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f130c0cdb20 0x7f130c1f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 --2- 172.21.15.148:0/2438678986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f130c0cc4f0 0x7f130c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:49.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.877+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 >> 172.21.15.148:0/2438678986 conn(0x7f130c06abe0 msgr2=0x7f130c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:49.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.882+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 shutdown_connections 2024-04-25T09:58:49.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.884+0000 7f1313476640 1 -- 172.21.15.148:0/2438678986 wait complete. 2024-04-25T09:58:49.901 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:50.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.996+0000 7f7cc446f640 1 Processor -- start 2024-04-25T09:58:50.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.996+0000 7f7cc446f640 1 -- start start 2024-04-25T09:58:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc446f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7cbc154ee0 0x7f7cbc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc446f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc071480 con 0x7f7cbc0ce590 2024-04-25T09:58:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc446f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cbc071760 con 0x7f7cbc0cfcf0 2024-04-25T09:58:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc446f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cbc072090 con 0x7f7cbc154ee0 2024-04-25T09:58:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc2a0c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7cbc0ce590 0x7f7cbc06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39594/0 (socket says 172.21.15.148:39594) 2024-04-25T09:58:50.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc2a0c640 1 -- 172.21.15.148:0/3698066974 learned_addr learned my addr 172.21.15.148:0/3698066974 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc220b640 1 --2- 172.21.15.148:0/3698066974 >> v2:172.21.15.188:3300/0 conn(0x7f7cbc154ee0 0x7f7cbc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450588613 0 0) 0x7f7cbc071480 con 0x7f7cbc0ce590 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ca8003520 con 0x7f7cbc0ce590 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3424783286 0 0) 0x7f7ca8003520 con 0x7f7cbc0ce590 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 >> v2:172.21.15.188:3300/0 conn(0x7f7cbc154ee0 msgr2=0x7f7cbc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 --2- 172.21.15.148:0/3698066974 >> v2:172.21.15.188:3300/0 conn(0x7f7cbc154ee0 0x7f7cbc0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 >> v1:172.21.15.188:6789/0 conn(0x7f7cbc0cfcf0 legacy=0x7f7cbc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc071a40 con 0x7f7cbc0ce590 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7cb40019f0 con 0x7f7cbc0ce590 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 >> v1:172.21.15.148:6789/0 conn(0x7f7cbc0ce590 legacy=0x7f7cbc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca80039b0 0x7f7ca8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca80043b0 0x7f7ca8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7ca800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca800bfb0 con 0x7f7ca8005080 2024-04-25T09:58:50.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca800c160 con 0x7f7ca80043b0 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.997+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca800e570 con 0x7f7ca80039b0 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7ca800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc220b640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca80039b0 0x7f7ca8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc1a0a640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca80043b0 0x7f7ca8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 -- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca80039b0 msgr2=0x7f7ca8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ca80039b0 0x7f7ca8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 -- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca80043b0 msgr2=0x7f7ca8004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ca80043b0 0x7f7ca8004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 -- 172.21.15.148:0/3698066974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8003520 con 0x7f7ca8005080 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.998+0000 7f7cc2a0c640 1 -- 172.21.15.148:0/3698066974 reap_dead start 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc2a0c640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7ca800b9c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400a800 tx=0x7f7ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93079c1965168e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400a450 con 0x7f7ca8005080 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb402f070 con 0x7f7ca8005080 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc1209640 1 -- 172.21.15.148:0/3698066974 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb402bc50 con 0x7f7ca8005080 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 -- 172.21.15.148:0/3698066974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 msgr2=0x7f7ca800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7ca800b9c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400a800 tx=0x7f7ca8005830 comp rx=0 tx=0).stop 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 -- 172.21.15.148:0/3698066974 shutdown_connections 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 --2- 172.21.15.148:0/3698066974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7ca800b9c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 -- 172.21.15.148:0/3698066974 >> 172.21.15.148:0/3698066974 conn(0x7f7cbc06abe0 msgr2=0x7f7cbc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 -- 172.21.15.148:0/3698066974 shutdown_connections 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:49.999+0000 7f7cc446f640 1 -- 172.21.15.148:0/3698066974 wait complete. 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.000+0000 7f7cc446f640 1 Processor -- start 2024-04-25T09:58:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 -- start start 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7cbc1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc153dd0 0x7f7cbc154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cbc1f2a20 0x7f7cbc1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1552c0 con 0x7f7ca8005080 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc071230 con 0x7f7cbc153dd0 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc446f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc155440 con 0x7f7cbc1f2a20 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc220b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7cbc1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc220b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7cbc1530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55938/0 (socket says 172.21.15.148:55938) 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.001+0000 7f7cc220b640 1 -- 172.21.15.148:0/2263451496 learned_addr learned my addr 172.21.15.148:0/2263451496 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc1a0a640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc153dd0 0x7f7cbc154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc2a0c640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cbc1f2a20 0x7f7cbc1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc220b640 1 -- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cbc1f2a20 msgr2=0x7f7cbc1f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc220b640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cbc1f2a20 0x7f7cbc1f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc220b640 1 -- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc153dd0 msgr2=0x7f7cbc154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc220b640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc153dd0 0x7f7cbc154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc220b640 1 -- 172.21.15.148:0/2263451496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc1f01d0 con 0x7f7ca8005080 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc2a0c640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cbc1f2a20 0x7f7cbc1f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc220b640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7cbc1530f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80118e0 tx=0x7f7cb8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800faf0 con 0x7f7ca8005080 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb800fc90 con 0x7f7ca8005080 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc1ef410 con 0x7f7ca8005080 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8014440 con 0x7f7ca8005080 2024-04-25T09:58:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.002+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc1ef9d0 con 0x7f7ca8005080 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.003+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc0b2590 con 0x7f7ca8005080 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.003+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cb8012040 con 0x7f7ca8005080 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.004+0000 7f7cb2ffd640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ca006b4d0 0x7f7ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.004+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7cb808aa90 con 0x7f7ca8005080 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.004+0000 7f7cc1a0a640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ca006b4d0 0x7f7ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.004+0000 7f7cc1a0a640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ca006b4d0 0x7f7ca006d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0adf90 tx=0x7f7cac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.008+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cbc0b2590 con 0x7f7ca8005080 2024-04-25T09:58:50.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.181+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cbc073990 con 0x7f7ca8005080 2024-04-25T09:58:50.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.182+0000 7f7cb2ffd640 1 -- 172.21.15.148:0/2263451496 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17006 (secure 0 0 0) 0x7f7cb8019030 con 0x7f7ca8005080 2024-04-25T09:58:50.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:50.189 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":82,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:49.193182+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ca006b4d0 msgr2=0x7f7ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ca006b4d0 0x7f7ca006d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0adf90 tx=0x7f7cac009040 comp rx=0 tx=0).stop 2024-04-25T09:58:50.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 msgr2=0x7f7cbc1530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7cbc1530f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80118e0 tx=0x7f7cb8011da0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 shutdown_connections 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ca006b4d0 0x7f7ca006d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cbc1f2a20 0x7f7cbc1f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cbc153dd0 0x7f7cbc154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 --2- 172.21.15.148:0/2263451496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ca8005080 0x7f7cbc1530f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.184+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 >> 172.21.15.148:0/2263451496 conn(0x7f7cbc06abe0 msgr2=0x7f7cbc15c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.185+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 shutdown_connections 2024-04-25T09:58:50.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.185+0000 7f7cc446f640 1 -- 172.21.15.148:0/2263451496 wait complete. 2024-04-25T09:58:50.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.292+0000 7f9bde093640 1 Processor -- start 2024-04-25T09:58:50.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.292+0000 7f9bde093640 1 -- start start 2024-04-25T09:58:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bde093640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9bd80b2cb0 0x7f9bd80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bde093640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd80d1020 0x7f9bd80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bde093640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd80b97a0 con 0x7f9bd815dbe0 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bde093640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd815dfc0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bde093640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80d1a00 con 0x7f9bd80d1020 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bceffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9bd80b2cb0 0x7f9bd80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.293+0000 7f9bd7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd80d1020 0x7f9bd80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.294+0000 7f9bd77fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9bd815dbe0 0x7f9bd80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:52480/0 (socket says 172.21.15.148:52480) 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.294+0000 7f9bd77fe640 1 -- 172.21.15.148:0/2329818242 learned_addr learned my addr 172.21.15.148:0/2329818242 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.294+0000 7f9bceffd640 1 -- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd80d1020 msgr2=0x7f9bd80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.294+0000 7f9bceffd640 1 --2- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd80d1020 0x7f9bd80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.294+0000 7f9bceffd640 1 -- 172.21.15.148:0/2329818242 >> v1:172.21.15.188:6789/0 conn(0x7f9bd815dbe0 legacy=0x7f9bd80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.294+0000 7f9bceffd640 1 -- 172.21.15.148:0/2329818242 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80d1c50 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bceffd640 1 --2- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd80d2730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00097a0 tx=0x7f9bc002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f8cf940027c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bd6ffd640 1 -- 172.21.15.148:0/2329818242 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0037070 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bd6ffd640 1 -- 172.21.15.148:0/2329818242 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0032440 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bd6ffd640 1 -- 172.21.15.148:0/2329818242 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0032810 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 -- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80b2cb0 msgr2=0x7f9bd80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 --2- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd80d2730 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00097a0 tx=0x7f9bc002a180 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 -- 172.21.15.148:0/2329818242 shutdown_connections 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 --2- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd80d2730 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 --2- 172.21.15.148:0/2329818242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd80d1020 0x7f9bd80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 -- 172.21.15.148:0/2329818242 >> 172.21.15.148:0/2329818242 conn(0x7f9bd8061a20 msgr2=0x7f9bd8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 -- 172.21.15.148:0/2329818242 shutdown_connections 2024-04-25T09:58:50.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.295+0000 7f9bde093640 1 -- 172.21.15.148:0/2329818242 wait complete. 2024-04-25T09:58:50.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.296+0000 7f9bde093640 1 Processor -- start 2024-04-25T09:58:50.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 -- start start 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80d1020 0x7f9bd8148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd815dbe0 0x7f9bd80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b92a0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80d2210 con 0x7f9bd80d1020 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bde093640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80b9420 con 0x7f9bd815dbe0 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.297+0000 7f9bd77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd815f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55942/0 (socket says 172.21.15.148:55942) 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 -- 172.21.15.148:0/2390232291 learned_addr learned my addr 172.21.15.148:0/2390232291 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bceffd640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80d1020 0x7f9bd8148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd7fff640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd815dbe0 0x7f9bd80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 -- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd815dbe0 msgr2=0x7f9bd80b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd815dbe0 0x7f9bd80b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 -- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80d1020 msgr2=0x7f9bd8148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80d1020 0x7f9bd8148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 -- 172.21.15.148:0/2390232291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd815e710 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd77fe640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd815f7b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800c3c0 tx=0x7f9bc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8017070 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8005ce0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd815e9f0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80072f0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.298+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd815efb0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.299+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd8073990 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.299+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bc801a070 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.300+0000 7f9bd4ff9640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bb806b4d0 0x7f9bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.300+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9bc80897c0 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.300+0000 7f9bceffd640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bb806b4d0 0x7f9bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.300+0000 7f9bceffd640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bb806b4d0 0x7f9bb806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0034000 tx=0x7f9bc002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.304+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bc8058420 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.476+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bd80d4790 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.477+0000 7f9bd4ff9640 1 -- 172.21.15.148:0/2390232291 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+17006 (secure 0 0 0) 0x7f9bc8057d20 con 0x7f9bd80b2cb0 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:50.481 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":82,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:49.193182+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bb806b4d0 msgr2=0x7f9bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bb806b4d0 0x7f9bb806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0034000 tx=0x7f9bc002e000 comp rx=0 tx=0).stop 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 msgr2=0x7f9bd815f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd815f7b0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800c3c0 tx=0x7f9bc800c880 comp rx=0 tx=0).stop 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 shutdown_connections 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bb806b4d0 0x7f9bb806d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bd815dbe0 0x7f9bd80b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd80d1020 0x7f9bd8148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 --2- 172.21.15.148:0/2390232291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd80b2cb0 0x7f9bd815f7b0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 >> 172.21.15.148:0/2390232291 conn(0x7f9bd8061a20 msgr2=0x7f9bd815a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 shutdown_connections 2024-04-25T09:58:50.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.479+0000 7f9bde093640 1 -- 172.21.15.148:0/2390232291 wait complete. 2024-04-25T09:58:50.494 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:58:50.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.591+0000 7f55be977640 1 Processor -- start 2024-04-25T09:58:50.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.591+0000 7f55be977640 1 -- start start 2024-04-25T09:58:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55be977640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f55b815bb20 0x7f55b80cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55be977640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55b80d00e0 0x7f55b80ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55be977640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b80cb190 con 0x7f55b80ccdd0 2024-04-25T09:58:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55be977640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b80cb470 con 0x7f55b80d00e0 2024-04-25T09:58:50.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55be977640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b80cb750 con 0x7f55b815bb20 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55bcf14640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55b80d00e0 0x7f55b80ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55bcf14640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55b80d00e0 0x7f55b80ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55956/0 (socket says 172.21.15.148:55956) 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.592+0000 7f55bcf14640 1 -- 172.21.15.148:0/2982414424 learned_addr learned my addr 172.21.15.148:0/2982414424 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55bcf14640 1 -- 172.21.15.148:0/2982414424 >> v2:172.21.15.188:3300/0 conn(0x7f55b815bb20 msgr2=0x7f55b80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55bcf14640 1 --2- 172.21.15.148:0/2982414424 >> v2:172.21.15.188:3300/0 conn(0x7f55b815bb20 0x7f55b80cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55bcf14640 1 -- 172.21.15.148:0/2982414424 >> v1:172.21.15.188:6789/0 conn(0x7f55b80ccdd0 legacy=0x7f55b80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55bcf14640 1 -- 172.21.15.148:0/2982414424 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80cba30 con 0x7f55b80d00e0 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55bcf14640 1 --2- 172.21.15.148:0/2982414424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b80d00e0 0x7f55b80ad600 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f55b80d1800 tx=0x7f55b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2555588176f68387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55b6ffd640 1 -- 172.21.15.148:0/2982414424 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0036070 con 0x7f55b80d00e0 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.593+0000 7f55b6ffd640 1 -- 172.21.15.148:0/2982414424 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0031600 con 0x7f55b80d00e0 2024-04-25T09:58:50.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55b6ffd640 1 -- 172.21.15.148:0/2982414424 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00319d0 con 0x7f55b80d00e0 2024-04-25T09:58:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 -- 172.21.15.148:0/2982414424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b80d00e0 msgr2=0x7f55b80ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 --2- 172.21.15.148:0/2982414424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b80d00e0 0x7f55b80ad600 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f55b80d1800 tx=0x7f55b002a660 comp rx=0 tx=0).stop 2024-04-25T09:58:50.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 -- 172.21.15.148:0/2982414424 shutdown_connections 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 --2- 172.21.15.148:0/2982414424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b80d00e0 0x7f55b80ad600 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 --2- 172.21.15.148:0/2982414424 >> v2:172.21.15.188:3300/0 conn(0x7f55b815bb20 0x7f55b80cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 -- 172.21.15.148:0/2982414424 >> 172.21.15.148:0/2982414424 conn(0x7f55b80b2390 msgr2=0x7f55b80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 -- 172.21.15.148:0/2982414424 shutdown_connections 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.594+0000 7f55be977640 1 -- 172.21.15.148:0/2982414424 wait complete. 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.595+0000 7f55be977640 1 Processor -- start 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.595+0000 7f55be977640 1 -- start start 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55be977640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80ccdd0 0x7f55b814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55be977640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80d00e0 0x7f55b8156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55be977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 0x7f55b80b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55be977640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b815bf00 con 0x7f55b815bb20 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55be977640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b80cc770 con 0x7f55b80ccdd0 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55be977640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b815c080 con 0x7f55b80d00e0 2024-04-25T09:58:50.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55bcf14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 0x7f55b80b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55bcf14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 0x7f55b80b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55968/0 (socket says 172.21.15.148:55968) 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55bcf14640 1 -- 172.21.15.148:0/2642581754 learned_addr learned my addr 172.21.15.148:0/2642581754 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55b7fff640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80ccdd0 0x7f55b814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.596+0000 7f55b77fe640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80d00e0 0x7f55b8156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55bcf14640 1 -- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80d00e0 msgr2=0x7f55b8156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55bcf14640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80d00e0 0x7f55b8156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55bcf14640 1 -- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80ccdd0 msgr2=0x7f55b814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55bcf14640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80ccdd0 0x7f55b814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55bcf14640 1 -- 172.21.15.148:0/2642581754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8157480 con 0x7f55b815bb20 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55b77fe640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80d00e0 0x7f55b8156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55b7fff640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80ccdd0 0x7f55b814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55bcf14640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 0x7f55b80b0670 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f55b0034040 tx=0x7f55b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0041020 con 0x7f55b815bb20 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0007780 con 0x7f55b815bb20 2024-04-25T09:58:50.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b8157760 con 0x7f55b815bb20 2024-04-25T09:58:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b0036040 con 0x7f55b815bb20 2024-04-25T09:58:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.597+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b80b4dd0 con 0x7f55b815bb20 2024-04-25T09:58:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.598+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b80b4fd0 con 0x7f55b815bb20 2024-04-25T09:58:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.598+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55b0044020 con 0x7f55b815bb20 2024-04-25T09:58:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.599+0000 7f55b4ff9640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55a406b4d0 0x7f55a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.599+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55b00aba90 con 0x7f55b815bb20 2024-04-25T09:58:50.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.599+0000 7f55b7fff640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55a406b4d0 0x7f55a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.599+0000 7f55b7fff640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55a406b4d0 0x7f55a406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009890 tx=0x7f55a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.603+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55b80b4fd0 con 0x7f55b815bb20 2024-04-25T09:58:50.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.808+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f55b8073a00 con 0x7f55b815bb20 2024-04-25T09:58:50.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.866+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2642581754 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f55b004b030 con 0x7f55b815bb20 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55a406b4d0 msgr2=0x7f55a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55a406b4d0 0x7f55a406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009890 tx=0x7f55a8008040 comp rx=0 tx=0).stop 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 msgr2=0x7f55b80b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 0x7f55b80b0670 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f55b0034040 tx=0x7f55b0004a40 comp rx=0 tx=0).stop 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 shutdown_connections 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55a406b4d0 0x7f55a406d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55b815bb20 0x7f55b80b0670 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55b80d00e0 0x7f55b8156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.868+0000 7f55be977640 1 --2- 172.21.15.148:0/2642581754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55b80ccdd0 0x7f55b814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.869+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 >> 172.21.15.148:0/2642581754 conn(0x7f55b80b2390 msgr2=0x7f55b80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.869+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 shutdown_connections 2024-04-25T09:58:50.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.869+0000 7f55be977640 1 -- 172.21.15.148:0/2642581754 wait complete. 2024-04-25T09:58:50.884 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:58:50.884 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:58:50.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.980+0000 7f562494c640 1 Processor -- start 2024-04-25T09:58:50.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.980+0000 7f562494c640 1 -- start start 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.981+0000 7f562494c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 0x7f5620168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.981+0000 7f562494c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56200d06c0 con 0x7f562015e0e0 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.981+0000 7f562494c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56200d0960 con 0x7f562015dd00 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.981+0000 7f562494c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200d1440 con 0x7f562006cf10 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.981+0000 7f561dd9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f562015e0e0 0x7f562009a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39598/0 (socket says 172.21.15.148:39598) 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.981+0000 7f561dd9b640 1 -- 172.21.15.148:0/792214892 learned_addr learned my addr 172.21.15.148:0/792214892 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561ed9d640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 0x7f5620168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401193904 0 0) 0x7f56200d06c0 con 0x7f562015e0e0 2024-04-25T09:58:50.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5608003520 con 0x7f562015e0e0 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 82353613 0 0) 0x7f5608003520 con 0x7f562015e0e0 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 msgr2=0x7f5620168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 0x7f5620168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 >> v1:172.21.15.188:6789/0 conn(0x7f562015dd00 legacy=0x7f5620099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200d0c40 con 0x7f562015e0e0 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5614001670 con 0x7f562015e0e0 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 >> v1:172.21.15.148:6789/0 conn(0x7f562015e0e0 legacy=0x7f562009a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56080039e0 0x7f5608003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5608004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 0x7f56080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200d1440 con 0x7f5608004800 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560800ba70 con 0x7f56080039e0 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560800bc70 con 0x7f562006cf10 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561dd9b640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5608004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561ed9d640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 0x7f56080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561e59c640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56080039e0 0x7f5608003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561dd9b640 1 -- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 msgr2=0x7f56080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561dd9b640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f562006cf10 0x7f56080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561dd9b640 1 -- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56080039e0 msgr2=0x7f5608003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561dd9b640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56080039e0 0x7f5608003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.982+0000 7f561dd9b640 1 -- 172.21.15.148:0/792214892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608003520 con 0x7f5608004800 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.983+0000 7f561dd9b640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5608004c80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5614001eb0 tx=0x7f5608005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b7031a0d7ddd80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.983+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614007d30 con 0x7f5608004800 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.983+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614005af0 con 0x7f5608004800 2024-04-25T09:58:50.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.983+0000 7f561d59a640 1 -- 172.21.15.148:0/792214892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614007070 con 0x7f5608004800 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f562494c640 1 -- 172.21.15.148:0/792214892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 msgr2=0x7f5608004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f562494c640 1 --2- 172.21.15.148:0/792214892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5608004c80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5614001eb0 tx=0x7f5608005830 comp rx=0 tx=0).stop 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f561ed9d640 1 -- 172.21.15.148:0/792214892 reap_dead start 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f562494c640 1 -- 172.21.15.148:0/792214892 shutdown_connections 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f562494c640 1 -- 172.21.15.148:0/792214892 >> 172.21.15.148:0/792214892 conn(0x7f56200b3290 msgr2=0x7f56200b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f562494c640 1 -- 172.21.15.148:0/792214892 shutdown_connections 2024-04-25T09:58:50.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.984+0000 7f562494c640 1 -- 172.21.15.148:0/792214892 wait complete. 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.985+0000 7f562494c640 1 Processor -- start 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.985+0000 7f562494c640 1 -- start start 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f562494c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5620154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f562494c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5620154bf0 0x7f56200b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f562494c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200b6920 0x7f562006fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f562494c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b2b40 con 0x7f5608004800 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f562494c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562009a950 con 0x7f56200b6920 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f562494c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b2cc0 con 0x7f5620154bf0 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5620154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5620154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55984/0 (socket says 172.21.15.148:55984) 2024-04-25T09:58:50.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 -- 172.21.15.148:0/394560436 learned_addr learned my addr 172.21.15.148:0/394560436 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561dd9b640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5620154bf0 0x7f56200b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561ed9d640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200b6920 0x7f562006fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 -- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5620154bf0 msgr2=0x7f56200b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5620154bf0 0x7f56200b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 -- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200b6920 msgr2=0x7f562006fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200b6920 0x7f562006fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561e59c640 1 -- 172.21.15.148:0/394560436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56201f6ba0 con 0x7f5608004800 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.986+0000 7f561dd9b640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5620154bf0 0x7f56200b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.987+0000 7f561e59c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5620154380 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f561000fb90 tx=0x7f561000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.987+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610003b80 con 0x7f5608004800 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.987+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610018070 con 0x7f5608004800 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.987+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561000d7f0 con 0x7f5608004800 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.987+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56201f6e40 con 0x7f5608004800 2024-04-25T09:58:50.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.987+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56201f7400 con 0x7f5608004800 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.988+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5620073a00 con 0x7f5608004800 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.988+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f561000d990 con 0x7f5608004800 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.989+0000 7f560effd640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560006b500 0x7f560006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.989+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f561008a4b0 con 0x7f5608004800 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.989+0000 7f561dd9b640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560006b500 0x7f560006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.989+0000 7f561dd9b640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560006b500 0x7f560006d9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5614032000 tx=0x7f561402d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:50.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:50.993+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5610058b50 con 0x7f5608004800 2024-04-25T09:58:51.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.201+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5620068080 con 0x7f5608004800 2024-04-25T09:58:51.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.874+0000 7f560effd640 1 -- 172.21.15.148:0/394560436 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f561005cbf0 con 0x7f5608004800 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560006b500 msgr2=0x7f560006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560006b500 0x7f560006d9c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5614032000 tx=0x7f561402d000 comp rx=0 tx=0).stop 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 msgr2=0x7f5620154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5620154380 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f561000fb90 tx=0x7f561000ff90 comp rx=0 tx=0).stop 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 shutdown_connections 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f560006b500 0x7f560006d9c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56200b6920 0x7f562006fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5620154bf0 0x7f56200b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 --2- 172.21.15.148:0/394560436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5608004800 0x7f5620154380 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:51.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 >> 172.21.15.148:0/394560436 conn(0x7f56200b3290 msgr2=0x7f5620099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:51.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 shutdown_connections 2024-04-25T09:58:51.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.877+0000 7f562494c640 1 -- 172.21.15.148:0/394560436 wait complete. 2024-04-25T09:58:51.894 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:58:51.894 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.993+0000 7fa103e2a640 1 Processor -- start 2024-04-25T09:58:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.993+0000 7fa103e2a640 1 -- start start 2024-04-25T09:58:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa103e2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc06abe0 0x7fa0fc06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa103e2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa0fc0ce330 0x7fa0fc06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa103e2a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc1626e0 con 0x7fa0fc0700a0 2024-04-25T09:58:52.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa103e2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc06b500 con 0x7fa0fc0ce330 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa103e2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0cd270 con 0x7fa0fc06abe0 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa1013c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa0fc0ce330 0x7fa0fc06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa1013c5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa0fc0ce330 0x7fa0fc06fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55988/0 (socket says 172.21.15.148:55988) 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa1013c5640 1 -- 172.21.15.148:0/2814760607 learned_addr learned my addr 172.21.15.148:0/2814760607 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.994+0000 7fa101bc6640 1 --2- 172.21.15.148:0/2814760607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc06abe0 0x7fa0fc06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa1013c5640 1 -- 172.21.15.148:0/2814760607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc06abe0 msgr2=0x7fa0fc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa1013c5640 1 --2- 172.21.15.148:0/2814760607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc06abe0 0x7fa0fc06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa1013c5640 1 -- 172.21.15.148:0/2814760607 >> v1:172.21.15.188:6789/0 conn(0x7fa0fc0700a0 legacy=0x7fa0fc06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa1013c5640 1 -- 172.21.15.148:0/2814760607 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0cd550 con 0x7fa0fc0ce330 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa1013c5640 1 --2- 172.21.15.148:0/2814760607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0ce330 0x7fa0fc06fb60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009870 tx=0x7fa0ec02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406f245effbe6927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa100bc4640 1 -- 172.21.15.148:0/2814760607 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec037070 con 0x7fa0fc0ce330 2024-04-25T09:58:52.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa100bc4640 1 -- 172.21.15.148:0/2814760607 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec032440 con 0x7fa0fc0ce330 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.995+0000 7fa100bc4640 1 -- 172.21.15.148:0/2814760607 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec032810 con 0x7fa0fc0ce330 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 -- 172.21.15.148:0/2814760607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0ce330 msgr2=0x7fa0fc06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 --2- 172.21.15.148:0/2814760607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0ce330 0x7fa0fc06fb60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009870 tx=0x7fa0ec02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 -- 172.21.15.148:0/2814760607 shutdown_connections 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 --2- 172.21.15.148:0/2814760607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0ce330 0x7fa0fc06fb60 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 --2- 172.21.15.148:0/2814760607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc06abe0 0x7fa0fc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 -- 172.21.15.148:0/2814760607 >> 172.21.15.148:0/2814760607 conn(0x7fa0fc0b2220 msgr2=0x7fa0fc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 -- 172.21.15.148:0/2814760607 shutdown_connections 2024-04-25T09:58:52.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.996+0000 7fa103e2a640 1 -- 172.21.15.148:0/2814760607 wait complete. 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.997+0000 7fa103e2a640 1 Processor -- start 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.997+0000 7fa103e2a640 1 -- start start 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa103e2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc06abe0 0x7fa0fc15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa103e2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 0x7fa0fc0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa103e2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc0ce330 0x7fa0fc0b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa103e2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc161780 con 0x7fa0fc0700a0 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa103e2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0cdb10 con 0x7fa0fc06abe0 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa103e2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc06eb00 con 0x7fa0fc0ce330 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 0x7fa0fc0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 0x7fa0fc0b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56002/0 (socket says 172.21.15.148:56002) 2024-04-25T09:58:52.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 -- 172.21.15.148:0/43040877 learned_addr learned my addr 172.21.15.148:0/43040877 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa101bc6640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc06abe0 0x7fa0fc15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1023c7640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc0ce330 0x7fa0fc0b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 -- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc0ce330 msgr2=0x7fa0fc0b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc0ce330 0x7fa0fc0b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 -- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc06abe0 msgr2=0x7fa0fc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc06abe0 0x7fa0fc15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa1013c5640 1 -- 172.21.15.148:0/43040877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc1f8300 con 0x7fa0fc0700a0 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.998+0000 7fa101bc6640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc06abe0 0x7fa0fc15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.999+0000 7fa1013c5640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 0x7fa0fc0b0170 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0090b0 tx=0x7fa0ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.999+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec041070 con 0x7fa0fc0700a0 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.999+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec03c8b0 con 0x7fa0fc0700a0 2024-04-25T09:58:52.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.999+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec037040 con 0x7fa0fc0700a0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.999+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc1f85e0 con 0x7fa0fc0700a0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:51.999+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc09ca40 con 0x7fa0fc0700a0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.000+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc06ef30 con 0x7fa0fc0700a0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.000+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0ec044070 con 0x7fa0fc0700a0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.001+0000 7fa0f27fc640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.001+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0ec0ab0f0 con 0x7fa0fc0700a0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.001+0000 7fa101bc6640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.001+0000 7fa101bc6640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0e006b4d0 0x7fa0e006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009b20 tx=0x7fa0f80039d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.005+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0fc06ef30 con 0x7fa0fc0700a0 2024-04-25T09:58:52.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.214+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0fc073990 con 0x7fa0fc0700a0 2024-04-25T09:58:52.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.215+0000 7fa0f27fc640 1 -- 172.21.15.148:0/43040877 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa0ec079f00 con 0x7fa0fc0700a0 2024-04-25T09:58:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:52.219 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:49.193553+0000","modified":"2024-04-25T09:58:51.202219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7284":{"gid":7284,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/82715169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":82715169},{"type":"v1","addr":"172.21.15.148:6837","nonce":82715169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:52.219 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 46 2024-04-25T09:58:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0e006b4d0 msgr2=0x7fa0e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0e006b4d0 0x7fa0e006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009b20 tx=0x7fa0f80039d0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 msgr2=0x7fa0fc0b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 0x7fa0fc0b0170 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0090b0 tx=0x7fa0ec0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 shutdown_connections 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0fc0ce330 0x7fa0fc0b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0fc0700a0 0x7fa0fc0b0170 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 --2- 172.21.15.148:0/43040877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0fc06abe0 0x7fa0fc15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 >> 172.21.15.148:0/43040877 conn(0x7fa0fc0b2220 msgr2=0x7fa0fc0ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 shutdown_connections 2024-04-25T09:58:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.217+0000 7fa103e2a640 1 -- 172.21.15.148:0/43040877 wait complete. 2024-04-25T09:58:52.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:58:52.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.326+0000 7fbf5b68f640 1 Processor -- start 2024-04-25T09:58:52.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.326+0000 7fbf5b68f640 1 -- start start 2024-04-25T09:58:52.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5b68f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf541602e0 0x7fbf540ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5b68f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 0x7fbf5415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5b68f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf540cd710 con 0x7fbf5415ba00 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5b68f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf54099b70 con 0x7fbf541602e0 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5b68f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54099e50 con 0x7fbf5415f5f0 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5942b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbf5415ba00 0x7fbf5409a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39610/0 (socket says 172.21.15.148:39610) 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf5942b640 1 -- 172.21.15.148:0/3218595605 learned_addr learned my addr 172.21.15.148:0/3218595605 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.327+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/3218595605 >> v2:172.21.15.188:3300/0 conn(0x7fbf541602e0 0x7fbf540ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf59c2c640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 0x7fbf5415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921210627 0 0) 0x7fbf50001110 con 0x7fbf5415ba00 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf40003520 con 0x7fbf5415ba00 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3343767291 0 0) 0x7fbf40003520 con 0x7fbf5415ba00 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 msgr2=0x7fbf5415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 0x7fbf5415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 >> v2:172.21.15.188:3300/0 conn(0x7fbf541602e0 msgr2=0x7fbf540ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 --2- 172.21.15.148:0/3218595605 >> v2:172.21.15.188:3300/0 conn(0x7fbf541602e0 0x7fbf540ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5409a130 con 0x7fbf5415ba00 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbf50001840 con 0x7fbf5415ba00 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/3218595605 >> v2:172.21.15.188:3300/0 conn(0x7fbf541602e0 0x7fbf540ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 >> v1:172.21.15.148:6789/0 conn(0x7fbf5415ba00 legacy=0x7fbf5409a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf400039b0 0x7fbf40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf40004800 0x7fbf40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 0x7fbf400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54099b70 con 0x7fbf40004800 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf54099e50 con 0x7fbf400039b0 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000ba40 con 0x7fbf5415f5f0 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf40004800 0x7fbf40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf5942b640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf400039b0 0x7fbf40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf59c2c640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 0x7fbf400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 -- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 msgr2=0x7fbf400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf5415f5f0 0x7fbf400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 -- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf400039b0 msgr2=0x7fbf40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf400039b0 0x7fbf40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.328+0000 7fbf58c2a640 1 -- 172.21.15.148:0/3218595605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40003520 con 0x7fbf40004800 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.329+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf40004800 0x7fbf40004c80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009c90 tx=0x7fbf40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d2b2172e0a9926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.329+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44036070 con 0x7fbf40004800 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.329+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44031400 con 0x7fbf40004800 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.329+0000 7fbf4bfff640 1 -- 172.21.15.148:0/3218595605 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44009860 con 0x7fbf40004800 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf5b68f640 1 -- 172.21.15.148:0/3218595605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf40004800 msgr2=0x7fbf40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/3218595605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf40004800 0x7fbf40004c80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009c90 tx=0x7fbf40005830 comp rx=0 tx=0).stop 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf59c2c640 1 -- 172.21.15.148:0/3218595605 reap_dead start 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf5b68f640 1 -- 172.21.15.148:0/3218595605 shutdown_connections 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf5b68f640 1 -- 172.21.15.148:0/3218595605 >> 172.21.15.148:0/3218595605 conn(0x7fbf540b3270 msgr2=0x7fbf540b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf5b68f640 1 -- 172.21.15.148:0/3218595605 shutdown_connections 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.330+0000 7fbf5b68f640 1 -- 172.21.15.148:0/3218595605 wait complete. 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.331+0000 7fbf5b68f640 1 Processor -- start 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.331+0000 7fbf5b68f640 1 -- start start 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf40004800 0x7fbf541f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 0x7fbf5415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf540ca350 0x7fbf540ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540cd460 con 0x7fbf5415e620 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf540cd2e0 con 0x7fbf40004800 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5415fa80 con 0x7fbf540ca350 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 0x7fbf5415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 0x7fbf5415eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56022/0 (socket says 172.21.15.148:56022) 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 -- 172.21.15.148:0/2381820528 learned_addr learned my addr 172.21.15.148:0/2381820528 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 -- 172.21.15.148:0/2381820528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf540ca350 msgr2=0x7fbf540ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf540ca350 0x7fbf540ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 -- 172.21.15.148:0/2381820528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf40004800 msgr2=0x7fbf541f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf40004800 0x7fbf541f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 -- 172.21.15.148:0/2381820528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf541f1740 con 0x7fbf5415e620 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf58c2a640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 0x7fbf5415eaa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009db0 tx=0x7fbf44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44040020 con 0x7fbf5415e620 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4403b680 con 0x7fbf5415e620 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf541f19c0 con 0x7fbf5415e620 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.332+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44036040 con 0x7fbf5415e620 2024-04-25T09:58:52.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.333+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf541f1f80 con 0x7fbf5415e620 2024-04-25T09:58:52.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.334+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf54073990 con 0x7fbf5415e620 2024-04-25T09:58:52.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.334+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf44043020 con 0x7fbf5415e620 2024-04-25T09:58:52.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.334+0000 7fbf49ffb640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3806b500 0x7fbf3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.334+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf440aad20 con 0x7fbf5415e620 2024-04-25T09:58:52.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.334+0000 7fbf5942b640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3806b500 0x7fbf3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.335+0000 7fbf5942b640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3806b500 0x7fbf3806d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf540cb950 tx=0x7fbf50007e60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.339+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf4404a020 con 0x7fbf5415e620 2024-04-25T09:58:52.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.509+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf540b40a0 con 0x7fbf5415e620 2024-04-25T09:58:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.510+0000 7fbf49ffb640 1 -- 172.21.15.148:0/2381820528 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16993 (secure 0 0 0) 0x7fbf541f1740 con 0x7fbf5415e620 2024-04-25T09:58:52.514 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:52.515 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":83,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:58:50.860742+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:58:52.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3806b500 msgr2=0x7fbf3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3806b500 0x7fbf3806d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf540cb950 tx=0x7fbf50007e60 comp rx=0 tx=0).stop 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 msgr2=0x7fbf5415eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 0x7fbf5415eaa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009db0 tx=0x7fbf44004730 comp rx=0 tx=0).stop 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 shutdown_connections 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf3806b500 0x7fbf3806d9c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf540ca350 0x7fbf540ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf5415e620 0x7fbf5415eaa0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 --2- 172.21.15.148:0/2381820528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf40004800 0x7fbf541f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.512+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 >> 172.21.15.148:0/2381820528 conn(0x7fbf540b3270 msgr2=0x7fbf5409a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.513+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 shutdown_connections 2024-04-25T09:58:52.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.513+0000 7fbf5b68f640 1 -- 172.21.15.148:0/2381820528 wait complete. 2024-04-25T09:58:52.528 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:58:52.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.622+0000 7f0fb8c07640 1 Processor -- start 2024-04-25T09:58:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.622+0000 7f0fb8c07640 1 -- start start 2024-04-25T09:58:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.623+0000 7f0fb8c07640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fb40ce590 0x7f0fb4165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.623+0000 7f0fb8c07640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb4071480 con 0x7f0fb4154ee0 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.623+0000 7f0fb8c07640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb4071760 con 0x7f0fb40cfec0 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.623+0000 7f0fb8c07640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb4072090 con 0x7f0fb40ce590 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fb40ce590 0x7f0fb4165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0fb4154ee0 0x7f0fb40cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39618/0 (socket says 172.21.15.148:39618) 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0fb40ce590 0x7f0fb4165900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56030/0 (socket says 172.21.15.148:56030) 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb259c640 1 -- 172.21.15.148:0/3724280713 learned_addr learned my addr 172.21.15.148:0/3724280713 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb2d9d640 1 -- 172.21.15.148:0/3724280713 >> v1:172.21.15.188:6789/0 conn(0x7f0fb40cfec0 legacy=0x7f0fb40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb2d9d640 1 -- 172.21.15.148:0/3724280713 >> v1:172.21.15.148:6789/0 conn(0x7f0fb4154ee0 legacy=0x7f0fb40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb2d9d640 1 -- 172.21.15.148:0/3724280713 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4071a40 con 0x7f0fb40ce590 2024-04-25T09:58:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.624+0000 7f0fb2d9d640 1 --2- 172.21.15.148:0/3724280713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb4165900 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40097a0 tx=0x7f0fa402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77975e650710bb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb159a640 1 -- 172.21.15.148:0/3724280713 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4037070 con 0x7f0fb40ce590 2024-04-25T09:58:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb159a640 1 -- 172.21.15.148:0/3724280713 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa4032440 con 0x7f0fb40ce590 2024-04-25T09:58:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb159a640 1 -- 172.21.15.148:0/3724280713 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4032810 con 0x7f0fb40ce590 2024-04-25T09:58:52.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 -- 172.21.15.148:0/3724280713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 msgr2=0x7f0fb4165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/3724280713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb4165900 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40097a0 tx=0x7f0fa402a230 comp rx=0 tx=0).stop 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 -- 172.21.15.148:0/3724280713 shutdown_connections 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/3724280713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb4165900 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 -- 172.21.15.148:0/3724280713 >> 172.21.15.148:0/3724280713 conn(0x7f0fb406abe0 msgr2=0x7f0fb406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 -- 172.21.15.148:0/3724280713 shutdown_connections 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.625+0000 7f0fb8c07640 1 -- 172.21.15.148:0/3724280713 wait complete. 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 Processor -- start 2024-04-25T09:58:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 -- start start 2024-04-25T09:58:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb40cfec0 0x7f0fb41f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb4154ee0 0x7f0fb414e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41552c0 con 0x7f0fb40ce590 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4071230 con 0x7f0fb40cfec0 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb8c07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4155440 con 0x7f0fb4154ee0 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.627+0000 7f0fb259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb414ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56034/0 (socket says 172.21.15.148:56034) 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 -- 172.21.15.148:0/900283161 learned_addr learned my addr 172.21.15.148:0/900283161 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 -- 172.21.15.148:0/900283161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb4154ee0 msgr2=0x7f0fb414e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb4154ee0 0x7f0fb414e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 -- 172.21.15.148:0/900283161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb40cfec0 msgr2=0x7f0fb41f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb40cfec0 0x7f0fb41f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 -- 172.21.15.148:0/900283161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb40b0df0 con 0x7f0fb40ce590 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb259c640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb414ba70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00cbf0 tx=0x7f0fac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac017070 con 0x7f0fb40ce590 2024-04-25T09:58:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac00aca0 con 0x7f0fb40ce590 2024-04-25T09:58:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac012600 con 0x7f0fb40ce590 2024-04-25T09:58:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb40af2c0 con 0x7f0fb40ce590 2024-04-25T09:58:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.628+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb40af880 con 0x7f0fb40ce590 2024-04-25T09:58:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.629+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb4151d40 con 0x7f0fb40ce590 2024-04-25T09:58:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.630+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fac01a070 con 0x7f0fb40ce590 2024-04-25T09:58:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.630+0000 7f0fa2ffd640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.630+0000 7f0fb1d9b640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.630+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fac089f60 con 0x7f0fb40ce590 2024-04-25T09:58:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.631+0000 7f0fb1d9b640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f9406b4d0 0x7f0f9406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000f60 tx=0x7f0fa80079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.634+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fb4151d40 con 0x7f0fb40ce590 2024-04-25T09:58:52.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.805+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0fb406d360 con 0x7f0f9406b4d0 2024-04-25T09:58:52.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.811+0000 7f0fa2ffd640 1 -- 172.21.15.148:0/900283161 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252004 (secure 0 0 0) 0x7f0fb406d360 con 0x7f0f9406b4d0 2024-04-25T09:58:52.818 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:52.819 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f9406b4d0 msgr2=0x7f0f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f9406b4d0 0x7f0f9406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8000f60 tx=0x7f0fa80079d0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 msgr2=0x7f0fb414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb414ba70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00cbf0 tx=0x7f0fac00a060 comp rx=0 tx=0).stop 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 shutdown_connections 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0fb4154ee0 0x7f0fb414e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fb40cfec0 0x7f0fb41f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.814+0000 7f0fb8c07640 1 --2- 172.21.15.148:0/900283161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fb40ce590 0x7f0fb414ba70 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.815+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 >> 172.21.15.148:0/900283161 conn(0x7f0fb406abe0 msgr2=0x7f0fb4151630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.815+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 shutdown_connections 2024-04-25T09:58:52.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.815+0000 7f0fb8c07640 1 -- 172.21.15.148:0/900283161 wait complete. 2024-04-25T09:58:52.829 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-25T09:58:51.074884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291716,"kb_used_data":8964,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754683004,"statfs":{"total":773094113280,"available":772795396096,"internally_reserved":0,"allocated":9179136,"data_stored":4844023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000658"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872471+0000","last_change":"2024-04-25T09:58:49.866402+0000","last_active":"2024-04-25T09:58:49.872471+0000","last_peered":"2024-04-25T09:58:49.872471+0000","last_clean":"2024-04-25T09:58:49.872471+0000","last_became_active":"2024-04-25T09:58:49.866123+0000","last_became_peered":"2024-04-25T09:58:49.866123+0000","last_unstale":"2024-04-25T09:58:49.872471+0000","last_undegraded":"2024-04-25T09:58:49.872471+0000","last_fullsized":"2024-04-25T09:58:49.872471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:16.672276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885419+0000","last_change":"2024-04-25T09:58:49.883642+0000","last_active":"2024-04-25T09:58:49.885419+0000","last_peered":"2024-04-25T09:58:49.885419+0000","last_clean":"2024-04-25T09:58:49.885419+0000","last_became_active":"2024-04-25T09:58:49.883378+0000","last_became_peered":"2024-04-25T09:58:49.883378+0000","last_unstale":"2024-04-25T09:58:49.885419+0000","last_undegraded":"2024-04-25T09:58:49.885419+0000","last_fullsized":"2024-04-25T09:58:49.885419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.136005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882914+0000","last_change":"2024-04-25T09:58:49.875977+0000","last_active":"2024-04-25T09:58:49.882914+0000","last_peered":"2024-04-25T09:58:49.882914+0000","last_clean":"2024-04-25T09:58:49.882914+0000","last_became_active":"2024-04-25T09:58:49.874129+0000","last_became_peered":"2024-04-25T09:58:49.874129+0000","last_unstale":"2024-04-25T09:58:49.882914+0000","last_undegraded":"2024-04-25T09:58:49.882914+0000","last_fullsized":"2024-04-25T09:58:49.882914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:15.168768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862634+0000","last_change":"2024-04-25T09:58:49.861515+0000","last_active":"2024-04-25T09:58:49.862634+0000","last_peered":"2024-04-25T09:58:49.862634+0000","last_clean":"2024-04-25T09:58:49.862634+0000","last_became_active":"2024-04-25T09:58:49.861238+0000","last_became_peered":"2024-04-25T09:58:49.861238+0000","last_unstale":"2024-04-25T09:58:49.862634+0000","last_undegraded":"2024-04-25T09:58:49.862634+0000","last_fullsized":"2024-04-25T09:58:49.862634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:16.672276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883180+0000","last_change":"2024-04-25T09:58:49.877651+0000","last_active":"2024-04-25T09:58:49.883180+0000","last_peered":"2024-04-25T09:58:49.883180+0000","last_clean":"2024-04-25T09:58:49.883180+0000","last_became_active":"2024-04-25T09:58:49.875944+0000","last_became_peered":"2024-04-25T09:58:49.875944+0000","last_unstale":"2024-04-25T09:58:49.883180+0000","last_undegraded":"2024-04-25T09:58:49.883180+0000","last_fullsized":"2024-04-25T09:58:49.883180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:20.990159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.868065+0000","last_change":"2024-04-25T09:58:49.867372+0000","last_active":"2024-04-25T09:58:49.868065+0000","last_peered":"2024-04-25T09:58:49.868065+0000","last_clean":"2024-04-25T09:58:49.868065+0000","last_became_active":"2024-04-25T09:58:49.866334+0000","last_became_peered":"2024-04-25T09:58:49.866334+0000","last_unstale":"2024-04-25T09:58:49.868065+0000","last_undegraded":"2024-04-25T09:58:49.868065+0000","last_fullsized":"2024-04-25T09:58:49.868065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:57.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.889010+0000","last_change":"2024-04-25T09:58:49.884475+0000","last_active":"2024-04-25T09:58:49.889010+0000","last_peered":"2024-04-25T09:58:49.889010+0000","last_clean":"2024-04-25T09:58:49.889010+0000","last_became_active":"2024-04-25T09:58:49.884215+0000","last_became_peered":"2024-04-25T09:58:49.884215+0000","last_unstale":"2024-04-25T09:58:49.889010+0000","last_undegraded":"2024-04-25T09:58:49.889010+0000","last_fullsized":"2024-04-25T09:58:49.889010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:47.622560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883755+0000","last_change":"2024-04-25T09:58:49.879132+0000","last_active":"2024-04-25T09:58:49.883755+0000","last_peered":"2024-04-25T09:58:49.883755+0000","last_clean":"2024-04-25T09:58:49.883755+0000","last_became_active":"2024-04-25T09:58:49.878844+0000","last_became_peered":"2024-04-25T09:58:49.878844+0000","last_unstale":"2024-04-25T09:58:49.883755+0000","last_undegraded":"2024-04-25T09:58:49.883755+0000","last_fullsized":"2024-04-25T09:58:49.883755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:49.217657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.865723+0000","last_change":"2024-04-25T09:58:49.862989+0000","last_active":"2024-04-25T09:58:49.865723+0000","last_peered":"2024-04-25T09:58:49.865723+0000","last_clean":"2024-04-25T09:58:49.865723+0000","last_became_active":"2024-04-25T09:58:49.862697+0000","last_became_peered":"2024-04-25T09:58:49.862697+0000","last_unstale":"2024-04-25T09:58:49.865723+0000","last_undegraded":"2024-04-25T09:58:49.865723+0000","last_fullsized":"2024-04-25T09:58:49.865723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:09.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880834+0000","last_change":"2024-04-25T09:58:49.875506+0000","last_active":"2024-04-25T09:58:49.880834+0000","last_peered":"2024-04-25T09:58:49.880834+0000","last_clean":"2024-04-25T09:58:49.880834+0000","last_became_active":"2024-04-25T09:58:49.875202+0000","last_became_peered":"2024-04-25T09:58:49.875202+0000","last_unstale":"2024-04-25T09:58:49.880834+0000","last_undegraded":"2024-04-25T09:58:49.880834+0000","last_fullsized":"2024-04-25T09:58:49.880834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:28.823821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885479+0000","last_change":"2024-04-25T09:58:49.884665+0000","last_active":"2024-04-25T09:58:49.885479+0000","last_peered":"2024-04-25T09:58:49.885479+0000","last_clean":"2024-04-25T09:58:49.885479+0000","last_became_active":"2024-04-25T09:58:49.883976+0000","last_became_peered":"2024-04-25T09:58:49.883976+0000","last_unstale":"2024-04-25T09:58:49.885479+0000","last_undegraded":"2024-04-25T09:58:49.885479+0000","last_fullsized":"2024-04-25T09:58:49.885479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:31.065429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880786+0000","last_change":"2024-04-25T09:58:49.875945+0000","last_active":"2024-04-25T09:58:49.880786+0000","last_peered":"2024-04-25T09:58:49.880786+0000","last_clean":"2024-04-25T09:58:49.880786+0000","last_became_active":"2024-04-25T09:58:49.875661+0000","last_became_peered":"2024-04-25T09:58:49.875661+0000","last_unstale":"2024-04-25T09:58:49.880786+0000","last_undegraded":"2024-04-25T09:58:49.880786+0000","last_fullsized":"2024-04-25T09:58:49.880786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.136005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.886019+0000","last_change":"2024-04-25T09:58:49.882170+0000","last_active":"2024-04-25T09:58:49.886019+0000","last_peered":"2024-04-25T09:58:49.886019+0000","last_clean":"2024-04-25T09:58:49.886019+0000","last_became_active":"2024-04-25T09:58:49.881887+0000","last_became_peered":"2024-04-25T09:58:49.881887+0000","last_unstale":"2024-04-25T09:58:49.886019+0000","last_undegraded":"2024-04-25T09:58:49.886019+0000","last_fullsized":"2024-04-25T09:58:49.886019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.656639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.867055+0000","last_change":"2024-04-25T09:58:49.865752+0000","last_active":"2024-04-25T09:58:49.867055+0000","last_peered":"2024-04-25T09:58:49.867055+0000","last_clean":"2024-04-25T09:58:49.867055+0000","last_became_active":"2024-04-25T09:58:49.865480+0000","last_became_peered":"2024-04-25T09:58:49.865480+0000","last_unstale":"2024-04-25T09:58:49.867055+0000","last_undegraded":"2024-04-25T09:58:49.867055+0000","last_fullsized":"2024-04-25T09:58:49.867055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:19.425038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879852+0000","last_change":"2024-04-25T09:58:49.877322+0000","last_active":"2024-04-25T09:58:49.879852+0000","last_peered":"2024-04-25T09:58:49.879852+0000","last_clean":"2024-04-25T09:58:49.879852+0000","last_became_active":"2024-04-25T09:58:49.876991+0000","last_became_peered":"2024-04-25T09:58:49.876991+0000","last_unstale":"2024-04-25T09:58:49.879852+0000","last_undegraded":"2024-04-25T09:58:49.879852+0000","last_fullsized":"2024-04-25T09:58:49.879852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.380686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.869742+0000","last_change":"2024-04-25T09:58:49.865189+0000","last_active":"2024-04-25T09:58:49.869742+0000","last_peered":"2024-04-25T09:58:49.869742+0000","last_clean":"2024-04-25T09:58:49.869742+0000","last_became_active":"2024-04-25T09:58:49.864898+0000","last_became_peered":"2024-04-25T09:58:49.864898+0000","last_unstale":"2024-04-25T09:58:49.869742+0000","last_undegraded":"2024-04-25T09:58:49.869742+0000","last_fullsized":"2024-04-25T09:58:49.869742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:40.141095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883472+0000","last_change":"2024-04-25T09:58:49.879974+0000","last_active":"2024-04-25T09:58:49.883472+0000","last_peered":"2024-04-25T09:58:49.883472+0000","last_clean":"2024-04-25T09:58:49.883472+0000","last_became_active":"2024-04-25T09:58:49.879710+0000","last_became_peered":"2024-04-25T09:58:49.879710+0000","last_unstale":"2024-04-25T09:58:49.883472+0000","last_undegraded":"2024-04-25T09:58:49.883472+0000","last_fullsized":"2024-04-25T09:58:49.883472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:56.581674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873824+0000","last_change":"2024-04-25T09:58:49.869032+0000","last_active":"2024-04-25T09:58:49.873824+0000","last_peered":"2024-04-25T09:58:49.873824+0000","last_clean":"2024-04-25T09:58:49.873824+0000","last_became_active":"2024-04-25T09:58:49.867788+0000","last_became_peered":"2024-04-25T09:58:49.867788+0000","last_unstale":"2024-04-25T09:58:49.873824+0000","last_undegraded":"2024-04-25T09:58:49.873824+0000","last_fullsized":"2024-04-25T09:58:49.873824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:56.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.874679+0000","last_change":"2024-04-25T09:58:49.870312+0000","last_active":"2024-04-25T09:58:49.874679+0000","last_peered":"2024-04-25T09:58:49.874679+0000","last_clean":"2024-04-25T09:58:49.874679+0000","last_became_active":"2024-04-25T09:58:49.870069+0000","last_became_peered":"2024-04-25T09:58:49.870069+0000","last_unstale":"2024-04-25T09:58:49.874679+0000","last_undegraded":"2024-04-25T09:58:49.874679+0000","last_fullsized":"2024-04-25T09:58:49.874679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:56.215164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883322+0000","last_change":"2024-04-25T09:58:49.880978+0000","last_active":"2024-04-25T09:58:49.883322+0000","last_peered":"2024-04-25T09:58:49.883322+0000","last_clean":"2024-04-25T09:58:49.883322+0000","last_became_active":"2024-04-25T09:58:49.880701+0000","last_became_peered":"2024-04-25T09:58:49.880701+0000","last_unstale":"2024-04-25T09:58:49.883322+0000","last_undegraded":"2024-04-25T09:58:49.883322+0000","last_fullsized":"2024-04-25T09:58:49.883322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.041253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881150+0000","last_change":"2024-04-25T09:58:49.877242+0000","last_active":"2024-04-25T09:58:49.881150+0000","last_peered":"2024-04-25T09:58:49.881150+0000","last_clean":"2024-04-25T09:58:49.881150+0000","last_became_active":"2024-04-25T09:58:49.875314+0000","last_became_peered":"2024-04-25T09:58:49.875314+0000","last_unstale":"2024-04-25T09:58:49.881150+0000","last_undegraded":"2024-04-25T09:58:49.881150+0000","last_fullsized":"2024-04-25T09:58:49.881150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:57.767275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884020+0000","last_change":"2024-04-25T09:58:49.881593+0000","last_active":"2024-04-25T09:58:49.884020+0000","last_peered":"2024-04-25T09:58:49.884020+0000","last_clean":"2024-04-25T09:58:49.884020+0000","last_became_active":"2024-04-25T09:58:49.881158+0000","last_became_peered":"2024-04-25T09:58:49.881158+0000","last_unstale":"2024-04-25T09:58:49.884020+0000","last_undegraded":"2024-04-25T09:58:49.884020+0000","last_fullsized":"2024-04-25T09:58:49.884020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:24.930190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866008+0000","last_change":"2024-04-25T09:58:49.865005+0000","last_active":"2024-04-25T09:58:49.866008+0000","last_peered":"2024-04-25T09:58:49.866008+0000","last_clean":"2024-04-25T09:58:49.866008+0000","last_became_active":"2024-04-25T09:58:49.864707+0000","last_became_peered":"2024-04-25T09:58:49.864707+0000","last_unstale":"2024-04-25T09:58:49.866008+0000","last_undegraded":"2024-04-25T09:58:49.866008+0000","last_fullsized":"2024-04-25T09:58:49.866008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:04.618516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885512+0000","last_change":"2024-04-25T09:58:49.882016+0000","last_active":"2024-04-25T09:58:49.885512+0000","last_peered":"2024-04-25T09:58:49.885512+0000","last_clean":"2024-04-25T09:58:49.885512+0000","last_became_active":"2024-04-25T09:58:49.879297+0000","last_became_peered":"2024-04-25T09:58:49.879297+0000","last_unstale":"2024-04-25T09:58:49.885512+0000","last_undegraded":"2024-04-25T09:58:49.885512+0000","last_fullsized":"2024-04-25T09:58:49.885512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:28.823821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873211+0000","last_change":"2024-04-25T09:58:49.871311+0000","last_active":"2024-04-25T09:58:49.873211+0000","last_peered":"2024-04-25T09:58:49.873211+0000","last_clean":"2024-04-25T09:58:49.873211+0000","last_became_active":"2024-04-25T09:58:49.871045+0000","last_became_peered":"2024-04-25T09:58:49.871045+0000","last_unstale":"2024-04-25T09:58:49.873211+0000","last_undegraded":"2024-04-25T09:58:49.873211+0000","last_fullsized":"2024-04-25T09:58:49.873211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:54.538905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872509+0000","last_change":"2024-04-25T09:58:49.871089+0000","last_active":"2024-04-25T09:58:49.872509+0000","last_peered":"2024-04-25T09:58:49.872509+0000","last_clean":"2024-04-25T09:58:49.872509+0000","last_became_active":"2024-04-25T09:58:49.870817+0000","last_became_peered":"2024-04-25T09:58:49.870817+0000","last_unstale":"2024-04-25T09:58:49.872509+0000","last_undegraded":"2024-04-25T09:58:49.872509+0000","last_fullsized":"2024-04-25T09:58:49.872509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.600001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872714+0000","last_change":"2024-04-25T09:58:49.867612+0000","last_active":"2024-04-25T09:58:49.872714+0000","last_peered":"2024-04-25T09:58:49.872714+0000","last_clean":"2024-04-25T09:58:49.872714+0000","last_became_active":"2024-04-25T09:58:49.867238+0000","last_became_peered":"2024-04-25T09:58:49.867238+0000","last_unstale":"2024-04-25T09:58:49.872714+0000","last_undegraded":"2024-04-25T09:58:49.872714+0000","last_fullsized":"2024-04-25T09:58:49.872714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.380686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883300+0000","last_change":"2024-04-25T09:58:49.880743+0000","last_active":"2024-04-25T09:58:49.883300+0000","last_peered":"2024-04-25T09:58:49.883300+0000","last_clean":"2024-04-25T09:58:49.883300+0000","last_became_active":"2024-04-25T09:58:49.880393+0000","last_became_peered":"2024-04-25T09:58:49.880393+0000","last_unstale":"2024-04-25T09:58:49.883300+0000","last_undegraded":"2024-04-25T09:58:49.883300+0000","last_fullsized":"2024-04-25T09:58:49.883300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:06.915021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881355+0000","last_change":"2024-04-25T09:58:49.878361+0000","last_active":"2024-04-25T09:58:49.881355+0000","last_peered":"2024-04-25T09:58:49.881355+0000","last_clean":"2024-04-25T09:58:49.881355+0000","last_became_active":"2024-04-25T09:58:49.877352+0000","last_became_peered":"2024-04-25T09:58:49.877352+0000","last_unstale":"2024-04-25T09:58:49.881355+0000","last_undegraded":"2024-04-25T09:58:49.881355+0000","last_fullsized":"2024-04-25T09:58:49.881355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.715358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872665+0000","last_change":"2024-04-25T09:58:49.869447+0000","last_active":"2024-04-25T09:58:49.872665+0000","last_peered":"2024-04-25T09:58:49.872665+0000","last_clean":"2024-04-25T09:58:49.872665+0000","last_became_active":"2024-04-25T09:58:49.869095+0000","last_became_peered":"2024-04-25T09:58:49.869095+0000","last_unstale":"2024-04-25T09:58:49.872665+0000","last_undegraded":"2024-04-25T09:58:49.872665+0000","last_fullsized":"2024-04-25T09:58:49.872665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.715358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882534+0000","last_change":"2024-04-25T09:58:49.875724+0000","last_active":"2024-04-25T09:58:49.882534+0000","last_peered":"2024-04-25T09:58:49.882534+0000","last_clean":"2024-04-25T09:58:49.882534+0000","last_became_active":"2024-04-25T09:58:49.874456+0000","last_became_peered":"2024-04-25T09:58:49.874456+0000","last_unstale":"2024-04-25T09:58:49.882534+0000","last_undegraded":"2024-04-25T09:58:49.882534+0000","last_fullsized":"2024-04-25T09:58:49.882534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.629636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.869721+0000","last_change":"2024-04-25T09:58:49.864476+0000","last_active":"2024-04-25T09:58:49.869721+0000","last_peered":"2024-04-25T09:58:49.869721+0000","last_clean":"2024-04-25T09:58:49.869721+0000","last_became_active":"2024-04-25T09:58:49.864002+0000","last_became_peered":"2024-04-25T09:58:49.864002+0000","last_unstale":"2024-04-25T09:58:49.869721+0000","last_undegraded":"2024-04-25T09:58:49.869721+0000","last_fullsized":"2024-04-25T09:58:49.869721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:54.538905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882729+0000","last_change":"2024-04-25T09:58:49.874988+0000","last_active":"2024-04-25T09:58:49.882729+0000","last_peered":"2024-04-25T09:58:49.882729+0000","last_clean":"2024-04-25T09:58:49.882729+0000","last_became_active":"2024-04-25T09:58:49.873842+0000","last_became_peered":"2024-04-25T09:58:49.873842+0000","last_unstale":"2024-04-25T09:58:49.882729+0000","last_undegraded":"2024-04-25T09:58:49.882729+0000","last_fullsized":"2024-04-25T09:58:49.882729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:42.918059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885809+0000","last_change":"2024-04-25T09:58:48.858763+0000","last_active":"2024-04-25T09:58:49.885809+0000","last_peered":"2024-04-25T09:58:49.885809+0000","last_clean":"2024-04-25T09:58:49.885809+0000","last_became_active":"2024-04-25T09:58:48.858498+0000","last_became_peered":"2024-04-25T09:58:48.858498+0000","last_unstale":"2024-04-25T09:58:49.885809+0000","last_undegraded":"2024-04-25T09:58:49.885809+0000","last_fullsized":"2024-04-25T09:58:49.885809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:10.787548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872553+0000","last_change":"2024-04-25T09:58:49.870634+0000","last_active":"2024-04-25T09:58:49.872553+0000","last_peered":"2024-04-25T09:58:49.872553+0000","last_clean":"2024-04-25T09:58:49.872553+0000","last_became_active":"2024-04-25T09:58:49.870241+0000","last_became_peered":"2024-04-25T09:58:49.870241+0000","last_unstale":"2024-04-25T09:58:49.872553+0000","last_undegraded":"2024-04-25T09:58:49.872553+0000","last_fullsized":"2024-04-25T09:58:49.872553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:24.362109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879056+0000","last_change":"2024-04-25T09:58:48.858296+0000","last_active":"2024-04-25T09:58:49.879056+0000","last_peered":"2024-04-25T09:58:49.879056+0000","last_clean":"2024-04-25T09:58:49.879056+0000","last_became_active":"2024-04-25T09:58:48.857952+0000","last_became_peered":"2024-04-25T09:58:48.857952+0000","last_unstale":"2024-04-25T09:58:49.879056+0000","last_undegraded":"2024-04-25T09:58:49.879056+0000","last_fullsized":"2024-04-25T09:58:49.879056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:38.892401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.868613+0000","last_change":"2024-04-25T09:58:49.867512+0000","last_active":"2024-04-25T09:58:49.868613+0000","last_peered":"2024-04-25T09:58:49.868613+0000","last_clean":"2024-04-25T09:58:49.868613+0000","last_became_active":"2024-04-25T09:58:49.867237+0000","last_became_peered":"2024-04-25T09:58:49.867237+0000","last_unstale":"2024-04-25T09:58:49.868613+0000","last_undegraded":"2024-04-25T09:58:49.868613+0000","last_fullsized":"2024-04-25T09:58:49.868613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:36.615889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883069+0000","last_change":"2024-04-25T09:58:48.864168+0000","last_active":"2024-04-25T09:58:49.883069+0000","last_peered":"2024-04-25T09:58:49.883069+0000","last_clean":"2024-04-25T09:58:49.883069+0000","last_became_active":"2024-04-25T09:58:48.863890+0000","last_became_peered":"2024-04-25T09:58:48.863890+0000","last_unstale":"2024-04-25T09:58:49.883069+0000","last_undegraded":"2024-04-25T09:58:49.883069+0000","last_fullsized":"2024-04-25T09:58:49.883069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:07.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.869998+0000","last_change":"2024-04-25T09:58:49.868845+0000","last_active":"2024-04-25T09:58:49.869998+0000","last_peered":"2024-04-25T09:58:49.869998+0000","last_clean":"2024-04-25T09:58:49.869998+0000","last_became_active":"2024-04-25T09:58:49.868480+0000","last_became_peered":"2024-04-25T09:58:49.868480+0000","last_unstale":"2024-04-25T09:58:49.869998+0000","last_undegraded":"2024-04-25T09:58:49.869998+0000","last_fullsized":"2024-04-25T09:58:49.869998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:02.681291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879268+0000","last_change":"2024-04-25T09:58:48.860848+0000","last_active":"2024-04-25T09:58:49.879268+0000","last_peered":"2024-04-25T09:58:49.879268+0000","last_clean":"2024-04-25T09:58:49.879268+0000","last_became_active":"2024-04-25T09:58:48.860335+0000","last_became_peered":"2024-04-25T09:58:48.860335+0000","last_unstale":"2024-04-25T09:58:49.879268+0000","last_undegraded":"2024-04-25T09:58:49.879268+0000","last_fullsized":"2024-04-25T09:58:49.879268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:08.275929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.886493+0000","last_change":"2024-04-25T09:58:49.885087+0000","last_active":"2024-04-25T09:58:49.886493+0000","last_peered":"2024-04-25T09:58:49.886493+0000","last_clean":"2024-04-25T09:58:49.886493+0000","last_became_active":"2024-04-25T09:58:49.884815+0000","last_became_peered":"2024-04-25T09:58:49.884815+0000","last_unstale":"2024-04-25T09:58:49.886493+0000","last_undegraded":"2024-04-25T09:58:49.886493+0000","last_fullsized":"2024-04-25T09:58:49.886493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:57.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.889180+0000","last_change":"2024-04-25T09:58:48.859800+0000","last_active":"2024-04-25T09:58:49.889180+0000","last_peered":"2024-04-25T09:58:49.889180+0000","last_clean":"2024-04-25T09:58:49.889180+0000","last_became_active":"2024-04-25T09:58:48.859535+0000","last_became_peered":"2024-04-25T09:58:48.859535+0000","last_unstale":"2024-04-25T09:58:49.889180+0000","last_undegraded":"2024-04-25T09:58:49.889180+0000","last_fullsized":"2024-04-25T09:58:49.889180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:38.747671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885458+0000","last_change":"2024-04-25T09:58:49.884455+0000","last_active":"2024-04-25T09:58:49.885458+0000","last_peered":"2024-04-25T09:58:49.885458+0000","last_clean":"2024-04-25T09:58:49.885458+0000","last_became_active":"2024-04-25T09:58:49.883633+0000","last_became_peered":"2024-04-25T09:58:49.883633+0000","last_unstale":"2024-04-25T09:58:49.885458+0000","last_undegraded":"2024-04-25T09:58:49.885458+0000","last_fullsized":"2024-04-25T09:58:49.885458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:19.425038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881521+0000","last_change":"2024-04-25T09:58:48.863123+0000","last_active":"2024-04-25T09:58:49.881521+0000","last_peered":"2024-04-25T09:58:49.881521+0000","last_clean":"2024-04-25T09:58:49.881521+0000","last_became_active":"2024-04-25T09:58:48.862845+0000","last_became_peered":"2024-04-25T09:58:48.862845+0000","last_unstale":"2024-04-25T09:58:49.881521+0000","last_undegraded":"2024-04-25T09:58:49.881521+0000","last_fullsized":"2024-04-25T09:58:49.881521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:23.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.864564+0000","last_change":"2024-04-25T09:58:49.862244+0000","last_active":"2024-04-25T09:58:49.864564+0000","last_peered":"2024-04-25T09:58:49.864564+0000","last_clean":"2024-04-25T09:58:49.864564+0000","last_became_active":"2024-04-25T09:58:49.861988+0000","last_became_peered":"2024-04-25T09:58:49.861988+0000","last_unstale":"2024-04-25T09:58:49.864564+0000","last_undegraded":"2024-04-25T09:58:49.864564+0000","last_fullsized":"2024-04-25T09:58:49.864564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:08.268371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.864437+0000","last_change":"2024-04-25T09:58:48.859923+0000","last_active":"2024-04-25T09:58:49.864437+0000","last_peered":"2024-04-25T09:58:49.864437+0000","last_clean":"2024-04-25T09:58:49.864437+0000","last_became_active":"2024-04-25T09:58:48.859642+0000","last_became_peered":"2024-04-25T09:58:48.859642+0000","last_unstale":"2024-04-25T09:58:49.864437+0000","last_undegraded":"2024-04-25T09:58:49.864437+0000","last_fullsized":"2024-04-25T09:58:49.864437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.926531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.888990+0000","last_change":"2024-04-25T09:58:49.884342+0000","last_active":"2024-04-25T09:58:49.888990+0000","last_peered":"2024-04-25T09:58:49.888990+0000","last_clean":"2024-04-25T09:58:49.888990+0000","last_became_active":"2024-04-25T09:58:49.884053+0000","last_became_peered":"2024-04-25T09:58:49.884053+0000","last_unstale":"2024-04-25T09:58:49.888990+0000","last_undegraded":"2024-04-25T09:58:49.888990+0000","last_fullsized":"2024-04-25T09:58:49.888990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:29.518080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859916+0000","last_change":"2024-04-25T09:58:48.855119+0000","last_active":"2024-04-25T09:58:49.859916+0000","last_peered":"2024-04-25T09:58:49.859916+0000","last_clean":"2024-04-25T09:58:49.859916+0000","last_became_active":"2024-04-25T09:58:48.854493+0000","last_became_peered":"2024-04-25T09:58:48.854493+0000","last_unstale":"2024-04-25T09:58:49.859916+0000","last_undegraded":"2024-04-25T09:58:49.859916+0000","last_fullsized":"2024-04-25T09:58:49.859916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:00.205579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882265+0000","last_change":"2024-04-25T09:58:49.879588+0000","last_active":"2024-04-25T09:58:49.882265+0000","last_peered":"2024-04-25T09:58:49.882265+0000","last_clean":"2024-04-25T09:58:49.882265+0000","last_became_active":"2024-04-25T09:58:49.878830+0000","last_became_peered":"2024-04-25T09:58:49.878830+0000","last_unstale":"2024-04-25T09:58:49.882265+0000","last_undegraded":"2024-04-25T09:58:49.882265+0000","last_fullsized":"2024-04-25T09:58:49.882265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.600001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862199+0000","last_change":"2024-04-25T09:58:48.858713+0000","last_active":"2024-04-25T09:58:49.862199+0000","last_peered":"2024-04-25T09:58:49.862199+0000","last_clean":"2024-04-25T09:58:49.862199+0000","last_became_active":"2024-04-25T09:58:48.858264+0000","last_became_peered":"2024-04-25T09:58:48.858264+0000","last_unstale":"2024-04-25T09:58:49.862199+0000","last_undegraded":"2024-04-25T09:58:49.862199+0000","last_fullsized":"2024-04-25T09:58:49.862199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:23.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866978+0000","last_change":"2024-04-25T09:58:49.863067+0000","last_active":"2024-04-25T09:58:49.866978+0000","last_peered":"2024-04-25T09:58:49.866978+0000","last_clean":"2024-04-25T09:58:49.866978+0000","last_became_active":"2024-04-25T09:58:49.862796+0000","last_became_peered":"2024-04-25T09:58:49.862796+0000","last_unstale":"2024-04-25T09:58:49.866978+0000","last_undegraded":"2024-04-25T09:58:49.866978+0000","last_fullsized":"2024-04-25T09:58:49.866978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.715358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.857560+0000","last_change":"2024-04-25T09:58:48.856626+0000","last_active":"2024-04-25T09:58:49.857560+0000","last_peered":"2024-04-25T09:58:49.857560+0000","last_clean":"2024-04-25T09:58:49.857560+0000","last_became_active":"2024-04-25T09:58:48.856336+0000","last_became_peered":"2024-04-25T09:58:48.856336+0000","last_unstale":"2024-04-25T09:58:49.857560+0000","last_undegraded":"2024-04-25T09:58:49.857560+0000","last_fullsized":"2024-04-25T09:58:49.857560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:39.134338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.867487+0000","last_change":"2024-04-25T09:58:49.866910+0000","last_active":"2024-04-25T09:58:49.867487+0000","last_peered":"2024-04-25T09:58:49.867487+0000","last_clean":"2024-04-25T09:58:49.867487+0000","last_became_active":"2024-04-25T09:58:49.866648+0000","last_became_peered":"2024-04-25T09:58:49.866648+0000","last_unstale":"2024-04-25T09:58:49.867487+0000","last_undegraded":"2024-04-25T09:58:49.867487+0000","last_fullsized":"2024-04-25T09:58:49.867487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:31.065429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884298+0000","last_change":"2024-04-25T09:58:48.857036+0000","last_active":"2024-04-25T09:58:49.884298+0000","last_peered":"2024-04-25T09:58:49.884298+0000","last_clean":"2024-04-25T09:58:49.884298+0000","last_became_active":"2024-04-25T09:58:48.856770+0000","last_became_peered":"2024-04-25T09:58:48.856770+0000","last_unstale":"2024-04-25T09:58:49.884298+0000","last_undegraded":"2024-04-25T09:58:49.884298+0000","last_fullsized":"2024-04-25T09:58:49.884298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:57.643448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882147+0000","last_change":"2024-04-25T09:58:49.873531+0000","last_active":"2024-04-25T09:58:49.882147+0000","last_peered":"2024-04-25T09:58:49.882147+0000","last_clean":"2024-04-25T09:58:49.882147+0000","last_became_active":"2024-04-25T09:58:49.873230+0000","last_became_peered":"2024-04-25T09:58:49.873230+0000","last_unstale":"2024-04-25T09:58:49.882147+0000","last_undegraded":"2024-04-25T09:58:49.882147+0000","last_fullsized":"2024-04-25T09:58:49.882147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:51.081158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.891989+0000","last_change":"2024-04-25T09:58:48.859642+0000","last_active":"2024-04-25T09:58:49.891989+0000","last_peered":"2024-04-25T09:58:49.891989+0000","last_clean":"2024-04-25T09:58:49.891989+0000","last_became_active":"2024-04-25T09:58:48.859342+0000","last_became_peered":"2024-04-25T09:58:48.859342+0000","last_unstale":"2024-04-25T09:58:49.891989+0000","last_undegraded":"2024-04-25T09:58:49.891989+0000","last_fullsized":"2024-04-25T09:58:49.891989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:55.253441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866766+0000","last_change":"2024-04-25T09:58:49.862416+0000","last_active":"2024-04-25T09:58:49.866766+0000","last_peered":"2024-04-25T09:58:49.866766+0000","last_clean":"2024-04-25T09:58:49.866766+0000","last_became_active":"2024-04-25T09:58:49.862141+0000","last_became_peered":"2024-04-25T09:58:49.862141+0000","last_unstale":"2024-04-25T09:58:49.866766+0000","last_undegraded":"2024-04-25T09:58:49.866766+0000","last_fullsized":"2024-04-25T09:58:49.866766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.380686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881629+0000","last_change":"2024-04-25T09:58:48.856870+0000","last_active":"2024-04-25T09:58:49.881629+0000","last_peered":"2024-04-25T09:58:49.881629+0000","last_clean":"2024-04-25T09:58:49.881629+0000","last_became_active":"2024-04-25T09:58:48.856380+0000","last_became_peered":"2024-04-25T09:58:48.856380+0000","last_unstale":"2024-04-25T09:58:49.881629+0000","last_undegraded":"2024-04-25T09:58:49.881629+0000","last_fullsized":"2024-04-25T09:58:49.881629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:27.889161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883689+0000","last_change":"2024-04-25T09:58:49.879122+0000","last_active":"2024-04-25T09:58:49.883689+0000","last_peered":"2024-04-25T09:58:49.883689+0000","last_clean":"2024-04-25T09:58:49.883689+0000","last_became_active":"2024-04-25T09:58:49.878848+0000","last_became_peered":"2024-04-25T09:58:49.878848+0000","last_unstale":"2024-04-25T09:58:49.883689+0000","last_undegraded":"2024-04-25T09:58:49.883689+0000","last_fullsized":"2024-04-25T09:58:49.883689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.629636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.860470+0000","last_change":"2024-04-25T09:58:48.859890+0000","last_active":"2024-04-25T09:58:49.860470+0000","last_peered":"2024-04-25T09:58:49.860470+0000","last_clean":"2024-04-25T09:58:49.860470+0000","last_became_active":"2024-04-25T09:58:48.859641+0000","last_became_peered":"2024-04-25T09:58:48.859641+0000","last_unstale":"2024-04-25T09:58:49.860470+0000","last_undegraded":"2024-04-25T09:58:49.860470+0000","last_fullsized":"2024-04-25T09:58:49.860470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.926531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.878372+0000","last_change":"2024-04-25T09:58:49.874843+0000","last_active":"2024-04-25T09:58:49.878372+0000","last_peered":"2024-04-25T09:58:49.878372+0000","last_clean":"2024-04-25T09:58:49.878372+0000","last_became_active":"2024-04-25T09:58:49.874550+0000","last_became_peered":"2024-04-25T09:58:49.874550+0000","last_unstale":"2024-04-25T09:58:49.878372+0000","last_undegraded":"2024-04-25T09:58:49.878372+0000","last_fullsized":"2024-04-25T09:58:49.878372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:42.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.860432+0000","last_change":"2024-04-25T09:58:48.863107+0000","last_active":"2024-04-25T09:58:49.860432+0000","last_peered":"2024-04-25T09:58:49.860432+0000","last_clean":"2024-04-25T09:58:49.860432+0000","last_became_active":"2024-04-25T09:58:48.862777+0000","last_became_peered":"2024-04-25T09:58:48.862777+0000","last_unstale":"2024-04-25T09:58:49.860432+0000","last_undegraded":"2024-04-25T09:58:49.860432+0000","last_fullsized":"2024-04-25T09:58:49.860432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:01.135899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880837+0000","last_change":"2024-04-25T09:58:49.878037+0000","last_active":"2024-04-25T09:58:49.880837+0000","last_peered":"2024-04-25T09:58:49.880837+0000","last_clean":"2024-04-25T09:58:49.880837+0000","last_became_active":"2024-04-25T09:58:49.877719+0000","last_became_peered":"2024-04-25T09:58:49.877719+0000","last_unstale":"2024-04-25T09:58:49.880837+0000","last_undegraded":"2024-04-25T09:58:49.880837+0000","last_fullsized":"2024-04-25T09:58:49.880837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:42.918059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.887096+0000","last_change":"2024-04-25T09:58:48.857534+0000","last_active":"2024-04-25T09:58:49.887096+0000","last_peered":"2024-04-25T09:58:49.887096+0000","last_clean":"2024-04-25T09:58:49.887096+0000","last_became_active":"2024-04-25T09:58:48.857290+0000","last_became_peered":"2024-04-25T09:58:48.857290+0000","last_unstale":"2024-04-25T09:58:49.887096+0000","last_undegraded":"2024-04-25T09:58:49.887096+0000","last_fullsized":"2024-04-25T09:58:49.887096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:29.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.875593+0000","last_change":"2024-04-25T09:58:49.872187+0000","last_active":"2024-04-25T09:58:49.875593+0000","last_peered":"2024-04-25T09:58:49.875593+0000","last_clean":"2024-04-25T09:58:49.875593+0000","last_became_active":"2024-04-25T09:58:49.871931+0000","last_became_peered":"2024-04-25T09:58:49.871931+0000","last_unstale":"2024-04-25T09:58:49.875593+0000","last_undegraded":"2024-04-25T09:58:49.875593+0000","last_fullsized":"2024-04-25T09:58:49.875593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:57.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884074+0000","last_change":"2024-04-25T09:58:48.861459+0000","last_active":"2024-04-25T09:58:49.884074+0000","last_peered":"2024-04-25T09:58:49.884074+0000","last_clean":"2024-04-25T09:58:49.884074+0000","last_became_active":"2024-04-25T09:58:48.861175+0000","last_became_peered":"2024-04-25T09:58:48.861175+0000","last_unstale":"2024-04-25T09:58:49.884074+0000","last_undegraded":"2024-04-25T09:58:49.884074+0000","last_fullsized":"2024-04-25T09:58:49.884074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:27.889161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883652+0000","last_change":"2024-04-25T09:58:49.880305+0000","last_active":"2024-04-25T09:58:49.883652+0000","last_peered":"2024-04-25T09:58:49.883652+0000","last_clean":"2024-04-25T09:58:49.883652+0000","last_became_active":"2024-04-25T09:58:49.880049+0000","last_became_peered":"2024-04-25T09:58:49.880049+0000","last_unstale":"2024-04-25T09:58:49.883652+0000","last_undegraded":"2024-04-25T09:58:49.883652+0000","last_fullsized":"2024-04-25T09:58:49.883652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:35.809943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.889180+0000","last_change":"2024-04-25T09:58:48.847300+0000","last_active":"2024-04-25T09:58:49.889180+0000","last_peered":"2024-04-25T09:58:49.889180+0000","last_clean":"2024-04-25T09:58:49.889180+0000","last_became_active":"2024-04-25T09:58:48.846972+0000","last_became_peered":"2024-04-25T09:58:48.846972+0000","last_unstale":"2024-04-25T09:58:49.889180+0000","last_undegraded":"2024-04-25T09:58:49.889180+0000","last_fullsized":"2024-04-25T09:58:49.889180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:45.276573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883999+0000","last_change":"2024-04-25T09:58:49.881395+0000","last_active":"2024-04-25T09:58:49.883999+0000","last_peered":"2024-04-25T09:58:49.883999+0000","last_clean":"2024-04-25T09:58:49.883999+0000","last_became_active":"2024-04-25T09:58:49.880893+0000","last_became_peered":"2024-04-25T09:58:49.880893+0000","last_unstale":"2024-04-25T09:58:49.883999+0000","last_undegraded":"2024-04-25T09:58:49.883999+0000","last_fullsized":"2024-04-25T09:58:49.883999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:36.208100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884682+0000","last_change":"2024-04-25T09:58:48.857850+0000","last_active":"2024-04-25T09:58:49.884682+0000","last_peered":"2024-04-25T09:58:49.884682+0000","last_clean":"2024-04-25T09:58:49.884682+0000","last_became_active":"2024-04-25T09:58:48.857588+0000","last_became_peered":"2024-04-25T09:58:48.857588+0000","last_unstale":"2024-04-25T09:58:49.884682+0000","last_undegraded":"2024-04-25T09:58:49.884682+0000","last_fullsized":"2024-04-25T09:58:49.884682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:45.874120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885212+0000","last_change":"2024-04-25T09:58:49.884415+0000","last_active":"2024-04-25T09:58:49.885212+0000","last_peered":"2024-04-25T09:58:49.885212+0000","last_clean":"2024-04-25T09:58:49.885212+0000","last_became_active":"2024-04-25T09:58:49.884170+0000","last_became_peered":"2024-04-25T09:58:49.884170+0000","last_unstale":"2024-04-25T09:58:49.885212+0000","last_undegraded":"2024-04-25T09:58:49.885212+0000","last_fullsized":"2024-04-25T09:58:49.885212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:12.604509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.868745+0000","last_change":"2024-04-25T09:58:48.854279+0000","last_active":"2024-04-25T09:58:49.868745+0000","last_peered":"2024-04-25T09:58:49.868745+0000","last_clean":"2024-04-25T09:58:49.868745+0000","last_became_active":"2024-04-25T09:58:48.853982+0000","last_became_peered":"2024-04-25T09:58:48.853982+0000","last_unstale":"2024-04-25T09:58:49.868745+0000","last_undegraded":"2024-04-25T09:58:49.868745+0000","last_fullsized":"2024-04-25T09:58:49.868745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:08.275929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880472+0000","last_change":"2024-04-25T09:58:49.873004+0000","last_active":"2024-04-25T09:58:49.880472+0000","last_peered":"2024-04-25T09:58:49.880472+0000","last_clean":"2024-04-25T09:58:49.880472+0000","last_became_active":"2024-04-25T09:58:49.871714+0000","last_became_peered":"2024-04-25T09:58:49.871714+0000","last_unstale":"2024-04-25T09:58:49.880472+0000","last_undegraded":"2024-04-25T09:58:49.880472+0000","last_fullsized":"2024-04-25T09:58:49.880472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:16.672276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.887508+0000","last_change":"2024-04-25T09:58:48.857386+0000","last_active":"2024-04-25T09:58:49.887508+0000","last_peered":"2024-04-25T09:58:49.887508+0000","last_clean":"2024-04-25T09:58:49.887508+0000","last_became_active":"2024-04-25T09:58:48.857092+0000","last_became_peered":"2024-04-25T09:58:48.857092+0000","last_unstale":"2024-04-25T09:58:49.887508+0000","last_undegraded":"2024-04-25T09:58:49.887508+0000","last_fullsized":"2024-04-25T09:58:49.887508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:27.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873694+0000","last_change":"2024-04-25T09:58:49.871837+0000","last_active":"2024-04-25T09:58:49.873694+0000","last_peered":"2024-04-25T09:58:49.873694+0000","last_clean":"2024-04-25T09:58:49.873694+0000","last_became_active":"2024-04-25T09:58:49.871581+0000","last_became_peered":"2024-04-25T09:58:49.871581+0000","last_unstale":"2024-04-25T09:58:49.873694+0000","last_undegraded":"2024-04-25T09:58:49.873694+0000","last_fullsized":"2024-04-25T09:58:49.873694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:40.141095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859691+0000","last_change":"2024-04-25T09:58:48.864266+0000","last_active":"2024-04-25T09:58:49.859691+0000","last_peered":"2024-04-25T09:58:49.859691+0000","last_clean":"2024-04-25T09:58:49.859691+0000","last_became_active":"2024-04-25T09:58:48.863962+0000","last_became_peered":"2024-04-25T09:58:48.863962+0000","last_unstale":"2024-04-25T09:58:49.859691+0000","last_undegraded":"2024-04-25T09:58:49.859691+0000","last_fullsized":"2024-04-25T09:58:49.859691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:24.593244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883765+0000","last_change":"2024-04-25T09:58:49.879900+0000","last_active":"2024-04-25T09:58:49.883765+0000","last_peered":"2024-04-25T09:58:49.883765+0000","last_clean":"2024-04-25T09:58:49.883765+0000","last_became_active":"2024-04-25T09:58:49.879653+0000","last_became_peered":"2024-04-25T09:58:49.879653+0000","last_unstale":"2024-04-25T09:58:49.883765+0000","last_undegraded":"2024-04-25T09:58:49.883765+0000","last_fullsized":"2024-04-25T09:58:49.883765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:15.168768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859560+0000","last_change":"2024-04-25T09:58:48.864351+0000","last_active":"2024-04-25T09:58:49.859560+0000","last_peered":"2024-04-25T09:58:49.859560+0000","last_clean":"2024-04-25T09:58:49.859560+0000","last_became_active":"2024-04-25T09:58:48.864022+0000","last_became_peered":"2024-04-25T09:58:48.864022+0000","last_unstale":"2024-04-25T09:58:49.859560+0000","last_undegraded":"2024-04-25T09:58:49.859560+0000","last_fullsized":"2024-04-25T09:58:49.859560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:54.708602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879353+0000","last_change":"2024-04-25T09:58:49.875240+0000","last_active":"2024-04-25T09:58:49.879353+0000","last_peered":"2024-04-25T09:58:49.879353+0000","last_clean":"2024-04-25T09:58:49.879353+0000","last_became_active":"2024-04-25T09:58:49.874793+0000","last_became_peered":"2024-04-25T09:58:49.874793+0000","last_unstale":"2024-04-25T09:58:49.879353+0000","last_undegraded":"2024-04-25T09:58:49.879353+0000","last_fullsized":"2024-04-25T09:58:49.879353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:04.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884292+0000","last_change":"2024-04-25T09:58:48.856595+0000","last_active":"2024-04-25T09:58:49.884292+0000","last_peered":"2024-04-25T09:58:49.884292+0000","last_clean":"2024-04-25T09:58:49.884292+0000","last_became_active":"2024-04-25T09:58:48.856243+0000","last_became_peered":"2024-04-25T09:58:48.856243+0000","last_unstale":"2024-04-25T09:58:49.884292+0000","last_undegraded":"2024-04-25T09:58:49.884292+0000","last_fullsized":"2024-04-25T09:58:49.884292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:53.532148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.886530+0000","last_change":"2024-04-25T09:58:49.881940+0000","last_active":"2024-04-25T09:58:49.886530+0000","last_peered":"2024-04-25T09:58:49.886530+0000","last_clean":"2024-04-25T09:58:49.886530+0000","last_became_active":"2024-04-25T09:58:49.881609+0000","last_became_peered":"2024-04-25T09:58:49.881609+0000","last_unstale":"2024-04-25T09:58:49.886530+0000","last_undegraded":"2024-04-25T09:58:49.886530+0000","last_fullsized":"2024-04-25T09:58:49.886530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:04.229506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866634+0000","last_change":"2024-04-25T09:58:48.864534+0000","last_active":"2024-04-25T09:58:49.866634+0000","last_peered":"2024-04-25T09:58:49.866634+0000","last_clean":"2024-04-25T09:58:49.866634+0000","last_became_active":"2024-04-25T09:58:48.864262+0000","last_became_peered":"2024-04-25T09:58:48.864262+0000","last_unstale":"2024-04-25T09:58:49.866634+0000","last_undegraded":"2024-04-25T09:58:49.866634+0000","last_fullsized":"2024-04-25T09:58:49.866634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:07.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.865729+0000","last_change":"2024-04-25T09:58:49.863198+0000","last_active":"2024-04-25T09:58:49.865729+0000","last_peered":"2024-04-25T09:58:49.865729+0000","last_clean":"2024-04-25T09:58:49.865729+0000","last_became_active":"2024-04-25T09:58:49.862904+0000","last_became_peered":"2024-04-25T09:58:49.862904+0000","last_unstale":"2024-04-25T09:58:49.865729+0000","last_undegraded":"2024-04-25T09:58:49.865729+0000","last_fullsized":"2024-04-25T09:58:49.865729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:12.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862124+0000","last_change":"2024-04-25T09:58:48.862663+0000","last_active":"2024-04-25T09:58:49.862124+0000","last_peered":"2024-04-25T09:58:49.862124+0000","last_clean":"2024-04-25T09:58:49.862124+0000","last_became_active":"2024-04-25T09:58:48.862366+0000","last_became_peered":"2024-04-25T09:58:48.862366+0000","last_unstale":"2024-04-25T09:58:49.862124+0000","last_undegraded":"2024-04-25T09:58:49.862124+0000","last_fullsized":"2024-04-25T09:58:49.862124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:29.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872923+0000","last_change":"2024-04-25T09:58:49.871194+0000","last_active":"2024-04-25T09:58:49.872923+0000","last_peered":"2024-04-25T09:58:49.872923+0000","last_clean":"2024-04-25T09:58:49.872923+0000","last_became_active":"2024-04-25T09:58:49.870762+0000","last_became_peered":"2024-04-25T09:58:49.870762+0000","last_unstale":"2024-04-25T09:58:49.872923+0000","last_undegraded":"2024-04-25T09:58:49.872923+0000","last_fullsized":"2024-04-25T09:58:49.872923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:28.895918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859020+0000","last_change":"2024-04-25T09:58:48.858993+0000","last_active":"2024-04-25T09:58:49.859020+0000","last_peered":"2024-04-25T09:58:49.859020+0000","last_clean":"2024-04-25T09:58:49.859020+0000","last_became_active":"2024-04-25T09:58:48.858719+0000","last_became_peered":"2024-04-25T09:58:48.858719+0000","last_unstale":"2024-04-25T09:58:49.859020+0000","last_undegraded":"2024-04-25T09:58:49.859020+0000","last_fullsized":"2024-04-25T09:58:49.859020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:10.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880917+0000","last_change":"2024-04-25T09:58:49.877968+0000","last_active":"2024-04-25T09:58:49.880917+0000","last_peered":"2024-04-25T09:58:49.880917+0000","last_clean":"2024-04-25T09:58:49.880917+0000","last_became_active":"2024-04-25T09:58:49.877689+0000","last_became_peered":"2024-04-25T09:58:49.877689+0000","last_unstale":"2024-04-25T09:58:49.880917+0000","last_undegraded":"2024-04-25T09:58:49.880917+0000","last_fullsized":"2024-04-25T09:58:49.880917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:51.081158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872937+0000","last_change":"2024-04-25T09:58:48.859743+0000","last_active":"2024-04-25T09:58:49.872937+0000","last_peered":"2024-04-25T09:58:49.872937+0000","last_clean":"2024-04-25T09:58:49.872937+0000","last_became_active":"2024-04-25T09:58:48.859205+0000","last_became_peered":"2024-04-25T09:58:48.859205+0000","last_unstale":"2024-04-25T09:58:49.872937+0000","last_undegraded":"2024-04-25T09:58:49.872937+0000","last_fullsized":"2024-04-25T09:58:49.872937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:03.407058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.891667+0000","last_change":"2024-04-25T09:58:49.886759+0000","last_active":"2024-04-25T09:58:49.891667+0000","last_peered":"2024-04-25T09:58:49.891667+0000","last_clean":"2024-04-25T09:58:49.891667+0000","last_became_active":"2024-04-25T09:58:49.886540+0000","last_became_peered":"2024-04-25T09:58:49.886540+0000","last_unstale":"2024-04-25T09:58:49.891667+0000","last_undegraded":"2024-04-25T09:58:49.891667+0000","last_fullsized":"2024-04-25T09:58:49.891667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:35.351668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862340+0000","last_change":"2024-04-25T09:58:48.859684+0000","last_active":"2024-04-25T09:58:49.862340+0000","last_peered":"2024-04-25T09:58:49.862340+0000","last_clean":"2024-04-25T09:58:49.862340+0000","last_became_active":"2024-04-25T09:58:48.859396+0000","last_became_peered":"2024-04-25T09:58:48.859396+0000","last_unstale":"2024-04-25T09:58:49.862340+0000","last_undegraded":"2024-04-25T09:58:49.862340+0000","last_fullsized":"2024-04-25T09:58:49.862340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:48.506129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.874197+0000","last_change":"2024-04-25T09:58:49.873526+0000","last_active":"2024-04-25T09:58:49.874197+0000","last_peered":"2024-04-25T09:58:49.874197+0000","last_clean":"2024-04-25T09:58:49.874197+0000","last_became_active":"2024-04-25T09:58:49.873257+0000","last_became_peered":"2024-04-25T09:58:49.873257+0000","last_unstale":"2024-04-25T09:58:49.874197+0000","last_undegraded":"2024-04-25T09:58:49.874197+0000","last_fullsized":"2024-04-25T09:58:49.874197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:42.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873172+0000","last_change":"2024-04-25T09:58:48.864595+0000","last_active":"2024-04-25T09:58:49.873172+0000","last_peered":"2024-04-25T09:58:49.873172+0000","last_clean":"2024-04-25T09:58:49.873172+0000","last_became_active":"2024-04-25T09:58:48.864323+0000","last_became_peered":"2024-04-25T09:58:48.864323+0000","last_unstale":"2024-04-25T09:58:49.873172+0000","last_undegraded":"2024-04-25T09:58:49.873172+0000","last_fullsized":"2024-04-25T09:58:49.873172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:56.760519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882627+0000","last_change":"2024-04-25T09:58:49.876262+0000","last_active":"2024-04-25T09:58:49.882627+0000","last_peered":"2024-04-25T09:58:49.882627+0000","last_clean":"2024-04-25T09:58:49.882627+0000","last_became_active":"2024-04-25T09:58:49.874737+0000","last_became_peered":"2024-04-25T09:58:49.874737+0000","last_unstale":"2024-04-25T09:58:49.882627+0000","last_undegraded":"2024-04-25T09:58:49.882627+0000","last_fullsized":"2024-04-25T09:58:49.882627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:04.229506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866360+0000","last_change":"2024-04-25T09:58:48.855658+0000","last_active":"2024-04-25T09:58:49.866360+0000","last_peered":"2024-04-25T09:58:49.866360+0000","last_clean":"2024-04-25T09:58:49.866360+0000","last_became_active":"2024-04-25T09:58:48.855373+0000","last_became_peered":"2024-04-25T09:58:48.855373+0000","last_unstale":"2024-04-25T09:58:49.866360+0000","last_undegraded":"2024-04-25T09:58:49.866360+0000","last_fullsized":"2024-04-25T09:58:49.866360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:56.760519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.878645+0000","last_change":"2024-04-25T09:58:49.876639+0000","last_active":"2024-04-25T09:58:49.878645+0000","last_peered":"2024-04-25T09:58:49.878645+0000","last_clean":"2024-04-25T09:58:49.878645+0000","last_became_active":"2024-04-25T09:58:49.876346+0000","last_became_peered":"2024-04-25T09:58:49.876346+0000","last_unstale":"2024-04-25T09:58:49.878645+0000","last_undegraded":"2024-04-25T09:58:49.878645+0000","last_fullsized":"2024-04-25T09:58:49.878645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.600001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.860005+0000","last_change":"2024-04-25T09:58:48.865178+0000","last_active":"2024-04-25T09:58:49.860005+0000","last_peered":"2024-04-25T09:58:49.860005+0000","last_clean":"2024-04-25T09:58:49.860005+0000","last_became_active":"2024-04-25T09:58:48.864898+0000","last_became_peered":"2024-04-25T09:58:48.864898+0000","last_unstale":"2024-04-25T09:58:49.860005+0000","last_undegraded":"2024-04-25T09:58:49.860005+0000","last_fullsized":"2024-04-25T09:58:49.860005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.129248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":284,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859112+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:58:49.859112+0000","last_peered":"2024-04-25T09:58:49.859112+0000","last_clean":"2024-04-25T09:58:49.859112+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:58:49.859112+0000","last_undegraded":"2024-04-25T09:58:49.859112+0000","last_fullsized":"2024-04-25T09:58:49.859112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1601536,"data_stored":1052563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335500,"statfs":{"total":96636764160,"available":96599552000,"internally_reserved":0,"allocated":1024000,"data_stored":463889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1597440,"data_stored":1052541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:52.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:58:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.963+0000 7f31aef10640 1 Processor -- start 2024-04-25T09:58:52.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.963+0000 7f31aef10640 1 -- start start 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31aef10640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31a815ef00 0x7f31a80d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31aef10640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80c7970 con 0x7f31a80c7310 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31aef10640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80c7c50 con 0x7f31a80d1ca0 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31aef10640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80d2110 con 0x7f31a815ef00 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31accac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31a815ef00 0x7f31a80d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31ad4ad640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f31a80c7310 0x7f31a80cc270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39634/0 (socket says 172.21.15.148:39634) 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31accac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31a815ef00 0x7f31a80d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56048/0 (socket says 172.21.15.148:56048) 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31ad4ad640 1 -- 172.21.15.148:0/332729353 learned_addr learned my addr 172.21.15.148:0/332729353 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31accac640 1 -- 172.21.15.148:0/332729353 >> v1:172.21.15.188:6789/0 conn(0x7f31a80d1ca0 legacy=0x7f31a80c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31accac640 1 -- 172.21.15.148:0/332729353 >> v1:172.21.15.148:6789/0 conn(0x7f31a80c7310 legacy=0x7f31a80cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31accac640 1 -- 172.21.15.148:0/332729353 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80c7f30 con 0x7f31a815ef00 2024-04-25T09:58:52.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.964+0000 7f31a77fe640 1 -- 172.21.15.148:0/332729353 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260744338 0 0) 0x7f31a80c7970 con 0x7f31a80c7310 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31accac640 1 --2- 172.21.15.148:0/332729353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a815ef00 0x7f31a80d1760 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f319c0097a0 tx=0x7f319c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a71b514d4bb7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31a77fe640 1 -- 172.21.15.148:0/332729353 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c037070 con 0x7f31a815ef00 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31a77fe640 1 -- 172.21.15.148:0/332729353 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c032640 con 0x7f31a815ef00 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31a77fe640 1 -- 172.21.15.148:0/332729353 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c032a10 con 0x7f31a815ef00 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31aef10640 1 -- 172.21.15.148:0/332729353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a815ef00 msgr2=0x7f31a80d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31aef10640 1 --2- 172.21.15.148:0/332729353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a815ef00 0x7f31a80d1760 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f319c0097a0 tx=0x7f319c02a2c0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31aef10640 1 -- 172.21.15.148:0/332729353 shutdown_connections 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.965+0000 7f31aef10640 1 --2- 172.21.15.148:0/332729353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a815ef00 0x7f31a80d1760 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.966+0000 7f31aef10640 1 -- 172.21.15.148:0/332729353 >> 172.21.15.148:0/332729353 conn(0x7f31a806f5d0 msgr2=0x7f31a806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.966+0000 7f31aef10640 1 -- 172.21.15.148:0/332729353 shutdown_connections 2024-04-25T09:58:52.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.966+0000 7f31aef10640 1 -- 172.21.15.148:0/332729353 wait complete. 2024-04-25T09:58:52.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.967+0000 7f31aef10640 1 Processor -- start 2024-04-25T09:58:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.967+0000 7f31aef10640 1 -- start start 2024-04-25T09:58:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 0x7f31a81481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a80d1ca0 0x7f31a8159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a815ef00 0x7f31a815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80c84f0 con 0x7f31a80c7310 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a815f2e0 con 0x7f31a815ef00 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80cc980 con 0x7f31a80d1ca0 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 0x7f31a81481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 0x7f31a81481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56052/0 (socket says 172.21.15.148:56052) 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 -- 172.21.15.148:0/3528809802 learned_addr learned my addr 172.21.15.148:0/3528809802 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 -- 172.21.15.148:0/3528809802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a80d1ca0 msgr2=0x7f31a8159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a80d1ca0 0x7f31a8159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 -- 172.21.15.148:0/3528809802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a815ef00 msgr2=0x7f31a815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:52.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a815ef00 0x7f31a815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 -- 172.21.15.148:0/3528809802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a806be90 con 0x7f31a80c7310 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31accac640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 0x7f31a81481b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f319c000c00 tx=0x7f319c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c041020 con 0x7f31a80c7310 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c030970 con 0x7f31a80c7310 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c037040 con 0x7f31a80c7310 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a806c130 con 0x7f31a80c7310 2024-04-25T09:58:52.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.968+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a806c6f0 con 0x7f31a80c7310 2024-04-25T09:58:52.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.969+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a8073990 con 0x7f31a80c7310 2024-04-25T09:58:52.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.970+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f319c044020 con 0x7f31a80c7310 2024-04-25T09:58:52.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.970+0000 7f31a57fa640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f318c06b4d0 0x7f318c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:52.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.970+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f319c0aace0 con 0x7f31a80c7310 2024-04-25T09:58:52.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.970+0000 7f31a7fff640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f318c06b4d0 0x7f318c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:52.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.971+0000 7f31a7fff640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f318c06b4d0 0x7f318c06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3194000f60 tx=0x7f31940079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:52.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:52.974+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f319c04b030 con 0x7f31a80c7310 2024-04-25T09:58:53.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.145+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31a80b0d90 con 0x7f318c06b4d0 2024-04-25T09:58:53.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.151+0000 7f31a57fa640 1 -- 172.21.15.148:0/3528809802 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252004 (secure 0 0 0) 0x7f31a80b0d90 con 0x7f318c06b4d0 2024-04-25T09:58:53.157 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:53.158 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f318c06b4d0 msgr2=0x7f318c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f318c06b4d0 0x7f318c06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3194000f60 tx=0x7f31940079b0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 msgr2=0x7f31a81481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 0x7f31a81481b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f319c000c00 tx=0x7f319c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 shutdown_connections 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f318c06b4d0 0x7f318c06d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31a815ef00 0x7f31a815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a80d1ca0 0x7f31a8159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 --2- 172.21.15.148:0/3528809802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31a80c7310 0x7f31a81481b0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 >> 172.21.15.148:0/3528809802 conn(0x7f31a806f5d0 msgr2=0x7f31a815db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 shutdown_connections 2024-04-25T09:58:53.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.154+0000 7f31aef10640 1 -- 172.21.15.148:0/3528809802 wait complete. 2024-04-25T09:58:53.169 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-04-25T09:58:53.075223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291716,"kb_used_data":8964,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754683004,"statfs":{"total":773094113280,"available":772795396096,"internally_reserved":0,"allocated":9179136,"data_stored":4844023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.235253"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872471+0000","last_change":"2024-04-25T09:58:49.866402+0000","last_active":"2024-04-25T09:58:49.872471+0000","last_peered":"2024-04-25T09:58:49.872471+0000","last_clean":"2024-04-25T09:58:49.872471+0000","last_became_active":"2024-04-25T09:58:49.866123+0000","last_became_peered":"2024-04-25T09:58:49.866123+0000","last_unstale":"2024-04-25T09:58:49.872471+0000","last_undegraded":"2024-04-25T09:58:49.872471+0000","last_fullsized":"2024-04-25T09:58:49.872471+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:16.672276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885419+0000","last_change":"2024-04-25T09:58:49.883642+0000","last_active":"2024-04-25T09:58:49.885419+0000","last_peered":"2024-04-25T09:58:49.885419+0000","last_clean":"2024-04-25T09:58:49.885419+0000","last_became_active":"2024-04-25T09:58:49.883378+0000","last_became_peered":"2024-04-25T09:58:49.883378+0000","last_unstale":"2024-04-25T09:58:49.885419+0000","last_undegraded":"2024-04-25T09:58:49.885419+0000","last_fullsized":"2024-04-25T09:58:49.885419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.136005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882914+0000","last_change":"2024-04-25T09:58:49.875977+0000","last_active":"2024-04-25T09:58:49.882914+0000","last_peered":"2024-04-25T09:58:49.882914+0000","last_clean":"2024-04-25T09:58:49.882914+0000","last_became_active":"2024-04-25T09:58:49.874129+0000","last_became_peered":"2024-04-25T09:58:49.874129+0000","last_unstale":"2024-04-25T09:58:49.882914+0000","last_undegraded":"2024-04-25T09:58:49.882914+0000","last_fullsized":"2024-04-25T09:58:49.882914+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:15.168768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862634+0000","last_change":"2024-04-25T09:58:49.861515+0000","last_active":"2024-04-25T09:58:49.862634+0000","last_peered":"2024-04-25T09:58:49.862634+0000","last_clean":"2024-04-25T09:58:49.862634+0000","last_became_active":"2024-04-25T09:58:49.861238+0000","last_became_peered":"2024-04-25T09:58:49.861238+0000","last_unstale":"2024-04-25T09:58:49.862634+0000","last_undegraded":"2024-04-25T09:58:49.862634+0000","last_fullsized":"2024-04-25T09:58:49.862634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:16.672276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883180+0000","last_change":"2024-04-25T09:58:49.877651+0000","last_active":"2024-04-25T09:58:49.883180+0000","last_peered":"2024-04-25T09:58:49.883180+0000","last_clean":"2024-04-25T09:58:49.883180+0000","last_became_active":"2024-04-25T09:58:49.875944+0000","last_became_peered":"2024-04-25T09:58:49.875944+0000","last_unstale":"2024-04-25T09:58:49.883180+0000","last_undegraded":"2024-04-25T09:58:49.883180+0000","last_fullsized":"2024-04-25T09:58:49.883180+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:20.990159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.868065+0000","last_change":"2024-04-25T09:58:49.867372+0000","last_active":"2024-04-25T09:58:49.868065+0000","last_peered":"2024-04-25T09:58:49.868065+0000","last_clean":"2024-04-25T09:58:49.868065+0000","last_became_active":"2024-04-25T09:58:49.866334+0000","last_became_peered":"2024-04-25T09:58:49.866334+0000","last_unstale":"2024-04-25T09:58:49.868065+0000","last_undegraded":"2024-04-25T09:58:49.868065+0000","last_fullsized":"2024-04-25T09:58:49.868065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:57.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.889010+0000","last_change":"2024-04-25T09:58:49.884475+0000","last_active":"2024-04-25T09:58:49.889010+0000","last_peered":"2024-04-25T09:58:49.889010+0000","last_clean":"2024-04-25T09:58:49.889010+0000","last_became_active":"2024-04-25T09:58:49.884215+0000","last_became_peered":"2024-04-25T09:58:49.884215+0000","last_unstale":"2024-04-25T09:58:49.889010+0000","last_undegraded":"2024-04-25T09:58:49.889010+0000","last_fullsized":"2024-04-25T09:58:49.889010+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:47.622560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883755+0000","last_change":"2024-04-25T09:58:49.879132+0000","last_active":"2024-04-25T09:58:49.883755+0000","last_peered":"2024-04-25T09:58:49.883755+0000","last_clean":"2024-04-25T09:58:49.883755+0000","last_became_active":"2024-04-25T09:58:49.878844+0000","last_became_peered":"2024-04-25T09:58:49.878844+0000","last_unstale":"2024-04-25T09:58:49.883755+0000","last_undegraded":"2024-04-25T09:58:49.883755+0000","last_fullsized":"2024-04-25T09:58:49.883755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:49.217657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.865723+0000","last_change":"2024-04-25T09:58:49.862989+0000","last_active":"2024-04-25T09:58:49.865723+0000","last_peered":"2024-04-25T09:58:49.865723+0000","last_clean":"2024-04-25T09:58:49.865723+0000","last_became_active":"2024-04-25T09:58:49.862697+0000","last_became_peered":"2024-04-25T09:58:49.862697+0000","last_unstale":"2024-04-25T09:58:49.865723+0000","last_undegraded":"2024-04-25T09:58:49.865723+0000","last_fullsized":"2024-04-25T09:58:49.865723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:09.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880834+0000","last_change":"2024-04-25T09:58:49.875506+0000","last_active":"2024-04-25T09:58:49.880834+0000","last_peered":"2024-04-25T09:58:49.880834+0000","last_clean":"2024-04-25T09:58:49.880834+0000","last_became_active":"2024-04-25T09:58:49.875202+0000","last_became_peered":"2024-04-25T09:58:49.875202+0000","last_unstale":"2024-04-25T09:58:49.880834+0000","last_undegraded":"2024-04-25T09:58:49.880834+0000","last_fullsized":"2024-04-25T09:58:49.880834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:28.823821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885479+0000","last_change":"2024-04-25T09:58:49.884665+0000","last_active":"2024-04-25T09:58:49.885479+0000","last_peered":"2024-04-25T09:58:49.885479+0000","last_clean":"2024-04-25T09:58:49.885479+0000","last_became_active":"2024-04-25T09:58:49.883976+0000","last_became_peered":"2024-04-25T09:58:49.883976+0000","last_unstale":"2024-04-25T09:58:49.885479+0000","last_undegraded":"2024-04-25T09:58:49.885479+0000","last_fullsized":"2024-04-25T09:58:49.885479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:31.065429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880786+0000","last_change":"2024-04-25T09:58:49.875945+0000","last_active":"2024-04-25T09:58:49.880786+0000","last_peered":"2024-04-25T09:58:49.880786+0000","last_clean":"2024-04-25T09:58:49.880786+0000","last_became_active":"2024-04-25T09:58:49.875661+0000","last_became_peered":"2024-04-25T09:58:49.875661+0000","last_unstale":"2024-04-25T09:58:49.880786+0000","last_undegraded":"2024-04-25T09:58:49.880786+0000","last_fullsized":"2024-04-25T09:58:49.880786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:03.136005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.886019+0000","last_change":"2024-04-25T09:58:49.882170+0000","last_active":"2024-04-25T09:58:49.886019+0000","last_peered":"2024-04-25T09:58:49.886019+0000","last_clean":"2024-04-25T09:58:49.886019+0000","last_became_active":"2024-04-25T09:58:49.881887+0000","last_became_peered":"2024-04-25T09:58:49.881887+0000","last_unstale":"2024-04-25T09:58:49.886019+0000","last_undegraded":"2024-04-25T09:58:49.886019+0000","last_fullsized":"2024-04-25T09:58:49.886019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.656639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.867055+0000","last_change":"2024-04-25T09:58:49.865752+0000","last_active":"2024-04-25T09:58:49.867055+0000","last_peered":"2024-04-25T09:58:49.867055+0000","last_clean":"2024-04-25T09:58:49.867055+0000","last_became_active":"2024-04-25T09:58:49.865480+0000","last_became_peered":"2024-04-25T09:58:49.865480+0000","last_unstale":"2024-04-25T09:58:49.867055+0000","last_undegraded":"2024-04-25T09:58:49.867055+0000","last_fullsized":"2024-04-25T09:58:49.867055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:19.425038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879852+0000","last_change":"2024-04-25T09:58:49.877322+0000","last_active":"2024-04-25T09:58:49.879852+0000","last_peered":"2024-04-25T09:58:49.879852+0000","last_clean":"2024-04-25T09:58:49.879852+0000","last_became_active":"2024-04-25T09:58:49.876991+0000","last_became_peered":"2024-04-25T09:58:49.876991+0000","last_unstale":"2024-04-25T09:58:49.879852+0000","last_undegraded":"2024-04-25T09:58:49.879852+0000","last_fullsized":"2024-04-25T09:58:49.879852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.380686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.869742+0000","last_change":"2024-04-25T09:58:49.865189+0000","last_active":"2024-04-25T09:58:49.869742+0000","last_peered":"2024-04-25T09:58:49.869742+0000","last_clean":"2024-04-25T09:58:49.869742+0000","last_became_active":"2024-04-25T09:58:49.864898+0000","last_became_peered":"2024-04-25T09:58:49.864898+0000","last_unstale":"2024-04-25T09:58:49.869742+0000","last_undegraded":"2024-04-25T09:58:49.869742+0000","last_fullsized":"2024-04-25T09:58:49.869742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:40.141095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883472+0000","last_change":"2024-04-25T09:58:49.879974+0000","last_active":"2024-04-25T09:58:49.883472+0000","last_peered":"2024-04-25T09:58:49.883472+0000","last_clean":"2024-04-25T09:58:49.883472+0000","last_became_active":"2024-04-25T09:58:49.879710+0000","last_became_peered":"2024-04-25T09:58:49.879710+0000","last_unstale":"2024-04-25T09:58:49.883472+0000","last_undegraded":"2024-04-25T09:58:49.883472+0000","last_fullsized":"2024-04-25T09:58:49.883472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:56.581674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873824+0000","last_change":"2024-04-25T09:58:49.869032+0000","last_active":"2024-04-25T09:58:49.873824+0000","last_peered":"2024-04-25T09:58:49.873824+0000","last_clean":"2024-04-25T09:58:49.873824+0000","last_became_active":"2024-04-25T09:58:49.867788+0000","last_became_peered":"2024-04-25T09:58:49.867788+0000","last_unstale":"2024-04-25T09:58:49.873824+0000","last_undegraded":"2024-04-25T09:58:49.873824+0000","last_fullsized":"2024-04-25T09:58:49.873824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:56.260197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.874679+0000","last_change":"2024-04-25T09:58:49.870312+0000","last_active":"2024-04-25T09:58:49.874679+0000","last_peered":"2024-04-25T09:58:49.874679+0000","last_clean":"2024-04-25T09:58:49.874679+0000","last_became_active":"2024-04-25T09:58:49.870069+0000","last_became_peered":"2024-04-25T09:58:49.870069+0000","last_unstale":"2024-04-25T09:58:49.874679+0000","last_undegraded":"2024-04-25T09:58:49.874679+0000","last_fullsized":"2024-04-25T09:58:49.874679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:56.215164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883322+0000","last_change":"2024-04-25T09:58:49.880978+0000","last_active":"2024-04-25T09:58:49.883322+0000","last_peered":"2024-04-25T09:58:49.883322+0000","last_clean":"2024-04-25T09:58:49.883322+0000","last_became_active":"2024-04-25T09:58:49.880701+0000","last_became_peered":"2024-04-25T09:58:49.880701+0000","last_unstale":"2024-04-25T09:58:49.883322+0000","last_undegraded":"2024-04-25T09:58:49.883322+0000","last_fullsized":"2024-04-25T09:58:49.883322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.041253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881150+0000","last_change":"2024-04-25T09:58:49.877242+0000","last_active":"2024-04-25T09:58:49.881150+0000","last_peered":"2024-04-25T09:58:49.881150+0000","last_clean":"2024-04-25T09:58:49.881150+0000","last_became_active":"2024-04-25T09:58:49.875314+0000","last_became_peered":"2024-04-25T09:58:49.875314+0000","last_unstale":"2024-04-25T09:58:49.881150+0000","last_undegraded":"2024-04-25T09:58:49.881150+0000","last_fullsized":"2024-04-25T09:58:49.881150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:57.767275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884020+0000","last_change":"2024-04-25T09:58:49.881593+0000","last_active":"2024-04-25T09:58:49.884020+0000","last_peered":"2024-04-25T09:58:49.884020+0000","last_clean":"2024-04-25T09:58:49.884020+0000","last_became_active":"2024-04-25T09:58:49.881158+0000","last_became_peered":"2024-04-25T09:58:49.881158+0000","last_unstale":"2024-04-25T09:58:49.884020+0000","last_undegraded":"2024-04-25T09:58:49.884020+0000","last_fullsized":"2024-04-25T09:58:49.884020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:24.930190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866008+0000","last_change":"2024-04-25T09:58:49.865005+0000","last_active":"2024-04-25T09:58:49.866008+0000","last_peered":"2024-04-25T09:58:49.866008+0000","last_clean":"2024-04-25T09:58:49.866008+0000","last_became_active":"2024-04-25T09:58:49.864707+0000","last_became_peered":"2024-04-25T09:58:49.864707+0000","last_unstale":"2024-04-25T09:58:49.866008+0000","last_undegraded":"2024-04-25T09:58:49.866008+0000","last_fullsized":"2024-04-25T09:58:49.866008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:04.618516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885512+0000","last_change":"2024-04-25T09:58:49.882016+0000","last_active":"2024-04-25T09:58:49.885512+0000","last_peered":"2024-04-25T09:58:49.885512+0000","last_clean":"2024-04-25T09:58:49.885512+0000","last_became_active":"2024-04-25T09:58:49.879297+0000","last_became_peered":"2024-04-25T09:58:49.879297+0000","last_unstale":"2024-04-25T09:58:49.885512+0000","last_undegraded":"2024-04-25T09:58:49.885512+0000","last_fullsized":"2024-04-25T09:58:49.885512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:28.823821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873211+0000","last_change":"2024-04-25T09:58:49.871311+0000","last_active":"2024-04-25T09:58:49.873211+0000","last_peered":"2024-04-25T09:58:49.873211+0000","last_clean":"2024-04-25T09:58:49.873211+0000","last_became_active":"2024-04-25T09:58:49.871045+0000","last_became_peered":"2024-04-25T09:58:49.871045+0000","last_unstale":"2024-04-25T09:58:49.873211+0000","last_undegraded":"2024-04-25T09:58:49.873211+0000","last_fullsized":"2024-04-25T09:58:49.873211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:54.538905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872509+0000","last_change":"2024-04-25T09:58:49.871089+0000","last_active":"2024-04-25T09:58:49.872509+0000","last_peered":"2024-04-25T09:58:49.872509+0000","last_clean":"2024-04-25T09:58:49.872509+0000","last_became_active":"2024-04-25T09:58:49.870817+0000","last_became_peered":"2024-04-25T09:58:49.870817+0000","last_unstale":"2024-04-25T09:58:49.872509+0000","last_undegraded":"2024-04-25T09:58:49.872509+0000","last_fullsized":"2024-04-25T09:58:49.872509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.600001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872714+0000","last_change":"2024-04-25T09:58:49.867612+0000","last_active":"2024-04-25T09:58:49.872714+0000","last_peered":"2024-04-25T09:58:49.872714+0000","last_clean":"2024-04-25T09:58:49.872714+0000","last_became_active":"2024-04-25T09:58:49.867238+0000","last_became_peered":"2024-04-25T09:58:49.867238+0000","last_unstale":"2024-04-25T09:58:49.872714+0000","last_undegraded":"2024-04-25T09:58:49.872714+0000","last_fullsized":"2024-04-25T09:58:49.872714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.380686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883300+0000","last_change":"2024-04-25T09:58:49.880743+0000","last_active":"2024-04-25T09:58:49.883300+0000","last_peered":"2024-04-25T09:58:49.883300+0000","last_clean":"2024-04-25T09:58:49.883300+0000","last_became_active":"2024-04-25T09:58:49.880393+0000","last_became_peered":"2024-04-25T09:58:49.880393+0000","last_unstale":"2024-04-25T09:58:49.883300+0000","last_undegraded":"2024-04-25T09:58:49.883300+0000","last_fullsized":"2024-04-25T09:58:49.883300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:06.915021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881355+0000","last_change":"2024-04-25T09:58:49.878361+0000","last_active":"2024-04-25T09:58:49.881355+0000","last_peered":"2024-04-25T09:58:49.881355+0000","last_clean":"2024-04-25T09:58:49.881355+0000","last_became_active":"2024-04-25T09:58:49.877352+0000","last_became_peered":"2024-04-25T09:58:49.877352+0000","last_unstale":"2024-04-25T09:58:49.881355+0000","last_undegraded":"2024-04-25T09:58:49.881355+0000","last_fullsized":"2024-04-25T09:58:49.881355+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.715358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872665+0000","last_change":"2024-04-25T09:58:49.869447+0000","last_active":"2024-04-25T09:58:49.872665+0000","last_peered":"2024-04-25T09:58:49.872665+0000","last_clean":"2024-04-25T09:58:49.872665+0000","last_became_active":"2024-04-25T09:58:49.869095+0000","last_became_peered":"2024-04-25T09:58:49.869095+0000","last_unstale":"2024-04-25T09:58:49.872665+0000","last_undegraded":"2024-04-25T09:58:49.872665+0000","last_fullsized":"2024-04-25T09:58:49.872665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.715358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882534+0000","last_change":"2024-04-25T09:58:49.875724+0000","last_active":"2024-04-25T09:58:49.882534+0000","last_peered":"2024-04-25T09:58:49.882534+0000","last_clean":"2024-04-25T09:58:49.882534+0000","last_became_active":"2024-04-25T09:58:49.874456+0000","last_became_peered":"2024-04-25T09:58:49.874456+0000","last_unstale":"2024-04-25T09:58:49.882534+0000","last_undegraded":"2024-04-25T09:58:49.882534+0000","last_fullsized":"2024-04-25T09:58:49.882534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.629636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.869721+0000","last_change":"2024-04-25T09:58:49.864476+0000","last_active":"2024-04-25T09:58:49.869721+0000","last_peered":"2024-04-25T09:58:49.869721+0000","last_clean":"2024-04-25T09:58:49.869721+0000","last_became_active":"2024-04-25T09:58:49.864002+0000","last_became_peered":"2024-04-25T09:58:49.864002+0000","last_unstale":"2024-04-25T09:58:49.869721+0000","last_undegraded":"2024-04-25T09:58:49.869721+0000","last_fullsized":"2024-04-25T09:58:49.869721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:54.538905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882729+0000","last_change":"2024-04-25T09:58:49.874988+0000","last_active":"2024-04-25T09:58:49.882729+0000","last_peered":"2024-04-25T09:58:49.882729+0000","last_clean":"2024-04-25T09:58:49.882729+0000","last_became_active":"2024-04-25T09:58:49.873842+0000","last_became_peered":"2024-04-25T09:58:49.873842+0000","last_unstale":"2024-04-25T09:58:49.882729+0000","last_undegraded":"2024-04-25T09:58:49.882729+0000","last_fullsized":"2024-04-25T09:58:49.882729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:42.918059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885809+0000","last_change":"2024-04-25T09:58:48.858763+0000","last_active":"2024-04-25T09:58:49.885809+0000","last_peered":"2024-04-25T09:58:49.885809+0000","last_clean":"2024-04-25T09:58:49.885809+0000","last_became_active":"2024-04-25T09:58:48.858498+0000","last_became_peered":"2024-04-25T09:58:48.858498+0000","last_unstale":"2024-04-25T09:58:49.885809+0000","last_undegraded":"2024-04-25T09:58:49.885809+0000","last_fullsized":"2024-04-25T09:58:49.885809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:10.787548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872553+0000","last_change":"2024-04-25T09:58:49.870634+0000","last_active":"2024-04-25T09:58:49.872553+0000","last_peered":"2024-04-25T09:58:49.872553+0000","last_clean":"2024-04-25T09:58:49.872553+0000","last_became_active":"2024-04-25T09:58:49.870241+0000","last_became_peered":"2024-04-25T09:58:49.870241+0000","last_unstale":"2024-04-25T09:58:49.872553+0000","last_undegraded":"2024-04-25T09:58:49.872553+0000","last_fullsized":"2024-04-25T09:58:49.872553+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:24.362109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879056+0000","last_change":"2024-04-25T09:58:48.858296+0000","last_active":"2024-04-25T09:58:49.879056+0000","last_peered":"2024-04-25T09:58:49.879056+0000","last_clean":"2024-04-25T09:58:49.879056+0000","last_became_active":"2024-04-25T09:58:48.857952+0000","last_became_peered":"2024-04-25T09:58:48.857952+0000","last_unstale":"2024-04-25T09:58:49.879056+0000","last_undegraded":"2024-04-25T09:58:49.879056+0000","last_fullsized":"2024-04-25T09:58:49.879056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:38.892401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.868613+0000","last_change":"2024-04-25T09:58:49.867512+0000","last_active":"2024-04-25T09:58:49.868613+0000","last_peered":"2024-04-25T09:58:49.868613+0000","last_clean":"2024-04-25T09:58:49.868613+0000","last_became_active":"2024-04-25T09:58:49.867237+0000","last_became_peered":"2024-04-25T09:58:49.867237+0000","last_unstale":"2024-04-25T09:58:49.868613+0000","last_undegraded":"2024-04-25T09:58:49.868613+0000","last_fullsized":"2024-04-25T09:58:49.868613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:36.615889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883069+0000","last_change":"2024-04-25T09:58:48.864168+0000","last_active":"2024-04-25T09:58:49.883069+0000","last_peered":"2024-04-25T09:58:49.883069+0000","last_clean":"2024-04-25T09:58:49.883069+0000","last_became_active":"2024-04-25T09:58:48.863890+0000","last_became_peered":"2024-04-25T09:58:48.863890+0000","last_unstale":"2024-04-25T09:58:49.883069+0000","last_undegraded":"2024-04-25T09:58:49.883069+0000","last_fullsized":"2024-04-25T09:58:49.883069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:07.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.869998+0000","last_change":"2024-04-25T09:58:49.868845+0000","last_active":"2024-04-25T09:58:49.869998+0000","last_peered":"2024-04-25T09:58:49.869998+0000","last_clean":"2024-04-25T09:58:49.869998+0000","last_became_active":"2024-04-25T09:58:49.868480+0000","last_became_peered":"2024-04-25T09:58:49.868480+0000","last_unstale":"2024-04-25T09:58:49.869998+0000","last_undegraded":"2024-04-25T09:58:49.869998+0000","last_fullsized":"2024-04-25T09:58:49.869998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:02.681291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879268+0000","last_change":"2024-04-25T09:58:48.860848+0000","last_active":"2024-04-25T09:58:49.879268+0000","last_peered":"2024-04-25T09:58:49.879268+0000","last_clean":"2024-04-25T09:58:49.879268+0000","last_became_active":"2024-04-25T09:58:48.860335+0000","last_became_peered":"2024-04-25T09:58:48.860335+0000","last_unstale":"2024-04-25T09:58:49.879268+0000","last_undegraded":"2024-04-25T09:58:49.879268+0000","last_fullsized":"2024-04-25T09:58:49.879268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:08.275929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.886493+0000","last_change":"2024-04-25T09:58:49.885087+0000","last_active":"2024-04-25T09:58:49.886493+0000","last_peered":"2024-04-25T09:58:49.886493+0000","last_clean":"2024-04-25T09:58:49.886493+0000","last_became_active":"2024-04-25T09:58:49.884815+0000","last_became_peered":"2024-04-25T09:58:49.884815+0000","last_unstale":"2024-04-25T09:58:49.886493+0000","last_undegraded":"2024-04-25T09:58:49.886493+0000","last_fullsized":"2024-04-25T09:58:49.886493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:57.214533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.889180+0000","last_change":"2024-04-25T09:58:48.859800+0000","last_active":"2024-04-25T09:58:49.889180+0000","last_peered":"2024-04-25T09:58:49.889180+0000","last_clean":"2024-04-25T09:58:49.889180+0000","last_became_active":"2024-04-25T09:58:48.859535+0000","last_became_peered":"2024-04-25T09:58:48.859535+0000","last_unstale":"2024-04-25T09:58:49.889180+0000","last_undegraded":"2024-04-25T09:58:49.889180+0000","last_fullsized":"2024-04-25T09:58:49.889180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:38.747671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885458+0000","last_change":"2024-04-25T09:58:49.884455+0000","last_active":"2024-04-25T09:58:49.885458+0000","last_peered":"2024-04-25T09:58:49.885458+0000","last_clean":"2024-04-25T09:58:49.885458+0000","last_became_active":"2024-04-25T09:58:49.883633+0000","last_became_peered":"2024-04-25T09:58:49.883633+0000","last_unstale":"2024-04-25T09:58:49.885458+0000","last_undegraded":"2024-04-25T09:58:49.885458+0000","last_fullsized":"2024-04-25T09:58:49.885458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:19.425038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881521+0000","last_change":"2024-04-25T09:58:48.863123+0000","last_active":"2024-04-25T09:58:49.881521+0000","last_peered":"2024-04-25T09:58:49.881521+0000","last_clean":"2024-04-25T09:58:49.881521+0000","last_became_active":"2024-04-25T09:58:48.862845+0000","last_became_peered":"2024-04-25T09:58:48.862845+0000","last_unstale":"2024-04-25T09:58:49.881521+0000","last_undegraded":"2024-04-25T09:58:49.881521+0000","last_fullsized":"2024-04-25T09:58:49.881521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:23.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.864564+0000","last_change":"2024-04-25T09:58:49.862244+0000","last_active":"2024-04-25T09:58:49.864564+0000","last_peered":"2024-04-25T09:58:49.864564+0000","last_clean":"2024-04-25T09:58:49.864564+0000","last_became_active":"2024-04-25T09:58:49.861988+0000","last_became_peered":"2024-04-25T09:58:49.861988+0000","last_unstale":"2024-04-25T09:58:49.864564+0000","last_undegraded":"2024-04-25T09:58:49.864564+0000","last_fullsized":"2024-04-25T09:58:49.864564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:08.268371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.864437+0000","last_change":"2024-04-25T09:58:48.859923+0000","last_active":"2024-04-25T09:58:49.864437+0000","last_peered":"2024-04-25T09:58:49.864437+0000","last_clean":"2024-04-25T09:58:49.864437+0000","last_became_active":"2024-04-25T09:58:48.859642+0000","last_became_peered":"2024-04-25T09:58:48.859642+0000","last_unstale":"2024-04-25T09:58:49.864437+0000","last_undegraded":"2024-04-25T09:58:49.864437+0000","last_fullsized":"2024-04-25T09:58:49.864437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.926531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.888990+0000","last_change":"2024-04-25T09:58:49.884342+0000","last_active":"2024-04-25T09:58:49.888990+0000","last_peered":"2024-04-25T09:58:49.888990+0000","last_clean":"2024-04-25T09:58:49.888990+0000","last_became_active":"2024-04-25T09:58:49.884053+0000","last_became_peered":"2024-04-25T09:58:49.884053+0000","last_unstale":"2024-04-25T09:58:49.888990+0000","last_undegraded":"2024-04-25T09:58:49.888990+0000","last_fullsized":"2024-04-25T09:58:49.888990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:29.518080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859916+0000","last_change":"2024-04-25T09:58:48.855119+0000","last_active":"2024-04-25T09:58:49.859916+0000","last_peered":"2024-04-25T09:58:49.859916+0000","last_clean":"2024-04-25T09:58:49.859916+0000","last_became_active":"2024-04-25T09:58:48.854493+0000","last_became_peered":"2024-04-25T09:58:48.854493+0000","last_unstale":"2024-04-25T09:58:49.859916+0000","last_undegraded":"2024-04-25T09:58:49.859916+0000","last_fullsized":"2024-04-25T09:58:49.859916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:00.205579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882265+0000","last_change":"2024-04-25T09:58:49.879588+0000","last_active":"2024-04-25T09:58:49.882265+0000","last_peered":"2024-04-25T09:58:49.882265+0000","last_clean":"2024-04-25T09:58:49.882265+0000","last_became_active":"2024-04-25T09:58:49.878830+0000","last_became_peered":"2024-04-25T09:58:49.878830+0000","last_unstale":"2024-04-25T09:58:49.882265+0000","last_undegraded":"2024-04-25T09:58:49.882265+0000","last_fullsized":"2024-04-25T09:58:49.882265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.600001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862199+0000","last_change":"2024-04-25T09:58:48.858713+0000","last_active":"2024-04-25T09:58:49.862199+0000","last_peered":"2024-04-25T09:58:49.862199+0000","last_clean":"2024-04-25T09:58:49.862199+0000","last_became_active":"2024-04-25T09:58:48.858264+0000","last_became_peered":"2024-04-25T09:58:48.858264+0000","last_unstale":"2024-04-25T09:58:49.862199+0000","last_undegraded":"2024-04-25T09:58:49.862199+0000","last_fullsized":"2024-04-25T09:58:49.862199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:23.355352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866978+0000","last_change":"2024-04-25T09:58:49.863067+0000","last_active":"2024-04-25T09:58:49.866978+0000","last_peered":"2024-04-25T09:58:49.866978+0000","last_clean":"2024-04-25T09:58:49.866978+0000","last_became_active":"2024-04-25T09:58:49.862796+0000","last_became_peered":"2024-04-25T09:58:49.862796+0000","last_unstale":"2024-04-25T09:58:49.866978+0000","last_undegraded":"2024-04-25T09:58:49.866978+0000","last_fullsized":"2024-04-25T09:58:49.866978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:55.715358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.857560+0000","last_change":"2024-04-25T09:58:48.856626+0000","last_active":"2024-04-25T09:58:49.857560+0000","last_peered":"2024-04-25T09:58:49.857560+0000","last_clean":"2024-04-25T09:58:49.857560+0000","last_became_active":"2024-04-25T09:58:48.856336+0000","last_became_peered":"2024-04-25T09:58:48.856336+0000","last_unstale":"2024-04-25T09:58:49.857560+0000","last_undegraded":"2024-04-25T09:58:49.857560+0000","last_fullsized":"2024-04-25T09:58:49.857560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:39.134338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.867487+0000","last_change":"2024-04-25T09:58:49.866910+0000","last_active":"2024-04-25T09:58:49.867487+0000","last_peered":"2024-04-25T09:58:49.867487+0000","last_clean":"2024-04-25T09:58:49.867487+0000","last_became_active":"2024-04-25T09:58:49.866648+0000","last_became_peered":"2024-04-25T09:58:49.866648+0000","last_unstale":"2024-04-25T09:58:49.867487+0000","last_undegraded":"2024-04-25T09:58:49.867487+0000","last_fullsized":"2024-04-25T09:58:49.867487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:31.065429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884298+0000","last_change":"2024-04-25T09:58:48.857036+0000","last_active":"2024-04-25T09:58:49.884298+0000","last_peered":"2024-04-25T09:58:49.884298+0000","last_clean":"2024-04-25T09:58:49.884298+0000","last_became_active":"2024-04-25T09:58:48.856770+0000","last_became_peered":"2024-04-25T09:58:48.856770+0000","last_unstale":"2024-04-25T09:58:49.884298+0000","last_undegraded":"2024-04-25T09:58:49.884298+0000","last_fullsized":"2024-04-25T09:58:49.884298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:57.643448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882147+0000","last_change":"2024-04-25T09:58:49.873531+0000","last_active":"2024-04-25T09:58:49.882147+0000","last_peered":"2024-04-25T09:58:49.882147+0000","last_clean":"2024-04-25T09:58:49.882147+0000","last_became_active":"2024-04-25T09:58:49.873230+0000","last_became_peered":"2024-04-25T09:58:49.873230+0000","last_unstale":"2024-04-25T09:58:49.882147+0000","last_undegraded":"2024-04-25T09:58:49.882147+0000","last_fullsized":"2024-04-25T09:58:49.882147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:51.081158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.891989+0000","last_change":"2024-04-25T09:58:48.859642+0000","last_active":"2024-04-25T09:58:49.891989+0000","last_peered":"2024-04-25T09:58:49.891989+0000","last_clean":"2024-04-25T09:58:49.891989+0000","last_became_active":"2024-04-25T09:58:48.859342+0000","last_became_peered":"2024-04-25T09:58:48.859342+0000","last_unstale":"2024-04-25T09:58:49.891989+0000","last_undegraded":"2024-04-25T09:58:49.891989+0000","last_fullsized":"2024-04-25T09:58:49.891989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:55.253441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866766+0000","last_change":"2024-04-25T09:58:49.862416+0000","last_active":"2024-04-25T09:58:49.866766+0000","last_peered":"2024-04-25T09:58:49.866766+0000","last_clean":"2024-04-25T09:58:49.866766+0000","last_became_active":"2024-04-25T09:58:49.862141+0000","last_became_peered":"2024-04-25T09:58:49.862141+0000","last_unstale":"2024-04-25T09:58:49.866766+0000","last_undegraded":"2024-04-25T09:58:49.866766+0000","last_fullsized":"2024-04-25T09:58:49.866766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:08.380686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.881629+0000","last_change":"2024-04-25T09:58:48.856870+0000","last_active":"2024-04-25T09:58:49.881629+0000","last_peered":"2024-04-25T09:58:49.881629+0000","last_clean":"2024-04-25T09:58:49.881629+0000","last_became_active":"2024-04-25T09:58:48.856380+0000","last_became_peered":"2024-04-25T09:58:48.856380+0000","last_unstale":"2024-04-25T09:58:49.881629+0000","last_undegraded":"2024-04-25T09:58:49.881629+0000","last_fullsized":"2024-04-25T09:58:49.881629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:27.889161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883689+0000","last_change":"2024-04-25T09:58:49.879122+0000","last_active":"2024-04-25T09:58:49.883689+0000","last_peered":"2024-04-25T09:58:49.883689+0000","last_clean":"2024-04-25T09:58:49.883689+0000","last_became_active":"2024-04-25T09:58:49.878848+0000","last_became_peered":"2024-04-25T09:58:49.878848+0000","last_unstale":"2024-04-25T09:58:49.883689+0000","last_undegraded":"2024-04-25T09:58:49.883689+0000","last_fullsized":"2024-04-25T09:58:49.883689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:25.629636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.860470+0000","last_change":"2024-04-25T09:58:48.859890+0000","last_active":"2024-04-25T09:58:49.860470+0000","last_peered":"2024-04-25T09:58:49.860470+0000","last_clean":"2024-04-25T09:58:49.860470+0000","last_became_active":"2024-04-25T09:58:48.859641+0000","last_became_peered":"2024-04-25T09:58:48.859641+0000","last_unstale":"2024-04-25T09:58:49.860470+0000","last_undegraded":"2024-04-25T09:58:49.860470+0000","last_fullsized":"2024-04-25T09:58:49.860470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.926531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.878372+0000","last_change":"2024-04-25T09:58:49.874843+0000","last_active":"2024-04-25T09:58:49.878372+0000","last_peered":"2024-04-25T09:58:49.878372+0000","last_clean":"2024-04-25T09:58:49.878372+0000","last_became_active":"2024-04-25T09:58:49.874550+0000","last_became_peered":"2024-04-25T09:58:49.874550+0000","last_unstale":"2024-04-25T09:58:49.878372+0000","last_undegraded":"2024-04-25T09:58:49.878372+0000","last_fullsized":"2024-04-25T09:58:49.878372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:42.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.860432+0000","last_change":"2024-04-25T09:58:48.863107+0000","last_active":"2024-04-25T09:58:49.860432+0000","last_peered":"2024-04-25T09:58:49.860432+0000","last_clean":"2024-04-25T09:58:49.860432+0000","last_became_active":"2024-04-25T09:58:48.862777+0000","last_became_peered":"2024-04-25T09:58:48.862777+0000","last_unstale":"2024-04-25T09:58:49.860432+0000","last_undegraded":"2024-04-25T09:58:49.860432+0000","last_fullsized":"2024-04-25T09:58:49.860432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:01.135899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880837+0000","last_change":"2024-04-25T09:58:49.878037+0000","last_active":"2024-04-25T09:58:49.880837+0000","last_peered":"2024-04-25T09:58:49.880837+0000","last_clean":"2024-04-25T09:58:49.880837+0000","last_became_active":"2024-04-25T09:58:49.877719+0000","last_became_peered":"2024-04-25T09:58:49.877719+0000","last_unstale":"2024-04-25T09:58:49.880837+0000","last_undegraded":"2024-04-25T09:58:49.880837+0000","last_fullsized":"2024-04-25T09:58:49.880837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:42.918059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.887096+0000","last_change":"2024-04-25T09:58:48.857534+0000","last_active":"2024-04-25T09:58:49.887096+0000","last_peered":"2024-04-25T09:58:49.887096+0000","last_clean":"2024-04-25T09:58:49.887096+0000","last_became_active":"2024-04-25T09:58:48.857290+0000","last_became_peered":"2024-04-25T09:58:48.857290+0000","last_unstale":"2024-04-25T09:58:49.887096+0000","last_undegraded":"2024-04-25T09:58:49.887096+0000","last_fullsized":"2024-04-25T09:58:49.887096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:29.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.875593+0000","last_change":"2024-04-25T09:58:49.872187+0000","last_active":"2024-04-25T09:58:49.875593+0000","last_peered":"2024-04-25T09:58:49.875593+0000","last_clean":"2024-04-25T09:58:49.875593+0000","last_became_active":"2024-04-25T09:58:49.871931+0000","last_became_peered":"2024-04-25T09:58:49.871931+0000","last_unstale":"2024-04-25T09:58:49.875593+0000","last_undegraded":"2024-04-25T09:58:49.875593+0000","last_fullsized":"2024-04-25T09:58:49.875593+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:57.394666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884074+0000","last_change":"2024-04-25T09:58:48.861459+0000","last_active":"2024-04-25T09:58:49.884074+0000","last_peered":"2024-04-25T09:58:49.884074+0000","last_clean":"2024-04-25T09:58:49.884074+0000","last_became_active":"2024-04-25T09:58:48.861175+0000","last_became_peered":"2024-04-25T09:58:48.861175+0000","last_unstale":"2024-04-25T09:58:49.884074+0000","last_undegraded":"2024-04-25T09:58:49.884074+0000","last_fullsized":"2024-04-25T09:58:49.884074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:27.889161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883652+0000","last_change":"2024-04-25T09:58:49.880305+0000","last_active":"2024-04-25T09:58:49.883652+0000","last_peered":"2024-04-25T09:58:49.883652+0000","last_clean":"2024-04-25T09:58:49.883652+0000","last_became_active":"2024-04-25T09:58:49.880049+0000","last_became_peered":"2024-04-25T09:58:49.880049+0000","last_unstale":"2024-04-25T09:58:49.883652+0000","last_undegraded":"2024-04-25T09:58:49.883652+0000","last_fullsized":"2024-04-25T09:58:49.883652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:35.809943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.889180+0000","last_change":"2024-04-25T09:58:48.847300+0000","last_active":"2024-04-25T09:58:49.889180+0000","last_peered":"2024-04-25T09:58:49.889180+0000","last_clean":"2024-04-25T09:58:49.889180+0000","last_became_active":"2024-04-25T09:58:48.846972+0000","last_became_peered":"2024-04-25T09:58:48.846972+0000","last_unstale":"2024-04-25T09:58:49.889180+0000","last_undegraded":"2024-04-25T09:58:49.889180+0000","last_fullsized":"2024-04-25T09:58:49.889180+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:45.276573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883999+0000","last_change":"2024-04-25T09:58:49.881395+0000","last_active":"2024-04-25T09:58:49.883999+0000","last_peered":"2024-04-25T09:58:49.883999+0000","last_clean":"2024-04-25T09:58:49.883999+0000","last_became_active":"2024-04-25T09:58:49.880893+0000","last_became_peered":"2024-04-25T09:58:49.880893+0000","last_unstale":"2024-04-25T09:58:49.883999+0000","last_undegraded":"2024-04-25T09:58:49.883999+0000","last_fullsized":"2024-04-25T09:58:49.883999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:36.208100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884682+0000","last_change":"2024-04-25T09:58:48.857850+0000","last_active":"2024-04-25T09:58:49.884682+0000","last_peered":"2024-04-25T09:58:49.884682+0000","last_clean":"2024-04-25T09:58:49.884682+0000","last_became_active":"2024-04-25T09:58:48.857588+0000","last_became_peered":"2024-04-25T09:58:48.857588+0000","last_unstale":"2024-04-25T09:58:49.884682+0000","last_undegraded":"2024-04-25T09:58:49.884682+0000","last_fullsized":"2024-04-25T09:58:49.884682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:45.874120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.885212+0000","last_change":"2024-04-25T09:58:49.884415+0000","last_active":"2024-04-25T09:58:49.885212+0000","last_peered":"2024-04-25T09:58:49.885212+0000","last_clean":"2024-04-25T09:58:49.885212+0000","last_became_active":"2024-04-25T09:58:49.884170+0000","last_became_peered":"2024-04-25T09:58:49.884170+0000","last_unstale":"2024-04-25T09:58:49.885212+0000","last_undegraded":"2024-04-25T09:58:49.885212+0000","last_fullsized":"2024-04-25T09:58:49.885212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:12.604509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.868745+0000","last_change":"2024-04-25T09:58:48.854279+0000","last_active":"2024-04-25T09:58:49.868745+0000","last_peered":"2024-04-25T09:58:49.868745+0000","last_clean":"2024-04-25T09:58:49.868745+0000","last_became_active":"2024-04-25T09:58:48.853982+0000","last_became_peered":"2024-04-25T09:58:48.853982+0000","last_unstale":"2024-04-25T09:58:49.868745+0000","last_undegraded":"2024-04-25T09:58:49.868745+0000","last_fullsized":"2024-04-25T09:58:49.868745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:08.275929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880472+0000","last_change":"2024-04-25T09:58:49.873004+0000","last_active":"2024-04-25T09:58:49.880472+0000","last_peered":"2024-04-25T09:58:49.880472+0000","last_clean":"2024-04-25T09:58:49.880472+0000","last_became_active":"2024-04-25T09:58:49.871714+0000","last_became_peered":"2024-04-25T09:58:49.871714+0000","last_unstale":"2024-04-25T09:58:49.880472+0000","last_undegraded":"2024-04-25T09:58:49.880472+0000","last_fullsized":"2024-04-25T09:58:49.880472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:16.672276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.887508+0000","last_change":"2024-04-25T09:58:48.857386+0000","last_active":"2024-04-25T09:58:49.887508+0000","last_peered":"2024-04-25T09:58:49.887508+0000","last_clean":"2024-04-25T09:58:49.887508+0000","last_became_active":"2024-04-25T09:58:48.857092+0000","last_became_peered":"2024-04-25T09:58:48.857092+0000","last_unstale":"2024-04-25T09:58:49.887508+0000","last_undegraded":"2024-04-25T09:58:49.887508+0000","last_fullsized":"2024-04-25T09:58:49.887508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:27.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873694+0000","last_change":"2024-04-25T09:58:49.871837+0000","last_active":"2024-04-25T09:58:49.873694+0000","last_peered":"2024-04-25T09:58:49.873694+0000","last_clean":"2024-04-25T09:58:49.873694+0000","last_became_active":"2024-04-25T09:58:49.871581+0000","last_became_peered":"2024-04-25T09:58:49.871581+0000","last_unstale":"2024-04-25T09:58:49.873694+0000","last_undegraded":"2024-04-25T09:58:49.873694+0000","last_fullsized":"2024-04-25T09:58:49.873694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:40.141095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859691+0000","last_change":"2024-04-25T09:58:48.864266+0000","last_active":"2024-04-25T09:58:49.859691+0000","last_peered":"2024-04-25T09:58:49.859691+0000","last_clean":"2024-04-25T09:58:49.859691+0000","last_became_active":"2024-04-25T09:58:48.863962+0000","last_became_peered":"2024-04-25T09:58:48.863962+0000","last_unstale":"2024-04-25T09:58:49.859691+0000","last_undegraded":"2024-04-25T09:58:49.859691+0000","last_fullsized":"2024-04-25T09:58:49.859691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:24.593244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.883765+0000","last_change":"2024-04-25T09:58:49.879900+0000","last_active":"2024-04-25T09:58:49.883765+0000","last_peered":"2024-04-25T09:58:49.883765+0000","last_clean":"2024-04-25T09:58:49.883765+0000","last_became_active":"2024-04-25T09:58:49.879653+0000","last_became_peered":"2024-04-25T09:58:49.879653+0000","last_unstale":"2024-04-25T09:58:49.883765+0000","last_undegraded":"2024-04-25T09:58:49.883765+0000","last_fullsized":"2024-04-25T09:58:49.883765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:15.168768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859560+0000","last_change":"2024-04-25T09:58:48.864351+0000","last_active":"2024-04-25T09:58:49.859560+0000","last_peered":"2024-04-25T09:58:49.859560+0000","last_clean":"2024-04-25T09:58:49.859560+0000","last_became_active":"2024-04-25T09:58:48.864022+0000","last_became_peered":"2024-04-25T09:58:48.864022+0000","last_unstale":"2024-04-25T09:58:49.859560+0000","last_undegraded":"2024-04-25T09:58:49.859560+0000","last_fullsized":"2024-04-25T09:58:49.859560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:54.708602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.879353+0000","last_change":"2024-04-25T09:58:49.875240+0000","last_active":"2024-04-25T09:58:49.879353+0000","last_peered":"2024-04-25T09:58:49.879353+0000","last_clean":"2024-04-25T09:58:49.879353+0000","last_became_active":"2024-04-25T09:58:49.874793+0000","last_became_peered":"2024-04-25T09:58:49.874793+0000","last_unstale":"2024-04-25T09:58:49.879353+0000","last_undegraded":"2024-04-25T09:58:49.879353+0000","last_fullsized":"2024-04-25T09:58:49.879353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:04.413815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.884292+0000","last_change":"2024-04-25T09:58:48.856595+0000","last_active":"2024-04-25T09:58:49.884292+0000","last_peered":"2024-04-25T09:58:49.884292+0000","last_clean":"2024-04-25T09:58:49.884292+0000","last_became_active":"2024-04-25T09:58:48.856243+0000","last_became_peered":"2024-04-25T09:58:48.856243+0000","last_unstale":"2024-04-25T09:58:49.884292+0000","last_undegraded":"2024-04-25T09:58:49.884292+0000","last_fullsized":"2024-04-25T09:58:49.884292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:53.532148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.886530+0000","last_change":"2024-04-25T09:58:49.881940+0000","last_active":"2024-04-25T09:58:49.886530+0000","last_peered":"2024-04-25T09:58:49.886530+0000","last_clean":"2024-04-25T09:58:49.886530+0000","last_became_active":"2024-04-25T09:58:49.881609+0000","last_became_peered":"2024-04-25T09:58:49.881609+0000","last_unstale":"2024-04-25T09:58:49.886530+0000","last_undegraded":"2024-04-25T09:58:49.886530+0000","last_fullsized":"2024-04-25T09:58:49.886530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:04.229506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866634+0000","last_change":"2024-04-25T09:58:48.864534+0000","last_active":"2024-04-25T09:58:49.866634+0000","last_peered":"2024-04-25T09:58:49.866634+0000","last_clean":"2024-04-25T09:58:49.866634+0000","last_became_active":"2024-04-25T09:58:48.864262+0000","last_became_peered":"2024-04-25T09:58:48.864262+0000","last_unstale":"2024-04-25T09:58:49.866634+0000","last_undegraded":"2024-04-25T09:58:49.866634+0000","last_fullsized":"2024-04-25T09:58:49.866634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:07.628252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.865729+0000","last_change":"2024-04-25T09:58:49.863198+0000","last_active":"2024-04-25T09:58:49.865729+0000","last_peered":"2024-04-25T09:58:49.865729+0000","last_clean":"2024-04-25T09:58:49.865729+0000","last_became_active":"2024-04-25T09:58:49.862904+0000","last_became_peered":"2024-04-25T09:58:49.862904+0000","last_unstale":"2024-04-25T09:58:49.865729+0000","last_undegraded":"2024-04-25T09:58:49.865729+0000","last_fullsized":"2024-04-25T09:58:49.865729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:12.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862124+0000","last_change":"2024-04-25T09:58:48.862663+0000","last_active":"2024-04-25T09:58:49.862124+0000","last_peered":"2024-04-25T09:58:49.862124+0000","last_clean":"2024-04-25T09:58:49.862124+0000","last_became_active":"2024-04-25T09:58:48.862366+0000","last_became_peered":"2024-04-25T09:58:48.862366+0000","last_unstale":"2024-04-25T09:58:49.862124+0000","last_undegraded":"2024-04-25T09:58:49.862124+0000","last_fullsized":"2024-04-25T09:58:49.862124+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:29.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872923+0000","last_change":"2024-04-25T09:58:49.871194+0000","last_active":"2024-04-25T09:58:49.872923+0000","last_peered":"2024-04-25T09:58:49.872923+0000","last_clean":"2024-04-25T09:58:49.872923+0000","last_became_active":"2024-04-25T09:58:49.870762+0000","last_became_peered":"2024-04-25T09:58:49.870762+0000","last_unstale":"2024-04-25T09:58:49.872923+0000","last_undegraded":"2024-04-25T09:58:49.872923+0000","last_fullsized":"2024-04-25T09:58:49.872923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:28.895918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859020+0000","last_change":"2024-04-25T09:58:48.858993+0000","last_active":"2024-04-25T09:58:49.859020+0000","last_peered":"2024-04-25T09:58:49.859020+0000","last_clean":"2024-04-25T09:58:49.859020+0000","last_became_active":"2024-04-25T09:58:48.858719+0000","last_became_peered":"2024-04-25T09:58:48.858719+0000","last_unstale":"2024-04-25T09:58:49.859020+0000","last_undegraded":"2024-04-25T09:58:49.859020+0000","last_fullsized":"2024-04-25T09:58:49.859020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:10.184432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.880917+0000","last_change":"2024-04-25T09:58:49.877968+0000","last_active":"2024-04-25T09:58:49.880917+0000","last_peered":"2024-04-25T09:58:49.880917+0000","last_clean":"2024-04-25T09:58:49.880917+0000","last_became_active":"2024-04-25T09:58:49.877689+0000","last_became_peered":"2024-04-25T09:58:49.877689+0000","last_unstale":"2024-04-25T09:58:49.880917+0000","last_undegraded":"2024-04-25T09:58:49.880917+0000","last_fullsized":"2024-04-25T09:58:49.880917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:51.081158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.872937+0000","last_change":"2024-04-25T09:58:48.859743+0000","last_active":"2024-04-25T09:58:49.872937+0000","last_peered":"2024-04-25T09:58:49.872937+0000","last_clean":"2024-04-25T09:58:49.872937+0000","last_became_active":"2024-04-25T09:58:48.859205+0000","last_became_peered":"2024-04-25T09:58:48.859205+0000","last_unstale":"2024-04-25T09:58:49.872937+0000","last_undegraded":"2024-04-25T09:58:49.872937+0000","last_fullsized":"2024-04-25T09:58:49.872937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:03.407058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.891667+0000","last_change":"2024-04-25T09:58:49.886759+0000","last_active":"2024-04-25T09:58:49.891667+0000","last_peered":"2024-04-25T09:58:49.891667+0000","last_clean":"2024-04-25T09:58:49.891667+0000","last_became_active":"2024-04-25T09:58:49.886540+0000","last_became_peered":"2024-04-25T09:58:49.886540+0000","last_unstale":"2024-04-25T09:58:49.891667+0000","last_undegraded":"2024-04-25T09:58:49.891667+0000","last_fullsized":"2024-04-25T09:58:49.891667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:35.351668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.862340+0000","last_change":"2024-04-25T09:58:48.859684+0000","last_active":"2024-04-25T09:58:49.862340+0000","last_peered":"2024-04-25T09:58:49.862340+0000","last_clean":"2024-04-25T09:58:49.862340+0000","last_became_active":"2024-04-25T09:58:48.859396+0000","last_became_peered":"2024-04-25T09:58:48.859396+0000","last_unstale":"2024-04-25T09:58:49.862340+0000","last_undegraded":"2024-04-25T09:58:49.862340+0000","last_fullsized":"2024-04-25T09:58:49.862340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:48.506129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.874197+0000","last_change":"2024-04-25T09:58:49.873526+0000","last_active":"2024-04-25T09:58:49.874197+0000","last_peered":"2024-04-25T09:58:49.874197+0000","last_clean":"2024-04-25T09:58:49.874197+0000","last_became_active":"2024-04-25T09:58:49.873257+0000","last_became_peered":"2024-04-25T09:58:49.873257+0000","last_unstale":"2024-04-25T09:58:49.874197+0000","last_undegraded":"2024-04-25T09:58:49.874197+0000","last_fullsized":"2024-04-25T09:58:49.874197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:42.664588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.873172+0000","last_change":"2024-04-25T09:58:48.864595+0000","last_active":"2024-04-25T09:58:49.873172+0000","last_peered":"2024-04-25T09:58:49.873172+0000","last_clean":"2024-04-25T09:58:49.873172+0000","last_became_active":"2024-04-25T09:58:48.864323+0000","last_became_peered":"2024-04-25T09:58:48.864323+0000","last_unstale":"2024-04-25T09:58:49.873172+0000","last_undegraded":"2024-04-25T09:58:49.873172+0000","last_fullsized":"2024-04-25T09:58:49.873172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:56.760519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.882627+0000","last_change":"2024-04-25T09:58:49.876262+0000","last_active":"2024-04-25T09:58:49.882627+0000","last_peered":"2024-04-25T09:58:49.882627+0000","last_clean":"2024-04-25T09:58:49.882627+0000","last_became_active":"2024-04-25T09:58:49.874737+0000","last_became_peered":"2024-04-25T09:58:49.874737+0000","last_unstale":"2024-04-25T09:58:49.882627+0000","last_undegraded":"2024-04-25T09:58:49.882627+0000","last_fullsized":"2024-04-25T09:58:49.882627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:04.229506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.866360+0000","last_change":"2024-04-25T09:58:48.855658+0000","last_active":"2024-04-25T09:58:49.866360+0000","last_peered":"2024-04-25T09:58:49.866360+0000","last_clean":"2024-04-25T09:58:49.866360+0000","last_became_active":"2024-04-25T09:58:48.855373+0000","last_became_peered":"2024-04-25T09:58:48.855373+0000","last_unstale":"2024-04-25T09:58:49.866360+0000","last_undegraded":"2024-04-25T09:58:49.866360+0000","last_fullsized":"2024-04-25T09:58:49.866360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:56.760519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.878645+0000","last_change":"2024-04-25T09:58:49.876639+0000","last_active":"2024-04-25T09:58:49.878645+0000","last_peered":"2024-04-25T09:58:49.878645+0000","last_clean":"2024-04-25T09:58:49.878645+0000","last_became_active":"2024-04-25T09:58:49.876346+0000","last_became_peered":"2024-04-25T09:58:49.876346+0000","last_unstale":"2024-04-25T09:58:49.878645+0000","last_undegraded":"2024-04-25T09:58:49.878645+0000","last_fullsized":"2024-04-25T09:58:49.878645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:48.839531+0000","last_clean_scrub_stamp":"2024-04-25T09:58:48.839531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.600001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.860005+0000","last_change":"2024-04-25T09:58:48.865178+0000","last_active":"2024-04-25T09:58:49.860005+0000","last_peered":"2024-04-25T09:58:49.860005+0000","last_clean":"2024-04-25T09:58:49.860005+0000","last_became_active":"2024-04-25T09:58:48.864898+0000","last_became_peered":"2024-04-25T09:58:48.864898+0000","last_unstale":"2024-04-25T09:58:49.860005+0000","last_undegraded":"2024-04-25T09:58:49.860005+0000","last_fullsized":"2024-04-25T09:58:49.860005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:58:47.832774+0000","last_clean_scrub_stamp":"2024-04-25T09:58:47.832774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.129248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":284,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T09:58:49.859112+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T09:58:49.859112+0000","last_peered":"2024-04-25T09:58:49.859112+0000","last_clean":"2024-04-25T09:58:49.859112+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T09:58:49.859112+0000","last_undegraded":"2024-04-25T09:58:49.859112+0000","last_fullsized":"2024-04-25T09:58:49.859112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1601536,"data_stored":1052563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36340,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335500,"statfs":{"total":96636764160,"available":96599552000,"internally_reserved":0,"allocated":1024000,"data_stored":463889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335528,"statfs":{"total":96636764160,"available":96599580672,"internally_reserved":0,"allocated":991232,"data_stored":455006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36908,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334932,"statfs":{"total":96636764160,"available":96598970368,"internally_reserved":0,"allocated":1597440,"data_stored":1052541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:58:53.172 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:58:53.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:58:53.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.310+0000 7fa20533b640 1 Processor -- start 2024-04-25T09:58:53.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.310+0000 7fa20533b640 1 -- start start 2024-04-25T09:58:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.310+0000 7fa20533b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2000cd4c0 0x7fa2000cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.310+0000 7fa20533b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa20015b1c0 0x7fa20015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa20533b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20015e6e0 con 0x7fa20015e1a0 2024-04-25T09:58:53.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa20533b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20015b620 con 0x7fa20015b1c0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa20533b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015e9c0 con 0x7fa2000cd4c0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fe7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa20015b1c0 0x7fa20015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1ff7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa20015e1a0 0x7fa200150b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:39638/0 (socket says 172.21.15.148:39638) 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fe7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa20015b1c0 0x7fa20015dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56056/0 (socket says 172.21.15.148:56056) 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1ff7fe640 1 -- 172.21.15.148:0/2729724857 learned_addr learned my addr 172.21.15.148:0/2729724857 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1feffd640 1 --2- 172.21.15.148:0/2729724857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2000cd4c0 0x7fa2000cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2729724857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2000cd4c0 msgr2=0x7fa2000cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fe7fc640 1 --2- 172.21.15.148:0/2729724857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2000cd4c0 0x7fa2000cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2729724857 >> v1:172.21.15.148:6789/0 conn(0x7fa20015e1a0 legacy=0x7fa200150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fdffb640 1 -- 172.21.15.148:0/2729724857 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385887665 0 0) 0x7fa20015e6e0 con 0x7fa20015e1a0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.311+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2729724857 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20015a9c0 con 0x7fa20015b1c0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa1fe7fc640 1 --2- 172.21.15.148:0/2729724857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015dc60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097a0 tx=0x7fa1f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce7e036c2c3b6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa1fdffb640 1 -- 172.21.15.148:0/2729724857 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4037070 con 0x7fa20015b1c0 2024-04-25T09:58:53.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa1fdffb640 1 -- 172.21.15.148:0/2729724857 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4032440 con 0x7fa20015b1c0 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa1fdffb640 1 -- 172.21.15.148:0/2729724857 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4032810 con 0x7fa20015b1c0 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 -- 172.21.15.148:0/2729724857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 msgr2=0x7fa20015dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 --2- 172.21.15.148:0/2729724857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015dc60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40097a0 tx=0x7fa1f402a180 comp rx=0 tx=0).stop 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 -- 172.21.15.148:0/2729724857 shutdown_connections 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 --2- 172.21.15.148:0/2729724857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015dc60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 --2- 172.21.15.148:0/2729724857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2000cd4c0 0x7fa2000cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 -- 172.21.15.148:0/2729724857 >> 172.21.15.148:0/2729724857 conn(0x7fa20006abe0 msgr2=0x7fa20006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 -- 172.21.15.148:0/2729724857 shutdown_connections 2024-04-25T09:58:53.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.312+0000 7fa20533b640 1 -- 172.21.15.148:0/2729724857 wait complete. 2024-04-25T09:58:53.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 Processor -- start 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 -- start start 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2000cd4c0 0x7fa2001f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20015e1a0 0x7fa2001f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20006eca0 con 0x7fa20015b1c0 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015a670 con 0x7fa2000cd4c0 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.314+0000 7fa20533b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20006ee20 con 0x7fa20015e1a0 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56072/0 (socket says 172.21.15.148:56072) 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2217133898 learned_addr learned my addr 172.21.15.148:0/2217133898 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1feffd640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2000cd4c0 0x7fa2001f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1ff7fe640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20015e1a0 0x7fa2001f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20015e1a0 msgr2=0x7fa2001f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20015e1a0 0x7fa2001f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2000cd4c0 msgr2=0x7fa2001f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2000cd4c0 0x7fa2001f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 -- 172.21.15.148:0/2217133898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000ad830 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1ff7fe640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20015e1a0 0x7fa2001f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1fe7fc640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015cf30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000c00 tx=0x7fa1f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4037040 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4042070 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2000adb10 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4030bb0 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.315+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2000ae0d0 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.316+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2000b4a70 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.317+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1f4047460 con 0x7fa20015b1c0 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.317+0000 7fa1e77fe640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.317+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1f40aaeb0 con 0x7fa20015b1c0 2024-04-25T09:58:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.317+0000 7fa1feffd640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.318+0000 7fa1feffd640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0006fd0 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.321+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2000b4a70 con 0x7fa20015b1c0 2024-04-25T09:58:53.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.556+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa200073a00 con 0x7fa20015b1c0 2024-04-25T09:58:53.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.558+0000 7fa1e77fe640 1 -- 172.21.15.148:0/2217133898 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa1f4032e20 con 0x7fa20015b1c0 2024-04-25T09:58:53.562 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T09:58:53.562 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T09:58:53.562 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T09:58:53.562 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T09:58:53.562 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:58:53.563 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T09:58:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1ec06b4d0 msgr2=0x7fa1ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0006fd0 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2024-04-25T09:58:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 msgr2=0x7fa20015cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015cf30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4000c00 tx=0x7fa1f40046c0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 shutdown_connections 2024-04-25T09:58:53.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1ec06b4d0 0x7fa1ec06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20015e1a0 0x7fa2001f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20015b1c0 0x7fa20015cf30 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 --2- 172.21.15.148:0/2217133898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2000cd4c0 0x7fa2001f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 >> 172.21.15.148:0/2217133898 conn(0x7fa20006abe0 msgr2=0x7fa20014fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 shutdown_connections 2024-04-25T09:58:53.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.560+0000 7fa20533b640 1 -- 172.21.15.148:0/2217133898 wait complete. 2024-04-25T09:58:53.575 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:58:53.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.668+0000 7fa120c9c640 1 Processor -- start 2024-04-25T09:58:53.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.668+0000 7fa120c9c640 1 -- start start 2024-04-25T09:58:53.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.669+0000 7fa120c9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa11c15dbe0 0x7fa11c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.669+0000 7fa120c9c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0b6220 con 0x7fa11c0b92a0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.669+0000 7fa120c9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0b97a0 con 0x7fa11c0b2cb0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.669+0000 7fa120c9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c15dfc0 con 0x7fa11c15dbe0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11ad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa11c0b92a0 0x7fa11c0b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46766/0 (socket says 172.21.15.148:46766) 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11ad9d640 1 -- 172.21.15.148:0/3284134332 learned_addr learned my addr 172.21.15.148:0/3284134332 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 --2- 172.21.15.148:0/3284134332 >> v2:172.21.15.188:3300/0 conn(0x7fa11c15dbe0 0x7fa11c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072525332 0 0) 0x7fa11c0b6220 con 0x7fa11c0b92a0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa104003520 con 0x7fa11c0b92a0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203968435 0 0) 0x7fa104003520 con 0x7fa11c0b92a0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 >> v2:172.21.15.188:3300/0 conn(0x7fa11c15dbe0 msgr2=0x7fa11c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 --2- 172.21.15.148:0/3284134332 >> v2:172.21.15.188:3300/0 conn(0x7fa11c15dbe0 0x7fa11c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 >> v1:172.21.15.188:6789/0 conn(0x7fa11c0b2cb0 legacy=0x7fa11c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0d1c50 con 0x7fa11c0b92a0 2024-04-25T09:58:53.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa1140019f0 con 0x7fa11c0b92a0 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 --2- 172.21.15.148:0/3284134332 >> v2:172.21.15.188:3300/0 conn(0x7fa11c15dbe0 0x7fa11c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 >> v1:172.21.15.148:6789/0 conn(0x7fa11c0b92a0 legacy=0x7fa11c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa104004800 0x7fa104004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040051c0 0x7fa10400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10400c130 con 0x7fa1040039b0 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10400c2e0 con 0x7fa1040051c0 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10400c4e0 con 0x7fa104004800 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11ad9d640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040051c0 0x7fa10400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa119d9b640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa104004800 0x7fa104004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 -- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa104004800 msgr2=0x7fa104004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa104004800 0x7fa104004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 -- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040051c0 msgr2=0x7fa10400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:53.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040051c0 0x7fa10400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11a59c640 1 -- 172.21.15.148:0/3284134332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104003520 con 0x7fa1040039b0 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11ad9d640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040051c0 0x7fa10400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.670+0000 7fa11ad9d640 1 -- 172.21.15.148:0/3284134332 reap_dead start 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.671+0000 7fa11a59c640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa110009c90 tx=0x7fa104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba48cf881b41a33b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.671+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110036070 con 0x7fa1040039b0 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.671+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa110031400 con 0x7fa1040039b0 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.671+0000 7fa11959a640 1 -- 172.21.15.148:0/3284134332 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110009860 con 0x7fa1040039b0 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 -- 172.21.15.148:0/3284134332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1040039b0 msgr2=0x7fa104003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa110009c90 tx=0x7fa104005830 comp rx=0 tx=0).stop 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 -- 172.21.15.148:0/3284134332 shutdown_connections 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 --2- 172.21.15.148:0/3284134332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1040039b0 0x7fa104003dd0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 -- 172.21.15.148:0/3284134332 >> 172.21.15.148:0/3284134332 conn(0x7fa11c061a20 msgr2=0x7fa11c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 -- 172.21.15.148:0/3284134332 shutdown_connections 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.672+0000 7fa120c9c640 1 -- 172.21.15.148:0/3284134332 wait complete. 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.673+0000 7fa120c9c640 1 Processor -- start 2024-04-25T09:58:53.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.673+0000 7fa120c9c640 1 -- start start 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa120c9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040039b0 0x7fa11c0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa120c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa11c0b5f80 0x7fa11c151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa120c9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 0x7fa11c15f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa120c9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0d2210 con 0x7fa11c15f220 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa120c9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0d1a00 con 0x7fa1040039b0 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa120c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0b6ab0 con 0x7fa11c0b5f80 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 0x7fa11c15f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 0x7fa11c15f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54884/0 (socket says 172.21.15.148:54884) 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 -- 172.21.15.148:0/2137532972 learned_addr learned my addr 172.21.15.148:0/2137532972 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11a59c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040039b0 0x7fa11c0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa119d9b640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa11c0b5f80 0x7fa11c151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 -- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa11c0b5f80 msgr2=0x7fa11c151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa11c0b5f80 0x7fa11c151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 -- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040039b0 msgr2=0x7fa11c0b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040039b0 0x7fa11c0b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.674+0000 7fa11ad9d640 1 -- 172.21.15.148:0/2137532972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0b4c30 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.675+0000 7fa11ad9d640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 0x7fa11c15f6a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cfb60 tx=0x7fa11400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.675+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400eb90 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.675+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114018070 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.675+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c0b3650 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.675+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114013420 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.675+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c0b3c10 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.676+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c073990 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.676+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1140135c0 con 0x7fa11c15f220 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.676+0000 7fa10affd640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:53.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.677+0000 7fa11a59c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:53.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.677+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa11408a5f0 con 0x7fa11c15f220 2024-04-25T09:58:53.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.677+0000 7fa11a59c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa110033000 tx=0x7fa110008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:53.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.681+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa114058c90 con 0x7fa11c15f220 2024-04-25T09:58:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.887+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa11c158b50 con 0x7fa11c15f220 2024-04-25T09:58:53.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.888+0000 7fa10affd640 1 -- 172.21.15.148:0/2137532972 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa11405cd30 con 0x7fa11c15f220 2024-04-25T09:58:53.892 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T09:58:53.892 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:58:49.193553+0000","modified":"2024-04-25T09:58:51.202219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7284":{"gid":7284,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/82715169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":82715169},{"type":"v1","addr":"172.21.15.148:6837","nonce":82715169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:58:53.892 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 46 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0fc06b4d0 msgr2=0x7fa0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa110033000 tx=0x7fa110008030 comp rx=0 tx=0).stop 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 msgr2=0x7fa11c15f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 0x7fa11c15f6a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cfb60 tx=0x7fa11400e530 comp rx=0 tx=0).stop 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 shutdown_connections 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0fc06b4d0 0x7fa0fc06d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c15f220 0x7fa11c15f6a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa11c0b5f80 0x7fa11c151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 --2- 172.21.15.148:0/2137532972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1040039b0 0x7fa11c0b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 >> 172.21.15.148:0/2137532972 conn(0x7fa11c061a20 msgr2=0x7fa11c158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 shutdown_connections 2024-04-25T09:58:53.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:53.890+0000 7fa120c9c640 1 -- 172.21.15.148:0/2137532972 wait complete. 2024-04-25T09:58:54.148 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:54.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:58:54.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:58:49.193553+0000', 'modified': '2024-04-25T09:58:51.202219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7284': {'gid': 7284, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/82715169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 82715169}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 82715169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:58:54.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:58:54.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:58:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.403+0000 7f211a0c4640 1 Processor -- start 2024-04-25T09:58:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.403+0000 7f211a0c4640 1 -- start start 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f211a0c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21140ad570 0x7f211406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f211a0c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21140b3930 con 0x7f211406abe0 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f211a0c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21140b3c10 con 0x7f211415dd80 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f211a0c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211406e410 con 0x7f21140ad570 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21137fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f211406abe0 0x7f211406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46782/0 (socket says 172.21.15.148:46782) 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21137fe640 1 -- 172.21.15.148:0/3012350820 learned_addr learned my addr 172.21.15.148:0/3012350820 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934117943 0 0) 0x7f21140b3930 con 0x7f211406abe0 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f2113fff640 1 --2- 172.21.15.148:0/3012350820 >> v2:172.21.15.188:3300/0 conn(0x7f21140ad570 0x7f211406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20fc003520 con 0x7f211406abe0 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207170133 0 0) 0x7f20fc003520 con 0x7f211406abe0 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 >> v2:172.21.15.188:3300/0 conn(0x7f21140ad570 msgr2=0x7f211406ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 --2- 172.21.15.148:0/3012350820 >> v2:172.21.15.188:3300/0 conn(0x7f21140ad570 0x7f211406ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 >> v1:172.21.15.188:6789/0 conn(0x7f211415dd80 legacy=0x7f21140b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211406e660 con 0x7f211406abe0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f2104001670 con 0x7f211406abe0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 >> v1:172.21.15.148:6789/0 conn(0x7f211406abe0 legacy=0x7f211406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0039b0 0x7f20fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc004800 0x7f20fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f20fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211406e410 con 0x7f20fc0051c0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc00c060 con 0x7f20fc004800 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.404+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc00c260 con 0x7f20fc0039b0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f20fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f21137fe640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0039b0 0x7f20fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2112ffd640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc004800 0x7f20fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 -- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0039b0 msgr2=0x7f20fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20fc0039b0 0x7f20fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 -- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc004800 msgr2=0x7f20fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc004800 0x7f20fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 -- 172.21.15.148:0/3012350820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc003520 con 0x7f20fc0051c0 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2112ffd640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20fc004800 0x7f20fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 -- 172.21.15.148:0/3012350820 reap_dead start 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.405+0000 7f2113fff640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f20fc00b9c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f210c009fc0 tx=0x7f20fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe3988546d93502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c035020 con 0x7f20fc0051c0 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c003bd0 con 0x7f20fc0051c0 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f21127fc640 1 -- 172.21.15.148:0/3012350820 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c009c10 con 0x7f20fc0051c0 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 -- 172.21.15.148:0/3012350820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 msgr2=0x7f20fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f20fc00b9c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f210c009fc0 tx=0x7f20fc005830 comp rx=0 tx=0).stop 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 -- 172.21.15.148:0/3012350820 shutdown_connections 2024-04-25T09:58:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 --2- 172.21.15.148:0/3012350820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f20fc00b9c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 -- 172.21.15.148:0/3012350820 >> 172.21.15.148:0/3012350820 conn(0x7f21140b5af0 msgr2=0x7f21140b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 -- 172.21.15.148:0/3012350820 shutdown_connections 2024-04-25T09:58:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.406+0000 7f211a0c4640 1 -- 172.21.15.148:0/3012350820 wait complete. 2024-04-25T09:58:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.407+0000 7f211a0c4640 1 Processor -- start 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 -- start start 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f2114160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21141420d0 0x7f211413ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2114141480 0x7f2114141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114070400 con 0x7f20fc0051c0 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114070280 con 0x7f2114141480 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f211a0c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21140b3590 con 0x7f21141420d0 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f21137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f2114160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f21137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f2114160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54900/0 (socket says 172.21.15.148:54900) 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f21137fe640 1 -- 172.21.15.148:0/1286158019 learned_addr learned my addr 172.21.15.148:0/1286158019 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:54.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.408+0000 7f2113fff640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2114141480 0x7f2114141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21141420d0 0x7f211413ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f21137fe640 1 -- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21141420d0 msgr2=0x7f211413ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f21137fe640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21141420d0 0x7f211413ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f21137fe640 1 -- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2114141480 msgr2=0x7f2114141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f21137fe640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2114141480 0x7f2114141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f21137fe640 1 -- 172.21.15.148:0/1286158019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211413f0f0 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f21137fe640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f2114160b40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f21040072c0 tx=0x7f2104007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f20fbfff640 1 -- 172.21.15.148:0/1286158019 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210400f960 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f20fbfff640 1 -- 172.21.15.148:0/1286158019 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2104018070 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211409a840 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f20fbfff640 1 -- 172.21.15.148:0/1286158019 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210400fd80 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.409+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211415e5c0 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.410+0000 7f20fbfff640 1 -- 172.21.15.148:0/1286158019 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f210401d460 con 0x7f20fc0051c0 2024-04-25T09:58:54.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.411+0000 7f20fbfff640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f210006b500 0x7f210006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.411+0000 7f20fbfff640 1 -- 172.21.15.148:0/1286158019 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f210408a390 con 0x7f20fc0051c0 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.411+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f210006b500 0x7f210006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.411+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f210006b500 0x7f210006d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2108000c00 tx=0x7f2108009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.412+0000 7f211cdc7740 1 Processor -- start 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.412+0000 7f211cdc7740 1 -- start start 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a65610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a82ad0 0x5622b9a83020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 0x5622b9a81190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f211cdc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b999da10 con 0x5622b9a3af10 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b9946b30 con 0x5622b9a82ad0 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622b9999c50 con 0x5622b9a65230 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 0x5622b9a81190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 0x5622b9a81190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54906/0 (socket says 172.21.15.148:54906) 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 -- 172.21.15.148:0/2558282015 learned_addr learned my addr 172.21.15.148:0/2558282015 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f21137fe640 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a65610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2112ffd640 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a82ad0 0x5622b9a83020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 -- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 msgr2=0x5622b9a65610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a65610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 -- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a82ad0 msgr2=0x5622b9a83020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a82ad0 0x5622b9a83020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 -- 172.21.15.148:0/2558282015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b995be10 con 0x5622b9a3af10 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f21137fe640 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a65610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f2113fff640 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 0x5622b9a81190 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f90 tx=0x7f210c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbacc43a285e7cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.413+0000 7f20f97fa640 1 -- 172.21.15.148:0/2558282015 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c040070 con 0x5622b9a3af10 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f20f97fa640 1 -- 172.21.15.148:0/2558282015 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c0317b0 con 0x5622b9a3af10 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f20f97fa640 1 -- 172.21.15.148:0/2558282015 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c035040 con 0x5622b9a3af10 2024-04-25T09:58:54.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 -- 172.21.15.148:0/2558282015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 msgr2=0x5622b9a81190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 0x5622b9a81190 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f210c009f90 tx=0x7f210c02d000 comp rx=0 tx=0).stop 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 -- 172.21.15.148:0/2558282015 shutdown_connections 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a3af10 0x5622b9a81190 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a82ad0 0x5622b9a83020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2558282015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a65610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 -- 172.21.15.148:0/2558282015 >> 172.21.15.148:0/2558282015 conn(0x5622b9a37840 msgr2=0x5622b9a80040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 -- 172.21.15.148:0/2558282015 shutdown_connections 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.414+0000 7f211cdc7740 1 -- 172.21.15.148:0/2558282015 wait complete. 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.415+0000 7f211cdc7740 1 -- start start 2024-04-25T09:58:54.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.415+0000 7f211cdc7740 1 Processor -- start 2024-04-25T09:58:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a4ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b9a3bcb0 con 0x5622b9a65230 2024-04-25T09:58:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b995a0c0 con 0x5622b9a77630 2024-04-25T09:58:54.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622b9a35770 con 0x5622b9a82ad0 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 10 client.0 _ll_register_callbacks cb 0x5622b994dec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54922/0 (socket says 172.21.15.148:54922) 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 -- 172.21.15.148:0/2025115445 learned_addr learned my addr 172.21.15.148:0/2025115445 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f2112ffd640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f2113fff640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a4ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 -- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 msgr2=0x5622b9a770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a770f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 -- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 msgr2=0x5622b9a4ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a4ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 -- 172.21.15.148:0/2025115445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622b96b22d0 con 0x5622b9a65230 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f2113fff640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a4ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f21137fe640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a348a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f210400f5a0 tx=0x7f2104059080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5442acc779b16532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f20f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f20f27fc640 1 -- 172.21.15.148:0/2025115445 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210405cad0 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f20f27fc640 1 -- 172.21.15.148:0/2025115445 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2104060c00 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f210c0317b0 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f20f27fc640 1 -- 172.21.15.148:0/2025115445 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2104061090 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 10 client.7428 fetch_fsmap learned FSMap version 46 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.416+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x5622b977c6b0 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f20f27fc640 1 -- 172.21.15.148:0/2025115445 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c0317b0 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f20f27fc640 1 -- 172.21.15.148:0/2025115445 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2104088150 con 0x5622b9a65230 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f211cdc7740 10 client.7428 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f211cdc7740 10 client.7428 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f211cdc7740 20 client.7428 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9a939e0 0x5622b9a93dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f211cdc7740 4 client.7428 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622b99fc920 con 0x5622b9a939e0 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f2112ffd640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9a939e0 0x5622b9a93dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f2112ffd640 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9a939e0 0x5622b9a93dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d6e447a6efe167 server_cookie=afe2a18e5031604c in_seq=0 out_seq=0 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.417+0000 7f20f27fc640 10 client.7428 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f20f27fc640 1 -- 172.21.15.148:0/2025115445 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622b99fc920 con 0x5622b9a939e0 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f20f27fc640 10 client.7428 handle_command_reply: tid=0 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 client.7428 shutdown 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 msgr2=0x5622b9a348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a348a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f210400f5a0 tx=0x7f2104059080 comp rx=0 tx=0).stop 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 shutdown_connections 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a4ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9a939e0 0x5622b9a93dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 --2- 172.21.15.148:0/2025115445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a348a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f20f27fc640 0 client.7428 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:58:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 >> 172.21.15.148:0/2025115445 conn(0x5622b9a3af10 msgr2=0x5622b9a835c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 shutdown_connections 2024-04-25T09:58:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 1 -- 172.21.15.148:0/2025115445 wait complete. 2024-04-25T09:58:54.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.418+0000 7f211cdc7740 20 client.7428 trim_cache size 0 max 16384 2024-04-25T09:58:54.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.434+0000 7f211cdc7740 1 Processor -- start 2024-04-25T09:58:54.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.434+0000 7f211cdc7740 1 -- start start 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a35060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a3b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 0x5622b9a3b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f211cdc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21141ffb50 con 0x5622b9a82ad0 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21141fef50 con 0x5622b9a77630 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21141ff550 con 0x5622b9a65230 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 0x5622b9a3b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 0x5622b9a3b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54936/0 (socket says 172.21.15.148:54936) 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 -- 172.21.15.148:0/1600540757 learned_addr learned my addr 172.21.15.148:0/1600540757 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f21137fe640 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a35060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a3b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 -- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 msgr2=0x5622b9a35060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a35060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 -- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 msgr2=0x5622b9a3b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a3b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 -- 172.21.15.148:0/1600540757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622b9a31550 con 0x5622b9a82ad0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a3b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f2113fff640 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 0x5622b9a3b850 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f210c009fd0 tx=0x7f210c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca45230fd929a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f20f8ff9640 1 -- 172.21.15.148:0/1600540757 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c035070 con 0x5622b9a82ad0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f20f8ff9640 1 -- 172.21.15.148:0/1600540757 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c0317b0 con 0x5622b9a82ad0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.435+0000 7f20f8ff9640 1 -- 172.21.15.148:0/1600540757 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c040040 con 0x5622b9a82ad0 2024-04-25T09:58:54.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 -- 172.21.15.148:0/1600540757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 msgr2=0x5622b9a3b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 0x5622b9a3b850 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f210c009fd0 tx=0x7f210c030000 comp rx=0 tx=0).stop 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 -- 172.21.15.148:0/1600540757 shutdown_connections 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a82ad0 0x5622b9a3b850 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a3b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1600540757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a65230 0x5622b9a35060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 -- 172.21.15.148:0/1600540757 >> 172.21.15.148:0/1600540757 conn(0x5622b9a3af10 msgr2=0x5622b9a346b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 -- 172.21.15.148:0/1600540757 shutdown_connections 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 -- 172.21.15.148:0/1600540757 wait complete. 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 -- start start 2024-04-25T09:58:54.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.436+0000 7f211cdc7740 1 Processor -- start 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a30bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a3b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b9a66a10 con 0x5622b9a65230 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622b9a4fa90 con 0x5622b9a77630 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622b9a63b80 con 0x5622b9a82ad0 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f211cdc7740 10 client.0 _ll_register_callbacks cb 0x5622b9a35c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:58:54.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f21137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f21137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a4bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54946/0 (socket says 172.21.15.148:54946) 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f21137fe640 1 -- 172.21.15.148:0/1864137664 learned_addr learned my addr 172.21.15.148:0/1864137664 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a30bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.437+0000 7f2113fff640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a3b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f21137fe640 1 -- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 msgr2=0x5622b9a3b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f21137fe640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a3b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f21137fe640 1 -- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 msgr2=0x5622b9a30bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f21137fe640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a30bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f21137fe640 1 -- 172.21.15.148:0/1864137664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622b9a78270 con 0x5622b9a65230 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f21137fe640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a4bb60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2104010fd0 tx=0x7f210405b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecad975abea4d491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f20f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f20f27fc640 1 -- 172.21.15.148:0/1864137664 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104074050 con 0x5622b9a65230 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f20f27fc640 1 -- 172.21.15.148:0/1864137664 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21040773d0 con 0x5622b9a65230 2024-04-25T09:58:54.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f210c0317b0 con 0x5622b9a65230 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f20f27fc640 1 -- 172.21.15.148:0/1864137664 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2104085020 con 0x5622b9a65230 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 10 client.7434 fetch_fsmap learned FSMap version 46 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x5622b9aeba10 con 0x5622b9a65230 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f20f27fc640 1 -- 172.21.15.148:0/1864137664 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c0317b0 con 0x5622b9a65230 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f20f27fc640 1 -- 172.21.15.148:0/1864137664 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2104088b50 con 0x5622b9a65230 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 10 client.7434 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 10 client.7434 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 20 client.7434 populate_metadata read hostname 'smithi148' 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9aeb410 0x5622b9a921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 4 client.7434 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5622b99aaf50 con 0x5622b9aeb410 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.438+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9aeb410 0x5622b9a921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f2112ffd640 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9aeb410 0x5622b9a921f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14cafc79e213cef server_cookie=bbfa594bf21281fa in_seq=0 out_seq=0 2024-04-25T09:58:54.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f20f27fc640 10 client.7434 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f20f27fc640 1 -- 172.21.15.148:0/1864137664 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5622b99aaf50 con 0x5622b9aeb410 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f20f27fc640 10 client.7434 handle_command_reply: tid=0 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 client.7434 shutdown 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 msgr2=0x5622b9a4bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a4bb60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2104010fd0 tx=0x7f210405b040 comp rx=0 tx=0).stop 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 shutdown_connections 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5622b9aeb410 0x5622b9a921f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622b9a82ad0 0x5622b9a3b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622b9a77630 0x5622b9a30bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f20f27fc640 0 client.7434 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 --2- 172.21.15.148:0/1864137664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5622b9a65230 0x5622b9a4bb60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 >> 172.21.15.148:0/1864137664 conn(0x5622b9a3af10 msgr2=0x5622b9a76f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 shutdown_connections 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 1 -- 172.21.15.148:0/1864137664 wait complete. 2024-04-25T09:58:54.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211cdc7740 20 client.7434 trim_cache size 0 max 16384 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 7284, 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.5788789870000004, 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 46, 2024-04-25T09:58:54.450 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 83, 2024-04-25T09:58:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 83, 2024-04-25T09:58:54.451 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 9.6566694420000001 2024-04-25T09:58:54.451 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:58:54.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f210006b500 msgr2=0x7f210006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211a0c4640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f210006b500 0x7f210006d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2108000c00 tx=0x7f2108009040 comp rx=0 tx=0).stop 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 msgr2=0x7f2114160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.439+0000 7f211a0c4640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f2114160b40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f21040072c0 tx=0x7f2104007780 comp rx=0 tx=0).stop 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 shutdown_connections 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f210006b500 0x7f210006d9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2114141480 0x7f2114141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21141420d0 0x7f211413ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 --2- 172.21.15.148:0/1286158019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20fc0051c0 0x7f2114160b40 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 >> 172.21.15.148:0/1286158019 conn(0x7f21140b5af0 msgr2=0x7f21140b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 shutdown_connections 2024-04-25T09:58:54.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.440+0000 7f211a0c4640 1 -- 172.21.15.148:0/1286158019 wait complete. 2024-04-25T09:58:54.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:58:54.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.547+0000 7f2644688640 1 Processor -- start 2024-04-25T09:58:54.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.547+0000 7f2644688640 1 -- start start 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2644688640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f263c15dbe0 0x7f263c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2644688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 0x7f263c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2644688640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c0b97a0 con 0x7f263c0b2cb0 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2644688640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c15dfc0 con 0x7f263c15dbe0 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2644688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0d1a00 con 0x7f263c0b92a0 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2641c23640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f263c0b2cb0 0x7f263c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46786/0 (socket says 172.21.15.148:46786) 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.548+0000 7f2641c23640 1 -- 172.21.15.148:0/3879836813 learned_addr learned my addr 172.21.15.148:0/3879836813 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:58:54.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2642424640 1 --2- 172.21.15.148:0/3879836813 >> v2:172.21.15.188:3300/0 conn(0x7f263c15dbe0 0x7f263c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2642c25640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 0x7f263c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91420628 0 0) 0x7f263c0b97a0 con 0x7f263c0b2cb0 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2628003520 con 0x7f263c0b2cb0 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4025669502 0 0) 0x7f2628003520 con 0x7f263c0b2cb0 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 msgr2=0x7f263c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 0x7f263c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 >> v2:172.21.15.188:3300/0 conn(0x7f263c15dbe0 msgr2=0x7f263c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 --2- 172.21.15.148:0/3879836813 >> v2:172.21.15.188:3300/0 conn(0x7f263c15dbe0 0x7f263c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0d1c50 con 0x7f263c0b2cb0 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f262c001670 con 0x7f263c0b2cb0 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 >> v1:172.21.15.148:6789/0 conn(0x7f263c0b2cb0 legacy=0x7f263c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 0x7f26280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0d1a00 con 0x7f2628004d40 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262800b9c0 con 0x7f2628003f30 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262800bb40 con 0x7f263c0b92a0 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2642424640 1 --2- 172.21.15.148:0/3879836813 >> v2:172.21.15.188:3300/0 conn(0x7f263c15dbe0 0x7f263c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2642c25640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2641c23640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.549+0000 7f2642424640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 0x7f26280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2642c25640 1 -- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 msgr2=0x7f26280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2642c25640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b92a0 0x7f26280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2642c25640 1 -- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628003f30 msgr2=0x7f2628004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2642c25640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2642c25640 1 -- 172.21.15.148:0/3879836813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628003520 con 0x7f2628004d40 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2642c25640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2628004d40 0x7f26280051c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2634009f90 tx=0x7f2628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24296e14a74ef7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634009c10 con 0x7f2628004d40 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2634036070 con 0x7f2628004d40 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.550+0000 7f2641422640 1 -- 172.21.15.148:0/3879836813 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634031670 con 0x7f2628004d40 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2644688640 1 -- 172.21.15.148:0/3879836813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2628004d40 msgr2=0x7f26280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2644688640 1 --2- 172.21.15.148:0/3879836813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2628004d40 0x7f26280051c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2634009f90 tx=0x7f2628005830 comp rx=0 tx=0).stop 2024-04-25T09:58:54.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2642c25640 1 -- 172.21.15.148:0/3879836813 reap_dead start 2024-04-25T09:58:54.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2644688640 1 -- 172.21.15.148:0/3879836813 shutdown_connections 2024-04-25T09:58:54.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2644688640 1 -- 172.21.15.148:0/3879836813 >> 172.21.15.148:0/3879836813 conn(0x7f263c061a20 msgr2=0x7f263c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2644688640 1 -- 172.21.15.148:0/3879836813 shutdown_connections 2024-04-25T09:58:54.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.551+0000 7f2644688640 1 -- 172.21.15.148:0/3879836813 wait complete. 2024-04-25T09:58:54.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.552+0000 7f2644688640 1 Processor -- start 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.552+0000 7f2644688640 1 -- start start 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2644688640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628004d40 0x7f263c152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2644688640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b54c0 0x7f263c0b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2644688640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 0x7f263c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2644688640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b6ab0 con 0x7f263c156f90 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2644688640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0d2210 con 0x7f2628004d40 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2644688640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b6c30 con 0x7f263c0b54c0 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 0x7f263c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 0x7f263c157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54958/0 (socket says 172.21.15.148:54958) 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 -- 172.21.15.148:0/2327339730 learned_addr learned my addr 172.21.15.148:0/2327339730 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:54.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642424640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628004d40 0x7f263c152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2641c23640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b54c0 0x7f263c0b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 -- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b54c0 msgr2=0x7f263c0b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b54c0 0x7f263c0b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 -- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628004d40 msgr2=0x7f263c152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628004d40 0x7f263c152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2642c25640 1 -- 172.21.15.148:0/2327339730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0cd870 con 0x7f263c156f90 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.553+0000 7f2641c23640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b54c0 0x7f263c0b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.554+0000 7f2642c25640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 0x7f263c157410 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2634009f90 tx=0x7f2634009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.554+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634009c10 con 0x7f263c156f90 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.554+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2634031dd0 con 0x7f263c156f90 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.554+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0caa50 con 0x7f263c156f90 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.554+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634003bd0 con 0x7f263c156f90 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.554+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c0cb010 con 0x7f263c156f90 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.555+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c073990 con 0x7f263c156f90 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.555+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2634005d10 con 0x7f263c156f90 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.556+0000 7f2632ffd640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f262006b500 0x7f262006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.556+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26340ab5a0 con 0x7f263c156f90 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.556+0000 7f2642424640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f262006b500 0x7f262006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.556+0000 7f2642424640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f262006b500 0x7f262006d9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2638009dc0 tx=0x7f2638009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:54.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.560+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2634049060 con 0x7f263c156f90 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.766+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f263c159f60 con 0x7f263c156f90 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.766+0000 7f2632ffd640 1 -- 172.21.15.148:0/2327339730 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f263407a320 con 0x7f263c156f90 2024-04-25T09:58:54.770 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f262006b500 msgr2=0x7f262006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f262006b500 0x7f262006d9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2638009dc0 tx=0x7f2638009190 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 msgr2=0x7f263c157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 0x7f263c157410 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2634009f90 tx=0x7f2634009750 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 shutdown_connections 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f262006b500 0x7f262006d9c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f263c156f90 0x7f263c157410 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f263c0b54c0 0x7f263c0b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 --2- 172.21.15.148:0/2327339730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2628004d40 0x7f263c152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 >> 172.21.15.148:0/2327339730 conn(0x7f263c061a20 msgr2=0x7f263c0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 shutdown_connections 2024-04-25T09:58:54.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:54.768+0000 7f2644688640 1 -- 172.21.15.148:0/2327339730 wait complete. 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:58:54.783 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:54.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:54.783 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T09:58:54.798 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:58:54.799 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42501sec preferred_lft 42501sec 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.800 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:58:54.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:54.801 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:58:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T09:58:54.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:54.862 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T09:58:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T09:58:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:58:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T09:58:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T09:58:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:58:54.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:54.919 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T09:58:54.919 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T09:58:54.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T09:58:54.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T09:58:54.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T09:58:54.920 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T09:58:55.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:55.003 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T09:58:55.057 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:58:55.058 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:58:55.058 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:55.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:55.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T09:58:55.120 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-04-25T09:58:55.116+0000 7febe01d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:58:55.121 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:55.121 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:55.176 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:58:55.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:58:55.177 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:55.231 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T09:58:55.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:55.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:55.312 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:55.312 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:55.312 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:55.320 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:55.321 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:55.321 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:55.321 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:58:55.321 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:58:55.336 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T09:58:55.336 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T09:58:55.336 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T09:58:55.337 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T09:58:55.338 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T09:58:55.338 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T09:58:55.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:55.339 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:58:55.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:55.403 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T09:58:55.464 INFO:teuthology.orchestra.run.smithi148.stdout:client5810 2024-04-25T09:58:55.464 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:55.464 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:55.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T09:58:55.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 Processor -- start 2024-04-25T09:58:55.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 -- start start 2024-04-25T09:58:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f872815dc20 0x7f87280ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f87280d32f0 0x7f87280cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872815e000 con 0x7f8728161710 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87280d1de0 con 0x7f872815dc20 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872f091640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8728161c60 con 0x7f87280d32f0 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872ce2d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f872815dc20 0x7f87280ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872ce2d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f872815dc20 0x7f87280ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54966/0 (socket says 172.21.15.148:54966) 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.658+0000 7f872ce2d640 1 -- 172.21.15.148:0/3352564970 learned_addr learned my addr 172.21.15.148:0/3352564970 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872d62e640 1 --2- 172.21.15.148:0/3352564970 >> v2:172.21.15.188:3300/0 conn(0x7f87280d32f0 0x7f87280cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872ce2d640 1 -- 172.21.15.148:0/3352564970 >> v2:172.21.15.188:3300/0 conn(0x7f87280d32f0 msgr2=0x7f87280cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872ce2d640 1 --2- 172.21.15.148:0/3352564970 >> v2:172.21.15.188:3300/0 conn(0x7f87280d32f0 0x7f87280cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872ce2d640 1 -- 172.21.15.148:0/3352564970 >> v1:172.21.15.188:6789/0 conn(0x7f8728161710 legacy=0x7f872806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872ce2d640 1 -- 172.21.15.148:0/3352564970 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728161f40 con 0x7f872815dc20 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872d62e640 1 --2- 172.21.15.148:0/3352564970 >> v2:172.21.15.188:3300/0 conn(0x7f87280d32f0 0x7f87280cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.659+0000 7f872ce2d640 1 --2- 172.21.15.148:0/3352564970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f87280ce670 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f87280b6910 tx=0x7f871c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee9c74af0905439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f87277fe640 1 -- 172.21.15.148:0/3352564970 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c037070 con 0x7f872815dc20 2024-04-25T09:58:55.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f87277fe640 1 -- 172.21.15.148:0/3352564970 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c032440 con 0x7f872815dc20 2024-04-25T09:58:55.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f87277fe640 1 -- 172.21.15.148:0/3352564970 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c032810 con 0x7f872815dc20 2024-04-25T09:58:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 -- 172.21.15.148:0/3352564970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 msgr2=0x7f87280ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 --2- 172.21.15.148:0/3352564970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f87280ce670 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f87280b6910 tx=0x7f871c02a230 comp rx=0 tx=0).stop 2024-04-25T09:58:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 -- 172.21.15.148:0/3352564970 shutdown_connections 2024-04-25T09:58:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 --2- 172.21.15.148:0/3352564970 >> v2:172.21.15.188:3300/0 conn(0x7f87280d32f0 0x7f87280cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 --2- 172.21.15.148:0/3352564970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f87280ce670 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 -- 172.21.15.148:0/3352564970 >> 172.21.15.148:0/3352564970 conn(0x7f87280d0420 msgr2=0x7f87280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 -- 172.21.15.148:0/3352564970 shutdown_connections 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.660+0000 7f872f091640 1 -- 172.21.15.148:0/3352564970 wait complete. 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.661+0000 7f872f091640 1 Processor -- start 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.661+0000 7f872f091640 1 -- start start 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872f091640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87280d32f0 0x7f872815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872f091640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f872815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872f091640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8728161710 0x7f872815c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872f091640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87280b5e50 con 0x7f872815dc20 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872f091640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8728162500 con 0x7f87280d32f0 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872f091640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87280b5fd0 con 0x7f8728161710 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f8727fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f872815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f8727fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f872815bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54974/0 (socket says 172.21.15.148:54974) 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f8727fff640 1 -- 172.21.15.148:0/53629891 learned_addr learned my addr 172.21.15.148:0/53629891 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872ce2d640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87280d32f0 0x7f872815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.662+0000 7f872d62e640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8728161710 0x7f872815c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f8727fff640 1 -- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8728161710 msgr2=0x7f872815c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f8727fff640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8728161710 0x7f872815c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f8727fff640 1 -- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87280d32f0 msgr2=0x7f872815e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f8727fff640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87280d32f0 0x7f872815e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f8727fff640 1 -- 172.21.15.148:0/53629891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872815f080 con 0x7f872815dc20 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f8727fff640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f872815bcb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f871400b3f0 tx=0x7f871400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714015020 con 0x7f872815dc20 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714011830 con 0x7f872815dc20 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872815f360 con 0x7f872815dc20 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714011c20 con 0x7f872815dc20 2024-04-25T09:58:55.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.663+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87281f3760 con 0x7f872815dc20 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.664+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8728073a00 con 0x7f872815dc20 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.664+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8714018020 con 0x7f872815dc20 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.665+0000 7f87257fa640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.665+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8714089e60 con 0x7f872815dc20 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.665+0000 7f872ce2d640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:58:55.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.665+0000 7f872ce2d640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f871c034000 tx=0x7f871c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:58:55.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.669+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f871401f030 con 0x7f872815dc20 2024-04-25T09:58:55.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.878+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f87280d16b0 con 0x7f872815dc20 2024-04-25T09:58:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.888+0000 7f87257fa640 1 -- 172.21.15.148:0/53629891 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v47) v1 ==== 167+0+0 (secure 0 0 0) 0x7f8714058500 con 0x7f872815dc20 2024-04-25T09:58:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:enabled new snapshots 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.890+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 msgr2=0x7f870c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.890+0000 7f872f091640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f871c034000 tx=0x7f871c02e000 comp rx=0 tx=0).stop 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.890+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 msgr2=0x7f872815bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.890+0000 7f872f091640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f872815bcb0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f871400b3f0 tx=0x7f871400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 shutdown_connections 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8728161710 0x7f872815c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872815dc20 0x7f872815bcb0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 --2- 172.21.15.148:0/53629891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87280d32f0 0x7f872815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 >> 172.21.15.148:0/53629891 conn(0x7f87280d0420 msgr2=0x7f87280b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 shutdown_connections 2024-04-25T09:58:55.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:58:55.891+0000 7f872f091640 1 -- 172.21.15.148:0/53629891 wait complete. 2024-04-25T09:58:55.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:58:55.905 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T09:58:55.924 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T09:58:55.924 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:58:55.980 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> import uuid 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> n = 2000 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> create_hard_links = False 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> if 3 > 0: 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> create_hard_links = True 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> dpath = os.path.dirname(path) 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> fnameprefix = os.path.basename(path) 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> os.makedirs(dpath, exist_ok=True) 2024-04-25T09:58:55.981 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> for i in range(n): 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> f.write(f"{i}") 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> f.flush() 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f.fileno()) 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink(fpath) 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(dirfd) 2024-04-25T09:58:55.982 DEBUG:teuthology.orchestra.run.smithi148:> if create_hard_links: 2024-04-25T09:58:55.983 DEBUG:teuthology.orchestra.run.smithi148:> for j in range(3): 2024-04-25T09:58:55.983 DEBUG:teuthology.orchestra.run.smithi148:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T09:58:55.983 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T09:58:55.983 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(dirfd) 2024-04-25T09:58:55.983 DEBUG:teuthology.orchestra.run.smithi148:> finally: 2024-04-25T09:58:55.984 DEBUG:teuthology.orchestra.run.smithi148:> os.close(dirfd) 2024-04-25T09:58:55.984 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T09:59:12.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:59:12.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:12.010 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:59:12.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:12.282 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T09:59:24.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:24.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:38.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:38.514 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T09:59:38.611 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:59:38.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.726+0000 7ff3d6a4a640 1 Processor -- start 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.726+0000 7ff3d6a4a640 1 -- start start 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3d6a4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3d00cc4f0 0x7ff3d00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3d6a4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d00cdb20 0x7ff3d00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3d6a4a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3d015b270 0x7ff3d015b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3d6a4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d00ce4c0 con 0x7ff3d00cc4f0 2024-04-25T09:59:38.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3d6a4a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d0071270 con 0x7ff3d015b270 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3d6a4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0072280 con 0x7ff3d00cdb20 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3d00cc4f0 0x7ff3d00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3d00cc4f0 0x7ff3d00cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60026/0 (socket says 172.21.15.148:60026) 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 -- 172.21.15.148:0/3311921616 learned_addr learned my addr 172.21.15.148:0/3311921616 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/3311921616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d00cdb20 0x7ff3d00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 -- 172.21.15.148:0/3311921616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d00cdb20 msgr2=0x7ff3d00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 --2- 172.21.15.148:0/3311921616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d00cdb20 0x7ff3d00cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 -- 172.21.15.148:0/3311921616 >> v2:172.21.15.188:3300/0 conn(0x7ff3d015b270 msgr2=0x7ff3d015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 --2- 172.21.15.148:0/3311921616 >> v2:172.21.15.188:3300/0 conn(0x7ff3d015b270 0x7ff3d015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.727+0000 7ff3cffff640 1 -- 172.21.15.148:0/3311921616 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00714c0 con 0x7ff3d00cc4f0 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.728+0000 7ff3cffff640 1 --2- 172.21.15.148:0/3311921616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d00cc8d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0158aa0 tx=0x7ff3c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfeeda8df520486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.728+0000 7ff3ceffd640 1 -- 172.21.15.148:0/3311921616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0037070 con 0x7ff3d00cc4f0 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.728+0000 7ff3ceffd640 1 -- 172.21.15.148:0/3311921616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0032440 con 0x7ff3d00cc4f0 2024-04-25T09:59:38.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.728+0000 7ff3ceffd640 1 -- 172.21.15.148:0/3311921616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0032810 con 0x7ff3d00cc4f0 2024-04-25T09:59:38.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/3311921616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cc4f0 msgr2=0x7ff3d00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/3311921616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d00cc8d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0158aa0 tx=0x7ff3c002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/3311921616 shutdown_connections 2024-04-25T09:59:38.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/3311921616 >> v2:172.21.15.188:3300/0 conn(0x7ff3d015b270 0x7ff3d015b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/3311921616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d00cdb20 0x7ff3d00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/3311921616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d00cc8d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/3311921616 >> 172.21.15.148:0/3311921616 conn(0x7ff3d006abe0 msgr2=0x7ff3d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/3311921616 shutdown_connections 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.729+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/3311921616 wait complete. 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.730+0000 7ff3d6a4a640 1 Processor -- start 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.730+0000 7ff3d6a4a640 1 -- start start 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d6a4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d0144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d6a4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 0x7ff3d01ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d6a4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d015b270 0x7ff3d0156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d6a4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0071d60 con 0x7ff3d00cdb20 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d6a4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0071a80 con 0x7ff3d00cc4f0 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d6a4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d015bbc0 con 0x7ff3d015b270 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 0x7ff3d01ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 0x7ff3d01ef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60032/0 (socket says 172.21.15.148:60032) 2024-04-25T09:59:38.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/448272579 learned_addr learned my addr 172.21.15.148:0/448272579 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cffff640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d0144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d015b270 0x7ff3d0156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d015b270 msgr2=0x7ff3d0156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d015b270 0x7ff3d0156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00cc4f0 msgr2=0x7ff3d0144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d0144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.731+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/448272579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0154320 con 0x7ff3d00cdb20 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.732+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 0x7ff3d01ef080 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c7f0 tx=0x7ff3c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.732+0000 7ff3ccff9640 1 -- 172.21.15.148:0/448272579 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4015070 con 0x7ff3d00cdb20 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.732+0000 7ff3ccff9640 1 -- 172.21.15.148:0/448272579 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c4004220 con 0x7ff3d00cdb20 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.732+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d0154600 con 0x7ff3d00cdb20 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.732+0000 7ff3ccff9640 1 -- 172.21.15.148:0/448272579 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4007620 con 0x7ff3d00cdb20 2024-04-25T09:59:38.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.732+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d0154bc0 con 0x7ff3d00cdb20 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.733+0000 7ff3ccff9640 1 -- 172.21.15.148:0/448272579 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3c40079d0 con 0x7ff3d00cdb20 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.734+0000 7ff3ccff9640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3bc06b4d0 0x7ff3bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.734+0000 7ff3ccff9640 1 -- 172.21.15.148:0/448272579 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff3c408a350 con 0x7ff3d00cdb20 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.734+0000 7ff3cffff640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3bc06b4d0 0x7ff3bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.734+0000 7ff3cffff640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3bc06b4d0 0x7ff3bc06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0034000 tx=0x7ff3c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 Processor -- start 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 -- start start 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4b82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4b81080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4babbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4ae3b90 con 0x55e8e4bab760 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4a8ce40 con 0x55e8e4bbd370 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.735+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4adfc30 con 0x55e8e4b80ca0 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4babbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4babbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60040/0 (socket says 172.21.15.148:60040) 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 -- 172.21.15.148:0/3751063627 learned_addr learned my addr 172.21.15.148:0/3751063627 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4b81080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3cffff640 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4b82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 -- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 msgr2=0x55e8e4b82e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4b82e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 -- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 msgr2=0x55e8e4b81080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4b81080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 -- 172.21.15.148:0/3751063627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8e48c27d0 con 0x55e8e4bab760 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3cffff640 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4b82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4babbe0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800a4b0 tx=0x7ff3c8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1c372886647853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3b5ffb640 1 -- 172.21.15.148:0/3751063627 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8013790 con 0x55e8e4bab760 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3b5ffb640 1 -- 172.21.15.148:0/3751063627 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c8019070 con 0x55e8e4bab760 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.736+0000 7ff3b5ffb640 1 -- 172.21.15.148:0/3751063627 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8013b70 con 0x55e8e4bab760 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 -- 172.21.15.148:0/3751063627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 msgr2=0x55e8e4babbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4babbe0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800a4b0 tx=0x7ff3c8009650 comp rx=0 tx=0).stop 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 -- 172.21.15.148:0/3751063627 shutdown_connections 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4babbe0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4b81080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 --2- 172.21.15.148:0/3751063627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4b82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 -- 172.21.15.148:0/3751063627 >> 172.21.15.148:0/3751063627 conn(0x55e8e4b48870 msgr2=0x55e8e4bc6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 -- 172.21.15.148:0/3751063627 shutdown_connections 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 -- 172.21.15.148:0/3751063627 wait complete. 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.737+0000 7ff3d9826740 1 -- start start 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 Processor -- start 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4b80ca0 0x55e8e4b9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4bbefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4baec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4aa03e0 con 0x55e8e4bab760 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4ba92b0 con 0x55e8e4b80ca0 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4bade20 con 0x55e8e4bbd370 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d9826740 10 client.0 _ll_register_callbacks cb 0x55e8e4a9fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:38.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4bbefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4bbefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60042/0 (socket says 172.21.15.148:60042) 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/256248715 learned_addr learned my addr 172.21.15.148:0/256248715 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3cffff640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4b80ca0 0x55e8e4b9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.738+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4baec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 msgr2=0x55e8e4baec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4baec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4b80ca0 msgr2=0x55e8e4b9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4b80ca0 0x55e8e4b9bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/256248715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8e47b80f0 con 0x55e8e4bab760 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4bbefb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4058980 tx=0x7ff3c4058b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46ed55bbf3b3b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3aeffd640 1 -- 172.21.15.148:0/256248715 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c401e040 con 0x55e8e4bab760 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3aeffd640 1 -- 172.21.15.148:0/256248715 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff3c4007cc0 con 0x55e8e4bab760 2024-04-25T09:59:38.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3c8019070 con 0x55e8e4bab760 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3aeffd640 1 -- 172.21.15.148:0/256248715 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3c4057e40 con 0x55e8e4bab760 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 10 client.7458 fetch_fsmap learned FSMap version 47 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55e8e4922f60 con 0x55e8e4bab760 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3aeffd640 1 -- 172.21.15.148:0/256248715 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c8019070 con 0x55e8e4bab760 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3aeffd640 1 -- 172.21.15.148:0/256248715 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff3c4057fe0 con 0x55e8e4bab760 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 10 client.7458 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 10 client.7458 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 20 client.7458 populate_metadata read hostname 'smithi148' 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4ba7360 0x55e8e4bd9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 4 client.7458 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.739+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8e4af10d0 con 0x55e8e4ba7360 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3cffff640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4ba7360 0x55e8e4bd9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3aeffd640 10 client.7458 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3cffff640 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4ba7360 0x55e8e4bd9ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af889598a20b08d3 server_cookie=60e32f3c42d906f8 in_seq=0 out_seq=0 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3aeffd640 1 -- 172.21.15.148:0/256248715 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8e4af10d0 con 0x55e8e4ba7360 2024-04-25T09:59:38.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3aeffd640 10 client.7458 handle_command_reply: tid=0 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3d9826740 1 client.7458 shutdown 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 msgr2=0x55e8e4bbefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3d9826740 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4bbefb0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4058980 tx=0x7ff3c4058b30 comp rx=0 tx=0).stop 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 shutdown_connections 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.740+0000 7ff3d9826740 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4ba7360 0x55e8e4bd9ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4baec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4bbefb0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 1 --2- 172.21.15.148:0/256248715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4b80ca0 0x55e8e4b9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 >> 172.21.15.148:0/256248715 conn(0x55e8e4b48870 msgr2=0x55e8e4bc9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3aeffd640 0 client.7458 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 shutdown_connections 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 1 -- 172.21.15.148:0/256248715 wait complete. 2024-04-25T09:59:38.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.741+0000 7ff3d9826740 20 client.7458 trim_cache size 0 max 16384 2024-04-25T09:59:38.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 Processor -- start 2024-04-25T09:59:38.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 -- start start 2024-04-25T09:59:38.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 0x55e8e4baaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bab760 0x55e8e4bc9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4bc9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01fc180 con 0x55e8e4b80ca0 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01fb380 con 0x55e8e4bab760 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.755+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0155e70 con 0x55e8e4bbd370 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 0x55e8e4baaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 0x55e8e4baaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60050/0 (socket says 172.21.15.148:60050) 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 -- 172.21.15.148:0/748242585 learned_addr learned my addr 172.21.15.148:0/748242585 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bab760 0x55e8e4bc9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4bc9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 -- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 msgr2=0x55e8e4bc9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4bc9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 -- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bab760 msgr2=0x55e8e4bc9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bab760 0x55e8e4bc9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 -- 172.21.15.148:0/748242585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8e4bbf320 con 0x55e8e4b80ca0 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bab760 0x55e8e4bc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3cffff640 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 0x55e8e4baaf80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff3c003e040 tx=0x7ff3c002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78366e691a3d68e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3b57fa640 1 -- 172.21.15.148:0/748242585 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0037040 con 0x55e8e4b80ca0 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3b57fa640 1 -- 172.21.15.148:0/748242585 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0053030 con 0x55e8e4b80ca0 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.756+0000 7ff3b57fa640 1 -- 172.21.15.148:0/748242585 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c004f400 con 0x55e8e4b80ca0 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 -- 172.21.15.148:0/748242585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 msgr2=0x55e8e4baaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 0x55e8e4baaf80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff3c003e040 tx=0x7ff3c002c000 comp rx=0 tx=0).stop 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 -- 172.21.15.148:0/748242585 shutdown_connections 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4bbd370 0x55e8e4bc9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bab760 0x55e8e4bc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 --2- 172.21.15.148:0/748242585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4b80ca0 0x55e8e4baaf80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 -- 172.21.15.148:0/748242585 >> 172.21.15.148:0/748242585 conn(0x55e8e4b48870 msgr2=0x55e8e4b81b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 -- 172.21.15.148:0/748242585 shutdown_connections 2024-04-25T09:59:38.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 -- 172.21.15.148:0/748242585 wait complete. 2024-04-25T09:59:38.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.757+0000 7ff3d9826740 1 -- start start 2024-04-25T09:59:38.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 Processor -- start 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4c33150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4be9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4bc5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01fad80 con 0x55e8e4bab760 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4b7e0f0 con 0x55e8e4bbd370 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e4bc6100 con 0x55e8e4b80ca0 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d9826740 10 client.0 _ll_register_callbacks cb 0x7ff3c4058350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4be9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4be9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60052/0 (socket says 172.21.15.148:60052) 2024-04-25T09:59:38.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/1494864127 learned_addr learned my addr 172.21.15.148:0/1494864127 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3cffff640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4c33150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.758+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4bc5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 msgr2=0x55e8e4c33150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4c33150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 msgr2=0x55e8e4bc5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4bc5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cf7fe640 1 -- 172.21.15.148:0/1494864127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8e47911b0 con 0x55e8e4bab760 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d4fe7640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4bc5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cf7fe640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4be9310 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4088740 tx=0x7ff3c40885c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f537238a1944c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3aeffd640 1 -- 172.21.15.148:0/1494864127 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c401e070 con 0x55e8e4bab760 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3aeffd640 1 -- 172.21.15.148:0/1494864127 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff3c408a0c0 con 0x55e8e4bab760 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3c0053030 con 0x55e8e4bab760 2024-04-25T09:59:38.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3aeffd640 1 -- 172.21.15.148:0/1494864127 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3c4007cc0 con 0x55e8e4bab760 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 10 client.7464 fetch_fsmap learned FSMap version 47 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55e8e4b80380 con 0x55e8e4bab760 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3aeffd640 1 -- 172.21.15.148:0/1494864127 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0053030 con 0x55e8e4bab760 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3aeffd640 1 -- 172.21.15.148:0/1494864127 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff3c4057fe0 con 0x55e8e4bab760 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 10 client.7464 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 10 client.7464 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 20 client.7464 populate_metadata read hostname 'smithi148' 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4b7b880 0x55e8e4bc5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 4 client.7464 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55e8e4aeb420 con 0x55e8e4b7b880 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.759+0000 7ff3cffff640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4b7b880 0x55e8e4bc5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3aeffd640 10 client.7464 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3cffff640 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4b7b880 0x55e8e4bc5b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7652723c64785b7 server_cookie=8333f508836a8015 in_seq=0 out_seq=0 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3aeffd640 1 -- 172.21.15.148:0/1494864127 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17918 (crc 0 0 0) 0x55e8e4aeb420 con 0x55e8e4b7b880 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3aeffd640 10 client.7464 handle_command_reply: tid=0 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 client.7464 shutdown 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 msgr2=0x55e8e4be9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4be9310 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4088740 tx=0x7ff3c40885c0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 shutdown_connections 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8e4bbd370 0x55e8e4bc5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8e4bab760 0x55e8e4be9310 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55e8e4b7b880 0x55e8e4bc5b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 --2- 172.21.15.148:0/1494864127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e8e4b80ca0 0x55e8e4c33150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3aeffd640 0 client.7464 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 >> 172.21.15.148:0/1494864127 conn(0x55e8e4b48870 msgr2=0x55e8e4baaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 shutdown_connections 2024-04-25T09:59:38.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.760+0000 7ff3d9826740 1 -- 172.21.15.148:0/1494864127 wait complete. 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d9826740 20 client.7464 trim_cache size 0 max 16384 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1335, 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1336, 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 536014, 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 19794427, 2024-04-25T09:59:38.772 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 9, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.183104278, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.102554321, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.026958836, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.041656202, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1334, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.324024205, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000242896 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000183, 2024-04-25T09:59:38.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000091 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 27814, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 50302, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 8778410, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 57160834, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 2.849925225, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 2.221217968, 2024-04-25T09:59:38.774 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.838217149, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.038991942, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 50300, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.358253061, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000027003 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 48965, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.008111698, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000165 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 104784 2024-04-25T09:59:38.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2440, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2434, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 705656, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 46929170, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.311950892, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.179510272, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.049400889, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.073796680, 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:59:38.776 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2434, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.353414008, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000145198 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 115616, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 16704 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-25T09:59:38.777 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1235, 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.501658761, 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.013361667 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T09:59:38.778 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1834, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.169661232, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000637765 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 25280, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 25280, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 25280, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 354.911135465, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.014039206 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-25T09:59:38.779 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 12, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 19816, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 19816, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 19816, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 16, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 16, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 45659, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 45240, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 96, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-25T09:59:38.781 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 1062, 2024-04-25T09:59:38.782 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 4035, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 3765, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 3232, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 51, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 13896 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-25T09:59:38.783 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 2000, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 182, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 4066, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 1818, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 2160, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-25T09:59:38.784 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 26148, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 26148, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-25T09:59:38.785 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 35, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 35, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 4, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 98561453, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2498, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 103.600579062, 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.041473410 2024-04-25T09:59:38.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 2014, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 2014, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 0, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 13, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 13, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 0, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 19816, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 19817, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 1, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 2, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 6268, 2024-04-25T09:59:38.787 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 6266, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 115568, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215396 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 28062, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 25280, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 3, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2000, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.470355994, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000735177 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-25T09:59:38.788 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 29, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003990102, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000137589 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6000, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.790735975, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001131789 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-25T09:59:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7078, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.357593790, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000898218 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.790 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000050882, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000050882 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.791 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002811367, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002811367 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-25T09:59:38.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.211655970, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.211655970 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.441922598, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.180240324 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2160, 2024-04-25T09:59:38.793 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 322.221959028, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.149176832 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-25T09:59:38.794 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.079928256, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.079928256 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.795 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000244572, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000244572 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.796 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.797 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8000, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.329886931, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002041235 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.798 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 18591, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 18591, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 39, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:59:38.799 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:59:38.800 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:59:38.801 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 17396644, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 6395, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 440, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 5, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-25T09:59:38.802 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 5424, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 76, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 20413527, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 862886, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.803 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 4531, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 94843009, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 31, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 4531, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4531, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 22.026464195, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004861280 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-25T09:59:38.804 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4531, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 4619 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 4531, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 43, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 4488, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 43, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 2541, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 28, 2024-04-25T09:59:38.805 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 1818, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 34, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-25T09:59:38.806 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-25T09:59:38.807 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 83, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 1, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 8, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 8, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-25T09:59:38.808 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 94, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 28 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 32, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 1818, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 1818, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 31585, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 7604844, 2024-04-25T09:59:38.810 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 31585, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 31585, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 7604844, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T09:59:38.811 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4531, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 95041171, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4531, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 95041171, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.812 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 4531, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 4531, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 4531, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 4531, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.813 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1818, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 198162, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1818, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:59:38.814 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 43, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 198162, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55c2f99f26e0": { 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 84, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 26148, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 94367149, 2024-04-25T09:59:38.815 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 26148, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2498, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 94367065, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3bc06b4d0 msgr2=0x7ff3bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3bc06b4d0 0x7ff3bc06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0034000 tx=0x7ff3c002e000 comp rx=0 tx=0).stop 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 msgr2=0x7ff3d01ef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 0x7ff3d01ef080 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c7f0 tx=0x7ff3c400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 shutdown_connections 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3bc06b4d0 0x7ff3bc06d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d015b270 0x7ff3d0156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3d00cdb20 0x7ff3d01ef080 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 --2- 172.21.15.148:0/448272579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d00cc4f0 0x7ff3d0144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 >> 172.21.15.148:0/448272579 conn(0x7ff3d006abe0 msgr2=0x7ff3d006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 shutdown_connections 2024-04-25T09:59:38.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.761+0000 7ff3d6a4a640 1 -- 172.21.15.148:0/448272579 wait complete. 2024-04-25T09:59:38.819 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T09:59:38.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:38.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T09:59:38.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.918+0000 7fb95891d640 1 Processor -- start 2024-04-25T09:59:38.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.918+0000 7fb95891d640 1 -- start start 2024-04-25T09:59:38.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb95891d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9541446d0 0x7fb9540b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb95891d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb95406e8a0 0x7fb9540b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb95891d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9540cccc0 con 0x7fb954144d50 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb95891d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9540b2b20 con 0x7fb95406e8a0 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb95891d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95406d400 con 0x7fb9541446d0 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb952d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb95406e8a0 0x7fb9540b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb952d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb95406e8a0 0x7fb9540b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60054/0 (socket says 172.21.15.148:60054) 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb952d9d640 1 -- 172.21.15.148:0/2345947936 learned_addr learned my addr 172.21.15.148:0/2345947936 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb95259c640 1 --2- 172.21.15.148:0/2345947936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9541446d0 0x7fb9540b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb952d9d640 1 -- 172.21.15.148:0/2345947936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9541446d0 msgr2=0x7fb9540b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb952d9d640 1 --2- 172.21.15.148:0/2345947936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9541446d0 0x7fb9540b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.919+0000 7fb952d9d640 1 -- 172.21.15.148:0/2345947936 >> v1:172.21.15.188:6789/0 conn(0x7fb954144d50 legacy=0x7fb9540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.920+0000 7fb952d9d640 1 -- 172.21.15.148:0/2345947936 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540cc140 con 0x7fb95406e8a0 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.920+0000 7fb952d9d640 1 --2- 172.21.15.148:0/2345947936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95406e8a0 0x7fb9540b4460 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009870 tx=0x7fb94c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec9fe5363c15ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.920+0000 7fb95159a640 1 -- 172.21.15.148:0/2345947936 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c037070 con 0x7fb95406e8a0 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.920+0000 7fb95159a640 1 -- 172.21.15.148:0/2345947936 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c032440 con 0x7fb95406e8a0 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.920+0000 7fb95159a640 1 -- 172.21.15.148:0/2345947936 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c032810 con 0x7fb95406e8a0 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 -- 172.21.15.148:0/2345947936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95406e8a0 msgr2=0x7fb9540b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 --2- 172.21.15.148:0/2345947936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95406e8a0 0x7fb9540b4460 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009870 tx=0x7fb94c02a270 comp rx=0 tx=0).stop 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 -- 172.21.15.148:0/2345947936 shutdown_connections 2024-04-25T09:59:38.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 --2- 172.21.15.148:0/2345947936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb95406e8a0 0x7fb9540b4460 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 --2- 172.21.15.148:0/2345947936 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9541446d0 0x7fb9540b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 -- 172.21.15.148:0/2345947936 >> 172.21.15.148:0/2345947936 conn(0x7fb95406abe0 msgr2=0x7fb95406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 -- 172.21.15.148:0/2345947936 shutdown_connections 2024-04-25T09:59:38.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.921+0000 7fb95891d640 1 -- 172.21.15.148:0/2345947936 wait complete. 2024-04-25T09:59:38.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.922+0000 7fb95891d640 1 Processor -- start 2024-04-25T09:59:38.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.922+0000 7fb95891d640 1 -- start start 2024-04-25T09:59:38.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95891d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb95406e8a0 0x7fb95413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95891d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 0x7fb9541f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95891d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb954144d50 0x7fb9541e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95891d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cc700 con 0x7fb9541446d0 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95891d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540b49a0 con 0x7fb95406e8a0 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95891d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540cc9e0 con 0x7fb954144d50 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 0x7fb9541f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 0x7fb9541f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60064/0 (socket says 172.21.15.148:60064) 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 -- 172.21.15.148:0/3270279476 learned_addr learned my addr 172.21.15.148:0/3270279476 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb95259c640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb95406e8a0 0x7fb95413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb952d9d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb954144d50 0x7fb9541e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 -- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb954144d50 msgr2=0x7fb9541e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb954144d50 0x7fb9541e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 -- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb95406e8a0 msgr2=0x7fb95413fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb95406e8a0 0x7fb95413fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 -- 172.21.15.148:0/3270279476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95413cf30 con 0x7fb9541446d0 2024-04-25T09:59:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.923+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 0x7fb9541f37b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb94800c400 tx=0x7fb94800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.924+0000 7fb942ffd640 1 -- 172.21.15.148:0/3270279476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948017070 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.924+0000 7fb942ffd640 1 -- 172.21.15.148:0/3270279476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9480052d0 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.924+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95413d6b0 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.924+0000 7fb942ffd640 1 -- 172.21.15.148:0/3270279476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948003830 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.924+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95415b100 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.925+0000 7fb942ffd640 1 -- 172.21.15.148:0/3270279476 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb94801a070 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.925+0000 7fb942ffd640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.926+0000 7fb942ffd640 1 -- 172.21.15.148:0/3270279476 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb9480895a0 con 0x7fb9541446d0 2024-04-25T09:59:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.926+0000 7fb95259c640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.926+0000 7fb95259c640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb9540cf4c0 tx=0x7fb94400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 Processor -- start 2024-04-25T09:59:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 -- start start 2024-04-25T09:59:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb983200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb983740 0x558fcb9ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb8ec0a0 con 0x558fcb9b3080 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb894d40 con 0x558fcb9b3aa0 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.927+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fcb8e8140 con 0x558fcb983740 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60070/0 (socket says 172.21.15.148:60070) 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 -- 172.21.15.148:0/185061363 learned_addr learned my addr 172.21.15.148:0/185061363 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb952d9d640 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb983740 0x558fcb9ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb951d9b640 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb983200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 -- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb983740 msgr2=0x558fcb9ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb983740 0x558fcb9ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 -- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 msgr2=0x558fcb983200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb983200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 -- 172.21.15.148:0/185061363 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fcb6cca20 con 0x558fcb9b3080 2024-04-25T09:59:38.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb951d9b640 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb983200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb95259c640 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9b3460 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb944003d60 tx=0x7fb944014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217fc1860ef62865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb933fff640 1 -- 172.21.15.148:0/185061363 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94401d050 con 0x558fcb9b3080 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb933fff640 1 -- 172.21.15.148:0/185061363 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944027070 con 0x558fcb9b3080 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.928+0000 7fb933fff640 1 -- 172.21.15.148:0/185061363 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944015a60 con 0x558fcb9b3080 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 -- 172.21.15.148:0/185061363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 msgr2=0x558fcb9b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9b3460 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb944003d60 tx=0x7fb944014000 comp rx=0 tx=0).stop 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 -- 172.21.15.148:0/185061363 shutdown_connections 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb983740 0x558fcb9ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb983200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 --2- 172.21.15.148:0/185061363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9b3460 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 -- 172.21.15.148:0/185061363 >> 172.21.15.148:0/185061363 conn(0x558fcb985470 msgr2=0x558fcb9cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 -- 172.21.15.148:0/185061363 shutdown_connections 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 -- 172.21.15.148:0/185061363 wait complete. 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.929+0000 7fb95b61a740 1 -- start start 2024-04-25T09:59:38.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 Processor -- start 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb9b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b4b90 0x558fcb9b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9af7d0 con 0x558fcb9b3080 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb88a3d0 con 0x558fcb9b3aa0 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.930+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9afa70 con 0x558fcb9b4b90 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95b61a740 10 client.0 _ll_register_callbacks cb 0x558fcb5ffe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60082/0 (socket says 172.21.15.148:60082) 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 -- 172.21.15.148:0/4124536990 learned_addr learned my addr 172.21.15.148:0/4124536990 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb951d9b640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb9b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb952d9d640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b4b90 0x558fcb9b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 -- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b4b90 msgr2=0x558fcb9b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b4b90 0x558fcb9b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 -- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 msgr2=0x558fcb9b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb9b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 -- 172.21.15.148:0/4124536990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fcb5c0310 con 0x558fcb9b3080 2024-04-25T09:59:38.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95259c640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9deb00 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb94400afd0 tx=0x7fb944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3f0451536fb2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb9317fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb9317fa640 1 -- 172.21.15.148:0/4124536990 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944015d10 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb9317fa640 1 -- 172.21.15.148:0/4124536990 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb944036c70 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb944027070 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb9317fa640 1 -- 172.21.15.148:0/4124536990 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb94401d020 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95b61a740 10 client.7476 fetch_fsmap learned FSMap version 47 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.931+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x558fcb72d1b0 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb9317fa640 1 -- 172.21.15.148:0/4124536990 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944027070 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb9317fa640 1 -- 172.21.15.148:0/4124536990 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb944022440 con 0x558fcb9b3080 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb95b61a740 10 client.7476 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb95b61a740 10 client.7476 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb95b61a740 20 client.7476 populate_metadata read hostname 'smithi148' 2024-04-25T09:59:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb95b61a740 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9a7b60 0x558fcb9a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb95b61a740 4 client.7476 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fcb8f95e0 con 0x558fcb9a7b60 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb951d9b640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9a7b60 0x558fcb9a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb9317fa640 10 client.7476 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.932+0000 7fb951d9b640 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9a7b60 0x558fcb9a7f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4faed5c0ffece51f server_cookie=ee6604631dbfb44a in_seq=0 out_seq=0 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb9317fa640 1 -- 172.21.15.148:0/4124536990 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fcb8f95e0 con 0x558fcb9a7b60 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb9317fa640 10 client.7476 handle_command_reply: tid=0 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 client.7476 shutdown 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 msgr2=0x558fcb9deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9deb00 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb94400afd0 tx=0x7fb944004a40 comp rx=0 tx=0).stop 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 shutdown_connections 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9a7b60 0x558fcb9a7f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b4b90 0x558fcb9b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3aa0 0x558fcb9b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb9317fa640 0 client.7476 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 --2- 172.21.15.148:0/4124536990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3080 0x558fcb9deb00 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 >> 172.21.15.148:0/4124536990 conn(0x558fcb983740 msgr2=0x558fcb9a95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 shutdown_connections 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 1 -- 172.21.15.148:0/4124536990 wait complete. 2024-04-25T09:59:38.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.933+0000 7fb95b61a740 20 client.7476 trim_cache size 0 max 16384 2024-04-25T09:59:38.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.948+0000 7fb95b61a740 1 Processor -- start 2024-04-25T09:59:38.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.948+0000 7fb95b61a740 1 -- start start 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb983740 0x558fcb9dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3080 0x558fcb9f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 0x558fcb9a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95b61a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9542005c0 con 0x558fcb9b3aa0 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9541ff7f0 con 0x558fcb983740 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540b1280 con 0x558fcb9b3080 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 0x558fcb9a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 0x558fcb9a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60088/0 (socket says 172.21.15.148:60088) 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 -- 172.21.15.148:0/3700268722 learned_addr learned my addr 172.21.15.148:0/3700268722 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95259c640 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb983740 0x558fcb9dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3080 0x558fcb9f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 -- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3080 msgr2=0x558fcb9f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3080 0x558fcb9f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 -- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb983740 msgr2=0x558fcb9dd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb983740 0x558fcb9dd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 -- 172.21.15.148:0/3700268722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fcb599490 con 0x558fcb9b3aa0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb95259c640 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb983740 0x558fcb9dd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.949+0000 7fb952d9d640 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 0x558fcb9a6a20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009870 tx=0x7fb94c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5820a37f2d1ae25a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb9337fe640 1 -- 172.21.15.148:0/3700268722 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c04b070 con 0x558fcb9b3aa0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb9337fe640 1 -- 172.21.15.148:0/3700268722 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0328b0 con 0x558fcb9b3aa0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb9337fe640 1 -- 172.21.15.148:0/3700268722 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c037040 con 0x558fcb9b3aa0 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 -- 172.21.15.148:0/3700268722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 msgr2=0x558fcb9a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 0x558fcb9a6a20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009870 tx=0x7fb94c02e000 comp rx=0 tx=0).stop 2024-04-25T09:59:38.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 -- 172.21.15.148:0/3700268722 shutdown_connections 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb9b3aa0 0x558fcb9a6a20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3080 0x558fcb9f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3700268722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb983740 0x558fcb9dd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 -- 172.21.15.148:0/3700268722 >> 172.21.15.148:0/3700268722 conn(0x558fcb986380 msgr2=0x558fcb9b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 -- 172.21.15.148:0/3700268722 shutdown_connections 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.950+0000 7fb95b61a740 1 -- 172.21.15.148:0/3700268722 wait complete. 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.951+0000 7fb95b61a740 1 -- start start 2024-04-25T09:59:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.951+0000 7fb95b61a740 1 Processor -- start 2024-04-25T09:59:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 0x558fcb9b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3080 0x558fcb9a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3aa0 0x558fcb9a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540b1880 con 0x558fcb983740 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9a86e0 con 0x558fcb9b3080 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fcb9cd870 con 0x558fcb9b3aa0 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95b61a740 10 client.0 _ll_register_callbacks cb 0x7fb944009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 0x558fcb9b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 0x558fcb9b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60104/0 (socket says 172.21.15.148:60104) 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 -- 172.21.15.148:0/3374832919 learned_addr learned my addr 172.21.15.148:0/3374832919 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb952d9d640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3aa0 0x558fcb9a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3080 0x558fcb9a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 -- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3aa0 msgr2=0x558fcb9a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3aa0 0x558fcb9a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 -- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3080 msgr2=0x558fcb9a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3080 0x558fcb9a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 -- 172.21.15.148:0/3374832919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fcb9def50 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb95259c640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 0x558fcb9b0a70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb9440093e0 tx=0x7fb944018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdc41ca0d591713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.952+0000 7fb9317fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb9317fa640 1 -- 172.21.15.148:0/3374832919 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944022990 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb9317fa640 1 -- 172.21.15.148:0/3374832919 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb944022b30 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb94c0328b0 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb9317fa640 1 -- 172.21.15.148:0/3374832919 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb944015970 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 10 client.7482 fetch_fsmap learned FSMap version 47 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x558fcb9a5a80 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb9317fa640 1 -- 172.21.15.148:0/3374832919 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0328b0 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb9317fa640 1 -- 172.21.15.148:0/3374832919 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb944015b10 con 0x558fcb983740 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 10 client.7482 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 10 client.7482 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:59:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 20 client.7482 populate_metadata read hostname 'smithi148' 2024-04-25T09:59:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9b4b90 0x558fcb9a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 4 client.7482 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T09:59:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x558fcb8f3930 con 0x558fcb9b4b90 2024-04-25T09:59:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9b4b90 0x558fcb9a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb9317fa640 10 client.7482 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.953+0000 7fb951d9b640 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9b4b90 0x558fcb9a4b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa3917cc6bc16a5 server_cookie=aa460da6913ee17a in_seq=0 out_seq=0 2024-04-25T09:59:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb9317fa640 1 -- 172.21.15.148:0/3374832919 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558fcb8f3930 con 0x558fcb9b4b90 2024-04-25T09:59:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb9317fa640 10 client.7482 handle_command_reply: tid=0 2024-04-25T09:59:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 client.7482 shutdown 2024-04-25T09:59:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 msgr2=0x558fcb9b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 0x558fcb9b0a70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb9440093e0 tx=0x7fb944018000 comp rx=0 tx=0).stop 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 shutdown_connections 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x558fcb9b4b90 0x558fcb9a4b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558fcb9b3aa0 0x558fcb9a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558fcb9b3080 0x558fcb9a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 --2- 172.21.15.148:0/3374832919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fcb983740 0x558fcb9b0a70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 >> 172.21.15.148:0/3374832919 conn(0x558fcb986380 msgr2=0x558fcb9a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb9317fa640 0 client.7482 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 shutdown_connections 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 1 -- 172.21.15.148:0/3374832919 wait complete. 2024-04-25T09:59:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.959+0000 7fb95b61a740 20 client.7482 trim_cache size 0 max 16384 2024-04-25T09:59:38.966 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:59:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T09:59:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "cf9467b2-c102-4ebb-8903-c0b814e157d1", 2024-04-25T09:59:38.966 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T09:59:38.966 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 msgr2=0x7fb93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb9540cf4c0 tx=0x7fb94400b040 comp rx=0 tx=0).stop 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 msgr2=0x7fb9541f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 0x7fb9541f37b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb94800c400 tx=0x7fb94800c8c0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 shutdown_connections 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb93406b4d0 0x7fb93406d990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb954144d50 0x7fb9541e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9541446d0 0x7fb9541f37b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 --2- 172.21.15.148:0/3270279476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb95406e8a0 0x7fb95413fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 >> 172.21.15.148:0/3270279476 conn(0x7fb95406abe0 msgr2=0x7fb9540af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:38.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 shutdown_connections 2024-04-25T09:59:38.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:38.960+0000 7fb95891d640 1 -- 172.21.15.148:0/3270279476 wait complete. 2024-04-25T09:59:38.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:59:38.979 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:59:39.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.119+0000 7f68d8df5640 1 Processor -- start 2024-04-25T09:59:39.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.119+0000 7f68d8df5640 1 -- start start 2024-04-25T09:59:39.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d8df5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68d4161ef0 0x7f68d40716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d8df5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40c6910 con 0x7f68d40cf590 2024-04-25T09:59:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d8df5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40c6bf0 con 0x7f68d40ce7b0 2024-04-25T09:59:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d8df5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40c76d0 con 0x7f68d4161ef0 2024-04-25T09:59:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d1d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f68d40cf590 0x7f68d40ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36954/0 (socket says 172.21.15.148:36954) 2024-04-25T09:59:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d1d9b640 1 -- 172.21.15.148:0/554427434 learned_addr learned my addr 172.21.15.148:0/554427434 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T09:59:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.120+0000 7f68d259c640 1 --2- 172.21.15.148:0/554427434 >> v2:172.21.15.188:3300/0 conn(0x7f68d4161ef0 0x7f68d40716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263442161 0 0) 0x7f68d40c6910 con 0x7f68d40cf590 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c0003520 con 0x7f68d40cf590 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075882849 0 0) 0x7f68d40c6bf0 con 0x7f68d40ce7b0 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d40c6910 con 0x7f68d40ce7b0 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d259c640 1 -- 172.21.15.148:0/554427434 >> v1:172.21.15.188:6789/0 conn(0x7f68d40ce7b0 legacy=0x7f68d4165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d259c640 1 -- 172.21.15.148:0/554427434 >> v1:172.21.15.148:6789/0 conn(0x7f68d40cf590 legacy=0x7f68d40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.121+0000 7f68d259c640 1 -- 172.21.15.148:0/554427434 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d40c6ed0 con 0x7f68d4161ef0 2024-04-25T09:59:39.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.122+0000 7f68d259c640 1 --2- 172.21.15.148:0/554427434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d4161ef0 0x7f68d40716d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f68d40c7850 tx=0x7f68c402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898825876c918849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.122+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4037070 con 0x7f68d4161ef0 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.122+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c4032440 con 0x7f68d4161ef0 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.122+0000 7f68d159a640 1 -- 172.21.15.148:0/554427434 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4032810 con 0x7f68d4161ef0 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 -- 172.21.15.148:0/554427434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d4161ef0 msgr2=0x7f68d40716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 --2- 172.21.15.148:0/554427434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d4161ef0 0x7f68d40716d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f68d40c7850 tx=0x7f68c402a210 comp rx=0 tx=0).stop 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 -- 172.21.15.148:0/554427434 shutdown_connections 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 --2- 172.21.15.148:0/554427434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d4161ef0 0x7f68d40716d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 -- 172.21.15.148:0/554427434 >> 172.21.15.148:0/554427434 conn(0x7f68d406f500 msgr2=0x7f68d406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 -- 172.21.15.148:0/554427434 shutdown_connections 2024-04-25T09:59:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.123+0000 7f68d8df5640 1 -- 172.21.15.148:0/554427434 wait complete. 2024-04-25T09:59:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.125+0000 7f68d8df5640 1 Processor -- start 2024-04-25T09:59:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.125+0000 7f68d8df5640 1 -- start start 2024-04-25T09:59:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d8df5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 0x7f68d414ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d8df5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40cf590 0x7f68d41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d8df5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d4161ef0 0x7f68d414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40cf590 0x7f68d41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 0x7f68d414ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 0x7f68d414ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:45700/0 (socket says 172.21.15.148:45700) 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40cf590 0x7f68d41562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60118/0 (socket says 172.21.15.148:60118) 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d8df5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41622d0 con 0x7f68d40cf590 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d259c640 1 -- 172.21.15.148:0/2211962721 learned_addr learned my addr 172.21.15.148:0/2211962721 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d8df5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40b0b50 con 0x7f68d40ce7b0 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.127+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40b0e00 con 0x7f68d4161ef0 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d4161ef0 0x7f68d414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d259c640 1 -- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d4161ef0 msgr2=0x7f68d414c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d259c640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d4161ef0 0x7f68d414c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d259c640 1 -- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40cf590 msgr2=0x7f68d41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d259c640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40cf590 0x7f68d41562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d259c640 1 -- 172.21.15.148:0/2211962721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4156800 con 0x7f68d40ce7b0 2024-04-25T09:59:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68d259c640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 0x7f68d414ca50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f68c402a800 tx=0x7f68c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.128+0000 7f68caffd640 1 -- 172.21.15.148:0/2211962721 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4037050 con 0x7f68d40ce7b0 2024-04-25T09:59:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.129+0000 7f68caffd640 1 -- 172.21.15.148:0/2211962721 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c4043070 con 0x7f68d40ce7b0 2024-04-25T09:59:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.129+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d40b56a0 con 0x7f68d40ce7b0 2024-04-25T09:59:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.129+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d40b5c60 con 0x7f68d40ce7b0 2024-04-25T09:59:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.129+0000 7f68caffd640 1 -- 172.21.15.148:0/2211962721 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c403cb10 con 0x7f68d40ce7b0 2024-04-25T09:59:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.131+0000 7f68caffd640 1 -- 172.21.15.148:0/2211962721 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68c403ccb0 con 0x7f68d40ce7b0 2024-04-25T09:59:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.131+0000 7f68caffd640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68b406b4d0 0x7f68b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.132+0000 7f68caffd640 1 -- 172.21.15.148:0/2211962721 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68c40aaf00 con 0x7f68d40ce7b0 2024-04-25T09:59:39.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.132+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68b406b4d0 0x7f68b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.132+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68b406b4d0 0x7f68b406d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f68d4156270 tx=0x7f68bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.133+0000 7f68dbbd2740 1 Processor -- start 2024-04-25T09:59:39.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.133+0000 7f68dbbd2740 1 -- start start 2024-04-25T09:59:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.134+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eb10 0x5620a3331320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.134+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3331860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.134+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 0x5620a3304f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.134+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a326ada0 con 0x5620a332eef0 2024-04-25T09:59:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.134+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a3213b80 con 0x5620a332eb10 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.134+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a3266e40 con 0x5620a3301770 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 0x5620a3304f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eb10 0x5620a3331320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 0x5620a3304f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:36856/0 (socket says 172.21.15.148:36856) 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 -- 172.21.15.148:0/1684516475 learned_addr learned my addr 172.21.15.148:0/1684516475 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3331860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 -- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eb10 msgr2=0x5620a3331320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eb10 0x5620a3331320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 -- 172.21.15.148:0/1684516475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 msgr2=0x5620a3331860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3331860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 -- 172.21.15.148:0/1684516475 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a321df10 con 0x5620a3301770 2024-04-25T09:59:39.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.135+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 0x5620a3304f80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68cc001f20 tx=0x7f68cc0016d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfda35f83eb4e761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.136+0000 7f68b3fff640 1 -- 172.21.15.148:0/1684516475 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00fe50 con 0x5620a3301770 2024-04-25T09:59:39.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.136+0000 7f68b3fff640 1 -- 172.21.15.148:0/1684516475 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc029070 con 0x5620a3301770 2024-04-25T09:59:39.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.136+0000 7f68b3fff640 1 -- 172.21.15.148:0/1684516475 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc003bb0 con 0x5620a3301770 2024-04-25T09:59:39.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 msgr2=0x5620a3304f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 0x5620a3304f80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f68cc001f20 tx=0x7f68cc0016d0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1684516475 shutdown_connections 2024-04-25T09:59:39.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a3301770 0x5620a3304f80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3331860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1684516475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eb10 0x5620a3331320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1684516475 >> 172.21.15.148:0/1684516475 conn(0x5620a33039e0 msgr2=0x5620a334c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1684516475 shutdown_connections 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.137+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1684516475 wait complete. 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.138+0000 7f68dbbd2740 1 -- start start 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.138+0000 7f68dbbd2740 1 Processor -- start 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.139+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a332eb10 0x5620a3328e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.139+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3349080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.139+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a334f530 0x5620a334f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.139+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a3306b90 con 0x5620a332eef0 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.139+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a32094b0 con 0x5620a334f530 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.139+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a3306e70 con 0x5620a332eb10 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3349080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3349080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60144/0 (socket says 172.21.15.148:60144) 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 -- 172.21.15.148:0/1172481790 learned_addr learned my addr 172.21.15.148:0/1172481790 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a334f530 0x5620a334f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d259c640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a332eb10 0x5620a3328e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68dbbd2740 10 client.0 _ll_register_callbacks cb 0x5620a3249180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 -- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a332eb10 msgr2=0x5620a3328e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a332eb10 0x5620a3328e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 -- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a334f530 msgr2=0x5620a334f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a334f530 0x5620a334f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 -- 172.21.15.148:0/1172481790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620a30497d0 con 0x5620a332eef0 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d259c640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a332eb10 0x5620a3328e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3349080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0010e0 tx=0x7f68bc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4db35be33e16f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:59:39.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68b17fa640 1 -- 172.21.15.148:0/1172481790 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc025020 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68b17fa640 1 -- 172.21.15.148:0/1172481790 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68bc013680 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.140+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68cc029070 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68b17fa640 1 -- 172.21.15.148:0/1172481790 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68bc02f050 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 10 client.7494 fetch_fsmap learned FSMap version 47 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x5620a2f7f3f0 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68b17fa640 1 -- 172.21.15.148:0/1172481790 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc01b070 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68b17fa640 1 -- 172.21.15.148:0/1172481790 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68bc032020 con 0x5620a332eef0 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 10 client.7494 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 10 client.7494 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 20 client.7494 populate_metadata read hostname 'smithi148' 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a3346e20 0x5620a3347200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 4 client.7494 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:59:39.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68cc029070 con 0x5620a3346e20 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.141+0000 7f68d259c640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a3346e20 0x5620a3347200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.142+0000 7f68b17fa640 10 client.7494 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.142+0000 7f68d259c640 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a3346e20 0x5620a3347200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c4d5c4fc4c4d12 server_cookie=5c1a83dab1249361 in_seq=0 out_seq=0 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68b17fa640 1 -- 172.21.15.148:0/1172481790 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f68cc029070 con 0x5620a3346e20 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68b17fa640 10 client.7494 handle_command_reply: tid=0 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68dbbd2740 1 client.7494 shutdown 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 msgr2=0x5620a3349080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3349080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0010e0 tx=0x7f68bc000ed0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 shutdown_connections 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.143+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a334f530 0x5620a334f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eef0 0x5620a3349080 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a3346e20 0x5620a3347200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/1172481790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a332eb10 0x5620a3328e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 >> 172.21.15.148:0/1172481790 conn(0x5620a3301770 msgr2=0x5620a3302420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68b17fa640 0 client.7494 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:39.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 shutdown_connections 2024-04-25T09:59:39.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 1 -- 172.21.15.148:0/1172481790 wait complete. 2024-04-25T09:59:39.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.144+0000 7f68dbbd2740 20 client.7494 trim_cache size 0 max 16384 2024-04-25T09:59:39.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.168+0000 7f68dbbd2740 1 Processor -- start 2024-04-25T09:59:39.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.168+0000 7f68dbbd2740 1 -- start start 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 0x5620a33293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a32fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68d259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 0x5620a33293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a334f530 0x5620a334bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68d259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 0x5620a33293d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60160/0 (socket says 172.21.15.148:60160) 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68dbbd2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d412bcc0 con 0x5620a332eb10 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4158300 con 0x5620a332eef0 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68d259c640 1 -- 172.21.15.148:0/2554536272 learned_addr learned my addr 172.21.15.148:0/2554536272 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4149ba0 con 0x5620a334f530 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a334f530 0x5620a334bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.169+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a32fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d259c640 1 -- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a334f530 msgr2=0x5620a334bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d259c640 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a334f530 0x5620a334bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d259c640 1 -- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 msgr2=0x5620a32fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d259c640 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a32fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d259c640 1 -- 172.21.15.148:0/2554536272 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a2f3f0f0 con 0x5620a332eb10 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a334f530 0x5620a334bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68d259c640 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 0x5620a33293d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f68c4079520 tx=0x7f68c40a9160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b683b9d2aa2d7866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68b37fe640 1 -- 172.21.15.148:0/2554536272 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c407d630 con 0x5620a332eb10 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68b37fe640 1 -- 172.21.15.148:0/2554536272 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c4038040 con 0x5620a332eb10 2024-04-25T09:59:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.170+0000 7f68b37fe640 1 -- 172.21.15.148:0/2554536272 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4081bf0 con 0x5620a332eb10 2024-04-25T09:59:39.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.171+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 msgr2=0x5620a33293d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.171+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 0x5620a33293d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f68c4079520 tx=0x7f68c40a9160 comp rx=0 tx=0).stop 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.171+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 shutdown_connections 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.172+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a334f530 0x5620a334bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.172+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a32fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.172+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/2554536272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a332eb10 0x5620a33293d0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.172+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 >> 172.21.15.148:0/2554536272 conn(0x5620a3301770 msgr2=0x5620a33313e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.172+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 shutdown_connections 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.172+0000 7f68dbbd2740 1 -- 172.21.15.148:0/2554536272 wait complete. 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.173+0000 7f68dbbd2740 1 -- start start 2024-04-25T09:59:39.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.173+0000 7f68dbbd2740 1 Processor -- start 2024-04-25T09:59:39.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.174+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a3305440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.174+0000 7f68dbbd2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 0x5620a3305980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.174+0000 7f68d259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a3305440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.174+0000 7f68d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 0x5620a3305980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.174+0000 7f68d1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 0x5620a3305980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60168/0 (socket says 172.21.15.148:60168) 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d1d9b640 1 -- 172.21.15.148:0/611999624 learned_addr learned my addr 172.21.15.148:0/611999624 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a33b5c80 0x5620a33271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a3306520 con 0x5620a334f530 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4157d00 con 0x5620a332eef0 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620a32fd430 con 0x5620a33b5c80 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d2d9d640 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a33b5c80 0x5620a33271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d1d9b640 1 -- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a33b5c80 msgr2=0x5620a33271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a33b5c80 0x5620a33271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d1d9b640 1 -- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 msgr2=0x5620a3305440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a3305440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68d1d9b640 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620a33548d0 con 0x5620a334f530 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68dbbd2740 10 client.0 _ll_register_callbacks cb 0x7f68cc0019c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:59:39.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.175+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68c4038040 con 0x5620a334f530 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68d1d9b640 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 0x5620a3305980 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000f60 tx=0x7f68bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc4e06005bcc98b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68b17fa640 10 client.7500 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68b17fa640 1 -- 172.21.15.148:0/611999624 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc025070 con 0x5620a334f530 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68b17fa640 1 -- 172.21.15.148:0/611999624 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68bc013d90 con 0x5620a334f530 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68b17fa640 1 -- 172.21.15.148:0/611999624 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68bc013ae0 con 0x5620a334f530 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68dbbd2740 10 client.7500 fetch_fsmap learned FSMap version 47 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.176+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=47}) v3 -- 0x5620a3327f60 con 0x5620a334f530 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68b17fa640 1 -- 172.21.15.148:0/611999624 <== mon.0 v2:172.21.15.148:3300/0 4 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68bc032040 con 0x5620a334f530 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68dbbd2740 10 client.7500 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68dbbd2740 10 client.7500 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68dbbd2740 20 client.7500 populate_metadata read hostname 'smithi148' 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a33295d0 0x5620a3327720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68dbbd2740 4 client.7500 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620a332a290 con 0x5620a33295d0 2024-04-25T09:59:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68d259c640 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a33295d0 0x5620a3327720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:59:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68b17fa640 10 client.7500 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:39.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.177+0000 7f68d259c640 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a33295d0 0x5620a3327720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2adcbadf5adeeb server_cookie=4ba54f80f579527b in_seq=0 out_seq=0 2024-04-25T09:59:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.178+0000 7f68b17fa640 1 -- 172.21.15.148:0/611999624 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+264 (crc 0 0 0) 0x5620a332a290 con 0x5620a33295d0 2024-04-25T09:59:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.178+0000 7f68b17fa640 10 client.7500 handle_command_reply: tid=0 2024-04-25T09:59:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.178+0000 7f68dbbd2740 1 client.7500 shutdown 2024-04-25T09:59:39.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.178+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 msgr2=0x5620a3305980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.178+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 0x5620a3305980 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000f60 tx=0x7f68bc014000 comp rx=0 tx=0).stop 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 shutdown_connections 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5620a33295d0 0x5620a3327720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620a33b5c80 0x5620a33271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620a334f530 0x5620a3305980 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 --2- 172.21.15.148:0/611999624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620a332eef0 0x5620a3305440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 >> 172.21.15.148:0/611999624 conn(0x5620a3301770 msgr2=0x5620a3328ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68b17fa640 0 client.7500 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 shutdown_connections 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 1 -- 172.21.15.148:0/611999624 wait complete. 2024-04-25T09:59:39.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.179+0000 7f68dbbd2740 20 client.7500 trim_cache size 0 max 16384 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "scrub active (94 inodes in the stack)", 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "cf9467b2-c102-4ebb-8903-c0b814e157d1": { 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "~mds0", 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "cf9467b2-c102-4ebb-8903-c0b814e157d1", 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T09:59:39.187 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T09:59:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68b406b4d0 msgr2=0x7f68b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68b406b4d0 0x7f68b406d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f68d4156270 tx=0x7f68bc009040 comp rx=0 tx=0).stop 2024-04-25T09:59:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 msgr2=0x7f68d414ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:59:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 0x7f68d414ca50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f68c402a800 tx=0x7f68c40046c0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 shutdown_connections 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68b406b4d0 0x7f68b406d990 secure :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f68d4156270 tx=0x7f68bc009040 comp rx=0 tx=0).stop 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68d4161ef0 0x7f68d414c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68d40cf590 0x7f68d41562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 --2- 172.21.15.148:0/2211962721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68d40ce7b0 0x7f68d414ca50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 >> 172.21.15.148:0/2211962721 conn(0x7f68d406f500 msgr2=0x7f68d4160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 shutdown_connections 2024-04-25T09:59:39.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T09:59:39.180+0000 7f68d8df5640 1 -- 172.21.15.148:0/2211962721 wait complete. 2024-04-25T09:59:39.197 INFO:tasks.cephfs.filesystem:scrub status for tag:cf9467b2-c102-4ebb-8903-c0b814e157d1 - {'path': '~mds0', 'tag': 'cf9467b2-c102-4ebb-8903-c0b814e157d1', 'options': 'recursive'} 2024-04-25T09:59:54.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:59:54.350 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:09.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:09.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:00:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.308+0000 7f1039cfc640 1 Processor -- start 2024-04-25T10:00:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.308+0000 7f1039cfc640 1 -- start start 2024-04-25T10:00:09.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1039cfc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 0x7f103406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1039cfc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10340cccc0 con 0x7f10341452a0 2024-04-25T10:00:09.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1039cfc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10340cc140 con 0x7f10341446d0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1039cfc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10340b2b20 con 0x7f1034144d50 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1033fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f10341452a0 0x7f103406ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36014/0 (socket says 172.21.15.148:36014) 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1033fff640 1 -- 172.21.15.148:0/744895928 learned_addr learned my addr 172.21.15.148:0/744895928 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f1032ffd640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 0x7f103406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038447555 0 0) 0x7f10340cccc0 con 0x7f10341452a0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.309+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c003520 con 0x7f10341452a0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 380443720 0 0) 0x7f101c003520 con 0x7f10341452a0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 msgr2=0x7f103406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 0x7f103406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 >> v1:172.21.15.188:6789/0 conn(0x7f10341446d0 legacy=0x7f10340b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340cc420 con 0x7f10341452a0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f102c0019f0 con 0x7f10341452a0 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 >> v1:172.21.15.148:6789/0 conn(0x7f10341452a0 legacy=0x7f103406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f101c0039e0 0x7f101c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 0x7f101c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c004d40 0x7f101c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10340b2b20 con 0x7f101c004d40 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c00ba70 con 0x7f101c0039e0 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c00bc70 con 0x7f1034144d50 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1033fff640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c004d40 0x7f101c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f10337fe640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f101c0039e0 0x7f101c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1032ffd640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 0x7f101c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1033fff640 1 -- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 msgr2=0x7f101c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1033fff640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1034144d50 0x7f101c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1033fff640 1 -- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f101c0039e0 msgr2=0x7f101c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1033fff640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f101c0039e0 0x7f101c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.310+0000 7f1033fff640 1 -- 172.21.15.148:0/744895928 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c003520 con 0x7f101c004d40 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.311+0000 7f1033fff640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c004d40 0x7f101c0051c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f102c00a810 tx=0x7f101c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5befd35ecc7e1b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.311+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c00a460 con 0x7f101c004d40 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.311+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c02f070 con 0x7f101c004d40 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.311+0000 7f10327fc640 1 -- 172.21.15.148:0/744895928 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c02bc90 con 0x7f101c004d40 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1039cfc640 1 -- 172.21.15.148:0/744895928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c004d40 msgr2=0x7f101c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1039cfc640 1 --2- 172.21.15.148:0/744895928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f101c004d40 0x7f101c0051c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f102c00a810 tx=0x7f101c005830 comp rx=0 tx=0).stop 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1033fff640 1 -- 172.21.15.148:0/744895928 reap_dead start 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1039cfc640 1 -- 172.21.15.148:0/744895928 shutdown_connections 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1039cfc640 1 -- 172.21.15.148:0/744895928 >> 172.21.15.148:0/744895928 conn(0x7f103406abe0 msgr2=0x7f103406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1039cfc640 1 -- 172.21.15.148:0/744895928 shutdown_connections 2024-04-25T10:00:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.312+0000 7f1039cfc640 1 -- 172.21.15.148:0/744895928 wait complete. 2024-04-25T10:00:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.313+0000 7f1039cfc640 1 Processor -- start 2024-04-25T10:00:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.313+0000 7f1039cfc640 1 -- start start 2024-04-25T10:00:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1039cfc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f103415a4e0 0x7f10341f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1039cfc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103413e3b0 0x7f103413c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1039cfc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 0x7f103413c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1039cfc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10340cc9e0 con 0x7f103415a8c0 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1039cfc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103406d400 con 0x7f103413e3b0 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1039cfc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1034144ab0 con 0x7f103415a4e0 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 0x7f103413c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 0x7f103413c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54816/0 (socket says 172.21.15.148:54816) 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 -- 172.21.15.148:0/2993310596 learned_addr learned my addr 172.21.15.148:0/2993310596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f10337fe640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f103415a4e0 0x7f10341f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1032ffd640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103413e3b0 0x7f103413c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 -- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f103415a4e0 msgr2=0x7f10341f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f103415a4e0 0x7f10341f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 -- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103413e3b0 msgr2=0x7f103413c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103413e3b0 0x7f103413c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 -- 172.21.15.148:0/2993310596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10341f4730 con 0x7f103415a8c0 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.314+0000 7f1033fff640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 0x7f103413c760 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f102c000f80 tx=0x7f103415bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.315+0000 7f101bfff640 1 -- 172.21.15.148:0/2993310596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c045070 con 0x7f103415a8c0 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.315+0000 7f101bfff640 1 -- 172.21.15.148:0/2993310596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c040480 con 0x7f103415a8c0 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.315+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10341f4a10 con 0x7f103415a8c0 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.315+0000 7f101bfff640 1 -- 172.21.15.148:0/2993310596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c00a460 con 0x7f103415a8c0 2024-04-25T10:00:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.315+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10341f4cf0 con 0x7f103415a8c0 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.316+0000 7f101bfff640 1 -- 172.21.15.148:0/2993310596 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f102c048070 con 0x7f103415a8c0 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.316+0000 7f101bfff640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f102006b500 0x7f102006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.317+0000 7f101bfff640 1 -- 172.21.15.148:0/2993310596 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f102c0aa140 con 0x7f103415a8c0 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.317+0000 7f10337fe640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f102006b500 0x7f102006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.317+0000 7f10337fe640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f102006b500 0x7f102006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f10240017e0 tx=0x7f1024009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.318+0000 7f103c9f7740 1 Processor -- start 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.318+0000 7f103c9f7740 1 -- start start 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511b10 0x55819e514320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511ef0 0x55819e514860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 0x55819e4e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f103c9f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819e44dda0 con 0x55819e4e4770 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819e3f6b80 con 0x55819e511b10 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55819e449e40 con 0x55819e511ef0 2024-04-25T10:00:09.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 0x55819e4e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 0x55819e4e7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54820/0 (socket says 172.21.15.148:54820) 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 -- 172.21.15.148:0/1754132097 learned_addr learned my addr 172.21.15.148:0/1754132097 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f10337fe640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511b10 0x55819e514320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1032ffd640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511ef0 0x55819e514860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 -- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511ef0 msgr2=0x55819e514860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511ef0 0x55819e514860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 -- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511b10 msgr2=0x55819e514320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511b10 0x55819e514320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 -- 172.21.15.148:0/1754132097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55819e400f10 con 0x55819e4e4770 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f10337fe640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511b10 0x55819e514320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1032ffd640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511ef0 0x55819e514860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f1033fff640 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 0x55819e4e7f80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f102c080b60 tx=0x7f102c074e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5119eea3b69dff8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f10197fa640 1 -- 172.21.15.148:0/1754132097 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c07cad0 con 0x55819e4e4770 2024-04-25T10:00:09.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f10197fa640 1 -- 172.21.15.148:0/1754132097 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c02f040 con 0x55819e4e4770 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.319+0000 7f10197fa640 1 -- 172.21.15.148:0/1754132097 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c081090 con 0x55819e4e4770 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 -- 172.21.15.148:0/1754132097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 msgr2=0x55819e4e7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 0x55819e4e7f80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f102c080b60 tx=0x7f102c074e00 comp rx=0 tx=0).stop 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 -- 172.21.15.148:0/1754132097 shutdown_connections 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e4e4770 0x55819e4e7f80 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511ef0 0x55819e514860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 --2- 172.21.15.148:0/1754132097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511b10 0x55819e514320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 -- 172.21.15.148:0/1754132097 >> 172.21.15.148:0/1754132097 conn(0x55819e4e69e0 msgr2=0x55819e52f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 -- 172.21.15.148:0/1754132097 shutdown_connections 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.320+0000 7f103c9f7740 1 -- 172.21.15.148:0/1754132097 wait complete. 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.321+0000 7f103c9f7740 1 -- start start 2024-04-25T10:00:09.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.321+0000 7f103c9f7740 1 Processor -- start 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.321+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e531b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.321+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.321+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e5327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819e3f6b80 con 0x55819e511ef0 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819e3ec4b0 con 0x55819e52a360 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55819e44dda0 con 0x55819e511b10 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 10 client.0 _ll_register_callbacks cb 0x55819e42c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:09.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54822/0 (socket says 172.21.15.148:54822) 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 -- 172.21.15.148:0/79665263 learned_addr learned my addr 172.21.15.148:0/79665263 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1033fff640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e5327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f10337fe640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e531b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 -- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 msgr2=0x55819e531b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e531b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 -- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 msgr2=0x55819e5327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e5327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 -- 172.21.15.148:0/79665263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55819e22c7d0 con 0x55819e511ef0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f10337fe640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e531b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1033fff640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e5327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f1032ffd640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50c5d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f10280177b0 tx=0x7f1028017c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27207f8570a6d886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f10127fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f10127fc640 1 -- 172.21.15.148:0/79665263 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028017e60 con 0x55819e511ef0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f10127fc640 1 -- 172.21.15.148:0/79665263 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f1028015c10 con 0x55819e511ef0 2024-04-25T10:00:09.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f102c02f040 con 0x55819e511ef0 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f10127fc640 1 -- 172.21.15.148:0/79665263 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1028015e10 con 0x55819e511ef0 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 10 client.7512 fetch_fsmap learned FSMap version 47 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.322+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55819e1623f0 con 0x55819e511ef0 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f10127fc640 1 -- 172.21.15.148:0/79665263 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c02f040 con 0x55819e511ef0 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f10127fc640 1 -- 172.21.15.148:0/79665263 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f102802c080 con 0x55819e511ef0 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f103c9f7740 10 client.7512 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f103c9f7740 10 client.7512 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f103c9f7740 20 client.7512 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f103c9f7740 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50d850 0x55819e50dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f103c9f7740 4 client.7512 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55819e45b2e0 con 0x55819e50d850 2024-04-25T10:00:09.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f10337fe640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50d850 0x55819e50dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f10337fe640 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50d850 0x55819e50dc30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90503046bfe4a919 server_cookie=a059f226e0ec644c in_seq=0 out_seq=0 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.323+0000 7f10127fc640 10 client.7512 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f10127fc640 1 -- 172.21.15.148:0/79665263 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55819e45b2e0 con 0x55819e50d850 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f10127fc640 10 client.7512 handle_command_reply: tid=0 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 client.7512 shutdown 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 msgr2=0x55819e50c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50c5d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f10280177b0 tx=0x7f1028017c70 comp rx=0 tx=0).stop 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 shutdown_connections 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e5327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50c5d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50d850 0x55819e50dc30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 --2- 172.21.15.148:0/79665263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e531b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f10127fc640 0 client.7512 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 >> 172.21.15.148:0/79665263 conn(0x55819e4e4770 msgr2=0x55819e4e9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 shutdown_connections 2024-04-25T10:00:09.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 1 -- 172.21.15.148:0/79665263 wait complete. 2024-04-25T10:00:09.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.324+0000 7f103c9f7740 20 client.7512 trim_cache size 0 max 16384 2024-04-25T10:00:09.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.339+0000 7f103c9f7740 1 Processor -- start 2024-04-25T10:00:09.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.339+0000 7f103c9f7740 1 -- start start 2024-04-25T10:00:09.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e4e4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511ef0 0x55819e50fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 0x55819e551870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f103c9f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034201160 con 0x55819e52a360 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034200360 con 0x55819e511ef0 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10341ff790 con 0x55819e511b10 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 0x55819e551870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 0x55819e551870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54830/0 (socket says 172.21.15.148:54830) 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 -- 172.21.15.148:0/297195738 learned_addr learned my addr 172.21.15.148:0/297195738 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f10337fe640 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e4e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1032ffd640 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511ef0 0x55819e50fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 -- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 msgr2=0x55819e4e4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e4e4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 -- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511ef0 msgr2=0x55819e50fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511ef0 0x55819e50fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 -- 172.21.15.148:0/297195738 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55819e1220f0 con 0x55819e52a360 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1033fff640 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 0x55819e551870 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f102c002230 tx=0x7f102c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351e6454e821395b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.340+0000 7f1018ff9640 1 -- 172.21.15.148:0/297195738 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0a7070 con 0x55819e52a360 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f1018ff9640 1 -- 172.21.15.148:0/297195738 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c0a5070 con 0x55819e52a360 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f1018ff9640 1 -- 172.21.15.148:0/297195738 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c0a8270 con 0x55819e52a360 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 -- 172.21.15.148:0/297195738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 msgr2=0x55819e551870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 0x55819e551870 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f102c002230 tx=0x7f102c04c000 comp rx=0 tx=0).stop 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 -- 172.21.15.148:0/297195738 shutdown_connections 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e52a360 0x55819e551870 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e511ef0 0x55819e50fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 --2- 172.21.15.148:0/297195738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e4e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 -- 172.21.15.148:0/297195738 >> 172.21.15.148:0/297195738 conn(0x55819e4e4770 msgr2=0x55819e4e3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 -- 172.21.15.148:0/297195738 shutdown_connections 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.341+0000 7f103c9f7740 1 -- 172.21.15.148:0/297195738 wait complete. 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.342+0000 7f103c9f7740 1 -- start start 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.342+0000 7f103c9f7740 1 Processor -- start 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e599010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e50aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341ffd60 con 0x55819e511ef0 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819e5291f0 con 0x55819e52a360 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55819e5172f0 con 0x55819e511b10 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 10 client.0 _ll_register_callbacks cb 0x7f102c002180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54842/0 (socket says 172.21.15.148:54842) 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 -- 172.21.15.148:0/3664958154 learned_addr learned my addr 172.21.15.148:0/3664958154 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1033fff640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e50aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f10337fe640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e599010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 -- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 msgr2=0x55819e599010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e599010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 -- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 msgr2=0x55819e50aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e50aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 -- 172.21.15.148:0/3664958154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55819e52a140 con 0x55819e511ef0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f1032ffd640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50a4c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f10280177b0 tx=0x7f1028004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39a9accc7eb0fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f10127fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f10127fc640 1 -- 172.21.15.148:0/3664958154 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028030070 con 0x55819e511ef0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f10127fc640 1 -- 172.21.15.148:0/3664958154 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f10280044c0 con 0x55819e511ef0 2024-04-25T10:00:09.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.343+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f102c0a5070 con 0x55819e511ef0 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10127fc640 1 -- 172.21.15.148:0/3664958154 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f102803b050 con 0x55819e511ef0 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 10 client.7518 fetch_fsmap learned FSMap version 47 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55819e531340 con 0x55819e511ef0 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10127fc640 1 -- 172.21.15.148:0/3664958154 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c0a5070 con 0x55819e511ef0 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10127fc640 1 -- 172.21.15.148:0/3664958154 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f102803e020 con 0x55819e511ef0 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 10 client.7518 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 10 client.7518 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 20 client.7518 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50bd00 0x55819e531ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 4 client.7518 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55819e455630 con 0x55819e50bd00 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10337fe640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50bd00 0x55819e531ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10127fc640 10 client.7518 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10337fe640 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50bd00 0x55819e531ab0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1d7029aeeb6fe7 server_cookie=207de2c74f700726 in_seq=0 out_seq=0 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10127fc640 1 -- 172.21.15.148:0/3664958154 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55819e455630 con 0x55819e50bd00 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f10127fc640 10 client.7518 handle_command_reply: tid=0 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 client.7518 shutdown 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 msgr2=0x55819e50a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50a4c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f10280177b0 tx=0x7f1028004730 comp rx=0 tx=0).stop 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 shutdown_connections 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55819e52a360 0x55819e50aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55819e511ef0 0x55819e50a4c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x55819e50bd00 0x55819e531ab0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.344+0000 7f103c9f7740 1 --2- 172.21.15.148:0/3664958154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55819e511b10 0x55819e599010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f10127fc640 0 client.7518 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 >> 172.21.15.148:0/3664958154 conn(0x55819e4e4770 msgr2=0x55819e4e3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 shutdown_connections 2024-04-25T10:00:09.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f103c9f7740 1 -- 172.21.15.148:0/3664958154 wait complete. 2024-04-25T10:00:09.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f103c9f7740 20 client.7518 trim_cache size 0 max 16384 2024-04-25T10:00:09.357 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:09.357 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:00:09.357 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:00:09.357 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f102006b500 msgr2=0x7f102006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f102006b500 0x7f102006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f10240017e0 tx=0x7f1024009040 comp rx=0 tx=0).stop 2024-04-25T10:00:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 msgr2=0x7f103413c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 0x7f103413c760 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f102c000f80 tx=0x7f103415bfe0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 shutdown_connections 2024-04-25T10:00:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f102006b500 0x7f102006d9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f103415a8c0 0x7f103413c760 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f103413e3b0 0x7f103413c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 --2- 172.21.15.148:0/2993310596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f103415a4e0 0x7f10341f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 >> 172.21.15.148:0/2993310596 conn(0x7f103406abe0 msgr2=0x7f10341433b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 shutdown_connections 2024-04-25T10:00:09.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.345+0000 7f1039cfc640 1 -- 172.21.15.148:0/2993310596 wait complete. 2024-04-25T10:00:09.360 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:00:09.361 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:00:09.361 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:00:09.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.456+0000 7f23797d5640 1 Processor -- start 2024-04-25T10:00:09.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.456+0000 7f23797d5640 1 -- start start 2024-04-25T10:00:09.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23797d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23740d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23797d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23740d1b80 0x7f237406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23797d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f237406c870 0x7f23740b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23797d5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2374158610 con 0x7f237406c870 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23797d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237406bea0 con 0x7f23740d1b80 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23797d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740d2020 con 0x7f2374158230 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f237406c870 0x7f23740b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f237406c870 0x7f23740b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54844/0 (socket says 172.21.15.148:54844) 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 -- 172.21.15.148:0/375572783 learned_addr learned my addr 172.21.15.148:0/375572783 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f236bfff640 1 --2- 172.21.15.148:0/375572783 >> v2:172.21.15.188:3300/0 conn(0x7f23740d1b80 0x7f237406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f2372ffd640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23740d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 -- 172.21.15.148:0/375572783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 msgr2=0x7f23740d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23740d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 -- 172.21.15.148:0/375572783 >> v2:172.21.15.188:3300/0 conn(0x7f23740d1b80 msgr2=0x7f237406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 --2- 172.21.15.148:0/375572783 >> v2:172.21.15.188:3300/0 conn(0x7f23740d1b80 0x7f237406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f23737fe640 1 -- 172.21.15.148:0/375572783 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740d2300 con 0x7f237406c870 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f236bfff640 1 --2- 172.21.15.148:0/375572783 >> v2:172.21.15.188:3300/0 conn(0x7f23740d1b80 0x7f237406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.457+0000 7f2372ffd640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23740d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23737fe640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740b4e60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f23740d2830 tx=0x7f236c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d04d9153e8e7fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23727fc640 1 -- 172.21.15.148:0/375572783 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c036070 con 0x7f237406c870 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23727fc640 1 -- 172.21.15.148:0/375572783 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c031600 con 0x7f237406c870 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23727fc640 1 -- 172.21.15.148:0/375572783 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0319d0 con 0x7f237406c870 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 -- 172.21.15.148:0/375572783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 msgr2=0x7f23740b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740b4e60 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f23740d2830 tx=0x7f236c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 -- 172.21.15.148:0/375572783 shutdown_connections 2024-04-25T10:00:09.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740b4e60 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 --2- 172.21.15.148:0/375572783 >> v2:172.21.15.188:3300/0 conn(0x7f23740d1b80 0x7f237406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 --2- 172.21.15.148:0/375572783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23740d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.458+0000 7f23797d5640 1 -- 172.21.15.148:0/375572783 >> 172.21.15.148:0/375572783 conn(0x7f2374061600 msgr2=0x7f23740619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.459+0000 7f23797d5640 1 -- 172.21.15.148:0/375572783 shutdown_connections 2024-04-25T10:00:09.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.459+0000 7f23797d5640 1 -- 172.21.15.148:0/375572783 wait complete. 2024-04-25T10:00:09.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 Processor -- start 2024-04-25T10:00:09.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 -- start start 2024-04-25T10:00:09.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23740d1b80 0x7f23741e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23741e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237415e320 con 0x7f237406c870 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740d25e0 con 0x7f23740d1b80 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f23797d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237415e4a0 con 0x7f2374158230 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f2372ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f2372ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54846/0 (socket says 172.21.15.148:54846) 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.460+0000 7f2372ffd640 1 -- 172.21.15.148:0/2518500888 learned_addr learned my addr 172.21.15.148:0/2518500888 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f2372ffd640 1 -- 172.21.15.148:0/2518500888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 msgr2=0x7f23741e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f2372ffd640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23741e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f2372ffd640 1 -- 172.21.15.148:0/2518500888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23740d1b80 msgr2=0x7f23741e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f2372ffd640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23740d1b80 0x7f23741e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f2372ffd640 1 -- 172.21.15.148:0/2518500888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740cbbd0 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f2372ffd640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740cb690 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f236400c7f0 tx=0x7f236400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f236b7fe640 1 -- 172.21.15.148:0/2518500888 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364015070 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f236b7fe640 1 -- 172.21.15.148:0/2518500888 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364007a30 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23741e6100 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f236b7fe640 1 -- 172.21.15.148:0/2518500888 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364007e00 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.461+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23741e66c0 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.462+0000 7f236b7fe640 1 -- 172.21.15.148:0/2518500888 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2364018070 con 0x7f237406c870 2024-04-25T10:00:09.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.463+0000 7f236b7fe640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f235406b4d0 0x7f235406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.463+0000 7f236bfff640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f235406b4d0 0x7f235406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.463+0000 7f236b7fe640 1 -- 172.21.15.148:0/2518500888 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f236408a230 con 0x7f237406c870 2024-04-25T10:00:09.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.463+0000 7f236bfff640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f235406b4d0 0x7f235406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f23740b0d70 tx=0x7f235c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.464+0000 7f237c5ba740 1 Processor -- start 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.464+0000 7f237c5ba740 1 -- start start 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557684380ca0 0x557684382e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x557684381080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f237c5ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576842e3b90 con 0x5576843ab760 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768428ce40 con 0x5576843bd370 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576842dfc30 con 0x557684380ca0 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54850/0 (socket says 172.21.15.148:54850) 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 -- 172.21.15.148:0/1672952136 learned_addr learned my addr 172.21.15.148:0/1672952136 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f236bfff640 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x557684381080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f2372ffd640 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557684380ca0 0x557684382e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 -- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557684380ca0 msgr2=0x557684382e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557684380ca0 0x557684382e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 -- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 msgr2=0x557684381080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x557684381080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 -- 172.21.15.148:0/1672952136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576840c27d0 con 0x5576843ab760 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f236bfff640 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x557684381080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f23737fe640 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843abbe0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f236c009f40 tx=0x7f236c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786ed25885addadc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f2368ff9640 1 -- 172.21.15.148:0/1672952136 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c036050 con 0x5576843ab760 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f2368ff9640 1 -- 172.21.15.148:0/1672952136 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c042070 con 0x5576843ab760 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.465+0000 7f2368ff9640 1 -- 172.21.15.148:0/1672952136 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c03d8b0 con 0x5576843ab760 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 -- 172.21.15.148:0/1672952136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 msgr2=0x5576843abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843abbe0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f236c009f40 tx=0x7f236c02c000 comp rx=0 tx=0).stop 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 -- 172.21.15.148:0/1672952136 shutdown_connections 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843abbe0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x557684381080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 --2- 172.21.15.148:0/1672952136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557684380ca0 0x557684382e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 -- 172.21.15.148:0/1672952136 >> 172.21.15.148:0/1672952136 conn(0x557684348870 msgr2=0x5576843c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 -- 172.21.15.148:0/1672952136 shutdown_connections 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.466+0000 7f237c5ba740 1 -- 172.21.15.148:0/1672952136 wait complete. 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.467+0000 7f237c5ba740 1 -- start start 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.467+0000 7f237c5ba740 1 Processor -- start 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.467+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x5576843a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.467+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576842a03e0 con 0x5576843ab760 2024-04-25T10:00:09.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576843a92b0 con 0x557684380ca0 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x557684396c20 con 0x5576843bd370 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 10 client.0 _ll_register_callbacks cb 0x55768429fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54864/0 (socket says 172.21.15.148:54864) 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f2372ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x5576843a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 -- 172.21.15.148:0/393195293 learned_addr learned my addr 172.21.15.148:0/393195293 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f23737fe640 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 -- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 msgr2=0x5576843c0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843c0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 -- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 msgr2=0x5576843a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x5576843a7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 -- 172.21.15.148:0/393195293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557683fb80f0 con 0x5576843ab760 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f236bfff640 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843c9110 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f235c00cda0 tx=0x7f235c006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0f29cb9ec94da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f2351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f2351ffb640 1 -- 172.21.15.148:0/393195293 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c01f420 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f2351ffb640 1 -- 172.21.15.148:0/393195293 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f235c01f5c0 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f236c042070 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f2351ffb640 1 -- 172.21.15.148:0/393195293 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f235c01f7c0 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 10 client.7530 fetch_fsmap learned FSMap version 47 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x557684122f60 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.468+0000 7f2351ffb640 1 -- 172.21.15.148:0/393195293 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c042070 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f2351ffb640 1 -- 172.21.15.148:0/393195293 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f235c02f080 con 0x5576843ab760 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f237c5ba740 10 client.7530 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f237c5ba740 10 client.7530 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f237c5ba740 20 client.7530 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f237c5ba740 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843d6760 0x5576843d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f237c5ba740 4 client.7530 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:09.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576842f10d0 con 0x5576843d6760 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f2372ffd640 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843d6760 0x5576843d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f2351ffb640 10 client.7530 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f2372ffd640 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843d6760 0x5576843d6b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb390b572f4e8f90 server_cookie=d6a1a0c940030365 in_seq=0 out_seq=0 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f2351ffb640 1 -- 172.21.15.148:0/393195293 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576842f10d0 con 0x5576843d6760 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.469+0000 7f2351ffb640 10 client.7530 handle_command_reply: tid=0 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 client.7530 shutdown 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 msgr2=0x5576843c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843c9110 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f235c00cda0 tx=0x7f235c006be0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 shutdown_connections 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843d6760 0x5576843d6b60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843c0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f2351ffb640 0 client.7530 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843c9110 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 --2- 172.21.15.148:0/393195293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x5576843a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 >> 172.21.15.148:0/393195293 conn(0x557684348870 msgr2=0x5576843a88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 shutdown_connections 2024-04-25T10:00:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 1 -- 172.21.15.148:0/393195293 wait complete. 2024-04-25T10:00:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.470+0000 7f237c5ba740 20 client.7530 trim_cache size 0 max 16384 2024-04-25T10:00:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 Processor -- start 2024-04-25T10:00:09.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 -- start start 2024-04-25T10:00:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 0x5576843d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843ab760 0x5576843d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x5576843a9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23742003d0 con 0x557684380ca0 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741ff5b0 con 0x5576843bd370 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.484+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2374155cf0 con 0x5576843ab760 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 0x5576843d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 0x5576843d8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54876/0 (socket says 172.21.15.148:54876) 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 -- 172.21.15.148:0/754286993 learned_addr learned my addr 172.21.15.148:0/754286993 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 -- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843ab760 msgr2=0x5576843d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843ab760 0x5576843d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 -- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 msgr2=0x5576843a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f23737fe640 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x5576843a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x5576843a9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 -- 172.21.15.148:0/754286993 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557683f911b0 con 0x557684380ca0 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2372ffd640 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 0x5576843d8b50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2364058590 tx=0x7f236401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4055078059a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2353fff640 1 -- 172.21.15.148:0/754286993 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236401f040 con 0x557684380ca0 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2353fff640 1 -- 172.21.15.148:0/754286993 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364056b90 con 0x557684380ca0 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.485+0000 7f2353fff640 1 -- 172.21.15.148:0/754286993 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364059bc0 con 0x557684380ca0 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 -- 172.21.15.148:0/754286993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 msgr2=0x5576843d8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 0x5576843d8b50 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2364058590 tx=0x7f236401c040 comp rx=0 tx=0).stop 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 -- 172.21.15.148:0/754286993 shutdown_connections 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5576843bd370 0x5576843a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843ab760 0x5576843d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 --2- 172.21.15.148:0/754286993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557684380ca0 0x5576843d8b50 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 -- 172.21.15.148:0/754286993 >> 172.21.15.148:0/754286993 conn(0x557684348870 msgr2=0x5576843c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 -- 172.21.15.148:0/754286993 shutdown_connections 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 -- 172.21.15.148:0/754286993 wait complete. 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 -- start start 2024-04-25T10:00:09.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.486+0000 7f237c5ba740 1 Processor -- start 2024-04-25T10:00:09.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x557684397090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576842f10d0 con 0x5576843ab760 2024-04-25T10:00:09.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23741562f0 con 0x557684380ca0 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576843a86e0 con 0x5576843bd370 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f237c5ba740 10 client.0 _ll_register_callbacks cb 0x7f235c00a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54890/0 (socket says 172.21.15.148:54890) 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 -- 172.21.15.148:0/882632424 learned_addr learned my addr 172.21.15.148:0/882632424 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f2372ffd640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x557684397090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f23737fe640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 -- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 msgr2=0x5576843a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843a9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 -- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 msgr2=0x557684397090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x557684397090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f236bfff640 1 -- 172.21.15.148:0/882632424 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557684433720 con 0x5576843ab760 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.487+0000 7f2372ffd640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x557684397090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f236bfff640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843975d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c710 tx=0x7f235c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318ee35a71a5bdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 1 -- 172.21.15.148:0/882632424 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c036420 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 1 -- 172.21.15.148:0/882632424 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f235c037e20 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2364056b90 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 1 -- 172.21.15.148:0/882632424 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f235c0353c0 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 10 client.7536 fetch_fsmap learned FSMap version 47 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55768443c940 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 1 -- 172.21.15.148:0/882632424 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364056b90 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 1 -- 172.21.15.148:0/882632424 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f235c03e050 con 0x5576843ab760 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 10 client.7536 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 10 client.7536 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 20 client.7536 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843a9890 0x55768437f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 4 client.7536 mds_command: new command op to 7284 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:00:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 --> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5576842eb420 con 0x5576843a9890 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2372ffd640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843a9890 0x55768437f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2351ffb640 10 client.7536 ms_handle_connect on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.488+0000 7f2372ffd640 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843a9890 0x55768437f460 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1146077e54143970 server_cookie=8cd6c28b1ba3b930 in_seq=0 out_seq=0 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f2351ffb640 1 -- 172.21.15.148:0/882632424 <== mds.0 v2:172.21.15.148:6835/82715169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17932 (crc 0 0 0) 0x5576842eb420 con 0x5576843a9890 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f2351ffb640 10 client.7536 handle_command_reply: tid=0 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 client.7536 shutdown 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 msgr2=0x5576843975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843975d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f235c00c710 tx=0x7f235c011000 comp rx=0 tx=0).stop 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 shutdown_connections 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:6835/82715169,v1:172.21.15.148:6837/82715169] conn(0x5576843a9890 0x55768437f460 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5576843bd370 0x5576843a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5576843ab760 0x5576843975d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 --2- 172.21.15.148:0/882632424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557684380ca0 0x557684397090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f2351ffb640 0 client.7536 ms_handle_reset on v2:172.21.15.148:6835/82715169 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 >> 172.21.15.148:0/882632424 conn(0x557684348870 msgr2=0x5576843c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 shutdown_connections 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 1 -- 172.21.15.148:0/882632424 wait complete. 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.489+0000 7f237c5ba740 20 client.7536 trim_cache size 0 max 16384 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:00:09.500 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3061, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3062, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1175916, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 21867772, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 17, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.299689776, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.153308735, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.051769974, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.076066716, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3060, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.444944510, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000145406 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.501 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000001149, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000114 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 28760, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 51224, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 9022938, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 57472488, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-04-25T10:00:09.502 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 2.907990477, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 2.238231652, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.851747030, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.062290324, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 51222, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.446636626, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000028242 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 48967, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.008113143, 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000165 2024-04-25T10:00:09.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 115536 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3848, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3842, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 1064514, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 47394497, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.406743551, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.221691314, 2024-04-25T10:00:09.504 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.068107290, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.100524476, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3842, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.455135629, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000118463 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 119488, 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 22944 2024-04-25T10:00:09.505 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2650, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.854528348, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006360199 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-04-25T10:00:09.506 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3938, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.380600192, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000350584 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 25280, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 25280, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 25280, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 354.911135465, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.014039206 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.507 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 12, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 17816, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 17816, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 17816, 2024-04-25T10:00:09.508 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 2, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 2, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 45660, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 45241, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 96, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-04-25T10:00:09.509 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 2011, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 2011, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 11, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T10:00:09.510 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 2000, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 1062, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 4035, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 3765, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 3232, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 51, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 13896 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-04-25T10:00:09.511 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 4066, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 3818, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 2160, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-04-25T10:00:09.512 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-04-25T10:00:09.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:00:09.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-04-25T10:00:09.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-04-25T10:00:09.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:00:09.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-04-25T10:00:09.513 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-04-25T10:00:09.514 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.514 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-04-25T10:00:09.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-04-25T10:00:09.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 28155, 2024-04-25T10:00:09.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-04-25T10:00:09.514 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 28155, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 37, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 37, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 4, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-04-25T10:00:09.515 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 99771965, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2502, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 103.621689653, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.041415543 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.516 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 14, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 2015, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 2001, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 13, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 13, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 0, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 17816, 2024-04-25T10:00:09.517 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 19817, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 2001, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 2, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 6268, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 6266, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 100508, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215396 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 28062, 2024-04-25T10:00:09.518 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 25280, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 5, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2000, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.470355994, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000735177 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 29, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003990102, 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000137589 2024-04-25T10:00:09.519 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6000, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.790735975, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001131789 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7078, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 6.357593790, 2024-04-25T10:00:09.520 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000898218 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.521 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000050882, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000050882 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-04-25T10:00:09.522 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002811367, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002811367 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.523 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.211655970, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.211655970 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-04-25T10:00:09.524 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.441922598, 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.180240324 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2160, 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 322.221959028, 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.149176832 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.525 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.526 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.079928256, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.079928256 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000244572, 2024-04-25T10:00:09.527 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000244572 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.528 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.529 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8000, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 16.329886931, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002041235 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.530 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 18591, 2024-04-25T10:00:09.531 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 18591, 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 69, 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-04-25T10:00:09.532 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:00:09.533 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:00:09.534 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:00:09.535 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 250233, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 92, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 440, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 5, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 5528, 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 78, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 13415253, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 660636, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-04-25T10:00:09.537 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 8558, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 96268030, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 31, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 8558, 2024-04-25T10:00:09.538 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8558, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 32.909199422, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003845431 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8558, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 8648 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 8558, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 2059, 2024-04-25T10:00:09.539 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 6499, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 47, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 2548, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 30, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 1818, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 2034, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-04-25T10:00:09.540 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 2012, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:00:09.541 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-04-25T10:00:09.542 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 85, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 3, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 8, 2024-04-25T10:00:09.543 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 8, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 97, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 29 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-04-25T10:00:09.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 32, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 3818, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 3818, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.545 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 35665, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 8480823, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 35665, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 35665, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8480823, 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T10:00:09.546 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8558, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 96668192, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8558, 2024-04-25T10:00:09.547 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 96668192, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T10:00:09.548 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 8558, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 8558, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 8558, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 8558, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.549 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 3818, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 400162, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 3818, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 46, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 400162, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.550 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55c2f99f26e0": { 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 28155, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 95577661, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 28155, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2502, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 95577661, 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-04-25T10:00:09.551 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-04-25T10:00:09.552 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-04-25T10:00:09.552 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-04-25T10:00:09.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.552 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:09.552 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f235406b4d0 msgr2=0x7f235406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f235406b4d0 0x7f235406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f23740b0d70 tx=0x7f235c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:09.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 msgr2=0x7f23740cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740cb690 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f236400c7f0 tx=0x7f236400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 shutdown_connections 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f235406b4d0 0x7f235406d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2374158230 0x7f23741e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23740d1b80 0x7f23741e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 --2- 172.21.15.148:0/2518500888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f237406c870 0x7f23740cb690 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 >> 172.21.15.148:0/2518500888 conn(0x7f2374061600 msgr2=0x7f23740ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 shutdown_connections 2024-04-25T10:00:09.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.490+0000 7f23797d5640 1 -- 172.21.15.148:0/2518500888 wait complete. 2024-04-25T10:00:09.555 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:00:09.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:09.611 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:09.653 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:00:09.653 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:09.653 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:09.668 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:09.669 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:09.724 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:09.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.857+0000 7fd29ace1640 1 Processor -- start 2024-04-25T10:00:09.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.857+0000 7fd29ace1640 1 -- start start 2024-04-25T10:00:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd29ace1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd29415ef00 0x7fd2940d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd29ace1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2940c7310 0x7fd2940cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd29ace1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2940d20e0 con 0x7fd2940d1ca0 2024-04-25T10:00:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd29ace1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2940c7730 con 0x7fd2940c7310 2024-04-25T10:00:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd29ace1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29415f2e0 con 0x7fd29415ef00 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd293fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd2940d1ca0 0x7fd2940c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36022/0 (socket says 172.21.15.148:36022) 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd293fff640 1 -- 172.21.15.148:0/853554612 learned_addr learned my addr 172.21.15.148:0/853554612 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd29927e640 1 --2- 172.21.15.148:0/853554612 >> v2:172.21.15.188:3300/0 conn(0x7fd2940c7310 0x7fd2940cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd298a7d640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd29415ef00 0x7fd2940d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986220975 0 0) 0x7fd2940d20e0 con 0x7fd2940d1ca0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.858+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd27c003520 con 0x7fd2940d1ca0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2050431054 0 0) 0x7fd27c003520 con 0x7fd2940d1ca0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd29415ef00 msgr2=0x7fd2940d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd29415ef00 0x7fd2940d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 >> v2:172.21.15.188:3300/0 conn(0x7fd2940c7310 msgr2=0x7fd2940cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 --2- 172.21.15.148:0/853554612 >> v2:172.21.15.188:3300/0 conn(0x7fd2940c7310 0x7fd2940cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940c7970 con 0x7fd2940d1ca0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd288001670 con 0x7fd2940d1ca0 2024-04-25T10:00:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 >> v1:172.21.15.148:6789/0 conn(0x7fd2940d1ca0 legacy=0x7fd2940c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c0039b0 0x7fd27c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c004800 0x7fd27c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0051c0 0x7fd27c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c7730 con 0x7fd27c0051c0 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29415f2e0 con 0x7fd27c0039b0 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00bf00 con 0x7fd27c004800 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd298a7d640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd29415ef00 0x7fd2940d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 --2- 172.21.15.148:0/853554612 >> v2:172.21.15.188:3300/0 conn(0x7fd2940c7310 0x7fd2940cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0051c0 0x7fd27c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd293fff640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c004800 0x7fd27c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd298a7d640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c0039b0 0x7fd27c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 -- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c004800 msgr2=0x7fd27c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c004800 0x7fd27c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 -- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c0039b0 msgr2=0x7fd27c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c0039b0 0x7fd27c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd29927e640 1 -- 172.21.15.148:0/853554612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c003520 con 0x7fd27c0051c0 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.859+0000 7fd298a7d640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd27c0039b0 0x7fd27c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.860+0000 7fd29927e640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0051c0 0x7fd27c00b9c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00e350 tx=0x7fd27c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed09505448cff98e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.860+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c009dc0 con 0x7fd27c0051c0 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.860+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c003b90 con 0x7fd27c0051c0 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.860+0000 7fd2937fe640 1 -- 172.21.15.148:0/853554612 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c004020 con 0x7fd27c0051c0 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29ace1640 1 -- 172.21.15.148:0/853554612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0051c0 msgr2=0x7fd27c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29ace1640 1 --2- 172.21.15.148:0/853554612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd27c0051c0 0x7fd27c00b9c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00e350 tx=0x7fd27c005830 comp rx=0 tx=0).stop 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29927e640 1 -- 172.21.15.148:0/853554612 reap_dead start 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29ace1640 1 -- 172.21.15.148:0/853554612 shutdown_connections 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29ace1640 1 -- 172.21.15.148:0/853554612 >> 172.21.15.148:0/853554612 conn(0x7fd29406f5d0 msgr2=0x7fd29406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29ace1640 1 -- 172.21.15.148:0/853554612 shutdown_connections 2024-04-25T10:00:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.861+0000 7fd29ace1640 1 -- 172.21.15.148:0/853554612 wait complete. 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.862+0000 7fd29ace1640 1 Processor -- start 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.862+0000 7fd29ace1640 1 -- start start 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29ace1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0051c0 0x7fd29414f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29ace1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd294151d90 0x7fd2941588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29ace1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 0x7fd294158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29ace1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c8210 con 0x7fd294152210 2024-04-25T10:00:09.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29ace1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c7f30 con 0x7fd294151d90 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29ace1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940c84f0 con 0x7fd27c0051c0 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 0x7fd294158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 0x7fd294158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54918/0 (socket says 172.21.15.148:54918) 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 -- 172.21.15.148:0/2218011445 learned_addr learned my addr 172.21.15.148:0/2218011445 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd298a7d640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0051c0 0x7fd29414f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd293fff640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd294151d90 0x7fd2941588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 -- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0051c0 msgr2=0x7fd29414f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0051c0 0x7fd29414f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 -- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd294151d90 msgr2=0x7fd2941588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd294151d90 0x7fd2941588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 -- 172.21.15.148:0/2218011445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2941527e0 con 0x7fd294152210 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd29927e640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 0x7fd294158e10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00e470 tx=0x7fd28c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.863+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c009dc0 con 0x7fd294152210 2024-04-25T10:00:09.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.864+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c02ee50 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.864+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2941f3af0 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.864+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c036610 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.864+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941f40b0 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.865+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd294073990 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.865+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28c0367b0 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.865+0000 7fd2917fa640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd28006b500 0x7fd28006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.866+0000 7fd298a7d640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd28006b500 0x7fd28006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.866+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fd28c0aae70 con 0x7fd294152210 2024-04-25T10:00:09.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.866+0000 7fd298a7d640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd28006b500 0x7fd28006d9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd2940d2470 tx=0x7fd284008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:09.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:09.870+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd28c032030 con 0x7fd294152210 2024-04-25T10:00:10.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.044+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd29415a040 con 0x7fd294152210 2024-04-25T10:00:10.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.045+0000 7fd2917fa640 1 -- 172.21.15.148:0/2218011445 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17059 (secure 0 0 0) 0x7fd28c079bd0 con 0x7fd294152210 2024-04-25T10:00:10.049 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:10.050 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:59:48.744514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd28006b500 msgr2=0x7fd28006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd28006b500 0x7fd28006d9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd2940d2470 tx=0x7fd284008040 comp rx=0 tx=0).stop 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 msgr2=0x7fd294158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 0x7fd294158e10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00e470 tx=0x7fd28c009750 comp rx=0 tx=0).stop 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 shutdown_connections 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd28006b500 0x7fd28006d9c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd294152210 0x7fd294158e10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd294151d90 0x7fd2941588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 --2- 172.21.15.148:0/2218011445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd27c0051c0 0x7fd29414f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 >> 172.21.15.148:0/2218011445 conn(0x7fd29406f5d0 msgr2=0x7fd2940ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 shutdown_connections 2024-04-25T10:00:10.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.047+0000 7fd29ace1640 1 -- 172.21.15.148:0/2218011445 wait complete. 2024-04-25T10:00:10.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:47.401474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:48.167626+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:10.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:10.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.159+0000 7fbf8eb02640 1 Processor -- start 2024-04-25T10:00:10.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.159+0000 7fbf8eb02640 1 -- start start 2024-04-25T10:00:10.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8eb02640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf880cc4f0 0x7fbf880cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8eb02640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf880cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8eb02640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf88072280 con 0x7fbf880cdb20 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8eb02640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf88071270 con 0x7fbf880cc4f0 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8eb02640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880714c0 con 0x7fbf8815b980 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf87fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbf880cdb20 0x7fbf8815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36034/0 (socket says 172.21.15.148:36034) 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf87fff640 1 -- 172.21.15.148:0/880844719 learned_addr learned my addr 172.21.15.148:0/880844719 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/880844719 >> v2:172.21.15.188:3300/0 conn(0x7fbf880cc4f0 0x7fbf880cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf880cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266393216 0 0) 0x7fbf88072280 con 0x7fbf880cdb20 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf70003520 con 0x7fbf880cdb20 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.160+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288956520 0 0) 0x7fbf70003520 con 0x7fbf880cdb20 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 msgr2=0x7fbf880cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf880cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 >> v2:172.21.15.188:3300/0 conn(0x7fbf880cc4f0 msgr2=0x7fbf880cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 --2- 172.21.15.148:0/880844719 >> v2:172.21.15.188:3300/0 conn(0x7fbf880cc4f0 0x7fbf880cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880717a0 con 0x7fbf880cdb20 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbf7c001690 con 0x7fbf880cdb20 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 >> v1:172.21.15.148:6789/0 conn(0x7fbf880cdb20 legacy=0x7fbf8815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf70004d40 0x7fbf700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88071270 con 0x7fbf70004d40 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880714c0 con 0x7fbf700039b0 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7000b9c0 con 0x7fbf8815b980 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/880844719 >> v2:172.21.15.188:3300/0 conn(0x7fbf880cc4f0 0x7fbf880cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf70004d40 0x7fbf700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf87fff640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8d09f640 1 -- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 msgr2=0x7fbf70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8d09f640 1 -- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf700039b0 msgr2=0x7fbf70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8d09f640 1 -- 172.21.15.148:0/880844719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70003520 con 0x7fbf70004d40 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.161+0000 7fbf87fff640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b980 0x7fbf70004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf70004d40 0x7fbf700051c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009fc0 tx=0x7fbf70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331cb39a7634b1e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80009c10 con 0x7fbf70004d40 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80036070 con 0x7fbf70004d40 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf877fe640 1 -- 172.21.15.148:0/880844719 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80031880 con 0x7fbf70004d40 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8eb02640 1 -- 172.21.15.148:0/880844719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf70004d40 msgr2=0x7fbf700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/880844719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf70004d40 0x7fbf700051c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009fc0 tx=0x7fbf70005830 comp rx=0 tx=0).stop 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8d09f640 1 -- 172.21.15.148:0/880844719 reap_dead start 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8eb02640 1 -- 172.21.15.148:0/880844719 shutdown_connections 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8eb02640 1 -- 172.21.15.148:0/880844719 >> 172.21.15.148:0/880844719 conn(0x7fbf8806abe0 msgr2=0x7fbf8806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8eb02640 1 -- 172.21.15.148:0/880844719 shutdown_connections 2024-04-25T10:00:10.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.162+0000 7fbf8eb02640 1 -- 172.21.15.148:0/880844719 wait complete. 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 Processor -- start 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 -- start start 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf70004d40 0x7fbf88153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf881515a0 0x7fbf88148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 0x7fbf88152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880cdfb0 con 0x7fbf88151fc0 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88071d60 con 0x7fbf70004d40 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.164+0000 7fbf8eb02640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880ce130 con 0x7fbf881515a0 2024-04-25T10:00:10.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 0x7fbf88152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 0x7fbf88152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54946/0 (socket says 172.21.15.148:54946) 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 -- 172.21.15.148:0/2561041470 learned_addr learned my addr 172.21.15.148:0/2561041470 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 -- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf881515a0 msgr2=0x7fbf88148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf881515a0 0x7fbf88148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf70004d40 0x7fbf88153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 -- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf70004d40 msgr2=0x7fbf88153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf70004d40 0x7fbf88153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 -- 172.21.15.148:0/2561041470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf88153980 con 0x7fbf88151fc0 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8d09f640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 0x7fbf88152450 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbf80033040 tx=0x7fbf80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80009c10 con 0x7fbf88151fc0 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80031db0 con 0x7fbf88151fc0 2024-04-25T10:00:10.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf88153c60 con 0x7fbf88151fc0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80003bd0 con 0x7fbf88151fc0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.165+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf88146fa0 con 0x7fbf88151fc0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.166+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf88153e30 con 0x7fbf88151fc0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.167+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf80005d10 con 0x7fbf88151fc0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.167+0000 7fbf857fa640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf7406b4d0 0x7fbf7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.167+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf7406b4d0 0x7fbf7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.167+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fbf800abaf0 con 0x7fbf88151fc0 2024-04-25T10:00:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.168+0000 7fbf8c89e640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf7406b4d0 0x7fbf7406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009a40 tx=0x7fbf78008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.171+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf8004a060 con 0x7fbf88151fc0 2024-04-25T10:00:10.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.340+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf88073a00 con 0x7fbf88151fc0 2024-04-25T10:00:10.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.341+0000 7fbf857fa640 1 -- 172.21.15.148:0/2561041470 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf8007a7c0 con 0x7fbf88151fc0 2024-04-25T10:00:10.345 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:00:10.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf7406b4d0 msgr2=0x7fbf7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf7406b4d0 0x7fbf7406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009a40 tx=0x7fbf78008040 comp rx=0 tx=0).stop 2024-04-25T10:00:10.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 msgr2=0x7fbf88152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 0x7fbf88152450 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbf80033040 tx=0x7fbf80009750 comp rx=0 tx=0).stop 2024-04-25T10:00:10.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 shutdown_connections 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbf7406b4d0 0x7fbf7406d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf88151fc0 0x7fbf88152450 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf881515a0 0x7fbf88148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 --2- 172.21.15.148:0/2561041470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf70004d40 0x7fbf88153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 >> 172.21.15.148:0/2561041470 conn(0x7fbf8806abe0 msgr2=0x7fbf8815b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 shutdown_connections 2024-04-25T10:00:10.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.343+0000 7fbf8eb02640 1 -- 172.21.15.148:0/2561041470 wait complete. 2024-04-25T10:00:10.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:10.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.454+0000 7f4fa5913640 1 Processor -- start 2024-04-25T10:00:10.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.455+0000 7f4fa5913640 1 -- start start 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.455+0000 7f4fa5913640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa00cadb0 0x7f4fa00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.455+0000 7f4fa5913640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4fa00ccfe0 0x7f4fa00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.455+0000 7f4fa5913640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa00afc80 con 0x7f4fa015de80 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.455+0000 7f4fa5913640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa00cb210 con 0x7f4fa00ccfe0 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.455+0000 7f4fa5913640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00ce4d0 con 0x7f4fa00cadb0 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4fa00ccfe0 0x7f4fa00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4fa00ccfe0 0x7f4fa00cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54952/0 (socket says 172.21.15.148:54952) 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 -- 172.21.15.148:0/3786895980 learned_addr learned my addr 172.21.15.148:0/3786895980 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:10.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9e7fc640 1 --2- 172.21.15.148:0/3786895980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa00cadb0 0x7f4fa00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 -- 172.21.15.148:0/3786895980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa00cadb0 msgr2=0x7f4fa00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 --2- 172.21.15.148:0/3786895980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa00cadb0 0x7f4fa00cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 -- 172.21.15.148:0/3786895980 >> v1:172.21.15.188:6789/0 conn(0x7f4fa015de80 legacy=0x7f4fa00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.456+0000 7f4f9f7fe640 1 -- 172.21.15.148:0/3786895980 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00ce7b0 con 0x7f4fa00ccfe0 2024-04-25T10:00:10.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4f9f7fe640 1 --2- 172.21.15.148:0/3786895980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00ccfe0 0x7f4fa00cd470 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009870 tx=0x7f4f9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f89718cf2e32e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4f9dffb640 1 -- 172.21.15.148:0/3786895980 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98037070 con 0x7f4fa00ccfe0 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4f9dffb640 1 -- 172.21.15.148:0/3786895980 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98032440 con 0x7f4fa00ccfe0 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4f9dffb640 1 -- 172.21.15.148:0/3786895980 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98032810 con 0x7f4fa00ccfe0 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 -- 172.21.15.148:0/3786895980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00ccfe0 msgr2=0x7f4fa00cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3786895980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00ccfe0 0x7f4fa00cd470 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009870 tx=0x7f4f9802a270 comp rx=0 tx=0).stop 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 -- 172.21.15.148:0/3786895980 shutdown_connections 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3786895980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00ccfe0 0x7f4fa00cd470 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3786895980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa00cadb0 0x7f4fa00cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 -- 172.21.15.148:0/3786895980 >> 172.21.15.148:0/3786895980 conn(0x7f4fa00b2390 msgr2=0x7f4fa00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 -- 172.21.15.148:0/3786895980 shutdown_connections 2024-04-25T10:00:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.457+0000 7f4fa5913640 1 -- 172.21.15.148:0/3786895980 wait complete. 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.458+0000 7f4fa5913640 1 Processor -- start 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 -- start start 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 0x7f4fa0152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fa00ccfe0 0x7f4fa00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa015de80 0x7f4fa00b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa015e260 con 0x7f4fa00cadb0 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00cfc00 con 0x7f4fa00ccfe0 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4fa5913640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00b2eb0 con 0x7f4fa015de80 2024-04-25T10:00:10.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4f9effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 0x7f4fa0152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4f9effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 0x7f4fa0152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54962/0 (socket says 172.21.15.148:54962) 2024-04-25T10:00:10.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.459+0000 7f4f9effd640 1 -- 172.21.15.148:0/3764316088 learned_addr learned my addr 172.21.15.148:0/3764316088 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:10.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f9effd640 1 -- 172.21.15.148:0/3764316088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa015de80 msgr2=0x7f4fa00b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f9effd640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa015de80 0x7f4fa00b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f9effd640 1 -- 172.21.15.148:0/3764316088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fa00ccfe0 msgr2=0x7f4fa00b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f9effd640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fa00ccfe0 0x7f4fa00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f9effd640 1 -- 172.21.15.148:0/3764316088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00b5470 con 0x7f4fa00cadb0 2024-04-25T10:00:10.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f9effd640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 0x7f4fa0152210 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000c730 tx=0x7f4f9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90016020 con 0x7f4fa00cadb0 2024-04-25T10:00:10.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90006d50 con 0x7f4fa00cadb0 2024-04-25T10:00:10.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa00b4170 con 0x7f4fa00cadb0 2024-04-25T10:00:10.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90003bb0 con 0x7f4fa00cadb0 2024-04-25T10:00:10.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.460+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa00b4730 con 0x7f4fa00cadb0 2024-04-25T10:00:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.461+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa0073a00 con 0x7f4fa00cadb0 2024-04-25T10:00:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.461+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f9000a850 con 0x7f4fa00cadb0 2024-04-25T10:00:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.462+0000 7f4f87fff640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.462+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f4f9008a000 con 0x7f4fa00cadb0 2024-04-25T10:00:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.462+0000 7f4f9e7fc640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.462+0000 7f4f9e7fc640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cbb50 tx=0x7f4f94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.466+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f90020020 con 0x7f4fa00cadb0 2024-04-25T10:00:10.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.638+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fa015d6a0 con 0x7f4fa00cadb0 2024-04-25T10:00:10.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.639+0000 7f4f87fff640 1 -- 172.21.15.148:0/3764316088 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f90019310 con 0x7f4fa00cadb0 2024-04-25T10:00:10.643 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f8c06b4d0 msgr2=0x7f4f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00cbb50 tx=0x7f4f94008040 comp rx=0 tx=0).stop 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 msgr2=0x7f4fa0152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 0x7f4fa0152210 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000c730 tx=0x7f4f9000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 shutdown_connections 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f8c06b4d0 0x7f4f8c06d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fa015de80 0x7f4fa00b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fa00ccfe0 0x7f4fa00b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 --2- 172.21.15.148:0/3764316088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4fa00cadb0 0x7f4fa0152210 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 >> 172.21.15.148:0/3764316088 conn(0x7f4fa00b2390 msgr2=0x7f4fa006e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 shutdown_connections 2024-04-25T10:00:10.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.641+0000 7f4fa5913640 1 -- 172.21.15.148:0/3764316088 wait complete. 2024-04-25T10:00:10.656 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:10.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.752+0000 7f487c142640 1 Processor -- start 2024-04-25T10:00:10.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.752+0000 7f487c142640 1 -- start start 2024-04-25T10:00:10.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f487c142640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48740cd8a0 0x7f487415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f487c142640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f4874151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f487c142640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487415e6e0 con 0x7f48740cd4c0 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f487c142640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4874151560 con 0x7f48740cd8a0 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f487c142640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487415e9c0 con 0x7f4874150b90 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48740cd8a0 0x7f487415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48740cd8a0 0x7f487415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54966/0 (socket says 172.21.15.148:54966) 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 -- 172.21.15.148:0/4112336964 learned_addr learned my addr 172.21.15.148:0/4112336964 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f487a6df640 1 --2- 172.21.15.148:0/4112336964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f4874151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 -- 172.21.15.148:0/4112336964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 msgr2=0x7f4874151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 --2- 172.21.15.148:0/4112336964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f4874151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 -- 172.21.15.148:0/4112336964 >> v1:172.21.15.188:6789/0 conn(0x7f48740cd4c0 legacy=0x7f487415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:10.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.753+0000 7f48796dd640 1 -- 172.21.15.148:0/4112336964 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487415a9c0 con 0x7f48740cd8a0 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.754+0000 7f48796dd640 1 --2- 172.21.15.148:0/4112336964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd8a0 0x7f487415b1c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4864009870 tx=0x7f486402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de7d8bde8ee934f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.754+0000 7f4878edc640 1 -- 172.21.15.148:0/4112336964 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864037070 con 0x7f48740cd8a0 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.754+0000 7f4878edc640 1 -- 172.21.15.148:0/4112336964 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864032440 con 0x7f48740cd8a0 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.754+0000 7f4878edc640 1 -- 172.21.15.148:0/4112336964 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864032810 con 0x7f48740cd8a0 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.754+0000 7f487c142640 1 -- 172.21.15.148:0/4112336964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd8a0 msgr2=0x7f487415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.754+0000 7f487c142640 1 --2- 172.21.15.148:0/4112336964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd8a0 0x7f487415b1c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f4864009870 tx=0x7f486402a210 comp rx=0 tx=0).stop 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.755+0000 7f487c142640 1 -- 172.21.15.148:0/4112336964 shutdown_connections 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.755+0000 7f487c142640 1 --2- 172.21.15.148:0/4112336964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd8a0 0x7f487415b1c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.755+0000 7f487c142640 1 --2- 172.21.15.148:0/4112336964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f4874151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.755+0000 7f487c142640 1 -- 172.21.15.148:0/4112336964 >> 172.21.15.148:0/4112336964 conn(0x7f487406abe0 msgr2=0x7f487406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.755+0000 7f487c142640 1 -- 172.21.15.148:0/4112336964 shutdown_connections 2024-04-25T10:00:10.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.755+0000 7f487c142640 1 -- 172.21.15.148:0/4112336964 wait complete. 2024-04-25T10:00:10.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.756+0000 7f487c142640 1 Processor -- start 2024-04-25T10:00:10.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.756+0000 7f487c142640 1 -- start start 2024-04-25T10:00:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 0x7f4874144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48740cd8a0 0x7f4874144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f487415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487415e370 con 0x7f48740cd4c0 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487415a670 con 0x7f48740cd8a0 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487406eca0 con 0x7f4874150b90 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 0x7f4874144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 0x7f4874144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54974/0 (socket says 172.21.15.148:54974) 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 -- 172.21.15.148:0/3367425318 learned_addr learned my addr 172.21.15.148:0/3367425318 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 -- 172.21.15.148:0/3367425318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 msgr2=0x7f487415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f487415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 -- 172.21.15.148:0/3367425318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48740cd8a0 msgr2=0x7f4874144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48740cd8a0 0x7f4874144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 -- 172.21.15.148:0/3367425318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740b1d20 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f4879ede640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 0x7f4874144410 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f487000c430 tx=0x7f487000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870011790 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870017070 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4874157830 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870011bb0 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.757+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4874157df0 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.758+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48740739e0 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.759+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f487001c460 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.759+0000 7f486a7fc640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f485806b4d0 0x7f485806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.759+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f487008a1d0 con 0x7f48740cd4c0 2024-04-25T10:00:10.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.759+0000 7f48796dd640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f485806b4d0 0x7f485806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:10.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.759+0000 7f48796dd640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f485806b4d0 0x7f485806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4864034000 tx=0x7f486402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:10.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.763+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48740b1d20 con 0x7f48740cd4c0 2024-04-25T10:00:10.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.934+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f487406b1d0 con 0x7f48740cd4c0 2024-04-25T10:00:10.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.935+0000 7f486a7fc640 1 -- 172.21.15.148:0/3367425318 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4870058f30 con 0x7f48740cd4c0 2024-04-25T10:00:10.939 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.936+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f485806b4d0 msgr2=0x7f485806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.936+0000 7f487c142640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f485806b4d0 0x7f485806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4864034000 tx=0x7f486402c000 comp rx=0 tx=0).stop 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.936+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 msgr2=0x7f4874144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.936+0000 7f487c142640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 0x7f4874144410 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f487000c430 tx=0x7f487000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 shutdown_connections 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f485806b4d0 0x7f485806d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4874150b90 0x7f487415cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48740cd8a0 0x7f4874144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 --2- 172.21.15.148:0/3367425318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48740cd4c0 0x7f4874144410 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:10.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 >> 172.21.15.148:0/3367425318 conn(0x7f487406abe0 msgr2=0x7f4874158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:10.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 shutdown_connections 2024-04-25T10:00:10.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:10.937+0000 7f487c142640 1 -- 172.21.15.148:0/3367425318 wait complete. 2024-04-25T10:00:10.952 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:00:11.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.046+0000 7f7f72bfe640 1 Processor -- start 2024-04-25T10:00:11.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.046+0000 7f7f72bfe640 1 -- start start 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.047+0000 7f7f72bfe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f6c15dbb0 0x7f7f6c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.047+0000 7f7f72bfe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.047+0000 7f7f72bfe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0cae60 con 0x7f7f6c0cc360 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.047+0000 7f7f72bfe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cb140 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.047+0000 7f7f72bfe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cb420 con 0x7f7f6c0cbf00 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f7099a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f6c15dbb0 0x7f7f6c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f7119b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7f6c0cc360 0x7f7f6c06db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53070/0 (socket says 172.21.15.148:53070) 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f7119b640 1 -- 172.21.15.148:0/3971362636 learned_addr learned my addr 172.21.15.148:0/3971362636 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f6bfff640 1 -- 172.21.15.148:0/3971362636 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c15dbb0 msgr2=0x7f7f6c06fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f6bfff640 1 --2- 172.21.15.148:0/3971362636 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c15dbb0 0x7f7f6c06fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f6bfff640 1 -- 172.21.15.148:0/3971362636 >> v1:172.21.15.188:6789/0 conn(0x7f7f6c0cc360 legacy=0x7f7f6c06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f6bfff640 1 -- 172.21.15.148:0/3971362636 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0cb700 con 0x7f7f6c0cbf00 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.048+0000 7f7f7099a640 1 --2- 172.21.15.148:0/3971362636 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c15dbb0 0x7f7f6c06fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f6bfff640 1 --2- 172.21.15.148:0/3971362636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c06bbf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097a0 tx=0x7f7f6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b6870b4bdea94c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f6b7fe640 1 -- 172.21.15.148:0/3971362636 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60037070 con 0x7f7f6c0cbf00 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f6b7fe640 1 -- 172.21.15.148:0/3971362636 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60032440 con 0x7f7f6c0cbf00 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f6b7fe640 1 -- 172.21.15.148:0/3971362636 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60032810 con 0x7f7f6c0cbf00 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 -- 172.21.15.148:0/3971362636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 msgr2=0x7f7f6c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/3971362636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c06bbf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097a0 tx=0x7f7f6002a180 comp rx=0 tx=0).stop 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 -- 172.21.15.148:0/3971362636 shutdown_connections 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/3971362636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c06bbf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/3971362636 >> v2:172.21.15.188:3300/0 conn(0x7f7f6c15dbb0 0x7f7f6c06fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 -- 172.21.15.148:0/3971362636 >> 172.21.15.148:0/3971362636 conn(0x7f7f6c071af0 msgr2=0x7f7f6c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 -- 172.21.15.148:0/3971362636 shutdown_connections 2024-04-25T10:00:11.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.049+0000 7f7f72bfe640 1 -- 172.21.15.148:0/3971362636 wait complete. 2024-04-25T10:00:11.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.050+0000 7f7f72bfe640 1 Processor -- start 2024-04-25T10:00:11.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 -- start start 2024-04-25T10:00:11.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f6c0cc360 0x7f7f6c0b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 0x7f7f6c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cab10 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c15df90 con 0x7f7f6c0cc360 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f72bfe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0703e0 con 0x7f7f6c0cbf00 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.051+0000 7f7f7119b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 0x7f7f6c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 0x7f7f6c0b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54984/0 (socket says 172.21.15.148:54984) 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 -- 172.21.15.148:0/1801436442 learned_addr learned my addr 172.21.15.148:0/1801436442 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f6bfff640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f6c0cc360 0x7f7f6c0b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7099a640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 -- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 msgr2=0x7f7f6c15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 -- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f6c0cc360 msgr2=0x7f7f6c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f6c0cc360 0x7f7f6c0b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 -- 172.21.15.148:0/1801436442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0b4e80 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f7119b640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 0x7f7f6c0b3e00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400ca60 tx=0x7f7f6400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64017070 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400abd0 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c0b3640 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64012400 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.052+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c0b5120 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.053+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c073a00 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.054+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f6401a070 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.054+0000 7f7f697fa640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f5806b4d0 0x7f7f5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.054+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f7f6408a110 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.054+0000 7f7f7099a640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f5806b4d0 0x7f7f5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.054+0000 7f7f7099a640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f5806b4d0 0x7f7f5806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c15b8f0 tx=0x7f7f5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.058+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f64058d40 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.240+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f6c061b50 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.241+0000 7f7f697fa640 1 -- 172.21.15.148:0/1801436442 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7f64058640 con 0x7f7f6c15dbb0 2024-04-25T10:00:11.245 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f5806b4d0 msgr2=0x7f7f5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f5806b4d0 0x7f7f5806d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c15b8f0 tx=0x7f7f5c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 msgr2=0x7f7f6c0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 0x7f7f6c0b3e00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400ca60 tx=0x7f7f6400cf20 comp rx=0 tx=0).stop 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 shutdown_connections 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7f5806b4d0 0x7f7f5806d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f6c15dbb0 0x7f7f6c0b3e00 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f6c0cc360 0x7f7f6c0b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 --2- 172.21.15.148:0/1801436442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f6c0cbf00 0x7f7f6c15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 >> 172.21.15.148:0/1801436442 conn(0x7f7f6c071af0 msgr2=0x7f7f6c15b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 shutdown_connections 2024-04-25T10:00:11.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.243+0000 7f7f72bfe640 1 -- 172.21.15.148:0/1801436442 wait complete. 2024-04-25T10:00:11.258 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:00:11.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:11.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:11.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:00:11.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:11.295 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:00:11.350 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:00:11.350 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:11.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:11.368 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:11.368 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:11.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:11.405 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:00:11.405 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:11.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:11.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.539+0000 7f986562a640 1 Processor -- start 2024-04-25T10:00:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.539+0000 7f986562a640 1 -- start start 2024-04-25T10:00:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f986562a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9860154ee0 0x7f98600cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f986562a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98600ce590 0x7f986006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f986562a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9860071480 con 0x7f98600cfcf0 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f986562a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9860072090 con 0x7f98600ce590 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f986562a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9860071230 con 0x7f9860154ee0 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98600ce590 0x7f986006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98600ce590 0x7f986006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54990/0 (socket says 172.21.15.148:54990) 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 -- 172.21.15.148:0/4157168536 learned_addr learned my addr 172.21.15.148:0/4157168536 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985effd640 1 --2- 172.21.15.148:0/4157168536 >> v2:172.21.15.188:3300/0 conn(0x7f9860154ee0 0x7f98600cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 -- 172.21.15.148:0/4157168536 >> v2:172.21.15.188:3300/0 conn(0x7f9860154ee0 msgr2=0x7f98600cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 --2- 172.21.15.148:0/4157168536 >> v2:172.21.15.188:3300/0 conn(0x7f9860154ee0 0x7f98600cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 -- 172.21.15.148:0/4157168536 >> v1:172.21.15.188:6789/0 conn(0x7f98600cfcf0 legacy=0x7f98600cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.540+0000 7f985f7fe640 1 -- 172.21.15.148:0/4157168536 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860071760 con 0x7f98600ce590 2024-04-25T10:00:11.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.541+0000 7f985f7fe640 1 --2- 172.21.15.148:0/4157168536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f986006d6f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f98600d02e0 tx=0x7f985802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5fde90429cb337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.541+0000 7f985e7fc640 1 -- 172.21.15.148:0/4157168536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858036070 con 0x7f98600ce590 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.541+0000 7f985e7fc640 1 -- 172.21.15.148:0/4157168536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858031600 con 0x7f98600ce590 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.541+0000 7f985e7fc640 1 -- 172.21.15.148:0/4157168536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580319d0 con 0x7f98600ce590 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 -- 172.21.15.148:0/4157168536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 msgr2=0x7f986006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 --2- 172.21.15.148:0/4157168536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f986006d6f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f98600d02e0 tx=0x7f985802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 -- 172.21.15.148:0/4157168536 shutdown_connections 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 --2- 172.21.15.148:0/4157168536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f986006d6f0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 --2- 172.21.15.148:0/4157168536 >> v2:172.21.15.188:3300/0 conn(0x7f9860154ee0 0x7f98600cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 -- 172.21.15.148:0/4157168536 >> 172.21.15.148:0/4157168536 conn(0x7f986006abe0 msgr2=0x7f986006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 -- 172.21.15.148:0/4157168536 shutdown_connections 2024-04-25T10:00:11.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.542+0000 7f986562a640 1 -- 172.21.15.148:0/4157168536 wait complete. 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 Processor -- start 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 -- start start 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f9860158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98600cfcf0 0x7f9860152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9860154ee0 0x7f98601534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601552c0 con 0x7f98600ce590 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860071d20 con 0x7f9860154ee0 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.543+0000 7f986562a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9860155440 con 0x7f98600cfcf0 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f9860158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f9860158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55004/0 (socket says 172.21.15.148:55004) 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 -- 172.21.15.148:0/1614212021 learned_addr learned my addr 172.21.15.148:0/1614212021 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:11.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985f7fe640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9860154ee0 0x7f98601534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f9857fff640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98600cfcf0 0x7f9860152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 -- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98600cfcf0 msgr2=0x7f9860152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98600cfcf0 0x7f9860152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 -- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9860154ee0 msgr2=0x7f98601534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9860154ee0 0x7f98601534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 -- 172.21.15.148:0/1614212021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98601589d0 con 0x7f98600ce590 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985f7fe640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9860154ee0 0x7f98601534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f985effd640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f9860158490 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f985000c7f0 tx=0x7f985000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850015070 con 0x7f98600ce590 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9850004220 con 0x7f98600ce590 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98600a8930 con 0x7f98600ce590 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850003610 con 0x7f98600ce590 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.544+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601e5760 con 0x7f98600ce590 2024-04-25T10:00:11.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.545+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9860073990 con 0x7f98600ce590 2024-04-25T10:00:11.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.546+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98500039c0 con 0x7f98600ce590 2024-04-25T10:00:11.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.546+0000 7f98577fe640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f984006b4d0 0x7f984006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.546+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f985008a5d0 con 0x7f98600ce590 2024-04-25T10:00:11.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.546+0000 7f9857fff640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f984006b4d0 0x7f984006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.546+0000 7f9857fff640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f984006b4d0 0x7f984006d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98600af460 tx=0x7f9848009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.550+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9850059330 con 0x7f98600ce590 2024-04-25T10:00:11.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.756+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f986006e4e0 con 0x7f98600ce590 2024-04-25T10:00:11.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.757+0000 7f98577fe640 1 -- 172.21.15.148:0/1614212021 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f9850058c30 con 0x7f98600ce590 2024-04-25T10:00:11.761 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:11.762 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:58:49.193553+0000","modified":"2024-04-25T09:58:55.879362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7284":{"gid":7284,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/82715169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":82715169},{"type":"v1","addr":"172.21.15.148:6837","nonce":82715169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:00:11.762 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 47 2024-04-25T10:00:11.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f984006b4d0 msgr2=0x7f984006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f984006b4d0 0x7f984006d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98600af460 tx=0x7f9848009040 comp rx=0 tx=0).stop 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 msgr2=0x7f9860158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f9860158490 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f985000c7f0 tx=0x7f985000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 shutdown_connections 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f984006b4d0 0x7f984006d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9860154ee0 0x7f98601534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98600cfcf0 0x7f9860152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 --2- 172.21.15.148:0/1614212021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98600ce590 0x7f9860158490 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 >> 172.21.15.148:0/1614212021 conn(0x7f986006abe0 msgr2=0x7f986015d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 shutdown_connections 2024-04-25T10:00:11.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.759+0000 7f986562a640 1 -- 172.21.15.148:0/1614212021 wait complete. 2024-04-25T10:00:11.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.868+0000 7f40bd390640 1 Processor -- start 2024-04-25T10:00:11.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.868+0000 7f40bd390640 1 -- start start 2024-04-25T10:00:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40bd390640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f40b80af970 0x7f40b80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40bd390640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b806abe0 con 0x7f40b80d0010 2024-04-25T10:00:11.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40bd390640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b80ccc50 con 0x7f40b815dce0 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40bd390640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b806b4f0 con 0x7f40b80af970 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40b67fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f40b80af970 0x7f40b80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40b77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f40b80d0010 0x7f40b80cd4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36048/0 (socket says 172.21.15.148:36048) 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40b77fe640 1 -- 172.21.15.148:0/3615853681 learned_addr learned my addr 172.21.15.148:0/3615853681 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.869+0000 7f40b67fc640 1 --2- 172.21.15.148:0/3615853681 >> v2:172.21.15.148:3300/0 conn(0x7f40b80af970 0x7f40b80ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55018/0 (socket says 172.21.15.148:55018) 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b67fc640 1 -- 172.21.15.148:0/3615853681 >> v1:172.21.15.188:6789/0 conn(0x7f40b815dce0 legacy=0x7f40b80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b67fc640 1 -- 172.21.15.148:0/3615853681 >> v1:172.21.15.148:6789/0 conn(0x7f40b80d0010 legacy=0x7f40b80cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b67fc640 1 -- 172.21.15.148:0/3615853681 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b806aea0 con 0x7f40b80af970 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b5ffb640 1 -- 172.21.15.148:0/3615853681 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380634728 0 0) 0x7f40b806abe0 con 0x7f40b80d0010 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b67fc640 1 --2- 172.21.15.148:0/3615853681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b80ccf70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0097a0 tx=0x7f40ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b11597a13cfa575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b5ffb640 1 -- 172.21.15.148:0/3615853681 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac037070 con 0x7f40b80af970 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b5ffb640 1 -- 172.21.15.148:0/3615853681 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac02ae50 con 0x7f40b80af970 2024-04-25T10:00:11.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.870+0000 7f40b5ffb640 1 -- 172.21.15.148:0/3615853681 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac032660 con 0x7f40b80af970 2024-04-25T10:00:11.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 -- 172.21.15.148:0/3615853681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 msgr2=0x7f40b80ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 --2- 172.21.15.148:0/3615853681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b80ccf70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0097a0 tx=0x7f40ac02a140 comp rx=0 tx=0).stop 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 -- 172.21.15.148:0/3615853681 shutdown_connections 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 --2- 172.21.15.148:0/3615853681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b80ccf70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 -- 172.21.15.148:0/3615853681 >> 172.21.15.148:0/3615853681 conn(0x7f40b80b2290 msgr2=0x7f40b80b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 -- 172.21.15.148:0/3615853681 shutdown_connections 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.871+0000 7f40bd390640 1 -- 172.21.15.148:0/3615853681 wait complete. 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.872+0000 7f40bd390640 1 Processor -- start 2024-04-25T10:00:11.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.872+0000 7f40bd390640 1 -- start start 2024-04-25T10:00:11.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40bd390640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40bd390640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b80d0010 0x7f40b80b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40bd390640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b815dce0 0x7f40b80b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40bd390640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b815e0c0 con 0x7f40b80af970 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40bd390640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b806b980 con 0x7f40b815dce0 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40bd390640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80c4ef0 con 0x7f40b80d0010 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b815aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55030/0 (socket says 172.21.15.148:55030) 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 -- 172.21.15.148:0/2808956003 learned_addr learned my addr 172.21.15.148:0/2808956003 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b67fc640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b80d0010 0x7f40b80b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b77fe640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b815dce0 0x7f40b80b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 -- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b80d0010 msgr2=0x7f40b80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b80d0010 0x7f40b80b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 -- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b815dce0 msgr2=0x7f40b80b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:11.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b815dce0 0x7f40b80b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b6ffd640 1 -- 172.21.15.148:0/2808956003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b81f7820 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b67fc640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b80d0010 0x7f40b80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.873+0000 7f40b77fe640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b815dce0 0x7f40b80b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.874+0000 7f40b6ffd640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b815aeb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c7f0 tx=0x7f40a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.874+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8016020 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.874+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a800aa50 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.874+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b81f7b00 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.874+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a800ae40 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.874+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b80b46e0 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.875+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b806cc80 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.875+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40a8019020 con 0x7f40b80af970 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.876+0000 7f409f7fe640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40a406b4d0 0x7f40a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.876+0000 7f40b67fc640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40a406b4d0 0x7f40a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:11.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.876+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f40a808a0d0 con 0x7f40b80af970 2024-04-25T10:00:11.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.876+0000 7f40b67fc640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40a406b4d0 0x7f40a406d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f40ac034000 tx=0x7f40ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:11.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:11.880+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40a8020070 con 0x7f40b80af970 2024-04-25T10:00:12.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.051+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b8073990 con 0x7f40b80af970 2024-04-25T10:00:12.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.053+0000 7f409f7fe640 1 -- 172.21.15.148:0/2808956003 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17059 (secure 0 0 0) 0x7f40a8058e30 con 0x7f40b80af970 2024-04-25T10:00:12.057 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:12.057 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:59:48.744514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:12.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.054+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40a406b4d0 msgr2=0x7f40a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.054+0000 7f40bd390640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40a406b4d0 0x7f40a406d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f40ac034000 tx=0x7f40ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:12.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 msgr2=0x7f40b815aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b815aeb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c7f0 tx=0x7f40a800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 shutdown_connections 2024-04-25T10:00:12.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40a406b4d0 0x7f40a406d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40b815dce0 0x7f40b80b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40b80d0010 0x7f40b80b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 --2- 172.21.15.148:0/2808956003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40b80af970 0x7f40b815aeb0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 >> 172.21.15.148:0/2808956003 conn(0x7f40b80b2290 msgr2=0x7f40b80ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 shutdown_connections 2024-04-25T10:00:12.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.055+0000 7f40bd390640 1 -- 172.21.15.148:0/2808956003 wait complete. 2024-04-25T10:00:12.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:58:47.401474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:58:48.167626+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:12.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.167+0000 7f6edefe0640 1 Processor -- start 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.167+0000 7f6edefe0640 1 -- start start 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edefe0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 0x7f6ed806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edefe0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ed80ac8b0 0x7f6ed80acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edefe0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80cfd80 con 0x7f6ed815b710 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edefe0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed806f190 con 0x7f6ed80ac8b0 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edefe0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed815e490 con 0x7f6ed80cf640 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edcd7c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6ed815b710 0x7f6ed815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36060/0 (socket says 172.21.15.148:36060) 2024-04-25T10:00:12.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1021620731 learned_addr learned my addr 172.21.15.148:0/1021620731 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6edd57d640 1 --2- 172.21.15.148:0/1021620731 >> v2:172.21.15.188:3300/0 conn(0x7f6ed80ac8b0 0x7f6ed80acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed7fff640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 0x7f6ed806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525369430 0 0) 0x7f6ed80cfd80 con 0x7f6ed815b710 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec8003520 con 0x7f6ed815b710 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941331913 0 0) 0x7f6ec8003520 con 0x7f6ed815b710 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 msgr2=0x7f6ed806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 0x7f6ed806ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 >> v2:172.21.15.188:3300/0 conn(0x7f6ed80ac8b0 msgr2=0x7f6ed80acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 --2- 172.21.15.148:0/1021620731 >> v2:172.21.15.188:3300/0 conn(0x7f6ed80ac8b0 0x7f6ed80acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80d0060 con 0x7f6ed815b710 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6ecc001670 con 0x7f6ed815b710 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 >> v1:172.21.15.148:6789/0 conn(0x7f6ed815b710 legacy=0x7f6ed815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.168+0000 7f6ed77fe640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6ed77fe640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ec8004800 0x7f6ec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6ed77fe640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 0x7f6ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806f190 con 0x7f6ec80039b0 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed815e490 con 0x7f6ec8004800 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec800b9c0 con 0x7f6ed80cf640 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6ed7fff640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ec8004800 0x7f6ec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edd57d640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 0x7f6ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 msgr2=0x7f6ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed80cf640 0x7f6ec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ec8004800 msgr2=0x7f6ec8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ec8004800 0x7f6ec8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1021620731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8003520 con 0x7f6ec80039b0 2024-04-25T10:00:12.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.169+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ec8003dd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c240 tx=0x7f6ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8800bdfb3170b29a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc02e070 con 0x7f6ec80039b0 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc005af0 con 0x7f6ec80039b0 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6ed77fe640 1 -- 172.21.15.148:0/1021620731 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc007d30 con 0x7f6ec80039b0 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edefe0640 1 -- 172.21.15.148:0/1021620731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 msgr2=0x7f6ec8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1021620731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ec8003dd0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c240 tx=0x7f6ec8005830 comp rx=0 tx=0).stop 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edd57d640 1 -- 172.21.15.148:0/1021620731 reap_dead start 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edefe0640 1 -- 172.21.15.148:0/1021620731 shutdown_connections 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edefe0640 1 -- 172.21.15.148:0/1021620731 >> 172.21.15.148:0/1021620731 conn(0x7f6ed806abe0 msgr2=0x7f6ed806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edefe0640 1 -- 172.21.15.148:0/1021620731 shutdown_connections 2024-04-25T10:00:12.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.170+0000 7f6edefe0640 1 -- 172.21.15.148:0/1021620731 wait complete. 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.171+0000 7f6edefe0640 1 Processor -- start 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.171+0000 7f6edefe0640 1 -- start start 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edefe0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ed8152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edefe0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed8152670 0x7f6ed8156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edefe0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ed8156f70 0x7f6ed8153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edefe0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed815baf0 con 0x7f6ec80039b0 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edefe0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80cfaa0 con 0x7f6ed8156f70 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edefe0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed815bc70 con 0x7f6ed8152670 2024-04-25T10:00:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edcd7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ed8152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edcd7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ed8152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55050/0 (socket says 172.21.15.148:55050) 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.172+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1686872427 learned_addr learned my addr 172.21.15.148:0/1686872427 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6ed7fff640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed8152670 0x7f6ed8156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edd57d640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ed8156f70 0x7f6ed8153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed8152670 msgr2=0x7f6ed8156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed8152670 0x7f6ed8156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ed8156f70 msgr2=0x7f6ed8153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ed8156f70 0x7f6ed8153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edcd7c640 1 -- 172.21.15.148:0/1686872427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8157760 con 0x7f6ec80039b0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edcd7c640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ed8152130 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c360 tx=0x7f6ed80d0230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc042070 con 0x7f6ec80039b0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc007480 con 0x7f6ec80039b0 2024-04-25T10:00:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed8157a40 con 0x7f6ec80039b0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc001c70 con 0x7f6ec80039b0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.173+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed8157420 con 0x7f6ec80039b0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.174+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8073a00 con 0x7f6ec80039b0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.174+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ecc045070 con 0x7f6ec80039b0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.175+0000 7f6ed57fa640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ebc06b4d0 0x7f6ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.175+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6ecc0aaf40 con 0x7f6ec80039b0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.175+0000 7f6ed7fff640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ebc06b4d0 0x7f6ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.175+0000 7f6ed7fff640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ebc06b4d0 0x7f6ebc06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8152a50 tx=0x7f6ec4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.179+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ecc005ac0 con 0x7f6ec80039b0 2024-04-25T10:00:12.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.350+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ed806c250 con 0x7f6ec80039b0 2024-04-25T10:00:12.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.350+0000 7f6ed57fa640 1 -- 172.21.15.148:0/1686872427 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ecc079d30 con 0x7f6ec80039b0 2024-04-25T10:00:12.354 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:00:12.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ebc06b4d0 msgr2=0x7f6ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ebc06b4d0 0x7f6ebc06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8152a50 tx=0x7f6ec4008040 comp rx=0 tx=0).stop 2024-04-25T10:00:12.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 msgr2=0x7f6ed8152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ed8152130 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c360 tx=0x7f6ed80d0230 comp rx=0 tx=0).stop 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 shutdown_connections 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ebc06b4d0 0x7f6ebc06d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ed8156f70 0x7f6ed8153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ed8152670 0x7f6ed8156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 --2- 172.21.15.148:0/1686872427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ec80039b0 0x7f6ed8152130 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 >> 172.21.15.148:0/1686872427 conn(0x7f6ed806abe0 msgr2=0x7f6ed815cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 shutdown_connections 2024-04-25T10:00:12.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.352+0000 7f6edefe0640 1 -- 172.21.15.148:0/1686872427 wait complete. 2024-04-25T10:00:12.367 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:12.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.464+0000 7f9fbd010640 1 Processor -- start 2024-04-25T10:00:12.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.464+0000 7f9fbd010640 1 -- start start 2024-04-25T10:00:12.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fbd010640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb815eec0 0x7f9fb81689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fbd010640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb80d1770 con 0x7f9fb806ccd0 2024-04-25T10:00:12.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fbd010640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb8162740 con 0x7f9fb8070dc0 2024-04-25T10:00:12.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fbd010640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80b0eb0 con 0x7f9fb815eec0 2024-04-25T10:00:12.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9faffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9fb806ccd0 0x7f9fb80b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36064/0 (socket says 172.21.15.148:36064) 2024-04-25T10:00:12.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9faffff640 1 -- 172.21.15.148:0/2660139910 learned_addr learned my addr 172.21.15.148:0/2660139910 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb759e640 1 --2- 172.21.15.148:0/2660139910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb815eec0 0x7f9fb81689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569781960 0 0) 0x7f9fb80d1770 con 0x7f9fb806ccd0 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4003520 con 0x7f9fb806ccd0 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124474889 0 0) 0x7f9fb8162740 con 0x7f9fb8070dc0 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb80d1770 con 0x7f9fb8070dc0 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 82424790 0 0) 0x7f9fa4003520 con 0x7f9fb806ccd0 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb759e640 1 -- 172.21.15.148:0/2660139910 >> v1:172.21.15.188:6789/0 conn(0x7f9fb8070dc0 legacy=0x7f9fb80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb759e640 1 -- 172.21.15.148:0/2660139910 >> v1:172.21.15.148:6789/0 conn(0x7f9fb806ccd0 legacy=0x7f9fb80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.465+0000 7f9fb759e640 1 -- 172.21.15.148:0/2660139910 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb8161980 con 0x7f9fb815eec0 2024-04-25T10:00:12.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.466+0000 7f9fb759e640 1 --2- 172.21.15.148:0/2660139910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb815eec0 0x7f9fb81689d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009dc0 tx=0x7f9fb002a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9997d54719dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.466+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0001670 con 0x7f9fb815eec0 2024-04-25T10:00:12.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.466+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb0007540 con 0x7f9fb815eec0 2024-04-25T10:00:12.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.466+0000 7f9fb659c640 1 -- 172.21.15.148:0/2660139910 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0007910 con 0x7f9fb815eec0 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 -- 172.21.15.148:0/2660139910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb815eec0 msgr2=0x7f9fb81689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 --2- 172.21.15.148:0/2660139910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb815eec0 0x7f9fb81689d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009dc0 tx=0x7f9fb002a540 comp rx=0 tx=0).stop 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 -- 172.21.15.148:0/2660139910 shutdown_connections 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 --2- 172.21.15.148:0/2660139910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb815eec0 0x7f9fb81689d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 -- 172.21.15.148:0/2660139910 >> 172.21.15.148:0/2660139910 conn(0x7f9fb80b3530 msgr2=0x7f9fb80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 -- 172.21.15.148:0/2660139910 shutdown_connections 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.467+0000 7f9fbd010640 1 -- 172.21.15.148:0/2660139910 wait complete. 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.468+0000 7f9fbd010640 1 Processor -- start 2024-04-25T10:00:12.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.468+0000 7f9fbd010640 1 -- start start 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 0x7f9fb814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb8070dc0 0x7f9fb80b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fb815eec0 0x7f9fb80b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80b10f0 con 0x7f9fb806ccd0 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81622b0 con 0x7f9fb815eec0 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb8161f40 con 0x7f9fb8070dc0 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 0x7f9fb814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 0x7f9fb814ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55058/0 (socket says 172.21.15.148:55058) 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 -- 172.21.15.148:0/575968110 learned_addr learned my addr 172.21.15.148:0/575968110 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9faffff640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb8070dc0 0x7f9fb80b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb759e640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fb815eec0 0x7f9fb80b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 -- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb8070dc0 msgr2=0x7f9fb80b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb8070dc0 0x7f9fb80b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 -- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fb815eec0 msgr2=0x7f9fb80b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fb815eec0 0x7f9fb80b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 -- 172.21.15.148:0/575968110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb80cb8e0 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fb6d9d640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 0x7f9fb814ec90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c4c0 tx=0x7f9fa800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80052d0 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8005470 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8012870 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.469+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb80cbbc0 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.470+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb80cc180 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.470+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb8073a00 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.471+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fa8012a10 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.471+0000 7f9faf7fe640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f9806b4d0 0x7f9f9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.471+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f9fa808a010 con 0x7f9fb806ccd0 2024-04-25T10:00:12.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.471+0000 7f9faffff640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f9806b4d0 0x7f9f9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.472+0000 7f9faffff640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f9806b4d0 0x7f9f9806d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9fb81f6870 tx=0x7f9fa0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.475+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fa8058d70 con 0x7f9fb806ccd0 2024-04-25T10:00:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.646+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fb80b4730 con 0x7f9fb806ccd0 2024-04-25T10:00:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.647+0000 7f9faf7fe640 1 -- 172.21.15.148:0/575968110 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fa8058670 con 0x7f9fb806ccd0 2024-04-25T10:00:12.651 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:00:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f9806b4d0 msgr2=0x7f9f9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f9806b4d0 0x7f9f9806d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9fb81f6870 tx=0x7f9fa0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 msgr2=0x7f9fb814ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 0x7f9fb814ec90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9fa800c4c0 tx=0x7f9fa800c980 comp rx=0 tx=0).stop 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 shutdown_connections 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f9806b4d0 0x7f9f9806d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fb815eec0 0x7f9fb80b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fb8070dc0 0x7f9fb80b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 --2- 172.21.15.148:0/575968110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fb806ccd0 0x7f9fb814ec90 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 >> 172.21.15.148:0/575968110 conn(0x7f9fb80b3530 msgr2=0x7f9fb806ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 shutdown_connections 2024-04-25T10:00:12.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.649+0000 7f9fbd010640 1 -- 172.21.15.148:0/575968110 wait complete. 2024-04-25T10:00:12.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.759+0000 7fb684c8f640 1 Processor -- start 2024-04-25T10:00:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.759+0000 7fb684c8f640 1 -- start start 2024-04-25T10:00:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.760+0000 7fb684c8f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb68015bb20 0x7fb6800cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.760+0000 7fb684c8f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6800d00e0 0x7fb6800ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.760+0000 7fb684c8f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6800cb190 con 0x7fb6800ccdd0 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.760+0000 7fb684c8f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800cb470 con 0x7fb68015bb20 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.760+0000 7fb684c8f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800cb750 con 0x7fb6800d00e0 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb68015bb20 0x7fb6800cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb68015bb20 0x7fb6800cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55066/0 (socket says 172.21.15.148:55066) 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 -- 172.21.15.148:0/2251248401 learned_addr learned my addr 172.21.15.148:0/2251248401 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67ed9d640 1 --2- 172.21.15.148:0/2251248401 >> v2:172.21.15.188:3300/0 conn(0x7fb6800d00e0 0x7fb6800ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 -- 172.21.15.148:0/2251248401 >> v2:172.21.15.188:3300/0 conn(0x7fb6800d00e0 msgr2=0x7fb6800ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 --2- 172.21.15.148:0/2251248401 >> v2:172.21.15.188:3300/0 conn(0x7fb6800d00e0 0x7fb6800ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 -- 172.21.15.148:0/2251248401 >> v1:172.21.15.188:6789/0 conn(0x7fb6800ccdd0 legacy=0x7fb6800cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:12.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.761+0000 7fb67e59c640 1 -- 172.21.15.148:0/2251248401 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800cba30 con 0x7fb68015bb20 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb67e59c640 1 --2- 172.21.15.148:0/2251248401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb6800cc230 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb6800af5d0 tx=0x7fb67802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b11565ad233f522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb67d59a640 1 -- 172.21.15.148:0/2251248401 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678037070 con 0x7fb68015bb20 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb67d59a640 1 -- 172.21.15.148:0/2251248401 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678032440 con 0x7fb68015bb20 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb67d59a640 1 -- 172.21.15.148:0/2251248401 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678032810 con 0x7fb68015bb20 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 -- 172.21.15.148:0/2251248401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 msgr2=0x7fb6800cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 --2- 172.21.15.148:0/2251248401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb6800cc230 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb6800af5d0 tx=0x7fb67802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 -- 172.21.15.148:0/2251248401 shutdown_connections 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 --2- 172.21.15.148:0/2251248401 >> v2:172.21.15.188:3300/0 conn(0x7fb6800d00e0 0x7fb6800ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 --2- 172.21.15.148:0/2251248401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb6800cc230 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 -- 172.21.15.148:0/2251248401 >> 172.21.15.148:0/2251248401 conn(0x7fb6800b2390 msgr2=0x7fb6800b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 -- 172.21.15.148:0/2251248401 shutdown_connections 2024-04-25T10:00:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.762+0000 7fb684c8f640 1 -- 172.21.15.148:0/2251248401 wait complete. 2024-04-25T10:00:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 Processor -- start 2024-04-25T10:00:12.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 -- start start 2024-04-25T10:00:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6800ccdd0 0x7fb68014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6800d00e0 0x7fb68015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb68015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015bf00 con 0x7fb68015bb20 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800cc770 con 0x7fb6800d00e0 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.764+0000 7fb684c8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015c080 con 0x7fb6800ccdd0 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb68015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb68015a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55078/0 (socket says 172.21.15.148:55078) 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 -- 172.21.15.148:0/3717362108 learned_addr learned my addr 172.21.15.148:0/3717362108 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67dd9b640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6800d00e0 0x7fb68015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67e59c640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6800ccdd0 0x7fb68014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 -- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6800ccdd0 msgr2=0x7fb68014aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6800ccdd0 0x7fb68014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 -- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6800d00e0 msgr2=0x7fb68015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6800d00e0 0x7fb68015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 -- 172.21.15.148:0/3717362108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68014b910 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67dd9b640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6800d00e0 0x7fb68015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb67ed9d640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb68015a0a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d800 tx=0x7fb67000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000b6c0 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb670019070 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800c8b20 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000bac0 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.765+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6801598a0 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.766+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6800c9040 con 0x7fb68015bb20 2024-04-25T10:00:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.767+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb67000bc60 con 0x7fb68015bb20 2024-04-25T10:00:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.767+0000 7fb66effd640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb66006b4d0 0x7fb66006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.767+0000 7fb67e59c640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb66006b4d0 0x7fb66006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.767+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb67008a890 con 0x7fb68015bb20 2024-04-25T10:00:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.768+0000 7fb67e59c640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb66006b4d0 0x7fb66006d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb678034000 tx=0x7fb67802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:12.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.771+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6800c9040 con 0x7fb68015bb20 2024-04-25T10:00:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.939+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb680073a00 con 0x7fb68015bb20 2024-04-25T10:00:12.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.940+0000 7fb66effd640 1 -- 172.21.15.148:0/3717362108 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb68014b910 con 0x7fb68015bb20 2024-04-25T10:00:12.944 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:00:12.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb66006b4d0 msgr2=0x7fb66006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb66006b4d0 0x7fb66006d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb678034000 tx=0x7fb67802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:12.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 msgr2=0x7fb68015a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:12.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb68015a0a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d800 tx=0x7fb67000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 shutdown_connections 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb66006b4d0 0x7fb66006d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68015bb20 0x7fb68015a0a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6800d00e0 0x7fb68015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 --2- 172.21.15.148:0/3717362108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6800ccdd0 0x7fb68014aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 >> 172.21.15.148:0/3717362108 conn(0x7fb6800b2390 msgr2=0x7fb680151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 shutdown_connections 2024-04-25T10:00:12.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:12.942+0000 7fb684c8f640 1 -- 172.21.15.148:0/3717362108 wait complete. 2024-04-25T10:00:12.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:13.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.050+0000 7f68f057e640 1 Processor -- start 2024-04-25T10:00:13.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.050+0000 7f68f057e640 1 -- start start 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.051+0000 7f68f057e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68e80b0b90 0x7f68e80cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.051+0000 7f68f057e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 0x7f68e8099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.051+0000 7f68f057e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80cc3c0 con 0x7f68e815f020 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.051+0000 7f68f057e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e80cccb0 con 0x7f68e80b0b90 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.051+0000 7f68f057e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815f400 con 0x7f68e8099a40 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68eeb1b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 0x7f68e8099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68eeb1b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 0x7f68e8099ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:45426/0 (socket says 172.21.15.148:45426) 2024-04-25T10:00:13.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68ee31a640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f68e815f020 0x7f68e80b10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53116/0 (socket says 172.21.15.148:53116) 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68ee31a640 1 -- 172.21.15.148:0/3023954170 learned_addr learned my addr 172.21.15.148:0/3023954170 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68edb19640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68e80b0b90 0x7f68e80cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68ed318640 1 -- 172.21.15.148:0/3023954170 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682827182 0 0) 0x7f68e80cc3c0 con 0x7f68e815f020 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68ed318640 1 -- 172.21.15.148:0/3023954170 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d4003520 con 0x7f68e815f020 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68eeb1b640 1 -- 172.21.15.148:0/3023954170 >> v2:172.21.15.188:3300/0 conn(0x7f68e80b0b90 msgr2=0x7f68e80cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68eeb1b640 1 --2- 172.21.15.148:0/3023954170 >> v2:172.21.15.188:3300/0 conn(0x7f68e80b0b90 0x7f68e80cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68eeb1b640 1 -- 172.21.15.148:0/3023954170 >> v1:172.21.15.188:6789/0 conn(0x7f68e815f020 legacy=0x7f68e80b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68eeb1b640 1 -- 172.21.15.148:0/3023954170 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80d0550 con 0x7f68e8099a40 2024-04-25T10:00:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.052+0000 7f68edb19640 1 --2- 172.21.15.148:0/3023954170 >> v2:172.21.15.188:3300/0 conn(0x7f68e80b0b90 0x7f68e80cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:13.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.053+0000 7f68eeb1b640 1 --2- 172.21.15.148:0/3023954170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 0x7f68e8099ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f68e00097a0 tx=0x7f68e002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b942375d6f7ba4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.053+0000 7f68ed318640 1 -- 172.21.15.148:0/3023954170 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0037070 con 0x7f68e8099a40 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.053+0000 7f68ed318640 1 -- 172.21.15.148:0/3023954170 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e0032440 con 0x7f68e8099a40 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.053+0000 7f68ed318640 1 -- 172.21.15.148:0/3023954170 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0032810 con 0x7f68e8099a40 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 -- 172.21.15.148:0/3023954170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 msgr2=0x7f68e8099ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 --2- 172.21.15.148:0/3023954170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 0x7f68e8099ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f68e00097a0 tx=0x7f68e002a180 comp rx=0 tx=0).stop 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 -- 172.21.15.148:0/3023954170 shutdown_connections 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 --2- 172.21.15.148:0/3023954170 >> v2:172.21.15.188:3300/0 conn(0x7f68e80b0b90 0x7f68e80cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 --2- 172.21.15.148:0/3023954170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e8099a40 0x7f68e8099ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 -- 172.21.15.148:0/3023954170 >> 172.21.15.148:0/3023954170 conn(0x7f68e80b3310 msgr2=0x7f68e80b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 -- 172.21.15.148:0/3023954170 shutdown_connections 2024-04-25T10:00:13.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.054+0000 7f68f057e640 1 -- 172.21.15.148:0/3023954170 wait complete. 2024-04-25T10:00:13.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.055+0000 7f68f057e640 1 Processor -- start 2024-04-25T10:00:13.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.055+0000 7f68f057e640 1 -- start start 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 0x7f68e8151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68e80b0b90 0x7f68e815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e815f020 0x7f68e80c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80cc940 con 0x7f68e8099a40 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80cc660 con 0x7f68e80b0b90 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80cd140 con 0x7f68e815f020 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 0x7f68e8151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 0x7f68e8151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55092/0 (socket says 172.21.15.148:55092) 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 -- 172.21.15.148:0/1367146133 learned_addr learned my addr 172.21.15.148:0/1367146133 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68eeb1b640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e815f020 0x7f68e80c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68edb19640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68e80b0b90 0x7f68e815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 -- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e815f020 msgr2=0x7f68e80c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e815f020 0x7f68e80c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 -- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68e80b0b90 msgr2=0x7f68e815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68e80b0b90 0x7f68e815ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 -- 172.21.15.148:0/1367146133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80c7fe0 con 0x7f68e8099a40 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68edb19640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68e80b0b90 0x7f68e815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68ee31a640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 0x7f68e8151b20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f68e400cad0 tx=0x7f68e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4017070 con 0x7f68e8099a40 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400ac30 con 0x7f68e8099a40 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e80aebf0 con 0x7f68e8099a40 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4012400 con 0x7f68e8099a40 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.056+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e80af1b0 con 0x7f68e8099a40 2024-04-25T10:00:13.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.057+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e8073990 con 0x7f68e8099a40 2024-04-25T10:00:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.058+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68e401a070 con 0x7f68e8099a40 2024-04-25T10:00:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.058+0000 7f68deffd640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68cc06b500 0x7f68cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.058+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f68e408a090 con 0x7f68e8099a40 2024-04-25T10:00:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.058+0000 7f68edb19640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68cc06b500 0x7f68cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.059+0000 7f68edb19640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68cc06b500 0x7f68cc06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f68e80705d0 tx=0x7f68d8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.062+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68e4058df0 con 0x7f68e8099a40 2024-04-25T10:00:13.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.268+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68e80b77a0 con 0x7f68e8099a40 2024-04-25T10:00:13.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.269+0000 7f68deffd640 1 -- 172.21.15.148:0/1367146133 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f68e40586f0 con 0x7f68e8099a40 2024-04-25T10:00:13.273 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:13.273 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:58:49.193553+0000","modified":"2024-04-25T09:58:55.879362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7284":{"gid":7284,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/82715169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":82715169},{"type":"v1","addr":"172.21.15.148:6837","nonce":82715169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:00:13.274 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 47 2024-04-25T10:00:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68cc06b500 msgr2=0x7f68cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68cc06b500 0x7f68cc06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f68e80705d0 tx=0x7f68d8008040 comp rx=0 tx=0).stop 2024-04-25T10:00:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 msgr2=0x7f68e8151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 0x7f68e8151b20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f68e400cad0 tx=0x7f68e400cf90 comp rx=0 tx=0).stop 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 shutdown_connections 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f68cc06b500 0x7f68cc06d9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68e815f020 0x7f68e80c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68e80b0b90 0x7f68e815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 --2- 172.21.15.148:0/1367146133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68e8099a40 0x7f68e8151b20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 >> 172.21.15.148:0/1367146133 conn(0x7f68e80b3310 msgr2=0x7f68e815feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 shutdown_connections 2024-04-25T10:00:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.271+0000 7f68f057e640 1 -- 172.21.15.148:0/1367146133 wait complete. 2024-04-25T10:00:13.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:13.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.379+0000 7fe0b3d53640 1 Processor -- start 2024-04-25T10:00:13.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.379+0000 7fe0b3d53640 1 -- start start 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b3d53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0ac0af720 0x7fe0ac0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b3d53640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac06fe00 con 0x7fe0ac0d02f0 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b3d53640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac0700e0 con 0x7fe0ac158220 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b3d53640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ac0b3670 con 0x7fe0ac0af720 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b12ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0ac0af720 0x7fe0ac0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b12ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0ac0af720 0x7fe0ac0afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55102/0 (socket says 172.21.15.148:55102) 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.380+0000 7fe0b12ee640 1 -- 172.21.15.148:0/104636650 learned_addr learned my addr 172.21.15.148:0/104636650 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.381+0000 7fe0b12ee640 1 -- 172.21.15.148:0/104636650 >> v1:172.21.15.188:6789/0 conn(0x7fe0ac158220 legacy=0x7fe0ac15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.381+0000 7fe0b12ee640 1 -- 172.21.15.148:0/104636650 >> v1:172.21.15.148:6789/0 conn(0x7fe0ac0d02f0 legacy=0x7fe0ac0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.381+0000 7fe0b12ee640 1 -- 172.21.15.148:0/104636650 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0703c0 con 0x7fe0ac0af720 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.381+0000 7fe0b12ee640 1 --2- 172.21.15.148:0/104636650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac0af720 0x7fe0ac0afba0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009870 tx=0x7fe09c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e255bb054ba4048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.381+0000 7fe0b0aed640 1 -- 172.21.15.148:0/104636650 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c037070 con 0x7fe0ac0af720 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b0aed640 1 -- 172.21.15.148:0/104636650 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c02ae50 con 0x7fe0ac0af720 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b0aed640 1 -- 172.21.15.148:0/104636650 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c032860 con 0x7fe0ac0af720 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 -- 172.21.15.148:0/104636650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac0af720 msgr2=0x7fe0ac0afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/104636650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac0af720 0x7fe0ac0afba0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009870 tx=0x7fe09c02a300 comp rx=0 tx=0).stop 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 -- 172.21.15.148:0/104636650 shutdown_connections 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/104636650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac0af720 0x7fe0ac0afba0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 -- 172.21.15.148:0/104636650 >> 172.21.15.148:0/104636650 conn(0x7fe0ac0b2370 msgr2=0x7fe0ac0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 -- 172.21.15.148:0/104636650 shutdown_connections 2024-04-25T10:00:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.382+0000 7fe0b3d53640 1 -- 172.21.15.148:0/104636650 wait complete. 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.383+0000 7fe0b3d53640 1 Processor -- start 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.383+0000 7fe0b3d53640 1 -- start start 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b3d53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ac0af720 0x7fe0ac15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b3d53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0ac0d02f0 0x7fe0ac15e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b3d53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 0x7fe0ac0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b3d53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac158600 con 0x7fe0ac158220 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b3d53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac070a10 con 0x7fe0ac0af720 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b3d53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac072960 con 0x7fe0ac0d02f0 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 0x7fe0ac0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 0x7fe0ac0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55104/0 (socket says 172.21.15.148:55104) 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 -- 172.21.15.148:0/4052365179 learned_addr learned my addr 172.21.15.148:0/4052365179 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b12ee640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0ac0d02f0 0x7fe0ac15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b1aef640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ac0af720 0x7fe0ac15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 -- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0ac0d02f0 msgr2=0x7fe0ac15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0ac0d02f0 0x7fe0ac15e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 -- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ac0af720 msgr2=0x7fe0ac15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ac0af720 0x7fe0ac15b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 -- 172.21.15.148:0/4052365179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0cc830 con 0x7fe0ac158220 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b1aef640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ac0af720 0x7fe0ac15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b12ee640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0ac0d02f0 0x7fe0ac15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.384+0000 7fe0b22f0640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 0x7fe0ac0cc2f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4001eb0 tx=0x7fe0a400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.385+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4017070 con 0x7fe0ac158220 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.385+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a40052d0 con 0x7fe0ac158220 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.385+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac1598c0 con 0x7fe0ac158220 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.385+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4003810 con 0x7fe0ac158220 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.385+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac159e80 con 0x7fe0ac158220 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.386+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac073990 con 0x7fe0ac158220 2024-04-25T10:00:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.386+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0a401a070 con 0x7fe0ac158220 2024-04-25T10:00:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.386+0000 7fe0a27fc640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe09006b4d0 0x7fe09006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.387+0000 7fe0b1aef640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe09006b4d0 0x7fe09006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.387+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe0a40896a0 con 0x7fe0ac158220 2024-04-25T10:00:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.387+0000 7fe0b1aef640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe09006b4d0 0x7fe09006d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8001e00 tx=0x7fe0a8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.391+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0a4058400 con 0x7fe0ac158220 2024-04-25T10:00:13.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.562+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0ac15d020 con 0x7fe0ac158220 2024-04-25T10:00:13.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.563+0000 7fe0a27fc640 1 -- 172.21.15.148:0/4052365179 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17059 (secure 0 0 0) 0x7fe0a4057d00 con 0x7fe0ac158220 2024-04-25T10:00:13.567 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:13.568 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:59:48.744514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe09006b4d0 msgr2=0x7fe09006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe09006b4d0 0x7fe09006d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8001e00 tx=0x7fe0a8009040 comp rx=0 tx=0).stop 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 msgr2=0x7fe0ac0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 0x7fe0ac0cc2f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4001eb0 tx=0x7fe0a400c890 comp rx=0 tx=0).stop 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 shutdown_connections 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe09006b4d0 0x7fe09006d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0ac158220 0x7fe0ac0cc2f0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0ac0d02f0 0x7fe0ac15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 --2- 172.21.15.148:0/4052365179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0ac0af720 0x7fe0ac15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 >> 172.21.15.148:0/4052365179 conn(0x7fe0ac0b2370 msgr2=0x7fe0ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 shutdown_connections 2024-04-25T10:00:13.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.565+0000 7fe0b3d53640 1 -- 172.21.15.148:0/4052365179 wait complete. 2024-04-25T10:00:13.580 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:00:13.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:13.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.672+0000 7fca0f563640 1 Processor -- start 2024-04-25T10:00:13.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.673+0000 7fca0f563640 1 -- start start 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.673+0000 7fca0f563640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca080b2cb0 0x7fca080d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.673+0000 7fca0f563640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca080b97a0 con 0x7fca0815dbe0 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.673+0000 7fca0f563640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca080d1c50 con 0x7fca080d1020 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.673+0000 7fca0f563640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0815dfc0 con 0x7fca080b2cb0 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fca0d2ff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fca0815dbe0 0x7fca080d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57210/0 (socket says 172.21.15.148:57210) 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fca0d2ff640 1 -- 172.21.15.148:0/3333945850 learned_addr learned my addr 172.21.15.148:0/3333945850 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:13.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fca0cafe640 1 --2- 172.21.15.148:0/3333945850 >> v2:172.21.15.188:3300/0 conn(0x7fca080b2cb0 0x7fca080d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fc9fffff640 1 -- 172.21.15.148:0/3333945850 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134790268 0 0) 0x7fca080b97a0 con 0x7fca0815dbe0 2024-04-25T10:00:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fc9fffff640 1 -- 172.21.15.148:0/3333945850 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9f4003520 con 0x7fca0815dbe0 2024-04-25T10:00:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fca0cafe640 1 -- 172.21.15.148:0/3333945850 >> v1:172.21.15.188:6789/0 conn(0x7fca080d1020 legacy=0x7fca080d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fca0cafe640 1 -- 172.21.15.148:0/3333945850 >> v1:172.21.15.148:6789/0 conn(0x7fca0815dbe0 legacy=0x7fca080d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.674+0000 7fca0cafe640 1 -- 172.21.15.148:0/3333945850 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080d1f30 con 0x7fca080b2cb0 2024-04-25T10:00:13.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0cafe640 1 --2- 172.21.15.148:0/3333945850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca080b2cb0 0x7fca080d2730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80097a0 tx=0x7fc9f802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619874033970be38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fc9fffff640 1 -- 172.21.15.148:0/3333945850 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8037070 con 0x7fca080b2cb0 2024-04-25T10:00:13.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fc9fffff640 1 -- 172.21.15.148:0/3333945850 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f8032440 con 0x7fca080b2cb0 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fc9fffff640 1 -- 172.21.15.148:0/3333945850 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80327f0 con 0x7fca080b2cb0 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 -- 172.21.15.148:0/3333945850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca080b2cb0 msgr2=0x7fca080d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 --2- 172.21.15.148:0/3333945850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca080b2cb0 0x7fca080d2730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80097a0 tx=0x7fc9f802a230 comp rx=0 tx=0).stop 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 -- 172.21.15.148:0/3333945850 shutdown_connections 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 --2- 172.21.15.148:0/3333945850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca080b2cb0 0x7fca080d2730 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 -- 172.21.15.148:0/3333945850 >> 172.21.15.148:0/3333945850 conn(0x7fca08061a20 msgr2=0x7fca08061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 -- 172.21.15.148:0/3333945850 shutdown_connections 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.675+0000 7fca0f563640 1 -- 172.21.15.148:0/3333945850 wait complete. 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 Processor -- start 2024-04-25T10:00:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 -- start start 2024-04-25T10:00:13.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 0x7fca080b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca080d1020 0x7fca0814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0815dbe0 0x7fca0815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca080b92a0 con 0x7fca080b2cb0 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca080d1a00 con 0x7fca0815dbe0 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0f563640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080b9420 con 0x7fca080d1020 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0d2ff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 0x7fca080b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0d2ff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 0x7fca080b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60632/0 (socket says 172.21.15.148:60632) 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.677+0000 7fca0d2ff640 1 -- 172.21.15.148:0/3942702862 learned_addr learned my addr 172.21.15.148:0/3942702862 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0db00640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0815dbe0 0x7fca0815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0cafe640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca080d1020 0x7fca0814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0d2ff640 1 -- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca080d1020 msgr2=0x7fca0814cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0d2ff640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca080d1020 0x7fca0814cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0d2ff640 1 -- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0815dbe0 msgr2=0x7fca0815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0d2ff640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0815dbe0 0x7fca0815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0d2ff640 1 -- 172.21.15.148:0/3942702862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0814d480 con 0x7fca080b2cb0 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0db00640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0815dbe0 0x7fca0815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0d2ff640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 0x7fca080b3090 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c440 tx=0x7fca0400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04017070 con 0x7fca080b2cb0 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca04005ce0 con 0x7fca080b2cb0 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca081601d0 con 0x7fca080b2cb0 2024-04-25T10:00:13.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040072b0 con 0x7fca080b2cb0 2024-04-25T10:00:13.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.678+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca080b73a0 con 0x7fca080b2cb0 2024-04-25T10:00:13.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.679+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca08073990 con 0x7fca080b2cb0 2024-04-25T10:00:13.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.679+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca0401a070 con 0x7fca080b2cb0 2024-04-25T10:00:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.680+0000 7fc9fdffb640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9ec06b4d0 0x7fc9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.680+0000 7fca0cafe640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9ec06b4d0 0x7fc9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.680+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fca040897b0 con 0x7fca080b2cb0 2024-04-25T10:00:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.680+0000 7fca0cafe640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9ec06b4d0 0x7fc9ec06d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8034000 tx=0x7fc9f802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.684+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca040584c0 con 0x7fca080b2cb0 2024-04-25T10:00:13.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.888+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca0806cb00 con 0x7fca080b2cb0 2024-04-25T10:00:13.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.889+0000 7fc9fdffb640 1 -- 172.21.15.148:0/3942702862 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fca04057dc0 con 0x7fca080b2cb0 2024-04-25T10:00:13.893 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:13.893 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:58:49.193553+0000","modified":"2024-04-25T09:58:55.879362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7284":{"gid":7284,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/82715169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":82715169},{"type":"v1","addr":"172.21.15.148:6837","nonce":82715169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:00:13.893 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 47 2024-04-25T10:00:13.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9ec06b4d0 msgr2=0x7fc9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9ec06b4d0 0x7fc9ec06d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8034000 tx=0x7fc9f802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 msgr2=0x7fca080b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 0x7fca080b3090 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fca0400c440 tx=0x7fca0400c900 comp rx=0 tx=0).stop 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 shutdown_connections 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc9ec06b4d0 0x7fc9ec06d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca0815dbe0 0x7fca0815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca080d1020 0x7fca0814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 --2- 172.21.15.148:0/3942702862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca080b2cb0 0x7fca080b3090 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 >> 172.21.15.148:0/3942702862 conn(0x7fca08061a20 msgr2=0x7fca0815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 shutdown_connections 2024-04-25T10:00:13.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.891+0000 7fca0f563640 1 -- 172.21.15.148:0/3942702862 wait complete. 2024-04-25T10:00:13.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:14.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.999+0000 7ff766942640 1 Processor -- start 2024-04-25T10:00:14.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:13.999+0000 7ff766942640 1 -- start start 2024-04-25T10:00:14.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff766942640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff766942640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7600b2cb0 0x7ff7600d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff766942640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600b97a0 con 0x7ff7600b92a0 2024-04-25T10:00:14.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff766942640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76015dfc0 con 0x7ff7600b2cb0 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff766942640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600d1a00 con 0x7ff76015dbe0 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:32990/0 (socket says 172.21.15.148:32990) 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75f7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7600b2cb0 0x7ff7600d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 -- 172.21.15.148:0/2704563278 learned_addr learned my addr 172.21.15.148:0/2704563278 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 -- 172.21.15.148:0/2704563278 >> v2:172.21.15.188:3300/0 conn(0x7ff7600b2cb0 msgr2=0x7ff7600d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 --2- 172.21.15.148:0/2704563278 >> v2:172.21.15.188:3300/0 conn(0x7ff7600b2cb0 0x7ff7600d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 -- 172.21.15.148:0/2704563278 >> v1:172.21.15.188:6789/0 conn(0x7ff7600b92a0 legacy=0x7ff7600b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.000+0000 7ff75ffff640 1 -- 172.21.15.148:0/2704563278 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600d1c50 con 0x7ff76015dbe0 2024-04-25T10:00:14.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.001+0000 7ff75ffff640 1 --2- 172.21.15.148:0/2704563278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600d0910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff750009870 tx=0x7ff75002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d62762920ef2d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.003+0000 7ff75effd640 1 -- 172.21.15.148:0/2704563278 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750037070 con 0x7ff76015dbe0 2024-04-25T10:00:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.003+0000 7ff75effd640 1 -- 172.21.15.148:0/2704563278 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff750032440 con 0x7ff76015dbe0 2024-04-25T10:00:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.003+0000 7ff75effd640 1 -- 172.21.15.148:0/2704563278 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750032810 con 0x7ff76015dbe0 2024-04-25T10:00:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 -- 172.21.15.148:0/2704563278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 msgr2=0x7ff7600d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 --2- 172.21.15.148:0/2704563278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600d0910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff750009870 tx=0x7ff75002a270 comp rx=0 tx=0).stop 2024-04-25T10:00:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 -- 172.21.15.148:0/2704563278 shutdown_connections 2024-04-25T10:00:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 --2- 172.21.15.148:0/2704563278 >> v2:172.21.15.188:3300/0 conn(0x7ff7600b2cb0 0x7ff7600d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 --2- 172.21.15.148:0/2704563278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600d0910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 -- 172.21.15.148:0/2704563278 >> 172.21.15.148:0/2704563278 conn(0x7ff760061a20 msgr2=0x7ff760061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 -- 172.21.15.148:0/2704563278 shutdown_connections 2024-04-25T10:00:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.004+0000 7ff766942640 1 -- 172.21.15.148:0/2704563278 wait complete. 2024-04-25T10:00:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.005+0000 7ff766942640 1 Processor -- start 2024-04-25T10:00:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.006+0000 7ff766942640 1 -- start start 2024-04-25T10:00:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.006+0000 7ff766942640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7600b2cb0 0x7ff7600cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.006+0000 7ff766942640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 0x7ff7600caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.006+0000 7ff766942640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff766942640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600b6ab0 con 0x7ff7600b92a0 2024-04-25T10:00:14.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff766942640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600d2210 con 0x7ff7600b2cb0 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff766942640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600b6c30 con 0x7ff76015dbe0 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 0x7ff7600caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 0x7ff7600caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60636/0 (socket says 172.21.15.148:60636) 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 -- 172.21.15.148:0/1782104769 learned_addr learned my addr 172.21.15.148:0/1782104769 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75ffff640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7600b2cb0 0x7ff7600cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff764edf640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 -- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 msgr2=0x7ff7600caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 -- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7600b2cb0 msgr2=0x7ff7600cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7600b2cb0 0x7ff7600cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 -- 172.21.15.148:0/1782104769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600cb630 con 0x7ff7600b92a0 2024-04-25T10:00:14.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75f7fe640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 0x7ff7600caa50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e9b0 tx=0x7ff75400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754017070 con 0x7ff7600b92a0 2024-04-25T10:00:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400c9a0 con 0x7ff7600b92a0 2024-04-25T10:00:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7600cb910 con 0x7ff7600b92a0 2024-04-25T10:00:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400cd90 con 0x7ff7600b92a0 2024-04-25T10:00:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.007+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7600cbed0 con 0x7ff7600b92a0 2024-04-25T10:00:14.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.008+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76015b100 con 0x7ff7600b92a0 2024-04-25T10:00:14.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.009+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff75401a070 con 0x7ff7600b92a0 2024-04-25T10:00:14.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.009+0000 7ff75cff9640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff74c06b500 0x7ff74c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.009+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7ff75408a2d0 con 0x7ff7600b92a0 2024-04-25T10:00:14.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.009+0000 7ff75ffff640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff74c06b500 0x7ff74c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.010+0000 7ff75ffff640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff74c06b500 0x7ff74c06d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff750034000 tx=0x7ff75002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.013+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff76015b100 con 0x7ff7600b92a0 2024-04-25T10:00:14.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.217+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff760073990 con 0x7ff7600b92a0 2024-04-25T10:00:14.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.218+0000 7ff75cff9640 1 -- 172.21.15.148:0/1782104769 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff754058950 con 0x7ff7600b92a0 2024-04-25T10:00:14.222 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:14.222 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T09:58:49.193553+0000","modified":"2024-04-25T09:58:55.879362+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7284":{"gid":7284,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/82715169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":82715169},{"type":"v1","addr":"172.21.15.148:6837","nonce":82715169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:00:14.223 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 47 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.219+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff74c06b500 msgr2=0x7ff74c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.219+0000 7ff766942640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff74c06b500 0x7ff74c06d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff750034000 tx=0x7ff75002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 msgr2=0x7ff7600caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 0x7ff7600caa50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e9b0 tx=0x7ff75400ee70 comp rx=0 tx=0).stop 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 shutdown_connections 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff74c06b500 0x7ff74c06d9c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76015dbe0 0x7ff7600caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7600b92a0 0x7ff7600caa50 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 --2- 172.21.15.148:0/1782104769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7600b2cb0 0x7ff7600cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 >> 172.21.15.148:0/1782104769 conn(0x7ff760061a20 msgr2=0x7ff76015be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 shutdown_connections 2024-04-25T10:00:14.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.220+0000 7ff766942640 1 -- 172.21.15.148:0/1782104769 wait complete. 2024-04-25T10:00:14.235 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:14.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.329+0000 7f52eb4d2640 1 Processor -- start 2024-04-25T10:00:14.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.329+0000 7f52eb4d2640 1 -- start start 2024-04-25T10:00:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52eb4d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e41597d0 0x7f52e406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52eb4d2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52e40b2d40 0x7f52e406b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52eb4d2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52e40cc320 0x7f52e40ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52eb4d2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e40d0480 con 0x7f52e40cc320 2024-04-25T10:00:14.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52eb4d2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e415c1b0 con 0x7f52e40b2d40 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52eb4d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e415c490 con 0x7f52e41597d0 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52e40cc320 0x7f52e40ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52e40cc320 0x7f52e40ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60648/0 (socket says 172.21.15.148:60648) 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 -- 172.21.15.148:0/4159300426 learned_addr learned my addr 172.21.15.148:0/4159300426 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e8a6d640 1 --2- 172.21.15.148:0/4159300426 >> v2:172.21.15.188:3300/0 conn(0x7f52e40b2d40 0x7f52e406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e926e640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e41597d0 0x7f52e406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 -- 172.21.15.148:0/4159300426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e41597d0 msgr2=0x7f52e406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e41597d0 0x7f52e406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 -- 172.21.15.148:0/4159300426 >> v2:172.21.15.188:3300/0 conn(0x7f52e40b2d40 msgr2=0x7f52e406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 --2- 172.21.15.148:0/4159300426 >> v2:172.21.15.188:3300/0 conn(0x7f52e40b2d40 0x7f52e406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e9a6f640 1 -- 172.21.15.148:0/4159300426 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e415c770 con 0x7f52e40cc320 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e8a6d640 1 --2- 172.21.15.148:0/4159300426 >> v2:172.21.15.188:3300/0 conn(0x7f52e40b2d40 0x7f52e406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.330+0000 7f52e926e640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e41597d0 0x7f52e406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:14.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.331+0000 7f52e9a6f640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40cc320 0x7f52e40ce2c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f52e40d17f0 tx=0x7f52e002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92ee14a04d9cf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.331+0000 7f52dbfff640 1 -- 172.21.15.148:0/4159300426 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0036070 con 0x7f52e40cc320 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.331+0000 7f52dbfff640 1 -- 172.21.15.148:0/4159300426 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e0031600 con 0x7f52e40cc320 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.331+0000 7f52dbfff640 1 -- 172.21.15.148:0/4159300426 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00319d0 con 0x7f52e40cc320 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 -- 172.21.15.148:0/4159300426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40cc320 msgr2=0x7f52e40ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40cc320 0x7f52e40ce2c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f52e40d17f0 tx=0x7f52e002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 -- 172.21.15.148:0/4159300426 shutdown_connections 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40cc320 0x7f52e40ce2c0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/4159300426 >> v2:172.21.15.188:3300/0 conn(0x7f52e40b2d40 0x7f52e406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/4159300426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e41597d0 0x7f52e406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 -- 172.21.15.148:0/4159300426 >> 172.21.15.148:0/4159300426 conn(0x7f52e40b2220 msgr2=0x7f52e40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 -- 172.21.15.148:0/4159300426 shutdown_connections 2024-04-25T10:00:14.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.332+0000 7f52eb4d2640 1 -- 172.21.15.148:0/4159300426 wait complete. 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.333+0000 7f52eb4d2640 1 Processor -- start 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.333+0000 7f52eb4d2640 1 -- start start 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 0x7f52e4153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e40cc320 0x7f52e406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e41597d0 0x7f52e40b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e406b920 con 0x7f52e40b2d40 2024-04-25T10:00:14.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e406b7a0 con 0x7f52e41597d0 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40cf570 con 0x7f52e40cc320 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 0x7f52e4153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 0x7f52e4153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60656/0 (socket says 172.21.15.148:60656) 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 -- 172.21.15.148:0/963847311 learned_addr learned my addr 172.21.15.148:0/963847311 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e8a6d640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e40cc320 0x7f52e406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e9a6f640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e41597d0 0x7f52e40b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 -- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e40cc320 msgr2=0x7f52e406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e40cc320 0x7f52e406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 -- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e41597d0 msgr2=0x7f52e40b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e41597d0 0x7f52e40b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 -- 172.21.15.148:0/963847311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4154490 con 0x7f52e40b2d40 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52e926e640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 0x7f52e4153f50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c7f0 tx=0x7f52dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc015070 con 0x7f52e40b2d40 2024-04-25T10:00:14.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc007820 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc007bd0 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.334+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e406ead0 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.335+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e406f090 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.336+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e4073990 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.336+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52dc018070 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.336+0000 7f52d9ffb640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.336+0000 7f52e8a6d640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.336+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f52dc08a460 con 0x7f52e40b2d40 2024-04-25T10:00:14.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.337+0000 7f52e8a6d640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f52e40b21f0 tx=0x7f52d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.341+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52dc0591c0 con 0x7f52e40b2d40 2024-04-25T10:00:14.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.512+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52e406d370 con 0x7f52e40b2d40 2024-04-25T10:00:14.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.513+0000 7f52d9ffb640 1 -- 172.21.15.148:0/963847311 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+17059 (secure 0 0 0) 0x7f52dc058ac0 con 0x7f52e40b2d40 2024-04-25T10:00:14.517 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:14.517 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":85,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T09:59:48.744514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:58:47.401474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T09:58:48.167626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 msgr2=0x7f52c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f52e40b21f0 tx=0x7f52d4008040 comp rx=0 tx=0).stop 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 msgr2=0x7f52e4153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 0x7f52e4153f50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00c7f0 tx=0x7f52dc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 shutdown_connections 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e41597d0 0x7f52e40b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e40cc320 0x7f52e406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 --2- 172.21.15.148:0/963847311 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52e40b2d40 0x7f52e4153f50 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 >> 172.21.15.148:0/963847311 conn(0x7f52e40b2220 msgr2=0x7f52e40b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 shutdown_connections 2024-04-25T10:00:14.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.515+0000 7f52eb4d2640 1 -- 172.21.15.148:0/963847311 wait complete. 2024-04-25T10:00:14.530 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:00:14.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.624+0000 7fa479163640 1 Processor -- start 2024-04-25T10:00:14.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.624+0000 7fa479163640 1 -- start start 2024-04-25T10:00:14.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa479163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa47406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa479163640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa47406b8c0 0x7fa4740b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa479163640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4740b49f0 0x7fa47406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa479163640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4740b3450 con 0x7fa47406b8c0 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa479163640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4740b36f0 con 0x7fa4740b49f0 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa479163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740b39d0 con 0x7fa47415dcd0 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa47406b8c0 0x7fa4740b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa47406b8c0 0x7fa4740b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60668/0 (socket says 172.21.15.148:60668) 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 -- 172.21.15.148:0/1391758800 learned_addr learned my addr 172.21.15.148:0/1391758800 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa472d9d640 1 --2- 172.21.15.148:0/1391758800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa47406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa47359e640 1 --2- 172.21.15.148:0/1391758800 >> v2:172.21.15.188:3300/0 conn(0x7fa4740b49f0 0x7fa47406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 -- 172.21.15.148:0/1391758800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 msgr2=0x7fa47406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 --2- 172.21.15.148:0/1391758800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa47406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 -- 172.21.15.148:0/1391758800 >> v2:172.21.15.188:3300/0 conn(0x7fa4740b49f0 msgr2=0x7fa47406e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 --2- 172.21.15.148:0/1391758800 >> v2:172.21.15.188:3300/0 conn(0x7fa4740b49f0 0x7fa47406e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.625+0000 7fa46a59c640 1 -- 172.21.15.148:0/1391758800 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740b3cb0 con 0x7fa47406b8c0 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa46a59c640 1 --2- 172.21.15.148:0/1391758800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa47406b8c0 0x7fa4740b44b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa474071670 tx=0x7fa45c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c5033ee99ce289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa47259c640 1 -- 172.21.15.148:0/1391758800 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c037070 con 0x7fa47406b8c0 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa47259c640 1 -- 172.21.15.148:0/1391758800 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c032440 con 0x7fa47406b8c0 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa47259c640 1 -- 172.21.15.148:0/1391758800 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c032810 con 0x7fa47406b8c0 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 -- 172.21.15.148:0/1391758800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa47406b8c0 msgr2=0x7fa4740b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 --2- 172.21.15.148:0/1391758800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa47406b8c0 0x7fa4740b44b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa474071670 tx=0x7fa45c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 -- 172.21.15.148:0/1391758800 shutdown_connections 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 --2- 172.21.15.148:0/1391758800 >> v2:172.21.15.188:3300/0 conn(0x7fa4740b49f0 0x7fa47406e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 --2- 172.21.15.148:0/1391758800 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa47406b8c0 0x7fa4740b44b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 --2- 172.21.15.148:0/1391758800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa47406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 -- 172.21.15.148:0/1391758800 >> 172.21.15.148:0/1391758800 conn(0x7fa47406abe0 msgr2=0x7fa47406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 -- 172.21.15.148:0/1391758800 shutdown_connections 2024-04-25T10:00:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.626+0000 7fa479163640 1 -- 172.21.15.148:0/1391758800 wait complete. 2024-04-25T10:00:14.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 Processor -- start 2024-04-25T10:00:14.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 -- start start 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa47406b8c0 0x7fa47415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 0x7fa474155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa474154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47406dda0 con 0x7fa4740b49f0 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47406dc20 con 0x7fa47406b8c0 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.628+0000 7fa479163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47406e830 con 0x7fa47415dcd0 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 0x7fa474155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 0x7fa474155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60680/0 (socket says 172.21.15.148:60680) 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 -- 172.21.15.148:0/2950450704 learned_addr learned my addr 172.21.15.148:0/2950450704 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa47359e640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa474154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa472d9d640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa47406b8c0 0x7fa47415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 -- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 msgr2=0x7fa474154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa474154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 -- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa47406b8c0 msgr2=0x7fa47415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa47406b8c0 0x7fa47415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 -- 172.21.15.148:0/2950450704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4741f7b60 con 0x7fa4740b49f0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa472d9d640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa47406b8c0 0x7fa47415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46a59c640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 0x7fa474155830 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa45c000c00 tx=0x7fa45c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c041070 con 0x7fa4740b49f0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c03c870 con 0x7fa4740b49f0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4740cdcd0 con 0x7fa4740b49f0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c037040 con 0x7fa4740b49f0 2024-04-25T10:00:14.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.629+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4740ce290 con 0x7fa4740b49f0 2024-04-25T10:00:14.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.630+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa474073a00 con 0x7fa4740b49f0 2024-04-25T10:00:14.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.630+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa45c044070 con 0x7fa4740b49f0 2024-04-25T10:00:14.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.631+0000 7fa46bfff640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa45406b4d0 0x7fa45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.631+0000 7fa472d9d640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa45406b4d0 0x7fa45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.631+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fa45c0ab0d0 con 0x7fa4740b49f0 2024-04-25T10:00:14.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.631+0000 7fa472d9d640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa45406b4d0 0x7fa45406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.635+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa45c079f50 con 0x7fa4740b49f0 2024-04-25T10:00:14.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.841+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa4740b47d0 con 0x7fa4740b49f0 2024-04-25T10:00:14.846 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:00:14.842+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:00:14.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.843+0000 7fa46bfff640 1 -- 172.21.15.148:0/2950450704 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa45c079850 con 0x7fa4740b49f0 2024-04-25T10:00:14.847 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:00:14.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa45406b4d0 msgr2=0x7fa45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa45406b4d0 0x7fa45406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464008040 comp rx=0 tx=0).stop 2024-04-25T10:00:14.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 msgr2=0x7fa474155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 0x7fa474155830 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa45c000c00 tx=0x7fa45c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 shutdown_connections 2024-04-25T10:00:14.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa45406b4d0 0x7fa45406d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa47415dcd0 0x7fa474154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4740b49f0 0x7fa474155830 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 --2- 172.21.15.148:0/2950450704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa47406b8c0 0x7fa47415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 >> 172.21.15.148:0/2950450704 conn(0x7fa47406abe0 msgr2=0x7fa47415cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 shutdown_connections 2024-04-25T10:00:14.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.846+0000 7fa479163640 1 -- 172.21.15.148:0/2950450704 wait complete. 2024-04-25T10:00:14.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:00:14.889 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:00:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.960+0000 7f10c395c640 1 Processor -- start 2024-04-25T10:00:14.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.960+0000 7f10c395c640 1 -- start start 2024-04-25T10:00:14.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c395c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f10bc06c330 0x7f10bc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c395c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10bc06c7e0 0x7f10bc0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c395c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc0d1d50 con 0x7f10bc158230 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c395c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc158610 con 0x7f10bc06c7e0 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c395c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc06bea0 con 0x7f10bc06c330 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10bc06c7e0 0x7f10bc0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f10bc06c7e0 0x7f10bc0b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60698/0 (socket says 172.21.15.148:60698) 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 -- 172.21.15.148:0/1135247716 learned_addr learned my addr 172.21.15.148:0/1135247716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c0ef7640 1 --2- 172.21.15.148:0/1135247716 >> v2:172.21.15.188:3300/0 conn(0x7f10bc06c330 0x7f10bc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 -- 172.21.15.148:0/1135247716 >> v2:172.21.15.188:3300/0 conn(0x7f10bc06c330 msgr2=0x7f10bc0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 --2- 172.21.15.148:0/1135247716 >> v2:172.21.15.188:3300/0 conn(0x7f10bc06c330 0x7f10bc0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 -- 172.21.15.148:0/1135247716 >> v1:172.21.15.188:6789/0 conn(0x7f10bc158230 legacy=0x7f10bc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c1ef9640 1 -- 172.21.15.148:0/1135247716 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0d2020 con 0x7f10bc06c7e0 2024-04-25T10:00:14.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.961+0000 7f10c0ef7640 1 --2- 172.21.15.148:0/1135247716 >> v2:172.21.15.188:3300/0 conn(0x7f10bc06c330 0x7f10bc0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:14.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.962+0000 7f10c1ef9640 1 --2- 172.21.15.148:0/1135247716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0b53a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0b4800 tx=0x7f10b402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fb54a8c4fbb695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.962+0000 7f10b3fff640 1 -- 172.21.15.148:0/1135247716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4036070 con 0x7f10bc06c7e0 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.962+0000 7f10b3fff640 1 -- 172.21.15.148:0/1135247716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b4031600 con 0x7f10bc06c7e0 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.962+0000 7f10b3fff640 1 -- 172.21.15.148:0/1135247716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40319d0 con 0x7f10bc06c7e0 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 -- 172.21.15.148:0/1135247716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c7e0 msgr2=0x7f10bc0b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 --2- 172.21.15.148:0/1135247716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0b53a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0b4800 tx=0x7f10b402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 -- 172.21.15.148:0/1135247716 shutdown_connections 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 --2- 172.21.15.148:0/1135247716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0b53a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 --2- 172.21.15.148:0/1135247716 >> v2:172.21.15.188:3300/0 conn(0x7f10bc06c330 0x7f10bc0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 -- 172.21.15.148:0/1135247716 >> 172.21.15.148:0/1135247716 conn(0x7f10bc061600 msgr2=0x7f10bc0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 -- 172.21.15.148:0/1135247716 shutdown_connections 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.963+0000 7f10c395c640 1 -- 172.21.15.148:0/1135247716 wait complete. 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.964+0000 7f10c395c640 1 Processor -- start 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.964+0000 7f10c395c640 1 -- start start 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 0x7f10bc1f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10bc158230 0x7f10bc1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0d25e0 con 0x7f10bc06c330 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc06bb30 con 0x7f10bc06c7e0 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15e320 con 0x7f10bc158230 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 0x7f10bc1f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 0x7f10bc1f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60700/0 (socket says 172.21.15.148:60700) 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 -- 172.21.15.148:0/945507636 learned_addr learned my addr 172.21.15.148:0/945507636 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c0ef7640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c1ef9640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10bc158230 0x7f10bc1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 -- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10bc158230 msgr2=0x7f10bc1565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10bc158230 0x7f10bc1565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 -- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10bc06c7e0 msgr2=0x7f10bc0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:14.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 -- 172.21.15.148:0/945507636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0cd820 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c1ef9640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10bc158230 0x7f10bc1565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c0ef7640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c16f8640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 0x7f10bc1f4630 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00c430 tx=0x7f10ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac011790 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac017070 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc158a00 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.965+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac011bb0 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.966+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc158fc0 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.967+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc06d3a0 con 0x7f10bc06c330 2024-04-25T10:00:14.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.967+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10ac01c460 con 0x7f10bc06c330 2024-04-25T10:00:14.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.967+0000 7f10b1ffb640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10a006b500 0x7f10a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:14.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.967+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f10ac08a1e0 con 0x7f10bc06c330 2024-04-25T10:00:14.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.967+0000 7f10c0ef7640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10a006b500 0x7f10a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:14.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.968+0000 7f10c0ef7640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10a006b500 0x7f10a006d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f10bc151b70 tx=0x7f10b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:14.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:14.971+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10bc06d3a0 con 0x7f10bc06c330 2024-04-25T10:00:15.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.179+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f10bc06de50 con 0x7f10bc06c330 2024-04-25T10:00:15.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.845+0000 7f10b1ffb640 1 -- 172.21.15.148:0/945507636 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f10ac058f10 con 0x7f10bc06c330 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.847+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10a006b500 msgr2=0x7f10a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.847+0000 7f10c395c640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10a006b500 0x7f10a006d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f10bc151b70 tx=0x7f10b8008040 comp rx=0 tx=0).stop 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.847+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 msgr2=0x7f10bc1f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.847+0000 7f10c395c640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 0x7f10bc1f4630 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00c430 tx=0x7f10ac00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 shutdown_connections 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f10a006b500 0x7f10a006d9c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10bc158230 0x7f10bc1565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10bc06c7e0 0x7f10bc0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 --2- 172.21.15.148:0/945507636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10bc06c330 0x7f10bc1f4630 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:15.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 >> 172.21.15.148:0/945507636 conn(0x7f10bc061600 msgr2=0x7f10bc0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:15.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 shutdown_connections 2024-04-25T10:00:15.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:15.848+0000 7f10c395c640 1 -- 172.21.15.148:0/945507636 wait complete. 2024-04-25T10:00:15.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:00:16.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.011+0000 7f8587010640 1 Processor -- start 2024-04-25T10:00:16.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.011+0000 7f8587010640 1 -- start start 2024-04-25T10:00:16.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f8587010640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f858006bbf0 0x7f858006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f8587010640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85800cb0e0 con 0x7f85800caa60 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f8587010640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85800cb3c0 con 0x7f858015dbd0 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f8587010640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85800cae90 con 0x7f858006bbf0 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f85855ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f858006bbf0 0x7f858006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f857ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f85800caa60 0x7f85800cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57228/0 (socket says 172.21.15.148:57228) 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f857ffff640 1 -- 172.21.15.148:0/3087746970 learned_addr learned my addr 172.21.15.148:0/3087746970 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f85855ad640 1 -- 172.21.15.148:0/3087746970 >> v1:172.21.15.188:6789/0 conn(0x7f858015dbd0 legacy=0x7f858006ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f85855ad640 1 -- 172.21.15.148:0/3087746970 >> v1:172.21.15.148:6789/0 conn(0x7f85800caa60 legacy=0x7f85800cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f85855ad640 1 -- 172.21.15.148:0/3087746970 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800cb6a0 con 0x7f858006bbf0 2024-04-25T10:00:16.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.012+0000 7f857f7fe640 1 -- 172.21.15.148:0/3087746970 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602971072 0 0) 0x7f85800cb0e0 con 0x7f85800caa60 2024-04-25T10:00:16.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.013+0000 7f85855ad640 1 --2- 172.21.15.148:0/3087746970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858006bbf0 0x7f858006c080 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f85800cb010 tx=0x7f857802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f63e1afaf87c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.013+0000 7f857f7fe640 1 -- 172.21.15.148:0/3087746970 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578007740 con 0x7f858006bbf0 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.013+0000 7f857f7fe640 1 -- 172.21.15.148:0/3087746970 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85780078e0 con 0x7f858006bbf0 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.013+0000 7f857f7fe640 1 -- 172.21.15.148:0/3087746970 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578007cb0 con 0x7f858006bbf0 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 -- 172.21.15.148:0/3087746970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858006bbf0 msgr2=0x7f858006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 --2- 172.21.15.148:0/3087746970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858006bbf0 0x7f858006c080 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f85800cb010 tx=0x7f857802a710 comp rx=0 tx=0).stop 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 -- 172.21.15.148:0/3087746970 shutdown_connections 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 --2- 172.21.15.148:0/3087746970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858006bbf0 0x7f858006c080 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 -- 172.21.15.148:0/3087746970 >> 172.21.15.148:0/3087746970 conn(0x7f8580071af0 msgr2=0x7f8580071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 -- 172.21.15.148:0/3087746970 shutdown_connections 2024-04-25T10:00:16.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.014+0000 7f8587010640 1 -- 172.21.15.148:0/3087746970 wait complete. 2024-04-25T10:00:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.015+0000 7f8587010640 1 Processor -- start 2024-04-25T10:00:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.015+0000 7f8587010640 1 -- start start 2024-04-25T10:00:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8587010640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006bbf0 0x7f85800c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8587010640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 0x7f85800ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8587010640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858015dbd0 0x7f8580145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8587010640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858015dfb0 con 0x7f85800caa60 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8587010640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800cbc60 con 0x7f858015dbd0 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8587010640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858015e130 con 0x7f858006bbf0 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 0x7f85800ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 0x7f85800ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60716/0 (socket says 172.21.15.148:60716) 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 -- 172.21.15.148:0/3414403801 learned_addr learned my addr 172.21.15.148:0/3414403801 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f85855ad640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858015dbd0 0x7f8580145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f8584dac640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006bbf0 0x7f85800c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 -- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006bbf0 msgr2=0x7f85800c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006bbf0 0x7f85800c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 -- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858015dbd0 msgr2=0x7f8580145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858015dbd0 0x7f8580145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:16.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.016+0000 7f857ffff640 1 -- 172.21.15.148:0/3414403801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858015e450 con 0x7f85800caa60 2024-04-25T10:00:16.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.017+0000 7f857ffff640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 0x7f85800ae950 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c4a0 tx=0x7f856c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.017+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c018070 con 0x7f85800caa60 2024-04-25T10:00:16.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.017+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c0074e0 con 0x7f85800caa60 2024-04-25T10:00:16.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.017+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858015e6f0 con 0x7f85800caa60 2024-04-25T10:00:16.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.017+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c004a20 con 0x7f85800caa60 2024-04-25T10:00:16.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.017+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8580146520 con 0x7f85800caa60 2024-04-25T10:00:16.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.018+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8580073a00 con 0x7f85800caa60 2024-04-25T10:00:16.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.018+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f856c01b070 con 0x7f85800caa60 2024-04-25T10:00:16.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.018+0000 7f857d7fa640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856406b4d0 0x7f856406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:16.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.019+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f856c089580 con 0x7f85800caa60 2024-04-25T10:00:16.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.019+0000 7f8584dac640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856406b4d0 0x7f856406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:16.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.019+0000 7f8584dac640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856406b4d0 0x7f856406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8574000f60 tx=0x7f8574009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:16.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.023+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f856c057c60 con 0x7f85800caa60 2024-04-25T10:00:16.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:16.194+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8580061b30 con 0x7f85800caa60 2024-04-25T10:00:17.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.016+0000 7f857d7fa640 1 -- 172.21.15.148:0/3414403801 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7f856c05bd00 con 0x7f85800caa60 2024-04-25T10:00:17.020 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:00:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856406b4d0 msgr2=0x7f856406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856406b4d0 0x7f856406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8574000f60 tx=0x7f8574009040 comp rx=0 tx=0).stop 2024-04-25T10:00:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 msgr2=0x7f85800ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 0x7f85800ae950 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c4a0 tx=0x7f856c00c960 comp rx=0 tx=0).stop 2024-04-25T10:00:17.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 shutdown_connections 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f856406b4d0 0x7f856406d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858015dbd0 0x7f8580145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85800caa60 0x7f85800ae950 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 --2- 172.21.15.148:0/3414403801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f858006bbf0 0x7f85800c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 >> 172.21.15.148:0/3414403801 conn(0x7f8580071af0 msgr2=0x7f85800b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 shutdown_connections 2024-04-25T10:00:17.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.019+0000 7f8587010640 1 -- 172.21.15.148:0/3414403801 wait complete. 2024-04-25T10:00:17.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:00:17.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.140+0000 7f9f00b6a640 1 Processor -- start 2024-04-25T10:00:17.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.141+0000 7f9f00b6a640 1 -- start start 2024-04-25T10:00:17.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.141+0000 7f9f00b6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9efc15dc20 0x7f9efc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.141+0000 7f9f00b6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 0x7f9efc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.141+0000 7f9f00b6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc15e000 con 0x7f9efc0b9330 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.141+0000 7f9f00b6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc0d1de0 con 0x7f9efc15dc20 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.141+0000 7f9f00b6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc161c60 con 0x7f9efc161710 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9efad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9efc0b9330 0x7f9efc06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57230/0 (socket says 172.21.15.148:57230) 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9efad9d640 1 -- 172.21.15.148:0/969622612 learned_addr learned my addr 172.21.15.148:0/969622612 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054810398 0 0) 0x7f9efc15e000 con 0x7f9efc0b9330 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee8003520 con 0x7f9efc0b9330 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 0x7f9efc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9efa59c640 1 --2- 172.21.15.148:0/969622612 >> v2:172.21.15.188:3300/0 conn(0x7f9efc15dc20 0x7f9efc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3427631407 0 0) 0x7f9ee8003520 con 0x7f9efc0b9330 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 msgr2=0x7f9efc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 0x7f9efc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 >> v2:172.21.15.188:3300/0 conn(0x7f9efc15dc20 msgr2=0x7f9efc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 --2- 172.21.15.148:0/969622612 >> v2:172.21.15.188:3300/0 conn(0x7f9efc15dc20 0x7f9efc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc161f40 con 0x7f9efc0b9330 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9ef40019f0 con 0x7f9efc0b9330 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 >> v1:172.21.15.148:6789/0 conn(0x7f9efc0b9330 legacy=0x7f9efc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 0x7f9ee80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9ee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee8004d40 0x7f9ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0d1de0 con 0x7f9ee8003f30 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc161c60 con 0x7f9ee8004d40 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee800b9c0 con 0x7f9efc161710 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9ee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9efa59c640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 0x7f9ee80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.142+0000 7f9efad9d640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee8004d40 0x7f9ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef3fff640 1 -- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 msgr2=0x7f9ee80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:17.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc161710 0x7f9ee80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef3fff640 1 -- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee8004d40 msgr2=0x7f9ee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee8004d40 0x7f9ee80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef3fff640 1 -- 172.21.15.148:0/969622612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee8003520 con 0x7f9ee8003f30 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9efad9d640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee8004d40 0x7f9ee80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9ee8004800 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009ac0 tx=0x7f9ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83c6956ab61c8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4036070 con 0x7f9ee8003f30 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee4031400 con 0x7f9ee8003f30 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.143+0000 7f9ef9d9b640 1 -- 172.21.15.148:0/969622612 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4009690 con 0x7f9ee8003f30 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9f00b6a640 1 -- 172.21.15.148:0/969622612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 msgr2=0x7f9ee8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/969622612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9ee8004800 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009ac0 tx=0x7f9ee8005830 comp rx=0 tx=0).stop 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9efad9d640 1 -- 172.21.15.148:0/969622612 reap_dead start 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9f00b6a640 1 -- 172.21.15.148:0/969622612 shutdown_connections 2024-04-25T10:00:17.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9f00b6a640 1 -- 172.21.15.148:0/969622612 >> 172.21.15.148:0/969622612 conn(0x7f9efc0d0420 msgr2=0x7f9efc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:17.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9f00b6a640 1 -- 172.21.15.148:0/969622612 shutdown_connections 2024-04-25T10:00:17.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.144+0000 7f9f00b6a640 1 -- 172.21.15.148:0/969622612 wait complete. 2024-04-25T10:00:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.145+0000 7f9f00b6a640 1 Processor -- start 2024-04-25T10:00:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.145+0000 7f9f00b6a640 1 -- start start 2024-04-25T10:00:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9f00b6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9efc0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9f00b6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9efc15a3f0 0x7f9efc15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9f00b6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc0b45f0 0x7f9efc0b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9f00b6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b5e50 con 0x7f9ee8003f30 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9f00b6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc162500 con 0x7f9efc15a3f0 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9f00b6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0b5fd0 con 0x7f9efc0b45f0 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9efc0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9efc0b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60726/0 (socket says 172.21.15.148:60726) 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 -- 172.21.15.148:0/842338373 learned_addr learned my addr 172.21.15.148:0/842338373 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9efc15a3f0 0x7f9efc15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efad9d640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc0b45f0 0x7f9efc0b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 -- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc0b45f0 msgr2=0x7f9efc0b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc0b45f0 0x7f9efc0b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 -- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9efc15a3f0 msgr2=0x7f9efc15e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:17.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9efc15a3f0 0x7f9efc15e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9efa59c640 1 -- 172.21.15.148:0/842338373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc15e540 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.146+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9efc15a3f0 0x7f9efc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.147+0000 7f9efa59c640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9efc0b26f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9eec011ad0 tx=0x7f9eec011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.147+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00fb50 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.147+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec00fcf0 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.147+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc15e7c0 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.147+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00fea0 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.147+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc15ed80 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.148+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc073a00 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.148+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9eec012040 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.149+0000 7f9ef2ffd640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9edc06b500 0x7f9edc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.149+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f9eec08a8e0 con 0x7f9ee8003f30 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.149+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9edc06b500 0x7f9edc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.149+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9edc06b500 0x7f9edc06d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4033000 tx=0x7f9ee4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:17.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.153+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9efc15e540 con 0x7f9ee8003f30 2024-04-25T10:00:17.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:17.337+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9efc0d16b0 con 0x7f9ee8003f30 2024-04-25T10:00:18.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.020+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/842338373 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9eec019030 con 0x7f9ee8003f30 2024-04-25T10:00:18.025 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:00:18.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9edc06b500 msgr2=0x7f9edc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9edc06b500 0x7f9edc06d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4033000 tx=0x7f9ee4008030 comp rx=0 tx=0).stop 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 msgr2=0x7f9efc0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9efc0b26f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9eec011ad0 tx=0x7f9eec011f90 comp rx=0 tx=0).stop 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 shutdown_connections 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9edc06b500 0x7f9edc06d9c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9efc0b45f0 0x7f9efc0b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9efc15a3f0 0x7f9efc15e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 --2- 172.21.15.148:0/842338373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee8003f30 0x7f9efc0b26f0 secure :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9eec011ad0 tx=0x7f9eec011f90 comp rx=0 tx=0).stop 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 >> 172.21.15.148:0/842338373 conn(0x7f9efc0d0420 msgr2=0x7f9efc0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 shutdown_connections 2024-04-25T10:00:18.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.024+0000 7f9f00b6a640 1 -- 172.21.15.148:0/842338373 wait complete. 2024-04-25T10:00:18.044 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:00:18.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.139+0000 7ffaa5be0640 1 Processor -- start 2024-04-25T10:00:18.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.139+0000 7ffaa5be0640 1 -- start start 2024-04-25T10:00:18.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffaa5be0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc4f0 0x7ffaa00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffaa5be0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffaa5be0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffaa015b270 0x7ffaa015b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffaa5be0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa00ce4c0 con 0x7ffaa015b270 2024-04-25T10:00:18.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffaa5be0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa0071270 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffaa5be0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0072280 con 0x7ffaa00cdb20 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffaa015b270 0x7ffaa015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ffaa015b270 0x7ffaa015b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60742/0 (socket says 172.21.15.148:60742) 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 -- 172.21.15.148:0/25611454 learned_addr learned my addr 172.21.15.148:0/25611454 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 -- 172.21.15.148:0/25611454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 msgr2=0x7ffaa00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9f7fe640 1 --2- 172.21.15.148:0/25611454 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc4f0 0x7ffaa00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9effd640 1 --2- 172.21.15.148:0/25611454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/25611454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 -- 172.21.15.148:0/25611454 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc4f0 msgr2=0x7ffaa00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/25611454 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc4f0 0x7ffaa00cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9ffff640 1 -- 172.21.15.148:0/25611454 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00714c0 con 0x7ffaa015b270 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.140+0000 7ffa9f7fe640 1 --2- 172.21.15.148:0/25611454 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc4f0 0x7ffaa00cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.141+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/25611454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa015b270 0x7ffaa015b680 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00cd290 tx=0x7ffa9802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73ef4574f3e8303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.141+0000 7ffa9e7fc640 1 -- 172.21.15.148:0/25611454 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98036070 con 0x7ffaa015b270 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.141+0000 7ffa9e7fc640 1 -- 172.21.15.148:0/25611454 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98031600 con 0x7ffaa015b270 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.141+0000 7ffa9e7fc640 1 -- 172.21.15.148:0/25611454 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa980319d0 con 0x7ffaa015b270 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 -- 172.21.15.148:0/25611454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa015b270 msgr2=0x7ffaa015b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/25611454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa015b270 0x7ffaa015b680 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00cd290 tx=0x7ffa9802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 -- 172.21.15.148:0/25611454 shutdown_connections 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/25611454 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa015b270 0x7ffaa015b680 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/25611454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/25611454 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc4f0 0x7ffaa00cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 -- 172.21.15.148:0/25611454 >> 172.21.15.148:0/25611454 conn(0x7ffaa006abe0 msgr2=0x7ffaa006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 -- 172.21.15.148:0/25611454 shutdown_connections 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.142+0000 7ffaa5be0640 1 -- 172.21.15.148:0/25611454 wait complete. 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.143+0000 7ffaa5be0640 1 Processor -- start 2024-04-25T10:00:18.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.143+0000 7ffaa5be0640 1 -- start start 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 0x7ffaa01451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa015b270 0x7ffaa00c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0071d60 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0071a80 con 0x7ffaa015b270 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa015bbc0 con 0x7ffaa00cdb20 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 0x7ffaa01451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 0x7ffaa01451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60758/0 (socket says 172.21.15.148:60758) 2024-04-25T10:00:18.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 -- 172.21.15.148:0/130828175 learned_addr learned my addr 172.21.15.148:0/130828175 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 -- 172.21.15.148:0/130828175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 msgr2=0x7ffaa00c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 -- 172.21.15.148:0/130828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa015b270 msgr2=0x7ffaa00c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa015b270 0x7ffaa00c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 -- 172.21.15.148:0/130828175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0145720 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa9f7fe640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 0x7ffaa01451e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c7f0 tx=0x7ffa9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90015070 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa90007820 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90007c10 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa00c5190 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.144+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa00c5750 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.145+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa0073a00 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.146+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa90018070 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.146+0000 7ffa87fff640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b4d0 0x7ffa8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.146+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ffa9008a140 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.146+0000 7ffa9effd640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b4d0 0x7ffa8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.147+0000 7ffa9effd640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b4d0 0x7ffa8c06d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffaa006eee0 tx=0x7ffa94008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.150+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa90059140 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.319+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ffaa006d3a0 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.322+0000 7ffa87fff640 1 -- 172.21.15.148:0/130828175 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa90058a40 con 0x7ffaa00cc4f0 2024-04-25T10:00:18.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b4d0 msgr2=0x7ffa8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b4d0 0x7ffa8c06d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffaa006eee0 tx=0x7ffa94008040 comp rx=0 tx=0).stop 2024-04-25T10:00:18.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 msgr2=0x7ffaa01451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 0x7ffaa01451e0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000c7f0 tx=0x7ffa9000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 shutdown_connections 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b4d0 0x7ffa8c06d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa015b270 0x7ffaa00c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa00cdb20 0x7ffaa00c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 --2- 172.21.15.148:0/130828175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa00cc4f0 0x7ffaa01451e0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 >> 172.21.15.148:0/130828175 conn(0x7ffaa006abe0 msgr2=0x7ffaa006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 shutdown_connections 2024-04-25T10:00:18.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.324+0000 7ffaa5be0640 1 -- 172.21.15.148:0/130828175 wait complete. 2024-04-25T10:00:18.339 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:00:18.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.433+0000 7ff0e8807640 1 Processor -- start 2024-04-25T10:00:18.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.433+0000 7ff0e8807640 1 -- start start 2024-04-25T10:00:18.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.435+0000 7ff0e8807640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0e4162020 0x7ff0e4070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.435+0000 7ff0e8807640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0e406b2f0 0x7ff0e40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.435+0000 7ff0e8807640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40b37f0 con 0x7ff0e40709d0 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.435+0000 7ff0e8807640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40b3ad0 con 0x7ff0e406b2f0 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.435+0000 7ff0e8807640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40b3db0 con 0x7ff0e4162020 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.436+0000 7ff0e1d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff0e40709d0 0x7ff0e406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57232/0 (socket says 172.21.15.148:57232) 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.436+0000 7ff0e1d9b640 1 -- 172.21.15.148:0/1953463834 learned_addr learned my addr 172.21.15.148:0/1953463834 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.436+0000 7ff0e2d9d640 1 --2- 172.21.15.148:0/1953463834 >> v2:172.21.15.148:3300/0 conn(0x7ff0e406b2f0 0x7ff0e40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.436+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142069672 0 0) 0x7ff0e40b37f0 con 0x7ff0e40709d0 2024-04-25T10:00:18.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.436+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0cc003520 con 0x7ff0e40709d0 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 644981360 0 0) 0x7ff0cc003520 con 0x7ff0e40709d0 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 >> v2:172.21.15.188:3300/0 conn(0x7ff0e4162020 msgr2=0x7ff0e4070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 --2- 172.21.15.148:0/1953463834 >> v2:172.21.15.188:3300/0 conn(0x7ff0e4162020 0x7ff0e4070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 >> v2:172.21.15.148:3300/0 conn(0x7ff0e406b2f0 msgr2=0x7ff0e40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 --2- 172.21.15.148:0/1953463834 >> v2:172.21.15.148:3300/0 conn(0x7ff0e406b2f0 0x7ff0e40cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40b4090 con 0x7ff0e40709d0 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff0dc0019f0 con 0x7ff0e40709d0 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 >> v1:172.21.15.148:6789/0 conn(0x7ff0e40709d0 legacy=0x7ff0e406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0cc003a00 0x7ff0cc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc004800 0x7ff0cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc005210 0x7ff0cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40b3ad0 con 0x7ff0cc005210 2024-04-25T10:00:18.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40b3db0 con 0x7ff0cc003a00 2024-04-25T10:00:18.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc00c1e0 con 0x7ff0cc004800 2024-04-25T10:00:18.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.437+0000 7ff0e1d9b640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc004800 0x7ff0cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e259c640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0cc003a00 0x7ff0cc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e2d9d640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc005210 0x7ff0cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e259c640 1 -- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc004800 msgr2=0x7ff0cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e259c640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc004800 0x7ff0cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e259c640 1 -- 172.21.15.148:0/1953463834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc005210 msgr2=0x7ff0cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e259c640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc005210 0x7ff0cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.438+0000 7ff0e259c640 1 -- 172.21.15.148:0/1953463834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc003520 con 0x7ff0cc003a00 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.439+0000 7ff0e2d9d640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0cc005210 0x7ff0cc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.439+0000 7ff0e2d9d640 1 -- 172.21.15.148:0/1953463834 reap_dead start 2024-04-25T10:00:18.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.440+0000 7ff0e259c640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0cc003a00 0x7ff0cc003e20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009860 tx=0x7ff0cc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d828c401d8191ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.440+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4037070 con 0x7ff0cc003a00 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.440+0000 7ff0e159a640 1 -- 172.21.15.148:0/1953463834 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d402aa60 con 0x7ff0cc003a00 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.441+0000 7ff0e8807640 1 -- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0cc003a00 msgr2=0x7ff0cc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.441+0000 7ff0e8807640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0cc003a00 0x7ff0cc003e20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009860 tx=0x7ff0cc005830 comp rx=0 tx=0).stop 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.441+0000 7ff0e8807640 1 -- 172.21.15.148:0/1953463834 shutdown_connections 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.441+0000 7ff0e8807640 1 --2- 172.21.15.148:0/1953463834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0cc003a00 0x7ff0cc003e20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.441+0000 7ff0e8807640 1 -- 172.21.15.148:0/1953463834 >> 172.21.15.148:0/1953463834 conn(0x7ff0e40b2370 msgr2=0x7ff0e40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:18.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.446+0000 7ff0e8807640 1 -- 172.21.15.148:0/1953463834 shutdown_connections 2024-04-25T10:00:18.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.446+0000 7ff0e8807640 1 -- 172.21.15.148:0/1953463834 wait complete. 2024-04-25T10:00:18.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.448+0000 7ff0e8807640 1 Processor -- start 2024-04-25T10:00:18.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.449+0000 7ff0e8807640 1 -- start start 2024-04-25T10:00:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.450+0000 7ff0e8807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc003a00 0x7ff0e40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.450+0000 7ff0e8807640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e4159360 0x7ff0e4157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.450+0000 7ff0e8807640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 0x7ff0e4149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.450+0000 7ff0e8807640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4072a00 con 0x7ff0e4159360 2024-04-25T10:00:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.450+0000 7ff0e8807640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4162400 con 0x7ff0e4157cb0 2024-04-25T10:00:18.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.450+0000 7ff0e8807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40b4890 con 0x7ff0cc003a00 2024-04-25T10:00:18.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.451+0000 7ff0e2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 0x7ff0e4149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.451+0000 7ff0e2d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 0x7ff0e4149520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:54784/0 (socket says 172.21.15.148:54784) 2024-04-25T10:00:18.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.451+0000 7ff0e2d9d640 1 -- 172.21.15.148:0/2951193062 learned_addr learned my addr 172.21.15.148:0/2951193062 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:18.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.451+0000 7ff0e2d9d640 1 -- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc003a00 msgr2=0x7ff0e40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:18.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.451+0000 7ff0e2d9d640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc003a00 0x7ff0e40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.451+0000 7ff0e2d9d640 1 -- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e4159360 msgr2=0x7ff0e4157770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:00:18.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.452+0000 7ff0e1d9b640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e4159360 0x7ff0e4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.453+0000 7ff0e2d9d640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e4159360 0x7ff0e4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:18.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.453+0000 7ff0e2d9d640 1 -- 172.21.15.148:0/2951193062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e41582c0 con 0x7ff0e4157cb0 2024-04-25T10:00:18.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.453+0000 7ff0e1d9b640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e4159360 0x7ff0e4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:18.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.454+0000 7ff0e2d9d640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 0x7ff0e4149520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40b3020 tx=0x7ff0d8003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.454+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800b5e0 con 0x7ff0e4157cb0 2024-04-25T10:00:18.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.454+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d801b070 con 0x7ff0e4157cb0 2024-04-25T10:00:18.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.454+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8010930 con 0x7ff0e4157cb0 2024-04-25T10:00:18.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.455+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e41585a0 con 0x7ff0e4157cb0 2024-04-25T10:00:18.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.455+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e4158b60 con 0x7ff0e4157cb0 2024-04-25T10:00:18.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.456+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4073990 con 0x7ff0e4157cb0 2024-04-25T10:00:18.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.456+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0d8012040 con 0x7ff0e4157cb0 2024-04-25T10:00:18.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.457+0000 7ff0d2ffd640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0c406b4d0 0x7ff0c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:18.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.457+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0d8089ba0 con 0x7ff0e4157cb0 2024-04-25T10:00:18.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.457+0000 7ff0e259c640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0c406b4d0 0x7ff0c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:18.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.457+0000 7ff0e259c640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0c406b4d0 0x7ff0c406d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4034000 tx=0x7ff0d402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:18.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.461+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0d8025020 con 0x7ff0e4157cb0 2024-04-25T10:00:18.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:18.695+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7ff0e40b5120 con 0x7ff0e4157cb0 2024-04-25T10:00:19.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.024+0000 7ff0d2ffd640 1 -- 172.21.15.148:0/2951193062 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v293) v1 ==== 158+0+0 (secure 0 0 0) 0x7ff0d8058af0 con 0x7ff0e4157cb0 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0c406b4d0 msgr2=0x7ff0c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0c406b4d0 0x7ff0c406d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4034000 tx=0x7ff0d402c000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 msgr2=0x7ff0e4149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 0x7ff0e4149520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40b3020 tx=0x7ff0d8003cd0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 shutdown_connections 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0c406b4d0 0x7ff0c406d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e4157cb0 0x7ff0e4149520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0e4159360 0x7ff0e4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 --2- 172.21.15.148:0/2951193062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0cc003a00 0x7ff0e40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 >> 172.21.15.148:0/2951193062 conn(0x7ff0e40b2370 msgr2=0x7ff0e40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 shutdown_connections 2024-04-25T10:00:19.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.027+0000 7ff0e8807640 1 -- 172.21.15.148:0/2951193062 wait complete. 2024-04-25T10:00:19.042 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T10:00:19.043 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:00:19.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.137+0000 7fa0e32fc640 1 Processor -- start 2024-04-25T10:00:19.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e32fc640 1 -- start start 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e32fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0dc09a150 0x7fa0dc0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e32fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc0cc3c0 con 0x7fa0dc0b0b90 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e32fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc15f400 con 0x7fa0dc15f020 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e32fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc0cccb0 con 0x7fa0dc09a150 2024-04-25T10:00:19.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e0897640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa0dc0b0b90 0x7fa0dc099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57238/0 (socket says 172.21.15.148:57238) 2024-04-25T10:00:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.138+0000 7fa0e0897640 1 -- 172.21.15.148:0/1132668337 learned_addr learned my addr 172.21.15.148:0/1132668337 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038416795 0 0) 0x7fa0dc0cc3c0 con 0x7fa0dc0b0b90 2024-04-25T10:00:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c4003520 con 0x7fa0dc0b0b90 2024-04-25T10:00:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e1899640 1 --2- 172.21.15.148:0/1132668337 >> v2:172.21.15.188:3300/0 conn(0x7fa0dc09a150 0x7fa0dc0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3966842393 0 0) 0x7fa0c4003520 con 0x7fa0dc0b0b90 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 >> v2:172.21.15.188:3300/0 conn(0x7fa0dc09a150 msgr2=0x7fa0dc0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 --2- 172.21.15.148:0/1132668337 >> v2:172.21.15.188:3300/0 conn(0x7fa0dc09a150 0x7fa0dc0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 >> v1:172.21.15.188:6789/0 conn(0x7fa0dc15f020 legacy=0x7fa0dc0b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc0d0550 con 0x7fa0dc0b0b90 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa0d0001670 con 0x7fa0dc0b0b90 2024-04-25T10:00:19.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 >> v1:172.21.15.148:6789/0 conn(0x7fa0dc0b0b90 legacy=0x7fa0dc099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c40039b0 0x7fa0c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40051c0 0x7fa0c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0cccb0 con 0x7fa0c4004800 2024-04-25T10:00:19.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c400c060 con 0x7fa0c40039b0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c400c260 con 0x7fa0c40051c0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e0897640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e1098640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c40039b0 0x7fa0c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e1899640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40051c0 0x7fa0c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e0897640 1 -- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40051c0 msgr2=0x7fa0c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e0897640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c40051c0 0x7fa0c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e0897640 1 -- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c40039b0 msgr2=0x7fa0c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e0897640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c40039b0 0x7fa0c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e0897640 1 -- 172.21.15.148:0/1132668337 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4003520 con 0x7fa0c4004800 2024-04-25T10:00:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.139+0000 7fa0e1899640 1 -- 172.21.15.148:0/1132668337 reap_dead start 2024-04-25T10:00:19.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.140+0000 7fa0e0897640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000c210 tx=0x7fa0c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d0a283337857d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.140+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0007d30 con 0x7fa0c4004800 2024-04-25T10:00:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.140+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0005ad0 con 0x7fa0c4004800 2024-04-25T10:00:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.140+0000 7fa0dbfff640 1 -- 172.21.15.148:0/1132668337 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0007070 con 0x7fa0c4004800 2024-04-25T10:00:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 -- 172.21.15.148:0/1132668337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 msgr2=0x7fa0c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000c210 tx=0x7fa0c4005830 comp rx=0 tx=0).stop 2024-04-25T10:00:19.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 -- 172.21.15.148:0/1132668337 shutdown_connections 2024-04-25T10:00:19.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/1132668337 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0c4004c80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 -- 172.21.15.148:0/1132668337 >> 172.21.15.148:0/1132668337 conn(0x7fa0dc0b3310 msgr2=0x7fa0dc0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 -- 172.21.15.148:0/1132668337 shutdown_connections 2024-04-25T10:00:19.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.141+0000 7fa0e32fc640 1 -- 172.21.15.148:0/1132668337 wait complete. 2024-04-25T10:00:19.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.142+0000 7fa0e32fc640 1 Processor -- start 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.142+0000 7fa0e32fc640 1 -- start start 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0dc158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0dc0c5d60 0x7fa0dc0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0dc0b5ed0 0x7fa0dc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0cc940 con 0x7fa0c4004800 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0cc660 con 0x7fa0dc0b5ed0 2024-04-25T10:00:19.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0cfe00 con 0x7fa0dc0c5d60 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0dc158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0dc158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60798/0 (socket says 172.21.15.148:60798) 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 -- 172.21.15.148:0/971052405 learned_addr learned my addr 172.21.15.148:0/971052405 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e0897640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0dc0c5d60 0x7fa0dc0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1899640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0dc0b5ed0 0x7fa0dc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 -- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0dc0c5d60 msgr2=0x7fa0dc0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0dc0c5d60 0x7fa0dc0b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 -- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0dc0b5ed0 msgr2=0x7fa0dc0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0dc0b5ed0 0x7fa0dc0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 -- 172.21.15.148:0/971052405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc1587b0 con 0x7fa0c4004800 2024-04-25T10:00:19.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e1098640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0dc158270 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400f7a0 tx=0x7fa0d400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40050d0 con 0x7fa0c4004800 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d4018070 con 0x7fa0c4004800 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc0c8a20 con 0x7fa0c4004800 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.143+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4005490 con 0x7fa0c4004800 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.144+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc0c8fe0 con 0x7fa0c4004800 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.145+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc073990 con 0x7fa0c4004800 2024-04-25T10:00:19.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.145+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0d400d730 con 0x7fa0c4004800 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.145+0000 7fa0d9ffb640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0c806b500 0x7fa0c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.146+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa0d4089d20 con 0x7fa0c4004800 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.146+0000 7fa0e0897640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0c806b500 0x7fa0c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.146+0000 7fa0e0897640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0c806b500 0x7fa0c806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0032000 tx=0x7fa0d002d000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.150+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0d4058da0 con 0x7fa0c4004800 2024-04-25T10:00:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.322+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0dc0b77a0 con 0x7fa0c4004800 2024-04-25T10:00:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.323+0000 7fa0d9ffb640 1 -- 172.21.15.148:0/971052405 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13340 (secure 0 0 0) 0x7fa0d40586a0 con 0x7fa0c4004800 2024-04-25T10:00:19.327 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:19.327 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":88,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:18.017265+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/82715169":"2024-04-26T10:00:14.842864+0000","172.21.15.148:6835/82715169":"2024-04-26T10:00:14.842864+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:19.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0c806b500 msgr2=0x7fa0c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0c806b500 0x7fa0c806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0032000 tx=0x7fa0d002d000 comp rx=0 tx=0).stop 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 msgr2=0x7fa0dc158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0dc158270 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400f7a0 tx=0x7fa0d400fc60 comp rx=0 tx=0).stop 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 shutdown_connections 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa0c806b500 0x7fa0c806d9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0dc0b5ed0 0x7fa0dc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0dc0c5d60 0x7fa0dc0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 --2- 172.21.15.148:0/971052405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0c4004800 0x7fa0dc158270 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 >> 172.21.15.148:0/971052405 conn(0x7fa0dc0b3310 msgr2=0x7fa0dc09a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 shutdown_connections 2024-04-25T10:00:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.325+0000 7fa0e32fc640 1 -- 172.21.15.148:0/971052405 wait complete. 2024-04-25T10:00:19.341 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:19.341 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.436+0000 7faa860cb640 1 Processor -- start 2024-04-25T10:00:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.436+0000 7faa860cb640 1 -- start start 2024-04-25T10:00:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.437+0000 7faa860cb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa8015a9f0 0x7faa800c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.437+0000 7faa860cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.437+0000 7faa860cb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa800c6940 con 0x7faa800cd520 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.437+0000 7faa860cb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa800b0020 con 0x7faa8015a9f0 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.437+0000 7faa860cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa800cb5a0 con 0x7faa800cdbf0 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.437+0000 7faa7f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa8015a9f0 0x7faa800c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7faa8015a9f0 0x7faa800c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60800/0 (socket says 172.21.15.148:60800) 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 -- 172.21.15.148:0/2964458980 learned_addr learned my addr 172.21.15.148:0/2964458980 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7ffff640 1 --1- 172.21.15.148:0/2964458980 >> v1:172.21.15.148:6789/0 conn(0x7faa800cd520 0x7faa800ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57246/0 (socket says 172.21.15.148:57246) 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 -- 172.21.15.148:0/2964458980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 msgr2=0x7faa800ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 --2- 172.21.15.148:0/2964458980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 -- 172.21.15.148:0/2964458980 >> v1:172.21.15.148:6789/0 conn(0x7faa800cd520 legacy=0x7faa800ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 -- 172.21.15.148:0/2964458980 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8015add0 con 0x7faa8015a9f0 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7e7fc640 1 -- 172.21.15.148:0/2964458980 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461458087 0 0) 0x7faa78001490 con 0x7faa800cd520 2024-04-25T10:00:19.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.438+0000 7faa7f7fe640 1 --2- 172.21.15.148:0/2964458980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa800c6010 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7faa7000bb10 tx=0x7faa7002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7519e5e91a2470b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa7e7fc640 1 -- 172.21.15.148:0/2964458980 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70036020 con 0x7faa8015a9f0 2024-04-25T10:00:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa7e7fc640 1 -- 172.21.15.148:0/2964458980 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa70032820 con 0x7faa8015a9f0 2024-04-25T10:00:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa7e7fc640 1 -- 172.21.15.148:0/2964458980 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70032bf0 con 0x7faa8015a9f0 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 -- 172.21.15.148:0/2964458980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 msgr2=0x7faa800c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 --2- 172.21.15.148:0/2964458980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa800c6010 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7faa7000bb10 tx=0x7faa7002c920 comp rx=0 tx=0).stop 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 -- 172.21.15.148:0/2964458980 shutdown_connections 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 --2- 172.21.15.148:0/2964458980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 --2- 172.21.15.148:0/2964458980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa800c6010 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 -- 172.21.15.148:0/2964458980 >> 172.21.15.148:0/2964458980 conn(0x7faa800b2390 msgr2=0x7faa800b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 -- 172.21.15.148:0/2964458980 shutdown_connections 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.439+0000 7faa860cb640 1 -- 172.21.15.148:0/2964458980 wait complete. 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.440+0000 7faa860cb640 1 Processor -- start 2024-04-25T10:00:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 -- start start 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa800cd520 0x7faa800b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa80156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800cb230 con 0x7faa8015a9f0 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa800caf50 con 0x7faa800cd520 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa860cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa800ce070 con 0x7faa800cdbf0 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa7ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa80156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa7ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa80156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60808/0 (socket says 172.21.15.148:60808) 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.441+0000 7faa7ffff640 1 -- 172.21.15.148:0/2774587016 learned_addr learned my addr 172.21.15.148:0/2774587016 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7f7fe640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa800cd520 0x7faa800b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7ffff640 1 -- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 msgr2=0x7faa800af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7effd640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7ffff640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7ffff640 1 -- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa800cd520 msgr2=0x7faa800b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7ffff640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa800cd520 0x7faa800b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7ffff640 1 -- 172.21.15.148:0/2774587016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa800b3af0 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa7ffff640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa80156130 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c7d0 tx=0x7faa7800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78018070 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7800a640 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa800b3dd0 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7800aab0 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.442+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa800b4390 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.443+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8006cc80 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.443+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa7801b070 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.444+0000 7faa67fff640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faa6c06b4d0 0x7faa6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.444+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7faa78089680 con 0x7faa8015a9f0 2024-04-25T10:00:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.444+0000 7faa7f7fe640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faa6c06b4d0 0x7faa6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.444+0000 7faa7f7fe640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faa6c06b4d0 0x7faa6c06d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faa70037000 tx=0x7faa70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.448+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa800c6940 con 0x7faa8015a9f0 2024-04-25T10:00:19.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.617+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa800cd9b0 con 0x7faa8015a9f0 2024-04-25T10:00:19.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.618+0000 7faa67fff640 1 -- 172.21.15.148:0/2774587016 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa800cd9b0 con 0x7faa8015a9f0 2024-04-25T10:00:19.622 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faa6c06b4d0 msgr2=0x7faa6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faa6c06b4d0 0x7faa6c06d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7faa70037000 tx=0x7faa70008040 comp rx=0 tx=0).stop 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 msgr2=0x7faa80156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa80156130 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7faa7800c7d0 tx=0x7faa7800cc90 comp rx=0 tx=0).stop 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 shutdown_connections 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7faa6c06b4d0 0x7faa6c06d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa8015a9f0 0x7faa80156130 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa800cdbf0 0x7faa800af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 --2- 172.21.15.148:0/2774587016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa800cd520 0x7faa800b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 >> 172.21.15.148:0/2774587016 conn(0x7faa800b2390 msgr2=0x7faa8006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 shutdown_connections 2024-04-25T10:00:19.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.620+0000 7faa860cb640 1 -- 172.21.15.148:0/2774587016 wait complete. 2024-04-25T10:00:19.635 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:00:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.728+0000 7f9896bc4640 1 Processor -- start 2024-04-25T10:00:19.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.728+0000 7f9896bc4640 1 -- start start 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9896bc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98900cfec0 0x7f98900cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9896bc4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98900ce3c0 0x7f989006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9896bc4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9890071480 con 0x7f9890154ee0 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9896bc4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9890072090 con 0x7f98900ce3c0 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9896bc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9890071230 con 0x7f98900cfec0 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98900ce3c0 0x7f989006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98900ce3c0 0x7f989006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60820/0 (socket says 172.21.15.148:60820) 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 -- 172.21.15.148:0/4160940892 learned_addr learned my addr 172.21.15.148:0/4160940892 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f988ffff640 1 --2- 172.21.15.148:0/4160940892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98900cfec0 0x7f98900cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 -- 172.21.15.148:0/4160940892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98900cfec0 msgr2=0x7f98900cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 --2- 172.21.15.148:0/4160940892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98900cfec0 0x7f98900cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 -- 172.21.15.148:0/4160940892 >> v1:172.21.15.188:6789/0 conn(0x7f9890154ee0 legacy=0x7f98900cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:19.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.729+0000 7f9895161640 1 -- 172.21.15.148:0/4160940892 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890071760 con 0x7f98900ce3c0 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.730+0000 7f9895161640 1 --2- 172.21.15.148:0/4160940892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f989006d6f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f988800d150 tx=0x7f988802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bf3f69f4bd4e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.730+0000 7f988f7fe640 1 -- 172.21.15.148:0/4160940892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800b2e0 con 0x7f98900ce3c0 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.730+0000 7f988f7fe640 1 -- 172.21.15.148:0/4160940892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9888009600 con 0x7f98900ce3c0 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.730+0000 7f988f7fe640 1 -- 172.21.15.148:0/4160940892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800b480 con 0x7f98900ce3c0 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 -- 172.21.15.148:0/4160940892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 msgr2=0x7f989006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 --2- 172.21.15.148:0/4160940892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f989006d6f0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f988800d150 tx=0x7f988802c8e0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 -- 172.21.15.148:0/4160940892 shutdown_connections 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 --2- 172.21.15.148:0/4160940892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f989006d6f0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 --2- 172.21.15.148:0/4160940892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98900cfec0 0x7f98900cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 -- 172.21.15.148:0/4160940892 >> 172.21.15.148:0/4160940892 conn(0x7f989006abe0 msgr2=0x7f989006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 -- 172.21.15.148:0/4160940892 shutdown_connections 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.731+0000 7f9896bc4640 1 -- 172.21.15.148:0/4160940892 wait complete. 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.732+0000 7f9896bc4640 1 Processor -- start 2024-04-25T10:00:19.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.732+0000 7f9896bc4640 1 -- start start 2024-04-25T10:00:19.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9896bc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f9890146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9896bc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98900cfec0 0x7f9890143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9896bc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9890154ee0 0x7f98900b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9896bc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9890071d20 con 0x7f98900ce3c0 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9896bc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900ce800 con 0x7f98900cfec0 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9896bc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98901552c0 con 0x7f9890154ee0 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f9890146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f9890146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60824/0 (socket says 172.21.15.148:60824) 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 -- 172.21.15.148:0/2147632099 learned_addr learned my addr 172.21.15.148:0/2147632099 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f988ffff640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98900cfec0 0x7f9890143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9895161640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9890154ee0 0x7f98900b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 -- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9890154ee0 msgr2=0x7f98900b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9890154ee0 0x7f98900b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 -- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98900cfec0 msgr2=0x7f9890143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98900cfec0 0x7f9890143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 -- 172.21.15.148:0/2147632099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900b1560 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f988ffff640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98900cfec0 0x7f9890143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.733+0000 7f9894960640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f9890146b80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f988000c430 tx=0x7f988000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.734+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880011790 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.734+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.734+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98900ac8d0 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.734+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880011bb0 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.734+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98900ad170 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.735+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98900ad370 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.735+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f988001c460 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.735+0000 7f988d7fa640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f987c06b4d0 0x7f987c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.736+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9880089f70 con 0x7f98900ce3c0 2024-04-25T10:00:19.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.736+0000 7f988ffff640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f987c06b4d0 0x7f987c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:19.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.736+0000 7f988ffff640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f987c06b4d0 0x7f987c06d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f98901f6c90 tx=0x7f9884008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:19.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.740+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98900ad370 con 0x7f98900ce3c0 2024-04-25T10:00:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:19.973+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f9890073990 con 0x7f98900ce3c0 2024-04-25T10:00:20.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.029+0000 7f988d7fa640 1 -- 172.21.15.148:0/2147632099 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v294) v1 ==== 147+0+0 (secure 0 0 0) 0x7f9880058f60 con 0x7f98900ce3c0 2024-04-25T10:00:20.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f987c06b4d0 msgr2=0x7f987c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f987c06b4d0 0x7f987c06d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f98901f6c90 tx=0x7f9884008040 comp rx=0 tx=0).stop 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 msgr2=0x7f9890146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f9890146b80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f988000c430 tx=0x7f988000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 shutdown_connections 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f987c06b4d0 0x7f987c06d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9890154ee0 0x7f98900b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98900cfec0 0x7f9890143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 --2- 172.21.15.148:0/2147632099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98900ce3c0 0x7f9890146b80 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 >> 172.21.15.148:0/2147632099 conn(0x7f989006abe0 msgr2=0x7f98900cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 shutdown_connections 2024-04-25T10:00:20.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.031+0000 7f9896bc4640 1 -- 172.21.15.148:0/2147632099 wait complete. 2024-04-25T10:00:20.047 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:00:20.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.144+0000 7f6a0d7c8640 1 Processor -- start 2024-04-25T10:00:20.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.144+0000 7f6a0d7c8640 1 -- start start 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a0d7c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a080b5620 0x7f6a080b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a0d7c8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a0806c2d0 0x7f6a0806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a0d7c8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0806cc80 con 0x7f6a0815b7d0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a0d7c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0806cf50 con 0x7f6a080b5620 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a0d7c8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a080ce530 con 0x7f6a0806c2d0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a080b5620 0x7f6a080b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a080b5620 0x7f6a080b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60836/0 (socket says 172.21.15.148:60836) 2024-04-25T10:00:20.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 -- 172.21.15.148:0/3972594220 learned_addr learned my addr 172.21.15.148:0/3972594220 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a077fe640 1 --2- 172.21.15.148:0/3972594220 >> v2:172.21.15.188:3300/0 conn(0x7f6a0806c2d0 0x7f6a0806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 -- 172.21.15.148:0/3972594220 >> v2:172.21.15.188:3300/0 conn(0x7f6a0806c2d0 msgr2=0x7f6a0806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 --2- 172.21.15.148:0/3972594220 >> v2:172.21.15.188:3300/0 conn(0x7f6a0806c2d0 0x7f6a0806c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 -- 172.21.15.148:0/3972594220 >> v1:172.21.15.188:6789/0 conn(0x7f6a0815b7d0 legacy=0x7f6a080cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.145+0000 7f6a067fc640 1 -- 172.21.15.148:0/3972594220 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080ce810 con 0x7f6a080b5620 2024-04-25T10:00:20.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.146+0000 7f6a067fc640 1 --2- 172.21.15.148:0/3972594220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080b5620 0x7f6a080b5ab0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6a08161890 tx=0x7f69fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be3f19dd48861fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.146+0000 7f6a05ffb640 1 -- 172.21.15.148:0/3972594220 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037070 con 0x7f6a080b5620 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.146+0000 7f6a05ffb640 1 -- 172.21.15.148:0/3972594220 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc032440 con 0x7f6a080b5620 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.146+0000 7f6a05ffb640 1 -- 172.21.15.148:0/3972594220 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc032810 con 0x7f6a080b5620 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/3972594220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080b5620 msgr2=0x7f6a080b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/3972594220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080b5620 0x7f6a080b5ab0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6a08161890 tx=0x7f69fc02a230 comp rx=0 tx=0).stop 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/3972594220 shutdown_connections 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/3972594220 >> v2:172.21.15.188:3300/0 conn(0x7f6a0806c2d0 0x7f6a0806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/3972594220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080b5620 0x7f6a080b5ab0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/3972594220 >> 172.21.15.148:0/3972594220 conn(0x7f6a080d0800 msgr2=0x7f6a080d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/3972594220 shutdown_connections 2024-04-25T10:00:20.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.147+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/3972594220 wait complete. 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.148+0000 7f6a0d7c8640 1 Processor -- start 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.148+0000 7f6a0d7c8640 1 -- start start 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a0d7c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0806c2d0 0x7f6a080ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a0d7c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a080b5620 0x7f6a08158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a0d7c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 0x7f6a080d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a0d7c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0815bbb0 con 0x7f6a0815b7d0 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a0d7c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0806d230 con 0x7f6a0806c2d0 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a0d7c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0815bd30 con 0x7f6a080b5620 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 0x7f6a080d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 0x7f6a080d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60846/0 (socket says 172.21.15.148:60846) 2024-04-25T10:00:20.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 -- 172.21.15.148:0/566913040 learned_addr learned my addr 172.21.15.148:0/566913040 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a06ffd640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0806c2d0 0x7f6a080ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a067fc640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a080b5620 0x7f6a08158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 -- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a080b5620 msgr2=0x7f6a08158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a080b5620 0x7f6a08158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 -- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0806c2d0 msgr2=0x7f6a080ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.149+0000 7f6a077fe640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0806c2d0 0x7f6a080ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f6a077fe640 1 -- 172.21.15.148:0/566913040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a081592c0 con 0x7f6a0815b7d0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f6a077fe640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 0x7f6a080d1cb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000cd20 tx=0x7f6a0000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00003b90 con 0x7f6a0815b7d0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00003d30 con 0x7f6a0815b7d0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a080c7920 con 0x7f6a0815b7d0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00004240 con 0x7f6a0815b7d0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.150+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a080c7ee0 con 0x7f6a0815b7d0 2024-04-25T10:00:20.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.151+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0806c740 con 0x7f6a0815b7d0 2024-04-25T10:00:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.151+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a000043e0 con 0x7f6a0815b7d0 2024-04-25T10:00:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.152+0000 7f69ef7fe640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69f406b4d0 0x7f69f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.152+0000 7f6a06ffd640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69f406b4d0 0x7f69f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.152+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a0008a8f0 con 0x7f6a0815b7d0 2024-04-25T10:00:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.152+0000 7f6a06ffd640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69f406b4d0 0x7f69f406d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006fd0 tx=0x7f69f8009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.156+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a0806c740 con 0x7f6a0815b7d0 2024-04-25T10:00:20.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.325+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6a08073a00 con 0x7f6a0815b7d0 2024-04-25T10:00:20.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.330+0000 7f69ef7fe640 1 -- 172.21.15.148:0/566913040 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a00059320 con 0x7f6a0815b7d0 2024-04-25T10:00:20.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69f406b4d0 msgr2=0x7f69f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69f406b4d0 0x7f69f406d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006fd0 tx=0x7f69f8009060 comp rx=0 tx=0).stop 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 msgr2=0x7f6a080d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 0x7f6a080d1cb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000cd20 tx=0x7f6a0000a660 comp rx=0 tx=0).stop 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 shutdown_connections 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69f406b4d0 0x7f69f406d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0815b7d0 0x7f6a080d1cb0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a080b5620 0x7f6a08158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 --2- 172.21.15.148:0/566913040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0806c2d0 0x7f6a080ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 >> 172.21.15.148:0/566913040 conn(0x7f6a080d0800 msgr2=0x7f6a0815fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 shutdown_connections 2024-04-25T10:00:20.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.332+0000 7f6a0d7c8640 1 -- 172.21.15.148:0/566913040 wait complete. 2024-04-25T10:00:20.348 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:00:20.348 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:20.390 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:20.435 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:00:20.435 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:20.435 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:20.450 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:20.450 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:00:20.450 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:20.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:20.493 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:20.493 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:20.508 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:20.508 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:00:20.508 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:20.551 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:20.551 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:00:20.551 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.644+0000 7ff719522640 1 Processor -- start 2024-04-25T10:00:20.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.645+0000 7ff719522640 1 -- start start 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.645+0000 7ff719522640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff71415dbe0 0x7ff7140d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.645+0000 7ff719522640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7140b92a0 0x7ff7140d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.645+0000 7ff719522640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7140b97a0 con 0x7ff7140b2cb0 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.645+0000 7ff719522640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71415dfc0 con 0x7ff7140b92a0 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.645+0000 7ff719522640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140d1a00 con 0x7ff71415dbe0 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff712ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff71415dbe0 0x7ff7140d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff7137fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7140b92a0 0x7ff7140d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff7127fc640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff7140b2cb0 0x7ff7140b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:52644/0 (socket says 172.21.15.148:52644) 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff7127fc640 1 -- 172.21.15.148:0/3581390656 learned_addr learned my addr 172.21.15.148:0/3581390656 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff712ffd640 1 -- 172.21.15.148:0/3581390656 >> v2:172.21.15.188:3300/0 conn(0x7ff7140b92a0 msgr2=0x7ff7140d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff712ffd640 1 --2- 172.21.15.148:0/3581390656 >> v2:172.21.15.188:3300/0 conn(0x7ff7140b92a0 0x7ff7140d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff712ffd640 1 -- 172.21.15.148:0/3581390656 >> v1:172.21.15.188:6789/0 conn(0x7ff7140b2cb0 legacy=0x7ff7140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.646+0000 7ff712ffd640 1 -- 172.21.15.148:0/3581390656 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140d1c50 con 0x7ff71415dbe0 2024-04-25T10:00:20.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.647+0000 7ff712ffd640 1 --2- 172.21.15.148:0/3581390656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff71415dbe0 0x7ff7140d0910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff704009d90 tx=0x7ff70402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c20dbbfa9f2cdd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.647+0000 7ff711ffb640 1 -- 172.21.15.148:0/3581390656 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704007740 con 0x7ff71415dbe0 2024-04-25T10:00:20.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.647+0000 7ff711ffb640 1 -- 172.21.15.148:0/3581390656 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7040078e0 con 0x7ff71415dbe0 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.647+0000 7ff711ffb640 1 -- 172.21.15.148:0/3581390656 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704007cb0 con 0x7ff71415dbe0 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.647+0000 7ff719522640 1 -- 172.21.15.148:0/3581390656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff71415dbe0 msgr2=0x7ff7140d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.647+0000 7ff719522640 1 --2- 172.21.15.148:0/3581390656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff71415dbe0 0x7ff7140d0910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff704009d90 tx=0x7ff70402a660 comp rx=0 tx=0).stop 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.648+0000 7ff719522640 1 -- 172.21.15.148:0/3581390656 shutdown_connections 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.648+0000 7ff719522640 1 --2- 172.21.15.148:0/3581390656 >> v2:172.21.15.188:3300/0 conn(0x7ff7140b92a0 0x7ff7140d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.648+0000 7ff719522640 1 --2- 172.21.15.148:0/3581390656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff71415dbe0 0x7ff7140d0910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.648+0000 7ff719522640 1 -- 172.21.15.148:0/3581390656 >> 172.21.15.148:0/3581390656 conn(0x7ff714061a20 msgr2=0x7ff714061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.648+0000 7ff719522640 1 -- 172.21.15.148:0/3581390656 shutdown_connections 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.648+0000 7ff719522640 1 -- 172.21.15.148:0/3581390656 wait complete. 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.649+0000 7ff719522640 1 Processor -- start 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.649+0000 7ff719522640 1 -- start start 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7140b2cb0 0x7ff7141480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 0x7ff7140caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff71415dbe0 0x7ff7140b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b6ab0 con 0x7ff7140b92a0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140d2210 con 0x7ff71415dbe0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7140b6c30 con 0x7ff7140b2cb0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 0x7ff7140caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 0x7ff7140caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60848/0 (socket says 172.21.15.148:60848) 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 -- 172.21.15.148:0/2575008092 learned_addr learned my addr 172.21.15.148:0/2575008092 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7137fe640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff71415dbe0 0x7ff7140b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff712ffd640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7140b2cb0 0x7ff7141480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 -- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7140b2cb0 msgr2=0x7ff7141480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7140b2cb0 0x7ff7141480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 -- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff71415dbe0 msgr2=0x7ff7140b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff71415dbe0 0x7ff7140b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 -- 172.21.15.148:0/2575008092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140cc5f0 con 0x7ff7140b92a0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff7127fc640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 0x7ff7140caa50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c3c0 tx=0x7ff70800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708017070 con 0x7ff7140b92a0 2024-04-25T10:00:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708005ce0 con 0x7ff7140b92a0 2024-04-25T10:00:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080072f0 con 0x7ff7140b92a0 2024-04-25T10:00:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7140cb550 con 0x7ff7140b92a0 2024-04-25T10:00:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.650+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7140cbb10 con 0x7ff7140b92a0 2024-04-25T10:00:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.651+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff714073990 con 0x7ff7140b92a0 2024-04-25T10:00:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.652+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff70801a070 con 0x7ff7140b92a0 2024-04-25T10:00:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.652+0000 7ff6fb7fe640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff70006b4d0 0x7ff70006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.652+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff708089380 con 0x7ff7140b92a0 2024-04-25T10:00:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.652+0000 7ff712ffd640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff70006b4d0 0x7ff70006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.653+0000 7ff712ffd640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff70006b4d0 0x7ff70006d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff704034000 tx=0x7ff70402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.656+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff708057d20 con 0x7ff7140b92a0 2024-04-25T10:00:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.862+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7140b7950 con 0x7ff7140b92a0 2024-04-25T10:00:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.862+0000 7ff6fb7fe640 1 -- 172.21.15.148:0/2575008092 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ff70805bdc0 con 0x7ff7140b92a0 2024-04-25T10:00:20.867 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:20.867 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1598609717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1598609717},{"type":"v1","addr":"172.21.15.188:6835","nonce":1598609717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/29539896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":29539896},{"type":"v1","addr":"172.21.15.188:6834","nonce":29539896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1964991678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1964991678},{"type":"v1","addr":"172.21.15.148:6836","nonce":1964991678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/88882848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":88882848},{"type":"v1","addr":"172.21.15.148:6837","nonce":88882848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[]} 2024-04-25T10:00:20.867 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 49 2024-04-25T10:00:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff70006b4d0 msgr2=0x7ff70006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff70006b4d0 0x7ff70006d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff704034000 tx=0x7ff70402f000 comp rx=0 tx=0).stop 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 msgr2=0x7ff7140caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 0x7ff7140caa50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c3c0 tx=0x7ff70800c880 comp rx=0 tx=0).stop 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 shutdown_connections 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff70006b4d0 0x7ff70006d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff71415dbe0 0x7ff7140b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7140b92a0 0x7ff7140caa50 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 --2- 172.21.15.148:0/2575008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7140b2cb0 0x7ff7141480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 >> 172.21.15.148:0/2575008092 conn(0x7ff714061a20 msgr2=0x7ff714160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 shutdown_connections 2024-04-25T10:00:20.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:20.864+0000 7ff719522640 1 -- 172.21.15.148:0/2575008092 wait complete. 2024-04-25T10:00:20.880 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:00:20.880 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:00:20.880 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:00:20.880 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:20.881 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:00:20.881 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:00:20.881 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:00:20.881 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:20.881 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:00:20.881 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:00:20.882 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:00:20.882 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:20.882 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:00:20.882 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:00:20.882 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:00:20.882 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:00:20.889 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:00:20.889 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:00:20.890 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:00:20.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:00:20.893 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:00:20.894 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:00:20.894 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:00:20.895 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:00:20.933 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:00:20.933 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:20.934 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:00:20.934 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:20.937 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:00:20.937 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:20.937 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:00:20.938 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:20.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:00:21.005 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:00:21.006 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:00:21.016 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:00:21.016 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:00:21.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.037+0000 7f76d60e1640 1 Processor -- start 2024-04-25T10:00:21.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.037+0000 7f76d60e1640 1 -- start start 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76d60e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d01610f0 0x7f76d006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76d60e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76d01616f0 0x7f76d0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76d60e1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d00c6960 con 0x7f76d00ce0a0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76d60e1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d01614d0 con 0x7f76d01616f0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76d60e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0162260 con 0x7f76d01610f0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76cffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f76d00ce0a0 0x7f76d00d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57256/0 (socket says 172.21.15.148:57256) 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76cffff640 1 -- 172.21.15.148:0/3992772250 learned_addr learned my addr 172.21.15.148:0/3992772250 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76ceffd640 1 --2- 172.21.15.148:0/3992772250 >> v2:172.21.15.188:3300/0 conn(0x7f76d01616f0 0x7f76d0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.038+0000 7f76cf7fe640 1 --2- 172.21.15.148:0/3992772250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d01610f0 0x7f76d006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76ce7fc640 1 -- 172.21.15.148:0/3992772250 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554522947 0 0) 0x7f76d00c6960 con 0x7f76d00ce0a0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76ce7fc640 1 -- 172.21.15.148:0/3992772250 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b8003520 con 0x7f76d00ce0a0 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76cf7fe640 1 -- 172.21.15.148:0/3992772250 >> v2:172.21.15.188:3300/0 conn(0x7f76d01616f0 msgr2=0x7f76d0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:21.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76cf7fe640 1 --2- 172.21.15.148:0/3992772250 >> v2:172.21.15.188:3300/0 conn(0x7f76d01616f0 0x7f76d0161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76cf7fe640 1 -- 172.21.15.148:0/3992772250 >> v1:172.21.15.148:6789/0 conn(0x7f76d00ce0a0 legacy=0x7f76d00d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76cf7fe640 1 -- 172.21.15.148:0/3992772250 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0162540 con 0x7f76d01610f0 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.039+0000 7f76cf7fe640 1 --2- 172.21.15.148:0/3992772250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d01610f0 0x7f76d006c1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76c00097a0 tx=0x7f76c002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4359fed88598931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76ce7fc640 1 -- 172.21.15.148:0/3992772250 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037070 con 0x7f76d01610f0 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76ce7fc640 1 -- 172.21.15.148:0/3992772250 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c0032440 con 0x7f76d01610f0 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76ce7fc640 1 -- 172.21.15.148:0/3992772250 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0032810 con 0x7f76d01610f0 2024-04-25T10:00:21.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 -- 172.21.15.148:0/3992772250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d01610f0 msgr2=0x7f76d006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 --2- 172.21.15.148:0/3992772250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d01610f0 0x7f76d006c1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76c00097a0 tx=0x7f76c002a180 comp rx=0 tx=0).stop 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 -- 172.21.15.148:0/3992772250 shutdown_connections 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 --2- 172.21.15.148:0/3992772250 >> v2:172.21.15.188:3300/0 conn(0x7f76d01616f0 0x7f76d0161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 --2- 172.21.15.148:0/3992772250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d01610f0 0x7f76d006c1c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 -- 172.21.15.148:0/3992772250 >> 172.21.15.148:0/3992772250 conn(0x7f76d006f500 msgr2=0x7f76d006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 -- 172.21.15.148:0/3992772250 shutdown_connections 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.040+0000 7f76d60e1640 1 -- 172.21.15.148:0/3992772250 wait complete. 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.041+0000 7f76d60e1640 1 Processor -- start 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.042+0000 7f76d60e1640 1 -- start start 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.042+0000 7f76d60e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d00ce0a0 0x7f76d01483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.042+0000 7f76d60e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d01610f0 0x7f76d00c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.042+0000 7f76d60e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 0x7f76d00c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76d60e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d00cf690 con 0x7f76d01616f0 2024-04-25T10:00:21.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 0x7f76d00c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76d60e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d00cfce0 con 0x7f76d01610f0 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cf7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d00ce0a0 0x7f76d01483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76d60e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d006c700 con 0x7f76d00ce0a0 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 0x7f76d00c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60886/0 (socket says 172.21.15.148:60886) 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 -- 172.21.15.148:0/2889168441 learned_addr learned my addr 172.21.15.148:0/2889168441 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76ceffd640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d01610f0 0x7f76d00c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 -- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d00ce0a0 msgr2=0x7f76d01483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d00ce0a0 0x7f76d01483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 -- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d01610f0 msgr2=0x7f76d00c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d01610f0 0x7f76d00c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 -- 172.21.15.148:0/2889168441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d00ad460 con 0x7f76d01616f0 2024-04-25T10:00:21.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.043+0000 7f76cffff640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 0x7f76d00c97b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f76c800ca20 tx=0x7f76c800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.046+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8017070 con 0x7f76d01616f0 2024-04-25T10:00:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.046+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d00ad740 con 0x7f76d01616f0 2024-04-25T10:00:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.046+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d00add00 con 0x7f76d01616f0 2024-04-25T10:00:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.046+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c800abd0 con 0x7f76d01616f0 2024-04-25T10:00:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.046+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8021060 con 0x7f76d01616f0 2024-04-25T10:00:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.047+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b0005290 con 0x7f76d01616f0 2024-04-25T10:00:21.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.048+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76c801a070 con 0x7f76d01616f0 2024-04-25T10:00:21.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.048+0000 7f76b7fff640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76bc06b4d0 0x7f76bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:21.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.048+0000 7f76cf7fe640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76bc06b4d0 0x7f76bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:21.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.048+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f76c8089c20 con 0x7f76d01616f0 2024-04-25T10:00:21.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.049+0000 7f76cf7fe640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76bc06b4d0 0x7f76bc06d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f76c0034000 tx=0x7f76c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:21.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.053+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76c8058650 con 0x7f76d01616f0 2024-04-25T10:00:21.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:21.237+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f76b0005850 con 0x7f76d01616f0 2024-04-25T10:00:22.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.066+0000 7f76b7fff640 1 -- 172.21.15.148:0/2889168441 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f76c805c6f0 con 0x7f76d01616f0 2024-04-25T10:00:22.071 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:00:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76bc06b4d0 msgr2=0x7f76bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76bc06b4d0 0x7f76bc06d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f76c0034000 tx=0x7f76c002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 msgr2=0x7f76d00c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 0x7f76d00c97b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f76c800ca20 tx=0x7f76c800cee0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 shutdown_connections 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76bc06b4d0 0x7f76bc06d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76d01616f0 0x7f76d00c97b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d01610f0 0x7f76d00c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 --2- 172.21.15.148:0/2889168441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76d00ce0a0 0x7f76d01483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 >> 172.21.15.148:0/2889168441 conn(0x7f76d006f500 msgr2=0x7f76d0071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 shutdown_connections 2024-04-25T10:00:22.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.069+0000 7f76d60e1640 1 -- 172.21.15.148:0/2889168441 wait complete. 2024-04-25T10:00:22.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:00:22.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.181+0000 7f468f36a640 1 Processor -- start 2024-04-25T10:00:22.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.181+0000 7f468f36a640 1 -- start start 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468f36a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 0x7f46881625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468f36a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880cce50 con 0x7f468815dc90 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468f36a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4688161af0 con 0x7f46880cc8e0 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468f36a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4688161dd0 con 0x7f46880d1f20 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468d106640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f468815dc90 0x7f468815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57262/0 (socket says 172.21.15.148:57262) 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468d106640 1 -- 172.21.15.148:0/266579621 learned_addr learned my addr 172.21.15.148:0/266579621 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f468d907640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 0x7f46881625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264722310 0 0) 0x7f46880cce50 con 0x7f468815dc90 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4674003520 con 0x7f468815dc90 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769904728 0 0) 0x7f4674003520 con 0x7f468815dc90 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 msgr2=0x7f46881625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 0x7f46881625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 >> v1:172.21.15.188:6789/0 conn(0x7f46880cc8e0 legacy=0x7f46880d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46881620b0 con 0x7f468815dc90 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4684001670 con 0x7f468815dc90 2024-04-25T10:00:22.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 >> v1:172.21.15.148:6789/0 conn(0x7f468815dc90 legacy=0x7f468815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.182+0000 7f467ffff640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 0x7f46740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f467ffff640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674003f60 0x7f4674004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f467ffff640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4674004d40 0x7f46740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4688161dd0 con 0x7f4674004d40 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467400bb50 con 0x7f4674003f60 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467400bd50 con 0x7f46880d1f20 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468c905640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674003f60 0x7f4674004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d907640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4674004d40 0x7f46740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d106640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 0x7f46740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d907640 1 -- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 msgr2=0x7f46740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d907640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880d1f20 0x7f46740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d907640 1 -- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674003f60 msgr2=0x7f4674004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d907640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674003f60 0x7f4674004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.183+0000 7f468d907640 1 -- 172.21.15.148:0/266579621 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674003520 con 0x7f4674004d40 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468d907640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4674004d40 0x7f46740051c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4680009f90 tx=0x7f4674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169a32dff09bad96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680009c10 con 0x7f4674004d40 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4680036070 con 0x7f4674004d40 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f467ffff640 1 -- 172.21.15.148:0/266579621 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680031850 con 0x7f4674004d40 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468f36a640 1 -- 172.21.15.148:0/266579621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4674004d40 msgr2=0x7f46740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468f36a640 1 --2- 172.21.15.148:0/266579621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4674004d40 0x7f46740051c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4680009f90 tx=0x7f4674005830 comp rx=0 tx=0).stop 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468d907640 1 -- 172.21.15.148:0/266579621 reap_dead start 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468f36a640 1 -- 172.21.15.148:0/266579621 shutdown_connections 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468f36a640 1 -- 172.21.15.148:0/266579621 >> 172.21.15.148:0/266579621 conn(0x7f468806f590 msgr2=0x7f468806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468f36a640 1 -- 172.21.15.148:0/266579621 shutdown_connections 2024-04-25T10:00:22.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.184+0000 7f468f36a640 1 -- 172.21.15.148:0/266579621 wait complete. 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 Processor -- start 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 -- start start 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674004d40 0x7f468814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880b34a0 0x7f46880c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 0x7f46880c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468815e1f0 con 0x7f46880b3920 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468815e070 con 0x7f4674004d40 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468f36a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46881625d0 con 0x7f46880b34a0 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468d907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 0x7f46880c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468d907640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 0x7f46880c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60908/0 (socket says 172.21.15.148:60908) 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.186+0000 7f468d907640 1 -- 172.21.15.148:0/3470090796 learned_addr learned my addr 172.21.15.148:0/3470090796 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468c905640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880b34a0 0x7f46880c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d106640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674004d40 0x7f468814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d907640 1 -- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880b34a0 msgr2=0x7f46880c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d907640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880b34a0 0x7f46880c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d907640 1 -- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674004d40 msgr2=0x7f468814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d907640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674004d40 0x7f468814dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d907640 1 -- 172.21.15.148:0/3470090796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688159470 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468d907640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 0x7f46880c8fb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4680033040 tx=0x7f4680009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680009c10 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4680003990 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46880caee0 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680003d80 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.187+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46880cb4a0 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.188+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4688073990 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.189+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4680005d10 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.189+0000 7f467dffb640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b500 0x7f466c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.189+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f46800ab740 con 0x7f46880b3920 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.189+0000 7f468d106640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b500 0x7f466c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.189+0000 7f468d106640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b500 0x7f466c06d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f46880cba70 tx=0x7f4684009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.193+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f468004a060 con 0x7f46880b3920 2024-04-25T10:00:22.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.427+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4688067df0 con 0x7f46880b3920 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.428+0000 7f467dffb640 1 -- 172.21.15.148:0/3470090796 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f468002e050 con 0x7f46880b3920 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:00:22.432 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:00:22.433 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.434 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:00:22.435 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.436 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.437 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.438 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.439 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:00:22.440 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:00:22.441 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:00:22.442 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.443 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:00:22.444 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:00:22.445 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.429+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b500 msgr2=0x7f466c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.429+0000 7f468f36a640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b500 0x7f466c06d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f46880cba70 tx=0x7f4684009040 comp rx=0 tx=0).stop 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 msgr2=0x7f46880c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 0x7f46880c8fb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4680033040 tx=0x7f4680009750 comp rx=0 tx=0).stop 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 shutdown_connections 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f466c06b500 0x7f466c06d9c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46880b3920 0x7f46880c8fb0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46880b34a0 0x7f46880c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 --2- 172.21.15.148:0/3470090796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674004d40 0x7f468814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 >> 172.21.15.148:0/3470090796 conn(0x7f468806f590 msgr2=0x7f46880d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 shutdown_connections 2024-04-25T10:00:22.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.430+0000 7f468f36a640 1 -- 172.21.15.148:0/3470090796 wait complete. 2024-04-25T10:00:22.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.541+0000 7fd2ff093640 1 Processor -- start 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.541+0000 7fd2ff093640 1 -- start start 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2ff093640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2f81602e0 0x7fd2f80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2ff093640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2ff093640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f80cd710 con 0x7fd2f815ba00 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2ff093640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f8099b70 con 0x7fd2f81602e0 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2ff093640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8099e50 con 0x7fd2f815f5f0 2024-04-25T10:00:22.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2f7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd2f81602e0 0x7fd2f80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2fce2f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd2f815ba00 0x7fd2f809a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57268/0 (socket says 172.21.15.148:57268) 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.542+0000 7fd2fce2f640 1 -- 172.21.15.148:0/807890415 learned_addr learned my addr 172.21.15.148:0/807890415 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2fd630640 1 --2- 172.21.15.148:0/807890415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f7fff640 1 -- 172.21.15.148:0/807890415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 msgr2=0x7fd2f815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f7fff640 1 --2- 172.21.15.148:0/807890415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f7fff640 1 -- 172.21.15.148:0/807890415 >> v1:172.21.15.148:6789/0 conn(0x7fd2f815ba00 legacy=0x7fd2f809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f77fe640 1 -- 172.21.15.148:0/807890415 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986911311 0 0) 0x7fd2e8001110 con 0x7fd2f815ba00 2024-04-25T10:00:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f7fff640 1 -- 172.21.15.148:0/807890415 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f809a130 con 0x7fd2f81602e0 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f7fff640 1 --2- 172.21.15.148:0/807890415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f81602e0 0x7fd2f80ccda0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097a0 tx=0x7fd2ec02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725a64f4cfcc4b5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.543+0000 7fd2f77fe640 1 -- 172.21.15.148:0/807890415 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec037070 con 0x7fd2f81602e0 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2f77fe640 1 -- 172.21.15.148:0/807890415 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec02adb0 con 0x7fd2f81602e0 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2f77fe640 1 -- 172.21.15.148:0/807890415 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec032660 con 0x7fd2f81602e0 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 -- 172.21.15.148:0/807890415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f81602e0 msgr2=0x7fd2f80ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 --2- 172.21.15.148:0/807890415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f81602e0 0x7fd2f80ccda0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0097a0 tx=0x7fd2ec02a060 comp rx=0 tx=0).stop 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 -- 172.21.15.148:0/807890415 shutdown_connections 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 --2- 172.21.15.148:0/807890415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 --2- 172.21.15.148:0/807890415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f81602e0 0x7fd2f80ccda0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 -- 172.21.15.148:0/807890415 >> 172.21.15.148:0/807890415 conn(0x7fd2f80b3270 msgr2=0x7fd2f80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 -- 172.21.15.148:0/807890415 shutdown_connections 2024-04-25T10:00:22.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.544+0000 7fd2ff093640 1 -- 172.21.15.148:0/807890415 wait complete. 2024-04-25T10:00:22.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.545+0000 7fd2ff093640 1 Processor -- start 2024-04-25T10:00:22.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.545+0000 7fd2ff093640 1 -- start start 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2ff093640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 0x7fd2f81575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2ff093640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2ff093640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f81602e0 0x7fd2f8070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2ff093640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80cd460 con 0x7fd2f815ba00 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2ff093640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80cd2e0 con 0x7fd2f81602e0 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2ff093640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f80c52d0 con 0x7fd2f815f5f0 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2fce2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 0x7fd2f81575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2fce2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 0x7fd2f81575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60916/0 (socket says 172.21.15.148:60916) 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.546+0000 7fd2fce2f640 1 -- 172.21.15.148:0/2222896510 learned_addr learned my addr 172.21.15.148:0/2222896510 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fd630640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f81602e0 0x7fd2f8070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2f7fff640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fce2f640 1 -- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 msgr2=0x7fd2f815e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fce2f640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fce2f640 1 -- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f81602e0 msgr2=0x7fd2f8070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fce2f640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f81602e0 0x7fd2f8070940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fce2f640 1 -- 172.21.15.148:0/2222896510 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8157b20 con 0x7fd2f815ba00 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2fce2f640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 0x7fd2f81575e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c780 tx=0x7fd2e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8018070 con 0x7fd2f815ba00 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e80072d0 con 0x7fd2f815ba00 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80047e0 con 0x7fd2f815ba00 2024-04-25T10:00:22.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f80c9da0 con 0x7fd2f815ba00 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.547+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f806be90 con 0x7fd2f815ba00 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.548+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f8073990 con 0x7fd2f815ba00 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.549+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2e801b070 con 0x7fd2f815ba00 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.549+0000 7fd2f57fa640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd2e406b4d0 0x7fd2e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.549+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd2e8089290 con 0x7fd2f815ba00 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.549+0000 7fd2f7fff640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd2e406b4d0 0x7fd2e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.549+0000 7fd2f7fff640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd2e406b4d0 0x7fd2e406d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec034000 tx=0x7fd2ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.553+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2f80cd710 con 0x7fd2f815ba00 2024-04-25T10:00:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.724+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2f80b40a0 con 0x7fd2f815ba00 2024-04-25T10:00:22.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.724+0000 7fd2f57fa640 1 -- 172.21.15.148:0/2222896510 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13213 (secure 0 0 0) 0x7fd2e8058360 con 0x7fd2f815ba00 2024-04-25T10:00:22.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:22.729 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:22.059422+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.726+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd2e406b4d0 msgr2=0x7fd2e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.726+0000 7fd2ff093640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd2e406b4d0 0x7fd2e406d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec034000 tx=0x7fd2ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.726+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 msgr2=0x7fd2f81575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.726+0000 7fd2ff093640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 0x7fd2f81575e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c780 tx=0x7fd2e800cc40 comp rx=0 tx=0).stop 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 shutdown_connections 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd2e406b4d0 0x7fd2e406d990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2f81602e0 0x7fd2f8070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f815f5f0 0x7fd2f815e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 --2- 172.21.15.148:0/2222896510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2f815ba00 0x7fd2f81575e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 >> 172.21.15.148:0/2222896510 conn(0x7fd2f80b3270 msgr2=0x7fd2f80c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 shutdown_connections 2024-04-25T10:00:22.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.727+0000 7fd2ff093640 1 -- 172.21.15.148:0/2222896510 wait complete. 2024-04-25T10:00:22.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:22.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:22.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.838+0000 7fe84c41c640 1 Processor -- start 2024-04-25T10:00:22.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.838+0000 7fe84c41c640 1 -- start start 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84c41c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe84415dc20 0x7fe8440ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84c41c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8440d32f0 0x7fe8440cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84c41c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84415e000 con 0x7fe844161710 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84c41c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8440d1de0 con 0x7fe8440d32f0 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84c41c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe844161c60 con 0x7fe84415dc20 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84a9b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8440d32f0 0x7fe8440cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84a9b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8440d32f0 0x7fe8440cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60924/0 (socket says 172.21.15.148:60924) 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.839+0000 7fe84a9b9640 1 -- 172.21.15.148:0/3879595656 learned_addr learned my addr 172.21.15.148:0/3879595656 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe84a1b8640 1 --2- 172.21.15.148:0/3879595656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe84415dc20 0x7fe8440ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe84a9b9640 1 -- 172.21.15.148:0/3879595656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe84415dc20 msgr2=0x7fe8440ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe84a9b9640 1 --2- 172.21.15.148:0/3879595656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe84415dc20 0x7fe8440ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe84a9b9640 1 -- 172.21.15.148:0/3879595656 >> v1:172.21.15.148:6789/0 conn(0x7fe844161710 legacy=0x7fe84406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe8491b6640 1 -- 172.21.15.148:0/3879595656 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263808052 0 0) 0x7fe84415e000 con 0x7fe844161710 2024-04-25T10:00:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe84a9b9640 1 -- 172.21.15.148:0/3879595656 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844161f40 con 0x7fe8440d32f0 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.840+0000 7fe84a9b9640 1 --2- 172.21.15.148:0/3879595656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe8440cebb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009870 tx=0x7fe83c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d87e4be13927f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe8491b6640 1 -- 172.21.15.148:0/3879595656 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c037070 con 0x7fe8440d32f0 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe8491b6640 1 -- 172.21.15.148:0/3879595656 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c032440 con 0x7fe8440d32f0 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe8491b6640 1 -- 172.21.15.148:0/3879595656 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c032810 con 0x7fe8440d32f0 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 -- 172.21.15.148:0/3879595656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 msgr2=0x7fe8440cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 --2- 172.21.15.148:0/3879595656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe8440cebb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009870 tx=0x7fe83c02a270 comp rx=0 tx=0).stop 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 -- 172.21.15.148:0/3879595656 shutdown_connections 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 --2- 172.21.15.148:0/3879595656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe8440cebb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 --2- 172.21.15.148:0/3879595656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe84415dc20 0x7fe8440ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 -- 172.21.15.148:0/3879595656 >> 172.21.15.148:0/3879595656 conn(0x7fe8440d0420 msgr2=0x7fe8440d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 -- 172.21.15.148:0/3879595656 shutdown_connections 2024-04-25T10:00:22.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.841+0000 7fe84c41c640 1 -- 172.21.15.148:0/3879595656 wait complete. 2024-04-25T10:00:22.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.842+0000 7fe84c41c640 1 Processor -- start 2024-04-25T10:00:22.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.842+0000 7fe84c41c640 1 -- start start 2024-04-25T10:00:22.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84c41c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe844159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84c41c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe84415dc20 0x7fe8440b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84c41c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe844161710 0x7fe8440ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84c41c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440b5e50 con 0x7fe8440d32f0 2024-04-25T10:00:22.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84c41c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe844162500 con 0x7fe84415dc20 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84c41c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8440b5fd0 con 0x7fe844161710 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84a1b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe844159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84a1b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe844159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60934/0 (socket says 172.21.15.148:60934) 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84a1b8640 1 -- 172.21.15.148:0/1208547303 learned_addr learned my addr 172.21.15.148:0/1208547303 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe84a9b9640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe844161710 0x7fe8440ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.843+0000 7fe8499b7640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe84415dc20 0x7fe8440b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a1b8640 1 -- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe844161710 msgr2=0x7fe8440ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a1b8640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe844161710 0x7fe8440ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a1b8640 1 -- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe84415dc20 msgr2=0x7fe8440b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a1b8640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe84415dc20 0x7fe8440b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a1b8640 1 -- 172.21.15.148:0/1208547303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84415e2b0 con 0x7fe8440d32f0 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a9b9640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe844161710 0x7fe8440ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84a1b8640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe844159c50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe840007e40 tx=0x7fe84000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840014620 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe840019070 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840014a00 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84415e590 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.844+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8440b25e0 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.845+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe844073a00 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.845+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe840014ba0 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.846+0000 7fe83affd640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe82806b4d0 0x7fe82806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.846+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe84008a870 con 0x7fe8440d32f0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.846+0000 7fe8499b7640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe82806b4d0 0x7fe82806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.846+0000 7fe8499b7640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe82806b4d0 0x7fe82806d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe834007fd0 tx=0x7fe834009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:22.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:22.850+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe840059940 con 0x7fe8440d32f0 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.018+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8440d16b0 con 0x7fe8440d32f0 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.019+0000 7fe83affd640 1 -- 172.21.15.148:0/1208547303 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe840059240 con 0x7fe8440d32f0 2024-04-25T10:00:23.023 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:00:23.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe82806b4d0 msgr2=0x7fe82806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe82806b4d0 0x7fe82806d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe834007fd0 tx=0x7fe834009040 comp rx=0 tx=0).stop 2024-04-25T10:00:23.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 msgr2=0x7fe844159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe844159c50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe840007e40 tx=0x7fe84000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 shutdown_connections 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe82806b4d0 0x7fe82806d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe844161710 0x7fe8440ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe84415dc20 0x7fe8440b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 --2- 172.21.15.148:0/1208547303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8440d32f0 0x7fe844159c50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.020+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 >> 172.21.15.148:0/1208547303 conn(0x7fe8440d0420 msgr2=0x7fe8440b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.021+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 shutdown_connections 2024-04-25T10:00:23.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.021+0000 7fe84c41c640 1 -- 172.21.15.148:0/1208547303 wait complete. 2024-04-25T10:00:23.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:00:23.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.129+0000 7fdcd2f22640 1 Processor -- start 2024-04-25T10:00:23.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.129+0000 7fdcd2f22640 1 -- start start 2024-04-25T10:00:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd2f22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc1602e0 0x7fdccc0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd2f22640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdccc15f5f0 0x7fdccc15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd2f22640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0cd710 con 0x7fdccc15ba00 2024-04-25T10:00:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd2f22640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc099b70 con 0x7fdccc15f5f0 2024-04-25T10:00:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd2f22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc099e50 con 0x7fdccc1602e0 2024-04-25T10:00:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd0cbe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fdccc15ba00 0x7fdccc09a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:52672/0 (socket says 172.21.15.148:52672) 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd0cbe640 1 -- 172.21.15.148:0/3951203625 learned_addr learned my addr 172.21.15.148:0/3951203625 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdccbfff640 1 --2- 172.21.15.148:0/3951203625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc1602e0 0x7fdccc0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.130+0000 7fdcd14bf640 1 --2- 172.21.15.148:0/3951203625 >> v2:172.21.15.188:3300/0 conn(0x7fdccc15f5f0 0x7fdccc15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.131+0000 7fdccb7fe640 1 -- 172.21.15.148:0/3951203625 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825840963 0 0) 0x7fdcbc001110 con 0x7fdccc15ba00 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.131+0000 7fdccb7fe640 1 -- 172.21.15.148:0/3951203625 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb4003520 con 0x7fdccc15ba00 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.131+0000 7fdccbfff640 1 -- 172.21.15.148:0/3951203625 >> v2:172.21.15.188:3300/0 conn(0x7fdccc15f5f0 msgr2=0x7fdccc15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.131+0000 7fdccbfff640 1 --2- 172.21.15.148:0/3951203625 >> v2:172.21.15.188:3300/0 conn(0x7fdccc15f5f0 0x7fdccc15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.131+0000 7fdccbfff640 1 -- 172.21.15.148:0/3951203625 >> v1:172.21.15.188:6789/0 conn(0x7fdccc15ba00 legacy=0x7fdccc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.131+0000 7fdccbfff640 1 -- 172.21.15.148:0/3951203625 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc09a130 con 0x7fdccc1602e0 2024-04-25T10:00:23.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdccbfff640 1 --2- 172.21.15.148:0/3951203625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc1602e0 0x7fdccc0ccda0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00097a0 tx=0x7fdcc002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe211882656889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdccb7fe640 1 -- 172.21.15.148:0/3951203625 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0037070 con 0x7fdccc1602e0 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdccb7fe640 1 -- 172.21.15.148:0/3951203625 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc002adb0 con 0x7fdccc1602e0 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdccb7fe640 1 -- 172.21.15.148:0/3951203625 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0032640 con 0x7fdccc1602e0 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 -- 172.21.15.148:0/3951203625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc1602e0 msgr2=0x7fdccc0ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/3951203625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc1602e0 0x7fdccc0ccda0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00097a0 tx=0x7fdcc002a060 comp rx=0 tx=0).stop 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 -- 172.21.15.148:0/3951203625 shutdown_connections 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/3951203625 >> v2:172.21.15.188:3300/0 conn(0x7fdccc15f5f0 0x7fdccc15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/3951203625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc1602e0 0x7fdccc0ccda0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 -- 172.21.15.148:0/3951203625 >> 172.21.15.148:0/3951203625 conn(0x7fdccc0b3270 msgr2=0x7fdccc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 -- 172.21.15.148:0/3951203625 shutdown_connections 2024-04-25T10:00:23.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.132+0000 7fdcd2f22640 1 -- 172.21.15.148:0/3951203625 wait complete. 2024-04-25T10:00:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.133+0000 7fdcd2f22640 1 Processor -- start 2024-04-25T10:00:23.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.133+0000 7fdcd2f22640 1 -- start start 2024-04-25T10:00:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.134+0000 7fdcd2f22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc15ba00 0x7fdccc148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.134+0000 7fdcd2f22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 0x7fdccc15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.134+0000 7fdcd2f22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdccc1602e0 0x7fdccc0c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.134+0000 7fdcd2f22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0cd460 con 0x7fdccc15f5f0 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.134+0000 7fdcd2f22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0cd2e0 con 0x7fdccc1602e0 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.134+0000 7fdcd2f22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0c52d0 con 0x7fdccc15ba00 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 0x7fdccc15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 0x7fdccc15d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60948/0 (socket says 172.21.15.148:60948) 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 -- 172.21.15.148:0/2274280497 learned_addr learned my addr 172.21.15.148:0/2274280497 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcd0cbe640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc15ba00 0x7fdccc148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcd14bf640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdccc1602e0 0x7fdccc0c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 -- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc15ba00 msgr2=0x7fdccc148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc15ba00 0x7fdccc148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 -- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdccc1602e0 msgr2=0x7fdccc0c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdccc1602e0 0x7fdccc0c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 -- 172.21.15.148:0/2274280497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc15df60 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcd14bf640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdccc1602e0 0x7fdccc0c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdccbfff640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 0x7fdccc15d5f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0035040 tx=0x7fdcc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0041020 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc003c670 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc15e240 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0037040 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.135+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc15e800 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.136+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc073990 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.137+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcc0044020 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.137+0000 7fdcc97fa640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcb806b4d0 0x7fdcb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.137+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdcc00aa660 con 0x7fdccc15f5f0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.137+0000 7fdcd0cbe640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcb806b4d0 0x7fdcb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.138+0000 7fdcd0cbe640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcb806b4d0 0x7fdcb806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005880 tx=0x7fdcbc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.141+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcc004b030 con 0x7fdccc15f5f0 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.315+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdccc0cc620 con 0x7fdccc15f5f0 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.316+0000 7fdcc97fa640 1 -- 172.21.15.148:0/2274280497 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdcc00796a0 con 0x7fdccc15f5f0 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:00:23.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:23.321 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:00:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcb806b4d0 msgr2=0x7fdcb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcb806b4d0 0x7fdcb806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005880 tx=0x7fdcbc009040 comp rx=0 tx=0).stop 2024-04-25T10:00:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 msgr2=0x7fdccc15d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 0x7fdccc15d5f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0035040 tx=0x7fdcc00046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 shutdown_connections 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdcb806b4d0 0x7fdcb806d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdccc1602e0 0x7fdccc0c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdccc15f5f0 0x7fdccc15d5f0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 --2- 172.21.15.148:0/2274280497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdccc15ba00 0x7fdccc148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 >> 172.21.15.148:0/2274280497 conn(0x7fdccc0b3270 msgr2=0x7fdccc070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 shutdown_connections 2024-04-25T10:00:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.318+0000 7fdcd2f22640 1 -- 172.21.15.148:0/2274280497 wait complete. 2024-04-25T10:00:23.333 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:00:23.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:00:23.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.430+0000 7f56e65c8640 1 Processor -- start 2024-04-25T10:00:23.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.430+0000 7f56e65c8640 1 -- start start 2024-04-25T10:00:23.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56e65c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e0070dc0 0x7f56e006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56e65c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e00d1770 con 0x7f56e00711a0 2024-04-25T10:00:23.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56e65c8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e0162740 con 0x7f56e015deb0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56e65c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00b0eb0 con 0x7f56e0070dc0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56df7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f56e00711a0 0x7f56e00b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57278/0 (socket says 172.21.15.148:57278) 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56df7fe640 1 -- 172.21.15.148:0/1319286448 learned_addr learned my addr 172.21.15.148:0/1319286448 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.431+0000 7f56dffff640 1 --2- 172.21.15.148:0/1319286448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e0070dc0 0x7f56e006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546572949 0 0) 0x7f56e00d1770 con 0x7f56e00711a0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c8003520 con 0x7f56e00711a0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697913637 0 0) 0x7f56e0162740 con 0x7f56e015deb0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e00d1770 con 0x7f56e015deb0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353851464 0 0) 0x7f56c8003520 con 0x7f56e00711a0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56dffff640 1 -- 172.21.15.148:0/1319286448 >> v1:172.21.15.188:6789/0 conn(0x7f56e015deb0 legacy=0x7f56e015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56dffff640 1 -- 172.21.15.148:0/1319286448 >> v1:172.21.15.148:6789/0 conn(0x7f56e00711a0 legacy=0x7f56e00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.432+0000 7f56dffff640 1 -- 172.21.15.148:0/1319286448 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e0161980 con 0x7f56e0070dc0 2024-04-25T10:00:23.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56dffff640 1 --2- 172.21.15.148:0/1319286448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e0070dc0 0x7f56e006ccd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f56d00097d0 tx=0x7f56d002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71b591fac869063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0037070 con 0x7f56e0070dc0 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d002ab10 con 0x7f56e0070dc0 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56deffd640 1 -- 172.21.15.148:0/1319286448 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4001670 con 0x7f56e0070dc0 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 -- 172.21.15.148:0/1319286448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e0070dc0 msgr2=0x7f56e006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 --2- 172.21.15.148:0/1319286448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e0070dc0 0x7f56e006ccd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f56d00097d0 tx=0x7f56d002a080 comp rx=0 tx=0).stop 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 -- 172.21.15.148:0/1319286448 shutdown_connections 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 --2- 172.21.15.148:0/1319286448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e0070dc0 0x7f56e006ccd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 -- 172.21.15.148:0/1319286448 >> 172.21.15.148:0/1319286448 conn(0x7f56e00b3530 msgr2=0x7f56e00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 -- 172.21.15.148:0/1319286448 shutdown_connections 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.433+0000 7f56e65c8640 1 -- 172.21.15.148:0/1319286448 wait complete. 2024-04-25T10:00:23.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 Processor -- start 2024-04-25T10:00:23.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 -- start start 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 0x7f56e0152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e00711a0 0x7f56e015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e015deb0 0x7f56e0153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00b10f0 con 0x7f56e0070dc0 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e01622b0 con 0x7f56e015deb0 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.435+0000 7f56e65c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e0161f40 con 0x7f56e00711a0 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 0x7f56e0152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 0x7f56e0152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60952/0 (socket says 172.21.15.148:60952) 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 -- 172.21.15.148:0/2881418371 learned_addr learned my addr 172.21.15.148:0/2881418371 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56e4b65640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e015deb0 0x7f56e0153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56df7fe640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e00711a0 0x7f56e015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 -- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e00711a0 msgr2=0x7f56e015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e00711a0 0x7f56e015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 -- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e015deb0 msgr2=0x7f56e0153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e015deb0 0x7f56e0153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 -- 172.21.15.148:0/2881418371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e0156210 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56e4b65640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e015deb0 0x7f56e0153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56df7fe640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e00711a0 0x7f56e015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dffff640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 0x7f56e0152570 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f56d0035040 tx=0x7f56d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0037050 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0043070 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e01564f0 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d003ca30 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.436+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e0156ab0 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.437+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e0073a00 con 0x7f56e0070dc0 2024-04-25T10:00:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.438+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56d003cbd0 con 0x7f56e0070dc0 2024-04-25T10:00:23.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.438+0000 7f56dcff9640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:23.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.438+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f56d00aae40 con 0x7f56e0070dc0 2024-04-25T10:00:23.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.438+0000 7f56df7fe640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:23.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.439+0000 7f56df7fe640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f56d4001380 tx=0x7f56d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:23.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.442+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56d0079fa0 con 0x7f56e0070dc0 2024-04-25T10:00:23.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:23.612+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f56e00b2510 con 0x7f56e0070dc0 2024-04-25T10:00:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.072+0000 7f56dcff9640 1 -- 172.21.15.148:0/2881418371 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) v1 ==== 138+0+0 (secure 0 0 0) 0x7f56d00798a0 con 0x7f56e0070dc0 2024-04-25T10:00:24.077 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:00:24.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56cc06b4d0 msgr2=0x7f56cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f56d4001380 tx=0x7f56d4009040 comp rx=0 tx=0).stop 2024-04-25T10:00:24.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 msgr2=0x7f56e0152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 0x7f56e0152570 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f56d0035040 tx=0x7f56d00046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 shutdown_connections 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56e015deb0 0x7f56e0153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56e00711a0 0x7f56e015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 --2- 172.21.15.148:0/2881418371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56e0070dc0 0x7f56e0152570 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 >> 172.21.15.148:0/2881418371 conn(0x7f56e00b3530 msgr2=0x7f56e015d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.074+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 shutdown_connections 2024-04-25T10:00:24.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.075+0000 7f56e65c8640 1 -- 172.21.15.148:0/2881418371 wait complete. 2024-04-25T10:00:24.094 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:00:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.197+0000 7f9705483640 1 Processor -- start 2024-04-25T10:00:24.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.197+0000 7f9705483640 1 -- start start 2024-04-25T10:00:24.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f9705483640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f9705483640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f9705483640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97000c6940 con 0x7f97000cdbf0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f9705483640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000b0020 con 0x7f970015a9f0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f9705483640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000cb5a0 con 0x7f97000cd6f0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f96feffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f96feffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50182/0 (socket says 172.21.15.148:50182) 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.198+0000 7f96feffd640 1 -- 172.21.15.148:0/4087727153 learned_addr learned my addr 172.21.15.148:0/4087727153 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96feffd640 1 -- 172.21.15.148:0/4087727153 >> v2:172.21.15.188:3300/0 conn(0x7f97000cd6f0 msgr2=0x7f97000cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96ff7fe640 1 --2- 172.21.15.148:0/4087727153 >> v2:172.21.15.188:3300/0 conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96feffd640 1 --2- 172.21.15.148:0/4087727153 >> v2:172.21.15.188:3300/0 conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96feffd640 1 -- 172.21.15.148:0/4087727153 >> v1:172.21.15.188:6789/0 conn(0x7f97000cdbf0 legacy=0x7f97000ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96feffd640 1 -- 172.21.15.148:0/4087727153 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970015add0 con 0x7f970015a9f0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96ff7fe640 1 --2- 172.21.15.148:0/4087727153 >> v2:172.21.15.188:3300/0 conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96feffd640 1 --2- 172.21.15.148:0/4087727153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970015a9f0 0x7f97000c6010 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f97000cb510 tx=0x7f96f002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52fb2e8a85a6dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96fdffb640 1 -- 172.21.15.148:0/4087727153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0009600 con 0x7f970015a9f0 2024-04-25T10:00:24.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96fdffb640 1 -- 172.21.15.148:0/4087727153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f00097a0 con 0x7f970015a9f0 2024-04-25T10:00:24.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.199+0000 7f96fdffb640 1 -- 172.21.15.148:0/4087727153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0009b70 con 0x7f970015a9f0 2024-04-25T10:00:24.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 -- 172.21.15.148:0/4087727153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970015a9f0 msgr2=0x7f97000c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 --2- 172.21.15.148:0/4087727153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970015a9f0 0x7f97000c6010 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f97000cb510 tx=0x7f96f002c900 comp rx=0 tx=0).stop 2024-04-25T10:00:24.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 -- 172.21.15.148:0/4087727153 shutdown_connections 2024-04-25T10:00:24.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 --2- 172.21.15.148:0/4087727153 >> v2:172.21.15.188:3300/0 conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 --2- 172.21.15.148:0/4087727153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 -- 172.21.15.148:0/4087727153 >> 172.21.15.148:0/4087727153 conn(0x7f97000b2390 msgr2=0x7f97000b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 -- 172.21.15.148:0/4087727153 shutdown_connections 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.200+0000 7f9705483640 1 -- 172.21.15.148:0/4087727153 wait complete. 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.201+0000 7f9705483640 1 Processor -- start 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.201+0000 7f9705483640 1 -- start start 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f9705483640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 0x7f97001427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f9705483640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97000cdbf0 0x7f9700145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f9705483640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f970015a9f0 0x7f970006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f9705483640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000cb230 con 0x7f97000cd6f0 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f9705483640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000caf50 con 0x7f97000cdbf0 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f9705483640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ce070 con 0x7f970015a9f0 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 0x7f97001427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 0x7f97001427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50192/0 (socket says 172.21.15.148:50192) 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 -- 172.21.15.148:0/330435572 learned_addr learned my addr 172.21.15.148:0/330435572 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96fe7fc640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97000cdbf0 0x7f9700145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96ff7fe640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f970015a9f0 0x7f970006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 -- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f970015a9f0 msgr2=0x7f970006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f970015a9f0 0x7f970006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 -- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97000cdbf0 msgr2=0x7f9700145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97000cdbf0 0x7f9700145930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96feffd640 1 -- 172.21.15.148:0/330435572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700145e70 con 0x7f97000cd6f0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.202+0000 7f96ff7fe640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f970015a9f0 0x7f970006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.203+0000 7f96feffd640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 0x7f97001427e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f96f0000f80 tx=0x7f96f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.203+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0009600 con 0x7f97000cd6f0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.203+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0034c80 con 0x7f97000cd6f0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.203+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000b720 con 0x7f97000cd6f0 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.203+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9700156270 con 0x7f97000cd6f0 2024-04-25T10:00:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.203+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9700156830 con 0x7f97000cd6f0 2024-04-25T10:00:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.204+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970006cc80 con 0x7f97000cd6f0 2024-04-25T10:00:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.204+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96f0045020 con 0x7f97000cd6f0 2024-04-25T10:00:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.205+0000 7f96e77fe640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96ec06b4d0 0x7f96ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.205+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f96f00ad570 con 0x7f97000cd6f0 2024-04-25T10:00:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.205+0000 7f96fe7fc640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96ec06b4d0 0x7f96ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.205+0000 7f96fe7fc640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96ec06b4d0 0x7f96ec06d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9700159b10 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.209+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9700145e70 con 0x7f97000cd6f0 2024-04-25T10:00:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:24.382+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f97000d01a0 con 0x7f97000cd6f0 2024-04-25T10:00:24.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:24.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:25.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.074+0000 7f96e77fe640 1 -- 172.21.15.148:0/330435572 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) v1 ==== 170+0+0 (secure 0 0 0) 0x7f97000c6940 con 0x7f97000cd6f0 2024-04-25T10:00:25.079 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:00:25.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96ec06b4d0 msgr2=0x7f96ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96ec06b4d0 0x7f96ec06d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9700159b10 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2024-04-25T10:00:25.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 msgr2=0x7f97001427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 0x7f97001427e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f96f0000f80 tx=0x7f96f0004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 shutdown_connections 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96ec06b4d0 0x7f96ec06d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f970015a9f0 0x7f970006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97000cdbf0 0x7f9700145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 --2- 172.21.15.148:0/330435572 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97000cd6f0 0x7f97001427e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 >> 172.21.15.148:0/330435572 conn(0x7f97000b2390 msgr2=0x7f97000cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 shutdown_connections 2024-04-25T10:00:25.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.077+0000 7f9705483640 1 -- 172.21.15.148:0/330435572 wait complete. 2024-04-25T10:00:25.108 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:00:25.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.206+0000 7f422c21f640 1 Processor -- start 2024-04-25T10:00:25.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.206+0000 7f422c21f640 1 -- start start 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f422c21f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42240d0ab0 0x7f4224168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f422c21f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240d2420 con 0x7f42240d34c0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f422c21f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240d2700 con 0x7f422415dac0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f422c21f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42240d3030 con 0x7f42240d0ab0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f42297ba640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f42240d34c0 0x7f42240d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33294/0 (socket says 172.21.15.148:33294) 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f42297ba640 1 -- 172.21.15.148:0/3243215040 learned_addr learned my addr 172.21.15.148:0/3243215040 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918504397 0 0) 0x7f42240d2420 con 0x7f42240d34c0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4210003520 con 0x7f42240d34c0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f422a7bc640 1 --2- 172.21.15.148:0/3243215040 >> v2:172.21.15.188:3300/0 conn(0x7f42240d0ab0 0x7f4224168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 68768091 0 0) 0x7f4210003520 con 0x7f42240d34c0 2024-04-25T10:00:25.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 >> v2:172.21.15.188:3300/0 conn(0x7f42240d0ab0 msgr2=0x7f4224168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 --2- 172.21.15.148:0/3243215040 >> v2:172.21.15.188:3300/0 conn(0x7f42240d0ab0 0x7f4224168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 >> v1:172.21.15.188:6789/0 conn(0x7f422415dac0 legacy=0x7f42240d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240d29e0 con 0x7f42240d34c0 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f42140017f0 con 0x7f42240d34c0 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 >> v1:172.21.15.148:6789/0 conn(0x7f42240d34c0 legacy=0x7f42240d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42100039b0 0x7f4210003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4210004800 0x7f4210004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100051c0 0x7f421000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240d3030 con 0x7f4210004800 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421000c070 con 0x7f42100039b0 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.207+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421000c270 con 0x7f42100051c0 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4210004800 0x7f4210004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f4229fbb640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42100039b0 0x7f4210003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f422a7bc640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100051c0 0x7f421000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 -- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100051c0 msgr2=0x7f421000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100051c0 0x7f421000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 -- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42100039b0 msgr2=0x7f4210003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42100039b0 0x7f4210003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 -- 172.21.15.148:0/3243215040 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210003520 con 0x7f4210004800 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f422a7bc640 1 -- 172.21.15.148:0/3243215040 reap_dead start 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.208+0000 7f42297ba640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4210004800 0x7f4210004c80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f421400c240 tx=0x7f4210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305a99c356e031e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214007ce0 con 0x7f4210004800 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4214007270 con 0x7f4210004800 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f4228fb9640 1 -- 172.21.15.148:0/3243215040 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42140057a0 con 0x7f4210004800 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 -- 172.21.15.148:0/3243215040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4210004800 msgr2=0x7f4210004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4210004800 0x7f4210004c80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f421400c240 tx=0x7f4210005830 comp rx=0 tx=0).stop 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 -- 172.21.15.148:0/3243215040 shutdown_connections 2024-04-25T10:00:25.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 --2- 172.21.15.148:0/3243215040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4210004800 0x7f4210004c80 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 -- 172.21.15.148:0/3243215040 >> 172.21.15.148:0/3243215040 conn(0x7f4224061c30 msgr2=0x7f422406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:25.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 -- 172.21.15.148:0/3243215040 shutdown_connections 2024-04-25T10:00:25.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.209+0000 7f422c21f640 1 -- 172.21.15.148:0/3243215040 wait complete. 2024-04-25T10:00:25.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.210+0000 7f422c21f640 1 Processor -- start 2024-04-25T10:00:25.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 -- start start 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4210004800 0x7f42240b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 0x7f4224160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4224144410 0x7f4224160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b97d0 con 0x7f4224143fb0 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422415dea0 con 0x7f4224144410 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422c21f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b9950 con 0x7f4210004800 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f42297ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 0x7f4224160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f42297ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 0x7f4224160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50208/0 (socket says 172.21.15.148:50208) 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f42297ba640 1 -- 172.21.15.148:0/4156065980 learned_addr learned my addr 172.21.15.148:0/4156065980 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f422a7bc640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4224144410 0x7f4224160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.211+0000 7f4229fbb640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4210004800 0x7f42240b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f42297ba640 1 -- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4210004800 msgr2=0x7f42240b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f42297ba640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4210004800 0x7f42240b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f42297ba640 1 -- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4224144410 msgr2=0x7f4224160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f42297ba640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4224144410 0x7f4224160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f42297ba640 1 -- 172.21.15.148:0/4156065980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240cb5f0 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f42297ba640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 0x7f4224160600 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f421400c240 tx=0x7f42240d28d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421403f070 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42140015f0 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42240cb890 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214007420 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.212+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42240cbe50 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.213+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4224073a00 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.213+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4214042070 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.214+0000 7f421a7fc640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f420806b500 0x7f420806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.214+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f42140a9930 con 0x7f4224143fb0 2024-04-25T10:00:25.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.214+0000 7f4229fbb640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f420806b500 0x7f420806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:25.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.214+0000 7f4229fbb640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f420806b500 0x7f420806d9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4220000c00 tx=0x7f4220009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:25.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.218+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4214049070 con 0x7f4224143fb0 2024-04-25T10:00:25.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:25.423+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f422406b210 con 0x7f4224143fb0 2024-04-25T10:00:26.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.088+0000 7f421a7fc640 1 -- 172.21.15.148:0/4156065980 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4214078030 con 0x7f4224143fb0 2024-04-25T10:00:26.092 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T10:00:26.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f420806b500 msgr2=0x7f420806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f420806b500 0x7f420806d9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4220000c00 tx=0x7f4220009040 comp rx=0 tx=0).stop 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 msgr2=0x7f4224160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 0x7f4224160600 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f421400c240 tx=0x7f42240d28d0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 shutdown_connections 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f420806b500 0x7f420806d9c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4224144410 0x7f4224160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4224143fb0 0x7f4224160600 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 --2- 172.21.15.148:0/4156065980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4210004800 0x7f42240b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 >> 172.21.15.148:0/4156065980 conn(0x7f4224061c30 msgr2=0x7f42240c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 shutdown_connections 2024-04-25T10:00:26.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.090+0000 7f422c21f640 1 -- 172.21.15.148:0/4156065980 wait complete. 2024-04-25T10:00:26.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:26.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.220+0000 7f2c58d3b640 1 Processor -- start 2024-04-25T10:00:26.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.220+0000 7f2c58d3b640 1 -- start start 2024-04-25T10:00:26.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.221+0000 7f2c58d3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 0x7f2c5409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.221+0000 7f2c58d3b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c5415f5f0 0x7f2c5415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.221+0000 7f2c58d3b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c540cd710 con 0x7f2c541602e0 2024-04-25T10:00:26.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.221+0000 7f2c58d3b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c540cd4b0 con 0x7f2c5415f5f0 2024-04-25T10:00:26.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.221+0000 7f2c58d3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54099b70 con 0x7f2c5415ba00 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 0x7f2c5409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c52d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c5415f5f0 0x7f2c5415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 0x7f2c5409a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:48326/0 (socket says 172.21.15.148:48326) 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c51d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f2c541602e0 0x7f2c540ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43266/0 (socket says 172.21.15.148:43266) 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c52d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c5415f5f0 0x7f2c5415faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:37410/0 (socket says 172.21.15.148:37410) 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c5259c640 1 -- 172.21.15.148:0/1437644036 learned_addr learned my addr 172.21.15.148:0/1437644036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c5159a640 1 -- 172.21.15.148:0/1437644036 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149272486 0 0) 0x7f2c48001110 con 0x7f2c541602e0 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c5159a640 1 -- 172.21.15.148:0/1437644036 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c3c003520 con 0x7f2c541602e0 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c52d9d640 1 -- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 msgr2=0x7f2c5409a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c52d9d640 1 --2- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 0x7f2c5409a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c52d9d640 1 -- 172.21.15.148:0/1437644036 >> v1:172.21.15.188:6789/0 conn(0x7f2c541602e0 legacy=0x7f2c540ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c52d9d640 1 -- 172.21.15.148:0/1437644036 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54099e50 con 0x7f2c5415f5f0 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.222+0000 7f2c5259c640 1 --2- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 0x7f2c5409a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.223+0000 7f2c52d9d640 1 --2- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5415f5f0 0x7f2c5415faa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0097a0 tx=0x7f2c4c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5891feeecfa95f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.223+0000 7f2c5159a640 1 -- 172.21.15.148:0/1437644036 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c037070 con 0x7f2c5415f5f0 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.223+0000 7f2c5159a640 1 -- 172.21.15.148:0/1437644036 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c02adb0 con 0x7f2c5415f5f0 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.223+0000 7f2c5159a640 1 -- 172.21.15.148:0/1437644036 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c032640 con 0x7f2c5415f5f0 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 -- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5415f5f0 msgr2=0x7f2c5415faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5415f5f0 0x7f2c5415faa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0097a0 tx=0x7f2c4c02a060 comp rx=0 tx=0).stop 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 -- 172.21.15.148:0/1437644036 shutdown_connections 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c5415f5f0 0x7f2c5415faa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/1437644036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415ba00 0x7f2c5409a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 -- 172.21.15.148:0/1437644036 >> 172.21.15.148:0/1437644036 conn(0x7f2c540b3270 msgr2=0x7f2c540b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 -- 172.21.15.148:0/1437644036 shutdown_connections 2024-04-25T10:00:26.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.224+0000 7f2c58d3b640 1 -- 172.21.15.148:0/1437644036 wait complete. 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.225+0000 7f2c58d3b640 1 Processor -- start 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.225+0000 7f2c58d3b640 1 -- start start 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c58d3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 0x7f2c54156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c58d3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415f5f0 0x7f2c5415da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c58d3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c541602e0 0x7f2c5415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c58d3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5415bde0 con 0x7f2c5415ba00 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c58d3b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5409ae70 con 0x7f2c541602e0 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c58d3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5406c780 con 0x7f2c5415f5f0 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 0x7f2c54156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 0x7f2c54156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50218/0 (socket says 172.21.15.148:50218) 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 -- 172.21.15.148:0/3247719639 learned_addr learned my addr 172.21.15.148:0/3247719639 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c52d9d640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c541602e0 0x7f2c5415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c51d9b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415f5f0 0x7f2c5415da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 -- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415f5f0 msgr2=0x7f2c5415da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415f5f0 0x7f2c5415da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 -- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c541602e0 msgr2=0x7f2c5415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c541602e0 0x7f2c5415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 -- 172.21.15.148:0/3247719639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c540c73c0 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c5259c640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 0x7f2c54156340 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400efc0 tx=0x7f2c4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.226+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400ce60 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.227+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c44019070 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.227+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44014680 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.227+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5406be70 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.227+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5406c410 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.228+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c54073990 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.228+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c44005d10 con 0x7f2c5415ba00 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.228+0000 7f2c42ffd640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c3406b4d0 0x7f2c3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.228+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c4408a720 con 0x7f2c5415ba00 2024-04-25T10:00:26.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.228+0000 7f2c51d9b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c3406b4d0 0x7f2c3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.229+0000 7f2c51d9b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c3406b4d0 0x7f2c3406d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c541f6ab0 tx=0x7f2c48007b30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.233+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c44010040 con 0x7f2c5415ba00 2024-04-25T10:00:26.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.404+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c540ce9a0 con 0x7f2c5415ba00 2024-04-25T10:00:26.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.405+0000 7f2c42ffd640 1 -- 172.21.15.148:0/3247719639 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16946 (secure 0 0 0) 0x7f2c44058dd0 con 0x7f2c5415ba00 2024-04-25T10:00:26.410 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:26.410 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":96,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:25.424346+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c3406b4d0 msgr2=0x7f2c3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c3406b4d0 0x7f2c3406d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c541f6ab0 tx=0x7f2c48007b30 comp rx=0 tx=0).stop 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 msgr2=0x7f2c54156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 0x7f2c54156340 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400efc0 tx=0x7f2c4400c490 comp rx=0 tx=0).stop 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 shutdown_connections 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2c3406b4d0 0x7f2c3406d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c541602e0 0x7f2c5415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c5415f5f0 0x7f2c5415da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 --2- 172.21.15.148:0/3247719639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c5415ba00 0x7f2c54156340 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 >> 172.21.15.148:0/3247719639 conn(0x7f2c540b3270 msgr2=0x7f2c540cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 shutdown_connections 2024-04-25T10:00:26.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.408+0000 7f2c58d3b640 1 -- 172.21.15.148:0/3247719639 wait complete. 2024-04-25T10:00:26.423 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:26.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.515+0000 7fa0a7990640 1 Processor -- start 2024-04-25T10:00:26.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.515+0000 7fa0a7990640 1 -- start start 2024-04-25T10:00:26.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a7990640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0a013d190 0x7fa0a00cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a7990640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a0071ed0 con 0x7fa0a0071940 2024-04-25T10:00:26.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a7990640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a015a840 con 0x7fa0a015b320 2024-04-25T10:00:26.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a7990640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a013d570 con 0x7fa0a013d190 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a5f2d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa0a0071940 0x7fa0a015dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33310/0 (socket says 172.21.15.148:33310) 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a5f2d640 1 -- 172.21.15.148:0/3233926011 learned_addr learned my addr 172.21.15.148:0/3233926011 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa0a572c640 1 --2- 172.21.15.148:0/3233926011 >> v2:172.21.15.188:3300/0 conn(0x7fa0a013d190 0x7fa0a00cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.516+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195483873 0 0) 0x7fa0a0071ed0 con 0x7fa0a0071940 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa08c005730 con 0x7fa0a0071940 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533475470 0 0) 0x7fa08c005730 con 0x7fa0a0071940 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 >> v2:172.21.15.188:3300/0 conn(0x7fa0a013d190 msgr2=0x7fa0a00cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 --2- 172.21.15.148:0/3233926011 >> v2:172.21.15.188:3300/0 conn(0x7fa0a013d190 0x7fa0a00cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 >> v1:172.21.15.188:6789/0 conn(0x7fa0a015b320 legacy=0x7fa0a0071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a015ab20 con 0x7fa0a0071940 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa098001670 con 0x7fa0a0071940 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 >> v1:172.21.15.148:6789/0 conn(0x7fa0a0071940 legacy=0x7fa0a015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa08c005bc0 0x7fa08c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa08c006a10 0x7fa08c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c0073d0 0x7fa08c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a013d570 con 0x7fa08c005bc0 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c00e370 con 0x7fa08c0073d0 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c00e570 con 0x7fa08c006a10 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a572c640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa08c005bc0 0x7fa08c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a5f2d640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c0073d0 0x7fa08c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a4f2b640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa08c006a10 0x7fa08c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a572c640 1 -- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa08c006a10 msgr2=0x7fa08c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a572c640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa08c006a10 0x7fa08c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a572c640 1 -- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c0073d0 msgr2=0x7fa08c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a572c640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c0073d0 0x7fa08c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a572c640 1 -- 172.21.15.148:0/3233926011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c005730 con 0x7fa08c005bc0 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a5f2d640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c0073d0 0x7fa08c00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:26.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.517+0000 7fa0a5f2d640 1 -- 172.21.15.148:0/3233926011 reap_dead start 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.518+0000 7fa0a572c640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa08c005bc0 0x7fa08c005fe0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009fc0 tx=0x7fa08c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123b6418a963ccef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.518+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c009c10 con 0x7fa08c005bc0 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.518+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c036070 con 0x7fa08c005bc0 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.518+0000 7fa097fff640 1 -- 172.21.15.148:0/3233926011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c031880 con 0x7fa08c005bc0 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.518+0000 7fa0a7990640 1 -- 172.21.15.148:0/3233926011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa08c005bc0 msgr2=0x7fa08c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.518+0000 7fa0a7990640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa08c005bc0 0x7fa08c005fe0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009fc0 tx=0x7fa08c007a40 comp rx=0 tx=0).stop 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.519+0000 7fa0a7990640 1 -- 172.21.15.148:0/3233926011 shutdown_connections 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.519+0000 7fa0a7990640 1 --2- 172.21.15.148:0/3233926011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa08c005bc0 0x7fa08c005fe0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.519+0000 7fa0a7990640 1 -- 172.21.15.148:0/3233926011 >> 172.21.15.148:0/3233926011 conn(0x7fa0a006abe0 msgr2=0x7fa0a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.519+0000 7fa0a7990640 1 -- 172.21.15.148:0/3233926011 shutdown_connections 2024-04-25T10:00:26.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.519+0000 7fa0a7990640 1 -- 172.21.15.148:0/3233926011 wait complete. 2024-04-25T10:00:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 Processor -- start 2024-04-25T10:00:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 -- start start 2024-04-25T10:00:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c005bc0 0x7fa0a015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0a00b0f20 0x7fa0a00b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 0x7fa0a0130e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00713b0 con 0x7fa0a0130980 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0071230 con 0x7fa08c005bc0 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.520+0000 7fa0a7990640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0071530 con 0x7fa0a00b0f20 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 0x7fa0a0130e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 0x7fa0a0130e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50238/0 (socket says 172.21.15.148:50238) 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 -- 172.21.15.148:0/2640841392 learned_addr learned my addr 172.21.15.148:0/2640841392 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a572c640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c005bc0 0x7fa0a015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a4f2b640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0a00b0f20 0x7fa0a00b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 -- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0a00b0f20 msgr2=0x7fa0a00b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0a00b0f20 0x7fa0a00b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 -- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c005bc0 msgr2=0x7fa0a015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c005bc0 0x7fa0a015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 -- 172.21.15.148:0/2640841392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00b18e0 con 0x7fa0a0130980 2024-04-25T10:00:26.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a572c640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c005bc0 0x7fa0a015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a5f2d640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 0x7fa0a0130e00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa098000c00 tx=0x7fa09800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09800c610 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098018070 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a00b2a10 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09800c9f0 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.521+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a00b2fd0 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.522+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0073990 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.523+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa09800cb90 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.523+0000 7fa095ffb640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa08406b4d0 0x7fa08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.523+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa098089fa0 con 0x7fa0a0130980 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.523+0000 7fa0a572c640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa08406b4d0 0x7fa08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.524+0000 7fa0a572c640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa08406b4d0 0x7fa08406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa09c035000 tx=0x7fa09c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.527+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa098058c90 con 0x7fa0a0130980 2024-04-25T10:00:26.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.696+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0a00b0b70 con 0x7fa0a0130980 2024-04-25T10:00:26.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.697+0000 7fa095ffb640 1 -- 172.21.15.148:0/2640841392 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16946 (secure 0 0 0) 0x7fa098058590 con 0x7fa0a0130980 2024-04-25T10:00:26.702 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:26.702 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":96,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:25.424346+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa08406b4d0 msgr2=0x7fa08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa08406b4d0 0x7fa08406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa09c035000 tx=0x7fa09c02f000 comp rx=0 tx=0).stop 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 msgr2=0x7fa0a0130e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 0x7fa0a0130e00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa098000c00 tx=0x7fa09800eb90 comp rx=0 tx=0).stop 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 shutdown_connections 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa08406b4d0 0x7fa08406d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0130980 0x7fa0a0130e00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0a00b0f20 0x7fa0a00b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 --2- 172.21.15.148:0/2640841392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa08c005bc0 0x7fa0a015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 >> 172.21.15.148:0/2640841392 conn(0x7fa0a006abe0 msgr2=0x7fa0a00cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 shutdown_connections 2024-04-25T10:00:26.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.699+0000 7fa0a7990640 1 -- 172.21.15.148:0/2640841392 wait complete. 2024-04-25T10:00:26.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:00:26.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.811+0000 7f928891d640 1 Processor -- start 2024-04-25T10:00:26.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.811+0000 7f928891d640 1 -- start start 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928891d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9284070dc0 0x7f928406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928891d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92840711a0 0x7f92840b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928891d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840b4830 0x7f92840b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928891d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92841622b0 con 0x7f9284070dc0 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928891d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840b10f0 con 0x7f92840711a0 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928891d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840d1770 con 0x7f92840b4830 2024-04-25T10:00:26.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9284070dc0 0x7f928406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9284070dc0 0x7f928406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50242/0 (socket says 172.21.15.148:50242) 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 -- 172.21.15.148:0/1291880137 learned_addr learned my addr 172.21.15.148:0/1291880137 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f9281d9b640 1 --2- 172.21.15.148:0/1291880137 >> v2:172.21.15.188:3300/0 conn(0x7f92840711a0 0x7f92840b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f9282d9d640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840b4830 0x7f92840b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 -- 172.21.15.148:0/1291880137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840b4830 msgr2=0x7f92840b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840b4830 0x7f92840b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 -- 172.21.15.148:0/1291880137 >> v2:172.21.15.188:3300/0 conn(0x7f92840711a0 msgr2=0x7f92840b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 --2- 172.21.15.148:0/1291880137 >> v2:172.21.15.188:3300/0 conn(0x7f92840711a0 0x7f92840b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f928259c640 1 -- 172.21.15.148:0/1291880137 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284162740 con 0x7f9284070dc0 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.812+0000 7f9282d9d640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840b4830 0x7f92840b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:26.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.813+0000 7f928259c640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9284070dc0 0x7f928406ccd0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f928404efc0 tx=0x7f927402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f07af3f270154a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.813+0000 7f928159a640 1 -- 172.21.15.148:0/1291880137 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274036070 con 0x7f9284070dc0 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.813+0000 7f928159a640 1 -- 172.21.15.148:0/1291880137 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9274031600 con 0x7f9284070dc0 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.813+0000 7f928159a640 1 -- 172.21.15.148:0/1291880137 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740319d0 con 0x7f9284070dc0 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 -- 172.21.15.148:0/1291880137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9284070dc0 msgr2=0x7f928406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9284070dc0 0x7f928406ccd0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f928404efc0 tx=0x7f927402a5b0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 -- 172.21.15.148:0/1291880137 shutdown_connections 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840b4830 0x7f92840b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 --2- 172.21.15.148:0/1291880137 >> v2:172.21.15.188:3300/0 conn(0x7f92840711a0 0x7f92840b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 --2- 172.21.15.148:0/1291880137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9284070dc0 0x7f928406ccd0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 -- 172.21.15.148:0/1291880137 >> 172.21.15.148:0/1291880137 conn(0x7f92840b3530 msgr2=0x7f92840b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 -- 172.21.15.148:0/1291880137 shutdown_connections 2024-04-25T10:00:26.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.814+0000 7f928891d640 1 -- 172.21.15.148:0/1291880137 wait complete. 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.815+0000 7f928891d640 1 Processor -- start 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.815+0000 7f928891d640 1 -- start start 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928891d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9284070dc0 0x7f9284151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928891d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840711a0 0x7f9284148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928891d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 0x7f9284152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928891d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9284161f40 con 0x7f92840b4830 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928891d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9284161c60 con 0x7f9284070dc0 2024-04-25T10:00:26.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928891d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9284071630 con 0x7f92840711a0 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 0x7f9284152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 0x7f9284152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50256/0 (socket says 172.21.15.148:50256) 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 -- 172.21.15.148:0/1334911772 learned_addr learned my addr 172.21.15.148:0/1334911772 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f928259c640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9284070dc0 0x7f9284151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9281d9b640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840711a0 0x7f9284148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 -- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840711a0 msgr2=0x7f9284148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840711a0 0x7f9284148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 -- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9284070dc0 msgr2=0x7f9284151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9284070dc0 0x7f9284151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 -- 172.21.15.148:0/1334911772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284148f60 con 0x7f92840b4830 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.816+0000 7f9282d9d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 0x7f9284152b70 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f927c00d7d0 tx=0x7f927c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.817+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c004030 con 0x7f92840b4830 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.817+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c0041d0 con 0x7f92840b4830 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.817+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0096a0 con 0x7f92840b4830 2024-04-25T10:00:26.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.817+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92841533c0 con 0x7f92840b4830 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.817+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9284153980 con 0x7f92840b4830 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.818+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9284073a00 con 0x7f92840b4830 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.818+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f927c009840 con 0x7f92840b4830 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.818+0000 7f9272ffd640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926406b500 0x7f926406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.819+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f927c08a4f0 con 0x7f92840b4830 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.819+0000 7f928259c640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926406b500 0x7f926406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:26.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.819+0000 7f928259c640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926406b500 0x7f926406d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9274033000 tx=0x7f9274009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:26.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:26.823+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9284148f60 con 0x7f92840b4830 2024-04-25T10:00:27.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.032+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f928406caa0 con 0x7f92840b4830 2024-04-25T10:00:27.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.105+0000 7f9272ffd640 1 -- 172.21.15.148:0/1334911772 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f927c058b90 con 0x7f92840b4830 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926406b500 msgr2=0x7f926406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926406b500 0x7f926406d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9274033000 tx=0x7f9274009000 comp rx=0 tx=0).stop 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 msgr2=0x7f9284152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 0x7f9284152b70 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f927c00d7d0 tx=0x7f927c00dc90 comp rx=0 tx=0).stop 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 shutdown_connections 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926406b500 0x7f926406d9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840b4830 0x7f9284152b70 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840711a0 0x7f9284148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 --2- 172.21.15.148:0/1334911772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9284070dc0 0x7f9284151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 >> 172.21.15.148:0/1334911772 conn(0x7f92840b3530 msgr2=0x7f928406ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 shutdown_connections 2024-04-25T10:00:27.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.107+0000 7f928891d640 1 -- 172.21.15.148:0/1334911772 wait complete. 2024-04-25T10:00:27.123 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:00:27.123 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:00:27.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.218+0000 7f5404ccb640 1 Processor -- start 2024-04-25T10:00:27.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.218+0000 7f5404ccb640 1 -- start start 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f5404ccb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f540015bb50 0x7f54000c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f5404ccb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54000c70d0 0x7f54000cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f5404ccb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54000c75e0 con 0x7f54000cfb40 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f5404ccb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000c8680 con 0x7f54000c70d0 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f5404ccb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000c78c0 con 0x7f540015bb50 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f53fdd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54000c70d0 0x7f54000cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f53fed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f54000cfb40 0x7f5400070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33320/0 (socket says 172.21.15.148:33320) 2024-04-25T10:00:27.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.219+0000 7f53fed9d640 1 -- 172.21.15.148:0/180224004 learned_addr learned my addr 172.21.15.148:0/180224004 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fe59c640 1 --2- 172.21.15.148:0/180224004 >> v2:172.21.15.188:3300/0 conn(0x7f540015bb50 0x7f54000c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fdd9b640 1 -- 172.21.15.148:0/180224004 >> v2:172.21.15.188:3300/0 conn(0x7f540015bb50 msgr2=0x7f54000c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fdd9b640 1 --2- 172.21.15.148:0/180224004 >> v2:172.21.15.188:3300/0 conn(0x7f540015bb50 0x7f54000c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fdd9b640 1 -- 172.21.15.148:0/180224004 >> v1:172.21.15.148:6789/0 conn(0x7f54000cfb40 legacy=0x7f5400070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fdd9b640 1 -- 172.21.15.148:0/180224004 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000c7ba0 con 0x7f54000c70d0 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fd59a640 1 -- 172.21.15.148:0/180224004 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039499871 0 0) 0x7f54000c75e0 con 0x7f54000cfb40 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.220+0000 7f53fdd9b640 1 --2- 172.21.15.148:0/180224004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f54000cf600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5400070ab0 tx=0x7f53f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c0020de8902a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f53fd59a640 1 -- 172.21.15.148:0/180224004 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4037070 con 0x7f54000c70d0 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f53fd59a640 1 -- 172.21.15.148:0/180224004 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f4032440 con 0x7f54000c70d0 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f53fd59a640 1 -- 172.21.15.148:0/180224004 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4032810 con 0x7f54000c70d0 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 -- 172.21.15.148:0/180224004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 msgr2=0x7f54000cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 --2- 172.21.15.148:0/180224004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f54000cf600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5400070ab0 tx=0x7f53f402a230 comp rx=0 tx=0).stop 2024-04-25T10:00:27.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 -- 172.21.15.148:0/180224004 shutdown_connections 2024-04-25T10:00:27.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 --2- 172.21.15.148:0/180224004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f54000cf600 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 --2- 172.21.15.148:0/180224004 >> v2:172.21.15.188:3300/0 conn(0x7f540015bb50 0x7f54000c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 -- 172.21.15.148:0/180224004 >> 172.21.15.148:0/180224004 conn(0x7f540006f500 msgr2=0x7f540006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:27.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 -- 172.21.15.148:0/180224004 shutdown_connections 2024-04-25T10:00:27.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.221+0000 7f5404ccb640 1 -- 172.21.15.148:0/180224004 wait complete. 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 Processor -- start 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 -- start start 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f5400147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54000cfb40 0x7f540015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540015bb50 0x7f54000ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540015f6d0 con 0x7f54000c70d0 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000c8160 con 0x7f54000cfb40 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f5404ccb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540015f850 con 0x7f540015bb50 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f53fe59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f5400147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f53fe59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f5400147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50274/0 (socket says 172.21.15.148:50274) 2024-04-25T10:00:27.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.223+0000 7f53fe59c640 1 -- 172.21.15.148:0/211502043 learned_addr learned my addr 172.21.15.148:0/211502043 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fe59c640 1 -- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540015bb50 msgr2=0x7f54000ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fe59c640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540015bb50 0x7f54000ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fe59c640 1 -- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54000cfb40 msgr2=0x7f540015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fdd9b640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54000cfb40 0x7f540015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fe59c640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54000cfb40 0x7f540015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fe59c640 1 -- 172.21.15.148:0/211502043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540015ced0 con 0x7f54000c70d0 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fdd9b640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54000cfb40 0x7f540015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53fe59c640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f5400147150 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f53f000efc0 tx=0x7f53f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0018020 con 0x7f54000c70d0 2024-04-25T10:00:27.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f0014480 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0014850 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540015c910 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.224+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540015d440 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.225+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5400073a00 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.225+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53f0005d10 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.226+0000 7f53eeffd640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e006b4d0 0x7f53e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.226+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53f008ad70 con 0x7f54000c70d0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.226+0000 7f53fdd9b640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e006b4d0 0x7f53e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.226+0000 7f53fdd9b640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e006b4d0 0x7f53e006d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f53f4034000 tx=0x7f53f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:27.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.230+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53f0022070 con 0x7f54000c70d0 2024-04-25T10:00:27.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:27.436+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f54000cf150 con 0x7f54000c70d0 2024-04-25T10:00:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.116+0000 7f53eeffd640 1 -- 172.21.15.148:0/211502043 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53f0010040 con 0x7f54000c70d0 2024-04-25T10:00:28.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e006b4d0 msgr2=0x7f53e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e006b4d0 0x7f53e006d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f53f4034000 tx=0x7f53f402e000 comp rx=0 tx=0).stop 2024-04-25T10:00:28.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 msgr2=0x7f5400147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f5400147150 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f53f000efc0 tx=0x7f53f000c490 comp rx=0 tx=0).stop 2024-04-25T10:00:28.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 shutdown_connections 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e006b4d0 0x7f53e006d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f540015bb50 0x7f54000ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54000cfb40 0x7f540015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 --2- 172.21.15.148:0/211502043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54000c70d0 0x7f5400147150 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 >> 172.21.15.148:0/211502043 conn(0x7f540006f500 msgr2=0x7f54000d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 shutdown_connections 2024-04-25T10:00:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.119+0000 7f5404ccb640 1 -- 172.21.15.148:0/211502043 wait complete. 2024-04-25T10:00:28.136 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:00:28.136 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.234+0000 7f938834c640 1 Processor -- start 2024-04-25T10:00:28.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.234+0000 7f938834c640 1 -- start start 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f938834c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f93800aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f938834c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9380071ac0 0x7f93800ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f938834c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938015ec60 con 0x7f938006c1d0 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f938834c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938015ef40 con 0x7f9380071ac0 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f938834c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938006db90 con 0x7f9380154ed0 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93868e9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f938006c1d0 0x7f93801617d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33332/0 (socket says 172.21.15.148:33332) 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93868e9640 1 -- 172.21.15.148:0/1098690795 learned_addr learned my addr 172.21.15.148:0/1098690795 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:28.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93858e7640 1 --2- 172.21.15.148:0/1098690795 >> v2:172.21.15.188:3300/0 conn(0x7f9380071ac0 0x7f93800ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93860e8640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f93800aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696373372 0 0) 0x7f938015ec60 con 0x7f938006c1d0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f936c003520 con 0x7f938006c1d0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.235+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827290237 0 0) 0x7f936c003520 con 0x7f938006c1d0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 msgr2=0x7f93800aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f93800aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 >> v2:172.21.15.188:3300/0 conn(0x7f9380071ac0 msgr2=0x7f93800ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 --2- 172.21.15.148:0/1098690795 >> v2:172.21.15.188:3300/0 conn(0x7f9380071ac0 0x7f93800ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938006de70 con 0x7f938006c1d0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f93780019f0 con 0x7f938006c1d0 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 >> v1:172.21.15.148:6789/0 conn(0x7f938006c1d0 legacy=0x7f93801617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f936c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f936c003f30 0x7f936c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f936c004d40 0x7f936c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938015ef40 con 0x7f936c003f30 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938006db90 con 0x7f936c004d40 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c00b9c0 con 0x7f9380154ed0 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93858e7640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f936c003f30 0x7f936c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93860e8640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f936c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93868e9640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f936c004d40 0x7f936c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93858e7640 1 -- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 msgr2=0x7f936c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93858e7640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f936c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93858e7640 1 -- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f936c004d40 msgr2=0x7f936c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93858e7640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f936c004d40 0x7f936c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93858e7640 1 -- 172.21.15.148:0/1098690795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c003520 con 0x7f936c003f30 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93860e8640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380154ed0 0x7f936c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.236+0000 7f93868e9640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f936c004d40 0x7f936c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f93858e7640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f936c003f30 0x7f936c004800 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9370009a90 tx=0x7f936c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a810744200421aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370036070 con 0x7f936c003f30 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9370031400 con 0x7f936c003f30 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f93850e6640 1 -- 172.21.15.148:0/1098690795 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370009690 con 0x7f936c003f30 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f938834c640 1 -- 172.21.15.148:0/1098690795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f936c003f30 msgr2=0x7f936c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f938834c640 1 --2- 172.21.15.148:0/1098690795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f936c003f30 0x7f936c004800 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9370009a90 tx=0x7f936c005830 comp rx=0 tx=0).stop 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f93868e9640 1 -- 172.21.15.148:0/1098690795 reap_dead start 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f938834c640 1 -- 172.21.15.148:0/1098690795 shutdown_connections 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f938834c640 1 -- 172.21.15.148:0/1098690795 >> 172.21.15.148:0/1098690795 conn(0x7f93800af670 msgr2=0x7f93800b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f938834c640 1 -- 172.21.15.148:0/1098690795 shutdown_connections 2024-04-25T10:00:28.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.237+0000 7f938834c640 1 -- 172.21.15.148:0/1098690795 wait complete. 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 Processor -- start 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 -- start start 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380142960 0x7f9380156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93801570d0 0x7f93800cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 0x7f93800cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93800ad950 con 0x7f93800cdf30 2024-04-25T10:00:28.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93800ad120 con 0x7f93801570d0 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f938834c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93800adad0 con 0x7f9380142960 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.239+0000 7f93868e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 0x7f93800cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 0x7f93800cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50286/0 (socket says 172.21.15.148:50286) 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 -- 172.21.15.148:0/1320831878 learned_addr learned my addr 172.21.15.148:0/1320831878 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93858e7640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93801570d0 0x7f93800cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93860e8640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380142960 0x7f9380156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 -- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380142960 msgr2=0x7f9380156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380142960 0x7f9380156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 -- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93801570d0 msgr2=0x7f93800cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93801570d0 0x7f93800cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 -- 172.21.15.148:0/1320831878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380142d40 con 0x7f93800cdf30 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93858e7640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93801570d0 0x7f93800cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f93868e9640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 0x7f93800cc420 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9378000f80 tx=0x7f937800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937800eb90 con 0x7f93800cdf30 2024-04-25T10:00:28.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937800ed50 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93801575a0 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378014670 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.240+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9380157b60 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.241+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9380073990 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.242+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9378014810 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.242+0000 7f9376ffd640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f936406b4d0 0x7f936406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.242+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f937808a220 con 0x7f93800cdf30 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.242+0000 7f93860e8640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f936406b4d0 0x7f936406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.242+0000 7f93860e8640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f936406b4d0 0x7f936406d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f937c0098d0 tx=0x7f937c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.246+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9378058f10 con 0x7f93800cdf30 2024-04-25T10:00:28.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.451+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f938006e930 con 0x7f93800cdf30 2024-04-25T10:00:28.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.452+0000 7f9376ffd640 1 -- 172.21.15.148:0/1320831878 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f9378058810 con 0x7f93800cdf30 2024-04-25T10:00:28.456 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:28.456 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:28.456 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:28.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f936406b4d0 msgr2=0x7f936406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f936406b4d0 0x7f936406d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f937c0098d0 tx=0x7f937c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:28.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 msgr2=0x7f93800cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 0x7f93800cc420 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9378000f80 tx=0x7f937800e530 comp rx=0 tx=0).stop 2024-04-25T10:00:28.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 shutdown_connections 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f936406b4d0 0x7f936406d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93800cdf30 0x7f93800cc420 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93801570d0 0x7f93800cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 --2- 172.21.15.148:0/1320831878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9380142960 0x7f9380156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 >> 172.21.15.148:0/1320831878 conn(0x7f936c003f30 msgr2=0x7f938015e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 shutdown_connections 2024-04-25T10:00:28.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.454+0000 7f938834c640 1 -- 172.21.15.148:0/1320831878 wait complete. 2024-04-25T10:00:28.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:28.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.562+0000 7fcb148f5640 1 Processor -- start 2024-04-25T10:00:28.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.562+0000 7fcb148f5640 1 -- start start 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb148f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcb1015dc00 0x7fcb100c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb148f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1015dfe0 con 0x7fcb100d1dd0 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb148f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb100d00c0 con 0x7fcb100ceb20 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb148f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb100d03a0 con 0x7fcb1015dc00 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0ed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb100d1dd0 0x7fcb100ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33336/0 (socket says 172.21.15.148:33336) 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0ed9d640 1 -- 172.21.15.148:0/2349726175 learned_addr learned my addr 172.21.15.148:0/2349726175 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731596397 0 0) 0x7fcb08001490 con 0x7fcb100d1dd0 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/2349726175 >> v2:172.21.15.188:3300/0 conn(0x7fcb1015dc00 0x7fcb100c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcafc003520 con 0x7fcb100d1dd0 2024-04-25T10:00:28.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792046567 0 0) 0x7fcafc003520 con 0x7fcb100d1dd0 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 >> v2:172.21.15.188:3300/0 conn(0x7fcb1015dc00 msgr2=0x7fcb100c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 --2- 172.21.15.148:0/2349726175 >> v2:172.21.15.188:3300/0 conn(0x7fcb1015dc00 0x7fcb100c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 >> v1:172.21.15.188:6789/0 conn(0x7fcb100ceb20 legacy=0x7fcb100d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100cc440 con 0x7fcb100d1dd0 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fcb08001bc0 con 0x7fcb100d1dd0 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 >> v1:172.21.15.148:6789/0 conn(0x7fcb100d1dd0 legacy=0x7fcb100ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcafc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.563+0000 7fcb0dd9b640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc004800 0x7fcafc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0dd9b640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0051c0 0x7fcafc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100d03a0 con 0x7fcafc0039b0 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc00c060 con 0x7fcafc004800 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc00c260 con 0x7fcafc0051c0 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcafc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb07fff640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc004800 0x7fcafc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0ed9d640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0051c0 0x7fcafc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0e59c640 1 -- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0051c0 msgr2=0x7fcafc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0051c0 0x7fcafc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0e59c640 1 -- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc004800 msgr2=0x7fcafc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc004800 0x7fcafc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0e59c640 1 -- 172.21.15.148:0/2349726175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc003520 con 0x7fcafc0039b0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0ed9d640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0051c0 0x7fcafc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.564+0000 7fcb0ed9d640 1 -- 172.21.15.148:0/2349726175 reap_dead start 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcafc003dd0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009ac0 tx=0x7fcafc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556645438502ee35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00036070 con 0x7fcafc0039b0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb00031400 con 0x7fcafc0039b0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb0dd9b640 1 -- 172.21.15.148:0/2349726175 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00009690 con 0x7fcafc0039b0 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 -- 172.21.15.148:0/2349726175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 msgr2=0x7fcafc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcafc003dd0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009ac0 tx=0x7fcafc005830 comp rx=0 tx=0).stop 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 -- 172.21.15.148:0/2349726175 shutdown_connections 2024-04-25T10:00:28.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 --2- 172.21.15.148:0/2349726175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcafc003dd0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 -- 172.21.15.148:0/2349726175 >> 172.21.15.148:0/2349726175 conn(0x7fcb1006f5d0 msgr2=0x7fcb1006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 -- 172.21.15.148:0/2349726175 shutdown_connections 2024-04-25T10:00:28.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.565+0000 7fcb148f5640 1 -- 172.21.15.148:0/2349726175 wait complete. 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 Processor -- start 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 -- start start 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcb1015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb1015fb20 0x7fcb1015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb10148bb0 0x7fcb1014b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100cfd90 con 0x7fcafc0039b0 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100cc130 con 0x7fcb10148bb0 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb148f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100cdd70 con 0x7fcb1015fb20 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb0e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcb1015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb0e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcb1015cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50310/0 (socket says 172.21.15.148:50310) 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.567+0000 7fcb0e59c640 1 -- 172.21.15.148:0/4131391323 learned_addr learned my addr 172.21.15.148:0/4131391323 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0ed9d640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb10148bb0 0x7fcb1014b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb07fff640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb1015fb20 0x7fcb1015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0e59c640 1 -- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb1015fb20 msgr2=0x7fcb1015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb1015fb20 0x7fcb1015e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0e59c640 1 -- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb10148bb0 msgr2=0x7fcb1014b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb10148bb0 0x7fcb1014b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0e59c640 1 -- 172.21.15.148:0/4131391323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10149750 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0ed9d640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb10148bb0 0x7fcb1014b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb0e59c640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcb1015cf50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcb00034040 tx=0x7fcb00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0003f020 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb00037040 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb10149a30 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00036070 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.568+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb10149ff0 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.569+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1015ff00 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.570+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb00042020 con 0x7fcafc0039b0 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.570+0000 7fcb06ffd640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.570+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb000aa3e0 con 0x7fcafc0039b0 2024-04-25T10:00:28.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.570+0000 7fcb07fff640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.570+0000 7fcb07fff640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcaf006b4d0 0x7fcaf006d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000c00 tx=0x7fcaf8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.574+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb00049030 con 0x7fcafc0039b0 2024-04-25T10:00:28.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.743+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb10073990 con 0x7fcafc0039b0 2024-04-25T10:00:28.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.745+0000 7fcb06ffd640 1 -- 172.21.15.148:0/4131391323 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fcb00079040 con 0x7fcafc0039b0 2024-04-25T10:00:28.749 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:28.749 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:27.088834+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:28.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcaf006b4d0 msgr2=0x7fcaf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcaf006b4d0 0x7fcaf006d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000c00 tx=0x7fcaf8009040 comp rx=0 tx=0).stop 2024-04-25T10:00:28.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 msgr2=0x7fcb1015cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcb1015cf50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcb00034040 tx=0x7fcb00004730 comp rx=0 tx=0).stop 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 shutdown_connections 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb10148bb0 0x7fcb1014b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb1015fb20 0x7fcb1015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 --2- 172.21.15.148:0/4131391323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcafc0039b0 0x7fcb1015cf50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 >> 172.21.15.148:0/4131391323 conn(0x7fcb1006f5d0 msgr2=0x7fcb1006ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 shutdown_connections 2024-04-25T10:00:28.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.747+0000 7fcb148f5640 1 -- 172.21.15.148:0/4131391323 wait complete. 2024-04-25T10:00:28.763 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:00:28.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:28.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.855+0000 7fe7df1f5640 1 Processor -- start 2024-04-25T10:00:28.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.855+0000 7fe7df1f5640 1 -- start start 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.855+0000 7fe7df1f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7d80cc8e0 0x7fe7d81625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.855+0000 7fe7df1f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7d80d1810 0x7fe7d80d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.855+0000 7fe7df1f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d80cce50 con 0x7fe7d815dc90 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7df1f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d80d21c0 con 0x7fe7d80d1810 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7df1f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d8161af0 con 0x7fe7d80cc8e0 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7d80d1810 0x7fe7d80d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe7d80d1810 0x7fe7d80d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50316/0 (socket says 172.21.15.148:50316) 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 -- 172.21.15.148:0/4229492099 learned_addr learned my addr 172.21.15.148:0/4229492099 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 -- 172.21.15.148:0/4229492099 >> v2:172.21.15.188:3300/0 conn(0x7fe7d80cc8e0 msgr2=0x7fe7d81625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7d7fff640 1 --2- 172.21.15.148:0/4229492099 >> v2:172.21.15.188:3300/0 conn(0x7fe7d80cc8e0 0x7fe7d81625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 --2- 172.21.15.148:0/4229492099 >> v2:172.21.15.188:3300/0 conn(0x7fe7d80cc8e0 0x7fe7d81625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 -- 172.21.15.148:0/4229492099 >> v1:172.21.15.188:6789/0 conn(0x7fe7d815dc90 legacy=0x7fe7d815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7dd792640 1 -- 172.21.15.148:0/4229492099 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8161dd0 con 0x7fe7d80d1810 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.856+0000 7fe7d7fff640 1 --2- 172.21.15.148:0/4229492099 >> v2:172.21.15.188:3300/0 conn(0x7fe7d80cc8e0 0x7fe7d81625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7dd792640 1 --2- 172.21.15.148:0/4229492099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d80d1810 0x7fe7d80d1c80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80c88a0 tx=0x7fe7d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30efdaaae23b6bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7d77fe640 1 -- 172.21.15.148:0/4229492099 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0036070 con 0x7fe7d80d1810 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7d77fe640 1 -- 172.21.15.148:0/4229492099 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0031600 con 0x7fe7d80d1810 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7d77fe640 1 -- 172.21.15.148:0/4229492099 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00319d0 con 0x7fe7d80d1810 2024-04-25T10:00:28.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 -- 172.21.15.148:0/4229492099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d80d1810 msgr2=0x7fe7d80d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/4229492099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d80d1810 0x7fe7d80d1c80 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80c88a0 tx=0x7fe7d002a660 comp rx=0 tx=0).stop 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 -- 172.21.15.148:0/4229492099 shutdown_connections 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/4229492099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d80d1810 0x7fe7d80d1c80 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/4229492099 >> v2:172.21.15.188:3300/0 conn(0x7fe7d80cc8e0 0x7fe7d81625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 -- 172.21.15.148:0/4229492099 >> 172.21.15.148:0/4229492099 conn(0x7fe7d806f590 msgr2=0x7fe7d806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 -- 172.21.15.148:0/4229492099 shutdown_connections 2024-04-25T10:00:28.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.857+0000 7fe7df1f5640 1 -- 172.21.15.148:0/4229492099 wait complete. 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.858+0000 7fe7df1f5640 1 Processor -- start 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 -- start start 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7d80cc8e0 0x7fe7d8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d80d1810 0x7fe7d80b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 0x7fe7d815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d815e070 con 0x7fe7d815dc90 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d80cd130 con 0x7fe7d80d1810 2024-04-25T10:00:28.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7df1f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d815e1f0 con 0x7fe7d80cc8e0 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.859+0000 7fe7dd792640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 0x7fe7d815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 0x7fe7d815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50328/0 (socket says 172.21.15.148:50328) 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 -- 172.21.15.148:0/2499286758 learned_addr learned my addr 172.21.15.148:0/2499286758 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7d7fff640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d80d1810 0x7fe7d80b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dcf91640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7d80cc8e0 0x7fe7d8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 -- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7d80cc8e0 msgr2=0x7fe7d8155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7d80cc8e0 0x7fe7d8155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 -- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d80d1810 msgr2=0x7fe7d80b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d80d1810 0x7fe7d80b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 -- 172.21.15.148:0/2499286758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d815b570 con 0x7fe7d815dc90 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dcf91640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7d80cc8e0 0x7fe7d8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7d7fff640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d80d1810 0x7fe7d80b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:28.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7dd792640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 0x7fe7d815ac70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009f40 tx=0x7fe7d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0036050 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0042070 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d815b850 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0009510 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.860+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d80b49e0 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.861+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d8073990 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.862+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7d0007bf0 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.862+0000 7fe7d57fa640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.862+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7d00ab210 con 0x7fe7d815dc90 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.862+0000 7fe7dcf91640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:28.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.862+0000 7fe7dcf91640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00a830 tx=0x7fe7cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:28.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:28.866+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7d0079940 con 0x7fe7d815dc90 2024-04-25T10:00:29.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.034+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7d80cf6b0 con 0x7fe7bc06b4d0 2024-04-25T10:00:29.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.040+0000 7fe7d57fa640 1 -- 172.21.15.148:0/2499286758 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246298 (secure 0 0 0) 0x7fe7d80cf6b0 con 0x7fe7bc06b4d0 2024-04-25T10:00:29.047 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:29.049 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7bc06b4d0 msgr2=0x7fe7bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00a830 tx=0x7fe7cc008040 comp rx=0 tx=0).stop 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 msgr2=0x7fe7d815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 0x7fe7d815ac70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009f40 tx=0x7fe7d0004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 shutdown_connections 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe7bc06b4d0 0x7fe7bc06d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d815dc90 0x7fe7d815ac70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d80d1810 0x7fe7d80b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 --2- 172.21.15.148:0/2499286758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7d80cc8e0 0x7fe7d8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 >> 172.21.15.148:0/2499286758 conn(0x7fe7d806f590 msgr2=0x7fe7d80c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 shutdown_connections 2024-04-25T10:00:29.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.043+0000 7fe7df1f5640 1 -- 172.21.15.148:0/2499286758 wait complete. 2024-04-25T10:00:29.058 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-04-25T10:00:27.094938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":192,"acting":192,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374880,"kb_used_data":10216,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599840,"statfs":{"total":773094113280,"available":772710236160,"internally_reserved":0,"allocated":10461184,"data_stored":5458264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000761"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082127+0000","last_change":"2024-04-25T10:00:25.082127+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082127+0000","last_undegraded":"2024-04-25T10:00:25.082127+0000","last_fullsized":"2024-04-25T10:00:25.082127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081711+0000","last_change":"2024-04-25T10:00:25.081711+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081711+0000","last_undegraded":"2024-04-25T10:00:25.081711+0000","last_fullsized":"2024-04-25T10:00:25.081711+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086478+0000","last_change":"2024-04-25T10:00:25.086478+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086478+0000","last_undegraded":"2024-04-25T10:00:25.086478+0000","last_fullsized":"2024-04-25T10:00:25.086478+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:00:25.073196+0000","last_change":"2024-04-25T10:00:25.073196+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.073196+0000","last_undegraded":"2024-04-25T10:00:25.073196+0000","last_fullsized":"2024-04-25T10:00:25.073196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087074+0000","last_change":"2024-04-25T10:00:25.087074+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087074+0000","last_undegraded":"2024-04-25T10:00:25.087074+0000","last_fullsized":"2024-04-25T10:00:25.087074+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085115+0000","last_change":"2024-04-25T10:00:25.085115+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085115+0000","last_undegraded":"2024-04-25T10:00:25.085115+0000","last_fullsized":"2024-04-25T10:00:25.085115+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085698+0000","last_change":"2024-04-25T10:00:25.085698+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085698+0000","last_undegraded":"2024-04-25T10:00:25.085698+0000","last_fullsized":"2024-04-25T10:00:25.085698+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086878+0000","last_change":"2024-04-25T10:00:25.086878+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086878+0000","last_undegraded":"2024-04-25T10:00:25.086878+0000","last_fullsized":"2024-04-25T10:00:25.086878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084003+0000","last_change":"2024-04-25T10:00:25.084003+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084003+0000","last_undegraded":"2024-04-25T10:00:25.084003+0000","last_fullsized":"2024-04-25T10:00:25.084003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086635+0000","last_change":"2024-04-25T10:00:25.086635+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086635+0000","last_undegraded":"2024-04-25T10:00:25.086635+0000","last_fullsized":"2024-04-25T10:00:25.086635+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083320+0000","last_change":"2024-04-25T10:00:25.083320+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083320+0000","last_undegraded":"2024-04-25T10:00:25.083320+0000","last_fullsized":"2024-04-25T10:00:25.083320+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084513+0000","last_change":"2024-04-25T10:00:25.084513+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084513+0000","last_undegraded":"2024-04-25T10:00:25.084513+0000","last_fullsized":"2024-04-25T10:00:25.084513+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086055+0000","last_change":"2024-04-25T10:00:25.086055+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086055+0000","last_undegraded":"2024-04-25T10:00:25.086055+0000","last_fullsized":"2024-04-25T10:00:25.086055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085240+0000","last_change":"2024-04-25T10:00:25.085240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085240+0000","last_undegraded":"2024-04-25T10:00:25.085240+0000","last_fullsized":"2024-04-25T10:00:25.085240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084527+0000","last_change":"2024-04-25T10:00:25.084527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084527+0000","last_undegraded":"2024-04-25T10:00:25.084527+0000","last_fullsized":"2024-04-25T10:00:25.084527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081678+0000","last_change":"2024-04-25T10:00:25.081678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081678+0000","last_undegraded":"2024-04-25T10:00:25.081678+0000","last_fullsized":"2024-04-25T10:00:25.081678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086106+0000","last_change":"2024-04-25T10:00:25.086106+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086106+0000","last_undegraded":"2024-04-25T10:00:25.086106+0000","last_fullsized":"2024-04-25T10:00:25.086106+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083550+0000","last_change":"2024-04-25T10:00:25.083550+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083550+0000","last_undegraded":"2024-04-25T10:00:25.083550+0000","last_fullsized":"2024-04-25T10:00:25.083550+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086039+0000","last_change":"2024-04-25T10:00:25.086039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086039+0000","last_undegraded":"2024-04-25T10:00:25.086039+0000","last_fullsized":"2024-04-25T10:00:25.086039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085527+0000","last_change":"2024-04-25T10:00:25.085527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085527+0000","last_undegraded":"2024-04-25T10:00:25.085527+0000","last_fullsized":"2024-04-25T10:00:25.085527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081240+0000","last_change":"2024-04-25T10:00:25.081240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081240+0000","last_undegraded":"2024-04-25T10:00:25.081240+0000","last_fullsized":"2024-04-25T10:00:25.081240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086088+0000","last_change":"2024-04-25T10:00:25.086088+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086088+0000","last_undegraded":"2024-04-25T10:00:25.086088+0000","last_fullsized":"2024-04-25T10:00:25.086088+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086491+0000","last_change":"2024-04-25T10:00:25.086491+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086491+0000","last_undegraded":"2024-04-25T10:00:25.086491+0000","last_fullsized":"2024-04-25T10:00:25.086491+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083164+0000","last_change":"2024-04-25T10:00:25.083164+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083164+0000","last_undegraded":"2024-04-25T10:00:25.083164+0000","last_fullsized":"2024-04-25T10:00:25.083164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083140+0000","last_change":"2024-04-25T10:00:25.083140+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083140+0000","last_undegraded":"2024-04-25T10:00:25.083140+0000","last_fullsized":"2024-04-25T10:00:25.083140+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082536+0000","last_change":"2024-04-25T10:00:25.082536+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082536+0000","last_undegraded":"2024-04-25T10:00:25.082536+0000","last_fullsized":"2024-04-25T10:00:25.082536+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084678+0000","last_change":"2024-04-25T10:00:25.084678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084678+0000","last_undegraded":"2024-04-25T10:00:25.084678+0000","last_fullsized":"2024-04-25T10:00:25.084678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083510+0000","last_change":"2024-04-25T10:00:25.083510+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083510+0000","last_undegraded":"2024-04-25T10:00:25.083510+0000","last_fullsized":"2024-04-25T10:00:25.083510+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082308+0000","last_change":"2024-04-25T10:00:25.082308+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082308+0000","last_undegraded":"2024-04-25T10:00:25.082308+0000","last_fullsized":"2024-04-25T10:00:25.082308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083370+0000","last_change":"2024-04-25T10:00:25.083370+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083370+0000","last_undegraded":"2024-04-25T10:00:25.083370+0000","last_fullsized":"2024-04-25T10:00:25.083370+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100472+0000","last_change":"2024-04-25T10:00:25.098352+0000","last_active":"2024-04-25T10:00:25.100472+0000","last_peered":"2024-04-25T10:00:25.100472+0000","last_clean":"2024-04-25T10:00:25.100472+0000","last_became_active":"2024-04-25T10:00:25.098081+0000","last_became_peered":"2024-04-25T10:00:25.098081+0000","last_unstale":"2024-04-25T10:00:25.100472+0000","last_undegraded":"2024-04-25T10:00:25.100472+0000","last_fullsized":"2024-04-25T10:00:25.100472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:10.923042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081425+0000","last_change":"2024-04-25T10:00:25.081425+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081425+0000","last_undegraded":"2024-04-25T10:00:25.081425+0000","last_fullsized":"2024-04-25T10:00:25.081425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085428+0000","last_change":"2024-04-25T10:00:25.085428+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085428+0000","last_undegraded":"2024-04-25T10:00:25.085428+0000","last_fullsized":"2024-04-25T10:00:25.085428+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082454+0000","last_change":"2024-04-25T10:00:25.082454+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082454+0000","last_undegraded":"2024-04-25T10:00:25.082454+0000","last_fullsized":"2024-04-25T10:00:25.082454+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088926+0000","last_change":"2024-04-25T10:00:25.087470+0000","last_active":"2024-04-25T10:00:25.088926+0000","last_peered":"2024-04-25T10:00:25.088926+0000","last_clean":"2024-04-25T10:00:25.088926+0000","last_became_active":"2024-04-25T10:00:25.087191+0000","last_became_peered":"2024-04-25T10:00:25.087191+0000","last_unstale":"2024-04-25T10:00:25.088926+0000","last_undegraded":"2024-04-25T10:00:25.088926+0000","last_fullsized":"2024-04-25T10:00:25.088926+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:29.174918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081884+0000","last_change":"2024-04-25T10:00:25.081884+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081884+0000","last_undegraded":"2024-04-25T10:00:25.081884+0000","last_fullsized":"2024-04-25T10:00:25.081884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100393+0000","last_change":"2024-04-25T10:00:25.096412+0000","last_active":"2024-04-25T10:00:25.100393+0000","last_peered":"2024-04-25T10:00:25.100393+0000","last_clean":"2024-04-25T10:00:25.100393+0000","last_became_active":"2024-04-25T10:00:25.095816+0000","last_became_peered":"2024-04-25T10:00:25.095816+0000","last_unstale":"2024-04-25T10:00:25.100393+0000","last_undegraded":"2024-04-25T10:00:25.100393+0000","last_fullsized":"2024-04-25T10:00:25.100393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:34.162551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084889+0000","last_change":"2024-04-25T10:00:25.084889+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084889+0000","last_undegraded":"2024-04-25T10:00:25.084889+0000","last_fullsized":"2024-04-25T10:00:25.084889+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084057+0000","last_change":"2024-04-25T10:00:25.084057+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084057+0000","last_undegraded":"2024-04-25T10:00:25.084057+0000","last_fullsized":"2024-04-25T10:00:25.084057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.088983+0000","last_change":"2024-04-25T10:00:25.088983+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.088983+0000","last_undegraded":"2024-04-25T10:00:25.088983+0000","last_fullsized":"2024-04-25T10:00:25.088983+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092854+0000","last_change":"2024-04-25T10:00:25.090629+0000","last_active":"2024-04-25T10:00:25.092854+0000","last_peered":"2024-04-25T10:00:25.092854+0000","last_clean":"2024-04-25T10:00:25.092854+0000","last_became_active":"2024-04-25T10:00:25.090354+0000","last_became_peered":"2024-04-25T10:00:25.090354+0000","last_unstale":"2024-04-25T10:00:25.092854+0000","last_undegraded":"2024-04-25T10:00:25.092854+0000","last_fullsized":"2024-04-25T10:00:25.092854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:06.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082091+0000","last_change":"2024-04-25T10:00:25.082091+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082091+0000","last_undegraded":"2024-04-25T10:00:25.082091+0000","last_fullsized":"2024-04-25T10:00:25.082091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088390+0000","last_change":"2024-04-25T10:00:25.086108+0000","last_active":"2024-04-25T10:00:25.088390+0000","last_peered":"2024-04-25T10:00:25.088390+0000","last_clean":"2024-04-25T10:00:25.088390+0000","last_became_active":"2024-04-25T10:00:25.085829+0000","last_became_peered":"2024-04-25T10:00:25.085829+0000","last_unstale":"2024-04-25T10:00:25.088390+0000","last_undegraded":"2024-04-25T10:00:25.088390+0000","last_fullsized":"2024-04-25T10:00:25.088390+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083970+0000","last_change":"2024-04-25T10:00:25.083970+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083970+0000","last_undegraded":"2024-04-25T10:00:25.083970+0000","last_fullsized":"2024-04-25T10:00:25.083970+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098752+0000","last_change":"2024-04-25T10:00:25.097674+0000","last_active":"2024-04-25T10:00:25.098752+0000","last_peered":"2024-04-25T10:00:25.098752+0000","last_clean":"2024-04-25T10:00:25.098752+0000","last_became_active":"2024-04-25T10:00:25.097383+0000","last_became_peered":"2024-04-25T10:00:25.097383+0000","last_unstale":"2024-04-25T10:00:25.098752+0000","last_undegraded":"2024-04-25T10:00:25.098752+0000","last_fullsized":"2024-04-25T10:00:25.098752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084573+0000","last_change":"2024-04-25T10:00:25.084573+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084573+0000","last_undegraded":"2024-04-25T10:00:25.084573+0000","last_fullsized":"2024-04-25T10:00:25.084573+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098649+0000","last_change":"2024-04-25T10:00:25.096537+0000","last_active":"2024-04-25T10:00:25.098649+0000","last_peered":"2024-04-25T10:00:25.098649+0000","last_clean":"2024-04-25T10:00:25.098649+0000","last_became_active":"2024-04-25T10:00:25.096221+0000","last_became_peered":"2024-04-25T10:00:25.096221+0000","last_unstale":"2024-04-25T10:00:25.098649+0000","last_undegraded":"2024-04-25T10:00:25.098649+0000","last_fullsized":"2024-04-25T10:00:25.098649+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082880+0000","last_change":"2024-04-25T10:00:25.082880+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082880+0000","last_undegraded":"2024-04-25T10:00:25.082880+0000","last_fullsized":"2024-04-25T10:00:25.082880+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094336+0000","last_change":"2024-04-25T10:00:25.093114+0000","last_active":"2024-04-25T10:00:25.094336+0000","last_peered":"2024-04-25T10:00:25.094336+0000","last_clean":"2024-04-25T10:00:25.094336+0000","last_became_active":"2024-04-25T10:00:25.092835+0000","last_became_peered":"2024-04-25T10:00:25.092835+0000","last_unstale":"2024-04-25T10:00:25.094336+0000","last_undegraded":"2024-04-25T10:00:25.094336+0000","last_fullsized":"2024-04-25T10:00:25.094336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:40.180045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081242+0000","last_change":"2024-04-25T10:00:25.081242+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081242+0000","last_undegraded":"2024-04-25T10:00:25.081242+0000","last_fullsized":"2024-04-25T10:00:25.081242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084272+0000","last_change":"2024-04-25T10:00:25.084272+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084272+0000","last_undegraded":"2024-04-25T10:00:25.084272+0000","last_fullsized":"2024-04-25T10:00:25.084272+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082937+0000","last_change":"2024-04-25T10:00:25.082937+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082937+0000","last_undegraded":"2024-04-25T10:00:25.082937+0000","last_fullsized":"2024-04-25T10:00:25.082937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.097897+0000","last_change":"2024-04-25T10:00:25.097116+0000","last_active":"2024-04-25T10:00:25.097897+0000","last_peered":"2024-04-25T10:00:25.097897+0000","last_clean":"2024-04-25T10:00:25.097897+0000","last_became_active":"2024-04-25T10:00:25.096795+0000","last_became_peered":"2024-04-25T10:00:25.096795+0000","last_unstale":"2024-04-25T10:00:25.097897+0000","last_undegraded":"2024-04-25T10:00:25.097897+0000","last_fullsized":"2024-04-25T10:00:25.097897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.429544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084469+0000","last_change":"2024-04-25T10:00:25.084469+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084469+0000","last_undegraded":"2024-04-25T10:00:25.084469+0000","last_fullsized":"2024-04-25T10:00:25.084469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098728+0000","last_change":"2024-04-25T10:00:25.096994+0000","last_active":"2024-04-25T10:00:25.098728+0000","last_peered":"2024-04-25T10:00:25.098728+0000","last_clean":"2024-04-25T10:00:25.098728+0000","last_became_active":"2024-04-25T10:00:25.096686+0000","last_became_peered":"2024-04-25T10:00:25.096686+0000","last_unstale":"2024-04-25T10:00:25.098728+0000","last_undegraded":"2024-04-25T10:00:25.098728+0000","last_fullsized":"2024-04-25T10:00:25.098728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084087+0000","last_change":"2024-04-25T10:00:25.084087+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084087+0000","last_undegraded":"2024-04-25T10:00:25.084087+0000","last_fullsized":"2024-04-25T10:00:25.084087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094843+0000","last_change":"2024-04-25T10:00:25.092552+0000","last_active":"2024-04-25T10:00:25.094843+0000","last_peered":"2024-04-25T10:00:25.094843+0000","last_clean":"2024-04-25T10:00:25.094843+0000","last_became_active":"2024-04-25T10:00:25.092297+0000","last_became_peered":"2024-04-25T10:00:25.092297+0000","last_unstale":"2024-04-25T10:00:25.094843+0000","last_undegraded":"2024-04-25T10:00:25.094843+0000","last_fullsized":"2024-04-25T10:00:25.094843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:20.732580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080612+0000","last_change":"2024-04-25T10:00:25.080612+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080612+0000","last_undegraded":"2024-04-25T10:00:25.080612+0000","last_fullsized":"2024-04-25T10:00:25.080612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096185+0000","last_change":"2024-04-25T10:00:25.094479+0000","last_active":"2024-04-25T10:00:25.096185+0000","last_peered":"2024-04-25T10:00:25.096185+0000","last_clean":"2024-04-25T10:00:25.096185+0000","last_became_active":"2024-04-25T10:00:25.094171+0000","last_became_peered":"2024-04-25T10:00:25.094171+0000","last_unstale":"2024-04-25T10:00:25.096185+0000","last_undegraded":"2024-04-25T10:00:25.096185+0000","last_fullsized":"2024-04-25T10:00:25.096185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:51.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082611+0000","last_change":"2024-04-25T10:00:25.082611+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082611+0000","last_undegraded":"2024-04-25T10:00:25.082611+0000","last_fullsized":"2024-04-25T10:00:25.082611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092829+0000","last_change":"2024-04-25T10:00:25.091554+0000","last_active":"2024-04-25T10:00:25.092829+0000","last_peered":"2024-04-25T10:00:25.092829+0000","last_clean":"2024-04-25T10:00:25.092829+0000","last_became_active":"2024-04-25T10:00:25.091292+0000","last_became_peered":"2024-04-25T10:00:25.091292+0000","last_unstale":"2024-04-25T10:00:25.092829+0000","last_undegraded":"2024-04-25T10:00:25.092829+0000","last_fullsized":"2024-04-25T10:00:25.092829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:43.856414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083001+0000","last_change":"2024-04-25T10:00:25.083001+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083001+0000","last_undegraded":"2024-04-25T10:00:25.083001+0000","last_fullsized":"2024-04-25T10:00:25.083001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100381+0000","last_change":"2024-04-25T10:00:25.096404+0000","last_active":"2024-04-25T10:00:25.100381+0000","last_peered":"2024-04-25T10:00:25.100381+0000","last_clean":"2024-04-25T10:00:25.100381+0000","last_became_active":"2024-04-25T10:00:25.096120+0000","last_became_peered":"2024-04-25T10:00:25.096120+0000","last_unstale":"2024-04-25T10:00:25.100381+0000","last_undegraded":"2024-04-25T10:00:25.100381+0000","last_fullsized":"2024-04-25T10:00:25.100381+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.897809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.091793+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.091513+0000","last_became_peered":"2024-04-25T10:00:25.091513+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085785+0000","last_change":"2024-04-25T10:00:25.085785+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085785+0000","last_undegraded":"2024-04-25T10:00:25.085785+0000","last_fullsized":"2024-04-25T10:00:25.085785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083952+0000","last_change":"2024-04-25T10:00:25.083952+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083952+0000","last_undegraded":"2024-04-25T10:00:25.083952+0000","last_fullsized":"2024-04-25T10:00:25.083952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080419+0000","last_change":"2024-04-25T10:00:25.080419+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080419+0000","last_undegraded":"2024-04-25T10:00:25.080419+0000","last_fullsized":"2024-04-25T10:00:25.080419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096106+0000","last_change":"2024-04-25T10:00:25.093815+0000","last_active":"2024-04-25T10:00:25.096106+0000","last_peered":"2024-04-25T10:00:25.096106+0000","last_clean":"2024-04-25T10:00:25.096106+0000","last_became_active":"2024-04-25T10:00:25.093502+0000","last_became_peered":"2024-04-25T10:00:25.093502+0000","last_unstale":"2024-04-25T10:00:25.096106+0000","last_undegraded":"2024-04-25T10:00:25.096106+0000","last_fullsized":"2024-04-25T10:00:25.096106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083008+0000","last_change":"2024-04-25T10:00:25.083008+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083008+0000","last_undegraded":"2024-04-25T10:00:25.083008+0000","last_fullsized":"2024-04-25T10:00:25.083008+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094580+0000","last_change":"2024-04-25T10:00:25.093636+0000","last_active":"2024-04-25T10:00:25.094580+0000","last_peered":"2024-04-25T10:00:25.094580+0000","last_clean":"2024-04-25T10:00:25.094580+0000","last_became_active":"2024-04-25T10:00:25.093327+0000","last_became_peered":"2024-04-25T10:00:25.093327+0000","last_unstale":"2024-04-25T10:00:25.094580+0000","last_undegraded":"2024-04-25T10:00:25.094580+0000","last_fullsized":"2024-04-25T10:00:25.094580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:06.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087746+0000","last_change":"2024-04-25T10:00:25.087746+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087746+0000","last_undegraded":"2024-04-25T10:00:25.087746+0000","last_fullsized":"2024-04-25T10:00:25.087746+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098073+0000","last_change":"2024-04-25T10:00:25.094685+0000","last_active":"2024-04-25T10:00:25.098073+0000","last_peered":"2024-04-25T10:00:25.098073+0000","last_clean":"2024-04-25T10:00:25.098073+0000","last_became_active":"2024-04-25T10:00:25.094421+0000","last_became_peered":"2024-04-25T10:00:25.094421+0000","last_unstale":"2024-04-25T10:00:25.098073+0000","last_undegraded":"2024-04-25T10:00:25.098073+0000","last_fullsized":"2024-04-25T10:00:25.098073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081435+0000","last_change":"2024-04-25T10:00:25.081435+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081435+0000","last_undegraded":"2024-04-25T10:00:25.081435+0000","last_fullsized":"2024-04-25T10:00:25.081435+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.087730+0000","last_change":"2024-04-25T10:00:25.087285+0000","last_active":"2024-04-25T10:00:25.087730+0000","last_peered":"2024-04-25T10:00:25.087730+0000","last_clean":"2024-04-25T10:00:25.087730+0000","last_became_active":"2024-04-25T10:00:25.087017+0000","last_became_peered":"2024-04-25T10:00:25.087017+0000","last_unstale":"2024-04-25T10:00:25.087730+0000","last_undegraded":"2024-04-25T10:00:25.087730+0000","last_fullsized":"2024-04-25T10:00:25.087730+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:10.581104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.092039+0000","last_change":"2024-04-25T10:00:25.092039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.092039+0000","last_undegraded":"2024-04-25T10:00:25.092039+0000","last_fullsized":"2024-04-25T10:00:25.092039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098652+0000","last_change":"2024-04-25T10:00:25.095926+0000","last_active":"2024-04-25T10:00:25.098652+0000","last_peered":"2024-04-25T10:00:25.098652+0000","last_clean":"2024-04-25T10:00:25.098652+0000","last_became_active":"2024-04-25T10:00:25.095588+0000","last_became_peered":"2024-04-25T10:00:25.095588+0000","last_unstale":"2024-04-25T10:00:25.098652+0000","last_undegraded":"2024-04-25T10:00:25.098652+0000","last_fullsized":"2024-04-25T10:00:25.098652+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081036+0000","last_change":"2024-04-25T10:00:25.081036+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081036+0000","last_undegraded":"2024-04-25T10:00:25.081036+0000","last_fullsized":"2024-04-25T10:00:25.081036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085107+0000","last_change":"2024-04-25T10:00:25.085107+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085107+0000","last_undegraded":"2024-04-25T10:00:25.085107+0000","last_fullsized":"2024-04-25T10:00:25.085107+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080448+0000","last_change":"2024-04-25T10:00:25.080448+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080448+0000","last_undegraded":"2024-04-25T10:00:25.080448+0000","last_fullsized":"2024-04-25T10:00:25.080448+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083562+0000","last_change":"2024-04-25T10:00:25.083562+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083562+0000","last_undegraded":"2024-04-25T10:00:25.083562+0000","last_fullsized":"2024-04-25T10:00:25.083562+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085633+0000","last_change":"2024-04-25T10:00:25.085633+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085633+0000","last_undegraded":"2024-04-25T10:00:25.085633+0000","last_fullsized":"2024-04-25T10:00:25.085633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100414+0000","last_change":"2024-04-25T10:00:25.096895+0000","last_active":"2024-04-25T10:00:25.100414+0000","last_peered":"2024-04-25T10:00:25.100414+0000","last_clean":"2024-04-25T10:00:25.100414+0000","last_became_active":"2024-04-25T10:00:25.096545+0000","last_became_peered":"2024-04-25T10:00:25.096545+0000","last_unstale":"2024-04-25T10:00:25.100414+0000","last_undegraded":"2024-04-25T10:00:25.100414+0000","last_fullsized":"2024-04-25T10:00:25.100414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:51.071469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080086+0000","last_change":"2024-04-25T10:00:25.080086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080086+0000","last_undegraded":"2024-04-25T10:00:25.080086+0000","last_fullsized":"2024-04-25T10:00:25.080086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096760+0000","last_change":"2024-04-25T10:00:25.096062+0000","last_active":"2024-04-25T10:00:25.096760+0000","last_peered":"2024-04-25T10:00:25.096760+0000","last_clean":"2024-04-25T10:00:25.096760+0000","last_became_active":"2024-04-25T10:00:25.095754+0000","last_became_peered":"2024-04-25T10:00:25.095754+0000","last_unstale":"2024-04-25T10:00:25.096760+0000","last_undegraded":"2024-04-25T10:00:25.096760+0000","last_fullsized":"2024-04-25T10:00:25.096760+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:04.002047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081155+0000","last_change":"2024-04-25T10:00:25.081155+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081155+0000","last_undegraded":"2024-04-25T10:00:25.081155+0000","last_fullsized":"2024-04-25T10:00:25.081155+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.090791+0000","last_change":"2024-04-25T10:00:25.089349+0000","last_active":"2024-04-25T10:00:25.090791+0000","last_peered":"2024-04-25T10:00:25.090791+0000","last_clean":"2024-04-25T10:00:25.090791+0000","last_became_active":"2024-04-25T10:00:25.089074+0000","last_became_peered":"2024-04-25T10:00:25.089074+0000","last_unstale":"2024-04-25T10:00:25.090791+0000","last_undegraded":"2024-04-25T10:00:25.090791+0000","last_fullsized":"2024-04-25T10:00:25.090791+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.089230+0000","last_change":"2024-04-25T10:00:25.089230+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.089230+0000","last_undegraded":"2024-04-25T10:00:25.089230+0000","last_fullsized":"2024-04-25T10:00:25.089230+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100433+0000","last_change":"2024-04-25T10:00:25.097664+0000","last_active":"2024-04-25T10:00:25.100433+0000","last_peered":"2024-04-25T10:00:25.100433+0000","last_clean":"2024-04-25T10:00:25.100433+0000","last_became_active":"2024-04-25T10:00:25.097390+0000","last_became_peered":"2024-04-25T10:00:25.097390+0000","last_unstale":"2024-04-25T10:00:25.100433+0000","last_undegraded":"2024-04-25T10:00:25.100433+0000","last_fullsized":"2024-04-25T10:00:25.100433+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:24.773637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081086+0000","last_change":"2024-04-25T10:00:25.081086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081086+0000","last_undegraded":"2024-04-25T10:00:25.081086+0000","last_fullsized":"2024-04-25T10:00:25.081086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.092559+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.092275+0000","last_became_peered":"2024-04-25T10:00:25.092275+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081859+0000","last_change":"2024-04-25T10:00:25.081859+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081859+0000","last_undegraded":"2024-04-25T10:00:25.081859+0000","last_fullsized":"2024-04-25T10:00:25.081859+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092495+0000","last_change":"2024-04-25T10:00:25.090514+0000","last_active":"2024-04-25T10:00:25.092495+0000","last_peered":"2024-04-25T10:00:25.092495+0000","last_clean":"2024-04-25T10:00:25.092495+0000","last_became_active":"2024-04-25T10:00:25.090248+0000","last_became_peered":"2024-04-25T10:00:25.090248+0000","last_unstale":"2024-04-25T10:00:25.092495+0000","last_undegraded":"2024-04-25T10:00:25.092495+0000","last_fullsized":"2024-04-25T10:00:25.092495+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:34.979909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":308,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.083144+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:00:25.083144+0000","last_peered":"2024-04-25T10:00:25.083144+0000","last_clean":"2024-04-25T10:00:25.083144+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:00:25.083144+0000","last_undegraded":"2024-04-25T10:00:25.083144+0000","last_fullsized":"2024-04-25T10:00:25.083144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":126,"acting":126,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46780,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325060,"statfs":{"total":96636764160,"available":96588861440,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:29.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:29.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.191+0000 7fc8100e4640 1 Processor -- start 2024-04-25T10:00:29.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.191+0000 7fc8100e4640 1 -- start start 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc8100e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080b6cb0 0x7fc8080b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc8100e4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8080b5a60 0x7fc8080d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc8100e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8080ceee0 con 0x7fc80815db70 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc8100e4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8080d1d10 con 0x7fc8080b5a60 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc8100e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080d1ff0 con 0x7fc8080b6cb0 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8080b5a60 0x7fc8080d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc8080b5a60 0x7fc8080d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50338/0 (socket says 172.21.15.148:50338) 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 -- 172.21.15.148:0/131603768 learned_addr learned my addr 172.21.15.148:0/131603768 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 -- 172.21.15.148:0/131603768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080b6cb0 msgr2=0x7fc8080b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 --2- 172.21.15.148:0/131603768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080b6cb0 0x7fc8080b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 -- 172.21.15.148:0/131603768 >> v1:172.21.15.188:6789/0 conn(0x7fc80815db70 legacy=0x7fc808162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.192+0000 7fc80e681640 1 -- 172.21.15.148:0/131603768 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080d22d0 con 0x7fc8080b5a60 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.193+0000 7fc80e681640 1 --2- 172.21.15.148:0/131603768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080d27f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fc800009e50 tx=0x7fc80002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6520295bb07776ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.193+0000 7fc80ce7e640 1 -- 172.21.15.148:0/131603768 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800036070 con 0x7fc8080b5a60 2024-04-25T10:00:29.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.193+0000 7fc80ce7e640 1 -- 172.21.15.148:0/131603768 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc800031600 con 0x7fc8080b5a60 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.193+0000 7fc80ce7e640 1 -- 172.21.15.148:0/131603768 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000319d0 con 0x7fc8080b5a60 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 -- 172.21.15.148:0/131603768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 msgr2=0x7fc8080d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 --2- 172.21.15.148:0/131603768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080d27f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fc800009e50 tx=0x7fc80002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 -- 172.21.15.148:0/131603768 shutdown_connections 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 --2- 172.21.15.148:0/131603768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080d27f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 --2- 172.21.15.148:0/131603768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080b6cb0 0x7fc8080b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 -- 172.21.15.148:0/131603768 >> 172.21.15.148:0/131603768 conn(0x7fc8080d0520 msgr2=0x7fc8080d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 -- 172.21.15.148:0/131603768 shutdown_connections 2024-04-25T10:00:29.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.194+0000 7fc8100e4640 1 -- 172.21.15.148:0/131603768 wait complete. 2024-04-25T10:00:29.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.195+0000 7fc8100e4640 1 Processor -- start 2024-04-25T10:00:29.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.195+0000 7fc8100e4640 1 -- start start 2024-04-25T10:00:29.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080b6cb0 0x7fc808155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815db70 0x7fc8080cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080cf1c0 con 0x7fc8080b5a60 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080ce530 con 0x7fc8080b6cb0 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080cf340 con 0x7fc80815db70 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50342/0 (socket says 172.21.15.148:50342) 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 -- 172.21.15.148:0/4269387491 learned_addr learned my addr 172.21.15.148:0/4269387491 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80d67f640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080b6cb0 0x7fc808155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80e681640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815db70 0x7fc8080cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 -- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815db70 msgr2=0x7fc8080cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815db70 0x7fc8080cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 -- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080b6cb0 msgr2=0x7fc808155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080b6cb0 0x7fc808155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 -- 172.21.15.148:0/4269387491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80815ebe0 con 0x7fc8080b5a60 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80d67f640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080b6cb0 0x7fc808155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80e681640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815db70 0x7fc8080cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc80de80640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080b90d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b3f0 tx=0x7fc80400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804015020 con 0x7fc8080b5a60 2024-04-25T10:00:29.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804011830 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8080c9560 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.196+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804011c20 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.197+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8080c9b20 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.198+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808073990 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.198+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc804018020 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.198+0000 7fc7fe7fc640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7ec06b500 0x7fc7ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.198+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc804089f40 con 0x7fc8080b5a60 2024-04-25T10:00:29.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.198+0000 7fc80d67f640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7ec06b500 0x7fc7ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.199+0000 7fc80d67f640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7ec06b500 0x7fc7ec06d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc808161660 tx=0x7fc7f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.203+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc80401f030 con 0x7fc8080b5a60 2024-04-25T10:00:29.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.370+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc808155740 con 0x7fc7ec06b500 2024-04-25T10:00:29.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.376+0000 7fc7fe7fc640 1 -- 172.21.15.148:0/4269387491 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246298 (secure 0 0 0) 0x7fc808155740 con 0x7fc7ec06b500 2024-04-25T10:00:29.384 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:29.386 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:00:29.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7ec06b500 msgr2=0x7fc7ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7ec06b500 0x7fc7ec06d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc808161660 tx=0x7fc7f8008040 comp rx=0 tx=0).stop 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 msgr2=0x7fc8080b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080b90d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b3f0 tx=0x7fc80400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 shutdown_connections 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7ec06b500 0x7fc7ec06d9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815db70 0x7fc8080cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080b6cb0 0x7fc808155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 --2- 172.21.15.148:0/4269387491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8080b5a60 0x7fc8080b90d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 >> 172.21.15.148:0/4269387491 conn(0x7fc8080d0520 msgr2=0x7fc80815b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 shutdown_connections 2024-04-25T10:00:29.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.379+0000 7fc8100e4640 1 -- 172.21.15.148:0/4269387491 wait complete. 2024-04-25T10:00:29.394 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-25T10:00:29.095272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":192,"acting":192,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374880,"kb_used_data":10216,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599840,"statfs":{"total":773094113280,"available":772710236160,"internally_reserved":0,"allocated":10461184,"data_stored":5458264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021231"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082127+0000","last_change":"2024-04-25T10:00:25.082127+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082127+0000","last_undegraded":"2024-04-25T10:00:25.082127+0000","last_fullsized":"2024-04-25T10:00:25.082127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081711+0000","last_change":"2024-04-25T10:00:25.081711+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081711+0000","last_undegraded":"2024-04-25T10:00:25.081711+0000","last_fullsized":"2024-04-25T10:00:25.081711+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086478+0000","last_change":"2024-04-25T10:00:25.086478+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086478+0000","last_undegraded":"2024-04-25T10:00:25.086478+0000","last_fullsized":"2024-04-25T10:00:25.086478+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:00:25.073196+0000","last_change":"2024-04-25T10:00:25.073196+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.073196+0000","last_undegraded":"2024-04-25T10:00:25.073196+0000","last_fullsized":"2024-04-25T10:00:25.073196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087074+0000","last_change":"2024-04-25T10:00:25.087074+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087074+0000","last_undegraded":"2024-04-25T10:00:25.087074+0000","last_fullsized":"2024-04-25T10:00:25.087074+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085115+0000","last_change":"2024-04-25T10:00:25.085115+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085115+0000","last_undegraded":"2024-04-25T10:00:25.085115+0000","last_fullsized":"2024-04-25T10:00:25.085115+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085698+0000","last_change":"2024-04-25T10:00:25.085698+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085698+0000","last_undegraded":"2024-04-25T10:00:25.085698+0000","last_fullsized":"2024-04-25T10:00:25.085698+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086878+0000","last_change":"2024-04-25T10:00:25.086878+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086878+0000","last_undegraded":"2024-04-25T10:00:25.086878+0000","last_fullsized":"2024-04-25T10:00:25.086878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084003+0000","last_change":"2024-04-25T10:00:25.084003+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084003+0000","last_undegraded":"2024-04-25T10:00:25.084003+0000","last_fullsized":"2024-04-25T10:00:25.084003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086635+0000","last_change":"2024-04-25T10:00:25.086635+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086635+0000","last_undegraded":"2024-04-25T10:00:25.086635+0000","last_fullsized":"2024-04-25T10:00:25.086635+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083320+0000","last_change":"2024-04-25T10:00:25.083320+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083320+0000","last_undegraded":"2024-04-25T10:00:25.083320+0000","last_fullsized":"2024-04-25T10:00:25.083320+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084513+0000","last_change":"2024-04-25T10:00:25.084513+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084513+0000","last_undegraded":"2024-04-25T10:00:25.084513+0000","last_fullsized":"2024-04-25T10:00:25.084513+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086055+0000","last_change":"2024-04-25T10:00:25.086055+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086055+0000","last_undegraded":"2024-04-25T10:00:25.086055+0000","last_fullsized":"2024-04-25T10:00:25.086055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085240+0000","last_change":"2024-04-25T10:00:25.085240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085240+0000","last_undegraded":"2024-04-25T10:00:25.085240+0000","last_fullsized":"2024-04-25T10:00:25.085240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084527+0000","last_change":"2024-04-25T10:00:25.084527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084527+0000","last_undegraded":"2024-04-25T10:00:25.084527+0000","last_fullsized":"2024-04-25T10:00:25.084527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081678+0000","last_change":"2024-04-25T10:00:25.081678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081678+0000","last_undegraded":"2024-04-25T10:00:25.081678+0000","last_fullsized":"2024-04-25T10:00:25.081678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086106+0000","last_change":"2024-04-25T10:00:25.086106+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086106+0000","last_undegraded":"2024-04-25T10:00:25.086106+0000","last_fullsized":"2024-04-25T10:00:25.086106+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083550+0000","last_change":"2024-04-25T10:00:25.083550+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083550+0000","last_undegraded":"2024-04-25T10:00:25.083550+0000","last_fullsized":"2024-04-25T10:00:25.083550+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086039+0000","last_change":"2024-04-25T10:00:25.086039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086039+0000","last_undegraded":"2024-04-25T10:00:25.086039+0000","last_fullsized":"2024-04-25T10:00:25.086039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085527+0000","last_change":"2024-04-25T10:00:25.085527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085527+0000","last_undegraded":"2024-04-25T10:00:25.085527+0000","last_fullsized":"2024-04-25T10:00:25.085527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081240+0000","last_change":"2024-04-25T10:00:25.081240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081240+0000","last_undegraded":"2024-04-25T10:00:25.081240+0000","last_fullsized":"2024-04-25T10:00:25.081240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086088+0000","last_change":"2024-04-25T10:00:25.086088+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086088+0000","last_undegraded":"2024-04-25T10:00:25.086088+0000","last_fullsized":"2024-04-25T10:00:25.086088+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086491+0000","last_change":"2024-04-25T10:00:25.086491+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086491+0000","last_undegraded":"2024-04-25T10:00:25.086491+0000","last_fullsized":"2024-04-25T10:00:25.086491+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083164+0000","last_change":"2024-04-25T10:00:25.083164+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083164+0000","last_undegraded":"2024-04-25T10:00:25.083164+0000","last_fullsized":"2024-04-25T10:00:25.083164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083140+0000","last_change":"2024-04-25T10:00:25.083140+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083140+0000","last_undegraded":"2024-04-25T10:00:25.083140+0000","last_fullsized":"2024-04-25T10:00:25.083140+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082536+0000","last_change":"2024-04-25T10:00:25.082536+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082536+0000","last_undegraded":"2024-04-25T10:00:25.082536+0000","last_fullsized":"2024-04-25T10:00:25.082536+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084678+0000","last_change":"2024-04-25T10:00:25.084678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084678+0000","last_undegraded":"2024-04-25T10:00:25.084678+0000","last_fullsized":"2024-04-25T10:00:25.084678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083510+0000","last_change":"2024-04-25T10:00:25.083510+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083510+0000","last_undegraded":"2024-04-25T10:00:25.083510+0000","last_fullsized":"2024-04-25T10:00:25.083510+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082308+0000","last_change":"2024-04-25T10:00:25.082308+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082308+0000","last_undegraded":"2024-04-25T10:00:25.082308+0000","last_fullsized":"2024-04-25T10:00:25.082308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083370+0000","last_change":"2024-04-25T10:00:25.083370+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083370+0000","last_undegraded":"2024-04-25T10:00:25.083370+0000","last_fullsized":"2024-04-25T10:00:25.083370+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100472+0000","last_change":"2024-04-25T10:00:25.098352+0000","last_active":"2024-04-25T10:00:25.100472+0000","last_peered":"2024-04-25T10:00:25.100472+0000","last_clean":"2024-04-25T10:00:25.100472+0000","last_became_active":"2024-04-25T10:00:25.098081+0000","last_became_peered":"2024-04-25T10:00:25.098081+0000","last_unstale":"2024-04-25T10:00:25.100472+0000","last_undegraded":"2024-04-25T10:00:25.100472+0000","last_fullsized":"2024-04-25T10:00:25.100472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:10.923042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081425+0000","last_change":"2024-04-25T10:00:25.081425+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081425+0000","last_undegraded":"2024-04-25T10:00:25.081425+0000","last_fullsized":"2024-04-25T10:00:25.081425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085428+0000","last_change":"2024-04-25T10:00:25.085428+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085428+0000","last_undegraded":"2024-04-25T10:00:25.085428+0000","last_fullsized":"2024-04-25T10:00:25.085428+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082454+0000","last_change":"2024-04-25T10:00:25.082454+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082454+0000","last_undegraded":"2024-04-25T10:00:25.082454+0000","last_fullsized":"2024-04-25T10:00:25.082454+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088926+0000","last_change":"2024-04-25T10:00:25.087470+0000","last_active":"2024-04-25T10:00:25.088926+0000","last_peered":"2024-04-25T10:00:25.088926+0000","last_clean":"2024-04-25T10:00:25.088926+0000","last_became_active":"2024-04-25T10:00:25.087191+0000","last_became_peered":"2024-04-25T10:00:25.087191+0000","last_unstale":"2024-04-25T10:00:25.088926+0000","last_undegraded":"2024-04-25T10:00:25.088926+0000","last_fullsized":"2024-04-25T10:00:25.088926+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:29.174918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081884+0000","last_change":"2024-04-25T10:00:25.081884+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081884+0000","last_undegraded":"2024-04-25T10:00:25.081884+0000","last_fullsized":"2024-04-25T10:00:25.081884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100393+0000","last_change":"2024-04-25T10:00:25.096412+0000","last_active":"2024-04-25T10:00:25.100393+0000","last_peered":"2024-04-25T10:00:25.100393+0000","last_clean":"2024-04-25T10:00:25.100393+0000","last_became_active":"2024-04-25T10:00:25.095816+0000","last_became_peered":"2024-04-25T10:00:25.095816+0000","last_unstale":"2024-04-25T10:00:25.100393+0000","last_undegraded":"2024-04-25T10:00:25.100393+0000","last_fullsized":"2024-04-25T10:00:25.100393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:34.162551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084889+0000","last_change":"2024-04-25T10:00:25.084889+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084889+0000","last_undegraded":"2024-04-25T10:00:25.084889+0000","last_fullsized":"2024-04-25T10:00:25.084889+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084057+0000","last_change":"2024-04-25T10:00:25.084057+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084057+0000","last_undegraded":"2024-04-25T10:00:25.084057+0000","last_fullsized":"2024-04-25T10:00:25.084057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.088983+0000","last_change":"2024-04-25T10:00:25.088983+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.088983+0000","last_undegraded":"2024-04-25T10:00:25.088983+0000","last_fullsized":"2024-04-25T10:00:25.088983+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092854+0000","last_change":"2024-04-25T10:00:25.090629+0000","last_active":"2024-04-25T10:00:25.092854+0000","last_peered":"2024-04-25T10:00:25.092854+0000","last_clean":"2024-04-25T10:00:25.092854+0000","last_became_active":"2024-04-25T10:00:25.090354+0000","last_became_peered":"2024-04-25T10:00:25.090354+0000","last_unstale":"2024-04-25T10:00:25.092854+0000","last_undegraded":"2024-04-25T10:00:25.092854+0000","last_fullsized":"2024-04-25T10:00:25.092854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:06.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082091+0000","last_change":"2024-04-25T10:00:25.082091+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082091+0000","last_undegraded":"2024-04-25T10:00:25.082091+0000","last_fullsized":"2024-04-25T10:00:25.082091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088390+0000","last_change":"2024-04-25T10:00:25.086108+0000","last_active":"2024-04-25T10:00:25.088390+0000","last_peered":"2024-04-25T10:00:25.088390+0000","last_clean":"2024-04-25T10:00:25.088390+0000","last_became_active":"2024-04-25T10:00:25.085829+0000","last_became_peered":"2024-04-25T10:00:25.085829+0000","last_unstale":"2024-04-25T10:00:25.088390+0000","last_undegraded":"2024-04-25T10:00:25.088390+0000","last_fullsized":"2024-04-25T10:00:25.088390+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083970+0000","last_change":"2024-04-25T10:00:25.083970+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083970+0000","last_undegraded":"2024-04-25T10:00:25.083970+0000","last_fullsized":"2024-04-25T10:00:25.083970+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098752+0000","last_change":"2024-04-25T10:00:25.097674+0000","last_active":"2024-04-25T10:00:25.098752+0000","last_peered":"2024-04-25T10:00:25.098752+0000","last_clean":"2024-04-25T10:00:25.098752+0000","last_became_active":"2024-04-25T10:00:25.097383+0000","last_became_peered":"2024-04-25T10:00:25.097383+0000","last_unstale":"2024-04-25T10:00:25.098752+0000","last_undegraded":"2024-04-25T10:00:25.098752+0000","last_fullsized":"2024-04-25T10:00:25.098752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084573+0000","last_change":"2024-04-25T10:00:25.084573+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084573+0000","last_undegraded":"2024-04-25T10:00:25.084573+0000","last_fullsized":"2024-04-25T10:00:25.084573+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098649+0000","last_change":"2024-04-25T10:00:25.096537+0000","last_active":"2024-04-25T10:00:25.098649+0000","last_peered":"2024-04-25T10:00:25.098649+0000","last_clean":"2024-04-25T10:00:25.098649+0000","last_became_active":"2024-04-25T10:00:25.096221+0000","last_became_peered":"2024-04-25T10:00:25.096221+0000","last_unstale":"2024-04-25T10:00:25.098649+0000","last_undegraded":"2024-04-25T10:00:25.098649+0000","last_fullsized":"2024-04-25T10:00:25.098649+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082880+0000","last_change":"2024-04-25T10:00:25.082880+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082880+0000","last_undegraded":"2024-04-25T10:00:25.082880+0000","last_fullsized":"2024-04-25T10:00:25.082880+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094336+0000","last_change":"2024-04-25T10:00:25.093114+0000","last_active":"2024-04-25T10:00:25.094336+0000","last_peered":"2024-04-25T10:00:25.094336+0000","last_clean":"2024-04-25T10:00:25.094336+0000","last_became_active":"2024-04-25T10:00:25.092835+0000","last_became_peered":"2024-04-25T10:00:25.092835+0000","last_unstale":"2024-04-25T10:00:25.094336+0000","last_undegraded":"2024-04-25T10:00:25.094336+0000","last_fullsized":"2024-04-25T10:00:25.094336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:40.180045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081242+0000","last_change":"2024-04-25T10:00:25.081242+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081242+0000","last_undegraded":"2024-04-25T10:00:25.081242+0000","last_fullsized":"2024-04-25T10:00:25.081242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084272+0000","last_change":"2024-04-25T10:00:25.084272+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084272+0000","last_undegraded":"2024-04-25T10:00:25.084272+0000","last_fullsized":"2024-04-25T10:00:25.084272+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082937+0000","last_change":"2024-04-25T10:00:25.082937+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082937+0000","last_undegraded":"2024-04-25T10:00:25.082937+0000","last_fullsized":"2024-04-25T10:00:25.082937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.097897+0000","last_change":"2024-04-25T10:00:25.097116+0000","last_active":"2024-04-25T10:00:25.097897+0000","last_peered":"2024-04-25T10:00:25.097897+0000","last_clean":"2024-04-25T10:00:25.097897+0000","last_became_active":"2024-04-25T10:00:25.096795+0000","last_became_peered":"2024-04-25T10:00:25.096795+0000","last_unstale":"2024-04-25T10:00:25.097897+0000","last_undegraded":"2024-04-25T10:00:25.097897+0000","last_fullsized":"2024-04-25T10:00:25.097897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.429544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084469+0000","last_change":"2024-04-25T10:00:25.084469+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084469+0000","last_undegraded":"2024-04-25T10:00:25.084469+0000","last_fullsized":"2024-04-25T10:00:25.084469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098728+0000","last_change":"2024-04-25T10:00:25.096994+0000","last_active":"2024-04-25T10:00:25.098728+0000","last_peered":"2024-04-25T10:00:25.098728+0000","last_clean":"2024-04-25T10:00:25.098728+0000","last_became_active":"2024-04-25T10:00:25.096686+0000","last_became_peered":"2024-04-25T10:00:25.096686+0000","last_unstale":"2024-04-25T10:00:25.098728+0000","last_undegraded":"2024-04-25T10:00:25.098728+0000","last_fullsized":"2024-04-25T10:00:25.098728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084087+0000","last_change":"2024-04-25T10:00:25.084087+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084087+0000","last_undegraded":"2024-04-25T10:00:25.084087+0000","last_fullsized":"2024-04-25T10:00:25.084087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094843+0000","last_change":"2024-04-25T10:00:25.092552+0000","last_active":"2024-04-25T10:00:25.094843+0000","last_peered":"2024-04-25T10:00:25.094843+0000","last_clean":"2024-04-25T10:00:25.094843+0000","last_became_active":"2024-04-25T10:00:25.092297+0000","last_became_peered":"2024-04-25T10:00:25.092297+0000","last_unstale":"2024-04-25T10:00:25.094843+0000","last_undegraded":"2024-04-25T10:00:25.094843+0000","last_fullsized":"2024-04-25T10:00:25.094843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:20.732580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080612+0000","last_change":"2024-04-25T10:00:25.080612+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080612+0000","last_undegraded":"2024-04-25T10:00:25.080612+0000","last_fullsized":"2024-04-25T10:00:25.080612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096185+0000","last_change":"2024-04-25T10:00:25.094479+0000","last_active":"2024-04-25T10:00:25.096185+0000","last_peered":"2024-04-25T10:00:25.096185+0000","last_clean":"2024-04-25T10:00:25.096185+0000","last_became_active":"2024-04-25T10:00:25.094171+0000","last_became_peered":"2024-04-25T10:00:25.094171+0000","last_unstale":"2024-04-25T10:00:25.096185+0000","last_undegraded":"2024-04-25T10:00:25.096185+0000","last_fullsized":"2024-04-25T10:00:25.096185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:51.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082611+0000","last_change":"2024-04-25T10:00:25.082611+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082611+0000","last_undegraded":"2024-04-25T10:00:25.082611+0000","last_fullsized":"2024-04-25T10:00:25.082611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092829+0000","last_change":"2024-04-25T10:00:25.091554+0000","last_active":"2024-04-25T10:00:25.092829+0000","last_peered":"2024-04-25T10:00:25.092829+0000","last_clean":"2024-04-25T10:00:25.092829+0000","last_became_active":"2024-04-25T10:00:25.091292+0000","last_became_peered":"2024-04-25T10:00:25.091292+0000","last_unstale":"2024-04-25T10:00:25.092829+0000","last_undegraded":"2024-04-25T10:00:25.092829+0000","last_fullsized":"2024-04-25T10:00:25.092829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:43.856414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083001+0000","last_change":"2024-04-25T10:00:25.083001+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083001+0000","last_undegraded":"2024-04-25T10:00:25.083001+0000","last_fullsized":"2024-04-25T10:00:25.083001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100381+0000","last_change":"2024-04-25T10:00:25.096404+0000","last_active":"2024-04-25T10:00:25.100381+0000","last_peered":"2024-04-25T10:00:25.100381+0000","last_clean":"2024-04-25T10:00:25.100381+0000","last_became_active":"2024-04-25T10:00:25.096120+0000","last_became_peered":"2024-04-25T10:00:25.096120+0000","last_unstale":"2024-04-25T10:00:25.100381+0000","last_undegraded":"2024-04-25T10:00:25.100381+0000","last_fullsized":"2024-04-25T10:00:25.100381+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.897809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.091793+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.091513+0000","last_became_peered":"2024-04-25T10:00:25.091513+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085785+0000","last_change":"2024-04-25T10:00:25.085785+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085785+0000","last_undegraded":"2024-04-25T10:00:25.085785+0000","last_fullsized":"2024-04-25T10:00:25.085785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083952+0000","last_change":"2024-04-25T10:00:25.083952+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083952+0000","last_undegraded":"2024-04-25T10:00:25.083952+0000","last_fullsized":"2024-04-25T10:00:25.083952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080419+0000","last_change":"2024-04-25T10:00:25.080419+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080419+0000","last_undegraded":"2024-04-25T10:00:25.080419+0000","last_fullsized":"2024-04-25T10:00:25.080419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096106+0000","last_change":"2024-04-25T10:00:25.093815+0000","last_active":"2024-04-25T10:00:25.096106+0000","last_peered":"2024-04-25T10:00:25.096106+0000","last_clean":"2024-04-25T10:00:25.096106+0000","last_became_active":"2024-04-25T10:00:25.093502+0000","last_became_peered":"2024-04-25T10:00:25.093502+0000","last_unstale":"2024-04-25T10:00:25.096106+0000","last_undegraded":"2024-04-25T10:00:25.096106+0000","last_fullsized":"2024-04-25T10:00:25.096106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083008+0000","last_change":"2024-04-25T10:00:25.083008+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083008+0000","last_undegraded":"2024-04-25T10:00:25.083008+0000","last_fullsized":"2024-04-25T10:00:25.083008+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094580+0000","last_change":"2024-04-25T10:00:25.093636+0000","last_active":"2024-04-25T10:00:25.094580+0000","last_peered":"2024-04-25T10:00:25.094580+0000","last_clean":"2024-04-25T10:00:25.094580+0000","last_became_active":"2024-04-25T10:00:25.093327+0000","last_became_peered":"2024-04-25T10:00:25.093327+0000","last_unstale":"2024-04-25T10:00:25.094580+0000","last_undegraded":"2024-04-25T10:00:25.094580+0000","last_fullsized":"2024-04-25T10:00:25.094580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:06.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087746+0000","last_change":"2024-04-25T10:00:25.087746+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087746+0000","last_undegraded":"2024-04-25T10:00:25.087746+0000","last_fullsized":"2024-04-25T10:00:25.087746+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098073+0000","last_change":"2024-04-25T10:00:25.094685+0000","last_active":"2024-04-25T10:00:25.098073+0000","last_peered":"2024-04-25T10:00:25.098073+0000","last_clean":"2024-04-25T10:00:25.098073+0000","last_became_active":"2024-04-25T10:00:25.094421+0000","last_became_peered":"2024-04-25T10:00:25.094421+0000","last_unstale":"2024-04-25T10:00:25.098073+0000","last_undegraded":"2024-04-25T10:00:25.098073+0000","last_fullsized":"2024-04-25T10:00:25.098073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081435+0000","last_change":"2024-04-25T10:00:25.081435+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081435+0000","last_undegraded":"2024-04-25T10:00:25.081435+0000","last_fullsized":"2024-04-25T10:00:25.081435+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.087730+0000","last_change":"2024-04-25T10:00:25.087285+0000","last_active":"2024-04-25T10:00:25.087730+0000","last_peered":"2024-04-25T10:00:25.087730+0000","last_clean":"2024-04-25T10:00:25.087730+0000","last_became_active":"2024-04-25T10:00:25.087017+0000","last_became_peered":"2024-04-25T10:00:25.087017+0000","last_unstale":"2024-04-25T10:00:25.087730+0000","last_undegraded":"2024-04-25T10:00:25.087730+0000","last_fullsized":"2024-04-25T10:00:25.087730+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:10.581104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.092039+0000","last_change":"2024-04-25T10:00:25.092039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.092039+0000","last_undegraded":"2024-04-25T10:00:25.092039+0000","last_fullsized":"2024-04-25T10:00:25.092039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098652+0000","last_change":"2024-04-25T10:00:25.095926+0000","last_active":"2024-04-25T10:00:25.098652+0000","last_peered":"2024-04-25T10:00:25.098652+0000","last_clean":"2024-04-25T10:00:25.098652+0000","last_became_active":"2024-04-25T10:00:25.095588+0000","last_became_peered":"2024-04-25T10:00:25.095588+0000","last_unstale":"2024-04-25T10:00:25.098652+0000","last_undegraded":"2024-04-25T10:00:25.098652+0000","last_fullsized":"2024-04-25T10:00:25.098652+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081036+0000","last_change":"2024-04-25T10:00:25.081036+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081036+0000","last_undegraded":"2024-04-25T10:00:25.081036+0000","last_fullsized":"2024-04-25T10:00:25.081036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085107+0000","last_change":"2024-04-25T10:00:25.085107+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085107+0000","last_undegraded":"2024-04-25T10:00:25.085107+0000","last_fullsized":"2024-04-25T10:00:25.085107+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080448+0000","last_change":"2024-04-25T10:00:25.080448+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080448+0000","last_undegraded":"2024-04-25T10:00:25.080448+0000","last_fullsized":"2024-04-25T10:00:25.080448+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083562+0000","last_change":"2024-04-25T10:00:25.083562+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083562+0000","last_undegraded":"2024-04-25T10:00:25.083562+0000","last_fullsized":"2024-04-25T10:00:25.083562+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085633+0000","last_change":"2024-04-25T10:00:25.085633+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085633+0000","last_undegraded":"2024-04-25T10:00:25.085633+0000","last_fullsized":"2024-04-25T10:00:25.085633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100414+0000","last_change":"2024-04-25T10:00:25.096895+0000","last_active":"2024-04-25T10:00:25.100414+0000","last_peered":"2024-04-25T10:00:25.100414+0000","last_clean":"2024-04-25T10:00:25.100414+0000","last_became_active":"2024-04-25T10:00:25.096545+0000","last_became_peered":"2024-04-25T10:00:25.096545+0000","last_unstale":"2024-04-25T10:00:25.100414+0000","last_undegraded":"2024-04-25T10:00:25.100414+0000","last_fullsized":"2024-04-25T10:00:25.100414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:51.071469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080086+0000","last_change":"2024-04-25T10:00:25.080086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080086+0000","last_undegraded":"2024-04-25T10:00:25.080086+0000","last_fullsized":"2024-04-25T10:00:25.080086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096760+0000","last_change":"2024-04-25T10:00:25.096062+0000","last_active":"2024-04-25T10:00:25.096760+0000","last_peered":"2024-04-25T10:00:25.096760+0000","last_clean":"2024-04-25T10:00:25.096760+0000","last_became_active":"2024-04-25T10:00:25.095754+0000","last_became_peered":"2024-04-25T10:00:25.095754+0000","last_unstale":"2024-04-25T10:00:25.096760+0000","last_undegraded":"2024-04-25T10:00:25.096760+0000","last_fullsized":"2024-04-25T10:00:25.096760+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:04.002047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081155+0000","last_change":"2024-04-25T10:00:25.081155+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081155+0000","last_undegraded":"2024-04-25T10:00:25.081155+0000","last_fullsized":"2024-04-25T10:00:25.081155+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.090791+0000","last_change":"2024-04-25T10:00:25.089349+0000","last_active":"2024-04-25T10:00:25.090791+0000","last_peered":"2024-04-25T10:00:25.090791+0000","last_clean":"2024-04-25T10:00:25.090791+0000","last_became_active":"2024-04-25T10:00:25.089074+0000","last_became_peered":"2024-04-25T10:00:25.089074+0000","last_unstale":"2024-04-25T10:00:25.090791+0000","last_undegraded":"2024-04-25T10:00:25.090791+0000","last_fullsized":"2024-04-25T10:00:25.090791+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.089230+0000","last_change":"2024-04-25T10:00:25.089230+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.089230+0000","last_undegraded":"2024-04-25T10:00:25.089230+0000","last_fullsized":"2024-04-25T10:00:25.089230+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100433+0000","last_change":"2024-04-25T10:00:25.097664+0000","last_active":"2024-04-25T10:00:25.100433+0000","last_peered":"2024-04-25T10:00:25.100433+0000","last_clean":"2024-04-25T10:00:25.100433+0000","last_became_active":"2024-04-25T10:00:25.097390+0000","last_became_peered":"2024-04-25T10:00:25.097390+0000","last_unstale":"2024-04-25T10:00:25.100433+0000","last_undegraded":"2024-04-25T10:00:25.100433+0000","last_fullsized":"2024-04-25T10:00:25.100433+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:24.773637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081086+0000","last_change":"2024-04-25T10:00:25.081086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081086+0000","last_undegraded":"2024-04-25T10:00:25.081086+0000","last_fullsized":"2024-04-25T10:00:25.081086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.092559+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.092275+0000","last_became_peered":"2024-04-25T10:00:25.092275+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081859+0000","last_change":"2024-04-25T10:00:25.081859+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081859+0000","last_undegraded":"2024-04-25T10:00:25.081859+0000","last_fullsized":"2024-04-25T10:00:25.081859+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092495+0000","last_change":"2024-04-25T10:00:25.090514+0000","last_active":"2024-04-25T10:00:25.092495+0000","last_peered":"2024-04-25T10:00:25.092495+0000","last_clean":"2024-04-25T10:00:25.092495+0000","last_became_active":"2024-04-25T10:00:25.090248+0000","last_became_peered":"2024-04-25T10:00:25.090248+0000","last_unstale":"2024-04-25T10:00:25.092495+0000","last_undegraded":"2024-04-25T10:00:25.092495+0000","last_fullsized":"2024-04-25T10:00:25.092495+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:34.979909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":308,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.083144+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:00:25.083144+0000","last_peered":"2024-04-25T10:00:25.083144+0000","last_clean":"2024-04-25T10:00:25.083144+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:00:25.083144+0000","last_undegraded":"2024-04-25T10:00:25.083144+0000","last_fullsized":"2024-04-25T10:00:25.083144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":126,"acting":126,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46780,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325060,"statfs":{"total":96636764160,"available":96588861440,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:29.399 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:29.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.528+0000 7fb9010bc640 1 Processor -- start 2024-04-25T10:00:29.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.528+0000 7fb9010bc640 1 -- start start 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb9010bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc15dc90 0x7fb8fc1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb9010bc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8fc0d1810 0x7fb8fc0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb9010bc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc0cce50 con 0x7fb8fc0cc8e0 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb9010bc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc0d2180 con 0x7fb8fc0d1810 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb9010bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc161af0 con 0x7fb8fc15dc90 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fb59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8fc0d1810 0x7fb8fc0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc15dc90 0x7fb8fc1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8f3fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb8fc0cc8e0 0x7fb8fc15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43306/0 (socket says 172.21.15.148:43306) 2024-04-25T10:00:29.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc15dc90 0x7fb8fc1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:48454/0 (socket says 172.21.15.148:48454) 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fb59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8fc0d1810 0x7fb8fc0d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:37572/0 (socket says 172.21.15.148:37572) 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8f3fff640 1 -- 172.21.15.148:0/2895822621 learned_addr learned my addr 172.21.15.148:0/2895822621 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc15dc90 msgr2=0x7fb8fc1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fb59e640 1 --2- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc15dc90 0x7fb8fc1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2895822621 >> v1:172.21.15.188:6789/0 conn(0x7fb8fc0cc8e0 legacy=0x7fb8fc15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fa59c640 1 -- 172.21.15.148:0/2895822621 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603049451 0 0) 0x7fb8fc0cce50 con 0x7fb8fc0cc8e0 2024-04-25T10:00:29.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.529+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2895822621 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc161dd0 con 0x7fb8fc0d1810 2024-04-25T10:00:29.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.530+0000 7fb8fb59e640 1 --2- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0d1810 0x7fb8fc0d1c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009d90 tx=0x7fb8f402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dca8f4783dfc3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.530+0000 7fb8fa59c640 1 -- 172.21.15.148:0/2895822621 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4031600 con 0x7fb8fc0d1810 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.530+0000 7fb8fa59c640 1 -- 172.21.15.148:0/2895822621 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f40317a0 con 0x7fb8fc0d1810 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.530+0000 7fb8fa59c640 1 -- 172.21.15.148:0/2895822621 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4031b70 con 0x7fb8fc0d1810 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 -- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0d1810 msgr2=0x7fb8fc0d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0d1810 0x7fb8fc0d1c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009d90 tx=0x7fb8f402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 -- 172.21.15.148:0/2895822621 shutdown_connections 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0d1810 0x7fb8fc0d1c40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2895822621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc15dc90 0x7fb8fc1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 -- 172.21.15.148:0/2895822621 >> 172.21.15.148:0/2895822621 conn(0x7fb8fc06f590 msgr2=0x7fb8fc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 -- 172.21.15.148:0/2895822621 shutdown_connections 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.531+0000 7fb9010bc640 1 -- 172.21.15.148:0/2895822621 wait complete. 2024-04-25T10:00:29.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.532+0000 7fb9010bc640 1 Processor -- start 2024-04-25T10:00:29.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.532+0000 7fb9010bc640 1 -- start start 2024-04-25T10:00:29.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb9010bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0cc8e0 0x7fb8fc0b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb9010bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc0d1810 0x7fb8fc1f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb9010bc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 0x7fb8fc1f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb9010bc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc15e070 con 0x7fb8fc15dc90 2024-04-25T10:00:29.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb9010bc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc0cd130 con 0x7fb8fc0cc8e0 2024-04-25T10:00:29.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb9010bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc15e1f0 con 0x7fb8fc0d1810 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 0x7fb8fc1f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 0x7fb8fc1f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50352/0 (socket says 172.21.15.148:50352) 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2777887409 learned_addr learned my addr 172.21.15.148:0/2777887409 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fad9d640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0cc8e0 0x7fb8fc0b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8f3fff640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc0d1810 0x7fb8fc1f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc0d1810 msgr2=0x7fb8fc1f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc0d1810 0x7fb8fc1f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0cc8e0 msgr2=0x7fb8fc0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0cc8e0 0x7fb8fc0b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fb59e640 1 -- 172.21.15.148:0/2777887409 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc0b7180 con 0x7fb8fc15dc90 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.533+0000 7fb8fad9d640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0cc8e0 0x7fb8fc0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.534+0000 7fb8fb59e640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 0x7fb8fc1f4530 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009d90 tx=0x7fb8f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.534+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4040070 con 0x7fb8fc15dc90 2024-04-25T10:00:29.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.534+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f4003b90 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.534+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc0b7460 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.534+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f403c6f0 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.534+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc1e7870 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.535+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc073990 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.535+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8f4039040 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.536+0000 7fb8f37fe640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8dc06b4d0 0x7fb8dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.536+0000 7fb8fad9d640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8dc06b4d0 0x7fb8dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.536+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8f40aae00 con 0x7fb8fc15dc90 2024-04-25T10:00:29.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.536+0000 7fb8fad9d640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8dc06b4d0 0x7fb8dc06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc15cca0 tx=0x7fb8ec009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.540+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8f40794a0 con 0x7fb8fc15dc90 2024-04-25T10:00:29.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.706+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8fc1e7ab0 con 0x7fb8dc06b4d0 2024-04-25T10:00:29.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.711+0000 7fb8f37fe640 1 -- 172.21.15.148:0/2777887409 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246298 (secure 0 0 0) 0x7fb8ec011dd0 con 0x7fb8dc06b4d0 2024-04-25T10:00:29.719 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:29.720 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:00:29.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.714+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8dc06b4d0 msgr2=0x7fb8dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.714+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8dc06b4d0 0x7fb8dc06d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc15cca0 tx=0x7fb8ec009190 comp rx=0 tx=0).stop 2024-04-25T10:00:29.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 msgr2=0x7fb8fc1f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 0x7fb8fc1f4530 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009d90 tx=0x7fb8f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 shutdown_connections 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8dc06b4d0 0x7fb8dc06d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8fc15dc90 0x7fb8fc1f4530 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8fc0d1810 0x7fb8fc1f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 --2- 172.21.15.148:0/2777887409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8fc0cc8e0 0x7fb8fc0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 >> 172.21.15.148:0/2777887409 conn(0x7fb8fc06f590 msgr2=0x7fb8fc15ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 shutdown_connections 2024-04-25T10:00:29.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.715+0000 7fb9010bc640 1 -- 172.21.15.148:0/2777887409 wait complete. 2024-04-25T10:00:29.730 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-25T10:00:29.095272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":192,"acting":192,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374880,"kb_used_data":10216,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599840,"statfs":{"total":773094113280,"available":772710236160,"internally_reserved":0,"allocated":10461184,"data_stored":5458264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021231"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082127+0000","last_change":"2024-04-25T10:00:25.082127+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082127+0000","last_undegraded":"2024-04-25T10:00:25.082127+0000","last_fullsized":"2024-04-25T10:00:25.082127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081711+0000","last_change":"2024-04-25T10:00:25.081711+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081711+0000","last_undegraded":"2024-04-25T10:00:25.081711+0000","last_fullsized":"2024-04-25T10:00:25.081711+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086478+0000","last_change":"2024-04-25T10:00:25.086478+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086478+0000","last_undegraded":"2024-04-25T10:00:25.086478+0000","last_fullsized":"2024-04-25T10:00:25.086478+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:00:25.073196+0000","last_change":"2024-04-25T10:00:25.073196+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.073196+0000","last_undegraded":"2024-04-25T10:00:25.073196+0000","last_fullsized":"2024-04-25T10:00:25.073196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087074+0000","last_change":"2024-04-25T10:00:25.087074+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087074+0000","last_undegraded":"2024-04-25T10:00:25.087074+0000","last_fullsized":"2024-04-25T10:00:25.087074+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085115+0000","last_change":"2024-04-25T10:00:25.085115+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085115+0000","last_undegraded":"2024-04-25T10:00:25.085115+0000","last_fullsized":"2024-04-25T10:00:25.085115+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085698+0000","last_change":"2024-04-25T10:00:25.085698+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085698+0000","last_undegraded":"2024-04-25T10:00:25.085698+0000","last_fullsized":"2024-04-25T10:00:25.085698+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086878+0000","last_change":"2024-04-25T10:00:25.086878+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086878+0000","last_undegraded":"2024-04-25T10:00:25.086878+0000","last_fullsized":"2024-04-25T10:00:25.086878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084003+0000","last_change":"2024-04-25T10:00:25.084003+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084003+0000","last_undegraded":"2024-04-25T10:00:25.084003+0000","last_fullsized":"2024-04-25T10:00:25.084003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086635+0000","last_change":"2024-04-25T10:00:25.086635+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086635+0000","last_undegraded":"2024-04-25T10:00:25.086635+0000","last_fullsized":"2024-04-25T10:00:25.086635+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083320+0000","last_change":"2024-04-25T10:00:25.083320+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083320+0000","last_undegraded":"2024-04-25T10:00:25.083320+0000","last_fullsized":"2024-04-25T10:00:25.083320+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084513+0000","last_change":"2024-04-25T10:00:25.084513+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084513+0000","last_undegraded":"2024-04-25T10:00:25.084513+0000","last_fullsized":"2024-04-25T10:00:25.084513+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086055+0000","last_change":"2024-04-25T10:00:25.086055+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086055+0000","last_undegraded":"2024-04-25T10:00:25.086055+0000","last_fullsized":"2024-04-25T10:00:25.086055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085240+0000","last_change":"2024-04-25T10:00:25.085240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085240+0000","last_undegraded":"2024-04-25T10:00:25.085240+0000","last_fullsized":"2024-04-25T10:00:25.085240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084527+0000","last_change":"2024-04-25T10:00:25.084527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084527+0000","last_undegraded":"2024-04-25T10:00:25.084527+0000","last_fullsized":"2024-04-25T10:00:25.084527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081678+0000","last_change":"2024-04-25T10:00:25.081678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081678+0000","last_undegraded":"2024-04-25T10:00:25.081678+0000","last_fullsized":"2024-04-25T10:00:25.081678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086106+0000","last_change":"2024-04-25T10:00:25.086106+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086106+0000","last_undegraded":"2024-04-25T10:00:25.086106+0000","last_fullsized":"2024-04-25T10:00:25.086106+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083550+0000","last_change":"2024-04-25T10:00:25.083550+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083550+0000","last_undegraded":"2024-04-25T10:00:25.083550+0000","last_fullsized":"2024-04-25T10:00:25.083550+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086039+0000","last_change":"2024-04-25T10:00:25.086039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086039+0000","last_undegraded":"2024-04-25T10:00:25.086039+0000","last_fullsized":"2024-04-25T10:00:25.086039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085527+0000","last_change":"2024-04-25T10:00:25.085527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085527+0000","last_undegraded":"2024-04-25T10:00:25.085527+0000","last_fullsized":"2024-04-25T10:00:25.085527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081240+0000","last_change":"2024-04-25T10:00:25.081240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081240+0000","last_undegraded":"2024-04-25T10:00:25.081240+0000","last_fullsized":"2024-04-25T10:00:25.081240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086088+0000","last_change":"2024-04-25T10:00:25.086088+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086088+0000","last_undegraded":"2024-04-25T10:00:25.086088+0000","last_fullsized":"2024-04-25T10:00:25.086088+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086491+0000","last_change":"2024-04-25T10:00:25.086491+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086491+0000","last_undegraded":"2024-04-25T10:00:25.086491+0000","last_fullsized":"2024-04-25T10:00:25.086491+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083164+0000","last_change":"2024-04-25T10:00:25.083164+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083164+0000","last_undegraded":"2024-04-25T10:00:25.083164+0000","last_fullsized":"2024-04-25T10:00:25.083164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083140+0000","last_change":"2024-04-25T10:00:25.083140+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083140+0000","last_undegraded":"2024-04-25T10:00:25.083140+0000","last_fullsized":"2024-04-25T10:00:25.083140+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082536+0000","last_change":"2024-04-25T10:00:25.082536+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082536+0000","last_undegraded":"2024-04-25T10:00:25.082536+0000","last_fullsized":"2024-04-25T10:00:25.082536+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084678+0000","last_change":"2024-04-25T10:00:25.084678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084678+0000","last_undegraded":"2024-04-25T10:00:25.084678+0000","last_fullsized":"2024-04-25T10:00:25.084678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083510+0000","last_change":"2024-04-25T10:00:25.083510+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083510+0000","last_undegraded":"2024-04-25T10:00:25.083510+0000","last_fullsized":"2024-04-25T10:00:25.083510+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082308+0000","last_change":"2024-04-25T10:00:25.082308+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082308+0000","last_undegraded":"2024-04-25T10:00:25.082308+0000","last_fullsized":"2024-04-25T10:00:25.082308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083370+0000","last_change":"2024-04-25T10:00:25.083370+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083370+0000","last_undegraded":"2024-04-25T10:00:25.083370+0000","last_fullsized":"2024-04-25T10:00:25.083370+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100472+0000","last_change":"2024-04-25T10:00:25.098352+0000","last_active":"2024-04-25T10:00:25.100472+0000","last_peered":"2024-04-25T10:00:25.100472+0000","last_clean":"2024-04-25T10:00:25.100472+0000","last_became_active":"2024-04-25T10:00:25.098081+0000","last_became_peered":"2024-04-25T10:00:25.098081+0000","last_unstale":"2024-04-25T10:00:25.100472+0000","last_undegraded":"2024-04-25T10:00:25.100472+0000","last_fullsized":"2024-04-25T10:00:25.100472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:10.923042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081425+0000","last_change":"2024-04-25T10:00:25.081425+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081425+0000","last_undegraded":"2024-04-25T10:00:25.081425+0000","last_fullsized":"2024-04-25T10:00:25.081425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085428+0000","last_change":"2024-04-25T10:00:25.085428+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085428+0000","last_undegraded":"2024-04-25T10:00:25.085428+0000","last_fullsized":"2024-04-25T10:00:25.085428+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082454+0000","last_change":"2024-04-25T10:00:25.082454+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082454+0000","last_undegraded":"2024-04-25T10:00:25.082454+0000","last_fullsized":"2024-04-25T10:00:25.082454+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088926+0000","last_change":"2024-04-25T10:00:25.087470+0000","last_active":"2024-04-25T10:00:25.088926+0000","last_peered":"2024-04-25T10:00:25.088926+0000","last_clean":"2024-04-25T10:00:25.088926+0000","last_became_active":"2024-04-25T10:00:25.087191+0000","last_became_peered":"2024-04-25T10:00:25.087191+0000","last_unstale":"2024-04-25T10:00:25.088926+0000","last_undegraded":"2024-04-25T10:00:25.088926+0000","last_fullsized":"2024-04-25T10:00:25.088926+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:29.174918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081884+0000","last_change":"2024-04-25T10:00:25.081884+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081884+0000","last_undegraded":"2024-04-25T10:00:25.081884+0000","last_fullsized":"2024-04-25T10:00:25.081884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100393+0000","last_change":"2024-04-25T10:00:25.096412+0000","last_active":"2024-04-25T10:00:25.100393+0000","last_peered":"2024-04-25T10:00:25.100393+0000","last_clean":"2024-04-25T10:00:25.100393+0000","last_became_active":"2024-04-25T10:00:25.095816+0000","last_became_peered":"2024-04-25T10:00:25.095816+0000","last_unstale":"2024-04-25T10:00:25.100393+0000","last_undegraded":"2024-04-25T10:00:25.100393+0000","last_fullsized":"2024-04-25T10:00:25.100393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:34.162551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084889+0000","last_change":"2024-04-25T10:00:25.084889+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084889+0000","last_undegraded":"2024-04-25T10:00:25.084889+0000","last_fullsized":"2024-04-25T10:00:25.084889+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084057+0000","last_change":"2024-04-25T10:00:25.084057+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084057+0000","last_undegraded":"2024-04-25T10:00:25.084057+0000","last_fullsized":"2024-04-25T10:00:25.084057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.088983+0000","last_change":"2024-04-25T10:00:25.088983+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.088983+0000","last_undegraded":"2024-04-25T10:00:25.088983+0000","last_fullsized":"2024-04-25T10:00:25.088983+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092854+0000","last_change":"2024-04-25T10:00:25.090629+0000","last_active":"2024-04-25T10:00:25.092854+0000","last_peered":"2024-04-25T10:00:25.092854+0000","last_clean":"2024-04-25T10:00:25.092854+0000","last_became_active":"2024-04-25T10:00:25.090354+0000","last_became_peered":"2024-04-25T10:00:25.090354+0000","last_unstale":"2024-04-25T10:00:25.092854+0000","last_undegraded":"2024-04-25T10:00:25.092854+0000","last_fullsized":"2024-04-25T10:00:25.092854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:06.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082091+0000","last_change":"2024-04-25T10:00:25.082091+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082091+0000","last_undegraded":"2024-04-25T10:00:25.082091+0000","last_fullsized":"2024-04-25T10:00:25.082091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088390+0000","last_change":"2024-04-25T10:00:25.086108+0000","last_active":"2024-04-25T10:00:25.088390+0000","last_peered":"2024-04-25T10:00:25.088390+0000","last_clean":"2024-04-25T10:00:25.088390+0000","last_became_active":"2024-04-25T10:00:25.085829+0000","last_became_peered":"2024-04-25T10:00:25.085829+0000","last_unstale":"2024-04-25T10:00:25.088390+0000","last_undegraded":"2024-04-25T10:00:25.088390+0000","last_fullsized":"2024-04-25T10:00:25.088390+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083970+0000","last_change":"2024-04-25T10:00:25.083970+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083970+0000","last_undegraded":"2024-04-25T10:00:25.083970+0000","last_fullsized":"2024-04-25T10:00:25.083970+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098752+0000","last_change":"2024-04-25T10:00:25.097674+0000","last_active":"2024-04-25T10:00:25.098752+0000","last_peered":"2024-04-25T10:00:25.098752+0000","last_clean":"2024-04-25T10:00:25.098752+0000","last_became_active":"2024-04-25T10:00:25.097383+0000","last_became_peered":"2024-04-25T10:00:25.097383+0000","last_unstale":"2024-04-25T10:00:25.098752+0000","last_undegraded":"2024-04-25T10:00:25.098752+0000","last_fullsized":"2024-04-25T10:00:25.098752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084573+0000","last_change":"2024-04-25T10:00:25.084573+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084573+0000","last_undegraded":"2024-04-25T10:00:25.084573+0000","last_fullsized":"2024-04-25T10:00:25.084573+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098649+0000","last_change":"2024-04-25T10:00:25.096537+0000","last_active":"2024-04-25T10:00:25.098649+0000","last_peered":"2024-04-25T10:00:25.098649+0000","last_clean":"2024-04-25T10:00:25.098649+0000","last_became_active":"2024-04-25T10:00:25.096221+0000","last_became_peered":"2024-04-25T10:00:25.096221+0000","last_unstale":"2024-04-25T10:00:25.098649+0000","last_undegraded":"2024-04-25T10:00:25.098649+0000","last_fullsized":"2024-04-25T10:00:25.098649+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082880+0000","last_change":"2024-04-25T10:00:25.082880+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082880+0000","last_undegraded":"2024-04-25T10:00:25.082880+0000","last_fullsized":"2024-04-25T10:00:25.082880+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094336+0000","last_change":"2024-04-25T10:00:25.093114+0000","last_active":"2024-04-25T10:00:25.094336+0000","last_peered":"2024-04-25T10:00:25.094336+0000","last_clean":"2024-04-25T10:00:25.094336+0000","last_became_active":"2024-04-25T10:00:25.092835+0000","last_became_peered":"2024-04-25T10:00:25.092835+0000","last_unstale":"2024-04-25T10:00:25.094336+0000","last_undegraded":"2024-04-25T10:00:25.094336+0000","last_fullsized":"2024-04-25T10:00:25.094336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:40.180045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081242+0000","last_change":"2024-04-25T10:00:25.081242+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081242+0000","last_undegraded":"2024-04-25T10:00:25.081242+0000","last_fullsized":"2024-04-25T10:00:25.081242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084272+0000","last_change":"2024-04-25T10:00:25.084272+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084272+0000","last_undegraded":"2024-04-25T10:00:25.084272+0000","last_fullsized":"2024-04-25T10:00:25.084272+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082937+0000","last_change":"2024-04-25T10:00:25.082937+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082937+0000","last_undegraded":"2024-04-25T10:00:25.082937+0000","last_fullsized":"2024-04-25T10:00:25.082937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.097897+0000","last_change":"2024-04-25T10:00:25.097116+0000","last_active":"2024-04-25T10:00:25.097897+0000","last_peered":"2024-04-25T10:00:25.097897+0000","last_clean":"2024-04-25T10:00:25.097897+0000","last_became_active":"2024-04-25T10:00:25.096795+0000","last_became_peered":"2024-04-25T10:00:25.096795+0000","last_unstale":"2024-04-25T10:00:25.097897+0000","last_undegraded":"2024-04-25T10:00:25.097897+0000","last_fullsized":"2024-04-25T10:00:25.097897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.429544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084469+0000","last_change":"2024-04-25T10:00:25.084469+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084469+0000","last_undegraded":"2024-04-25T10:00:25.084469+0000","last_fullsized":"2024-04-25T10:00:25.084469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098728+0000","last_change":"2024-04-25T10:00:25.096994+0000","last_active":"2024-04-25T10:00:25.098728+0000","last_peered":"2024-04-25T10:00:25.098728+0000","last_clean":"2024-04-25T10:00:25.098728+0000","last_became_active":"2024-04-25T10:00:25.096686+0000","last_became_peered":"2024-04-25T10:00:25.096686+0000","last_unstale":"2024-04-25T10:00:25.098728+0000","last_undegraded":"2024-04-25T10:00:25.098728+0000","last_fullsized":"2024-04-25T10:00:25.098728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084087+0000","last_change":"2024-04-25T10:00:25.084087+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084087+0000","last_undegraded":"2024-04-25T10:00:25.084087+0000","last_fullsized":"2024-04-25T10:00:25.084087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094843+0000","last_change":"2024-04-25T10:00:25.092552+0000","last_active":"2024-04-25T10:00:25.094843+0000","last_peered":"2024-04-25T10:00:25.094843+0000","last_clean":"2024-04-25T10:00:25.094843+0000","last_became_active":"2024-04-25T10:00:25.092297+0000","last_became_peered":"2024-04-25T10:00:25.092297+0000","last_unstale":"2024-04-25T10:00:25.094843+0000","last_undegraded":"2024-04-25T10:00:25.094843+0000","last_fullsized":"2024-04-25T10:00:25.094843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:20.732580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080612+0000","last_change":"2024-04-25T10:00:25.080612+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080612+0000","last_undegraded":"2024-04-25T10:00:25.080612+0000","last_fullsized":"2024-04-25T10:00:25.080612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096185+0000","last_change":"2024-04-25T10:00:25.094479+0000","last_active":"2024-04-25T10:00:25.096185+0000","last_peered":"2024-04-25T10:00:25.096185+0000","last_clean":"2024-04-25T10:00:25.096185+0000","last_became_active":"2024-04-25T10:00:25.094171+0000","last_became_peered":"2024-04-25T10:00:25.094171+0000","last_unstale":"2024-04-25T10:00:25.096185+0000","last_undegraded":"2024-04-25T10:00:25.096185+0000","last_fullsized":"2024-04-25T10:00:25.096185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:51.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082611+0000","last_change":"2024-04-25T10:00:25.082611+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082611+0000","last_undegraded":"2024-04-25T10:00:25.082611+0000","last_fullsized":"2024-04-25T10:00:25.082611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092829+0000","last_change":"2024-04-25T10:00:25.091554+0000","last_active":"2024-04-25T10:00:25.092829+0000","last_peered":"2024-04-25T10:00:25.092829+0000","last_clean":"2024-04-25T10:00:25.092829+0000","last_became_active":"2024-04-25T10:00:25.091292+0000","last_became_peered":"2024-04-25T10:00:25.091292+0000","last_unstale":"2024-04-25T10:00:25.092829+0000","last_undegraded":"2024-04-25T10:00:25.092829+0000","last_fullsized":"2024-04-25T10:00:25.092829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:43.856414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083001+0000","last_change":"2024-04-25T10:00:25.083001+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083001+0000","last_undegraded":"2024-04-25T10:00:25.083001+0000","last_fullsized":"2024-04-25T10:00:25.083001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100381+0000","last_change":"2024-04-25T10:00:25.096404+0000","last_active":"2024-04-25T10:00:25.100381+0000","last_peered":"2024-04-25T10:00:25.100381+0000","last_clean":"2024-04-25T10:00:25.100381+0000","last_became_active":"2024-04-25T10:00:25.096120+0000","last_became_peered":"2024-04-25T10:00:25.096120+0000","last_unstale":"2024-04-25T10:00:25.100381+0000","last_undegraded":"2024-04-25T10:00:25.100381+0000","last_fullsized":"2024-04-25T10:00:25.100381+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.897809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.091793+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.091513+0000","last_became_peered":"2024-04-25T10:00:25.091513+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085785+0000","last_change":"2024-04-25T10:00:25.085785+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085785+0000","last_undegraded":"2024-04-25T10:00:25.085785+0000","last_fullsized":"2024-04-25T10:00:25.085785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083952+0000","last_change":"2024-04-25T10:00:25.083952+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083952+0000","last_undegraded":"2024-04-25T10:00:25.083952+0000","last_fullsized":"2024-04-25T10:00:25.083952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080419+0000","last_change":"2024-04-25T10:00:25.080419+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080419+0000","last_undegraded":"2024-04-25T10:00:25.080419+0000","last_fullsized":"2024-04-25T10:00:25.080419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096106+0000","last_change":"2024-04-25T10:00:25.093815+0000","last_active":"2024-04-25T10:00:25.096106+0000","last_peered":"2024-04-25T10:00:25.096106+0000","last_clean":"2024-04-25T10:00:25.096106+0000","last_became_active":"2024-04-25T10:00:25.093502+0000","last_became_peered":"2024-04-25T10:00:25.093502+0000","last_unstale":"2024-04-25T10:00:25.096106+0000","last_undegraded":"2024-04-25T10:00:25.096106+0000","last_fullsized":"2024-04-25T10:00:25.096106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083008+0000","last_change":"2024-04-25T10:00:25.083008+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083008+0000","last_undegraded":"2024-04-25T10:00:25.083008+0000","last_fullsized":"2024-04-25T10:00:25.083008+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094580+0000","last_change":"2024-04-25T10:00:25.093636+0000","last_active":"2024-04-25T10:00:25.094580+0000","last_peered":"2024-04-25T10:00:25.094580+0000","last_clean":"2024-04-25T10:00:25.094580+0000","last_became_active":"2024-04-25T10:00:25.093327+0000","last_became_peered":"2024-04-25T10:00:25.093327+0000","last_unstale":"2024-04-25T10:00:25.094580+0000","last_undegraded":"2024-04-25T10:00:25.094580+0000","last_fullsized":"2024-04-25T10:00:25.094580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:06.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087746+0000","last_change":"2024-04-25T10:00:25.087746+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087746+0000","last_undegraded":"2024-04-25T10:00:25.087746+0000","last_fullsized":"2024-04-25T10:00:25.087746+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098073+0000","last_change":"2024-04-25T10:00:25.094685+0000","last_active":"2024-04-25T10:00:25.098073+0000","last_peered":"2024-04-25T10:00:25.098073+0000","last_clean":"2024-04-25T10:00:25.098073+0000","last_became_active":"2024-04-25T10:00:25.094421+0000","last_became_peered":"2024-04-25T10:00:25.094421+0000","last_unstale":"2024-04-25T10:00:25.098073+0000","last_undegraded":"2024-04-25T10:00:25.098073+0000","last_fullsized":"2024-04-25T10:00:25.098073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081435+0000","last_change":"2024-04-25T10:00:25.081435+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081435+0000","last_undegraded":"2024-04-25T10:00:25.081435+0000","last_fullsized":"2024-04-25T10:00:25.081435+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.087730+0000","last_change":"2024-04-25T10:00:25.087285+0000","last_active":"2024-04-25T10:00:25.087730+0000","last_peered":"2024-04-25T10:00:25.087730+0000","last_clean":"2024-04-25T10:00:25.087730+0000","last_became_active":"2024-04-25T10:00:25.087017+0000","last_became_peered":"2024-04-25T10:00:25.087017+0000","last_unstale":"2024-04-25T10:00:25.087730+0000","last_undegraded":"2024-04-25T10:00:25.087730+0000","last_fullsized":"2024-04-25T10:00:25.087730+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:10.581104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.092039+0000","last_change":"2024-04-25T10:00:25.092039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.092039+0000","last_undegraded":"2024-04-25T10:00:25.092039+0000","last_fullsized":"2024-04-25T10:00:25.092039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098652+0000","last_change":"2024-04-25T10:00:25.095926+0000","last_active":"2024-04-25T10:00:25.098652+0000","last_peered":"2024-04-25T10:00:25.098652+0000","last_clean":"2024-04-25T10:00:25.098652+0000","last_became_active":"2024-04-25T10:00:25.095588+0000","last_became_peered":"2024-04-25T10:00:25.095588+0000","last_unstale":"2024-04-25T10:00:25.098652+0000","last_undegraded":"2024-04-25T10:00:25.098652+0000","last_fullsized":"2024-04-25T10:00:25.098652+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081036+0000","last_change":"2024-04-25T10:00:25.081036+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081036+0000","last_undegraded":"2024-04-25T10:00:25.081036+0000","last_fullsized":"2024-04-25T10:00:25.081036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085107+0000","last_change":"2024-04-25T10:00:25.085107+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085107+0000","last_undegraded":"2024-04-25T10:00:25.085107+0000","last_fullsized":"2024-04-25T10:00:25.085107+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080448+0000","last_change":"2024-04-25T10:00:25.080448+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080448+0000","last_undegraded":"2024-04-25T10:00:25.080448+0000","last_fullsized":"2024-04-25T10:00:25.080448+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083562+0000","last_change":"2024-04-25T10:00:25.083562+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083562+0000","last_undegraded":"2024-04-25T10:00:25.083562+0000","last_fullsized":"2024-04-25T10:00:25.083562+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085633+0000","last_change":"2024-04-25T10:00:25.085633+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085633+0000","last_undegraded":"2024-04-25T10:00:25.085633+0000","last_fullsized":"2024-04-25T10:00:25.085633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100414+0000","last_change":"2024-04-25T10:00:25.096895+0000","last_active":"2024-04-25T10:00:25.100414+0000","last_peered":"2024-04-25T10:00:25.100414+0000","last_clean":"2024-04-25T10:00:25.100414+0000","last_became_active":"2024-04-25T10:00:25.096545+0000","last_became_peered":"2024-04-25T10:00:25.096545+0000","last_unstale":"2024-04-25T10:00:25.100414+0000","last_undegraded":"2024-04-25T10:00:25.100414+0000","last_fullsized":"2024-04-25T10:00:25.100414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:51.071469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080086+0000","last_change":"2024-04-25T10:00:25.080086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080086+0000","last_undegraded":"2024-04-25T10:00:25.080086+0000","last_fullsized":"2024-04-25T10:00:25.080086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096760+0000","last_change":"2024-04-25T10:00:25.096062+0000","last_active":"2024-04-25T10:00:25.096760+0000","last_peered":"2024-04-25T10:00:25.096760+0000","last_clean":"2024-04-25T10:00:25.096760+0000","last_became_active":"2024-04-25T10:00:25.095754+0000","last_became_peered":"2024-04-25T10:00:25.095754+0000","last_unstale":"2024-04-25T10:00:25.096760+0000","last_undegraded":"2024-04-25T10:00:25.096760+0000","last_fullsized":"2024-04-25T10:00:25.096760+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:04.002047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081155+0000","last_change":"2024-04-25T10:00:25.081155+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081155+0000","last_undegraded":"2024-04-25T10:00:25.081155+0000","last_fullsized":"2024-04-25T10:00:25.081155+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.090791+0000","last_change":"2024-04-25T10:00:25.089349+0000","last_active":"2024-04-25T10:00:25.090791+0000","last_peered":"2024-04-25T10:00:25.090791+0000","last_clean":"2024-04-25T10:00:25.090791+0000","last_became_active":"2024-04-25T10:00:25.089074+0000","last_became_peered":"2024-04-25T10:00:25.089074+0000","last_unstale":"2024-04-25T10:00:25.090791+0000","last_undegraded":"2024-04-25T10:00:25.090791+0000","last_fullsized":"2024-04-25T10:00:25.090791+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.089230+0000","last_change":"2024-04-25T10:00:25.089230+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.089230+0000","last_undegraded":"2024-04-25T10:00:25.089230+0000","last_fullsized":"2024-04-25T10:00:25.089230+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100433+0000","last_change":"2024-04-25T10:00:25.097664+0000","last_active":"2024-04-25T10:00:25.100433+0000","last_peered":"2024-04-25T10:00:25.100433+0000","last_clean":"2024-04-25T10:00:25.100433+0000","last_became_active":"2024-04-25T10:00:25.097390+0000","last_became_peered":"2024-04-25T10:00:25.097390+0000","last_unstale":"2024-04-25T10:00:25.100433+0000","last_undegraded":"2024-04-25T10:00:25.100433+0000","last_fullsized":"2024-04-25T10:00:25.100433+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:24.773637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081086+0000","last_change":"2024-04-25T10:00:25.081086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081086+0000","last_undegraded":"2024-04-25T10:00:25.081086+0000","last_fullsized":"2024-04-25T10:00:25.081086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.092559+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.092275+0000","last_became_peered":"2024-04-25T10:00:25.092275+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081859+0000","last_change":"2024-04-25T10:00:25.081859+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081859+0000","last_undegraded":"2024-04-25T10:00:25.081859+0000","last_fullsized":"2024-04-25T10:00:25.081859+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092495+0000","last_change":"2024-04-25T10:00:25.090514+0000","last_active":"2024-04-25T10:00:25.092495+0000","last_peered":"2024-04-25T10:00:25.092495+0000","last_clean":"2024-04-25T10:00:25.092495+0000","last_became_active":"2024-04-25T10:00:25.090248+0000","last_became_peered":"2024-04-25T10:00:25.090248+0000","last_unstale":"2024-04-25T10:00:25.092495+0000","last_undegraded":"2024-04-25T10:00:25.092495+0000","last_fullsized":"2024-04-25T10:00:25.092495+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:34.979909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":308,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.083144+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:00:25.083144+0000","last_peered":"2024-04-25T10:00:25.083144+0000","last_clean":"2024-04-25T10:00:25.083144+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:00:25.083144+0000","last_undegraded":"2024-04-25T10:00:25.083144+0000","last_fullsized":"2024-04-25T10:00:25.083144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":126,"acting":126,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46780,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325060,"statfs":{"total":96636764160,"available":96588861440,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082127+0000', 'last_change': '2024-04-25T10:00:25.082127+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082127+0000', 'last_undegraded': '2024-04-25T10:00:25.082127+0000', 'last_fullsized': '2024-04-25T10:00:25.082127+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086340+0000', 'last_change': '2024-04-25T10:00:25.086340+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086340+0000', 'last_undegraded': '2024-04-25T10:00:25.086340+0000', 'last_fullsized': '2024-04-25T10:00:25.086340+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081711+0000', 'last_change': '2024-04-25T10:00:25.081711+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081711+0000', 'last_undegraded': '2024-04-25T10:00:25.081711+0000', 'last_fullsized': '2024-04-25T10:00:25.081711+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086478+0000', 'last_change': '2024-04-25T10:00:25.086478+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086478+0000', 'last_undegraded': '2024-04-25T10:00:25.086478+0000', 'last_fullsized': '2024-04-25T10:00:25.086478+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.737 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:00:25.073196+0000', 'last_change': '2024-04-25T10:00:25.073196+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.073196+0000', 'last_undegraded': '2024-04-25T10:00:25.073196+0000', 'last_fullsized': '2024-04-25T10:00:25.073196+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.087074+0000', 'last_change': '2024-04-25T10:00:25.087074+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.087074+0000', 'last_undegraded': '2024-04-25T10:00:25.087074+0000', 'last_fullsized': '2024-04-25T10:00:25.087074+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.085115+0000', 'last_change': '2024-04-25T10:00:25.085115+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085115+0000', 'last_undegraded': '2024-04-25T10:00:25.085115+0000', 'last_fullsized': '2024-04-25T10:00:25.085115+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.085698+0000', 'last_change': '2024-04-25T10:00:25.085698+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085698+0000', 'last_undegraded': '2024-04-25T10:00:25.085698+0000', 'last_fullsized': '2024-04-25T10:00:25.085698+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086878+0000', 'last_change': '2024-04-25T10:00:25.086878+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086878+0000', 'last_undegraded': '2024-04-25T10:00:25.086878+0000', 'last_fullsized': '2024-04-25T10:00:25.086878+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-25T10:00:29.738 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084003+0000', 'last_change': '2024-04-25T10:00:25.084003+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084003+0000', 'last_undegraded': '2024-04-25T10:00:25.084003+0000', 'last_fullsized': '2024-04-25T10:00:25.084003+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086635+0000', 'last_change': '2024-04-25T10:00:25.086635+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086635+0000', 'last_undegraded': '2024-04-25T10:00:25.086635+0000', 'last_fullsized': '2024-04-25T10:00:25.086635+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083320+0000', 'last_change': '2024-04-25T10:00:25.083320+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083320+0000', 'last_undegraded': '2024-04-25T10:00:25.083320+0000', 'last_fullsized': '2024-04-25T10:00:25.083320+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084513+0000', 'last_change': '2024-04-25T10:00:25.084513+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084513+0000', 'last_undegraded': '2024-04-25T10:00:25.084513+0000', 'last_fullsized': '2024-04-25T10:00:25.084513+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086055+0000', 'last_change': '2024-04-25T10:00:25.086055+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086055+0000', 'last_undegraded': '2024-04-25T10:00:25.086055+0000', 'last_fullsized': '2024-04-25T10:00:25.086055+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.085240+0000', 'last_change': '2024-04-25T10:00:25.085240+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085240+0000', 'last_undegraded': '2024-04-25T10:00:25.085240+0000', 'last_fullsized': '2024-04-25T10:00:25.085240+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.739 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086340+0000', 'last_change': '2024-04-25T10:00:25.086340+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086340+0000', 'last_undegraded': '2024-04-25T10:00:25.086340+0000', 'last_fullsized': '2024-04-25T10:00:25.086340+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084527+0000', 'last_change': '2024-04-25T10:00:25.084527+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084527+0000', 'last_undegraded': '2024-04-25T10:00:25.084527+0000', 'last_fullsized': '2024-04-25T10:00:25.084527+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081678+0000', 'last_change': '2024-04-25T10:00:25.081678+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081678+0000', 'last_undegraded': '2024-04-25T10:00:25.081678+0000', 'last_fullsized': '2024-04-25T10:00:25.081678+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086106+0000', 'last_change': '2024-04-25T10:00:25.086106+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086106+0000', 'last_undegraded': '2024-04-25T10:00:25.086106+0000', 'last_fullsized': '2024-04-25T10:00:25.086106+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083550+0000', 'last_change': '2024-04-25T10:00:25.083550+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083550+0000', 'last_undegraded': '2024-04-25T10:00:25.083550+0000', 'last_fullsized': '2024-04-25T10:00:25.083550+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-25T10:00:29.740 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086039+0000', 'last_change': '2024-04-25T10:00:25.086039+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086039+0000', 'last_undegraded': '2024-04-25T10:00:25.086039+0000', 'last_fullsized': '2024-04-25T10:00:25.086039+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.085527+0000', 'last_change': '2024-04-25T10:00:25.085527+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085527+0000', 'last_undegraded': '2024-04-25T10:00:25.085527+0000', 'last_fullsized': '2024-04-25T10:00:25.085527+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.091346+0000', 'last_change': '2024-04-25T10:00:25.091346+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.091346+0000', 'last_undegraded': '2024-04-25T10:00:25.091346+0000', 'last_fullsized': '2024-04-25T10:00:25.091346+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081240+0000', 'last_change': '2024-04-25T10:00:25.081240+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081240+0000', 'last_undegraded': '2024-04-25T10:00:25.081240+0000', 'last_fullsized': '2024-04-25T10:00:25.081240+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086088+0000', 'last_change': '2024-04-25T10:00:25.086088+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086088+0000', 'last_undegraded': '2024-04-25T10:00:25.086088+0000', 'last_fullsized': '2024-04-25T10:00:25.086088+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.086491+0000', 'last_change': '2024-04-25T10:00:25.086491+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.086491+0000', 'last_undegraded': '2024-04-25T10:00:25.086491+0000', 'last_fullsized': '2024-04-25T10:00:25.086491+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.741 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083164+0000', 'last_change': '2024-04-25T10:00:25.083164+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083164+0000', 'last_undegraded': '2024-04-25T10:00:25.083164+0000', 'last_fullsized': '2024-04-25T10:00:25.083164+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083140+0000', 'last_change': '2024-04-25T10:00:25.083140+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083140+0000', 'last_undegraded': '2024-04-25T10:00:25.083140+0000', 'last_fullsized': '2024-04-25T10:00:25.083140+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082536+0000', 'last_change': '2024-04-25T10:00:25.082536+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082536+0000', 'last_undegraded': '2024-04-25T10:00:25.082536+0000', 'last_fullsized': '2024-04-25T10:00:25.082536+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084678+0000', 'last_change': '2024-04-25T10:00:25.084678+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084678+0000', 'last_undegraded': '2024-04-25T10:00:25.084678+0000', 'last_fullsized': '2024-04-25T10:00:25.084678+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083510+0000', 'last_change': '2024-04-25T10:00:25.083510+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083510+0000', 'last_undegraded': '2024-04-25T10:00:25.083510+0000', 'last_fullsized': '2024-04-25T10:00:25.083510+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:00:29.742 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082308+0000', 'last_change': '2024-04-25T10:00:25.082308+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082308+0000', 'last_undegraded': '2024-04-25T10:00:25.082308+0000', 'last_fullsized': '2024-04-25T10:00:25.082308+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083370+0000', 'last_change': '2024-04-25T10:00:25.083370+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083370+0000', 'last_undegraded': '2024-04-25T10:00:25.083370+0000', 'last_fullsized': '2024-04-25T10:00:25.083370+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081425+0000', 'last_change': '2024-04-25T10:00:25.081425+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081425+0000', 'last_undegraded': '2024-04-25T10:00:25.081425+0000', 'last_fullsized': '2024-04-25T10:00:25.081425+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 95, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:00:25.085428+0000', 'last_change': '2024-04-25T10:00:25.085428+0000', 'last_active': '2024-04-25T10:00:24.068967+0000', 'last_peered': '2024-04-25T10:00:24.068967+0000', 'last_clean': '2024-04-25T10:00:24.068967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085428+0000', 'last_undegraded': '2024-04-25T10:00:25.085428+0000', 'last_fullsized': '2024-04-25T10:00:25.085428+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082454+0000', 'last_change': '2024-04-25T10:00:25.082454+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082454+0000', 'last_undegraded': '2024-04-25T10:00:25.082454+0000', 'last_fullsized': '2024-04-25T10:00:25.082454+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T10:00:29.743 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081884+0000', 'last_change': '2024-04-25T10:00:25.081884+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081884+0000', 'last_undegraded': '2024-04-25T10:00:25.081884+0000', 'last_fullsized': '2024-04-25T10:00:25.081884+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084889+0000', 'last_change': '2024-04-25T10:00:25.084889+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084889+0000', 'last_undegraded': '2024-04-25T10:00:25.084889+0000', 'last_fullsized': '2024-04-25T10:00:25.084889+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 95, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:00:25.084057+0000', 'last_change': '2024-04-25T10:00:25.084057+0000', 'last_active': '2024-04-25T10:00:24.068967+0000', 'last_peered': '2024-04-25T10:00:24.068967+0000', 'last_clean': '2024-04-25T10:00:24.068967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084057+0000', 'last_undegraded': '2024-04-25T10:00:25.084057+0000', 'last_fullsized': '2024-04-25T10:00:25.084057+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.088983+0000', 'last_change': '2024-04-25T10:00:25.088983+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.088983+0000', 'last_undegraded': '2024-04-25T10:00:25.088983+0000', 'last_fullsized': '2024-04-25T10:00:25.088983+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082091+0000', 'last_change': '2024-04-25T10:00:25.082091+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082091+0000', 'last_undegraded': '2024-04-25T10:00:25.082091+0000', 'last_fullsized': '2024-04-25T10:00:25.082091+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T10:00:29.744 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083970+0000', 'last_change': '2024-04-25T10:00:25.083970+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083970+0000', 'last_undegraded': '2024-04-25T10:00:25.083970+0000', 'last_fullsized': '2024-04-25T10:00:25.083970+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084573+0000', 'last_change': '2024-04-25T10:00:25.084573+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084573+0000', 'last_undegraded': '2024-04-25T10:00:25.084573+0000', 'last_fullsized': '2024-04-25T10:00:25.084573+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082880+0000', 'last_change': '2024-04-25T10:00:25.082880+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082880+0000', 'last_undegraded': '2024-04-25T10:00:25.082880+0000', 'last_fullsized': '2024-04-25T10:00:25.082880+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081242+0000', 'last_change': '2024-04-25T10:00:25.081242+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081242+0000', 'last_undegraded': '2024-04-25T10:00:25.081242+0000', 'last_fullsized': '2024-04-25T10:00:25.081242+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 95, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:00:25.084272+0000', 'last_change': '2024-04-25T10:00:25.084272+0000', 'last_active': '2024-04-25T10:00:24.068967+0000', 'last_peered': '2024-04-25T10:00:24.068967+0000', 'last_clean': '2024-04-25T10:00:24.068967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084272+0000', 'last_undegraded': '2024-04-25T10:00:25.084272+0000', 'last_fullsized': '2024-04-25T10:00:25.084272+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082937+0000', 'last_change': '2024-04-25T10:00:25.082937+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082937+0000', 'last_undegraded': '2024-04-25T10:00:25.082937+0000', 'last_fullsized': '2024-04-25T10:00:25.082937+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T10:00:29.745 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084469+0000', 'last_change': '2024-04-25T10:00:25.084469+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084469+0000', 'last_undegraded': '2024-04-25T10:00:25.084469+0000', 'last_fullsized': '2024-04-25T10:00:25.084469+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.084087+0000', 'last_change': '2024-04-25T10:00:25.084087+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.084087+0000', 'last_undegraded': '2024-04-25T10:00:25.084087+0000', 'last_fullsized': '2024-04-25T10:00:25.084087+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.080612+0000', 'last_change': '2024-04-25T10:00:25.080612+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.080612+0000', 'last_undegraded': '2024-04-25T10:00:25.080612+0000', 'last_fullsized': '2024-04-25T10:00:25.080612+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.082611+0000', 'last_change': '2024-04-25T10:00:25.082611+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.082611+0000', 'last_undegraded': '2024-04-25T10:00:25.082611+0000', 'last_fullsized': '2024-04-25T10:00:25.082611+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083001+0000', 'last_change': '2024-04-25T10:00:25.083001+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083001+0000', 'last_undegraded': '2024-04-25T10:00:25.083001+0000', 'last_fullsized': '2024-04-25T10:00:25.083001+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.091346+0000', 'last_change': '2024-04-25T10:00:25.091346+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.091346+0000', 'last_undegraded': '2024-04-25T10:00:25.091346+0000', 'last_fullsized': '2024-04-25T10:00:25.091346+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:00:29.746 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.085785+0000', 'last_change': '2024-04-25T10:00:25.085785+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085785+0000', 'last_undegraded': '2024-04-25T10:00:25.085785+0000', 'last_fullsized': '2024-04-25T10:00:25.085785+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 95, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:00:25.083952+0000', 'last_change': '2024-04-25T10:00:25.083952+0000', 'last_active': '2024-04-25T10:00:24.068967+0000', 'last_peered': '2024-04-25T10:00:24.068967+0000', 'last_clean': '2024-04-25T10:00:24.068967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083952+0000', 'last_undegraded': '2024-04-25T10:00:25.083952+0000', 'last_fullsized': '2024-04-25T10:00:25.083952+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': ['4', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.080419+0000', 'last_change': '2024-04-25T10:00:25.080419+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.080419+0000', 'last_undegraded': '2024-04-25T10:00:25.080419+0000', 'last_fullsized': '2024-04-25T10:00:25.080419+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.083008+0000', 'last_change': '2024-04-25T10:00:25.083008+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083008+0000', 'last_undegraded': '2024-04-25T10:00:25.083008+0000', 'last_fullsized': '2024-04-25T10:00:25.083008+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.087746+0000', 'last_change': '2024-04-25T10:00:25.087746+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.087746+0000', 'last_undegraded': '2024-04-25T10:00:25.087746+0000', 'last_fullsized': '2024-04-25T10:00:25.087746+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T10:00:29.747 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081435+0000', 'last_change': '2024-04-25T10:00:25.081435+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081435+0000', 'last_undegraded': '2024-04-25T10:00:25.081435+0000', 'last_fullsized': '2024-04-25T10:00:25.081435+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.092039+0000', 'last_change': '2024-04-25T10:00:25.092039+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.092039+0000', 'last_undegraded': '2024-04-25T10:00:25.092039+0000', 'last_fullsized': '2024-04-25T10:00:25.092039+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081036+0000', 'last_change': '2024-04-25T10:00:25.081036+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081036+0000', 'last_undegraded': '2024-04-25T10:00:25.081036+0000', 'last_fullsized': '2024-04-25T10:00:25.081036+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 95, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:00:25.085107+0000', 'last_change': '2024-04-25T10:00:25.085107+0000', 'last_active': '2024-04-25T10:00:24.068967+0000', 'last_peered': '2024-04-25T10:00:24.068967+0000', 'last_clean': '2024-04-25T10:00:24.068967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085107+0000', 'last_undegraded': '2024-04-25T10:00:25.085107+0000', 'last_fullsized': '2024-04-25T10:00:25.085107+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.080448+0000', 'last_change': '2024-04-25T10:00:25.080448+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.080448+0000', 'last_undegraded': '2024-04-25T10:00:25.080448+0000', 'last_fullsized': '2024-04-25T10:00:25.080448+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 95, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:00:25.083562+0000', 'last_change': '2024-04-25T10:00:25.083562+0000', 'last_active': '2024-04-25T10:00:24.068967+0000', 'last_peered': '2024-04-25T10:00:24.068967+0000', 'last_clean': '2024-04-25T10:00:24.068967+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.083562+0000', 'last_undegraded': '2024-04-25T10:00:25.083562+0000', 'last_fullsized': '2024-04-25T10:00:25.083562+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:24.068967+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T10:00:29.748 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.085633+0000', 'last_change': '2024-04-25T10:00:25.085633+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.085633+0000', 'last_undegraded': '2024-04-25T10:00:25.085633+0000', 'last_fullsized': '2024-04-25T10:00:25.085633+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.080086+0000', 'last_change': '2024-04-25T10:00:25.080086+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.080086+0000', 'last_undegraded': '2024-04-25T10:00:25.080086+0000', 'last_fullsized': '2024-04-25T10:00:25.080086+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081155+0000', 'last_change': '2024-04-25T10:00:25.081155+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081155+0000', 'last_undegraded': '2024-04-25T10:00:25.081155+0000', 'last_fullsized': '2024-04-25T10:00:25.081155+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.089230+0000', 'last_change': '2024-04-25T10:00:25.089230+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.089230+0000', 'last_undegraded': '2024-04-25T10:00:25.089230+0000', 'last_fullsized': '2024-04-25T10:00:25.089230+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081086+0000', 'last_change': '2024-04-25T10:00:25.081086+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081086+0000', 'last_undegraded': '2024-04-25T10:00:25.081086+0000', 'last_fullsized': '2024-04-25T10:00:25.081086+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T10:00:29.749 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 95, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:00:25.081859+0000', 'last_change': '2024-04-25T10:00:25.081859+0000', 'last_active': '2024-04-25T10:00:25.073196+0000', 'last_peered': '2024-04-25T10:00:25.073196+0000', 'last_clean': '2024-04-25T10:00:25.073196+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:00:25.081859+0000', 'last_undegraded': '2024-04-25T10:00:25.081859+0000', 'last_fullsized': '2024-04-25T10:00:25.081859+0000', 'mapping_epoch': 95, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 95, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'last_clean_scrub_stamp': '2024-04-25T10:00:25.073196+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:00:29.750 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:29.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.862+0000 7fee3ec1c640 1 Processor -- start 2024-04-25T10:00:29.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.862+0000 7fee3ec1c640 1 -- start start 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3ec1c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee380d1dd0 0x7fee380ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3ec1c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3815dfe0 con 0x7fee3815dc00 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3ec1c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380d00c0 con 0x7fee380c6df0 2024-04-25T10:00:29.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3ec1c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee380d03a0 con 0x7fee380d1dd0 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3d1b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee380d1dd0 0x7fee380ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3c9b8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee3815dc00 0x7fee380ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33348/0 (socket says 172.21.15.148:33348) 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3c9b8640 1 -- 172.21.15.148:0/1255739943 learned_addr learned my addr 172.21.15.148:0/1255739943 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3d1b9640 1 -- 172.21.15.148:0/1255739943 >> v1:172.21.15.188:6789/0 conn(0x7fee380c6df0 legacy=0x7fee380d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3d1b9640 1 -- 172.21.15.148:0/1255739943 >> v1:172.21.15.148:6789/0 conn(0x7fee3815dc00 legacy=0x7fee380ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee3d1b9640 1 -- 172.21.15.148:0/1255739943 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380cc440 con 0x7fee380d1dd0 2024-04-25T10:00:29.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.863+0000 7fee377fe640 1 -- 172.21.15.148:0/1255739943 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637978951 0 0) 0x7fee2c001110 con 0x7fee3815dc00 2024-04-25T10:00:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.864+0000 7fee3d1b9640 1 --2- 172.21.15.148:0/1255739943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee380d1dd0 0x7fee380ccc40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fee30009d90 tx=0x7fee3002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ceffe7f2ce0f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.864+0000 7fee377fe640 1 -- 172.21.15.148:0/1255739943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30007740 con 0x7fee380d1dd0 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.864+0000 7fee377fe640 1 -- 172.21.15.148:0/1255739943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee300078e0 con 0x7fee380d1dd0 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.864+0000 7fee377fe640 1 -- 172.21.15.148:0/1255739943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30007cb0 con 0x7fee380d1dd0 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.864+0000 7fee3ec1c640 1 -- 172.21.15.148:0/1255739943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee380d1dd0 msgr2=0x7fee380ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.864+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/1255739943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee380d1dd0 0x7fee380ccc40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fee30009d90 tx=0x7fee3002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.865+0000 7fee3ec1c640 1 -- 172.21.15.148:0/1255739943 shutdown_connections 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.865+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/1255739943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee380d1dd0 0x7fee380ccc40 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.865+0000 7fee3ec1c640 1 -- 172.21.15.148:0/1255739943 >> 172.21.15.148:0/1255739943 conn(0x7fee3806f5d0 msgr2=0x7fee3806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.865+0000 7fee3ec1c640 1 -- 172.21.15.148:0/1255739943 shutdown_connections 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.865+0000 7fee3ec1c640 1 -- 172.21.15.148:0/1255739943 wait complete. 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.866+0000 7fee3ec1c640 1 Processor -- start 2024-04-25T10:00:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.866+0000 7fee3ec1c640 1 -- start start 2024-04-25T10:00:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee380c6df0 0x7fee381e8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee380d1dd0 0x7fee380b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 0x7fee381533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380cfd90 con 0x7fee3815dc00 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380cc130 con 0x7fee380d1dd0 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380cdd70 con 0x7fee380c6df0 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 0x7fee381533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 0x7fee381533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50378/0 (socket says 172.21.15.148:50378) 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 -- 172.21.15.148:0/356793470 learned_addr learned my addr 172.21.15.148:0/356793470 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee37fff640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee380d1dd0 0x7fee380b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 -- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee380c6df0 msgr2=0x7fee381e8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee380c6df0 0x7fee381e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 -- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee380d1dd0 msgr2=0x7fee380b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee380d1dd0 0x7fee380b5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 -- 172.21.15.148:0/356793470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380b6440 con 0x7fee3815dc00 2024-04-25T10:00:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee37fff640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee380d1dd0 0x7fee380b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3d1b9640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 0x7fee381533d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fee30009eb0 tx=0x7fee30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30040070 con 0x7fee3815dc00 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30031c90 con 0x7fee3815dc00 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee380b66e0 con 0x7fee3815dc00 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3003c640 con 0x7fee3815dc00 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.867+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee380b6f80 con 0x7fee3815dc00 2024-04-25T10:00:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.868+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee38073990 con 0x7fee3815dc00 2024-04-25T10:00:29.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.869+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee30039040 con 0x7fee3815dc00 2024-04-25T10:00:29.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.869+0000 7fee357fa640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1c06b4d0 0x7fee1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:29.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.869+0000 7fee3c9b8640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1c06b4d0 0x7fee1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:29.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.869+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee300ab9e0 con 0x7fee3815dc00 2024-04-25T10:00:29.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.870+0000 7fee3c9b8640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1c06b4d0 0x7fee1c06d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000c00 tx=0x7fee2c0079c0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:29.874+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee3007a790 con 0x7fee3815dc00 2024-04-25T10:00:30.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.112+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fee380c66c0 con 0x7fee3815dc00 2024-04-25T10:00:30.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.112+0000 7fee357fa640 1 -- 172.21.15.148:0/356793470 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1158 (secure 0 0 0) 0x7fee3007a090 con 0x7fee3815dc00 2024-04-25T10:00:30.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:30.118 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":391,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":63},{"state_name":"active+clean","count":27},{"state_name":"creating+activating","count":6},{"state_name":"unknown","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":383877120,"bytes_avail":772710236160,"bytes_total":773094113280,"unknown_pgs_ratio":0.010309278033673763,"inactive_pgs_ratio":0.71134018898010254},"fsmap":{"epoch":53,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7725}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T09:59:57.088810+0000","services":{}},"progress_events":{}} 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1c06b4d0 msgr2=0x7fee1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1c06b4d0 0x7fee1c06d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000c00 tx=0x7fee2c0079c0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 msgr2=0x7fee381533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 0x7fee381533d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fee30009eb0 tx=0x7fee30004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 shutdown_connections 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1c06b4d0 0x7fee1c06d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee3815dc00 0x7fee381533d0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee380d1dd0 0x7fee380b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 --2- 172.21.15.148:0/356793470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee380c6df0 0x7fee381e8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 >> 172.21.15.148:0/356793470 conn(0x7fee3806f5d0 msgr2=0x7fee380cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 shutdown_connections 2024-04-25T10:00:30.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.114+0000 7fee3ec1c640 1 -- 172.21.15.148:0/356793470 wait complete. 2024-04-25T10:00:30.131 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:00:30.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:30.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.223+0000 7f0ea729a640 1 Processor -- start 2024-04-25T10:00:30.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.224+0000 7f0ea729a640 1 -- start start 2024-04-25T10:00:30.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.224+0000 7f0ea729a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea01602e0 0x7f0ea00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.224+0000 7f0ea729a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0ea015f5f0 0x7f0ea015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.224+0000 7f0ea729a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea00cd710 con 0x7f0ea015ba00 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.224+0000 7f0ea729a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea0099b70 con 0x7f0ea015f5f0 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.224+0000 7f0ea729a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea0099e50 con 0x7f0ea01602e0 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea5837640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0ea015f5f0 0x7f0ea015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea4835640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea01602e0 0x7f0ea00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea5036640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0ea015ba00 0x7f0ea009a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43318/0 (socket says 172.21.15.148:43318) 2024-04-25T10:00:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea5036640 1 -- 172.21.15.148:0/2519906498 learned_addr learned my addr 172.21.15.148:0/2519906498 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea4835640 1 -- 172.21.15.148:0/2519906498 >> v2:172.21.15.188:3300/0 conn(0x7f0ea015f5f0 msgr2=0x7f0ea015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea4835640 1 --2- 172.21.15.148:0/2519906498 >> v2:172.21.15.188:3300/0 conn(0x7f0ea015f5f0 0x7f0ea015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea4835640 1 -- 172.21.15.148:0/2519906498 >> v1:172.21.15.188:6789/0 conn(0x7f0ea015ba00 legacy=0x7f0ea009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea4835640 1 -- 172.21.15.148:0/2519906498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea009a130 con 0x7f0ea01602e0 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.225+0000 7f0ea5837640 1 --2- 172.21.15.148:0/2519906498 >> v2:172.21.15.188:3300/0 conn(0x7f0ea015f5f0 0x7f0ea015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0ea4835640 1 --2- 172.21.15.148:0/2519906498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea01602e0 0x7f0ea00ccda0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e900097a0 tx=0x7f0e9002a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8abbfec64dede31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0e97fff640 1 -- 172.21.15.148:0/2519906498 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90037070 con 0x7f0ea01602e0 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0e97fff640 1 -- 172.21.15.148:0/2519906498 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9002adb0 con 0x7f0ea01602e0 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0e97fff640 1 -- 172.21.15.148:0/2519906498 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90032660 con 0x7f0ea01602e0 2024-04-25T10:00:30.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0ea729a640 1 -- 172.21.15.148:0/2519906498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea01602e0 msgr2=0x7f0ea00ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0ea729a640 1 --2- 172.21.15.148:0/2519906498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea01602e0 0x7f0ea00ccda0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0e900097a0 tx=0x7f0e9002a060 comp rx=0 tx=0).stop 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.226+0000 7f0ea729a640 1 -- 172.21.15.148:0/2519906498 shutdown_connections 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.227+0000 7f0ea729a640 1 --2- 172.21.15.148:0/2519906498 >> v2:172.21.15.188:3300/0 conn(0x7f0ea015f5f0 0x7f0ea015faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.227+0000 7f0ea729a640 1 --2- 172.21.15.148:0/2519906498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea01602e0 0x7f0ea00ccda0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.227+0000 7f0ea729a640 1 -- 172.21.15.148:0/2519906498 >> 172.21.15.148:0/2519906498 conn(0x7f0ea00b3270 msgr2=0x7f0ea00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.227+0000 7f0ea729a640 1 -- 172.21.15.148:0/2519906498 shutdown_connections 2024-04-25T10:00:30.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.227+0000 7f0ea729a640 1 -- 172.21.15.148:0/2519906498 wait complete. 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 Processor -- start 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 -- start start 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 0x7f0ea00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea015f5f0 0x7f0ea014a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea01602e0 0x7f0ea00c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00cd460 con 0x7f0ea015ba00 2024-04-25T10:00:30.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00cd2e0 con 0x7f0ea01602e0 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.228+0000 7f0ea729a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00c52d0 con 0x7f0ea015f5f0 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 0x7f0ea00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 0x7f0ea00b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50384/0 (socket says 172.21.15.148:50384) 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 -- 172.21.15.148:0/23599365 learned_addr learned my addr 172.21.15.148:0/23599365 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5837640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea01602e0 0x7f0ea00c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea4835640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea015f5f0 0x7f0ea014a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 -- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea015f5f0 msgr2=0x7f0ea014a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea015f5f0 0x7f0ea014a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 -- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea01602e0 msgr2=0x7f0ea00c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea01602e0 0x7f0ea00c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 -- 172.21.15.148:0/23599365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00c8d50 con 0x7f0ea015ba00 2024-04-25T10:00:30.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea4835640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea015f5f0 0x7f0ea014a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5837640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea01602e0 0x7f0ea00c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea5036640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 0x7f0ea00b4d60 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00cb10 tx=0x7f0e9c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c017070 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00ac70 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c012620 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea00c9030 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.229+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea00c95f0 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.230+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea0073990 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.231+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e9c01a070 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.231+0000 7f0e95ffb640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e8406b500 0x7f0e8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.231+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e9c08a0a0 con 0x7f0ea015ba00 2024-04-25T10:00:30.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.231+0000 7f0ea4835640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e8406b500 0x7f0e8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:30.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.231+0000 7f0ea4835640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e8406b500 0x7f0e8406d9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e90034000 tx=0x7f0e9002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:30.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.235+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ea00cd710 con 0x7f0ea015ba00 2024-04-25T10:00:30.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.401+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ea00c7c50 con 0x7f0e8406b500 2024-04-25T10:00:30.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.407+0000 7f0e95ffb640 1 -- 172.21.15.148:0/23599365 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246298 (secure 0 0 0) 0x7f0e9007e230 con 0x7f0e8406b500 2024-04-25T10:00:30.415 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:30.416 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e8406b500 msgr2=0x7f0e8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e8406b500 0x7f0e8406d9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0e90034000 tx=0x7f0e9002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 msgr2=0x7f0ea00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 0x7f0ea00b4d60 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00cb10 tx=0x7f0e9c00a060 comp rx=0 tx=0).stop 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 shutdown_connections 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e8406b500 0x7f0e8406d9c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea01602e0 0x7f0ea00c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea015f5f0 0x7f0ea014a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 --2- 172.21.15.148:0/23599365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ea015ba00 0x7f0ea00b4d60 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 >> 172.21.15.148:0/23599365 conn(0x7f0ea00b3270 msgr2=0x7f0ea009a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 shutdown_connections 2024-04-25T10:00:30.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:30.410+0000 7f0ea729a640 1 -- 172.21.15.148:0/23599365 wait complete. 2024-04-25T10:00:30.426 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-04-25T10:00:29.095272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":192,"acting":192,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374880,"kb_used_data":10216,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599840,"statfs":{"total":773094113280,"available":772710236160,"internally_reserved":0,"allocated":10461184,"data_stored":5458264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.021231"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082127+0000","last_change":"2024-04-25T10:00:25.082127+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082127+0000","last_undegraded":"2024-04-25T10:00:25.082127+0000","last_fullsized":"2024-04-25T10:00:25.082127+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081711+0000","last_change":"2024-04-25T10:00:25.081711+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081711+0000","last_undegraded":"2024-04-25T10:00:25.081711+0000","last_fullsized":"2024-04-25T10:00:25.081711+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086478+0000","last_change":"2024-04-25T10:00:25.086478+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086478+0000","last_undegraded":"2024-04-25T10:00:25.086478+0000","last_fullsized":"2024-04-25T10:00:25.086478+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:00:25.073196+0000","last_change":"2024-04-25T10:00:25.073196+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.073196+0000","last_undegraded":"2024-04-25T10:00:25.073196+0000","last_fullsized":"2024-04-25T10:00:25.073196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087074+0000","last_change":"2024-04-25T10:00:25.087074+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087074+0000","last_undegraded":"2024-04-25T10:00:25.087074+0000","last_fullsized":"2024-04-25T10:00:25.087074+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085115+0000","last_change":"2024-04-25T10:00:25.085115+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085115+0000","last_undegraded":"2024-04-25T10:00:25.085115+0000","last_fullsized":"2024-04-25T10:00:25.085115+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085698+0000","last_change":"2024-04-25T10:00:25.085698+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085698+0000","last_undegraded":"2024-04-25T10:00:25.085698+0000","last_fullsized":"2024-04-25T10:00:25.085698+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086878+0000","last_change":"2024-04-25T10:00:25.086878+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086878+0000","last_undegraded":"2024-04-25T10:00:25.086878+0000","last_fullsized":"2024-04-25T10:00:25.086878+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084003+0000","last_change":"2024-04-25T10:00:25.084003+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084003+0000","last_undegraded":"2024-04-25T10:00:25.084003+0000","last_fullsized":"2024-04-25T10:00:25.084003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086635+0000","last_change":"2024-04-25T10:00:25.086635+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086635+0000","last_undegraded":"2024-04-25T10:00:25.086635+0000","last_fullsized":"2024-04-25T10:00:25.086635+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083320+0000","last_change":"2024-04-25T10:00:25.083320+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083320+0000","last_undegraded":"2024-04-25T10:00:25.083320+0000","last_fullsized":"2024-04-25T10:00:25.083320+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084513+0000","last_change":"2024-04-25T10:00:25.084513+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084513+0000","last_undegraded":"2024-04-25T10:00:25.084513+0000","last_fullsized":"2024-04-25T10:00:25.084513+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086055+0000","last_change":"2024-04-25T10:00:25.086055+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086055+0000","last_undegraded":"2024-04-25T10:00:25.086055+0000","last_fullsized":"2024-04-25T10:00:25.086055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085240+0000","last_change":"2024-04-25T10:00:25.085240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085240+0000","last_undegraded":"2024-04-25T10:00:25.085240+0000","last_fullsized":"2024-04-25T10:00:25.085240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086340+0000","last_change":"2024-04-25T10:00:25.086340+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086340+0000","last_undegraded":"2024-04-25T10:00:25.086340+0000","last_fullsized":"2024-04-25T10:00:25.086340+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084527+0000","last_change":"2024-04-25T10:00:25.084527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084527+0000","last_undegraded":"2024-04-25T10:00:25.084527+0000","last_fullsized":"2024-04-25T10:00:25.084527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081678+0000","last_change":"2024-04-25T10:00:25.081678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081678+0000","last_undegraded":"2024-04-25T10:00:25.081678+0000","last_fullsized":"2024-04-25T10:00:25.081678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086106+0000","last_change":"2024-04-25T10:00:25.086106+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086106+0000","last_undegraded":"2024-04-25T10:00:25.086106+0000","last_fullsized":"2024-04-25T10:00:25.086106+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083550+0000","last_change":"2024-04-25T10:00:25.083550+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083550+0000","last_undegraded":"2024-04-25T10:00:25.083550+0000","last_fullsized":"2024-04-25T10:00:25.083550+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086039+0000","last_change":"2024-04-25T10:00:25.086039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086039+0000","last_undegraded":"2024-04-25T10:00:25.086039+0000","last_fullsized":"2024-04-25T10:00:25.086039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085527+0000","last_change":"2024-04-25T10:00:25.085527+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085527+0000","last_undegraded":"2024-04-25T10:00:25.085527+0000","last_fullsized":"2024-04-25T10:00:25.085527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081240+0000","last_change":"2024-04-25T10:00:25.081240+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081240+0000","last_undegraded":"2024-04-25T10:00:25.081240+0000","last_fullsized":"2024-04-25T10:00:25.081240+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086088+0000","last_change":"2024-04-25T10:00:25.086088+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086088+0000","last_undegraded":"2024-04-25T10:00:25.086088+0000","last_fullsized":"2024-04-25T10:00:25.086088+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.086491+0000","last_change":"2024-04-25T10:00:25.086491+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.086491+0000","last_undegraded":"2024-04-25T10:00:25.086491+0000","last_fullsized":"2024-04-25T10:00:25.086491+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083164+0000","last_change":"2024-04-25T10:00:25.083164+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083164+0000","last_undegraded":"2024-04-25T10:00:25.083164+0000","last_fullsized":"2024-04-25T10:00:25.083164+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083140+0000","last_change":"2024-04-25T10:00:25.083140+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083140+0000","last_undegraded":"2024-04-25T10:00:25.083140+0000","last_fullsized":"2024-04-25T10:00:25.083140+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082536+0000","last_change":"2024-04-25T10:00:25.082536+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082536+0000","last_undegraded":"2024-04-25T10:00:25.082536+0000","last_fullsized":"2024-04-25T10:00:25.082536+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084678+0000","last_change":"2024-04-25T10:00:25.084678+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084678+0000","last_undegraded":"2024-04-25T10:00:25.084678+0000","last_fullsized":"2024-04-25T10:00:25.084678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083510+0000","last_change":"2024-04-25T10:00:25.083510+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083510+0000","last_undegraded":"2024-04-25T10:00:25.083510+0000","last_fullsized":"2024-04-25T10:00:25.083510+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082308+0000","last_change":"2024-04-25T10:00:25.082308+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082308+0000","last_undegraded":"2024-04-25T10:00:25.082308+0000","last_fullsized":"2024-04-25T10:00:25.082308+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083370+0000","last_change":"2024-04-25T10:00:25.083370+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083370+0000","last_undegraded":"2024-04-25T10:00:25.083370+0000","last_fullsized":"2024-04-25T10:00:25.083370+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100472+0000","last_change":"2024-04-25T10:00:25.098352+0000","last_active":"2024-04-25T10:00:25.100472+0000","last_peered":"2024-04-25T10:00:25.100472+0000","last_clean":"2024-04-25T10:00:25.100472+0000","last_became_active":"2024-04-25T10:00:25.098081+0000","last_became_peered":"2024-04-25T10:00:25.098081+0000","last_unstale":"2024-04-25T10:00:25.100472+0000","last_undegraded":"2024-04-25T10:00:25.100472+0000","last_fullsized":"2024-04-25T10:00:25.100472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:10.923042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081425+0000","last_change":"2024-04-25T10:00:25.081425+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081425+0000","last_undegraded":"2024-04-25T10:00:25.081425+0000","last_fullsized":"2024-04-25T10:00:25.081425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085428+0000","last_change":"2024-04-25T10:00:25.085428+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085428+0000","last_undegraded":"2024-04-25T10:00:25.085428+0000","last_fullsized":"2024-04-25T10:00:25.085428+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082454+0000","last_change":"2024-04-25T10:00:25.082454+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082454+0000","last_undegraded":"2024-04-25T10:00:25.082454+0000","last_fullsized":"2024-04-25T10:00:25.082454+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088926+0000","last_change":"2024-04-25T10:00:25.087470+0000","last_active":"2024-04-25T10:00:25.088926+0000","last_peered":"2024-04-25T10:00:25.088926+0000","last_clean":"2024-04-25T10:00:25.088926+0000","last_became_active":"2024-04-25T10:00:25.087191+0000","last_became_peered":"2024-04-25T10:00:25.087191+0000","last_unstale":"2024-04-25T10:00:25.088926+0000","last_undegraded":"2024-04-25T10:00:25.088926+0000","last_fullsized":"2024-04-25T10:00:25.088926+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:29.174918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081884+0000","last_change":"2024-04-25T10:00:25.081884+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081884+0000","last_undegraded":"2024-04-25T10:00:25.081884+0000","last_fullsized":"2024-04-25T10:00:25.081884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100393+0000","last_change":"2024-04-25T10:00:25.096412+0000","last_active":"2024-04-25T10:00:25.100393+0000","last_peered":"2024-04-25T10:00:25.100393+0000","last_clean":"2024-04-25T10:00:25.100393+0000","last_became_active":"2024-04-25T10:00:25.095816+0000","last_became_peered":"2024-04-25T10:00:25.095816+0000","last_unstale":"2024-04-25T10:00:25.100393+0000","last_undegraded":"2024-04-25T10:00:25.100393+0000","last_fullsized":"2024-04-25T10:00:25.100393+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:34.162551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084889+0000","last_change":"2024-04-25T10:00:25.084889+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084889+0000","last_undegraded":"2024-04-25T10:00:25.084889+0000","last_fullsized":"2024-04-25T10:00:25.084889+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084057+0000","last_change":"2024-04-25T10:00:25.084057+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084057+0000","last_undegraded":"2024-04-25T10:00:25.084057+0000","last_fullsized":"2024-04-25T10:00:25.084057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.088983+0000","last_change":"2024-04-25T10:00:25.088983+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.088983+0000","last_undegraded":"2024-04-25T10:00:25.088983+0000","last_fullsized":"2024-04-25T10:00:25.088983+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092854+0000","last_change":"2024-04-25T10:00:25.090629+0000","last_active":"2024-04-25T10:00:25.092854+0000","last_peered":"2024-04-25T10:00:25.092854+0000","last_clean":"2024-04-25T10:00:25.092854+0000","last_became_active":"2024-04-25T10:00:25.090354+0000","last_became_peered":"2024-04-25T10:00:25.090354+0000","last_unstale":"2024-04-25T10:00:25.092854+0000","last_undegraded":"2024-04-25T10:00:25.092854+0000","last_fullsized":"2024-04-25T10:00:25.092854+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:06.477981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082091+0000","last_change":"2024-04-25T10:00:25.082091+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082091+0000","last_undegraded":"2024-04-25T10:00:25.082091+0000","last_fullsized":"2024-04-25T10:00:25.082091+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.088390+0000","last_change":"2024-04-25T10:00:25.086108+0000","last_active":"2024-04-25T10:00:25.088390+0000","last_peered":"2024-04-25T10:00:25.088390+0000","last_clean":"2024-04-25T10:00:25.088390+0000","last_became_active":"2024-04-25T10:00:25.085829+0000","last_became_peered":"2024-04-25T10:00:25.085829+0000","last_unstale":"2024-04-25T10:00:25.088390+0000","last_undegraded":"2024-04-25T10:00:25.088390+0000","last_fullsized":"2024-04-25T10:00:25.088390+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083970+0000","last_change":"2024-04-25T10:00:25.083970+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083970+0000","last_undegraded":"2024-04-25T10:00:25.083970+0000","last_fullsized":"2024-04-25T10:00:25.083970+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098752+0000","last_change":"2024-04-25T10:00:25.097674+0000","last_active":"2024-04-25T10:00:25.098752+0000","last_peered":"2024-04-25T10:00:25.098752+0000","last_clean":"2024-04-25T10:00:25.098752+0000","last_became_active":"2024-04-25T10:00:25.097383+0000","last_became_peered":"2024-04-25T10:00:25.097383+0000","last_unstale":"2024-04-25T10:00:25.098752+0000","last_undegraded":"2024-04-25T10:00:25.098752+0000","last_fullsized":"2024-04-25T10:00:25.098752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084573+0000","last_change":"2024-04-25T10:00:25.084573+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084573+0000","last_undegraded":"2024-04-25T10:00:25.084573+0000","last_fullsized":"2024-04-25T10:00:25.084573+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098649+0000","last_change":"2024-04-25T10:00:25.096537+0000","last_active":"2024-04-25T10:00:25.098649+0000","last_peered":"2024-04-25T10:00:25.098649+0000","last_clean":"2024-04-25T10:00:25.098649+0000","last_became_active":"2024-04-25T10:00:25.096221+0000","last_became_peered":"2024-04-25T10:00:25.096221+0000","last_unstale":"2024-04-25T10:00:25.098649+0000","last_undegraded":"2024-04-25T10:00:25.098649+0000","last_fullsized":"2024-04-25T10:00:25.098649+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082880+0000","last_change":"2024-04-25T10:00:25.082880+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082880+0000","last_undegraded":"2024-04-25T10:00:25.082880+0000","last_fullsized":"2024-04-25T10:00:25.082880+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094336+0000","last_change":"2024-04-25T10:00:25.093114+0000","last_active":"2024-04-25T10:00:25.094336+0000","last_peered":"2024-04-25T10:00:25.094336+0000","last_clean":"2024-04-25T10:00:25.094336+0000","last_became_active":"2024-04-25T10:00:25.092835+0000","last_became_peered":"2024-04-25T10:00:25.092835+0000","last_unstale":"2024-04-25T10:00:25.094336+0000","last_undegraded":"2024-04-25T10:00:25.094336+0000","last_fullsized":"2024-04-25T10:00:25.094336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:40.180045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081242+0000","last_change":"2024-04-25T10:00:25.081242+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081242+0000","last_undegraded":"2024-04-25T10:00:25.081242+0000","last_fullsized":"2024-04-25T10:00:25.081242+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.084272+0000","last_change":"2024-04-25T10:00:25.084272+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084272+0000","last_undegraded":"2024-04-25T10:00:25.084272+0000","last_fullsized":"2024-04-25T10:00:25.084272+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082937+0000","last_change":"2024-04-25T10:00:25.082937+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082937+0000","last_undegraded":"2024-04-25T10:00:25.082937+0000","last_fullsized":"2024-04-25T10:00:25.082937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.097897+0000","last_change":"2024-04-25T10:00:25.097116+0000","last_active":"2024-04-25T10:00:25.097897+0000","last_peered":"2024-04-25T10:00:25.097897+0000","last_clean":"2024-04-25T10:00:25.097897+0000","last_became_active":"2024-04-25T10:00:25.096795+0000","last_became_peered":"2024-04-25T10:00:25.096795+0000","last_unstale":"2024-04-25T10:00:25.097897+0000","last_undegraded":"2024-04-25T10:00:25.097897+0000","last_fullsized":"2024-04-25T10:00:25.097897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.429544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084469+0000","last_change":"2024-04-25T10:00:25.084469+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084469+0000","last_undegraded":"2024-04-25T10:00:25.084469+0000","last_fullsized":"2024-04-25T10:00:25.084469+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098728+0000","last_change":"2024-04-25T10:00:25.096994+0000","last_active":"2024-04-25T10:00:25.098728+0000","last_peered":"2024-04-25T10:00:25.098728+0000","last_clean":"2024-04-25T10:00:25.098728+0000","last_became_active":"2024-04-25T10:00:25.096686+0000","last_became_peered":"2024-04-25T10:00:25.096686+0000","last_unstale":"2024-04-25T10:00:25.098728+0000","last_undegraded":"2024-04-25T10:00:25.098728+0000","last_fullsized":"2024-04-25T10:00:25.098728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:02.701707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.084087+0000","last_change":"2024-04-25T10:00:25.084087+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.084087+0000","last_undegraded":"2024-04-25T10:00:25.084087+0000","last_fullsized":"2024-04-25T10:00:25.084087+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094843+0000","last_change":"2024-04-25T10:00:25.092552+0000","last_active":"2024-04-25T10:00:25.094843+0000","last_peered":"2024-04-25T10:00:25.094843+0000","last_clean":"2024-04-25T10:00:25.094843+0000","last_became_active":"2024-04-25T10:00:25.092297+0000","last_became_peered":"2024-04-25T10:00:25.092297+0000","last_unstale":"2024-04-25T10:00:25.094843+0000","last_undegraded":"2024-04-25T10:00:25.094843+0000","last_fullsized":"2024-04-25T10:00:25.094843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:20.732580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080612+0000","last_change":"2024-04-25T10:00:25.080612+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080612+0000","last_undegraded":"2024-04-25T10:00:25.080612+0000","last_fullsized":"2024-04-25T10:00:25.080612+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096185+0000","last_change":"2024-04-25T10:00:25.094479+0000","last_active":"2024-04-25T10:00:25.096185+0000","last_peered":"2024-04-25T10:00:25.096185+0000","last_clean":"2024-04-25T10:00:25.096185+0000","last_became_active":"2024-04-25T10:00:25.094171+0000","last_became_peered":"2024-04-25T10:00:25.094171+0000","last_unstale":"2024-04-25T10:00:25.096185+0000","last_undegraded":"2024-04-25T10:00:25.096185+0000","last_fullsized":"2024-04-25T10:00:25.096185+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:51.073666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.082611+0000","last_change":"2024-04-25T10:00:25.082611+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.082611+0000","last_undegraded":"2024-04-25T10:00:25.082611+0000","last_fullsized":"2024-04-25T10:00:25.082611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092829+0000","last_change":"2024-04-25T10:00:25.091554+0000","last_active":"2024-04-25T10:00:25.092829+0000","last_peered":"2024-04-25T10:00:25.092829+0000","last_clean":"2024-04-25T10:00:25.092829+0000","last_became_active":"2024-04-25T10:00:25.091292+0000","last_became_peered":"2024-04-25T10:00:25.091292+0000","last_unstale":"2024-04-25T10:00:25.092829+0000","last_undegraded":"2024-04-25T10:00:25.092829+0000","last_fullsized":"2024-04-25T10:00:25.092829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:43.856414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083001+0000","last_change":"2024-04-25T10:00:25.083001+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083001+0000","last_undegraded":"2024-04-25T10:00:25.083001+0000","last_fullsized":"2024-04-25T10:00:25.083001+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100381+0000","last_change":"2024-04-25T10:00:25.096404+0000","last_active":"2024-04-25T10:00:25.100381+0000","last_peered":"2024-04-25T10:00:25.100381+0000","last_clean":"2024-04-25T10:00:25.100381+0000","last_became_active":"2024-04-25T10:00:25.096120+0000","last_became_peered":"2024-04-25T10:00:25.096120+0000","last_unstale":"2024-04-25T10:00:25.100381+0000","last_undegraded":"2024-04-25T10:00:25.100381+0000","last_fullsized":"2024-04-25T10:00:25.100381+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.897809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.091346+0000","last_change":"2024-04-25T10:00:25.091346+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.091346+0000","last_undegraded":"2024-04-25T10:00:25.091346+0000","last_fullsized":"2024-04-25T10:00:25.091346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.091793+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.091513+0000","last_became_peered":"2024-04-25T10:00:25.091513+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085785+0000","last_change":"2024-04-25T10:00:25.085785+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085785+0000","last_undegraded":"2024-04-25T10:00:25.085785+0000","last_fullsized":"2024-04-25T10:00:25.085785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083952+0000","last_change":"2024-04-25T10:00:25.083952+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083952+0000","last_undegraded":"2024-04-25T10:00:25.083952+0000","last_fullsized":"2024-04-25T10:00:25.083952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080419+0000","last_change":"2024-04-25T10:00:25.080419+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080419+0000","last_undegraded":"2024-04-25T10:00:25.080419+0000","last_fullsized":"2024-04-25T10:00:25.080419+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096106+0000","last_change":"2024-04-25T10:00:25.093815+0000","last_active":"2024-04-25T10:00:25.096106+0000","last_peered":"2024-04-25T10:00:25.096106+0000","last_clean":"2024-04-25T10:00:25.096106+0000","last_became_active":"2024-04-25T10:00:25.093502+0000","last_became_peered":"2024-04-25T10:00:25.093502+0000","last_unstale":"2024-04-25T10:00:25.096106+0000","last_undegraded":"2024-04-25T10:00:25.096106+0000","last_fullsized":"2024-04-25T10:00:25.096106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.083008+0000","last_change":"2024-04-25T10:00:25.083008+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083008+0000","last_undegraded":"2024-04-25T10:00:25.083008+0000","last_fullsized":"2024-04-25T10:00:25.083008+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.094580+0000","last_change":"2024-04-25T10:00:25.093636+0000","last_active":"2024-04-25T10:00:25.094580+0000","last_peered":"2024-04-25T10:00:25.094580+0000","last_clean":"2024-04-25T10:00:25.094580+0000","last_became_active":"2024-04-25T10:00:25.093327+0000","last_became_peered":"2024-04-25T10:00:25.093327+0000","last_unstale":"2024-04-25T10:00:25.094580+0000","last_undegraded":"2024-04-25T10:00:25.094580+0000","last_fullsized":"2024-04-25T10:00:25.094580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:06.677307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.087746+0000","last_change":"2024-04-25T10:00:25.087746+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.087746+0000","last_undegraded":"2024-04-25T10:00:25.087746+0000","last_fullsized":"2024-04-25T10:00:25.087746+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098073+0000","last_change":"2024-04-25T10:00:25.094685+0000","last_active":"2024-04-25T10:00:25.098073+0000","last_peered":"2024-04-25T10:00:25.098073+0000","last_clean":"2024-04-25T10:00:25.098073+0000","last_became_active":"2024-04-25T10:00:25.094421+0000","last_became_peered":"2024-04-25T10:00:25.094421+0000","last_unstale":"2024-04-25T10:00:25.098073+0000","last_undegraded":"2024-04-25T10:00:25.098073+0000","last_fullsized":"2024-04-25T10:00:25.098073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081435+0000","last_change":"2024-04-25T10:00:25.081435+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081435+0000","last_undegraded":"2024-04-25T10:00:25.081435+0000","last_fullsized":"2024-04-25T10:00:25.081435+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.087730+0000","last_change":"2024-04-25T10:00:25.087285+0000","last_active":"2024-04-25T10:00:25.087730+0000","last_peered":"2024-04-25T10:00:25.087730+0000","last_clean":"2024-04-25T10:00:25.087730+0000","last_became_active":"2024-04-25T10:00:25.087017+0000","last_became_peered":"2024-04-25T10:00:25.087017+0000","last_unstale":"2024-04-25T10:00:25.087730+0000","last_undegraded":"2024-04-25T10:00:25.087730+0000","last_fullsized":"2024-04-25T10:00:25.087730+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:10.581104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.092039+0000","last_change":"2024-04-25T10:00:25.092039+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.092039+0000","last_undegraded":"2024-04-25T10:00:25.092039+0000","last_fullsized":"2024-04-25T10:00:25.092039+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.098652+0000","last_change":"2024-04-25T10:00:25.095926+0000","last_active":"2024-04-25T10:00:25.098652+0000","last_peered":"2024-04-25T10:00:25.098652+0000","last_clean":"2024-04-25T10:00:25.098652+0000","last_became_active":"2024-04-25T10:00:25.095588+0000","last_became_peered":"2024-04-25T10:00:25.095588+0000","last_unstale":"2024-04-25T10:00:25.098652+0000","last_undegraded":"2024-04-25T10:00:25.098652+0000","last_fullsized":"2024-04-25T10:00:25.098652+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:20.122711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081036+0000","last_change":"2024-04-25T10:00:25.081036+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081036+0000","last_undegraded":"2024-04-25T10:00:25.081036+0000","last_fullsized":"2024-04-25T10:00:25.081036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.085107+0000","last_change":"2024-04-25T10:00:25.085107+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085107+0000","last_undegraded":"2024-04-25T10:00:25.085107+0000","last_fullsized":"2024-04-25T10:00:25.085107+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080448+0000","last_change":"2024-04-25T10:00:25.080448+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080448+0000","last_undegraded":"2024-04-25T10:00:25.080448+0000","last_fullsized":"2024-04-25T10:00:25.080448+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":3,"reported_epoch":95,"state":"creating+activating","last_fresh":"2024-04-25T10:00:25.083562+0000","last_change":"2024-04-25T10:00:25.083562+0000","last_active":"2024-04-25T10:00:24.068967+0000","last_peered":"2024-04-25T10:00:24.068967+0000","last_clean":"2024-04-25T10:00:24.068967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.083562+0000","last_undegraded":"2024-04-25T10:00:25.083562+0000","last_fullsized":"2024-04-25T10:00:25.083562+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.085633+0000","last_change":"2024-04-25T10:00:25.085633+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.085633+0000","last_undegraded":"2024-04-25T10:00:25.085633+0000","last_fullsized":"2024-04-25T10:00:25.085633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100414+0000","last_change":"2024-04-25T10:00:25.096895+0000","last_active":"2024-04-25T10:00:25.100414+0000","last_peered":"2024-04-25T10:00:25.100414+0000","last_clean":"2024-04-25T10:00:25.100414+0000","last_became_active":"2024-04-25T10:00:25.096545+0000","last_became_peered":"2024-04-25T10:00:25.096545+0000","last_unstale":"2024-04-25T10:00:25.100414+0000","last_undegraded":"2024-04-25T10:00:25.100414+0000","last_fullsized":"2024-04-25T10:00:25.100414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:51.071469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.080086+0000","last_change":"2024-04-25T10:00:25.080086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.080086+0000","last_undegraded":"2024-04-25T10:00:25.080086+0000","last_fullsized":"2024-04-25T10:00:25.080086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.096760+0000","last_change":"2024-04-25T10:00:25.096062+0000","last_active":"2024-04-25T10:00:25.096760+0000","last_peered":"2024-04-25T10:00:25.096760+0000","last_clean":"2024-04-25T10:00:25.096760+0000","last_became_active":"2024-04-25T10:00:25.095754+0000","last_became_peered":"2024-04-25T10:00:25.095754+0000","last_unstale":"2024-04-25T10:00:25.096760+0000","last_undegraded":"2024-04-25T10:00:25.096760+0000","last_fullsized":"2024-04-25T10:00:25.096760+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:04.002047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081155+0000","last_change":"2024-04-25T10:00:25.081155+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081155+0000","last_undegraded":"2024-04-25T10:00:25.081155+0000","last_fullsized":"2024-04-25T10:00:25.081155+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.090791+0000","last_change":"2024-04-25T10:00:25.089349+0000","last_active":"2024-04-25T10:00:25.090791+0000","last_peered":"2024-04-25T10:00:25.090791+0000","last_clean":"2024-04-25T10:00:25.090791+0000","last_became_active":"2024-04-25T10:00:25.089074+0000","last_became_peered":"2024-04-25T10:00:25.089074+0000","last_unstale":"2024-04-25T10:00:25.090791+0000","last_undegraded":"2024-04-25T10:00:25.090791+0000","last_fullsized":"2024-04-25T10:00:25.090791+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:21.487187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.089230+0000","last_change":"2024-04-25T10:00:25.089230+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.089230+0000","last_undegraded":"2024-04-25T10:00:25.089230+0000","last_fullsized":"2024-04-25T10:00:25.089230+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.100433+0000","last_change":"2024-04-25T10:00:25.097664+0000","last_active":"2024-04-25T10:00:25.100433+0000","last_peered":"2024-04-25T10:00:25.100433+0000","last_clean":"2024-04-25T10:00:25.100433+0000","last_became_active":"2024-04-25T10:00:25.097390+0000","last_became_peered":"2024-04-25T10:00:25.097390+0000","last_unstale":"2024-04-25T10:00:25.100433+0000","last_undegraded":"2024-04-25T10:00:25.100433+0000","last_fullsized":"2024-04-25T10:00:25.100433+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:24.773637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081086+0000","last_change":"2024-04-25T10:00:25.081086+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081086+0000","last_undegraded":"2024-04-25T10:00:25.081086+0000","last_fullsized":"2024-04-25T10:00:25.081086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.095329+0000","last_change":"2024-04-25T10:00:25.092559+0000","last_active":"2024-04-25T10:00:25.095329+0000","last_peered":"2024-04-25T10:00:25.095329+0000","last_clean":"2024-04-25T10:00:25.095329+0000","last_became_active":"2024-04-25T10:00:25.092275+0000","last_became_peered":"2024-04-25T10:00:25.092275+0000","last_unstale":"2024-04-25T10:00:25.095329+0000","last_undegraded":"2024-04-25T10:00:25.095329+0000","last_fullsized":"2024-04-25T10:00:25.095329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:07.600785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:00:25.081859+0000","last_change":"2024-04-25T10:00:25.081859+0000","last_active":"2024-04-25T10:00:25.073196+0000","last_peered":"2024-04-25T10:00:25.073196+0000","last_clean":"2024-04-25T10:00:25.073196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:00:25.081859+0000","last_undegraded":"2024-04-25T10:00:25.081859+0000","last_fullsized":"2024-04-25T10:00:25.081859+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.092495+0000","last_change":"2024-04-25T10:00:25.090514+0000","last_active":"2024-04-25T10:00:25.092495+0000","last_peered":"2024-04-25T10:00:25.092495+0000","last_clean":"2024-04-25T10:00:25.092495+0000","last_became_active":"2024-04-25T10:00:25.090248+0000","last_became_peered":"2024-04-25T10:00:25.090248+0000","last_unstale":"2024-04-25T10:00:25.092495+0000","last_undegraded":"2024-04-25T10:00:25.092495+0000","last_fullsized":"2024-04-25T10:00:25.092495+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:34.979909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":308,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:00:25.083144+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:00:25.083144+0000","last_peered":"2024-04-25T10:00:25.083144+0000","last_clean":"2024-04-25T10:00:25.083144+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:00:25.083144+0000","last_undegraded":"2024-04-25T10:00:25.083144+0000","last_fullsized":"2024-04-25T10:00:25.083144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":126,"acting":126,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46204,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325636,"statfs":{"total":96636764160,"available":96589451264,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62840,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309000,"statfs":{"total":96636764160,"available":96572416000,"internally_reserved":0,"allocated":1159168,"data_stored":534691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46780,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325060,"statfs":{"total":96636764160,"available":96588861440,"internally_reserved":0,"allocated":1753088,"data_stored":1125059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:31.341 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-25T10:00:31.337+0000 7f6cac478640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:00:33.431 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:00:33.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.535+0000 7fa5517ee640 1 Processor -- start 2024-04-25T10:00:33.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.535+0000 7fa5517ee640 1 -- start start 2024-04-25T10:00:33.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa5517ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa54c15dbe0 0x7fa54c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa5517ee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c0b6220 con 0x7fa54c0b2cb0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa5517ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c0b97a0 con 0x7fa54c0b92a0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa5517ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c15dfc0 con 0x7fa54c15dbe0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa54a7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa54c0b2cb0 0x7fa54c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59574/0 (socket says 172.21.15.148:59574) 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa54a7fc640 1 -- 172.21.15.148:0/83822546 learned_addr learned my addr 172.21.15.148:0/83822546 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa54affd640 1 --2- 172.21.15.148:0/83822546 >> v2:172.21.15.188:3300/0 conn(0x7fa54c15dbe0 0x7fa54c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737390066 0 0) 0x7fa54c0b6220 con 0x7fa54c0b2cb0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.536+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa534003520 con 0x7fa54c0b2cb0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362519468 0 0) 0x7fa534003520 con 0x7fa54c0b2cb0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 >> v2:172.21.15.188:3300/0 conn(0x7fa54c15dbe0 msgr2=0x7fa54c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 --2- 172.21.15.148:0/83822546 >> v2:172.21.15.188:3300/0 conn(0x7fa54c15dbe0 0x7fa54c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 >> v1:172.21.15.188:6789/0 conn(0x7fa54c0b92a0 legacy=0x7fa54c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0d1c50 con 0x7fa54c0b2cb0 2024-04-25T10:00:33.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa540001670 con 0x7fa54c0b2cb0 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 >> v1:172.21.15.148:6789/0 conn(0x7fa54c0b2cb0 legacy=0x7fa54c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5340039b0 0x7fa534003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa534004800 0x7fa534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340051c0 0x7fa53400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53400c130 con 0x7fa5340051c0 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53400c2e0 con 0x7fa5340039b0 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53400c4e0 con 0x7fa534004800 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340051c0 0x7fa53400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54affd640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5340039b0 0x7fa534003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54a7fc640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa534004800 0x7fa534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54b7fe640 1 -- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa534004800 msgr2=0x7fa534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa534004800 0x7fa534004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54b7fe640 1 -- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5340039b0 msgr2=0x7fa534003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5340039b0 0x7fa534003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.537+0000 7fa54b7fe640 1 -- 172.21.15.148:0/83822546 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa534003520 con 0x7fa5340051c0 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.538+0000 7fa54b7fe640 1 -- 172.21.15.148:0/83822546 reap_dead start 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.538+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340051c0 0x7fa53400b9c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa544001e20 tx=0x7fa534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a5f196409a401a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.538+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544001a70 con 0x7fa5340051c0 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.538+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54402ae40 con 0x7fa5340051c0 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.538+0000 7fa549ffb640 1 -- 172.21.15.148:0/83822546 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544003bb0 con 0x7fa5340051c0 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.538+0000 7fa5517ee640 1 -- 172.21.15.148:0/83822546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340051c0 msgr2=0x7fa53400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.539+0000 7fa5517ee640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340051c0 0x7fa53400b9c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa544001e20 tx=0x7fa534005830 comp rx=0 tx=0).stop 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.539+0000 7fa5517ee640 1 -- 172.21.15.148:0/83822546 shutdown_connections 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.539+0000 7fa5517ee640 1 --2- 172.21.15.148:0/83822546 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5340051c0 0x7fa53400b9c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.539+0000 7fa5517ee640 1 -- 172.21.15.148:0/83822546 >> 172.21.15.148:0/83822546 conn(0x7fa54c061a20 msgr2=0x7fa54c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.539+0000 7fa5517ee640 1 -- 172.21.15.148:0/83822546 shutdown_connections 2024-04-25T10:00:33.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.539+0000 7fa5517ee640 1 -- 172.21.15.148:0/83822546 wait complete. 2024-04-25T10:00:33.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.540+0000 7fa5517ee640 1 Processor -- start 2024-04-25T10:00:33.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.540+0000 7fa5517ee640 1 -- start start 2024-04-25T10:00:33.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5340051c0 0x7fa54c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa54c0ab060 0x7fa54c15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 0x7fa54c0aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0d2210 con 0x7fa54c15fa50 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0d1a00 con 0x7fa54c0ab060 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c0b6ab0 con 0x7fa5340051c0 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 0x7fa54c0aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 0x7fa54c0aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34842/0 (socket says 172.21.15.148:34842) 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 -- 172.21.15.148:0/2264884018 learned_addr learned my addr 172.21.15.148:0/2264884018 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54affd640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5340051c0 0x7fa54c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54a7fc640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa54c0ab060 0x7fa54c15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 -- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5340051c0 msgr2=0x7fa54c159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5340051c0 0x7fa54c159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 -- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa54c0ab060 msgr2=0x7fa54c15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa54c0ab060 0x7fa54c15f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 -- 172.21.15.148:0/2264884018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c15ff70 con 0x7fa54c15fa50 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa54b7fe640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 0x7fa54c0aa370 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa544001f40 tx=0x7fa5440014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544007db0 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544006d60 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c160250 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.541+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440056c0 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.542+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c0cbeb0 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.543+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c073990 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.543+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa544007030 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.543+0000 7fa5337fe640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa53806b500 0x7fa53806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.544+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5440a9db0 con 0x7fa54c15fa50 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.544+0000 7fa54affd640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa53806b500 0x7fa53806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.544+0000 7fa54affd640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa53806b500 0x7fa53806d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009a80 tx=0x7fa53c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.548+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa544043060 con 0x7fa54c15fa50 2024-04-25T10:00:33.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.714+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa54c159ea0 con 0x7fa53806b500 2024-04-25T10:00:33.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.720+0000 7fa5337fe640 1 -- 172.21.15.148:0/2264884018 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252076 (secure 0 0 0) 0x7fa54c159ea0 con 0x7fa53806b500 2024-04-25T10:00:33.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:33.730 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:00:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa53806b500 msgr2=0x7fa53806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa53806b500 0x7fa53806d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009a80 tx=0x7fa53c008040 comp rx=0 tx=0).stop 2024-04-25T10:00:33.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 msgr2=0x7fa54c0aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 0x7fa54c0aa370 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fa544001f40 tx=0x7fa5440014a0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 shutdown_connections 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa53806b500 0x7fa53806d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa54c15fa50 0x7fa54c0aa370 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa54c0ab060 0x7fa54c15f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.723+0000 7fa5517ee640 1 --2- 172.21.15.148:0/2264884018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5340051c0 0x7fa54c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.724+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 >> 172.21.15.148:0/2264884018 conn(0x7fa54c061a20 msgr2=0x7fa54c0d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.724+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 shutdown_connections 2024-04-25T10:00:33.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.724+0000 7fa5517ee640 1 -- 172.21.15.148:0/2264884018 wait complete. 2024-04-25T10:00:33.740 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-04-25T10:00:33.096310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":375224,"kb_used_data":10568,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599496,"statfs":{"total":773094113280,"available":772709883904,"internally_reserved":0,"allocated":10821632,"data_stored":5627292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001813"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636408+0000","last_change":"2024-04-25T10:00:26.117159+0000","last_active":"2024-04-25T10:00:27.636408+0000","last_peered":"2024-04-25T10:00:27.636408+0000","last_clean":"2024-04-25T10:00:27.636408+0000","last_became_active":"2024-04-25T10:00:26.116890+0000","last_became_peered":"2024-04-25T10:00:26.116890+0000","last_unstale":"2024-04-25T10:00:27.636408+0000","last_undegraded":"2024-04-25T10:00:27.636408+0000","last_fullsized":"2024-04-25T10:00:27.636408+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:21.032139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636432+0000","last_change":"2024-04-25T10:00:26.099830+0000","last_active":"2024-04-25T10:00:27.636432+0000","last_peered":"2024-04-25T10:00:27.636432+0000","last_clean":"2024-04-25T10:00:27.636432+0000","last_became_active":"2024-04-25T10:00:26.099534+0000","last_became_peered":"2024-04-25T10:00:26.099534+0000","last_unstale":"2024-04-25T10:00:27.636432+0000","last_undegraded":"2024-04-25T10:00:27.636432+0000","last_fullsized":"2024-04-25T10:00:27.636432+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:01.444519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636734+0000","last_change":"2024-04-25T10:00:26.116788+0000","last_active":"2024-04-25T10:00:27.636734+0000","last_peered":"2024-04-25T10:00:27.636734+0000","last_clean":"2024-04-25T10:00:27.636734+0000","last_became_active":"2024-04-25T10:00:26.116524+0000","last_became_peered":"2024-04-25T10:00:26.116524+0000","last_unstale":"2024-04-25T10:00:27.636734+0000","last_undegraded":"2024-04-25T10:00:27.636734+0000","last_fullsized":"2024-04-25T10:00:27.636734+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:21.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637128+0000","last_change":"2024-04-25T10:00:26.117694+0000","last_active":"2024-04-25T10:00:27.637128+0000","last_peered":"2024-04-25T10:00:27.637128+0000","last_clean":"2024-04-25T10:00:27.637128+0000","last_became_active":"2024-04-25T10:00:26.117418+0000","last_became_peered":"2024-04-25T10:00:26.117418+0000","last_unstale":"2024-04-25T10:00:27.637128+0000","last_undegraded":"2024-04-25T10:00:27.637128+0000","last_fullsized":"2024-04-25T10:00:27.637128+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:08.605014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093316+0000","last_change":"2024-04-25T10:00:26.103858+0000","last_active":"2024-04-25T10:00:27.093316+0000","last_peered":"2024-04-25T10:00:27.093316+0000","last_clean":"2024-04-25T10:00:27.093316+0000","last_became_active":"2024-04-25T10:00:26.103558+0000","last_became_peered":"2024-04-25T10:00:26.103558+0000","last_unstale":"2024-04-25T10:00:27.093316+0000","last_undegraded":"2024-04-25T10:00:27.093316+0000","last_fullsized":"2024-04-25T10:00:27.093316+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:46.760399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091611+0000","last_change":"2024-04-25T10:00:26.122625+0000","last_active":"2024-04-25T10:00:27.091611+0000","last_peered":"2024-04-25T10:00:27.091611+0000","last_clean":"2024-04-25T10:00:27.091611+0000","last_became_active":"2024-04-25T10:00:26.122342+0000","last_became_peered":"2024-04-25T10:00:26.122342+0000","last_unstale":"2024-04-25T10:00:27.091611+0000","last_undegraded":"2024-04-25T10:00:27.091611+0000","last_fullsized":"2024-04-25T10:00:27.091611+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:57.814727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636542+0000","last_change":"2024-04-25T10:00:26.098729+0000","last_active":"2024-04-25T10:00:27.636542+0000","last_peered":"2024-04-25T10:00:27.636542+0000","last_clean":"2024-04-25T10:00:27.636542+0000","last_became_active":"2024-04-25T10:00:26.098441+0000","last_became_peered":"2024-04-25T10:00:26.098441+0000","last_unstale":"2024-04-25T10:00:27.636542+0000","last_undegraded":"2024-04-25T10:00:27.636542+0000","last_fullsized":"2024-04-25T10:00:27.636542+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:18.152201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636435+0000","last_change":"2024-04-25T10:00:26.115297+0000","last_active":"2024-04-25T10:00:27.636435+0000","last_peered":"2024-04-25T10:00:27.636435+0000","last_clean":"2024-04-25T10:00:27.636435+0000","last_became_active":"2024-04-25T10:00:26.113853+0000","last_became_peered":"2024-04-25T10:00:26.113853+0000","last_unstale":"2024-04-25T10:00:27.636435+0000","last_undegraded":"2024-04-25T10:00:27.636435+0000","last_fullsized":"2024-04-25T10:00:27.636435+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:49.888391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093549+0000","last_change":"2024-04-25T10:00:26.110045+0000","last_active":"2024-04-25T10:00:27.093549+0000","last_peered":"2024-04-25T10:00:27.093549+0000","last_clean":"2024-04-25T10:00:27.093549+0000","last_became_active":"2024-04-25T10:00:26.109674+0000","last_became_peered":"2024-04-25T10:00:26.109674+0000","last_unstale":"2024-04-25T10:00:27.093549+0000","last_undegraded":"2024-04-25T10:00:27.093549+0000","last_fullsized":"2024-04-25T10:00:27.093549+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:24.676020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636829+0000","last_change":"2024-04-25T10:00:26.116281+0000","last_active":"2024-04-25T10:00:27.636829+0000","last_peered":"2024-04-25T10:00:27.636829+0000","last_clean":"2024-04-25T10:00:27.636829+0000","last_became_active":"2024-04-25T10:00:26.116004+0000","last_became_peered":"2024-04-25T10:00:26.116004+0000","last_unstale":"2024-04-25T10:00:27.636829+0000","last_undegraded":"2024-04-25T10:00:27.636829+0000","last_fullsized":"2024-04-25T10:00:27.636829+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:00.595774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093412+0000","last_change":"2024-04-25T10:00:26.111831+0000","last_active":"2024-04-25T10:00:27.093412+0000","last_peered":"2024-04-25T10:00:27.093412+0000","last_clean":"2024-04-25T10:00:27.093412+0000","last_became_active":"2024-04-25T10:00:26.111075+0000","last_became_peered":"2024-04-25T10:00:26.111075+0000","last_unstale":"2024-04-25T10:00:27.093412+0000","last_undegraded":"2024-04-25T10:00:27.093412+0000","last_fullsized":"2024-04-25T10:00:27.093412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:57.671341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636875+0000","last_change":"2024-04-25T10:00:26.112932+0000","last_active":"2024-04-25T10:00:27.636875+0000","last_peered":"2024-04-25T10:00:27.636875+0000","last_clean":"2024-04-25T10:00:27.636875+0000","last_became_active":"2024-04-25T10:00:26.112514+0000","last_became_peered":"2024-04-25T10:00:26.112514+0000","last_unstale":"2024-04-25T10:00:27.636875+0000","last_undegraded":"2024-04-25T10:00:27.636875+0000","last_fullsized":"2024-04-25T10:00:27.636875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:05.129583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636726+0000","last_change":"2024-04-25T10:00:26.108916+0000","last_active":"2024-04-25T10:00:27.636726+0000","last_peered":"2024-04-25T10:00:27.636726+0000","last_clean":"2024-04-25T10:00:27.636726+0000","last_became_active":"2024-04-25T10:00:26.107937+0000","last_became_peered":"2024-04-25T10:00:26.107937+0000","last_unstale":"2024-04-25T10:00:27.636726+0000","last_undegraded":"2024-04-25T10:00:27.636726+0000","last_fullsized":"2024-04-25T10:00:27.636726+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:37.942553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092866+0000","last_change":"2024-04-25T10:00:26.121025+0000","last_active":"2024-04-25T10:00:27.092866+0000","last_peered":"2024-04-25T10:00:27.092866+0000","last_clean":"2024-04-25T10:00:27.092866+0000","last_became_active":"2024-04-25T10:00:26.120805+0000","last_became_peered":"2024-04-25T10:00:26.120805+0000","last_unstale":"2024-04-25T10:00:27.092866+0000","last_undegraded":"2024-04-25T10:00:27.092866+0000","last_fullsized":"2024-04-25T10:00:27.092866+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:30.179146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637407+0000","last_change":"2024-04-25T10:00:26.111340+0000","last_active":"2024-04-25T10:00:27.637407+0000","last_peered":"2024-04-25T10:00:27.637407+0000","last_clean":"2024-04-25T10:00:27.637407+0000","last_became_active":"2024-04-25T10:00:26.110736+0000","last_became_peered":"2024-04-25T10:00:26.110736+0000","last_unstale":"2024-04-25T10:00:27.637407+0000","last_undegraded":"2024-04-25T10:00:27.637407+0000","last_fullsized":"2024-04-25T10:00:27.637407+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:46.760399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091574+0000","last_change":"2024-04-25T10:00:26.112469+0000","last_active":"2024-04-25T10:00:27.091574+0000","last_peered":"2024-04-25T10:00:27.091574+0000","last_clean":"2024-04-25T10:00:27.091574+0000","last_became_active":"2024-04-25T10:00:26.112082+0000","last_became_peered":"2024-04-25T10:00:26.112082+0000","last_unstale":"2024-04-25T10:00:27.091574+0000","last_undegraded":"2024-04-25T10:00:27.091574+0000","last_fullsized":"2024-04-25T10:00:27.091574+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:38.205031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091979+0000","last_change":"2024-04-25T10:00:26.113232+0000","last_active":"2024-04-25T10:00:27.091979+0000","last_peered":"2024-04-25T10:00:27.091979+0000","last_clean":"2024-04-25T10:00:27.091979+0000","last_became_active":"2024-04-25T10:00:26.112962+0000","last_became_peered":"2024-04-25T10:00:26.112962+0000","last_unstale":"2024-04-25T10:00:27.091979+0000","last_undegraded":"2024-04-25T10:00:27.091979+0000","last_fullsized":"2024-04-25T10:00:27.091979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:32.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636592+0000","last_change":"2024-04-25T10:00:26.102579+0000","last_active":"2024-04-25T10:00:27.636592+0000","last_peered":"2024-04-25T10:00:27.636592+0000","last_clean":"2024-04-25T10:00:27.636592+0000","last_became_active":"2024-04-25T10:00:26.102277+0000","last_became_peered":"2024-04-25T10:00:26.102277+0000","last_unstale":"2024-04-25T10:00:27.636592+0000","last_undegraded":"2024-04-25T10:00:27.636592+0000","last_fullsized":"2024-04-25T10:00:27.636592+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:28.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093555+0000","last_change":"2024-04-25T10:00:26.117311+0000","last_active":"2024-04-25T10:00:27.093555+0000","last_peered":"2024-04-25T10:00:27.093555+0000","last_clean":"2024-04-25T10:00:27.093555+0000","last_became_active":"2024-04-25T10:00:26.116986+0000","last_became_peered":"2024-04-25T10:00:26.116986+0000","last_unstale":"2024-04-25T10:00:27.093555+0000","last_undegraded":"2024-04-25T10:00:27.093555+0000","last_fullsized":"2024-04-25T10:00:27.093555+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:52.221201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637226+0000","last_change":"2024-04-25T10:00:26.115453+0000","last_active":"2024-04-25T10:00:27.637226+0000","last_peered":"2024-04-25T10:00:27.637226+0000","last_clean":"2024-04-25T10:00:27.637226+0000","last_became_active":"2024-04-25T10:00:26.114870+0000","last_became_peered":"2024-04-25T10:00:26.114870+0000","last_unstale":"2024-04-25T10:00:27.637226+0000","last_undegraded":"2024-04-25T10:00:27.637226+0000","last_fullsized":"2024-04-25T10:00:27.637226+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:21.126939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092995+0000","last_change":"2024-04-25T10:00:26.109209+0000","last_active":"2024-04-25T10:00:27.092995+0000","last_peered":"2024-04-25T10:00:27.092995+0000","last_clean":"2024-04-25T10:00:27.092995+0000","last_became_active":"2024-04-25T10:00:26.108345+0000","last_became_peered":"2024-04-25T10:00:26.108345+0000","last_unstale":"2024-04-25T10:00:27.092995+0000","last_undegraded":"2024-04-25T10:00:27.092995+0000","last_fullsized":"2024-04-25T10:00:27.092995+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:21.126939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092923+0000","last_change":"2024-04-25T10:00:26.120219+0000","last_active":"2024-04-25T10:00:27.092923+0000","last_peered":"2024-04-25T10:00:27.092923+0000","last_clean":"2024-04-25T10:00:27.092923+0000","last_became_active":"2024-04-25T10:00:26.119997+0000","last_became_peered":"2024-04-25T10:00:26.119997+0000","last_unstale":"2024-04-25T10:00:27.092923+0000","last_undegraded":"2024-04-25T10:00:27.092923+0000","last_fullsized":"2024-04-25T10:00:27.092923+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:08.605014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637566+0000","last_change":"2024-04-25T10:00:26.122080+0000","last_active":"2024-04-25T10:00:27.637566+0000","last_peered":"2024-04-25T10:00:27.637566+0000","last_clean":"2024-04-25T10:00:27.637566+0000","last_became_active":"2024-04-25T10:00:26.121782+0000","last_became_peered":"2024-04-25T10:00:26.121782+0000","last_unstale":"2024-04-25T10:00:27.637566+0000","last_undegraded":"2024-04-25T10:00:27.637566+0000","last_fullsized":"2024-04-25T10:00:27.637566+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:21.126939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636202+0000","last_change":"2024-04-25T10:00:26.117768+0000","last_active":"2024-04-25T10:00:27.636202+0000","last_peered":"2024-04-25T10:00:27.636202+0000","last_clean":"2024-04-25T10:00:27.636202+0000","last_became_active":"2024-04-25T10:00:26.117334+0000","last_became_peered":"2024-04-25T10:00:26.117334+0000","last_unstale":"2024-04-25T10:00:27.636202+0000","last_undegraded":"2024-04-25T10:00:27.636202+0000","last_fullsized":"2024-04-25T10:00:27.636202+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:08.058617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636906+0000","last_change":"2024-04-25T10:00:26.103096+0000","last_active":"2024-04-25T10:00:27.636906+0000","last_peered":"2024-04-25T10:00:27.636906+0000","last_clean":"2024-04-25T10:00:27.636906+0000","last_became_active":"2024-04-25T10:00:26.102749+0000","last_became_peered":"2024-04-25T10:00:26.102749+0000","last_unstale":"2024-04-25T10:00:27.636906+0000","last_undegraded":"2024-04-25T10:00:27.636906+0000","last_fullsized":"2024-04-25T10:00:27.636906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:18.152201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093412+0000","last_change":"2024-04-25T10:00:26.117308+0000","last_active":"2024-04-25T10:00:27.093412+0000","last_peered":"2024-04-25T10:00:27.093412+0000","last_clean":"2024-04-25T10:00:27.093412+0000","last_became_active":"2024-04-25T10:00:26.117002+0000","last_became_peered":"2024-04-25T10:00:26.117002+0000","last_unstale":"2024-04-25T10:00:27.093412+0000","last_undegraded":"2024-04-25T10:00:27.093412+0000","last_fullsized":"2024-04-25T10:00:27.093412+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:32.433774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636918+0000","last_change":"2024-04-25T10:00:26.104220+0000","last_active":"2024-04-25T10:00:27.636918+0000","last_peered":"2024-04-25T10:00:27.636918+0000","last_clean":"2024-04-25T10:00:27.636918+0000","last_became_active":"2024-04-25T10:00:26.103933+0000","last_became_peered":"2024-04-25T10:00:26.103933+0000","last_unstale":"2024-04-25T10:00:27.636918+0000","last_undegraded":"2024-04-25T10:00:27.636918+0000","last_fullsized":"2024-04-25T10:00:27.636918+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:30.772570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637346+0000","last_change":"2024-04-25T10:00:26.108635+0000","last_active":"2024-04-25T10:00:27.637346+0000","last_peered":"2024-04-25T10:00:27.637346+0000","last_clean":"2024-04-25T10:00:27.637346+0000","last_became_active":"2024-04-25T10:00:26.108273+0000","last_became_peered":"2024-04-25T10:00:26.108273+0000","last_unstale":"2024-04-25T10:00:27.637346+0000","last_undegraded":"2024-04-25T10:00:27.637346+0000","last_fullsized":"2024-04-25T10:00:27.637346+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:03.705935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636864+0000","last_change":"2024-04-25T10:00:26.100799+0000","last_active":"2024-04-25T10:00:27.636864+0000","last_peered":"2024-04-25T10:00:27.636864+0000","last_clean":"2024-04-25T10:00:27.636864+0000","last_became_active":"2024-04-25T10:00:26.100482+0000","last_became_peered":"2024-04-25T10:00:26.100482+0000","last_unstale":"2024-04-25T10:00:27.636864+0000","last_undegraded":"2024-04-25T10:00:27.636864+0000","last_fullsized":"2024-04-25T10:00:27.636864+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:21.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093086+0000","last_change":"2024-04-25T10:00:26.112039+0000","last_active":"2024-04-25T10:00:27.093086+0000","last_peered":"2024-04-25T10:00:27.093086+0000","last_clean":"2024-04-25T10:00:27.093086+0000","last_became_active":"2024-04-25T10:00:26.111757+0000","last_became_peered":"2024-04-25T10:00:26.111757+0000","last_unstale":"2024-04-25T10:00:27.093086+0000","last_undegraded":"2024-04-25T10:00:27.093086+0000","last_fullsized":"2024-04-25T10:00:27.093086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:03.705935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636472+0000","last_change":"2024-04-25T10:00:26.102639+0000","last_active":"2024-04-25T10:00:27.636472+0000","last_peered":"2024-04-25T10:00:27.636472+0000","last_clean":"2024-04-25T10:00:27.636472+0000","last_became_active":"2024-04-25T10:00:26.102382+0000","last_became_peered":"2024-04-25T10:00:26.102382+0000","last_unstale":"2024-04-25T10:00:27.636472+0000","last_undegraded":"2024-04-25T10:00:27.636472+0000","last_fullsized":"2024-04-25T10:00:27.636472+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:21.032139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091655+0000","last_change":"2024-04-25T10:00:26.112998+0000","last_active":"2024-04-25T10:00:27.091655+0000","last_peered":"2024-04-25T10:00:27.091655+0000","last_clean":"2024-04-25T10:00:27.091655+0000","last_became_active":"2024-04-25T10:00:26.112535+0000","last_became_peered":"2024-04-25T10:00:26.112535+0000","last_unstale":"2024-04-25T10:00:27.091655+0000","last_undegraded":"2024-04-25T10:00:27.091655+0000","last_fullsized":"2024-04-25T10:00:27.091655+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:37.550361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091759+0000","last_change":"2024-04-25T10:00:26.108680+0000","last_active":"2024-04-25T10:00:27.091759+0000","last_peered":"2024-04-25T10:00:27.091759+0000","last_clean":"2024-04-25T10:00:27.091759+0000","last_became_active":"2024-04-25T10:00:26.108406+0000","last_became_peered":"2024-04-25T10:00:26.108406+0000","last_unstale":"2024-04-25T10:00:27.091759+0000","last_undegraded":"2024-04-25T10:00:27.091759+0000","last_fullsized":"2024-04-25T10:00:27.091759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:39.680442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637204+0000","last_change":"2024-04-25T10:00:25.098352+0000","last_active":"2024-04-25T10:00:27.637204+0000","last_peered":"2024-04-25T10:00:27.637204+0000","last_clean":"2024-04-25T10:00:27.637204+0000","last_became_active":"2024-04-25T10:00:25.098081+0000","last_became_peered":"2024-04-25T10:00:25.098081+0000","last_unstale":"2024-04-25T10:00:27.637204+0000","last_undegraded":"2024-04-25T10:00:27.637204+0000","last_fullsized":"2024-04-25T10:00:27.637204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:41.508550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092145+0000","last_change":"2024-04-25T10:00:26.107693+0000","last_active":"2024-04-25T10:00:27.092145+0000","last_peered":"2024-04-25T10:00:27.092145+0000","last_clean":"2024-04-25T10:00:27.092145+0000","last_became_active":"2024-04-25T10:00:26.107343+0000","last_became_peered":"2024-04-25T10:00:26.107343+0000","last_unstale":"2024-04-25T10:00:27.092145+0000","last_undegraded":"2024-04-25T10:00:27.092145+0000","last_fullsized":"2024-04-25T10:00:27.092145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:54.927060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093530+0000","last_change":"2024-04-25T10:00:25.093361+0000","last_active":"2024-04-25T10:00:27.093530+0000","last_peered":"2024-04-25T10:00:27.093530+0000","last_clean":"2024-04-25T10:00:27.093530+0000","last_became_active":"2024-04-25T10:00:25.092838+0000","last_became_peered":"2024-04-25T10:00:25.092838+0000","last_unstale":"2024-04-25T10:00:27.093530+0000","last_undegraded":"2024-04-25T10:00:27.093530+0000","last_fullsized":"2024-04-25T10:00:27.093530+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:26.596265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636765+0000","last_change":"2024-04-25T10:00:26.100809+0000","last_active":"2024-04-25T10:00:27.636765+0000","last_peered":"2024-04-25T10:00:27.636765+0000","last_clean":"2024-04-25T10:00:27.636765+0000","last_became_active":"2024-04-25T10:00:26.100540+0000","last_became_peered":"2024-04-25T10:00:26.100540+0000","last_unstale":"2024-04-25T10:00:27.636765+0000","last_undegraded":"2024-04-25T10:00:27.636765+0000","last_fullsized":"2024-04-25T10:00:27.636765+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:28.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636676+0000","last_change":"2024-04-25T10:00:25.087470+0000","last_active":"2024-04-25T10:00:27.636676+0000","last_peered":"2024-04-25T10:00:27.636676+0000","last_clean":"2024-04-25T10:00:27.636676+0000","last_became_active":"2024-04-25T10:00:25.087191+0000","last_became_peered":"2024-04-25T10:00:25.087191+0000","last_unstale":"2024-04-25T10:00:27.636676+0000","last_undegraded":"2024-04-25T10:00:27.636676+0000","last_fullsized":"2024-04-25T10:00:27.636676+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:23.671791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092022+0000","last_change":"2024-04-25T10:00:26.106644+0000","last_active":"2024-04-25T10:00:27.092022+0000","last_peered":"2024-04-25T10:00:27.092022+0000","last_clean":"2024-04-25T10:00:27.092022+0000","last_became_active":"2024-04-25T10:00:26.106357+0000","last_became_peered":"2024-04-25T10:00:26.106357+0000","last_unstale":"2024-04-25T10:00:27.092022+0000","last_undegraded":"2024-04-25T10:00:27.092022+0000","last_fullsized":"2024-04-25T10:00:27.092022+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:35.639385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637537+0000","last_change":"2024-04-25T10:00:25.096412+0000","last_active":"2024-04-25T10:00:27.637537+0000","last_peered":"2024-04-25T10:00:27.637537+0000","last_clean":"2024-04-25T10:00:27.637537+0000","last_became_active":"2024-04-25T10:00:25.095816+0000","last_became_peered":"2024-04-25T10:00:25.095816+0000","last_unstale":"2024-04-25T10:00:27.637537+0000","last_undegraded":"2024-04-25T10:00:27.637537+0000","last_fullsized":"2024-04-25T10:00:27.637537+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:31.055739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636527+0000","last_change":"2024-04-25T10:00:26.111255+0000","last_active":"2024-04-25T10:00:27.636527+0000","last_peered":"2024-04-25T10:00:27.636527+0000","last_clean":"2024-04-25T10:00:27.636527+0000","last_became_active":"2024-04-25T10:00:26.110415+0000","last_became_peered":"2024-04-25T10:00:26.110415+0000","last_unstale":"2024-04-25T10:00:27.636527+0000","last_undegraded":"2024-04-25T10:00:27.636527+0000","last_fullsized":"2024-04-25T10:00:27.636527+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:22.840981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093326+0000","last_change":"2024-04-25T10:00:25.094826+0000","last_active":"2024-04-25T10:00:27.093326+0000","last_peered":"2024-04-25T10:00:27.093326+0000","last_clean":"2024-04-25T10:00:27.093326+0000","last_became_active":"2024-04-25T10:00:25.094268+0000","last_became_peered":"2024-04-25T10:00:25.094268+0000","last_unstale":"2024-04-25T10:00:27.093326+0000","last_undegraded":"2024-04-25T10:00:27.093326+0000","last_fullsized":"2024-04-25T10:00:27.093326+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:25.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637595+0000","last_change":"2024-04-25T10:00:26.101813+0000","last_active":"2024-04-25T10:00:27.637595+0000","last_peered":"2024-04-25T10:00:27.637595+0000","last_clean":"2024-04-25T10:00:27.637595+0000","last_became_active":"2024-04-25T10:00:26.101543+0000","last_became_peered":"2024-04-25T10:00:26.101543+0000","last_unstale":"2024-04-25T10:00:27.637595+0000","last_undegraded":"2024-04-25T10:00:27.637595+0000","last_fullsized":"2024-04-25T10:00:27.637595+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:03.705935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637248+0000","last_change":"2024-04-25T10:00:25.090629+0000","last_active":"2024-04-25T10:00:27.637248+0000","last_peered":"2024-04-25T10:00:27.637248+0000","last_clean":"2024-04-25T10:00:27.637248+0000","last_became_active":"2024-04-25T10:00:25.090354+0000","last_became_peered":"2024-04-25T10:00:25.090354+0000","last_unstale":"2024-04-25T10:00:27.637248+0000","last_undegraded":"2024-04-25T10:00:27.637248+0000","last_fullsized":"2024-04-25T10:00:27.637248+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:31.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091660+0000","last_change":"2024-04-25T10:00:26.115143+0000","last_active":"2024-04-25T10:00:27.091660+0000","last_peered":"2024-04-25T10:00:27.091660+0000","last_clean":"2024-04-25T10:00:27.091660+0000","last_became_active":"2024-04-25T10:00:26.114877+0000","last_became_peered":"2024-04-25T10:00:26.114877+0000","last_unstale":"2024-04-25T10:00:27.091660+0000","last_undegraded":"2024-04-25T10:00:27.091660+0000","last_fullsized":"2024-04-25T10:00:27.091660+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:38.929614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636571+0000","last_change":"2024-04-25T10:00:25.086108+0000","last_active":"2024-04-25T10:00:27.636571+0000","last_peered":"2024-04-25T10:00:27.636571+0000","last_clean":"2024-04-25T10:00:27.636571+0000","last_became_active":"2024-04-25T10:00:25.085829+0000","last_became_peered":"2024-04-25T10:00:25.085829+0000","last_unstale":"2024-04-25T10:00:27.636571+0000","last_undegraded":"2024-04-25T10:00:27.636571+0000","last_fullsized":"2024-04-25T10:00:27.636571+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:56.667112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636804+0000","last_change":"2024-04-25T10:00:26.111525+0000","last_active":"2024-04-25T10:00:27.636804+0000","last_peered":"2024-04-25T10:00:27.636804+0000","last_clean":"2024-04-25T10:00:27.636804+0000","last_became_active":"2024-04-25T10:00:26.108293+0000","last_became_peered":"2024-04-25T10:00:26.108293+0000","last_unstale":"2024-04-25T10:00:27.636804+0000","last_undegraded":"2024-04-25T10:00:27.636804+0000","last_fullsized":"2024-04-25T10:00:27.636804+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:30.179146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636886+0000","last_change":"2024-04-25T10:00:25.097674+0000","last_active":"2024-04-25T10:00:27.636886+0000","last_peered":"2024-04-25T10:00:27.636886+0000","last_clean":"2024-04-25T10:00:27.636886+0000","last_became_active":"2024-04-25T10:00:25.097383+0000","last_became_peered":"2024-04-25T10:00:25.097383+0000","last_unstale":"2024-04-25T10:00:27.636886+0000","last_undegraded":"2024-04-25T10:00:27.636886+0000","last_fullsized":"2024-04-25T10:00:27.636886+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:40.180045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093375+0000","last_change":"2024-04-25T10:00:26.112958+0000","last_active":"2024-04-25T10:00:27.093375+0000","last_peered":"2024-04-25T10:00:27.093375+0000","last_clean":"2024-04-25T10:00:27.093375+0000","last_became_active":"2024-04-25T10:00:26.112687+0000","last_became_peered":"2024-04-25T10:00:26.112687+0000","last_unstale":"2024-04-25T10:00:27.093375+0000","last_undegraded":"2024-04-25T10:00:27.093375+0000","last_fullsized":"2024-04-25T10:00:27.093375+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:41.184274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637150+0000","last_change":"2024-04-25T10:00:25.096537+0000","last_active":"2024-04-25T10:00:27.637150+0000","last_peered":"2024-04-25T10:00:27.637150+0000","last_clean":"2024-04-25T10:00:27.637150+0000","last_became_active":"2024-04-25T10:00:25.096221+0000","last_became_peered":"2024-04-25T10:00:25.096221+0000","last_unstale":"2024-04-25T10:00:27.637150+0000","last_undegraded":"2024-04-25T10:00:27.637150+0000","last_fullsized":"2024-04-25T10:00:27.637150+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:40.180045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636656+0000","last_change":"2024-04-25T10:00:26.106251+0000","last_active":"2024-04-25T10:00:27.636656+0000","last_peered":"2024-04-25T10:00:27.636656+0000","last_clean":"2024-04-25T10:00:27.636656+0000","last_became_active":"2024-04-25T10:00:26.105974+0000","last_became_peered":"2024-04-25T10:00:26.105974+0000","last_unstale":"2024-04-25T10:00:27.636656+0000","last_undegraded":"2024-04-25T10:00:27.636656+0000","last_fullsized":"2024-04-25T10:00:27.636656+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:21.032139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636508+0000","last_change":"2024-04-25T10:00:25.093114+0000","last_active":"2024-04-25T10:00:27.636508+0000","last_peered":"2024-04-25T10:00:27.636508+0000","last_clean":"2024-04-25T10:00:27.636508+0000","last_became_active":"2024-04-25T10:00:25.092835+0000","last_became_peered":"2024-04-25T10:00:25.092835+0000","last_unstale":"2024-04-25T10:00:27.636508+0000","last_undegraded":"2024-04-25T10:00:27.636508+0000","last_fullsized":"2024-04-25T10:00:27.636508+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:31.429546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636416+0000","last_change":"2024-04-25T10:00:26.118603+0000","last_active":"2024-04-25T10:00:27.636416+0000","last_peered":"2024-04-25T10:00:27.636416+0000","last_clean":"2024-04-25T10:00:27.636416+0000","last_became_active":"2024-04-25T10:00:26.118340+0000","last_became_peered":"2024-04-25T10:00:26.118340+0000","last_unstale":"2024-04-25T10:00:27.636416+0000","last_undegraded":"2024-04-25T10:00:27.636416+0000","last_fullsized":"2024-04-25T10:00:27.636416+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:18.152201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093661+0000","last_change":"2024-04-25T10:00:25.097773+0000","last_active":"2024-04-25T10:00:27.093661+0000","last_peered":"2024-04-25T10:00:27.093661+0000","last_clean":"2024-04-25T10:00:27.093661+0000","last_became_active":"2024-04-25T10:00:25.097130+0000","last_became_peered":"2024-04-25T10:00:25.097130+0000","last_unstale":"2024-04-25T10:00:27.093661+0000","last_undegraded":"2024-04-25T10:00:27.093661+0000","last_fullsized":"2024-04-25T10:00:27.093661+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:31.530314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636933+0000","last_change":"2024-04-25T10:00:26.106308+0000","last_active":"2024-04-25T10:00:27.636933+0000","last_peered":"2024-04-25T10:00:27.636933+0000","last_clean":"2024-04-25T10:00:27.636933+0000","last_became_active":"2024-04-25T10:00:26.105673+0000","last_became_peered":"2024-04-25T10:00:26.105673+0000","last_unstale":"2024-04-25T10:00:27.636933+0000","last_undegraded":"2024-04-25T10:00:27.636933+0000","last_fullsized":"2024-04-25T10:00:27.636933+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:22.491416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092038+0000","last_change":"2024-04-25T10:00:25.097116+0000","last_active":"2024-04-25T10:00:27.092038+0000","last_peered":"2024-04-25T10:00:27.092038+0000","last_clean":"2024-04-25T10:00:27.092038+0000","last_became_active":"2024-04-25T10:00:25.096795+0000","last_became_peered":"2024-04-25T10:00:25.096795+0000","last_unstale":"2024-04-25T10:00:27.092038+0000","last_undegraded":"2024-04-25T10:00:27.092038+0000","last_fullsized":"2024-04-25T10:00:27.092038+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:35.733723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636579+0000","last_change":"2024-04-25T10:00:26.103029+0000","last_active":"2024-04-25T10:00:27.636579+0000","last_peered":"2024-04-25T10:00:27.636579+0000","last_clean":"2024-04-25T10:00:27.636579+0000","last_became_active":"2024-04-25T10:00:26.102264+0000","last_became_peered":"2024-04-25T10:00:26.102264+0000","last_unstale":"2024-04-25T10:00:27.636579+0000","last_undegraded":"2024-04-25T10:00:27.636579+0000","last_fullsized":"2024-04-25T10:00:27.636579+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:12.214321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636948+0000","last_change":"2024-04-25T10:00:25.096994+0000","last_active":"2024-04-25T10:00:27.636948+0000","last_peered":"2024-04-25T10:00:27.636948+0000","last_clean":"2024-04-25T10:00:27.636948+0000","last_became_active":"2024-04-25T10:00:25.096686+0000","last_became_peered":"2024-04-25T10:00:25.096686+0000","last_unstale":"2024-04-25T10:00:27.636948+0000","last_undegraded":"2024-04-25T10:00:27.636948+0000","last_fullsized":"2024-04-25T10:00:27.636948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:22.589059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092838+0000","last_change":"2024-04-25T10:00:26.103127+0000","last_active":"2024-04-25T10:00:27.092838+0000","last_peered":"2024-04-25T10:00:27.092838+0000","last_clean":"2024-04-25T10:00:27.092838+0000","last_became_active":"2024-04-25T10:00:26.102811+0000","last_became_peered":"2024-04-25T10:00:26.102811+0000","last_unstale":"2024-04-25T10:00:27.092838+0000","last_undegraded":"2024-04-25T10:00:27.092838+0000","last_fullsized":"2024-04-25T10:00:27.092838+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:22.491416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091760+0000","last_change":"2024-04-25T10:00:25.092552+0000","last_active":"2024-04-25T10:00:27.091760+0000","last_peered":"2024-04-25T10:00:27.091760+0000","last_clean":"2024-04-25T10:00:27.091760+0000","last_became_active":"2024-04-25T10:00:25.092297+0000","last_became_peered":"2024-04-25T10:00:25.092297+0000","last_unstale":"2024-04-25T10:00:27.091760+0000","last_undegraded":"2024-04-25T10:00:27.091760+0000","last_fullsized":"2024-04-25T10:00:27.091760+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:21.441609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636079+0000","last_change":"2024-04-25T10:00:26.119651+0000","last_active":"2024-04-25T10:00:27.636079+0000","last_peered":"2024-04-25T10:00:27.636079+0000","last_clean":"2024-04-25T10:00:27.636079+0000","last_became_active":"2024-04-25T10:00:26.119380+0000","last_became_peered":"2024-04-25T10:00:26.119380+0000","last_unstale":"2024-04-25T10:00:27.636079+0000","last_undegraded":"2024-04-25T10:00:27.636079+0000","last_fullsized":"2024-04-25T10:00:27.636079+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:07.047721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637458+0000","last_change":"2024-04-25T10:00:25.094479+0000","last_active":"2024-04-25T10:00:27.637458+0000","last_peered":"2024-04-25T10:00:27.637458+0000","last_clean":"2024-04-25T10:00:27.637458+0000","last_became_active":"2024-04-25T10:00:25.094171+0000","last_became_peered":"2024-04-25T10:00:25.094171+0000","last_unstale":"2024-04-25T10:00:27.637458+0000","last_undegraded":"2024-04-25T10:00:27.637458+0000","last_fullsized":"2024-04-25T10:00:27.637458+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:25.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637073+0000","last_change":"2024-04-25T10:00:26.104418+0000","last_active":"2024-04-25T10:00:27.637073+0000","last_peered":"2024-04-25T10:00:27.637073+0000","last_clean":"2024-04-25T10:00:27.637073+0000","last_became_active":"2024-04-25T10:00:26.104117+0000","last_became_peered":"2024-04-25T10:00:26.104117+0000","last_unstale":"2024-04-25T10:00:27.637073+0000","last_undegraded":"2024-04-25T10:00:27.637073+0000","last_fullsized":"2024-04-25T10:00:27.637073+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:16.374760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636621+0000","last_change":"2024-04-25T10:00:25.091554+0000","last_active":"2024-04-25T10:00:27.636621+0000","last_peered":"2024-04-25T10:00:27.636621+0000","last_clean":"2024-04-25T10:00:27.636621+0000","last_became_active":"2024-04-25T10:00:25.091292+0000","last_became_peered":"2024-04-25T10:00:25.091292+0000","last_unstale":"2024-04-25T10:00:27.636621+0000","last_undegraded":"2024-04-25T10:00:27.636621+0000","last_fullsized":"2024-04-25T10:00:27.636621+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:26.596265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092027+0000","last_change":"2024-04-25T10:00:26.108382+0000","last_active":"2024-04-25T10:00:27.092027+0000","last_peered":"2024-04-25T10:00:27.092027+0000","last_clean":"2024-04-25T10:00:27.092027+0000","last_became_active":"2024-04-25T10:00:26.105965+0000","last_became_peered":"2024-04-25T10:00:26.105965+0000","last_unstale":"2024-04-25T10:00:27.092027+0000","last_undegraded":"2024-04-25T10:00:27.092027+0000","last_fullsized":"2024-04-25T10:00:27.092027+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:06.469833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637408+0000","last_change":"2024-04-25T10:00:25.096404+0000","last_active":"2024-04-25T10:00:27.637408+0000","last_peered":"2024-04-25T10:00:27.637408+0000","last_clean":"2024-04-25T10:00:27.637408+0000","last_became_active":"2024-04-25T10:00:25.096120+0000","last_became_peered":"2024-04-25T10:00:25.096120+0000","last_unstale":"2024-04-25T10:00:27.637408+0000","last_undegraded":"2024-04-25T10:00:27.637408+0000","last_fullsized":"2024-04-25T10:00:27.637408+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.281637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637096+0000","last_change":"2024-04-25T10:00:26.105547+0000","last_active":"2024-04-25T10:00:27.637096+0000","last_peered":"2024-04-25T10:00:27.637096+0000","last_clean":"2024-04-25T10:00:27.637096+0000","last_became_active":"2024-04-25T10:00:26.104666+0000","last_became_peered":"2024-04-25T10:00:26.104666+0000","last_unstale":"2024-04-25T10:00:27.637096+0000","last_undegraded":"2024-04-25T10:00:27.637096+0000","last_fullsized":"2024-04-25T10:00:27.637096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:22.491416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636684+0000","last_change":"2024-04-25T10:00:25.091793+0000","last_active":"2024-04-25T10:00:27.636684+0000","last_peered":"2024-04-25T10:00:27.636684+0000","last_clean":"2024-04-25T10:00:27.636684+0000","last_became_active":"2024-04-25T10:00:25.091513+0000","last_became_peered":"2024-04-25T10:00:25.091513+0000","last_unstale":"2024-04-25T10:00:27.636684+0000","last_undegraded":"2024-04-25T10:00:27.636684+0000","last_fullsized":"2024-04-25T10:00:27.636684+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:31.429546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093530+0000","last_change":"2024-04-25T10:00:26.103770+0000","last_active":"2024-04-25T10:00:27.093530+0000","last_peered":"2024-04-25T10:00:27.093530+0000","last_clean":"2024-04-25T10:00:27.093530+0000","last_became_active":"2024-04-25T10:00:26.103496+0000","last_became_peered":"2024-04-25T10:00:26.103496+0000","last_unstale":"2024-04-25T10:00:27.093530+0000","last_undegraded":"2024-04-25T10:00:27.093530+0000","last_fullsized":"2024-04-25T10:00:27.093530+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:23.593287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093680+0000","last_change":"2024-04-25T10:00:25.093711+0000","last_active":"2024-04-25T10:00:27.093680+0000","last_peered":"2024-04-25T10:00:27.093680+0000","last_clean":"2024-04-25T10:00:27.093680+0000","last_became_active":"2024-04-25T10:00:25.093429+0000","last_became_peered":"2024-04-25T10:00:25.093429+0000","last_unstale":"2024-04-25T10:00:27.093680+0000","last_undegraded":"2024-04-25T10:00:27.093680+0000","last_fullsized":"2024-04-25T10:00:27.093680+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:17.897809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091743+0000","last_change":"2024-04-25T10:00:26.117042+0000","last_active":"2024-04-25T10:00:27.091743+0000","last_peered":"2024-04-25T10:00:27.091743+0000","last_clean":"2024-04-25T10:00:27.091743+0000","last_became_active":"2024-04-25T10:00:26.115309+0000","last_became_peered":"2024-04-25T10:00:26.115309+0000","last_unstale":"2024-04-25T10:00:27.091743+0000","last_undegraded":"2024-04-25T10:00:27.091743+0000","last_fullsized":"2024-04-25T10:00:27.091743+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:26.724704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636162+0000","last_change":"2024-04-25T10:00:25.093815+0000","last_active":"2024-04-25T10:00:27.636162+0000","last_peered":"2024-04-25T10:00:27.636162+0000","last_clean":"2024-04-25T10:00:27.636162+0000","last_became_active":"2024-04-25T10:00:25.093502+0000","last_became_peered":"2024-04-25T10:00:25.093502+0000","last_unstale":"2024-04-25T10:00:27.636162+0000","last_undegraded":"2024-04-25T10:00:27.636162+0000","last_fullsized":"2024-04-25T10:00:27.636162+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:56.667112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637314+0000","last_change":"2024-04-25T10:00:26.100569+0000","last_active":"2024-04-25T10:00:27.637314+0000","last_peered":"2024-04-25T10:00:27.637314+0000","last_clean":"2024-04-25T10:00:27.637314+0000","last_became_active":"2024-04-25T10:00:26.100257+0000","last_became_peered":"2024-04-25T10:00:26.100257+0000","last_unstale":"2024-04-25T10:00:27.637314+0000","last_undegraded":"2024-04-25T10:00:27.637314+0000","last_fullsized":"2024-04-25T10:00:27.637314+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:21.736808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636162+0000","last_change":"2024-04-25T10:00:25.093636+0000","last_active":"2024-04-25T10:00:27.636162+0000","last_peered":"2024-04-25T10:00:27.636162+0000","last_clean":"2024-04-25T10:00:27.636162+0000","last_became_active":"2024-04-25T10:00:25.093327+0000","last_became_peered":"2024-04-25T10:00:25.093327+0000","last_unstale":"2024-04-25T10:00:27.636162+0000","last_undegraded":"2024-04-25T10:00:27.636162+0000","last_fullsized":"2024-04-25T10:00:27.636162+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:31.561872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636347+0000","last_change":"2024-04-25T10:00:26.113811+0000","last_active":"2024-04-25T10:00:27.636347+0000","last_peered":"2024-04-25T10:00:27.636347+0000","last_clean":"2024-04-25T10:00:27.636347+0000","last_became_active":"2024-04-25T10:00:26.113457+0000","last_became_peered":"2024-04-25T10:00:26.113457+0000","last_unstale":"2024-04-25T10:00:27.636347+0000","last_undegraded":"2024-04-25T10:00:27.636347+0000","last_fullsized":"2024-04-25T10:00:27.636347+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:08.058617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636582+0000","last_change":"2024-04-25T10:00:25.094685+0000","last_active":"2024-04-25T10:00:27.636582+0000","last_peered":"2024-04-25T10:00:27.636582+0000","last_clean":"2024-04-25T10:00:27.636582+0000","last_became_active":"2024-04-25T10:00:25.094421+0000","last_became_peered":"2024-04-25T10:00:25.094421+0000","last_unstale":"2024-04-25T10:00:27.636582+0000","last_undegraded":"2024-04-25T10:00:27.636582+0000","last_fullsized":"2024-04-25T10:00:27.636582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:45.756171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091851+0000","last_change":"2024-04-25T10:00:26.107442+0000","last_active":"2024-04-25T10:00:27.091851+0000","last_peered":"2024-04-25T10:00:27.091851+0000","last_clean":"2024-04-25T10:00:27.091851+0000","last_became_active":"2024-04-25T10:00:26.107170+0000","last_became_peered":"2024-04-25T10:00:26.107170+0000","last_unstale":"2024-04-25T10:00:27.091851+0000","last_undegraded":"2024-04-25T10:00:27.091851+0000","last_fullsized":"2024-04-25T10:00:27.091851+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:21.915015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.635927+0000","last_change":"2024-04-25T10:00:25.087285+0000","last_active":"2024-04-25T10:00:27.635927+0000","last_peered":"2024-04-25T10:00:27.635927+0000","last_clean":"2024-04-25T10:00:27.635927+0000","last_became_active":"2024-04-25T10:00:25.087017+0000","last_became_peered":"2024-04-25T10:00:25.087017+0000","last_unstale":"2024-04-25T10:00:27.635927+0000","last_undegraded":"2024-04-25T10:00:27.635927+0000","last_fullsized":"2024-04-25T10:00:27.635927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.006391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637472+0000","last_change":"2024-04-25T10:00:26.101120+0000","last_active":"2024-04-25T10:00:27.637472+0000","last_peered":"2024-04-25T10:00:27.637472+0000","last_clean":"2024-04-25T10:00:27.637472+0000","last_became_active":"2024-04-25T10:00:26.100826+0000","last_became_peered":"2024-04-25T10:00:26.100826+0000","last_unstale":"2024-04-25T10:00:27.637472+0000","last_undegraded":"2024-04-25T10:00:27.637472+0000","last_fullsized":"2024-04-25T10:00:27.637472+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:08.605014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637094+0000","last_change":"2024-04-25T10:00:25.095926+0000","last_active":"2024-04-25T10:00:27.637094+0000","last_peered":"2024-04-25T10:00:27.637094+0000","last_clean":"2024-04-25T10:00:27.637094+0000","last_became_active":"2024-04-25T10:00:25.095588+0000","last_became_peered":"2024-04-25T10:00:25.095588+0000","last_unstale":"2024-04-25T10:00:27.637094+0000","last_undegraded":"2024-04-25T10:00:27.637094+0000","last_fullsized":"2024-04-25T10:00:27.637094+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:51.216973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092222+0000","last_change":"2024-04-25T10:00:26.106847+0000","last_active":"2024-04-25T10:00:27.092222+0000","last_peered":"2024-04-25T10:00:27.092222+0000","last_clean":"2024-04-25T10:00:27.092222+0000","last_became_active":"2024-04-25T10:00:26.106584+0000","last_became_peered":"2024-04-25T10:00:26.106584+0000","last_unstale":"2024-04-25T10:00:27.092222+0000","last_undegraded":"2024-04-25T10:00:27.092222+0000","last_fullsized":"2024-04-25T10:00:27.092222+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.830410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093703+0000","last_change":"2024-04-25T10:00:25.093139+0000","last_active":"2024-04-25T10:00:27.093703+0000","last_peered":"2024-04-25T10:00:27.093703+0000","last_clean":"2024-04-25T10:00:27.093703+0000","last_became_active":"2024-04-25T10:00:25.092500+0000","last_became_peered":"2024-04-25T10:00:25.092500+0000","last_unstale":"2024-04-25T10:00:27.093703+0000","last_undegraded":"2024-04-25T10:00:27.093703+0000","last_fullsized":"2024-04-25T10:00:27.093703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:06.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.092028+0000","last_change":"2024-04-25T10:00:26.106942+0000","last_active":"2024-04-25T10:00:27.092028+0000","last_peered":"2024-04-25T10:00:27.092028+0000","last_clean":"2024-04-25T10:00:27.092028+0000","last_became_active":"2024-04-25T10:00:26.105394+0000","last_became_peered":"2024-04-25T10:00:26.105394+0000","last_unstale":"2024-04-25T10:00:27.092028+0000","last_undegraded":"2024-04-25T10:00:27.092028+0000","last_fullsized":"2024-04-25T10:00:27.092028+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:14.994000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.093744+0000","last_change":"2024-04-25T10:00:25.097566+0000","last_active":"2024-04-25T10:00:27.093744+0000","last_peered":"2024-04-25T10:00:27.093744+0000","last_clean":"2024-04-25T10:00:27.093744+0000","last_became_active":"2024-04-25T10:00:25.096800+0000","last_became_peered":"2024-04-25T10:00:25.096800+0000","last_unstale":"2024-04-25T10:00:27.093744+0000","last_undegraded":"2024-04-25T10:00:27.093744+0000","last_fullsized":"2024-04-25T10:00:27.093744+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:34.162551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637058+0000","last_change":"2024-04-25T10:00:26.108694+0000","last_active":"2024-04-25T10:00:27.637058+0000","last_peered":"2024-04-25T10:00:27.637058+0000","last_clean":"2024-04-25T10:00:27.637058+0000","last_became_active":"2024-04-25T10:00:26.108439+0000","last_became_peered":"2024-04-25T10:00:26.108439+0000","last_unstale":"2024-04-25T10:00:27.637058+0000","last_undegraded":"2024-04-25T10:00:27.637058+0000","last_fullsized":"2024-04-25T10:00:27.637058+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:28.447001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637587+0000","last_change":"2024-04-25T10:00:25.096895+0000","last_active":"2024-04-25T10:00:27.637587+0000","last_peered":"2024-04-25T10:00:27.637587+0000","last_clean":"2024-04-25T10:00:27.637587+0000","last_became_active":"2024-04-25T10:00:25.096545+0000","last_became_peered":"2024-04-25T10:00:25.096545+0000","last_unstale":"2024-04-25T10:00:27.637587+0000","last_undegraded":"2024-04-25T10:00:27.637587+0000","last_fullsized":"2024-04-25T10:00:27.637587+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:39.070111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.635959+0000","last_change":"2024-04-25T10:00:26.101900+0000","last_active":"2024-04-25T10:00:27.635959+0000","last_peered":"2024-04-25T10:00:27.635959+0000","last_clean":"2024-04-25T10:00:27.635959+0000","last_became_active":"2024-04-25T10:00:26.101628+0000","last_became_peered":"2024-04-25T10:00:26.101628+0000","last_unstale":"2024-04-25T10:00:27.635959+0000","last_undegraded":"2024-04-25T10:00:27.635959+0000","last_fullsized":"2024-04-25T10:00:27.635959+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:44.868674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091981+0000","last_change":"2024-04-25T10:00:25.096062+0000","last_active":"2024-04-25T10:00:27.091981+0000","last_peered":"2024-04-25T10:00:27.091981+0000","last_clean":"2024-04-25T10:00:27.091981+0000","last_became_active":"2024-04-25T10:00:25.095754+0000","last_became_peered":"2024-04-25T10:00:25.095754+0000","last_unstale":"2024-04-25T10:00:27.091981+0000","last_undegraded":"2024-04-25T10:00:27.091981+0000","last_fullsized":"2024-04-25T10:00:27.091981+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:23.642194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636406+0000","last_change":"2024-04-25T10:00:26.116204+0000","last_active":"2024-04-25T10:00:27.636406+0000","last_peered":"2024-04-25T10:00:27.636406+0000","last_clean":"2024-04-25T10:00:27.636406+0000","last_became_active":"2024-04-25T10:00:26.115878+0000","last_became_peered":"2024-04-25T10:00:26.115878+0000","last_unstale":"2024-04-25T10:00:27.636406+0000","last_undegraded":"2024-04-25T10:00:27.636406+0000","last_fullsized":"2024-04-25T10:00:27.636406+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:01.444519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636282+0000","last_change":"2024-04-25T10:00:25.089349+0000","last_active":"2024-04-25T10:00:27.636282+0000","last_peered":"2024-04-25T10:00:27.636282+0000","last_clean":"2024-04-25T10:00:27.636282+0000","last_became_active":"2024-04-25T10:00:25.089074+0000","last_became_peered":"2024-04-25T10:00:25.089074+0000","last_unstale":"2024-04-25T10:00:27.636282+0000","last_undegraded":"2024-04-25T10:00:27.636282+0000","last_fullsized":"2024-04-25T10:00:27.636282+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:31.429546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637633+0000","last_change":"2024-04-25T10:00:26.122197+0000","last_active":"2024-04-25T10:00:27.637633+0000","last_peered":"2024-04-25T10:00:27.637633+0000","last_clean":"2024-04-25T10:00:27.637633+0000","last_became_active":"2024-04-25T10:00:26.121897+0000","last_became_peered":"2024-04-25T10:00:26.121897+0000","last_unstale":"2024-04-25T10:00:27.637633+0000","last_undegraded":"2024-04-25T10:00:27.637633+0000","last_fullsized":"2024-04-25T10:00:27.637633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:46.760399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637692+0000","last_change":"2024-04-25T10:00:25.097664+0000","last_active":"2024-04-25T10:00:27.637692+0000","last_peered":"2024-04-25T10:00:27.637692+0000","last_clean":"2024-04-25T10:00:27.637692+0000","last_became_active":"2024-04-25T10:00:25.097390+0000","last_became_peered":"2024-04-25T10:00:25.097390+0000","last_unstale":"2024-04-25T10:00:27.637692+0000","last_undegraded":"2024-04-25T10:00:27.637692+0000","last_fullsized":"2024-04-25T10:00:27.637692+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:47.445114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.091891+0000","last_change":"2024-04-25T10:00:26.111508+0000","last_active":"2024-04-25T10:00:27.091891+0000","last_peered":"2024-04-25T10:00:27.091891+0000","last_clean":"2024-04-25T10:00:27.091891+0000","last_became_active":"2024-04-25T10:00:26.111272+0000","last_became_peered":"2024-04-25T10:00:26.111272+0000","last_unstale":"2024-04-25T10:00:27.091891+0000","last_undegraded":"2024-04-25T10:00:27.091891+0000","last_fullsized":"2024-04-25T10:00:27.091891+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:03.096027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636568+0000","last_change":"2024-04-25T10:00:25.092559+0000","last_active":"2024-04-25T10:00:27.636568+0000","last_peered":"2024-04-25T10:00:27.636568+0000","last_clean":"2024-04-25T10:00:27.636568+0000","last_became_active":"2024-04-25T10:00:25.092275+0000","last_became_peered":"2024-04-25T10:00:25.092275+0000","last_unstale":"2024-04-25T10:00:27.636568+0000","last_undegraded":"2024-04-25T10:00:27.636568+0000","last_fullsized":"2024-04-25T10:00:27.636568+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:56.667112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637044+0000","last_change":"2024-04-25T10:00:26.099744+0000","last_active":"2024-04-25T10:00:27.637044+0000","last_peered":"2024-04-25T10:00:27.637044+0000","last_clean":"2024-04-25T10:00:27.637044+0000","last_became_active":"2024-04-25T10:00:26.099316+0000","last_became_peered":"2024-04-25T10:00:26.099316+0000","last_unstale":"2024-04-25T10:00:27.637044+0000","last_undegraded":"2024-04-25T10:00:27.637044+0000","last_fullsized":"2024-04-25T10:00:27.637044+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:25.073196+0000","last_clean_scrub_stamp":"2024-04-25T10:00:25.073196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:05.006276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.637344+0000","last_change":"2024-04-25T10:00:25.090514+0000","last_active":"2024-04-25T10:00:27.637344+0000","last_peered":"2024-04-25T10:00:27.637344+0000","last_clean":"2024-04-25T10:00:27.637344+0000","last_became_active":"2024-04-25T10:00:25.090248+0000","last_became_peered":"2024-04-25T10:00:25.090248+0000","last_unstale":"2024-04-25T10:00:27.637344+0000","last_undegraded":"2024-04-25T10:00:27.637344+0000","last_fullsized":"2024-04-25T10:00:27.637344+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:00:24.068967+0000","last_clean_scrub_stamp":"2024-04-25T10:00:24.068967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:06.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T10:00:27.636706+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:00:27.636706+0000","last_peered":"2024-04-25T10:00:27.636706+0000","last_clean":"2024-04-25T10:00:27.636706+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:00:27.636706+0000","last_undegraded":"2024-04-25T10:00:27.636706+0000","last_fullsized":"2024-04-25T10:00:27.636706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41256,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330584,"statfs":{"total":96636764160,"available":96594518016,"internally_reserved":0,"allocated":1798144,"data_stored":1145641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1216512,"data_stored":557433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40668,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331172,"statfs":{"total":96636764160,"available":96595120128,"internally_reserved":0,"allocated":1196032,"data_stored":555217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46252,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325588,"statfs":{"total":96636764160,"available":96589402112,"internally_reserved":0,"allocated":1204224,"data_stored":555377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50396,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321444,"statfs":{"total":96636764160,"available":96585158656,"internally_reserved":0,"allocated":1196032,"data_stored":555217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46240,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325600,"statfs":{"total":96636764160,"available":96589414400,"internally_reserved":0,"allocated":1196032,"data_stored":555217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62880,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308960,"statfs":{"total":96636764160,"available":96572375040,"internally_reserved":0,"allocated":1200128,"data_stored":555287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46844,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324996,"statfs":{"total":96636764160,"available":96588795904,"internally_reserved":0,"allocated":1814528,"data_stored":1147903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:00:33.744 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:00:33.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:00:33.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.879+0000 7fc0b1c9c640 1 Processor -- start 2024-04-25T10:00:33.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.879+0000 7fc0b1c9c640 1 -- start start 2024-04-25T10:00:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0b1c9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0ac155f80 0x7fc0ac06f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0b1c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0ce900 0x7fc0ac0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0b1c9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac0b4d80 con 0x7fc0ac06fa00 2024-04-25T10:00:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0b1c9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac0b5390 con 0x7fc0ac155f80 2024-04-25T10:00:33.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0b1c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0d07f0 con 0x7fc0ac0ce900 2024-04-25T10:00:33.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0ab7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0ac155f80 0x7fc0ac06f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0ab7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0ac155f80 0x7fc0ac06f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:51502/0 (socket says 172.21.15.148:51502) 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0a3fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc0ac06fa00 0x7fc0ac0ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:33234/0 (socket says 172.21.15.148:33234) 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0a3fff640 1 -- 172.21.15.148:0/1119829150 learned_addr learned my addr 172.21.15.148:0/1119829150 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.880+0000 7fc0abfff640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0ce900 0x7fc0ac0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0ab7fe640 1 -- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0ce900 msgr2=0x7fc0ac0b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0ab7fe640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0ce900 0x7fc0ac0b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0ab7fe640 1 -- 172.21.15.148:0/1119829150 >> v1:172.21.15.188:6789/0 conn(0x7fc0ac06fa00 legacy=0x7fc0ac0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0ab7fe640 1 -- 172.21.15.148:0/1119829150 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0cef70 con 0x7fc0ac155f80 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0aaffd640 1 -- 172.21.15.148:0/1119829150 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219336251 0 0) 0x7fc0ac0b4d80 con 0x7fc0ac06fa00 2024-04-25T10:00:33.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0abfff640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0ce900 0x7fc0ac0b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:33.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.881+0000 7fc0ab7fe640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac155f80 0x7fc0ac06f4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0b5510 tx=0x7fc09c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9418c99f7e594a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0aaffd640 1 -- 172.21.15.148:0/1119829150 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c037070 con 0x7fc0ac155f80 2024-04-25T10:00:33.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0aaffd640 1 -- 172.21.15.148:0/1119829150 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c032440 con 0x7fc0ac155f80 2024-04-25T10:00:33.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0aaffd640 1 -- 172.21.15.148:0/1119829150 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c032810 con 0x7fc0ac155f80 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac155f80 msgr2=0x7fc0ac06f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac155f80 0x7fc0ac06f4c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0b5510 tx=0x7fc09c02a180 comp rx=0 tx=0).stop 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/1119829150 shutdown_connections 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0ce900 0x7fc0ac0b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/1119829150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac155f80 0x7fc0ac06f4c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/1119829150 >> 172.21.15.148:0/1119829150 conn(0x7fc0ac061600 msgr2=0x7fc0ac0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/1119829150 shutdown_connections 2024-04-25T10:00:33.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.882+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/1119829150 wait complete. 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.883+0000 7fc0b1c9c640 1 Processor -- start 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.883+0000 7fc0b1c9c640 1 -- start start 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0b1c9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac06fa00 0x7fc0ac1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0b1c9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 0x7fc0ac149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0b1c9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac155f80 0x7fc0ac1f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0b1c9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cf250 con 0x7fc0ac0ce900 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0b1c9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac156360 con 0x7fc0ac06fa00 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0b1c9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cf3d0 con 0x7fc0ac155f80 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 0x7fc0ac149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0a3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 0x7fc0ac149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34850/0 (socket says 172.21.15.148:34850) 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0a3fff640 1 -- 172.21.15.148:0/2433285756 learned_addr learned my addr 172.21.15.148:0/2433285756 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0abfff640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac155f80 0x7fc0ac1f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0ab7fe640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac06fa00 0x7fc0ac1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0a3fff640 1 -- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac155f80 msgr2=0x7fc0ac1f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.884+0000 7fc0a3fff640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac155f80 0x7fc0ac1f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a3fff640 1 -- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac06fa00 msgr2=0x7fc0ac1f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a3fff640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac06fa00 0x7fc0ac1f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a3fff640 1 -- 172.21.15.148:0/2433285756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac1f28c0 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a3fff640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 0x7fc0ac149bc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c460 tx=0x7fc09400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940074e0 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094007680 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac1f2ba0 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094004a50 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.885+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac1f5870 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.886+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac073a00 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.886+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc094011070 con 0x7fc0ac0ce900 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.887+0000 7fc0a8ff9640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc08c06b4d0 0x7fc08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:33.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.887+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc094089e20 con 0x7fc0ac0ce900 2024-04-25T10:00:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.887+0000 7fc0ab7fe640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc08c06b4d0 0x7fc08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:33.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.887+0000 7fc0ab7fe640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc08c06b4d0 0x7fc08c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc09c034000 tx=0x7fc09c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:33.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:33.891+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc094058c60 con 0x7fc0ac0ce900 2024-04-25T10:00:34.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.130+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc0ac1e9200 con 0x7fc0ac0ce900 2024-04-25T10:00:34.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.132+0000 7fc0a8ff9640 1 -- 172.21.15.148:0/2433285756 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc094058560 con 0x7fc0ac0ce900 2024-04-25T10:00:34.138 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:00:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:00:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:00:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:00:34.138 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:00:34.138 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:00:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc08c06b4d0 msgr2=0x7fc08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc08c06b4d0 0x7fc08c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc09c034000 tx=0x7fc09c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 msgr2=0x7fc0ac149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 0x7fc0ac149bc0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c460 tx=0x7fc09400c860 comp rx=0 tx=0).stop 2024-04-25T10:00:34.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 shutdown_connections 2024-04-25T10:00:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc08c06b4d0 0x7fc08c06d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac155f80 0x7fc0ac1f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0ac0ce900 0x7fc0ac149bc0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 --2- 172.21.15.148:0/2433285756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac06fa00 0x7fc0ac1f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 >> 172.21.15.148:0/2433285756 conn(0x7fc0ac061600 msgr2=0x7fc0ac155330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 shutdown_connections 2024-04-25T10:00:34.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.135+0000 7fc0b1c9c640 1 -- 172.21.15.148:0/2433285756 wait complete. 2024-04-25T10:00:34.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:34.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.242+0000 7f508d0f5640 1 Processor -- start 2024-04-25T10:00:34.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.242+0000 7f508d0f5640 1 -- start start 2024-04-25T10:00:34.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.242+0000 7f508d0f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50880c6df0 0x7f50880d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.242+0000 7f508d0f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508815dfe0 con 0x7f50880d1c00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.242+0000 7f508d0f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50880d00c0 con 0x7f508815dc00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.242+0000 7f508d0f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50880d2070 con 0x7f50880c6df0 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f508759e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50880d1c00 0x7f50880ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:59582/0 (socket says 172.21.15.148:59582) 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f508759e640 1 -- 172.21.15.148:0/9903371 learned_addr learned my addr 172.21.15.148:0/9903371 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f508659c640 1 --2- 172.21.15.148:0/9903371 >> v2:172.21.15.188:3300/0 conn(0x7f50880c6df0 0x7f50880d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407633394 0 0) 0x7f5080001490 con 0x7f50880d1c00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5070003520 con 0x7f50880d1c00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342962170 0 0) 0x7f5070003520 con 0x7f50880d1c00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 >> v2:172.21.15.188:3300/0 conn(0x7f50880c6df0 msgr2=0x7f50880d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 --2- 172.21.15.148:0/9903371 >> v2:172.21.15.188:3300/0 conn(0x7f50880c6df0 0x7f50880d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 >> v1:172.21.15.188:6789/0 conn(0x7f508815dc00 legacy=0x7f50880ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880d03a0 con 0x7f50880d1c00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5080001bc0 con 0x7f50880d1c00 2024-04-25T10:00:34.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f508659c640 1 --2- 172.21.15.148:0/9903371 >> v2:172.21.15.188:3300/0 conn(0x7f50880c6df0 0x7f50880d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 >> v1:172.21.15.148:6789/0 conn(0x7f50880d1c00 legacy=0x7f50880ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50700039b0 0x7f5070003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5070004800 0x7f5070004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f507000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880d2070 con 0x7f50700051c0 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507000c060 con 0x7f5070004800 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507000c260 con 0x7f50700039b0 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f508759e640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f507000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f508659c640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5070004800 0x7f5070004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.243+0000 7f5086d9d640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50700039b0 0x7f5070003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 -- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50700039b0 msgr2=0x7f5070003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50700039b0 0x7f5070003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 -- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5070004800 msgr2=0x7f5070004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5070004800 0x7f5070004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 -- 172.21.15.148:0/9903371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070003520 con 0x7f50700051c0 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 -- 172.21.15.148:0/9903371 reap_dead start 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f508759e640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f507000b9c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f508000aa50 tx=0x7f5070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac81a62e3590936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508000a670 con 0x7f50700051c0 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508002f070 con 0x7f50700051c0 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.244+0000 7f5085d9b640 1 -- 172.21.15.148:0/9903371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080034470 con 0x7f50700051c0 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 -- 172.21.15.148:0/9903371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 msgr2=0x7f507000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f507000b9c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f508000aa50 tx=0x7f5070005830 comp rx=0 tx=0).stop 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 -- 172.21.15.148:0/9903371 shutdown_connections 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 --2- 172.21.15.148:0/9903371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f507000b9c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 -- 172.21.15.148:0/9903371 >> 172.21.15.148:0/9903371 conn(0x7f508806f5d0 msgr2=0x7f508806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 -- 172.21.15.148:0/9903371 shutdown_connections 2024-04-25T10:00:34.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.245+0000 7f508d0f5640 1 -- 172.21.15.148:0/9903371 wait complete. 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.246+0000 7f508d0f5640 1 Processor -- start 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.246+0000 7f508d0f5640 1 -- start start 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508d0f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f50880b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508d0f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50880b4610 0x7f50881523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508d0f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50881f3f80 0x7f50881f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508d0f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880cc130 con 0x7f50700051c0 2024-04-25T10:00:34.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508d0f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880cc720 con 0x7f50880b4610 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508d0f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880cfd90 con 0x7f50881f3f80 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f5086d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f50880b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f5086d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f50880b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34870/0 (socket says 172.21.15.148:34870) 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f5086d9d640 1 -- 172.21.15.148:0/2747101306 learned_addr learned my addr 172.21.15.148:0/2747101306 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508659c640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50880b4610 0x7f50881523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.247+0000 7f508759e640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50881f3f80 0x7f50881f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f5086d9d640 1 -- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50881f3f80 msgr2=0x7f50881f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f5086d9d640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50881f3f80 0x7f50881f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f5086d9d640 1 -- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50880b4610 msgr2=0x7f50881523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f5086d9d640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50880b4610 0x7f50881523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f5086d9d640 1 -- 172.21.15.148:0/2747101306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880c8b40 con 0x7f50700051c0 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f508759e640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50881f3f80 0x7f50881f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f5086d9d640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f50880b5520 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f507800e480 tx=0x7f507800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780052e0 con 0x7f50700051c0 2024-04-25T10:00:34.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078018070 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50880c8e20 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078003830 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.248+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50880b7170 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.249+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5088073990 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.249+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50780039d0 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.250+0000 7f506f7fe640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507406b500 0x7f507406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.250+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5078089e80 con 0x7f50700051c0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.250+0000 7f508659c640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507406b500 0x7f507406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.250+0000 7f508659c640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507406b500 0x7f507406d9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f50880ce730 tx=0x7f507c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.254+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5078058c00 con 0x7f50700051c0 2024-04-25T10:00:34.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.460+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50880c66c0 con 0x7f50700051c0 2024-04-25T10:00:34.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.460+0000 7f506f7fe640 1 -- 172.21.15.148:0/2747101306 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f5078058500 con 0x7f50700051c0 2024-04-25T10:00:34.466 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:34.466 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:34.466 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507406b500 msgr2=0x7f507406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507406b500 0x7f507406d9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f50880ce730 tx=0x7f507c009190 comp rx=0 tx=0).stop 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 msgr2=0x7f50880b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f50880b5520 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f507800e480 tx=0x7f507800e940 comp rx=0 tx=0).stop 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 shutdown_connections 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507406b500 0x7f507406d9c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50881f3f80 0x7f50881f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50880b4610 0x7f50881523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 --2- 172.21.15.148:0/2747101306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50700051c0 0x7f50880b5520 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 >> 172.21.15.148:0/2747101306 conn(0x7f508806f5d0 msgr2=0x7f508806ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 shutdown_connections 2024-04-25T10:00:34.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.462+0000 7f508d0f5640 1 -- 172.21.15.148:0/2747101306 wait complete. 2024-04-25T10:00:34.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:00:25.424657+0000', 'modified': '2024-04-25T10:00:27.438215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7725': {'gid': 7725, 'name': 'd', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6834/1505603179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 1505603179}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 1505603179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:00:34.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:00:34.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:00:34.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 Processor -- start 2024-04-25T10:00:34.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 -- start start 2024-04-25T10:00:34.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58bc06bd20 0x7f58bc06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc15e020 con 0x7f58bc06e360 2024-04-25T10:00:34.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc06b890 con 0x7f58bc06bd20 2024-04-25T10:00:34.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.976+0000 7f58c20ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc06ac80 con 0x7f58bc0b38d0 2024-04-25T10:00:34.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58bc06bd20 0x7f58bc06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f58bc06bd20 0x7f58bc06c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34872/0 (socket says 172.21.15.148:34872) 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 -- 172.21.15.148:0/2812216874 learned_addr learned my addr 172.21.15.148:0/2812216874 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58baffd640 1 --2- 172.21.15.148:0/2812216874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 -- 172.21.15.148:0/2812216874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 msgr2=0x7f58bc0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 --2- 172.21.15.148:0/2812216874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 -- 172.21.15.148:0/2812216874 >> v1:172.21.15.188:6789/0 conn(0x7f58bc06e360 legacy=0x7f58bc06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 -- 172.21.15.148:0/2812216874 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc06af60 con 0x7f58bc06bd20 2024-04-25T10:00:34.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.977+0000 7f58bbfff640 1 --2- 172.21.15.148:0/2812216874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc06c1b0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009e50 tx=0x7f58b402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e3e7bff178eabe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58ba7fc640 1 -- 172.21.15.148:0/2812216874 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4036070 con 0x7f58bc06bd20 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58ba7fc640 1 -- 172.21.15.148:0/2812216874 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4031600 con 0x7f58bc06bd20 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58ba7fc640 1 -- 172.21.15.148:0/2812216874 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40319f0 con 0x7f58bc06bd20 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 -- 172.21.15.148:0/2812216874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 msgr2=0x7f58bc06c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 --2- 172.21.15.148:0/2812216874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc06c1b0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009e50 tx=0x7f58b402a7c0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 -- 172.21.15.148:0/2812216874 shutdown_connections 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 --2- 172.21.15.148:0/2812216874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc06c1b0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 --2- 172.21.15.148:0/2812216874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 -- 172.21.15.148:0/2812216874 >> 172.21.15.148:0/2812216874 conn(0x7f58bc0cfec0 msgr2=0x7f58bc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 -- 172.21.15.148:0/2812216874 shutdown_connections 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.978+0000 7f58c20ac640 1 -- 172.21.15.148:0/2812216874 wait complete. 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.979+0000 7f58c20ac640 1 Processor -- start 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 -- start start 2024-04-25T10:00:34.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc1f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc06e360 0x7f58bc0ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc1f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc06ee50 con 0x7f58bc06bd20 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc06b520 con 0x7f58bc06e360 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58c20ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0b2fd0 con 0x7f58bc0b38d0 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc1f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc1f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34886/0 (socket says 172.21.15.148:34886) 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.980+0000 7f58bb7fe640 1 -- 172.21.15.148:0/3543065665 learned_addr learned my addr 172.21.15.148:0/3543065665 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bbfff640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc1f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58baffd640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc06e360 0x7f58bc0ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bb7fe640 1 -- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 msgr2=0x7f58bc1f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc1f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bb7fe640 1 -- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc06e360 msgr2=0x7f58bc0ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc06e360 0x7f58bc0ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bb7fe640 1 -- 172.21.15.148:0/3543065665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc156f60 con 0x7f58bc06bd20 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58baffd640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc06e360 0x7f58bc0ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc1f6cf0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c400 tx=0x7f58b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58a3fff640 1 -- 172.21.15.148:0/3543065665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00052d0 con 0x7f58bc06bd20 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58a3fff640 1 -- 172.21.15.148:0/3543065665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0005490 con 0x7f58bc06bd20 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58a3fff640 1 -- 172.21.15.148:0/3543065665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0007260 con 0x7f58bc06bd20 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc157240 con 0x7f58bc06bd20 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.981+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc157800 con 0x7f58bc06bd20 2024-04-25T10:00:34.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.983+0000 7f58a3fff640 1 -- 172.21.15.148:0/3543065665 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58b0007400 con 0x7f58bc06bd20 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.983+0000 7f58a3fff640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a806b4d0 0x7f58a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.983+0000 7f58a3fff640 1 -- 172.21.15.148:0/3543065665 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58b00898e0 con 0x7f58bc06bd20 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.983+0000 7f58baffd640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a806b4d0 0x7f58a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.983+0000 7f58baffd640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a806b4d0 0x7f58a806d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f58ac006fd0 tx=0x7f58ac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.984+0000 7f58c4daa740 1 Processor -- start 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.984+0000 7f58c4daa740 1 -- start start 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b28020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4adff10 0x55c4f4b26190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58c4daa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4a42a10 con 0x55c4f4b27ad0 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f49ebb30 con 0x55c4f4b0a230 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4a3ec50 con 0x55c4f4adff10 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b28020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b28020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34900/0 (socket says 172.21.15.148:34900) 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 -- 172.21.15.148:0/3827204355 learned_addr learned my addr 172.21.15.148:0/3827204355 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 -- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4adff10 msgr2=0x55c4f4b26190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58bbfff640 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4adff10 0x55c4f4b26190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4adff10 0x55c4f4b26190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 -- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 msgr2=0x55c4f4b0a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b0a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 -- 172.21.15.148:0/3827204355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f4a00e10 con 0x55c4f4b27ad0 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58baffd640 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b28020 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0041f0 tx=0x7f58ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c212e361c3d9dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58a17fa640 1 -- 172.21.15.148:0/3827204355 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00f040 con 0x55c4f4b27ad0 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58a17fa640 1 -- 172.21.15.148:0/3827204355 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00ee40 con 0x55c4f4b27ad0 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.985+0000 7f58a17fa640 1 -- 172.21.15.148:0/3827204355 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac003810 con 0x55c4f4b27ad0 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 -- 172.21.15.148:0/3827204355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 msgr2=0x55c4f4b28020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b28020 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0041f0 tx=0x7f58ac011000 comp rx=0 tx=0).stop 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 -- 172.21.15.148:0/3827204355 shutdown_connections 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4adff10 0x55c4f4b26190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b28020 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3827204355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 -- 172.21.15.148:0/3827204355 >> 172.21.15.148:0/3827204355 conn(0x55c4f4adc840 msgr2=0x55c4f4b25040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 -- 172.21.15.148:0/3827204355 shutdown_connections 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.986+0000 7f58c4daa740 1 -- 172.21.15.148:0/3827204355 wait complete. 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 -- start start 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 Processor -- start 2024-04-25T10:00:34.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b1c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b1dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b1cab0 0x55c4f4b1e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4ae0cb0 con 0x55c4f4b27ad0 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f49ff0c0 con 0x55c4f4b0a230 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4ada770 con 0x55c4f4b1cab0 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.987+0000 7f58c4daa740 10 client.0 _ll_register_callbacks cb 0x55c4f49f2ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b1dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b1dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34906/0 (socket says 172.21.15.148:34906) 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 -- 172.21.15.148:0/3744326813 learned_addr learned my addr 172.21.15.148:0/3744326813 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58bbfff640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b1cab0 0x55c4f4b1e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b1c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 -- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b1cab0 msgr2=0x55c4f4b1e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b1cab0 0x55c4f4b1e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 -- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 msgr2=0x55c4f4b1c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b1c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 -- 172.21.15.148:0/3744326813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4f47572d0 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58baffd640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b1dec0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f58ac008010 tx=0x7f58ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab9b4b5c127bbad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f589a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f589a7fc640 1 -- 172.21.15.148:0/3744326813 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac005e40 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f589a7fc640 1 -- 172.21.15.148:0/3744326813 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58ac02bc40 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58ac00ee40 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f589a7fc640 1 -- 172.21.15.148:0/3744326813 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ac02be40 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58c4daa740 10 client.7866 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55c4f48216b0 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.988+0000 7f589a7fc640 1 -- 172.21.15.148:0/3744326813 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac00ee40 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f589a7fc640 1 -- 172.21.15.148:0/3744326813 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58ac012070 con 0x55c4f4b27ad0 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58c4daa740 10 client.7866 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58c4daa740 10 client.7866 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58c4daa740 20 client.7866 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b39160 0x55c4f4b39540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58c4daa740 4 client.7866 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4f4aa1920 con 0x55c4f4b39160 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b39160 0x55c4f4b39540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f589a7fc640 10 client.7866 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.989+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b39160 0x55c4f4b39540 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5756ccc429e11e7d server_cookie=d05600853b239ba2 in_seq=0 out_seq=0 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f589a7fc640 1 -- 172.21.15.148:0/3744326813 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4f4aa1920 con 0x55c4f4b39160 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f589a7fc640 10 client.7866 handle_command_reply: tid=0 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 client.7866 shutdown 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 msgr2=0x55c4f4b1dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b1dec0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f58ac008010 tx=0x7f58ac004730 comp rx=0 tx=0).stop 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 shutdown_connections 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b39160 0x55c4f4b39540 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b1cab0 0x55c4f4b1e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b1dec0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 --2- 172.21.15.148:0/3744326813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b1c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f589a7fc640 0 client.7866 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:35.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 >> 172.21.15.148:0/3744326813 conn(0x55c4f4adff10 msgr2=0x55c4f4b285c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 shutdown_connections 2024-04-25T10:00:35.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 1 -- 172.21.15.148:0/3744326813 wait complete. 2024-04-25T10:00:35.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:34.990+0000 7f58c4daa740 20 client.7866 trim_cache size 0 max 16384 2024-04-25T10:00:35.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.006+0000 7f58c4daa740 1 Processor -- start 2024-04-25T10:00:35.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.006+0000 7f58c4daa740 1 -- start start 2024-04-25T10:00:35.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b3f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b09270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b27ad0 0x55c4f4b097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58c4daa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1f6540 con 0x55c4f4b1cab0 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1f4ca0 con 0x55c4f4b0a230 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1f5f40 con 0x55c4f4b27ad0 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b09270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b09270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34912/0 (socket says 172.21.15.148:34912) 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 -- 172.21.15.148:0/2489171025 learned_addr learned my addr 172.21.15.148:0/2489171025 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 -- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b27ad0 msgr2=0x55c4f4b097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b27ad0 0x55c4f4b097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 -- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 msgr2=0x55c4f4b3f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b3f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b3f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 -- 172.21.15.148:0/2489171025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f4b0c430 con 0x55c4f4b1cab0 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b3f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58baffd640 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b09270 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f58ac004170 tx=0x7f58ac024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9400cf4c325274f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58a0ff9640 1 -- 172.21.15.148:0/2489171025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac038070 con 0x55c4f4b1cab0 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58a0ff9640 1 -- 172.21.15.148:0/2489171025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac003820 con 0x55c4f4b1cab0 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.007+0000 7f58a0ff9640 1 -- 172.21.15.148:0/2489171025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac005ca0 con 0x55c4f4b1cab0 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 -- 172.21.15.148:0/2489171025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 msgr2=0x55c4f4b09270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b09270 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f58ac004170 tx=0x7f58ac024040 comp rx=0 tx=0).stop 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 -- 172.21.15.148:0/2489171025 shutdown_connections 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b27ad0 0x55c4f4b097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b09270 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2489171025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b0a230 0x55c4f4b3f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 -- 172.21.15.148:0/2489171025 >> 172.21.15.148:0/2489171025 conn(0x55c4f4adff10 msgr2=0x55c4f4ae1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 -- 172.21.15.148:0/2489171025 shutdown_connections 2024-04-25T10:00:35.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 -- 172.21.15.148:0/2489171025 wait complete. 2024-04-25T10:00:35.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.008+0000 7f58c4daa740 1 -- start start 2024-04-25T10:00:35.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 Processor -- start 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b0a230 0x55c4f4b09270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b25f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4b35ce0 con 0x55c4f4b1cab0 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4b0b490 con 0x55c4f4b27ad0 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4b0c160 con 0x55c4f4b0a230 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58c4daa740 10 client.0 _ll_register_callbacks cb 0x55c4f4adac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58baffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b097b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34924/0 (socket says 172.21.15.148:34924) 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58baffd640 1 -- 172.21.15.148:0/2692903596 learned_addr learned my addr 172.21.15.148:0/2692903596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58bbfff640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b25f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.009+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b0a230 0x55c4f4b09270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58baffd640 1 -- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b0a230 msgr2=0x55c4f4b09270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58baffd640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b0a230 0x55c4f4b09270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58baffd640 1 -- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b27ad0 msgr2=0x55c4f4b25f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58baffd640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b25f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58baffd640 1 -- 172.21.15.148:0/2692903596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4f4b1bbb0 con 0x55c4f4b1cab0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58bbfff640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b25f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58baffd640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b097b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e790 tx=0x7f58ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1720e1aa82fbd429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f589a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f589a7fc640 1 -- 172.21.15.148:0/2692903596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac040020 con 0x55c4f4b1cab0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f589a7fc640 1 -- 172.21.15.148:0/2692903596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58ac006390 con 0x55c4f4b1cab0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58ac003820 con 0x55c4f4b1cab0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f589a7fc640 1 -- 172.21.15.148:0/2692903596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ac04a050 con 0x55c4f4b1cab0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 10 client.7872 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55c4f4b22830 con 0x55c4f4b1cab0 2024-04-25T10:00:35.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f589a7fc640 1 -- 172.21.15.148:0/2692903596 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac003820 con 0x55c4f4b1cab0 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f589a7fc640 1 -- 172.21.15.148:0/2692903596 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58ac04d020 con 0x55c4f4b1cab0 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 10 client.7872 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 10 client.7872 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 20 client.7872 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b26a80 0x55c4f4b8fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 4 client.7872 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.010+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c4f4a4ff50 con 0x55c4f4b26a80 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b26a80 0x55c4f4b8fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58bb7fe640 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b26a80 0x55c4f4b8fb10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0cdfdfe6490949 server_cookie=2154a1ab7131e1ea in_seq=0 out_seq=0 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f589a7fc640 10 client.7872 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f589a7fc640 1 -- 172.21.15.148:0/2692903596 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c4f4a4ff50 con 0x55c4f4b26a80 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f589a7fc640 10 client.7872 handle_command_reply: tid=0 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 client.7872 shutdown 2024-04-25T10:00:35.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 msgr2=0x55c4f4b097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b097b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00e790 tx=0x7f58ac014000 comp rx=0 tx=0).stop 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 shutdown_connections 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c4f4b26a80 0x55c4f4b8fb10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4f4b27ad0 0x55c4f4b25f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4f4b1cab0 0x55c4f4b097b0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 --2- 172.21.15.148:0/2692903596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c4f4b0a230 0x55c4f4b09270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 >> 172.21.15.148:0/2692903596 conn(0x55c4f4adff10 msgr2=0x55c4f4ae1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f589a7fc640 0 client.7872 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 shutdown_connections 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 1 -- 172.21.15.148:0/2692903596 wait complete. 2024-04-25T10:00:35.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.011+0000 7f58c4daa740 20 client.7872 trim_cache size 0 max 16384 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 7725, 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 8.9238732029999994, 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 53, 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 97, 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 13.999095624000001 2024-04-25T10:00:35.024 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:35.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a806b4d0 msgr2=0x7f58a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a806b4d0 0x7f58a806d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f58ac006fd0 tx=0x7f58ac008040 comp rx=0 tx=0).stop 2024-04-25T10:00:35.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 msgr2=0x7f58bc1f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc1f6cf0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c400 tx=0x7f58b000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 shutdown_connections 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58a806b4d0 0x7f58a806d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58bc0b38d0 0x7f58bc1f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58bc06e360 0x7f58bc0ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 --2- 172.21.15.148:0/3543065665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58bc06bd20 0x7f58bc1f6cf0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 >> 172.21.15.148:0/3543065665 conn(0x7f58bc0cfec0 msgr2=0x7f58bc160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 shutdown_connections 2024-04-25T10:00:35.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.012+0000 7f58c20ac640 1 -- 172.21.15.148:0/3543065665 wait complete. 2024-04-25T10:00:35.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:00:35.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.121+0000 7fbb1fb6a640 1 Processor -- start 2024-04-25T10:00:35.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.121+0000 7fbb1fb6a640 1 -- start start 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1fb6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18071ac0 0x7fbb180ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1fb6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb1806c1d0 0x7fbb1806afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1fb6a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1815ec60 con 0x7fbb18154ed0 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1fb6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1815ef40 con 0x7fbb1806c1d0 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1fb6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806db90 con 0x7fbb18071ac0 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1d105640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18071ac0 0x7fbb180ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1d105640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18071ac0 0x7fbb180ad5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:42276/0 (socket says 172.21.15.148:42276) 2024-04-25T10:00:35.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1e107640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb1806c1d0 0x7fbb1806afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.122+0000 7fbb1d105640 1 -- 172.21.15.148:0/4123303290 learned_addr learned my addr 172.21.15.148:0/4123303290 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:35.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.123+0000 7fbb1e107640 1 -- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18071ac0 msgr2=0x7fbb180ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:35.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.123+0000 7fbb1e107640 1 --2- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18071ac0 0x7fbb180ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.123+0000 7fbb1e107640 1 -- 172.21.15.148:0/4123303290 >> v1:172.21.15.188:6789/0 conn(0x7fbb18154ed0 legacy=0x7fbb180aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.123+0000 7fbb1e107640 1 -- 172.21.15.148:0/4123303290 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1806de70 con 0x7fbb1806c1d0 2024-04-25T10:00:35.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.123+0000 7fbb1c904640 1 -- 172.21.15.148:0/4123303290 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674456673 0 0) 0x7fbb1815ec60 con 0x7fbb18154ed0 2024-04-25T10:00:35.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.123+0000 7fbb1e107640 1 --2- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb1806c1d0 0x7fbb1806afa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009d90 tx=0x7fbb1002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53da31dee4c47c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1c904640 1 -- 172.21.15.148:0/4123303290 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10007740 con 0x7fbb1806c1d0 2024-04-25T10:00:35.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1c904640 1 -- 172.21.15.148:0/4123303290 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb100078e0 con 0x7fbb1806c1d0 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1c904640 1 -- 172.21.15.148:0/4123303290 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10007cb0 con 0x7fbb1806c1d0 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb1806c1d0 msgr2=0x7fbb1806afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb1806c1d0 0x7fbb1806afa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009d90 tx=0x7fbb1002a660 comp rx=0 tx=0).stop 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/4123303290 shutdown_connections 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb1806c1d0 0x7fbb1806afa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/4123303290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18071ac0 0x7fbb180ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/4123303290 >> 172.21.15.148:0/4123303290 conn(0x7fbb180af670 msgr2=0x7fbb180b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/4123303290 shutdown_connections 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.124+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/4123303290 wait complete. 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.125+0000 7fbb1fb6a640 1 Processor -- start 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 -- start start 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18071ac0 0x7fbb1813dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18154ed0 0x7fbb18150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 0x7fbb181f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806b660 con 0x7fbb181512b0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806b4e0 con 0x7fbb18071ac0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1fb6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806b7e0 con 0x7fbb18154ed0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1e107640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 0x7fbb181f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1e107640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 0x7fbb181f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34928/0 (socket says 172.21.15.148:34928) 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1e107640 1 -- 172.21.15.148:0/2941657502 learned_addr learned my addr 172.21.15.148:0/2941657502 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.126+0000 7fbb1d105640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18154ed0 0x7fbb18150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1d906640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18071ac0 0x7fbb1813dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1e107640 1 -- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18154ed0 msgr2=0x7fbb18150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1e107640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18154ed0 0x7fbb18150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1e107640 1 -- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18071ac0 msgr2=0x7fbb1813dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1e107640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18071ac0 0x7fbb1813dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1e107640 1 -- 172.21.15.148:0/2941657502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb181f7ba0 con 0x7fbb181512b0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1d906640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18071ac0 0x7fbb1813dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1e107640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 0x7fbb181f6e30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009d90 tx=0x7fbb10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10040070 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10031c40 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb18140560 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10009450 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.127+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb18140b20 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.128+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb18073990 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.128+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb10039040 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.129+0000 7fbb0e7fc640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.129+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb100ab290 con 0x7fbb181512b0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.129+0000 7fbb1d906640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:35.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.129+0000 7fbb1d906640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbafc06b4d0 0x7fbafc06d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb08000f60 tx=0x7fbb08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:35.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.133+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb10079f00 con 0x7fbb181512b0 2024-04-25T10:00:35.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.341+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbb180cf6e0 con 0x7fbb181512b0 2024-04-25T10:00:35.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.341+0000 7fbb0e7fc640 1 -- 172.21.15.148:0/2941657502 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbb10079800 con 0x7fbb181512b0 2024-04-25T10:00:35.347 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:00:35.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbafc06b4d0 msgr2=0x7fbafc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbafc06b4d0 0x7fbafc06d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb08000f60 tx=0x7fbb08009040 comp rx=0 tx=0).stop 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 msgr2=0x7fbb181f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 0x7fbb181f6e30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009d90 tx=0x7fbb10004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 shutdown_connections 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb181512b0 0x7fbb181f6e30 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb18154ed0 0x7fbb18150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 --2- 172.21.15.148:0/2941657502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb18071ac0 0x7fbb1813dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 >> 172.21.15.148:0/2941657502 conn(0x7fbb1806c1d0 msgr2=0x7fbb1814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 shutdown_connections 2024-04-25T10:00:35.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:35.344+0000 7fbb1fb6a640 1 -- 172.21.15.148:0/2941657502 wait complete. 2024-04-25T10:00:35.359 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:00:35.360 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:35.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.360 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:35.376 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42400sec preferred_lft 42400sec 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.377 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:00:35.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.379 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:00:35.438 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:00:35.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.439 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:00:35.496 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:00:35.496 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:00:35.497 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:00:35.497 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:00:35.497 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:00:35.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.497 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:00:35.497 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:35.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:00:35.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:00:35.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:00:35.497 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:35.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.581 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:00:35.636 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:00:35.636 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:00:35.636 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:35.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:00:35.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:00:35.700 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:35.754 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:00:35.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:00:35.755 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:35.809 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:00:35.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.809 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:00:35.890 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:00:35.890 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:00:35.890 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:00:35.898 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:00:35.899 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:00:35.899 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:00:35.899 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:00:35.899 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:00:35.915 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:00:35.916 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:00:35.916 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:00:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:00:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:00:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:00:35.916 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:00:35.917 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:00:35.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:35.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:35.982 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:00:36.039 INFO:teuthology.orchestra.run.smithi148.stdout:client6071 2024-04-25T10:00:36.039 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:36.039 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:36.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:36.094 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:36.094 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:36.094 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:00:36.094 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:36.095 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:36.095 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:36.095 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766606 s, 137 MB/s 2024-04-25T10:00:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829724 s, 126 MB/s 2024-04-25T10:00:36.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787338 s, 133 MB/s 2024-04-25T10:00:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866365 s, 121 MB/s 2024-04-25T10:00:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724342 s, 145 MB/s 2024-04-25T10:00:36.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635405 s, 165 MB/s 2024-04-25T10:00:36.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069807 s, 150 MB/s 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.220 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055167 s, 190 MB/s 2024-04-25T10:00:36.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.228 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734125 s, 143 MB/s 2024-04-25T10:00:36.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814181 s, 129 MB/s 2024-04-25T10:00:36.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701785 s, 149 MB/s 2024-04-25T10:00:36.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.253 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721904 s, 145 MB/s 2024-04-25T10:00:36.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068603 s, 153 MB/s 2024-04-25T10:00:36.267 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585908 s, 179 MB/s 2024-04-25T10:00:36.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068528 s, 153 MB/s 2024-04-25T10:00:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760034 s, 138 MB/s 2024-04-25T10:00:36.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685771 s, 153 MB/s 2024-04-25T10:00:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655762 s, 160 MB/s 2024-04-25T10:00:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069634 s, 151 MB/s 2024-04-25T10:00:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579816 s, 181 MB/s 2024-04-25T10:00:36.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638683 s, 164 MB/s 2024-04-25T10:00:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.329 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007217 s, 145 MB/s 2024-04-25T10:00:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596612 s, 176 MB/s 2024-04-25T10:00:36.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.344 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745204 s, 141 MB/s 2024-04-25T10:00:36.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.351 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582466 s, 180 MB/s 2024-04-25T10:00:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.357 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054978 s, 191 MB/s 2024-04-25T10:00:36.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.365 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659497 s, 159 MB/s 2024-04-25T10:00:36.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644304 s, 163 MB/s 2024-04-25T10:00:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713527 s, 147 MB/s 2024-04-25T10:00:36.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578791 s, 181 MB/s 2024-04-25T10:00:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660913 s, 159 MB/s 2024-04-25T10:00:36.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.402 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554907 s, 189 MB/s 2024-04-25T10:00:36.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:36.402 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738512 s, 142 MB/s 2024-04-25T10:00:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572694 s, 183 MB/s 2024-04-25T10:00:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659107 s, 159 MB/s 2024-04-25T10:00:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561899 s, 187 MB/s 2024-04-25T10:00:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.496 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711783 s, 147 MB/s 2024-04-25T10:00:36.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654445 s, 160 MB/s 2024-04-25T10:00:36.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.511 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686251 s, 153 MB/s 2024-04-25T10:00:36.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.518 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567634 s, 185 MB/s 2024-04-25T10:00:36.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566698 s, 185 MB/s 2024-04-25T10:00:36.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.532 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064872 s, 162 MB/s 2024-04-25T10:00:36.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.540 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734399 s, 143 MB/s 2024-04-25T10:00:36.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.547 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632032 s, 166 MB/s 2024-04-25T10:00:36.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.556 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763431 s, 137 MB/s 2024-04-25T10:00:36.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.562 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569829 s, 184 MB/s 2024-04-25T10:00:36.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.571 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774647 s, 135 MB/s 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576927 s, 182 MB/s 2024-04-25T10:00:36.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622984 s, 168 MB/s 2024-04-25T10:00:36.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.592 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556722 s, 188 MB/s 2024-04-25T10:00:36.599 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.599 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730974 s, 143 MB/s 2024-04-25T10:00:36.606 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.606 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.606 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576543 s, 182 MB/s 2024-04-25T10:00:36.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.614 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652236 s, 161 MB/s 2024-04-25T10:00:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.621 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006528 s, 161 MB/s 2024-04-25T10:00:36.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.628 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.628 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557044 s, 188 MB/s 2024-04-25T10:00:36.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.636 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759575 s, 138 MB/s 2024-04-25T10:00:36.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.643 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618006 s, 170 MB/s 2024-04-25T10:00:36.651 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754215 s, 139 MB/s 2024-04-25T10:00:36.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065752 s, 159 MB/s 2024-04-25T10:00:36.665 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.665 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.665 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552711 s, 190 MB/s 2024-04-25T10:00:36.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.672 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556423 s, 188 MB/s 2024-04-25T10:00:36.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.679 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573021 s, 183 MB/s 2024-04-25T10:00:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749741 s, 140 MB/s 2024-04-25T10:00:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715614 s, 147 MB/s 2024-04-25T10:00:36.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:36.696 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:36.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560886 s, 187 MB/s 2024-04-25T10:00:36.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.767 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666457 s, 157 MB/s 2024-04-25T10:00:36.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.773 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558528 s, 188 MB/s 2024-04-25T10:00:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542086 s, 193 MB/s 2024-04-25T10:00:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055244 s, 190 MB/s 2024-04-25T10:00:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057792 s, 181 MB/s 2024-04-25T10:00:36.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.801 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756126 s, 139 MB/s 2024-04-25T10:00:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559584 s, 187 MB/s 2024-04-25T10:00:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054629 s, 192 MB/s 2024-04-25T10:00:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621556 s, 169 MB/s 2024-04-25T10:00:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.828 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058111 s, 180 MB/s 2024-04-25T10:00:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.835 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571498 s, 183 MB/s 2024-04-25T10:00:36.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682791 s, 154 MB/s 2024-04-25T10:00:36.849 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.849 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.849 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579382 s, 181 MB/s 2024-04-25T10:00:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.857 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693201 s, 151 MB/s 2024-04-25T10:00:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.865 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683747 s, 153 MB/s 2024-04-25T10:00:36.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.871 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565417 s, 185 MB/s 2024-04-25T10:00:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701481 s, 149 MB/s 2024-04-25T10:00:36.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.886 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546031 s, 192 MB/s 2024-04-25T10:00:36.893 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.894 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577142 s, 182 MB/s 2024-04-25T10:00:36.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.899 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559032 s, 188 MB/s 2024-04-25T10:00:36.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690967 s, 152 MB/s 2024-04-25T10:00:36.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.914 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626944 s, 167 MB/s 2024-04-25T10:00:36.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637505 s, 164 MB/s 2024-04-25T10:00:36.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.929 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723641 s, 145 MB/s 2024-04-25T10:00:36.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.936 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564734 s, 186 MB/s 2024-04-25T10:00:36.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566911 s, 185 MB/s 2024-04-25T10:00:36.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710405 s, 148 MB/s 2024-04-25T10:00:36.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554331 s, 189 MB/s 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.964 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687605 s, 152 MB/s 2024-04-25T10:00:36.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.972 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070665 s, 148 MB/s 2024-04-25T10:00:36.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:36.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:36.979 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00543832 s, 193 MB/s 2024-04-25T10:00:36.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:36.980 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:37.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661896 s, 158 MB/s 2024-04-25T10:00:37.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.051 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558307 s, 188 MB/s 2024-04-25T10:00:37.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.058 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584303 s, 179 MB/s 2024-04-25T10:00:37.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.064 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554405 s, 189 MB/s 2024-04-25T10:00:37.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724954 s, 145 MB/s 2024-04-25T10:00:37.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560186 s, 187 MB/s 2024-04-25T10:00:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711875 s, 147 MB/s 2024-04-25T10:00:37.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634685 s, 165 MB/s 2024-04-25T10:00:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.102 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635653 s, 165 MB/s 2024-04-25T10:00:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615118 s, 170 MB/s 2024-04-25T10:00:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579197 s, 181 MB/s 2024-04-25T10:00:37.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595705 s, 176 MB/s 2024-04-25T10:00:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572575 s, 183 MB/s 2024-04-25T10:00:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679097 s, 154 MB/s 2024-04-25T10:00:37.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062383 s, 168 MB/s 2024-04-25T10:00:37.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658247 s, 159 MB/s 2024-04-25T10:00:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700569 s, 150 MB/s 2024-04-25T10:00:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581411 s, 180 MB/s 2024-04-25T10:00:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634346 s, 165 MB/s 2024-04-25T10:00:37.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.180 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657798 s, 159 MB/s 2024-04-25T10:00:37.188 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.188 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.188 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734641 s, 143 MB/s 2024-04-25T10:00:37.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.195 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615822 s, 170 MB/s 2024-04-25T10:00:37.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679848 s, 154 MB/s 2024-04-25T10:00:37.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.210 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063007 s, 166 MB/s 2024-04-25T10:00:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.217 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595829 s, 176 MB/s 2024-04-25T10:00:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.224 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649202 s, 162 MB/s 2024-04-25T10:00:37.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.232 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713019 s, 147 MB/s 2024-04-25T10:00:37.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073408 s, 143 MB/s 2024-04-25T10:00:37.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713328 s, 147 MB/s 2024-04-25T10:00:37.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.255 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562324 s, 186 MB/s 2024-04-25T10:00:37.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705788 s, 149 MB/s 2024-04-25T10:00:37.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655555 s, 160 MB/s 2024-04-25T10:00:37.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:37.272 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:37.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636765 s, 165 MB/s 2024-04-25T10:00:37.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590115 s, 178 MB/s 2024-04-25T10:00:37.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.349 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00539264 s, 194 MB/s 2024-04-25T10:00:37.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.356 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549247 s, 191 MB/s 2024-04-25T10:00:37.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.364 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725638 s, 145 MB/s 2024-04-25T10:00:37.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713628 s, 147 MB/s 2024-04-25T10:00:37.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682962 s, 154 MB/s 2024-04-25T10:00:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638052 s, 164 MB/s 2024-04-25T10:00:37.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632373 s, 166 MB/s 2024-04-25T10:00:37.400 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572409 s, 183 MB/s 2024-04-25T10:00:37.407 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.407 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.407 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055071 s, 190 MB/s 2024-04-25T10:00:37.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.414 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555408 s, 189 MB/s 2024-04-25T10:00:37.422 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.422 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.422 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752783 s, 139 MB/s 2024-04-25T10:00:37.429 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.429 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.429 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567505 s, 185 MB/s 2024-04-25T10:00:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.436 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622494 s, 168 MB/s 2024-04-25T10:00:37.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.444 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729232 s, 144 MB/s 2024-04-25T10:00:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.451 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575619 s, 182 MB/s 2024-04-25T10:00:37.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.459 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071149 s, 147 MB/s 2024-04-25T10:00:37.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764799 s, 137 MB/s 2024-04-25T10:00:37.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.475 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070418 s, 149 MB/s 2024-04-25T10:00:37.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.483 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624568 s, 168 MB/s 2024-04-25T10:00:37.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608001 s, 172 MB/s 2024-04-25T10:00:37.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.497 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067313 s, 156 MB/s 2024-04-25T10:00:37.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.503 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544211 s, 193 MB/s 2024-04-25T10:00:37.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.511 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643746 s, 163 MB/s 2024-04-25T10:00:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563837 s, 186 MB/s 2024-04-25T10:00:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.524 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612319 s, 171 MB/s 2024-04-25T10:00:37.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.532 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710045 s, 148 MB/s 2024-04-25T10:00:37.539 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.539 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.539 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575903 s, 182 MB/s 2024-04-25T10:00:37.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.546 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645368 s, 162 MB/s 2024-04-25T10:00:37.552 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.553 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566068 s, 185 MB/s 2024-04-25T10:00:37.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.560 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586934 s, 179 MB/s 2024-04-25T10:00:37.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:37.561 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:37.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.627 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745219 s, 141 MB/s 2024-04-25T10:00:37.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.635 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069161 s, 152 MB/s 2024-04-25T10:00:37.642 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.642 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.642 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006556 s, 160 MB/s 2024-04-25T10:00:37.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.649 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614899 s, 171 MB/s 2024-04-25T10:00:37.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.657 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647012 s, 162 MB/s 2024-04-25T10:00:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.665 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710035 s, 148 MB/s 2024-04-25T10:00:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.672 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688911 s, 152 MB/s 2024-04-25T10:00:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625064 s, 168 MB/s 2024-04-25T10:00:37.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.686 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605595 s, 173 MB/s 2024-04-25T10:00:37.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.693 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574902 s, 182 MB/s 2024-04-25T10:00:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702508 s, 149 MB/s 2024-04-25T10:00:37.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.708 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645507 s, 162 MB/s 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057473 s, 182 MB/s 2024-04-25T10:00:37.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644493 s, 163 MB/s 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.729 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635069 s, 165 MB/s 2024-04-25T10:00:37.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694937 s, 151 MB/s 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.747 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627715 s, 167 MB/s 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.752 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641431 s, 163 MB/s 2024-04-25T10:00:37.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.758 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576379 s, 182 MB/s 2024-04-25T10:00:37.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694353 s, 151 MB/s 2024-04-25T10:00:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067345 s, 156 MB/s 2024-04-25T10:00:37.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564293 s, 186 MB/s 2024-04-25T10:00:37.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.787 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587421 s, 179 MB/s 2024-04-25T10:00:37.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.794 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578529 s, 181 MB/s 2024-04-25T10:00:37.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.802 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701214 s, 150 MB/s 2024-04-25T10:00:37.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720375 s, 146 MB/s 2024-04-25T10:00:37.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.817 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062318 s, 168 MB/s 2024-04-25T10:00:37.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716646 s, 146 MB/s 2024-04-25T10:00:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707057 s, 148 MB/s 2024-04-25T10:00:37.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645592 s, 162 MB/s 2024-04-25T10:00:37.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.848 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.848 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650915 s, 161 MB/s 2024-04-25T10:00:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561813 s, 187 MB/s 2024-04-25T10:00:37.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:37.856 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:37.856 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:37.857 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:00:37.857 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:37.858 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:37.858 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:37.859 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:37.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.922 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654937 s, 160 MB/s 2024-04-25T10:00:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.929 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729 s, 144 MB/s 2024-04-25T10:00:37.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.936 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.936 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577974 s, 181 MB/s 2024-04-25T10:00:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554378 s, 189 MB/s 2024-04-25T10:00:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696637 s, 151 MB/s 2024-04-25T10:00:37.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.957 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570662 s, 184 MB/s 2024-04-25T10:00:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583686 s, 180 MB/s 2024-04-25T10:00:37.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.971 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651461 s, 161 MB/s 2024-04-25T10:00:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.979 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658265 s, 159 MB/s 2024-04-25T10:00:37.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555021 s, 189 MB/s 2024-04-25T10:00:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634229 s, 165 MB/s 2024-04-25T10:00:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058225 s, 180 MB/s 2024-04-25T10:00:38.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.007 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721084 s, 145 MB/s 2024-04-25T10:00:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.014 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636682 s, 165 MB/s 2024-04-25T10:00:38.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070955 s, 148 MB/s 2024-04-25T10:00:38.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.029 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561704 s, 187 MB/s 2024-04-25T10:00:38.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744039 s, 141 MB/s 2024-04-25T10:00:38.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.044 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615199 s, 170 MB/s 2024-04-25T10:00:38.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.051 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623 s, 168 MB/s 2024-04-25T10:00:38.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069886 s, 150 MB/s 2024-04-25T10:00:38.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609783 s, 172 MB/s 2024-04-25T10:00:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617424 s, 170 MB/s 2024-04-25T10:00:38.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622405 s, 168 MB/s 2024-04-25T10:00:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629915 s, 166 MB/s 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553573 s, 189 MB/s 2024-04-25T10:00:38.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.102 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007094 s, 148 MB/s 2024-04-25T10:00:38.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.109 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615066 s, 170 MB/s 2024-04-25T10:00:38.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.116 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566863 s, 185 MB/s 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.125 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717249 s, 146 MB/s 2024-04-25T10:00:38.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613639 s, 171 MB/s 2024-04-25T10:00:38.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683543 s, 153 MB/s 2024-04-25T10:00:38.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629792 s, 166 MB/s 2024-04-25T10:00:38.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:38.147 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:38.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066012 s, 159 MB/s 2024-04-25T10:00:38.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.219 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067141 s, 156 MB/s 2024-04-25T10:00:38.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.227 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708879 s, 148 MB/s 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719101 s, 146 MB/s 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573744 s, 183 MB/s 2024-04-25T10:00:38.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.249 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055413 s, 189 MB/s 2024-04-25T10:00:38.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.256 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629152 s, 167 MB/s 2024-04-25T10:00:38.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.262 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550104 s, 191 MB/s 2024-04-25T10:00:38.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630751 s, 166 MB/s 2024-04-25T10:00:38.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.277 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709195 s, 148 MB/s 2024-04-25T10:00:38.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.284 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580684 s, 181 MB/s 2024-04-25T10:00:38.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073008 s, 144 MB/s 2024-04-25T10:00:38.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558875 s, 188 MB/s 2024-04-25T10:00:38.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.306 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610272 s, 172 MB/s 2024-04-25T10:00:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.313 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643006 s, 163 MB/s 2024-04-25T10:00:38.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.321 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716299 s, 146 MB/s 2024-04-25T10:00:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626064 s, 167 MB/s 2024-04-25T10:00:38.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633281 s, 166 MB/s 2024-04-25T10:00:38.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.342 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568876 s, 184 MB/s 2024-04-25T10:00:38.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.348 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055315 s, 190 MB/s 2024-04-25T10:00:38.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.356 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.356 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646219 s, 162 MB/s 2024-04-25T10:00:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648362 s, 162 MB/s 2024-04-25T10:00:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593329 s, 177 MB/s 2024-04-25T10:00:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624601 s, 168 MB/s 2024-04-25T10:00:38.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.385 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695262 s, 151 MB/s 2024-04-25T10:00:38.392 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.392 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.392 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643973 s, 163 MB/s 2024-04-25T10:00:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629599 s, 167 MB/s 2024-04-25T10:00:38.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.406 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583222 s, 180 MB/s 2024-04-25T10:00:38.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.413 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562565 s, 186 MB/s 2024-04-25T10:00:38.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.420 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626146 s, 167 MB/s 2024-04-25T10:00:38.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722146 s, 145 MB/s 2024-04-25T10:00:38.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.435 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613547 s, 171 MB/s 2024-04-25T10:00:38.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:38.436 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:38.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.500 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562939 s, 186 MB/s 2024-04-25T10:00:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555318 s, 189 MB/s 2024-04-25T10:00:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562743 s, 186 MB/s 2024-04-25T10:00:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075344 s, 139 MB/s 2024-04-25T10:00:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064788 s, 162 MB/s 2024-04-25T10:00:38.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615418 s, 170 MB/s 2024-04-25T10:00:38.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.542 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548811 s, 191 MB/s 2024-04-25T10:00:38.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719612 s, 146 MB/s 2024-04-25T10:00:38.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612155 s, 171 MB/s 2024-04-25T10:00:38.563 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574387 s, 183 MB/s 2024-04-25T10:00:38.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.570 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558567 s, 188 MB/s 2024-04-25T10:00:38.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.577 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623658 s, 168 MB/s 2024-04-25T10:00:38.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706023 s, 149 MB/s 2024-04-25T10:00:38.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.591 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552065 s, 190 MB/s 2024-04-25T10:00:38.598 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.598 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.598 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565483 s, 185 MB/s 2024-04-25T10:00:38.605 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.605 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.605 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652335 s, 161 MB/s 2024-04-25T10:00:38.613 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.614 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737654 s, 142 MB/s 2024-04-25T10:00:38.620 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.620 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.620 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055684 s, 188 MB/s 2024-04-25T10:00:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690425 s, 152 MB/s 2024-04-25T10:00:38.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.636 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735752 s, 143 MB/s 2024-04-25T10:00:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.644 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708396 s, 148 MB/s 2024-04-25T10:00:38.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572056 s, 183 MB/s 2024-04-25T10:00:38.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728823 s, 144 MB/s 2024-04-25T10:00:38.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.666 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625706 s, 168 MB/s 2024-04-25T10:00:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550326 s, 191 MB/s 2024-04-25T10:00:38.679 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.679 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.680 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620052 s, 169 MB/s 2024-04-25T10:00:38.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544902 s, 192 MB/s 2024-04-25T10:00:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706072 s, 149 MB/s 2024-04-25T10:00:38.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.701 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647492 s, 162 MB/s 2024-04-25T10:00:38.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.709 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711615 s, 147 MB/s 2024-04-25T10:00:38.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.717 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675608 s, 155 MB/s 2024-04-25T10:00:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.723 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561151 s, 187 MB/s 2024-04-25T10:00:38.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:38.725 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793723 s, 132 MB/s 2024-04-25T10:00:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.799 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695416 s, 151 MB/s 2024-04-25T10:00:38.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.805 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058435 s, 179 MB/s 2024-04-25T10:00:38.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.812 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547832 s, 191 MB/s 2024-04-25T10:00:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563767 s, 186 MB/s 2024-04-25T10:00:38.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.826 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063395 s, 165 MB/s 2024-04-25T10:00:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668043 s, 157 MB/s 2024-04-25T10:00:38.841 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.841 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.841 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666133 s, 157 MB/s 2024-04-25T10:00:38.849 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.849 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.849 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730711 s, 144 MB/s 2024-04-25T10:00:38.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.857 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711377 s, 147 MB/s 2024-04-25T10:00:38.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.863 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556806 s, 188 MB/s 2024-04-25T10:00:38.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.871 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711133 s, 147 MB/s 2024-04-25T10:00:38.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.879 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706401 s, 148 MB/s 2024-04-25T10:00:38.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.886 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547074 s, 192 MB/s 2024-04-25T10:00:38.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.892 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054511 s, 192 MB/s 2024-04-25T10:00:38.900 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.900 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.900 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699911 s, 150 MB/s 2024-04-25T10:00:38.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572309 s, 183 MB/s 2024-04-25T10:00:38.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.914 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649113 s, 162 MB/s 2024-04-25T10:00:38.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.922 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710786 s, 148 MB/s 2024-04-25T10:00:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.930 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724413 s, 145 MB/s 2024-04-25T10:00:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.937 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065169 s, 161 MB/s 2024-04-25T10:00:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.945 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702341 s, 149 MB/s 2024-04-25T10:00:38.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.952 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637902 s, 164 MB/s 2024-04-25T10:00:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620908 s, 169 MB/s 2024-04-25T10:00:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720849 s, 145 MB/s 2024-04-25T10:00:38.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.975 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630483 s, 166 MB/s 2024-04-25T10:00:38.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.981 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595609 s, 176 MB/s 2024-04-25T10:00:38.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.989 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697803 s, 150 MB/s 2024-04-25T10:00:38.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:38.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:38.996 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562573 s, 186 MB/s 2024-04-25T10:00:39.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.002 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569303 s, 184 MB/s 2024-04-25T10:00:39.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.010 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647138 s, 162 MB/s 2024-04-25T10:00:39.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562855 s, 186 MB/s 2024-04-25T10:00:39.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.017 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.017 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:39.017 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:00:39.017 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.018 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.018 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:39.018 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:39.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.083 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073321 s, 143 MB/s 2024-04-25T10:00:39.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.089 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557296 s, 188 MB/s 2024-04-25T10:00:39.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.096 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563085 s, 186 MB/s 2024-04-25T10:00:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.103 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644679 s, 163 MB/s 2024-04-25T10:00:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.110 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579137 s, 181 MB/s 2024-04-25T10:00:39.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.118 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.118 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658283 s, 159 MB/s 2024-04-25T10:00:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697523 s, 150 MB/s 2024-04-25T10:00:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064148 s, 163 MB/s 2024-04-25T10:00:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.140 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622201 s, 169 MB/s 2024-04-25T10:00:39.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.147 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062903 s, 167 MB/s 2024-04-25T10:00:39.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.154 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651621 s, 161 MB/s 2024-04-25T10:00:39.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.162 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678749 s, 154 MB/s 2024-04-25T10:00:39.170 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.170 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.170 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743425 s, 141 MB/s 2024-04-25T10:00:39.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.177 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567924 s, 185 MB/s 2024-04-25T10:00:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552955 s, 190 MB/s 2024-04-25T10:00:39.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603611 s, 174 MB/s 2024-04-25T10:00:39.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706952 s, 148 MB/s 2024-04-25T10:00:39.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.205 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603957 s, 174 MB/s 2024-04-25T10:00:39.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621208 s, 169 MB/s 2024-04-25T10:00:39.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.219 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601502 s, 174 MB/s 2024-04-25T10:00:39.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617625 s, 170 MB/s 2024-04-25T10:00:39.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.233 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619277 s, 169 MB/s 2024-04-25T10:00:39.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639163 s, 164 MB/s 2024-04-25T10:00:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711422 s, 147 MB/s 2024-04-25T10:00:39.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.256 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615498 s, 170 MB/s 2024-04-25T10:00:39.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.262 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546019 s, 192 MB/s 2024-04-25T10:00:39.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708681 s, 148 MB/s 2024-04-25T10:00:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710207 s, 148 MB/s 2024-04-25T10:00:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.285 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057761 s, 182 MB/s 2024-04-25T10:00:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00543488 s, 193 MB/s 2024-04-25T10:00:39.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708424 s, 148 MB/s 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615705 s, 170 MB/s 2024-04-25T10:00:39.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.307 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.307 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:39.307 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:00:39.307 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.307 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.308 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:39.308 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:39.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.371 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558307 s, 188 MB/s 2024-04-25T10:00:39.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.378 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652043 s, 161 MB/s 2024-04-25T10:00:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.386 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707403 s, 148 MB/s 2024-04-25T10:00:39.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690632 s, 152 MB/s 2024-04-25T10:00:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659376 s, 159 MB/s 2024-04-25T10:00:39.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074422 s, 141 MB/s 2024-04-25T10:00:39.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694103 s, 151 MB/s 2024-04-25T10:00:39.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578261 s, 181 MB/s 2024-04-25T10:00:39.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685919 s, 153 MB/s 2024-04-25T10:00:39.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.440 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757601 s, 138 MB/s 2024-04-25T10:00:39.446 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.446 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.446 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586825 s, 179 MB/s 2024-04-25T10:00:39.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.453 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566896 s, 185 MB/s 2024-04-25T10:00:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634942 s, 165 MB/s 2024-04-25T10:00:39.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.468 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701027 s, 150 MB/s 2024-04-25T10:00:39.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.475 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552367 s, 190 MB/s 2024-04-25T10:00:39.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.481 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00537349 s, 195 MB/s 2024-04-25T10:00:39.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.488 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650249 s, 161 MB/s 2024-04-25T10:00:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606941 s, 173 MB/s 2024-04-25T10:00:39.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.503 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072014 s, 146 MB/s 2024-04-25T10:00:39.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.510 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562372 s, 186 MB/s 2024-04-25T10:00:39.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641869 s, 163 MB/s 2024-04-25T10:00:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644778 s, 163 MB/s 2024-04-25T10:00:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.531 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565008 s, 186 MB/s 2024-04-25T10:00:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.538 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610085 s, 172 MB/s 2024-04-25T10:00:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.546 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694218 s, 151 MB/s 2024-04-25T10:00:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.554 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682256 s, 154 MB/s 2024-04-25T10:00:39.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.563 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711718 s, 147 MB/s 2024-04-25T10:00:39.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.568 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570224 s, 184 MB/s 2024-04-25T10:00:39.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.574 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558459 s, 188 MB/s 2024-04-25T10:00:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629958 s, 166 MB/s 2024-04-25T10:00:39.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578236 s, 181 MB/s 2024-04-25T10:00:39.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711634 s, 147 MB/s 2024-04-25T10:00:39.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.597 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.598 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:39.598 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:00:39.598 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.598 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.598 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:39.598 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:39.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652265 s, 161 MB/s 2024-04-25T10:00:39.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.671 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693957 s, 151 MB/s 2024-04-25T10:00:39.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.678 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632374 s, 166 MB/s 2024-04-25T10:00:39.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.686 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729054 s, 144 MB/s 2024-04-25T10:00:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.694 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631424 s, 166 MB/s 2024-04-25T10:00:39.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.702 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716568 s, 146 MB/s 2024-04-25T10:00:39.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.709 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687048 s, 153 MB/s 2024-04-25T10:00:39.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.716 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627211 s, 167 MB/s 2024-04-25T10:00:39.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.723 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610695 s, 172 MB/s 2024-04-25T10:00:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583551 s, 180 MB/s 2024-04-25T10:00:39.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556742 s, 188 MB/s 2024-04-25T10:00:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706242 s, 148 MB/s 2024-04-25T10:00:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610762 s, 172 MB/s 2024-04-25T10:00:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564926 s, 186 MB/s 2024-04-25T10:00:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627722 s, 167 MB/s 2024-04-25T10:00:39.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.772 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610358 s, 172 MB/s 2024-04-25T10:00:39.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.779 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643658 s, 163 MB/s 2024-04-25T10:00:39.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550455 s, 190 MB/s 2024-04-25T10:00:39.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.792 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544293 s, 193 MB/s 2024-04-25T10:00:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.799 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056637 s, 185 MB/s 2024-04-25T10:00:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630576 s, 166 MB/s 2024-04-25T10:00:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.812 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554469 s, 189 MB/s 2024-04-25T10:00:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.819 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551965 s, 190 MB/s 2024-04-25T10:00:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.826 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610626 s, 172 MB/s 2024-04-25T10:00:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631564 s, 166 MB/s 2024-04-25T10:00:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715354 s, 147 MB/s 2024-04-25T10:00:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619435 s, 169 MB/s 2024-04-25T10:00:39.856 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.856 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.856 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070183 s, 149 MB/s 2024-04-25T10:00:39.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.863 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545162 s, 192 MB/s 2024-04-25T10:00:39.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552892 s, 190 MB/s 2024-04-25T10:00:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.877 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698799 s, 150 MB/s 2024-04-25T10:00:39.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.884 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631781 s, 166 MB/s 2024-04-25T10:00:39.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:39.885 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:39.886 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:39.886 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:00:39.886 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:39.886 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:39.886 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:39.886 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:39.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630196 s, 166 MB/s 2024-04-25T10:00:39.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750405 s, 140 MB/s 2024-04-25T10:00:39.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.965 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565395 s, 185 MB/s 2024-04-25T10:00:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579837 s, 181 MB/s 2024-04-25T10:00:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.979 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623345 s, 168 MB/s 2024-04-25T10:00:39.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.986 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592263 s, 177 MB/s 2024-04-25T10:00:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:39.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:39.994 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666773 s, 157 MB/s 2024-04-25T10:00:40.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616831 s, 170 MB/s 2024-04-25T10:00:40.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.007 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560734 s, 187 MB/s 2024-04-25T10:00:40.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.013 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551508 s, 190 MB/s 2024-04-25T10:00:40.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.021 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648854 s, 162 MB/s 2024-04-25T10:00:40.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617672 s, 170 MB/s 2024-04-25T10:00:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561469 s, 187 MB/s 2024-04-25T10:00:40.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626716 s, 167 MB/s 2024-04-25T10:00:40.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637631 s, 164 MB/s 2024-04-25T10:00:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618325 s, 170 MB/s 2024-04-25T10:00:40.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.063 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626386 s, 167 MB/s 2024-04-25T10:00:40.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.070 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647154 s, 162 MB/s 2024-04-25T10:00:40.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070694 s, 148 MB/s 2024-04-25T10:00:40.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720666 s, 146 MB/s 2024-04-25T10:00:40.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.093 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580722 s, 181 MB/s 2024-04-25T10:00:40.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.100 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641285 s, 164 MB/s 2024-04-25T10:00:40.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664548 s, 158 MB/s 2024-04-25T10:00:40.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619091 s, 169 MB/s 2024-04-25T10:00:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627284 s, 167 MB/s 2024-04-25T10:00:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.130 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699451 s, 150 MB/s 2024-04-25T10:00:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742029 s, 141 MB/s 2024-04-25T10:00:40.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.146 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708826 s, 148 MB/s 2024-04-25T10:00:40.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.154 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740515 s, 142 MB/s 2024-04-25T10:00:40.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.162 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653128 s, 161 MB/s 2024-04-25T10:00:40.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.170 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.170 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069648 s, 151 MB/s 2024-04-25T10:00:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.177 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636501 s, 165 MB/s 2024-04-25T10:00:40.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:40.178 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:40.178 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:40.178 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:00:40.178 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:40.179 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:40.179 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:40.179 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708576 s, 148 MB/s 2024-04-25T10:00:40.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.251 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658514 s, 159 MB/s 2024-04-25T10:00:40.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.257 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057938 s, 181 MB/s 2024-04-25T10:00:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693359 s, 151 MB/s 2024-04-25T10:00:40.272 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.272 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.272 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629498 s, 167 MB/s 2024-04-25T10:00:40.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.280 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724819 s, 145 MB/s 2024-04-25T10:00:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.287 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055492 s, 189 MB/s 2024-04-25T10:00:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.294 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652707 s, 161 MB/s 2024-04-25T10:00:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691755 s, 152 MB/s 2024-04-25T10:00:40.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.308 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054242 s, 193 MB/s 2024-04-25T10:00:40.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568464 s, 184 MB/s 2024-04-25T10:00:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696535 s, 151 MB/s 2024-04-25T10:00:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685702 s, 153 MB/s 2024-04-25T10:00:40.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.337 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562057 s, 187 MB/s 2024-04-25T10:00:40.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.345 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695992 s, 151 MB/s 2024-04-25T10:00:40.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.354 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647814 s, 162 MB/s 2024-04-25T10:00:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578664 s, 181 MB/s 2024-04-25T10:00:40.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065315 s, 161 MB/s 2024-04-25T10:00:40.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639405 s, 164 MB/s 2024-04-25T10:00:40.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.381 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614382 s, 171 MB/s 2024-04-25T10:00:40.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.388 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617629 s, 170 MB/s 2024-04-25T10:00:40.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550371 s, 191 MB/s 2024-04-25T10:00:40.400 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.400 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559277 s, 187 MB/s 2024-04-25T10:00:40.407 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.408 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618296 s, 170 MB/s 2024-04-25T10:00:40.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.416 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707485 s, 148 MB/s 2024-04-25T10:00:40.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.423 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693375 s, 151 MB/s 2024-04-25T10:00:40.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715844 s, 146 MB/s 2024-04-25T10:00:40.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694157 s, 151 MB/s 2024-04-25T10:00:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.446 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557937 s, 188 MB/s 2024-04-25T10:00:40.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704422 s, 149 MB/s 2024-04-25T10:00:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570904 s, 184 MB/s 2024-04-25T10:00:40.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726555 s, 144 MB/s 2024-04-25T10:00:40.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:40.470 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:40.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636775 s, 165 MB/s 2024-04-25T10:00:40.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571547 s, 183 MB/s 2024-04-25T10:00:40.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631292 s, 166 MB/s 2024-04-25T10:00:40.554 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.555 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593887 s, 177 MB/s 2024-04-25T10:00:40.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.561 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569414 s, 184 MB/s 2024-04-25T10:00:40.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.568 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564479 s, 186 MB/s 2024-04-25T10:00:40.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.575 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057263 s, 183 MB/s 2024-04-25T10:00:40.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627008 s, 167 MB/s 2024-04-25T10:00:40.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.590 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641073 s, 164 MB/s 2024-04-25T10:00:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634983 s, 165 MB/s 2024-04-25T10:00:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063366 s, 165 MB/s 2024-04-25T10:00:40.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631783 s, 166 MB/s 2024-04-25T10:00:40.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558076 s, 188 MB/s 2024-04-25T10:00:40.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628409 s, 167 MB/s 2024-04-25T10:00:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.631 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565399 s, 185 MB/s 2024-04-25T10:00:40.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.638 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617523 s, 170 MB/s 2024-04-25T10:00:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608587 s, 172 MB/s 2024-04-25T10:00:40.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631552 s, 166 MB/s 2024-04-25T10:00:40.658 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.658 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.658 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567886 s, 185 MB/s 2024-04-25T10:00:40.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.666 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697161 s, 150 MB/s 2024-04-25T10:00:40.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.673 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619241 s, 169 MB/s 2024-04-25T10:00:40.680 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.680 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.681 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621307 s, 169 MB/s 2024-04-25T10:00:40.687 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.687 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.687 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557033 s, 188 MB/s 2024-04-25T10:00:40.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.694 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599496 s, 175 MB/s 2024-04-25T10:00:40.702 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.702 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.702 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738239 s, 142 MB/s 2024-04-25T10:00:40.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.710 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704751 s, 149 MB/s 2024-04-25T10:00:40.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715023 s, 147 MB/s 2024-04-25T10:00:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.724 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545807 s, 192 MB/s 2024-04-25T10:00:40.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.733 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719378 s, 146 MB/s 2024-04-25T10:00:40.739 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.740 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635512 s, 165 MB/s 2024-04-25T10:00:40.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.747 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641685 s, 163 MB/s 2024-04-25T10:00:40.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.754 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580528 s, 181 MB/s 2024-04-25T10:00:40.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:40.755 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:40.755 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:40.755 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:00:40.755 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:40.755 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:40.755 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:40.756 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:40.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.819 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556867 s, 188 MB/s 2024-04-25T10:00:40.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.827 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649439 s, 161 MB/s 2024-04-25T10:00:40.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.835 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.835 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711414 s, 147 MB/s 2024-04-25T10:00:40.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642619 s, 163 MB/s 2024-04-25T10:00:40.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.850 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717369 s, 146 MB/s 2024-04-25T10:00:40.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.857 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642578 s, 163 MB/s 2024-04-25T10:00:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638344 s, 164 MB/s 2024-04-25T10:00:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.871 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555924 s, 189 MB/s 2024-04-25T10:00:40.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.879 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717867 s, 146 MB/s 2024-04-25T10:00:40.886 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.886 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.886 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580687 s, 181 MB/s 2024-04-25T10:00:40.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.894 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707384 s, 148 MB/s 2024-04-25T10:00:40.901 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.901 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.901 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062842 s, 167 MB/s 2024-04-25T10:00:40.909 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.909 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.909 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722051 s, 145 MB/s 2024-04-25T10:00:40.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.916 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572709 s, 183 MB/s 2024-04-25T10:00:40.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.923 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635877 s, 165 MB/s 2024-04-25T10:00:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.931 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726273 s, 144 MB/s 2024-04-25T10:00:40.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.938 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647207 s, 162 MB/s 2024-04-25T10:00:40.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.946 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706049 s, 149 MB/s 2024-04-25T10:00:40.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619797 s, 169 MB/s 2024-04-25T10:00:40.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.961 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647355 s, 162 MB/s 2024-04-25T10:00:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.967 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054485 s, 192 MB/s 2024-04-25T10:00:40.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576178 s, 182 MB/s 2024-04-25T10:00:40.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701961 s, 149 MB/s 2024-04-25T10:00:40.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.989 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620771 s, 169 MB/s 2024-04-25T10:00:40.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:40.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:40.996 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613247 s, 171 MB/s 2024-04-25T10:00:41.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.002 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603298 s, 174 MB/s 2024-04-25T10:00:41.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.009 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060815 s, 172 MB/s 2024-04-25T10:00:41.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557909 s, 188 MB/s 2024-04-25T10:00:41.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657224 s, 160 MB/s 2024-04-25T10:00:41.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.031 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658464 s, 159 MB/s 2024-04-25T10:00:41.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.038 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663167 s, 158 MB/s 2024-04-25T10:00:41.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.047 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743402 s, 141 MB/s 2024-04-25T10:00:41.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:41.048 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:41.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661434 s, 159 MB/s 2024-04-25T10:00:41.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711486 s, 147 MB/s 2024-04-25T10:00:41.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585525 s, 179 MB/s 2024-04-25T10:00:41.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.135 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550844 s, 190 MB/s 2024-04-25T10:00:41.141 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.141 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.141 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574457 s, 183 MB/s 2024-04-25T10:00:41.148 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.148 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.148 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562701 s, 186 MB/s 2024-04-25T10:00:41.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.155 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592473 s, 177 MB/s 2024-04-25T10:00:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561491 s, 187 MB/s 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071625 s, 146 MB/s 2024-04-25T10:00:41.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.177 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631562 s, 166 MB/s 2024-04-25T10:00:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700603 s, 150 MB/s 2024-04-25T10:00:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.191 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603192 s, 174 MB/s 2024-04-25T10:00:41.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.199 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648338 s, 162 MB/s 2024-04-25T10:00:41.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.207 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732157 s, 143 MB/s 2024-04-25T10:00:41.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.215 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701554 s, 149 MB/s 2024-04-25T10:00:41.223 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.223 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.223 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711427 s, 147 MB/s 2024-04-25T10:00:41.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.230 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651835 s, 161 MB/s 2024-04-25T10:00:41.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587231 s, 179 MB/s 2024-04-25T10:00:41.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.244 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611536 s, 171 MB/s 2024-04-25T10:00:41.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.251 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627937 s, 167 MB/s 2024-04-25T10:00:41.259 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.259 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.259 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063232 s, 166 MB/s 2024-04-25T10:00:41.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.266 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616522 s, 170 MB/s 2024-04-25T10:00:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623644 s, 168 MB/s 2024-04-25T10:00:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638349 s, 164 MB/s 2024-04-25T10:00:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.287 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616716 s, 170 MB/s 2024-04-25T10:00:41.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.294 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564655 s, 186 MB/s 2024-04-25T10:00:41.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721113 s, 145 MB/s 2024-04-25T10:00:41.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620382 s, 169 MB/s 2024-04-25T10:00:41.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.316 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063388 s, 165 MB/s 2024-04-25T10:00:41.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632166 s, 166 MB/s 2024-04-25T10:00:41.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.331 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691857 s, 152 MB/s 2024-04-25T10:00:41.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688925 s, 152 MB/s 2024-04-25T10:00:41.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:41.340 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:41.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568233 s, 185 MB/s 2024-04-25T10:00:41.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.411 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659724 s, 159 MB/s 2024-04-25T10:00:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.418 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659777 s, 159 MB/s 2024-04-25T10:00:41.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.426 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627586 s, 167 MB/s 2024-04-25T10:00:41.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573582 s, 183 MB/s 2024-04-25T10:00:41.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.440 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699809 s, 150 MB/s 2024-04-25T10:00:41.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578682 s, 181 MB/s 2024-04-25T10:00:41.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714428 s, 147 MB/s 2024-04-25T10:00:41.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636999 s, 165 MB/s 2024-04-25T10:00:41.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063448 s, 165 MB/s 2024-04-25T10:00:41.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.477 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695516 s, 151 MB/s 2024-04-25T10:00:41.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.485 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637441 s, 164 MB/s 2024-04-25T10:00:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630496 s, 166 MB/s 2024-04-25T10:00:41.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.499 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703713 s, 149 MB/s 2024-04-25T10:00:41.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.507 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700546 s, 150 MB/s 2024-04-25T10:00:41.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062541 s, 168 MB/s 2024-04-25T10:00:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582951 s, 180 MB/s 2024-04-25T10:00:41.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.527 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556324 s, 188 MB/s 2024-04-25T10:00:41.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693106 s, 151 MB/s 2024-04-25T10:00:41.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.543 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660016 s, 159 MB/s 2024-04-25T10:00:41.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627511 s, 167 MB/s 2024-04-25T10:00:41.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.556 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553955 s, 189 MB/s 2024-04-25T10:00:41.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698946 s, 150 MB/s 2024-04-25T10:00:41.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.571 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645429 s, 162 MB/s 2024-04-25T10:00:41.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.579 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701559 s, 149 MB/s 2024-04-25T10:00:41.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.586 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562655 s, 186 MB/s 2024-04-25T10:00:41.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643105 s, 163 MB/s 2024-04-25T10:00:41.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585518 s, 179 MB/s 2024-04-25T10:00:41.607 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.607 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.607 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616733 s, 170 MB/s 2024-04-25T10:00:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717729 s, 146 MB/s 2024-04-25T10:00:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.623 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709248 s, 148 MB/s 2024-04-25T10:00:41.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.630 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637843 s, 164 MB/s 2024-04-25T10:00:41.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.631 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.631 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:41.631 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:00:41.632 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.632 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.632 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:41.632 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:41.695 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.695 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.695 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553997 s, 189 MB/s 2024-04-25T10:00:41.702 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.703 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650838 s, 161 MB/s 2024-04-25T10:00:41.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.709 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569898 s, 184 MB/s 2024-04-25T10:00:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690747 s, 152 MB/s 2024-04-25T10:00:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630736 s, 166 MB/s 2024-04-25T10:00:41.731 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.731 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.731 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619721 s, 169 MB/s 2024-04-25T10:00:41.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.738 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564583 s, 186 MB/s 2024-04-25T10:00:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057827 s, 181 MB/s 2024-04-25T10:00:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.752 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715368 s, 147 MB/s 2024-04-25T10:00:41.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.760 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702425 s, 149 MB/s 2024-04-25T10:00:41.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638223 s, 164 MB/s 2024-04-25T10:00:41.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.775 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622379 s, 168 MB/s 2024-04-25T10:00:41.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.781 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577501 s, 182 MB/s 2024-04-25T10:00:41.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.788 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567875 s, 185 MB/s 2024-04-25T10:00:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.794 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00541422 s, 194 MB/s 2024-04-25T10:00:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.802 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700435 s, 150 MB/s 2024-04-25T10:00:41.809 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063248 s, 166 MB/s 2024-04-25T10:00:41.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.817 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634539 s, 165 MB/s 2024-04-25T10:00:41.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714902 s, 147 MB/s 2024-04-25T10:00:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576222 s, 182 MB/s 2024-04-25T10:00:41.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568414 s, 184 MB/s 2024-04-25T10:00:41.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645951 s, 162 MB/s 2024-04-25T10:00:41.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071709 s, 146 MB/s 2024-04-25T10:00:41.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703809 s, 149 MB/s 2024-04-25T10:00:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554034 s, 189 MB/s 2024-04-25T10:00:41.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660104 s, 159 MB/s 2024-04-25T10:00:41.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702118 s, 149 MB/s 2024-04-25T10:00:41.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612078 s, 171 MB/s 2024-04-25T10:00:41.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622868 s, 168 MB/s 2024-04-25T10:00:41.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.905 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704288 s, 149 MB/s 2024-04-25T10:00:41.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618627 s, 170 MB/s 2024-04-25T10:00:41.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696445 s, 151 MB/s 2024-04-25T10:00:41.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:41.922 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654209 s, 160 MB/s 2024-04-25T10:00:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:41.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602108 s, 174 MB/s 2024-04-25T10:00:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697494 s, 150 MB/s 2024-04-25T10:00:42.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.009 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742129 s, 141 MB/s 2024-04-25T10:00:42.017 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.017 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.018 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576986 s, 182 MB/s 2024-04-25T10:00:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563895 s, 186 MB/s 2024-04-25T10:00:42.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.030 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635514 s, 165 MB/s 2024-04-25T10:00:42.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.038 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724315 s, 145 MB/s 2024-04-25T10:00:42.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632882 s, 166 MB/s 2024-04-25T10:00:42.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568592 s, 184 MB/s 2024-04-25T10:00:42.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637082 s, 165 MB/s 2024-04-25T10:00:42.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.065 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575601 s, 182 MB/s 2024-04-25T10:00:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583521 s, 180 MB/s 2024-04-25T10:00:42.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559674 s, 187 MB/s 2024-04-25T10:00:42.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584614 s, 179 MB/s 2024-04-25T10:00:42.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.093 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069532 s, 151 MB/s 2024-04-25T10:00:42.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637016 s, 165 MB/s 2024-04-25T10:00:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740038 s, 142 MB/s 2024-04-25T10:00:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642811 s, 163 MB/s 2024-04-25T10:00:42.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635441 s, 165 MB/s 2024-04-25T10:00:42.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630892 s, 166 MB/s 2024-04-25T10:00:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690736 s, 152 MB/s 2024-04-25T10:00:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593917 s, 177 MB/s 2024-04-25T10:00:42.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.152 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549311 s, 191 MB/s 2024-04-25T10:00:42.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.160 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706649 s, 148 MB/s 2024-04-25T10:00:42.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623088 s, 168 MB/s 2024-04-25T10:00:42.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670902 s, 156 MB/s 2024-04-25T10:00:42.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722392 s, 145 MB/s 2024-04-25T10:00:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.189 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613391 s, 171 MB/s 2024-04-25T10:00:42.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.196 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594367 s, 176 MB/s 2024-04-25T10:00:42.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.204 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670108 s, 156 MB/s 2024-04-25T10:00:42.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622023 s, 169 MB/s 2024-04-25T10:00:42.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:42.213 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:42.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006414 s, 163 MB/s 2024-04-25T10:00:42.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697966 s, 150 MB/s 2024-04-25T10:00:42.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.293 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678159 s, 155 MB/s 2024-04-25T10:00:42.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732809 s, 143 MB/s 2024-04-25T10:00:42.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.308 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584495 s, 179 MB/s 2024-04-25T10:00:42.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.316 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683546 s, 153 MB/s 2024-04-25T10:00:42.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056928 s, 184 MB/s 2024-04-25T10:00:42.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641354 s, 163 MB/s 2024-04-25T10:00:42.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.337 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617619 s, 170 MB/s 2024-04-25T10:00:42.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550043 s, 191 MB/s 2024-04-25T10:00:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.351 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657896 s, 159 MB/s 2024-04-25T10:00:42.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658612 s, 159 MB/s 2024-04-25T10:00:42.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.365 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062008 s, 169 MB/s 2024-04-25T10:00:42.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569418 s, 184 MB/s 2024-04-25T10:00:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718013 s, 146 MB/s 2024-04-25T10:00:42.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.388 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721763 s, 145 MB/s 2024-04-25T10:00:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575514 s, 182 MB/s 2024-04-25T10:00:42.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570969 s, 184 MB/s 2024-04-25T10:00:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643202 s, 163 MB/s 2024-04-25T10:00:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.416 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582076 s, 180 MB/s 2024-04-25T10:00:42.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.423 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652337 s, 161 MB/s 2024-04-25T10:00:42.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.430 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654893 s, 160 MB/s 2024-04-25T10:00:42.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.438 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647034 s, 162 MB/s 2024-04-25T10:00:42.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064082 s, 164 MB/s 2024-04-25T10:00:42.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.452 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610139 s, 172 MB/s 2024-04-25T10:00:42.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.459 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624096 s, 168 MB/s 2024-04-25T10:00:42.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070699 s, 148 MB/s 2024-04-25T10:00:42.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062003 s, 169 MB/s 2024-04-25T10:00:42.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.480 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577233 s, 182 MB/s 2024-04-25T10:00:42.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568082 s, 185 MB/s 2024-04-25T10:00:42.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.494 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561813 s, 187 MB/s 2024-04-25T10:00:42.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.501 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637713 s, 164 MB/s 2024-04-25T10:00:42.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:42.502 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:42.502 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:42.502 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:00:42.502 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:42.502 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:42.503 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:42.503 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665488 s, 158 MB/s 2024-04-25T10:00:42.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057346 s, 183 MB/s 2024-04-25T10:00:42.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.579 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548922 s, 191 MB/s 2024-04-25T10:00:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.587 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580929 s, 180 MB/s 2024-04-25T10:00:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626372 s, 167 MB/s 2024-04-25T10:00:42.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.601 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632411 s, 166 MB/s 2024-04-25T10:00:42.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724063 s, 145 MB/s 2024-04-25T10:00:42.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.616 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697693 s, 150 MB/s 2024-04-25T10:00:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.623 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632211 s, 166 MB/s 2024-04-25T10:00:42.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.630 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551461 s, 190 MB/s 2024-04-25T10:00:42.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.637 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065164 s, 161 MB/s 2024-04-25T10:00:42.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074317 s, 141 MB/s 2024-04-25T10:00:42.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644291 s, 163 MB/s 2024-04-25T10:00:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620694 s, 169 MB/s 2024-04-25T10:00:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.667 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650821 s, 161 MB/s 2024-04-25T10:00:42.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.674 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614136 s, 171 MB/s 2024-04-25T10:00:42.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686031 s, 153 MB/s 2024-04-25T10:00:42.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.688 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542063 s, 193 MB/s 2024-04-25T10:00:42.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.696 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686949 s, 153 MB/s 2024-04-25T10:00:42.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.703 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586649 s, 179 MB/s 2024-04-25T10:00:42.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.711 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672368 s, 156 MB/s 2024-04-25T10:00:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.717 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570263 s, 184 MB/s 2024-04-25T10:00:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.725 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643164 s, 163 MB/s 2024-04-25T10:00:42.731 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.731 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.731 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557256 s, 188 MB/s 2024-04-25T10:00:42.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.738 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575185 s, 182 MB/s 2024-04-25T10:00:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654402 s, 160 MB/s 2024-04-25T10:00:42.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.753 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701027 s, 150 MB/s 2024-04-25T10:00:42.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.760 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624731 s, 168 MB/s 2024-04-25T10:00:42.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.767 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627311 s, 167 MB/s 2024-04-25T10:00:42.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061123 s, 172 MB/s 2024-04-25T10:00:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.781 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591956 s, 177 MB/s 2024-04-25T10:00:42.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.788 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620225 s, 169 MB/s 2024-04-25T10:00:42.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:42.789 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:42.789 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:42.790 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:00:42.790 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:42.790 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:42.790 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:42.790 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:42.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655133 s, 160 MB/s 2024-04-25T10:00:42.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572771 s, 183 MB/s 2024-04-25T10:00:42.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706932 s, 148 MB/s 2024-04-25T10:00:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654431 s, 160 MB/s 2024-04-25T10:00:42.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557642 s, 188 MB/s 2024-04-25T10:00:42.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585118 s, 179 MB/s 2024-04-25T10:00:42.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657375 s, 160 MB/s 2024-04-25T10:00:42.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629492 s, 167 MB/s 2024-04-25T10:00:42.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709426 s, 148 MB/s 2024-04-25T10:00:42.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.923 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985274 s, 106 MB/s 2024-04-25T10:00:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.929 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563439 s, 186 MB/s 2024-04-25T10:00:42.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.938 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644962 s, 163 MB/s 2024-04-25T10:00:42.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.944 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641934 s, 163 MB/s 2024-04-25T10:00:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.951 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601392 s, 174 MB/s 2024-04-25T10:00:42.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.960 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723129 s, 145 MB/s 2024-04-25T10:00:42.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.967 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665756 s, 158 MB/s 2024-04-25T10:00:42.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709884 s, 148 MB/s 2024-04-25T10:00:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639991 s, 164 MB/s 2024-04-25T10:00:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.989 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064426 s, 163 MB/s 2024-04-25T10:00:42.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:42.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:42.996 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545753 s, 192 MB/s 2024-04-25T10:00:43.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.004 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705435 s, 149 MB/s 2024-04-25T10:00:43.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.011 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657066 s, 160 MB/s 2024-04-25T10:00:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.018 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617301 s, 170 MB/s 2024-04-25T10:00:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.025 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.026 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636087 s, 165 MB/s 2024-04-25T10:00:43.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740546 s, 142 MB/s 2024-04-25T10:00:43.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.042 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691694 s, 152 MB/s 2024-04-25T10:00:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702738 s, 149 MB/s 2024-04-25T10:00:43.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576375 s, 182 MB/s 2024-04-25T10:00:43.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.063 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617356 s, 170 MB/s 2024-04-25T10:00:43.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.071 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714578 s, 147 MB/s 2024-04-25T10:00:43.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583954 s, 180 MB/s 2024-04-25T10:00:43.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668157 s, 157 MB/s 2024-04-25T10:00:43.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:43.087 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:43.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.153 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712004 s, 147 MB/s 2024-04-25T10:00:43.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566738 s, 185 MB/s 2024-04-25T10:00:43.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674741 s, 155 MB/s 2024-04-25T10:00:43.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640373 s, 164 MB/s 2024-04-25T10:00:43.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.182 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705879 s, 149 MB/s 2024-04-25T10:00:43.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743024 s, 141 MB/s 2024-04-25T10:00:43.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704776 s, 149 MB/s 2024-04-25T10:00:43.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.205 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620112 s, 169 MB/s 2024-04-25T10:00:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.213 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679026 s, 154 MB/s 2024-04-25T10:00:43.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.220 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628782 s, 167 MB/s 2024-04-25T10:00:43.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.227 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630667 s, 166 MB/s 2024-04-25T10:00:43.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708517 s, 148 MB/s 2024-04-25T10:00:43.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631247 s, 166 MB/s 2024-04-25T10:00:43.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.250 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705074 s, 149 MB/s 2024-04-25T10:00:43.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.258 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074178 s, 141 MB/s 2024-04-25T10:00:43.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.266 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063836 s, 164 MB/s 2024-04-25T10:00:43.272 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.272 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.273 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606068 s, 173 MB/s 2024-04-25T10:00:43.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.281 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697594 s, 150 MB/s 2024-04-25T10:00:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.288 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607144 s, 173 MB/s 2024-04-25T10:00:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553219 s, 190 MB/s 2024-04-25T10:00:43.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.300 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547142 s, 192 MB/s 2024-04-25T10:00:43.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.308 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650618 s, 161 MB/s 2024-04-25T10:00:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577753 s, 181 MB/s 2024-04-25T10:00:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.321 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559098 s, 188 MB/s 2024-04-25T10:00:43.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557617 s, 188 MB/s 2024-04-25T10:00:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.334 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054118 s, 194 MB/s 2024-04-25T10:00:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.342 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726562 s, 144 MB/s 2024-04-25T10:00:43.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.350 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073476 s, 143 MB/s 2024-04-25T10:00:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618995 s, 169 MB/s 2024-04-25T10:00:43.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.365 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623154 s, 168 MB/s 2024-04-25T10:00:43.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.373 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069882 s, 150 MB/s 2024-04-25T10:00:43.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700518 s, 150 MB/s 2024-04-25T10:00:43.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.381 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.381 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:43.381 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:00:43.382 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.382 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.382 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:43.382 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:43.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718977 s, 146 MB/s 2024-04-25T10:00:43.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573279 s, 183 MB/s 2024-04-25T10:00:43.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723617 s, 145 MB/s 2024-04-25T10:00:43.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628382 s, 167 MB/s 2024-04-25T10:00:43.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600544 s, 175 MB/s 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.483 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610562 s, 172 MB/s 2024-04-25T10:00:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727645 s, 144 MB/s 2024-04-25T10:00:43.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.499 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006218 s, 169 MB/s 2024-04-25T10:00:43.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705696 s, 149 MB/s 2024-04-25T10:00:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677392 s, 155 MB/s 2024-04-25T10:00:43.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.522 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071256 s, 147 MB/s 2024-04-25T10:00:43.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.530 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696568 s, 151 MB/s 2024-04-25T10:00:43.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.536 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570755 s, 184 MB/s 2024-04-25T10:00:43.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650164 s, 161 MB/s 2024-04-25T10:00:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572235 s, 183 MB/s 2024-04-25T10:00:43.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555538 s, 189 MB/s 2024-04-25T10:00:43.563 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584334 s, 179 MB/s 2024-04-25T10:00:43.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710678 s, 148 MB/s 2024-04-25T10:00:43.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.580 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698609 s, 150 MB/s 2024-04-25T10:00:43.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.586 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575035 s, 182 MB/s 2024-04-25T10:00:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.594 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064188 s, 163 MB/s 2024-04-25T10:00:43.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.602 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719283 s, 146 MB/s 2024-04-25T10:00:43.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623535 s, 168 MB/s 2024-04-25T10:00:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716895 s, 146 MB/s 2024-04-25T10:00:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588048 s, 178 MB/s 2024-04-25T10:00:43.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.631 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617962 s, 170 MB/s 2024-04-25T10:00:43.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.638 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698801 s, 150 MB/s 2024-04-25T10:00:43.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561109 s, 187 MB/s 2024-04-25T10:00:43.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638952 s, 164 MB/s 2024-04-25T10:00:43.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663129 s, 158 MB/s 2024-04-25T10:00:43.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.668 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723822 s, 145 MB/s 2024-04-25T10:00:43.675 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.675 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.675 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645203 s, 163 MB/s 2024-04-25T10:00:43.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.676 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.676 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:43.676 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:00:43.677 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.677 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.677 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:43.677 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:43.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.742 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623097 s, 168 MB/s 2024-04-25T10:00:43.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.747 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554959 s, 189 MB/s 2024-04-25T10:00:43.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.754 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056108 s, 187 MB/s 2024-04-25T10:00:43.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.763 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711367 s, 147 MB/s 2024-04-25T10:00:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056441 s, 186 MB/s 2024-04-25T10:00:43.776 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.776 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.776 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640792 s, 164 MB/s 2024-04-25T10:00:43.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.783 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635542 s, 165 MB/s 2024-04-25T10:00:43.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.789 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548558 s, 191 MB/s 2024-04-25T10:00:43.796 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.796 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.796 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560983 s, 187 MB/s 2024-04-25T10:00:43.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.803 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579426 s, 181 MB/s 2024-04-25T10:00:43.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065461 s, 160 MB/s 2024-04-25T10:00:43.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.816 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00543027 s, 193 MB/s 2024-04-25T10:00:43.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.823 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559821 s, 187 MB/s 2024-04-25T10:00:43.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.831 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069688 s, 150 MB/s 2024-04-25T10:00:43.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631698 s, 166 MB/s 2024-04-25T10:00:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563384 s, 186 MB/s 2024-04-25T10:00:43.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065639 s, 160 MB/s 2024-04-25T10:00:43.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629046 s, 167 MB/s 2024-04-25T10:00:43.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.866 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562246 s, 186 MB/s 2024-04-25T10:00:43.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.872 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558012 s, 188 MB/s 2024-04-25T10:00:43.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.880 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709854 s, 148 MB/s 2024-04-25T10:00:43.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.887 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638801 s, 164 MB/s 2024-04-25T10:00:43.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.894 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580896 s, 181 MB/s 2024-04-25T10:00:43.901 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.901 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.901 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625945 s, 168 MB/s 2024-04-25T10:00:43.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.908 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.908 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563705 s, 186 MB/s 2024-04-25T10:00:43.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.916 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734337 s, 143 MB/s 2024-04-25T10:00:43.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.923 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564029 s, 186 MB/s 2024-04-25T10:00:43.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.930 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617948 s, 170 MB/s 2024-04-25T10:00:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.937 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624328 s, 168 MB/s 2024-04-25T10:00:43.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.945 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718949 s, 146 MB/s 2024-04-25T10:00:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644028 s, 163 MB/s 2024-04-25T10:00:43.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:43.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:43.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582693 s, 180 MB/s 2024-04-25T10:00:43.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:43.960 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:43.960 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:43.960 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:00:43.960 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:43.961 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:43.961 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:43.961 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:44.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.024 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.024 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577126 s, 182 MB/s 2024-04-25T10:00:44.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.030 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575696 s, 182 MB/s 2024-04-25T10:00:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564087 s, 186 MB/s 2024-04-25T10:00:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720149 s, 146 MB/s 2024-04-25T10:00:44.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057207 s, 183 MB/s 2024-04-25T10:00:44.061 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.061 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.062 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631063 s, 166 MB/s 2024-04-25T10:00:44.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640037 s, 164 MB/s 2024-04-25T10:00:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605749 s, 173 MB/s 2024-04-25T10:00:44.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554613 s, 189 MB/s 2024-04-25T10:00:44.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547073 s, 192 MB/s 2024-04-25T10:00:44.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.093 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650523 s, 161 MB/s 2024-04-25T10:00:44.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611829 s, 171 MB/s 2024-04-25T10:00:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670905 s, 156 MB/s 2024-04-25T10:00:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.116 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070294 s, 149 MB/s 2024-04-25T10:00:44.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611765 s, 171 MB/s 2024-04-25T10:00:44.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.130 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577202 s, 182 MB/s 2024-04-25T10:00:44.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573432 s, 183 MB/s 2024-04-25T10:00:44.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.144 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617331 s, 170 MB/s 2024-04-25T10:00:44.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548371 s, 191 MB/s 2024-04-25T10:00:44.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.157 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647906 s, 162 MB/s 2024-04-25T10:00:44.164 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.164 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.164 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056192 s, 187 MB/s 2024-04-25T10:00:44.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735169 s, 143 MB/s 2024-04-25T10:00:44.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.180 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690159 s, 152 MB/s 2024-04-25T10:00:44.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.187 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063788 s, 164 MB/s 2024-04-25T10:00:44.194 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.195 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064872 s, 162 MB/s 2024-04-25T10:00:44.202 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.202 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.202 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634132 s, 165 MB/s 2024-04-25T10:00:44.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.209 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590435 s, 178 MB/s 2024-04-25T10:00:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.216 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611508 s, 171 MB/s 2024-04-25T10:00:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.223 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604243 s, 174 MB/s 2024-04-25T10:00:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.230 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621511 s, 169 MB/s 2024-04-25T10:00:44.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571873 s, 183 MB/s 2024-04-25T10:00:44.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.243 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560728 s, 187 MB/s 2024-04-25T10:00:44.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:44.244 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:44.244 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:44.244 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:00:44.244 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:44.244 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:44.245 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:44.245 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674565 s, 155 MB/s 2024-04-25T10:00:44.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.317 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703282 s, 149 MB/s 2024-04-25T10:00:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.324 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694328 s, 151 MB/s 2024-04-25T10:00:44.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.331 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575653 s, 182 MB/s 2024-04-25T10:00:44.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689318 s, 152 MB/s 2024-04-25T10:00:44.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584524 s, 179 MB/s 2024-04-25T10:00:44.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.354 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715367 s, 147 MB/s 2024-04-25T10:00:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687033 s, 153 MB/s 2024-04-25T10:00:44.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.368 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632078 s, 166 MB/s 2024-04-25T10:00:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.377 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771305 s, 136 MB/s 2024-04-25T10:00:44.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.385 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652862 s, 161 MB/s 2024-04-25T10:00:44.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716477 s, 146 MB/s 2024-04-25T10:00:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.399 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576897 s, 182 MB/s 2024-04-25T10:00:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.407 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640043 s, 164 MB/s 2024-04-25T10:00:44.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.413 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582176 s, 180 MB/s 2024-04-25T10:00:44.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.421 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696302 s, 151 MB/s 2024-04-25T10:00:44.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062233 s, 168 MB/s 2024-04-25T10:00:44.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.435 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578452 s, 181 MB/s 2024-04-25T10:00:44.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.443 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700591 s, 150 MB/s 2024-04-25T10:00:44.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.451 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.451 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699015 s, 150 MB/s 2024-04-25T10:00:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.457 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055468 s, 189 MB/s 2024-04-25T10:00:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.464 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637713 s, 164 MB/s 2024-04-25T10:00:44.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.472 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690874 s, 152 MB/s 2024-04-25T10:00:44.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.479 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635613 s, 165 MB/s 2024-04-25T10:00:44.486 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.486 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.486 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584609 s, 179 MB/s 2024-04-25T10:00:44.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.493 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619805 s, 169 MB/s 2024-04-25T10:00:44.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.500 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626315 s, 167 MB/s 2024-04-25T10:00:44.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.507 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621664 s, 169 MB/s 2024-04-25T10:00:44.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582389 s, 180 MB/s 2024-04-25T10:00:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634845 s, 165 MB/s 2024-04-25T10:00:44.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.530 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750354 s, 140 MB/s 2024-04-25T10:00:44.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.538 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726849 s, 144 MB/s 2024-04-25T10:00:44.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:44.539 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:44.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578765 s, 181 MB/s 2024-04-25T10:00:44.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569067 s, 184 MB/s 2024-04-25T10:00:44.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662494 s, 158 MB/s 2024-04-25T10:00:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.623 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057396 s, 183 MB/s 2024-04-25T10:00:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.630 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632526 s, 166 MB/s 2024-04-25T10:00:44.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.637 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585825 s, 179 MB/s 2024-04-25T10:00:44.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070403 s, 149 MB/s 2024-04-25T10:00:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561741 s, 187 MB/s 2024-04-25T10:00:44.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632328 s, 166 MB/s 2024-04-25T10:00:44.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.667 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631693 s, 166 MB/s 2024-04-25T10:00:44.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.674 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647077 s, 162 MB/s 2024-04-25T10:00:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706299 s, 148 MB/s 2024-04-25T10:00:44.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.688 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572506 s, 183 MB/s 2024-04-25T10:00:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.696 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707464 s, 148 MB/s 2024-04-25T10:00:44.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.703 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572849 s, 183 MB/s 2024-04-25T10:00:44.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.709 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545501 s, 192 MB/s 2024-04-25T10:00:44.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.717 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642242 s, 163 MB/s 2024-04-25T10:00:44.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.723 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576691 s, 182 MB/s 2024-04-25T10:00:44.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.731 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650633 s, 161 MB/s 2024-04-25T10:00:44.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.738 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635991 s, 165 MB/s 2024-04-25T10:00:44.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631448 s, 166 MB/s 2024-04-25T10:00:44.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.753 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071587 s, 146 MB/s 2024-04-25T10:00:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.761 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717803 s, 146 MB/s 2024-04-25T10:00:44.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549607 s, 191 MB/s 2024-04-25T10:00:44.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580555 s, 181 MB/s 2024-04-25T10:00:44.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.781 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617517 s, 170 MB/s 2024-04-25T10:00:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.790 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756509 s, 139 MB/s 2024-04-25T10:00:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.797 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586514 s, 179 MB/s 2024-04-25T10:00:44.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.803 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583722 s, 180 MB/s 2024-04-25T10:00:44.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.811 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712257 s, 147 MB/s 2024-04-25T10:00:44.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584996 s, 179 MB/s 2024-04-25T10:00:44.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557242 s, 188 MB/s 2024-04-25T10:00:44.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:44.826 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:44.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559708 s, 187 MB/s 2024-04-25T10:00:44.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749384 s, 140 MB/s 2024-04-25T10:00:44.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567303 s, 185 MB/s 2024-04-25T10:00:44.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070969 s, 148 MB/s 2024-04-25T10:00:44.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734501 s, 143 MB/s 2024-04-25T10:00:44.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.929 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702891 s, 149 MB/s 2024-04-25T10:00:44.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.935 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595202 s, 176 MB/s 2024-04-25T10:00:44.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614857 s, 171 MB/s 2024-04-25T10:00:44.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659526 s, 159 MB/s 2024-04-25T10:00:44.957 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.957 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.957 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632158 s, 166 MB/s 2024-04-25T10:00:44.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.965 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715864 s, 146 MB/s 2024-04-25T10:00:44.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.972 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559572 s, 187 MB/s 2024-04-25T10:00:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626711 s, 167 MB/s 2024-04-25T10:00:44.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.986 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595785 s, 176 MB/s 2024-04-25T10:00:44.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:44.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:44.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674575 s, 155 MB/s 2024-04-25T10:00:45.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728535 s, 144 MB/s 2024-04-25T10:00:45.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.009 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071199 s, 147 MB/s 2024-04-25T10:00:45.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586463 s, 179 MB/s 2024-04-25T10:00:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568524 s, 184 MB/s 2024-04-25T10:00:45.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.029 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542426 s, 193 MB/s 2024-04-25T10:00:45.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689823 s, 152 MB/s 2024-04-25T10:00:45.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737803 s, 142 MB/s 2024-04-25T10:00:45.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555049 s, 189 MB/s 2024-04-25T10:00:45.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.060 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723995 s, 145 MB/s 2024-04-25T10:00:45.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559525 s, 187 MB/s 2024-04-25T10:00:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.074 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711328 s, 147 MB/s 2024-04-25T10:00:45.081 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.081 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.081 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634494 s, 165 MB/s 2024-04-25T10:00:45.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072413 s, 145 MB/s 2024-04-25T10:00:45.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.097 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006271 s, 167 MB/s 2024-04-25T10:00:45.105 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.105 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.105 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694724 s, 151 MB/s 2024-04-25T10:00:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.112 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613918 s, 171 MB/s 2024-04-25T10:00:45.119 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.120 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645213 s, 163 MB/s 2024-04-25T10:00:45.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:00:45.121 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:00:45.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.185 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.185 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590844 s, 177 MB/s 2024-04-25T10:00:45.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.193 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704782 s, 149 MB/s 2024-04-25T10:00:45.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.201 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647322 s, 162 MB/s 2024-04-25T10:00:45.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.207 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654782 s, 160 MB/s 2024-04-25T10:00:45.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572003 s, 183 MB/s 2024-04-25T10:00:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.221 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642456 s, 163 MB/s 2024-04-25T10:00:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.228 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550633 s, 190 MB/s 2024-04-25T10:00:45.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633953 s, 165 MB/s 2024-04-25T10:00:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568096 s, 185 MB/s 2024-04-25T10:00:45.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606302 s, 173 MB/s 2024-04-25T10:00:45.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.256 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006213 s, 169 MB/s 2024-04-25T10:00:45.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623258 s, 168 MB/s 2024-04-25T10:00:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620352 s, 169 MB/s 2024-04-25T10:00:45.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709946 s, 148 MB/s 2024-04-25T10:00:45.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.285 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615572 s, 170 MB/s 2024-04-25T10:00:45.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645963 s, 162 MB/s 2024-04-25T10:00:45.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.300 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718419 s, 146 MB/s 2024-04-25T10:00:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588912 s, 178 MB/s 2024-04-25T10:00:45.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594578 s, 176 MB/s 2024-04-25T10:00:45.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734786 s, 143 MB/s 2024-04-25T10:00:45.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.329 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635379 s, 165 MB/s 2024-04-25T10:00:45.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593671 s, 177 MB/s 2024-04-25T10:00:45.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649865 s, 161 MB/s 2024-04-25T10:00:45.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.350 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576252 s, 182 MB/s 2024-04-25T10:00:45.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064257 s, 163 MB/s 2024-04-25T10:00:45.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734278 s, 143 MB/s 2024-04-25T10:00:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547493 s, 192 MB/s 2024-04-25T10:00:45.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646835 s, 162 MB/s 2024-04-25T10:00:45.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.388 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777095 s, 135 MB/s 2024-04-25T10:00:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.396 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703 s, 149 MB/s 2024-04-25T10:00:45.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.403 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564609 s, 186 MB/s 2024-04-25T10:00:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:00:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:00:45.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569614 s, 184 MB/s 2024-04-25T10:00:45.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.410 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:00:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.546+0000 7fa23b64a640 1 Processor -- start 2024-04-25T10:00:45.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.546+0000 7fa23b64a640 1 -- start start 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa23b64a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2340d0a00 0x7fa234162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa23b64a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340d0de0 0x7fa2340cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa23b64a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2340cd540 0x7fa2340caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa23b64a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23415e3a0 con 0x7fa2340d0a00 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa23b64a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340cd9d0 con 0x7fa2340cd540 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa23b64a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340cdc70 con 0x7fa2340d0de0 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa2393e6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2340d0a00 0x7fa234162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa2393e6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa2340d0a00 0x7fa234162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60064/0 (socket says 172.21.15.148:60064) 2024-04-25T10:00:45.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.547+0000 7fa2393e6640 1 -- 172.21.15.148:0/1288378405 learned_addr learned my addr 172.21.15.148:0/1288378405 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa239be7640 1 --2- 172.21.15.148:0/1288378405 >> v2:172.21.15.188:3300/0 conn(0x7fa2340cd540 0x7fa2340caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa238be5640 1 --2- 172.21.15.148:0/1288378405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340d0de0 0x7fa2340cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa2393e6640 1 -- 172.21.15.148:0/1288378405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340d0de0 msgr2=0x7fa2340cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa2393e6640 1 --2- 172.21.15.148:0/1288378405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340d0de0 0x7fa2340cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa2393e6640 1 -- 172.21.15.148:0/1288378405 >> v2:172.21.15.188:3300/0 conn(0x7fa2340cd540 msgr2=0x7fa2340caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa2393e6640 1 --2- 172.21.15.148:0/1288378405 >> v2:172.21.15.188:3300/0 conn(0x7fa2340cd540 0x7fa2340caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa2393e6640 1 -- 172.21.15.148:0/1288378405 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340cdf50 con 0x7fa2340d0a00 2024-04-25T10:00:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.548+0000 7fa2393e6640 1 --2- 172.21.15.148:0/1288378405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0a00 0x7fa234162140 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa2340731b0 tx=0x7fa23002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec9554d6cdab8ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa22bfff640 1 -- 172.21.15.148:0/1288378405 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230036070 con 0x7fa2340d0a00 2024-04-25T10:00:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa22bfff640 1 -- 172.21.15.148:0/1288378405 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa230031600 con 0x7fa2340d0a00 2024-04-25T10:00:45.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa22bfff640 1 -- 172.21.15.148:0/1288378405 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2300319d0 con 0x7fa2340d0a00 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 -- 172.21.15.148:0/1288378405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0a00 msgr2=0x7fa234162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 --2- 172.21.15.148:0/1288378405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0a00 0x7fa234162140 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa2340731b0 tx=0x7fa23002a610 comp rx=0 tx=0).stop 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 -- 172.21.15.148:0/1288378405 shutdown_connections 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 --2- 172.21.15.148:0/1288378405 >> v2:172.21.15.188:3300/0 conn(0x7fa2340cd540 0x7fa2340caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 --2- 172.21.15.148:0/1288378405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340d0de0 0x7fa2340cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 --2- 172.21.15.148:0/1288378405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0a00 0x7fa234162140 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 -- 172.21.15.148:0/1288378405 >> 172.21.15.148:0/1288378405 conn(0x7fa2340d17c0 msgr2=0x7fa2340d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 -- 172.21.15.148:0/1288378405 shutdown_connections 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.549+0000 7fa23b64a640 1 -- 172.21.15.148:0/1288378405 wait complete. 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.550+0000 7fa23b64a640 1 Processor -- start 2024-04-25T10:00:45.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.550+0000 7fa23b64a640 1 -- start start 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa23b64a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340cd540 0x7fa2340a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa23b64a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340d0a00 0x7fa2340b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa23b64a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 0x7fa2341f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa23b64a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234161c20 con 0x7fa2340d0de0 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa23b64a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa234161940 con 0x7fa2340d0a00 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa23b64a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340d1260 con 0x7fa2340cd540 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa239be7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 0x7fa2341f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa239be7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 0x7fa2341f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60068/0 (socket says 172.21.15.148:60068) 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa239be7640 1 -- 172.21.15.148:0/2187641835 learned_addr learned my addr 172.21.15.148:0/2187641835 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa238be5640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340d0a00 0x7fa2340b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.551+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340cd540 0x7fa2340a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa239be7640 1 -- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340cd540 msgr2=0x7fa2340a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa239be7640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340cd540 0x7fa2340a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa239be7640 1 -- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340d0a00 msgr2=0x7fa2340b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa239be7640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340d0a00 0x7fa2340b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa239be7640 1 -- 172.21.15.148:0/2187641835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340a8ac0 con 0x7fa2340d0de0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa239be7640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 0x7fa2341f7110 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00d800 tx=0x7fa22c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa229ffb640 1 -- 172.21.15.148:0/2187641835 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00b6c0 con 0x7fa2340d0de0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa229ffb640 1 -- 172.21.15.148:0/2187641835 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c019070 con 0x7fa2340d0de0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2340b6560 con 0x7fa2340d0de0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa229ffb640 1 -- 172.21.15.148:0/2187641835 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00bac0 con 0x7fa2340d0de0 2024-04-25T10:00:45.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.552+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2340b6b20 con 0x7fa2340d0de0 2024-04-25T10:00:45.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.554+0000 7fa229ffb640 1 -- 172.21.15.148:0/2187641835 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa22c00bc60 con 0x7fa2340d0de0 2024-04-25T10:00:45.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.554+0000 7fa229ffb640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21806b4d0 0x7fa21806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.554+0000 7fa229ffb640 1 -- 172.21.15.148:0/2187641835 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa22c08a640 con 0x7fa2340d0de0 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.554+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21806b4d0 0x7fa21806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.554+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21806b4d0 0x7fa21806d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa230033000 tx=0x7fa230009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.555+0000 7fa23e423740 1 Processor -- start 2024-04-25T10:00:45.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.555+0000 7fa23e423740 1 -- start start 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa3306360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa23e423740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa332ecc0 0x55efa332f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa23e423740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efa3268000 con 0x55efa33408d0 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efa3210f20 con 0x55efa3304200 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efa32640a0 con 0x55efa332ecc0 2024-04-25T10:00:45.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60070/0 (socket says 172.21.15.148:60070) 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 -- 172.21.15.148:0/146174301 learned_addr learned my addr 172.21.15.148:0/146174301 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa2393e6640 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa3306360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa239be7640 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa332ecc0 0x55efa332f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 -- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa332ecc0 msgr2=0x55efa332f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa332ecc0 0x55efa332f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 -- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 msgr2=0x55efa3306360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa3306360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 -- 172.21.15.148:0/146174301 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efa3048ba0 con 0x55efa33408d0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa238be5640 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33045e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa224007840 tx=0x7fa224007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdac06187b1562f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.556+0000 7fa21effd640 1 -- 172.21.15.148:0/146174301 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224018070 con 0x55efa33408d0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa21effd640 1 -- 172.21.15.148:0/146174301 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240109e0 con 0x55efa33408d0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa21effd640 1 -- 172.21.15.148:0/146174301 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224010dd0 con 0x55efa33408d0 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 -- 172.21.15.148:0/146174301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 msgr2=0x55efa33045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33045e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa224007840 tx=0x7fa224007d00 comp rx=0 tx=0).stop 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 -- 172.21.15.148:0/146174301 shutdown_connections 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa332ecc0 0x55efa332f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33045e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 --2- 172.21.15.148:0/146174301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa3306360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 -- 172.21.15.148:0/146174301 >> 172.21.15.148:0/146174301 conn(0x55efa32cc050 msgr2=0x55efa3349cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 -- 172.21.15.148:0/146174301 shutdown_connections 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.557+0000 7fa23e423740 1 -- 172.21.15.148:0/146174301 wait complete. 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.558+0000 7fa23e423740 1 -- start start 2024-04-25T10:00:45.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.558+0000 7fa23e423740 1 Processor -- start 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa331b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa332fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa33408d0 0x55efa332b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efa3206610 con 0x55efa332ecc0 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efa332c810 con 0x55efa3304200 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efa331ba60 con 0x55efa33408d0 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 10 client.0 _ll_register_callbacks cb 0x55efa321ef20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa332fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa332fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60078/0 (socket says 172.21.15.148:60078) 2024-04-25T10:00:45.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 -- 172.21.15.148:0/2154550746 learned_addr learned my addr 172.21.15.148:0/2154550746 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa331b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa239be7640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa33408d0 0x55efa332b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 -- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa33408d0 msgr2=0x55efa332b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa33408d0 0x55efa332b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 -- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 msgr2=0x55efa331b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa331b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 -- 172.21.15.148:0/2154550746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efa2f3c490 con 0x55efa332ecc0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa238be5640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa332fb50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa224010620 tx=0x7fa224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f398f18aadd37d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa213fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa213fff640 1 -- 172.21.15.148:0/2154550746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224026060 con 0x55efa332ecc0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa213fff640 1 -- 172.21.15.148:0/2154550746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa224004230 con 0x55efa332ecc0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.559+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2240109e0 con 0x55efa332ecc0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa213fff640 1 -- 172.21.15.148:0/2154550746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa22402c050 con 0x55efa332ecc0 2024-04-25T10:00:45.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 10 client.7887 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55efa30a9330 con 0x55efa332ecc0 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa213fff640 1 -- 172.21.15.148:0/2154550746 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240109e0 con 0x55efa332ecc0 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa213fff640 1 -- 172.21.15.148:0/2154550746 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa22402f020 con 0x55efa332ecc0 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 10 client.7887 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 10 client.7887 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 20 client.7887 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa335d100 0x55efa335d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 4 client.7887 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efa3275540 con 0x55efa335d100 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa335d100 0x55efa335d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa213fff640 10 client.7887 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.560+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa335d100 0x55efa335d520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b95c519bd27b5e0 server_cookie=e3f8f8ba4d43117 in_seq=0 out_seq=0 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa213fff640 1 -- 172.21.15.148:0/2154550746 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efa3275540 con 0x55efa335d100 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa213fff640 10 client.7887 handle_command_reply: tid=0 2024-04-25T10:00:45.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 client.7887 shutdown 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 msgr2=0x55efa332fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa332fb50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa224010620 tx=0x7fa224004730 comp rx=0 tx=0).stop 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 shutdown_connections 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa335d100 0x55efa335d520 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa33408d0 0x55efa332b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa213fff640 0 client.7887 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa332fb50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 --2- 172.21.15.148:0/2154550746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa3304200 0x55efa331b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 >> 172.21.15.148:0/2154550746 conn(0x55efa32cc050 msgr2=0x55efa332cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 shutdown_connections 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 1 -- 172.21.15.148:0/2154550746 wait complete. 2024-04-25T10:00:45.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.561+0000 7fa23e423740 20 client.7887 trim_cache size 0 max 16384 2024-04-25T10:00:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.576+0000 7fa23e423740 1 Processor -- start 2024-04-25T10:00:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.576+0000 7fa23e423740 1 -- start start 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa33aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa23e423740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa3363510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa33408d0 0x55efa3363a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa23e423740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2342005b0 con 0x55efa332ecc0 2024-04-25T10:00:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341ff760 con 0x55efa33408d0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa234135570 con 0x55efa3304200 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa3363510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa3363510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60082/0 (socket says 172.21.15.148:60082) 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 -- 172.21.15.148:0/2229957089 learned_addr learned my addr 172.21.15.148:0/2229957089 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa239be7640 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa33408d0 0x55efa3363a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa33aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 -- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 msgr2=0x55efa33aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa33aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 -- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa33408d0 msgr2=0x55efa3363a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa33408d0 0x55efa3363a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 -- 172.21.15.148:0/2229957089 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efa32ff8f0 con 0x55efa332ecc0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa33aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.577+0000 7fa238be5640 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa3363510 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa22401d6e0 tx=0x7fa224020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be7d8ded4ff7dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa21e7fc640 1 -- 172.21.15.148:0/2229957089 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224018040 con 0x55efa332ecc0 2024-04-25T10:00:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa21e7fc640 1 -- 172.21.15.148:0/2229957089 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22401d9a0 con 0x55efa332ecc0 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa21e7fc640 1 -- 172.21.15.148:0/2229957089 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22401dd70 con 0x55efa332ecc0 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 -- 172.21.15.148:0/2229957089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 msgr2=0x55efa3363510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa3363510 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa22401d6e0 tx=0x7fa224020040 comp rx=0 tx=0).stop 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 -- 172.21.15.148:0/2229957089 shutdown_connections 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa33408d0 0x55efa3363a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa332ecc0 0x55efa3363510 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 --2- 172.21.15.148:0/2229957089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa33aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 -- 172.21.15.148:0/2229957089 >> 172.21.15.148:0/2229957089 conn(0x55efa32cc050 msgr2=0x55efa332aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 -- 172.21.15.148:0/2229957089 shutdown_connections 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.578+0000 7fa23e423740 1 -- 172.21.15.148:0/2229957089 wait complete. 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.579+0000 7fa23e423740 1 -- start start 2024-04-25T10:00:45.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.579+0000 7fa23e423740 1 Processor -- start 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa3363c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa332ecc0 0x55efa33a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efa3330210 con 0x55efa33408d0 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2341ff260 con 0x55efa332ecc0 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efa3343110 con 0x55efa3304200 2024-04-25T10:00:45.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 10 client.0 _ll_register_callbacks cb 0x7fa224007450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60094/0 (socket says 172.21.15.148:60094) 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 -- 172.21.15.148:0/2536847032 learned_addr learned my addr 172.21.15.148:0/2536847032 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa238be5640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa332ecc0 0x55efa33a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa3363c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 -- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 msgr2=0x55efa3363c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa3363c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 -- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa332ecc0 msgr2=0x55efa33a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa332ecc0 0x55efa33a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 -- 172.21.15.148:0/2536847032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efa2f15610 con 0x55efa33408d0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa239be7640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33582c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa22c060e10 tx=0x7fa22c055010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871521705c3a507a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa213fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa213fff640 1 -- 172.21.15.148:0/2536847032 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c061340 con 0x55efa33408d0 2024-04-25T10:00:45.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa213fff640 1 -- 172.21.15.148:0/2536847032 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa22c055350 con 0x55efa33408d0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa22401d9a0 con 0x55efa33408d0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa213fff640 1 -- 172.21.15.148:0/2536847032 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa22c058130 con 0x55efa33408d0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 10 client.7893 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.580+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55efa332e190 con 0x55efa33408d0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa213fff640 1 -- 172.21.15.148:0/2536847032 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22401d9a0 con 0x55efa33408d0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa213fff640 1 -- 172.21.15.148:0/2536847032 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa22c013070 con 0x55efa33408d0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa23e423740 10 client.7893 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa23e423740 10 client.7893 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa23e423740 20 client.7893 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa23e423740 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa33588c0 0x55efa3358cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa23e423740 4 client.7893 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55efa326f890 con 0x55efa33588c0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa33588c0 0x55efa3358cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa213fff640 10 client.7893 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.581+0000 7fa2393e6640 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa33588c0 0x55efa3358cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6504a65314763cdb server_cookie=702adb6e5aaeb4be in_seq=0 out_seq=0 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa213fff640 1 -- 172.21.15.148:0/2536847032 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55efa326f890 con 0x55efa33588c0 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa213fff640 10 client.7893 handle_command_reply: tid=0 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 client.7893 shutdown 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 msgr2=0x55efa33582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33582c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa22c060e10 tx=0x7fa22c055010 comp rx=0 tx=0).stop 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 shutdown_connections 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55efa33588c0 0x55efa3358cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55efa33408d0 0x55efa33582c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efa332ecc0 0x55efa33a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa213fff640 0 client.7893 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 --2- 172.21.15.148:0/2536847032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55efa3304200 0x55efa3363c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 >> 172.21.15.148:0/2536847032 conn(0x55efa32cc050 msgr2=0x55efa3363510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 shutdown_connections 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 1 -- 172.21.15.148:0/2536847032 wait complete. 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.585+0000 7fa23e423740 20 client.7893 trim_cache size 0 max 16384 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "a2449663-3825-45a8-9e47-9b0f64a78bd9", 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:00:45.593 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21806b4d0 msgr2=0x7fa21806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21806b4d0 0x7fa21806d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa230033000 tx=0x7fa230009000 comp rx=0 tx=0).stop 2024-04-25T10:00:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 msgr2=0x7fa2341f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 0x7fa2341f7110 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00d800 tx=0x7fa22c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 shutdown_connections 2024-04-25T10:00:45.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa21806b4d0 0x7fa21806d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2340d0de0 0x7fa2341f7110 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2340d0a00 0x7fa2340b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 --2- 172.21.15.148:0/2187641835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2340cd540 0x7fa2340a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 >> 172.21.15.148:0/2187641835 conn(0x7fa2340d17c0 msgr2=0x7fa234143240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 shutdown_connections 2024-04-25T10:00:45.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.586+0000 7fa23b64a640 1 -- 172.21.15.148:0/2187641835 wait complete. 2024-04-25T10:00:45.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:45.601 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T10:00:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.702+0000 7f8aed9a5640 1 Processor -- start 2024-04-25T10:00:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.702+0000 7f8aed9a5640 1 -- start start 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8aed9a5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8ae806c9e0 0x7f8ae80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8aed9a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae806cf50 con 0x7f8ae815b7d0 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8aed9a5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80ce530 con 0x7f8ae80b5620 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8aed9a5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80ce810 con 0x7f8ae806c9e0 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ae815b7d0 0x7f8ae80cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57094/0 (socket says 172.21.15.148:57094) 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/3655859652 learned_addr learned my addr 172.21.15.148:0/3655859652 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/3655859652 >> v2:172.21.15.188:3300/0 conn(0x7f8ae806c9e0 0x7f8ae80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943209103 0 0) 0x7f8ae806cf50 con 0x7f8ae815b7d0 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad4003520 con 0x7f8ae815b7d0 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843465491 0 0) 0x7f8ad4003520 con 0x7f8ae815b7d0 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 >> v2:172.21.15.188:3300/0 conn(0x7f8ae806c9e0 msgr2=0x7f8ae80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 --2- 172.21.15.148:0/3655859652 >> v2:172.21.15.188:3300/0 conn(0x7f8ae806c9e0 0x7f8ae80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 >> v1:172.21.15.188:6789/0 conn(0x7f8ae80b5620 legacy=0x7f8ae806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80ceaf0 con 0x7f8ae815b7d0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f8ad8001670 con 0x7f8ae815b7d0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 >> v1:172.21.15.148:6789/0 conn(0x7f8ae815b7d0 legacy=0x7f8ae80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.703+0000 7f8ae67fc640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad40039b0 0x7f8ad4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae67fc640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ad4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae67fc640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad4005080 0x7f8ad400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80ce810 con 0x7f8ad40043b0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400bfe0 con 0x7f8ad40039b0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400c1e0 con 0x7f8ad4005080 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8adffff640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ad4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad40039b0 0x7f8ad4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad4005080 0x7f8ad400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8adffff640 1 -- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad4005080 msgr2=0x7f8ad400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8adffff640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad4005080 0x7f8ad400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8adffff640 1 -- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad40039b0 msgr2=0x7f8ad4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8adffff640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad40039b0 0x7f8ad4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8ae77fe640 1 -- 172.21.15.148:0/3655859652 reap_dead start 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.704+0000 7f8adffff640 1 -- 172.21.15.148:0/3655859652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4003520 con 0x7f8ad40043b0 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8adffff640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ad4004880 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0001860 tx=0x7f8ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361eaf49d8924434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0037070 con 0x7f8ad40043b0 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad002abe0 con 0x7f8ad40043b0 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8ae67fc640 1 -- 172.21.15.148:0/3655859652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0001430 con 0x7f8ad40043b0 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 -- 172.21.15.148:0/3655859652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 msgr2=0x7f8ad4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ad4004880 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0001860 tx=0x7f8ad4005830 comp rx=0 tx=0).stop 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 -- 172.21.15.148:0/3655859652 shutdown_connections 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/3655859652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ad4004880 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 -- 172.21.15.148:0/3655859652 >> 172.21.15.148:0/3655859652 conn(0x7f8ae80d0800 msgr2=0x7f8ae80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 -- 172.21.15.148:0/3655859652 shutdown_connections 2024-04-25T10:00:45.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.705+0000 7f8aed9a5640 1 -- 172.21.15.148:0/3655859652 wait complete. 2024-04-25T10:00:45.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.706+0000 7f8aed9a5640 1 Processor -- start 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 -- start start 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ae8159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ae80cc6d0 0x7f8ae80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ae80cef10 0x7f8ae80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae815bd30 con 0x7f8ad40043b0 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae815bbb0 con 0x7f8ae80cc6d0 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8aed9a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80b5ab0 con 0x7f8ae80cef10 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ae8159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ae8159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60114/0 (socket says 172.21.15.148:60114) 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/4161014808 learned_addr learned my addr 172.21.15.148:0/4161014808 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.707+0000 7f8adffff640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ae80cc6d0 0x7f8ae80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ae80cef10 msgr2=0x7f8ae80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ae80cef10 0x7f8ae80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ae80cc6d0 msgr2=0x7f8ae80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ae80cc6d0 0x7f8ae80cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/4161014808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80ccb60 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8adffff640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ae80cc6d0 0x7f8ae80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ae8159c50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80072c0 tx=0x7f8ad8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae4ff9640 1 -- 172.21.15.148:0/4161014808 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800f8e0 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae4ff9640 1 -- 172.21.15.148:0/4161014808 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8018070 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae80cce40 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8ae4ff9640 1 -- 172.21.15.148:0/4161014808 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800fca0 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.708+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae80c7570 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.709+0000 7f8ae4ff9640 1 -- 172.21.15.148:0/4161014808 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ad801d460 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.710+0000 7f8ae4ff9640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ac806b4d0 0x7f8ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.710+0000 7f8ae4ff9640 1 -- 172.21.15.148:0/4161014808 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ad808a2d0 con 0x7f8ad40043b0 2024-04-25T10:00:45.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.710+0000 7f8adffff640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ac806b4d0 0x7f8ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.710+0000 7f8adffff640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ac806b4d0 0x7f8ac806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0034000 tx=0x7f8ad0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 Processor -- start 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 -- start start 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df629e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df628080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df652be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557df58ab90 con 0x5557df627ca0 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.711+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557df533e40 con 0x5557df652760 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557df586c30 con 0x5557df664370 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df629e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df629e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60118/0 (socket says 172.21.15.148:60118) 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/900280433 learned_addr learned my addr 172.21.15.148:0/900280433 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8adffff640 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df628080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df652be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 msgr2=0x5557df628080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df628080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 msgr2=0x5557df652be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df652be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/900280433 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557df3697d0 con 0x5557df627ca0 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df652be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df629e00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8088670 tx=0x7f8ad8058850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1283f7e6e201326b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8add7fa640 1 -- 172.21.15.148:0/900280433 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8061060 con 0x5557df627ca0 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8add7fa640 1 -- 172.21.15.148:0/900280433 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad80569f0 con 0x5557df627ca0 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.712+0000 7f8add7fa640 1 -- 172.21.15.148:0/900280433 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8054fc0 con 0x5557df627ca0 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 -- 172.21.15.148:0/900280433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 msgr2=0x5557df629e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df629e00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8088670 tx=0x7f8ad8058850 comp rx=0 tx=0).stop 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 -- 172.21.15.148:0/900280433 shutdown_connections 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df652be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df628080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 --2- 172.21.15.148:0/900280433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df629e00 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 -- 172.21.15.148:0/900280433 >> 172.21.15.148:0/900280433 conn(0x5557df5ef870 msgr2=0x5557df66d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 -- 172.21.15.148:0/900280433 shutdown_connections 2024-04-25T10:00:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.713+0000 7f8af0783740 1 -- 172.21.15.148:0/900280433 wait complete. 2024-04-25T10:00:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 -- start start 2024-04-25T10:00:45.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 Processor -- start 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df627ca0 0x5557df6440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df654e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 0x5557df66ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557df5473e0 con 0x5557df664370 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557df6502b0 con 0x5557df627ca0 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557df67d620 con 0x5557df652760 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8af0783740 10 client.0 _ll_register_callbacks cb 0x5557df546db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.714+0000 7f8ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 0x5557df66ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 0x5557df66ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60128/0 (socket says 172.21.15.148:60128) 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 -- 172.21.15.148:0/2430174921 learned_addr learned my addr 172.21.15.148:0/2430174921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df627ca0 0x5557df6440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8adffff640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df654e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 -- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 msgr2=0x5557df654e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df654e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 -- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df627ca0 msgr2=0x5557df6440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df627ca0 0x5557df6440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 -- 172.21.15.148:0/2430174921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557df25f0f0 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 0x5557df66ad30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806b4d0 tx=0x7f8ae0007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9e391110f44adf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ac67fc640 1 -- 172.21.15.148:0/2430174921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae001c020 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ac67fc640 1 -- 172.21.15.148:0/2430174921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ae0003da0 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad80569f0 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ac67fc640 1 -- 172.21.15.148:0/2430174921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ae0026050 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 10 client.7905 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5557df3c9f60 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ac67fc640 1 -- 172.21.15.148:0/2430174921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad80569f0 con 0x5557df664370 2024-04-25T10:00:45.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8ac67fc640 1 -- 172.21.15.148:0/2430174921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ae0029020 con 0x5557df664370 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 10 client.7905 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 10 client.7905 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 20 client.7905 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df680a90 0x5557df680ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.715+0000 7f8af0783740 4 client.7905 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.716+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557df5980d0 con 0x5557df680a90 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.716+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df680a90 0x5557df680ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.716+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df680a90 0x5557df680ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564e67499185d6df server_cookie=2039f56979564fb4 in_seq=0 out_seq=0 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.716+0000 7f8ac67fc640 10 client.7905 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8ac67fc640 1 -- 172.21.15.148:0/2430174921 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557df5980d0 con 0x5557df680a90 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8ac67fc640 10 client.7905 handle_command_reply: tid=0 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 client.7905 shutdown 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 msgr2=0x5557df66ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 0x5557df66ad30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806b4d0 tx=0x7f8ae0007dd0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 shutdown_connections 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df680a90 0x5557df680ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df664370 0x5557df66ad30 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8ac67fc640 0 client.7905 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df654e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 --2- 172.21.15.148:0/2430174921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df627ca0 0x5557df6440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 >> 172.21.15.148:0/2430174921 conn(0x5557df5ef870 msgr2=0x5557df64f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 shutdown_connections 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 1 -- 172.21.15.148:0/2430174921 wait complete. 2024-04-25T10:00:45.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.717+0000 7f8af0783740 20 client.7905 trim_cache size 0 max 16384 2024-04-25T10:00:45.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.731+0000 7f8af0783740 1 Processor -- start 2024-04-25T10:00:45.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.731+0000 7f8af0783740 1 -- start start 2024-04-25T10:00:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8af0783740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df663c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df6692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df664370 0x5557df669800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8af0783740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8200960 con 0x5557df627ca0 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81ffae0 con 0x5557df664370 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80c8eb0 con 0x5557df652760 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df663c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df663c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60130/0 (socket says 172.21.15.148:60130) 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1371946748 learned_addr learned my addr 172.21.15.148:0/1371946748 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1371946748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 msgr2=0x5557df6692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df6692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.732+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1371946748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df664370 msgr2=0x5557df669800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df664370 0x5557df669800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1371946748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557df626000 con 0x5557df627ca0 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df663c80 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80588e0 tx=0x7f8ad8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e1c3c07270dacc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8adcff9640 1 -- 172.21.15.148:0/1371946748 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8085040 con 0x5557df627ca0 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8adcff9640 1 -- 172.21.15.148:0/1371946748 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad80569f0 con 0x5557df627ca0 2024-04-25T10:00:45.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8adcff9640 1 -- 172.21.15.148:0/1371946748 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8056c50 con 0x5557df627ca0 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 -- 172.21.15.148:0/1371946748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 msgr2=0x5557df663c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df663c80 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80588e0 tx=0x7f8ad8009040 comp rx=0 tx=0).stop 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 -- 172.21.15.148:0/1371946748 shutdown_connections 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df664370 0x5557df669800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df652760 0x5557df6692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 --2- 172.21.15.148:0/1371946748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df663c80 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 -- 172.21.15.148:0/1371946748 >> 172.21.15.148:0/1371946748 conn(0x5557df5ef870 msgr2=0x5557df643ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 -- 172.21.15.148:0/1371946748 shutdown_connections 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.733+0000 7f8af0783740 1 -- 172.21.15.148:0/1371946748 wait complete. 2024-04-25T10:00:45.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.734+0000 7f8af0783740 1 -- start start 2024-04-25T10:00:45.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.734+0000 7f8af0783740 1 Processor -- start 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df6507c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df6dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df666760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81ff4e0 con 0x5557df627ca0 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557df667560 con 0x5557df652760 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557df669210 con 0x5557df664370 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 10 client.0 _ll_register_callbacks cb 0x7f8ae8062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df6507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df6507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60138/0 (socket says 172.21.15.148:60138) 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1314187137 learned_addr learned my addr 172.21.15.148:0/1314187137 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8adffff640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df6dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df666760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 msgr2=0x5557df666760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df666760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 msgr2=0x5557df6dab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df6dab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 -- 172.21.15.148:0/1314187137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557df2381b0 con 0x5557df627ca0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae77fe640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df666760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ae6ffd640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df6507c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8058650 tx=0x7f8ad805b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a42ff53c0136c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ac67fc640 1 -- 172.21.15.148:0/1314187137 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8085040 con 0x5557df627ca0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8ac67fc640 1 -- 172.21.15.148:0/1314187137 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ad8088380 con 0x5557df627ca0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.735+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad80569f0 con 0x5557df627ca0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8ac67fc640 1 -- 172.21.15.148:0/1314187137 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad807f050 con 0x5557df627ca0 2024-04-25T10:00:45.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 10 client.7911 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5557df667300 con 0x5557df627ca0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8ac67fc640 1 -- 172.21.15.148:0/1314187137 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad80569f0 con 0x5557df627ca0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8ac67fc640 1 -- 172.21.15.148:0/1314187137 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ad8054fc0 con 0x5557df627ca0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 10 client.7911 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 10 client.7911 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 20 client.7911 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df6276a0 0x5557df666ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 4 client.7911 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5557df592420 con 0x5557df6276a0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8adffff640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df6276a0 0x5557df666ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8ac67fc640 10 client.7911 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8adffff640 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df6276a0 0x5557df666ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b62181c5cd4694 server_cookie=923f54a8aea5210 in_seq=0 out_seq=0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8ac67fc640 1 -- 172.21.15.148:0/1314187137 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5557df592420 con 0x5557df6276a0 2024-04-25T10:00:45.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8ac67fc640 10 client.7911 handle_command_reply: tid=0 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.736+0000 7f8af0783740 1 client.7911 shutdown 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 msgr2=0x5557df6507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df6507c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8058650 tx=0x7f8ad805b070 comp rx=0 tx=0).stop 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 shutdown_connections 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5557df6276a0 0x5557df666ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5557df664370 0x5557df666760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557df652760 0x5557df6dab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8ac67fc640 0 client.7911 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 --2- 172.21.15.148:0/1314187137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557df627ca0 0x5557df6507c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 >> 172.21.15.148:0/1314187137 conn(0x5557df5ef870 msgr2=0x5557df690150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 shutdown_connections 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 1 -- 172.21.15.148:0/1314187137 wait complete. 2024-04-25T10:00:45.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8af0783740 20 client.7911 trim_cache size 0 max 16384 2024-04-25T10:00:45.749 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:45.749 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:00:45.749 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ac806b4d0 msgr2=0x7f8ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ac806b4d0 0x7f8ac806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0034000 tx=0x7f8ad0009030 comp rx=0 tx=0).stop 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 msgr2=0x7f8ae8159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ae8159c50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80072c0 tx=0x7f8ad8007780 comp rx=0 tx=0).stop 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 shutdown_connections 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ac806b4d0 0x7f8ac806d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ae80cef10 0x7f8ae80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ae80cc6d0 0x7f8ae80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 --2- 172.21.15.148:0/4161014808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ad40043b0 0x7f8ae8159c50 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 >> 172.21.15.148:0/4161014808 conn(0x7f8ae80d0800 msgr2=0x7f8ae80b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 shutdown_connections 2024-04-25T10:00:45.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.737+0000 7f8aed9a5640 1 -- 172.21.15.148:0/4161014808 wait complete. 2024-04-25T10:00:45.752 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:00:45.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:00:45.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.847+0000 7f2da5d42640 1 Processor -- start 2024-04-25T10:00:45.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.847+0000 7f2da5d42640 1 -- start start 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2da5d42640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2da006abe0 0x7f2da006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2da5d42640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da006b500 con 0x7f2da015dd80 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2da5d42640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00b3930 con 0x7f2da00ad570 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2da5d42640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da006e410 con 0x7f2da006abe0 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2da006abe0 0x7f2da006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2da015dd80 0x7f2da00b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57096/0 (socket says 172.21.15.148:57096) 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9effd640 1 -- 172.21.15.148:0/2364912216 learned_addr learned my addr 172.21.15.148:0/2364912216 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/2364912216 >> v1:172.21.15.188:6789/0 conn(0x7f2da00ad570 legacy=0x7f2da006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/2364912216 >> v1:172.21.15.148:6789/0 conn(0x7f2da015dd80 legacy=0x7f2da00b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9e7fc640 1 -- 172.21.15.148:0/2364912216 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741891766 0 0) 0x7f2da006b500 con 0x7f2da015dd80 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.848+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/2364912216 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00b3c10 con 0x7f2da006abe0 2024-04-25T10:00:45.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/2364912216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da006abe0 0x7f2da006afc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097a0 tx=0x7f2d9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554bd3797523abb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2d9e7fc640 1 -- 172.21.15.148:0/2364912216 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90037070 con 0x7f2da006abe0 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2d9e7fc640 1 -- 172.21.15.148:0/2364912216 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90032440 con 0x7f2da006abe0 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2d9e7fc640 1 -- 172.21.15.148:0/2364912216 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90032810 con 0x7f2da006abe0 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 -- 172.21.15.148:0/2364912216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da006abe0 msgr2=0x7f2da006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 --2- 172.21.15.148:0/2364912216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da006abe0 0x7f2da006afc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2d900097a0 tx=0x7f2d9002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 -- 172.21.15.148:0/2364912216 shutdown_connections 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 --2- 172.21.15.148:0/2364912216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da006abe0 0x7f2da006afc0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 -- 172.21.15.148:0/2364912216 >> 172.21.15.148:0/2364912216 conn(0x7f2da00b5af0 msgr2=0x7f2da00b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 -- 172.21.15.148:0/2364912216 shutdown_connections 2024-04-25T10:00:45.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.849+0000 7f2da5d42640 1 -- 172.21.15.148:0/2364912216 wait complete. 2024-04-25T10:00:45.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.850+0000 7f2da5d42640 1 Processor -- start 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 -- start start 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2da006abe0 0x7f2da009df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2da00ad570 0x7f2da00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 0x7f2da00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0070280 con 0x7f2da015dd80 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da006e940 con 0x7f2da006abe0 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2da5d42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0070400 con 0x7f2da00ad570 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.851+0000 7f2d9ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 0x7f2da00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 0x7f2da00b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60164/0 (socket says 172.21.15.148:60164) 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 -- 172.21.15.148:0/1383865866 learned_addr learned my addr 172.21.15.148:0/1383865866 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2da006abe0 0x7f2da009df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9effd640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2da00ad570 0x7f2da00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 -- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2da00ad570 msgr2=0x7f2da00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2da00ad570 0x7f2da00b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 -- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2da006abe0 msgr2=0x7f2da009df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2da006abe0 0x7f2da009df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 -- 172.21.15.148:0/1383865866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0149330 con 0x7f2da015dd80 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2da006abe0 0x7f2da009df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 0x7f2da00b27b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c780 tx=0x7f2d9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d87fff640 1 -- 172.21.15.148:0/1383865866 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98004b60 con 0x7f2da015dd80 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d87fff640 1 -- 172.21.15.148:0/1383865866 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98004d00 con 0x7f2da015dd80 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2d87fff640 1 -- 172.21.15.148:0/1383865866 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800aa90 con 0x7f2da015dd80 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da0149610 con 0x7f2da015dd80 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.852+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da0149bd0 con 0x7f2da015dd80 2024-04-25T10:00:45.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.854+0000 7f2d87fff640 1 -- 172.21.15.148:0/1383865866 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d9800ac30 con 0x7f2da015dd80 2024-04-25T10:00:45.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.854+0000 7f2d87fff640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2d8c06b4d0 0x7f2d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.854+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2d8c06b4d0 0x7f2d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.854+0000 7f2d87fff640 1 -- 172.21.15.148:0/1383865866 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d9808a690 con 0x7f2da015dd80 2024-04-25T10:00:45.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.855+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2d8c06b4d0 0x7f2d8c06d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2d90034000 tx=0x7f2d9002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.855+0000 7f2da8a4a740 1 Processor -- start 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.855+0000 7f2da8a4a740 1 -- start start 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa49320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa49860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa19770 0x55c87aa1cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87a982da0 con 0x55c87aa46ef0 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87a92bb80 con 0x55c87aa19770 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87a97ee40 con 0x55c87aa46b10 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa49860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa49860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60178/0 (socket says 172.21.15.148:60178) 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 -- 172.21.15.148:0/3461771611 learned_addr learned my addr 172.21.15.148:0/3461771611 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa19770 0x55c87aa1cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa49320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 -- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 msgr2=0x55c87aa49320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa49320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 -- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa19770 msgr2=0x55c87aa1cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa19770 0x55c87aa1cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 -- 172.21.15.148:0/3461771611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c87a935f10 con 0x55c87aa46ef0 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa19770 0x55c87aa1cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.856+0000 7f2d9effd640 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa49860 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001630 tx=0x7f2d94001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e971bde02f0e4457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2d857fa640 1 -- 172.21.15.148:0/3461771611 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94001d40 con 0x55c87aa46ef0 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2d857fa640 1 -- 172.21.15.148:0/3461771611 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d940074e0 con 0x55c87aa46ef0 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2d857fa640 1 -- 172.21.15.148:0/3461771611 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94004a00 con 0x55c87aa46ef0 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3461771611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 msgr2=0x55c87aa49860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa49860 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001630 tx=0x7f2d94001af0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3461771611 shutdown_connections 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa19770 0x55c87aa1cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa49860 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3461771611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa49320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3461771611 >> 172.21.15.148:0/3461771611 conn(0x55c87aa1b9e0 msgr2=0x55c87aa64320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3461771611 shutdown_connections 2024-04-25T10:00:45.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.857+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3461771611 wait complete. 2024-04-25T10:00:45.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.858+0000 7f2da8a4a740 1 -- start start 2024-04-25T10:00:45.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.858+0000 7f2da8a4a740 1 Processor -- start 2024-04-25T10:00:45.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa6b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46ef0 0x55c87aa689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa68f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87aa1eb90 con 0x55c87aa46b10 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87a9214b0 con 0x55c87aa6b9c0 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87aa1ee70 con 0x55c87aa46ef0 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2da8a4a740 10 client.0 _ll_register_callbacks cb 0x55c87a961180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa6b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa6b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60184/0 (socket says 172.21.15.148:60184) 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/4116296066 learned_addr learned my addr 172.21.15.148:0/4116296066 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa68f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9effd640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46ef0 0x55c87aa689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46ef0 msgr2=0x55c87aa689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46ef0 0x55c87aa689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 msgr2=0x55c87aa68f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa68f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/4116296066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c87a7617d0 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa6b480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f2d9002a7d0 tx=0x7f2d90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700c3e6e277e10f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.859+0000 7f2d7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/4116296066 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90037040 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/4116296066 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d9003e9f0 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d940074e0 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/4116296066 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d90055050 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 10 client.7926 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55c87a6973f0 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/4116296066 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d940074e0 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/4116296066 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d90058020 con 0x55c87aa46b10 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 10 client.7926 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 10 client.7926 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:45.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 20 client.7926 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa4aff0 0x55c87aa67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 4 client.7926 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c87a9902e0 con 0x55c87aa4aff0 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.860+0000 7f2d9effd640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa4aff0 0x55c87aa67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2d7e7fc640 10 client.7926 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2d9effd640 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa4aff0 0x55c87aa67cc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44d1e7c14e81092 server_cookie=bcd4d43efe246618 in_seq=0 out_seq=0 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/4116296066 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c87a9902e0 con 0x55c87aa4aff0 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2d7e7fc640 10 client.7926 handle_command_reply: tid=0 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 client.7926 shutdown 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 msgr2=0x55c87aa6b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa6b480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f2d9002a7d0 tx=0x7f2d90004730 comp rx=0 tx=0).stop 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 shutdown_connections 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa4aff0 0x55c87aa67cc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa68f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46ef0 0x55c87aa689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2d7e7fc640 0 client.7926 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/4116296066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa6b480 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 >> 172.21.15.148:0/4116296066 conn(0x55c87aa19770 msgr2=0x55c87aa1a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 shutdown_connections 2024-04-25T10:00:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 1 -- 172.21.15.148:0/4116296066 wait complete. 2024-04-25T10:00:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.861+0000 7f2da8a4a740 20 client.7926 trim_cache size 0 max 16384 2024-04-25T10:00:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.876+0000 7f2da8a4a740 1 Processor -- start 2024-04-25T10:00:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.876+0000 7f2da8a4a740 1 -- start start 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa18e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0200460 con 0x55c87aa46ef0 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01ff670 con 0x55c87aa6b9c0 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da009c180 con 0x55c87aa46b10 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa61da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60188/0 (socket says 172.21.15.148:60188) 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 -- 172.21.15.148:0/285215669 learned_addr learned my addr 172.21.15.148:0/285215669 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa18e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 -- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 msgr2=0x55c87aa18e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa18e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 -- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 msgr2=0x55c87aa622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.877+0000 7f2d9effd640 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2d9effd640 1 -- 172.21.15.148:0/285215669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c87a6570f0 con 0x55c87aa46ef0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2d9effd640 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa61da0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001580 tx=0x7f2d94006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec110002f0256a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2d84ff9640 1 -- 172.21.15.148:0/285215669 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9402c070 con 0x55c87aa46ef0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2d84ff9640 1 -- 172.21.15.148:0/285215669 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94007360 con 0x55c87aa46ef0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2d84ff9640 1 -- 172.21.15.148:0/285215669 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94027d20 con 0x55c87aa46ef0 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 -- 172.21.15.148:0/285215669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 msgr2=0x55c87aa61da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa61da0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001580 tx=0x7f2d94006000 comp rx=0 tx=0).stop 2024-04-25T10:00:45.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 -- 172.21.15.148:0/285215669 shutdown_connections 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa6b9c0 0x55c87aa622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46ef0 0x55c87aa61da0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/285215669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa46b10 0x55c87aa18e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.878+0000 7f2da8a4a740 1 -- 172.21.15.148:0/285215669 >> 172.21.15.148:0/285215669 conn(0x55c87aa19770 msgr2=0x55c87aa14b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.879+0000 7f2da8a4a740 1 -- 172.21.15.148:0/285215669 shutdown_connections 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.879+0000 7f2da8a4a740 1 -- 172.21.15.148:0/285215669 wait complete. 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.879+0000 7f2da8a4a740 1 -- start start 2024-04-25T10:00:45.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.879+0000 7f2da8a4a740 1 Processor -- start 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa76120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa46ef0 0x55c87aa45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa6b9c0 0x55c87aa1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c87aa61570 con 0x55c87aa46b10 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da009c780 con 0x55c87aa46ef0 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c87a985240 con 0x55c87aa6b9c0 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2da8a4a740 10 client.0 _ll_register_callbacks cb 0x7f2d94007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa76120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa76120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60202/0 (socket says 172.21.15.148:60202) 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/3584942740 learned_addr learned my addr 172.21.15.148:0/3584942740 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9effd640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa46ef0 0x55c87aa45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9ffff640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa6b9c0 0x55c87aa1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa6b9c0 msgr2=0x55c87aa1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa6b9c0 0x55c87aa1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa46ef0 msgr2=0x55c87aa45360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa46ef0 0x55c87aa45360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 -- 172.21.15.148:0/3584942740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c87aa68e50 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d9f7fe640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa76120 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009450 tx=0x7f2d90009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4102b5cf2dd65b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.880+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/3584942740 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9005e060 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/3584942740 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d90004450 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d94007360 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/3584942740 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d90037040 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 10 client.7932 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55c87aa45b20 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/3584942740 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94007360 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/3584942740 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d90058040 con 0x55c87aa46b10 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 10 client.7932 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 10 client.7932 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 20 client.7932 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa1a1e0 0x55c87aa35140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 4 client.7932 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c87a98a630 con 0x55c87aa1a1e0 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d9effd640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa1a1e0 0x55c87aa35140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d7e7fc640 10 client.7932 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.881+0000 7f2d9effd640 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa1a1e0 0x55c87aa35140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99605c92253c22b8 server_cookie=593d17f90eeebb1c in_seq=0 out_seq=0 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2d7e7fc640 1 -- 172.21.15.148:0/3584942740 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55c87a98a630 con 0x55c87aa1a1e0 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2d7e7fc640 10 client.7932 handle_command_reply: tid=0 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 client.7932 shutdown 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 msgr2=0x55c87aa76120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa76120 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009450 tx=0x7f2d90009190 comp rx=0 tx=0).stop 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 shutdown_connections 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x55c87aa1a1e0 0x55c87aa35140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c87aa6b9c0 0x55c87aa1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2d7e7fc640 0 client.7932 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c87aa46ef0 0x55c87aa45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 --2- 172.21.15.148:0/3584942740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c87aa46b10 0x55c87aa76120 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 >> 172.21.15.148:0/3584942740 conn(0x55c87aa19770 msgr2=0x55c87aa1e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 shutdown_connections 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 1 -- 172.21.15.148:0/3584942740 wait complete. 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da8a4a740 20 client.7932 trim_cache size 0 max 16384 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:00:45.892 INFO:teuthology.orchestra.run.smithi148.stdout: "a2449663-3825-45a8-9e47-9b0f64a78bd9": { 2024-04-25T10:00:45.893 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:00:45.893 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "a2449663-3825-45a8-9e47-9b0f64a78bd9", 2024-04-25T10:00:45.893 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-04-25T10:00:45.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:45.893 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:00:45.893 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2d8c06b4d0 msgr2=0x7f2d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2d8c06b4d0 0x7f2d8c06d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2d90034000 tx=0x7f2d9002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 msgr2=0x7f2da00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 0x7f2da00b27b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c780 tx=0x7f2d9800cc40 comp rx=0 tx=0).stop 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 shutdown_connections 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2d8c06b4d0 0x7f2d8c06d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2da015dd80 0x7f2da00b27b0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2da00ad570 0x7f2da00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 --2- 172.21.15.148:0/1383865866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2da006abe0 0x7f2da009df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:45.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.882+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 >> 172.21.15.148:0/1383865866 conn(0x7f2da00b5af0 msgr2=0x7f2da00ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:45.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.883+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 shutdown_connections 2024-04-25T10:00:45.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:45.883+0000 7f2da5d42640 1 -- 172.21.15.148:0/1383865866 wait complete. 2024-04-25T10:00:50.899 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:00:50.899 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:00:51.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.005+0000 7f7791b1b640 1 Processor -- start 2024-04-25T10:00:51.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.006+0000 7f7791b1b640 1 -- start start 2024-04-25T10:00:51.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.006+0000 7f7791b1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f778c15de80 0x7f778c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.006+0000 7f7791b1b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0ccfe0 0x7f778c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.006+0000 7f7791b1b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778c0afc80 con 0x7f778c0cefb0 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.006+0000 7f7791b1b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c0ce4d0 con 0x7f778c15de80 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.006+0000 7f7791b1b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0ce7b0 con 0x7f778c0ccfe0 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f778c15de80 0x7f778c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f778c15de80 0x7f778c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60206/0 (socket says 172.21.15.148:60206) 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 -- 172.21.15.148:0/1724754294 learned_addr learned my addr 172.21.15.148:0/1724754294 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778bfff640 1 --2- 172.21.15.148:0/1724754294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0ccfe0 0x7f778c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 -- 172.21.15.148:0/1724754294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0ccfe0 msgr2=0x7f778c0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1724754294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0ccfe0 0x7f778c0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 -- 172.21.15.148:0/1724754294 >> v1:172.21.15.188:6789/0 conn(0x7f778c0cefb0 legacy=0x7f778c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.007+0000 7f778b7fe640 1 -- 172.21.15.148:0/1724754294 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0cea90 con 0x7f778c15de80 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1724754294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15de80 0x7f778c0cadb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f777c009e50 tx=0x7f777c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bdbbd8bffb7167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f778a7fc640 1 -- 172.21.15.148:0/1724754294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c036070 con 0x7f778c15de80 2024-04-25T10:00:51.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f778a7fc640 1 -- 172.21.15.148:0/1724754294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c031600 con 0x7f778c15de80 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f778a7fc640 1 -- 172.21.15.148:0/1724754294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0319d0 con 0x7f778c15de80 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 -- 172.21.15.148:0/1724754294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15de80 msgr2=0x7f778c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1724754294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15de80 0x7f778c0cadb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f777c009e50 tx=0x7f777c02a710 comp rx=0 tx=0).stop 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 -- 172.21.15.148:0/1724754294 shutdown_connections 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1724754294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0ccfe0 0x7f778c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1724754294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15de80 0x7f778c0cadb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 -- 172.21.15.148:0/1724754294 >> 172.21.15.148:0/1724754294 conn(0x7f778c0b2390 msgr2=0x7f778c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 -- 172.21.15.148:0/1724754294 shutdown_connections 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.008+0000 7f7791b1b640 1 -- 172.21.15.148:0/1724754294 wait complete. 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.010+0000 7f7791b1b640 1 Processor -- start 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.010+0000 7f7791b1b640 1 -- start start 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.010+0000 7f7791b1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 0x7f778c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c0cefb0 0x7f778c0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c15de80 0x7f778c0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0cfb50 con 0x7f778c0ccfe0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0b2eb0 con 0x7f778c0cefb0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0c8090 con 0x7f778c15de80 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 0x7f778c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 0x7f778c0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60216/0 (socket says 172.21.15.148:60216) 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 -- 172.21.15.148:0/1580332812 learned_addr learned my addr 172.21.15.148:0/1580332812 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778affd640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c0cefb0 0x7f778c0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778bfff640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c15de80 0x7f778c0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 -- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c15de80 msgr2=0x7f778c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c15de80 0x7f778c0ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 -- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c0cefb0 msgr2=0x7f778c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c0cefb0 0x7f778c0adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 -- 172.21.15.148:0/1580332812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0c7830 con 0x7f778c0ccfe0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 0x7f778c0c9fb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f777c034040 tx=0x7f777c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7773fff640 1 -- 172.21.15.148:0/1580332812 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c03f020 con 0x7f778c0ccfe0 2024-04-25T10:00:51.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7773fff640 1 -- 172.21.15.148:0/1580332812 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c037040 con 0x7f778c0ccfe0 2024-04-25T10:00:51.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c0c7ad0 con 0x7f778c0ccfe0 2024-04-25T10:00:51.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7773fff640 1 -- 172.21.15.148:0/1580332812 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c036070 con 0x7f778c0ccfe0 2024-04-25T10:00:51.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.011+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c158320 con 0x7f778c0ccfe0 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.013+0000 7f7773fff640 1 -- 172.21.15.148:0/1580332812 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f777c042020 con 0x7f778c0ccfe0 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.013+0000 7f7773fff640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b500 0x7f777806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.013+0000 7f7773fff640 1 -- 172.21.15.148:0/1580332812 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777c0aa8c0 con 0x7f778c0ccfe0 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.013+0000 7f778affd640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b500 0x7f777806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.014+0000 7f778affd640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b500 0x7f777806d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f778c0ae990 tx=0x7f7780009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.014+0000 7f779481e740 1 Processor -- start 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.014+0000 7f779481e740 1 -- start start 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b8692320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f779481e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 0x5643b8692860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8662770 0x5643b8665f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f779481e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b85cbda0 con 0x5643b868fef0 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b8574b80 con 0x5643b8662770 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b85c7e40 con 0x5643b868fb10 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 0x5643b8692860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 0x5643b8692860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60232/0 (socket says 172.21.15.148:60232) 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 -- 172.21.15.148:0/2893852844 learned_addr learned my addr 172.21.15.148:0/2893852844 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778b7fe640 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b8692320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778bfff640 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8662770 0x5643b8665f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 -- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 msgr2=0x5643b8692320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b8692320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 -- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8662770 msgr2=0x5643b8665f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8662770 0x5643b8665f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778affd640 1 -- 172.21.15.148:0/2893852844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643b857ef10 con 0x5643b868fef0 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.015+0000 7f778b7fe640 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b8692320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.016+0000 7f778affd640 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 0x5643b8692860 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7780005260 tx=0x7f7780011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88b3fac79d96e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.016+0000 7f77717fa640 1 -- 172.21.15.148:0/2893852844 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000e500 con 0x5643b868fef0 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.016+0000 7f77717fa640 1 -- 172.21.15.148:0/2893852844 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778000e6a0 con 0x5643b868fef0 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.016+0000 7f77717fa640 1 -- 172.21.15.148:0/2893852844 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000ea50 con 0x5643b868fef0 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.016+0000 7f779481e740 1 -- 172.21.15.148:0/2893852844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 msgr2=0x5643b8692860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.016+0000 7f779481e740 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 0x5643b8692860 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7780005260 tx=0x7f7780011000 comp rx=0 tx=0).stop 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 -- 172.21.15.148:0/2893852844 shutdown_connections 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8662770 0x5643b8665f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fef0 0x5643b8692860 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 --2- 172.21.15.148:0/2893852844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b8692320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 -- 172.21.15.148:0/2893852844 >> 172.21.15.148:0/2893852844 conn(0x5643b86649e0 msgr2=0x5643b86ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 -- 172.21.15.148:0/2893852844 shutdown_connections 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 -- 172.21.15.148:0/2893852844 wait complete. 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 -- start start 2024-04-25T10:00:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.017+0000 7f779481e740 1 Processor -- start 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b86ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b8666f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b86674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b8667b90 con 0x5643b868fb10 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b856a4b0 con 0x5643b8677cd0 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b8667e70 con 0x5643b868fef0 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f779481e740 10 client.0 _ll_register_callbacks cb 0x5643b85aa180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b86ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b86ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60236/0 (socket says 172.21.15.148:60236) 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 -- 172.21.15.148:0/2810663678 learned_addr learned my addr 172.21.15.148:0/2810663678 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778bfff640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b86674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 -- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 msgr2=0x5643b8666f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778affd640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b8666f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b8666f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 -- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 msgr2=0x5643b86674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b86674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.018+0000 7f778b7fe640 1 -- 172.21.15.148:0/2810663678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643b83aa7d0 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f778bfff640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b86674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f778b7fe640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b86ae450 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f777c0072a0 tx=0x7f777c078840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b97ee3918f8076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f776a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f776a7fc640 1 -- 172.21.15.148:0/2810663678 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0a7060 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f776a7fc640 1 -- 172.21.15.148:0/2810663678 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777c078470 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f778000e6a0 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f776a7fc640 1 -- 172.21.15.148:0/2810663678 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f777c095050 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 10 client.7944 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5643b82e03f0 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f776a7fc640 1 -- 172.21.15.148:0/2810663678 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778000e6a0 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f776a7fc640 1 -- 172.21.15.148:0/2810663678 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f777c074b70 con 0x5643b868fb10 2024-04-25T10:00:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 10 client.7944 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 10 client.7944 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 20 client.7944 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b8693e70 0x5643b8686070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 4 client.7944 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.019+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643b85d92e0 con 0x5643b8693e70 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.020+0000 7f778affd640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b8693e70 0x5643b8686070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.020+0000 7f776a7fc640 10 client.7944 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.020+0000 7f778affd640 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b8693e70 0x5643b8686070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83d1a186607d4f0 server_cookie=b8c0d7941611d77b in_seq=0 out_seq=0 2024-04-25T10:00:51.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f776a7fc640 1 -- 172.21.15.148:0/2810663678 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643b85d92e0 con 0x5643b8693e70 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f776a7fc640 10 client.7944 handle_command_reply: tid=0 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 client.7944 shutdown 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 msgr2=0x5643b86ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b86ae450 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f777c0072a0 tx=0x7f777c078840 comp rx=0 tx=0).stop 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 shutdown_connections 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b8693e70 0x5643b8686070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b86674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b8666f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f776a7fc640 0 client.7944 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 --2- 172.21.15.148:0/2810663678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b86ae450 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 >> 172.21.15.148:0/2810663678 conn(0x5643b8662770 msgr2=0x5643b868ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 shutdown_connections 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 1 -- 172.21.15.148:0/2810663678 wait complete. 2024-04-25T10:00:51.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.021+0000 7f779481e740 20 client.7944 trim_cache size 0 max 16384 2024-04-25T10:00:51.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.036+0000 7f779481e740 1 Processor -- start 2024-04-25T10:00:51.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.036+0000 7f779481e740 1 -- start start 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f779481e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 0x5643b8661a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b868edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b868fef0 0x5643b868f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f779481e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c200500 con 0x5643b8677cd0 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1ff700 con 0x5643b868fef0 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c159860 con 0x5643b868fb10 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 0x5643b8661a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 0x5643b8661a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60238/0 (socket says 172.21.15.148:60238) 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 -- 172.21.15.148:0/3358761973 learned_addr learned my addr 172.21.15.148:0/3358761973 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 -- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 msgr2=0x5643b868edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b868edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778bfff640 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b868fef0 0x5643b868f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 -- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b868fef0 msgr2=0x5643b868f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b868fef0 0x5643b868f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 -- 172.21.15.148:0/3358761973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643b82a00f0 con 0x5643b8677cd0 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778bfff640 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b868fef0 0x5643b868f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f778b7fe640 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 0x5643b8661a20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f777c0aabe0 tx=0x7f777c07c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d69bf241d8a140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f7770ff9640 1 -- 172.21.15.148:0/3358761973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c09b030 con 0x5643b8677cd0 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.037+0000 7f7770ff9640 1 -- 172.21.15.148:0/3358761973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c09d050 con 0x5643b8677cd0 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f7770ff9640 1 -- 172.21.15.148:0/3358761973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c090070 con 0x5643b8677cd0 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 -- 172.21.15.148:0/3358761973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 msgr2=0x5643b8661a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 0x5643b8661a20 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f777c0aabe0 tx=0x7f777c07c000 comp rx=0 tx=0).stop 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 -- 172.21.15.148:0/3358761973 shutdown_connections 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b868fef0 0x5643b868f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fb10 0x5643b868edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 --2- 172.21.15.148:0/3358761973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b8677cd0 0x5643b8661a20 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 -- 172.21.15.148:0/3358761973 >> 172.21.15.148:0/3358761973 conn(0x5643b8662770 msgr2=0x5643b86687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 -- 172.21.15.148:0/3358761973 shutdown_connections 2024-04-25T10:00:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.038+0000 7f779481e740 1 -- 172.21.15.148:0/3358761973 wait complete. 2024-04-25T10:00:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 -- start start 2024-04-25T10:00:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 Processor -- start 2024-04-25T10:00:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b871a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b8665f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b86664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b8669ad0 con 0x5643b868fb10 2024-04-25T10:00:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.039+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b8693060 con 0x5643b8677cd0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f779481e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b8663ae0 con 0x5643b868fef0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f779481e740 10 client.0 _ll_register_callbacks cb 0x7f77800012e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b8665f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b8665f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60246/0 (socket says 172.21.15.148:60246) 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 -- 172.21.15.148:0/3164171018 learned_addr learned my addr 172.21.15.148:0/3164171018 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778b7fe640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b871a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778bfff640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b86664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 -- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 msgr2=0x5643b86664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b86664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 -- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 msgr2=0x5643b871a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b871a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 -- 172.21.15.148:0/3164171018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643b86ad3c0 con 0x5643b868fb10 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f778affd640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b8665f80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7780014000 tx=0x7f7780000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87c0f474fc3c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f776a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f776a7fc640 1 -- 172.21.15.148:0/3164171018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780029be0 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f776a7fc640 1 -- 172.21.15.148:0/3164171018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7780029d80 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f777c09d050 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f776a7fc640 1 -- 172.21.15.148:0/3164171018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7780035050 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f779481e740 10 client.7950 fetch_fsmap learned FSMap version 53 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.040+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5643b8666ac0 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f776a7fc640 1 -- 172.21.15.148:0/3164171018 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c09d050 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f776a7fc640 1 -- 172.21.15.148:0/3164171018 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7780038020 con 0x5643b868fb10 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 10 client.7950 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 10 client.7950 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 20 client.7950 populate_metadata read hostname 'smithi148' 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b86adea0 0x5643b868edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 4 client.7950 mds_command: new command op to 7725 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 --> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5643b85d3630 con 0x5643b86adea0 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f778b7fe640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b86adea0 0x5643b868edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f776a7fc640 10 client.7950 ms_handle_connect on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f778b7fe640 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b86adea0 0x5643b868edf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48d27cb8c2dfd6d server_cookie=d5929eb4c39beae2 in_seq=0 out_seq=0 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f776a7fc640 1 -- 172.21.15.148:0/3164171018 <== mds.0 v2:172.21.15.188:6832/1505603179 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5643b85d3630 con 0x5643b86adea0 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f776a7fc640 10 client.7950 handle_command_reply: tid=0 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 1 client.7950 shutdown 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 msgr2=0x5643b8665f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.041+0000 7f779481e740 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b8665f80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7780014000 tx=0x7f7780000ed0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 shutdown_connections 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:6832/1505603179,v1:172.21.15.188:6834/1505603179] conn(0x5643b86adea0 0x5643b868edf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5643b868fef0 0x5643b86664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f776a7fc640 0 client.7950 ms_handle_reset on v2:172.21.15.188:6832/1505603179 2024-04-25T10:00:51.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643b868fb10 0x5643b8665f80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 --2- 172.21.15.148:0/3164171018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5643b8677cd0 0x5643b871a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 >> 172.21.15.148:0/3164171018 conn(0x5643b8662770 msgr2=0x5643b86cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 shutdown_connections 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 1 -- 172.21.15.148:0/3164171018 wait complete. 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f779481e740 20 client.7950 trim_cache size 0 max 16384 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:00:51.053 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:00:51.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b500 msgr2=0x7f777806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b500 0x7f777806d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f778c0ae990 tx=0x7f7780009040 comp rx=0 tx=0).stop 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 msgr2=0x7f778c0c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 0x7f778c0c9fb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f777c034040 tx=0x7f777c004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 shutdown_connections 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b500 0x7f777806d9c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c15de80 0x7f778c0ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c0cefb0 0x7f778c0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 --2- 172.21.15.148:0/1580332812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c0ccfe0 0x7f778c0c9fb0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 >> 172.21.15.148:0/1580332812 conn(0x7f778c0b2390 msgr2=0x7f778c154270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 shutdown_connections 2024-04-25T10:00:51.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.042+0000 7f7791b1b640 1 -- 172.21.15.148:0/1580332812 wait complete. 2024-04-25T10:00:51.057 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:00:51.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:51.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.151+0000 7f4c5fae1640 1 Processor -- start 2024-04-25T10:00:51.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.151+0000 7f4c5fae1640 1 -- start start 2024-04-25T10:00:51.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5fae1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 0x7f4c580d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5fae1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c580c7730 0x7f4c580ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5fae1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c580c7b90 0x7f4c580ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5fae1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c580c8030 con 0x7f4c580c7730 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5fae1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c580c8310 con 0x7f4c580c7b90 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5fae1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c580c85f0 con 0x7f4c5815da70 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c580c7730 0x7f4c580ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4c580c7730 0x7f4c580ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60252/0 (socket says 172.21.15.148:60252) 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 -- 172.21.15.148:0/4040536755 learned_addr learned my addr 172.21.15.148:0/4040536755 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5e07e640 1 --2- 172.21.15.148:0/4040536755 >> v2:172.21.15.188:3300/0 conn(0x7f4c580c7b90 0x7f4c580ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 -- 172.21.15.148:0/4040536755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 msgr2=0x7f4c580d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 --2- 172.21.15.148:0/4040536755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 0x7f4c580d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 -- 172.21.15.148:0/4040536755 >> v2:172.21.15.188:3300/0 conn(0x7f4c580c7b90 msgr2=0x7f4c580ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 --2- 172.21.15.148:0/4040536755 >> v2:172.21.15.188:3300/0 conn(0x7f4c580c7b90 0x7f4c580ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5d07c640 1 -- 172.21.15.148:0/4040536755 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5806bb30 con 0x7f4c580c7730 2024-04-25T10:00:51.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.152+0000 7f4c5e07e640 1 --2- 172.21.15.148:0/4040536755 >> v2:172.21.15.188:3300/0 conn(0x7f4c580c7b90 0x7f4c580ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:51.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.153+0000 7f4c5d07c640 1 --2- 172.21.15.148:0/4040536755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c580ce1c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f4c580c67e0 tx=0x7f4c4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a835d0f0297bef1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.153+0000 7f4c5c87b640 1 -- 172.21.15.148:0/4040536755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48037070 con 0x7f4c580c7730 2024-04-25T10:00:51.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.153+0000 7f4c5c87b640 1 -- 172.21.15.148:0/4040536755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48032440 con 0x7f4c580c7730 2024-04-25T10:00:51.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.153+0000 7f4c5c87b640 1 -- 172.21.15.148:0/4040536755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48032810 con 0x7f4c580c7730 2024-04-25T10:00:51.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.153+0000 7f4c5fae1640 1 -- 172.21.15.148:0/4040536755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 msgr2=0x7f4c580ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.153+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/4040536755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c580ce1c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f4c580c67e0 tx=0x7f4c4802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 -- 172.21.15.148:0/4040536755 shutdown_connections 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/4040536755 >> v2:172.21.15.188:3300/0 conn(0x7f4c580c7b90 0x7f4c580ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/4040536755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c580ce1c0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/4040536755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 0x7f4c580d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 -- 172.21.15.148:0/4040536755 >> 172.21.15.148:0/4040536755 conn(0x7f4c580b33e0 msgr2=0x7f4c580b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 -- 172.21.15.148:0/4040536755 shutdown_connections 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.154+0000 7f4c5fae1640 1 -- 172.21.15.148:0/4040536755 wait complete. 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 Processor -- start 2024-04-25T10:00:51.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 -- start start 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c58146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c580c7b90 0x7f4c580b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 0x7f4c5815f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58070c60 con 0x7f4c580c7730 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5815de50 con 0x7f4c580c7b90 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.155+0000 7f4c5fae1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58070de0 con 0x7f4c5815da70 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c58146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c58146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60264/0 (socket says 172.21.15.148:60264) 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 -- 172.21.15.148:0/624897325 learned_addr learned my addr 172.21.15.148:0/624897325 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d07c640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c580c7b90 0x7f4c580b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 -- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 msgr2=0x7f4c5815f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 0x7f4c5815f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 -- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c580c7b90 msgr2=0x7f4c580b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c580c7b90 0x7f4c580b6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 -- 172.21.15.148:0/624897325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58146760 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5d87d640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c58146220 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400b440 tx=0x7f4c5400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54015020 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c54011890 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5815e2f0 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c54011c80 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.156+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c581608d0 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.157+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5806de80 con 0x7f4c580c7730 2024-04-25T10:00:51.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.158+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c54018020 con 0x7f4c580c7730 2024-04-25T10:00:51.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.158+0000 7f4c4e7fc640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c3c06b4d0 0x7f4c3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.158+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c5408a3f0 con 0x7f4c580c7730 2024-04-25T10:00:51.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.158+0000 7f4c5d07c640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c3c06b4d0 0x7f4c3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.158+0000 7f4c5d07c640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c3c06b4d0 0x7f4c3c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4c48034000 tx=0x7f4c4802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.162+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c5401f020 con 0x7f4c580c7730 2024-04-25T10:00:51.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.367+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c580b15c0 con 0x7f4c580c7730 2024-04-25T10:00:51.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.367+0000 7f4c4e7fc640 1 -- 172.21.15.148:0/624897325 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f4c58146760 con 0x7f4c580c7730 2024-04-25T10:00:51.372 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:51.372 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:51.372 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:51.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.369+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c3c06b4d0 msgr2=0x7f4c3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.369+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c3c06b4d0 0x7f4c3c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4c48034000 tx=0x7f4c4802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:51.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.369+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 msgr2=0x7f4c58146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.369+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c58146220 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400b440 tx=0x7f4c5400b900 comp rx=0 tx=0).stop 2024-04-25T10:00:51.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 shutdown_connections 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c3c06b4d0 0x7f4c3c06d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c5815da70 0x7f4c5815f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c580c7b90 0x7f4c580b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 --2- 172.21.15.148:0/624897325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c580c7730 0x7f4c58146220 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 >> 172.21.15.148:0/624897325 conn(0x7f4c580b33e0 msgr2=0x7f4c5806d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 shutdown_connections 2024-04-25T10:00:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.370+0000 7f4c5fae1640 1 -- 172.21.15.148:0/624897325 wait complete. 2024-04-25T10:00:51.384 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:00:51.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:00:51.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.476+0000 7f8916e4a640 1 Processor -- start 2024-04-25T10:00:51.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.477+0000 7f8916e4a640 1 -- start start 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.477+0000 7f8916e4a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f891015dbe0 0x7f89100d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f8916e4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89100b6220 con 0x7f89100b92a0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f8916e4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89100b97a0 con 0x7f89100b2cb0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f8916e4a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891015dfc0 con 0x7f891015dbe0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f89153e7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f89100b92a0 0x7f89100b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57106/0 (socket says 172.21.15.148:57106) 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f89153e7640 1 -- 172.21.15.148:0/3444455839 learned_addr learned my addr 172.21.15.148:0/3444455839 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f8914be6640 1 --2- 172.21.15.148:0/3444455839 >> v2:172.21.15.188:3300/0 conn(0x7f891015dbe0 0x7f89100d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325639077 0 0) 0x7f89100b6220 con 0x7f89100b92a0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f8003520 con 0x7f89100b92a0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300478832 0 0) 0x7f88f8003520 con 0x7f89100b92a0 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 >> v2:172.21.15.188:3300/0 conn(0x7f891015dbe0 msgr2=0x7f89100d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 --2- 172.21.15.148:0/3444455839 >> v2:172.21.15.188:3300/0 conn(0x7f891015dbe0 0x7f89100d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 >> v1:172.21.15.188:6789/0 conn(0x7f89100b2cb0 legacy=0x7f89100b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100d1c50 con 0x7f89100b92a0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f89080019f0 con 0x7f89100b92a0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 >> v1:172.21.15.148:6789/0 conn(0x7f89100b92a0 legacy=0x7f89100b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f80039b0 0x7f88f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80043b0 0x7f88f8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8005120 0x7f88f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f800c030 con 0x7f88f80039b0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f800c1e0 con 0x7f88f8005120 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f800c3e0 con 0x7f88f80043b0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f8914be6640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f80039b0 0x7f88f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f89153e7640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8005120 0x7f88f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.478+0000 7f890ffff640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80043b0 0x7f88f8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f8914be6640 1 -- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80043b0 msgr2=0x7f88f8004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f8914be6640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80043b0 0x7f88f8004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f8914be6640 1 -- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8005120 msgr2=0x7f88f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f8914be6640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f8005120 0x7f88f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f8914be6640 1 -- 172.21.15.148:0/3444455839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8003520 con 0x7f88f80039b0 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f89153e7640 1 -- 172.21.15.148:0/3444455839 reap_dead start 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f8914be6640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f80039b0 0x7f88f8003d90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8900009ac0 tx=0x7f88f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb99d540729a73c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900036070 con 0x7f88f80039b0 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8900031400 con 0x7f88f80039b0 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.479+0000 7f890f7fe640 1 -- 172.21.15.148:0/3444455839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900009690 con 0x7f88f80039b0 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 -- 172.21.15.148:0/3444455839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f80039b0 msgr2=0x7f88f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f80039b0 0x7f88f8003d90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f8900009ac0 tx=0x7f88f8005830 comp rx=0 tx=0).stop 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 -- 172.21.15.148:0/3444455839 shutdown_connections 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 --2- 172.21.15.148:0/3444455839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f88f80039b0 0x7f88f8003d90 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 -- 172.21.15.148:0/3444455839 >> 172.21.15.148:0/3444455839 conn(0x7f8910061a20 msgr2=0x7f8910061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 -- 172.21.15.148:0/3444455839 shutdown_connections 2024-04-25T10:00:51.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.480+0000 7f8916e4a640 1 -- 172.21.15.148:0/3444455839 wait complete. 2024-04-25T10:00:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.481+0000 7f8916e4a640 1 Processor -- start 2024-04-25T10:00:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.481+0000 7f8916e4a640 1 -- start start 2024-04-25T10:00:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8916e4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80039b0 0x7f89101f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8916e4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 0x7f8910155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8916e4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89101563a0 0x7f89100b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8916e4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100d2210 con 0x7f89101f08a0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8916e4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100d1a00 con 0x7f89101563a0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8916e4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89100b6ab0 con 0x7f88f80039b0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 0x7f8910155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 0x7f8910155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60292/0 (socket says 172.21.15.148:60292) 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 -- 172.21.15.148:0/1556756523 learned_addr learned my addr 172.21.15.148:0/1556756523 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f8914be6640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80039b0 0x7f89101f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f89153e7640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89101563a0 0x7f89100b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 -- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80039b0 msgr2=0x7f89101f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80039b0 0x7f89101f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 -- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89101563a0 msgr2=0x7f89100b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89101563a0 0x7f89100b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.482+0000 7f890ffff640 1 -- 172.21.15.148:0/1556756523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89101f3a70 con 0x7f89101f08a0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.483+0000 7f890ffff640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 0x7f8910155af0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f890400e7b0 tx=0x7f890400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.483+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89040050d0 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.483+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904018070 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.483+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89101f3d50 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.483+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89040054b0 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.483+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89101f4310 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.484+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910073990 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.484+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f890400c640 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.484+0000 7f890d7fa640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88fc06b4d0 0x7f88fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.485+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8904089f70 con 0x7f89101f08a0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.485+0000 7f8914be6640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88fc06b4d0 0x7f88fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.485+0000 7f8914be6640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88fc06b4d0 0x7f88fc06d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8900033000 tx=0x7f8900008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:51.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.489+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8904058cf0 con 0x7f89101f08a0 2024-04-25T10:00:51.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.723+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f89100b7970 con 0x7f89101f08a0 2024-04-25T10:00:51.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.724+0000 7f890d7fa640 1 -- 172.21.15.148:0/1556756523 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1026 (secure 0 0 0) 0x7f89040585f0 con 0x7f89101f08a0 2024-04-25T10:00:51.728 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:51.728 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":412,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2529034240,"bytes_avail":770565079040,"bytes_total":773094113280,"write_bytes_sec":89652057,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":53,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"d","status":"up:active","gid":7725}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T09:59:57.088810+0000","services":{}},"progress_events":{}} 2024-04-25T10:00:51.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88fc06b4d0 msgr2=0x7f88fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88fc06b4d0 0x7f88fc06d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8900033000 tx=0x7f8900008030 comp rx=0 tx=0).stop 2024-04-25T10:00:51.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 msgr2=0x7f8910155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 0x7f8910155af0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f890400e7b0 tx=0x7f890400ebb0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 shutdown_connections 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f88fc06b4d0 0x7f88fc06d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89101563a0 0x7f89100b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89101f08a0 0x7f8910155af0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 --2- 172.21.15.148:0/1556756523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f80039b0 0x7f89101f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 >> 172.21.15.148:0/1556756523 conn(0x7f8910061a20 msgr2=0x7f89100d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 shutdown_connections 2024-04-25T10:00:51.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:51.726+0000 7f8916e4a640 1 -- 172.21.15.148:0/1556756523 wait complete. 2024-04-25T10:00:51.741 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:00:51.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:51.778 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:00:51.824 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:00:51.825 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:51.825 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:51.843 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:51.843 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:51.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:52.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.031+0000 7f1e75ac4640 1 Processor -- start 2024-04-25T10:00:52.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.031+0000 7f1e75ac4640 1 -- start start 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e75ac4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e7006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e75ac4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1e7006eb10 0x7f1e700ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e75ac4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e700cee60 0x7f1e7015f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e75ac4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e70144b30 con 0x7f1e7006eb10 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e75ac4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e700712d0 con 0x7f1e700cee60 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e75ac4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700715b0 con 0x7f1e70144750 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1e7006eb10 0x7f1e700ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1e7006eb10 0x7f1e700ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60308/0 (socket says 172.21.15.148:60308) 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 -- 172.21.15.148:0/2396449981 learned_addr learned my addr 172.21.15.148:0/2396449981 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6ffff640 1 --2- 172.21.15.148:0/2396449981 >> v2:172.21.15.188:3300/0 conn(0x7f1e700cee60 0x7f1e7015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6f7fe640 1 --2- 172.21.15.148:0/2396449981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e7006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 -- 172.21.15.148:0/2396449981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 msgr2=0x7f1e7006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 --2- 172.21.15.148:0/2396449981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e7006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 -- 172.21.15.148:0/2396449981 >> v2:172.21.15.188:3300/0 conn(0x7f1e700cee60 msgr2=0x7f1e7015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 --2- 172.21.15.148:0/2396449981 >> v2:172.21.15.188:3300/0 conn(0x7f1e700cee60 0x7f1e7015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.032+0000 7f1e6effd640 1 -- 172.21.15.148:0/2396449981 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70071890 con 0x7f1e7006eb10 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.033+0000 7f1e6effd640 1 --2- 172.21.15.148:0/2396449981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e700ce920 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1e700b4aa0 tx=0x7f1e6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a37937ea48c1343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.033+0000 7f1e6e7fc640 1 -- 172.21.15.148:0/2396449981 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64037070 con 0x7f1e7006eb10 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.033+0000 7f1e6e7fc640 1 -- 172.21.15.148:0/2396449981 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e64032440 con 0x7f1e7006eb10 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.033+0000 7f1e6e7fc640 1 -- 172.21.15.148:0/2396449981 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64032810 con 0x7f1e7006eb10 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 -- 172.21.15.148:0/2396449981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 msgr2=0x7f1e700ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/2396449981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e700ce920 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1e700b4aa0 tx=0x7f1e6402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 -- 172.21.15.148:0/2396449981 shutdown_connections 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/2396449981 >> v2:172.21.15.188:3300/0 conn(0x7f1e700cee60 0x7f1e7015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/2396449981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e700ce920 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/2396449981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e7006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 -- 172.21.15.148:0/2396449981 >> 172.21.15.148:0/2396449981 conn(0x7f1e7006abe0 msgr2=0x7f1e7006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 -- 172.21.15.148:0/2396449981 shutdown_connections 2024-04-25T10:00:52.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.034+0000 7f1e75ac4640 1 -- 172.21.15.148:0/2396449981 wait complete. 2024-04-25T10:00:52.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.035+0000 7f1e75ac4640 1 Processor -- start 2024-04-25T10:00:52.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.035+0000 7f1e75ac4640 1 -- start start 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e70143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e700cee60 0x7f1e701f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e701f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70145450 con 0x7f1e7006eb10 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7006f020 con 0x7f1e700cee60 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e701455d0 con 0x7f1e70144750 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e70143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e70143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60310/0 (socket says 172.21.15.148:60310) 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 -- 172.21.15.148:0/3090898445 learned_addr learned my addr 172.21.15.148:0/3090898445 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 -- 172.21.15.148:0/3090898445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 msgr2=0x7f1e701f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e701f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 -- 172.21.15.148:0/3090898445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e700cee60 msgr2=0x7f1e701f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e700cee60 0x7f1e701f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 -- 172.21.15.148:0/3090898445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70141340 con 0x7f1e7006eb10 2024-04-25T10:00:52.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e6f7fe640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e70143b10 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000d800 tx=0x7f1e6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000b660 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60019070 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e70141620 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000ba00 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.036+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e70141be0 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.037+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70073990 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.038+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e6000bba0 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.038+0000 7f1e57fff640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1e5c06b500 0x7f1e5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.038+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e6008a7a0 con 0x7f1e7006eb10 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.038+0000 7f1e6effd640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1e5c06b500 0x7f1e5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.039+0000 7f1e6effd640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1e5c06b500 0x7f1e5c06d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1e64034000 tx=0x7f1e6402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.042+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e60059490 con 0x7f1e7006eb10 2024-04-25T10:00:52.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.213+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e700d0510 con 0x7f1e7006eb10 2024-04-25T10:00:52.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.214+0000 7f1e57fff640 1 -- 172.21.15.148:0/3090898445 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f1e60058d90 con 0x7f1e7006eb10 2024-04-25T10:00:52.219 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:52.219 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:27.088834+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1e5c06b500 msgr2=0x7f1e5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1e5c06b500 0x7f1e5c06d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1e64034000 tx=0x7f1e6402c000 comp rx=0 tx=0).stop 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 msgr2=0x7f1e70143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e70143b10 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000d800 tx=0x7f1e6000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 shutdown_connections 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1e5c06b500 0x7f1e5c06d9c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70144750 0x7f1e701f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e700cee60 0x7f1e701f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 --2- 172.21.15.148:0/3090898445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7006eb10 0x7f1e70143b10 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.216+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 >> 172.21.15.148:0/3090898445 conn(0x7f1e7006abe0 msgr2=0x7f1e700af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.217+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 shutdown_connections 2024-04-25T10:00:52.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.217+0000 7f1e75ac4640 1 -- 172.21.15.148:0/3090898445 wait complete. 2024-04-25T10:00:52.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:00:23.631066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:00:24.401378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:52.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:52.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.327+0000 7f250d942640 1 Processor -- start 2024-04-25T10:00:52.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.327+0000 7f250d942640 1 -- start start 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f250d942640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f250815dc20 0x7f25080ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f250d942640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2508161710 0x7f25080cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f250d942640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250815e000 con 0x7f25080b9330 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f250d942640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25080d1de0 con 0x7f2508161710 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f250d942640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2508161c60 con 0x7f250815dc20 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25067fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2508161710 0x7f25080cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25077fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f25080b9330 0x7f250806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57110/0 (socket says 172.21.15.148:57110) 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25067fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2508161710 0x7f25080cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60324/0 (socket says 172.21.15.148:60324) 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25077fe640 1 -- 172.21.15.148:0/417968979 learned_addr learned my addr 172.21.15.148:0/417968979 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f2506ffd640 1 --2- 172.21.15.148:0/417968979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f250815dc20 0x7f25080ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25067fc640 1 -- 172.21.15.148:0/417968979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f250815dc20 msgr2=0x7f25080ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25067fc640 1 --2- 172.21.15.148:0/417968979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f250815dc20 0x7f25080ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25067fc640 1 -- 172.21.15.148:0/417968979 >> v1:172.21.15.148:6789/0 conn(0x7f25080b9330 legacy=0x7f250806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f2505ffb640 1 -- 172.21.15.148:0/417968979 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473900271 0 0) 0x7f250815e000 con 0x7f25080b9330 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.328+0000 7f25067fc640 1 -- 172.21.15.148:0/417968979 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508161f40 con 0x7f2508161710 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f25067fc640 1 --2- 172.21.15.148:0/417968979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2508161710 0x7f25080cebb0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0097a0 tx=0x7f24fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba556e77945f034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f2505ffb640 1 -- 172.21.15.148:0/417968979 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc037070 con 0x7f2508161710 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f2505ffb640 1 -- 172.21.15.148:0/417968979 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc032440 con 0x7f2508161710 2024-04-25T10:00:52.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f2505ffb640 1 -- 172.21.15.148:0/417968979 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc032810 con 0x7f2508161710 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f250d942640 1 -- 172.21.15.148:0/417968979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2508161710 msgr2=0x7f25080cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f250d942640 1 --2- 172.21.15.148:0/417968979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2508161710 0x7f25080cebb0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0097a0 tx=0x7f24fc02a180 comp rx=0 tx=0).stop 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f250d942640 1 -- 172.21.15.148:0/417968979 shutdown_connections 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f250d942640 1 --2- 172.21.15.148:0/417968979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2508161710 0x7f25080cebb0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.329+0000 7f250d942640 1 --2- 172.21.15.148:0/417968979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f250815dc20 0x7f25080ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.330+0000 7f250d942640 1 -- 172.21.15.148:0/417968979 >> 172.21.15.148:0/417968979 conn(0x7f25080d0420 msgr2=0x7f25080d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.330+0000 7f250d942640 1 -- 172.21.15.148:0/417968979 shutdown_connections 2024-04-25T10:00:52.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.330+0000 7f250d942640 1 -- 172.21.15.148:0/417968979 wait complete. 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.331+0000 7f250d942640 1 Processor -- start 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.331+0000 7f250d942640 1 -- start start 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25080b9330 0x7f25081551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 0x7f2508158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2508161710 0x7f2508156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25080b5e50 con 0x7f250815dc20 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508162500 con 0x7f2508161710 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25080b5fd0 con 0x7f25080b9330 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 0x7f2508158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 0x7f2508158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60328/0 (socket says 172.21.15.148:60328) 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 -- 172.21.15.148:0/1145926130 learned_addr learned my addr 172.21.15.148:0/1145926130 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25077fe640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2508161710 0x7f2508156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f2506ffd640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25080b9330 0x7f25081551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 -- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25080b9330 msgr2=0x7f25081551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25080b9330 0x7f25081551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 -- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2508161710 msgr2=0x7f2508156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2508161710 0x7f2508156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 -- 172.21.15.148:0/1145926130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250815ec30 con 0x7f250815dc20 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f25067fc640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 0x7f2508158470 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f24fc035040 tx=0x7f24fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc037040 con 0x7f250815dc20 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc030770 con 0x7f250815dc20 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc030b80 con 0x7f250815dc20 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250815ef10 con 0x7f250815dc20 2024-04-25T10:00:52.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.332+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250815f4d0 con 0x7f250815dc20 2024-04-25T10:00:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.333+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2508073a00 con 0x7f250815dc20 2024-04-25T10:00:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.334+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24fc044020 con 0x7f250815dc20 2024-04-25T10:00:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.334+0000 7f24effff640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f24f406b4d0 0x7f24f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.334+0000 7f2506ffd640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f24f406b4d0 0x7f24f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.334+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24fc0aa8f0 con 0x7f250815dc20 2024-04-25T10:00:52.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.335+0000 7f2506ffd640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f24f406b4d0 0x7f24f406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f24f8006fd0 tx=0x7f24f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.339+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24fc032e20 con 0x7f250815dc20 2024-04-25T10:00:52.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.507+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25080d16b0 con 0x7f250815dc20 2024-04-25T10:00:52.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.507+0000 7f24effff640 1 -- 172.21.15.148:0/1145926130 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24fc0313a0 con 0x7f250815dc20 2024-04-25T10:00:52.511 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:00:52.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f24f406b4d0 msgr2=0x7f24f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f24f406b4d0 0x7f24f406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f24f8006fd0 tx=0x7f24f8008040 comp rx=0 tx=0).stop 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 msgr2=0x7f2508158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 0x7f2508158470 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f24fc035040 tx=0x7f24fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 shutdown_connections 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f24f406b4d0 0x7f24f406d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2508161710 0x7f2508156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250815dc20 0x7f2508158470 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 --2- 172.21.15.148:0/1145926130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25080b9330 0x7f25081551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 >> 172.21.15.148:0/1145926130 conn(0x7f25080d0420 msgr2=0x7f25080d2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 shutdown_connections 2024-04-25T10:00:52.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.509+0000 7f250d942640 1 -- 172.21.15.148:0/1145926130 wait complete. 2024-04-25T10:00:52.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.620+0000 7f62fed9e640 1 Processor -- start 2024-04-25T10:00:52.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.620+0000 7f62fed9e640 1 -- start start 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.620+0000 7f62fed9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62f806ccd0 0x7f62f80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62fed9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f815deb0 0x7f62f815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62fed9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f80d1770 con 0x7f62f8070dc0 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62fed9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f81622b0 con 0x7f62f806ccd0 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62fed9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f80b10f0 con 0x7f62f815deb0 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62f806ccd0 0x7f62f80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62f806ccd0 0x7f62f80b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60330/0 (socket says 172.21.15.148:60330) 2024-04-25T10:00:52.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 -- 172.21.15.148:0/113337588 learned_addr learned my addr 172.21.15.148:0/113337588 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 -- 172.21.15.148:0/113337588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f815deb0 msgr2=0x7f62f815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 --2- 172.21.15.148:0/113337588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f815deb0 0x7f62f815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 -- 172.21.15.148:0/113337588 >> v1:172.21.15.188:6789/0 conn(0x7f62f8070dc0 legacy=0x7f62f80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.621+0000 7f62f7fff640 1 -- 172.21.15.148:0/113337588 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8162740 con 0x7f62f806ccd0 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62f7fff640 1 --2- 172.21.15.148:0/113337588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f806ccd0 0x7f62f80b07a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009870 tx=0x7f62e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c908cf93a28cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62f77fe640 1 -- 172.21.15.148:0/113337588 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8037070 con 0x7f62f806ccd0 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62f77fe640 1 -- 172.21.15.148:0/113337588 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e802ae10 con 0x7f62f806ccd0 2024-04-25T10:00:52.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62f77fe640 1 -- 172.21.15.148:0/113337588 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8032660 con 0x7f62f806ccd0 2024-04-25T10:00:52.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 -- 172.21.15.148:0/113337588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f806ccd0 msgr2=0x7f62f80b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 --2- 172.21.15.148:0/113337588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f806ccd0 0x7f62f80b07a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009870 tx=0x7f62e802a100 comp rx=0 tx=0).stop 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 -- 172.21.15.148:0/113337588 shutdown_connections 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 --2- 172.21.15.148:0/113337588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f806ccd0 0x7f62f80b07a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 --2- 172.21.15.148:0/113337588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f815deb0 0x7f62f815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 -- 172.21.15.148:0/113337588 >> 172.21.15.148:0/113337588 conn(0x7f62f80b3530 msgr2=0x7f62f80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 -- 172.21.15.148:0/113337588 shutdown_connections 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.622+0000 7f62fed9e640 1 -- 172.21.15.148:0/113337588 wait complete. 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 Processor -- start 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 -- start start 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f806ccd0 0x7f62f80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f8070dc0 0x7f62f814bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 0x7f62f80a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f8161f40 con 0x7f62f815deb0 2024-04-25T10:00:52.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f8161c60 con 0x7f62f8070dc0 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.624+0000 7f62fed9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f80b5840 con 0x7f62f806ccd0 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 0x7f62f80a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 0x7f62f80a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60338/0 (socket says 172.21.15.148:60338) 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 -- 172.21.15.148:0/2451341224 learned_addr learned my addr 172.21.15.148:0/2451341224 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62f7fff640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f8070dc0 0x7f62f814bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fcb3a640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f806ccd0 0x7f62f80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 -- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f806ccd0 msgr2=0x7f62f80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f806ccd0 0x7f62f80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 -- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f8070dc0 msgr2=0x7f62f814bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f8070dc0 0x7f62f814bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 -- 172.21.15.148:0/2451341224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f80a8a50 con 0x7f62f815deb0 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fcb3a640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f806ccd0 0x7f62f80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:52.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fd33b640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 0x7f62f80a8510 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f62e000b440 tx=0x7f62e000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0015020 con 0x7f62f815deb0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e0011830 con 0x7f62f815deb0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f8149370 con 0x7f62f815deb0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0011c20 con 0x7f62f815deb0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.625+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f8149930 con 0x7f62f815deb0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.626+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f8073a00 con 0x7f62f815deb0 2024-04-25T10:00:52.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.627+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62e0018020 con 0x7f62f815deb0 2024-04-25T10:00:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.627+0000 7f62f57fa640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62d806b4d0 0x7f62d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.627+0000 7f62fcb3a640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62d806b4d0 0x7f62d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.627+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62e008a170 con 0x7f62f815deb0 2024-04-25T10:00:52.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.627+0000 7f62fcb3a640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62d806b4d0 0x7f62d806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62f00045b0 tx=0x7f62f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.631+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62e001f020 con 0x7f62f815deb0 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.801+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62f814a090 con 0x7f62f815deb0 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.802+0000 7f62f57fa640 1 -- 172.21.15.148:0/2451341224 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62e0058e60 con 0x7f62f815deb0 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.803+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62d806b4d0 msgr2=0x7f62d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.803+0000 7f62fed9e640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62d806b4d0 0x7f62d806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62f00045b0 tx=0x7f62f0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.803+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 msgr2=0x7f62f80a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.803+0000 7f62fed9e640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 0x7f62f80a8510 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f62e000b440 tx=0x7f62e000b900 comp rx=0 tx=0).stop 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 shutdown_connections 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62d806b4d0 0x7f62d806d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f815deb0 0x7f62f80a8510 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62f8070dc0 0x7f62f814bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 --2- 172.21.15.148:0/2451341224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62f806ccd0 0x7f62f80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 >> 172.21.15.148:0/2451341224 conn(0x7f62f80b3530 msgr2=0x7f62f806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 shutdown_connections 2024-04-25T10:00:52.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.804+0000 7f62fed9e640 1 -- 172.21.15.148:0/2451341224 wait complete. 2024-04-25T10:00:52.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:52.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.914+0000 7f95766ac640 1 Processor -- start 2024-04-25T10:00:52.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.914+0000 7f95766ac640 1 -- start start 2024-04-25T10:00:52.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f95766ac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f957006abe0 0x7f957006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f95766ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f95766ac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957006b500 con 0x7f95700ad570 2024-04-25T10:00:52.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f95766ac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957006e410 con 0x7f957006abe0 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f95766ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95700b3930 con 0x7f957015dd80 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f9574c49640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f95700ad570 0x7f957006ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57114/0 (socket says 172.21.15.148:57114) 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f9574c49640 1 -- 172.21.15.148:0/54062674 learned_addr learned my addr 172.21.15.148:0/54062674 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956ffff640 1 --2- 172.21.15.148:0/54062674 >> v2:172.21.15.188:3300/0 conn(0x7f957006abe0 0x7f957006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956f7fe640 1 --2- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956effd640 1 -- 172.21.15.148:0/54062674 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790203721 0 0) 0x7f957006b500 con 0x7f95700ad570 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956effd640 1 -- 172.21.15.148:0/54062674 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9558003520 con 0x7f95700ad570 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956ffff640 1 -- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 msgr2=0x7f95700b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956ffff640 1 --2- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956ffff640 1 -- 172.21.15.148:0/54062674 >> v1:172.21.15.148:6789/0 conn(0x7f95700ad570 legacy=0x7f957006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956effd640 1 -- 172.21.15.148:0/54062674 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734013389 0 0) 0x7f9558003520 con 0x7f95700ad570 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.915+0000 7f956ffff640 1 -- 172.21.15.148:0/54062674 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700b3c10 con 0x7f957006abe0 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.916+0000 7f956ffff640 1 --2- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957006abe0 0x7f957006afc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f95600097a0 tx=0x7f956002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4084643c65ce81c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.916+0000 7f956effd640 1 -- 172.21.15.148:0/54062674 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560037070 con 0x7f957006abe0 2024-04-25T10:00:52.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.916+0000 7f956effd640 1 -- 172.21.15.148:0/54062674 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956002aba0 con 0x7f957006abe0 2024-04-25T10:00:52.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.916+0000 7f956effd640 1 -- 172.21.15.148:0/54062674 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680019f0 con 0x7f957006abe0 2024-04-25T10:00:52.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 -- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957006abe0 msgr2=0x7f957006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 --2- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957006abe0 0x7f957006afc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f95600097a0 tx=0x7f956002a110 comp rx=0 tx=0).stop 2024-04-25T10:00:52.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 -- 172.21.15.148:0/54062674 shutdown_connections 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 --2- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 --2- 172.21.15.148:0/54062674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f957006abe0 0x7f957006afc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 -- 172.21.15.148:0/54062674 >> 172.21.15.148:0/54062674 conn(0x7f95700b5af0 msgr2=0x7f95700b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 -- 172.21.15.148:0/54062674 shutdown_connections 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.917+0000 7f95766ac640 1 -- 172.21.15.148:0/54062674 wait complete. 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.918+0000 7f95766ac640 1 Processor -- start 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.918+0000 7f95766ac640 1 -- start start 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f95766ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 0x7f9570147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f95766ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95700ad570 0x7f957015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f95766ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f95766ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95700b57d0 con 0x7f957006abe0 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f95766ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957006e940 con 0x7f95700ad570 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f95766ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9570070280 con 0x7f957015dd80 2024-04-25T10:00:52.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 0x7f9570147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 0x7f9570147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60352/0 (socket says 172.21.15.148:60352) 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 -- 172.21.15.148:0/3115761958 learned_addr learned my addr 172.21.15.148:0/3115761958 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956f7fe640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95700ad570 0x7f957015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f9574c49640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 -- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 msgr2=0x7f95700b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 -- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95700ad570 msgr2=0x7f957015c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95700ad570 0x7f957015c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 -- 172.21.15.148:0/3115761958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957015aba0 con 0x7f957006abe0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.919+0000 7f956ffff640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 0x7f9570147130 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9560035040 tx=0x7f95600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.920+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560042020 con 0x7f957006abe0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.920+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956003d670 con 0x7f957006abe0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.920+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957015ae80 con 0x7f957006abe0 2024-04-25T10:00:52.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.920+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560037040 con 0x7f957006abe0 2024-04-25T10:00:52.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.920+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957015b440 con 0x7f957006abe0 2024-04-25T10:00:52.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.921+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9570073a00 con 0x7f957006abe0 2024-04-25T10:00:52.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.921+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9560045020 con 0x7f957006abe0 2024-04-25T10:00:52.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.922+0000 7f956cff9640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f955c06b4d0 0x7f955c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:52.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.922+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95600aab80 con 0x7f957006abe0 2024-04-25T10:00:52.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.922+0000 7f956f7fe640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f955c06b4d0 0x7f955c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:52.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.922+0000 7f956f7fe640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f955c06b4d0 0x7f955c06d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f95701539c0 tx=0x7f9564009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:52.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:52.926+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f956004c030 con 0x7f957006abe0 2024-04-25T10:00:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.095+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f957006b1d0 con 0x7f957006abe0 2024-04-25T10:00:53.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.096+0000 7f956cff9640 1 -- 172.21.15.148:0/3115761958 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9560079870 con 0x7f957006abe0 2024-04-25T10:00:53.100 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f955c06b4d0 msgr2=0x7f955c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f955c06b4d0 0x7f955c06d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f95701539c0 tx=0x7f9564009190 comp rx=0 tx=0).stop 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 msgr2=0x7f9570147130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 0x7f9570147130 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9560035040 tx=0x7f95600046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 shutdown_connections 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f955c06b4d0 0x7f955c06d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f957015dd80 0x7f95700b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95700ad570 0x7f957015c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 --2- 172.21.15.148:0/3115761958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f957006abe0 0x7f9570147130 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 >> 172.21.15.148:0/3115761958 conn(0x7f95700b5af0 msgr2=0x7f95700ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:53.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 shutdown_connections 2024-04-25T10:00:53.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.098+0000 7f95766ac640 1 -- 172.21.15.148:0/3115761958 wait complete. 2024-04-25T10:00:53.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:00:53.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.209+0000 7f21d737e640 1 Processor -- start 2024-04-25T10:00:53.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.209+0000 7f21d737e640 1 -- start start 2024-04-25T10:00:53.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d737e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21d00cd4c0 0x7f21d00cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d737e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d0151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d737e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d015e6e0 con 0x7f21d015b1c0 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d737e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d0151560 con 0x7f21d00cd4c0 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d737e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d015e9c0 con 0x7f21d0150b90 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d591b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d0151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d511a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21d00cd4c0 0x7f21d00cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d591b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d0151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:40274/0 (socket says 172.21.15.148:40274) 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d591b640 1 -- 172.21.15.148:0/3617745541 learned_addr learned my addr 172.21.15.148:0/3617745541 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d4919640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f21d015b1c0 0x7f21d015dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34510/0 (socket says 172.21.15.148:34510) 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d511a640 1 -- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 msgr2=0x7f21d0151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d511a640 1 --2- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d0151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d511a640 1 -- 172.21.15.148:0/3617745541 >> v1:172.21.15.188:6789/0 conn(0x7f21d015b1c0 legacy=0x7f21d015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.210+0000 7f21d511a640 1 -- 172.21.15.148:0/3617745541 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d015a9c0 con 0x7f21d00cd4c0 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.211+0000 7f21d511a640 1 --2- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00cd8a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097a0 tx=0x7f21cc02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf1cb3eb8e7738f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.211+0000 7f21c7fff640 1 -- 172.21.15.148:0/3617745541 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc037070 con 0x7f21d00cd4c0 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.211+0000 7f21c7fff640 1 -- 172.21.15.148:0/3617745541 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc02ad10 con 0x7f21d00cd4c0 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.211+0000 7f21c7fff640 1 -- 172.21.15.148:0/3617745541 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc032660 con 0x7f21d00cd4c0 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 -- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 msgr2=0x7f21d00cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 --2- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00cd8a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097a0 tx=0x7f21cc02a080 comp rx=0 tx=0).stop 2024-04-25T10:00:53.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 -- 172.21.15.148:0/3617745541 shutdown_connections 2024-04-25T10:00:53.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 --2- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d0151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 --2- 172.21.15.148:0/3617745541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00cd8a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 -- 172.21.15.148:0/3617745541 >> 172.21.15.148:0/3617745541 conn(0x7f21d006abe0 msgr2=0x7f21d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:53.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 -- 172.21.15.148:0/3617745541 shutdown_connections 2024-04-25T10:00:53.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.212+0000 7f21d737e640 1 -- 172.21.15.148:0/3617745541 wait complete. 2024-04-25T10:00:53.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.213+0000 7f21d737e640 1 Processor -- start 2024-04-25T10:00:53.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.213+0000 7f21d737e640 1 -- start start 2024-04-25T10:00:53.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d737e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d737e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d013f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d737e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 0x7f21d00b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d737e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d015e370 con 0x7f21d015b1c0 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d737e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d015a670 con 0x7f21d00cd4c0 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d737e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d006eca0 con 0x7f21d0150b90 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 0x7f21d00b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 0x7f21d00b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60354/0 (socket says 172.21.15.148:60354) 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 -- 172.21.15.148:0/1738158064 learned_addr learned my addr 172.21.15.148:0/1738158064 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d511a640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d4919640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d013f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 -- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 msgr2=0x7f21d013f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d013f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 -- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 msgr2=0x7f21d00b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 -- 172.21.15.148:0/1738158064 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d013fc10 con 0x7f21d015b1c0 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d511a640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:53.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21d591b640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 0x7f21d00b1090 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f21c800efc0 tx=0x7f21c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800cd20 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.214+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8003990 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.215+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d0154fb0 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.215+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8003d80 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.215+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d013d3c0 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.216+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d0073a00 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.216+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21c8005d10 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.216+0000 7f21c5ffb640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21b406b4d0 0x7f21b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.217+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21c808a9c0 con 0x7f21d015b1c0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.217+0000 7f21d511a640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21b406b4d0 0x7f21b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:53.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.217+0000 7f21d511a640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21b406b4d0 0x7f21b406d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f21cc034000 tx=0x7f21cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:53.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.221+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21c808ce20 con 0x7f21d015b1c0 2024-04-25T10:00:53.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.402+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f21d00ccb10 con 0x7f21d015b1c0 2024-04-25T10:00:53.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.402+0000 7f21c5ffb640 1 -- 172.21.15.148:0/1738158064 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) v1 ==== 80+0+0 (secure 0 0 0) 0x7f21c8022020 con 0x7f21d015b1c0 2024-04-25T10:00:53.407 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:00:53.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21b406b4d0 msgr2=0x7f21b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21b406b4d0 0x7f21b406d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f21cc034000 tx=0x7f21cc02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 msgr2=0x7f21d00b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 0x7f21d00b1090 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f21c800efc0 tx=0x7f21c800c490 comp rx=0 tx=0).stop 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 shutdown_connections 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f21b406b4d0 0x7f21b406d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21d015b1c0 0x7f21d00b1090 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d0150b90 0x7f21d013f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 --2- 172.21.15.148:0/1738158064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d00cd4c0 0x7f21d00b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:53.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 >> 172.21.15.148:0/1738158064 conn(0x7f21d006abe0 msgr2=0x7f21d015c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:53.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 shutdown_connections 2024-04-25T10:00:53.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:53.405+0000 7f21d737e640 1 -- 172.21.15.148:0/1738158064 wait complete. 2024-04-25T10:00:53.420 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:00:53.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:53.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:00:54.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:00:54.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:00:54.029 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:00:54.051 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:00:54.051 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:54.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:54.066 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:00:54.066 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:54.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:00:54.109 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:00:54.109 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:00:54.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:00:54.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:54.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.242+0000 7f4898bd5640 1 Processor -- start 2024-04-25T10:00:54.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.242+0000 7f4898bd5640 1 -- start start 2024-04-25T10:00:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4898bd5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f48940cc8e0 0x7f48941625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4898bd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f48940d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4898bd5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48940cce50 con 0x7f489415dc90 2024-04-25T10:00:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4898bd5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48940d21c0 con 0x7f48940cc8e0 2024-04-25T10:00:54.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4898bd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4894161af0 con 0x7f48940d1810 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f489259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f489415dc90 0x7f489415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44468/0 (socket says 172.21.15.148:44468) 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f489259c640 1 -- 172.21.15.148:0/367353222 learned_addr learned my addr 172.21.15.148:0/367353222 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f488bfff640 1 --2- 172.21.15.148:0/367353222 >> v2:172.21.15.188:3300/0 conn(0x7f48940cc8e0 0x7f48941625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4892d9d640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f48940d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4891d9b640 1 -- 172.21.15.148:0/367353222 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77620807 0 0) 0x7f48940cce50 con 0x7f489415dc90 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4891d9b640 1 -- 172.21.15.148:0/367353222 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4880003520 con 0x7f489415dc90 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f488bfff640 1 -- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 msgr2=0x7f48940d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f488bfff640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f48940d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f488bfff640 1 -- 172.21.15.148:0/367353222 >> v1:172.21.15.148:6789/0 conn(0x7f489415dc90 legacy=0x7f489415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f488bfff640 1 -- 172.21.15.148:0/367353222 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894161dd0 con 0x7f48940cc8e0 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.243+0000 7f4892d9d640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f48940d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.244+0000 7f488bfff640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941625d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f487c0097a0 tx=0x7f487c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f78c4a9a5c1b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.244+0000 7f4891d9b640 1 -- 172.21.15.148:0/367353222 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c037070 con 0x7f48940cc8e0 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.244+0000 7f4891d9b640 1 -- 172.21.15.148:0/367353222 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c02ae50 con 0x7f48940cc8e0 2024-04-25T10:00:54.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.244+0000 7f4891d9b640 1 -- 172.21.15.148:0/367353222 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c032660 con 0x7f48940cc8e0 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 -- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 msgr2=0x7f48941625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941625d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f487c0097a0 tx=0x7f487c02a100 comp rx=0 tx=0).stop 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 -- 172.21.15.148:0/367353222 shutdown_connections 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f48940d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 --2- 172.21.15.148:0/367353222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941625d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 -- 172.21.15.148:0/367353222 >> 172.21.15.148:0/367353222 conn(0x7f489406f590 msgr2=0x7f489406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 -- 172.21.15.148:0/367353222 shutdown_connections 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.245+0000 7f4898bd5640 1 -- 172.21.15.148:0/367353222 wait complete. 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.246+0000 7f4898bd5640 1 Processor -- start 2024-04-25T10:00:54.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.246+0000 7f4898bd5640 1 -- start start 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4898bd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4898bd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f4894148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4898bd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 0x7f4894145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4898bd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489415e070 con 0x7f489415dc90 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4898bd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940cd130 con 0x7f48940cc8e0 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4898bd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489415e1f0 con 0x7f48940d1810 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 0x7f4894145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 0x7f4894145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58398/0 (socket says 172.21.15.148:58398) 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 -- 172.21.15.148:0/1864543390 learned_addr learned my addr 172.21.15.148:0/1864543390 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f488bfff640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f4894148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f489259c640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 -- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 msgr2=0x7f4894148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f4894148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 -- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 msgr2=0x7f48941e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941e74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 -- 172.21.15.148:0/1864543390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894149110 con 0x7f489415dc90 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f488bfff640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f4894148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f489259c640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.247+0000 7f4892d9d640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 0x7f4894145810 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f488c00efc0 tx=0x7f488c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.248+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c018030 con 0x7f489415dc90 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.248+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00cd30 con 0x7f489415dc90 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.248+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4894146220 con 0x7f489415dc90 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.248+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c003bb0 con 0x7f489415dc90 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.248+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48941467e0 con 0x7f489415dc90 2024-04-25T10:00:54.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.249+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4894073990 con 0x7f489415dc90 2024-04-25T10:00:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.249+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f488c01b070 con 0x7f489415dc90 2024-04-25T10:00:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.249+0000 7f488affd640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f487406b4d0 0x7f487406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.250+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f488c08ad80 con 0x7f489415dc90 2024-04-25T10:00:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.250+0000 7f489259c640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f487406b4d0 0x7f487406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.250+0000 7f489259c640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f487406b4d0 0x7f487406d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4884007fd0 tx=0x7f4884009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.254+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f488c010040 con 0x7f489415dc90 2024-04-25T10:00:54.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.459+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48940cf6b0 con 0x7f489415dc90 2024-04-25T10:00:54.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.459+0000 7f488affd640 1 -- 172.21.15.148:0/1864543390 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f488c012340 con 0x7f489415dc90 2024-04-25T10:00:54.464 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:54.464 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:54.465 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:54.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f487406b4d0 msgr2=0x7f487406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f487406b4d0 0x7f487406d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4884007fd0 tx=0x7f4884009040 comp rx=0 tx=0).stop 2024-04-25T10:00:54.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 msgr2=0x7f4894145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 0x7f4894145810 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f488c00efc0 tx=0x7f488c00c490 comp rx=0 tx=0).stop 2024-04-25T10:00:54.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 shutdown_connections 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f487406b4d0 0x7f487406d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f489415dc90 0x7f4894145810 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48940d1810 0x7f4894148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 --2- 172.21.15.148:0/1864543390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48940cc8e0 0x7f48941e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 >> 172.21.15.148:0/1864543390 conn(0x7f489406f590 msgr2=0x7f48940b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 shutdown_connections 2024-04-25T10:00:54.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.462+0000 7f4898bd5640 1 -- 172.21.15.148:0/1864543390 wait complete. 2024-04-25T10:00:54.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 Processor -- start 2024-04-25T10:00:54.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 -- start start 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f508c15dc70 0x7f508c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f508c06ce10 0x7f508c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c06c220 con 0x7f508c09a450 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0d0c90 con 0x7f508c06ce10 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.570+0000 7f50929b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0d0080 con 0x7f508c15dc70 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f5090f4d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f508c06ce10 0x7f508c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f508b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f508c09a450 0x7f508c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44484/0 (socket says 172.21.15.148:44484) 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f508b7fe640 1 -- 172.21.15.148:0/3746387429 learned_addr learned my addr 172.21.15.148:0/3746387429 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f508bfff640 1 --2- 172.21.15.148:0/3746387429 >> v2:172.21.15.188:3300/0 conn(0x7f508c15dc70 0x7f508c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f5090f4d640 1 -- 172.21.15.148:0/3746387429 >> v2:172.21.15.188:3300/0 conn(0x7f508c15dc70 msgr2=0x7f508c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:54.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f5090f4d640 1 --2- 172.21.15.148:0/3746387429 >> v2:172.21.15.188:3300/0 conn(0x7f508c15dc70 0x7f508c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f5090f4d640 1 -- 172.21.15.148:0/3746387429 >> v1:172.21.15.148:6789/0 conn(0x7f508c09a450 legacy=0x7f508c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f5090f4d640 1 -- 172.21.15.148:0/3746387429 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0d0360 con 0x7f508c06ce10 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.571+0000 7f508affd640 1 -- 172.21.15.148:0/3746387429 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032448952 0 0) 0x7f508c06c220 con 0x7f508c09a450 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f5090f4d640 1 --2- 172.21.15.148:0/3746387429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c0cf9c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f508c0726a0 tx=0x7f508402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ee53fddfe2af67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f508affd640 1 -- 172.21.15.148:0/3746387429 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084007740 con 0x7f508c06ce10 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f508affd640 1 -- 172.21.15.148:0/3746387429 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50840078e0 con 0x7f508c06ce10 2024-04-25T10:00:54.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f508affd640 1 -- 172.21.15.148:0/3746387429 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084007cb0 con 0x7f508c06ce10 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 -- 172.21.15.148:0/3746387429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 msgr2=0x7f508c0cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 --2- 172.21.15.148:0/3746387429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c0cf9c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f508c0726a0 tx=0x7f508402a680 comp rx=0 tx=0).stop 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 -- 172.21.15.148:0/3746387429 shutdown_connections 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 --2- 172.21.15.148:0/3746387429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c0cf9c0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 --2- 172.21.15.148:0/3746387429 >> v2:172.21.15.188:3300/0 conn(0x7f508c15dc70 0x7f508c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 -- 172.21.15.148:0/3746387429 >> 172.21.15.148:0/3746387429 conn(0x7f508c0b3270 msgr2=0x7f508c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 -- 172.21.15.148:0/3746387429 shutdown_connections 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.572+0000 7f50929b0640 1 -- 172.21.15.148:0/3746387429 wait complete. 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.573+0000 7f50929b0640 1 Processor -- start 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 -- start start 2024-04-25T10:00:54.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c09a450 0x7f508c1f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f508c15dc70 0x7f508c1f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15e050 con 0x7f508c06ce10 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0d0920 con 0x7f508c15dc70 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f50929b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cbdd0 con 0x7f508c09a450 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f508bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f508bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c09b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58418/0 (socket says 172.21.15.148:58418) 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.574+0000 7f508bfff640 1 -- 172.21.15.148:0/2292932539 learned_addr learned my addr 172.21.15.148:0/2292932539 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f5090f4d640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f508c15dc70 0x7f508c1f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508b7fe640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c09a450 0x7f508c1f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508bfff640 1 -- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c09a450 msgr2=0x7f508c1f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508bfff640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c09a450 0x7f508c1f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508bfff640 1 -- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f508c15dc70 msgr2=0x7f508c1f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508bfff640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f508c15dc70 0x7f508c1f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508bfff640 1 -- 172.21.15.148:0/2292932539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c09e3d0 con 0x7f508c06ce10 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f5090f4d640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f508c15dc70 0x7f508c1f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f508bfff640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c09b5a0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f507c00d620 tx=0x7f507c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00b620 con 0x7f508c06ce10 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c019070 con 0x7f508c06ce10 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00ba00 con 0x7f508c06ce10 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c09db30 con 0x7f508c06ce10 2024-04-25T10:00:54.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.575+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c09e0f0 con 0x7f508c06ce10 2024-04-25T10:00:54.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.576+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c073990 con 0x7f508c06ce10 2024-04-25T10:00:54.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.576+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f507c00bba0 con 0x7f508c06ce10 2024-04-25T10:00:54.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.577+0000 7f5088ff9640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507806b4d0 0x7f507806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.577+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f507c08a740 con 0x7f508c06ce10 2024-04-25T10:00:54.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.577+0000 7f508b7fe640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507806b4d0 0x7f507806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.577+0000 7f508b7fe640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507806b4d0 0x7f507806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f508c06cde0 tx=0x7f5080007bb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.581+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f507c0594c0 con 0x7f508c06ce10 2024-04-25T10:00:54.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:54.624 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:54.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.758+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f508c0b40a0 con 0x7f508c06ce10 2024-04-25T10:00:54.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.759+0000 7f5088ff9640 1 -- 172.21.15.148:0/2292932539 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f507c058dc0 con 0x7f508c06ce10 2024-04-25T10:00:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:54.764 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:27.088834+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507806b4d0 msgr2=0x7f507806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507806b4d0 0x7f507806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f508c06cde0 tx=0x7f5080007bb0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 msgr2=0x7f508c09b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c09b5a0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f507c00d620 tx=0x7f507c00dae0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 shutdown_connections 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f507806b4d0 0x7f507806d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f508c15dc70 0x7f508c1f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c09a450 0x7f508c1f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 --2- 172.21.15.148:0/2292932539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f508c06ce10 0x7f508c09b5a0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 >> 172.21.15.148:0/2292932539 conn(0x7f508c0b3270 msgr2=0x7f508c0caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.761+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 shutdown_connections 2024-04-25T10:00:54.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.762+0000 7f50929b0640 1 -- 172.21.15.148:0/2292932539 wait complete. 2024-04-25T10:00:54.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:00:23.631066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:00:24.401378+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:00:54.778 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:00:54.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.874+0000 7f60828e2640 1 Processor -- start 2024-04-25T10:00:54.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.874+0000 7f60828e2640 1 -- start start 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.874+0000 7f60828e2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f607c161710 0x7f607c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.874+0000 7f60828e2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c15e000 con 0x7f607c0ced80 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.874+0000 7f60828e2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c0d1de0 con 0x7f607c15dc20 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.874+0000 7f60828e2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607c161c60 con 0x7f607c161710 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f6080e7f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f607c161710 0x7f607c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f607b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f607c0ced80 0x7f607c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44486/0 (socket says 172.21.15.148:44486) 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f607b7fe640 1 -- 172.21.15.148:0/2932159434 learned_addr learned my addr 172.21.15.148:0/2932159434 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f6080e7f640 1 -- 172.21.15.148:0/2932159434 >> v1:172.21.15.188:6789/0 conn(0x7f607c15dc20 legacy=0x7f607c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f6080e7f640 1 -- 172.21.15.148:0/2932159434 >> v1:172.21.15.148:6789/0 conn(0x7f607c0ced80 legacy=0x7f607c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f6080e7f640 1 -- 172.21.15.148:0/2932159434 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c161f40 con 0x7f607c161710 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.875+0000 7f6080e7f640 1 --2- 172.21.15.148:0/2932159434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c0d29f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6074009d90 tx=0x7f607402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c539e6bba3ed6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f607affd640 1 -- 172.21.15.148:0/2932159434 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074007740 con 0x7f607c161710 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f607affd640 1 -- 172.21.15.148:0/2932159434 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60740078e0 con 0x7f607c161710 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f607affd640 1 -- 172.21.15.148:0/2932159434 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074007cb0 con 0x7f607c161710 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 -- 172.21.15.148:0/2932159434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 msgr2=0x7f607c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 --2- 172.21.15.148:0/2932159434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c0d29f0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6074009d90 tx=0x7f607402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 -- 172.21.15.148:0/2932159434 shutdown_connections 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 --2- 172.21.15.148:0/2932159434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c0d29f0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 -- 172.21.15.148:0/2932159434 >> 172.21.15.148:0/2932159434 conn(0x7f607c0d0420 msgr2=0x7f607c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 -- 172.21.15.148:0/2932159434 shutdown_connections 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.876+0000 7f60828e2640 1 -- 172.21.15.148:0/2932159434 wait complete. 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.877+0000 7f60828e2640 1 Processor -- start 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 -- start start 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f607c0ced80 0x7f607c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607c15dc20 0x7f607c1e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0b5e50 con 0x7f607c161710 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c162500 con 0x7f607c0ced80 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f60828e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0b5fd0 con 0x7f607c15dc20 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f6080e7f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f6080e7f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58440/0 (socket says 172.21.15.148:58440) 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.878+0000 7f6080e7f640 1 -- 172.21.15.148:0/3410118776 learned_addr learned my addr 172.21.15.148:0/3410118776 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6080e7f640 1 -- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607c15dc20 msgr2=0x7f607c1e9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6080e7f640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607c15dc20 0x7f607c1e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f607bfff640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f607c0ced80 0x7f607c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6080e7f640 1 -- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f607c0ced80 msgr2=0x7f607c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6080e7f640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f607c0ced80 0x7f607c160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6080e7f640 1 -- 172.21.15.148:0/3410118776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c161020 con 0x7f607c161710 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f607bfff640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f607c0ced80 0x7f607c160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6080e7f640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c15e210 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6074009eb0 tx=0x7f6074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074040070 con 0x7f607c161710 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6074031bc0 con 0x7f607c161710 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c0c96e0 con 0x7f607c161710 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607403c420 con 0x7f607c161710 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.879+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1e7890 con 0x7f607c161710 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.880+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c073a00 con 0x7f607c161710 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.881+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6074039040 con 0x7f607c161710 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.881+0000 7f6078ff9640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f606806b4d0 0x7f606806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.881+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60740ab8e0 con 0x7f607c161710 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.881+0000 7f607bfff640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f606806b4d0 0x7f606806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.881+0000 7f607bfff640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f606806b4d0 0x7f606806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f607c1f7090 tx=0x7f606c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:54.885+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f607c161020 con 0x7f607c161710 2024-04-25T10:00:55.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.056+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f607c0d16b0 con 0x7f607c161710 2024-04-25T10:00:55.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.056+0000 7f6078ff9640 1 -- 172.21.15.148:0/3410118776 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f607407a040 con 0x7f607c161710 2024-04-25T10:00:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f606806b4d0 msgr2=0x7f606806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f606806b4d0 0x7f606806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f607c1f7090 tx=0x7f606c009040 comp rx=0 tx=0).stop 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 msgr2=0x7f607c15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c15e210 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6074009eb0 tx=0x7f6074004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 shutdown_connections 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f606806b4d0 0x7f606806d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c161710 0x7f607c15e210 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f607c15dc20 0x7f607c1e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 --2- 172.21.15.148:0/3410118776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f607c0ced80 0x7f607c160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 >> 172.21.15.148:0/3410118776 conn(0x7f607c0d0420 msgr2=0x7f607c0d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 shutdown_connections 2024-04-25T10:00:55.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.058+0000 7f60828e2640 1 -- 172.21.15.148:0/3410118776 wait complete. 2024-04-25T10:00:55.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:00:55.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.170+0000 7f7242313640 1 Processor -- start 2024-04-25T10:00:55.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.170+0000 7f7242313640 1 -- start start 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f7242313640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f723c158230 0x7f723c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f7242313640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f723c0b4e60 0x7f723c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f7242313640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723c0d2020 con 0x7f723c0d1b80 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f7242313640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c158610 con 0x7f723c0b4e60 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f7242313640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c0b57c0 con 0x7f723c158230 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f723c0b4e60 0x7f723c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f723c0b4e60 0x7f723c0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58456/0 (socket says 172.21.15.148:58456) 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 -- 172.21.15.148:0/3734571491 learned_addr learned my addr 172.21.15.148:0/3734571491 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f723bfff640 1 --2- 172.21.15.148:0/3734571491 >> v2:172.21.15.188:3300/0 conn(0x7f723c158230 0x7f723c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 -- 172.21.15.148:0/3734571491 >> v2:172.21.15.188:3300/0 conn(0x7f723c158230 msgr2=0x7f723c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 --2- 172.21.15.148:0/3734571491 >> v2:172.21.15.188:3300/0 conn(0x7f723c158230 0x7f723c0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 -- 172.21.15.148:0/3734571491 >> v1:172.21.15.188:6789/0 conn(0x7f723c0d1b80 legacy=0x7f723c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.171+0000 7f72408b0640 1 -- 172.21.15.148:0/3734571491 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c0d2300 con 0x7f723c0b4e60 2024-04-25T10:00:55.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.172+0000 7f72408b0640 1 --2- 172.21.15.148:0/3734571491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c0b4e60 0x7f723c0b5280 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f723c070460 tx=0x7f723402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4664c371e92523c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.175+0000 7f723affd640 1 -- 172.21.15.148:0/3734571491 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234036070 con 0x7f723c0b4e60 2024-04-25T10:00:55.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.175+0000 7f723affd640 1 -- 172.21.15.148:0/3734571491 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234031600 con 0x7f723c0b4e60 2024-04-25T10:00:55.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.175+0000 7f723affd640 1 -- 172.21.15.148:0/3734571491 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340319d0 con 0x7f723c0b4e60 2024-04-25T10:00:55.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 -- 172.21.15.148:0/3734571491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c0b4e60 msgr2=0x7f723c0b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 --2- 172.21.15.148:0/3734571491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c0b4e60 0x7f723c0b5280 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f723c070460 tx=0x7f723402a730 comp rx=0 tx=0).stop 2024-04-25T10:00:55.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 -- 172.21.15.148:0/3734571491 shutdown_connections 2024-04-25T10:00:55.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 --2- 172.21.15.148:0/3734571491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c0b4e60 0x7f723c0b5280 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 --2- 172.21.15.148:0/3734571491 >> v2:172.21.15.188:3300/0 conn(0x7f723c158230 0x7f723c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 -- 172.21.15.148:0/3734571491 >> 172.21.15.148:0/3734571491 conn(0x7f723c061600 msgr2=0x7f723c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 -- 172.21.15.148:0/3734571491 shutdown_connections 2024-04-25T10:00:55.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.176+0000 7f7242313640 1 -- 172.21.15.148:0/3734571491 wait complete. 2024-04-25T10:00:55.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.177+0000 7f7242313640 1 Processor -- start 2024-04-25T10:00:55.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.177+0000 7f7242313640 1 -- start start 2024-04-25T10:00:55.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f7242313640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f723c0b4e60 0x7f723c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f7242313640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f723c0d1b80 0x7f723c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f7242313640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 0x7f723c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f7242313640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0d25e0 con 0x7f723c158230 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f7242313640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c06bea0 con 0x7f723c0b4e60 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f7242313640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c06ca40 con 0x7f723c0d1b80 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 0x7f723c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 0x7f723c0cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58468/0 (socket says 172.21.15.148:58468) 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 -- 172.21.15.148:0/919559764 learned_addr learned my addr 172.21.15.148:0/919559764 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f723b7fe640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f723c0d1b80 0x7f723c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f723bfff640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f723c0b4e60 0x7f723c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 -- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f723c0d1b80 msgr2=0x7f723c156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f723c0d1b80 0x7f723c156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 -- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f723c0b4e60 msgr2=0x7f723c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f723c0b4e60 0x7f723c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.178+0000 7f72408b0640 1 -- 172.21.15.148:0/919559764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c0cc2e0 con 0x7f723c158230 2024-04-25T10:00:55.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.179+0000 7f72408b0640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 0x7f723c0cd380 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7234009f40 tx=0x7f7234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.180+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234036050 con 0x7f723c158230 2024-04-25T10:00:55.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.180+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234042070 con 0x7f723c158230 2024-04-25T10:00:55.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.180+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234009510 con 0x7f723c158230 2024-04-25T10:00:55.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.180+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723c0cc5c0 con 0x7f723c158230 2024-04-25T10:00:55.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.180+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723c0cce60 con 0x7f723c158230 2024-04-25T10:00:55.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.181+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723c0cc790 con 0x7f723c158230 2024-04-25T10:00:55.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.182+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7234007bf0 con 0x7f723c158230 2024-04-25T10:00:55.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.182+0000 7f7238ff9640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f722806b4d0 0x7f722806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.182+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72340aaf30 con 0x7f723c158230 2024-04-25T10:00:55.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.182+0000 7f723bfff640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f722806b4d0 0x7f722806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.183+0000 7f723bfff640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f722806b4d0 0x7f722806d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f722c009aa0 tx=0x7f722c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.186+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72340795d0 con 0x7f723c158230 2024-04-25T10:00:55.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.355+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f723c073a00 con 0x7f723c158230 2024-04-25T10:00:55.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.355+0000 7f7238ff9640 1 -- 172.21.15.148:0/919559764 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f723407d670 con 0x7f723c158230 2024-04-25T10:00:55.360 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f722806b4d0 msgr2=0x7f722806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f722806b4d0 0x7f722806d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f722c009aa0 tx=0x7f722c009190 comp rx=0 tx=0).stop 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 msgr2=0x7f723c0cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 0x7f723c0cd380 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7234009f40 tx=0x7f7234004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 shutdown_connections 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f722806b4d0 0x7f722806d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f723c158230 0x7f723c0cd380 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f723c0d1b80 0x7f723c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 --2- 172.21.15.148:0/919559764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f723c0b4e60 0x7f723c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 >> 172.21.15.148:0/919559764 conn(0x7f723c061600 msgr2=0x7f723c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 shutdown_connections 2024-04-25T10:00:55.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.357+0000 7f7242313640 1 -- 172.21.15.148:0/919559764 wait complete. 2024-04-25T10:00:55.373 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:00:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.468+0000 7f940325d640 1 Processor -- start 2024-04-25T10:00:55.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.468+0000 7f940325d640 1 -- start start 2024-04-25T10:00:55.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f940325d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93fc0b5570 0x7f93fc06f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f940325d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc0d1d50 con 0x7f93fc158230 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f940325d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc0d2020 con 0x7f93fc06c330 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f940325d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc158610 con 0x7f93fc0b5570 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f9400ff9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f93fc158230 0x7f93fc0d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44488/0 (socket says 172.21.15.148:44488) 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f9400ff9640 1 -- 172.21.15.148:0/1219267371 learned_addr learned my addr 172.21.15.148:0/1219267371 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092614017 0 0) 0x7f93fc0d1d50 con 0x7f93fc158230 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93ec003520 con 0x7f93fc158230 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f94017fa640 1 --2- 172.21.15.148:0/1219267371 >> v2:172.21.15.188:3300/0 conn(0x7f93fc0b5570 0x7f93fc06f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378907844 0 0) 0x7f93ec003520 con 0x7f93fc158230 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 >> v2:172.21.15.188:3300/0 conn(0x7f93fc0b5570 msgr2=0x7f93fc06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 --2- 172.21.15.148:0/1219267371 >> v2:172.21.15.188:3300/0 conn(0x7f93fc0b5570 0x7f93fc06f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 >> v1:172.21.15.188:6789/0 conn(0x7f93fc06c330 legacy=0x7f93fc0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0d2300 con 0x7f93fc158230 2024-04-25T10:00:55.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f93f0001670 con 0x7f93fc158230 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f94017fa640 1 --2- 172.21.15.148:0/1219267371 >> v2:172.21.15.188:3300/0 conn(0x7f93fc0b5570 0x7f93fc06f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 >> v1:172.21.15.148:6789/0 conn(0x7f93fc158230 legacy=0x7f93fc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.469+0000 7f93fb7fe640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93ec0039b0 0x7f93ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f93fb7fe640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f93fb7fe640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec0051c0 0x7f93ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc158610 con 0x7f93ec0051c0 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec00c110 con 0x7f93ec004800 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec00c310 con 0x7f93ec0039b0 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f94017fa640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec0051c0 0x7f93ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f9400ff9640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93ec0039b0 0x7f93ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f93fbfff640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f94017fa640 1 -- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93ec0039b0 msgr2=0x7f93ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f94017fa640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93ec0039b0 0x7f93ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f94017fa640 1 -- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec004800 msgr2=0x7f93ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f94017fa640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec004800 0x7f93ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f94017fa640 1 -- 172.21.15.148:0/1219267371 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec003520 con 0x7f93ec0051c0 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.470+0000 7f9400ff9640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93ec0039b0 0x7f93ec003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f94017fa640 1 -- 172.21.15.148:0/1219267371 reap_dead start 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f94017fa640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec0051c0 0x7f93ec00b9c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f93f40073e0 tx=0x7f93ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c44da19e784c55a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4009c80 con 0x7f93ec0051c0 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f402c6c0 con 0x7f93ec0051c0 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f93fb7fe640 1 -- 172.21.15.148:0/1219267371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f402cab0 con 0x7f93ec0051c0 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 -- 172.21.15.148:0/1219267371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec0051c0 msgr2=0x7f93ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec0051c0 0x7f93ec00b9c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f93f40073e0 tx=0x7f93ec005830 comp rx=0 tx=0).stop 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 -- 172.21.15.148:0/1219267371 shutdown_connections 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 --2- 172.21.15.148:0/1219267371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ec0051c0 0x7f93ec00b9c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 -- 172.21.15.148:0/1219267371 >> 172.21.15.148:0/1219267371 conn(0x7f93fc061600 msgr2=0x7f93fc0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 -- 172.21.15.148:0/1219267371 shutdown_connections 2024-04-25T10:00:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.471+0000 7f940325d640 1 -- 172.21.15.148:0/1219267371 wait complete. 2024-04-25T10:00:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.472+0000 7f940325d640 1 Processor -- start 2024-04-25T10:00:55.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 -- start start 2024-04-25T10:00:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec0051c0 0x7f93fc153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 0x7f93fc158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93fc158cd0 0x7f93fc0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0d25e0 con 0x7f93fc156690 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06bea0 con 0x7f93ec0051c0 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f940325d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc15e320 con 0x7f93fc158cd0 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 0x7f93fc158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 0x7f93fc158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58480/0 (socket says 172.21.15.148:58480) 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.473+0000 7f93fbfff640 1 -- 172.21.15.148:0/2470235178 learned_addr learned my addr 172.21.15.148:0/2470235178 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f9400ff9640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec0051c0 0x7f93fc153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f94017fa640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93fc158cd0 0x7f93fc0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93fbfff640 1 -- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93fc158cd0 msgr2=0x7f93fc0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93fbfff640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93fc158cd0 0x7f93fc0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93fbfff640 1 -- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec0051c0 msgr2=0x7f93fc153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93fbfff640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec0051c0 0x7f93fc153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93fbfff640 1 -- 172.21.15.148:0/2470235178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc1591e0 con 0x7f93fc156690 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f94017fa640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93fc158cd0 0x7f93fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:55.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93fbfff640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 0x7f93fc158790 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f93e8001f00 tx=0x7f93e800fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e801a070 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e8005320 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc1594c0 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8006a40 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.474+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc159d60 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.475+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc1598b0 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.476+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93e801d070 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.476+0000 7f93f97fa640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f93e006b500 0x7f93e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.476+0000 7f9400ff9640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f93e006b500 0x7f93e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.476+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93e8089b30 con 0x7f93fc156690 2024-04-25T10:00:55.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.477+0000 7f9400ff9640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f93e006b500 0x7f93e006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f93f00059b0 tx=0x7f93f0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.480+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93e8058820 con 0x7f93fc156690 2024-04-25T10:00:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.650+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93fc073a00 con 0x7f93fc156690 2024-04-25T10:00:55.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.650+0000 7f93f97fa640 1 -- 172.21.15.148:0/2470235178 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93e8058120 con 0x7f93fc156690 2024-04-25T10:00:55.655 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f93e006b500 msgr2=0x7f93e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f93e006b500 0x7f93e006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f93f00059b0 tx=0x7f93f0009040 comp rx=0 tx=0).stop 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 msgr2=0x7f93fc158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 0x7f93fc158790 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f93e8001f00 tx=0x7f93e800fa50 comp rx=0 tx=0).stop 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 shutdown_connections 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f93e006b500 0x7f93e006d9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93fc158cd0 0x7f93fc0cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93fc156690 0x7f93fc158790 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 --2- 172.21.15.148:0/2470235178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec0051c0 0x7f93fc153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 >> 172.21.15.148:0/2470235178 conn(0x7f93fc061600 msgr2=0x7f93fc0b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 shutdown_connections 2024-04-25T10:00:55.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.652+0000 7f940325d640 1 -- 172.21.15.148:0/2470235178 wait complete. 2024-04-25T10:00:55.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:55.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.761+0000 7f109da93640 1 Processor -- start 2024-04-25T10:00:55.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.761+0000 7f109da93640 1 -- start start 2024-04-25T10:00:55.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f109da93640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f109815dc90 0x7f10981625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f109da93640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10980cc8e0 0x7f109815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f109da93640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109815f390 con 0x7f10980d1810 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f109da93640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10980cce50 con 0x7f109815dc90 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f109da93640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1098161af0 con 0x7f10980cc8e0 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10980cc8e0 0x7f109815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f109815dc90 0x7f10981625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f109815dc90 0x7f10981625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:54388/0 (socket says 172.21.15.148:54388) 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10980cc8e0 0x7f109815ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:39222/0 (socket says 172.21.15.148:39222) 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f1097fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f10980d1810 0x7f10980cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34238/0 (socket says 172.21.15.148:34238) 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 -- 172.21.15.148:0/143481020 learned_addr learned my addr 172.21.15.148:0/143481020 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 -- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10980cc8e0 msgr2=0x7f109815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 --2- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10980cc8e0 0x7f109815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 -- 172.21.15.148:0/143481020 >> v1:172.21.15.188:6789/0 conn(0x7f10980d1810 legacy=0x7f10980cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10977fe640 1 -- 172.21.15.148:0/143481020 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098161dd0 con 0x7f109815dc90 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.762+0000 7f10967fc640 1 -- 172.21.15.148:0/143481020 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233258910 0 0) 0x7f1090001490 con 0x7f10980d1810 2024-04-25T10:00:55.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.763+0000 7f10977fe640 1 --2- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f109815dc90 0x7f10981625d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1088009870 tx=0x7f108802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b0e0efbe3e9be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.763+0000 7f10967fc640 1 -- 172.21.15.148:0/143481020 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088037070 con 0x7f109815dc90 2024-04-25T10:00:55.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.763+0000 7f10967fc640 1 -- 172.21.15.148:0/143481020 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1088032440 con 0x7f109815dc90 2024-04-25T10:00:55.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.763+0000 7f10967fc640 1 -- 172.21.15.148:0/143481020 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088032810 con 0x7f109815dc90 2024-04-25T10:00:55.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 -- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f109815dc90 msgr2=0x7f10981625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 --2- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f109815dc90 0x7f10981625d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1088009870 tx=0x7f108802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 -- 172.21.15.148:0/143481020 shutdown_connections 2024-04-25T10:00:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 --2- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10980cc8e0 0x7f109815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 --2- 172.21.15.148:0/143481020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f109815dc90 0x7f10981625d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 -- 172.21.15.148:0/143481020 >> 172.21.15.148:0/143481020 conn(0x7f109806f590 msgr2=0x7f109806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 -- 172.21.15.148:0/143481020 shutdown_connections 2024-04-25T10:00:55.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.764+0000 7f109da93640 1 -- 172.21.15.148:0/143481020 wait complete. 2024-04-25T10:00:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.765+0000 7f109da93640 1 Processor -- start 2024-04-25T10:00:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.765+0000 7f109da93640 1 -- start start 2024-04-25T10:00:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f109da93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10980cc8e0 0x7f10980b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f109da93640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 0x7f10980ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f109da93640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109815dc90 0x7f10980c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f109da93640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109815e070 con 0x7f10980d1810 2024-04-25T10:00:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f109da93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980cd130 con 0x7f10980cc8e0 2024-04-25T10:00:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f109da93640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109815e1f0 con 0x7f109815dc90 2024-04-25T10:00:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 0x7f10980ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 0x7f10980ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58496/0 (socket says 172.21.15.148:58496) 2024-04-25T10:00:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 -- 172.21.15.148:0/3795671979 learned_addr learned my addr 172.21.15.148:0/3795671979 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f10977fe640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10980cc8e0 0x7f10980b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1097fff640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109815dc90 0x7f10980c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 -- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109815dc90 msgr2=0x7f10980c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109815dc90 0x7f10980c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 -- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10980cc8e0 msgr2=0x7f10980b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10980cc8e0 0x7f10980b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.766+0000 7f1096ffd640 1 -- 172.21.15.148:0/3795671979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980c8e30 con 0x7f10980d1810 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.767+0000 7f1096ffd640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 0x7f10980ae0b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e870 tx=0x7f108c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.767+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c017070 con 0x7f10980d1810 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.767+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c00c940 con 0x7f10980d1810 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.767+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10980c9380 con 0x7f10980d1810 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.767+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00cd30 con 0x7f10980d1810 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.767+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10980c9940 con 0x7f10980d1810 2024-04-25T10:00:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.768+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098073990 con 0x7f10980d1810 2024-04-25T10:00:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.768+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f108c01a070 con 0x7f10980d1810 2024-04-25T10:00:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.769+0000 7f107ffff640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108406b4d0 0x7f108406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.769+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f108c08a110 con 0x7f10980d1810 2024-04-25T10:00:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.769+0000 7f10977fe640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108406b4d0 0x7f108406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.769+0000 7f10977fe640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108406b4d0 0x7f108406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1088034000 tx=0x7f108802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:55.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.773+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f108c058840 con 0x7f10980d1810 2024-04-25T10:00:55.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.980+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10980cc070 con 0x7f10980d1810 2024-04-25T10:00:55.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.980+0000 7f107ffff640 1 -- 172.21.15.148:0/3795671979 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f108c05c8e0 con 0x7f10980d1810 2024-04-25T10:00:55.985 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:55.985 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:55.985 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108406b4d0 msgr2=0x7f108406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108406b4d0 0x7f108406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1088034000 tx=0x7f108802e000 comp rx=0 tx=0).stop 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 msgr2=0x7f10980ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 0x7f10980ae0b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f108c00e870 tx=0x7f108c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 shutdown_connections 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f108406b4d0 0x7f108406d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f109815dc90 0x7f10980c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10980d1810 0x7f10980ae0b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 --2- 172.21.15.148:0/3795671979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10980cc8e0 0x7f10980b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:55.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 >> 172.21.15.148:0/3795671979 conn(0x7f109806f590 msgr2=0x7f109815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:55.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 shutdown_connections 2024-04-25T10:00:55.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:55.982+0000 7f109da93640 1 -- 172.21.15.148:0/3795671979 wait complete. 2024-04-25T10:00:55.998 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:56.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.091+0000 7f0440fc1640 1 Processor -- start 2024-04-25T10:00:56.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.091+0000 7f0440fc1640 1 -- start start 2024-04-25T10:00:56.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0440fc1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043c0ce330 0x7f043c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0440fc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0700a0 0x7f043c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0440fc1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c1626e0 con 0x7f043c06abe0 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0440fc1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c0cd270 con 0x7f043c0ce330 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0440fc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0cd550 con 0x7f043c0700a0 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0439d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043c0ce330 0x7f043c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0439d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f043c0ce330 0x7f043c06fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58506/0 (socket says 172.21.15.148:58506) 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f0439d9b640 1 -- 172.21.15.148:0/1249339400 learned_addr learned my addr 172.21.15.148:0/1249339400 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.092+0000 7f043ad9d640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0700a0 0x7f043c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f0439d9b640 1 -- 172.21.15.148:0/1249339400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0700a0 msgr2=0x7f043c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f0439d9b640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0700a0 0x7f043c15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f0439d9b640 1 -- 172.21.15.148:0/1249339400 >> v1:172.21.15.188:6789/0 conn(0x7f043c06abe0 legacy=0x7f043c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f0439d9b640 1 -- 172.21.15.148:0/1249339400 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c0cd830 con 0x7f043c0ce330 2024-04-25T10:00:56.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f043ad9d640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0700a0 0x7f043c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:56.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f0439d9b640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c0ce330 0x7f043c06fb60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0430009870 tx=0x7f043002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a76db5e389e8aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.093+0000 7f043959a640 1 -- 172.21.15.148:0/1249339400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430037070 con 0x7f043c0ce330 2024-04-25T10:00:56.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f043959a640 1 -- 172.21.15.148:0/1249339400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430032440 con 0x7f043c0ce330 2024-04-25T10:00:56.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f043959a640 1 -- 172.21.15.148:0/1249339400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430032810 con 0x7f043c0ce330 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 -- 172.21.15.148:0/1249339400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c0ce330 msgr2=0x7f043c06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c0ce330 0x7f043c06fb60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0430009870 tx=0x7f043002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 -- 172.21.15.148:0/1249339400 shutdown_connections 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c0ce330 0x7f043c06fb60 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 --2- 172.21.15.148:0/1249339400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0700a0 0x7f043c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 -- 172.21.15.148:0/1249339400 >> 172.21.15.148:0/1249339400 conn(0x7f043c0b2220 msgr2=0x7f043c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 -- 172.21.15.148:0/1249339400 shutdown_connections 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.094+0000 7f0440fc1640 1 -- 172.21.15.148:0/1249339400 wait complete. 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 Processor -- start 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 -- start start 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 0x7f043c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f043c0700a0 0x7f043c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0ce330 0x7f043c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c15d8b0 con 0x7f043c06abe0 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c161780 con 0x7f043c0700a0 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f0440fc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c15da30 con 0x7f043c0ce330 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f043a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 0x7f043c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f043a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 0x7f043c15ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58514/0 (socket says 172.21.15.148:58514) 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.096+0000 7f043a59c640 1 -- 172.21.15.148:0/241257283 learned_addr learned my addr 172.21.15.148:0/241257283 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f0439d9b640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f043c0700a0 0x7f043c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043ad9d640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0ce330 0x7f043c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043a59c640 1 -- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0ce330 msgr2=0x7f043c0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043a59c640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0ce330 0x7f043c0ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043a59c640 1 -- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f043c0700a0 msgr2=0x7f043c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043a59c640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f043c0700a0 0x7f043c0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043a59c640 1 -- 172.21.15.148:0/241257283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c09dda0 con 0x7f043c06abe0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f043a59c640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 0x7f043c15ccc0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f042c00c400 tx=0x7f042c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c0052d0 con 0x7f043c06abe0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c005490 con 0x7f043c06abe0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c15ec40 con 0x7f043c06abe0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c007260 con 0x7f043c06abe0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.097+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c15f200 con 0x7f043c06abe0 2024-04-25T10:00:56.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.098+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c073990 con 0x7f043c06abe0 2024-04-25T10:00:56.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.099+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f042c007400 con 0x7f043c06abe0 2024-04-25T10:00:56.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.099+0000 7f042affd640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f041c06b4d0 0x7f041c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.099+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f042c0897a0 con 0x7f043c06abe0 2024-04-25T10:00:56.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.099+0000 7f0439d9b640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f041c06b4d0 0x7f041c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.099+0000 7f0439d9b640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f041c06b4d0 0x7f041c06d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0430034000 tx=0x7f043002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.103+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f042c057ed0 con 0x7f043c06abe0 2024-04-25T10:00:56.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.278+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f043c0b2ff0 con 0x7f043c06abe0 2024-04-25T10:00:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.279+0000 7f042affd640 1 -- 172.21.15.148:0/241257283 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f042c05bf70 con 0x7f043c06abe0 2024-04-25T10:00:56.284 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:56.284 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:27.088834+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f041c06b4d0 msgr2=0x7f041c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f041c06b4d0 0x7f041c06d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0430034000 tx=0x7f043002e000 comp rx=0 tx=0).stop 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 msgr2=0x7f043c15ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 0x7f043c15ccc0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f042c00c400 tx=0x7f042c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 shutdown_connections 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f041c06b4d0 0x7f041c06d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f043c0ce330 0x7f043c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f043c0700a0 0x7f043c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 --2- 172.21.15.148:0/241257283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f043c06abe0 0x7f043c15ccc0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 >> 172.21.15.148:0/241257283 conn(0x7f043c0b2220 msgr2=0x7f043c0aeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 shutdown_connections 2024-04-25T10:00:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.281+0000 7f0440fc1640 1 -- 172.21.15.148:0/241257283 wait complete. 2024-04-25T10:00:56.297 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:00:56.297 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:56.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.390+0000 7f3b9005a640 1 Processor -- start 2024-04-25T10:00:56.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.391+0000 7f3b9005a640 1 -- start start 2024-04-25T10:00:56.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.391+0000 7f3b9005a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b88154ed0 0x7f3b880aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.391+0000 7f3b9005a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b880ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.391+0000 7f3b9005a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8815ec60 con 0x7f3b8806c1d0 2024-04-25T10:00:56.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.391+0000 7f3b9005a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8815ef40 con 0x7f3b88154ed0 2024-04-25T10:00:56.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.391+0000 7f3b9005a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8806db90 con 0x7f3b88071ac0 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b88154ed0 0x7f3b880aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b88154ed0 0x7f3b880aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58526/0 (socket says 172.21.15.148:58526) 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 -- 172.21.15.148:0/1644432880 learned_addr learned my addr 172.21.15.148:0/1644432880 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8d5f5640 1 --2- 172.21.15.148:0/1644432880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b880ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 -- 172.21.15.148:0/1644432880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 msgr2=0x7f3b880ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 --2- 172.21.15.148:0/1644432880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b880ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 -- 172.21.15.148:0/1644432880 >> v1:172.21.15.188:6789/0 conn(0x7f3b8806c1d0 legacy=0x7f3b881617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.392+0000 7f3b8ddf6640 1 -- 172.21.15.148:0/1644432880 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8806de70 con 0x7f3b88154ed0 2024-04-25T10:00:56.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b8ddf6640 1 --2- 172.21.15.148:0/1644432880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b88154ed0 0x7f3b880aced0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009870 tx=0x7f3b7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cc75c2d7600f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b8cdf4640 1 -- 172.21.15.148:0/1644432880 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037070 con 0x7f3b88154ed0 2024-04-25T10:00:56.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b8cdf4640 1 -- 172.21.15.148:0/1644432880 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c032440 con 0x7f3b88154ed0 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b8cdf4640 1 -- 172.21.15.148:0/1644432880 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c032810 con 0x7f3b88154ed0 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b9005a640 1 -- 172.21.15.148:0/1644432880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b88154ed0 msgr2=0x7f3b880aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b9005a640 1 --2- 172.21.15.148:0/1644432880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b88154ed0 0x7f3b880aced0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009870 tx=0x7f3b7c02a270 comp rx=0 tx=0).stop 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b9005a640 1 -- 172.21.15.148:0/1644432880 shutdown_connections 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b9005a640 1 --2- 172.21.15.148:0/1644432880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b880ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b9005a640 1 --2- 172.21.15.148:0/1644432880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b88154ed0 0x7f3b880aced0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.393+0000 7f3b9005a640 1 -- 172.21.15.148:0/1644432880 >> 172.21.15.148:0/1644432880 conn(0x7f3b880af670 msgr2=0x7f3b880b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.394+0000 7f3b9005a640 1 -- 172.21.15.148:0/1644432880 shutdown_connections 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.394+0000 7f3b9005a640 1 -- 172.21.15.148:0/1644432880 wait complete. 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 Processor -- start 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 -- start start 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b88153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b88154ed0 0x7f3b880ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 0x7f3b880c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880add20 con 0x7f3b880ca840 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880adba0 con 0x7f3b88154ed0 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.395+0000 7f3b9005a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88161ee0 con 0x7f3b88071ac0 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 0x7f3b880c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 0x7f3b880c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58542/0 (socket says 172.21.15.148:58542) 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 -- 172.21.15.148:0/3836321871 learned_addr learned my addr 172.21.15.148:0/3836321871 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8ddf6640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b88153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8d5f5640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b88154ed0 0x7f3b880ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 -- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 msgr2=0x7f3b88153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b88153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 -- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b88154ed0 msgr2=0x7f3b880ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b88154ed0 0x7f3b880ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 -- 172.21.15.148:0/3836321871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b880c7130 con 0x7f3b880ca840 2024-04-25T10:00:56.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8d5f5640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b88154ed0 0x7f3b880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:56.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8ddf6640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b88153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b8e5f7640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 0x7f3b880c8310 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000c730 tx=0x7f3b8000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80003b70 con 0x7f3b880ca840 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b80003d10 con 0x7f3b880ca840 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b880cd5a0 con 0x7f3b880ca840 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80005310 con 0x7f3b880ca840 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.396+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b880cdb60 con 0x7f3b880ca840 2024-04-25T10:00:56.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.397+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b880b2b10 con 0x7f3b880ca840 2024-04-25T10:00:56.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.398+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b800054b0 con 0x7f3b880ca840 2024-04-25T10:00:56.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.398+0000 7f3b7a7fc640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b6c06b4d0 0x7f3b6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.398+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b8008a100 con 0x7f3b880ca840 2024-04-25T10:00:56.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.398+0000 7f3b8ddf6640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b6c06b4d0 0x7f3b6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.399+0000 7f3b8ddf6640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b6c06b4d0 0x7f3b6c06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c034000 tx=0x7f3b7c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.402+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b80058e80 con 0x7f3b880ca840 2024-04-25T10:00:56.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.607+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b88162320 con 0x7f3b880ca840 2024-04-25T10:00:56.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.608+0000 7f3b7a7fc640 1 -- 172.21.15.148:0/3836321871 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f3b88162320 con 0x7f3b880ca840 2024-04-25T10:00:56.612 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:56.613 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:56.613 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:56.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b6c06b4d0 msgr2=0x7f3b6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b6c06b4d0 0x7f3b6c06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c034000 tx=0x7f3b7c02e000 comp rx=0 tx=0).stop 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 msgr2=0x7f3b880c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 0x7f3b880c8310 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000c730 tx=0x7f3b8000cb30 comp rx=0 tx=0).stop 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 shutdown_connections 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b6c06b4d0 0x7f3b6c06d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b880ca840 0x7f3b880c8310 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b88154ed0 0x7f3b880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 --2- 172.21.15.148:0/3836321871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b88071ac0 0x7f3b88153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 >> 172.21.15.148:0/3836321871 conn(0x7f3b8806c1d0 msgr2=0x7f3b880b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 shutdown_connections 2024-04-25T10:00:56.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.610+0000 7f3b9005a640 1 -- 172.21.15.148:0/3836321871 wait complete. 2024-04-25T10:00:56.626 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:00:56.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.719+0000 7f73e5cb6640 1 Processor -- start 2024-04-25T10:00:56.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.719+0000 7f73e5cb6640 1 -- start start 2024-04-25T10:00:56.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73e5cb6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73e00d0aa0 0x7f73e00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73e5cb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e006fea0 0x7f73e00b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73e5cb6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e00b3710 con 0x7f73e015a790 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73e5cb6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e00cf0e0 con 0x7f73e00d0aa0 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73e5cb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00cf3c0 con 0x7f73e006fea0 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73e00d0aa0 0x7f73e00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73e00d0aa0 0x7f73e00b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58548/0 (socket says 172.21.15.148:58548) 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 -- 172.21.15.148:0/1018997646 learned_addr learned my addr 172.21.15.148:0/1018997646 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 -- 172.21.15.148:0/1018997646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e006fea0 msgr2=0x7f73e00b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 --2- 172.21.15.148:0/1018997646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e006fea0 0x7f73e00b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 -- 172.21.15.148:0/1018997646 >> v1:172.21.15.188:6789/0 conn(0x7f73e015a790 legacy=0x7f73e00d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:56.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.720+0000 7f73d7fff640 1 -- 172.21.15.148:0/1018997646 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00cf6a0 con 0x7f73e00d0aa0 2024-04-25T10:00:56.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73d7fff640 1 --2- 172.21.15.148:0/1018997646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e00d0aa0 0x7f73e00b3ea0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f73c800bb10 tx=0x7f73c802c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87de414101a576b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73deffd640 1 -- 172.21.15.148:0/1018997646 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8032790 con 0x7f73e00d0aa0 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73deffd640 1 -- 172.21.15.148:0/1018997646 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c8038070 con 0x7f73e00d0aa0 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73deffd640 1 -- 172.21.15.148:0/1018997646 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8032b50 con 0x7f73e00d0aa0 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 -- 172.21.15.148:0/1018997646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e00d0aa0 msgr2=0x7f73e00b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/1018997646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e00d0aa0 0x7f73e00b3ea0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f73c800bb10 tx=0x7f73c802c8d0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 -- 172.21.15.148:0/1018997646 shutdown_connections 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/1018997646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e00d0aa0 0x7f73e00b3ea0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/1018997646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e006fea0 0x7f73e00b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 -- 172.21.15.148:0/1018997646 >> 172.21.15.148:0/1018997646 conn(0x7f73e00b2370 msgr2=0x7f73e00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 -- 172.21.15.148:0/1018997646 shutdown_connections 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.721+0000 7f73e5cb6640 1 -- 172.21.15.148:0/1018997646 wait complete. 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 Processor -- start 2024-04-25T10:00:56.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 -- start start 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e006fea0 0x7f73e0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e00d0aa0 0x7f73e015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 0x7f73e00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e015ab70 con 0x7f73e015a790 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00cf980 con 0x7f73e006fea0 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73e5cb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e015acf0 con 0x7f73e00d0aa0 2024-04-25T10:00:56.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73dffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 0x7f73e00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73dffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 0x7f73e00ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58550/0 (socket says 172.21.15.148:58550) 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.723+0000 7f73dffff640 1 -- 172.21.15.148:0/2240546799 learned_addr learned my addr 172.21.15.148:0/2240546799 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73df7fe640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e006fea0 0x7f73e0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73d7fff640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e00d0aa0 0x7f73e015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dffff640 1 -- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e00d0aa0 msgr2=0x7f73e015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dffff640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e00d0aa0 0x7f73e015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dffff640 1 -- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e006fea0 msgr2=0x7f73e0153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dffff640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e006fea0 0x7f73e0153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dffff640 1 -- 172.21.15.148:0/2240546799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00ad620 con 0x7f73e015a790 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dffff640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 0x7f73e00ad0e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b9a0 tx=0x7f73d800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8019070 con 0x7f73e015a790 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d800cbc0 con 0x7f73e015a790 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8014440 con 0x7f73e015a790 2024-04-25T10:00:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e00addc0 con 0x7f73e015a790 2024-04-25T10:00:56.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.724+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e00ae380 con 0x7f73e015a790 2024-04-25T10:00:56.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.725+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e0073990 con 0x7f73e015a790 2024-04-25T10:00:56.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.726+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73d801c070 con 0x7f73e015a790 2024-04-25T10:00:56.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.726+0000 7f73dcff9640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f73c006b4d0 0x7f73c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:56.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.726+0000 7f73df7fe640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f73c006b4d0 0x7f73c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:56.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.726+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73d808a0f0 con 0x7f73e015a790 2024-04-25T10:00:56.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.726+0000 7f73df7fe640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f73c006b4d0 0x7f73c006d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006fd0 tx=0x7f73d0009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:56.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.730+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73e00ad620 con 0x7f73e015a790 2024-04-25T10:00:56.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.933+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73e00b2960 con 0x7f73e015a790 2024-04-25T10:00:56.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.934+0000 7f73dcff9640 1 -- 172.21.15.148:0/2240546799 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f73d8058700 con 0x7f73e015a790 2024-04-25T10:00:56.938 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:56.938 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:00:25.424657+0000","modified":"2024-04-25T10:00:27.438215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7725":{"gid":7725,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/1505603179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1505603179},{"type":"v1","addr":"172.21.15.188:6834","nonce":1505603179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:00:56.939 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 53 2024-04-25T10:00:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f73c006b4d0 msgr2=0x7f73c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f73c006b4d0 0x7f73c006d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006fd0 tx=0x7f73d0009060 comp rx=0 tx=0).stop 2024-04-25T10:00:56.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 msgr2=0x7f73e00ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 0x7f73e00ad0e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f73d800b9a0 tx=0x7f73d800be60 comp rx=0 tx=0).stop 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 shutdown_connections 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f73c006b4d0 0x7f73c006d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e015a790 0x7f73e00ad0e0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e00d0aa0 0x7f73e015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 --2- 172.21.15.148:0/2240546799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e006fea0 0x7f73e0153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 >> 172.21.15.148:0/2240546799 conn(0x7f73e00b2370 msgr2=0x7f73e0158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 shutdown_connections 2024-04-25T10:00:56.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:56.936+0000 7f73e5cb6640 1 -- 172.21.15.148:0/2240546799 wait complete. 2024-04-25T10:00:56.952 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:00:57.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.045+0000 7f40c9fce640 1 Processor -- start 2024-04-25T10:00:57.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.045+0000 7f40c9fce640 1 -- start start 2024-04-25T10:00:57.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c9fce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f40c40ce330 0x7f40c406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c9fce640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f40c40700a0 0x7f40c415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c9fce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c41626e0 con 0x7f40c406abe0 2024-04-25T10:00:57.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c9fce640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40cd270 con 0x7f40c40700a0 2024-04-25T10:00:57.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c9fce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40cd550 con 0x7f40c40ce330 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f40c40700a0 0x7f40c415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f40c406abe0 0x7f40c406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44490/0 (socket says 172.21.15.148:44490) 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c37fe640 1 -- 172.21.15.148:0/1886244027 learned_addr learned my addr 172.21.15.148:0/1886244027 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c2ffd640 1 --2- 172.21.15.148:0/1886244027 >> v2:172.21.15.188:3300/0 conn(0x7f40c40ce330 0x7f40c406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c3fff640 1 -- 172.21.15.148:0/1886244027 >> v2:172.21.15.188:3300/0 conn(0x7f40c40ce330 msgr2=0x7f40c406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c3fff640 1 --2- 172.21.15.148:0/1886244027 >> v2:172.21.15.188:3300/0 conn(0x7f40c40ce330 0x7f40c406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c3fff640 1 -- 172.21.15.148:0/1886244027 >> v1:172.21.15.148:6789/0 conn(0x7f40c406abe0 legacy=0x7f40c406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c27fc640 1 -- 172.21.15.148:0/1886244027 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921776279 0 0) 0x7f40c41626e0 con 0x7f40c406abe0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c3fff640 1 -- 172.21.15.148:0/1886244027 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40cd830 con 0x7f40c40700a0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.046+0000 7f40c2ffd640 1 --2- 172.21.15.148:0/1886244027 >> v2:172.21.15.188:3300/0 conn(0x7f40c40ce330 0x7f40c406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.047+0000 7f40c3fff640 1 --2- 172.21.15.148:0/1886244027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c40700a0 0x7f40c415d370 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f40c4068860 tx=0x7f40bc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d453d44845e7509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.047+0000 7f40c27fc640 1 -- 172.21.15.148:0/1886244027 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc007740 con 0x7f40c40700a0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.047+0000 7f40c27fc640 1 -- 172.21.15.148:0/1886244027 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc0078e0 con 0x7f40c40700a0 2024-04-25T10:00:57.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.047+0000 7f40c27fc640 1 -- 172.21.15.148:0/1886244027 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc007cb0 con 0x7f40c40700a0 2024-04-25T10:00:57.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 -- 172.21.15.148:0/1886244027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c40700a0 msgr2=0x7f40c415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1886244027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c40700a0 0x7f40c415d370 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f40c4068860 tx=0x7f40bc02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 -- 172.21.15.148:0/1886244027 shutdown_connections 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1886244027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c40700a0 0x7f40c415d370 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1886244027 >> v2:172.21.15.188:3300/0 conn(0x7f40c40ce330 0x7f40c406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 -- 172.21.15.148:0/1886244027 >> 172.21.15.148:0/1886244027 conn(0x7f40c40b2220 msgr2=0x7f40c40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 -- 172.21.15.148:0/1886244027 shutdown_connections 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.048+0000 7f40c9fce640 1 -- 172.21.15.148:0/1886244027 wait complete. 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.049+0000 7f40c9fce640 1 Processor -- start 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.049+0000 7f40c9fce640 1 -- start start 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c9fce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 0x7f40c40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c9fce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40c40700a0 0x7f40c41f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c9fce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40c40ce330 0x7f40c415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c9fce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c415d8b0 con 0x7f40c406abe0 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c9fce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c4161780 con 0x7f40c40ce330 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c9fce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c415da30 con 0x7f40c40700a0 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 0x7f40c40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 0x7f40c40b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58576/0 (socket says 172.21.15.148:58576) 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 -- 172.21.15.148:0/1760058343 learned_addr learned my addr 172.21.15.148:0/1760058343 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c3fff640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40c40ce330 0x7f40c415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c2ffd640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40c40700a0 0x7f40c41f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 -- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40c40700a0 msgr2=0x7f40c41f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40c40700a0 0x7f40c41f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 -- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40c40ce330 msgr2=0x7f40c415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40c40ce330 0x7f40c415f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 -- 172.21.15.148:0/1760058343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c41f4ae0 con 0x7f40c406abe0 2024-04-25T10:00:57.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c3fff640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40c40ce330 0x7f40c415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40c37fe640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 0x7f40c40b32f0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c500 tx=0x7f40b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.050+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4018070 con 0x7f40c406abe0 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.051+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4007580 con 0x7f40c406abe0 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.051+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4004a20 con 0x7f40c406abe0 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.051+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c41f4dc0 con 0x7f40c406abe0 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.051+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c41f5380 con 0x7f40c406abe0 2024-04-25T10:00:57.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.051+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c40b63d0 con 0x7f40c406abe0 2024-04-25T10:00:57.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.052+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40b401b070 con 0x7f40c406abe0 2024-04-25T10:00:57.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.052+0000 7f40abfff640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40b006b4d0 0x7f40b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.052+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40b4089630 con 0x7f40c406abe0 2024-04-25T10:00:57.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.052+0000 7f40c2ffd640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40b006b4d0 0x7f40b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.053+0000 7f40c2ffd640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40b006b4d0 0x7f40b006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f40c40b3830 tx=0x7f40b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.056+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40b40583b0 con 0x7f40c406abe0 2024-04-25T10:00:57.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.233+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40c40c73a0 con 0x7f40c406abe0 2024-04-25T10:00:57.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.234+0000 7f40abfff640 1 -- 172.21.15.148:0/1760058343 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f40b4057cb0 con 0x7f40c406abe0 2024-04-25T10:00:57.239 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:00:57.239 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":97,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:00:27.088834+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:00:23.631066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:00:24.401378+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:00:57.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.236+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40b006b4d0 msgr2=0x7f40b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.236+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40b006b4d0 0x7f40b006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f40c40b3830 tx=0x7f40b8008040 comp rx=0 tx=0).stop 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.236+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 msgr2=0x7f40c40b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.236+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 0x7f40c40b32f0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f40b400c500 tx=0x7f40b400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.236+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 shutdown_connections 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f40b006b4d0 0x7f40b006d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40c40ce330 0x7f40c415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40c40700a0 0x7f40c41f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 --2- 172.21.15.148:0/1760058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40c406abe0 0x7f40c40b32f0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 >> 172.21.15.148:0/1760058343 conn(0x7f40c40b2220 msgr2=0x7f40c40b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 shutdown_connections 2024-04-25T10:00:57.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.237+0000 7f40c9fce640 1 -- 172.21.15.148:0/1760058343 wait complete. 2024-04-25T10:00:57.253 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:00:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.346+0000 7f839fe53640 1 Processor -- start 2024-04-25T10:00:57.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.346+0000 7f839fe53640 1 -- start start 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839fe53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839fe53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83980b5620 0x7f83980b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839fe53640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839806cf50 con 0x7f839806c2d0 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839fe53640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83980b5fc0 con 0x7f83980b5620 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839fe53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980ce530 con 0x7f839815b7d0 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839d3ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83980b5620 0x7f83980b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839e3f0640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f839806c2d0 0x7f839806c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44506/0 (socket says 172.21.15.148:44506) 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.347+0000 7f839e3f0640 1 -- 172.21.15.148:0/3654189997 learned_addr learned my addr 172.21.15.148:0/3654189997 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839dbef640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839d3ee640 1 -- 172.21.15.148:0/3654189997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 msgr2=0x7f83980cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839d3ee640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839d3ee640 1 -- 172.21.15.148:0/3654189997 >> v1:172.21.15.148:6789/0 conn(0x7f839806c2d0 legacy=0x7f839806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839cbed640 1 -- 172.21.15.148:0/3654189997 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552456695 0 0) 0x7f839806cf50 con 0x7f839806c2d0 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839d3ee640 1 -- 172.21.15.148:0/3654189997 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980ce810 con 0x7f83980b5620 2024-04-25T10:00:57.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839dbef640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839d3ee640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980b5a80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f83880097a0 tx=0x7f838802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae31ef37e7ff74e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.348+0000 7f839cbed640 1 -- 172.21.15.148:0/3654189997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388037070 con 0x7f83980b5620 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839cbed640 1 -- 172.21.15.148:0/3654189997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388032440 con 0x7f83980b5620 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839cbed640 1 -- 172.21.15.148:0/3654189997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388032810 con 0x7f83980b5620 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 -- 172.21.15.148:0/3654189997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 msgr2=0x7f83980b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980b5a80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f83880097a0 tx=0x7f838802a180 comp rx=0 tx=0).stop 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 -- 172.21.15.148:0/3654189997 shutdown_connections 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980b5a80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 --2- 172.21.15.148:0/3654189997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 -- 172.21.15.148:0/3654189997 >> 172.21.15.148:0/3654189997 conn(0x7f83980d0800 msgr2=0x7f83980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 -- 172.21.15.148:0/3654189997 shutdown_connections 2024-04-25T10:00:57.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.349+0000 7f839fe53640 1 -- 172.21.15.148:0/3654189997 wait complete. 2024-04-25T10:00:57.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.350+0000 7f839fe53640 1 Processor -- start 2024-04-25T10:00:57.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.350+0000 7f839fe53640 1 -- start start 2024-04-25T10:00:57.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839fe53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839806c2d0 0x7f8398159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839fe53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839fe53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839fe53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83980cf550 con 0x7f83980b5620 2024-04-25T10:00:57.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839fe53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839806d230 con 0x7f839806c2d0 2024-04-25T10:00:57.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839fe53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980cf6d0 con 0x7f839815b7d0 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58594/0 (socket says 172.21.15.148:58594) 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 -- 172.21.15.148:0/3328178716 learned_addr learned my addr 172.21.15.148:0/3328178716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839dbef640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839806c2d0 0x7f8398159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839e3f0640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 -- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 msgr2=0x7f83980b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 -- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839806c2d0 msgr2=0x7f8398159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839806c2d0 0x7f8398159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.351+0000 7f839d3ee640 1 -- 172.21.15.148:0/3328178716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980cae10 con 0x7f83980b5620 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.352+0000 7f839d3ee640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980cc600 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f83880098c0 tx=0x7f83880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.352+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388037040 con 0x7f83980b5620 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.352+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388030770 con 0x7f83980b5620 2024-04-25T10:00:57.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.352+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388030b40 con 0x7f83980b5620 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.352+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83980cb0b0 con 0x7f83980b5620 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.352+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83980cb670 con 0x7f83980b5620 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.353+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8398073a00 con 0x7f83980b5620 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.353+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8388044020 con 0x7f83980b5620 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.353+0000 7f838e7fc640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.354+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83880aa8d0 con 0x7f83980b5620 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.354+0000 7f839dbef640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.354+0000 7f839dbef640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f837c06b4d0 0x7f837c06d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8394007510 tx=0x7f83940074a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.358+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8388032e20 con 0x7f83980b5620 2024-04-25T10:00:57.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.562+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83980b53e0 con 0x7f83980b5620 2024-04-25T10:00:57.567 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:00:57.563+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:00:57.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.564+0000 7f838e7fc640 1 -- 172.21.15.148:0/3328178716 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8388079650 con 0x7f83980b5620 2024-04-25T10:00:57.568 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:00:57.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f837c06b4d0 msgr2=0x7f837c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f837c06b4d0 0x7f837c06d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8394007510 tx=0x7f83940074a0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 msgr2=0x7f83980cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980cc600 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f83880098c0 tx=0x7f83880046c0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 shutdown_connections 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f839815b7d0 0x7f83980b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83980b5620 0x7f83980cc600 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 --2- 172.21.15.148:0/3328178716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839806c2d0 0x7f8398159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 >> 172.21.15.148:0/3328178716 conn(0x7f83980d0800 msgr2=0x7f8398160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 shutdown_connections 2024-04-25T10:00:57.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.567+0000 7f839fe53640 1 -- 172.21.15.148:0/3328178716 wait complete. 2024-04-25T10:00:57.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:00:57.611 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:00:57.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.678+0000 7f7b535a5640 1 Processor -- start 2024-04-25T10:00:57.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.678+0000 7f7b535a5640 1 -- start start 2024-04-25T10:00:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b535a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c15b980 0x7f7b4c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b535a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c072280 con 0x7f7b4c0cc4f0 2024-04-25T10:00:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b535a5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c0714c0 con 0x7f7b4c0ce230 2024-04-25T10:00:57.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b535a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c15bdc0 con 0x7f7b4c15b980 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b51341640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7b4c0cc4f0 0x7f7b4c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44520/0 (socket says 172.21.15.148:44520) 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b51341640 1 -- 172.21.15.148:0/3084776612 learned_addr learned my addr 172.21.15.148:0/3084776612 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b51b42640 1 --2- 172.21.15.148:0/3084776612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c15b980 0x7f7b4c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284305205 0 0) 0x7f7b4c072280 con 0x7f7b4c0cc4f0 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b38003520 con 0x7f7b4c0cc4f0 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065919760 0 0) 0x7f7b4c0714c0 con 0x7f7b4c0ce230 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c072280 con 0x7f7b4c0ce230 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b51b42640 1 -- 172.21.15.148:0/3084776612 >> v1:172.21.15.188:6789/0 conn(0x7f7b4c0ce230 legacy=0x7f7b4c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b51b42640 1 -- 172.21.15.148:0/3084776612 >> v1:172.21.15.148:6789/0 conn(0x7f7b4c0cc4f0 legacy=0x7f7b4c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.679+0000 7f7b51b42640 1 -- 172.21.15.148:0/3084776612 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0717a0 con 0x7f7b4c15b980 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.680+0000 7f7b51b42640 1 --2- 172.21.15.148:0/3084776612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c15b980 0x7f7b4c0cc8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009dc0 tx=0x7f7b4402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4c23874232667d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.680+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44007740 con 0x7f7b4c15b980 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.680+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b440078e0 con 0x7f7b4c15b980 2024-04-25T10:00:57.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.680+0000 7f7b43fff640 1 -- 172.21.15.148:0/3084776612 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44007c90 con 0x7f7b4c15b980 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 -- 172.21.15.148:0/3084776612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c15b980 msgr2=0x7f7b4c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3084776612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c15b980 0x7f7b4c0cc8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009dc0 tx=0x7f7b4402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 -- 172.21.15.148:0/3084776612 shutdown_connections 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3084776612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c15b980 0x7f7b4c0cc8d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 -- 172.21.15.148:0/3084776612 >> 172.21.15.148:0/3084776612 conn(0x7f7b4c06abe0 msgr2=0x7f7b4c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 -- 172.21.15.148:0/3084776612 shutdown_connections 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.681+0000 7f7b535a5640 1 -- 172.21.15.148:0/3084776612 wait complete. 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.682+0000 7f7b535a5640 1 Processor -- start 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.682+0000 7f7b535a5640 1 -- start start 2024-04-25T10:00:57.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c0cc4f0 0x7f7b4c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b4c0ce230 0x7f7b4c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 0x7f7b4c0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c071d60 con 0x7f7b4c15b980 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c071270 con 0x7f7b4c0ce230 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c06e8a0 con 0x7f7b4c0cc4f0 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 0x7f7b4c0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 0x7f7b4c0cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58610/0 (socket says 172.21.15.148:58610) 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 -- 172.21.15.148:0/3742417902 learned_addr learned my addr 172.21.15.148:0/3742417902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b50b40640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b4c0ce230 0x7f7b4c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51341640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c0cc4f0 0x7f7b4c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 -- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c0cc4f0 msgr2=0x7f7b4c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c0cc4f0 0x7f7b4c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 -- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b4c0ce230 msgr2=0x7f7b4c15c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b4c0ce230 0x7f7b4c15c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:57.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 -- 172.21.15.148:0/3742417902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c1586b0 con 0x7f7b4c15b980 2024-04-25T10:00:57.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b51b42640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 0x7f7b4c0cb2a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7b4402acc0 tx=0x7f7b44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44040070 con 0x7f7b4c15b980 2024-04-25T10:00:57.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b44031c40 con 0x7f7b4c15b980 2024-04-25T10:00:57.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44009450 con 0x7f7b4c15b980 2024-04-25T10:00:57.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.683+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c158990 con 0x7f7b4c15b980 2024-04-25T10:00:57.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.684+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c158f50 con 0x7f7b4c15b980 2024-04-25T10:00:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.685+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c073a00 con 0x7f7b4c15b980 2024-04-25T10:00:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.685+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b44039040 con 0x7f7b4c15b980 2024-04-25T10:00:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.685+0000 7f7b41ffb640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b3006b4d0 0x7f7b3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.685+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7b440ab990 con 0x7f7b4c15b980 2024-04-25T10:00:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.686+0000 7f7b51341640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b3006b4d0 0x7f7b3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:57.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.686+0000 7f7b51341640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b3006b4d0 0x7f7b3006d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000f60 tx=0x7f7b48007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:57.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.690+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b4407a190 con 0x7f7b4c15b980 2024-04-25T10:00:57.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:57.897+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7b4c15a9a0 con 0x7f7b4c15b980 2024-04-25T10:00:58.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.566+0000 7f7b41ffb640 1 -- 172.21.15.148:0/3742417902 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7b4407e230 con 0x7f7b4c15b980 2024-04-25T10:00:58.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b3006b4d0 msgr2=0x7f7b3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b3006b4d0 0x7f7b3006d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000f60 tx=0x7f7b48007d50 comp rx=0 tx=0).stop 2024-04-25T10:00:58.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 msgr2=0x7f7b4c0cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 0x7f7b4c0cb2a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7b4402acc0 tx=0x7f7b44004a40 comp rx=0 tx=0).stop 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 shutdown_connections 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7b3006b4d0 0x7f7b3006d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b4c15b980 0x7f7b4c0cb2a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b4c0ce230 0x7f7b4c15c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 --2- 172.21.15.148:0/3742417902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b4c0cc4f0 0x7f7b4c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.568+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 >> 172.21.15.148:0/3742417902 conn(0x7f7b4c06abe0 msgr2=0x7f7b4c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.569+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 shutdown_connections 2024-04-25T10:00:58.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.569+0000 7f7b535a5640 1 -- 172.21.15.148:0/3742417902 wait complete. 2024-04-25T10:00:58.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:00:58.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.688+0000 7fbabbf7e640 1 Processor -- start 2024-04-25T10:00:58.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.688+0000 7fbabbf7e640 1 -- start start 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbabbf7e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbab415dc20 0x7fbab40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbabbf7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbab4161710 0x7fbab40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbabbf7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab415e000 con 0x7fbab40b9330 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbabbf7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40d1de0 con 0x7fbab4161710 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbabbf7e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab4161c60 con 0x7fbab415dc20 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbab4161710 0x7fbab40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbab4161710 0x7fbab40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58622/0 (socket says 172.21.15.148:58622) 2024-04-25T10:00:58.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 -- 172.21.15.148:0/2139607255 learned_addr learned my addr 172.21.15.148:0/2139607255 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9d1a640 1 --2- 172.21.15.148:0/2139607255 >> v2:172.21.15.188:3300/0 conn(0x7fbab415dc20 0x7fbab40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 -- 172.21.15.148:0/2139607255 >> v2:172.21.15.188:3300/0 conn(0x7fbab415dc20 msgr2=0x7fbab40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 --2- 172.21.15.148:0/2139607255 >> v2:172.21.15.188:3300/0 conn(0x7fbab415dc20 0x7fbab40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 -- 172.21.15.148:0/2139607255 >> v1:172.21.15.188:6789/0 conn(0x7fbab40b9330 legacy=0x7fbab406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9519640 1 -- 172.21.15.148:0/2139607255 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4161f40 con 0x7fbab4161710 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.689+0000 7fbab9d1a640 1 --2- 172.21.15.148:0/2139607255 >> v2:172.21.15.188:3300/0 conn(0x7fbab415dc20 0x7fbab40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:00:58.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.690+0000 7fbab9519640 1 --2- 172.21.15.148:0/2139607255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40cebb0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fbab40d1520 tx=0x7fbaa402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60467f53e7b234d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.690+0000 7fbab8d18640 1 -- 172.21.15.148:0/2139607255 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4037070 con 0x7fbab4161710 2024-04-25T10:00:58.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.690+0000 7fbab8d18640 1 -- 172.21.15.148:0/2139607255 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa4032440 con 0x7fbab4161710 2024-04-25T10:00:58.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.690+0000 7fbab8d18640 1 -- 172.21.15.148:0/2139607255 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4032810 con 0x7fbab4161710 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 -- 172.21.15.148:0/2139607255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 msgr2=0x7fbab40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/2139607255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40cebb0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fbab40d1520 tx=0x7fbaa402a230 comp rx=0 tx=0).stop 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 -- 172.21.15.148:0/2139607255 shutdown_connections 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/2139607255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40cebb0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/2139607255 >> v2:172.21.15.188:3300/0 conn(0x7fbab415dc20 0x7fbab40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 -- 172.21.15.148:0/2139607255 >> 172.21.15.148:0/2139607255 conn(0x7fbab40d0420 msgr2=0x7fbab40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 -- 172.21.15.148:0/2139607255 shutdown_connections 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.691+0000 7fbabbf7e640 1 -- 172.21.15.148:0/2139607255 wait complete. 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.692+0000 7fbabbf7e640 1 Processor -- start 2024-04-25T10:00:58.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.692+0000 7fbabbf7e640 1 -- start start 2024-04-25T10:00:58.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbabbf7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab40b9330 0x7fbab40aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbabbf7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbab415dc20 0x7fbab41e9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbabbf7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbabbf7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40b5e50 con 0x7fbab4161710 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbabbf7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab4162500 con 0x7fbab40b9330 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbabbf7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40b5fd0 con 0x7fbab415dc20 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58634/0 (socket says 172.21.15.148:58634) 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 -- 172.21.15.148:0/4286094677 learned_addr learned my addr 172.21.15.148:0/4286094677 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbab9d1a640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab40b9330 0x7fbab40aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbab9519640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbab415dc20 0x7fbab41e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 -- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbab415dc20 msgr2=0x7fbab41e9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbab415dc20 0x7fbab41e9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 -- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab40b9330 msgr2=0x7fbab40aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab40b9330 0x7fbab40aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 -- 172.21.15.148:0/4286094677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab41e8570 con 0x7fbab4161710 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.693+0000 7fbaba51b640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40b0ff0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00ba10 tx=0x7fbaac00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.694+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac019070 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.694+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac00cac0 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.694+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab41e8850 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.694+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac014440 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.694+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab41e8e10 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.695+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab40cff60 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.695+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaac01c070 con 0x7fbab4161710 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.695+0000 7fbaaa7fc640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba9806b4d0 0x7fba9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.696+0000 7fbab9d1a640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba9806b4d0 0x7fba9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.696+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbaac089ff0 con 0x7fbab4161710 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.696+0000 7fbab9d1a640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba9806b4d0 0x7fba9806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbab000a950 tx=0x7fbab0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.700+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbab40cff60 con 0x7fbab4161710 2024-04-25T10:00:58.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:58.872+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbab40b2f30 con 0x7fbab4161710 2024-04-25T10:00:59.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.571+0000 7fbaaa7fc640 1 -- 172.21.15.148:0/4286094677 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbaac058c90 con 0x7fbab4161710 2024-04-25T10:00:59.576 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba9806b4d0 msgr2=0x7fba9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba9806b4d0 0x7fba9806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbab000a950 tx=0x7fbab0008040 comp rx=0 tx=0).stop 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 msgr2=0x7fbab40b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40b0ff0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00ba10 tx=0x7fbaac00bed0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 shutdown_connections 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba9806b4d0 0x7fba9806d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbab4161710 0x7fbab40b0ff0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbab415dc20 0x7fbab41e9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 --2- 172.21.15.148:0/4286094677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab40b9330 0x7fbab40aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 >> 172.21.15.148:0/4286094677 conn(0x7fbab40d0420 msgr2=0x7fbab40b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 shutdown_connections 2024-04-25T10:00:59.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.574+0000 7fbabbf7e640 1 -- 172.21.15.148:0/4286094677 wait complete. 2024-04-25T10:00:59.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:00:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 Processor -- start 2024-04-25T10:00:59.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 -- start start 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff4161890 0x7feff40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feff40b54a0 0x7feff40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff40cf1d0 con 0x7feff4161240 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff40b7120 con 0x7feff40b54a0 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.692+0000 7feff8e4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff4161620 con 0x7feff4161890 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feff40b54a0 0x7feff40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feff40b54a0 0x7feff40b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58644/0 (socket says 172.21.15.148:58644) 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 -- 172.21.15.148:0/3655213575 learned_addr learned my addr 172.21.15.148:0/3655213575 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7fefebfff640 1 --2- 172.21.15.148:0/3655213575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff4161890 0x7feff40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 -- 172.21.15.148:0/3655213575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff4161890 msgr2=0x7feff40b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 --2- 172.21.15.148:0/3655213575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff4161890 0x7feff40b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 -- 172.21.15.148:0/3655213575 >> v1:172.21.15.188:6789/0 conn(0x7feff4161240 legacy=0x7feff40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.693+0000 7feff2d9d640 1 -- 172.21.15.148:0/3655213575 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4161e00 con 0x7feff40b54a0 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff2d9d640 1 --2- 172.21.15.148:0/3655213575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40b54a0 0x7feff40b5930 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fefec009e50 tx=0x7fefec02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81bcdaad24b7301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff1d9b640 1 -- 172.21.15.148:0/3655213575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec036070 con 0x7feff40b54a0 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff1d9b640 1 -- 172.21.15.148:0/3655213575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec031600 con 0x7feff40b54a0 2024-04-25T10:00:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff1d9b640 1 -- 172.21.15.148:0/3655213575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0319d0 con 0x7feff40b54a0 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 -- 172.21.15.148:0/3655213575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40b54a0 msgr2=0x7feff40b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 --2- 172.21.15.148:0/3655213575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40b54a0 0x7feff40b5930 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fefec009e50 tx=0x7fefec02a710 comp rx=0 tx=0).stop 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 -- 172.21.15.148:0/3655213575 shutdown_connections 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 --2- 172.21.15.148:0/3655213575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff40b54a0 0x7feff40b5930 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 --2- 172.21.15.148:0/3655213575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff4161890 0x7feff40b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 -- 172.21.15.148:0/3655213575 >> 172.21.15.148:0/3655213575 conn(0x7feff40d04a0 msgr2=0x7feff40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:00:59.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 -- 172.21.15.148:0/3655213575 shutdown_connections 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.694+0000 7feff8e4a640 1 -- 172.21.15.148:0/3655213575 wait complete. 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 Processor -- start 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 -- start start 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff40b54a0 0x7feff415fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 0x7feff40cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff4161890 0x7feff40cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff41626a0 con 0x7feff4161240 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff41623c0 con 0x7feff4161890 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7feff8e4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40d15f0 con 0x7feff40b54a0 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7fefebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 0x7feff40cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7fefebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 0x7feff40cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58654/0 (socket says 172.21.15.148:58654) 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.696+0000 7fefebfff640 1 -- 172.21.15.148:0/1382483819 learned_addr learned my addr 172.21.15.148:0/1382483819 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7feff259c640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff40b54a0 0x7feff415fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7feff2d9d640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff4161890 0x7feff40cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefebfff640 1 -- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff40b54a0 msgr2=0x7feff415fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefebfff640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff40b54a0 0x7feff415fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefebfff640 1 -- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff4161890 msgr2=0x7feff40cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefebfff640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff4161890 0x7feff40cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefebfff640 1 -- 172.21.15.148:0/1382483819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40c80f0 con 0x7feff4161240 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefebfff640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 0x7feff40cbd30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00d5f0 tx=0x7fefdc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0125f0 con 0x7feff4161240 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc019070 con 0x7feff4161240 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff40cc8a0 con 0x7feff4161240 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0129d0 con 0x7feff4161240 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.697+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff40cce60 con 0x7feff4161240 2024-04-25T10:00:59.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.698+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff4073a00 con 0x7feff4161240 2024-04-25T10:00:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.699+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefdc012b70 con 0x7feff4161240 2024-04-25T10:00:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.699+0000 7fefeaffd640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fefd406b4d0 0x7fefd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:00:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.699+0000 7feff259c640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fefd406b4d0 0x7fefd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:00:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.699+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fefdc08a690 con 0x7feff4161240 2024-04-25T10:00:59.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.699+0000 7feff259c640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fefd406b4d0 0x7fefd406d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefe4001640 tx=0x7fefe4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:00:59.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.703+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefdc059450 con 0x7feff4161240 2024-04-25T10:00:59.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:00:59.875+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feff406d240 con 0x7feff4161240 2024-04-25T10:01:00.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.579+0000 7fefeaffd640 1 -- 172.21.15.148:0/1382483819 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7fefdc058d50 con 0x7feff4161240 2024-04-25T10:01:00.583 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:01:00.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.582+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fefd406b4d0 msgr2=0x7fefd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.582+0000 7feff8e4a640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fefd406b4d0 0x7fefd406d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefe4001640 tx=0x7fefe4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:00.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 msgr2=0x7feff40cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 0x7feff40cbd30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00d5f0 tx=0x7fefdc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 shutdown_connections 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fefd406b4d0 0x7fefd406d990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feff4161890 0x7feff40cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feff4161240 0x7feff40cbd30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 --2- 172.21.15.148:0/1382483819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feff40b54a0 0x7feff415fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.583+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 >> 172.21.15.148:0/1382483819 conn(0x7feff40d04a0 msgr2=0x7feff40b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.584+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 shutdown_connections 2024-04-25T10:01:00.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.584+0000 7feff8e4a640 1 -- 172.21.15.148:0/1382483819 wait complete. 2024-04-25T10:01:00.605 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:01:00.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.707+0000 7f513cabd640 1 Processor -- start 2024-04-25T10:01:00.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.707+0000 7f513cabd640 1 -- start start 2024-04-25T10:01:00.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513cabd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5138158230 0x7f51380d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513cabd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51380d2020 con 0x7f51380d1b80 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513cabd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51380d2300 con 0x7f51380b4e60 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513cabd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5138158610 con 0x7f5138158230 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5138158230 0x7f51380d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f512ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f51380d1b80 0x7f513806c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44522/0 (socket says 172.21.15.148:44522) 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f512ffff640 1 -- 172.21.15.148:0/2397531045 learned_addr learned my addr 172.21.15.148:0/2397531045 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513659c640 1 -- 172.21.15.148:0/2397531045 >> v1:172.21.15.188:6789/0 conn(0x7f51380b4e60 legacy=0x7f513806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513659c640 1 -- 172.21.15.148:0/2397531045 >> v1:172.21.15.148:6789/0 conn(0x7f51380d1b80 legacy=0x7f513806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.708+0000 7f513659c640 1 -- 172.21.15.148:0/2397531045 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513806bb30 con 0x7f5138158230 2024-04-25T10:01:00.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.709+0000 7f513659c640 1 --2- 172.21.15.148:0/2397531045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5138158230 0x7f51380d1640 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f51280097a0 tx=0x7f512802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d04758c0b88c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.709+0000 7f5135d9b640 1 -- 172.21.15.148:0/2397531045 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128037070 con 0x7f5138158230 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.709+0000 7f5135d9b640 1 -- 172.21.15.148:0/2397531045 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128032440 con 0x7f5138158230 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.709+0000 7f5135d9b640 1 -- 172.21.15.148:0/2397531045 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128032810 con 0x7f5138158230 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 -- 172.21.15.148:0/2397531045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5138158230 msgr2=0x7f51380d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 --2- 172.21.15.148:0/2397531045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5138158230 0x7f51380d1640 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f51280097a0 tx=0x7f512802a230 comp rx=0 tx=0).stop 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 -- 172.21.15.148:0/2397531045 shutdown_connections 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 --2- 172.21.15.148:0/2397531045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5138158230 0x7f51380d1640 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 -- 172.21.15.148:0/2397531045 >> 172.21.15.148:0/2397531045 conn(0x7f5138061600 msgr2=0x7f51380619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 -- 172.21.15.148:0/2397531045 shutdown_connections 2024-04-25T10:01:00.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.710+0000 7f513cabd640 1 -- 172.21.15.148:0/2397531045 wait complete. 2024-04-25T10:01:00.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.711+0000 7f513cabd640 1 Processor -- start 2024-04-25T10:01:00.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.711+0000 7f513cabd640 1 -- start start 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51380b4e60 0x7f5138159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 0x7f51381e5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5138158230 0x7f51381587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513806ca40 con 0x7f51380d1b80 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513806bea0 con 0x7f5138158230 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513815e320 con 0x7f51380b4e60 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 0x7f51381e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 0x7f51381e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58672/0 (socket says 172.21.15.148:58672) 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 -- 172.21.15.148:0/849844923 learned_addr learned my addr 172.21.15.148:0/849844923 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 -- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51380b4e60 msgr2=0x7f5138159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51380b4e60 0x7f5138159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 -- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5138158230 msgr2=0x7f51381587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f5136d9d640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5138158230 0x7f51381587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5138158230 0x7f51381587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 -- 172.21.15.148:0/849844923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138158e90 con 0x7f51380d1b80 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f5136d9d640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5138158230 0x7f51381587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512ffff640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 0x7f51381e5140 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f512000c3c0 tx=0x7f512000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120017070 con 0x7f51380d1b80 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51200074e0 con 0x7f51380d1b80 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51200059e0 con 0x7f51380d1b80 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.712+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5138159170 con 0x7f51380d1b80 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.713+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5138159730 con 0x7f51380d1b80 2024-04-25T10:01:00.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.713+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513806de50 con 0x7f51380d1b80 2024-04-25T10:01:00.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.714+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f512001a070 con 0x7f51380d1b80 2024-04-25T10:01:00.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.714+0000 7f512effd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f511806b500 0x7f511806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:00.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.714+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5120089420 con 0x7f51380d1b80 2024-04-25T10:01:00.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.714+0000 7f513659c640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f511806b500 0x7f511806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:00.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.715+0000 7f513659c640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f511806b500 0x7f511806d9c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5128034000 tx=0x7f512802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:00.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.719+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5120058440 con 0x7f51380d1b80 2024-04-25T10:01:00.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.887+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f51380b2950 con 0x7f51380d1b80 2024-04-25T10:01:00.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.890+0000 7f512effd640 1 -- 172.21.15.148:0/849844923 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5120057d40 con 0x7f51380d1b80 2024-04-25T10:01:00.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f511806b500 msgr2=0x7f511806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f511806b500 0x7f511806d9c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5128034000 tx=0x7f512802c000 comp rx=0 tx=0).stop 2024-04-25T10:01:00.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 msgr2=0x7f51381e5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:00.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 0x7f51381e5140 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f512000c3c0 tx=0x7f512000c880 comp rx=0 tx=0).stop 2024-04-25T10:01:00.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 shutdown_connections 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f511806b500 0x7f511806d9c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5138158230 0x7f51381587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51380d1b80 0x7f51381e5140 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 --2- 172.21.15.148:0/849844923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51380b4e60 0x7f5138159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 >> 172.21.15.148:0/849844923 conn(0x7f5138061600 msgr2=0x7f51380b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 shutdown_connections 2024-04-25T10:01:00.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:00.893+0000 7f513cabd640 1 -- 172.21.15.148:0/849844923 wait complete. 2024-04-25T10:01:00.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:01:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.016+0000 7f59a0210640 1 Processor -- start 2024-04-25T10:01:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.016+0000 7f59a0210640 1 -- start start 2024-04-25T10:01:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.016+0000 7f59a0210640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59980d0ad0 0x7f5998168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f59a0210640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980b9940 con 0x7f599815dd30 2024-04-25T10:01:01.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f59a0210640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980d1df0 con 0x7f59980d2e90 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f59a0210640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59980b7a50 con 0x7f59980d0ad0 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599e7ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59980d0ad0 0x7f5998168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599e7ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59980d0ad0 0x7f5998168760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58680/0 (socket says 172.21.15.148:58680) 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599e7ad640 1 -- 172.21.15.148:0/3734550820 learned_addr learned my addr 172.21.15.148:0/3734550820 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599dfac640 1 --1- 172.21.15.148:0/3734550820 >> v1:172.21.15.148:6789/0 conn(0x7f599815dd30 0x7f59980b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44534/0 (socket says 172.21.15.148:44534) 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599e7ad640 1 -- 172.21.15.148:0/3734550820 >> v1:172.21.15.188:6789/0 conn(0x7f59980d2e90 legacy=0x7f59980d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599e7ad640 1 -- 172.21.15.148:0/3734550820 >> v1:172.21.15.148:6789/0 conn(0x7f599815dd30 legacy=0x7f59980b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599e7ad640 1 -- 172.21.15.148:0/3734550820 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980d20d0 con 0x7f59980d0ad0 2024-04-25T10:01:01.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.017+0000 7f599cfaa640 1 -- 172.21.15.148:0/3734550820 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534507140 0 0) 0x7f59980b9940 con 0x7f599815dd30 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f599e7ad640 1 --2- 172.21.15.148:0/3734550820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f5998168760 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5990009e50 tx=0x7f599002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af97f66b8514891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f599cfaa640 1 -- 172.21.15.148:0/3734550820 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990036070 con 0x7f59980d0ad0 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f599cfaa640 1 -- 172.21.15.148:0/3734550820 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5990031600 con 0x7f59980d0ad0 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f599cfaa640 1 -- 172.21.15.148:0/3734550820 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59900319d0 con 0x7f59980d0ad0 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 -- 172.21.15.148:0/3734550820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 msgr2=0x7f5998168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 --2- 172.21.15.148:0/3734550820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f5998168760 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5990009e50 tx=0x7f599002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 -- 172.21.15.148:0/3734550820 shutdown_connections 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 --2- 172.21.15.148:0/3734550820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f5998168760 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 -- 172.21.15.148:0/3734550820 >> 172.21.15.148:0/3734550820 conn(0x7f5998061c30 msgr2=0x7f599806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 -- 172.21.15.148:0/3734550820 shutdown_connections 2024-04-25T10:01:01.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.018+0000 7f59a0210640 1 -- 172.21.15.148:0/3734550820 wait complete. 2024-04-25T10:01:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 Processor -- start 2024-04-25T10:01:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 -- start start 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f59981f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59980d2e90 0x7f59980b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f599815dd30 0x7f59980b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980d2970 con 0x7f59980d0ad0 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980d2690 con 0x7f59980d2e90 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f59a0210640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980d1a10 con 0x7f599815dd30 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f599dfac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f59981f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f599dfac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f59981f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58682/0 (socket says 172.21.15.148:58682) 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.020+0000 7f599dfac640 1 -- 172.21.15.148:0/2766317570 learned_addr learned my addr 172.21.15.148:0/2766317570 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f599dfac640 1 -- 172.21.15.148:0/2766317570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f599815dd30 msgr2=0x7f59980b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f599dfac640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f599815dd30 0x7f59980b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f599dfac640 1 -- 172.21.15.148:0/2766317570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59980d2e90 msgr2=0x7f59980b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f599dfac640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59980d2e90 0x7f59980b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f599dfac640 1 -- 172.21.15.148:0/2766317570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59981f49f0 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f599dfac640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f59981f38e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f599400c4a0 tx=0x7f599400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994018070 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59940075a0 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59981f4cd0 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994004a20 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.021+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59981f5290 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.022+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5998073990 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.022+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f599401b070 con 0x7f59980d0ad0 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.023+0000 7f598e7fc640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f597c06b4d0 0x7f597c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.023+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f59940893d0 con 0x7f59980d0ad0 2024-04-25T10:01:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.023+0000 7f599d7ab640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f597c06b4d0 0x7f597c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.023+0000 7f599d7ab640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f597c06b4d0 0x7f597c06d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5988007fd0 tx=0x7f59880079a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.027+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59940583c0 con 0x7f59980d0ad0 2024-04-25T10:01:01.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.269+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f59980c8c40 con 0x7f59980d0ad0 2024-04-25T10:01:01.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.580+0000 7f598e7fc640 1 -- 172.21.15.148:0/2766317570 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v327) v1 ==== 144+0+0 (secure 0 0 0) 0x7f599405e0c0 con 0x7f59980d0ad0 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f597c06b4d0 msgr2=0x7f597c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f597c06b4d0 0x7f597c06d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5988007fd0 tx=0x7f59880079a0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 msgr2=0x7f59981f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f59981f38e0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f599400c4a0 tx=0x7f599400c960 comp rx=0 tx=0).stop 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 shutdown_connections 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f597c06b4d0 0x7f597c06d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f599815dd30 0x7f59980b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59980d2e90 0x7f59980b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 --2- 172.21.15.148:0/2766317570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59980d0ad0 0x7f59981f38e0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 >> 172.21.15.148:0/2766317570 conn(0x7f5998061c30 msgr2=0x7f599815cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 shutdown_connections 2024-04-25T10:01:01.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.582+0000 7f59a0210640 1 -- 172.21.15.148:0/2766317570 wait complete. 2024-04-25T10:01:01.598 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:01:01.598 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:01:01.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:01.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.691+0000 7f92b4476640 1 Processor -- start 2024-04-25T10:01:01.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.691+0000 7f92b4476640 1 -- start start 2024-04-25T10:01:01.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.692+0000 7f92b4476640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92ac0d2e90 0x7f92ac0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.692+0000 7f92b4476640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92ac0d0900 0x7f92ac0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.692+0000 7f92b4476640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac0b9940 con 0x7f92ac15dd30 2024-04-25T10:01:01.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.692+0000 7f92b4476640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac0d0d40 con 0x7f92ac0d0900 2024-04-25T10:01:01.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.692+0000 7f92b4476640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac0d1df0 con 0x7f92ac0d2e90 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92ac0d0900 0x7f92ac0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92ac0d0900 0x7f92ac0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58698/0 (socket says 172.21.15.148:58698) 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 -- 172.21.15.148:0/452927811 learned_addr learned my addr 172.21.15.148:0/452927811 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b1a11640 1 --2- 172.21.15.148:0/452927811 >> v2:172.21.15.188:3300/0 conn(0x7f92ac0d2e90 0x7f92ac0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 -- 172.21.15.148:0/452927811 >> v2:172.21.15.188:3300/0 conn(0x7f92ac0d2e90 msgr2=0x7f92ac0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 --2- 172.21.15.148:0/452927811 >> v2:172.21.15.188:3300/0 conn(0x7f92ac0d2e90 0x7f92ac0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 -- 172.21.15.148:0/452927811 >> v1:172.21.15.188:6789/0 conn(0x7f92ac15dd30 legacy=0x7f92ac0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.693+0000 7f92b2a13640 1 -- 172.21.15.148:0/452927811 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0d20d0 con 0x7f92ac0d0900 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b2a13640 1 --2- 172.21.15.148:0/452927811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac0d0900 0x7f92ac0b6a40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0b6540 tx=0x7f92a402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cd73c519a2952b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b1210640 1 -- 172.21.15.148:0/452927811 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4036070 con 0x7f92ac0d0900 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b1210640 1 -- 172.21.15.148:0/452927811 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4031600 con 0x7f92ac0d0900 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b1210640 1 -- 172.21.15.148:0/452927811 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40319d0 con 0x7f92ac0d0900 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 -- 172.21.15.148:0/452927811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac0d0900 msgr2=0x7f92ac0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 --2- 172.21.15.148:0/452927811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac0d0900 0x7f92ac0b6a40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0b6540 tx=0x7f92a402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 -- 172.21.15.148:0/452927811 shutdown_connections 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 --2- 172.21.15.148:0/452927811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac0d0900 0x7f92ac0b6a40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 --2- 172.21.15.148:0/452927811 >> v2:172.21.15.188:3300/0 conn(0x7f92ac0d2e90 0x7f92ac0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 -- 172.21.15.148:0/452927811 >> 172.21.15.148:0/452927811 conn(0x7f92ac061c30 msgr2=0x7f92ac06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 -- 172.21.15.148:0/452927811 shutdown_connections 2024-04-25T10:01:01.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.694+0000 7f92b4476640 1 -- 172.21.15.148:0/452927811 wait complete. 2024-04-25T10:01:01.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 Processor -- start 2024-04-25T10:01:01.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 -- start start 2024-04-25T10:01:01.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac0d0900 0x7f92ac1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac0d2e90 0x7f92ac1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 0x7f92ac14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0d2970 con 0x7f92ac15dd30 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0d2690 con 0x7f92ac0d2e90 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b4476640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0d1a10 con 0x7f92ac0d0900 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b2a13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 0x7f92ac14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b2a13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 0x7f92ac14e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58700/0 (socket says 172.21.15.148:58700) 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.696+0000 7f92b2a13640 1 -- 172.21.15.148:0/2840649117 learned_addr learned my addr 172.21.15.148:0/2840649117 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b2a13640 1 -- 172.21.15.148:0/2840649117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac0d0900 msgr2=0x7f92ac1f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b2a13640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac0d0900 0x7f92ac1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b2a13640 1 -- 172.21.15.148:0/2840649117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac0d2e90 msgr2=0x7f92ac1f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b2a13640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac0d2e90 0x7f92ac1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b2a13640 1 -- 172.21.15.148:0/2840649117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac14f040 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b2a13640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 0x7f92ac14e580 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009f40 tx=0x7f92a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4036050 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a4042070 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac14f320 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4007bd0 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.697+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac1f6e30 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.698+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac073990 con 0x7f92ac15dd30 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.698+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92a4047460 con 0x7f92ac15dd30 2024-04-25T10:01:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.699+0000 7f92a2ffd640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f929006b4d0 0x7f929006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.699+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f92a40aafb0 con 0x7f92ac15dd30 2024-04-25T10:01:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.699+0000 7f92b2212640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f929006b4d0 0x7f929006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.699+0000 7f92b2212640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f929006b4d0 0x7f929006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f929c006fd0 tx=0x7f929c006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.703+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92a407a030 con 0x7f92ac15dd30 2024-04-25T10:01:01.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.873+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92ac06b210 con 0x7f92ac15dd30 2024-04-25T10:01:01.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.874+0000 7f92a2ffd640 1 -- 172.21.15.148:0/2840649117 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13345 (secure 0 0 0) 0x7f92a4079930 con 0x7f92ac15dd30 2024-04-25T10:01:01.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:01.879 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":100,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:00.572637+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6834/1505603179":"2024-04-26T10:00:57.563666+0000","172.21.15.188:6832/1505603179":"2024-04-26T10:00:57.563666+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f929006b4d0 msgr2=0x7f929006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f929006b4d0 0x7f929006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f929c006fd0 tx=0x7f929c006bc0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 msgr2=0x7f92ac14e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 0x7f92ac14e580 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f92a4009f40 tx=0x7f92a4004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 shutdown_connections 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f929006b4d0 0x7f929006d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ac15dd30 0x7f92ac14e580 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac0d2e90 0x7f92ac1f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 --2- 172.21.15.148:0/2840649117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac0d0900 0x7f92ac1f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 >> 172.21.15.148:0/2840649117 conn(0x7f92ac061c30 msgr2=0x7f92ac0c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:01.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 shutdown_connections 2024-04-25T10:01:01.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.876+0000 7f92b4476640 1 -- 172.21.15.148:0/2840649117 wait complete. 2024-04-25T10:01:01.892 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:01.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.988+0000 7f56073e0640 1 Processor -- start 2024-04-25T10:01:01.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.988+0000 7f56073e0640 1 -- start start 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.988+0000 7f56073e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f56000cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.988+0000 7f56073e0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560006cf50 con 0x7f560006c9e0 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.988+0000 7f56073e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56000ce530 con 0x7f56000b5620 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.988+0000 7f56073e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56000ce810 con 0x7f560015b7d0 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f560597d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f560006c9e0 0x7f56000d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44550/0 (socket says 172.21.15.148:44550) 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f560597d640 1 -- 172.21.15.148:0/62252765 learned_addr learned my addr 172.21.15.148:0/62252765 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f560517c640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f56000cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901535876 0 0) 0x7f560006cf50 con 0x7f560006c9e0 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55ec003520 con 0x7f560006c9e0 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600736584 0 0) 0x7f56000ce530 con 0x7f56000b5620 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f560006cf50 con 0x7f56000b5620 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2800334769 0 0) 0x7f55ec003520 con 0x7f560006c9e0 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 msgr2=0x7f56000cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f56000cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 >> v1:172.21.15.188:6789/0 conn(0x7f56000b5620 legacy=0x7f560006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000ceaf0 con 0x7f560006c9e0 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f55f8001670 con 0x7f560006c9e0 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 >> v1:172.21.15.148:6789/0 conn(0x7f560006c9e0 legacy=0x7f56000d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f55ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec004d40 0x7f55ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56000ce810 con 0x7f55ec003b70 2024-04-25T10:01:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec00ba70 con 0x7f55ec004d40 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec00bc20 con 0x7f560015b7d0 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f560517c640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.989+0000 7f560497b640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f55ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560597d640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec004d40 0x7f55ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560517c640 1 -- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 msgr2=0x7f55ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560517c640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f55ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560517c640 1 -- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec004d40 msgr2=0x7f55ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560517c640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec004d40 0x7f55ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560517c640 1 -- 172.21.15.148:0/62252765 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec003520 con 0x7f55ec003b70 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560497b640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f560015b7d0 0x7f55ec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f560517c640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009fc0 tx=0x7f55ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7707c02160317c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc009c10 con 0x7f55ec003b70 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc036070 con 0x7f55ec003b70 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.990+0000 7f55f7fff640 1 -- 172.21.15.148:0/62252765 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc031880 con 0x7f55ec003b70 2024-04-25T10:01:01.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f56073e0640 1 -- 172.21.15.148:0/62252765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ec003b70 msgr2=0x7f55ec003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f56073e0640 1 --2- 172.21.15.148:0/62252765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ec003b70 0x7f55ec003f70 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009fc0 tx=0x7f55ec005830 comp rx=0 tx=0).stop 2024-04-25T10:01:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f560597d640 1 -- 172.21.15.148:0/62252765 reap_dead start 2024-04-25T10:01:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f56073e0640 1 -- 172.21.15.148:0/62252765 shutdown_connections 2024-04-25T10:01:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f56073e0640 1 -- 172.21.15.148:0/62252765 >> 172.21.15.148:0/62252765 conn(0x7f56000d0800 msgr2=0x7f56000d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f56073e0640 1 -- 172.21.15.148:0/62252765 shutdown_connections 2024-04-25T10:01:02.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.991+0000 7f56073e0640 1 -- 172.21.15.148:0/62252765 wait complete. 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.992+0000 7f56073e0640 1 Processor -- start 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.992+0000 7f56073e0640 1 -- start start 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f56073e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec003b70 0x7f56000caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f56073e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56001f80e0 0x7f560014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f56073e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 0x7f56001f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f56073e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560015bd30 con 0x7f56001f7540 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f56073e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560015bbb0 con 0x7f55ec003b70 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f56073e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56000d2650 con 0x7f56001f80e0 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 0x7f56001f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 0x7f56001f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58716/0 (socket says 172.21.15.148:58716) 2024-04-25T10:01:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 -- 172.21.15.148:0/1669303630 learned_addr learned my addr 172.21.15.148:0/1669303630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560517c640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec003b70 0x7f56000caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560497b640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56001f80e0 0x7f560014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 -- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56001f80e0 msgr2=0x7f560014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56001f80e0 0x7f560014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 -- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec003b70 msgr2=0x7f56000caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec003b70 0x7f56000caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 -- 172.21.15.148:0/1669303630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000aa5b0 con 0x7f56001f7540 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560517c640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec003b70 0x7f56000caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560497b640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56001f80e0 0x7f560014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.993+0000 7f560597d640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 0x7f56001f79c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f55f8000c00 tx=0x7f55f800e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.994+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8010510 con 0x7f56001f7540 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.994+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f80106b0 con 0x7f56001f7540 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.994+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56000aa890 con 0x7f56001f7540 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.994+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8010aa0 con 0x7f56001f7540 2024-04-25T10:01:02.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.994+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56000aab70 con 0x7f56001f7540 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.995+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5600073a00 con 0x7f56001f7540 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.995+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55f8010c40 con 0x7f56001f7540 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.995+0000 7f55f5ffb640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55e406b4d0 0x7f55e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.995+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f55f8089690 con 0x7f56001f7540 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.995+0000 7f560517c640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55e406b4d0 0x7f55e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.996+0000 7f560517c640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55e406b4d0 0x7f55e406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f55fc035000 tx=0x7f55fc02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:01.999+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55f8058680 con 0x7f56001f7540 2024-04-25T10:01:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.172+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f560015a630 con 0x7f56001f7540 2024-04-25T10:01:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.172+0000 7f55f5ffb640 1 -- 172.21.15.148:0/1669303630 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55f8057f80 con 0x7f56001f7540 2024-04-25T10:01:02.176 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55e406b4d0 msgr2=0x7f55e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55e406b4d0 0x7f55e406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f55fc035000 tx=0x7f55fc02f000 comp rx=0 tx=0).stop 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 msgr2=0x7f56001f79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 0x7f56001f79c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f55f8000c00 tx=0x7f55f800e890 comp rx=0 tx=0).stop 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 shutdown_connections 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55e406b4d0 0x7f55e406d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56001f7540 0x7f56001f79c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56001f80e0 0x7f560014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 --2- 172.21.15.148:0/1669303630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55ec003b70 0x7f56000caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 >> 172.21.15.148:0/1669303630 conn(0x7f56000d0800 msgr2=0x7f56000cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 shutdown_connections 2024-04-25T10:01:02.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.174+0000 7f56073e0640 1 -- 172.21.15.148:0/1669303630 wait complete. 2024-04-25T10:01:02.190 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:01:02.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.284+0000 7f256705d640 1 Processor -- start 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.284+0000 7f256705d640 1 -- start start 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.284+0000 7f256705d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25600b5f00 0x7f25600ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.284+0000 7f256705d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25600cec00 con 0x7f25600d0b10 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.284+0000 7f256705d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25600ceee0 con 0x7f256015db60 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.284+0000 7f256705d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600ce930 con 0x7f25600b5f00 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f25600d0b10 0x7f25600d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44556/0 (socket says 172.21.15.148:44556) 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255ffff640 1 -- 172.21.15.148:0/3034728566 learned_addr learned my addr 172.21.15.148:0/3034728566 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881503928 0 0) 0x7f25600cec00 con 0x7f25600d0b10 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2550003520 con 0x7f25600d0b10 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f25655fa640 1 --2- 172.21.15.148:0/3034728566 >> v2:172.21.15.188:3300/0 conn(0x7f25600b5f00 0x7f25600ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 993834063 0 0) 0x7f2550003520 con 0x7f25600d0b10 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 >> v2:172.21.15.188:3300/0 conn(0x7f25600b5f00 msgr2=0x7f25600ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 --2- 172.21.15.148:0/3034728566 >> v2:172.21.15.188:3300/0 conn(0x7f25600b5f00 0x7f25600ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 >> v1:172.21.15.188:6789/0 conn(0x7f256015db60 legacy=0x7f25600d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600cf1c0 con 0x7f25600d0b10 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f254c001670 con 0x7f25600d0b10 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 >> v1:172.21.15.148:6789/0 conn(0x7f25600d0b10 legacy=0x7f25600d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2550004800 0x7f2550004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500051c0 0x7f255000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600ce930 con 0x7f25500039b0 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255000c160 con 0x7f25500051c0 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255000c360 con 0x7f2550004800 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f2564df9640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f25655fa640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500051c0 0x7f255000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.285+0000 7f255ffff640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2550004800 0x7f2550004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f2564df9640 1 -- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2550004800 msgr2=0x7f2550004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f2564df9640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2550004800 0x7f2550004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f2564df9640 1 -- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500051c0 msgr2=0x7f255000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f2564df9640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500051c0 0x7f255000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f2564df9640 1 -- 172.21.15.148:0/3034728566 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550003520 con 0x7f25500039b0 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f25655fa640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500051c0 0x7f255000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f25655fa640 1 -- 172.21.15.148:0/3034728566 reap_dead start 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f2564df9640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2554001750 tx=0x7f2550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f0bdeea8c89ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554001370 con 0x7f25500039b0 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25540052d0 con 0x7f25500039b0 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.286+0000 7f255f7fe640 1 -- 172.21.15.148:0/3034728566 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255402abc0 con 0x7f25500039b0 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 -- 172.21.15.148:0/3034728566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25500039b0 msgr2=0x7f2550003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2554001750 tx=0x7f2550005830 comp rx=0 tx=0).stop 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 -- 172.21.15.148:0/3034728566 shutdown_connections 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 --2- 172.21.15.148:0/3034728566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25500039b0 0x7f2550003dd0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 -- 172.21.15.148:0/3034728566 >> 172.21.15.148:0/3034728566 conn(0x7f2560061600 msgr2=0x7f2560061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 -- 172.21.15.148:0/3034728566 shutdown_connections 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.287+0000 7f256705d640 1 -- 172.21.15.148:0/3034728566 wait complete. 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.288+0000 7f256705d640 1 Processor -- start 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.288+0000 7f256705d640 1 -- start start 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f256705d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500039b0 0x7f2560158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f256705d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f256015e450 0x7f256015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f256705d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 0x7f2560160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f256705d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256015df40 con 0x7f256015ee10 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f256705d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b6450 con 0x7f25500039b0 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f256705d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256015e0c0 con 0x7f256015e450 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f25655fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 0x7f2560160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f25655fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 0x7f2560160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58736/0 (socket says 172.21.15.148:58736) 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f25655fa640 1 -- 172.21.15.148:0/710490535 learned_addr learned my addr 172.21.15.148:0/710490535 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f255ffff640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f256015e450 0x7f256015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f2564df9640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500039b0 0x7f2560158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f25655fa640 1 -- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f256015e450 msgr2=0x7f256015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.289+0000 7f25655fa640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f256015e450 0x7f256015e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f25655fa640 1 -- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500039b0 msgr2=0x7f2560158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f25655fa640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500039b0 0x7f2560158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f25655fa640 1 -- 172.21.15.148:0/710490535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256015f290 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f2564df9640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500039b0 0x7f2560158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f25655fa640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 0x7f2560160060 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f256008c1a0 tx=0x7f2558010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558010da0 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2558003990 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600c8330 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558003d80 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.290+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600c88f0 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.291+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25600c8f40 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.291+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f255801e020 con 0x7f256015ee10 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.292+0000 7f255d7fa640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f254406b500 0x7f254406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.292+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f255808a6c0 con 0x7f256015ee10 2024-04-25T10:01:02.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.292+0000 7f2564df9640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f254406b500 0x7f254406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.292+0000 7f2564df9640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f254406b500 0x7f254406d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2554032000 tx=0x7f255402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.296+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25580596b0 con 0x7f256015ee10 2024-04-25T10:01:02.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.532+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f2560073270 con 0x7f256015ee10 2024-04-25T10:01:02.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.581+0000 7f255d7fa640 1 -- 172.21.15.148:0/710490535 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v328) v1 ==== 158+0+0 (secure 0 0 0) 0x7f2558058fb0 con 0x7f256015ee10 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f254406b500 msgr2=0x7f254406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f254406b500 0x7f254406d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2554032000 tx=0x7f255402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 msgr2=0x7f2560160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 0x7f2560160060 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f256008c1a0 tx=0x7f2558010590 comp rx=0 tx=0).stop 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 shutdown_connections 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f254406b500 0x7f254406d9c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f256015ee10 0x7f2560160060 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f256015e450 0x7f256015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 --2- 172.21.15.148:0/710490535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25500039b0 0x7f2560158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.583+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 >> 172.21.15.148:0/710490535 conn(0x7f2560061600 msgr2=0x7f25600b88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.584+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 shutdown_connections 2024-04-25T10:01:02.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.584+0000 7f256705d640 1 -- 172.21.15.148:0/710490535 wait complete. 2024-04-25T10:01:02.599 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:01:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.696+0000 7f05b879d640 1 Processor -- start 2024-04-25T10:01:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.696+0000 7f05b879d640 1 -- start start 2024-04-25T10:01:02.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b879d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f05b015b980 0x7f05b00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b879d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b0072280 con 0x7f05b00cc4f0 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b879d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b00714c0 con 0x7f05b00ce230 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b879d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b0071270 con 0x7f05b015b980 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b6539640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f05b00cc4f0 0x7f05b00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44562/0 (socket says 172.21.15.148:44562) 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b6539640 1 -- 172.21.15.148:0/289647896 learned_addr learned my addr 172.21.15.148:0/289647896 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b6d3a640 1 --2- 172.21.15.148:0/289647896 >> v2:172.21.15.188:3300/0 conn(0x7f05b015b980 0x7f05b00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021108958 0 0) 0x7f05b0072280 con 0x7f05b00cc4f0 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f059c003520 con 0x7f05b00cc4f0 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2652076713 0 0) 0x7f059c003520 con 0x7f05b00cc4f0 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 >> v2:172.21.15.188:3300/0 conn(0x7f05b015b980 msgr2=0x7f05b00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 --2- 172.21.15.148:0/289647896 >> v2:172.21.15.188:3300/0 conn(0x7f05b015b980 0x7f05b00cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 >> v1:172.21.15.188:6789/0 conn(0x7f05b00ce230 legacy=0x7f05b015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00717a0 con 0x7f05b00cc4f0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f05ac001670 con 0x7f05b00cc4f0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 >> v1:172.21.15.148:6789/0 conn(0x7f05b00cc4f0 legacy=0x7f05b00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f059c0039b0 0x7f059c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f059c004800 0x7f059c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0071270 con 0x7f059c0039b0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c00c060 con 0x7f059c0051c0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c00c260 con 0x7f059c004800 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.697+0000 7f05b6539640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f059c0039b0 0x7f059c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b5d38640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f059c004800 0x7f059c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6d3a640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6539640 1 -- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f059c004800 msgr2=0x7f059c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6539640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f059c004800 0x7f059c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6539640 1 -- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0051c0 msgr2=0x7f059c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6539640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0051c0 0x7f059c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6d3a640 1 -- 172.21.15.148:0/289647896 reap_dead start 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6539640 1 -- 172.21.15.148:0/289647896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c003520 con 0x7f059c0039b0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.698+0000 7f05b6539640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f059c0039b0 0x7f059c003dd0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c240 tx=0x7f059c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812ac0eb371654f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac007d10 con 0x7f059c0039b0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0058d0 con 0x7f059c0039b0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b5537640 1 -- 172.21.15.148:0/289647896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac005d20 con 0x7f059c0039b0 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 -- 172.21.15.148:0/289647896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f059c0039b0 msgr2=0x7f059c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f059c0039b0 0x7f059c003dd0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c240 tx=0x7f059c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 -- 172.21.15.148:0/289647896 shutdown_connections 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 --2- 172.21.15.148:0/289647896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f059c0039b0 0x7f059c003dd0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 -- 172.21.15.148:0/289647896 >> 172.21.15.148:0/289647896 conn(0x7f05b006abe0 msgr2=0x7f05b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 -- 172.21.15.148:0/289647896 shutdown_connections 2024-04-25T10:01:02.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.699+0000 7f05b879d640 1 -- 172.21.15.148:0/289647896 wait complete. 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.700+0000 7f05b879d640 1 Processor -- start 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.700+0000 7f05b879d640 1 -- start start 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b879d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0039b0 0x7f05b0143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b879d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 0x7f05b0158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b879d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05b00ca490 0x7f05b00ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b879d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00cc8d0 con 0x7f05b0158ad0 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b879d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0071d60 con 0x7f059c0039b0 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b879d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00cca50 con 0x7f05b00ca490 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 0x7f05b0158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 0x7f05b0158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58756/0 (socket says 172.21.15.148:58756) 2024-04-25T10:01:02.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 -- 172.21.15.148:0/3954376527 learned_addr learned my addr 172.21.15.148:0/3954376527 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b6539640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0039b0 0x7f05b0143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b6d3a640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05b00ca490 0x7f05b00ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 -- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05b00ca490 msgr2=0x7f05b00ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05b00ca490 0x7f05b00ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 -- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0039b0 msgr2=0x7f05b0143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0039b0 0x7f05b0143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b5d38640 1 -- 172.21.15.148:0/3954376527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b0159470 con 0x7f05b0158ad0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.701+0000 7f05b6539640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0039b0 0x7f05b0143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.702+0000 7f05b5d38640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 0x7f05b0158f30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e410 tx=0x7f05a000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.702+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0019070 con 0x7f05b0158ad0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.702+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00052e0 con 0x7f05b0158ad0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.702+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b00cb250 con 0x7f05b0158ad0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.702+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0003830 con 0x7f05b0158ad0 2024-04-25T10:01:02.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.702+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b00cb810 con 0x7f05b0158ad0 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.703+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b00cf200 con 0x7f05b0158ad0 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.703+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05a001c070 con 0x7f05b0158ad0 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.703+0000 7f05a6ffd640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f059406b4d0 0x7f059406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.704+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f05a0089520 con 0x7f05b0158ad0 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.704+0000 7f05b6539640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f059406b4d0 0x7f059406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.704+0000 7f05b6539640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f059406b4d0 0x7f059406d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05ac033000 tx=0x7f05ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:02.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.708+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05b00cf200 con 0x7f05b0158ad0 2024-04-25T10:01:02.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.878+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05b0143700 con 0x7f05b0158ad0 2024-04-25T10:01:02.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.882+0000 7f05a6ffd640 1 -- 172.21.15.148:0/3954376527 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05b0159470 con 0x7f05b0158ad0 2024-04-25T10:01:02.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f059406b4d0 msgr2=0x7f059406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f059406b4d0 0x7f059406d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05ac033000 tx=0x7f05ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 msgr2=0x7f05b0158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 0x7f05b0158f30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f05a000e410 tx=0x7f05a000e8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 shutdown_connections 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f059406b4d0 0x7f059406d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05b00ca490 0x7f05b00ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05b0158ad0 0x7f05b0158f30 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 --2- 172.21.15.148:0/3954376527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f059c0039b0 0x7f05b0143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 >> 172.21.15.148:0/3954376527 conn(0x7f05b006abe0 msgr2=0x7f05b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 shutdown_connections 2024-04-25T10:01:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:02.884+0000 7f05b879d640 1 -- 172.21.15.148:0/3954376527 wait complete. 2024-04-25T10:01:02.900 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:01:02.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:02.948 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:02.996 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:01:02.996 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:02.996 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:03.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:03.012 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:01:03.012 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:03.029 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:03.029 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:03.029 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:03.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:03.067 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:01:03.067 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:03.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:03.086 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:01:03.086 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:03.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.200+0000 7fccaa7ce640 1 Processor -- start 2024-04-25T10:01:03.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.200+0000 7fccaa7ce640 1 -- start start 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fccaa7ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca40cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fccaa7ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 0x7fcca406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fccaa7ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca4071480 con 0x7fcca40cfcf0 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fccaa7ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca4072090 con 0x7fcca40ce590 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fccaa7ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40d0150 con 0x7fcca4154ee0 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fcca8d6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 0x7fcca406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca40cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fcca37fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fcca40cfcf0 0x7fcca40cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34318/0 (socket says 172.21.15.148:34318) 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca40cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:39390/0 (socket says 172.21.15.148:39390) 2024-04-25T10:01:03.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fcca8d6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 0x7fcca406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:54618/0 (socket says 172.21.15.148:54618) 2024-04-25T10:01:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.201+0000 7fcca37fe640 1 -- 172.21.15.148:0/3210972047 learned_addr learned my addr 172.21.15.148:0/3210972047 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.202+0000 7fcca3fff640 1 -- 172.21.15.148:0/3210972047 >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 msgr2=0x7fcca406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.202+0000 7fcca3fff640 1 --2- 172.21.15.148:0/3210972047 >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 0x7fcca406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.202+0000 7fcca3fff640 1 -- 172.21.15.148:0/3210972047 >> v1:172.21.15.188:6789/0 conn(0x7fcca40cfcf0 legacy=0x7fcca40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.202+0000 7fcca3fff640 1 -- 172.21.15.148:0/3210972047 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca4071760 con 0x7fcca4154ee0 2024-04-25T10:01:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.202+0000 7fcca8d6b640 1 --2- 172.21.15.148:0/3210972047 >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 0x7fcca406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.202+0000 7fcca3fff640 1 --2- 172.21.15.148:0/3210972047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca40cf7b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097a0 tx=0x7fcc9402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb49702b17a430e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fcca2ffd640 1 -- 172.21.15.148:0/3210972047 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94037070 con 0x7fcca4154ee0 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fcca2ffd640 1 -- 172.21.15.148:0/3210972047 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc94032440 con 0x7fcca4154ee0 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fcca2ffd640 1 -- 172.21.15.148:0/3210972047 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94032810 con 0x7fcca4154ee0 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 -- 172.21.15.148:0/3210972047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 msgr2=0x7fcca40cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/3210972047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca40cf7b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097a0 tx=0x7fcc9402a180 comp rx=0 tx=0).stop 2024-04-25T10:01:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 -- 172.21.15.148:0/3210972047 shutdown_connections 2024-04-25T10:01:03.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/3210972047 >> v2:172.21.15.188:3300/0 conn(0x7fcca40ce590 0x7fcca406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/3210972047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca40cf7b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 -- 172.21.15.148:0/3210972047 >> 172.21.15.148:0/3210972047 conn(0x7fcca406abe0 msgr2=0x7fcca406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 -- 172.21.15.148:0/3210972047 shutdown_connections 2024-04-25T10:01:03.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.203+0000 7fccaa7ce640 1 -- 172.21.15.148:0/3210972047 wait complete. 2024-04-25T10:01:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.204+0000 7fccaa7ce640 1 Processor -- start 2024-04-25T10:01:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 -- start start 2024-04-25T10:01:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcca40ce590 0x7fcca4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 0x7fcca41f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4071d20 con 0x7fcca40cfcf0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4071230 con 0x7fcca40ce590 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fccaa7ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca41552c0 con 0x7fcca4154ee0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fcca37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 0x7fcca41f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fcca37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 0x7fcca41f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58762/0 (socket says 172.21.15.148:58762) 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fcca37fe640 1 -- 172.21.15.148:0/140480891 learned_addr learned my addr 172.21.15.148:0/140480891 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fcca3fff640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcca40ce590 0x7fcca4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.205+0000 7fcca8d6b640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca37fe640 1 -- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 msgr2=0x7fcca4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca37fe640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca4159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca37fe640 1 -- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcca40ce590 msgr2=0x7fcca4154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca37fe640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcca40ce590 0x7fcca4154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca37fe640 1 -- 172.21.15.148:0/140480891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca4158cb0 con 0x7fcca40cfcf0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca37fe640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 0x7fcca41f4d90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c400 tx=0x7fcc9800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98017070 con 0x7fcca40cfcf0 2024-04-25T10:01:03.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc98005ce0 con 0x7fcca40cfcf0 2024-04-25T10:01:03.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca4158f90 con 0x7fcca40cfcf0 2024-04-25T10:01:03.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc980072f0 con 0x7fcca40cfcf0 2024-04-25T10:01:03.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.206+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca4159550 con 0x7fcca40cfcf0 2024-04-25T10:01:03.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.207+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca4073990 con 0x7fcca40cfcf0 2024-04-25T10:01:03.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.207+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc9801a070 con 0x7fcca40cfcf0 2024-04-25T10:01:03.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.208+0000 7fcca0ff9640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.208+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcc980892f0 con 0x7fcca40cfcf0 2024-04-25T10:01:03.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.208+0000 7fcca3fff640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.208+0000 7fcca3fff640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc9006b4d0 0x7fcc9006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcc94034000 tx=0x7fcc9402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.212+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc98057d00 con 0x7fcca40cfcf0 2024-04-25T10:01:03.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.419+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca406d360 con 0x7fcca40cfcf0 2024-04-25T10:01:03.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.420+0000 7fcca0ff9640 1 -- 172.21.15.148:0/140480891 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fcc9805bda0 con 0x7fcca40cfcf0 2024-04-25T10:01:03.424 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:03.425 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6577,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/320485721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":320485721},{"type":"v1","addr":"172.21.15.188:6835","nonce":320485721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1422333811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1422333811},{"type":"v1","addr":"172.21.15.188:6834","nonce":1422333811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7710,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3996460828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3996460828},{"type":"v1","addr":"172.21.15.148:6836","nonce":3996460828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1935451908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1935451908},{"type":"v1","addr":"172.21.15.148:6837","nonce":1935451908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[]} 2024-04-25T10:01:03.425 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 55 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc9006b4d0 msgr2=0x7fcc9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc9006b4d0 0x7fcc9006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcc94034000 tx=0x7fcc9402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 msgr2=0x7fcca41f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 0x7fcca41f4d90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c400 tx=0x7fcc9800c800 comp rx=0 tx=0).stop 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 shutdown_connections 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcca4154ee0 0x7fcca4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca40cfcf0 0x7fcca41f4d90 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 --2- 172.21.15.148:0/140480891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcca40ce590 0x7fcca4154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 >> 172.21.15.148:0/140480891 conn(0x7fcca406abe0 msgr2=0x7fcca40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 shutdown_connections 2024-04-25T10:01:03.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.422+0000 7fccaa7ce640 1 -- 172.21.15.148:0/140480891 wait complete. 2024-04-25T10:01:03.439 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:03.439 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:01:03.440 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:01:03.440 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:03.440 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:03.441 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:01:03.441 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:01:03.441 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:03.442 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:03.442 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:01:03.442 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:01:03.443 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:03.443 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:03.443 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:01:03.444 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:01:03.444 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:03.450 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:01:03.451 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:03.452 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:01:03.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:03.454 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:01:03.454 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:03.456 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:01:03.456 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:03.493 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:03.493 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:03.494 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:03.494 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:03.497 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:03.497 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:03.498 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:03.499 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:03.499 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:01:03.566 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:01:03.566 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:01:03.576 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:01:03.577 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:01:03.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.598+0000 7fc8a3d16640 1 Processor -- start 2024-04-25T10:01:03.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.599+0000 7fc8a3d16640 1 -- start start 2024-04-25T10:01:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.599+0000 7fc8a3d16640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc89c15e0e0 0x7fc89c09a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.599+0000 7fc8a3d16640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0d06c0 con 0x7fc89c15dd00 2024-04-25T10:01:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.599+0000 7fc8a3d16640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c09a690 con 0x7fc89c06bf00 2024-04-25T10:01:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.599+0000 7fc8a3d16640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0d1440 con 0x7fc89c15e0e0 2024-04-25T10:01:03.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a1ab2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc89c15dd00 0x7fc89c099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33960/0 (socket says 172.21.15.148:33960) 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a1ab2640 1 -- 172.21.15.148:0/3769513798 learned_addr learned my addr 172.21.15.148:0/3769513798 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a12b1640 1 --2- 172.21.15.148:0/3769513798 >> v2:172.21.15.188:3300/0 conn(0x7fc89c15e0e0 0x7fc89c09a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229282284 0 0) 0x7fc89c0d06c0 con 0x7fc89c15dd00 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc888003520 con 0x7fc89c15dd00 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207351235 0 0) 0x7fc89c09a690 con 0x7fc89c06bf00 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc89c0d06c0 con 0x7fc89c06bf00 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a12b1640 1 -- 172.21.15.148:0/3769513798 >> v1:172.21.15.188:6789/0 conn(0x7fc89c06bf00 legacy=0x7fc89c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a12b1640 1 -- 172.21.15.148:0/3769513798 >> v1:172.21.15.148:6789/0 conn(0x7fc89c15dd00 legacy=0x7fc89c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.600+0000 7fc8a12b1640 1 -- 172.21.15.148:0/3769513798 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0d0960 con 0x7fc89c15e0e0 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a12b1640 1 --2- 172.21.15.148:0/3769513798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c15e0e0 0x7fc89c09a150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097a0 tx=0x7fc88c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10dfbda8debaa0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c037070 con 0x7fc89c15e0e0 2024-04-25T10:01:03.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c032440 con 0x7fc89c15e0e0 2024-04-25T10:01:03.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a0ab0640 1 -- 172.21.15.148:0/3769513798 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c032810 con 0x7fc89c15e0e0 2024-04-25T10:01:03.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a3d16640 1 -- 172.21.15.148:0/3769513798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c15e0e0 msgr2=0x7fc89c09a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/3769513798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c15e0e0 0x7fc89c09a150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097a0 tx=0x7fc88c02a230 comp rx=0 tx=0).stop 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a3d16640 1 -- 172.21.15.148:0/3769513798 shutdown_connections 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/3769513798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c15e0e0 0x7fc89c09a150 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.601+0000 7fc8a3d16640 1 -- 172.21.15.148:0/3769513798 >> 172.21.15.148:0/3769513798 conn(0x7fc89c0b3290 msgr2=0x7fc89c0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.602+0000 7fc8a3d16640 1 -- 172.21.15.148:0/3769513798 shutdown_connections 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.602+0000 7fc8a3d16640 1 -- 172.21.15.148:0/3769513798 wait complete. 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 Processor -- start 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 -- start start 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c06bf00 0x7fc89c1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 0x7fc89c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c15e0e0 0x7fc89c1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c09a950 con 0x7fc89c15dd00 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0d0f20 con 0x7fc89c06bf00 2024-04-25T10:01:03.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a3d16640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0b2b40 con 0x7fc89c15e0e0 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.603+0000 7fc8a12b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 0x7fc89c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 0x7fc89c15e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59622/0 (socket says 172.21.15.148:59622) 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 -- 172.21.15.148:0/2197386646 learned_addr learned my addr 172.21.15.148:0/2197386646 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a1ab2640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c06bf00 0x7fc89c1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a22b3640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c15e0e0 0x7fc89c1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 -- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c15e0e0 msgr2=0x7fc89c1f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c15e0e0 0x7fc89c1f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 -- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c06bf00 msgr2=0x7fc89c1482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c06bf00 0x7fc89c1482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 -- 172.21.15.148:0/2197386646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c161280 con 0x7fc89c15dd00 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a22b3640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c15e0e0 0x7fc89c1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8a12b1640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 0x7fc89c15e690 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc88c000c00 tx=0x7fc88c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c041020 con 0x7fc89c15dd00 2024-04-25T10:01:03.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c03d640 con 0x7fc89c15dd00 2024-04-25T10:01:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.604+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c037040 con 0x7fc89c15dd00 2024-04-25T10:01:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.605+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c161560 con 0x7fc89c15dd00 2024-04-25T10:01:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.605+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c148c60 con 0x7fc89c15dd00 2024-04-25T10:01:03.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.606+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c073a00 con 0x7fc89c15dd00 2024-04-25T10:01:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.606+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc88c044020 con 0x7fc89c15dd00 2024-04-25T10:01:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.607+0000 7fc8927fc640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc88006b4d0 0x7fc88006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.607+0000 7fc8a1ab2640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc88006b4d0 0x7fc88006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.607+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc88c0aa7e0 con 0x7fc89c15dd00 2024-04-25T10:01:03.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.607+0000 7fc8a1ab2640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc88006b4d0 0x7fc88006d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc898000f60 tx=0x7fc898009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.611+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc88c04b030 con 0x7fc89c15dd00 2024-04-25T10:01:03.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:01:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:03.794+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc89c09b0d0 con 0x7fc89c15dd00 2024-04-25T10:01:04.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.596+0000 7fc8927fc640 1 -- 172.21.15.148:0/2197386646 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc88c0797e0 con 0x7fc89c15dd00 2024-04-25T10:01:04.600 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:01:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc88006b4d0 msgr2=0x7fc88006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc88006b4d0 0x7fc88006d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc898000f60 tx=0x7fc898009040 comp rx=0 tx=0).stop 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 msgr2=0x7fc89c15e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 0x7fc89c15e690 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc88c000c00 tx=0x7fc88c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 shutdown_connections 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc88006b4d0 0x7fc88006d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c15e0e0 0x7fc89c1f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc89c15dd00 0x7fc89c15e690 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 --2- 172.21.15.148:0/2197386646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c06bf00 0x7fc89c1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 >> 172.21.15.148:0/2197386646 conn(0x7fc89c0b3290 msgr2=0x7fc89c15cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 shutdown_connections 2024-04-25T10:01:04.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.599+0000 7fc8a3d16640 1 -- 172.21.15.148:0/2197386646 wait complete. 2024-04-25T10:01:04.617 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:01:04.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.717+0000 7f6bade09640 1 Processor -- start 2024-04-25T10:01:04.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.717+0000 7f6bade09640 1 -- start start 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.717+0000 7f6bade09640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ba815dbe0 0x7f6ba80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.717+0000 7f6bade09640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ba80b2cb0 0x7f6ba80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6bade09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80b97a0 con 0x7f6ba80b92a0 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6bade09640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba815dfc0 con 0x7f6ba815dbe0 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6bade09640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba80d1a00 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ba815dbe0 0x7f6ba80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6ba80b92a0 0x7f6ba80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33974/0 (socket says 172.21.15.148:33974) 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba7fff640 1 -- 172.21.15.148:0/2337756702 learned_addr learned my addr 172.21.15.148:0/2337756702 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6b9ffff640 1 --2- 172.21.15.148:0/2337756702 >> v2:172.21.15.188:3300/0 conn(0x7f6ba80b2cb0 0x7f6ba80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba77fe640 1 -- 172.21.15.148:0/2337756702 >> v2:172.21.15.188:3300/0 conn(0x7f6ba80b2cb0 msgr2=0x7f6ba80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:04.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba77fe640 1 --2- 172.21.15.148:0/2337756702 >> v2:172.21.15.188:3300/0 conn(0x7f6ba80b2cb0 0x7f6ba80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba6ffd640 1 -- 172.21.15.148:0/2337756702 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134004955 0 0) 0x7f6ba80b97a0 con 0x7f6ba80b92a0 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba77fe640 1 -- 172.21.15.148:0/2337756702 >> v1:172.21.15.148:6789/0 conn(0x7f6ba80b92a0 legacy=0x7f6ba80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6ba77fe640 1 -- 172.21.15.148:0/2337756702 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80d1c50 con 0x7f6ba815dbe0 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.718+0000 7f6b9ffff640 1 --2- 172.21.15.148:0/2337756702 >> v2:172.21.15.188:3300/0 conn(0x7f6ba80b2cb0 0x7f6ba80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6ba77fe640 1 --2- 172.21.15.148:0/2337756702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba815dbe0 0x7f6ba80d0910 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6ba806cff0 tx=0x7f6b9802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ce9f3d35383f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6ba6ffd640 1 -- 172.21.15.148:0/2337756702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037070 con 0x7f6ba815dbe0 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6ba6ffd640 1 -- 172.21.15.148:0/2337756702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b98032440 con 0x7f6ba815dbe0 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6ba6ffd640 1 -- 172.21.15.148:0/2337756702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9802ae60 con 0x7f6ba815dbe0 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 -- 172.21.15.148:0/2337756702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba815dbe0 msgr2=0x7f6ba80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 --2- 172.21.15.148:0/2337756702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba815dbe0 0x7f6ba80d0910 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6ba806cff0 tx=0x7f6b9802a110 comp rx=0 tx=0).stop 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 -- 172.21.15.148:0/2337756702 shutdown_connections 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 --2- 172.21.15.148:0/2337756702 >> v2:172.21.15.188:3300/0 conn(0x7f6ba80b2cb0 0x7f6ba80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 --2- 172.21.15.148:0/2337756702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba815dbe0 0x7f6ba80d0910 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 -- 172.21.15.148:0/2337756702 >> 172.21.15.148:0/2337756702 conn(0x7f6ba8061a20 msgr2=0x7f6ba8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.719+0000 7f6bade09640 1 -- 172.21.15.148:0/2337756702 shutdown_connections 2024-04-25T10:01:04.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.720+0000 7f6bade09640 1 -- 172.21.15.148:0/2337756702 wait complete. 2024-04-25T10:01:04.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 Processor -- start 2024-04-25T10:01:04.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 -- start start 2024-04-25T10:01:04.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 0x7f6ba8151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ba80b92a0 0x7f6ba80cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ba815dbe0 0x7f6ba815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80b6ab0 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80d2210 con 0x7f6ba80b92a0 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6bade09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80b6c30 con 0x7f6ba815dbe0 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 0x7f6ba8151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6ba77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 0x7f6ba8151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59636/0 (socket says 172.21.15.148:59636) 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.721+0000 7f6ba77fe640 1 -- 172.21.15.148:0/1062309362 learned_addr learned my addr 172.21.15.148:0/1062309362 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba77fe640 1 -- 172.21.15.148:0/1062309362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ba815dbe0 msgr2=0x7f6ba815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba77fe640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ba815dbe0 0x7f6ba815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba77fe640 1 -- 172.21.15.148:0/1062309362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ba80b92a0 msgr2=0x7f6ba80cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba77fe640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ba80b92a0 0x7f6ba80cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba77fe640 1 -- 172.21.15.148:0/1062309362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba815ea40 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba77fe640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 0x7f6ba8151e70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037040 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b98042070 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba815ed20 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98030bd0 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.722+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba815f2e0 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.723+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba8073990 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.724+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b98047460 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.724+0000 7f6ba4ff9640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b8806b4d0 0x7f6b8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.724+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6b980aace0 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.724+0000 7f6b9ffff640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b8806b4d0 0x7f6b8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:04.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.724+0000 7f6b9ffff640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b8806b4d0 0x7f6b8806d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6b90006fd0 tx=0x7f6b90008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:04.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.728+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b98032e20 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.969+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6ba815d1d0 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.970+0000 7f6ba4ff9640 1 -- 172.21.15.148:0/1062309362 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6b98045360 con 0x7f6ba80b2cb0 2024-04-25T10:01:04.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:04.974 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:01:04.975 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:04.976 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.977 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.978 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:01:04.979 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.980 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.981 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.982 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:01:04.983 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.984 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:04.985 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:04.986 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:01:04.987 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b8806b4d0 msgr2=0x7f6b8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b8806b4d0 0x7f6b8806d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6b90006fd0 tx=0x7f6b90008040 comp rx=0 tx=0).stop 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 msgr2=0x7f6ba8151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 0x7f6ba8151e70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f6b98000c00 tx=0x7f6b980046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 shutdown_connections 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6b8806b4d0 0x7f6b8806d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ba815dbe0 0x7f6ba815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ba80b92a0 0x7f6ba80cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 --2- 172.21.15.148:0/1062309362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ba80b2cb0 0x7f6ba8151e70 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 >> 172.21.15.148:0/1062309362 conn(0x7f6ba8061a20 msgr2=0x7f6ba815ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 shutdown_connections 2024-04-25T10:01:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:04.972+0000 7f6bade09640 1 -- 172.21.15.148:0/1062309362 wait complete. 2024-04-25T10:01:04.990 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.082+0000 7f464165d640 1 Processor -- start 2024-04-25T10:01:05.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.082+0000 7f464165d640 1 -- start start 2024-04-25T10:01:05.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f464165d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 0x7f463c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f464165d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c0cec00 con 0x7f463c15db60 2024-04-25T10:01:05.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f464165d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c0ceee0 con 0x7f463c0b5f00 2024-04-25T10:01:05.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f464165d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0ce930 con 0x7f463c0d0b10 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f463affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f463c15db60 0x7f463c0d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33978/0 (socket says 172.21.15.148:33978) 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f463affd640 1 -- 172.21.15.148:0/4128692797 learned_addr learned my addr 172.21.15.148:0/4128692797 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.083+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667895890 0 0) 0x7f463c0cec00 con 0x7f463c15db60 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4624003520 con 0x7f463c15db60 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 0x7f463c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1030097279 0 0) 0x7f4624003520 con 0x7f463c15db60 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 msgr2=0x7f463c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 0x7f463c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 >> v1:172.21.15.188:6789/0 conn(0x7f463c0b5f00 legacy=0x7f463c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0cf1c0 con 0x7f463c15db60 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f462c0019f0 con 0x7f463c15db60 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 >> v1:172.21.15.148:6789/0 conn(0x7f463c15db60 legacy=0x7f463c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 0x7f46240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624003f60 0x7f4624004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4624004d40 0x7f46240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0ce930 con 0x7f4624003f60 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462400bb50 con 0x7f4624004d40 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462400bd50 con 0x7f463c0d0b10 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624003f60 0x7f4624004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463affd640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 0x7f46240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463b7fe640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4624004d40 0x7f46240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 -- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 msgr2=0x7f46240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f463c0d0b10 0x7f46240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 -- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4624004d40 msgr2=0x7f46240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4624004d40 0x7f46240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463a7fc640 1 -- 172.21.15.148:0/4128692797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624003520 con 0x7f4624003f60 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.084+0000 7f463b7fe640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4624004d40 0x7f46240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.085+0000 7f463a7fc640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624003f60 0x7f4624004800 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ac0 tx=0x7f4624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90630e0a6d00f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.085+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630036070 con 0x7f4624003f60 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.085+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4630031400 con 0x7f4624003f60 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.085+0000 7f4639ffb640 1 -- 172.21.15.148:0/4128692797 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630009690 con 0x7f4624003f60 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f464165d640 1 -- 172.21.15.148:0/4128692797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624003f60 msgr2=0x7f4624004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f464165d640 1 --2- 172.21.15.148:0/4128692797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624003f60 0x7f4624004800 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ac0 tx=0x7f4624005830 comp rx=0 tx=0).stop 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f463b7fe640 1 -- 172.21.15.148:0/4128692797 reap_dead start 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f464165d640 1 -- 172.21.15.148:0/4128692797 shutdown_connections 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f464165d640 1 -- 172.21.15.148:0/4128692797 >> 172.21.15.148:0/4128692797 conn(0x7f463c061600 msgr2=0x7f463c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f464165d640 1 -- 172.21.15.148:0/4128692797 shutdown_connections 2024-04-25T10:01:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.086+0000 7f464165d640 1 -- 172.21.15.148:0/4128692797 wait complete. 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.087+0000 7f464165d640 1 Processor -- start 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.087+0000 7f464165d640 1 -- start start 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f464165d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624003f60 0x7f463c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f464165d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 0x7f463c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f464165d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c15acb0 0x7f463c15fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f464165d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c15df40 con 0x7f463c151f60 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f464165d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0b6450 con 0x7f463c15acb0 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f464165d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c15e0c0 con 0x7f4624003f60 2024-04-25T10:01:05.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 0x7f463c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 0x7f463c15a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59662/0 (socket says 172.21.15.148:59662) 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 -- 172.21.15.148:0/1547846884 learned_addr learned my addr 172.21.15.148:0/1547846884 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c15acb0 0x7f463c15fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463affd640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624003f60 0x7f463c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 -- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624003f60 msgr2=0x7f463c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624003f60 0x7f463c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 -- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c15acb0 msgr2=0x7f463c15fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c15acb0 0x7f463c15fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 -- 172.21.15.148:0/1547846884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c15e5b0 con 0x7f463c151f60 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463b7fe640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c15acb0 0x7f463c15fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463affd640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624003f60 0x7f463c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f463a7fc640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 0x7f463c15a770 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4630009be0 tx=0x7f4630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.088+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630040020 con 0x7f463c151f60 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.089+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4630003bc0 con 0x7f463c151f60 2024-04-25T10:01:05.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.089+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c15e890 con 0x7f463c151f60 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.089+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630036040 con 0x7f463c151f60 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.089+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c15ee50 con 0x7f463c151f60 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.090+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463c073270 con 0x7f463c151f60 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.090+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4630043020 con 0x7f463c151f60 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.090+0000 7f46237fe640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.090+0000 7f463affd640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.090+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f46300aa180 con 0x7f463c151f60 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.091+0000 7f463affd640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f462c004d00 tx=0x7f462c006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.094+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f463004a070 con 0x7f463c151f60 2024-04-25T10:01:05.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.267+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f463c15fb90 con 0x7f463c151f60 2024-04-25T10:01:05.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.268+0000 7f46237fe640 1 -- 172.21.15.148:0/1547846884 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+13214 (secure 0 0 0) 0x7f46300044e0 con 0x7f463c151f60 2024-04-25T10:01:05.272 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:05.272 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":105,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:04.595474+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 msgr2=0x7f462806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f462c004d00 tx=0x7f462c006040 comp rx=0 tx=0).stop 2024-04-25T10:01:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 msgr2=0x7f463c15a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 0x7f463c15a770 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4630009be0 tx=0x7f4630004730 comp rx=0 tx=0).stop 2024-04-25T10:01:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 shutdown_connections 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f462806b500 0x7f462806d9c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f463c15acb0 0x7f463c15fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463c151f60 0x7f463c15a770 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 --2- 172.21.15.148:0/1547846884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4624003f60 0x7f463c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 >> 172.21.15.148:0/1547846884 conn(0x7f463c061600 msgr2=0x7f463c15bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 shutdown_connections 2024-04-25T10:01:05.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.270+0000 7f464165d640 1 -- 172.21.15.148:0/1547846884 wait complete. 2024-04-25T10:01:05.285 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:05.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:05.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.381+0000 7f4e3fb32640 1 Processor -- start 2024-04-25T10:01:05.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.381+0000 7f4e3fb32640 1 -- start start 2024-04-25T10:01:05.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3fb32640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e380cf600 0x7f4e380cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3fb32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cbf20 0x7f4e380cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3fb32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e380c75e0 con 0x7f4e3815bb50 2024-04-25T10:01:05.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3fb32640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e380c8680 con 0x7f4e380cf600 2024-04-25T10:01:05.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3fb32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380cff80 con 0x7f4e380cbf20 2024-04-25T10:01:05.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3d0cd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4e380cf600 0x7f4e380cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3d8ce640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4e3815bb50 0x7f4e380c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33984/0 (socket says 172.21.15.148:33984) 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.382+0000 7f4e3d8ce640 1 -- 172.21.15.148:0/2556552486 learned_addr learned my addr 172.21.15.148:0/2556552486 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3d0cd640 1 -- 172.21.15.148:0/2556552486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cbf20 msgr2=0x7f4e380cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3d0cd640 1 --2- 172.21.15.148:0/2556552486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cbf20 0x7f4e380cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3d0cd640 1 -- 172.21.15.148:0/2556552486 >> v1:172.21.15.148:6789/0 conn(0x7f4e3815bb50 legacy=0x7f4e380c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3c8cc640 1 -- 172.21.15.148:0/2556552486 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990284705 0 0) 0x7f4e380c75e0 con 0x7f4e3815bb50 2024-04-25T10:01:05.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3d0cd640 1 -- 172.21.15.148:0/2556552486 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380c78c0 con 0x7f4e380cf600 2024-04-25T10:01:05.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3d0cd640 1 --2- 172.21.15.148:0/2556552486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cf600 0x7f4e380cfa40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4e280097a0 tx=0x7f4e2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ff01dc2dae2fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.383+0000 7f4e3c8cc640 1 -- 172.21.15.148:0/2556552486 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28037070 con 0x7f4e380cf600 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3c8cc640 1 -- 172.21.15.148:0/2556552486 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e28032440 con 0x7f4e380cf600 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3c8cc640 1 -- 172.21.15.148:0/2556552486 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28032810 con 0x7f4e380cf600 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 -- 172.21.15.148:0/2556552486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cf600 msgr2=0x7f4e380cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/2556552486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cf600 0x7f4e380cfa40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4e280097a0 tx=0x7f4e2802a210 comp rx=0 tx=0).stop 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 -- 172.21.15.148:0/2556552486 shutdown_connections 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/2556552486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cbf20 0x7f4e380cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/2556552486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cf600 0x7f4e380cfa40 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 -- 172.21.15.148:0/2556552486 >> 172.21.15.148:0/2556552486 conn(0x7f4e3806f500 msgr2=0x7f4e3806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 -- 172.21.15.148:0/2556552486 shutdown_connections 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.384+0000 7f4e3fb32640 1 -- 172.21.15.148:0/2556552486 wait complete. 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.385+0000 7f4e3fb32640 1 Processor -- start 2024-04-25T10:01:05.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.385+0000 7f4e3fb32640 1 -- start start 2024-04-25T10:01:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3fb32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 0x7f4e381550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3fb32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cf600 0x7f4e3814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3fb32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3815bb50 0x7f4e38156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3fb32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380c8160 con 0x7f4e380cbf20 2024-04-25T10:01:05.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3fb32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380c7e80 con 0x7f4e3815bb50 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3fb32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e380c72a0 con 0x7f4e380cf600 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 0x7f4e381550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 0x7f4e381550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59686/0 (socket says 172.21.15.148:59686) 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 -- 172.21.15.148:0/4020575973 learned_addr learned my addr 172.21.15.148:0/4020575973 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 -- 172.21.15.148:0/4020575973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cf600 msgr2=0x7f4e3814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cf600 0x7f4e3814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 -- 172.21.15.148:0/4020575973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3815bb50 msgr2=0x7f4e38156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3815bb50 0x7f4e38156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.386+0000 7f4e3d8ce640 1 -- 172.21.15.148:0/4020575973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38154300 con 0x7f4e380cbf20 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.387+0000 7f4e3d8ce640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 0x7f4e381550a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400c7f0 tx=0x7f4e3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.387+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34018070 con 0x7f4e380cbf20 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.387+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3400a640 con 0x7f4e380cbf20 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.387+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3806ca70 con 0x7f4e380cbf20 2024-04-25T10:01:05.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.387+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400aa30 con 0x7f4e380cbf20 2024-04-25T10:01:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.387+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e381548a0 con 0x7f4e380cbf20 2024-04-25T10:01:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.388+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e38073a00 con 0x7f4e380cbf20 2024-04-25T10:01:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.388+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e3401b070 con 0x7f4e380cbf20 2024-04-25T10:01:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.389+0000 7f4e2e7fc640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.389+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4e34089690 con 0x7f4e380cbf20 2024-04-25T10:01:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.389+0000 7f4e3d0cd640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.389+0000 7f4e3d0cd640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4e28034000 tx=0x7f4e2802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.393+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e34058640 con 0x7f4e380cbf20 2024-04-25T10:01:05.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.563+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e38159140 con 0x7f4e380cbf20 2024-04-25T10:01:05.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.564+0000 7f4e2e7fc640 1 -- 172.21.15.148:0/4020575973 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e34057f40 con 0x7f4e380cbf20 2024-04-25T10:01:05.568 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:01:05.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e1c06b4d0 msgr2=0x7f4e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4e28034000 tx=0x7f4e2802c000 comp rx=0 tx=0).stop 2024-04-25T10:01:05.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 msgr2=0x7f4e381550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 0x7f4e381550a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400c7f0 tx=0x7f4e3400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 shutdown_connections 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e3815bb50 0x7f4e38156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e380cf600 0x7f4e3814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 --2- 172.21.15.148:0/4020575973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e380cbf20 0x7f4e381550a0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 >> 172.21.15.148:0/4020575973 conn(0x7f4e3806f500 msgr2=0x7f4e380b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 shutdown_connections 2024-04-25T10:01:05.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.566+0000 7f4e3fb32640 1 -- 172.21.15.148:0/4020575973 wait complete. 2024-04-25T10:01:05.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:01:05.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.674+0000 7fca90e8f640 1 Processor -- start 2024-04-25T10:01:05.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.674+0000 7fca90e8f640 1 -- start start 2024-04-25T10:01:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca90e8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c15dc90 0x7fca8c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca90e8f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca8c0cc8e0 0x7fca8c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca90e8f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8c15f390 con 0x7fca8c0d1810 2024-04-25T10:01:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca90e8f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0cce50 con 0x7fca8c0cc8e0 2024-04-25T10:01:05.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca90e8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c161af0 con 0x7fca8c15dc90 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca8c0cc8e0 0x7fca8c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fca8c0cc8e0 0x7fca8c15ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59692/0 (socket says 172.21.15.148:59692) 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 -- 172.21.15.148:0/2108393042 learned_addr learned my addr 172.21.15.148:0/2108393042 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca8a59c640 1 --2- 172.21.15.148:0/2108393042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c15dc90 0x7fca8c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 -- 172.21.15.148:0/2108393042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c15dc90 msgr2=0x7fca8c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 --2- 172.21.15.148:0/2108393042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c15dc90 0x7fca8c1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 -- 172.21.15.148:0/2108393042 >> v1:172.21.15.188:6789/0 conn(0x7fca8c0d1810 legacy=0x7fca8c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.675+0000 7fca89d9b640 1 -- 172.21.15.148:0/2108393042 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c161dd0 con 0x7fca8c0cc8e0 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.676+0000 7fca89d9b640 1 --2- 172.21.15.148:0/2108393042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c15ee50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fca80009870 tx=0x7fca8002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d13f3e55dc5f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.676+0000 7fca8959a640 1 -- 172.21.15.148:0/2108393042 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80037070 con 0x7fca8c0cc8e0 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.676+0000 7fca8959a640 1 -- 172.21.15.148:0/2108393042 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80032440 con 0x7fca8c0cc8e0 2024-04-25T10:01:05.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.676+0000 7fca8959a640 1 -- 172.21.15.148:0/2108393042 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80032810 con 0x7fca8c0cc8e0 2024-04-25T10:01:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 -- 172.21.15.148:0/2108393042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c0cc8e0 msgr2=0x7fca8c15ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2108393042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c15ee50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fca80009870 tx=0x7fca8002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 -- 172.21.15.148:0/2108393042 shutdown_connections 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2108393042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c15ee50 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2108393042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c15dc90 0x7fca8c1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 -- 172.21.15.148:0/2108393042 >> 172.21.15.148:0/2108393042 conn(0x7fca8c06f590 msgr2=0x7fca8c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 -- 172.21.15.148:0/2108393042 shutdown_connections 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.677+0000 7fca90e8f640 1 -- 172.21.15.148:0/2108393042 wait complete. 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.678+0000 7fca90e8f640 1 Processor -- start 2024-04-25T10:01:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.678+0000 7fca90e8f640 1 -- start start 2024-04-25T10:01:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c1f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c0d1810 0x7fca8c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 0x7fca8c1f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c15e070 con 0x7fca8c15dc90 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0cd130 con 0x7fca8c0cc8e0 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c15e1f0 con 0x7fca8c0d1810 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 0x7fca8c1f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 0x7fca8c1f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59696/0 (socket says 172.21.15.148:59696) 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 -- 172.21.15.148:0/2701798577 learned_addr learned my addr 172.21.15.148:0/2701798577 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8a59c640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c1f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca89d9b640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c0d1810 0x7fca8c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 -- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c0d1810 msgr2=0x7fca8c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c0d1810 0x7fca8c1f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 -- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8c0cc8e0 msgr2=0x7fca8c1f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c1f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 -- 172.21.15.148:0/2701798577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c1f72d0 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca8ad9d640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 0x7fca8c1f7510 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c760 tx=0x7fca8400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84003b70 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84003d30 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.679+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c159300 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.680+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400aa90 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.680+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c1598c0 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.680+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c073990 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.681+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca8400ac30 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.681+0000 7fca7affd640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca6c06b4d0 0x7fca6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.681+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fca84089ec0 con 0x7fca8c15dc90 2024-04-25T10:01:05.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.681+0000 7fca8a59c640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca6c06b4d0 0x7fca6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.682+0000 7fca8a59c640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca6c06b4d0 0x7fca6c06d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fca7c006fd0 tx=0x7fca7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.685+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca8c15f390 con 0x7fca8c15dc90 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.858+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fca8c06cc60 con 0x7fca8c15dc90 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.859+0000 7fca7affd640 1 -- 172.21.15.148:0/2701798577 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) v1 ==== 84+0+65 (secure 0 0 0) 0x7fca84058f00 con 0x7fca8c15dc90 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:01:05.863 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:05.864 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:01:05.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.860+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca6c06b4d0 msgr2=0x7fca6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.860+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca6c06b4d0 0x7fca6c06d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fca7c006fd0 tx=0x7fca7c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:05.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.860+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 msgr2=0x7fca8c1f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.860+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 0x7fca8c1f7510 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fca8400c760 tx=0x7fca8400cc20 comp rx=0 tx=0).stop 2024-04-25T10:01:05.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 shutdown_connections 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fca6c06b4d0 0x7fca6c06d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca8c15dc90 0x7fca8c1f7510 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8c0d1810 0x7fca8c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 --2- 172.21.15.148:0/2701798577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8c0cc8e0 0x7fca8c1f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 >> 172.21.15.148:0/2701798577 conn(0x7fca8c06f590 msgr2=0x7fca8c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 shutdown_connections 2024-04-25T10:01:05.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.861+0000 7fca90e8f640 1 -- 172.21.15.148:0/2701798577 wait complete. 2024-04-25T10:01:05.876 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:01:05.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:01:05.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.972+0000 7fa1fe3d5640 1 Processor -- start 2024-04-25T10:01:05.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.972+0000 7fa1fe3d5640 1 -- start start 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1fe3d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f806c330 0x7fa1f80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1fe3d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f80d1d50 con 0x7fa1f8158230 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1fe3d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f80d2020 con 0x7fa1f806c7e0 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1fe3d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f8158610 con 0x7fa1f806c330 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1f7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa1f8158230 0x7fa1f80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33988/0 (socket says 172.21.15.148:33988) 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1f7fff640 1 -- 172.21.15.148:0/3720358575 learned_addr learned my addr 172.21.15.148:0/3720358575 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3720358575 >> v2:172.21.15.188:3300/0 conn(0x7fa1f806c330 0x7fa1f80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087971191 0 0) 0x7fa1f80d1d50 con 0x7fa1f8158230 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.973+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e0003520 con 0x7fa1f8158230 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564433905 0 0) 0x7fa1f80d2020 con 0x7fa1f806c7e0 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f80d1d50 con 0x7fa1f806c7e0 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3878960063 0 0) 0x7fa1e0003520 con 0x7fa1f8158230 2024-04-25T10:01:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 >> v2:172.21.15.188:3300/0 conn(0x7fa1f806c330 msgr2=0x7fa1f80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 --2- 172.21.15.148:0/3720358575 >> v2:172.21.15.188:3300/0 conn(0x7fa1f806c330 0x7fa1f80b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 >> v1:172.21.15.188:6789/0 conn(0x7fa1f806c7e0 legacy=0x7fa1f806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80d2300 con 0x7fa1f8158230 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa1e8001670 con 0x7fa1f8158230 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 >> v1:172.21.15.148:6789/0 conn(0x7fa1f8158230 legacy=0x7fa1f80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e0003b40 0x7fa1e0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1e0004800 0x7fa1e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00051c0 0x7fa1e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8158610 con 0x7fa1e0004800 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e000c040 con 0x7fa1e0003b40 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e000c240 con 0x7fa1e00051c0 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1e0004800 0x7fa1e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f7fff640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e0003b40 0x7fa1e0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1fc972640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00051c0 0x7fa1e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00051c0 msgr2=0x7fa1e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00051c0 0x7fa1e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e0003b40 msgr2=0x7fa1e0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e0003b40 0x7fa1e0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3720358575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0003520 con 0x7fa1e0004800 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.974+0000 7fa1fc972640 1 -- 172.21.15.148:0/3720358575 reap_dead start 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1e0004800 0x7fa1e0004c80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009ab0 tx=0x7fa1e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2129c88ffa7a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec036070 con 0x7fa1e0004800 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec005990 con 0x7fa1e0004800 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1f6ffd640 1 -- 172.21.15.148:0/3720358575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec009680 con 0x7fa1e0004800 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3720358575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1e0004800 msgr2=0x7fa1e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1e0004800 0x7fa1e0004c80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009ab0 tx=0x7fa1e0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3720358575 shutdown_connections 2024-04-25T10:01:05.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3720358575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1e0004800 0x7fa1e0004c80 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3720358575 >> 172.21.15.148:0/3720358575 conn(0x7fa1f8061600 msgr2=0x7fa1f80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:05.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3720358575 shutdown_connections 2024-04-25T10:01:05.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.975+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3720358575 wait complete. 2024-04-25T10:01:05.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 Processor -- start 2024-04-25T10:01:05.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 -- start start 2024-04-25T10:01:05.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e0004800 0x7fa1f81ed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 0x7fa1f8158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f81e3fa0 0x7fa1f81e4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80d25e0 con 0x7fa1f8158790 2024-04-25T10:01:05.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f806bea0 con 0x7fa1f81e3fa0 2024-04-25T10:01:05.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.977+0000 7fa1fe3d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f815e320 con 0x7fa1e0004800 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 0x7fa1f8158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 0x7fa1f8158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59726/0 (socket says 172.21.15.148:59726) 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3696911145 learned_addr learned my addr 172.21.15.148:0/3696911145 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1fc972640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f81e3fa0 0x7fa1f81e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e0004800 msgr2=0x7fa1f81ed9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f7fff640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e0004800 0x7fa1f81ed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e0004800 0x7fa1f81ed9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f81e3fa0 msgr2=0x7fa1f81e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f81e3fa0 0x7fa1f81e4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 -- 172.21.15.148:0/3696911145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f81edf10 con 0x7fa1f8158790 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f7fff640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e0004800 0x7fa1f81ed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1fc972640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f81e3fa0 0x7fa1f81e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f77fe640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 0x7fa1f8158c10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec034040 tx=0x7fa1ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec040020 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec006660 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f8159300 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec036040 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.978+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f81598c0 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.979+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f806de50 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.980+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1ec043020 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.980+0000 7fa1f4ff9640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1e406b4d0 0x7fa1e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.980+0000 7fa1f7fff640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1e406b4d0 0x7fa1e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.980+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa1ec0aa440 con 0x7fa1f8158790 2024-04-25T10:01:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.981+0000 7fa1f7fff640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1e406b4d0 0x7fa1e406d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa1f806ff60 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:05.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:05.984+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1ec04a030 con 0x7fa1f8158790 2024-04-25T10:01:06.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.156+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa1f8073a00 con 0x7fa1f8158790 2024-04-25T10:01:06.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.602+0000 7fa1f4ff9640 1 -- 172.21.15.148:0/3696911145 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa1ec079480 con 0x7fa1f8158790 2024-04-25T10:01:06.607 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:01:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.604+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1e406b4d0 msgr2=0x7fa1e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.604+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1e406b4d0 0x7fa1e406d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa1f806ff60 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2024-04-25T10:01:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.604+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 msgr2=0x7fa1f8158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.604+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 0x7fa1f8158c10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec034040 tx=0x7fa1ec004730 comp rx=0 tx=0).stop 2024-04-25T10:01:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 shutdown_connections 2024-04-25T10:01:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1e406b4d0 0x7fa1e406d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f81e3fa0 0x7fa1f81e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f8158790 0x7fa1f8158c10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 --2- 172.21.15.148:0/3696911145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e0004800 0x7fa1f81ed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 >> 172.21.15.148:0/3696911145 conn(0x7fa1f8061600 msgr2=0x7fa1f80ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 shutdown_connections 2024-04-25T10:01:06.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.605+0000 7fa1fe3d5640 1 -- 172.21.15.148:0/3696911145 wait complete. 2024-04-25T10:01:06.624 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:01:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.723+0000 7f41a60ac640 1 Processor -- start 2024-04-25T10:01:06.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.723+0000 7f41a60ac640 1 -- start start 2024-04-25T10:01:06.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.724+0000 7f41a60ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 0x7f41a00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.724+0000 7f41a60ac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a00cccc0 con 0x7f41a0144d50 2024-04-25T10:01:06.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.724+0000 7f41a60ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a00d0460 con 0x7f41a006e8a0 2024-04-25T10:01:06.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.724+0000 7f41a60ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00b2b20 con 0x7f41a01446d0 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.724+0000 7f419effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f41a0144d50 0x7f41a00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34004/0 (socket says 172.21.15.148:34004) 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.724+0000 7f419effd640 1 -- 172.21.15.148:0/1730435368 learned_addr learned my addr 172.21.15.148:0/1730435368 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419f7fe640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 0x7f41a00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035495147 0 0) 0x7f41a00cccc0 con 0x7f41a0144d50 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4188003520 con 0x7f41a0144d50 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802784305 0 0) 0x7f4188003520 con 0x7f41a0144d50 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 msgr2=0x7f41a00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 0x7f41a00b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 >> v1:172.21.15.188:6789/0 conn(0x7f41a006e8a0 legacy=0x7f41a00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a00cc140 con 0x7f41a0144d50 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4194001670 con 0x7f41a0144d50 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 >> v1:172.21.15.148:6789/0 conn(0x7f41a0144d50 legacy=0x7f41a00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880039e0 0x7f4188003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f4188004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 0x7f41880050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00b2b20 con 0x7f4188004390 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418800b9c0 con 0x7f41880039e0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418800bbc0 con 0x7f41a01446d0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419effd640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f4188004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419f7fe640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880039e0 0x7f4188003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419ffff640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 0x7f41880050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419effd640 1 -- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 msgr2=0x7f41880050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419effd640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a01446d0 0x7f41880050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419effd640 1 -- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880039e0 msgr2=0x7f4188003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419effd640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41880039e0 0x7f4188003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.725+0000 7f419effd640 1 -- 172.21.15.148:0/1730435368 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188003520 con 0x7f4188004390 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.726+0000 7f419effd640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f4188004860 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f419400c290 tx=0x7f4188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf11bb4a906cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.726+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194007d50 con 0x7f4188004390 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.726+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4194005af0 con 0x7f4188004390 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.726+0000 7f419e7fc640 1 -- 172.21.15.148:0/1730435368 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194007070 con 0x7f4188004390 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f41a60ac640 1 -- 172.21.15.148:0/1730435368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 msgr2=0x7f4188004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f41a60ac640 1 --2- 172.21.15.148:0/1730435368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f4188004860 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f419400c290 tx=0x7f4188005830 comp rx=0 tx=0).stop 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f419ffff640 1 -- 172.21.15.148:0/1730435368 reap_dead start 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f41a60ac640 1 -- 172.21.15.148:0/1730435368 shutdown_connections 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f41a60ac640 1 -- 172.21.15.148:0/1730435368 >> 172.21.15.148:0/1730435368 conn(0x7f41a006abe0 msgr2=0x7f41a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f41a60ac640 1 -- 172.21.15.148:0/1730435368 shutdown_connections 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.727+0000 7f41a60ac640 1 -- 172.21.15.148:0/1730435368 wait complete. 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.728+0000 7f41a60ac640 1 Processor -- start 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.728+0000 7f41a60ac640 1 -- start start 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f41a00b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a013c610 0x7f41a00b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a015ad80 0x7f41a00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00cc700 con 0x7f4188004390 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a006d400 con 0x7f41a015ad80 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00cc9e0 con 0x7f41a013c610 2024-04-25T10:01:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f41a00b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f41a00b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59746/0 (socket says 172.21.15.148:59746) 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 -- 172.21.15.148:0/2666506732 learned_addr learned my addr 172.21.15.148:0/2666506732 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 -- 172.21.15.148:0/2666506732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a013c610 msgr2=0x7f41a00b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a013c610 0x7f41a00b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 -- 172.21.15.148:0/2666506732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a015ad80 msgr2=0x7f41a00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a015ad80 0x7f41a00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 -- 172.21.15.148:0/2666506732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a013cda0 con 0x7f4188004390 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f419f7fe640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f41a00b14b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f41900119d0 tx=0x7f4190011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000fa90 con 0x7f4188004390 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419000fc30 con 0x7f4188004390 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a013d080 con 0x7f4188004390 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.729+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190014440 con 0x7f4188004390 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.730+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a013d640 con 0x7f4188004390 2024-04-25T10:01:06.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.730+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a0073a00 con 0x7f4188004390 2024-04-25T10:01:06.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.731+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4190012040 con 0x7f4188004390 2024-04-25T10:01:06.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.731+0000 7f4187fff640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f418c06b500 0x7f418c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:06.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.732+0000 7f419effd640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f418c06b500 0x7f418c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:06.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.732+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f419008a850 con 0x7f4188004390 2024-04-25T10:01:06.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.732+0000 7f419effd640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f418c06b500 0x7f418c06d9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f419400c290 tx=0x7f4194009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:06.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.736+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4190019030 con 0x7f4188004390 2024-04-25T10:01:06.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:06.907+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f41a013e570 con 0x7f4188004390 2024-04-25T10:01:07.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.604+0000 7f4187fff640 1 -- 172.21.15.148:0/2666506732 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4190059720 con 0x7f4188004390 2024-04-25T10:01:07.609 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:01:07.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f418c06b500 msgr2=0x7f418c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f418c06b500 0x7f418c06d9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f419400c290 tx=0x7f4194009030 comp rx=0 tx=0).stop 2024-04-25T10:01:07.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 msgr2=0x7f41a00b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f41a00b14b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f41900119d0 tx=0x7f4190011e90 comp rx=0 tx=0).stop 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 shutdown_connections 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f418c06b500 0x7f418c06d9c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a015ad80 0x7f41a00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a013c610 0x7f41a00b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 --2- 172.21.15.148:0/2666506732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4188004390 0x7f41a00b14b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 >> 172.21.15.148:0/2666506732 conn(0x7f41a006abe0 msgr2=0x7f41a01433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 shutdown_connections 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.607+0000 7f41a60ac640 1 -- 172.21.15.148:0/2666506732 wait complete. 2024-04-25T10:01:07.636 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.736+0000 7f6f71ab3640 1 Processor -- start 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.736+0000 7f6f71ab3640 1 -- start start 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f71ab3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f6c0b5620 0x7f6f6c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f71ab3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c06cf50 con 0x7f6f6c06c2d0 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f71ab3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c0ce530 con 0x7f6f6c15b7d0 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f71ab3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c0ce810 con 0x7f6f6c0b5620 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f6c0b5620 0x7f6f6c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6f6c06c2d0 0x7f6f6c06c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34020/0 (socket says 172.21.15.148:34020) 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6bfff640 1 -- 172.21.15.148:0/2568301850 learned_addr learned my addr 172.21.15.148:0/2568301850 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f6c0b5620 0x7f6f6c0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59756/0 (socket says 172.21.15.148:59756) 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6affd640 1 -- 172.21.15.148:0/2568301850 >> v1:172.21.15.188:6789/0 conn(0x7f6f6c15b7d0 legacy=0x7f6f6c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6affd640 1 -- 172.21.15.148:0/2568301850 >> v1:172.21.15.148:6789/0 conn(0x7f6f6c06c2d0 legacy=0x7f6f6c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6affd640 1 -- 172.21.15.148:0/2568301850 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c0ceaf0 con 0x7f6f6c0b5620 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.737+0000 7f6f6a7fc640 1 -- 172.21.15.148:0/2568301850 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000014739 0 0) 0x7f6f6c06cf50 con 0x7f6f6c06c2d0 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.738+0000 7f6f6affd640 1 --2- 172.21.15.148:0/2568301850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0b5ab0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6f600097a0 tx=0x7f6f6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432fdf4feb1d86fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.738+0000 7f6f6a7fc640 1 -- 172.21.15.148:0/2568301850 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60037070 con 0x7f6f6c0b5620 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.738+0000 7f6f6a7fc640 1 -- 172.21.15.148:0/2568301850 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60032440 con 0x7f6f6c0b5620 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.738+0000 7f6f6a7fc640 1 -- 172.21.15.148:0/2568301850 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60032810 con 0x7f6f6c0b5620 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 -- 172.21.15.148:0/2568301850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 msgr2=0x7f6f6c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/2568301850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0b5ab0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6f600097a0 tx=0x7f6f6002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 -- 172.21.15.148:0/2568301850 shutdown_connections 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/2568301850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0b5ab0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 -- 172.21.15.148:0/2568301850 >> 172.21.15.148:0/2568301850 conn(0x7f6f6c0d0800 msgr2=0x7f6f6c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 -- 172.21.15.148:0/2568301850 shutdown_connections 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.739+0000 7f6f71ab3640 1 -- 172.21.15.148:0/2568301850 wait complete. 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.740+0000 7f6f71ab3640 1 Processor -- start 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.740+0000 7f6f71ab3640 1 -- start start 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f71ab3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f6c06c2d0 0x7f6f6c0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f71ab3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f71ab3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f6c15b7d0 0x7f6f6c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f71ab3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c15bd30 con 0x7f6f6c0b5620 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f71ab3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c15bbb0 con 0x7f6f6c15b7d0 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f71ab3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0d1f40 con 0x7f6f6c06c2d0 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59770/0 (socket says 172.21.15.148:59770) 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 -- 172.21.15.148:0/1016951329 learned_addr learned my addr 172.21.15.148:0/1016951329 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6bfff640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f6c15b7d0 0x7f6f6c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6b7fe640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f6c06c2d0 0x7f6f6c0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 -- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f6c06c2d0 msgr2=0x7f6f6c0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f6c06c2d0 0x7f6f6c0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 -- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f6c15b7d0 msgr2=0x7f6f6c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f6c15b7d0 0x7f6f6c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 -- 172.21.15.148:0/1016951329 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c1488c0 con 0x7f6f6c0b5620 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6b7fe640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f6c06c2d0 0x7f6f6c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6bfff640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f6c15b7d0 0x7f6f6c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f6affd640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0c7b50 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6f600098c0 tx=0x7f6f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60041070 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.741+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6003c870 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.742+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60037040 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.742+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c148ba0 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.742+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c0c9a40 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.743+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6c073a00 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.743+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f60044070 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.743+0000 7f6f53fff640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f5806b500 0x7f6f5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.743+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f6f600aaea0 con 0x7f6f6c0b5620 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.744+0000 7f6f6b7fe640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f5806b500 0x7f6f5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.744+0000 7f6f6b7fe640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f5806b500 0x7f6f5806d9c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c006fd0 tx=0x7f6f5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.748+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f60079d10 con 0x7f6f6c0b5620 2024-04-25T10:01:07.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:07.956+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6f6c0b6940 con 0x7f6f6c0b5620 2024-04-25T10:01:08.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.629+0000 7f6f53fff640 1 -- 172.21.15.148:0/1016951329 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6f60079610 con 0x7f6f6c0b5620 2024-04-25T10:01:08.634 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.632+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f5806b500 msgr2=0x7f6f5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.632+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f5806b500 0x7f6f5806d9c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c006fd0 tx=0x7f6f5c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 msgr2=0x7f6f6c0c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0c7b50 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6f600098c0 tx=0x7f6f600046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 shutdown_connections 2024-04-25T10:01:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f5806b500 0x7f6f5806d9c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f6c15b7d0 0x7f6f6c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f6c0b5620 0x7f6f6c0c7b50 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 --2- 172.21.15.148:0/1016951329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f6c06c2d0 0x7f6f6c0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.633+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 >> 172.21.15.148:0/1016951329 conn(0x7f6f6c0d0800 msgr2=0x7f6f6c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.634+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 shutdown_connections 2024-04-25T10:01:08.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.635+0000 7f6f71ab3640 1 -- 172.21.15.148:0/1016951329 wait complete. 2024-04-25T10:01:08.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.763+0000 7f8caa7ee640 1 Processor -- start 2024-04-25T10:01:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.763+0000 7f8caa7ee640 1 -- start start 2024-04-25T10:01:08.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.763+0000 7f8caa7ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ca415b320 0x7f8ca4164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.763+0000 7f8caa7ee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca4071ed0 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.763+0000 7f8caa7ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca415a840 con 0x7f8ca413d190 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.763+0000 7f8caa7ee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca413d570 con 0x7f8ca415b320 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca8d8b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ca415b320 0x7f8ca4164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca8d8b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ca415b320 0x7f8ca4164fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59780/0 (socket says 172.21.15.148:59780) 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca8d8b640 1 -- 172.21.15.148:0/2179327686 learned_addr learned my addr 172.21.15.148:0/2179327686 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ca40cc6c0 0x7f8ca4071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34034/0 (socket says 172.21.15.148:34034) 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca8d8b640 1 -- 172.21.15.148:0/2179327686 >> v1:172.21.15.188:6789/0 conn(0x7f8ca413d190 legacy=0x7f8ca40cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca8d8b640 1 -- 172.21.15.148:0/2179327686 >> v1:172.21.15.148:6789/0 conn(0x7f8ca40cc6c0 legacy=0x7f8ca4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca8d8b640 1 -- 172.21.15.148:0/2179327686 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca415ab20 con 0x7f8ca415b320 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.764+0000 7f8ca2ffd640 1 -- 172.21.15.148:0/2179327686 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875465863 0 0) 0x7f8ca4071ed0 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8ca8d8b640 1 --2- 172.21.15.148:0/2179327686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca415b320 0x7f8ca4164fb0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8ca413d6f0 tx=0x7f8c9c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90c10992a2080d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8ca2ffd640 1 -- 172.21.15.148:0/2179327686 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c036070 con 0x7f8ca415b320 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8ca2ffd640 1 -- 172.21.15.148:0/2179327686 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c031600 con 0x7f8ca415b320 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8ca2ffd640 1 -- 172.21.15.148:0/2179327686 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0319d0 con 0x7f8ca415b320 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2179327686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca415b320 msgr2=0x7f8ca4164fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2179327686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca415b320 0x7f8ca4164fb0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8ca413d6f0 tx=0x7f8c9c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2179327686 shutdown_connections 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2179327686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca415b320 0x7f8ca4164fb0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2179327686 >> 172.21.15.148:0/2179327686 conn(0x7f8ca406abe0 msgr2=0x7f8ca406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2179327686 shutdown_connections 2024-04-25T10:01:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.765+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2179327686 wait complete. 2024-04-25T10:01:08.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.766+0000 7f8caa7ee640 1 Processor -- start 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 -- start start 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 0x7f8ca413a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ca413d190 0x7f8ca40b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca415b320 0x7f8ca40b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4071ac0 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4071940 con 0x7f8ca415b320 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8caa7ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4071c40 con 0x7f8ca413d190 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 0x7f8ca413a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 0x7f8ca413a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59790/0 (socket says 172.21.15.148:59790) 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.767+0000 7f8ca3fff640 1 -- 172.21.15.148:0/2274934549 learned_addr learned my addr 172.21.15.148:0/2274934549 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca3fff640 1 -- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ca413d190 msgr2=0x7f8ca40b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca3fff640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ca413d190 0x7f8ca40b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca3fff640 1 -- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca415b320 msgr2=0x7f8ca40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca8d8b640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca415b320 0x7f8ca40b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca3fff640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca415b320 0x7f8ca40b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca3fff640 1 -- 172.21.15.148:0/2274934549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40b2e80 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca8d8b640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca415b320 0x7f8ca40b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca3fff640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 0x7f8ca413a1f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c460 tx=0x7f8c9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94016020 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94012870 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca40b0b50 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94012c60 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.768+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca40b1110 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.769+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca4073990 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.769+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c94019020 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.770+0000 7f8ca0ff9640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b500 0x7f8c9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:08.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.770+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c94089e00 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.770+0000 7f8ca37fe640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b500 0x7f8c9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:08.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.770+0000 7f8ca37fe640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b500 0x7f8c9006d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8ca413c400 tx=0x7f8c98007940 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:08.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.774+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c94020070 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.894 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-25T10:01:08.889+0000 7f57d1526640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:08.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.945+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ca40b2130 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.947+0000 7f8ca0ff9640 1 -- 172.21.15.148:0/2274934549 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f8ca40b2e80 con 0x7f8ca40cc6c0 2024-04-25T10:01:08.951 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:08.951 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":108,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:07.957838+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:08.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b500 msgr2=0x7f8c9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b500 0x7f8c9006d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8ca413c400 tx=0x7f8c98007940 comp rx=0 tx=0).stop 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 msgr2=0x7f8ca413a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 0x7f8ca413a1f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c460 tx=0x7f8c9400c920 comp rx=0 tx=0).stop 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 shutdown_connections 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c9006b500 0x7f8c9006d9c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ca415b320 0x7f8ca40b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ca413d190 0x7f8ca40b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 --2- 172.21.15.148:0/2274934549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ca40cc6c0 0x7f8ca413a1f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 >> 172.21.15.148:0/2274934549 conn(0x7f8ca406abe0 msgr2=0x7f8ca4158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 shutdown_connections 2024-04-25T10:01:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:08.949+0000 7f8caa7ee640 1 -- 172.21.15.148:0/2274934549 wait complete. 2024-04-25T10:01:08.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:09.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.058+0000 7f4bbb607640 1 Processor -- start 2024-04-25T10:01:09.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.058+0000 7f4bbb607640 1 -- start start 2024-04-25T10:01:09.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bbb607640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bb40cfec0 0x7f4bb40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bbb607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bbb607640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb4071480 con 0x7f4bb4154ee0 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bbb607640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb40ce830 con 0x7f4bb40cfec0 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bbb607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb4072090 con 0x7f4bb40ce3c0 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bb40cfec0 0x7f4bb40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bb40cfec0 0x7f4bb40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59794/0 (socket says 172.21.15.148:59794) 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/248061986 learned_addr learned my addr 172.21.15.148:0/248061986 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb9ba4640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/248061986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 msgr2=0x7f4bb406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/248061986 >> v1:172.21.15.188:6789/0 conn(0x7f4bb4154ee0 legacy=0x7f4bb40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/248061986 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4071760 con 0x7f4bb40cfec0 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.059+0000 7f4bb9ba4640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.060+0000 7f4bb8ba2640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cde80 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ebb5f2f4405691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.060+0000 7f4babfff640 1 -- 172.21.15.148:0/248061986 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4037070 con 0x7f4bb40cfec0 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.060+0000 7f4babfff640 1 -- 172.21.15.148:0/248061986 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4032440 con 0x7f4bb40cfec0 2024-04-25T10:01:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.060+0000 7f4babfff640 1 -- 172.21.15.148:0/248061986 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4032810 con 0x7f4bb40cfec0 2024-04-25T10:01:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 -- 172.21.15.148:0/248061986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 msgr2=0x7f4bb40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cde80 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba402a210 comp rx=0 tx=0).stop 2024-04-25T10:01:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 -- 172.21.15.148:0/248061986 shutdown_connections 2024-04-25T10:01:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cde80 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 --2- 172.21.15.148:0/248061986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 -- 172.21.15.148:0/248061986 >> 172.21.15.148:0/248061986 conn(0x7f4bb406abe0 msgr2=0x7f4bb406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 -- 172.21.15.148:0/248061986 shutdown_connections 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.061+0000 7f4bbb607640 1 -- 172.21.15.148:0/248061986 wait complete. 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.062+0000 7f4bbb607640 1 Processor -- start 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.062+0000 7f4bbb607640 1 -- start start 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb41f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4154ee0 0x7f4bb40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb4071d20 con 0x7f4bb40cfec0 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb4071230 con 0x7f4bb4154ee0 2024-04-25T10:01:09.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb41552c0 con 0x7f4bb40ce3c0 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59800/0 (socket says 172.21.15.148:59800) 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/740149962 learned_addr learned my addr 172.21.15.148:0/740149962 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb93a3640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb41f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb9ba4640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4154ee0 0x7f4bb40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 msgr2=0x7f4bb41f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb41f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4154ee0 msgr2=0x7f4bb40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4154ee0 0x7f4bb40cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 -- 172.21.15.148:0/740149962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb41f2710 con 0x7f4bb40cfec0 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bb8ba2640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cc0a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4041020 con 0x7f4bb40cfec0 2024-04-25T10:01:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4030970 con 0x7f4bb40cfec0 2024-04-25T10:01:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4037040 con 0x7f4bb40cfec0 2024-04-25T10:01:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.063+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb41ef880 con 0x7f4bb40cfec0 2024-04-25T10:01:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.064+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb41f3120 con 0x7f4bb40cfec0 2024-04-25T10:01:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.065+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb4073990 con 0x7f4bb40cfec0 2024-04-25T10:01:09.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.065+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ba4044020 con 0x7f4bb40cfec0 2024-04-25T10:01:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.065+0000 7f4ba9ffb640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b4d0 0x7f4b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.066+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ba40aad00 con 0x7f4bb40cfec0 2024-04-25T10:01:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.066+0000 7f4bb93a3640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b4d0 0x7f4b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.066+0000 7f4bb93a3640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b4d0 0x7f4b9806d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0006fd0 tx=0x7f4bb0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.070+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ba404b030 con 0x7f4bb40cfec0 2024-04-25T10:01:09.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.244+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bb406d360 con 0x7f4bb40cfec0 2024-04-25T10:01:09.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.245+0000 7f4ba9ffb640 1 -- 172.21.15.148:0/740149962 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f4bb41f2710 con 0x7f4bb40cfec0 2024-04-25T10:01:09.250 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:09.250 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":108,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:07.957838+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b4d0 msgr2=0x7f4b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b4d0 0x7f4b9806d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0006fd0 tx=0x7f4bb0008040 comp rx=0 tx=0).stop 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 msgr2=0x7f4bb40cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cc0a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009870 tx=0x7f4ba40046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 shutdown_connections 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4b9806b4d0 0x7f4b9806d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bb4154ee0 0x7f4bb40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bb40cfec0 0x7f4bb40cc0a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 --2- 172.21.15.148:0/740149962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bb40ce3c0 0x7f4bb41f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 >> 172.21.15.148:0/740149962 conn(0x7f4bb406abe0 msgr2=0x7f4bb415c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 shutdown_connections 2024-04-25T10:01:09.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.247+0000 7f4bbb607640 1 -- 172.21.15.148:0/740149962 wait complete. 2024-04-25T10:01:09.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:01:09.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.359+0000 7fa3cba88640 1 Processor -- start 2024-04-25T10:01:09.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.359+0000 7fa3cba88640 1 -- start start 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3cba88640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3cba88640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 0x7fa3c40cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3cba88640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c415e3a0 con 0x7fa3c40cd540 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3cba88640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c40cd9d0 con 0x7fa3c40d0de0 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3cba88640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40cdc70 con 0x7fa3c40d0a00 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9824640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9023640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 0x7fa3c40cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9023640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 0x7fa3c40cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:44198/0 (socket says 172.21.15.148:44198) 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9023640 1 -- 172.21.15.148:0/2315937826 learned_addr learned my addr 172.21.15.148:0/2315937826 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3ca025640 1 --1- 172.21.15.148:0/2315937826 >> v1:172.21.15.188:6789/0 conn(0x7fa3c40cd540 0x7fa3c40caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:54650/0 (socket says 172.21.15.148:54650) 2024-04-25T10:01:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9824640 1 -- 172.21.15.148:0/2315937826 >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 msgr2=0x7fa3c40cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9824640 1 --2- 172.21.15.148:0/2315937826 >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 0x7fa3c40cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9824640 1 -- 172.21.15.148:0/2315937826 >> v1:172.21.15.188:6789/0 conn(0x7fa3c40cd540 legacy=0x7fa3c40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9824640 1 -- 172.21.15.148:0/2315937826 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40cdf50 con 0x7fa3c40d0a00 2024-04-25T10:01:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.360+0000 7fa3c9023640 1 --2- 172.21.15.148:0/2315937826 >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 0x7fa3c40cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.361+0000 7fa3c9824640 1 --2- 172.21.15.148:0/2315937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4162140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00097a0 tx=0x7fa3c002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cabcc6b5367aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.361+0000 7fa3c8822640 1 -- 172.21.15.148:0/2315937826 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0037070 con 0x7fa3c40d0a00 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3c8822640 1 -- 172.21.15.148:0/2315937826 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c002ae50 con 0x7fa3c40d0a00 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3c8822640 1 -- 172.21.15.148:0/2315937826 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0032660 con 0x7fa3c40d0a00 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 -- 172.21.15.148:0/2315937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 msgr2=0x7fa3c4162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 --2- 172.21.15.148:0/2315937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4162140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00097a0 tx=0x7fa3c002a100 comp rx=0 tx=0).stop 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 -- 172.21.15.148:0/2315937826 shutdown_connections 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 --2- 172.21.15.148:0/2315937826 >> v2:172.21.15.188:3300/0 conn(0x7fa3c40d0de0 0x7fa3c40cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 --2- 172.21.15.148:0/2315937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4162140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 -- 172.21.15.148:0/2315937826 >> 172.21.15.148:0/2315937826 conn(0x7fa3c40d17c0 msgr2=0x7fa3c40d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 -- 172.21.15.148:0/2315937826 shutdown_connections 2024-04-25T10:01:09.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.362+0000 7fa3cba88640 1 -- 172.21.15.148:0/2315937826 wait complete. 2024-04-25T10:01:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.363+0000 7fa3cba88640 1 Processor -- start 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.363+0000 7fa3cba88640 1 -- start start 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3cba88640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c40cd540 0x7fa3c41f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3cba88640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3cba88640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 0x7fa3c4146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3cba88640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4161c20 con 0x7fa3c40d0de0 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3cba88640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4161940 con 0x7fa3c40cd540 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3cba88640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40d1260 con 0x7fa3c40d0a00 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3ca025640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 0x7fa3c4146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3ca025640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 0x7fa3c4146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59806/0 (socket says 172.21.15.148:59806) 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3ca025640 1 -- 172.21.15.148:0/1331268694 learned_addr learned my addr 172.21.15.148:0/1331268694 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3c9824640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c40cd540 0x7fa3c41f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3c9023640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3ca025640 1 -- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 msgr2=0x7fa3c4145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.364+0000 7fa3ca025640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ca025640 1 -- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c40cd540 msgr2=0x7fa3c41f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ca025640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c40cd540 0x7fa3c41f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ca025640 1 -- 172.21.15.148:0/1331268694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40b68e0 con 0x7fa3c40d0de0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ca025640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 0x7fa3c4146120 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c780 tx=0x7fa3bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc017070 con 0x7fa3c40d0de0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc00a640 con 0x7fa3c40d0de0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c4143a50 con 0x7fa3c40d0de0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00a9f0 con 0x7fa3c40d0de0 2024-04-25T10:01:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.365+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c40b6bc0 con 0x7fa3c40d0de0 2024-04-25T10:01:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.366+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c4073a00 con 0x7fa3c40d0de0 2024-04-25T10:01:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.366+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3bc01a070 con 0x7fa3c40d0de0 2024-04-25T10:01:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.367+0000 7fa3ba7fc640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a806b4d0 0x7fa3a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.367+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3bc089820 con 0x7fa3c40d0de0 2024-04-25T10:01:09.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.367+0000 7fa3c9824640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a806b4d0 0x7fa3a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.367+0000 7fa3c9824640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a806b4d0 0x7fa3a806d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0034000 tx=0x7fa3c002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.371+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3bc057f50 con 0x7fa3c40d0de0 2024-04-25T10:01:09.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.577+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa3c40d0360 con 0x7fa3c40d0de0 2024-04-25T10:01:09.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.760+0000 7fa3ba7fc640 1 -- 172.21.15.148:0/1331268694 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa3bc05bff0 con 0x7fa3c40d0de0 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a806b4d0 msgr2=0x7fa3a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a806b4d0 0x7fa3a806d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0034000 tx=0x7fa3c002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 msgr2=0x7fa3c4146120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 0x7fa3c4146120 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c780 tx=0x7fa3bc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 shutdown_connections 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a806b4d0 0x7fa3a806d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3c40d0de0 0x7fa3c4146120 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3c40d0a00 0x7fa3c4145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 --2- 172.21.15.148:0/1331268694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3c40cd540 0x7fa3c41f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 >> 172.21.15.148:0/1331268694 conn(0x7fa3c40d17c0 msgr2=0x7fa3c415f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 shutdown_connections 2024-04-25T10:01:09.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.763+0000 7fa3cba88640 1 -- 172.21.15.148:0/1331268694 wait complete. 2024-04-25T10:01:09.779 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:01:09.779 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:01:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.874+0000 7fe3ac370640 1 Processor -- start 2024-04-25T10:01:09.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.874+0000 7fe3ac370640 1 -- start start 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.875+0000 7fe3ac370640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 0x7fe3a40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.875+0000 7fe3ac370640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe3a40cfd10 0x7fe3a40cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.875+0000 7fe3ac370640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a40c75e0 con 0x7fe3a40c70d0 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.875+0000 7fe3ac370640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a40c8680 con 0x7fe3a40cfd10 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.875+0000 7fe3ac370640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40c78c0 con 0x7fe3a415bb50 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a990b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe3a40c70d0 0x7fe3a40cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34040/0 (socket says 172.21.15.148:34040) 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a990b640 1 -- 172.21.15.148:0/76705756 learned_addr learned my addr 172.21.15.148:0/76705756 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974104139 0 0) 0x7fe3a40c75e0 con 0x7fe3a40c70d0 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe390003520 con 0x7fe3a40c70d0 2024-04-25T10:01:09.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3aa90d640 1 --2- 172.21.15.148:0/76705756 >> v2:172.21.15.188:3300/0 conn(0x7fe3a40cfd10 0x7fe3a40cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3aa10c640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 0x7fe3a40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281829502 0 0) 0x7fe390003520 con 0x7fe3a40c70d0 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 msgr2=0x7fe3a40c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 0x7fe3a40c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 >> v2:172.21.15.188:3300/0 conn(0x7fe3a40cfd10 msgr2=0x7fe3a40cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 --2- 172.21.15.148:0/76705756 >> v2:172.21.15.188:3300/0 conn(0x7fe3a40cfd10 0x7fe3a40cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40c7ba0 con 0x7fe3a40c70d0 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe3a0001670 con 0x7fe3a40c70d0 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3aa90d640 1 --2- 172.21.15.148:0/76705756 >> v2:172.21.15.188:3300/0 conn(0x7fe3a40cfd10 0x7fe3a40cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 >> v1:172.21.15.148:6789/0 conn(0x7fe3a40c70d0 legacy=0x7fe3a40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3900039b0 0x7fe390003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe390004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 0x7fe3900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40c8680 con 0x7fe390004800 2024-04-25T10:01:09.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40c78c0 con 0x7fe3900039b0 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39000ba70 con 0x7fe3a415bb50 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3a990b640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe390004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3aa10c640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3900039b0 0x7fe390003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.876+0000 7fe3aa90d640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 0x7fe3900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a990b640 1 -- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 msgr2=0x7fe3900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a990b640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a415bb50 0x7fe3900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a990b640 1 -- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3900039b0 msgr2=0x7fe390003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a990b640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3900039b0 0x7fe390003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a990b640 1 -- 172.21.15.148:0/76705756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390003520 con 0x7fe390004800 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a990b640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe390004c80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0001050 tx=0x7fe390005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ae4c7ba985fa33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0007d70 con 0x7fe390004800 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0005af0 con 0x7fe390004800 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.877+0000 7fe3a910a640 1 -- 172.21.15.148:0/76705756 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0007070 con 0x7fe390004800 2024-04-25T10:01:09.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3ac370640 1 -- 172.21.15.148:0/76705756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 msgr2=0x7fe390004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3ac370640 1 --2- 172.21.15.148:0/76705756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe390004c80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0001050 tx=0x7fe390005830 comp rx=0 tx=0).stop 2024-04-25T10:01:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3aa90d640 1 -- 172.21.15.148:0/76705756 reap_dead start 2024-04-25T10:01:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3ac370640 1 -- 172.21.15.148:0/76705756 shutdown_connections 2024-04-25T10:01:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3ac370640 1 -- 172.21.15.148:0/76705756 >> 172.21.15.148:0/76705756 conn(0x7fe3a406f500 msgr2=0x7fe3a406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3ac370640 1 -- 172.21.15.148:0/76705756 shutdown_connections 2024-04-25T10:01:09.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.878+0000 7fe3ac370640 1 -- 172.21.15.148:0/76705756 wait complete. 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.879+0000 7fe3ac370640 1 Processor -- start 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.879+0000 7fe3ac370640 1 -- start start 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3ac370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe3a40b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3ac370640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a414c2f0 0x7fe3a40b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3ac370640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a40b65f0 0x7fe3a415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3ac370640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a415f6d0 con 0x7fe390004800 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3ac370640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40c8160 con 0x7fe3a40b65f0 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3ac370640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a415f850 con 0x7fe3a414c2f0 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe3a40b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe3a40b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59816/0 (socket says 172.21.15.148:59816) 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 -- 172.21.15.148:0/3695794205 learned_addr learned my addr 172.21.15.148:0/3695794205 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3a990b640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a414c2f0 0x7fe3a40b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa90d640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a40b65f0 0x7fe3a415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 -- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a414c2f0 msgr2=0x7fe3a40b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a414c2f0 0x7fe3a40b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 -- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a40b65f0 msgr2=0x7fe3a415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a40b65f0 0x7fe3a415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 -- 172.21.15.148:0/3695794205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40b50b0 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.880+0000 7fe3aa10c640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe3a40b6e70 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe3940119a0 tx=0x7fe394011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.881+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400fa90 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.881+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39400fc30 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.881+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a40b5390 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.881+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394014440 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.881+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a40b5950 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.882+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a4073a00 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.882+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe394012040 con 0x7fe390004800 2024-04-25T10:01:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.882+0000 7fe39affd640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38806b500 0x7fe38806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.883+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe39408a850 con 0x7fe390004800 2024-04-25T10:01:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.883+0000 7fe3a990b640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38806b500 0x7fe38806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.883+0000 7fe3a990b640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38806b500 0x7fe38806d9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0033000 tx=0x7fe3a0009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:09.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:09.887+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe394019030 con 0x7fe390004800 2024-04-25T10:01:10.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.094+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe3a40b6c50 con 0x7fe390004800 2024-04-25T10:01:10.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:01:10.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.762+0000 7fe39affd640 1 -- 172.21.15.148:0/3695794205 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe394058f80 con 0x7fe390004800 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38806b500 msgr2=0x7fe38806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38806b500 0x7fe38806d9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0033000 tx=0x7fe3a0009030 comp rx=0 tx=0).stop 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 msgr2=0x7fe3a40b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe3a40b6e70 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe3940119a0 tx=0x7fe394011e60 comp rx=0 tx=0).stop 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 shutdown_connections 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38806b500 0x7fe38806d9c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a40b65f0 0x7fe3a415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a414c2f0 0x7fe3a40b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 --2- 172.21.15.148:0/3695794205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe390004800 0x7fe3a40b6e70 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 >> 172.21.15.148:0/3695794205 conn(0x7fe3a406f500 msgr2=0x7fe3a4070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 shutdown_connections 2024-04-25T10:01:10.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.764+0000 7fe3ac370640 1 -- 172.21.15.148:0/3695794205 wait complete. 2024-04-25T10:01:10.781 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:01:10.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:10.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.874+0000 7f3ba9f6a640 1 Processor -- start 2024-04-25T10:01:10.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 -- start start 2024-04-25T10:01:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba415dbe0 0x7f3ba40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ba40b2cb0 0x7f3ba40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ba40b92a0 0x7f3ba40b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba415dfc0 con 0x7f3ba40b92a0 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40d1a00 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba9f6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b97a0 con 0x7f3ba415dbe0 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ba40b92a0 0x7f3ba40b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.875+0000 7f3ba3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3ba40b92a0 0x7f3ba40b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59818/0 (socket says 172.21.15.148:59818) 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 -- 172.21.15.148:0/1629988666 learned_addr learned my addr 172.21.15.148:0/1629988666 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 -- 172.21.15.148:0/1629988666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba415dbe0 msgr2=0x7f3ba40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba37fe640 1 --2- 172.21.15.148:0/1629988666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba415dbe0 0x7f3ba40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba2ffd640 1 --2- 172.21.15.148:0/1629988666 >> v2:172.21.15.188:3300/0 conn(0x7f3ba40b2cb0 0x7f3ba40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 --2- 172.21.15.148:0/1629988666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba415dbe0 0x7f3ba40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 -- 172.21.15.148:0/1629988666 >> v2:172.21.15.188:3300/0 conn(0x7f3ba40b2cb0 msgr2=0x7f3ba40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 --2- 172.21.15.148:0/1629988666 >> v2:172.21.15.188:3300/0 conn(0x7f3ba40b2cb0 0x7f3ba40d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 -- 172.21.15.148:0/1629988666 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40d1c50 con 0x7f3ba40b92a0 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.876+0000 7f3ba3fff640 1 --2- 172.21.15.148:0/1629988666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b92a0 0x7f3ba40b5400 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b64d0 tx=0x7f3b9c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c318a31500963bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba27fc640 1 -- 172.21.15.148:0/1629988666 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c036070 con 0x7f3ba40b92a0 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba27fc640 1 -- 172.21.15.148:0/1629988666 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c031600 con 0x7f3ba40b92a0 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba27fc640 1 -- 172.21.15.148:0/1629988666 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0319d0 con 0x7f3ba40b92a0 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/1629988666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b92a0 msgr2=0x7f3ba40b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/1629988666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b92a0 0x7f3ba40b5400 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b64d0 tx=0x7f3b9c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/1629988666 shutdown_connections 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/1629988666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b92a0 0x7f3ba40b5400 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/1629988666 >> v2:172.21.15.188:3300/0 conn(0x7f3ba40b2cb0 0x7f3ba40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/1629988666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba415dbe0 0x7f3ba40d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/1629988666 >> 172.21.15.148:0/1629988666 conn(0x7f3ba4061a20 msgr2=0x7f3ba4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/1629988666 shutdown_connections 2024-04-25T10:01:10.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.877+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/1629988666 wait complete. 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.878+0000 7f3ba9f6a640 1 Processor -- start 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 -- start start 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 0x7f3ba4154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba40b92a0 0x7f3ba41f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba415dbe0 0x7f3ba41f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b6ab0 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40d2210 con 0x7f3ba415dbe0 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba9f6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b6c30 con 0x7f3ba40b92a0 2024-04-25T10:01:10.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 0x7f3ba4154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 0x7f3ba4154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59826/0 (socket says 172.21.15.148:59826) 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.879+0000 7f3ba37fe640 1 -- 172.21.15.148:0/3368774551 learned_addr learned my addr 172.21.15.148:0/3368774551 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba2ffd640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba40b92a0 0x7f3ba41f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba3fff640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba415dbe0 0x7f3ba41f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba37fe640 1 -- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba40b92a0 msgr2=0x7f3ba41f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba37fe640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba40b92a0 0x7f3ba41f3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba37fe640 1 -- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba415dbe0 msgr2=0x7f3ba41f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba37fe640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba415dbe0 0x7f3ba41f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba37fe640 1 -- 172.21.15.148:0/3368774551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba4152b40 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba37fe640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 0x7f3ba4154cd0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c7f0 tx=0x7f3b9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94015070 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94004220 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba4152e20 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94007620 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.880+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41533e0 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.881+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4073990 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.881+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b940079d0 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.882+0000 7f3b8bfff640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b9006b4d0 0x7f3b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.882+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b9408a2a0 con 0x7f3ba40b2cb0 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.882+0000 7f3ba2ffd640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b9006b4d0 0x7f3b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.882+0000 7f3ba2ffd640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b9006b4d0 0x7f3b9006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b98006fd0 tx=0x7f3b98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:10.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:10.886+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b94059020 con 0x7f3ba40b2cb0 2024-04-25T10:01:11.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.091+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba40d2bd0 con 0x7f3ba40b2cb0 2024-04-25T10:01:11.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.092+0000 7f3b8bfff640 1 -- 172.21.15.148:0/3368774551 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f3b9408a0c0 con 0x7f3ba40b2cb0 2024-04-25T10:01:11.097 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:11.097 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:11.098 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:11.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b9006b4d0 msgr2=0x7f3b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b9006b4d0 0x7f3b9006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b98006fd0 tx=0x7f3b98008040 comp rx=0 tx=0).stop 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 msgr2=0x7f3ba4154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 0x7f3ba4154cd0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c7f0 tx=0x7f3b9400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 shutdown_connections 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b9006b4d0 0x7f3b9006d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ba415dbe0 0x7f3ba41f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ba40b92a0 0x7f3ba41f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 --2- 172.21.15.148:0/3368774551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ba40b2cb0 0x7f3ba4154cd0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 >> 172.21.15.148:0/3368774551 conn(0x7f3ba4061a20 msgr2=0x7f3ba4158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 shutdown_connections 2024-04-25T10:01:11.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.094+0000 7f3ba9f6a640 1 -- 172.21.15.148:0/3368774551 wait complete. 2024-04-25T10:01:11.110 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:11.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.201+0000 7f6f97e41640 1 Processor -- start 2024-04-25T10:01:11.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f97e41640 1 -- start start 2024-04-25T10:01:11.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f97e41640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f90154ee0 0x7f6f900cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f97e41640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f900cfcf0 0x7f6f900cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f97e41640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f90071480 con 0x7f6f900ce3c0 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f97e41640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f900ce830 con 0x7f6f900cfcf0 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f97e41640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f90072090 con 0x7f6f90154ee0 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.202+0000 7f6f953dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f900cfcf0 0x7f6f900cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6f900cfcf0 0x7f6f900cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59832/0 (socket says 172.21.15.148:59832) 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 -- 172.21.15.148:0/3229490844 learned_addr learned my addr 172.21.15.148:0/3229490844 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f95bdd640 1 --2- 172.21.15.148:0/3229490844 >> v2:172.21.15.188:3300/0 conn(0x7f6f90154ee0 0x7f6f900cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 -- 172.21.15.148:0/3229490844 >> v2:172.21.15.188:3300/0 conn(0x7f6f90154ee0 msgr2=0x7f6f900cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 --2- 172.21.15.148:0/3229490844 >> v2:172.21.15.188:3300/0 conn(0x7f6f90154ee0 0x7f6f900cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 -- 172.21.15.148:0/3229490844 >> v1:172.21.15.188:6789/0 conn(0x7f6f900ce3c0 legacy=0x7f6f9015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 -- 172.21.15.148:0/3229490844 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90071760 con 0x7f6f900cfcf0 2024-04-25T10:01:11.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.203+0000 7f6f953dc640 1 --2- 172.21.15.148:0/3229490844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900cfcf0 0x7f6f900cde80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6f900d02e0 tx=0x7f6f8002c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60b9d2c70cbebe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f94bdb640 1 -- 172.21.15.148:0/3229490844 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80036020 con 0x7f6f900cfcf0 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f94bdb640 1 -- 172.21.15.148:0/3229490844 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f80032640 con 0x7f6f900cfcf0 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f94bdb640 1 -- 172.21.15.148:0/3229490844 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80032a10 con 0x7f6f900cfcf0 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 -- 172.21.15.148:0/3229490844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900cfcf0 msgr2=0x7f6f900cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 --2- 172.21.15.148:0/3229490844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900cfcf0 0x7f6f900cde80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6f900d02e0 tx=0x7f6f8002c940 comp rx=0 tx=0).stop 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 -- 172.21.15.148:0/3229490844 shutdown_connections 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 --2- 172.21.15.148:0/3229490844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900cfcf0 0x7f6f900cde80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 --2- 172.21.15.148:0/3229490844 >> v2:172.21.15.188:3300/0 conn(0x7f6f90154ee0 0x7f6f900cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 -- 172.21.15.148:0/3229490844 >> 172.21.15.148:0/3229490844 conn(0x7f6f9006abe0 msgr2=0x7f6f9006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 -- 172.21.15.148:0/3229490844 shutdown_connections 2024-04-25T10:01:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.204+0000 7f6f97e41640 1 -- 172.21.15.148:0/3229490844 wait complete. 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 Processor -- start 2024-04-25T10:01:11.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 -- start start 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 0x7f6f90157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f900cfcf0 0x7f6f90159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f90154ee0 0x7f6f901555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90071d20 con 0x7f6f900ce3c0 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90071230 con 0x7f6f90154ee0 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f97e41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f901552c0 con 0x7f6f900cfcf0 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f95bdd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 0x7f6f90157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f95bdd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 0x7f6f90157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59836/0 (socket says 172.21.15.148:59836) 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.206+0000 7f6f95bdd640 1 -- 172.21.15.148:0/2422480279 learned_addr learned my addr 172.21.15.148:0/2422480279 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f963de640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f90154ee0 0x7f6f901555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f95bdd640 1 -- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f900cfcf0 msgr2=0x7f6f90159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f95bdd640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f900cfcf0 0x7f6f90159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f95bdd640 1 -- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f90154ee0 msgr2=0x7f6f901555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f95bdd640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f90154ee0 0x7f6f901555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f95bdd640 1 -- 172.21.15.148:0/2422480279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90151ce0 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f95bdd640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 0x7f6f90157080 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800d5f0 tx=0x7f6f8800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880125f0 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f88019070 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880129d0 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f90151a00 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.207+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f90159310 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.208+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f90073990 con 0x7f6f900ce3c0 2024-04-25T10:01:11.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.208+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f88012b70 con 0x7f6f900ce3c0 2024-04-25T10:01:11.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.209+0000 7f6f867fc640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f7406b4d0 0x7f6f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.209+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f8808a7a0 con 0x7f6f900ce3c0 2024-04-25T10:01:11.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.209+0000 7f6f953dc640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f7406b4d0 0x7f6f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.209+0000 7f6f953dc640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f7406b4d0 0x7f6f7406d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6f80037000 tx=0x7f6f80031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.213+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f88059490 con 0x7f6f900ce3c0 2024-04-25T10:01:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.389+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f90158e20 con 0x7f6f900ce3c0 2024-04-25T10:01:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.390+0000 7f6f867fc640 1 -- 172.21.15.148:0/2422480279 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f6f88058d90 con 0x7f6f900ce3c0 2024-04-25T10:01:11.396 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:11.396 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":109,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:09.636711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f7406b4d0 msgr2=0x7f6f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f7406b4d0 0x7f6f7406d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6f80037000 tx=0x7f6f80031000 comp rx=0 tx=0).stop 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 msgr2=0x7f6f90157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 0x7f6f90157080 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800d5f0 tx=0x7f6f8800dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 shutdown_connections 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f7406b4d0 0x7f6f7406d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f90154ee0 0x7f6f901555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f900cfcf0 0x7f6f90159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 --2- 172.21.15.148:0/2422480279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f900ce3c0 0x7f6f90157080 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 >> 172.21.15.148:0/2422480279 conn(0x7f6f9006abe0 msgr2=0x7f6f900b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 shutdown_connections 2024-04-25T10:01:11.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.392+0000 7f6f97e41640 1 -- 172.21.15.148:0/2422480279 wait complete. 2024-04-25T10:01:11.409 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:01:11.409 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:11.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.501+0000 7f3ac9b09640 1 Processor -- start 2024-04-25T10:01:11.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.501+0000 7f3ac9b09640 1 -- start start 2024-04-25T10:01:11.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac9b09640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ac415dc20 0x7f3ac41616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac9b09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac415e000 con 0x7f3ac40cedf0 2024-04-25T10:01:11.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac9b09640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac40b6bb0 con 0x7f3ac40b93a0 2024-04-25T10:01:11.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac9b09640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac40b6030 con 0x7f3ac415dc20 2024-04-25T10:01:11.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3ac40cedf0 0x7f3ac40d2510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34056/0 (socket says 172.21.15.148:34056) 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac3fff640 1 -- 172.21.15.148:0/2656100232 learned_addr learned my addr 172.21.15.148:0/2656100232 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2656100232 >> v2:172.21.15.188:3300/0 conn(0x7f3ac415dc20 0x7f3ac41616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303627700 0 0) 0x7f3abc001490 con 0x7f3ac40cedf0 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.502+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aac003520 con 0x7f3ac40cedf0 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1180321989 0 0) 0x7f3aac003520 con 0x7f3ac40cedf0 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 >> v2:172.21.15.188:3300/0 conn(0x7f3ac415dc20 msgr2=0x7f3ac41616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 --2- 172.21.15.148:0/2656100232 >> v2:172.21.15.188:3300/0 conn(0x7f3ac415dc20 0x7f3ac41616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 >> v1:172.21.15.188:6789/0 conn(0x7f3ac40b93a0 legacy=0x7f3ac40ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4161de0 con 0x7f3ac40cedf0 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3abc001bc0 con 0x7f3ac40cedf0 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 >> v1:172.21.15.148:6789/0 conn(0x7f3ac40cedf0 legacy=0x7f3ac40d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac0039b0 0x7f3aac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0043b0 0x7f3aac004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aac005080 0x7f3aac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac40b6030 con 0x7f3aac0039b0 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac00bfe0 con 0x7f3aac005080 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac00c1e0 con 0x7f3aac0043b0 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac0039b0 0x7f3aac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac3fff640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aac005080 0x7f3aac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac2ffd640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0043b0 0x7f3aac004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac37fe640 1 -- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0043b0 msgr2=0x7f3aac004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0043b0 0x7f3aac004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac37fe640 1 -- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aac005080 msgr2=0x7f3aac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aac005080 0x7f3aac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac37fe640 1 -- 172.21.15.148:0/2656100232 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac003520 con 0x7f3aac0039b0 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac2ffd640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0043b0 0x7f3aac004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac3fff640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aac005080 0x7f3aac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:11.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.503+0000 7f3ac3fff640 1 -- 172.21.15.148:0/2656100232 reap_dead start 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac0039b0 0x7f3aac003d90 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009ac0 tx=0x7f3aac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5274e5d7ab506a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4036070 con 0x7f3aac0039b0 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab4031400 con 0x7f3aac0039b0 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac27fc640 1 -- 172.21.15.148:0/2656100232 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4009690 con 0x7f3aac0039b0 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2656100232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac0039b0 msgr2=0x7f3aac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac0039b0 0x7f3aac003d90 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009ac0 tx=0x7f3aac005830 comp rx=0 tx=0).stop 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2656100232 shutdown_connections 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2656100232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac0039b0 0x7f3aac003d90 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.504+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2656100232 >> 172.21.15.148:0/2656100232 conn(0x7f3ac40d0490 msgr2=0x7f3ac40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.505+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2656100232 shutdown_connections 2024-04-25T10:01:11.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.505+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2656100232 wait complete. 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.506+0000 7f3ac9b09640 1 Processor -- start 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.506+0000 7f3ac9b09640 1 -- start start 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.506+0000 7f3ac9b09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0039b0 0x7f3ac415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ac415f240 0x7f3ac41f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 0x7f3ac41f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41616b0 con 0x7f3ac415f6c0 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac41623a0 con 0x7f3ac415f240 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac4161830 con 0x7f3aac0039b0 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 0x7f3ac41f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 0x7f3ac41f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59850/0 (socket says 172.21.15.148:59850) 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 -- 172.21.15.148:0/2300925350 learned_addr learned my addr 172.21.15.148:0/2300925350 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 -- 172.21.15.148:0/2300925350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0039b0 msgr2=0x7f3ac415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0039b0 0x7f3ac415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 -- 172.21.15.148:0/2300925350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ac415f240 msgr2=0x7f3ac41f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ac415f240 0x7f3ac41f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 -- 172.21.15.148:0/2300925350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac415e1e0 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac3fff640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 0x7f3ac41f4110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40b6600 tx=0x7f3abc007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc019070 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc00ea80 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00ee30 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac415e480 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.507+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac415ea40 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.508+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac4073990 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.509+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3abc01c070 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.509+0000 7f3aabfff640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.509+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3abc089f00 con 0x7f3ac415f6c0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.509+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.510+0000 7f3ac37fe640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab006b4d0 0x7f3ab006d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4033000 tx=0x7f3ab402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.513+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ac415e000 con 0x7f3ac415f6c0 2024-04-25T10:01:11.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.684+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ac40d1720 con 0x7f3ab006b4d0 2024-04-25T10:01:11.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.690+0000 7f3aabfff640 1 -- 172.21.15.148:0/2300925350 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252450 (secure 0 0 0) 0x7f3ab407ea40 con 0x7f3ab006b4d0 2024-04-25T10:01:11.698 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab006b4d0 msgr2=0x7f3ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab006b4d0 0x7f3ab006d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4033000 tx=0x7f3ab402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 msgr2=0x7f3ac41f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 0x7f3ac41f4110 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40b6600 tx=0x7f3abc007bc0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 shutdown_connections 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab006b4d0 0x7f3ab006d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.693+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ac415f6c0 0x7f3ac41f4110 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.694+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ac415f240 0x7f3ac41f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.694+0000 7f3ac9b09640 1 --2- 172.21.15.148:0/2300925350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aac0039b0 0x7f3ac415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.694+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 >> 172.21.15.148:0/2300925350 conn(0x7f3ac40d0490 msgr2=0x7f3ac40b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.694+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 shutdown_connections 2024-04-25T10:01:11.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.694+0000 7f3ac9b09640 1 -- 172.21.15.148:0/2300925350 wait complete. 2024-04-25T10:01:11.713 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-04-25T10:01:11.104108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":369448,"kb_used_data":11688,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754605272,"statfs":{"total":773094113280,"available":772715798528,"internally_reserved":0,"allocated":11968512,"data_stored":6159508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":366202416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.500230"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966648+0000","last_change":"2024-04-25T10:01:08.647566+0000","last_active":"2024-04-25T10:01:09.966648+0000","last_peered":"2024-04-25T10:01:09.966648+0000","last_clean":"2024-04-25T10:01:09.966648+0000","last_became_active":"2024-04-25T10:01:08.646882+0000","last_became_peered":"2024-04-25T10:01:08.646882+0000","last_unstale":"2024-04-25T10:01:09.966648+0000","last_undegraded":"2024-04-25T10:01:09.966648+0000","last_fullsized":"2024-04-25T10:01:09.966648+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:24.047699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660615+0000","last_change":"2024-04-25T10:01:08.659726+0000","last_active":"2024-04-25T10:01:08.660615+0000","last_peered":"2024-04-25T10:01:08.660615+0000","last_clean":"2024-04-25T10:01:08.660615+0000","last_became_active":"2024-04-25T10:01:08.659434+0000","last_became_peered":"2024-04-25T10:01:08.659434+0000","last_unstale":"2024-04-25T10:01:08.660615+0000","last_undegraded":"2024-04-25T10:01:08.660615+0000","last_fullsized":"2024-04-25T10:01:08.660615+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:06.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966569+0000","last_change":"2024-04-25T10:01:08.657288+0000","last_active":"2024-04-25T10:01:09.966569+0000","last_peered":"2024-04-25T10:01:09.966569+0000","last_clean":"2024-04-25T10:01:09.966569+0000","last_became_active":"2024-04-25T10:01:08.657027+0000","last_became_peered":"2024-04-25T10:01:08.657027+0000","last_unstale":"2024-04-25T10:01:09.966569+0000","last_undegraded":"2024-04-25T10:01:09.966569+0000","last_fullsized":"2024-04-25T10:01:09.966569+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:47.589375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.654696+0000","last_change":"2024-04-25T10:01:08.652623+0000","last_active":"2024-04-25T10:01:08.654696+0000","last_peered":"2024-04-25T10:01:08.654696+0000","last_clean":"2024-04-25T10:01:08.654696+0000","last_became_active":"2024-04-25T10:01:08.652327+0000","last_became_peered":"2024-04-25T10:01:08.652327+0000","last_unstale":"2024-04-25T10:01:08.654696+0000","last_undegraded":"2024-04-25T10:01:08.654696+0000","last_fullsized":"2024-04-25T10:01:08.654696+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658205+0000","last_change":"2024-04-25T10:01:08.655399+0000","last_active":"2024-04-25T10:01:08.658205+0000","last_peered":"2024-04-25T10:01:08.658205+0000","last_clean":"2024-04-25T10:01:08.658205+0000","last_became_active":"2024-04-25T10:01:08.655098+0000","last_became_peered":"2024-04-25T10:01:08.655098+0000","last_unstale":"2024-04-25T10:01:08.658205+0000","last_undegraded":"2024-04-25T10:01:08.658205+0000","last_fullsized":"2024-04-25T10:01:08.658205+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:42.660604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.657790+0000","last_change":"2024-04-25T10:01:08.656730+0000","last_active":"2024-04-25T10:01:08.657790+0000","last_peered":"2024-04-25T10:01:08.657790+0000","last_clean":"2024-04-25T10:01:08.657790+0000","last_became_active":"2024-04-25T10:01:08.654062+0000","last_became_peered":"2024-04-25T10:01:08.654062+0000","last_unstale":"2024-04-25T10:01:08.657790+0000","last_undegraded":"2024-04-25T10:01:08.657790+0000","last_fullsized":"2024-04-25T10:01:08.657790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:10.050353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.661418+0000","last_change":"2024-04-25T10:01:08.660907+0000","last_active":"2024-04-25T10:01:08.661418+0000","last_peered":"2024-04-25T10:01:08.661418+0000","last_clean":"2024-04-25T10:01:08.661418+0000","last_became_active":"2024-04-25T10:01:08.660556+0000","last_became_peered":"2024-04-25T10:01:08.660556+0000","last_unstale":"2024-04-25T10:01:08.661418+0000","last_undegraded":"2024-04-25T10:01:08.661418+0000","last_fullsized":"2024-04-25T10:01:08.661418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:47.879606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639182+0000","last_change":"2024-04-25T10:01:08.650932+0000","last_active":"2024-04-25T10:01:09.639182+0000","last_peered":"2024-04-25T10:01:09.639182+0000","last_clean":"2024-04-25T10:01:09.639182+0000","last_became_active":"2024-04-25T10:01:08.650667+0000","last_became_peered":"2024-04-25T10:01:08.650667+0000","last_unstale":"2024-04-25T10:01:09.639182+0000","last_undegraded":"2024-04-25T10:01:09.639182+0000","last_fullsized":"2024-04-25T10:01:09.639182+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:05.371195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660928+0000","last_change":"2024-04-25T10:01:08.660465+0000","last_active":"2024-04-25T10:01:08.660928+0000","last_peered":"2024-04-25T10:01:08.660928+0000","last_clean":"2024-04-25T10:01:08.660928+0000","last_became_active":"2024-04-25T10:01:08.660165+0000","last_became_peered":"2024-04-25T10:01:08.660165+0000","last_unstale":"2024-04-25T10:01:08.660928+0000","last_undegraded":"2024-04-25T10:01:08.660928+0000","last_fullsized":"2024-04-25T10:01:08.660928+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:05.836377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659903+0000","last_change":"2024-04-25T10:01:08.657254+0000","last_active":"2024-04-25T10:01:08.659903+0000","last_peered":"2024-04-25T10:01:08.659903+0000","last_clean":"2024-04-25T10:01:08.659903+0000","last_became_active":"2024-04-25T10:01:08.656957+0000","last_became_peered":"2024-04-25T10:01:08.656957+0000","last_unstale":"2024-04-25T10:01:08.659903+0000","last_undegraded":"2024-04-25T10:01:08.659903+0000","last_fullsized":"2024-04-25T10:01:08.659903+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:10.050353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651563+0000","last_change":"2024-04-25T10:01:08.649835+0000","last_active":"2024-04-25T10:01:08.651563+0000","last_peered":"2024-04-25T10:01:08.651563+0000","last_clean":"2024-04-25T10:01:08.651563+0000","last_became_active":"2024-04-25T10:01:08.649566+0000","last_became_peered":"2024-04-25T10:01:08.649566+0000","last_unstale":"2024-04-25T10:01:08.651563+0000","last_undegraded":"2024-04-25T10:01:08.651563+0000","last_fullsized":"2024-04-25T10:01:08.651563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642817+0000","last_change":"2024-04-25T10:01:08.639879+0000","last_active":"2024-04-25T10:01:09.642817+0000","last_peered":"2024-04-25T10:01:09.642817+0000","last_clean":"2024-04-25T10:01:09.642817+0000","last_became_active":"2024-04-25T10:01:08.639533+0000","last_became_peered":"2024-04-25T10:01:08.639533+0000","last_unstale":"2024-04-25T10:01:09.642817+0000","last_undegraded":"2024-04-25T10:01:09.642817+0000","last_fullsized":"2024-04-25T10:01:09.642817+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.655825+0000","last_change":"2024-04-25T10:01:08.652617+0000","last_active":"2024-04-25T10:01:08.655825+0000","last_peered":"2024-04-25T10:01:08.655825+0000","last_clean":"2024-04-25T10:01:08.655825+0000","last_became_active":"2024-04-25T10:01:08.652296+0000","last_became_peered":"2024-04-25T10:01:08.652296+0000","last_unstale":"2024-04-25T10:01:08.655825+0000","last_undegraded":"2024-04-25T10:01:08.655825+0000","last_fullsized":"2024-04-25T10:01:08.655825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.643276+0000","last_change":"2024-04-25T10:01:08.642861+0000","last_active":"2024-04-25T10:01:08.643276+0000","last_peered":"2024-04-25T10:01:08.643276+0000","last_clean":"2024-04-25T10:01:08.643276+0000","last_became_active":"2024-04-25T10:01:08.642564+0000","last_became_peered":"2024-04-25T10:01:08.642564+0000","last_unstale":"2024-04-25T10:01:08.643276+0000","last_undegraded":"2024-04-25T10:01:08.643276+0000","last_fullsized":"2024-04-25T10:01:08.643276+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651528+0000","last_change":"2024-04-25T10:01:08.649954+0000","last_active":"2024-04-25T10:01:08.651528+0000","last_peered":"2024-04-25T10:01:08.651528+0000","last_clean":"2024-04-25T10:01:08.651528+0000","last_became_active":"2024-04-25T10:01:08.649696+0000","last_became_peered":"2024-04-25T10:01:08.649696+0000","last_unstale":"2024-04-25T10:01:08.651528+0000","last_undegraded":"2024-04-25T10:01:08.651528+0000","last_fullsized":"2024-04-25T10:01:08.651528+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.652304+0000","last_change":"2024-04-25T10:01:08.651124+0000","last_active":"2024-04-25T10:01:08.652304+0000","last_peered":"2024-04-25T10:01:08.652304+0000","last_clean":"2024-04-25T10:01:08.652304+0000","last_became_active":"2024-04-25T10:01:08.650871+0000","last_became_peered":"2024-04-25T10:01:08.650871+0000","last_unstale":"2024-04-25T10:01:08.652304+0000","last_undegraded":"2024-04-25T10:01:08.652304+0000","last_fullsized":"2024-04-25T10:01:08.652304+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:18.169599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966690+0000","last_change":"2024-04-25T10:01:08.658019+0000","last_active":"2024-04-25T10:01:09.966690+0000","last_peered":"2024-04-25T10:01:09.966690+0000","last_clean":"2024-04-25T10:01:09.966690+0000","last_became_active":"2024-04-25T10:01:08.657652+0000","last_became_peered":"2024-04-25T10:01:08.657652+0000","last_unstale":"2024-04-25T10:01:09.966690+0000","last_undegraded":"2024-04-25T10:01:09.966690+0000","last_fullsized":"2024-04-25T10:01:09.966690+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:12.079829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660332+0000","last_change":"2024-04-25T10:01:08.659396+0000","last_active":"2024-04-25T10:01:08.660332+0000","last_peered":"2024-04-25T10:01:08.660332+0000","last_clean":"2024-04-25T10:01:08.660332+0000","last_became_active":"2024-04-25T10:01:08.659071+0000","last_became_peered":"2024-04-25T10:01:08.659071+0000","last_unstale":"2024-04-25T10:01:08.660332+0000","last_undegraded":"2024-04-25T10:01:08.660332+0000","last_fullsized":"2024-04-25T10:01:08.660332+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:26.192422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966624+0000","last_change":"2024-04-25T10:01:08.656704+0000","last_active":"2024-04-25T10:01:09.966624+0000","last_peered":"2024-04-25T10:01:09.966624+0000","last_clean":"2024-04-25T10:01:09.966624+0000","last_became_active":"2024-04-25T10:01:08.653888+0000","last_became_peered":"2024-04-25T10:01:08.653888+0000","last_unstale":"2024-04-25T10:01:09.966624+0000","last_undegraded":"2024-04-25T10:01:09.966624+0000","last_fullsized":"2024-04-25T10:01:09.966624+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:22.665485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642860+0000","last_change":"2024-04-25T10:01:08.641167+0000","last_active":"2024-04-25T10:01:09.642860+0000","last_peered":"2024-04-25T10:01:09.642860+0000","last_clean":"2024-04-25T10:01:09.642860+0000","last_became_active":"2024-04-25T10:01:08.640835+0000","last_became_peered":"2024-04-25T10:01:08.640835+0000","last_unstale":"2024-04-25T10:01:09.642860+0000","last_undegraded":"2024-04-25T10:01:09.642860+0000","last_fullsized":"2024-04-25T10:01:09.642860+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:18.169599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639608+0000","last_change":"2024-04-25T10:01:08.647220+0000","last_active":"2024-04-25T10:01:09.639608+0000","last_peered":"2024-04-25T10:01:09.639608+0000","last_clean":"2024-04-25T10:01:09.639608+0000","last_became_active":"2024-04-25T10:01:08.646913+0000","last_became_peered":"2024-04-25T10:01:08.646913+0000","last_unstale":"2024-04-25T10:01:09.639608+0000","last_undegraded":"2024-04-25T10:01:09.639608+0000","last_fullsized":"2024-04-25T10:01:09.639608+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:23.714488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651680+0000","last_change":"2024-04-25T10:01:08.650009+0000","last_active":"2024-04-25T10:01:08.651680+0000","last_peered":"2024-04-25T10:01:08.651680+0000","last_clean":"2024-04-25T10:01:08.651680+0000","last_became_active":"2024-04-25T10:01:08.649454+0000","last_became_peered":"2024-04-25T10:01:08.649454+0000","last_unstale":"2024-04-25T10:01:08.651680+0000","last_undegraded":"2024-04-25T10:01:08.651680+0000","last_fullsized":"2024-04-25T10:01:08.651680+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:04.944422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966513+0000","last_change":"2024-04-25T10:01:08.653957+0000","last_active":"2024-04-25T10:01:09.966513+0000","last_peered":"2024-04-25T10:01:09.966513+0000","last_clean":"2024-04-25T10:01:09.966513+0000","last_became_active":"2024-04-25T10:01:08.653160+0000","last_became_peered":"2024-04-25T10:01:08.653160+0000","last_unstale":"2024-04-25T10:01:09.966513+0000","last_undegraded":"2024-04-25T10:01:09.966513+0000","last_fullsized":"2024-04-25T10:01:09.966513+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:55.438978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.650623+0000","last_change":"2024-04-25T10:01:08.648591+0000","last_active":"2024-04-25T10:01:08.650623+0000","last_peered":"2024-04-25T10:01:08.650623+0000","last_clean":"2024-04-25T10:01:08.650623+0000","last_became_active":"2024-04-25T10:01:08.648271+0000","last_became_peered":"2024-04-25T10:01:08.648271+0000","last_unstale":"2024-04-25T10:01:08.650623+0000","last_undegraded":"2024-04-25T10:01:08.650623+0000","last_fullsized":"2024-04-25T10:01:08.650623+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:07.176637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642605+0000","last_change":"2024-04-25T10:01:08.651605+0000","last_active":"2024-04-25T10:01:09.642605+0000","last_peered":"2024-04-25T10:01:09.642605+0000","last_clean":"2024-04-25T10:01:09.642605+0000","last_became_active":"2024-04-25T10:01:08.651308+0000","last_became_peered":"2024-04-25T10:01:08.651308+0000","last_unstale":"2024-04-25T10:01:09.642605+0000","last_undegraded":"2024-04-25T10:01:09.642605+0000","last_fullsized":"2024-04-25T10:01:09.642605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642576+0000","last_change":"2024-04-25T10:01:08.654590+0000","last_active":"2024-04-25T10:01:09.642576+0000","last_peered":"2024-04-25T10:01:09.642576+0000","last_clean":"2024-04-25T10:01:09.642576+0000","last_became_active":"2024-04-25T10:01:08.654230+0000","last_became_peered":"2024-04-25T10:01:08.654230+0000","last_unstale":"2024-04-25T10:01:09.642576+0000","last_undegraded":"2024-04-25T10:01:09.642576+0000","last_fullsized":"2024-04-25T10:01:09.642576+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651469+0000","last_change":"2024-04-25T10:01:08.646886+0000","last_active":"2024-04-25T10:01:08.651469+0000","last_peered":"2024-04-25T10:01:08.651469+0000","last_clean":"2024-04-25T10:01:08.651469+0000","last_became_active":"2024-04-25T10:01:08.646205+0000","last_became_peered":"2024-04-25T10:01:08.646205+0000","last_unstale":"2024-04-25T10:01:08.651469+0000","last_undegraded":"2024-04-25T10:01:08.651469+0000","last_fullsized":"2024-04-25T10:01:08.651469+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:40.344941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660320+0000","last_change":"2024-04-25T10:01:08.657289+0000","last_active":"2024-04-25T10:01:08.660320+0000","last_peered":"2024-04-25T10:01:08.660320+0000","last_clean":"2024-04-25T10:01:08.660320+0000","last_became_active":"2024-04-25T10:01:08.656965+0000","last_became_peered":"2024-04-25T10:01:08.656965+0000","last_unstale":"2024-04-25T10:01:08.660320+0000","last_undegraded":"2024-04-25T10:01:08.660320+0000","last_fullsized":"2024-04-25T10:01:08.660320+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:27.203638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639692+0000","last_change":"2024-04-25T10:01:08.647041+0000","last_active":"2024-04-25T10:01:09.639692+0000","last_peered":"2024-04-25T10:01:09.639692+0000","last_clean":"2024-04-25T10:01:09.639692+0000","last_became_active":"2024-04-25T10:01:08.646548+0000","last_became_peered":"2024-04-25T10:01:08.646548+0000","last_unstale":"2024-04-25T10:01:09.639692+0000","last_undegraded":"2024-04-25T10:01:09.639692+0000","last_fullsized":"2024-04-25T10:01:09.639692+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:14.963989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642545+0000","last_change":"2024-04-25T10:01:08.651842+0000","last_active":"2024-04-25T10:01:09.642545+0000","last_peered":"2024-04-25T10:01:09.642545+0000","last_clean":"2024-04-25T10:01:09.642545+0000","last_became_active":"2024-04-25T10:01:08.651513+0000","last_became_peered":"2024-04-25T10:01:08.651513+0000","last_unstale":"2024-04-25T10:01:09.642545+0000","last_undegraded":"2024-04-25T10:01:09.642545+0000","last_fullsized":"2024-04-25T10:01:09.642545+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:21.459828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.657563+0000","last_change":"2024-04-25T10:01:08.650755+0000","last_active":"2024-04-25T10:01:08.657563+0000","last_peered":"2024-04-25T10:01:08.657563+0000","last_clean":"2024-04-25T10:01:08.657563+0000","last_became_active":"2024-04-25T10:01:08.650259+0000","last_became_peered":"2024-04-25T10:01:08.650259+0000","last_unstale":"2024-04-25T10:01:08.657563+0000","last_undegraded":"2024-04-25T10:01:08.657563+0000","last_fullsized":"2024-04-25T10:01:08.657563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:29.785417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639258+0000","last_change":"2024-04-25T10:01:08.637412+0000","last_active":"2024-04-25T10:01:09.639258+0000","last_peered":"2024-04-25T10:01:09.639258+0000","last_clean":"2024-04-25T10:01:09.639258+0000","last_became_active":"2024-04-25T10:01:08.636910+0000","last_became_peered":"2024-04-25T10:01:08.636910+0000","last_unstale":"2024-04-25T10:01:09.639258+0000","last_undegraded":"2024-04-25T10:01:09.639258+0000","last_fullsized":"2024-04-25T10:01:09.639258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:54.744535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966807+0000","last_change":"2024-04-25T10:01:08.660401+0000","last_active":"2024-04-25T10:01:09.966807+0000","last_peered":"2024-04-25T10:01:09.966807+0000","last_clean":"2024-04-25T10:01:09.966807+0000","last_became_active":"2024-04-25T10:01:08.659797+0000","last_became_peered":"2024-04-25T10:01:08.659797+0000","last_unstale":"2024-04-25T10:01:09.966807+0000","last_undegraded":"2024-04-25T10:01:09.966807+0000","last_fullsized":"2024-04-25T10:01:09.966807+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:33.359857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"108'4","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659580+0000","last_change":"2024-04-25T10:01:07.627632+0000","last_active":"2024-04-25T10:01:08.659580+0000","last_peered":"2024-04-25T10:01:08.659580+0000","last_clean":"2024-04-25T10:01:08.659580+0000","last_became_active":"2024-04-25T10:01:07.627373+0000","last_became_peered":"2024-04-25T10:01:07.627373+0000","last_unstale":"2024-04-25T10:01:08.659580+0000","last_undegraded":"2024-04-25T10:01:08.659580+0000","last_fullsized":"2024-04-25T10:01:08.659580+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:19.078513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639382+0000","last_change":"2024-04-25T10:01:08.650263+0000","last_active":"2024-04-25T10:01:09.639382+0000","last_peered":"2024-04-25T10:01:09.639382+0000","last_clean":"2024-04-25T10:01:09.639382+0000","last_became_active":"2024-04-25T10:01:08.650000+0000","last_became_peered":"2024-04-25T10:01:08.650000+0000","last_unstale":"2024-04-25T10:01:09.639382+0000","last_undegraded":"2024-04-25T10:01:09.639382+0000","last_fullsized":"2024-04-25T10:01:09.639382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:05.021630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639536+0000","last_change":"2024-04-25T10:01:07.620530+0000","last_active":"2024-04-25T10:01:09.639536+0000","last_peered":"2024-04-25T10:01:09.639536+0000","last_clean":"2024-04-25T10:01:09.639536+0000","last_became_active":"2024-04-25T10:01:07.619062+0000","last_became_peered":"2024-04-25T10:01:07.619062+0000","last_unstale":"2024-04-25T10:01:09.639536+0000","last_undegraded":"2024-04-25T10:01:09.639536+0000","last_fullsized":"2024-04-25T10:01:09.639536+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:14.062695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642716+0000","last_change":"2024-04-25T10:01:08.655561+0000","last_active":"2024-04-25T10:01:09.642716+0000","last_peered":"2024-04-25T10:01:09.642716+0000","last_clean":"2024-04-25T10:01:09.642716+0000","last_became_active":"2024-04-25T10:01:08.652594+0000","last_became_peered":"2024-04-25T10:01:08.652594+0000","last_unstale":"2024-04-25T10:01:09.642716+0000","last_undegraded":"2024-04-25T10:01:09.642716+0000","last_fullsized":"2024-04-25T10:01:09.642716+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642503+0000","last_change":"2024-04-25T10:01:07.625695+0000","last_active":"2024-04-25T10:01:09.642503+0000","last_peered":"2024-04-25T10:01:09.642503+0000","last_clean":"2024-04-25T10:01:09.642503+0000","last_became_active":"2024-04-25T10:01:07.625326+0000","last_became_peered":"2024-04-25T10:01:07.625326+0000","last_unstale":"2024-04-25T10:01:09.642503+0000","last_undegraded":"2024-04-25T10:01:09.642503+0000","last_fullsized":"2024-04-25T10:01:09.642503+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.102035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659826+0000","last_change":"2024-04-25T10:01:08.656149+0000","last_active":"2024-04-25T10:01:08.659826+0000","last_peered":"2024-04-25T10:01:08.659826+0000","last_clean":"2024-04-25T10:01:08.659826+0000","last_became_active":"2024-04-25T10:01:08.655844+0000","last_became_peered":"2024-04-25T10:01:08.655844+0000","last_unstale":"2024-04-25T10:01:08.659826+0000","last_undegraded":"2024-04-25T10:01:08.659826+0000","last_fullsized":"2024-04-25T10:01:08.659826+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.655871+0000","last_change":"2024-04-25T10:01:07.616666+0000","last_active":"2024-04-25T10:01:08.655871+0000","last_peered":"2024-04-25T10:01:08.655871+0000","last_clean":"2024-04-25T10:01:08.655871+0000","last_became_active":"2024-04-25T10:01:07.616325+0000","last_became_peered":"2024-04-25T10:01:07.616325+0000","last_unstale":"2024-04-25T10:01:08.655871+0000","last_undegraded":"2024-04-25T10:01:08.655871+0000","last_fullsized":"2024-04-25T10:01:08.655871+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.961926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.645496+0000","last_change":"2024-04-25T10:01:08.644638+0000","last_active":"2024-04-25T10:01:08.645496+0000","last_peered":"2024-04-25T10:01:08.645496+0000","last_clean":"2024-04-25T10:01:08.645496+0000","last_became_active":"2024-04-25T10:01:08.644370+0000","last_became_peered":"2024-04-25T10:01:08.644370+0000","last_unstale":"2024-04-25T10:01:08.645496+0000","last_undegraded":"2024-04-25T10:01:08.645496+0000","last_fullsized":"2024-04-25T10:01:08.645496+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:21.459828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658975+0000","last_change":"2024-04-25T10:01:07.627666+0000","last_active":"2024-04-25T10:01:08.658975+0000","last_peered":"2024-04-25T10:01:08.658975+0000","last_clean":"2024-04-25T10:01:08.658975+0000","last_became_active":"2024-04-25T10:01:07.627311+0000","last_became_peered":"2024-04-25T10:01:07.627311+0000","last_unstale":"2024-04-25T10:01:08.658975+0000","last_undegraded":"2024-04-25T10:01:08.658975+0000","last_fullsized":"2024-04-25T10:01:08.658975+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:19.078513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966792+0000","last_change":"2024-04-25T10:01:08.654445+0000","last_active":"2024-04-25T10:01:09.966792+0000","last_peered":"2024-04-25T10:01:09.966792+0000","last_clean":"2024-04-25T10:01:09.966792+0000","last_became_active":"2024-04-25T10:01:08.653208+0000","last_became_peered":"2024-04-25T10:01:08.653208+0000","last_unstale":"2024-04-25T10:01:09.966792+0000","last_undegraded":"2024-04-25T10:01:09.966792+0000","last_fullsized":"2024-04-25T10:01:09.966792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:36.551347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656349+0000","last_change":"2024-04-25T10:01:07.622473+0000","last_active":"2024-04-25T10:01:08.656349+0000","last_peered":"2024-04-25T10:01:08.656349+0000","last_clean":"2024-04-25T10:01:08.656349+0000","last_became_active":"2024-04-25T10:01:07.621618+0000","last_became_peered":"2024-04-25T10:01:07.621618+0000","last_unstale":"2024-04-25T10:01:08.656349+0000","last_undegraded":"2024-04-25T10:01:08.656349+0000","last_fullsized":"2024-04-25T10:01:08.656349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:19.475669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.657672+0000","last_change":"2024-04-25T10:01:08.656318+0000","last_active":"2024-04-25T10:01:08.657672+0000","last_peered":"2024-04-25T10:01:08.657672+0000","last_clean":"2024-04-25T10:01:08.657672+0000","last_became_active":"2024-04-25T10:01:08.653440+0000","last_became_peered":"2024-04-25T10:01:08.653440+0000","last_unstale":"2024-04-25T10:01:08.657672+0000","last_undegraded":"2024-04-25T10:01:08.657672+0000","last_fullsized":"2024-04-25T10:01:08.657672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:42.660604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642189+0000","last_change":"2024-04-25T10:01:07.624656+0000","last_active":"2024-04-25T10:01:09.642189+0000","last_peered":"2024-04-25T10:01:09.642189+0000","last_clean":"2024-04-25T10:01:09.642189+0000","last_became_active":"2024-04-25T10:01:07.624326+0000","last_became_peered":"2024-04-25T10:01:07.624326+0000","last_unstale":"2024-04-25T10:01:09.642189+0000","last_undegraded":"2024-04-25T10:01:09.642189+0000","last_fullsized":"2024-04-25T10:01:09.642189+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:26.201576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.649824+0000","last_change":"2024-04-25T10:01:08.648418+0000","last_active":"2024-04-25T10:01:08.649824+0000","last_peered":"2024-04-25T10:01:08.649824+0000","last_clean":"2024-04-25T10:01:08.649824+0000","last_became_active":"2024-04-25T10:01:08.648164+0000","last_became_peered":"2024-04-25T10:01:08.648164+0000","last_unstale":"2024-04-25T10:01:08.649824+0000","last_undegraded":"2024-04-25T10:01:08.649824+0000","last_fullsized":"2024-04-25T10:01:08.649824+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.637179+0000","last_change":"2024-04-25T10:01:07.621178+0000","last_active":"2024-04-25T10:01:08.637179+0000","last_peered":"2024-04-25T10:01:08.637179+0000","last_clean":"2024-04-25T10:01:08.637179+0000","last_became_active":"2024-04-25T10:01:07.620918+0000","last_became_peered":"2024-04-25T10:01:07.620918+0000","last_unstale":"2024-04-25T10:01:08.637179+0000","last_undegraded":"2024-04-25T10:01:08.637179+0000","last_fullsized":"2024-04-25T10:01:08.637179+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966683+0000","last_change":"2024-04-25T10:01:08.660501+0000","last_active":"2024-04-25T10:01:09.966683+0000","last_peered":"2024-04-25T10:01:09.966683+0000","last_clean":"2024-04-25T10:01:09.966683+0000","last_became_active":"2024-04-25T10:01:08.660237+0000","last_became_peered":"2024-04-25T10:01:08.660237+0000","last_unstale":"2024-04-25T10:01:09.966683+0000","last_undegraded":"2024-04-25T10:01:09.966683+0000","last_fullsized":"2024-04-25T10:01:09.966683+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:41.524775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642674+0000","last_change":"2024-04-25T10:01:07.624534+0000","last_active":"2024-04-25T10:01:09.642674+0000","last_peered":"2024-04-25T10:01:09.642674+0000","last_clean":"2024-04-25T10:01:09.642674+0000","last_became_active":"2024-04-25T10:01:07.624256+0000","last_became_peered":"2024-04-25T10:01:07.624256+0000","last_unstale":"2024-04-25T10:01:09.642674+0000","last_undegraded":"2024-04-25T10:01:09.642674+0000","last_fullsized":"2024-04-25T10:01:09.642674+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:41.658543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651659+0000","last_change":"2024-04-25T10:01:08.649945+0000","last_active":"2024-04-25T10:01:08.651659+0000","last_peered":"2024-04-25T10:01:08.651659+0000","last_clean":"2024-04-25T10:01:08.651659+0000","last_became_active":"2024-04-25T10:01:08.649629+0000","last_became_peered":"2024-04-25T10:01:08.649629+0000","last_unstale":"2024-04-25T10:01:08.651659+0000","last_undegraded":"2024-04-25T10:01:08.651659+0000","last_fullsized":"2024-04-25T10:01:08.651659+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:29.791247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966456+0000","last_change":"2024-04-25T10:01:07.621825+0000","last_active":"2024-04-25T10:01:09.966456+0000","last_peered":"2024-04-25T10:01:09.966456+0000","last_clean":"2024-04-25T10:01:09.966456+0000","last_became_active":"2024-04-25T10:01:07.621570+0000","last_became_peered":"2024-04-25T10:01:07.621570+0000","last_unstale":"2024-04-25T10:01:09.966456+0000","last_undegraded":"2024-04-25T10:01:09.966456+0000","last_fullsized":"2024-04-25T10:01:09.966456+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639360+0000","last_change":"2024-04-25T10:01:08.651676+0000","last_active":"2024-04-25T10:01:09.639360+0000","last_peered":"2024-04-25T10:01:09.639360+0000","last_clean":"2024-04-25T10:01:09.639360+0000","last_became_active":"2024-04-25T10:01:08.651351+0000","last_became_peered":"2024-04-25T10:01:08.651351+0000","last_unstale":"2024-04-25T10:01:09.639360+0000","last_undegraded":"2024-04-25T10:01:09.639360+0000","last_fullsized":"2024-04-25T10:01:09.639360+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:12.709360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658123+0000","last_change":"2024-04-25T10:01:07.621881+0000","last_active":"2024-04-25T10:01:08.658123+0000","last_peered":"2024-04-25T10:01:08.658123+0000","last_clean":"2024-04-25T10:01:08.658123+0000","last_became_active":"2024-04-25T10:01:07.621635+0000","last_became_peered":"2024-04-25T10:01:07.621635+0000","last_unstale":"2024-04-25T10:01:08.658123+0000","last_undegraded":"2024-04-25T10:01:08.658123+0000","last_fullsized":"2024-04-25T10:01:08.658123+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:20.457767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656725+0000","last_change":"2024-04-25T10:01:08.653783+0000","last_active":"2024-04-25T10:01:08.656725+0000","last_peered":"2024-04-25T10:01:08.656725+0000","last_clean":"2024-04-25T10:01:08.656725+0000","last_became_active":"2024-04-25T10:01:08.653445+0000","last_became_peered":"2024-04-25T10:01:08.653445+0000","last_unstale":"2024-04-25T10:01:08.656725+0000","last_undegraded":"2024-04-25T10:01:08.656725+0000","last_fullsized":"2024-04-25T10:01:08.656725+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:29.785417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.662151+0000","last_change":"2024-04-25T10:01:07.625818+0000","last_active":"2024-04-25T10:01:08.662151+0000","last_peered":"2024-04-25T10:01:08.662151+0000","last_clean":"2024-04-25T10:01:08.662151+0000","last_became_active":"2024-04-25T10:01:07.625208+0000","last_became_peered":"2024-04-25T10:01:07.625208+0000","last_unstale":"2024-04-25T10:01:08.662151+0000","last_undegraded":"2024-04-25T10:01:08.662151+0000","last_fullsized":"2024-04-25T10:01:08.662151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:41.642231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.653854+0000","last_change":"2024-04-25T10:01:08.649337+0000","last_active":"2024-04-25T10:01:08.653854+0000","last_peered":"2024-04-25T10:01:08.653854+0000","last_clean":"2024-04-25T10:01:08.653854+0000","last_became_active":"2024-04-25T10:01:08.649008+0000","last_became_peered":"2024-04-25T10:01:08.649008+0000","last_unstale":"2024-04-25T10:01:08.653854+0000","last_undegraded":"2024-04-25T10:01:08.653854+0000","last_fullsized":"2024-04-25T10:01:08.653854+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659145+0000","last_change":"2024-04-25T10:01:07.628914+0000","last_active":"2024-04-25T10:01:08.659145+0000","last_peered":"2024-04-25T10:01:08.659145+0000","last_clean":"2024-04-25T10:01:08.659145+0000","last_became_active":"2024-04-25T10:01:07.628154+0000","last_became_peered":"2024-04-25T10:01:07.628154+0000","last_unstale":"2024-04-25T10:01:08.659145+0000","last_undegraded":"2024-04-25T10:01:08.659145+0000","last_fullsized":"2024-04-25T10:01:08.659145+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:46.534428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.650363+0000","last_change":"2024-04-25T10:01:08.649599+0000","last_active":"2024-04-25T10:01:08.650363+0000","last_peered":"2024-04-25T10:01:08.650363+0000","last_clean":"2024-04-25T10:01:08.650363+0000","last_became_active":"2024-04-25T10:01:08.649336+0000","last_became_peered":"2024-04-25T10:01:08.649336+0000","last_unstale":"2024-04-25T10:01:08.650363+0000","last_undegraded":"2024-04-25T10:01:08.650363+0000","last_fullsized":"2024-04-25T10:01:08.650363+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:29.791247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.637485+0000","last_change":"2024-04-25T10:01:07.616480+0000","last_active":"2024-04-25T10:01:08.637485+0000","last_peered":"2024-04-25T10:01:08.637485+0000","last_clean":"2024-04-25T10:01:08.637485+0000","last_became_active":"2024-04-25T10:01:07.616218+0000","last_became_peered":"2024-04-25T10:01:07.616218+0000","last_unstale":"2024-04-25T10:01:08.637485+0000","last_undegraded":"2024-04-25T10:01:08.637485+0000","last_fullsized":"2024-04-25T10:01:08.637485+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.342879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639474+0000","last_change":"2024-04-25T10:01:08.641341+0000","last_active":"2024-04-25T10:01:09.639474+0000","last_peered":"2024-04-25T10:01:09.639474+0000","last_clean":"2024-04-25T10:01:09.639474+0000","last_became_active":"2024-04-25T10:01:08.640054+0000","last_became_peered":"2024-04-25T10:01:08.640054+0000","last_unstale":"2024-04-25T10:01:09.639474+0000","last_undegraded":"2024-04-25T10:01:09.639474+0000","last_fullsized":"2024-04-25T10:01:09.639474+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:46.236149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639206+0000","last_change":"2024-04-25T10:01:07.619939+0000","last_active":"2024-04-25T10:01:09.639206+0000","last_peered":"2024-04-25T10:01:09.639206+0000","last_clean":"2024-04-25T10:01:09.639206+0000","last_became_active":"2024-04-25T10:01:07.619608+0000","last_became_peered":"2024-04-25T10:01:07.619608+0000","last_unstale":"2024-04-25T10:01:09.639206+0000","last_undegraded":"2024-04-25T10:01:09.639206+0000","last_fullsized":"2024-04-25T10:01:09.639206+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.606047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966793+0000","last_change":"2024-04-25T10:01:08.657650+0000","last_active":"2024-04-25T10:01:09.966793+0000","last_peered":"2024-04-25T10:01:09.966793+0000","last_clean":"2024-04-25T10:01:09.966793+0000","last_became_active":"2024-04-25T10:01:08.657355+0000","last_became_peered":"2024-04-25T10:01:08.657355+0000","last_unstale":"2024-04-25T10:01:09.966793+0000","last_undegraded":"2024-04-25T10:01:09.966793+0000","last_fullsized":"2024-04-25T10:01:09.966793+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:03.837528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642167+0000","last_change":"2024-04-25T10:01:07.618962+0000","last_active":"2024-04-25T10:01:09.642167+0000","last_peered":"2024-04-25T10:01:09.642167+0000","last_clean":"2024-04-25T10:01:09.642167+0000","last_became_active":"2024-04-25T10:01:07.618628+0000","last_became_peered":"2024-04-25T10:01:07.618628+0000","last_unstale":"2024-04-25T10:01:09.642167+0000","last_undegraded":"2024-04-25T10:01:09.642167+0000","last_fullsized":"2024-04-25T10:01:09.642167+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:09.048291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639519+0000","last_change":"2024-04-25T10:01:08.645514+0000","last_active":"2024-04-25T10:01:09.639519+0000","last_peered":"2024-04-25T10:01:09.639519+0000","last_clean":"2024-04-25T10:01:09.639519+0000","last_became_active":"2024-04-25T10:01:08.645251+0000","last_became_peered":"2024-04-25T10:01:08.645251+0000","last_unstale":"2024-04-25T10:01:09.639519+0000","last_undegraded":"2024-04-25T10:01:09.639519+0000","last_fullsized":"2024-04-25T10:01:09.639519+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:29.290614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658902+0000","last_change":"2024-04-25T10:01:07.626839+0000","last_active":"2024-04-25T10:01:08.658902+0000","last_peered":"2024-04-25T10:01:08.658902+0000","last_clean":"2024-04-25T10:01:08.658902+0000","last_became_active":"2024-04-25T10:01:07.626229+0000","last_became_peered":"2024-04-25T10:01:07.626229+0000","last_unstale":"2024-04-25T10:01:08.658902+0000","last_undegraded":"2024-04-25T10:01:08.658902+0000","last_fullsized":"2024-04-25T10:01:08.658902+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.174575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.650175+0000","last_change":"2024-04-25T10:01:08.649611+0000","last_active":"2024-04-25T10:01:08.650175+0000","last_peered":"2024-04-25T10:01:08.650175+0000","last_clean":"2024-04-25T10:01:08.650175+0000","last_became_active":"2024-04-25T10:01:08.649321+0000","last_became_peered":"2024-04-25T10:01:08.649321+0000","last_unstale":"2024-04-25T10:01:08.650175+0000","last_undegraded":"2024-04-25T10:01:08.650175+0000","last_fullsized":"2024-04-25T10:01:08.650175+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:04.944422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651626+0000","last_change":"2024-04-25T10:01:07.621132+0000","last_active":"2024-04-25T10:01:08.651626+0000","last_peered":"2024-04-25T10:01:08.651626+0000","last_clean":"2024-04-25T10:01:08.651626+0000","last_became_active":"2024-04-25T10:01:07.620856+0000","last_became_peered":"2024-04-25T10:01:07.620856+0000","last_unstale":"2024-04-25T10:01:08.651626+0000","last_undegraded":"2024-04-25T10:01:08.651626+0000","last_fullsized":"2024-04-25T10:01:08.651626+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:08.252856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.644731+0000","last_change":"2024-04-25T10:01:08.643843+0000","last_active":"2024-04-25T10:01:08.644731+0000","last_peered":"2024-04-25T10:01:08.644731+0000","last_clean":"2024-04-25T10:01:08.644731+0000","last_became_active":"2024-04-25T10:01:08.643309+0000","last_became_peered":"2024-04-25T10:01:08.643309+0000","last_unstale":"2024-04-25T10:01:08.644731+0000","last_undegraded":"2024-04-25T10:01:08.644731+0000","last_fullsized":"2024-04-25T10:01:08.644731+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.645281+0000","last_change":"2024-04-25T10:01:07.621971+0000","last_active":"2024-04-25T10:01:08.645281+0000","last_peered":"2024-04-25T10:01:08.645281+0000","last_clean":"2024-04-25T10:01:08.645281+0000","last_became_active":"2024-04-25T10:01:07.621650+0000","last_became_peered":"2024-04-25T10:01:07.621650+0000","last_unstale":"2024-04-25T10:01:08.645281+0000","last_undegraded":"2024-04-25T10:01:08.645281+0000","last_fullsized":"2024-04-25T10:01:08.645281+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966513+0000","last_change":"2024-04-25T10:01:08.642726+0000","last_active":"2024-04-25T10:01:09.966513+0000","last_peered":"2024-04-25T10:01:09.966513+0000","last_clean":"2024-04-25T10:01:09.966513+0000","last_became_active":"2024-04-25T10:01:08.642428+0000","last_became_peered":"2024-04-25T10:01:08.642428+0000","last_unstale":"2024-04-25T10:01:09.966513+0000","last_undegraded":"2024-04-25T10:01:09.966513+0000","last_fullsized":"2024-04-25T10:01:09.966513+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:32.717258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639659+0000","last_change":"2024-04-25T10:01:07.629013+0000","last_active":"2024-04-25T10:01:09.639659+0000","last_peered":"2024-04-25T10:01:09.639659+0000","last_clean":"2024-04-25T10:01:09.639659+0000","last_became_active":"2024-04-25T10:01:07.628383+0000","last_became_peered":"2024-04-25T10:01:07.628383+0000","last_unstale":"2024-04-25T10:01:09.639659+0000","last_undegraded":"2024-04-25T10:01:09.639659+0000","last_fullsized":"2024-04-25T10:01:09.639659+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:48.862207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966715+0000","last_change":"2024-04-25T10:01:08.656442+0000","last_active":"2024-04-25T10:01:09.966715+0000","last_peered":"2024-04-25T10:01:09.966715+0000","last_clean":"2024-04-25T10:01:09.966715+0000","last_became_active":"2024-04-25T10:01:08.654125+0000","last_became_peered":"2024-04-25T10:01:08.654125+0000","last_unstale":"2024-04-25T10:01:09.966715+0000","last_undegraded":"2024-04-25T10:01:09.966715+0000","last_fullsized":"2024-04-25T10:01:09.966715+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:12.719132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639163+0000","last_change":"2024-04-25T10:01:07.621828+0000","last_active":"2024-04-25T10:01:09.639163+0000","last_peered":"2024-04-25T10:01:09.639163+0000","last_clean":"2024-04-25T10:01:09.639163+0000","last_became_active":"2024-04-25T10:01:07.621541+0000","last_became_peered":"2024-04-25T10:01:07.621541+0000","last_unstale":"2024-04-25T10:01:09.639163+0000","last_undegraded":"2024-04-25T10:01:09.639163+0000","last_fullsized":"2024-04-25T10:01:09.639163+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:49.010362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659108+0000","last_change":"2024-04-25T10:01:08.657148+0000","last_active":"2024-04-25T10:01:08.659108+0000","last_peered":"2024-04-25T10:01:08.659108+0000","last_clean":"2024-04-25T10:01:08.659108+0000","last_became_active":"2024-04-25T10:01:08.656868+0000","last_became_peered":"2024-04-25T10:01:08.656868+0000","last_unstale":"2024-04-25T10:01:08.659108+0000","last_undegraded":"2024-04-25T10:01:08.659108+0000","last_fullsized":"2024-04-25T10:01:08.659108+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:18.169599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.643331+0000","last_change":"2024-04-25T10:01:07.623823+0000","last_active":"2024-04-25T10:01:08.643331+0000","last_peered":"2024-04-25T10:01:08.643331+0000","last_clean":"2024-04-25T10:01:08.643331+0000","last_became_active":"2024-04-25T10:01:07.623540+0000","last_became_peered":"2024-04-25T10:01:07.623540+0000","last_unstale":"2024-04-25T10:01:08.643331+0000","last_undegraded":"2024-04-25T10:01:08.643331+0000","last_fullsized":"2024-04-25T10:01:08.643331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:18.266105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639431+0000","last_change":"2024-04-25T10:01:08.642443+0000","last_active":"2024-04-25T10:01:09.639431+0000","last_peered":"2024-04-25T10:01:09.639431+0000","last_clean":"2024-04-25T10:01:09.639431+0000","last_became_active":"2024-04-25T10:01:08.642173+0000","last_became_peered":"2024-04-25T10:01:08.642173+0000","last_unstale":"2024-04-25T10:01:09.639431+0000","last_undegraded":"2024-04-25T10:01:09.639431+0000","last_fullsized":"2024-04-25T10:01:09.639431+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:03.657153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639358+0000","last_change":"2024-04-25T10:01:07.622643+0000","last_active":"2024-04-25T10:01:09.639358+0000","last_peered":"2024-04-25T10:01:09.639358+0000","last_clean":"2024-04-25T10:01:09.639358+0000","last_became_active":"2024-04-25T10:01:07.622210+0000","last_became_peered":"2024-04-25T10:01:07.622210+0000","last_unstale":"2024-04-25T10:01:09.639358+0000","last_undegraded":"2024-04-25T10:01:09.639358+0000","last_fullsized":"2024-04-25T10:01:09.639358+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.805840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966772+0000","last_change":"2024-04-25T10:01:08.647311+0000","last_active":"2024-04-25T10:01:09.966772+0000","last_peered":"2024-04-25T10:01:09.966772+0000","last_clean":"2024-04-25T10:01:09.966772+0000","last_became_active":"2024-04-25T10:01:08.646589+0000","last_became_peered":"2024-04-25T10:01:08.646589+0000","last_unstale":"2024-04-25T10:01:09.966772+0000","last_undegraded":"2024-04-25T10:01:09.966772+0000","last_fullsized":"2024-04-25T10:01:09.966772+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:57.642202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656404+0000","last_change":"2024-04-25T10:01:07.628529+0000","last_active":"2024-04-25T10:01:08.656404+0000","last_peered":"2024-04-25T10:01:08.656404+0000","last_clean":"2024-04-25T10:01:08.656404+0000","last_became_active":"2024-04-25T10:01:07.628185+0000","last_became_peered":"2024-04-25T10:01:07.628185+0000","last_unstale":"2024-04-25T10:01:08.656404+0000","last_undegraded":"2024-04-25T10:01:08.656404+0000","last_fullsized":"2024-04-25T10:01:08.656404+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:41.642231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642227+0000","last_change":"2024-04-25T10:01:08.643343+0000","last_active":"2024-04-25T10:01:09.642227+0000","last_peered":"2024-04-25T10:01:09.642227+0000","last_clean":"2024-04-25T10:01:09.642227+0000","last_became_active":"2024-04-25T10:01:08.643066+0000","last_became_peered":"2024-04-25T10:01:08.643066+0000","last_unstale":"2024-04-25T10:01:09.642227+0000","last_undegraded":"2024-04-25T10:01:09.642227+0000","last_fullsized":"2024-04-25T10:01:09.642227+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.080575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966505+0000","last_change":"2024-04-25T10:01:07.626502+0000","last_active":"2024-04-25T10:01:09.966505+0000","last_peered":"2024-04-25T10:01:09.966505+0000","last_clean":"2024-04-25T10:01:09.966505+0000","last_became_active":"2024-04-25T10:01:07.626187+0000","last_became_peered":"2024-04-25T10:01:07.626187+0000","last_unstale":"2024-04-25T10:01:09.966505+0000","last_undegraded":"2024-04-25T10:01:09.966505+0000","last_fullsized":"2024-04-25T10:01:09.966505+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:42.812396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659660+0000","last_change":"2024-04-25T10:01:08.656378+0000","last_active":"2024-04-25T10:01:08.659660+0000","last_peered":"2024-04-25T10:01:08.659660+0000","last_clean":"2024-04-25T10:01:08.659660+0000","last_became_active":"2024-04-25T10:01:08.656090+0000","last_became_peered":"2024-04-25T10:01:08.656090+0000","last_unstale":"2024-04-25T10:01:08.659660+0000","last_undegraded":"2024-04-25T10:01:08.659660+0000","last_fullsized":"2024-04-25T10:01:08.659660+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.639731+0000","last_change":"2024-04-25T10:01:07.623261+0000","last_active":"2024-04-25T10:01:08.639731+0000","last_peered":"2024-04-25T10:01:08.639731+0000","last_clean":"2024-04-25T10:01:08.639731+0000","last_became_active":"2024-04-25T10:01:07.622935+0000","last_became_peered":"2024-04-25T10:01:07.622935+0000","last_unstale":"2024-04-25T10:01:08.639731+0000","last_undegraded":"2024-04-25T10:01:08.639731+0000","last_fullsized":"2024-04-25T10:01:08.639731+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.961926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656011+0000","last_change":"2024-04-25T10:01:08.653766+0000","last_active":"2024-04-25T10:01:08.656011+0000","last_peered":"2024-04-25T10:01:08.656011+0000","last_clean":"2024-04-25T10:01:08.656011+0000","last_became_active":"2024-04-25T10:01:08.653271+0000","last_became_peered":"2024-04-25T10:01:08.653271+0000","last_unstale":"2024-04-25T10:01:08.656011+0000","last_undegraded":"2024-04-25T10:01:08.656011+0000","last_fullsized":"2024-04-25T10:01:08.656011+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.641404+0000","last_change":"2024-04-25T10:01:07.623766+0000","last_active":"2024-04-25T10:01:08.641404+0000","last_peered":"2024-04-25T10:01:08.641404+0000","last_clean":"2024-04-25T10:01:08.641404+0000","last_became_active":"2024-04-25T10:01:07.623498+0000","last_became_peered":"2024-04-25T10:01:07.623498+0000","last_unstale":"2024-04-25T10:01:08.641404+0000","last_undegraded":"2024-04-25T10:01:08.641404+0000","last_fullsized":"2024-04-25T10:01:08.641404+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:18.266105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639273+0000","last_change":"2024-04-25T10:01:08.642642+0000","last_active":"2024-04-25T10:01:09.639273+0000","last_peered":"2024-04-25T10:01:09.639273+0000","last_clean":"2024-04-25T10:01:09.639273+0000","last_became_active":"2024-04-25T10:01:08.642385+0000","last_became_peered":"2024-04-25T10:01:08.642385+0000","last_unstale":"2024-04-25T10:01:09.639273+0000","last_undegraded":"2024-04-25T10:01:09.639273+0000","last_fullsized":"2024-04-25T10:01:09.639273+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:20.472767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642348+0000","last_change":"2024-04-25T10:01:07.627410+0000","last_active":"2024-04-25T10:01:09.642348+0000","last_peered":"2024-04-25T10:01:09.642348+0000","last_clean":"2024-04-25T10:01:09.642348+0000","last_became_active":"2024-04-25T10:01:07.627089+0000","last_became_peered":"2024-04-25T10:01:07.627089+0000","last_unstale":"2024-04-25T10:01:09.642348+0000","last_undegraded":"2024-04-25T10:01:09.642348+0000","last_fullsized":"2024-04-25T10:01:09.642348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:25.190361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639406+0000","last_change":"2024-04-25T10:01:08.642060+0000","last_active":"2024-04-25T10:01:09.639406+0000","last_peered":"2024-04-25T10:01:09.639406+0000","last_clean":"2024-04-25T10:01:09.639406+0000","last_became_active":"2024-04-25T10:01:08.641779+0000","last_became_peered":"2024-04-25T10:01:08.641779+0000","last_unstale":"2024-04-25T10:01:09.639406+0000","last_undegraded":"2024-04-25T10:01:09.639406+0000","last_fullsized":"2024-04-25T10:01:09.639406+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.135228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639464+0000","last_change":"2024-04-25T10:01:07.620335+0000","last_active":"2024-04-25T10:01:09.639464+0000","last_peered":"2024-04-25T10:01:09.639464+0000","last_clean":"2024-04-25T10:01:09.639464+0000","last_became_active":"2024-04-25T10:01:07.619641+0000","last_became_peered":"2024-04-25T10:01:07.619641+0000","last_unstale":"2024-04-25T10:01:09.639464+0000","last_undegraded":"2024-04-25T10:01:09.639464+0000","last_fullsized":"2024-04-25T10:01:09.639464+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:26.388795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651574+0000","last_change":"2024-04-25T10:01:08.648935+0000","last_active":"2024-04-25T10:01:08.651574+0000","last_peered":"2024-04-25T10:01:08.651574+0000","last_clean":"2024-04-25T10:01:08.651574+0000","last_became_active":"2024-04-25T10:01:08.648677+0000","last_became_peered":"2024-04-25T10:01:08.648677+0000","last_unstale":"2024-04-25T10:01:08.651574+0000","last_undegraded":"2024-04-25T10:01:08.651574+0000","last_fullsized":"2024-04-25T10:01:08.651574+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.643483+0000","last_change":"2024-04-25T10:01:07.626934+0000","last_active":"2024-04-25T10:01:08.643483+0000","last_peered":"2024-04-25T10:01:08.643483+0000","last_clean":"2024-04-25T10:01:08.643483+0000","last_became_active":"2024-04-25T10:01:07.626558+0000","last_became_peered":"2024-04-25T10:01:07.626558+0000","last_unstale":"2024-04-25T10:01:08.643483+0000","last_undegraded":"2024-04-25T10:01:08.643483+0000","last_fullsized":"2024-04-25T10:01:08.643483+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:03.973991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.654256+0000","last_change":"2024-04-25T10:01:08.647871+0000","last_active":"2024-04-25T10:01:08.654256+0000","last_peered":"2024-04-25T10:01:08.654256+0000","last_clean":"2024-04-25T10:01:08.654256+0000","last_became_active":"2024-04-25T10:01:08.646567+0000","last_became_peered":"2024-04-25T10:01:08.646567+0000","last_unstale":"2024-04-25T10:01:08.654256+0000","last_undegraded":"2024-04-25T10:01:08.654256+0000","last_fullsized":"2024-04-25T10:01:08.654256+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:21.459828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639454+0000","last_change":"2024-04-25T10:01:07.620838+0000","last_active":"2024-04-25T10:01:09.639454+0000","last_peered":"2024-04-25T10:01:09.639454+0000","last_clean":"2024-04-25T10:01:09.639454+0000","last_became_active":"2024-04-25T10:01:07.620560+0000","last_became_peered":"2024-04-25T10:01:07.620560+0000","last_unstale":"2024-04-25T10:01:09.639454+0000","last_undegraded":"2024-04-25T10:01:09.639454+0000","last_fullsized":"2024-04-25T10:01:09.639454+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:08.363320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651392+0000","last_change":"2024-04-25T10:01:08.647793+0000","last_active":"2024-04-25T10:01:08.651392+0000","last_peered":"2024-04-25T10:01:08.651392+0000","last_clean":"2024-04-25T10:01:08.651392+0000","last_became_active":"2024-04-25T10:01:08.647500+0000","last_became_peered":"2024-04-25T10:01:08.647500+0000","last_unstale":"2024-04-25T10:01:08.651392+0000","last_undegraded":"2024-04-25T10:01:08.651392+0000","last_fullsized":"2024-04-25T10:01:08.651392+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.080575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639438+0000","last_change":"2024-04-25T10:01:07.625148+0000","last_active":"2024-04-25T10:01:09.639438+0000","last_peered":"2024-04-25T10:01:09.639438+0000","last_clean":"2024-04-25T10:01:09.639438+0000","last_became_active":"2024-04-25T10:01:07.624773+0000","last_became_peered":"2024-04-25T10:01:07.624773+0000","last_unstale":"2024-04-25T10:01:09.639438+0000","last_undegraded":"2024-04-25T10:01:09.639438+0000","last_fullsized":"2024-04-25T10:01:09.639438+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:09.128646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":332,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.637490+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:08.637490+0000","last_peered":"2024-04-25T10:01:08.637490+0000","last_clean":"2024-04-25T10:01:08.637490+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:08.637490+0000","last_undegraded":"2024-04-25T10:01:08.637490+0000","last_fullsized":"2024-04-25T10:01:08.637490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41396,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330444,"statfs":{"total":96636764160,"available":96594374656,"internally_reserved":0,"allocated":1941504,"data_stored":1212272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46404,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325436,"statfs":{"total":96636764160,"available":96589246464,"internally_reserved":0,"allocated":1359872,"data_stored":623960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40808,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331032,"statfs":{"total":96636764160,"available":96594976768,"internally_reserved":0,"allocated":1339392,"data_stored":621744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1347584,"data_stored":621836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50548,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321292,"statfs":{"total":96636764160,"available":96585003008,"internally_reserved":0,"allocated":1343488,"data_stored":622326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46384,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325456,"statfs":{"total":96636764160,"available":96589266944,"internally_reserved":0,"allocated":1343488,"data_stored":621778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46380,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325460,"statfs":{"total":96636764160,"available":96589271040,"internally_reserved":0,"allocated":1343488,"data_stored":621778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46976,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324864,"statfs":{"total":96636764160,"available":96588660736,"internally_reserved":0,"allocated":1949696,"data_stored":1213814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:11.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:11.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 Processor -- start 2024-04-25T10:01:11.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 -- start start 2024-04-25T10:01:11.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d1806f360 0x7f3d180725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3d180aefa0 0x7f3d1806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d1815be50 con 0x7f3d1815ba70 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d180717f0 con 0x7f3d1806f360 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.855+0000 7f3d1eabc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d18071ad0 con 0x7f3d180aefa0 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d1806f360 0x7f3d180725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3d1806f360 0x7f3d180725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59860/0 (socket says 172.21.15.148:59860) 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 -- 172.21.15.148:0/4133666613 learned_addr learned my addr 172.21.15.148:0/4133666613 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d1d059640 1 --2- 172.21.15.148:0/4133666613 >> v2:172.21.15.188:3300/0 conn(0x7f3d180aefa0 0x7f3d1806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 -- 172.21.15.148:0/4133666613 >> v2:172.21.15.188:3300/0 conn(0x7f3d180aefa0 msgr2=0x7f3d1806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 --2- 172.21.15.148:0/4133666613 >> v2:172.21.15.188:3300/0 conn(0x7f3d180aefa0 0x7f3d1806d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 -- 172.21.15.148:0/4133666613 >> v1:172.21.15.188:6789/0 conn(0x7f3d1815ba70 legacy=0x7f3d180cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.856+0000 7f3d17fff640 1 -- 172.21.15.148:0/4133666613 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d18071db0 con 0x7f3d1806f360 2024-04-25T10:01:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d17fff640 1 --2- 172.21.15.148:0/4133666613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d180725b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3d18070f80 tx=0x7f3d0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a46caea52676db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d177fe640 1 -- 172.21.15.148:0/4133666613 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08037070 con 0x7f3d1806f360 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d177fe640 1 -- 172.21.15.148:0/4133666613 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08032440 con 0x7f3d1806f360 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d177fe640 1 -- 172.21.15.148:0/4133666613 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08032810 con 0x7f3d1806f360 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 -- 172.21.15.148:0/4133666613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 msgr2=0x7f3d180725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/4133666613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d180725b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3d18070f80 tx=0x7f3d0802a270 comp rx=0 tx=0).stop 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 -- 172.21.15.148:0/4133666613 shutdown_connections 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/4133666613 >> v2:172.21.15.188:3300/0 conn(0x7f3d180aefa0 0x7f3d1806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/4133666613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d180725b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 -- 172.21.15.148:0/4133666613 >> 172.21.15.148:0/4133666613 conn(0x7f3d180cfae0 msgr2=0x7f3d180cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 -- 172.21.15.148:0/4133666613 shutdown_connections 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.857+0000 7f3d1eabc640 1 -- 172.21.15.148:0/4133666613 wait complete. 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.858+0000 7f3d1eabc640 1 Processor -- start 2024-04-25T10:01:11.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 -- start start 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d1815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d180aefa0 0x7f3d1814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1815ba70 0x7f3d18153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180cd420 con 0x7f3d1806f360 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180cd7e0 con 0x7f3d180aefa0 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1eabc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180cd5a0 con 0x7f3d1815ba70 2024-04-25T10:01:11.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.859+0000 7f3d1c858640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d1815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d1815c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59868/0 (socket says 172.21.15.148:59868) 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 -- 172.21.15.148:0/3641919161 learned_addr learned my addr 172.21.15.148:0/3641919161 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d17fff640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d180aefa0 0x7f3d1814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1d059640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1815ba70 0x7f3d18153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 -- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1815ba70 msgr2=0x7f3d18153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1815ba70 0x7f3d18153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 -- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d180aefa0 msgr2=0x7f3d1814d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d180aefa0 0x7f3d1814d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 -- 172.21.15.148:0/3641919161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d18154460 con 0x7f3d1806f360 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d17fff640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d180aefa0 0x7f3d1814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1c858640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d1815c030 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c790 tx=0x7f3d1000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10016020 con 0x7f3d1806f360 2024-04-25T10:01:11.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d10006d50 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d181f5000 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10003bb0 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.860+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d181f55c0 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.861+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d180728b0 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.862+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d1000a850 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.862+0000 7f3d157fa640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cf806b500 0x7f3cf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.862+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d1008a030 con 0x7f3d1806f360 2024-04-25T10:01:11.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.862+0000 7f3d17fff640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cf806b500 0x7f3cf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.862+0000 7f3d17fff640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cf806b500 0x7f3cf806d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3d08034000 tx=0x7f3d0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:11.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:11.866+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d180728b0 con 0x7f3d1806f360 2024-04-25T10:01:12.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.034+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d18073990 con 0x7f3cf806b500 2024-04-25T10:01:12.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.040+0000 7f3d157fa640 1 -- 172.21.15.148:0/3641919161 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252450 (secure 0 0 0) 0x7f3d18073990 con 0x7f3cf806b500 2024-04-25T10:01:12.047 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:12.049 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:01:12.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.042+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cf806b500 msgr2=0x7f3cf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.042+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cf806b500 0x7f3cf806d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3d08034000 tx=0x7f3d0802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:12.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 msgr2=0x7f3d1815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d1815c030 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000c790 tx=0x7f3d1000cc50 comp rx=0 tx=0).stop 2024-04-25T10:01:12.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 shutdown_connections 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3cf806b500 0x7f3cf806d9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d1815ba70 0x7f3d18153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d180aefa0 0x7f3d1814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 --2- 172.21.15.148:0/3641919161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d1806f360 0x7f3d1815c030 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 >> 172.21.15.148:0/3641919161 conn(0x7f3d180cfae0 msgr2=0x7f3d180afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 shutdown_connections 2024-04-25T10:01:12.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.043+0000 7f3d1eabc640 1 -- 172.21.15.148:0/3641919161 wait complete. 2024-04-25T10:01:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-04-25T10:01:11.104108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":369448,"kb_used_data":11688,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754605272,"statfs":{"total":773094113280,"available":772715798528,"internally_reserved":0,"allocated":11968512,"data_stored":6159508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":366202416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.500230"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966648+0000","last_change":"2024-04-25T10:01:08.647566+0000","last_active":"2024-04-25T10:01:09.966648+0000","last_peered":"2024-04-25T10:01:09.966648+0000","last_clean":"2024-04-25T10:01:09.966648+0000","last_became_active":"2024-04-25T10:01:08.646882+0000","last_became_peered":"2024-04-25T10:01:08.646882+0000","last_unstale":"2024-04-25T10:01:09.966648+0000","last_undegraded":"2024-04-25T10:01:09.966648+0000","last_fullsized":"2024-04-25T10:01:09.966648+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:24.047699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660615+0000","last_change":"2024-04-25T10:01:08.659726+0000","last_active":"2024-04-25T10:01:08.660615+0000","last_peered":"2024-04-25T10:01:08.660615+0000","last_clean":"2024-04-25T10:01:08.660615+0000","last_became_active":"2024-04-25T10:01:08.659434+0000","last_became_peered":"2024-04-25T10:01:08.659434+0000","last_unstale":"2024-04-25T10:01:08.660615+0000","last_undegraded":"2024-04-25T10:01:08.660615+0000","last_fullsized":"2024-04-25T10:01:08.660615+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:06.104096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966569+0000","last_change":"2024-04-25T10:01:08.657288+0000","last_active":"2024-04-25T10:01:09.966569+0000","last_peered":"2024-04-25T10:01:09.966569+0000","last_clean":"2024-04-25T10:01:09.966569+0000","last_became_active":"2024-04-25T10:01:08.657027+0000","last_became_peered":"2024-04-25T10:01:08.657027+0000","last_unstale":"2024-04-25T10:01:09.966569+0000","last_undegraded":"2024-04-25T10:01:09.966569+0000","last_fullsized":"2024-04-25T10:01:09.966569+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:47.589375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.654696+0000","last_change":"2024-04-25T10:01:08.652623+0000","last_active":"2024-04-25T10:01:08.654696+0000","last_peered":"2024-04-25T10:01:08.654696+0000","last_clean":"2024-04-25T10:01:08.654696+0000","last_became_active":"2024-04-25T10:01:08.652327+0000","last_became_peered":"2024-04-25T10:01:08.652327+0000","last_unstale":"2024-04-25T10:01:08.654696+0000","last_undegraded":"2024-04-25T10:01:08.654696+0000","last_fullsized":"2024-04-25T10:01:08.654696+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658205+0000","last_change":"2024-04-25T10:01:08.655399+0000","last_active":"2024-04-25T10:01:08.658205+0000","last_peered":"2024-04-25T10:01:08.658205+0000","last_clean":"2024-04-25T10:01:08.658205+0000","last_became_active":"2024-04-25T10:01:08.655098+0000","last_became_peered":"2024-04-25T10:01:08.655098+0000","last_unstale":"2024-04-25T10:01:08.658205+0000","last_undegraded":"2024-04-25T10:01:08.658205+0000","last_fullsized":"2024-04-25T10:01:08.658205+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:42.660604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.657790+0000","last_change":"2024-04-25T10:01:08.656730+0000","last_active":"2024-04-25T10:01:08.657790+0000","last_peered":"2024-04-25T10:01:08.657790+0000","last_clean":"2024-04-25T10:01:08.657790+0000","last_became_active":"2024-04-25T10:01:08.654062+0000","last_became_peered":"2024-04-25T10:01:08.654062+0000","last_unstale":"2024-04-25T10:01:08.657790+0000","last_undegraded":"2024-04-25T10:01:08.657790+0000","last_fullsized":"2024-04-25T10:01:08.657790+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:10.050353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.661418+0000","last_change":"2024-04-25T10:01:08.660907+0000","last_active":"2024-04-25T10:01:08.661418+0000","last_peered":"2024-04-25T10:01:08.661418+0000","last_clean":"2024-04-25T10:01:08.661418+0000","last_became_active":"2024-04-25T10:01:08.660556+0000","last_became_peered":"2024-04-25T10:01:08.660556+0000","last_unstale":"2024-04-25T10:01:08.661418+0000","last_undegraded":"2024-04-25T10:01:08.661418+0000","last_fullsized":"2024-04-25T10:01:08.661418+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:47.879606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639182+0000","last_change":"2024-04-25T10:01:08.650932+0000","last_active":"2024-04-25T10:01:09.639182+0000","last_peered":"2024-04-25T10:01:09.639182+0000","last_clean":"2024-04-25T10:01:09.639182+0000","last_became_active":"2024-04-25T10:01:08.650667+0000","last_became_peered":"2024-04-25T10:01:08.650667+0000","last_unstale":"2024-04-25T10:01:09.639182+0000","last_undegraded":"2024-04-25T10:01:09.639182+0000","last_fullsized":"2024-04-25T10:01:09.639182+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:05.371195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660928+0000","last_change":"2024-04-25T10:01:08.660465+0000","last_active":"2024-04-25T10:01:08.660928+0000","last_peered":"2024-04-25T10:01:08.660928+0000","last_clean":"2024-04-25T10:01:08.660928+0000","last_became_active":"2024-04-25T10:01:08.660165+0000","last_became_peered":"2024-04-25T10:01:08.660165+0000","last_unstale":"2024-04-25T10:01:08.660928+0000","last_undegraded":"2024-04-25T10:01:08.660928+0000","last_fullsized":"2024-04-25T10:01:08.660928+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:05.836377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659903+0000","last_change":"2024-04-25T10:01:08.657254+0000","last_active":"2024-04-25T10:01:08.659903+0000","last_peered":"2024-04-25T10:01:08.659903+0000","last_clean":"2024-04-25T10:01:08.659903+0000","last_became_active":"2024-04-25T10:01:08.656957+0000","last_became_peered":"2024-04-25T10:01:08.656957+0000","last_unstale":"2024-04-25T10:01:08.659903+0000","last_undegraded":"2024-04-25T10:01:08.659903+0000","last_fullsized":"2024-04-25T10:01:08.659903+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:10.050353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651563+0000","last_change":"2024-04-25T10:01:08.649835+0000","last_active":"2024-04-25T10:01:08.651563+0000","last_peered":"2024-04-25T10:01:08.651563+0000","last_clean":"2024-04-25T10:01:08.651563+0000","last_became_active":"2024-04-25T10:01:08.649566+0000","last_became_peered":"2024-04-25T10:01:08.649566+0000","last_unstale":"2024-04-25T10:01:08.651563+0000","last_undegraded":"2024-04-25T10:01:08.651563+0000","last_fullsized":"2024-04-25T10:01:08.651563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642817+0000","last_change":"2024-04-25T10:01:08.639879+0000","last_active":"2024-04-25T10:01:09.642817+0000","last_peered":"2024-04-25T10:01:09.642817+0000","last_clean":"2024-04-25T10:01:09.642817+0000","last_became_active":"2024-04-25T10:01:08.639533+0000","last_became_peered":"2024-04-25T10:01:08.639533+0000","last_unstale":"2024-04-25T10:01:09.642817+0000","last_undegraded":"2024-04-25T10:01:09.642817+0000","last_fullsized":"2024-04-25T10:01:09.642817+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.655825+0000","last_change":"2024-04-25T10:01:08.652617+0000","last_active":"2024-04-25T10:01:08.655825+0000","last_peered":"2024-04-25T10:01:08.655825+0000","last_clean":"2024-04-25T10:01:08.655825+0000","last_became_active":"2024-04-25T10:01:08.652296+0000","last_became_peered":"2024-04-25T10:01:08.652296+0000","last_unstale":"2024-04-25T10:01:08.655825+0000","last_undegraded":"2024-04-25T10:01:08.655825+0000","last_fullsized":"2024-04-25T10:01:08.655825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.643276+0000","last_change":"2024-04-25T10:01:08.642861+0000","last_active":"2024-04-25T10:01:08.643276+0000","last_peered":"2024-04-25T10:01:08.643276+0000","last_clean":"2024-04-25T10:01:08.643276+0000","last_became_active":"2024-04-25T10:01:08.642564+0000","last_became_peered":"2024-04-25T10:01:08.642564+0000","last_unstale":"2024-04-25T10:01:08.643276+0000","last_undegraded":"2024-04-25T10:01:08.643276+0000","last_fullsized":"2024-04-25T10:01:08.643276+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651528+0000","last_change":"2024-04-25T10:01:08.649954+0000","last_active":"2024-04-25T10:01:08.651528+0000","last_peered":"2024-04-25T10:01:08.651528+0000","last_clean":"2024-04-25T10:01:08.651528+0000","last_became_active":"2024-04-25T10:01:08.649696+0000","last_became_peered":"2024-04-25T10:01:08.649696+0000","last_unstale":"2024-04-25T10:01:08.651528+0000","last_undegraded":"2024-04-25T10:01:08.651528+0000","last_fullsized":"2024-04-25T10:01:08.651528+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.652304+0000","last_change":"2024-04-25T10:01:08.651124+0000","last_active":"2024-04-25T10:01:08.652304+0000","last_peered":"2024-04-25T10:01:08.652304+0000","last_clean":"2024-04-25T10:01:08.652304+0000","last_became_active":"2024-04-25T10:01:08.650871+0000","last_became_peered":"2024-04-25T10:01:08.650871+0000","last_unstale":"2024-04-25T10:01:08.652304+0000","last_undegraded":"2024-04-25T10:01:08.652304+0000","last_fullsized":"2024-04-25T10:01:08.652304+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:18.169599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966690+0000","last_change":"2024-04-25T10:01:08.658019+0000","last_active":"2024-04-25T10:01:09.966690+0000","last_peered":"2024-04-25T10:01:09.966690+0000","last_clean":"2024-04-25T10:01:09.966690+0000","last_became_active":"2024-04-25T10:01:08.657652+0000","last_became_peered":"2024-04-25T10:01:08.657652+0000","last_unstale":"2024-04-25T10:01:09.966690+0000","last_undegraded":"2024-04-25T10:01:09.966690+0000","last_fullsized":"2024-04-25T10:01:09.966690+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:12.079829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660332+0000","last_change":"2024-04-25T10:01:08.659396+0000","last_active":"2024-04-25T10:01:08.660332+0000","last_peered":"2024-04-25T10:01:08.660332+0000","last_clean":"2024-04-25T10:01:08.660332+0000","last_became_active":"2024-04-25T10:01:08.659071+0000","last_became_peered":"2024-04-25T10:01:08.659071+0000","last_unstale":"2024-04-25T10:01:08.660332+0000","last_undegraded":"2024-04-25T10:01:08.660332+0000","last_fullsized":"2024-04-25T10:01:08.660332+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:26.192422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966624+0000","last_change":"2024-04-25T10:01:08.656704+0000","last_active":"2024-04-25T10:01:09.966624+0000","last_peered":"2024-04-25T10:01:09.966624+0000","last_clean":"2024-04-25T10:01:09.966624+0000","last_became_active":"2024-04-25T10:01:08.653888+0000","last_became_peered":"2024-04-25T10:01:08.653888+0000","last_unstale":"2024-04-25T10:01:09.966624+0000","last_undegraded":"2024-04-25T10:01:09.966624+0000","last_fullsized":"2024-04-25T10:01:09.966624+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:22.665485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642860+0000","last_change":"2024-04-25T10:01:08.641167+0000","last_active":"2024-04-25T10:01:09.642860+0000","last_peered":"2024-04-25T10:01:09.642860+0000","last_clean":"2024-04-25T10:01:09.642860+0000","last_became_active":"2024-04-25T10:01:08.640835+0000","last_became_peered":"2024-04-25T10:01:08.640835+0000","last_unstale":"2024-04-25T10:01:09.642860+0000","last_undegraded":"2024-04-25T10:01:09.642860+0000","last_fullsized":"2024-04-25T10:01:09.642860+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:18.169599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639608+0000","last_change":"2024-04-25T10:01:08.647220+0000","last_active":"2024-04-25T10:01:09.639608+0000","last_peered":"2024-04-25T10:01:09.639608+0000","last_clean":"2024-04-25T10:01:09.639608+0000","last_became_active":"2024-04-25T10:01:08.646913+0000","last_became_peered":"2024-04-25T10:01:08.646913+0000","last_unstale":"2024-04-25T10:01:09.639608+0000","last_undegraded":"2024-04-25T10:01:09.639608+0000","last_fullsized":"2024-04-25T10:01:09.639608+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:23.714488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651680+0000","last_change":"2024-04-25T10:01:08.650009+0000","last_active":"2024-04-25T10:01:08.651680+0000","last_peered":"2024-04-25T10:01:08.651680+0000","last_clean":"2024-04-25T10:01:08.651680+0000","last_became_active":"2024-04-25T10:01:08.649454+0000","last_became_peered":"2024-04-25T10:01:08.649454+0000","last_unstale":"2024-04-25T10:01:08.651680+0000","last_undegraded":"2024-04-25T10:01:08.651680+0000","last_fullsized":"2024-04-25T10:01:08.651680+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:04.944422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966513+0000","last_change":"2024-04-25T10:01:08.653957+0000","last_active":"2024-04-25T10:01:09.966513+0000","last_peered":"2024-04-25T10:01:09.966513+0000","last_clean":"2024-04-25T10:01:09.966513+0000","last_became_active":"2024-04-25T10:01:08.653160+0000","last_became_peered":"2024-04-25T10:01:08.653160+0000","last_unstale":"2024-04-25T10:01:09.966513+0000","last_undegraded":"2024-04-25T10:01:09.966513+0000","last_fullsized":"2024-04-25T10:01:09.966513+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:55.438978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.650623+0000","last_change":"2024-04-25T10:01:08.648591+0000","last_active":"2024-04-25T10:01:08.650623+0000","last_peered":"2024-04-25T10:01:08.650623+0000","last_clean":"2024-04-25T10:01:08.650623+0000","last_became_active":"2024-04-25T10:01:08.648271+0000","last_became_peered":"2024-04-25T10:01:08.648271+0000","last_unstale":"2024-04-25T10:01:08.650623+0000","last_undegraded":"2024-04-25T10:01:08.650623+0000","last_fullsized":"2024-04-25T10:01:08.650623+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:07.176637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642605+0000","last_change":"2024-04-25T10:01:08.651605+0000","last_active":"2024-04-25T10:01:09.642605+0000","last_peered":"2024-04-25T10:01:09.642605+0000","last_clean":"2024-04-25T10:01:09.642605+0000","last_became_active":"2024-04-25T10:01:08.651308+0000","last_became_peered":"2024-04-25T10:01:08.651308+0000","last_unstale":"2024-04-25T10:01:09.642605+0000","last_undegraded":"2024-04-25T10:01:09.642605+0000","last_fullsized":"2024-04-25T10:01:09.642605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:22.210656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642576+0000","last_change":"2024-04-25T10:01:08.654590+0000","last_active":"2024-04-25T10:01:09.642576+0000","last_peered":"2024-04-25T10:01:09.642576+0000","last_clean":"2024-04-25T10:01:09.642576+0000","last_became_active":"2024-04-25T10:01:08.654230+0000","last_became_peered":"2024-04-25T10:01:08.654230+0000","last_unstale":"2024-04-25T10:01:09.642576+0000","last_undegraded":"2024-04-25T10:01:09.642576+0000","last_fullsized":"2024-04-25T10:01:09.642576+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651469+0000","last_change":"2024-04-25T10:01:08.646886+0000","last_active":"2024-04-25T10:01:08.651469+0000","last_peered":"2024-04-25T10:01:08.651469+0000","last_clean":"2024-04-25T10:01:08.651469+0000","last_became_active":"2024-04-25T10:01:08.646205+0000","last_became_peered":"2024-04-25T10:01:08.646205+0000","last_unstale":"2024-04-25T10:01:08.651469+0000","last_undegraded":"2024-04-25T10:01:08.651469+0000","last_fullsized":"2024-04-25T10:01:08.651469+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:40.344941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.660320+0000","last_change":"2024-04-25T10:01:08.657289+0000","last_active":"2024-04-25T10:01:08.660320+0000","last_peered":"2024-04-25T10:01:08.660320+0000","last_clean":"2024-04-25T10:01:08.660320+0000","last_became_active":"2024-04-25T10:01:08.656965+0000","last_became_peered":"2024-04-25T10:01:08.656965+0000","last_unstale":"2024-04-25T10:01:08.660320+0000","last_undegraded":"2024-04-25T10:01:08.660320+0000","last_fullsized":"2024-04-25T10:01:08.660320+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:27.203638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639692+0000","last_change":"2024-04-25T10:01:08.647041+0000","last_active":"2024-04-25T10:01:09.639692+0000","last_peered":"2024-04-25T10:01:09.639692+0000","last_clean":"2024-04-25T10:01:09.639692+0000","last_became_active":"2024-04-25T10:01:08.646548+0000","last_became_peered":"2024-04-25T10:01:08.646548+0000","last_unstale":"2024-04-25T10:01:09.639692+0000","last_undegraded":"2024-04-25T10:01:09.639692+0000","last_fullsized":"2024-04-25T10:01:09.639692+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:14.963989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642545+0000","last_change":"2024-04-25T10:01:08.651842+0000","last_active":"2024-04-25T10:01:09.642545+0000","last_peered":"2024-04-25T10:01:09.642545+0000","last_clean":"2024-04-25T10:01:09.642545+0000","last_became_active":"2024-04-25T10:01:08.651513+0000","last_became_peered":"2024-04-25T10:01:08.651513+0000","last_unstale":"2024-04-25T10:01:09.642545+0000","last_undegraded":"2024-04-25T10:01:09.642545+0000","last_fullsized":"2024-04-25T10:01:09.642545+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:21.459828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.657563+0000","last_change":"2024-04-25T10:01:08.650755+0000","last_active":"2024-04-25T10:01:08.657563+0000","last_peered":"2024-04-25T10:01:08.657563+0000","last_clean":"2024-04-25T10:01:08.657563+0000","last_became_active":"2024-04-25T10:01:08.650259+0000","last_became_peered":"2024-04-25T10:01:08.650259+0000","last_unstale":"2024-04-25T10:01:08.657563+0000","last_undegraded":"2024-04-25T10:01:08.657563+0000","last_fullsized":"2024-04-25T10:01:08.657563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:29.785417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639258+0000","last_change":"2024-04-25T10:01:08.637412+0000","last_active":"2024-04-25T10:01:09.639258+0000","last_peered":"2024-04-25T10:01:09.639258+0000","last_clean":"2024-04-25T10:01:09.639258+0000","last_became_active":"2024-04-25T10:01:08.636910+0000","last_became_peered":"2024-04-25T10:01:08.636910+0000","last_unstale":"2024-04-25T10:01:09.639258+0000","last_undegraded":"2024-04-25T10:01:09.639258+0000","last_fullsized":"2024-04-25T10:01:09.639258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:54.744535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966807+0000","last_change":"2024-04-25T10:01:08.660401+0000","last_active":"2024-04-25T10:01:09.966807+0000","last_peered":"2024-04-25T10:01:09.966807+0000","last_clean":"2024-04-25T10:01:09.966807+0000","last_became_active":"2024-04-25T10:01:08.659797+0000","last_became_peered":"2024-04-25T10:01:08.659797+0000","last_unstale":"2024-04-25T10:01:09.966807+0000","last_undegraded":"2024-04-25T10:01:09.966807+0000","last_fullsized":"2024-04-25T10:01:09.966807+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:33.359857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"108'4","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659580+0000","last_change":"2024-04-25T10:01:07.627632+0000","last_active":"2024-04-25T10:01:08.659580+0000","last_peered":"2024-04-25T10:01:08.659580+0000","last_clean":"2024-04-25T10:01:08.659580+0000","last_became_active":"2024-04-25T10:01:07.627373+0000","last_became_peered":"2024-04-25T10:01:07.627373+0000","last_unstale":"2024-04-25T10:01:08.659580+0000","last_undegraded":"2024-04-25T10:01:08.659580+0000","last_fullsized":"2024-04-25T10:01:08.659580+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:19.078513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639382+0000","last_change":"2024-04-25T10:01:08.650263+0000","last_active":"2024-04-25T10:01:09.639382+0000","last_peered":"2024-04-25T10:01:09.639382+0000","last_clean":"2024-04-25T10:01:09.639382+0000","last_became_active":"2024-04-25T10:01:08.650000+0000","last_became_peered":"2024-04-25T10:01:08.650000+0000","last_unstale":"2024-04-25T10:01:09.639382+0000","last_undegraded":"2024-04-25T10:01:09.639382+0000","last_fullsized":"2024-04-25T10:01:09.639382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:05.021630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639536+0000","last_change":"2024-04-25T10:01:07.620530+0000","last_active":"2024-04-25T10:01:09.639536+0000","last_peered":"2024-04-25T10:01:09.639536+0000","last_clean":"2024-04-25T10:01:09.639536+0000","last_became_active":"2024-04-25T10:01:07.619062+0000","last_became_peered":"2024-04-25T10:01:07.619062+0000","last_unstale":"2024-04-25T10:01:09.639536+0000","last_undegraded":"2024-04-25T10:01:09.639536+0000","last_fullsized":"2024-04-25T10:01:09.639536+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:14.062695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642716+0000","last_change":"2024-04-25T10:01:08.655561+0000","last_active":"2024-04-25T10:01:09.642716+0000","last_peered":"2024-04-25T10:01:09.642716+0000","last_clean":"2024-04-25T10:01:09.642716+0000","last_became_active":"2024-04-25T10:01:08.652594+0000","last_became_peered":"2024-04-25T10:01:08.652594+0000","last_unstale":"2024-04-25T10:01:09.642716+0000","last_undegraded":"2024-04-25T10:01:09.642716+0000","last_fullsized":"2024-04-25T10:01:09.642716+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642503+0000","last_change":"2024-04-25T10:01:07.625695+0000","last_active":"2024-04-25T10:01:09.642503+0000","last_peered":"2024-04-25T10:01:09.642503+0000","last_clean":"2024-04-25T10:01:09.642503+0000","last_became_active":"2024-04-25T10:01:07.625326+0000","last_became_peered":"2024-04-25T10:01:07.625326+0000","last_unstale":"2024-04-25T10:01:09.642503+0000","last_undegraded":"2024-04-25T10:01:09.642503+0000","last_fullsized":"2024-04-25T10:01:09.642503+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:05.102035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659826+0000","last_change":"2024-04-25T10:01:08.656149+0000","last_active":"2024-04-25T10:01:08.659826+0000","last_peered":"2024-04-25T10:01:08.659826+0000","last_clean":"2024-04-25T10:01:08.659826+0000","last_became_active":"2024-04-25T10:01:08.655844+0000","last_became_peered":"2024-04-25T10:01:08.655844+0000","last_unstale":"2024-04-25T10:01:08.659826+0000","last_undegraded":"2024-04-25T10:01:08.659826+0000","last_fullsized":"2024-04-25T10:01:08.659826+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.655871+0000","last_change":"2024-04-25T10:01:07.616666+0000","last_active":"2024-04-25T10:01:08.655871+0000","last_peered":"2024-04-25T10:01:08.655871+0000","last_clean":"2024-04-25T10:01:08.655871+0000","last_became_active":"2024-04-25T10:01:07.616325+0000","last_became_peered":"2024-04-25T10:01:07.616325+0000","last_unstale":"2024-04-25T10:01:08.655871+0000","last_undegraded":"2024-04-25T10:01:08.655871+0000","last_fullsized":"2024-04-25T10:01:08.655871+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.961926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.645496+0000","last_change":"2024-04-25T10:01:08.644638+0000","last_active":"2024-04-25T10:01:08.645496+0000","last_peered":"2024-04-25T10:01:08.645496+0000","last_clean":"2024-04-25T10:01:08.645496+0000","last_became_active":"2024-04-25T10:01:08.644370+0000","last_became_peered":"2024-04-25T10:01:08.644370+0000","last_unstale":"2024-04-25T10:01:08.645496+0000","last_undegraded":"2024-04-25T10:01:08.645496+0000","last_fullsized":"2024-04-25T10:01:08.645496+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:21.459828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658975+0000","last_change":"2024-04-25T10:01:07.627666+0000","last_active":"2024-04-25T10:01:08.658975+0000","last_peered":"2024-04-25T10:01:08.658975+0000","last_clean":"2024-04-25T10:01:08.658975+0000","last_became_active":"2024-04-25T10:01:07.627311+0000","last_became_peered":"2024-04-25T10:01:07.627311+0000","last_unstale":"2024-04-25T10:01:08.658975+0000","last_undegraded":"2024-04-25T10:01:08.658975+0000","last_fullsized":"2024-04-25T10:01:08.658975+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:19.078513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966792+0000","last_change":"2024-04-25T10:01:08.654445+0000","last_active":"2024-04-25T10:01:09.966792+0000","last_peered":"2024-04-25T10:01:09.966792+0000","last_clean":"2024-04-25T10:01:09.966792+0000","last_became_active":"2024-04-25T10:01:08.653208+0000","last_became_peered":"2024-04-25T10:01:08.653208+0000","last_unstale":"2024-04-25T10:01:09.966792+0000","last_undegraded":"2024-04-25T10:01:09.966792+0000","last_fullsized":"2024-04-25T10:01:09.966792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:36.551347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656349+0000","last_change":"2024-04-25T10:01:07.622473+0000","last_active":"2024-04-25T10:01:08.656349+0000","last_peered":"2024-04-25T10:01:08.656349+0000","last_clean":"2024-04-25T10:01:08.656349+0000","last_became_active":"2024-04-25T10:01:07.621618+0000","last_became_peered":"2024-04-25T10:01:07.621618+0000","last_unstale":"2024-04-25T10:01:08.656349+0000","last_undegraded":"2024-04-25T10:01:08.656349+0000","last_fullsized":"2024-04-25T10:01:08.656349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:19.475669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.657672+0000","last_change":"2024-04-25T10:01:08.656318+0000","last_active":"2024-04-25T10:01:08.657672+0000","last_peered":"2024-04-25T10:01:08.657672+0000","last_clean":"2024-04-25T10:01:08.657672+0000","last_became_active":"2024-04-25T10:01:08.653440+0000","last_became_peered":"2024-04-25T10:01:08.653440+0000","last_unstale":"2024-04-25T10:01:08.657672+0000","last_undegraded":"2024-04-25T10:01:08.657672+0000","last_fullsized":"2024-04-25T10:01:08.657672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:42.660604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642189+0000","last_change":"2024-04-25T10:01:07.624656+0000","last_active":"2024-04-25T10:01:09.642189+0000","last_peered":"2024-04-25T10:01:09.642189+0000","last_clean":"2024-04-25T10:01:09.642189+0000","last_became_active":"2024-04-25T10:01:07.624326+0000","last_became_peered":"2024-04-25T10:01:07.624326+0000","last_unstale":"2024-04-25T10:01:09.642189+0000","last_undegraded":"2024-04-25T10:01:09.642189+0000","last_fullsized":"2024-04-25T10:01:09.642189+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:26.201576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.649824+0000","last_change":"2024-04-25T10:01:08.648418+0000","last_active":"2024-04-25T10:01:08.649824+0000","last_peered":"2024-04-25T10:01:08.649824+0000","last_clean":"2024-04-25T10:01:08.649824+0000","last_became_active":"2024-04-25T10:01:08.648164+0000","last_became_peered":"2024-04-25T10:01:08.648164+0000","last_unstale":"2024-04-25T10:01:08.649824+0000","last_undegraded":"2024-04-25T10:01:08.649824+0000","last_fullsized":"2024-04-25T10:01:08.649824+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.637179+0000","last_change":"2024-04-25T10:01:07.621178+0000","last_active":"2024-04-25T10:01:08.637179+0000","last_peered":"2024-04-25T10:01:08.637179+0000","last_clean":"2024-04-25T10:01:08.637179+0000","last_became_active":"2024-04-25T10:01:07.620918+0000","last_became_peered":"2024-04-25T10:01:07.620918+0000","last_unstale":"2024-04-25T10:01:08.637179+0000","last_undegraded":"2024-04-25T10:01:08.637179+0000","last_fullsized":"2024-04-25T10:01:08.637179+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966683+0000","last_change":"2024-04-25T10:01:08.660501+0000","last_active":"2024-04-25T10:01:09.966683+0000","last_peered":"2024-04-25T10:01:09.966683+0000","last_clean":"2024-04-25T10:01:09.966683+0000","last_became_active":"2024-04-25T10:01:08.660237+0000","last_became_peered":"2024-04-25T10:01:08.660237+0000","last_unstale":"2024-04-25T10:01:09.966683+0000","last_undegraded":"2024-04-25T10:01:09.966683+0000","last_fullsized":"2024-04-25T10:01:09.966683+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:41.524775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642674+0000","last_change":"2024-04-25T10:01:07.624534+0000","last_active":"2024-04-25T10:01:09.642674+0000","last_peered":"2024-04-25T10:01:09.642674+0000","last_clean":"2024-04-25T10:01:09.642674+0000","last_became_active":"2024-04-25T10:01:07.624256+0000","last_became_peered":"2024-04-25T10:01:07.624256+0000","last_unstale":"2024-04-25T10:01:09.642674+0000","last_undegraded":"2024-04-25T10:01:09.642674+0000","last_fullsized":"2024-04-25T10:01:09.642674+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:41.658543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651659+0000","last_change":"2024-04-25T10:01:08.649945+0000","last_active":"2024-04-25T10:01:08.651659+0000","last_peered":"2024-04-25T10:01:08.651659+0000","last_clean":"2024-04-25T10:01:08.651659+0000","last_became_active":"2024-04-25T10:01:08.649629+0000","last_became_peered":"2024-04-25T10:01:08.649629+0000","last_unstale":"2024-04-25T10:01:08.651659+0000","last_undegraded":"2024-04-25T10:01:08.651659+0000","last_fullsized":"2024-04-25T10:01:08.651659+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:29.791247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966456+0000","last_change":"2024-04-25T10:01:07.621825+0000","last_active":"2024-04-25T10:01:09.966456+0000","last_peered":"2024-04-25T10:01:09.966456+0000","last_clean":"2024-04-25T10:01:09.966456+0000","last_became_active":"2024-04-25T10:01:07.621570+0000","last_became_peered":"2024-04-25T10:01:07.621570+0000","last_unstale":"2024-04-25T10:01:09.966456+0000","last_undegraded":"2024-04-25T10:01:09.966456+0000","last_fullsized":"2024-04-25T10:01:09.966456+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639360+0000","last_change":"2024-04-25T10:01:08.651676+0000","last_active":"2024-04-25T10:01:09.639360+0000","last_peered":"2024-04-25T10:01:09.639360+0000","last_clean":"2024-04-25T10:01:09.639360+0000","last_became_active":"2024-04-25T10:01:08.651351+0000","last_became_peered":"2024-04-25T10:01:08.651351+0000","last_unstale":"2024-04-25T10:01:09.639360+0000","last_undegraded":"2024-04-25T10:01:09.639360+0000","last_fullsized":"2024-04-25T10:01:09.639360+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:12.709360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"108'3","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658123+0000","last_change":"2024-04-25T10:01:07.621881+0000","last_active":"2024-04-25T10:01:08.658123+0000","last_peered":"2024-04-25T10:01:08.658123+0000","last_clean":"2024-04-25T10:01:08.658123+0000","last_became_active":"2024-04-25T10:01:07.621635+0000","last_became_peered":"2024-04-25T10:01:07.621635+0000","last_unstale":"2024-04-25T10:01:08.658123+0000","last_undegraded":"2024-04-25T10:01:08.658123+0000","last_fullsized":"2024-04-25T10:01:08.658123+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:20.457767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656725+0000","last_change":"2024-04-25T10:01:08.653783+0000","last_active":"2024-04-25T10:01:08.656725+0000","last_peered":"2024-04-25T10:01:08.656725+0000","last_clean":"2024-04-25T10:01:08.656725+0000","last_became_active":"2024-04-25T10:01:08.653445+0000","last_became_peered":"2024-04-25T10:01:08.653445+0000","last_unstale":"2024-04-25T10:01:08.656725+0000","last_undegraded":"2024-04-25T10:01:08.656725+0000","last_fullsized":"2024-04-25T10:01:08.656725+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:29.785417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"108'6","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.662151+0000","last_change":"2024-04-25T10:01:07.625818+0000","last_active":"2024-04-25T10:01:08.662151+0000","last_peered":"2024-04-25T10:01:08.662151+0000","last_clean":"2024-04-25T10:01:08.662151+0000","last_became_active":"2024-04-25T10:01:07.625208+0000","last_became_peered":"2024-04-25T10:01:07.625208+0000","last_unstale":"2024-04-25T10:01:08.662151+0000","last_undegraded":"2024-04-25T10:01:08.662151+0000","last_fullsized":"2024-04-25T10:01:08.662151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:41.642231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.653854+0000","last_change":"2024-04-25T10:01:08.649337+0000","last_active":"2024-04-25T10:01:08.653854+0000","last_peered":"2024-04-25T10:01:08.653854+0000","last_clean":"2024-04-25T10:01:08.653854+0000","last_became_active":"2024-04-25T10:01:08.649008+0000","last_became_peered":"2024-04-25T10:01:08.649008+0000","last_unstale":"2024-04-25T10:01:08.653854+0000","last_undegraded":"2024-04-25T10:01:08.653854+0000","last_fullsized":"2024-04-25T10:01:08.653854+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:09.254918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659145+0000","last_change":"2024-04-25T10:01:07.628914+0000","last_active":"2024-04-25T10:01:08.659145+0000","last_peered":"2024-04-25T10:01:08.659145+0000","last_clean":"2024-04-25T10:01:08.659145+0000","last_became_active":"2024-04-25T10:01:07.628154+0000","last_became_peered":"2024-04-25T10:01:07.628154+0000","last_unstale":"2024-04-25T10:01:08.659145+0000","last_undegraded":"2024-04-25T10:01:08.659145+0000","last_fullsized":"2024-04-25T10:01:08.659145+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:46.534428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.650363+0000","last_change":"2024-04-25T10:01:08.649599+0000","last_active":"2024-04-25T10:01:08.650363+0000","last_peered":"2024-04-25T10:01:08.650363+0000","last_clean":"2024-04-25T10:01:08.650363+0000","last_became_active":"2024-04-25T10:01:08.649336+0000","last_became_peered":"2024-04-25T10:01:08.649336+0000","last_unstale":"2024-04-25T10:01:08.650363+0000","last_undegraded":"2024-04-25T10:01:08.650363+0000","last_fullsized":"2024-04-25T10:01:08.650363+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:29.791247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.637485+0000","last_change":"2024-04-25T10:01:07.616480+0000","last_active":"2024-04-25T10:01:08.637485+0000","last_peered":"2024-04-25T10:01:08.637485+0000","last_clean":"2024-04-25T10:01:08.637485+0000","last_became_active":"2024-04-25T10:01:07.616218+0000","last_became_peered":"2024-04-25T10:01:07.616218+0000","last_unstale":"2024-04-25T10:01:08.637485+0000","last_undegraded":"2024-04-25T10:01:08.637485+0000","last_fullsized":"2024-04-25T10:01:08.637485+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.342879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639474+0000","last_change":"2024-04-25T10:01:08.641341+0000","last_active":"2024-04-25T10:01:09.639474+0000","last_peered":"2024-04-25T10:01:09.639474+0000","last_clean":"2024-04-25T10:01:09.639474+0000","last_became_active":"2024-04-25T10:01:08.640054+0000","last_became_peered":"2024-04-25T10:01:08.640054+0000","last_unstale":"2024-04-25T10:01:09.639474+0000","last_undegraded":"2024-04-25T10:01:09.639474+0000","last_fullsized":"2024-04-25T10:01:09.639474+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:46.236149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639206+0000","last_change":"2024-04-25T10:01:07.619939+0000","last_active":"2024-04-25T10:01:09.639206+0000","last_peered":"2024-04-25T10:01:09.639206+0000","last_clean":"2024-04-25T10:01:09.639206+0000","last_became_active":"2024-04-25T10:01:07.619608+0000","last_became_peered":"2024-04-25T10:01:07.619608+0000","last_unstale":"2024-04-25T10:01:09.639206+0000","last_undegraded":"2024-04-25T10:01:09.639206+0000","last_fullsized":"2024-04-25T10:01:09.639206+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:33.606047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966793+0000","last_change":"2024-04-25T10:01:08.657650+0000","last_active":"2024-04-25T10:01:09.966793+0000","last_peered":"2024-04-25T10:01:09.966793+0000","last_clean":"2024-04-25T10:01:09.966793+0000","last_became_active":"2024-04-25T10:01:08.657355+0000","last_became_peered":"2024-04-25T10:01:08.657355+0000","last_unstale":"2024-04-25T10:01:09.966793+0000","last_undegraded":"2024-04-25T10:01:09.966793+0000","last_fullsized":"2024-04-25T10:01:09.966793+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:03.837528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642167+0000","last_change":"2024-04-25T10:01:07.618962+0000","last_active":"2024-04-25T10:01:09.642167+0000","last_peered":"2024-04-25T10:01:09.642167+0000","last_clean":"2024-04-25T10:01:09.642167+0000","last_became_active":"2024-04-25T10:01:07.618628+0000","last_became_peered":"2024-04-25T10:01:07.618628+0000","last_unstale":"2024-04-25T10:01:09.642167+0000","last_undegraded":"2024-04-25T10:01:09.642167+0000","last_fullsized":"2024-04-25T10:01:09.642167+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:09.048291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639519+0000","last_change":"2024-04-25T10:01:08.645514+0000","last_active":"2024-04-25T10:01:09.639519+0000","last_peered":"2024-04-25T10:01:09.639519+0000","last_clean":"2024-04-25T10:01:09.639519+0000","last_became_active":"2024-04-25T10:01:08.645251+0000","last_became_peered":"2024-04-25T10:01:08.645251+0000","last_unstale":"2024-04-25T10:01:09.639519+0000","last_undegraded":"2024-04-25T10:01:09.639519+0000","last_fullsized":"2024-04-25T10:01:09.639519+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:29.290614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.658902+0000","last_change":"2024-04-25T10:01:07.626839+0000","last_active":"2024-04-25T10:01:08.658902+0000","last_peered":"2024-04-25T10:01:08.658902+0000","last_clean":"2024-04-25T10:01:08.658902+0000","last_became_active":"2024-04-25T10:01:07.626229+0000","last_became_peered":"2024-04-25T10:01:07.626229+0000","last_unstale":"2024-04-25T10:01:08.658902+0000","last_undegraded":"2024-04-25T10:01:08.658902+0000","last_fullsized":"2024-04-25T10:01:08.658902+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:06.174575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.650175+0000","last_change":"2024-04-25T10:01:08.649611+0000","last_active":"2024-04-25T10:01:08.650175+0000","last_peered":"2024-04-25T10:01:08.650175+0000","last_clean":"2024-04-25T10:01:08.650175+0000","last_became_active":"2024-04-25T10:01:08.649321+0000","last_became_peered":"2024-04-25T10:01:08.649321+0000","last_unstale":"2024-04-25T10:01:08.650175+0000","last_undegraded":"2024-04-25T10:01:08.650175+0000","last_fullsized":"2024-04-25T10:01:08.650175+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:04.944422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"108'1","reported_seq":13,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651626+0000","last_change":"2024-04-25T10:01:07.621132+0000","last_active":"2024-04-25T10:01:08.651626+0000","last_peered":"2024-04-25T10:01:08.651626+0000","last_clean":"2024-04-25T10:01:08.651626+0000","last_became_active":"2024-04-25T10:01:07.620856+0000","last_became_peered":"2024-04-25T10:01:07.620856+0000","last_unstale":"2024-04-25T10:01:08.651626+0000","last_undegraded":"2024-04-25T10:01:08.651626+0000","last_fullsized":"2024-04-25T10:01:08.651626+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:08.252856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.644731+0000","last_change":"2024-04-25T10:01:08.643843+0000","last_active":"2024-04-25T10:01:08.644731+0000","last_peered":"2024-04-25T10:01:08.644731+0000","last_clean":"2024-04-25T10:01:08.644731+0000","last_became_active":"2024-04-25T10:01:08.643309+0000","last_became_peered":"2024-04-25T10:01:08.643309+0000","last_unstale":"2024-04-25T10:01:08.644731+0000","last_undegraded":"2024-04-25T10:01:08.644731+0000","last_fullsized":"2024-04-25T10:01:08.644731+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.976052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.645281+0000","last_change":"2024-04-25T10:01:07.621971+0000","last_active":"2024-04-25T10:01:08.645281+0000","last_peered":"2024-04-25T10:01:08.645281+0000","last_clean":"2024-04-25T10:01:08.645281+0000","last_became_active":"2024-04-25T10:01:07.621650+0000","last_became_peered":"2024-04-25T10:01:07.621650+0000","last_unstale":"2024-04-25T10:01:08.645281+0000","last_undegraded":"2024-04-25T10:01:08.645281+0000","last_fullsized":"2024-04-25T10:01:08.645281+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966513+0000","last_change":"2024-04-25T10:01:08.642726+0000","last_active":"2024-04-25T10:01:09.966513+0000","last_peered":"2024-04-25T10:01:09.966513+0000","last_clean":"2024-04-25T10:01:09.966513+0000","last_became_active":"2024-04-25T10:01:08.642428+0000","last_became_peered":"2024-04-25T10:01:08.642428+0000","last_unstale":"2024-04-25T10:01:09.966513+0000","last_undegraded":"2024-04-25T10:01:09.966513+0000","last_fullsized":"2024-04-25T10:01:09.966513+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:32.717258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639659+0000","last_change":"2024-04-25T10:01:07.629013+0000","last_active":"2024-04-25T10:01:09.639659+0000","last_peered":"2024-04-25T10:01:09.639659+0000","last_clean":"2024-04-25T10:01:09.639659+0000","last_became_active":"2024-04-25T10:01:07.628383+0000","last_became_peered":"2024-04-25T10:01:07.628383+0000","last_unstale":"2024-04-25T10:01:09.639659+0000","last_undegraded":"2024-04-25T10:01:09.639659+0000","last_fullsized":"2024-04-25T10:01:09.639659+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:48.862207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966715+0000","last_change":"2024-04-25T10:01:08.656442+0000","last_active":"2024-04-25T10:01:09.966715+0000","last_peered":"2024-04-25T10:01:09.966715+0000","last_clean":"2024-04-25T10:01:09.966715+0000","last_became_active":"2024-04-25T10:01:08.654125+0000","last_became_peered":"2024-04-25T10:01:08.654125+0000","last_unstale":"2024-04-25T10:01:09.966715+0000","last_undegraded":"2024-04-25T10:01:09.966715+0000","last_fullsized":"2024-04-25T10:01:09.966715+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:12.719132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639163+0000","last_change":"2024-04-25T10:01:07.621828+0000","last_active":"2024-04-25T10:01:09.639163+0000","last_peered":"2024-04-25T10:01:09.639163+0000","last_clean":"2024-04-25T10:01:09.639163+0000","last_became_active":"2024-04-25T10:01:07.621541+0000","last_became_peered":"2024-04-25T10:01:07.621541+0000","last_unstale":"2024-04-25T10:01:09.639163+0000","last_undegraded":"2024-04-25T10:01:09.639163+0000","last_fullsized":"2024-04-25T10:01:09.639163+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:49.010362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659108+0000","last_change":"2024-04-25T10:01:08.657148+0000","last_active":"2024-04-25T10:01:08.659108+0000","last_peered":"2024-04-25T10:01:08.659108+0000","last_clean":"2024-04-25T10:01:08.659108+0000","last_became_active":"2024-04-25T10:01:08.656868+0000","last_became_peered":"2024-04-25T10:01:08.656868+0000","last_unstale":"2024-04-25T10:01:08.659108+0000","last_undegraded":"2024-04-25T10:01:08.659108+0000","last_fullsized":"2024-04-25T10:01:08.659108+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:18.169599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.643331+0000","last_change":"2024-04-25T10:01:07.623823+0000","last_active":"2024-04-25T10:01:08.643331+0000","last_peered":"2024-04-25T10:01:08.643331+0000","last_clean":"2024-04-25T10:01:08.643331+0000","last_became_active":"2024-04-25T10:01:07.623540+0000","last_became_peered":"2024-04-25T10:01:07.623540+0000","last_unstale":"2024-04-25T10:01:08.643331+0000","last_undegraded":"2024-04-25T10:01:08.643331+0000","last_fullsized":"2024-04-25T10:01:08.643331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:18.266105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639431+0000","last_change":"2024-04-25T10:01:08.642443+0000","last_active":"2024-04-25T10:01:09.639431+0000","last_peered":"2024-04-25T10:01:09.639431+0000","last_clean":"2024-04-25T10:01:09.639431+0000","last_became_active":"2024-04-25T10:01:08.642173+0000","last_became_peered":"2024-04-25T10:01:08.642173+0000","last_unstale":"2024-04-25T10:01:09.639431+0000","last_undegraded":"2024-04-25T10:01:09.639431+0000","last_fullsized":"2024-04-25T10:01:09.639431+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:03.657153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639358+0000","last_change":"2024-04-25T10:01:07.622643+0000","last_active":"2024-04-25T10:01:09.639358+0000","last_peered":"2024-04-25T10:01:09.639358+0000","last_clean":"2024-04-25T10:01:09.639358+0000","last_became_active":"2024-04-25T10:01:07.622210+0000","last_became_peered":"2024-04-25T10:01:07.622210+0000","last_unstale":"2024-04-25T10:01:09.639358+0000","last_undegraded":"2024-04-25T10:01:09.639358+0000","last_fullsized":"2024-04-25T10:01:09.639358+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.805840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966772+0000","last_change":"2024-04-25T10:01:08.647311+0000","last_active":"2024-04-25T10:01:09.966772+0000","last_peered":"2024-04-25T10:01:09.966772+0000","last_clean":"2024-04-25T10:01:09.966772+0000","last_became_active":"2024-04-25T10:01:08.646589+0000","last_became_peered":"2024-04-25T10:01:08.646589+0000","last_unstale":"2024-04-25T10:01:09.966772+0000","last_undegraded":"2024-04-25T10:01:09.966772+0000","last_fullsized":"2024-04-25T10:01:09.966772+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:57.642202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"108'2","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656404+0000","last_change":"2024-04-25T10:01:07.628529+0000","last_active":"2024-04-25T10:01:08.656404+0000","last_peered":"2024-04-25T10:01:08.656404+0000","last_clean":"2024-04-25T10:01:08.656404+0000","last_became_active":"2024-04-25T10:01:07.628185+0000","last_became_peered":"2024-04-25T10:01:07.628185+0000","last_unstale":"2024-04-25T10:01:08.656404+0000","last_undegraded":"2024-04-25T10:01:08.656404+0000","last_fullsized":"2024-04-25T10:01:08.656404+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:41.642231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642227+0000","last_change":"2024-04-25T10:01:08.643343+0000","last_active":"2024-04-25T10:01:09.642227+0000","last_peered":"2024-04-25T10:01:09.642227+0000","last_clean":"2024-04-25T10:01:09.642227+0000","last_became_active":"2024-04-25T10:01:08.643066+0000","last_became_peered":"2024-04-25T10:01:08.643066+0000","last_unstale":"2024-04-25T10:01:09.642227+0000","last_undegraded":"2024-04-25T10:01:09.642227+0000","last_fullsized":"2024-04-25T10:01:09.642227+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.080575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.966505+0000","last_change":"2024-04-25T10:01:07.626502+0000","last_active":"2024-04-25T10:01:09.966505+0000","last_peered":"2024-04-25T10:01:09.966505+0000","last_clean":"2024-04-25T10:01:09.966505+0000","last_became_active":"2024-04-25T10:01:07.626187+0000","last_became_peered":"2024-04-25T10:01:07.626187+0000","last_unstale":"2024-04-25T10:01:09.966505+0000","last_undegraded":"2024-04-25T10:01:09.966505+0000","last_fullsized":"2024-04-25T10:01:09.966505+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:42.812396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.659660+0000","last_change":"2024-04-25T10:01:08.656378+0000","last_active":"2024-04-25T10:01:08.659660+0000","last_peered":"2024-04-25T10:01:08.659660+0000","last_clean":"2024-04-25T10:01:08.659660+0000","last_became_active":"2024-04-25T10:01:08.656090+0000","last_became_peered":"2024-04-25T10:01:08.656090+0000","last_unstale":"2024-04-25T10:01:08.659660+0000","last_undegraded":"2024-04-25T10:01:08.659660+0000","last_fullsized":"2024-04-25T10:01:08.659660+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.626241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.639731+0000","last_change":"2024-04-25T10:01:07.623261+0000","last_active":"2024-04-25T10:01:08.639731+0000","last_peered":"2024-04-25T10:01:08.639731+0000","last_clean":"2024-04-25T10:01:08.639731+0000","last_became_active":"2024-04-25T10:01:07.622935+0000","last_became_peered":"2024-04-25T10:01:07.622935+0000","last_unstale":"2024-04-25T10:01:08.639731+0000","last_undegraded":"2024-04-25T10:01:08.639731+0000","last_fullsized":"2024-04-25T10:01:08.639731+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.961926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.656011+0000","last_change":"2024-04-25T10:01:08.653766+0000","last_active":"2024-04-25T10:01:08.656011+0000","last_peered":"2024-04-25T10:01:08.656011+0000","last_clean":"2024-04-25T10:01:08.656011+0000","last_became_active":"2024-04-25T10:01:08.653271+0000","last_became_peered":"2024-04-25T10:01:08.653271+0000","last_unstale":"2024-04-25T10:01:08.656011+0000","last_undegraded":"2024-04-25T10:01:08.656011+0000","last_fullsized":"2024-04-25T10:01:08.656011+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.641404+0000","last_change":"2024-04-25T10:01:07.623766+0000","last_active":"2024-04-25T10:01:08.641404+0000","last_peered":"2024-04-25T10:01:08.641404+0000","last_clean":"2024-04-25T10:01:08.641404+0000","last_became_active":"2024-04-25T10:01:07.623498+0000","last_became_peered":"2024-04-25T10:01:07.623498+0000","last_unstale":"2024-04-25T10:01:08.641404+0000","last_undegraded":"2024-04-25T10:01:08.641404+0000","last_fullsized":"2024-04-25T10:01:08.641404+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:18.266105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639273+0000","last_change":"2024-04-25T10:01:08.642642+0000","last_active":"2024-04-25T10:01:09.639273+0000","last_peered":"2024-04-25T10:01:09.639273+0000","last_clean":"2024-04-25T10:01:09.639273+0000","last_became_active":"2024-04-25T10:01:08.642385+0000","last_became_peered":"2024-04-25T10:01:08.642385+0000","last_unstale":"2024-04-25T10:01:09.639273+0000","last_undegraded":"2024-04-25T10:01:09.639273+0000","last_fullsized":"2024-04-25T10:01:09.639273+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:20.472767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.642348+0000","last_change":"2024-04-25T10:01:07.627410+0000","last_active":"2024-04-25T10:01:09.642348+0000","last_peered":"2024-04-25T10:01:09.642348+0000","last_clean":"2024-04-25T10:01:09.642348+0000","last_became_active":"2024-04-25T10:01:07.627089+0000","last_became_peered":"2024-04-25T10:01:07.627089+0000","last_unstale":"2024-04-25T10:01:09.642348+0000","last_undegraded":"2024-04-25T10:01:09.642348+0000","last_fullsized":"2024-04-25T10:01:09.642348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:25.190361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639406+0000","last_change":"2024-04-25T10:01:08.642060+0000","last_active":"2024-04-25T10:01:09.639406+0000","last_peered":"2024-04-25T10:01:09.639406+0000","last_clean":"2024-04-25T10:01:09.639406+0000","last_became_active":"2024-04-25T10:01:08.641779+0000","last_became_peered":"2024-04-25T10:01:08.641779+0000","last_unstale":"2024-04-25T10:01:09.639406+0000","last_undegraded":"2024-04-25T10:01:09.639406+0000","last_fullsized":"2024-04-25T10:01:09.639406+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:51.135228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639464+0000","last_change":"2024-04-25T10:01:07.620335+0000","last_active":"2024-04-25T10:01:09.639464+0000","last_peered":"2024-04-25T10:01:09.639464+0000","last_clean":"2024-04-25T10:01:09.639464+0000","last_became_active":"2024-04-25T10:01:07.619641+0000","last_became_peered":"2024-04-25T10:01:07.619641+0000","last_unstale":"2024-04-25T10:01:09.639464+0000","last_undegraded":"2024-04-25T10:01:09.639464+0000","last_fullsized":"2024-04-25T10:01:09.639464+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:26.388795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651574+0000","last_change":"2024-04-25T10:01:08.648935+0000","last_active":"2024-04-25T10:01:08.651574+0000","last_peered":"2024-04-25T10:01:08.651574+0000","last_clean":"2024-04-25T10:01:08.651574+0000","last_became_active":"2024-04-25T10:01:08.648677+0000","last_became_peered":"2024-04-25T10:01:08.648677+0000","last_unstale":"2024-04-25T10:01:08.651574+0000","last_undegraded":"2024-04-25T10:01:08.651574+0000","last_fullsized":"2024-04-25T10:01:08.651574+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.445229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.643483+0000","last_change":"2024-04-25T10:01:07.626934+0000","last_active":"2024-04-25T10:01:08.643483+0000","last_peered":"2024-04-25T10:01:08.643483+0000","last_clean":"2024-04-25T10:01:08.643483+0000","last_became_active":"2024-04-25T10:01:07.626558+0000","last_became_peered":"2024-04-25T10:01:07.626558+0000","last_unstale":"2024-04-25T10:01:08.643483+0000","last_undegraded":"2024-04-25T10:01:08.643483+0000","last_fullsized":"2024-04-25T10:01:08.643483+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:03.973991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.654256+0000","last_change":"2024-04-25T10:01:08.647871+0000","last_active":"2024-04-25T10:01:08.654256+0000","last_peered":"2024-04-25T10:01:08.654256+0000","last_clean":"2024-04-25T10:01:08.654256+0000","last_became_active":"2024-04-25T10:01:08.646567+0000","last_became_peered":"2024-04-25T10:01:08.646567+0000","last_unstale":"2024-04-25T10:01:08.654256+0000","last_undegraded":"2024-04-25T10:01:08.654256+0000","last_fullsized":"2024-04-25T10:01:08.654256+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:21.459828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639454+0000","last_change":"2024-04-25T10:01:07.620838+0000","last_active":"2024-04-25T10:01:09.639454+0000","last_peered":"2024-04-25T10:01:09.639454+0000","last_clean":"2024-04-25T10:01:09.639454+0000","last_became_active":"2024-04-25T10:01:07.620560+0000","last_became_peered":"2024-04-25T10:01:07.620560+0000","last_unstale":"2024-04-25T10:01:09.639454+0000","last_undegraded":"2024-04-25T10:01:09.639454+0000","last_fullsized":"2024-04-25T10:01:09.639454+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:08.363320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.651392+0000","last_change":"2024-04-25T10:01:08.647793+0000","last_active":"2024-04-25T10:01:08.651392+0000","last_peered":"2024-04-25T10:01:08.651392+0000","last_clean":"2024-04-25T10:01:08.651392+0000","last_became_active":"2024-04-25T10:01:08.647500+0000","last_became_peered":"2024-04-25T10:01:08.647500+0000","last_unstale":"2024-04-25T10:01:08.651392+0000","last_undegraded":"2024-04-25T10:01:08.651392+0000","last_fullsized":"2024-04-25T10:01:08.651392+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:07.603410+0000","last_clean_scrub_stamp":"2024-04-25T10:01:07.603410+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.080575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:01:09.639438+0000","last_change":"2024-04-25T10:01:07.625148+0000","last_active":"2024-04-25T10:01:09.639438+0000","last_peered":"2024-04-25T10:01:09.639438+0000","last_clean":"2024-04-25T10:01:09.639438+0000","last_became_active":"2024-04-25T10:01:07.624773+0000","last_became_peered":"2024-04-25T10:01:07.624773+0000","last_unstale":"2024-04-25T10:01:09.639438+0000","last_undegraded":"2024-04-25T10:01:09.639438+0000","last_fullsized":"2024-04-25T10:01:09.639438+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:06.601348+0000","last_clean_scrub_stamp":"2024-04-25T10:01:06.601348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:09.128646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":332,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T10:01:08.637490+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:08.637490+0000","last_peered":"2024-04-25T10:01:08.637490+0000","last_clean":"2024-04-25T10:01:08.637490+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:08.637490+0000","last_undegraded":"2024-04-25T10:01:08.637490+0000","last_fullsized":"2024-04-25T10:01:08.637490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41396,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330444,"statfs":{"total":96636764160,"available":96594374656,"internally_reserved":0,"allocated":1941504,"data_stored":1212272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46404,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325436,"statfs":{"total":96636764160,"available":96589246464,"internally_reserved":0,"allocated":1359872,"data_stored":623960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40808,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331032,"statfs":{"total":96636764160,"available":96594976768,"internally_reserved":0,"allocated":1339392,"data_stored":621744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1347584,"data_stored":621836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50548,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321292,"statfs":{"total":96636764160,"available":96585003008,"internally_reserved":0,"allocated":1343488,"data_stored":622326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46384,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325456,"statfs":{"total":96636764160,"available":96589266944,"internally_reserved":0,"allocated":1343488,"data_stored":621778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46380,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325460,"statfs":{"total":96636764160,"available":96589271040,"internally_reserved":0,"allocated":1343488,"data_stored":621778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46976,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324864,"statfs":{"total":96636764160,"available":96588660736,"internally_reserved":0,"allocated":1949696,"data_stored":1213814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:12.063 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:01:12.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:01:12.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.198+0000 7f3d33fa5640 1 Processor -- start 2024-04-25T10:01:12.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.198+0000 7f3d33fa5640 1 -- start start 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d33fa5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3d2c070dc0 0x7f3d2c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d33fa5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d33fa5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c0d1770 con 0x7f3d2c0711a0 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d33fa5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c0b0eb0 con 0x7f3d2c070dc0 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d33fa5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1622b0 con 0x7f3d2c15deb0 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d31d41640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3d2c070dc0 0x7f3d2c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d32542640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d31540640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3d2c0711a0 0x7f3d2c0b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:54690/0 (socket says 172.21.15.148:54690) 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d31d41640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3d2c070dc0 0x7f3d2c06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:44342/0 (socket says 172.21.15.148:44342) 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d32542640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55432/0 (socket says 172.21.15.148:55432) 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.199+0000 7f3d31540640 1 -- 172.21.15.148:0/1483168670 learned_addr learned my addr 172.21.15.148:0/1483168670 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.200+0000 7f3d31d41640 1 -- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 msgr2=0x7f3d2c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.200+0000 7f3d31d41640 1 --2- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c15e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.200+0000 7f3d31d41640 1 -- 172.21.15.148:0/1483168670 >> v1:172.21.15.188:6789/0 conn(0x7f3d2c0711a0 legacy=0x7f3d2c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.200+0000 7f3d31d41640 1 -- 172.21.15.148:0/1483168670 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c162740 con 0x7f3d2c070dc0 2024-04-25T10:01:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.200+0000 7f3d32542640 1 --2- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.200+0000 7f3d31d41640 1 --2- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c06ccd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d240097a0 tx=0x7f3d2402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f65b48b3eca0518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d30d3f640 1 -- 172.21.15.148:0/1483168670 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24037070 con 0x7f3d2c070dc0 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d30d3f640 1 -- 172.21.15.148:0/1483168670 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2402adb0 con 0x7f3d2c070dc0 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d30d3f640 1 -- 172.21.15.148:0/1483168670 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24032660 con 0x7f3d2c070dc0 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 -- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 msgr2=0x7f3d2c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c06ccd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d240097a0 tx=0x7f3d2402a060 comp rx=0 tx=0).stop 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 -- 172.21.15.148:0/1483168670 shutdown_connections 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c15e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/1483168670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c06ccd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 -- 172.21.15.148:0/1483168670 >> 172.21.15.148:0/1483168670 conn(0x7f3d2c0b3530 msgr2=0x7f3d2c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 -- 172.21.15.148:0/1483168670 shutdown_connections 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.201+0000 7f3d33fa5640 1 -- 172.21.15.148:0/1483168670 wait complete. 2024-04-25T10:01:12.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.202+0000 7f3d33fa5640 1 Processor -- start 2024-04-25T10:01:12.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.202+0000 7f3d33fa5640 1 -- start start 2024-04-25T10:01:12.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d33fa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c1f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d33fa5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 0x7f3d2c1f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d33fa5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c1f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d33fa5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c161c60 con 0x7f3d2c0711a0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d33fa5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0b10f0 con 0x7f3d2c070dc0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d33fa5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c161f40 con 0x7f3d2c15deb0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d31540640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 0x7f3d2c1f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d31540640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 0x7f3d2c1f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59872/0 (socket says 172.21.15.148:59872) 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d31540640 1 -- 172.21.15.148:0/973390716 learned_addr learned my addr 172.21.15.148:0/973390716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d32542640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c1f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.203+0000 7f3d31d41640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c1f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d31540640 1 -- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 msgr2=0x7f3d2c1f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d31540640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c1f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d31540640 1 -- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 msgr2=0x7f3d2c1f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d31540640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c1f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d31540640 1 -- 172.21.15.148:0/973390716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c1f5120 con 0x7f3d2c0711a0 2024-04-25T10:01:12.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d31540640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 0x7f3d2c1f0c70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c400 tx=0x7f3d1c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c017070 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c1f5400 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c005ce0 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0072f0 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.204+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c1f59c0 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.205+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c073a00 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.205+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d1c01a070 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.206+0000 7f3d227fc640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d1006b4d0 0x7f3d1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.206+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d1c0896f0 con 0x7f3d2c0711a0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.206+0000 7f3d31d41640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d1006b4d0 0x7f3d1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.206+0000 7f3d31d41640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d1006b4d0 0x7f3d1006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3d24034000 tx=0x7f3d2402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.210+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d1c057d10 con 0x7f3d2c0711a0 2024-04-25T10:01:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.447+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3d2c0b5440 con 0x7f3d2c0711a0 2024-04-25T10:01:12.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.449+0000 7f3d227fc640 1 -- 172.21.15.148:0/973390716 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3d1c05bdb0 con 0x7f3d2c0711a0 2024-04-25T10:01:12.454 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:01:12.454 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:01:12.455 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:01:12.455 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:01:12.455 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:12.455 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:01:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d1006b4d0 msgr2=0x7f3d1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d1006b4d0 0x7f3d1006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3d24034000 tx=0x7f3d2402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 msgr2=0x7f3d2c1f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 0x7f3d2c1f0c70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c400 tx=0x7f3d1c00c800 comp rx=0 tx=0).stop 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 shutdown_connections 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3d1006b4d0 0x7f3d1006d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c15deb0 0x7f3d2c1f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d2c0711a0 0x7f3d2c1f0c70 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 --2- 172.21.15.148:0/973390716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2c070dc0 0x7f3d2c1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 >> 172.21.15.148:0/973390716 conn(0x7f3d2c0b3530 msgr2=0x7f3d2c0d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 shutdown_connections 2024-04-25T10:01:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.451+0000 7f3d33fa5640 1 -- 172.21.15.148:0/973390716 wait complete. 2024-04-25T10:01:12.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:12.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.561+0000 7ff3ca5f5640 1 Processor -- start 2024-04-25T10:01:12.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.561+0000 7ff3ca5f5640 1 -- start start 2024-04-25T10:01:12.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.561+0000 7ff3ca5f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3c40d1f20 0x7ff3c41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.561+0000 7ff3ca5f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40cce50 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.561+0000 7ff3ca5f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c4161af0 con 0x7ff3c415dc90 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.561+0000 7ff3ca5f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c4161dd0 con 0x7ff3c40d1f20 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c8b92640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3c40d1f20 0x7ff3c41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c8b92640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3c40d1f20 0x7ff3c41625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59884/0 (socket says 172.21.15.148:59884) 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff3c40cc8e0 0x7ff3c40d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34070/0 (socket says 172.21.15.148:34070) 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c8b92640 1 -- 172.21.15.148:0/105989 learned_addr learned my addr 172.21.15.148:0/105989 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c8b92640 1 -- 172.21.15.148:0/105989 >> v1:172.21.15.188:6789/0 conn(0x7ff3c415dc90 legacy=0x7ff3c415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c8b92640 1 -- 172.21.15.148:0/105989 >> v1:172.21.15.148:6789/0 conn(0x7ff3c40cc8e0 legacy=0x7ff3c40d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c8b92640 1 -- 172.21.15.148:0/105989 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c41620b0 con 0x7ff3c40d1f20 2024-04-25T10:01:12.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.562+0000 7ff3c2ffd640 1 -- 172.21.15.148:0/105989 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662411012 0 0) 0x7ff3c40cce50 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3c8b92640 1 --2- 172.21.15.148:0/105989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41625d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009e50 tx=0x7ff3bc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd22aa18691283c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3c2ffd640 1 -- 172.21.15.148:0/105989 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc036070 con 0x7ff3c40d1f20 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3c2ffd640 1 -- 172.21.15.148:0/105989 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc031600 con 0x7ff3c40d1f20 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3c2ffd640 1 -- 172.21.15.148:0/105989 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0319d0 con 0x7ff3c40d1f20 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/105989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40d1f20 msgr2=0x7ff3c41625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/105989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41625d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009e50 tx=0x7ff3bc02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/105989 shutdown_connections 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/105989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41625d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/105989 >> 172.21.15.148:0/105989 conn(0x7ff3c406f590 msgr2=0x7ff3c406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/105989 shutdown_connections 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.563+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/105989 wait complete. 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.564+0000 7ff3ca5f5640 1 Processor -- start 2024-04-25T10:01:12.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 -- start start 2024-04-25T10:01:12.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 0x7ff3c40c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c415dc90 0x7ff3c4150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c415e1f0 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c415e070 con 0x7ff3c40d1f20 2024-04-25T10:01:12.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3ca5f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40cfde0 con 0x7ff3c415dc90 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 0x7ff3c40c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 0x7ff3c40c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59894/0 (socket says 172.21.15.148:59894) 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.565+0000 7ff3c3fff640 1 -- 172.21.15.148:0/684751362 learned_addr learned my addr 172.21.15.148:0/684751362 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c37fe640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c8b92640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c415dc90 0x7ff3c4150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c3fff640 1 -- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c415dc90 msgr2=0x7ff3c4150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c3fff640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c415dc90 0x7ff3c4150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c3fff640 1 -- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c40d1f20 msgr2=0x7ff3c41f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c3fff640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c3fff640 1 -- 172.21.15.148:0/684751362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c41f3fe0 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c8b92640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c415dc90 0x7ff3c4150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c3fff640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 0x7ff3c40c6860 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400b3f0 tx=0x7ff3b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4015020 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4011830 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c41f4280 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4011c20 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.566+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c41f4840 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.567+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c4073990 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.567+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3b4018020 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.568+0000 7ff3c0ff9640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3b006b4d0 0x7ff3b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.568+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff3b408a220 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.568+0000 7ff3c37fe640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3b006b4d0 0x7ff3b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:12.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.568+0000 7ff3c37fe640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3b006b4d0 0x7ff3b006d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40c5130 tx=0x7ff3b8007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:12.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.572+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3b401f020 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.777+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3c40cf6b0 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.778+0000 7ff3c0ff9640 1 -- 172.21.15.148:0/684751362 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ff3b40172c0 con 0x7ff3c40cc8e0 2024-04-25T10:01:12.783 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:12.783 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:12.784 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:12.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3b006b4d0 msgr2=0x7ff3b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3b006b4d0 0x7ff3b006d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40c5130 tx=0x7ff3b8007b50 comp rx=0 tx=0).stop 2024-04-25T10:01:12.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 msgr2=0x7ff3c40c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 0x7ff3c40c6860 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400b3f0 tx=0x7ff3b400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 shutdown_connections 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff3b006b4d0 0x7ff3b006d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3c415dc90 0x7ff3c4150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3c40d1f20 0x7ff3c41f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 --2- 172.21.15.148:0/684751362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3c40cc8e0 0x7ff3c40c6860 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 >> 172.21.15.148:0/684751362 conn(0x7ff3c406f590 msgr2=0x7ff3c40b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 shutdown_connections 2024-04-25T10:01:12.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:12.780+0000 7ff3ca5f5640 1 -- 172.21.15.148:0/684751362 wait complete. 2024-04-25T10:01:13.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:07.958159+0000', 'modified': '2024-04-25T10:01:10.095327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8163}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8163': {'gid': 8163, 'name': 'd', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6834/3455709247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 3455709247}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 3455709247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:01:13.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:01:13.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:01:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.294+0000 7f5a6f55a640 1 Processor -- start 2024-04-25T10:01:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.294+0000 7f5a6f55a640 1 -- start start 2024-04-25T10:01:13.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6f55a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a6806ccd0 0x7f5a680b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6f55a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a6815deb0 0x7f5a6815e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6f55a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680d1770 con 0x7f5a68070dc0 2024-04-25T10:01:13.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6f55a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a681622b0 con 0x7f5a6806ccd0 2024-04-25T10:01:13.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6f55a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680b10f0 con 0x7f5a6815deb0 2024-04-25T10:01:13.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a6806ccd0 0x7f5a680b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a6806ccd0 0x7f5a680b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59904/0 (socket says 172.21.15.148:59904) 2024-04-25T10:01:13.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 -- 172.21.15.148:0/1196259099 learned_addr learned my addr 172.21.15.148:0/1196259099 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 -- 172.21.15.148:0/1196259099 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815deb0 msgr2=0x7f5a6815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/1196259099 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815deb0 0x7f5a6815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/1196259099 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815deb0 0x7f5a6815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 -- 172.21.15.148:0/1196259099 >> v1:172.21.15.188:6789/0 conn(0x7f5a68070dc0 legacy=0x7f5a680711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.295+0000 7f5a6caf5640 1 -- 172.21.15.148:0/1196259099 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a68162740 con 0x7f5a6806ccd0 2024-04-25T10:01:13.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.296+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/1196259099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6806ccd0 0x7f5a680b07a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5a6806de30 tx=0x7f5a5802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cdb94d754e0418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.296+0000 7f5a5ffff640 1 -- 172.21.15.148:0/1196259099 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58037070 con 0x7f5a6806ccd0 2024-04-25T10:01:13.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.296+0000 7f5a5ffff640 1 -- 172.21.15.148:0/1196259099 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5802ae20 con 0x7f5a6806ccd0 2024-04-25T10:01:13.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.296+0000 7f5a5ffff640 1 -- 172.21.15.148:0/1196259099 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58032660 con 0x7f5a6806ccd0 2024-04-25T10:01:13.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 -- 172.21.15.148:0/1196259099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6806ccd0 msgr2=0x7f5a680b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/1196259099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6806ccd0 0x7f5a680b07a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5a6806de30 tx=0x7f5a5802a110 comp rx=0 tx=0).stop 2024-04-25T10:01:13.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 -- 172.21.15.148:0/1196259099 shutdown_connections 2024-04-25T10:01:13.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/1196259099 >> v2:172.21.15.188:3300/0 conn(0x7f5a6815deb0 0x7f5a6815e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/1196259099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6806ccd0 0x7f5a680b07a0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 -- 172.21.15.148:0/1196259099 >> 172.21.15.148:0/1196259099 conn(0x7f5a680b3530 msgr2=0x7f5a680b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 -- 172.21.15.148:0/1196259099 shutdown_connections 2024-04-25T10:01:13.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.297+0000 7f5a6f55a640 1 -- 172.21.15.148:0/1196259099 wait complete. 2024-04-25T10:01:13.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.298+0000 7f5a6f55a640 1 Processor -- start 2024-04-25T10:01:13.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.298+0000 7f5a6f55a640 1 -- start start 2024-04-25T10:01:13.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6806ccd0 0x7f5a6814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a68070dc0 0x7f5a680cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 0x7f5a68147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68161f40 con 0x7f5a6815deb0 2024-04-25T10:01:13.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68161c60 con 0x7f5a6806ccd0 2024-04-25T10:01:13.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680b5840 con 0x7f5a68070dc0 2024-04-25T10:01:13.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 0x7f5a68147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 0x7f5a68147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59912/0 (socket says 172.21.15.148:59912) 2024-04-25T10:01:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 -- 172.21.15.148:0/848262952 learned_addr learned my addr 172.21.15.148:0/848262952 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 -- 172.21.15.148:0/848262952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a68070dc0 msgr2=0x7f5a680cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a68070dc0 0x7f5a680cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 -- 172.21.15.148:0/848262952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6806ccd0 msgr2=0x7f5a6814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6806ccd0 0x7f5a6814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 -- 172.21.15.148:0/848262952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680cb0f0 con 0x7f5a6815deb0 2024-04-25T10:01:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 0x7f5a68147e40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cc30 tx=0x7f5a6000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a5dffb640 1 -- 172.21.15.148:0/848262952 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000ce40 con 0x7f5a6815deb0 2024-04-25T10:01:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a5dffb640 1 -- 172.21.15.148:0/848262952 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60003990 con 0x7f5a6815deb0 2024-04-25T10:01:13.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a68148380 con 0x7f5a6815deb0 2024-04-25T10:01:13.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a5dffb640 1 -- 172.21.15.148:0/848262952 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60003d40 con 0x7f5a6815deb0 2024-04-25T10:01:13.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.299+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6815ede0 con 0x7f5a6815deb0 2024-04-25T10:01:13.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.301+0000 7f5a5dffb640 1 -- 172.21.15.148:0/848262952 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a60004000 con 0x7f5a6815deb0 2024-04-25T10:01:13.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.302+0000 7f5a5dffb640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.302+0000 7f5a5dffb640 1 -- 172.21.15.148:0/848262952 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a6008a840 con 0x7f5a6815deb0 2024-04-25T10:01:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.302+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.302+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5a64003cc0 tx=0x7f5a640079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 Processor -- start 2024-04-25T10:01:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 -- start start 2024-04-25T10:01:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362c06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c24020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 0x55a362c22190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362b3ea10 con 0x55a362bdbf10 2024-04-25T10:01:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362ae7b30 con 0x55a362c06230 2024-04-25T10:01:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a362b3ac50 con 0x55a362c23ad0 2024-04-25T10:01:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a6daf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 0x55a362c22190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a6daf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 0x55a362c22190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59916/0 (socket says 172.21.15.148:59916) 2024-04-25T10:01:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.303+0000 7f5a6daf7640 1 -- 172.21.15.148:0/2343556528 learned_addr learned my addr 172.21.15.148:0/2343556528 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c24020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362c06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6daf7640 1 -- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 msgr2=0x55a362c24020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c24020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6daf7640 1 -- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 msgr2=0x55a362c06610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362c06610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6daf7640 1 -- 172.21.15.148:0/2343556528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a362afce10 con 0x55a362bdbf10 2024-04-25T10:01:13.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c24020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:13.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 0x55a362c22190 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5a60055290 tx=0x7f5a600595f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255c9f283238232f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a52ffd640 1 -- 172.21.15.148:0/2343556528 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6001d040 con 0x55a362bdbf10 2024-04-25T10:01:13.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a52ffd640 1 -- 172.21.15.148:0/2343556528 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60061540 con 0x55a362bdbf10 2024-04-25T10:01:13.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a52ffd640 1 -- 172.21.15.148:0/2343556528 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a600554a0 con 0x55a362bdbf10 2024-04-25T10:01:13.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 -- 172.21.15.148:0/2343556528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 msgr2=0x55a362c22190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 0x55a362c22190 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5a60055290 tx=0x7f5a600595f0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 -- 172.21.15.148:0/2343556528 shutdown_connections 2024-04-25T10:01:13.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362bdbf10 0x55a362c22190 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c24020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 --2- 172.21.15.148:0/2343556528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362c06610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.304+0000 7f5a72258740 1 -- 172.21.15.148:0/2343556528 >> 172.21.15.148:0/2343556528 conn(0x55a362bd8840 msgr2=0x55a362c21040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.305+0000 7f5a72258740 1 -- 172.21.15.148:0/2343556528 shutdown_connections 2024-04-25T10:01:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.305+0000 7f5a72258740 1 -- 172.21.15.148:0/2343556528 wait complete. 2024-04-25T10:01:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.305+0000 7f5a72258740 1 -- start start 2024-04-25T10:01:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.305+0000 7f5a72258740 1 Processor -- start 2024-04-25T10:01:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c32530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362bdccb0 con 0x55a362c06230 2024-04-25T10:01:13.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362afb0c0 con 0x55a362c18630 2024-04-25T10:01:13.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a362bd6770 con 0x55a362c23ad0 2024-04-25T10:01:13.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a72258740 10 client.0 _ll_register_callbacks cb 0x55a362aeeec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:13.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59926/0 (socket says 172.21.15.148:59926) 2024-04-25T10:01:13.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/2480123382 learned_addr learned my addr 172.21.15.148:0/2480123382 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c32530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 msgr2=0x55a362c180f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c180f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 msgr2=0x55a362c32530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c32530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/2480123382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3628532d0 con 0x55a362c06230 2024-04-25T10:01:13.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.306+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c046f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5a64014e80 tx=0x7f5a64003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac9a51f0dcb5456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a47fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a47fff640 1 -- 172.21.15.148:0/2480123382 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64024060 con 0x55a362c06230 2024-04-25T10:01:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a47fff640 1 -- 172.21.15.148:0/2480123382 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a6401f600 con 0x55a362c06230 2024-04-25T10:01:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a60061540 con 0x55a362c06230 2024-04-25T10:01:13.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a47fff640 1 -- 172.21.15.148:0/2480123382 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6401f800 con 0x55a362c06230 2024-04-25T10:01:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 10 client.8295 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55a36291d6b0 con 0x55a362c06230 2024-04-25T10:01:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a47fff640 1 -- 172.21.15.148:0/2480123382 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60061540 con 0x55a362c06230 2024-04-25T10:01:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a47fff640 1 -- 172.21.15.148:0/2480123382 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a6402d080 con 0x55a362c06230 2024-04-25T10:01:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 10 client.8295 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:13.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 10 client.8295 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:01:13.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 20 client.8295 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:13.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c3d140 0x55a362c3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 4 client.8295 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:13.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a362b9d920 con 0x55a362c3d140 2024-04-25T10:01:13.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.307+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c3d140 0x55a362c3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.308+0000 7f5a47fff640 10 client.8295 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:13.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.308+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c3d140 0x55a362c3f530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab190531431a1ce server_cookie=6fb927678ab11543 in_seq=0 out_seq=0 2024-04-25T10:01:13.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.308+0000 7f5a47fff640 1 -- 172.21.15.148:0/2480123382 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a362b9d920 con 0x55a362c3d140 2024-04-25T10:01:13.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.308+0000 7f5a47fff640 10 client.8295 handle_command_reply: tid=0 2024-04-25T10:01:13.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.308+0000 7f5a72258740 1 client.8295 shutdown 2024-04-25T10:01:13.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 msgr2=0x55a362c046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c046f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5a64014e80 tx=0x7f5a64003760 comp rx=0 tx=0).stop 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 shutdown_connections 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c3d140 0x55a362c3f530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c32530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 --2- 172.21.15.148:0/2480123382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c046f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a47fff640 0 client.8295 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 >> 172.21.15.148:0/2480123382 conn(0x55a362bdbf10 msgr2=0x55a362c24e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 shutdown_connections 2024-04-25T10:01:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 1 -- 172.21.15.148:0/2480123382 wait complete. 2024-04-25T10:01:13.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.309+0000 7f5a72258740 20 client.8295 trim_cache size 0 max 16384 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.324+0000 7f5a72258740 1 Processor -- start 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.324+0000 7f5a72258740 1 -- start start 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362bdc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a72258740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 0x55a362c3a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c3ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a72258740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681ff7a0 con 0x55a362c18630 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681f7f40 con 0x55a362c06230 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cdd20 con 0x55a362c23ad0 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 0x55a362c3a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 0x55a362c3a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59932/0 (socket says 172.21.15.148:59932) 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 -- 172.21.15.148:0/3925357300 learned_addr learned my addr 172.21.15.148:0/3925357300 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362bdc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c3ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 -- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 msgr2=0x55a362c3ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c3ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 -- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 msgr2=0x55a362bdc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362bdc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 -- 172.21.15.148:0/3925357300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a362b73460 con 0x55a362c18630 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c3ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362bdc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 0x55a362c3a6e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5a5802a150 tx=0x7f5a5802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c196426c596fce47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a527fc640 1 -- 172.21.15.148:0/3925357300 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58047070 con 0x55a362c18630 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a527fc640 1 -- 172.21.15.148:0/3925357300 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58042410 con 0x55a362c18630 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.325+0000 7f5a527fc640 1 -- 172.21.15.148:0/3925357300 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58037040 con 0x55a362c18630 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 -- 172.21.15.148:0/3925357300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 msgr2=0x55a362c3a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 0x55a362c3a6e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5a5802a150 tx=0x7f5a5802c000 comp rx=0 tx=0).stop 2024-04-25T10:01:13.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 -- 172.21.15.148:0/3925357300 shutdown_connections 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c3ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c18630 0x55a362c3a6e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 --2- 172.21.15.148:0/3925357300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c06230 0x55a362bdc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 -- 172.21.15.148:0/3925357300 >> 172.21.15.148:0/3925357300 conn(0x55a362bdbf10 msgr2=0x55a362bdb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 -- 172.21.15.148:0/3925357300 shutdown_connections 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 -- 172.21.15.148:0/3925357300 wait complete. 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 -- start start 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.326+0000 7f5a72258740 1 Processor -- start 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c8b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c43550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c43a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362c31ed0 con 0x55a362c06230 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a362c31910 con 0x55a362c18630 2024-04-25T10:01:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a362c33570 con 0x55a362c23ad0 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a72258740 10 client.0 _ll_register_callbacks cb 0x55a362bd6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a6d2f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c8b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a6d2f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c8b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59942/0 (socket says 172.21.15.148:59942) 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.327+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/660223548 learned_addr learned my addr 172.21.15.148:0/660223548 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c43a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c43550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 msgr2=0x55a362c43a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c43a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 msgr2=0x55a362c43550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c43550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6d2f6640 1 -- 172.21.15.148:0/660223548 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a362c16cb0 con 0x55a362c06230 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6daf7640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c43a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c43550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a6d2f6640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c8b980 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5a64014010 tx=0x7f5a64013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6012d01d68f030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a47fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a47fff640 1 -- 172.21.15.148:0/660223548 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64024050 con 0x55a362c06230 2024-04-25T10:01:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a47fff640 1 -- 172.21.15.148:0/660223548 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a6400cc50 con 0x55a362c06230 2024-04-25T10:01:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a58042410 con 0x55a362c06230 2024-04-25T10:01:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a47fff640 1 -- 172.21.15.148:0/660223548 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6400ce90 con 0x55a362c06230 2024-04-25T10:01:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 10 client.8301 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55a362c8bec0 con 0x55a362c06230 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a47fff640 1 -- 172.21.15.148:0/660223548 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58042410 con 0x55a362c06230 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a47fff640 1 -- 172.21.15.148:0/660223548 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a64005020 con 0x55a362c06230 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 10 client.8301 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 10 client.8301 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 20 client.8301 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c30f30 0x55a362c09420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 4 client.8301 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.328+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a362b4bf50 con 0x55a362c30f30 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c30f30 0x55a362c09420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a47fff640 10 client.8301 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a6caf5640 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c30f30 0x55a362c09420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af3305bdc0bbdf3 server_cookie=1b9ecffdc0577768 in_seq=0 out_seq=0 2024-04-25T10:01:13.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a47fff640 1 -- 172.21.15.148:0/660223548 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a362b4bf50 con 0x55a362c30f30 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a47fff640 10 client.8301 handle_command_reply: tid=0 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 client.8301 shutdown 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 msgr2=0x55a362c8b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c8b980 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5a64014010 tx=0x7f5a64013000 comp rx=0 tx=0).stop 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 shutdown_connections 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a362c23ad0 0x55a362c43a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a362c30f30 0x55a362c09420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a362c18630 0x55a362c43550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 --2- 172.21.15.148:0/660223548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a362c06230 0x55a362c8b980 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a47fff640 0 client.8301 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.329+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 >> 172.21.15.148:0/660223548 conn(0x55a362bdbf10 msgr2=0x55a362bdb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 shutdown_connections 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a72258740 1 -- 172.21.15.148:0/660223548 wait complete. 2024-04-25T10:01:13.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a72258740 20 client.8301 trim_cache size 0 max 16384 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 8163, 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.7003150930000004, 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 60, 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 109, 2024-04-25T10:01:13.360 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 109, 2024-04-25T10:01:13.361 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 9.7566029370000003 2024-04-25T10:01:13.361 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:13.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4c06b4d0 msgr2=0x7f5a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5a64003cc0 tx=0x7f5a640079b0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 msgr2=0x7f5a68147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 0x7f5a68147e40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000cc30 tx=0x7f5a6000a5c0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 shutdown_connections 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a6815deb0 0x7f5a68147e40 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a68070dc0 0x7f5a680cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 --2- 172.21.15.148:0/848262952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a6806ccd0 0x7f5a6814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 >> 172.21.15.148:0/848262952 conn(0x7f5a680b3530 msgr2=0x7f5a680ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 shutdown_connections 2024-04-25T10:01:13.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.330+0000 7f5a6f55a640 1 -- 172.21.15.148:0/848262952 wait complete. 2024-04-25T10:01:13.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:01:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.455+0000 7f550b728640 1 Processor -- start 2024-04-25T10:01:13.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.455+0000 7f550b728640 1 -- start start 2024-04-25T10:01:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f550b728640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f550415dd30 0x7f55040b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f550b728640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 0x7f55040b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f550b728640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040d1df0 con 0x7f55040b9770 2024-04-25T10:01:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f550b728640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55040d20d0 con 0x7f550415dd30 2024-04-25T10:01:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f550b728640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55040d23b0 con 0x7f55040d0ad0 2024-04-25T10:01:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f5508cc3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f55040b9770 0x7f55040d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34074/0 (socket says 172.21.15.148:34074) 2024-04-25T10:01:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.456+0000 7f5508cc3640 1 -- 172.21.15.148:0/374803282 learned_addr learned my addr 172.21.15.148:0/374803282 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254456841 0 0) 0x7f55040d1df0 con 0x7f55040b9770 2024-04-25T10:01:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f5509cc5640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 0x7f55040b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 --2- 172.21.15.148:0/374803282 >> v2:172.21.15.188:3300/0 conn(0x7f550415dd30 0x7f55040b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f0003520 con 0x7f55040b9770 2024-04-25T10:01:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756381928 0 0) 0x7f54f0003520 con 0x7f55040b9770 2024-04-25T10:01:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 msgr2=0x7f55040b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 0x7f55040b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 >> v2:172.21.15.188:3300/0 conn(0x7f550415dd30 msgr2=0x7f55040b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 --2- 172.21.15.148:0/374803282 >> v2:172.21.15.188:3300/0 conn(0x7f550415dd30 0x7f55040b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55040d2690 con 0x7f55040b9770 2024-04-25T10:01:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f54f40017f0 con 0x7f55040b9770 2024-04-25T10:01:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 >> v1:172.21.15.148:6789/0 conn(0x7f55040b9770 legacy=0x7f55040d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f54f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f0004800 0x7f54f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 0x7f54f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040d20d0 con 0x7f54f00039b0 2024-04-25T10:01:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040d23b0 con 0x7f54f0004800 2024-04-25T10:01:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f000b9f0 con 0x7f55040d0ad0 2024-04-25T10:01:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f54f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f5508cc3640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f0004800 0x7f54f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f5509cc5640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 0x7f54f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 -- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 msgr2=0x7f54f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55040d0ad0 0x7f54f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 -- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f0004800 msgr2=0x7f54f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f0004800 0x7f54f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f55094c4640 1 -- 172.21.15.148:0/374803282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0003520 con 0x7f54f00039b0 2024-04-25T10:01:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.457+0000 7f5508cc3640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f0004800 0x7f54f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.458+0000 7f55094c4640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f54f0003dd0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5500009fc0 tx=0x7f54f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9b9537a259647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.458+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500009c10 con 0x7f54f00039b0 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.458+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500036070 con 0x7f54f00039b0 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.458+0000 7f54fbfff640 1 -- 172.21.15.148:0/374803282 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550002d070 con 0x7f54f00039b0 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f550b728640 1 -- 172.21.15.148:0/374803282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 msgr2=0x7f54f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f550b728640 1 --2- 172.21.15.148:0/374803282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f54f0003dd0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5500009fc0 tx=0x7f54f0005830 comp rx=0 tx=0).stop 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f5509cc5640 1 -- 172.21.15.148:0/374803282 reap_dead start 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f550b728640 1 -- 172.21.15.148:0/374803282 shutdown_connections 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f550b728640 1 -- 172.21.15.148:0/374803282 >> 172.21.15.148:0/374803282 conn(0x7f5504061c30 msgr2=0x7f550406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f550b728640 1 -- 172.21.15.148:0/374803282 shutdown_connections 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.459+0000 7f550b728640 1 -- 172.21.15.148:0/374803282 wait complete. 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.460+0000 7f550b728640 1 Processor -- start 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.460+0000 7f550b728640 1 -- start start 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.460+0000 7f550b728640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f550415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5504158cc0 0x7f5504150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f550415ff40 0x7f55040aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040d1b90 con 0x7f54f00039b0 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55040d1a10 con 0x7f550415ff40 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55040d2e90 con 0x7f5504158cc0 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f550415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f550415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:59962/0 (socket says 172.21.15.148:59962) 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 -- 172.21.15.148:0/261421707 learned_addr learned my addr 172.21.15.148:0/261421707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f5508cc3640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5504158cc0 0x7f5504150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f5509cc5640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f550415ff40 0x7f55040aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 -- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5504158cc0 msgr2=0x7f5504150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5504158cc0 0x7f5504150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 -- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f550415ff40 msgr2=0x7f55040aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f550415ff40 0x7f55040aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 -- 172.21.15.148:0/261421707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55041591f0 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f55094c4640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f550415a290 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f550002af10 tx=0x7f5500009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500009c10 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5500003bb0 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55041594d0 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500004020 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.461+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5504159a90 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.462+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5504073990 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.463+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5500005d10 con 0x7f54f00039b0 2024-04-25T10:01:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.463+0000 7f54f9ffb640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54e806b4d0 0x7f54e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:13.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.463+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55000ab820 con 0x7f54f00039b0 2024-04-25T10:01:13.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.463+0000 7f5508cc3640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54e806b4d0 0x7f54e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:13.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.464+0000 7f5508cc3640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54e806b4d0 0x7f54e806d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5504159120 tx=0x7f54f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:13.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.468+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f550007a510 con 0x7f54f00039b0 2024-04-25T10:01:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.673+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f55041512d0 con 0x7f54f00039b0 2024-04-25T10:01:13.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.674+0000 7f54f9ffb640 1 -- 172.21.15.148:0/261421707 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5500079e10 con 0x7f54f00039b0 2024-04-25T10:01:13.679 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:01:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.675+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54e806b4d0 msgr2=0x7f54e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54e806b4d0 0x7f54e806d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5504159120 tx=0x7f54f4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 msgr2=0x7f550415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:13.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f550415a290 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f550002af10 tx=0x7f5500009750 comp rx=0 tx=0).stop 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 shutdown_connections 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54e806b4d0 0x7f54e806d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f550415ff40 0x7f55040aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5504158cc0 0x7f5504150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 --2- 172.21.15.148:0/261421707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54f00039b0 0x7f550415a290 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 >> 172.21.15.148:0/261421707 conn(0x7f5504061c30 msgr2=0x7f550415e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 shutdown_connections 2024-04-25T10:01:13.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:13.676+0000 7f550b728640 1 -- 172.21.15.148:0/261421707 wait complete. 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:01:13.692 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:13.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:13.693 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:01:13.708 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:01:13.708 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:01:13.708 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:01:13.708 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.708 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:01:13.708 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42362sec preferred_lft 42362sec 2024-04-25T10:01:13.709 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:01:13.710 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:01:13.711 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:01:13.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:13.711 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:01:13.771 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:01:13.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:13.772 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:01:13.827 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:01:13.828 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:01:13.828 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:01:13.828 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:01:13.828 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:01:13.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:13.828 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:01:13.828 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:13.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:01:13.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:01:13.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:01:13.829 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:13.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:13.912 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:01:13.927 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:01:13.927 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:01:13.927 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:13.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:13.927 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:01:13.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:13.993 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:14.048 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:01:14.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:01:14.048 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:14.102 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:01:14.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:14.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:01:14.183 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:01:14.184 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:01:14.184 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:01:14.192 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:01:14.192 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:01:14.192 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:01:14.193 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:01:14.193 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:01:14.208 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:01:14.209 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:01:14.209 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:01:14.209 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:01:14.209 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:01:14.209 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:01:14.210 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:01:14.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:14.210 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:14.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:14.274 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:01:14.337 INFO:teuthology.orchestra.run.smithi148.stdout:client7030 2024-04-25T10:01:14.338 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:14.338 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:14.393 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:14.393 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T10:01:14.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:14.394 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00991631 s, 106 MB/s 2024-04-25T10:01:14.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.479 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808731 s, 130 MB/s 2024-04-25T10:01:14.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722739 s, 145 MB/s 2024-04-25T10:01:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.495 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673378 s, 156 MB/s 2024-04-25T10:01:14.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778615 s, 135 MB/s 2024-04-25T10:01:14.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.511 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621788 s, 169 MB/s 2024-04-25T10:01:14.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.519 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682547 s, 154 MB/s 2024-04-25T10:01:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836759 s, 125 MB/s 2024-04-25T10:01:14.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.536 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006915 s, 152 MB/s 2024-04-25T10:01:14.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.543 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662292 s, 158 MB/s 2024-04-25T10:01:14.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650817 s, 161 MB/s 2024-04-25T10:01:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580508 s, 181 MB/s 2024-04-25T10:01:14.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.565 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716361 s, 146 MB/s 2024-04-25T10:01:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717257 s, 146 MB/s 2024-04-25T10:01:14.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657918 s, 159 MB/s 2024-04-25T10:01:14.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059678 s, 176 MB/s 2024-04-25T10:01:14.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723155 s, 145 MB/s 2024-04-25T10:01:14.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671178 s, 156 MB/s 2024-04-25T10:01:14.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055927 s, 187 MB/s 2024-04-25T10:01:14.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653353 s, 160 MB/s 2024-04-25T10:01:14.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586368 s, 179 MB/s 2024-04-25T10:01:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696375 s, 151 MB/s 2024-04-25T10:01:14.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733195 s, 143 MB/s 2024-04-25T10:01:14.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.648 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741743 s, 141 MB/s 2024-04-25T10:01:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.656 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642405 s, 163 MB/s 2024-04-25T10:01:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640707 s, 164 MB/s 2024-04-25T10:01:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631257 s, 166 MB/s 2024-04-25T10:01:14.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.678 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675529 s, 155 MB/s 2024-04-25T10:01:14.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.686 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715381 s, 147 MB/s 2024-04-25T10:01:14.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.694 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739784 s, 142 MB/s 2024-04-25T10:01:14.702 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.702 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.702 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669306 s, 157 MB/s 2024-04-25T10:01:14.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.708 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566557 s, 185 MB/s 2024-04-25T10:01:14.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:14.709 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:14.709 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:14.709 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:01:14.710 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:14.710 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:14.710 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:14.710 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:14.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716083 s, 146 MB/s 2024-04-25T10:01:14.782 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.782 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.782 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772257 s, 136 MB/s 2024-04-25T10:01:14.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.789 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590608 s, 178 MB/s 2024-04-25T10:01:14.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.797 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669083 s, 157 MB/s 2024-04-25T10:01:14.804 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.804 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.804 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581902 s, 180 MB/s 2024-04-25T10:01:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.811 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614998 s, 171 MB/s 2024-04-25T10:01:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.819 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718933 s, 146 MB/s 2024-04-25T10:01:14.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.826 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594169 s, 176 MB/s 2024-04-25T10:01:14.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557645 s, 188 MB/s 2024-04-25T10:01:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653526 s, 160 MB/s 2024-04-25T10:01:14.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.847 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624812 s, 168 MB/s 2024-04-25T10:01:14.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629988 s, 166 MB/s 2024-04-25T10:01:14.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643458 s, 163 MB/s 2024-04-25T10:01:14.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706004 s, 149 MB/s 2024-04-25T10:01:14.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647592 s, 162 MB/s 2024-04-25T10:01:14.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.884 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006673 s, 157 MB/s 2024-04-25T10:01:14.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.892 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739919 s, 142 MB/s 2024-04-25T10:01:14.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.900 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.900 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663335 s, 158 MB/s 2024-04-25T10:01:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.907 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621519 s, 169 MB/s 2024-04-25T10:01:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578206 s, 181 MB/s 2024-04-25T10:01:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598938 s, 175 MB/s 2024-04-25T10:01:14.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.928 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641051 s, 164 MB/s 2024-04-25T10:01:14.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.935 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623412 s, 168 MB/s 2024-04-25T10:01:14.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670623 s, 156 MB/s 2024-04-25T10:01:14.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.949 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619958 s, 169 MB/s 2024-04-25T10:01:14.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641091 s, 164 MB/s 2024-04-25T10:01:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616178 s, 170 MB/s 2024-04-25T10:01:14.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.970 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624296 s, 168 MB/s 2024-04-25T10:01:14.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646125 s, 162 MB/s 2024-04-25T10:01:14.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683808 s, 153 MB/s 2024-04-25T10:01:14.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:14.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:14.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712032 s, 147 MB/s 2024-04-25T10:01:15.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636177 s, 165 MB/s 2024-04-25T10:01:15.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:15.002 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:15.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588524 s, 178 MB/s 2024-04-25T10:01:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063538 s, 165 MB/s 2024-04-25T10:01:15.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062978 s, 166 MB/s 2024-04-25T10:01:15.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.087 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619811 s, 169 MB/s 2024-04-25T10:01:15.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608283 s, 172 MB/s 2024-04-25T10:01:15.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620388 s, 169 MB/s 2024-04-25T10:01:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649718 s, 161 MB/s 2024-04-25T10:01:15.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604672 s, 173 MB/s 2024-04-25T10:01:15.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704533 s, 149 MB/s 2024-04-25T10:01:15.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.130 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581536 s, 180 MB/s 2024-04-25T10:01:15.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.137 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612551 s, 171 MB/s 2024-04-25T10:01:15.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.144 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643506 s, 163 MB/s 2024-04-25T10:01:15.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.152 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694847 s, 151 MB/s 2024-04-25T10:01:15.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617144 s, 170 MB/s 2024-04-25T10:01:15.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663475 s, 158 MB/s 2024-04-25T10:01:15.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.175 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712516 s, 147 MB/s 2024-04-25T10:01:15.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.182 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626839 s, 167 MB/s 2024-04-25T10:01:15.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070034 s, 150 MB/s 2024-04-25T10:01:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.197 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575977 s, 182 MB/s 2024-04-25T10:01:15.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.204 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635252 s, 165 MB/s 2024-04-25T10:01:15.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.211 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609355 s, 172 MB/s 2024-04-25T10:01:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720456 s, 146 MB/s 2024-04-25T10:01:15.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586752 s, 179 MB/s 2024-04-25T10:01:15.234 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.234 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.234 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718851 s, 146 MB/s 2024-04-25T10:01:15.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578816 s, 181 MB/s 2024-04-25T10:01:15.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064154 s, 163 MB/s 2024-04-25T10:01:15.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056274 s, 186 MB/s 2024-04-25T10:01:15.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705458 s, 149 MB/s 2024-04-25T10:01:15.271 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.271 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.271 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787324 s, 133 MB/s 2024-04-25T10:01:15.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615453 s, 170 MB/s 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690689 s, 152 MB/s 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657142 s, 160 MB/s 2024-04-25T10:01:15.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:15.295 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.361 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739313 s, 142 MB/s 2024-04-25T10:01:15.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.368 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625145 s, 168 MB/s 2024-04-25T10:01:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.375 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623124 s, 168 MB/s 2024-04-25T10:01:15.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.383 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710582 s, 148 MB/s 2024-04-25T10:01:15.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.390 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633403 s, 166 MB/s 2024-04-25T10:01:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681952 s, 154 MB/s 2024-04-25T10:01:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635199 s, 165 MB/s 2024-04-25T10:01:15.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596421 s, 176 MB/s 2024-04-25T10:01:15.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.420 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731882 s, 143 MB/s 2024-04-25T10:01:15.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.427 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610272 s, 172 MB/s 2024-04-25T10:01:15.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.434 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631481 s, 166 MB/s 2024-04-25T10:01:15.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.442 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696452 s, 151 MB/s 2024-04-25T10:01:15.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581934 s, 180 MB/s 2024-04-25T10:01:15.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.456 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642259 s, 163 MB/s 2024-04-25T10:01:15.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.464 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736212 s, 142 MB/s 2024-04-25T10:01:15.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.472 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703212 s, 149 MB/s 2024-04-25T10:01:15.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.480 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691612 s, 152 MB/s 2024-04-25T10:01:15.486 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.486 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576988 s, 182 MB/s 2024-04-25T10:01:15.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.493 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567875 s, 185 MB/s 2024-04-25T10:01:15.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.501 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711856 s, 147 MB/s 2024-04-25T10:01:15.508 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.508 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.509 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642466 s, 163 MB/s 2024-04-25T10:01:15.516 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.516 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.516 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659727 s, 159 MB/s 2024-04-25T10:01:15.523 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.523 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.523 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602693 s, 174 MB/s 2024-04-25T10:01:15.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.530 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662493 s, 158 MB/s 2024-04-25T10:01:15.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.537 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579919 s, 181 MB/s 2024-04-25T10:01:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578254 s, 181 MB/s 2024-04-25T10:01:15.552 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.552 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.552 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736259 s, 142 MB/s 2024-04-25T10:01:15.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.560 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735601 s, 143 MB/s 2024-04-25T10:01:15.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553949 s, 189 MB/s 2024-04-25T10:01:15.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.574 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060659 s, 173 MB/s 2024-04-25T10:01:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644976 s, 163 MB/s 2024-04-25T10:01:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625472 s, 168 MB/s 2024-04-25T10:01:15.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:15.589 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:15.589 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:15.590 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:01:15.590 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:15.590 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:15.590 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:15.590 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:15.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.654 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646194 s, 162 MB/s 2024-04-25T10:01:15.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.662 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722191 s, 145 MB/s 2024-04-25T10:01:15.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686711 s, 153 MB/s 2024-04-25T10:01:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649137 s, 162 MB/s 2024-04-25T10:01:15.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.685 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007376 s, 142 MB/s 2024-04-25T10:01:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.693 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712957 s, 147 MB/s 2024-04-25T10:01:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579363 s, 181 MB/s 2024-04-25T10:01:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055207 s, 190 MB/s 2024-04-25T10:01:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.714 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719456 s, 146 MB/s 2024-04-25T10:01:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.721 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645485 s, 162 MB/s 2024-04-25T10:01:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610773 s, 172 MB/s 2024-04-25T10:01:15.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.736 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699832 s, 150 MB/s 2024-04-25T10:01:15.743 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654925 s, 160 MB/s 2024-04-25T10:01:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625122 s, 168 MB/s 2024-04-25T10:01:15.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.758 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634891 s, 165 MB/s 2024-04-25T10:01:15.765 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.765 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.765 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640435 s, 164 MB/s 2024-04-25T10:01:15.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.772 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620728 s, 169 MB/s 2024-04-25T10:01:15.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.779 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602764 s, 174 MB/s 2024-04-25T10:01:15.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.787 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638418 s, 164 MB/s 2024-04-25T10:01:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557951 s, 188 MB/s 2024-04-25T10:01:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.801 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708595 s, 148 MB/s 2024-04-25T10:01:15.809 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.809 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.809 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688394 s, 152 MB/s 2024-04-25T10:01:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621611 s, 169 MB/s 2024-04-25T10:01:15.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.824 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702262 s, 149 MB/s 2024-04-25T10:01:15.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.831 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064847 s, 162 MB/s 2024-04-25T10:01:15.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583896 s, 180 MB/s 2024-04-25T10:01:15.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.846 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704989 s, 149 MB/s 2024-04-25T10:01:15.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701959 s, 149 MB/s 2024-04-25T10:01:15.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617915 s, 170 MB/s 2024-04-25T10:01:15.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724626 s, 145 MB/s 2024-04-25T10:01:15.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618176 s, 170 MB/s 2024-04-25T10:01:15.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588733 s, 178 MB/s 2024-04-25T10:01:15.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:15.884 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:15.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695409 s, 151 MB/s 2024-04-25T10:01:15.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614167 s, 171 MB/s 2024-04-25T10:01:15.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.964 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625997 s, 168 MB/s 2024-04-25T10:01:15.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.970 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618219 s, 170 MB/s 2024-04-25T10:01:15.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612704 s, 171 MB/s 2024-04-25T10:01:15.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652043 s, 161 MB/s 2024-04-25T10:01:15.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580572 s, 181 MB/s 2024-04-25T10:01:15.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:15.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701921 s, 149 MB/s 2024-04-25T10:01:16.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.006 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595352 s, 176 MB/s 2024-04-25T10:01:16.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.014 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569623 s, 184 MB/s 2024-04-25T10:01:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.020 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626431 s, 167 MB/s 2024-04-25T10:01:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678561 s, 155 MB/s 2024-04-25T10:01:16.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.036 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069233 s, 151 MB/s 2024-04-25T10:01:16.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.043 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621723 s, 169 MB/s 2024-04-25T10:01:16.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545299 s, 192 MB/s 2024-04-25T10:01:16.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.057 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673405 s, 156 MB/s 2024-04-25T10:01:16.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.064 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063516 s, 165 MB/s 2024-04-25T10:01:16.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669512 s, 157 MB/s 2024-04-25T10:01:16.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616553 s, 170 MB/s 2024-04-25T10:01:16.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.087 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715287 s, 147 MB/s 2024-04-25T10:01:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639579 s, 164 MB/s 2024-04-25T10:01:16.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644095 s, 163 MB/s 2024-04-25T10:01:16.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575411 s, 182 MB/s 2024-04-25T10:01:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562345 s, 186 MB/s 2024-04-25T10:01:16.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625891 s, 168 MB/s 2024-04-25T10:01:16.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.130 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694656 s, 151 MB/s 2024-04-25T10:01:16.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.137 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651584 s, 161 MB/s 2024-04-25T10:01:16.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.144 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614338 s, 171 MB/s 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624667 s, 168 MB/s 2024-04-25T10:01:16.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00543466 s, 193 MB/s 2024-04-25T10:01:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575867 s, 182 MB/s 2024-04-25T10:01:16.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.171 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593613 s, 177 MB/s 2024-04-25T10:01:16.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:16.172 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:16.172 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:16.172 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:01:16.173 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:16.173 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:16.173 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:16.173 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.236 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569385 s, 184 MB/s 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.244 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711611 s, 147 MB/s 2024-04-25T10:01:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.251 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570077 s, 184 MB/s 2024-04-25T10:01:16.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.288 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363634 s, 28.8 MB/s 2024-04-25T10:01:16.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.296 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752177 s, 139 MB/s 2024-04-25T10:01:16.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.303 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591593 s, 177 MB/s 2024-04-25T10:01:16.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.311 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662568 s, 158 MB/s 2024-04-25T10:01:16.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.319 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681216 s, 154 MB/s 2024-04-25T10:01:16.325 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.325 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.325 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614383 s, 171 MB/s 2024-04-25T10:01:16.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.333 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668041 s, 157 MB/s 2024-04-25T10:01:16.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.340 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593848 s, 177 MB/s 2024-04-25T10:01:16.347 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.347 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.347 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632279 s, 166 MB/s 2024-04-25T10:01:16.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.354 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610112 s, 172 MB/s 2024-04-25T10:01:16.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.362 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686189 s, 153 MB/s 2024-04-25T10:01:16.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.370 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072766 s, 144 MB/s 2024-04-25T10:01:16.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.378 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733864 s, 143 MB/s 2024-04-25T10:01:16.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.386 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635717 s, 165 MB/s 2024-04-25T10:01:16.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633137 s, 166 MB/s 2024-04-25T10:01:16.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703959 s, 149 MB/s 2024-04-25T10:01:16.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072491 s, 145 MB/s 2024-04-25T10:01:16.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073839 s, 142 MB/s 2024-04-25T10:01:16.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620551 s, 169 MB/s 2024-04-25T10:01:16.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578811 s, 181 MB/s 2024-04-25T10:01:16.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.437 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571444 s, 183 MB/s 2024-04-25T10:01:16.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.444 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581146 s, 180 MB/s 2024-04-25T10:01:16.451 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.451 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.452 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623845 s, 168 MB/s 2024-04-25T10:01:16.458 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.458 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.458 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584207 s, 179 MB/s 2024-04-25T10:01:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.465 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056203 s, 187 MB/s 2024-04-25T10:01:16.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.472 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675094 s, 155 MB/s 2024-04-25T10:01:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.479 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556207 s, 189 MB/s 2024-04-25T10:01:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754373 s, 139 MB/s 2024-04-25T10:01:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.494 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586938 s, 179 MB/s 2024-04-25T10:01:16.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:16.495 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:16.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.560 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617354 s, 170 MB/s 2024-04-25T10:01:16.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642524 s, 163 MB/s 2024-04-25T10:01:16.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.574 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657416 s, 159 MB/s 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621555 s, 169 MB/s 2024-04-25T10:01:16.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606251 s, 173 MB/s 2024-04-25T10:01:16.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.595 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639176 s, 164 MB/s 2024-04-25T10:01:16.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.602 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584666 s, 179 MB/s 2024-04-25T10:01:16.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729395 s, 144 MB/s 2024-04-25T10:01:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715247 s, 147 MB/s 2024-04-25T10:01:16.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.627 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746137 s, 141 MB/s 2024-04-25T10:01:16.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576577 s, 182 MB/s 2024-04-25T10:01:16.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.641 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710899 s, 148 MB/s 2024-04-25T10:01:16.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.649 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634941 s, 165 MB/s 2024-04-25T10:01:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.657 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744238 s, 141 MB/s 2024-04-25T10:01:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584916 s, 179 MB/s 2024-04-25T10:01:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.671 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641949 s, 163 MB/s 2024-04-25T10:01:16.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.678 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614958 s, 171 MB/s 2024-04-25T10:01:16.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.685 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572404 s, 183 MB/s 2024-04-25T10:01:16.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.691 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557861 s, 188 MB/s 2024-04-25T10:01:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713355 s, 147 MB/s 2024-04-25T10:01:16.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612749 s, 171 MB/s 2024-04-25T10:01:16.715 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.715 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.715 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746561 s, 140 MB/s 2024-04-25T10:01:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.721 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058079 s, 181 MB/s 2024-04-25T10:01:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.729 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629398 s, 167 MB/s 2024-04-25T10:01:16.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727426 s, 144 MB/s 2024-04-25T10:01:16.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063818 s, 164 MB/s 2024-04-25T10:01:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640751 s, 164 MB/s 2024-04-25T10:01:16.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.758 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582487 s, 180 MB/s 2024-04-25T10:01:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689414 s, 152 MB/s 2024-04-25T10:01:16.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.773 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620634 s, 169 MB/s 2024-04-25T10:01:16.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633637 s, 165 MB/s 2024-04-25T10:01:16.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.789 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752043 s, 139 MB/s 2024-04-25T10:01:16.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:16.790 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:16.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.855 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630228 s, 166 MB/s 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.862 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063559 s, 165 MB/s 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061457 s, 171 MB/s 2024-04-25T10:01:16.877 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.877 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.877 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615414 s, 170 MB/s 2024-04-25T10:01:16.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632417 s, 166 MB/s 2024-04-25T10:01:16.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628108 s, 167 MB/s 2024-04-25T10:01:16.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620776 s, 169 MB/s 2024-04-25T10:01:16.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062355 s, 168 MB/s 2024-04-25T10:01:16.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613286 s, 171 MB/s 2024-04-25T10:01:16.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.918 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644321 s, 163 MB/s 2024-04-25T10:01:16.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.926 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617617 s, 170 MB/s 2024-04-25T10:01:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.932 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554113 s, 189 MB/s 2024-04-25T10:01:16.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.939 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570647 s, 184 MB/s 2024-04-25T10:01:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548819 s, 191 MB/s 2024-04-25T10:01:16.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694938 s, 151 MB/s 2024-04-25T10:01:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072335 s, 145 MB/s 2024-04-25T10:01:16.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576606 s, 182 MB/s 2024-04-25T10:01:16.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00539525 s, 194 MB/s 2024-04-25T10:01:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722497 s, 145 MB/s 2024-04-25T10:01:16.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.990 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.990 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645858 s, 162 MB/s 2024-04-25T10:01:16.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:16.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:16.997 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063065 s, 166 MB/s 2024-04-25T10:01:17.005 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.005 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.005 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071471 s, 147 MB/s 2024-04-25T10:01:17.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.012 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639256 s, 164 MB/s 2024-04-25T10:01:17.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630011 s, 166 MB/s 2024-04-25T10:01:17.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.027 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665267 s, 158 MB/s 2024-04-25T10:01:17.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616022 s, 170 MB/s 2024-04-25T10:01:17.040 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.040 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615932 s, 170 MB/s 2024-04-25T10:01:17.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.048 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628161 s, 167 MB/s 2024-04-25T10:01:17.054 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.054 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.054 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588399 s, 178 MB/s 2024-04-25T10:01:17.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.062 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642498 s, 163 MB/s 2024-04-25T10:01:17.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.069 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632576 s, 166 MB/s 2024-04-25T10:01:17.076 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.076 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.076 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642771 s, 163 MB/s 2024-04-25T10:01:17.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:17.077 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:17.077 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:17.078 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:01:17.078 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:17.078 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:17.078 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:17.078 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632 s, 166 MB/s 2024-04-25T10:01:17.148 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.148 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.149 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635452 s, 165 MB/s 2024-04-25T10:01:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.156 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626682 s, 167 MB/s 2024-04-25T10:01:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.162 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056289 s, 186 MB/s 2024-04-25T10:01:17.170 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.170 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.170 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713838 s, 147 MB/s 2024-04-25T10:01:17.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.177 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553914 s, 189 MB/s 2024-04-25T10:01:17.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066151 s, 159 MB/s 2024-04-25T10:01:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574117 s, 183 MB/s 2024-04-25T10:01:17.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616858 s, 170 MB/s 2024-04-25T10:01:17.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660421 s, 159 MB/s 2024-04-25T10:01:17.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.213 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621897 s, 169 MB/s 2024-04-25T10:01:17.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.221 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703002 s, 149 MB/s 2024-04-25T10:01:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.227 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584498 s, 179 MB/s 2024-04-25T10:01:17.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722361 s, 145 MB/s 2024-04-25T10:01:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619898 s, 169 MB/s 2024-04-25T10:01:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720086 s, 146 MB/s 2024-04-25T10:01:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558368 s, 188 MB/s 2024-04-25T10:01:17.264 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.264 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.264 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594387 s, 176 MB/s 2024-04-25T10:01:17.271 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.271 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.271 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592598 s, 177 MB/s 2024-04-25T10:01:17.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631209 s, 166 MB/s 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736811 s, 142 MB/s 2024-04-25T10:01:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717759 s, 146 MB/s 2024-04-25T10:01:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593184 s, 177 MB/s 2024-04-25T10:01:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062854 s, 167 MB/s 2024-04-25T10:01:17.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.316 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640412 s, 164 MB/s 2024-04-25T10:01:17.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603235 s, 174 MB/s 2024-04-25T10:01:17.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624635 s, 168 MB/s 2024-04-25T10:01:17.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.338 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072269 s, 145 MB/s 2024-04-25T10:01:17.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.345 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597147 s, 176 MB/s 2024-04-25T10:01:17.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.352 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552015 s, 190 MB/s 2024-04-25T10:01:17.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596589 s, 176 MB/s 2024-04-25T10:01:17.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641663 s, 163 MB/s 2024-04-25T10:01:17.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:17.367 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:17.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580544 s, 181 MB/s 2024-04-25T10:01:17.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061039 s, 172 MB/s 2024-04-25T10:01:17.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.446 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611083 s, 172 MB/s 2024-04-25T10:01:17.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586034 s, 179 MB/s 2024-04-25T10:01:17.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629991 s, 166 MB/s 2024-04-25T10:01:17.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062911 s, 167 MB/s 2024-04-25T10:01:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658926 s, 159 MB/s 2024-04-25T10:01:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.481 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622312 s, 168 MB/s 2024-04-25T10:01:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.489 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645055 s, 163 MB/s 2024-04-25T10:01:17.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.497 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720877 s, 145 MB/s 2024-04-25T10:01:17.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713122 s, 147 MB/s 2024-04-25T10:01:17.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732422 s, 143 MB/s 2024-04-25T10:01:17.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706936 s, 148 MB/s 2024-04-25T10:01:17.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626706 s, 167 MB/s 2024-04-25T10:01:17.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058218 s, 180 MB/s 2024-04-25T10:01:17.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.542 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649096 s, 162 MB/s 2024-04-25T10:01:17.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.551 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770716 s, 136 MB/s 2024-04-25T10:01:17.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.559 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687103 s, 153 MB/s 2024-04-25T10:01:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573758 s, 183 MB/s 2024-04-25T10:01:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593339 s, 177 MB/s 2024-04-25T10:01:17.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.578 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545333 s, 192 MB/s 2024-04-25T10:01:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695646 s, 151 MB/s 2024-04-25T10:01:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577484 s, 182 MB/s 2024-04-25T10:01:17.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.601 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702791 s, 149 MB/s 2024-04-25T10:01:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718041 s, 146 MB/s 2024-04-25T10:01:17.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.616 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638308 s, 164 MB/s 2024-04-25T10:01:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631392 s, 166 MB/s 2024-04-25T10:01:17.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.630 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574727 s, 182 MB/s 2024-04-25T10:01:17.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.637 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631096 s, 166 MB/s 2024-04-25T10:01:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665375 s, 158 MB/s 2024-04-25T10:01:17.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611515 s, 171 MB/s 2024-04-25T10:01:17.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618169 s, 170 MB/s 2024-04-25T10:01:17.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:17.660 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:17.660 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:17.660 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:01:17.660 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:17.660 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:17.660 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:17.661 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.725 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630178 s, 166 MB/s 2024-04-25T10:01:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.733 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730677 s, 144 MB/s 2024-04-25T10:01:17.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.740 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644114 s, 163 MB/s 2024-04-25T10:01:17.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.747 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593816 s, 177 MB/s 2024-04-25T10:01:17.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.754 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636375 s, 165 MB/s 2024-04-25T10:01:17.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.761 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601947 s, 174 MB/s 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648038 s, 162 MB/s 2024-04-25T10:01:17.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.775 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628488 s, 167 MB/s 2024-04-25T10:01:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.783 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063568 s, 165 MB/s 2024-04-25T10:01:17.790 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.790 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.790 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643744 s, 163 MB/s 2024-04-25T10:01:17.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.797 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657762 s, 159 MB/s 2024-04-25T10:01:17.804 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.805 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632327 s, 166 MB/s 2024-04-25T10:01:17.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.812 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644963 s, 163 MB/s 2024-04-25T10:01:17.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.819 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641863 s, 163 MB/s 2024-04-25T10:01:17.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.826 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610157 s, 172 MB/s 2024-04-25T10:01:17.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608552 s, 172 MB/s 2024-04-25T10:01:17.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056001 s, 187 MB/s 2024-04-25T10:01:17.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.846 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556133 s, 189 MB/s 2024-04-25T10:01:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630879 s, 166 MB/s 2024-04-25T10:01:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634383 s, 165 MB/s 2024-04-25T10:01:17.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675488 s, 155 MB/s 2024-04-25T10:01:17.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643993 s, 163 MB/s 2024-04-25T10:01:17.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.884 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715679 s, 147 MB/s 2024-04-25T10:01:17.891 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.891 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.891 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642941 s, 163 MB/s 2024-04-25T10:01:17.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062124 s, 169 MB/s 2024-04-25T10:01:17.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691548 s, 152 MB/s 2024-04-25T10:01:17.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635815 s, 165 MB/s 2024-04-25T10:01:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707962 s, 148 MB/s 2024-04-25T10:01:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.928 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620287 s, 169 MB/s 2024-04-25T10:01:17.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.935 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572435 s, 183 MB/s 2024-04-25T10:01:17.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647136 s, 162 MB/s 2024-04-25T10:01:17.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:17.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:17.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721408 s, 145 MB/s 2024-04-25T10:01:17.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:17.951 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:17.951 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:17.952 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:01:17.952 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:17.952 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:17.952 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:17.952 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:18.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728881 s, 144 MB/s 2024-04-25T10:01:18.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.024 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.024 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687658 s, 152 MB/s 2024-04-25T10:01:18.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.032 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705832 s, 149 MB/s 2024-04-25T10:01:18.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.039 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638367 s, 164 MB/s 2024-04-25T10:01:18.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574039 s, 183 MB/s 2024-04-25T10:01:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597986 s, 175 MB/s 2024-04-25T10:01:18.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.060 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716582 s, 146 MB/s 2024-04-25T10:01:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.067 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564279 s, 186 MB/s 2024-04-25T10:01:18.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.074 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573072 s, 183 MB/s 2024-04-25T10:01:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565727 s, 185 MB/s 2024-04-25T10:01:18.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.088 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630955 s, 166 MB/s 2024-04-25T10:01:18.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604986 s, 173 MB/s 2024-04-25T10:01:18.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623473 s, 168 MB/s 2024-04-25T10:01:18.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619576 s, 169 MB/s 2024-04-25T10:01:18.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.116 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702557 s, 149 MB/s 2024-04-25T10:01:18.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.124 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657105 s, 160 MB/s 2024-04-25T10:01:18.132 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.132 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.132 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708019 s, 148 MB/s 2024-04-25T10:01:18.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.139 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646515 s, 162 MB/s 2024-04-25T10:01:18.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.146 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649536 s, 161 MB/s 2024-04-25T10:01:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.153 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584961 s, 179 MB/s 2024-04-25T10:01:18.160 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.160 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.161 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649222 s, 162 MB/s 2024-04-25T10:01:18.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622098 s, 169 MB/s 2024-04-25T10:01:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592036 s, 177 MB/s 2024-04-25T10:01:18.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.182 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661007 s, 159 MB/s 2024-04-25T10:01:18.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.189 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636939 s, 165 MB/s 2024-04-25T10:01:18.197 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.197 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.197 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699107 s, 150 MB/s 2024-04-25T10:01:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612937 s, 171 MB/s 2024-04-25T10:01:18.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714982 s, 147 MB/s 2024-04-25T10:01:18.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.219 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613241 s, 171 MB/s 2024-04-25T10:01:18.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061087 s, 172 MB/s 2024-04-25T10:01:18.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.233 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631155 s, 166 MB/s 2024-04-25T10:01:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714171 s, 147 MB/s 2024-04-25T10:01:18.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:18.242 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:18.242 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:18.242 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:01:18.242 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:18.242 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:18.243 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:18.243 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:18.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677771 s, 155 MB/s 2024-04-25T10:01:18.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650512 s, 161 MB/s 2024-04-25T10:01:18.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728396 s, 144 MB/s 2024-04-25T10:01:18.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.331 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072598 s, 144 MB/s 2024-04-25T10:01:18.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697218 s, 150 MB/s 2024-04-25T10:01:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617117 s, 170 MB/s 2024-04-25T10:01:18.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.353 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649232 s, 162 MB/s 2024-04-25T10:01:18.360 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.360 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.360 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567767 s, 185 MB/s 2024-04-25T10:01:18.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562707 s, 186 MB/s 2024-04-25T10:01:18.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.374 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602947 s, 174 MB/s 2024-04-25T10:01:18.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576328 s, 182 MB/s 2024-04-25T10:01:18.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635708 s, 165 MB/s 2024-04-25T10:01:18.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558439 s, 188 MB/s 2024-04-25T10:01:18.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.402 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734857 s, 143 MB/s 2024-04-25T10:01:18.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.410 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718131 s, 146 MB/s 2024-04-25T10:01:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.418 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708236 s, 148 MB/s 2024-04-25T10:01:18.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.426 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674851 s, 155 MB/s 2024-04-25T10:01:18.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.434 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702675 s, 149 MB/s 2024-04-25T10:01:18.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.440 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605059 s, 173 MB/s 2024-04-25T10:01:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618366 s, 170 MB/s 2024-04-25T10:01:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067657 s, 155 MB/s 2024-04-25T10:01:18.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626703 s, 167 MB/s 2024-04-25T10:01:18.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.470 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070709 s, 148 MB/s 2024-04-25T10:01:18.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.477 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561001 s, 187 MB/s 2024-04-25T10:01:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630032 s, 166 MB/s 2024-04-25T10:01:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625295 s, 168 MB/s 2024-04-25T10:01:18.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630304 s, 166 MB/s 2024-04-25T10:01:18.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733829 s, 143 MB/s 2024-04-25T10:01:18.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595135 s, 176 MB/s 2024-04-25T10:01:18.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.522 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742939 s, 141 MB/s 2024-04-25T10:01:18.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.529 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614553 s, 171 MB/s 2024-04-25T10:01:18.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582032 s, 180 MB/s 2024-04-25T10:01:18.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:18.536 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:18.537 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:18.537 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:01:18.537 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:18.537 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:18.537 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:18.537 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:18.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674499 s, 155 MB/s 2024-04-25T10:01:18.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613738 s, 171 MB/s 2024-04-25T10:01:18.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718319 s, 146 MB/s 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.627 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776304 s, 135 MB/s 2024-04-25T10:01:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.636 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791567 s, 132 MB/s 2024-04-25T10:01:18.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.643 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633783 s, 165 MB/s 2024-04-25T10:01:18.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.650 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620206 s, 169 MB/s 2024-04-25T10:01:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.658 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629524 s, 167 MB/s 2024-04-25T10:01:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.666 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745882 s, 141 MB/s 2024-04-25T10:01:18.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660367 s, 159 MB/s 2024-04-25T10:01:18.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.681 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683665 s, 153 MB/s 2024-04-25T10:01:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.690 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728614 s, 144 MB/s 2024-04-25T10:01:18.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.698 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705137 s, 149 MB/s 2024-04-25T10:01:18.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.705 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059825 s, 175 MB/s 2024-04-25T10:01:18.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717841 s, 146 MB/s 2024-04-25T10:01:18.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.720 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650302 s, 161 MB/s 2024-04-25T10:01:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704418 s, 149 MB/s 2024-04-25T10:01:18.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.736 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066344 s, 158 MB/s 2024-04-25T10:01:18.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.743 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.743 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630622 s, 166 MB/s 2024-04-25T10:01:18.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.750 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624158 s, 168 MB/s 2024-04-25T10:01:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.756 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565513 s, 185 MB/s 2024-04-25T10:01:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.763 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607313 s, 173 MB/s 2024-04-25T10:01:18.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612738 s, 171 MB/s 2024-04-25T10:01:18.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631702 s, 166 MB/s 2024-04-25T10:01:18.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.785 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645565 s, 162 MB/s 2024-04-25T10:01:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782169 s, 134 MB/s 2024-04-25T10:01:18.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574905 s, 182 MB/s 2024-04-25T10:01:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619081 s, 169 MB/s 2024-04-25T10:01:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571204 s, 184 MB/s 2024-04-25T10:01:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599681 s, 175 MB/s 2024-04-25T10:01:18.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.828 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615263 s, 170 MB/s 2024-04-25T10:01:18.835 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.835 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.835 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644546 s, 163 MB/s 2024-04-25T10:01:18.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:18.836 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:18.837 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:18.837 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:01:18.837 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:18.837 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:18.837 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:18.837 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:18.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.902 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721272 s, 145 MB/s 2024-04-25T10:01:18.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.910 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729221 s, 144 MB/s 2024-04-25T10:01:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.917 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611449 s, 171 MB/s 2024-04-25T10:01:18.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.924 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617654 s, 170 MB/s 2024-04-25T10:01:18.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.931 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622546 s, 168 MB/s 2024-04-25T10:01:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.938 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559185 s, 188 MB/s 2024-04-25T10:01:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.945 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620688 s, 169 MB/s 2024-04-25T10:01:18.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071521 s, 147 MB/s 2024-04-25T10:01:18.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.960 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629107 s, 167 MB/s 2024-04-25T10:01:18.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.967 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583147 s, 180 MB/s 2024-04-25T10:01:18.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.975 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672323 s, 156 MB/s 2024-04-25T10:01:18.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.983 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734203 s, 143 MB/s 2024-04-25T10:01:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733143 s, 143 MB/s 2024-04-25T10:01:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689578 s, 152 MB/s 2024-04-25T10:01:19.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.006 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683694 s, 153 MB/s 2024-04-25T10:01:19.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.014 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630555 s, 166 MB/s 2024-04-25T10:01:19.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727714 s, 144 MB/s 2024-04-25T10:01:19.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.029 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656854 s, 160 MB/s 2024-04-25T10:01:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.036 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642894 s, 163 MB/s 2024-04-25T10:01:19.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.043 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581884 s, 180 MB/s 2024-04-25T10:01:19.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.050 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624954 s, 168 MB/s 2024-04-25T10:01:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594879 s, 176 MB/s 2024-04-25T10:01:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.065 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703918 s, 149 MB/s 2024-04-25T10:01:19.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599841 s, 175 MB/s 2024-04-25T10:01:19.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064592 s, 162 MB/s 2024-04-25T10:01:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560862 s, 187 MB/s 2024-04-25T10:01:19.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.093 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565581 s, 185 MB/s 2024-04-25T10:01:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719545 s, 146 MB/s 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.107 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558861 s, 188 MB/s 2024-04-25T10:01:19.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647311 s, 162 MB/s 2024-04-25T10:01:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067235 s, 156 MB/s 2024-04-25T10:01:19.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587977 s, 178 MB/s 2024-04-25T10:01:19.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:19.130 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:19.130 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:19.130 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:01:19.130 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:19.130 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:19.131 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:19.131 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.195 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722775 s, 145 MB/s 2024-04-25T10:01:19.202 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661017 s, 159 MB/s 2024-04-25T10:01:19.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.210 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592737 s, 177 MB/s 2024-04-25T10:01:19.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725421 s, 145 MB/s 2024-04-25T10:01:19.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.225 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653927 s, 160 MB/s 2024-04-25T10:01:19.231 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.232 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562732 s, 186 MB/s 2024-04-25T10:01:19.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.239 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654693 s, 160 MB/s 2024-04-25T10:01:19.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.247 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663048 s, 158 MB/s 2024-04-25T10:01:19.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634262 s, 165 MB/s 2024-04-25T10:01:19.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617638 s, 170 MB/s 2024-04-25T10:01:19.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655028 s, 160 MB/s 2024-04-25T10:01:19.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.275 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603419 s, 174 MB/s 2024-04-25T10:01:19.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.282 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599341 s, 175 MB/s 2024-04-25T10:01:19.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.289 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612681 s, 171 MB/s 2024-04-25T10:01:19.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.297 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656741 s, 160 MB/s 2024-04-25T10:01:19.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.304 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619722 s, 169 MB/s 2024-04-25T10:01:19.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.311 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634802 s, 165 MB/s 2024-04-25T10:01:19.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.318 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634679 s, 165 MB/s 2024-04-25T10:01:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063493 s, 165 MB/s 2024-04-25T10:01:19.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.333 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602148 s, 174 MB/s 2024-04-25T10:01:19.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.340 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624811 s, 168 MB/s 2024-04-25T10:01:19.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582046 s, 180 MB/s 2024-04-25T10:01:19.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.353 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625329 s, 168 MB/s 2024-04-25T10:01:19.360 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.360 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.361 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616888 s, 170 MB/s 2024-04-25T10:01:19.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.368 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614735 s, 171 MB/s 2024-04-25T10:01:19.374 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.374 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.375 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061054 s, 172 MB/s 2024-04-25T10:01:19.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.382 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715671 s, 147 MB/s 2024-04-25T10:01:19.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.391 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735037 s, 143 MB/s 2024-04-25T10:01:19.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.399 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711016 s, 147 MB/s 2024-04-25T10:01:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580497 s, 181 MB/s 2024-04-25T10:01:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059123 s, 177 MB/s 2024-04-25T10:01:19.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563314 s, 186 MB/s 2024-04-25T10:01:19.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:19.420 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:19.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.485 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642135 s, 163 MB/s 2024-04-25T10:01:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.494 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701673 s, 149 MB/s 2024-04-25T10:01:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.500 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700027 s, 150 MB/s 2024-04-25T10:01:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.507 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574573 s, 182 MB/s 2024-04-25T10:01:19.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587373 s, 179 MB/s 2024-04-25T10:01:19.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616595 s, 170 MB/s 2024-04-25T10:01:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.529 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708566 s, 148 MB/s 2024-04-25T10:01:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.536 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628833 s, 167 MB/s 2024-04-25T10:01:19.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.543 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619297 s, 169 MB/s 2024-04-25T10:01:19.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.549 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570098 s, 184 MB/s 2024-04-25T10:01:19.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651844 s, 161 MB/s 2024-04-25T10:01:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627309 s, 167 MB/s 2024-04-25T10:01:19.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.572 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676336 s, 155 MB/s 2024-04-25T10:01:19.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.579 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617669 s, 170 MB/s 2024-04-25T10:01:19.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.586 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660784 s, 159 MB/s 2024-04-25T10:01:19.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058022 s, 181 MB/s 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642987 s, 163 MB/s 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.607 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602106 s, 174 MB/s 2024-04-25T10:01:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.614 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575837 s, 182 MB/s 2024-04-25T10:01:19.621 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.621 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.621 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639996 s, 164 MB/s 2024-04-25T10:01:19.628 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.629 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.629 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649236 s, 162 MB/s 2024-04-25T10:01:19.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.635 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581304 s, 180 MB/s 2024-04-25T10:01:19.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.643 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643728 s, 163 MB/s 2024-04-25T10:01:19.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.650 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634224 s, 165 MB/s 2024-04-25T10:01:19.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.657 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622472 s, 168 MB/s 2024-04-25T10:01:19.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.664 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620316 s, 169 MB/s 2024-04-25T10:01:19.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.670 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564914 s, 186 MB/s 2024-04-25T10:01:19.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618397 s, 170 MB/s 2024-04-25T10:01:19.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636756 s, 165 MB/s 2024-04-25T10:01:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574647 s, 182 MB/s 2024-04-25T10:01:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594825 s, 176 MB/s 2024-04-25T10:01:19.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637828 s, 164 MB/s 2024-04-25T10:01:19.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:19.707 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667706 s, 157 MB/s 2024-04-25T10:01:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.779 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641095 s, 164 MB/s 2024-04-25T10:01:19.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.788 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077076 s, 136 MB/s 2024-04-25T10:01:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.795 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678867 s, 154 MB/s 2024-04-25T10:01:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.803 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751528 s, 140 MB/s 2024-04-25T10:01:19.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.811 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629385 s, 167 MB/s 2024-04-25T10:01:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.817 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584294 s, 179 MB/s 2024-04-25T10:01:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654179 s, 160 MB/s 2024-04-25T10:01:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644104 s, 163 MB/s 2024-04-25T10:01:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064269 s, 163 MB/s 2024-04-25T10:01:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614854 s, 171 MB/s 2024-04-25T10:01:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621326 s, 169 MB/s 2024-04-25T10:01:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668852 s, 157 MB/s 2024-04-25T10:01:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577162 s, 182 MB/s 2024-04-25T10:01:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628965 s, 167 MB/s 2024-04-25T10:01:19.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.882 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642418 s, 163 MB/s 2024-04-25T10:01:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701543 s, 149 MB/s 2024-04-25T10:01:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607547 s, 173 MB/s 2024-04-25T10:01:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.905 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728229 s, 144 MB/s 2024-04-25T10:01:19.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604528 s, 173 MB/s 2024-04-25T10:01:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.919 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061893 s, 169 MB/s 2024-04-25T10:01:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.926 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627019 s, 167 MB/s 2024-04-25T10:01:19.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.933 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622831 s, 168 MB/s 2024-04-25T10:01:19.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.940 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577038 s, 182 MB/s 2024-04-25T10:01:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654452 s, 160 MB/s 2024-04-25T10:01:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.954 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588682 s, 178 MB/s 2024-04-25T10:01:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.962 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643762 s, 163 MB/s 2024-04-25T10:01:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640347 s, 164 MB/s 2024-04-25T10:01:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070013 s, 150 MB/s 2024-04-25T10:01:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.984 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637379 s, 165 MB/s 2024-04-25T10:01:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.991 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649865 s, 161 MB/s 2024-04-25T10:01:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:19.999 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701755 s, 149 MB/s 2024-04-25T10:01:20.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:20.000 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:20.001 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:20.001 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:01:20.001 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:20.001 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:20.001 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:20.001 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750121 s, 140 MB/s 2024-04-25T10:01:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574957 s, 182 MB/s 2024-04-25T10:01:20.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616409 s, 170 MB/s 2024-04-25T10:01:20.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623522 s, 168 MB/s 2024-04-25T10:01:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701847 s, 149 MB/s 2024-04-25T10:01:20.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642805 s, 163 MB/s 2024-04-25T10:01:20.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622259 s, 169 MB/s 2024-04-25T10:01:20.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622803 s, 168 MB/s 2024-04-25T10:01:20.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.124 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748828 s, 140 MB/s 2024-04-25T10:01:20.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658406 s, 159 MB/s 2024-04-25T10:01:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687717 s, 152 MB/s 2024-04-25T10:01:20.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.147 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749146 s, 140 MB/s 2024-04-25T10:01:20.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.154 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624003 s, 168 MB/s 2024-04-25T10:01:20.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.161 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598085 s, 175 MB/s 2024-04-25T10:01:20.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555279 s, 189 MB/s 2024-04-25T10:01:20.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.175 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611732 s, 171 MB/s 2024-04-25T10:01:20.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580922 s, 181 MB/s 2024-04-25T10:01:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.194 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113192 s, 92.6 MB/s 2024-04-25T10:01:20.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.201 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655325 s, 160 MB/s 2024-04-25T10:01:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.208 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629776 s, 166 MB/s 2024-04-25T10:01:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608908 s, 172 MB/s 2024-04-25T10:01:20.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559729 s, 187 MB/s 2024-04-25T10:01:20.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.229 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625184 s, 168 MB/s 2024-04-25T10:01:20.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735155 s, 143 MB/s 2024-04-25T10:01:20.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.244 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658542 s, 159 MB/s 2024-04-25T10:01:20.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.252 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731076 s, 143 MB/s 2024-04-25T10:01:20.259 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.260 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628679 s, 167 MB/s 2024-04-25T10:01:20.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.267 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.267 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613151 s, 171 MB/s 2024-04-25T10:01:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622324 s, 168 MB/s 2024-04-25T10:01:20.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.281 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628264 s, 167 MB/s 2024-04-25T10:01:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645061 s, 163 MB/s 2024-04-25T10:01:20.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058881 s, 178 MB/s 2024-04-25T10:01:20.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:20.296 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:20.296 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:20.296 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:01:20.296 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:20.296 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:20.296 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:20.297 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:20.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.362 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763086 s, 137 MB/s 2024-04-25T10:01:20.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.369 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061763 s, 170 MB/s 2024-04-25T10:01:20.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.376 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683046 s, 154 MB/s 2024-04-25T10:01:20.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.384 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722465 s, 145 MB/s 2024-04-25T10:01:20.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.391 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629713 s, 167 MB/s 2024-04-25T10:01:20.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.398 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596006 s, 176 MB/s 2024-04-25T10:01:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589391 s, 178 MB/s 2024-04-25T10:01:20.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626329 s, 167 MB/s 2024-04-25T10:01:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.420 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640814 s, 164 MB/s 2024-04-25T10:01:20.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.427 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676576 s, 155 MB/s 2024-04-25T10:01:20.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.435 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071468 s, 147 MB/s 2024-04-25T10:01:20.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.442 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614758 s, 171 MB/s 2024-04-25T10:01:20.449 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.449 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.449 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619117 s, 169 MB/s 2024-04-25T10:01:20.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.456 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627291 s, 167 MB/s 2024-04-25T10:01:20.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.463 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573694 s, 183 MB/s 2024-04-25T10:01:20.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.470 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642523 s, 163 MB/s 2024-04-25T10:01:20.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.478 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641112 s, 164 MB/s 2024-04-25T10:01:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.484 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587829 s, 178 MB/s 2024-04-25T10:01:20.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563403 s, 186 MB/s 2024-04-25T10:01:20.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667715 s, 157 MB/s 2024-04-25T10:01:20.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713834 s, 147 MB/s 2024-04-25T10:01:20.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558819 s, 188 MB/s 2024-04-25T10:01:20.520 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.520 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.520 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659225 s, 159 MB/s 2024-04-25T10:01:20.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.529 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713674 s, 147 MB/s 2024-04-25T10:01:20.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.537 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070512 s, 149 MB/s 2024-04-25T10:01:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632685 s, 166 MB/s 2024-04-25T10:01:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.551 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687207 s, 153 MB/s 2024-04-25T10:01:20.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.559 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663467 s, 158 MB/s 2024-04-25T10:01:20.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.566 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571622 s, 183 MB/s 2024-04-25T10:01:20.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.572 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567487 s, 185 MB/s 2024-04-25T10:01:20.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.580 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659504 s, 159 MB/s 2024-04-25T10:01:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.587 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640783 s, 164 MB/s 2024-04-25T10:01:20.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:20.588 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:20.588 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:20.588 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:01:20.588 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:20.588 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:20.588 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:20.589 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:20.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637503 s, 164 MB/s 2024-04-25T10:01:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589355 s, 178 MB/s 2024-04-25T10:01:20.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.667 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696608 s, 151 MB/s 2024-04-25T10:01:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626342 s, 167 MB/s 2024-04-25T10:01:20.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.681 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632916 s, 166 MB/s 2024-04-25T10:01:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570321 s, 184 MB/s 2024-04-25T10:01:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069513 s, 151 MB/s 2024-04-25T10:01:20.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.703 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646458 s, 162 MB/s 2024-04-25T10:01:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.710 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581308 s, 180 MB/s 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.717 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589719 s, 178 MB/s 2024-04-25T10:01:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.724 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611394 s, 172 MB/s 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.732 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742308 s, 141 MB/s 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.739 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631757 s, 166 MB/s 2024-04-25T10:01:20.746 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.747 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657273 s, 160 MB/s 2024-04-25T10:01:20.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.755 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731079 s, 143 MB/s 2024-04-25T10:01:20.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.763 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714955 s, 147 MB/s 2024-04-25T10:01:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062528 s, 168 MB/s 2024-04-25T10:01:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.777 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629377 s, 167 MB/s 2024-04-25T10:01:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.785 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725035 s, 145 MB/s 2024-04-25T10:01:20.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.792 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608872 s, 172 MB/s 2024-04-25T10:01:20.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663032 s, 158 MB/s 2024-04-25T10:01:20.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062204 s, 169 MB/s 2024-04-25T10:01:20.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.814 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006322 s, 166 MB/s 2024-04-25T10:01:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576412 s, 182 MB/s 2024-04-25T10:01:20.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.828 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006282 s, 167 MB/s 2024-04-25T10:01:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735428 s, 143 MB/s 2024-04-25T10:01:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596594 s, 176 MB/s 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.851 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721803 s, 145 MB/s 2024-04-25T10:01:20.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.858 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627605 s, 167 MB/s 2024-04-25T10:01:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.866 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732611 s, 143 MB/s 2024-04-25T10:01:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.873 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607848 s, 173 MB/s 2024-04-25T10:01:20.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710714 s, 148 MB/s 2024-04-25T10:01:20.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:20.882 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:20.883 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:20.883 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:01:20.883 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:20.883 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:20.883 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:20.883 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:20.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580878 s, 181 MB/s 2024-04-25T10:01:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627121 s, 167 MB/s 2024-04-25T10:01:20.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768434 s, 136 MB/s 2024-04-25T10:01:20.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.971 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728434 s, 144 MB/s 2024-04-25T10:01:20.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611215 s, 172 MB/s 2024-04-25T10:01:20.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.986 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713817 s, 147 MB/s 2024-04-25T10:01:20.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:20.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:20.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625181 s, 168 MB/s 2024-04-25T10:01:21.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630566 s, 166 MB/s 2024-04-25T10:01:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.008 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708073 s, 148 MB/s 2024-04-25T10:01:21.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.014 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576011 s, 182 MB/s 2024-04-25T10:01:21.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714457 s, 147 MB/s 2024-04-25T10:01:21.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.031 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071346 s, 147 MB/s 2024-04-25T10:01:21.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.038 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655421 s, 160 MB/s 2024-04-25T10:01:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608127 s, 172 MB/s 2024-04-25T10:01:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.051 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576781 s, 182 MB/s 2024-04-25T10:01:21.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611432 s, 171 MB/s 2024-04-25T10:01:21.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.067 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758445 s, 138 MB/s 2024-04-25T10:01:21.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.074 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637306 s, 165 MB/s 2024-04-25T10:01:21.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.082 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722976 s, 145 MB/s 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.089 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576286 s, 182 MB/s 2024-04-25T10:01:21.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.096 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654982 s, 160 MB/s 2024-04-25T10:01:21.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.103 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577958 s, 181 MB/s 2024-04-25T10:01:21.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.111 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708725 s, 148 MB/s 2024-04-25T10:01:21.119 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.119 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.119 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710942 s, 147 MB/s 2024-04-25T10:01:21.127 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748417 s, 140 MB/s 2024-04-25T10:01:21.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.135 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640014 s, 164 MB/s 2024-04-25T10:01:21.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721014 s, 145 MB/s 2024-04-25T10:01:21.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071076 s, 148 MB/s 2024-04-25T10:01:21.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698269 s, 150 MB/s 2024-04-25T10:01:21.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.166 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678184 s, 155 MB/s 2024-04-25T10:01:21.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634739 s, 165 MB/s 2024-04-25T10:01:21.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645196 s, 163 MB/s 2024-04-25T10:01:21.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:21.182 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:21.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.247 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645499 s, 162 MB/s 2024-04-25T10:01:21.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599126 s, 175 MB/s 2024-04-25T10:01:21.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.260 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560131 s, 187 MB/s 2024-04-25T10:01:21.267 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638912 s, 164 MB/s 2024-04-25T10:01:21.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.275 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641263 s, 164 MB/s 2024-04-25T10:01:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655652 s, 160 MB/s 2024-04-25T10:01:21.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.290 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648244 s, 162 MB/s 2024-04-25T10:01:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.297 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641986 s, 163 MB/s 2024-04-25T10:01:21.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.305 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732924 s, 143 MB/s 2024-04-25T10:01:21.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.313 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601086 s, 174 MB/s 2024-04-25T10:01:21.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.319 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613301 s, 171 MB/s 2024-04-25T10:01:21.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.326 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636619 s, 165 MB/s 2024-04-25T10:01:21.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.334 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708709 s, 148 MB/s 2024-04-25T10:01:21.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.342 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672556 s, 156 MB/s 2024-04-25T10:01:21.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.349 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063083 s, 166 MB/s 2024-04-25T10:01:21.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.356 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563854 s, 186 MB/s 2024-04-25T10:01:21.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646055 s, 162 MB/s 2024-04-25T10:01:21.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.369 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554631 s, 189 MB/s 2024-04-25T10:01:21.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.377 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650159 s, 161 MB/s 2024-04-25T10:01:21.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.384 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632375 s, 166 MB/s 2024-04-25T10:01:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.391 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590591 s, 178 MB/s 2024-04-25T10:01:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896456 s, 117 MB/s 2024-04-25T10:01:21.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742665 s, 141 MB/s 2024-04-25T10:01:21.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690094 s, 152 MB/s 2024-04-25T10:01:21.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636105 s, 165 MB/s 2024-04-25T10:01:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070042 s, 150 MB/s 2024-04-25T10:01:21.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587965 s, 178 MB/s 2024-04-25T10:01:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553138 s, 190 MB/s 2024-04-25T10:01:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577998 s, 181 MB/s 2024-04-25T10:01:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686196 s, 153 MB/s 2024-04-25T10:01:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.466 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608396 s, 172 MB/s 2024-04-25T10:01:21.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.473 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562278 s, 186 MB/s 2024-04-25T10:01:21.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.474 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:21.474 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:21.475 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:01:21.475 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:21.475 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:21.475 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:21.475 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:21.539 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.539 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.539 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644391 s, 163 MB/s 2024-04-25T10:01:21.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.545 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606856 s, 173 MB/s 2024-04-25T10:01:21.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.553 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649619 s, 161 MB/s 2024-04-25T10:01:21.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.560 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588706 s, 178 MB/s 2024-04-25T10:01:21.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708346 s, 148 MB/s 2024-04-25T10:01:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.575 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066875 s, 157 MB/s 2024-04-25T10:01:21.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638371 s, 164 MB/s 2024-04-25T10:01:21.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.592 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922106 s, 114 MB/s 2024-04-25T10:01:21.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722488 s, 145 MB/s 2024-04-25T10:01:21.607 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.607 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.607 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582018 s, 180 MB/s 2024-04-25T10:01:21.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.614 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.614 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633543 s, 166 MB/s 2024-04-25T10:01:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.622 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643814 s, 163 MB/s 2024-04-25T10:01:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.629 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631533 s, 166 MB/s 2024-04-25T10:01:21.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.637 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746666 s, 140 MB/s 2024-04-25T10:01:21.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681087 s, 154 MB/s 2024-04-25T10:01:21.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678851 s, 154 MB/s 2024-04-25T10:01:21.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.662 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871312 s, 120 MB/s 2024-04-25T10:01:21.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572531 s, 183 MB/s 2024-04-25T10:01:21.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732254 s, 143 MB/s 2024-04-25T10:01:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627318 s, 167 MB/s 2024-04-25T10:01:21.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.691 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594304 s, 176 MB/s 2024-04-25T10:01:21.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072781 s, 144 MB/s 2024-04-25T10:01:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.707 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647112 s, 162 MB/s 2024-04-25T10:01:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.714 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644725 s, 163 MB/s 2024-04-25T10:01:21.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.721 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617236 s, 170 MB/s 2024-04-25T10:01:21.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.729 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703779 s, 149 MB/s 2024-04-25T10:01:21.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.736 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639258 s, 164 MB/s 2024-04-25T10:01:21.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652063 s, 161 MB/s 2024-04-25T10:01:21.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006285 s, 167 MB/s 2024-04-25T10:01:21.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711839 s, 147 MB/s 2024-04-25T10:01:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642741 s, 163 MB/s 2024-04-25T10:01:21.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709507 s, 148 MB/s 2024-04-25T10:01:21.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:21.775 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:21.775 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:21.775 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:01:21.775 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:21.775 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:21.776 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:21.776 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:21.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.841 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071508 s, 147 MB/s 2024-04-25T10:01:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.847 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565593 s, 185 MB/s 2024-04-25T10:01:21.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645796 s, 162 MB/s 2024-04-25T10:01:21.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.862 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640416 s, 164 MB/s 2024-04-25T10:01:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600155 s, 175 MB/s 2024-04-25T10:01:21.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628521 s, 167 MB/s 2024-04-25T10:01:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.885 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843206 s, 124 MB/s 2024-04-25T10:01:21.893 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.893 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.893 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728594 s, 144 MB/s 2024-04-25T10:01:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.900 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630433 s, 166 MB/s 2024-04-25T10:01:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.907 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580098 s, 181 MB/s 2024-04-25T10:01:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.914 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630494 s, 166 MB/s 2024-04-25T10:01:21.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.925 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951788 s, 110 MB/s 2024-04-25T10:01:21.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.932 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562777 s, 186 MB/s 2024-04-25T10:01:21.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865878 s, 121 MB/s 2024-04-25T10:01:21.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.949 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717126 s, 146 MB/s 2024-04-25T10:01:21.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630512 s, 166 MB/s 2024-04-25T10:01:21.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616904 s, 170 MB/s 2024-04-25T10:01:21.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.970 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682463 s, 154 MB/s 2024-04-25T10:01:21.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666165 s, 157 MB/s 2024-04-25T10:01:21.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604012 s, 174 MB/s 2024-04-25T10:01:21.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:21.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:21.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680859 s, 154 MB/s 2024-04-25T10:01:22.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735257 s, 143 MB/s 2024-04-25T10:01:22.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.008 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646425 s, 162 MB/s 2024-04-25T10:01:22.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.015 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603032 s, 174 MB/s 2024-04-25T10:01:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628603 s, 167 MB/s 2024-04-25T10:01:22.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.030 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673326 s, 156 MB/s 2024-04-25T10:01:22.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623207 s, 168 MB/s 2024-04-25T10:01:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.044 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709621 s, 148 MB/s 2024-04-25T10:01:22.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.051 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581401 s, 180 MB/s 2024-04-25T10:01:22.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.058 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057232 s, 183 MB/s 2024-04-25T10:01:22.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696927 s, 150 MB/s 2024-04-25T10:01:22.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.074 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695642 s, 151 MB/s 2024-04-25T10:01:22.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:22.075 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:22.140 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.140 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.141 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644726 s, 163 MB/s 2024-04-25T10:01:22.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.146 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632125 s, 166 MB/s 2024-04-25T10:01:22.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.154 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065154 s, 161 MB/s 2024-04-25T10:01:22.160 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.160 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.160 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583605 s, 180 MB/s 2024-04-25T10:01:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676516 s, 155 MB/s 2024-04-25T10:01:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.175 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065482 s, 160 MB/s 2024-04-25T10:01:22.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639554 s, 164 MB/s 2024-04-25T10:01:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.191 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756522 s, 139 MB/s 2024-04-25T10:01:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635048 s, 165 MB/s 2024-04-25T10:01:22.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065043 s, 161 MB/s 2024-04-25T10:01:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723753 s, 145 MB/s 2024-04-25T10:01:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562897 s, 186 MB/s 2024-04-25T10:01:22.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.229 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739238 s, 142 MB/s 2024-04-25T10:01:22.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.239 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932984 s, 112 MB/s 2024-04-25T10:01:22.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560683 s, 187 MB/s 2024-04-25T10:01:22.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.252 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593149 s, 177 MB/s 2024-04-25T10:01:22.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762909 s, 137 MB/s 2024-04-25T10:01:22.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624873 s, 168 MB/s 2024-04-25T10:01:22.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.275 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671974 s, 156 MB/s 2024-04-25T10:01:22.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.282 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602237 s, 174 MB/s 2024-04-25T10:01:22.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654512 s, 160 MB/s 2024-04-25T10:01:22.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.298 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706555 s, 148 MB/s 2024-04-25T10:01:22.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.305 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639929 s, 164 MB/s 2024-04-25T10:01:22.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.313 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701188 s, 150 MB/s 2024-04-25T10:01:22.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.320 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601566 s, 174 MB/s 2024-04-25T10:01:22.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690556 s, 152 MB/s 2024-04-25T10:01:22.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.334 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593671 s, 177 MB/s 2024-04-25T10:01:22.341 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.341 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.341 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059871 s, 175 MB/s 2024-04-25T10:01:22.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.349 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701114 s, 150 MB/s 2024-04-25T10:01:22.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.357 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734942 s, 143 MB/s 2024-04-25T10:01:22.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.365 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729422 s, 144 MB/s 2024-04-25T10:01:22.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.375 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885171 s, 118 MB/s 2024-04-25T10:01:22.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:22.376 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:22.376 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:22.376 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:01:22.377 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:22.377 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:22.377 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:22.377 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:22.441 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.441 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.442 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725824 s, 144 MB/s 2024-04-25T10:01:22.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629911 s, 166 MB/s 2024-04-25T10:01:22.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.456 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632294 s, 166 MB/s 2024-04-25T10:01:22.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.464 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753084 s, 139 MB/s 2024-04-25T10:01:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748026 s, 140 MB/s 2024-04-25T10:01:22.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.481 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718651 s, 146 MB/s 2024-04-25T10:01:22.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603948 s, 174 MB/s 2024-04-25T10:01:22.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.494 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058079 s, 181 MB/s 2024-04-25T10:01:22.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.501 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624003 s, 168 MB/s 2024-04-25T10:01:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.509 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067883 s, 154 MB/s 2024-04-25T10:01:22.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.518 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085127 s, 123 MB/s 2024-04-25T10:01:22.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.526 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634191 s, 165 MB/s 2024-04-25T10:01:22.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.533 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709181 s, 148 MB/s 2024-04-25T10:01:22.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613391 s, 171 MB/s 2024-04-25T10:01:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650564 s, 161 MB/s 2024-04-25T10:01:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.555 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609793 s, 172 MB/s 2024-04-25T10:01:22.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.565 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931317 s, 113 MB/s 2024-04-25T10:01:22.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729956 s, 144 MB/s 2024-04-25T10:01:22.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713943 s, 147 MB/s 2024-04-25T10:01:22.589 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.589 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.589 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661532 s, 159 MB/s 2024-04-25T10:01:22.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626326 s, 167 MB/s 2024-04-25T10:01:22.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638098 s, 164 MB/s 2024-04-25T10:01:22.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752996 s, 139 MB/s 2024-04-25T10:01:22.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.620 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.620 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734056 s, 143 MB/s 2024-04-25T10:01:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.626 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574955 s, 182 MB/s 2024-04-25T10:01:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.634 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712734 s, 147 MB/s 2024-04-25T10:01:22.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.642 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634476 s, 165 MB/s 2024-04-25T10:01:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641891 s, 163 MB/s 2024-04-25T10:01:22.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.657 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706797 s, 148 MB/s 2024-04-25T10:01:22.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.664 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574178 s, 183 MB/s 2024-04-25T10:01:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.672 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071889 s, 146 MB/s 2024-04-25T10:01:22.680 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.680 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.680 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735684 s, 143 MB/s 2024-04-25T10:01:22.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:22.681 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:22.746 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.746 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.746 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652592 s, 161 MB/s 2024-04-25T10:01:22.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.754 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751162 s, 140 MB/s 2024-04-25T10:01:22.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.761 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579353 s, 181 MB/s 2024-04-25T10:01:22.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598177 s, 175 MB/s 2024-04-25T10:01:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.775 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065652 s, 160 MB/s 2024-04-25T10:01:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725139 s, 145 MB/s 2024-04-25T10:01:22.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.792 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727069 s, 144 MB/s 2024-04-25T10:01:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726938 s, 144 MB/s 2024-04-25T10:01:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057203 s, 183 MB/s 2024-04-25T10:01:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.815 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715447 s, 147 MB/s 2024-04-25T10:01:22.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.823 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714207 s, 147 MB/s 2024-04-25T10:01:22.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.829 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567577 s, 185 MB/s 2024-04-25T10:01:22.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.837 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691537 s, 152 MB/s 2024-04-25T10:01:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680849 s, 154 MB/s 2024-04-25T10:01:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620068 s, 169 MB/s 2024-04-25T10:01:22.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633331 s, 166 MB/s 2024-04-25T10:01:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557969 s, 188 MB/s 2024-04-25T10:01:22.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.873 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639502 s, 164 MB/s 2024-04-25T10:01:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736583 s, 142 MB/s 2024-04-25T10:01:22.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725532 s, 145 MB/s 2024-04-25T10:01:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835876 s, 125 MB/s 2024-04-25T10:01:22.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.907 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741919 s, 141 MB/s 2024-04-25T10:01:22.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561532 s, 187 MB/s 2024-04-25T10:01:22.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671512 s, 156 MB/s 2024-04-25T10:01:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561571 s, 187 MB/s 2024-04-25T10:01:22.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.935 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694886 s, 151 MB/s 2024-04-25T10:01:22.943 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.943 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.943 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734824 s, 143 MB/s 2024-04-25T10:01:22.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579309 s, 181 MB/s 2024-04-25T10:01:22.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.958 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627834 s, 167 MB/s 2024-04-25T10:01:22.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.965 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707987 s, 148 MB/s 2024-04-25T10:01:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006418 s, 163 MB/s 2024-04-25T10:01:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006593 s, 159 MB/s 2024-04-25T10:01:22.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:22.981 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:22.981 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:22.981 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:01:22.981 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:22.982 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:22.982 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:22.982 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:23.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892202 s, 118 MB/s 2024-04-25T10:01:23.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679522 s, 154 MB/s 2024-04-25T10:01:23.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.063 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576451 s, 182 MB/s 2024-04-25T10:01:23.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.071 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725355 s, 145 MB/s 2024-04-25T10:01:23.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620727 s, 169 MB/s 2024-04-25T10:01:23.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666345 s, 157 MB/s 2024-04-25T10:01:23.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.092 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618439 s, 170 MB/s 2024-04-25T10:01:23.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.099 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574425 s, 183 MB/s 2024-04-25T10:01:23.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593995 s, 177 MB/s 2024-04-25T10:01:23.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565186 s, 186 MB/s 2024-04-25T10:01:23.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.120 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650596 s, 161 MB/s 2024-04-25T10:01:23.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713724 s, 147 MB/s 2024-04-25T10:01:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.135 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627887 s, 167 MB/s 2024-04-25T10:01:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746566 s, 140 MB/s 2024-04-25T10:01:23.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652663 s, 161 MB/s 2024-04-25T10:01:23.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625459 s, 168 MB/s 2024-04-25T10:01:23.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626625 s, 167 MB/s 2024-04-25T10:01:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620958 s, 169 MB/s 2024-04-25T10:01:23.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.180 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753051 s, 139 MB/s 2024-04-25T10:01:23.188 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.188 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.188 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636201 s, 165 MB/s 2024-04-25T10:01:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710665 s, 148 MB/s 2024-04-25T10:01:23.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637338 s, 165 MB/s 2024-04-25T10:01:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.211 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729048 s, 144 MB/s 2024-04-25T10:01:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630139 s, 166 MB/s 2024-04-25T10:01:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604859 s, 173 MB/s 2024-04-25T10:01:23.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.233 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721889 s, 145 MB/s 2024-04-25T10:01:23.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703084 s, 149 MB/s 2024-04-25T10:01:23.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581724 s, 180 MB/s 2024-04-25T10:01:23.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554531 s, 189 MB/s 2024-04-25T10:01:23.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630049 s, 166 MB/s 2024-04-25T10:01:23.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609278 s, 172 MB/s 2024-04-25T10:01:23.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.277 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633109 s, 166 MB/s 2024-04-25T10:01:23.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:23.277 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:23.277 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:23.277 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:01:23.277 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:23.278 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:23.278 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:23.278 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:23.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742698 s, 141 MB/s 2024-04-25T10:01:23.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.350 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627468 s, 167 MB/s 2024-04-25T10:01:23.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695817 s, 151 MB/s 2024-04-25T10:01:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756744 s, 139 MB/s 2024-04-25T10:01:23.374 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.374 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.374 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710381 s, 148 MB/s 2024-04-25T10:01:23.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.382 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063624 s, 165 MB/s 2024-04-25T10:01:23.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.389 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643762 s, 163 MB/s 2024-04-25T10:01:23.396 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.397 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067356 s, 156 MB/s 2024-04-25T10:01:23.404 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.404 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.404 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657716 s, 159 MB/s 2024-04-25T10:01:23.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.411 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648628 s, 162 MB/s 2024-04-25T10:01:23.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702003 s, 149 MB/s 2024-04-25T10:01:23.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798267 s, 131 MB/s 2024-04-25T10:01:23.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.435 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627082 s, 167 MB/s 2024-04-25T10:01:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.443 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676123 s, 155 MB/s 2024-04-25T10:01:23.449 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.450 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646514 s, 162 MB/s 2024-04-25T10:01:23.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.457 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631294 s, 166 MB/s 2024-04-25T10:01:23.465 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.465 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.465 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716195 s, 146 MB/s 2024-04-25T10:01:23.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.473 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746653 s, 140 MB/s 2024-04-25T10:01:23.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.481 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650239 s, 161 MB/s 2024-04-25T10:01:23.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.489 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757226 s, 138 MB/s 2024-04-25T10:01:23.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.496 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641206 s, 164 MB/s 2024-04-25T10:01:23.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735534 s, 143 MB/s 2024-04-25T10:01:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.511 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058293 s, 180 MB/s 2024-04-25T10:01:23.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.519 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638448 s, 164 MB/s 2024-04-25T10:01:23.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.527 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668064 s, 157 MB/s 2024-04-25T10:01:23.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637135 s, 165 MB/s 2024-04-25T10:01:23.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670189 s, 156 MB/s 2024-04-25T10:01:23.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623208 s, 168 MB/s 2024-04-25T10:01:23.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.555 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600326 s, 175 MB/s 2024-04-25T10:01:23.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.562 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623655 s, 168 MB/s 2024-04-25T10:01:23.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.569 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617047 s, 170 MB/s 2024-04-25T10:01:23.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.578 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073792 s, 142 MB/s 2024-04-25T10:01:23.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:01:23.579 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:01:23.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589494 s, 178 MB/s 2024-04-25T10:01:23.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.705 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710607 s, 148 MB/s 2024-04-25T10:01:23.712 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.712 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.712 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576451 s, 182 MB/s 2024-04-25T10:01:23.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.720 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703602 s, 149 MB/s 2024-04-25T10:01:23.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685507 s, 153 MB/s 2024-04-25T10:01:23.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.735 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616702 s, 170 MB/s 2024-04-25T10:01:23.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.742 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596977 s, 176 MB/s 2024-04-25T10:01:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625289 s, 168 MB/s 2024-04-25T10:01:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626456 s, 167 MB/s 2024-04-25T10:01:23.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.763 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604597 s, 173 MB/s 2024-04-25T10:01:23.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625129 s, 168 MB/s 2024-04-25T10:01:23.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700343 s, 150 MB/s 2024-04-25T10:01:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.789 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893149 s, 117 MB/s 2024-04-25T10:01:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.796 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755637 s, 139 MB/s 2024-04-25T10:01:23.804 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.804 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.804 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724499 s, 145 MB/s 2024-04-25T10:01:23.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.811 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.811 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637015 s, 165 MB/s 2024-04-25T10:01:23.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581568 s, 180 MB/s 2024-04-25T10:01:23.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573592 s, 183 MB/s 2024-04-25T10:01:23.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.832 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630107 s, 166 MB/s 2024-04-25T10:01:23.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.839 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595254 s, 176 MB/s 2024-04-25T10:01:23.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056294 s, 186 MB/s 2024-04-25T10:01:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767431 s, 137 MB/s 2024-04-25T10:01:23.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.862 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735284 s, 143 MB/s 2024-04-25T10:01:23.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607185 s, 173 MB/s 2024-04-25T10:01:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.877 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647666 s, 162 MB/s 2024-04-25T10:01:23.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563206 s, 186 MB/s 2024-04-25T10:01:23.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.891 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653827 s, 160 MB/s 2024-04-25T10:01:23.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675726 s, 155 MB/s 2024-04-25T10:01:23.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.905 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627595 s, 167 MB/s 2024-04-25T10:01:23.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.915 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906711 s, 116 MB/s 2024-04-25T10:01:23.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.923 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720742 s, 145 MB/s 2024-04-25T10:01:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:01:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:01:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626832 s, 167 MB/s 2024-04-25T10:01:23.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:23.932 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:01:24.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.028+0000 7f6132330640 1 Processor -- start 2024-04-25T10:01:24.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.028+0000 7f6132330640 1 -- start start 2024-04-25T10:01:24.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f6132330640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c15dc20 0x7f612c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f6132330640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f612c161710 0x7f612c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f6132330640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c15e000 con 0x7f612c0b9330 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f6132330640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c0d1de0 con 0x7f612c161710 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f6132330640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c161c60 con 0x7f612c15dc20 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f612b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f612c161710 0x7f612c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f612b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f612c161710 0x7f612c0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33084/0 (socket says 172.21.15.148:33084) 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.029+0000 7f612b7fe640 1 -- 172.21.15.148:0/2628676466 learned_addr learned my addr 172.21.15.148:0/2628676466 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612bfff640 1 --2- 172.21.15.148:0/2628676466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c15dc20 0x7f612c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612b7fe640 1 -- 172.21.15.148:0/2628676466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c15dc20 msgr2=0x7f612c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612b7fe640 1 --2- 172.21.15.148:0/2628676466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c15dc20 0x7f612c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612b7fe640 1 -- 172.21.15.148:0/2628676466 >> v1:172.21.15.188:6789/0 conn(0x7f612c0b9330 legacy=0x7f612c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612b7fe640 1 -- 172.21.15.148:0/2628676466 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c161f40 con 0x7f612c161710 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612b7fe640 1 --2- 172.21.15.148:0/2628676466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c0cebb0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6120009870 tx=0x7f612002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55771f8c299520bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.030+0000 7f612affd640 1 -- 172.21.15.148:0/2628676466 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120037070 con 0x7f612c161710 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f612affd640 1 -- 172.21.15.148:0/2628676466 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6120032440 con 0x7f612c161710 2024-04-25T10:01:24.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f612affd640 1 -- 172.21.15.148:0/2628676466 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120032810 con 0x7f612c161710 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 -- 172.21.15.148:0/2628676466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 msgr2=0x7f612c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 --2- 172.21.15.148:0/2628676466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c0cebb0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6120009870 tx=0x7f612002a210 comp rx=0 tx=0).stop 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 -- 172.21.15.148:0/2628676466 shutdown_connections 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 --2- 172.21.15.148:0/2628676466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c0cebb0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 --2- 172.21.15.148:0/2628676466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c15dc20 0x7f612c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 -- 172.21.15.148:0/2628676466 >> 172.21.15.148:0/2628676466 conn(0x7f612c0d0420 msgr2=0x7f612c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 -- 172.21.15.148:0/2628676466 shutdown_connections 2024-04-25T10:01:24.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.031+0000 7f6132330640 1 -- 172.21.15.148:0/2628676466 wait complete. 2024-04-25T10:01:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.032+0000 7f6132330640 1 Processor -- start 2024-04-25T10:01:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.032+0000 7f6132330640 1 -- start start 2024-04-25T10:01:24.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f6132330640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c0b9330 0x7f612c1e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f6132330640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f612c15dc20 0x7f612c15fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f6132330640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f6132330640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b5e50 con 0x7f612c161710 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f6132330640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c162500 con 0x7f612c15dc20 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f6132330640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b5fd0 con 0x7f612c0b9330 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f61308cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f61308cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c15e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33098/0 (socket says 172.21.15.148:33098) 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f61308cd640 1 -- 172.21.15.148:0/3708216281 learned_addr learned my addr 172.21.15.148:0/3708216281 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f612b7fe640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f612c15dc20 0x7f612c15fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f612bfff640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c0b9330 0x7f612c1e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f61308cd640 1 -- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c0b9330 msgr2=0x7f612c1e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f61308cd640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c0b9330 0x7f612c1e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.033+0000 7f61308cd640 1 -- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f612c15dc20 msgr2=0x7f612c15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f61308cd640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f612c15dc20 0x7f612c15fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f61308cd640 1 -- 172.21.15.148:0/3708216281 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c1e83b0 con 0x7f612c161710 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f61308cd640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c15e1d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f612400b9e0 tx=0x7f612400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f6128ff9640 1 -- 172.21.15.148:0/3708216281 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124003b70 con 0x7f612c161710 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f6128ff9640 1 -- 172.21.15.148:0/3708216281 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124003d10 con 0x7f612c161710 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c1e8690 con 0x7f612c161710 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f6128ff9640 1 -- 172.21.15.148:0/3708216281 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400cae0 con 0x7f612c161710 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.034+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c1e8c50 con 0x7f612c161710 2024-04-25T10:01:24.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.035+0000 7f6128ff9640 1 -- 172.21.15.148:0/3708216281 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f612400cc80 con 0x7f612c161710 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.036+0000 7f6128ff9640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f611806b4d0 0x7f611806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.036+0000 7f6128ff9640 1 -- 172.21.15.148:0/3708216281 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f612408a030 con 0x7f612c161710 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.036+0000 7f612bfff640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f611806b4d0 0x7f611806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.036+0000 7f612bfff640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f611806b4d0 0x7f611806d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f612c15d030 tx=0x7f611c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 Processor -- start 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 -- start start 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e97360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34e955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f34df9000 con 0x558f34ebfcc0 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f34da1f20 con 0x558f34e95200 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f34df50a0 con 0x558f34ed18d0 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f61308cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f61308cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33108/0 (socket says 172.21.15.148:33108) 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.037+0000 7f61308cd640 1 -- 172.21.15.148:0/1269809253 learned_addr learned my addr 172.21.15.148:0/1269809253 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f612bfff640 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e97360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f612b7fe640 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34e955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f61308cd640 1 -- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 msgr2=0x558f34e955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f61308cd640 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34e955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f61308cd640 1 -- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 msgr2=0x558f34e97360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f61308cd640 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e97360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f61308cd640 1 -- 172.21.15.148:0/1269809253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f34bd9ba0 con 0x558f34ebfcc0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f612bfff640 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e97360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f61308cd640 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0140 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6124088400 tx=0x7f6124058760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efca8903f4c134d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f6111ffb640 1 -- 172.21.15.148:0/1269809253 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124060dc0 con 0x558f34ebfcc0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f6111ffb640 1 -- 172.21.15.148:0/1269809253 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124070170 con 0x558f34ebfcc0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.038+0000 7f6111ffb640 1 -- 172.21.15.148:0/1269809253 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124054dd0 con 0x558f34ebfcc0 2024-04-25T10:01:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 -- 172.21.15.148:0/1269809253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 msgr2=0x558f34ec0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0140 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6124088400 tx=0x7f6124058760 comp rx=0 tx=0).stop 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 -- 172.21.15.148:0/1269809253 shutdown_connections 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0140 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34e955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 --2- 172.21.15.148:0/1269809253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e97360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 -- 172.21.15.148:0/1269809253 >> 172.21.15.148:0/1269809253 conn(0x558f34e5d050 msgr2=0x558f34edacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 -- 172.21.15.148:0/1269809253 shutdown_connections 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 -- 172.21.15.148:0/1269809253 wait complete. 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 -- start start 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.039+0000 7f6135034740 1 Processor -- start 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34edd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34ebe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f34d97610 con 0x558f34ebfcc0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f34ebd810 con 0x558f34e95200 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f34ec1fa0 con 0x558f34ed18d0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f6135034740 10 client.0 _ll_register_callbacks cb 0x558f34daff20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33110/0 (socket says 172.21.15.148:33110) 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f612b7fe640 1 -- 172.21.15.148:0/931036977 learned_addr learned my addr 172.21.15.148:0/931036977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f612bfff640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34edd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.040+0000 7f61308cd640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34ebe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f612b7fe640 1 -- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 msgr2=0x558f34ebe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f612b7fe640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34ebe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f612b7fe640 1 -- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 msgr2=0x558f34edd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f612b7fe640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34edd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f612b7fe640 1 -- 172.21.15.148:0/931036977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f34acd490 con 0x558f34ebfcc0 2024-04-25T10:01:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f612b7fe640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0af0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6120009870 tx=0x7f6120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f808b43029587d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f610affd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f610affd640 1 -- 172.21.15.148:0/931036977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120037040 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f610affd640 1 -- 172.21.15.148:0/931036977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6120047c30 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6124070170 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f610affd640 1 -- 172.21.15.148:0/931036977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6120047e30 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 10 client.8319 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x558f34c3a330 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f610affd640 1 -- 172.21.15.148:0/931036977 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124070170 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f610affd640 1 -- 172.21.15.148:0/931036977 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f612004e050 con 0x558f34ebfcc0 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 10 client.8319 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 10 client.8319 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 20 client.8319 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34ebeaa0 0x558f34ea74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 4 client.8319 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.041+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f34e06540 con 0x558f34ebeaa0 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.042+0000 7f612bfff640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34ebeaa0 0x558f34ea74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.042+0000 7f610affd640 10 client.8319 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.042+0000 7f612bfff640 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34ebeaa0 0x558f34ea74d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30a4478320156a0f server_cookie=ce9c4a3db33124e in_seq=0 out_seq=0 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.042+0000 7f610affd640 1 -- 172.21.15.148:0/931036977 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f34e06540 con 0x558f34ebeaa0 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.042+0000 7f610affd640 10 client.8319 handle_command_reply: tid=0 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.042+0000 7f6135034740 1 client.8319 shutdown 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 msgr2=0x558f34ec0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0af0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6120009870 tx=0x7f6120004730 comp rx=0 tx=0).stop 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 shutdown_connections 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34ebe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34ec0af0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34ebeaa0 0x558f34ea74d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 --2- 172.21.15.148:0/931036977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34edd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f610affd640 0 client.8319 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 >> 172.21.15.148:0/931036977 conn(0x558f34e5d050 msgr2=0x558f34ede240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 shutdown_connections 2024-04-25T10:01:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 1 -- 172.21.15.148:0/931036977 wait complete. 2024-04-25T10:01:24.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.043+0000 7f6135034740 20 client.8319 trim_cache size 0 max 16384 2024-04-25T10:01:24.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 Processor -- start 2024-04-25T10:01:24.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 -- start start 2024-04-25T10:01:24.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e8b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ebfcc0 0x558f34ed2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 0x558f34ed2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c2006c0 con 0x558f34ed18d0 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1ff890 con 0x558f34e95200 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.058+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0cc830 con 0x558f34ebfcc0 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 0x558f34ed2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 0x558f34ed2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33118/0 (socket says 172.21.15.148:33118) 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 -- 172.21.15.148:0/2614380383 learned_addr learned my addr 172.21.15.148:0/2614380383 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f612bfff640 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e8b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f612b7fe640 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ebfcc0 0x558f34ed2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 -- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ebfcc0 msgr2=0x558f34ed2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ebfcc0 0x558f34ed2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 -- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 msgr2=0x558f34e8b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e8b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 -- 172.21.15.148:0/2614380383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f34ed3570 con 0x558f34ed18d0 2024-04-25T10:01:24.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61308cd640 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 0x558f34ed2660 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f61240884b0 tx=0x7f6124089000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2d4a08a7fc7cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61117fa640 1 -- 172.21.15.148:0/2614380383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124070170 con 0x558f34ed18d0 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61117fa640 1 -- 172.21.15.148:0/2614380383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124056a60 con 0x558f34ed18d0 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.059+0000 7f61117fa640 1 -- 172.21.15.148:0/2614380383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124060930 con 0x558f34ed18d0 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 -- 172.21.15.148:0/2614380383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 msgr2=0x558f34ed2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 0x558f34ed2660 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f61240884b0 tx=0x7f6124089000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 -- 172.21.15.148:0/2614380383 shutdown_connections 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ed18d0 0x558f34ed2660 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ebfcc0 0x558f34ed2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 --2- 172.21.15.148:0/2614380383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34e8b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 -- 172.21.15.148:0/2614380383 >> 172.21.15.148:0/2614380383 conn(0x558f34e5d050 msgr2=0x558f34ebe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 -- 172.21.15.148:0/2614380383 shutdown_connections 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 -- 172.21.15.148:0/2614380383 wait complete. 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 -- start start 2024-04-25T10:01:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.060+0000 7f6135034740 1 Processor -- start 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34f39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34f4a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34f36f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f34ec08a0 con 0x558f34ebfcc0 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0cce30 con 0x558f34e95200 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f34ea6ee0 con 0x558f34ed18d0 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f6135034740 10 client.0 _ll_register_callbacks cb 0x7f611c007120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:24.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34f4a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34f4a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33126/0 (socket says 172.21.15.148:33126) 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 -- 172.21.15.148:0/2846341025 learned_addr learned my addr 172.21.15.148:0/2846341025 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612bfff640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34f39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f61308cd640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34f36f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 -- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 msgr2=0x558f34f36f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34f36f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 -- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 msgr2=0x558f34f39f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34f39f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612b7fe640 1 -- 172.21.15.148:0/2846341025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f34aa6610 con 0x558f34ebfcc0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.061+0000 7f612bfff640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34f39f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f612b7fe640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34f4a030 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6120009630 tx=0x7f6120009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ae0adaf5dd891e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f610affd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f610affd640 1 -- 172.21.15.148:0/2846341025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120037070 con 0x558f34ebfcc0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f610affd640 1 -- 172.21.15.148:0/2846341025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6120003610 con 0x558f34ebfcc0 2024-04-25T10:01:24.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6124056a60 con 0x558f34ebfcc0 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f610affd640 1 -- 172.21.15.148:0/2846341025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f612005a050 con 0x558f34ebfcc0 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 10 client.8325 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x558f34ed1540 con 0x558f34ebfcc0 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f610affd640 1 -- 172.21.15.148:0/2846341025 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124056a60 con 0x558f34ebfcc0 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f610affd640 1 -- 172.21.15.148:0/2846341025 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f612005d020 con 0x558f34ebfcc0 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 10 client.8325 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 10 client.8325 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 20 client.8325 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34e94b90 0x558f34f374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 4 client.8325 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x558f34e00890 con 0x558f34e94b90 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.062+0000 7f612bfff640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34e94b90 0x558f34f374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.063+0000 7f610affd640 10 client.8325 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.063+0000 7f612bfff640 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34e94b90 0x558f34f374b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6a5111fc390dac server_cookie=d9f9092971b1179 in_seq=0 out_seq=0 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f610affd640 1 -- 172.21.15.148:0/2846341025 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558f34e00890 con 0x558f34e94b90 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f610affd640 10 client.8325 handle_command_reply: tid=0 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 client.8325 shutdown 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 msgr2=0x558f34f4a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34f4a030 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f6120009630 tx=0x7f6120009240 comp rx=0 tx=0).stop 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 shutdown_connections 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558f34ed18d0 0x558f34f36f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558f34ebfcc0 0x558f34f4a030 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x558f34e94b90 0x558f34f374b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 --2- 172.21.15.148:0/2846341025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558f34e95200 0x558f34f39f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f610affd640 0 client.8325 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 >> 172.21.15.148:0/2846341025 conn(0x558f34e5d050 msgr2=0x558f34ed2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 shutdown_connections 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 1 -- 172.21.15.148:0/2846341025 wait complete. 2024-04-25T10:01:24.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.065+0000 7f6135034740 20 client.8325 trim_cache size 0 max 16384 2024-04-25T10:01:24.074 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:01:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "fcd45599-579f-4875-8c15-54c8e8662918", 2024-04-25T10:01:24.074 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:01:24.074 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f611806b4d0 msgr2=0x7f611806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f611806b4d0 0x7f611806d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f612c15d030 tx=0x7f611c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 msgr2=0x7f612c15e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c15e1d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f612400b9e0 tx=0x7f612400bea0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 shutdown_connections 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f611806b4d0 0x7f611806d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f612c161710 0x7f612c15e1d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f612c15dc20 0x7f612c15fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 --2- 172.21.15.148:0/3708216281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f612c0b9330 0x7f612c1e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 >> 172.21.15.148:0/3708216281 conn(0x7f612c0d0420 msgr2=0x7f612c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 shutdown_connections 2024-04-25T10:01:24.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.066+0000 7f6132330640 1 -- 172.21.15.148:0/3708216281 wait complete. 2024-04-25T10:01:24.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:24.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T10:01:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.190+0000 7f928bf29640 1 Processor -- start 2024-04-25T10:01:24.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.190+0000 7f928bf29640 1 -- start start 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f928bf29640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92840cadb0 0x7f92840cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f928bf29640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92840ccfe0 0x7f92840cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f928bf29640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92840afc80 con 0x7f928415de80 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f928bf29640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840cb240 con 0x7f92840cadb0 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f928bf29640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840ce4d0 con 0x7f92840ccfe0 2024-04-25T10:01:24.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f92894c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92840cadb0 0x7f92840cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f92894c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92840cadb0 0x7f92840cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33134/0 (socket says 172.21.15.148:33134) 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f92894c4640 1 -- 172.21.15.148:0/3885777212 learned_addr learned my addr 172.21.15.148:0/3885777212 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.191+0000 7f9289cc5640 1 --1- 172.21.15.148:0/3885777212 >> v1:172.21.15.148:6789/0 conn(0x7f928415de80 0x7f92840cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49010/0 (socket says 172.21.15.148:49010) 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f928a4c6640 1 --2- 172.21.15.148:0/3885777212 >> v2:172.21.15.188:3300/0 conn(0x7f92840ccfe0 0x7f92840cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f92894c4640 1 -- 172.21.15.148:0/3885777212 >> v2:172.21.15.188:3300/0 conn(0x7f92840ccfe0 msgr2=0x7f92840cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f92894c4640 1 --2- 172.21.15.148:0/3885777212 >> v2:172.21.15.188:3300/0 conn(0x7f92840ccfe0 0x7f92840cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f92894c4640 1 -- 172.21.15.148:0/3885777212 >> v1:172.21.15.148:6789/0 conn(0x7f928415de80 legacy=0x7f92840cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f9288cc3640 1 -- 172.21.15.148:0/3885777212 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716891499 0 0) 0x7f92840afc80 con 0x7f928415de80 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f92894c4640 1 -- 172.21.15.148:0/3885777212 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840ce7b0 con 0x7f92840cadb0 2024-04-25T10:01:24.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.192+0000 7f92894c4640 1 --2- 172.21.15.148:0/3885777212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840cadb0 0x7f92840cf6c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f928406e2a0 tx=0x7f927402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b38fee5e8ebbad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f9288cc3640 1 -- 172.21.15.148:0/3885777212 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274037070 con 0x7f92840cadb0 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f9288cc3640 1 -- 172.21.15.148:0/3885777212 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9274032440 con 0x7f92840cadb0 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f9288cc3640 1 -- 172.21.15.148:0/3885777212 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274032810 con 0x7f92840cadb0 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 -- 172.21.15.148:0/3885777212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840cadb0 msgr2=0x7f92840cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 --2- 172.21.15.148:0/3885777212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840cadb0 0x7f92840cf6c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f928406e2a0 tx=0x7f927402a270 comp rx=0 tx=0).stop 2024-04-25T10:01:24.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 -- 172.21.15.148:0/3885777212 shutdown_connections 2024-04-25T10:01:24.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 --2- 172.21.15.148:0/3885777212 >> v2:172.21.15.188:3300/0 conn(0x7f92840ccfe0 0x7f92840cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 --2- 172.21.15.148:0/3885777212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92840cadb0 0x7f92840cf6c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 -- 172.21.15.148:0/3885777212 >> 172.21.15.148:0/3885777212 conn(0x7f92840b2390 msgr2=0x7f92840b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 -- 172.21.15.148:0/3885777212 shutdown_connections 2024-04-25T10:01:24.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.193+0000 7f928bf29640 1 -- 172.21.15.148:0/3885777212 wait complete. 2024-04-25T10:01:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.194+0000 7f928bf29640 1 Processor -- start 2024-04-25T10:01:24.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.194+0000 7f928bf29640 1 -- start start 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928bf29640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92840cadb0 0x7f92841526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928bf29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840ccfe0 0x7f92840aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928bf29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 0x7f928406f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928bf29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928415e260 con 0x7f928415de80 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928bf29640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840cfc00 con 0x7f92840cadb0 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928bf29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b2eb0 con 0x7f92840ccfe0 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928a4c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 0x7f928406f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928a4c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 0x7f928406f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33136/0 (socket says 172.21.15.148:33136) 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928a4c6640 1 -- 172.21.15.148:0/4165515685 learned_addr learned my addr 172.21.15.148:0/4165515685 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f9289cc5640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92840cadb0 0x7f92841526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f92894c4640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840ccfe0 0x7f92840aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.195+0000 7f928a4c6640 1 -- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840ccfe0 msgr2=0x7f92840aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928a4c6640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840ccfe0 0x7f92840aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928a4c6640 1 -- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92840cadb0 msgr2=0x7f92841526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928a4c6640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92840cadb0 0x7f92841526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928a4c6640 1 -- 172.21.15.148:0/4165515685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840ca240 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928a4c6640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 0x7f928406f460 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f927c00d800 tx=0x7f927c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f927a7fc640 1 -- 172.21.15.148:0/4165515685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00b6c0 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f927a7fc640 1 -- 172.21.15.148:0/4165515685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c019070 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9284152c10 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f927a7fc640 1 -- 172.21.15.148:0/4165515685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00bac0 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.196+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92840c7c90 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.197+0000 7f927a7fc640 1 -- 172.21.15.148:0/4165515685 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f927c00bc60 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.198+0000 7f927a7fc640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926806b4d0 0x7f926806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.198+0000 7f927a7fc640 1 -- 172.21.15.148:0/4165515685 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f927c08a770 con 0x7f928415de80 2024-04-25T10:01:24.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.198+0000 7f9289cc5640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926806b4d0 0x7f926806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.198+0000 7f9289cc5640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926806b4d0 0x7f926806d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9280007ed0 tx=0x7f9280007e60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.199+0000 7f928ec2c740 1 Processor -- start 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.199+0000 7f928ec2c740 1 -- start start 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.199+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e1763ca0 0x5620e1765e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.199+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e1764080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e178ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928ec2c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e16c6b90 con 0x5620e178e760 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e166fe40 con 0x5620e1763ca0 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620e16c2c30 con 0x5620e17a0370 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e178ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e178ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33150/0 (socket says 172.21.15.148:33150) 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 -- 172.21.15.148:0/2598645626 learned_addr learned my addr 172.21.15.148:0/2598645626 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f9289cc5640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e1763ca0 0x5620e1765e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f92894c4640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e1764080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 -- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 msgr2=0x5620e1764080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e1764080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 -- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e1763ca0 msgr2=0x5620e1765e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e1763ca0 0x5620e1765e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 -- 172.21.15.148:0/2598645626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e14a57d0 con 0x5620e178e760 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f92894c4640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e1764080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f9289cc5640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e1763ca0 0x5620e1765e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f928a4c6640 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e178ebe0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f927c088b40 tx=0x7f927c0594f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45047825bcc738bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f92677fe640 1 -- 172.21.15.148:0/2598645626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c05de60 con 0x5620e178e760 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f92677fe640 1 -- 172.21.15.148:0/2598645626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c05cf40 con 0x5620e178e760 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.200+0000 7f92677fe640 1 -- 172.21.15.148:0/2598645626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c061500 con 0x5620e178e760 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 -- 172.21.15.148:0/2598645626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 msgr2=0x5620e178ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e178ebe0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f927c088b40 tx=0x7f927c0594f0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 -- 172.21.15.148:0/2598645626 shutdown_connections 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e178ebe0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e1764080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2598645626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e1763ca0 0x5620e1765e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 -- 172.21.15.148:0/2598645626 >> 172.21.15.148:0/2598645626 conn(0x5620e172b870 msgr2=0x5620e17a9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 -- 172.21.15.148:0/2598645626 shutdown_connections 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.201+0000 7f928ec2c740 1 -- 172.21.15.148:0/2598645626 wait complete. 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.202+0000 7f928ec2c740 1 -- start start 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.202+0000 7f928ec2c740 1 Processor -- start 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.202+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 0x5620e17a4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e178e760 0x5620e17a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e17bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e16833e0 con 0x5620e1763ca0 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e178c2b0 con 0x5620e178e760 2024-04-25T10:01:24.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620e1790370 con 0x5620e17a0370 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 10 client.0 _ll_register_callbacks cb 0x5620e1682db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 0x5620e17a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 0x5620e17a4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33154/0 (socket says 172.21.15.148:33154) 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 -- 172.21.15.148:0/2561486866 learned_addr learned my addr 172.21.15.148:0/2561486866 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f92894c4640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e178e760 0x5620e17a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928a4c6640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e17bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 -- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 msgr2=0x5620e17bca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e17bca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 -- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e178e760 msgr2=0x5620e17a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e178e760 0x5620e17a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 -- 172.21.15.148:0/2561486866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e139b0f0 con 0x5620e1763ca0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9289cc5640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 0x5620e17a4990 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9280015720 tx=0x7f9280011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607630765cf80f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9264ff9640 1 -- 172.21.15.148:0/2561486866 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280012440 con 0x5620e1763ca0 2024-04-25T10:01:24.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9264ff9640 1 -- 172.21.15.148:0/2561486866 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9280021c10 con 0x5620e1763ca0 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f927c05cf40 con 0x5620e1763ca0 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9264ff9640 1 -- 172.21.15.148:0/2561486866 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9280021e10 con 0x5620e1763ca0 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 10 client.8340 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x5620e1505f60 con 0x5620e1763ca0 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.203+0000 7f9264ff9640 1 -- 172.21.15.148:0/2561486866 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c05cf40 con 0x5620e1763ca0 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f9264ff9640 1 -- 172.21.15.148:0/2561486866 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9280021420 con 0x5620e1763ca0 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f928ec2c740 10 client.8340 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f928ec2c740 10 client.8340 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f928ec2c740 20 client.8340 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e177f230 0x5620e17bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f928ec2c740 4 client.8340 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620e16d40d0 con 0x5620e177f230 2024-04-25T10:01:24.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f92894c4640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e177f230 0x5620e17bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f9264ff9640 10 client.8340 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.204+0000 7f92894c4640 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e177f230 0x5620e17bcf80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e60b45744597df server_cookie=4c4898ef33436e1 in_seq=0 out_seq=0 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f9264ff9640 1 -- 172.21.15.148:0/2561486866 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620e16d40d0 con 0x5620e177f230 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f9264ff9640 10 client.8340 handle_command_reply: tid=0 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 client.8340 shutdown 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 msgr2=0x5620e17a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 0x5620e17a4990 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f9280015720 tx=0x7f9280011000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 shutdown_connections 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e17a0370 0x5620e17bca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e177f230 0x5620e17bcf80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e178e760 0x5620e17a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 --2- 172.21.15.148:0/2561486866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e1763ca0 0x5620e17a4990 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f9264ff9640 0 client.8340 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 >> 172.21.15.148:0/2561486866 conn(0x5620e172b870 msgr2=0x5620e17a27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 shutdown_connections 2024-04-25T10:01:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 1 -- 172.21.15.148:0/2561486866 wait complete. 2024-04-25T10:01:24.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.205+0000 7f928ec2c740 20 client.8340 trim_cache size 0 max 16384 2024-04-25T10:01:24.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.220+0000 7f928ec2c740 1 Processor -- start 2024-04-25T10:01:24.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.220+0000 7f928ec2c740 1 -- start start 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e178a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e17a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928ec2c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9284200350 con 0x5620e178e760 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841ff510 con 0x5620e17a0370 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840adfa0 con 0x5620e1763ca0 2024-04-25T10:01:24.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e17a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e17a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33158/0 (socket says 172.21.15.148:33158) 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 -- 172.21.15.148:0/3039677080 learned_addr learned my addr 172.21.15.148:0/3039677080 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f9289cc5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e178a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f928a4c6640 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 -- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 msgr2=0x5620e178a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e178a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 -- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 msgr2=0x5620e17a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 -- 172.21.15.148:0/3039677080 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620e13741b0 con 0x5620e178e760 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f92894c4640 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e17a2950 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9274009960 tx=0x7f927402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb4f9d5c62b2939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f9266ffd640 1 -- 172.21.15.148:0/3039677080 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274037040 con 0x5620e178e760 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f9266ffd640 1 -- 172.21.15.148:0/3039677080 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9274032880 con 0x5620e178e760 2024-04-25T10:01:24.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.221+0000 7f9266ffd640 1 -- 172.21.15.148:0/3039677080 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274032c70 con 0x5620e178e760 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 -- 172.21.15.148:0/3039677080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 msgr2=0x5620e17a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e17a2950 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f9274009960 tx=0x7f927402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 -- 172.21.15.148:0/3039677080 shutdown_connections 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e17a2950 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 --2- 172.21.15.148:0/3039677080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e178a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 -- 172.21.15.148:0/3039677080 >> 172.21.15.148:0/3039677080 conn(0x5620e172b870 msgr2=0x5620e1759f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 -- 172.21.15.148:0/3039677080 shutdown_connections 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.222+0000 7f928ec2c740 1 -- 172.21.15.148:0/3039677080 wait complete. 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.223+0000 7f928ec2c740 1 -- start start 2024-04-25T10:01:24.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.223+0000 7f928ec2c740 1 Processor -- start 2024-04-25T10:01:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e17a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e1813ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840ae5a0 con 0x5620e178e760 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620e1761c90 con 0x5620e17a0370 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620e1814020 con 0x5620e1763ca0 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 10 client.0 _ll_register_callbacks cb 0x7f9280005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e1813ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e1813ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33166/0 (socket says 172.21.15.148:33166) 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 -- 172.21.15.148:0/671244626 learned_addr learned my addr 172.21.15.148:0/671244626 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928a4c6640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f9289cc5640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e17a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 -- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 msgr2=0x5620e17a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e17a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 -- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 msgr2=0x5620e17c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 -- 172.21.15.148:0/671244626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620e1814810 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f92894c4640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e1813ae0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f927402a860 tx=0x7f9274004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa378d04213a8af2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f9264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f9264ff9640 1 -- 172.21.15.148:0/671244626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274037050 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f9264ff9640 1 -- 172.21.15.148:0/671244626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9274032a30 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.224+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9274032880 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f9264ff9640 1 -- 172.21.15.148:0/671244626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9274058050 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 10 client.8346 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x5620e180a620 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f9264ff9640 1 -- 172.21.15.148:0/671244626 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9274032880 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f9264ff9640 1 -- 172.21.15.148:0/671244626 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f927405b020 con 0x5620e178e760 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 10 client.8346 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 10 client.8346 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:24.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 20 client.8346 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e17c9760 0x5620e17c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 4 client.8346 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5620e16ce420 con 0x5620e17c9760 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f9289cc5640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e17c9760 0x5620e17c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f9264ff9640 10 client.8346 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.225+0000 7f9289cc5640 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e17c9760 0x5620e17c9b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0235f5264033527 server_cookie=1f253284899b0b0c in_seq=0 out_seq=0 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f9264ff9640 1 -- 172.21.15.148:0/671244626 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5620e16ce420 con 0x5620e17c9760 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f9264ff9640 10 client.8346 handle_command_reply: tid=0 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 client.8346 shutdown 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 msgr2=0x5620e1813ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e1813ae0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f927402a860 tx=0x7f9274004730 comp rx=0 tx=0).stop 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 shutdown_connections 2024-04-25T10:01:24.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x5620e17c9760 0x5620e17c9b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e17a0370 0x5620e17c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f9264ff9640 0 client.8346 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620e178e760 0x5620e1813ae0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 --2- 172.21.15.148:0/671244626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5620e1763ca0 0x5620e17a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 >> 172.21.15.148:0/671244626 conn(0x5620e172b870 msgr2=0x5620e177e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 shutdown_connections 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 1 -- 172.21.15.148:0/671244626 wait complete. 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928ec2c740 20 client.8346 trim_cache size 0 max 16384 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:24.237 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:01:24.238 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:24.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926806b4d0 msgr2=0x7f926806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926806b4d0 0x7f926806d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9280007ed0 tx=0x7f9280007e60 comp rx=0 tx=0).stop 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 msgr2=0x7f928406f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 0x7f928406f460 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f927c00d800 tx=0x7f927c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 shutdown_connections 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f926806b4d0 0x7f926806d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928415de80 0x7f928406f460 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92840ccfe0 0x7f92840aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 --2- 172.21.15.148:0/4165515685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92840cadb0 0x7f92841526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.226+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 >> 172.21.15.148:0/4165515685 conn(0x7f92840b2390 msgr2=0x7f928415c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.227+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 shutdown_connections 2024-04-25T10:01:24.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.227+0000 7f928bf29640 1 -- 172.21.15.148:0/4165515685 wait complete. 2024-04-25T10:01:24.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:24.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:01:24.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 Processor -- start 2024-04-25T10:01:24.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 -- start start 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb380cadb0 0x7fcb380cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380ccfe0 0x7fcb380cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb380afc80 con 0x7fcb3815de80 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb380cb240 con 0x7fcb380cadb0 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.337+0000 7fcb3ed30640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380ce4d0 con 0x7fcb380ccfe0 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb37fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcb380cadb0 0x7fcb380cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb3cacc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcb3815de80 0x7fcb380cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49018/0 (socket says 172.21.15.148:49018) 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb3cacc640 1 -- 172.21.15.148:0/2167608807 learned_addr learned my addr 172.21.15.148:0/2167608807 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2167608807 >> v2:172.21.15.148:3300/0 conn(0x7fcb380cadb0 0x7fcb380cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33174/0 (socket says 172.21.15.148:33174) 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/2167608807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380ccfe0 0x7fcb380cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb37fff640 1 -- 172.21.15.148:0/2167608807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380ccfe0 msgr2=0x7fcb380cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2167608807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380ccfe0 0x7fcb380cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb37fff640 1 -- 172.21.15.148:0/2167608807 >> v1:172.21.15.148:6789/0 conn(0x7fcb3815de80 legacy=0x7fcb380cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.338+0000 7fcb37fff640 1 -- 172.21.15.148:0/2167608807 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380ce7b0 con 0x7fcb380cadb0 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2167608807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb380cadb0 0x7fcb380cf6c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fcb280097a0 tx=0x7fcb2802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6b599ccfe59afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb377fe640 1 -- 172.21.15.148:0/2167608807 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28037070 con 0x7fcb380cadb0 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb377fe640 1 -- 172.21.15.148:0/2167608807 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2802ad10 con 0x7fcb380cadb0 2024-04-25T10:01:24.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb377fe640 1 -- 172.21.15.148:0/2167608807 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28032660 con 0x7fcb380cadb0 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 -- 172.21.15.148:0/2167608807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb380cadb0 msgr2=0x7fcb380cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/2167608807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb380cadb0 0x7fcb380cf6c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fcb280097a0 tx=0x7fcb2802a080 comp rx=0 tx=0).stop 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 -- 172.21.15.148:0/2167608807 shutdown_connections 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/2167608807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380ccfe0 0x7fcb380cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/2167608807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb380cadb0 0x7fcb380cf6c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 -- 172.21.15.148:0/2167608807 >> 172.21.15.148:0/2167608807 conn(0x7fcb380b2390 msgr2=0x7fcb380b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 -- 172.21.15.148:0/2167608807 shutdown_connections 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.339+0000 7fcb3ed30640 1 -- 172.21.15.148:0/2167608807 wait complete. 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.340+0000 7fcb3ed30640 1 Processor -- start 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 -- start start 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380cadb0 0x7fcb3814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb380ccfe0 0x7fcb381598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 0x7fcb38159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3815e260 con 0x7fcb3815de80 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380cfc00 con 0x7fcb380ccfe0 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3ed30640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380b2eb0 con 0x7fcb380cadb0 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.341+0000 7fcb3d2cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 0x7fcb38159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 0x7fcb38159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33188/0 (socket says 172.21.15.148:33188) 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 -- 172.21.15.148:0/3590478345 learned_addr learned my addr 172.21.15.148:0/3590478345 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb37fff640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb380ccfe0 0x7fcb381598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380cadb0 0x7fcb3814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 -- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380cadb0 msgr2=0x7fcb3814f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380cadb0 0x7fcb3814f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 -- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb380ccfe0 msgr2=0x7fcb381598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb380ccfe0 0x7fcb381598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 -- 172.21.15.148:0/3590478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3814fa40 con 0x7fcb3815de80 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb37fff640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb380ccfe0 0x7fcb381598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 0x7fcb38159e30 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000e930 tx=0x7fcb2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb357fa640 1 -- 172.21.15.148:0/3590478345 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20017070 con 0x7fcb3815de80 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb357fa640 1 -- 172.21.15.148:0/3590478345 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2000c980 con 0x7fcb3815de80 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb38156eb0 con 0x7fcb3815de80 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb357fa640 1 -- 172.21.15.148:0/3590478345 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000cd70 con 0x7fcb3815de80 2024-04-25T10:01:24.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.342+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb38157470 con 0x7fcb3815de80 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.344+0000 7fcb357fa640 1 -- 172.21.15.148:0/3590478345 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb2001a070 con 0x7fcb3815de80 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.344+0000 7fcb357fa640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb1806b4d0 0x7fcb1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.344+0000 7fcb357fa640 1 -- 172.21.15.148:0/3590478345 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb2008a350 con 0x7fcb3815de80 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.344+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb1806b4d0 0x7fcb1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.345+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb1806b4d0 0x7fcb1806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb300012e0 tx=0x7fcb30001810 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.345+0000 7fcb41a32740 1 Processor -- start 2024-04-25T10:01:24.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.345+0000 7fcb41a32740 1 -- start start 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c359320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c359860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c329770 0x56366c32cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb41a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366c292da0 con 0x56366c356ef0 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366c23bb80 con 0x56366c329770 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56366c28ee40 con 0x56366c356b10 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c359860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c359860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33202/0 (socket says 172.21.15.148:33202) 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 -- 172.21.15.148:0/847130332 learned_addr learned my addr 172.21.15.148:0/847130332 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c329770 0x56366c32cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c359320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 -- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 msgr2=0x56366c359320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c359320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 -- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c329770 msgr2=0x56366c32cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c329770 0x56366c32cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 -- 172.21.15.148:0/847130332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56366c245f10 con 0x56366c356ef0 2024-04-25T10:01:24.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb37fff640 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c359860 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fcb28035040 tx=0x7fcb2802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41317ae3aa3139bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb2e7fc640 1 -- 172.21.15.148:0/847130332 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28037040 con 0x56366c356ef0 2024-04-25T10:01:24.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb2e7fc640 1 -- 172.21.15.148:0/847130332 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28004040 con 0x56366c356ef0 2024-04-25T10:01:24.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.346+0000 7fcb2e7fc640 1 -- 172.21.15.148:0/847130332 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004410 con 0x56366c356ef0 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 -- 172.21.15.148:0/847130332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 msgr2=0x56366c359860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c359860 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fcb28035040 tx=0x7fcb2802c000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 -- 172.21.15.148:0/847130332 shutdown_connections 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c329770 0x56366c32cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c359860 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 --2- 172.21.15.148:0/847130332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c359320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 -- 172.21.15.148:0/847130332 >> 172.21.15.148:0/847130332 conn(0x56366c32b9e0 msgr2=0x56366c374320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 -- 172.21.15.148:0/847130332 shutdown_connections 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.347+0000 7fcb41a32740 1 -- 172.21.15.148:0/847130332 wait complete. 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.348+0000 7fcb41a32740 1 -- start start 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.348+0000 7fcb41a32740 1 Processor -- start 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c3531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c35ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c379f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366c32eb90 con 0x56366c356ef0 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366c2314b0 con 0x56366c34c6a0 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56366c32ee70 con 0x56366c356b10 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 10 client.0 _ll_register_callbacks cb 0x56366c271180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c35ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c35ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33210/0 (socket says 172.21.15.148:33210) 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 -- 172.21.15.148:0/3498629431 learned_addr learned my addr 172.21.15.148:0/3498629431 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c3531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c379f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 -- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 msgr2=0x56366c3531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c3531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 -- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 msgr2=0x56366c379f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c379f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 -- 172.21.15.148:0/3498629431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56366c0717d0 con 0x56366c356ef0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb37fff640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c35ca60 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802a670 tx=0x7fcb28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af6c2eb91143e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb177fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb177fe640 1 -- 172.21.15.148:0/3498629431 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28037040 con 0x56366c356ef0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb177fe640 1 -- 172.21.15.148:0/3498629431 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb28003ae0 con 0x56366c356ef0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb28004040 con 0x56366c356ef0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb177fe640 1 -- 172.21.15.148:0/3498629431 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb28003ce0 con 0x56366c356ef0 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 10 client.8361 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:24.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.349+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x56366bfa73f0 con 0x56366c356ef0 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb177fe640 1 -- 172.21.15.148:0/3498629431 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28004040 con 0x56366c356ef0 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb177fe640 1 -- 172.21.15.148:0/3498629431 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb28053080 con 0x56366c356ef0 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb41a32740 10 client.8361 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb41a32740 10 client.8361 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb41a32740 20 client.8361 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb41a32740 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c378040 0x56366c3556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb41a32740 4 client.8361 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56366c2a02e0 con 0x56366c378040 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c378040 0x56366c3556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb177fe640 10 client.8361 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.350+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c378040 0x56366c3556c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a51e004e14fa9df server_cookie=69709c479a877791 in_seq=0 out_seq=0 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb177fe640 1 -- 172.21.15.148:0/3498629431 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56366c2a02e0 con 0x56366c378040 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb177fe640 10 client.8361 handle_command_reply: tid=0 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 client.8361 shutdown 2024-04-25T10:01:24.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 msgr2=0x56366c35ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c35ca60 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802a670 tx=0x7fcb28004730 comp rx=0 tx=0).stop 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 shutdown_connections 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c378040 0x56366c3556c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c379f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356ef0 0x56366c35ca60 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 --2- 172.21.15.148:0/3498629431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356b10 0x56366c3531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb177fe640 0 client.8361 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 >> 172.21.15.148:0/3498629431 conn(0x56366c329770 msgr2=0x56366c377460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 shutdown_connections 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 1 -- 172.21.15.148:0/3498629431 wait complete. 2024-04-25T10:01:24.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.351+0000 7fcb41a32740 20 client.8361 trim_cache size 0 max 16384 2024-04-25T10:01:24.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.366+0000 7fcb41a32740 1 Processor -- start 2024-04-25T10:01:24.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.366+0000 7fcb41a32740 1 -- start start 2024-04-25T10:01:24.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c370fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c37ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356ef0 0x56366c37bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb41a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb38200310 con 0x56366c356b10 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb381ff4d0 con 0x56366c34c6a0 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380c8520 con 0x56366c356ef0 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c37ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c37ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33220/0 (socket says 172.21.15.148:33220) 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 -- 172.21.15.148:0/2925078075 learned_addr learned my addr 172.21.15.148:0/2925078075 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c370fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356ef0 0x56366c37bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 -- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356ef0 msgr2=0x56366c37bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356ef0 0x56366c37bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 -- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 msgr2=0x56366c370fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c370fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 -- 172.21.15.148:0/2925078075 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56366bf670f0 con 0x56366c356b10 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c370fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c37ad00 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009160 tx=0x7fcb28034070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2426234bf74ea7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb2dffb640 1 -- 172.21.15.148:0/2925078075 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28037040 con 0x56366c356b10 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb2dffb640 1 -- 172.21.15.148:0/2925078075 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2802ad10 con 0x56366c356b10 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.367+0000 7fcb2dffb640 1 -- 172.21.15.148:0/2925078075 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28000e60 con 0x56366c356b10 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 -- 172.21.15.148:0/2925078075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 msgr2=0x56366c37ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c37ad00 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009160 tx=0x7fcb28034070 comp rx=0 tx=0).stop 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 -- 172.21.15.148:0/2925078075 shutdown_connections 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c356ef0 0x56366c37bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c37ad00 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2925078075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c34c6a0 0x56366c370fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 -- 172.21.15.148:0/2925078075 >> 172.21.15.148:0/2925078075 conn(0x56366c329770 msgr2=0x56366c35c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 -- 172.21.15.148:0/2925078075 shutdown_connections 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 -- 172.21.15.148:0/2925078075 wait complete. 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.368+0000 7fcb41a32740 1 -- start start 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 Processor -- start 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c34c6a0 0x56366c3ddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c36a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c356ef0 0x56366c3533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb381fef00 con 0x56366c356b10 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56366c35d600 con 0x56366c356ef0 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56366c34b1c0 con 0x56366c34c6a0 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.369+0000 7fcb41a32740 10 client.0 _ll_register_callbacks cb 0x7fcb28035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:24.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c36a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c36a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33224/0 (socket says 172.21.15.148:33224) 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 -- 172.21.15.148:0/2665343670 learned_addr learned my addr 172.21.15.148:0/2665343670 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb3d2cd640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c356ef0 0x56366c3533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c34c6a0 0x56366c3ddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 -- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c34c6a0 msgr2=0x56366c3ddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c34c6a0 0x56366c3ddf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 -- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c356ef0 msgr2=0x56366c3533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c356ef0 0x56366c3533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 -- 172.21.15.148:0/2665343670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56366c32a050 con 0x56366c356b10 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb37fff640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c36a420 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802a670 tx=0x7fcb2802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7e9461b6af06c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb177fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:24.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb177fe640 1 -- 172.21.15.148:0/2665343670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28037040 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb177fe640 1 -- 172.21.15.148:0/2665343670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb2805bc40 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb2802ad10 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb177fe640 1 -- 172.21.15.148:0/2665343670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb2805be40 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb41a32740 10 client.8367 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x56366c32a050 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb177fe640 1 -- 172.21.15.148:0/2665343670 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2802ad10 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.370+0000 7fcb177fe640 1 -- 172.21.15.148:0/2665343670 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb28053070 con 0x56366c356b10 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 10 client.8367 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 10 client.8367 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 20 client.8367 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c374f00 0x56366c3742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 4 client.8367 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56366c330b10 con 0x56366c374f00 2024-04-25T10:01:24.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c374f00 0x56366c3742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb177fe640 10 client.8367 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb3cacc640 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c374f00 0x56366c3742c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53a94e19d21d8e4 server_cookie=9207f54ec8d6b72c in_seq=0 out_seq=0 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb177fe640 1 -- 172.21.15.148:0/2665343670 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x56366c330b10 con 0x56366c374f00 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb177fe640 10 client.8367 handle_command_reply: tid=0 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 client.8367 shutdown 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 msgr2=0x56366c36a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c36a420 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802a670 tx=0x7fcb2802e000 comp rx=0 tx=0).stop 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 shutdown_connections 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x56366c374f00 0x56366c3742c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56366c356ef0 0x56366c3533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366c356b10 0x56366c36a420 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb177fe640 0 client.8367 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 --2- 172.21.15.148:0/2665343670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56366c34c6a0 0x56366c3ddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.371+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 >> 172.21.15.148:0/2665343670 conn(0x56366c329770 msgr2=0x56366c37b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 shutdown_connections 2024-04-25T10:01:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb41a32740 1 -- 172.21.15.148:0/2665343670 wait complete. 2024-04-25T10:01:24.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb41a32740 20 client.8367 trim_cache size 0 max 16384 2024-04-25T10:01:24.389 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:24.389 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:01:24.389 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout: "fcd45599-579f-4875-8c15-54c8e8662918": { 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "fcd45599-579f-4875-8c15-54c8e8662918", 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:24.390 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:24.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb1806b4d0 msgr2=0x7fcb1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb1806b4d0 0x7fcb1806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb300012e0 tx=0x7fcb30001810 comp rx=0 tx=0).stop 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 msgr2=0x7fcb38159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 0x7fcb38159e30 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000e930 tx=0x7fcb2000edf0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 shutdown_connections 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb1806b4d0 0x7fcb1806d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb3815de80 0x7fcb38159e30 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb380ccfe0 0x7fcb381598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 --2- 172.21.15.148:0/3590478345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb380cadb0 0x7fcb3814f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 >> 172.21.15.148:0/3590478345 conn(0x7fcb380b2390 msgr2=0x7fcb380cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 shutdown_connections 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.372+0000 7fcb3ed30640 1 -- 172.21.15.148:0/3590478345 wait complete. 2024-04-25T10:01:24.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:24.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.485+0000 7fd8609f9640 1 Processor -- start 2024-04-25T10:01:24.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.485+0000 7fd8609f9640 1 -- start start 2024-04-25T10:01:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.486+0000 7fd8609f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 0x7fd85c06c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.486+0000 7fd8609f9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c06b0e0 con 0x7fd85c06f790 2024-04-25T10:01:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.486+0000 7fd8609f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c06b3c0 con 0x7fd85c0b2e80 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.486+0000 7fd8609f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c06b6a0 con 0x7fd85c15dc20 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd859d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd85c06f790 0x7fd85c0ace60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49030/0 (socket says 172.21.15.148:49030) 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd859d9b640 1 -- 172.21.15.148:0/2382384625 learned_addr learned my addr 172.21.15.148:0/2382384625 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 0x7fd85c06c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084995371 0 0) 0x7fd85c06b0e0 con 0x7fd85c06f790 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd844003520 con 0x7fd85c06f790 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674825861 0 0) 0x7fd844003520 con 0x7fd85c06f790 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 msgr2=0x7fd85c06c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 0x7fd85c06c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 >> v1:172.21.15.188:6789/0 conn(0x7fd85c0b2e80 legacy=0x7fd85c0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c06b980 con 0x7fd85c06f790 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd850001670 con 0x7fd85c06f790 2024-04-25T10:01:24.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 >> v1:172.21.15.148:6789/0 conn(0x7fd85c06f790 legacy=0x7fd85c0ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 0x7fd8440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd844003f20 0x7fd844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd844004d40 0x7fd8440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c06b6a0 con 0x7fd844004d40 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84400ba20 con 0x7fd844003f20 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84400bc20 con 0x7fd85c15dc20 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd844004d40 0x7fd8440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85a59c640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 0x7fd8440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd859d9b640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd844003f20 0x7fd844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 -- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 msgr2=0x7fd8440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd85c15dc20 0x7fd8440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 -- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd844003f20 msgr2=0x7fd844004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd844003f20 0x7fd844004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd85ad9d640 1 -- 172.21.15.148:0/2382384625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd844003520 con 0x7fd844004d40 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.487+0000 7fd859d9b640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd844003f20 0x7fd844004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.488+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd844004d40 0x7fd8440051c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd854009a90 tx=0x7fd844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa4c8b6b85b8243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.488+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854036070 con 0x7fd844004d40 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.488+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854031400 con 0x7fd844004d40 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.488+0000 7fd85959a640 1 -- 172.21.15.148:0/2382384625 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854009690 con 0x7fd844004d40 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd8609f9640 1 -- 172.21.15.148:0/2382384625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd844004d40 msgr2=0x7fd8440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd8609f9640 1 --2- 172.21.15.148:0/2382384625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd844004d40 0x7fd8440051c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd854009a90 tx=0x7fd844005830 comp rx=0 tx=0).stop 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd85ad9d640 1 -- 172.21.15.148:0/2382384625 reap_dead start 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd8609f9640 1 -- 172.21.15.148:0/2382384625 shutdown_connections 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd8609f9640 1 -- 172.21.15.148:0/2382384625 >> 172.21.15.148:0/2382384625 conn(0x7fd85c0cfe10 msgr2=0x7fd85c0d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd8609f9640 1 -- 172.21.15.148:0/2382384625 shutdown_connections 2024-04-25T10:01:24.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.489+0000 7fd8609f9640 1 -- 172.21.15.148:0/2382384625 wait complete. 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.490+0000 7fd8609f9640 1 Processor -- start 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.490+0000 7fd8609f9640 1 -- start start 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd8609f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd844004d40 0x7fd85c0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd8609f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd85c160a10 0x7fd85c0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd8609f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 0x7fd85c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd8609f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c06ad60 con 0x7fd85c155b00 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd8609f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c06abe0 con 0x7fd85c160a10 2024-04-25T10:01:24.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd8609f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c06dc00 con 0x7fd844004d40 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 0x7fd85c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 0x7fd85c155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33232/0 (socket says 172.21.15.148:33232) 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 -- 172.21.15.148:0/1408181083 learned_addr learned my addr 172.21.15.148:0/1408181083 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd859d9b640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd85c160a10 0x7fd85c0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85a59c640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd844004d40 0x7fd85c0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 -- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd844004d40 msgr2=0x7fd85c0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd844004d40 0x7fd85c0cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 -- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd85c160a10 msgr2=0x7fd85c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd85c160a10 0x7fd85c0ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.491+0000 7fd85ad9d640 1 -- 172.21.15.148:0/1408181083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0ccef0 con 0x7fd85c155b00 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.492+0000 7fd85ad9d640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 0x7fd85c155f80 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd854009a90 tx=0x7fd854004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.492+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854036040 con 0x7fd85c155b00 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.492+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854041070 con 0x7fd85c155b00 2024-04-25T10:01:24.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.492+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c0cd1d0 con 0x7fd85c155b00 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.492+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854003c80 con 0x7fd85c155b00 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.492+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0cd790 con 0x7fd85c155b00 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.493+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c073990 con 0x7fd85c155b00 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.493+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd854039070 con 0x7fd85c155b00 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.494+0000 7fd84affd640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd83c06b500 0x7fd83c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.494+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8540aa160 con 0x7fd85c155b00 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.494+0000 7fd85a59c640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd83c06b500 0x7fd83c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.494+0000 7fd85a59c640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd83c06b500 0x7fd83c06d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd84c003ea0 tx=0x7fd84c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.498+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd854078dc0 con 0x7fd85c155b00 2024-04-25T10:01:24.697 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:24.700 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.709+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd85c15afd0 con 0x7fd85c155b00 2024-04-25T10:01:24.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.709+0000 7fd84affd640 1 -- 172.21.15.148:0/1408181083 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fd8540786c0 con 0x7fd85c155b00 2024-04-25T10:01:24.714 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:24.714 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:24.714 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd83c06b500 msgr2=0x7fd83c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd83c06b500 0x7fd83c06d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd84c003ea0 tx=0x7fd84c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 msgr2=0x7fd85c155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 0x7fd85c155f80 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fd854009a90 tx=0x7fd854004730 comp rx=0 tx=0).stop 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 shutdown_connections 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd83c06b500 0x7fd83c06d9c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd85c155b00 0x7fd85c155f80 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd85c160a10 0x7fd85c0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 --2- 172.21.15.148:0/1408181083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd844004d40 0x7fd85c0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 >> 172.21.15.148:0/1408181083 conn(0x7fd85c0cfe10 msgr2=0x7fd85c0cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 shutdown_connections 2024-04-25T10:01:24.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.711+0000 7fd8609f9640 1 -- 172.21.15.148:0/1408181083 wait complete. 2024-04-25T10:01:24.727 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:24.727 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:24.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.877+0000 7fdae4589640 1 Processor -- start 2024-04-25T10:01:24.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.877+0000 7fdae4589640 1 -- start start 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae4589640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdadc0b6cb0 0x7fdadc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae4589640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdadc0ceee0 con 0x7fdadc15db70 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae4589640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdadc0d1d10 con 0x7fdadc0b5a60 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae4589640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdadc0d1ff0 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae1b24640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdadc0b6cb0 0x7fdadc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae1b24640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdadc0b6cb0 0x7fdadc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33238/0 (socket says 172.21.15.148:33238) 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae2325640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdadc15db70 0x7fdadc162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49046/0 (socket says 172.21.15.148:49046) 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae1b24640 1 -- 172.21.15.148:0/3689362290 learned_addr learned my addr 172.21.15.148:0/3689362290 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae1b24640 1 -- 172.21.15.148:0/3689362290 >> v1:172.21.15.188:6789/0 conn(0x7fdadc0b5a60 legacy=0x7fdadc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae1b24640 1 -- 172.21.15.148:0/3689362290 >> v1:172.21.15.148:6789/0 conn(0x7fdadc15db70 legacy=0x7fdadc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.878+0000 7fdae1b24640 1 -- 172.21.15.148:0/3689362290 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0d22d0 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.879+0000 7fdae1b24640 1 --2- 172.21.15.148:0/3689362290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0b5520 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a2c89af5b712d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.879+0000 7fdae1323640 1 -- 172.21.15.148:0/3689362290 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc037070 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.879+0000 7fdae1323640 1 -- 172.21.15.148:0/3689362290 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc032440 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.879+0000 7fdae1323640 1 -- 172.21.15.148:0/3689362290 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc032810 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 -- 172.21.15.148:0/3689362290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 msgr2=0x7fdadc0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 --2- 172.21.15.148:0/3689362290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0b5520 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc02a210 comp rx=0 tx=0).stop 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 -- 172.21.15.148:0/3689362290 shutdown_connections 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 --2- 172.21.15.148:0/3689362290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0b5520 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 -- 172.21.15.148:0/3689362290 >> 172.21.15.148:0/3689362290 conn(0x7fdadc0d0520 msgr2=0x7fdadc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 -- 172.21.15.148:0/3689362290 shutdown_connections 2024-04-25T10:01:24.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.880+0000 7fdae4589640 1 -- 172.21.15.148:0/3689362290 wait complete. 2024-04-25T10:01:24.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.881+0000 7fdae4589640 1 Processor -- start 2024-04-25T10:01:24.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.881+0000 7fdae4589640 1 -- start start 2024-04-25T10:01:24.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdadc0b5a60 0x7fdadc156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdadc15db70 0x7fdadc1f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0cf1c0 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0ce530 con 0x7fdadc0b5a60 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc0cf340 con 0x7fdadc15db70 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33248/0 (socket says 172.21.15.148:33248) 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 -- 172.21.15.148:0/1208688941 learned_addr learned my addr 172.21.15.148:0/1208688941 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae2325640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdadc0b5a60 0x7fdadc156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae2b26640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdadc15db70 0x7fdadc1f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 -- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdadc15db70 msgr2=0x7fdadc1f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdadc15db70 0x7fdadc1f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 -- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdadc0b5a60 msgr2=0x7fdadc156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdadc0b5a60 0x7fdadc156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:24.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 -- 172.21.15.148:0/1208688941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc1f11f0 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae2325640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdadc0b5a60 0x7fdadc156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae2b26640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdadc15db70 0x7fdadc1f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae1b24640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0c9a40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fdacc000c00 tx=0x7fdacc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc041070 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc03c870 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdadc0c7760 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc037040 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.882+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdadc0c7d20 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.883+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdadc073990 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.884+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdacc044070 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.884+0000 7fdad2ffd640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdac006b4d0 0x7fdac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.884+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdacc0aaeb0 con 0x7fdadc0b6cb0 2024-04-25T10:01:24.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.884+0000 7fdae2325640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdac006b4d0 0x7fdac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:24.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.884+0000 7fdae2325640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdac006b4d0 0x7fdac006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdad8000f60 tx=0x7fdad8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:24.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:24.888+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdacc079c30 con 0x7fdadc0b6cb0 2024-04-25T10:01:25.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.128+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdadc0d1820 con 0x7fdadc0b6cb0 2024-04-25T10:01:25.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.128+0000 7fdad2ffd640 1 -- 172.21.15.148:0/1208688941 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1025 (secure 0 0 0) 0x7fdacc079530 con 0x7fdadc0b6cb0 2024-04-25T10:01:25.133 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:25.133 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":446,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":639,"data_bytes":646440244,"bytes_used":1567342592,"bytes_avail":771526770688,"bytes_total":773094113280,"write_bytes_sec":53814114,"read_op_per_sec":0,"write_op_per_sec":52},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T09:59:57.088810+0000","services":{}},"progress_events":{}} 2024-04-25T10:01:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdac006b4d0 msgr2=0x7fdac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdac006b4d0 0x7fdac006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdad8000f60 tx=0x7fdad8009040 comp rx=0 tx=0).stop 2024-04-25T10:01:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 msgr2=0x7fdadc0c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0c9a40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fdacc000c00 tx=0x7fdacc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 shutdown_connections 2024-04-25T10:01:25.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdac006b4d0 0x7fdac006d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdadc15db70 0x7fdadc1f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdadc0b6cb0 0x7fdadc0c9a40 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 --2- 172.21.15.148:0/1208688941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdadc0b5a60 0x7fdadc156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 >> 172.21.15.148:0/1208688941 conn(0x7fdadc0d0520 msgr2=0x7fdadc0d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.130+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 shutdown_connections 2024-04-25T10:01:25.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:25.131+0000 7fdae4589640 1 -- 172.21.15.148:0/1208688941 wait complete. 2024-04-25T10:01:26.146 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 Processor -- start 2024-04-25T10:01:26.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 -- start start 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcbc15dbe0 0x7fdcbc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcbc0b92a0 0x7fdcbc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc0b97a0 con 0x7fdcbc0b2cb0 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc15dfc0 con 0x7fdcbc0b92a0 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.252+0000 7fdcc115c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc0d1a00 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcbc0b92a0 0x7fdcbc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdcbc0b92a0 0x7fdcbc0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33256/0 (socket says 172.21.15.148:33256) 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 -- 172.21.15.148:0/191874423 learned_addr learned my addr 172.21.15.148:0/191874423 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbad9d640 1 --2- 172.21.15.148:0/191874423 >> v2:172.21.15.188:3300/0 conn(0x7fdcbc15dbe0 0x7fdcbc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 -- 172.21.15.148:0/191874423 >> v2:172.21.15.188:3300/0 conn(0x7fdcbc15dbe0 msgr2=0x7fdcbc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 --2- 172.21.15.148:0/191874423 >> v2:172.21.15.188:3300/0 conn(0x7fdcbc15dbe0 0x7fdcbc0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 -- 172.21.15.148:0/191874423 >> v1:172.21.15.188:6789/0 conn(0x7fdcbc0b2cb0 legacy=0x7fdcbc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.253+0000 7fdcbb59e640 1 -- 172.21.15.148:0/191874423 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0d1c50 con 0x7fdcbc0b92a0 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcbb59e640 1 --2- 172.21.15.148:0/191874423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0d0e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc06c140 tx=0x7fdcb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a502a9e60fd3306 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcba59c640 1 -- 172.21.15.148:0/191874423 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4036070 con 0x7fdcbc0b92a0 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcba59c640 1 -- 172.21.15.148:0/191874423 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4031600 con 0x7fdcbc0b92a0 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcba59c640 1 -- 172.21.15.148:0/191874423 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40319d0 con 0x7fdcbc0b92a0 2024-04-25T10:01:26.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 -- 172.21.15.148:0/191874423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc0b92a0 msgr2=0x7fdcbc0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 --2- 172.21.15.148:0/191874423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0d0e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc06c140 tx=0x7fdcb402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 -- 172.21.15.148:0/191874423 shutdown_connections 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 --2- 172.21.15.148:0/191874423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0d0e50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 --2- 172.21.15.148:0/191874423 >> v2:172.21.15.188:3300/0 conn(0x7fdcbc15dbe0 0x7fdcbc0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 -- 172.21.15.148:0/191874423 >> 172.21.15.148:0/191874423 conn(0x7fdcbc061a20 msgr2=0x7fdcbc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 -- 172.21.15.148:0/191874423 shutdown_connections 2024-04-25T10:01:26.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.254+0000 7fdcc115c640 1 -- 172.21.15.148:0/191874423 wait complete. 2024-04-25T10:01:26.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.256+0000 7fdcc115c640 1 Processor -- start 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.256+0000 7fdcc115c640 1 -- start start 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcbc0b2cb0 0x7fdcbc0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 0x7fdcbc146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b6ab0 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0d2210 con 0x7fdcbc0b92a0 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b6c30 con 0x7fdcbc0b2cb0 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 0x7fdcbc146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 0x7fdcbc146460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33260/0 (socket says 172.21.15.148:33260) 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 -- 172.21.15.148:0/1586983435 learned_addr learned my addr 172.21.15.148:0/1586983435 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcb3fff640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbad9d640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcbc0b2cb0 0x7fdcbc0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 -- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcbc0b2cb0 msgr2=0x7fdcbc0cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcbc0b2cb0 0x7fdcbc0cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 -- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcbc0b92a0 msgr2=0x7fdcbc0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 -- 172.21.15.148:0/1586983435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0b62b0 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcb3fff640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcbb59e640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 0x7fdcbc146460 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009f40 tx=0x7fdcb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4036050 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4042070 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc0ccf80 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4009510 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.257+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc0cd540 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.258+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc159cb0 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.259+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcb4007bf0 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.259+0000 7fdcb37fe640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.259+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdcb40ab110 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.259+0000 7fdcbad9d640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.260+0000 7fdcbad9d640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009a30 tx=0x7fdcac008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.263+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcbc159cb0 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.470+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcbc073990 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.470+0000 7fdcb37fe640 1 -- 172.21.15.148:0/1586983435 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fdcb4079e90 con 0x7fdcbc15dbe0 2024-04-25T10:01:26.475 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:26.475 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:26.475 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 msgr2=0x7fdc9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009a30 tx=0x7fdcac008040 comp rx=0 tx=0).stop 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 msgr2=0x7fdcbc146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 0x7fdcbc146460 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009f40 tx=0x7fdcb4004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 shutdown_connections 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcbc15dbe0 0x7fdcbc146460 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcbc0b92a0 0x7fdcbc0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 --2- 172.21.15.148:0/1586983435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcbc0b2cb0 0x7fdcbc0cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 >> 172.21.15.148:0/1586983435 conn(0x7fdcbc061a20 msgr2=0x7fdcbc0b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 shutdown_connections 2024-04-25T10:01:26.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.472+0000 7fdcc115c640 1 -- 172.21.15.148:0/1586983435 wait complete. 2024-04-25T10:01:26.487 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:26.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:26.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.579+0000 7f9444e4c640 1 Processor -- start 2024-04-25T10:01:26.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.579+0000 7f9444e4c640 1 -- start start 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f9444e4c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f94400b2cb0 0x7f94400d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f9444e4c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400b97a0 con 0x7f944015dbe0 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f9444e4c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400d1c50 con 0x7f94400d1020 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f9444e4c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944015dfc0 con 0x7f94400b2cb0 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f943e59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f944015dbe0 0x7f94400d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49050/0 (socket says 172.21.15.148:49050) 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f943e59c640 1 -- 172.21.15.148:0/919692345 learned_addr learned my addr 172.21.15.148:0/919692345 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:26.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.580+0000 7f943dd9b640 1 --2- 172.21.15.148:0/919692345 >> v2:172.21.15.148:3300/0 conn(0x7f94400b2cb0 0x7f94400d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.581+0000 7f943dd9b640 1 -- 172.21.15.148:0/919692345 >> v1:172.21.15.188:6789/0 conn(0x7f94400d1020 legacy=0x7f94400d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.581+0000 7f943dd9b640 1 -- 172.21.15.148:0/919692345 >> v1:172.21.15.148:6789/0 conn(0x7f944015dbe0 legacy=0x7f94400d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.581+0000 7f943dd9b640 1 -- 172.21.15.148:0/919692345 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400d1f30 con 0x7f94400b2cb0 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.581+0000 7f943d59a640 1 -- 172.21.15.148:0/919692345 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833860072 0 0) 0x7f94400b97a0 con 0x7f944015dbe0 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.581+0000 7f943dd9b640 1 --2- 172.21.15.148:0/919692345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400b2cb0 0x7f94400d2730 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f94340097a0 tx=0x7f943402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2abf54e2442f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f943d59a640 1 -- 172.21.15.148:0/919692345 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434037070 con 0x7f94400b2cb0 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f943d59a640 1 -- 172.21.15.148:0/919692345 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434032440 con 0x7f94400b2cb0 2024-04-25T10:01:26.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f943d59a640 1 -- 172.21.15.148:0/919692345 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434032810 con 0x7f94400b2cb0 2024-04-25T10:01:26.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 -- 172.21.15.148:0/919692345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400b2cb0 msgr2=0x7f94400d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 --2- 172.21.15.148:0/919692345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400b2cb0 0x7f94400d2730 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f94340097a0 tx=0x7f943402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 -- 172.21.15.148:0/919692345 shutdown_connections 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 --2- 172.21.15.148:0/919692345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400b2cb0 0x7f94400d2730 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 -- 172.21.15.148:0/919692345 >> 172.21.15.148:0/919692345 conn(0x7f9440061a20 msgr2=0x7f9440061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 -- 172.21.15.148:0/919692345 shutdown_connections 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.582+0000 7f9444e4c640 1 -- 172.21.15.148:0/919692345 wait complete. 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.583+0000 7f9444e4c640 1 Processor -- start 2024-04-25T10:01:26.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 -- start start 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94400b2cb0 0x7f9440151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 0x7f94400b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f944015dbe0 0x7f94401605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b92a0 con 0x7f94400d1020 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400d1a00 con 0x7f944015dbe0 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f9444e4c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b9420 con 0x7f94400b2cb0 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f943dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 0x7f94400b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f943dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 0x7f94400b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33270/0 (socket says 172.21.15.148:33270) 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f943dd9b640 1 -- 172.21.15.148:0/853012052 learned_addr learned my addr 172.21.15.148:0/853012052 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f943e59c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94400b2cb0 0x7f9440151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.584+0000 7f943ed9d640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f944015dbe0 0x7f94401605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943dd9b640 1 -- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94400b2cb0 msgr2=0x7f9440151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943dd9b640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94400b2cb0 0x7f9440151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943dd9b640 1 -- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f944015dbe0 msgr2=0x7f94401605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943dd9b640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f944015dbe0 0x7f94401605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943dd9b640 1 -- 172.21.15.148:0/853012052 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400b31c0 con 0x7f94400d1020 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943ed9d640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f944015dbe0 0x7f94401605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f943dd9b640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 0x7f94400b52a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f94340098c0 tx=0x7f94340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434041070 con 0x7f94400d1020 2024-04-25T10:01:26.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943403c870 con 0x7f94400d1020 2024-04-25T10:01:26.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94400b34a0 con 0x7f94400d1020 2024-04-25T10:01:26.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434037040 con 0x7f94400d1020 2024-04-25T10:01:26.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.585+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94400b3a60 con 0x7f94400d1020 2024-04-25T10:01:26.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.586+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9440073990 con 0x7f94400d1020 2024-04-25T10:01:26.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.586+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9434044070 con 0x7f94400d1020 2024-04-25T10:01:26.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.587+0000 7f942effd640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f942006b4d0 0x7f942006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:26.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.587+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94340ab010 con 0x7f94400d1020 2024-04-25T10:01:26.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.587+0000 7f943e59c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f942006b4d0 0x7f942006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:26.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.587+0000 7f943e59c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f942006b4d0 0x7f942006d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f94380012e0 tx=0x7f9438005eb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:26.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.591+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9434079e30 con 0x7f94400d1020 2024-04-25T10:01:26.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.829+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f94400c9f80 con 0x7f94400d1020 2024-04-25T10:01:26.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.829+0000 7f942effd640 1 -- 172.21.15.148:0/853012052 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1025 (secure 0 0 0) 0x7f9434079730 con 0x7f94400d1020 2024-04-25T10:01:26.834 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:26.834 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":447,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":738,"data_bytes":750249268,"bytes_used":1751891968,"bytes_avail":771342221312,"bytes_total":773094113280,"write_bytes_sec":62463944,"read_op_per_sec":0,"write_op_per_sec":60},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T09:59:57.088810+0000","services":{}},"progress_events":{}} 2024-04-25T10:01:26.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f942006b4d0 msgr2=0x7f942006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f942006b4d0 0x7f942006d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f94380012e0 tx=0x7f9438005eb0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 msgr2=0x7f94400b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:26.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 0x7f94400b52a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f94340098c0 tx=0x7f94340046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 shutdown_connections 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f942006b4d0 0x7f942006d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f944015dbe0 0x7f94401605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94400d1020 0x7f94400b52a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 --2- 172.21.15.148:0/853012052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94400b2cb0 0x7f9440151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 >> 172.21.15.148:0/853012052 conn(0x7f9440061a20 msgr2=0x7f94400b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:26.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 shutdown_connections 2024-04-25T10:01:26.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:26.831+0000 7f9444e4c640 1 -- 172.21.15.148:0/853012052 wait complete. 2024-04-25T10:01:27.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:27.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.952+0000 7fb29803f640 1 Processor -- start 2024-04-25T10:01:27.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.952+0000 7fb29803f640 1 -- start start 2024-04-25T10:01:27.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb29803f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29006abe0 0x7fb29006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb29803f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb29015d370 0x7fb29015d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb29803f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2901626e0 con 0x7fb2900ce330 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb29803f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29006b500 con 0x7fb29015d370 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb29803f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb290070270 con 0x7fb29006abe0 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb2965dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb29015d370 0x7fb29015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb2965dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb29015d370 0x7fb29015d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33284/0 (socket says 172.21.15.148:33284) 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb2965dc640 1 -- 172.21.15.148:0/2092483219 learned_addr learned my addr 172.21.15.148:0/2092483219 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb295ddb640 1 --2- 172.21.15.148:0/2092483219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29006abe0 0x7fb29006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb2965dc640 1 -- 172.21.15.148:0/2092483219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29006abe0 msgr2=0x7fb29006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb2965dc640 1 --2- 172.21.15.148:0/2092483219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29006abe0 0x7fb29006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.953+0000 7fb2965dc640 1 -- 172.21.15.148:0/2092483219 >> v1:172.21.15.188:6789/0 conn(0x7fb2900ce330 legacy=0x7fb29006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.954+0000 7fb2965dc640 1 -- 172.21.15.148:0/2092483219 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900cd270 con 0x7fb29015d370 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.954+0000 7fb2965dc640 1 --2- 172.21.15.148:0/2092483219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29015d370 0x7fb29015d800 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e50 tx=0x7fb28802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c379f5ddeb900a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.954+0000 7fb294dd9640 1 -- 172.21.15.148:0/2092483219 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288036070 con 0x7fb29015d370 2024-04-25T10:01:27.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.954+0000 7fb294dd9640 1 -- 172.21.15.148:0/2092483219 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288031600 con 0x7fb29015d370 2024-04-25T10:01:27.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.954+0000 7fb294dd9640 1 -- 172.21.15.148:0/2092483219 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880319d0 con 0x7fb29015d370 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 -- 172.21.15.148:0/2092483219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29015d370 msgr2=0x7fb29015d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 --2- 172.21.15.148:0/2092483219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29015d370 0x7fb29015d800 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e50 tx=0x7fb28802a630 comp rx=0 tx=0).stop 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 -- 172.21.15.148:0/2092483219 shutdown_connections 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 --2- 172.21.15.148:0/2092483219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29015d370 0x7fb29015d800 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 --2- 172.21.15.148:0/2092483219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29006abe0 0x7fb29006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 -- 172.21.15.148:0/2092483219 >> 172.21.15.148:0/2092483219 conn(0x7fb2900b2220 msgr2=0x7fb2900b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 -- 172.21.15.148:0/2092483219 shutdown_connections 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.955+0000 7fb29803f640 1 -- 172.21.15.148:0/2092483219 wait complete. 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.956+0000 7fb29803f640 1 Processor -- start 2024-04-25T10:01:27.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.956+0000 7fb29803f640 1 -- start start 2024-04-25T10:01:27.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb29803f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 0x7fb29015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb29803f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2900ce330 0x7fb2901483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb29803f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29015d370 0x7fb29014a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb29803f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900cdb10 con 0x7fb29006abe0 2024-04-25T10:01:27.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb29803f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900cd830 con 0x7fb29015d370 2024-04-25T10:01:27.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb29803f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb290161780 con 0x7fb2900ce330 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb295ddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 0x7fb29015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb295ddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 0x7fb29015efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33296/0 (socket says 172.21.15.148:33296) 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.957+0000 7fb295ddb640 1 -- 172.21.15.148:0/1172773992 learned_addr learned my addr 172.21.15.148:0/1172773992 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb2965dc640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29015d370 0x7fb29014a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb2955da640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2900ce330 0x7fb2901483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb295ddb640 1 -- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2900ce330 msgr2=0x7fb2901483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb295ddb640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2900ce330 0x7fb2901483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb295ddb640 1 -- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29015d370 msgr2=0x7fb29014a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb295ddb640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29015d370 0x7fb29014a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb295ddb640 1 -- 172.21.15.148:0/1172773992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290148e70 con 0x7fb29006abe0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb295ddb640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 0x7fb29015efb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c820 tx=0x7fb28c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c015070 con 0x7fb29006abe0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c004020 con 0x7fb29006abe0 2024-04-25T10:01:27.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb290149150 con 0x7fb29006abe0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0043f0 con 0x7fb29006abe0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.958+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb290149710 con 0x7fb29006abe0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.959+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2900afe50 con 0x7fb29006abe0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.959+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb28c007830 con 0x7fb29006abe0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.960+0000 7fb2867fc640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb27406b4d0 0x7fb27406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.960+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb28c08a1b0 con 0x7fb29006abe0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.960+0000 7fb2955da640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb27406b4d0 0x7fb27406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:27.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.960+0000 7fb2955da640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb27406b4d0 0x7fb27406d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb2900b09c0 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:27.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:27.964+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb28c058ea0 con 0x7fb29006abe0 2024-04-25T10:01:28.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.170+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2900b6670 con 0x7fb29006abe0 2024-04-25T10:01:28.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.170+0000 7fb2867fc640 1 -- 172.21.15.148:0/1172773992 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fb28c0587a0 con 0x7fb29006abe0 2024-04-25T10:01:28.175 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:28.175 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:28.175 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb27406b4d0 msgr2=0x7fb27406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb27406b4d0 0x7fb27406d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb2900b09c0 tx=0x7fb280009040 comp rx=0 tx=0).stop 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 msgr2=0x7fb29015efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 0x7fb29015efb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00c820 tx=0x7fb28c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 shutdown_connections 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb27406b4d0 0x7fb27406d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb29015d370 0x7fb29014a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2900ce330 0x7fb2901483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 --2- 172.21.15.148:0/1172773992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb29006abe0 0x7fb29015efb0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 >> 172.21.15.148:0/1172773992 conn(0x7fb2900b2220 msgr2=0x7fb2900b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 shutdown_connections 2024-04-25T10:01:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.172+0000 7fb29803f640 1 -- 172.21.15.148:0/1172773992 wait complete. 2024-04-25T10:01:28.188 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:28.188 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.279+0000 7fd6cd84d640 1 Processor -- start 2024-04-25T10:01:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.279+0000 7fd6cd84d640 1 -- start start 2024-04-25T10:01:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6cd84d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 0x7fd6c80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6cd84d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6c806ce10 0x7fd6c80cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6cd84d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c806c220 con 0x7fd6c809a450 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6cd84d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c809a8b0 con 0x7fd6c806ce10 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6cd84d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80d0c90 con 0x7fd6c815dc70 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6c67fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd6c809a450 0x7fd6c806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49054/0 (socket says 172.21.15.148:49054) 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.280+0000 7fd6c67fc640 1 -- 172.21.15.148:0/3386264248 learned_addr learned my addr 172.21.15.148:0/3386264248 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c6ffd640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 0x7fd6c80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/3386264248 >> v2:172.21.15.188:3300/0 conn(0x7fd6c806ce10 0x7fd6c80cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519895240 0 0) 0x7fd6c806c220 con 0x7fd6c809a450 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4003520 con 0x7fd6c809a450 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594277294 0 0) 0x7fd6b4003520 con 0x7fd6c809a450 2024-04-25T10:01:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 msgr2=0x7fd6c80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 0x7fd6c80d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 >> v2:172.21.15.188:3300/0 conn(0x7fd6c806ce10 msgr2=0x7fd6c80cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 --2- 172.21.15.148:0/3386264248 >> v2:172.21.15.188:3300/0 conn(0x7fd6c806ce10 0x7fd6c80cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80d0080 con 0x7fd6c809a450 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd6b0001670 con 0x7fd6c809a450 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 >> v1:172.21.15.148:6789/0 conn(0x7fd6c809a450 legacy=0x7fd6c806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 0x7fd6b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6b4003f30 0x7fd6b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c809a8b0 con 0x7fd6b4004d40 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80d0c90 con 0x7fd6b4003f30 2024-04-25T10:01:28.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400b9c0 con 0x7fd6c815dc70 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c67fc640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6b4003f30 0x7fd6b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c6ffd640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 0x7fd6b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 -- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 msgr2=0x7fd6b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c815dc70 0x7fd6b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 -- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6b4003f30 msgr2=0x7fd6b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6b4003f30 0x7fd6b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.281+0000 7fd6c77fe640 1 -- 172.21.15.148:0/3386264248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4003520 con 0x7fd6b4004d40 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.282+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009ac0 tx=0x7fd6b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d96bff911da38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.282+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0036070 con 0x7fd6b4004d40 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.282+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0031400 con 0x7fd6b4004d40 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.282+0000 7fd6c5ffb640 1 -- 172.21.15.148:0/3386264248 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0009690 con 0x7fd6b4004d40 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6cd84d640 1 -- 172.21.15.148:0/3386264248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b4004d40 msgr2=0x7fd6b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/3386264248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009ac0 tx=0x7fd6b4005830 comp rx=0 tx=0).stop 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6c77fe640 1 -- 172.21.15.148:0/3386264248 reap_dead start 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6cd84d640 1 -- 172.21.15.148:0/3386264248 shutdown_connections 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6cd84d640 1 -- 172.21.15.148:0/3386264248 >> 172.21.15.148:0/3386264248 conn(0x7fd6c80b3270 msgr2=0x7fd6c80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6cd84d640 1 -- 172.21.15.148:0/3386264248 shutdown_connections 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.283+0000 7fd6cd84d640 1 -- 172.21.15.148:0/3386264248 wait complete. 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.284+0000 7fd6cd84d640 1 Processor -- start 2024-04-25T10:01:28.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.284+0000 7fd6cd84d640 1 -- start start 2024-04-25T10:01:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6cd84d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6b4004d40 0x7fd6c8160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6cd84d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c80c7e40 0x7fd6c80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6cd84d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 0x7fd6c8157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6cd84d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80d0920 con 0x7fd6c8157a80 2024-04-25T10:01:28.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6cd84d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80d0640 con 0x7fd6c80c7e40 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6cd84d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815e050 con 0x7fd6b4004d40 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 0x7fd6c8157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 0x7fd6c8157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33320/0 (socket says 172.21.15.148:33320) 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 -- 172.21.15.148:0/172438728 learned_addr learned my addr 172.21.15.148:0/172438728 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c67fc640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c80c7e40 0x7fd6c80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c6ffd640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6b4004d40 0x7fd6c8160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 -- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6b4004d40 msgr2=0x7fd6c8160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6b4004d40 0x7fd6c8160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 -- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c80c7e40 msgr2=0x7fd6c80cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c80c7e40 0x7fd6c80cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 -- 172.21.15.148:0/172438728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80c61d0 con 0x7fd6c8157a80 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.285+0000 7fd6c77fe640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 0x7fd6c8157f00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009be0 tx=0x7fd6c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.286+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0036040 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.286+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0041070 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.286+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0009690 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.286+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c80c64b0 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.286+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80c6a70 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.287+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c8073990 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.287+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6c0039070 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.287+0000 7fd6bffff640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6a806b4d0 0x7fd6a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.287+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6c00a9e10 con 0x7fd6c8157a80 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.287+0000 7fd6c6ffd640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6a806b4d0 0x7fd6a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:28.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.288+0000 7fd6c6ffd640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6a806b4d0 0x7fd6a806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8007520 tx=0x7fd6b800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:28.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.292+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6c0078a70 con 0x7fd6c8157a80 2024-04-25T10:01:28.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.525+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd6c8158440 con 0x7fd6c8157a80 2024-04-25T10:01:28.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.526+0000 7fd6bffff640 1 -- 172.21.15.148:0/172438728 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7fd6c0078370 con 0x7fd6c8157a80 2024-04-25T10:01:28.530 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:28.530 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076490554,"bytes_used":2542919680,"bytes_avail":770551193600,"bytes_total":773094113280,"write_bytes_sec":88689750,"read_op_per_sec":0,"write_op_per_sec":86},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-25T09:59:57.088810+0000","services":{}},"progress_events":{}} 2024-04-25T10:01:28.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.527+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6a806b4d0 msgr2=0x7fd6a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.527+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6a806b4d0 0x7fd6a806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8007520 tx=0x7fd6b800b040 comp rx=0 tx=0).stop 2024-04-25T10:01:28.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.527+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 msgr2=0x7fd6c8157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:28.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.527+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 0x7fd6c8157f00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009be0 tx=0x7fd6c0004730 comp rx=0 tx=0).stop 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 shutdown_connections 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6a806b4d0 0x7fd6a806d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c8157a80 0x7fd6c8157f00 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c80c7e40 0x7fd6c80cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 --2- 172.21.15.148:0/172438728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6b4004d40 0x7fd6c8160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 >> 172.21.15.148:0/172438728 conn(0x7fd6c80b3270 msgr2=0x7fd6c80b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 shutdown_connections 2024-04-25T10:01:28.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:28.528+0000 7fd6cd84d640 1 -- 172.21.15.148:0/172438728 wait complete. 2024-04-25T10:01:29.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:29.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.648+0000 7f72dc7f9640 1 Processor -- start 2024-04-25T10:01:29.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.648+0000 7f72dc7f9640 1 -- start start 2024-04-25T10:01:29.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.649+0000 7f72dc7f9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72d415db70 0x7f72d4162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.649+0000 7f72dc7f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72d40b5c30 0x7f72d40d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.649+0000 7f72dc7f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d4162690 con 0x7f72d40b6cb0 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.649+0000 7f72dc7f9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d40ceee0 con 0x7f72d40b5c30 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.649+0000 7f72dc7f9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d40d1d10 con 0x7f72d415db70 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.649+0000 7f72dad96640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72d40b5c30 0x7f72d40d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f72d40b5c30 0x7f72d40d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33324/0 (socket says 172.21.15.148:33324) 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 -- 172.21.15.148:0/3760005893 learned_addr learned my addr 172.21.15.148:0/3760005893 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72da595640 1 --2- 172.21.15.148:0/3760005893 >> v2:172.21.15.188:3300/0 conn(0x7f72d415db70 0x7f72d4162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 -- 172.21.15.148:0/3760005893 >> v2:172.21.15.188:3300/0 conn(0x7f72d415db70 msgr2=0x7f72d4162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 --2- 172.21.15.148:0/3760005893 >> v2:172.21.15.188:3300/0 conn(0x7f72d415db70 0x7f72d4162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 -- 172.21.15.148:0/3760005893 >> v1:172.21.15.188:6789/0 conn(0x7f72d40b6cb0 legacy=0x7f72d40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 -- 172.21.15.148:0/3760005893 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40d1ff0 con 0x7f72d40b5c30 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.650+0000 7f72dad96640 1 --2- 172.21.15.148:0/3760005893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40d27f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f72d40b6570 tx=0x7f72cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bb0fe7213884f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72d9593640 1 -- 172.21.15.148:0/3760005893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc037070 con 0x7f72d40b5c30 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72d9593640 1 -- 172.21.15.148:0/3760005893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc032440 con 0x7f72d40b5c30 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72d9593640 1 -- 172.21.15.148:0/3760005893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc032810 con 0x7f72d40b5c30 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 -- 172.21.15.148:0/3760005893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 msgr2=0x7f72d40d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/3760005893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40d27f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f72d40b6570 tx=0x7f72cc02a270 comp rx=0 tx=0).stop 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 -- 172.21.15.148:0/3760005893 shutdown_connections 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/3760005893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40d27f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/3760005893 >> v2:172.21.15.188:3300/0 conn(0x7f72d415db70 0x7f72d4162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 -- 172.21.15.148:0/3760005893 >> 172.21.15.148:0/3760005893 conn(0x7f72d40d0520 msgr2=0x7f72d40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 -- 172.21.15.148:0/3760005893 shutdown_connections 2024-04-25T10:01:29.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.651+0000 7f72dc7f9640 1 -- 172.21.15.148:0/3760005893 wait complete. 2024-04-25T10:01:29.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.652+0000 7f72dc7f9640 1 Processor -- start 2024-04-25T10:01:29.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 -- start start 2024-04-25T10:01:29.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d40b6cb0 0x7f72d41f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d415db70 0x7f72d41f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40ce530 con 0x7f72d40b5c30 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d415df50 con 0x7f72d40b6cb0 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72dc7f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40cf1c0 con 0x7f72d415db70 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72da595640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72da595640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33326/0 (socket says 172.21.15.148:33326) 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72da595640 1 -- 172.21.15.148:0/10626708 learned_addr learned my addr 172.21.15.148:0/10626708 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.653+0000 7f72d9d94640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d40b6cb0 0x7f72d41f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72da595640 1 -- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d415db70 msgr2=0x7f72d41f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72dad96640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d415db70 0x7f72d41f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72da595640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d415db70 0x7f72d41f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72da595640 1 -- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d40b6cb0 msgr2=0x7f72d41f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72da595640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d40b6cb0 0x7f72d41f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72da595640 1 -- 172.21.15.148:0/10626708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40cd1e0 con 0x7f72d40b5c30 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72d9d94640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d40b6cb0 0x7f72d41f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72dad96640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d415db70 0x7f72d41f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:29.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72da595640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40cc970 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f72d000efc0 tx=0x7f72d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0018020 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0014480 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0014850 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d40cd4c0 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.654+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d40cda80 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.655+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d4073990 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.655+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72d0005d10 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.656+0000 7f72caffd640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72b806b4d0 0x7f72b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.656+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72d008aea0 con 0x7f72d40b5c30 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.656+0000 7f72d9d94640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72b806b4d0 0x7f72b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.656+0000 7f72d9d94640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72b806b4d0 0x7f72b806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f72d41ea100 tx=0x7f72c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.660+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72d0022070 con 0x7f72d40b5c30 2024-04-25T10:01:29.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.866+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72d40d1820 con 0x7f72d40b5c30 2024-04-25T10:01:29.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.867+0000 7f72caffd640 1 -- 172.21.15.148:0/10626708 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f72d0010040 con 0x7f72d40b5c30 2024-04-25T10:01:29.871 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:29.871 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:29.872 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72b806b4d0 msgr2=0x7f72b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72b806b4d0 0x7f72b806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f72d41ea100 tx=0x7f72c4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 msgr2=0x7f72d40cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40cc970 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f72d000efc0 tx=0x7f72d000c490 comp rx=0 tx=0).stop 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 shutdown_connections 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f72b806b4d0 0x7f72b806d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72d415db70 0x7f72d41f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72d40b6cb0 0x7f72d41f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 --2- 172.21.15.148:0/10626708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72d40b5c30 0x7f72d40cc970 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 >> 172.21.15.148:0/10626708 conn(0x7f72d40d0520 msgr2=0x7f72d415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 shutdown_connections 2024-04-25T10:01:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.869+0000 7f72dc7f9640 1 -- 172.21.15.148:0/10626708 wait complete. 2024-04-25T10:01:29.884 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:29.884 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:29.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.976+0000 7fba3db5c640 1 Processor -- start 2024-04-25T10:01:29.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.976+0000 7fba3db5c640 1 -- start start 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.976+0000 7fba3db5c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba380cfec0 0x7fba380cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.976+0000 7fba3db5c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba38071480 con 0x7fba380ce3c0 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.976+0000 7fba3db5c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba38071760 con 0x7fba38154ee0 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.976+0000 7fba3db5c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba380ce830 con 0x7fba380cfec0 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba37fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fba380ce3c0 0x7fba3806d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49062/0 (socket says 172.21.15.148:49062) 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba37fff640 1 -- 172.21.15.148:0/3871144734 learned_addr learned my addr 172.21.15.148:0/3871144734 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba36ffd640 1 --2- 172.21.15.148:0/3871144734 >> v2:172.21.15.188:3300/0 conn(0x7fba380cfec0 0x7fba380cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba367fc640 1 -- 172.21.15.148:0/3871144734 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858878096 0 0) 0x7fba38071480 con 0x7fba380ce3c0 2024-04-25T10:01:29.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba367fc640 1 -- 172.21.15.148:0/3871144734 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba20003520 con 0x7fba380ce3c0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba36ffd640 1 -- 172.21.15.148:0/3871144734 >> v1:172.21.15.188:6789/0 conn(0x7fba38154ee0 legacy=0x7fba380cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba367fc640 1 -- 172.21.15.148:0/3871144734 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128665290 0 0) 0x7fba20003520 con 0x7fba380ce3c0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba36ffd640 1 -- 172.21.15.148:0/3871144734 >> v1:172.21.15.148:6789/0 conn(0x7fba380ce3c0 legacy=0x7fba3806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.977+0000 7fba36ffd640 1 -- 172.21.15.148:0/3871144734 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba38071a40 con 0x7fba380cfec0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.978+0000 7fba36ffd640 1 --2- 172.21.15.148:0/3871144734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba380cde80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f6b85ab9a94a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.978+0000 7fba367fc640 1 -- 172.21.15.148:0/3871144734 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c037070 con 0x7fba380cfec0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.978+0000 7fba367fc640 1 -- 172.21.15.148:0/3871144734 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c02ac50 con 0x7fba380cfec0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.978+0000 7fba367fc640 1 -- 172.21.15.148:0/3871144734 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba300019f0 con 0x7fba380cfec0 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 -- 172.21.15.148:0/3871144734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 msgr2=0x7fba380cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 --2- 172.21.15.148:0/3871144734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba380cde80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 -- 172.21.15.148:0/3871144734 shutdown_connections 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 --2- 172.21.15.148:0/3871144734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba380cde80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 -- 172.21.15.148:0/3871144734 >> 172.21.15.148:0/3871144734 conn(0x7fba3806abe0 msgr2=0x7fba3806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:29.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 -- 172.21.15.148:0/3871144734 shutdown_connections 2024-04-25T10:01:29.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.979+0000 7fba3db5c640 1 -- 172.21.15.148:0/3871144734 wait complete. 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.980+0000 7fba3db5c640 1 Processor -- start 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.980+0000 7fba3db5c640 1 -- start start 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 0x7fba38146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba381e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba38154ee0 0x7fba381e6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba38071230 con 0x7fba380ce3c0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba381552c0 con 0x7fba380cfec0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba38155440 con 0x7fba38154ee0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 0x7fba38146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 0x7fba38146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33338/0 (socket says 172.21.15.148:33338) 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 -- 172.21.15.148:0/4097379576 learned_addr learned my addr 172.21.15.148:0/4097379576 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba36ffd640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba381e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba37fff640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba38154ee0 0x7fba381e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 -- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba38154ee0 msgr2=0x7fba381e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba38154ee0 0x7fba381e6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 -- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 msgr2=0x7fba381e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba381e5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 -- 172.21.15.148:0/4097379576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba38154350 con 0x7fba380ce3c0 2024-04-25T10:01:29.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba37fff640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba38154ee0 0x7fba381e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba377fe640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 0x7fba38146f70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c430 tx=0x7fba2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28017070 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba280052d0 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba381e6790 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28003830 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.981+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba38156640 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.982+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba381e6cb0 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.983+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba2801a070 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.983+0000 7fba1ffff640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba2406b4d0 0x7fba2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.983+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba28089760 con 0x7fba380ce3c0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.983+0000 7fba36ffd640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba2406b4d0 0x7fba2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:29.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.984+0000 7fba36ffd640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba2406b4d0 0x7fba2406d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fba2c034000 tx=0x7fba2c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:29.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:29.988+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba28058450 con 0x7fba380ce3c0 2024-04-25T10:01:30.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.223+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fba38073990 con 0x7fba380ce3c0 2024-04-25T10:01:30.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.224+0000 7fba1ffff640 1 -- 172.21.15.148:0/4097379576 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fba28057d50 con 0x7fba380ce3c0 2024-04-25T10:01:30.228 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:30.228 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076490554,"bytes_used":2542919680,"bytes_avail":770551193600,"bytes_total":773094113280,"write_bytes_sec":83445712,"read_op_per_sec":0,"write_op_per_sec":81},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-25T10:01:29.108234+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:01:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba2406b4d0 msgr2=0x7fba2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba2406b4d0 0x7fba2406d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fba2c034000 tx=0x7fba2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 msgr2=0x7fba38146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 0x7fba38146f70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c430 tx=0x7fba2800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 shutdown_connections 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba2406b4d0 0x7fba2406d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba38154ee0 0x7fba381e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba380cfec0 0x7fba381e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 --2- 172.21.15.148:0/4097379576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba380ce3c0 0x7fba38146f70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 >> 172.21.15.148:0/4097379576 conn(0x7fba3806abe0 msgr2=0x7fba380af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 shutdown_connections 2024-04-25T10:01:30.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.226+0000 7fba3db5c640 1 -- 172.21.15.148:0/4097379576 wait complete. 2024-04-25T10:01:30.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:01:30.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:30.241 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T10:01:30.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.335+0000 7f32525e7640 1 Processor -- start 2024-04-25T10:01:30.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.335+0000 7f32525e7640 1 -- start start 2024-04-25T10:01:30.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f32525e7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f324c158230 0x7f324c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f32525e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f324c0d1b80 0x7f324c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f32525e7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324c0d2020 con 0x7f324c06c870 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f32525e7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c158610 con 0x7f324c158230 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f32525e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c06bea0 con 0x7f324c0d1b80 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f324c158230 0x7f324c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f3250b84640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f324c06c870 0x7f324c0b4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49068/0 (socket says 172.21.15.148:49068) 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f324c158230 0x7f324c0d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33342/0 (socket says 172.21.15.148:33342) 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f3250b84640 1 -- 172.21.15.148:0/896152850 learned_addr learned my addr 172.21.15.148:0/896152850 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324b7fe640 1 --2- 172.21.15.148:0/896152850 >> v2:172.21.15.188:3300/0 conn(0x7f324c0d1b80 0x7f324c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324bfff640 1 -- 172.21.15.148:0/896152850 >> v2:172.21.15.188:3300/0 conn(0x7f324c0d1b80 msgr2=0x7f324c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324bfff640 1 --2- 172.21.15.148:0/896152850 >> v2:172.21.15.188:3300/0 conn(0x7f324c0d1b80 0x7f324c06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324bfff640 1 -- 172.21.15.148:0/896152850 >> v1:172.21.15.148:6789/0 conn(0x7f324c06c870 legacy=0x7f324c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.336+0000 7f324affd640 1 -- 172.21.15.148:0/896152850 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783954781 0 0) 0x7f324c0d2020 con 0x7f324c06c870 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.337+0000 7f324bfff640 1 -- 172.21.15.148:0/896152850 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c0d2300 con 0x7f324c158230 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.337+0000 7f324b7fe640 1 --2- 172.21.15.148:0/896152850 >> v2:172.21.15.188:3300/0 conn(0x7f324c0d1b80 0x7f324c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:30.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.337+0000 7f324bfff640 1 --2- 172.21.15.148:0/896152850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c158230 0x7f324c0d1640 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f324c15bcc0 tx=0x7f323c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546904cda3d1e915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.337+0000 7f324affd640 1 -- 172.21.15.148:0/896152850 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c037070 con 0x7f324c158230 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.337+0000 7f324affd640 1 -- 172.21.15.148:0/896152850 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c032440 con 0x7f324c158230 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.337+0000 7f324affd640 1 -- 172.21.15.148:0/896152850 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c032810 con 0x7f324c158230 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 -- 172.21.15.148:0/896152850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c158230 msgr2=0x7f324c0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 --2- 172.21.15.148:0/896152850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c158230 0x7f324c0d1640 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f324c15bcc0 tx=0x7f323c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 -- 172.21.15.148:0/896152850 shutdown_connections 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 --2- 172.21.15.148:0/896152850 >> v2:172.21.15.188:3300/0 conn(0x7f324c0d1b80 0x7f324c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 --2- 172.21.15.148:0/896152850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c158230 0x7f324c0d1640 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 -- 172.21.15.148:0/896152850 >> 172.21.15.148:0/896152850 conn(0x7f324c061600 msgr2=0x7f324c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 -- 172.21.15.148:0/896152850 shutdown_connections 2024-04-25T10:01:30.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.338+0000 7f32525e7640 1 -- 172.21.15.148:0/896152850 wait complete. 2024-04-25T10:01:30.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.339+0000 7f32525e7640 1 Processor -- start 2024-04-25T10:01:30.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.339+0000 7f32525e7640 1 -- start start 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f32525e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 0x7f324c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f32525e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f324c0d1b80 0x7f324c14ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f32525e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f324c158230 0x7f324c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f32525e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c15e320 con 0x7f324c06c870 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f32525e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0d25e0 con 0x7f324c158230 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f32525e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c15e4a0 con 0x7f324c0d1b80 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 0x7f324c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 0x7f324c0cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33352/0 (socket says 172.21.15.148:33352) 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 -- 172.21.15.148:0/868000137 learned_addr learned my addr 172.21.15.148:0/868000137 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f3250b84640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f324c158230 0x7f324c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324b7fe640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f324c0d1b80 0x7f324c14ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 -- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f324c0d1b80 msgr2=0x7f324c14ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f324c0d1b80 0x7f324c14ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 -- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f324c158230 msgr2=0x7f324c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f324c158230 0x7f324c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324bfff640 1 -- 172.21.15.148:0/868000137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c14d720 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f3250b84640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f324c158230 0x7f324c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.340+0000 7f324b7fe640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f324c0d1b80 0x7f324c14ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.341+0000 7f324bfff640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 0x7f324c0cd790 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f323c000c00 tx=0x7f323c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.341+0000 7f3248ff9640 1 -- 172.21.15.148:0/868000137 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c037050 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.341+0000 7f3248ff9640 1 -- 172.21.15.148:0/868000137 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c038040 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.341+0000 7f3248ff9640 1 -- 172.21.15.148:0/868000137 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c006e10 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.341+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c14da00 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.341+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c14dfc0 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.342+0000 7f3248ff9640 1 -- 172.21.15.148:0/868000137 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f323c030980 con 0x7f324c06c870 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.343+0000 7f3248ff9640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f323806b4d0 0x7f323806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.343+0000 7f3248ff9640 1 -- 172.21.15.148:0/868000137 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f323c0aaf20 con 0x7f324c06c870 2024-04-25T10:01:30.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.343+0000 7f324b7fe640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f323806b4d0 0x7f323806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.343+0000 7f324b7fe640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f323806b4d0 0x7f323806d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f324c0ccc50 tx=0x7f3240008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 Processor -- start 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 -- start start 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa43e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa43e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa40e770 0x55a3aa411f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa377da0 con 0x55a3aa43bef0 2024-04-25T10:01:30.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa320b80 con 0x55a3aa40e770 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.344+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa373e40 con 0x55a3aa43bb10 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa43e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa43e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33360/0 (socket says 172.21.15.148:33360) 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 -- 172.21.15.148:0/1214160847 learned_addr learned my addr 172.21.15.148:0/1214160847 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 -- 172.21.15.148:0/1214160847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 msgr2=0x55a3aa43e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa43e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 -- 172.21.15.148:0/1214160847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa40e770 msgr2=0x55a3aa411f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa40e770 0x55a3aa411f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 -- 172.21.15.148:0/1214160847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3aa32af10 con 0x55a3aa43bef0 2024-04-25T10:01:30.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f324b7fe640 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa43e860 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f324000f930 tx=0x7f3240012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f05dc2ad764ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f3231ffb640 1 -- 172.21.15.148:0/1214160847 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324001b050 con 0x55a3aa43bef0 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f3231ffb640 1 -- 172.21.15.148:0/1214160847 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3240025070 con 0x55a3aa43bef0 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.345+0000 7f3231ffb640 1 -- 172.21.15.148:0/1214160847 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240014de0 con 0x55a3aa43bef0 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 -- 172.21.15.148:0/1214160847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 msgr2=0x55a3aa43e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa43e860 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f324000f930 tx=0x7f3240012000 comp rx=0 tx=0).stop 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 -- 172.21.15.148:0/1214160847 shutdown_connections 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa40e770 0x55a3aa411f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa43e860 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 --2- 172.21.15.148:0/1214160847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa43e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 -- 172.21.15.148:0/1214160847 >> 172.21.15.148:0/1214160847 conn(0x55a3aa4109e0 msgr2=0x55a3aa459320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 -- 172.21.15.148:0/1214160847 shutdown_connections 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 -- 172.21.15.148:0/1214160847 wait complete. 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 -- start start 2024-04-25T10:01:30.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.346+0000 7f32553c7740 1 Processor -- start 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 0x55a3aa45bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bef0 0x55a3aa45c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa433050 0x55a3aa46c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa413b90 con 0x55a3aa43bb10 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa3164b0 con 0x55a3aa43bef0 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa413e70 con 0x55a3aa433050 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f32553c7740 10 client.0 _ll_register_callbacks cb 0x55a3aa356180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f324bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 0x55a3aa45bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f324bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 0x55a3aa45bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33364/0 (socket says 172.21.15.148:33364) 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f324bfff640 1 -- 172.21.15.148:0/3612864006 learned_addr learned my addr 172.21.15.148:0/3612864006 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.347+0000 7f324bfff640 1 -- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa433050 msgr2=0x55a3aa46c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f324bfff640 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa433050 0x55a3aa46c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f324bfff640 1 -- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bef0 msgr2=0x55a3aa45c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f324bfff640 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bef0 0x55a3aa45c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f324bfff640 1 -- 172.21.15.148:0/3612864006 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3aa1567d0 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f324bfff640 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 0x55a3aa45bf20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f323c02a1e0 tx=0x7f323c079e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deabc88d454bd64c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f322affd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f322affd640 1 -- 172.21.15.148:0/3612864006 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0a7060 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f322affd640 1 -- 172.21.15.148:0/3612864006 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f323c030c70 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3240025070 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f322affd640 1 -- 172.21.15.148:0/3612864006 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f323c086020 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 10 client.8436 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55a3aa08c3f0 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f322affd640 1 -- 172.21.15.148:0/3612864006 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3240025070 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f322affd640 1 -- 172.21.15.148:0/3612864006 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f323c081940 con 0x55a3aa43bb10 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 10 client.8436 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 10 client.8436 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 20 client.8436 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:30.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa46ce40 0x55a3aa45adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 4 client.8436 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3aa3852e0 con 0x55a3aa46ce40 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.348+0000 7f324b7fe640 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa46ce40 0x55a3aa45adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.349+0000 7f322affd640 10 client.8436 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.349+0000 7f324b7fe640 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa46ce40 0x55a3aa45adf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81689c8df05bf9af server_cookie=5de3657d80abf5ae in_seq=0 out_seq=0 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.349+0000 7f322affd640 1 -- 172.21.15.148:0/3612864006 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a3aa3852e0 con 0x55a3aa46ce40 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.349+0000 7f322affd640 10 client.8436 handle_command_reply: tid=0 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.349+0000 7f32553c7740 1 client.8436 shutdown 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 msgr2=0x55a3aa45bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 0x55a3aa45bf20 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f323c02a1e0 tx=0x7f323c079e80 comp rx=0 tx=0).stop 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 shutdown_connections 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa433050 0x55a3aa46c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa46ce40 0x55a3aa45adf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bef0 0x55a3aa45c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f322affd640 0 client.8436 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 --2- 172.21.15.148:0/3612864006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bb10 0x55a3aa45bf20 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 >> 172.21.15.148:0/3612864006 conn(0x55a3aa40e770 msgr2=0x55a3aa40f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 shutdown_connections 2024-04-25T10:01:30.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 1 -- 172.21.15.148:0/3612864006 wait complete. 2024-04-25T10:01:30.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.350+0000 7f32553c7740 20 client.8436 trim_cache size 0 max 16384 2024-04-25T10:01:30.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.364+0000 7f32553c7740 1 Processor -- start 2024-04-25T10:01:30.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.364+0000 7f32553c7740 1 -- start start 2024-04-25T10:01:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32553c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 0x55a3aa459420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bb10 0x55a3aa4135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bef0 0x55a3aa4394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32553c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c2003c0 con 0x55a3aa433050 2024-04-25T10:01:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c1ff5c0 con 0x55a3aa43bb10 2024-04-25T10:01:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0b2340 con 0x55a3aa43bef0 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 0x55a3aa459420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 0x55a3aa459420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33376/0 (socket says 172.21.15.148:33376) 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 -- 172.21.15.148:0/3697953854 learned_addr learned my addr 172.21.15.148:0/3697953854 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 -- 172.21.15.148:0/3697953854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bef0 msgr2=0x55a3aa4394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bef0 0x55a3aa4394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 -- 172.21.15.148:0/3697953854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bb10 msgr2=0x55a3aa4135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bb10 0x55a3aa4135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 -- 172.21.15.148:0/3697953854 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3aa04c0f0 con 0x55a3aa433050 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f324bfff640 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 0x55a3aa459420 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f323c032eb0 tx=0x7f323c08e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1048a61de201b107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.365+0000 7f32317fa640 1 -- 172.21.15.148:0/3697953854 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c091070 con 0x55a3aa433050 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32317fa640 1 -- 172.21.15.148:0/3697953854 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c091210 con 0x55a3aa433050 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32317fa640 1 -- 172.21.15.148:0/3697953854 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c079020 con 0x55a3aa433050 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 -- 172.21.15.148:0/3697953854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 msgr2=0x55a3aa459420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 0x55a3aa459420 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f323c032eb0 tx=0x7f323c08e000 comp rx=0 tx=0).stop 2024-04-25T10:01:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 -- 172.21.15.148:0/3697953854 shutdown_connections 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bef0 0x55a3aa4394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa43bb10 0x55a3aa4135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 --2- 172.21.15.148:0/3697953854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa433050 0x55a3aa459420 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 -- 172.21.15.148:0/3697953854 >> 172.21.15.148:0/3697953854 conn(0x55a3aa40e770 msgr2=0x55a3aa43e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 -- 172.21.15.148:0/3697953854 shutdown_connections 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.366+0000 7f32553c7740 1 -- 172.21.15.148:0/3697953854 wait complete. 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.367+0000 7f32553c7740 1 -- start start 2024-04-25T10:01:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.367+0000 7f32553c7740 1 Processor -- start 2024-04-25T10:01:30.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.367+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa433050 0x55a3aa4c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.367+0000 7f32553c7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa4394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa4399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0b2940 con 0x55a3aa43bef0 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa46c780 con 0x55a3aa433050 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a3aa46aed0 con 0x55a3aa43bb10 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 10 client.0 _ll_register_callbacks cb 0x7f32400091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa4399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa4399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33388/0 (socket says 172.21.15.148:33388) 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 -- 172.21.15.148:0/2152586766 learned_addr learned my addr 172.21.15.148:0/2152586766 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f324bfff640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa433050 0x55a3aa4c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f324b7fe640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa4394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 -- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 msgr2=0x55a3aa4394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa4394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 -- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa433050 msgr2=0x55a3aa4c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa433050 0x55a3aa4c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 -- 172.21.15.148:0/2152586766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3aa412230 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f324b7fe640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa4394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f3250b84640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa4399f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3244000f80 tx=0x7f324400d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145090d119a0e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f322affd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f322affd640 1 -- 172.21.15.148:0/2152586766 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440070d0 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f322affd640 1 -- 172.21.15.148:0/2152586766 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3244013c10 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f323c091210 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f322affd640 1 -- 172.21.15.148:0/2152586766 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3244013e10 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 10 client.8442 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55a3aa44f650 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.368+0000 7f322affd640 1 -- 172.21.15.148:0/2152586766 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c091210 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f322affd640 1 -- 172.21.15.148:0/2152586766 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3244013420 con 0x55a3aa43bef0 2024-04-25T10:01:30.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 10 client.8442 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 10 client.8442 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 20 client.8442 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa43e690 0x55a3aa40d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 4 client.8442 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55a3aa37f630 con 0x55a3aa43e690 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f324bfff640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa43e690 0x55a3aa40d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f322affd640 10 client.8442 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f324bfff640 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa43e690 0x55a3aa40d600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947f84632d9d517b server_cookie=3087917366a0a69 in_seq=0 out_seq=0 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f322affd640 1 -- 172.21.15.148:0/2152586766 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55a3aa37f630 con 0x55a3aa43e690 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f322affd640 10 client.8442 handle_command_reply: tid=0 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 1 client.8442 shutdown 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 msgr2=0x55a3aa4399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.369+0000 7f32553c7740 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa4399f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3244000f80 tx=0x7f324400d860 comp rx=0 tx=0).stop 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 shutdown_connections 2024-04-25T10:01:30.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a3aa43bef0 0x55a3aa4399f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a3aa43bb10 0x55a3aa4394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55a3aa43e690 0x55a3aa40d600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 --2- 172.21.15.148:0/2152586766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3aa433050 0x55a3aa4c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f322affd640 0 client.8442 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 >> 172.21.15.148:0/2152586766 conn(0x55a3aa40e770 msgr2=0x55a3aa409d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 shutdown_connections 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 1 -- 172.21.15.148:0/2152586766 wait complete. 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32553c7740 20 client.8442 trim_cache size 0 max 16384 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:30.380 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:01:30.381 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f323806b4d0 msgr2=0x7f323806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f323806b4d0 0x7f323806d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f324c0ccc50 tx=0x7f3240008040 comp rx=0 tx=0).stop 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 msgr2=0x7f324c0cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 0x7f324c0cd790 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f323c000c00 tx=0x7f323c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 shutdown_connections 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f323806b4d0 0x7f323806d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f324c158230 0x7f324c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f324c0d1b80 0x7f324c14ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 --2- 172.21.15.148:0/868000137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f324c06c870 0x7f324c0cd790 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 >> 172.21.15.148:0/868000137 conn(0x7f324c061600 msgr2=0x7f324c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 shutdown_connections 2024-04-25T10:01:30.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.370+0000 7f32525e7640 1 -- 172.21.15.148:0/868000137 wait complete. 2024-04-25T10:01:30.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:30.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:01:30.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.480+0000 7f1382083640 1 Processor -- start 2024-04-25T10:01:30.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.480+0000 7f1382083640 1 -- start start 2024-04-25T10:01:30.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f1382083640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c15dac0 0x7f137c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f1382083640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f137c0d1f30 0x7f137c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f1382083640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137c0d2420 con 0x7f137c0d34c0 2024-04-25T10:01:30.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f1382083640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f137c0d3030 con 0x7f137c0d1f30 2024-04-25T10:01:30.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f1382083640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0d2700 con 0x7f137c15dac0 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f137affd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f137c0d1f30 0x7f137c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c15dac0 0x7f137c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f137bfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f137c0d34c0 0x7f137c0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:58686/0 (socket says 172.21.15.148:58686) 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.481+0000 7f137affd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f137c0d1f30 0x7f137c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57620/0 (socket says 172.21.15.148:57620) 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.482+0000 7f137bfff640 1 -- 172.21.15.148:0/2024999451 learned_addr learned my addr 172.21.15.148:0/2024999451 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.482+0000 7f137affd640 1 -- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c15dac0 msgr2=0x7f137c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.482+0000 7f137affd640 1 --2- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c15dac0 0x7f137c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.482+0000 7f137affd640 1 -- 172.21.15.148:0/2024999451 >> v1:172.21.15.188:6789/0 conn(0x7f137c0d34c0 legacy=0x7f137c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.482+0000 7f137affd640 1 -- 172.21.15.148:0/2024999451 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0d29e0 con 0x7f137c0d1f30 2024-04-25T10:01:30.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.482+0000 7f137b7fe640 1 --2- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c15dac0 0x7f137c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f137affd640 1 --2- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c0d1f30 0x7f137c0d03a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1370009790 tx=0x7f137002a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b3161c7e1760e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f137a7fc640 1 -- 172.21.15.148:0/2024999451 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370037070 con 0x7f137c0d1f30 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f137a7fc640 1 -- 172.21.15.148:0/2024999451 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1370032440 con 0x7f137c0d1f30 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f137a7fc640 1 -- 172.21.15.148:0/2024999451 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370032810 con 0x7f137c0d1f30 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 -- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c0d1f30 msgr2=0x7f137c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 --2- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c0d1f30 0x7f137c0d03a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1370009790 tx=0x7f137002a200 comp rx=0 tx=0).stop 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 -- 172.21.15.148:0/2024999451 shutdown_connections 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 --2- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c0d1f30 0x7f137c0d03a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 --2- 172.21.15.148:0/2024999451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c15dac0 0x7f137c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 -- 172.21.15.148:0/2024999451 >> 172.21.15.148:0/2024999451 conn(0x7f137c061c30 msgr2=0x7f137c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 -- 172.21.15.148:0/2024999451 shutdown_connections 2024-04-25T10:01:30.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.483+0000 7f1382083640 1 -- 172.21.15.148:0/2024999451 wait complete. 2024-04-25T10:01:30.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.484+0000 7f1382083640 1 Processor -- start 2024-04-25T10:01:30.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 -- start start 2024-04-25T10:01:30.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c0d1f30 0x7f137c14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 0x7f137c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c15dac0 0x7f137c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b97d0 con 0x7f137c0d34c0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c15dea0 con 0x7f137c15dac0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f1382083640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b9950 con 0x7f137c0d1f30 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.485+0000 7f137affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 0x7f137c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 0x7f137c0caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33398/0 (socket says 172.21.15.148:33398) 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 -- 172.21.15.148:0/753346786 learned_addr learned my addr 172.21.15.148:0/753346786 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137b7fe640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c0d1f30 0x7f137c14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137bfff640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c15dac0 0x7f137c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 -- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c0d1f30 msgr2=0x7f137c14d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c0d1f30 0x7f137c14d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 -- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c15dac0 msgr2=0x7f137c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c15dac0 0x7f137c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 -- 172.21.15.148:0/753346786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c14d660 con 0x7f137c0d34c0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f137affd640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 0x7f137c0caaa0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1370035040 tx=0x7f13700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f1363fff640 1 -- 172.21.15.148:0/753346786 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370037040 con 0x7f137c0d34c0 2024-04-25T10:01:30.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f1363fff640 1 -- 172.21.15.148:0/753346786 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1370005bd0 con 0x7f137c0d34c0 2024-04-25T10:01:30.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c0c72b0 con 0x7f137c0d34c0 2024-04-25T10:01:30.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f1363fff640 1 -- 172.21.15.148:0/753346786 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370006030 con 0x7f137c0d34c0 2024-04-25T10:01:30.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.486+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c0c7870 con 0x7f137c0d34c0 2024-04-25T10:01:30.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.488+0000 7f1363fff640 1 -- 172.21.15.148:0/753346786 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13700061d0 con 0x7f137c0d34c0 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.488+0000 7f1363fff640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f136806b500 0x7f136806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.488+0000 7f1363fff640 1 -- 172.21.15.148:0/753346786 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13700aa9d0 con 0x7f137c0d34c0 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.488+0000 7f137b7fe640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f136806b500 0x7f136806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.489+0000 7f137b7fe640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f136806b500 0x7f136806d9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f136c009560 tx=0x7f136c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.489+0000 7f1384d82740 1 Processor -- start 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.489+0000 7f1384d82740 1 -- start start 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f1384d82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd871320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd871860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd841770 0x55aecd844f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f1384d82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecd7aada0 con 0x55aecd86eb10 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecd753b80 con 0x55aecd841770 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aecd7a6e40 con 0x55aecd86eef0 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd871320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd871320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33406/0 (socket says 172.21.15.148:33406) 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 -- 172.21.15.148:0/898853155 learned_addr learned my addr 172.21.15.148:0/898853155 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137bfff640 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd841770 0x55aecd844f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 -- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 msgr2=0x55aecd871860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd871860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 -- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd841770 msgr2=0x55aecd844f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd841770 0x55aecd844f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 -- 172.21.15.148:0/898853155 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aecd75df10 con 0x55aecd86eb10 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f137b7fe640 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd871320 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f136c011dc0 tx=0x7f136c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d48a8d8657c5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f13617fa640 1 -- 172.21.15.148:0/898853155 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c014bd0 con 0x55aecd86eb10 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.490+0000 7f13617fa640 1 -- 172.21.15.148:0/898853155 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c01b070 con 0x55aecd86eb10 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f13617fa640 1 -- 172.21.15.148:0/898853155 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c004030 con 0x55aecd86eb10 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 -- 172.21.15.148:0/898853155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 msgr2=0x55aecd871320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd871320 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f136c011dc0 tx=0x7f136c014000 comp rx=0 tx=0).stop 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 -- 172.21.15.148:0/898853155 shutdown_connections 2024-04-25T10:01:30.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd841770 0x55aecd844f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd871860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 --2- 172.21.15.148:0/898853155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd871320 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 -- 172.21.15.148:0/898853155 >> 172.21.15.148:0/898853155 conn(0x55aecd8439e0 msgr2=0x55aecd88c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 -- 172.21.15.148:0/898853155 shutdown_connections 2024-04-25T10:01:30.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.491+0000 7f1384d82740 1 -- 172.21.15.148:0/898853155 wait complete. 2024-04-25T10:01:30.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.492+0000 7f1384d82740 1 -- start start 2024-04-25T10:01:30.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.492+0000 7f1384d82740 1 Processor -- start 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd892a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 0x55aecd890f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecd846b90 con 0x55aecd86e160 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecd7494b0 con 0x55aecd86eb10 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aecd846e70 con 0x55aecd86eef0 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 10 client.0 _ll_register_callbacks cb 0x55aecd789180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 0x55aecd890f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 0x55aecd890f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33408/0 (socket says 172.21.15.148:33408) 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 -- 172.21.15.148:0/4082130131 learned_addr learned my addr 172.21.15.148:0/4082130131 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137b7fe640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd892a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137affd640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 -- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 msgr2=0x55aecd8909f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8909f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 -- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 msgr2=0x55aecd892a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd892a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 -- 172.21.15.148:0/4082130131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aecd5897d0 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137b7fe640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd892a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137affd640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f137bfff640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 0x55aecd890f30 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f13740075b0 tx=0x7f13740076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecab1be5f8c08762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f135a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f135a7fc640 1 -- 172.21.15.148:0/4082130131 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374018030 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f135a7fc640 1 -- 172.21.15.148:0/4082130131 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1374014610 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f136c01b070 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.493+0000 7f135a7fc640 1 -- 172.21.15.148:0/4082130131 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1374021050 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 10 client.8451 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55aecd4bf3f0 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f135a7fc640 1 -- 172.21.15.148:0/4082130131 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c01b070 con 0x55aecd86e160 2024-04-25T10:01:30.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f135a7fc640 1 -- 172.21.15.148:0/4082130131 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1374024020 con 0x55aecd86e160 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 10 client.8451 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 10 client.8451 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 20 client.8451 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd865210 0x55aecd865620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 4 client.8451 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aecd7b82e0 con 0x55aecd865210 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f137b7fe640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd865210 0x55aecd865620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f135a7fc640 10 client.8451 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.494+0000 7f137b7fe640 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd865210 0x55aecd865620 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831cd868a91659bd server_cookie=45fa7858e5fd6175 in_seq=0 out_seq=0 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f135a7fc640 1 -- 172.21.15.148:0/4082130131 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aecd7b82e0 con 0x55aecd865210 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f135a7fc640 10 client.8451 handle_command_reply: tid=0 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 client.8451 shutdown 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 msgr2=0x55aecd890f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 0x55aecd890f30 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f13740075b0 tx=0x7f13740076a0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 shutdown_connections 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86e160 0x55aecd890f30 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8909f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd865210 0x55aecd865620 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 --2- 172.21.15.148:0/4082130131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd892a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f135a7fc640 0 client.8451 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 >> 172.21.15.148:0/4082130131 conn(0x55aecd841770 msgr2=0x55aecd842420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 shutdown_connections 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 1 -- 172.21.15.148:0/4082130131 wait complete. 2024-04-25T10:01:30.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.495+0000 7f1384d82740 20 client.8451 trim_cache size 0 max 16384 2024-04-25T10:01:30.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.510+0000 7f1384d82740 1 Processor -- start 2024-04-25T10:01:30.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.510+0000 7f1384d82740 1 -- start start 2024-04-25T10:01:30.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86e160 0x55aecd8646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd8ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1384d82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 0x55aecd874ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1384d82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c2005c0 con 0x55aecd86eef0 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c1ff7d0 con 0x55aecd86eb10 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0c9420 con 0x55aecd86e160 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 0x55aecd874ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 0x55aecd874ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33420/0 (socket says 172.21.15.148:33420) 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 -- 172.21.15.148:0/2417144552 learned_addr learned my addr 172.21.15.148:0/2417144552 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137affd640 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd8ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137b7fe640 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86e160 0x55aecd8646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 -- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86e160 msgr2=0x55aecd8646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86e160 0x55aecd8646e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 -- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 msgr2=0x55aecd8ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd8ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 -- 172.21.15.148:0/2417144552 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aecd47f0f0 con 0x55aecd86eef0 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f137bfff640 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 0x55aecd874ed0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1374005c00 tx=0x7f137400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547b6af9136bb43b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1360ff9640 1 -- 172.21.15.148:0/2417144552 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374018040 con 0x55aecd86eef0 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1360ff9640 1 -- 172.21.15.148:0/2417144552 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374014610 con 0x55aecd86eef0 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.511+0000 7f1360ff9640 1 -- 172.21.15.148:0/2417144552 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13740070e0 con 0x55aecd86eef0 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 -- 172.21.15.148:0/2417144552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 msgr2=0x55aecd874ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 0x55aecd874ed0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1374005c00 tx=0x7f137400c000 comp rx=0 tx=0).stop 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 -- 172.21.15.148:0/2417144552 shutdown_connections 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eef0 0x55aecd874ed0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86eb10 0x55aecd8ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 --2- 172.21.15.148:0/2417144552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86e160 0x55aecd8646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 -- 172.21.15.148:0/2417144552 >> 172.21.15.148:0/2417144552 conn(0x55aecd841770 msgr2=0x55aecd8942d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 -- 172.21.15.148:0/2417144552 shutdown_connections 2024-04-25T10:01:30.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.512+0000 7f1384d82740 1 -- 172.21.15.148:0/2417144552 wait complete. 2024-04-25T10:01:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 -- start start 2024-04-25T10:01:30.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 Processor -- start 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86e160 0x55aecd840a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd8f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aecd8487a0 con 0x55aecd86eb10 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c1ff1d0 con 0x55aecd86e160 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aecd74b010 con 0x55aecd86eef0 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f1384d82740 10 client.0 _ll_register_callbacks cb 0x7f136c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f137affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd8f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f137affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd8f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33430/0 (socket says 172.21.15.148:33430) 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.513+0000 7f137affd640 1 -- 172.21.15.148:0/3734717336 learned_addr learned my addr 172.21.15.148:0/3734717336 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137b7fe640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86e160 0x55aecd840a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137bfff640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137affd640 1 -- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 msgr2=0x55aecd8f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137affd640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137affd640 1 -- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86e160 msgr2=0x55aecd840a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137affd640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86e160 0x55aecd840a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137affd640 1 -- 172.21.15.148:0/3734717336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aecd88e3e0 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f137affd640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd8f5950 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1370000c00 tx=0x7f13700789e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286579181a4cf5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f135a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f135a7fc640 1 -- 172.21.15.148:0/3734717336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700816e0 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f135a7fc640 1 -- 172.21.15.148:0/3734717336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1370081250 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1374014610 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f135a7fc640 1 -- 172.21.15.148:0/3734717336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1370081470 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 10 client.8457 fetch_fsmap learned FSMap version 60 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x55aecd840f40 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f135a7fc640 1 -- 172.21.15.148:0/3734717336 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374014610 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f135a7fc640 1 -- 172.21.15.148:0/3734717336 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f137003a070 con 0x55aecd86eb10 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 10 client.8457 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T10:01:30.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 10 client.8457 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 20 client.8457 populate_metadata read hostname 'smithi148' 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd893800 0x55aecd88d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 4 client.8457 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.514+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 --> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aecd7b2630 con 0x55aecd893800 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f137b7fe640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd893800 0x55aecd88d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f135a7fc640 10 client.8457 ms_handle_connect on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f137b7fe640 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd893800 0x55aecd88d590 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e126128590998c5 server_cookie=c7be978bc4a78e33 in_seq=0 out_seq=0 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f135a7fc640 1 -- 172.21.15.148:0/3734717336 <== mds.0 v2:172.21.15.188:6832/3455709247 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aecd7b2630 con 0x55aecd893800 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f135a7fc640 10 client.8457 handle_command_reply: tid=0 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 client.8457 shutdown 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 msgr2=0x55aecd8f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd8f5950 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1370000c00 tx=0x7f13700789e0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 shutdown_connections 2024-04-25T10:01:30.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aecd86eef0 0x55aecd8f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aecd86eb10 0x55aecd8f5950 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:6832/3455709247,v1:172.21.15.188:6834/3455709247] conn(0x55aecd893800 0x55aecd88d590 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 --2- 172.21.15.148:0/3734717336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aecd86e160 0x55aecd840a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f135a7fc640 0 client.8457 ms_handle_reset on v2:172.21.15.188:6832/3455709247 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 >> 172.21.15.148:0/3734717336 conn(0x55aecd841770 msgr2=0x55aecd874ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 shutdown_connections 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.515+0000 7f1384d82740 1 -- 172.21.15.148:0/3734717336 wait complete. 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1384d82740 20 client.8457 trim_cache size 0 max 16384 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:30.526 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:01:30.527 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:01:30.527 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:30.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f136806b500 msgr2=0x7f136806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f136806b500 0x7f136806d9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f136c009560 tx=0x7f136c009190 comp rx=0 tx=0).stop 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 msgr2=0x7f137c0caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 0x7f137c0caaa0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1370035040 tx=0x7f13700046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 shutdown_connections 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f136806b500 0x7f136806d9c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137c15dac0 0x7f137c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f137c0d34c0 0x7f137c0caaa0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 --2- 172.21.15.148:0/753346786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137c0d1f30 0x7f137c14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 >> 172.21.15.148:0/753346786 conn(0x7f137c061c30 msgr2=0x7f137c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 shutdown_connections 2024-04-25T10:01:30.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.516+0000 7f1382083640 1 -- 172.21.15.148:0/753346786 wait complete. 2024-04-25T10:01:30.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.624+0000 7fd48ede7640 1 Processor -- start 2024-04-25T10:01:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.624+0000 7fd48ede7640 1 -- start start 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.624+0000 7fd48ede7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 0x7fd4880c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.624+0000 7fd48ede7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4880ceb20 0x7fd4880d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48ede7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48815dfe0 con 0x7fd4880d1c00 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48ede7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880d2060 con 0x7fd4880ceb20 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48ede7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880d00c0 con 0x7fd48815dc00 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48cb83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 0x7fd4880c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd487fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4880ceb20 0x7fd4880d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48d384640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd4880d1c00 0x7fd4880ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:58702/0 (socket says 172.21.15.148:58702) 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48cb83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 0x7fd4880c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:54058/0 (socket says 172.21.15.148:54058) 2024-04-25T10:01:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48d384640 1 -- 172.21.15.148:0/3821108920 learned_addr learned my addr 172.21.15.148:0/3821108920 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd487fff640 1 --2- 172.21.15.148:0/3821108920 >> v2:172.21.15.188:3300/0 conn(0x7fd4880ceb20 0x7fd4880d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:57672/0 (socket says 172.21.15.148:57672) 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd4877fe640 1 -- 172.21.15.148:0/3821108920 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663019937 0 0) 0x7fd480001490 con 0x7fd4880d1c00 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd4877fe640 1 -- 172.21.15.148:0/3821108920 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd478003520 con 0x7fd4880d1c00 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd487fff640 1 -- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 msgr2=0x7fd4880c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd487fff640 1 --2- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 0x7fd4880c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd487fff640 1 -- 172.21.15.148:0/3821108920 >> v1:172.21.15.188:6789/0 conn(0x7fd4880d1c00 legacy=0x7fd4880ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd487fff640 1 -- 172.21.15.148:0/3821108920 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4880d03a0 con 0x7fd4880ceb20 2024-04-25T10:01:30.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.625+0000 7fd48cb83640 1 --2- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 0x7fd4880c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.626+0000 7fd487fff640 1 --2- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880ceb20 0x7fd4880d16c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097a0 tx=0x7fd47402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9902d728af219d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.626+0000 7fd4877fe640 1 -- 172.21.15.148:0/3821108920 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474037070 con 0x7fd4880ceb20 2024-04-25T10:01:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.626+0000 7fd4877fe640 1 -- 172.21.15.148:0/3821108920 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474032440 con 0x7fd4880ceb20 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.626+0000 7fd4877fe640 1 -- 172.21.15.148:0/3821108920 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474032810 con 0x7fd4880ceb20 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 -- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880ceb20 msgr2=0x7fd4880d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880ceb20 0x7fd4880d16c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097a0 tx=0x7fd47402a180 comp rx=0 tx=0).stop 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 -- 172.21.15.148:0/3821108920 shutdown_connections 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880ceb20 0x7fd4880d16c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3821108920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd48815dc00 0x7fd4880c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 -- 172.21.15.148:0/3821108920 >> 172.21.15.148:0/3821108920 conn(0x7fd48806f5d0 msgr2=0x7fd48806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 -- 172.21.15.148:0/3821108920 shutdown_connections 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.627+0000 7fd48ede7640 1 -- 172.21.15.148:0/3821108920 wait complete. 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.628+0000 7fd48ede7640 1 Processor -- start 2024-04-25T10:01:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.628+0000 7fd48ede7640 1 -- start start 2024-04-25T10:01:30.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48ede7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4880ceb20 0x7fd488147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48ede7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880d1c00 0x7fd488156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48ede7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 0x7fd488156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48ede7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880cc130 con 0x7fd48815dc00 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48ede7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880cc720 con 0x7fd4880d1c00 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48ede7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880cfd90 con 0x7fd4880ceb20 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 0x7fd488156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 0x7fd488156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33446/0 (socket says 172.21.15.148:33446) 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 -- 172.21.15.148:0/3375096077 learned_addr learned my addr 172.21.15.148:0/3375096077 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48cb83640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4880ceb20 0x7fd488147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd487fff640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880d1c00 0x7fd488156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 -- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4880ceb20 msgr2=0x7fd488147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4880ceb20 0x7fd488147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 -- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880d1c00 msgr2=0x7fd488156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880d1c00 0x7fd488156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 -- 172.21.15.148:0/3375096077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488152a20 con 0x7fd48815dc00 2024-04-25T10:01:30.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd487fff640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880d1c00 0x7fd488156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd48d384640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 0x7fd488156dd0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd48000cb50 tx=0x7fd48000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.629+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480017070 con 0x7fd48815dc00 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.630+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48000ad40 con 0x7fd48815dc00 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.630+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488152d00 con 0x7fd48815dc00 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.630+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480012620 con 0x7fd48815dc00 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.630+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4881532c0 con 0x7fd48815dc00 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.631+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488073990 con 0x7fd48815dc00 2024-04-25T10:01:30.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.631+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd48001a070 con 0x7fd48815dc00 2024-04-25T10:01:30.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.632+0000 7fd4857fa640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd46c06b4d0 0x7fd46c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.632+0000 7fd48cb83640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd46c06b4d0 0x7fd46c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.632+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd48008a0d0 con 0x7fd48815dc00 2024-04-25T10:01:30.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.632+0000 7fd48cb83640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd46c06b4d0 0x7fd46c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009560 tx=0x7fd47c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.636+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd48815dfe0 con 0x7fd48815dc00 2024-04-25T10:01:30.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.842+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd48806dc00 con 0x7fd48815dc00 2024-04-25T10:01:30.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.842+0000 7fd4857fa640 1 -- 172.21.15.148:0/3375096077 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fd480058800 con 0x7fd48815dc00 2024-04-25T10:01:30.847 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:30.847 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:30.847 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd46c06b4d0 msgr2=0x7fd46c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd46c06b4d0 0x7fd46c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009560 tx=0x7fd47c009190 comp rx=0 tx=0).stop 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 msgr2=0x7fd488156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 0x7fd488156dd0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fd48000cb50 tx=0x7fd48000a430 comp rx=0 tx=0).stop 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 shutdown_connections 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd46c06b4d0 0x7fd46c06d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48815dc00 0x7fd488156dd0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4880d1c00 0x7fd488156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 --2- 172.21.15.148:0/3375096077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4880ceb20 0x7fd488147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 >> 172.21.15.148:0/3375096077 conn(0x7fd48806f5d0 msgr2=0x7fd4880c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 shutdown_connections 2024-04-25T10:01:30.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.844+0000 7fd48ede7640 1 -- 172.21.15.148:0/3375096077 wait complete. 2024-04-25T10:01:30.859 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:30.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.951+0000 7f63851b0640 1 Processor -- start 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.951+0000 7f63851b0640 1 -- start start 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f63851b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f638015dc00 0x7f63800c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f63851b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638015dfe0 con 0x7f63800d1dd0 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f63851b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63800d00c0 con 0x7f63800ceb20 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f63851b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63800d03a0 con 0x7f638015dc00 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637f59e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f63800d1dd0 0x7f63800ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49078/0 (socket says 172.21.15.148:49078) 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637f59e640 1 -- 172.21.15.148:0/4154899153 learned_addr learned my addr 172.21.15.148:0/4154899153 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472909733 0 0) 0x7f6378001490 con 0x7f63800d1dd0 2024-04-25T10:01:30.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f636c003520 con 0x7f63800d1dd0 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637ed9d640 1 --2- 172.21.15.148:0/4154899153 >> v2:172.21.15.188:3300/0 conn(0x7f638015dc00 0x7f63800c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1013218262 0 0) 0x7f636c003520 con 0x7f63800d1dd0 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 >> v2:172.21.15.188:3300/0 conn(0x7f638015dc00 msgr2=0x7f63800c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 --2- 172.21.15.148:0/4154899153 >> v2:172.21.15.188:3300/0 conn(0x7f638015dc00 0x7f63800c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 >> v1:172.21.15.188:6789/0 conn(0x7f63800ceb20 legacy=0x7f63800d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800cc440 con 0x7f63800d1dd0 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6378001bc0 con 0x7f63800d1dd0 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 >> v1:172.21.15.148:6789/0 conn(0x7f63800d1dd0 legacy=0x7f63800ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0039b0 0x7f636c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c004800 0x7f636c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f636c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800d03a0 con 0x7f636c0051c0 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c00c0f0 con 0x7f636c004800 2024-04-25T10:01:30.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.952+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c00c2f0 con 0x7f636c0039b0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f636c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637ed9d640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0039b0 0x7f636c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f6377fff640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c004800 0x7f636c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 -- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0039b0 msgr2=0x7f636c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c0039b0 0x7f636c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 -- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c004800 msgr2=0x7f636c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c004800 0x7f636c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 -- 172.21.15.148:0/4154899153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c003520 con 0x7f636c0051c0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 -- 172.21.15.148:0/4154899153 reap_dead start 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.953+0000 7f637f59e640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f636c00b9c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f63800b1060 tx=0x7f636c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29535bf79a652d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637800a690 con 0x7f636c0051c0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637802f070 con 0x7f636c0051c0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f637e59c640 1 -- 172.21.15.148:0/4154899153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637802be10 con 0x7f636c0051c0 2024-04-25T10:01:30.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 -- 172.21.15.148:0/4154899153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 msgr2=0x7f636c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f636c00b9c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f63800b1060 tx=0x7f636c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 -- 172.21.15.148:0/4154899153 shutdown_connections 2024-04-25T10:01:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 --2- 172.21.15.148:0/4154899153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f636c00b9c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 -- 172.21.15.148:0/4154899153 >> 172.21.15.148:0/4154899153 conn(0x7f638006f5d0 msgr2=0x7f638006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 -- 172.21.15.148:0/4154899153 shutdown_connections 2024-04-25T10:01:30.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.954+0000 7f63851b0640 1 -- 172.21.15.148:0/4154899153 wait complete. 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 Processor -- start 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 -- start start 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f6380155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63801f3fa0 0x7f6380153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800ca950 0x7f63800cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800cfd90 con 0x7f636c0051c0 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800cc130 con 0x7f63801f3fa0 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f63851b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63800cdd70 con 0x7f63800ca950 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f637ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f6380155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f637ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f6380155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33462/0 (socket says 172.21.15.148:33462) 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.956+0000 7f637ed9d640 1 -- 172.21.15.148:0/2411993067 learned_addr learned my addr 172.21.15.148:0/2411993067 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637f59e640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800ca950 0x7f63800cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f6377fff640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63801f3fa0 0x7f6380153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637ed9d640 1 -- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800ca950 msgr2=0x7f63800cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637ed9d640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800ca950 0x7f63800cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637ed9d640 1 -- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63801f3fa0 msgr2=0x7f6380153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637ed9d640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63801f3fa0 0x7f6380153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637ed9d640 1 -- 172.21.15.148:0/2411993067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6380156e40 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f637ed9d640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f6380155c50 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f637000b7c0 tx=0x7f637000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370018070 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6370010830 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63801562c0 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370010c00 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.957+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6380156880 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.958+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6380073990 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.958+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f637001b070 con 0x7f636c0051c0 2024-04-25T10:01:30.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.959+0000 7f63777fe640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f636006b500 0x7f636006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:30.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.959+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f637008a7c0 con 0x7f636c0051c0 2024-04-25T10:01:30.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.959+0000 7f6377fff640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f636006b500 0x7f636006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:30.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.959+0000 7f6377fff640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f636006b500 0x7f636006d9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6380159440 tx=0x7f6368009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:30.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:30.963+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6370059540 con 0x7f636c0051c0 2024-04-25T10:01:31.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.199+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f63800c66c0 con 0x7f636c0051c0 2024-04-25T10:01:31.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.199+0000 7f63777fe640 1 -- 172.21.15.148:0/2411993067 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f6370058e40 con 0x7f636c0051c0 2024-04-25T10:01:31.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:31.204 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":452,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076490554,"bytes_used":2542919680,"bytes_avail":770551193600,"bytes_total":773094113280,"write_bytes_sec":83445712,"read_op_per_sec":0,"write_op_per_sec":81},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-25T10:01:29.108234+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f636006b500 msgr2=0x7f636006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f636006b500 0x7f636006d9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6380159440 tx=0x7f6368009040 comp rx=0 tx=0).stop 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 msgr2=0x7f6380155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f6380155c50 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f637000b7c0 tx=0x7f637000bc80 comp rx=0 tx=0).stop 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 shutdown_connections 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f636006b500 0x7f636006d9c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800ca950 0x7f63800cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63801f3fa0 0x7f6380153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 --2- 172.21.15.148:0/2411993067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636c0051c0 0x7f6380155c50 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:31.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 >> 172.21.15.148:0/2411993067 conn(0x7f638006f5d0 msgr2=0x7f63800b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:31.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 shutdown_connections 2024-04-25T10:01:31.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:31.201+0000 7f63851b0640 1 -- 172.21.15.148:0/2411993067 wait complete. 2024-04-25T10:01:31.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:01:32.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:32.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.395+0000 7fcbadcee640 1 Processor -- start 2024-04-25T10:01:32.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.395+0000 7fcbadcee640 1 -- start start 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcbadcee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcba806fea0 0x7fcba80b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcbadcee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80b3710 con 0x7fcba815a790 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcbadcee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80cf0e0 con 0x7fcba80d0aa0 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcbadcee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba80cf3c0 con 0x7fcba806fea0 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcba815a790 0x7fcba80d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49090/0 (socket says 172.21.15.148:49090) 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba77fe640 1 -- 172.21.15.148:0/3670841013 learned_addr learned my addr 172.21.15.148:0/3670841013 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba7fff640 1 --2- 172.21.15.148:0/3670841013 >> v2:172.21.15.188:3300/0 conn(0x7fcba806fea0 0x7fcba80b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019701415 0 0) 0x7fcba80b3710 con 0x7fcba815a790 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb94005730 con 0x7fcba815a790 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189759348 0 0) 0x7fcb94005730 con 0x7fcba815a790 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 >> v2:172.21.15.188:3300/0 conn(0x7fcba806fea0 msgr2=0x7fcba80b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 --2- 172.21.15.148:0/3670841013 >> v2:172.21.15.188:3300/0 conn(0x7fcba806fea0 0x7fcba80b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 >> v1:172.21.15.188:6789/0 conn(0x7fcba80d0aa0 legacy=0x7fcba80b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80cf6a0 con 0x7fcba815a790 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fcb98001670 con 0x7fcba815a790 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 >> v1:172.21.15.148:6789/0 conn(0x7fcba815a790 legacy=0x7fcba80d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb94005bc0 0x7fcb94005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcb94006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb940073d0 0x7fcb9400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80cf3c0 con 0x7fcb94006a10 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9400e270 con 0x7fcb94005bc0 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.396+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9400e470 con 0x7fcb940073d0 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcb94006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcba77fe640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb94005bc0 0x7fcb94005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcba7fff640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb940073d0 0x7fcb9400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 -- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb940073d0 msgr2=0x7fcb9400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb940073d0 0x7fcb9400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 -- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb94005bc0 msgr2=0x7fcb94005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb94005bc0 0x7fcb94005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 -- 172.21.15.148:0/3670841013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94005730 con 0x7fcb94006a10 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcba7fff640 1 -- 172.21.15.148:0/3670841013 reap_dead start 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.397+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcb94006e90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fcb90001750 tx=0x7fcb94007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe947c5392c562b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90001370 con 0x7fcb94006a10 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb900052d0 con 0x7fcb94006a10 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcba6ffd640 1 -- 172.21.15.148:0/3670841013 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9002abc0 con 0x7fcb94006a10 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 -- 172.21.15.148:0/3670841013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 msgr2=0x7fcb94006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcb94006e90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fcb90001750 tx=0x7fcb94007a40 comp rx=0 tx=0).stop 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 -- 172.21.15.148:0/3670841013 shutdown_connections 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 --2- 172.21.15.148:0/3670841013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcb94006e90 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 -- 172.21.15.148:0/3670841013 >> 172.21.15.148:0/3670841013 conn(0x7fcba80b2370 msgr2=0x7fcba80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 -- 172.21.15.148:0/3670841013 shutdown_connections 2024-04-25T10:01:32.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.398+0000 7fcbadcee640 1 -- 172.21.15.148:0/3670841013 wait complete. 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.399+0000 7fcbadcee640 1 Processor -- start 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.399+0000 7fcbadcee640 1 -- start start 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcbadcee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcba81480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcbadcee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcba815d950 0x7fcba81e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcbadcee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcba8156830 0x7fcba8156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcbadcee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba815ab70 con 0x7fcb94006a10 2024-04-25T10:01:32.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcbadcee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80cf980 con 0x7fcba8156830 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcbadcee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba815acf0 con 0x7fcba815d950 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcba77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcba81480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcba77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcba81480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33484/0 (socket says 172.21.15.148:33484) 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcba77fe640 1 -- 172.21.15.148:0/92231453 learned_addr learned my addr 172.21.15.148:0/92231453 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcba7fff640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcba8156830 0x7fcba8156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcba815d950 0x7fcba81e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcba77fe640 1 -- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcba815d950 msgr2=0x7fcba81e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.400+0000 7fcba77fe640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcba815d950 0x7fcba81e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba77fe640 1 -- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcba8156830 msgr2=0x7fcba8156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba77fe640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcba8156830 0x7fcba8156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba77fe640 1 -- 172.21.15.148:0/92231453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba81e35e0 con 0x7fcb94006a10 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba7fff640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcba8156830 0x7fcba8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:32.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba77fe640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcba81480b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fcb98007320 tx=0x7fcb980077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9801a070 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9800f8e0 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba81e38c0 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800fcd0 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.401+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba81e3e80 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.402+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba8073990 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.402+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb9801d070 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.403+0000 7fcba4ff9640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb8806b4d0 0x7fcb8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.403+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb8806b4d0 0x7fcb8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.403+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb98089c80 con 0x7fcb94006a10 2024-04-25T10:01:32.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.403+0000 7fcb9ffff640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb8806b4d0 0x7fcb8806d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb90032000 tx=0x7fcb9002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.407+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb98058a00 con 0x7fcb94006a10 2024-04-25T10:01:32.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.611+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcba806ad80 con 0x7fcb94006a10 2024-04-25T10:01:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.612+0000 7fcba4ff9640 1 -- 172.21.15.148:0/92231453 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fcb98058300 con 0x7fcb94006a10 2024-04-25T10:01:32.616 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:32.616 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:32.616 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb8806b4d0 msgr2=0x7fcb8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb8806b4d0 0x7fcb8806d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb90032000 tx=0x7fcb9002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 msgr2=0x7fcba81480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcba81480b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fcb98007320 tx=0x7fcb980077e0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 shutdown_connections 2024-04-25T10:01:32.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcb8806b4d0 0x7fcb8806d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcba8156830 0x7fcba8156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcba815d950 0x7fcba81e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 --2- 172.21.15.148:0/92231453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb94006a10 0x7fcba81480b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.613+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 >> 172.21.15.148:0/92231453 conn(0x7fcba80b2370 msgr2=0x7fcba8159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.614+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 shutdown_connections 2024-04-25T10:01:32.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.614+0000 7fcbadcee640 1 -- 172.21.15.148:0/92231453 wait complete. 2024-04-25T10:01:32.629 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:32.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:32.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.721+0000 7f757fa8f640 1 Processor -- start 2024-04-25T10:01:32.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.722+0000 7f757fa8f640 1 -- start start 2024-04-25T10:01:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.722+0000 7f757fa8f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f757815dc00 0x7f75780c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.722+0000 7f757fa8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780ceb20 0x7f75780d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.722+0000 7f757fa8f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757815dfe0 con 0x7f75780d1c00 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.722+0000 7f757fa8f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75780d2060 con 0x7f757815dc00 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.722+0000 7f757fa8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75780d00c0 con 0x7f75780ceb20 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f757815dc00 0x7f75780c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f757815dc00 0x7f75780c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33490/0 (socket says 172.21.15.148:33490) 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 -- 172.21.15.148:0/635643270 learned_addr learned my addr 172.21.15.148:0/635643270 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d02a640 1 --2- 172.21.15.148:0/635643270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780ceb20 0x7f75780d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 -- 172.21.15.148:0/635643270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780ceb20 msgr2=0x7f75780d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 --2- 172.21.15.148:0/635643270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780ceb20 0x7f75780d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 -- 172.21.15.148:0/635643270 >> v1:172.21.15.148:6789/0 conn(0x7f75780d1c00 legacy=0x7f75780ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757c829640 1 -- 172.21.15.148:0/635643270 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391089203 0 0) 0x7f7574001490 con 0x7f75780d1c00 2024-04-25T10:01:32.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.723+0000 7f757d82b640 1 -- 172.21.15.148:0/635643270 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780d03a0 con 0x7f757815dc00 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757d82b640 1 --2- 172.21.15.148:0/635643270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757815dc00 0x7f75780c6df0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7570009870 tx=0x7f757002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59aa0dbc7030d08f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757c829640 1 -- 172.21.15.148:0/635643270 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570037070 con 0x7f757815dc00 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757c829640 1 -- 172.21.15.148:0/635643270 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570032440 con 0x7f757815dc00 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757c829640 1 -- 172.21.15.148:0/635643270 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570032810 con 0x7f757815dc00 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757fa8f640 1 -- 172.21.15.148:0/635643270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757815dc00 msgr2=0x7f75780c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757fa8f640 1 --2- 172.21.15.148:0/635643270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757815dc00 0x7f75780c6df0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7570009870 tx=0x7f757002a270 comp rx=0 tx=0).stop 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757fa8f640 1 -- 172.21.15.148:0/635643270 shutdown_connections 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757fa8f640 1 --2- 172.21.15.148:0/635643270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780ceb20 0x7f75780d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757fa8f640 1 --2- 172.21.15.148:0/635643270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f757815dc00 0x7f75780c6df0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.724+0000 7f757fa8f640 1 -- 172.21.15.148:0/635643270 >> 172.21.15.148:0/635643270 conn(0x7f757806f5d0 msgr2=0x7f757806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.725+0000 7f757fa8f640 1 -- 172.21.15.148:0/635643270 shutdown_connections 2024-04-25T10:01:32.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.725+0000 7f757fa8f640 1 -- 172.21.15.148:0/635643270 wait complete. 2024-04-25T10:01:32.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 Processor -- start 2024-04-25T10:01:32.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 -- start start 2024-04-25T10:01:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 0x7f757815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780d1c00 0x7f75780ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757815dc00 0x7f7578158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780cc130 con 0x7f75780ceb20 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75780cc720 con 0x7f757815dc00 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757fa8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75780cfd90 con 0x7f75780d1c00 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757d82b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 0x7f757815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757d82b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 0x7f757815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33494/0 (socket says 172.21.15.148:33494) 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.726+0000 7f757d82b640 1 -- 172.21.15.148:0/2161625503 learned_addr learned my addr 172.21.15.148:0/2161625503 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d02a640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780d1c00 0x7f75780ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757e02c640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757815dc00 0x7f7578158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d82b640 1 -- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780d1c00 msgr2=0x7f75780ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d82b640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780d1c00 0x7f75780ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d82b640 1 -- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757815dc00 msgr2=0x7f7578158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d82b640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757815dc00 0x7f7578158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d82b640 1 -- 172.21.15.148:0/2161625503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780b4960 con 0x7f75780ceb20 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757d82b640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 0x7f757815c980 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7570000c00 tx=0x7f75700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570041020 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700306c0 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75781e9a90 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570037040 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.727+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75781ea050 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.728+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578073990 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.728+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7570044020 con 0x7f75780ceb20 2024-04-25T10:01:32.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.729+0000 7f756e7fc640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755c06b4d0 0x7f755c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.729+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75700aab40 con 0x7f75780ceb20 2024-04-25T10:01:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.729+0000 7f757d02a640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755c06b4d0 0x7f755c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:32.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.729+0000 7f757d02a640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755c06b4d0 0x7f755c06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f75780b1040 tx=0x7f7568008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:32.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.733+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f757004b030 con 0x7f75780ceb20 2024-04-25T10:01:32.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.969+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f75780ce340 con 0x7f75780ceb20 2024-04-25T10:01:32.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.970+0000 7f756e7fc640 1 -- 172.21.15.148:0/2161625503 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f7570079830 con 0x7f75780ceb20 2024-04-25T10:01:32.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:32.974 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":454,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2543058944,"bytes_avail":770551054336,"bytes_total":773094113280,"write_bytes_sec":83453079,"read_op_per_sec":0,"write_op_per_sec":81},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-25T10:01:29.108234+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:01:32.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755c06b4d0 msgr2=0x7f755c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755c06b4d0 0x7f755c06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f75780b1040 tx=0x7f7568008040 comp rx=0 tx=0).stop 2024-04-25T10:01:32.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 msgr2=0x7f757815c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 0x7f757815c980 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7570000c00 tx=0x7f75700046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 shutdown_connections 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755c06b4d0 0x7f755c06d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f757815dc00 0x7f7578158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75780d1c00 0x7f75780ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 --2- 172.21.15.148:0/2161625503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75780ceb20 0x7f757815c980 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 >> 172.21.15.148:0/2161625503 conn(0x7f757806f5d0 msgr2=0x7f757806c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 shutdown_connections 2024-04-25T10:01:32.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:32.972+0000 7f757fa8f640 1 -- 172.21.15.148:0/2161625503 wait complete. 2024-04-25T10:01:32.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:01:33.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:34.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.092+0000 7fc444334640 1 Processor -- start 2024-04-25T10:01:34.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.092+0000 7fc444334640 1 -- start start 2024-04-25T10:01:34.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc444334640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc43c15dc90 0x7fc43c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc444334640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc43c0d1810 0x7fc43c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc444334640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c0cce50 con 0x7fc43c0cc8e0 2024-04-25T10:01:34.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc444334640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c0d21c0 con 0x7fc43c15dc90 2024-04-25T10:01:34.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc444334640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c161af0 con 0x7fc43c0d1810 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc4420d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc43c15dc90 0x7fc43c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc4420d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc43c15dc90 0x7fc43c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34040/0 (socket says 172.21.15.148:34040) 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.093+0000 7fc4420d0640 1 -- 172.21.15.148:0/1413041078 learned_addr learned my addr 172.21.15.148:0/1413041078 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.094+0000 7fc4420d0640 1 -- 172.21.15.148:0/1413041078 >> v2:172.21.15.188:3300/0 conn(0x7fc43c0d1810 msgr2=0x7fc43c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.094+0000 7fc4420d0640 1 --2- 172.21.15.148:0/1413041078 >> v2:172.21.15.188:3300/0 conn(0x7fc43c0d1810 0x7fc43c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.094+0000 7fc4420d0640 1 -- 172.21.15.148:0/1413041078 >> v1:172.21.15.188:6789/0 conn(0x7fc43c0cc8e0 legacy=0x7fc43c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.094+0000 7fc4420d0640 1 -- 172.21.15.148:0/1413041078 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c161dd0 con 0x7fc43c15dc90 2024-04-25T10:01:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.094+0000 7fc4420d0640 1 --2- 172.21.15.148:0/1413041078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c15dc90 0x7fc43c1625d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0cdbb0 tx=0x7fc43402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eacbd38a2c4f9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc4410ce640 1 -- 172.21.15.148:0/1413041078 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434036070 con 0x7fc43c15dc90 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc4410ce640 1 -- 172.21.15.148:0/1413041078 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434031600 con 0x7fc43c15dc90 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc4410ce640 1 -- 172.21.15.148:0/1413041078 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4340319d0 con 0x7fc43c15dc90 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 -- 172.21.15.148:0/1413041078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c15dc90 msgr2=0x7fc43c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 --2- 172.21.15.148:0/1413041078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c15dc90 0x7fc43c1625d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0cdbb0 tx=0x7fc43402a660 comp rx=0 tx=0).stop 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 -- 172.21.15.148:0/1413041078 shutdown_connections 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 --2- 172.21.15.148:0/1413041078 >> v2:172.21.15.188:3300/0 conn(0x7fc43c0d1810 0x7fc43c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 --2- 172.21.15.148:0/1413041078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c15dc90 0x7fc43c1625d0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 -- 172.21.15.148:0/1413041078 >> 172.21.15.148:0/1413041078 conn(0x7fc43c06f590 msgr2=0x7fc43c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 -- 172.21.15.148:0/1413041078 shutdown_connections 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.095+0000 7fc444334640 1 -- 172.21.15.148:0/1413041078 wait complete. 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.096+0000 7fc444334640 1 Processor -- start 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 -- start start 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43c0cc8e0 0x7fc43c1f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 0x7fc43c0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc43c15dc90 0x7fc43c1f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15e070 con 0x7fc43c0d1810 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c0cd130 con 0x7fc43c0cc8e0 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc444334640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c15e1f0 con 0x7fc43c15dc90 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc4418cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 0x7fc43c0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc4418cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 0x7fc43c0b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34052/0 (socket says 172.21.15.148:34052) 2024-04-25T10:01:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.097+0000 7fc4418cf640 1 -- 172.21.15.148:0/4032269530 learned_addr learned my addr 172.21.15.148:0/4032269530 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4420d0640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43c0cc8e0 0x7fc43c1f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4418cf640 1 -- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc43c15dc90 msgr2=0x7fc43c1f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4428d1640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc43c15dc90 0x7fc43c1f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4418cf640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc43c15dc90 0x7fc43c1f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4418cf640 1 -- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43c0cc8e0 msgr2=0x7fc43c1f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4418cf640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43c0cc8e0 0x7fc43c1f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4418cf640 1 -- 172.21.15.148:0/4032269530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c0b6140 con 0x7fc43c0d1810 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc4418cf640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 0x7fc43c0b5c00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc43000c410 tx=0x7fc43000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430017070 con 0x7fc43c0d1810 2024-04-25T10:01:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300035f0 con 0x7fc43c0d1810 2024-04-25T10:01:34.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c1f4b00 con 0x7fc43c0d1810 2024-04-25T10:01:34.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300039e0 con 0x7fc43c0d1810 2024-04-25T10:01:34.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.098+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c15f620 con 0x7fc43c0d1810 2024-04-25T10:01:34.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.099+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c073990 con 0x7fc43c0d1810 2024-04-25T10:01:34.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.100+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc43001a070 con 0x7fc43c0d1810 2024-04-25T10:01:34.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.100+0000 7fc43affd640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.100+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc430089800 con 0x7fc43c0d1810 2024-04-25T10:01:34.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.100+0000 7fc4420d0640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.101+0000 7fc4420d0640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc42806b4d0 0x7fc42806d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc434033000 tx=0x7fc434009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.104+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc43c0b6140 con 0x7fc43c0d1810 2024-04-25T10:01:34.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.311+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc43c0cf6b0 con 0x7fc43c0d1810 2024-04-25T10:01:34.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.312+0000 7fc43affd640 1 -- 172.21.15.148:0/4032269530 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fc4300584f0 con 0x7fc43c0d1810 2024-04-25T10:01:34.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:34.317 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:34.317 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:34.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc42806b4d0 msgr2=0x7fc42806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc42806b4d0 0x7fc42806d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc434033000 tx=0x7fc434009000 comp rx=0 tx=0).stop 2024-04-25T10:01:34.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 msgr2=0x7fc43c0b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 0x7fc43c0b5c00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc43000c410 tx=0x7fc43000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 shutdown_connections 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc42806b4d0 0x7fc42806d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc43c15dc90 0x7fc43c1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc43c0d1810 0x7fc43c0b5c00 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 --2- 172.21.15.148:0/4032269530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc43c0cc8e0 0x7fc43c1f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 >> 172.21.15.148:0/4032269530 conn(0x7fc43c06f590 msgr2=0x7fc43c0b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 shutdown_connections 2024-04-25T10:01:34.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.314+0000 7fc444334640 1 -- 172.21.15.148:0/4032269530 wait complete. 2024-04-25T10:01:34.330 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:34.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.421+0000 7fa212dc1640 1 Processor -- start 2024-04-25T10:01:34.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.421+0000 7fa212dc1640 1 -- start start 2024-04-25T10:01:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa212dc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c0cc8e0 0x7fa20c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa212dc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa20c0d1810 0x7fa20c0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa212dc1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c0cce50 con 0x7fa20c15dc90 2024-04-25T10:01:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa212dc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c161af0 con 0x7fa20c0d1810 2024-04-25T10:01:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa212dc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c161dd0 con 0x7fa20c0cc8e0 2024-04-25T10:01:34.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa210b5d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa20c15dc90 0x7fa20c15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46598/0 (socket says 172.21.15.148:46598) 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa210b5d640 1 -- 172.21.15.148:0/1758373792 learned_addr learned my addr 172.21.15.148:0/1758373792 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa21135e640 1 --2- 172.21.15.148:0/1758373792 >> v2:172.21.15.188:3300/0 conn(0x7fa20c0d1810 0x7fa20c0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa20bfff640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c0cc8e0 0x7fa20c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa20b7fe640 1 -- 172.21.15.148:0/1758373792 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540980003 0 0) 0x7fa20c0cce50 con 0x7fa20c15dc90 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa20b7fe640 1 -- 172.21.15.148:0/1758373792 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1fc003520 con 0x7fa20c15dc90 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa21135e640 1 -- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c0cc8e0 msgr2=0x7fa20c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa21135e640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c0cc8e0 0x7fa20c1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa21135e640 1 -- 172.21.15.148:0/1758373792 >> v1:172.21.15.148:6789/0 conn(0x7fa20c15dc90 legacy=0x7fa20c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa21135e640 1 -- 172.21.15.148:0/1758373792 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c1620b0 con 0x7fa20c0d1810 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.422+0000 7fa20bfff640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c0cc8e0 0x7fa20c1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.423+0000 7fa21135e640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0d1810 0x7fa20c0d1ca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa204009d90 tx=0x7fa20402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d18e47c861e1093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.423+0000 7fa20b7fe640 1 -- 172.21.15.148:0/1758373792 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204031600 con 0x7fa20c0d1810 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.423+0000 7fa20b7fe640 1 -- 172.21.15.148:0/1758373792 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2040317a0 con 0x7fa20c0d1810 2024-04-25T10:01:34.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.423+0000 7fa20b7fe640 1 -- 172.21.15.148:0/1758373792 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204031b70 con 0x7fa20c0d1810 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 -- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0d1810 msgr2=0x7fa20c0d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0d1810 0x7fa20c0d1ca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa204009d90 tx=0x7fa20402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 -- 172.21.15.148:0/1758373792 shutdown_connections 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0d1810 0x7fa20c0d1ca0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 --2- 172.21.15.148:0/1758373792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c0cc8e0 0x7fa20c1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 -- 172.21.15.148:0/1758373792 >> 172.21.15.148:0/1758373792 conn(0x7fa20c06f590 msgr2=0x7fa20c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 -- 172.21.15.148:0/1758373792 shutdown_connections 2024-04-25T10:01:34.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.424+0000 7fa212dc1640 1 -- 172.21.15.148:0/1758373792 wait complete. 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.425+0000 7fa212dc1640 1 Processor -- start 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.425+0000 7fa212dc1640 1 -- start start 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa212dc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0cc8e0 0x7fa20c158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa212dc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 0x7fa20c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa212dc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c15dc90 0x7fa20c1e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa212dc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15e1f0 con 0x7fa20c0d1810 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa212dc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15e070 con 0x7fa20c0cc8e0 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa212dc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c0cfde0 con 0x7fa20c15dc90 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 0x7fa20c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 0x7fa20c0cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34054/0 (socket says 172.21.15.148:34054) 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 -- 172.21.15.148:0/2860026885 learned_addr learned my addr 172.21.15.148:0/2860026885 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa210b5d640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0cc8e0 0x7fa20c158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa21135e640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c15dc90 0x7fa20c1e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 -- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c15dc90 msgr2=0x7fa20c1e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c15dc90 0x7fa20c1e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 -- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0cc8e0 msgr2=0x7fa20c158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0cc8e0 0x7fa20c158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 -- 172.21.15.148:0/2860026885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0cb640 con 0x7fa20c0d1810 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa21135e640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c15dc90 0x7fa20c1e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.426+0000 7fa20bfff640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 0x7fa20c0cb100 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800ebe0 tx=0x7fa1f800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.427+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8017070 con 0x7fa20c0d1810 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.427+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800cc30 con 0x7fa20c0d1810 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.427+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c0cb900 con 0x7fa20c0d1810 2024-04-25T10:01:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.427+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8013470 con 0x7fa20c0d1810 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.427+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c0cbec0 con 0x7fa20c0d1810 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.428+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c0cc0c0 con 0x7fa20c0d1810 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.428+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1f801a070 con 0x7fa20c0d1810 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.428+0000 7fa2097fa640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.429+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1f808a510 con 0x7fa20c0d1810 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.429+0000 7fa210b5d640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.429+0000 7fa210b5d640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1f006b4d0 0x7fa1f006d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa200005880 tx=0x7fa200009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.433+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1f8058c40 con 0x7fa20c0d1810 2024-04-25T10:01:34.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.669+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa20c073990 con 0x7fa20c0d1810 2024-04-25T10:01:34.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.669+0000 7fa2097fa640 1 -- 172.21.15.148:0/2860026885 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fa1f805cce0 con 0x7fa20c0d1810 2024-04-25T10:01:34.674 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:34.674 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":455,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2543058944,"bytes_avail":770551054336,"bytes_total":773094113280,"write_bytes_sec":35840997,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-25T10:01:29.108234+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1f006b4d0 msgr2=0x7fa1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1f006b4d0 0x7fa1f006d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa200005880 tx=0x7fa200009040 comp rx=0 tx=0).stop 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 msgr2=0x7fa20c0cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 0x7fa20c0cb100 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800ebe0 tx=0x7fa1f800c520 comp rx=0 tx=0).stop 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 shutdown_connections 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20c15dc90 0x7fa20c1e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa20c0d1810 0x7fa20c0cb100 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 --2- 172.21.15.148:0/2860026885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20c0cc8e0 0x7fa20c158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 >> 172.21.15.148:0/2860026885 conn(0x7fa20c06f590 msgr2=0x7fa20c0cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 shutdown_connections 2024-04-25T10:01:34.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:34.671+0000 7fa212dc1640 1 -- 172.21.15.148:0/2860026885 wait complete. 2024-04-25T10:01:34.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:01:35.687 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:35.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.792+0000 7ff0af02a640 1 Processor -- start 2024-04-25T10:01:35.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.792+0000 7ff0af02a640 1 -- start start 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0af02a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0a815dd30 0x7ff0a80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0af02a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0a80b9770 0x7ff0a80d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0af02a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a80d1df0 con 0x7ff0a80d2e90 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0af02a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80b7a50 con 0x7ff0a80b9770 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0af02a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80d20d0 con 0x7ff0a815dd30 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0a80b9770 0x7ff0a80d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0a80b9770 0x7ff0a80d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34058/0 (socket says 172.21.15.148:34058) 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 -- 172.21.15.148:0/1321494798 learned_addr learned my addr 172.21.15.148:0/1321494798 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0acdc6640 1 --2- 172.21.15.148:0/1321494798 >> v2:172.21.15.188:3300/0 conn(0x7ff0a815dd30 0x7ff0a80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 -- 172.21.15.148:0/1321494798 >> v2:172.21.15.188:3300/0 conn(0x7ff0a815dd30 msgr2=0x7ff0a80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 --2- 172.21.15.148:0/1321494798 >> v2:172.21.15.188:3300/0 conn(0x7ff0a815dd30 0x7ff0a80b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 -- 172.21.15.148:0/1321494798 >> v1:172.21.15.188:6789/0 conn(0x7ff0a80d2e90 legacy=0x7ff0a80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.793+0000 7ff0a7fff640 1 -- 172.21.15.148:0/1321494798 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80d23b0 con 0x7ff0a80b9770 2024-04-25T10:01:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.794+0000 7ff0a7fff640 1 --2- 172.21.15.148:0/1321494798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80b9770 0x7ff0a80d03c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff0a806d000 tx=0x7ff09402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba684fa54315d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.794+0000 7ff0a77fe640 1 -- 172.21.15.148:0/1321494798 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09402ae10 con 0x7ff0a80b9770 2024-04-25T10:01:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.794+0000 7ff0a77fe640 1 -- 172.21.15.148:0/1321494798 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094004030 con 0x7ff0a80b9770 2024-04-25T10:01:35.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.794+0000 7ff0a77fe640 1 -- 172.21.15.148:0/1321494798 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094004460 con 0x7ff0a80b9770 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 -- 172.21.15.148:0/1321494798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80b9770 msgr2=0x7ff0a80d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 --2- 172.21.15.148:0/1321494798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80b9770 0x7ff0a80d03c0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff0a806d000 tx=0x7ff09402a100 comp rx=0 tx=0).stop 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 -- 172.21.15.148:0/1321494798 shutdown_connections 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 --2- 172.21.15.148:0/1321494798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a80b9770 0x7ff0a80d03c0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 --2- 172.21.15.148:0/1321494798 >> v2:172.21.15.188:3300/0 conn(0x7ff0a815dd30 0x7ff0a80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 -- 172.21.15.148:0/1321494798 >> 172.21.15.148:0/1321494798 conn(0x7ff0a8061c30 msgr2=0x7ff0a806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 -- 172.21.15.148:0/1321494798 shutdown_connections 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.795+0000 7ff0af02a640 1 -- 172.21.15.148:0/1321494798 wait complete. 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.796+0000 7ff0af02a640 1 Processor -- start 2024-04-25T10:01:35.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.796+0000 7ff0af02a640 1 -- start start 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0af02a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a80b9770 0x7ff0a814cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0af02a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a80d2e90 0x7ff0a81e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0af02a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 0x7ff0a814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0af02a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80d1a10 con 0x7ff0a815dd30 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0af02a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80d2970 con 0x7ff0a80b9770 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0af02a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80d1b90 con 0x7ff0a80d2e90 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 0x7ff0a814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 0x7ff0a814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34070/0 (socket says 172.21.15.148:34070) 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 -- 172.21.15.148:0/2288781916 learned_addr learned my addr 172.21.15.148:0/2288781916 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0acdc6640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a80b9770 0x7ff0a814cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0a7fff640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a80d2e90 0x7ff0a81e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 -- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a80d2e90 msgr2=0x7ff0a81e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a80d2e90 0x7ff0a81e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 -- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a80b9770 msgr2=0x7ff0a814cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a80b9770 0x7ff0a814cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.797+0000 7ff0ad5c7640 1 -- 172.21.15.148:0/2288781916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a814d320 con 0x7ff0a815dd30 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.798+0000 7ff0ad5c7640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 0x7ff0a814d830 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c780 tx=0x7ff0a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.798+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0017070 con 0x7ff0a815dd30 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.798+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0003da0 con 0x7ff0a815dd30 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.798+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a80cbbc0 con 0x7ff0a815dd30 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.798+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00052b0 con 0x7ff0a815dd30 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.798+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a80cc180 con 0x7ff0a815dd30 2024-04-25T10:01:35.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.799+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a8073990 con 0x7ff0a815dd30 2024-04-25T10:01:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.799+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0a001a070 con 0x7ff0a815dd30 2024-04-25T10:01:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.800+0000 7ff0a57fa640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff08c06b4d0 0x7ff08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.800+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0a00899b0 con 0x7ff0a815dd30 2024-04-25T10:01:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.800+0000 7ff0acdc6640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff08c06b4d0 0x7ff08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.800+0000 7ff0acdc6640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff08c06b4d0 0x7ff08c06d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00aa20 tx=0x7ff09c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:35.804+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0a814d320 con 0x7ff0a815dd30 2024-04-25T10:01:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.013+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0a80c7480 con 0x7ff0a815dd30 2024-04-25T10:01:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.013+0000 7ff0a57fa640 1 -- 172.21.15.148:0/2288781916 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ff0a0057fc0 con 0x7ff0a815dd30 2024-04-25T10:01:36.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:36.018 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:36.018 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff08c06b4d0 msgr2=0x7ff08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff08c06b4d0 0x7ff08c06d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00aa20 tx=0x7ff09c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 msgr2=0x7ff0a814d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 0x7ff0a814d830 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c780 tx=0x7ff0a000cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 shutdown_connections 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff08c06b4d0 0x7ff08c06d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0a815dd30 0x7ff0a814d830 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a80d2e90 0x7ff0a81e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 --2- 172.21.15.148:0/2288781916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a80b9770 0x7ff0a814cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 >> 172.21.15.148:0/2288781916 conn(0x7ff0a8061c30 msgr2=0x7ff0a80c7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 shutdown_connections 2024-04-25T10:01:36.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.016+0000 7ff0af02a640 1 -- 172.21.15.148:0/2288781916 wait complete. 2024-04-25T10:01:36.031 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:01:36.031 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:36.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 Processor -- start 2024-04-25T10:01:36.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 -- start start 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c600d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c600b92a0 0x7f4c600d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600b97a0 con 0x7f4c600b2cb0 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6015dfc0 con 0x7f4c600b92a0 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.123+0000 7f4c651ed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600d1a00 con 0x7f4c6015dbe0 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c57fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4c600b2cb0 0x7f4c600b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46614/0 (socket says 172.21.15.148:46614) 2024-04-25T10:01:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c57fff640 1 -- 172.21.15.148:0/2022485461 learned_addr learned my addr 172.21.15.148:0/2022485461 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5f59e640 1 --2- 172.21.15.148:0/2022485461 >> v2:172.21.15.188:3300/0 conn(0x7f4c600b92a0 0x7f4c600d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755050113 0 0) 0x7f4c600b97a0 con 0x7f4c600b2cb0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c003520 con 0x7f4c600b2cb0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5ed9d640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c600d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669425574 0 0) 0x7f4c4c003520 con 0x7f4c600b2cb0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 msgr2=0x7f4c600d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c600d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 >> v2:172.21.15.188:3300/0 conn(0x7f4c600b92a0 msgr2=0x7f4c600d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 --2- 172.21.15.148:0/2022485461 >> v2:172.21.15.188:3300/0 conn(0x7f4c600b92a0 0x7f4c600d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600d1c50 con 0x7f4c600b2cb0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4c48001670 con 0x7f4c600b2cb0 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 >> v1:172.21.15.148:6789/0 conn(0x7f4c600b2cb0 legacy=0x7f4c600b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0039b0 0x7f4c4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600d1a00 con 0x7f4c4c004800 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c00b9c0 con 0x7f4c4c0039b0 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c00bb40 con 0x7f4c6015dbe0 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c57fff640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5ed9d640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0039b0 0x7f4c4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.124+0000 7f4c5f59e640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c57fff640 1 -- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 msgr2=0x7f4c4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c57fff640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c57fff640 1 -- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0039b0 msgr2=0x7f4c4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c57fff640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c0039b0 0x7f4c4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c57fff640 1 -- 172.21.15.148:0/2022485461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c003520 con 0x7f4c4c004800 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c5f59e640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015dbe0 0x7f4c4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:36.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c57fff640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4c48007fd0 tx=0x7f4c4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e40ac3ff6d843d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48007d50 con 0x7f4c4c004800 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48005ad0 con 0x7f4c4c004800 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.125+0000 7f4c5e59c640 1 -- 172.21.15.148:0/2022485461 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48007070 con 0x7f4c4c004800 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c651ed640 1 -- 172.21.15.148:0/2022485461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c004800 msgr2=0x7f4c4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c651ed640 1 --2- 172.21.15.148:0/2022485461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4c48007fd0 tx=0x7f4c4c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c5f59e640 1 -- 172.21.15.148:0/2022485461 reap_dead start 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c651ed640 1 -- 172.21.15.148:0/2022485461 shutdown_connections 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c651ed640 1 -- 172.21.15.148:0/2022485461 >> 172.21.15.148:0/2022485461 conn(0x7f4c60061a20 msgr2=0x7f4c60061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c651ed640 1 -- 172.21.15.148:0/2022485461 shutdown_connections 2024-04-25T10:01:36.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.126+0000 7f4c651ed640 1 -- 172.21.15.148:0/2022485461 wait complete. 2024-04-25T10:01:36.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.127+0000 7f4c651ed640 1 Processor -- start 2024-04-25T10:01:36.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.127+0000 7f4c651ed640 1 -- start start 2024-04-25T10:01:36.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c651ed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c004800 0x7f4c60151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c651ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 0x7f4c601f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c651ed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015c770 0x7f4c6015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c651ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b6ab0 con 0x7f4c601f66b0 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c651ed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600d2210 con 0x7f4c4c004800 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c651ed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b6c30 con 0x7f4c6015c770 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 0x7f4c601f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 0x7f4c601f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34092/0 (socket says 172.21.15.148:34092) 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.128+0000 7f4c57fff640 1 -- 172.21.15.148:0/3946263382 learned_addr learned my addr 172.21.15.148:0/3946263382 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c5ed9d640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c004800 0x7f4c60151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c57fff640 1 -- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015c770 msgr2=0x7f4c6015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c5f59e640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015c770 0x7f4c6015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c57fff640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015c770 0x7f4c6015cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c57fff640 1 -- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c004800 msgr2=0x7f4c60151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c57fff640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c004800 0x7f4c60151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c57fff640 1 -- 172.21.15.148:0/3946263382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6015ec50 con 0x7f4c601f66b0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c57fff640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 0x7f4c601f6b10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800c2d0 tx=0x7f4c600b9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48004bf0 con 0x7f4c601f66b0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48043070 con 0x7f4c601f66b0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c601f7200 con 0x7f4c601f66b0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48007070 con 0x7f4c601f66b0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.129+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c600b4720 con 0x7f4c601f66b0 2024-04-25T10:01:36.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.130+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60073990 con 0x7f4c601f66b0 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.130+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c48007210 con 0x7f4c601f66b0 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.131+0000 7f4c577fe640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4006b500 0x7f4c4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.131+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c480aab70 con 0x7f4c601f66b0 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.131+0000 7f4c5ed9d640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4006b500 0x7f4c4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.131+0000 7f4c5ed9d640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4006b500 0x7f4c4006d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4c500098a0 tx=0x7f4c50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.135+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c48079a10 con 0x7f4c601f66b0 2024-04-25T10:01:36.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.375+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4c6015ab50 con 0x7f4c601f66b0 2024-04-25T10:01:36.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.375+0000 7f4c577fe640 1 -- 172.21.15.148:0/3946263382 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f4c48079310 con 0x7f4c601f66b0 2024-04-25T10:01:36.380 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:36.380 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":457,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2543058944,"bytes_avail":770551054336,"bytes_total":773094113280,"write_bytes_sec":35841476,"read_op_per_sec":0,"write_op_per_sec":35},"fsmap":{"epoch":60,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8163}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T10:01:35.109573+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4006b500 msgr2=0x7f4c4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4006b500 0x7f4c4006d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4c500098a0 tx=0x7f4c50008040 comp rx=0 tx=0).stop 2024-04-25T10:01:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 msgr2=0x7f4c601f6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 0x7f4c601f6b10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800c2d0 tx=0x7f4c600b9970 comp rx=0 tx=0).stop 2024-04-25T10:01:36.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 shutdown_connections 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4c4006b500 0x7f4c4006d9c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c6015c770 0x7f4c6015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c601f66b0 0x7f4c601f6b10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 --2- 172.21.15.148:0/3946263382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c4c004800 0x7f4c60151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 >> 172.21.15.148:0/3946263382 conn(0x7f4c60061a20 msgr2=0x7f4c6015ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 shutdown_connections 2024-04-25T10:01:36.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.377+0000 7f4c651ed640 1 -- 172.21.15.148:0/3946263382 wait complete. 2024-04-25T10:01:36.393 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:01:36.393 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:01:36.393 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:36.431 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:36.477 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:01:36.477 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:36.478 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:36.493 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:36.493 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:36.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:36.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.680+0000 7f2723e5d640 1 Processor -- start 2024-04-25T10:01:36.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.681+0000 7f2723e5d640 1 -- start start 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.681+0000 7f2723e5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0b2cb0 0x7f271c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.681+0000 7f2723e5d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f271c0d1020 0x7f271c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.681+0000 7f2723e5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c0b97a0 con 0x7f271c15dbe0 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.681+0000 7f2723e5d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c15dfc0 con 0x7f271c0d1020 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.681+0000 7f2723e5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0d1a00 con 0x7f271c0b2cb0 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f271c0d1020 0x7f271c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f271c0d1020 0x7f271c0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34106/0 (socket says 172.21.15.148:34106) 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 -- 172.21.15.148:0/84446011 learned_addr learned my addr 172.21.15.148:0/84446011 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 -- 172.21.15.148:0/84446011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0b2cb0 msgr2=0x7f271c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27213f8640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0b2cb0 0x7f271c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0b2cb0 0x7f271c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 -- 172.21.15.148:0/84446011 >> v1:172.21.15.148:6789/0 conn(0x7f271c15dbe0 legacy=0x7f271c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f2720bf7640 1 -- 172.21.15.148:0/84446011 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396001695 0 0) 0x7f271c0b97a0 con 0x7f271c15dbe0 2024-04-25T10:01:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27223fa640 1 -- 172.21.15.148:0/84446011 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0d1c50 con 0x7f271c0d1020 2024-04-25T10:01:36.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.682+0000 7f27213f8640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0b2cb0 0x7f271c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f27223fa640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c0d1020 0x7f271c0d2c70 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2714009e50 tx=0x7f271402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fede529f708f582c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2720bf7640 1 -- 172.21.15.148:0/84446011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714036070 con 0x7f271c0d1020 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2720bf7640 1 -- 172.21.15.148:0/84446011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714031600 con 0x7f271c0d1020 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2720bf7640 1 -- 172.21.15.148:0/84446011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27140319d0 con 0x7f271c0d1020 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 -- 172.21.15.148:0/84446011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c0d1020 msgr2=0x7f271c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c0d1020 0x7f271c0d2c70 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f2714009e50 tx=0x7f271402a710 comp rx=0 tx=0).stop 2024-04-25T10:01:36.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 -- 172.21.15.148:0/84446011 shutdown_connections 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c0d1020 0x7f271c0d2c70 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 --2- 172.21.15.148:0/84446011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0b2cb0 0x7f271c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 -- 172.21.15.148:0/84446011 >> 172.21.15.148:0/84446011 conn(0x7f271c061a20 msgr2=0x7f271c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 -- 172.21.15.148:0/84446011 shutdown_connections 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.683+0000 7f2723e5d640 1 -- 172.21.15.148:0/84446011 wait complete. 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.684+0000 7f2723e5d640 1 Processor -- start 2024-04-25T10:01:36.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 -- start start 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f271c0b2cb0 0x7f271c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0d1020 0x7f271c1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 0x7f271c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0b92a0 con 0x7f271c15dbe0 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0d2210 con 0x7f271c0b2cb0 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f2723e5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0b9420 con 0x7f271c0d1020 2024-04-25T10:01:36.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f27223fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 0x7f271c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f27223fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 0x7f271c159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34120/0 (socket says 172.21.15.148:34120) 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.685+0000 7f27223fa640 1 -- 172.21.15.148:0/2849950184 learned_addr learned my addr 172.21.15.148:0/2849950184 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27213f8640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0d1020 0x7f271c1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f2721bf9640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f271c0b2cb0 0x7f271c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27223fa640 1 -- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0d1020 msgr2=0x7f271c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27223fa640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0d1020 0x7f271c1560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27223fa640 1 -- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f271c0b2cb0 msgr2=0x7f271c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27223fa640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f271c0b2cb0 0x7f271c154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27223fa640 1 -- 172.21.15.148:0/2849950184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c158bf0 con 0x7f271c15dbe0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27223fa640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 0x7f271c159c90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2714009e50 tx=0x7f2714004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714041020 con 0x7f271c15dbe0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714037040 con 0x7f271c15dbe0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c158ed0 con 0x7f271c15dbe0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714036050 con 0x7f271c15dbe0 2024-04-25T10:01:36.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.686+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c159490 con 0x7f271c15dbe0 2024-04-25T10:01:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.687+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c073990 con 0x7f271c15dbe0 2024-04-25T10:01:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.687+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2714007c30 con 0x7f271c15dbe0 2024-04-25T10:01:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.688+0000 7f27127fc640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f270006b500 0x7f270006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.688+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27140aac40 con 0x7f271c15dbe0 2024-04-25T10:01:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.688+0000 7f2721bf9640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f270006b500 0x7f270006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.688+0000 7f2721bf9640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f270006b500 0x7f270006d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2718000f60 tx=0x7f2718009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.692+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f271404b060 con 0x7f271c15dbe0 2024-04-25T10:01:36.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.865+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f271c06b080 con 0x7f271c15dbe0 2024-04-25T10:01:36.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.866+0000 7f27127fc640 1 -- 172.21.15.148:0/2849950184 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f27140799c0 con 0x7f271c15dbe0 2024-04-25T10:01:36.871 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:36.871 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":109,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:09.636711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f270006b500 msgr2=0x7f270006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f270006b500 0x7f270006d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2718000f60 tx=0x7f2718009040 comp rx=0 tx=0).stop 2024-04-25T10:01:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 msgr2=0x7f271c159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 0x7f271c159c90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2714009e50 tx=0x7f2714004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 shutdown_connections 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f270006b500 0x7f270006d9c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f271c15dbe0 0x7f271c159c90 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f271c0d1020 0x7f271c1560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 --2- 172.21.15.148:0/2849950184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f271c0b2cb0 0x7f271c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 >> 172.21.15.148:0/2849950184 conn(0x7f271c061a20 msgr2=0x7f271c160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 shutdown_connections 2024-04-25T10:01:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.868+0000 7f2723e5d640 1 -- 172.21.15.148:0/2849950184 wait complete. 2024-04-25T10:01:36.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:06.175634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:06.926396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:36.884 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.979+0000 7fa72bea3640 1 Processor -- start 2024-04-25T10:01:36.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.979+0000 7fa72bea3640 1 -- start start 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72bea3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 0x7fa7240ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72bea3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa7240d32f0 0x7fa7240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72bea3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72415e000 con 0x7fa724161710 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72bea3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7240d1de0 con 0x7fa7240d32f0 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72bea3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa724161c60 con 0x7fa72415dc20 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72943e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa724161710 0x7fa72406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46632/0 (socket says 172.21.15.148:46632) 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72943e640 1 -- 172.21.15.148:0/3740575951 learned_addr learned my addr 172.21.15.148:0/3740575951 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa72a440640 1 --2- 172.21.15.148:0/3740575951 >> v2:172.21.15.188:3300/0 conn(0x7fa7240d32f0 0x7fa7240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.980+0000 7fa729c3f640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 0x7fa7240ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125663885 0 0) 0x7fa72415e000 con 0x7fa724161710 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c003520 con 0x7fa724161710 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2308710344 0 0) 0x7fa70c003520 con 0x7fa724161710 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 msgr2=0x7fa7240ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 0x7fa7240ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 >> v2:172.21.15.188:3300/0 conn(0x7fa7240d32f0 msgr2=0x7fa7240cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 --2- 172.21.15.148:0/3740575951 >> v2:172.21.15.188:3300/0 conn(0x7fa7240d32f0 0x7fa7240cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724161f40 con 0x7fa724161710 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa718001670 con 0x7fa724161710 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72a440640 1 --2- 172.21.15.148:0/3740575951 >> v2:172.21.15.188:3300/0 conn(0x7fa7240d32f0 0x7fa7240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 >> v1:172.21.15.148:6789/0 conn(0x7fa724161710 legacy=0x7fa72406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 0x7fa70c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa70c003f30 0x7fa70c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa70c004d40 0x7fa70c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240d1de0 con 0x7fa70c003f30 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa724161c60 con 0x7fa70c004d40 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c00b9c0 con 0x7fa72415dc20 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72943e640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa70c003f30 0x7fa70c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa729c3f640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 0x7fa70c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72a440640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa70c004d40 0x7fa70c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72943e640 1 -- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 msgr2=0x7fa70c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72943e640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72415dc20 0x7fa70c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72943e640 1 -- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa70c004d40 msgr2=0x7fa70c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72943e640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa70c004d40 0x7fa70c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.981+0000 7fa72943e640 1 -- 172.21.15.148:0/3740575951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c003520 con 0x7fa70c003f30 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.982+0000 7fa72943e640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa70c003f30 0x7fa70c004800 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa71800c290 tx=0x7fa70c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520cb100363f84c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.982+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718007d50 con 0x7fa70c003f30 2024-04-25T10:01:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.982+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71802e070 con 0x7fa70c003f30 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.982+0000 7fa728c3d640 1 -- 172.21.15.148:0/3740575951 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718005d70 con 0x7fa70c003f30 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72bea3640 1 -- 172.21.15.148:0/3740575951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa70c003f30 msgr2=0x7fa70c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72bea3640 1 --2- 172.21.15.148:0/3740575951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa70c003f30 0x7fa70c004800 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa71800c290 tx=0x7fa70c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72a440640 1 -- 172.21.15.148:0/3740575951 reap_dead start 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72bea3640 1 -- 172.21.15.148:0/3740575951 shutdown_connections 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72bea3640 1 -- 172.21.15.148:0/3740575951 >> 172.21.15.148:0/3740575951 conn(0x7fa7240d0420 msgr2=0x7fa7240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72bea3640 1 -- 172.21.15.148:0/3740575951 shutdown_connections 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.983+0000 7fa72bea3640 1 -- 172.21.15.148:0/3740575951 wait complete. 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.984+0000 7fa72bea3640 1 Processor -- start 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.984+0000 7fa72bea3640 1 -- start start 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72bea3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa70c003f30 0x7fa7241f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72bea3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7240cd280 0x7fa7240cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72bea3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 0x7fa7240b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72bea3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240b5e50 con 0x7fa7240b7b50 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72bea3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa724162500 con 0x7fa7240cd280 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72bea3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240b5fd0 con 0x7fa70c003f30 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 0x7fa7240b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 0x7fa7240b7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34142/0 (socket says 172.21.15.148:34142) 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 -- 172.21.15.148:0/775479041 learned_addr learned my addr 172.21.15.148:0/775479041 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa729c3f640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa70c003f30 0x7fa7241f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72943e640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7240cd280 0x7fa7240cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 -- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa70c003f30 msgr2=0x7fa7241f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa70c003f30 0x7fa7241f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 -- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7240cd280 msgr2=0x7fa7240cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7240cd280 0x7fa7240cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 -- 172.21.15.148:0/775479041 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240cdd40 con 0x7fa7240b7b50 2024-04-25T10:01:36.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa72a440640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 0x7fa7240b7fd0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa7240d3780 tx=0x7fa71c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c019070 con 0x7fa7240b7b50 2024-04-25T10:01:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.985+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c00f950 con 0x7fa7240b7b50 2024-04-25T10:01:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.986+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00fd40 con 0x7fa7240b7b50 2024-04-25T10:01:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.986+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7240ce020 con 0x7fa7240b7b50 2024-04-25T10:01:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.986+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7240ce5e0 con 0x7fa7240b7b50 2024-04-25T10:01:36.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.987+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7240ca780 con 0x7fa7240b7b50 2024-04-25T10:01:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.987+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa71c01c070 con 0x7fa7240b7b50 2024-04-25T10:01:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.987+0000 7fa7227fc640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.987+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa71c08a580 con 0x7fa7240b7b50 2024-04-25T10:01:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.987+0000 7fa729c3f640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.988+0000 7fa729c3f640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa7240b9690 tx=0x7fa71400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:36.992+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa71c058c20 con 0x7fa7240b7b50 2024-04-25T10:01:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.164+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7240d16b0 con 0x7fa7240b7b50 2024-04-25T10:01:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.164+0000 7fa7227fc640 1 -- 172.21.15.148:0/775479041 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa71c05ccc0 con 0x7fa7240b7b50 2024-04-25T10:01:37.169 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:01:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 msgr2=0x7fa71006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa7240b9690 tx=0x7fa71400b040 comp rx=0 tx=0).stop 2024-04-25T10:01:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 msgr2=0x7fa7240b7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 0x7fa7240b7fd0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa7240d3780 tx=0x7fa71c011cf0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 shutdown_connections 2024-04-25T10:01:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa71006b4d0 0x7fa71006d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7240b7b50 0x7fa7240b7fd0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7240cd280 0x7fa7240cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 --2- 172.21.15.148:0/775479041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa70c003f30 0x7fa7241f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 >> 172.21.15.148:0/775479041 conn(0x7fa7240d0420 msgr2=0x7fa7240c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 shutdown_connections 2024-04-25T10:01:37.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.166+0000 7fa72bea3640 1 -- 172.21.15.148:0/775479041 wait complete. 2024-04-25T10:01:37.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:37.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.277+0000 7fbc6a61f640 1 Processor -- start 2024-04-25T10:01:37.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.277+0000 7fbc6a61f640 1 -- start start 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc6a61f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc64158230 0x7fbc640d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc6a61f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc640d1b80 0x7fbc6406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc6a61f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640d2020 con 0x7fbc6406c870 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc6a61f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc64158610 con 0x7fbc64158230 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc6a61f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6406bea0 con 0x7fbc640d1b80 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc64158230 0x7fbc640d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbc64158230 0x7fbc640d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34158/0 (socket says 172.21.15.148:34158) 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 -- 172.21.15.148:0/1620600142 learned_addr learned my addr 172.21.15.148:0/1620600142 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 -- 172.21.15.148:0/1620600142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc640d1b80 msgr2=0x7fbc6406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 --2- 172.21.15.148:0/1620600142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc640d1b80 0x7fbc6406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 -- 172.21.15.148:0/1620600142 >> v1:172.21.15.188:6789/0 conn(0x7fbc6406c870 legacy=0x7fbc640b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.278+0000 7fbc63fff640 1 -- 172.21.15.148:0/1620600142 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640d2300 con 0x7fbc64158230 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc63fff640 1 --2- 172.21.15.148:0/1620600142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc64158230 0x7fbc640d1640 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009870 tx=0x7fbc5402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e56ab7c3f8c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc62ffd640 1 -- 172.21.15.148:0/1620600142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54037070 con 0x7fbc64158230 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc62ffd640 1 -- 172.21.15.148:0/1620600142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc54032440 con 0x7fbc64158230 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc62ffd640 1 -- 172.21.15.148:0/1620600142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54032810 con 0x7fbc64158230 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc6a61f640 1 -- 172.21.15.148:0/1620600142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc64158230 msgr2=0x7fbc640d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/1620600142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc64158230 0x7fbc640d1640 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009870 tx=0x7fbc5402a210 comp rx=0 tx=0).stop 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc6a61f640 1 -- 172.21.15.148:0/1620600142 shutdown_connections 2024-04-25T10:01:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/1620600142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc640d1b80 0x7fbc6406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.279+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/1620600142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc64158230 0x7fbc640d1640 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.280+0000 7fbc6a61f640 1 -- 172.21.15.148:0/1620600142 >> 172.21.15.148:0/1620600142 conn(0x7fbc64061600 msgr2=0x7fbc640619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.280+0000 7fbc6a61f640 1 -- 172.21.15.148:0/1620600142 shutdown_connections 2024-04-25T10:01:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.280+0000 7fbc6a61f640 1 -- 172.21.15.148:0/1620600142 wait complete. 2024-04-25T10:01:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.281+0000 7fbc6a61f640 1 Processor -- start 2024-04-25T10:01:37.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.281+0000 7fbc6a61f640 1 -- start start 2024-04-25T10:01:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 0x7fbc640caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc640d1b80 0x7fbc64156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc64158230 0x7fbc640cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6415e320 con 0x7fbc6406c870 2024-04-25T10:01:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc640d25e0 con 0x7fbc640d1b80 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6415e4a0 con 0x7fbc64158230 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 0x7fbc640caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 0x7fbc640caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34168/0 (socket says 172.21.15.148:34168) 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 -- 172.21.15.148:0/238685878 learned_addr learned my addr 172.21.15.148:0/238685878 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 -- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc64158230 msgr2=0x7fbc640cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc637fe640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc640d1b80 0x7fbc64156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc68bbc640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc64158230 0x7fbc640cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc64158230 0x7fbc640cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 -- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc640d1b80 msgr2=0x7fbc64156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc640d1b80 0x7fbc64156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 -- 172.21.15.148:0/238685878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640cb120 con 0x7fbc6406c870 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc637fe640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc640d1b80 0x7fbc64156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc68bbc640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc64158230 0x7fbc640cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc63fff640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 0x7fbc640caa60 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000c00 tx=0x7fbc540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54041070 con 0x7fbc6406c870 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5403c870 con 0x7fbc6406c870 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc640cb400 con 0x7fbc6406c870 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54037040 con 0x7fbc6406c870 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.282+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc640cb9c0 con 0x7fbc6406c870 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.284+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6406de50 con 0x7fbc6406c870 2024-04-25T10:01:37.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.284+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc54044070 con 0x7fbc6406c870 2024-04-25T10:01:37.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.284+0000 7fbc60ff9640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc5006b500 0x7fbc5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.285+0000 7fbc637fe640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc5006b500 0x7fbc5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.285+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc540aae40 con 0x7fbc6406c870 2024-04-25T10:01:37.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.285+0000 7fbc637fe640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc5006b500 0x7fbc5006d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbc6414b210 tx=0x7fbc58008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.289+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc54079b30 con 0x7fbc6406c870 2024-04-25T10:01:37.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.457+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc64073a00 con 0x7fbc6406c870 2024-04-25T10:01:37.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.458+0000 7fbc60ff9640 1 -- 172.21.15.148:0/238685878 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc54079430 con 0x7fbc6406c870 2024-04-25T10:01:37.463 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc5006b500 msgr2=0x7fbc5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc5006b500 0x7fbc5006d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbc6414b210 tx=0x7fbc58008040 comp rx=0 tx=0).stop 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 msgr2=0x7fbc640caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 0x7fbc640caa60 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000c00 tx=0x7fbc540046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 shutdown_connections 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbc5006b500 0x7fbc5006d9c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc64158230 0x7fbc640cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc640d1b80 0x7fbc64156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 --2- 172.21.15.148:0/238685878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc6406c870 0x7fbc640caa60 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 >> 172.21.15.148:0/238685878 conn(0x7fbc64061600 msgr2=0x7fbc64153330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 shutdown_connections 2024-04-25T10:01:37.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.460+0000 7fbc6a61f640 1 -- 172.21.15.148:0/238685878 wait complete. 2024-04-25T10:01:37.475 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:37.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.571+0000 7f033eb7e640 1 Processor -- start 2024-04-25T10:01:37.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.571+0000 7f033eb7e640 1 -- start start 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033eb7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03380ce230 0x7f03380cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033eb7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f033815b270 0x7f033815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033eb7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0338072280 con 0x7f03380cc4f0 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033eb7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0338071270 con 0x7f03380ce230 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033eb7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380714c0 con 0x7f033815b270 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03380ce230 0x7f03380cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03380ce230 0x7f03380cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34182/0 (socket says 172.21.15.148:34182) 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 -- 172.21.15.148:0/3813123259 learned_addr learned my addr 172.21.15.148:0/3813123259 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033d11b640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f033815b270 0x7f033815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 -- 172.21.15.148:0/3813123259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f033815b270 msgr2=0x7f033815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f033815b270 0x7f033815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 -- 172.21.15.148:0/3813123259 >> v1:172.21.15.188:6789/0 conn(0x7f03380cc4f0 legacy=0x7f03380cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f0337fff640 1 -- 172.21.15.148:0/3813123259 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380717a0 con 0x7f03380ce230 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.572+0000 7f033d11b640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f033815b270 0x7f033815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.573+0000 7f0337fff640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380ce230 0x7f03380cc8d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f032c00bb10 tx=0x7f032c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e204b2003c752df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.573+0000 7f03377fe640 1 -- 172.21.15.148:0/3813123259 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c036020 con 0x7f03380ce230 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.573+0000 7f03377fe640 1 -- 172.21.15.148:0/3813123259 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c032820 con 0x7f03380ce230 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.573+0000 7f03377fe640 1 -- 172.21.15.148:0/3813123259 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c032bf0 con 0x7f03380ce230 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 -- 172.21.15.148:0/3813123259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380ce230 msgr2=0x7f03380cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380ce230 0x7f03380cc8d0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f032c00bb10 tx=0x7f032c02c920 comp rx=0 tx=0).stop 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 -- 172.21.15.148:0/3813123259 shutdown_connections 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380ce230 0x7f03380cc8d0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 --2- 172.21.15.148:0/3813123259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f033815b270 0x7f033815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 -- 172.21.15.148:0/3813123259 >> 172.21.15.148:0/3813123259 conn(0x7f033806abe0 msgr2=0x7f033806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 -- 172.21.15.148:0/3813123259 shutdown_connections 2024-04-25T10:01:37.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.574+0000 7f033eb7e640 1 -- 172.21.15.148:0/3813123259 wait complete. 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.575+0000 7f033eb7e640 1 Processor -- start 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.575+0000 7f033eb7e640 1 -- start start 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033eb7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 0x7f0338147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033eb7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03380ce230 0x7f03381e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033eb7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f033815b270 0x7f0338153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033eb7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033815bbf0 con 0x7f03380cc4f0 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033eb7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338071d60 con 0x7f033815b270 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033eb7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033815bd70 con 0x7f03380ce230 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 0x7f0338147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 0x7f0338147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34198/0 (socket says 172.21.15.148:34198) 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 -- 172.21.15.148:0/2951526413 learned_addr learned my addr 172.21.15.148:0/2951526413 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033d11b640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f033815b270 0x7f0338153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f0337fff640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03380ce230 0x7f03381e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 -- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03380ce230 msgr2=0x7f03381e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03380ce230 0x7f03381e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 -- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f033815b270 msgr2=0x7f0338153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f033815b270 0x7f0338153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 -- 172.21.15.148:0/2951526413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338152190 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.576+0000 7f033c91a640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 0x7f0338147d70 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3f0 tx=0x7f032800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.577+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03280052d0 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.577+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0328005490 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.577+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0338152470 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.577+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328007260 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.577+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0338152a30 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.578+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0338073a00 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.578+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0328007400 con 0x7f03380cc4f0 2024-04-25T10:01:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.578+0000 7f03357fa640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032406b4d0 0x7f032406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.578+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0328089730 con 0x7f03380cc4f0 2024-04-25T10:01:37.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.578+0000 7f0337fff640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032406b4d0 0x7f032406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.579+0000 7f0337fff640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032406b4d0 0x7f032406d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f032c037000 tx=0x7f032c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.583+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0328057e60 con 0x7f03380cc4f0 2024-04-25T10:01:37.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.753+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f033806c250 con 0x7f03380cc4f0 2024-04-25T10:01:37.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.753+0000 7f03357fa640 1 -- 172.21.15.148:0/2951526413 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f032805ce20 con 0x7f03380cc4f0 2024-04-25T10:01:37.758 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:01:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032406b4d0 msgr2=0x7f032406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032406b4d0 0x7f032406d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f032c037000 tx=0x7f032c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 msgr2=0x7f0338147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 0x7f0338147d70 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f032800c3f0 tx=0x7f032800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 shutdown_connections 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f032406b4d0 0x7f032406d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f033815b270 0x7f0338153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03380ce230 0x7f03381e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 --2- 172.21.15.148:0/2951526413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03380cc4f0 0x7f0338147d70 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 >> 172.21.15.148:0/2951526413 conn(0x7f033806abe0 msgr2=0x7f033806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.755+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 shutdown_connections 2024-04-25T10:01:37.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.756+0000 7f033eb7e640 1 -- 172.21.15.148:0/2951526413 wait complete. 2024-04-25T10:01:37.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:01:37.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.865+0000 7f83e8e4a640 1 Processor -- start 2024-04-25T10:01:37.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.865+0000 7f83e8e4a640 1 -- start start 2024-04-25T10:01:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e8e4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83e415b7d0 0x7f83e40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e8e4a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f83e40b5620 0x7f83e40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e8e4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e406cf50 con 0x7f83e406c2d0 2024-04-25T10:01:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e8e4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e40ce530 con 0x7f83e415b7d0 2024-04-25T10:01:37.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e8e4a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e40ce810 con 0x7f83e40b5620 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83e415b7d0 0x7f83e40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f83e415b7d0 0x7f83e40cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34200/0 (socket says 172.21.15.148:34200) 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 -- 172.21.15.148:0/2229270902 learned_addr learned my addr 172.21.15.148:0/2229270902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83dbfff640 1 --2- 172.21.15.148:0/2229270902 >> v2:172.21.15.188:3300/0 conn(0x7f83e40b5620 0x7f83e40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 -- 172.21.15.148:0/2229270902 >> v2:172.21.15.188:3300/0 conn(0x7f83e40b5620 msgr2=0x7f83e40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 --2- 172.21.15.148:0/2229270902 >> v2:172.21.15.188:3300/0 conn(0x7f83e40b5620 0x7f83e40b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 -- 172.21.15.148:0/2229270902 >> v1:172.21.15.188:6789/0 conn(0x7f83e406c2d0 legacy=0x7f83e406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.866+0000 7f83e259c640 1 -- 172.21.15.148:0/2229270902 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40ceaf0 con 0x7f83e415b7d0 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e259c640 1 --2- 172.21.15.148:0/2229270902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e415b7d0 0x7f83e40cf010 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ce180 tx=0x7f83d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aca791bda80b33a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e1d9b640 1 -- 172.21.15.148:0/2229270902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4037070 con 0x7f83e415b7d0 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e1d9b640 1 -- 172.21.15.148:0/2229270902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d4032440 con 0x7f83e415b7d0 2024-04-25T10:01:37.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e1d9b640 1 -- 172.21.15.148:0/2229270902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4032810 con 0x7f83e415b7d0 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2229270902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e415b7d0 msgr2=0x7f83e40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2229270902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e415b7d0 0x7f83e40cf010 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ce180 tx=0x7f83d402a230 comp rx=0 tx=0).stop 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2229270902 shutdown_connections 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2229270902 >> v2:172.21.15.188:3300/0 conn(0x7f83e40b5620 0x7f83e40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2229270902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e415b7d0 0x7f83e40cf010 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.867+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2229270902 >> 172.21.15.148:0/2229270902 conn(0x7f83e40d0800 msgr2=0x7f83e40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.868+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2229270902 shutdown_connections 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.868+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2229270902 wait complete. 2024-04-25T10:01:37.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.869+0000 7f83e8e4a640 1 Processor -- start 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.869+0000 7f83e8e4a640 1 -- start start 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.869+0000 7f83e8e4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 0x7f83e41f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.869+0000 7f83e8e4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83e40b5620 0x7f83e41f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.869+0000 7f83e8e4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83e415b7d0 0x7f83e40c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e8e4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40cf6d0 con 0x7f83e406c2d0 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e8e4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40cf550 con 0x7f83e40b5620 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e8e4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e415bbb0 con 0x7f83e415b7d0 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 0x7f83e41f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 0x7f83e41f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34212/0 (socket says 172.21.15.148:34212) 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 -- 172.21.15.148:0/2517933087 learned_addr learned my addr 172.21.15.148:0/2517933087 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 -- 172.21.15.148:0/2517933087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83e415b7d0 msgr2=0x7f83e40c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83e415b7d0 0x7f83e40c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 -- 172.21.15.148:0/2517933087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83e40b5620 msgr2=0x7f83e41f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83e40b5620 0x7f83e41f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 -- 172.21.15.148:0/2517933087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40c9520 con 0x7f83e406c2d0 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e259c640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 0x7f83e41f3710 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000c00 tx=0x7f83d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4041020 con 0x7f83e406c2d0 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d40306c0 con 0x7f83e406c2d0 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e414a4d0 con 0x7f83e406c2d0 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4037040 con 0x7f83e406c2d0 2024-04-25T10:01:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.870+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e414aa90 con 0x7f83e406c2d0 2024-04-25T10:01:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.871+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e4073a00 con 0x7f83e406c2d0 2024-04-25T10:01:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.872+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83d4044020 con 0x7f83e406c2d0 2024-04-25T10:01:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.872+0000 7f83daffd640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f83c406b4d0 0x7f83c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.872+0000 7f83dbfff640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f83c406b4d0 0x7f83c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.872+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83d40aabe0 con 0x7f83e406c2d0 2024-04-25T10:01:37.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.873+0000 7f83dbfff640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f83c406b4d0 0x7f83c406d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f83e40cb1f0 tx=0x7f83cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:37.876+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83d404b030 con 0x7f83e406c2d0 2024-04-25T10:01:38.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.062+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83e415b390 con 0x7f83e406c2d0 2024-04-25T10:01:38.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.062+0000 7f83daffd640 1 -- 172.21.15.148:0/2517933087 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f83d40798d0 con 0x7f83e406c2d0 2024-04-25T10:01:38.067 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:01:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f83c406b4d0 msgr2=0x7f83c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f83c406b4d0 0x7f83c406d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f83e40cb1f0 tx=0x7f83cc008040 comp rx=0 tx=0).stop 2024-04-25T10:01:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 msgr2=0x7f83e41f3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 0x7f83e41f3710 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000c00 tx=0x7f83d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 shutdown_connections 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f83c406b4d0 0x7f83c406d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f83e415b7d0 0x7f83e40c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83e40b5620 0x7f83e41f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 --2- 172.21.15.148:0/2517933087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83e406c2d0 0x7f83e41f3710 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 >> 172.21.15.148:0/2517933087 conn(0x7f83e40d0800 msgr2=0x7f83e40d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.064+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 shutdown_connections 2024-04-25T10:01:38.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.065+0000 7f83e8e4a640 1 -- 172.21.15.148:0/2517933087 wait complete. 2024-04-25T10:01:38.080 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:01:38.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:38.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:01:38.704 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:01:38.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:01:38.705 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:01:38.719 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:01:38.719 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:38.734 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:38.734 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:38.735 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:38.774 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:38.774 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:01:38.774 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:38.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:38.790 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:38.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.906+0000 7fb6b27b0640 1 Processor -- start 2024-04-25T10:01:38.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.906+0000 7fb6b27b0640 1 -- start start 2024-04-25T10:01:38.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6b27b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6ac15ba00 0x7fb6ac09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6b27b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0cd710 con 0x7fb6ac1602e0 2024-04-25T10:01:38.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6b27b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac099b70 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6b27b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0cd4b0 con 0x7fb6ac15ba00 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6abfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6ac15ba00 0x7fb6ac09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6ab7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb6ac1602e0 0x7fb6ac0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46646/0 (socket says 172.21.15.148:46646) 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6ab7fe640 1 -- 172.21.15.148:0/3060471630 learned_addr learned my addr 172.21.15.148:0/3060471630 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.907+0000 7fb6abfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6ac15ba00 0x7fb6ac09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34220/0 (socket says 172.21.15.148:34220) 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.908+0000 7fb6abfff640 1 -- 172.21.15.148:0/3060471630 >> v1:172.21.15.188:6789/0 conn(0x7fb6ac15f5f0 legacy=0x7fb6ac0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.908+0000 7fb6abfff640 1 -- 172.21.15.148:0/3060471630 >> v1:172.21.15.148:6789/0 conn(0x7fb6ac1602e0 legacy=0x7fb6ac0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.908+0000 7fb6abfff640 1 -- 172.21.15.148:0/3060471630 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac099e50 con 0x7fb6ac15ba00 2024-04-25T10:01:38.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.908+0000 7fb6aaffd640 1 -- 172.21.15.148:0/3060471630 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045375249 0 0) 0x7fb6a0001110 con 0x7fb6ac1602e0 2024-04-25T10:01:38.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.908+0000 7fb6abfff640 1 --2- 172.21.15.148:0/3060471630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac09a930 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0097a0 tx=0x7fb69c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1c307078f1fba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.908+0000 7fb6aaffd640 1 -- 172.21.15.148:0/3060471630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c037070 con 0x7fb6ac15ba00 2024-04-25T10:01:38.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6aaffd640 1 -- 172.21.15.148:0/3060471630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c032440 con 0x7fb6ac15ba00 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6aaffd640 1 -- 172.21.15.148:0/3060471630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c02ae60 con 0x7fb6ac15ba00 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3060471630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15ba00 msgr2=0x7fb6ac09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3060471630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac09a930 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0097a0 tx=0x7fb69c02a110 comp rx=0 tx=0).stop 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3060471630 shutdown_connections 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3060471630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac09a930 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3060471630 >> 172.21.15.148:0/3060471630 conn(0x7fb6ac0b3270 msgr2=0x7fb6ac0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3060471630 shutdown_connections 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.909+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3060471630 wait complete. 2024-04-25T10:01:38.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.910+0000 7fb6b27b0640 1 Processor -- start 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.910+0000 7fb6b27b0640 1 -- start start 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b27b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac06c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b27b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 0x7fb6ac15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b27b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6ac1602e0 0x7fb6ac15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b27b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac15bde0 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b27b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac09ae70 con 0x7fb6ac15ba00 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b27b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac1f51d0 con 0x7fb6ac1602e0 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6ab7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 0x7fb6ac15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6ab7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 0x7fb6ac15ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34226/0 (socket says 172.21.15.148:34226) 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6ab7fe640 1 -- 172.21.15.148:0/3218767242 learned_addr learned my addr 172.21.15.148:0/3218767242 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:38.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.911+0000 7fb6b0d4d640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6ac1602e0 0x7fb6ac15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6ab7fe640 1 -- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6ac1602e0 msgr2=0x7fb6ac15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6ab7fe640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6ac1602e0 0x7fb6ac15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6ab7fe640 1 -- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6ac15ba00 msgr2=0x7fb6ac06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6ab7fe640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6ab7fe640 1 -- 172.21.15.148:0/3218767242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac06cf90 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6ab7fe640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 0x7fb6ac15ab10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c7b0 tx=0x7fb6a000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0018070 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00072d0 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00047e0 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac1e55d0 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.912+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac1e5b90 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.913+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac067be0 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.913+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6a001b070 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.914+0000 7fb6a8ff9640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb69806b4d0 0x7fb69806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.914+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6a0089630 con 0x7fb6ac15f5f0 2024-04-25T10:01:38.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.914+0000 7fb6abfff640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb69806b4d0 0x7fb69806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.914+0000 7fb6abfff640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb69806b4d0 0x7fb69806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb69c034000 tx=0x7fb69c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:38.918+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6ac067be0 con 0x7fb6ac15f5f0 2024-04-25T10:01:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.124+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6ac073990 con 0x7fb6ac15f5f0 2024-04-25T10:01:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.125+0000 7fb6a8ff9640 1 -- 172.21.15.148:0/3218767242 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fb6ac0cd710 con 0x7fb6ac15f5f0 2024-04-25T10:01:39.129 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:39.129 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:39.130 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:39.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb69806b4d0 msgr2=0x7fb69806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb69806b4d0 0x7fb69806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb69c034000 tx=0x7fb69c02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 msgr2=0x7fb6ac15ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 0x7fb6ac15ab10 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c7b0 tx=0x7fb6a000cc70 comp rx=0 tx=0).stop 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 shutdown_connections 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb69806b4d0 0x7fb69806d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6ac1602e0 0x7fb6ac15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6ac15f5f0 0x7fb6ac15ab10 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 --2- 172.21.15.148:0/3218767242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6ac15ba00 0x7fb6ac06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 >> 172.21.15.148:0/3218767242 conn(0x7fb6ac0b3270 msgr2=0x7fb6ac070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 shutdown_connections 2024-04-25T10:01:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.127+0000 7fb6b27b0640 1 -- 172.21.15.148:0/3218767242 wait complete. 2024-04-25T10:01:39.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:39.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.236+0000 7f5152015640 1 Processor -- start 2024-04-25T10:01:39.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.236+0000 7f5152015640 1 -- start start 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f5152015640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f514c0ce330 0x7f514c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f5152015640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c1626e0 con 0x7f514c0700a0 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f5152015640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c0cd270 con 0x7f514c06abe0 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f5152015640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c0cd550 con 0x7f514c0ce330 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f514affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f514c0ce330 0x7f514c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f514bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f514c0700a0 0x7f514c15d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46658/0 (socket says 172.21.15.148:46658) 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f514bfff640 1 -- 172.21.15.148:0/4112183099 learned_addr learned my addr 172.21.15.148:0/4112183099 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f514affd640 1 -- 172.21.15.148:0/4112183099 >> v1:172.21.15.188:6789/0 conn(0x7f514c06abe0 legacy=0x7f514c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f514affd640 1 -- 172.21.15.148:0/4112183099 >> v1:172.21.15.148:6789/0 conn(0x7f514c0700a0 legacy=0x7f514c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.237+0000 7f514affd640 1 -- 172.21.15.148:0/4112183099 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0cd830 con 0x7f514c0ce330 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.238+0000 7f514affd640 1 --2- 172.21.15.148:0/4112183099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ce330 0x7f514c06fb60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f51400097a0 tx=0x7f514002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f0d8a60e8803f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.238+0000 7f514a7fc640 1 -- 172.21.15.148:0/4112183099 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140037070 con 0x7f514c0ce330 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.238+0000 7f514a7fc640 1 -- 172.21.15.148:0/4112183099 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5140032440 con 0x7f514c0ce330 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.238+0000 7f514a7fc640 1 -- 172.21.15.148:0/4112183099 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140032810 con 0x7f514c0ce330 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 -- 172.21.15.148:0/4112183099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ce330 msgr2=0x7f514c06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 --2- 172.21.15.148:0/4112183099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ce330 0x7f514c06fb60 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f51400097a0 tx=0x7f514002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 -- 172.21.15.148:0/4112183099 shutdown_connections 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 --2- 172.21.15.148:0/4112183099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0ce330 0x7f514c06fb60 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 -- 172.21.15.148:0/4112183099 >> 172.21.15.148:0/4112183099 conn(0x7f514c0b2220 msgr2=0x7f514c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 -- 172.21.15.148:0/4112183099 shutdown_connections 2024-04-25T10:01:39.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.239+0000 7f5152015640 1 -- 172.21.15.148:0/4112183099 wait complete. 2024-04-25T10:01:39.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.240+0000 7f5152015640 1 Processor -- start 2024-04-25T10:01:39.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.240+0000 7f5152015640 1 -- start start 2024-04-25T10:01:39.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f5152015640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f514c06abe0 0x7f514c151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f5152015640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 0x7f514c0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f5152015640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f514c0ce330 0x7f514c0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f5152015640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c15ea90 con 0x7f514c0700a0 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f5152015640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c161780 con 0x7f514c06abe0 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f5152015640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c15ec10 con 0x7f514c0ce330 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 0x7f514c0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 0x7f514c0b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34236/0 (socket says 172.21.15.148:34236) 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 -- 172.21.15.148:0/1412769588 learned_addr learned my addr 172.21.15.148:0/1412769588 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514bfff640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f514c0ce330 0x7f514c0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514b7fe640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f514c06abe0 0x7f514c151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 -- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f514c0ce330 msgr2=0x7f514c0b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f514c0ce330 0x7f514c0b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 -- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f514c06abe0 msgr2=0x7f514c151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f514c06abe0 0x7f514c151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 -- 172.21.15.148:0/1412769588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0caf70 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.241+0000 7f514affd640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 0x7f514c0b0270 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f5140000c00 tx=0x7f51400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.242+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140037040 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.242+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5140042070 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.242+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0cb250 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.242+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140030bd0 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.242+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c0cb810 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.243+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c073990 con 0x7f514c0700a0 2024-04-25T10:01:39.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.243+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5140047460 con 0x7f514c0700a0 2024-04-25T10:01:39.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.243+0000 7f5133fff640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f513806b4d0 0x7f513806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.244+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51400aaf10 con 0x7f514c0700a0 2024-04-25T10:01:39.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.244+0000 7f514b7fe640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f513806b4d0 0x7f513806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.244+0000 7f514b7fe640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f513806b4d0 0x7f513806d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f514c0b0fa0 tx=0x7f513c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.248+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5140032e20 con 0x7f514c0700a0 2024-04-25T10:01:39.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.424+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514c0b2ff0 con 0x7f514c0700a0 2024-04-25T10:01:39.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.425+0000 7f5133fff640 1 -- 172.21.15.148:0/1412769588 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f51400795b0 con 0x7f514c0700a0 2024-04-25T10:01:39.430 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:39.430 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":109,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:09.636711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f513806b4d0 msgr2=0x7f513806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f513806b4d0 0x7f513806d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f514c0b0fa0 tx=0x7f513c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 msgr2=0x7f514c0b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 0x7f514c0b0270 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f5140000c00 tx=0x7f51400046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 shutdown_connections 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f513806b4d0 0x7f513806d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f514c0ce330 0x7f514c0b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f514c0700a0 0x7f514c0b0270 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 --2- 172.21.15.148:0/1412769588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f514c06abe0 0x7f514c151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 >> 172.21.15.148:0/1412769588 conn(0x7f514c0b2220 msgr2=0x7f514c06f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 shutdown_connections 2024-04-25T10:01:39.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.427+0000 7f5152015640 1 -- 172.21.15.148:0/1412769588 wait complete. 2024-04-25T10:01:39.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:06.175634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:06.926396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:39.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:39.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.538+0000 7fdcbf81f640 1 Processor -- start 2024-04-25T10:01:39.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.539+0000 7fdcbf81f640 1 -- start start 2024-04-25T10:01:39.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.539+0000 7fdcbf81f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb80cadb0 0x7fdcb80cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.539+0000 7fdcbf81f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcb80ccfe0 0x7fdcb80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.539+0000 7fdcbf81f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80afc80 con 0x7fdcb815de80 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.539+0000 7fdcbf81f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb80cb210 con 0x7fdcb80ccfe0 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.539+0000 7fdcbf81f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80ce4d0 con 0x7fdcb80cadb0 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbddbc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcb80ccfe0 0x7fdcb80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbd5bb640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fdcb815de80 0x7fdcb80cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:37270/0 (socket says 172.21.15.148:37270) 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbddbc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcb80ccfe0 0x7fdcb80cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:39748/0 (socket says 172.21.15.148:39748) 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbd5bb640 1 -- 172.21.15.148:0/1228972155 learned_addr learned my addr 172.21.15.148:0/1228972155 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbcdba640 1 --2- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb80cadb0 0x7fdcb80cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcaffff640 1 -- 172.21.15.148:0/1228972155 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604805898 0 0) 0x7fdcb80afc80 con 0x7fdcb815de80 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcaffff640 1 -- 172.21.15.148:0/1228972155 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca4003520 con 0x7fdcb815de80 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbddbc640 1 -- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb80cadb0 msgr2=0x7fdcb80cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:39.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbddbc640 1 --2- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb80cadb0 0x7fdcb80cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbddbc640 1 -- 172.21.15.148:0/1228972155 >> v1:172.21.15.188:6789/0 conn(0x7fdcb815de80 legacy=0x7fdcb80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.540+0000 7fdcbddbc640 1 -- 172.21.15.148:0/1228972155 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80ce7b0 con 0x7fdcb80ccfe0 2024-04-25T10:01:39.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.541+0000 7fdcbddbc640 1 --2- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80cd470 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14dc6480cf5f277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.541+0000 7fdcaffff640 1 -- 172.21.15.148:0/1228972155 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0007740 con 0x7fdcb80ccfe0 2024-04-25T10:01:39.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.541+0000 7fdcaffff640 1 -- 172.21.15.148:0/1228972155 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb00078e0 con 0x7fdcb80ccfe0 2024-04-25T10:01:39.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.541+0000 7fdcaffff640 1 -- 172.21.15.148:0/1228972155 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0007cb0 con 0x7fdcb80ccfe0 2024-04-25T10:01:39.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 -- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 msgr2=0x7fdcb80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80cd470 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb002a660 comp rx=0 tx=0).stop 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 -- 172.21.15.148:0/1228972155 shutdown_connections 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80cd470 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/1228972155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb80cadb0 0x7fdcb80cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 -- 172.21.15.148:0/1228972155 >> 172.21.15.148:0/1228972155 conn(0x7fdcb80b2390 msgr2=0x7fdcb80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 -- 172.21.15.148:0/1228972155 shutdown_connections 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.542+0000 7fdcbf81f640 1 -- 172.21.15.148:0/1228972155 wait complete. 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.543+0000 7fdcbf81f640 1 Processor -- start 2024-04-25T10:01:39.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.543+0000 7fdcbf81f640 1 -- start start 2024-04-25T10:01:39.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 0x7fdcb80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb815de80 0x7fdcb80d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb815e260 con 0x7fdcb80cadb0 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80cfc00 con 0x7fdcb80ccfe0 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80b2eb0 con 0x7fdcb815de80 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 0x7fdcb80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 0x7fdcb80b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34244/0 (socket says 172.21.15.148:34244) 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 -- 172.21.15.148:0/697137896 learned_addr learned my addr 172.21.15.148:0/697137896 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbcdba640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbddbc640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb815de80 0x7fdcb80d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 -- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb815de80 msgr2=0x7fdcb80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb815de80 0x7fdcb80d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 -- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 msgr2=0x7fdcb80b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 -- 172.21.15.148:0/697137896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80b39c0 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbddbc640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb815de80 0x7fdcb80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbd5bb640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 0x7fdcb80b3480 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c3c0 tx=0x7fdcb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4017070 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4005ce0 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb81471f0 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.544+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40072f0 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.545+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb80b5510 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.546+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80ae610 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.546+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcb401a070 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.546+0000 7fdcadffb640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.546+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdcb4089630 con 0x7fdcb80cadb0 2024-04-25T10:01:39.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.546+0000 7fdcbcdba640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.547+0000 7fdcbcdba640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8158c40 tx=0x7fdca8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.550+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcb80ae610 con 0x7fdcb80cadb0 2024-04-25T10:01:39.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.721+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcb8073a00 con 0x7fdcb80cadb0 2024-04-25T10:01:39.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.722+0000 7fdcadffb640 1 -- 172.21.15.148:0/697137896 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcb4057cd0 con 0x7fdcb80cadb0 2024-04-25T10:01:39.726 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:01:39.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.723+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 msgr2=0x7fdc9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.723+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8158c40 tx=0x7fdca8009190 comp rx=0 tx=0).stop 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 msgr2=0x7fdcb80b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 0x7fdcb80b3480 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c3c0 tx=0x7fdcb400c880 comp rx=0 tx=0).stop 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 shutdown_connections 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9c06b500 0x7fdc9c06d9c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcb815de80 0x7fdcb80d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcb80ccfe0 0x7fdcb80b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 --2- 172.21.15.148:0/697137896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcb80cadb0 0x7fdcb80b3480 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 >> 172.21.15.148:0/697137896 conn(0x7fdcb80b2390 msgr2=0x7fdcb806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 shutdown_connections 2024-04-25T10:01:39.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.724+0000 7fdcbf81f640 1 -- 172.21.15.148:0/697137896 wait complete. 2024-04-25T10:01:39.739 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:01:39.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.835+0000 7f44474b3640 1 Processor -- start 2024-04-25T10:01:39.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.835+0000 7f44474b3640 1 -- start start 2024-04-25T10:01:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f44474b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44400ced80 0x7f44400d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f44474b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4440161710 0x7f44400d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f44474b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444015e000 con 0x7f444015dc20 2024-04-25T10:01:39.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f44474b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44400d1de0 con 0x7f4440161710 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f44474b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4440161c60 con 0x7f44400ced80 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f4445a50640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4440161710 0x7f44400d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f444524f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f444015dc20 0x7f44400ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46662/0 (socket says 172.21.15.148:46662) 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.836+0000 7f444524f640 1 -- 172.21.15.148:0/3838399922 learned_addr learned my addr 172.21.15.148:0/3838399922 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.837+0000 7f4444a4e640 1 --2- 172.21.15.148:0/3838399922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44400ced80 0x7f44400d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.837+0000 7f4445a50640 1 -- 172.21.15.148:0/3838399922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44400ced80 msgr2=0x7f44400d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.837+0000 7f4445a50640 1 --2- 172.21.15.148:0/3838399922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44400ced80 0x7f44400d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.837+0000 7f4445a50640 1 -- 172.21.15.148:0/3838399922 >> v1:172.21.15.148:6789/0 conn(0x7f444015dc20 legacy=0x7f44400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.837+0000 7f4445a50640 1 -- 172.21.15.148:0/3838399922 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440161f40 con 0x7f4440161710 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.837+0000 7f4445a50640 1 --2- 172.21.15.148:0/3838399922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4440161710 0x7f44400d2820 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4438009d90 tx=0x7f443802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5101e8e3e4bb5799 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f4437fff640 1 -- 172.21.15.148:0/3838399922 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438007740 con 0x7f4440161710 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f4437fff640 1 -- 172.21.15.148:0/3838399922 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44380078e0 con 0x7f4440161710 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f4437fff640 1 -- 172.21.15.148:0/3838399922 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438007cb0 con 0x7f4440161710 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 -- 172.21.15.148:0/3838399922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4440161710 msgr2=0x7f44400d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 --2- 172.21.15.148:0/3838399922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4440161710 0x7f44400d2820 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4438009d90 tx=0x7f443802a660 comp rx=0 tx=0).stop 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 -- 172.21.15.148:0/3838399922 shutdown_connections 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 --2- 172.21.15.148:0/3838399922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4440161710 0x7f44400d2820 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 --2- 172.21.15.148:0/3838399922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44400ced80 0x7f44400d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 -- 172.21.15.148:0/3838399922 >> 172.21.15.148:0/3838399922 conn(0x7f44400d0420 msgr2=0x7f44400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 -- 172.21.15.148:0/3838399922 shutdown_connections 2024-04-25T10:01:39.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.838+0000 7f44474b3640 1 -- 172.21.15.148:0/3838399922 wait complete. 2024-04-25T10:01:39.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.839+0000 7f44474b3640 1 Processor -- start 2024-04-25T10:01:39.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 -- start start 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 0x7f444014fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f444015dc20 0x7f44400b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4440161710 0x7f44400b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44400b5e50 con 0x7f44400ced80 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4440162500 con 0x7f444015dc20 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f44474b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44400b5fd0 con 0x7f4440161710 2024-04-25T10:01:39.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f444524f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 0x7f444014fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f444524f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 0x7f444014fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34256/0 (socket says 172.21.15.148:34256) 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f444524f640 1 -- 172.21.15.148:0/3241378420 learned_addr learned my addr 172.21.15.148:0/3241378420 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f4445a50640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4440161710 0x7f44400b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.840+0000 7f4444a4e640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f444015dc20 0x7f44400b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f444524f640 1 -- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4440161710 msgr2=0x7f44400b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f444524f640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4440161710 0x7f44400b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f444524f640 1 -- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f444015dc20 msgr2=0x7f44400b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f444524f640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f444015dc20 0x7f44400b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f444524f640 1 -- 172.21.15.148:0/3241378420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44401e8f10 con 0x7f44400ced80 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f4445a50640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4440161710 0x7f44400b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f4444a4e640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f444015dc20 0x7f44400b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f444524f640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 0x7f444014fd70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f443000c3c0 tx=0x7f443000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430017070 con 0x7f44400ced80 2024-04-25T10:01:39.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4430005ce0 con 0x7f44400ced80 2024-04-25T10:01:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44401e91f0 con 0x7f44400ced80 2024-04-25T10:01:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.841+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44300072f0 con 0x7f44400ced80 2024-04-25T10:01:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.842+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44401e97b0 con 0x7f44400ced80 2024-04-25T10:01:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.843+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44400cf6c0 con 0x7f44400ced80 2024-04-25T10:01:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.843+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f443001a070 con 0x7f44400ced80 2024-04-25T10:01:39.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.843+0000 7f4435ffb640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f442406b4d0 0x7f442406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.843+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4430089770 con 0x7f44400ced80 2024-04-25T10:01:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.843+0000 7f4444a4e640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f442406b4d0 0x7f442406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.844+0000 7f4444a4e640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f442406b4d0 0x7f442406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f44400b5400 tx=0x7f443c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:39.847+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44400cf6c0 con 0x7f44400ced80 2024-04-25T10:01:40.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.015+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4440073a00 con 0x7f44400ced80 2024-04-25T10:01:40.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.016+0000 7f4435ffb640 1 -- 172.21.15.148:0/3241378420 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4430058460 con 0x7f44400ced80 2024-04-25T10:01:40.020 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:01:40.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f442406b4d0 msgr2=0x7f442406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f442406b4d0 0x7f442406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f44400b5400 tx=0x7f443c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 msgr2=0x7f444014fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 0x7f444014fd70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f443000c3c0 tx=0x7f443000c880 comp rx=0 tx=0).stop 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 shutdown_connections 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f442406b4d0 0x7f442406d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4440161710 0x7f44400b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f444015dc20 0x7f44400b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 --2- 172.21.15.148:0/3241378420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44400ced80 0x7f444014fd70 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 >> 172.21.15.148:0/3241378420 conn(0x7f44400d0420 msgr2=0x7f44400b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 shutdown_connections 2024-04-25T10:01:40.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.018+0000 7f44474b3640 1 -- 172.21.15.148:0/3241378420 wait complete. 2024-04-25T10:01:40.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:01:40.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.130+0000 7fc469ac4640 1 Processor -- start 2024-04-25T10:01:40.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.130+0000 7fc469ac4640 1 -- start start 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc469ac4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc4640ce900 0x7fc464166870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc469ac4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640ced00 con 0x7fc46406fbd0 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc469ac4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640b4d80 con 0x7fc464155f80 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc469ac4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4640b5390 con 0x7fc4640ce900 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc463fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc4640ce900 0x7fc464166870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc462ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc46406fbd0 0x7fc4640ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46668/0 (socket says 172.21.15.148:46668) 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc462ffd640 1 -- 172.21.15.148:0/3743031781 learned_addr learned my addr 172.21.15.148:0/3743031781 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc463fff640 1 -- 172.21.15.148:0/3743031781 >> v1:172.21.15.188:6789/0 conn(0x7fc464155f80 legacy=0x7fc46406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc463fff640 1 -- 172.21.15.148:0/3743031781 >> v1:172.21.15.148:6789/0 conn(0x7fc46406fbd0 legacy=0x7fc4640ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.131+0000 7fc463fff640 1 -- 172.21.15.148:0/3743031781 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640cef70 con 0x7fc4640ce900 2024-04-25T10:01:40.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.132+0000 7fc463fff640 1 --2- 172.21.15.148:0/3743031781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4640ce900 0x7fc464166870 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc4640b5510 tx=0x7fc45c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a34070f32bd635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.132+0000 7fc4627fc640 1 -- 172.21.15.148:0/3743031781 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c007740 con 0x7fc4640ce900 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.132+0000 7fc4627fc640 1 -- 172.21.15.148:0/3743031781 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c0078e0 con 0x7fc4640ce900 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.132+0000 7fc4627fc640 1 -- 172.21.15.148:0/3743031781 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c007cb0 con 0x7fc4640ce900 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 -- 172.21.15.148:0/3743031781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4640ce900 msgr2=0x7fc464166870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3743031781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4640ce900 0x7fc464166870 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc4640b5510 tx=0x7fc45c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 -- 172.21.15.148:0/3743031781 shutdown_connections 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3743031781 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4640ce900 0x7fc464166870 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 -- 172.21.15.148:0/3743031781 >> 172.21.15.148:0/3743031781 conn(0x7fc464061600 msgr2=0x7fc4640619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 -- 172.21.15.148:0/3743031781 shutdown_connections 2024-04-25T10:01:40.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.133+0000 7fc469ac4640 1 -- 172.21.15.148:0/3743031781 wait complete. 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.134+0000 7fc469ac4640 1 Processor -- start 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.134+0000 7fc469ac4640 1 -- start start 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 0x7fc4641f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4640ce900 0x7fc4641e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc464155f80 0x7fc46414a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464156360 con 0x7fc46406fbd0 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640d07f0 con 0x7fc4640ce900 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640cf250 con 0x7fc464155f80 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 0x7fc4641f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 0x7fc4641f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34278/0 (socket says 172.21.15.148:34278) 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 -- 172.21.15.148:0/3868670713 learned_addr learned my addr 172.21.15.148:0/3868670713 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc463fff640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc464155f80 0x7fc46414a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc462ffd640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4640ce900 0x7fc4641e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 -- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc464155f80 msgr2=0x7fc46414a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc464155f80 0x7fc46414a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 -- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4640ce900 msgr2=0x7fc4641e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4640ce900 0x7fc4641e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 -- 172.21.15.148:0/3868670713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46406c240 con 0x7fc46406fbd0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc4637fe640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 0x7fc4641f8070 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc45400b3f0 tx=0x7fc45400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454015020 con 0x7fc46406fbd0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454011830 con 0x7fc46406fbd0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46406c520 con 0x7fc46406fbd0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454011c20 con 0x7fc46406fbd0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.135+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46406cae0 con 0x7fc46406fbd0 2024-04-25T10:01:40.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.136+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc464073a00 con 0x7fc46406fbd0 2024-04-25T10:01:40.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.137+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc454018020 con 0x7fc46406fbd0 2024-04-25T10:01:40.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.137+0000 7fc44bfff640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.137+0000 7fc462ffd640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.137+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc45408a290 con 0x7fc46406fbd0 2024-04-25T10:01:40.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.138+0000 7fc462ffd640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc45006b4d0 0x7fc45006d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc4640b08c0 tx=0x7fc458009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.141+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc45401f020 con 0x7fc46406fbd0 2024-04-25T10:01:40.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.312+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4640b1030 con 0x7fc46406fbd0 2024-04-25T10:01:40.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.312+0000 7fc44bfff640 1 -- 172.21.15.148:0/3868670713 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4540172c0 con 0x7fc46406fbd0 2024-04-25T10:01:40.317 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:01:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc45006b4d0 msgr2=0x7fc45006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc45006b4d0 0x7fc45006d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc4640b08c0 tx=0x7fc458009040 comp rx=0 tx=0).stop 2024-04-25T10:01:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 msgr2=0x7fc4641f8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 0x7fc4641f8070 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc45400b3f0 tx=0x7fc45400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 shutdown_connections 2024-04-25T10:01:40.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc464155f80 0x7fc46414a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4640ce900 0x7fc4641e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 --2- 172.21.15.148:0/3868670713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc46406fbd0 0x7fc4641f8070 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 >> 172.21.15.148:0/3868670713 conn(0x7fc464061600 msgr2=0x7fc46406f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 shutdown_connections 2024-04-25T10:01:40.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.314+0000 7fc469ac4640 1 -- 172.21.15.148:0/3868670713 wait complete. 2024-04-25T10:01:40.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.423+0000 7fe4d013b640 1 Processor -- start 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.423+0000 7fe4d013b640 1 -- start start 2024-04-25T10:01:40.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4d013b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4c80c6df0 0x7fe4c80d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4d013b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4c80d1c00 0x7fe4c80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4d013b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c815dfe0 con 0x7fe4c815dc00 2024-04-25T10:01:40.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4d013b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80d2070 con 0x7fe4c80c6df0 2024-04-25T10:01:40.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4d013b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c80d00c0 con 0x7fe4c80d1c00 2024-04-25T10:01:40.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4c80c6df0 0x7fe4c80d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4c80c6df0 0x7fe4c80d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34280/0 (socket says 172.21.15.148:34280) 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 -- 172.21.15.148:0/812722526 learned_addr learned my addr 172.21.15.148:0/812722526 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4ce6d8640 1 --2- 172.21.15.148:0/812722526 >> v2:172.21.15.188:3300/0 conn(0x7fe4c80d1c00 0x7fe4c80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 -- 172.21.15.148:0/812722526 >> v2:172.21.15.188:3300/0 conn(0x7fe4c80d1c00 msgr2=0x7fe4c80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 --2- 172.21.15.148:0/812722526 >> v2:172.21.15.188:3300/0 conn(0x7fe4c80d1c00 0x7fe4c80ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 -- 172.21.15.148:0/812722526 >> v1:172.21.15.188:6789/0 conn(0x7fe4c815dc00 legacy=0x7fe4c80ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.424+0000 7fe4cd6d6640 1 -- 172.21.15.148:0/812722526 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80d03a0 con 0x7fe4c80c6df0 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4cd6d6640 1 --2- 172.21.15.148:0/812722526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80d16c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80b1040 tx=0x7fe4b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba969f598317a8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4cced5640 1 -- 172.21.15.148:0/812722526 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8037070 con 0x7fe4c80c6df0 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4cced5640 1 -- 172.21.15.148:0/812722526 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b8032440 con 0x7fe4c80c6df0 2024-04-25T10:01:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4cced5640 1 -- 172.21.15.148:0/812722526 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8032810 con 0x7fe4c80c6df0 2024-04-25T10:01:40.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4d013b640 1 -- 172.21.15.148:0/812722526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c80c6df0 msgr2=0x7fe4c80d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4d013b640 1 --2- 172.21.15.148:0/812722526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80d16c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fe4c80b1040 tx=0x7fe4b802a230 comp rx=0 tx=0).stop 2024-04-25T10:01:40.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4d013b640 1 -- 172.21.15.148:0/812722526 shutdown_connections 2024-04-25T10:01:40.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4d013b640 1 --2- 172.21.15.148:0/812722526 >> v2:172.21.15.188:3300/0 conn(0x7fe4c80d1c00 0x7fe4c80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4d013b640 1 --2- 172.21.15.148:0/812722526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80d16c0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.425+0000 7fe4d013b640 1 -- 172.21.15.148:0/812722526 >> 172.21.15.148:0/812722526 conn(0x7fe4c806f5d0 msgr2=0x7fe4c806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.426+0000 7fe4d013b640 1 -- 172.21.15.148:0/812722526 shutdown_connections 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.426+0000 7fe4d013b640 1 -- 172.21.15.148:0/812722526 wait complete. 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 Processor -- start 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 -- start start 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4c80d1c00 0x7fe4c81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 0x7fe4c815c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80cc130 con 0x7fe4c815dc00 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80cc720 con 0x7fe4c80c6df0 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4d013b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c80cfd90 con 0x7fe4c80d1c00 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4ce6d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 0x7fe4c815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4ce6d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 0x7fe4c815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34284/0 (socket says 172.21.15.148:34284) 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.427+0000 7fe4ce6d8640 1 -- 172.21.15.148:0/672040782 learned_addr learned my addr 172.21.15.148:0/672040782 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4cd6d6640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4c80d1c00 0x7fe4c81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4cded7640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4ce6d8640 1 -- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4c80d1c00 msgr2=0x7fe4c81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4ce6d8640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4c80d1c00 0x7fe4c81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4ce6d8640 1 -- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4c80c6df0 msgr2=0x7fe4c80b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4ce6d8640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4ce6d8640 1 -- 172.21.15.148:0/672040782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c80b5590 con 0x7fe4c815dc00 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4cded7640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4ce6d8640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 0x7fe4c815c240 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0007e70 tx=0x7fe4c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0014620 con 0x7fe4c815dc00 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0019070 con 0x7fe4c815dc00 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0014a20 con 0x7fe4c815dc00 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c815af80 con 0x7fe4c815dc00 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.428+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c815a9c0 con 0x7fe4c815dc00 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.429+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c8073990 con 0x7fe4c815dc00 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.429+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4c0014bc0 con 0x7fe4c815dc00 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.430+0000 7fe4be7fc640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4ac06b4d0 0x7fe4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.430+0000 7fe4cded7640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4ac06b4d0 0x7fe4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.430+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4c008ab70 con 0x7fe4c815dc00 2024-04-25T10:01:40.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.430+0000 7fe4cded7640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4ac06b4d0 0x7fe4ac06d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006fd0 tx=0x7fe4c4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.434+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4c80b5590 con 0x7fe4c815dc00 2024-04-25T10:01:40.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.640+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4c80c66c0 con 0x7fe4c815dc00 2024-04-25T10:01:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.640+0000 7fe4be7fc640 1 -- 172.21.15.148:0/672040782 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fe4c00598f0 con 0x7fe4c815dc00 2024-04-25T10:01:40.645 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:40.645 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:40.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4ac06b4d0 msgr2=0x7fe4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4ac06b4d0 0x7fe4ac06d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006fd0 tx=0x7fe4c4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:40.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 msgr2=0x7fe4c815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 0x7fe4c815c240 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0007e70 tx=0x7fe4c000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 shutdown_connections 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4ac06b4d0 0x7fe4ac06d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4c815dc00 0x7fe4c815c240 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4c80d1c00 0x7fe4c81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 --2- 172.21.15.148:0/672040782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4c80c6df0 0x7fe4c80b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 >> 172.21.15.148:0/672040782 conn(0x7fe4c806f5d0 msgr2=0x7fe4c80c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 shutdown_connections 2024-04-25T10:01:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.642+0000 7fe4d013b640 1 -- 172.21.15.148:0/672040782 wait complete. 2024-04-25T10:01:40.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:40.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.752+0000 7f3a66ebc640 1 Processor -- start 2024-04-25T10:01:40.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.752+0000 7f3a66ebc640 1 -- start start 2024-04-25T10:01:40.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a66ebc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3a600ce590 0x7f3a60165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a66ebc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a60071480 con 0x7f3a60154ee0 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a66ebc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a60071760 con 0x7f3a600cfec0 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a66ebc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a60072090 con 0x7f3a600ce590 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a64c58640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3a60154ee0 0x7f3a600cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46670/0 (socket says 172.21.15.148:46670) 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a64c58640 1 -- 172.21.15.148:0/3419348293 learned_addr learned my addr 172.21.15.148:0/3419348293 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a65459640 1 --2- 172.21.15.148:0/3419348293 >> v2:172.21.15.148:3300/0 conn(0x7f3a600ce590 0x7f3a60165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a65459640 1 -- 172.21.15.148:0/3419348293 >> v1:172.21.15.188:6789/0 conn(0x7f3a600cfec0 legacy=0x7f3a600cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a65459640 1 -- 172.21.15.148:0/3419348293 >> v1:172.21.15.148:6789/0 conn(0x7f3a60154ee0 legacy=0x7f3a600cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.753+0000 7f3a65459640 1 -- 172.21.15.148:0/3419348293 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60071a40 con 0x7f3a600ce590 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a65459640 1 --2- 172.21.15.148:0/3419348293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a600ce590 0x7f3a60165900 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009d90 tx=0x7f3a5802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd74b8bcbe6e7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a5f7fe640 1 -- 172.21.15.148:0/3419348293 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58007740 con 0x7f3a600ce590 2024-04-25T10:01:40.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a5f7fe640 1 -- 172.21.15.148:0/3419348293 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a580078e0 con 0x7f3a600ce590 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a5f7fe640 1 -- 172.21.15.148:0/3419348293 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58007cb0 con 0x7f3a600ce590 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a66ebc640 1 -- 172.21.15.148:0/3419348293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a600ce590 msgr2=0x7f3a60165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/3419348293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a600ce590 0x7f3a60165900 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009d90 tx=0x7f3a5802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a66ebc640 1 -- 172.21.15.148:0/3419348293 shutdown_connections 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/3419348293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a600ce590 0x7f3a60165900 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.754+0000 7f3a66ebc640 1 -- 172.21.15.148:0/3419348293 >> 172.21.15.148:0/3419348293 conn(0x7f3a6006abe0 msgr2=0x7f3a6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.755+0000 7f3a66ebc640 1 -- 172.21.15.148:0/3419348293 shutdown_connections 2024-04-25T10:01:40.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.755+0000 7f3a66ebc640 1 -- 172.21.15.148:0/3419348293 wait complete. 2024-04-25T10:01:40.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 Processor -- start 2024-04-25T10:01:40.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 -- start start 2024-04-25T10:01:40.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a600ce590 0x7f3a6014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a600cfec0 0x7f3a6014d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 0x7f3a601e3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601552c0 con 0x7f3a60154ee0 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60071230 con 0x7f3a600ce590 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.756+0000 7f3a66ebc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60155440 con 0x7f3a600cfec0 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 0x7f3a601e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 0x7f3a601e3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34300/0 (socket says 172.21.15.148:34300) 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 -- 172.21.15.148:0/1158818098 learned_addr learned my addr 172.21.15.148:0/1158818098 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a64c58640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a600ce590 0x7f3a6014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a5ffff640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a600cfec0 0x7f3a6014d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 -- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a600cfec0 msgr2=0x7f3a6014d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a600cfec0 0x7f3a6014d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 -- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a600ce590 msgr2=0x7f3a6014ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a600ce590 0x7f3a6014ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 -- 172.21.15.148:0/1158818098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6014dd60 con 0x7f3a60154ee0 2024-04-25T10:01:40.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a65459640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 0x7f3a601e3420 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009eb0 tx=0x7f3a58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58040070 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a58031c90 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6014e040 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58045680 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.757+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6014e600 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.758+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60073990 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.759+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a58039040 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.759+0000 7f3a5d7fa640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a4406b4d0 0x7f3a4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.759+0000 7f3a64c58640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a4406b4d0 0x7f3a4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.759+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a580abd80 con 0x7f3a60154ee0 2024-04-25T10:01:40.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.760+0000 7f3a64c58640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a4406b4d0 0x7f3a4406d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005880 tx=0x7f3a54009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.763+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a58043360 con 0x7f3a60154ee0 2024-04-25T10:01:40.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.936+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a6006d360 con 0x7f3a60154ee0 2024-04-25T10:01:40.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.937+0000 7f3a5d7fa640 1 -- 172.21.15.148:0/1158818098 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f3a5807ab30 con 0x7f3a60154ee0 2024-04-25T10:01:40.942 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:40.942 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":109,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:09.636711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a4406b4d0 msgr2=0x7f3a4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a4406b4d0 0x7f3a4406d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005880 tx=0x7f3a54009040 comp rx=0 tx=0).stop 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 msgr2=0x7f3a601e3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 0x7f3a601e3420 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009eb0 tx=0x7f3a58004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 shutdown_connections 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a4406b4d0 0x7f3a4406d990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a60154ee0 0x7f3a601e3420 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a600cfec0 0x7f3a6014d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 --2- 172.21.15.148:0/1158818098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a600ce590 0x7f3a6014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 >> 172.21.15.148:0/1158818098 conn(0x7f3a6006abe0 msgr2=0x7f3a600b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 shutdown_connections 2024-04-25T10:01:40.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:40.939+0000 7f3a66ebc640 1 -- 172.21.15.148:0/1158818098 wait complete. 2024-04-25T10:01:40.955 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:01:40.955 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:41.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.048+0000 7fef305f5640 1 Processor -- start 2024-04-25T10:01:41.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.048+0000 7fef305f5640 1 -- start start 2024-04-25T10:01:41.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.049+0000 7fef305f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef2815dc70 0x7fef280d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.049+0000 7fef305f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef2809a450 0x7fef280cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.049+0000 7fef305f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2806c700 0x7fef2806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.049+0000 7fef305f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef280d0c90 con 0x7fef2815dc70 2024-04-25T10:01:41.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.049+0000 7fef305f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2806c220 con 0x7fef2809a450 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.049+0000 7fef305f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280d0080 con 0x7fef2806c700 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef2815dc70 0x7fef280d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef2815dc70 0x7fef280d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34306/0 (socket says 172.21.15.148:34306) 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 -- 172.21.15.148:0/1893366890 learned_addr learned my addr 172.21.15.148:0/1893366890 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2eb92640 1 --2- 172.21.15.148:0/1893366890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2806c700 0x7fef2806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2db90640 1 --2- 172.21.15.148:0/1893366890 >> v2:172.21.15.188:3300/0 conn(0x7fef2809a450 0x7fef280cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 -- 172.21.15.148:0/1893366890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2806c700 msgr2=0x7fef2806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 --2- 172.21.15.148:0/1893366890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2806c700 0x7fef2806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 -- 172.21.15.148:0/1893366890 >> v2:172.21.15.188:3300/0 conn(0x7fef2809a450 msgr2=0x7fef280cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 --2- 172.21.15.148:0/1893366890 >> v2:172.21.15.188:3300/0 conn(0x7fef2809a450 0x7fef280cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.050+0000 7fef2e391640 1 -- 172.21.15.148:0/1893366890 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280d0360 con 0x7fef2815dc70 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef2e391640 1 --2- 172.21.15.148:0/1893366890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2815dc70 0x7fef280d1120 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fef280733b0 tx=0x7fef2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75e9b014aad0a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef2d38f640 1 -- 172.21.15.148:0/1893366890 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24037070 con 0x7fef2815dc70 2024-04-25T10:01:41.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef2d38f640 1 -- 172.21.15.148:0/1893366890 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef24032440 con 0x7fef2815dc70 2024-04-25T10:01:41.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef2d38f640 1 -- 172.21.15.148:0/1893366890 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24032810 con 0x7fef2815dc70 2024-04-25T10:01:41.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 -- 172.21.15.148:0/1893366890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2815dc70 msgr2=0x7fef280d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 --2- 172.21.15.148:0/1893366890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2815dc70 0x7fef280d1120 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fef280733b0 tx=0x7fef2402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 -- 172.21.15.148:0/1893366890 shutdown_connections 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 --2- 172.21.15.148:0/1893366890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2806c700 0x7fef2806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 --2- 172.21.15.148:0/1893366890 >> v2:172.21.15.188:3300/0 conn(0x7fef2809a450 0x7fef280cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 --2- 172.21.15.148:0/1893366890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2815dc70 0x7fef280d1120 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 -- 172.21.15.148:0/1893366890 >> 172.21.15.148:0/1893366890 conn(0x7fef280b3270 msgr2=0x7fef280b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.051+0000 7fef305f5640 1 -- 172.21.15.148:0/1893366890 shutdown_connections 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.052+0000 7fef305f5640 1 -- 172.21.15.148:0/1893366890 wait complete. 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 Processor -- start 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 -- start start 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 0x7fef28157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2809a450 0x7fef2815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2815dc70 0x7fef2815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2815e050 con 0x7fef2806c700 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280d0920 con 0x7fef2815dc70 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef305f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280cbdd0 con 0x7fef2809a450 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef2e391640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 0x7fef28157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef2e391640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 0x7fef28157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34314/0 (socket says 172.21.15.148:34314) 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.053+0000 7fef2e391640 1 -- 172.21.15.148:0/421695066 learned_addr learned my addr 172.21.15.148:0/421695066 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2db90640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2809a450 0x7fef2815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2eb92640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2815dc70 0x7fef2815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2e391640 1 -- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2809a450 msgr2=0x7fef2815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2e391640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2809a450 0x7fef2815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2e391640 1 -- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2815dc70 msgr2=0x7fef2815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2e391640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2815dc70 0x7fef2815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2e391640 1 -- 172.21.15.148:0/421695066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280c79f0 con 0x7fef2806c700 2024-04-25T10:01:41.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef2e391640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 0x7fef28157830 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fef240097c0 tx=0x7fef240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24041020 con 0x7fef2806c700 2024-04-25T10:01:41.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef240306c0 con 0x7fef2806c700 2024-04-25T10:01:41.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef280c7cd0 con 0x7fef2806c700 2024-04-25T10:01:41.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef24037040 con 0x7fef2806c700 2024-04-25T10:01:41.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.054+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef280c9b50 con 0x7fef2806c700 2024-04-25T10:01:41.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.055+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef28073990 con 0x7fef2806c700 2024-04-25T10:01:41.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.056+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef24044020 con 0x7fef2806c700 2024-04-25T10:01:41.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.056+0000 7fef1effd640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef0c06b4d0 0x7fef0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.056+0000 7fef2db90640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef0c06b4d0 0x7fef0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.056+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fef240aaab0 con 0x7fef2806c700 2024-04-25T10:01:41.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.057+0000 7fef2db90640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef0c06b4d0 0x7fef0c06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fef281f7240 tx=0x7fef18008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.060+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef2404b030 con 0x7fef2806c700 2024-04-25T10:01:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.268+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef280d1120 con 0x7fef2806c700 2024-04-25T10:01:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.268+0000 7fef1effd640 1 -- 172.21.15.148:0/421695066 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fef24079830 con 0x7fef2806c700 2024-04-25T10:01:41.273 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:41.273 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef0c06b4d0 msgr2=0x7fef0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef0c06b4d0 0x7fef0c06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fef281f7240 tx=0x7fef18008040 comp rx=0 tx=0).stop 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 msgr2=0x7fef28157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 0x7fef28157830 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fef240097c0 tx=0x7fef240046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 shutdown_connections 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef0c06b4d0 0x7fef0c06d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2815dc70 0x7fef2815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2809a450 0x7fef2815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 --2- 172.21.15.148:0/421695066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2806c700 0x7fef28157830 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.270+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 >> 172.21.15.148:0/421695066 conn(0x7fef280b3270 msgr2=0x7fef280ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.271+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 shutdown_connections 2024-04-25T10:01:41.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.271+0000 7fef305f5640 1 -- 172.21.15.148:0/421695066 wait complete. 2024-04-25T10:01:41.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:41.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.380+0000 7f3837c6b640 1 Processor -- start 2024-04-25T10:01:41.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.381+0000 7f3837c6b640 1 -- start start 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.381+0000 7f3837c6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015db60 0x7f38300ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.381+0000 7f3837c6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38300b5f00 0x7f38300d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.381+0000 7f3837c6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38300cec00 con 0x7f383015df40 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.381+0000 7f3837c6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38300ce930 con 0x7f38300b5f00 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.381+0000 7f3837c6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38300b6450 con 0x7f383015db60 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38300b5f00 0x7f38300d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38300b5f00 0x7f38300d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34316/0 (socket says 172.21.15.148:34316) 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 -- 172.21.15.148:0/3108097735 learned_addr learned my addr 172.21.15.148:0/3108097735 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835a07640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015db60 0x7f38300ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 -- 172.21.15.148:0/3108097735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015db60 msgr2=0x7f38300ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015db60 0x7f38300ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 -- 172.21.15.148:0/3108097735 >> v1:172.21.15.188:6789/0 conn(0x7f383015df40 legacy=0x7f3830161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 -- 172.21.15.148:0/3108097735 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300ceee0 con 0x7f38300b5f00 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835a07640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015db60 0x7f38300ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.382+0000 7f3835206640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f38300d0400 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f38200098a0 tx=0x7f382002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d2dbb31d632830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3834a05640 1 -- 172.21.15.148:0/3108097735 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820037070 con 0x7f38300b5f00 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3834a05640 1 -- 172.21.15.148:0/3108097735 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3820032440 con 0x7f38300b5f00 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3834a05640 1 -- 172.21.15.148:0/3108097735 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820032810 con 0x7f38300b5f00 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 -- 172.21.15.148:0/3108097735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 msgr2=0x7f38300d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f38300d0400 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f38200098a0 tx=0x7f382002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 -- 172.21.15.148:0/3108097735 shutdown_connections 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f38300d0400 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 --2- 172.21.15.148:0/3108097735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015db60 0x7f38300ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 -- 172.21.15.148:0/3108097735 >> 172.21.15.148:0/3108097735 conn(0x7f3830061600 msgr2=0x7f3830061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 -- 172.21.15.148:0/3108097735 shutdown_connections 2024-04-25T10:01:41.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.383+0000 7f3837c6b640 1 -- 172.21.15.148:0/3108097735 wait complete. 2024-04-25T10:01:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.384+0000 7f3837c6b640 1 Processor -- start 2024-04-25T10:01:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 -- start start 2024-04-25T10:01:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f383014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f383015db60 0x7f38300ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015df40 0x7f38300d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38300d3280 con 0x7f38300b5f00 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38300d1d50 con 0x7f383015db60 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3837c6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38300d3400 con 0x7f383015df40 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3835a07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f383014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3835a07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f383014efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34332/0 (socket says 172.21.15.148:34332) 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3835a07640 1 -- 172.21.15.148:0/4058429471 learned_addr learned my addr 172.21.15.148:0/4058429471 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3836208640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015df40 0x7f38300d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.385+0000 7f3835206640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f383015db60 0x7f38300ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835a07640 1 -- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015df40 msgr2=0x7f38300d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835a07640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015df40 0x7f38300d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835a07640 1 -- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f383015db60 msgr2=0x7f38300ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835a07640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f383015db60 0x7f38300ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835a07640 1 -- 172.21.15.148:0/4058429471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300c79f0 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3836208640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015df40 0x7f38300d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835206640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f383015db60 0x7f38300ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3835a07640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f383014efc0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f382c00cf60 tx=0x7f382c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c01a070 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c00d840 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38300c7cd0 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00dc10 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.386+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38300c8290 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.387+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3830073270 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.387+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f382c01d070 con 0x7f38300b5f00 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.388+0000 7f38267fc640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f381406b4d0 0x7f381406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.388+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f382c08aaf0 con 0x7f38300b5f00 2024-04-25T10:01:41.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.388+0000 7f3835206640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f381406b4d0 0x7f381406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.388+0000 7f3835206640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f381406b4d0 0x7f381406d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3820034000 tx=0x7f382002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.392+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f382c059190 con 0x7f38300b5f00 2024-04-25T10:01:41.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.597+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3830073990 con 0x7f38300b5f00 2024-04-25T10:01:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.597+0000 7f38267fc640 1 -- 172.21.15.148:0/4058429471 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f382c05d230 con 0x7f38300b5f00 2024-04-25T10:01:41.602 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:41.602 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":60,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:07.958159+0000","modified":"2024-04-25T10:01:10.095327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"d","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6834/3455709247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3455709247},{"type":"v1","addr":"172.21.15.188:6834","nonce":3455709247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:01:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 60 2024-04-25T10:01:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.599+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f381406b4d0 msgr2=0x7f381406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.599+0000 7f3837c6b640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f381406b4d0 0x7f381406d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3820034000 tx=0x7f382002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.599+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 msgr2=0x7f383014efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.599+0000 7f3837c6b640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f383014efc0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f382c00cf60 tx=0x7f382c00cf90 comp rx=0 tx=0).stop 2024-04-25T10:01:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 shutdown_connections 2024-04-25T10:01:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f381406b4d0 0x7f381406d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f383015df40 0x7f38300d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f383015db60 0x7f38300ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 --2- 172.21.15.148:0/4058429471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38300b5f00 0x7f383014efc0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 >> 172.21.15.148:0/4058429471 conn(0x7f3830061600 msgr2=0x7f38300b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 shutdown_connections 2024-04-25T10:01:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.600+0000 7f3837c6b640 1 -- 172.21.15.148:0/4058429471 wait complete. 2024-04-25T10:01:41.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:41.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.709+0000 7f5698377640 1 Processor -- start 2024-04-25T10:01:41.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.709+0000 7f5698377640 1 -- start start 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5698377640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f569015da70 0x7f56900d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5698377640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f569006c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5698377640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56900c8030 con 0x7f56900c7730 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5698377640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56900c8310 con 0x7f569015da70 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5698377640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c85f0 con 0x7f56900ce8d0 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695912640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f56900c7730 0x7f56900ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46686/0 (socket says 172.21.15.148:46686) 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695912640 1 -- 172.21.15.148:0/388648299 learned_addr learned my addr 172.21.15.148:0/388648299 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166389354 0 0) 0x7f56900c8030 con 0x7f56900c7730 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f567c003520 con 0x7f56900c7730 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5696914640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f569006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5696113640 1 --2- 172.21.15.148:0/388648299 >> v2:172.21.15.188:3300/0 conn(0x7f569015da70 0x7f56900d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3479907478 0 0) 0x7f567c003520 con 0x7f56900c7730 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 msgr2=0x7f569006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f569006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 >> v2:172.21.15.188:3300/0 conn(0x7f569015da70 msgr2=0x7f56900d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 --2- 172.21.15.148:0/388648299 >> v2:172.21.15.188:3300/0 conn(0x7f569015da70 0x7f56900d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569006bb30 con 0x7f56900c7730 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5680001670 con 0x7f56900c7730 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 >> v1:172.21.15.148:6789/0 conn(0x7f56900c7730 legacy=0x7f56900ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0039b0 0x7f567c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f567c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c8310 con 0x7f567c0039b0 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c85f0 con 0x7f567c004800 2024-04-25T10:01:41.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.710+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c00baa0 con 0x7f56900ce8d0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0039b0 0x7f567c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5695912640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696914640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f567c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 -- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 msgr2=0x7f567c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f567c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 -- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c004800 msgr2=0x7f567c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 -- 172.21.15.148:0/388648299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c003520 con 0x7f567c0039b0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5695912640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696914640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56900ce8d0 0x7f567c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.711+0000 7f5696113640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0039b0 0x7f567c003dd0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f568c00ba00 tx=0x7f567c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79b16276f2e6e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c030070 con 0x7f567c0039b0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c02adb0 con 0x7f567c0039b0 2024-04-25T10:01:41.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5695111640 1 -- 172.21.15.148:0/388648299 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c00b600 con 0x7f567c0039b0 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5698377640 1 -- 172.21.15.148:0/388648299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0039b0 msgr2=0x7f567c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5698377640 1 --2- 172.21.15.148:0/388648299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567c0039b0 0x7f567c003dd0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f568c00ba00 tx=0x7f567c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5696914640 1 -- 172.21.15.148:0/388648299 reap_dead start 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5698377640 1 -- 172.21.15.148:0/388648299 shutdown_connections 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5698377640 1 -- 172.21.15.148:0/388648299 >> 172.21.15.148:0/388648299 conn(0x7f56900b33e0 msgr2=0x7f56900b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5698377640 1 -- 172.21.15.148:0/388648299 shutdown_connections 2024-04-25T10:01:41.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.712+0000 7f5698377640 1 -- 172.21.15.148:0/388648299 wait complete. 2024-04-25T10:01:41.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.713+0000 7f5698377640 1 Processor -- start 2024-04-25T10:01:41.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.713+0000 7f5698377640 1 -- start start 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5698377640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c0039b0 0x7f5690149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5698377640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f569015c1d0 0x7f569015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5698377640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 0x7f56900b6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5698377640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c7bc0 con 0x7f56900b60e0 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5698377640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569015de50 con 0x7f567c0039b0 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5698377640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c7d40 con 0x7f569015c1d0 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 0x7f56900b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 0x7f56900b6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34348/0 (socket says 172.21.15.148:34348) 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 -- 172.21.15.148:0/925224630 learned_addr learned my addr 172.21.15.148:0/925224630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 -- 172.21.15.148:0/925224630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f569015c1d0 msgr2=0x7f569015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f569015c1d0 0x7f569015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 -- 172.21.15.148:0/925224630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c0039b0 msgr2=0x7f5690149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c0039b0 0x7f5690149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.714+0000 7f5696914640 1 -- 172.21.15.148:0/925224630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690149b80 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.715+0000 7f5696914640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 0x7f56900b6560 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f569006b8b0 tx=0x7f5688004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.715+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688007c50 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.715+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5688019070 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.715+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56900b7130 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.715+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688014650 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.715+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5690152d30 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.716+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5690073990 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.716+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56880147f0 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.717+0000 7f5686ffd640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b4d0 0x7f567406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.717+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f568808a960 con 0x7f56900b60e0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.717+0000 7f5696113640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b4d0 0x7f567406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.717+0000 7f5696113640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b4d0 0x7f567406d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f567c0057e0 tx=0x7f568c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.721+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5688059650 con 0x7f56900b60e0 2024-04-25T10:01:41.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.894+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f569006d230 con 0x7f56900b60e0 2024-04-25T10:01:41.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.895+0000 7f5686ffd640 1 -- 172.21.15.148:0/925224630 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f5688058f50 con 0x7f56900b60e0 2024-04-25T10:01:41.900 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:41.900 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":109,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:09.636711+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:06.175634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:06.926396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:41.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b4d0 msgr2=0x7f567406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b4d0 0x7f567406d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f567c0057e0 tx=0x7f568c02c000 comp rx=0 tx=0).stop 2024-04-25T10:01:41.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 msgr2=0x7f56900b6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 0x7f56900b6560 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f569006b8b0 tx=0x7f5688004840 comp rx=0 tx=0).stop 2024-04-25T10:01:41.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 shutdown_connections 2024-04-25T10:01:41.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b4d0 0x7f567406d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56900b60e0 0x7f56900b6560 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f569015c1d0 0x7f569015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 --2- 172.21.15.148:0/925224630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f567c0039b0 0x7f5690149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 >> 172.21.15.148:0/925224630 conn(0x7f56900b33e0 msgr2=0x7f56900cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 shutdown_connections 2024-04-25T10:01:41.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:41.897+0000 7f5698377640 1 -- 172.21.15.148:0/925224630 wait complete. 2024-04-25T10:01:41.914 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:01:42.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.007+0000 7fe94b35c640 1 Processor -- start 2024-04-25T10:01:42.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.007+0000 7fe94b35c640 1 -- start start 2024-04-25T10:01:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe94b35c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cf600 0x7fe9440cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe94b35c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9440cbf20 0x7fe9440cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe94b35c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9440c75e0 con 0x7fe94415bb50 2024-04-25T10:01:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe94b35c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9440c8680 con 0x7fe9440cbf20 2024-04-25T10:01:42.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe94b35c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440cff80 con 0x7fe9440cf600 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe9498f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9440cbf20 0x7fe9440cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe9498f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe9440cbf20 0x7fe9440cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34364/0 (socket says 172.21.15.148:34364) 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.008+0000 7fe9498f9640 1 -- 172.21.15.148:0/1283498224 learned_addr learned my addr 172.21.15.148:0/1283498224 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.009+0000 7fe9488f7640 1 --2- 172.21.15.148:0/1283498224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cf600 0x7fe9440cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.009+0000 7fe9498f9640 1 -- 172.21.15.148:0/1283498224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cf600 msgr2=0x7fe9440cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.009+0000 7fe9498f9640 1 --2- 172.21.15.148:0/1283498224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cf600 0x7fe9440cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.009+0000 7fe9498f9640 1 -- 172.21.15.148:0/1283498224 >> v1:172.21.15.188:6789/0 conn(0x7fe94415bb50 legacy=0x7fe9440c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.009+0000 7fe9498f9640 1 -- 172.21.15.148:0/1283498224 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440c78c0 con 0x7fe9440cbf20 2024-04-25T10:01:42.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.009+0000 7fe9498f9640 1 --2- 172.21.15.148:0/1283498224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9440cbf20 0x7fe9440cc3b0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe934009870 tx=0x7fe93402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29c89ee3bd8b570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe943fff640 1 -- 172.21.15.148:0/1283498224 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934037070 con 0x7fe9440cbf20 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe943fff640 1 -- 172.21.15.148:0/1283498224 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93402ae50 con 0x7fe9440cbf20 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe943fff640 1 -- 172.21.15.148:0/1283498224 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934032860 con 0x7fe9440cbf20 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 -- 172.21.15.148:0/1283498224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9440cbf20 msgr2=0x7fe9440cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 --2- 172.21.15.148:0/1283498224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9440cbf20 0x7fe9440cc3b0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe934009870 tx=0x7fe93402a300 comp rx=0 tx=0).stop 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 -- 172.21.15.148:0/1283498224 shutdown_connections 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 --2- 172.21.15.148:0/1283498224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9440cbf20 0x7fe9440cc3b0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 --2- 172.21.15.148:0/1283498224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cf600 0x7fe9440cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 -- 172.21.15.148:0/1283498224 >> 172.21.15.148:0/1283498224 conn(0x7fe94406f500 msgr2=0x7fe94406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 -- 172.21.15.148:0/1283498224 shutdown_connections 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.010+0000 7fe94b35c640 1 -- 172.21.15.148:0/1283498224 wait complete. 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.011+0000 7fe94b35c640 1 Processor -- start 2024-04-25T10:01:42.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.011+0000 7fe94b35c640 1 -- start start 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe94b35c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cbf20 0x7fe9440ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe94b35c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9440cf600 0x7fe9440cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe94b35c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 0x7fe9441586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe94b35c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440c8160 con 0x7fe94415bb50 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe94b35c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440c7e80 con 0x7fe9440cf600 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe94b35c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440c72a0 con 0x7fe9440cbf20 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe9498f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 0x7fe9441586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe9498f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 0x7fe9441586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34378/0 (socket says 172.21.15.148:34378) 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe9498f9640 1 -- 172.21.15.148:0/3277531877 learned_addr learned my addr 172.21.15.148:0/3277531877 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:42.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe9490f8640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cbf20 0x7fe9440ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe9488f7640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9440cf600 0x7fe9440cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.012+0000 7fe9498f9640 1 -- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cbf20 msgr2=0x7fe9440ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe9498f9640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cbf20 0x7fe9440ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe9498f9640 1 -- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9440cf600 msgr2=0x7fe9440cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe9498f9640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9440cf600 0x7fe9440cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe9498f9640 1 -- 172.21.15.148:0/3277531877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440b50d0 con 0x7fe94415bb50 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe9498f9640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 0x7fe9441586e0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe93402a340 tx=0x7fe9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934042070 con 0x7fe94415bb50 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93403e660 con 0x7fe94415bb50 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9440b53b0 con 0x7fe94415bb50 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934037040 con 0x7fe94415bb50 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.013+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9440b5970 con 0x7fe94415bb50 2024-04-25T10:01:42.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.014+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe944158500 con 0x7fe94415bb50 2024-04-25T10:01:42.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.014+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe934045070 con 0x7fe94415bb50 2024-04-25T10:01:42.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.015+0000 7fe941ffb640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.015+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9340aaa50 con 0x7fe94415bb50 2024-04-25T10:01:42.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.015+0000 7fe9490f8640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.015+0000 7fe9490f8640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe9441f4d00 tx=0x7fe93c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.019+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe944158500 con 0x7fe94415bb50 2024-04-25T10:01:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.225+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe944157400 con 0x7fe94415bb50 2024-04-25T10:01:42.230 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:01:42.226+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:01:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.227+0000 7fe941ffb640 1 -- 172.21.15.148:0/3277531877 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe9340792a0 con 0x7fe94415bb50 2024-04-25T10:01:42.231 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 msgr2=0x7fe93006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe9441f4d00 tx=0x7fe93c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 msgr2=0x7fe9441586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 0x7fe9441586e0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe93402a340 tx=0x7fe9340046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 shutdown_connections 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94415bb50 0x7fe9441586e0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9440cf600 0x7fe9440cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 --2- 172.21.15.148:0/3277531877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9440cbf20 0x7fe9440ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 >> 172.21.15.148:0/3277531877 conn(0x7fe94406f500 msgr2=0x7fe9440ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 shutdown_connections 2024-04-25T10:01:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.229+0000 7fe94b35c640 1 -- 172.21.15.148:0/3277531877 wait complete. 2024-04-25T10:01:42.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:01:42.275 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.340+0000 7f09f7cee640 1 Processor -- start 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.341+0000 7f09f7cee640 1 -- start start 2024-04-25T10:01:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.341+0000 7f09f7cee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f015de80 0x7f09f00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.341+0000 7f09f7cee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09f00cefb0 0x7f09f00cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.341+0000 7f09f7cee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f00afc80 con 0x7f09f00cf950 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.341+0000 7f09f7cee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f00ce4d0 con 0x7f09f00cefb0 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.341+0000 7f09f7cee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00ce7b0 con 0x7f09f015de80 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f628b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f09f00cf950 0x7f09f006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:46688/0 (socket says 172.21.15.148:46688) 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f628b640 1 -- 172.21.15.148:0/1224123879 learned_addr learned my addr 172.21.15.148:0/1224123879 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5a8a640 1 --2- 172.21.15.148:0/1224123879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f015de80 0x7f09f00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5289640 1 --2- 172.21.15.148:0/1224123879 >> v2:172.21.15.188:3300/0 conn(0x7f09f00cefb0 0x7f09f00cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f4a88640 1 -- 172.21.15.148:0/1224123879 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824059680 0 0) 0x7f09f00afc80 con 0x7f09f00cf950 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f4a88640 1 -- 172.21.15.148:0/1224123879 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09dc003520 con 0x7f09f00cf950 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5a8a640 1 -- 172.21.15.148:0/1224123879 >> v2:172.21.15.188:3300/0 conn(0x7f09f00cefb0 msgr2=0x7f09f00cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5a8a640 1 --2- 172.21.15.148:0/1224123879 >> v2:172.21.15.188:3300/0 conn(0x7f09f00cefb0 0x7f09f00cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5a8a640 1 -- 172.21.15.148:0/1224123879 >> v1:172.21.15.148:6789/0 conn(0x7f09f00cf950 legacy=0x7f09f006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5a8a640 1 -- 172.21.15.148:0/1224123879 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00cea90 con 0x7f09f015de80 2024-04-25T10:01:42.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.342+0000 7f09f5289640 1 --2- 172.21.15.148:0/1224123879 >> v2:172.21.15.188:3300/0 conn(0x7f09f00cefb0 0x7f09f00cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.343+0000 7f09f5a8a640 1 --2- 172.21.15.148:0/1224123879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f015de80 0x7f09f00cadb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0097a0 tx=0x7f09ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b927ab06d212d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.343+0000 7f09f4a88640 1 -- 172.21.15.148:0/1224123879 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec037070 con 0x7f09f015de80 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.343+0000 7f09f4a88640 1 -- 172.21.15.148:0/1224123879 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec032440 con 0x7f09f015de80 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.343+0000 7f09f4a88640 1 -- 172.21.15.148:0/1224123879 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec032810 con 0x7f09f015de80 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.343+0000 7f09f7cee640 1 -- 172.21.15.148:0/1224123879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f015de80 msgr2=0x7f09f00cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.343+0000 7f09f7cee640 1 --2- 172.21.15.148:0/1224123879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f015de80 0x7f09f00cadb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0097a0 tx=0x7f09ec02a180 comp rx=0 tx=0).stop 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.344+0000 7f09f7cee640 1 -- 172.21.15.148:0/1224123879 shutdown_connections 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.344+0000 7f09f7cee640 1 --2- 172.21.15.148:0/1224123879 >> v2:172.21.15.188:3300/0 conn(0x7f09f00cefb0 0x7f09f00cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.344+0000 7f09f7cee640 1 --2- 172.21.15.148:0/1224123879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f015de80 0x7f09f00cadb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.344+0000 7f09f7cee640 1 -- 172.21.15.148:0/1224123879 >> 172.21.15.148:0/1224123879 conn(0x7f09f00b2390 msgr2=0x7f09f00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.344+0000 7f09f7cee640 1 -- 172.21.15.148:0/1224123879 shutdown_connections 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.344+0000 7f09f7cee640 1 -- 172.21.15.148:0/1224123879 wait complete. 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 Processor -- start 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 -- start start 2024-04-25T10:01:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 0x7f09f01506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f00cf950 0x7f09f00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f015de80 0x7f09f00cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00b3030 con 0x7f09f00cefb0 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00b2eb0 con 0x7f09f015de80 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.345+0000 7f09f7cee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00c5600 con 0x7f09f00cf950 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 0x7f09f01506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 0x7f09f01506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34386/0 (socket says 172.21.15.148:34386) 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 -- 172.21.15.148:0/4160590951 learned_addr learned my addr 172.21.15.148:0/4160590951 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f628b640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f015de80 0x7f09f00cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5289640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f00cf950 0x7f09f00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 -- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f00cf950 msgr2=0x7f09f00adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f00cf950 0x7f09f00adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 -- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f015de80 msgr2=0x7f09f00cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f015de80 0x7f09f00cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 -- 172.21.15.148:0/4160590951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f0150bf0 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f5a8a640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 0x7f09f01506b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec041020 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec03d640 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f00aeab0 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec037040 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.346+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f00c8930 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.347+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f0073a00 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.348+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09ec044020 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.348+0000 7f09e67fc640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09d406b4d0 0x7f09d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.348+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f09ec0aab50 con 0x7f09f00cefb0 2024-04-25T10:01:42.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.348+0000 7f09f5289640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09d406b4d0 0x7f09d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.349+0000 7f09f5289640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09d406b4d0 0x7f09d406d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009560 tx=0x7f09e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.352+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09ec04b030 con 0x7f09f00cefb0 2024-04-25T10:01:42.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:42.557+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09f00c8710 con 0x7f09f00cefb0 2024-04-25T10:01:43.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.228+0000 7f09e67fc640 1 -- 172.21.15.148:0/4160590951 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09f0150bf0 con 0x7f09f00cefb0 2024-04-25T10:01:43.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09d406b4d0 msgr2=0x7f09d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09d406b4d0 0x7f09d406d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009560 tx=0x7f09e0008040 comp rx=0 tx=0).stop 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 msgr2=0x7f09f01506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 0x7f09f01506b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 shutdown_connections 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09d406b4d0 0x7f09d406d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f015de80 0x7f09f00cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f00cf950 0x7f09f00adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 --2- 172.21.15.148:0/4160590951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09f00cefb0 0x7f09f01506b0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 >> 172.21.15.148:0/4160590951 conn(0x7f09f00b2390 msgr2=0x7f09f00b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 shutdown_connections 2024-04-25T10:01:43.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.231+0000 7f09f7cee640 1 -- 172.21.15.148:0/4160590951 wait complete. 2024-04-25T10:01:43.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:01:43.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.349+0000 7fe5765b0640 1 Processor -- start 2024-04-25T10:01:43.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.349+0000 7fe5765b0640 1 -- start start 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe5765b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe570162140 0x7fe5700cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe5765b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe5765b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57015e3a0 con 0x7fe5700d0a00 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe5765b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5701625c0 con 0x7fe570162140 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe5765b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700cd9d0 con 0x7fe5700cd540 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe56f7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe570162140 0x7fe5700cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.350+0000 7fe574b4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe56ffff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe5700d0a00 0x7fe5700d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:37316/0 (socket says 172.21.15.148:37316) 2024-04-25T10:01:43.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe56ffff640 1 -- 172.21.15.148:0/1646464517 learned_addr learned my addr 172.21.15.148:0/1646464517 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe574b4d640 1 -- 172.21.15.148:0/1646464517 >> v2:172.21.15.188:3300/0 conn(0x7fe570162140 msgr2=0x7fe5700cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe574b4d640 1 --2- 172.21.15.148:0/1646464517 >> v2:172.21.15.188:3300/0 conn(0x7fe570162140 0x7fe5700cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe574b4d640 1 -- 172.21.15.148:0/1646464517 >> v1:172.21.15.188:6789/0 conn(0x7fe5700d0a00 legacy=0x7fe5700d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe574b4d640 1 -- 172.21.15.148:0/1646464517 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700cdc70 con 0x7fe5700cd540 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.351+0000 7fe56f7fe640 1 --2- 172.21.15.148:0/1646464517 >> v2:172.21.15.188:3300/0 conn(0x7fe570162140 0x7fe5700cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe574b4d640 1 --2- 172.21.15.148:0/1646464517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700caad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe568009d90 tx=0x7fe56802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f57c46b98e38407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe56effd640 1 -- 172.21.15.148:0/1646464517 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568031600 con 0x7fe5700cd540 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe56effd640 1 -- 172.21.15.148:0/1646464517 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5680317a0 con 0x7fe5700cd540 2024-04-25T10:01:43.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe56effd640 1 -- 172.21.15.148:0/1646464517 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568031b70 con 0x7fe5700cd540 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 -- 172.21.15.148:0/1646464517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 msgr2=0x7fe5700caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 --2- 172.21.15.148:0/1646464517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700caad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe568009d90 tx=0x7fe56802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 -- 172.21.15.148:0/1646464517 shutdown_connections 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 --2- 172.21.15.148:0/1646464517 >> v2:172.21.15.188:3300/0 conn(0x7fe570162140 0x7fe5700cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 --2- 172.21.15.148:0/1646464517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700caad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 -- 172.21.15.148:0/1646464517 >> 172.21.15.148:0/1646464517 conn(0x7fe5700d17c0 msgr2=0x7fe5700d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 -- 172.21.15.148:0/1646464517 shutdown_connections 2024-04-25T10:01:43.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.352+0000 7fe5765b0640 1 -- 172.21.15.148:0/1646464517 wait complete. 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.353+0000 7fe5765b0640 1 Processor -- start 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 -- start start 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5700d0a00 0x7fe570146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 0x7fe5701f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe570161940 con 0x7fe570162140 2024-04-25T10:01:43.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe570161660 con 0x7fe5700d0a00 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.354+0000 7fe5765b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe570161c20 con 0x7fe5700cd540 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 0x7fe5701f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 0x7fe5701f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34396/0 (socket says 172.21.15.148:34396) 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 -- 172.21.15.148:0/2876047798 learned_addr learned my addr 172.21.15.148:0/2876047798 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56f7fe640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5700d0a00 0x7fe570146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56ffff640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 -- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 msgr2=0x7fe5700a87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700a87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 -- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5700d0a00 msgr2=0x7fe570146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5700d0a00 0x7fe570146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 -- 172.21.15.148:0/2876047798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe570146d50 con 0x7fe570162140 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56ffff640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700a87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56f7fe640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5700d0a00 0x7fe570146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe574b4d640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 0x7fe5701f6f10 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe568009eb0 tx=0x7fe568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568040070 con 0x7fe570162140 2024-04-25T10:01:43.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568030c00 con 0x7fe570162140 2024-04-25T10:01:43.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568009450 con 0x7fe570162140 2024-04-25T10:01:43.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5700b5590 con 0x7fe570162140 2024-04-25T10:01:43.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.355+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5700b5b30 con 0x7fe570162140 2024-04-25T10:01:43.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.356+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe570073a00 con 0x7fe570162140 2024-04-25T10:01:43.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.357+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe568039040 con 0x7fe570162140 2024-04-25T10:01:43.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.357+0000 7fe56cff9640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe55c06b4d0 0x7fe55c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.357+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe5680ab710 con 0x7fe570162140 2024-04-25T10:01:43.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.357+0000 7fe56ffff640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe55c06b4d0 0x7fe55c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.358+0000 7fe56ffff640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe55c06b4d0 0x7fe55c06d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe560000f60 tx=0x7fe560009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.361+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe56807a3b0 con 0x7fe570162140 2024-04-25T10:01:43.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:43.531+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe570071c50 con 0x7fe570162140 2024-04-25T10:01:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.230+0000 7fe56cff9640 1 -- 172.21.15.148:0/2876047798 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe568079cb0 con 0x7fe570162140 2024-04-25T10:01:44.235 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe55c06b4d0 msgr2=0x7fe55c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe55c06b4d0 0x7fe55c06d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe560000f60 tx=0x7fe560009040 comp rx=0 tx=0).stop 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 msgr2=0x7fe5701f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 0x7fe5701f6f10 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe568009eb0 tx=0x7fe568004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 shutdown_connections 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe55c06b4d0 0x7fe55c06d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe570162140 0x7fe5701f6f10 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5700d0a00 0x7fe570146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 --2- 172.21.15.148:0/2876047798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cd540 0x7fe5700a87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 >> 172.21.15.148:0/2876047798 conn(0x7fe5700d17c0 msgr2=0x7fe570070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.234+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 shutdown_connections 2024-04-25T10:01:44.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.235+0000 7fe5765b0640 1 -- 172.21.15.148:0/2876047798 wait complete. 2024-04-25T10:01:44.252 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:01:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.353+0000 7f107d57e640 1 Processor -- start 2024-04-25T10:01:44.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.353+0000 7f107d57e640 1 -- start start 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f107d57e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1078161710 0x7f10780d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f107d57e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107815e000 con 0x7f10780ced80 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f107d57e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10780d1de0 con 0x7f107815dc20 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f107d57e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1078161c60 con 0x7f1078161710 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f10777fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1078161710 0x7f10780d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f10767fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f10780ced80 0x7f10780d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50030/0 (socket says 172.21.15.148:50030) 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f10767fc640 1 -- 172.21.15.148:0/297233688 learned_addr learned my addr 172.21.15.148:0/297233688 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:44.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f10777fe640 1 -- 172.21.15.148:0/297233688 >> v1:172.21.15.188:6789/0 conn(0x7f107815dc20 legacy=0x7f10780ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f10777fe640 1 -- 172.21.15.148:0/297233688 >> v1:172.21.15.148:6789/0 conn(0x7f10780ced80 legacy=0x7f10780d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f10777fe640 1 -- 172.21.15.148:0/297233688 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1078161f40 con 0x7f1078161710 2024-04-25T10:01:44.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.354+0000 7f1075ffb640 1 -- 172.21.15.148:0/297233688 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731784545 0 0) 0x7f107815e000 con 0x7f10780ced80 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.355+0000 7f10777fe640 1 --2- 172.21.15.148:0/297233688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1078161710 0x7f10780d29f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1070009d90 tx=0x7f107002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb2110ac03409d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.355+0000 7f1075ffb640 1 -- 172.21.15.148:0/297233688 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070007740 con 0x7f1078161710 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.355+0000 7f1075ffb640 1 -- 172.21.15.148:0/297233688 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10700078e0 con 0x7f1078161710 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.355+0000 7f1075ffb640 1 -- 172.21.15.148:0/297233688 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070007cb0 con 0x7f1078161710 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 -- 172.21.15.148:0/297233688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1078161710 msgr2=0x7f10780d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 --2- 172.21.15.148:0/297233688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1078161710 0x7f10780d29f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1070009d90 tx=0x7f107002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 -- 172.21.15.148:0/297233688 shutdown_connections 2024-04-25T10:01:44.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 --2- 172.21.15.148:0/297233688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1078161710 0x7f10780d29f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 -- 172.21.15.148:0/297233688 >> 172.21.15.148:0/297233688 conn(0x7f10780d0420 msgr2=0x7f10780d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 -- 172.21.15.148:0/297233688 shutdown_connections 2024-04-25T10:01:44.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.356+0000 7f107d57e640 1 -- 172.21.15.148:0/297233688 wait complete. 2024-04-25T10:01:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.357+0000 7f107d57e640 1 Processor -- start 2024-04-25T10:01:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.357+0000 7f107d57e640 1 -- start start 2024-04-25T10:01:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f107d57e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 0x7f107815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f107d57e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107815dc20 0x7f10781600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f107d57e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1078161710 0x7f107815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f107d57e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10780b5e50 con 0x7f10780ced80 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f107d57e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078162500 con 0x7f107815dc20 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f107d57e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10780b5fd0 con 0x7f1078161710 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 0x7f107815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 0x7f107815e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42888/0 (socket says 172.21.15.148:42888) 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 -- 172.21.15.148:0/1692238767 learned_addr learned my addr 172.21.15.148:0/1692238767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f10767fc640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107815dc20 0x7f10781600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 -- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1078161710 msgr2=0x7f107815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1078161710 0x7f107815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 -- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107815dc20 msgr2=0x7f10781600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107815dc20 0x7f10781600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 -- 172.21.15.148:0/1692238767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107815e8d0 con 0x7f10780ced80 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f1076ffd640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 0x7f107815e210 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f106800c430 tx=0x7f106800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.358+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068011790 con 0x7f10780ced80 2024-04-25T10:01:44.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.359+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068017070 con 0x7f10780ced80 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.359+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068011bb0 con 0x7f10780ced80 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.359+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107815ebb0 con 0x7f10780ced80 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.359+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107815f170 con 0x7f10780ced80 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.359+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1078073a00 con 0x7f10780ced80 2024-04-25T10:01:44.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.360+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f106801c460 con 0x7f10780ced80 2024-04-25T10:01:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.360+0000 7f105f7fe640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.360+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f106808a110 con 0x7f10780ced80 2024-04-25T10:01:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.360+0000 7f10767fc640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.361+0000 7f10767fc640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f107815fd00 tx=0x7f106c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.365+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1068058f60 con 0x7f10780ced80 2024-04-25T10:01:44.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:44.534+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10780d16b0 con 0x7f10780ced80 2024-04-25T10:01:45.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.232+0000 7f105f7fe640 1 -- 172.21.15.148:0/1692238767 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1068058860 con 0x7f10780ced80 2024-04-25T10:01:45.236 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f106406b4d0 msgr2=0x7f106406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f107815fd00 tx=0x7f106c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 msgr2=0x7f107815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 0x7f107815e210 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f106800c430 tx=0x7f106800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 shutdown_connections 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1078161710 0x7f107815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107815dc20 0x7f10781600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 --2- 172.21.15.148:0/1692238767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10780ced80 0x7f107815e210 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 >> 172.21.15.148:0/1692238767 conn(0x7f10780d0420 msgr2=0x7f10780d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 shutdown_connections 2024-04-25T10:01:45.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.236+0000 7f107d57e640 1 -- 172.21.15.148:0/1692238767 wait complete. 2024-04-25T10:01:45.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:01:45.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.360+0000 7f398bcdd640 1 Processor -- start 2024-04-25T10:01:45.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 -- start start 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f398415dc20 0x7f39840ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3984161710 0x7f39840cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f39840b9330 0x7f398406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398415e000 con 0x7f398415dc20 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39840d1de0 con 0x7f39840b9330 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f398bcdd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3984161c60 con 0x7f3984161710 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f3989a79640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f398415dc20 0x7f39840ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f3989a79640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f398415dc20 0x7f39840ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42904/0 (socket says 172.21.15.148:42904) 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.361+0000 7f3989a79640 1 -- 172.21.15.148:0/178821860 learned_addr learned my addr 172.21.15.148:0/178821860 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:45.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f398a27a640 1 --2- 172.21.15.148:0/178821860 >> v2:172.21.15.188:3300/0 conn(0x7f39840b9330 0x7f398406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989278640 1 --2- 172.21.15.148:0/178821860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3984161710 0x7f39840cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989a79640 1 -- 172.21.15.148:0/178821860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3984161710 msgr2=0x7f39840cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989a79640 1 --2- 172.21.15.148:0/178821860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3984161710 0x7f39840cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989a79640 1 -- 172.21.15.148:0/178821860 >> v2:172.21.15.188:3300/0 conn(0x7f39840b9330 msgr2=0x7f398406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989a79640 1 --2- 172.21.15.148:0/178821860 >> v2:172.21.15.188:3300/0 conn(0x7f39840b9330 0x7f398406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989a79640 1 -- 172.21.15.148:0/178821860 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984161f40 con 0x7f398415dc20 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.362+0000 7f3989a79640 1 --2- 172.21.15.148:0/178821860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398415dc20 0x7f39840ce670 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f39840b6570 tx=0x7f398002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830302e4e95c157a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f3988a77640 1 -- 172.21.15.148:0/178821860 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980036070 con 0x7f398415dc20 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f3988a77640 1 -- 172.21.15.148:0/178821860 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3980031600 con 0x7f398415dc20 2024-04-25T10:01:45.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f3988a77640 1 -- 172.21.15.148:0/178821860 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39800319d0 con 0x7f398415dc20 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 -- 172.21.15.148:0/178821860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398415dc20 msgr2=0x7f39840ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 --2- 172.21.15.148:0/178821860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398415dc20 0x7f39840ce670 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f39840b6570 tx=0x7f398002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 -- 172.21.15.148:0/178821860 shutdown_connections 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 --2- 172.21.15.148:0/178821860 >> v2:172.21.15.188:3300/0 conn(0x7f39840b9330 0x7f398406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 --2- 172.21.15.148:0/178821860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3984161710 0x7f39840cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 --2- 172.21.15.148:0/178821860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f398415dc20 0x7f39840ce670 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 -- 172.21.15.148:0/178821860 >> 172.21.15.148:0/178821860 conn(0x7f39840d0420 msgr2=0x7f39840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 -- 172.21.15.148:0/178821860 shutdown_connections 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.363+0000 7f398bcdd640 1 -- 172.21.15.148:0/178821860 wait complete. 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.364+0000 7f398bcdd640 1 Processor -- start 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 -- start start 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 0x7f39841e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415dc20 0x7f39841f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984161710 0x7f3984149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39840b5e50 con 0x7f39840b9330 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3984162500 con 0x7f3984161710 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f398bcdd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39840b5fd0 con 0x7f398415dc20 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f3989a79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 0x7f39841e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f3989a79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 0x7f39841e8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42908/0 (socket says 172.21.15.148:42908) 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.365+0000 7f3989a79640 1 -- 172.21.15.148:0/1876898914 learned_addr learned my addr 172.21.15.148:0/1876898914 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989278640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415dc20 0x7f39841f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989a79640 1 -- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415dc20 msgr2=0x7f39841f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f398a27a640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984161710 0x7f3984149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989a79640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415dc20 0x7f39841f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989a79640 1 -- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984161710 msgr2=0x7f3984149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989a79640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984161710 0x7f3984149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989a79640 1 -- 172.21.15.148:0/1876898914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39841e9490 con 0x7f39840b9330 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f398a27a640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984161710 0x7f3984149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989278640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415dc20 0x7f39841f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f3989a79640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 0x7f39841e8f50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f3980034040 tx=0x7f3980004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980041020 con 0x7f39840b9330 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3980007800 con 0x7f39840b9330 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980036040 con 0x7f39840b9330 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39840caf70 con 0x7f39840b9330 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.366+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39840cb530 con 0x7f39840b9330 2024-04-25T10:01:45.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.367+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3984073a00 con 0x7f39840b9330 2024-04-25T10:01:45.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.367+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3980044020 con 0x7f39840b9330 2024-04-25T10:01:45.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.368+0000 7f397a7fc640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f396806b500 0x7f396806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.368+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f39800aada0 con 0x7f39840b9330 2024-04-25T10:01:45.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.368+0000 7f3989278640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f396806b500 0x7f396806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.368+0000 7f3989278640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f396806b500 0x7f396806d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3974006fd0 tx=0x7f3974008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.372+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f398004b030 con 0x7f39840b9330 2024-04-25T10:01:45.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.542+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f39840d16b0 con 0x7f39840b9330 2024-04-25T10:01:45.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.545+0000 7f397a7fc640 1 -- 172.21.15.148:0/1876898914 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3980079e50 con 0x7f39840b9330 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f396806b500 msgr2=0x7f396806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f396806b500 0x7f396806d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3974006fd0 tx=0x7f3974008040 comp rx=0 tx=0).stop 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 msgr2=0x7f39841e8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 0x7f39841e8f50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f3980034040 tx=0x7f3980004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 shutdown_connections 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f396806b500 0x7f396806d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3984161710 0x7f3984149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415dc20 0x7f39841f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 --2- 172.21.15.148:0/1876898914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39840b9330 0x7f39841e8f50 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 >> 172.21.15.148:0/1876898914 conn(0x7f39840d0420 msgr2=0x7f39840b7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.552+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 shutdown_connections 2024-04-25T10:01:45.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.553+0000 7f398bcdd640 1 -- 172.21.15.148:0/1876898914 wait complete. 2024-04-25T10:01:45.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:01:45.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.690+0000 7fd170cb3640 1 Processor -- start 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.690+0000 7fd170cb3640 1 -- start start 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd170cb3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd16c0ce330 0x7fd16c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd170cb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd170cb3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c1626e0 con 0x7fd16c06abe0 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd170cb3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c0cd270 con 0x7fd16c0ce330 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd170cb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0cd550 con 0x7fd16c0700a0 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd16a59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd16c06abe0 0x7fd16c06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50040/0 (socket says 172.21.15.148:50040) 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd16a59c640 1 -- 172.21.15.148:0/2104936186 learned_addr learned my addr 172.21.15.148:0/2104936186 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd169d9b640 1 --2- 172.21.15.148:0/2104936186 >> v2:172.21.15.148:3300/0 conn(0x7fd16c0ce330 0x7fd16c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd16ad9d640 1 --2- 172.21.15.148:0/2104936186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd16959a640 1 -- 172.21.15.148:0/2104936186 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284327476 0 0) 0x7fd16c1626e0 con 0x7fd16c06abe0 2024-04-25T10:01:45.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd16959a640 1 -- 172.21.15.148:0/2104936186 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd154003520 con 0x7fd16c06abe0 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd169d9b640 1 -- 172.21.15.148:0/2104936186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 msgr2=0x7fd16c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd169d9b640 1 --2- 172.21.15.148:0/2104936186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd169d9b640 1 -- 172.21.15.148:0/2104936186 >> v1:172.21.15.148:6789/0 conn(0x7fd16c06abe0 legacy=0x7fd16c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.691+0000 7fd169d9b640 1 -- 172.21.15.148:0/2104936186 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c0cd830 con 0x7fd16c0ce330 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.692+0000 7fd169d9b640 1 --2- 172.21.15.148:0/2104936186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c06fb60 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd1600097a0 tx=0x7fd16002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919834d6f90bac09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.692+0000 7fd16959a640 1 -- 172.21.15.148:0/2104936186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160037070 con 0x7fd16c0ce330 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.692+0000 7fd16959a640 1 -- 172.21.15.148:0/2104936186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16002ae50 con 0x7fd16c0ce330 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.692+0000 7fd16959a640 1 -- 172.21.15.148:0/2104936186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160032660 con 0x7fd16c0ce330 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 -- 172.21.15.148:0/2104936186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 msgr2=0x7fd16c06fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 --2- 172.21.15.148:0/2104936186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c06fb60 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd1600097a0 tx=0x7fd16002a100 comp rx=0 tx=0).stop 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 -- 172.21.15.148:0/2104936186 shutdown_connections 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 --2- 172.21.15.148:0/2104936186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 --2- 172.21.15.148:0/2104936186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c06fb60 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 -- 172.21.15.148:0/2104936186 >> 172.21.15.148:0/2104936186 conn(0x7fd16c0b2220 msgr2=0x7fd16c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 -- 172.21.15.148:0/2104936186 shutdown_connections 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.693+0000 7fd170cb3640 1 -- 172.21.15.148:0/2104936186 wait complete. 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.694+0000 7fd170cb3640 1 Processor -- start 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.694+0000 7fd170cb3640 1 -- start start 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd170cb3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd16c06abe0 0x7fd16c1e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd170cb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd170cb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd170cb3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c15d8b0 con 0x7fd16c0ce330 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd170cb3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c161780 con 0x7fd16c06abe0 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd170cb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c15da30 con 0x7fd16c0700a0 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd169d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c0b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42926/0 (socket says 172.21.15.148:42926) 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 -- 172.21.15.148:0/953568380 learned_addr learned my addr 172.21.15.148:0/953568380 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd169d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:60338/0 (socket says 172.21.15.148:60338) 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16a59c640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd16c06abe0 0x7fd16c1e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 -- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 msgr2=0x7fd16c15e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 -- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd16c06abe0 msgr2=0x7fd16c1e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd16c06abe0 0x7fd16c1e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.695+0000 7fd16ad9d640 1 -- 172.21.15.148:0/953568380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c15f210 con 0x7fd16c0ce330 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.696+0000 7fd16ad9d640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c0b5100 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd164007e30 tx=0x7fd16400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.696+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164014620 con 0x7fd16c0ce330 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.696+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164019070 con 0x7fd16c0ce330 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.696+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c1e7710 con 0x7fd16c0ce330 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.696+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164014a00 con 0x7fd16c0ce330 2024-04-25T10:01:45.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.696+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c1e7cd0 con 0x7fd16c0ce330 2024-04-25T10:01:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.697+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c073990 con 0x7fd16c0ce330 2024-04-25T10:01:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.697+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd164014ba0 con 0x7fd16c0ce330 2024-04-25T10:01:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.697+0000 7fd15affd640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd14c06b500 0x7fd14c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.698+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd16408a750 con 0x7fd16c0ce330 2024-04-25T10:01:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.698+0000 7fd16a59c640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd14c06b500 0x7fd14c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.698+0000 7fd16a59c640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd14c06b500 0x7fd14c06d9c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd15c005880 tx=0x7fd15c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.702+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd164059180 con 0x7fd16c0ce330 2024-04-25T10:01:45.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:45.940+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fd16c0afe50 con 0x7fd16c0ce330 2024-04-25T10:01:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.235+0000 7fd15affd640 1 -- 172.21.15.148:0/953568380 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v364) v1 ==== 155+0+0 (secure 0 0 0) 0x7fd16405d220 con 0x7fd16c0ce330 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd14c06b500 msgr2=0x7fd14c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd14c06b500 0x7fd14c06d9c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd15c005880 tx=0x7fd15c009040 comp rx=0 tx=0).stop 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 msgr2=0x7fd16c0b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c0b5100 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd164007e30 tx=0x7fd16400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 shutdown_connections 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd14c06b500 0x7fd14c06d9c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd16c0ce330 0x7fd16c0b5100 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd16c0700a0 0x7fd16c15e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 --2- 172.21.15.148:0/953568380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd16c06abe0 0x7fd16c1e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 >> 172.21.15.148:0/953568380 conn(0x7fd16c0b2220 msgr2=0x7fd16c06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 shutdown_connections 2024-04-25T10:01:46.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.237+0000 7fd170cb3640 1 -- 172.21.15.148:0/953568380 wait complete. 2024-04-25T10:01:46.255 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:01:46.256 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:01:46.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:46.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 Processor -- start 2024-04-25T10:01:46.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 -- start start 2024-04-25T10:01:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59b00cc8e0 0x7f59b015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b015f390 con 0x7f59b00d1810 2024-04-25T10:01:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00cce50 con 0x7f59b00cc8e0 2024-04-25T10:01:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.352+0000 7f59b693b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0161af0 con 0x7f59b015dc90 2024-04-25T10:01:46.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59b00cc8e0 0x7f59b015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59b00cc8e0 0x7f59b015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42940/0 (socket says 172.21.15.148:42940) 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 -- 172.21.15.148:0/1554021060 learned_addr learned my addr 172.21.15.148:0/1554021060 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 -- 172.21.15.148:0/1554021060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 msgr2=0x7f59b01625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 --2- 172.21.15.148:0/1554021060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 -- 172.21.15.148:0/1554021060 >> v1:172.21.15.188:6789/0 conn(0x7f59b00d1810 legacy=0x7f59b00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.353+0000 7f59af7fe640 1 -- 172.21.15.148:0/1554021060 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0161dd0 con 0x7f59b00cc8e0 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59af7fe640 1 --2- 172.21.15.148:0/1554021060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00cc8e0 0x7f59b015ee50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009870 tx=0x7f59a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8596487a86e624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59aeffd640 1 -- 172.21.15.148:0/1554021060 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4037070 con 0x7f59b00cc8e0 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59aeffd640 1 -- 172.21.15.148:0/1554021060 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4032440 con 0x7f59b00cc8e0 2024-04-25T10:01:46.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59aeffd640 1 -- 172.21.15.148:0/1554021060 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4032810 con 0x7f59b00cc8e0 2024-04-25T10:01:46.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 -- 172.21.15.148:0/1554021060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00cc8e0 msgr2=0x7f59b015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 --2- 172.21.15.148:0/1554021060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00cc8e0 0x7f59b015ee50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009870 tx=0x7f59a402a180 comp rx=0 tx=0).stop 2024-04-25T10:01:46.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 -- 172.21.15.148:0/1554021060 shutdown_connections 2024-04-25T10:01:46.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 --2- 172.21.15.148:0/1554021060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00cc8e0 0x7f59b015ee50 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 --2- 172.21.15.148:0/1554021060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b01625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 -- 172.21.15.148:0/1554021060 >> 172.21.15.148:0/1554021060 conn(0x7f59b006f590 msgr2=0x7f59b006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 -- 172.21.15.148:0/1554021060 shutdown_connections 2024-04-25T10:01:46.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.354+0000 7f59b693b640 1 -- 172.21.15.148:0/1554021060 wait complete. 2024-04-25T10:01:46.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 Processor -- start 2024-04-25T10:01:46.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 -- start start 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b00cc8e0 0x7f59b01f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 0x7f59b00ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b00ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b015e070 con 0x7f59b00d1810 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b00cd130 con 0x7f59b00cc8e0 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.356+0000 7f59b693b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b015e1f0 con 0x7f59b015dc90 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 0x7f59b00ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 0x7f59b00ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42954/0 (socket says 172.21.15.148:42954) 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 -- 172.21.15.148:0/918067557 learned_addr learned my addr 172.21.15.148:0/918067557 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59affff640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b00cc8e0 0x7f59b01f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59b4ed8640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b00ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 -- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 msgr2=0x7f59b00ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b00ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 -- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b00cc8e0 msgr2=0x7f59b01f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b00cc8e0 0x7f59b01f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 -- 172.21.15.148:0/918067557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b014e540 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59af7fe640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 0x7f59b00ac930 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009870 tx=0x7f59a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4041070 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a403c870 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b014e810 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4037040 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.357+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b014edd0 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.358+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b0073990 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.359+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59a4044070 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.359+0000 7f59acff9640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f599c06b4d0 0x7f599c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.359+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f59a40aacc0 con 0x7f59b00d1810 2024-04-25T10:01:46.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.359+0000 7f59affff640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f599c06b4d0 0x7f599c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.360+0000 7f59affff640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f599c06b4d0 0x7f599c06d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009990 tx=0x7f59a0009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.363+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59b015f390 con 0x7f59b00d1810 2024-04-25T10:01:46.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.534+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59b00ce970 con 0x7f59b00d1810 2024-04-25T10:01:46.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.535+0000 7f59acff9640 1 -- 172.21.15.148:0/918067557 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+13353 (secure 0 0 0) 0x7f59a4079780 con 0x7f59b00d1810 2024-04-25T10:01:46.539 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:46.539 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":112,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:45.231164+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6834/3455709247":"2024-04-26T10:01:42.226549+0000","172.21.15.188:6832/3455709247":"2024-04-26T10:01:42.226549+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:46.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f599c06b4d0 msgr2=0x7f599c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f599c06b4d0 0x7f599c06d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009990 tx=0x7f59a0009110 comp rx=0 tx=0).stop 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 msgr2=0x7f59b00ac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 0x7f59b00ac930 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009870 tx=0x7f59a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 shutdown_connections 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f599c06b4d0 0x7f599c06d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59b015dc90 0x7f59b00ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59b00d1810 0x7f59b00ac930 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 --2- 172.21.15.148:0/918067557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59b00cc8e0 0x7f59b01f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 >> 172.21.15.148:0/918067557 conn(0x7f59b006f590 msgr2=0x7f59b00cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 shutdown_connections 2024-04-25T10:01:46.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.537+0000 7f59b693b640 1 -- 172.21.15.148:0/918067557 wait complete. 2024-04-25T10:01:46.552 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:46.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:46.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.649+0000 7f9e4b7ce640 1 Processor -- start 2024-04-25T10:01:46.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.649+0000 7f9e4b7ce640 1 -- start start 2024-04-25T10:01:46.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.650+0000 7f9e4b7ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 0x7f9e440cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.650+0000 7f9e4b7ce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440cb190 con 0x7f9e440d00e0 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.650+0000 7f9e4b7ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440cb470 con 0x7f9e440ccdd0 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.650+0000 7f9e4b7ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440cb750 con 0x7f9e4415bb20 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.650+0000 7f9e49d6b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9e440d00e0 0x7f9e440ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50044/0 (socket says 172.21.15.148:50044) 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.650+0000 7f9e49d6b640 1 -- 172.21.15.148:0/3204077704 learned_addr learned my addr 172.21.15.148:0/3204077704 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749403997 0 0) 0x7f9e440cb190 con 0x7f9e440d00e0 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 0x7f9e440cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e30003520 con 0x7f9e440d00e0 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3161269564 0 0) 0x7f9e30003520 con 0x7f9e440d00e0 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 msgr2=0x7f9e440cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 0x7f9e440cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 >> v1:172.21.15.188:6789/0 conn(0x7f9e440ccdd0 legacy=0x7f9e440cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440cba30 con 0x7f9e440d00e0 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9e3c001670 con 0x7f9e440d00e0 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 >> v1:172.21.15.148:6789/0 conn(0x7f9e440d00e0 legacy=0x7f9e440ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 0x7f9e30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e30004d40 0x7f9e300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440cb750 con 0x7f9e300039e0 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000baf0 con 0x7f9e30004d40 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000bcf0 con 0x7f9e4415bb20 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e49d6b640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e30004d40 0x7f9e300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e48d69640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 0x7f9e30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 -- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 msgr2=0x7f9e30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e4415bb20 0x7f9e30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 -- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e30004d40 msgr2=0x7f9e300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e30004d40 0x7f9e300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.651+0000 7f9e4956a640 1 -- 172.21.15.148:0/3204077704 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30003520 con 0x7f9e300039e0 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.652+0000 7f9e4956a640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e30003e00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fc0 tx=0x7f9e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea3861be8e87c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.652+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40037070 con 0x7f9e300039e0 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.652+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e400325f0 con 0x7f9e300039e0 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.652+0000 7f9e3bfff640 1 -- 172.21.15.148:0/3204077704 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40009c10 con 0x7f9e300039e0 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/3204077704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 msgr2=0x7f9e30003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/3204077704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e30003e00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fc0 tx=0x7f9e30005830 comp rx=0 tx=0).stop 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e49d6b640 1 -- 172.21.15.148:0/3204077704 reap_dead start 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/3204077704 shutdown_connections 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/3204077704 >> 172.21.15.148:0/3204077704 conn(0x7f9e440b2390 msgr2=0x7f9e440b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/3204077704 shutdown_connections 2024-04-25T10:01:46.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.653+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/3204077704 wait complete. 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.654+0000 7f9e4b7ce640 1 Processor -- start 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.654+0000 7f9e4b7ce640 1 -- start start 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e44145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e441596a0 0x7f9e44159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e4415a060 0x7f9e44148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4415c080 con 0x7f9e300039e0 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4415bf00 con 0x7f9e4415a060 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440cadb0 con 0x7f9e441596a0 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e44145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e44145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42984/0 (socket says 172.21.15.148:42984) 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 -- 172.21.15.148:0/1769525949 learned_addr learned my addr 172.21.15.148:0/1769525949 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e48d69640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e441596a0 0x7f9e44159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e49d6b640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e4415a060 0x7f9e44148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 -- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e441596a0 msgr2=0x7f9e44159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e441596a0 0x7f9e44159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 -- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e4415a060 msgr2=0x7f9e44148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e4415a060 0x7f9e44148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 -- 172.21.15.148:0/1769525949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440c7960 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e49d6b640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e4415a060 0x7f9e44148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4956a640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e44145d80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e170 tx=0x7f9e40009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40042030 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e400323f0 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.655+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440c7c40 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.656+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40037040 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.656+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4415a4c0 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.657+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44073a00 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.657+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e40045070 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.657+0000 7f9e39ffb640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9e2806b500 0x7f9e2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.658+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9e400aa9f0 con 0x7f9e300039e0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.658+0000 7f9e48d69640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9e2806b500 0x7f9e2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.658+0000 7f9e48d69640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9e2806b500 0x7f9e2806d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9e4415aa10 tx=0x7f9e34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.662+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e4002f070 con 0x7f9e300039e0 2024-04-25T10:01:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.831+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e440c7e10 con 0x7f9e300039e0 2024-04-25T10:01:46.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.831+0000 7f9e39ffb640 1 -- 172.21.15.148:0/1769525949 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e400325c0 con 0x7f9e300039e0 2024-04-25T10:01:46.836 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:01:46.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9e2806b500 msgr2=0x7f9e2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9e2806b500 0x7f9e2806d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9e4415aa10 tx=0x7f9e34009040 comp rx=0 tx=0).stop 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 msgr2=0x7f9e44145d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e44145d80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e170 tx=0x7f9e40009750 comp rx=0 tx=0).stop 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 shutdown_connections 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9e2806b500 0x7f9e2806d9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e4415a060 0x7f9e44148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e441596a0 0x7f9e44159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 --2- 172.21.15.148:0/1769525949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e300039e0 0x7f9e44145d80 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 >> 172.21.15.148:0/1769525949 conn(0x7f9e440b2390 msgr2=0x7f9e4406e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 shutdown_connections 2024-04-25T10:01:46.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.833+0000 7f9e4b7ce640 1 -- 172.21.15.148:0/1769525949 wait complete. 2024-04-25T10:01:46.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:01:46.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.942+0000 7f5f06dc1640 1 Processor -- start 2024-04-25T10:01:46.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.942+0000 7f5f06dc1640 1 -- start start 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5f06dc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f0015dc90 0x7f5f001625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5f06dc1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f000cce50 con 0x7f5f000cc8e0 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5f06dc1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f00161af0 con 0x7f5f000d1810 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5f06dc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f00161dd0 con 0x7f5f0015dc90 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5efffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5f000cc8e0 0x7f5f0015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50060/0 (socket says 172.21.15.148:50060) 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5efffff640 1 -- 172.21.15.148:0/1178560783 learned_addr learned my addr 172.21.15.148:0/1178560783 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5f04b5d640 1 --2- 172.21.15.148:0/1178560783 >> v2:172.21.15.188:3300/0 conn(0x7f5f0015dc90 0x7f5f001625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5eff7fe640 1 -- 172.21.15.148:0/1178560783 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211637428 0 0) 0x7f5f000cce50 con 0x7f5f000cc8e0 2024-04-25T10:01:46.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.943+0000 7f5eff7fe640 1 -- 172.21.15.148:0/1178560783 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee8003520 con 0x7f5f000cc8e0 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.944+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1178560783 >> v1:172.21.15.188:6789/0 conn(0x7f5f000d1810 legacy=0x7f5f000cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.944+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1178560783 >> v1:172.21.15.148:6789/0 conn(0x7f5f000cc8e0 legacy=0x7f5f0015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.944+0000 7f5eff7fe640 1 -- 172.21.15.148:0/1178560783 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3761226083 0 0) 0x7f5ee8003520 con 0x7f5f000cc8e0 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.944+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1178560783 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f001620b0 con 0x7f5f0015dc90 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.944+0000 7f5f04b5d640 1 --2- 172.21.15.148:0/1178560783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0015dc90 0x7f5f001625d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00097a0 tx=0x7f5ef002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4787c1d925a6375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5eff7fe640 1 -- 172.21.15.148:0/1178560783 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0037070 con 0x7f5f0015dc90 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5eff7fe640 1 -- 172.21.15.148:0/1178560783 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef002abc0 con 0x7f5f0015dc90 2024-04-25T10:01:46.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5eff7fe640 1 -- 172.21.15.148:0/1178560783 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4001670 con 0x7f5f0015dc90 2024-04-25T10:01:46.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1178560783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0015dc90 msgr2=0x7f5f001625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1178560783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0015dc90 0x7f5f001625d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00097a0 tx=0x7f5ef002a130 comp rx=0 tx=0).stop 2024-04-25T10:01:46.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1178560783 shutdown_connections 2024-04-25T10:01:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1178560783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0015dc90 0x7f5f001625d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1178560783 >> 172.21.15.148:0/1178560783 conn(0x7f5f0006f590 msgr2=0x7f5f0006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1178560783 shutdown_connections 2024-04-25T10:01:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.945+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1178560783 wait complete. 2024-04-25T10:01:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.946+0000 7f5f06dc1640 1 Processor -- start 2024-04-25T10:01:46.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.946+0000 7f5f06dc1640 1 -- start start 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f06dc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 0x7f5f001f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f06dc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f000d1810 0x7f5f000c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f06dc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0015dc90 0x7f5f000cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f06dc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015e1f0 con 0x7f5f000cc8e0 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f06dc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015e070 con 0x7f5f000d1810 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f06dc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000d1c80 con 0x7f5f0015dc90 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f04b5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 0x7f5f001f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f04b5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 0x7f5f001f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43000/0 (socket says 172.21.15.148:43000) 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.947+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1356165357 learned_addr learned my addr 172.21.15.148:0/1356165357 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5efffff640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f000d1810 0x7f5f000c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f0535e640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0015dc90 0x7f5f000cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0015dc90 msgr2=0x7f5f000cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f04b5d640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0015dc90 0x7f5f000cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f000d1810 msgr2=0x7f5f000c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f04b5d640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f000d1810 0x7f5f000c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f04b5d640 1 -- 172.21.15.148:0/1356165357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f001f7440 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f0535e640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0015dc90 0x7f5f000cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5efffff640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f000d1810 0x7f5f000c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f04b5d640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 0x7f5f001f6f00 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00098c0 tx=0x7f5ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0042020 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef003d670 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f000c7560 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0037040 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.948+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f000c7e00 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.949+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f000c7950 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.949+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ef0045020 con 0x7f5f000cc8e0 2024-04-25T10:01:46.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.950+0000 7f5efd7fa640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5eec06b4d0 0x7f5eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.950+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5ef00aa6b0 con 0x7f5f000cc8e0 2024-04-25T10:01:46.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.950+0000 7f5efffff640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5eec06b4d0 0x7f5eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.950+0000 7f5efffff640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5eec06b4d0 0x7f5eec06d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5f000d1420 tx=0x7f5ef4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:46.954+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ef004c070 con 0x7f5f000cc8e0 2024-04-25T10:01:47.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.190+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f5f00073990 con 0x7f5f000cc8e0 2024-04-25T10:01:47.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.236+0000 7f5efd7fa640 1 -- 172.21.15.148:0/1356165357 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v365) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5ef00796a0 con 0x7f5f000cc8e0 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5eec06b4d0 msgr2=0x7f5eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5eec06b4d0 0x7f5eec06d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5f000d1420 tx=0x7f5ef4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 msgr2=0x7f5f001f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 0x7f5f001f6f00 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00098c0 tx=0x7f5ef00046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 shutdown_connections 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5eec06b4d0 0x7f5eec06d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f0015dc90 0x7f5f000cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f000d1810 0x7f5f000c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 --2- 172.21.15.148:0/1356165357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f000cc8e0 0x7f5f001f6f00 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 >> 172.21.15.148:0/1356165357 conn(0x7f5f0006f590 msgr2=0x7f5f0015b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 shutdown_connections 2024-04-25T10:01:47.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.238+0000 7f5f06dc1640 1 -- 172.21.15.148:0/1356165357 wait complete. 2024-04-25T10:01:47.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:01:47.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.349+0000 7fa13ae53640 1 Processor -- start 2024-04-25T10:01:47.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.349+0000 7fa13ae53640 1 -- start start 2024-04-25T10:01:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.350+0000 7fa13ae53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa1340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.350+0000 7fa13ae53640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13406cda0 con 0x7fa134161710 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.350+0000 7fa13ae53640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13415e000 con 0x7fa1340d32f0 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.350+0000 7fa13ae53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340d1de0 con 0x7fa13415dc20 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.350+0000 7fa133fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa134161710 0x7fa13406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50062/0 (socket says 172.21.15.148:50062) 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.350+0000 7fa133fff640 1 -- 172.21.15.148:0/2496546254 learned_addr learned my addr 172.21.15.148:0/2496546254 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa1340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562352223 0 0) 0x7fa13406cda0 con 0x7fa134161710 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa11c003520 con 0x7fa134161710 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741338156 0 0) 0x7fa13415e000 con 0x7fa1340d32f0 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa13406cda0 con 0x7fa1340d32f0 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2513081325 0 0) 0x7fa11c003520 con 0x7fa134161710 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 msgr2=0x7fa1340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa1340ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 >> v1:172.21.15.188:6789/0 conn(0x7fa1340d32f0 legacy=0x7fa13406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134161c60 con 0x7fa134161710 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa128001670 con 0x7fa134161710 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 >> v1:172.21.15.148:6789/0 conn(0x7fa134161710 legacy=0x7fa13406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c003b70 0x7fa11c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c004800 0x7fa11c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa11c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340d1de0 con 0x7fa11c003b70 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c00ba70 con 0x7fa11c004800 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c00bc40 con 0x7fa13415dc20 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c003b70 0x7fa11c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa133fff640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c004800 0x7fa11c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1393f0640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa11c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 -- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 msgr2=0x7fa11c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa11c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 -- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c004800 msgr2=0x7fa11c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c004800 0x7fa11c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa138bef640 1 -- 172.21.15.148:0/2496546254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c003520 con 0x7fa11c003b70 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.351+0000 7fa1393f0640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13415dc20 0x7fa11c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.352+0000 7fa138bef640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c003b70 0x7fa11c003f70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ac0 tx=0x7fa11c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2995436e66a7b765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.352+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124036070 con 0x7fa11c003b70 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.352+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa124031400 con 0x7fa11c003b70 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.352+0000 7fa1337fe640 1 -- 172.21.15.148:0/2496546254 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124009690 con 0x7fa11c003b70 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa13ae53640 1 -- 172.21.15.148:0/2496546254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c003b70 msgr2=0x7fa11c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa13ae53640 1 --2- 172.21.15.148:0/2496546254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa11c003b70 0x7fa11c003f70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa124009ac0 tx=0x7fa11c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa1393f0640 1 -- 172.21.15.148:0/2496546254 reap_dead start 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa13ae53640 1 -- 172.21.15.148:0/2496546254 shutdown_connections 2024-04-25T10:01:47.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa13ae53640 1 -- 172.21.15.148:0/2496546254 >> 172.21.15.148:0/2496546254 conn(0x7fa1340d0420 msgr2=0x7fa1340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:47.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa13ae53640 1 -- 172.21.15.148:0/2496546254 shutdown_connections 2024-04-25T10:01:47.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.353+0000 7fa13ae53640 1 -- 172.21.15.148:0/2496546254 wait complete. 2024-04-25T10:01:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.354+0000 7fa13ae53640 1 Processor -- start 2024-04-25T10:01:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.354+0000 7fa13ae53640 1 -- start start 2024-04-25T10:01:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa13ae53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c003b70 0x7fa1340cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa13ae53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1341599f0 0x7fa1340cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa13ae53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 0x7fa1340cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa13ae53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa134162500 con 0x7fa134148f90 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa13ae53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa134162220 con 0x7fa11c003b70 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa13ae53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340b5e50 con 0x7fa1341599f0 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 0x7fa1340cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 0x7fa1340cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43018/0 (socket says 172.21.15.148:43018) 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 -- 172.21.15.148:0/4197987184 learned_addr learned my addr 172.21.15.148:0/4197987184 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa133fff640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1341599f0 0x7fa1340cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa138bef640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c003b70 0x7fa1340cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 -- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1341599f0 msgr2=0x7fa1340cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1341599f0 0x7fa1340cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 -- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c003b70 msgr2=0x7fa1340cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c003b70 0x7fa1340cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 -- 172.21.15.148:0/4197987184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134159e80 con 0x7fa134148f90 2024-04-25T10:01:47.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1393f0640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 0x7fa1340cb1f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa12c001330 tx=0x7fa12c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c006b50 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.355+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c006d10 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.356+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13415a160 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.356+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c003970 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.356+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13415a720 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.357+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134073a00 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.357+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa12c01a020 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.357+0000 7fa1317fa640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12006b4d0 0x7fa12006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.358+0000 7fa138bef640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12006b4d0 0x7fa12006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.358+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa12c0896e0 con 0x7fa134148f90 2024-04-25T10:01:47.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.358+0000 7fa138bef640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12006b4d0 0x7fa12006d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa124033000 tx=0x7fa124008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:47.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.362+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa12c0586d0 con 0x7fa134148f90 2024-04-25T10:01:47.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.532+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa1340b5a60 con 0x7fa134148f90 2024-04-25T10:01:47.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.539+0000 7fa1317fa640 1 -- 172.21.15.148:0/4197987184 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa12c057fd0 con 0x7fa134148f90 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12006b4d0 msgr2=0x7fa12006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12006b4d0 0x7fa12006d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa124033000 tx=0x7fa124008030 comp rx=0 tx=0).stop 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 msgr2=0x7fa1340cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 0x7fa1340cb1f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fa12c001330 tx=0x7fa12c00eb90 comp rx=0 tx=0).stop 2024-04-25T10:01:47.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 shutdown_connections 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12006b4d0 0x7fa12006d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa134148f90 0x7fa1340cb1f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1341599f0 0x7fa1340cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 --2- 172.21.15.148:0/4197987184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa11c003b70 0x7fa1340cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 >> 172.21.15.148:0/4197987184 conn(0x7fa1340d0420 msgr2=0x7fa1340b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 shutdown_connections 2024-04-25T10:01:47.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.541+0000 7fa13ae53640 1 -- 172.21.15.148:0/4197987184 wait complete. 2024-04-25T10:01:47.556 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:01:47.557 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:47.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:01:47.648 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:01:47.649 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:47.649 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:47.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:47.665 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:01:47.665 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:47.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:47.683 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:01:47.684 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:47.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:01:47.720 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:01:47.721 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:01:47.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:01:47.743 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:01:47.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:47.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.854+0000 7f8cc9e12640 1 Processor -- start 2024-04-25T10:01:47.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.854+0000 7f8cc9e12640 1 -- start start 2024-04-25T10:01:47.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.854+0000 7f8cc9e12640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8cc406abe0 0x7f8cc406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.854+0000 7f8cc9e12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.854+0000 7f8cc9e12640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8cc40700a0 0x7f8cc406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.854+0000 7f8cc9e12640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc41626e0 con 0x7f8cc40700a0 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc9e12640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc406b500 con 0x7f8cc406abe0 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc9e12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40cd270 con 0x7f8cc40ce330 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8cc40700a0 0x7f8cc406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8cc40700a0 0x7f8cc406e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43032/0 (socket says 172.21.15.148:43032) 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 -- 172.21.15.148:0/722947599 learned_addr learned my addr 172.21.15.148:0/722947599 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc37fe640 1 --2- 172.21.15.148:0/722947599 >> v2:172.21.15.188:3300/0 conn(0x7f8cc406abe0 0x7f8cc406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc2ffd640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 -- 172.21.15.148:0/722947599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 msgr2=0x7f8cc406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 -- 172.21.15.148:0/722947599 >> v2:172.21.15.188:3300/0 conn(0x7f8cc406abe0 msgr2=0x7f8cc406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 --2- 172.21.15.148:0/722947599 >> v2:172.21.15.188:3300/0 conn(0x7f8cc406abe0 0x7f8cc406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc3fff640 1 -- 172.21.15.148:0/722947599 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40cd550 con 0x7f8cc40700a0 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc37fe640 1 --2- 172.21.15.148:0/722947599 >> v2:172.21.15.188:3300/0 conn(0x7f8cc406abe0 0x7f8cc406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.855+0000 7f8cc2ffd640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc3fff640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc406e3f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f8cc406e200 tx=0x7f8cbc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9ea5aa48a5b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc27fc640 1 -- 172.21.15.148:0/722947599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc036070 con 0x7f8cc40700a0 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc27fc640 1 -- 172.21.15.148:0/722947599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc031600 con 0x7f8cc40700a0 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc27fc640 1 -- 172.21.15.148:0/722947599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0319d0 con 0x7f8cc40700a0 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 -- 172.21.15.148:0/722947599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 msgr2=0x7f8cc406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc406e3f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f8cc406e200 tx=0x7f8cbc02a610 comp rx=0 tx=0).stop 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 -- 172.21.15.148:0/722947599 shutdown_connections 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc406e3f0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/722947599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/722947599 >> v2:172.21.15.188:3300/0 conn(0x7f8cc406abe0 0x7f8cc406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 -- 172.21.15.148:0/722947599 >> 172.21.15.148:0/722947599 conn(0x7f8cc40b2220 msgr2=0x7f8cc40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:47.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 -- 172.21.15.148:0/722947599 shutdown_connections 2024-04-25T10:01:47.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.856+0000 7f8cc9e12640 1 -- 172.21.15.148:0/722947599 wait complete. 2024-04-25T10:01:47.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.857+0000 7f8cc9e12640 1 Processor -- start 2024-04-25T10:01:47.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 -- start start 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc406abe0 0x7f8cc40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc40b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4161780 con 0x7f8cc40700a0 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40cdb10 con 0x7f8cc406abe0 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc9e12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc406e930 con 0x7f8cc40ce330 2024-04-25T10:01:47.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc40b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc40b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43042/0 (socket says 172.21.15.148:43042) 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.858+0000 7f8cc2ffd640 1 -- 172.21.15.148:0/3638898322 learned_addr learned my addr 172.21.15.148:0/3638898322 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc37fe640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc406abe0 0x7f8cc40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc3fff640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc2ffd640 1 -- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 msgr2=0x7f8cc415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc2ffd640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc2ffd640 1 -- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc406abe0 msgr2=0x7f8cc40b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc2ffd640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc406abe0 0x7f8cc40b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc2ffd640 1 -- 172.21.15.148:0/3638898322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc415d580 con 0x7f8cc40700a0 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc37fe640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc406abe0 0x7f8cc40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc2ffd640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc40b41c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c7f0 tx=0x7f8cb400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:47.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4015070 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb4007820 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc415d300 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4007bd0 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.859+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc40b36e0 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.860+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc40afe50 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.860+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cb4018070 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.861+0000 7f8cabfff640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.861+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8cb408a230 con 0x7f8cc40700a0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.861+0000 7f8cc37fe640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:47.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.861+0000 7f8cc37fe640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8cb006b4d0 0x7f8cb006d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006b10 tx=0x7f8cb8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:47.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:47.865+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cc415d300 con 0x7f8cc40700a0 2024-04-25T10:01:48.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.073+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc40b6670 con 0x7f8cc40700a0 2024-04-25T10:01:48.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.073+0000 7f8cabfff640 1 -- 172.21.15.148:0/3638898322 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f8cb4059180 con 0x7f8cc40700a0 2024-04-25T10:01:48.078 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:48.078 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2519292283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2519292283},{"type":"v1","addr":"172.21.15.148:6837","nonce":2519292283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":6221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3618697403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3618697403},{"type":"v1","addr":"172.21.15.188:6835","nonce":3618697403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1238434469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1238434469},{"type":"v1","addr":"172.21.15.188:6834","nonce":1238434469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":8148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/671313066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":671313066},{"type":"v1","addr":"172.21.15.148:6836","nonce":671313066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2024-04-25T10:01:48.078 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 62 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8cb006b4d0 msgr2=0x7f8cb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8cb006b4d0 0x7f8cb006d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006b10 tx=0x7f8cb8008040 comp rx=0 tx=0).stop 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 msgr2=0x7f8cc40b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc40b41c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400c7f0 tx=0x7f8cb400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 shutdown_connections 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8cb006b4d0 0x7f8cb006d990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc40ce330 0x7f8cc415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8cc40700a0 0x7f8cc40b41c0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 --2- 172.21.15.148:0/3638898322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc406abe0 0x7f8cc40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 >> 172.21.15.148:0/3638898322 conn(0x7f8cc40b2220 msgr2=0x7f8cc4159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 shutdown_connections 2024-04-25T10:01:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.075+0000 7f8cc9e12640 1 -- 172.21.15.148:0/3638898322 wait complete. 2024-04-25T10:01:48.091 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:48.091 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:01:48.091 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:01:48.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:48.092 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:48.092 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:01:48.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:01:48.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:48.092 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:48.093 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:01:48.093 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:01:48.093 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:48.093 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:48.093 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:01:48.093 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:01:48.093 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:01:48.103 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:01:48.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:48.104 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:01:48.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:48.105 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:01:48.105 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:48.106 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:01:48.106 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:48.146 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:48.149 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:48.151 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:48.151 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:48.152 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:48.152 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:48.153 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:48.153 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:48.153 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:01:48.219 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:01:48.219 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:01:48.234 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:01:48.234 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:01:48.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.253+0000 7f808283f640 1 Processor -- start 2024-04-25T10:01:48.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.253+0000 7f808283f640 1 -- start start 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f808283f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f807c0cadb0 0x7f807c0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f808283f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f807c0ccfe0 0x7f807c0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f808283f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c0afc80 con 0x7f807c15de80 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f808283f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0cb240 con 0x7f807c0cadb0 2024-04-25T10:01:48.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f808283f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0ce4d0 con 0x7f807c0ccfe0 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f807c15de80 0x7f807c0cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50066/0 (socket says 172.21.15.148:50066) 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807bfff640 1 -- 172.21.15.148:0/4255369230 learned_addr learned my addr 172.21.15.148:0/4255369230 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807b7fe640 1 --2- 172.21.15.148:0/4255369230 >> v2:172.21.15.148:3300/0 conn(0x7f807c0cadb0 0x7f807c0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f8080ddc640 1 --2- 172.21.15.148:0/4255369230 >> v2:172.21.15.188:3300/0 conn(0x7f807c0ccfe0 0x7f807c0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807b7fe640 1 -- 172.21.15.148:0/4255369230 >> v2:172.21.15.188:3300/0 conn(0x7f807c0ccfe0 msgr2=0x7f807c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807b7fe640 1 --2- 172.21.15.148:0/4255369230 >> v2:172.21.15.188:3300/0 conn(0x7f807c0ccfe0 0x7f807c0cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807b7fe640 1 -- 172.21.15.148:0/4255369230 >> v1:172.21.15.148:6789/0 conn(0x7f807c15de80 legacy=0x7f807c0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.254+0000 7f807b7fe640 1 -- 172.21.15.148:0/4255369230 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0ce7b0 con 0x7f807c0cadb0 2024-04-25T10:01:48.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.255+0000 7f807b7fe640 1 --2- 172.21.15.148:0/4255369230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c0cf6c0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f807c06e2a0 tx=0x7f807002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b68abd19c589d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.256+0000 7f807affd640 1 -- 172.21.15.148:0/4255369230 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070037070 con 0x7f807c0cadb0 2024-04-25T10:01:48.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.256+0000 7f807affd640 1 -- 172.21.15.148:0/4255369230 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070032440 con 0x7f807c0cadb0 2024-04-25T10:01:48.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.256+0000 7f807affd640 1 -- 172.21.15.148:0/4255369230 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070032810 con 0x7f807c0cadb0 2024-04-25T10:01:48.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.256+0000 7f808283f640 1 -- 172.21.15.148:0/4255369230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 msgr2=0x7f807c0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.256+0000 7f808283f640 1 --2- 172.21.15.148:0/4255369230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c0cf6c0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f807c06e2a0 tx=0x7f807002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.257+0000 7f808283f640 1 -- 172.21.15.148:0/4255369230 shutdown_connections 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.257+0000 7f808283f640 1 --2- 172.21.15.148:0/4255369230 >> v2:172.21.15.188:3300/0 conn(0x7f807c0ccfe0 0x7f807c0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.257+0000 7f808283f640 1 --2- 172.21.15.148:0/4255369230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c0cf6c0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.257+0000 7f808283f640 1 -- 172.21.15.148:0/4255369230 >> 172.21.15.148:0/4255369230 conn(0x7f807c0b2390 msgr2=0x7f807c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.257+0000 7f808283f640 1 -- 172.21.15.148:0/4255369230 shutdown_connections 2024-04-25T10:01:48.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.257+0000 7f808283f640 1 -- 172.21.15.148:0/4255369230 wait complete. 2024-04-25T10:01:48.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.258+0000 7f808283f640 1 Processor -- start 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.258+0000 7f808283f640 1 -- start start 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807c0ccfe0 0x7f807c157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f807c15de80 0x7f807c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c15e260 con 0x7f807c0cadb0 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0cfc00 con 0x7f807c0ccfe0 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0b2eb0 con 0x7f807c15de80 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43070/0 (socket says 172.21.15.148:43070) 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 -- 172.21.15.148:0/669099934 learned_addr learned my addr 172.21.15.148:0/669099934 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:48.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807b7fe640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807c0ccfe0 0x7f807c157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f8080ddc640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f807c15de80 0x7f807c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 -- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f807c15de80 msgr2=0x7f807c1579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f807c15de80 0x7f807c1579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 -- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807c0ccfe0 msgr2=0x7f807c157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807c0ccfe0 0x7f807c157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 -- 172.21.15.148:0/669099934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0b6470 con 0x7f807c0cadb0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f807bfff640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c15aa20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f806c00c3c0 tx=0x7f806c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c017070 con 0x7f807c0cadb0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c005ce0 con 0x7f807c0cadb0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0072f0 con 0x7f807c0cadb0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c1577a0 con 0x7f807c0cadb0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.259+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c159f40 con 0x7f807c0cadb0 2024-04-25T10:01:48.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.260+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c073a00 con 0x7f807c0cadb0 2024-04-25T10:01:48.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.261+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f806c01a070 con 0x7f807c0cadb0 2024-04-25T10:01:48.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.261+0000 7f8078ff9640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:48.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.261+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f806c089390 con 0x7f807c0cadb0 2024-04-25T10:01:48.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.261+0000 7f807b7fe640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:48.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.262+0000 7f807b7fe640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f806806b4d0 0x7f806806d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8070034000 tx=0x7f807002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:48.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.265+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f807c0b6470 con 0x7f807c0cadb0 2024-04-25T10:01:48.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:48.449+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f807c158cc0 con 0x7f807c0cadb0 2024-04-25T10:01:49.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.269+0000 7f8078ff9640 1 -- 172.21.15.148:0/669099934 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7f806c057d20 con 0x7f807c0cadb0 2024-04-25T10:01:49.274 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:01:49.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f806806b4d0 msgr2=0x7f806806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f806806b4d0 0x7f806806d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8070034000 tx=0x7f807002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 msgr2=0x7f807c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c15aa20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f806c00c3c0 tx=0x7f806c00c880 comp rx=0 tx=0).stop 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 shutdown_connections 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f807c15de80 0x7f807c1579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807c0ccfe0 0x7f807c157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 --2- 172.21.15.148:0/669099934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807c0cadb0 0x7f807c15aa20 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 >> 172.21.15.148:0/669099934 conn(0x7f807c0b2390 msgr2=0x7f807c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 shutdown_connections 2024-04-25T10:01:49.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.273+0000 7f808283f640 1 -- 172.21.15.148:0/669099934 wait complete. 2024-04-25T10:01:49.291 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:01:49.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.390+0000 7ff835607640 1 Processor -- start 2024-04-25T10:01:49.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.390+0000 7ff835607640 1 -- start start 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff835607640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff830158220 0x7ff83015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff835607640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff8300af720 0x7ff8300afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff835607640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83015f180 con 0x7ff8301617b0 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff835607640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8300b3670 con 0x7ff830158220 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff835607640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff830070a10 con 0x7ff8300af720 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff82effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff830158220 0x7ff83015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff82effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff830158220 0x7ff83015ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43080/0 (socket says 172.21.15.148:43080) 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff82effd640 1 -- 172.21.15.148:0/1024666515 learned_addr learned my addr 172.21.15.148:0/1024666515 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.391+0000 7ff82f7fe640 1 --1- 172.21.15.148:0/1024666515 >> v1:172.21.15.148:6789/0 conn(0x7ff8301617b0 0x7ff830161be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50068/0 (socket says 172.21.15.148:50068) 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82effd640 1 -- 172.21.15.148:0/1024666515 >> v2:172.21.15.188:3300/0 conn(0x7ff8300af720 msgr2=0x7ff8300afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82effd640 1 --2- 172.21.15.148:0/1024666515 >> v2:172.21.15.188:3300/0 conn(0x7ff8300af720 0x7ff8300afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82effd640 1 -- 172.21.15.148:0/1024666515 >> v1:172.21.15.148:6789/0 conn(0x7ff8301617b0 legacy=0x7ff830161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82effd640 1 -- 172.21.15.148:0/1024666515 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83006fe00 con 0x7ff830158220 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82effd640 1 --2- 172.21.15.148:0/1024666515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015ec40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff8300cfc70 tx=0x7ff82002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd06fa15361a005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82dffb640 1 -- 172.21.15.148:0/1024666515 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820036070 con 0x7ff830158220 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82dffb640 1 -- 172.21.15.148:0/1024666515 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff820031600 con 0x7ff830158220 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.392+0000 7ff82dffb640 1 -- 172.21.15.148:0/1024666515 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8200319d0 con 0x7ff830158220 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 -- 172.21.15.148:0/1024666515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 msgr2=0x7ff83015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 --2- 172.21.15.148:0/1024666515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015ec40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ff8300cfc70 tx=0x7ff82002a730 comp rx=0 tx=0).stop 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 -- 172.21.15.148:0/1024666515 shutdown_connections 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 --2- 172.21.15.148:0/1024666515 >> v2:172.21.15.188:3300/0 conn(0x7ff8300af720 0x7ff8300afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 --2- 172.21.15.148:0/1024666515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015ec40 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 -- 172.21.15.148:0/1024666515 >> 172.21.15.148:0/1024666515 conn(0x7ff8300b2370 msgr2=0x7ff8300b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 -- 172.21.15.148:0/1024666515 shutdown_connections 2024-04-25T10:01:49.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.393+0000 7ff835607640 1 -- 172.21.15.148:0/1024666515 wait complete. 2024-04-25T10:01:49.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.394+0000 7ff835607640 1 Processor -- start 2024-04-25T10:01:49.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.394+0000 7ff835607640 1 -- start start 2024-04-25T10:01:49.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff835607640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300af720 0x7ff83014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff835607640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff835607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8301617b0 0x7ff83015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff835607640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8300706a0 con 0x7ff830158220 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff835607640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8300703c0 con 0x7ff8300af720 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff835607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff830158600 con 0x7ff8301617b0 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff82e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff82e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43090/0 (socket says 172.21.15.148:43090) 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff82e7fc640 1 -- 172.21.15.148:0/1450421022 learned_addr learned my addr 172.21.15.148:0/1450421022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff82effd640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300af720 0x7ff83014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.395+0000 7ff82f7fe640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8301617b0 0x7ff83015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff82e7fc640 1 -- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8301617b0 msgr2=0x7ff83015a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff82e7fc640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8301617b0 0x7ff83015a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff82e7fc640 1 -- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300af720 msgr2=0x7ff83014b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff82e7fc640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300af720 0x7ff83014b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff82e7fc640 1 -- 172.21.15.148:0/1450421022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830158b60 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff82e7fc640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015a220 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7ff82400b410 tx=0x7ff82400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824015020 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff824011850 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824011c40 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff830158e40 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.396+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff830159400 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.397+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff830073990 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.397+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff824018020 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.398+0000 7ff8177fe640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff81c06b4d0 0x7ff81c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.398+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff824089fa0 con 0x7ff830158220 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.398+0000 7ff82effd640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff81c06b4d0 0x7ff81c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.398+0000 7ff82effd640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff81c06b4d0 0x7ff81c06d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff820033000 tx=0x7ff820009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.402+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff82401f020 con 0x7ff830158220 2024-04-25T10:01:49.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.642+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff8300b3800 con 0x7ff830158220 2024-04-25T10:01:49.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.643+0000 7ff8177fe640 1 -- 172.21.15.148:0/1450421022 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff8240172c0 con 0x7ff830158220 2024-04-25T10:01:49.647 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.648 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.649 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.650 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.651 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.652 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.653 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.654 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.655 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.656 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:01:49.657 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.658 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.659 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:49.660 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:01:49.661 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff81c06b4d0 msgr2=0x7ff81c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff81c06b4d0 0x7ff81c06d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff820033000 tx=0x7ff820009000 comp rx=0 tx=0).stop 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 msgr2=0x7ff83015a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015a220 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7ff82400b410 tx=0x7ff82400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 shutdown_connections 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff81c06b4d0 0x7ff81c06d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8301617b0 0x7ff83015a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff830158220 0x7ff83015a220 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 --2- 172.21.15.148:0/1450421022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8300af720 0x7ff83014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 >> 172.21.15.148:0/1450421022 conn(0x7ff8300b2370 msgr2=0x7ff8300b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 shutdown_connections 2024-04-25T10:01:49.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.645+0000 7ff835607640 1 -- 172.21.15.148:0/1450421022 wait complete. 2024-04-25T10:01:49.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:49.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.757+0000 7f2ef5ecb640 1 Processor -- start 2024-04-25T10:01:49.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.757+0000 7f2ef5ecb640 1 -- start start 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.757+0000 7f2ef5ecb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2ef00b2cb0 0x7f2ef00d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.757+0000 7f2ef5ecb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00b97a0 con 0x7f2ef00d1020 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.757+0000 7f2ef5ecb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef00d1c50 con 0x7f2ef015dbe0 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.757+0000 7f2ef5ecb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef015dfc0 con 0x7f2ef00b2cb0 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eeffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2ef00d1020 0x7f2ef00d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50080/0 (socket says 172.21.15.148:50080) 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eeffff640 1 -- 172.21.15.148:0/1921028893 learned_addr learned my addr 172.21.15.148:0/1921028893 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247536161 0 0) 0x7f2ef00b97a0 con 0x7f2ef00d1020 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eeeffd640 1 --2- 172.21.15.148:0/1921028893 >> v2:172.21.15.188:3300/0 conn(0x7f2ef00b2cb0 0x7f2ef00d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed8003520 con 0x7f2ef00d1020 2024-04-25T10:01:49.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1595301719 0 0) 0x7f2ed8003520 con 0x7f2ef00d1020 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 >> v2:172.21.15.188:3300/0 conn(0x7f2ef00b2cb0 msgr2=0x7f2ef00d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 --2- 172.21.15.148:0/1921028893 >> v2:172.21.15.188:3300/0 conn(0x7f2ef00b2cb0 0x7f2ef00d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 >> v1:172.21.15.188:6789/0 conn(0x7f2ef015dbe0 legacy=0x7f2ef00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00d1f30 con 0x7f2ef00d1020 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f2ee80019f0 con 0x7f2ef00d1020 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 >> v1:172.21.15.148:6789/0 conn(0x7f2ef00d1020 legacy=0x7f2ef00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ed8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ed8004800 0x7f2ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed80051c0 0x7f2ed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed800c030 con 0x7f2ed80039b0 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed800c1e0 con 0x7f2ed80051c0 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed800c3e0 con 0x7f2ed8004800 2024-04-25T10:01:49.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ed8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eeffff640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed80051c0 0x7f2ed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.758+0000 7f2eeeffd640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ed8004800 0x7f2ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eef7fe640 1 -- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ed8004800 msgr2=0x7f2ed8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ed8004800 0x7f2ed8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eef7fe640 1 -- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed80051c0 msgr2=0x7f2ed800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ed80051c0 0x7f2ed800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eeffff640 1 -- 172.21.15.148:0/1921028893 reap_dead start 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eef7fe640 1 -- 172.21.15.148:0/1921028893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8003520 con 0x7f2ed80039b0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eeeffd640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ed8004800 0x7f2ed8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ed8003dd0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4001750 tx=0x7f2ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8366fb45aed720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4001370 con 0x7f2ed80039b0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee40052d0 con 0x7f2ed80039b0 2024-04-25T10:01:49.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.759+0000 7f2eee7fc640 1 -- 172.21.15.148:0/1921028893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee402abc0 con 0x7f2ed80039b0 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/1921028893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 msgr2=0x7f2ed8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ed8003dd0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4001750 tx=0x7f2ed8005830 comp rx=0 tx=0).stop 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/1921028893 shutdown_connections 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/1921028893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ed8003dd0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/1921028893 >> 172.21.15.148:0/1921028893 conn(0x7f2ef0061a20 msgr2=0x7f2ef0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/1921028893 shutdown_connections 2024-04-25T10:01:49.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.760+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/1921028893 wait complete. 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.761+0000 7f2ef5ecb640 1 Processor -- start 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.761+0000 7f2ef5ecb640 1 -- start start 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2ef5ecb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ef00b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2ef5ecb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef00b5280 0x7f2ef00b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2ef5ecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef014d830 0x7f2ef014dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2ef5ecb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b92a0 con 0x7f2ed80039b0 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2ef5ecb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00d1a00 con 0x7f2ef00b5280 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2ef5ecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b9420 con 0x7f2ef014d830 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ef00b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ef00b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43106/0 (socket says 172.21.15.148:43106) 2024-04-25T10:01:49.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 -- 172.21.15.148:0/279220914 learned_addr learned my addr 172.21.15.148:0/279220914 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eeffff640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef014d830 0x7f2ef014dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eeeffd640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef00b5280 0x7f2ef00b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 -- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef014d830 msgr2=0x7f2ef014dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef014d830 0x7f2ef014dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 -- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef00b5280 msgr2=0x7f2ef00b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef00b5280 0x7f2ef00b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.762+0000 7f2eef7fe640 1 -- 172.21.15.148:0/279220914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef014e590 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.763+0000 7f2eef7fe640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ef00b1a40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4001870 tx=0x7f2ee4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.763+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee403f030 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.763+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee4005480 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.763+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef014e870 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.763+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4037040 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.763+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef014ee30 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.764+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0073990 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.764+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ee4042070 con 0x7f2ed80039b0 2024-04-25T10:01:49.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.765+0000 7f2ed7fff640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2edc06b4d0 0x7f2edc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:49.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.765+0000 7f2eeeffd640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2edc06b4d0 0x7f2edc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:49.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.765+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2ee40a9670 con 0x7f2ed80039b0 2024-04-25T10:01:49.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.765+0000 7f2eeeffd640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2edc06b4d0 0x7f2edc06d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00b6e90 tx=0x7f2ee0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:49.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.769+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ee4078620 con 0x7f2ed80039b0 2024-04-25T10:01:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.942+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ef00b9080 con 0x7f2ed80039b0 2024-04-25T10:01:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.943+0000 7f2ed7fff640 1 -- 172.21.15.148:0/279220914 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13222 (secure 0 0 0) 0x7f2ee4077f20 con 0x7f2ed80039b0 2024-04-25T10:01:49.948 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:49.948 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":117,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:49.266816+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.945+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2edc06b4d0 msgr2=0x7f2edc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.945+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2edc06b4d0 0x7f2edc06d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00b6e90 tx=0x7f2ee0008040 comp rx=0 tx=0).stop 2024-04-25T10:01:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.945+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 msgr2=0x7f2ef00b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.945+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ef00b1a40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4001870 tx=0x7f2ee4001120 comp rx=0 tx=0).stop 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.945+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 shutdown_connections 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2edc06b4d0 0x7f2edc06d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ef014d830 0x7f2ef014dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ef00b5280 0x7f2ef00b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 --2- 172.21.15.148:0/279220914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed80039b0 0x7f2ef00b1a40 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 >> 172.21.15.148:0/279220914 conn(0x7f2ef0061a20 msgr2=0x7f2ef015b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 shutdown_connections 2024-04-25T10:01:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:49.946+0000 7f2ef5ecb640 1 -- 172.21.15.148:0/279220914 wait complete. 2024-04-25T10:01:49.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:49.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:50.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.058+0000 7fb63dc46640 1 Processor -- start 2024-04-25T10:01:50.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.058+0000 7fb63dc46640 1 -- start start 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb63dc46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6380ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb63dc46640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb638161710 0x7fb6380cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb63dc46640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63815e000 con 0x7fb6380b9330 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb63dc46640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6380d1de0 con 0x7fb638161710 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb63dc46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb638161c60 con 0x7fb63815dc20 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb637fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb6380b9330 0x7fb63806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50096/0 (socket says 172.21.15.148:50096) 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.059+0000 7fb637fff640 1 -- 172.21.15.148:0/2166596293 learned_addr learned my addr 172.21.15.148:0/2166596293 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 --2- 172.21.15.148:0/2166596293 >> v2:172.21.15.188:3300/0 conn(0x7fb638161710 0x7fb6380cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957878677 0 0) 0x7fb63815e000 con 0x7fb6380b9330 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6377fe640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6380ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb620003520 con 0x7fb6380b9330 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3163820940 0 0) 0x7fb620003520 con 0x7fb6380b9330 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 msgr2=0x7fb6380ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6380ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 >> v2:172.21.15.188:3300/0 conn(0x7fb638161710 msgr2=0x7fb6380cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 --2- 172.21.15.148:0/2166596293 >> v2:172.21.15.188:3300/0 conn(0x7fb638161710 0x7fb6380cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638161f40 con 0x7fb6380b9330 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb6300019f0 con 0x7fb6380b9330 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 >> v1:172.21.15.148:6789/0 conn(0x7fb6380b9330 legacy=0x7fb63806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6200039b0 0x7fb620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb620004800 0x7fb620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380d1de0 con 0x7fb620004800 2024-04-25T10:01:50.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638161c60 con 0x7fb6200039b0 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62000b9c0 con 0x7fb63815dc20 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb620004800 0x7fb620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb6377fe640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6200039b0 0x7fb620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb637fff640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 -- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 msgr2=0x7fb6200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 -- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6200039b0 msgr2=0x7fb620003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6200039b0 0x7fb620003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb636ffd640 1 -- 172.21.15.148:0/2166596293 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620003520 con 0x7fb620004800 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.060+0000 7fb637fff640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb63815dc20 0x7fb6200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.061+0000 7fb636ffd640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb620004800 0x7fb620004c80 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009ac0 tx=0x7fb620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c723b430d7e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.061+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c036070 con 0x7fb620004800 2024-04-25T10:01:50.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.061+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c031400 con 0x7fb620004800 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.061+0000 7fb6367fc640 1 -- 172.21.15.148:0/2166596293 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c009690 con 0x7fb620004800 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb63dc46640 1 -- 172.21.15.148:0/2166596293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb620004800 msgr2=0x7fb620004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb63dc46640 1 --2- 172.21.15.148:0/2166596293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb620004800 0x7fb620004c80 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009ac0 tx=0x7fb620005830 comp rx=0 tx=0).stop 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb637fff640 1 -- 172.21.15.148:0/2166596293 reap_dead start 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb63dc46640 1 -- 172.21.15.148:0/2166596293 shutdown_connections 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb63dc46640 1 -- 172.21.15.148:0/2166596293 >> 172.21.15.148:0/2166596293 conn(0x7fb6380d0420 msgr2=0x7fb6380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb63dc46640 1 -- 172.21.15.148:0/2166596293 shutdown_connections 2024-04-25T10:01:50.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.062+0000 7fb63dc46640 1 -- 172.21.15.148:0/2166596293 wait complete. 2024-04-25T10:01:50.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.063+0000 7fb63dc46640 1 Processor -- start 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.063+0000 7fb63dc46640 1 -- start start 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb63dc46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb620004800 0x7fb6380c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb63dc46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 0x7fb638159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb63dc46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb638157bc0 0x7fb6381f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb63dc46640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380b5e50 con 0x7fb638146a60 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb63dc46640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638162500 con 0x7fb638157bc0 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb63dc46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380b5fd0 con 0x7fb620004800 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 0x7fb638159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 0x7fb638159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43120/0 (socket says 172.21.15.148:43120) 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 -- 172.21.15.148:0/3660922283 learned_addr learned my addr 172.21.15.148:0/3660922283 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb637fff640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb638157bc0 0x7fb6381f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb6377fe640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb620004800 0x7fb6380c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 -- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb620004800 msgr2=0x7fb6380c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb620004800 0x7fb6380c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 -- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb638157bc0 msgr2=0x7fb6381f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb638157bc0 0x7fb6381f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 -- 172.21.15.148:0/3660922283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380ca2f0 con 0x7fb638146a60 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb637fff640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb638157bc0 0x7fb6381f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb636ffd640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 0x7fb638159d60 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb62c034040 tx=0x7fb62c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.064+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c040020 con 0x7fb638146a60 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.065+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c003950 con 0x7fb638146a60 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.065+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63815a9f0 con 0x7fb638146a60 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.065+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c036040 con 0x7fb638146a60 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.065+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63815e7b0 con 0x7fb638146a60 2024-04-25T10:01:50.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.065+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb638073a00 con 0x7fb638146a60 2024-04-25T10:01:50.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.066+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb62c043020 con 0x7fb638146a60 2024-04-25T10:01:50.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.066+0000 7fb61ffff640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb62406b4d0 0x7fb62406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.066+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb62c0aa190 con 0x7fb638146a60 2024-04-25T10:01:50.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.066+0000 7fb6377fe640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb62406b4d0 0x7fb62406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.067+0000 7fb6377fe640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb62406b4d0 0x7fb62406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb628009900 tx=0x7fb628008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.071+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb62c04a070 con 0x7fb638146a60 2024-04-25T10:01:50.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.239+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6380cdde0 con 0x7fb638146a60 2024-04-25T10:01:50.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.240+0000 7fb61ffff640 1 -- 172.21.15.148:0/3660922283 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb62c079260 con 0x7fb638146a60 2024-04-25T10:01:50.244 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:01:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.241+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb62406b4d0 msgr2=0x7fb62406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.241+0000 7fb63dc46640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb62406b4d0 0x7fb62406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb628009900 tx=0x7fb628008040 comp rx=0 tx=0).stop 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.241+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 msgr2=0x7fb638159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.241+0000 7fb63dc46640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 0x7fb638159d60 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb62c034040 tx=0x7fb62c004730 comp rx=0 tx=0).stop 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 shutdown_connections 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb62406b4d0 0x7fb62406d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb638157bc0 0x7fb6381f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb638146a60 0x7fb638159d60 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 --2- 172.21.15.148:0/3660922283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb620004800 0x7fb6380c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 >> 172.21.15.148:0/3660922283 conn(0x7fb6380d0420 msgr2=0x7fb6380b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 shutdown_connections 2024-04-25T10:01:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.242+0000 7fb63dc46640 1 -- 172.21.15.148:0/3660922283 wait complete. 2024-04-25T10:01:50.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:01:50.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.351+0000 7f356d51b640 1 Processor -- start 2024-04-25T10:01:50.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.351+0000 7f356d51b640 1 -- start start 2024-04-25T10:01:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f356d51b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35680cfec0 0x7f35680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f356d51b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3568071480 con 0x7f35680ce3c0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f356d51b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3568071760 con 0x7f3568154ee0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f356d51b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680ce830 con 0x7f35680cfec0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35677fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f35680ce3c0 0x7f356806d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50098/0 (socket says 172.21.15.148:50098) 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35677fe640 1 -- 172.21.15.148:0/3802517702 learned_addr learned my addr 172.21.15.148:0/3802517702 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f355ffff640 1 --2- 172.21.15.148:0/3802517702 >> v2:172.21.15.188:3300/0 conn(0x7f35680cfec0 0x7f35680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231402238 0 0) 0x7f3568071480 con 0x7f35680ce3c0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3554003520 con 0x7f35680ce3c0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900439977 0 0) 0x7f3554003520 con 0x7f35680ce3c0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 >> v2:172.21.15.188:3300/0 conn(0x7f35680cfec0 msgr2=0x7f35680cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 --2- 172.21.15.148:0/3802517702 >> v2:172.21.15.188:3300/0 conn(0x7f35680cfec0 0x7f35680cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 >> v1:172.21.15.188:6789/0 conn(0x7f3568154ee0 legacy=0x7f35680cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568071a40 con 0x7f35680ce3c0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.352+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f35600019f0 con 0x7f35680ce3c0 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 >> v1:172.21.15.148:6789/0 conn(0x7f35680ce3c0 legacy=0x7f356806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35540039b0 0x7f3554003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f3554004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35540051c0 0x7f355400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680ce830 con 0x7f3554004800 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355400c060 con 0x7f35540039b0 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355400c260 con 0x7f35540051c0 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f355ffff640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f3554004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f3566ffd640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35540039b0 0x7f3554003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35677fe640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35540051c0 0x7f355400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f355ffff640 1 -- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35540051c0 msgr2=0x7f355400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f355ffff640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35540051c0 0x7f355400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f355ffff640 1 -- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35540039b0 msgr2=0x7f3554003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f355ffff640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35540039b0 0x7f3554003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f355ffff640 1 -- 172.21.15.148:0/3802517702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554003520 con 0x7f3554004800 2024-04-25T10:01:50.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f35677fe640 1 -- 172.21.15.148:0/3802517702 reap_dead start 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.353+0000 7f3566ffd640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35540039b0 0x7f3554003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f355ffff640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f3554004c80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3550009ac0 tx=0x7f3554005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4365a785001cc72d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550036070 con 0x7f3554004800 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3550031400 con 0x7f3554004800 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f35667fc640 1 -- 172.21.15.148:0/3802517702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550009690 con 0x7f3554004800 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 -- 172.21.15.148:0/3802517702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 msgr2=0x7f3554004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f3554004c80 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3550009ac0 tx=0x7f3554005830 comp rx=0 tx=0).stop 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 -- 172.21.15.148:0/3802517702 shutdown_connections 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 --2- 172.21.15.148:0/3802517702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f3554004c80 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 -- 172.21.15.148:0/3802517702 >> 172.21.15.148:0/3802517702 conn(0x7f356806abe0 msgr2=0x7f356806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 -- 172.21.15.148:0/3802517702 shutdown_connections 2024-04-25T10:01:50.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.354+0000 7f356d51b640 1 -- 172.21.15.148:0/3802517702 wait complete. 2024-04-25T10:01:50.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 Processor -- start 2024-04-25T10:01:50.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 -- start start 2024-04-25T10:01:50.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f35681555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3568159f30 0x7f3568145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3568157090 0x7f3568157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568071230 con 0x7f3554004800 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35681552c0 con 0x7f3568157090 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f356d51b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3568155440 con 0x7f3568159f30 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f3566ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f35681555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f3566ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f35681555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43134/0 (socket says 172.21.15.148:43134) 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.356+0000 7f3566ffd640 1 -- 172.21.15.148:0/1752291096 learned_addr learned my addr 172.21.15.148:0/1752291096 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f355ffff640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3568159f30 0x7f3568145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f35677fe640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3568157090 0x7f3568157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f3566ffd640 1 -- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3568159f30 msgr2=0x7f3568145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f3566ffd640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3568159f30 0x7f3568145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f3566ffd640 1 -- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3568157090 msgr2=0x7f3568157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f3566ffd640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3568157090 0x7f3568157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f3566ffd640 1 -- 172.21.15.148:0/1752291096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568145760 con 0x7f3554004800 2024-04-25T10:01:50.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f35677fe640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3568157090 0x7f3568157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f3566ffd640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f35681555c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f355800e480 tx=0x7f355800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580052e0 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558007020 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3568155c10 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007210 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.357+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35681561d0 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.358+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3568073990 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.359+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f355801a020 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.359+0000 7f355f7fe640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f354806b500 0x7f354806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.359+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f35580894b0 con 0x7f3554004800 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.359+0000 7f355ffff640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f354806b500 0x7f354806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.359+0000 7f355ffff640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f354806b500 0x7f354806d9c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3550033000 tx=0x7f3550008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.363+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3558058580 con 0x7f3554004800 2024-04-25T10:01:50.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.533+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f356806d360 con 0x7f3554004800 2024-04-25T10:01:50.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.533+0000 7f355f7fe640 1 -- 172.21.15.148:0/1752291096 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3558057e80 con 0x7f3554004800 2024-04-25T10:01:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:50.538 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:01:50.538 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:01:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:01:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:01:50.539 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:01:50.539 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:01:50.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.535+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f354806b500 msgr2=0x7f354806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.535+0000 7f356d51b640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f354806b500 0x7f354806d9c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3550033000 tx=0x7f3550008030 comp rx=0 tx=0).stop 2024-04-25T10:01:50.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.535+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 msgr2=0x7f35681555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.535+0000 7f356d51b640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f35681555c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f355800e480 tx=0x7f355800e940 comp rx=0 tx=0).stop 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 shutdown_connections 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f354806b500 0x7f354806d9c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3568157090 0x7f3568157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3568159f30 0x7f3568145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 --2- 172.21.15.148:0/1752291096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3554004800 0x7f35681555c0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 >> 172.21.15.148:0/1752291096 conn(0x7f356806abe0 msgr2=0x7f35680cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 shutdown_connections 2024-04-25T10:01:50.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.536+0000 7f356d51b640 1 -- 172.21.15.148:0/1752291096 wait complete. 2024-04-25T10:01:50.551 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:01:50.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:01:50.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.648+0000 7fe1e7dce640 1 Processor -- start 2024-04-25T10:01:50.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.648+0000 7fe1e7dce640 1 -- start start 2024-04-25T10:01:50.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e7dce640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1e015dac0 0x7fe1e00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e7dce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe1e00d1f30 0x7fe1e00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e7dce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d34c0 0x7fe1e00d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e7dce640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e00d3030 con 0x7fe1e015dac0 2024-04-25T10:01:50.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e7dce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e00d2420 con 0x7fe1e00d1f30 2024-04-25T10:01:50.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e7dce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00d2700 con 0x7fe1e00d34c0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1e015dac0 0x7fe1e00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe1e015dac0 0x7fe1e00d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43138/0 (socket says 172.21.15.148:43138) 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 -- 172.21.15.148:0/1617756508 learned_addr learned my addr 172.21.15.148:0/1617756508 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e636b640 1 --2- 172.21.15.148:0/1617756508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d34c0 0x7fe1e00d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5369640 1 --2- 172.21.15.148:0/1617756508 >> v2:172.21.15.188:3300/0 conn(0x7fe1e00d1f30 0x7fe1e00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 -- 172.21.15.148:0/1617756508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d34c0 msgr2=0x7fe1e00d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/1617756508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d34c0 0x7fe1e00d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 -- 172.21.15.148:0/1617756508 >> v2:172.21.15.188:3300/0 conn(0x7fe1e00d1f30 msgr2=0x7fe1e00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/1617756508 >> v2:172.21.15.188:3300/0 conn(0x7fe1e00d1f30 0x7fe1e00d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.649+0000 7fe1e5b6a640 1 -- 172.21.15.148:0/1617756508 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00d29e0 con 0x7fe1e015dac0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/1617756508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00d19f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00d2910 tx=0x7fe1dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf26dadfc33f0814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e4b68640 1 -- 172.21.15.148:0/1617756508 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc037070 con 0x7fe1e015dac0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e4b68640 1 -- 172.21.15.148:0/1617756508 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc032440 con 0x7fe1e015dac0 2024-04-25T10:01:50.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e4b68640 1 -- 172.21.15.148:0/1617756508 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc032810 con 0x7fe1e015dac0 2024-04-25T10:01:50.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e7dce640 1 -- 172.21.15.148:0/1617756508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 msgr2=0x7fe1e00d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/1617756508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00d19f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00d2910 tx=0x7fe1dc02a270 comp rx=0 tx=0).stop 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e7dce640 1 -- 172.21.15.148:0/1617756508 shutdown_connections 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.650+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/1617756508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d34c0 0x7fe1e00d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.651+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/1617756508 >> v2:172.21.15.188:3300/0 conn(0x7fe1e00d1f30 0x7fe1e00d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.651+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/1617756508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00d19f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.651+0000 7fe1e7dce640 1 -- 172.21.15.148:0/1617756508 >> 172.21.15.148:0/1617756508 conn(0x7fe1e0061c30 msgr2=0x7fe1e006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.651+0000 7fe1e7dce640 1 -- 172.21.15.148:0/1617756508 shutdown_connections 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.651+0000 7fe1e7dce640 1 -- 172.21.15.148:0/1617756508 wait complete. 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 Processor -- start 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 -- start start 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d1f30 0x7fe1e00cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e00d34c0 0x7fe1e0143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00b97d0 con 0x7fe1e015dac0 2024-04-25T10:01:50.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e015dea0 con 0x7fe1e00d34c0 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.652+0000 7fe1e7dce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00b9950 con 0x7fe1e00d1f30 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43152/0 (socket says 172.21.15.148:43152) 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 -- 172.21.15.148:0/4089664920 learned_addr learned my addr 172.21.15.148:0/4089664920 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d1f30 0x7fe1e00cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e5369640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e00d34c0 0x7fe1e0143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 -- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d1f30 msgr2=0x7fe1e00cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d1f30 0x7fe1e00cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 -- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e00d34c0 msgr2=0x7fe1e0143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e00d34c0 0x7fe1e0143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 -- 172.21.15.148:0/4089664920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00cd9c0 con 0x7fe1e015dac0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d1f30 0x7fe1e00cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e5369640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e00d34c0 0x7fe1e0143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e636b640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00caaa0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007e40 tx=0x7fe1d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8014620 con 0x7fe1e015dac0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d8019070 con 0x7fe1e015dac0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8014a20 con 0x7fe1e015dac0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e00b45b0 con 0x7fe1e015dac0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.653+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e00b4b70 con 0x7fe1e015dac0 2024-04-25T10:01:50.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.654+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e0073a00 con 0x7fe1e015dac0 2024-04-25T10:01:50.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.655+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1d8014bc0 con 0x7fe1e015dac0 2024-04-25T10:01:50.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.655+0000 7fe1d67fc640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe1c406b4d0 0x7fe1c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.655+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe1d808a820 con 0x7fe1e015dac0 2024-04-25T10:01:50.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.655+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe1c406b4d0 0x7fe1c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.656+0000 7fe1e5b6a640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe1c406b4d0 0x7fe1c406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc034000 tx=0x7fe1dc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.659+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1d80598f0 con 0x7fe1e015dac0 2024-04-25T10:01:50.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:50.830+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe1e00c84c0 con 0x7fe1e015dac0 2024-04-25T10:01:51.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.277+0000 7fe1d67fc640 1 -- 172.21.15.148:0/4089664920 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe1d80591f0 con 0x7fe1e015dac0 2024-04-25T10:01:51.282 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe1c406b4d0 msgr2=0x7fe1c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe1c406b4d0 0x7fe1c406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc034000 tx=0x7fe1dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 msgr2=0x7fe1e00caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00caaa0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007e40 tx=0x7fe1d800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 shutdown_connections 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe1c406b4d0 0x7fe1c406d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1e015dac0 0x7fe1e00caaa0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1e00d34c0 0x7fe1e0143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 --2- 172.21.15.148:0/4089664920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1e00d1f30 0x7fe1e00cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 >> 172.21.15.148:0/4089664920 conn(0x7fe1e0061c30 msgr2=0x7fe1e00ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 shutdown_connections 2024-04-25T10:01:51.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.279+0000 7fe1e7dce640 1 -- 172.21.15.148:0/4089664920 wait complete. 2024-04-25T10:01:51.298 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:01:51.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.400+0000 7f9efa3d5640 1 Processor -- start 2024-04-25T10:01:51.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.400+0000 7f9efa3d5640 1 -- start start 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9efa3d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ef41454b0 0x7f9ef4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9efa3d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ef4071300 0x7f9ef40b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9efa3d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40ceba0 con 0x7f9ef4144d20 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9efa3d5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef40b4830 con 0x7f9ef41454b0 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9efa3d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef4071730 con 0x7f9ef4071300 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ef41454b0 0x7f9ef4070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9ef41454b0 0x7f9ef4070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43154/0 (socket says 172.21.15.148:43154) 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 -- 172.21.15.148:0/2235541272 learned_addr learned my addr 172.21.15.148:0/2235541272 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef8972640 1 --2- 172.21.15.148:0/2235541272 >> v2:172.21.15.188:3300/0 conn(0x7f9ef4071300 0x7f9ef40b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 -- 172.21.15.148:0/2235541272 >> v2:172.21.15.188:3300/0 conn(0x7f9ef4071300 msgr2=0x7f9ef40b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 --2- 172.21.15.148:0/2235541272 >> v2:172.21.15.188:3300/0 conn(0x7f9ef4071300 0x7f9ef40b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 -- 172.21.15.148:0/2235541272 >> v1:172.21.15.188:6789/0 conn(0x7f9ef4144d20 legacy=0x7f9ef40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:51.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.401+0000 7f9ef37fe640 1 -- 172.21.15.148:0/2235541272 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40cee80 con 0x7f9ef41454b0 2024-04-25T10:01:51.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.402+0000 7f9ef37fe640 1 --2- 172.21.15.148:0/2235541272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef41454b0 0x7f9ef4070dc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f9ef406bcc0 tx=0x7f9ee802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d30d7ae2eb446d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.402+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/2235541272 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8037070 con 0x7f9ef41454b0 2024-04-25T10:01:51.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.402+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/2235541272 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8032440 con 0x7f9ef41454b0 2024-04-25T10:01:51.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.402+0000 7f9ef2ffd640 1 -- 172.21.15.148:0/2235541272 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8032810 con 0x7f9ef41454b0 2024-04-25T10:01:51.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 -- 172.21.15.148:0/2235541272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef41454b0 msgr2=0x7f9ef4070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/2235541272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef41454b0 0x7f9ef4070dc0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f9ef406bcc0 tx=0x7f9ee802a230 comp rx=0 tx=0).stop 2024-04-25T10:01:51.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 -- 172.21.15.148:0/2235541272 shutdown_connections 2024-04-25T10:01:51.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/2235541272 >> v2:172.21.15.188:3300/0 conn(0x7f9ef4071300 0x7f9ef40b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/2235541272 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef41454b0 0x7f9ef4070dc0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 -- 172.21.15.148:0/2235541272 >> 172.21.15.148:0/2235541272 conn(0x7f9ef40b3530 msgr2=0x7f9ef40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 -- 172.21.15.148:0/2235541272 shutdown_connections 2024-04-25T10:01:51.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.403+0000 7f9efa3d5640 1 -- 172.21.15.148:0/2235541272 wait complete. 2024-04-25T10:01:51.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.404+0000 7f9efa3d5640 1 Processor -- start 2024-04-25T10:01:51.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 -- start start 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef4071300 0x7f9ef40c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 0x7f9ef4155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef41454b0 0x7f9ef413d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4145100 con 0x7f9ef4144d20 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40cf440 con 0x7f9ef41454b0 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9efa3d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4145280 con 0x7f9ef4071300 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9ef37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 0x7f9ef4155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9ef37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 0x7f9ef4155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43160/0 (socket says 172.21.15.148:43160) 2024-04-25T10:01:51.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.405+0000 7f9ef37fe640 1 -- 172.21.15.148:0/1285577712 learned_addr learned my addr 172.21.15.148:0/1285577712 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef37fe640 1 -- 172.21.15.148:0/1285577712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef4071300 msgr2=0x7f9ef40c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef37fe640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef4071300 0x7f9ef40c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef37fe640 1 -- 172.21.15.148:0/1285577712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef41454b0 msgr2=0x7f9ef413d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef37fe640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef41454b0 0x7f9ef413d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef37fe640 1 -- 172.21.15.148:0/1285577712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40c7b60 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef37fe640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 0x7f9ef4155610 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8000c00 tx=0x7f9ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8041020 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8030970 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8037040 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40c7e40 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.406+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef4153050 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.407+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef40b0f50 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.407+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ee8044020 con 0x7f9ef4144d20 2024-04-25T10:01:51.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.408+0000 7f9ef0ff9640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ee006b4d0 0x7f9ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:51.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.408+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f9ee80aabb0 con 0x7f9ef4144d20 2024-04-25T10:01:51.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.408+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ee006b4d0 0x7f9ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:51.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.408+0000 7f9ef3fff640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ee006b4d0 0x7f9ee006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009650 tx=0x7f9ee4006ba0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:51.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.412+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ee804b030 con 0x7f9ef4144d20 2024-04-25T10:01:51.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:51.583+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9ef40b4200 con 0x7f9ef4144d20 2024-04-25T10:01:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.279+0000 7f9ef0ff9640 1 -- 172.21.15.148:0/1285577712 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9ee8079a80 con 0x7f9ef4144d20 2024-04-25T10:01:52.284 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:01:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ee006b4d0 msgr2=0x7f9ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ee006b4d0 0x7f9ee006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009650 tx=0x7f9ee4006ba0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 msgr2=0x7f9ef4155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 0x7f9ef4155610 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8000c00 tx=0x7f9ee80046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 shutdown_connections 2024-04-25T10:01:52.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ee006b4d0 0x7f9ee006d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef41454b0 0x7f9ef413d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4144d20 0x7f9ef4155610 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 --2- 172.21.15.148:0/1285577712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ef4071300 0x7f9ef40c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 >> 172.21.15.148:0/1285577712 conn(0x7f9ef40b3530 msgr2=0x7f9ef40b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 shutdown_connections 2024-04-25T10:01:52.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.281+0000 7f9efa3d5640 1 -- 172.21.15.148:0/1285577712 wait complete. 2024-04-25T10:01:52.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:01:52.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.416+0000 7f50ee7c4640 1 Processor -- start 2024-04-25T10:01:52.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.416+0000 7f50ee7c4640 1 -- start start 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50ee7c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50e806ccd0 0x7f50e80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50ee7c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80d1770 con 0x7f50e815deb0 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50ee7c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e8162740 con 0x7f50e8070dc0 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50ee7c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e81622b0 con 0x7f50e806ccd0 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50e77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50e806ccd0 0x7f50e80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50ecd61640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f50e815deb0 0x7f50e815e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50112/0 (socket says 172.21.15.148:50112) 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50e77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f50e806ccd0 0x7f50e80b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43166/0 (socket says 172.21.15.148:43166) 2024-04-25T10:01:52.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50ecd61640 1 -- 172.21.15.148:0/2505358602 learned_addr learned my addr 172.21.15.148:0/2505358602 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:52.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50e77fe640 1 -- 172.21.15.148:0/2505358602 >> v1:172.21.15.188:6789/0 conn(0x7f50e8070dc0 legacy=0x7f50e80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:52.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50e77fe640 1 -- 172.21.15.148:0/2505358602 >> v1:172.21.15.148:6789/0 conn(0x7f50e815deb0 legacy=0x7f50e815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.417+0000 7f50e77fe640 1 -- 172.21.15.148:0/2505358602 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8161980 con 0x7f50e806ccd0 2024-04-25T10:01:52.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.418+0000 7f50e77fe640 1 --2- 172.21.15.148:0/2505358602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e806ccd0 0x7f50e80b07a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0097a0 tx=0x7f50dc02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5590b2bbc782a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.418+0000 7f50e6ffd640 1 -- 172.21.15.148:0/2505358602 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc037070 con 0x7f50e806ccd0 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.418+0000 7f50e6ffd640 1 -- 172.21.15.148:0/2505358602 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc02adc0 con 0x7f50e806ccd0 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.418+0000 7f50e6ffd640 1 -- 172.21.15.148:0/2505358602 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc032660 con 0x7f50e806ccd0 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2505358602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e806ccd0 msgr2=0x7f50e80b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2505358602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e806ccd0 0x7f50e80b07a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0097a0 tx=0x7f50dc02a0b0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2505358602 shutdown_connections 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2505358602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e806ccd0 0x7f50e80b07a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2505358602 >> 172.21.15.148:0/2505358602 conn(0x7f50e80b3530 msgr2=0x7f50e80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2505358602 shutdown_connections 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.419+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2505358602 wait complete. 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.420+0000 7f50ee7c4640 1 Processor -- start 2024-04-25T10:01:52.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.420+0000 7f50ee7c4640 1 -- start start 2024-04-25T10:01:52.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ee7c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e806ccd0 0x7f50e814b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ee7c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 0x7f50e80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ee7c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e815deb0 0x7f50e80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ee7c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e8161f40 con 0x7f50e8070dc0 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ee7c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80b10f0 con 0x7f50e806ccd0 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ee7c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80b5840 con 0x7f50e815deb0 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 0x7f50e80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 0x7f50e80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43174/0 (socket says 172.21.15.148:43174) 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 -- 172.21.15.148:0/2232471228 learned_addr learned my addr 172.21.15.148:0/2232471228 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e7fff640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e806ccd0 0x7f50e814b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ecd61640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e815deb0 0x7f50e80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 -- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e815deb0 msgr2=0x7f50e80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e815deb0 0x7f50e80cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 -- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e806ccd0 msgr2=0x7f50e814b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e806ccd0 0x7f50e814b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 -- 172.21.15.148:0/2232471228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e80cac50 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50ecd61640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e815deb0 0x7f50e80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.421+0000 7f50e77fe640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 0x7f50e80ccff0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f50dc02a0f0 tx=0x7f50dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.422+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc037040 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.422+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc042070 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.422+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e815f010 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.422+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc03b870 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.422+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e815f5d0 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.423+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e8073a00 con 0x7f50e8070dc0 2024-04-25T10:01:52.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.423+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50dc03ba10 con 0x7f50e8070dc0 2024-04-25T10:01:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.424+0000 7f50e4ff9640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50d406b4d0 0x7f50d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.424+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f50dc0aa950 con 0x7f50e8070dc0 2024-04-25T10:01:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.424+0000 7f50e7fff640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50d406b4d0 0x7f50d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.424+0000 7f50e7fff640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50d406b4d0 0x7f50d406d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f50d8004f60 tx=0x7f50d8007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.428+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50dc079730 con 0x7f50e8070dc0 2024-04-25T10:01:52.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:52.634+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f50e80ce580 con 0x7f50e8070dc0 2024-04-25T10:01:53.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.302+0000 7f50e4ff9640 1 -- 172.21.15.148:0/2232471228 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f50dc079030 con 0x7f50e8070dc0 2024-04-25T10:01:53.308 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.312+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50d406b4d0 msgr2=0x7f50d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.312+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50d406b4d0 0x7f50d406d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f50d8004f60 tx=0x7f50d8007e30 comp rx=0 tx=0).stop 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.312+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 msgr2=0x7f50e80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.312+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 0x7f50e80ccff0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f50dc02a0f0 tx=0x7f50dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 shutdown_connections 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50d406b4d0 0x7f50d406d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50e815deb0 0x7f50e80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50e8070dc0 0x7f50e80ccff0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 --2- 172.21.15.148:0/2232471228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50e806ccd0 0x7f50e814b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 >> 172.21.15.148:0/2232471228 conn(0x7f50e80b3530 msgr2=0x7f50e80cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.313+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 shutdown_connections 2024-04-25T10:01:53.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.314+0000 7f50ee7c4640 1 -- 172.21.15.148:0/2232471228 wait complete. 2024-04-25T10:01:53.339 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:53.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.434+0000 7fe478e46640 1 Processor -- start 2024-04-25T10:01:53.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.434+0000 7fe478e46640 1 -- start start 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe478e46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 0x7fe4740d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe478e46640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4740b92a0 0x7fe4740d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe478e46640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4740b97a0 con 0x7fe4740b2cb0 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe478e46640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47415dfc0 con 0x7fe4740b92a0 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe478e46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740d1a00 con 0x7fe47415dbe0 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe471d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe4740b2cb0 0x7fe4740b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:50120/0 (socket says 172.21.15.148:50120) 2024-04-25T10:01:53.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe471d9b640 1 -- 172.21.15.148:0/1681380505 learned_addr learned my addr 172.21.15.148:0/1681380505 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360634685 0 0) 0x7fe4740b97a0 con 0x7fe4740b2cb0 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47259c640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 0x7fe4740d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe472d9d640 1 --2- 172.21.15.148:0/1681380505 >> v2:172.21.15.188:3300/0 conn(0x7fe4740b92a0 0x7fe4740d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe45c003520 con 0x7fe4740b2cb0 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 830540789 0 0) 0x7fe45c003520 con 0x7fe4740b2cb0 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 msgr2=0x7fe4740d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 0x7fe4740d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 >> v2:172.21.15.188:3300/0 conn(0x7fe4740b92a0 msgr2=0x7fe4740d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 --2- 172.21.15.148:0/1681380505 >> v2:172.21.15.188:3300/0 conn(0x7fe4740b92a0 0x7fe4740d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740d1c50 con 0x7fe4740b2cb0 2024-04-25T10:01:53.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe468001670 con 0x7fe4740b2cb0 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 >> v1:172.21.15.148:6789/0 conn(0x7fe4740b2cb0 legacy=0x7fe4740b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c0039b0 0x7fe45c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 0x7fe45c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c004d40 0x7fe45c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740d1a00 con 0x7fe45c0039b0 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c00b9c0 con 0x7fe45c004d40 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c00bb40 con 0x7fe47415dbe0 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.435+0000 7fe47259c640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c0039b0 0x7fe45c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe471d9b640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 0x7fe45c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe472d9d640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c004d40 0x7fe45c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47259c640 1 -- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 msgr2=0x7fe45c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47259c640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415dbe0 0x7fe45c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47259c640 1 -- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c004d40 msgr2=0x7fe45c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47259c640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c004d40 0x7fe45c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47259c640 1 -- 172.21.15.148:0/1681380505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c003520 con 0x7fe45c0039b0 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe472d9d640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c004d40 0x7fe45c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47259c640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c0039b0 0x7fe45c003dd0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009ac0 tx=0x7fe45c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c8fa0c6e0f1353 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.436+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c036070 con 0x7fe45c0039b0 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c031400 con 0x7fe45c0039b0 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe47159a640 1 -- 172.21.15.148:0/1681380505 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c009690 con 0x7fe45c0039b0 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe478e46640 1 -- 172.21.15.148:0/1681380505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c0039b0 msgr2=0x7fe45c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe478e46640 1 --2- 172.21.15.148:0/1681380505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe45c0039b0 0x7fe45c003dd0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009ac0 tx=0x7fe45c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe472d9d640 1 -- 172.21.15.148:0/1681380505 reap_dead start 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe478e46640 1 -- 172.21.15.148:0/1681380505 shutdown_connections 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe478e46640 1 -- 172.21.15.148:0/1681380505 >> 172.21.15.148:0/1681380505 conn(0x7fe474061a20 msgr2=0x7fe474061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe478e46640 1 -- 172.21.15.148:0/1681380505 shutdown_connections 2024-04-25T10:01:53.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.437+0000 7fe478e46640 1 -- 172.21.15.148:0/1681380505 wait complete. 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.438+0000 7fe478e46640 1 Processor -- start 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 -- start start 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c0039b0 0x7fe47414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 0x7fe47415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415cad0 0x7fe4740caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b6ab0 con 0x7fe4740cc6d0 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740d2210 con 0x7fe45c0039b0 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe478e46640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b6c30 con 0x7fe47415cad0 2024-04-25T10:01:53.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe471d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 0x7fe47415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe471d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 0x7fe47415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43182/0 (socket says 172.21.15.148:43182) 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe471d9b640 1 -- 172.21.15.148:0/1983706716 learned_addr learned my addr 172.21.15.148:0/1983706716 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe472d9d640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415cad0 0x7fe4740caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.439+0000 7fe47259c640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c0039b0 0x7fe47414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe471d9b640 1 -- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415cad0 msgr2=0x7fe4740caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe471d9b640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415cad0 0x7fe4740caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe471d9b640 1 -- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c0039b0 msgr2=0x7fe47414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe471d9b640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c0039b0 0x7fe47414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe471d9b640 1 -- 172.21.15.148:0/1983706716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740cb630 con 0x7fe4740cc6d0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe47259c640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c0039b0 0x7fe47414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe471d9b640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 0x7fe47415c590 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe468007340 tx=0x7fe468007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800f8e0 con 0x7fe4740cc6d0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe468018070 con 0x7fe4740cc6d0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800fcc0 con 0x7fe4740cc6d0 2024-04-25T10:01:53.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4740cb910 con 0x7fe4740cc6d0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.440+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4740cbed0 con 0x7fe4740cc6d0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.441+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe474073990 con 0x7fe4740cc6d0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.441+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe46801d460 con 0x7fe4740cc6d0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.442+0000 7fe462ffd640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe45406b4d0 0x7fe45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.442+0000 7fe47259c640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe45406b4d0 0x7fe45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.442+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe46808a460 con 0x7fe4740cc6d0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.442+0000 7fe47259c640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe45406b4d0 0x7fe45406d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe46c033000 tx=0x7fe46c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.446+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe46800fe60 con 0x7fe4740cc6d0 2024-04-25T10:01:53.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.621+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe47415ceb0 con 0x7fe4740cc6d0 2024-04-25T10:01:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.622+0000 7fe462ffd640 1 -- 172.21.15.148:0/1983706716 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17013 (secure 0 0 0) 0x7fe468059150 con 0x7fe4740cc6d0 2024-04-25T10:01:53.628 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:53.628 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":120,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:52.636877+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe45406b4d0 msgr2=0x7fe45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe45406b4d0 0x7fe45406d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe46c033000 tx=0x7fe46c008030 comp rx=0 tx=0).stop 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 msgr2=0x7fe47415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 0x7fe47415c590 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe468007340 tx=0x7fe468007800 comp rx=0 tx=0).stop 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 shutdown_connections 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe45406b4d0 0x7fe45406d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47415cad0 0x7fe4740caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4740cc6d0 0x7fe47415c590 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 --2- 172.21.15.148:0/1983706716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c0039b0 0x7fe47414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.624+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 >> 172.21.15.148:0/1983706716 conn(0x7fe474061a20 msgr2=0x7fe47415ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.625+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 shutdown_connections 2024-04-25T10:01:53.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.625+0000 7fe478e46640 1 -- 172.21.15.148:0/1983706716 wait complete. 2024-04-25T10:01:53.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:53.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.734+0000 7f7ee4c6b640 1 Processor -- start 2024-04-25T10:01:53.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.734+0000 7f7ee4c6b640 1 -- start start 2024-04-25T10:01:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ee4c6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee015b9c0 0x7f7ee015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ee4c6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ee006abe0 0x7f7ee006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ee4c6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee00cfa60 con 0x7f7ee0072620 2024-04-25T10:01:53.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ee4c6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee00731a0 con 0x7f7ee006abe0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ee4c6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee006b580 con 0x7f7ee015b9c0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ed7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ee006abe0 0x7f7ee006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7eded9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7ee0072620 0x7f7ee0072a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42936/0 (socket says 172.21.15.148:42936) 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7eded9d640 1 -- 172.21.15.148:0/174589429 learned_addr learned my addr 172.21.15.148:0/174589429 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ede59c640 1 --2- 172.21.15.148:0/174589429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee015b9c0 0x7f7ee015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ed7fff640 1 -- 172.21.15.148:0/174589429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee015b9c0 msgr2=0x7f7ee015bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ed7fff640 1 --2- 172.21.15.148:0/174589429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee015b9c0 0x7f7ee015bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ed7fff640 1 -- 172.21.15.148:0/174589429 >> v1:172.21.15.148:6789/0 conn(0x7f7ee0072620 legacy=0x7f7ee0072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7ed7fff640 1 -- 172.21.15.148:0/174589429 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00cfd40 con 0x7f7ee006abe0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.735+0000 7f7eddd9b640 1 -- 172.21.15.148:0/174589429 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149133548 0 0) 0x7f7ee00cfa60 con 0x7f7ee0072620 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ed7fff640 1 --2- 172.21.15.148:0/174589429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee006abe0 0x7f7ee006b040 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097a0 tx=0x7f7ec802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824fedda91f09a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7eddd9b640 1 -- 172.21.15.148:0/174589429 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8037070 con 0x7f7ee006abe0 2024-04-25T10:01:53.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7eddd9b640 1 -- 172.21.15.148:0/174589429 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8032440 con 0x7f7ee006abe0 2024-04-25T10:01:53.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7eddd9b640 1 -- 172.21.15.148:0/174589429 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8032810 con 0x7f7ee006abe0 2024-04-25T10:01:53.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/174589429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee006abe0 msgr2=0x7f7ee006b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/174589429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee006abe0 0x7f7ee006b040 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7ec80097a0 tx=0x7f7ec802a180 comp rx=0 tx=0).stop 2024-04-25T10:01:53.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/174589429 shutdown_connections 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/174589429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee006abe0 0x7f7ee006b040 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/174589429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee015b9c0 0x7f7ee015bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/174589429 >> 172.21.15.148:0/174589429 conn(0x7f7ee00b2370 msgr2=0x7f7ee00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.736+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/174589429 shutdown_connections 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.737+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/174589429 wait complete. 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 Processor -- start 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 -- start start 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee006abe0 0x7f7ee0146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee0072620 0x7f7ee0157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 0x7f7ee01e5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00cf710 con 0x7f7ee015b9c0 2024-04-25T10:01:53.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00d0300 con 0x7f7ee006abe0 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.738+0000 7f7ee4c6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee015c2e0 con 0x7f7ee0072620 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 0x7f7ee01e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 0x7f7ee01e5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42672/0 (socket says 172.21.15.148:42672) 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 -- 172.21.15.148:0/4237769647 learned_addr learned my addr 172.21.15.148:0/4237769647 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ed7fff640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee0072620 0x7f7ee0157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ede59c640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee006abe0 0x7f7ee0146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 -- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee0072620 msgr2=0x7f7ee0157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee0072620 0x7f7ee0157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 -- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee006abe0 msgr2=0x7f7ee0146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee006abe0 0x7f7ee0146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 -- 172.21.15.148:0/4237769647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0146640 con 0x7f7ee015b9c0 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ed7fff640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee0072620 0x7f7ee0157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ede59c640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee006abe0 0x7f7ee0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:53.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7eded9d640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 0x7f7ee01e5720 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c7d0 tx=0x7f7ed800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8018070 con 0x7f7ee015b9c0 2024-04-25T10:01:53.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed800a640 con 0x7f7ee015b9c0 2024-04-25T10:01:53.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800aa70 con 0x7f7ee015b9c0 2024-04-25T10:01:53.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee0157ec0 con 0x7f7ee015b9c0 2024-04-25T10:01:53.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.739+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee0158480 con 0x7f7ee015b9c0 2024-04-25T10:01:53.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.740+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee0073990 con 0x7f7ee015b9c0 2024-04-25T10:01:53.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.741+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ed801b070 con 0x7f7ee015b9c0 2024-04-25T10:01:53.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.741+0000 7f7ed6ffd640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ec006b500 0x7f7ec006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.741+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ed8089920 con 0x7f7ee015b9c0 2024-04-25T10:01:53.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.741+0000 7f7ede59c640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ec006b500 0x7f7ec006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.741+0000 7f7ede59c640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ec006b500 0x7f7ec006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0006fd0 tx=0x7f7ed0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.745+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ed8058610 con 0x7f7ee015b9c0 2024-04-25T10:01:53.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.919+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ee015a6f0 con 0x7f7ee015b9c0 2024-04-25T10:01:53.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.920+0000 7f7ed6ffd640 1 -- 172.21.15.148:0/4237769647 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+17013 (secure 0 0 0) 0x7f7ed8057f10 con 0x7f7ee015b9c0 2024-04-25T10:01:53.926 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:53.926 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":120,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:52.636877+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.922+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ec006b500 msgr2=0x7f7ec006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.922+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ec006b500 0x7f7ec006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0006fd0 tx=0x7f7ed0008040 comp rx=0 tx=0).stop 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.922+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 msgr2=0x7f7ee01e5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.922+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 0x7f7ee01e5720 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c7d0 tx=0x7f7ed800cc90 comp rx=0 tx=0).stop 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 shutdown_connections 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ec006b500 0x7f7ec006d9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee015b9c0 0x7f7ee01e5720 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ee0072620 0x7f7ee0157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 --2- 172.21.15.148:0/4237769647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ee006abe0 0x7f7ee0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 >> 172.21.15.148:0/4237769647 conn(0x7f7ee00b2370 msgr2=0x7f7ee01597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 shutdown_connections 2024-04-25T10:01:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:53.923+0000 7f7ee4c6b640 1 -- 172.21.15.148:0/4237769647 wait complete. 2024-04-25T10:01:53.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:01:54.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.035+0000 7fa88579e640 1 Processor -- start 2024-04-25T10:01:54.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.035+0000 7fa88579e640 1 -- start start 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa88579e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa88015de80 0x7fa8800cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa88579e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa88579e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8800cf950 0x7fa88006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa88579e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800afc80 con 0x7fa8800cf950 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa88579e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800ce4d0 con 0x7fa88015de80 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa88579e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800ce7b0 con 0x7fa8800cefb0 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa87f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8800cf950 0x7fa88006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa87f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa8800cf950 0x7fa88006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42680/0 (socket says 172.21.15.148:42680) 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa87f7fe640 1 -- 172.21.15.148:0/3851455173 learned_addr learned my addr 172.21.15.148:0/3851455173 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:54.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa87e7fc640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.036+0000 7fa87effd640 1 --2- 172.21.15.148:0/3851455173 >> v2:172.21.15.188:3300/0 conn(0x7fa88015de80 0x7fa8800cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87f7fe640 1 -- 172.21.15.148:0/3851455173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 msgr2=0x7fa8800cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87f7fe640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87f7fe640 1 -- 172.21.15.148:0/3851455173 >> v2:172.21.15.188:3300/0 conn(0x7fa88015de80 msgr2=0x7fa8800cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87f7fe640 1 --2- 172.21.15.148:0/3851455173 >> v2:172.21.15.188:3300/0 conn(0x7fa88015de80 0x7fa8800cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87f7fe640 1 -- 172.21.15.148:0/3851455173 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800cea90 con 0x7fa8800cf950 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87effd640 1 --2- 172.21.15.148:0/3851455173 >> v2:172.21.15.188:3300/0 conn(0x7fa88015de80 0x7fa8800cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87e7fc640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87f7fe640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800cf950 0x7fa88006abe0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa8800d1800 tx=0x7fa87802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb2a60ea0c4d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87dffb640 1 -- 172.21.15.148:0/3851455173 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878037070 con 0x7fa8800cf950 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87dffb640 1 -- 172.21.15.148:0/3851455173 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa878032440 con 0x7fa8800cf950 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.037+0000 7fa87dffb640 1 -- 172.21.15.148:0/3851455173 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878032810 con 0x7fa8800cf950 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 -- 172.21.15.148:0/3851455173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800cf950 msgr2=0x7fa88006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800cf950 0x7fa88006abe0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa8800d1800 tx=0x7fa87802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 -- 172.21.15.148:0/3851455173 shutdown_connections 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8800cf950 0x7fa88006abe0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 --2- 172.21.15.148:0/3851455173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 --2- 172.21.15.148:0/3851455173 >> v2:172.21.15.188:3300/0 conn(0x7fa88015de80 0x7fa8800cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 -- 172.21.15.148:0/3851455173 >> 172.21.15.148:0/3851455173 conn(0x7fa8800b2390 msgr2=0x7fa8800b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 -- 172.21.15.148:0/3851455173 shutdown_connections 2024-04-25T10:01:54.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.038+0000 7fa88579e640 1 -- 172.21.15.148:0/3851455173 wait complete. 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.039+0000 7fa88579e640 1 Processor -- start 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.039+0000 7fa88579e640 1 -- start start 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa88579e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa88579e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8800cf950 0x7fa8800b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa88579e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 0x7fa8801f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa88579e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800b3030 con 0x7fa88015de80 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa88579e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800b2eb0 con 0x7fa8800cf950 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa88579e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800c5600 con 0x7fa8800cefb0 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 0x7fa8801f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 0x7fa8801f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42694/0 (socket says 172.21.15.148:42694) 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 -- 172.21.15.148:0/1390659884 learned_addr learned my addr 172.21.15.148:0/1390659884 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87effd640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87e7fc640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8800cf950 0x7fa8800b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 -- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 msgr2=0x7fa8800cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 -- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8800cf950 msgr2=0x7fa8800b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8800cf950 0x7fa8800b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.040+0000 7fa87f7fe640 1 -- 172.21.15.148:0/1390659884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800b5f10 con 0x7fa88015de80 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.041+0000 7fa87f7fe640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 0x7fa8801f7060 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa87802a220 tx=0x7fa8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.041+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878041070 con 0x7fa88015de80 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.041+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87803c870 con 0x7fa88015de80 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.041+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878037040 con 0x7fa88015de80 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.041+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8800b61f0 con 0x7fa88015de80 2024-04-25T10:01:54.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.041+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8800c8980 con 0x7fa88015de80 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.042+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88015d6a0 con 0x7fa88015de80 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.042+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa878044070 con 0x7fa88015de80 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.043+0000 7fa8677fe640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa86c06b4d0 0x7fa86c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.043+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa8780aaf70 con 0x7fa88015de80 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.043+0000 7fa87effd640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa86c06b4d0 0x7fa86c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.043+0000 7fa87effd640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa86c06b4d0 0x7fa86c06d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa8800cf5f0 tx=0x7fa87000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.047+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa878079e10 con 0x7fa88015de80 2024-04-25T10:01:54.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.256+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa8800739e0 con 0x7fa88015de80 2024-04-25T10:01:54.760 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:54.763 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:54.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.762+0000 7fa8677fe640 1 -- 172.21.15.148:0/1390659884 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa878079710 con 0x7fa88015de80 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.764+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa86c06b4d0 msgr2=0x7fa86c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.764+0000 7fa88579e640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa86c06b4d0 0x7fa86c06d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa8800cf5f0 tx=0x7fa87000b040 comp rx=0 tx=0).stop 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.764+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 msgr2=0x7fa8801f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.764+0000 7fa88579e640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 0x7fa8801f7060 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fa87802a220 tx=0x7fa8780046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 shutdown_connections 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa86c06b4d0 0x7fa86c06d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa88015de80 0x7fa8801f7060 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8800cf950 0x7fa8800b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 --2- 172.21.15.148:0/1390659884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8800cefb0 0x7fa8800cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 >> 172.21.15.148:0/1390659884 conn(0x7fa8800b2390 msgr2=0x7fa88006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 shutdown_connections 2024-04-25T10:01:54.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.765+0000 7fa88579e640 1 -- 172.21.15.148:0/1390659884 wait complete. 2024-04-25T10:01:54.786 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:01:54.787 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:01:54.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.924+0000 7f54e9977640 1 Processor -- start 2024-04-25T10:01:54.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.924+0000 7f54e9977640 1 -- start start 2024-04-25T10:01:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e9977640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f54e415b7d0 0x7f54e40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e9977640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54e406c9e0 0x7f54e40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e9977640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e406cf50 con 0x7f54e40b5620 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e9977640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40ce530 con 0x7f54e406c9e0 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e9977640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40ce810 con 0x7f54e415b7d0 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54e406c9e0 0x7f54e40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54e406c9e0 0x7f54e40b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42706/0 (socket says 172.21.15.148:42706) 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.925+0000 7f54e37fe640 1 -- 172.21.15.148:0/2665015818 learned_addr learned my addr 172.21.15.148:0/2665015818 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.926+0000 7f54e37fe640 1 -- 172.21.15.148:0/2665015818 >> v2:172.21.15.188:3300/0 conn(0x7f54e415b7d0 msgr2=0x7f54e40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.926+0000 7f54e37fe640 1 --2- 172.21.15.148:0/2665015818 >> v2:172.21.15.188:3300/0 conn(0x7f54e415b7d0 0x7f54e40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.926+0000 7f54e37fe640 1 -- 172.21.15.148:0/2665015818 >> v1:172.21.15.188:6789/0 conn(0x7f54e40b5620 legacy=0x7f54e406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.926+0000 7f54e37fe640 1 -- 172.21.15.148:0/2665015818 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40ceaf0 con 0x7f54e406c9e0 2024-04-25T10:01:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.926+0000 7f54e37fe640 1 --2- 172.21.15.148:0/2665015818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e406c9e0 0x7f54e40b5ab0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f54e408c0d0 tx=0x7f54dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeb4d7d90acf239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e1ffb640 1 -- 172.21.15.148:0/2665015818 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc036070 con 0x7f54e406c9e0 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e1ffb640 1 -- 172.21.15.148:0/2665015818 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc031600 con 0x7f54e406c9e0 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e1ffb640 1 -- 172.21.15.148:0/2665015818 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0319d0 con 0x7f54e406c9e0 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 -- 172.21.15.148:0/2665015818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e406c9e0 msgr2=0x7f54e40b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 --2- 172.21.15.148:0/2665015818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e406c9e0 0x7f54e40b5ab0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f54e408c0d0 tx=0x7f54dc02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 -- 172.21.15.148:0/2665015818 shutdown_connections 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 --2- 172.21.15.148:0/2665015818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e406c9e0 0x7f54e40b5ab0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 --2- 172.21.15.148:0/2665015818 >> v2:172.21.15.188:3300/0 conn(0x7f54e415b7d0 0x7f54e40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 -- 172.21.15.148:0/2665015818 >> 172.21.15.148:0/2665015818 conn(0x7f54e40d0800 msgr2=0x7f54e40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 -- 172.21.15.148:0/2665015818 shutdown_connections 2024-04-25T10:01:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.927+0000 7f54e9977640 1 -- 172.21.15.148:0/2665015818 wait complete. 2024-04-25T10:01:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.929+0000 7f54e9977640 1 Processor -- start 2024-04-25T10:01:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.929+0000 7f54e9977640 1 -- start start 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e9977640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e406c9e0 0x7f54e4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e9977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 0x7f54e40b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e9977640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e415b7d0 0x7f54e40b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e9977640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40cf6d0 con 0x7f54e40b5620 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e9977640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40cf550 con 0x7f54e415b7d0 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e9977640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e415bbb0 con 0x7f54e406c9e0 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 0x7f54e40b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 0x7f54e40b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42720/0 (socket says 172.21.15.148:42720) 2024-04-25T10:01:54.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.930+0000 7f54e27fc640 1 -- 172.21.15.148:0/2199203211 learned_addr learned my addr 172.21.15.148:0/2199203211 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e2ffd640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e406c9e0 0x7f54e4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e37fe640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e415b7d0 0x7f54e40b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e27fc640 1 -- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e406c9e0 msgr2=0x7f54e4150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e27fc640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e406c9e0 0x7f54e4150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e27fc640 1 -- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e415b7d0 msgr2=0x7f54e40b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e27fc640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e415b7d0 0x7f54e40b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e27fc640 1 -- 172.21.15.148:0/2199203211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40cad10 con 0x7f54e40b5620 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e37fe640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e415b7d0 0x7f54e40b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e2ffd640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e406c9e0 0x7f54e4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e27fc640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 0x7f54e40b4550 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f54d400c410 tx=0x7f54d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4017070 con 0x7f54e40b5620 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d40035f0 con 0x7f54e40b5620 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e40cafb0 con 0x7f54e40b5620 2024-04-25T10:01:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d4003a20 con 0x7f54e40b5620 2024-04-25T10:01:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.931+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e40cb570 con 0x7f54e40b5620 2024-04-25T10:01:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.932+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e4073a00 con 0x7f54e40b5620 2024-04-25T10:01:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.933+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54d401a070 con 0x7f54e40b5620 2024-04-25T10:01:54.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.933+0000 7f54dbfff640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54c406b4d0 0x7f54c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.933+0000 7f54e2ffd640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54c406b4d0 0x7f54c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.933+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54d4089840 con 0x7f54e40b5620 2024-04-25T10:01:54.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.934+0000 7f54e2ffd640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54c406b4d0 0x7f54c406d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00a870 tx=0x7f54cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:54.937+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54d4058520 con 0x7f54e40b5620 2024-04-25T10:01:55.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.151+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f54e40b3240 con 0x7f54e40b5620 2024-04-25T10:01:55.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.771+0000 7f54dbfff640 1 -- 172.21.15.148:0/2199203211 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) v1 ==== 122+0+0 (secure 0 0 0) 0x7f54d40010c0 con 0x7f54e40b5620 2024-04-25T10:01:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.773+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54c406b4d0 msgr2=0x7f54c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.773+0000 7f54e9977640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54c406b4d0 0x7f54c406d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00a870 tx=0x7f54cc008040 comp rx=0 tx=0).stop 2024-04-25T10:01:55.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.773+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 msgr2=0x7f54e40b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.773+0000 7f54e9977640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 0x7f54e40b4550 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f54d400c410 tx=0x7f54d400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 shutdown_connections 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54c406b4d0 0x7f54c406d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54e415b7d0 0x7f54e40b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54e40b5620 0x7f54e40b4550 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 --2- 172.21.15.148:0/2199203211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54e406c9e0 0x7f54e4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 >> 172.21.15.148:0/2199203211 conn(0x7f54e40d0800 msgr2=0x7f54e406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 shutdown_connections 2024-04-25T10:01:55.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.774+0000 7f54e9977640 1 -- 172.21.15.148:0/2199203211 wait complete. 2024-04-25T10:01:55.790 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:01:55.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.884+0000 7f07f18f7640 1 Processor -- start 2024-04-25T10:01:55.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.884+0000 7f07f18f7640 1 -- start start 2024-04-25T10:01:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07f18f7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f07ec0b5f00 0x7f07ec0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07f18f7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ec0cec00 con 0x7f07ec15db60 2024-04-25T10:01:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07f18f7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ec0ceee0 con 0x7f07ec0d0b10 2024-04-25T10:01:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07f18f7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec0ce930 con 0x7f07ec0b5f00 2024-04-25T10:01:55.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07eb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f07ec0b5f00 0x7f07ec0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07eaffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f07ec15db60 0x7f07ec0d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42950/0 (socket says 172.21.15.148:42950) 2024-04-25T10:01:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.885+0000 7f07eaffd640 1 -- 172.21.15.148:0/3656824952 learned_addr learned my addr 172.21.15.148:0/3656824952 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.886+0000 7f07eb7fe640 1 -- 172.21.15.148:0/3656824952 >> v1:172.21.15.188:6789/0 conn(0x7f07ec0d0b10 legacy=0x7f07ec0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.886+0000 7f07eb7fe640 1 -- 172.21.15.148:0/3656824952 >> v1:172.21.15.148:6789/0 conn(0x7f07ec15db60 legacy=0x7f07ec0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.886+0000 7f07e9ffb640 1 -- 172.21.15.148:0/3656824952 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041538768 0 0) 0x7f07ec0cec00 con 0x7f07ec15db60 2024-04-25T10:01:55.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.886+0000 7f07eb7fe640 1 -- 172.21.15.148:0/3656824952 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0cf1c0 con 0x7f07ec0b5f00 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.886+0000 7f07eb7fe640 1 --2- 172.21.15.148:0/3656824952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec0b5f00 0x7f07ec0ce3f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009dc0 tx=0x7f07e402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c650402a640b660c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.886+0000 7f07e9ffb640 1 -- 172.21.15.148:0/3656824952 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4007740 con 0x7f07ec0b5f00 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07e9ffb640 1 -- 172.21.15.148:0/3656824952 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e40078e0 con 0x7f07ec0b5f00 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07e9ffb640 1 -- 172.21.15.148:0/3656824952 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4007cb0 con 0x7f07ec0b5f00 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 -- 172.21.15.148:0/3656824952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec0b5f00 msgr2=0x7f07ec0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 --2- 172.21.15.148:0/3656824952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec0b5f00 0x7f07ec0ce3f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009dc0 tx=0x7f07e402a640 comp rx=0 tx=0).stop 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 -- 172.21.15.148:0/3656824952 shutdown_connections 2024-04-25T10:01:55.893 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 --2- 172.21.15.148:0/3656824952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec0b5f00 0x7f07ec0ce3f0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 -- 172.21.15.148:0/3656824952 >> 172.21.15.148:0/3656824952 conn(0x7f07ec061600 msgr2=0x7f07ec061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 -- 172.21.15.148:0/3656824952 shutdown_connections 2024-04-25T10:01:55.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.887+0000 7f07f18f7640 1 -- 172.21.15.148:0/3656824952 wait complete. 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.888+0000 7f07f18f7640 1 Processor -- start 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.888+0000 7f07f18f7640 1 -- start start 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07f18f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ec0b5f00 0x7f07ec0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07f18f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ec0d0b10 0x7f07ec15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07f18f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 0x7f07ec151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07f18f7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec15df40 con 0x7f07ec15db60 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07f18f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec0b6450 con 0x7f07ec0d0b10 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07f18f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec15e0c0 con 0x7f07ec0b5f00 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07eb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 0x7f07ec151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07eb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 0x7f07ec151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42736/0 (socket says 172.21.15.148:42736) 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.889+0000 7f07eb7fe640 1 -- 172.21.15.148:0/1573497964 learned_addr learned my addr 172.21.15.148:0/1573497964 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:55.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eaffd640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ec0b5f00 0x7f07ec0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07ea7fc640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ec0d0b10 0x7f07ec15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eb7fe640 1 -- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ec0b5f00 msgr2=0x7f07ec0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eb7fe640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ec0b5f00 0x7f07ec0c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eb7fe640 1 -- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ec0d0b10 msgr2=0x7f07ec15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eb7fe640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ec0d0b10 0x7f07ec15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eb7fe640 1 -- 172.21.15.148:0/1573497964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec15ed20 con 0x7f07ec15db60 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07eb7fe640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 0x7f07ec151b50 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009ee0 tx=0x7f07e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4040070 con 0x7f07ec15db60 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e4031c40 con 0x7f07ec15db60 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec15f2e0 con 0x7f07ec15db60 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e403c420 con 0x7f07ec15db60 2024-04-25T10:01:55.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.890+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec15f8a0 con 0x7f07ec15db60 2024-04-25T10:01:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.891+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ec073270 con 0x7f07ec15db60 2024-04-25T10:01:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.892+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07e4039040 con 0x7f07ec15db60 2024-04-25T10:01:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.892+0000 7f07e3fff640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f07cc06b500 0x7f07cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.892+0000 7f07eaffd640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f07cc06b500 0x7f07cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.892+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07e40ab8a0 con 0x7f07ec15db60 2024-04-25T10:01:55.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.893+0000 7f07eaffd640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f07cc06b500 0x7f07cc06d9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f07d4007fd0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:55.896+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07e402ae20 con 0x7f07ec15db60 2024-04-25T10:01:56.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.106+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07ec15faa0 con 0x7f07ec15db60 2024-04-25T10:01:56.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.107+0000 7f07e3fff640 1 -- 172.21.15.148:0/1573497964 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f07ec15faa0 con 0x7f07ec15db60 2024-04-25T10:01:56.113 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:56.113 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:01:56.113 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f07cc06b500 msgr2=0x7f07cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f07cc06b500 0x7f07cc06d9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f07d4007fd0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 msgr2=0x7f07ec151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 0x7f07ec151b50 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009ee0 tx=0x7f07e4004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 shutdown_connections 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f07cc06b500 0x7f07cc06d9c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07ec15db60 0x7f07ec151b50 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07ec0d0b10 0x7f07ec15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 --2- 172.21.15.148:0/1573497964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07ec0b5f00 0x7f07ec0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 >> 172.21.15.148:0/1573497964 conn(0x7f07ec061600 msgr2=0x7f07ec15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 shutdown_connections 2024-04-25T10:01:56.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.109+0000 7f07f18f7640 1 -- 172.21.15.148:0/1573497964 wait complete. 2024-04-25T10:01:56.126 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.217+0000 7f680b3c1640 1 Processor -- start 2024-04-25T10:01:56.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.217+0000 7f680b3c1640 1 -- start start 2024-04-25T10:01:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680b3c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804154ee0 0x7f68040cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680b3c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68040ce590 0x7f680406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680b3c1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6804071480 con 0x7f68040cfcf0 2024-04-25T10:01:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680b3c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6804072090 con 0x7f68040ce590 2024-04-25T10:01:56.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680b3c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040d0150 con 0x7f6804154ee0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680995e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68040ce590 0x7f680406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680915d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804154ee0 0x7f68040cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680895c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f68040cfcf0 0x7f68040cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:35494/0 (socket says 172.21.15.148:35494) 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680895c640 1 -- 172.21.15.148:0/3460092012 learned_addr learned my addr 172.21.15.148:0/3460092012 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680915d640 1 -- 172.21.15.148:0/3460092012 >> v2:172.21.15.188:3300/0 conn(0x7f68040ce590 msgr2=0x7f680406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680915d640 1 --2- 172.21.15.148:0/3460092012 >> v2:172.21.15.188:3300/0 conn(0x7f68040ce590 0x7f680406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.218+0000 7f680915d640 1 -- 172.21.15.148:0/3460092012 >> v1:172.21.15.188:6789/0 conn(0x7f68040cfcf0 legacy=0x7f68040cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.219+0000 7f680915d640 1 -- 172.21.15.148:0/3460092012 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6804071760 con 0x7f6804154ee0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.219+0000 7f680995e640 1 --2- 172.21.15.148:0/3460092012 >> v2:172.21.15.188:3300/0 conn(0x7f68040ce590 0x7f680406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.219+0000 7f680915d640 1 --2- 172.21.15.148:0/3460092012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804154ee0 0x7f68040cf7b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68000097a0 tx=0x7f680002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4899fa5dc7a18898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.219+0000 7f67fbfff640 1 -- 172.21.15.148:0/3460092012 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800037070 con 0x7f6804154ee0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.219+0000 7f67fbfff640 1 -- 172.21.15.148:0/3460092012 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800032440 con 0x7f6804154ee0 2024-04-25T10:01:56.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.219+0000 7f67fbfff640 1 -- 172.21.15.148:0/3460092012 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800032810 con 0x7f6804154ee0 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 -- 172.21.15.148:0/3460092012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804154ee0 msgr2=0x7f68040cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 --2- 172.21.15.148:0/3460092012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804154ee0 0x7f68040cf7b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68000097a0 tx=0x7f680002a180 comp rx=0 tx=0).stop 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 -- 172.21.15.148:0/3460092012 shutdown_connections 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 --2- 172.21.15.148:0/3460092012 >> v2:172.21.15.188:3300/0 conn(0x7f68040ce590 0x7f680406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 --2- 172.21.15.148:0/3460092012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6804154ee0 0x7f68040cf7b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 -- 172.21.15.148:0/3460092012 >> 172.21.15.148:0/3460092012 conn(0x7f680406abe0 msgr2=0x7f680406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 -- 172.21.15.148:0/3460092012 shutdown_connections 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.220+0000 7f680b3c1640 1 -- 172.21.15.148:0/3460092012 wait complete. 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.221+0000 7f680b3c1640 1 Processor -- start 2024-04-25T10:01:56.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.221+0000 7f680b3c1640 1 -- start start 2024-04-25T10:01:56.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680b3c1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 0x7f68040ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680b3c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68040cfcf0 0x7f68040a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680b3c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804154ee0 0x7f68040a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680b3c1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804071d20 con 0x7f68040ce590 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680b3c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804071230 con 0x7f6804154ee0 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680b3c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68041552c0 con 0x7f68040cfcf0 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 0x7f68040ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 0x7f68040ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42742/0 (socket says 172.21.15.148:42742) 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 -- 172.21.15.148:0/2711162729 learned_addr learned my addr 172.21.15.148:0/2711162729 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680995e640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804154ee0 0x7f68040a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680895c640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68040cfcf0 0x7f68040a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 -- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68040cfcf0 msgr2=0x7f68040a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68040cfcf0 0x7f68040a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 -- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804154ee0 msgr2=0x7f68040a9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804154ee0 0x7f68040a9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680915d640 1 -- 172.21.15.148:0/2711162729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68040abf70 con 0x7f68040ce590 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.222+0000 7f680995e640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804154ee0 0x7f68040a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:56.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.223+0000 7f680915d640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 0x7f68040ab870 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6800035040 tx=0x7f68000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.223+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800037040 con 0x7f68040ce590 2024-04-25T10:01:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.223+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800030500 con 0x7f68040ce590 2024-04-25T10:01:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.223+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68040ac250 con 0x7f68040ce590 2024-04-25T10:01:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.223+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800030910 con 0x7f68040ce590 2024-04-25T10:01:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.223+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68040ac810 con 0x7f68040ce590 2024-04-25T10:01:56.231 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.224+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6804154200 con 0x7f68040ce590 2024-04-25T10:01:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.224+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6800044020 con 0x7f68040ce590 2024-04-25T10:01:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.225+0000 7f67f9ffb640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67e806b4d0 0x7f67e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.225+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68000aa5a0 con 0x7f68040ce590 2024-04-25T10:01:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.225+0000 7f680895c640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67e806b4d0 0x7f67e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.225+0000 7f680895c640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67e806b4d0 0x7f67e806d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f680415e230 tx=0x7f67f4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.229+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6804154200 con 0x7f68040ce590 2024-04-25T10:01:56.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.404+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6804073990 con 0x7f68040ce590 2024-04-25T10:01:56.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.405+0000 7f67f9ffb640 1 -- 172.21.15.148:0/2711162729 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f6800079320 con 0x7f68040ce590 2024-04-25T10:01:56.411 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:56.411 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":121,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:54.304228+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67e806b4d0 msgr2=0x7f67e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67e806b4d0 0x7f67e806d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f680415e230 tx=0x7f67f4009040 comp rx=0 tx=0).stop 2024-04-25T10:01:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 msgr2=0x7f68040ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 0x7f68040ab870 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f6800035040 tx=0x7f68000046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 shutdown_connections 2024-04-25T10:01:56.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67e806b4d0 0x7f67e806d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6804154ee0 0x7f68040a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68040cfcf0 0x7f68040a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 --2- 172.21.15.148:0/2711162729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68040ce590 0x7f68040ab870 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.407+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 >> 172.21.15.148:0/2711162729 conn(0x7f680406abe0 msgr2=0x7f68040b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.408+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 shutdown_connections 2024-04-25T10:01:56.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.408+0000 7f680b3c1640 1 -- 172.21.15.148:0/2711162729 wait complete. 2024-04-25T10:01:56.425 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:01:56.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:56.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.517+0000 7f84f7874640 1 Processor -- start 2024-04-25T10:01:56.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.517+0000 7f84f7874640 1 -- start start 2024-04-25T10:01:56.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f7874640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84f00cc8e0 0x7f84f01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f7874640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f84f00d1810 0x7f84f00d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f7874640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00cce50 con 0x7f84f015dc90 2024-04-25T10:01:56.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f7874640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f00d21c0 con 0x7f84f00cc8e0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f7874640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f0161af0 con 0x7f84f00d1810 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84f00cc8e0 0x7f84f01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84f00cc8e0 0x7f84f01625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42754/0 (socket says 172.21.15.148:42754) 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 -- 172.21.15.148:0/2251785103 learned_addr learned my addr 172.21.15.148:0/2251785103 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f5e11640 1 --2- 172.21.15.148:0/2251785103 >> v2:172.21.15.188:3300/0 conn(0x7f84f00d1810 0x7f84f00d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 -- 172.21.15.148:0/2251785103 >> v2:172.21.15.188:3300/0 conn(0x7f84f00d1810 msgr2=0x7f84f00d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 --2- 172.21.15.148:0/2251785103 >> v2:172.21.15.188:3300/0 conn(0x7f84f00d1810 0x7f84f00d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 -- 172.21.15.148:0/2251785103 >> v1:172.21.15.188:6789/0 conn(0x7f84f015dc90 legacy=0x7f84f015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.518+0000 7f84f4e0f640 1 -- 172.21.15.148:0/2251785103 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0161dd0 con 0x7f84f00cc8e0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.519+0000 7f84f4e0f640 1 --2- 172.21.15.148:0/2251785103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f01625d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f84f00cdbb0 tx=0x7f84e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7006f83c730c7225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.519+0000 7f84e7fff640 1 -- 172.21.15.148:0/2251785103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0037070 con 0x7f84f00cc8e0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.519+0000 7f84e7fff640 1 -- 172.21.15.148:0/2251785103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0032440 con 0x7f84f00cc8e0 2024-04-25T10:01:56.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.519+0000 7f84e7fff640 1 -- 172.21.15.148:0/2251785103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0032810 con 0x7f84f00cc8e0 2024-04-25T10:01:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.519+0000 7f84f7874640 1 -- 172.21.15.148:0/2251785103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 msgr2=0x7f84f01625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.519+0000 7f84f7874640 1 --2- 172.21.15.148:0/2251785103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f01625d0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f84f00cdbb0 tx=0x7f84e002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.520+0000 7f84f7874640 1 -- 172.21.15.148:0/2251785103 shutdown_connections 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.520+0000 7f84f7874640 1 --2- 172.21.15.148:0/2251785103 >> v2:172.21.15.188:3300/0 conn(0x7f84f00d1810 0x7f84f00d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.520+0000 7f84f7874640 1 --2- 172.21.15.148:0/2251785103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f01625d0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.520+0000 7f84f7874640 1 -- 172.21.15.148:0/2251785103 >> 172.21.15.148:0/2251785103 conn(0x7f84f006f590 msgr2=0x7f84f006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.520+0000 7f84f7874640 1 -- 172.21.15.148:0/2251785103 shutdown_connections 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.520+0000 7f84f7874640 1 -- 172.21.15.148:0/2251785103 wait complete. 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.521+0000 7f84f7874640 1 Processor -- start 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.521+0000 7f84f7874640 1 -- start start 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.521+0000 7f84f7874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84f00d1810 0x7f84f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84f015dc90 0x7f84f00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015e070 con 0x7f84f00cc8e0 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00cd130 con 0x7f84f00d1810 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015e1f0 con 0x7f84f015dc90 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f014dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42764/0 (socket says 172.21.15.148:42764) 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 -- 172.21.15.148:0/2098207115 learned_addr learned my addr 172.21.15.148:0/2098207115 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 -- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84f015dc90 msgr2=0x7f84f00b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84f015dc90 0x7f84f00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 -- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84f00d1810 msgr2=0x7f84f006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f4e0f640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84f00d1810 0x7f84f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84f00d1810 0x7f84f006d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 -- 172.21.15.148:0/2098207115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f014b240 con 0x7f84f00cc8e0 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f4e0f640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84f00d1810 0x7f84f006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f5610640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f014dc90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00b3f0 tx=0x7f84ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec015020 con 0x7f84f00cc8e0 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec011830 con 0x7f84f00cc8e0 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f014e1d0 con 0x7f84f00cc8e0 2024-04-25T10:01:56.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011c20 con 0x7f84f00cc8e0 2024-04-25T10:01:56.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.522+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f014b7c0 con 0x7f84f00cc8e0 2024-04-25T10:01:56.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.523+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f0073990 con 0x7f84f00cc8e0 2024-04-25T10:01:56.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.524+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84ec018020 con 0x7f84f00cc8e0 2024-04-25T10:01:56.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.524+0000 7f84e5ffb640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f84d406b4d0 0x7f84d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.524+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84ec08a050 con 0x7f84f00cc8e0 2024-04-25T10:01:56.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.524+0000 7f84f4e0f640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f84d406b4d0 0x7f84d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.525+0000 7f84f4e0f640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f84d406b4d0 0x7f84d406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84e0034000 tx=0x7f84e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.528+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84ec01f030 con 0x7f84f00cc8e0 2024-04-25T10:01:56.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.696+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84f006c920 con 0x7f84d406b4d0 2024-04-25T10:01:56.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.701+0000 7f84e5ffb640 1 -- 172.21.15.148:0/2098207115 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240142 (secure 0 0 0) 0x7f84f006c920 con 0x7f84d406b4d0 2024-04-25T10:01:56.711 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:56.713 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f84d406b4d0 msgr2=0x7f84d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f84d406b4d0 0x7f84d406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f84e0034000 tx=0x7f84e002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 msgr2=0x7f84f014dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f014dc90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00b3f0 tx=0x7f84ec00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 shutdown_connections 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f84d406b4d0 0x7f84d406d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84f015dc90 0x7f84f00b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84f00d1810 0x7f84f006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 --2- 172.21.15.148:0/2098207115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84f00cc8e0 0x7f84f014dc90 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 >> 172.21.15.148:0/2098207115 conn(0x7f84f006f590 msgr2=0x7f84f015b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.704+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 shutdown_connections 2024-04-25T10:01:56.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.705+0000 7f84f7874640 1 -- 172.21.15.148:0/2098207115 wait complete. 2024-04-25T10:01:56.721 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-25T10:01:55.113629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":386880,"kb_used_data":12492,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587840,"statfs":{"total":773094113280,"available":772697948160,"internally_reserved":0,"allocated":12791808,"data_stored":6557784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000659"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312163+0000","last_change":"2024-04-25T10:01:53.308968+0000","last_active":"2024-04-25T10:01:54.312163+0000","last_peered":"2024-04-25T10:01:54.312163+0000","last_clean":"2024-04-25T10:01:54.312163+0000","last_became_active":"2024-04-25T10:01:53.308683+0000","last_became_peered":"2024-04-25T10:01:53.308683+0000","last_unstale":"2024-04-25T10:01:54.312163+0000","last_undegraded":"2024-04-25T10:01:54.312163+0000","last_fullsized":"2024-04-25T10:01:54.312163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311504+0000","last_change":"2024-04-25T10:01:53.313912+0000","last_active":"2024-04-25T10:01:54.311504+0000","last_peered":"2024-04-25T10:01:54.311504+0000","last_clean":"2024-04-25T10:01:54.311504+0000","last_became_active":"2024-04-25T10:01:53.313635+0000","last_became_peered":"2024-04-25T10:01:53.313635+0000","last_unstale":"2024-04-25T10:01:54.311504+0000","last_undegraded":"2024-04-25T10:01:54.311504+0000","last_fullsized":"2024-04-25T10:01:54.311504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312000+0000","last_change":"2024-04-25T10:01:53.315244+0000","last_active":"2024-04-25T10:01:54.312000+0000","last_peered":"2024-04-25T10:01:54.312000+0000","last_clean":"2024-04-25T10:01:54.312000+0000","last_became_active":"2024-04-25T10:01:53.314991+0000","last_became_peered":"2024-04-25T10:01:53.314991+0000","last_unstale":"2024-04-25T10:01:54.312000+0000","last_undegraded":"2024-04-25T10:01:54.312000+0000","last_fullsized":"2024-04-25T10:01:54.312000+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:10.260048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311370+0000","last_change":"2024-04-25T10:01:53.314815+0000","last_active":"2024-04-25T10:01:54.311370+0000","last_peered":"2024-04-25T10:01:54.311370+0000","last_clean":"2024-04-25T10:01:54.311370+0000","last_became_active":"2024-04-25T10:01:53.314511+0000","last_became_peered":"2024-04-25T10:01:53.314511+0000","last_unstale":"2024-04-25T10:01:54.311370+0000","last_undegraded":"2024-04-25T10:01:54.311370+0000","last_fullsized":"2024-04-25T10:01:54.311370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347573+0000","last_change":"2024-04-25T10:01:53.307833+0000","last_active":"2024-04-25T10:01:54.347573+0000","last_peered":"2024-04-25T10:01:54.347573+0000","last_clean":"2024-04-25T10:01:54.347573+0000","last_became_active":"2024-04-25T10:01:53.307583+0000","last_became_peered":"2024-04-25T10:01:53.307583+0000","last_unstale":"2024-04-25T10:01:54.347573+0000","last_undegraded":"2024-04-25T10:01:54.347573+0000","last_fullsized":"2024-04-25T10:01:54.347573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:14.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311689+0000","last_change":"2024-04-25T10:01:53.317853+0000","last_active":"2024-04-25T10:01:54.311689+0000","last_peered":"2024-04-25T10:01:54.311689+0000","last_clean":"2024-04-25T10:01:54.311689+0000","last_became_active":"2024-04-25T10:01:53.317561+0000","last_became_peered":"2024-04-25T10:01:53.317561+0000","last_unstale":"2024-04-25T10:01:54.311689+0000","last_undegraded":"2024-04-25T10:01:54.311689+0000","last_fullsized":"2024-04-25T10:01:54.311689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.139972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347834+0000","last_change":"2024-04-25T10:01:53.306396+0000","last_active":"2024-04-25T10:01:54.347834+0000","last_peered":"2024-04-25T10:01:54.347834+0000","last_clean":"2024-04-25T10:01:54.347834+0000","last_became_active":"2024-04-25T10:01:53.306127+0000","last_became_peered":"2024-04-25T10:01:53.306127+0000","last_unstale":"2024-04-25T10:01:54.347834+0000","last_undegraded":"2024-04-25T10:01:54.347834+0000","last_fullsized":"2024-04-25T10:01:54.347834+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:27.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347913+0000","last_change":"2024-04-25T10:01:53.316236+0000","last_active":"2024-04-25T10:01:54.347913+0000","last_peered":"2024-04-25T10:01:54.347913+0000","last_clean":"2024-04-25T10:01:54.347913+0000","last_became_active":"2024-04-25T10:01:53.315521+0000","last_became_peered":"2024-04-25T10:01:53.315521+0000","last_unstale":"2024-04-25T10:01:54.347913+0000","last_undegraded":"2024-04-25T10:01:54.347913+0000","last_fullsized":"2024-04-25T10:01:54.347913+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.725287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311619+0000","last_change":"2024-04-25T10:01:52.300611+0000","last_active":"2024-04-25T10:01:54.311619+0000","last_peered":"2024-04-25T10:01:54.311619+0000","last_clean":"2024-04-25T10:01:54.311619+0000","last_became_active":"2024-04-25T10:01:52.300310+0000","last_became_peered":"2024-04-25T10:01:52.300310+0000","last_unstale":"2024-04-25T10:01:54.311619+0000","last_undegraded":"2024-04-25T10:01:54.311619+0000","last_fullsized":"2024-04-25T10:01:54.311619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311437+0000","last_change":"2024-04-25T10:01:53.308986+0000","last_active":"2024-04-25T10:01:54.311437+0000","last_peered":"2024-04-25T10:01:54.311437+0000","last_clean":"2024-04-25T10:01:54.311437+0000","last_became_active":"2024-04-25T10:01:53.308652+0000","last_became_peered":"2024-04-25T10:01:53.308652+0000","last_unstale":"2024-04-25T10:01:54.311437+0000","last_undegraded":"2024-04-25T10:01:54.311437+0000","last_fullsized":"2024-04-25T10:01:54.311437+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347539+0000","last_change":"2024-04-25T10:01:53.311926+0000","last_active":"2024-04-25T10:01:54.347539+0000","last_peered":"2024-04-25T10:01:54.347539+0000","last_clean":"2024-04-25T10:01:54.347539+0000","last_became_active":"2024-04-25T10:01:53.309452+0000","last_became_peered":"2024-04-25T10:01:53.309452+0000","last_unstale":"2024-04-25T10:01:54.347539+0000","last_undegraded":"2024-04-25T10:01:54.347539+0000","last_fullsized":"2024-04-25T10:01:54.347539+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.619357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312173+0000","last_change":"2024-04-25T10:01:53.315399+0000","last_active":"2024-04-25T10:01:54.312173+0000","last_peered":"2024-04-25T10:01:54.312173+0000","last_clean":"2024-04-25T10:01:54.312173+0000","last_became_active":"2024-04-25T10:01:53.315137+0000","last_became_peered":"2024-04-25T10:01:53.315137+0000","last_unstale":"2024-04-25T10:01:54.312173+0000","last_undegraded":"2024-04-25T10:01:54.312173+0000","last_fullsized":"2024-04-25T10:01:54.312173+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.634999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347825+0000","last_change":"2024-04-25T10:01:52.292697+0000","last_active":"2024-04-25T10:01:54.347825+0000","last_peered":"2024-04-25T10:01:54.347825+0000","last_clean":"2024-04-25T10:01:54.347825+0000","last_became_active":"2024-04-25T10:01:52.292316+0000","last_became_peered":"2024-04-25T10:01:52.292316+0000","last_unstale":"2024-04-25T10:01:54.347825+0000","last_undegraded":"2024-04-25T10:01:54.347825+0000","last_fullsized":"2024-04-25T10:01:54.347825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:31.205159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348125+0000","last_change":"2024-04-25T10:01:53.312259+0000","last_active":"2024-04-25T10:01:54.348125+0000","last_peered":"2024-04-25T10:01:54.348125+0000","last_clean":"2024-04-25T10:01:54.348125+0000","last_became_active":"2024-04-25T10:01:53.311938+0000","last_became_peered":"2024-04-25T10:01:53.311938+0000","last_unstale":"2024-04-25T10:01:54.348125+0000","last_undegraded":"2024-04-25T10:01:54.348125+0000","last_fullsized":"2024-04-25T10:01:54.348125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:11.878572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311723+0000","last_change":"2024-04-25T10:01:52.289539+0000","last_active":"2024-04-25T10:01:54.311723+0000","last_peered":"2024-04-25T10:01:54.311723+0000","last_clean":"2024-04-25T10:01:54.311723+0000","last_became_active":"2024-04-25T10:01:52.289261+0000","last_became_peered":"2024-04-25T10:01:52.289261+0000","last_unstale":"2024-04-25T10:01:54.311723+0000","last_undegraded":"2024-04-25T10:01:54.311723+0000","last_fullsized":"2024-04-25T10:01:54.311723+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311806+0000","last_change":"2024-04-25T10:01:52.292733+0000","last_active":"2024-04-25T10:01:54.311806+0000","last_peered":"2024-04-25T10:01:54.311806+0000","last_clean":"2024-04-25T10:01:54.311806+0000","last_became_active":"2024-04-25T10:01:52.292040+0000","last_became_peered":"2024-04-25T10:01:52.292040+0000","last_unstale":"2024-04-25T10:01:54.311806+0000","last_undegraded":"2024-04-25T10:01:54.311806+0000","last_fullsized":"2024-04-25T10:01:54.311806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":345,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:01:49.019604+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:49.019604+0000","last_peered":"2024-04-25T10:01:49.019604+0000","last_clean":"2024-04-25T10:01:49.019604+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:49.019604+0000","last_undegraded":"2024-04-25T10:01:49.019604+0000","last_fullsized":"2024-04-25T10:01:49.019604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41492,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330348,"statfs":{"total":96636764160,"available":96594276352,"internally_reserved":0,"allocated":2039808,"data_stored":1259769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46480,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325360,"statfs":{"total":96636764160,"available":96589168640,"internally_reserved":0,"allocated":1445888,"data_stored":669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45072,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326768,"statfs":{"total":96636764160,"available":96590610432,"internally_reserved":0,"allocated":1445888,"data_stored":669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1519616,"data_stored":703267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50640,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1445888,"data_stored":669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1507328,"data_stored":701655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50596,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1396736,"data_stored":647261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51176,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320664,"statfs":{"total":96636764160,"available":96584359936,"internally_reserved":0,"allocated":1990656,"data_stored":1237629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:56.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:56.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.853+0000 7fad82bc8640 1 Processor -- start 2024-04-25T10:01:56.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.853+0000 7fad82bc8640 1 -- start start 2024-04-25T10:01:56.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad82bc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad7c06f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad82bc8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c06c730 con 0x7fad7c15bb60 2024-04-25T10:01:56.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad82bc8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c06b690 con 0x7fad7c0721d0 2024-04-25T10:01:56.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad82bc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c06b970 con 0x7fad7c06d9f0 2024-04-25T10:01:56.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad80964640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fad7c15bb60 0x7fad7c071ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42966/0 (socket says 172.21.15.148:42966) 2024-04-25T10:01:56.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad80964640 1 -- 172.21.15.148:0/2159070408 learned_addr learned my addr 172.21.15.148:0/2159070408 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:56.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920727252 0 0) 0x7fad7c06c730 con 0x7fad7c15bb60 2024-04-25T10:01:56.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c003520 con 0x7fad7c15bb60 2024-04-25T10:01:56.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad81165640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad7c06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143167256 0 0) 0x7fad6c003520 con 0x7fad7c15bb60 2024-04-25T10:01:56.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 msgr2=0x7fad7c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad7c06f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 >> v1:172.21.15.188:6789/0 conn(0x7fad7c0721d0 legacy=0x7fad7c15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c06bc50 con 0x7fad7c15bb60 2024-04-25T10:01:56.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fad70001670 con 0x7fad7c15bb60 2024-04-25T10:01:56.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 >> v1:172.21.15.148:6789/0 conn(0x7fad7c15bb60 legacy=0x7fad7c071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6c0039e0 0x7fad6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad6c004800 0x7fad6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c06b970 con 0x7fad6c0039e0 2024-04-25T10:01:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00ba70 con 0x7fad6c004800 2024-04-25T10:01:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.854+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00bc70 con 0x7fad7c06d9f0 2024-04-25T10:01:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6c0039e0 0x7fad6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad81165640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad7bfff640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad6c004800 0x7fad6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 -- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 msgr2=0x7fad6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 -- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad6c004800 msgr2=0x7fad6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad6c004800 0x7fad6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 -- 172.21.15.148:0/2159070408 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c003520 con 0x7fad6c0039e0 2024-04-25T10:01:56.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad81165640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c06d9f0 0x7fad6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:56.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.855+0000 7fad80964640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6c0039e0 0x7fad6c003e00 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fad70001eb0 tx=0x7fad6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18a3c6d0ee401cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70007d30 con 0x7fad6c0039e0 2024-04-25T10:01:56.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70005af0 con 0x7fad6c0039e0 2024-04-25T10:01:56.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad7b7fe640 1 -- 172.21.15.148:0/2159070408 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70007070 con 0x7fad6c0039e0 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad82bc8640 1 -- 172.21.15.148:0/2159070408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6c0039e0 msgr2=0x7fad6c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad82bc8640 1 --2- 172.21.15.148:0/2159070408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad6c0039e0 0x7fad6c003e00 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fad70001eb0 tx=0x7fad6c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad81165640 1 -- 172.21.15.148:0/2159070408 reap_dead start 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad82bc8640 1 -- 172.21.15.148:0/2159070408 shutdown_connections 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad82bc8640 1 -- 172.21.15.148:0/2159070408 >> 172.21.15.148:0/2159070408 conn(0x7fad7c0af670 msgr2=0x7fad7c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad82bc8640 1 -- 172.21.15.148:0/2159070408 shutdown_connections 2024-04-25T10:01:56.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.856+0000 7fad82bc8640 1 -- 172.21.15.148:0/2159070408 wait complete. 2024-04-25T10:01:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.857+0000 7fad82bc8640 1 Processor -- start 2024-04-25T10:01:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.857+0000 7fad82bc8640 1 -- start start 2024-04-25T10:01:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad82bc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 0x7fad7c09aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad82bc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c09afd0 0x7fad7c1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad82bc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c09d410 0x7fad7c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad82bc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c0727f0 con 0x7fad7c09d030 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad82bc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c06c210 con 0x7fad7c09d410 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad82bc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c072970 con 0x7fad7c09afd0 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad80964640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 0x7fad7c09aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad80964640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 0x7fad7c09aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42770/0 (socket says 172.21.15.148:42770) 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad80964640 1 -- 172.21.15.148:0/3699145007 learned_addr learned my addr 172.21.15.148:0/3699145007 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad7bfff640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c09afd0 0x7fad7c1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.858+0000 7fad81165640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c09d410 0x7fad7c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad80964640 1 -- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c09afd0 msgr2=0x7fad7c1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad80964640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c09afd0 0x7fad7c1540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad80964640 1 -- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c09d410 msgr2=0x7fad7c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad80964640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c09d410 0x7fad7c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad80964640 1 -- 172.21.15.148:0/3699145007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c1581e0 con 0x7fad7c09d030 2024-04-25T10:01:56.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad81165640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c09d410 0x7fad7c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad80964640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 0x7fad7c09aa90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c300 tx=0x7fad70001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70001630 con 0x7fad7c09d030 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70042070 con 0x7fad7c09d030 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c1584c0 con 0x7fad7c09d030 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700072d0 con 0x7fad7c09d030 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.859+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c146d80 con 0x7fad7c09d030 2024-04-25T10:01:56.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.860+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c073990 con 0x7fad7c09d030 2024-04-25T10:01:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.860+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad70007470 con 0x7fad7c09d030 2024-04-25T10:01:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.861+0000 7fad797fa640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6006b500 0x7fad6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.861+0000 7fad7bfff640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6006b500 0x7fad6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.861+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad700aa420 con 0x7fad7c09d030 2024-04-25T10:01:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.861+0000 7fad7bfff640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6006b500 0x7fad6006d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fad7c09b3d0 tx=0x7fad68009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:56.865+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad700791a0 con 0x7fad7c09d030 2024-04-25T10:01:57.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.031+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad7c159a10 con 0x7fad6006b500 2024-04-25T10:01:57.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.036+0000 7fad797fa640 1 -- 172.21.15.148:0/3699145007 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240142 (secure 0 0 0) 0x7fad7c159a10 con 0x7fad6006b500 2024-04-25T10:01:57.044 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:57.045 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6006b500 msgr2=0x7fad6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6006b500 0x7fad6006d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fad7c09b3d0 tx=0x7fad68009040 comp rx=0 tx=0).stop 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 msgr2=0x7fad7c09aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 0x7fad7c09aa90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c300 tx=0x7fad70001a40 comp rx=0 tx=0).stop 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 shutdown_connections 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fad6006b500 0x7fad6006d9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad7c09d410 0x7fad7c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad7c09afd0 0x7fad7c1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 --2- 172.21.15.148:0/3699145007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad7c09d030 0x7fad7c09aa90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 >> 172.21.15.148:0/3699145007 conn(0x7fad6c0039e0 msgr2=0x7fad7c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 shutdown_connections 2024-04-25T10:01:57.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.039+0000 7fad82bc8640 1 -- 172.21.15.148:0/3699145007 wait complete. 2024-04-25T10:01:57.055 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-25T10:01:55.113629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":34,"acting":34,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":386880,"kb_used_data":12492,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587840,"statfs":{"total":773094113280,"available":772697948160,"internally_reserved":0,"allocated":12791808,"data_stored":6557784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000659"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312163+0000","last_change":"2024-04-25T10:01:53.308968+0000","last_active":"2024-04-25T10:01:54.312163+0000","last_peered":"2024-04-25T10:01:54.312163+0000","last_clean":"2024-04-25T10:01:54.312163+0000","last_became_active":"2024-04-25T10:01:53.308683+0000","last_became_peered":"2024-04-25T10:01:53.308683+0000","last_unstale":"2024-04-25T10:01:54.312163+0000","last_undegraded":"2024-04-25T10:01:54.312163+0000","last_fullsized":"2024-04-25T10:01:54.312163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311504+0000","last_change":"2024-04-25T10:01:53.313912+0000","last_active":"2024-04-25T10:01:54.311504+0000","last_peered":"2024-04-25T10:01:54.311504+0000","last_clean":"2024-04-25T10:01:54.311504+0000","last_became_active":"2024-04-25T10:01:53.313635+0000","last_became_peered":"2024-04-25T10:01:53.313635+0000","last_unstale":"2024-04-25T10:01:54.311504+0000","last_undegraded":"2024-04-25T10:01:54.311504+0000","last_fullsized":"2024-04-25T10:01:54.311504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312000+0000","last_change":"2024-04-25T10:01:53.315244+0000","last_active":"2024-04-25T10:01:54.312000+0000","last_peered":"2024-04-25T10:01:54.312000+0000","last_clean":"2024-04-25T10:01:54.312000+0000","last_became_active":"2024-04-25T10:01:53.314991+0000","last_became_peered":"2024-04-25T10:01:53.314991+0000","last_unstale":"2024-04-25T10:01:54.312000+0000","last_undegraded":"2024-04-25T10:01:54.312000+0000","last_fullsized":"2024-04-25T10:01:54.312000+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:10.260048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311370+0000","last_change":"2024-04-25T10:01:53.314815+0000","last_active":"2024-04-25T10:01:54.311370+0000","last_peered":"2024-04-25T10:01:54.311370+0000","last_clean":"2024-04-25T10:01:54.311370+0000","last_became_active":"2024-04-25T10:01:53.314511+0000","last_became_peered":"2024-04-25T10:01:53.314511+0000","last_unstale":"2024-04-25T10:01:54.311370+0000","last_undegraded":"2024-04-25T10:01:54.311370+0000","last_fullsized":"2024-04-25T10:01:54.311370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347573+0000","last_change":"2024-04-25T10:01:53.307833+0000","last_active":"2024-04-25T10:01:54.347573+0000","last_peered":"2024-04-25T10:01:54.347573+0000","last_clean":"2024-04-25T10:01:54.347573+0000","last_became_active":"2024-04-25T10:01:53.307583+0000","last_became_peered":"2024-04-25T10:01:53.307583+0000","last_unstale":"2024-04-25T10:01:54.347573+0000","last_undegraded":"2024-04-25T10:01:54.347573+0000","last_fullsized":"2024-04-25T10:01:54.347573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:14.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311689+0000","last_change":"2024-04-25T10:01:53.317853+0000","last_active":"2024-04-25T10:01:54.311689+0000","last_peered":"2024-04-25T10:01:54.311689+0000","last_clean":"2024-04-25T10:01:54.311689+0000","last_became_active":"2024-04-25T10:01:53.317561+0000","last_became_peered":"2024-04-25T10:01:53.317561+0000","last_unstale":"2024-04-25T10:01:54.311689+0000","last_undegraded":"2024-04-25T10:01:54.311689+0000","last_fullsized":"2024-04-25T10:01:54.311689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.139972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347834+0000","last_change":"2024-04-25T10:01:53.306396+0000","last_active":"2024-04-25T10:01:54.347834+0000","last_peered":"2024-04-25T10:01:54.347834+0000","last_clean":"2024-04-25T10:01:54.347834+0000","last_became_active":"2024-04-25T10:01:53.306127+0000","last_became_peered":"2024-04-25T10:01:53.306127+0000","last_unstale":"2024-04-25T10:01:54.347834+0000","last_undegraded":"2024-04-25T10:01:54.347834+0000","last_fullsized":"2024-04-25T10:01:54.347834+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:27.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347913+0000","last_change":"2024-04-25T10:01:53.316236+0000","last_active":"2024-04-25T10:01:54.347913+0000","last_peered":"2024-04-25T10:01:54.347913+0000","last_clean":"2024-04-25T10:01:54.347913+0000","last_became_active":"2024-04-25T10:01:53.315521+0000","last_became_peered":"2024-04-25T10:01:53.315521+0000","last_unstale":"2024-04-25T10:01:54.347913+0000","last_undegraded":"2024-04-25T10:01:54.347913+0000","last_fullsized":"2024-04-25T10:01:54.347913+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.725287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311619+0000","last_change":"2024-04-25T10:01:52.300611+0000","last_active":"2024-04-25T10:01:54.311619+0000","last_peered":"2024-04-25T10:01:54.311619+0000","last_clean":"2024-04-25T10:01:54.311619+0000","last_became_active":"2024-04-25T10:01:52.300310+0000","last_became_peered":"2024-04-25T10:01:52.300310+0000","last_unstale":"2024-04-25T10:01:54.311619+0000","last_undegraded":"2024-04-25T10:01:54.311619+0000","last_fullsized":"2024-04-25T10:01:54.311619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311437+0000","last_change":"2024-04-25T10:01:53.308986+0000","last_active":"2024-04-25T10:01:54.311437+0000","last_peered":"2024-04-25T10:01:54.311437+0000","last_clean":"2024-04-25T10:01:54.311437+0000","last_became_active":"2024-04-25T10:01:53.308652+0000","last_became_peered":"2024-04-25T10:01:53.308652+0000","last_unstale":"2024-04-25T10:01:54.311437+0000","last_undegraded":"2024-04-25T10:01:54.311437+0000","last_fullsized":"2024-04-25T10:01:54.311437+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347539+0000","last_change":"2024-04-25T10:01:53.311926+0000","last_active":"2024-04-25T10:01:54.347539+0000","last_peered":"2024-04-25T10:01:54.347539+0000","last_clean":"2024-04-25T10:01:54.347539+0000","last_became_active":"2024-04-25T10:01:53.309452+0000","last_became_peered":"2024-04-25T10:01:53.309452+0000","last_unstale":"2024-04-25T10:01:54.347539+0000","last_undegraded":"2024-04-25T10:01:54.347539+0000","last_fullsized":"2024-04-25T10:01:54.347539+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.619357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312173+0000","last_change":"2024-04-25T10:01:53.315399+0000","last_active":"2024-04-25T10:01:54.312173+0000","last_peered":"2024-04-25T10:01:54.312173+0000","last_clean":"2024-04-25T10:01:54.312173+0000","last_became_active":"2024-04-25T10:01:53.315137+0000","last_became_peered":"2024-04-25T10:01:53.315137+0000","last_unstale":"2024-04-25T10:01:54.312173+0000","last_undegraded":"2024-04-25T10:01:54.312173+0000","last_fullsized":"2024-04-25T10:01:54.312173+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.634999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347825+0000","last_change":"2024-04-25T10:01:52.292697+0000","last_active":"2024-04-25T10:01:54.347825+0000","last_peered":"2024-04-25T10:01:54.347825+0000","last_clean":"2024-04-25T10:01:54.347825+0000","last_became_active":"2024-04-25T10:01:52.292316+0000","last_became_peered":"2024-04-25T10:01:52.292316+0000","last_unstale":"2024-04-25T10:01:54.347825+0000","last_undegraded":"2024-04-25T10:01:54.347825+0000","last_fullsized":"2024-04-25T10:01:54.347825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:31.205159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348125+0000","last_change":"2024-04-25T10:01:53.312259+0000","last_active":"2024-04-25T10:01:54.348125+0000","last_peered":"2024-04-25T10:01:54.348125+0000","last_clean":"2024-04-25T10:01:54.348125+0000","last_became_active":"2024-04-25T10:01:53.311938+0000","last_became_peered":"2024-04-25T10:01:53.311938+0000","last_unstale":"2024-04-25T10:01:54.348125+0000","last_undegraded":"2024-04-25T10:01:54.348125+0000","last_fullsized":"2024-04-25T10:01:54.348125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:11.878572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311723+0000","last_change":"2024-04-25T10:01:52.289539+0000","last_active":"2024-04-25T10:01:54.311723+0000","last_peered":"2024-04-25T10:01:54.311723+0000","last_clean":"2024-04-25T10:01:54.311723+0000","last_became_active":"2024-04-25T10:01:52.289261+0000","last_became_peered":"2024-04-25T10:01:52.289261+0000","last_unstale":"2024-04-25T10:01:54.311723+0000","last_undegraded":"2024-04-25T10:01:54.311723+0000","last_fullsized":"2024-04-25T10:01:54.311723+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311806+0000","last_change":"2024-04-25T10:01:52.292733+0000","last_active":"2024-04-25T10:01:54.311806+0000","last_peered":"2024-04-25T10:01:54.311806+0000","last_clean":"2024-04-25T10:01:54.311806+0000","last_became_active":"2024-04-25T10:01:52.292040+0000","last_became_peered":"2024-04-25T10:01:52.292040+0000","last_unstale":"2024-04-25T10:01:54.311806+0000","last_undegraded":"2024-04-25T10:01:54.311806+0000","last_fullsized":"2024-04-25T10:01:54.311806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:52.278344+0000","last_change":"2024-04-25T10:01:52.278344+0000","last_active":"2024-04-25T10:01:52.278344+0000","last_peered":"2024-04-25T10:01:52.278344+0000","last_clean":"2024-04-25T10:01:52.278344+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:52.278344+0000","last_undegraded":"2024-04-25T10:01:52.278344+0000","last_fullsized":"2024-04-25T10:01:52.278344+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:01:51.272079+0000","last_change":"2024-04-25T10:01:51.272079+0000","last_active":"2024-04-25T10:01:51.272079+0000","last_peered":"2024-04-25T10:01:51.272079+0000","last_clean":"2024-04-25T10:01:51.272079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:01:51.272079+0000","last_undegraded":"2024-04-25T10:01:51.272079+0000","last_fullsized":"2024-04-25T10:01:51.272079+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":345,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-25T10:01:49.019604+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:49.019604+0000","last_peered":"2024-04-25T10:01:49.019604+0000","last_clean":"2024-04-25T10:01:49.019604+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:49.019604+0000","last_undegraded":"2024-04-25T10:01:49.019604+0000","last_fullsized":"2024-04-25T10:01:49.019604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":2},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41492,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330348,"statfs":{"total":96636764160,"available":96594276352,"internally_reserved":0,"allocated":2039808,"data_stored":1259769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46480,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325360,"statfs":{"total":96636764160,"available":96589168640,"internally_reserved":0,"allocated":1445888,"data_stored":669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45072,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326768,"statfs":{"total":96636764160,"available":96590610432,"internally_reserved":0,"allocated":1445888,"data_stored":669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1519616,"data_stored":703267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50640,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1445888,"data_stored":669401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1507328,"data_stored":701655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50596,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321244,"statfs":{"total":96636764160,"available":96584953856,"internally_reserved":0,"allocated":1396736,"data_stored":647261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51176,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320664,"statfs":{"total":96636764160,"available":96584359936,"internally_reserved":0,"allocated":1990656,"data_stored":1237629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:57.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:57.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.189+0000 7efe2355c640 1 Processor -- start 2024-04-25T10:01:57.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.189+0000 7efe2355c640 1 -- start start 2024-04-25T10:01:57.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe2355c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe1c154ee0 0x7efe1c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe2355c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe2355c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c071480 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe2355c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c0ce830 con 0x7efe1c154ee0 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe2355c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c072090 con 0x7efe1c0cfcf0 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe1c154ee0 0x7efe1c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe1c154ee0 0x7efe1c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42776/0 (socket says 172.21.15.148:42776) 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 -- 172.21.15.148:0/1273010655 learned_addr learned my addr 172.21.15.148:0/1273010655 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe20af7640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 -- 172.21.15.148:0/1273010655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 msgr2=0x7efe1c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 -- 172.21.15.148:0/1273010655 >> v1:172.21.15.188:6789/0 conn(0x7efe1c0ce3c0 legacy=0x7efe1c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe212f8640 1 -- 172.21.15.148:0/1273010655 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c071760 con 0x7efe1c154ee0 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.190+0000 7efe20af7640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.191+0000 7efe212f8640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c154ee0 0x7efe1c0cf7b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7efe1800bb10 tx=0x7efe1802c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b226724f2ba1d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.191+0000 7efe13fff640 1 -- 172.21.15.148:0/1273010655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18032790 con 0x7efe1c154ee0 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.191+0000 7efe13fff640 1 -- 172.21.15.148:0/1273010655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe18038070 con 0x7efe1c154ee0 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.191+0000 7efe13fff640 1 -- 172.21.15.148:0/1273010655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18032b50 con 0x7efe1c154ee0 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 -- 172.21.15.148:0/1273010655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c154ee0 msgr2=0x7efe1c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c154ee0 0x7efe1c0cf7b0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7efe1800bb10 tx=0x7efe1802c8d0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 -- 172.21.15.148:0/1273010655 shutdown_connections 2024-04-25T10:01:57.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 --2- 172.21.15.148:0/1273010655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c154ee0 0x7efe1c0cf7b0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 -- 172.21.15.148:0/1273010655 >> 172.21.15.148:0/1273010655 conn(0x7efe1c06abe0 msgr2=0x7efe1c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 -- 172.21.15.148:0/1273010655 shutdown_connections 2024-04-25T10:01:57.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.192+0000 7efe2355c640 1 -- 172.21.15.148:0/1273010655 wait complete. 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 Processor -- start 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 -- start start 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 0x7efe1c1ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1c154ee0 0x7efe1c151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c071230 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.193+0000 7efe2355c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c071d20 con 0x7efe1c154ee0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe2355c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c1552c0 con 0x7efe1c0cfcf0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 0x7efe1c1ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 0x7efe1c1ef260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42792/0 (socket says 172.21.15.148:42792) 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 -- 172.21.15.148:0/3854886856 learned_addr learned my addr 172.21.15.148:0/3854886856 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 -- 172.21.15.148:0/3854886856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 msgr2=0x7efe1c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 -- 172.21.15.148:0/3854886856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1c154ee0 msgr2=0x7efe1c151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1c154ee0 0x7efe1c151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 -- 172.21.15.148:0/3854886856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c13eb80 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe212f8640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 0x7efe1c1ef260 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b2c0 tx=0x7efe180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18033620 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe180337c0 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18033bb0 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c1e4970 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.194+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c14f220 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.195+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1c073990 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.196+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe1804d060 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.196+0000 7efe11ffb640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe0006b4d0 0x7efe0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.196+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe180acfa0 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.196+0000 7efe20af7640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe0006b4d0 0x7efe0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.197+0000 7efe20af7640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe0006b4d0 0x7efe0006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efe0c006fd0 tx=0x7efe0c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.201+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe18041070 con 0x7efe1c0ce3c0 2024-04-25T10:01:57.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.367+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe1c06d360 con 0x7efe0006b4d0 2024-04-25T10:01:57.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.373+0000 7efe11ffb640 1 -- 172.21.15.148:0/3854886856 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7efe1c06d360 con 0x7efe0006b4d0 2024-04-25T10:01:57.381 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:57.382 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:01:57.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe0006b4d0 msgr2=0x7efe0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe0006b4d0 0x7efe0006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efe0c006fd0 tx=0x7efe0c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:57.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 msgr2=0x7efe1c1ef260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 0x7efe1c1ef260 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b2c0 tx=0x7efe180046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 shutdown_connections 2024-04-25T10:01:57.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe0006b4d0 0x7efe0006d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe1c154ee0 0x7efe1c151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe1c0cfcf0 0x7efe1c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 --2- 172.21.15.148:0/3854886856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe1c0ce3c0 0x7efe1c1ef260 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 >> 172.21.15.148:0/3854886856 conn(0x7efe1c06abe0 msgr2=0x7efe1c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 shutdown_connections 2024-04-25T10:01:57.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.376+0000 7efe2355c640 1 -- 172.21.15.148:0/3854886856 wait complete. 2024-04-25T10:01:57.392 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-04-25T10:01:57.113907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":387356,"kb_used_data":12964,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587364,"statfs":{"total":773094113280,"available":772697460736,"internally_reserved":0,"allocated":13275136,"data_stored":6778720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.832860"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312163+0000","last_change":"2024-04-25T10:01:53.308968+0000","last_active":"2024-04-25T10:01:54.312163+0000","last_peered":"2024-04-25T10:01:54.312163+0000","last_clean":"2024-04-25T10:01:54.312163+0000","last_became_active":"2024-04-25T10:01:53.308683+0000","last_became_peered":"2024-04-25T10:01:53.308683+0000","last_unstale":"2024-04-25T10:01:54.312163+0000","last_undegraded":"2024-04-25T10:01:54.312163+0000","last_fullsized":"2024-04-25T10:01:54.312163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348370+0000","last_change":"2024-04-25T10:01:53.326585+0000","last_active":"2024-04-25T10:01:54.348370+0000","last_peered":"2024-04-25T10:01:54.348370+0000","last_clean":"2024-04-25T10:01:54.348370+0000","last_became_active":"2024-04-25T10:01:53.325978+0000","last_became_peered":"2024-04-25T10:01:53.325978+0000","last_unstale":"2024-04-25T10:01:54.348370+0000","last_undegraded":"2024-04-25T10:01:54.348370+0000","last_fullsized":"2024-04-25T10:01:54.348370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:04.286812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348198+0000","last_change":"2024-04-25T10:01:53.319634+0000","last_active":"2024-04-25T10:01:54.348198+0000","last_peered":"2024-04-25T10:01:54.348198+0000","last_clean":"2024-04-25T10:01:54.348198+0000","last_became_active":"2024-04-25T10:01:53.319370+0000","last_became_peered":"2024-04-25T10:01:53.319370+0000","last_unstale":"2024-04-25T10:01:54.348198+0000","last_undegraded":"2024-04-25T10:01:54.348198+0000","last_fullsized":"2024-04-25T10:01:54.348198+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:18.034791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348276+0000","last_change":"2024-04-25T10:01:53.314499+0000","last_active":"2024-04-25T10:01:54.348276+0000","last_peered":"2024-04-25T10:01:54.348276+0000","last_clean":"2024-04-25T10:01:54.348276+0000","last_became_active":"2024-04-25T10:01:53.314191+0000","last_became_peered":"2024-04-25T10:01:53.314191+0000","last_unstale":"2024-04-25T10:01:54.348276+0000","last_undegraded":"2024-04-25T10:01:54.348276+0000","last_fullsized":"2024-04-25T10:01:54.348276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:26.199709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311504+0000","last_change":"2024-04-25T10:01:53.313912+0000","last_active":"2024-04-25T10:01:54.311504+0000","last_peered":"2024-04-25T10:01:54.311504+0000","last_clean":"2024-04-25T10:01:54.311504+0000","last_became_active":"2024-04-25T10:01:53.313635+0000","last_became_peered":"2024-04-25T10:01:53.313635+0000","last_unstale":"2024-04-25T10:01:54.311504+0000","last_undegraded":"2024-04-25T10:01:54.311504+0000","last_fullsized":"2024-04-25T10:01:54.311504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320356+0000","last_change":"2024-04-25T10:01:53.317523+0000","last_active":"2024-04-25T10:01:54.320356+0000","last_peered":"2024-04-25T10:01:54.320356+0000","last_clean":"2024-04-25T10:01:54.320356+0000","last_became_active":"2024-04-25T10:01:53.317248+0000","last_became_peered":"2024-04-25T10:01:53.317248+0000","last_unstale":"2024-04-25T10:01:54.320356+0000","last_undegraded":"2024-04-25T10:01:54.320356+0000","last_fullsized":"2024-04-25T10:01:54.320356+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:41.914139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320327+0000","last_change":"2024-04-25T10:01:53.314768+0000","last_active":"2024-04-25T10:01:54.320327+0000","last_peered":"2024-04-25T10:01:54.320327+0000","last_clean":"2024-04-25T10:01:54.320327+0000","last_became_active":"2024-04-25T10:01:53.314504+0000","last_became_peered":"2024-04-25T10:01:53.314504+0000","last_unstale":"2024-04-25T10:01:54.320327+0000","last_undegraded":"2024-04-25T10:01:54.320327+0000","last_fullsized":"2024-04-25T10:01:54.320327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:40.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348425+0000","last_change":"2024-04-25T10:01:53.320926+0000","last_active":"2024-04-25T10:01:54.348425+0000","last_peered":"2024-04-25T10:01:54.348425+0000","last_clean":"2024-04-25T10:01:54.348425+0000","last_became_active":"2024-04-25T10:01:53.320663+0000","last_became_peered":"2024-04-25T10:01:53.320663+0000","last_unstale":"2024-04-25T10:01:54.348425+0000","last_undegraded":"2024-04-25T10:01:54.348425+0000","last_fullsized":"2024-04-25T10:01:54.348425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:38.131544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348471+0000","last_change":"2024-04-25T10:01:53.320793+0000","last_active":"2024-04-25T10:01:54.348471+0000","last_peered":"2024-04-25T10:01:54.348471+0000","last_clean":"2024-04-25T10:01:54.348471+0000","last_became_active":"2024-04-25T10:01:53.320516+0000","last_became_peered":"2024-04-25T10:01:53.320516+0000","last_unstale":"2024-04-25T10:01:54.348471+0000","last_undegraded":"2024-04-25T10:01:54.348471+0000","last_fullsized":"2024-04-25T10:01:54.348471+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:21.226281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348274+0000","last_change":"2024-04-25T10:01:53.308828+0000","last_active":"2024-04-25T10:01:54.348274+0000","last_peered":"2024-04-25T10:01:54.348274+0000","last_clean":"2024-04-25T10:01:54.348274+0000","last_became_active":"2024-04-25T10:01:53.308560+0000","last_became_peered":"2024-04-25T10:01:53.308560+0000","last_unstale":"2024-04-25T10:01:54.348274+0000","last_undegraded":"2024-04-25T10:01:54.348274+0000","last_fullsized":"2024-04-25T10:01:54.348274+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312000+0000","last_change":"2024-04-25T10:01:53.315244+0000","last_active":"2024-04-25T10:01:54.312000+0000","last_peered":"2024-04-25T10:01:54.312000+0000","last_clean":"2024-04-25T10:01:54.312000+0000","last_became_active":"2024-04-25T10:01:53.314991+0000","last_became_peered":"2024-04-25T10:01:53.314991+0000","last_unstale":"2024-04-25T10:01:54.312000+0000","last_undegraded":"2024-04-25T10:01:54.312000+0000","last_fullsized":"2024-04-25T10:01:54.312000+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:10.260048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320663+0000","last_change":"2024-04-25T10:01:53.314287+0000","last_active":"2024-04-25T10:01:54.320663+0000","last_peered":"2024-04-25T10:01:54.320663+0000","last_clean":"2024-04-25T10:01:54.320663+0000","last_became_active":"2024-04-25T10:01:53.313201+0000","last_became_peered":"2024-04-25T10:01:53.313201+0000","last_unstale":"2024-04-25T10:01:54.320663+0000","last_undegraded":"2024-04-25T10:01:54.320663+0000","last_fullsized":"2024-04-25T10:01:54.320663+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.462869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347955+0000","last_change":"2024-04-25T10:01:53.307341+0000","last_active":"2024-04-25T10:01:54.347955+0000","last_peered":"2024-04-25T10:01:54.347955+0000","last_clean":"2024-04-25T10:01:54.347955+0000","last_became_active":"2024-04-25T10:01:53.307024+0000","last_became_peered":"2024-04-25T10:01:53.307024+0000","last_unstale":"2024-04-25T10:01:54.347955+0000","last_undegraded":"2024-04-25T10:01:54.347955+0000","last_fullsized":"2024-04-25T10:01:54.347955+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:40.966537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348781+0000","last_change":"2024-04-25T10:01:53.326968+0000","last_active":"2024-04-25T10:01:54.348781+0000","last_peered":"2024-04-25T10:01:54.348781+0000","last_clean":"2024-04-25T10:01:54.348781+0000","last_became_active":"2024-04-25T10:01:53.326663+0000","last_became_peered":"2024-04-25T10:01:53.326663+0000","last_unstale":"2024-04-25T10:01:54.348781+0000","last_undegraded":"2024-04-25T10:01:54.348781+0000","last_fullsized":"2024-04-25T10:01:54.348781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:03.834566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311370+0000","last_change":"2024-04-25T10:01:53.314815+0000","last_active":"2024-04-25T10:01:54.311370+0000","last_peered":"2024-04-25T10:01:54.311370+0000","last_clean":"2024-04-25T10:01:54.311370+0000","last_became_active":"2024-04-25T10:01:53.314511+0000","last_became_peered":"2024-04-25T10:01:53.314511+0000","last_unstale":"2024-04-25T10:01:54.311370+0000","last_undegraded":"2024-04-25T10:01:54.311370+0000","last_fullsized":"2024-04-25T10:01:54.311370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320222+0000","last_change":"2024-04-25T10:01:53.309219+0000","last_active":"2024-04-25T10:01:54.320222+0000","last_peered":"2024-04-25T10:01:54.320222+0000","last_clean":"2024-04-25T10:01:54.320222+0000","last_became_active":"2024-04-25T10:01:53.308965+0000","last_became_peered":"2024-04-25T10:01:53.308965+0000","last_unstale":"2024-04-25T10:01:54.320222+0000","last_undegraded":"2024-04-25T10:01:54.320222+0000","last_fullsized":"2024-04-25T10:01:54.320222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:00.180829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348329+0000","last_change":"2024-04-25T10:01:53.315092+0000","last_active":"2024-04-25T10:01:54.348329+0000","last_peered":"2024-04-25T10:01:54.348329+0000","last_clean":"2024-04-25T10:01:54.348329+0000","last_became_active":"2024-04-25T10:01:53.314365+0000","last_became_peered":"2024-04-25T10:01:53.314365+0000","last_unstale":"2024-04-25T10:01:54.348329+0000","last_undegraded":"2024-04-25T10:01:54.348329+0000","last_fullsized":"2024-04-25T10:01:54.348329+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348538+0000","last_change":"2024-04-25T10:01:53.326698+0000","last_active":"2024-04-25T10:01:54.348538+0000","last_peered":"2024-04-25T10:01:54.348538+0000","last_clean":"2024-04-25T10:01:54.348538+0000","last_became_active":"2024-04-25T10:01:53.325993+0000","last_became_peered":"2024-04-25T10:01:53.325993+0000","last_unstale":"2024-04-25T10:01:54.348538+0000","last_undegraded":"2024-04-25T10:01:54.348538+0000","last_fullsized":"2024-04-25T10:01:54.348538+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318420+0000","last_change":"2024-04-25T10:01:53.320785+0000","last_active":"2024-04-25T10:01:54.318420+0000","last_peered":"2024-04-25T10:01:54.318420+0000","last_clean":"2024-04-25T10:01:54.318420+0000","last_became_active":"2024-04-25T10:01:53.319963+0000","last_became_peered":"2024-04-25T10:01:53.319963+0000","last_unstale":"2024-04-25T10:01:54.318420+0000","last_undegraded":"2024-04-25T10:01:54.318420+0000","last_fullsized":"2024-04-25T10:01:54.318420+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347573+0000","last_change":"2024-04-25T10:01:53.307833+0000","last_active":"2024-04-25T10:01:54.347573+0000","last_peered":"2024-04-25T10:01:54.347573+0000","last_clean":"2024-04-25T10:01:54.347573+0000","last_became_active":"2024-04-25T10:01:53.307583+0000","last_became_peered":"2024-04-25T10:01:53.307583+0000","last_unstale":"2024-04-25T10:01:54.347573+0000","last_undegraded":"2024-04-25T10:01:54.347573+0000","last_fullsized":"2024-04-25T10:01:54.347573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:14.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348432+0000","last_change":"2024-04-25T10:01:53.313805+0000","last_active":"2024-04-25T10:01:54.348432+0000","last_peered":"2024-04-25T10:01:54.348432+0000","last_clean":"2024-04-25T10:01:54.348432+0000","last_became_active":"2024-04-25T10:01:53.313489+0000","last_became_peered":"2024-04-25T10:01:53.313489+0000","last_unstale":"2024-04-25T10:01:54.348432+0000","last_undegraded":"2024-04-25T10:01:54.348432+0000","last_fullsized":"2024-04-25T10:01:54.348432+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:57.394066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311689+0000","last_change":"2024-04-25T10:01:53.317853+0000","last_active":"2024-04-25T10:01:54.311689+0000","last_peered":"2024-04-25T10:01:54.311689+0000","last_clean":"2024-04-25T10:01:54.311689+0000","last_became_active":"2024-04-25T10:01:53.317561+0000","last_became_peered":"2024-04-25T10:01:53.317561+0000","last_unstale":"2024-04-25T10:01:54.311689+0000","last_undegraded":"2024-04-25T10:01:54.311689+0000","last_fullsized":"2024-04-25T10:01:54.311689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.139972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348127+0000","last_change":"2024-04-25T10:01:53.318514+0000","last_active":"2024-04-25T10:01:54.348127+0000","last_peered":"2024-04-25T10:01:54.348127+0000","last_clean":"2024-04-25T10:01:54.348127+0000","last_became_active":"2024-04-25T10:01:53.318206+0000","last_became_peered":"2024-04-25T10:01:53.318206+0000","last_unstale":"2024-04-25T10:01:54.348127+0000","last_undegraded":"2024-04-25T10:01:54.348127+0000","last_fullsized":"2024-04-25T10:01:54.348127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:08.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.309900+0000","last_change":"2024-04-25T10:01:53.316554+0000","last_active":"2024-04-25T10:01:54.309900+0000","last_peered":"2024-04-25T10:01:54.309900+0000","last_clean":"2024-04-25T10:01:54.309900+0000","last_became_active":"2024-04-25T10:01:53.316218+0000","last_became_peered":"2024-04-25T10:01:53.316218+0000","last_unstale":"2024-04-25T10:01:54.309900+0000","last_undegraded":"2024-04-25T10:01:54.309900+0000","last_fullsized":"2024-04-25T10:01:54.309900+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.340419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310385+0000","last_change":"2024-04-25T10:01:53.307075+0000","last_active":"2024-04-25T10:01:54.310385+0000","last_peered":"2024-04-25T10:01:54.310385+0000","last_clean":"2024-04-25T10:01:54.310385+0000","last_became_active":"2024-04-25T10:01:53.304942+0000","last_became_peered":"2024-04-25T10:01:53.304942+0000","last_unstale":"2024-04-25T10:01:54.310385+0000","last_undegraded":"2024-04-25T10:01:54.310385+0000","last_fullsized":"2024-04-25T10:01:54.310385+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:21.226281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347834+0000","last_change":"2024-04-25T10:01:53.306396+0000","last_active":"2024-04-25T10:01:54.347834+0000","last_peered":"2024-04-25T10:01:54.347834+0000","last_clean":"2024-04-25T10:01:54.347834+0000","last_became_active":"2024-04-25T10:01:53.306127+0000","last_became_peered":"2024-04-25T10:01:53.306127+0000","last_unstale":"2024-04-25T10:01:54.347834+0000","last_undegraded":"2024-04-25T10:01:54.347834+0000","last_fullsized":"2024-04-25T10:01:54.347834+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:27.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318479+0000","last_change":"2024-04-25T10:01:53.319940+0000","last_active":"2024-04-25T10:01:54.318479+0000","last_peered":"2024-04-25T10:01:54.318479+0000","last_clean":"2024-04-25T10:01:54.318479+0000","last_became_active":"2024-04-25T10:01:53.319590+0000","last_became_peered":"2024-04-25T10:01:53.319590+0000","last_unstale":"2024-04-25T10:01:54.318479+0000","last_undegraded":"2024-04-25T10:01:54.318479+0000","last_fullsized":"2024-04-25T10:01:54.318479+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320676+0000","last_change":"2024-04-25T10:01:53.316178+0000","last_active":"2024-04-25T10:01:54.320676+0000","last_peered":"2024-04-25T10:01:54.320676+0000","last_clean":"2024-04-25T10:01:54.320676+0000","last_became_active":"2024-04-25T10:01:53.314889+0000","last_became_peered":"2024-04-25T10:01:53.314889+0000","last_unstale":"2024-04-25T10:01:54.320676+0000","last_undegraded":"2024-04-25T10:01:54.320676+0000","last_fullsized":"2024-04-25T10:01:54.320676+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:28.026534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347913+0000","last_change":"2024-04-25T10:01:53.316236+0000","last_active":"2024-04-25T10:01:54.347913+0000","last_peered":"2024-04-25T10:01:54.347913+0000","last_clean":"2024-04-25T10:01:54.347913+0000","last_became_active":"2024-04-25T10:01:53.315521+0000","last_became_peered":"2024-04-25T10:01:53.315521+0000","last_unstale":"2024-04-25T10:01:54.347913+0000","last_undegraded":"2024-04-25T10:01:54.347913+0000","last_fullsized":"2024-04-25T10:01:54.347913+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.725287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348895+0000","last_change":"2024-04-25T10:01:53.317694+0000","last_active":"2024-04-25T10:01:54.348895+0000","last_peered":"2024-04-25T10:01:54.348895+0000","last_clean":"2024-04-25T10:01:54.348895+0000","last_became_active":"2024-04-25T10:01:53.317254+0000","last_became_peered":"2024-04-25T10:01:53.317254+0000","last_unstale":"2024-04-25T10:01:54.348895+0000","last_undegraded":"2024-04-25T10:01:54.348895+0000","last_fullsized":"2024-04-25T10:01:54.348895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:59.725012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310217+0000","last_change":"2024-04-25T10:01:53.303113+0000","last_active":"2024-04-25T10:01:54.310217+0000","last_peered":"2024-04-25T10:01:54.310217+0000","last_clean":"2024-04-25T10:01:54.310217+0000","last_became_active":"2024-04-25T10:01:53.302788+0000","last_became_peered":"2024-04-25T10:01:53.302788+0000","last_unstale":"2024-04-25T10:01:54.310217+0000","last_undegraded":"2024-04-25T10:01:54.310217+0000","last_fullsized":"2024-04-25T10:01:54.310217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:56.754763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.309915+0000","last_change":"2024-04-25T10:01:53.316148+0000","last_active":"2024-04-25T10:01:54.309915+0000","last_peered":"2024-04-25T10:01:54.309915+0000","last_clean":"2024-04-25T10:01:54.309915+0000","last_became_active":"2024-04-25T10:01:53.315873+0000","last_became_peered":"2024-04-25T10:01:53.315873+0000","last_unstale":"2024-04-25T10:01:54.309915+0000","last_undegraded":"2024-04-25T10:01:54.309915+0000","last_fullsized":"2024-04-25T10:01:54.309915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:08.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348109+0000","last_change":"2024-04-25T10:01:53.310855+0000","last_active":"2024-04-25T10:01:54.348109+0000","last_peered":"2024-04-25T10:01:54.348109+0000","last_clean":"2024-04-25T10:01:54.348109+0000","last_became_active":"2024-04-25T10:01:53.310567+0000","last_became_peered":"2024-04-25T10:01:53.310567+0000","last_unstale":"2024-04-25T10:01:54.348109+0000","last_undegraded":"2024-04-25T10:01:54.348109+0000","last_fullsized":"2024-04-25T10:01:54.348109+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.215530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318170+0000","last_change":"2024-04-25T10:01:52.296531+0000","last_active":"2024-04-25T10:01:54.318170+0000","last_peered":"2024-04-25T10:01:54.318170+0000","last_clean":"2024-04-25T10:01:54.318170+0000","last_became_active":"2024-04-25T10:01:52.296192+0000","last_became_peered":"2024-04-25T10:01:52.296192+0000","last_unstale":"2024-04-25T10:01:54.318170+0000","last_undegraded":"2024-04-25T10:01:54.318170+0000","last_fullsized":"2024-04-25T10:01:54.318170+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320541+0000","last_change":"2024-04-25T10:01:53.314991+0000","last_active":"2024-04-25T10:01:54.320541+0000","last_peered":"2024-04-25T10:01:54.320541+0000","last_clean":"2024-04-25T10:01:54.320541+0000","last_became_active":"2024-04-25T10:01:53.314698+0000","last_became_peered":"2024-04-25T10:01:53.314698+0000","last_unstale":"2024-04-25T10:01:54.320541+0000","last_undegraded":"2024-04-25T10:01:54.320541+0000","last_fullsized":"2024-04-25T10:01:54.320541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:49.360827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348276+0000","last_change":"2024-04-25T10:01:52.299627+0000","last_active":"2024-04-25T10:01:54.348276+0000","last_peered":"2024-04-25T10:01:54.348276+0000","last_clean":"2024-04-25T10:01:54.348276+0000","last_became_active":"2024-04-25T10:01:52.299139+0000","last_became_peered":"2024-04-25T10:01:52.299139+0000","last_unstale":"2024-04-25T10:01:54.348276+0000","last_undegraded":"2024-04-25T10:01:54.348276+0000","last_fullsized":"2024-04-25T10:01:54.348276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.686423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310355+0000","last_change":"2024-04-25T10:01:53.310586+0000","last_active":"2024-04-25T10:01:54.310355+0000","last_peered":"2024-04-25T10:01:54.310355+0000","last_clean":"2024-04-25T10:01:54.310355+0000","last_became_active":"2024-04-25T10:01:53.309807+0000","last_became_peered":"2024-04-25T10:01:53.309807+0000","last_unstale":"2024-04-25T10:01:54.310355+0000","last_undegraded":"2024-04-25T10:01:54.310355+0000","last_fullsized":"2024-04-25T10:01:54.310355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:42.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348318+0000","last_change":"2024-04-25T10:01:52.300227+0000","last_active":"2024-04-25T10:01:54.348318+0000","last_peered":"2024-04-25T10:01:54.348318+0000","last_clean":"2024-04-25T10:01:54.348318+0000","last_became_active":"2024-04-25T10:01:52.299809+0000","last_became_peered":"2024-04-25T10:01:52.299809+0000","last_unstale":"2024-04-25T10:01:54.348318+0000","last_undegraded":"2024-04-25T10:01:54.348318+0000","last_fullsized":"2024-04-25T10:01:54.348318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:22.491159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348484+0000","last_change":"2024-04-25T10:01:53.313976+0000","last_active":"2024-04-25T10:01:54.348484+0000","last_peered":"2024-04-25T10:01:54.348484+0000","last_clean":"2024-04-25T10:01:54.348484+0000","last_became_active":"2024-04-25T10:01:53.313672+0000","last_became_peered":"2024-04-25T10:01:53.313672+0000","last_unstale":"2024-04-25T10:01:54.348484+0000","last_undegraded":"2024-04-25T10:01:54.348484+0000","last_fullsized":"2024-04-25T10:01:54.348484+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.512462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348104+0000","last_change":"2024-04-25T10:01:52.304486+0000","last_active":"2024-04-25T10:01:54.348104+0000","last_peered":"2024-04-25T10:01:54.348104+0000","last_clean":"2024-04-25T10:01:54.348104+0000","last_became_active":"2024-04-25T10:01:52.304196+0000","last_became_peered":"2024-04-25T10:01:52.304196+0000","last_unstale":"2024-04-25T10:01:54.348104+0000","last_undegraded":"2024-04-25T10:01:54.348104+0000","last_fullsized":"2024-04-25T10:01:54.348104+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:39.943512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347804+0000","last_change":"2024-04-25T10:01:53.315595+0000","last_active":"2024-04-25T10:01:54.347804+0000","last_peered":"2024-04-25T10:01:54.347804+0000","last_clean":"2024-04-25T10:01:54.347804+0000","last_became_active":"2024-04-25T10:01:53.315254+0000","last_became_peered":"2024-04-25T10:01:53.315254+0000","last_unstale":"2024-04-25T10:01:54.347804+0000","last_undegraded":"2024-04-25T10:01:54.347804+0000","last_fullsized":"2024-04-25T10:01:54.347804+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:28.489392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311619+0000","last_change":"2024-04-25T10:01:52.300611+0000","last_active":"2024-04-25T10:01:54.311619+0000","last_peered":"2024-04-25T10:01:54.311619+0000","last_clean":"2024-04-25T10:01:54.311619+0000","last_became_active":"2024-04-25T10:01:52.300310+0000","last_became_peered":"2024-04-25T10:01:52.300310+0000","last_unstale":"2024-04-25T10:01:54.311619+0000","last_undegraded":"2024-04-25T10:01:54.311619+0000","last_fullsized":"2024-04-25T10:01:54.311619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311437+0000","last_change":"2024-04-25T10:01:53.308986+0000","last_active":"2024-04-25T10:01:54.311437+0000","last_peered":"2024-04-25T10:01:54.311437+0000","last_clean":"2024-04-25T10:01:54.311437+0000","last_became_active":"2024-04-25T10:01:53.308652+0000","last_became_peered":"2024-04-25T10:01:53.308652+0000","last_unstale":"2024-04-25T10:01:54.311437+0000","last_undegraded":"2024-04-25T10:01:54.311437+0000","last_fullsized":"2024-04-25T10:01:54.311437+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320437+0000","last_change":"2024-04-25T10:01:52.301727+0000","last_active":"2024-04-25T10:01:54.320437+0000","last_peered":"2024-04-25T10:01:54.320437+0000","last_clean":"2024-04-25T10:01:54.320437+0000","last_became_active":"2024-04-25T10:01:52.301473+0000","last_became_peered":"2024-04-25T10:01:52.301473+0000","last_unstale":"2024-04-25T10:01:54.320437+0000","last_undegraded":"2024-04-25T10:01:54.320437+0000","last_fullsized":"2024-04-25T10:01:54.320437+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:29.764946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348929+0000","last_change":"2024-04-25T10:01:53.321778+0000","last_active":"2024-04-25T10:01:54.348929+0000","last_peered":"2024-04-25T10:01:54.348929+0000","last_clean":"2024-04-25T10:01:54.348929+0000","last_became_active":"2024-04-25T10:01:53.318367+0000","last_became_peered":"2024-04-25T10:01:53.318367+0000","last_unstale":"2024-04-25T10:01:54.348929+0000","last_undegraded":"2024-04-25T10:01:54.348929+0000","last_fullsized":"2024-04-25T10:01:54.348929+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:43.815198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310339+0000","last_change":"2024-04-25T10:01:52.292390+0000","last_active":"2024-04-25T10:01:54.310339+0000","last_peered":"2024-04-25T10:01:54.310339+0000","last_clean":"2024-04-25T10:01:54.310339+0000","last_became_active":"2024-04-25T10:01:52.292077+0000","last_became_peered":"2024-04-25T10:01:52.292077+0000","last_unstale":"2024-04-25T10:01:54.310339+0000","last_undegraded":"2024-04-25T10:01:54.310339+0000","last_fullsized":"2024-04-25T10:01:54.310339+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:39.960271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348105+0000","last_change":"2024-04-25T10:01:53.314768+0000","last_active":"2024-04-25T10:01:54.348105+0000","last_peered":"2024-04-25T10:01:54.348105+0000","last_clean":"2024-04-25T10:01:54.348105+0000","last_became_active":"2024-04-25T10:01:53.314481+0000","last_became_peered":"2024-04-25T10:01:53.314481+0000","last_unstale":"2024-04-25T10:01:54.348105+0000","last_undegraded":"2024-04-25T10:01:54.348105+0000","last_fullsized":"2024-04-25T10:01:54.348105+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:23.497424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320449+0000","last_change":"2024-04-25T10:01:52.303125+0000","last_active":"2024-04-25T10:01:54.320449+0000","last_peered":"2024-04-25T10:01:54.320449+0000","last_clean":"2024-04-25T10:01:54.320449+0000","last_became_active":"2024-04-25T10:01:52.302491+0000","last_became_peered":"2024-04-25T10:01:52.302491+0000","last_unstale":"2024-04-25T10:01:54.320449+0000","last_undegraded":"2024-04-25T10:01:54.320449+0000","last_fullsized":"2024-04-25T10:01:54.320449+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:18.878013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318179+0000","last_change":"2024-04-25T10:01:53.320517+0000","last_active":"2024-04-25T10:01:54.318179+0000","last_peered":"2024-04-25T10:01:54.318179+0000","last_clean":"2024-04-25T10:01:54.318179+0000","last_became_active":"2024-04-25T10:01:53.319589+0000","last_became_peered":"2024-04-25T10:01:53.319589+0000","last_unstale":"2024-04-25T10:01:54.318179+0000","last_undegraded":"2024-04-25T10:01:54.318179+0000","last_fullsized":"2024-04-25T10:01:54.318179+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.215530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318256+0000","last_change":"2024-04-25T10:01:52.301928+0000","last_active":"2024-04-25T10:01:54.318256+0000","last_peered":"2024-04-25T10:01:54.318256+0000","last_clean":"2024-04-25T10:01:54.318256+0000","last_became_active":"2024-04-25T10:01:52.301620+0000","last_became_peered":"2024-04-25T10:01:52.301620+0000","last_unstale":"2024-04-25T10:01:54.318256+0000","last_undegraded":"2024-04-25T10:01:54.318256+0000","last_fullsized":"2024-04-25T10:01:54.318256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310596+0000","last_change":"2024-04-25T10:01:53.310558+0000","last_active":"2024-04-25T10:01:54.310596+0000","last_peered":"2024-04-25T10:01:54.310596+0000","last_clean":"2024-04-25T10:01:54.310596+0000","last_became_active":"2024-04-25T10:01:53.310042+0000","last_became_peered":"2024-04-25T10:01:53.310042+0000","last_unstale":"2024-04-25T10:01:54.310596+0000","last_undegraded":"2024-04-25T10:01:54.310596+0000","last_fullsized":"2024-04-25T10:01:54.310596+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:04.286812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310155+0000","last_change":"2024-04-25T10:01:52.293082+0000","last_active":"2024-04-25T10:01:54.310155+0000","last_peered":"2024-04-25T10:01:54.310155+0000","last_clean":"2024-04-25T10:01:54.310155+0000","last_became_active":"2024-04-25T10:01:52.292730+0000","last_became_peered":"2024-04-25T10:01:52.292730+0000","last_unstale":"2024-04-25T10:01:54.310155+0000","last_undegraded":"2024-04-25T10:01:54.310155+0000","last_fullsized":"2024-04-25T10:01:54.310155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:22.491159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348694+0000","last_change":"2024-04-25T10:01:53.319606+0000","last_active":"2024-04-25T10:01:54.348694+0000","last_peered":"2024-04-25T10:01:54.348694+0000","last_clean":"2024-04-25T10:01:54.348694+0000","last_became_active":"2024-04-25T10:01:53.319303+0000","last_became_peered":"2024-04-25T10:01:53.319303+0000","last_unstale":"2024-04-25T10:01:54.348694+0000","last_undegraded":"2024-04-25T10:01:54.348694+0000","last_fullsized":"2024-04-25T10:01:54.348694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:31.529520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310559+0000","last_change":"2024-04-25T10:01:52.296284+0000","last_active":"2024-04-25T10:01:54.310559+0000","last_peered":"2024-04-25T10:01:54.310559+0000","last_clean":"2024-04-25T10:01:54.310559+0000","last_became_active":"2024-04-25T10:01:52.295533+0000","last_became_peered":"2024-04-25T10:01:52.295533+0000","last_unstale":"2024-04-25T10:01:54.310559+0000","last_undegraded":"2024-04-25T10:01:54.310559+0000","last_fullsized":"2024-04-25T10:01:54.310559+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.557737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310419+0000","last_change":"2024-04-25T10:01:53.306729+0000","last_active":"2024-04-25T10:01:54.310419+0000","last_peered":"2024-04-25T10:01:54.310419+0000","last_clean":"2024-04-25T10:01:54.310419+0000","last_became_active":"2024-04-25T10:01:53.305882+0000","last_became_peered":"2024-04-25T10:01:53.305882+0000","last_unstale":"2024-04-25T10:01:54.310419+0000","last_undegraded":"2024-04-25T10:01:54.310419+0000","last_fullsized":"2024-04-25T10:01:54.310419+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.512462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349089+0000","last_change":"2024-04-25T10:01:52.303545+0000","last_active":"2024-04-25T10:01:54.349089+0000","last_peered":"2024-04-25T10:01:54.349089+0000","last_clean":"2024-04-25T10:01:54.349089+0000","last_became_active":"2024-04-25T10:01:52.303100+0000","last_became_peered":"2024-04-25T10:01:52.303100+0000","last_unstale":"2024-04-25T10:01:54.349089+0000","last_undegraded":"2024-04-25T10:01:54.349089+0000","last_fullsized":"2024-04-25T10:01:54.349089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:54.567517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318394+0000","last_change":"2024-04-25T10:01:53.327840+0000","last_active":"2024-04-25T10:01:54.318394+0000","last_peered":"2024-04-25T10:01:54.318394+0000","last_clean":"2024-04-25T10:01:54.318394+0000","last_became_active":"2024-04-25T10:01:53.327636+0000","last_became_peered":"2024-04-25T10:01:53.327636+0000","last_unstale":"2024-04-25T10:01:54.318394+0000","last_undegraded":"2024-04-25T10:01:54.318394+0000","last_fullsized":"2024-04-25T10:01:54.318394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348429+0000","last_change":"2024-04-25T10:01:52.301182+0000","last_active":"2024-04-25T10:01:54.348429+0000","last_peered":"2024-04-25T10:01:54.348429+0000","last_clean":"2024-04-25T10:01:54.348429+0000","last_became_active":"2024-04-25T10:01:52.300836+0000","last_became_peered":"2024-04-25T10:01:52.300836+0000","last_unstale":"2024-04-25T10:01:54.348429+0000","last_undegraded":"2024-04-25T10:01:54.348429+0000","last_fullsized":"2024-04-25T10:01:54.348429+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:05.209265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347876+0000","last_change":"2024-04-25T10:01:53.308947+0000","last_active":"2024-04-25T10:01:54.347876+0000","last_peered":"2024-04-25T10:01:54.347876+0000","last_clean":"2024-04-25T10:01:54.347876+0000","last_became_active":"2024-04-25T10:01:53.308682+0000","last_became_peered":"2024-04-25T10:01:53.308682+0000","last_unstale":"2024-04-25T10:01:54.347876+0000","last_undegraded":"2024-04-25T10:01:54.347876+0000","last_fullsized":"2024-04-25T10:01:54.347876+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:09.641026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349073+0000","last_change":"2024-04-25T10:01:52.299596+0000","last_active":"2024-04-25T10:01:54.349073+0000","last_peered":"2024-04-25T10:01:54.349073+0000","last_clean":"2024-04-25T10:01:54.349073+0000","last_became_active":"2024-04-25T10:01:52.299303+0000","last_became_peered":"2024-04-25T10:01:52.299303+0000","last_unstale":"2024-04-25T10:01:54.349073+0000","last_undegraded":"2024-04-25T10:01:54.349073+0000","last_fullsized":"2024-04-25T10:01:54.349073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.253783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347539+0000","last_change":"2024-04-25T10:01:53.311926+0000","last_active":"2024-04-25T10:01:54.347539+0000","last_peered":"2024-04-25T10:01:54.347539+0000","last_clean":"2024-04-25T10:01:54.347539+0000","last_became_active":"2024-04-25T10:01:53.309452+0000","last_became_peered":"2024-04-25T10:01:53.309452+0000","last_unstale":"2024-04-25T10:01:54.347539+0000","last_undegraded":"2024-04-25T10:01:54.347539+0000","last_fullsized":"2024-04-25T10:01:54.347539+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.619357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348648+0000","last_change":"2024-04-25T10:01:52.301037+0000","last_active":"2024-04-25T10:01:54.348648+0000","last_peered":"2024-04-25T10:01:54.348648+0000","last_clean":"2024-04-25T10:01:54.348648+0000","last_became_active":"2024-04-25T10:01:52.300739+0000","last_became_peered":"2024-04-25T10:01:52.300739+0000","last_unstale":"2024-04-25T10:01:54.348648+0000","last_undegraded":"2024-04-25T10:01:54.348648+0000","last_fullsized":"2024-04-25T10:01:54.348648+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:43.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310139+0000","last_change":"2024-04-25T10:01:53.310352+0000","last_active":"2024-04-25T10:01:54.310139+0000","last_peered":"2024-04-25T10:01:54.310139+0000","last_clean":"2024-04-25T10:01:54.310139+0000","last_became_active":"2024-04-25T10:01:53.310055+0000","last_became_peered":"2024-04-25T10:01:53.310055+0000","last_unstale":"2024-04-25T10:01:54.310139+0000","last_undegraded":"2024-04-25T10:01:54.310139+0000","last_fullsized":"2024-04-25T10:01:54.310139+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:26.199709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348498+0000","last_change":"2024-04-25T10:01:52.301288+0000","last_active":"2024-04-25T10:01:54.348498+0000","last_peered":"2024-04-25T10:01:54.348498+0000","last_clean":"2024-04-25T10:01:54.348498+0000","last_became_active":"2024-04-25T10:01:52.301022+0000","last_became_peered":"2024-04-25T10:01:52.301022+0000","last_unstale":"2024-04-25T10:01:54.348498+0000","last_undegraded":"2024-04-25T10:01:54.348498+0000","last_fullsized":"2024-04-25T10:01:54.348498+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:07.389506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312173+0000","last_change":"2024-04-25T10:01:53.315399+0000","last_active":"2024-04-25T10:01:54.312173+0000","last_peered":"2024-04-25T10:01:54.312173+0000","last_clean":"2024-04-25T10:01:54.312173+0000","last_became_active":"2024-04-25T10:01:53.315137+0000","last_became_peered":"2024-04-25T10:01:53.315137+0000","last_unstale":"2024-04-25T10:01:54.312173+0000","last_undegraded":"2024-04-25T10:01:54.312173+0000","last_fullsized":"2024-04-25T10:01:54.312173+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.634999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347825+0000","last_change":"2024-04-25T10:01:52.292697+0000","last_active":"2024-04-25T10:01:54.347825+0000","last_peered":"2024-04-25T10:01:54.347825+0000","last_clean":"2024-04-25T10:01:54.347825+0000","last_became_active":"2024-04-25T10:01:52.292316+0000","last_became_peered":"2024-04-25T10:01:52.292316+0000","last_unstale":"2024-04-25T10:01:54.347825+0000","last_undegraded":"2024-04-25T10:01:54.347825+0000","last_fullsized":"2024-04-25T10:01:54.347825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:31.205159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320518+0000","last_change":"2024-04-25T10:01:53.313773+0000","last_active":"2024-04-25T10:01:54.320518+0000","last_peered":"2024-04-25T10:01:54.320518+0000","last_clean":"2024-04-25T10:01:54.320518+0000","last_became_active":"2024-04-25T10:01:53.313469+0000","last_became_peered":"2024-04-25T10:01:53.313469+0000","last_unstale":"2024-04-25T10:01:54.320518+0000","last_undegraded":"2024-04-25T10:01:54.320518+0000","last_fullsized":"2024-04-25T10:01:54.320518+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:52.218455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348408+0000","last_change":"2024-04-25T10:01:52.299811+0000","last_active":"2024-04-25T10:01:54.348408+0000","last_peered":"2024-04-25T10:01:54.348408+0000","last_clean":"2024-04-25T10:01:54.348408+0000","last_became_active":"2024-04-25T10:01:52.299468+0000","last_became_peered":"2024-04-25T10:01:52.299468+0000","last_unstale":"2024-04-25T10:01:54.348408+0000","last_undegraded":"2024-04-25T10:01:54.348408+0000","last_fullsized":"2024-04-25T10:01:54.348408+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:05.209265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310557+0000","last_change":"2024-04-25T10:01:53.309636+0000","last_active":"2024-04-25T10:01:54.310557+0000","last_peered":"2024-04-25T10:01:54.310557+0000","last_clean":"2024-04-25T10:01:54.310557+0000","last_became_active":"2024-04-25T10:01:53.309375+0000","last_became_peered":"2024-04-25T10:01:53.309375+0000","last_unstale":"2024-04-25T10:01:54.310557+0000","last_undegraded":"2024-04-25T10:01:54.310557+0000","last_fullsized":"2024-04-25T10:01:54.310557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:18.034791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348215+0000","last_change":"2024-04-25T10:01:52.300012+0000","last_active":"2024-04-25T10:01:54.348215+0000","last_peered":"2024-04-25T10:01:54.348215+0000","last_clean":"2024-04-25T10:01:54.348215+0000","last_became_active":"2024-04-25T10:01:52.299713+0000","last_became_peered":"2024-04-25T10:01:52.299713+0000","last_unstale":"2024-04-25T10:01:54.348215+0000","last_undegraded":"2024-04-25T10:01:54.348215+0000","last_fullsized":"2024-04-25T10:01:54.348215+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:39.960271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320576+0000","last_change":"2024-04-25T10:01:53.318376+0000","last_active":"2024-04-25T10:01:54.320576+0000","last_peered":"2024-04-25T10:01:54.320576+0000","last_clean":"2024-04-25T10:01:54.320576+0000","last_became_active":"2024-04-25T10:01:53.318005+0000","last_became_peered":"2024-04-25T10:01:53.318005+0000","last_unstale":"2024-04-25T10:01:54.320576+0000","last_undegraded":"2024-04-25T10:01:54.320576+0000","last_fullsized":"2024-04-25T10:01:54.320576+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:30.968859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320314+0000","last_change":"2024-04-25T10:01:52.294015+0000","last_active":"2024-04-25T10:01:54.320314+0000","last_peered":"2024-04-25T10:01:54.320314+0000","last_clean":"2024-04-25T10:01:54.320314+0000","last_became_active":"2024-04-25T10:01:52.293769+0000","last_became_peered":"2024-04-25T10:01:52.293769+0000","last_unstale":"2024-04-25T10:01:54.320314+0000","last_undegraded":"2024-04-25T10:01:54.320314+0000","last_fullsized":"2024-04-25T10:01:54.320314+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:22.062751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348551+0000","last_change":"2024-04-25T10:01:53.316595+0000","last_active":"2024-04-25T10:01:54.348551+0000","last_peered":"2024-04-25T10:01:54.348551+0000","last_clean":"2024-04-25T10:01:54.348551+0000","last_became_active":"2024-04-25T10:01:53.313992+0000","last_became_peered":"2024-04-25T10:01:53.313992+0000","last_unstale":"2024-04-25T10:01:54.348551+0000","last_undegraded":"2024-04-25T10:01:54.348551+0000","last_fullsized":"2024-04-25T10:01:54.348551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:27.623538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347985+0000","last_change":"2024-04-25T10:01:52.299879+0000","last_active":"2024-04-25T10:01:54.347985+0000","last_peered":"2024-04-25T10:01:54.347985+0000","last_clean":"2024-04-25T10:01:54.347985+0000","last_became_active":"2024-04-25T10:01:52.299301+0000","last_became_peered":"2024-04-25T10:01:52.299301+0000","last_unstale":"2024-04-25T10:01:54.347985+0000","last_undegraded":"2024-04-25T10:01:54.347985+0000","last_fullsized":"2024-04-25T10:01:54.347985+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.557737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349026+0000","last_change":"2024-04-25T10:01:53.322694+0000","last_active":"2024-04-25T10:01:54.349026+0000","last_peered":"2024-04-25T10:01:54.349026+0000","last_clean":"2024-04-25T10:01:54.349026+0000","last_became_active":"2024-04-25T10:01:53.322132+0000","last_became_peered":"2024-04-25T10:01:53.322132+0000","last_unstale":"2024-04-25T10:01:54.349026+0000","last_undegraded":"2024-04-25T10:01:54.349026+0000","last_fullsized":"2024-04-25T10:01:54.349026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:52.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348684+0000","last_change":"2024-04-25T10:01:52.302593+0000","last_active":"2024-04-25T10:01:54.348684+0000","last_peered":"2024-04-25T10:01:54.348684+0000","last_clean":"2024-04-25T10:01:54.348684+0000","last_became_active":"2024-04-25T10:01:52.300953+0000","last_became_peered":"2024-04-25T10:01:52.300953+0000","last_unstale":"2024-04-25T10:01:54.348684+0000","last_undegraded":"2024-04-25T10:01:54.348684+0000","last_fullsized":"2024-04-25T10:01:54.348684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310301+0000","last_change":"2024-04-25T10:01:53.305776+0000","last_active":"2024-04-25T10:01:54.310301+0000","last_peered":"2024-04-25T10:01:54.310301+0000","last_clean":"2024-04-25T10:01:54.310301+0000","last_became_active":"2024-04-25T10:01:53.304775+0000","last_became_peered":"2024-04-25T10:01:53.304775+0000","last_unstale":"2024-04-25T10:01:54.310301+0000","last_undegraded":"2024-04-25T10:01:54.310301+0000","last_fullsized":"2024-04-25T10:01:54.310301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:57.394066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347849+0000","last_change":"2024-04-25T10:01:52.300698+0000","last_active":"2024-04-25T10:01:54.347849+0000","last_peered":"2024-04-25T10:01:54.347849+0000","last_clean":"2024-04-25T10:01:54.347849+0000","last_became_active":"2024-04-25T10:01:52.299332+0000","last_became_peered":"2024-04-25T10:01:52.299332+0000","last_unstale":"2024-04-25T10:01:54.347849+0000","last_undegraded":"2024-04-25T10:01:54.347849+0000","last_fullsized":"2024-04-25T10:01:54.347849+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:54.567517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320740+0000","last_change":"2024-04-25T10:01:53.307063+0000","last_active":"2024-04-25T10:01:54.320740+0000","last_peered":"2024-04-25T10:01:54.320740+0000","last_clean":"2024-04-25T10:01:54.320740+0000","last_became_active":"2024-04-25T10:01:53.306775+0000","last_became_peered":"2024-04-25T10:01:53.306775+0000","last_unstale":"2024-04-25T10:01:54.320740+0000","last_undegraded":"2024-04-25T10:01:54.320740+0000","last_fullsized":"2024-04-25T10:01:54.320740+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:36.442879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320616+0000","last_change":"2024-04-25T10:01:52.300081+0000","last_active":"2024-04-25T10:01:54.320616+0000","last_peered":"2024-04-25T10:01:54.320616+0000","last_clean":"2024-04-25T10:01:54.320616+0000","last_became_active":"2024-04-25T10:01:52.299737+0000","last_became_peered":"2024-04-25T10:01:52.299737+0000","last_unstale":"2024-04-25T10:01:54.320616+0000","last_undegraded":"2024-04-25T10:01:54.320616+0000","last_fullsized":"2024-04-25T10:01:54.320616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:47.176599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347945+0000","last_change":"2024-04-25T10:01:53.314636+0000","last_active":"2024-04-25T10:01:54.347945+0000","last_peered":"2024-04-25T10:01:54.347945+0000","last_clean":"2024-04-25T10:01:54.347945+0000","last_became_active":"2024-04-25T10:01:53.314349+0000","last_became_peered":"2024-04-25T10:01:53.314349+0000","last_unstale":"2024-04-25T10:01:54.347945+0000","last_undegraded":"2024-04-25T10:01:54.347945+0000","last_fullsized":"2024-04-25T10:01:54.347945+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:04.564002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.319936+0000","last_change":"2024-04-25T10:01:52.292204+0000","last_active":"2024-04-25T10:01:54.319936+0000","last_peered":"2024-04-25T10:01:54.319936+0000","last_clean":"2024-04-25T10:01:54.319936+0000","last_became_active":"2024-04-25T10:01:52.291829+0000","last_became_peered":"2024-04-25T10:01:52.291829+0000","last_unstale":"2024-04-25T10:01:54.319936+0000","last_undegraded":"2024-04-25T10:01:54.319936+0000","last_fullsized":"2024-04-25T10:01:54.319936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:30.816226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348453+0000","last_change":"2024-04-25T10:01:53.306759+0000","last_active":"2024-04-25T10:01:54.348453+0000","last_peered":"2024-04-25T10:01:54.348453+0000","last_clean":"2024-04-25T10:01:54.348453+0000","last_became_active":"2024-04-25T10:01:53.306408+0000","last_became_peered":"2024-04-25T10:01:53.306408+0000","last_unstale":"2024-04-25T10:01:54.348453+0000","last_undegraded":"2024-04-25T10:01:54.348453+0000","last_fullsized":"2024-04-25T10:01:54.348453+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:42.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318511+0000","last_change":"2024-04-25T10:01:52.293197+0000","last_active":"2024-04-25T10:01:54.318511+0000","last_peered":"2024-04-25T10:01:54.318511+0000","last_clean":"2024-04-25T10:01:54.318511+0000","last_became_active":"2024-04-25T10:01:52.292609+0000","last_became_peered":"2024-04-25T10:01:52.292609+0000","last_unstale":"2024-04-25T10:01:54.318511+0000","last_undegraded":"2024-04-25T10:01:54.318511+0000","last_fullsized":"2024-04-25T10:01:54.318511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348125+0000","last_change":"2024-04-25T10:01:53.312259+0000","last_active":"2024-04-25T10:01:54.348125+0000","last_peered":"2024-04-25T10:01:54.348125+0000","last_clean":"2024-04-25T10:01:54.348125+0000","last_became_active":"2024-04-25T10:01:53.311938+0000","last_became_peered":"2024-04-25T10:01:53.311938+0000","last_unstale":"2024-04-25T10:01:54.348125+0000","last_undegraded":"2024-04-25T10:01:54.348125+0000","last_fullsized":"2024-04-25T10:01:54.348125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:11.878572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320728+0000","last_change":"2024-04-25T10:01:52.299885+0000","last_active":"2024-04-25T10:01:54.320728+0000","last_peered":"2024-04-25T10:01:54.320728+0000","last_clean":"2024-04-25T10:01:54.320728+0000","last_became_active":"2024-04-25T10:01:52.299576+0000","last_became_peered":"2024-04-25T10:01:52.299576+0000","last_unstale":"2024-04-25T10:01:54.320728+0000","last_undegraded":"2024-04-25T10:01:54.320728+0000","last_fullsized":"2024-04-25T10:01:54.320728+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:09.750891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320361+0000","last_change":"2024-04-25T10:01:53.313711+0000","last_active":"2024-04-25T10:01:54.320361+0000","last_peered":"2024-04-25T10:01:54.320361+0000","last_clean":"2024-04-25T10:01:54.320361+0000","last_became_active":"2024-04-25T10:01:53.313416+0000","last_became_peered":"2024-04-25T10:01:53.313416+0000","last_unstale":"2024-04-25T10:01:54.320361+0000","last_undegraded":"2024-04-25T10:01:54.320361+0000","last_fullsized":"2024-04-25T10:01:54.320361+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:39.432005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311723+0000","last_change":"2024-04-25T10:01:52.289539+0000","last_active":"2024-04-25T10:01:54.311723+0000","last_peered":"2024-04-25T10:01:54.311723+0000","last_clean":"2024-04-25T10:01:54.311723+0000","last_became_active":"2024-04-25T10:01:52.289261+0000","last_became_peered":"2024-04-25T10:01:52.289261+0000","last_unstale":"2024-04-25T10:01:54.311723+0000","last_undegraded":"2024-04-25T10:01:54.311723+0000","last_fullsized":"2024-04-25T10:01:54.311723+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348597+0000","last_change":"2024-04-25T10:01:53.322700+0000","last_active":"2024-04-25T10:01:54.348597+0000","last_peered":"2024-04-25T10:01:54.348597+0000","last_clean":"2024-04-25T10:01:54.348597+0000","last_became_active":"2024-04-25T10:01:53.320500+0000","last_became_peered":"2024-04-25T10:01:53.320500+0000","last_unstale":"2024-04-25T10:01:54.348597+0000","last_undegraded":"2024-04-25T10:01:54.348597+0000","last_fullsized":"2024-04-25T10:01:54.348597+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.362351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311806+0000","last_change":"2024-04-25T10:01:52.292733+0000","last_active":"2024-04-25T10:01:54.311806+0000","last_peered":"2024-04-25T10:01:54.311806+0000","last_clean":"2024-04-25T10:01:54.311806+0000","last_became_active":"2024-04-25T10:01:52.292040+0000","last_became_peered":"2024-04-25T10:01:52.292040+0000","last_unstale":"2024-04-25T10:01:54.311806+0000","last_undegraded":"2024-04-25T10:01:54.311806+0000","last_fullsized":"2024-04-25T10:01:54.311806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348029+0000","last_change":"2024-04-25T10:01:53.311823+0000","last_active":"2024-04-25T10:01:54.348029+0000","last_peered":"2024-04-25T10:01:54.348029+0000","last_clean":"2024-04-25T10:01:54.348029+0000","last_became_active":"2024-04-25T10:01:53.311509+0000","last_became_peered":"2024-04-25T10:01:53.311509+0000","last_unstale":"2024-04-25T10:01:54.348029+0000","last_undegraded":"2024-04-25T10:01:54.348029+0000","last_fullsized":"2024-04-25T10:01:54.348029+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.340419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320280+0000","last_change":"2024-04-25T10:01:52.299938+0000","last_active":"2024-04-25T10:01:54.320280+0000","last_peered":"2024-04-25T10:01:54.320280+0000","last_clean":"2024-04-25T10:01:54.320280+0000","last_became_active":"2024-04-25T10:01:52.299592+0000","last_became_peered":"2024-04-25T10:01:52.299592+0000","last_unstale":"2024-04-25T10:01:54.320280+0000","last_undegraded":"2024-04-25T10:01:54.320280+0000","last_fullsized":"2024-04-25T10:01:54.320280+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:03.815918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348251+0000","last_change":"2024-04-25T10:01:53.308285+0000","last_active":"2024-04-25T10:01:54.348251+0000","last_peered":"2024-04-25T10:01:54.348251+0000","last_clean":"2024-04-25T10:01:54.348251+0000","last_became_active":"2024-04-25T10:01:53.308018+0000","last_became_peered":"2024-04-25T10:01:53.308018+0000","last_unstale":"2024-04-25T10:01:54.348251+0000","last_undegraded":"2024-04-25T10:01:54.348251+0000","last_fullsized":"2024-04-25T10:01:54.348251+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347851+0000","last_change":"2024-04-25T10:01:52.301345+0000","last_active":"2024-04-25T10:01:54.347851+0000","last_peered":"2024-04-25T10:01:54.347851+0000","last_clean":"2024-04-25T10:01:54.347851+0000","last_became_active":"2024-04-25T10:01:52.301002+0000","last_became_peered":"2024-04-25T10:01:52.301002+0000","last_unstale":"2024-04-25T10:01:54.347851+0000","last_undegraded":"2024-04-25T10:01:54.347851+0000","last_fullsized":"2024-04-25T10:01:54.347851+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348318+0000","last_change":"2024-04-25T10:01:53.314890+0000","last_active":"2024-04-25T10:01:54.348318+0000","last_peered":"2024-04-25T10:01:54.348318+0000","last_clean":"2024-04-25T10:01:54.348318+0000","last_became_active":"2024-04-25T10:01:53.314629+0000","last_became_peered":"2024-04-25T10:01:53.314629+0000","last_unstale":"2024-04-25T10:01:54.348318+0000","last_undegraded":"2024-04-25T10:01:54.348318+0000","last_fullsized":"2024-04-25T10:01:54.348318+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:56.754763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348508+0000","last_change":"2024-04-25T10:01:52.300200+0000","last_active":"2024-04-25T10:01:54.348508+0000","last_peered":"2024-04-25T10:01:54.348508+0000","last_clean":"2024-04-25T10:01:54.348508+0000","last_became_active":"2024-04-25T10:01:52.299829+0000","last_became_peered":"2024-04-25T10:01:52.299829+0000","last_unstale":"2024-04-25T10:01:54.348508+0000","last_undegraded":"2024-04-25T10:01:54.348508+0000","last_fullsized":"2024-04-25T10:01:54.348508+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":355,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347939+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:54.347939+0000","last_peered":"2024-04-25T10:01:54.347939+0000","last_clean":"2024-04-25T10:01:54.347939+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:54.347939+0000","last_undegraded":"2024-04-25T10:01:54.347939+0000","last_fullsized":"2024-04-25T10:01:54.347939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2117632,"data_stored":1292799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46536,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325304,"statfs":{"total":96636764160,"available":96589111296,"internally_reserved":0,"allocated":1499136,"data_stored":695545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45140,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326700,"statfs":{"total":96636764160,"available":96590540800,"internally_reserved":0,"allocated":1515520,"data_stored":701815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1519616,"data_stored":703267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50716,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321124,"statfs":{"total":96636764160,"available":96584830976,"internally_reserved":0,"allocated":1523712,"data_stored":703301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1507328,"data_stored":701655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50700,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321140,"statfs":{"total":96636764160,"available":96584847360,"internally_reserved":0,"allocated":1499136,"data_stored":694985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51272,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320568,"statfs":{"total":96636764160,"available":96584261632,"internally_reserved":0,"allocated":2093056,"data_stored":1285353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:57.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:01:57.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.529+0000 7fcdd6ac4640 1 Processor -- start 2024-04-25T10:01:57.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.529+0000 7fcdd6ac4640 1 -- start start 2024-04-25T10:01:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd6ac4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcdd015dd30 0x7fcdd00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd6ac4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcdd00b9770 0x7fcdd00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd6ac4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd00d1df0 con 0x7fcdd00d2e90 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd6ac4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd00b7a50 con 0x7fcdd015dd30 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd6ac4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd00d20d0 con 0x7fcdd00b9770 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd4860640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcdd015dd30 0x7fcdd00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd5061640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcdd00d2e90 0x7fcdd00b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42976/0 (socket says 172.21.15.148:42976) 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd4860640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcdd015dd30 0x7fcdd00b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42802/0 (socket says 172.21.15.148:42802) 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd5061640 1 -- 172.21.15.148:0/2175503266 learned_addr learned my addr 172.21.15.148:0/2175503266 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd4860640 1 -- 172.21.15.148:0/2175503266 >> v2:172.21.15.188:3300/0 conn(0x7fcdd00b9770 msgr2=0x7fcdd00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdcffff640 1 --2- 172.21.15.148:0/2175503266 >> v2:172.21.15.188:3300/0 conn(0x7fcdd00b9770 0x7fcdd00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd4860640 1 --2- 172.21.15.148:0/2175503266 >> v2:172.21.15.188:3300/0 conn(0x7fcdd00b9770 0x7fcdd00d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd4860640 1 -- 172.21.15.148:0/2175503266 >> v1:172.21.15.148:6789/0 conn(0x7fcdd00d2e90 legacy=0x7fcdd00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdcf7fe640 1 -- 172.21.15.148:0/2175503266 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867285159 0 0) 0x7fcdd00d1df0 con 0x7fcdd00d2e90 2024-04-25T10:01:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdd4860640 1 -- 172.21.15.148:0/2175503266 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00d23b0 con 0x7fcdd015dd30 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.530+0000 7fcdcffff640 1 --2- 172.21.15.148:0/2175503266 >> v2:172.21.15.188:3300/0 conn(0x7fcdd00b9770 0x7fcdd00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdd4860640 1 --2- 172.21.15.148:0/2175503266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00b9230 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00b6540 tx=0x7fcdc402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260d015327708552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdcf7fe640 1 -- 172.21.15.148:0/2175503266 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4037070 con 0x7fcdd015dd30 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdcf7fe640 1 -- 172.21.15.148:0/2175503266 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc402add0 con 0x7fcdd015dd30 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdcf7fe640 1 -- 172.21.15.148:0/2175503266 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4032660 con 0x7fcdd015dd30 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2175503266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 msgr2=0x7fcdd00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2175503266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00b9230 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00b6540 tx=0x7fcdc402a080 comp rx=0 tx=0).stop 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.531+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2175503266 shutdown_connections 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.532+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2175503266 >> v2:172.21.15.188:3300/0 conn(0x7fcdd00b9770 0x7fcdd00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.532+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2175503266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00b9230 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.532+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2175503266 >> 172.21.15.148:0/2175503266 conn(0x7fcdd0061c30 msgr2=0x7fcdd006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.532+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2175503266 shutdown_connections 2024-04-25T10:01:57.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.532+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2175503266 wait complete. 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.533+0000 7fcdd6ac4640 1 Processor -- start 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.533+0000 7fcdd6ac4640 1 -- start start 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.533+0000 7fcdd6ac4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd00b9770 0x7fcdd00c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.533+0000 7fcdd6ac4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdd00d2e90 0x7fcdd00ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd6ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd6ac4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00d1a10 con 0x7fcdd015dd30 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd6ac4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00d2970 con 0x7fcdd00b9770 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd6ac4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00d1b90 con 0x7fcdd00d2e90 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42818/0 (socket says 172.21.15.148:42818) 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 -- 172.21.15.148:0/2023868166 learned_addr learned my addr 172.21.15.148:0/2023868166 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 -- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdd00d2e90 msgr2=0x7fcdd00ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd4860640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd00b9770 0x7fcdd00c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdcffff640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdd00d2e90 0x7fcdd00ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdd00d2e90 0x7fcdd00ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 -- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd00b9770 msgr2=0x7fcdd00c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd00b9770 0x7fcdd00c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 -- 172.21.15.148:0/2023868166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd01f75d0 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd5061640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00cd280 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800c7a0 tx=0x7fcdc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8018070 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc800a640 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd01f7870 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800aab0 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.534+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd01f7e30 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.535+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0073990 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.536+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdc801b070 con 0x7fcdd015dd30 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.536+0000 7fcdcd7fa640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.536+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdc8089920 con 0x7fcdd015dd30 2024-04-25T10:01:57.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.536+0000 7fcdd4860640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.537+0000 7fcdd4860640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcdb406b4d0 0x7fcdb406d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4034000 tx=0x7fcdc402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.540+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdc80586a0 con 0x7fcdd015dd30 2024-04-25T10:01:57.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.777+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcdd00b8750 con 0x7fcdd015dd30 2024-04-25T10:01:57.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.777+0000 7fcdcd7fa640 1 -- 172.21.15.148:0/2023868166 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1434 (secure 0 0 0) 0x7fcdc8057fa0 con 0x7fcdd015dd30 2024-04-25T10:01:57.782 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:57.782 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":478,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":396652544,"bytes_avail":772697460736,"bytes_total":773094113280,"write_bytes_sec":2457,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:01:43.111276+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcdb406b4d0 msgr2=0x7fcdb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcdb406b4d0 0x7fcdb406d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4034000 tx=0x7fcdc402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 msgr2=0x7fcdd00cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00cd280 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800c7a0 tx=0x7fcdc800cc60 comp rx=0 tx=0).stop 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 shutdown_connections 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdd015dd30 0x7fcdd00cd280 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdd00d2e90 0x7fcdd00ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 --2- 172.21.15.148:0/2023868166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdd00b9770 0x7fcdd00c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 >> 172.21.15.148:0/2023868166 conn(0x7fcdd0061c30 msgr2=0x7fcdd00d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 shutdown_connections 2024-04-25T10:01:57.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.779+0000 7fcdd6ac4640 1 -- 172.21.15.148:0/2023868166 wait complete. 2024-04-25T10:01:57.795 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:01:57.795 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:01:57.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.887+0000 7f6f54e5a640 1 Processor -- start 2024-04-25T10:01:57.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.887+0000 7f6f54e5a640 1 -- start start 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f54e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f5015dc90 0x7f6f501625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f54e5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f500d1810 0x7f6f500d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f54e5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f500cce50 con 0x7f6f500cc8e0 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f54e5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f500d2180 con 0x7f6f500d1810 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f54e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50161af0 con 0x7f6f5015dc90 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f4dd9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6f500cc8e0 0x7f6f5015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42990/0 (socket says 172.21.15.148:42990) 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f4dd9b640 1 -- 172.21.15.148:0/687171035 learned_addr learned my addr 172.21.15.148:0/687171035 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f4e59c640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f5015dc90 0x7f6f501625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/687171035 >> v2:172.21.15.188:3300/0 conn(0x7f6f500d1810 0x7f6f500d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557645448 0 0) 0x7f6f500cce50 con 0x7f6f500cc8e0 2024-04-25T10:01:57.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.888+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f38003520 con 0x7f6f500cc8e0 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642741189 0 0) 0x7f6f38003520 con 0x7f6f500cc8e0 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f5015dc90 msgr2=0x7f6f501625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f5015dc90 0x7f6f501625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 >> v2:172.21.15.188:3300/0 conn(0x7f6f500d1810 msgr2=0x7f6f500d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 --2- 172.21.15.148:0/687171035 >> v2:172.21.15.188:3300/0 conn(0x7f6f500d1810 0x7f6f500d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50161dd0 con 0x7f6f500cc8e0 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6f44001670 con 0x7f6f500cc8e0 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 >> v1:172.21.15.148:6789/0 conn(0x7f6f500cc8e0 legacy=0x7f6f5015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380039b0 0x7f6f38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f38004800 0x7f6f38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f380051c0 0x7f6f3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500d2180 con 0x7f6f380051c0 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50161af0 con 0x7f6f380039b0 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3800bf50 con 0x7f6f38004800 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4e59c640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f5015dc90 0x7f6f501625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f380051c0 0x7f6f3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4dd9b640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f38004800 0x7f6f38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4e59c640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380039b0 0x7f6f38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f38004800 msgr2=0x7f6f38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f38004800 0x7f6f38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380039b0 msgr2=0x7f6f38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380039b0 0x7f6f38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.889+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/687171035 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38003520 con 0x7f6f380051c0 2024-04-25T10:01:57.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f380051c0 0x7f6f3800b9c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e170 tx=0x7f6f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7462bc0e2af31f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48034020 con 0x7f6f380051c0 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48030430 con 0x7f6f380051c0 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f4d59a640 1 -- 172.21.15.148:0/687171035 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48009c10 con 0x7f6f380051c0 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f54e5a640 1 -- 172.21.15.148:0/687171035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f380051c0 msgr2=0x7f6f3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/687171035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f380051c0 0x7f6f3800b9c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e170 tx=0x7f6f38005830 comp rx=0 tx=0).stop 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.890+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/687171035 reap_dead start 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.891+0000 7f6f54e5a640 1 -- 172.21.15.148:0/687171035 shutdown_connections 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.891+0000 7f6f54e5a640 1 -- 172.21.15.148:0/687171035 >> 172.21.15.148:0/687171035 conn(0x7f6f5006f590 msgr2=0x7f6f5006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.891+0000 7f6f54e5a640 1 -- 172.21.15.148:0/687171035 shutdown_connections 2024-04-25T10:01:57.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.891+0000 7f6f54e5a640 1 -- 172.21.15.148:0/687171035 wait complete. 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.892+0000 7f6f54e5a640 1 Processor -- start 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.892+0000 7f6f54e5a640 1 -- start start 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380051c0 0x7f6f50155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f500c8e50 0x7f6f500c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 0x7f6f500b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5015e070 con 0x7f6f500b6f00 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500cd130 con 0x7f6f380051c0 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5015e1f0 con 0x7f6f500c8e50 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 0x7f6f500b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 0x7f6f500b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42838/0 (socket says 172.21.15.148:42838) 2024-04-25T10:01:57.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/1892097540 learned_addr learned my addr 172.21.15.148:0/1892097540 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/1892097540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f500c8e50 msgr2=0x7f6f500c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f500c8e50 0x7f6f500c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/1892097540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380051c0 msgr2=0x7f6f50155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380051c0 0x7f6f50155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 -- 172.21.15.148:0/1892097540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f501555b0 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f4ed9d640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 0x7f6f500b7380 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e170 tx=0x7f6f48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48034040 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48003b90 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f500b6140 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48009c10 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.893+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f500b6700 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.894+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f50073990 con 0x7f6f500b6f00 2024-04-25T10:01:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.895+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f48037020 con 0x7f6f500b6f00 2024-04-25T10:01:57.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.895+0000 7f6f3effd640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f3006b500 0x7f6f3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.895+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f480ab220 con 0x7f6f500b6f00 2024-04-25T10:01:57.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.895+0000 7f6f4e59c640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f3006b500 0x7f6f3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.896+0000 7f6f4e59c640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f3006b500 0x7f6f3006d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6f5006b820 tx=0x7f6f40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:57.899+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f48049060 con 0x7f6f500b6f00 2024-04-25T10:01:58.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.068+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f500cf6b0 con 0x7f6f3006b500 2024-04-25T10:01:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.074+0000 7f6f3effd640 1 -- 172.21.15.148:0/1892097540 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f6f500cf6b0 con 0x7f6f3006b500 2024-04-25T10:01:58.081 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:01:58.082 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f3006b500 msgr2=0x7f6f3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f3006b500 0x7f6f3006d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6f5006b820 tx=0x7f6f40008040 comp rx=0 tx=0).stop 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 msgr2=0x7f6f500b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 0x7f6f500b7380 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800e170 tx=0x7f6f48009750 comp rx=0 tx=0).stop 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 shutdown_connections 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6f3006b500 0x7f6f3006d9c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f500b6f00 0x7f6f500b7380 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f500c8e50 0x7f6f500c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 --2- 172.21.15.148:0/1892097540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f380051c0 0x7f6f50155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 >> 172.21.15.148:0/1892097540 conn(0x7f6f5006f590 msgr2=0x7f6f5006bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 shutdown_connections 2024-04-25T10:01:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:01:58.077+0000 7f6f54e5a640 1 -- 172.21.15.148:0/1892097540 wait complete. 2024-04-25T10:01:58.093 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-04-25T10:01:57.113907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":387356,"kb_used_data":12964,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587364,"statfs":{"total":773094113280,"available":772697460736,"internally_reserved":0,"allocated":13275136,"data_stored":6778720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.832860"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312163+0000","last_change":"2024-04-25T10:01:53.308968+0000","last_active":"2024-04-25T10:01:54.312163+0000","last_peered":"2024-04-25T10:01:54.312163+0000","last_clean":"2024-04-25T10:01:54.312163+0000","last_became_active":"2024-04-25T10:01:53.308683+0000","last_became_peered":"2024-04-25T10:01:53.308683+0000","last_unstale":"2024-04-25T10:01:54.312163+0000","last_undegraded":"2024-04-25T10:01:54.312163+0000","last_fullsized":"2024-04-25T10:01:54.312163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348370+0000","last_change":"2024-04-25T10:01:53.326585+0000","last_active":"2024-04-25T10:01:54.348370+0000","last_peered":"2024-04-25T10:01:54.348370+0000","last_clean":"2024-04-25T10:01:54.348370+0000","last_became_active":"2024-04-25T10:01:53.325978+0000","last_became_peered":"2024-04-25T10:01:53.325978+0000","last_unstale":"2024-04-25T10:01:54.348370+0000","last_undegraded":"2024-04-25T10:01:54.348370+0000","last_fullsized":"2024-04-25T10:01:54.348370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:04.286812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348198+0000","last_change":"2024-04-25T10:01:53.319634+0000","last_active":"2024-04-25T10:01:54.348198+0000","last_peered":"2024-04-25T10:01:54.348198+0000","last_clean":"2024-04-25T10:01:54.348198+0000","last_became_active":"2024-04-25T10:01:53.319370+0000","last_became_peered":"2024-04-25T10:01:53.319370+0000","last_unstale":"2024-04-25T10:01:54.348198+0000","last_undegraded":"2024-04-25T10:01:54.348198+0000","last_fullsized":"2024-04-25T10:01:54.348198+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:18.034791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348276+0000","last_change":"2024-04-25T10:01:53.314499+0000","last_active":"2024-04-25T10:01:54.348276+0000","last_peered":"2024-04-25T10:01:54.348276+0000","last_clean":"2024-04-25T10:01:54.348276+0000","last_became_active":"2024-04-25T10:01:53.314191+0000","last_became_peered":"2024-04-25T10:01:53.314191+0000","last_unstale":"2024-04-25T10:01:54.348276+0000","last_undegraded":"2024-04-25T10:01:54.348276+0000","last_fullsized":"2024-04-25T10:01:54.348276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:26.199709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311504+0000","last_change":"2024-04-25T10:01:53.313912+0000","last_active":"2024-04-25T10:01:54.311504+0000","last_peered":"2024-04-25T10:01:54.311504+0000","last_clean":"2024-04-25T10:01:54.311504+0000","last_became_active":"2024-04-25T10:01:53.313635+0000","last_became_peered":"2024-04-25T10:01:53.313635+0000","last_unstale":"2024-04-25T10:01:54.311504+0000","last_undegraded":"2024-04-25T10:01:54.311504+0000","last_fullsized":"2024-04-25T10:01:54.311504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320356+0000","last_change":"2024-04-25T10:01:53.317523+0000","last_active":"2024-04-25T10:01:54.320356+0000","last_peered":"2024-04-25T10:01:54.320356+0000","last_clean":"2024-04-25T10:01:54.320356+0000","last_became_active":"2024-04-25T10:01:53.317248+0000","last_became_peered":"2024-04-25T10:01:53.317248+0000","last_unstale":"2024-04-25T10:01:54.320356+0000","last_undegraded":"2024-04-25T10:01:54.320356+0000","last_fullsized":"2024-04-25T10:01:54.320356+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:41.914139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320327+0000","last_change":"2024-04-25T10:01:53.314768+0000","last_active":"2024-04-25T10:01:54.320327+0000","last_peered":"2024-04-25T10:01:54.320327+0000","last_clean":"2024-04-25T10:01:54.320327+0000","last_became_active":"2024-04-25T10:01:53.314504+0000","last_became_peered":"2024-04-25T10:01:53.314504+0000","last_unstale":"2024-04-25T10:01:54.320327+0000","last_undegraded":"2024-04-25T10:01:54.320327+0000","last_fullsized":"2024-04-25T10:01:54.320327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:40.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348425+0000","last_change":"2024-04-25T10:01:53.320926+0000","last_active":"2024-04-25T10:01:54.348425+0000","last_peered":"2024-04-25T10:01:54.348425+0000","last_clean":"2024-04-25T10:01:54.348425+0000","last_became_active":"2024-04-25T10:01:53.320663+0000","last_became_peered":"2024-04-25T10:01:53.320663+0000","last_unstale":"2024-04-25T10:01:54.348425+0000","last_undegraded":"2024-04-25T10:01:54.348425+0000","last_fullsized":"2024-04-25T10:01:54.348425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:38.131544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348471+0000","last_change":"2024-04-25T10:01:53.320793+0000","last_active":"2024-04-25T10:01:54.348471+0000","last_peered":"2024-04-25T10:01:54.348471+0000","last_clean":"2024-04-25T10:01:54.348471+0000","last_became_active":"2024-04-25T10:01:53.320516+0000","last_became_peered":"2024-04-25T10:01:53.320516+0000","last_unstale":"2024-04-25T10:01:54.348471+0000","last_undegraded":"2024-04-25T10:01:54.348471+0000","last_fullsized":"2024-04-25T10:01:54.348471+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:21.226281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348274+0000","last_change":"2024-04-25T10:01:53.308828+0000","last_active":"2024-04-25T10:01:54.348274+0000","last_peered":"2024-04-25T10:01:54.348274+0000","last_clean":"2024-04-25T10:01:54.348274+0000","last_became_active":"2024-04-25T10:01:53.308560+0000","last_became_peered":"2024-04-25T10:01:53.308560+0000","last_unstale":"2024-04-25T10:01:54.348274+0000","last_undegraded":"2024-04-25T10:01:54.348274+0000","last_fullsized":"2024-04-25T10:01:54.348274+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312000+0000","last_change":"2024-04-25T10:01:53.315244+0000","last_active":"2024-04-25T10:01:54.312000+0000","last_peered":"2024-04-25T10:01:54.312000+0000","last_clean":"2024-04-25T10:01:54.312000+0000","last_became_active":"2024-04-25T10:01:53.314991+0000","last_became_peered":"2024-04-25T10:01:53.314991+0000","last_unstale":"2024-04-25T10:01:54.312000+0000","last_undegraded":"2024-04-25T10:01:54.312000+0000","last_fullsized":"2024-04-25T10:01:54.312000+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:10.260048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320663+0000","last_change":"2024-04-25T10:01:53.314287+0000","last_active":"2024-04-25T10:01:54.320663+0000","last_peered":"2024-04-25T10:01:54.320663+0000","last_clean":"2024-04-25T10:01:54.320663+0000","last_became_active":"2024-04-25T10:01:53.313201+0000","last_became_peered":"2024-04-25T10:01:53.313201+0000","last_unstale":"2024-04-25T10:01:54.320663+0000","last_undegraded":"2024-04-25T10:01:54.320663+0000","last_fullsized":"2024-04-25T10:01:54.320663+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.462869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347955+0000","last_change":"2024-04-25T10:01:53.307341+0000","last_active":"2024-04-25T10:01:54.347955+0000","last_peered":"2024-04-25T10:01:54.347955+0000","last_clean":"2024-04-25T10:01:54.347955+0000","last_became_active":"2024-04-25T10:01:53.307024+0000","last_became_peered":"2024-04-25T10:01:53.307024+0000","last_unstale":"2024-04-25T10:01:54.347955+0000","last_undegraded":"2024-04-25T10:01:54.347955+0000","last_fullsized":"2024-04-25T10:01:54.347955+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:40.966537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348781+0000","last_change":"2024-04-25T10:01:53.326968+0000","last_active":"2024-04-25T10:01:54.348781+0000","last_peered":"2024-04-25T10:01:54.348781+0000","last_clean":"2024-04-25T10:01:54.348781+0000","last_became_active":"2024-04-25T10:01:53.326663+0000","last_became_peered":"2024-04-25T10:01:53.326663+0000","last_unstale":"2024-04-25T10:01:54.348781+0000","last_undegraded":"2024-04-25T10:01:54.348781+0000","last_fullsized":"2024-04-25T10:01:54.348781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:03.834566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311370+0000","last_change":"2024-04-25T10:01:53.314815+0000","last_active":"2024-04-25T10:01:54.311370+0000","last_peered":"2024-04-25T10:01:54.311370+0000","last_clean":"2024-04-25T10:01:54.311370+0000","last_became_active":"2024-04-25T10:01:53.314511+0000","last_became_peered":"2024-04-25T10:01:53.314511+0000","last_unstale":"2024-04-25T10:01:54.311370+0000","last_undegraded":"2024-04-25T10:01:54.311370+0000","last_fullsized":"2024-04-25T10:01:54.311370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320222+0000","last_change":"2024-04-25T10:01:53.309219+0000","last_active":"2024-04-25T10:01:54.320222+0000","last_peered":"2024-04-25T10:01:54.320222+0000","last_clean":"2024-04-25T10:01:54.320222+0000","last_became_active":"2024-04-25T10:01:53.308965+0000","last_became_peered":"2024-04-25T10:01:53.308965+0000","last_unstale":"2024-04-25T10:01:54.320222+0000","last_undegraded":"2024-04-25T10:01:54.320222+0000","last_fullsized":"2024-04-25T10:01:54.320222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:00.180829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348329+0000","last_change":"2024-04-25T10:01:53.315092+0000","last_active":"2024-04-25T10:01:54.348329+0000","last_peered":"2024-04-25T10:01:54.348329+0000","last_clean":"2024-04-25T10:01:54.348329+0000","last_became_active":"2024-04-25T10:01:53.314365+0000","last_became_peered":"2024-04-25T10:01:53.314365+0000","last_unstale":"2024-04-25T10:01:54.348329+0000","last_undegraded":"2024-04-25T10:01:54.348329+0000","last_fullsized":"2024-04-25T10:01:54.348329+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348538+0000","last_change":"2024-04-25T10:01:53.326698+0000","last_active":"2024-04-25T10:01:54.348538+0000","last_peered":"2024-04-25T10:01:54.348538+0000","last_clean":"2024-04-25T10:01:54.348538+0000","last_became_active":"2024-04-25T10:01:53.325993+0000","last_became_peered":"2024-04-25T10:01:53.325993+0000","last_unstale":"2024-04-25T10:01:54.348538+0000","last_undegraded":"2024-04-25T10:01:54.348538+0000","last_fullsized":"2024-04-25T10:01:54.348538+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318420+0000","last_change":"2024-04-25T10:01:53.320785+0000","last_active":"2024-04-25T10:01:54.318420+0000","last_peered":"2024-04-25T10:01:54.318420+0000","last_clean":"2024-04-25T10:01:54.318420+0000","last_became_active":"2024-04-25T10:01:53.319963+0000","last_became_peered":"2024-04-25T10:01:53.319963+0000","last_unstale":"2024-04-25T10:01:54.318420+0000","last_undegraded":"2024-04-25T10:01:54.318420+0000","last_fullsized":"2024-04-25T10:01:54.318420+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347573+0000","last_change":"2024-04-25T10:01:53.307833+0000","last_active":"2024-04-25T10:01:54.347573+0000","last_peered":"2024-04-25T10:01:54.347573+0000","last_clean":"2024-04-25T10:01:54.347573+0000","last_became_active":"2024-04-25T10:01:53.307583+0000","last_became_peered":"2024-04-25T10:01:53.307583+0000","last_unstale":"2024-04-25T10:01:54.347573+0000","last_undegraded":"2024-04-25T10:01:54.347573+0000","last_fullsized":"2024-04-25T10:01:54.347573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:14.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348432+0000","last_change":"2024-04-25T10:01:53.313805+0000","last_active":"2024-04-25T10:01:54.348432+0000","last_peered":"2024-04-25T10:01:54.348432+0000","last_clean":"2024-04-25T10:01:54.348432+0000","last_became_active":"2024-04-25T10:01:53.313489+0000","last_became_peered":"2024-04-25T10:01:53.313489+0000","last_unstale":"2024-04-25T10:01:54.348432+0000","last_undegraded":"2024-04-25T10:01:54.348432+0000","last_fullsized":"2024-04-25T10:01:54.348432+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:57.394066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311689+0000","last_change":"2024-04-25T10:01:53.317853+0000","last_active":"2024-04-25T10:01:54.311689+0000","last_peered":"2024-04-25T10:01:54.311689+0000","last_clean":"2024-04-25T10:01:54.311689+0000","last_became_active":"2024-04-25T10:01:53.317561+0000","last_became_peered":"2024-04-25T10:01:53.317561+0000","last_unstale":"2024-04-25T10:01:54.311689+0000","last_undegraded":"2024-04-25T10:01:54.311689+0000","last_fullsized":"2024-04-25T10:01:54.311689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.139972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348127+0000","last_change":"2024-04-25T10:01:53.318514+0000","last_active":"2024-04-25T10:01:54.348127+0000","last_peered":"2024-04-25T10:01:54.348127+0000","last_clean":"2024-04-25T10:01:54.348127+0000","last_became_active":"2024-04-25T10:01:53.318206+0000","last_became_peered":"2024-04-25T10:01:53.318206+0000","last_unstale":"2024-04-25T10:01:54.348127+0000","last_undegraded":"2024-04-25T10:01:54.348127+0000","last_fullsized":"2024-04-25T10:01:54.348127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:08.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.309900+0000","last_change":"2024-04-25T10:01:53.316554+0000","last_active":"2024-04-25T10:01:54.309900+0000","last_peered":"2024-04-25T10:01:54.309900+0000","last_clean":"2024-04-25T10:01:54.309900+0000","last_became_active":"2024-04-25T10:01:53.316218+0000","last_became_peered":"2024-04-25T10:01:53.316218+0000","last_unstale":"2024-04-25T10:01:54.309900+0000","last_undegraded":"2024-04-25T10:01:54.309900+0000","last_fullsized":"2024-04-25T10:01:54.309900+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.340419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310385+0000","last_change":"2024-04-25T10:01:53.307075+0000","last_active":"2024-04-25T10:01:54.310385+0000","last_peered":"2024-04-25T10:01:54.310385+0000","last_clean":"2024-04-25T10:01:54.310385+0000","last_became_active":"2024-04-25T10:01:53.304942+0000","last_became_peered":"2024-04-25T10:01:53.304942+0000","last_unstale":"2024-04-25T10:01:54.310385+0000","last_undegraded":"2024-04-25T10:01:54.310385+0000","last_fullsized":"2024-04-25T10:01:54.310385+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:21.226281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347834+0000","last_change":"2024-04-25T10:01:53.306396+0000","last_active":"2024-04-25T10:01:54.347834+0000","last_peered":"2024-04-25T10:01:54.347834+0000","last_clean":"2024-04-25T10:01:54.347834+0000","last_became_active":"2024-04-25T10:01:53.306127+0000","last_became_peered":"2024-04-25T10:01:53.306127+0000","last_unstale":"2024-04-25T10:01:54.347834+0000","last_undegraded":"2024-04-25T10:01:54.347834+0000","last_fullsized":"2024-04-25T10:01:54.347834+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:27.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318479+0000","last_change":"2024-04-25T10:01:53.319940+0000","last_active":"2024-04-25T10:01:54.318479+0000","last_peered":"2024-04-25T10:01:54.318479+0000","last_clean":"2024-04-25T10:01:54.318479+0000","last_became_active":"2024-04-25T10:01:53.319590+0000","last_became_peered":"2024-04-25T10:01:53.319590+0000","last_unstale":"2024-04-25T10:01:54.318479+0000","last_undegraded":"2024-04-25T10:01:54.318479+0000","last_fullsized":"2024-04-25T10:01:54.318479+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320676+0000","last_change":"2024-04-25T10:01:53.316178+0000","last_active":"2024-04-25T10:01:54.320676+0000","last_peered":"2024-04-25T10:01:54.320676+0000","last_clean":"2024-04-25T10:01:54.320676+0000","last_became_active":"2024-04-25T10:01:53.314889+0000","last_became_peered":"2024-04-25T10:01:53.314889+0000","last_unstale":"2024-04-25T10:01:54.320676+0000","last_undegraded":"2024-04-25T10:01:54.320676+0000","last_fullsized":"2024-04-25T10:01:54.320676+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:28.026534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347913+0000","last_change":"2024-04-25T10:01:53.316236+0000","last_active":"2024-04-25T10:01:54.347913+0000","last_peered":"2024-04-25T10:01:54.347913+0000","last_clean":"2024-04-25T10:01:54.347913+0000","last_became_active":"2024-04-25T10:01:53.315521+0000","last_became_peered":"2024-04-25T10:01:53.315521+0000","last_unstale":"2024-04-25T10:01:54.347913+0000","last_undegraded":"2024-04-25T10:01:54.347913+0000","last_fullsized":"2024-04-25T10:01:54.347913+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.725287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348895+0000","last_change":"2024-04-25T10:01:53.317694+0000","last_active":"2024-04-25T10:01:54.348895+0000","last_peered":"2024-04-25T10:01:54.348895+0000","last_clean":"2024-04-25T10:01:54.348895+0000","last_became_active":"2024-04-25T10:01:53.317254+0000","last_became_peered":"2024-04-25T10:01:53.317254+0000","last_unstale":"2024-04-25T10:01:54.348895+0000","last_undegraded":"2024-04-25T10:01:54.348895+0000","last_fullsized":"2024-04-25T10:01:54.348895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:59.725012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310217+0000","last_change":"2024-04-25T10:01:53.303113+0000","last_active":"2024-04-25T10:01:54.310217+0000","last_peered":"2024-04-25T10:01:54.310217+0000","last_clean":"2024-04-25T10:01:54.310217+0000","last_became_active":"2024-04-25T10:01:53.302788+0000","last_became_peered":"2024-04-25T10:01:53.302788+0000","last_unstale":"2024-04-25T10:01:54.310217+0000","last_undegraded":"2024-04-25T10:01:54.310217+0000","last_fullsized":"2024-04-25T10:01:54.310217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:56.754763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.309915+0000","last_change":"2024-04-25T10:01:53.316148+0000","last_active":"2024-04-25T10:01:54.309915+0000","last_peered":"2024-04-25T10:01:54.309915+0000","last_clean":"2024-04-25T10:01:54.309915+0000","last_became_active":"2024-04-25T10:01:53.315873+0000","last_became_peered":"2024-04-25T10:01:53.315873+0000","last_unstale":"2024-04-25T10:01:54.309915+0000","last_undegraded":"2024-04-25T10:01:54.309915+0000","last_fullsized":"2024-04-25T10:01:54.309915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:08.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348109+0000","last_change":"2024-04-25T10:01:53.310855+0000","last_active":"2024-04-25T10:01:54.348109+0000","last_peered":"2024-04-25T10:01:54.348109+0000","last_clean":"2024-04-25T10:01:54.348109+0000","last_became_active":"2024-04-25T10:01:53.310567+0000","last_became_peered":"2024-04-25T10:01:53.310567+0000","last_unstale":"2024-04-25T10:01:54.348109+0000","last_undegraded":"2024-04-25T10:01:54.348109+0000","last_fullsized":"2024-04-25T10:01:54.348109+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.215530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318170+0000","last_change":"2024-04-25T10:01:52.296531+0000","last_active":"2024-04-25T10:01:54.318170+0000","last_peered":"2024-04-25T10:01:54.318170+0000","last_clean":"2024-04-25T10:01:54.318170+0000","last_became_active":"2024-04-25T10:01:52.296192+0000","last_became_peered":"2024-04-25T10:01:52.296192+0000","last_unstale":"2024-04-25T10:01:54.318170+0000","last_undegraded":"2024-04-25T10:01:54.318170+0000","last_fullsized":"2024-04-25T10:01:54.318170+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320541+0000","last_change":"2024-04-25T10:01:53.314991+0000","last_active":"2024-04-25T10:01:54.320541+0000","last_peered":"2024-04-25T10:01:54.320541+0000","last_clean":"2024-04-25T10:01:54.320541+0000","last_became_active":"2024-04-25T10:01:53.314698+0000","last_became_peered":"2024-04-25T10:01:53.314698+0000","last_unstale":"2024-04-25T10:01:54.320541+0000","last_undegraded":"2024-04-25T10:01:54.320541+0000","last_fullsized":"2024-04-25T10:01:54.320541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:49.360827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348276+0000","last_change":"2024-04-25T10:01:52.299627+0000","last_active":"2024-04-25T10:01:54.348276+0000","last_peered":"2024-04-25T10:01:54.348276+0000","last_clean":"2024-04-25T10:01:54.348276+0000","last_became_active":"2024-04-25T10:01:52.299139+0000","last_became_peered":"2024-04-25T10:01:52.299139+0000","last_unstale":"2024-04-25T10:01:54.348276+0000","last_undegraded":"2024-04-25T10:01:54.348276+0000","last_fullsized":"2024-04-25T10:01:54.348276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.686423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310355+0000","last_change":"2024-04-25T10:01:53.310586+0000","last_active":"2024-04-25T10:01:54.310355+0000","last_peered":"2024-04-25T10:01:54.310355+0000","last_clean":"2024-04-25T10:01:54.310355+0000","last_became_active":"2024-04-25T10:01:53.309807+0000","last_became_peered":"2024-04-25T10:01:53.309807+0000","last_unstale":"2024-04-25T10:01:54.310355+0000","last_undegraded":"2024-04-25T10:01:54.310355+0000","last_fullsized":"2024-04-25T10:01:54.310355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:42.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348318+0000","last_change":"2024-04-25T10:01:52.300227+0000","last_active":"2024-04-25T10:01:54.348318+0000","last_peered":"2024-04-25T10:01:54.348318+0000","last_clean":"2024-04-25T10:01:54.348318+0000","last_became_active":"2024-04-25T10:01:52.299809+0000","last_became_peered":"2024-04-25T10:01:52.299809+0000","last_unstale":"2024-04-25T10:01:54.348318+0000","last_undegraded":"2024-04-25T10:01:54.348318+0000","last_fullsized":"2024-04-25T10:01:54.348318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:22.491159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348484+0000","last_change":"2024-04-25T10:01:53.313976+0000","last_active":"2024-04-25T10:01:54.348484+0000","last_peered":"2024-04-25T10:01:54.348484+0000","last_clean":"2024-04-25T10:01:54.348484+0000","last_became_active":"2024-04-25T10:01:53.313672+0000","last_became_peered":"2024-04-25T10:01:53.313672+0000","last_unstale":"2024-04-25T10:01:54.348484+0000","last_undegraded":"2024-04-25T10:01:54.348484+0000","last_fullsized":"2024-04-25T10:01:54.348484+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.512462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348104+0000","last_change":"2024-04-25T10:01:52.304486+0000","last_active":"2024-04-25T10:01:54.348104+0000","last_peered":"2024-04-25T10:01:54.348104+0000","last_clean":"2024-04-25T10:01:54.348104+0000","last_became_active":"2024-04-25T10:01:52.304196+0000","last_became_peered":"2024-04-25T10:01:52.304196+0000","last_unstale":"2024-04-25T10:01:54.348104+0000","last_undegraded":"2024-04-25T10:01:54.348104+0000","last_fullsized":"2024-04-25T10:01:54.348104+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:39.943512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347804+0000","last_change":"2024-04-25T10:01:53.315595+0000","last_active":"2024-04-25T10:01:54.347804+0000","last_peered":"2024-04-25T10:01:54.347804+0000","last_clean":"2024-04-25T10:01:54.347804+0000","last_became_active":"2024-04-25T10:01:53.315254+0000","last_became_peered":"2024-04-25T10:01:53.315254+0000","last_unstale":"2024-04-25T10:01:54.347804+0000","last_undegraded":"2024-04-25T10:01:54.347804+0000","last_fullsized":"2024-04-25T10:01:54.347804+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:28.489392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311619+0000","last_change":"2024-04-25T10:01:52.300611+0000","last_active":"2024-04-25T10:01:54.311619+0000","last_peered":"2024-04-25T10:01:54.311619+0000","last_clean":"2024-04-25T10:01:54.311619+0000","last_became_active":"2024-04-25T10:01:52.300310+0000","last_became_peered":"2024-04-25T10:01:52.300310+0000","last_unstale":"2024-04-25T10:01:54.311619+0000","last_undegraded":"2024-04-25T10:01:54.311619+0000","last_fullsized":"2024-04-25T10:01:54.311619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311437+0000","last_change":"2024-04-25T10:01:53.308986+0000","last_active":"2024-04-25T10:01:54.311437+0000","last_peered":"2024-04-25T10:01:54.311437+0000","last_clean":"2024-04-25T10:01:54.311437+0000","last_became_active":"2024-04-25T10:01:53.308652+0000","last_became_peered":"2024-04-25T10:01:53.308652+0000","last_unstale":"2024-04-25T10:01:54.311437+0000","last_undegraded":"2024-04-25T10:01:54.311437+0000","last_fullsized":"2024-04-25T10:01:54.311437+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320437+0000","last_change":"2024-04-25T10:01:52.301727+0000","last_active":"2024-04-25T10:01:54.320437+0000","last_peered":"2024-04-25T10:01:54.320437+0000","last_clean":"2024-04-25T10:01:54.320437+0000","last_became_active":"2024-04-25T10:01:52.301473+0000","last_became_peered":"2024-04-25T10:01:52.301473+0000","last_unstale":"2024-04-25T10:01:54.320437+0000","last_undegraded":"2024-04-25T10:01:54.320437+0000","last_fullsized":"2024-04-25T10:01:54.320437+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:29.764946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348929+0000","last_change":"2024-04-25T10:01:53.321778+0000","last_active":"2024-04-25T10:01:54.348929+0000","last_peered":"2024-04-25T10:01:54.348929+0000","last_clean":"2024-04-25T10:01:54.348929+0000","last_became_active":"2024-04-25T10:01:53.318367+0000","last_became_peered":"2024-04-25T10:01:53.318367+0000","last_unstale":"2024-04-25T10:01:54.348929+0000","last_undegraded":"2024-04-25T10:01:54.348929+0000","last_fullsized":"2024-04-25T10:01:54.348929+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:43.815198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310339+0000","last_change":"2024-04-25T10:01:52.292390+0000","last_active":"2024-04-25T10:01:54.310339+0000","last_peered":"2024-04-25T10:01:54.310339+0000","last_clean":"2024-04-25T10:01:54.310339+0000","last_became_active":"2024-04-25T10:01:52.292077+0000","last_became_peered":"2024-04-25T10:01:52.292077+0000","last_unstale":"2024-04-25T10:01:54.310339+0000","last_undegraded":"2024-04-25T10:01:54.310339+0000","last_fullsized":"2024-04-25T10:01:54.310339+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:39.960271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348105+0000","last_change":"2024-04-25T10:01:53.314768+0000","last_active":"2024-04-25T10:01:54.348105+0000","last_peered":"2024-04-25T10:01:54.348105+0000","last_clean":"2024-04-25T10:01:54.348105+0000","last_became_active":"2024-04-25T10:01:53.314481+0000","last_became_peered":"2024-04-25T10:01:53.314481+0000","last_unstale":"2024-04-25T10:01:54.348105+0000","last_undegraded":"2024-04-25T10:01:54.348105+0000","last_fullsized":"2024-04-25T10:01:54.348105+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:23.497424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320449+0000","last_change":"2024-04-25T10:01:52.303125+0000","last_active":"2024-04-25T10:01:54.320449+0000","last_peered":"2024-04-25T10:01:54.320449+0000","last_clean":"2024-04-25T10:01:54.320449+0000","last_became_active":"2024-04-25T10:01:52.302491+0000","last_became_peered":"2024-04-25T10:01:52.302491+0000","last_unstale":"2024-04-25T10:01:54.320449+0000","last_undegraded":"2024-04-25T10:01:54.320449+0000","last_fullsized":"2024-04-25T10:01:54.320449+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:18.878013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318179+0000","last_change":"2024-04-25T10:01:53.320517+0000","last_active":"2024-04-25T10:01:54.318179+0000","last_peered":"2024-04-25T10:01:54.318179+0000","last_clean":"2024-04-25T10:01:54.318179+0000","last_became_active":"2024-04-25T10:01:53.319589+0000","last_became_peered":"2024-04-25T10:01:53.319589+0000","last_unstale":"2024-04-25T10:01:54.318179+0000","last_undegraded":"2024-04-25T10:01:54.318179+0000","last_fullsized":"2024-04-25T10:01:54.318179+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.215530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318256+0000","last_change":"2024-04-25T10:01:52.301928+0000","last_active":"2024-04-25T10:01:54.318256+0000","last_peered":"2024-04-25T10:01:54.318256+0000","last_clean":"2024-04-25T10:01:54.318256+0000","last_became_active":"2024-04-25T10:01:52.301620+0000","last_became_peered":"2024-04-25T10:01:52.301620+0000","last_unstale":"2024-04-25T10:01:54.318256+0000","last_undegraded":"2024-04-25T10:01:54.318256+0000","last_fullsized":"2024-04-25T10:01:54.318256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310596+0000","last_change":"2024-04-25T10:01:53.310558+0000","last_active":"2024-04-25T10:01:54.310596+0000","last_peered":"2024-04-25T10:01:54.310596+0000","last_clean":"2024-04-25T10:01:54.310596+0000","last_became_active":"2024-04-25T10:01:53.310042+0000","last_became_peered":"2024-04-25T10:01:53.310042+0000","last_unstale":"2024-04-25T10:01:54.310596+0000","last_undegraded":"2024-04-25T10:01:54.310596+0000","last_fullsized":"2024-04-25T10:01:54.310596+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:04.286812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310155+0000","last_change":"2024-04-25T10:01:52.293082+0000","last_active":"2024-04-25T10:01:54.310155+0000","last_peered":"2024-04-25T10:01:54.310155+0000","last_clean":"2024-04-25T10:01:54.310155+0000","last_became_active":"2024-04-25T10:01:52.292730+0000","last_became_peered":"2024-04-25T10:01:52.292730+0000","last_unstale":"2024-04-25T10:01:54.310155+0000","last_undegraded":"2024-04-25T10:01:54.310155+0000","last_fullsized":"2024-04-25T10:01:54.310155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:22.491159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348694+0000","last_change":"2024-04-25T10:01:53.319606+0000","last_active":"2024-04-25T10:01:54.348694+0000","last_peered":"2024-04-25T10:01:54.348694+0000","last_clean":"2024-04-25T10:01:54.348694+0000","last_became_active":"2024-04-25T10:01:53.319303+0000","last_became_peered":"2024-04-25T10:01:53.319303+0000","last_unstale":"2024-04-25T10:01:54.348694+0000","last_undegraded":"2024-04-25T10:01:54.348694+0000","last_fullsized":"2024-04-25T10:01:54.348694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:31.529520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310559+0000","last_change":"2024-04-25T10:01:52.296284+0000","last_active":"2024-04-25T10:01:54.310559+0000","last_peered":"2024-04-25T10:01:54.310559+0000","last_clean":"2024-04-25T10:01:54.310559+0000","last_became_active":"2024-04-25T10:01:52.295533+0000","last_became_peered":"2024-04-25T10:01:52.295533+0000","last_unstale":"2024-04-25T10:01:54.310559+0000","last_undegraded":"2024-04-25T10:01:54.310559+0000","last_fullsized":"2024-04-25T10:01:54.310559+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.557737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310419+0000","last_change":"2024-04-25T10:01:53.306729+0000","last_active":"2024-04-25T10:01:54.310419+0000","last_peered":"2024-04-25T10:01:54.310419+0000","last_clean":"2024-04-25T10:01:54.310419+0000","last_became_active":"2024-04-25T10:01:53.305882+0000","last_became_peered":"2024-04-25T10:01:53.305882+0000","last_unstale":"2024-04-25T10:01:54.310419+0000","last_undegraded":"2024-04-25T10:01:54.310419+0000","last_fullsized":"2024-04-25T10:01:54.310419+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.512462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349089+0000","last_change":"2024-04-25T10:01:52.303545+0000","last_active":"2024-04-25T10:01:54.349089+0000","last_peered":"2024-04-25T10:01:54.349089+0000","last_clean":"2024-04-25T10:01:54.349089+0000","last_became_active":"2024-04-25T10:01:52.303100+0000","last_became_peered":"2024-04-25T10:01:52.303100+0000","last_unstale":"2024-04-25T10:01:54.349089+0000","last_undegraded":"2024-04-25T10:01:54.349089+0000","last_fullsized":"2024-04-25T10:01:54.349089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:54.567517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318394+0000","last_change":"2024-04-25T10:01:53.327840+0000","last_active":"2024-04-25T10:01:54.318394+0000","last_peered":"2024-04-25T10:01:54.318394+0000","last_clean":"2024-04-25T10:01:54.318394+0000","last_became_active":"2024-04-25T10:01:53.327636+0000","last_became_peered":"2024-04-25T10:01:53.327636+0000","last_unstale":"2024-04-25T10:01:54.318394+0000","last_undegraded":"2024-04-25T10:01:54.318394+0000","last_fullsized":"2024-04-25T10:01:54.318394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348429+0000","last_change":"2024-04-25T10:01:52.301182+0000","last_active":"2024-04-25T10:01:54.348429+0000","last_peered":"2024-04-25T10:01:54.348429+0000","last_clean":"2024-04-25T10:01:54.348429+0000","last_became_active":"2024-04-25T10:01:52.300836+0000","last_became_peered":"2024-04-25T10:01:52.300836+0000","last_unstale":"2024-04-25T10:01:54.348429+0000","last_undegraded":"2024-04-25T10:01:54.348429+0000","last_fullsized":"2024-04-25T10:01:54.348429+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:05.209265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347876+0000","last_change":"2024-04-25T10:01:53.308947+0000","last_active":"2024-04-25T10:01:54.347876+0000","last_peered":"2024-04-25T10:01:54.347876+0000","last_clean":"2024-04-25T10:01:54.347876+0000","last_became_active":"2024-04-25T10:01:53.308682+0000","last_became_peered":"2024-04-25T10:01:53.308682+0000","last_unstale":"2024-04-25T10:01:54.347876+0000","last_undegraded":"2024-04-25T10:01:54.347876+0000","last_fullsized":"2024-04-25T10:01:54.347876+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:09.641026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349073+0000","last_change":"2024-04-25T10:01:52.299596+0000","last_active":"2024-04-25T10:01:54.349073+0000","last_peered":"2024-04-25T10:01:54.349073+0000","last_clean":"2024-04-25T10:01:54.349073+0000","last_became_active":"2024-04-25T10:01:52.299303+0000","last_became_peered":"2024-04-25T10:01:52.299303+0000","last_unstale":"2024-04-25T10:01:54.349073+0000","last_undegraded":"2024-04-25T10:01:54.349073+0000","last_fullsized":"2024-04-25T10:01:54.349073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.253783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347539+0000","last_change":"2024-04-25T10:01:53.311926+0000","last_active":"2024-04-25T10:01:54.347539+0000","last_peered":"2024-04-25T10:01:54.347539+0000","last_clean":"2024-04-25T10:01:54.347539+0000","last_became_active":"2024-04-25T10:01:53.309452+0000","last_became_peered":"2024-04-25T10:01:53.309452+0000","last_unstale":"2024-04-25T10:01:54.347539+0000","last_undegraded":"2024-04-25T10:01:54.347539+0000","last_fullsized":"2024-04-25T10:01:54.347539+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.619357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348648+0000","last_change":"2024-04-25T10:01:52.301037+0000","last_active":"2024-04-25T10:01:54.348648+0000","last_peered":"2024-04-25T10:01:54.348648+0000","last_clean":"2024-04-25T10:01:54.348648+0000","last_became_active":"2024-04-25T10:01:52.300739+0000","last_became_peered":"2024-04-25T10:01:52.300739+0000","last_unstale":"2024-04-25T10:01:54.348648+0000","last_undegraded":"2024-04-25T10:01:54.348648+0000","last_fullsized":"2024-04-25T10:01:54.348648+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:43.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310139+0000","last_change":"2024-04-25T10:01:53.310352+0000","last_active":"2024-04-25T10:01:54.310139+0000","last_peered":"2024-04-25T10:01:54.310139+0000","last_clean":"2024-04-25T10:01:54.310139+0000","last_became_active":"2024-04-25T10:01:53.310055+0000","last_became_peered":"2024-04-25T10:01:53.310055+0000","last_unstale":"2024-04-25T10:01:54.310139+0000","last_undegraded":"2024-04-25T10:01:54.310139+0000","last_fullsized":"2024-04-25T10:01:54.310139+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:26.199709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348498+0000","last_change":"2024-04-25T10:01:52.301288+0000","last_active":"2024-04-25T10:01:54.348498+0000","last_peered":"2024-04-25T10:01:54.348498+0000","last_clean":"2024-04-25T10:01:54.348498+0000","last_became_active":"2024-04-25T10:01:52.301022+0000","last_became_peered":"2024-04-25T10:01:52.301022+0000","last_unstale":"2024-04-25T10:01:54.348498+0000","last_undegraded":"2024-04-25T10:01:54.348498+0000","last_fullsized":"2024-04-25T10:01:54.348498+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:07.389506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312173+0000","last_change":"2024-04-25T10:01:53.315399+0000","last_active":"2024-04-25T10:01:54.312173+0000","last_peered":"2024-04-25T10:01:54.312173+0000","last_clean":"2024-04-25T10:01:54.312173+0000","last_became_active":"2024-04-25T10:01:53.315137+0000","last_became_peered":"2024-04-25T10:01:53.315137+0000","last_unstale":"2024-04-25T10:01:54.312173+0000","last_undegraded":"2024-04-25T10:01:54.312173+0000","last_fullsized":"2024-04-25T10:01:54.312173+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.634999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347825+0000","last_change":"2024-04-25T10:01:52.292697+0000","last_active":"2024-04-25T10:01:54.347825+0000","last_peered":"2024-04-25T10:01:54.347825+0000","last_clean":"2024-04-25T10:01:54.347825+0000","last_became_active":"2024-04-25T10:01:52.292316+0000","last_became_peered":"2024-04-25T10:01:52.292316+0000","last_unstale":"2024-04-25T10:01:54.347825+0000","last_undegraded":"2024-04-25T10:01:54.347825+0000","last_fullsized":"2024-04-25T10:01:54.347825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:31.205159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320518+0000","last_change":"2024-04-25T10:01:53.313773+0000","last_active":"2024-04-25T10:01:54.320518+0000","last_peered":"2024-04-25T10:01:54.320518+0000","last_clean":"2024-04-25T10:01:54.320518+0000","last_became_active":"2024-04-25T10:01:53.313469+0000","last_became_peered":"2024-04-25T10:01:53.313469+0000","last_unstale":"2024-04-25T10:01:54.320518+0000","last_undegraded":"2024-04-25T10:01:54.320518+0000","last_fullsized":"2024-04-25T10:01:54.320518+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:52.218455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348408+0000","last_change":"2024-04-25T10:01:52.299811+0000","last_active":"2024-04-25T10:01:54.348408+0000","last_peered":"2024-04-25T10:01:54.348408+0000","last_clean":"2024-04-25T10:01:54.348408+0000","last_became_active":"2024-04-25T10:01:52.299468+0000","last_became_peered":"2024-04-25T10:01:52.299468+0000","last_unstale":"2024-04-25T10:01:54.348408+0000","last_undegraded":"2024-04-25T10:01:54.348408+0000","last_fullsized":"2024-04-25T10:01:54.348408+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:05.209265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310557+0000","last_change":"2024-04-25T10:01:53.309636+0000","last_active":"2024-04-25T10:01:54.310557+0000","last_peered":"2024-04-25T10:01:54.310557+0000","last_clean":"2024-04-25T10:01:54.310557+0000","last_became_active":"2024-04-25T10:01:53.309375+0000","last_became_peered":"2024-04-25T10:01:53.309375+0000","last_unstale":"2024-04-25T10:01:54.310557+0000","last_undegraded":"2024-04-25T10:01:54.310557+0000","last_fullsized":"2024-04-25T10:01:54.310557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:18.034791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348215+0000","last_change":"2024-04-25T10:01:52.300012+0000","last_active":"2024-04-25T10:01:54.348215+0000","last_peered":"2024-04-25T10:01:54.348215+0000","last_clean":"2024-04-25T10:01:54.348215+0000","last_became_active":"2024-04-25T10:01:52.299713+0000","last_became_peered":"2024-04-25T10:01:52.299713+0000","last_unstale":"2024-04-25T10:01:54.348215+0000","last_undegraded":"2024-04-25T10:01:54.348215+0000","last_fullsized":"2024-04-25T10:01:54.348215+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:39.960271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320576+0000","last_change":"2024-04-25T10:01:53.318376+0000","last_active":"2024-04-25T10:01:54.320576+0000","last_peered":"2024-04-25T10:01:54.320576+0000","last_clean":"2024-04-25T10:01:54.320576+0000","last_became_active":"2024-04-25T10:01:53.318005+0000","last_became_peered":"2024-04-25T10:01:53.318005+0000","last_unstale":"2024-04-25T10:01:54.320576+0000","last_undegraded":"2024-04-25T10:01:54.320576+0000","last_fullsized":"2024-04-25T10:01:54.320576+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:30.968859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320314+0000","last_change":"2024-04-25T10:01:52.294015+0000","last_active":"2024-04-25T10:01:54.320314+0000","last_peered":"2024-04-25T10:01:54.320314+0000","last_clean":"2024-04-25T10:01:54.320314+0000","last_became_active":"2024-04-25T10:01:52.293769+0000","last_became_peered":"2024-04-25T10:01:52.293769+0000","last_unstale":"2024-04-25T10:01:54.320314+0000","last_undegraded":"2024-04-25T10:01:54.320314+0000","last_fullsized":"2024-04-25T10:01:54.320314+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:22.062751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348551+0000","last_change":"2024-04-25T10:01:53.316595+0000","last_active":"2024-04-25T10:01:54.348551+0000","last_peered":"2024-04-25T10:01:54.348551+0000","last_clean":"2024-04-25T10:01:54.348551+0000","last_became_active":"2024-04-25T10:01:53.313992+0000","last_became_peered":"2024-04-25T10:01:53.313992+0000","last_unstale":"2024-04-25T10:01:54.348551+0000","last_undegraded":"2024-04-25T10:01:54.348551+0000","last_fullsized":"2024-04-25T10:01:54.348551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:27.623538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347985+0000","last_change":"2024-04-25T10:01:52.299879+0000","last_active":"2024-04-25T10:01:54.347985+0000","last_peered":"2024-04-25T10:01:54.347985+0000","last_clean":"2024-04-25T10:01:54.347985+0000","last_became_active":"2024-04-25T10:01:52.299301+0000","last_became_peered":"2024-04-25T10:01:52.299301+0000","last_unstale":"2024-04-25T10:01:54.347985+0000","last_undegraded":"2024-04-25T10:01:54.347985+0000","last_fullsized":"2024-04-25T10:01:54.347985+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.557737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349026+0000","last_change":"2024-04-25T10:01:53.322694+0000","last_active":"2024-04-25T10:01:54.349026+0000","last_peered":"2024-04-25T10:01:54.349026+0000","last_clean":"2024-04-25T10:01:54.349026+0000","last_became_active":"2024-04-25T10:01:53.322132+0000","last_became_peered":"2024-04-25T10:01:53.322132+0000","last_unstale":"2024-04-25T10:01:54.349026+0000","last_undegraded":"2024-04-25T10:01:54.349026+0000","last_fullsized":"2024-04-25T10:01:54.349026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:52.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348684+0000","last_change":"2024-04-25T10:01:52.302593+0000","last_active":"2024-04-25T10:01:54.348684+0000","last_peered":"2024-04-25T10:01:54.348684+0000","last_clean":"2024-04-25T10:01:54.348684+0000","last_became_active":"2024-04-25T10:01:52.300953+0000","last_became_peered":"2024-04-25T10:01:52.300953+0000","last_unstale":"2024-04-25T10:01:54.348684+0000","last_undegraded":"2024-04-25T10:01:54.348684+0000","last_fullsized":"2024-04-25T10:01:54.348684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310301+0000","last_change":"2024-04-25T10:01:53.305776+0000","last_active":"2024-04-25T10:01:54.310301+0000","last_peered":"2024-04-25T10:01:54.310301+0000","last_clean":"2024-04-25T10:01:54.310301+0000","last_became_active":"2024-04-25T10:01:53.304775+0000","last_became_peered":"2024-04-25T10:01:53.304775+0000","last_unstale":"2024-04-25T10:01:54.310301+0000","last_undegraded":"2024-04-25T10:01:54.310301+0000","last_fullsized":"2024-04-25T10:01:54.310301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:57.394066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347849+0000","last_change":"2024-04-25T10:01:52.300698+0000","last_active":"2024-04-25T10:01:54.347849+0000","last_peered":"2024-04-25T10:01:54.347849+0000","last_clean":"2024-04-25T10:01:54.347849+0000","last_became_active":"2024-04-25T10:01:52.299332+0000","last_became_peered":"2024-04-25T10:01:52.299332+0000","last_unstale":"2024-04-25T10:01:54.347849+0000","last_undegraded":"2024-04-25T10:01:54.347849+0000","last_fullsized":"2024-04-25T10:01:54.347849+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:54.567517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320740+0000","last_change":"2024-04-25T10:01:53.307063+0000","last_active":"2024-04-25T10:01:54.320740+0000","last_peered":"2024-04-25T10:01:54.320740+0000","last_clean":"2024-04-25T10:01:54.320740+0000","last_became_active":"2024-04-25T10:01:53.306775+0000","last_became_peered":"2024-04-25T10:01:53.306775+0000","last_unstale":"2024-04-25T10:01:54.320740+0000","last_undegraded":"2024-04-25T10:01:54.320740+0000","last_fullsized":"2024-04-25T10:01:54.320740+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:36.442879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320616+0000","last_change":"2024-04-25T10:01:52.300081+0000","last_active":"2024-04-25T10:01:54.320616+0000","last_peered":"2024-04-25T10:01:54.320616+0000","last_clean":"2024-04-25T10:01:54.320616+0000","last_became_active":"2024-04-25T10:01:52.299737+0000","last_became_peered":"2024-04-25T10:01:52.299737+0000","last_unstale":"2024-04-25T10:01:54.320616+0000","last_undegraded":"2024-04-25T10:01:54.320616+0000","last_fullsized":"2024-04-25T10:01:54.320616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:47.176599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347945+0000","last_change":"2024-04-25T10:01:53.314636+0000","last_active":"2024-04-25T10:01:54.347945+0000","last_peered":"2024-04-25T10:01:54.347945+0000","last_clean":"2024-04-25T10:01:54.347945+0000","last_became_active":"2024-04-25T10:01:53.314349+0000","last_became_peered":"2024-04-25T10:01:53.314349+0000","last_unstale":"2024-04-25T10:01:54.347945+0000","last_undegraded":"2024-04-25T10:01:54.347945+0000","last_fullsized":"2024-04-25T10:01:54.347945+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:04.564002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.319936+0000","last_change":"2024-04-25T10:01:52.292204+0000","last_active":"2024-04-25T10:01:54.319936+0000","last_peered":"2024-04-25T10:01:54.319936+0000","last_clean":"2024-04-25T10:01:54.319936+0000","last_became_active":"2024-04-25T10:01:52.291829+0000","last_became_peered":"2024-04-25T10:01:52.291829+0000","last_unstale":"2024-04-25T10:01:54.319936+0000","last_undegraded":"2024-04-25T10:01:54.319936+0000","last_fullsized":"2024-04-25T10:01:54.319936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:30.816226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348453+0000","last_change":"2024-04-25T10:01:53.306759+0000","last_active":"2024-04-25T10:01:54.348453+0000","last_peered":"2024-04-25T10:01:54.348453+0000","last_clean":"2024-04-25T10:01:54.348453+0000","last_became_active":"2024-04-25T10:01:53.306408+0000","last_became_peered":"2024-04-25T10:01:53.306408+0000","last_unstale":"2024-04-25T10:01:54.348453+0000","last_undegraded":"2024-04-25T10:01:54.348453+0000","last_fullsized":"2024-04-25T10:01:54.348453+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:42.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318511+0000","last_change":"2024-04-25T10:01:52.293197+0000","last_active":"2024-04-25T10:01:54.318511+0000","last_peered":"2024-04-25T10:01:54.318511+0000","last_clean":"2024-04-25T10:01:54.318511+0000","last_became_active":"2024-04-25T10:01:52.292609+0000","last_became_peered":"2024-04-25T10:01:52.292609+0000","last_unstale":"2024-04-25T10:01:54.318511+0000","last_undegraded":"2024-04-25T10:01:54.318511+0000","last_fullsized":"2024-04-25T10:01:54.318511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348125+0000","last_change":"2024-04-25T10:01:53.312259+0000","last_active":"2024-04-25T10:01:54.348125+0000","last_peered":"2024-04-25T10:01:54.348125+0000","last_clean":"2024-04-25T10:01:54.348125+0000","last_became_active":"2024-04-25T10:01:53.311938+0000","last_became_peered":"2024-04-25T10:01:53.311938+0000","last_unstale":"2024-04-25T10:01:54.348125+0000","last_undegraded":"2024-04-25T10:01:54.348125+0000","last_fullsized":"2024-04-25T10:01:54.348125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:11.878572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320728+0000","last_change":"2024-04-25T10:01:52.299885+0000","last_active":"2024-04-25T10:01:54.320728+0000","last_peered":"2024-04-25T10:01:54.320728+0000","last_clean":"2024-04-25T10:01:54.320728+0000","last_became_active":"2024-04-25T10:01:52.299576+0000","last_became_peered":"2024-04-25T10:01:52.299576+0000","last_unstale":"2024-04-25T10:01:54.320728+0000","last_undegraded":"2024-04-25T10:01:54.320728+0000","last_fullsized":"2024-04-25T10:01:54.320728+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:09.750891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320361+0000","last_change":"2024-04-25T10:01:53.313711+0000","last_active":"2024-04-25T10:01:54.320361+0000","last_peered":"2024-04-25T10:01:54.320361+0000","last_clean":"2024-04-25T10:01:54.320361+0000","last_became_active":"2024-04-25T10:01:53.313416+0000","last_became_peered":"2024-04-25T10:01:53.313416+0000","last_unstale":"2024-04-25T10:01:54.320361+0000","last_undegraded":"2024-04-25T10:01:54.320361+0000","last_fullsized":"2024-04-25T10:01:54.320361+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:39.432005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311723+0000","last_change":"2024-04-25T10:01:52.289539+0000","last_active":"2024-04-25T10:01:54.311723+0000","last_peered":"2024-04-25T10:01:54.311723+0000","last_clean":"2024-04-25T10:01:54.311723+0000","last_became_active":"2024-04-25T10:01:52.289261+0000","last_became_peered":"2024-04-25T10:01:52.289261+0000","last_unstale":"2024-04-25T10:01:54.311723+0000","last_undegraded":"2024-04-25T10:01:54.311723+0000","last_fullsized":"2024-04-25T10:01:54.311723+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348597+0000","last_change":"2024-04-25T10:01:53.322700+0000","last_active":"2024-04-25T10:01:54.348597+0000","last_peered":"2024-04-25T10:01:54.348597+0000","last_clean":"2024-04-25T10:01:54.348597+0000","last_became_active":"2024-04-25T10:01:53.320500+0000","last_became_peered":"2024-04-25T10:01:53.320500+0000","last_unstale":"2024-04-25T10:01:54.348597+0000","last_undegraded":"2024-04-25T10:01:54.348597+0000","last_fullsized":"2024-04-25T10:01:54.348597+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.362351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311806+0000","last_change":"2024-04-25T10:01:52.292733+0000","last_active":"2024-04-25T10:01:54.311806+0000","last_peered":"2024-04-25T10:01:54.311806+0000","last_clean":"2024-04-25T10:01:54.311806+0000","last_became_active":"2024-04-25T10:01:52.292040+0000","last_became_peered":"2024-04-25T10:01:52.292040+0000","last_unstale":"2024-04-25T10:01:54.311806+0000","last_undegraded":"2024-04-25T10:01:54.311806+0000","last_fullsized":"2024-04-25T10:01:54.311806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348029+0000","last_change":"2024-04-25T10:01:53.311823+0000","last_active":"2024-04-25T10:01:54.348029+0000","last_peered":"2024-04-25T10:01:54.348029+0000","last_clean":"2024-04-25T10:01:54.348029+0000","last_became_active":"2024-04-25T10:01:53.311509+0000","last_became_peered":"2024-04-25T10:01:53.311509+0000","last_unstale":"2024-04-25T10:01:54.348029+0000","last_undegraded":"2024-04-25T10:01:54.348029+0000","last_fullsized":"2024-04-25T10:01:54.348029+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.340419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320280+0000","last_change":"2024-04-25T10:01:52.299938+0000","last_active":"2024-04-25T10:01:54.320280+0000","last_peered":"2024-04-25T10:01:54.320280+0000","last_clean":"2024-04-25T10:01:54.320280+0000","last_became_active":"2024-04-25T10:01:52.299592+0000","last_became_peered":"2024-04-25T10:01:52.299592+0000","last_unstale":"2024-04-25T10:01:54.320280+0000","last_undegraded":"2024-04-25T10:01:54.320280+0000","last_fullsized":"2024-04-25T10:01:54.320280+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:03.815918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348251+0000","last_change":"2024-04-25T10:01:53.308285+0000","last_active":"2024-04-25T10:01:54.348251+0000","last_peered":"2024-04-25T10:01:54.348251+0000","last_clean":"2024-04-25T10:01:54.348251+0000","last_became_active":"2024-04-25T10:01:53.308018+0000","last_became_peered":"2024-04-25T10:01:53.308018+0000","last_unstale":"2024-04-25T10:01:54.348251+0000","last_undegraded":"2024-04-25T10:01:54.348251+0000","last_fullsized":"2024-04-25T10:01:54.348251+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347851+0000","last_change":"2024-04-25T10:01:52.301345+0000","last_active":"2024-04-25T10:01:54.347851+0000","last_peered":"2024-04-25T10:01:54.347851+0000","last_clean":"2024-04-25T10:01:54.347851+0000","last_became_active":"2024-04-25T10:01:52.301002+0000","last_became_peered":"2024-04-25T10:01:52.301002+0000","last_unstale":"2024-04-25T10:01:54.347851+0000","last_undegraded":"2024-04-25T10:01:54.347851+0000","last_fullsized":"2024-04-25T10:01:54.347851+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348318+0000","last_change":"2024-04-25T10:01:53.314890+0000","last_active":"2024-04-25T10:01:54.348318+0000","last_peered":"2024-04-25T10:01:54.348318+0000","last_clean":"2024-04-25T10:01:54.348318+0000","last_became_active":"2024-04-25T10:01:53.314629+0000","last_became_peered":"2024-04-25T10:01:53.314629+0000","last_unstale":"2024-04-25T10:01:54.348318+0000","last_undegraded":"2024-04-25T10:01:54.348318+0000","last_fullsized":"2024-04-25T10:01:54.348318+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:56.754763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348508+0000","last_change":"2024-04-25T10:01:52.300200+0000","last_active":"2024-04-25T10:01:54.348508+0000","last_peered":"2024-04-25T10:01:54.348508+0000","last_clean":"2024-04-25T10:01:54.348508+0000","last_became_active":"2024-04-25T10:01:52.299829+0000","last_became_peered":"2024-04-25T10:01:52.299829+0000","last_unstale":"2024-04-25T10:01:54.348508+0000","last_undegraded":"2024-04-25T10:01:54.348508+0000","last_fullsized":"2024-04-25T10:01:54.348508+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":355,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347939+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:54.347939+0000","last_peered":"2024-04-25T10:01:54.347939+0000","last_clean":"2024-04-25T10:01:54.347939+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:54.347939+0000","last_undegraded":"2024-04-25T10:01:54.347939+0000","last_fullsized":"2024-04-25T10:01:54.347939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2117632,"data_stored":1292799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46536,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325304,"statfs":{"total":96636764160,"available":96589111296,"internally_reserved":0,"allocated":1499136,"data_stored":695545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45140,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326700,"statfs":{"total":96636764160,"available":96590540800,"internally_reserved":0,"allocated":1515520,"data_stored":701815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1519616,"data_stored":703267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50716,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321124,"statfs":{"total":96636764160,"available":96584830976,"internally_reserved":0,"allocated":1523712,"data_stored":703301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1507328,"data_stored":701655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50700,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321140,"statfs":{"total":96636764160,"available":96584847360,"internally_reserved":0,"allocated":1499136,"data_stored":694985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51272,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320568,"statfs":{"total":96636764160,"available":96584261632,"internally_reserved":0,"allocated":2093056,"data_stored":1285353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:01:58.549 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-25T10:01:58.544+0000 7f85e1348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:01.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.203+0000 7f57af370640 1 Processor -- start 2024-04-25T10:02:01.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.203+0000 7f57af370640 1 -- start start 2024-04-25T10:02:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57af370640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57a815dc90 0x7f57a81625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57af370640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57a80d1810 0x7f57a80d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57af370640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80cce50 con 0x7f57a80cc8e0 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57af370640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80d21c0 con 0x7f57a815dc90 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57af370640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a8161af0 con 0x7f57a80d1810 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57ad10c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57a815dc90 0x7f57a81625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57ac90b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f57a80cc8e0 0x7f57a815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42998/0 (socket says 172.21.15.148:42998) 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.204+0000 7f57ac90b640 1 -- 172.21.15.148:0/2389918741 learned_addr learned my addr 172.21.15.148:0/2389918741 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f57ad90d640 1 --2- 172.21.15.148:0/2389918741 >> v2:172.21.15.188:3300/0 conn(0x7f57a80d1810 0x7f57a80d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f57ad10c640 1 -- 172.21.15.148:0/2389918741 >> v2:172.21.15.188:3300/0 conn(0x7f57a80d1810 msgr2=0x7f57a80d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f57ad10c640 1 --2- 172.21.15.148:0/2389918741 >> v2:172.21.15.188:3300/0 conn(0x7f57a80d1810 0x7f57a80d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f57ad10c640 1 -- 172.21.15.148:0/2389918741 >> v1:172.21.15.148:6789/0 conn(0x7f57a80cc8e0 legacy=0x7f57a815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f579ffff640 1 -- 172.21.15.148:0/2389918741 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628909655 0 0) 0x7f57a80cce50 con 0x7f57a80cc8e0 2024-04-25T10:02:01.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f57ad10c640 1 -- 172.21.15.148:0/2389918741 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8161dd0 con 0x7f57a815dc90 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f57ad10c640 1 --2- 172.21.15.148:0/2389918741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a81625d0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cdbb0 tx=0x7f57a402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534075440ca50da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.205+0000 7f579ffff640 1 -- 172.21.15.148:0/2389918741 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4037070 con 0x7f57a815dc90 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f579ffff640 1 -- 172.21.15.148:0/2389918741 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a4032440 con 0x7f57a815dc90 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f579ffff640 1 -- 172.21.15.148:0/2389918741 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a402ae60 con 0x7f57a815dc90 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 -- 172.21.15.148:0/2389918741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 msgr2=0x7f57a81625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 --2- 172.21.15.148:0/2389918741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a81625d0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cdbb0 tx=0x7f57a402a110 comp rx=0 tx=0).stop 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 -- 172.21.15.148:0/2389918741 shutdown_connections 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 --2- 172.21.15.148:0/2389918741 >> v2:172.21.15.188:3300/0 conn(0x7f57a80d1810 0x7f57a80d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 --2- 172.21.15.148:0/2389918741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a81625d0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 -- 172.21.15.148:0/2389918741 >> 172.21.15.148:0/2389918741 conn(0x7f57a806f590 msgr2=0x7f57a806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 -- 172.21.15.148:0/2389918741 shutdown_connections 2024-04-25T10:02:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.206+0000 7f57af370640 1 -- 172.21.15.148:0/2389918741 wait complete. 2024-04-25T10:02:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.207+0000 7f57af370640 1 Processor -- start 2024-04-25T10:02:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.207+0000 7f57af370640 1 -- start start 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57af370640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57a80cc8e0 0x7f57a80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57af370640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57a80d1810 0x7f57a80b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57af370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a815fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57af370640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a815e070 con 0x7f57a815dc90 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57af370640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80cd130 con 0x7f57a80cc8e0 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57af370640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a815e1f0 con 0x7f57a80d1810 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57ad90d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a815fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57ad90d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a815fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42858/0 (socket says 172.21.15.148:42858) 2024-04-25T10:02:01.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57ad90d640 1 -- 172.21.15.148:0/2199057426 learned_addr learned my addr 172.21.15.148:0/2199057426 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57ad10c640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57a80cc8e0 0x7f57a80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.208+0000 7f57ac90b640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57a80d1810 0x7f57a80b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57ad90d640 1 -- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57a80d1810 msgr2=0x7f57a80b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57ad90d640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57a80d1810 0x7f57a80b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57ad90d640 1 -- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57a80cc8e0 msgr2=0x7f57a80b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57ad90d640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57a80cc8e0 0x7f57a80b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57ad90d640 1 -- 172.21.15.148:0/2199057426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80c54c0 con 0x7f57a815dc90 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57ad90d640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a815fc00 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f57a0007dc0 tx=0x7f57a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0014420 con 0x7f57a815dc90 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0019070 con 0x7f57a815dc90 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a80c57a0 con 0x7f57a815dc90 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0014820 con 0x7f57a815dc90 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.209+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a80c5d60 con 0x7f57a815dc90 2024-04-25T10:02:01.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.210+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8073990 con 0x7f57a815dc90 2024-04-25T10:02:01.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.210+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57a00149c0 con 0x7f57a815dc90 2024-04-25T10:02:01.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.211+0000 7f579dffb640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.211+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57a008aa80 con 0x7f57a815dc90 2024-04-25T10:02:01.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.211+0000 7f57ad10c640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.211+0000 7f57ad10c640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b4d0 0x7f578c06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.215+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57a0059800 con 0x7f57a815dc90 2024-04-25T10:02:01.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.387+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57a80cf6b0 con 0x7f578c06b4d0 2024-04-25T10:02:01.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.394+0000 7f579dffb640 1 -- 172.21.15.148:0/2199057426 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252376 (secure 0 0 0) 0x7f57a80cf6b0 con 0x7f578c06b4d0 2024-04-25T10:02:01.401 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:01.402 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:02:01.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b4d0 msgr2=0x7f578c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b4d0 0x7f578c06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f57a4034000 tx=0x7f57a402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:01.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 msgr2=0x7f57a815fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a815fc00 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f57a0007dc0 tx=0x7f57a000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 shutdown_connections 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57a815dc90 0x7f57a815fc00 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57a80d1810 0x7f57a80b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 --2- 172.21.15.148:0/2199057426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57a80cc8e0 0x7f57a80b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 >> 172.21.15.148:0/2199057426 conn(0x7f57a806f590 msgr2=0x7f57a80cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 shutdown_connections 2024-04-25T10:02:01.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.397+0000 7f57af370640 1 -- 172.21.15.148:0/2199057426 wait complete. 2024-04-25T10:02:01.412 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-04-25T10:02:01.114786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":387392,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":374259,"kb_avail":754587328,"statfs":{"total":773094113280,"available":772697423872,"internally_reserved":0,"allocated":13312000,"data_stored":6798796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":383241776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001503"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312163+0000","last_change":"2024-04-25T10:01:53.308968+0000","last_active":"2024-04-25T10:01:54.312163+0000","last_peered":"2024-04-25T10:01:54.312163+0000","last_clean":"2024-04-25T10:01:54.312163+0000","last_became_active":"2024-04-25T10:01:53.308683+0000","last_became_peered":"2024-04-25T10:01:53.308683+0000","last_unstale":"2024-04-25T10:01:54.312163+0000","last_undegraded":"2024-04-25T10:01:54.312163+0000","last_fullsized":"2024-04-25T10:01:54.312163+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:13.369773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348370+0000","last_change":"2024-04-25T10:01:53.326585+0000","last_active":"2024-04-25T10:01:54.348370+0000","last_peered":"2024-04-25T10:01:54.348370+0000","last_clean":"2024-04-25T10:01:54.348370+0000","last_became_active":"2024-04-25T10:01:53.325978+0000","last_became_peered":"2024-04-25T10:01:53.325978+0000","last_unstale":"2024-04-25T10:01:54.348370+0000","last_undegraded":"2024-04-25T10:01:54.348370+0000","last_fullsized":"2024-04-25T10:01:54.348370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:04.286812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348198+0000","last_change":"2024-04-25T10:01:53.319634+0000","last_active":"2024-04-25T10:01:54.348198+0000","last_peered":"2024-04-25T10:01:54.348198+0000","last_clean":"2024-04-25T10:01:54.348198+0000","last_became_active":"2024-04-25T10:01:53.319370+0000","last_became_peered":"2024-04-25T10:01:53.319370+0000","last_unstale":"2024-04-25T10:01:54.348198+0000","last_undegraded":"2024-04-25T10:01:54.348198+0000","last_fullsized":"2024-04-25T10:01:54.348198+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:18.034791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348276+0000","last_change":"2024-04-25T10:01:53.314499+0000","last_active":"2024-04-25T10:01:54.348276+0000","last_peered":"2024-04-25T10:01:54.348276+0000","last_clean":"2024-04-25T10:01:54.348276+0000","last_became_active":"2024-04-25T10:01:53.314191+0000","last_became_peered":"2024-04-25T10:01:53.314191+0000","last_unstale":"2024-04-25T10:01:54.348276+0000","last_undegraded":"2024-04-25T10:01:54.348276+0000","last_fullsized":"2024-04-25T10:01:54.348276+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:26.199709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311504+0000","last_change":"2024-04-25T10:01:53.313912+0000","last_active":"2024-04-25T10:01:54.311504+0000","last_peered":"2024-04-25T10:01:54.311504+0000","last_clean":"2024-04-25T10:01:54.311504+0000","last_became_active":"2024-04-25T10:01:53.313635+0000","last_became_peered":"2024-04-25T10:01:53.313635+0000","last_unstale":"2024-04-25T10:01:54.311504+0000","last_undegraded":"2024-04-25T10:01:54.311504+0000","last_fullsized":"2024-04-25T10:01:54.311504+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320356+0000","last_change":"2024-04-25T10:01:53.317523+0000","last_active":"2024-04-25T10:01:54.320356+0000","last_peered":"2024-04-25T10:01:54.320356+0000","last_clean":"2024-04-25T10:01:54.320356+0000","last_became_active":"2024-04-25T10:01:53.317248+0000","last_became_peered":"2024-04-25T10:01:53.317248+0000","last_unstale":"2024-04-25T10:01:54.320356+0000","last_undegraded":"2024-04-25T10:01:54.320356+0000","last_fullsized":"2024-04-25T10:01:54.320356+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:41.914139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320327+0000","last_change":"2024-04-25T10:01:53.314768+0000","last_active":"2024-04-25T10:01:54.320327+0000","last_peered":"2024-04-25T10:01:54.320327+0000","last_clean":"2024-04-25T10:01:54.320327+0000","last_became_active":"2024-04-25T10:01:53.314504+0000","last_became_peered":"2024-04-25T10:01:53.314504+0000","last_unstale":"2024-04-25T10:01:54.320327+0000","last_undegraded":"2024-04-25T10:01:54.320327+0000","last_fullsized":"2024-04-25T10:01:54.320327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:40.662254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348425+0000","last_change":"2024-04-25T10:01:53.320926+0000","last_active":"2024-04-25T10:01:54.348425+0000","last_peered":"2024-04-25T10:01:54.348425+0000","last_clean":"2024-04-25T10:01:54.348425+0000","last_became_active":"2024-04-25T10:01:53.320663+0000","last_became_peered":"2024-04-25T10:01:53.320663+0000","last_unstale":"2024-04-25T10:01:54.348425+0000","last_undegraded":"2024-04-25T10:01:54.348425+0000","last_fullsized":"2024-04-25T10:01:54.348425+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:38.131544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348471+0000","last_change":"2024-04-25T10:01:53.320793+0000","last_active":"2024-04-25T10:01:54.348471+0000","last_peered":"2024-04-25T10:01:54.348471+0000","last_clean":"2024-04-25T10:01:54.348471+0000","last_became_active":"2024-04-25T10:01:53.320516+0000","last_became_peered":"2024-04-25T10:01:53.320516+0000","last_unstale":"2024-04-25T10:01:54.348471+0000","last_undegraded":"2024-04-25T10:01:54.348471+0000","last_fullsized":"2024-04-25T10:01:54.348471+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:21.226281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348274+0000","last_change":"2024-04-25T10:01:53.308828+0000","last_active":"2024-04-25T10:01:54.348274+0000","last_peered":"2024-04-25T10:01:54.348274+0000","last_clean":"2024-04-25T10:01:54.348274+0000","last_became_active":"2024-04-25T10:01:53.308560+0000","last_became_peered":"2024-04-25T10:01:53.308560+0000","last_unstale":"2024-04-25T10:01:54.348274+0000","last_undegraded":"2024-04-25T10:01:54.348274+0000","last_fullsized":"2024-04-25T10:01:54.348274+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312000+0000","last_change":"2024-04-25T10:01:53.315244+0000","last_active":"2024-04-25T10:01:54.312000+0000","last_peered":"2024-04-25T10:01:54.312000+0000","last_clean":"2024-04-25T10:01:54.312000+0000","last_became_active":"2024-04-25T10:01:53.314991+0000","last_became_peered":"2024-04-25T10:01:53.314991+0000","last_unstale":"2024-04-25T10:01:54.312000+0000","last_undegraded":"2024-04-25T10:01:54.312000+0000","last_fullsized":"2024-04-25T10:01:54.312000+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:10.260048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320663+0000","last_change":"2024-04-25T10:01:53.314287+0000","last_active":"2024-04-25T10:01:54.320663+0000","last_peered":"2024-04-25T10:01:54.320663+0000","last_clean":"2024-04-25T10:01:54.320663+0000","last_became_active":"2024-04-25T10:01:53.313201+0000","last_became_peered":"2024-04-25T10:01:53.313201+0000","last_unstale":"2024-04-25T10:01:54.320663+0000","last_undegraded":"2024-04-25T10:01:54.320663+0000","last_fullsized":"2024-04-25T10:01:54.320663+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.462869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347955+0000","last_change":"2024-04-25T10:01:53.307341+0000","last_active":"2024-04-25T10:01:54.347955+0000","last_peered":"2024-04-25T10:01:54.347955+0000","last_clean":"2024-04-25T10:01:54.347955+0000","last_became_active":"2024-04-25T10:01:53.307024+0000","last_became_peered":"2024-04-25T10:01:53.307024+0000","last_unstale":"2024-04-25T10:01:54.347955+0000","last_undegraded":"2024-04-25T10:01:54.347955+0000","last_fullsized":"2024-04-25T10:01:54.347955+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:40.966537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348781+0000","last_change":"2024-04-25T10:01:53.326968+0000","last_active":"2024-04-25T10:01:54.348781+0000","last_peered":"2024-04-25T10:01:54.348781+0000","last_clean":"2024-04-25T10:01:54.348781+0000","last_became_active":"2024-04-25T10:01:53.326663+0000","last_became_peered":"2024-04-25T10:01:53.326663+0000","last_unstale":"2024-04-25T10:01:54.348781+0000","last_undegraded":"2024-04-25T10:01:54.348781+0000","last_fullsized":"2024-04-25T10:01:54.348781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:03.834566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311370+0000","last_change":"2024-04-25T10:01:53.314815+0000","last_active":"2024-04-25T10:01:54.311370+0000","last_peered":"2024-04-25T10:01:54.311370+0000","last_clean":"2024-04-25T10:01:54.311370+0000","last_became_active":"2024-04-25T10:01:53.314511+0000","last_became_peered":"2024-04-25T10:01:53.314511+0000","last_unstale":"2024-04-25T10:01:54.311370+0000","last_undegraded":"2024-04-25T10:01:54.311370+0000","last_fullsized":"2024-04-25T10:01:54.311370+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320222+0000","last_change":"2024-04-25T10:01:53.309219+0000","last_active":"2024-04-25T10:01:54.320222+0000","last_peered":"2024-04-25T10:01:54.320222+0000","last_clean":"2024-04-25T10:01:54.320222+0000","last_became_active":"2024-04-25T10:01:53.308965+0000","last_became_peered":"2024-04-25T10:01:53.308965+0000","last_unstale":"2024-04-25T10:01:54.320222+0000","last_undegraded":"2024-04-25T10:01:54.320222+0000","last_fullsized":"2024-04-25T10:01:54.320222+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:00.180829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348329+0000","last_change":"2024-04-25T10:01:53.315092+0000","last_active":"2024-04-25T10:01:54.348329+0000","last_peered":"2024-04-25T10:01:54.348329+0000","last_clean":"2024-04-25T10:01:54.348329+0000","last_became_active":"2024-04-25T10:01:53.314365+0000","last_became_peered":"2024-04-25T10:01:53.314365+0000","last_unstale":"2024-04-25T10:01:54.348329+0000","last_undegraded":"2024-04-25T10:01:54.348329+0000","last_fullsized":"2024-04-25T10:01:54.348329+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348538+0000","last_change":"2024-04-25T10:01:53.326698+0000","last_active":"2024-04-25T10:01:54.348538+0000","last_peered":"2024-04-25T10:01:54.348538+0000","last_clean":"2024-04-25T10:01:54.348538+0000","last_became_active":"2024-04-25T10:01:53.325993+0000","last_became_peered":"2024-04-25T10:01:53.325993+0000","last_unstale":"2024-04-25T10:01:54.348538+0000","last_undegraded":"2024-04-25T10:01:54.348538+0000","last_fullsized":"2024-04-25T10:01:54.348538+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318420+0000","last_change":"2024-04-25T10:01:53.320785+0000","last_active":"2024-04-25T10:01:54.318420+0000","last_peered":"2024-04-25T10:01:54.318420+0000","last_clean":"2024-04-25T10:01:54.318420+0000","last_became_active":"2024-04-25T10:01:53.319963+0000","last_became_peered":"2024-04-25T10:01:53.319963+0000","last_unstale":"2024-04-25T10:01:54.318420+0000","last_undegraded":"2024-04-25T10:01:54.318420+0000","last_fullsized":"2024-04-25T10:01:54.318420+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347573+0000","last_change":"2024-04-25T10:01:53.307833+0000","last_active":"2024-04-25T10:01:54.347573+0000","last_peered":"2024-04-25T10:01:54.347573+0000","last_clean":"2024-04-25T10:01:54.347573+0000","last_became_active":"2024-04-25T10:01:53.307583+0000","last_became_peered":"2024-04-25T10:01:53.307583+0000","last_unstale":"2024-04-25T10:01:54.347573+0000","last_undegraded":"2024-04-25T10:01:54.347573+0000","last_fullsized":"2024-04-25T10:01:54.347573+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:14.460352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348432+0000","last_change":"2024-04-25T10:01:53.313805+0000","last_active":"2024-04-25T10:01:54.348432+0000","last_peered":"2024-04-25T10:01:54.348432+0000","last_clean":"2024-04-25T10:01:54.348432+0000","last_became_active":"2024-04-25T10:01:53.313489+0000","last_became_peered":"2024-04-25T10:01:53.313489+0000","last_unstale":"2024-04-25T10:01:54.348432+0000","last_undegraded":"2024-04-25T10:01:54.348432+0000","last_fullsized":"2024-04-25T10:01:54.348432+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:57.394066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311689+0000","last_change":"2024-04-25T10:01:53.317853+0000","last_active":"2024-04-25T10:01:54.311689+0000","last_peered":"2024-04-25T10:01:54.311689+0000","last_clean":"2024-04-25T10:01:54.311689+0000","last_became_active":"2024-04-25T10:01:53.317561+0000","last_became_peered":"2024-04-25T10:01:53.317561+0000","last_unstale":"2024-04-25T10:01:54.311689+0000","last_undegraded":"2024-04-25T10:01:54.311689+0000","last_fullsized":"2024-04-25T10:01:54.311689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:06.139972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348127+0000","last_change":"2024-04-25T10:01:53.318514+0000","last_active":"2024-04-25T10:01:54.348127+0000","last_peered":"2024-04-25T10:01:54.348127+0000","last_clean":"2024-04-25T10:01:54.348127+0000","last_became_active":"2024-04-25T10:01:53.318206+0000","last_became_peered":"2024-04-25T10:01:53.318206+0000","last_unstale":"2024-04-25T10:01:54.348127+0000","last_undegraded":"2024-04-25T10:01:54.348127+0000","last_fullsized":"2024-04-25T10:01:54.348127+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:08.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.309900+0000","last_change":"2024-04-25T10:01:53.316554+0000","last_active":"2024-04-25T10:01:54.309900+0000","last_peered":"2024-04-25T10:01:54.309900+0000","last_clean":"2024-04-25T10:01:54.309900+0000","last_became_active":"2024-04-25T10:01:53.316218+0000","last_became_peered":"2024-04-25T10:01:53.316218+0000","last_unstale":"2024-04-25T10:01:54.309900+0000","last_undegraded":"2024-04-25T10:01:54.309900+0000","last_fullsized":"2024-04-25T10:01:54.309900+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.340419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310385+0000","last_change":"2024-04-25T10:01:53.307075+0000","last_active":"2024-04-25T10:01:54.310385+0000","last_peered":"2024-04-25T10:01:54.310385+0000","last_clean":"2024-04-25T10:01:54.310385+0000","last_became_active":"2024-04-25T10:01:53.304942+0000","last_became_peered":"2024-04-25T10:01:53.304942+0000","last_unstale":"2024-04-25T10:01:54.310385+0000","last_undegraded":"2024-04-25T10:01:54.310385+0000","last_fullsized":"2024-04-25T10:01:54.310385+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:21.226281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347834+0000","last_change":"2024-04-25T10:01:53.306396+0000","last_active":"2024-04-25T10:01:54.347834+0000","last_peered":"2024-04-25T10:01:54.347834+0000","last_clean":"2024-04-25T10:01:54.347834+0000","last_became_active":"2024-04-25T10:01:53.306127+0000","last_became_peered":"2024-04-25T10:01:53.306127+0000","last_unstale":"2024-04-25T10:01:54.347834+0000","last_undegraded":"2024-04-25T10:01:54.347834+0000","last_fullsized":"2024-04-25T10:01:54.347834+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:27.335539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318479+0000","last_change":"2024-04-25T10:01:53.319940+0000","last_active":"2024-04-25T10:01:54.318479+0000","last_peered":"2024-04-25T10:01:54.318479+0000","last_clean":"2024-04-25T10:01:54.318479+0000","last_became_active":"2024-04-25T10:01:53.319590+0000","last_became_peered":"2024-04-25T10:01:53.319590+0000","last_unstale":"2024-04-25T10:01:54.318479+0000","last_undegraded":"2024-04-25T10:01:54.318479+0000","last_fullsized":"2024-04-25T10:01:54.318479+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:44.238342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320676+0000","last_change":"2024-04-25T10:01:53.316178+0000","last_active":"2024-04-25T10:01:54.320676+0000","last_peered":"2024-04-25T10:01:54.320676+0000","last_clean":"2024-04-25T10:01:54.320676+0000","last_became_active":"2024-04-25T10:01:53.314889+0000","last_became_peered":"2024-04-25T10:01:53.314889+0000","last_unstale":"2024-04-25T10:01:54.320676+0000","last_undegraded":"2024-04-25T10:01:54.320676+0000","last_fullsized":"2024-04-25T10:01:54.320676+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:28.026534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347913+0000","last_change":"2024-04-25T10:01:53.316236+0000","last_active":"2024-04-25T10:01:54.347913+0000","last_peered":"2024-04-25T10:01:54.347913+0000","last_clean":"2024-04-25T10:01:54.347913+0000","last_became_active":"2024-04-25T10:01:53.315521+0000","last_became_peered":"2024-04-25T10:01:53.315521+0000","last_unstale":"2024-04-25T10:01:54.347913+0000","last_undegraded":"2024-04-25T10:01:54.347913+0000","last_fullsized":"2024-04-25T10:01:54.347913+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:54.725287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348895+0000","last_change":"2024-04-25T10:01:53.317694+0000","last_active":"2024-04-25T10:01:54.348895+0000","last_peered":"2024-04-25T10:01:54.348895+0000","last_clean":"2024-04-25T10:01:54.348895+0000","last_became_active":"2024-04-25T10:01:53.317254+0000","last_became_peered":"2024-04-25T10:01:53.317254+0000","last_unstale":"2024-04-25T10:01:54.348895+0000","last_undegraded":"2024-04-25T10:01:54.348895+0000","last_fullsized":"2024-04-25T10:01:54.348895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:59.725012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310217+0000","last_change":"2024-04-25T10:01:53.303113+0000","last_active":"2024-04-25T10:01:54.310217+0000","last_peered":"2024-04-25T10:01:54.310217+0000","last_clean":"2024-04-25T10:01:54.310217+0000","last_became_active":"2024-04-25T10:01:53.302788+0000","last_became_peered":"2024-04-25T10:01:53.302788+0000","last_unstale":"2024-04-25T10:01:54.310217+0000","last_undegraded":"2024-04-25T10:01:54.310217+0000","last_fullsized":"2024-04-25T10:01:54.310217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:56.754763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.309915+0000","last_change":"2024-04-25T10:01:53.316148+0000","last_active":"2024-04-25T10:01:54.309915+0000","last_peered":"2024-04-25T10:01:54.309915+0000","last_clean":"2024-04-25T10:01:54.309915+0000","last_became_active":"2024-04-25T10:01:53.315873+0000","last_became_peered":"2024-04-25T10:01:53.315873+0000","last_unstale":"2024-04-25T10:01:54.309915+0000","last_undegraded":"2024-04-25T10:01:54.309915+0000","last_fullsized":"2024-04-25T10:01:54.309915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:08.722633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348109+0000","last_change":"2024-04-25T10:01:53.310855+0000","last_active":"2024-04-25T10:01:54.348109+0000","last_peered":"2024-04-25T10:01:54.348109+0000","last_clean":"2024-04-25T10:01:54.348109+0000","last_became_active":"2024-04-25T10:01:53.310567+0000","last_became_peered":"2024-04-25T10:01:53.310567+0000","last_unstale":"2024-04-25T10:01:54.348109+0000","last_undegraded":"2024-04-25T10:01:54.348109+0000","last_fullsized":"2024-04-25T10:01:54.348109+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.215530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318170+0000","last_change":"2024-04-25T10:01:52.296531+0000","last_active":"2024-04-25T10:01:54.318170+0000","last_peered":"2024-04-25T10:01:54.318170+0000","last_clean":"2024-04-25T10:01:54.318170+0000","last_became_active":"2024-04-25T10:01:52.296192+0000","last_became_peered":"2024-04-25T10:01:52.296192+0000","last_unstale":"2024-04-25T10:01:54.318170+0000","last_undegraded":"2024-04-25T10:01:54.318170+0000","last_fullsized":"2024-04-25T10:01:54.318170+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320541+0000","last_change":"2024-04-25T10:01:53.314991+0000","last_active":"2024-04-25T10:01:54.320541+0000","last_peered":"2024-04-25T10:01:54.320541+0000","last_clean":"2024-04-25T10:01:54.320541+0000","last_became_active":"2024-04-25T10:01:53.314698+0000","last_became_peered":"2024-04-25T10:01:53.314698+0000","last_unstale":"2024-04-25T10:01:54.320541+0000","last_undegraded":"2024-04-25T10:01:54.320541+0000","last_fullsized":"2024-04-25T10:01:54.320541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:49.360827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348276+0000","last_change":"2024-04-25T10:01:52.299627+0000","last_active":"2024-04-25T10:01:54.348276+0000","last_peered":"2024-04-25T10:01:54.348276+0000","last_clean":"2024-04-25T10:01:54.348276+0000","last_became_active":"2024-04-25T10:01:52.299139+0000","last_became_peered":"2024-04-25T10:01:52.299139+0000","last_unstale":"2024-04-25T10:01:54.348276+0000","last_undegraded":"2024-04-25T10:01:54.348276+0000","last_fullsized":"2024-04-25T10:01:54.348276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.686423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310355+0000","last_change":"2024-04-25T10:01:53.310586+0000","last_active":"2024-04-25T10:01:54.310355+0000","last_peered":"2024-04-25T10:01:54.310355+0000","last_clean":"2024-04-25T10:01:54.310355+0000","last_became_active":"2024-04-25T10:01:53.309807+0000","last_became_peered":"2024-04-25T10:01:53.309807+0000","last_unstale":"2024-04-25T10:01:54.310355+0000","last_undegraded":"2024-04-25T10:01:54.310355+0000","last_fullsized":"2024-04-25T10:01:54.310355+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:42.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348318+0000","last_change":"2024-04-25T10:01:52.300227+0000","last_active":"2024-04-25T10:01:54.348318+0000","last_peered":"2024-04-25T10:01:54.348318+0000","last_clean":"2024-04-25T10:01:54.348318+0000","last_became_active":"2024-04-25T10:01:52.299809+0000","last_became_peered":"2024-04-25T10:01:52.299809+0000","last_unstale":"2024-04-25T10:01:54.348318+0000","last_undegraded":"2024-04-25T10:01:54.348318+0000","last_fullsized":"2024-04-25T10:01:54.348318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:22.491159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348484+0000","last_change":"2024-04-25T10:01:53.313976+0000","last_active":"2024-04-25T10:01:54.348484+0000","last_peered":"2024-04-25T10:01:54.348484+0000","last_clean":"2024-04-25T10:01:54.348484+0000","last_became_active":"2024-04-25T10:01:53.313672+0000","last_became_peered":"2024-04-25T10:01:53.313672+0000","last_unstale":"2024-04-25T10:01:54.348484+0000","last_undegraded":"2024-04-25T10:01:54.348484+0000","last_fullsized":"2024-04-25T10:01:54.348484+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.512462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348104+0000","last_change":"2024-04-25T10:01:52.304486+0000","last_active":"2024-04-25T10:01:54.348104+0000","last_peered":"2024-04-25T10:01:54.348104+0000","last_clean":"2024-04-25T10:01:54.348104+0000","last_became_active":"2024-04-25T10:01:52.304196+0000","last_became_peered":"2024-04-25T10:01:52.304196+0000","last_unstale":"2024-04-25T10:01:54.348104+0000","last_undegraded":"2024-04-25T10:01:54.348104+0000","last_fullsized":"2024-04-25T10:01:54.348104+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:39.943512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347804+0000","last_change":"2024-04-25T10:01:53.315595+0000","last_active":"2024-04-25T10:01:54.347804+0000","last_peered":"2024-04-25T10:01:54.347804+0000","last_clean":"2024-04-25T10:01:54.347804+0000","last_became_active":"2024-04-25T10:01:53.315254+0000","last_became_peered":"2024-04-25T10:01:53.315254+0000","last_unstale":"2024-04-25T10:01:54.347804+0000","last_undegraded":"2024-04-25T10:01:54.347804+0000","last_fullsized":"2024-04-25T10:01:54.347804+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:28.489392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311619+0000","last_change":"2024-04-25T10:01:52.300611+0000","last_active":"2024-04-25T10:01:54.311619+0000","last_peered":"2024-04-25T10:01:54.311619+0000","last_clean":"2024-04-25T10:01:54.311619+0000","last_became_active":"2024-04-25T10:01:52.300310+0000","last_became_peered":"2024-04-25T10:01:52.300310+0000","last_unstale":"2024-04-25T10:01:54.311619+0000","last_undegraded":"2024-04-25T10:01:54.311619+0000","last_fullsized":"2024-04-25T10:01:54.311619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311437+0000","last_change":"2024-04-25T10:01:53.308986+0000","last_active":"2024-04-25T10:01:54.311437+0000","last_peered":"2024-04-25T10:01:54.311437+0000","last_clean":"2024-04-25T10:01:54.311437+0000","last_became_active":"2024-04-25T10:01:53.308652+0000","last_became_peered":"2024-04-25T10:01:53.308652+0000","last_unstale":"2024-04-25T10:01:54.311437+0000","last_undegraded":"2024-04-25T10:01:54.311437+0000","last_fullsized":"2024-04-25T10:01:54.311437+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:20.339224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320437+0000","last_change":"2024-04-25T10:01:52.301727+0000","last_active":"2024-04-25T10:01:54.320437+0000","last_peered":"2024-04-25T10:01:54.320437+0000","last_clean":"2024-04-25T10:01:54.320437+0000","last_became_active":"2024-04-25T10:01:52.301473+0000","last_became_peered":"2024-04-25T10:01:52.301473+0000","last_unstale":"2024-04-25T10:01:54.320437+0000","last_undegraded":"2024-04-25T10:01:54.320437+0000","last_fullsized":"2024-04-25T10:01:54.320437+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:29.764946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348929+0000","last_change":"2024-04-25T10:01:53.321778+0000","last_active":"2024-04-25T10:01:54.348929+0000","last_peered":"2024-04-25T10:01:54.348929+0000","last_clean":"2024-04-25T10:01:54.348929+0000","last_became_active":"2024-04-25T10:01:53.318367+0000","last_became_peered":"2024-04-25T10:01:53.318367+0000","last_unstale":"2024-04-25T10:01:54.348929+0000","last_undegraded":"2024-04-25T10:01:54.348929+0000","last_fullsized":"2024-04-25T10:01:54.348929+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:43.815198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310339+0000","last_change":"2024-04-25T10:01:52.292390+0000","last_active":"2024-04-25T10:01:54.310339+0000","last_peered":"2024-04-25T10:01:54.310339+0000","last_clean":"2024-04-25T10:01:54.310339+0000","last_became_active":"2024-04-25T10:01:52.292077+0000","last_became_peered":"2024-04-25T10:01:52.292077+0000","last_unstale":"2024-04-25T10:01:54.310339+0000","last_undegraded":"2024-04-25T10:01:54.310339+0000","last_fullsized":"2024-04-25T10:01:54.310339+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:39.960271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348105+0000","last_change":"2024-04-25T10:01:53.314768+0000","last_active":"2024-04-25T10:01:54.348105+0000","last_peered":"2024-04-25T10:01:54.348105+0000","last_clean":"2024-04-25T10:01:54.348105+0000","last_became_active":"2024-04-25T10:01:53.314481+0000","last_became_peered":"2024-04-25T10:01:53.314481+0000","last_unstale":"2024-04-25T10:01:54.348105+0000","last_undegraded":"2024-04-25T10:01:54.348105+0000","last_fullsized":"2024-04-25T10:01:54.348105+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:23.497424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320449+0000","last_change":"2024-04-25T10:01:52.303125+0000","last_active":"2024-04-25T10:01:54.320449+0000","last_peered":"2024-04-25T10:01:54.320449+0000","last_clean":"2024-04-25T10:01:54.320449+0000","last_became_active":"2024-04-25T10:01:52.302491+0000","last_became_peered":"2024-04-25T10:01:52.302491+0000","last_unstale":"2024-04-25T10:01:54.320449+0000","last_undegraded":"2024-04-25T10:01:54.320449+0000","last_fullsized":"2024-04-25T10:01:54.320449+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:18.878013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318179+0000","last_change":"2024-04-25T10:01:53.320517+0000","last_active":"2024-04-25T10:01:54.318179+0000","last_peered":"2024-04-25T10:01:54.318179+0000","last_clean":"2024-04-25T10:01:54.318179+0000","last_became_active":"2024-04-25T10:01:53.319589+0000","last_became_peered":"2024-04-25T10:01:53.319589+0000","last_unstale":"2024-04-25T10:01:54.318179+0000","last_undegraded":"2024-04-25T10:01:54.318179+0000","last_fullsized":"2024-04-25T10:01:54.318179+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.215530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318256+0000","last_change":"2024-04-25T10:01:52.301928+0000","last_active":"2024-04-25T10:01:54.318256+0000","last_peered":"2024-04-25T10:01:54.318256+0000","last_clean":"2024-04-25T10:01:54.318256+0000","last_became_active":"2024-04-25T10:01:52.301620+0000","last_became_peered":"2024-04-25T10:01:52.301620+0000","last_unstale":"2024-04-25T10:01:54.318256+0000","last_undegraded":"2024-04-25T10:01:54.318256+0000","last_fullsized":"2024-04-25T10:01:54.318256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310596+0000","last_change":"2024-04-25T10:01:53.310558+0000","last_active":"2024-04-25T10:01:54.310596+0000","last_peered":"2024-04-25T10:01:54.310596+0000","last_clean":"2024-04-25T10:01:54.310596+0000","last_became_active":"2024-04-25T10:01:53.310042+0000","last_became_peered":"2024-04-25T10:01:53.310042+0000","last_unstale":"2024-04-25T10:01:54.310596+0000","last_undegraded":"2024-04-25T10:01:54.310596+0000","last_fullsized":"2024-04-25T10:01:54.310596+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:04.286812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310155+0000","last_change":"2024-04-25T10:01:52.293082+0000","last_active":"2024-04-25T10:01:54.310155+0000","last_peered":"2024-04-25T10:01:54.310155+0000","last_clean":"2024-04-25T10:01:54.310155+0000","last_became_active":"2024-04-25T10:01:52.292730+0000","last_became_peered":"2024-04-25T10:01:52.292730+0000","last_unstale":"2024-04-25T10:01:54.310155+0000","last_undegraded":"2024-04-25T10:01:54.310155+0000","last_fullsized":"2024-04-25T10:01:54.310155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:22.491159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348694+0000","last_change":"2024-04-25T10:01:53.319606+0000","last_active":"2024-04-25T10:01:54.348694+0000","last_peered":"2024-04-25T10:01:54.348694+0000","last_clean":"2024-04-25T10:01:54.348694+0000","last_became_active":"2024-04-25T10:01:53.319303+0000","last_became_peered":"2024-04-25T10:01:53.319303+0000","last_unstale":"2024-04-25T10:01:54.348694+0000","last_undegraded":"2024-04-25T10:01:54.348694+0000","last_fullsized":"2024-04-25T10:01:54.348694+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:31.529520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310559+0000","last_change":"2024-04-25T10:01:52.296284+0000","last_active":"2024-04-25T10:01:54.310559+0000","last_peered":"2024-04-25T10:01:54.310559+0000","last_clean":"2024-04-25T10:01:54.310559+0000","last_became_active":"2024-04-25T10:01:52.295533+0000","last_became_peered":"2024-04-25T10:01:52.295533+0000","last_unstale":"2024-04-25T10:01:54.310559+0000","last_undegraded":"2024-04-25T10:01:54.310559+0000","last_fullsized":"2024-04-25T10:01:54.310559+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.557737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310419+0000","last_change":"2024-04-25T10:01:53.306729+0000","last_active":"2024-04-25T10:01:54.310419+0000","last_peered":"2024-04-25T10:01:54.310419+0000","last_clean":"2024-04-25T10:01:54.310419+0000","last_became_active":"2024-04-25T10:01:53.305882+0000","last_became_peered":"2024-04-25T10:01:53.305882+0000","last_unstale":"2024-04-25T10:01:54.310419+0000","last_undegraded":"2024-04-25T10:01:54.310419+0000","last_fullsized":"2024-04-25T10:01:54.310419+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.512462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349089+0000","last_change":"2024-04-25T10:01:52.303545+0000","last_active":"2024-04-25T10:01:54.349089+0000","last_peered":"2024-04-25T10:01:54.349089+0000","last_clean":"2024-04-25T10:01:54.349089+0000","last_became_active":"2024-04-25T10:01:52.303100+0000","last_became_peered":"2024-04-25T10:01:52.303100+0000","last_unstale":"2024-04-25T10:01:54.349089+0000","last_undegraded":"2024-04-25T10:01:54.349089+0000","last_fullsized":"2024-04-25T10:01:54.349089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:54.567517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318394+0000","last_change":"2024-04-25T10:01:53.327840+0000","last_active":"2024-04-25T10:01:54.318394+0000","last_peered":"2024-04-25T10:01:54.318394+0000","last_clean":"2024-04-25T10:01:54.318394+0000","last_became_active":"2024-04-25T10:01:53.327636+0000","last_became_peered":"2024-04-25T10:01:53.327636+0000","last_unstale":"2024-04-25T10:01:54.318394+0000","last_undegraded":"2024-04-25T10:01:54.318394+0000","last_fullsized":"2024-04-25T10:01:54.318394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348429+0000","last_change":"2024-04-25T10:01:52.301182+0000","last_active":"2024-04-25T10:01:54.348429+0000","last_peered":"2024-04-25T10:01:54.348429+0000","last_clean":"2024-04-25T10:01:54.348429+0000","last_became_active":"2024-04-25T10:01:52.300836+0000","last_became_peered":"2024-04-25T10:01:52.300836+0000","last_unstale":"2024-04-25T10:01:54.348429+0000","last_undegraded":"2024-04-25T10:01:54.348429+0000","last_fullsized":"2024-04-25T10:01:54.348429+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:05.209265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347876+0000","last_change":"2024-04-25T10:01:53.308947+0000","last_active":"2024-04-25T10:01:54.347876+0000","last_peered":"2024-04-25T10:01:54.347876+0000","last_clean":"2024-04-25T10:01:54.347876+0000","last_became_active":"2024-04-25T10:01:53.308682+0000","last_became_peered":"2024-04-25T10:01:53.308682+0000","last_unstale":"2024-04-25T10:01:54.347876+0000","last_undegraded":"2024-04-25T10:01:54.347876+0000","last_fullsized":"2024-04-25T10:01:54.347876+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:09.641026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349073+0000","last_change":"2024-04-25T10:01:52.299596+0000","last_active":"2024-04-25T10:01:54.349073+0000","last_peered":"2024-04-25T10:01:54.349073+0000","last_clean":"2024-04-25T10:01:54.349073+0000","last_became_active":"2024-04-25T10:01:52.299303+0000","last_became_peered":"2024-04-25T10:01:52.299303+0000","last_unstale":"2024-04-25T10:01:54.349073+0000","last_undegraded":"2024-04-25T10:01:54.349073+0000","last_fullsized":"2024-04-25T10:01:54.349073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.253783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347539+0000","last_change":"2024-04-25T10:01:53.311926+0000","last_active":"2024-04-25T10:01:54.347539+0000","last_peered":"2024-04-25T10:01:54.347539+0000","last_clean":"2024-04-25T10:01:54.347539+0000","last_became_active":"2024-04-25T10:01:53.309452+0000","last_became_peered":"2024-04-25T10:01:53.309452+0000","last_unstale":"2024-04-25T10:01:54.347539+0000","last_undegraded":"2024-04-25T10:01:54.347539+0000","last_fullsized":"2024-04-25T10:01:54.347539+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.619357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348648+0000","last_change":"2024-04-25T10:01:52.301037+0000","last_active":"2024-04-25T10:01:54.348648+0000","last_peered":"2024-04-25T10:01:54.348648+0000","last_clean":"2024-04-25T10:01:54.348648+0000","last_became_active":"2024-04-25T10:01:52.300739+0000","last_became_peered":"2024-04-25T10:01:52.300739+0000","last_unstale":"2024-04-25T10:01:54.348648+0000","last_undegraded":"2024-04-25T10:01:54.348648+0000","last_fullsized":"2024-04-25T10:01:54.348648+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:43.232077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310139+0000","last_change":"2024-04-25T10:01:53.310352+0000","last_active":"2024-04-25T10:01:54.310139+0000","last_peered":"2024-04-25T10:01:54.310139+0000","last_clean":"2024-04-25T10:01:54.310139+0000","last_became_active":"2024-04-25T10:01:53.310055+0000","last_became_peered":"2024-04-25T10:01:53.310055+0000","last_unstale":"2024-04-25T10:01:54.310139+0000","last_undegraded":"2024-04-25T10:01:54.310139+0000","last_fullsized":"2024-04-25T10:01:54.310139+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:26.199709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348498+0000","last_change":"2024-04-25T10:01:52.301288+0000","last_active":"2024-04-25T10:01:54.348498+0000","last_peered":"2024-04-25T10:01:54.348498+0000","last_clean":"2024-04-25T10:01:54.348498+0000","last_became_active":"2024-04-25T10:01:52.301022+0000","last_became_peered":"2024-04-25T10:01:52.301022+0000","last_unstale":"2024-04-25T10:01:54.348498+0000","last_undegraded":"2024-04-25T10:01:54.348498+0000","last_fullsized":"2024-04-25T10:01:54.348498+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:07.389506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.312173+0000","last_change":"2024-04-25T10:01:53.315399+0000","last_active":"2024-04-25T10:01:54.312173+0000","last_peered":"2024-04-25T10:01:54.312173+0000","last_clean":"2024-04-25T10:01:54.312173+0000","last_became_active":"2024-04-25T10:01:53.315137+0000","last_became_peered":"2024-04-25T10:01:53.315137+0000","last_unstale":"2024-04-25T10:01:54.312173+0000","last_undegraded":"2024-04-25T10:01:54.312173+0000","last_fullsized":"2024-04-25T10:01:54.312173+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.634999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347825+0000","last_change":"2024-04-25T10:01:52.292697+0000","last_active":"2024-04-25T10:01:54.347825+0000","last_peered":"2024-04-25T10:01:54.347825+0000","last_clean":"2024-04-25T10:01:54.347825+0000","last_became_active":"2024-04-25T10:01:52.292316+0000","last_became_peered":"2024-04-25T10:01:52.292316+0000","last_unstale":"2024-04-25T10:01:54.347825+0000","last_undegraded":"2024-04-25T10:01:54.347825+0000","last_fullsized":"2024-04-25T10:01:54.347825+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:31.205159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320518+0000","last_change":"2024-04-25T10:01:53.313773+0000","last_active":"2024-04-25T10:01:54.320518+0000","last_peered":"2024-04-25T10:01:54.320518+0000","last_clean":"2024-04-25T10:01:54.320518+0000","last_became_active":"2024-04-25T10:01:53.313469+0000","last_became_peered":"2024-04-25T10:01:53.313469+0000","last_unstale":"2024-04-25T10:01:54.320518+0000","last_undegraded":"2024-04-25T10:01:54.320518+0000","last_fullsized":"2024-04-25T10:01:54.320518+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:52.218455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348408+0000","last_change":"2024-04-25T10:01:52.299811+0000","last_active":"2024-04-25T10:01:54.348408+0000","last_peered":"2024-04-25T10:01:54.348408+0000","last_clean":"2024-04-25T10:01:54.348408+0000","last_became_active":"2024-04-25T10:01:52.299468+0000","last_became_peered":"2024-04-25T10:01:52.299468+0000","last_unstale":"2024-04-25T10:01:54.348408+0000","last_undegraded":"2024-04-25T10:01:54.348408+0000","last_fullsized":"2024-04-25T10:01:54.348408+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:05.209265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310557+0000","last_change":"2024-04-25T10:01:53.309636+0000","last_active":"2024-04-25T10:01:54.310557+0000","last_peered":"2024-04-25T10:01:54.310557+0000","last_clean":"2024-04-25T10:01:54.310557+0000","last_became_active":"2024-04-25T10:01:53.309375+0000","last_became_peered":"2024-04-25T10:01:53.309375+0000","last_unstale":"2024-04-25T10:01:54.310557+0000","last_undegraded":"2024-04-25T10:01:54.310557+0000","last_fullsized":"2024-04-25T10:01:54.310557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:18.034791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348215+0000","last_change":"2024-04-25T10:01:52.300012+0000","last_active":"2024-04-25T10:01:54.348215+0000","last_peered":"2024-04-25T10:01:54.348215+0000","last_clean":"2024-04-25T10:01:54.348215+0000","last_became_active":"2024-04-25T10:01:52.299713+0000","last_became_peered":"2024-04-25T10:01:52.299713+0000","last_unstale":"2024-04-25T10:01:54.348215+0000","last_undegraded":"2024-04-25T10:01:54.348215+0000","last_fullsized":"2024-04-25T10:01:54.348215+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:39.960271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320576+0000","last_change":"2024-04-25T10:01:53.318376+0000","last_active":"2024-04-25T10:01:54.320576+0000","last_peered":"2024-04-25T10:01:54.320576+0000","last_clean":"2024-04-25T10:01:54.320576+0000","last_became_active":"2024-04-25T10:01:53.318005+0000","last_became_peered":"2024-04-25T10:01:53.318005+0000","last_unstale":"2024-04-25T10:01:54.320576+0000","last_undegraded":"2024-04-25T10:01:54.320576+0000","last_fullsized":"2024-04-25T10:01:54.320576+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:30.968859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320314+0000","last_change":"2024-04-25T10:01:52.294015+0000","last_active":"2024-04-25T10:01:54.320314+0000","last_peered":"2024-04-25T10:01:54.320314+0000","last_clean":"2024-04-25T10:01:54.320314+0000","last_became_active":"2024-04-25T10:01:52.293769+0000","last_became_peered":"2024-04-25T10:01:52.293769+0000","last_unstale":"2024-04-25T10:01:54.320314+0000","last_undegraded":"2024-04-25T10:01:54.320314+0000","last_fullsized":"2024-04-25T10:01:54.320314+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:22.062751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348551+0000","last_change":"2024-04-25T10:01:53.316595+0000","last_active":"2024-04-25T10:01:54.348551+0000","last_peered":"2024-04-25T10:01:54.348551+0000","last_clean":"2024-04-25T10:01:54.348551+0000","last_became_active":"2024-04-25T10:01:53.313992+0000","last_became_peered":"2024-04-25T10:01:53.313992+0000","last_unstale":"2024-04-25T10:01:54.348551+0000","last_undegraded":"2024-04-25T10:01:54.348551+0000","last_fullsized":"2024-04-25T10:01:54.348551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:27.623538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347985+0000","last_change":"2024-04-25T10:01:52.299879+0000","last_active":"2024-04-25T10:01:54.347985+0000","last_peered":"2024-04-25T10:01:54.347985+0000","last_clean":"2024-04-25T10:01:54.347985+0000","last_became_active":"2024-04-25T10:01:52.299301+0000","last_became_peered":"2024-04-25T10:01:52.299301+0000","last_unstale":"2024-04-25T10:01:54.347985+0000","last_undegraded":"2024-04-25T10:01:54.347985+0000","last_fullsized":"2024-04-25T10:01:54.347985+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.557737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.349026+0000","last_change":"2024-04-25T10:01:53.322694+0000","last_active":"2024-04-25T10:01:54.349026+0000","last_peered":"2024-04-25T10:01:54.349026+0000","last_clean":"2024-04-25T10:01:54.349026+0000","last_became_active":"2024-04-25T10:01:53.322132+0000","last_became_peered":"2024-04-25T10:01:53.322132+0000","last_unstale":"2024-04-25T10:01:54.349026+0000","last_undegraded":"2024-04-25T10:01:54.349026+0000","last_fullsized":"2024-04-25T10:01:54.349026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:52.522758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348684+0000","last_change":"2024-04-25T10:01:52.302593+0000","last_active":"2024-04-25T10:01:54.348684+0000","last_peered":"2024-04-25T10:01:54.348684+0000","last_clean":"2024-04-25T10:01:54.348684+0000","last_became_active":"2024-04-25T10:01:52.300953+0000","last_became_peered":"2024-04-25T10:01:52.300953+0000","last_unstale":"2024-04-25T10:01:54.348684+0000","last_undegraded":"2024-04-25T10:01:54.348684+0000","last_fullsized":"2024-04-25T10:01:54.348684+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.310301+0000","last_change":"2024-04-25T10:01:53.305776+0000","last_active":"2024-04-25T10:01:54.310301+0000","last_peered":"2024-04-25T10:01:54.310301+0000","last_clean":"2024-04-25T10:01:54.310301+0000","last_became_active":"2024-04-25T10:01:53.304775+0000","last_became_peered":"2024-04-25T10:01:53.304775+0000","last_unstale":"2024-04-25T10:01:54.310301+0000","last_undegraded":"2024-04-25T10:01:54.310301+0000","last_fullsized":"2024-04-25T10:01:54.310301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:57.394066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347849+0000","last_change":"2024-04-25T10:01:52.300698+0000","last_active":"2024-04-25T10:01:54.347849+0000","last_peered":"2024-04-25T10:01:54.347849+0000","last_clean":"2024-04-25T10:01:54.347849+0000","last_became_active":"2024-04-25T10:01:52.299332+0000","last_became_peered":"2024-04-25T10:01:52.299332+0000","last_unstale":"2024-04-25T10:01:54.347849+0000","last_undegraded":"2024-04-25T10:01:54.347849+0000","last_fullsized":"2024-04-25T10:01:54.347849+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:54.567517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320740+0000","last_change":"2024-04-25T10:01:53.307063+0000","last_active":"2024-04-25T10:01:54.320740+0000","last_peered":"2024-04-25T10:01:54.320740+0000","last_clean":"2024-04-25T10:01:54.320740+0000","last_became_active":"2024-04-25T10:01:53.306775+0000","last_became_peered":"2024-04-25T10:01:53.306775+0000","last_unstale":"2024-04-25T10:01:54.320740+0000","last_undegraded":"2024-04-25T10:01:54.320740+0000","last_fullsized":"2024-04-25T10:01:54.320740+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:36.442879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320616+0000","last_change":"2024-04-25T10:01:52.300081+0000","last_active":"2024-04-25T10:01:54.320616+0000","last_peered":"2024-04-25T10:01:54.320616+0000","last_clean":"2024-04-25T10:01:54.320616+0000","last_became_active":"2024-04-25T10:01:52.299737+0000","last_became_peered":"2024-04-25T10:01:52.299737+0000","last_unstale":"2024-04-25T10:01:54.320616+0000","last_undegraded":"2024-04-25T10:01:54.320616+0000","last_fullsized":"2024-04-25T10:01:54.320616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:47.176599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347945+0000","last_change":"2024-04-25T10:01:53.314636+0000","last_active":"2024-04-25T10:01:54.347945+0000","last_peered":"2024-04-25T10:01:54.347945+0000","last_clean":"2024-04-25T10:01:54.347945+0000","last_became_active":"2024-04-25T10:01:53.314349+0000","last_became_peered":"2024-04-25T10:01:53.314349+0000","last_unstale":"2024-04-25T10:01:54.347945+0000","last_undegraded":"2024-04-25T10:01:54.347945+0000","last_fullsized":"2024-04-25T10:01:54.347945+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:04.564002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.319936+0000","last_change":"2024-04-25T10:01:52.292204+0000","last_active":"2024-04-25T10:01:54.319936+0000","last_peered":"2024-04-25T10:01:54.319936+0000","last_clean":"2024-04-25T10:01:54.319936+0000","last_became_active":"2024-04-25T10:01:52.291829+0000","last_became_peered":"2024-04-25T10:01:52.291829+0000","last_unstale":"2024-04-25T10:01:54.319936+0000","last_undegraded":"2024-04-25T10:01:54.319936+0000","last_fullsized":"2024-04-25T10:01:54.319936+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:30.816226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348453+0000","last_change":"2024-04-25T10:01:53.306759+0000","last_active":"2024-04-25T10:01:54.348453+0000","last_peered":"2024-04-25T10:01:54.348453+0000","last_clean":"2024-04-25T10:01:54.348453+0000","last_became_active":"2024-04-25T10:01:53.306408+0000","last_became_peered":"2024-04-25T10:01:53.306408+0000","last_unstale":"2024-04-25T10:01:54.348453+0000","last_undegraded":"2024-04-25T10:01:54.348453+0000","last_fullsized":"2024-04-25T10:01:54.348453+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:42.317137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.318511+0000","last_change":"2024-04-25T10:01:52.293197+0000","last_active":"2024-04-25T10:01:54.318511+0000","last_peered":"2024-04-25T10:01:54.318511+0000","last_clean":"2024-04-25T10:01:54.318511+0000","last_became_active":"2024-04-25T10:01:52.292609+0000","last_became_peered":"2024-04-25T10:01:52.292609+0000","last_unstale":"2024-04-25T10:01:54.318511+0000","last_undegraded":"2024-04-25T10:01:54.318511+0000","last_fullsized":"2024-04-25T10:01:54.318511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:55.502905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348125+0000","last_change":"2024-04-25T10:01:53.312259+0000","last_active":"2024-04-25T10:01:54.348125+0000","last_peered":"2024-04-25T10:01:54.348125+0000","last_clean":"2024-04-25T10:01:54.348125+0000","last_became_active":"2024-04-25T10:01:53.311938+0000","last_became_peered":"2024-04-25T10:01:53.311938+0000","last_unstale":"2024-04-25T10:01:54.348125+0000","last_undegraded":"2024-04-25T10:01:54.348125+0000","last_fullsized":"2024-04-25T10:01:54.348125+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:11.878572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320728+0000","last_change":"2024-04-25T10:01:52.299885+0000","last_active":"2024-04-25T10:01:54.320728+0000","last_peered":"2024-04-25T10:01:54.320728+0000","last_clean":"2024-04-25T10:01:54.320728+0000","last_became_active":"2024-04-25T10:01:52.299576+0000","last_became_peered":"2024-04-25T10:01:52.299576+0000","last_unstale":"2024-04-25T10:01:54.320728+0000","last_undegraded":"2024-04-25T10:01:54.320728+0000","last_fullsized":"2024-04-25T10:01:54.320728+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:09.750891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320361+0000","last_change":"2024-04-25T10:01:53.313711+0000","last_active":"2024-04-25T10:01:54.320361+0000","last_peered":"2024-04-25T10:01:54.320361+0000","last_clean":"2024-04-25T10:01:54.320361+0000","last_became_active":"2024-04-25T10:01:53.313416+0000","last_became_peered":"2024-04-25T10:01:53.313416+0000","last_unstale":"2024-04-25T10:01:54.320361+0000","last_undegraded":"2024-04-25T10:01:54.320361+0000","last_fullsized":"2024-04-25T10:01:54.320361+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:39.432005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311723+0000","last_change":"2024-04-25T10:01:52.289539+0000","last_active":"2024-04-25T10:01:54.311723+0000","last_peered":"2024-04-25T10:01:54.311723+0000","last_clean":"2024-04-25T10:01:54.311723+0000","last_became_active":"2024-04-25T10:01:52.289261+0000","last_became_peered":"2024-04-25T10:01:52.289261+0000","last_unstale":"2024-04-25T10:01:54.311723+0000","last_undegraded":"2024-04-25T10:01:54.311723+0000","last_fullsized":"2024-04-25T10:01:54.311723+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:14.608806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348597+0000","last_change":"2024-04-25T10:01:53.322700+0000","last_active":"2024-04-25T10:01:54.348597+0000","last_peered":"2024-04-25T10:01:54.348597+0000","last_clean":"2024-04-25T10:01:54.348597+0000","last_became_active":"2024-04-25T10:01:53.320500+0000","last_became_peered":"2024-04-25T10:01:53.320500+0000","last_unstale":"2024-04-25T10:01:54.348597+0000","last_undegraded":"2024-04-25T10:01:54.348597+0000","last_fullsized":"2024-04-25T10:01:54.348597+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.362351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.311806+0000","last_change":"2024-04-25T10:01:52.292733+0000","last_active":"2024-04-25T10:01:54.311806+0000","last_peered":"2024-04-25T10:01:54.311806+0000","last_clean":"2024-04-25T10:01:54.311806+0000","last_became_active":"2024-04-25T10:01:52.292040+0000","last_became_peered":"2024-04-25T10:01:52.292040+0000","last_unstale":"2024-04-25T10:01:54.311806+0000","last_undegraded":"2024-04-25T10:01:54.311806+0000","last_fullsized":"2024-04-25T10:01:54.311806+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:25.222701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348029+0000","last_change":"2024-04-25T10:01:53.311823+0000","last_active":"2024-04-25T10:01:54.348029+0000","last_peered":"2024-04-25T10:01:54.348029+0000","last_clean":"2024-04-25T10:01:54.348029+0000","last_became_active":"2024-04-25T10:01:53.311509+0000","last_became_peered":"2024-04-25T10:01:53.311509+0000","last_unstale":"2024-04-25T10:01:54.348029+0000","last_undegraded":"2024-04-25T10:01:54.348029+0000","last_fullsized":"2024-04-25T10:01:54.348029+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:07.340419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.320280+0000","last_change":"2024-04-25T10:01:52.299938+0000","last_active":"2024-04-25T10:01:54.320280+0000","last_peered":"2024-04-25T10:01:54.320280+0000","last_clean":"2024-04-25T10:01:54.320280+0000","last_became_active":"2024-04-25T10:01:52.299592+0000","last_became_peered":"2024-04-25T10:01:52.299592+0000","last_unstale":"2024-04-25T10:01:54.320280+0000","last_undegraded":"2024-04-25T10:01:54.320280+0000","last_fullsized":"2024-04-25T10:01:54.320280+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:03.815918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348251+0000","last_change":"2024-04-25T10:01:53.308285+0000","last_active":"2024-04-25T10:01:54.348251+0000","last_peered":"2024-04-25T10:01:54.348251+0000","last_clean":"2024-04-25T10:01:54.348251+0000","last_became_active":"2024-04-25T10:01:53.308018+0000","last_became_peered":"2024-04-25T10:01:53.308018+0000","last_unstale":"2024-04-25T10:01:54.348251+0000","last_undegraded":"2024-04-25T10:01:54.348251+0000","last_fullsized":"2024-04-25T10:01:54.348251+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:55.573783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347851+0000","last_change":"2024-04-25T10:01:52.301345+0000","last_active":"2024-04-25T10:01:54.347851+0000","last_peered":"2024-04-25T10:01:54.347851+0000","last_clean":"2024-04-25T10:01:54.347851+0000","last_became_active":"2024-04-25T10:01:52.301002+0000","last_became_peered":"2024-04-25T10:01:52.301002+0000","last_unstale":"2024-04-25T10:01:54.347851+0000","last_undegraded":"2024-04-25T10:01:54.347851+0000","last_fullsized":"2024-04-25T10:01:54.347851+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348318+0000","last_change":"2024-04-25T10:01:53.314890+0000","last_active":"2024-04-25T10:01:54.348318+0000","last_peered":"2024-04-25T10:01:54.348318+0000","last_clean":"2024-04-25T10:01:54.348318+0000","last_became_active":"2024-04-25T10:01:53.314629+0000","last_became_peered":"2024-04-25T10:01:53.314629+0000","last_unstale":"2024-04-25T10:01:54.348318+0000","last_undegraded":"2024-04-25T10:01:54.348318+0000","last_fullsized":"2024-04-25T10:01:54.348318+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.278344+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.278344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:56.754763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.348508+0000","last_change":"2024-04-25T10:01:52.300200+0000","last_active":"2024-04-25T10:01:54.348508+0000","last_peered":"2024-04-25T10:01:54.348508+0000","last_clean":"2024-04-25T10:01:54.348508+0000","last_became_active":"2024-04-25T10:01:52.299829+0000","last_became_peered":"2024-04-25T10:01:52.299829+0000","last_unstale":"2024-04-25T10:01:54.348508+0000","last_undegraded":"2024-04-25T10:01:54.348508+0000","last_fullsized":"2024-04-25T10:01:54.348508+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:51.272079+0000","last_clean_scrub_stamp":"2024-04-25T10:01:51.272079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:19.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":355,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:01:54.347939+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:01:54.347939+0000","last_peered":"2024-04-25T10:01:54.347939+0000","last_clean":"2024-04-25T10:01:54.347939+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:01:54.347939+0000","last_undegraded":"2024-04-25T10:01:54.347939+0000","last_fullsized":"2024-04-25T10:01:54.347939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41568,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330272,"statfs":{"total":96636764160,"available":96594198528,"internally_reserved":0,"allocated":2117632,"data_stored":1292799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46552,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325288,"statfs":{"total":96636764160,"available":96589094912,"internally_reserved":0,"allocated":1511424,"data_stored":702237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45140,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326700,"statfs":{"total":96636764160,"available":96590540800,"internally_reserved":0,"allocated":1515520,"data_stored":701815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1519616,"data_stored":703267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50716,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321124,"statfs":{"total":96636764160,"available":96584830976,"internally_reserved":0,"allocated":1523712,"data_stored":703301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1507328,"data_stored":701655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1511424,"data_stored":701677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51288,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320552,"statfs":{"total":96636764160,"available":96584245248,"internally_reserved":0,"allocated":2105344,"data_stored":1292045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:01.417 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:01.417 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.552+0000 7f1616cbc640 1 Processor -- start 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.552+0000 7f1616cbc640 1 -- start start 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f1616cbc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f16100b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f1616cbc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161015e020 con 0x7f161006bd20 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f1616cbc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161006ac80 con 0x7f161006e360 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f1616cbc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161006b890 con 0x7f16100b38d0 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f1615259640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f161006bd20 0x7f16100acea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43014/0 (socket says 172.21.15.148:43014) 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f1615259640 1 -- 172.21.15.148:0/2995444094 learned_addr learned my addr 172.21.15.148:0/2995444094 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160ffff640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f16100b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380940909 0 0) 0x7f161015e020 con 0x7f161006bd20 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f8003520 con 0x7f161006bd20 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158090196 0 0) 0x7f161006ac80 con 0x7f161006e360 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161015e020 con 0x7f161006e360 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540853534 0 0) 0x7f15f8003520 con 0x7f161006bd20 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 msgr2=0x7f16100b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f16100b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 >> v1:172.21.15.188:6789/0 conn(0x7f161006e360 legacy=0x7f161006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161006af60 con 0x7f161006bd20 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f16080019f0 con 0x7f161006bd20 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160ffff640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f16100b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 >> v1:172.21.15.148:6789/0 conn(0x7f161006bd20 legacy=0x7f16100acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f8003b70 0x7f15f8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f15f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f8004d40 0x7f15f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161006b890 con 0x7f15f8004d40 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f800ba70 con 0x7f15f8003b70 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.553+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f800bbf0 con 0x7f16100b38d0 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f8004d40 0x7f15f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f160ffff640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f15f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1614a58640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f8003b70 0x7f15f8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 -- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 msgr2=0x7f15f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16100b38d0 0x7f15f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 -- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f8003b70 msgr2=0x7f15f8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f8003b70 0x7f15f8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 -- 172.21.15.148:0/2995444094 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8003520 con 0x7f15f8004d40 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.554+0000 7f1615259640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f8004d40 0x7f15f80051c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f160800a7f0 tx=0x7f15f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18be76d49e05f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800a410 con 0x7f15f8004d40 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160802f070 con 0x7f15f8004d40 2024-04-25T10:02:01.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f160f7fe640 1 -- 172.21.15.148:0/2995444094 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160802bc10 con 0x7f15f8004d40 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1616cbc640 1 -- 172.21.15.148:0/2995444094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f8004d40 msgr2=0x7f15f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1616cbc640 1 --2- 172.21.15.148:0/2995444094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15f8004d40 0x7f15f80051c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f160800a7f0 tx=0x7f15f8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1615259640 1 -- 172.21.15.148:0/2995444094 reap_dead start 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1616cbc640 1 -- 172.21.15.148:0/2995444094 shutdown_connections 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1616cbc640 1 -- 172.21.15.148:0/2995444094 >> 172.21.15.148:0/2995444094 conn(0x7f16100cfec0 msgr2=0x7f16100d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1616cbc640 1 -- 172.21.15.148:0/2995444094 shutdown_connections 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.555+0000 7f1616cbc640 1 -- 172.21.15.148:0/2995444094 wait complete. 2024-04-25T10:02:01.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.556+0000 7f1616cbc640 1 Processor -- start 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.556+0000 7f1616cbc640 1 -- start start 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f1616cbc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16101427b0 0x7f1610154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f1616cbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 0x7f16101451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f1616cbc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1610142b90 0x7f16101456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f1616cbc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161006ee50 con 0x7f1610145ea0 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f1616cbc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161006b520 con 0x7f16101427b0 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f1616cbc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16100b2fd0 con 0x7f1610142b90 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f160ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 0x7f16101451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f160ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 0x7f16101451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42870/0 (socket says 172.21.15.148:42870) 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.557+0000 7f160ffff640 1 -- 172.21.15.148:0/3448860119 learned_addr learned my addr 172.21.15.148:0/3448860119 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f1614a58640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16101427b0 0x7f1610154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160ffff640 1 -- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1610142b90 msgr2=0x7f16101456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f1615259640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1610142b90 0x7f16101456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160ffff640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1610142b90 0x7f16101456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160ffff640 1 -- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16101427b0 msgr2=0x7f1610154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160ffff640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16101427b0 0x7f1610154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160ffff640 1 -- 172.21.15.148:0/3448860119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16101549d0 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160ffff640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 0x7f16101451b0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f16040119d0 tx=0x7f1604011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160400faf0 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160400fc90 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1610154cb0 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604004030 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.558+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1610143120 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.559+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1610073a00 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.559+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1604012040 con 0x7f1610145ea0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.560+0000 7f160d7fa640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15fc06b4d0 0x7f15fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.560+0000 7f1614a58640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15fc06b4d0 0x7f15fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.560+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f160408a7c0 con 0x7f1610145ea0 2024-04-25T10:02:01.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.560+0000 7f1614a58640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15fc06b4d0 0x7f15fc06d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1600005530 tx=0x7f1600009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.564+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1604019070 con 0x7f1610145ea0 2024-04-25T10:02:01.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.798+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f161015a860 con 0x7f1610145ea0 2024-04-25T10:02:01.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.801+0000 7f160d7fa640 1 -- 172.21.15.148:0/3448860119 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f160405de20 con 0x7f1610145ea0 2024-04-25T10:02:01.805 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:02:01.805 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:02:01.806 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:02:01.806 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:02:01.806 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:01.806 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15fc06b4d0 msgr2=0x7f15fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15fc06b4d0 0x7f15fc06d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1600005530 tx=0x7f1600009040 comp rx=0 tx=0).stop 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 msgr2=0x7f16101451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 0x7f16101451b0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f16040119d0 tx=0x7f1604011e90 comp rx=0 tx=0).stop 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 shutdown_connections 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15fc06b4d0 0x7f15fc06d990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1610142b90 0x7f16101456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1610145ea0 0x7f16101451b0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 --2- 172.21.15.148:0/3448860119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16101427b0 0x7f1610154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 >> 172.21.15.148:0/3448860119 conn(0x7f16100cfec0 msgr2=0x7f1610159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 shutdown_connections 2024-04-25T10:02:01.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.803+0000 7f1616cbc640 1 -- 172.21.15.148:0/3448860119 wait complete. 2024-04-25T10:02:01.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:01.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.912+0000 7f46c6f29640 1 Processor -- start 2024-04-25T10:02:01.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.912+0000 7f46c6f29640 1 -- start start 2024-04-25T10:02:01.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c6f29640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46c015f020 0x7f46c00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c6f29640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c00cc3c0 con 0x7f46c0099a40 2024-04-25T10:02:01.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c6f29640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c015f400 con 0x7f46c00b0b90 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c6f29640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c00cccb0 con 0x7f46c015f020 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c4cc5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46c015f020 0x7f46c00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c54c6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f46c0099a40 0x7f46c00cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43026/0 (socket says 172.21.15.148:43026) 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c54c6640 1 -- 172.21.15.148:0/1855626810 learned_addr learned my addr 172.21.15.148:0/1855626810 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.913+0000 7f46c4cc5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f46c015f020 0x7f46c00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42882/0 (socket says 172.21.15.148:42882) 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.914+0000 7f46c4cc5640 1 -- 172.21.15.148:0/1855626810 >> v1:172.21.15.188:6789/0 conn(0x7f46c00b0b90 legacy=0x7f46c00b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.914+0000 7f46c4cc5640 1 -- 172.21.15.148:0/1855626810 >> v1:172.21.15.148:6789/0 conn(0x7f46c0099a40 legacy=0x7f46c00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.914+0000 7f46c4cc5640 1 -- 172.21.15.148:0/1855626810 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00d0550 con 0x7f46c015f020 2024-04-25T10:02:01.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.914+0000 7f46bf7fe640 1 -- 172.21.15.148:0/1855626810 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487433789 0 0) 0x7f46c00cc3c0 con 0x7f46c0099a40 2024-04-25T10:02:01.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.914+0000 7f46c4cc5640 1 --2- 172.21.15.148:0/1855626810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c015f020 0x7f46c00cfe00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f46b00097a0 tx=0x7f46b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc066a7e2297d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46bf7fe640 1 -- 172.21.15.148:0/1855626810 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0037070 con 0x7f46c015f020 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46bf7fe640 1 -- 172.21.15.148:0/1855626810 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b0032440 con 0x7f46c015f020 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46bf7fe640 1 -- 172.21.15.148:0/1855626810 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0032810 con 0x7f46c015f020 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 -- 172.21.15.148:0/1855626810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c015f020 msgr2=0x7f46c00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 --2- 172.21.15.148:0/1855626810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c015f020 0x7f46c00cfe00 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f46b00097a0 tx=0x7f46b002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 -- 172.21.15.148:0/1855626810 shutdown_connections 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 --2- 172.21.15.148:0/1855626810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c015f020 0x7f46c00cfe00 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 -- 172.21.15.148:0/1855626810 >> 172.21.15.148:0/1855626810 conn(0x7f46c00b3310 msgr2=0x7f46c00b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 -- 172.21.15.148:0/1855626810 shutdown_connections 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.915+0000 7f46c6f29640 1 -- 172.21.15.148:0/1855626810 wait complete. 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.916+0000 7f46c6f29640 1 Processor -- start 2024-04-25T10:02:01.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.916+0000 7f46c6f29640 1 -- start start 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c6f29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 0x7f46c00b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c6f29640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00b0b90 0x7f46c00c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c6f29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c015f020 0x7f46c00b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c6f29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00cc940 con 0x7f46c0099a40 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c6f29640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00cc660 con 0x7f46c00b0b90 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c6f29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c00cd850 con 0x7f46c015f020 2024-04-25T10:02:01.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c4cc5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 0x7f46c00b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c4cc5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 0x7f46c00b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42894/0 (socket says 172.21.15.148:42894) 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c4cc5640 1 -- 172.21.15.148:0/2957032888 learned_addr learned my addr 172.21.15.148:0/2957032888 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46bffff640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00b0b90 0x7f46c00c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.917+0000 7f46c54c6640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c015f020 0x7f46c00b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c4cc5640 1 -- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c015f020 msgr2=0x7f46c00b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c4cc5640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c015f020 0x7f46c00b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c4cc5640 1 -- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00b0b90 msgr2=0x7f46c00c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c4cc5640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00b0b90 0x7f46c00c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c4cc5640 1 -- 172.21.15.148:0/2957032888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c00c9c00 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c54c6640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c015f020 0x7f46c00b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c4cc5640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 0x7f46c00b46e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f46b00097a0 tx=0x7f46b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0037040 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b0004250 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c00c9ee0 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0032e20 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.918+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c00ca4a0 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.919+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c0073990 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.919+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46b0030770 con 0x7f46c0099a40 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.920+0000 7f46bd7fa640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f46ac06b4d0 0x7f46ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.920+0000 7f46bffff640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f46ac06b4d0 0x7f46ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.920+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46b00ab670 con 0x7f46c0099a40 2024-04-25T10:02:01.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.920+0000 7f46bffff640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f46ac06b4d0 0x7f46ac06d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f46b4007fd0 tx=0x7f46b4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:01.924+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46b007a360 con 0x7f46c0099a40 2024-04-25T10:02:02.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.132+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46c00b77a0 con 0x7f46c0099a40 2024-04-25T10:02:02.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.132+0000 7f46bd7fa640 1 -- 172.21.15.148:0/2957032888 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f46b0079c60 con 0x7f46c0099a40 2024-04-25T10:02:02.137 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:02.137 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:02.137 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:02.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f46ac06b4d0 msgr2=0x7f46ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f46ac06b4d0 0x7f46ac06d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f46b4007fd0 tx=0x7f46b4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:02.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 msgr2=0x7f46c00b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 0x7f46c00b46e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f46b00097a0 tx=0x7f46b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 shutdown_connections 2024-04-25T10:02:02.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f46ac06b4d0 0x7f46ac06d990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46c015f020 0x7f46c00b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00b0b90 0x7f46c00c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 --2- 172.21.15.148:0/2957032888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46c0099a40 0x7f46c00b46e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 >> 172.21.15.148:0/2957032888 conn(0x7f46c00b3310 msgr2=0x7f46c015f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 shutdown_connections 2024-04-25T10:02:02.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.134+0000 7f46c6f29640 1 -- 172.21.15.148:0/2957032888 wait complete. 2024-04-25T10:02:02.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:52.637253+0000', 'modified': '2024-04-25T10:01:55.153797+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8709}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8709': {'gid': 8709, 'name': 'd', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/609722939', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 609722939}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 609722939}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:02.553 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:02.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:02:02.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.650+0000 7fd931f28640 1 Processor -- start 2024-04-25T10:02:02.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.650+0000 7fd931f28640 1 -- start start 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.650+0000 7fd931f28640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd92c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.650+0000 7fd931f28640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c06b500 con 0x7fd92c15dd80 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.650+0000 7fd931f28640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c0b3930 con 0x7fd92c0ad570 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.650+0000 7fd931f28640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c06e410 con 0x7fd92c06abe0 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd92c15dd80 0x7fd92c0b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43038/0 (socket says 172.21.15.148:43038) 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92affd640 1 -- 172.21.15.148:0/4220132305 learned_addr learned my addr 172.21.15.148:0/4220132305 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd92c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998697263 0 0) 0x7fd92c06b500 con 0x7fd92c15dd80 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd914003520 con 0x7fd92c15dd80 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795168154 0 0) 0x7fd92c0b3930 con 0x7fd92c0ad570 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c06b500 con 0x7fd92c0ad570 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755891003 0 0) 0x7fd914003520 con 0x7fd92c15dd80 2024-04-25T10:02:02.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 msgr2=0x7fd92c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd92c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 >> v1:172.21.15.188:6789/0 conn(0x7fd92c0ad570 legacy=0x7fd92c06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c0b3c10 con 0x7fd92c15dd80 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd91c001670 con 0x7fd92c15dd80 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 >> v1:172.21.15.148:6789/0 conn(0x7fd92c15dd80 legacy=0x7fd92c0b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914003b70 0x7fd914003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd914004800 0x7fd914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd9140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c06e410 con 0x7fd914003b70 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91400ba70 con 0x7fd914004800 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91400bc40 con 0x7fd92c06abe0 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914003b70 0x7fd914003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92affd640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd914004800 0x7fd914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.651+0000 7fd92bfff640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd9140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92b7fe640 1 -- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 msgr2=0x7fd9140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd9140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92b7fe640 1 -- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd914004800 msgr2=0x7fd914004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd914004800 0x7fd914004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92b7fe640 1 -- 172.21.15.148:0/4220132305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914003520 con 0x7fd914003b70 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92bfff640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd92c06abe0 0x7fd9140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914003b70 0x7fd914003f70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd920009ac0 tx=0x7fd914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1de5744d110acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920036070 con 0x7fd914003b70 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920031400 con 0x7fd914003b70 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.652+0000 7fd92a7fc640 1 -- 172.21.15.148:0/4220132305 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920009690 con 0x7fd914003b70 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd931f28640 1 -- 172.21.15.148:0/4220132305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914003b70 msgr2=0x7fd914003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd931f28640 1 --2- 172.21.15.148:0/4220132305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd914003b70 0x7fd914003f70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd920009ac0 tx=0x7fd914005830 comp rx=0 tx=0).stop 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd92bfff640 1 -- 172.21.15.148:0/4220132305 reap_dead start 2024-04-25T10:02:02.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd931f28640 1 -- 172.21.15.148:0/4220132305 shutdown_connections 2024-04-25T10:02:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd931f28640 1 -- 172.21.15.148:0/4220132305 >> 172.21.15.148:0/4220132305 conn(0x7fd92c0b5af0 msgr2=0x7fd92c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd931f28640 1 -- 172.21.15.148:0/4220132305 shutdown_connections 2024-04-25T10:02:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.653+0000 7fd931f28640 1 -- 172.21.15.148:0/4220132305 wait complete. 2024-04-25T10:02:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.654+0000 7fd931f28640 1 Processor -- start 2024-04-25T10:02:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.654+0000 7fd931f28640 1 -- start start 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd931f28640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd914003b70 0x7fd92c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd931f28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd92c156e20 0x7fd92c154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd931f28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 0x7fd92c0ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd931f28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c070280 con 0x7fd92c1f7210 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd931f28640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c06e940 con 0x7fd92c156e20 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd931f28640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c070400 con 0x7fd914003b70 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 0x7fd92c0ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 0x7fd92c0ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42916/0 (socket says 172.21.15.148:42916) 2024-04-25T10:02:02.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 -- 172.21.15.148:0/3840259792 learned_addr learned my addr 172.21.15.148:0/3840259792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92affd640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd92c156e20 0x7fd92c154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd914003b70 0x7fd92c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 -- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd914003b70 msgr2=0x7fd92c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd914003b70 0x7fd92c153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 -- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd92c156e20 msgr2=0x7fd92c154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd92c156e20 0x7fd92c154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92bfff640 1 -- 172.21.15.148:0/3840259792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c1f3be0 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.655+0000 7fd92affd640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd92c156e20 0x7fd92c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.656+0000 7fd92bfff640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 0x7fd92c0ada00 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd924001330 tx=0x7fd92400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.656+0000 7fd913fff640 1 -- 172.21.15.148:0/3840259792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400ee50 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.656+0000 7fd913fff640 1 -- 172.21.15.148:0/3840259792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924003b80 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.656+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c1f7710 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.656+0000 7fd913fff640 1 -- 172.21.15.148:0/3840259792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924003d70 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.656+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c1f7cd0 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.657+0000 7fd913fff640 1 -- 172.21.15.148:0/3840259792 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd92401a020 con 0x7fd92c1f7210 2024-04-25T10:02:02.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.658+0000 7fd913fff640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd91806b4d0 0x7fd91806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.658+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd91806b4d0 0x7fd91806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.658+0000 7fd913fff640 1 -- 172.21.15.148:0/3840259792 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd924089800 con 0x7fd92c1f7210 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.658+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd91806b4d0 0x7fd91806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd920033000 tx=0x7fd920008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.659+0000 7fd934c1f740 1 Processor -- start 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.659+0000 7fd934c1f740 1 -- start start 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e2c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e4a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e01f10 0x55a5a8e48190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd934c1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8d64a10 con 0x55a5a8e49ad0 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8d0db30 con 0x55a5a8e01f10 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8d60c50 con 0x55a5a8e2c230 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e4a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e4a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42928/0 (socket says 172.21.15.148:42928) 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 -- 172.21.15.148:0/340113150 learned_addr learned my addr 172.21.15.148:0/340113150 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e2c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92bfff640 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e01f10 0x55a5a8e48190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 -- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 msgr2=0x55a5a8e2c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e2c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 -- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e01f10 msgr2=0x55a5a8e48190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e01f10 0x55a5a8e48190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 -- 172.21.15.148:0/340113150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a8d22e10 con 0x55a5a8e49ad0 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd92affd640 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e4a020 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd92c147b70 tx=0x7fd91c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8afcf2e9a14d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd9117fa640 1 -- 172.21.15.148:0/340113150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004ce0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd9117fa640 1 -- 172.21.15.148:0/340113150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c01a070 con 0x55a5a8e49ad0 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.660+0000 7fd9117fa640 1 -- 172.21.15.148:0/340113150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c015670 con 0x55a5a8e49ad0 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 -- 172.21.15.148:0/340113150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 msgr2=0x55a5a8e4a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e4a020 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd92c147b70 tx=0x7fd91c004bb0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 -- 172.21.15.148:0/340113150 shutdown_connections 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e01f10 0x55a5a8e48190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e4a020 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 --2- 172.21.15.148:0/340113150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e2c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 -- 172.21.15.148:0/340113150 >> 172.21.15.148:0/340113150 conn(0x55a5a8dfe840 msgr2=0x55a5a8e47040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 -- 172.21.15.148:0/340113150 shutdown_connections 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.661+0000 7fd934c1f740 1 -- 172.21.15.148:0/340113150 wait complete. 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.662+0000 7fd934c1f740 1 -- start start 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.662+0000 7fd934c1f740 1 Processor -- start 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.662+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2c230 0x55a5a8e409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.662+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8dffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e00320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8e02cb0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8d210c0 con 0x55a5a8e2c230 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8dfc770 con 0x55a5a8e2ff30 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 10 client.0 _ll_register_callbacks cb 0x55a5a8d14ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8dffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8dffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42942/0 (socket says 172.21.15.148:42942) 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 -- 172.21.15.148:0/3147456150 learned_addr learned my addr 172.21.15.148:0/3147456150 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2c230 0x55a5a8e409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92bfff640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e00320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 -- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 msgr2=0x55a5a8e00320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e00320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 -- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2c230 msgr2=0x55a5a8e409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2c230 0x55a5a8e409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 -- 172.21.15.148:0/3147456150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a8a792d0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92bfff640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e00320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2c230 0x55a5a8e409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd92affd640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8dffde0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd91c017040 tx=0x7fd91c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43583a591d5db467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd90a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd90a7fc640 1 -- 172.21.15.148:0/3147456150 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c015c30 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd90a7fc640 1 -- 172.21.15.148:0/3147456150 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd91c015dd0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd91c01a070 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd90a7fc640 1 -- 172.21.15.148:0/3147456150 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd91c02f050 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 10 client.8874 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.663+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55a5a8b436b0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd90a7fc640 1 -- 172.21.15.148:0/3147456150 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c01a070 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd90a7fc640 1 -- 172.21.15.148:0/3147456150 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd91c032020 con 0x55a5a8e49ad0 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd934c1f740 10 client.8874 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd934c1f740 10 client.8874 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd934c1f740 20 client.8874 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8e3e580 0x55a5a8e3e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd934c1f740 4 client.8874 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5a8dc3920 con 0x55a5a8e3e580 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8e3e580 0x55a5a8e3e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd90a7fc640 10 client.8874 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.664+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8e3e580 0x55a5a8e3e960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcfc858f195d4f5 server_cookie=45d38217eab73129 in_seq=0 out_seq=0 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd90a7fc640 1 -- 172.21.15.148:0/3147456150 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5a8dc3920 con 0x55a5a8e3e580 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd90a7fc640 10 client.8874 handle_command_reply: tid=0 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 client.8874 shutdown 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 msgr2=0x55a5a8dffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8dffde0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd91c017040 tx=0x7fd91c001a30 comp rx=0 tx=0).stop 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 shutdown_connections 2024-04-25T10:02:02.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8e3e580 0x55a5a8e3e960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e00320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8dffde0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3147456150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2c230 0x55a5a8e409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 >> 172.21.15.148:0/3147456150 conn(0x55a5a8e01f10 msgr2=0x55a5a8e4a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd90a7fc640 0 client.8874 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 shutdown_connections 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 1 -- 172.21.15.148:0/3147456150 wait complete. 2024-04-25T10:02:02.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.665+0000 7fd934c1f740 20 client.8874 trim_cache size 0 max 16384 2024-04-25T10:02:02.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.681+0000 7fd934c1f740 1 Processor -- start 2024-04-25T10:02:02.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.681+0000 7fd934c1f740 1 -- start start 2024-04-25T10:02:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 0x55a5a8e3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e3f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd934c1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c1ffe70 con 0x55a5a8e2c230 2024-04-25T10:02:02.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c152890 con 0x55a5a8e49ad0 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c1ff870 con 0x55a5a8e2ff30 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 0x55a5a8e3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 0x55a5a8e3fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42948/0 (socket says 172.21.15.148:42948) 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 -- 172.21.15.148:0/3730209861 learned_addr learned my addr 172.21.15.148:0/3730209861 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 -- 172.21.15.148:0/3730209861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 msgr2=0x55a5a8e56cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 -- 172.21.15.148:0/3730209861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e49ad0 msgr2=0x55a5a8e3f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e3f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 -- 172.21.15.148:0/3730209861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a8e495d0 con 0x55a5a8e2c230 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.682+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 0x55a5a8e3fff0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd920009be0 tx=0x7fd92002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8060f18cbbb42c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd910ff9640 1 -- 172.21.15.148:0/3730209861 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920036040 con 0x55a5a8e2c230 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd910ff9640 1 -- 172.21.15.148:0/3730209861 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92004ed00 con 0x55a5a8e2c230 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd910ff9640 1 -- 172.21.15.148:0/3730209861 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92004e500 con 0x55a5a8e2c230 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 -- 172.21.15.148:0/3730209861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 msgr2=0x55a5a8e3fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 0x55a5a8e3fff0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd920009be0 tx=0x7fd92002f000 comp rx=0 tx=0).stop 2024-04-25T10:02:02.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 -- 172.21.15.148:0/3730209861 shutdown_connections 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e49ad0 0x55a5a8e3f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2ff30 0x55a5a8e56cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 --2- 172.21.15.148:0/3730209861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e2c230 0x55a5a8e3fff0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 -- 172.21.15.148:0/3730209861 >> 172.21.15.148:0/3730209861 conn(0x55a5a8e01f10 msgr2=0x55a5a8e28320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 -- 172.21.15.148:0/3730209861 shutdown_connections 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.683+0000 7fd934c1f740 1 -- 172.21.15.148:0/3730209861 wait complete. 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.684+0000 7fd934c1f740 1 -- start start 2024-04-25T10:02:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.684+0000 7fd934c1f740 1 Processor -- start 2024-04-25T10:02:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e4a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2ff30 0x55a5a8e4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8eb55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8e3f2e0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8e2f000 con 0x55a5a8e2ff30 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8e5b320 con 0x55a5a8e2c230 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 10 client.0 _ll_register_callbacks cb 0x55a5a8dfcc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8eb55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8eb55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42958/0 (socket says 172.21.15.148:42958) 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 -- 172.21.15.148:0/1333787464 learned_addr learned my addr 172.21.15.148:0/1333787464 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e4a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92affd640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2ff30 0x55a5a8e4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 -- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 msgr2=0x55a5a8e4a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e4a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 -- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2ff30 msgr2=0x55a5a8e4a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2ff30 0x55a5a8e4a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 -- 172.21.15.148:0/1333787464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a8e3ef00 con 0x55a5a8e49ad0 2024-04-25T10:02:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e4a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92affd640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2ff30 0x55a5a8e4a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd92bfff640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8eb55c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd924060060 tx=0x7fd9240586c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11226a8110731f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd90a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd90a7fc640 1 -- 172.21.15.148:0/1333787464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924060590 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd90a7fc640 1 -- 172.21.15.148:0/1333787464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9240544f0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd92004ed00 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.685+0000 7fd90a7fc640 1 -- 172.21.15.148:0/1333787464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd92405e0b0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 10 client.8880 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55a5a8eb6220 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd90a7fc640 1 -- 172.21.15.148:0/1333787464 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92004ed00 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd90a7fc640 1 -- 172.21.15.148:0/1333787464 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9240572d0 con 0x55a5a8e49ad0 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 10 client.8880 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 10 client.8880 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 20 client.8880 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:02.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8eb5cc0 0x55a5a8e46530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 4 client.8880 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a5a8e41f50 con 0x55a5a8eb5cc0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8eb5cc0 0x55a5a8e46530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd90a7fc640 10 client.8880 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd92b7fe640 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8eb5cc0 0x55a5a8e46530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f3e71531c18862 server_cookie=410b944a3a3c04b5 in_seq=0 out_seq=0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.686+0000 7fd90a7fc640 1 -- 172.21.15.148:0/1333787464 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a5a8e41f50 con 0x55a5a8eb5cc0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd90a7fc640 10 client.8880 handle_command_reply: tid=0 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 client.8880 shutdown 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 msgr2=0x55a5a8eb55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8eb55c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fd924060060 tx=0x7fd9240586c0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 shutdown_connections 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55a5a8eb5cc0 0x55a5a8e46530 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5a8e49ad0 0x55a5a8eb55c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd90a7fc640 0 client.8880 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a5a8e2ff30 0x55a5a8e4a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 --2- 172.21.15.148:0/1333787464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a5a8e2c230 0x55a5a8e4a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 >> 172.21.15.148:0/1333787464 conn(0x55a5a8e01f10 msgr2=0x55a5a8e3f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 shutdown_connections 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 1 -- 172.21.15.148:0/1333787464 wait complete. 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd934c1f740 20 client.8880 trim_cache size 0 max 16384 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:02:02.698 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 8709, 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 9.3822540239999999, 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 67, 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 121, 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 121, 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 14.455390080999999 2024-04-25T10:02:02.699 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd91806b4d0 msgr2=0x7fd91806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd91806b4d0 0x7fd91806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd920033000 tx=0x7fd920008030 comp rx=0 tx=0).stop 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 msgr2=0x7fd92c0ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 0x7fd92c0ada00 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd924001330 tx=0x7fd92400ec00 comp rx=0 tx=0).stop 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 shutdown_connections 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd91806b4d0 0x7fd91806d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd92c1f7210 0x7fd92c0ada00 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd92c156e20 0x7fd92c154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 --2- 172.21.15.148:0/3840259792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd914003b70 0x7fd92c153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 >> 172.21.15.148:0/3840259792 conn(0x7fd92c0b5af0 msgr2=0x7fd92c0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 shutdown_connections 2024-04-25T10:02:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.687+0000 7fd931f28640 1 -- 172.21.15.148:0/3840259792 wait complete. 2024-04-25T10:02:02.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:02.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.796+0000 7f23b7c1b640 1 Processor -- start 2024-04-25T10:02:02.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.796+0000 7f23b7c1b640 1 -- start start 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b7c1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f23b015dc90 0x7f23b01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b7c1b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b00cce50 con 0x7f23b00cc8e0 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b7c1b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b0161af0 con 0x7f23b00d1810 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b7c1b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b0161dd0 con 0x7f23b015dc90 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b59b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f23b015dc90 0x7f23b01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b59b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f23b015dc90 0x7f23b01625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42964/0 (socket says 172.21.15.148:42964) 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b51b6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f23b00cc8e0 0x7f23b015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:43048/0 (socket says 172.21.15.148:43048) 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b59b7640 1 -- 172.21.15.148:0/1423537641 learned_addr learned my addr 172.21.15.148:0/1423537641 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b59b7640 1 -- 172.21.15.148:0/1423537641 >> v1:172.21.15.188:6789/0 conn(0x7f23b00d1810 legacy=0x7f23b00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b59b7640 1 -- 172.21.15.148:0/1423537641 >> v1:172.21.15.148:6789/0 conn(0x7f23b00cc8e0 legacy=0x7f23b015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.797+0000 7f23b59b7640 1 -- 172.21.15.148:0/1423537641 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b01620b0 con 0x7f23b015dc90 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.798+0000 7f23b59b7640 1 --2- 172.21.15.148:0/1423537641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b01625d0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009870 tx=0x7f23ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0c01341618b1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.798+0000 7f23b49b5640 1 -- 172.21.15.148:0/1423537641 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac037070 con 0x7f23b015dc90 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.798+0000 7f23b49b5640 1 -- 172.21.15.148:0/1423537641 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac032440 con 0x7f23b015dc90 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.798+0000 7f23b49b5640 1 -- 172.21.15.148:0/1423537641 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac032810 con 0x7f23b015dc90 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.798+0000 7f23b7c1b640 1 -- 172.21.15.148:0/1423537641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 msgr2=0x7f23b01625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.798+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/1423537641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b01625d0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009870 tx=0x7f23ac02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.799+0000 7f23b7c1b640 1 -- 172.21.15.148:0/1423537641 shutdown_connections 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.799+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/1423537641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b01625d0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.799+0000 7f23b7c1b640 1 -- 172.21.15.148:0/1423537641 >> 172.21.15.148:0/1423537641 conn(0x7f23b006f590 msgr2=0x7f23b006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.799+0000 7f23b7c1b640 1 -- 172.21.15.148:0/1423537641 shutdown_connections 2024-04-25T10:02:02.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.799+0000 7f23b7c1b640 1 -- 172.21.15.148:0/1423537641 wait complete. 2024-04-25T10:02:02.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 Processor -- start 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 -- start start 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23b00cc8e0 0x7f23b0155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23b00d1810 0x7f23b015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b0160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b015e1f0 con 0x7f23b015dc90 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b015e070 con 0x7f23b00d1810 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.800+0000 7f23b7c1b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00d1c80 con 0x7f23b00cc8e0 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b0160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b0160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42980/0 (socket says 172.21.15.148:42980) 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 -- 172.21.15.148:0/753617749 learned_addr learned my addr 172.21.15.148:0/753617749 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b51b6640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23b00d1810 0x7f23b015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b59b7640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23b00cc8e0 0x7f23b0155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 -- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23b00cc8e0 msgr2=0x7f23b0155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23b00cc8e0 0x7f23b0155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 -- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23b00d1810 msgr2=0x7f23b015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23b00d1810 0x7f23b015fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 -- 172.21.15.148:0/753617749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b01575e0 con 0x7f23b015dc90 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b61b8640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b0160140 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c730 tx=0x7f23a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8003b70 con 0x7f23b015dc90 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a8003d30 con 0x7f23b015dc90 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b01578c0 con 0x7f23b015dc90 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8005310 con 0x7f23b015dc90 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.801+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b01580c0 con 0x7f23b015dc90 2024-04-25T10:02:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.802+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b006de50 con 0x7f23b015dc90 2024-04-25T10:02:02.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.803+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23a80054b0 con 0x7f23b015dc90 2024-04-25T10:02:02.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.803+0000 7f23a67fc640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f239406b4d0 0x7f239406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.803+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23a808a1a0 con 0x7f23b015dc90 2024-04-25T10:02:02.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.803+0000 7f23b59b7640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f239406b4d0 0x7f239406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.804+0000 7f23b59b7640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f239406b4d0 0x7f239406d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f23ac034000 tx=0x7f23ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:02.807+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23a8058f20 con 0x7f23b015dc90 2024-04-25T10:02:03.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.013+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23b00cf6b0 con 0x7f23b015dc90 2024-04-25T10:02:03.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.013+0000 7f23a67fc640 1 -- 172.21.15.148:0/753617749 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f23a8058820 con 0x7f23b015dc90 2024-04-25T10:02:03.018 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f239406b4d0 msgr2=0x7f239406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f239406b4d0 0x7f239406d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f23ac034000 tx=0x7f23ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 msgr2=0x7f23b0160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b0160140 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f23a800c730 tx=0x7f23a800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 shutdown_connections 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f239406b4d0 0x7f239406d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23b015dc90 0x7f23b0160140 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23b00d1810 0x7f23b015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 --2- 172.21.15.148:0/753617749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23b00cc8e0 0x7f23b0155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 >> 172.21.15.148:0/753617749 conn(0x7f23b006f590 msgr2=0x7f23b00cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 shutdown_connections 2024-04-25T10:02:03.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:03.015+0000 7f23b7c1b640 1 -- 172.21.15.148:0/753617749 wait complete. 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:03.031 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:03.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.032 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:02:03.047 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:03.047 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:03.047 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:03.048 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42312sec preferred_lft 42312sec 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:03.049 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:03.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.050 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:03.110 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:02:03.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.111 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:02:03.167 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:02:03.167 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:03.167 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:03.167 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:03.167 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:03.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.167 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:02:03.168 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:03.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:03.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:03.168 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:03.168 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:03.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.252 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:02:03.266 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:02:03.266 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:03.267 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:03.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:03.331 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:03.385 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:03.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:03.386 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:03.440 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:03.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:03.521 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:03.521 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:03.521 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:02:03.529 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:03.530 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:03.530 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:02:03.530 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:02:03.530 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:03.548 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:03.548 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:03.548 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:03.549 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:03.550 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:03.551 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:03.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.551 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:03.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.616 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:03.680 INFO:teuthology.orchestra.run.smithi148.stdout:client7420 2024-04-25T10:02:03.680 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:03.681 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:03.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:03.736 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:03.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.813 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839243 s, 125 MB/s 2024-04-25T10:02:03.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.822 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775747 s, 135 MB/s 2024-04-25T10:02:03.830 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.830 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.830 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759552 s, 138 MB/s 2024-04-25T10:02:03.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703392 s, 149 MB/s 2024-04-25T10:02:03.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.847 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798394 s, 131 MB/s 2024-04-25T10:02:03.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.855 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698276 s, 150 MB/s 2024-04-25T10:02:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.864 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785491 s, 133 MB/s 2024-04-25T10:02:03.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068724 s, 153 MB/s 2024-04-25T10:02:03.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.879 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064102 s, 164 MB/s 2024-04-25T10:02:03.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.888 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766994 s, 137 MB/s 2024-04-25T10:02:03.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.895 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639702 s, 164 MB/s 2024-04-25T10:02:03.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802639 s, 131 MB/s 2024-04-25T10:02:03.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721805 s, 145 MB/s 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638886 s, 164 MB/s 2024-04-25T10:02:03.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.927 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736139 s, 142 MB/s 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776599 s, 135 MB/s 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760432 s, 138 MB/s 2024-04-25T10:02:03.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.951 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063291 s, 166 MB/s 2024-04-25T10:02:03.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.960 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733062 s, 143 MB/s 2024-04-25T10:02:03.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.966 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577312 s, 182 MB/s 2024-04-25T10:02:03.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.973 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628627 s, 167 MB/s 2024-04-25T10:02:03.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.981 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713814 s, 147 MB/s 2024-04-25T10:02:03.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594518 s, 176 MB/s 2024-04-25T10:02:03.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:03.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:03.996 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690962 s, 152 MB/s 2024-04-25T10:02:04.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.003 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647471 s, 162 MB/s 2024-04-25T10:02:04.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.011 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640831 s, 164 MB/s 2024-04-25T10:02:04.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.018 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661915 s, 158 MB/s 2024-04-25T10:02:04.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.026 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668175 s, 157 MB/s 2024-04-25T10:02:04.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.033 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606936 s, 173 MB/s 2024-04-25T10:02:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.040 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587011 s, 179 MB/s 2024-04-25T10:02:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.047 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635139 s, 165 MB/s 2024-04-25T10:02:04.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.055 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698454 s, 150 MB/s 2024-04-25T10:02:04.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:04.056 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:04.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651147 s, 161 MB/s 2024-04-25T10:02:04.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744724 s, 141 MB/s 2024-04-25T10:02:04.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.137 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699322 s, 150 MB/s 2024-04-25T10:02:04.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693321 s, 151 MB/s 2024-04-25T10:02:04.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.155 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892431 s, 117 MB/s 2024-04-25T10:02:04.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.162 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625764 s, 168 MB/s 2024-04-25T10:02:04.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.170 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702779 s, 149 MB/s 2024-04-25T10:02:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.176 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601139 s, 174 MB/s 2024-04-25T10:02:04.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625128 s, 168 MB/s 2024-04-25T10:02:04.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.191 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006296 s, 167 MB/s 2024-04-25T10:02:04.197 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.197 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.197 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588517 s, 178 MB/s 2024-04-25T10:02:04.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.205 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642384 s, 163 MB/s 2024-04-25T10:02:04.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604603 s, 173 MB/s 2024-04-25T10:02:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.219 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621308 s, 169 MB/s 2024-04-25T10:02:04.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595311 s, 176 MB/s 2024-04-25T10:02:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.233 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680586 s, 154 MB/s 2024-04-25T10:02:04.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565313 s, 185 MB/s 2024-04-25T10:02:04.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.249 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840791 s, 125 MB/s 2024-04-25T10:02:04.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.257 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647504 s, 162 MB/s 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616964 s, 170 MB/s 2024-04-25T10:02:04.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594651 s, 176 MB/s 2024-04-25T10:02:04.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701222 s, 150 MB/s 2024-04-25T10:02:04.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063456 s, 165 MB/s 2024-04-25T10:02:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.293 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579633 s, 181 MB/s 2024-04-25T10:02:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613068 s, 171 MB/s 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067369 s, 156 MB/s 2024-04-25T10:02:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620984 s, 169 MB/s 2024-04-25T10:02:04.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.321 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636642 s, 165 MB/s 2024-04-25T10:02:04.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.329 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673064 s, 156 MB/s 2024-04-25T10:02:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.335 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567859 s, 185 MB/s 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.342 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622455 s, 168 MB/s 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633701 s, 165 MB/s 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:04.351 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:04.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745325 s, 141 MB/s 2024-04-25T10:02:04.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.425 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710634 s, 148 MB/s 2024-04-25T10:02:04.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656539 s, 160 MB/s 2024-04-25T10:02:04.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626407 s, 167 MB/s 2024-04-25T10:02:04.446 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.446 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.446 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560714 s, 187 MB/s 2024-04-25T10:02:04.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.453 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574954 s, 182 MB/s 2024-04-25T10:02:04.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062244 s, 168 MB/s 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627243 s, 167 MB/s 2024-04-25T10:02:04.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621416 s, 169 MB/s 2024-04-25T10:02:04.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.482 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066368 s, 158 MB/s 2024-04-25T10:02:04.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.489 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630112 s, 166 MB/s 2024-04-25T10:02:04.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.496 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639177 s, 164 MB/s 2024-04-25T10:02:04.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725096 s, 145 MB/s 2024-04-25T10:02:04.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.511 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055539 s, 189 MB/s 2024-04-25T10:02:04.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.519 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.519 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723159 s, 145 MB/s 2024-04-25T10:02:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697714 s, 150 MB/s 2024-04-25T10:02:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631862 s, 166 MB/s 2024-04-25T10:02:04.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653051 s, 161 MB/s 2024-04-25T10:02:04.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710921 s, 147 MB/s 2024-04-25T10:02:04.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629929 s, 166 MB/s 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631816 s, 166 MB/s 2024-04-25T10:02:04.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.571 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632684 s, 166 MB/s 2024-04-25T10:02:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.578 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063935 s, 164 MB/s 2024-04-25T10:02:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063846 s, 164 MB/s 2024-04-25T10:02:04.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068386 s, 153 MB/s 2024-04-25T10:02:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906999 s, 116 MB/s 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642829 s, 163 MB/s 2024-04-25T10:02:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643957 s, 163 MB/s 2024-04-25T10:02:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556576 s, 188 MB/s 2024-04-25T10:02:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063226 s, 166 MB/s 2024-04-25T10:02:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704361 s, 149 MB/s 2024-04-25T10:02:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576686 s, 182 MB/s 2024-04-25T10:02:04.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:04.647 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737448 s, 142 MB/s 2024-04-25T10:02:04.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.720 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666756 s, 157 MB/s 2024-04-25T10:02:04.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.727 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637382 s, 165 MB/s 2024-04-25T10:02:04.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00539229 s, 194 MB/s 2024-04-25T10:02:04.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.740 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589436 s, 178 MB/s 2024-04-25T10:02:04.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.748 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.748 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629778 s, 166 MB/s 2024-04-25T10:02:04.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.755 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067751 s, 155 MB/s 2024-04-25T10:02:04.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.762 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623163 s, 168 MB/s 2024-04-25T10:02:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007276 s, 144 MB/s 2024-04-25T10:02:04.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.777 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580556 s, 181 MB/s 2024-04-25T10:02:04.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.784 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061386 s, 171 MB/s 2024-04-25T10:02:04.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.794 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845457 s, 124 MB/s 2024-04-25T10:02:04.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589806 s, 178 MB/s 2024-04-25T10:02:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.809 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715611 s, 147 MB/s 2024-04-25T10:02:04.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.817 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722873 s, 145 MB/s 2024-04-25T10:02:04.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.824 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062131 s, 169 MB/s 2024-04-25T10:02:04.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704761 s, 149 MB/s 2024-04-25T10:02:04.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.839 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571379 s, 184 MB/s 2024-04-25T10:02:04.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.847 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752184 s, 139 MB/s 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620723 s, 169 MB/s 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624919 s, 168 MB/s 2024-04-25T10:02:04.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619477 s, 169 MB/s 2024-04-25T10:02:04.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564955 s, 186 MB/s 2024-04-25T10:02:04.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709872 s, 148 MB/s 2024-04-25T10:02:04.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596477 s, 176 MB/s 2024-04-25T10:02:04.896 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.896 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.896 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615557 s, 170 MB/s 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618502 s, 170 MB/s 2024-04-25T10:02:04.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716776 s, 146 MB/s 2024-04-25T10:02:04.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.919 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006283 s, 167 MB/s 2024-04-25T10:02:04.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.926 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587207 s, 179 MB/s 2024-04-25T10:02:04.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696134 s, 151 MB/s 2024-04-25T10:02:04.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:04.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:04.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630995 s, 166 MB/s 2024-04-25T10:02:04.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:04.942 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:05.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.007 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632204 s, 166 MB/s 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.013 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582247 s, 180 MB/s 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.020 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567849 s, 185 MB/s 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664246 s, 158 MB/s 2024-04-25T10:02:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060515 s, 173 MB/s 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567664 s, 185 MB/s 2024-04-25T10:02:05.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.048 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585842 s, 179 MB/s 2024-04-25T10:02:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724034 s, 145 MB/s 2024-04-25T10:02:05.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.063 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625262 s, 168 MB/s 2024-04-25T10:02:05.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.070 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616128 s, 170 MB/s 2024-04-25T10:02:05.077 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664078 s, 158 MB/s 2024-04-25T10:02:05.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621531 s, 169 MB/s 2024-04-25T10:02:05.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814197 s, 129 MB/s 2024-04-25T10:02:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616048 s, 170 MB/s 2024-04-25T10:02:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641577 s, 163 MB/s 2024-04-25T10:02:05.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604629 s, 173 MB/s 2024-04-25T10:02:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552535 s, 190 MB/s 2024-04-25T10:02:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064623 s, 162 MB/s 2024-04-25T10:02:05.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594036 s, 177 MB/s 2024-04-25T10:02:05.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623516 s, 168 MB/s 2024-04-25T10:02:05.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743993 s, 141 MB/s 2024-04-25T10:02:05.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697783 s, 150 MB/s 2024-04-25T10:02:05.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710525 s, 148 MB/s 2024-04-25T10:02:05.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627111 s, 167 MB/s 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920671 s, 114 MB/s 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.192 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074351 s, 141 MB/s 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.200 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631608 s, 166 MB/s 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.207 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630703 s, 166 MB/s 2024-04-25T10:02:05.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061781 s, 170 MB/s 2024-04-25T10:02:05.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729266 s, 144 MB/s 2024-04-25T10:02:05.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.229 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580482 s, 181 MB/s 2024-04-25T10:02:05.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577151 s, 182 MB/s 2024-04-25T10:02:05.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:05.237 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:05.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.301 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064779 s, 162 MB/s 2024-04-25T10:02:05.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.308 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632737 s, 166 MB/s 2024-04-25T10:02:05.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635168 s, 165 MB/s 2024-04-25T10:02:05.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745404 s, 141 MB/s 2024-04-25T10:02:05.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612957 s, 171 MB/s 2024-04-25T10:02:05.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077332 s, 136 MB/s 2024-04-25T10:02:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630752 s, 166 MB/s 2024-04-25T10:02:05.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.355 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782408 s, 134 MB/s 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553172 s, 190 MB/s 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642067 s, 163 MB/s 2024-04-25T10:02:05.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.376 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635722 s, 165 MB/s 2024-04-25T10:02:05.383 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.383 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.383 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648338 s, 162 MB/s 2024-04-25T10:02:05.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.391 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644281 s, 163 MB/s 2024-04-25T10:02:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.398 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642636 s, 163 MB/s 2024-04-25T10:02:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662823 s, 158 MB/s 2024-04-25T10:02:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601252 s, 174 MB/s 2024-04-25T10:02:05.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.420 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641729 s, 163 MB/s 2024-04-25T10:02:05.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722932 s, 145 MB/s 2024-04-25T10:02:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.436 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719077 s, 146 MB/s 2024-04-25T10:02:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.443 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630219 s, 166 MB/s 2024-04-25T10:02:05.451 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.451 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.451 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665355 s, 158 MB/s 2024-04-25T10:02:05.458 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.459 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676086 s, 155 MB/s 2024-04-25T10:02:05.466 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.466 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.466 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061821 s, 170 MB/s 2024-04-25T10:02:05.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726256 s, 144 MB/s 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726857 s, 144 MB/s 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707425 s, 148 MB/s 2024-04-25T10:02:05.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071154 s, 147 MB/s 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059109 s, 177 MB/s 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730033 s, 144 MB/s 2024-04-25T10:02:05.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661788 s, 158 MB/s 2024-04-25T10:02:05.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.527 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596971 s, 176 MB/s 2024-04-25T10:02:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593414 s, 177 MB/s 2024-04-25T10:02:05.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:05.535 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:05.535 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:05.535 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:02:05.536 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:05.536 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:05.536 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:05.536 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:05.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.601 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704856 s, 149 MB/s 2024-04-25T10:02:05.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713776 s, 147 MB/s 2024-04-25T10:02:05.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.616 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639922 s, 164 MB/s 2024-04-25T10:02:05.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715779 s, 146 MB/s 2024-04-25T10:02:05.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693231 s, 151 MB/s 2024-04-25T10:02:05.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.639 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617289 s, 170 MB/s 2024-04-25T10:02:05.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576364 s, 182 MB/s 2024-04-25T10:02:05.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722619 s, 145 MB/s 2024-04-25T10:02:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582312 s, 180 MB/s 2024-04-25T10:02:05.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.668 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065167 s, 161 MB/s 2024-04-25T10:02:05.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741024 s, 142 MB/s 2024-04-25T10:02:05.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.683 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578353 s, 181 MB/s 2024-04-25T10:02:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.690 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644775 s, 163 MB/s 2024-04-25T10:02:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.697 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631214 s, 166 MB/s 2024-04-25T10:02:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.705 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704557 s, 149 MB/s 2024-04-25T10:02:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.712 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585941 s, 179 MB/s 2024-04-25T10:02:05.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.719 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620984 s, 169 MB/s 2024-04-25T10:02:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.727 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723059 s, 145 MB/s 2024-04-25T10:02:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613842 s, 171 MB/s 2024-04-25T10:02:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.741 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638897 s, 164 MB/s 2024-04-25T10:02:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.748 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607919 s, 172 MB/s 2024-04-25T10:02:05.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.755 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612999 s, 171 MB/s 2024-04-25T10:02:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.762 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606218 s, 173 MB/s 2024-04-25T10:02:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722471 s, 145 MB/s 2024-04-25T10:02:05.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685007 s, 153 MB/s 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728153 s, 144 MB/s 2024-04-25T10:02:05.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054893 s, 191 MB/s 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626724 s, 167 MB/s 2024-04-25T10:02:05.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.808 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691269 s, 152 MB/s 2024-04-25T10:02:05.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.816 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711318 s, 147 MB/s 2024-04-25T10:02:05.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854195 s, 123 MB/s 2024-04-25T10:02:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739787 s, 142 MB/s 2024-04-25T10:02:05.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:05.834 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:05.834 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:05.835 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:02:05.835 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:05.835 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:05.835 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:05.835 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:05.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621107 s, 169 MB/s 2024-04-25T10:02:05.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710845 s, 148 MB/s 2024-04-25T10:02:05.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609086 s, 172 MB/s 2024-04-25T10:02:05.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688959 s, 152 MB/s 2024-04-25T10:02:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.931 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885578 s, 118 MB/s 2024-04-25T10:02:05.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.938 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627184 s, 167 MB/s 2024-04-25T10:02:05.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.946 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701874 s, 149 MB/s 2024-04-25T10:02:05.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612437 s, 171 MB/s 2024-04-25T10:02:05.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.961 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698888 s, 150 MB/s 2024-04-25T10:02:05.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628945 s, 167 MB/s 2024-04-25T10:02:05.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565301 s, 185 MB/s 2024-04-25T10:02:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605049 s, 173 MB/s 2024-04-25T10:02:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605622 s, 173 MB/s 2024-04-25T10:02:05.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:05.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:05.996 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618631 s, 169 MB/s 2024-04-25T10:02:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.003 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612255 s, 171 MB/s 2024-04-25T10:02:06.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.012 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776691 s, 135 MB/s 2024-04-25T10:02:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651933 s, 161 MB/s 2024-04-25T10:02:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610534 s, 172 MB/s 2024-04-25T10:02:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581711 s, 180 MB/s 2024-04-25T10:02:06.040 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.040 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.040 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628579 s, 167 MB/s 2024-04-25T10:02:06.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076191 s, 138 MB/s 2024-04-25T10:02:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718389 s, 146 MB/s 2024-04-25T10:02:06.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.065 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776477 s, 135 MB/s 2024-04-25T10:02:06.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.074 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732249 s, 143 MB/s 2024-04-25T10:02:06.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.082 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736073 s, 142 MB/s 2024-04-25T10:02:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072942 s, 144 MB/s 2024-04-25T10:02:06.098 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.098 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.098 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719059 s, 146 MB/s 2024-04-25T10:02:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723182 s, 145 MB/s 2024-04-25T10:02:06.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618663 s, 169 MB/s 2024-04-25T10:02:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692741 s, 151 MB/s 2024-04-25T10:02:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063019 s, 166 MB/s 2024-04-25T10:02:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.135 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626316 s, 167 MB/s 2024-04-25T10:02:06.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:06.136 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:06.137 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:06.137 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:02:06.137 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:06.137 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:06.137 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:06.137 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:06.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.201 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646798 s, 162 MB/s 2024-04-25T10:02:06.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.208 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612901 s, 171 MB/s 2024-04-25T10:02:06.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.215 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571264 s, 184 MB/s 2024-04-25T10:02:06.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663439 s, 158 MB/s 2024-04-25T10:02:06.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.229 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624339 s, 168 MB/s 2024-04-25T10:02:06.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.238 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726527 s, 144 MB/s 2024-04-25T10:02:06.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617303 s, 170 MB/s 2024-04-25T10:02:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.252 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625872 s, 168 MB/s 2024-04-25T10:02:06.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.260 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722123 s, 145 MB/s 2024-04-25T10:02:06.267 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068157 s, 154 MB/s 2024-04-25T10:02:06.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.274 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588952 s, 178 MB/s 2024-04-25T10:02:06.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.282 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070951 s, 148 MB/s 2024-04-25T10:02:06.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.290 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644823 s, 163 MB/s 2024-04-25T10:02:06.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.298 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731377 s, 143 MB/s 2024-04-25T10:02:06.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.304 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576134 s, 182 MB/s 2024-04-25T10:02:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709314 s, 148 MB/s 2024-04-25T10:02:06.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.320 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670733 s, 156 MB/s 2024-04-25T10:02:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739856 s, 142 MB/s 2024-04-25T10:02:06.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007043 s, 149 MB/s 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569922 s, 184 MB/s 2024-04-25T10:02:06.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.351 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729901 s, 144 MB/s 2024-04-25T10:02:06.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556029 s, 189 MB/s 2024-04-25T10:02:06.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.365 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600981 s, 174 MB/s 2024-04-25T10:02:06.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663002 s, 158 MB/s 2024-04-25T10:02:06.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.379 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612694 s, 171 MB/s 2024-04-25T10:02:06.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708637 s, 148 MB/s 2024-04-25T10:02:06.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637716 s, 164 MB/s 2024-04-25T10:02:06.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.403 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712241 s, 147 MB/s 2024-04-25T10:02:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089956 s, 117 MB/s 2024-04-25T10:02:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.420 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634925 s, 165 MB/s 2024-04-25T10:02:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.427 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632793 s, 166 MB/s 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657736 s, 159 MB/s 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:06.435 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:06.436 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:06.436 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:02:06.436 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:06.436 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:06.436 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:06.436 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:06.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.499 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605067 s, 173 MB/s 2024-04-25T10:02:06.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.508 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748686 s, 140 MB/s 2024-04-25T10:02:06.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599702 s, 175 MB/s 2024-04-25T10:02:06.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714867 s, 147 MB/s 2024-04-25T10:02:06.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.530 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625061 s, 168 MB/s 2024-04-25T10:02:06.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.538 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071823 s, 146 MB/s 2024-04-25T10:02:06.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.546 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716072 s, 146 MB/s 2024-04-25T10:02:06.554 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.554 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.554 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758476 s, 138 MB/s 2024-04-25T10:02:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.562 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629053 s, 167 MB/s 2024-04-25T10:02:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.570 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739364 s, 142 MB/s 2024-04-25T10:02:06.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.576 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579708 s, 181 MB/s 2024-04-25T10:02:06.584 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.584 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.584 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649906 s, 161 MB/s 2024-04-25T10:02:06.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.591 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644502 s, 163 MB/s 2024-04-25T10:02:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.598 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638111 s, 164 MB/s 2024-04-25T10:02:06.605 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.605 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.605 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602634 s, 174 MB/s 2024-04-25T10:02:06.613 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.613 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.613 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722693 s, 145 MB/s 2024-04-25T10:02:06.620 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.621 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.621 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639657 s, 164 MB/s 2024-04-25T10:02:06.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.628 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.628 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060506 s, 173 MB/s 2024-04-25T10:02:06.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719785 s, 146 MB/s 2024-04-25T10:02:06.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.643 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623556 s, 168 MB/s 2024-04-25T10:02:06.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.651 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707082 s, 148 MB/s 2024-04-25T10:02:06.658 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705783 s, 149 MB/s 2024-04-25T10:02:06.665 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.665 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.665 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569057 s, 184 MB/s 2024-04-25T10:02:06.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.673 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711993 s, 147 MB/s 2024-04-25T10:02:06.680 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.681 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655773 s, 160 MB/s 2024-04-25T10:02:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.688 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653328 s, 160 MB/s 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062635 s, 167 MB/s 2024-04-25T10:02:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718645 s, 146 MB/s 2024-04-25T10:02:06.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.711 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716485 s, 146 MB/s 2024-04-25T10:02:06.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632152 s, 166 MB/s 2024-04-25T10:02:06.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.726 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701084 s, 150 MB/s 2024-04-25T10:02:06.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627619 s, 167 MB/s 2024-04-25T10:02:06.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:06.735 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:06.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675183 s, 155 MB/s 2024-04-25T10:02:06.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640837 s, 164 MB/s 2024-04-25T10:02:06.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.814 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592751 s, 177 MB/s 2024-04-25T10:02:06.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586249 s, 179 MB/s 2024-04-25T10:02:06.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.828 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581649 s, 180 MB/s 2024-04-25T10:02:06.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917706 s, 114 MB/s 2024-04-25T10:02:06.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.846 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683661 s, 153 MB/s 2024-04-25T10:02:06.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708217 s, 148 MB/s 2024-04-25T10:02:06.860 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.860 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.860 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613191 s, 171 MB/s 2024-04-25T10:02:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764232 s, 137 MB/s 2024-04-25T10:02:06.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564864 s, 186 MB/s 2024-04-25T10:02:06.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647114 s, 162 MB/s 2024-04-25T10:02:06.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630099 s, 166 MB/s 2024-04-25T10:02:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599137 s, 175 MB/s 2024-04-25T10:02:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.905 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720177 s, 146 MB/s 2024-04-25T10:02:06.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590361 s, 178 MB/s 2024-04-25T10:02:06.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.919 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625223 s, 168 MB/s 2024-04-25T10:02:06.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.926 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644628 s, 163 MB/s 2024-04-25T10:02:06.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703866 s, 149 MB/s 2024-04-25T10:02:06.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716793 s, 146 MB/s 2024-04-25T10:02:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691298 s, 152 MB/s 2024-04-25T10:02:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.958 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740807 s, 142 MB/s 2024-04-25T10:02:06.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.967 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739793 s, 142 MB/s 2024-04-25T10:02:06.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.975 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718352 s, 146 MB/s 2024-04-25T10:02:06.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063366 s, 165 MB/s 2024-04-25T10:02:06.990 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.990 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.990 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067254 s, 156 MB/s 2024-04-25T10:02:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:06.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:06.998 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708544 s, 148 MB/s 2024-04-25T10:02:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.004 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613822 s, 171 MB/s 2024-04-25T10:02:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.014 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887073 s, 118 MB/s 2024-04-25T10:02:07.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656494 s, 160 MB/s 2024-04-25T10:02:07.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.030 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728817 s, 144 MB/s 2024-04-25T10:02:07.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064055 s, 164 MB/s 2024-04-25T10:02:07.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.038 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:07.038 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:07.038 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:02:07.038 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:07.038 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:07.039 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:07.039 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.102 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579926 s, 181 MB/s 2024-04-25T10:02:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661151 s, 159 MB/s 2024-04-25T10:02:07.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.120 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950835 s, 110 MB/s 2024-04-25T10:02:07.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.126 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557587 s, 188 MB/s 2024-04-25T10:02:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879383 s, 119 MB/s 2024-04-25T10:02:07.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664818 s, 158 MB/s 2024-04-25T10:02:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072379 s, 145 MB/s 2024-04-25T10:02:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612151 s, 171 MB/s 2024-04-25T10:02:07.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898368 s, 117 MB/s 2024-04-25T10:02:07.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.175 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062384 s, 168 MB/s 2024-04-25T10:02:07.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.182 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622182 s, 169 MB/s 2024-04-25T10:02:07.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.189 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590538 s, 178 MB/s 2024-04-25T10:02:07.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611524 s, 171 MB/s 2024-04-25T10:02:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606583 s, 173 MB/s 2024-04-25T10:02:07.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.210 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634166 s, 165 MB/s 2024-04-25T10:02:07.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633065 s, 166 MB/s 2024-04-25T10:02:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.225 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632353 s, 166 MB/s 2024-04-25T10:02:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.232 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657676 s, 159 MB/s 2024-04-25T10:02:07.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.239 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611214 s, 172 MB/s 2024-04-25T10:02:07.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821632 s, 128 MB/s 2024-04-25T10:02:07.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.255 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633059 s, 166 MB/s 2024-04-25T10:02:07.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706017 s, 149 MB/s 2024-04-25T10:02:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586962 s, 179 MB/s 2024-04-25T10:02:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627017 s, 167 MB/s 2024-04-25T10:02:07.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.285 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712254 s, 147 MB/s 2024-04-25T10:02:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.293 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710019 s, 148 MB/s 2024-04-25T10:02:07.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.300 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575168 s, 182 MB/s 2024-04-25T10:02:07.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574193 s, 183 MB/s 2024-04-25T10:02:07.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643157 s, 163 MB/s 2024-04-25T10:02:07.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694852 s, 151 MB/s 2024-04-25T10:02:07.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.329 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594678 s, 176 MB/s 2024-04-25T10:02:07.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.337 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689403 s, 152 MB/s 2024-04-25T10:02:07.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.338 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:07.338 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:07.338 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:02:07.338 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:07.338 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:07.338 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:07.339 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:07.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.404 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721172 s, 145 MB/s 2024-04-25T10:02:07.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726675 s, 144 MB/s 2024-04-25T10:02:07.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608957 s, 172 MB/s 2024-04-25T10:02:07.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.426 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599538 s, 175 MB/s 2024-04-25T10:02:07.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589887 s, 178 MB/s 2024-04-25T10:02:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581738 s, 180 MB/s 2024-04-25T10:02:07.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684977 s, 153 MB/s 2024-04-25T10:02:07.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642183 s, 163 MB/s 2024-04-25T10:02:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.463 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763398 s, 137 MB/s 2024-04-25T10:02:07.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.470 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631882 s, 166 MB/s 2024-04-25T10:02:07.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.478 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646866 s, 162 MB/s 2024-04-25T10:02:07.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.484 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629178 s, 167 MB/s 2024-04-25T10:02:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.492 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647364 s, 162 MB/s 2024-04-25T10:02:07.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.499 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638247 s, 164 MB/s 2024-04-25T10:02:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.507 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714304 s, 147 MB/s 2024-04-25T10:02:07.515 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.515 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.515 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702248 s, 149 MB/s 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.522 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585553 s, 179 MB/s 2024-04-25T10:02:07.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586967 s, 179 MB/s 2024-04-25T10:02:07.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555986 s, 189 MB/s 2024-04-25T10:02:07.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.542 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622606 s, 168 MB/s 2024-04-25T10:02:07.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.549 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651599 s, 161 MB/s 2024-04-25T10:02:07.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.559 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904315 s, 116 MB/s 2024-04-25T10:02:07.566 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.566 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.566 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062146 s, 169 MB/s 2024-04-25T10:02:07.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639187 s, 164 MB/s 2024-04-25T10:02:07.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691869 s, 152 MB/s 2024-04-25T10:02:07.589 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.589 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.589 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700709 s, 150 MB/s 2024-04-25T10:02:07.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062064 s, 169 MB/s 2024-04-25T10:02:07.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.604 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667764 s, 157 MB/s 2024-04-25T10:02:07.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637525 s, 164 MB/s 2024-04-25T10:02:07.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571532 s, 183 MB/s 2024-04-25T10:02:07.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613961 s, 171 MB/s 2024-04-25T10:02:07.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630712 s, 166 MB/s 2024-04-25T10:02:07.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:07.633 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:07.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771078 s, 136 MB/s 2024-04-25T10:02:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.707 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717179 s, 146 MB/s 2024-04-25T10:02:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.714 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628248 s, 167 MB/s 2024-04-25T10:02:07.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.721 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591442 s, 177 MB/s 2024-04-25T10:02:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643226 s, 163 MB/s 2024-04-25T10:02:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678851 s, 154 MB/s 2024-04-25T10:02:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755204 s, 139 MB/s 2024-04-25T10:02:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668823 s, 157 MB/s 2024-04-25T10:02:07.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580904 s, 181 MB/s 2024-04-25T10:02:07.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631283 s, 166 MB/s 2024-04-25T10:02:07.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.773 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630603 s, 166 MB/s 2024-04-25T10:02:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623164 s, 168 MB/s 2024-04-25T10:02:07.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.787 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590933 s, 177 MB/s 2024-04-25T10:02:07.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.795 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704994 s, 149 MB/s 2024-04-25T10:02:07.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.803 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716848 s, 146 MB/s 2024-04-25T10:02:07.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685729 s, 153 MB/s 2024-04-25T10:02:07.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715128 s, 147 MB/s 2024-04-25T10:02:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.826 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659482 s, 159 MB/s 2024-04-25T10:02:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.834 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716068 s, 146 MB/s 2024-04-25T10:02:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733043 s, 143 MB/s 2024-04-25T10:02:07.849 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.849 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.849 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568935 s, 184 MB/s 2024-04-25T10:02:07.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.857 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745165 s, 141 MB/s 2024-04-25T10:02:07.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.864 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613323 s, 171 MB/s 2024-04-25T10:02:07.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.871 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.872 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064574 s, 162 MB/s 2024-04-25T10:02:07.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.879 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701567 s, 149 MB/s 2024-04-25T10:02:07.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.887 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630291 s, 166 MB/s 2024-04-25T10:02:07.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.895 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075621 s, 139 MB/s 2024-04-25T10:02:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621009 s, 169 MB/s 2024-04-25T10:02:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586994 s, 179 MB/s 2024-04-25T10:02:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634407 s, 165 MB/s 2024-04-25T10:02:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734206 s, 143 MB/s 2024-04-25T10:02:07.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.932 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648803 s, 162 MB/s 2024-04-25T10:02:07.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.933 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:07.933 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:07.933 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:02:07.933 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:07.933 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:07.933 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:07.934 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:07.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:07.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:07.998 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641928 s, 163 MB/s 2024-04-25T10:02:08.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.004 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615692 s, 170 MB/s 2024-04-25T10:02:08.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.011 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616304 s, 170 MB/s 2024-04-25T10:02:08.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.020 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744962 s, 141 MB/s 2024-04-25T10:02:08.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714448 s, 147 MB/s 2024-04-25T10:02:08.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.035 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648469 s, 162 MB/s 2024-04-25T10:02:08.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.042 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606931 s, 173 MB/s 2024-04-25T10:02:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593228 s, 177 MB/s 2024-04-25T10:02:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652382 s, 161 MB/s 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.064 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659463 s, 159 MB/s 2024-04-25T10:02:08.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698222 s, 150 MB/s 2024-04-25T10:02:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621371 s, 169 MB/s 2024-04-25T10:02:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646101 s, 162 MB/s 2024-04-25T10:02:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643266 s, 163 MB/s 2024-04-25T10:02:08.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.100 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616281 s, 170 MB/s 2024-04-25T10:02:08.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712426 s, 147 MB/s 2024-04-25T10:02:08.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619504 s, 169 MB/s 2024-04-25T10:02:08.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631772 s, 166 MB/s 2024-04-25T10:02:08.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725037 s, 145 MB/s 2024-04-25T10:02:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066739 s, 157 MB/s 2024-04-25T10:02:08.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580301 s, 181 MB/s 2024-04-25T10:02:08.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.152 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609347 s, 172 MB/s 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626586 s, 167 MB/s 2024-04-25T10:02:08.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.166 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062897 s, 167 MB/s 2024-04-25T10:02:08.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.173 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611942 s, 171 MB/s 2024-04-25T10:02:08.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072553 s, 145 MB/s 2024-04-25T10:02:08.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.189 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648471 s, 162 MB/s 2024-04-25T10:02:08.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.196 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064719 s, 162 MB/s 2024-04-25T10:02:08.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627527 s, 167 MB/s 2024-04-25T10:02:08.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.210 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602105 s, 174 MB/s 2024-04-25T10:02:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627391 s, 167 MB/s 2024-04-25T10:02:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622069 s, 169 MB/s 2024-04-25T10:02:08.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:08.226 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759327 s, 138 MB/s 2024-04-25T10:02:08.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.300 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660031 s, 159 MB/s 2024-04-25T10:02:08.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.308 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748736 s, 140 MB/s 2024-04-25T10:02:08.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.316 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667813 s, 157 MB/s 2024-04-25T10:02:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571315 s, 184 MB/s 2024-04-25T10:02:08.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662425 s, 158 MB/s 2024-04-25T10:02:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652154 s, 161 MB/s 2024-04-25T10:02:08.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712226 s, 147 MB/s 2024-04-25T10:02:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666992 s, 157 MB/s 2024-04-25T10:02:08.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.361 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653521 s, 160 MB/s 2024-04-25T10:02:08.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.368 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633373 s, 166 MB/s 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643432 s, 163 MB/s 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632 s, 166 MB/s 2024-04-25T10:02:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.390 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629808 s, 166 MB/s 2024-04-25T10:02:08.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.397 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630501 s, 166 MB/s 2024-04-25T10:02:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.404 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612272 s, 171 MB/s 2024-04-25T10:02:08.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.411 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631315 s, 166 MB/s 2024-04-25T10:02:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072308 s, 145 MB/s 2024-04-25T10:02:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.426 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574894 s, 182 MB/s 2024-04-25T10:02:08.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.433 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627625 s, 167 MB/s 2024-04-25T10:02:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.441 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650876 s, 161 MB/s 2024-04-25T10:02:08.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593082 s, 177 MB/s 2024-04-25T10:02:08.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579259 s, 181 MB/s 2024-04-25T10:02:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600809 s, 175 MB/s 2024-04-25T10:02:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634646 s, 165 MB/s 2024-04-25T10:02:08.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620379 s, 169 MB/s 2024-04-25T10:02:08.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.483 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638681 s, 164 MB/s 2024-04-25T10:02:08.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596503 s, 176 MB/s 2024-04-25T10:02:08.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.497 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626159 s, 167 MB/s 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069544 s, 151 MB/s 2024-04-25T10:02:08.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.512 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064998 s, 161 MB/s 2024-04-25T10:02:08.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.522 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872555 s, 120 MB/s 2024-04-25T10:02:08.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:08.523 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:08.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643514 s, 163 MB/s 2024-04-25T10:02:08.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.595 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633148 s, 166 MB/s 2024-04-25T10:02:08.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648576 s, 162 MB/s 2024-04-25T10:02:08.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714195 s, 147 MB/s 2024-04-25T10:02:08.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698254 s, 150 MB/s 2024-04-25T10:02:08.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.626 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664571 s, 158 MB/s 2024-04-25T10:02:08.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635016 s, 165 MB/s 2024-04-25T10:02:08.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590607 s, 178 MB/s 2024-04-25T10:02:08.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.648 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731732 s, 143 MB/s 2024-04-25T10:02:08.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.655 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644723 s, 163 MB/s 2024-04-25T10:02:08.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711944 s, 147 MB/s 2024-04-25T10:02:08.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.672 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726233 s, 144 MB/s 2024-04-25T10:02:08.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.679 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617032 s, 170 MB/s 2024-04-25T10:02:08.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.687 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.687 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722404 s, 145 MB/s 2024-04-25T10:02:08.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.694 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627133 s, 167 MB/s 2024-04-25T10:02:08.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.701 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651369 s, 161 MB/s 2024-04-25T10:02:08.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.709 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720956 s, 145 MB/s 2024-04-25T10:02:08.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755477 s, 139 MB/s 2024-04-25T10:02:08.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.726 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748726 s, 140 MB/s 2024-04-25T10:02:08.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696584 s, 151 MB/s 2024-04-25T10:02:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.741 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649936 s, 161 MB/s 2024-04-25T10:02:08.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.749 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710362 s, 148 MB/s 2024-04-25T10:02:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.756 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652747 s, 161 MB/s 2024-04-25T10:02:08.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.763 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609479 s, 172 MB/s 2024-04-25T10:02:08.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.771 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654933 s, 160 MB/s 2024-04-25T10:02:08.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619077 s, 169 MB/s 2024-04-25T10:02:08.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.785 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610618 s, 172 MB/s 2024-04-25T10:02:08.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.791 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588841 s, 178 MB/s 2024-04-25T10:02:08.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.799 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623216 s, 168 MB/s 2024-04-25T10:02:08.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064199 s, 163 MB/s 2024-04-25T10:02:08.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.813 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577525 s, 182 MB/s 2024-04-25T10:02:08.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.820 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627093 s, 167 MB/s 2024-04-25T10:02:08.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:08.821 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:08.886 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.886 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.886 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647147 s, 162 MB/s 2024-04-25T10:02:08.893 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.893 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.893 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633722 s, 165 MB/s 2024-04-25T10:02:08.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.899 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057708 s, 182 MB/s 2024-04-25T10:02:08.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.907 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627023 s, 167 MB/s 2024-04-25T10:02:08.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.915 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724321 s, 145 MB/s 2024-04-25T10:02:08.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580887 s, 181 MB/s 2024-04-25T10:02:08.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.928 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591019 s, 177 MB/s 2024-04-25T10:02:08.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.936 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.936 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648848 s, 162 MB/s 2024-04-25T10:02:08.943 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.943 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.943 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062293 s, 168 MB/s 2024-04-25T10:02:08.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.951 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744177 s, 141 MB/s 2024-04-25T10:02:08.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734746 s, 143 MB/s 2024-04-25T10:02:08.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.966 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592195 s, 177 MB/s 2024-04-25T10:02:08.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709432 s, 148 MB/s 2024-04-25T10:02:08.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674124 s, 156 MB/s 2024-04-25T10:02:08.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:08.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:08.989 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645266 s, 163 MB/s 2024-04-25T10:02:08.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959694 s, 109 MB/s 2024-04-25T10:02:09.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.007 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606832 s, 173 MB/s 2024-04-25T10:02:09.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.015 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751591 s, 140 MB/s 2024-04-25T10:02:09.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635076 s, 165 MB/s 2024-04-25T10:02:09.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.029 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630378 s, 166 MB/s 2024-04-25T10:02:09.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.036 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612968 s, 171 MB/s 2024-04-25T10:02:09.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.043 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627976 s, 167 MB/s 2024-04-25T10:02:09.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.050 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623453 s, 168 MB/s 2024-04-25T10:02:09.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756833 s, 139 MB/s 2024-04-25T10:02:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.067 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698233 s, 150 MB/s 2024-04-25T10:02:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597782 s, 175 MB/s 2024-04-25T10:02:09.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061142 s, 171 MB/s 2024-04-25T10:02:09.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.089 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708194 s, 148 MB/s 2024-04-25T10:02:09.095 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.096 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629593 s, 167 MB/s 2024-04-25T10:02:09.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.104 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708712 s, 148 MB/s 2024-04-25T10:02:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.111 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627789 s, 167 MB/s 2024-04-25T10:02:09.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.118 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585958 s, 179 MB/s 2024-04-25T10:02:09.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:09.119 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:09.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066654 s, 157 MB/s 2024-04-25T10:02:09.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.192 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710099 s, 148 MB/s 2024-04-25T10:02:09.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.199 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061901 s, 169 MB/s 2024-04-25T10:02:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635088 s, 165 MB/s 2024-04-25T10:02:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573392 s, 183 MB/s 2024-04-25T10:02:09.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.221 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718096 s, 146 MB/s 2024-04-25T10:02:09.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.228 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060997 s, 172 MB/s 2024-04-25T10:02:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.236 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729539 s, 144 MB/s 2024-04-25T10:02:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658802 s, 159 MB/s 2024-04-25T10:02:09.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.251 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.251 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639291 s, 164 MB/s 2024-04-25T10:02:09.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.258 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713667 s, 147 MB/s 2024-04-25T10:02:09.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.266 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655316 s, 160 MB/s 2024-04-25T10:02:09.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.273 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609254 s, 172 MB/s 2024-04-25T10:02:09.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.280 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640668 s, 164 MB/s 2024-04-25T10:02:09.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.289 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803889 s, 130 MB/s 2024-04-25T10:02:09.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.296 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634685 s, 165 MB/s 2024-04-25T10:02:09.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.303 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060271 s, 174 MB/s 2024-04-25T10:02:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.311 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699088 s, 150 MB/s 2024-04-25T10:02:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066768 s, 157 MB/s 2024-04-25T10:02:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723677 s, 145 MB/s 2024-04-25T10:02:09.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.334 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648344 s, 162 MB/s 2024-04-25T10:02:09.341 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.341 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.341 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571089 s, 184 MB/s 2024-04-25T10:02:09.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.349 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.349 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708867 s, 148 MB/s 2024-04-25T10:02:09.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.357 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720023 s, 146 MB/s 2024-04-25T10:02:09.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.364 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571251 s, 184 MB/s 2024-04-25T10:02:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748092 s, 140 MB/s 2024-04-25T10:02:09.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720741 s, 145 MB/s 2024-04-25T10:02:09.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.388 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665113 s, 158 MB/s 2024-04-25T10:02:09.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613172 s, 171 MB/s 2024-04-25T10:02:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651103 s, 161 MB/s 2024-04-25T10:02:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615252 s, 170 MB/s 2024-04-25T10:02:09.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854009 s, 123 MB/s 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:09.420 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:09.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.484 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652726 s, 161 MB/s 2024-04-25T10:02:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565972 s, 185 MB/s 2024-04-25T10:02:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658784 s, 159 MB/s 2024-04-25T10:02:09.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062952 s, 167 MB/s 2024-04-25T10:02:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694532 s, 151 MB/s 2024-04-25T10:02:09.523 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.523 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.523 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973978 s, 108 MB/s 2024-04-25T10:02:09.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.530 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.530 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600671 s, 175 MB/s 2024-04-25T10:02:09.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.537 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583316 s, 180 MB/s 2024-04-25T10:02:09.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654969 s, 160 MB/s 2024-04-25T10:02:09.552 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.552 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.552 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649853 s, 161 MB/s 2024-04-25T10:02:09.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.559 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624294 s, 168 MB/s 2024-04-25T10:02:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573209 s, 183 MB/s 2024-04-25T10:02:09.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648753 s, 162 MB/s 2024-04-25T10:02:09.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.580 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631747 s, 166 MB/s 2024-04-25T10:02:09.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744869 s, 141 MB/s 2024-04-25T10:02:09.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.595 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577522 s, 182 MB/s 2024-04-25T10:02:09.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726614 s, 144 MB/s 2024-04-25T10:02:09.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609504 s, 172 MB/s 2024-04-25T10:02:09.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685035 s, 153 MB/s 2024-04-25T10:02:09.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.625 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567746 s, 185 MB/s 2024-04-25T10:02:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647377 s, 162 MB/s 2024-04-25T10:02:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.639 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638864 s, 164 MB/s 2024-04-25T10:02:09.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605608 s, 173 MB/s 2024-04-25T10:02:09.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.654 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066012 s, 159 MB/s 2024-04-25T10:02:09.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.661 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628217 s, 167 MB/s 2024-04-25T10:02:09.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.668 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613585 s, 171 MB/s 2024-04-25T10:02:09.675 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.675 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.675 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062968 s, 167 MB/s 2024-04-25T10:02:09.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631961 s, 166 MB/s 2024-04-25T10:02:09.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.689 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627699 s, 167 MB/s 2024-04-25T10:02:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.697 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063128 s, 166 MB/s 2024-04-25T10:02:09.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.705 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719456 s, 146 MB/s 2024-04-25T10:02:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635263 s, 165 MB/s 2024-04-25T10:02:09.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:09.713 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:09.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736548 s, 142 MB/s 2024-04-25T10:02:09.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673421 s, 156 MB/s 2024-04-25T10:02:09.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.794 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.794 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761784 s, 138 MB/s 2024-04-25T10:02:09.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.801 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629465 s, 167 MB/s 2024-04-25T10:02:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829017 s, 126 MB/s 2024-04-25T10:02:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647535 s, 162 MB/s 2024-04-25T10:02:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.824 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603172 s, 174 MB/s 2024-04-25T10:02:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.831 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584024 s, 180 MB/s 2024-04-25T10:02:09.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649937 s, 161 MB/s 2024-04-25T10:02:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.846 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685684 s, 153 MB/s 2024-04-25T10:02:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620951 s, 169 MB/s 2024-04-25T10:02:09.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709531 s, 148 MB/s 2024-04-25T10:02:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618104 s, 170 MB/s 2024-04-25T10:02:09.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569348 s, 184 MB/s 2024-04-25T10:02:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.882 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580397 s, 181 MB/s 2024-04-25T10:02:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717119 s, 146 MB/s 2024-04-25T10:02:09.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621927 s, 169 MB/s 2024-04-25T10:02:09.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645314 s, 162 MB/s 2024-04-25T10:02:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690513 s, 152 MB/s 2024-04-25T10:02:09.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669418 s, 157 MB/s 2024-04-25T10:02:09.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.927 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619745 s, 169 MB/s 2024-04-25T10:02:09.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654709 s, 160 MB/s 2024-04-25T10:02:09.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587917 s, 178 MB/s 2024-04-25T10:02:09.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.948 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594519 s, 176 MB/s 2024-04-25T10:02:09.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754252 s, 139 MB/s 2024-04-25T10:02:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.964 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070308 s, 149 MB/s 2024-04-25T10:02:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.972 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700666 s, 150 MB/s 2024-04-25T10:02:09.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.979 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655812 s, 160 MB/s 2024-04-25T10:02:09.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.986 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.986 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611554 s, 171 MB/s 2024-04-25T10:02:09.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:09.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:09.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586482 s, 179 MB/s 2024-04-25T10:02:10.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664614 s, 158 MB/s 2024-04-25T10:02:10.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.009 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724457 s, 145 MB/s 2024-04-25T10:02:10.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:10.010 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:10.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.075 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668149 s, 157 MB/s 2024-04-25T10:02:10.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.082 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638966 s, 164 MB/s 2024-04-25T10:02:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.091 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771376 s, 136 MB/s 2024-04-25T10:02:10.098 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.098 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.098 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663882 s, 158 MB/s 2024-04-25T10:02:10.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671203 s, 156 MB/s 2024-04-25T10:02:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613872 s, 171 MB/s 2024-04-25T10:02:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.120 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630242 s, 166 MB/s 2024-04-25T10:02:10.127 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.127 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.127 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640997 s, 164 MB/s 2024-04-25T10:02:10.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739397 s, 142 MB/s 2024-04-25T10:02:10.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645974 s, 162 MB/s 2024-04-25T10:02:10.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611971 s, 171 MB/s 2024-04-25T10:02:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624001 s, 168 MB/s 2024-04-25T10:02:10.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668585 s, 157 MB/s 2024-04-25T10:02:10.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.171 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582821 s, 180 MB/s 2024-04-25T10:02:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.179 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644076 s, 163 MB/s 2024-04-25T10:02:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.186 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620184 s, 169 MB/s 2024-04-25T10:02:10.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.193 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637037 s, 165 MB/s 2024-04-25T10:02:10.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.200 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618201 s, 170 MB/s 2024-04-25T10:02:10.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.208 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658716 s, 159 MB/s 2024-04-25T10:02:10.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602407 s, 174 MB/s 2024-04-25T10:02:10.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712392 s, 147 MB/s 2024-04-25T10:02:10.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.230 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616994 s, 170 MB/s 2024-04-25T10:02:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720918 s, 145 MB/s 2024-04-25T10:02:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.246 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730413 s, 144 MB/s 2024-04-25T10:02:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065007 s, 161 MB/s 2024-04-25T10:02:10.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070798 s, 148 MB/s 2024-04-25T10:02:10.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622275 s, 169 MB/s 2024-04-25T10:02:10.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.275 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573666 s, 183 MB/s 2024-04-25T10:02:10.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.282 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615973 s, 170 MB/s 2024-04-25T10:02:10.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.289 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.289 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646205 s, 162 MB/s 2024-04-25T10:02:10.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.296 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617732 s, 170 MB/s 2024-04-25T10:02:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.304 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705265 s, 149 MB/s 2024-04-25T10:02:10.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:10.305 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:10.305 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:10.305 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:02:10.305 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:10.306 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:10.306 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:10.306 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061093 s, 172 MB/s 2024-04-25T10:02:10.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.376 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.376 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587692 s, 178 MB/s 2024-04-25T10:02:10.383 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.383 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.383 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661728 s, 158 MB/s 2024-04-25T10:02:10.390 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635452 s, 165 MB/s 2024-04-25T10:02:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633282 s, 166 MB/s 2024-04-25T10:02:10.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650441 s, 161 MB/s 2024-04-25T10:02:10.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613194 s, 171 MB/s 2024-04-25T10:02:10.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.420 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640096 s, 164 MB/s 2024-04-25T10:02:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725338 s, 145 MB/s 2024-04-25T10:02:10.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.436 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742359 s, 141 MB/s 2024-04-25T10:02:10.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.444 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740906 s, 142 MB/s 2024-04-25T10:02:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630074 s, 166 MB/s 2024-04-25T10:02:10.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731802 s, 143 MB/s 2024-04-25T10:02:10.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607035 s, 173 MB/s 2024-04-25T10:02:10.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.473 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.473 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059674 s, 176 MB/s 2024-04-25T10:02:10.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.481 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642232 s, 163 MB/s 2024-04-25T10:02:10.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.488 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627512 s, 167 MB/s 2024-04-25T10:02:10.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.495 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618012 s, 170 MB/s 2024-04-25T10:02:10.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715982 s, 146 MB/s 2024-04-25T10:02:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638229 s, 164 MB/s 2024-04-25T10:02:10.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.518 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639948 s, 164 MB/s 2024-04-25T10:02:10.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.526 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757135 s, 138 MB/s 2024-04-25T10:02:10.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700282 s, 150 MB/s 2024-04-25T10:02:10.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566031 s, 185 MB/s 2024-04-25T10:02:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623462 s, 168 MB/s 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674626 s, 155 MB/s 2024-04-25T10:02:10.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.562 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063038 s, 166 MB/s 2024-04-25T10:02:10.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.569 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594341 s, 176 MB/s 2024-04-25T10:02:10.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.577 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628116 s, 167 MB/s 2024-04-25T10:02:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767093 s, 137 MB/s 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646691 s, 162 MB/s 2024-04-25T10:02:10.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.601 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674807 s, 155 MB/s 2024-04-25T10:02:10.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:10.601 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:10.601 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:10.601 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:02:10.601 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:10.601 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:10.602 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:10.602 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644525 s, 163 MB/s 2024-04-25T10:02:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754922 s, 139 MB/s 2024-04-25T10:02:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.681 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625006 s, 168 MB/s 2024-04-25T10:02:10.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.690 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724695 s, 145 MB/s 2024-04-25T10:02:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.697 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064053 s, 164 MB/s 2024-04-25T10:02:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.704 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626949 s, 167 MB/s 2024-04-25T10:02:10.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.711 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626747 s, 167 MB/s 2024-04-25T10:02:10.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.719 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720538 s, 146 MB/s 2024-04-25T10:02:10.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.726 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640302 s, 164 MB/s 2024-04-25T10:02:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.733 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579327 s, 181 MB/s 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574123 s, 183 MB/s 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669565 s, 157 MB/s 2024-04-25T10:02:10.754 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.755 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654269 s, 160 MB/s 2024-04-25T10:02:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.762 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063258 s, 166 MB/s 2024-04-25T10:02:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.769 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656789 s, 160 MB/s 2024-04-25T10:02:10.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732903 s, 143 MB/s 2024-04-25T10:02:10.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.785 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660289 s, 159 MB/s 2024-04-25T10:02:10.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740732 s, 142 MB/s 2024-04-25T10:02:10.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613148 s, 171 MB/s 2024-04-25T10:02:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592873 s, 177 MB/s 2024-04-25T10:02:10.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.815 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666085 s, 157 MB/s 2024-04-25T10:02:10.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.822 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624282 s, 168 MB/s 2024-04-25T10:02:10.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.829 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059529 s, 176 MB/s 2024-04-25T10:02:10.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709677 s, 148 MB/s 2024-04-25T10:02:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.844 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063319 s, 166 MB/s 2024-04-25T10:02:10.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747685 s, 140 MB/s 2024-04-25T10:02:10.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063015 s, 166 MB/s 2024-04-25T10:02:10.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719327 s, 146 MB/s 2024-04-25T10:02:10.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655721 s, 160 MB/s 2024-04-25T10:02:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.882 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641624 s, 163 MB/s 2024-04-25T10:02:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630053 s, 166 MB/s 2024-04-25T10:02:10.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730691 s, 144 MB/s 2024-04-25T10:02:10.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:10.899 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590689 s, 178 MB/s 2024-04-25T10:02:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587855 s, 178 MB/s 2024-04-25T10:02:10.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725874 s, 144 MB/s 2024-04-25T10:02:10.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638857 s, 164 MB/s 2024-04-25T10:02:10.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:10.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:10.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726528 s, 144 MB/s 2024-04-25T10:02:11.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646926 s, 162 MB/s 2024-04-25T10:02:11.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.008 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722653 s, 145 MB/s 2024-04-25T10:02:11.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.015 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626844 s, 167 MB/s 2024-04-25T10:02:11.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596087 s, 176 MB/s 2024-04-25T10:02:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669974 s, 157 MB/s 2024-04-25T10:02:11.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675255 s, 155 MB/s 2024-04-25T10:02:11.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.045 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652858 s, 161 MB/s 2024-04-25T10:02:11.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658417 s, 159 MB/s 2024-04-25T10:02:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.060 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722053 s, 145 MB/s 2024-04-25T10:02:11.068 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.068 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.068 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719228 s, 146 MB/s 2024-04-25T10:02:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653738 s, 160 MB/s 2024-04-25T10:02:11.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.083 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641904 s, 163 MB/s 2024-04-25T10:02:11.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615386 s, 170 MB/s 2024-04-25T10:02:11.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.097 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630116 s, 166 MB/s 2024-04-25T10:02:11.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.104 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619472 s, 169 MB/s 2024-04-25T10:02:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.112 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739802 s, 142 MB/s 2024-04-25T10:02:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762725 s, 137 MB/s 2024-04-25T10:02:11.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659107 s, 159 MB/s 2024-04-25T10:02:11.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691534 s, 152 MB/s 2024-04-25T10:02:11.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616929 s, 170 MB/s 2024-04-25T10:02:11.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615315 s, 170 MB/s 2024-04-25T10:02:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752731 s, 139 MB/s 2024-04-25T10:02:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584344 s, 179 MB/s 2024-04-25T10:02:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007044 s, 149 MB/s 2024-04-25T10:02:11.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.180 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574478 s, 183 MB/s 2024-04-25T10:02:11.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.187 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631345 s, 166 MB/s 2024-04-25T10:02:11.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.196 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763368 s, 137 MB/s 2024-04-25T10:02:11.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:11.197 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:11.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777679 s, 135 MB/s 2024-04-25T10:02:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639963 s, 164 MB/s 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.279 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715375 s, 147 MB/s 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650693 s, 161 MB/s 2024-04-25T10:02:11.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.293 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619318 s, 169 MB/s 2024-04-25T10:02:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.301 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711127 s, 147 MB/s 2024-04-25T10:02:11.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006801 s, 154 MB/s 2024-04-25T10:02:11.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.317 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729456 s, 144 MB/s 2024-04-25T10:02:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.325 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704844 s, 149 MB/s 2024-04-25T10:02:11.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.333 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752737 s, 139 MB/s 2024-04-25T10:02:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.340 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643088 s, 163 MB/s 2024-04-25T10:02:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.348 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006417 s, 163 MB/s 2024-04-25T10:02:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.356 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740692 s, 142 MB/s 2024-04-25T10:02:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632415 s, 166 MB/s 2024-04-25T10:02:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584767 s, 179 MB/s 2024-04-25T10:02:11.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.377 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660974 s, 159 MB/s 2024-04-25T10:02:11.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.384 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.384 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622291 s, 169 MB/s 2024-04-25T10:02:11.392 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.392 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.392 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657258 s, 160 MB/s 2024-04-25T10:02:11.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.399 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607936 s, 172 MB/s 2024-04-25T10:02:11.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.406 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655647 s, 160 MB/s 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065174 s, 161 MB/s 2024-04-25T10:02:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.421 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647237 s, 162 MB/s 2024-04-25T10:02:11.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.429 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645367 s, 162 MB/s 2024-04-25T10:02:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.437 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007199 s, 146 MB/s 2024-04-25T10:02:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586098 s, 179 MB/s 2024-04-25T10:02:11.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.450 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601926 s, 174 MB/s 2024-04-25T10:02:11.458 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.458 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.458 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646351 s, 162 MB/s 2024-04-25T10:02:11.465 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.465 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.465 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641781 s, 163 MB/s 2024-04-25T10:02:11.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.472 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590278 s, 178 MB/s 2024-04-25T10:02:11.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.480 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728953 s, 144 MB/s 2024-04-25T10:02:11.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.488 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715885 s, 146 MB/s 2024-04-25T10:02:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.496 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631841 s, 166 MB/s 2024-04-25T10:02:11.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:11.496 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:11.497 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:11.497 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:02:11.497 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:11.497 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:11.497 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:11.497 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644918 s, 163 MB/s 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581336 s, 180 MB/s 2024-04-25T10:02:11.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.574 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565925 s, 185 MB/s 2024-04-25T10:02:11.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625029 s, 168 MB/s 2024-04-25T10:02:11.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619197 s, 169 MB/s 2024-04-25T10:02:11.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730782 s, 143 MB/s 2024-04-25T10:02:11.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624433 s, 168 MB/s 2024-04-25T10:02:11.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641033 s, 164 MB/s 2024-04-25T10:02:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569012 s, 184 MB/s 2024-04-25T10:02:11.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664519 s, 158 MB/s 2024-04-25T10:02:11.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722255 s, 145 MB/s 2024-04-25T10:02:11.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624516 s, 168 MB/s 2024-04-25T10:02:11.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.647 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623348 s, 168 MB/s 2024-04-25T10:02:11.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.655 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727402 s, 144 MB/s 2024-04-25T10:02:11.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705065 s, 149 MB/s 2024-04-25T10:02:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.670 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628147 s, 167 MB/s 2024-04-25T10:02:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.678 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726384 s, 144 MB/s 2024-04-25T10:02:11.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.685 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060367 s, 174 MB/s 2024-04-25T10:02:11.692 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665756 s, 158 MB/s 2024-04-25T10:02:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062381 s, 168 MB/s 2024-04-25T10:02:11.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.707 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631055 s, 166 MB/s 2024-04-25T10:02:11.715 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.715 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.715 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712448 s, 147 MB/s 2024-04-25T10:02:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627738 s, 167 MB/s 2024-04-25T10:02:11.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729829 s, 144 MB/s 2024-04-25T10:02:11.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625629 s, 168 MB/s 2024-04-25T10:02:11.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682352 s, 154 MB/s 2024-04-25T10:02:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572493 s, 183 MB/s 2024-04-25T10:02:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644207 s, 163 MB/s 2024-04-25T10:02:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.767 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736166 s, 142 MB/s 2024-04-25T10:02:11.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.775 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660262 s, 159 MB/s 2024-04-25T10:02:11.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.782 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.782 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612475 s, 171 MB/s 2024-04-25T10:02:11.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.789 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.790 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634872 s, 165 MB/s 2024-04-25T10:02:11.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:11.790 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:11.790 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:11.790 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:02:11.790 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:11.791 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:11.791 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:11.791 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.857 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072659 s, 144 MB/s 2024-04-25T10:02:11.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.863 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063594 s, 165 MB/s 2024-04-25T10:02:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.870 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659076 s, 159 MB/s 2024-04-25T10:02:11.878 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.878 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.878 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673715 s, 156 MB/s 2024-04-25T10:02:11.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.885 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.885 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061361 s, 171 MB/s 2024-04-25T10:02:11.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.893 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.893 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706388 s, 148 MB/s 2024-04-25T10:02:11.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.899 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586946 s, 179 MB/s 2024-04-25T10:02:11.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.907 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635966 s, 165 MB/s 2024-04-25T10:02:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.914 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622091 s, 169 MB/s 2024-04-25T10:02:11.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647389 s, 162 MB/s 2024-04-25T10:02:11.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.930 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075398 s, 139 MB/s 2024-04-25T10:02:11.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.937 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647498 s, 162 MB/s 2024-04-25T10:02:11.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.944 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633081 s, 166 MB/s 2024-04-25T10:02:11.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.952 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636904 s, 165 MB/s 2024-04-25T10:02:11.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064439 s, 163 MB/s 2024-04-25T10:02:11.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.966 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618734 s, 169 MB/s 2024-04-25T10:02:11.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.973 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651722 s, 161 MB/s 2024-04-25T10:02:11.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.980 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006307 s, 166 MB/s 2024-04-25T10:02:11.987 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629676 s, 167 MB/s 2024-04-25T10:02:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:11.995 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640676 s, 164 MB/s 2024-04-25T10:02:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.002 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630138 s, 166 MB/s 2024-04-25T10:02:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.010 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720328 s, 146 MB/s 2024-04-25T10:02:12.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.018 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731525 s, 143 MB/s 2024-04-25T10:02:12.025 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.025 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.025 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568399 s, 184 MB/s 2024-04-25T10:02:12.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.033 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712471 s, 147 MB/s 2024-04-25T10:02:12.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074231 s, 141 MB/s 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641834 s, 163 MB/s 2024-04-25T10:02:12.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630406 s, 166 MB/s 2024-04-25T10:02:12.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.063 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630265 s, 166 MB/s 2024-04-25T10:02:12.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.071 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071776 s, 146 MB/s 2024-04-25T10:02:12.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600998 s, 174 MB/s 2024-04-25T10:02:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652204 s, 161 MB/s 2024-04-25T10:02:12.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:12.087 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:12.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643453 s, 163 MB/s 2024-04-25T10:02:12.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618461 s, 170 MB/s 2024-04-25T10:02:12.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064499 s, 163 MB/s 2024-04-25T10:02:12.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577865 s, 181 MB/s 2024-04-25T10:02:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.179 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613895 s, 171 MB/s 2024-04-25T10:02:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638989 s, 164 MB/s 2024-04-25T10:02:12.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.195 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074622 s, 141 MB/s 2024-04-25T10:02:12.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732746 s, 143 MB/s 2024-04-25T10:02:12.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.211 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726107 s, 144 MB/s 2024-04-25T10:02:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615384 s, 170 MB/s 2024-04-25T10:02:12.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.228 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879936 s, 119 MB/s 2024-04-25T10:02:12.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646535 s, 162 MB/s 2024-04-25T10:02:12.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059998 s, 175 MB/s 2024-04-25T10:02:12.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.250 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660812 s, 159 MB/s 2024-04-25T10:02:12.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.257 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616502 s, 170 MB/s 2024-04-25T10:02:12.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.265 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757346 s, 138 MB/s 2024-04-25T10:02:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675517 s, 155 MB/s 2024-04-25T10:02:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059345 s, 177 MB/s 2024-04-25T10:02:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675021 s, 155 MB/s 2024-04-25T10:02:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644796 s, 163 MB/s 2024-04-25T10:02:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633755 s, 165 MB/s 2024-04-25T10:02:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607463 s, 173 MB/s 2024-04-25T10:02:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633788 s, 165 MB/s 2024-04-25T10:02:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622946 s, 168 MB/s 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057777 s, 181 MB/s 2024-04-25T10:02:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.338 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699968 s, 150 MB/s 2024-04-25T10:02:12.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.345 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.345 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061674 s, 170 MB/s 2024-04-25T10:02:12.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.353 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664762 s, 158 MB/s 2024-04-25T10:02:12.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.359 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565717 s, 185 MB/s 2024-04-25T10:02:12.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.367 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729103 s, 144 MB/s 2024-04-25T10:02:12.374 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.374 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.374 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562754 s, 186 MB/s 2024-04-25T10:02:12.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.382 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659188 s, 159 MB/s 2024-04-25T10:02:12.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:12.383 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668438 s, 157 MB/s 2024-04-25T10:02:12.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623127 s, 168 MB/s 2024-04-25T10:02:12.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.463 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731584 s, 143 MB/s 2024-04-25T10:02:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.471 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650734 s, 161 MB/s 2024-04-25T10:02:12.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.480 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810359 s, 129 MB/s 2024-04-25T10:02:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643261 s, 163 MB/s 2024-04-25T10:02:12.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.494 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606007 s, 173 MB/s 2024-04-25T10:02:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.501 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642338 s, 163 MB/s 2024-04-25T10:02:12.508 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.508 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.508 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057383 s, 183 MB/s 2024-04-25T10:02:12.515 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.515 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.515 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660084 s, 159 MB/s 2024-04-25T10:02:12.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.522 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.522 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062344 s, 168 MB/s 2024-04-25T10:02:12.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.529 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619383 s, 169 MB/s 2024-04-25T10:02:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639855 s, 164 MB/s 2024-04-25T10:02:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635445 s, 165 MB/s 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609584 s, 172 MB/s 2024-04-25T10:02:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616437 s, 170 MB/s 2024-04-25T10:02:12.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.566 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663686 s, 158 MB/s 2024-04-25T10:02:12.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.572 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598574 s, 175 MB/s 2024-04-25T10:02:12.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910117 s, 115 MB/s 2024-04-25T10:02:12.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.590 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720765 s, 145 MB/s 2024-04-25T10:02:12.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.597 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615564 s, 170 MB/s 2024-04-25T10:02:12.605 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.605 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.605 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657059 s, 160 MB/s 2024-04-25T10:02:12.612 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.612 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.612 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617898 s, 170 MB/s 2024-04-25T10:02:12.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.619 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667746 s, 157 MB/s 2024-04-25T10:02:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.626 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590596 s, 178 MB/s 2024-04-25T10:02:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.634 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726664 s, 144 MB/s 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.644 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809859 s, 129 MB/s 2024-04-25T10:02:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733815 s, 143 MB/s 2024-04-25T10:02:12.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649298 s, 161 MB/s 2024-04-25T10:02:12.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.668 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755569 s, 139 MB/s 2024-04-25T10:02:12.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.674 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585669 s, 179 MB/s 2024-04-25T10:02:12.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662015 s, 158 MB/s 2024-04-25T10:02:12.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.683 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:12.683 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:12.683 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:02:12.683 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:12.683 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:12.683 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:12.684 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.748 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733538 s, 143 MB/s 2024-04-25T10:02:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.755 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625941 s, 168 MB/s 2024-04-25T10:02:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.762 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063124 s, 166 MB/s 2024-04-25T10:02:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630507 s, 166 MB/s 2024-04-25T10:02:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719611 s, 146 MB/s 2024-04-25T10:02:12.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.785 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649591 s, 161 MB/s 2024-04-25T10:02:12.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.792 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572725 s, 183 MB/s 2024-04-25T10:02:12.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.799 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656769 s, 160 MB/s 2024-04-25T10:02:12.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620836 s, 169 MB/s 2024-04-25T10:02:12.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.815 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073295 s, 143 MB/s 2024-04-25T10:02:12.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.823 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723112 s, 145 MB/s 2024-04-25T10:02:12.830 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.830 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.830 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635794 s, 165 MB/s 2024-04-25T10:02:12.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726772 s, 144 MB/s 2024-04-25T10:02:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609711 s, 172 MB/s 2024-04-25T10:02:12.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748012 s, 140 MB/s 2024-04-25T10:02:12.860 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.860 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.860 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628597 s, 167 MB/s 2024-04-25T10:02:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.867 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616103 s, 170 MB/s 2024-04-25T10:02:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.874 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057786 s, 181 MB/s 2024-04-25T10:02:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597778 s, 175 MB/s 2024-04-25T10:02:12.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.888 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624433 s, 168 MB/s 2024-04-25T10:02:12.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715065 s, 147 MB/s 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621965 s, 169 MB/s 2024-04-25T10:02:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717469 s, 146 MB/s 2024-04-25T10:02:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082662 s, 127 MB/s 2024-04-25T10:02:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.927 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057395 s, 183 MB/s 2024-04-25T10:02:12.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575398 s, 182 MB/s 2024-04-25T10:02:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630355 s, 166 MB/s 2024-04-25T10:02:12.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.948 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631742 s, 166 MB/s 2024-04-25T10:02:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.955 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621446 s, 169 MB/s 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.962 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588628 s, 178 MB/s 2024-04-25T10:02:12.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.970 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711249 s, 147 MB/s 2024-04-25T10:02:12.976 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:12.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:12.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564206 s, 186 MB/s 2024-04-25T10:02:12.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:12.978 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:13.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.043 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643737 s, 163 MB/s 2024-04-25T10:02:13.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.050 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061156 s, 171 MB/s 2024-04-25T10:02:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.057 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633043 s, 166 MB/s 2024-04-25T10:02:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.064 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622418 s, 168 MB/s 2024-04-25T10:02:13.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.071 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626742 s, 167 MB/s 2024-04-25T10:02:13.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727111 s, 144 MB/s 2024-04-25T10:02:13.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.086 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.086 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641168 s, 164 MB/s 2024-04-25T10:02:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616488 s, 170 MB/s 2024-04-25T10:02:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068861 s, 152 MB/s 2024-04-25T10:02:13.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.108 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.108 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652407 s, 161 MB/s 2024-04-25T10:02:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.116 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647952 s, 162 MB/s 2024-04-25T10:02:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621895 s, 169 MB/s 2024-04-25T10:02:13.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722305 s, 145 MB/s 2024-04-25T10:02:13.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062907 s, 167 MB/s 2024-04-25T10:02:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559357 s, 187 MB/s 2024-04-25T10:02:13.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.238 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.238 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865326 s, 121 MB/s 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771377 s, 136 MB/s 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071928 s, 146 MB/s 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619439 s, 169 MB/s 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632414 s, 166 MB/s 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709446 s, 148 MB/s 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711675 s, 147 MB/s 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635528 s, 165 MB/s 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630738 s, 166 MB/s 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633896 s, 165 MB/s 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058695 s, 179 MB/s 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638566 s, 164 MB/s 2024-04-25T10:02:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075889 s, 138 MB/s 2024-04-25T10:02:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731984 s, 143 MB/s 2024-04-25T10:02:13.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666522 s, 157 MB/s 2024-04-25T10:02:13.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671622 s, 156 MB/s 2024-04-25T10:02:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582569 s, 180 MB/s 2024-04-25T10:02:13.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.415+0000 7fa1f4aee640 1 Processor -- start 2024-04-25T10:02:13.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.415+0000 7fa1f4aee640 1 -- start start 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1f4aee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1f4aee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1f015b270 0x7fa1f015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1f4aee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f0072280 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1f4aee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f0071270 con 0x7fa1f015b270 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1f4aee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f00714c0 con 0x7fa1f00ce230 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1ee59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa1f00cc4f0 0x7fa1f00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:57348/0 (socket says 172.21.15.148:57348) 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.416+0000 7fa1ee59c640 1 -- 172.21.15.148:0/2040141870 learned_addr learned my addr 172.21.15.148:0/2040141870 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/2040141870 >> v2:172.21.15.188:3300/0 conn(0x7fa1f015b270 0x7fa1f015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1ed59a640 1 -- 172.21.15.148:0/2040141870 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440396993 0 0) 0x7fa1f0072280 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1ed59a640 1 -- 172.21.15.148:0/2040141870 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8003520 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1eed9d640 1 -- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 msgr2=0x7fa1f00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f00cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1eed9d640 1 -- 172.21.15.148:0/2040141870 >> v1:172.21.15.148:6789/0 conn(0x7fa1f00cc4f0 legacy=0x7fa1f00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1eed9d640 1 -- 172.21.15.148:0/2040141870 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00717a0 con 0x7fa1f015b270 2024-04-25T10:02:13.423 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.417+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f00cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:13.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f015b6c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80097a0 tx=0x7fa1e802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec9607892954637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1ed59a640 1 -- 172.21.15.148:0/2040141870 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037070 con 0x7fa1f015b270 2024-04-25T10:02:13.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1ed59a640 1 -- 172.21.15.148:0/2040141870 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8032440 con 0x7fa1f015b270 2024-04-25T10:02:13.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1ed59a640 1 -- 172.21.15.148:0/2040141870 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80327f0 con 0x7fa1f015b270 2024-04-25T10:02:13.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 -- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 msgr2=0x7fa1f015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f015b6c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80097a0 tx=0x7fa1e802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 -- 172.21.15.148:0/2040141870 shutdown_connections 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f015b6c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/2040141870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f00cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 -- 172.21.15.148:0/2040141870 >> 172.21.15.148:0/2040141870 conn(0x7fa1f006abe0 msgr2=0x7fa1f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 -- 172.21.15.148:0/2040141870 shutdown_connections 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.418+0000 7fa1f4aee640 1 -- 172.21.15.148:0/2040141870 wait complete. 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 Processor -- start 2024-04-25T10:02:13.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 -- start start 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 0x7fa1f01447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f0144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f0151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f015bc00 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0071d60 con 0x7fa1f015b270 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.420+0000 7fa1f4aee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f015bd80 con 0x7fa1f00ce230 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 0x7fa1f01447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 0x7fa1f01447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48006/0 (socket says 172.21.15.148:48006) 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3732566300 learned_addr learned my addr 172.21.15.148:0/3732566300 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f0144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f0151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 msgr2=0x7fa1f0144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f0144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 msgr2=0x7fa1f0151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f0151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3732566300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0146b10 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f0151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 0x7fa1f01447d0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000cad0 tx=0x7fa1e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1deffd640 1 -- 172.21.15.148:0/3732566300 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0017070 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1deffd640 1 -- 172.21.15.148:0/3732566300 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e000ac30 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f0145250 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1deffd640 1 -- 172.21.15.148:0/3732566300 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0012400 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.421+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f0147060 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.423+0000 7fa1deffd640 1 -- 172.21.15.148:0/3732566300 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1e001a070 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.423+0000 7fa1deffd640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1d006b4d0 0x7fa1d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.423+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1d006b4d0 0x7fa1d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.423+0000 7fa1deffd640 1 -- 172.21.15.148:0/3732566300 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1e008a110 con 0x7fa1f00cc4f0 2024-04-25T10:02:13.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.424+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1d006b4d0 0x7fa1d006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009560 tx=0x7fa1e4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.424+0000 7fa1f77ec740 1 Processor -- start 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.424+0000 7fa1f77ec740 1 -- start start 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b0d200 0x560805b0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b498d0 0x560805b0d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 0x560805b38140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805a71000 con 0x560805b37cc0 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805a19f20 con 0x560805b498d0 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560805a6d0a0 con 0x560805b0d200 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 0x560805b38140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 0x560805b38140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48020/0 (socket says 172.21.15.148:48020) 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 -- 172.21.15.148:0/3693938437 learned_addr learned my addr 172.21.15.148:0/3693938437 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b498d0 0x560805b0d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b0d200 0x560805b0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 -- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b0d200 msgr2=0x560805b0f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b0d200 0x560805b0f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 -- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b498d0 msgr2=0x560805b0d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b498d0 0x560805b0d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.425+0000 7fa1eed9d640 1 -- 172.21.15.148:0/3693938437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560805851ba0 con 0x560805b37cc0 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 0x560805b38140 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80098c0 tx=0x7fa1e802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a019fd0b955ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1cffff640 1 -- 172.21.15.148:0/3693938437 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037040 con 0x560805b37cc0 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1cffff640 1 -- 172.21.15.148:0/3693938437 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e803e660 con 0x560805b37cc0 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1cffff640 1 -- 172.21.15.148:0/3693938437 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e803ea10 con 0x560805b37cc0 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3693938437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 msgr2=0x560805b38140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 0x560805b38140 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80098c0 tx=0x7fa1e802c000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3693938437 shutdown_connections 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b37cc0 0x560805b38140 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b498d0 0x560805b0d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3693938437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b0d200 0x560805b0f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.426+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3693938437 >> 172.21.15.148:0/3693938437 conn(0x560805ad5050 msgr2=0x560805b52cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.427+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3693938437 shutdown_connections 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.427+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3693938437 wait complete. 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.427+0000 7fa1f77ec740 1 -- start start 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.427+0000 7fa1f77ec740 1 Processor -- start 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805b54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b3a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805a0f610 con 0x560805b0d200 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805b35810 con 0x560805b37cc0 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560805b54d80 con 0x560805b498d0 2024-04-25T10:02:13.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1f77ec740 10 client.0 _ll_register_callbacks cb 0x560805a27f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b22fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48028/0 (socket says 172.21.15.148:48028) 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 -- 172.21.15.148:0/1269563316 learned_addr learned my addr 172.21.15.148:0/1269563316 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805b54410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 -- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 msgr2=0x560805b3a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b3a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 -- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 msgr2=0x560805b54410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805b54410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.428+0000 7fa1ee59c640 1 -- 172.21.15.148:0/1269563316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560805745490 con 0x560805b0d200 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b22fc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00883b0 tx=0x7fa1e005c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929ddd6929f11d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:13.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/1269563316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0070070 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/1269563316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1e0057b60 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1e803e660 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/1269563316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e0057d60 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 10 client.8901 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5608058b2330 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/1269563316 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e803e660 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/1269563316 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa1e0054460 con 0x560805b0d200 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 10 client.8901 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 10 client.8901 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 20 client.8901 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b25140 0x560805b4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 4 client.8901 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560805a7e540 con 0x560805b25140 2024-04-25T10:02:13.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.429+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b25140 0x560805b4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b25140 0x560805b4d1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce67d8e4ce054568 server_cookie=7ae27ea1ec47471f in_seq=0 out_seq=0 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1cd7fa640 10 client.8901 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/1269563316 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560805a7e540 con 0x560805b25140 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1cd7fa640 10 client.8901 handle_command_reply: tid=0 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 client.8901 shutdown 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 msgr2=0x560805b22fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b22fc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00883b0 tx=0x7fa1e005c870 comp rx=0 tx=0).stop 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 shutdown_connections 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b25140 0x560805b4d1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b3a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805b54410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1269563316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b22fc0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1cd7fa640 0 client.8901 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.430+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 >> 172.21.15.148:0/1269563316 conn(0x560805ad5050 msgr2=0x560805b34f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.431+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 shutdown_connections 2024-04-25T10:02:13.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.431+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1269563316 wait complete. 2024-04-25T10:02:13.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.431+0000 7fa1f77ec740 20 client.8901 trim_cache size 0 max 16384 2024-04-25T10:02:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.445+0000 7fa1f77ec740 1 Processor -- start 2024-04-25T10:02:13.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.445+0000 7fa1f77ec740 1 -- start start 2024-04-25T10:02:13.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b0d200 0x560805b35160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b37cc0 0x560805b35c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 0x560805b36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01fe0d0 con 0x560805b498d0 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01fd270 con 0x560805b0d200 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f01484c0 con 0x560805b37cc0 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1eed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 0x560805b36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1eed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 0x560805b36150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48040/0 (socket says 172.21.15.148:48040) 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1eed9d640 1 -- 172.21.15.148:0/1835454871 learned_addr learned my addr 172.21.15.148:0/1835454871 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.446+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b0d200 0x560805b35160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1eed9d640 1 -- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b37cc0 msgr2=0x560805b35c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b37cc0 0x560805b35c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1eed9d640 1 -- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b0d200 msgr2=0x560805b35160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b0d200 0x560805b35160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1eed9d640 1 -- 172.21.15.148:0/1835454871 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56080571e610 con 0x560805b498d0 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 0x560805b36150 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802a770 tx=0x7fa1e8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db65c6810002d5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1cf7fe640 1 -- 172.21.15.148:0/1835454871 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037040 con 0x560805b498d0 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1cf7fe640 1 -- 172.21.15.148:0/1835454871 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e804a020 con 0x560805b498d0 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1cf7fe640 1 -- 172.21.15.148:0/1835454871 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003610 con 0x560805b498d0 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1835454871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 msgr2=0x560805b36150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 0x560805b36150 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802a770 tx=0x7fa1e8008030 comp rx=0 tx=0).stop 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1835454871 shutdown_connections 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b498d0 0x560805b36150 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b37cc0 0x560805b35c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/1835454871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b0d200 0x560805b35160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1835454871 >> 172.21.15.148:0/1835454871 conn(0x560805ad5050 msgr2=0x560805b33e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1835454871 shutdown_connections 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.447+0000 7fa1f77ec740 1 -- 172.21.15.148:0/1835454871 wait complete. 2024-04-25T10:02:13.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.448+0000 7fa1f77ec740 1 -- start start 2024-04-25T10:02:13.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.448+0000 7fa1f77ec740 1 Processor -- start 2024-04-25T10:02:13.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805bbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f0148ac0 con 0x560805b0d200 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805b24a30 con 0x560805b37cc0 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560805bc31d0 con 0x560805b498d0 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1f77ec740 10 client.0 _ll_register_callbacks cb 0x7fa1e8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b38c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48054/0 (socket says 172.21.15.148:48054) 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3431514892 learned_addr learned my addr 172.21.15.148:0/3431514892 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805bbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1eed9d640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 msgr2=0x560805b380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 msgr2=0x560805bbbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805bbbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 -- 172.21.15.148:0/3431514892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560805b39b60 con 0x560805b0d200 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1ee59c640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b38c10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005c20 tx=0x7fa1e00882e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee016cc5871647dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.449+0000 7fa1cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/3431514892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0070040 con 0x560805b0d200 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/3431514892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1e00577e0 con 0x560805b0d200 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1e804a020 con 0x560805b0d200 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/3431514892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e0058070 con 0x560805b0d200 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 10 client.8907 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x560805b35370 con 0x560805b0d200 2024-04-25T10:02:13.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/3431514892 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e804a020 con 0x560805b0d200 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/3431514892 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa1e0077020 con 0x560805b0d200 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 10 client.8907 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 10 client.8907 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 20 client.8907 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b385e0 0x560805b48650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 4 client.8907 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x560805a78890 con 0x560805b385e0 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b385e0 0x560805b48650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1cd7fa640 10 client.8907 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.450+0000 7fa1edd9b640 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b385e0 0x560805b48650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca867fe65e574c5c server_cookie=4080752fe11a76fd in_seq=0 out_seq=0 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1cd7fa640 1 -- 172.21.15.148:0/3431514892 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560805a78890 con 0x560805b385e0 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1cd7fa640 10 client.8907 handle_command_reply: tid=0 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 client.8907 shutdown 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 msgr2=0x560805b38c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b38c10 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0005c20 tx=0x7fa1e00882e0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 shutdown_connections 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x560805b385e0 0x560805b48650 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560805b498d0 0x560805b380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560805b37cc0 0x560805bbbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1cd7fa640 0 client.8907 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 --2- 172.21.15.148:0/3431514892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560805b0d200 0x560805b38c10 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 >> 172.21.15.148:0/3431514892 conn(0x560805ad5050 msgr2=0x560805b54020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 shutdown_connections 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 1 -- 172.21.15.148:0/3431514892 wait complete. 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.454+0000 7fa1f77ec740 20 client.8907 trim_cache size 0 max 16384 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "cc1db65d-4e06-43ed-a25b-2b485ad11758", 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:02:13.463 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:13.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1d006b4d0 msgr2=0x7fa1d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1d006b4d0 0x7fa1d006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009560 tx=0x7fa1e4009190 comp rx=0 tx=0).stop 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 msgr2=0x7fa1f01447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 0x7fa1f01447d0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000cad0 tx=0x7fa1e000cf90 comp rx=0 tx=0).stop 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 shutdown_connections 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa1d006b4d0 0x7fa1d006d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1f015b270 0x7fa1f0151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1f00ce230 0x7fa1f0144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 --2- 172.21.15.148:0/3732566300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1f00cc4f0 0x7fa1f01447d0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 >> 172.21.15.148:0/3732566300 conn(0x7fa1f006abe0 msgr2=0x7fa1f015c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 shutdown_connections 2024-04-25T10:02:13.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.455+0000 7fa1f4aee640 1 -- 172.21.15.148:0/3732566300 wait complete. 2024-04-25T10:02:13.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T10:02:13.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 Processor -- start 2024-04-25T10:02:13.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 -- start start 2024-04-25T10:02:13.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a84161710 0x7f1a840d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8415e000 con 0x7f1a8415dc20 2024-04-25T10:02:13.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a840d1de0 con 0x7f1a84161710 2024-04-25T10:02:13.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.581+0000 7f1a89276640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84161c60 con 0x7f1a840ced80 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a84161710 0x7f1a840d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a84161710 0x7f1a840d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51824/0 (socket says 172.21.15.148:51824) 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 -- 172.21.15.148:0/1029203316 learned_addr learned my addr 172.21.15.148:0/1029203316 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 -- 172.21.15.148:0/1029203316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 msgr2=0x7f1a840d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a827fc640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 -- 172.21.15.148:0/1029203316 >> v1:172.21.15.188:6789/0 conn(0x7f1a8415dc20 legacy=0x7f1a840ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a837fe640 1 -- 172.21.15.148:0/1029203316 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84161f40 con 0x7f1a84161710 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.582+0000 7f1a827fc640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a837fe640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a84161710 0x7f1a840d2820 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009e50 tx=0x7f1a7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74be896a3f66f848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a81ffb640 1 -- 172.21.15.148:0/1029203316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c036070 con 0x7f1a84161710 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a81ffb640 1 -- 172.21.15.148:0/1029203316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c031600 con 0x7f1a84161710 2024-04-25T10:02:13.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a81ffb640 1 -- 172.21.15.148:0/1029203316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0319d0 con 0x7f1a84161710 2024-04-25T10:02:13.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 -- 172.21.15.148:0/1029203316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a84161710 msgr2=0x7f1a840d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a84161710 0x7f1a840d2820 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009e50 tx=0x7f1a7c02a710 comp rx=0 tx=0).stop 2024-04-25T10:02:13.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 -- 172.21.15.148:0/1029203316 shutdown_connections 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a84161710 0x7f1a840d2820 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 --2- 172.21.15.148:0/1029203316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 -- 172.21.15.148:0/1029203316 >> 172.21.15.148:0/1029203316 conn(0x7f1a840d0420 msgr2=0x7f1a840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 -- 172.21.15.148:0/1029203316 shutdown_connections 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.583+0000 7f1a89276640 1 -- 172.21.15.148:0/1029203316 wait complete. 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 Processor -- start 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 -- start start 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 0x7f1a840c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a84161710 0x7f1a840c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840b5e50 con 0x7f1a8415dc20 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84162500 con 0x7f1a84161710 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a89276640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840b5fd0 con 0x7f1a840ced80 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 0x7f1a840c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 0x7f1a840c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51838/0 (socket says 172.21.15.148:51838) 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a827fc640 1 -- 172.21.15.148:0/4082665647 learned_addr learned my addr 172.21.15.148:0/4082665647 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.585+0000 7f1a837fe640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a84161710 0x7f1a840c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a827fc640 1 -- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 msgr2=0x7f1a840d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a827fc640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a827fc640 1 -- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a84161710 msgr2=0x7f1a840c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a827fc640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a84161710 0x7f1a840c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a827fc640 1 -- 172.21.15.148:0/4082665647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a841e9c40 con 0x7f1a8415dc20 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a827fc640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 0x7f1a840c8ff0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800c7f0 tx=0x7f1a7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a6b7fe640 1 -- 172.21.15.148:0/4082665647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78015070 con 0x7f1a8415dc20 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a6b7fe640 1 -- 172.21.15.148:0/4082665647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a78007820 con 0x7f1a8415dc20 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a841497c0 con 0x7f1a8415dc20 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a6b7fe640 1 -- 172.21.15.148:0/4082665647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78007bd0 con 0x7f1a8415dc20 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.586+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a84149d80 con 0x7f1a8415dc20 2024-04-25T10:02:13.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.588+0000 7f1a6b7fe640 1 -- 172.21.15.148:0/4082665647 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a78018070 con 0x7f1a8415dc20 2024-04-25T10:02:13.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.588+0000 7f1a6b7fe640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a7006b4d0 0x7f1a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.588+0000 7f1a6b7fe640 1 -- 172.21.15.148:0/4082665647 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a7808a450 con 0x7f1a8415dc20 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.588+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a7006b4d0 0x7f1a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.589+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a7006b4d0 0x7f1a7006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1a8415c870 tx=0x7f1a74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.589+0000 7f1a8bf7d740 1 Processor -- start 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.589+0000 7f1a8bf7d740 1 -- start start 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf612ca0 0x5639cf614e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 0x5639cf613080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cf575b90 con 0x5639cf64f370 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cf51ee40 con 0x5639cf612ca0 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639cf571c30 con 0x5639cf63d760 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 0x5639cf613080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 0x5639cf613080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51854/0 (socket says 172.21.15.148:51854) 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 -- 172.21.15.148:0/179410711 learned_addr learned my addr 172.21.15.148:0/179410711 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf612ca0 0x5639cf614e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a837fe640 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 -- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 msgr2=0x5639cf63dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 -- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf612ca0 msgr2=0x5639cf614e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf612ca0 0x5639cf614e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 -- 172.21.15.148:0/179410711 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639cf3547d0 con 0x5639cf64f370 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.590+0000 7f1a827fc640 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 0x5639cf613080 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1a78088760 tx=0x7f1a7805cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904474123b3122ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a68ff9640 1 -- 172.21.15.148:0/179410711 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7801f030 con 0x5639cf64f370 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a68ff9640 1 -- 172.21.15.148:0/179410711 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a78056df0 con 0x5639cf64f370 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a68ff9640 1 -- 172.21.15.148:0/179410711 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7808a0b0 con 0x5639cf64f370 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/179410711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 msgr2=0x5639cf613080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 0x5639cf613080 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1a78088760 tx=0x7f1a7805cb90 comp rx=0 tx=0).stop 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/179410711 shutdown_connections 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf64f370 0x5639cf613080 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/179410711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf612ca0 0x5639cf614e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/179410711 >> 172.21.15.148:0/179410711 conn(0x5639cf5da870 msgr2=0x5639cf658760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/179410711 shutdown_connections 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.591+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/179410711 wait complete. 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.592+0000 7f1a8bf7d740 1 -- start start 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.592+0000 7f1a8bf7d740 1 Processor -- start 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf640f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf652c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cf5323e0 con 0x5639cf612ca0 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cf63b2b0 con 0x5639cf64f370 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639cf6282d0 con 0x5639cf63d760 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a8bf7d740 10 client.0 _ll_register_callbacks cb 0x5639cf531db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf640f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf640f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51862/0 (socket says 172.21.15.148:51862) 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 -- 172.21.15.148:0/718955350 learned_addr learned my addr 172.21.15.148:0/718955350 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a827fc640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a837fe640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf652c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 -- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 msgr2=0x5639cf63b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 -- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 msgr2=0x5639cf652c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf652c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 -- 172.21.15.148:0/718955350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639cf24a0f0 con 0x5639cf612ca0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf640f20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f1a74014700 tx=0x7f1a74000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d79c2335393e3de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a61ffb640 1 -- 172.21.15.148:0/718955350 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74025020 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.593+0000 7f1a61ffb640 1 -- 172.21.15.148:0/718955350 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a74013680 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a78056df0 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a61ffb640 1 -- 172.21.15.148:0/718955350 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7402f050 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 10 client.8919 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5639cf3b4f60 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a61ffb640 1 -- 172.21.15.148:0/718955350 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a78056df0 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a61ffb640 1 -- 172.21.15.148:0/718955350 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a74032020 con 0x5639cf612ca0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 10 client.8919 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 10 client.8919 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 20 client.8919 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf66b8b0 0x5639cf66bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 4 client.8919 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639cf5830d0 con 0x5639cf66b8b0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a827fc640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf66b8b0 0x5639cf66bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a61ffb640 10 client.8919 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.594+0000 7f1a827fc640 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf66b8b0 0x5639cf66bc90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26cb2f53302e49f server_cookie=c2bfff960b4cfb9d in_seq=0 out_seq=0 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a61ffb640 1 -- 172.21.15.148:0/718955350 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639cf5830d0 con 0x5639cf66b8b0 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a61ffb640 10 client.8919 handle_command_reply: tid=0 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 client.8919 shutdown 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 msgr2=0x5639cf640f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf640f20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f1a74014700 tx=0x7f1a74000ed0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 shutdown_connections 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf66b8b0 0x5639cf66bc90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf652c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a61ffb640 0 client.8919 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf63b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/718955350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf640f20 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 >> 172.21.15.148:0/718955350 conn(0x5639cf5da870 msgr2=0x5639cf65bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 shutdown_connections 2024-04-25T10:02:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/718955350 wait complete. 2024-04-25T10:02:13.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.595+0000 7f1a8bf7d740 20 client.8919 trim_cache size 0 max 16384 2024-04-25T10:02:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 Processor -- start 2024-04-25T10:02:13.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 -- start start 2024-04-25T10:02:13.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf612ca0 0x5639cf641110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 0x5639cf638560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf638aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84200600 con 0x5639cf63d760 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841ff7c0 con 0x5639cf64f370 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8414b4a0 con 0x5639cf612ca0 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.610+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 0x5639cf638560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 0x5639cf638560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51864/0 (socket says 172.21.15.148:51864) 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 -- 172.21.15.148:0/2319593701 learned_addr learned my addr 172.21.15.148:0/2319593701 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 -- 172.21.15.148:0/2319593701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf612ca0 msgr2=0x5639cf641110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf612ca0 0x5639cf641110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 -- 172.21.15.148:0/2319593701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 msgr2=0x5639cf638aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf638aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 -- 172.21.15.148:0/2319593701 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639cf2231b0 con 0x5639cf63d760 2024-04-25T10:02:13.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a827fc640 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 0x5639cf638560 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1a7808a770 tx=0x7f1a78012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcbb4afae590f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a63fff640 1 -- 172.21.15.148:0/2319593701 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78088070 con 0x5639cf63d760 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a63fff640 1 -- 172.21.15.148:0/2319593701 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7808a0b0 con 0x5639cf63d760 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a63fff640 1 -- 172.21.15.148:0/2319593701 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7801f040 con 0x5639cf63d760 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/2319593701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 msgr2=0x5639cf638560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.611+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 0x5639cf638560 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1a7808a770 tx=0x7f1a78012000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/2319593701 shutdown_connections 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf638aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf63d760 0x5639cf638560 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/2319593701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf612ca0 0x5639cf641110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/2319593701 >> 172.21.15.148:0/2319593701 conn(0x5639cf5da870 msgr2=0x5639cf6540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/2319593701 shutdown_connections 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/2319593701 wait complete. 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 -- start start 2024-04-25T10:02:13.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.612+0000 7f1a8bf7d740 1 Processor -- start 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf66b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf6a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf6255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841ff1c0 con 0x5639cf612ca0 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cf60dd10 con 0x5639cf64f370 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639cf6a9270 con 0x5639cf63d760 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a8bf7d740 10 client.0 _ll_register_callbacks cb 0x7f1a740013c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf66b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf66b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51880/0 (socket says 172.21.15.148:51880) 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 -- 172.21.15.148:0/3160225631 learned_addr learned my addr 172.21.15.148:0/3160225631 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a827fc640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf6a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a837fe640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf6255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 -- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 msgr2=0x5639cf6a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf6a8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 -- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 msgr2=0x5639cf6255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf6255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 -- 172.21.15.148:0/3160225631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639cf627030 con 0x5639cf612ca0 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a82ffd640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf66b340 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1a74000fb0 tx=0x7f1a74014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7239ecbf28e5e99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a61ffb640 1 -- 172.21.15.148:0/3160225631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74038060 con 0x5639cf612ca0 2024-04-25T10:02:13.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.613+0000 7f1a61ffb640 1 -- 172.21.15.148:0/3160225631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a74021400 con 0x5639cf612ca0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a7808a0b0 con 0x5639cf612ca0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a61ffb640 1 -- 172.21.15.148:0/3160225631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7403e050 con 0x5639cf612ca0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 10 client.8925 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5639cf625af0 con 0x5639cf612ca0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a61ffb640 1 -- 172.21.15.148:0/3160225631 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7808a0b0 con 0x5639cf612ca0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a61ffb640 1 -- 172.21.15.148:0/3160225631 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a74041020 con 0x5639cf612ca0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 10 client.8925 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 10 client.8925 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 20 client.8925 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf626100 0x5639cf6518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 4 client.8925 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5639cf629cf0 con 0x5639cf626100 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a827fc640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf626100 0x5639cf6518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a827fc640 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf626100 0x5639cf6518c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391b8570173da040 server_cookie=3f70439249084438 in_seq=0 out_seq=0 2024-04-25T10:02:13.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a61ffb640 10 client.8925 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a61ffb640 1 -- 172.21.15.148:0/3160225631 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5639cf629cf0 con 0x5639cf626100 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a61ffb640 10 client.8925 handle_command_reply: tid=0 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.614+0000 7f1a8bf7d740 1 client.8925 shutdown 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 msgr2=0x5639cf66b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf66b340 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1a74000fb0 tx=0x7f1a74014000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 shutdown_connections 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x5639cf626100 0x5639cf6518c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639cf64f370 0x5639cf6255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a61ffb640 0 client.8925 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639cf63d760 0x5639cf6a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 --2- 172.21.15.148:0/3160225631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5639cf612ca0 0x5639cf66b340 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 >> 172.21.15.148:0/3160225631 conn(0x5639cf5da870 msgr2=0x5639cf60cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 shutdown_connections 2024-04-25T10:02:13.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 1 -- 172.21.15.148:0/3160225631 wait complete. 2024-04-25T10:02:13.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a8bf7d740 20 client.8925 trim_cache size 0 max 16384 2024-04-25T10:02:13.626 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:13.626 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:02:13.626 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a7006b4d0 msgr2=0x7f1a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a7006b4d0 0x7f1a7006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1a8415c870 tx=0x7f1a74009040 comp rx=0 tx=0).stop 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 msgr2=0x7f1a840c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 0x7f1a840c8ff0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800c7f0 tx=0x7f1a7800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 shutdown_connections 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a7006b4d0 0x7f1a7006d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a84161710 0x7f1a840c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a8415dc20 0x7f1a840c8ff0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 --2- 172.21.15.148:0/4082665647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a840ced80 0x7f1a840d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 >> 172.21.15.148:0/4082665647 conn(0x7f1a840d0420 msgr2=0x7f1a840b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 shutdown_connections 2024-04-25T10:02:13.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.615+0000 7f1a89276640 1 -- 172.21.15.148:0/4082665647 wait complete. 2024-04-25T10:02:13.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.631 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:02:13.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.725+0000 7f55d5777640 1 Processor -- start 2024-04-25T10:02:13.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.725+0000 7f55d5777640 1 -- start start 2024-04-25T10:02:13.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55d5777640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55d00d34c0 0x7f55d00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55d5777640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d00d2420 con 0x7f55d015dac0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55d5777640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d00d2700 con 0x7f55d00d08e0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55d5777640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d00d0d50 con 0x7f55d00d34c0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55ce7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f55d00d34c0 0x7f55d00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55ceffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f55d015dac0 0x7f55d00d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60282/0 (socket says 172.21.15.148:60282) 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55ceffd640 1 -- 172.21.15.148:0/1465230724 learned_addr learned my addr 172.21.15.148:0/1465230724 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55ce7fc640 1 -- 172.21.15.148:0/1465230724 >> v1:172.21.15.188:6789/0 conn(0x7f55d00d08e0 legacy=0x7f55d00b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55ce7fc640 1 -- 172.21.15.148:0/1465230724 >> v1:172.21.15.148:6789/0 conn(0x7f55d015dac0 legacy=0x7f55d00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.726+0000 7f55ce7fc640 1 -- 172.21.15.148:0/1465230724 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d00d29e0 con 0x7f55d00d34c0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.727+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/1465230724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d00d34c0 0x7f55d00d03a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009790 tx=0x7f55c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d8eec73ce68c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.727+0000 7f55cdffb640 1 -- 172.21.15.148:0/1465230724 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4037070 con 0x7f55d00d34c0 2024-04-25T10:02:13.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.727+0000 7f55cdffb640 1 -- 172.21.15.148:0/1465230724 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c4032440 con 0x7f55d00d34c0 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.727+0000 7f55cdffb640 1 -- 172.21.15.148:0/1465230724 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4032810 con 0x7f55d00d34c0 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 -- 172.21.15.148:0/1465230724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d00d34c0 msgr2=0x7f55d00d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 --2- 172.21.15.148:0/1465230724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d00d34c0 0x7f55d00d03a0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009790 tx=0x7f55c402a270 comp rx=0 tx=0).stop 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 -- 172.21.15.148:0/1465230724 shutdown_connections 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 --2- 172.21.15.148:0/1465230724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d00d34c0 0x7f55d00d03a0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 -- 172.21.15.148:0/1465230724 >> 172.21.15.148:0/1465230724 conn(0x7f55d0061c30 msgr2=0x7f55d006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 -- 172.21.15.148:0/1465230724 shutdown_connections 2024-04-25T10:02:13.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.728+0000 7f55d5777640 1 -- 172.21.15.148:0/1465230724 wait complete. 2024-04-25T10:02:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.729+0000 7f55d5777640 1 Processor -- start 2024-04-25T10:02:13.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.729+0000 7f55d5777640 1 -- start start 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55d5777640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55d00d08e0 0x7f55d00cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55d5777640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d00d34c0 0x7f55d0151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55d5777640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 0x7f55d00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55d5777640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d00d2100 con 0x7f55d015dac0 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55d5777640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d00d3030 con 0x7f55d00d34c0 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55d5777640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d015dea0 con 0x7f55d00d08e0 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 0x7f55d00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 0x7f55d00cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51884/0 (socket says 172.21.15.148:51884) 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 -- 172.21.15.148:0/1068660675 learned_addr learned my addr 172.21.15.148:0/1068660675 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55ceffd640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55d00d08e0 0x7f55d00cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d00d34c0 0x7f55d0151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 -- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55d00d08e0 msgr2=0x7f55d00cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55d00d08e0 0x7f55d00cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 -- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d00d34c0 msgr2=0x7f55d0151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d00d34c0 0x7f55d0151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 -- 172.21.15.148:0/1068660675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d015f580 con 0x7f55d015dac0 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.730+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 0x7f55d00cd1c0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c410 tx=0x7f55c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.731+0000 7f55b77fe640 1 -- 172.21.15.148:0/1068660675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8011790 con 0x7f55d015dac0 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.731+0000 7f55b77fe640 1 -- 172.21.15.148:0/1068660675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8017070 con 0x7f55d015dac0 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.731+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d015f860 con 0x7f55d015dac0 2024-04-25T10:02:13.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.731+0000 7f55b77fe640 1 -- 172.21.15.148:0/1068660675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8011bb0 con 0x7f55d015dac0 2024-04-25T10:02:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.731+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d00cd700 con 0x7f55d015dac0 2024-04-25T10:02:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.732+0000 7f55b77fe640 1 -- 172.21.15.148:0/1068660675 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55c801c460 con 0x7f55d015dac0 2024-04-25T10:02:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.732+0000 7f55b77fe640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55bc06b500 0x7f55bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.732+0000 7f55b77fe640 1 -- 172.21.15.148:0/1068660675 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55c8089e00 con 0x7f55d015dac0 2024-04-25T10:02:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.733+0000 7f55ceffd640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55bc06b500 0x7f55bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.733+0000 7f55ceffd640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55bc06b500 0x7f55bc06d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f55c00012e0 tx=0x7f55c0001990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.733+0000 7f55d847e740 1 Processor -- start 2024-04-25T10:02:13.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 -- start start 2024-04-25T10:02:13.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8ef0 0x55e5001cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 0x55e50019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500104da0 con 0x55e50019b770 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5000adb80 con 0x55e5001c8ef0 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e500100e40 con 0x55e5001c8b10 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 0x55e50019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.734+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 0x55e50019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51892/0 (socket says 172.21.15.148:51892) 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2560304278 learned_addr learned my addr 172.21.15.148:0/2560304278 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55ceffd640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 msgr2=0x55e5001cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8ef0 0x55e5001cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8ef0 msgr2=0x55e5001cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8ef0 0x55e5001cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2560304278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5000b7f10 con 0x55e50019b770 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55ceffd640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8ef0 0x55e5001cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 0x55e50019ef80 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f55c800a040 tx=0x7f55c805c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b841ecb87f33a106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2560304278 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8011d50 con 0x55e50019b770 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2560304278 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8054b10 con 0x55e50019b770 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.735+0000 7f55b4ff9640 1 -- 172.21.15.148:0/2560304278 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80578f0 con 0x55e50019b770 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 -- 172.21.15.148:0/2560304278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 msgr2=0x55e50019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 0x55e50019ef80 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f55c800a040 tx=0x7f55c805c680 comp rx=0 tx=0).stop 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 -- 172.21.15.148:0/2560304278 shutdown_connections 2024-04-25T10:02:13.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e50019b770 0x55e50019ef80 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8ef0 0x55e5001cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 --2- 172.21.15.148:0/2560304278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 -- 172.21.15.148:0/2560304278 >> 172.21.15.148:0/2560304278 conn(0x55e50019d9e0 msgr2=0x55e5001e6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 -- 172.21.15.148:0/2560304278 shutdown_connections 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 -- 172.21.15.148:0/2560304278 wait complete. 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 -- start start 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.736+0000 7f55d847e740 1 Processor -- start 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 0x55e50019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5001a0b90 con 0x55e5001c8ef0 2024-04-25T10:02:13.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5000a34b0 con 0x55e5001e95f0 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5001a0e70 con 0x55e5001c8b10 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55d847e740 10 client.0 _ll_register_callbacks cb 0x55e5000e3180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55ce7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 0x55e50019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55ce7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 0x55e50019ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51904/0 (socket says 172.21.15.148:51904) 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55ce7fc640 1 -- 172.21.15.148:0/3353149888 learned_addr learned my addr 172.21.15.148:0/3353149888 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55ceffd640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55ce7fc640 1 -- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 msgr2=0x55e5001c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.737+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55ce7fc640 1 -- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 msgr2=0x55e5001a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55ce7fc640 1 -- 172.21.15.148:0/3353149888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4ffee37d0 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55ceffd640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 0x55e50019ff90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f55c4035040 tx=0x7f55c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4316af7380581d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55adffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55adffb640 1 -- 172.21.15.148:0/3353149888 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c4043070 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55c8054b10 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55adffb640 1 -- 172.21.15.148:0/3353149888 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55c403c440 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55adffb640 1 -- 172.21.15.148:0/3353149888 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c404e050 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 10 client.8940 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55e4ffe193f0 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55adffb640 1 -- 172.21.15.148:0/3353149888 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8054b10 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55adffb640 1 -- 172.21.15.148:0/3353149888 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55c4051020 con 0x55e5001c8ef0 2024-04-25T10:02:13.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 10 client.8940 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 10 client.8940 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 20 client.8940 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e5001c42d0 0x55e5001c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 4 client.8940 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5001122e0 con 0x55e5001c42d0 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.738+0000 7f55ceffd640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e5001c42d0 0x55e5001c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55adffb640 10 client.8940 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55ceffd640 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e5001c42d0 0x55e5001c46b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6eeac40434f828 server_cookie=81f01cdd6e2fbfb5 in_seq=0 out_seq=0 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55adffb640 1 -- 172.21.15.148:0/3353149888 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5001122e0 con 0x55e5001c42d0 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55adffb640 10 client.8940 handle_command_reply: tid=0 2024-04-25T10:02:13.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55d847e740 1 client.8940 shutdown 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 msgr2=0x55e50019ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.739+0000 7f55d847e740 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 0x55e50019ff90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f55c4035040 tx=0x7f55c4004730 comp rx=0 tx=0).stop 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 shutdown_connections 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e5001c42d0 0x55e5001c46b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55adffb640 0 client.8940 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8ef0 0x55e50019ff90 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 --2- 172.21.15.148:0/3353149888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8b10 0x55e5001c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 >> 172.21.15.148:0/3353149888 conn(0x55e50019b770 msgr2=0x55e5001c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 shutdown_connections 2024-04-25T10:02:13.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 1 -- 172.21.15.148:0/3353149888 wait complete. 2024-04-25T10:02:13.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.740+0000 7f55d847e740 20 client.8940 trim_cache size 0 max 16384 2024-04-25T10:02:13.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 Processor -- start 2024-04-25T10:02:13.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 -- start start 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8b10 0x55e5001a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 0x55e5001cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5001cc4c0 con 0x55e5001e95f0 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500200460 con 0x55e5001c8b10 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.755+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d02006e0 con 0x55e5001c8ef0 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 0x55e5001cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 0x55e5001cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51910/0 (socket says 172.21.15.148:51910) 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2531498907 learned_addr learned my addr 172.21.15.148:0/2531498907 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55ceffd640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8b10 0x55e5001a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 msgr2=0x55e5001cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8b10 msgr2=0x55e5001a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8b10 0x55e5001a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 -- 172.21.15.148:0/2531498907 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4ffdd90f0 con 0x55e5001e95f0 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55ceffd640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8b10 0x55e5001a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 0x55e5001cb820 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f55c8058430 tx=0x7f55c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff113f828e803ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55affff640 1 -- 172.21.15.148:0/2531498907 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8056860 con 0x55e5001e95f0 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55affff640 1 -- 172.21.15.148:0/2531498907 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8054b10 con 0x55e5001e95f0 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.756+0000 7f55affff640 1 -- 172.21.15.148:0/2531498907 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80578f0 con 0x55e5001e95f0 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 -- 172.21.15.148:0/2531498907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 msgr2=0x55e5001cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 0x55e5001cb820 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f55c8058430 tx=0x7f55c8014000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 -- 172.21.15.148:0/2531498907 shutdown_connections 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001e95f0 0x55e5001cb820 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 --2- 172.21.15.148:0/2531498907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001c8b10 0x55e5001a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 -- 172.21.15.148:0/2531498907 >> 172.21.15.148:0/2531498907 conn(0x55e50019b770 msgr2=0x55e500208910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 -- 172.21.15.148:0/2531498907 shutdown_connections 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 -- 172.21.15.148:0/2531498907 wait complete. 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 -- start start 2024-04-25T10:02:13.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.757+0000 7f55d847e740 1 Processor -- start 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 0x55e5001e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d00c9dc0 con 0x55e5001c8b10 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d01ff860 con 0x55e5001e95f0 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d01ff260 con 0x55e5001c8ef0 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55d847e740 10 client.0 _ll_register_callbacks cb 0x7f55c8058270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 0x55e5001e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 0x55e5001e6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51926/0 (socket says 172.21.15.148:51926) 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 -- 172.21.15.148:0/1511949262 learned_addr learned my addr 172.21.15.148:0/1511949262 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 -- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 msgr2=0x55e5001cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 -- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 msgr2=0x55e5001cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55ceffd640 1 -- 172.21.15.148:0/1511949262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5001f5c40 con 0x55e5001c8b10 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.758+0000 7f55cf7fe640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55ceffd640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 0x55e5001e6320 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f55c00074f0 tx=0x7f55c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc02490624458aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55adffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55adffb640 1 -- 172.21.15.148:0/1511949262 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0019040 con 0x55e5001c8b10 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55adffb640 1 -- 172.21.15.148:0/1511949262 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55c0030c60 con 0x55e5001c8b10 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55c8054b10 con 0x55e5001c8b10 2024-04-25T10:02:13.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55adffb640 1 -- 172.21.15.148:0/1511949262 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c0030e60 con 0x55e5001c8b10 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 10 client.8946 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55e5001f50f0 con 0x55e5001c8b10 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55adffb640 1 -- 172.21.15.148:0/1511949262 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8054b10 con 0x55e5001c8b10 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55adffb640 1 -- 172.21.15.148:0/1511949262 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55c0037050 con 0x55e5001c8b10 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 10 client.8946 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 10 client.8946 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 20 client.8946 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e50024d920 0x55e5001cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 4 client.8946 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e50010c630 con 0x55e50024d920 2024-04-25T10:02:13.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.759+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e50024d920 0x55e5001cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55adffb640 10 client.8946 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55ce7fc640 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e50024d920 0x55e5001cb2e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b75401c57b335cd4 server_cookie=4ec5ad0a6cb1bbef in_seq=0 out_seq=0 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55adffb640 1 -- 172.21.15.148:0/1511949262 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55e50010c630 con 0x55e50024d920 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55adffb640 10 client.8946 handle_command_reply: tid=0 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 client.8946 shutdown 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 msgr2=0x55e5001e6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 0x55e5001e6320 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f55c00074f0 tx=0x7f55c0012000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 shutdown_connections 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e50024d920 0x55e5001cb2e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e5001e95f0 0x55e5001cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55adffb640 0 client.8946 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e5001c8ef0 0x55e5001cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 --2- 172.21.15.148:0/1511949262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5001c8b10 0x55e5001e6320 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 >> 172.21.15.148:0/1511949262 conn(0x55e50019b770 msgr2=0x55e500196b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 shutdown_connections 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 1 -- 172.21.15.148:0/1511949262 wait complete. 2024-04-25T10:02:13.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.760+0000 7f55d847e740 20 client.8946 trim_cache size 0 max 16384 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: "cc1db65d-4e06-43ed-a25b-2b485ad11758": { 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "cc1db65d-4e06-43ed-a25b-2b485ad11758", 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:13.775 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55bc06b500 msgr2=0x7f55bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55bc06b500 0x7f55bc06d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f55c00012e0 tx=0x7f55c0001990 comp rx=0 tx=0).stop 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 msgr2=0x7f55d00cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 0x7f55d00cd1c0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c410 tx=0x7f55c800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 shutdown_connections 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f55bc06b500 0x7f55bc06d9c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55d015dac0 0x7f55d00cd1c0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d00d34c0 0x7f55d0151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 --2- 172.21.15.148:0/1068660675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55d00d08e0 0x7f55d00cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 >> 172.21.15.148:0/1068660675 conn(0x7f55d0061c30 msgr2=0x7f55d00d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 shutdown_connections 2024-04-25T10:02:13.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.761+0000 7f55d5777640 1 -- 172.21.15.148:0/1068660675 wait complete. 2024-04-25T10:02:13.778 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.871+0000 7f01c2ab6640 1 Processor -- start 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.871+0000 7f01c2ab6640 1 -- start start 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c2ab6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dd30 0x7f01bc0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c2ab6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0d0ad0 0x7f01bc0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c2ab6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01bc0d1df0 con 0x7f01bc0b9770 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c2ab6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc0d20d0 con 0x7f01bc15dd30 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c2ab6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0d23b0 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01bbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f01bc0b9770 0x7f01bc0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60294/0 (socket says 172.21.15.148:60294) 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01bbfff640 1 -- 172.21.15.148:0/1596337290 learned_addr learned my addr 172.21.15.148:0/1596337290 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c0852640 1 --2- 172.21.15.148:0/1596337290 >> v2:172.21.15.188:3300/0 conn(0x7f01bc15dd30 0x7f01bc0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.872+0000 7f01c1053640 1 --2- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0d0ad0 0x7f01bc0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01bb7fe640 1 -- 172.21.15.148:0/1596337290 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737670434 0 0) 0x7f01bc0d1df0 con 0x7f01bc0b9770 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01bb7fe640 1 -- 172.21.15.148:0/1596337290 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a4003520 con 0x7f01bc0b9770 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01c0852640 1 -- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0d0ad0 msgr2=0x7f01bc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01c0852640 1 --2- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0d0ad0 0x7f01bc0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01c0852640 1 -- 172.21.15.148:0/1596337290 >> v1:172.21.15.148:6789/0 conn(0x7f01bc0b9770 legacy=0x7f01bc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01c0852640 1 -- 172.21.15.148:0/1596337290 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc0d2690 con 0x7f01bc15dd30 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.873+0000 7f01c0852640 1 --2- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc0b9230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0097a0 tx=0x7f01ac02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0070e725e71344a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01bb7fe640 1 -- 172.21.15.148:0/1596337290 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac037070 con 0x7f01bc15dd30 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01bb7fe640 1 -- 172.21.15.148:0/1596337290 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac02ae50 con 0x7f01bc15dd30 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01bb7fe640 1 -- 172.21.15.148:0/1596337290 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac032640 con 0x7f01bc15dd30 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 -- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 msgr2=0x7f01bc0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc0b9230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0097a0 tx=0x7f01ac02a100 comp rx=0 tx=0).stop 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 -- 172.21.15.148:0/1596337290 shutdown_connections 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0d0ad0 0x7f01bc0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/1596337290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc0b9230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 -- 172.21.15.148:0/1596337290 >> 172.21.15.148:0/1596337290 conn(0x7f01bc061c30 msgr2=0x7f01bc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 -- 172.21.15.148:0/1596337290 shutdown_connections 2024-04-25T10:02:13.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.874+0000 7f01c2ab6640 1 -- 172.21.15.148:0/1596337290 wait complete. 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.875+0000 7f01c2ab6640 1 Processor -- start 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.875+0000 7f01c2ab6640 1 -- start start 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c2ab6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0b9770 0x7f01bc0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c2ab6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 0x7f01bc1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c2ab6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c2ab6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0d1b90 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c2ab6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0d1a10 con 0x7f01bc15dd30 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c2ab6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0d2e90 con 0x7f01bc0b9770 2024-04-25T10:02:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 0x7f01bc1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 0x7f01bc1529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51936/0 (socket says 172.21.15.148:51936) 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01bbfff640 1 -- 172.21.15.148:0/801069635 learned_addr learned my addr 172.21.15.148:0/801069635 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c1053640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.876+0000 7f01c0852640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0b9770 0x7f01bc0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01bbfff640 1 -- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0b9770 msgr2=0x7f01bc0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01bbfff640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0b9770 0x7f01bc0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01bbfff640 1 -- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 msgr2=0x7f01bc160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01bbfff640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01bbfff640 1 -- 172.21.15.148:0/801069635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc1613a0 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01bbfff640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 0x7f01bc1529f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c730 tx=0x7f01b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0017070 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b0007290 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc0b4250 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0001d40 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.877+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc0b4810 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.878+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc0ca320 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.879+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01b001a070 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.879+0000 7f01b97fa640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a806b4d0 0x7f01a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.879+0000 7f01c0852640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a806b4d0 0x7f01a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.879+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f01b0089730 con 0x7f01bc0d0ad0 2024-04-25T10:02:13.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.879+0000 7f01c0852640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a806b4d0 0x7f01a806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f01ac034000 tx=0x7f01ac02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:13.883+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01bc0ca320 con 0x7f01bc0d0ad0 2024-04-25T10:02:14.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.089+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01bc073990 con 0x7f01bc0d0ad0 2024-04-25T10:02:14.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.090+0000 7f01b97fa640 1 -- 172.21.15.148:0/801069635 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f01bc1613a0 con 0x7f01bc0d0ad0 2024-04-25T10:02:14.095 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:14.095 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:14.095 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a806b4d0 msgr2=0x7f01a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a806b4d0 0x7f01a806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f01ac034000 tx=0x7f01ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 msgr2=0x7f01bc1529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 0x7f01bc1529f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f01b000c730 tx=0x7f01b000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 shutdown_connections 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f01a806b4d0 0x7f01a806d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01bc15dd30 0x7f01bc160e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01bc0d0ad0 0x7f01bc1529f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 --2- 172.21.15.148:0/801069635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01bc0b9770 0x7f01bc0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 >> 172.21.15.148:0/801069635 conn(0x7f01bc061c30 msgr2=0x7f01bc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 shutdown_connections 2024-04-25T10:02:14.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.092+0000 7f01c2ab6640 1 -- 172.21.15.148:0/801069635 wait complete. 2024-04-25T10:02:14.108 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:14.108 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:14.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.199+0000 7f6e9f0d2640 1 Processor -- start 2024-04-25T10:02:14.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e9f0d2640 1 -- start start 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e9f0d2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e9815dc70 0x7f6e980d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e9f0d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 0x7f6e980cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e9f0d2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9806c220 con 0x7f6e9809a450 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e9f0d2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e980d0c90 con 0x7f6e9815dc70 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e9f0d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980d0080 con 0x7f6e9806ce10 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e97fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6e9809a450 0x7f6e9806c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60298/0 (socket says 172.21.15.148:60298) 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.200+0000 7f6e97fff640 1 -- 172.21.15.148:0/1100450895 learned_addr learned my addr 172.21.15.148:0/1100450895 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523003750 0 0) 0x7f6e9806c220 con 0x7f6e9809a450 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e9ce6e640 1 --2- 172.21.15.148:0/1100450895 >> v2:172.21.15.188:3300/0 conn(0x7f6e9815dc70 0x7f6e980d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e88003520 con 0x7f6e9809a450 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e9d66f640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 0x7f6e980cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3817118965 0 0) 0x7f6e88003520 con 0x7f6e9809a450 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 msgr2=0x7f6e980cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 0x7f6e980cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 >> v2:172.21.15.188:3300/0 conn(0x7f6e9815dc70 msgr2=0x7f6e980d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 --2- 172.21.15.148:0/1100450895 >> v2:172.21.15.188:3300/0 conn(0x7f6e9815dc70 0x7f6e980d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980d0360 con 0x7f6e9809a450 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6e84001670 con 0x7f6e9809a450 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 >> v1:172.21.15.148:6789/0 conn(0x7f6e9809a450 legacy=0x7f6e9806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e880039b0 0x7f6e88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88004800 0x7f6e88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 0x7f6e880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980d0c90 con 0x7f6e88004800 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980d0080 con 0x7f6e880039b0 2024-04-25T10:02:14.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8800b9c0 con 0x7f6e9806ce10 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e97fff640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88004800 0x7f6e88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e9ce6e640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e880039b0 0x7f6e88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e9d66f640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 0x7f6e880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e97fff640 1 -- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 msgr2=0x7f6e880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e97fff640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9806ce10 0x7f6e880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e97fff640 1 -- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e880039b0 msgr2=0x7f6e88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e97fff640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e880039b0 0x7f6e88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.201+0000 7f6e97fff640 1 -- 172.21.15.148:0/1100450895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e88003520 con 0x7f6e88004800 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.202+0000 7f6e97fff640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88004800 0x7f6e88004c80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c2c0 tx=0x7f6e88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0d33caf3553a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.202+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84007d50 con 0x7f6e88004800 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.202+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8402e070 con 0x7f6e88004800 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.202+0000 7f6e977fe640 1 -- 172.21.15.148:0/1100450895 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84005d30 con 0x7f6e88004800 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1100450895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88004800 msgr2=0x7f6e88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1100450895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88004800 0x7f6e88004c80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c2c0 tx=0x7f6e88005830 comp rx=0 tx=0).stop 2024-04-25T10:02:14.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9d66f640 1 -- 172.21.15.148:0/1100450895 reap_dead start 2024-04-25T10:02:14.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1100450895 shutdown_connections 2024-04-25T10:02:14.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1100450895 >> 172.21.15.148:0/1100450895 conn(0x7f6e980b3270 msgr2=0x7f6e980b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1100450895 shutdown_connections 2024-04-25T10:02:14.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.203+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1100450895 wait complete. 2024-04-25T10:02:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.204+0000 7f6e9f0d2640 1 Processor -- start 2024-04-25T10:02:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.204+0000 7f6e9f0d2640 1 -- start start 2024-04-25T10:02:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9f0d2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e88004800 0x7f6e981589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9f0d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e980c6e60 0x7f6e980c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9f0d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 0x7f6e981f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9f0d2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9815e050 con 0x7f6e981f3ed0 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9f0d2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980d0920 con 0x7f6e88004800 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9f0d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980cf9c0 con 0x7f6e980c6e60 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 0x7f6e981f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 0x7f6e981f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51950/0 (socket says 172.21.15.148:51950) 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 -- 172.21.15.148:0/1069045645 learned_addr learned my addr 172.21.15.148:0/1069045645 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9ce6e640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e88004800 0x7f6e981589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e97fff640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e980c6e60 0x7f6e980c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 -- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e980c6e60 msgr2=0x7f6e980c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e980c6e60 0x7f6e980c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 -- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e88004800 msgr2=0x7f6e981589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e88004800 0x7f6e981589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 -- 172.21.15.148:0/1069045645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980c9ad0 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9ce6e640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e88004800 0x7f6e981589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.205+0000 7f6e9d66f640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 0x7f6e981f4350 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6e90011bf0 tx=0x7f6e9000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.206+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000fdb0 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.206+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e90015430 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.206+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e980c9db0 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.206+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90015820 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.206+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e980c6150 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.207+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e98073990 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.207+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e900159c0 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.207+0000 7f6e957fa640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7c06b500 0x7f6e7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.208+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e9008b070 con 0x7f6e981f3ed0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.208+0000 7f6e9ce6e640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7c06b500 0x7f6e7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.208+0000 7f6e9ce6e640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7c06b500 0x7f6e7c06d9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009bc0 tx=0x7f6e8c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.212+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e90019030 con 0x7f6e981f3ed0 2024-04-25T10:02:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.450+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6e980cbba0 con 0x7f6e981f3ed0 2024-04-25T10:02:14.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.450+0000 7f6e957fa640 1 -- 172.21.15.148:0/1069045645 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f6e90010040 con 0x7f6e981f3ed0 2024-04-25T10:02:14.455 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:14.455 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":495,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":702,"data_bytes":712569707,"bytes_used":1727717376,"bytes_avail":771366395904,"bytes_total":773094113280,"write_bytes_sec":59325256,"read_op_per_sec":0,"write_op_per_sec":57},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:01:43.111276+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7c06b500 msgr2=0x7f6e7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7c06b500 0x7f6e7c06d9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009bc0 tx=0x7f6e8c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 msgr2=0x7f6e981f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 0x7f6e981f4350 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6e90011bf0 tx=0x7f6e9000f580 comp rx=0 tx=0).stop 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 shutdown_connections 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7c06b500 0x7f6e7c06d9c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e981f3ed0 0x7f6e981f4350 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e980c6e60 0x7f6e980c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 --2- 172.21.15.148:0/1069045645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e88004800 0x7f6e981589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 >> 172.21.15.148:0/1069045645 conn(0x7f6e980b3270 msgr2=0x7f6e980b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 shutdown_connections 2024-04-25T10:02:14.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:14.452+0000 7f6e9f0d2640 1 -- 172.21.15.148:0/1069045645 wait complete. 2024-04-25T10:02:14.468 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:02:15.469 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:15.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.574+0000 7fb7746ea640 1 Processor -- start 2024-04-25T10:02:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.574+0000 7fb7746ea640 1 -- start start 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb7746ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d13f0 0x7fb76c0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb7746ea640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb76c0d1d90 0x7fb76c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb7746ea640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76c0d0350 con 0x7fb76c15bd30 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb7746ea640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76c0d0630 con 0x7fb76c0d1d90 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb7746ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0d0910 con 0x7fb76c0d13f0 2024-04-25T10:02:15.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb772486640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb76c15bd30 0x7fb76c0cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60312/0 (socket says 172.21.15.148:60312) 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb772486640 1 -- 172.21.15.148:0/14310866 learned_addr learned my addr 172.21.15.148:0/14310866 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb772c87640 1 --2- 172.21.15.148:0/14310866 >> v2:172.21.15.188:3300/0 conn(0x7fb76c0d1d90 0x7fb76c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb771c85640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d13f0 0x7fb76c0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb771484640 1 -- 172.21.15.148:0/14310866 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498558769 0 0) 0x7fb76c0d0350 con 0x7fb76c15bd30 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.575+0000 7fb771484640 1 -- 172.21.15.148:0/14310866 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb758003520 con 0x7fb76c15bd30 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb771484640 1 -- 172.21.15.148:0/14310866 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1924397232 0 0) 0x7fb758003520 con 0x7fb76c15bd30 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb772c87640 1 -- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d13f0 msgr2=0x7fb76c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb772c87640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d13f0 0x7fb76c0d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb772c87640 1 -- 172.21.15.148:0/14310866 >> v1:172.21.15.148:6789/0 conn(0x7fb76c15bd30 legacy=0x7fb76c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb772c87640 1 -- 172.21.15.148:0/14310866 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0d0bf0 con 0x7fb76c0d1d90 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb771c85640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d13f0 0x7fb76c0d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.576+0000 7fb772c87640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c0d1d90 0x7fb76c0b4a30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7640097a0 tx=0x7fb76402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1559bf61191c0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb771484640 1 -- 172.21.15.148:0/14310866 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb764037070 con 0x7fb76c0d1d90 2024-04-25T10:02:15.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb771484640 1 -- 172.21.15.148:0/14310866 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76402ac30 con 0x7fb76c0d1d90 2024-04-25T10:02:15.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb771484640 1 -- 172.21.15.148:0/14310866 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680019f0 con 0x7fb76c0d1d90 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 -- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c0d1d90 msgr2=0x7fb76c0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c0d1d90 0x7fb76c0b4a30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7640097a0 tx=0x7fb76402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 -- 172.21.15.148:0/14310866 shutdown_connections 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c0d1d90 0x7fb76c0b4a30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 --2- 172.21.15.148:0/14310866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d13f0 0x7fb76c0d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 -- 172.21.15.148:0/14310866 >> 172.21.15.148:0/14310866 conn(0x7fb76c06f590 msgr2=0x7fb76c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 -- 172.21.15.148:0/14310866 shutdown_connections 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.577+0000 7fb7746ea640 1 -- 172.21.15.148:0/14310866 wait complete. 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.578+0000 7fb7746ea640 1 Processor -- start 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 -- start start 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 0x7fb76c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d1d90 0x7fb76c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c15bd30 0x7fb76c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0d0010 con 0x7fb76c0d13f0 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c15c110 con 0x7fb76c15bd30 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb7746ea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0b73b0 con 0x7fb76c0d1d90 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb772486640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 0x7fb76c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb772486640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 0x7fb76c14cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51956/0 (socket says 172.21.15.148:51956) 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.579+0000 7fb772486640 1 -- 172.21.15.148:0/2010786514 learned_addr learned my addr 172.21.15.148:0/2010786514 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb771c85640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d1d90 0x7fb76c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772c87640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c15bd30 0x7fb76c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772486640 1 -- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d1d90 msgr2=0x7fb76c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772486640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d1d90 0x7fb76c0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772486640 1 -- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c15bd30 msgr2=0x7fb76c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772486640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c15bd30 0x7fb76c0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772486640 1 -- 172.21.15.148:0/2010786514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0acab0 con 0x7fb76c0d13f0 2024-04-25T10:02:15.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb771c85640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d1d90 0x7fb76c0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772c87640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c15bd30 0x7fb76c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb772486640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 0x7fb76c14cf60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb76800cc70 tx=0x7fb76800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768017070 con 0x7fb76c0d13f0 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768012400 con 0x7fb76c0d13f0 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680127f0 con 0x7fb76c0d13f0 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c0acd90 con 0x7fb76c0d13f0 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.580+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c0ad350 con 0x7fb76c0d13f0 2024-04-25T10:02:15.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.581+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76c073990 con 0x7fb76c0d13f0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.582+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb76801a070 con 0x7fb76c0d13f0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.582+0000 7fb762ffd640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb75006b4d0 0x7fb75006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.582+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb76808a1c0 con 0x7fb76c0d13f0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.582+0000 7fb771c85640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb75006b4d0 0x7fb75006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.583+0000 7fb771c85640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb75006b4d0 0x7fb75006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0b15e0 tx=0x7fb75c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.586+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb768058860 con 0x7fb76c0d13f0 2024-04-25T10:02:15.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.792+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb76c0b6b00 con 0x7fb76c0d13f0 2024-04-25T10:02:15.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.793+0000 7fb762ffd640 1 -- 172.21.15.148:0/2010786514 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb76805c900 con 0x7fb76c0d13f0 2024-04-25T10:02:15.797 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:15.798 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:15.798 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.794+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb75006b4d0 msgr2=0x7fb75006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.794+0000 7fb7746ea640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb75006b4d0 0x7fb75006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0b15e0 tx=0x7fb75c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:15.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.794+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 msgr2=0x7fb76c14cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.794+0000 7fb7746ea640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 0x7fb76c14cf60 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fb76800cc70 tx=0x7fb76800a430 comp rx=0 tx=0).stop 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 shutdown_connections 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb75006b4d0 0x7fb75006d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb76c15bd30 0x7fb76c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb76c0d1d90 0x7fb76c0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 --2- 172.21.15.148:0/2010786514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb76c0d13f0 0x7fb76c14cf60 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 >> 172.21.15.148:0/2010786514 conn(0x7fb76c06f590 msgr2=0x7fb76c0cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 shutdown_connections 2024-04-25T10:02:15.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.795+0000 7fb7746ea640 1 -- 172.21.15.148:0/2010786514 wait complete. 2024-04-25T10:02:15.810 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:15.810 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:15.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.901+0000 7f4a6cb09640 1 Processor -- start 2024-04-25T10:02:15.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.902+0000 7f4a6cb09640 1 -- start start 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.902+0000 7f4a6cb09640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a68144750 0x7f4a6806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6cb09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 0x7f4a6806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6cb09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a68144b30 con 0x7f4a6806eb10 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6cb09640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a680712d0 con 0x7f4a68144750 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6cb09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680715b0 con 0x7f4a680cf030 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a65d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4a6806eb10 0x7f4a680ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60326/0 (socket says 172.21.15.148:60326) 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a65d9b640 1 -- 172.21.15.148:0/1464356863 learned_addr learned my addr 172.21.15.148:0/1464356863 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6659c640 1 --2- 172.21.15.148:0/1464356863 >> v2:172.21.15.188:3300/0 conn(0x7f4a68144750 0x7f4a6806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321150981 0 0) 0x7f4a68144b30 con 0x7f4a6806eb10 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 0x7f4a6806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a50003520 con 0x7f4a6806eb10 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1074688844 0 0) 0x7f4a50003520 con 0x7f4a6806eb10 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 msgr2=0x7f4a6806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 0x7f4a6806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 >> v2:172.21.15.188:3300/0 conn(0x7f4a68144750 msgr2=0x7f4a6806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 --2- 172.21.15.148:0/1464356863 >> v2:172.21.15.188:3300/0 conn(0x7f4a68144750 0x7f4a6806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68071890 con 0x7f4a6806eb10 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4a58001670 con 0x7f4a6806eb10 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 >> v1:172.21.15.148:6789/0 conn(0x7f4a6806eb10 legacy=0x7f4a680ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 0x7f4a500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50003f30 0x7f4a50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a50004d40 0x7f4a500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680712d0 con 0x7f4a50004d40 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680715b0 con 0x7f4a50003f30 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5000ba10 con 0x7f4a680cf030 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.903+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a50004d40 0x7f4a500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a6659c640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 0x7f4a500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a65d9b640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50003f30 0x7f4a50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 msgr2=0x7f4a500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a680cf030 0x7f4a500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50003f30 msgr2=0x7f4a50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50003f30 0x7f4a50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1464356863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50003520 con 0x7f4a50004d40 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.904+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a50004d40 0x7f4a500051c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e170 tx=0x7f4a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e64a636ca53309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60034020 con 0x7f4a50004d40 2024-04-25T10:02:15.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a600305f0 con 0x7f4a50004d40 2024-04-25T10:02:15.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6559a640 1 -- 172.21.15.148:0/1464356863 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60009c10 con 0x7f4a50004d40 2024-04-25T10:02:15.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1464356863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a50004d40 msgr2=0x7f4a500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1464356863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a50004d40 0x7f4a500051c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e170 tx=0x7f4a50005830 comp rx=0 tx=0).stop 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1464356863 reap_dead start 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1464356863 shutdown_connections 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1464356863 >> 172.21.15.148:0/1464356863 conn(0x7f4a6806abe0 msgr2=0x7f4a6806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1464356863 shutdown_connections 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.905+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1464356863 wait complete. 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.906+0000 7f4a6cb09640 1 Processor -- start 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.906+0000 7f4a6cb09640 1 -- start start 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a6cb09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50004d40 0x7f4a680ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a6cb09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a6813cac0 0x7f4a6813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a6cb09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 0x7f4a6814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a6cb09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a681455d0 con 0x7f4a6814eef0 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a6cb09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68145450 con 0x7f4a50004d40 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a6cb09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6815c000 con 0x7f4a6813cac0 2024-04-25T10:02:15.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a66d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 0x7f4a6814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a66d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 0x7f4a6814f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51980/0 (socket says 172.21.15.148:51980) 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.907+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1593413977 learned_addr learned my addr 172.21.15.148:0/1593413977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a6813cac0 msgr2=0x7f4a6813cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a6813cac0 0x7f4a6813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a6659c640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50004d40 0x7f4a680ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50004d40 msgr2=0x7f4a680ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50004d40 0x7f4a680ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a66d9d640 1 -- 172.21.15.148:0/1593413977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6813ba20 con 0x7f4a6814eef0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a66d9d640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 0x7f4a6814f370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e170 tx=0x7f4a60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60034070 con 0x7f4a6814eef0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a60003990 con 0x7f4a6814eef0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60009c10 con 0x7f4a6814eef0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6813bd00 con 0x7f4a6814eef0 2024-04-25T10:02:15.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.908+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6813c2c0 con 0x7f4a6814eef0 2024-04-25T10:02:15.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.909+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a68073990 con 0x7f4a6814eef0 2024-04-25T10:02:15.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.909+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a60037020 con 0x7f4a6814eef0 2024-04-25T10:02:15.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.910+0000 7f4a56ffd640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a4806b500 0x7f4a4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.910+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a600aa730 con 0x7f4a6814eef0 2024-04-25T10:02:15.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.910+0000 7f4a6659c640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a4806b500 0x7f4a4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.910+0000 7f4a6659c640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a4806b500 0x7f4a4806d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009a60 tx=0x7f4a5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:15.914+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a60079400 con 0x7f4a6814eef0 2024-04-25T10:02:16.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.156+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4a680d0510 con 0x7f4a6814eef0 2024-04-25T10:02:16.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.157+0000 7f4a56ffd640 1 -- 172.21.15.148:0/1593413977 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f4a60078d00 con 0x7f4a6814eef0 2024-04-25T10:02:16.161 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:16.162 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":764,"data_bytes":777581419,"bytes_used":1905061888,"bytes_avail":771189051392,"bytes_total":773094113280,"write_bytes_sec":64742869,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:01:43.111276+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a4806b500 msgr2=0x7f4a4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a4806b500 0x7f4a4806d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009a60 tx=0x7f4a5c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 msgr2=0x7f4a6814f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 0x7f4a6814f370 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000e170 tx=0x7f4a60009750 comp rx=0 tx=0).stop 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 shutdown_connections 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4a4806b500 0x7f4a4806d9c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a6814eef0 0x7f4a6814f370 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a6813cac0 0x7f4a6813cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 --2- 172.21.15.148:0/1593413977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a50004d40 0x7f4a680ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 >> 172.21.15.148:0/1593413977 conn(0x7f4a6806abe0 msgr2=0x7f4a68140c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 shutdown_connections 2024-04-25T10:02:16.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:16.159+0000 7f4a6cb09640 1 -- 172.21.15.148:0/1593413977 wait complete. 2024-04-25T10:02:16.174 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:02:17.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:17.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.279+0000 7fe567d22640 1 Processor -- start 2024-04-25T10:02:17.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.280+0000 7fe567d22640 1 -- start start 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.280+0000 7fe567d22640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5600aecf0 0x7fe5600cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe567d22640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5600b2650 0x7fe5600cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe567d22640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5600cf330 con 0x7fe560165680 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe567d22640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe560072340 con 0x7fe5600b2650 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe567d22640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe560071670 con 0x7fe5600aecf0 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5600b2650 0x7fe5600cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe5600b2650 0x7fe5600cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51994/0 (socket says 172.21.15.148:51994) 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 -- 172.21.15.148:0/3242784687 learned_addr learned my addr 172.21.15.148:0/3242784687 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:17.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 -- 172.21.15.148:0/3242784687 >> v2:172.21.15.188:3300/0 conn(0x7fe5600aecf0 msgr2=0x7fe5600cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5652bd640 1 --2- 172.21.15.148:0/3242784687 >> v2:172.21.15.188:3300/0 conn(0x7fe5600aecf0 0x7fe5600cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 --2- 172.21.15.148:0/3242784687 >> v2:172.21.15.188:3300/0 conn(0x7fe5600aecf0 0x7fe5600cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 -- 172.21.15.148:0/3242784687 >> v1:172.21.15.188:6789/0 conn(0x7fe560165680 legacy=0x7fe5600cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5662bf640 1 -- 172.21.15.148:0/3242784687 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560071d80 con 0x7fe5600b2650 2024-04-25T10:02:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.281+0000 7fe5652bd640 1 --2- 172.21.15.148:0/3242784687 >> v2:172.21.15.188:3300/0 conn(0x7fe5600aecf0 0x7fe5600cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe5662bf640 1 --2- 172.21.15.148:0/3242784687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5600b2650 0x7fe5600cf580 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe5600724c0 tx=0x7fe55802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b6eedb611e1833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe564abc640 1 -- 172.21.15.148:0/3242784687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558036070 con 0x7fe5600b2650 2024-04-25T10:02:17.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe564abc640 1 -- 172.21.15.148:0/3242784687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558031600 con 0x7fe5600b2650 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe564abc640 1 -- 172.21.15.148:0/3242784687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580319d0 con 0x7fe5600b2650 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe567d22640 1 -- 172.21.15.148:0/3242784687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5600b2650 msgr2=0x7fe5600cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe567d22640 1 --2- 172.21.15.148:0/3242784687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5600b2650 0x7fe5600cf580 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe5600724c0 tx=0x7fe55802a640 comp rx=0 tx=0).stop 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe567d22640 1 -- 172.21.15.148:0/3242784687 shutdown_connections 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe567d22640 1 --2- 172.21.15.148:0/3242784687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5600b2650 0x7fe5600cf580 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe567d22640 1 --2- 172.21.15.148:0/3242784687 >> v2:172.21.15.188:3300/0 conn(0x7fe5600aecf0 0x7fe5600cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.282+0000 7fe567d22640 1 -- 172.21.15.148:0/3242784687 >> 172.21.15.148:0/3242784687 conn(0x7fe56006abe0 msgr2=0x7fe56006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.283+0000 7fe567d22640 1 -- 172.21.15.148:0/3242784687 shutdown_connections 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.283+0000 7fe567d22640 1 -- 172.21.15.148:0/3242784687 wait complete. 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.284+0000 7fe567d22640 1 Processor -- start 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.284+0000 7fe567d22640 1 -- start start 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.284+0000 7fe567d22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600aecf0 0x7fe5601509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.284+0000 7fe567d22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5600b2650 0x7fe560151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.284+0000 7fe567d22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 0x7fe560152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.284+0000 7fe567d22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe560165a60 con 0x7fe560165680 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe567d22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56006ea30 con 0x7fe5600aecf0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe567d22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b2a60 con 0x7fe5600b2650 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 0x7fe560152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 0x7fe560152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52004/0 (socket says 172.21.15.148:52004) 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 -- 172.21.15.148:0/2019746441 learned_addr learned my addr 172.21.15.148:0/2019746441 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe565abe640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600aecf0 0x7fe5601509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5652bd640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5600b2650 0x7fe560151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 -- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5600b2650 msgr2=0x7fe560151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5600b2650 0x7fe560151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 -- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600aecf0 msgr2=0x7fe5601509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600aecf0 0x7fe5601509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 -- 172.21.15.148:0/2019746441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5601fb160 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5652bd640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5600b2650 0x7fe560151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5662bf640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 0x7fe560152360 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe55802ac30 tx=0x7fe558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558036050 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558042070 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe560150f20 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558009510 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.285+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5601fb6e0 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.286+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe560073990 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.287+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe558007bd0 con 0x7fe560165680 2024-04-25T10:02:17.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.287+0000 7fe5527fc640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.287+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5580ab350 con 0x7fe560165680 2024-04-25T10:02:17.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.287+0000 7fe565abe640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.287+0000 7fe565abe640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe554006fd0 tx=0x7fe554008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.291+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe55802ae20 con 0x7fe560165680 2024-04-25T10:02:17.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.501+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe560151400 con 0x7fe560165680 2024-04-25T10:02:17.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.502+0000 7fe5527fc640 1 -- 172.21.15.148:0/2019746441 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe55807a040 con 0x7fe560165680 2024-04-25T10:02:17.507 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:17.507 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:17.507 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 msgr2=0x7fe54406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe554006fd0 tx=0x7fe554008040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 msgr2=0x7fe560152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 0x7fe560152360 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe55802ac30 tx=0x7fe558004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 shutdown_connections 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe560165680 0x7fe560152360 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5600b2650 0x7fe560151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 --2- 172.21.15.148:0/2019746441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5600aecf0 0x7fe5601509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 >> 172.21.15.148:0/2019746441 conn(0x7fe56006abe0 msgr2=0x7fe56006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 shutdown_connections 2024-04-25T10:02:17.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.504+0000 7fe567d22640 1 -- 172.21.15.148:0/2019746441 wait complete. 2024-04-25T10:02:17.519 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:17.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:17.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.611+0000 7f80913e0640 1 Processor -- start 2024-04-25T10:02:17.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.611+0000 7f80913e0640 1 -- start start 2024-04-25T10:02:17.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f80913e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f80913e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f808c0700a0 0x7f808c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f80913e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c1626e0 con 0x7f808c06abe0 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f80913e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808c0cd270 con 0x7f808c0700a0 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f80913e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0cd550 con 0x7f808c0ce330 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f808b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f808c0700a0 0x7f808c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f808a7fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f808b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f808c0700a0 0x7f808c15d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:48854/0 (socket says 172.21.15.148:48854) 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.612+0000 7f808b7fe640 1 -- 172.21.15.148:0/362645418 learned_addr learned my addr 172.21.15.148:0/362645418 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.613+0000 7f808b7fe640 1 -- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 msgr2=0x7f808c06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.613+0000 7f808b7fe640 1 --2- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c06fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.613+0000 7f808b7fe640 1 -- 172.21.15.148:0/362645418 >> v1:172.21.15.188:6789/0 conn(0x7f808c06abe0 legacy=0x7f808c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.613+0000 7f808b7fe640 1 -- 172.21.15.148:0/362645418 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0cd830 con 0x7f808c0700a0 2024-04-25T10:02:17.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f808b7fe640 1 --2- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c0700a0 0x7f808c15d370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8084009870 tx=0x7f808402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9520f11e76eab2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f8089ffb640 1 -- 172.21.15.148:0/362645418 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084037070 con 0x7f808c0700a0 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f8089ffb640 1 -- 172.21.15.148:0/362645418 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084032440 con 0x7f808c0700a0 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f8089ffb640 1 -- 172.21.15.148:0/362645418 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084032810 con 0x7f808c0700a0 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 -- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c0700a0 msgr2=0x7f808c15d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 --2- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c0700a0 0x7f808c15d370 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8084009870 tx=0x7f808402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 -- 172.21.15.148:0/362645418 shutdown_connections 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 --2- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c0700a0 0x7f808c15d370 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 --2- 172.21.15.148:0/362645418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c06fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 -- 172.21.15.148:0/362645418 >> 172.21.15.148:0/362645418 conn(0x7f808c0b2220 msgr2=0x7f808c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 -- 172.21.15.148:0/362645418 shutdown_connections 2024-04-25T10:02:17.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.614+0000 7f80913e0640 1 -- 172.21.15.148:0/362645418 wait complete. 2024-04-25T10:02:17.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 Processor -- start 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 -- start start 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06abe0 0x7f808c149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 0x7f808c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c15d8b0 con 0x7f808c0700a0 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c161780 con 0x7f808c06abe0 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f80913e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c15da30 con 0x7f808c0ce330 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f808a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 0x7f808c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f808a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 0x7f808c148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52020/0 (socket says 172.21.15.148:52020) 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.616+0000 7f808a7fc640 1 -- 172.21.15.148:0/2479993973 learned_addr learned my addr 172.21.15.148:0/2479993973 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808affd640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06abe0 0x7f808c149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808b7fe640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808a7fc640 1 -- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 msgr2=0x7f808c14acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808a7fc640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c14acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808a7fc640 1 -- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06abe0 msgr2=0x7f808c149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808a7fc640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06abe0 0x7f808c149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808a7fc640 1 -- 172.21.15.148:0/2479993973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c149c20 con 0x7f808c0700a0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808affd640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06abe0 0x7f808c149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f808a7fc640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 0x7f808c148c10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f808000e870 tx=0x7f808000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080017070 con 0x7f808c0700a0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808000c940 con 0x7f808c0700a0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c149f00 con 0x7f808c0700a0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000cd30 con 0x7f808c0700a0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.617+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c14a4c0 con 0x7f808c0700a0 2024-04-25T10:02:17.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.618+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c0cf2f0 con 0x7f808c0700a0 2024-04-25T10:02:17.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.619+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f808001a070 con 0x7f808c0700a0 2024-04-25T10:02:17.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.619+0000 7f80737fe640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f807806b500 0x7f807806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.619+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f808008a510 con 0x7f808c0700a0 2024-04-25T10:02:17.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.619+0000 7f808affd640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f807806b500 0x7f807806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.619+0000 7f808affd640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f807806b500 0x7f807806d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f807c001e00 tx=0x7f807c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.623+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8080059290 con 0x7f808c0700a0 2024-04-25T10:02:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.858+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f808c0b2ff0 con 0x7f808c0700a0 2024-04-25T10:02:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.858+0000 7f80737fe640 1 -- 172.21.15.148:0/2479993973 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7f8080058b90 con 0x7f808c0700a0 2024-04-25T10:02:17.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:17.863 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":498,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562785280,"bytes_avail":770531328000,"bytes_total":773094113280,"write_bytes_sec":87299163,"read_op_per_sec":0,"write_op_per_sec":85},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T10:01:43.111276+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:02:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f807806b500 msgr2=0x7f807806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f807806b500 0x7f807806d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f807c001e00 tx=0x7f807c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 msgr2=0x7f808c148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 0x7f808c148c10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f808000e870 tx=0x7f808000ed30 comp rx=0 tx=0).stop 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 shutdown_connections 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f807806b500 0x7f807806d9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f808c0ce330 0x7f808c14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f808c0700a0 0x7f808c148c10 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 --2- 172.21.15.148:0/2479993973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c06abe0 0x7f808c149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 >> 172.21.15.148:0/2479993973 conn(0x7f808c0b2220 msgr2=0x7f808c0adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.860+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 shutdown_connections 2024-04-25T10:02:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:17.861+0000 7f80913e0640 1 -- 172.21.15.148:0/2479993973 wait complete. 2024-04-25T10:02:17.876 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:02:18.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:18.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.981+0000 7fa42a3ce640 1 Processor -- start 2024-04-25T10:02:18.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.981+0000 7fa42a3ce640 1 -- start start 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa42a3ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 0x7fa42406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa42a3ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4240700a0 0x7fa42415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa42a3ce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4241626e0 con 0x7fa42406abe0 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa42a3ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4240cd270 con 0x7fa4240700a0 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa42a3ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4240cd550 con 0x7fa4240ce330 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa423fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa42406abe0 0x7fa42406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60342/0 (socket says 172.21.15.148:60342) 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa423fff640 1 -- 172.21.15.148:0/708682388 learned_addr learned my addr 172.21.15.148:0/708682388 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa42896b640 1 --2- 172.21.15.148:0/708682388 >> v2:172.21.15.188:3300/0 conn(0x7fa4240700a0 0x7fa42415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930518395 0 0) 0x7fa4241626e0 con 0x7fa42406abe0 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003520 con 0x7fa42406abe0 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.982+0000 7fa4237fe640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 0x7fa42406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696481223 0 0) 0x7fa40c003520 con 0x7fa42406abe0 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 msgr2=0x7fa42406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 0x7fa42406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 >> v2:172.21.15.188:3300/0 conn(0x7fa4240700a0 msgr2=0x7fa42415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 --2- 172.21.15.148:0/708682388 >> v2:172.21.15.188:3300/0 conn(0x7fa4240700a0 0x7fa42415d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4240cd830 con 0x7fa42406abe0 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa414001670 con 0x7fa42406abe0 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 >> v1:172.21.15.148:6789/0 conn(0x7fa42406abe0 legacy=0x7fa42406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c0039b0 0x7fa40c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 0x7fa40c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40c004d40 0x7fa40c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4240cd270 con 0x7fa40c004d40 2024-04-25T10:02:18.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4240cd550 con 0x7fa40c0039b0 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c00ba40 con 0x7fa4240ce330 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa42896b640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40c004d40 0x7fa40c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa423fff640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c0039b0 0x7fa40c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa4237fe640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 0x7fa40c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa42896b640 1 -- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 msgr2=0x7fa40c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa42896b640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4240ce330 0x7fa40c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa42896b640 1 -- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c0039b0 msgr2=0x7fa40c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa42896b640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c0039b0 0x7fa40c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.983+0000 7fa42896b640 1 -- 172.21.15.148:0/708682388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c003520 con 0x7fa40c004d40 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.984+0000 7fa42896b640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40c004d40 0x7fa40c0051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa42406bc10 tx=0x7fa40c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4916d89f51754c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.984+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c009c10 con 0x7fa40c004d40 2024-04-25T10:02:18.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.984+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c036070 con 0x7fa40c004d40 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.984+0000 7fa422ffd640 1 -- 172.21.15.148:0/708682388 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c031810 con 0x7fa40c004d40 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42a3ce640 1 -- 172.21.15.148:0/708682388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40c004d40 msgr2=0x7fa40c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/708682388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40c004d40 0x7fa40c0051c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa42406bc10 tx=0x7fa40c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42896b640 1 -- 172.21.15.148:0/708682388 reap_dead start 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42a3ce640 1 -- 172.21.15.148:0/708682388 shutdown_connections 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42a3ce640 1 -- 172.21.15.148:0/708682388 >> 172.21.15.148:0/708682388 conn(0x7fa4240b2220 msgr2=0x7fa4240b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42a3ce640 1 -- 172.21.15.148:0/708682388 shutdown_connections 2024-04-25T10:02:18.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.985+0000 7fa42a3ce640 1 -- 172.21.15.148:0/708682388 wait complete. 2024-04-25T10:02:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.986+0000 7fa42a3ce640 1 Processor -- start 2024-04-25T10:02:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.986+0000 7fa42a3ce640 1 -- start start 2024-04-25T10:02:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42a3ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c004d40 0x7fa4240b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42a3ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa42415a0e0 0x7fa4241f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42a3ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 0x7fa4240b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42a3ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42406fb60 con 0x7fa4241f5af0 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42a3ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa424161780 con 0x7fa42415a0e0 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42a3ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42406fce0 con 0x7fa40c004d40 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 0x7fa4240b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 0x7fa4240b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52036/0 (socket says 172.21.15.148:52036) 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 -- 172.21.15.148:0/1684927956 learned_addr learned my addr 172.21.15.148:0/1684927956 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa4237fe640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa42415a0e0 0x7fa4241f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa423fff640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c004d40 0x7fa4240b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 -- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c004d40 msgr2=0x7fa4240b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c004d40 0x7fa4240b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 -- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa42415a0e0 msgr2=0x7fa4241f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa42415a0e0 0x7fa4241f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.987+0000 7fa42896b640 1 -- 172.21.15.148:0/1684927956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4240b3470 con 0x7fa4241f5af0 2024-04-25T10:02:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.988+0000 7fa42896b640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 0x7fa4240b4510 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fa41c033040 tx=0x7fa41c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.988+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c009c10 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.988+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c045650 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.988+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4240b3750 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.988+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c045a40 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.988+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4240b3d10 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.989+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa424073990 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.989+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa41c005d10 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.989+0000 7fa420ff9640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa41006b4d0 0x7fa41006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.990+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa41c0ab250 con 0x7fa4241f5af0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.990+0000 7fa423fff640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa41006b4d0 0x7fa41006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.990+0000 7fa423fff640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa41006b4d0 0x7fa41006d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4140017e0 tx=0x7fa414001e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:18.994+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa41c02e050 con 0x7fa4241f5af0 2024-04-25T10:02:19.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.203+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4240afe50 con 0x7fa4241f5af0 2024-04-25T10:02:19.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.204+0000 7fa420ff9640 1 -- 172.21.15.148:0/1684927956 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa41c079f40 con 0x7fa4241f5af0 2024-04-25T10:02:19.209 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:19.209 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:19.209 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa41006b4d0 msgr2=0x7fa41006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa41006b4d0 0x7fa41006d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4140017e0 tx=0x7fa414001e30 comp rx=0 tx=0).stop 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 msgr2=0x7fa4240b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 0x7fa4240b4510 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fa41c033040 tx=0x7fa41c009750 comp rx=0 tx=0).stop 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 shutdown_connections 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa41006b4d0 0x7fa41006d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4241f5af0 0x7fa4240b4510 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa42415a0e0 0x7fa4241f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 --2- 172.21.15.148:0/1684927956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c004d40 0x7fa4240b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 >> 172.21.15.148:0/1684927956 conn(0x7fa4240b2220 msgr2=0x7fa42406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 shutdown_connections 2024-04-25T10:02:19.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.206+0000 7fa42a3ce640 1 -- 172.21.15.148:0/1684927956 wait complete. 2024-04-25T10:02:19.222 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:19.222 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:19.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.314+0000 7f57376cb640 1 Processor -- start 2024-04-25T10:02:19.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.314+0000 7f57376cb640 1 -- start start 2024-04-25T10:02:19.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f57376cb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57300b3410 0x7f57300720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f57376cb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57300b37f0 0x7f57300b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f57376cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730161870 0x7f5730161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f57376cb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573006df30 con 0x7f57300b3410 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f57376cb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57300ace70 con 0x7f57300b37f0 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f57376cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57300726d0 con 0x7f5730161870 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57300b3410 0x7f57300720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57300b3410 0x7f57300720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52038/0 (socket says 172.21.15.148:52038) 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 -- 172.21.15.148:0/26377355 learned_addr learned my addr 172.21.15.148:0/26377355 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735c68640 1 --2- 172.21.15.148:0/26377355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730161870 0x7f5730161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5734c66640 1 --2- 172.21.15.148:0/26377355 >> v2:172.21.15.188:3300/0 conn(0x7f57300b37f0 0x7f57300b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 -- 172.21.15.148:0/26377355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730161870 msgr2=0x7f5730161d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 --2- 172.21.15.148:0/26377355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730161870 0x7f5730161d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 -- 172.21.15.148:0/26377355 >> v2:172.21.15.188:3300/0 conn(0x7f57300b37f0 msgr2=0x7f57300b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 --2- 172.21.15.148:0/26377355 >> v2:172.21.15.188:3300/0 conn(0x7f57300b37f0 0x7f57300b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.315+0000 7f5735467640 1 -- 172.21.15.148:0/26377355 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573006d8e0 con 0x7f57300b3410 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.316+0000 7f5735467640 1 --2- 172.21.15.148:0/26377355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57300b3410 0x7f57300720f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f573006bc50 tx=0x7f572c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6ff4486c5e685d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.316+0000 7f5727fff640 1 -- 172.21.15.148:0/26377355 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c037070 con 0x7f57300b3410 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.316+0000 7f5727fff640 1 -- 172.21.15.148:0/26377355 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c032440 con 0x7f57300b3410 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.316+0000 7f5727fff640 1 -- 172.21.15.148:0/26377355 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c032810 con 0x7f57300b3410 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 -- 172.21.15.148:0/26377355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57300b3410 msgr2=0x7f57300720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 --2- 172.21.15.148:0/26377355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57300b3410 0x7f57300720f0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f573006bc50 tx=0x7f572c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 -- 172.21.15.148:0/26377355 shutdown_connections 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 --2- 172.21.15.148:0/26377355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730161870 0x7f5730161d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 --2- 172.21.15.148:0/26377355 >> v2:172.21.15.188:3300/0 conn(0x7f57300b37f0 0x7f57300b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 --2- 172.21.15.148:0/26377355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57300b3410 0x7f57300720f0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 -- 172.21.15.148:0/26377355 >> 172.21.15.148:0/26377355 conn(0x7f57300cfae0 msgr2=0x7f57300cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 -- 172.21.15.148:0/26377355 shutdown_connections 2024-04-25T10:02:19.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.317+0000 7f57376cb640 1 -- 172.21.15.148:0/26377355 wait complete. 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.318+0000 7f57376cb640 1 Processor -- start 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.318+0000 7f57376cb640 1 -- start start 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57300b3410 0x7f573013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57300b37f0 0x7f57301f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 0x7f5730146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730147230 con 0x7f5730161870 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730147890 con 0x7f57300b37f0 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57301473b0 con 0x7f57300b3410 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 0x7f5730146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 0x7f5730146440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52042/0 (socket says 172.21.15.148:52042) 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 -- 172.21.15.148:0/2148683203 learned_addr learned my addr 172.21.15.148:0/2148683203 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 -- 172.21.15.148:0/2148683203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57300b3410 msgr2=0x7f573013d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57300b3410 0x7f573013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 -- 172.21.15.148:0/2148683203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57300b37f0 msgr2=0x7f57301f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57300b37f0 0x7f57301f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 -- 172.21.15.148:0/2148683203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573013d9e0 con 0x7f5730161870 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5735c68640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 0x7f5730146440 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5728007e40 tx=0x7f572800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728014620 con 0x7f5730161870 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728019070 con 0x7f5730161870 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573013dcc0 con 0x7f5730161870 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.319+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728014a20 con 0x7f5730161870 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.320+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573013e280 con 0x7f5730161870 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.320+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5730073990 con 0x7f5730161870 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.321+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5728014bc0 con 0x7f5730161870 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.321+0000 7f5725ffb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f571406b500 0x7f571406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.321+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f572808ac60 con 0x7f5730161870 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.321+0000 7f5735467640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f571406b500 0x7f571406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.322+0000 7f5735467640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f571406b500 0x7f571406d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f572c034000 tx=0x7f572c02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.325+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5728059950 con 0x7f5730161870 2024-04-25T10:02:19.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.562+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f57300d0800 con 0x7f5730161870 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.562+0000 7f5725ffb640 1 -- 172.21.15.148:0/2148683203 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f5728059250 con 0x7f5730161870 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:19.567 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":500,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562785280,"bytes_avail":770531328000,"bytes_total":773094113280,"write_bytes_sec":77074825,"read_op_per_sec":0,"write_op_per_sec":75},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:02:19.118650+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f571406b500 msgr2=0x7f571406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f571406b500 0x7f571406d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f572c034000 tx=0x7f572c02c000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 msgr2=0x7f5730146440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 0x7f5730146440 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5728007e40 tx=0x7f572800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 shutdown_connections 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f571406b500 0x7f571406d9c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5730161870 0x7f5730146440 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57300b37f0 0x7f57301f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 --2- 172.21.15.148:0/2148683203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57300b3410 0x7f573013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 >> 172.21.15.148:0/2148683203 conn(0x7f57300cfae0 msgr2=0x7f57300716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 shutdown_connections 2024-04-25T10:02:19.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.564+0000 7f57376cb640 1 -- 172.21.15.148:0/2148683203 wait complete. 2024-04-25T10:02:19.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:02:19.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:19.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T10:02:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.673+0000 7ff56c31d640 1 Processor -- start 2024-04-25T10:02:19.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.673+0000 7ff56c31d640 1 -- start start 2024-04-25T10:02:19.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.674+0000 7ff56c31d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5641610f0 0x7ff56406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.674+0000 7ff56c31d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5641616f0 0x7ff564161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.674+0000 7ff56c31d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5640c6960 con 0x7ff5640ce0a0 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.674+0000 7ff56c31d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5641614d0 con 0x7ff5641610f0 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.674+0000 7ff56c31d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff564162260 con 0x7ff5641616f0 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5641610f0 0x7ff56406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5641610f0 0x7ff56406c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52058/0 (socket says 172.21.15.148:52058) 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 -- 172.21.15.148:0/2060274963 learned_addr learned my addr 172.21.15.148:0/2060274963 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 -- 172.21.15.148:0/2060274963 >> v2:172.21.15.188:3300/0 conn(0x7ff5641616f0 msgr2=0x7ff564161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/2060274963 >> v2:172.21.15.188:3300/0 conn(0x7ff5641616f0 0x7ff564161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 -- 172.21.15.148:0/2060274963 >> v1:172.21.15.188:6789/0 conn(0x7ff5640ce0a0 legacy=0x7ff5640d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 -- 172.21.15.148:0/2060274963 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff564162540 con 0x7ff5641610f0 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.675+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/2060274963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5641610f0 0x7ff56406c1c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff56406f220 tx=0x7ff56002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73b1431326bf3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff5690b7640 1 -- 172.21.15.148:0/2060274963 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560037070 con 0x7ff5641610f0 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff5690b7640 1 -- 172.21.15.148:0/2060274963 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff560032440 con 0x7ff5641610f0 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff5690b7640 1 -- 172.21.15.148:0/2060274963 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560032810 con 0x7ff5641610f0 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 -- 172.21.15.148:0/2060274963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5641610f0 msgr2=0x7ff56406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 --2- 172.21.15.148:0/2060274963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5641610f0 0x7ff56406c1c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ff56406f220 tx=0x7ff56002a230 comp rx=0 tx=0).stop 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 -- 172.21.15.148:0/2060274963 shutdown_connections 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 --2- 172.21.15.148:0/2060274963 >> v2:172.21.15.188:3300/0 conn(0x7ff5641616f0 0x7ff564161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 --2- 172.21.15.148:0/2060274963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5641610f0 0x7ff56406c1c0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 -- 172.21.15.148:0/2060274963 >> 172.21.15.148:0/2060274963 conn(0x7ff56406f500 msgr2=0x7ff56406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 -- 172.21.15.148:0/2060274963 shutdown_connections 2024-04-25T10:02:19.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.676+0000 7ff56c31d640 1 -- 172.21.15.148:0/2060274963 wait complete. 2024-04-25T10:02:19.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.677+0000 7ff56c31d640 1 Processor -- start 2024-04-25T10:02:19.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 -- start start 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 0x7ff56415d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5641610f0 0x7ff5640b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5641616f0 0x7ff5641f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640cf690 con 0x7ff5640ce0a0 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640cfce0 con 0x7ff5641610f0 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56c31d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406c700 con 0x7ff5641616f0 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56a0b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 0x7ff56415d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56a0b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 0x7ff56415d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52072/0 (socket says 172.21.15.148:52072) 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.678+0000 7ff56a0b9640 1 -- 172.21.15.148:0/4115258605 learned_addr learned my addr 172.21.15.148:0/4115258605 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56a0b9640 1 -- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5641616f0 msgr2=0x7ff5641f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5641616f0 0x7ff5641f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56a0b9640 1 -- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5641610f0 msgr2=0x7ff5640b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff5698b8640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5641610f0 0x7ff5640b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5641610f0 0x7ff5640b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56a0b9640 1 -- 172.21.15.148:0/4115258605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56415dd50 con 0x7ff5640ce0a0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 0x7ff56415d0f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff560000c00 tx=0x7ff5600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff55affd640 1 -- 172.21.15.148:0/4115258605 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560041070 con 0x7ff5640ce0a0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff55affd640 1 -- 172.21.15.148:0/4115258605 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56003c870 con 0x7ff5640ce0a0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5640b61e0 con 0x7ff5640ce0a0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff55affd640 1 -- 172.21.15.148:0/4115258605 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560037040 con 0x7ff5640ce0a0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.679+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5640b6f20 con 0x7ff5640ce0a0 2024-04-25T10:02:19.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.680+0000 7ff55affd640 1 -- 172.21.15.148:0/4115258605 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff560044070 con 0x7ff5640ce0a0 2024-04-25T10:02:19.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.681+0000 7ff55affd640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff54806b4d0 0x7ff54806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.681+0000 7ff55affd640 1 -- 172.21.15.148:0/4115258605 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5600aaef0 con 0x7ff5640ce0a0 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.681+0000 7ff5698b8640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff54806b4d0 0x7ff54806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.681+0000 7ff5698b8640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff54806b4d0 0x7ff54806d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff56415d630 tx=0x7ff554008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.682+0000 7ff56f01c740 1 Processor -- start 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.682+0000 7ff56f01c740 1 -- start start 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fad1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fad0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fafa760 0x56297fafabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56f01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297fa32b90 con 0x56297facfca0 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297f9dbe40 con 0x56297fb0c370 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56297fa2ec30 con 0x56297fafa760 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fad1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fad1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52088/0 (socket says 172.21.15.148:52088) 2024-04-25T10:02:19.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 -- 172.21.15.148:0/1608803114 learned_addr learned my addr 172.21.15.148:0/1608803114 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff5698b8640 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fad0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a8ba640 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fafa760 0x56297fafabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 -- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fafa760 msgr2=0x56297fafabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fafa760 0x56297fafabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 -- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 msgr2=0x56297fad0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fad0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 -- 172.21.15.148:0/1608803114 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297f8117d0 con 0x56297facfca0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fad1e00 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff560081750 tx=0x7ff560079c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f71ba17491a3333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.683+0000 7ff547fff640 1 -- 172.21.15.148:0/1608803114 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56004b040 con 0x56297facfca0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff547fff640 1 -- 172.21.15.148:0/1608803114 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff560081c80 con 0x56297facfca0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff547fff640 1 -- 172.21.15.148:0/1608803114 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560075be0 con 0x56297facfca0 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 -- 172.21.15.148:0/1608803114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 msgr2=0x56297fad1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fad1e00 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff560081750 tx=0x7ff560079c70 comp rx=0 tx=0).stop 2024-04-25T10:02:19.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 -- 172.21.15.148:0/1608803114 shutdown_connections 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fafa760 0x56297fafabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fad0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1608803114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fad1e00 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 -- 172.21.15.148:0/1608803114 >> 172.21.15.148:0/1608803114 conn(0x56297fa97870 msgr2=0x56297fb15760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 -- 172.21.15.148:0/1608803114 shutdown_connections 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.684+0000 7ff56f01c740 1 -- 172.21.15.148:0/1608803114 wait complete. 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.685+0000 7ff56f01c740 1 -- start start 2024-04-25T10:02:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.685+0000 7ff56f01c740 1 Processor -- start 2024-04-25T10:02:19.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fb11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fafa760 0x56297fb145b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297fb121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297fa32b90 con 0x56297facfca0 2024-04-25T10:02:19.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297f9ef3e0 con 0x56297fafa760 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56297fb0dfd0 con 0x56297fb0c370 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56f01c740 10 client.0 _ll_register_callbacks cb 0x56297f9eedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fb11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fb11670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52098/0 (socket says 172.21.15.148:52098) 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 -- 172.21.15.148:0/3701978915 learned_addr learned my addr 172.21.15.148:0/3701978915 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff5698b8640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fafa760 0x56297fb145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a8ba640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297fb121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 -- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 msgr2=0x56297fb121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297fb121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 -- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fafa760 msgr2=0x56297fb145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fafa760 0x56297fb145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 -- 172.21.15.148:0/3701978915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56297f7070f0 con 0x56297facfca0 2024-04-25T10:02:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fb11670 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff560079730 tx=0x7ff5600091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc462856f3082e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff5457fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff5457fa640 1 -- 172.21.15.148:0/3701978915 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56004b040 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.686+0000 7ff5457fa640 1 -- 172.21.15.148:0/3701978915 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5600799d0 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff560081c80 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff5457fa640 1 -- 172.21.15.148:0/3701978915 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff56009d050 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 10 client.9003 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56297f871f60 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff5457fa640 1 -- 172.21.15.148:0/3701978915 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff560081c80 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff5457fa640 1 -- 172.21.15.148:0/3701978915 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff5600817f0 con 0x56297facfca0 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 10 client.9003 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 10 client.9003 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 20 client.9003 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb25750 0x56297fb28bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 4 client.9003 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56297fa400d0 con 0x56297fb25750 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.687+0000 7ff5698b8640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb25750 0x56297fb28bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff5457fa640 10 client.9003 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff5698b8640 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb25750 0x56297fb28bf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da103d5658186eac server_cookie=f548e05731e16d88 in_seq=0 out_seq=0 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff5457fa640 1 -- 172.21.15.148:0/3701978915 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56297fa400d0 con 0x56297fb25750 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff5457fa640 10 client.9003 handle_command_reply: tid=0 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 client.9003 shutdown 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 msgr2=0x56297fb11670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fb11670 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff560079730 tx=0x7ff5600091c0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 shutdown_connections 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb25750 0x56297fb28bf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297fb121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff5457fa640 0 client.9003 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fafa760 0x56297fb145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3701978915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297facfca0 0x56297fb11670 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 >> 172.21.15.148:0/3701978915 conn(0x56297fa97870 msgr2=0x56297fb16890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 shutdown_connections 2024-04-25T10:02:19.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.688+0000 7ff56f01c740 1 -- 172.21.15.148:0/3701978915 wait complete. 2024-04-25T10:02:19.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.689+0000 7ff56f01c740 20 client.9003 trim_cache size 0 max 16384 2024-04-25T10:02:19.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.703+0000 7ff56f01c740 1 Processor -- start 2024-04-25T10:02:19.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.703+0000 7ff56f01c740 1 -- start start 2024-04-25T10:02:19.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297facfca0 0x56297fb0b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb0ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297faf98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56f01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff564200610 con 0x56297fafa760 2024-04-25T10:02:19.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff564160460 con 0x56297facfca0 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56415f860 con 0x56297fb0c370 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb0ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb0ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52104/0 (socket says 172.21.15.148:52104) 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 -- 172.21.15.148:0/1585077808 learned_addr learned my addr 172.21.15.148:0/1585077808 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297facfca0 0x56297fb0b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56a8ba640 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297faf98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 -- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 msgr2=0x56297faf98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297faf98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 -- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297facfca0 msgr2=0x56297fb0b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297facfca0 0x56297fb0b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 -- 172.21.15.148:0/1585077808 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297fb14920 con 0x56297fafa760 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff56a8ba640 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297faf98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5698b8640 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb0ba50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ff564160b00 tx=0x7ff554012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d41560f8402c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.704+0000 7ff5477fe640 1 -- 172.21.15.148:0/1585077808 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554027b00 con 0x56297fafa760 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff5477fe640 1 -- 172.21.15.148:0/1585077808 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554027ca0 con 0x56297fafa760 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff5477fe640 1 -- 172.21.15.148:0/1585077808 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5540139f0 con 0x56297fafa760 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 -- 172.21.15.148:0/1585077808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 msgr2=0x56297fb0ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb0ba50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7ff564160b00 tx=0x7ff554012000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 -- 172.21.15.148:0/1585077808 shutdown_connections 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297fb0c370 0x56297faf98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb0ba50 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 --2- 172.21.15.148:0/1585077808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297facfca0 0x56297fb0b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 -- 172.21.15.148:0/1585077808 >> 172.21.15.148:0/1585077808 conn(0x56297fa97870 msgr2=0x56297faca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 -- 172.21.15.148:0/1585077808 shutdown_connections 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.705+0000 7ff56f01c740 1 -- 172.21.15.148:0/1585077808 wait complete. 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.706+0000 7ff56f01c740 1 -- start start 2024-04-25T10:02:19.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.706+0000 7ff56f01c740 1 Processor -- start 2024-04-25T10:02:19.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297facfca0 0x56297fb7d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb11480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fb19280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56415fe60 con 0x56297fafa760 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297fb0eb80 con 0x56297fb0c370 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56297fb80cc0 con 0x56297facfca0 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 10 client.0 _ll_register_callbacks cb 0x7ff560090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb11480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb11480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52110/0 (socket says 172.21.15.148:52110) 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 -- 172.21.15.148:0/3283072496 learned_addr learned my addr 172.21.15.148:0/3283072496 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297facfca0 0x56297fb7d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56a8ba640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fb19280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 -- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297facfca0 msgr2=0x56297fb7d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297facfca0 0x56297fb7d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 -- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 msgr2=0x56297fb19280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fb19280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 -- 172.21.15.148:0/3283072496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56297f6e01b0 con 0x56297fafa760 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5698b8640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb11480 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ff564146b30 tx=0x7ff5540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef7677ccd79a505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5457fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5457fa640 1 -- 172.21.15.148:0/3283072496 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554013d00 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5457fa640 1 -- 172.21.15.148:0/3283072496 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5540271b0 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff554027ca0 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff5457fa640 1 -- 172.21.15.148:0/3283072496 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff55403e050 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 10 client.9009 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.707+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56297fb119c0 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff5457fa640 1 -- 172.21.15.148:0/3283072496 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554027ca0 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff5457fa640 1 -- 172.21.15.148:0/3283072496 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff554041020 con 0x56297fafa760 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 10 client.9009 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 10 client.9009 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 20 client.9009 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb260d0 0x56297fb17450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 4 client.9009 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x56297fa3a420 con 0x56297fb260d0 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb260d0 0x56297fb17450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff5457fa640 10 client.9009 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56a0b9640 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb260d0 0x56297fb17450 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5592e73aeab533 server_cookie=dd564c7fa41b1087 in_seq=0 out_seq=0 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff5457fa640 1 -- 172.21.15.148:0/3283072496 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56297fa3a420 con 0x56297fb260d0 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff5457fa640 10 client.9009 handle_command_reply: tid=0 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 1 client.9009 shutdown 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 msgr2=0x56297fb11480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.708+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb11480 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7ff564146b30 tx=0x7ff5540046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 shutdown_connections 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x56297fb260d0 0x56297fb17450 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56297fb0c370 0x56297fb19280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56297fafa760 0x56297fb11480 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 --2- 172.21.15.148:0/3283072496 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56297facfca0 0x56297fb7d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff5457fa640 0 client.9009 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 >> 172.21.15.148:0/3283072496 conn(0x56297fa97870 msgr2=0x56297fb10b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 shutdown_connections 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 1 -- 172.21.15.148:0/3283072496 wait complete. 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56f01c740 20 client.9009 trim_cache size 0 max 16384 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:02:19.720 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:19.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff54806b4d0 msgr2=0x7ff54806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff54806b4d0 0x7ff54806d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff56415d630 tx=0x7ff554008040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 msgr2=0x7ff56415d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.721 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 0x7ff56415d0f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff560000c00 tx=0x7ff5600046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 shutdown_connections 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff54806b4d0 0x7ff54806d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5641616f0 0x7ff5641f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5641610f0 0x7ff5640b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 --2- 172.21.15.148:0/4115258605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5640ce0a0 0x7ff56415d0f0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 >> 172.21.15.148:0/4115258605 conn(0x7ff56406f500 msgr2=0x7ff5640b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 shutdown_connections 2024-04-25T10:02:19.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.709+0000 7ff56c31d640 1 -- 172.21.15.148:0/4115258605 wait complete. 2024-04-25T10:02:19.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:19.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:02:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.818+0000 7f68502cb640 1 Processor -- start 2024-04-25T10:02:19.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.818+0000 7f68502cb640 1 -- start start 2024-04-25T10:02:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f68502cb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6848099cb0 0x7f68480b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f68502cb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f684815f3d0 0x7f68480ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f68502cb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68480cc5e0 con 0x7f68480cfd30 2024-04-25T10:02:19.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f68502cb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68480cc3b0 con 0x7f684815f3d0 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f68502cb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68480d0210 con 0x7f6848099cb0 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f684e868640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f684815f3d0 0x7f68480ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f684d866640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f68480cfd30 0x7f684815ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60356/0 (socket says 172.21.15.148:60356) 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.819+0000 7f684d866640 1 -- 172.21.15.148:0/1365611087 learned_addr learned my addr 172.21.15.148:0/1365611087 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684e067640 1 --2- 172.21.15.148:0/1365611087 >> v2:172.21.15.188:3300/0 conn(0x7f6848099cb0 0x7f68480b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684e868640 1 -- 172.21.15.148:0/1365611087 >> v2:172.21.15.188:3300/0 conn(0x7f6848099cb0 msgr2=0x7f68480b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684e868640 1 --2- 172.21.15.148:0/1365611087 >> v2:172.21.15.188:3300/0 conn(0x7f6848099cb0 0x7f68480b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684e868640 1 -- 172.21.15.148:0/1365611087 >> v1:172.21.15.148:6789/0 conn(0x7f68480cfd30 legacy=0x7f684815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684d065640 1 -- 172.21.15.148:0/1365611087 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450389414 0 0) 0x7f68480cc5e0 con 0x7f68480cfd30 2024-04-25T10:02:19.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684e868640 1 -- 172.21.15.148:0/1365611087 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480d04f0 con 0x7f684815f3d0 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684e868640 1 --2- 172.21.15.148:0/1365611087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f684815f3d0 0x7f68480ccda0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f68480728e0 tx=0x7f684002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f22851eaa19b98d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.820+0000 7f684d065640 1 -- 172.21.15.148:0/1365611087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840037070 con 0x7f684815f3d0 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f684d065640 1 -- 172.21.15.148:0/1365611087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840032440 con 0x7f684815f3d0 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f684d065640 1 -- 172.21.15.148:0/1365611087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840032810 con 0x7f684815f3d0 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 -- 172.21.15.148:0/1365611087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f684815f3d0 msgr2=0x7f68480ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 --2- 172.21.15.148:0/1365611087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f684815f3d0 0x7f68480ccda0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f68480728e0 tx=0x7f684002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 -- 172.21.15.148:0/1365611087 shutdown_connections 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 --2- 172.21.15.148:0/1365611087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f684815f3d0 0x7f68480ccda0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 --2- 172.21.15.148:0/1365611087 >> v2:172.21.15.188:3300/0 conn(0x7f6848099cb0 0x7f68480b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 -- 172.21.15.148:0/1365611087 >> 172.21.15.148:0/1365611087 conn(0x7f68480b3290 msgr2=0x7f68480b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 -- 172.21.15.148:0/1365611087 shutdown_connections 2024-04-25T10:02:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.821+0000 7f68502cb640 1 -- 172.21.15.148:0/1365611087 wait complete. 2024-04-25T10:02:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.822+0000 7f68502cb640 1 Processor -- start 2024-04-25T10:02:19.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.822+0000 7f68502cb640 1 -- start start 2024-04-25T10:02:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f68502cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848099cb0 0x7f68480b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f68502cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 0x7f68481f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f68502cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f684815f3d0 0x7f684806c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f68502cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480c52f0 con 0x7f68480cfd30 2024-04-25T10:02:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f68502cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480d07d0 con 0x7f6848099cb0 2024-04-25T10:02:19.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f68502cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68480c5470 con 0x7f684815f3d0 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 0x7f68481f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 0x7f68481f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52134/0 (socket says 172.21.15.148:52134) 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 -- 172.21.15.148:0/1781748568 learned_addr learned my addr 172.21.15.148:0/1781748568 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684e868640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f684815f3d0 0x7f684806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684e067640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848099cb0 0x7f68480b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 -- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f684815f3d0 msgr2=0x7f684806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f684815f3d0 0x7f684806c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 -- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848099cb0 msgr2=0x7f68480b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848099cb0 0x7f68480b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684d866640 1 -- 172.21.15.148:0/1781748568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68481f4300 con 0x7f68480cfd30 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.823+0000 7f684e067640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848099cb0 0x7f68480b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.824+0000 7f684d866640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 0x7f68481f3ab0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f683800c410 tx=0x7f683800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.824+0000 7f683effd640 1 -- 172.21.15.148:0/1781748568 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838017070 con 0x7f68480cfd30 2024-04-25T10:02:19.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.824+0000 7f683effd640 1 -- 172.21.15.148:0/1781748568 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6838005ce0 con 0x7f68480cfd30 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.824+0000 7f683effd640 1 -- 172.21.15.148:0/1781748568 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380072d0 con 0x7f68480cfd30 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.824+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68481f45e0 con 0x7f68480cfd30 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.824+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68481f3ff0 con 0x7f68480cfd30 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.825+0000 7f683effd640 1 -- 172.21.15.148:0/1781748568 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f683801a070 con 0x7f68480cfd30 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.826+0000 7f683effd640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f682c06b500 0x7f682c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.826+0000 7f683effd640 1 -- 172.21.15.148:0/1781748568 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6838089510 con 0x7f68480cfd30 2024-04-25T10:02:19.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.826+0000 7f684e067640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f682c06b500 0x7f682c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.826+0000 7f684e067640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f682c06b500 0x7f682c06d9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f684815e2c0 tx=0x7f68440039f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.827+0000 7f6852fcf740 1 Processor -- start 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.827+0000 7f6852fcf740 1 -- start start 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14b07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 0x55ad14adaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f6852fcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14a40da0 con 0x55ad14ad7770 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad149e9b80 con 0x55ad14b04ef0 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad14a3ce40 con 0x55ad14b04b10 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 0x55ad14adaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 0x55ad14adaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52150/0 (socket says 172.21.15.148:52150) 2024-04-25T10:02:19.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 -- 172.21.15.148:0/2054658307 learned_addr learned my addr 172.21.15.148:0/2054658307 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684d866640 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14b07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e067640 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 -- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 msgr2=0x55ad14b07320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b07320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 -- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 msgr2=0x55ad14b07860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14b07860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 -- 172.21.15.148:0/2054658307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad149f3f10 con 0x55ad14ad7770 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e067640 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b07320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f684e868640 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 0x55ad14adaf80 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f684002a1e0 tx=0x7f684002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe30b8bc68c599b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.828+0000 7f682bfff640 1 -- 172.21.15.148:0/2054658307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840037040 con 0x55ad14ad7770 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f682bfff640 1 -- 172.21.15.148:0/2054658307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684003e660 con 0x55ad14ad7770 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f682bfff640 1 -- 172.21.15.148:0/2054658307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684003ea10 con 0x55ad14ad7770 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 -- 172.21.15.148:0/2054658307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 msgr2=0x55ad14adaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 0x55ad14adaf80 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f684002a1e0 tx=0x7f684002c000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 -- 172.21.15.148:0/2054658307 shutdown_connections 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14ad7770 0x55ad14adaf80 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14b07860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2054658307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b07320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 -- 172.21.15.148:0/2054658307 >> 172.21.15.148:0/2054658307 conn(0x55ad14ad99e0 msgr2=0x55ad14b22320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 -- 172.21.15.148:0/2054658307 shutdown_connections 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.829+0000 7f6852fcf740 1 -- 172.21.15.148:0/2054658307 wait complete. 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.830+0000 7f6852fcf740 1 -- start start 2024-04-25T10:02:19.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.830+0000 7f6852fcf740 1 Processor -- start 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b25610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14afe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14afa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14adcb90 con 0x55ad14b04ef0 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad149df4b0 con 0x55ad14afd3e0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad14adce70 con 0x55ad14b04b10 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f6852fcf740 10 client.0 _ll_register_callbacks cb 0x55ad14a1f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14afe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14afe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52164/0 (socket says 172.21.15.148:52164) 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 -- 172.21.15.148:0/3184636617 learned_addr learned my addr 172.21.15.148:0/3184636617 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684e868640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14afa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684e067640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b25610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 -- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 msgr2=0x55ad14b25610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b25610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 -- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 msgr2=0x55ad14afa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14afa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 -- 172.21.15.148:0/3184636617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad1481f7d0 con 0x55ad14b04ef0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f684d866640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14afe5c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f6838087940 tx=0x7f683805bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b09298500e56cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.831+0000 7f68297fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f68297fa640 1 -- 172.21.15.148:0/3184636617 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838056000 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f68297fa640 1 -- 172.21.15.148:0/3184636617 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6838007470 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f684003e660 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f68297fa640 1 -- 172.21.15.148:0/3184636617 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6838007670 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 10 client.9024 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55ad147553f0 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f68297fa640 1 -- 172.21.15.148:0/3184636617 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684003e660 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f68297fa640 1 -- 172.21.15.148:0/3184636617 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f683806bb50 con 0x55ad14b04ef0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 10 client.9024 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 10 client.9024 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 20 client.9024 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b02ce0 0x55ad14adc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 4 client.9024 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad14a4e2e0 con 0x55ad14b02ce0 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.832+0000 7f684e067640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b02ce0 0x55ad14adc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f68297fa640 10 client.9024 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f684e067640 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b02ce0 0x55ad14adc460 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1cc89b0efa0067 server_cookie=53e18e642b11b18b in_seq=0 out_seq=0 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f68297fa640 1 -- 172.21.15.148:0/3184636617 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad14a4e2e0 con 0x55ad14b02ce0 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f68297fa640 10 client.9024 handle_command_reply: tid=0 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 client.9024 shutdown 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 msgr2=0x55ad14afe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14afe5c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f6838087940 tx=0x7f683805bda0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 shutdown_connections 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b02ce0 0x55ad14adc460 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14afa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14afe5c0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 --2- 172.21.15.148:0/3184636617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b25610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f68297fa640 0 client.9024 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 >> 172.21.15.148:0/3184636617 conn(0x55ad14ad7770 msgr2=0x55ad14ad8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 shutdown_connections 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 1 -- 172.21.15.148:0/3184636617 wait complete. 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.833+0000 7f6852fcf740 20 client.9024 trim_cache size 0 max 16384 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.848+0000 7f6852fcf740 1 Processor -- start 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.848+0000 7f6852fcf740 1 -- start start 2024-04-25T10:02:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14afd3e0 0x55ad14b02250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 0x55ad14afb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14adb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f6852fcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68482002f0 con 0x55ad14b04b10 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68481ff640 con 0x55ad14b04ef0 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68481f7360 con 0x55ad14afd3e0 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 0x55ad14afb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 0x55ad14afb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52178/0 (socket says 172.21.15.148:52178) 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 -- 172.21.15.148:0/2874596975 learned_addr learned my addr 172.21.15.148:0/2874596975 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684e067640 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14afd3e0 0x55ad14b02250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684e868640 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14adb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 -- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14afd3e0 msgr2=0x55ad14b02250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14afd3e0 0x55ad14b02250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 -- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 msgr2=0x55ad14adb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14adb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684d866640 1 -- 172.21.15.148:0/2874596975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad147150f0 con 0x55ad14b04b10 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.849+0000 7f684e868640 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14adb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f684d866640 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 0x55ad14afb690 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6838000ff0 tx=0x7f683805a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7552ec57d8eb1a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f682b7fe640 1 -- 172.21.15.148:0/2874596975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838056000 con 0x55ad14b04b10 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f682b7fe640 1 -- 172.21.15.148:0/2874596975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683806bb50 con 0x55ad14b04b10 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f682b7fe640 1 -- 172.21.15.148:0/2874596975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838057090 con 0x55ad14b04b10 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 -- 172.21.15.148:0/2874596975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 msgr2=0x55ad14afb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 0x55ad14afb690 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f6838000ff0 tx=0x7f683805a000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 -- 172.21.15.148:0/2874596975 shutdown_connections 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14b04ef0 0x55ad14adb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04b10 0x55ad14afb690 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 --2- 172.21.15.148:0/2874596975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14afd3e0 0x55ad14b02250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 -- 172.21.15.148:0/2874596975 >> 172.21.15.148:0/2874596975 conn(0x55ad14ad7770 msgr2=0x55ad14add7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 -- 172.21.15.148:0/2874596975 shutdown_connections 2024-04-25T10:02:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.850+0000 7f6852fcf740 1 -- 172.21.15.148:0/2874596975 wait complete. 2024-04-25T10:02:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.851+0000 7f6852fcf740 1 -- start start 2024-04-25T10:02:19.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.851+0000 7f6852fcf740 1 Processor -- start 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14ad6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14b72af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68481f7960 con 0x55ad14b04ef0 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad14b21d90 con 0x55ad14afd3e0 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad14af92f0 con 0x55ad14b04b10 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f6852fcf740 10 client.0 _ll_register_callbacks cb 0x7f6840035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14b72af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14b72af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52180/0 (socket says 172.21.15.148:52180) 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 -- 172.21.15.148:0/4074352789 learned_addr learned my addr 172.21.15.148:0/4074352789 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e067640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14ad6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684d866640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 -- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 msgr2=0x55ad14b725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 -- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 msgr2=0x55ad14ad6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14ad6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 -- 172.21.15.148:0/4074352789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad14ad6f40 con 0x55ad14b04ef0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684d866640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f684e868640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14b72af0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f684002a790 tx=0x7f6840004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcac8ed7d6fca129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f68297fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.852+0000 7f68297fa640 1 -- 172.21.15.148:0/4074352789 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840037040 con 0x55ad14b04ef0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f68297fa640 1 -- 172.21.15.148:0/4074352789 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6840003810 con 0x55ad14b04ef0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f683806bb50 con 0x55ad14b04ef0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f68297fa640 1 -- 172.21.15.148:0/4074352789 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6840055050 con 0x55ad14b04ef0 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 10 client.9030 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:19.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55ad14adb6a0 con 0x55ad14b04ef0 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f68297fa640 1 -- 172.21.15.148:0/4074352789 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683806bb50 con 0x55ad14b04ef0 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f68297fa640 1 -- 172.21.15.148:0/4074352789 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6840058020 con 0x55ad14b04ef0 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 10 client.9030 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 10 client.9030 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 20 client.9030 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b84510 0x55ad14b06ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 4 client.9030 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad14a48630 con 0x55ad14b84510 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f684e067640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b84510 0x55ad14b06ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f68297fa640 10 client.9030 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.853+0000 7f684e067640 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b84510 0x55ad14b06ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234ae5560701f45c server_cookie=3029c5004be2e859 in_seq=0 out_seq=0 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68297fa640 1 -- 172.21.15.148:0/4074352789 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55ad14a48630 con 0x55ad14b84510 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68297fa640 10 client.9030 handle_command_reply: tid=0 2024-04-25T10:02:19.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 client.9030 shutdown 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 msgr2=0x55ad14b72af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14b72af0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f684002a790 tx=0x7f6840004730 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 shutdown_connections 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55ad14b84510 0x55ad14b06ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad14b04ef0 0x55ad14b72af0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad14b04b10 0x55ad14b725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68297fa640 0 client.9030 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 --2- 172.21.15.148:0/4074352789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad14afd3e0 0x55ad14ad6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 >> 172.21.15.148:0/4074352789 conn(0x55ad14ad7770 msgr2=0x55ad14b03110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 shutdown_connections 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 1 -- 172.21.15.148:0/4074352789 wait complete. 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f6852fcf740 20 client.9030 trim_cache size 0 max 16384 2024-04-25T10:02:19.865 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:19.866 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:02:19.866 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:02:19.866 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f682c06b500 msgr2=0x7f682c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68502cb640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f682c06b500 0x7f682c06d9c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f684815e2c0 tx=0x7f68440039f0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 msgr2=0x7f68481f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.854+0000 7f68502cb640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 0x7f68481f3ab0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f683800c410 tx=0x7f683800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 shutdown_connections 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f682c06b500 0x7f682c06d9c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f684815f3d0 0x7f684806c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68480cfd30 0x7f68481f3ab0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 --2- 172.21.15.148:0/1781748568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6848099cb0 0x7f68480b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 >> 172.21.15.148:0/1781748568 conn(0x7f68480b3290 msgr2=0x7f68480c7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 shutdown_connections 2024-04-25T10:02:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.855+0000 7f68502cb640 1 -- 172.21.15.148:0/1781748568 wait complete. 2024-04-25T10:02:19.870 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.962+0000 7fd33a3b0640 1 Processor -- start 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.962+0000 7fd33a3b0640 1 -- start start 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd33a3b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd33415bd30 0x7fd3340cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd33a3b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340d1b00 0x7fd3340b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd33a3b0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3340d0350 con 0x7fd3340cfe40 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd33a3b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3340d0630 con 0x7fd33415bd30 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd33a3b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340d0910 con 0x7fd3340d1b00 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd33415bd30 0x7fd3340cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd33415bd30 0x7fd3340cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52184/0 (socket says 172.21.15.148:52184) 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 -- 172.21.15.148:0/1853282260 learned_addr learned my addr 172.21.15.148:0/1853282260 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd33894d640 1 --2- 172.21.15.148:0/1853282260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340d1b00 0x7fd3340b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 -- 172.21.15.148:0/1853282260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340d1b00 msgr2=0x7fd3340b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 --2- 172.21.15.148:0/1853282260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340d1b00 0x7fd3340b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 -- 172.21.15.148:0/1853282260 >> v1:172.21.15.188:6789/0 conn(0x7fd3340cfe40 legacy=0x7fd3340d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.963+0000 7fd333fff640 1 -- 172.21.15.148:0/1853282260 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3340d0bf0 con 0x7fd33415bd30 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd333fff640 1 --2- 172.21.15.148:0/1853282260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340cf900 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fd324009870 tx=0x7fd32402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbc28491701a0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd332ffd640 1 -- 172.21.15.148:0/1853282260 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324037070 con 0x7fd33415bd30 2024-04-25T10:02:19.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd332ffd640 1 -- 172.21.15.148:0/1853282260 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32402ae50 con 0x7fd33415bd30 2024-04-25T10:02:19.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd332ffd640 1 -- 172.21.15.148:0/1853282260 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324032860 con 0x7fd33415bd30 2024-04-25T10:02:19.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1853282260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 msgr2=0x7fd3340cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1853282260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340cf900 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fd324009870 tx=0x7fd32402a300 comp rx=0 tx=0).stop 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.964+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1853282260 shutdown_connections 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.965+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1853282260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340d1b00 0x7fd3340b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.965+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1853282260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340cf900 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.965+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1853282260 >> 172.21.15.148:0/1853282260 conn(0x7fd33406f590 msgr2=0x7fd33406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.965+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1853282260 shutdown_connections 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.965+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1853282260 wait complete. 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.966+0000 7fd33a3b0640 1 Processor -- start 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.966+0000 7fd33a3b0640 1 -- start start 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340cfe40 0x7fd33414ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3340d1b00 0x7fd33415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340b73b0 con 0x7fd33415bd30 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33415c110 con 0x7fd3340d1b00 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3340b7530 con 0x7fd3340cfe40 2024-04-25T10:02:19.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52186/0 (socket says 172.21.15.148:52186) 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 -- 172.21.15.148:0/1448324609 learned_addr learned my addr 172.21.15.148:0/1448324609 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd333fff640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340cfe40 0x7fd33414ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd3337fe640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3340d1b00 0x7fd33415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 -- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340cfe40 msgr2=0x7fd33414ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340cfe40 0x7fd33414ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 -- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3340d1b00 msgr2=0x7fd33415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3340d1b00 0x7fd33415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 -- 172.21.15.148:0/1448324609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33415c8e0 con 0x7fd33415bd30 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd3337fe640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3340d1b00 0x7fd33415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33894d640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340c93f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd32c007d90 tx=0x7fd32c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c014420 con 0x7fd33415bd30 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c019070 con 0x7fd33415bd30 2024-04-25T10:02:19.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c014820 con 0x7fd33415bd30 2024-04-25T10:02:19.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33415cbc0 con 0x7fd33415bd30 2024-04-25T10:02:19.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.967+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33415d180 con 0x7fd33415bd30 2024-04-25T10:02:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.968+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd334073990 con 0x7fd33415bd30 2024-04-25T10:02:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.969+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd32c0149c0 con 0x7fd33415bd30 2024-04-25T10:02:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.969+0000 7fd330ff9640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd32006b500 0x7fd32006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.969+0000 7fd333fff640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd32006b500 0x7fd32006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.969+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd32c08aa50 con 0x7fd33415bd30 2024-04-25T10:02:19.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.970+0000 7fd333fff640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd32006b500 0x7fd32006d9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd324034000 tx=0x7fd32402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:19.974+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd32c0597d0 con 0x7fd33415bd30 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.180+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3340743e0 con 0x7fd33415bd30 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.181+0000 7fd330ff9640 1 -- 172.21.15.148:0/1448324609 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd32c0590d0 con 0x7fd33415bd30 2024-04-25T10:02:20.186 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:20.186 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:20.186 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd32006b500 msgr2=0x7fd32006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd32006b500 0x7fd32006d9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd324034000 tx=0x7fd32402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 msgr2=0x7fd3340c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340c93f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd32c007d90 tx=0x7fd32c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 shutdown_connections 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd32006b500 0x7fd32006d9c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33415bd30 0x7fd3340c93f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3340d1b00 0x7fd33415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 --2- 172.21.15.148:0/1448324609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3340cfe40 0x7fd33414ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 >> 172.21.15.148:0/1448324609 conn(0x7fd33406f590 msgr2=0x7fd3340b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 shutdown_connections 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.183+0000 7fd33a3b0640 1 -- 172.21.15.148:0/1448324609 wait complete. 2024-04-25T10:02:20.198 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:20.199 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:20.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.290+0000 7f377c0fc640 1 Processor -- start 2024-04-25T10:02:20.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.290+0000 7f377c0fc640 1 -- start start 2024-04-25T10:02:20.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f377c0fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3774154ee0 0x7f37740cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f377c0fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3774071480 con 0x7f37740cfcf0 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f377c0fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3774071760 con 0x7f37740ce590 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f377c0fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3774072090 con 0x7f3774154ee0 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3779e98640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3774154ee0 0x7f37740cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3779697640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f37740cfcf0 0x7f37740cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60368/0 (socket says 172.21.15.148:60368) 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3779697640 1 -- 172.21.15.148:0/263703450 learned_addr learned my addr 172.21.15.148:0/263703450 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3779e98640 1 -- 172.21.15.148:0/263703450 >> v1:172.21.15.188:6789/0 conn(0x7f37740ce590 legacy=0x7f377406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3779e98640 1 -- 172.21.15.148:0/263703450 >> v1:172.21.15.148:6789/0 conn(0x7f37740cfcf0 legacy=0x7f37740cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3778e96640 1 -- 172.21.15.148:0/263703450 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881275418 0 0) 0x7f3774071480 con 0x7f37740cfcf0 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.291+0000 7f3779e98640 1 -- 172.21.15.148:0/263703450 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3774071a40 con 0x7f3774154ee0 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.292+0000 7f3779e98640 1 --2- 172.21.15.148:0/263703450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3774154ee0 0x7f37740cf7b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f376c0097a0 tx=0x7f376c02c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d72b53835e5b249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.292+0000 7f3778e96640 1 -- 172.21.15.148:0/263703450 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c038070 con 0x7f3774154ee0 2024-04-25T10:02:20.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.292+0000 7f3778e96640 1 -- 172.21.15.148:0/263703450 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c031710 con 0x7f3774154ee0 2024-04-25T10:02:20.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.292+0000 7f3778e96640 1 -- 172.21.15.148:0/263703450 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c031ae0 con 0x7f3774154ee0 2024-04-25T10:02:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 -- 172.21.15.148:0/263703450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3774154ee0 msgr2=0x7f37740cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 --2- 172.21.15.148:0/263703450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3774154ee0 0x7f37740cf7b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f376c0097a0 tx=0x7f376c02c440 comp rx=0 tx=0).stop 2024-04-25T10:02:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 -- 172.21.15.148:0/263703450 shutdown_connections 2024-04-25T10:02:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 --2- 172.21.15.148:0/263703450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3774154ee0 0x7f37740cf7b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 -- 172.21.15.148:0/263703450 >> 172.21.15.148:0/263703450 conn(0x7f377406abe0 msgr2=0x7f377406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 -- 172.21.15.148:0/263703450 shutdown_connections 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.293+0000 7f377c0fc640 1 -- 172.21.15.148:0/263703450 wait complete. 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.294+0000 7f377c0fc640 1 Processor -- start 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.294+0000 7f377c0fc640 1 -- start start 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 0x7f3774156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37740cfcf0 0x7f37741f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3774154ee0 0x7f377415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37741552c0 con 0x7f37740ce590 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3774071230 con 0x7f3774154ee0 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3774155440 con 0x7f37740cfcf0 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 0x7f3774156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 0x7f3774156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52200/0 (socket says 172.21.15.148:52200) 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 -- 172.21.15.148:0/2911890498 learned_addr learned my addr 172.21.15.148:0/2911890498 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 -- 172.21.15.148:0/2911890498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37740cfcf0 msgr2=0x7f37741f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37740cfcf0 0x7f37741f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 -- 172.21.15.148:0/2911890498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3774154ee0 msgr2=0x7f377415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3774154ee0 0x7f377415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 -- 172.21.15.148:0/2911890498 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37741573e0 con 0x7f37740ce590 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f3779e98640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 0x7f3774156320 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f376c0098c0 tx=0x7f376c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c042020 con 0x7f37740ce590 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c033a40 con 0x7f37740ce590 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c038040 con 0x7f37740ce590 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37741576c0 con 0x7f37740ce590 2024-04-25T10:02:20.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.295+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3774157c80 con 0x7f37740ce590 2024-04-25T10:02:20.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.296+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3774073990 con 0x7f37740ce590 2024-04-25T10:02:20.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.297+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f376c045020 con 0x7f37740ce590 2024-04-25T10:02:20.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.297+0000 7f376a7fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f375806b4d0 0x7f375806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.297+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f376c0ad000 con 0x7f37740ce590 2024-04-25T10:02:20.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.297+0000 7f3779697640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f375806b4d0 0x7f375806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.298+0000 7f3779697640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f375806b4d0 0x7f375806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3764007fd0 tx=0x7f3764007900 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.301+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f376c04c030 con 0x7f37740ce590 2024-04-25T10:02:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.538+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f377406d360 con 0x7f37740ce590 2024-04-25T10:02:20.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.538+0000 7f376a7fc640 1 -- 172.21.15.148:0/2911890498 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f376c07bcf0 con 0x7f37740ce590 2024-04-25T10:02:20.543 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:20.543 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":501,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562785280,"bytes_avail":770531328000,"bytes_total":773094113280,"write_bytes_sec":77074825,"read_op_per_sec":0,"write_op_per_sec":75},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:02:19.118650+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:02:20.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f375806b4d0 msgr2=0x7f375806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f375806b4d0 0x7f375806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3764007fd0 tx=0x7f3764007900 comp rx=0 tx=0).stop 2024-04-25T10:02:20.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 msgr2=0x7f3774156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 0x7f3774156320 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f376c0098c0 tx=0x7f376c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 shutdown_connections 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f375806b4d0 0x7f375806d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3774154ee0 0x7f377415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37740cfcf0 0x7f37741f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 --2- 172.21.15.148:0/2911890498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37740ce590 0x7f3774156320 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 >> 172.21.15.148:0/2911890498 conn(0x7f377406abe0 msgr2=0x7f377406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 shutdown_connections 2024-04-25T10:02:20.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.540+0000 7f377c0fc640 1 -- 172.21.15.148:0/2911890498 wait complete. 2024-04-25T10:02:20.557 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:02:20.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:20.557 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T10:02:20.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.652+0000 7efe06322640 1 Processor -- start 2024-04-25T10:02:20.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.652+0000 7efe06322640 1 -- start start 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efe06322640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe001454b0 0x7efe00070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efe06322640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000ceba0 con 0x7efe00071300 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efe06322640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000cee80 con 0x7efe00144d20 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efe06322640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000b4830 con 0x7efe001454b0 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efdff7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe001454b0 0x7efe00070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efe048bf640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe00071300 0x7efe000b4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60374/0 (socket says 172.21.15.148:60374) 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efe048bf640 1 -- 172.21.15.148:0/2218578694 learned_addr learned my addr 172.21.15.148:0/2218578694 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:20.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efdff7fe640 1 -- 172.21.15.148:0/2218578694 >> v1:172.21.15.188:6789/0 conn(0x7efe00144d20 legacy=0x7efe000ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efdff7fe640 1 -- 172.21.15.148:0/2218578694 >> v1:172.21.15.148:6789/0 conn(0x7efe00071300 legacy=0x7efe000b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efdff7fe640 1 -- 172.21.15.148:0/2218578694 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000cf160 con 0x7efe001454b0 2024-04-25T10:02:20.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.653+0000 7efdfeffd640 1 -- 172.21.15.148:0/2218578694 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523590011 0 0) 0x7efe000ceba0 con 0x7efe00071300 2024-04-25T10:02:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efdff7fe640 1 --2- 172.21.15.148:0/2218578694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe001454b0 0x7efe00070dc0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7efe000b49b0 tx=0x7efdf402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8a45232b5b41d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efdfeffd640 1 -- 172.21.15.148:0/2218578694 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4037070 con 0x7efe001454b0 2024-04-25T10:02:20.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efdfeffd640 1 -- 172.21.15.148:0/2218578694 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4032440 con 0x7efe001454b0 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efdfeffd640 1 -- 172.21.15.148:0/2218578694 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4032810 con 0x7efe001454b0 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efe06322640 1 -- 172.21.15.148:0/2218578694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe001454b0 msgr2=0x7efe00070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efe06322640 1 --2- 172.21.15.148:0/2218578694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe001454b0 0x7efe00070dc0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7efe000b49b0 tx=0x7efdf402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efe06322640 1 -- 172.21.15.148:0/2218578694 shutdown_connections 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efe06322640 1 --2- 172.21.15.148:0/2218578694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe001454b0 0x7efe00070dc0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.654+0000 7efe06322640 1 -- 172.21.15.148:0/2218578694 >> 172.21.15.148:0/2218578694 conn(0x7efe000b3530 msgr2=0x7efe000b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.655+0000 7efe06322640 1 -- 172.21.15.148:0/2218578694 shutdown_connections 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.655+0000 7efe06322640 1 -- 172.21.15.148:0/2218578694 wait complete. 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.656+0000 7efe06322640 1 Processor -- start 2024-04-25T10:02:20.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.656+0000 7efe06322640 1 -- start start 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.656+0000 7efe06322640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 0x7efe001f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00144d20 0x7efe000afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe001454b0 0x7efe0013ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00145100 con 0x7efe00071300 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00071730 con 0x7efe00144d20 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe00145280 con 0x7efe001454b0 2024-04-25T10:02:20.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 0x7efe001f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 0x7efe001f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52214/0 (socket says 172.21.15.148:52214) 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 -- 172.21.15.148:0/3448147445 learned_addr learned my addr 172.21.15.148:0/3448147445 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdff7fe640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00144d20 0x7efe000afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe048bf640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe001454b0 0x7efe0013ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 -- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe001454b0 msgr2=0x7efe0013ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe001454b0 0x7efe0013ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 -- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00144d20 msgr2=0x7efe000afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00144d20 0x7efe000afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 -- 172.21.15.148:0/3448147445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe001f3220 con 0x7efe00071300 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdff7fe640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00144d20 0x7efe000afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfffff640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 0x7efe001f2ce0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c400 tx=0x7efdf000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfcff9640 1 -- 172.21.15.148:0/3448147445 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0017070 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfcff9640 1 -- 172.21.15.148:0/3448147445 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00052d0 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe000cb9c0 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efdfcff9640 1 -- 172.21.15.148:0/3448147445 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003830 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.657+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe000c7ae0 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.659+0000 7efdfcff9640 1 -- 172.21.15.148:0/3448147445 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdf001a070 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.659+0000 7efdfcff9640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdec06b4d0 0x7efdec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.659+0000 7efdfcff9640 1 -- 172.21.15.148:0/3448147445 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efdf0089720 con 0x7efe00071300 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.659+0000 7efdff7fe640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdec06b4d0 0x7efdec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.660+0000 7efdff7fe640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdec06b4d0 0x7efdec06d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efdf4034000 tx=0x7efdf402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.660+0000 7efe09029740 1 Processor -- start 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.660+0000 7efe09029740 1 -- start start 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c43320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe09029740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c43860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c13770 0x55e2e1c16f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe09029740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1b7cda0 con 0x55e2e1c40ef0 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1b25b80 con 0x55e2e1c13770 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1b78e40 con 0x55e2e1c40b10 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c43860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c43860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52224/0 (socket says 172.21.15.148:52224) 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 -- 172.21.15.148:0/305094852 learned_addr learned my addr 172.21.15.148:0/305094852 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdfffff640 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c43320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efe048bf640 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c13770 0x55e2e1c16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 -- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 msgr2=0x55e2e1c43320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c43320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 -- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c13770 msgr2=0x55e2e1c16f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c13770 0x55e2e1c16f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.661+0000 7efdff7fe640 1 -- 172.21.15.148:0/305094852 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e1b2ff10 con 0x55e2e1c40ef0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efdff7fe640 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c43860 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009860 tx=0x7efdf402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f625a18e68906d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efde5ffb640 1 -- 172.21.15.148:0/305094852 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf404d070 con 0x55e2e1c40ef0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efde5ffb640 1 -- 172.21.15.148:0/305094852 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4048470 con 0x55e2e1c40ef0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efde5ffb640 1 -- 172.21.15.148:0/305094852 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4037040 con 0x55e2e1c40ef0 2024-04-25T10:02:20.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 -- 172.21.15.148:0/305094852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 msgr2=0x55e2e1c43860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c43860 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009860 tx=0x7efdf402c000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 -- 172.21.15.148:0/305094852 shutdown_connections 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c13770 0x55e2e1c16f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c43860 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 --2- 172.21.15.148:0/305094852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c43320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 -- 172.21.15.148:0/305094852 >> 172.21.15.148:0/305094852 conn(0x55e2e1c159e0 msgr2=0x55e2e1c5e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 -- 172.21.15.148:0/305094852 shutdown_connections 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.662+0000 7efe09029740 1 -- 172.21.15.148:0/305094852 wait complete. 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.663+0000 7efe09029740 1 -- start start 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.663+0000 7efe09029740 1 Processor -- start 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c30cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c65410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c62e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1c18b90 con 0x55e2e1c629f0 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1b1b4b0 con 0x55e2e1c40ef0 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1c18e70 con 0x55e2e1c40b10 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 10 client.0 _ll_register_callbacks cb 0x55e2e1b5b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c62e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c62e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52236/0 (socket says 172.21.15.148:52236) 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 -- 172.21.15.148:0/2507142863 learned_addr learned my addr 172.21.15.148:0/2507142863 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efdff7fe640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c65410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efdfffff640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c30cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 -- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 msgr2=0x55e2e1c30cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c30cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 -- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40ef0 msgr2=0x55e2e1c65410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c65410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 -- 172.21.15.148:0/2507142863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2e195b7d0 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe048bf640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c62e70 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7efdf8001540 tx=0x7efdf8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04d124d46a762e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efddeffd640 1 -- 172.21.15.148:0/2507142863 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8019060 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efddeffd640 1 -- 172.21.15.148:0/2507142863 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efdf8014c30 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.664+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdf4048470 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efddeffd640 1 -- 172.21.15.148:0/2507142863 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf8014e50 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 10 client.9060 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55e2e18913f0 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efddeffd640 1 -- 172.21.15.148:0/2507142863 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4048470 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efddeffd640 1 -- 172.21.15.148:0/2507142863 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efdf8011580 con 0x55e2e1c629f0 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 10 client.9060 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 10 client.9060 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 20 client.9060 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:20.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c3efc0 0x55e2e1c3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 4 client.9060 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2e1b8a2e0 con 0x55e2e1c3efc0 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efdfffff640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c3efc0 0x55e2e1c3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efddeffd640 10 client.9060 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.665+0000 7efdfffff640 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c3efc0 0x55e2e1c3f3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482e2a2e7cef4d94 server_cookie=ac3bf0c2f58a1f3a in_seq=0 out_seq=0 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efddeffd640 1 -- 172.21.15.148:0/2507142863 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2e1b8a2e0 con 0x55e2e1c3efc0 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efddeffd640 10 client.9060 handle_command_reply: tid=0 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 client.9060 shutdown 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 msgr2=0x55e2e1c62e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c62e70 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7efdf8001540 tx=0x7efdf8001a00 comp rx=0 tx=0).stop 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 shutdown_connections 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c3efc0 0x55e2e1c3f3d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c62e70 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40ef0 0x55e2e1c65410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efddeffd640 0 client.9060 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 --2- 172.21.15.148:0/2507142863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40b10 0x55e2e1c30cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 >> 172.21.15.148:0/2507142863 conn(0x55e2e1c13770 msgr2=0x55e2e1c147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 shutdown_connections 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.666+0000 7efe09029740 1 -- 172.21.15.148:0/2507142863 wait complete. 2024-04-25T10:02:20.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.667+0000 7efe09029740 20 client.9060 trim_cache size 0 max 16384 2024-04-25T10:02:20.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.681+0000 7efe09029740 1 Processor -- start 2024-04-25T10:02:20.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.681+0000 7efe09029740 1 -- start start 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 0x55e2e1c14520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 0x55e2e1c14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe09029740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c3df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe09029740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe001fe100 con 0x55e2e1c629f0 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe001fd300 con 0x55e2e1c40b10 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe001f65b0 con 0x55e2e1c40ef0 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c3df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c3df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52238/0 (socket says 172.21.15.148:52238) 2024-04-25T10:02:20.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 -- 172.21.15.148:0/970691002 learned_addr learned my addr 172.21.15.148:0/970691002 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 -- 172.21.15.148:0/970691002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 msgr2=0x55e2e1c14a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 0x55e2e1c14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 -- 172.21.15.148:0/970691002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 msgr2=0x55e2e1c14520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 0x55e2e1c14520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 -- 172.21.15.148:0/970691002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2e18510f0 con 0x55e2e1c629f0 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efe048bf640 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c3df00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7efdf8000f80 tx=0x7efdf8007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acebadfe964fead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efde57fa640 1 -- 172.21.15.148:0/970691002 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8011e30 con 0x55e2e1c629f0 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efde57fa640 1 -- 172.21.15.148:0/970691002 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8011ba0 con 0x55e2e1c629f0 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.682+0000 7efde57fa640 1 -- 172.21.15.148:0/970691002 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8004b60 con 0x55e2e1c629f0 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 -- 172.21.15.148:0/970691002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 msgr2=0x55e2e1c3df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c3df00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7efdf8000f80 tx=0x7efdf8007000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 -- 172.21.15.148:0/970691002 shutdown_connections 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c3df00 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 0x55e2e1c14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 --2- 172.21.15.148:0/970691002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 0x55e2e1c14520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 -- 172.21.15.148:0/970691002 >> 172.21.15.148:0/970691002 conn(0x55e2e1c13770 msgr2=0x55e2e1c19810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 -- 172.21.15.148:0/970691002 shutdown_connections 2024-04-25T10:02:20.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 -- 172.21.15.148:0/970691002 wait complete. 2024-04-25T10:02:20.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.683+0000 7efe09029740 1 -- start start 2024-04-25T10:02:20.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 Processor -- start 2024-04-25T10:02:20.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 0x55e2e1c3f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 0x55e2e1c3f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c5f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1c80880 con 0x55e2e1c629f0 2024-04-25T10:02:20.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe001fcd30 con 0x55e2e1c40b10 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2e1c5cb80 con 0x55e2e1c40ef0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.684+0000 7efe09029740 10 client.0 _ll_register_callbacks cb 0x7efdf4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c5f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c5f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52248/0 (socket says 172.21.15.148:52248) 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 -- 172.21.15.148:0/2161165487 learned_addr learned my addr 172.21.15.148:0/2161165487 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 -- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 msgr2=0x55e2e1c3f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 0x55e2e1c3f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 -- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 msgr2=0x55e2e1c3f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 0x55e2e1c3f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 -- 172.21.15.148:0/2161165487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2e1c14cd0 con 0x55e2e1c629f0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe048bf640 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c5f450 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7efdf8011b70 tx=0x7efdf8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e61648e9e9c9f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efddeffd640 1 -- 172.21.15.148:0/2161165487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8011e30 con 0x55e2e1c629f0 2024-04-25T10:02:20.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efddeffd640 1 -- 172.21.15.148:0/2161165487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efdf8004b60 con 0x55e2e1c629f0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdf8011ba0 con 0x55e2e1c629f0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efddeffd640 1 -- 172.21.15.148:0/2161165487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf8038050 con 0x55e2e1c629f0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 10 client.9066 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55e2e1c5d8b0 con 0x55e2e1c629f0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efddeffd640 1 -- 172.21.15.148:0/2161165487 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8011ba0 con 0x55e2e1c629f0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efddeffd640 1 -- 172.21.15.148:0/2161165487 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efdf803b020 con 0x55e2e1c629f0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 10 client.9066 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 10 client.9066 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 20 client.9066 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.685+0000 7efe09029740 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c624b0 0x55e2e1c5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 4 client.9066 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55e2e1c63cd0 con 0x55e2e1c624b0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efdfffff640 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c624b0 0x55e2e1c5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efddeffd640 10 client.9066 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efdfffff640 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c624b0 0x55e2e1c5f990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb24c3d7e8f4967 server_cookie=4f9e78bd1b99794 in_seq=0 out_seq=0 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efddeffd640 1 -- 172.21.15.148:0/2161165487 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55e2e1c63cd0 con 0x55e2e1c624b0 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efddeffd640 10 client.9066 handle_command_reply: tid=0 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 client.9066 shutdown 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 msgr2=0x55e2e1c5f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c5f450 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7efdf8011b70 tx=0x7efdf8001250 comp rx=0 tx=0).stop 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 shutdown_connections 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55e2e1c624b0 0x55e2e1c5f990 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2e1c629f0 0x55e2e1c5f450 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2e1c40ef0 0x55e2e1c3f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efddeffd640 0 client.9066 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 --2- 172.21.15.148:0/2161165487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2e1c40b10 0x55e2e1c3f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 >> 172.21.15.148:0/2161165487 conn(0x55e2e1c13770 msgr2=0x55e2e1c43e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.686+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 shutdown_connections 2024-04-25T10:02:20.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe09029740 1 -- 172.21.15.148:0/2161165487 wait complete. 2024-04-25T10:02:20.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe09029740 20 client.9066 trim_cache size 0 max 16384 2024-04-25T10:02:20.697 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:20.697 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:02:20.697 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdec06b4d0 msgr2=0x7efdec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdec06b4d0 0x7efdec06d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efdf4034000 tx=0x7efdf402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 msgr2=0x7efe001f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 0x7efe001f2ce0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7efdf000c400 tx=0x7efdf000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 shutdown_connections 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efdec06b4d0 0x7efdec06d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe001454b0 0x7efe0013ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe00144d20 0x7efe000afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 --2- 172.21.15.148:0/3448147445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe00071300 0x7efe001f2ce0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 >> 172.21.15.148:0/3448147445 conn(0x7efe000b3530 msgr2=0x7efe001415a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 shutdown_connections 2024-04-25T10:02:20.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.687+0000 7efe06322640 1 -- 172.21.15.148:0/3448147445 wait complete. 2024-04-25T10:02:20.702 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:02:20.702 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:02:20.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.797+0000 7f21b04f5640 1 Processor -- start 2024-04-25T10:02:20.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.797+0000 7f21b04f5640 1 -- start start 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21b04f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21b04f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21a80ad570 0x7f21a806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21b04f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a806b500 con 0x7f21a815dd80 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21b04f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a806e410 con 0x7f21a80ad570 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21b04f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a80b3930 con 0x7f21a806abe0 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21a80ad570 0x7f21a806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f21a80ad570 0x7f21a806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52254/0 (socket says 172.21.15.148:52254) 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 -- 172.21.15.148:0/4227790625 learned_addr learned my addr 172.21.15.148:0/4227790625 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21ae291640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 -- 172.21.15.148:0/4227790625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 msgr2=0x7f21a806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 -- 172.21.15.148:0/4227790625 >> v1:172.21.15.188:6789/0 conn(0x7f21a815dd80 legacy=0x7f21a80b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21aea92640 1 -- 172.21.15.148:0/4227790625 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80b3c10 con 0x7f21a80ad570 2024-04-25T10:02:20.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.798+0000 7f21ae291640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.799+0000 7f21aea92640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a80ad570 0x7f21a806ee60 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009e50 tx=0x7f21a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e861dd5340aaa3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.799+0000 7f21ad28f640 1 -- 172.21.15.148:0/4227790625 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0036070 con 0x7f21a80ad570 2024-04-25T10:02:20.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.799+0000 7f21ad28f640 1 -- 172.21.15.148:0/4227790625 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a0031600 con 0x7f21a80ad570 2024-04-25T10:02:20.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.799+0000 7f21ad28f640 1 -- 172.21.15.148:0/4227790625 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00319d0 con 0x7f21a80ad570 2024-04-25T10:02:20.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 -- 172.21.15.148:0/4227790625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a80ad570 msgr2=0x7f21a806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a80ad570 0x7f21a806ee60 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009e50 tx=0x7f21a002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 -- 172.21.15.148:0/4227790625 shutdown_connections 2024-04-25T10:02:20.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a80ad570 0x7f21a806ee60 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 --2- 172.21.15.148:0/4227790625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 -- 172.21.15.148:0/4227790625 >> 172.21.15.148:0/4227790625 conn(0x7f21a80b5af0 msgr2=0x7f21a80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 -- 172.21.15.148:0/4227790625 shutdown_connections 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.800+0000 7f21b04f5640 1 -- 172.21.15.148:0/4227790625 wait complete. 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.801+0000 7f21b04f5640 1 Processor -- start 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.801+0000 7f21b04f5640 1 -- start start 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21b04f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21b04f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a80ad570 0x7f21a81592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21b04f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 0x7f21a81492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21b04f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8070280 con 0x7f21a815dd80 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21b04f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a806e940 con 0x7f21a80ad570 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21b04f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8070400 con 0x7f21a806abe0 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 0x7f21a81492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 0x7f21a81492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52260/0 (socket says 172.21.15.148:52260) 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 -- 172.21.15.148:0/2907359177 learned_addr learned my addr 172.21.15.148:0/2907359177 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21ada90640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a80ad570 0x7f21a81592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21ae291640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 -- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 msgr2=0x7f21a815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a815ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 -- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a80ad570 msgr2=0x7f21a81592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a80ad570 0x7f21a81592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 -- 172.21.15.148:0/2907359177 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8159800 con 0x7f21a815dd80 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21ae291640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.802+0000 7f21aea92640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 0x7f21a81492c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009e50 tx=0x7f21a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.803+0000 7f219effd640 1 -- 172.21.15.148:0/2907359177 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0036050 con 0x7f21a815dd80 2024-04-25T10:02:20.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.803+0000 7f219effd640 1 -- 172.21.15.148:0/2907359177 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a0042070 con 0x7f21a815dd80 2024-04-25T10:02:20.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.803+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a815f250 con 0x7f21a815dd80 2024-04-25T10:02:20.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.803+0000 7f219effd640 1 -- 172.21.15.148:0/2907359177 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0009510 con 0x7f21a815dd80 2024-04-25T10:02:20.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.803+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a815f810 con 0x7f21a815dd80 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.804+0000 7f219effd640 1 -- 172.21.15.148:0/2907359177 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21a0007bf0 con 0x7f21a815dd80 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.804+0000 7f219effd640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f218c06b4d0 0x7f218c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.805+0000 7f219effd640 1 -- 172.21.15.148:0/2907359177 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21a00ab1f0 con 0x7f21a815dd80 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.805+0000 7f21ae291640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f218c06b4d0 0x7f218c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.805+0000 7f21ae291640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f218c06b4d0 0x7f218c06d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f21a815cdb0 tx=0x7f21a4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 Processor -- start 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 -- start start 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c547d770 0x55c1c5480f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c53e6da0 con 0x55c1c54aaef0 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c538fb80 con 0x55c1c54aab10 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.806+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c53e2e40 con 0x55c1c547d770 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52274/0 (socket says 172.21.15.148:52274) 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 -- 172.21.15.148:0/2458167631 learned_addr learned my addr 172.21.15.148:0/2458167631 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ae291640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21aea92640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c547d770 0x55c1c5480f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 -- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c547d770 msgr2=0x55c1c5480f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c547d770 0x55c1c5480f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 -- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 msgr2=0x55c1c54ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 -- 172.21.15.148:0/2458167631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c5399f10 con 0x55c1c54aaef0 2024-04-25T10:02:20.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21aea92640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c547d770 0x55c1c5480f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ae291640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f21ada90640 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ad860 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f2198009910 tx=0x7f2198009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17159494374549c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f218bfff640 1 -- 172.21.15.148:0/2458167631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198017070 con 0x55c1c54aaef0 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f218bfff640 1 -- 172.21.15.148:0/2458167631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800ea20 con 0x55c1c54aaef0 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.807+0000 7f218bfff640 1 -- 172.21.15.148:0/2458167631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800ee10 con 0x55c1c54aaef0 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 -- 172.21.15.148:0/2458167631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 msgr2=0x55c1c54ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ad860 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f2198009910 tx=0x7f2198009dd0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 -- 172.21.15.148:0/2458167631 shutdown_connections 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c547d770 0x55c1c5480f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ad860 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 --2- 172.21.15.148:0/2458167631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 -- 172.21.15.148:0/2458167631 >> 172.21.15.148:0/2458167631 conn(0x55c1c547f9e0 msgr2=0x55c1c54c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 -- 172.21.15.148:0/2458167631 shutdown_connections 2024-04-25T10:02:20.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 -- 172.21.15.148:0/2458167631 wait complete. 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.808+0000 7f21b31f2740 1 -- start start 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 Processor -- start 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aaef0 0x55c1c54cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 0x55c1c54ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c5482b90 con 0x55c1c54c2950 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c53854b0 con 0x55c1c54aab10 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.809+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c5482e70 con 0x55c1c54aaef0 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21b31f2740 10 client.0 _ll_register_callbacks cb 0x55c1c53c5180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 0x55c1c54ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 0x55c1c54ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52276/0 (socket says 172.21.15.148:52276) 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 -- 172.21.15.148:0/3214947771 learned_addr learned my addr 172.21.15.148:0/3214947771 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 -- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aaef0 msgr2=0x55c1c54cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aaef0 0x55c1c54cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 -- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 msgr2=0x55c1c54a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 -- 172.21.15.148:0/3214947771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1c51c57d0 con 0x55c1c54c2950 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21aea92640 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 0x55c1c54ca3b0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f21a00073c0 tx=0x7f21a00ab050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6fa58ac5fe6523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21897fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21897fa640 1 -- 172.21.15.148:0/3214947771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00a7060 con 0x55c1c54c2950 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21897fa640 1 -- 172.21.15.148:0/3214947771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21a0091370 con 0x55c1c54c2950 2024-04-25T10:02:20.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f219800ea20 con 0x55c1c54c2950 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21897fa640 1 -- 172.21.15.148:0/3214947771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21a0096050 con 0x55c1c54c2950 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21b31f2740 10 client.9078 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55c1c50fb3f0 con 0x55c1c54c2950 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21897fa640 1 -- 172.21.15.148:0/3214947771 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800ea20 con 0x55c1c54c2950 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.810+0000 7f21897fa640 1 -- 172.21.15.148:0/3214947771 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21a00982e0 con 0x55c1c54c2950 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21b31f2740 10 client.9078 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21b31f2740 10 client.9078 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21b31f2740 20 client.9078 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54a3730 0x55c1c54a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21b31f2740 4 client.9078 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1c53f42e0 con 0x55c1c54a3730 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21ae291640 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54a3730 0x55c1c54a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21897fa640 10 client.9078 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.811+0000 7f21ae291640 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54a3730 0x55c1c54a3be0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b0a25d262846b3 server_cookie=a83de7514f460a2d in_seq=0 out_seq=0 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21897fa640 1 -- 172.21.15.148:0/3214947771 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1c53f42e0 con 0x55c1c54a3730 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21897fa640 10 client.9078 handle_command_reply: tid=0 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 client.9078 shutdown 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 msgr2=0x55c1c54ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 0x55c1c54ca3b0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f21a00073c0 tx=0x7f21a00ab050 comp rx=0 tx=0).stop 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 shutdown_connections 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54a3730 0x55c1c54a3be0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54c2950 0x55c1c54ca3b0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aaef0 0x55c1c54cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21897fa640 0 client.9078 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3214947771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c54a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 >> 172.21.15.148:0/3214947771 conn(0x55c1c547d770 msgr2=0x55c1c547e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 shutdown_connections 2024-04-25T10:02:20.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 1 -- 172.21.15.148:0/3214947771 wait complete. 2024-04-25T10:02:20.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.812+0000 7f21b31f2740 20 client.9078 trim_cache size 0 max 16384 2024-04-25T10:02:20.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.827+0000 7f21b31f2740 1 Processor -- start 2024-04-25T10:02:20.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.827+0000 7f21b31f2740 1 -- start start 2024-04-25T10:02:20.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aab10 0x55c1c54a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54c2950 0x55c1c54aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21b31f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a8200590 con 0x55c1c54aaef0 2024-04-25T10:02:20.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a81ff770 con 0x55c1c54c2950 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a814b8a0 con 0x55c1c54aab10 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52288/0 (socket says 172.21.15.148:52288) 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 -- 172.21.15.148:0/3544873996 learned_addr learned my addr 172.21.15.148:0/3544873996 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21aea92640 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54c2950 0x55c1c54aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ae291640 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aab10 0x55c1c54a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 -- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aab10 msgr2=0x55c1c54a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aab10 0x55c1c54a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 -- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54c2950 msgr2=0x55c1c54aa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54c2950 0x55c1c54aa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 -- 172.21.15.148:0/3544873996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c54adbd0 con 0x55c1c54aaef0 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f21ada90640 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ca390 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f2198016000 tx=0x7f2198011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709777100db7565c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f218b7fe640 1 -- 172.21.15.148:0/3544873996 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198029070 con 0x55c1c54aaef0 2024-04-25T10:02:20.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f218b7fe640 1 -- 172.21.15.148:0/3544873996 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800fcb0 con 0x55c1c54aaef0 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.828+0000 7f218b7fe640 1 -- 172.21.15.148:0/3544873996 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198017040 con 0x55c1c54aaef0 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 -- 172.21.15.148:0/3544873996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 msgr2=0x55c1c54ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ca390 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f2198016000 tx=0x7f2198011000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 -- 172.21.15.148:0/3544873996 shutdown_connections 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54c2950 0x55c1c54aa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54ca390 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 --2- 172.21.15.148:0/3544873996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54aab10 0x55c1c54a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 -- 172.21.15.148:0/3544873996 >> 172.21.15.148:0/3544873996 conn(0x55c1c547d770 msgr2=0x55c1c54c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 -- 172.21.15.148:0/3544873996 shutdown_connections 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.829+0000 7f21b31f2740 1 -- 172.21.15.148:0/3544873996 wait complete. 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 -- start start 2024-04-25T10:02:20.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 Processor -- start 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c5478e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54c2950 0x55c1c54c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c5483840 con 0x55c1c54aaef0 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a81ff170 con 0x55c1c54aab10 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.830+0000 7f21b31f2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c54ad2e0 con 0x55c1c54c2950 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21b31f2740 10 client.0 _ll_register_callbacks cb 0x7f2198009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52300/0 (socket says 172.21.15.148:52300) 2024-04-25T10:02:20.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 -- 172.21.15.148:0/1190339661 learned_addr learned my addr 172.21.15.148:0/1190339661 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ae291640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c5478e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21aea92640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54c2950 0x55c1c54c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 -- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54c2950 msgr2=0x55c1c54c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54c2950 0x55c1c54c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 -- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 msgr2=0x55c1c5478e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c5478e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 -- 172.21.15.148:0/1190339661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1c50bb0f0 con 0x55c1c54aaef0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21ada90640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54c3710 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2198006e40 tx=0x7f2198004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad0f106abbeab5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21897fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21897fa640 1 -- 172.21.15.148:0/1190339661 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198034060 con 0x55c1c54aaef0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21897fa640 1 -- 172.21.15.148:0/1190339661 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2198003800 con 0x55c1c54aaef0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f219800fcb0 con 0x55c1c54aaef0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21897fa640 1 -- 172.21.15.148:0/1190339661 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f219803a050 con 0x55c1c54aaef0 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21b31f2740 10 client.9084 fetch_fsmap learned FSMap version 67 2024-04-25T10:02:20.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55c1c54a1d80 con 0x55c1c54aaef0 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21897fa640 1 -- 172.21.15.148:0/1190339661 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800fcb0 con 0x55c1c54aaef0 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.831+0000 7f21897fa640 1 -- 172.21.15.148:0/1190339661 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f219803d020 con 0x55c1c54aaef0 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 10 client.9084 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 10 client.9084 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 20 client.9084 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54c3c50 0x55c1c54c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 4 client.9084 mds_command: new command op to 8709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 --> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1c54a2ce0 con 0x55c1c54c3c50 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21ae291640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54c3c50 0x55c1c54c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21897fa640 10 client.9084 ms_handle_connect on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21ae291640 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54c3c50 0x55c1c54c4b60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb88d7723bd1b2bf server_cookie=f0c90c377b61b7aa in_seq=0 out_seq=0 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21897fa640 1 -- 172.21.15.148:0/1190339661 <== mds.0 v2:172.21.15.188:6833/609722939 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c1c54a2ce0 con 0x55c1c54c3c50 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21897fa640 10 client.9084 handle_command_reply: tid=0 2024-04-25T10:02:20.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 client.9084 shutdown 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 msgr2=0x55c1c54c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54c3710 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2198006e40 tx=0x7f2198004680 comp rx=0 tx=0).stop 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 shutdown_connections 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:6833/609722939,v1:172.21.15.188:6835/609722939] conn(0x55c1c54c3c50 0x55c1c54c4b60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1c54c2950 0x55c1c54c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1c54aaef0 0x55c1c54c3710 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21897fa640 0 client.9084 ms_handle_reset on v2:172.21.15.188:6833/609722939 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 --2- 172.21.15.148:0/1190339661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1c54aab10 0x55c1c5478e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.832+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 >> 172.21.15.148:0/1190339661 conn(0x55c1c547d770 msgr2=0x55c1c54aa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 shutdown_connections 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b31f2740 1 -- 172.21.15.148:0/1190339661 wait complete. 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b31f2740 20 client.9084 trim_cache size 0 max 16384 2024-04-25T10:02:20.843 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:20.844 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:02:20.844 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:02:20.844 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f218c06b4d0 msgr2=0x7f218c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f218c06b4d0 0x7f218c06d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f21a815cdb0 tx=0x7f21a4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 msgr2=0x7f21a81492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 0x7f21a81492c0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009e50 tx=0x7f21a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 shutdown_connections 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f218c06b4d0 0x7f218c06d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a815dd80 0x7f21a81492c0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21a80ad570 0x7f21a81592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 --2- 172.21.15.148:0/2907359177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21a806abe0 0x7f21a815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 >> 172.21.15.148:0/2907359177 conn(0x7f21a80b5af0 msgr2=0x7f21a80b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 shutdown_connections 2024-04-25T10:02:20.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.833+0000 7f21b04f5640 1 -- 172.21.15.148:0/2907359177 wait complete. 2024-04-25T10:02:20.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:02:20.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:20.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.941+0000 7fe8a8807640 1 Processor -- start 2024-04-25T10:02:20.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.941+0000 7fe8a8807640 1 -- start start 2024-04-25T10:02:20.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.941+0000 7fe8a8807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a40d13f0 0x7fe8a40d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a8807640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8a40d1d90 0x7fe8a40b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a8807640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40d0350 con 0x7fe8a415bd30 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a8807640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40d0630 con 0x7fe8a40d1d90 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a8807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40d0910 con 0x7fe8a40d13f0 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8a40d1d90 0x7fe8a40b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8a40d1d90 0x7fe8a40b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52302/0 (socket says 172.21.15.148:52302) 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 -- 172.21.15.148:0/1007696870 learned_addr learned my addr 172.21.15.148:0/1007696870 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a1d9b640 1 --2- 172.21.15.148:0/1007696870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a40d13f0 0x7fe8a40d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 -- 172.21.15.148:0/1007696870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a40d13f0 msgr2=0x7fe8a40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 --2- 172.21.15.148:0/1007696870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a40d13f0 0x7fe8a40d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 -- 172.21.15.148:0/1007696870 >> v1:172.21.15.188:6789/0 conn(0x7fe8a415bd30 legacy=0x7fe8a40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.942+0000 7fe8a2d9d640 1 -- 172.21.15.148:0/1007696870 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40d0bf0 con 0x7fe8a40d1d90 2024-04-25T10:02:20.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a2d9d640 1 --2- 172.21.15.148:0/1007696870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a40b4a30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009e50 tx=0x7fe89c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b8ad81fc2e66cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a159a640 1 -- 172.21.15.148:0/1007696870 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c036070 con 0x7fe8a40d1d90 2024-04-25T10:02:20.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a159a640 1 -- 172.21.15.148:0/1007696870 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c031600 con 0x7fe8a40d1d90 2024-04-25T10:02:20.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a159a640 1 -- 172.21.15.148:0/1007696870 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0319d0 con 0x7fe8a40d1d90 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a8807640 1 -- 172.21.15.148:0/1007696870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 msgr2=0x7fe8a40b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a8807640 1 --2- 172.21.15.148:0/1007696870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a40b4a30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009e50 tx=0x7fe89c02a750 comp rx=0 tx=0).stop 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a8807640 1 -- 172.21.15.148:0/1007696870 shutdown_connections 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.943+0000 7fe8a8807640 1 --2- 172.21.15.148:0/1007696870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a40b4a30 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.944+0000 7fe8a8807640 1 --2- 172.21.15.148:0/1007696870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a40d13f0 0x7fe8a40d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.944+0000 7fe8a8807640 1 -- 172.21.15.148:0/1007696870 >> 172.21.15.148:0/1007696870 conn(0x7fe8a406f590 msgr2=0x7fe8a406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.944+0000 7fe8a8807640 1 -- 172.21.15.148:0/1007696870 shutdown_connections 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.944+0000 7fe8a8807640 1 -- 172.21.15.148:0/1007696870 wait complete. 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.945+0000 7fe8a8807640 1 Processor -- start 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.945+0000 7fe8a8807640 1 -- start start 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8a40d13f0 0x7fe8a41f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a4157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a415bd30 0x7fe8a4155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40d0010 con 0x7fe8a40d1d90 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415c110 con 0x7fe8a40d13f0 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b73b0 con 0x7fe8a415bd30 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a4157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a4157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52310/0 (socket says 172.21.15.148:52310) 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 -- 172.21.15.148:0/3479064962 learned_addr learned my addr 172.21.15.148:0/3479064962 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:20.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a259c640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8a40d13f0 0x7fe8a41f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a2d9d640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a415bd30 0x7fe8a4155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 -- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a415bd30 msgr2=0x7fe8a4155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a415bd30 0x7fe8a4155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 -- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8a40d13f0 msgr2=0x7fe8a41f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8a40d13f0 0x7fe8a41f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 -- 172.21.15.148:0/3479064962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a4157b30 con 0x7fe8a40d1d90 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a2d9d640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a415bd30 0x7fe8a4155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a259c640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8a40d13f0 0x7fe8a41f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a1d9b640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a4157470 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c7f0 tx=0x7fe88c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c015070 con 0x7fe8a40d1d90 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c004240 con 0x7fe8a40d1d90 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a4157e10 con 0x7fe8a40d1d90 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c007620 con 0x7fe8a40d1d90 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.946+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a41583d0 con 0x7fe8a40d1d90 2024-04-25T10:02:20.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.947+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4073990 con 0x7fe8a40d1d90 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.948+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe88c0079d0 con 0x7fe8a40d1d90 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.948+0000 7fe89affd640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe88406b4d0 0x7fe88406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.948+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe88c08a490 con 0x7fe8a40d1d90 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.948+0000 7fe8a259c640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe88406b4d0 0x7fe88406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.949+0000 7fe8a259c640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe88406b4d0 0x7fe88406d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40b2490 tx=0x7fe894008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:20.952+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe88c059180 con 0x7fe8a40d1d90 2024-04-25T10:02:21.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.161+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8a40b21e0 con 0x7fe8a40d1d90 2024-04-25T10:02:21.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.162+0000 7fe89affd640 1 -- 172.21.15.148:0/3479064962 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe88c058a80 con 0x7fe8a40d1d90 2024-04-25T10:02:21.167 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:21.167 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:21.167 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe88406b4d0 msgr2=0x7fe88406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe88406b4d0 0x7fe88406d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40b2490 tx=0x7fe894008040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 msgr2=0x7fe8a4157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a4157470 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c7f0 tx=0x7fe88c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 shutdown_connections 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe88406b4d0 0x7fe88406d990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8a415bd30 0x7fe8a4155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8a40d1d90 0x7fe8a4157470 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 --2- 172.21.15.148:0/3479064962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8a40d13f0 0x7fe8a41f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 >> 172.21.15.148:0/3479064962 conn(0x7fe8a406f590 msgr2=0x7fe8a40b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 shutdown_connections 2024-04-25T10:02:21.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.164+0000 7fe8a8807640 1 -- 172.21.15.148:0/3479064962 wait complete. 2024-04-25T10:02:21.179 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:21.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:21.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.271+0000 7f1a6d7f4640 1 Processor -- start 2024-04-25T10:02:21.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.271+0000 7f1a6d7f4640 1 -- start start 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a6d7f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a6815a790 0x7f1a680d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a6d7f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a6806fea0 0x7f1a680d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a6d7f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a680b3710 con 0x7f1a680af9c0 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a6d7f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680cf0e0 con 0x7f1a6806fea0 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a6d7f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680cf3c0 con 0x7f1a6815a790 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a6806fea0 0x7f1a680d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1a6806fea0 0x7f1a680d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52316/0 (socket says 172.21.15.148:52316) 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 -- 172.21.15.148:0/2536835103 learned_addr learned my addr 172.21.15.148:0/2536835103 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:21.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a66ffd640 1 --2- 172.21.15.148:0/2536835103 >> v2:172.21.15.188:3300/0 conn(0x7f1a6815a790 0x7f1a680d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 -- 172.21.15.148:0/2536835103 >> v2:172.21.15.188:3300/0 conn(0x7f1a6815a790 msgr2=0x7f1a680d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 --2- 172.21.15.148:0/2536835103 >> v2:172.21.15.188:3300/0 conn(0x7f1a6815a790 0x7f1a680d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 -- 172.21.15.148:0/2536835103 >> v1:172.21.15.188:6789/0 conn(0x7f1a680af9c0 legacy=0x7f1a68072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.272+0000 7f1a667fc640 1 -- 172.21.15.148:0/2536835103 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680cf6a0 con 0x7f1a6806fea0 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.273+0000 7f1a667fc640 1 --2- 172.21.15.148:0/2536835103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6806fea0 0x7f1a680d08d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f1a680b2e90 tx=0x7f1a5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcda8a78ec43f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.273+0000 7f1a65ffb640 1 -- 172.21.15.148:0/2536835103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c037070 con 0x7f1a6806fea0 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.273+0000 7f1a65ffb640 1 -- 172.21.15.148:0/2536835103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c032440 con 0x7f1a6806fea0 2024-04-25T10:02:21.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.273+0000 7f1a65ffb640 1 -- 172.21.15.148:0/2536835103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c032810 con 0x7f1a6806fea0 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/2536835103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6806fea0 msgr2=0x7f1a680d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/2536835103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6806fea0 0x7f1a680d08d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f1a680b2e90 tx=0x7f1a5c02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/2536835103 shutdown_connections 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/2536835103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6806fea0 0x7f1a680d08d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/2536835103 >> v2:172.21.15.188:3300/0 conn(0x7f1a6815a790 0x7f1a680d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/2536835103 >> 172.21.15.148:0/2536835103 conn(0x7f1a680b2370 msgr2=0x7f1a680b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/2536835103 shutdown_connections 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.274+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/2536835103 wait complete. 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.275+0000 7f1a6d7f4640 1 Processor -- start 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.275+0000 7f1a6d7f4640 1 -- start start 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a6d7f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6806fea0 0x7f1a681f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a6d7f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a680af9c0 0x7f1a68154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a6d7f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 0x7f1a681e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a6d7f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6815ab70 con 0x7f1a6815a790 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a6d7f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cf980 con 0x7f1a6806fea0 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a6d7f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6815acf0 con 0x7f1a680af9c0 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 0x7f1a681e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a66ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6806fea0 0x7f1a681f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 0x7f1a681e48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52330/0 (socket says 172.21.15.148:52330) 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 -- 172.21.15.148:0/3004162847 learned_addr learned my addr 172.21.15.148:0/3004162847 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a667fc640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a680af9c0 0x7f1a68154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 -- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a680af9c0 msgr2=0x7f1a68154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a680af9c0 0x7f1a68154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 -- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6806fea0 msgr2=0x7f1a681f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6806fea0 0x7f1a681f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a677fe640 1 -- 172.21.15.148:0/3004162847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a681f16f0 con 0x7f1a6815a790 2024-04-25T10:02:21.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.276+0000 7f1a66ffd640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6806fea0 0x7f1a681f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.277+0000 7f1a677fe640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 0x7f1a681e48a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c3c0 tx=0x7f1a6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.277+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60017070 con 0x7f1a6815a790 2024-04-25T10:02:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.277+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a600052d0 con 0x7f1a6815a790 2024-04-25T10:02:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.277+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60003850 con 0x7f1a6815a790 2024-04-25T10:02:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.277+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a681e3790 con 0x7f1a6815a790 2024-04-25T10:02:21.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.277+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a681e3d10 con 0x7f1a6815a790 2024-04-25T10:02:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.278+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a68158b90 con 0x7f1a6815a790 2024-04-25T10:02:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.278+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a6001a070 con 0x7f1a6815a790 2024-04-25T10:02:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.279+0000 7f1a4f7fe640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a5406b4d0 0x7f1a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.279+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a600895f0 con 0x7f1a6815a790 2024-04-25T10:02:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.279+0000 7f1a66ffd640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a5406b4d0 0x7f1a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.279+0000 7f1a66ffd640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a5406b4d0 0x7f1a5406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e740 tx=0x7f1a5800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.283+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a68158b90 con 0x7f1a6815a790 2024-04-25T10:02:21.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.519+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1a6806aff0 con 0x7f1a6815a790 2024-04-25T10:02:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.519+0000 7f1a4f7fe640 1 -- 172.21.15.148:0/3004162847 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f1a60057d20 con 0x7f1a6815a790 2024-04-25T10:02:21.524 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:21.524 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":502,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562785280,"bytes_avail":770531328000,"bytes_total":773094113280,"write_bytes_sec":77075918,"read_op_per_sec":0,"write_op_per_sec":75},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:02:19.118650+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a5406b4d0 msgr2=0x7f1a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a5406b4d0 0x7f1a5406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800e740 tx=0x7f1a5800b040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 msgr2=0x7f1a681e48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 0x7f1a681e48a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c3c0 tx=0x7f1a6000c880 comp rx=0 tx=0).stop 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 shutdown_connections 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a5406b4d0 0x7f1a5406d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6815a790 0x7f1a681e48a0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a680af9c0 0x7f1a68154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 --2- 172.21.15.148:0/3004162847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a6806fea0 0x7f1a681f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 >> 172.21.15.148:0/3004162847 conn(0x7f1a680b2370 msgr2=0x7f1a68157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 shutdown_connections 2024-04-25T10:02:21.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:21.521+0000 7f1a6d7f4640 1 -- 172.21.15.148:0/3004162847 wait complete. 2024-04-25T10:02:21.536 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:02:22.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.642+0000 7f0e919be640 1 Processor -- start 2024-04-25T10:02:22.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.642+0000 7f0e919be640 1 -- start start 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e919be640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e8c15dba0 0x7f0e8c06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e919be640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c06c9e0 0x7f0e8c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e919be640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e8c071200 0x7f0e8c161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e919be640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c15df80 con 0x7f0e8c071200 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e919be640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8c0cf380 con 0x7f0e8c15dba0 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e919be640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0ce770 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e8b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e8c071200 0x7f0e8c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e8b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e8c071200 0x7f0e8c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52336/0 (socket says 172.21.15.148:52336) 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.643+0000 7f0e8b7fe640 1 -- 172.21.15.148:0/808150400 learned_addr learned my addr 172.21.15.148:0/808150400 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8affd640 1 --2- 172.21.15.148:0/808150400 >> v2:172.21.15.188:3300/0 conn(0x7f0e8c15dba0 0x7f0e8c06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8a7fc640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c06c9e0 0x7f0e8c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8b7fe640 1 -- 172.21.15.148:0/808150400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c06c9e0 msgr2=0x7f0e8c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8b7fe640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c06c9e0 0x7f0e8c070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8b7fe640 1 -- 172.21.15.148:0/808150400 >> v2:172.21.15.188:3300/0 conn(0x7f0e8c15dba0 msgr2=0x7f0e8c06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8b7fe640 1 --2- 172.21.15.148:0/808150400 >> v2:172.21.15.188:3300/0 conn(0x7f0e8c15dba0 0x7f0e8c06c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8b7fe640 1 -- 172.21.15.148:0/808150400 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0cea50 con 0x7f0e8c071200 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8affd640 1 --2- 172.21.15.148:0/808150400 >> v2:172.21.15.188:3300/0 conn(0x7f0e8c15dba0 0x7f0e8c06c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8a7fc640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c06c9e0 0x7f0e8c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.644+0000 7f0e8b7fe640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c071200 0x7f0e8c161680 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009870 tx=0x7f0e8402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e314ea9f5821bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e89ffb640 1 -- 172.21.15.148:0/808150400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84037070 con 0x7f0e8c071200 2024-04-25T10:02:22.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e89ffb640 1 -- 172.21.15.148:0/808150400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e84032440 con 0x7f0e8c071200 2024-04-25T10:02:22.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e89ffb640 1 -- 172.21.15.148:0/808150400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84032810 con 0x7f0e8c071200 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 -- 172.21.15.148:0/808150400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c071200 msgr2=0x7f0e8c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c071200 0x7f0e8c161680 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009870 tx=0x7f0e8402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 -- 172.21.15.148:0/808150400 shutdown_connections 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c071200 0x7f0e8c161680 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 --2- 172.21.15.148:0/808150400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c06c9e0 0x7f0e8c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 --2- 172.21.15.148:0/808150400 >> v2:172.21.15.188:3300/0 conn(0x7f0e8c15dba0 0x7f0e8c06c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 -- 172.21.15.148:0/808150400 >> 172.21.15.148:0/808150400 conn(0x7f0e8c06f520 msgr2=0x7f0e8c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 -- 172.21.15.148:0/808150400 shutdown_connections 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.645+0000 7f0e919be640 1 -- 172.21.15.148:0/808150400 wait complete. 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.646+0000 7f0e919be640 1 Processor -- start 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 -- start start 2024-04-25T10:02:22.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 0x7f0e8c1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c071200 0x7f0e8c160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e8c15dba0 0x7f0e8c15fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0b0ce0 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0cf010 con 0x7f0e8c15dba0 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.647+0000 7f0e919be640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c161bc0 con 0x7f0e8c071200 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 0x7f0e8c1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 0x7f0e8c1f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52340/0 (socket says 172.21.15.148:52340) 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 -- 172.21.15.148:0/3904107579 learned_addr learned my addr 172.21.15.148:0/3904107579 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8b7fe640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e8c15dba0 0x7f0e8c15fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8a7fc640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c071200 0x7f0e8c160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 -- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c071200 msgr2=0x7f0e8c160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c071200 0x7f0e8c160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 -- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e8c15dba0 msgr2=0x7f0e8c15fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e8c15dba0 0x7f0e8c15fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 -- 172.21.15.148:0/3904107579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c161170 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8a7fc640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c071200 0x7f0e8c160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e8affd640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 0x7f0e8c1f4070 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c007ef0 tx=0x7f0e7c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01a070 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c003b90 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c15b5e0 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c004030 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.648+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c15bba0 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.649+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c073990 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.650+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e7c01d070 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.650+0000 7f0e73fff640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e7806b500 0x7f0e7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.650+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e7c08a450 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.650+0000 7f0e8a7fc640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e7806b500 0x7f0e7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.651+0000 7f0e8a7fc640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e7806b500 0x7f0e7806d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c06d520 tx=0x7f0e80008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.655+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e7c010070 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.858+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e8c0702e0 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.858+0000 7f0e73fff640 1 -- 172.21.15.148:0/3904107579 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0e7c058b80 con 0x7f0e8c06c9e0 2024-04-25T10:02:22.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:22.863 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:22.863 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e7806b500 msgr2=0x7f0e7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e7806b500 0x7f0e7806d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c06d520 tx=0x7f0e80008040 comp rx=0 tx=0).stop 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 msgr2=0x7f0e8c1f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 0x7f0e8c1f4070 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c007ef0 tx=0x7f0e7c00d4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 shutdown_connections 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e7806b500 0x7f0e7806d9c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e8c15dba0 0x7f0e8c15fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e8c071200 0x7f0e8c160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 --2- 172.21.15.148:0/3904107579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e8c06c9e0 0x7f0e8c1f4070 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 >> 172.21.15.148:0/3904107579 conn(0x7f0e8c06f520 msgr2=0x7f0e8c0d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 shutdown_connections 2024-04-25T10:02:22.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.860+0000 7f0e919be640 1 -- 172.21.15.148:0/3904107579 wait complete. 2024-04-25T10:02:22.876 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:22.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:22.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.967+0000 7f864a4e4640 1 Processor -- start 2024-04-25T10:02:22.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.967+0000 7f864a4e4640 1 -- start start 2024-04-25T10:02:22.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.967+0000 7f864a4e4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f864415b9c0 0x7f864415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.967+0000 7f864a4e4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8644073150 con 0x7f864406abe0 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.967+0000 7f864a4e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86440cfa60 con 0x7f8644072d30 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.967+0000 7f864a4e4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86440cfd40 con 0x7f864415b9c0 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f8643fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f864415b9c0 0x7f864415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f86437fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f864406abe0 0x7f8644072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:60380/0 (socket says 172.21.15.148:60380) 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f8643fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f864415b9c0 0x7f864415bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52348/0 (socket says 172.21.15.148:52348) 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f86437fe640 1 -- 172.21.15.148:0/2836411547 learned_addr learned my addr 172.21.15.148:0/2836411547 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f8643fff640 1 -- 172.21.15.148:0/2836411547 >> v1:172.21.15.188:6789/0 conn(0x7f8644072d30 legacy=0x7f86440d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f8643fff640 1 -- 172.21.15.148:0/2836411547 >> v1:172.21.15.148:6789/0 conn(0x7f864406abe0 legacy=0x7f8644072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.968+0000 7f8643fff640 1 -- 172.21.15.148:0/2836411547 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440d0020 con 0x7f864415b9c0 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f8643fff640 1 --2- 172.21.15.148:0/2836411547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864415b9c0 0x7f864415bda0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f86380097a0 tx=0x7f863802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a102a05b62d39da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f8642ffd640 1 -- 172.21.15.148:0/2836411547 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638037070 con 0x7f864415b9c0 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f8642ffd640 1 -- 172.21.15.148:0/2836411547 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8638032440 con 0x7f864415b9c0 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f8642ffd640 1 -- 172.21.15.148:0/2836411547 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638032810 con 0x7f864415b9c0 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 -- 172.21.15.148:0/2836411547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864415b9c0 msgr2=0x7f864415bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 --2- 172.21.15.148:0/2836411547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864415b9c0 0x7f864415bda0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f86380097a0 tx=0x7f863802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 -- 172.21.15.148:0/2836411547 shutdown_connections 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 --2- 172.21.15.148:0/2836411547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864415b9c0 0x7f864415bda0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 -- 172.21.15.148:0/2836411547 >> 172.21.15.148:0/2836411547 conn(0x7f86440b2370 msgr2=0x7f86440b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 -- 172.21.15.148:0/2836411547 shutdown_connections 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.969+0000 7f864a4e4640 1 -- 172.21.15.148:0/2836411547 wait complete. 2024-04-25T10:02:22.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.970+0000 7f864a4e4640 1 Processor -- start 2024-04-25T10:02:22.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 -- start start 2024-04-25T10:02:22.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 0x7f86440cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8644072d30 0x7f8644159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f864415b9c0 0x7f86440c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864415c2e0 con 0x7f864406abe0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cf710 con 0x7f8644072d30 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f864a4e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864406b070 con 0x7f864415b9c0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f8643fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 0x7f86440cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f8643fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 0x7f86440cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52356/0 (socket says 172.21.15.148:52356) 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.971+0000 7f8643fff640 1 -- 172.21.15.148:0/1153175294 learned_addr learned my addr 172.21.15.148:0/1153175294 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8643fff640 1 -- 172.21.15.148:0/1153175294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f864415b9c0 msgr2=0x7f86440c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8643fff640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f864415b9c0 0x7f86440c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8643fff640 1 -- 172.21.15.148:0/1153175294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8644072d30 msgr2=0x7f8644159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8643fff640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8644072d30 0x7f8644159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8643fff640 1 -- 172.21.15.148:0/1153175294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440b3c50 con 0x7f864406abe0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8643fff640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 0x7f86440cbb10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8638000c00 tx=0x7f86380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638041070 con 0x7f864406abe0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863803c930 con 0x7f864406abe0 2024-04-25T10:02:22.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86440b3f30 con 0x7f864406abe0 2024-04-25T10:02:22.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638037040 con 0x7f864406abe0 2024-04-25T10:02:22.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.972+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86440b44f0 con 0x7f864406abe0 2024-04-25T10:02:22.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.973+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8644157410 con 0x7f864406abe0 2024-04-25T10:02:22.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.974+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8638044070 con 0x7f864406abe0 2024-04-25T10:02:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.974+0000 7f8640ff9640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.974+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f86380ab330 con 0x7f864406abe0 2024-04-25T10:02:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.974+0000 7f86437fe640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.974+0000 7f86437fe640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f863006b4d0 0x7f863006d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8634000f60 tx=0x7f8634007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:22.978+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8644157410 con 0x7f864406abe0 2024-04-25T10:02:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.215+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f86440c99f0 con 0x7f864406abe0 2024-04-25T10:02:23.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.215+0000 7f8640ff9640 1 -- 172.21.15.148:0/1153175294 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f863807a0c0 con 0x7f864406abe0 2024-04-25T10:02:23.220 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:23.220 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":504,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562785280,"bytes_avail":770531328000,"bytes_total":773094113280,"write_bytes_sec":77075918,"read_op_per_sec":0,"write_op_per_sec":75},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:02:19.118650+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:02:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f863006b4d0 msgr2=0x7f863006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f863006b4d0 0x7f863006d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8634000f60 tx=0x7f8634007a80 comp rx=0 tx=0).stop 2024-04-25T10:02:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 msgr2=0x7f86440cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 0x7f86440cbb10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8638000c00 tx=0x7f86380046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 shutdown_connections 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f864415b9c0 0x7f86440c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8644072d30 0x7f8644159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 --2- 172.21.15.148:0/1153175294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864406abe0 0x7f86440cbb10 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 >> 172.21.15.148:0/1153175294 conn(0x7f86440b2370 msgr2=0x7f86440b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 shutdown_connections 2024-04-25T10:02:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:23.217+0000 7f864a4e4640 1 -- 172.21.15.148:0/1153175294 wait complete. 2024-04-25T10:02:23.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:02:24.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:24.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.339+0000 7fe0e0012640 1 Processor -- start 2024-04-25T10:02:24.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.339+0000 7fe0e0012640 1 -- start start 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0e0012640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d8154ee0 0x7fe0d80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0e0012640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0d80cfcf0 0x7fe0d80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0e0012640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d8071480 con 0x7fe0d80ce3c0 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0e0012640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d80ce830 con 0x7fe0d80cfcf0 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0e0012640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d8072090 con 0x7fe0d8154ee0 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0d80cfcf0 0x7fe0d80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0d80cfcf0 0x7fe0d80cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39868/0 (socket says 172.21.15.148:39868) 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 -- 172.21.15.148:0/471766467 learned_addr learned my addr 172.21.15.148:0/471766467 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dddae640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d8154ee0 0x7fe0d80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 -- 172.21.15.148:0/471766467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d8154ee0 msgr2=0x7fe0d80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d8154ee0 0x7fe0d80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 -- 172.21.15.148:0/471766467 >> v1:172.21.15.188:6789/0 conn(0x7fe0d80ce3c0 legacy=0x7fe0d815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dd5ad640 1 -- 172.21.15.148:0/471766467 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8071760 con 0x7fe0d80cfcf0 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.340+0000 7fe0dddae640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d8154ee0 0x7fe0d80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.341+0000 7fe0dd5ad640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d80cde80 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009870 tx=0x7fe0c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8d2e14bd65279f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.341+0000 7fe0dcdac640 1 -- 172.21.15.148:0/471766467 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8037070 con 0x7fe0d80cfcf0 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.341+0000 7fe0dcdac640 1 -- 172.21.15.148:0/471766467 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c8032440 con 0x7fe0d80cfcf0 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.341+0000 7fe0dcdac640 1 -- 172.21.15.148:0/471766467 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8032810 con 0x7fe0d80cfcf0 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 -- 172.21.15.148:0/471766467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d80cfcf0 msgr2=0x7fe0d80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d80cde80 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009870 tx=0x7fe0c802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 -- 172.21.15.148:0/471766467 shutdown_connections 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d80cde80 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 --2- 172.21.15.148:0/471766467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d8154ee0 0x7fe0d80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 -- 172.21.15.148:0/471766467 >> 172.21.15.148:0/471766467 conn(0x7fe0d806abe0 msgr2=0x7fe0d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 -- 172.21.15.148:0/471766467 shutdown_connections 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.342+0000 7fe0e0012640 1 -- 172.21.15.148:0/471766467 wait complete. 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.343+0000 7fe0e0012640 1 Processor -- start 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.343+0000 7fe0e0012640 1 -- start start 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d80ce3c0 0x7fe0d80b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 0x7fe0d8136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d8071230 con 0x7fe0d8154ee0 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d8071d20 con 0x7fe0d80cfcf0 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d81552c0 con 0x7fe0d80ce3c0 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 0x7fe0d8136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 0x7fe0d8136b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39876/0 (socket says 172.21.15.148:39876) 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 -- 172.21.15.148:0/19564129 learned_addr learned my addr 172.21.15.148:0/19564129 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0dd5ad640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0dddae640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d80ce3c0 0x7fe0d80b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 -- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d80ce3c0 msgr2=0x7fe0d80b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d80ce3c0 0x7fe0d80b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 -- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d80cfcf0 msgr2=0x7fe0d814c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d814c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 -- 172.21.15.148:0/19564129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d8154140 con 0x7fe0d8154ee0 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0de5af640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 0x7fe0d8136b00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000c7a0 tx=0x7fe0d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0003b70 con 0x7fe0d8154ee0 2024-04-25T10:02:24.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d0003d30 con 0x7fe0d8154ee0 2024-04-25T10:02:24.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d000aa90 con 0x7fe0d8154ee0 2024-04-25T10:02:24.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.344+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d8154420 con 0x7fe0d8154ee0 2024-04-25T10:02:24.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.345+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d81488f0 con 0x7fe0d8154ee0 2024-04-25T10:02:24.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.345+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d8073990 con 0x7fe0d8154ee0 2024-04-25T10:02:24.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.346+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0d000ac30 con 0x7fe0d8154ee0 2024-04-25T10:02:24.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.346+0000 7fe0ce7fc640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.346+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0d008a200 con 0x7fe0d8154ee0 2024-04-25T10:02:24.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.346+0000 7fe0dddae640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.347+0000 7fe0dddae640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe0d81f4f50 tx=0x7fe0d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.350+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0d0058820 con 0x7fe0d8154ee0 2024-04-25T10:02:24.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.559+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0d806d360 con 0x7fe0d8154ee0 2024-04-25T10:02:24.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.560+0000 7fe0ce7fc640 1 -- 172.21.15.148:0/19564129 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe0d005c8c0 con 0x7fe0d8154ee0 2024-04-25T10:02:24.564 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:24.565 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:24.565 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.561+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0bc06b4d0 msgr2=0x7fe0bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.561+0000 7fe0e0012640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe0d81f4f50 tx=0x7fe0d4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.561+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 msgr2=0x7fe0d8136b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 0x7fe0d8136b00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000c7a0 tx=0x7fe0d000cc60 comp rx=0 tx=0).stop 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 shutdown_connections 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0bc06b4d0 0x7fe0bc06d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d8154ee0 0x7fe0d8136b00 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d80cfcf0 0x7fe0d814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 --2- 172.21.15.148:0/19564129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d80ce3c0 0x7fe0d80b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 >> 172.21.15.148:0/19564129 conn(0x7fe0d806abe0 msgr2=0x7fe0d80cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 shutdown_connections 2024-04-25T10:02:24.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.562+0000 7fe0e0012640 1 -- 172.21.15.148:0/19564129 wait complete. 2024-04-25T10:02:24.577 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:24.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:24.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.668+0000 7f80023c4640 1 Processor -- start 2024-04-25T10:02:24.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.668+0000 7f80023c4640 1 -- start start 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f80023c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ffc15dbe0 0x7f7ffc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f80023c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b2cb0 0x7f7ffc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f80023c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ffc0b97a0 con 0x7f7ffc0b92a0 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f80023c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ffc15dfc0 con 0x7f7ffc15dbe0 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f80023c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0d1a00 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f7ffbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ffc15dbe0 0x7f7ffc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f7ffbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7ffc15dbe0 0x7f7ffc0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39882/0 (socket says 172.21.15.148:39882) 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.669+0000 7f7ffbfff640 1 -- 172.21.15.148:0/2980776983 learned_addr learned my addr 172.21.15.148:0/2980776983 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffbfff640 1 -- 172.21.15.148:0/2980776983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b2cb0 msgr2=0x7f7ffc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffbfff640 1 --2- 172.21.15.148:0/2980776983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b2cb0 0x7f7ffc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffbfff640 1 -- 172.21.15.148:0/2980776983 >> v1:172.21.15.188:6789/0 conn(0x7f7ffc0b92a0 legacy=0x7f7ffc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffbfff640 1 -- 172.21.15.148:0/2980776983 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc0d1c50 con 0x7f7ffc15dbe0 2024-04-25T10:02:24.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffbfff640 1 --2- 172.21.15.148:0/2980776983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0d0910 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b074b372bc5ade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffaffd640 1 -- 172.21.15.148:0/2980776983 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec037070 con 0x7f7ffc15dbe0 2024-04-25T10:02:24.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffaffd640 1 -- 172.21.15.148:0/2980776983 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec032440 con 0x7f7ffc15dbe0 2024-04-25T10:02:24.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.670+0000 7f7ffaffd640 1 -- 172.21.15.148:0/2980776983 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec032810 con 0x7f7ffc15dbe0 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 -- 172.21.15.148:0/2980776983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc15dbe0 msgr2=0x7f7ffc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 --2- 172.21.15.148:0/2980776983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0d0910 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec02a210 comp rx=0 tx=0).stop 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 -- 172.21.15.148:0/2980776983 shutdown_connections 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 --2- 172.21.15.148:0/2980776983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b2cb0 0x7f7ffc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 --2- 172.21.15.148:0/2980776983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0d0910 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 -- 172.21.15.148:0/2980776983 >> 172.21.15.148:0/2980776983 conn(0x7f7ffc061a20 msgr2=0x7f7ffc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 -- 172.21.15.148:0/2980776983 shutdown_connections 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.671+0000 7f80023c4640 1 -- 172.21.15.148:0/2980776983 wait complete. 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.672+0000 7f80023c4640 1 Processor -- start 2024-04-25T10:02:24.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.672+0000 7f80023c4640 1 -- start start 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f80023c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 0x7f7ffc1459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f80023c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b92a0 0x7f7ffc15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f80023c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f80023c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0b6ab0 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f80023c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0d2210 con 0x7f7ffc15dbe0 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f80023c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0b6c30 con 0x7f7ffc0b92a0 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 0x7f7ffc1459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 0x7f7ffc1459a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39888/0 (socket says 172.21.15.148:39888) 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 -- 172.21.15.148:0/3310494066 learned_addr learned my addr 172.21.15.148:0/3310494066 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f8000961640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffb7fe640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b92a0 0x7f7ffc15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 -- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b92a0 msgr2=0x7f7ffc15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b92a0 0x7f7ffc15f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 -- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc15dbe0 msgr2=0x7f7ffc0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.673+0000 7f7ffbfff640 1 -- 172.21.15.148:0/3310494066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc145ee0 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.674+0000 7f7ffbfff640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 0x7f7ffc1459a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.674+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec041070 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.674+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec03c870 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.674+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc0b4630 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.674+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec037040 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.674+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc0b4ed0 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.675+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ffc073990 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.675+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fec044070 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.676+0000 7f7ff8ff9640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe806b4d0 0x7f7fe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.676+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fec0aada0 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.676+0000 7f7ffb7fe640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe806b4d0 0x7f7fe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.676+0000 7f7ffb7fe640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe806b4d0 0x7f7fe806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0b4520 tx=0x7f7ff0009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.680+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fec0794d0 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:24.917 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:24.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.923+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7ffc0b8de0 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.923+0000 7f7ff8ff9640 1 -- 172.21.15.148:0/3310494066 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f7fec07d570 con 0x7f7ffc0b2cb0 2024-04-25T10:02:24.928 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:24.928 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":506,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562785280,"bytes_avail":770531328000,"bytes_total":773094113280,"write_bytes_sec":30331138,"read_op_per_sec":0,"write_op_per_sec":30},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T10:02:19.118650+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:02:24.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe806b4d0 msgr2=0x7f7fe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe806b4d0 0x7f7fe806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0b4520 tx=0x7f7ff0009060 comp rx=0 tx=0).stop 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 msgr2=0x7f7ffc1459a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 0x7f7ffc1459a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 shutdown_connections 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe806b4d0 0x7f7fe806d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc15dbe0 0x7f7ffc0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0b92a0 0x7f7ffc15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 --2- 172.21.15.148:0/3310494066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc0b2cb0 0x7f7ffc1459a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.925+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 >> 172.21.15.148:0/3310494066 conn(0x7f7ffc061a20 msgr2=0x7f7ffc0c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.926+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 shutdown_connections 2024-04-25T10:02:24.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:24.926+0000 7f80023c4640 1 -- 172.21.15.148:0/3310494066 wait complete. 2024-04-25T10:02:24.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:02:25.942 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:26.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.044+0000 7fc084add640 1 Processor -- start 2024-04-25T10:02:26.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.044+0000 7fc084add640 1 -- start start 2024-04-25T10:02:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc084add640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc080154ee0 0x7fc0800cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc084add640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc0800cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc084add640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc080071480 con 0x7fc0800ce3c0 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc084add640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0800ce830 con 0x7fc080154ee0 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc084add640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc080072090 con 0x7fc0800cfcf0 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc077fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc0800cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc07e59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc080154ee0 0x7fc0800cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc077fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc0800cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:38124/0 (socket says 172.21.15.148:38124) 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.045+0000 7fc077fff640 1 -- 172.21.15.148:0/670814103 learned_addr learned my addr 172.21.15.148:0/670814103 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.046+0000 7fc07dd9b640 1 -- 172.21.15.148:0/670814103 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417438204 0 0) 0x7fc080071480 con 0x7fc0800ce3c0 2024-04-25T10:02:26.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.046+0000 7fc077fff640 1 -- 172.21.15.148:0/670814103 >> v2:172.21.15.188:3300/0 conn(0x7fc080154ee0 msgr2=0x7fc0800cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.046+0000 7fc077fff640 1 --2- 172.21.15.148:0/670814103 >> v2:172.21.15.188:3300/0 conn(0x7fc080154ee0 0x7fc0800cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.046+0000 7fc077fff640 1 -- 172.21.15.148:0/670814103 >> v1:172.21.15.188:6789/0 conn(0x7fc0800ce3c0 legacy=0x7fc08015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.046+0000 7fc077fff640 1 -- 172.21.15.148:0/670814103 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080071760 con 0x7fc0800cfcf0 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.046+0000 7fc077fff640 1 --2- 172.21.15.148:0/670814103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc0800cde80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b920 tx=0x7fc06802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a819941aae3118c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc07dd9b640 1 -- 172.21.15.148:0/670814103 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06802f710 con 0x7fc0800cfcf0 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc07dd9b640 1 -- 172.21.15.148:0/670814103 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06802f8b0 con 0x7fc0800cfcf0 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc07dd9b640 1 -- 172.21.15.148:0/670814103 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06802fc80 con 0x7fc0800cfcf0 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 -- 172.21.15.148:0/670814103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 msgr2=0x7fc0800cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 --2- 172.21.15.148:0/670814103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc0800cde80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b920 tx=0x7fc06802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 -- 172.21.15.148:0/670814103 shutdown_connections 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 --2- 172.21.15.148:0/670814103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc0800cde80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 --2- 172.21.15.148:0/670814103 >> v2:172.21.15.188:3300/0 conn(0x7fc080154ee0 0x7fc0800cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 -- 172.21.15.148:0/670814103 >> 172.21.15.148:0/670814103 conn(0x7fc08006abe0 msgr2=0x7fc08006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 -- 172.21.15.148:0/670814103 shutdown_connections 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.047+0000 7fc084add640 1 -- 172.21.15.148:0/670814103 wait complete. 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 Processor -- start 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 -- start start 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0800ce3c0 0x7fc08014e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc08014e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 0x7fc080150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080071230 con 0x7fc080154ee0 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080071d20 con 0x7fc0800ce3c0 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc084add640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801552c0 con 0x7fc0800cfcf0 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc07ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 0x7fc080150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc07ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 0x7fc080150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39900/0 (socket says 172.21.15.148:39900) 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.049+0000 7fc07ed9d640 1 -- 172.21.15.148:0/3596351961 learned_addr learned my addr 172.21.15.148:0/3596351961 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc077fff640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc08014e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.061 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07e59c640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0800ce3c0 0x7fc08014e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07ed9d640 1 -- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 msgr2=0x7fc08014e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07ed9d640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc08014e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07ed9d640 1 -- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0800ce3c0 msgr2=0x7fc08014e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07ed9d640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0800ce3c0 0x7fc08014e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07ed9d640 1 -- 172.21.15.148:0/3596351961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08014e750 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc07ed9d640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 0x7fc080150f60 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c760 tx=0x7fc07800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078003da0 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078005070 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078005460 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0801f0090 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.050+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc080159720 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.051+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc080073990 con 0x7fc080154ee0 2024-04-25T10:02:26.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.052+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc078005600 con 0x7fc080154ee0 2024-04-25T10:02:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.052+0000 7fc076ffd640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc06006b4d0 0x7fc06006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.052+0000 7fc07e59c640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc06006b4d0 0x7fc06006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.052+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc07808a150 con 0x7fc080154ee0 2024-04-25T10:02:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.053+0000 7fc07e59c640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc06006b4d0 0x7fc06006d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc0800ccea0 tx=0x7fc07000f040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.057+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc08014e750 con 0x7fc080154ee0 2024-04-25T10:02:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.261+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc08006d360 con 0x7fc080154ee0 2024-04-25T10:02:26.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.262+0000 7fc076ffd640 1 -- 172.21.15.148:0/3596351961 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc0780587f0 con 0x7fc080154ee0 2024-04-25T10:02:26.267 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:26.267 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:26.267 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc06006b4d0 msgr2=0x7fc06006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc06006b4d0 0x7fc06006d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc0800ccea0 tx=0x7fc07000f040 comp rx=0 tx=0).stop 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 msgr2=0x7fc080150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 0x7fc080150f60 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c760 tx=0x7fc07800cc20 comp rx=0 tx=0).stop 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 shutdown_connections 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc06006b4d0 0x7fc06006d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc080154ee0 0x7fc080150f60 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0800cfcf0 0x7fc08014e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 --2- 172.21.15.148:0/3596351961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0800ce3c0 0x7fc08014e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 >> 172.21.15.148:0/3596351961 conn(0x7fc08006abe0 msgr2=0x7fc0800b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 shutdown_connections 2024-04-25T10:02:26.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.264+0000 7fc084add640 1 -- 172.21.15.148:0/3596351961 wait complete. 2024-04-25T10:02:26.279 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:02:26.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:02:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.371+0000 7f78ba2fc640 1 Processor -- start 2024-04-25T10:02:26.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.371+0000 7f78ba2fc640 1 -- start start 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78ba2fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78b406b2f0 0x7f78b415f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78ba2fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78ba2fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40d0480 con 0x7f78b41597d0 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78ba2fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b415c1b0 con 0x7f78b406b2f0 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78ba2fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b415c490 con 0x7f78b40b2d40 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f78b406b2f0 0x7f78b415f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f78b41597d0 0x7f78b406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56370/0 (socket says 172.21.15.148:56370) 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b3fff640 1 -- 172.21.15.148:0/491415301 learned_addr learned my addr 172.21.15.148:0/491415301 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b8899640 1 --2- 172.21.15.148:0/491415301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b2ffd640 1 -- 172.21.15.148:0/491415301 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666006453 0 0) 0x7f78b40d0480 con 0x7f78b41597d0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b2ffd640 1 -- 172.21.15.148:0/491415301 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f789c003520 con 0x7f78b41597d0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b37fe640 1 -- 172.21.15.148:0/491415301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 msgr2=0x7f78b415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b37fe640 1 --2- 172.21.15.148:0/491415301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b415fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b37fe640 1 -- 172.21.15.148:0/491415301 >> v1:172.21.15.148:6789/0 conn(0x7f78b41597d0 legacy=0x7f78b406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.372+0000 7f78b37fe640 1 -- 172.21.15.148:0/491415301 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b415c770 con 0x7f78b406b2f0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.373+0000 7f78b37fe640 1 --2- 172.21.15.148:0/491415301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b415f690 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f78a80097a0 tx=0x7f78a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bfa0f650c871b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.373+0000 7f78b2ffd640 1 -- 172.21.15.148:0/491415301 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8037070 con 0x7f78b406b2f0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.373+0000 7f78b2ffd640 1 -- 172.21.15.148:0/491415301 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8032440 con 0x7f78b406b2f0 2024-04-25T10:02:26.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.373+0000 7f78b2ffd640 1 -- 172.21.15.148:0/491415301 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8032810 con 0x7f78b406b2f0 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 -- 172.21.15.148:0/491415301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 msgr2=0x7f78b415f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/491415301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b415f690 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f78a80097a0 tx=0x7f78a802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 -- 172.21.15.148:0/491415301 shutdown_connections 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/491415301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/491415301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b415f690 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 -- 172.21.15.148:0/491415301 >> 172.21.15.148:0/491415301 conn(0x7f78b40b2220 msgr2=0x7f78b40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 -- 172.21.15.148:0/491415301 shutdown_connections 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.374+0000 7f78ba2fc640 1 -- 172.21.15.148:0/491415301 wait complete. 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.375+0000 7f78ba2fc640 1 Processor -- start 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.375+0000 7f78ba2fc640 1 -- start start 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78ba2fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b40ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78ba2fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78ba2fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b41597d0 0x7f78b40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78ba2fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b406b920 con 0x7f78b406b2f0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78ba2fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b406b7a0 con 0x7f78b41597d0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78ba2fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40cc320 con 0x7f78b40b2d40 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b40ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b40ae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39928/0 (socket says 172.21.15.148:39928) 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 -- 172.21.15.148:0/2716995723 learned_addr learned my addr 172.21.15.148:0/2716995723 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 -- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 msgr2=0x7f78b40c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b8899640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b41597d0 0x7f78b40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 -- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b41597d0 msgr2=0x7f78b40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b41597d0 0x7f78b40c5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.376+0000 7f78b3fff640 1 -- 172.21.15.148:0/2716995723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40c4670 con 0x7f78b406b2f0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.377+0000 7f78b3fff640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b40ae5e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c390 tx=0x7f78a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.377+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4017070 con 0x7f78b406b2f0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.377+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a4005ce0 con 0x7f78b406b2f0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.377+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40c4950 con 0x7f78b406b2f0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.377+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40072f0 con 0x7f78b406b2f0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.377+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b40c4f10 con 0x7f78b406b2f0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.378+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b406dfc0 con 0x7f78b406b2f0 2024-04-25T10:02:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.378+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78a401a070 con 0x7f78b406b2f0 2024-04-25T10:02:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.379+0000 7f78b0ff9640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f78a006b500 0x7f78a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.379+0000 7f78b37fe640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f78a006b500 0x7f78a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.379+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78a40896d0 con 0x7f78b406b2f0 2024-04-25T10:02:26.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.379+0000 7f78b37fe640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f78a006b500 0x7f78a006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f78a8034000 tx=0x7f78a802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.383+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78b406dfc0 con 0x7f78b406b2f0 2024-04-25T10:02:26.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.619+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f78b4073990 con 0x7f78b406b2f0 2024-04-25T10:02:26.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.619+0000 7f78b0ff9640 1 -- 172.21.15.148:0/2716995723 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f78a40583c0 con 0x7f78b406b2f0 2024-04-25T10:02:26.624 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:26.624 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":507,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2562789376,"bytes_avail":770531323904,"bytes_total":773094113280,"write_bytes_sec":30331606,"read_op_per_sec":0,"write_op_per_sec":30},"fsmap":{"epoch":67,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8709}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T10:02:25.119895+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f78a006b500 msgr2=0x7f78a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f78a006b500 0x7f78a006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f78a8034000 tx=0x7f78a802c000 comp rx=0 tx=0).stop 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 msgr2=0x7f78b40ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b40ae5e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c390 tx=0x7f78a400c850 comp rx=0 tx=0).stop 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 shutdown_connections 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f78a006b500 0x7f78a006d9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78b41597d0 0x7f78b40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78b40b2d40 0x7f78b40c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 --2- 172.21.15.148:0/2716995723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78b406b2f0 0x7f78b40ae5e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 >> 172.21.15.148:0/2716995723 conn(0x7f78b40b2220 msgr2=0x7f78b415e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 shutdown_connections 2024-04-25T10:02:26.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.621+0000 7f78ba2fc640 1 -- 172.21.15.148:0/2716995723 wait complete. 2024-04-25T10:02:26.637 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:02:26.637 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:26.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:26.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:26.723 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:26.723 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:26.723 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:26.738 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:26.738 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:26.795 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:26.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.929+0000 7f9eedd29640 1 Processor -- start 2024-04-25T10:02:26.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.929+0000 7f9eedd29640 1 -- start start 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.929+0000 7f9eedd29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee8158220 0x7f9ee815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.929+0000 7f9eedd29640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee815f180 con 0x7f9ee80afba0 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.929+0000 7f9eedd29640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee81625d0 con 0x7f9ee80af720 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.929+0000 7f9eedd29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80b3670 con 0x7f9ee8158220 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9ee80afba0 0x7f9ee8161ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56384/0 (socket says 172.21.15.148:56384) 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee7fff640 1 -- 172.21.15.148:0/2816029301 learned_addr learned my addr 172.21.15.148:0/2816029301 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee8158220 0x7f9ee815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756477675 0 0) 0x7f9ee0001490 con 0x7f9ee80afba0 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed4003520 con 0x7f9ee80afba0 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249804121 0 0) 0x7f9ed4003520 con 0x7f9ee80afba0 2024-04-25T10:02:26.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee8158220 msgr2=0x7f9ee815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee8158220 0x7f9ee815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 >> v1:172.21.15.188:6789/0 conn(0x7f9ee80af720 legacy=0x7f9ee81617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee80b0030 con 0x7f9ee80afba0 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9ee0001bc0 con 0x7f9ee80afba0 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 >> v1:172.21.15.148:6789/0 conn(0x7f9ee80afba0 legacy=0x7f9ee8161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ed40039e0 0x7f9ed4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed4004390 0x7f9ed4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed4005010 0x7f9ed400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed400bf00 con 0x7f9ed40039e0 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed400c0b0 con 0x7f9ed4005010 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed400c2b0 con 0x7f9ed4004390 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee8158220 0x7f9ee815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:26.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ed40039e0 0x7f9ed4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9edeffd640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed4004390 0x7f9ed4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.930+0000 7f9ee7fff640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed4005010 0x7f9ed400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee77fe640 1 -- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed4004390 msgr2=0x7f9ed4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed4004390 0x7f9ed4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee77fe640 1 -- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed4005010 msgr2=0x7f9ed400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed4005010 0x7f9ed400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee77fe640 1 -- 172.21.15.148:0/2816029301 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4003520 con 0x7f9ed40039e0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ed40039e0 0x7f9ed4003dc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80d16f0 tx=0x7f9ed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355f0acb8ffffb5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8036070 con 0x7f9ed40039e0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8004030 con 0x7f9ed40039e0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.931+0000 7f9ee6ffd640 1 -- 172.21.15.148:0/2816029301 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80097d0 con 0x7f9ed40039e0 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9eedd29640 1 -- 172.21.15.148:0/2816029301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ed40039e0 msgr2=0x7f9ed4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9eedd29640 1 --2- 172.21.15.148:0/2816029301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ed40039e0 0x7f9ed4003dc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80d16f0 tx=0x7f9ed4005830 comp rx=0 tx=0).stop 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9ee7fff640 1 -- 172.21.15.148:0/2816029301 reap_dead start 2024-04-25T10:02:26.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9eedd29640 1 -- 172.21.15.148:0/2816029301 shutdown_connections 2024-04-25T10:02:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9eedd29640 1 -- 172.21.15.148:0/2816029301 >> 172.21.15.148:0/2816029301 conn(0x7f9ee80b2370 msgr2=0x7f9ee80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9eedd29640 1 -- 172.21.15.148:0/2816029301 shutdown_connections 2024-04-25T10:02:26.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.932+0000 7f9eedd29640 1 -- 172.21.15.148:0/2816029301 wait complete. 2024-04-25T10:02:26.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.933+0000 7f9eedd29640 1 Processor -- start 2024-04-25T10:02:26.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.933+0000 7f9eedd29640 1 -- start start 2024-04-25T10:02:26.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9eedd29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed40039e0 0x7f9ee80cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9eedd29640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee815a860 0x7f9ee8149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9eedd29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 0x7f9ee814a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9eedd29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80700e0 con 0x7f9ee815acc0 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9eedd29640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8070a10 con 0x7f9ee815a860 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9eedd29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80703c0 con 0x7f9ed40039e0 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 0x7f9ee814a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 0x7f9ee814a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39944/0 (socket says 172.21.15.148:39944) 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 -- 172.21.15.148:0/1218031748 learned_addr learned my addr 172.21.15.148:0/1218031748 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9edeffd640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee815a860 0x7f9ee8149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed40039e0 0x7f9ee80cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 -- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed40039e0 msgr2=0x7f9ee80cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed40039e0 0x7f9ee80cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 -- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee815a860 msgr2=0x7f9ee8149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee815a860 0x7f9ee8149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee7fff640 1 -- 172.21.15.148:0/1218031748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee80ca890 con 0x7f9ee815acc0 2024-04-25T10:02:26.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.934+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed40039e0 0x7f9ee80cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.935+0000 7f9ee7fff640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 0x7f9ee814a0e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80aff80 tx=0x7f9ee000af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.935+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0017020 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.935+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0007780 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.935+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee80cab70 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.935+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00133f0 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.935+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee80cb130 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.936+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee806fb20 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.936+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ee001a020 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.937+0000 7f9ee4ff9640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ec806b500 0x7f9ec806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.937+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ee008a9a0 con 0x7f9ee815acc0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.937+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ec806b500 0x7f9ec806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.937+0000 7f9ee77fe640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ec806b500 0x7f9ec806d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8033000 tx=0x7f9ed8008070 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:26.941+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ee806fb20 con 0x7f9ee815acc0 2024-04-25T10:02:27.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.113+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ee8073990 con 0x7f9ee815acc0 2024-04-25T10:02:27.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.114+0000 7f9ee4ff9640 1 -- 172.21.15.148:0/1218031748 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f9ee0021020 con 0x7f9ee815acc0 2024-04-25T10:02:27.119 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:27.120 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":121,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:54.304228+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:27.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ec806b500 msgr2=0x7f9ec806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ec806b500 0x7f9ec806d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8033000 tx=0x7f9ed8008070 comp rx=0 tx=0).stop 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 msgr2=0x7f9ee814a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 0x7f9ee814a0e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80aff80 tx=0x7f9ee000af10 comp rx=0 tx=0).stop 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 shutdown_connections 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ec806b500 0x7f9ec806d9c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee815acc0 0x7f9ee814a0e0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee815a860 0x7f9ee8149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 --2- 172.21.15.148:0/1218031748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed40039e0 0x7f9ee80cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.116+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 >> 172.21.15.148:0/1218031748 conn(0x7f9ee80b2370 msgr2=0x7f9ee80b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.117+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 shutdown_connections 2024-04-25T10:02:27.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.117+0000 7f9eedd29640 1 -- 172.21.15.148:0/1218031748 wait complete. 2024-04-25T10:02:27.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:50.849774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:51.601981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:27.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.228+0000 7f932ad16640 1 Processor -- start 2024-04-25T10:02:27.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.228+0000 7f932ad16640 1 -- start start 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f932ad16640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f932415dc20 0x7f93240ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f932ad16640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9324161710 0x7f93240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f932ad16640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932415e000 con 0x7f93240b9330 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f932ad16640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93240d1de0 con 0x7f932415dc20 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f932ad16640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9324161c60 con 0x7f9324161710 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f9328ab2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f932415dc20 0x7f93240ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f9328ab2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f932415dc20 0x7f93240ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39948/0 (socket says 172.21.15.148:39948) 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.229+0000 7f9328ab2640 1 -- 172.21.15.148:0/3227626848 learned_addr learned my addr 172.21.15.148:0/3227626848 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.230+0000 7f9328ab2640 1 -- 172.21.15.148:0/3227626848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9324161710 msgr2=0x7f93240cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.230+0000 7f9328ab2640 1 --2- 172.21.15.148:0/3227626848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9324161710 0x7f93240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.230+0000 7f9328ab2640 1 -- 172.21.15.148:0/3227626848 >> v1:172.21.15.188:6789/0 conn(0x7f93240b9330 legacy=0x7f932406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.230+0000 7f9328ab2640 1 -- 172.21.15.148:0/3227626848 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324161f40 con 0x7f932415dc20 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.230+0000 7f9328ab2640 1 --2- 172.21.15.148:0/3227626848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f93240ce670 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f9314009870 tx=0x7f931402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f0357c18d05ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f93237fe640 1 -- 172.21.15.148:0/3227626848 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314037070 con 0x7f932415dc20 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f93237fe640 1 -- 172.21.15.148:0/3227626848 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9314032440 con 0x7f932415dc20 2024-04-25T10:02:27.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f93237fe640 1 -- 172.21.15.148:0/3227626848 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314032810 con 0x7f932415dc20 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 -- 172.21.15.148:0/3227626848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 msgr2=0x7f93240ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 --2- 172.21.15.148:0/3227626848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f93240ce670 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f9314009870 tx=0x7f931402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 -- 172.21.15.148:0/3227626848 shutdown_connections 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 --2- 172.21.15.148:0/3227626848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9324161710 0x7f93240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 --2- 172.21.15.148:0/3227626848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f93240ce670 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 -- 172.21.15.148:0/3227626848 >> 172.21.15.148:0/3227626848 conn(0x7f93240d0420 msgr2=0x7f93240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 -- 172.21.15.148:0/3227626848 shutdown_connections 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.231+0000 7f932ad16640 1 -- 172.21.15.148:0/3227626848 wait complete. 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.232+0000 7f932ad16640 1 Processor -- start 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.232+0000 7f932ad16640 1 -- start start 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f932ad16640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93240b9330 0x7f932414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f932ad16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f9324148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f932ad16640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9324161710 0x7f93240b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f932ad16640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b5e50 con 0x7f932415dc20 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f932ad16640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324162500 con 0x7f9324161710 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f932ad16640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93240b5fd0 con 0x7f93240b9330 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f9323fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f9324148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f9323fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f9324148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39956/0 (socket says 172.21.15.148:39956) 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f9323fff640 1 -- 172.21.15.148:0/2027588604 learned_addr learned my addr 172.21.15.148:0/2027588604 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f93292b3640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9324161710 0x7f93240b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.233+0000 7f9328ab2640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93240b9330 0x7f932414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f9323fff640 1 -- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93240b9330 msgr2=0x7f932414bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f9323fff640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93240b9330 0x7f932414bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f9323fff640 1 -- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9324161710 msgr2=0x7f93240b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f9323fff640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9324161710 0x7f93240b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f9323fff640 1 -- 172.21.15.148:0/2027588604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93241494a0 con 0x7f932415dc20 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f9323fff640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f9324148f60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f931800b440 tx=0x7f931800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318015020 con 0x7f932415dc20 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9318011890 con 0x7f932415dc20 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318011c80 con 0x7f932415dc20 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9324149740 con 0x7f932415dc20 2024-04-25T10:02:27.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.234+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9324149d00 con 0x7f932415dc20 2024-04-25T10:02:27.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.235+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9324073a00 con 0x7f932415dc20 2024-04-25T10:02:27.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.235+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9318018020 con 0x7f932415dc20 2024-04-25T10:02:27.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.236+0000 7f93217fa640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931006b4d0 0x7f931006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.236+0000 7f9328ab2640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931006b4d0 0x7f931006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.236+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f931808a480 con 0x7f932415dc20 2024-04-25T10:02:27.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.236+0000 7f9328ab2640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931006b4d0 0x7f931006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9314034000 tx=0x7f931402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.240+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f931801f020 con 0x7f932415dc20 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.410+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f932415c790 con 0x7f932415dc20 2024-04-25T10:02:27.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.411+0000 7f93217fa640 1 -- 172.21.15.148:0/2027588604 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f931800d040 con 0x7f932415dc20 2024-04-25T10:02:27.416 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:02:27.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931006b4d0 msgr2=0x7f931006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931006b4d0 0x7f931006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9314034000 tx=0x7f931402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 msgr2=0x7f9324148f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f9324148f60 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f931800b440 tx=0x7f931800b900 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 shutdown_connections 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931006b4d0 0x7f931006d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9324161710 0x7f93240b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f932415dc20 0x7f9324148f60 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 --2- 172.21.15.148:0/2027588604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93240b9330 0x7f932414bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 >> 172.21.15.148:0/2027588604 conn(0x7f93240d0420 msgr2=0x7f932415bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 shutdown_connections 2024-04-25T10:02:27.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.413+0000 7f932ad16640 1 -- 172.21.15.148:0/2027588604 wait complete. 2024-04-25T10:02:27.428 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:27.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.524+0000 7efc2a838640 1 Processor -- start 2024-04-25T10:02:27.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.524+0000 7efc2a838640 1 -- start start 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.524+0000 7efc2a838640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc240cd4c0 0x7efc240cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc2a838640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2415e6e0 con 0x7efc24150b90 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc2a838640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2415e9c0 con 0x7efc2415b1c0 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc2a838640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2415a9c0 con 0x7efc240cd4c0 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc23fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc240cd4c0 0x7efc240cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc28dd5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efc24150b90 0x7efc24071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56386/0 (socket says 172.21.15.148:56386) 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc28dd5640 1 -- 172.21.15.148:0/2862597066 learned_addr learned my addr 172.21.15.148:0/2862597066 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:27.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc23fff640 1 -- 172.21.15.148:0/2862597066 >> v1:172.21.15.188:6789/0 conn(0x7efc2415b1c0 legacy=0x7efc2415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc23fff640 1 -- 172.21.15.148:0/2862597066 >> v1:172.21.15.148:6789/0 conn(0x7efc24150b90 legacy=0x7efc24071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc23fff640 1 -- 172.21.15.148:0/2862597066 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2415aca0 con 0x7efc240cd4c0 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.525+0000 7efc22ffd640 1 -- 172.21.15.148:0/2862597066 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282596207 0 0) 0x7efc2415e6e0 con 0x7efc24150b90 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc23fff640 1 --2- 172.21.15.148:0/2862597066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc240cd4c0 0x7efc240cd8a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7efc180097a0 tx=0x7efc1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6043be887e16564c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc22ffd640 1 -- 172.21.15.148:0/2862597066 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18037070 con 0x7efc240cd4c0 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc22ffd640 1 -- 172.21.15.148:0/2862597066 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18032440 con 0x7efc240cd4c0 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc22ffd640 1 -- 172.21.15.148:0/2862597066 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18032810 con 0x7efc240cd4c0 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 -- 172.21.15.148:0/2862597066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc240cd4c0 msgr2=0x7efc240cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 --2- 172.21.15.148:0/2862597066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc240cd4c0 0x7efc240cd8a0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7efc180097a0 tx=0x7efc1802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 -- 172.21.15.148:0/2862597066 shutdown_connections 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 --2- 172.21.15.148:0/2862597066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc240cd4c0 0x7efc240cd8a0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 -- 172.21.15.148:0/2862597066 >> 172.21.15.148:0/2862597066 conn(0x7efc2406abe0 msgr2=0x7efc2406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 -- 172.21.15.148:0/2862597066 shutdown_connections 2024-04-25T10:02:27.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.526+0000 7efc2a838640 1 -- 172.21.15.148:0/2862597066 wait complete. 2024-04-25T10:02:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.527+0000 7efc2a838640 1 Processor -- start 2024-04-25T10:02:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 -- start start 2024-04-25T10:02:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc240cd4c0 0x7efc2414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 0x7efc241539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc2415b1c0 0x7efc24145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406eca0 con 0x7efc24150b90 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2415e370 con 0x7efc240cd4c0 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc2a838640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406ee20 con 0x7efc2415b1c0 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.528+0000 7efc237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 0x7efc241539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 0x7efc241539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39976/0 (socket says 172.21.15.148:39976) 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 -- 172.21.15.148:0/2487871718 learned_addr learned my addr 172.21.15.148:0/2487871718 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc23fff640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc240cd4c0 0x7efc2414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc28dd5640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc2415b1c0 0x7efc24145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 -- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc2415b1c0 msgr2=0x7efc24145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc2415b1c0 0x7efc24145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 -- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc240cd4c0 msgr2=0x7efc2414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc240cd4c0 0x7efc2414e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 -- 172.21.15.148:0/2487871718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2414e950 con 0x7efc24150b90 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc23fff640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc240cd4c0 0x7efc2414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc237fe640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 0x7efc241539f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7efc1400b420 tx=0x7efc1400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14011830 con 0x7efc24150b90 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14016070 con 0x7efc24150b90 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14011c10 con 0x7efc24150b90 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2414ec30 con 0x7efc24150b90 2024-04-25T10:02:27.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.529+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2414f1f0 con 0x7efc24150b90 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.530+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24073a00 con 0x7efc24150b90 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.530+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc1401b460 con 0x7efc24150b90 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.531+0000 7efc20ff9640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc1006b4d0 0x7efc1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.531+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efc14089de0 con 0x7efc24150b90 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.531+0000 7efc23fff640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc1006b4d0 0x7efc1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.531+0000 7efc23fff640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc1006b4d0 0x7efc1006d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efc18034000 tx=0x7efc1802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.535+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc14020020 con 0x7efc24150b90 2024-04-25T10:02:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.705+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc24159f20 con 0x7efc24150b90 2024-04-25T10:02:27.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.705+0000 7efc20ff9640 1 -- 172.21.15.148:0/2487871718 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc14058b60 con 0x7efc24150b90 2024-04-25T10:02:27.710 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:02:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc1006b4d0 msgr2=0x7efc1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc1006b4d0 0x7efc1006d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efc18034000 tx=0x7efc1802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 msgr2=0x7efc241539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 0x7efc241539f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7efc1400b420 tx=0x7efc1400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 shutdown_connections 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc1006b4d0 0x7efc1006d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc2415b1c0 0x7efc24145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24150b90 0x7efc241539f0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 --2- 172.21.15.148:0/2487871718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc240cd4c0 0x7efc2414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.707+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 >> 172.21.15.148:0/2487871718 conn(0x7efc2406abe0 msgr2=0x7efc2415c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.708+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 shutdown_connections 2024-04-25T10:02:27.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.708+0000 7efc2a838640 1 -- 172.21.15.148:0/2487871718 wait complete. 2024-04-25T10:02:27.723 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.819+0000 7f1dafb5c640 1 Processor -- start 2024-04-25T10:02:27.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.819+0000 7f1dafb5c640 1 -- start start 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dafb5c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1da80cc4f0 0x7f1da80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dafb5c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1da80cdb20 0x7f1da80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dafb5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dafb5c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da8071270 con 0x7f1da80cc4f0 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dafb5c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da8072280 con 0x7f1da80cdb20 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dafb5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80714c0 con 0x7f1da815b270 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1da80cc4f0 0x7f1da80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1da80cc4f0 0x7f1da80cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39984/0 (socket says 172.21.15.148:39984) 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 -- 172.21.15.148:0/1444631505 learned_addr learned my addr 172.21.15.148:0/1444631505 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad0f7640 1 --2- 172.21.15.148:0/1444631505 >> v2:172.21.15.188:3300/0 conn(0x7f1da80cdb20 0x7f1da80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dae0f9640 1 --2- 172.21.15.148:0/1444631505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 -- 172.21.15.148:0/1444631505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 msgr2=0x7f1da815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 --2- 172.21.15.148:0/1444631505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 -- 172.21.15.148:0/1444631505 >> v2:172.21.15.188:3300/0 conn(0x7f1da80cdb20 msgr2=0x7f1da80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 --2- 172.21.15.148:0/1444631505 >> v2:172.21.15.188:3300/0 conn(0x7f1da80cdb20 0x7f1da80cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad8f8640 1 -- 172.21.15.148:0/1444631505 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da80717a0 con 0x7f1da80cc4f0 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.820+0000 7f1dad0f7640 1 --2- 172.21.15.148:0/1444631505 >> v2:172.21.15.188:3300/0 conn(0x7f1da80cdb20 0x7f1da80cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.821+0000 7f1dad8f8640 1 --2- 172.21.15.148:0/1444631505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cc4f0 0x7f1da80cc8d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1da8158aa0 tx=0x7f1da402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b399f52132f3c59a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.821+0000 7f1dac8f6640 1 -- 172.21.15.148:0/1444631505 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4037070 con 0x7f1da80cc4f0 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.821+0000 7f1dac8f6640 1 -- 172.21.15.148:0/1444631505 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da4032440 con 0x7f1da80cc4f0 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.821+0000 7f1dac8f6640 1 -- 172.21.15.148:0/1444631505 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4032810 con 0x7f1da80cc4f0 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1444631505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cc4f0 msgr2=0x7f1da80cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1444631505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cc4f0 0x7f1da80cc8d0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1da8158aa0 tx=0x7f1da402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1444631505 shutdown_connections 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1444631505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1444631505 >> v2:172.21.15.188:3300/0 conn(0x7f1da80cdb20 0x7f1da80cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1444631505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cc4f0 0x7f1da80cc8d0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1444631505 >> 172.21.15.148:0/1444631505 conn(0x7f1da806abe0 msgr2=0x7f1da806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1444631505 shutdown_connections 2024-04-25T10:02:27.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.822+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1444631505 wait complete. 2024-04-25T10:02:27.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.823+0000 7f1dafb5c640 1 Processor -- start 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.823+0000 7f1dafb5c640 1 -- start start 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da80cc4f0 0x7f1da81f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 0x7f1da80c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da8146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815bc00 con 0x7f1da80cdb20 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8071d60 con 0x7f1da80cc4f0 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815bd80 con 0x7f1da815b270 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 0x7f1da80c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 0x7f1da80c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39992/0 (socket says 172.21.15.148:39992) 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 -- 172.21.15.148:0/1072417904 learned_addr learned my addr 172.21.15.148:0/1072417904 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dae0f9640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da8146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad8f8640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da80cc4f0 0x7f1da81f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 -- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 msgr2=0x7f1da8146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da8146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 -- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da80cc4f0 msgr2=0x7f1da81f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da80cc4f0 0x7f1da81f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 -- 172.21.15.148:0/1072417904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da814f2f0 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dae0f9640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da8146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dad0f7640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 0x7f1da80c60c0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800e840 tx=0x7f1d9800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800f040 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d98004030 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da8150690 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98004400 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.824+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da8159480 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.825+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da8073a00 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.826+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d9800c6d0 con 0x7f1da80cdb20 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.826+0000 7f1d9e7fc640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.826+0000 7f1dad8f8640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.826+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d9808a0e0 con 0x7f1da80cdb20 2024-04-25T10:02:27.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.827+0000 7f1dad8f8640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1da4034000 tx=0x7f1da402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:27.831+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d98017020 con 0x7f1da80cdb20 2024-04-25T10:02:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.000+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1da806d3a0 con 0x7f1da80cdb20 2024-04-25T10:02:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.000+0000 7f1d9e7fc640 1 -- 172.21.15.148:0/1072417904 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d98058d40 con 0x7f1da80cdb20 2024-04-25T10:02:28.005 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:02:28.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1d8c06b4d0 msgr2=0x7f1d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1da4034000 tx=0x7f1da402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 msgr2=0x7f1da80c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 0x7f1da80c60c0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800e840 tx=0x7f1d9800ed00 comp rx=0 tx=0).stop 2024-04-25T10:02:28.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 shutdown_connections 2024-04-25T10:02:28.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1d8c06b4d0 0x7f1d8c06d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1da815b270 0x7f1da8146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1da80cdb20 0x7f1da80c60c0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 --2- 172.21.15.148:0/1072417904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1da80cc4f0 0x7f1da81f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 >> 172.21.15.148:0/1072417904 conn(0x7f1da806abe0 msgr2=0x7f1da80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 shutdown_connections 2024-04-25T10:02:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.002+0000 7f1dafb5c640 1 -- 172.21.15.148:0/1072417904 wait complete. 2024-04-25T10:02:28.018 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:28.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.110+0000 7f757995c640 1 Processor -- start 2024-04-25T10:02:28.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.110+0000 7f757995c640 1 -- start start 2024-04-25T10:02:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f757995c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75740b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f757995c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75740b54a0 0x7f75740b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f757995c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75740cf1d0 con 0x7f7574161890 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f757995c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740b7120 con 0x7f75740b54a0 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f757995c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7574161620 con 0x7f7574161240 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f75727fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7574161890 0x7f75740ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56392/0 (socket says 172.21.15.148:56392) 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f75727fc640 1 -- 172.21.15.148:0/4144189171 learned_addr learned my addr 172.21.15.148:0/4144189171 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f7572ffd640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75740b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f75737fe640 1 --2- 172.21.15.148:0/4144189171 >> v2:172.21.15.188:3300/0 conn(0x7f75740b54a0 0x7f75740b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685603234 0 0) 0x7f75740cf1d0 con 0x7f7574161890 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.111+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7560003520 con 0x7f7574161890 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1243124622 0 0) 0x7f7560003520 con 0x7f7574161890 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 msgr2=0x7f75740b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75740b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 >> v2:172.21.15.188:3300/0 conn(0x7f75740b54a0 msgr2=0x7f75740b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 --2- 172.21.15.148:0/4144189171 >> v2:172.21.15.188:3300/0 conn(0x7f75740b54a0 0x7f75740b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574161e00 con 0x7f7574161890 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f755c001670 con 0x7f7574161890 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 >> v1:172.21.15.148:6789/0 conn(0x7f7574161890 legacy=0x7f75740ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7560003f30 0x7f7560004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7560004d40 0x7f75600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7574161620 con 0x7f7560004d40 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756000b9c0 con 0x7f7560003f30 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756000bb40 con 0x7f7574161240 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75737fe640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7560004d40 0x7f75600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75727fc640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7560003f30 0x7f7560004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7572ffd640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75737fe640 1 -- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 msgr2=0x7f75600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75737fe640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75737fe640 1 -- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7560003f30 msgr2=0x7f7560004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75737fe640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7560003f30 0x7f7560004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f75737fe640 1 -- 172.21.15.148:0/4144189171 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560003520 con 0x7f7560004d40 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.112+0000 7f7572ffd640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7574161240 0x7f75600039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f75737fe640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7560004d40 0x7f75600051c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7564009ac0 tx=0x7f7560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca508f97b4d0620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564036070 con 0x7f7560004d40 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564031400 con 0x7f7560004d40 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f7571ffb640 1 -- 172.21.15.148:0/4144189171 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564009690 con 0x7f7560004d40 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f757995c640 1 -- 172.21.15.148:0/4144189171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7560004d40 msgr2=0x7f75600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f757995c640 1 --2- 172.21.15.148:0/4144189171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7560004d40 0x7f75600051c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7564009ac0 tx=0x7f7560005830 comp rx=0 tx=0).stop 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f75737fe640 1 -- 172.21.15.148:0/4144189171 reap_dead start 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f757995c640 1 -- 172.21.15.148:0/4144189171 shutdown_connections 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f757995c640 1 -- 172.21.15.148:0/4144189171 >> 172.21.15.148:0/4144189171 conn(0x7f75740d04a0 msgr2=0x7f75740d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f757995c640 1 -- 172.21.15.148:0/4144189171 shutdown_connections 2024-04-25T10:02:28.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.113+0000 7f757995c640 1 -- 172.21.15.148:0/4144189171 wait complete. 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 Processor -- start 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 -- start start 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7560004d40 0x7f75740c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740ccfa0 0x7f75741f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 0x7f75741f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75741626a0 con 0x7f75741f6ba0 2024-04-25T10:02:28.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75741623c0 con 0x7f75740ccfa0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.115+0000 7f757995c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740d15f0 con 0x7f7560004d40 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 0x7f75741f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 0x7f75741f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40010/0 (socket says 172.21.15.148:40010) 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 -- 172.21.15.148:0/1601550524 learned_addr learned my addr 172.21.15.148:0/1601550524 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f7572ffd640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7560004d40 0x7f75740c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75727fc640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740ccfa0 0x7f75741f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 -- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7560004d40 msgr2=0x7f75740c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7560004d40 0x7f75740c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 -- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740ccfa0 msgr2=0x7f75741f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740ccfa0 0x7f75741f2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 -- 172.21.15.148:0/1601550524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75741f3450 con 0x7f75741f6ba0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f7572ffd640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7560004d40 0x7f75740c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f75737fe640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 0x7f75741f7020 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f7564034040 tx=0x7f7564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564036040 con 0x7f75741f6ba0 2024-04-25T10:02:28.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564041070 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75740cc500 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564003c80 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.116+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757415d390 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.117+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75741496a0 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.118+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7564039070 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.118+0000 7f756bfff640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755406b500 0x7f755406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.118+0000 7f7572ffd640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755406b500 0x7f755406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.118+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75640a9990 con 0x7f75741f6ba0 2024-04-25T10:02:28.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.119+0000 7f7572ffd640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755406b500 0x7f755406d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f756c007520 tx=0x7f756c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.122+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75741496a0 con 0x7f75741f6ba0 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.303+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7574073a00 con 0x7f75741f6ba0 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.303+0000 7f756bfff640 1 -- 172.21.15.148:0/1601550524 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f756404a020 con 0x7f75741f6ba0 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755406b500 msgr2=0x7f755406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755406b500 0x7f755406d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f756c007520 tx=0x7f756c00b040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 msgr2=0x7f75741f7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 0x7f75741f7020 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f7564034040 tx=0x7f7564004730 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 shutdown_connections 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f755406b500 0x7f755406d9c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75741f6ba0 0x7f75741f7020 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740ccfa0 0x7f75741f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 --2- 172.21.15.148:0/1601550524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7560004d40 0x7f75740c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 >> 172.21.15.148:0/1601550524 conn(0x7f75740d04a0 msgr2=0x7f757415b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 shutdown_connections 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:28.305+0000 7f757995c640 1 -- 172.21.15.148:0/1601550524 wait complete. 2024-04-25T10:02:28.321 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:02:28.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:28.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:02:28.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:28.932 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:02:28.948 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:02:28.948 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:28.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:28.964 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:28.964 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:29.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:29.003 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:02:29.004 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:29.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:29.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.136+0000 7f5a0f510640 1 Processor -- start 2024-04-25T10:02:29.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.136+0000 7f5a0f510640 1 -- start start 2024-04-25T10:02:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0f510640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a0815dc90 0x7f5a081625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0f510640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5a080cc8e0 0x7f5a0815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0f510640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0815f390 con 0x7f5a080d1810 2024-04-25T10:02:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0f510640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080cce50 con 0x7f5a080cc8e0 2024-04-25T10:02:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0f510640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a08161af0 con 0x7f5a0815dc90 2024-04-25T10:02:29.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0d2ac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a0815dc90 0x7f5a081625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0daad640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5a080d1810 0x7f5a080cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:52270/0 (socket says 172.21.15.148:52270) 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0daad640 1 -- 172.21.15.148:0/3939335109 learned_addr learned my addr 172.21.15.148:0/3939335109 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.137+0000 7f5a0caab640 1 --2- 172.21.15.148:0/3939335109 >> v2:172.21.15.148:3300/0 conn(0x7f5a080cc8e0 0x7f5a0815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f5a0d2ac640 1 -- 172.21.15.148:0/3939335109 >> v2:172.21.15.148:3300/0 conn(0x7f5a080cc8e0 msgr2=0x7f5a0815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f5a0d2ac640 1 --2- 172.21.15.148:0/3939335109 >> v2:172.21.15.148:3300/0 conn(0x7f5a080cc8e0 0x7f5a0815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f5a0d2ac640 1 -- 172.21.15.148:0/3939335109 >> v1:172.21.15.188:6789/0 conn(0x7f5a080d1810 legacy=0x7f5a080cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f59fffff640 1 -- 172.21.15.148:0/3939335109 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643698884 0 0) 0x7f5a00001490 con 0x7f5a080d1810 2024-04-25T10:02:29.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f5a0d2ac640 1 -- 172.21.15.148:0/3939335109 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08161dd0 con 0x7f5a0815dc90 2024-04-25T10:02:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f5a0d2ac640 1 --2- 172.21.15.148:0/3939335109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815dc90 0x7f5a081625d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5a0806b820 tx=0x7f5a0402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95efc1840756e17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f59fffff640 1 -- 172.21.15.148:0/3939335109 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04037070 con 0x7f5a0815dc90 2024-04-25T10:02:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.138+0000 7f59fffff640 1 -- 172.21.15.148:0/3939335109 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04032440 con 0x7f5a0815dc90 2024-04-25T10:02:29.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f59fffff640 1 -- 172.21.15.148:0/3939335109 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0402ae60 con 0x7f5a0815dc90 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 -- 172.21.15.148:0/3939335109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815dc90 msgr2=0x7f5a081625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 --2- 172.21.15.148:0/3939335109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815dc90 0x7f5a081625d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5a0806b820 tx=0x7f5a0402a110 comp rx=0 tx=0).stop 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 -- 172.21.15.148:0/3939335109 shutdown_connections 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 --2- 172.21.15.148:0/3939335109 >> v2:172.21.15.148:3300/0 conn(0x7f5a080cc8e0 0x7f5a0815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 --2- 172.21.15.148:0/3939335109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0815dc90 0x7f5a081625d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 -- 172.21.15.148:0/3939335109 >> 172.21.15.148:0/3939335109 conn(0x7f5a0806f590 msgr2=0x7f5a0806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 -- 172.21.15.148:0/3939335109 shutdown_connections 2024-04-25T10:02:29.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.139+0000 7f5a0f510640 1 -- 172.21.15.148:0/3939335109 wait complete. 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.140+0000 7f5a0f510640 1 Processor -- start 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.140+0000 7f5a0f510640 1 -- start start 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0f510640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a080cc8e0 0x7f5a081f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0f510640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 0x7f5a081e74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0f510640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0815dc90 0x7f5a08148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0f510640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0815e070 con 0x7f5a080d1810 2024-04-25T10:02:29.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0f510640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080cd130 con 0x7f5a080cc8e0 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0f510640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0815e1f0 con 0x7f5a0815dc90 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 0x7f5a081e74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 0x7f5a081e74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40026/0 (socket says 172.21.15.148:40026) 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 -- 172.21.15.148:0/1581663386 learned_addr learned my addr 172.21.15.148:0/1581663386 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0d2ac640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a080cc8e0 0x7f5a081f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0daad640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0815dc90 0x7f5a08148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 -- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0815dc90 msgr2=0x7f5a08148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0815dc90 0x7f5a08148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 -- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a080cc8e0 msgr2=0x7f5a081f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a080cc8e0 0x7f5a081f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.141+0000 7f5a0caab640 1 -- 172.21.15.148:0/1581663386 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a081f8440 con 0x7f5a080d1810 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.142+0000 7f5a0caab640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 0x7f5a081e74c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f59f800e9b0 tx=0x7f59f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.142+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8017070 con 0x7f5a080d1810 2024-04-25T10:02:29.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.142+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f800c9a0 con 0x7f5a080d1810 2024-04-25T10:02:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.142+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a081f45c0 con 0x7f5a080d1810 2024-04-25T10:02:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.142+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800cd90 con 0x7f5a080d1810 2024-04-25T10:02:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.142+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a081f4e00 con 0x7f5a080d1810 2024-04-25T10:02:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.143+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a08073990 con 0x7f5a080d1810 2024-04-25T10:02:29.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.143+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59f801a070 con 0x7f5a080d1810 2024-04-25T10:02:29.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.144+0000 7f59fdffb640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59ec06b4d0 0x7f59ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.144+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f59f808a320 con 0x7f5a080d1810 2024-04-25T10:02:29.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.144+0000 7f5a0d2ac640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59ec06b4d0 0x7f59ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.144+0000 7f5a0d2ac640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59ec06b4d0 0x7f59ec06d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5a04034000 tx=0x7f5a0402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.148+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a081f45c0 con 0x7f5a080d1810 2024-04-25T10:02:29.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.355+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a080cf6b0 con 0x7f5a080d1810 2024-04-25T10:02:29.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.355+0000 7f59fdffb640 1 -- 172.21.15.148:0/1581663386 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f59f80589c0 con 0x7f5a080d1810 2024-04-25T10:02:29.360 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:29.360 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:29.361 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:29.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.357+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59ec06b4d0 msgr2=0x7f59ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.357+0000 7f5a0f510640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59ec06b4d0 0x7f59ec06d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5a04034000 tx=0x7f5a0402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.357+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 msgr2=0x7f5a081e74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.357+0000 7f5a0f510640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 0x7f5a081e74c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f59f800e9b0 tx=0x7f59f800ee70 comp rx=0 tx=0).stop 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 shutdown_connections 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f59ec06b4d0 0x7f59ec06d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0815dc90 0x7f5a08148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a080d1810 0x7f5a081e74c0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 --2- 172.21.15.148:0/1581663386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a080cc8e0 0x7f5a081f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 >> 172.21.15.148:0/1581663386 conn(0x7f5a0806f590 msgr2=0x7f5a0815c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 shutdown_connections 2024-04-25T10:02:29.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.358+0000 7f5a0f510640 1 -- 172.21.15.148:0/1581663386 wait complete. 2024-04-25T10:02:29.374 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:29.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.468+0000 7f9a9fa22640 1 Processor -- start 2024-04-25T10:02:29.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.468+0000 7f9a9fa22640 1 -- start start 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9fa22640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a9815dc90 0x7f9a981625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9fa22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a980cc8e0 0x7f9a9815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9fa22640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9815f390 con 0x7f9a980d1810 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9fa22640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a980cce50 con 0x7f9a9815dc90 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9fa22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98161af0 con 0x7f9a980cc8e0 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9dfbf640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9a980d1810 0x7f9a980cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56406/0 (socket says 172.21.15.148:56406) 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9dfbf640 1 -- 172.21.15.148:0/1503213610 learned_addr learned my addr 172.21.15.148:0/1503213610 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9cfbd640 1 --2- 172.21.15.148:0/1503213610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a980cc8e0 0x7f9a9815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9d7be640 1 --2- 172.21.15.148:0/1503213610 >> v2:172.21.15.188:3300/0 conn(0x7f9a9815dc90 0x7f9a981625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a8ffff640 1 -- 172.21.15.148:0/1503213610 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118998444 0 0) 0x7f9a90001490 con 0x7f9a980d1810 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a8ffff640 1 -- 172.21.15.148:0/1503213610 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a84003520 con 0x7f9a980d1810 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a8ffff640 1 -- 172.21.15.148:0/1503213610 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3849619916 0 0) 0x7f9a84003520 con 0x7f9a980d1810 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/1503213610 >> v2:172.21.15.188:3300/0 conn(0x7f9a9815dc90 msgr2=0x7f9a981625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9cfbd640 1 --2- 172.21.15.148:0/1503213610 >> v2:172.21.15.188:3300/0 conn(0x7f9a9815dc90 0x7f9a981625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/1503213610 >> v1:172.21.15.148:6789/0 conn(0x7f9a980d1810 legacy=0x7f9a980cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.469+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/1503213610 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98161dd0 con 0x7f9a980cc8e0 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.470+0000 7f9a9cfbd640 1 --2- 172.21.15.148:0/1503213610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a980cc8e0 0x7f9a9815ee50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097a0 tx=0x7f9a8802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24b9c5612db45a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.470+0000 7f9a8ffff640 1 -- 172.21.15.148:0/1503213610 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88037070 con 0x7f9a980cc8e0 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.470+0000 7f9a8ffff640 1 -- 172.21.15.148:0/1503213610 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8802ab10 con 0x7f9a980cc8e0 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.470+0000 7f9a8ffff640 1 -- 172.21.15.148:0/1503213610 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90001bc0 con 0x7f9a980cc8e0 2024-04-25T10:02:29.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 -- 172.21.15.148:0/1503213610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a980cc8e0 msgr2=0x7f9a9815ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/1503213610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a980cc8e0 0x7f9a9815ee50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a880097a0 tx=0x7f9a8802a080 comp rx=0 tx=0).stop 2024-04-25T10:02:29.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 -- 172.21.15.148:0/1503213610 shutdown_connections 2024-04-25T10:02:29.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/1503213610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a980cc8e0 0x7f9a9815ee50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/1503213610 >> v2:172.21.15.188:3300/0 conn(0x7f9a9815dc90 0x7f9a981625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 -- 172.21.15.148:0/1503213610 >> 172.21.15.148:0/1503213610 conn(0x7f9a9806f590 msgr2=0x7f9a9806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 -- 172.21.15.148:0/1503213610 shutdown_connections 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.471+0000 7f9a9fa22640 1 -- 172.21.15.148:0/1503213610 wait complete. 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.472+0000 7f9a9fa22640 1 Processor -- start 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.472+0000 7f9a9fa22640 1 -- start start 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9fa22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a980cc8e0 0x7f9a980b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9fa22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 0x7f9a9814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9fa22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815dc90 0x7f9a981f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9fa22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9815e070 con 0x7f9a980d1810 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9fa22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980cd130 con 0x7f9a980cc8e0 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9fa22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9815e1f0 con 0x7f9a9815dc90 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 0x7f9a9814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 0x7f9a9814f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40038/0 (socket says 172.21.15.148:40038) 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/3437688347 learned_addr learned my addr 172.21.15.148:0/3437688347 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9d7be640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a980cc8e0 0x7f9a980b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9dfbf640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815dc90 0x7f9a981f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815dc90 msgr2=0x7f9a981f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815dc90 0x7f9a981f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a980cc8e0 msgr2=0x7f9a980b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a980cc8e0 0x7f9a980b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 -- 172.21.15.148:0/3437688347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a980b5c40 con 0x7f9a980d1810 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.473+0000 7f9a9cfbd640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 0x7f9a9814f410 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9a88035040 tx=0x7f9a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.474+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88037050 con 0x7f9a980d1810 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.474+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88043070 con 0x7f9a980d1810 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.474+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a980b5f20 con 0x7f9a980d1810 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.474+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88006e10 con 0x7f9a980d1810 2024-04-25T10:02:29.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.474+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a980b64e0 con 0x7f9a980d1810 2024-04-25T10:02:29.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.475+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98073990 con 0x7f9a980d1810 2024-04-25T10:02:29.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.475+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a8803ca10 con 0x7f9a980d1810 2024-04-25T10:02:29.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.476+0000 7f9a8dffb640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a7c06b4d0 0x7f9a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.476+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a880aaee0 con 0x7f9a980d1810 2024-04-25T10:02:29.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.476+0000 7f9a9d7be640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a7c06b4d0 0x7f9a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.476+0000 7f9a9d7be640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a7c06b4d0 0x7f9a7c06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009b40 tx=0x7f9a94009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.480+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a88079730 con 0x7f9a980d1810 2024-04-25T10:02:29.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.650+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a980cf6b0 con 0x7f9a980d1810 2024-04-25T10:02:29.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.651+0000 7f9a8dffb640 1 -- 172.21.15.148:0/3437688347 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f9a8807d7d0 con 0x7f9a980d1810 2024-04-25T10:02:29.656 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:29.656 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":121,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:54.304228+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:29.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a7c06b4d0 msgr2=0x7f9a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a7c06b4d0 0x7f9a7c06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009b40 tx=0x7f9a94009190 comp rx=0 tx=0).stop 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 msgr2=0x7f9a9814f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 0x7f9a9814f410 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9a88035040 tx=0x7f9a880046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 shutdown_connections 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9a7c06b4d0 0x7f9a7c06d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815dc90 0x7f9a981f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a980d1810 0x7f9a9814f410 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 --2- 172.21.15.148:0/3437688347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a980cc8e0 0x7f9a980b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 >> 172.21.15.148:0/3437688347 conn(0x7f9a9806f590 msgr2=0x7f9a9806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 shutdown_connections 2024-04-25T10:02:29.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.653+0000 7f9a9fa22640 1 -- 172.21.15.148:0/3437688347 wait complete. 2024-04-25T10:02:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:50.849774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:51.601981+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:29.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:29.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.765+0000 7f1068b32640 1 Processor -- start 2024-04-25T10:02:29.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.765+0000 7f1068b32640 1 -- start start 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.765+0000 7f1068b32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cd4c0 0x7f10640cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f1068b32640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106415e6e0 con 0x7f106415b1c0 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f1068b32640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106415e9c0 con 0x7f1064150b90 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f1068b32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106415b620 con 0x7f10640cd4c0 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f105bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f106415b1c0 0x7f106415dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56422/0 (socket says 172.21.15.148:56422) 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f105bfff640 1 -- 172.21.15.148:0/4152380128 learned_addr learned my addr 172.21.15.148:0/4152380128 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f106259c640 1 --2- 172.21.15.148:0/4152380128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cd4c0 0x7f10640cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f1061d9b640 1 -- 172.21.15.148:0/4152380128 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351576293 0 0) 0x7f106415e6e0 con 0x7f106415b1c0 2024-04-25T10:02:29.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f1061d9b640 1 -- 172.21.15.148:0/4152380128 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1050003520 con 0x7f106415b1c0 2024-04-25T10:02:29.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f106259c640 1 -- 172.21.15.148:0/4152380128 >> v1:172.21.15.188:6789/0 conn(0x7f1064150b90 legacy=0x7f1064071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f106259c640 1 -- 172.21.15.148:0/4152380128 >> v1:172.21.15.148:6789/0 conn(0x7f106415b1c0 legacy=0x7f106415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.766+0000 7f106259c640 1 -- 172.21.15.148:0/4152380128 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106415a9c0 con 0x7f10640cd4c0 2024-04-25T10:02:29.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.767+0000 7f106259c640 1 --2- 172.21.15.148:0/4152380128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cd4c0 0x7f10640cd8a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10540097d0 tx=0x7f105402b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6f0738bccab2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.767+0000 7f1061d9b640 1 -- 172.21.15.148:0/4152380128 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054038070 con 0x7f10640cd4c0 2024-04-25T10:02:29.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.767+0000 7f1061d9b640 1 -- 172.21.15.148:0/4152380128 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1054033640 con 0x7f10640cd4c0 2024-04-25T10:02:29.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.767+0000 7f1061d9b640 1 -- 172.21.15.148:0/4152380128 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054033a10 con 0x7f10640cd4c0 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.767+0000 7f1068b32640 1 -- 172.21.15.148:0/4152380128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cd4c0 msgr2=0x7f10640cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.768+0000 7f1068b32640 1 --2- 172.21.15.148:0/4152380128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cd4c0 0x7f10640cd8a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10540097d0 tx=0x7f105402b320 comp rx=0 tx=0).stop 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.768+0000 7f1068b32640 1 -- 172.21.15.148:0/4152380128 shutdown_connections 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.768+0000 7f1068b32640 1 --2- 172.21.15.148:0/4152380128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10640cd4c0 0x7f10640cd8a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.768+0000 7f1068b32640 1 -- 172.21.15.148:0/4152380128 >> 172.21.15.148:0/4152380128 conn(0x7f106406abe0 msgr2=0x7f106406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.768+0000 7f1068b32640 1 -- 172.21.15.148:0/4152380128 shutdown_connections 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.768+0000 7f1068b32640 1 -- 172.21.15.148:0/4152380128 wait complete. 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.769+0000 7f1068b32640 1 Processor -- start 2024-04-25T10:02:29.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.769+0000 7f1068b32640 1 -- start start 2024-04-25T10:02:29.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cd4c0 0x7f1064150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1064150b90 0x7f10641f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 0x7f10641f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106415e370 con 0x7f106415b1c0 2024-04-25T10:02:29.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106415a670 con 0x7f10640cd4c0 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106406eca0 con 0x7f1064150b90 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 0x7f10641f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 0x7f10641f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40040/0 (socket says 172.21.15.148:40040) 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 -- 172.21.15.148:0/4040378616 learned_addr learned my addr 172.21.15.148:0/4040378616 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f105bfff640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1064150b90 0x7f10641f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f106259c640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cd4c0 0x7f1064150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 -- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1064150b90 msgr2=0x7f10641f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1064150b90 0x7f10641f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 -- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cd4c0 msgr2=0x7f1064150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cd4c0 0x7f1064150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 -- 172.21.15.148:0/4040378616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106414b270 con 0x7f106415b1c0 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f106259c640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cd4c0 0x7f1064150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1062d9d640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 0x7f10641f54e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c7a0 tx=0x7f105c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c003b70 con 0x7f106415b1c0 2024-04-25T10:02:29.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c003d30 con 0x7f106415b1c0 2024-04-25T10:02:29.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106414b510 con 0x7f106415b1c0 2024-04-25T10:02:29.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00aa90 con 0x7f106415b1c0 2024-04-25T10:02:29.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.770+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106414c810 con 0x7f106415b1c0 2024-04-25T10:02:29.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.771+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1064073a00 con 0x7f106415b1c0 2024-04-25T10:02:29.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.772+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f105c00ac30 con 0x7f106415b1c0 2024-04-25T10:02:29.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.772+0000 7f105affd640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f104406b4d0 0x7f104406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.772+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f105c08a240 con 0x7f106415b1c0 2024-04-25T10:02:29.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.772+0000 7f106259c640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f104406b4d0 0x7f104406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.773+0000 7f106259c640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f104406b4d0 0x7f104406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1054035000 tx=0x7f105402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.777+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f105c058fc0 con 0x7f106415b1c0 2024-04-25T10:02:29.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.946+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1064165de0 con 0x7f106415b1c0 2024-04-25T10:02:29.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.947+0000 7f105affd640 1 -- 172.21.15.148:0/4040378616 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f105c0588c0 con 0x7f106415b1c0 2024-04-25T10:02:29.952 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f104406b4d0 msgr2=0x7f104406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f104406b4d0 0x7f104406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1054035000 tx=0x7f105402f000 comp rx=0 tx=0).stop 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 msgr2=0x7f10641f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 0x7f10641f54e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c7a0 tx=0x7f105c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 shutdown_connections 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f104406b4d0 0x7f104406d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f106415b1c0 0x7f10641f54e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1064150b90 0x7f10641f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 --2- 172.21.15.148:0/4040378616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10640cd4c0 0x7f1064150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 >> 172.21.15.148:0/4040378616 conn(0x7f106406abe0 msgr2=0x7f106415d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 shutdown_connections 2024-04-25T10:02:29.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:29.949+0000 7f1068b32640 1 -- 172.21.15.148:0/4040378616 wait complete. 2024-04-25T10:02:29.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:30.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.061+0000 7fee3129e640 1 Processor -- start 2024-04-25T10:02:30.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee3129e640 1 -- start start 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee3129e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee2c15dc20 0x7fee2c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee3129e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c161710 0x7fee2c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee3129e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c15e000 con 0x7fee2c0b9330 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee3129e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c0d1de0 con 0x7fee2c15dc20 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee3129e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c161c60 con 0x7fee2c161710 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee2affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee2c15dc20 0x7fee2c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee2b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee2c0b9330 0x7fee2c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56434/0 (socket says 172.21.15.148:56434) 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.062+0000 7fee2b7fe640 1 -- 172.21.15.148:0/3202891838 learned_addr learned my addr 172.21.15.148:0/3202891838 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:30.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee2a7fc640 1 --2- 172.21.15.148:0/3202891838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c161710 0x7fee2c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee2affd640 1 -- 172.21.15.148:0/3202891838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c161710 msgr2=0x7fee2c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee2affd640 1 --2- 172.21.15.148:0/3202891838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c161710 0x7fee2c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee2affd640 1 -- 172.21.15.148:0/3202891838 >> v1:172.21.15.148:6789/0 conn(0x7fee2c0b9330 legacy=0x7fee2c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee2affd640 1 -- 172.21.15.148:0/3202891838 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c161f40 con 0x7fee2c15dc20 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee29ffb640 1 -- 172.21.15.148:0/3202891838 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168903605 0 0) 0x7fee2c15e000 con 0x7fee2c0b9330 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.063+0000 7fee2affd640 1 --2- 172.21.15.148:0/3202891838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0ce670 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009d90 tx=0x7fee1c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9734ed68cacbc1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee29ffb640 1 -- 172.21.15.148:0/3202891838 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c007740 con 0x7fee2c15dc20 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee29ffb640 1 -- 172.21.15.148:0/3202891838 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c0078e0 con 0x7fee2c15dc20 2024-04-25T10:02:30.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee29ffb640 1 -- 172.21.15.148:0/3202891838 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c007cb0 con 0x7fee2c15dc20 2024-04-25T10:02:30.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 -- 172.21.15.148:0/3202891838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 msgr2=0x7fee2c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 --2- 172.21.15.148:0/3202891838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0ce670 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009d90 tx=0x7fee1c02a660 comp rx=0 tx=0).stop 2024-04-25T10:02:30.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 -- 172.21.15.148:0/3202891838 shutdown_connections 2024-04-25T10:02:30.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 --2- 172.21.15.148:0/3202891838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c161710 0x7fee2c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 --2- 172.21.15.148:0/3202891838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0ce670 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 -- 172.21.15.148:0/3202891838 >> 172.21.15.148:0/3202891838 conn(0x7fee2c0d0420 msgr2=0x7fee2c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 -- 172.21.15.148:0/3202891838 shutdown_connections 2024-04-25T10:02:30.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.064+0000 7fee3129e640 1 -- 172.21.15.148:0/3202891838 wait complete. 2024-04-25T10:02:30.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.065+0000 7fee3129e640 1 Processor -- start 2024-04-25T10:02:30.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.065+0000 7fee3129e640 1 -- start start 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee3129e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c0b9330 0x7fee2c1f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee3129e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee3129e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee2c161710 0x7fee2c0aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee3129e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b5e50 con 0x7fee2c15dc20 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee3129e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c162500 con 0x7fee2c161710 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee3129e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c0b5fd0 con 0x7fee2c0b9330 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee2a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee2a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40050/0 (socket says 172.21.15.148:40050) 2024-04-25T10:02:30.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee2a7fc640 1 -- 172.21.15.148:0/1495983335 learned_addr learned my addr 172.21.15.148:0/1495983335 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee2affd640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c0b9330 0x7fee2c1f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.066+0000 7fee2b7fe640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee2c161710 0x7fee2c0aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee2a7fc640 1 -- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c0b9330 msgr2=0x7fee2c1f0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee2a7fc640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c0b9330 0x7fee2c1f0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee2a7fc640 1 -- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee2c161710 msgr2=0x7fee2c0aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee2a7fc640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee2c161710 0x7fee2c0aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee2a7fc640 1 -- 172.21.15.148:0/1495983335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c1f0f80 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee2a7fc640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0aa690 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d620 tx=0x7fee2000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000b620 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20019070 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000ba00 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c0ab200 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.067+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c0ab7c0 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.068+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c0d16b0 con 0x7fee2c15dc20 2024-04-25T10:02:30.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.068+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee2000bba0 con 0x7fee2c15dc20 2024-04-25T10:02:30.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.069+0000 7fee137fe640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b4d0 0x7fee1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.069+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee2008a700 con 0x7fee2c15dc20 2024-04-25T10:02:30.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.069+0000 7fee2affd640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b4d0 0x7fee1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.069+0000 7fee2affd640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b4d0 0x7fee1806d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee1c034000 tx=0x7fee1c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.073+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee200593f0 con 0x7fee2c15dc20 2024-04-25T10:02:30.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.242+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee2c0ce470 con 0x7fee2c15dc20 2024-04-25T10:02:30.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.243+0000 7fee137fe640 1 -- 172.21.15.148:0/1495983335 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee2005f0f0 con 0x7fee2c15dc20 2024-04-25T10:02:30.248 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b4d0 msgr2=0x7fee1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b4d0 0x7fee1806d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee1c034000 tx=0x7fee1c02f000 comp rx=0 tx=0).stop 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 msgr2=0x7fee2c0aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0aa690 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d620 tx=0x7fee2000dae0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 shutdown_connections 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee1806b4d0 0x7fee1806d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee2c161710 0x7fee2c0aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee2c15dc20 0x7fee2c0aa690 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 --2- 172.21.15.148:0/1495983335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee2c0b9330 0x7fee2c1f0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 >> 172.21.15.148:0/1495983335 conn(0x7fee2c0d0420 msgr2=0x7fee2c0b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 shutdown_connections 2024-04-25T10:02:30.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.245+0000 7fee3129e640 1 -- 172.21.15.148:0/1495983335 wait complete. 2024-04-25T10:02:30.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:30.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.358+0000 7f054b19b640 1 Processor -- start 2024-04-25T10:02:30.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.358+0000 7f054b19b640 1 -- start start 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f054b19b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054415dac0 0x7f05440d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f054b19b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f05440d0ab0 0x7f05440b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f054b19b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05440d2420 con 0x7f05440d1f30 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f054b19b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05440d3030 con 0x7f05440d0ab0 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f054b19b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440d2700 con 0x7f054415dac0 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f0549738640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f05440d0ab0 0x7f05440b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f0543fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f05440d1f30 0x7f05440d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56448/0 (socket says 172.21.15.148:56448) 2024-04-25T10:02:30.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.359+0000 7f0543fff640 1 -- 172.21.15.148:0/3244162124 learned_addr learned my addr 172.21.15.148:0/3244162124 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0548f37640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054415dac0 0x7f05440d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0549738640 1 -- 172.21.15.148:0/3244162124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054415dac0 msgr2=0x7f05440d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0549738640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054415dac0 0x7f05440d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0549738640 1 -- 172.21.15.148:0/3244162124 >> v1:172.21.15.148:6789/0 conn(0x7f05440d1f30 legacy=0x7f05440d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0549738640 1 -- 172.21.15.148:0/3244162124 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440d29e0 con 0x7f05440d0ab0 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f05437fe640 1 -- 172.21.15.148:0/3244162124 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981506160 0 0) 0x7f05440d2420 con 0x7f05440d1f30 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0548f37640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054415dac0 0x7f05440d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f0549738640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d0ab0 0x7f05440b6990 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f053c009d90 tx=0x7f053c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efecff43a1251c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f05437fe640 1 -- 172.21.15.148:0/3244162124 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c007740 con 0x7f05440d0ab0 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f05437fe640 1 -- 172.21.15.148:0/3244162124 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c0078e0 con 0x7f05440d0ab0 2024-04-25T10:02:30.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.360+0000 7f05437fe640 1 -- 172.21.15.148:0/3244162124 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c007cb0 con 0x7f05440d0ab0 2024-04-25T10:02:30.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 -- 172.21.15.148:0/3244162124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d0ab0 msgr2=0x7f05440b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d0ab0 0x7f05440b6990 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f053c009d90 tx=0x7f053c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 -- 172.21.15.148:0/3244162124 shutdown_connections 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d0ab0 0x7f05440b6990 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 --2- 172.21.15.148:0/3244162124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054415dac0 0x7f05440d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 -- 172.21.15.148:0/3244162124 >> 172.21.15.148:0/3244162124 conn(0x7f0544061c30 msgr2=0x7f054406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 -- 172.21.15.148:0/3244162124 shutdown_connections 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.361+0000 7f054b19b640 1 -- 172.21.15.148:0/3244162124 wait complete. 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.362+0000 7f054b19b640 1 Processor -- start 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.362+0000 7f054b19b640 1 -- start start 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f054b19b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05440d0ab0 0x7f05440c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f054b19b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 0x7f0544149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f054b19b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054415dac0 0x7f05440c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f054b19b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b97d0 con 0x7f05440d1f30 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f054b19b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054415dea0 con 0x7f054415dac0 2024-04-25T10:02:30.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f054b19b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440b9950 con 0x7f05440d0ab0 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 0x7f0544149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 0x7f0544149ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40072/0 (socket says 172.21.15.148:40072) 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 -- 172.21.15.148:0/1417845055 learned_addr learned my addr 172.21.15.148:0/1417845055 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0548f37640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05440d0ab0 0x7f05440c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0549738640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054415dac0 0x7f05440c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 -- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05440d0ab0 msgr2=0x7f05440c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05440d0ab0 0x7f05440c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 -- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054415dac0 msgr2=0x7f05440c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054415dac0 0x7f05440c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.363+0000 7f0543fff640 1 -- 172.21.15.148:0/1417845055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440c9c00 con 0x7f05440d1f30 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.364+0000 7f0543fff640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 0x7f0544149ca0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f053800c730 tx=0x7f053800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.364+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538007270 con 0x7f05440d1f30 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.364+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538007410 con 0x7f05440d1f30 2024-04-25T10:02:30.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.364+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05440c9ee0 con 0x7f05440d1f30 2024-04-25T10:02:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.364+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538001d00 con 0x7f05440d1f30 2024-04-25T10:02:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.364+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05440c76c0 con 0x7f05440d1f30 2024-04-25T10:02:30.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.365+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0544073a00 con 0x7f05440d1f30 2024-04-25T10:02:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.365+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05380057a0 con 0x7f05440d1f30 2024-04-25T10:02:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.366+0000 7f05417fa640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f053006b500 0x7f053006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.366+0000 7f0548f37640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f053006b500 0x7f053006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.366+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f053808a090 con 0x7f05440d1f30 2024-04-25T10:02:30.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.366+0000 7f0548f37640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f053006b500 0x7f053006d9c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f053400a950 tx=0x7f0534008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.370+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0538058d80 con 0x7f05440d1f30 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.540+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f054415ac90 con 0x7f05440d1f30 2024-04-25T10:02:30.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.540+0000 7f05417fa640 1 -- 172.21.15.148:0/1417845055 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0538058680 con 0x7f05440d1f30 2024-04-25T10:02:30.545 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f053006b500 msgr2=0x7f053006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f053006b500 0x7f053006d9c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f053400a950 tx=0x7f0534008040 comp rx=0 tx=0).stop 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 msgr2=0x7f0544149ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 0x7f0544149ca0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f053800c730 tx=0x7f053800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 shutdown_connections 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f053006b500 0x7f053006d9c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054415dac0 0x7f05440c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05440d1f30 0x7f0544149ca0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 --2- 172.21.15.148:0/1417845055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05440d0ab0 0x7f05440c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 >> 172.21.15.148:0/1417845055 conn(0x7f0544061c30 msgr2=0x7f05440ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 shutdown_connections 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.542+0000 7f054b19b640 1 -- 172.21.15.148:0/1417845055 wait complete. 2024-04-25T10:02:30.558 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:30.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.650+0000 7f9201f70640 1 Processor -- start 2024-04-25T10:02:30.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.650+0000 7f9201f70640 1 -- start start 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f9201f70640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f91fc15dc00 0x7f91fc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f9201f70640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc15dfe0 con 0x7f91fc0d1dd0 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f9201f70640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc0d00c0 con 0x7f91fc0ceb20 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f9201f70640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc0d03a0 con 0x7f91fc15dc00 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f91fc15dc00 0x7f91fc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f91fc0d1dd0 0x7f91fc0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56456/0 (socket says 172.21.15.148:56456) 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fbfff640 1 -- 172.21.15.148:0/4157499476 learned_addr learned my addr 172.21.15.148:0/4157499476 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fb7fe640 1 -- 172.21.15.148:0/4157499476 >> v1:172.21.15.188:6789/0 conn(0x7f91fc0ceb20 legacy=0x7f91fc0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fb7fe640 1 -- 172.21.15.148:0/4157499476 >> v1:172.21.15.148:6789/0 conn(0x7f91fc0d1dd0 legacy=0x7f91fc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fb7fe640 1 -- 172.21.15.148:0/4157499476 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0cc440 con 0x7f91fc15dc00 2024-04-25T10:02:30.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.651+0000 7f91fa7fc640 1 -- 172.21.15.148:0/4157499476 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380134220 0 0) 0x7f91f4001490 con 0x7f91fc0d1dd0 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.652+0000 7f91fb7fe640 1 --2- 172.21.15.148:0/4157499476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc0c6df0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0097a0 tx=0x7f91ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663bb0ac18aac6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.652+0000 7f91fa7fc640 1 -- 172.21.15.148:0/4157499476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec037070 con 0x7f91fc15dc00 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.652+0000 7f91fa7fc640 1 -- 172.21.15.148:0/4157499476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec032440 con 0x7f91fc15dc00 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.652+0000 7f91fa7fc640 1 -- 172.21.15.148:0/4157499476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec032810 con 0x7f91fc15dc00 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 -- 172.21.15.148:0/4157499476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 msgr2=0x7f91fc0c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 --2- 172.21.15.148:0/4157499476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc0c6df0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0097a0 tx=0x7f91ec02a230 comp rx=0 tx=0).stop 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 -- 172.21.15.148:0/4157499476 shutdown_connections 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 --2- 172.21.15.148:0/4157499476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc0c6df0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 -- 172.21.15.148:0/4157499476 >> 172.21.15.148:0/4157499476 conn(0x7f91fc06f5d0 msgr2=0x7f91fc06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 -- 172.21.15.148:0/4157499476 shutdown_connections 2024-04-25T10:02:30.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.653+0000 7f9201f70640 1 -- 172.21.15.148:0/4157499476 wait complete. 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.654+0000 7f9201f70640 1 Processor -- start 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.654+0000 7f9201f70640 1 -- start start 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91fc0ceb20 0x7f91fc152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91fc0d1dd0 0x7f91fc1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc1e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0cfd90 con 0x7f91fc15dc00 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0cc130 con 0x7f91fc0d1dd0 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0cdd70 con 0x7f91fc0ceb20 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc1e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc1e8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40080/0 (socket says 172.21.15.148:40080) 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 -- 172.21.15.148:0/3864234825 learned_addr learned my addr 172.21.15.148:0/3864234825 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91faffd640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91fc0d1dd0 0x7f91fc1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fb7fe640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91fc0ceb20 0x7f91fc152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 -- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91fc0ceb20 msgr2=0x7f91fc152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91fc0ceb20 0x7f91fc152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 -- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91fc0d1dd0 msgr2=0x7f91fc1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91fc0d1dd0 0x7f91fc1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 -- 172.21.15.148:0/3864234825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0c9c40 con 0x7f91fc15dc00 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91fbfff640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc1e8b70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c760 tx=0x7f91f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4018070 con 0x7f91fc15dc00 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f400a640 con 0x7f91fc15dc00 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc0c9f20 con 0x7f91fc15dc00 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400aa30 con 0x7f91fc15dc00 2024-04-25T10:02:30.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.655+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc0ca4e0 con 0x7f91fc15dc00 2024-04-25T10:02:30.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.656+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc073990 con 0x7f91fc15dc00 2024-04-25T10:02:30.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.657+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91f401b070 con 0x7f91fc15dc00 2024-04-25T10:02:30.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.657+0000 7f91e3fff640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f91e806b500 0x7f91e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.657+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91f4089810 con 0x7f91fc15dc00 2024-04-25T10:02:30.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.657+0000 7f91fb7fe640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f91e806b500 0x7f91e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.658+0000 7f91fb7fe640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f91e806b500 0x7f91e806d9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f91ec034000 tx=0x7f91ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.662+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91fc15dfe0 con 0x7f91fc15dc00 2024-04-25T10:02:30.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.867+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91fc0ca6b0 con 0x7f91fc15dc00 2024-04-25T10:02:30.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.868+0000 7f91e3fff640 1 -- 172.21.15.148:0/3864234825 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f91f4058590 con 0x7f91fc15dc00 2024-04-25T10:02:30.872 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:30.873 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:30.873 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:30.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.869+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f91e806b500 msgr2=0x7f91e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.869+0000 7f9201f70640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f91e806b500 0x7f91e806d9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f91ec034000 tx=0x7f91ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.869+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 msgr2=0x7f91fc1e8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.869+0000 7f9201f70640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc1e8b70 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f91f400c760 tx=0x7f91f400cc20 comp rx=0 tx=0).stop 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 shutdown_connections 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f91e806b500 0x7f91e806d9c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91fc15dc00 0x7f91fc1e8b70 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91fc0d1dd0 0x7f91fc1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 --2- 172.21.15.148:0/3864234825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91fc0ceb20 0x7f91fc152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 >> 172.21.15.148:0/3864234825 conn(0x7f91fc06f5d0 msgr2=0x7f91fc0c7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 shutdown_connections 2024-04-25T10:02:30.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.870+0000 7f9201f70640 1 -- 172.21.15.148:0/3864234825 wait complete. 2024-04-25T10:02:30.886 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:30.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.978+0000 7fb05a1a5640 1 Processor -- start 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb05a1a5640 1 -- start start 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb05a1a5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0540d1020 0x7fb0540d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb05a1a5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540b97a0 con 0x7fb0540b2cb0 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb05a1a5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540d1c50 con 0x7fb05415dbe0 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb05a1a5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05415dfc0 con 0x7fb0540d1020 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb053fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0540d1020 0x7fb0540d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb052ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb0540b2cb0 0x7fb0540d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56460/0 (socket says 172.21.15.148:56460) 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.979+0000 7fb052ffd640 1 -- 172.21.15.148:0/2083891341 learned_addr learned my addr 172.21.15.148:0/2083891341 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.980+0000 7fb053fff640 1 -- 172.21.15.148:0/2083891341 >> v1:172.21.15.188:6789/0 conn(0x7fb05415dbe0 legacy=0x7fb0540d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.980+0000 7fb053fff640 1 -- 172.21.15.148:0/2083891341 >> v1:172.21.15.148:6789/0 conn(0x7fb0540b2cb0 legacy=0x7fb0540d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.980+0000 7fb053fff640 1 -- 172.21.15.148:0/2083891341 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540d1f30 con 0x7fb0540d1020 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.980+0000 7fb053fff640 1 --2- 172.21.15.148:0/2083891341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0540d1020 0x7fb0540d2e40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009d90 tx=0x7fb04c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de54d1deabed3c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb0527fc640 1 -- 172.21.15.148:0/2083891341 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c007740 con 0x7fb0540d1020 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb0527fc640 1 -- 172.21.15.148:0/2083891341 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c0078e0 con 0x7fb0540d1020 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb0527fc640 1 -- 172.21.15.148:0/2083891341 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c007cb0 con 0x7fb0540d1020 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 -- 172.21.15.148:0/2083891341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0540d1020 msgr2=0x7fb0540d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/2083891341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0540d1020 0x7fb0540d2e40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009d90 tx=0x7fb04c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 -- 172.21.15.148:0/2083891341 shutdown_connections 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/2083891341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0540d1020 0x7fb0540d2e40 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 -- 172.21.15.148:0/2083891341 >> 172.21.15.148:0/2083891341 conn(0x7fb054061a20 msgr2=0x7fb054061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 -- 172.21.15.148:0/2083891341 shutdown_connections 2024-04-25T10:02:30.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.981+0000 7fb05a1a5640 1 -- 172.21.15.148:0/2083891341 wait complete. 2024-04-25T10:02:30.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.982+0000 7fb05a1a5640 1 Processor -- start 2024-04-25T10:02:30.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.982+0000 7fb05a1a5640 1 -- start start 2024-04-25T10:02:30.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb05a1a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540b2cb0 0x7fb0540b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb05a1a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0540d1020 0x7fb0541556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb05a1a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 0x7fb0541547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb05a1a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b92a0 con 0x7fb05415dbe0 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb05a1a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540d1a00 con 0x7fb0540b2cb0 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb05a1a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b9420 con 0x7fb0540d1020 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb053fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 0x7fb0541547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb053fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 0x7fb0541547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40100/0 (socket says 172.21.15.148:40100) 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb053fff640 1 -- 172.21.15.148:0/1817861239 learned_addr learned my addr 172.21.15.148:0/1817861239 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb052ffd640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0540d1020 0x7fb0541556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.983+0000 7fb0537fe640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540b2cb0 0x7fb0540b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb053fff640 1 -- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0540d1020 msgr2=0x7fb0541556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb053fff640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0540d1020 0x7fb0541556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb053fff640 1 -- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540b2cb0 msgr2=0x7fb0540b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb053fff640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540b2cb0 0x7fb0540b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb053fff640 1 -- 172.21.15.148:0/1817861239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054155f30 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb052ffd640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0540d1020 0x7fb0541556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb0537fe640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540b2cb0 0x7fb0540b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb053fff640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 0x7fb0541547b0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009eb0 tx=0x7fb04c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c040070 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c031c40 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb054156210 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c03c420 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.984+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0541567d0 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.985+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0541595c0 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.985+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb04c039040 con 0x7fb05415dbe0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.986+0000 7fb03bfff640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb04006b4d0 0x7fb04006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.986+0000 7fb0537fe640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb04006b4d0 0x7fb04006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.986+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb04c0aba00 con 0x7fb05415dbe0 2024-04-25T10:02:30.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.986+0000 7fb0537fe640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb04006b4d0 0x7fb04006d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb044006fd0 tx=0x7fb044008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:30.990+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0541595c0 con 0x7fb05415dbe0 2024-04-25T10:02:31.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.160+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb054073990 con 0x7fb05415dbe0 2024-04-25T10:02:31.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.161+0000 7fb03bfff640 1 -- 172.21.15.148:0/1817861239 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17000 (secure 0 0 0) 0x7fb04c07a6f0 con 0x7fb05415dbe0 2024-04-25T10:02:31.166 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:31.167 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":121,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:54.304228+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.163+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb04006b4d0 msgr2=0x7fb04006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.163+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb04006b4d0 0x7fb04006d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb044006fd0 tx=0x7fb044008040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 msgr2=0x7fb0541547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 0x7fb0541547b0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009eb0 tx=0x7fb04c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 shutdown_connections 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb04006b4d0 0x7fb04006d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05415dbe0 0x7fb0541547b0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0540d1020 0x7fb0541556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 --2- 172.21.15.148:0/1817861239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0540b2cb0 0x7fb0540b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 >> 172.21.15.148:0/1817861239 conn(0x7fb054061a20 msgr2=0x7fb05415a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 shutdown_connections 2024-04-25T10:02:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.164+0000 7fb05a1a5640 1 -- 172.21.15.148:0/1817861239 wait complete. 2024-04-25T10:02:31.180 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:02:31.180 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:31.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 Processor -- start 2024-04-25T10:02:31.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 -- start start 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f341015db60 0x7f34100ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34100b5f00 0x7f34100d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34100cec00 con 0x7f341015df40 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34100ce930 con 0x7f341015db60 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.274+0000 7f341665d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34100b6450 con 0x7f34100b5f00 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f341015db60 0x7f34100ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f3414bfa640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f341015df40 0x7f3410161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56470/0 (socket says 172.21.15.148:56470) 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f3414bfa640 1 -- 172.21.15.148:0/458653398 learned_addr learned my addr 172.21.15.148:0/458653398 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340f7fe640 1 --2- 172.21.15.148:0/458653398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34100b5f00 0x7f34100d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340ffff640 1 -- 172.21.15.148:0/458653398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34100b5f00 msgr2=0x7f34100d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340ffff640 1 --2- 172.21.15.148:0/458653398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34100b5f00 0x7f34100d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340ffff640 1 -- 172.21.15.148:0/458653398 >> v1:172.21.15.148:6789/0 conn(0x7f341015df40 legacy=0x7f3410161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340ffff640 1 -- 172.21.15.148:0/458653398 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100ceee0 con 0x7f341015db60 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.275+0000 7f340ffff640 1 --2- 172.21.15.148:0/458653398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015db60 0x7f34100ce3f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f34040097d0 tx=0x7f340402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258bec83f47a93f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f340effd640 1 -- 172.21.15.148:0/458653398 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404037070 con 0x7f341015db60 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f340effd640 1 -- 172.21.15.148:0/458653398 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340402ae40 con 0x7f341015db60 2024-04-25T10:02:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f340effd640 1 -- 172.21.15.148:0/458653398 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404032660 con 0x7f341015db60 2024-04-25T10:02:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 -- 172.21.15.148:0/458653398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015db60 msgr2=0x7f34100ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 --2- 172.21.15.148:0/458653398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015db60 0x7f34100ce3f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f34040097d0 tx=0x7f340402a130 comp rx=0 tx=0).stop 2024-04-25T10:02:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 -- 172.21.15.148:0/458653398 shutdown_connections 2024-04-25T10:02:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 --2- 172.21.15.148:0/458653398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34100b5f00 0x7f34100d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 --2- 172.21.15.148:0/458653398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015db60 0x7f34100ce3f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 -- 172.21.15.148:0/458653398 >> 172.21.15.148:0/458653398 conn(0x7f3410061600 msgr2=0x7f3410061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 -- 172.21.15.148:0/458653398 shutdown_connections 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.276+0000 7f341665d640 1 -- 172.21.15.148:0/458653398 wait complete. 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.277+0000 7f341665d640 1 Processor -- start 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 -- start start 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34100b5f00 0x7f34101f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f341015db60 0x7f341015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 0x7f34101f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34100d3280 con 0x7f341015df40 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34100d1d50 con 0x7f34100b5f00 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f341665d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34100d3400 con 0x7f341015db60 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f3414bfa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 0x7f34101f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f3414bfa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 0x7f34101f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40116/0 (socket says 172.21.15.148:40116) 2024-04-25T10:02:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.278+0000 7f3414bfa640 1 -- 172.21.15.148:0/3803599233 learned_addr learned my addr 172.21.15.148:0/3803599233 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f340ffff640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34100b5f00 0x7f34101f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f340f7fe640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f341015db60 0x7f341015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f3414bfa640 1 -- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f341015db60 msgr2=0x7f341015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f3414bfa640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f341015db60 0x7f341015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f3414bfa640 1 -- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34100b5f00 msgr2=0x7f34101f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f3414bfa640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34100b5f00 0x7f34101f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f3414bfa640 1 -- 172.21.15.148:0/3803599233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34101f7840 con 0x7f341015df40 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f340ffff640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34100b5f00 0x7f34101f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f3414bfa640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 0x7f34101f6e60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f340800cbc0 tx=0x7f340800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408017070 con 0x7f341015df40 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800ad40 con 0x7f341015df40 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34101f7b20 con 0x7f341015df40 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408012600 con 0x7f341015df40 2024-04-25T10:02:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.279+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34101f3d60 con 0x7f341015df40 2024-04-25T10:02:31.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.280+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3410073270 con 0x7f341015df40 2024-04-25T10:02:31.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.280+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f340801a070 con 0x7f341015df40 2024-04-25T10:02:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.281+0000 7f340cff9640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33fc06b4d0 0x7f33fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.281+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3408089f20 con 0x7f341015df40 2024-04-25T10:02:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.281+0000 7f340ffff640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33fc06b4d0 0x7f33fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.281+0000 7f340ffff640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33fc06b4d0 0x7f33fc06d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3404034000 tx=0x7f340402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.285+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3408058650 con 0x7f341015df40 2024-04-25T10:02:31.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.491+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34101f3870 con 0x7f341015df40 2024-04-25T10:02:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.491+0000 7f340cff9640 1 -- 172.21.15.148:0/3803599233 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f340805c6f0 con 0x7f341015df40 2024-04-25T10:02:31.496 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:31.496 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:31.496 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33fc06b4d0 msgr2=0x7f33fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33fc06b4d0 0x7f33fc06d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3404034000 tx=0x7f340402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 msgr2=0x7f34101f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 0x7f34101f6e60 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f340800cbc0 tx=0x7f340800a430 comp rx=0 tx=0).stop 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 shutdown_connections 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33fc06b4d0 0x7f33fc06d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341015df40 0x7f34101f6e60 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f341015db60 0x7f341015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 --2- 172.21.15.148:0/3803599233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34100b5f00 0x7f34101f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 >> 172.21.15.148:0/3803599233 conn(0x7f3410061600 msgr2=0x7f34100d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 shutdown_connections 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.493+0000 7f341665d640 1 -- 172.21.15.148:0/3803599233 wait complete. 2024-04-25T10:02:31.509 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:31.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.601+0000 7f8ed9153640 1 Processor -- start 2024-04-25T10:02:31.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.601+0000 7f8ed9153640 1 -- start start 2024-04-25T10:02:31.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed9153640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed9153640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ed40afba0 0x7f8ed4161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed9153640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed415f180 con 0x7f8ed40af720 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed9153640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed40b3670 con 0x7f8ed40afba0 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed9153640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4070a10 con 0x7f8ed4158220 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ed40afba0 0x7f8ed4161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ed40afba0 0x7f8ed4161ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40124/0 (socket says 172.21.15.148:40124) 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed359e640 1 -- 172.21.15.148:0/708287407 learned_addr learned my addr 172.21.15.148:0/708287407 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.602+0000 7f8ed2d9d640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed359e640 1 -- 172.21.15.148:0/708287407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 msgr2=0x7f8ed415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed359e640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed359e640 1 -- 172.21.15.148:0/708287407 >> v1:172.21.15.188:6789/0 conn(0x7f8ed40af720 legacy=0x7f8ed41617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed359e640 1 -- 172.21.15.148:0/708287407 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed40b0030 con 0x7f8ed40afba0 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed2d9d640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:31.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed359e640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4161ec0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009870 tx=0x7f8ecc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aa242bd3b3e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed1d9b640 1 -- 172.21.15.148:0/708287407 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc037070 con 0x7f8ed40afba0 2024-04-25T10:02:31.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed1d9b640 1 -- 172.21.15.148:0/708287407 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc032440 con 0x7f8ed40afba0 2024-04-25T10:02:31.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.603+0000 7f8ed1d9b640 1 -- 172.21.15.148:0/708287407 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc032810 con 0x7f8ed40afba0 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 -- 172.21.15.148:0/708287407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 msgr2=0x7f8ed4161ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4161ec0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009870 tx=0x7f8ecc02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 -- 172.21.15.148:0/708287407 shutdown_connections 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4161ec0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 --2- 172.21.15.148:0/708287407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 -- 172.21.15.148:0/708287407 >> 172.21.15.148:0/708287407 conn(0x7f8ed40b2370 msgr2=0x7f8ed40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 -- 172.21.15.148:0/708287407 shutdown_connections 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.604+0000 7f8ed9153640 1 -- 172.21.15.148:0/708287407 wait complete. 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.605+0000 7f8ed9153640 1 Processor -- start 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 -- start start 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ed40af720 0x7f8ed4146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40703c0 con 0x7f8ed40afba0 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40700e0 con 0x7f8ed40af720 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed9153640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40706a0 con 0x7f8ed4158220 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40126/0 (socket says 172.21.15.148:40126) 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.606+0000 7f8ed259c640 1 -- 172.21.15.148:0/3901690910 learned_addr learned my addr 172.21.15.148:0/3901690910 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed2d9d640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ed40af720 0x7f8ed4146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed359e640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed259c640 1 -- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 msgr2=0x7f8ed415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed259c640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed259c640 1 -- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ed40af720 msgr2=0x7f8ed4146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed259c640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ed40af720 0x7f8ed4146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed259c640 1 -- 172.21.15.148:0/3901690910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4149940 con 0x7f8ed40afba0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed359e640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed259c640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4149400 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c3c0 tx=0x7f8ec800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80052d0 con 0x7f8ed40afba0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8005490 con 0x7f8ed40afba0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8007260 con 0x7f8ed40afba0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed415a750 con 0x7f8ed40afba0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.607+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed415ad10 con 0x7f8ed40afba0 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.608+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed4073990 con 0x7f8ed40afba0 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.608+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ec8007400 con 0x7f8ed40afba0 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.609+0000 7f8ebb7fe640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ec006b500 0x7f8ec006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.609+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ec80897e0 con 0x7f8ed40afba0 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.609+0000 7f8ed2d9d640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ec006b500 0x7f8ec006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.609+0000 7f8ed2d9d640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ec006b500 0x7f8ec006d9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4003fd0 tx=0x7f8ec40039f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.613+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ed415f180 con 0x7f8ed40afba0 2024-04-25T10:02:31.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.820+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ed406d330 con 0x7f8ed40afba0 2024-04-25T10:02:31.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.821+0000 7f8ebb7fe640 1 -- 172.21.15.148:0/3901690910 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8ec80584d0 con 0x7f8ed40afba0 2024-04-25T10:02:31.826 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:31.826 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:52.637253+0000","modified":"2024-04-25T10:01:55.153797+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"d","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/609722939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":609722939},{"type":"v1","addr":"172.21.15.188:6835","nonce":609722939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:02:31.826 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 67 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ec006b500 msgr2=0x7f8ec006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ec006b500 0x7f8ec006d9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4003fd0 tx=0x7f8ec40039f0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 msgr2=0x7f8ed4149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4149400 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c3c0 tx=0x7f8ec800c880 comp rx=0 tx=0).stop 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 shutdown_connections 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8ec006b500 0x7f8ec006d9c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ed4158220 0x7f8ed415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ed40afba0 0x7f8ed4149400 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 --2- 172.21.15.148:0/3901690910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ed40af720 0x7f8ed4146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 >> 172.21.15.148:0/3901690910 conn(0x7f8ed40b2370 msgr2=0x7f8ed40d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 shutdown_connections 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.823+0000 7f8ed9153640 1 -- 172.21.15.148:0/3901690910 wait complete. 2024-04-25T10:02:31.839 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.931+0000 7f16dfc6b640 1 Processor -- start 2024-04-25T10:02:31.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.931+0000 7f16dfc6b640 1 -- start start 2024-04-25T10:02:31.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dfc6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d8154ee0 0x7f16d80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dfc6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16d80cfcf0 0x7f16d80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dfc6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16d80ce3c0 0x7f16d815e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dfc6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80ce830 con 0x7f16d80cfcf0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dfc6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d8072090 con 0x7f16d80ce3c0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dfc6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80d0150 con 0x7f16d8154ee0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16d80cfcf0 0x7f16d80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16d80cfcf0 0x7f16d80cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40136/0 (socket says 172.21.15.148:40136) 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 -- 172.21.15.148:0/813204817 learned_addr learned my addr 172.21.15.148:0/813204817 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 -- 172.21.15.148:0/813204817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d8154ee0 msgr2=0x7f16d80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16de208640 1 --2- 172.21.15.148:0/813204817 >> v2:172.21.15.188:3300/0 conn(0x7f16d80ce3c0 0x7f16d815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dda07640 1 --2- 172.21.15.148:0/813204817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d8154ee0 0x7f16d80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 --2- 172.21.15.148:0/813204817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d8154ee0 0x7f16d80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 -- 172.21.15.148:0/813204817 >> v2:172.21.15.188:3300/0 conn(0x7f16d80ce3c0 msgr2=0x7f16d815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 --2- 172.21.15.148:0/813204817 >> v2:172.21.15.188:3300/0 conn(0x7f16d80ce3c0 0x7f16d815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.932+0000 7f16dd206640 1 -- 172.21.15.148:0/813204817 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8071480 con 0x7f16d80cfcf0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dd206640 1 --2- 172.21.15.148:0/813204817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d80cfcf0 0x7f16d80cde80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f16d806f6b0 tx=0x7f16c802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c771aee4d42513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dca05640 1 -- 172.21.15.148:0/813204817 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8037070 con 0x7f16d80cfcf0 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dca05640 1 -- 172.21.15.148:0/813204817 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8032440 con 0x7f16d80cfcf0 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dca05640 1 -- 172.21.15.148:0/813204817 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8032810 con 0x7f16d80cfcf0 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 -- 172.21.15.148:0/813204817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d80cfcf0 msgr2=0x7f16d80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/813204817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d80cfcf0 0x7f16d80cde80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f16d806f6b0 tx=0x7f16c802a210 comp rx=0 tx=0).stop 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 -- 172.21.15.148:0/813204817 shutdown_connections 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/813204817 >> v2:172.21.15.188:3300/0 conn(0x7f16d80ce3c0 0x7f16d815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/813204817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d80cfcf0 0x7f16d80cde80 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/813204817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d8154ee0 0x7f16d80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.933+0000 7f16dfc6b640 1 -- 172.21.15.148:0/813204817 >> 172.21.15.148:0/813204817 conn(0x7f16d806abe0 msgr2=0x7f16d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.934+0000 7f16dfc6b640 1 -- 172.21.15.148:0/813204817 shutdown_connections 2024-04-25T10:02:31.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.934+0000 7f16dfc6b640 1 -- 172.21.15.148:0/813204817 wait complete. 2024-04-25T10:02:31.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 Processor -- start 2024-04-25T10:02:31.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 -- start start 2024-04-25T10:02:31.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d80ce3c0 0x7f16d80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d80cfcf0 0x7f16d81f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 0x7f16d80aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8071a40 con 0x7f16d8154ee0 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8071230 con 0x7f16d80ce3c0 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.935+0000 7f16dfc6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8071d20 con 0x7f16d80cfcf0 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 0x7f16d80aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 0x7f16d80aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40150/0 (socket says 172.21.15.148:40150) 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 -- 172.21.15.148:0/299622158 learned_addr learned my addr 172.21.15.148:0/299622158 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16dda07640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d80ce3c0 0x7f16d80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16dd206640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d80cfcf0 0x7f16d81f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 -- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d80cfcf0 msgr2=0x7f16d81f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d80cfcf0 0x7f16d81f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 -- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d80ce3c0 msgr2=0x7f16d80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d80ce3c0 0x7f16d80b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 -- 172.21.15.148:0/299622158 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80a8540 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16dda07640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d80ce3c0 0x7f16d80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16de208640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 0x7f16d80aedf0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f16d000cd20 tx=0x7f16d000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0003b90 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d0003d30 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004240 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d81f3f30 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.936+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d81f44f0 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.937+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80a8710 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.938+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16d00043e0 con 0x7f16d8154ee0 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.938+0000 7f16ce7fc640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16bc06b4d0 0x7f16bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.938+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16d008ada0 con 0x7f16d8154ee0 2024-04-25T10:02:31.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.938+0000 7f16dda07640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16bc06b4d0 0x7f16bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.938+0000 7f16dda07640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16bc06b4d0 0x7f16bc06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f16d80b0000 tx=0x7f16d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:31.942+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16d0059b20 con 0x7f16d8154ee0 2024-04-25T10:02:32.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.115+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16d81f4830 con 0x7f16d8154ee0 2024-04-25T10:02:32.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.116+0000 7f16ce7fc640 1 -- 172.21.15.148:0/299622158 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f16d0059420 con 0x7f16d8154ee0 2024-04-25T10:02:32.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:32.121 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":121,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:01:54.304228+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:50.849774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:51.601981+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:32.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16bc06b4d0 msgr2=0x7f16bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16bc06b4d0 0x7f16bc06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f16d80b0000 tx=0x7f16d4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 msgr2=0x7f16d80aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 0x7f16d80aedf0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f16d000cd20 tx=0x7f16d000a660 comp rx=0 tx=0).stop 2024-04-25T10:02:32.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 shutdown_connections 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16bc06b4d0 0x7f16bc06d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d8154ee0 0x7f16d80aedf0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16d80cfcf0 0x7f16d81f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 --2- 172.21.15.148:0/299622158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16d80ce3c0 0x7f16d80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 >> 172.21.15.148:0/299622158 conn(0x7f16d806abe0 msgr2=0x7f16d8152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 shutdown_connections 2024-04-25T10:02:32.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.118+0000 7f16dfc6b640 1 -- 172.21.15.148:0/299622158 wait complete. 2024-04-25T10:02:32.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:02:32.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.228+0000 7f313049e640 1 Processor -- start 2024-04-25T10:02:32.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.228+0000 7f313049e640 1 -- start start 2024-04-25T10:02:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.229+0000 7f313049e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f312815bb50 0x7f31280c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.229+0000 7f313049e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280c75e0 con 0x7f31280c70d0 2024-04-25T10:02:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.229+0000 7f313049e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31280c78c0 con 0x7f31280cfd10 2024-04-25T10:02:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.229+0000 7f313049e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31280c8680 con 0x7f312815bb50 2024-04-25T10:02:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f31280c70d0 0x7f31280cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56482/0 (socket says 172.21.15.148:56482) 2024-04-25T10:02:32.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 -- 172.21.15.148:0/3273264073 learned_addr learned my addr 172.21.15.148:0/3273264073 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312e23a640 1 --2- 172.21.15.148:0/3273264073 >> v2:172.21.15.188:3300/0 conn(0x7f312815bb50 0x7f31280c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449778402 0 0) 0x7f31280c75e0 con 0x7f31280c70d0 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3114003520 con 0x7f31280c70d0 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2136432545 0 0) 0x7f3114003520 con 0x7f31280c70d0 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 >> v2:172.21.15.188:3300/0 conn(0x7f312815bb50 msgr2=0x7f31280c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 --2- 172.21.15.148:0/3273264073 >> v2:172.21.15.188:3300/0 conn(0x7f312815bb50 0x7f31280c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 >> v1:172.21.15.188:6789/0 conn(0x7f31280cfd10 legacy=0x7f31280cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280c7ba0 con 0x7f31280c70d0 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3118001670 con 0x7f31280c70d0 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 >> v1:172.21.15.148:6789/0 conn(0x7f31280c70d0 legacy=0x7f31280cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31140039b0 0x7f3114003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3114004800 0x7f3114004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31140051c0 0x7f311400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8680 con 0x7f3114004800 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311400c070 con 0x7f31140051c0 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311400c270 con 0x7f31140039b0 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3114004800 0x7f3114004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312e23a640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31140039b0 0x7f3114003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312ea3b640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31140051c0 0x7f311400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 -- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31140039b0 msgr2=0x7f3114003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31140039b0 0x7f3114003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 -- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31140051c0 msgr2=0x7f311400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.230+0000 7f312da39640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31140051c0 0x7f311400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.231+0000 7f312ea3b640 1 -- 172.21.15.148:0/3273264073 reap_dead start 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.231+0000 7f312da39640 1 -- 172.21.15.148:0/3273264073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3114003520 con 0x7f3114004800 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.231+0000 7f312da39640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3114004800 0x7f3114004c80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f311800c210 tx=0x7f3114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b223a0ddb9a0de17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.231+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118007d10 con 0x7f3114004800 2024-04-25T10:02:32.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.231+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31180058d0 con 0x7f3114004800 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.231+0000 7f312d238640 1 -- 172.21.15.148:0/3273264073 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118005d20 con 0x7f3114004800 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 -- 172.21.15.148:0/3273264073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3114004800 msgr2=0x7f3114004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3114004800 0x7f3114004c80 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f311800c210 tx=0x7f3114005830 comp rx=0 tx=0).stop 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 -- 172.21.15.148:0/3273264073 shutdown_connections 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 --2- 172.21.15.148:0/3273264073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3114004800 0x7f3114004c80 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 -- 172.21.15.148:0/3273264073 >> 172.21.15.148:0/3273264073 conn(0x7f312806f500 msgr2=0x7f312806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 -- 172.21.15.148:0/3273264073 shutdown_connections 2024-04-25T10:02:32.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.232+0000 7f313049e640 1 -- 172.21.15.148:0/3273264073 wait complete. 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.233+0000 7f313049e640 1 Processor -- start 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.233+0000 7f313049e640 1 -- start start 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f313049e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3114004800 0x7f312814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f313049e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 0x7f312815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f313049e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31280b56c0 0x7f31280cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f313049e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312815f6d0 con 0x7f31280cb270 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f313049e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8160 con 0x7f31280b56c0 2024-04-25T10:02:32.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f313049e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312815f850 con 0x7f3114004800 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 0x7f312815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 0x7f312815bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40164/0 (socket says 172.21.15.148:40164) 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 -- 172.21.15.148:0/2570394518 learned_addr learned my addr 172.21.15.148:0/2570394518 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312e23a640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3114004800 0x7f312814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312ea3b640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31280b56c0 0x7f31280cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 -- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3114004800 msgr2=0x7f312814cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3114004800 0x7f312814cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 -- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31280b56c0 msgr2=0x7f31280cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31280b56c0 0x7f31280cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 -- 172.21.15.148:0/2570394518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280b5f10 con 0x7f31280cb270 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f312da39640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 0x7f312815bfc0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f311800c210 tx=0x7f31280cbe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.234+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118007d10 con 0x7f31280cb270 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.235+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31180049f0 con 0x7f31280cb270 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.235+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118004de0 con 0x7f31280cb270 2024-04-25T10:02:32.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.235+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31280b61f0 con 0x7f31280cb270 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.235+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3128159c20 con 0x7f31280cb270 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.236+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3128073a00 con 0x7f31280cb270 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.236+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f311803b070 con 0x7f31280cb270 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.236+0000 7f311effd640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.237+0000 7f312e23a640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.237+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31180aa050 con 0x7f31280cb270 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.237+0000 7f312e23a640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f310c06b4d0 0x7f310c06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3124009b80 tx=0x7f3124008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.241+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3118078d90 con 0x7f31280cb270 2024-04-25T10:02:32.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.447+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f31280cf150 con 0x7f31280cb270 2024-04-25T10:02:32.452 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:02:32.448+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:02:32.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.450+0000 7f311effd640 1 -- 172.21.15.148:0/2570394518 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3118078690 con 0x7f31280cb270 2024-04-25T10:02:32.454 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f310c06b4d0 msgr2=0x7f310c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f310c06b4d0 0x7f310c06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3124009b80 tx=0x7f3124008040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 msgr2=0x7f312815bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 0x7f312815bfc0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f311800c210 tx=0x7f31280cbe70 comp rx=0 tx=0).stop 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 shutdown_connections 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31280b56c0 0x7f31280cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31280cb270 0x7f312815bfc0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 --2- 172.21.15.148:0/2570394518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3114004800 0x7f312814cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 >> 172.21.15.148:0/2570394518 conn(0x7f312806f500 msgr2=0x7f3128156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 shutdown_connections 2024-04-25T10:02:32.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.452+0000 7f313049e640 1 -- 172.21.15.148:0/2570394518 wait complete. 2024-04-25T10:02:32.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:02:32.498 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:02:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.563+0000 7f5bbbd42640 1 Processor -- start 2024-04-25T10:02:32.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.564+0000 7f5bbbd42640 1 -- start start 2024-04-25T10:02:32.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.564+0000 7f5bbbd42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.564+0000 7f5bbbd42640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb406cda0 con 0x7f5bb40d32f0 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bbbd42640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb415e000 con 0x7f5bb4161710 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bbbd42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40d1de0 con 0x7f5bb415dc20 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bba2df640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5bb40d32f0 0x7f5bb406c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56488/0 (socket says 172.21.15.148:56488) 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bba2df640 1 -- 172.21.15.148:0/1742730974 learned_addr learned my addr 172.21.15.148:0/1742730974 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb9ade640 1 --2- 172.21.15.148:0/1742730974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191732424 0 0) 0x7f5bb406cda0 con 0x7f5bb40d32f0 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba0003520 con 0x7f5bb40d32f0 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559400614 0 0) 0x7f5bb415e000 con 0x7f5bb4161710 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb406cda0 con 0x7f5bb4161710 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb9ade640 1 -- 172.21.15.148:0/1742730974 >> v1:172.21.15.188:6789/0 conn(0x7f5bb4161710 legacy=0x7f5bb406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb9ade640 1 -- 172.21.15.148:0/1742730974 >> v1:172.21.15.148:6789/0 conn(0x7f5bb40d32f0 legacy=0x7f5bb406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 244257564 0 0) 0x7f5ba0003520 con 0x7f5bb40d32f0 2024-04-25T10:02:32.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.565+0000 7f5bb9ade640 1 -- 172.21.15.148:0/1742730974 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4161c60 con 0x7f5bb415dc20 2024-04-25T10:02:32.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.566+0000 7f5bb9ade640 1 --2- 172.21.15.148:0/1742730974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb40ce670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40097d0 tx=0x7f5ba402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31294e25d8eebeb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.566+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4037070 con 0x7f5bb415dc20 2024-04-25T10:02:32.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.566+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba402ac30 con 0x7f5bb415dc20 2024-04-25T10:02:32.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.566+0000 7f5bb8adc640 1 -- 172.21.15.148:0/1742730974 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0019f0 con 0x7f5bb415dc20 2024-04-25T10:02:32.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 -- 172.21.15.148:0/1742730974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 msgr2=0x7f5bb40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/1742730974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb40ce670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40097d0 tx=0x7f5ba402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 -- 172.21.15.148:0/1742730974 shutdown_connections 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/1742730974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb40ce670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 -- 172.21.15.148:0/1742730974 >> 172.21.15.148:0/1742730974 conn(0x7f5bb40d0420 msgr2=0x7f5bb40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 -- 172.21.15.148:0/1742730974 shutdown_connections 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.567+0000 7f5bbbd42640 1 -- 172.21.15.148:0/1742730974 wait complete. 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.568+0000 7f5bbbd42640 1 Processor -- start 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.568+0000 7f5bbbd42640 1 -- start start 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 0x7f5bb41f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb4156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb4161710 0x7f5bb40b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4162500 con 0x7f5bb40d32f0 2024-04-25T10:02:32.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4162220 con 0x7f5bb4161710 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40b5e50 con 0x7f5bb415dc20 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 0x7f5bb41f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 0x7f5bb41f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40180/0 (socket says 172.21.15.148:40180) 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 -- 172.21.15.148:0/3333927253 learned_addr learned my addr 172.21.15.148:0/3333927253 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bba2df640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb4161710 0x7f5bb40b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 -- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 msgr2=0x7f5bb4156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb4156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 -- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb4161710 msgr2=0x7f5bb40b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb4161710 0x7f5bb40b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 -- 172.21.15.148:0/3333927253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb40ab550 con 0x7f5bb40d32f0 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bb9ade640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 0x7f5bb41f37c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4035040 tx=0x7f5ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4037040 con 0x7f5bb40d32f0 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4004230 con 0x7f5bb40d32f0 2024-04-25T10:02:32.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb40aaf90 con 0x7f5bb40d32f0 2024-04-25T10:02:32.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4006e10 con 0x7f5bb40d32f0 2024-04-25T10:02:32.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.569+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb415f270 con 0x7f5bb40d32f0 2024-04-25T10:02:32.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.570+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb4073a00 con 0x7f5bb40d32f0 2024-04-25T10:02:32.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.571+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ba4003610 con 0x7f5bb40d32f0 2024-04-25T10:02:32.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.571+0000 7f5baa7fc640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b9806b4d0 0x7f5b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.571+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5ba40ab1c0 con 0x7f5bb40d32f0 2024-04-25T10:02:32.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.571+0000 7f5bb92dd640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b9806b4d0 0x7f5b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.572+0000 7f5bb92dd640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b9806b4d0 0x7f5b9806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00010e0 tx=0x7f5bb0007950 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.575+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ba4079ef0 con 0x7f5bb40d32f0 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:32.782+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5bb40d16b0 con 0x7f5bb40d32f0 2024-04-25T10:02:33.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.450+0000 7f5baa7fc640 1 -- 172.21.15.148:0/3333927253 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5ba40797f0 con 0x7f5bb40d32f0 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b9806b4d0 msgr2=0x7f5b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b9806b4d0 0x7f5b9806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00010e0 tx=0x7f5bb0007950 comp rx=0 tx=0).stop 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 msgr2=0x7f5bb41f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 0x7f5bb41f37c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4035040 tx=0x7f5ba40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 shutdown_connections 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b9806b4d0 0x7f5b9806d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bb4161710 0x7f5bb40b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bb415dc20 0x7f5bb4156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 --2- 172.21.15.148:0/3333927253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bb40d32f0 0x7f5bb41f37c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 >> 172.21.15.148:0/3333927253 conn(0x7f5bb40d0420 msgr2=0x7f5bb40cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 shutdown_connections 2024-04-25T10:02:33.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.453+0000 7f5bbbd42640 1 -- 172.21.15.148:0/3333927253 wait complete. 2024-04-25T10:02:33.472 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:02:33.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.570+0000 7fafd9922640 1 Processor -- start 2024-04-25T10:02:33.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.570+0000 7fafd9922640 1 -- start start 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd9922640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafd4161710 0x7fafd40d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd9922640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd415e000 con 0x7fafd415dc20 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd9922640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd40d1de0 con 0x7fafd40ced80 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd9922640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd4161c60 con 0x7fafd4161710 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fafd415dc20 0x7fafd40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42926/0 (socket says 172.21.15.148:42926) 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd2ffd640 1 -- 172.21.15.148:0/4260444485 learned_addr learned my addr 172.21.15.148:0/4260444485 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071790104 0 0) 0x7fafd415e000 con 0x7fafd415dc20 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc0003520 con 0x7fafd415dc20 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd37fe640 1 --2- 172.21.15.148:0/4260444485 >> v2:172.21.15.188:3300/0 conn(0x7fafd4161710 0x7fafd40d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270664349 0 0) 0x7fafc0003520 con 0x7fafd415dc20 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 >> v2:172.21.15.188:3300/0 conn(0x7fafd4161710 msgr2=0x7fafd40d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 --2- 172.21.15.148:0/4260444485 >> v2:172.21.15.188:3300/0 conn(0x7fafd4161710 0x7fafd40d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 >> v1:172.21.15.188:6789/0 conn(0x7fafd40ced80 legacy=0x7fafd40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4161f40 con 0x7fafd415dc20 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fafc4001670 con 0x7fafd415dc20 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.571+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 >> v1:172.21.15.148:6789/0 conn(0x7fafd415dc20 legacy=0x7fafd40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd27fc640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd27fc640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd27fc640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafc00051c0 0x7fafc000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4161c60 con 0x7fafc00039b0 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc000c110 con 0x7fafc0004800 2024-04-25T10:02:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc000c310 con 0x7fafc00051c0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafcbfff640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd37fe640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafc00051c0 0x7fafc000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd2ffd640 1 -- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafc00051c0 msgr2=0x7fafc000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafc00051c0 0x7fafc000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd2ffd640 1 -- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc0004800 msgr2=0x7fafc0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd2ffd640 1 -- 172.21.15.148:0/4260444485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0003520 con 0x7fafc00039b0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafd37fe640 1 -- 172.21.15.148:0/4260444485 reap_dead start 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.572+0000 7fafcbfff640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c240 tx=0x7fafc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b465de72c649ec7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007d10 con 0x7fafc00039b0 2024-04-25T10:02:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc4005ad0 con 0x7fafc00039b0 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd27fc640 1 -- 172.21.15.148:0/4260444485 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007070 con 0x7fafc00039b0 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 -- 172.21.15.148:0/4260444485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 msgr2=0x7fafc0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c240 tx=0x7fafc0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 -- 172.21.15.148:0/4260444485 shutdown_connections 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 --2- 172.21.15.148:0/4260444485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 -- 172.21.15.148:0/4260444485 >> 172.21.15.148:0/4260444485 conn(0x7fafd40d0420 msgr2=0x7fafd40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 -- 172.21.15.148:0/4260444485 shutdown_connections 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.573+0000 7fafd9922640 1 -- 172.21.15.148:0/4260444485 wait complete. 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.574+0000 7fafd9922640 1 Processor -- start 2024-04-25T10:02:33.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.574+0000 7fafd9922640 1 -- start start 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd9922640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafd4160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd9922640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd415c5e0 0x7fafd40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd9922640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40cb480 0x7fafd40cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd9922640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd40b5e50 con 0x7fafc00039b0 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd9922640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4162500 con 0x7fafd415c5e0 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd9922640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd40b5fd0 con 0x7fafd40cb480 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafd4160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafd4160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50684/0 (socket says 172.21.15.148:50684) 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd2ffd640 1 -- 172.21.15.148:0/3206708810 learned_addr learned my addr 172.21.15.148:0/3206708810 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafcbfff640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd415c5e0 0x7fafd40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.575+0000 7fafd37fe640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40cb480 0x7fafd40cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd2ffd640 1 -- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40cb480 msgr2=0x7fafd40cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40cb480 0x7fafd40cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd2ffd640 1 -- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd415c5e0 msgr2=0x7fafd40caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd415c5e0 0x7fafd40caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd2ffd640 1 -- 172.21.15.148:0/3206708810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd41f5030 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafcbfff640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd415c5e0 0x7fafd40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd2ffd640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafd4160b50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c240 tx=0x7fafd4162110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4041020 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc4038040 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd415ec20 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007d10 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.576+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd415f1e0 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.577+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd4073a00 con 0x7fafc00039b0 2024-04-25T10:02:33.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.577+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafc4044020 con 0x7fafc00039b0 2024-04-25T10:02:33.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.578+0000 7fafd0ff9640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fafb406b500 0x7fafb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.578+0000 7fafcbfff640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fafb406b500 0x7fafb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.578+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fafc40aa640 con 0x7fafc00039b0 2024-04-25T10:02:33.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.578+0000 7fafcbfff640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fafb406b500 0x7fafb406d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fafbc000c00 tx=0x7fafbc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.582+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafc4005aa0 con 0x7fafc00039b0 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:33.754+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fafd40cb860 con 0x7fafc00039b0 2024-04-25T10:02:34.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.454+0000 7fafd0ff9640 1 -- 172.21.15.148:0/3206708810 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 181+0+0 (secure 0 0 0) 0x7fafc40792e0 con 0x7fafc00039b0 2024-04-25T10:02:34.460 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fafb406b500 msgr2=0x7fafb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fafb406b500 0x7fafb406d9c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fafbc000c00 tx=0x7fafbc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 msgr2=0x7fafd4160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafd4160b50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c240 tx=0x7fafd4162110 comp rx=0 tx=0).stop 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 shutdown_connections 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fafb406b500 0x7fafb406d9c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafd40cb480 0x7fafd40cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafd415c5e0 0x7fafd40caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 --2- 172.21.15.148:0/3206708810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafc00039b0 0x7fafd4160b50 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 >> 172.21.15.148:0/3206708810 conn(0x7fafd40d0420 msgr2=0x7fafd40d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 shutdown_connections 2024-04-25T10:02:34.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.457+0000 7fafd9922640 1 -- 172.21.15.148:0/3206708810 wait complete. 2024-04-25T10:02:34.477 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:02:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.583+0000 7f8a46561640 1 Processor -- start 2024-04-25T10:02:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.583+0000 7f8a46561640 1 -- start start 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a46561640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a400cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a46561640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a4015b980 0x7f8a400cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a46561640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a40072280 con 0x7f8a400cdb20 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a46561640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a400ce4c0 con 0x7f8a4015b980 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a46561640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40071270 con 0x7f8a400cc4f0 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a44afe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a4015b980 0x7f8a400cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a3f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8a400cdb20 0x7f8a4015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42932/0 (socket says 172.21.15.148:42932) 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a3f7fe640 1 -- 172.21.15.148:0/3327364301 learned_addr learned my addr 172.21.15.148:0/3327364301 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.584+0000 7f8a44afe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8a4015b980 0x7f8a400cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50696/0 (socket says 172.21.15.148:50696) 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a44afe640 1 -- 172.21.15.148:0/3327364301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 msgr2=0x7f8a400cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a44afe640 1 --2- 172.21.15.148:0/3327364301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a400cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a44afe640 1 -- 172.21.15.148:0/3327364301 >> v1:172.21.15.148:6789/0 conn(0x7f8a400cdb20 legacy=0x7f8a4015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a44afe640 1 -- 172.21.15.148:0/3327364301 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a400714c0 con 0x7f8a4015b980 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a3effd640 1 -- 172.21.15.148:0/3327364301 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628211055 0 0) 0x7f8a40072280 con 0x7f8a400cdb20 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a44afe640 1 --2- 172.21.15.148:0/3327364301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4015b980 0x7f8a400cdf80 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009d90 tx=0x7f8a3802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf20c2d3032d1b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a3effd640 1 -- 172.21.15.148:0/3327364301 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a38007740 con 0x7f8a4015b980 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a3effd640 1 -- 172.21.15.148:0/3327364301 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a380078e0 con 0x7f8a4015b980 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.585+0000 7f8a3effd640 1 -- 172.21.15.148:0/3327364301 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a38007cb0 con 0x7f8a4015b980 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 -- 172.21.15.148:0/3327364301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4015b980 msgr2=0x7f8a400cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 --2- 172.21.15.148:0/3327364301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4015b980 0x7f8a400cdf80 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009d90 tx=0x7f8a3802a660 comp rx=0 tx=0).stop 2024-04-25T10:02:34.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 -- 172.21.15.148:0/3327364301 shutdown_connections 2024-04-25T10:02:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 --2- 172.21.15.148:0/3327364301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a4015b980 0x7f8a400cdf80 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 --2- 172.21.15.148:0/3327364301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a400cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 -- 172.21.15.148:0/3327364301 >> 172.21.15.148:0/3327364301 conn(0x7f8a4006abe0 msgr2=0x7f8a4006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 -- 172.21.15.148:0/3327364301 shutdown_connections 2024-04-25T10:02:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.586+0000 7f8a46561640 1 -- 172.21.15.148:0/3327364301 wait complete. 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.587+0000 7f8a46561640 1 Processor -- start 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.587+0000 7f8a46561640 1 -- start start 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a46561640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a401462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a46561640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 0x7f8a40159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a46561640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a4015b980 0x7f8a4015c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a46561640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40071a80 con 0x7f8a400cdb20 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a46561640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4015bd90 con 0x7f8a4015b980 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a46561640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40071d60 con 0x7f8a400cc4f0 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 0x7f8a40159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 0x7f8a40159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50700/0 (socket says 172.21.15.148:50700) 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 -- 172.21.15.148:0/871229782 learned_addr learned my addr 172.21.15.148:0/871229782 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a44afe640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a4015b980 0x7f8a4015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3ffff640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a401462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 -- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 msgr2=0x7f8a401462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a401462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 -- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a4015b980 msgr2=0x7f8a4015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a4015b980 0x7f8a4015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a3f7fe640 1 -- 172.21.15.148:0/871229782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4015a3e0 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.588+0000 7f8a44afe640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a4015b980 0x7f8a4015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.589+0000 7f8a3f7fe640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 0x7f8a40159ea0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400c4d0 tx=0x7f8a3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.589+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34018070 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.589+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a34007500 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.589+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a40158e00 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.589+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34004a20 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.589+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a401593c0 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.590+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a40073a00 con 0x7f8a400cdb20 2024-04-25T10:02:34.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.590+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a3401b070 con 0x7f8a400cdb20 2024-04-25T10:02:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.591+0000 7f8a3cff9640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a2c06b500 0x7f8a2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.591+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f8a340894f0 con 0x7f8a400cdb20 2024-04-25T10:02:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.591+0000 7f8a3ffff640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a2c06b500 0x7f8a2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.591+0000 7f8a3ffff640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a2c06b500 0x7f8a2c06d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000b810 tx=0x7f8a30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.595+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a34058340 con 0x7f8a400cdb20 2024-04-25T10:02:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:34.766+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8a40152060 con 0x7f8a400cdb20 2024-04-25T10:02:35.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.455+0000 7f8a3cff9640 1 -- 172.21.15.148:0/871229782 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8a34057c40 con 0x7f8a400cdb20 2024-04-25T10:02:35.461 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.458+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a2c06b500 msgr2=0x7f8a2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.458+0000 7f8a46561640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a2c06b500 0x7f8a2c06d9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000b810 tx=0x7f8a30008040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.458+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 msgr2=0x7f8a40159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.458+0000 7f8a46561640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 0x7f8a40159ea0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400c4d0 tx=0x7f8a3400c990 comp rx=0 tx=0).stop 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 shutdown_connections 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8a2c06b500 0x7f8a2c06d9c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a4015b980 0x7f8a4015c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a400cdb20 0x7f8a40159ea0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 --2- 172.21.15.148:0/871229782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a400cc4f0 0x7f8a401462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 >> 172.21.15.148:0/871229782 conn(0x7f8a4006abe0 msgr2=0x7f8a4015b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 shutdown_connections 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.459+0000 7f8a46561640 1 -- 172.21.15.148:0/871229782 wait complete. 2024-04-25T10:02:35.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:02:35.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.575+0000 7fe77b2b3640 1 Processor -- start 2024-04-25T10:02:35.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.575+0000 7fe77b2b3640 1 -- start start 2024-04-25T10:02:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe77b2b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe77415b7d0 0x7fe7740cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe77b2b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe77b2b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77406cf50 con 0x7fe77406c2d0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe77b2b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7740b5fc0 con 0x7fe77415b7d0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe77b2b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740ce530 con 0x7fe7740b5620 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe779850640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe77406c2d0 0x7fe77406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42944/0 (socket says 172.21.15.148:42944) 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.576+0000 7fe779850640 1 -- 172.21.15.148:0/1060969566 learned_addr learned my addr 172.21.15.148:0/1060969566 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883210619 0 0) 0x7fe77406cf50 con 0x7fe77406c2d0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe760003520 con 0x7fe77406c2d0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77904f640 1 --2- 172.21.15.148:0/1060969566 >> v2:172.21.15.188:3300/0 conn(0x7fe77415b7d0 0x7fe7740cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901398604 0 0) 0x7fe760003520 con 0x7fe77406c2d0 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 msgr2=0x7fe7740b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 >> v2:172.21.15.188:3300/0 conn(0x7fe77415b7d0 msgr2=0x7fe7740cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 --2- 172.21.15.148:0/1060969566 >> v2:172.21.15.188:3300/0 conn(0x7fe77415b7d0 0x7fe7740cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740ce810 con 0x7fe77406c2d0 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe76c001670 con 0x7fe77406c2d0 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7740b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 >> v1:172.21.15.148:6789/0 conn(0x7fe77406c2d0 legacy=0x7fe77406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7600039b0 0x7fe760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004800 0x7fe760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740ce530 con 0x7fe760004800 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76000b9c0 con 0x7fe7600039b0 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76000bb70 con 0x7fe7740b5620 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004800 0x7fe760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77904f640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7600039b0 0x7fe760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe779850640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 -- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 msgr2=0x7fe7600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7740b5620 0x7fe7600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 -- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7600039b0 msgr2=0x7fe760003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7600039b0 0x7fe760003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.577+0000 7fe77884e640 1 -- 172.21.15.148:0/1060969566 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760003520 con 0x7fe760004800 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.578+0000 7fe77884e640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004800 0x7fe760004c80 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe7640098a0 tx=0x7fe760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fd48718b4a6629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.578+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76400f040 con 0x7fe760004800 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.578+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76402acc0 con 0x7fe760004800 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.578+0000 7fe76bfff640 1 -- 172.21.15.148:0/1060969566 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764004250 con 0x7fe760004800 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe77b2b3640 1 -- 172.21.15.148:0/1060969566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004800 msgr2=0x7fe760004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/1060969566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760004800 0x7fe760004c80 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe7640098a0 tx=0x7fe760005830 comp rx=0 tx=0).stop 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe779850640 1 -- 172.21.15.148:0/1060969566 reap_dead start 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe77b2b3640 1 -- 172.21.15.148:0/1060969566 shutdown_connections 2024-04-25T10:02:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe77b2b3640 1 -- 172.21.15.148:0/1060969566 >> 172.21.15.148:0/1060969566 conn(0x7fe7740d0800 msgr2=0x7fe7740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe77b2b3640 1 -- 172.21.15.148:0/1060969566 shutdown_connections 2024-04-25T10:02:35.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.579+0000 7fe77b2b3640 1 -- 172.21.15.148:0/1060969566 wait complete. 2024-04-25T10:02:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.580+0000 7fe77b2b3640 1 Processor -- start 2024-04-25T10:02:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.580+0000 7fe77b2b3640 1 -- start start 2024-04-25T10:02:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe760004800 0x7fe7740ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 0x7fe7741e7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741e7c90 0x7fe77415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740cf550 con 0x7fe77414b7e0 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77406d230 con 0x7fe7741e7c90 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740cf6d0 con 0x7fe760004800 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 0x7fe7741e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 0x7fe7741e7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50716/0 (socket says 172.21.15.148:50716) 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 -- 172.21.15.148:0/2578057025 learned_addr learned my addr 172.21.15.148:0/2578057025 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe779850640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741e7c90 0x7fe77415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77904f640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe760004800 0x7fe7740ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 -- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe760004800 msgr2=0x7fe7740ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe760004800 0x7fe7740ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 -- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741e7c90 msgr2=0x7fe77415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741e7c90 0x7fe77415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 -- 172.21.15.148:0/2578057025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740aa690 con 0x7fe77414b7e0 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77884e640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 0x7fe7741e7750 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe76402a7b0 tx=0x7fe764004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764041070 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76400f000 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7740caf90 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.581+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764003c50 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.582+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7740cb550 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.583+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe774073a00 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.583+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe764044070 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.583+0000 7fe769ffb640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75806b500 0x7fe75806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.583+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe7640aa680 con 0x7fe77414b7e0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.583+0000 7fe77904f640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75806b500 0x7fe75806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.584+0000 7fe77904f640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75806b500 0x7fe75806d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe770007330 tx=0x7fe77000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.587+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7640795e0 con 0x7fe77414b7e0 2024-04-25T10:02:35.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.757+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7741e8070 con 0x7fe77414b7e0 2024-04-25T10:02:35.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.760+0000 7fe769ffb640 1 -- 172.21.15.148:0/2578057025 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe764078ee0 con 0x7fe77414b7e0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75806b500 msgr2=0x7fe75806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75806b500 0x7fe75806d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe770007330 tx=0x7fe77000b040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 msgr2=0x7fe7741e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 0x7fe7741e7750 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe76402a7b0 tx=0x7fe764004730 comp rx=0 tx=0).stop 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 shutdown_connections 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe75806b500 0x7fe75806d9c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7741e7c90 0x7fe77415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77414b7e0 0x7fe7741e7750 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 --2- 172.21.15.148:0/2578057025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe760004800 0x7fe7740ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 >> 172.21.15.148:0/2578057025 conn(0x7fe7740d0800 msgr2=0x7fe77415ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.762+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 shutdown_connections 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.763+0000 7fe77b2b3640 1 -- 172.21.15.148:0/2578057025 wait complete. 2024-04-25T10:02:35.780 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:02:35.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.872+0000 7f0f3b75c640 1 Processor -- start 2024-04-25T10:02:35.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.872+0000 7f0f3b75c640 1 -- start start 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f3b75c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0f340cc8e0 0x7f0f341625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f3b75c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f340d1810 0x7f0f340d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f3b75c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f340cce50 con 0x7f0f3415dc90 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f3b75c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340d21c0 con 0x7f0f340cc8e0 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f3b75c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34161af0 con 0x7f0f340d1810 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f38cf7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0f340cc8e0 0x7f0f341625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f38cf7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0f340cc8e0 0x7f0f341625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50722/0 (socket says 172.21.15.148:50722) 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f38cf7640 1 -- 172.21.15.148:0/2966157538 learned_addr learned my addr 172.21.15.148:0/2966157538 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f39cf9640 1 --2- 172.21.15.148:0/2966157538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f340d1810 0x7f0f340d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f38cf7640 1 -- 172.21.15.148:0/2966157538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f340d1810 msgr2=0x7f0f340d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f38cf7640 1 --2- 172.21.15.148:0/2966157538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f340d1810 0x7f0f340d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.873+0000 7f0f38cf7640 1 -- 172.21.15.148:0/2966157538 >> v1:172.21.15.188:6789/0 conn(0x7f0f3415dc90 legacy=0x7f0f3415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.874+0000 7f0f38cf7640 1 -- 172.21.15.148:0/2966157538 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f34161dd0 con 0x7f0f340cc8e0 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.874+0000 7f0f38cf7640 1 --2- 172.21.15.148:0/2966157538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f341625d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009870 tx=0x7f0f2402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aac0406dbced0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.874+0000 7f0f2bfff640 1 -- 172.21.15.148:0/2966157538 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24037070 con 0x7f0f340cc8e0 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.874+0000 7f0f2bfff640 1 -- 172.21.15.148:0/2966157538 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f24032440 con 0x7f0f340cc8e0 2024-04-25T10:02:35.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.874+0000 7f0f2bfff640 1 -- 172.21.15.148:0/2966157538 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24032810 con 0x7f0f340cc8e0 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 -- 172.21.15.148:0/2966157538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 msgr2=0x7f0f341625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/2966157538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f341625d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009870 tx=0x7f0f2402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 -- 172.21.15.148:0/2966157538 shutdown_connections 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/2966157538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f341625d0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/2966157538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f340d1810 0x7f0f340d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 -- 172.21.15.148:0/2966157538 >> 172.21.15.148:0/2966157538 conn(0x7f0f3406f590 msgr2=0x7f0f3406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 -- 172.21.15.148:0/2966157538 shutdown_connections 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.875+0000 7f0f3b75c640 1 -- 172.21.15.148:0/2966157538 wait complete. 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.876+0000 7f0f3b75c640 1 Processor -- start 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.876+0000 7f0f3b75c640 1 -- start start 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f3b75c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f340c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f3b75c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f340d1810 0x7f0f34147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f3b75c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f3415dc90 0x7f0f34145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f3b75c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415e070 con 0x7f0f340cc8e0 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f3b75c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340cd130 con 0x7f0f340d1810 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f3b75c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415e1f0 con 0x7f0f3415dc90 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f340c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f340c4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50734/0 (socket says 172.21.15.148:50734) 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 -- 172.21.15.148:0/1343601688 learned_addr learned my addr 172.21.15.148:0/1343601688 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f38cf7640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f340d1810 0x7f0f34147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f39cf9640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f3415dc90 0x7f0f34145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 -- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f3415dc90 msgr2=0x7f0f34145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f3415dc90 0x7f0f34145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 -- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f340d1810 msgr2=0x7f0f34147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f340d1810 0x7f0f34147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 -- 172.21.15.148:0/1343601688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f34148510 con 0x7f0f340cc8e0 2024-04-25T10:02:35.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f394f8640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f340c4150 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c400 tx=0x7f0f3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.877+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f300052d0 con 0x7f0f340cc8e0 2024-04-25T10:02:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.878+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f30005490 con 0x7f0f340cc8e0 2024-04-25T10:02:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.878+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f341487f0 con 0x7f0f340cc8e0 2024-04-25T10:02:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.878+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30007260 con 0x7f0f340cc8e0 2024-04-25T10:02:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.878+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f340c7800 con 0x7f0f340cc8e0 2024-04-25T10:02:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.879+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f34073990 con 0x7f0f340cc8e0 2024-04-25T10:02:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.879+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f30007400 con 0x7f0f340cc8e0 2024-04-25T10:02:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.879+0000 7f0f29ffb640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f1806b4d0 0x7f0f1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.880+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0f30089530 con 0x7f0f340cc8e0 2024-04-25T10:02:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.880+0000 7f0f38cf7640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f1806b4d0 0x7f0f1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.880+0000 7f0f38cf7640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f1806b4d0 0x7f0f1806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0f24034000 tx=0x7f0f2402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:35.884+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f30057ed0 con 0x7f0f340cc8e0 2024-04-25T10:02:36.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.173+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f0f340cf6b0 con 0x7f0f340cc8e0 2024-04-25T10:02:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.457+0000 7f0f29ffb640 1 -- 172.21.15.148:0/1343601688 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v406) v1 ==== 166+0+0 (secure 0 0 0) 0x7f0f3005bf70 con 0x7f0f340cc8e0 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f1806b4d0 msgr2=0x7f0f1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f1806b4d0 0x7f0f1806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0f24034000 tx=0x7f0f2402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 msgr2=0x7f0f340c4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f340c4150 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c400 tx=0x7f0f3000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 shutdown_connections 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0f1806b4d0 0x7f0f1806d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f3415dc90 0x7f0f34145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f340d1810 0x7f0f34147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 --2- 172.21.15.148:0/1343601688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f340cc8e0 0x7f0f340c4150 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 >> 172.21.15.148:0/1343601688 conn(0x7f0f3406f590 msgr2=0x7f0f340c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 shutdown_connections 2024-04-25T10:02:36.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.459+0000 7f0f3b75c640 1 -- 172.21.15.148:0/1343601688 wait complete. 2024-04-25T10:02:36.477 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:02:36.477 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:02:36.477 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:36.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.569+0000 7f0abcb29640 1 Processor -- start 2024-04-25T10:02:36.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.569+0000 7f0abcb29640 1 -- start start 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0abcb29640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ab8158220 0x7f0ab815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0abcb29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80afba0 0x7f0ab8161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0abcb29640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab815f180 con 0x7f0ab80af720 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0abcb29640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80b3670 con 0x7f0ab8158220 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0abcb29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8070a10 con 0x7f0ab80afba0 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0ab659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ab8158220 0x7f0ab815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0aaffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0ab80af720 0x7f0ab81617b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42958/0 (socket says 172.21.15.148:42958) 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0ab659c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ab8158220 0x7f0ab815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50748/0 (socket says 172.21.15.148:50748) 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.570+0000 7f0aaffff640 1 -- 172.21.15.148:0/1674774627 learned_addr learned my addr 172.21.15.148:0/1674774627 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab6d9d640 1 --2- 172.21.15.148:0/1674774627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80afba0 0x7f0ab8161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab659c640 1 -- 172.21.15.148:0/1674774627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80afba0 msgr2=0x7f0ab8161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab659c640 1 --2- 172.21.15.148:0/1674774627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80afba0 0x7f0ab8161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab659c640 1 -- 172.21.15.148:0/1674774627 >> v1:172.21.15.148:6789/0 conn(0x7f0ab80af720 legacy=0x7f0ab81617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab5d9b640 1 -- 172.21.15.148:0/1674774627 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463851352 0 0) 0x7f0aa0001110 con 0x7f0ab80af720 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab659c640 1 -- 172.21.15.148:0/1674774627 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab806fe00 con 0x7f0ab8158220 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.571+0000 7f0ab659c640 1 --2- 172.21.15.148:0/1674774627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab815ec40 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097a0 tx=0x7f0aa802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138c511ff41328f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0ab5d9b640 1 -- 172.21.15.148:0/1674774627 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8037070 con 0x7f0ab8158220 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0ab5d9b640 1 -- 172.21.15.148:0/1674774627 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8032440 con 0x7f0ab8158220 2024-04-25T10:02:36.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0ab5d9b640 1 -- 172.21.15.148:0/1674774627 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8032810 con 0x7f0ab8158220 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 -- 172.21.15.148:0/1674774627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 msgr2=0x7f0ab815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 --2- 172.21.15.148:0/1674774627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab815ec40 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097a0 tx=0x7f0aa802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 -- 172.21.15.148:0/1674774627 shutdown_connections 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 --2- 172.21.15.148:0/1674774627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80afba0 0x7f0ab8161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 --2- 172.21.15.148:0/1674774627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab815ec40 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 -- 172.21.15.148:0/1674774627 >> 172.21.15.148:0/1674774627 conn(0x7f0ab80b2370 msgr2=0x7f0ab80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 -- 172.21.15.148:0/1674774627 shutdown_connections 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.572+0000 7f0abcb29640 1 -- 172.21.15.148:0/1674774627 wait complete. 2024-04-25T10:02:36.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.573+0000 7f0abcb29640 1 Processor -- start 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 -- start start 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80af720 0x7f0ab815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ab80afba0 0x7f0ab8160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80706a0 con 0x7f0ab8158220 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80703c0 con 0x7f0ab80afba0 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0abcb29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80b0010 con 0x7f0ab80af720 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0ab6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0ab6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab80cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50762/0 (socket says 172.21.15.148:50762) 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0ab6d9d640 1 -- 172.21.15.148:0/2119810705 learned_addr learned my addr 172.21.15.148:0/2119810705 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0aaffff640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ab80afba0 0x7f0ab8160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.574+0000 7f0ab659c640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80af720 0x7f0ab815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0ab6d9d640 1 -- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80af720 msgr2=0x7f0ab815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0ab6d9d640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80af720 0x7f0ab815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0ab6d9d640 1 -- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ab80afba0 msgr2=0x7f0ab8160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0ab6d9d640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ab80afba0 0x7f0ab8160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0ab6d9d640 1 -- 172.21.15.148:0/2119810705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8159b40 con 0x7f0ab8158220 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0ab6d9d640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab80cc1a0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007e40 tx=0x7f0ab000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0014620 con 0x7f0ab8158220 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0019070 con 0x7f0ab8158220 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0014a20 con 0x7f0ab8158220 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab8160e50 con 0x7f0ab8158220 2024-04-25T10:02:36.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.575+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab815a0a0 con 0x7f0ab8158220 2024-04-25T10:02:36.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.576+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab8073990 con 0x7f0ab8158220 2024-04-25T10:02:36.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.576+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ab0014bc0 con 0x7f0ab8158220 2024-04-25T10:02:36.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.577+0000 7f0aaeffd640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a9806b4d0 0x7f0a9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.577+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0ab008a9d0 con 0x7f0ab8158220 2024-04-25T10:02:36.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.577+0000 7f0ab659c640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a9806b4d0 0x7f0a9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.577+0000 7f0ab659c640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a9806b4d0 0x7f0a9806d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8034000 tx=0x7f0aa802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.581+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ab8160e50 con 0x7f0ab8158220 2024-04-25T10:02:36.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.751+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab80cbf80 con 0x7f0ab8158220 2024-04-25T10:02:36.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.752+0000 7f0aaeffd640 1 -- 172.21.15.148:0/2119810705 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+13351 (secure 0 0 0) 0x7f0ab8159b40 con 0x7f0ab8158220 2024-04-25T10:02:36.757 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:36.758 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":124,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:02:35.455826+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/609722939":"2024-04-26T10:02:32.449165+0000","172.21.15.188:6833/609722939":"2024-04-26T10:02:32.449165+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:36.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a9806b4d0 msgr2=0x7f0a9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a9806b4d0 0x7f0a9806d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8034000 tx=0x7f0aa802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 msgr2=0x7f0ab80cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab80cc1a0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007e40 tx=0x7f0ab000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 shutdown_connections 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0a9806b4d0 0x7f0a9806d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ab8158220 0x7f0ab80cc1a0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ab80afba0 0x7f0ab8160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 --2- 172.21.15.148:0/2119810705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ab80af720 0x7f0ab815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 >> 172.21.15.148:0/2119810705 conn(0x7f0ab80b2370 msgr2=0x7f0ab80b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 shutdown_connections 2024-04-25T10:02:36.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.754+0000 7f0abcb29640 1 -- 172.21.15.148:0/2119810705 wait complete. 2024-04-25T10:02:36.771 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:36.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:36.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 Processor -- start 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 -- start start 2024-04-25T10:02:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efc2415dc20 0x7efc240ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc24161710 0x7efc240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc2406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2415e000 con 0x7efc24161710 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc240d1de0 con 0x7efc2415dc20 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.866+0000 7efc28fa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc24161c60 con 0x7efc240b9330 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc24161710 0x7efc240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc24161710 0x7efc240cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50766/0 (socket says 172.21.15.148:50766) 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 -- 172.21.15.148:0/2469841022 learned_addr learned my addr 172.21.15.148:0/2469841022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 -- 172.21.15.148:0/2469841022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 msgr2=0x7efc2406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc2259c640 1 --2- 172.21.15.148:0/2469841022 >> v2:172.21.15.188:3300/0 conn(0x7efc2415dc20 0x7efc240ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc22d9d640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc2406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc2406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 -- 172.21.15.148:0/2469841022 >> v2:172.21.15.188:3300/0 conn(0x7efc2415dc20 msgr2=0x7efc240ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 --2- 172.21.15.148:0/2469841022 >> v2:172.21.15.188:3300/0 conn(0x7efc2415dc20 0x7efc240ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc1bfff640 1 -- 172.21.15.148:0/2469841022 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24161f40 con 0x7efc24161710 2024-04-25T10:02:36.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc2259c640 1 --2- 172.21.15.148:0/2469841022 >> v2:172.21.15.188:3300/0 conn(0x7efc2415dc20 0x7efc240ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.867+0000 7efc22d9d640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc2406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc1bfff640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240cebb0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7efc240d1520 tx=0x7efc0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d5b130f373b16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc21d9b640 1 -- 172.21.15.148:0/2469841022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c037070 con 0x7efc24161710 2024-04-25T10:02:36.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc21d9b640 1 -- 172.21.15.148:0/2469841022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0c032440 con 0x7efc24161710 2024-04-25T10:02:36.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc21d9b640 1 -- 172.21.15.148:0/2469841022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c032810 con 0x7efc24161710 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 -- 172.21.15.148:0/2469841022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 msgr2=0x7efc240cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240cebb0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7efc240d1520 tx=0x7efc0c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 -- 172.21.15.148:0/2469841022 shutdown_connections 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc2406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 --2- 172.21.15.148:0/2469841022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240cebb0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 --2- 172.21.15.148:0/2469841022 >> v2:172.21.15.188:3300/0 conn(0x7efc2415dc20 0x7efc240ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 -- 172.21.15.148:0/2469841022 >> 172.21.15.148:0/2469841022 conn(0x7efc240d0420 msgr2=0x7efc240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 -- 172.21.15.148:0/2469841022 shutdown_connections 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.868+0000 7efc28fa9640 1 -- 172.21.15.148:0/2469841022 wait complete. 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.870+0000 7efc28fa9640 1 Processor -- start 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.870+0000 7efc28fa9640 1 -- start start 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.870+0000 7efc28fa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc241519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415dc20 0x7efc240cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240b5e50 con 0x7efc24161710 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24162500 con 0x7efc2415dc20 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc240b5fd0 con 0x7efc240b9330 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240b8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50768/0 (socket says 172.21.15.148:50768) 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 -- 172.21.15.148:0/1332182121 learned_addr learned my addr 172.21.15.148:0/1332182121 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 -- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 msgr2=0x7efc241519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc241519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 -- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415dc20 msgr2=0x7efc240cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc1bfff640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415dc20 0x7efc240cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415dc20 0x7efc240cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 -- 172.21.15.148:0/1332182121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24152490 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc22d9d640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240b8270 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00cce0 tx=0x7efc1c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00f680 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00f820 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc24151f10 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00fc10 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.871+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc24160e70 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.872+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24073a00 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.873+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc1c0213f0 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.873+0000 7efc1affd640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0406b4d0 0x7efc0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.873+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efc1c08ac80 con 0x7efc24161710 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.873+0000 7efc2259c640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0406b4d0 0x7efc0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.874+0000 7efc2259c640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0406b4d0 0x7efc0406d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:36.877+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc24152490 con 0x7efc24161710 2024-04-25T10:02:37.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.047+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc240ce0d0 con 0x7efc24161710 2024-04-25T10:02:37.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.047+0000 7efc1affd640 1 -- 172.21.15.148:0/1332182121 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc1c014030 con 0x7efc24161710 2024-04-25T10:02:37.053 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0406b4d0 msgr2=0x7efc0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0406b4d0 0x7efc0406d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14008040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 msgr2=0x7efc240b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240b8270 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00cce0 tx=0x7efc1c00a650 comp rx=0 tx=0).stop 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 shutdown_connections 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc0406b4d0 0x7efc0406d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc24161710 0x7efc240b8270 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc2415dc20 0x7efc240cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 --2- 172.21.15.148:0/1332182121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc240b9330 0x7efc241519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 >> 172.21.15.148:0/1332182121 conn(0x7efc240d0420 msgr2=0x7efc240d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 shutdown_connections 2024-04-25T10:02:37.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.049+0000 7efc28fa9640 1 -- 172.21.15.148:0/1332182121 wait complete. 2024-04-25T10:02:37.066 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:02:37.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.158+0000 7f36476bc640 1 Processor -- start 2024-04-25T10:02:37.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.158+0000 7f36476bc640 1 -- start start 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f36476bc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f364015db70 0x7f3640162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f36476bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 0x7f36400d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f36476bc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3640162690 con 0x7f36400b6cb0 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f36476bc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36400ceee0 con 0x7f364015db70 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f36476bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36400d1d10 con 0x7f36400b5c30 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f3644c57640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f36400b6cb0 0x7f36400b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42968/0 (socket says 172.21.15.148:42968) 2024-04-25T10:02:37.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f3644c57640 1 -- 172.21.15.148:0/2219999748 learned_addr learned my addr 172.21.15.148:0/2219999748 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962394525 0 0) 0x7f3640162690 con 0x7f36400b6cb0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c003520 con 0x7f36400b6cb0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.159+0000 7f3645c59640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 0x7f36400d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645458640 1 --2- 172.21.15.148:0/2219999748 >> v2:172.21.15.188:3300/0 conn(0x7f364015db70 0x7f3640162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 424777080 0 0) 0x7f362c003520 con 0x7f36400b6cb0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 msgr2=0x7f36400d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 0x7f36400d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 >> v2:172.21.15.188:3300/0 conn(0x7f364015db70 msgr2=0x7f3640162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 --2- 172.21.15.148:0/2219999748 >> v2:172.21.15.188:3300/0 conn(0x7f364015db70 0x7f3640162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36400d1ff0 con 0x7f36400b6cb0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f3630001670 con 0x7f36400b6cb0 2024-04-25T10:02:37.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 >> v1:172.21.15.148:6789/0 conn(0x7f36400b6cb0 legacy=0x7f36400b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c0039b0 0x7f362c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 0x7f362c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f362c004d40 0x7f362c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36400ceee0 con 0x7f362c004d40 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36400d1d10 con 0x7f362c0039b0 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c00b9c0 con 0x7f36400b5c30 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645c59640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f362c004d40 0x7f362c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3644c57640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 0x7f362c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645458640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c0039b0 0x7f362c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645c59640 1 -- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 msgr2=0x7f362c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645c59640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36400b5c30 0x7f362c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645c59640 1 -- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c0039b0 msgr2=0x7f362c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645c59640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f362c0039b0 0x7f362c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.160+0000 7f3645c59640 1 -- 172.21.15.148:0/2219999748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c003520 con 0x7f362c004d40 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.161+0000 7f3645c59640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f362c004d40 0x7f362c0051c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f3638009a90 tx=0x7f362c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e443738c825244 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.161+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638036070 con 0x7f362c004d40 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.161+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638031400 con 0x7f362c004d40 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.161+0000 7f3637fff640 1 -- 172.21.15.148:0/2219999748 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638009690 con 0x7f362c004d40 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f36476bc640 1 -- 172.21.15.148:0/2219999748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f362c004d40 msgr2=0x7f362c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f36476bc640 1 --2- 172.21.15.148:0/2219999748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f362c004d40 0x7f362c0051c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f3638009a90 tx=0x7f362c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f3645c59640 1 -- 172.21.15.148:0/2219999748 reap_dead start 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f36476bc640 1 -- 172.21.15.148:0/2219999748 shutdown_connections 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f36476bc640 1 -- 172.21.15.148:0/2219999748 >> 172.21.15.148:0/2219999748 conn(0x7f36400d0520 msgr2=0x7f36400d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f36476bc640 1 -- 172.21.15.148:0/2219999748 shutdown_connections 2024-04-25T10:02:37.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.162+0000 7f36476bc640 1 -- 172.21.15.148:0/2219999748 wait complete. 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.163+0000 7f36476bc640 1 Processor -- start 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.163+0000 7f36476bc640 1 -- start start 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362c004d40 0x7f36400c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 0x7f36400cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36400cbe70 0x7f36400cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36400ce530 con 0x7f36400c9760 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364015df50 con 0x7f36400cbe70 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36400cf1c0 con 0x7f362c004d40 2024-04-25T10:02:37.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 0x7f36400cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 0x7f36400cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50782/0 (socket says 172.21.15.148:50782) 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 -- 172.21.15.148:0/605500325 learned_addr learned my addr 172.21.15.148:0/605500325 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3645458640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362c004d40 0x7f36400c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3645c59640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36400cbe70 0x7f36400cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 -- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362c004d40 msgr2=0x7f36400c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362c004d40 0x7f36400c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 -- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36400cbe70 msgr2=0x7f36400cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36400cbe70 0x7f36400cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 -- 172.21.15.148:0/605500325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364015e3e0 con 0x7f36400c9760 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3645458640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362c004d40 0x7f36400c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3645c59640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36400cbe70 0x7f36400cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3644c57640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 0x7f36400cb930 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f3630007340 tx=0x7f3630007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363000f980 con 0x7f36400c9760 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3630018070 con 0x7f36400c9760 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3640161470 con 0x7f36400c9760 2024-04-25T10:02:37.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.164+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363000fd60 con 0x7f36400c9760 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.165+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3640154660 con 0x7f36400c9760 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.166+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36400ce6b0 con 0x7f36400c9760 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.166+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f363001d460 con 0x7f36400c9760 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.166+0000 7f3635ffb640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f362406b500 0x7f362406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.166+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f363008a280 con 0x7f36400c9760 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.166+0000 7f3645458640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f362406b500 0x7f362406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.167+0000 7f3645458640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f362406b500 0x7f362406d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f363c007440 tx=0x7f363c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.170+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36400ce6b0 con 0x7f36400c9760 2024-04-25T10:02:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.403+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f3640073990 con 0x7f36400c9760 2024-04-25T10:02:37.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.465+0000 7f3635ffb640 1 -- 172.21.15.148:0/605500325 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v407) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3630059270 con 0x7f36400c9760 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f362406b500 msgr2=0x7f362406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f362406b500 0x7f362406d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f363c007440 tx=0x7f363c00b040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 msgr2=0x7f36400cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 0x7f36400cb930 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f3630007340 tx=0x7f3630007800 comp rx=0 tx=0).stop 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 shutdown_connections 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f362406b500 0x7f362406d9c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36400cbe70 0x7f36400cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36400c9760 0x7f36400cb930 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 --2- 172.21.15.148:0/605500325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f362c004d40 0x7f36400c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 >> 172.21.15.148:0/605500325 conn(0x7f36400d0520 msgr2=0x7f36400d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 shutdown_connections 2024-04-25T10:02:37.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.467+0000 7f36476bc640 1 -- 172.21.15.148:0/605500325 wait complete. 2024-04-25T10:02:37.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:02:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.578+0000 7f5e18e38640 1 Processor -- start 2024-04-25T10:02:37.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.578+0000 7f5e18e38640 1 -- start start 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e18e38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140ced80 0x7f5e140d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e18e38640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5e14161710 0x7f5e140d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e18e38640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1415e000 con 0x7f5e1415dc20 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e18e38640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e140d1de0 con 0x7f5e14161710 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e18e38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14161c60 con 0x7f5e140ced80 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e12d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5e14161710 0x7f5e140d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e12d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5e14161710 0x7f5e140d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50796/0 (socket says 172.21.15.148:50796) 2024-04-25T10:02:37.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.579+0000 7f5e12d9d640 1 -- 172.21.15.148:0/1267870029 learned_addr learned my addr 172.21.15.148:0/1267870029 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e11d9b640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140ced80 0x7f5e140d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e12d9d640 1 -- 172.21.15.148:0/1267870029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140ced80 msgr2=0x7f5e140d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e12d9d640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140ced80 0x7f5e140d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e12d9d640 1 -- 172.21.15.148:0/1267870029 >> v1:172.21.15.188:6789/0 conn(0x7f5e1415dc20 legacy=0x7f5e140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e12d9d640 1 -- 172.21.15.148:0/1267870029 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e14161f40 con 0x7f5e14161710 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e11d9b640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140ced80 0x7f5e140d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e12d9d640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e14161710 0x7f5e140d2820 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009870 tx=0x7f5e0c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d5ae68f004d390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.580+0000 7f5e1159a640 1 -- 172.21.15.148:0/1267870029 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c037070 con 0x7f5e14161710 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e1159a640 1 -- 172.21.15.148:0/1267870029 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0c032440 con 0x7f5e14161710 2024-04-25T10:02:37.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e1159a640 1 -- 172.21.15.148:0/1267870029 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c032810 con 0x7f5e14161710 2024-04-25T10:02:37.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 -- 172.21.15.148:0/1267870029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e14161710 msgr2=0x7f5e140d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e14161710 0x7f5e140d2820 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c009870 tx=0x7f5e0c02a210 comp rx=0 tx=0).stop 2024-04-25T10:02:37.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 -- 172.21.15.148:0/1267870029 shutdown_connections 2024-04-25T10:02:37.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e14161710 0x7f5e140d2820 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 --2- 172.21.15.148:0/1267870029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140ced80 0x7f5e140d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 -- 172.21.15.148:0/1267870029 >> 172.21.15.148:0/1267870029 conn(0x7f5e140d0420 msgr2=0x7f5e140d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 -- 172.21.15.148:0/1267870029 shutdown_connections 2024-04-25T10:02:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.581+0000 7f5e18e38640 1 -- 172.21.15.148:0/1267870029 wait complete. 2024-04-25T10:02:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.582+0000 7f5e18e38640 1 Processor -- start 2024-04-25T10:02:37.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.582+0000 7f5e18e38640 1 -- start start 2024-04-25T10:02:37.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e18e38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e140ced80 0x7f5e1414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e18e38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 0x7f5e140aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e18e38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e14161710 0x7f5e141f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e18e38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e140b5e50 con 0x7f5e1415dc20 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e18e38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14162500 con 0x7f5e140ced80 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e18e38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e140b5fd0 con 0x7f5e14161710 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e11d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 0x7f5e140aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e11d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 0x7f5e140aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50808/0 (socket says 172.21.15.148:50808) 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e11d9b640 1 -- 172.21.15.148:0/542798997 learned_addr learned my addr 172.21.15.148:0/542798997 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.583+0000 7f5e1259c640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e140ced80 0x7f5e1414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e12d9d640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e14161710 0x7f5e141f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e11d9b640 1 -- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e14161710 msgr2=0x7f5e141f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e11d9b640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e14161710 0x7f5e141f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e11d9b640 1 -- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e140ced80 msgr2=0x7f5e1414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e11d9b640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e140ced80 0x7f5e1414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e11d9b640 1 -- 172.21.15.148:0/542798997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e140cb450 con 0x7f5e1415dc20 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e11d9b640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 0x7f5e140aa380 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c7f0 tx=0x7f5e0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08015070 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e08007820 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e140cb730 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08007bd0 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.584+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e140aa8c0 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.585+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e14073a00 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.585+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e08018070 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.586+0000 7f5e02ffd640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5df406b4d0 0x7f5df406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.586+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5e0808a2a0 con 0x7f5e1415dc20 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.586+0000 7f5e1259c640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5df406b4d0 0x7f5df406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.586+0000 7f5e1259c640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5df406b4d0 0x7f5df406d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009f20 tx=0x7f5e040094d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.590+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e080591f0 con 0x7f5e1415dc20 2024-04-25T10:02:37.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.760+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5e140d16b0 con 0x7f5e1415dc20 2024-04-25T10:02:37.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.764+0000 7f5e02ffd640 1 -- 172.21.15.148:0/542798997 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e08058af0 con 0x7f5e1415dc20 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5df406b4d0 msgr2=0x7f5df406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5df406b4d0 0x7f5df406d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009f20 tx=0x7f5e040094d0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 msgr2=0x7f5e140aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 0x7f5e140aa380 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800c7f0 tx=0x7f5e0800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 shutdown_connections 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5df406b4d0 0x7f5df406d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e14161710 0x7f5e141f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e1415dc20 0x7f5e140aa380 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 --2- 172.21.15.148:0/542798997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e140ced80 0x7f5e1414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 >> 172.21.15.148:0/542798997 conn(0x7f5e140d0420 msgr2=0x7f5e141601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 shutdown_connections 2024-04-25T10:02:37.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:37.766+0000 7f5e18e38640 1 -- 172.21.15.148:0/542798997 wait complete. 2024-04-25T10:02:37.783 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:02:37.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:37.830 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:02:37.873 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:02:37.874 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:37.874 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:37.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:37.889 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:02:37.890 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:37.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:37.931 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:37.931 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:37.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:02:37.946 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:02:37.946 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:37.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:02:37.988 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:02:37.989 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 Processor -- start 2024-04-25T10:02:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 -- start start 2024-04-25T10:02:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f997415dbe0 0x7f99740d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b2cb0 0x7f99740d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99740b92a0 0x7f99740b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997415dfc0 con 0x7f997415dbe0 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740d1a00 con 0x7f99740b92a0 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.080+0000 7f9979d53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740b97a0 con 0x7f99740b2cb0 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f997415dbe0 0x7f99740d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f997415dbe0 0x7f99740d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50818/0 (socket says 172.21.15.148:50818) 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 -- 172.21.15.148:0/1733259371 learned_addr learned my addr 172.21.15.148:0/1733259371 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 -- 172.21.15.148:0/1733259371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b2cb0 msgr2=0x7f99740d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 --2- 172.21.15.148:0/1733259371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b2cb0 0x7f99740d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 -- 172.21.15.148:0/1733259371 >> v2:172.21.15.188:3300/0 conn(0x7f99740b92a0 msgr2=0x7f99740b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 --2- 172.21.15.148:0/1733259371 >> v2:172.21.15.188:3300/0 conn(0x7f99740b92a0 0x7f99740b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 -- 172.21.15.148:0/1733259371 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740d1c50 con 0x7f997415dbe0 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.081+0000 7f99737fe640 1 --2- 172.21.15.148:0/1733259371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740d0910 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f997406c640 tx=0x7f996402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f698af4fe9dc6c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f99727fc640 1 -- 172.21.15.148:0/1733259371 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964037070 con 0x7f997415dbe0 2024-04-25T10:02:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f99727fc640 1 -- 172.21.15.148:0/1733259371 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9964032440 con 0x7f997415dbe0 2024-04-25T10:02:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f99727fc640 1 -- 172.21.15.148:0/1733259371 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964032810 con 0x7f997415dbe0 2024-04-25T10:02:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 -- 172.21.15.148:0/1733259371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 msgr2=0x7f99740d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 --2- 172.21.15.148:0/1733259371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740d0910 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f997406c640 tx=0x7f996402a200 comp rx=0 tx=0).stop 2024-04-25T10:02:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 -- 172.21.15.148:0/1733259371 shutdown_connections 2024-04-25T10:02:38.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 --2- 172.21.15.148:0/1733259371 >> v2:172.21.15.188:3300/0 conn(0x7f99740b92a0 0x7f99740b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 --2- 172.21.15.148:0/1733259371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b2cb0 0x7f99740d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 --2- 172.21.15.148:0/1733259371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740d0910 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 -- 172.21.15.148:0/1733259371 >> 172.21.15.148:0/1733259371 conn(0x7f9974061a20 msgr2=0x7f9974061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 -- 172.21.15.148:0/1733259371 shutdown_connections 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.082+0000 7f9979d53640 1 -- 172.21.15.148:0/1733259371 wait complete. 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.083+0000 7f9979d53640 1 Processor -- start 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 -- start start 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99740b2cb0 0x7f99740cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b92a0 0x7f99740caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740b6ab0 con 0x7f997415dbe0 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740d2210 con 0x7f99740b2cb0 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9979d53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740b6c30 con 0x7f99740b92a0 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9973fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9973fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:50828/0 (socket says 172.21.15.148:50828) 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.084+0000 7f9973fff640 1 -- 172.21.15.148:0/4162647300 learned_addr learned my addr 172.21.15.148:0/4162647300 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9973fff640 1 -- 172.21.15.148:0/4162647300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b92a0 msgr2=0x7f99740caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9973fff640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b92a0 0x7f99740caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9973fff640 1 -- 172.21.15.148:0/4162647300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99740b2cb0 msgr2=0x7f99740cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9973fff640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99740b2cb0 0x7f99740cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9973fff640 1 -- 172.21.15.148:0/4162647300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740cc260 con 0x7f997415dbe0 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9973fff640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740cbd20 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f996c00b9e0 tx=0x7f996c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c019070 con 0x7f997415dbe0 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00cb20 con 0x7f997415dbe0 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740cdd60 con 0x7f997415dbe0 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c014440 con 0x7f997415dbe0 2024-04-25T10:02:38.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.085+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99740b3660 con 0x7f997415dbe0 2024-04-25T10:02:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.086+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974073990 con 0x7f997415dbe0 2024-04-25T10:02:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.086+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f996c01c070 con 0x7f997415dbe0 2024-04-25T10:02:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.087+0000 7f995bfff640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f996006b500 0x7f996006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.087+0000 7f99737fe640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f996006b500 0x7f996006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.087+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f996c089c60 con 0x7f997415dbe0 2024-04-25T10:02:38.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.087+0000 7f99737fe640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f996006b500 0x7f996006d9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9964034000 tx=0x7f996402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.091+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99740cc260 con 0x7f997415dbe0 2024-04-25T10:02:38.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.295+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f997415af70 con 0x7f997415dbe0 2024-04-25T10:02:38.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.296+0000 7f995bfff640 1 -- 172.21.15.148:0/4162647300 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f996c058ce0 con 0x7f997415dbe0 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/686524238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":686524238},{"type":"v1","addr":"172.21.15.188:6834","nonce":686524238}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7654,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2024415104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2024415104},{"type":"v1","addr":"172.21.15.188:6835","nonce":2024415104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8694,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2213381832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2213381832},{"type":"v1","addr":"172.21.15.148:6836","nonce":2213381832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3984562293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3984562293},{"type":"v1","addr":"172.21.15.148:6837","nonce":3984562293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[]} 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 69 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f996006b500 msgr2=0x7f996006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f996006b500 0x7f996006d9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9964034000 tx=0x7f996402c000 comp rx=0 tx=0).stop 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 msgr2=0x7f99740cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740cbd20 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f996c00b9e0 tx=0x7f996c00bea0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 shutdown_connections 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f996006b500 0x7f996006d9c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f997415dbe0 0x7f99740cbd20 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99740b92a0 0x7f99740caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 --2- 172.21.15.148:0/4162647300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99740b2cb0 0x7f99740cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 >> 172.21.15.148:0/4162647300 conn(0x7f9974061a20 msgr2=0x7f997415bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 shutdown_connections 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:38.299+0000 7f9979d53640 1 -- 172.21.15.148:0/4162647300 wait complete. 2024-04-25T10:02:38.315 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:02:38.315 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:02:38.315 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:02:38.315 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:38.316 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:02:38.316 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:02:38.316 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:02:38.316 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:38.316 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:02:38.316 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:02:38.316 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:02:38.316 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:38.317 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:02:38.317 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:02:38.317 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:02:38.317 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:38.327 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:02:38.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:02:38.328 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:02:38.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:02:38.329 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:02:38.329 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:02:38.370 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:02:38.370 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:38.371 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:02:38.371 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:38.372 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:02:38.372 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:38.442 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:02:38.449 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:02:38.450 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:02:38.450 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-25T10:02:38.445+0000 7f0ff577d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:02:44.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:02:44.327 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:02:44.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:02:44.330 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:02:44.331 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:44.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:02:44.413 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:02:44.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.437+0000 7f815797e640 1 Processor -- start 2024-04-25T10:02:44.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.437+0000 7f815797e640 1 -- start start 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f815797e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 0x7f81500cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f815797e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81500c75e0 con 0x7f815015bb50 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f815797e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81500c78c0 con 0x7f81500cbf20 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f815797e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500c8680 con 0x7f81500cf600 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f815571a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f815015bb50 0x7f81500c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56748/0 (socket says 172.21.15.148:56748) 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f815571a640 1 -- 172.21.15.148:0/1678171932 learned_addr learned my addr 172.21.15.148:0/1678171932 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8154f19640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 0x7f81500cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775843660 0 0) 0x7f81500c75e0 con 0x7f815015bb50 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f813c003520 con 0x7f815015bb50 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164948386 0 0) 0x7f813c003520 con 0x7f815015bb50 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 msgr2=0x7f81500cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 0x7f81500cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 >> v1:172.21.15.188:6789/0 conn(0x7f81500cbf20 legacy=0x7f81500cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500c7ba0 con 0x7f815015bb50 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.438+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f814c001670 con 0x7f815015bb50 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 >> v1:172.21.15.148:6789/0 conn(0x7f815015bb50 legacy=0x7f81500c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 0x7f813c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f813c003f20 0x7f813c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c004d40 0x7f813c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500c8680 con 0x7f813c003f20 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c00ba20 con 0x7f813c004d40 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c00bc20 con 0x7f81500cf600 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8154f19640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f813c003f20 0x7f813c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f815571a640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 0x7f813c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8155f1b640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c004d40 0x7f813c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8154f19640 1 -- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 msgr2=0x7f813c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8154f19640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81500cf600 0x7f813c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8154f19640 1 -- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c004d40 msgr2=0x7f813c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8154f19640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f813c004d40 0x7f813c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.439+0000 7f8154f19640 1 -- 172.21.15.148:0/1678171932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c003520 con 0x7f813c003f20 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f8154f19640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f813c003f20 0x7f813c004800 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ac0 tx=0x7f813c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f043eda236562b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140036070 con 0x7f813c003f20 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8140031400 con 0x7f813c003f20 2024-04-25T10:02:44.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f8147fff640 1 -- 172.21.15.148:0/1678171932 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140009690 con 0x7f813c003f20 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f815797e640 1 -- 172.21.15.148:0/1678171932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f813c003f20 msgr2=0x7f813c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f815797e640 1 --2- 172.21.15.148:0/1678171932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f813c003f20 0x7f813c004800 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f8140009ac0 tx=0x7f813c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f8155f1b640 1 -- 172.21.15.148:0/1678171932 reap_dead start 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f815797e640 1 -- 172.21.15.148:0/1678171932 shutdown_connections 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f815797e640 1 -- 172.21.15.148:0/1678171932 >> 172.21.15.148:0/1678171932 conn(0x7f815006f500 msgr2=0x7f815006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.440+0000 7f815797e640 1 -- 172.21.15.148:0/1678171932 shutdown_connections 2024-04-25T10:02:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.441+0000 7f815797e640 1 -- 172.21.15.148:0/1678171932 wait complete. 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 Processor -- start 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 -- start start 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c003f20 0x7f815015cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 0x7f81500cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81500c9760 0x7f81500c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500c72a0 con 0x7f81501558d0 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500c8160 con 0x7f81500c9760 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.442+0000 7f815797e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815015f6d0 con 0x7f813c003f20 2024-04-25T10:02:44.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 0x7f81500cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 0x7f81500cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34082/0 (socket says 172.21.15.148:34082) 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 -- 172.21.15.148:0/2385221748 learned_addr learned my addr 172.21.15.148:0/2385221748 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f815571a640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c003f20 0x7f815015cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8155f1b640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81500c9760 0x7f81500c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 -- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c003f20 msgr2=0x7f815015cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c003f20 0x7f815015cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 -- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81500c9760 msgr2=0x7f81500c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81500c9760 0x7f81500c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 -- 172.21.15.148:0/2385221748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500b7590 con 0x7f81501558d0 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f815571a640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c003f20 0x7f815015cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8155f1b640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81500c9760 0x7f81500c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:44.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8154f19640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 0x7f81500cb1e0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f8140034040 tx=0x7f8140004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140040020 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8140003a40 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81500b7870 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8140036040 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.443+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81500c8f60 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.444+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8150073a00 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.445+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8140043020 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.445+0000 7f8145ffb640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f813406b500 0x7f813406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.445+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f81400aa2e0 con 0x7f81501558d0 2024-04-25T10:02:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.445+0000 7f815571a640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f813406b500 0x7f813406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.446+0000 7f815571a640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f813406b500 0x7f813406d9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f814c001eb0 tx=0x7f814c001da0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.450+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f814004a070 con 0x7f81501558d0 2024-04-25T10:02:44.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:44.632+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f815015fc60 con 0x7f81501558d0 2024-04-25T10:02:45.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.782+0000 7f8145ffb640 1 -- 172.21.15.148:0/2385221748 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v129) v1 ==== 97+0+0 (secure 0 0 0) 0x7f81400392d0 con 0x7f81501558d0 2024-04-25T10:02:45.788 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f813406b500 msgr2=0x7f813406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f813406b500 0x7f813406d9c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f814c001eb0 tx=0x7f814c001da0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 msgr2=0x7f81500cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 0x7f81500cb1e0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f8140034040 tx=0x7f8140004730 comp rx=0 tx=0).stop 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 shutdown_connections 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f813406b500 0x7f813406d9c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81500c9760 0x7f81500c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81501558d0 0x7f81500cb1e0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 --2- 172.21.15.148:0/2385221748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f813c003f20 0x7f815015cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 >> 172.21.15.148:0/2385221748 conn(0x7f815006f500 msgr2=0x7f81500b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 shutdown_connections 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.786+0000 7f815797e640 1 -- 172.21.15.148:0/2385221748 wait complete. 2024-04-25T10:02:45.805 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:02:45.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.903+0000 7f7eb3ba5640 1 Processor -- start 2024-04-25T10:02:45.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.903+0000 7f7eb3ba5640 1 -- start start 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb3ba5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b3410 0x7f7eac0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb3ba5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7eac0b37f0 0x7f7eac0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb3ba5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eac0ace70 con 0x7f7eac0b41b0 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb3ba5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eac06df30 con 0x7f7eac0b37f0 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb3ba5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0726d0 con 0x7f7eac0b3410 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7eac0b37f0 0x7f7eac0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7eac0b37f0 0x7f7eac0b3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34090/0 (socket says 172.21.15.148:34090) 2024-04-25T10:02:45.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 -- 172.21.15.148:0/131331893 learned_addr learned my addr 172.21.15.148:0/131331893 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:45.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1941640 1 --2- 172.21.15.148:0/131331893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b3410 0x7f7eac0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 -- 172.21.15.148:0/131331893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b3410 msgr2=0x7f7eac0720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 --2- 172.21.15.148:0/131331893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b3410 0x7f7eac0720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 -- 172.21.15.148:0/131331893 >> v1:172.21.15.188:6789/0 conn(0x7f7eac0b41b0 legacy=0x7f7eac161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.904+0000 7f7eb1140640 1 -- 172.21.15.148:0/131331893 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac06d8e0 con 0x7f7eac0b37f0 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.905+0000 7f7eb1140640 1 --2- 172.21.15.148:0/131331893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b37f0 0x7f7eac0b3c70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009870 tx=0x7f7e9c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee588831334c37c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.905+0000 7f7eb093f640 1 -- 172.21.15.148:0/131331893 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c037070 con 0x7f7eac0b37f0 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.905+0000 7f7eb093f640 1 -- 172.21.15.148:0/131331893 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c032440 con 0x7f7eac0b37f0 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.905+0000 7f7eb093f640 1 -- 172.21.15.148:0/131331893 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c032810 con 0x7f7eac0b37f0 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/131331893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b37f0 msgr2=0x7f7eac0b3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/131331893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b37f0 0x7f7eac0b3c70 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009870 tx=0x7f7e9c02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:45.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/131331893 shutdown_connections 2024-04-25T10:02:45.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/131331893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b37f0 0x7f7eac0b3c70 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/131331893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b3410 0x7f7eac0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/131331893 >> 172.21.15.148:0/131331893 conn(0x7f7eac0cfae0 msgr2=0x7f7eac0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/131331893 shutdown_connections 2024-04-25T10:02:45.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.906+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/131331893 wait complete. 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.907+0000 7f7eb3ba5640 1 Processor -- start 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.907+0000 7f7eb3ba5640 1 -- start start 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eac0b3410 0x7f7eac143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b37f0 0x7f7eac1f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 0x7f7eac13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac06f3d0 con 0x7f7eac0b41b0 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac147890 con 0x7f7eac0b3410 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac06f550 con 0x7f7eac0b37f0 2024-04-25T10:02:45.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 0x7f7eac13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 0x7f7eac13e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34098/0 (socket says 172.21.15.148:34098) 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 -- 172.21.15.148:0/1174076599 learned_addr learned my addr 172.21.15.148:0/1174076599 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb1140640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b37f0 0x7f7eac1f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb1941640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eac0b3410 0x7f7eac143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 -- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b37f0 msgr2=0x7f7eac1f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b37f0 0x7f7eac1f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 -- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eac0b3410 msgr2=0x7f7eac143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eac0b3410 0x7f7eac143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 -- 172.21.15.148:0/1174076599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac143fd0 con 0x7f7eac0b41b0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb2142640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 0x7f7eac13e480 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c6b0 tx=0x7f7ea400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4003b70 con 0x7f7eac0b41b0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea4003d30 con 0x7f7eac0b41b0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac1442b0 con 0x7f7eac0b41b0 2024-04-25T10:02:45.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4005310 con 0x7f7eac0b41b0 2024-04-25T10:02:45.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.908+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac144870 con 0x7f7eac0b41b0 2024-04-25T10:02:45.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.909+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac147230 con 0x7f7eac0b41b0 2024-04-25T10:02:45.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.910+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ea40054b0 con 0x7f7eac0b41b0 2024-04-25T10:02:45.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.910+0000 7f7ea27fc640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e9006b4d0 0x7f7e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.910+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7ea4089f40 con 0x7f7eac0b41b0 2024-04-25T10:02:45.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.910+0000 7f7eb1941640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e9006b4d0 0x7f7e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.911+0000 7f7eb1941640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e9006b4d0 0x7f7e9006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006fd0 tx=0x7f7ea8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:45.914+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7eac147230 con 0x7f7eac0b41b0 2024-04-25T10:02:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.149+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7eac073990 con 0x7f7eac0b41b0 2024-04-25T10:02:46.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.149+0000 7f7ea27fc640 1 -- 172.21.15.148:0/1174076599 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7eac0ace70 con 0x7f7eac0b41b0 2024-04-25T10:02:46.154 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:46.154 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.155 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:02:46.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:02:46.159 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:46.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e9006b4d0 msgr2=0x7f7e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e9006b4d0 0x7f7e9006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006fd0 tx=0x7f7ea8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 msgr2=0x7f7eac13e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 0x7f7eac13e480 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c6b0 tx=0x7f7ea400cb70 comp rx=0 tx=0).stop 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 shutdown_connections 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e9006b4d0 0x7f7e9006d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eac0b41b0 0x7f7eac13e480 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7eac0b37f0 0x7f7eac1f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 --2- 172.21.15.148:0/1174076599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eac0b3410 0x7f7eac143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.151+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 >> 172.21.15.148:0/1174076599 conn(0x7f7eac0cfae0 msgr2=0x7f7eac141850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.152+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 shutdown_connections 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.152+0000 7f7eb3ba5640 1 -- 172.21.15.148:0/1174076599 wait complete. 2024-04-25T10:02:46.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.264+0000 7f575c72b640 1 Processor -- start 2024-04-25T10:02:46.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.264+0000 7f575c72b640 1 -- start start 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.265+0000 7f575c72b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f575415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.265+0000 7f575c72b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57540adbc0 0x7f5754071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.265+0000 7f575c72b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575406de40 con 0x7f5754072000 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.265+0000 7f575c72b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5754072470 con 0x7f57540adbc0 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.265+0000 7f575c72b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57540ad0e0 con 0x7f5754155fe0 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57540adbc0 0x7f5754071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f57540adbc0 0x7f5754071ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34100/0 (socket says 172.21.15.148:34100) 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 -- 172.21.15.148:0/2026864569 learned_addr learned my addr 172.21.15.148:0/2026864569 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f575a4c7640 1 --2- 172.21.15.148:0/2026864569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f575415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 -- 172.21.15.148:0/2026864569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 msgr2=0x7f575415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 --2- 172.21.15.148:0/2026864569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f575415c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 -- 172.21.15.148:0/2026864569 >> v1:172.21.15.188:6789/0 conn(0x7f5754072000 legacy=0x7f575406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.266+0000 7f5759cc6640 1 -- 172.21.15.148:0/2026864569 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540ad3c0 con 0x7f57540adbc0 2024-04-25T10:02:46.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f5759cc6640 1 --2- 172.21.15.148:0/2026864569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f5754071ac0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f575002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1cacb8cce8b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f57594c5640 1 -- 172.21.15.148:0/2026864569 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750037070 con 0x7f57540adbc0 2024-04-25T10:02:46.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f57594c5640 1 -- 172.21.15.148:0/2026864569 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750032440 con 0x7f57540adbc0 2024-04-25T10:02:46.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f57594c5640 1 -- 172.21.15.148:0/2026864569 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750032810 con 0x7f57540adbc0 2024-04-25T10:02:46.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 -- 172.21.15.148:0/2026864569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 msgr2=0x7f5754071ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 --2- 172.21.15.148:0/2026864569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f5754071ac0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f575002a210 comp rx=0 tx=0).stop 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 -- 172.21.15.148:0/2026864569 shutdown_connections 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 --2- 172.21.15.148:0/2026864569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f5754071ac0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 --2- 172.21.15.148:0/2026864569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f575415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 -- 172.21.15.148:0/2026864569 >> 172.21.15.148:0/2026864569 conn(0x7f57540af670 msgr2=0x7f57540b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 -- 172.21.15.148:0/2026864569 shutdown_connections 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.267+0000 7f575c72b640 1 -- 172.21.15.148:0/2026864569 wait complete. 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 Processor -- start 2024-04-25T10:02:46.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 -- start start 2024-04-25T10:02:46.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f57541f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f57541f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57541f3610 0x7f57541f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575406f880 con 0x7f57540adbc0 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57541563c0 con 0x7f57541f3610 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575c72b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575406fa00 con 0x7f5754155fe0 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575a4c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f57541f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575a4c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f57541f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34114/0 (socket says 172.21.15.148:34114) 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.269+0000 7f575a4c7640 1 -- 172.21.15.148:0/3084721393 learned_addr learned my addr 172.21.15.148:0/3084721393 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575a4c7640 1 -- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 msgr2=0x7f57541f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575a4c7640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f57541f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575a4c7640 1 -- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57541f3610 msgr2=0x7f57541f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575acc8640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57541f3610 0x7f57541f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575a4c7640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57541f3610 0x7f57541f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575a4c7640 1 -- 172.21.15.148:0/3084721393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57541f5eb0 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575a4c7640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f57541f5230 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f574400c7f0 tx=0x7f574400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744015070 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744004220 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57541f6190 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744007620 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.270+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57541f6750 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.271+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5754073990 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.271+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57440079d0 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.272+0000 7f574affd640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573806b4d0 0x7f573806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.272+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5744089fa0 con 0x7f57540adbc0 2024-04-25T10:02:46.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.272+0000 7f5759cc6640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573806b4d0 0x7f573806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.272+0000 7f5759cc6640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573806b4d0 0x7f573806d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5750034000 tx=0x7f575002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.276+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5744059070 con 0x7f57540adbc0 2024-04-25T10:02:46.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.446+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5754155090 con 0x7f57540adbc0 2024-04-25T10:02:46.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.446+0000 7f574affd640 1 -- 172.21.15.148:0/3084721393 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+13222 (secure 0 0 0) 0x7f5744058970 con 0x7f57540adbc0 2024-04-25T10:02:46.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:46.451 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":129,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:02:45.781996+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:46.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573806b4d0 msgr2=0x7f573806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573806b4d0 0x7f573806d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5750034000 tx=0x7f575002c000 comp rx=0 tx=0).stop 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 msgr2=0x7f57541f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f57541f5230 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f574400c7f0 tx=0x7f574400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 shutdown_connections 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573806b4d0 0x7f573806d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57541f3610 0x7f57541f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5754155fe0 0x7f57541f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 --2- 172.21.15.148:0/3084721393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57540adbc0 0x7f57541f5230 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.448+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 >> 172.21.15.148:0/3084721393 conn(0x7f5754072000 msgr2=0x7f57541545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.449+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 shutdown_connections 2024-04-25T10:02:46.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.449+0000 7f575c72b640 1 -- 172.21.15.148:0/3084721393 wait complete. 2024-04-25T10:02:46.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:46.465 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:46.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.562+0000 7fa5c0bc1640 1 Processor -- start 2024-04-25T10:02:46.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.562+0000 7fa5c0bc1640 1 -- start start 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5c0bc1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5bc0cd4c0 0x7fa5bc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5c0bc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc15b1c0 0x7fa5bc15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5c0bc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5bc15e1a0 0x7fa5bc150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5c0bc1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5bc15b620 con 0x7fa5bc0cd4c0 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5c0bc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5bc15e6e0 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5c0bc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc15e9c0 con 0x7fa5bc15b1c0 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5bc0cd4c0 0x7fa5bc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa5bc0cd4c0 0x7fa5bc0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34116/0 (socket says 172.21.15.148:34116) 2024-04-25T10:02:46.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 -- 172.21.15.148:0/2702578188 learned_addr learned my addr 172.21.15.148:0/2702578188 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5b9d9b640 1 --2- 172.21.15.148:0/2702578188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc15b1c0 0x7fa5bc15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5bad9d640 1 --2- 172.21.15.148:0/2702578188 >> v2:172.21.15.188:3300/0 conn(0x7fa5bc15e1a0 0x7fa5bc150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 -- 172.21.15.148:0/2702578188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc15b1c0 msgr2=0x7fa5bc15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/2702578188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc15b1c0 0x7fa5bc15dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 -- 172.21.15.148:0/2702578188 >> v2:172.21.15.188:3300/0 conn(0x7fa5bc15e1a0 msgr2=0x7fa5bc150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/2702578188 >> v2:172.21.15.188:3300/0 conn(0x7fa5bc15e1a0 0x7fa5bc150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.563+0000 7fa5ba59c640 1 -- 172.21.15.148:0/2702578188 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc15a9c0 con 0x7fa5bc0cd4c0 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.564+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/2702578188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc0cd4c0 0x7fa5bc0cd8a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0af3a0 tx=0x7fa5ac02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18893b16d0e83478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.564+0000 7fa5b959a640 1 -- 172.21.15.148:0/2702578188 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00b2e0 con 0x7fa5bc0cd4c0 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.564+0000 7fa5b959a640 1 -- 172.21.15.148:0/2702578188 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac009600 con 0x7fa5bc0cd4c0 2024-04-25T10:02:46.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.564+0000 7fa5b959a640 1 -- 172.21.15.148:0/2702578188 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00b480 con 0x7fa5bc0cd4c0 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/2702578188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc0cd4c0 msgr2=0x7fa5bc0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/2702578188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc0cd4c0 0x7fa5bc0cd8a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0af3a0 tx=0x7fa5ac02c8a0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/2702578188 shutdown_connections 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/2702578188 >> v2:172.21.15.188:3300/0 conn(0x7fa5bc15e1a0 0x7fa5bc150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/2702578188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc15b1c0 0x7fa5bc15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/2702578188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc0cd4c0 0x7fa5bc0cd8a0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/2702578188 >> 172.21.15.148:0/2702578188 conn(0x7fa5bc06abe0 msgr2=0x7fa5bc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/2702578188 shutdown_connections 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.565+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/2702578188 wait complete. 2024-04-25T10:02:46.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.566+0000 7fa5c0bc1640 1 Processor -- start 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.566+0000 7fa5c0bc1640 1 -- start start 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5c0bc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc0cd4c0 0x7fa5bc1f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5c0bc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5bc15b1c0 0x7fa5bc14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5c0bc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 0x7fa5bc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5c0bc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc06eca0 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5c0bc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc15a670 con 0x7fa5bc15b1c0 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5c0bc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc06ee20 con 0x7fa5bc0cd4c0 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5bad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 0x7fa5bc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5bad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 0x7fa5bc15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34132/0 (socket says 172.21.15.148:34132) 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5bad9d640 1 -- 172.21.15.148:0/4273059608 learned_addr learned my addr 172.21.15.148:0/4273059608 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc0cd4c0 0x7fa5bc1f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5b9d9b640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5bc15b1c0 0x7fa5bc14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5bad9d640 1 -- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc0cd4c0 msgr2=0x7fa5bc1f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.567+0000 7fa5bad9d640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc0cd4c0 0x7fa5bc1f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5bad9d640 1 -- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5bc15b1c0 msgr2=0x7fa5bc14cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5bad9d640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5bc15b1c0 0x7fa5bc14cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5bad9d640 1 -- 172.21.15.148:0/4273059608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc15d4f0 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5b9d9b640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5bc15b1c0 0x7fa5bc14cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc0cd4c0 0x7fa5bc1f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5bad9d640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 0x7fa5bc15c8e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d5a0 tx=0x7fa5b400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40125f0 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b4019070 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc15d210 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40129b0 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.568+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc1eff00 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.569+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc0cef90 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.569+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5b4012b50 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.570+0000 7fa5aaffd640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.570+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa5b408a3b0 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.570+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.570+0000 7fa5ba59c640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac036000 tx=0x7fa5ac00b000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.574+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5bc0cef90 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.744+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5bc073a00 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.745+0000 7fa5aaffd640 1 -- 172.21.15.148:0/4273059608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5b400f300 con 0x7fa5bc15e1a0 2024-04-25T10:02:46.750 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 msgr2=0x7fa59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac036000 tx=0x7fa5ac00b000 comp rx=0 tx=0).stop 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 msgr2=0x7fa5bc15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 0x7fa5bc15c8e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d5a0 tx=0x7fa5b400da60 comp rx=0 tx=0).stop 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 shutdown_connections 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc15e1a0 0x7fa5bc15c8e0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5bc15b1c0 0x7fa5bc14cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 --2- 172.21.15.148:0/4273059608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5bc0cd4c0 0x7fa5bc1f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 >> 172.21.15.148:0/4273059608 conn(0x7fa5bc06abe0 msgr2=0x7fa5bc14e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 shutdown_connections 2024-04-25T10:02:46.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.747+0000 7fa5c0bc1640 1 -- 172.21.15.148:0/4273059608 wait complete. 2024-04-25T10:02:46.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:02:46.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.855+0000 7f16be2a3640 1 Processor -- start 2024-04-25T10:02:46.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.855+0000 7f16be2a3640 1 -- start start 2024-04-25T10:02:46.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.856+0000 7f16be2a3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16b815db50 0x7f16b80c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.856+0000 7f16be2a3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b81625d0 con 0x7f16b80cfb30 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.856+0000 7f16be2a3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b80cc2b0 con 0x7f16b80c7050 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.856+0000 7f16be2a3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b815df30 con 0x7f16b815db50 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.856+0000 7f16bc840640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f16b80cfb30 0x7f16b8161860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56762/0 (socket says 172.21.15.148:56762) 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.856+0000 7f16bc840640 1 -- 172.21.15.148:0/2783355083 learned_addr learned my addr 172.21.15.148:0/2783355083 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 --2- 172.21.15.148:0/2783355083 >> v2:172.21.15.188:3300/0 conn(0x7f16b815db50 0x7f16b80c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047957505 0 0) 0x7f16b81625d0 con 0x7f16b80cfb30 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a0003520 con 0x7f16b80cfb30 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1248422433 0 0) 0x7f16a0003520 con 0x7f16b80cfb30 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 >> v2:172.21.15.188:3300/0 conn(0x7f16b815db50 msgr2=0x7f16b80c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 --2- 172.21.15.148:0/2783355083 >> v2:172.21.15.188:3300/0 conn(0x7f16b815db50 0x7f16b80c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 >> v1:172.21.15.188:6789/0 conn(0x7f16b80c7050 legacy=0x7f16b80cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80cc590 con 0x7f16b80cfb30 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f16b00019f0 con 0x7f16b80cfb30 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 >> v1:172.21.15.148:6789/0 conn(0x7f16b80cfb30 legacy=0x7f16b8161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a0004800 0x7f16a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a00051c0 0x7f16a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b815df30 con 0x7f16a00039b0 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a000c160 con 0x7f16a0004800 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a000c360 con 0x7f16a00051c0 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b77fe640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a0004800 0x7f16a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16bc840640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a00051c0 0x7f16a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 -- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a00051c0 msgr2=0x7f16a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16a00051c0 0x7f16a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 -- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a0004800 msgr2=0x7f16a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16a0004800 0x7f16a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16b7fff640 1 -- 172.21.15.148:0/2783355083 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0003520 con 0x7f16a00039b0 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.857+0000 7f16bc840640 1 -- 172.21.15.148:0/2783355083 reap_dead start 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.858+0000 7f16b7fff640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16a0003dd0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009ac0 tx=0x7f16a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a193875332e749eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.858+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8036070 con 0x7f16a00039b0 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.858+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8031400 con 0x7f16a00039b0 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.858+0000 7f16b6ffd640 1 -- 172.21.15.148:0/2783355083 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8009690 con 0x7f16a00039b0 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 -- 172.21.15.148:0/2783355083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 msgr2=0x7f16a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16a0003dd0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009ac0 tx=0x7f16a0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 -- 172.21.15.148:0/2783355083 shutdown_connections 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 --2- 172.21.15.148:0/2783355083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16a0003dd0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 -- 172.21.15.148:0/2783355083 >> 172.21.15.148:0/2783355083 conn(0x7f16b80d1800 msgr2=0x7f16b80d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 -- 172.21.15.148:0/2783355083 shutdown_connections 2024-04-25T10:02:46.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.859+0000 7f16be2a3640 1 -- 172.21.15.148:0/2783355083 wait complete. 2024-04-25T10:02:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.860+0000 7f16be2a3640 1 Processor -- start 2024-04-25T10:02:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.860+0000 7f16be2a3640 1 -- start start 2024-04-25T10:02:46.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16be2a3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16b81523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16be2a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16b80ca230 0x7f16b80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16be2a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16b80cb070 0x7f16b80c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16be2a3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80c6b10 con 0x7f16a00039b0 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16be2a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80ccb50 con 0x7f16b80cb070 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16be2a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b80c6c90 con 0x7f16b80ca230 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16b81523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16b81523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34150/0 (socket says 172.21.15.148:34150) 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 -- 172.21.15.148:0/1636785858 learned_addr learned my addr 172.21.15.148:0/1636785858 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b77fe640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16b80ca230 0x7f16b80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16bc840640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16b80cb070 0x7f16b80c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 -- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16b80ca230 msgr2=0x7f16b80cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16b80ca230 0x7f16b80cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 -- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16b80cb070 msgr2=0x7f16b80c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16b80cb070 0x7f16b80c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 -- 172.21.15.148:0/1636785858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8159ea0 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16bc840640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16b80cb070 0x7f16b80c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b7fff640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16b81523e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009be0 tx=0x7f16a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a803f020 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.861+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8037040 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.862+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b815a790 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.862+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8036070 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.862+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b815ad50 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.863+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b80cb480 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.863+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16a8042020 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.863+0000 7f16b4ff9640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16a406b4d0 0x7f16a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.863+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f16a80a9f50 con 0x7f16a00039b0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.863+0000 7f16b77fe640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16a406b4d0 0x7f16a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.864+0000 7f16b77fe640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16a406b4d0 0x7f16a406d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f16b8068730 tx=0x7f16ac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:46.867+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16b80cb480 con 0x7f16a00039b0 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.041+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f16b8073990 con 0x7f16a00039b0 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.041+0000 7f16b4ff9640 1 -- 172.21.15.148:0/1636785858 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v129) v1 ==== 84+0+65 (secure 0 0 0) 0x7f16a8049030 con 0x7f16a00039b0 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:02:47.046 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:02:47.047 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:02:47.047 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:02:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16a406b4d0 msgr2=0x7f16a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16a406b4d0 0x7f16a406d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f16b8068730 tx=0x7f16ac009040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 msgr2=0x7f16b81523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16b81523e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009be0 tx=0x7f16a8004730 comp rx=0 tx=0).stop 2024-04-25T10:02:47.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 shutdown_connections 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f16a406b4d0 0x7f16a406d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16b80cb070 0x7f16b80c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16b80ca230 0x7f16b80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 --2- 172.21.15.148:0/1636785858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a00039b0 0x7f16b81523e0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 >> 172.21.15.148:0/1636785858 conn(0x7f16b80d1800 msgr2=0x7f16b81584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 shutdown_connections 2024-04-25T10:02:47.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.043+0000 7f16be2a3640 1 -- 172.21.15.148:0/1636785858 wait complete. 2024-04-25T10:02:47.059 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:02:47.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:02:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.157+0000 7f7bf2efc640 1 Processor -- start 2024-04-25T10:02:47.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.157+0000 7f7bf2efc640 1 -- start start 2024-04-25T10:02:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf2efc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bec15b5a0 0x7f7bec0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf2efc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7bec15b980 0x7f7bec0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf2efc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec162450 0x7f7bec0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf2efc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec0cf320 con 0x7f7bec15b5a0 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf2efc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec0cf040 con 0x7f7bec15b980 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf2efc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec161970 con 0x7f7bec162450 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bec15b5a0 0x7f7bec0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7bec15b5a0 0x7f7bec0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34158/0 (socket says 172.21.15.148:34158) 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 -- 172.21.15.148:0/1010362503 learned_addr learned my addr 172.21.15.148:0/1010362503 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bebfff640 1 --2- 172.21.15.148:0/1010362503 >> v2:172.21.15.188:3300/0 conn(0x7f7bec15b980 0x7f7bec0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf1499640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec162450 0x7f7bec0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 -- 172.21.15.148:0/1010362503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec162450 msgr2=0x7f7bec0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec162450 0x7f7bec0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 -- 172.21.15.148:0/1010362503 >> v2:172.21.15.188:3300/0 conn(0x7f7bec15b980 msgr2=0x7f7bec0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 --2- 172.21.15.148:0/1010362503 >> v2:172.21.15.188:3300/0 conn(0x7f7bec15b980 0x7f7bec0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf0c98640 1 -- 172.21.15.148:0/1010362503 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec161c50 con 0x7f7bec15b5a0 2024-04-25T10:02:47.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bf1499640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec162450 0x7f7bec0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.158+0000 7f7bebfff640 1 --2- 172.21.15.148:0/1010362503 >> v2:172.21.15.188:3300/0 conn(0x7f7bec15b980 0x7f7bec0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.159+0000 7f7bf0c98640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec0b5530 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0b9bf0 tx=0x7f7be002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cce2bc393ffbaec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.159+0000 7f7beb7fe640 1 -- 172.21.15.148:0/1010362503 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0037070 con 0x7f7bec15b5a0 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.159+0000 7f7beb7fe640 1 -- 172.21.15.148:0/1010362503 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0032440 con 0x7f7bec15b5a0 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.159+0000 7f7beb7fe640 1 -- 172.21.15.148:0/1010362503 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0032810 con 0x7f7bec15b5a0 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 -- 172.21.15.148:0/1010362503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 msgr2=0x7f7bec0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec0b5530 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f7bec0b9bf0 tx=0x7f7be002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 -- 172.21.15.148:0/1010362503 shutdown_connections 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec162450 0x7f7bec0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/1010362503 >> v2:172.21.15.188:3300/0 conn(0x7f7bec15b980 0x7f7bec0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/1010362503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec0b5530 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 -- 172.21.15.148:0/1010362503 >> 172.21.15.148:0/1010362503 conn(0x7f7bec0d0530 msgr2=0x7f7bec0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 -- 172.21.15.148:0/1010362503 shutdown_connections 2024-04-25T10:02:47.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.160+0000 7f7bf2efc640 1 -- 172.21.15.148:0/1010362503 wait complete. 2024-04-25T10:02:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.161+0000 7f7bf2efc640 1 Processor -- start 2024-04-25T10:02:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.161+0000 7f7bf2efc640 1 -- start start 2024-04-25T10:02:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf2efc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf2efc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec15b980 0x7f7bec152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf2efc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bec162450 0x7f7bec0c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf2efc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0ce9d0 con 0x7f7bec15b5a0 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf2efc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0ce850 con 0x7f7bec162450 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf2efc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0ceb50 con 0x7f7bec15b980 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec1f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34172/0 (socket says 172.21.15.148:34172) 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 -- 172.21.15.148:0/2269980432 learned_addr learned my addr 172.21.15.148:0/2269980432 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf1499640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bec162450 0x7f7bec0c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bebfff640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec15b980 0x7f7bec152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 -- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec15b980 msgr2=0x7f7bec152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec15b980 0x7f7bec152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 -- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bec162450 msgr2=0x7f7bec0c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bec162450 0x7f7bec0c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 -- 172.21.15.148:0/2269980432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec152b90 con 0x7f7bec15b5a0 2024-04-25T10:02:47.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf1499640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bec162450 0x7f7bec0c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bebfff640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec15b980 0x7f7bec152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7bf0c98640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec1f3bf0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000c00 tx=0x7f7be00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.162+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0037040 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.163+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0042070 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.163+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec15f2b0 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.163+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0030bb0 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.163+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec15f870 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.164+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec0ca890 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.164+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7be0047460 con 0x7f7bec15b5a0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.164+0000 7f7be97fa640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7bd006b4d0 0x7f7bd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.165+0000 7f7bebfff640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7bd006b4d0 0x7f7bd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.165+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7be00aac70 con 0x7f7bec15b5a0 2024-04-25T10:02:47.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.165+0000 7f7bebfff640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7bd006b4d0 0x7f7bd006d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8006fd0 tx=0x7f7bd8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.169+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bec0ca890 con 0x7f7bec15b5a0 2024-04-25T10:02:47.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.340+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7bec0d2940 con 0x7f7bec15b5a0 2024-04-25T10:02:47.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.806+0000 7f7be97fa640 1 -- 172.21.15.148:0/2269980432 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v130) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7be0045360 con 0x7f7bec15b5a0 2024-04-25T10:02:47.811 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7bd006b4d0 msgr2=0x7f7bd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7bd006b4d0 0x7f7bd006d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8006fd0 tx=0x7f7bd8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 msgr2=0x7f7bec1f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec1f3bf0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f7be0000c00 tx=0x7f7be00046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 shutdown_connections 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7bd006b4d0 0x7f7bd006d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bec162450 0x7f7bec0c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bec15b980 0x7f7bec152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 --2- 172.21.15.148:0/2269980432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bec15b5a0 0x7f7bec1f3bf0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 >> 172.21.15.148:0/2269980432 conn(0x7f7bec0d0530 msgr2=0x7f7bec15be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 shutdown_connections 2024-04-25T10:02:47.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.808+0000 7f7bf2efc640 1 -- 172.21.15.148:0/2269980432 wait complete. 2024-04-25T10:02:47.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:02:47.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.928+0000 7f800439e640 1 Processor -- start 2024-04-25T10:02:47.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.929+0000 7f800439e640 1 -- start start 2024-04-25T10:02:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.929+0000 7f800439e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 0x7f7ffc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.929+0000 7f800439e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 0x7f7ffc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.929+0000 7f800439e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ffc15e000 con 0x7f7ffc15dc20 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f800439e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ffc0d1de0 con 0x7f7ffc161710 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f800439e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc161c60 con 0x7f7ffc0ced80 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 0x7f7ffc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f800293b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 0x7f7ffc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 0x7f7ffc0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:47482/0 (socket says 172.21.15.148:47482) 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f800293b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 0x7f7ffc0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:48618/0 (socket says 172.21.15.148:48618) 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 -- 172.21.15.148:0/914779671 learned_addr learned my addr 172.21.15.148:0/914779671 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:47.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f800213a640 1 --1- 172.21.15.148:0/914779671 >> v1:172.21.15.188:6789/0 conn(0x7f7ffc15dc20 0x7f7ffc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:38720/0 (socket says 172.21.15.148:38720) 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 -- 172.21.15.148:0/914779671 >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 msgr2=0x7f7ffc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 --2- 172.21.15.148:0/914779671 >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 0x7f7ffc0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 -- 172.21.15.148:0/914779671 >> v1:172.21.15.188:6789/0 conn(0x7f7ffc15dc20 legacy=0x7f7ffc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f8001939640 1 -- 172.21.15.148:0/914779671 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc161f40 con 0x7f7ffc0ced80 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.930+0000 7f800293b640 1 --2- 172.21.15.148:0/914779671 >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 0x7f7ffc0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.931+0000 7f8001939640 1 --2- 172.21.15.148:0/914779671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 0x7f7ffc0d22e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac268487791049f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.931+0000 7f8001138640 1 -- 172.21.15.148:0/914779671 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec037070 con 0x7f7ffc0ced80 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.931+0000 7f8001138640 1 -- 172.21.15.148:0/914779671 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec032440 con 0x7f7ffc0ced80 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.931+0000 7f8001138640 1 -- 172.21.15.148:0/914779671 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec032810 con 0x7f7ffc0ced80 2024-04-25T10:02:47.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 -- 172.21.15.148:0/914779671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 msgr2=0x7f7ffc0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 --2- 172.21.15.148:0/914779671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 0x7f7ffc0d22e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009870 tx=0x7f7fec02a270 comp rx=0 tx=0).stop 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 -- 172.21.15.148:0/914779671 shutdown_connections 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 --2- 172.21.15.148:0/914779671 >> v2:172.21.15.188:3300/0 conn(0x7f7ffc161710 0x7f7ffc0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 --2- 172.21.15.148:0/914779671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc0ced80 0x7f7ffc0d22e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 -- 172.21.15.148:0/914779671 >> 172.21.15.148:0/914779671 conn(0x7f7ffc0d0420 msgr2=0x7f7ffc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 -- 172.21.15.148:0/914779671 shutdown_connections 2024-04-25T10:02:47.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.932+0000 7f800439e640 1 -- 172.21.15.148:0/914779671 wait complete. 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.933+0000 7f800439e640 1 Processor -- start 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.933+0000 7f800439e640 1 -- start start 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800439e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc0ced80 0x7f7ffc15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800439e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc15dc20 0x7f7ffc0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800439e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 0x7f7ffc159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800439e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0b5e50 con 0x7f7ffc161710 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800439e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc162500 con 0x7f7ffc0ced80 2024-04-25T10:02:47.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800439e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0b5fd0 con 0x7f7ffc15dc20 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 0x7f7ffc159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 0x7f7ffc159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34176/0 (socket says 172.21.15.148:34176) 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 -- 172.21.15.148:0/3933783488 learned_addr learned my addr 172.21.15.148:0/3933783488 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f8001939640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc15dc20 0x7f7ffc0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800213a640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc0ced80 0x7f7ffc15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 -- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc15dc20 msgr2=0x7f7ffc0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc15dc20 0x7f7ffc0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 -- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc0ced80 msgr2=0x7f7ffc15d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc0ced80 0x7f7ffc15d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 -- 172.21.15.148:0/3933783488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc15a3e0 con 0x7f7ffc161710 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.934+0000 7f800293b640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 0x7f7ffc159ea0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400efc0 tx=0x7f7ff400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.935+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4018030 con 0x7f7ffc161710 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.935+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff400cd30 con 0x7f7ffc161710 2024-04-25T10:02:47.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.935+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc0cd560 con 0x7f7ffc161710 2024-04-25T10:02:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.935+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4003bb0 con 0x7f7ffc161710 2024-04-25T10:02:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.935+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc0cbb90 con 0x7f7ffc161710 2024-04-25T10:02:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.935+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ffc073a00 con 0x7f7ffc161710 2024-04-25T10:02:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.936+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ff4005d10 con 0x7f7ffc161710 2024-04-25T10:02:47.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.937+0000 7f7ff2ffd640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe006b4d0 0x7f7fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.937+0000 7f800213a640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe006b4d0 0x7f7fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.937+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f7ff408b030 con 0x7f7ffc161710 2024-04-25T10:02:47.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.937+0000 7f800213a640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe006b4d0 0x7f7fe006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8007fd0 tx=0x7f7ff8007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:47.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:47.940+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ff4022070 con 0x7f7ffc161710 2024-04-25T10:02:48.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.111+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7ffc0d16b0 con 0x7f7ffc161710 2024-04-25T10:02:48.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.808+0000 7f7ff2ffd640 1 -- 172.21.15.148:0/3933783488 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v131) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7ff4010040 con 0x7f7ffc161710 2024-04-25T10:02:48.813 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe006b4d0 msgr2=0x7f7fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe006b4d0 0x7f7fe006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8007fd0 tx=0x7f7ff8007e80 comp rx=0 tx=0).stop 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 msgr2=0x7f7ffc159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 0x7f7ffc159ea0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400efc0 tx=0x7f7ff400c490 comp rx=0 tx=0).stop 2024-04-25T10:02:48.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 shutdown_connections 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7fe006b4d0 0x7f7fe006d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ffc161710 0x7f7ffc159ea0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ffc15dc20 0x7f7ffc0cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 --2- 172.21.15.148:0/3933783488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ffc0ced80 0x7f7ffc15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 >> 172.21.15.148:0/3933783488 conn(0x7f7ffc0d0420 msgr2=0x7f7ffc0d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.811+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 shutdown_connections 2024-04-25T10:02:48.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.812+0000 7f800439e640 1 -- 172.21.15.148:0/3933783488 wait complete. 2024-04-25T10:02:48.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:02:48.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.940+0000 7f94caa76640 1 Processor -- start 2024-04-25T10:02:48.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.940+0000 7f94caa76640 1 -- start start 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.941+0000 7f94caa76640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f94c40d34c0 0x7f94c40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.941+0000 7f94caa76640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c40d2420 con 0x7f94c40d08e0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.941+0000 7f94caa76640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c40d2700 con 0x7f94c415dac0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.941+0000 7f94caa76640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c40d0d50 con 0x7f94c40d34c0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.941+0000 7f94c9013640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f94c40d08e0 0x7f94c40b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56764/0 (socket says 172.21.15.148:56764) 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.941+0000 7f94c9013640 1 -- 172.21.15.148:0/3898443872 learned_addr learned my addr 172.21.15.148:0/3898443872 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 --2- 172.21.15.148:0/3898443872 >> v2:172.21.15.188:3300/0 conn(0x7f94c40d34c0 0x7f94c40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686066372 0 0) 0x7f94c40d2420 con 0x7f94c40d08e0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94ac003520 con 0x7f94c40d08e0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060452396 0 0) 0x7f94ac003520 con 0x7f94c40d08e0 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 >> v2:172.21.15.188:3300/0 conn(0x7f94c40d34c0 msgr2=0x7f94c40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 --2- 172.21.15.148:0/3898443872 >> v2:172.21.15.188:3300/0 conn(0x7f94c40d34c0 0x7f94c40d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 >> v1:172.21.15.188:6789/0 conn(0x7f94c415dac0 legacy=0x7f94c40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40d29e0 con 0x7f94c40d08e0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f94bc0019f0 con 0x7f94c40d08e0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 >> v1:172.21.15.148:6789/0 conn(0x7f94c40d08e0 legacy=0x7f94c40b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ac0039b0 0x7f94ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ac0051c0 0x7f94ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40d0d50 con 0x7f94ac004800 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac00c070 con 0x7f94ac0051c0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac00c270 con 0x7f94ac0039b0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c9013640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ac0051c0 0x7f94ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c8812640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ac0039b0 0x7f94ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 -- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ac0039b0 msgr2=0x7f94ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ac0039b0 0x7f94ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 -- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ac0051c0 msgr2=0x7f94ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ac0051c0 0x7f94ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c9013640 1 -- 172.21.15.148:0/3898443872 reap_dead start 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.942+0000 7f94c3fff640 1 -- 172.21.15.148:0/3898443872 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac003520 con 0x7f94ac004800 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.943+0000 7f94c3fff640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94ac004c80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009ab0 tx=0x7f94ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3668b5d64f520e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.943+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8036070 con 0x7f94ac004800 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.943+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b8005990 con 0x7f94ac004800 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.943+0000 7f94c37fe640 1 -- 172.21.15.148:0/3898443872 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8009680 con 0x7f94ac004800 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 -- 172.21.15.148:0/3898443872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 msgr2=0x7f94ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94ac004c80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009ab0 tx=0x7f94ac005830 comp rx=0 tx=0).stop 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 -- 172.21.15.148:0/3898443872 shutdown_connections 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 --2- 172.21.15.148:0/3898443872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94ac004c80 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 -- 172.21.15.148:0/3898443872 >> 172.21.15.148:0/3898443872 conn(0x7f94c4061c30 msgr2=0x7f94c406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 -- 172.21.15.148:0/3898443872 shutdown_connections 2024-04-25T10:02:48.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.944+0000 7f94caa76640 1 -- 172.21.15.148:0/3898443872 wait complete. 2024-04-25T10:02:48.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.945+0000 7f94caa76640 1 Processor -- start 2024-04-25T10:02:48.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.945+0000 7f94caa76640 1 -- start start 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94caa76640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94c415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94caa76640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c41f7fa0 0x7f94c41f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94caa76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c41f7320 0x7f94c41f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94caa76640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c415dea0 con 0x7f94ac004800 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94caa76640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40d3030 con 0x7f94c41f7fa0 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94caa76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40b97d0 con 0x7f94c41f7320 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94c415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94c415e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34194/0 (socket says 172.21.15.148:34194) 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 -- 172.21.15.148:0/4020363675 learned_addr learned my addr 172.21.15.148:0/4020363675 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c9013640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c41f7320 0x7f94c41f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c3fff640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c41f7fa0 0x7f94c41f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 -- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c41f7320 msgr2=0x7f94c41f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c41f7320 0x7f94c41f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 -- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c41f7fa0 msgr2=0x7f94c41f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c41f7fa0 0x7f94c41f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 -- 172.21.15.148:0/4020363675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4156420 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.946+0000 7f94c8812640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94c415e020 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f94b400f820 tx=0x7f94b400fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.947+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40050d0 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.947+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4018070 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.947+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4155660 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.947+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b400d730 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.947+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c4155c20 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.948+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c4073a00 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.948+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94b400d8d0 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.948+0000 7f94c17fa640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f94b006b4d0 0x7f94b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.949+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f94b4089f70 con 0x7f94ac004800 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.949+0000 7f94c3fff640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f94b006b4d0 0x7f94b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.949+0000 7f94c3fff640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f94b006b4d0 0x7f94b006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f94b8033000 tx=0x7f94b8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:48.953+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94b4058d50 con 0x7f94ac004800 2024-04-25T10:02:49.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.162+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f94c4068090 con 0x7f94ac004800 2024-04-25T10:02:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.834+0000 7f94c17fa640 1 -- 172.21.15.148:0/4020363675 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v72) v1 ==== 174+0+0 (secure 0 0 0) 0x7f94b4058650 con 0x7f94ac004800 2024-04-25T10:02:49.839 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T10:02:49.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f94b006b4d0 msgr2=0x7f94b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f94b006b4d0 0x7f94b006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f94b8033000 tx=0x7f94b8008030 comp rx=0 tx=0).stop 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 msgr2=0x7f94c415e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94c415e020 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f94b400f820 tx=0x7f94b400fc20 comp rx=0 tx=0).stop 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 shutdown_connections 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f94b006b4d0 0x7f94b006d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94c41f7320 0x7f94c41f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94c41f7fa0 0x7f94c41f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 --2- 172.21.15.148:0/4020363675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94ac004800 0x7f94c415e020 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 >> 172.21.15.148:0/4020363675 conn(0x7f94c4061c30 msgr2=0x7f94c406c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.842+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 shutdown_connections 2024-04-25T10:02:49.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.843+0000 7f94caa76640 1 -- 172.21.15.148:0/4020363675 wait complete. 2024-04-25T10:02:49.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.956+0000 7f270f43f640 1 Processor -- start 2024-04-25T10:02:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.956+0000 7f270f43f640 1 -- start start 2024-04-25T10:02:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270f43f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f270815ba70 0x7f27080725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270f43f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f270806f360 0x7f27080cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270f43f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f27080cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270f43f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27080cf750 con 0x7f270815ba70 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270f43f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270815be50 con 0x7f270806f360 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270f43f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27080717f0 con 0x7f27080aefa0 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270d1db640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f270815ba70 0x7f27080725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270d1db640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f270815ba70 0x7f27080725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34198/0 (socket says 172.21.15.148:34198) 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.957+0000 7f270d1db640 1 -- 172.21.15.148:0/3923527806 learned_addr learned my addr 172.21.15.148:0/3923527806 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270d1db640 1 -- 172.21.15.148:0/3923527806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 msgr2=0x7f27080cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270d1db640 1 --2- 172.21.15.148:0/3923527806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f27080cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270d1db640 1 -- 172.21.15.148:0/3923527806 >> v2:172.21.15.188:3300/0 conn(0x7f270806f360 msgr2=0x7f27080cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270c9da640 1 --2- 172.21.15.148:0/3923527806 >> v2:172.21.15.188:3300/0 conn(0x7f270806f360 0x7f27080cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270d1db640 1 --2- 172.21.15.148:0/3923527806 >> v2:172.21.15.188:3300/0 conn(0x7f270806f360 0x7f27080cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270d1db640 1 -- 172.21.15.148:0/3923527806 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2708071ad0 con 0x7f270815ba70 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f270d1db640 1 --2- 172.21.15.148:0/3923527806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270815ba70 0x7f27080725b0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f27080712b0 tx=0x7f270402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec1fb98a8073b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f26fffff640 1 -- 172.21.15.148:0/3923527806 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704036070 con 0x7f270815ba70 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f26fffff640 1 -- 172.21.15.148:0/3923527806 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704031600 con 0x7f270815ba70 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.958+0000 7f26fffff640 1 -- 172.21.15.148:0/3923527806 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27040319d0 con 0x7f270815ba70 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 -- 172.21.15.148:0/3923527806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270815ba70 msgr2=0x7f27080725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 --2- 172.21.15.148:0/3923527806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270815ba70 0x7f27080725b0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f27080712b0 tx=0x7f270402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 -- 172.21.15.148:0/3923527806 shutdown_connections 2024-04-25T10:02:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 --2- 172.21.15.148:0/3923527806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f27080cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 --2- 172.21.15.148:0/3923527806 >> v2:172.21.15.188:3300/0 conn(0x7f270806f360 0x7f27080cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 --2- 172.21.15.148:0/3923527806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270815ba70 0x7f27080725b0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 -- 172.21.15.148:0/3923527806 >> 172.21.15.148:0/3923527806 conn(0x7f27080cfae0 msgr2=0x7f27080cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 -- 172.21.15.148:0/3923527806 shutdown_connections 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.959+0000 7f270f43f640 1 -- 172.21.15.148:0/3923527806 wait complete. 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.960+0000 7f270f43f640 1 Processor -- start 2024-04-25T10:02:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.960+0000 7f270f43f640 1 -- start start 2024-04-25T10:02:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270f43f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 0x7f27080b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270f43f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f270814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270f43f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270815ba70 0x7f2708150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270f43f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080cd7e0 con 0x7f270806f360 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270f43f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2708072090 con 0x7f270815ba70 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270f43f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27080cd420 con 0x7f27080aefa0 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 0x7f27080b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 0x7f27080b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34210/0 (socket says 172.21.15.148:34210) 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 -- 172.21.15.148:0/4137208770 learned_addr learned my addr 172.21.15.148:0/4137208770 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270c9da640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f270814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d9dc640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270815ba70 0x7f2708150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 -- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 msgr2=0x7f270814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f270814b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 -- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270815ba70 msgr2=0x7f2708150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270815ba70 0x7f2708150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.961+0000 7f270d1db640 1 -- 172.21.15.148:0/4137208770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2708153ac0 con 0x7f270806f360 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.962+0000 7f270d1db640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 0x7f27080b3800 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2704000f80 tx=0x7f2704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.962+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704036050 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.962+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704042070 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.962+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270814e3a0 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.962+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704009510 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.962+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270814e920 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.963+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2708073990 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.963+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2704007c70 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.964+0000 7f26fdffb640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f26ec06b500 0x7f26ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.964+0000 7f270c9da640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f26ec06b500 0x7f26ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.964+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27040ab2d0 con 0x7f270806f360 2024-04-25T10:02:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.964+0000 7f270c9da640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f26ec06b500 0x7f26ec06d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f27080b3790 tx=0x7f26f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:49.968+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f270407a050 con 0x7f270806f360 2024-04-25T10:02:50.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.141+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2708072870 con 0x7f270806f360 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.142+0000 7f26fdffb640 1 -- 172.21.15.148:0/4137208770 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16961 (secure 0 0 0) 0x7f2704079950 con 0x7f270806f360 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:50.147 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":132,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:02:49.163160+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:50.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f26ec06b500 msgr2=0x7f26ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f26ec06b500 0x7f26ec06d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f27080b3790 tx=0x7f26f8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:50.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 msgr2=0x7f27080b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 0x7f27080b3800 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2704000f80 tx=0x7f2704004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:50.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 shutdown_connections 2024-04-25T10:02:50.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f26ec06b500 0x7f26ec06d9c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270815ba70 0x7f2708150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27080aefa0 0x7f270814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 --2- 172.21.15.148:0/4137208770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f270806f360 0x7f27080b3800 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 >> 172.21.15.148:0/4137208770 conn(0x7f27080cfae0 msgr2=0x7f27080b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 shutdown_connections 2024-04-25T10:02:50.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.144+0000 7f270f43f640 1 -- 172.21.15.148:0/4137208770 wait complete. 2024-04-25T10:02:50.160 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.254+0000 7fa141f42640 1 Processor -- start 2024-04-25T10:02:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.254+0000 7fa141f42640 1 -- start start 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.254+0000 7fa141f42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 0x7fa13c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.254+0000 7fa141f42640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0cec00 con 0x7fa13c15db60 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.254+0000 7fa141f42640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0ceee0 con 0x7fa13c0b5f00 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa141f42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0ce930 con 0x7fa13c0d0b10 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa13c15db60 0x7fa13c0d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56772/0 (socket says 172.21.15.148:56772) 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13b7fe640 1 -- 172.21.15.148:0/777458014 learned_addr learned my addr 172.21.15.148:0/777458014 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999938479 0 0) 0x7fa13c0cec00 con 0x7fa13c15db60 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa124003520 con 0x7fa13c15db60 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13affd640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 0x7fa13c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2907774773 0 0) 0x7fa124003520 con 0x7fa13c15db60 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 msgr2=0x7fa13c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 0x7fa13c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 >> v1:172.21.15.188:6789/0 conn(0x7fa13c0b5f00 legacy=0x7fa13c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0cf1c0 con 0x7fa13c15db60 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa12c001670 con 0x7fa13c15db60 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 >> v1:172.21.15.148:6789/0 conn(0x7fa13c15db60 legacy=0x7fa13c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1240039e0 0x7fa124003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 0x7fa124004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa124004d40 0x7fa1240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0ce930 con 0x7fa1240039e0 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12400baf0 con 0x7fa124004d40 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12400bcf0 con 0x7fa13c0d0b10 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1240039e0 0x7fa124003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13affd640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 0x7fa124004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.255+0000 7fa13bfff640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa124004d40 0x7fa1240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13b7fe640 1 -- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 msgr2=0x7fa124004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa13c0d0b10 0x7fa124004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13b7fe640 1 -- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa124004d40 msgr2=0x7fa1240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa124004d40 0x7fa1240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13b7fe640 1 -- 172.21.15.148:0/777458014 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa124003520 con 0x7fa1240039e0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1240039e0 0x7fa124003e00 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00c240 tx=0x7fa124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7dc710c53c0a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c007cf0 con 0x7fa1240039e0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c005af0 con 0x7fa1240039e0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.256+0000 7fa13a7fc640 1 -- 172.21.15.148:0/777458014 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c007070 con 0x7fa1240039e0 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa141f42640 1 -- 172.21.15.148:0/777458014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1240039e0 msgr2=0x7fa124003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa141f42640 1 --2- 172.21.15.148:0/777458014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1240039e0 0x7fa124003e00 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00c240 tx=0x7fa124005830 comp rx=0 tx=0).stop 2024-04-25T10:02:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa13bfff640 1 -- 172.21.15.148:0/777458014 reap_dead start 2024-04-25T10:02:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa141f42640 1 -- 172.21.15.148:0/777458014 shutdown_connections 2024-04-25T10:02:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa141f42640 1 -- 172.21.15.148:0/777458014 >> 172.21.15.148:0/777458014 conn(0x7fa13c061600 msgr2=0x7fa13c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa141f42640 1 -- 172.21.15.148:0/777458014 shutdown_connections 2024-04-25T10:02:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.257+0000 7fa141f42640 1 -- 172.21.15.148:0/777458014 wait complete. 2024-04-25T10:02:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.258+0000 7fa141f42640 1 Processor -- start 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.258+0000 7fa141f42640 1 -- start start 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa141f42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1240039e0 0x7fa13c1551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa141f42640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c15ee20 0x7fa13c0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa141f42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 0x7fa13c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa141f42640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c15df40 con 0x7fa13c15a370 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa141f42640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0b6450 con 0x7fa13c15ee20 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa141f42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c15e0c0 con 0x7fa1240039e0 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 0x7fa13c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 0x7fa13c15a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34226/0 (socket says 172.21.15.148:34226) 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 -- 172.21.15.148:0/601137747 learned_addr learned my addr 172.21.15.148:0/601137747 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1240039e0 0x7fa13c1551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13affd640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c15ee20 0x7fa13c0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 -- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1240039e0 msgr2=0x7fa13c1551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1240039e0 0x7fa13c1551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 -- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c15ee20 msgr2=0x7fa13c0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c15ee20 0x7fa13c0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.259+0000 7fa13bfff640 1 -- 172.21.15.148:0/601137747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c155710 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.260+0000 7fa13bfff640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 0x7fa13c15a820 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa13c08c1a0 tx=0x7fa13400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.260+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13400ee90 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.260+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134018070 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.260+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c15f3c0 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.260+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134003bb0 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.260+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c15f980 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.261+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c073270 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.261+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1340050d0 con 0x7fa13c15a370 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.261+0000 7fa123fff640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12806b670 0x7fa12806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.262+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa134089f50 con 0x7fa13c15a370 2024-04-25T10:02:50.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.262+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12806b670 0x7fa12806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.262+0000 7fa13b7fe640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12806b670 0x7fa12806db30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa12c032000 tx=0x7fa12c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.266+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa134058c40 con 0x7fa13c15a370 2024-04-25T10:02:50.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.437+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa13c067be0 con 0x7fa13c15a370 2024-04-25T10:02:50.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.438+0000 7fa123fff640 1 -- 172.21.15.148:0/601137747 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16961 (secure 0 0 0) 0x7fa13c067be0 con 0x7fa13c15a370 2024-04-25T10:02:50.443 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:50.443 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":132,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:02:49.163160+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:50.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12806b670 msgr2=0x7fa12806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12806b670 0x7fa12806db30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa12c032000 tx=0x7fa12c009030 comp rx=0 tx=0).stop 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 msgr2=0x7fa13c15a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 0x7fa13c15a820 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa13c08c1a0 tx=0x7fa13400eb50 comp rx=0 tx=0).stop 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 shutdown_connections 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa12806b670 0x7fa12806db30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13c15a370 0x7fa13c15a820 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa13c15ee20 0x7fa13c0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 --2- 172.21.15.148:0/601137747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1240039e0 0x7fa13c1551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.440+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 >> 172.21.15.148:0/601137747 conn(0x7fa13c061600 msgr2=0x7fa13c15bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.441+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 shutdown_connections 2024-04-25T10:02:50.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.441+0000 7fa141f42640 1 -- 172.21.15.148:0/601137747 wait complete. 2024-04-25T10:02:50.457 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:50.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.552+0000 7ff603f70640 1 Processor -- start 2024-04-25T10:02:50.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff603f70640 1 -- start start 2024-04-25T10:02:50.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff603f70640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5fc0cc4f0 0x7ff5fc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff603f70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc15b980 0x7ff5fc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff603f70640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc072280 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff603f70640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc071270 con 0x7ff5fc0cc4f0 2024-04-25T10:02:50.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff603f70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0714c0 con 0x7ff5fc15b980 2024-04-25T10:02:50.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.553+0000 7ff601d0c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff5fc0cc4f0 0x7ff5fc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff60150b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff5fc0cdb20 0x7ff5fc15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56780/0 (socket says 172.21.15.148:56780) 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff60150b640 1 -- 172.21.15.148:0/2484537309 learned_addr learned my addr 172.21.15.148:0/2484537309 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff60250d640 1 --2- 172.21.15.148:0/2484537309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc15b980 0x7ff5fc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff601d0c640 1 -- 172.21.15.148:0/2484537309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc15b980 msgr2=0x7ff5fc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff601d0c640 1 --2- 172.21.15.148:0/2484537309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc15b980 0x7ff5fc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff601d0c640 1 -- 172.21.15.148:0/2484537309 >> v1:172.21.15.148:6789/0 conn(0x7ff5fc0cdb20 legacy=0x7ff5fc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff601d0c640 1 -- 172.21.15.148:0/2484537309 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0717a0 con 0x7ff5fc0cc4f0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.554+0000 7ff600d0a640 1 -- 172.21.15.148:0/2484537309 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314677302 0 0) 0x7ff5fc072280 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff601d0c640 1 --2- 172.21.15.148:0/2484537309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cc4f0 0x7ff5fc0cc8d0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097a0 tx=0x7ff5ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c564b2441657ae9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff600d0a640 1 -- 172.21.15.148:0/2484537309 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec037070 con 0x7ff5fc0cc4f0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff600d0a640 1 -- 172.21.15.148:0/2484537309 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec032440 con 0x7ff5fc0cc4f0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff600d0a640 1 -- 172.21.15.148:0/2484537309 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec032810 con 0x7ff5fc0cc4f0 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 -- 172.21.15.148:0/2484537309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cc4f0 msgr2=0x7ff5fc0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 --2- 172.21.15.148:0/2484537309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cc4f0 0x7ff5fc0cc8d0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097a0 tx=0x7ff5ec02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:50.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 -- 172.21.15.148:0/2484537309 shutdown_connections 2024-04-25T10:02:50.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 --2- 172.21.15.148:0/2484537309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc15b980 0x7ff5fc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 --2- 172.21.15.148:0/2484537309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cc4f0 0x7ff5fc0cc8d0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 -- 172.21.15.148:0/2484537309 >> 172.21.15.148:0/2484537309 conn(0x7ff5fc06abe0 msgr2=0x7ff5fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 -- 172.21.15.148:0/2484537309 shutdown_connections 2024-04-25T10:02:50.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.555+0000 7ff603f70640 1 -- 172.21.15.148:0/2484537309 wait complete. 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.556+0000 7ff603f70640 1 Processor -- start 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.556+0000 7ff603f70640 1 -- start start 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff603f70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc0cc4f0 0x7ff5fc1483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff603f70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 0x7ff5fc0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff603f70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc15b980 0x7ff5fc0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff603f70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0ae500 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff603f70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc071d60 con 0x7ff5fc15b980 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff603f70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0ae680 con 0x7ff5fc0cc4f0 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 0x7ff5fc0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 0x7ff5fc0b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34240/0 (socket says 172.21.15.148:34240) 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 -- 172.21.15.148:0/3303787630 learned_addr learned my addr 172.21.15.148:0/3303787630 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff601d0c640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc0cc4f0 0x7ff5fc1483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60250d640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc15b980 0x7ff5fc0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 -- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc0cc4f0 msgr2=0x7ff5fc1483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc0cc4f0 0x7ff5fc1483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 -- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc15b980 msgr2=0x7ff5fc0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.557+0000 7ff60150b640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc15b980 0x7ff5fc0b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff60150b640 1 -- 172.21.15.148:0/3303787630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc150720 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff60150b640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 0x7ff5fc0b0b00 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c4d0 tx=0x7ff5f800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8017070 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f8005d00 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc0afd40 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80072b0 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.558+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0b0300 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.559+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc073a00 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.559+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5f801a070 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.560+0000 7ff5f27fc640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e006b500 0x7ff5e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.560+0000 7ff601d0c640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e006b500 0x7ff5e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.560+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5f80896a0 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.560+0000 7ff601d0c640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e006b500 0x7ff5e006d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec034000 tx=0x7ff5ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.564+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5f8058420 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.770+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff5fc15b030 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.832+0000 7ff5f27fc640 1 -- 172.21.15.148:0/3303787630 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff5f8057d20 con 0x7ff5fc0cdb20 2024-04-25T10:02:50.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e006b500 msgr2=0x7ff5e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e006b500 0x7ff5e006d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec034000 tx=0x7ff5ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:50.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 msgr2=0x7ff5fc0b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 0x7ff5fc0b0b00 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c4d0 tx=0x7ff5f800c990 comp rx=0 tx=0).stop 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 shutdown_connections 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e006b500 0x7ff5e006d9c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5fc15b980 0x7ff5fc0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff5fc0cdb20 0x7ff5fc0b0b00 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 --2- 172.21.15.148:0/3303787630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5fc0cc4f0 0x7ff5fc1483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 >> 172.21.15.148:0/3303787630 conn(0x7ff5fc06abe0 msgr2=0x7ff5fc0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 shutdown_connections 2024-04-25T10:02:50.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.834+0000 7ff603f70640 1 -- 172.21.15.148:0/3303787630 wait complete. 2024-04-25T10:02:50.850 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:02:50.851 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.945+0000 7fdf14a09640 1 Processor -- start 2024-04-25T10:02:50.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.945+0000 7fdf14a09640 1 -- start start 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf14a09640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf100cd4c0 0x7fdf100cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf14a09640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf10150b90 0x7fdf10151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf14a09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1015e6e0 con 0x7fdf1015b1c0 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf14a09640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf10151560 con 0x7fdf10150b90 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf14a09640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1015e9c0 con 0x7fdf100cd4c0 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf10150b90 0x7fdf10151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdf10150b90 0x7fdf10151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34248/0 (socket says 172.21.15.148:34248) 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 -- 172.21.15.148:0/976489748 learned_addr learned my addr 172.21.15.148:0/976489748 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0dd9b640 1 --1- 172.21.15.148:0/976489748 >> v1:172.21.15.148:6789/0 conn(0x7fdf1015b1c0 0x7fdf1015dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56784/0 (socket says 172.21.15.148:56784) 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0e59c640 1 --2- 172.21.15.148:0/976489748 >> v2:172.21.15.188:3300/0 conn(0x7fdf100cd4c0 0x7fdf100cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 -- 172.21.15.148:0/976489748 >> v2:172.21.15.188:3300/0 conn(0x7fdf100cd4c0 msgr2=0x7fdf100cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 --2- 172.21.15.148:0/976489748 >> v2:172.21.15.188:3300/0 conn(0x7fdf100cd4c0 0x7fdf100cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 -- 172.21.15.148:0/976489748 >> v1:172.21.15.148:6789/0 conn(0x7fdf1015b1c0 legacy=0x7fdf1015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.946+0000 7fdf0ed9d640 1 -- 172.21.15.148:0/976489748 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1015a9c0 con 0x7fdf10150b90 2024-04-25T10:02:50.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.947+0000 7fdf0ed9d640 1 --2- 172.21.15.148:0/976489748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf10151020 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fdf1006b8e0 tx=0x7fdf0802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de3448d8fc6ebc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.947+0000 7fdf0d59a640 1 -- 172.21.15.148:0/976489748 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08036070 con 0x7fdf10150b90 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.947+0000 7fdf0d59a640 1 -- 172.21.15.148:0/976489748 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf08031600 con 0x7fdf10150b90 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.947+0000 7fdf0d59a640 1 -- 172.21.15.148:0/976489748 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080319d0 con 0x7fdf10150b90 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 -- 172.21.15.148:0/976489748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 msgr2=0x7fdf10151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 --2- 172.21.15.148:0/976489748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf10151020 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fdf1006b8e0 tx=0x7fdf0802a630 comp rx=0 tx=0).stop 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 -- 172.21.15.148:0/976489748 shutdown_connections 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 --2- 172.21.15.148:0/976489748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf10151020 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 --2- 172.21.15.148:0/976489748 >> v2:172.21.15.188:3300/0 conn(0x7fdf100cd4c0 0x7fdf100cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 -- 172.21.15.148:0/976489748 >> 172.21.15.148:0/976489748 conn(0x7fdf1006abe0 msgr2=0x7fdf1006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 -- 172.21.15.148:0/976489748 shutdown_connections 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.948+0000 7fdf14a09640 1 -- 172.21.15.148:0/976489748 wait complete. 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.949+0000 7fdf14a09640 1 Processor -- start 2024-04-25T10:02:50.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.949+0000 7fdf14a09640 1 -- start start 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf100cd4c0 0x7fdf1013b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf1013f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1015b1c0 0x7fdf1015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1015e370 con 0x7fdf10150b90 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1015a670 con 0x7fdf100cd4c0 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1006eca0 con 0x7fdf1015b1c0 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf1013f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf1013f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34262/0 (socket says 172.21.15.148:34262) 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 -- 172.21.15.148:0/3594661357 learned_addr learned my addr 172.21.15.148:0/3594661357 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 -- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1015b1c0 msgr2=0x7fdf1015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0ed9d640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1015b1c0 0x7fdf1015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1015b1c0 0x7fdf1015b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0e59c640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf100cd4c0 0x7fdf1013b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 -- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf100cd4c0 msgr2=0x7fdf1013b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf100cd4c0 0x7fdf1013b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 -- 172.21.15.148:0/3594661357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10140220 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf0dd9b640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf1013f7d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c570 tx=0x7fdf0400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04018070 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040074e0 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.950+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf10140500 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.951+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004a20 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.951+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf10140ac0 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.951+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf10073a00 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.952+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf0401b070 con 0x7fdf10150b90 2024-04-25T10:02:50.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.952+0000 7fdefeffd640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdef006b4d0 0x7fdef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.952+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdf040896f0 con 0x7fdf10150b90 2024-04-25T10:02:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.952+0000 7fdf0e59c640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdef006b4d0 0x7fdef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.953+0000 7fdf0e59c640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdef006b4d0 0x7fdef006d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdf00006fd0 tx=0x7fdf00008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:50.956+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf04058470 con 0x7fdf10150b90 2024-04-25T10:02:50.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:02:51.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.164+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdf10157df0 con 0x7fdf10150b90 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.846+0000 7fdefeffd640 1 -- 172.21.15.148:0/3594661357 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf04057d70 con 0x7fdf10150b90 2024-04-25T10:02:51.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdef006b4d0 msgr2=0x7fdef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdef006b4d0 0x7fdef006d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fdf00006fd0 tx=0x7fdf00008040 comp rx=0 tx=0).stop 2024-04-25T10:02:51.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 msgr2=0x7fdf1013f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf1013f7d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c570 tx=0x7fdf0400ca30 comp rx=0 tx=0).stop 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 shutdown_connections 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdef006b4d0 0x7fdef006d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1015b1c0 0x7fdf1015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10150b90 0x7fdf1013f7d0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 --2- 172.21.15.148:0/3594661357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf100cd4c0 0x7fdf1013b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 >> 172.21.15.148:0/3594661357 conn(0x7fdf1006abe0 msgr2=0x7fdf10158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.848+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 shutdown_connections 2024-04-25T10:02:51.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.849+0000 7fdf14a09640 1 -- 172.21.15.148:0/3594661357 wait complete. 2024-04-25T10:02:51.866 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:51.866 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:51.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.962+0000 7f1e00de7640 1 Processor -- start 2024-04-25T10:02:51.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.962+0000 7f1e00de7640 1 -- start start 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1e00de7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1dfc15ef00 0x7f1dfc0d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1e00de7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1dfc0d1ca0 0x7f1dfc0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1e00de7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0c7970 con 0x7f1dfc0c7140 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1e00de7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc0d2110 con 0x7f1dfc0d1ca0 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1e00de7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc15f2e0 con 0x7f1dfc15ef00 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1dfc0d1ca0 0x7f1dfc0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1dfc0d1ca0 0x7f1dfc0c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34270/0 (socket says 172.21.15.148:34270) 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 -- 172.21.15.148:0/906577393 learned_addr learned my addr 172.21.15.148:0/906577393 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1dfad9d640 1 --1- 172.21.15.148:0/906577393 >> v1:172.21.15.148:6789/0 conn(0x7f1dfc0c7140 0x7f1dfc15f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56796/0 (socket says 172.21.15.148:56796) 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1dfa59c640 1 --2- 172.21.15.148:0/906577393 >> v2:172.21.15.188:3300/0 conn(0x7f1dfc15ef00 0x7f1dfc0d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 -- 172.21.15.148:0/906577393 >> v2:172.21.15.188:3300/0 conn(0x7f1dfc15ef00 msgr2=0x7f1dfc0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 --2- 172.21.15.148:0/906577393 >> v2:172.21.15.188:3300/0 conn(0x7f1dfc15ef00 0x7f1dfc0d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 -- 172.21.15.148:0/906577393 >> v1:172.21.15.148:6789/0 conn(0x7f1dfc0c7140 legacy=0x7f1dfc15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df9d9b640 1 -- 172.21.15.148:0/906577393 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0c7c50 con 0x7f1dfc0d1ca0 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1df959a640 1 -- 172.21.15.148:0/906577393 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86740678 0 0) 0x7f1dfc0c7970 con 0x7f1dfc0c7140 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.963+0000 7f1dfa59c640 1 --2- 172.21.15.148:0/906577393 >> v2:172.21.15.188:3300/0 conn(0x7f1dfc15ef00 0x7f1dfc0d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.964+0000 7f1df9d9b640 1 --2- 172.21.15.148:0/906577393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc0c6c00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0d2470 tx=0x7f1df002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8411bc63408970a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.964+0000 7f1df959a640 1 -- 172.21.15.148:0/906577393 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037070 con 0x7f1dfc0d1ca0 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.964+0000 7f1df959a640 1 -- 172.21.15.148:0/906577393 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df002ae50 con 0x7f1dfc0d1ca0 2024-04-25T10:02:51.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.964+0000 7f1df959a640 1 -- 172.21.15.148:0/906577393 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0032860 con 0x7f1dfc0d1ca0 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.964+0000 7f1e00de7640 1 -- 172.21.15.148:0/906577393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0d1ca0 msgr2=0x7f1dfc0c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 --2- 172.21.15.148:0/906577393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc0c6c00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0d2470 tx=0x7f1df002a300 comp rx=0 tx=0).stop 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 -- 172.21.15.148:0/906577393 shutdown_connections 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 --2- 172.21.15.148:0/906577393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc0c6c00 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 --2- 172.21.15.148:0/906577393 >> v2:172.21.15.188:3300/0 conn(0x7f1dfc15ef00 0x7f1dfc0d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 -- 172.21.15.148:0/906577393 >> 172.21.15.148:0/906577393 conn(0x7f1dfc06f5d0 msgr2=0x7f1dfc06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 -- 172.21.15.148:0/906577393 shutdown_connections 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.965+0000 7f1e00de7640 1 -- 172.21.15.148:0/906577393 wait complete. 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.966+0000 7f1e00de7640 1 Processor -- start 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.966+0000 7f1e00de7640 1 -- start start 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 0x7f1dfc1f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc1eea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dfc15ef00 0x7f1dfc0cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c84f0 con 0x7f1dfc0c7140 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c8210 con 0x7f1dfc0d1ca0 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c7590 con 0x7f1dfc15ef00 2024-04-25T10:02:51.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 0x7f1dfc1f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 0x7f1dfc1f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34286/0 (socket says 172.21.15.148:34286) 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 -- 172.21.15.148:0/2369537675 learned_addr learned my addr 172.21.15.148:0/2369537675 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfad9d640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dfc15ef00 0x7f1dfc0cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1df9d9b640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc1eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 -- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dfc15ef00 msgr2=0x7f1dfc0cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dfc15ef00 0x7f1dfc0cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 -- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dfc0d1ca0 msgr2=0x7f1dfc1eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc1eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 -- 172.21.15.148:0/2369537675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc1eef60 con 0x7f1dfc0c7140 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfad9d640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dfc15ef00 0x7f1dfc0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:51.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1dfa59c640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 0x7f1dfc1f1fd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00c820 tx=0x7f1dec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec015070 con 0x7f1dfc0c7140 2024-04-25T10:02:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec007a30 con 0x7f1dfc0c7140 2024-04-25T10:02:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec007e00 con 0x7f1dfc0c7140 2024-04-25T10:02:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc0c8f20 con 0x7f1dfc0c7140 2024-04-25T10:02:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.967+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc0c94e0 con 0x7f1dfc0c7140 2024-04-25T10:02:51.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.968+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc073990 con 0x7f1dfc0c7140 2024-04-25T10:02:51.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.969+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1dec018070 con 0x7f1dfc0c7140 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.969+0000 7f1deaffd640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ddc06b4d0 0x7f1ddc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.969+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1dec08a2d0 con 0x7f1dfc0c7140 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.969+0000 7f1df9d9b640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ddc06b4d0 0x7f1ddc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.970+0000 7f1df9d9b640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ddc06b4d0 0x7f1ddc06d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1df0034000 tx=0x7f1df002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:51.973+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1dec058fc0 con 0x7f1dfc0c7140 2024-04-25T10:02:52.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.182+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dfc1554a0 con 0x7f1dfc0c7140 2024-04-25T10:02:52.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.182+0000 7f1deaffd640 1 -- 172.21.15.148:0/2369537675 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1dec0588c0 con 0x7f1dfc0c7140 2024-04-25T10:02:52.187 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:52.187 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:52.188 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:02:52.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ddc06b4d0 msgr2=0x7f1ddc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ddc06b4d0 0x7f1ddc06d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1df0034000 tx=0x7f1df002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:52.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 msgr2=0x7f1dfc1f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 0x7f1dfc1f1fd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00c820 tx=0x7f1dec00cc20 comp rx=0 tx=0).stop 2024-04-25T10:02:52.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 shutdown_connections 2024-04-25T10:02:52.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1ddc06b4d0 0x7f1ddc06d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1dfc15ef00 0x7f1dfc0cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1dfc0d1ca0 0x7f1dfc1eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 --2- 172.21.15.148:0/2369537675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dfc0c7140 0x7f1dfc1f1fd0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 >> 172.21.15.148:0/2369537675 conn(0x7f1dfc06f5d0 msgr2=0x7f1dfc15a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 shutdown_connections 2024-04-25T10:02:52.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.184+0000 7f1e00de7640 1 -- 172.21.15.148:0/2369537675 wait complete. 2024-04-25T10:02:52.200 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:52.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.293+0000 7f2973534640 1 Processor -- start 2024-04-25T10:02:52.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.293+0000 7f2973534640 1 -- start start 2024-04-25T10:02:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f2973534640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f296c15dc90 0x7f296c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f2973534640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c0cc8e0 0x7f296c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f2973534640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c15f390 con 0x7f296c0d1810 2024-04-25T10:02:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f2973534640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c0cce50 con 0x7f296c15dc90 2024-04-25T10:02:52.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f2973534640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c161af0 con 0x7f296c0cc8e0 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f296c15dc90 0x7f296c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f296c15dc90 0x7f296c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34302/0 (socket says 172.21.15.148:34302) 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 -- 172.21.15.148:0/2316799654 learned_addr learned my addr 172.21.15.148:0/2316799654 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 -- 172.21.15.148:0/2316799654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c0cc8e0 msgr2=0x7f296c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 --2- 172.21.15.148:0/2316799654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c0cc8e0 0x7f296c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 -- 172.21.15.148:0/2316799654 >> v1:172.21.15.188:6789/0 conn(0x7f296c0d1810 legacy=0x7f296c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.294+0000 7f29712d0640 1 -- 172.21.15.148:0/2316799654 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c161dd0 con 0x7f296c15dc90 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f29712d0640 1 --2- 172.21.15.148:0/2316799654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c15dc90 0x7f296c1625d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2968009870 tx=0x7f296802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c0999704477038 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2963fff640 1 -- 172.21.15.148:0/2316799654 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968037070 con 0x7f296c15dc90 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2963fff640 1 -- 172.21.15.148:0/2316799654 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2968032440 con 0x7f296c15dc90 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2963fff640 1 -- 172.21.15.148:0/2316799654 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968032810 con 0x7f296c15dc90 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2973534640 1 -- 172.21.15.148:0/2316799654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c15dc90 msgr2=0x7f296c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2973534640 1 --2- 172.21.15.148:0/2316799654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c15dc90 0x7f296c1625d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2968009870 tx=0x7f296802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:52.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2973534640 1 -- 172.21.15.148:0/2316799654 shutdown_connections 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2973534640 1 --2- 172.21.15.148:0/2316799654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c0cc8e0 0x7f296c15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2973534640 1 --2- 172.21.15.148:0/2316799654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c15dc90 0x7f296c1625d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.295+0000 7f2973534640 1 -- 172.21.15.148:0/2316799654 >> 172.21.15.148:0/2316799654 conn(0x7f296c06f590 msgr2=0x7f296c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.296+0000 7f2973534640 1 -- 172.21.15.148:0/2316799654 shutdown_connections 2024-04-25T10:02:52.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.296+0000 7f2973534640 1 -- 172.21.15.148:0/2316799654 wait complete. 2024-04-25T10:02:52.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 Processor -- start 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 -- start start 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c0cc8e0 0x7f296c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 0x7f296c1f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c15dc90 0x7f296c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c15e070 con 0x7f296c0d1810 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0cd130 con 0x7f296c0cc8e0 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2973534640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c15e1f0 con 0x7f296c15dc90 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2970acf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 0x7f296c1f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.297+0000 7f2970acf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 0x7f296c1f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34318/0 (socket says 172.21.15.148:34318) 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 -- 172.21.15.148:0/1477322009 learned_addr learned my addr 172.21.15.148:0/1477322009 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f29712d0640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c0cc8e0 0x7f296c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2971ad1640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c15dc90 0x7f296c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 -- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c15dc90 msgr2=0x7f296c0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c15dc90 0x7f296c0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 -- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c0cc8e0 msgr2=0x7f296c0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c0cc8e0 0x7f296c0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 -- 172.21.15.148:0/1477322009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0cb710 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2970acf640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 0x7f296c1f7650 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f295c00b4a0 tx=0x7f295c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c014070 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c010610 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c010a00 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c152570 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.298+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c0cb9f0 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.299+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c073990 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.299+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f295c017070 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.300+0000 7f2961ffb640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f295006b4d0 0x7f295006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.300+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f295c089d50 con 0x7f296c0d1810 2024-04-25T10:02:52.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.300+0000 7f29712d0640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f295006b4d0 0x7f295006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.300+0000 7f29712d0640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f295006b4d0 0x7f295006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2968034000 tx=0x7f296802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.304+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f295c08d030 con 0x7f296c0d1810 2024-04-25T10:02:52.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.474+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f296c0cf6b0 con 0x7f296c0d1810 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.475+0000 7f2961ffb640 1 -- 172.21.15.148:0/1477322009 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16948 (secure 0 0 0) 0x7f295c058ad0 con 0x7f296c0d1810 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":133,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:02:50.821131+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f295006b4d0 msgr2=0x7f295006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f295006b4d0 0x7f295006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2968034000 tx=0x7f296802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 msgr2=0x7f296c1f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 0x7f296c1f7650 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f295c00b4a0 tx=0x7f295c00b960 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 shutdown_connections 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f295006b4d0 0x7f295006d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f296c15dc90 0x7f296c0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f296c0d1810 0x7f296c1f7650 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 --2- 172.21.15.148:0/1477322009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c0cc8e0 0x7f296c0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.477+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 >> 172.21.15.148:0/1477322009 conn(0x7f296c06f590 msgr2=0x7f296c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.478+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 shutdown_connections 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.478+0000 7f2973534640 1 -- 172.21.15.148:0/1477322009 wait complete. 2024-04-25T10:02:52.494 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:52.494 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.587+0000 7f2913330640 1 Processor -- start 2024-04-25T10:02:52.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.587+0000 7f2913330640 1 -- start start 2024-04-25T10:02:52.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f2913330640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f290c15dc00 0x7f290c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f2913330640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c15dfe0 con 0x7f290c0d1dd0 2024-04-25T10:02:52.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f2913330640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0d00c0 con 0x7f290c0ceb20 2024-04-25T10:02:52.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f2913330640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290c0d03a0 con 0x7f290c15dc00 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f29118cd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f290c0d1dd0 0x7f290c0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56798/0 (socket says 172.21.15.148:56798) 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f29118cd640 1 -- 172.21.15.148:0/2418737054 learned_addr learned my addr 172.21.15.148:0/2418737054 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f29110cc640 1 --2- 172.21.15.148:0/2418737054 >> v2:172.21.15.188:3300/0 conn(0x7f290c15dc00 0x7f290c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678731602 0 0) 0x7f2904001110 con 0x7f290c0d1dd0 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.588+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f8003520 con 0x7f290c0d1dd0 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888636162 0 0) 0x7f28f8003520 con 0x7f290c0d1dd0 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 >> v2:172.21.15.188:3300/0 conn(0x7f290c15dc00 msgr2=0x7f290c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 --2- 172.21.15.148:0/2418737054 >> v2:172.21.15.188:3300/0 conn(0x7f290c15dc00 0x7f290c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 >> v1:172.21.15.188:6789/0 conn(0x7f290c0ceb20 legacy=0x7f290c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0cc440 con 0x7f290c0d1dd0 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f2904001840 con 0x7f290c0d1dd0 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 >> v1:172.21.15.148:6789/0 conn(0x7f290c0d1dd0 legacy=0x7f290c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80039b0 0x7f28f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f28f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80051c0 0x7f28f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0d03a0 con 0x7f28f8004800 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f800c060 con 0x7f28f80039b0 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f800c260 con 0x7f28f80051c0 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29108cb640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f28f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29110cc640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80039b0 0x7f28f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29118cd640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80051c0 0x7f28f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29108cb640 1 -- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80051c0 msgr2=0x7f28f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29108cb640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f80051c0 0x7f28f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29108cb640 1 -- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80039b0 msgr2=0x7f28f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29108cb640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f80039b0 0x7f28f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29108cb640 1 -- 172.21.15.148:0/2418737054 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8003520 con 0x7f28f8004800 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.589+0000 7f29118cd640 1 -- 172.21.15.148:0/2418737054 reap_dead start 2024-04-25T10:02:52.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.590+0000 7f29108cb640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f28f8004c80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f28fc001750 tx=0x7f28f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fcca6b96aae126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.590+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc001370 con 0x7f28f8004800 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.590+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc0052d0 con 0x7f28f8004800 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.590+0000 7f2903fff640 1 -- 172.21.15.148:0/2418737054 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02abc0 con 0x7f28f8004800 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.590+0000 7f2913330640 1 -- 172.21.15.148:0/2418737054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 msgr2=0x7f28f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.590+0000 7f2913330640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f28f8004c80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f28fc001750 tx=0x7f28f8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.591+0000 7f2913330640 1 -- 172.21.15.148:0/2418737054 shutdown_connections 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.591+0000 7f2913330640 1 --2- 172.21.15.148:0/2418737054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f28f8004c80 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.591+0000 7f2913330640 1 -- 172.21.15.148:0/2418737054 >> 172.21.15.148:0/2418737054 conn(0x7f290c06f5d0 msgr2=0x7f290c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.591+0000 7f2913330640 1 -- 172.21.15.148:0/2418737054 shutdown_connections 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.591+0000 7f2913330640 1 -- 172.21.15.148:0/2418737054 wait complete. 2024-04-25T10:02:52.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.592+0000 7f2913330640 1 Processor -- start 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.592+0000 7f2913330640 1 -- start start 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.592+0000 7f2913330640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f290c0b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.592+0000 7f2913330640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f290c1571f0 0x7f290c157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2913330640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f290c15c670 0x7f290c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2913330640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0cfd90 con 0x7f28f8004800 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2913330640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0cc130 con 0x7f290c15c670 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2913330640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0cdd70 con 0x7f290c1571f0 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f290c0b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f290c0b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34330/0 (socket says 172.21.15.148:34330) 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 -- 172.21.15.148:0/1462601588 learned_addr learned my addr 172.21.15.148:0/1462601588 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 -- 172.21.15.148:0/1462601588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f290c1571f0 msgr2=0x7f290c157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f290c1571f0 0x7f290c157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 -- 172.21.15.148:0/1462601588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f290c15c670 msgr2=0x7f290c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f290c15c670 0x7f290c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 -- 172.21.15.148:0/1462601588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c157c60 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f29110cc640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f290c0b0590 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2908009f90 tx=0x7f290800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908015430 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29080155d0 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080159a0 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c157f40 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.593+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c158500 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.594+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c073990 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.595+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2908015b40 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.595+0000 7f2901ffb640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f28f006b500 0x7f28f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.595+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f290808b550 con 0x7f28f8004800 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.595+0000 7f29108cb640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f28f006b500 0x7f28f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.596+0000 7f29108cb640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f28f006b500 0x7f28f006d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f28fc032000 tx=0x7f28fc02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.600+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2908019030 con 0x7f28f8004800 2024-04-25T10:02:52.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.766+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f290c0c66c0 con 0x7f28f006b500 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.772+0000 7f2901ffb640 1 -- 172.21.15.148:0/1462601588 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252414 (secure 0 0 0) 0x7f290c0c66c0 con 0x7f28f006b500 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:52.781 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f28f006b500 msgr2=0x7f28f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f28f006b500 0x7f28f006d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f28fc032000 tx=0x7f28fc02c000 comp rx=0 tx=0).stop 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 msgr2=0x7f290c0b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f290c0b0590 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2908009f90 tx=0x7f290800f580 comp rx=0 tx=0).stop 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 shutdown_connections 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f28f006b500 0x7f28f006d9c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f290c15c670 0x7f290c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f290c1571f0 0x7f290c157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 --2- 172.21.15.148:0/1462601588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28f8004800 0x7f290c0b0590 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 >> 172.21.15.148:0/1462601588 conn(0x7f290c06f5d0 msgr2=0x7f290c0ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 shutdown_connections 2024-04-25T10:02:52.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.775+0000 7f2913330640 1 -- 172.21.15.148:0/1462601588 wait complete. 2024-04-25T10:02:52.791 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-04-25T10:02:51.124608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389780,"kb_used_data":13976,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754584940,"statfs":{"total":773094113280,"available":772694978560,"internally_reserved":0,"allocated":14311424,"data_stored":7275981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000613"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846643+0000","last_change":"2024-04-25T10:02:49.844722+0000","last_active":"2024-04-25T10:02:49.846643+0000","last_peered":"2024-04-25T10:02:49.846643+0000","last_clean":"2024-04-25T10:02:49.846643+0000","last_became_active":"2024-04-25T10:02:49.844197+0000","last_became_peered":"2024-04-25T10:02:49.844197+0000","last_unstale":"2024-04-25T10:02:49.846643+0000","last_undegraded":"2024-04-25T10:02:49.846643+0000","last_fullsized":"2024-04-25T10:02:49.846643+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:25.017866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835870+0000","last_change":"2024-04-25T10:02:49.833348+0000","last_active":"2024-04-25T10:02:49.835870+0000","last_peered":"2024-04-25T10:02:49.835870+0000","last_clean":"2024-04-25T10:02:49.835870+0000","last_became_active":"2024-04-25T10:02:49.833081+0000","last_became_peered":"2024-04-25T10:02:49.833081+0000","last_unstale":"2024-04-25T10:02:49.835870+0000","last_undegraded":"2024-04-25T10:02:49.835870+0000","last_fullsized":"2024-04-25T10:02:49.835870+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:16.412752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844701+0000","last_change":"2024-04-25T10:02:49.843028+0000","last_active":"2024-04-25T10:02:49.844701+0000","last_peered":"2024-04-25T10:02:49.844701+0000","last_clean":"2024-04-25T10:02:49.844701+0000","last_became_active":"2024-04-25T10:02:49.838784+0000","last_became_peered":"2024-04-25T10:02:49.838784+0000","last_unstale":"2024-04-25T10:02:49.844701+0000","last_undegraded":"2024-04-25T10:02:49.844701+0000","last_fullsized":"2024-04-25T10:02:49.844701+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:19.597490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843642+0000","last_change":"2024-04-25T10:02:49.843054+0000","last_active":"2024-04-25T10:02:49.843642+0000","last_peered":"2024-04-25T10:02:49.843642+0000","last_clean":"2024-04-25T10:02:49.843642+0000","last_became_active":"2024-04-25T10:02:49.842770+0000","last_became_peered":"2024-04-25T10:02:49.842770+0000","last_unstale":"2024-04-25T10:02:49.843642+0000","last_undegraded":"2024-04-25T10:02:49.843642+0000","last_fullsized":"2024-04-25T10:02:49.843642+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.245807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840012+0000","last_change":"2024-04-25T10:02:49.836411+0000","last_active":"2024-04-25T10:02:49.840012+0000","last_peered":"2024-04-25T10:02:49.840012+0000","last_clean":"2024-04-25T10:02:49.840012+0000","last_became_active":"2024-04-25T10:02:49.836164+0000","last_became_peered":"2024-04-25T10:02:49.836164+0000","last_unstale":"2024-04-25T10:02:49.840012+0000","last_undegraded":"2024-04-25T10:02:49.840012+0000","last_fullsized":"2024-04-25T10:02:49.840012+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:48.746929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843984+0000","last_change":"2024-04-25T10:02:49.841469+0000","last_active":"2024-04-25T10:02:49.843984+0000","last_peered":"2024-04-25T10:02:49.843984+0000","last_clean":"2024-04-25T10:02:49.843984+0000","last_became_active":"2024-04-25T10:02:49.841214+0000","last_became_peered":"2024-04-25T10:02:49.841214+0000","last_unstale":"2024-04-25T10:02:49.843984+0000","last_undegraded":"2024-04-25T10:02:49.843984+0000","last_fullsized":"2024-04-25T10:02:49.843984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:01.155606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.848390+0000","last_change":"2024-04-25T10:02:49.846690+0000","last_active":"2024-04-25T10:02:49.848390+0000","last_peered":"2024-04-25T10:02:49.848390+0000","last_clean":"2024-04-25T10:02:49.848390+0000","last_became_active":"2024-04-25T10:02:49.845901+0000","last_became_peered":"2024-04-25T10:02:49.845901+0000","last_unstale":"2024-04-25T10:02:49.848390+0000","last_undegraded":"2024-04-25T10:02:49.848390+0000","last_fullsized":"2024-04-25T10:02:49.848390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:38.442612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839175+0000","last_change":"2024-04-25T10:02:49.836096+0000","last_active":"2024-04-25T10:02:49.839175+0000","last_peered":"2024-04-25T10:02:49.839175+0000","last_clean":"2024-04-25T10:02:49.839175+0000","last_became_active":"2024-04-25T10:02:49.835826+0000","last_became_peered":"2024-04-25T10:02:49.835826+0000","last_unstale":"2024-04-25T10:02:49.839175+0000","last_undegraded":"2024-04-25T10:02:49.839175+0000","last_fullsized":"2024-04-25T10:02:49.839175+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.350657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842646+0000","last_change":"2024-04-25T10:02:49.833852+0000","last_active":"2024-04-25T10:02:49.842646+0000","last_peered":"2024-04-25T10:02:49.842646+0000","last_clean":"2024-04-25T10:02:49.842646+0000","last_became_active":"2024-04-25T10:02:49.833417+0000","last_became_peered":"2024-04-25T10:02:49.833417+0000","last_unstale":"2024-04-25T10:02:49.842646+0000","last_undegraded":"2024-04-25T10:02:49.842646+0000","last_fullsized":"2024-04-25T10:02:49.842646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:13.515109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834984+0000","last_change":"2024-04-25T10:02:49.830643+0000","last_active":"2024-04-25T10:02:49.834984+0000","last_peered":"2024-04-25T10:02:49.834984+0000","last_clean":"2024-04-25T10:02:49.834984+0000","last_became_active":"2024-04-25T10:02:49.830367+0000","last_became_peered":"2024-04-25T10:02:49.830367+0000","last_unstale":"2024-04-25T10:02:49.834984+0000","last_undegraded":"2024-04-25T10:02:49.834984+0000","last_fullsized":"2024-04-25T10:02:49.834984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:41.743497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840636+0000","last_change":"2024-04-25T10:02:49.839883+0000","last_active":"2024-04-25T10:02:49.840636+0000","last_peered":"2024-04-25T10:02:49.840636+0000","last_clean":"2024-04-25T10:02:49.840636+0000","last_became_active":"2024-04-25T10:02:49.839575+0000","last_became_peered":"2024-04-25T10:02:49.839575+0000","last_unstale":"2024-04-25T10:02:49.840636+0000","last_undegraded":"2024-04-25T10:02:49.840636+0000","last_fullsized":"2024-04-25T10:02:49.840636+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.889150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839585+0000","last_change":"2024-04-25T10:02:49.837142+0000","last_active":"2024-04-25T10:02:49.839585+0000","last_peered":"2024-04-25T10:02:49.839585+0000","last_clean":"2024-04-25T10:02:49.839585+0000","last_became_active":"2024-04-25T10:02:49.836832+0000","last_became_peered":"2024-04-25T10:02:49.836832+0000","last_unstale":"2024-04-25T10:02:49.839585+0000","last_undegraded":"2024-04-25T10:02:49.839585+0000","last_fullsized":"2024-04-25T10:02:49.839585+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:22.757440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844020+0000","last_change":"2024-04-25T10:02:49.842819+0000","last_active":"2024-04-25T10:02:49.844020+0000","last_peered":"2024-04-25T10:02:49.844020+0000","last_clean":"2024-04-25T10:02:49.844020+0000","last_became_active":"2024-04-25T10:02:49.842510+0000","last_became_peered":"2024-04-25T10:02:49.842510+0000","last_unstale":"2024-04-25T10:02:49.844020+0000","last_undegraded":"2024-04-25T10:02:49.844020+0000","last_fullsized":"2024-04-25T10:02:49.844020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:41.207015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833809+0000","last_change":"2024-04-25T10:02:49.833102+0000","last_active":"2024-04-25T10:02:49.833809+0000","last_peered":"2024-04-25T10:02:49.833809+0000","last_clean":"2024-04-25T10:02:49.833809+0000","last_became_active":"2024-04-25T10:02:49.832831+0000","last_became_peered":"2024-04-25T10:02:49.832831+0000","last_unstale":"2024-04-25T10:02:49.833809+0000","last_undegraded":"2024-04-25T10:02:49.833809+0000","last_fullsized":"2024-04-25T10:02:49.833809+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:38.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.832520+0000","last_change":"2024-04-25T10:02:49.831636+0000","last_active":"2024-04-25T10:02:49.832520+0000","last_peered":"2024-04-25T10:02:49.832520+0000","last_clean":"2024-04-25T10:02:49.832520+0000","last_became_active":"2024-04-25T10:02:49.831381+0000","last_became_peered":"2024-04-25T10:02:49.831381+0000","last_unstale":"2024-04-25T10:02:49.832520+0000","last_undegraded":"2024-04-25T10:02:49.832520+0000","last_fullsized":"2024-04-25T10:02:49.832520+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:04.924245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842568+0000","last_change":"2024-04-25T10:02:49.836160+0000","last_active":"2024-04-25T10:02:49.842568+0000","last_peered":"2024-04-25T10:02:49.842568+0000","last_clean":"2024-04-25T10:02:49.842568+0000","last_became_active":"2024-04-25T10:02:49.835870+0000","last_became_peered":"2024-04-25T10:02:49.835870+0000","last_unstale":"2024-04-25T10:02:49.842568+0000","last_undegraded":"2024-04-25T10:02:49.842568+0000","last_fullsized":"2024-04-25T10:02:49.842568+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.350657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842668+0000","last_change":"2024-04-25T10:02:49.836609+0000","last_active":"2024-04-25T10:02:49.842668+0000","last_peered":"2024-04-25T10:02:49.842668+0000","last_clean":"2024-04-25T10:02:49.842668+0000","last_became_active":"2024-04-25T10:02:49.836304+0000","last_became_peered":"2024-04-25T10:02:49.836304+0000","last_unstale":"2024-04-25T10:02:49.842668+0000","last_undegraded":"2024-04-25T10:02:49.842668+0000","last_fullsized":"2024-04-25T10:02:49.842668+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:44.711338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843756+0000","last_change":"2024-04-25T10:02:49.835654+0000","last_active":"2024-04-25T10:02:49.843756+0000","last_peered":"2024-04-25T10:02:49.843756+0000","last_clean":"2024-04-25T10:02:49.843756+0000","last_became_active":"2024-04-25T10:02:49.835377+0000","last_became_peered":"2024-04-25T10:02:49.835377+0000","last_unstale":"2024-04-25T10:02:49.843756+0000","last_undegraded":"2024-04-25T10:02:49.843756+0000","last_fullsized":"2024-04-25T10:02:49.843756+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:35.960479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845975+0000","last_change":"2024-04-25T10:02:49.843381+0000","last_active":"2024-04-25T10:02:49.845975+0000","last_peered":"2024-04-25T10:02:49.845975+0000","last_clean":"2024-04-25T10:02:49.845975+0000","last_became_active":"2024-04-25T10:02:49.842321+0000","last_became_peered":"2024-04-25T10:02:49.842321+0000","last_unstale":"2024-04-25T10:02:49.845975+0000","last_undegraded":"2024-04-25T10:02:49.845975+0000","last_fullsized":"2024-04-25T10:02:49.845975+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:49.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842194+0000","last_change":"2024-04-25T10:02:49.841503+0000","last_active":"2024-04-25T10:02:49.842194+0000","last_peered":"2024-04-25T10:02:49.842194+0000","last_clean":"2024-04-25T10:02:49.842194+0000","last_became_active":"2024-04-25T10:02:49.841245+0000","last_became_peered":"2024-04-25T10:02:49.841245+0000","last_unstale":"2024-04-25T10:02:49.842194+0000","last_undegraded":"2024-04-25T10:02:49.842194+0000","last_fullsized":"2024-04-25T10:02:49.842194+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:45.889300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835911+0000","last_change":"2024-04-25T10:02:49.832441+0000","last_active":"2024-04-25T10:02:49.835911+0000","last_peered":"2024-04-25T10:02:49.835911+0000","last_clean":"2024-04-25T10:02:49.835911+0000","last_became_active":"2024-04-25T10:02:49.832161+0000","last_became_peered":"2024-04-25T10:02:49.832161+0000","last_unstale":"2024-04-25T10:02:49.835911+0000","last_undegraded":"2024-04-25T10:02:49.835911+0000","last_fullsized":"2024-04-25T10:02:49.835911+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:07.285630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839246+0000","last_change":"2024-04-25T10:02:49.837613+0000","last_active":"2024-04-25T10:02:49.839246+0000","last_peered":"2024-04-25T10:02:49.839246+0000","last_clean":"2024-04-25T10:02:49.839246+0000","last_became_active":"2024-04-25T10:02:49.837332+0000","last_became_peered":"2024-04-25T10:02:49.837332+0000","last_unstale":"2024-04-25T10:02:49.839246+0000","last_undegraded":"2024-04-25T10:02:49.839246+0000","last_fullsized":"2024-04-25T10:02:49.839246+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:12.143545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835936+0000","last_change":"2024-04-25T10:02:49.835003+0000","last_active":"2024-04-25T10:02:49.835936+0000","last_peered":"2024-04-25T10:02:49.835936+0000","last_clean":"2024-04-25T10:02:49.835936+0000","last_became_active":"2024-04-25T10:02:49.834748+0000","last_became_peered":"2024-04-25T10:02:49.834748+0000","last_unstale":"2024-04-25T10:02:49.835936+0000","last_undegraded":"2024-04-25T10:02:49.835936+0000","last_fullsized":"2024-04-25T10:02:49.835936+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:41.207015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840053+0000","last_change":"2024-04-25T10:02:49.832715+0000","last_active":"2024-04-25T10:02:49.840053+0000","last_peered":"2024-04-25T10:02:49.840053+0000","last_clean":"2024-04-25T10:02:49.840053+0000","last_became_active":"2024-04-25T10:02:49.832482+0000","last_became_peered":"2024-04-25T10:02:49.832482+0000","last_unstale":"2024-04-25T10:02:49.840053+0000","last_undegraded":"2024-04-25T10:02:49.840053+0000","last_fullsized":"2024-04-25T10:02:49.840053+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:06.208277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844562+0000","last_change":"2024-04-25T10:02:49.838816+0000","last_active":"2024-04-25T10:02:49.844562+0000","last_peered":"2024-04-25T10:02:49.844562+0000","last_clean":"2024-04-25T10:02:49.844562+0000","last_became_active":"2024-04-25T10:02:49.838488+0000","last_became_peered":"2024-04-25T10:02:49.838488+0000","last_unstale":"2024-04-25T10:02:49.844562+0000","last_undegraded":"2024-04-25T10:02:49.844562+0000","last_fullsized":"2024-04-25T10:02:49.844562+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:52.346670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845122+0000","last_change":"2024-04-25T10:02:49.843312+0000","last_active":"2024-04-25T10:02:49.845122+0000","last_peered":"2024-04-25T10:02:49.845122+0000","last_clean":"2024-04-25T10:02:49.845122+0000","last_became_active":"2024-04-25T10:02:49.843041+0000","last_became_peered":"2024-04-25T10:02:49.843041+0000","last_unstale":"2024-04-25T10:02:49.845122+0000","last_undegraded":"2024-04-25T10:02:49.845122+0000","last_fullsized":"2024-04-25T10:02:49.845122+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.299685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846718+0000","last_change":"2024-04-25T10:02:49.844459+0000","last_active":"2024-04-25T10:02:49.846718+0000","last_peered":"2024-04-25T10:02:49.846718+0000","last_clean":"2024-04-25T10:02:49.846718+0000","last_became_active":"2024-04-25T10:02:49.843914+0000","last_became_peered":"2024-04-25T10:02:49.843914+0000","last_unstale":"2024-04-25T10:02:49.846718+0000","last_undegraded":"2024-04-25T10:02:49.846718+0000","last_fullsized":"2024-04-25T10:02:49.846718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:01.845554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840370+0000","last_change":"2024-04-25T10:02:49.833068+0000","last_active":"2024-04-25T10:02:49.840370+0000","last_peered":"2024-04-25T10:02:49.840370+0000","last_clean":"2024-04-25T10:02:49.840370+0000","last_became_active":"2024-04-25T10:02:49.832617+0000","last_became_peered":"2024-04-25T10:02:49.832617+0000","last_unstale":"2024-04-25T10:02:49.840370+0000","last_undegraded":"2024-04-25T10:02:49.840370+0000","last_fullsized":"2024-04-25T10:02:49.840370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:09.252258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845414+0000","last_change":"2024-04-25T10:02:49.844765+0000","last_active":"2024-04-25T10:02:49.845414+0000","last_peered":"2024-04-25T10:02:49.845414+0000","last_clean":"2024-04-25T10:02:49.845414+0000","last_became_active":"2024-04-25T10:02:49.844491+0000","last_became_peered":"2024-04-25T10:02:49.844491+0000","last_unstale":"2024-04-25T10:02:49.845414+0000","last_undegraded":"2024-04-25T10:02:49.845414+0000","last_fullsized":"2024-04-25T10:02:49.845414+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:44.711338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844432+0000","last_change":"2024-04-25T10:02:49.841764+0000","last_active":"2024-04-25T10:02:49.844432+0000","last_peered":"2024-04-25T10:02:49.844432+0000","last_clean":"2024-04-25T10:02:49.844432+0000","last_became_active":"2024-04-25T10:02:49.841508+0000","last_became_peered":"2024-04-25T10:02:49.841508+0000","last_unstale":"2024-04-25T10:02:49.844432+0000","last_undegraded":"2024-04-25T10:02:49.844432+0000","last_fullsized":"2024-04-25T10:02:49.844432+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:13.515109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834876+0000","last_change":"2024-04-25T10:02:49.832255+0000","last_active":"2024-04-25T10:02:49.834876+0000","last_peered":"2024-04-25T10:02:49.834876+0000","last_clean":"2024-04-25T10:02:49.834876+0000","last_became_active":"2024-04-25T10:02:49.831921+0000","last_became_peered":"2024-04-25T10:02:49.831921+0000","last_unstale":"2024-04-25T10:02:49.834876+0000","last_undegraded":"2024-04-25T10:02:49.834876+0000","last_fullsized":"2024-04-25T10:02:49.834876+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:13.515109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835074+0000","last_change":"2024-04-25T10:02:49.829789+0000","last_active":"2024-04-25T10:02:49.835074+0000","last_peered":"2024-04-25T10:02:49.835074+0000","last_clean":"2024-04-25T10:02:49.835074+0000","last_became_active":"2024-04-25T10:02:49.829522+0000","last_became_peered":"2024-04-25T10:02:49.829522+0000","last_unstale":"2024-04-25T10:02:49.835074+0000","last_undegraded":"2024-04-25T10:02:49.835074+0000","last_fullsized":"2024-04-25T10:02:49.835074+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:26.913838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834940+0000","last_change":"2024-04-25T10:02:49.833240+0000","last_active":"2024-04-25T10:02:49.834940+0000","last_peered":"2024-04-25T10:02:49.834940+0000","last_clean":"2024-04-25T10:02:49.834940+0000","last_became_active":"2024-04-25T10:02:49.832961+0000","last_became_peered":"2024-04-25T10:02:49.832961+0000","last_unstale":"2024-04-25T10:02:49.834940+0000","last_undegraded":"2024-04-25T10:02:49.834940+0000","last_fullsized":"2024-04-25T10:02:49.834940+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.299685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"132'4","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.851735+0000","last_change":"2024-04-25T10:02:48.828764+0000","last_active":"2024-04-25T10:02:49.851735+0000","last_peered":"2024-04-25T10:02:49.851735+0000","last_clean":"2024-04-25T10:02:49.851735+0000","last_became_active":"2024-04-25T10:02:48.828233+0000","last_became_peered":"2024-04-25T10:02:48.828233+0000","last_unstale":"2024-04-25T10:02:49.851735+0000","last_undegraded":"2024-04-25T10:02:49.851735+0000","last_fullsized":"2024-04-25T10:02:49.851735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:48.046054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833056+0000","last_change":"2024-04-25T10:02:49.832355+0000","last_active":"2024-04-25T10:02:49.833056+0000","last_peered":"2024-04-25T10:02:49.833056+0000","last_clean":"2024-04-25T10:02:49.833056+0000","last_became_active":"2024-04-25T10:02:49.832116+0000","last_became_peered":"2024-04-25T10:02:49.832116+0000","last_unstale":"2024-04-25T10:02:49.833056+0000","last_undegraded":"2024-04-25T10:02:49.833056+0000","last_fullsized":"2024-04-25T10:02:49.833056+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:24.524473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.848489+0000","last_change":"2024-04-25T10:02:48.826403+0000","last_active":"2024-04-25T10:02:49.848489+0000","last_peered":"2024-04-25T10:02:49.848489+0000","last_clean":"2024-04-25T10:02:49.848489+0000","last_became_active":"2024-04-25T10:02:48.825714+0000","last_became_peered":"2024-04-25T10:02:48.825714+0000","last_unstale":"2024-04-25T10:02:49.848489+0000","last_undegraded":"2024-04-25T10:02:49.848489+0000","last_fullsized":"2024-04-25T10:02:49.848489+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:00.840377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835437+0000","last_change":"2024-04-25T10:02:49.834044+0000","last_active":"2024-04-25T10:02:49.835437+0000","last_peered":"2024-04-25T10:02:49.835437+0000","last_clean":"2024-04-25T10:02:49.835437+0000","last_became_active":"2024-04-25T10:02:49.833793+0000","last_became_peered":"2024-04-25T10:02:49.833793+0000","last_unstale":"2024-04-25T10:02:49.835437+0000","last_undegraded":"2024-04-25T10:02:49.835437+0000","last_fullsized":"2024-04-25T10:02:49.835437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:01.155606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.850590+0000","last_change":"2024-04-25T10:02:48.826757+0000","last_active":"2024-04-25T10:02:49.850590+0000","last_peered":"2024-04-25T10:02:49.850590+0000","last_clean":"2024-04-25T10:02:49.850590+0000","last_became_active":"2024-04-25T10:02:48.826295+0000","last_became_peered":"2024-04-25T10:02:48.826295+0000","last_unstale":"2024-04-25T10:02:49.850590+0000","last_undegraded":"2024-04-25T10:02:49.850590+0000","last_fullsized":"2024-04-25T10:02:49.850590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:40.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843847+0000","last_change":"2024-04-25T10:02:49.843091+0000","last_active":"2024-04-25T10:02:49.843847+0000","last_peered":"2024-04-25T10:02:49.843847+0000","last_clean":"2024-04-25T10:02:49.843847+0000","last_became_active":"2024-04-25T10:02:49.842824+0000","last_became_peered":"2024-04-25T10:02:49.842824+0000","last_unstale":"2024-04-25T10:02:49.843847+0000","last_undegraded":"2024-04-25T10:02:49.843847+0000","last_fullsized":"2024-04-25T10:02:49.843847+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:28.350965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.852238+0000","last_change":"2024-04-25T10:02:48.821036+0000","last_active":"2024-04-25T10:02:49.852238+0000","last_peered":"2024-04-25T10:02:49.852238+0000","last_clean":"2024-04-25T10:02:49.852238+0000","last_became_active":"2024-04-25T10:02:48.820769+0000","last_became_peered":"2024-04-25T10:02:48.820769+0000","last_unstale":"2024-04-25T10:02:49.852238+0000","last_undegraded":"2024-04-25T10:02:49.852238+0000","last_fullsized":"2024-04-25T10:02:49.852238+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:15.407574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835021+0000","last_change":"2024-04-25T10:02:49.832719+0000","last_active":"2024-04-25T10:02:49.835021+0000","last_peered":"2024-04-25T10:02:49.835021+0000","last_clean":"2024-04-25T10:02:49.835021+0000","last_became_active":"2024-04-25T10:02:49.832455+0000","last_became_peered":"2024-04-25T10:02:49.832455+0000","last_unstale":"2024-04-25T10:02:49.835021+0000","last_undegraded":"2024-04-25T10:02:49.835021+0000","last_fullsized":"2024-04-25T10:02:49.835021+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:19.597490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849488+0000","last_change":"2024-04-25T10:02:48.822773+0000","last_active":"2024-04-25T10:02:49.849488+0000","last_peered":"2024-04-25T10:02:49.849488+0000","last_clean":"2024-04-25T10:02:49.849488+0000","last_became_active":"2024-04-25T10:02:48.822535+0000","last_became_peered":"2024-04-25T10:02:48.822535+0000","last_unstale":"2024-04-25T10:02:49.849488+0000","last_undegraded":"2024-04-25T10:02:49.849488+0000","last_fullsized":"2024-04-25T10:02:49.849488+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:05.203099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.841576+0000","last_change":"2024-04-25T10:02:49.836677+0000","last_active":"2024-04-25T10:02:49.841576+0000","last_peered":"2024-04-25T10:02:49.841576+0000","last_clean":"2024-04-25T10:02:49.841576+0000","last_became_active":"2024-04-25T10:02:49.836402+0000","last_became_peered":"2024-04-25T10:02:49.836402+0000","last_unstale":"2024-04-25T10:02:49.841576+0000","last_undegraded":"2024-04-25T10:02:49.841576+0000","last_fullsized":"2024-04-25T10:02:49.841576+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:44.711338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.847742+0000","last_change":"2024-04-25T10:02:48.822313+0000","last_active":"2024-04-25T10:02:49.847742+0000","last_peered":"2024-04-25T10:02:49.847742+0000","last_clean":"2024-04-25T10:02:49.847742+0000","last_became_active":"2024-04-25T10:02:48.822039+0000","last_became_peered":"2024-04-25T10:02:48.822039+0000","last_unstale":"2024-04-25T10:02:49.847742+0000","last_undegraded":"2024-04-25T10:02:49.847742+0000","last_fullsized":"2024-04-25T10:02:49.847742+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:55.997076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845007+0000","last_change":"2024-04-25T10:02:49.843896+0000","last_active":"2024-04-25T10:02:49.845007+0000","last_peered":"2024-04-25T10:02:49.845007+0000","last_clean":"2024-04-25T10:02:49.845007+0000","last_became_active":"2024-04-25T10:02:49.843565+0000","last_became_peered":"2024-04-25T10:02:49.843565+0000","last_unstale":"2024-04-25T10:02:49.845007+0000","last_undegraded":"2024-04-25T10:02:49.845007+0000","last_fullsized":"2024-04-25T10:02:49.845007+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:48.991343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827555+0000","last_change":"2024-04-25T10:02:48.827407+0000","last_active":"2024-04-25T10:02:49.827555+0000","last_peered":"2024-04-25T10:02:49.827555+0000","last_clean":"2024-04-25T10:02:49.827555+0000","last_became_active":"2024-04-25T10:02:48.827117+0000","last_became_peered":"2024-04-25T10:02:48.827117+0000","last_unstale":"2024-04-25T10:02:49.827555+0000","last_undegraded":"2024-04-25T10:02:49.827555+0000","last_fullsized":"2024-04-25T10:02:49.827555+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:48.046054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846848+0000","last_change":"2024-04-25T10:02:49.844034+0000","last_active":"2024-04-25T10:02:49.846848+0000","last_peered":"2024-04-25T10:02:49.846848+0000","last_clean":"2024-04-25T10:02:49.846848+0000","last_became_active":"2024-04-25T10:02:49.843742+0000","last_became_peered":"2024-04-25T10:02:49.843742+0000","last_unstale":"2024-04-25T10:02:49.846848+0000","last_undegraded":"2024-04-25T10:02:49.846848+0000","last_fullsized":"2024-04-25T10:02:49.846848+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:57.002254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.836999+0000","last_change":"2024-04-25T10:02:48.830971+0000","last_active":"2024-04-25T10:02:49.836999+0000","last_peered":"2024-04-25T10:02:49.836999+0000","last_clean":"2024-04-25T10:02:49.836999+0000","last_became_active":"2024-04-25T10:02:48.830736+0000","last_became_peered":"2024-04-25T10:02:48.830736+0000","last_unstale":"2024-04-25T10:02:49.836999+0000","last_undegraded":"2024-04-25T10:02:49.836999+0000","last_fullsized":"2024-04-25T10:02:49.836999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:26.467368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839322+0000","last_change":"2024-04-25T10:02:49.838578+0000","last_active":"2024-04-25T10:02:49.839322+0000","last_peered":"2024-04-25T10:02:49.839322+0000","last_clean":"2024-04-25T10:02:49.839322+0000","last_became_active":"2024-04-25T10:02:49.838323+0000","last_became_peered":"2024-04-25T10:02:49.838323+0000","last_unstale":"2024-04-25T10:02:49.839322+0000","last_undegraded":"2024-04-25T10:02:49.839322+0000","last_fullsized":"2024-04-25T10:02:49.839322+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:53.037644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827088+0000","last_change":"2024-04-25T10:02:48.827636+0000","last_active":"2024-04-25T10:02:49.827088+0000","last_peered":"2024-04-25T10:02:49.827088+0000","last_clean":"2024-04-25T10:02:49.827088+0000","last_became_active":"2024-04-25T10:02:48.827360+0000","last_became_peered":"2024-04-25T10:02:48.827360+0000","last_unstale":"2024-04-25T10:02:49.827088+0000","last_undegraded":"2024-04-25T10:02:49.827088+0000","last_fullsized":"2024-04-25T10:02:49.827088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:26.294507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843942+0000","last_change":"2024-04-25T10:02:49.840669+0000","last_active":"2024-04-25T10:02:49.843942+0000","last_peered":"2024-04-25T10:02:49.843942+0000","last_clean":"2024-04-25T10:02:49.843942+0000","last_became_active":"2024-04-25T10:02:49.840413+0000","last_became_peered":"2024-04-25T10:02:49.840413+0000","last_unstale":"2024-04-25T10:02:49.843942+0000","last_undegraded":"2024-04-25T10:02:49.843942+0000","last_fullsized":"2024-04-25T10:02:49.843942+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:16.412752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.831880+0000","last_change":"2024-04-25T10:02:48.824375+0000","last_active":"2024-04-25T10:02:49.831880+0000","last_peered":"2024-04-25T10:02:49.831880+0000","last_clean":"2024-04-25T10:02:49.831880+0000","last_became_active":"2024-04-25T10:02:48.823657+0000","last_became_peered":"2024-04-25T10:02:48.823657+0000","last_unstale":"2024-04-25T10:02:49.831880+0000","last_undegraded":"2024-04-25T10:02:49.831880+0000","last_fullsized":"2024-04-25T10:02:49.831880+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:12.273179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844411+0000","last_change":"2024-04-25T10:02:49.837939+0000","last_active":"2024-04-25T10:02:49.844411+0000","last_peered":"2024-04-25T10:02:49.844411+0000","last_clean":"2024-04-25T10:02:49.844411+0000","last_became_active":"2024-04-25T10:02:49.837234+0000","last_became_peered":"2024-04-25T10:02:49.837234+0000","last_unstale":"2024-04-25T10:02:49.844411+0000","last_undegraded":"2024-04-25T10:02:49.844411+0000","last_fullsized":"2024-04-25T10:02:49.844411+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:54.280858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"132'3","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.852048+0000","last_change":"2024-04-25T10:02:48.826613+0000","last_active":"2024-04-25T10:02:49.852048+0000","last_peered":"2024-04-25T10:02:49.852048+0000","last_clean":"2024-04-25T10:02:49.852048+0000","last_became_active":"2024-04-25T10:02:48.826339+0000","last_became_peered":"2024-04-25T10:02:48.826339+0000","last_unstale":"2024-04-25T10:02:49.852048+0000","last_undegraded":"2024-04-25T10:02:49.852048+0000","last_fullsized":"2024-04-25T10:02:49.852048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:19.883973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845058+0000","last_change":"2024-04-25T10:02:49.842361+0000","last_active":"2024-04-25T10:02:49.845058+0000","last_peered":"2024-04-25T10:02:49.845058+0000","last_clean":"2024-04-25T10:02:49.845058+0000","last_became_active":"2024-04-25T10:02:49.840889+0000","last_became_peered":"2024-04-25T10:02:49.840889+0000","last_unstale":"2024-04-25T10:02:49.845058+0000","last_undegraded":"2024-04-25T10:02:49.845058+0000","last_fullsized":"2024-04-25T10:02:49.845058+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.350657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"132'6","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.853888+0000","last_change":"2024-04-25T10:02:48.829174+0000","last_active":"2024-04-25T10:02:49.853888+0000","last_peered":"2024-04-25T10:02:49.853888+0000","last_clean":"2024-04-25T10:02:49.853888+0000","last_became_active":"2024-04-25T10:02:48.828904+0000","last_became_peered":"2024-04-25T10:02:48.828904+0000","last_unstale":"2024-04-25T10:02:49.853888+0000","last_undegraded":"2024-04-25T10:02:49.853888+0000","last_fullsized":"2024-04-25T10:02:49.853888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:00.150429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840392+0000","last_change":"2024-04-25T10:02:49.834584+0000","last_active":"2024-04-25T10:02:49.840392+0000","last_peered":"2024-04-25T10:02:49.840392+0000","last_clean":"2024-04-25T10:02:49.840392+0000","last_became_active":"2024-04-25T10:02:49.834315+0000","last_became_peered":"2024-04-25T10:02:49.834315+0000","last_unstale":"2024-04-25T10:02:49.840392+0000","last_undegraded":"2024-04-25T10:02:49.840392+0000","last_fullsized":"2024-04-25T10:02:49.840392+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:40.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849522+0000","last_change":"2024-04-25T10:02:48.823023+0000","last_active":"2024-04-25T10:02:49.849522+0000","last_peered":"2024-04-25T10:02:49.849522+0000","last_clean":"2024-04-25T10:02:49.849522+0000","last_became_active":"2024-04-25T10:02:48.822747+0000","last_became_peered":"2024-04-25T10:02:48.822747+0000","last_unstale":"2024-04-25T10:02:49.849522+0000","last_undegraded":"2024-04-25T10:02:49.849522+0000","last_fullsized":"2024-04-25T10:02:49.849522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.473074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845679+0000","last_change":"2024-04-25T10:02:49.841886+0000","last_active":"2024-04-25T10:02:49.845679+0000","last_peered":"2024-04-25T10:02:49.845679+0000","last_clean":"2024-04-25T10:02:49.845679+0000","last_became_active":"2024-04-25T10:02:49.841602+0000","last_became_peered":"2024-04-25T10:02:49.841602+0000","last_unstale":"2024-04-25T10:02:49.845679+0000","last_undegraded":"2024-04-25T10:02:49.845679+0000","last_fullsized":"2024-04-25T10:02:49.845679+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.245807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.831528+0000","last_change":"2024-04-25T10:02:48.830219+0000","last_active":"2024-04-25T10:02:49.831528+0000","last_peered":"2024-04-25T10:02:49.831528+0000","last_clean":"2024-04-25T10:02:49.831528+0000","last_became_active":"2024-04-25T10:02:48.829919+0000","last_became_peered":"2024-04-25T10:02:48.829919+0000","last_unstale":"2024-04-25T10:02:49.831528+0000","last_undegraded":"2024-04-25T10:02:49.831528+0000","last_fullsized":"2024-04-25T10:02:49.831528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:55.248308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833376+0000","last_change":"2024-04-25T10:02:49.831993+0000","last_active":"2024-04-25T10:02:49.833376+0000","last_peered":"2024-04-25T10:02:49.833376+0000","last_clean":"2024-04-25T10:02:49.833376+0000","last_became_active":"2024-04-25T10:02:49.831728+0000","last_became_peered":"2024-04-25T10:02:49.831728+0000","last_unstale":"2024-04-25T10:02:49.833376+0000","last_undegraded":"2024-04-25T10:02:49.833376+0000","last_fullsized":"2024-04-25T10:02:49.833376+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:48.358626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"132'8","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.854283+0000","last_change":"2024-04-25T10:02:48.824872+0000","last_active":"2024-04-25T10:02:49.854283+0000","last_peered":"2024-04-25T10:02:49.854283+0000","last_clean":"2024-04-25T10:02:49.854283+0000","last_became_active":"2024-04-25T10:02:48.824143+0000","last_became_peered":"2024-04-25T10:02:48.824143+0000","last_unstale":"2024-04-25T10:02:49.854283+0000","last_undegraded":"2024-04-25T10:02:49.854283+0000","last_fullsized":"2024-04-25T10:02:49.854283+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:53.275680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844735+0000","last_change":"2024-04-25T10:02:49.841414+0000","last_active":"2024-04-25T10:02:49.844735+0000","last_peered":"2024-04-25T10:02:49.844735+0000","last_clean":"2024-04-25T10:02:49.844735+0000","last_became_active":"2024-04-25T10:02:49.841142+0000","last_became_peered":"2024-04-25T10:02:49.841142+0000","last_unstale":"2024-04-25T10:02:49.844735+0000","last_undegraded":"2024-04-25T10:02:49.844735+0000","last_fullsized":"2024-04-25T10:02:49.844735+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:06.208277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849273+0000","last_change":"2024-04-25T10:02:48.824817+0000","last_active":"2024-04-25T10:02:49.849273+0000","last_peered":"2024-04-25T10:02:49.849273+0000","last_clean":"2024-04-25T10:02:49.849273+0000","last_became_active":"2024-04-25T10:02:48.824432+0000","last_became_peered":"2024-04-25T10:02:48.824432+0000","last_unstale":"2024-04-25T10:02:49.849273+0000","last_undegraded":"2024-04-25T10:02:49.849273+0000","last_fullsized":"2024-04-25T10:02:49.849273+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:00.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846833+0000","last_change":"2024-04-25T10:02:49.845032+0000","last_active":"2024-04-25T10:02:49.846833+0000","last_peered":"2024-04-25T10:02:49.846833+0000","last_clean":"2024-04-25T10:02:49.846833+0000","last_became_active":"2024-04-25T10:02:49.844756+0000","last_became_peered":"2024-04-25T10:02:49.844756+0000","last_unstale":"2024-04-25T10:02:49.846833+0000","last_undegraded":"2024-04-25T10:02:49.846833+0000","last_fullsized":"2024-04-25T10:02:49.846833+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:48.358626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.850393+0000","last_change":"2024-04-25T10:02:48.828930+0000","last_active":"2024-04-25T10:02:49.850393+0000","last_peered":"2024-04-25T10:02:49.850393+0000","last_clean":"2024-04-25T10:02:49.850393+0000","last_became_active":"2024-04-25T10:02:48.828659+0000","last_became_peered":"2024-04-25T10:02:48.828659+0000","last_unstale":"2024-04-25T10:02:49.850393+0000","last_undegraded":"2024-04-25T10:02:49.850393+0000","last_fullsized":"2024-04-25T10:02:49.850393+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:39.338494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846293+0000","last_change":"2024-04-25T10:02:49.843998+0000","last_active":"2024-04-25T10:02:49.846293+0000","last_peered":"2024-04-25T10:02:49.846293+0000","last_clean":"2024-04-25T10:02:49.846293+0000","last_became_active":"2024-04-25T10:02:49.843711+0000","last_became_peered":"2024-04-25T10:02:49.843711+0000","last_unstale":"2024-04-25T10:02:49.846293+0000","last_undegraded":"2024-04-25T10:02:49.846293+0000","last_fullsized":"2024-04-25T10:02:49.846293+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:02.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.850736+0000","last_change":"2024-04-25T10:02:48.827675+0000","last_active":"2024-04-25T10:02:49.850736+0000","last_peered":"2024-04-25T10:02:49.850736+0000","last_clean":"2024-04-25T10:02:49.850736+0000","last_became_active":"2024-04-25T10:02:48.827288+0000","last_became_peered":"2024-04-25T10:02:48.827288+0000","last_unstale":"2024-04-25T10:02:49.850736+0000","last_undegraded":"2024-04-25T10:02:49.850736+0000","last_fullsized":"2024-04-25T10:02:49.850736+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:45.235658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839820+0000","last_change":"2024-04-25T10:02:49.838492+0000","last_active":"2024-04-25T10:02:49.839820+0000","last_peered":"2024-04-25T10:02:49.839820+0000","last_clean":"2024-04-25T10:02:49.839820+0000","last_became_active":"2024-04-25T10:02:49.838242+0000","last_became_peered":"2024-04-25T10:02:49.838242+0000","last_unstale":"2024-04-25T10:02:49.839820+0000","last_undegraded":"2024-04-25T10:02:49.839820+0000","last_fullsized":"2024-04-25T10:02:49.839820+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:22.290974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833022+0000","last_change":"2024-04-25T10:02:48.826489+0000","last_active":"2024-04-25T10:02:49.833022+0000","last_peered":"2024-04-25T10:02:49.833022+0000","last_clean":"2024-04-25T10:02:49.833022+0000","last_became_active":"2024-04-25T10:02:48.826208+0000","last_became_peered":"2024-04-25T10:02:48.826208+0000","last_unstale":"2024-04-25T10:02:49.833022+0000","last_undegraded":"2024-04-25T10:02:49.833022+0000","last_fullsized":"2024-04-25T10:02:49.833022+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:55.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839648+0000","last_change":"2024-04-25T10:02:49.834066+0000","last_active":"2024-04-25T10:02:49.839648+0000","last_peered":"2024-04-25T10:02:49.839648+0000","last_clean":"2024-04-25T10:02:49.839648+0000","last_became_active":"2024-04-25T10:02:49.833795+0000","last_became_peered":"2024-04-25T10:02:49.833795+0000","last_unstale":"2024-04-25T10:02:49.839648+0000","last_undegraded":"2024-04-25T10:02:49.839648+0000","last_fullsized":"2024-04-25T10:02:49.839648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.190728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.851088+0000","last_change":"2024-04-25T10:02:48.826053+0000","last_active":"2024-04-25T10:02:49.851088+0000","last_peered":"2024-04-25T10:02:49.851088+0000","last_clean":"2024-04-25T10:02:49.851088+0000","last_became_active":"2024-04-25T10:02:48.825783+0000","last_became_peered":"2024-04-25T10:02:48.825783+0000","last_unstale":"2024-04-25T10:02:49.851088+0000","last_undegraded":"2024-04-25T10:02:49.851088+0000","last_fullsized":"2024-04-25T10:02:49.851088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:45.235658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.832983+0000","last_change":"2024-04-25T10:02:49.831978+0000","last_active":"2024-04-25T10:02:49.832983+0000","last_peered":"2024-04-25T10:02:49.832983+0000","last_clean":"2024-04-25T10:02:49.832983+0000","last_became_active":"2024-04-25T10:02:49.831651+0000","last_became_peered":"2024-04-25T10:02:49.831651+0000","last_unstale":"2024-04-25T10:02:49.832983+0000","last_undegraded":"2024-04-25T10:02:49.832983+0000","last_fullsized":"2024-04-25T10:02:49.832983+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:39.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.828769+0000","last_change":"2024-04-25T10:02:48.820305+0000","last_active":"2024-04-25T10:02:49.828769+0000","last_peered":"2024-04-25T10:02:49.828769+0000","last_clean":"2024-04-25T10:02:49.828769+0000","last_became_active":"2024-04-25T10:02:48.820028+0000","last_became_peered":"2024-04-25T10:02:48.820028+0000","last_unstale":"2024-04-25T10:02:49.828769+0000","last_undegraded":"2024-04-25T10:02:49.828769+0000","last_fullsized":"2024-04-25T10:02:49.828769+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:37.215984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844987+0000","last_change":"2024-04-25T10:02:49.843680+0000","last_active":"2024-04-25T10:02:49.844987+0000","last_peered":"2024-04-25T10:02:49.844987+0000","last_clean":"2024-04-25T10:02:49.844987+0000","last_became_active":"2024-04-25T10:02:49.839409+0000","last_became_peered":"2024-04-25T10:02:49.839409+0000","last_unstale":"2024-04-25T10:02:49.844987+0000","last_undegraded":"2024-04-25T10:02:49.844987+0000","last_fullsized":"2024-04-25T10:02:49.844987+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:27.497333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.836391+0000","last_change":"2024-04-25T10:02:48.828163+0000","last_active":"2024-04-25T10:02:49.836391+0000","last_peered":"2024-04-25T10:02:49.836391+0000","last_clean":"2024-04-25T10:02:49.836391+0000","last_became_active":"2024-04-25T10:02:48.827888+0000","last_became_peered":"2024-04-25T10:02:48.827888+0000","last_unstale":"2024-04-25T10:02:49.836391+0000","last_undegraded":"2024-04-25T10:02:49.836391+0000","last_fullsized":"2024-04-25T10:02:49.836391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:05.266151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840608+0000","last_change":"2024-04-25T10:02:49.839919+0000","last_active":"2024-04-25T10:02:49.840608+0000","last_peered":"2024-04-25T10:02:49.840608+0000","last_clean":"2024-04-25T10:02:49.840608+0000","last_became_active":"2024-04-25T10:02:49.839561+0000","last_became_peered":"2024-04-25T10:02:49.839561+0000","last_unstale":"2024-04-25T10:02:49.840608+0000","last_undegraded":"2024-04-25T10:02:49.840608+0000","last_fullsized":"2024-04-25T10:02:49.840608+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:06.271328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.828057+0000","last_change":"2024-04-25T10:02:48.822741+0000","last_active":"2024-04-25T10:02:49.828057+0000","last_peered":"2024-04-25T10:02:49.828057+0000","last_clean":"2024-04-25T10:02:49.828057+0000","last_became_active":"2024-04-25T10:02:48.822505+0000","last_became_peered":"2024-04-25T10:02:48.822505+0000","last_unstale":"2024-04-25T10:02:49.828057+0000","last_undegraded":"2024-04-25T10:02:49.828057+0000","last_fullsized":"2024-04-25T10:02:49.828057+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:51.341492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843766+0000","last_change":"2024-04-25T10:02:49.834958+0000","last_active":"2024-04-25T10:02:49.843766+0000","last_peered":"2024-04-25T10:02:49.843766+0000","last_clean":"2024-04-25T10:02:49.843766+0000","last_became_active":"2024-04-25T10:02:49.834691+0000","last_became_peered":"2024-04-25T10:02:49.834691+0000","last_unstale":"2024-04-25T10:02:49.843766+0000","last_undegraded":"2024-04-25T10:02:49.843766+0000","last_fullsized":"2024-04-25T10:02:49.843766+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:38.442612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.851974+0000","last_change":"2024-04-25T10:02:48.826738+0000","last_active":"2024-04-25T10:02:49.851974+0000","last_peered":"2024-04-25T10:02:49.851974+0000","last_clean":"2024-04-25T10:02:49.851974+0000","last_became_active":"2024-04-25T10:02:48.825795+0000","last_became_peered":"2024-04-25T10:02:48.825795+0000","last_unstale":"2024-04-25T10:02:49.851974+0000","last_undegraded":"2024-04-25T10:02:49.851974+0000","last_fullsized":"2024-04-25T10:02:49.851974+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:30.862171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844476+0000","last_change":"2024-04-25T10:02:49.840172+0000","last_active":"2024-04-25T10:02:49.844476+0000","last_peered":"2024-04-25T10:02:49.844476+0000","last_clean":"2024-04-25T10:02:49.844476+0000","last_became_active":"2024-04-25T10:02:49.839886+0000","last_became_peered":"2024-04-25T10:02:49.839886+0000","last_unstale":"2024-04-25T10:02:49.844476+0000","last_undegraded":"2024-04-25T10:02:49.844476+0000","last_fullsized":"2024-04-25T10:02:49.844476+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.240835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827079+0000","last_change":"2024-04-25T10:02:48.821994+0000","last_active":"2024-04-25T10:02:49.827079+0000","last_peered":"2024-04-25T10:02:49.827079+0000","last_clean":"2024-04-25T10:02:49.827079+0000","last_became_active":"2024-04-25T10:02:48.819410+0000","last_became_peered":"2024-04-25T10:02:48.819410+0000","last_unstale":"2024-04-25T10:02:49.827079+0000","last_undegraded":"2024-04-25T10:02:49.827079+0000","last_fullsized":"2024-04-25T10:02:49.827079+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:54.528247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.847695+0000","last_change":"2024-04-25T10:02:49.846142+0000","last_active":"2024-04-25T10:02:49.847695+0000","last_peered":"2024-04-25T10:02:49.847695+0000","last_clean":"2024-04-25T10:02:49.847695+0000","last_became_active":"2024-04-25T10:02:49.845835+0000","last_became_peered":"2024-04-25T10:02:49.845835+0000","last_unstale":"2024-04-25T10:02:49.847695+0000","last_undegraded":"2024-04-25T10:02:49.847695+0000","last_fullsized":"2024-04-25T10:02:49.847695+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:56.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827025+0000","last_change":"2024-04-25T10:02:48.821477+0000","last_active":"2024-04-25T10:02:49.827025+0000","last_peered":"2024-04-25T10:02:49.827025+0000","last_clean":"2024-04-25T10:02:49.827025+0000","last_became_active":"2024-04-25T10:02:48.821201+0000","last_became_peered":"2024-04-25T10:02:48.821201+0000","last_unstale":"2024-04-25T10:02:49.827025+0000","last_undegraded":"2024-04-25T10:02:49.827025+0000","last_fullsized":"2024-04-25T10:02:49.827025+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:51.341492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.838635+0000","last_change":"2024-04-25T10:02:49.837705+0000","last_active":"2024-04-25T10:02:49.838635+0000","last_peered":"2024-04-25T10:02:49.838635+0000","last_clean":"2024-04-25T10:02:49.838635+0000","last_became_active":"2024-04-25T10:02:49.837452+0000","last_became_peered":"2024-04-25T10:02:49.837452+0000","last_unstale":"2024-04-25T10:02:49.838635+0000","last_undegraded":"2024-04-25T10:02:49.838635+0000","last_fullsized":"2024-04-25T10:02:49.838635+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:49.938913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.829666+0000","last_change":"2024-04-25T10:02:48.820572+0000","last_active":"2024-04-25T10:02:49.829666+0000","last_peered":"2024-04-25T10:02:49.829666+0000","last_clean":"2024-04-25T10:02:49.829666+0000","last_became_active":"2024-04-25T10:02:48.820294+0000","last_became_peered":"2024-04-25T10:02:48.820294+0000","last_unstale":"2024-04-25T10:02:49.829666+0000","last_undegraded":"2024-04-25T10:02:49.829666+0000","last_fullsized":"2024-04-25T10:02:49.829666+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:54.528247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842144+0000","last_change":"2024-04-25T10:02:49.835961+0000","last_active":"2024-04-25T10:02:49.842144+0000","last_peered":"2024-04-25T10:02:49.842144+0000","last_clean":"2024-04-25T10:02:49.842144+0000","last_became_active":"2024-04-25T10:02:49.835186+0000","last_became_peered":"2024-04-25T10:02:49.835186+0000","last_unstale":"2024-04-25T10:02:49.842144+0000","last_undegraded":"2024-04-25T10:02:49.842144+0000","last_fullsized":"2024-04-25T10:02:49.842144+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.299685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827520+0000","last_change":"2024-04-25T10:02:48.828657+0000","last_active":"2024-04-25T10:02:49.827520+0000","last_peered":"2024-04-25T10:02:49.827520+0000","last_clean":"2024-04-25T10:02:49.827520+0000","last_became_active":"2024-04-25T10:02:48.828398+0000","last_became_peered":"2024-04-25T10:02:48.828398+0000","last_unstale":"2024-04-25T10:02:49.827520+0000","last_undegraded":"2024-04-25T10:02:49.827520+0000","last_fullsized":"2024-04-25T10:02:49.827520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:43.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842009+0000","last_change":"2024-04-25T10:02:49.835574+0000","last_active":"2024-04-25T10:02:49.842009+0000","last_peered":"2024-04-25T10:02:49.842009+0000","last_clean":"2024-04-25T10:02:49.842009+0000","last_became_active":"2024-04-25T10:02:49.835307+0000","last_became_peered":"2024-04-25T10:02:49.835307+0000","last_unstale":"2024-04-25T10:02:49.842009+0000","last_undegraded":"2024-04-25T10:02:49.842009+0000","last_fullsized":"2024-04-25T10:02:49.842009+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:19.597490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834432+0000","last_change":"2024-04-25T10:02:48.828042+0000","last_active":"2024-04-25T10:02:49.834432+0000","last_peered":"2024-04-25T10:02:49.834432+0000","last_clean":"2024-04-25T10:02:49.834432+0000","last_became_active":"2024-04-25T10:02:48.827542+0000","last_became_peered":"2024-04-25T10:02:48.827542+0000","last_unstale":"2024-04-25T10:02:49.834432+0000","last_undegraded":"2024-04-25T10:02:49.834432+0000","last_fullsized":"2024-04-25T10:02:49.834432+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:06.280452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842114+0000","last_change":"2024-04-25T10:02:49.839532+0000","last_active":"2024-04-25T10:02:49.842114+0000","last_peered":"2024-04-25T10:02:49.842114+0000","last_clean":"2024-04-25T10:02:49.842114+0000","last_became_active":"2024-04-25T10:02:49.839271+0000","last_became_peered":"2024-04-25T10:02:49.839271+0000","last_unstale":"2024-04-25T10:02:49.842114+0000","last_undegraded":"2024-04-25T10:02:49.842114+0000","last_fullsized":"2024-04-25T10:02:49.842114+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:07.285630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827722+0000","last_change":"2024-04-25T10:02:48.821769+0000","last_active":"2024-04-25T10:02:49.827722+0000","last_peered":"2024-04-25T10:02:49.827722+0000","last_clean":"2024-04-25T10:02:49.827722+0000","last_became_active":"2024-04-25T10:02:48.821490+0000","last_became_peered":"2024-04-25T10:02:48.821490+0000","last_unstale":"2024-04-25T10:02:49.827722+0000","last_undegraded":"2024-04-25T10:02:49.827722+0000","last_fullsized":"2024-04-25T10:02:49.827722+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:26.467368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849201+0000","last_change":"2024-04-25T10:02:49.846635+0000","last_active":"2024-04-25T10:02:49.849201+0000","last_peered":"2024-04-25T10:02:49.849201+0000","last_clean":"2024-04-25T10:02:49.849201+0000","last_became_active":"2024-04-25T10:02:49.846302+0000","last_became_peered":"2024-04-25T10:02:49.846302+0000","last_unstale":"2024-04-25T10:02:49.849201+0000","last_undegraded":"2024-04-25T10:02:49.849201+0000","last_fullsized":"2024-04-25T10:02:49.849201+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.190728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827435+0000","last_change":"2024-04-25T10:02:48.828150+0000","last_active":"2024-04-25T10:02:49.827435+0000","last_peered":"2024-04-25T10:02:49.827435+0000","last_clean":"2024-04-25T10:02:49.827435+0000","last_became_active":"2024-04-25T10:02:48.827881+0000","last_became_peered":"2024-04-25T10:02:48.827881+0000","last_unstale":"2024-04-25T10:02:49.827435+0000","last_undegraded":"2024-04-25T10:02:49.827435+0000","last_fullsized":"2024-04-25T10:02:49.827435+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:01.500269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844682+0000","last_change":"2024-04-25T10:02:49.839068+0000","last_active":"2024-04-25T10:02:49.844682+0000","last_peered":"2024-04-25T10:02:49.844682+0000","last_clean":"2024-04-25T10:02:49.844682+0000","last_became_active":"2024-04-25T10:02:49.837711+0000","last_became_peered":"2024-04-25T10:02:49.837711+0000","last_unstale":"2024-04-25T10:02:49.844682+0000","last_undegraded":"2024-04-25T10:02:49.844682+0000","last_fullsized":"2024-04-25T10:02:49.844682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:55.533425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833954+0000","last_change":"2024-04-25T10:02:48.827809+0000","last_active":"2024-04-25T10:02:49.833954+0000","last_peered":"2024-04-25T10:02:49.833954+0000","last_clean":"2024-04-25T10:02:49.833954+0000","last_became_active":"2024-04-25T10:02:48.827164+0000","last_became_peered":"2024-04-25T10:02:48.827164+0000","last_unstale":"2024-04-25T10:02:49.833954+0000","last_undegraded":"2024-04-25T10:02:49.833954+0000","last_fullsized":"2024-04-25T10:02:49.833954+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:43.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":376,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833314+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:02:49.833314+0000","last_peered":"2024-04-25T10:02:49.833314+0000","last_clean":"2024-04-25T10:02:49.833314+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:02:49.833314+0000","last_undegraded":"2024-04-25T10:02:49.833314+0000","last_fullsized":"2024-04-25T10:02:49.833314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29204,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342636,"statfs":{"total":96636764160,"available":96606859264,"internally_reserved":0,"allocated":2232320,"data_stored":1351375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50828,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321012,"statfs":{"total":96636764160,"available":96584716288,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1654784,"data_stored":768196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54988,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316852,"statfs":{"total":96636764160,"available":96580456448,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54988,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316852,"statfs":{"total":96636764160,"available":96580456448,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50832,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321008,"statfs":{"total":96636764160,"available":96584712192,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50832,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321008,"statfs":{"total":96636764160,"available":96584712192,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51416,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320424,"statfs":{"total":96636764160,"available":96584114176,"internally_reserved":0,"allocated":2232320,"data_stored":1351375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:52.795 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:52.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 Processor -- start 2024-04-25T10:02:52.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 -- start start 2024-04-25T10:02:52.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09e80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09e80d1020 0x7f09e80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e80b97a0 con 0x7f09e815dbe0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e815dfc0 con 0x7f09e80d1020 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.924+0000 7f09ede9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80d1a00 con 0x7f09e80b2cb0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f09e815dbe0 0x7f09e80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:56802/0 (socket says 172.21.15.148:56802) 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e77fe640 1 -- 172.21.15.148:0/2851685404 learned_addr learned my addr 172.21.15.148:0/2851685404 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391424837 0 0) 0x7f09e80b97a0 con 0x7f09e815dbe0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d4003520 con 0x7f09e815dbe0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e7fff640 1 --2- 172.21.15.148:0/2851685404 >> v2:172.21.15.188:3300/0 conn(0x7f09e80d1020 0x7f09e80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09dffff640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09e80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2178218588 0 0) 0x7f09d4003520 con 0x7f09e815dbe0 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 msgr2=0x7f09e80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09e80d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 >> v2:172.21.15.188:3300/0 conn(0x7f09e80d1020 msgr2=0x7f09e80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 --2- 172.21.15.148:0/2851685404 >> v2:172.21.15.188:3300/0 conn(0x7f09e80d1020 0x7f09e80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e80d1c50 con 0x7f09e815dbe0 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f09d8001670 con 0x7f09e815dbe0 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09dffff640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09e80d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 >> v1:172.21.15.148:6789/0 conn(0x7f09e815dbe0 legacy=0x7f09e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d40039b0 0x7f09d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d4004800 0x7f09d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80d1a00 con 0x7f09d40039b0 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d400b9c0 con 0x7f09d4004800 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d400bb40 con 0x7f09e80b2cb0 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e77fe640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d40039b0 0x7f09d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09dffff640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d4004800 0x7f09d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.925+0000 7f09e7fff640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e77fe640 1 -- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 msgr2=0x7f09d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e77fe640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09e80b2cb0 0x7f09d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e77fe640 1 -- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d4004800 msgr2=0x7f09d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e77fe640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09d4004800 0x7f09d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e77fe640 1 -- 172.21.15.148:0/2851685404 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4003520 con 0x7f09d40039b0 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e77fe640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d40039b0 0x7f09d4003dd0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f09d800c2c0 tx=0x7f09d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5fec9673ceca28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8007d50 con 0x7f09d40039b0 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d802e070 con 0x7f09d40039b0 2024-04-25T10:02:52.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.926+0000 7f09e6ffd640 1 -- 172.21.15.148:0/2851685404 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8005d50 con 0x7f09d40039b0 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09ede9a640 1 -- 172.21.15.148:0/2851685404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d40039b0 msgr2=0x7f09d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09ede9a640 1 --2- 172.21.15.148:0/2851685404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09d40039b0 0x7f09d4003dd0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f09d800c2c0 tx=0x7f09d4005830 comp rx=0 tx=0).stop 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09e7fff640 1 -- 172.21.15.148:0/2851685404 reap_dead start 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09ede9a640 1 -- 172.21.15.148:0/2851685404 shutdown_connections 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09ede9a640 1 -- 172.21.15.148:0/2851685404 >> 172.21.15.148:0/2851685404 conn(0x7f09e8061a20 msgr2=0x7f09e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09ede9a640 1 -- 172.21.15.148:0/2851685404 shutdown_connections 2024-04-25T10:02:52.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.927+0000 7f09ede9a640 1 -- 172.21.15.148:0/2851685404 wait complete. 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.928+0000 7f09ede9a640 1 Processor -- start 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.928+0000 7f09ede9a640 1 -- start start 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09ede9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d40039b0 0x7f09e814be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09ede9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e8160610 0x7f09e8160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09ede9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 0x7f09e815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09ede9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80b92a0 con 0x7f09e815e180 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09ede9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80d2210 con 0x7f09e8160610 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09ede9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80b9420 con 0x7f09d40039b0 2024-04-25T10:02:52.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 0x7f09e815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 0x7f09e815e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34348/0 (socket says 172.21.15.148:34348) 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 -- 172.21.15.148:0/379320682 learned_addr learned my addr 172.21.15.148:0/379320682 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09dffff640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e8160610 0x7f09e8160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e77fe640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d40039b0 0x7f09e814be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 -- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d40039b0 msgr2=0x7f09e814be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d40039b0 0x7f09e814be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 -- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e8160610 msgr2=0x7f09e8160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e8160610 0x7f09e8160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 -- 172.21.15.148:0/379320682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e815eed0 con 0x7f09e815e180 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09dffff640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e8160610 0x7f09e8160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:52.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.929+0000 7f09e7fff640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 0x7f09e815e600 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f09e80b25c0 tx=0x7f09e000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.930+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000fdb0 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.930+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e001d070 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.930+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e815f1b0 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.930+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0015670 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.930+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e815f770 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.931+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e8073990 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.931+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09e0015810 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.931+0000 7f09e4ff9640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09c806b500 0x7f09c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.932+0000 7f09e77fe640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09c806b500 0x7f09c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.932+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09e008ac10 con 0x7f09e815e180 2024-04-25T10:02:52.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.932+0000 7f09e77fe640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09c806b500 0x7f09c806d9c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f09d8033000 tx=0x7f09d8009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:52.936+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09e0010040 con 0x7f09e815e180 2024-04-25T10:02:53.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.104+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09e80b32e0 con 0x7f09c806b500 2024-04-25T10:02:53.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.110+0000 7f09e4ff9640 1 -- 172.21.15.148:0/379320682 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252414 (secure 0 0 0) 0x7f09e80b32e0 con 0x7f09c806b500 2024-04-25T10:02:53.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:53.119 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09c806b500 msgr2=0x7f09c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09c806b500 0x7f09c806d9c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f09d8033000 tx=0x7f09d8009030 comp rx=0 tx=0).stop 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 msgr2=0x7f09e815e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 0x7f09e815e600 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f09e80b25c0 tx=0x7f09e000f580 comp rx=0 tx=0).stop 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 shutdown_connections 2024-04-25T10:02:53.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09c806b500 0x7f09c806d9c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e815e180 0x7f09e815e600 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09e8160610 0x7f09e8160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 --2- 172.21.15.148:0/379320682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09d40039b0 0x7f09e814be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 >> 172.21.15.148:0/379320682 conn(0x7f09e8061a20 msgr2=0x7f09e815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 shutdown_connections 2024-04-25T10:02:53.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.113+0000 7f09ede9a640 1 -- 172.21.15.148:0/379320682 wait complete. 2024-04-25T10:02:53.129 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-04-25T10:02:51.124608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":389780,"kb_used_data":13976,"kb_used_omap":12,"kb_used_meta":375667,"kb_avail":754584940,"statfs":{"total":773094113280,"available":772694978560,"internally_reserved":0,"allocated":14311424,"data_stored":7275981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":384683568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000613"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846643+0000","last_change":"2024-04-25T10:02:49.844722+0000","last_active":"2024-04-25T10:02:49.846643+0000","last_peered":"2024-04-25T10:02:49.846643+0000","last_clean":"2024-04-25T10:02:49.846643+0000","last_became_active":"2024-04-25T10:02:49.844197+0000","last_became_peered":"2024-04-25T10:02:49.844197+0000","last_unstale":"2024-04-25T10:02:49.846643+0000","last_undegraded":"2024-04-25T10:02:49.846643+0000","last_fullsized":"2024-04-25T10:02:49.846643+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:25.017866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835870+0000","last_change":"2024-04-25T10:02:49.833348+0000","last_active":"2024-04-25T10:02:49.835870+0000","last_peered":"2024-04-25T10:02:49.835870+0000","last_clean":"2024-04-25T10:02:49.835870+0000","last_became_active":"2024-04-25T10:02:49.833081+0000","last_became_peered":"2024-04-25T10:02:49.833081+0000","last_unstale":"2024-04-25T10:02:49.835870+0000","last_undegraded":"2024-04-25T10:02:49.835870+0000","last_fullsized":"2024-04-25T10:02:49.835870+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:16.412752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844701+0000","last_change":"2024-04-25T10:02:49.843028+0000","last_active":"2024-04-25T10:02:49.844701+0000","last_peered":"2024-04-25T10:02:49.844701+0000","last_clean":"2024-04-25T10:02:49.844701+0000","last_became_active":"2024-04-25T10:02:49.838784+0000","last_became_peered":"2024-04-25T10:02:49.838784+0000","last_unstale":"2024-04-25T10:02:49.844701+0000","last_undegraded":"2024-04-25T10:02:49.844701+0000","last_fullsized":"2024-04-25T10:02:49.844701+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:19.597490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843642+0000","last_change":"2024-04-25T10:02:49.843054+0000","last_active":"2024-04-25T10:02:49.843642+0000","last_peered":"2024-04-25T10:02:49.843642+0000","last_clean":"2024-04-25T10:02:49.843642+0000","last_became_active":"2024-04-25T10:02:49.842770+0000","last_became_peered":"2024-04-25T10:02:49.842770+0000","last_unstale":"2024-04-25T10:02:49.843642+0000","last_undegraded":"2024-04-25T10:02:49.843642+0000","last_fullsized":"2024-04-25T10:02:49.843642+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.245807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840012+0000","last_change":"2024-04-25T10:02:49.836411+0000","last_active":"2024-04-25T10:02:49.840012+0000","last_peered":"2024-04-25T10:02:49.840012+0000","last_clean":"2024-04-25T10:02:49.840012+0000","last_became_active":"2024-04-25T10:02:49.836164+0000","last_became_peered":"2024-04-25T10:02:49.836164+0000","last_unstale":"2024-04-25T10:02:49.840012+0000","last_undegraded":"2024-04-25T10:02:49.840012+0000","last_fullsized":"2024-04-25T10:02:49.840012+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:48.746929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843984+0000","last_change":"2024-04-25T10:02:49.841469+0000","last_active":"2024-04-25T10:02:49.843984+0000","last_peered":"2024-04-25T10:02:49.843984+0000","last_clean":"2024-04-25T10:02:49.843984+0000","last_became_active":"2024-04-25T10:02:49.841214+0000","last_became_peered":"2024-04-25T10:02:49.841214+0000","last_unstale":"2024-04-25T10:02:49.843984+0000","last_undegraded":"2024-04-25T10:02:49.843984+0000","last_fullsized":"2024-04-25T10:02:49.843984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:01.155606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.848390+0000","last_change":"2024-04-25T10:02:49.846690+0000","last_active":"2024-04-25T10:02:49.848390+0000","last_peered":"2024-04-25T10:02:49.848390+0000","last_clean":"2024-04-25T10:02:49.848390+0000","last_became_active":"2024-04-25T10:02:49.845901+0000","last_became_peered":"2024-04-25T10:02:49.845901+0000","last_unstale":"2024-04-25T10:02:49.848390+0000","last_undegraded":"2024-04-25T10:02:49.848390+0000","last_fullsized":"2024-04-25T10:02:49.848390+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:38.442612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839175+0000","last_change":"2024-04-25T10:02:49.836096+0000","last_active":"2024-04-25T10:02:49.839175+0000","last_peered":"2024-04-25T10:02:49.839175+0000","last_clean":"2024-04-25T10:02:49.839175+0000","last_became_active":"2024-04-25T10:02:49.835826+0000","last_became_peered":"2024-04-25T10:02:49.835826+0000","last_unstale":"2024-04-25T10:02:49.839175+0000","last_undegraded":"2024-04-25T10:02:49.839175+0000","last_fullsized":"2024-04-25T10:02:49.839175+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.350657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842646+0000","last_change":"2024-04-25T10:02:49.833852+0000","last_active":"2024-04-25T10:02:49.842646+0000","last_peered":"2024-04-25T10:02:49.842646+0000","last_clean":"2024-04-25T10:02:49.842646+0000","last_became_active":"2024-04-25T10:02:49.833417+0000","last_became_peered":"2024-04-25T10:02:49.833417+0000","last_unstale":"2024-04-25T10:02:49.842646+0000","last_undegraded":"2024-04-25T10:02:49.842646+0000","last_fullsized":"2024-04-25T10:02:49.842646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:13.515109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834984+0000","last_change":"2024-04-25T10:02:49.830643+0000","last_active":"2024-04-25T10:02:49.834984+0000","last_peered":"2024-04-25T10:02:49.834984+0000","last_clean":"2024-04-25T10:02:49.834984+0000","last_became_active":"2024-04-25T10:02:49.830367+0000","last_became_peered":"2024-04-25T10:02:49.830367+0000","last_unstale":"2024-04-25T10:02:49.834984+0000","last_undegraded":"2024-04-25T10:02:49.834984+0000","last_fullsized":"2024-04-25T10:02:49.834984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:41.743497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840636+0000","last_change":"2024-04-25T10:02:49.839883+0000","last_active":"2024-04-25T10:02:49.840636+0000","last_peered":"2024-04-25T10:02:49.840636+0000","last_clean":"2024-04-25T10:02:49.840636+0000","last_became_active":"2024-04-25T10:02:49.839575+0000","last_became_peered":"2024-04-25T10:02:49.839575+0000","last_unstale":"2024-04-25T10:02:49.840636+0000","last_undegraded":"2024-04-25T10:02:49.840636+0000","last_fullsized":"2024-04-25T10:02:49.840636+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:20.889150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839585+0000","last_change":"2024-04-25T10:02:49.837142+0000","last_active":"2024-04-25T10:02:49.839585+0000","last_peered":"2024-04-25T10:02:49.839585+0000","last_clean":"2024-04-25T10:02:49.839585+0000","last_became_active":"2024-04-25T10:02:49.836832+0000","last_became_peered":"2024-04-25T10:02:49.836832+0000","last_unstale":"2024-04-25T10:02:49.839585+0000","last_undegraded":"2024-04-25T10:02:49.839585+0000","last_fullsized":"2024-04-25T10:02:49.839585+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:22.757440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844020+0000","last_change":"2024-04-25T10:02:49.842819+0000","last_active":"2024-04-25T10:02:49.844020+0000","last_peered":"2024-04-25T10:02:49.844020+0000","last_clean":"2024-04-25T10:02:49.844020+0000","last_became_active":"2024-04-25T10:02:49.842510+0000","last_became_peered":"2024-04-25T10:02:49.842510+0000","last_unstale":"2024-04-25T10:02:49.844020+0000","last_undegraded":"2024-04-25T10:02:49.844020+0000","last_fullsized":"2024-04-25T10:02:49.844020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:41.207015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833809+0000","last_change":"2024-04-25T10:02:49.833102+0000","last_active":"2024-04-25T10:02:49.833809+0000","last_peered":"2024-04-25T10:02:49.833809+0000","last_clean":"2024-04-25T10:02:49.833809+0000","last_became_active":"2024-04-25T10:02:49.832831+0000","last_became_peered":"2024-04-25T10:02:49.832831+0000","last_unstale":"2024-04-25T10:02:49.833809+0000","last_undegraded":"2024-04-25T10:02:49.833809+0000","last_fullsized":"2024-04-25T10:02:49.833809+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:38.425880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.832520+0000","last_change":"2024-04-25T10:02:49.831636+0000","last_active":"2024-04-25T10:02:49.832520+0000","last_peered":"2024-04-25T10:02:49.832520+0000","last_clean":"2024-04-25T10:02:49.832520+0000","last_became_active":"2024-04-25T10:02:49.831381+0000","last_became_peered":"2024-04-25T10:02:49.831381+0000","last_unstale":"2024-04-25T10:02:49.832520+0000","last_undegraded":"2024-04-25T10:02:49.832520+0000","last_fullsized":"2024-04-25T10:02:49.832520+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:04.924245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842568+0000","last_change":"2024-04-25T10:02:49.836160+0000","last_active":"2024-04-25T10:02:49.842568+0000","last_peered":"2024-04-25T10:02:49.842568+0000","last_clean":"2024-04-25T10:02:49.842568+0000","last_became_active":"2024-04-25T10:02:49.835870+0000","last_became_peered":"2024-04-25T10:02:49.835870+0000","last_unstale":"2024-04-25T10:02:49.842568+0000","last_undegraded":"2024-04-25T10:02:49.842568+0000","last_fullsized":"2024-04-25T10:02:49.842568+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.350657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842668+0000","last_change":"2024-04-25T10:02:49.836609+0000","last_active":"2024-04-25T10:02:49.842668+0000","last_peered":"2024-04-25T10:02:49.842668+0000","last_clean":"2024-04-25T10:02:49.842668+0000","last_became_active":"2024-04-25T10:02:49.836304+0000","last_became_peered":"2024-04-25T10:02:49.836304+0000","last_unstale":"2024-04-25T10:02:49.842668+0000","last_undegraded":"2024-04-25T10:02:49.842668+0000","last_fullsized":"2024-04-25T10:02:49.842668+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:44.711338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843756+0000","last_change":"2024-04-25T10:02:49.835654+0000","last_active":"2024-04-25T10:02:49.843756+0000","last_peered":"2024-04-25T10:02:49.843756+0000","last_clean":"2024-04-25T10:02:49.843756+0000","last_became_active":"2024-04-25T10:02:49.835377+0000","last_became_peered":"2024-04-25T10:02:49.835377+0000","last_unstale":"2024-04-25T10:02:49.843756+0000","last_undegraded":"2024-04-25T10:02:49.843756+0000","last_fullsized":"2024-04-25T10:02:49.843756+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:35.960479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845975+0000","last_change":"2024-04-25T10:02:49.843381+0000","last_active":"2024-04-25T10:02:49.845975+0000","last_peered":"2024-04-25T10:02:49.845975+0000","last_clean":"2024-04-25T10:02:49.845975+0000","last_became_active":"2024-04-25T10:02:49.842321+0000","last_became_peered":"2024-04-25T10:02:49.842321+0000","last_unstale":"2024-04-25T10:02:49.845975+0000","last_undegraded":"2024-04-25T10:02:49.845975+0000","last_fullsized":"2024-04-25T10:02:49.845975+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:49.837087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842194+0000","last_change":"2024-04-25T10:02:49.841503+0000","last_active":"2024-04-25T10:02:49.842194+0000","last_peered":"2024-04-25T10:02:49.842194+0000","last_clean":"2024-04-25T10:02:49.842194+0000","last_became_active":"2024-04-25T10:02:49.841245+0000","last_became_peered":"2024-04-25T10:02:49.841245+0000","last_unstale":"2024-04-25T10:02:49.842194+0000","last_undegraded":"2024-04-25T10:02:49.842194+0000","last_fullsized":"2024-04-25T10:02:49.842194+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:45.889300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835911+0000","last_change":"2024-04-25T10:02:49.832441+0000","last_active":"2024-04-25T10:02:49.835911+0000","last_peered":"2024-04-25T10:02:49.835911+0000","last_clean":"2024-04-25T10:02:49.835911+0000","last_became_active":"2024-04-25T10:02:49.832161+0000","last_became_peered":"2024-04-25T10:02:49.832161+0000","last_unstale":"2024-04-25T10:02:49.835911+0000","last_undegraded":"2024-04-25T10:02:49.835911+0000","last_fullsized":"2024-04-25T10:02:49.835911+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:07.285630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839246+0000","last_change":"2024-04-25T10:02:49.837613+0000","last_active":"2024-04-25T10:02:49.839246+0000","last_peered":"2024-04-25T10:02:49.839246+0000","last_clean":"2024-04-25T10:02:49.839246+0000","last_became_active":"2024-04-25T10:02:49.837332+0000","last_became_peered":"2024-04-25T10:02:49.837332+0000","last_unstale":"2024-04-25T10:02:49.839246+0000","last_undegraded":"2024-04-25T10:02:49.839246+0000","last_fullsized":"2024-04-25T10:02:49.839246+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:12.143545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835936+0000","last_change":"2024-04-25T10:02:49.835003+0000","last_active":"2024-04-25T10:02:49.835936+0000","last_peered":"2024-04-25T10:02:49.835936+0000","last_clean":"2024-04-25T10:02:49.835936+0000","last_became_active":"2024-04-25T10:02:49.834748+0000","last_became_peered":"2024-04-25T10:02:49.834748+0000","last_unstale":"2024-04-25T10:02:49.835936+0000","last_undegraded":"2024-04-25T10:02:49.835936+0000","last_fullsized":"2024-04-25T10:02:49.835936+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:41.207015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840053+0000","last_change":"2024-04-25T10:02:49.832715+0000","last_active":"2024-04-25T10:02:49.840053+0000","last_peered":"2024-04-25T10:02:49.840053+0000","last_clean":"2024-04-25T10:02:49.840053+0000","last_became_active":"2024-04-25T10:02:49.832482+0000","last_became_peered":"2024-04-25T10:02:49.832482+0000","last_unstale":"2024-04-25T10:02:49.840053+0000","last_undegraded":"2024-04-25T10:02:49.840053+0000","last_fullsized":"2024-04-25T10:02:49.840053+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:06.208277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844562+0000","last_change":"2024-04-25T10:02:49.838816+0000","last_active":"2024-04-25T10:02:49.844562+0000","last_peered":"2024-04-25T10:02:49.844562+0000","last_clean":"2024-04-25T10:02:49.844562+0000","last_became_active":"2024-04-25T10:02:49.838488+0000","last_became_peered":"2024-04-25T10:02:49.838488+0000","last_unstale":"2024-04-25T10:02:49.844562+0000","last_undegraded":"2024-04-25T10:02:49.844562+0000","last_fullsized":"2024-04-25T10:02:49.844562+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:52.346670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845122+0000","last_change":"2024-04-25T10:02:49.843312+0000","last_active":"2024-04-25T10:02:49.845122+0000","last_peered":"2024-04-25T10:02:49.845122+0000","last_clean":"2024-04-25T10:02:49.845122+0000","last_became_active":"2024-04-25T10:02:49.843041+0000","last_became_peered":"2024-04-25T10:02:49.843041+0000","last_unstale":"2024-04-25T10:02:49.845122+0000","last_undegraded":"2024-04-25T10:02:49.845122+0000","last_fullsized":"2024-04-25T10:02:49.845122+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.299685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846718+0000","last_change":"2024-04-25T10:02:49.844459+0000","last_active":"2024-04-25T10:02:49.846718+0000","last_peered":"2024-04-25T10:02:49.846718+0000","last_clean":"2024-04-25T10:02:49.846718+0000","last_became_active":"2024-04-25T10:02:49.843914+0000","last_became_peered":"2024-04-25T10:02:49.843914+0000","last_unstale":"2024-04-25T10:02:49.846718+0000","last_undegraded":"2024-04-25T10:02:49.846718+0000","last_fullsized":"2024-04-25T10:02:49.846718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:01.845554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840370+0000","last_change":"2024-04-25T10:02:49.833068+0000","last_active":"2024-04-25T10:02:49.840370+0000","last_peered":"2024-04-25T10:02:49.840370+0000","last_clean":"2024-04-25T10:02:49.840370+0000","last_became_active":"2024-04-25T10:02:49.832617+0000","last_became_peered":"2024-04-25T10:02:49.832617+0000","last_unstale":"2024-04-25T10:02:49.840370+0000","last_undegraded":"2024-04-25T10:02:49.840370+0000","last_fullsized":"2024-04-25T10:02:49.840370+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:09.252258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845414+0000","last_change":"2024-04-25T10:02:49.844765+0000","last_active":"2024-04-25T10:02:49.845414+0000","last_peered":"2024-04-25T10:02:49.845414+0000","last_clean":"2024-04-25T10:02:49.845414+0000","last_became_active":"2024-04-25T10:02:49.844491+0000","last_became_peered":"2024-04-25T10:02:49.844491+0000","last_unstale":"2024-04-25T10:02:49.845414+0000","last_undegraded":"2024-04-25T10:02:49.845414+0000","last_fullsized":"2024-04-25T10:02:49.845414+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:44.711338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844432+0000","last_change":"2024-04-25T10:02:49.841764+0000","last_active":"2024-04-25T10:02:49.844432+0000","last_peered":"2024-04-25T10:02:49.844432+0000","last_clean":"2024-04-25T10:02:49.844432+0000","last_became_active":"2024-04-25T10:02:49.841508+0000","last_became_peered":"2024-04-25T10:02:49.841508+0000","last_unstale":"2024-04-25T10:02:49.844432+0000","last_undegraded":"2024-04-25T10:02:49.844432+0000","last_fullsized":"2024-04-25T10:02:49.844432+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:13.515109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834876+0000","last_change":"2024-04-25T10:02:49.832255+0000","last_active":"2024-04-25T10:02:49.834876+0000","last_peered":"2024-04-25T10:02:49.834876+0000","last_clean":"2024-04-25T10:02:49.834876+0000","last_became_active":"2024-04-25T10:02:49.831921+0000","last_became_peered":"2024-04-25T10:02:49.831921+0000","last_unstale":"2024-04-25T10:02:49.834876+0000","last_undegraded":"2024-04-25T10:02:49.834876+0000","last_fullsized":"2024-04-25T10:02:49.834876+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:13.515109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835074+0000","last_change":"2024-04-25T10:02:49.829789+0000","last_active":"2024-04-25T10:02:49.835074+0000","last_peered":"2024-04-25T10:02:49.835074+0000","last_clean":"2024-04-25T10:02:49.835074+0000","last_became_active":"2024-04-25T10:02:49.829522+0000","last_became_peered":"2024-04-25T10:02:49.829522+0000","last_unstale":"2024-04-25T10:02:49.835074+0000","last_undegraded":"2024-04-25T10:02:49.835074+0000","last_fullsized":"2024-04-25T10:02:49.835074+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:26.913838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834940+0000","last_change":"2024-04-25T10:02:49.833240+0000","last_active":"2024-04-25T10:02:49.834940+0000","last_peered":"2024-04-25T10:02:49.834940+0000","last_clean":"2024-04-25T10:02:49.834940+0000","last_became_active":"2024-04-25T10:02:49.832961+0000","last_became_peered":"2024-04-25T10:02:49.832961+0000","last_unstale":"2024-04-25T10:02:49.834940+0000","last_undegraded":"2024-04-25T10:02:49.834940+0000","last_fullsized":"2024-04-25T10:02:49.834940+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.299685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"132'4","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.851735+0000","last_change":"2024-04-25T10:02:48.828764+0000","last_active":"2024-04-25T10:02:49.851735+0000","last_peered":"2024-04-25T10:02:49.851735+0000","last_clean":"2024-04-25T10:02:49.851735+0000","last_became_active":"2024-04-25T10:02:48.828233+0000","last_became_peered":"2024-04-25T10:02:48.828233+0000","last_unstale":"2024-04-25T10:02:49.851735+0000","last_undegraded":"2024-04-25T10:02:49.851735+0000","last_fullsized":"2024-04-25T10:02:49.851735+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:48.046054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833056+0000","last_change":"2024-04-25T10:02:49.832355+0000","last_active":"2024-04-25T10:02:49.833056+0000","last_peered":"2024-04-25T10:02:49.833056+0000","last_clean":"2024-04-25T10:02:49.833056+0000","last_became_active":"2024-04-25T10:02:49.832116+0000","last_became_peered":"2024-04-25T10:02:49.832116+0000","last_unstale":"2024-04-25T10:02:49.833056+0000","last_undegraded":"2024-04-25T10:02:49.833056+0000","last_fullsized":"2024-04-25T10:02:49.833056+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:24.524473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.848489+0000","last_change":"2024-04-25T10:02:48.826403+0000","last_active":"2024-04-25T10:02:49.848489+0000","last_peered":"2024-04-25T10:02:49.848489+0000","last_clean":"2024-04-25T10:02:49.848489+0000","last_became_active":"2024-04-25T10:02:48.825714+0000","last_became_peered":"2024-04-25T10:02:48.825714+0000","last_unstale":"2024-04-25T10:02:49.848489+0000","last_undegraded":"2024-04-25T10:02:49.848489+0000","last_fullsized":"2024-04-25T10:02:49.848489+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:00.840377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835437+0000","last_change":"2024-04-25T10:02:49.834044+0000","last_active":"2024-04-25T10:02:49.835437+0000","last_peered":"2024-04-25T10:02:49.835437+0000","last_clean":"2024-04-25T10:02:49.835437+0000","last_became_active":"2024-04-25T10:02:49.833793+0000","last_became_peered":"2024-04-25T10:02:49.833793+0000","last_unstale":"2024-04-25T10:02:49.835437+0000","last_undegraded":"2024-04-25T10:02:49.835437+0000","last_fullsized":"2024-04-25T10:02:49.835437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:01.155606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.850590+0000","last_change":"2024-04-25T10:02:48.826757+0000","last_active":"2024-04-25T10:02:49.850590+0000","last_peered":"2024-04-25T10:02:49.850590+0000","last_clean":"2024-04-25T10:02:49.850590+0000","last_became_active":"2024-04-25T10:02:48.826295+0000","last_became_peered":"2024-04-25T10:02:48.826295+0000","last_unstale":"2024-04-25T10:02:49.850590+0000","last_undegraded":"2024-04-25T10:02:49.850590+0000","last_fullsized":"2024-04-25T10:02:49.850590+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:40.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843847+0000","last_change":"2024-04-25T10:02:49.843091+0000","last_active":"2024-04-25T10:02:49.843847+0000","last_peered":"2024-04-25T10:02:49.843847+0000","last_clean":"2024-04-25T10:02:49.843847+0000","last_became_active":"2024-04-25T10:02:49.842824+0000","last_became_peered":"2024-04-25T10:02:49.842824+0000","last_unstale":"2024-04-25T10:02:49.843847+0000","last_undegraded":"2024-04-25T10:02:49.843847+0000","last_fullsized":"2024-04-25T10:02:49.843847+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:28.350965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.852238+0000","last_change":"2024-04-25T10:02:48.821036+0000","last_active":"2024-04-25T10:02:49.852238+0000","last_peered":"2024-04-25T10:02:49.852238+0000","last_clean":"2024-04-25T10:02:49.852238+0000","last_became_active":"2024-04-25T10:02:48.820769+0000","last_became_peered":"2024-04-25T10:02:48.820769+0000","last_unstale":"2024-04-25T10:02:49.852238+0000","last_undegraded":"2024-04-25T10:02:49.852238+0000","last_fullsized":"2024-04-25T10:02:49.852238+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:15.407574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.835021+0000","last_change":"2024-04-25T10:02:49.832719+0000","last_active":"2024-04-25T10:02:49.835021+0000","last_peered":"2024-04-25T10:02:49.835021+0000","last_clean":"2024-04-25T10:02:49.835021+0000","last_became_active":"2024-04-25T10:02:49.832455+0000","last_became_peered":"2024-04-25T10:02:49.832455+0000","last_unstale":"2024-04-25T10:02:49.835021+0000","last_undegraded":"2024-04-25T10:02:49.835021+0000","last_fullsized":"2024-04-25T10:02:49.835021+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:19.597490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849488+0000","last_change":"2024-04-25T10:02:48.822773+0000","last_active":"2024-04-25T10:02:49.849488+0000","last_peered":"2024-04-25T10:02:49.849488+0000","last_clean":"2024-04-25T10:02:49.849488+0000","last_became_active":"2024-04-25T10:02:48.822535+0000","last_became_peered":"2024-04-25T10:02:48.822535+0000","last_unstale":"2024-04-25T10:02:49.849488+0000","last_undegraded":"2024-04-25T10:02:49.849488+0000","last_fullsized":"2024-04-25T10:02:49.849488+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:05.203099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.841576+0000","last_change":"2024-04-25T10:02:49.836677+0000","last_active":"2024-04-25T10:02:49.841576+0000","last_peered":"2024-04-25T10:02:49.841576+0000","last_clean":"2024-04-25T10:02:49.841576+0000","last_became_active":"2024-04-25T10:02:49.836402+0000","last_became_peered":"2024-04-25T10:02:49.836402+0000","last_unstale":"2024-04-25T10:02:49.841576+0000","last_undegraded":"2024-04-25T10:02:49.841576+0000","last_fullsized":"2024-04-25T10:02:49.841576+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:44.711338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.847742+0000","last_change":"2024-04-25T10:02:48.822313+0000","last_active":"2024-04-25T10:02:49.847742+0000","last_peered":"2024-04-25T10:02:49.847742+0000","last_clean":"2024-04-25T10:02:49.847742+0000","last_became_active":"2024-04-25T10:02:48.822039+0000","last_became_peered":"2024-04-25T10:02:48.822039+0000","last_unstale":"2024-04-25T10:02:49.847742+0000","last_undegraded":"2024-04-25T10:02:49.847742+0000","last_fullsized":"2024-04-25T10:02:49.847742+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:55.997076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845007+0000","last_change":"2024-04-25T10:02:49.843896+0000","last_active":"2024-04-25T10:02:49.845007+0000","last_peered":"2024-04-25T10:02:49.845007+0000","last_clean":"2024-04-25T10:02:49.845007+0000","last_became_active":"2024-04-25T10:02:49.843565+0000","last_became_peered":"2024-04-25T10:02:49.843565+0000","last_unstale":"2024-04-25T10:02:49.845007+0000","last_undegraded":"2024-04-25T10:02:49.845007+0000","last_fullsized":"2024-04-25T10:02:49.845007+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:48.991343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827555+0000","last_change":"2024-04-25T10:02:48.827407+0000","last_active":"2024-04-25T10:02:49.827555+0000","last_peered":"2024-04-25T10:02:49.827555+0000","last_clean":"2024-04-25T10:02:49.827555+0000","last_became_active":"2024-04-25T10:02:48.827117+0000","last_became_peered":"2024-04-25T10:02:48.827117+0000","last_unstale":"2024-04-25T10:02:49.827555+0000","last_undegraded":"2024-04-25T10:02:49.827555+0000","last_fullsized":"2024-04-25T10:02:49.827555+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:48.046054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846848+0000","last_change":"2024-04-25T10:02:49.844034+0000","last_active":"2024-04-25T10:02:49.846848+0000","last_peered":"2024-04-25T10:02:49.846848+0000","last_clean":"2024-04-25T10:02:49.846848+0000","last_became_active":"2024-04-25T10:02:49.843742+0000","last_became_peered":"2024-04-25T10:02:49.843742+0000","last_unstale":"2024-04-25T10:02:49.846848+0000","last_undegraded":"2024-04-25T10:02:49.846848+0000","last_fullsized":"2024-04-25T10:02:49.846848+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:57.002254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.836999+0000","last_change":"2024-04-25T10:02:48.830971+0000","last_active":"2024-04-25T10:02:49.836999+0000","last_peered":"2024-04-25T10:02:49.836999+0000","last_clean":"2024-04-25T10:02:49.836999+0000","last_became_active":"2024-04-25T10:02:48.830736+0000","last_became_peered":"2024-04-25T10:02:48.830736+0000","last_unstale":"2024-04-25T10:02:49.836999+0000","last_undegraded":"2024-04-25T10:02:49.836999+0000","last_fullsized":"2024-04-25T10:02:49.836999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:26.467368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839322+0000","last_change":"2024-04-25T10:02:49.838578+0000","last_active":"2024-04-25T10:02:49.839322+0000","last_peered":"2024-04-25T10:02:49.839322+0000","last_clean":"2024-04-25T10:02:49.839322+0000","last_became_active":"2024-04-25T10:02:49.838323+0000","last_became_peered":"2024-04-25T10:02:49.838323+0000","last_unstale":"2024-04-25T10:02:49.839322+0000","last_undegraded":"2024-04-25T10:02:49.839322+0000","last_fullsized":"2024-04-25T10:02:49.839322+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:53.037644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827088+0000","last_change":"2024-04-25T10:02:48.827636+0000","last_active":"2024-04-25T10:02:49.827088+0000","last_peered":"2024-04-25T10:02:49.827088+0000","last_clean":"2024-04-25T10:02:49.827088+0000","last_became_active":"2024-04-25T10:02:48.827360+0000","last_became_peered":"2024-04-25T10:02:48.827360+0000","last_unstale":"2024-04-25T10:02:49.827088+0000","last_undegraded":"2024-04-25T10:02:49.827088+0000","last_fullsized":"2024-04-25T10:02:49.827088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:26.294507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843942+0000","last_change":"2024-04-25T10:02:49.840669+0000","last_active":"2024-04-25T10:02:49.843942+0000","last_peered":"2024-04-25T10:02:49.843942+0000","last_clean":"2024-04-25T10:02:49.843942+0000","last_became_active":"2024-04-25T10:02:49.840413+0000","last_became_peered":"2024-04-25T10:02:49.840413+0000","last_unstale":"2024-04-25T10:02:49.843942+0000","last_undegraded":"2024-04-25T10:02:49.843942+0000","last_fullsized":"2024-04-25T10:02:49.843942+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:16.412752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.831880+0000","last_change":"2024-04-25T10:02:48.824375+0000","last_active":"2024-04-25T10:02:49.831880+0000","last_peered":"2024-04-25T10:02:49.831880+0000","last_clean":"2024-04-25T10:02:49.831880+0000","last_became_active":"2024-04-25T10:02:48.823657+0000","last_became_peered":"2024-04-25T10:02:48.823657+0000","last_unstale":"2024-04-25T10:02:49.831880+0000","last_undegraded":"2024-04-25T10:02:49.831880+0000","last_fullsized":"2024-04-25T10:02:49.831880+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:12.273179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844411+0000","last_change":"2024-04-25T10:02:49.837939+0000","last_active":"2024-04-25T10:02:49.844411+0000","last_peered":"2024-04-25T10:02:49.844411+0000","last_clean":"2024-04-25T10:02:49.844411+0000","last_became_active":"2024-04-25T10:02:49.837234+0000","last_became_peered":"2024-04-25T10:02:49.837234+0000","last_unstale":"2024-04-25T10:02:49.844411+0000","last_undegraded":"2024-04-25T10:02:49.844411+0000","last_fullsized":"2024-04-25T10:02:49.844411+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:54.280858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"132'3","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.852048+0000","last_change":"2024-04-25T10:02:48.826613+0000","last_active":"2024-04-25T10:02:49.852048+0000","last_peered":"2024-04-25T10:02:49.852048+0000","last_clean":"2024-04-25T10:02:49.852048+0000","last_became_active":"2024-04-25T10:02:48.826339+0000","last_became_peered":"2024-04-25T10:02:48.826339+0000","last_unstale":"2024-04-25T10:02:49.852048+0000","last_undegraded":"2024-04-25T10:02:49.852048+0000","last_fullsized":"2024-04-25T10:02:49.852048+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:19.883973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845058+0000","last_change":"2024-04-25T10:02:49.842361+0000","last_active":"2024-04-25T10:02:49.845058+0000","last_peered":"2024-04-25T10:02:49.845058+0000","last_clean":"2024-04-25T10:02:49.845058+0000","last_became_active":"2024-04-25T10:02:49.840889+0000","last_became_peered":"2024-04-25T10:02:49.840889+0000","last_unstale":"2024-04-25T10:02:49.845058+0000","last_undegraded":"2024-04-25T10:02:49.845058+0000","last_fullsized":"2024-04-25T10:02:49.845058+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.350657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"132'6","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.853888+0000","last_change":"2024-04-25T10:02:48.829174+0000","last_active":"2024-04-25T10:02:49.853888+0000","last_peered":"2024-04-25T10:02:49.853888+0000","last_clean":"2024-04-25T10:02:49.853888+0000","last_became_active":"2024-04-25T10:02:48.828904+0000","last_became_peered":"2024-04-25T10:02:48.828904+0000","last_unstale":"2024-04-25T10:02:49.853888+0000","last_undegraded":"2024-04-25T10:02:49.853888+0000","last_fullsized":"2024-04-25T10:02:49.853888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:00.150429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840392+0000","last_change":"2024-04-25T10:02:49.834584+0000","last_active":"2024-04-25T10:02:49.840392+0000","last_peered":"2024-04-25T10:02:49.840392+0000","last_clean":"2024-04-25T10:02:49.840392+0000","last_became_active":"2024-04-25T10:02:49.834315+0000","last_became_peered":"2024-04-25T10:02:49.834315+0000","last_unstale":"2024-04-25T10:02:49.840392+0000","last_undegraded":"2024-04-25T10:02:49.840392+0000","last_fullsized":"2024-04-25T10:02:49.840392+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:40.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849522+0000","last_change":"2024-04-25T10:02:48.823023+0000","last_active":"2024-04-25T10:02:49.849522+0000","last_peered":"2024-04-25T10:02:49.849522+0000","last_clean":"2024-04-25T10:02:49.849522+0000","last_became_active":"2024-04-25T10:02:48.822747+0000","last_became_peered":"2024-04-25T10:02:48.822747+0000","last_unstale":"2024-04-25T10:02:49.849522+0000","last_undegraded":"2024-04-25T10:02:49.849522+0000","last_fullsized":"2024-04-25T10:02:49.849522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:36.473074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.845679+0000","last_change":"2024-04-25T10:02:49.841886+0000","last_active":"2024-04-25T10:02:49.845679+0000","last_peered":"2024-04-25T10:02:49.845679+0000","last_clean":"2024-04-25T10:02:49.845679+0000","last_became_active":"2024-04-25T10:02:49.841602+0000","last_became_peered":"2024-04-25T10:02:49.841602+0000","last_unstale":"2024-04-25T10:02:49.845679+0000","last_undegraded":"2024-04-25T10:02:49.845679+0000","last_fullsized":"2024-04-25T10:02:49.845679+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:31.245807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.831528+0000","last_change":"2024-04-25T10:02:48.830219+0000","last_active":"2024-04-25T10:02:49.831528+0000","last_peered":"2024-04-25T10:02:49.831528+0000","last_clean":"2024-04-25T10:02:49.831528+0000","last_became_active":"2024-04-25T10:02:48.829919+0000","last_became_peered":"2024-04-25T10:02:48.829919+0000","last_unstale":"2024-04-25T10:02:49.831528+0000","last_undegraded":"2024-04-25T10:02:49.831528+0000","last_fullsized":"2024-04-25T10:02:49.831528+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:55.248308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833376+0000","last_change":"2024-04-25T10:02:49.831993+0000","last_active":"2024-04-25T10:02:49.833376+0000","last_peered":"2024-04-25T10:02:49.833376+0000","last_clean":"2024-04-25T10:02:49.833376+0000","last_became_active":"2024-04-25T10:02:49.831728+0000","last_became_peered":"2024-04-25T10:02:49.831728+0000","last_unstale":"2024-04-25T10:02:49.833376+0000","last_undegraded":"2024-04-25T10:02:49.833376+0000","last_fullsized":"2024-04-25T10:02:49.833376+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:48.358626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"132'8","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.854283+0000","last_change":"2024-04-25T10:02:48.824872+0000","last_active":"2024-04-25T10:02:49.854283+0000","last_peered":"2024-04-25T10:02:49.854283+0000","last_clean":"2024-04-25T10:02:49.854283+0000","last_became_active":"2024-04-25T10:02:48.824143+0000","last_became_peered":"2024-04-25T10:02:48.824143+0000","last_unstale":"2024-04-25T10:02:49.854283+0000","last_undegraded":"2024-04-25T10:02:49.854283+0000","last_fullsized":"2024-04-25T10:02:49.854283+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:53.275680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844735+0000","last_change":"2024-04-25T10:02:49.841414+0000","last_active":"2024-04-25T10:02:49.844735+0000","last_peered":"2024-04-25T10:02:49.844735+0000","last_clean":"2024-04-25T10:02:49.844735+0000","last_became_active":"2024-04-25T10:02:49.841142+0000","last_became_peered":"2024-04-25T10:02:49.841142+0000","last_unstale":"2024-04-25T10:02:49.844735+0000","last_undegraded":"2024-04-25T10:02:49.844735+0000","last_fullsized":"2024-04-25T10:02:49.844735+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:06.208277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849273+0000","last_change":"2024-04-25T10:02:48.824817+0000","last_active":"2024-04-25T10:02:49.849273+0000","last_peered":"2024-04-25T10:02:49.849273+0000","last_clean":"2024-04-25T10:02:49.849273+0000","last_became_active":"2024-04-25T10:02:48.824432+0000","last_became_peered":"2024-04-25T10:02:48.824432+0000","last_unstale":"2024-04-25T10:02:49.849273+0000","last_undegraded":"2024-04-25T10:02:49.849273+0000","last_fullsized":"2024-04-25T10:02:49.849273+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:00.345479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846833+0000","last_change":"2024-04-25T10:02:49.845032+0000","last_active":"2024-04-25T10:02:49.846833+0000","last_peered":"2024-04-25T10:02:49.846833+0000","last_clean":"2024-04-25T10:02:49.846833+0000","last_became_active":"2024-04-25T10:02:49.844756+0000","last_became_peered":"2024-04-25T10:02:49.844756+0000","last_unstale":"2024-04-25T10:02:49.846833+0000","last_undegraded":"2024-04-25T10:02:49.846833+0000","last_fullsized":"2024-04-25T10:02:49.846833+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:48.358626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.850393+0000","last_change":"2024-04-25T10:02:48.828930+0000","last_active":"2024-04-25T10:02:49.850393+0000","last_peered":"2024-04-25T10:02:49.850393+0000","last_clean":"2024-04-25T10:02:49.850393+0000","last_became_active":"2024-04-25T10:02:48.828659+0000","last_became_peered":"2024-04-25T10:02:48.828659+0000","last_unstale":"2024-04-25T10:02:49.850393+0000","last_undegraded":"2024-04-25T10:02:49.850393+0000","last_fullsized":"2024-04-25T10:02:49.850393+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:39.338494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.846293+0000","last_change":"2024-04-25T10:02:49.843998+0000","last_active":"2024-04-25T10:02:49.846293+0000","last_peered":"2024-04-25T10:02:49.846293+0000","last_clean":"2024-04-25T10:02:49.846293+0000","last_became_active":"2024-04-25T10:02:49.843711+0000","last_became_peered":"2024-04-25T10:02:49.843711+0000","last_unstale":"2024-04-25T10:02:49.846293+0000","last_undegraded":"2024-04-25T10:02:49.846293+0000","last_fullsized":"2024-04-25T10:02:49.846293+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:02.505446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"132'1","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.850736+0000","last_change":"2024-04-25T10:02:48.827675+0000","last_active":"2024-04-25T10:02:49.850736+0000","last_peered":"2024-04-25T10:02:49.850736+0000","last_clean":"2024-04-25T10:02:49.850736+0000","last_became_active":"2024-04-25T10:02:48.827288+0000","last_became_peered":"2024-04-25T10:02:48.827288+0000","last_unstale":"2024-04-25T10:02:49.850736+0000","last_undegraded":"2024-04-25T10:02:49.850736+0000","last_fullsized":"2024-04-25T10:02:49.850736+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:45.235658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839820+0000","last_change":"2024-04-25T10:02:49.838492+0000","last_active":"2024-04-25T10:02:49.839820+0000","last_peered":"2024-04-25T10:02:49.839820+0000","last_clean":"2024-04-25T10:02:49.839820+0000","last_became_active":"2024-04-25T10:02:49.838242+0000","last_became_peered":"2024-04-25T10:02:49.838242+0000","last_unstale":"2024-04-25T10:02:49.839820+0000","last_undegraded":"2024-04-25T10:02:49.839820+0000","last_fullsized":"2024-04-25T10:02:49.839820+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:22.290974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833022+0000","last_change":"2024-04-25T10:02:48.826489+0000","last_active":"2024-04-25T10:02:49.833022+0000","last_peered":"2024-04-25T10:02:49.833022+0000","last_clean":"2024-04-25T10:02:49.833022+0000","last_became_active":"2024-04-25T10:02:48.826208+0000","last_became_peered":"2024-04-25T10:02:48.826208+0000","last_unstale":"2024-04-25T10:02:49.833022+0000","last_undegraded":"2024-04-25T10:02:49.833022+0000","last_fullsized":"2024-04-25T10:02:49.833022+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:55.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.839648+0000","last_change":"2024-04-25T10:02:49.834066+0000","last_active":"2024-04-25T10:02:49.839648+0000","last_peered":"2024-04-25T10:02:49.839648+0000","last_clean":"2024-04-25T10:02:49.839648+0000","last_became_active":"2024-04-25T10:02:49.833795+0000","last_became_peered":"2024-04-25T10:02:49.833795+0000","last_unstale":"2024-04-25T10:02:49.839648+0000","last_undegraded":"2024-04-25T10:02:49.839648+0000","last_fullsized":"2024-04-25T10:02:49.839648+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.190728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.851088+0000","last_change":"2024-04-25T10:02:48.826053+0000","last_active":"2024-04-25T10:02:49.851088+0000","last_peered":"2024-04-25T10:02:49.851088+0000","last_clean":"2024-04-25T10:02:49.851088+0000","last_became_active":"2024-04-25T10:02:48.825783+0000","last_became_peered":"2024-04-25T10:02:48.825783+0000","last_unstale":"2024-04-25T10:02:49.851088+0000","last_undegraded":"2024-04-25T10:02:49.851088+0000","last_fullsized":"2024-04-25T10:02:49.851088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:45.235658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.832983+0000","last_change":"2024-04-25T10:02:49.831978+0000","last_active":"2024-04-25T10:02:49.832983+0000","last_peered":"2024-04-25T10:02:49.832983+0000","last_clean":"2024-04-25T10:02:49.832983+0000","last_became_active":"2024-04-25T10:02:49.831651+0000","last_became_peered":"2024-04-25T10:02:49.831651+0000","last_unstale":"2024-04-25T10:02:49.832983+0000","last_undegraded":"2024-04-25T10:02:49.832983+0000","last_fullsized":"2024-04-25T10:02:49.832983+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:39.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.828769+0000","last_change":"2024-04-25T10:02:48.820305+0000","last_active":"2024-04-25T10:02:49.828769+0000","last_peered":"2024-04-25T10:02:49.828769+0000","last_clean":"2024-04-25T10:02:49.828769+0000","last_became_active":"2024-04-25T10:02:48.820028+0000","last_became_peered":"2024-04-25T10:02:48.820028+0000","last_unstale":"2024-04-25T10:02:49.828769+0000","last_undegraded":"2024-04-25T10:02:49.828769+0000","last_fullsized":"2024-04-25T10:02:49.828769+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:37.215984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844987+0000","last_change":"2024-04-25T10:02:49.843680+0000","last_active":"2024-04-25T10:02:49.844987+0000","last_peered":"2024-04-25T10:02:49.844987+0000","last_clean":"2024-04-25T10:02:49.844987+0000","last_became_active":"2024-04-25T10:02:49.839409+0000","last_became_peered":"2024-04-25T10:02:49.839409+0000","last_unstale":"2024-04-25T10:02:49.844987+0000","last_undegraded":"2024-04-25T10:02:49.844987+0000","last_fullsized":"2024-04-25T10:02:49.844987+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:27.497333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.836391+0000","last_change":"2024-04-25T10:02:48.828163+0000","last_active":"2024-04-25T10:02:49.836391+0000","last_peered":"2024-04-25T10:02:49.836391+0000","last_clean":"2024-04-25T10:02:49.836391+0000","last_became_active":"2024-04-25T10:02:48.827888+0000","last_became_peered":"2024-04-25T10:02:48.827888+0000","last_unstale":"2024-04-25T10:02:49.836391+0000","last_undegraded":"2024-04-25T10:02:49.836391+0000","last_fullsized":"2024-04-25T10:02:49.836391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:05.266151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.840608+0000","last_change":"2024-04-25T10:02:49.839919+0000","last_active":"2024-04-25T10:02:49.840608+0000","last_peered":"2024-04-25T10:02:49.840608+0000","last_clean":"2024-04-25T10:02:49.840608+0000","last_became_active":"2024-04-25T10:02:49.839561+0000","last_became_peered":"2024-04-25T10:02:49.839561+0000","last_unstale":"2024-04-25T10:02:49.840608+0000","last_undegraded":"2024-04-25T10:02:49.840608+0000","last_fullsized":"2024-04-25T10:02:49.840608+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:06.271328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.828057+0000","last_change":"2024-04-25T10:02:48.822741+0000","last_active":"2024-04-25T10:02:49.828057+0000","last_peered":"2024-04-25T10:02:49.828057+0000","last_clean":"2024-04-25T10:02:49.828057+0000","last_became_active":"2024-04-25T10:02:48.822505+0000","last_became_peered":"2024-04-25T10:02:48.822505+0000","last_unstale":"2024-04-25T10:02:49.828057+0000","last_undegraded":"2024-04-25T10:02:49.828057+0000","last_fullsized":"2024-04-25T10:02:49.828057+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:51.341492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.843766+0000","last_change":"2024-04-25T10:02:49.834958+0000","last_active":"2024-04-25T10:02:49.843766+0000","last_peered":"2024-04-25T10:02:49.843766+0000","last_clean":"2024-04-25T10:02:49.843766+0000","last_became_active":"2024-04-25T10:02:49.834691+0000","last_became_peered":"2024-04-25T10:02:49.834691+0000","last_unstale":"2024-04-25T10:02:49.843766+0000","last_undegraded":"2024-04-25T10:02:49.843766+0000","last_fullsized":"2024-04-25T10:02:49.843766+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:38.442612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"132'2","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.851974+0000","last_change":"2024-04-25T10:02:48.826738+0000","last_active":"2024-04-25T10:02:49.851974+0000","last_peered":"2024-04-25T10:02:49.851974+0000","last_clean":"2024-04-25T10:02:49.851974+0000","last_became_active":"2024-04-25T10:02:48.825795+0000","last_became_peered":"2024-04-25T10:02:48.825795+0000","last_unstale":"2024-04-25T10:02:49.851974+0000","last_undegraded":"2024-04-25T10:02:49.851974+0000","last_fullsized":"2024-04-25T10:02:49.851974+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:30.862171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844476+0000","last_change":"2024-04-25T10:02:49.840172+0000","last_active":"2024-04-25T10:02:49.844476+0000","last_peered":"2024-04-25T10:02:49.844476+0000","last_clean":"2024-04-25T10:02:49.844476+0000","last_became_active":"2024-04-25T10:02:49.839886+0000","last_became_peered":"2024-04-25T10:02:49.839886+0000","last_unstale":"2024-04-25T10:02:49.844476+0000","last_undegraded":"2024-04-25T10:02:49.844476+0000","last_fullsized":"2024-04-25T10:02:49.844476+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:46.240835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827079+0000","last_change":"2024-04-25T10:02:48.821994+0000","last_active":"2024-04-25T10:02:49.827079+0000","last_peered":"2024-04-25T10:02:49.827079+0000","last_clean":"2024-04-25T10:02:49.827079+0000","last_became_active":"2024-04-25T10:02:48.819410+0000","last_became_peered":"2024-04-25T10:02:48.819410+0000","last_unstale":"2024-04-25T10:02:49.827079+0000","last_undegraded":"2024-04-25T10:02:49.827079+0000","last_fullsized":"2024-04-25T10:02:49.827079+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:54.528247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.847695+0000","last_change":"2024-04-25T10:02:49.846142+0000","last_active":"2024-04-25T10:02:49.847695+0000","last_peered":"2024-04-25T10:02:49.847695+0000","last_clean":"2024-04-25T10:02:49.847695+0000","last_became_active":"2024-04-25T10:02:49.845835+0000","last_became_peered":"2024-04-25T10:02:49.845835+0000","last_unstale":"2024-04-25T10:02:49.847695+0000","last_undegraded":"2024-04-25T10:02:49.847695+0000","last_fullsized":"2024-04-25T10:02:49.847695+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:56.709303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827025+0000","last_change":"2024-04-25T10:02:48.821477+0000","last_active":"2024-04-25T10:02:49.827025+0000","last_peered":"2024-04-25T10:02:49.827025+0000","last_clean":"2024-04-25T10:02:49.827025+0000","last_became_active":"2024-04-25T10:02:48.821201+0000","last_became_peered":"2024-04-25T10:02:48.821201+0000","last_unstale":"2024-04-25T10:02:49.827025+0000","last_undegraded":"2024-04-25T10:02:49.827025+0000","last_fullsized":"2024-04-25T10:02:49.827025+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:51.341492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.838635+0000","last_change":"2024-04-25T10:02:49.837705+0000","last_active":"2024-04-25T10:02:49.838635+0000","last_peered":"2024-04-25T10:02:49.838635+0000","last_clean":"2024-04-25T10:02:49.838635+0000","last_became_active":"2024-04-25T10:02:49.837452+0000","last_became_peered":"2024-04-25T10:02:49.837452+0000","last_unstale":"2024-04-25T10:02:49.838635+0000","last_undegraded":"2024-04-25T10:02:49.838635+0000","last_fullsized":"2024-04-25T10:02:49.838635+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:49.938913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.829666+0000","last_change":"2024-04-25T10:02:48.820572+0000","last_active":"2024-04-25T10:02:49.829666+0000","last_peered":"2024-04-25T10:02:49.829666+0000","last_clean":"2024-04-25T10:02:49.829666+0000","last_became_active":"2024-04-25T10:02:48.820294+0000","last_became_peered":"2024-04-25T10:02:48.820294+0000","last_unstale":"2024-04-25T10:02:49.829666+0000","last_undegraded":"2024-04-25T10:02:49.829666+0000","last_fullsized":"2024-04-25T10:02:49.829666+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:54.528247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842144+0000","last_change":"2024-04-25T10:02:49.835961+0000","last_active":"2024-04-25T10:02:49.842144+0000","last_peered":"2024-04-25T10:02:49.842144+0000","last_clean":"2024-04-25T10:02:49.842144+0000","last_became_active":"2024-04-25T10:02:49.835186+0000","last_became_peered":"2024-04-25T10:02:49.835186+0000","last_unstale":"2024-04-25T10:02:49.842144+0000","last_undegraded":"2024-04-25T10:02:49.842144+0000","last_fullsized":"2024-04-25T10:02:49.842144+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:27.299685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827520+0000","last_change":"2024-04-25T10:02:48.828657+0000","last_active":"2024-04-25T10:02:49.827520+0000","last_peered":"2024-04-25T10:02:49.827520+0000","last_clean":"2024-04-25T10:02:49.827520+0000","last_became_active":"2024-04-25T10:02:48.828398+0000","last_became_peered":"2024-04-25T10:02:48.828398+0000","last_unstale":"2024-04-25T10:02:49.827520+0000","last_undegraded":"2024-04-25T10:02:49.827520+0000","last_fullsized":"2024-04-25T10:02:49.827520+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:43.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842009+0000","last_change":"2024-04-25T10:02:49.835574+0000","last_active":"2024-04-25T10:02:49.842009+0000","last_peered":"2024-04-25T10:02:49.842009+0000","last_clean":"2024-04-25T10:02:49.842009+0000","last_became_active":"2024-04-25T10:02:49.835307+0000","last_became_peered":"2024-04-25T10:02:49.835307+0000","last_unstale":"2024-04-25T10:02:49.842009+0000","last_undegraded":"2024-04-25T10:02:49.842009+0000","last_fullsized":"2024-04-25T10:02:49.842009+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:19.597490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.834432+0000","last_change":"2024-04-25T10:02:48.828042+0000","last_active":"2024-04-25T10:02:49.834432+0000","last_peered":"2024-04-25T10:02:49.834432+0000","last_clean":"2024-04-25T10:02:49.834432+0000","last_became_active":"2024-04-25T10:02:48.827542+0000","last_became_peered":"2024-04-25T10:02:48.827542+0000","last_unstale":"2024-04-25T10:02:49.834432+0000","last_undegraded":"2024-04-25T10:02:49.834432+0000","last_fullsized":"2024-04-25T10:02:49.834432+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:06.280452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.842114+0000","last_change":"2024-04-25T10:02:49.839532+0000","last_active":"2024-04-25T10:02:49.842114+0000","last_peered":"2024-04-25T10:02:49.842114+0000","last_clean":"2024-04-25T10:02:49.842114+0000","last_became_active":"2024-04-25T10:02:49.839271+0000","last_became_peered":"2024-04-25T10:02:49.839271+0000","last_unstale":"2024-04-25T10:02:49.842114+0000","last_undegraded":"2024-04-25T10:02:49.842114+0000","last_fullsized":"2024-04-25T10:02:49.842114+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:07.285630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827722+0000","last_change":"2024-04-25T10:02:48.821769+0000","last_active":"2024-04-25T10:02:49.827722+0000","last_peered":"2024-04-25T10:02:49.827722+0000","last_clean":"2024-04-25T10:02:49.827722+0000","last_became_active":"2024-04-25T10:02:48.821490+0000","last_became_peered":"2024-04-25T10:02:48.821490+0000","last_unstale":"2024-04-25T10:02:49.827722+0000","last_undegraded":"2024-04-25T10:02:49.827722+0000","last_fullsized":"2024-04-25T10:02:49.827722+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:26.467368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.849201+0000","last_change":"2024-04-25T10:02:49.846635+0000","last_active":"2024-04-25T10:02:49.849201+0000","last_peered":"2024-04-25T10:02:49.849201+0000","last_clean":"2024-04-25T10:02:49.849201+0000","last_became_active":"2024-04-25T10:02:49.846302+0000","last_became_peered":"2024-04-25T10:02:49.846302+0000","last_unstale":"2024-04-25T10:02:49.849201+0000","last_undegraded":"2024-04-25T10:02:49.849201+0000","last_fullsized":"2024-04-25T10:02:49.849201+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.190728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.827435+0000","last_change":"2024-04-25T10:02:48.828150+0000","last_active":"2024-04-25T10:02:49.827435+0000","last_peered":"2024-04-25T10:02:49.827435+0000","last_clean":"2024-04-25T10:02:49.827435+0000","last_became_active":"2024-04-25T10:02:48.827881+0000","last_became_peered":"2024-04-25T10:02:48.827881+0000","last_unstale":"2024-04-25T10:02:49.827435+0000","last_undegraded":"2024-04-25T10:02:49.827435+0000","last_fullsized":"2024-04-25T10:02:49.827435+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:01.500269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.844682+0000","last_change":"2024-04-25T10:02:49.839068+0000","last_active":"2024-04-25T10:02:49.844682+0000","last_peered":"2024-04-25T10:02:49.844682+0000","last_clean":"2024-04-25T10:02:49.844682+0000","last_became_active":"2024-04-25T10:02:49.837711+0000","last_became_peered":"2024-04-25T10:02:49.837711+0000","last_unstale":"2024-04-25T10:02:49.844682+0000","last_undegraded":"2024-04-25T10:02:49.844682+0000","last_fullsized":"2024-04-25T10:02:49.844682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:48.806818+0000","last_clean_scrub_stamp":"2024-04-25T10:02:48.806818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:55.533425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833954+0000","last_change":"2024-04-25T10:02:48.827809+0000","last_active":"2024-04-25T10:02:49.833954+0000","last_peered":"2024-04-25T10:02:49.833954+0000","last_clean":"2024-04-25T10:02:49.833954+0000","last_became_active":"2024-04-25T10:02:48.827164+0000","last_became_peered":"2024-04-25T10:02:48.827164+0000","last_unstale":"2024-04-25T10:02:49.833954+0000","last_undegraded":"2024-04-25T10:02:49.833954+0000","last_fullsized":"2024-04-25T10:02:49.833954+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:47.801640+0000","last_clean_scrub_stamp":"2024-04-25T10:02:47.801640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:43.706160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":376,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T10:02:49.833314+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:02:49.833314+0000","last_peered":"2024-04-25T10:02:49.833314+0000","last_clean":"2024-04-25T10:02:49.833314+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:02:49.833314+0000","last_undegraded":"2024-04-25T10:02:49.833314+0000","last_fullsized":"2024-04-25T10:02:49.833314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29204,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342636,"statfs":{"total":96636764160,"available":96606859264,"internally_reserved":0,"allocated":2232320,"data_stored":1351375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50828,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321012,"statfs":{"total":96636764160,"available":96584716288,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46692,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325148,"statfs":{"total":96636764160,"available":96588951552,"internally_reserved":0,"allocated":1654784,"data_stored":768196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54988,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316852,"statfs":{"total":96636764160,"available":96580456448,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54988,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316852,"statfs":{"total":96636764160,"available":96580456448,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50832,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321008,"statfs":{"total":96636764160,"available":96584712192,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50832,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321008,"statfs":{"total":96636764160,"available":96584712192,"internally_reserved":0,"allocated":1638400,"data_stored":761007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51416,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320424,"statfs":{"total":96636764160,"available":96584114176,"internally_reserved":0,"allocated":2232320,"data_stored":1351375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:53.132 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:53.133 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:02:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.269+0000 7f8560390640 1 Processor -- start 2024-04-25T10:02:53.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.269+0000 7f8560390640 1 -- start start 2024-04-25T10:02:53.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.270+0000 7f8560390640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f855806b2f0 0x7f85580aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.270+0000 7f8560390640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f855815c970 0x7f855815d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.270+0000 7f8560390640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855815cde0 con 0x7f85581454a0 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.270+0000 7f8560390640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85580af540 con 0x7f855806b2f0 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.270+0000 7f8560390640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85580af820 con 0x7f855815c970 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f855806b2f0 0x7f85580aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f855806b2f0 0x7f85580aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34362/0 (socket says 172.21.15.148:34362) 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 -- 172.21.15.148:0/2378254359 learned_addr learned my addr 172.21.15.148:0/2378254359 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855e92d640 1 --2- 172.21.15.148:0/2378254359 >> v2:172.21.15.188:3300/0 conn(0x7f855815c970 0x7f855815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 -- 172.21.15.148:0/2378254359 >> v2:172.21.15.188:3300/0 conn(0x7f855815c970 msgr2=0x7f855815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 --2- 172.21.15.148:0/2378254359 >> v2:172.21.15.188:3300/0 conn(0x7f855815c970 0x7f855815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 -- 172.21.15.148:0/2378254359 >> v1:172.21.15.188:6789/0 conn(0x7f85581454a0 legacy=0x7f855806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 -- 172.21.15.148:0/2378254359 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580afb00 con 0x7f855806b2f0 2024-04-25T10:02:53.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.271+0000 7f855d92b640 1 --2- 172.21.15.148:0/2378254359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855806b2f0 0x7f85580aee60 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f85580719d0 tx=0x7f854802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f2b828f669f5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f855d12a640 1 -- 172.21.15.148:0/2378254359 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548037070 con 0x7f855806b2f0 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f855d12a640 1 -- 172.21.15.148:0/2378254359 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8548032440 con 0x7f855806b2f0 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f855d12a640 1 -- 172.21.15.148:0/2378254359 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548032810 con 0x7f855806b2f0 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 -- 172.21.15.148:0/2378254359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855806b2f0 msgr2=0x7f85580aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 --2- 172.21.15.148:0/2378254359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855806b2f0 0x7f85580aee60 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f85580719d0 tx=0x7f854802a230 comp rx=0 tx=0).stop 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 -- 172.21.15.148:0/2378254359 shutdown_connections 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 --2- 172.21.15.148:0/2378254359 >> v2:172.21.15.188:3300/0 conn(0x7f855815c970 0x7f855815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 --2- 172.21.15.148:0/2378254359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f855806b2f0 0x7f85580aee60 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 -- 172.21.15.148:0/2378254359 >> 172.21.15.148:0/2378254359 conn(0x7f85580b5af0 msgr2=0x7f85580b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 -- 172.21.15.148:0/2378254359 shutdown_connections 2024-04-25T10:02:53.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.272+0000 7f8560390640 1 -- 172.21.15.148:0/2378254359 wait complete. 2024-04-25T10:02:53.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.273+0000 7f8560390640 1 Processor -- start 2024-04-25T10:02:53.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.273+0000 7f8560390640 1 -- start start 2024-04-25T10:02:53.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f8560390640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f855806b2f0 0x7f85581e7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f8560390640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 0x7f8558144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f8560390640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855815c970 0x7f85581e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f8560390640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855815db50 con 0x7f85581454a0 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f8560390640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855815d9d0 con 0x7f855815c970 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f8560390640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855815dcd0 con 0x7f855806b2f0 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 0x7f8558144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 0x7f8558144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:34366/0 (socket says 172.21.15.148:34366) 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 -- 172.21.15.148:0/1942828388 learned_addr learned my addr 172.21.15.148:0/1942828388 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855e92d640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855815c970 0x7f85581e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 -- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f855806b2f0 msgr2=0x7f85581e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f855806b2f0 0x7f85581e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 -- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855815c970 msgr2=0x7f85581e5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.274+0000 7f855d92b640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855815c970 0x7f85581e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f855d92b640 1 -- 172.21.15.148:0/1942828388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85581f1bb0 con 0x7f85581454a0 2024-04-25T10:02:53.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f855d92b640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 0x7f8558144670 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f854802a270 tx=0x7f85480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548041020 con 0x7f85581454a0 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480306c0 con 0x7f85581454a0 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85581e5a70 con 0x7f85581454a0 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548037040 con 0x7f85581454a0 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.275+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581e6030 con 0x7f85581454a0 2024-04-25T10:02:53.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.276+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855806cc80 con 0x7f85581454a0 2024-04-25T10:02:53.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.276+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8548044020 con 0x7f85581454a0 2024-04-25T10:02:53.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.277+0000 7f854effd640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f853c06b500 0x7f853c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.277+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85480aaab0 con 0x7f85581454a0 2024-04-25T10:02:53.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.277+0000 7f855e12c640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f853c06b500 0x7f853c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.277+0000 7f855e12c640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f853c06b500 0x7f853c06d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8554003cc0 tx=0x7f85540079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.281+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f854804b030 con 0x7f85581454a0 2024-04-25T10:02:53.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.517+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8558073a00 con 0x7f85581454a0 2024-04-25T10:02:53.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.519+0000 7f854effd640 1 -- 172.21.15.148:0/1942828388 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8548079830 con 0x7f85581454a0 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f853c06b500 msgr2=0x7f853c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f853c06b500 0x7f853c06d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8554003cc0 tx=0x7f85540079b0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 msgr2=0x7f8558144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 0x7f8558144670 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f854802a270 tx=0x7f85480046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 shutdown_connections 2024-04-25T10:02:53.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f853c06b500 0x7f853c06d9c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f855815c970 0x7f85581e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85581454a0 0x7f8558144670 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 --2- 172.21.15.148:0/1942828388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f855806b2f0 0x7f85581e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 >> 172.21.15.148:0/1942828388 conn(0x7f85580b5af0 msgr2=0x7f855815fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 shutdown_connections 2024-04-25T10:02:53.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.521+0000 7f8560390640 1 -- 172.21.15.148:0/1942828388 wait complete. 2024-04-25T10:02:53.537 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:53.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 Processor -- start 2024-04-25T10:02:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 -- start start 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb43015b320 0x7fb430071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb430071ed0 con 0x7fb43013d190 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb430071cb0 con 0x7fb43015b320 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.630+0000 7fb436a6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43013d570 con 0x7fb4300cc6c0 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb43015b320 0x7fb430071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb43015b320 0x7fb430071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44110/0 (socket says 172.21.15.148:44110) 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 -- 172.21.15.148:0/3143208023 learned_addr learned my addr 172.21.15.148:0/3143208023 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb42ffff640 1 --2- 172.21.15.148:0/3143208023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 -- 172.21.15.148:0/3143208023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 msgr2=0x7fb430071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 --2- 172.21.15.148:0/3143208023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 -- 172.21.15.148:0/3143208023 >> v1:172.21.15.188:6789/0 conn(0x7fb43013d190 legacy=0x7fb4300cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.631+0000 7fb43500c640 1 -- 172.21.15.148:0/3143208023 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43015a840 con 0x7fb43015b320 2024-04-25T10:02:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb43500c640 1 --2- 172.21.15.148:0/3143208023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb430071770 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d150 tx=0x7fb42802c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b817a3344f5bd4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb42f7fe640 1 -- 172.21.15.148:0/3143208023 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42803b070 con 0x7fb43015b320 2024-04-25T10:02:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb42f7fe640 1 -- 172.21.15.148:0/3143208023 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428033670 con 0x7fb43015b320 2024-04-25T10:02:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb42f7fe640 1 -- 172.21.15.148:0/3143208023 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428033a40 con 0x7fb43015b320 2024-04-25T10:02:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 -- 172.21.15.148:0/3143208023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 msgr2=0x7fb430071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 --2- 172.21.15.148:0/3143208023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb430071770 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d150 tx=0x7fb42802c840 comp rx=0 tx=0).stop 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 -- 172.21.15.148:0/3143208023 shutdown_connections 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 --2- 172.21.15.148:0/3143208023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb430071770 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 --2- 172.21.15.148:0/3143208023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 -- 172.21.15.148:0/3143208023 >> 172.21.15.148:0/3143208023 conn(0x7fb43006abe0 msgr2=0x7fb43006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 -- 172.21.15.148:0/3143208023 shutdown_connections 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.632+0000 7fb436a6f640 1 -- 172.21.15.148:0/3143208023 wait complete. 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.633+0000 7fb436a6f640 1 Processor -- start 2024-04-25T10:02:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.633+0000 7fb436a6f640 1 -- start start 2024-04-25T10:02:53.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb436a6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb436a6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb43013d190 0x7fb430139ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb436a6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb4300b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb436a6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300cf620 con 0x7fb43015b320 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb436a6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43015ae00 con 0x7fb43013d190 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb436a6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300cf7a0 con 0x7fb4300cc6c0 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb43500c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb4300b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb43500c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb4300b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44124/0 (socket says 172.21.15.148:44124) 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.634+0000 7fb43500c640 1 -- 172.21.15.148:0/2783365133 learned_addr learned my addr 172.21.15.148:0/2783365133 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb42ffff640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb43013d190 0x7fb430139ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43480b640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43500c640 1 -- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 msgr2=0x7fb430156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43500c640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43500c640 1 -- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb43013d190 msgr2=0x7fb430139ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43500c640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb43013d190 0x7fb430139ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43500c640 1 -- 172.21.15.148:0/2783365133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4301556c0 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb43500c640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb4300b2420 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d270 tx=0x7fb428004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428046070 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42800b330 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4301559a0 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42803b070 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.635+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430155f60 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.636+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb430073990 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.637+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb428049070 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.637+0000 7fb42d7fa640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.637+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4280aa650 con 0x7fb43015b320 2024-04-25T10:02:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.637+0000 7fb43480b640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.637+0000 7fb43480b640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb41406b4d0 0x7fb41406d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb424003d90 tx=0x7fb424009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.641+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb428079340 con 0x7fb43015b320 2024-04-25T10:02:53.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.849+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4300cfeb0 con 0x7fb43015b320 2024-04-25T10:02:53.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.851+0000 7fb42d7fa640 1 -- 172.21.15.148:0/2783365133 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fb428078c40 con 0x7fb43015b320 2024-04-25T10:02:53.855 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:53.856 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:53.856 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb41406b4d0 msgr2=0x7fb41406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb41406b4d0 0x7fb41406d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb424003d90 tx=0x7fb424009040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 msgr2=0x7fb4300b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb4300b2420 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d270 tx=0x7fb428004ab0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 shutdown_connections 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb41406b4d0 0x7fb41406d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb43015b320 0x7fb4300b2420 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb43013d190 0x7fb430139ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 --2- 172.21.15.148:0/2783365133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4300cc6c0 0x7fb430156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 >> 172.21.15.148:0/2783365133 conn(0x7fb43006abe0 msgr2=0x7fb430158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 shutdown_connections 2024-04-25T10:02:53.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:53.853+0000 7fb436a6f640 1 -- 172.21.15.148:0/2783365133 wait complete. 2024-04-25T10:02:54.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:49.163477+0000', 'modified': '2024-04-25T10:02:51.165263+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9351}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9351': {'gid': 9351, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/1740205817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1740205817}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1740205817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:54.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:54.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:02:54.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.368+0000 7f65e7a3a640 1 Processor -- start 2024-04-25T10:02:54.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.368+0000 7f65e7a3a640 1 -- start start 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e7a3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f65e015dc20 0x7f65e00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e7a3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e7a3a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e015e000 con 0x7f65e0161710 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e7a3a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e00d1de0 con 0x7f65e015dc20 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e7a3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0161c60 con 0x7f65e00d32f0 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e57d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f65e015dc20 0x7f65e00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e57d6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f65e015dc20 0x7f65e00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44126/0 (socket says 172.21.15.148:44126) 2024-04-25T10:02:54.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.369+0000 7f65e57d6640 1 -- 172.21.15.148:0/1811023174 learned_addr learned my addr 172.21.15.148:0/1811023174 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e57d6640 1 -- 172.21.15.148:0/1811023174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 msgr2=0x7f65e00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e57d6640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e57d6640 1 -- 172.21.15.148:0/1811023174 >> v1:172.21.15.188:6789/0 conn(0x7f65e0161710 legacy=0x7f65e006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e57d6640 1 -- 172.21.15.148:0/1811023174 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0161f40 con 0x7f65e015dc20 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.370+0000 7f65e57d6640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e015dc20 0x7f65e00ce670 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009870 tx=0x7f65d802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8fee13d270c669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65dffff640 1 -- 172.21.15.148:0/1811023174 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8037070 con 0x7f65e015dc20 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65dffff640 1 -- 172.21.15.148:0/1811023174 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d8032440 con 0x7f65e015dc20 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65dffff640 1 -- 172.21.15.148:0/1811023174 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8032810 con 0x7f65e015dc20 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 -- 172.21.15.148:0/1811023174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e015dc20 msgr2=0x7f65e00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e015dc20 0x7f65e00ce670 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009870 tx=0x7f65d802a250 comp rx=0 tx=0).stop 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 -- 172.21.15.148:0/1811023174 shutdown_connections 2024-04-25T10:02:54.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/1811023174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e015dc20 0x7f65e00ce670 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 -- 172.21.15.148:0/1811023174 >> 172.21.15.148:0/1811023174 conn(0x7f65e00d0420 msgr2=0x7f65e00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 -- 172.21.15.148:0/1811023174 shutdown_connections 2024-04-25T10:02:54.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.371+0000 7f65e7a3a640 1 -- 172.21.15.148:0/1811023174 wait complete. 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.372+0000 7f65e7a3a640 1 Processor -- start 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.372+0000 7f65e7a3a640 1 -- start start 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e7a3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e7a3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e015dc20 0x7f65e015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e7a3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 0x7f65e01f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e7a3a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00b5e50 con 0x7f65e0161710 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e7a3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0162500 con 0x7f65e015dc20 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e7a3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00b5fd0 con 0x7f65e00d32f0 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 0x7f65e01f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 0x7f65e01f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44140/0 (socket says 172.21.15.148:44140) 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 -- 172.21.15.148:0/3492078471 learned_addr learned my addr 172.21.15.148:0/3492078471 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e015dc20 0x7f65e015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 -- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 msgr2=0x7f65e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 -- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e015dc20 msgr2=0x7f65e015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e015dc20 0x7f65e015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e5fd7640 1 -- 172.21.15.148:0/3492078471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e015d300 con 0x7f65e0161710 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.373+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e015dc20 0x7f65e015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.374+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 0x7f65e01f3d30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f65d000cd10 tx=0x7f65d000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.374+0000 7f65ddffb640 1 -- 172.21.15.148:0/3492078471 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d000f680 con 0x7f65e0161710 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.374+0000 7f65ddffb640 1 -- 172.21.15.148:0/3492078471 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d000f840 con 0x7f65e0161710 2024-04-25T10:02:54.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.374+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e015fc30 con 0x7f65e0161710 2024-04-25T10:02:54.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.374+0000 7f65ddffb640 1 -- 172.21.15.148:0/3492078471 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d000fc30 con 0x7f65e0161710 2024-04-25T10:02:54.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.374+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e01601b0 con 0x7f65e0161710 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.376+0000 7f65ddffb640 1 -- 172.21.15.148:0/3492078471 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65d00213f0 con 0x7f65e0161710 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.376+0000 7f65ddffb640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65cc06b4d0 0x7f65cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.376+0000 7f65ddffb640 1 -- 172.21.15.148:0/3492078471 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65d008b060 con 0x7f65e0161710 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.376+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65cc06b4d0 0x7f65cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.377+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65cc06b4d0 0x7f65cc06d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f65d8034000 tx=0x7f65d802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.377+0000 7f65ea810740 1 Processor -- start 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.377+0000 7f65ea810740 1 -- start start 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e333610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e350ad0 0x556a4e351020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65ea810740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 0x556a4e34f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65ea810740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4e26ba10 con 0x556a4e308f10 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4e214b30 con 0x556a4e333230 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a4e267c50 con 0x556a4e350ad0 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 0x556a4e34f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 0x556a4e34f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44152/0 (socket says 172.21.15.148:44152) 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 -- 172.21.15.148:0/682989441 learned_addr learned my addr 172.21.15.148:0/682989441 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e57d6640 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e333610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e350ad0 0x556a4e351020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 -- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e350ad0 msgr2=0x556a4e351020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e350ad0 0x556a4e351020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 -- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 msgr2=0x556a4e333610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e333610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 -- 172.21.15.148:0/682989441 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a4e229e10 con 0x556a4e308f10 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e57d6640 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e333610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 0x556a4e34f190 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f65d0059690 tx=0x7f65d0059810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f1801a605de78e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65c6ffd640 1 -- 172.21.15.148:0/682989441 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00711c0 con 0x556a4e308f10 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65c6ffd640 1 -- 172.21.15.148:0/682989441 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0057a00 con 0x556a4e308f10 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.378+0000 7f65c6ffd640 1 -- 172.21.15.148:0/682989441 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00618d0 con 0x556a4e308f10 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 -- 172.21.15.148:0/682989441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 msgr2=0x556a4e34f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 0x556a4e34f190 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f65d0059690 tx=0x7f65d0059810 comp rx=0 tx=0).stop 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 -- 172.21.15.148:0/682989441 shutdown_connections 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e308f10 0x556a4e34f190 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e350ad0 0x556a4e351020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 --2- 172.21.15.148:0/682989441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e333610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 -- 172.21.15.148:0/682989441 >> 172.21.15.148:0/682989441 conn(0x556a4e305840 msgr2=0x556a4e34e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 -- 172.21.15.148:0/682989441 shutdown_connections 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.379+0000 7f65ea810740 1 -- 172.21.15.148:0/682989441 wait complete. 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.380+0000 7f65ea810740 1 -- start start 2024-04-25T10:02:54.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.380+0000 7f65ea810740 1 Processor -- start 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e3028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e346ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e32f6a0 0x556a4e32fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4e309cb0 con 0x556a4e350ad0 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4e2280c0 con 0x556a4e32f6a0 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a4e303770 con 0x556a4e333230 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 10 client.0 _ll_register_callbacks cb 0x556a4e21bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e346ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e346ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44166/0 (socket says 172.21.15.148:44166) 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 -- 172.21.15.148:0/3961214047 learned_addr learned my addr 172.21.15.148:0/3961214047 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e3028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e32f6a0 0x556a4e32fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 -- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 msgr2=0x556a4e3028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e3028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 -- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e32f6a0 msgr2=0x556a4e32fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e32f6a0 0x556a4e32fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 -- 172.21.15.148:0/3961214047 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a4df802d0 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e3028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e346ed0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f65d4001e10 tx=0x7f65d400f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270f22cb892edcb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65bffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65bffff640 1 -- 172.21.15.148:0/3961214047 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40050d0 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65bffff640 1 -- 172.21.15.148:0/3961214047 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65d4005270 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65d0057a00 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65bffff640 1 -- 172.21.15.148:0/3961214047 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65d4020050 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 10 client.9477 fetch_fsmap learned FSMap version 74 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.381+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x556a4e04a6b0 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65bffff640 1 -- 172.21.15.148:0/3961214047 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0057a00 con 0x556a4e350ad0 2024-04-25T10:02:54.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65bffff640 1 -- 172.21.15.148:0/3961214047 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65d4023020 con 0x556a4e350ad0 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65ea810740 10 client.9477 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65ea810740 10 client.9477 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65ea810740 20 client.9477 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65ea810740 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e335770 0x556a4e335b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65ea810740 4 client.9477 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a4e2ca920 con 0x556a4e335770 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e335770 0x556a4e335b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65bffff640 10 client.9477 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.382+0000 7f65e57d6640 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e335770 0x556a4e335b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf68190e391030d server_cookie=8dfb0c3401d28036 in_seq=0 out_seq=0 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65bffff640 1 -- 172.21.15.148:0/3961214047 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a4e2ca920 con 0x556a4e335770 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65bffff640 10 client.9477 handle_command_reply: tid=0 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 client.9477 shutdown 2024-04-25T10:02:54.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 msgr2=0x556a4e346ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e346ed0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f65d4001e10 tx=0x7f65d400f660 comp rx=0 tx=0).stop 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 shutdown_connections 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e335770 0x556a4e335b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e32f6a0 0x556a4e32fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65bffff640 0 client.9477 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e346ed0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 --2- 172.21.15.148:0/3961214047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e3028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 >> 172.21.15.148:0/3961214047 conn(0x556a4e308f10 msgr2=0x556a4e3515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 shutdown_connections 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 1 -- 172.21.15.148:0/3961214047 wait complete. 2024-04-25T10:02:54.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.383+0000 7f65ea810740 20 client.9477 trim_cache size 0 max 16384 2024-04-25T10:02:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 Processor -- start 2024-04-25T10:02:54.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 -- start start 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 0x556a4e2fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e32f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e350ad0 0x556a4e346db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01ff700 con 0x556a4e32f6a0 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01e81d0 con 0x556a4e350ad0 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01e87d0 con 0x556a4e333230 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65e57d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 0x556a4e2fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65e57d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 0x556a4e2fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44174/0 (socket says 172.21.15.148:44174) 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.399+0000 7f65e57d6640 1 -- 172.21.15.148:0/380622886 learned_addr learned my addr 172.21.15.148:0/380622886 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65e57d6640 1 -- 172.21.15.148:0/380622886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 msgr2=0x556a4e32f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65e57d6640 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e32f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65e57d6640 1 -- 172.21.15.148:0/380622886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e350ad0 msgr2=0x556a4e346db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65e57d6640 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e350ad0 0x556a4e346db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65e57d6640 1 -- 172.21.15.148:0/380622886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a4e345a80 con 0x556a4e32f6a0 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65e57d6640 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 0x556a4e2fefa0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f65d80090b0 tx=0x7f65d802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8f18af6f025599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65c67fc640 1 -- 172.21.15.148:0/380622886 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8037040 con 0x556a4e32f6a0 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65c67fc640 1 -- 172.21.15.148:0/380622886 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d803e8b0 con 0x556a4e32f6a0 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65c67fc640 1 -- 172.21.15.148:0/380622886 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d803ec60 con 0x556a4e32f6a0 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 -- 172.21.15.148:0/380622886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 msgr2=0x556a4e2fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 0x556a4e2fefa0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f65d80090b0 tx=0x7f65d802c000 comp rx=0 tx=0).stop 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 -- 172.21.15.148:0/380622886 shutdown_connections 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e350ad0 0x556a4e346db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e333230 0x556a4e32f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 --2- 172.21.15.148:0/380622886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e32f6a0 0x556a4e2fefa0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 -- 172.21.15.148:0/380622886 >> 172.21.15.148:0/380622886 conn(0x556a4e308f10 msgr2=0x556a4e309350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.400+0000 7f65ea810740 1 -- 172.21.15.148:0/380622886 shutdown_connections 2024-04-25T10:02:54.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.401+0000 7f65ea810740 1 -- 172.21.15.148:0/380622886 wait complete. 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.401+0000 7f65ea810740 1 -- start start 2024-04-25T10:02:54.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.401+0000 7f65ea810740 1 Processor -- start 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e32f6a0 0x556a4e335c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e3b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e34ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4e347ab0 con 0x556a4e350ad0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a4e35f280 con 0x556a4e333230 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a4e3457b0 con 0x556a4e32f6a0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65ea810740 10 client.0 _ll_register_callbacks cb 0x556a4e303c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e34ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e34ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44178/0 (socket says 172.21.15.148:44178) 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 -- 172.21.15.148:0/212976391 learned_addr learned my addr 172.21.15.148:0/212976391 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e57d6640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e32f6a0 0x556a4e335c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e4fd5640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e3b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 -- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e32f6a0 msgr2=0x556a4e335c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e32f6a0 0x556a4e335c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 -- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 msgr2=0x556a4e3b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e3b9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e5fd7640 1 -- 172.21.15.148:0/212976391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a4e3097c0 con 0x556a4e350ad0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.402+0000 7f65e57d6640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e32f6a0 0x556a4e335c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65e5fd7640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e34ed50 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f65d005d960 tx=0x7f65d000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2636de909fc0f1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 1 -- 172.21.15.148:0/212976391 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0061ad0 con 0x556a4e350ad0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 1 -- 172.21.15.148:0/212976391 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65d0058b50 con 0x556a4e350ad0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65d803e8b0 con 0x556a4e350ad0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 1 -- 172.21.15.148:0/212976391 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65d00711c0 con 0x556a4e350ad0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 10 client.9483 fetch_fsmap learned FSMap version 74 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x556a4e351dc0 con 0x556a4e350ad0 2024-04-25T10:02:54.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 1 -- 172.21.15.148:0/212976391 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d803e8b0 con 0x556a4e350ad0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 1 -- 172.21.15.148:0/212976391 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65d0055450 con 0x556a4e350ad0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 10 client.9483 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 10 client.9483 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 20 client.9483 populate_metadata read hostname 'smithi148' 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e34f980 0x556a4e3439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 4 client.9483 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556a4e278f50 con 0x556a4e34f980 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65e57d6640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e34f980 0x556a4e3439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65bffff640 10 client.9483 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.403+0000 7f65e57d6640 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e34f980 0x556a4e3439d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f100710563fa456f server_cookie=3cf72ad99a60b2da in_seq=0 out_seq=0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65bffff640 1 -- 172.21.15.148:0/212976391 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556a4e278f50 con 0x556a4e34f980 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65bffff640 10 client.9483 handle_command_reply: tid=0 2024-04-25T10:02:54.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 client.9483 shutdown 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 msgr2=0x556a4e34ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e34ed50 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f65d005d960 tx=0x7f65d000af70 comp rx=0 tx=0).stop 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 shutdown_connections 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x556a4e34f980 0x556a4e3439d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a4e350ad0 0x556a4e34ed50 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65bffff640 0 client.9483 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a4e333230 0x556a4e3b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 --2- 172.21.15.148:0/212976391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a4e32f6a0 0x556a4e335c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 >> 172.21.15.148:0/212976391 conn(0x556a4e308f10 msgr2=0x556a4e332500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 shutdown_connections 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 1 -- 172.21.15.148:0/212976391 wait complete. 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.404+0000 7f65ea810740 20 client.9483 trim_cache size 0 max 16384 2024-04-25T10:02:54.415 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 9351, 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.5721411209999996, 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 74, 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 133, 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 133, 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 9.993628524 2024-04-25T10:02:54.416 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:02:54.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65cc06b4d0 msgr2=0x7f65cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65cc06b4d0 0x7f65cc06d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f65d8034000 tx=0x7f65d802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:54.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 msgr2=0x7f65e01f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 0x7f65e01f3d30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f65d000cd10 tx=0x7f65d000a650 comp rx=0 tx=0).stop 2024-04-25T10:02:54.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 shutdown_connections 2024-04-25T10:02:54.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65cc06b4d0 0x7f65cc06d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65e0161710 0x7f65e01f3d30 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65e015dc20 0x7f65e015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 --2- 172.21.15.148:0/3492078471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65e00d32f0 0x7f65e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 >> 172.21.15.148:0/3492078471 conn(0x7f65e00d0420 msgr2=0x7f65e00d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 shutdown_connections 2024-04-25T10:02:54.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.405+0000 7f65e7a3a640 1 -- 172.21.15.148:0/3492078471 wait complete. 2024-04-25T10:02:54.420 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.512+0000 7fe12a8db640 1 Processor -- start 2024-04-25T10:02:54.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.512+0000 7fe12a8db640 1 -- start start 2024-04-25T10:02:54.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe12a8db640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe1240ce920 0x7fe1240cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe12a8db640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe12406d290 0x7fe12406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe12a8db640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe124144b30 con 0x7fe124144750 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe12a8db640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240712d0 con 0x7fe12406d290 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe12a8db640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240715b0 con 0x7fe1240ce920 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe128e78640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe12406d290 0x7fe12406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe123fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe124144750 0x7fe12406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58068/0 (socket says 172.21.15.148:58068) 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe123fff640 1 -- 172.21.15.148:0/1507879542 learned_addr learned my addr 172.21.15.148:0/1507879542 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe1237fe640 1 --2- 172.21.15.148:0/1507879542 >> v2:172.21.15.188:3300/0 conn(0x7fe1240ce920 0x7fe1240cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe128e78640 1 -- 172.21.15.148:0/1507879542 >> v2:172.21.15.188:3300/0 conn(0x7fe1240ce920 msgr2=0x7fe1240cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe128e78640 1 --2- 172.21.15.148:0/1507879542 >> v2:172.21.15.188:3300/0 conn(0x7fe1240ce920 0x7fe1240cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe128e78640 1 -- 172.21.15.148:0/1507879542 >> v1:172.21.15.148:6789/0 conn(0x7fe124144750 legacy=0x7fe12406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe122ffd640 1 -- 172.21.15.148:0/1507879542 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495942901 0 0) 0x7fe124144b30 con 0x7fe124144750 2024-04-25T10:02:54.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe128e78640 1 -- 172.21.15.148:0/1507879542 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124071890 con 0x7fe12406d290 2024-04-25T10:02:54.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.513+0000 7fe1237fe640 1 --2- 172.21.15.148:0/1507879542 >> v2:172.21.15.188:3300/0 conn(0x7fe1240ce920 0x7fe1240cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.514+0000 7fe128e78640 1 --2- 172.21.15.148:0/1507879542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe12406d290 0x7fe12406d6c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe1241606b0 tx=0x7fe11c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e947569af014aca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.514+0000 7fe122ffd640 1 -- 172.21.15.148:0/1507879542 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007740 con 0x7fe12406d290 2024-04-25T10:02:54.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.514+0000 7fe122ffd640 1 -- 172.21.15.148:0/1507879542 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c0078e0 con 0x7fe12406d290 2024-04-25T10:02:54.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.514+0000 7fe122ffd640 1 -- 172.21.15.148:0/1507879542 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007cb0 con 0x7fe12406d290 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 -- 172.21.15.148:0/1507879542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe12406d290 msgr2=0x7fe12406d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 --2- 172.21.15.148:0/1507879542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe12406d290 0x7fe12406d6c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe1241606b0 tx=0x7fe11c02a680 comp rx=0 tx=0).stop 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 -- 172.21.15.148:0/1507879542 shutdown_connections 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 --2- 172.21.15.148:0/1507879542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe12406d290 0x7fe12406d6c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 --2- 172.21.15.148:0/1507879542 >> v2:172.21.15.188:3300/0 conn(0x7fe1240ce920 0x7fe1240cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 -- 172.21.15.148:0/1507879542 >> 172.21.15.148:0/1507879542 conn(0x7fe12406abe0 msgr2=0x7fe12406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 -- 172.21.15.148:0/1507879542 shutdown_connections 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.515+0000 7fe12a8db640 1 -- 172.21.15.148:0/1507879542 wait complete. 2024-04-25T10:02:54.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.516+0000 7fe12a8db640 1 Processor -- start 2024-04-25T10:02:54.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.516+0000 7fe12a8db640 1 -- start start 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe12a8db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12406d290 0x7fe124141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe12a8db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 0x7fe12415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe12a8db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe124144750 0x7fe1240ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe12a8db640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12406f020 con 0x7fe1240ce920 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe12a8db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12406ece0 con 0x7fe12406d290 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe12a8db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe124145450 con 0x7fe124144750 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 0x7fe12415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 0x7fe12415c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44200/0 (socket says 172.21.15.148:44200) 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 -- 172.21.15.148:0/3501637891 learned_addr learned my addr 172.21.15.148:0/3501637891 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe123fff640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12406d290 0x7fe124141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe128e78640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe124144750 0x7fe1240ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 -- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe124144750 msgr2=0x7fe1240ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe124144750 0x7fe1240ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 -- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12406d290 msgr2=0x7fe124141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12406d290 0x7fe124141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 -- 172.21.15.148:0/3501637891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12415db60 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe123fff640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12406d290 0x7fe124141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe1237fe640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 0x7fe12415c000 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c7f0 tx=0x7fe11800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.517+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118015070 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.518+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118007820 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.518+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1240b4570 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.518+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118007bd0 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.518+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12415e3c0 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.518+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe124073990 con 0x7fe1240ce920 2024-04-25T10:02:54.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.519+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe118018070 con 0x7fe1240ce920 2024-04-25T10:02:54.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.519+0000 7fe120ff9640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe11006b4d0 0x7fe11006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.519+0000 7fe123fff640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe11006b4d0 0x7fe11006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.519+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe11808a490 con 0x7fe1240ce920 2024-04-25T10:02:54.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.520+0000 7fe123fff640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe11006b4d0 0x7fe11006d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe114000f60 tx=0x7fe114009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.523+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe118059190 con 0x7fe1240ce920 2024-04-25T10:02:54.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.728+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1240d0510 con 0x7fe1240ce920 2024-04-25T10:02:54.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.728+0000 7fe120ff9640 1 -- 172.21.15.148:0/3501637891 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe118058a90 con 0x7fe1240ce920 2024-04-25T10:02:54.733 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:02:54.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe11006b4d0 msgr2=0x7fe11006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe11006b4d0 0x7fe11006d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe114000f60 tx=0x7fe114009040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 msgr2=0x7fe12415c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 0x7fe12415c000 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c7f0 tx=0x7fe11800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 shutdown_connections 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe11006b4d0 0x7fe11006d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe124144750 0x7fe1240ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1240ce920 0x7fe12415c000 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 --2- 172.21.15.148:0/3501637891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe12406d290 0x7fe124141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 >> 172.21.15.148:0/3501637891 conn(0x7fe12406abe0 msgr2=0x7fe12406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.730+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 shutdown_connections 2024-04-25T10:02:54.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:54.731+0000 7fe12a8db640 1 -- 172.21.15.148:0/3501637891 wait complete. 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:54.746 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:02:54.747 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:54.747 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:54.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:54.747 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:54.762 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:02:54.763 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42261sec preferred_lft 42261sec 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:54.764 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:02:54.765 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:54.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:54.765 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:54.792 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T10:02:54.787+0000 7f7c548ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:02:54.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:54.815 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:02:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:02:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:54.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:54.872 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:02:54.872 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:54.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:54.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:54.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:54.872 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:54.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:54.955 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:02:55.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:55.010 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:02:55.010 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:55.011 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:55.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:02:55.052 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:55.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:55.075 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:55.128 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:55.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:55.128 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:55.183 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:55.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.183 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:55.265 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:55.266 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:55.266 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:02:55.274 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:55.274 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:55.274 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:02:55.274 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:02:55.274 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:55.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:55.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.281 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:55.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:55.345 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:55.405 INFO:teuthology.orchestra.run.smithi148.stdout:client9495 2024-04-25T10:02:55.405 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:02:55.405 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:55.460 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.593+0000 7f0d8ca37640 1 Processor -- start 2024-04-25T10:02:55.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.593+0000 7f0d8ca37640 1 -- start start 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8ca37640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 0x7f0d880ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8ca37640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d8806d290 0x7f0d8806d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8ca37640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d88144b30 con 0x7f0d88144750 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8ca37640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d880712d0 con 0x7f0d8806d290 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8ca37640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880715b0 con 0x7f0d880ce920 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0d88144750 0x7f0d8806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:58082/0 (socket says 172.21.15.148:58082) 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d8659c640 1 -- 172.21.15.148:0/2325369742 learned_addr learned my addr 172.21.15.148:0/2325369742 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731425590 0 0) 0x7f0d88144b30 con 0x7f0d88144750 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d74003520 con 0x7f0d88144750 2024-04-25T10:02:55.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d7ffff640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 0x7f0d880ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/2325369742 >> v2:172.21.15.188:3300/0 conn(0x7f0d8806d290 0x7f0d8806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1564005711 0 0) 0x7f0d74003520 con 0x7f0d88144750 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 msgr2=0x7f0d880ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 0x7f0d880ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 >> v2:172.21.15.188:3300/0 conn(0x7f0d8806d290 msgr2=0x7f0d8806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 --2- 172.21.15.148:0/2325369742 >> v2:172.21.15.188:3300/0 conn(0x7f0d8806d290 0x7f0d8806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88071890 con 0x7f0d88144750 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f0d78001670 con 0x7f0d88144750 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.594+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/2325369742 >> v2:172.21.15.188:3300/0 conn(0x7f0d8806d290 0x7f0d8806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 >> v1:172.21.15.148:6789/0 conn(0x7f0d88144750 legacy=0x7f0d8806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d740039b0 0x7f0d74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 0x7f0d74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d74004d40 0x7f0d740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880712d0 con 0x7f0d74004d40 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880715b0 con 0x7f0d740039b0 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7400b9f0 con 0x7f0d880ce920 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d74004d40 0x7f0d740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d8659c640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d740039b0 0x7f0d74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d7ffff640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 0x7f0d74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d86d9d640 1 -- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 msgr2=0x7f0d74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880ce920 0x7f0d74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d86d9d640 1 -- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d740039b0 msgr2=0x7f0d74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d740039b0 0x7f0d74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.595+0000 7f0d86d9d640 1 -- 172.21.15.148:0/2325369742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74003520 con 0x7f0d74004d40 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.596+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d74004d40 0x7f0d740051c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0d800073e0 tx=0x7f0d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a494022350170b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.596+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80009c80 con 0x7f0d74004d40 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.596+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8002c680 con 0x7f0d74004d40 2024-04-25T10:02:55.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.596+0000 7f0d85d9b640 1 -- 172.21.15.148:0/2325369742 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8002ca30 con 0x7f0d74004d40 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d8ca37640 1 -- 172.21.15.148:0/2325369742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d74004d40 msgr2=0x7f0d740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/2325369742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d74004d40 0x7f0d740051c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0d800073e0 tx=0x7f0d74005830 comp rx=0 tx=0).stop 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d86d9d640 1 -- 172.21.15.148:0/2325369742 reap_dead start 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d8ca37640 1 -- 172.21.15.148:0/2325369742 shutdown_connections 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d8ca37640 1 -- 172.21.15.148:0/2325369742 >> 172.21.15.148:0/2325369742 conn(0x7f0d8806abe0 msgr2=0x7f0d8806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d8ca37640 1 -- 172.21.15.148:0/2325369742 shutdown_connections 2024-04-25T10:02:55.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.597+0000 7f0d8ca37640 1 -- 172.21.15.148:0/2325369742 wait complete. 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.598+0000 7f0d8ca37640 1 Processor -- start 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.598+0000 7f0d8ca37640 1 -- start start 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8ca37640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880b2a50 0x7f0d8813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8ca37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 0x7f0d8815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8ca37640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d8815f0d0 0x7f0d881404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8ca37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88145450 con 0x7f0d88143250 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8ca37640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8806f020 con 0x7f0d8815f0d0 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8ca37640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881455d0 con 0x7f0d880b2a50 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 0x7f0d8815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 0x7f0d8815eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44230/0 (socket says 172.21.15.148:44230) 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 -- 172.21.15.148:0/3468788785 learned_addr learned my addr 172.21.15.148:0/3468788785 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:55.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d8659c640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880b2a50 0x7f0d8813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d86d9d640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d8815f0d0 0x7f0d881404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 -- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880b2a50 msgr2=0x7f0d8813f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880b2a50 0x7f0d8813f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 -- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d8815f0d0 msgr2=0x7f0d881404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d8815f0d0 0x7f0d881404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 -- 172.21.15.148:0/3468788785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8815d850 con 0x7f0d88143250 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.599+0000 7f0d7ffff640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 0x7f0d8815eb90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0d700119d0 tx=0x7f0d70011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.600+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000fa70 con 0x7f0d88143250 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.600+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7000fc10 con 0x7f0d88143250 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.600+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70014440 con 0x7f0d88143250 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.600+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8815daf0 con 0x7f0d88143250 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.600+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8815e0b0 con 0x7f0d88143250 2024-04-25T10:02:55.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.601+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d88073990 con 0x7f0d88143250 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.601+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d70012040 con 0x7f0d88143250 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.601+0000 7f0d7effd640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d6806b4d0 0x7f0d6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.602+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d7008a860 con 0x7f0d88143250 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.602+0000 7f0d8659c640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d6806b4d0 0x7f0d6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.602+0000 7f0d8659c640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d6806b4d0 0x7f0d6806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0d78004840 tx=0x7f0d78007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.606+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d70019030 con 0x7f0d88143250 2024-04-25T10:02:55.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.810+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d880d0510 con 0x7f0d88143250 2024-04-25T10:02:55.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.811+0000 7f0d7effd640 1 -- 172.21.15.148:0/3468788785 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f0d700595e0 con 0x7f0d88143250 2024-04-25T10:02:55.816 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:55.816 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:55.816 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.812+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d6806b4d0 msgr2=0x7f0d6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.812+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d6806b4d0 0x7f0d6806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0d78004840 tx=0x7f0d78007970 comp rx=0 tx=0).stop 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.812+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 msgr2=0x7f0d8815eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.812+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 0x7f0d8815eb90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0d700119d0 tx=0x7f0d70011dd0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 shutdown_connections 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0d6806b4d0 0x7f0d6806d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d8815f0d0 0x7f0d881404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d88143250 0x7f0d8815eb90 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 --2- 172.21.15.148:0/3468788785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d880b2a50 0x7f0d8813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 >> 172.21.15.148:0/3468788785 conn(0x7f0d8806abe0 msgr2=0x7f0d8813cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 shutdown_connections 2024-04-25T10:02:55.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.813+0000 7f0d8ca37640 1 -- 172.21.15.148:0/3468788785 wait complete. 2024-04-25T10:02:55.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:55.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.923+0000 7fda291c4640 1 Processor -- start 2024-04-25T10:02:55.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.923+0000 7fda291c4640 1 -- start start 2024-04-25T10:02:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda291c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240cedf0 0x7fda241616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda291c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda240b93a0 0x7fda240d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda291c4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2415e000 con 0x7fda2415dc20 2024-04-25T10:02:55.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda291c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda240b6bb0 con 0x7fda240b93a0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda291c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda240b6030 con 0x7fda240cedf0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda240b93a0 0x7fda240d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fda240b93a0 0x7fda240d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44238/0 (socket says 172.21.15.148:44238) 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 -- 172.21.15.148:0/1855801028 learned_addr learned my addr 172.21.15.148:0/1855801028 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2259c640 1 --2- 172.21.15.148:0/1855801028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240cedf0 0x7fda241616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 -- 172.21.15.148:0/1855801028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240cedf0 msgr2=0x7fda241616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 --2- 172.21.15.148:0/1855801028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240cedf0 0x7fda241616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 -- 172.21.15.148:0/1855801028 >> v1:172.21.15.188:6789/0 conn(0x7fda2415dc20 legacy=0x7fda240ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.924+0000 7fda2359e640 1 -- 172.21.15.148:0/1855801028 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24161de0 con 0x7fda240b93a0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda2359e640 1 --2- 172.21.15.148:0/1855801028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240b93a0 0x7fda240d2510 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009e50 tx=0x7fda1c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a93fc4a08ea327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda21d9b640 1 -- 172.21.15.148:0/1855801028 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c036070 con 0x7fda240b93a0 2024-04-25T10:02:55.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda21d9b640 1 -- 172.21.15.148:0/1855801028 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c031600 con 0x7fda240b93a0 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda21d9b640 1 -- 172.21.15.148:0/1855801028 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0319d0 con 0x7fda240b93a0 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda291c4640 1 -- 172.21.15.148:0/1855801028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240b93a0 msgr2=0x7fda240d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda291c4640 1 --2- 172.21.15.148:0/1855801028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240b93a0 0x7fda240d2510 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fda1c009e50 tx=0x7fda1c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda291c4640 1 -- 172.21.15.148:0/1855801028 shutdown_connections 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda291c4640 1 --2- 172.21.15.148:0/1855801028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240b93a0 0x7fda240d2510 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda291c4640 1 --2- 172.21.15.148:0/1855801028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240cedf0 0x7fda241616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.925+0000 7fda291c4640 1 -- 172.21.15.148:0/1855801028 >> 172.21.15.148:0/1855801028 conn(0x7fda240d0490 msgr2=0x7fda240d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.926+0000 7fda291c4640 1 -- 172.21.15.148:0/1855801028 shutdown_connections 2024-04-25T10:02:55.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.926+0000 7fda291c4640 1 -- 172.21.15.148:0/1855801028 wait complete. 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.927+0000 7fda291c4640 1 Processor -- start 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.927+0000 7fda291c4640 1 -- start start 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda291c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240b93a0 0x7fda240cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda291c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 0x7fda24149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda291c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2415dc20 0x7fda2415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda291c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda240d2a50 con 0x7fda240cedf0 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda291c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda241623a0 con 0x7fda2415dc20 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda291c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda240d2bd0 con 0x7fda240b93a0 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 0x7fda24149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 0x7fda24149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44254/0 (socket says 172.21.15.148:44254) 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 -- 172.21.15.148:0/2487176793 learned_addr learned my addr 172.21.15.148:0/2487176793 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda22d9d640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240b93a0 0x7fda240cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2359e640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2415dc20 0x7fda2415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 -- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240b93a0 msgr2=0x7fda240cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240b93a0 0x7fda240cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 -- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2415dc20 msgr2=0x7fda2415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2415dc20 0x7fda2415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 -- 172.21.15.148:0/2487176793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24161a50 con 0x7fda240cedf0 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda2259c640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 0x7fda24149b90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c7f0 tx=0x7fda1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18015070 con 0x7fda240cedf0 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.928+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18004240 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.929+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda241e83e0 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.929+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007620 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.929+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda241e89a0 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.930+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda24073990 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.930+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda180079d0 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.930+0000 7fda0b7fe640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda1006b4d0 0x7fda1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.930+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda1808a760 con 0x7fda240cedf0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.930+0000 7fda22d9d640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda1006b4d0 0x7fda1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.931+0000 7fda22d9d640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda1006b4d0 0x7fda1006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fda14006fd0 tx=0x7fda14009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:55.935+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda18058e00 con 0x7fda240cedf0 2024-04-25T10:02:56.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.139+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda240d1720 con 0x7fda240cedf0 2024-04-25T10:02:56.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.140+0000 7fda0b7fe640 1 -- 172.21.15.148:0/2487176793 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fda1805cea0 con 0x7fda240cedf0 2024-04-25T10:02:56.144 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:02:56.145 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:02:56.145 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:02:56.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda1006b4d0 msgr2=0x7fda1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda1006b4d0 0x7fda1006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fda14006fd0 tx=0x7fda14009060 comp rx=0 tx=0).stop 2024-04-25T10:02:56.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 msgr2=0x7fda24149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 0x7fda24149b90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c7f0 tx=0x7fda1800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 shutdown_connections 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fda1006b4d0 0x7fda1006d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2415dc20 0x7fda2415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda240cedf0 0x7fda24149b90 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 --2- 172.21.15.148:0/2487176793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda240b93a0 0x7fda240cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.141+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 >> 172.21.15.148:0/2487176793 conn(0x7fda240d0490 msgr2=0x7fda240b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.142+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 shutdown_connections 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:02:56.142+0000 7fda291c4640 1 -- 172.21.15.148:0/2487176793 wait complete. 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:56.158 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:56.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.191 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.191 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101219 s, 104 MB/s 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.200 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831756 s, 126 MB/s 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.208 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657259 s, 160 MB/s 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.217 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832094 s, 126 MB/s 2024-04-25T10:02:56.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766922 s, 137 MB/s 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831597 s, 126 MB/s 2024-04-25T10:02:56.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.244 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.244 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787802 s, 133 MB/s 2024-04-25T10:02:56.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.250 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552971 s, 190 MB/s 2024-04-25T10:02:56.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.258 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718253 s, 146 MB/s 2024-04-25T10:02:56.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961366 s, 109 MB/s 2024-04-25T10:02:56.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.276 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688739 s, 152 MB/s 2024-04-25T10:02:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624127 s, 168 MB/s 2024-04-25T10:02:56.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.291 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690848 s, 152 MB/s 2024-04-25T10:02:56.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684303 s, 153 MB/s 2024-04-25T10:02:56.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.306 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656827 s, 160 MB/s 2024-04-25T10:02:56.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771935 s, 136 MB/s 2024-04-25T10:02:56.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685093 s, 153 MB/s 2024-04-25T10:02:56.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643702 s, 163 MB/s 2024-04-25T10:02:56.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.338 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673686 s, 156 MB/s 2024-04-25T10:02:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723683 s, 145 MB/s 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.354 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743782 s, 141 MB/s 2024-04-25T10:02:56.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.362 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707741 s, 148 MB/s 2024-04-25T10:02:56.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.370 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671557 s, 156 MB/s 2024-04-25T10:02:56.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.378 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730576 s, 144 MB/s 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606847 s, 173 MB/s 2024-04-25T10:02:56.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715531 s, 147 MB/s 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068285 s, 154 MB/s 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685107 s, 153 MB/s 2024-04-25T10:02:56.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955771 s, 110 MB/s 2024-04-25T10:02:56.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.426 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006437 s, 163 MB/s 2024-04-25T10:02:56.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.433 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634055 s, 165 MB/s 2024-04-25T10:02:56.441 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.441 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.441 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677817 s, 155 MB/s 2024-04-25T10:02:56.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:56.442 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:56.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.507 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.507 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691113 s, 152 MB/s 2024-04-25T10:02:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637796 s, 164 MB/s 2024-04-25T10:02:56.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618544 s, 170 MB/s 2024-04-25T10:02:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605459 s, 173 MB/s 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597107 s, 176 MB/s 2024-04-25T10:02:56.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774835 s, 135 MB/s 2024-04-25T10:02:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.551 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635259 s, 165 MB/s 2024-04-25T10:02:56.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.560 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760413 s, 138 MB/s 2024-04-25T10:02:56.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.568 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072905 s, 144 MB/s 2024-04-25T10:02:56.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.577 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769069 s, 136 MB/s 2024-04-25T10:02:56.584 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.584 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707451 s, 148 MB/s 2024-04-25T10:02:56.594 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.594 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.594 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878015 s, 119 MB/s 2024-04-25T10:02:56.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.602 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068068 s, 154 MB/s 2024-04-25T10:02:56.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723855 s, 145 MB/s 2024-04-25T10:02:56.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724776 s, 145 MB/s 2024-04-25T10:02:56.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.625 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649792 s, 161 MB/s 2024-04-25T10:02:56.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.636 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.636 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982062 s, 107 MB/s 2024-04-25T10:02:56.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.644 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652393 s, 161 MB/s 2024-04-25T10:02:56.651 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.651 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.651 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682257 s, 154 MB/s 2024-04-25T10:02:56.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723614 s, 145 MB/s 2024-04-25T10:02:56.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.668 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746322 s, 140 MB/s 2024-04-25T10:02:56.675 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705489 s, 149 MB/s 2024-04-25T10:02:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.683 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663858 s, 158 MB/s 2024-04-25T10:02:56.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.691 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698918 s, 150 MB/s 2024-04-25T10:02:56.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674888 s, 155 MB/s 2024-04-25T10:02:56.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682022 s, 154 MB/s 2024-04-25T10:02:56.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.714 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754152 s, 139 MB/s 2024-04-25T10:02:56.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.723 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.723 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714733 s, 147 MB/s 2024-04-25T10:02:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066285 s, 158 MB/s 2024-04-25T10:02:56.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.738 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664062 s, 158 MB/s 2024-04-25T10:02:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639573 s, 164 MB/s 2024-04-25T10:02:56.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.753 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075169 s, 139 MB/s 2024-04-25T10:02:56.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:56.754 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:56.755 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:56.755 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:02:56.755 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:56.755 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:56.755 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:56.755 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:56.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606351 s, 173 MB/s 2024-04-25T10:02:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.827 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073843 s, 142 MB/s 2024-04-25T10:02:56.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.834 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664595 s, 158 MB/s 2024-04-25T10:02:56.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665011 s, 158 MB/s 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889485 s, 118 MB/s 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665672 s, 158 MB/s 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661159 s, 159 MB/s 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071485 s, 147 MB/s 2024-04-25T10:02:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.882 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652429 s, 161 MB/s 2024-04-25T10:02:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.891 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.892 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877685 s, 119 MB/s 2024-04-25T10:02:56.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605893 s, 173 MB/s 2024-04-25T10:02:56.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628485 s, 167 MB/s 2024-04-25T10:02:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.914 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717133 s, 146 MB/s 2024-04-25T10:02:56.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065076 s, 161 MB/s 2024-04-25T10:02:56.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.929 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679438 s, 154 MB/s 2024-04-25T10:02:56.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.937 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696593 s, 151 MB/s 2024-04-25T10:02:56.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.946 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800028 s, 131 MB/s 2024-04-25T10:02:56.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640816 s, 164 MB/s 2024-04-25T10:02:56.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.960 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658207 s, 159 MB/s 2024-04-25T10:02:56.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646817 s, 162 MB/s 2024-04-25T10:02:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.976 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690776 s, 152 MB/s 2024-04-25T10:02:56.983 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.983 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.984 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690708 s, 152 MB/s 2024-04-25T10:02:56.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.991 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648353 s, 162 MB/s 2024-04-25T10:02:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:56.998 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628268 s, 167 MB/s 2024-04-25T10:02:57.005 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.005 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.005 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593981 s, 177 MB/s 2024-04-25T10:02:57.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.012 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639562 s, 164 MB/s 2024-04-25T10:02:57.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636857 s, 165 MB/s 2024-04-25T10:02:57.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.027 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648226 s, 162 MB/s 2024-04-25T10:02:57.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.033 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569106 s, 184 MB/s 2024-04-25T10:02:57.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667264 s, 157 MB/s 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694363 s, 151 MB/s 2024-04-25T10:02:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653823 s, 160 MB/s 2024-04-25T10:02:57.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:57.057 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:57.057 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:57.058 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:02:57.058 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:57.058 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:57.058 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:57.058 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:57.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570955 s, 184 MB/s 2024-04-25T10:02:57.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643913 s, 163 MB/s 2024-04-25T10:02:57.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636054 s, 165 MB/s 2024-04-25T10:02:57.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.142 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570545 s, 184 MB/s 2024-04-25T10:02:57.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649345 s, 161 MB/s 2024-04-25T10:02:57.156 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.156 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.156 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570597 s, 184 MB/s 2024-04-25T10:02:57.164 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.164 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.164 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665139 s, 158 MB/s 2024-04-25T10:02:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725471 s, 145 MB/s 2024-04-25T10:02:57.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.179 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063533 s, 165 MB/s 2024-04-25T10:02:57.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.187 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725363 s, 145 MB/s 2024-04-25T10:02:57.194 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.194 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.194 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639436 s, 164 MB/s 2024-04-25T10:02:57.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.201 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.201 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558012 s, 188 MB/s 2024-04-25T10:02:57.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.209 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702152 s, 149 MB/s 2024-04-25T10:02:57.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.217 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.217 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723852 s, 145 MB/s 2024-04-25T10:02:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652389 s, 161 MB/s 2024-04-25T10:02:57.231 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.231 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.231 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577149 s, 182 MB/s 2024-04-25T10:02:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.239 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697246 s, 150 MB/s 2024-04-25T10:02:57.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.247 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711205 s, 147 MB/s 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645355 s, 162 MB/s 2024-04-25T10:02:57.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619471 s, 169 MB/s 2024-04-25T10:02:57.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636257 s, 165 MB/s 2024-04-25T10:02:57.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.276 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659047 s, 159 MB/s 2024-04-25T10:02:57.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.282 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.283 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056587 s, 185 MB/s 2024-04-25T10:02:57.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.291 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748303 s, 140 MB/s 2024-04-25T10:02:57.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727194 s, 144 MB/s 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728698 s, 144 MB/s 2024-04-25T10:02:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692273 s, 151 MB/s 2024-04-25T10:02:57.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.323 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659148 s, 159 MB/s 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.331 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.331 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700198 s, 150 MB/s 2024-04-25T10:02:57.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727886 s, 144 MB/s 2024-04-25T10:02:57.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649393 s, 161 MB/s 2024-04-25T10:02:57.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.354 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743131 s, 141 MB/s 2024-04-25T10:02:57.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:57.355 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:57.355 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:57.356 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:02:57.356 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:57.356 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:57.356 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:57.356 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626226 s, 167 MB/s 2024-04-25T10:02:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.426 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595434 s, 176 MB/s 2024-04-25T10:02:57.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054959 s, 191 MB/s 2024-04-25T10:02:57.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620916 s, 169 MB/s 2024-04-25T10:02:57.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701867 s, 149 MB/s 2024-04-25T10:02:57.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646669 s, 162 MB/s 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.461 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058079 s, 181 MB/s 2024-04-25T10:02:57.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671429 s, 156 MB/s 2024-04-25T10:02:57.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644124 s, 163 MB/s 2024-04-25T10:02:57.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.484 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732616 s, 143 MB/s 2024-04-25T10:02:57.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570237 s, 184 MB/s 2024-04-25T10:02:57.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.497 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00538458 s, 195 MB/s 2024-04-25T10:02:57.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684364 s, 153 MB/s 2024-04-25T10:02:57.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713996 s, 147 MB/s 2024-04-25T10:02:57.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744721 s, 141 MB/s 2024-04-25T10:02:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.531 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879734 s, 119 MB/s 2024-04-25T10:02:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.538 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659365 s, 159 MB/s 2024-04-25T10:02:57.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.546 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.546 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629941 s, 166 MB/s 2024-04-25T10:02:57.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.554 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.554 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719936 s, 146 MB/s 2024-04-25T10:02:57.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.560 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.561 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589398 s, 178 MB/s 2024-04-25T10:02:57.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570021 s, 184 MB/s 2024-04-25T10:02:57.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.574 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.574 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566231 s, 185 MB/s 2024-04-25T10:02:57.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750381 s, 140 MB/s 2024-04-25T10:02:57.589 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.590 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662163 s, 158 MB/s 2024-04-25T10:02:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.597 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638494 s, 164 MB/s 2024-04-25T10:02:57.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.604 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621385 s, 169 MB/s 2024-04-25T10:02:57.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616071 s, 170 MB/s 2024-04-25T10:02:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649091 s, 162 MB/s 2024-04-25T10:02:57.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.626 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649832 s, 161 MB/s 2024-04-25T10:02:57.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636717 s, 165 MB/s 2024-04-25T10:02:57.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.641 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652482 s, 161 MB/s 2024-04-25T10:02:57.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.648 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613845 s, 171 MB/s 2024-04-25T10:02:57.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:57.649 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:57.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668864 s, 157 MB/s 2024-04-25T10:02:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.720 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634076 s, 165 MB/s 2024-04-25T10:02:57.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.727 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632196 s, 166 MB/s 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722252 s, 145 MB/s 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586701 s, 179 MB/s 2024-04-25T10:02:57.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.750 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675098 s, 155 MB/s 2024-04-25T10:02:57.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585668 s, 179 MB/s 2024-04-25T10:02:57.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.763 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.763 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585839 s, 179 MB/s 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.771 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708517 s, 148 MB/s 2024-04-25T10:02:57.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744291 s, 141 MB/s 2024-04-25T10:02:57.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.787 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067844 s, 155 MB/s 2024-04-25T10:02:57.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.795 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637532 s, 164 MB/s 2024-04-25T10:02:57.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.802 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656807 s, 160 MB/s 2024-04-25T10:02:57.809 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.809 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638824 s, 164 MB/s 2024-04-25T10:02:57.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.817 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.817 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680811 s, 154 MB/s 2024-04-25T10:02:57.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.825 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.825 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736849 s, 142 MB/s 2024-04-25T10:02:57.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.834 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739844 s, 142 MB/s 2024-04-25T10:02:57.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074088 s, 142 MB/s 2024-04-25T10:02:57.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.850 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719627 s, 146 MB/s 2024-04-25T10:02:57.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.857 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.858 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644575 s, 163 MB/s 2024-04-25T10:02:57.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.865 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704055 s, 149 MB/s 2024-04-25T10:02:57.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.872 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575181 s, 182 MB/s 2024-04-25T10:02:57.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764687 s, 137 MB/s 2024-04-25T10:02:57.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.887 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570904 s, 184 MB/s 2024-04-25T10:02:57.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.895 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683633 s, 153 MB/s 2024-04-25T10:02:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.903 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736025 s, 142 MB/s 2024-04-25T10:02:57.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709234 s, 148 MB/s 2024-04-25T10:02:57.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721346 s, 145 MB/s 2024-04-25T10:02:57.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.927 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633092 s, 166 MB/s 2024-04-25T10:02:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650934 s, 161 MB/s 2024-04-25T10:02:57.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595486 s, 176 MB/s 2024-04-25T10:02:57.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:57.949 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759546 s, 138 MB/s 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:57.951 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636687 s, 165 MB/s 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650718 s, 161 MB/s 2024-04-25T10:02:58.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.033 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907127 s, 116 MB/s 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.039 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591392 s, 177 MB/s 2024-04-25T10:02:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.047 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676183 s, 155 MB/s 2024-04-25T10:02:58.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.055 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711541 s, 147 MB/s 2024-04-25T10:02:58.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.062 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610208 s, 172 MB/s 2024-04-25T10:02:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.070 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715606 s, 147 MB/s 2024-04-25T10:02:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700102 s, 150 MB/s 2024-04-25T10:02:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660064 s, 159 MB/s 2024-04-25T10:02:58.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.092 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573935 s, 183 MB/s 2024-04-25T10:02:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.099 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617135 s, 170 MB/s 2024-04-25T10:02:58.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.107 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707717 s, 148 MB/s 2024-04-25T10:02:58.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579791 s, 181 MB/s 2024-04-25T10:02:58.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719056 s, 146 MB/s 2024-04-25T10:02:58.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644264 s, 163 MB/s 2024-04-25T10:02:58.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653158 s, 161 MB/s 2024-04-25T10:02:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.144 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062635 s, 167 MB/s 2024-04-25T10:02:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055421 s, 189 MB/s 2024-04-25T10:02:58.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.157 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656267 s, 160 MB/s 2024-04-25T10:02:58.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065177 s, 161 MB/s 2024-04-25T10:02:58.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.171 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.172 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573743 s, 183 MB/s 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737907 s, 142 MB/s 2024-04-25T10:02:58.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.186 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556147 s, 189 MB/s 2024-04-25T10:02:58.194 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.194 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.194 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665731 s, 158 MB/s 2024-04-25T10:02:58.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.204 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965171 s, 109 MB/s 2024-04-25T10:02:58.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.213 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736432 s, 142 MB/s 2024-04-25T10:02:58.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862958 s, 122 MB/s 2024-04-25T10:02:58.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.230 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658429 s, 159 MB/s 2024-04-25T10:02:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686555 s, 153 MB/s 2024-04-25T10:02:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701814 s, 149 MB/s 2024-04-25T10:02:58.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.253 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716467 s, 146 MB/s 2024-04-25T10:02:58.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:58.254 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:58.255 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:58.255 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:02:58.255 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:58.255 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:58.255 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:58.255 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:58.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.318 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552262 s, 190 MB/s 2024-04-25T10:02:58.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.326 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761095 s, 138 MB/s 2024-04-25T10:02:58.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875854 s, 120 MB/s 2024-04-25T10:02:58.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.344 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694423 s, 151 MB/s 2024-04-25T10:02:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562216 s, 187 MB/s 2024-04-25T10:02:58.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.357 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636902 s, 165 MB/s 2024-04-25T10:02:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563548 s, 186 MB/s 2024-04-25T10:02:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.371 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064347 s, 163 MB/s 2024-04-25T10:02:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709142 s, 148 MB/s 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660028 s, 159 MB/s 2024-04-25T10:02:58.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746045 s, 141 MB/s 2024-04-25T10:02:58.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.403 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661073 s, 159 MB/s 2024-04-25T10:02:58.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.410 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625535 s, 168 MB/s 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645201 s, 163 MB/s 2024-04-25T10:02:58.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059744 s, 176 MB/s 2024-04-25T10:02:58.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660327 s, 159 MB/s 2024-04-25T10:02:58.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.438 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586192 s, 179 MB/s 2024-04-25T10:02:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571887 s, 183 MB/s 2024-04-25T10:02:58.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.453 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712342 s, 147 MB/s 2024-04-25T10:02:58.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576525 s, 182 MB/s 2024-04-25T10:02:58.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.468 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712338 s, 147 MB/s 2024-04-25T10:02:58.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.475 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642033 s, 163 MB/s 2024-04-25T10:02:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.482 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585305 s, 179 MB/s 2024-04-25T10:02:58.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.489 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.489 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063718 s, 165 MB/s 2024-04-25T10:02:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649562 s, 161 MB/s 2024-04-25T10:02:58.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721016 s, 145 MB/s 2024-04-25T10:02:58.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721902 s, 145 MB/s 2024-04-25T10:02:58.520 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.520 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.520 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632028 s, 166 MB/s 2024-04-25T10:02:58.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.527 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621107 s, 169 MB/s 2024-04-25T10:02:58.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708413 s, 148 MB/s 2024-04-25T10:02:58.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.543 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690236 s, 152 MB/s 2024-04-25T10:02:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645846 s, 162 MB/s 2024-04-25T10:02:58.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:58.551 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:58.551 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:58.551 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:02:58.551 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:58.552 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:58.552 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:58.552 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:58.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729885 s, 144 MB/s 2024-04-25T10:02:58.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661861 s, 158 MB/s 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655756 s, 160 MB/s 2024-04-25T10:02:58.644 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.644 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.644 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115488 s, 90.8 MB/s 2024-04-25T10:02:58.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706926 s, 148 MB/s 2024-04-25T10:02:58.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701042 s, 150 MB/s 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664892 s, 158 MB/s 2024-04-25T10:02:58.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.674 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.674 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640715 s, 164 MB/s 2024-04-25T10:02:58.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720973 s, 145 MB/s 2024-04-25T10:02:58.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.690 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667138 s, 157 MB/s 2024-04-25T10:02:58.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.698 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613796 s, 171 MB/s 2024-04-25T10:02:58.704 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.704 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.704 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641492 s, 163 MB/s 2024-04-25T10:02:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571618 s, 183 MB/s 2024-04-25T10:02:58.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.717 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056002 s, 187 MB/s 2024-04-25T10:02:58.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.724 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596127 s, 176 MB/s 2024-04-25T10:02:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.732 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651241 s, 161 MB/s 2024-04-25T10:02:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.739 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592434 s, 177 MB/s 2024-04-25T10:02:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571431 s, 183 MB/s 2024-04-25T10:02:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.753 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679414 s, 154 MB/s 2024-04-25T10:02:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.761 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682491 s, 154 MB/s 2024-04-25T10:02:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642912 s, 163 MB/s 2024-04-25T10:02:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.775 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057587 s, 182 MB/s 2024-04-25T10:02:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.784 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871326 s, 120 MB/s 2024-04-25T10:02:58.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.792 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636248 s, 165 MB/s 2024-04-25T10:02:58.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.799 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066974 s, 157 MB/s 2024-04-25T10:02:58.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700145 s, 150 MB/s 2024-04-25T10:02:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.814 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622005 s, 169 MB/s 2024-04-25T10:02:58.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062704 s, 167 MB/s 2024-04-25T10:02:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.828 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572047 s, 183 MB/s 2024-04-25T10:02:58.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.836 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718202 s, 146 MB/s 2024-04-25T10:02:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807553 s, 130 MB/s 2024-04-25T10:02:58.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611111 s, 172 MB/s 2024-04-25T10:02:58.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:58.853 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:58.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.918 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685116 s, 153 MB/s 2024-04-25T10:02:58.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.925 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646058 s, 162 MB/s 2024-04-25T10:02:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.932 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650599 s, 161 MB/s 2024-04-25T10:02:58.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.940 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633962 s, 165 MB/s 2024-04-25T10:02:58.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636429 s, 165 MB/s 2024-04-25T10:02:58.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.955 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634348 s, 165 MB/s 2024-04-25T10:02:58.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.962 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711828 s, 147 MB/s 2024-04-25T10:02:58.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.970 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704063 s, 149 MB/s 2024-04-25T10:02:58.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665943 s, 157 MB/s 2024-04-25T10:02:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665088 s, 158 MB/s 2024-04-25T10:02:58.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:58.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:58.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621563 s, 169 MB/s 2024-04-25T10:02:59.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665624 s, 158 MB/s 2024-04-25T10:02:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.006 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556165 s, 189 MB/s 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558337 s, 188 MB/s 2024-04-25T10:02:59.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.020 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645655 s, 162 MB/s 2024-04-25T10:02:59.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657094 s, 160 MB/s 2024-04-25T10:02:59.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.036 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072791 s, 144 MB/s 2024-04-25T10:02:59.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.044 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716045 s, 146 MB/s 2024-04-25T10:02:59.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735432 s, 143 MB/s 2024-04-25T10:02:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.062 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866831 s, 121 MB/s 2024-04-25T10:02:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.069 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660127 s, 159 MB/s 2024-04-25T10:02:59.077 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.077 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.077 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701036 s, 150 MB/s 2024-04-25T10:02:59.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.084 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560347 s, 187 MB/s 2024-04-25T10:02:59.091 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.092 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707905 s, 148 MB/s 2024-04-25T10:02:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.099 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646503 s, 162 MB/s 2024-04-25T10:02:59.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672504 s, 156 MB/s 2024-04-25T10:02:59.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740413 s, 142 MB/s 2024-04-25T10:02:59.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663756 s, 158 MB/s 2024-04-25T10:02:59.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603651 s, 174 MB/s 2024-04-25T10:02:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.137 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688106 s, 152 MB/s 2024-04-25T10:02:59.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561925 s, 187 MB/s 2024-04-25T10:02:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656361 s, 160 MB/s 2024-04-25T10:02:59.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.152 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:59.152 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:59.152 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:02:59.152 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:59.152 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:59.152 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:59.153 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:59.216 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.216 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.216 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578908 s, 181 MB/s 2024-04-25T10:02:59.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.224 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704121 s, 149 MB/s 2024-04-25T10:02:59.234 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.234 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.234 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894293 s, 117 MB/s 2024-04-25T10:02:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709462 s, 148 MB/s 2024-04-25T10:02:59.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.249 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636208 s, 165 MB/s 2024-04-25T10:02:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635091 s, 165 MB/s 2024-04-25T10:02:59.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632259 s, 166 MB/s 2024-04-25T10:02:59.271 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.271 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.271 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703941 s, 149 MB/s 2024-04-25T10:02:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.278 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646597 s, 162 MB/s 2024-04-25T10:02:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066399 s, 158 MB/s 2024-04-25T10:02:59.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858381 s, 122 MB/s 2024-04-25T10:02:59.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.303 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633928 s, 165 MB/s 2024-04-25T10:02:59.310 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.310 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.310 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648078 s, 162 MB/s 2024-04-25T10:02:59.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.318 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065887 s, 159 MB/s 2024-04-25T10:02:59.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.324 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061381 s, 171 MB/s 2024-04-25T10:02:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.332 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636138 s, 165 MB/s 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628198 s, 167 MB/s 2024-04-25T10:02:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597318 s, 176 MB/s 2024-04-25T10:02:59.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.353 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553205 s, 190 MB/s 2024-04-25T10:02:59.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.359 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608792 s, 172 MB/s 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.367 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624107 s, 168 MB/s 2024-04-25T10:02:59.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.373 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593741 s, 177 MB/s 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.382 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715127 s, 147 MB/s 2024-04-25T10:02:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.389 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638102 s, 164 MB/s 2024-04-25T10:02:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567769 s, 185 MB/s 2024-04-25T10:02:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.403 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708772 s, 148 MB/s 2024-04-25T10:02:59.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.411 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.411 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657991 s, 159 MB/s 2024-04-25T10:02:59.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721779 s, 145 MB/s 2024-04-25T10:02:59.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.427 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687595 s, 152 MB/s 2024-04-25T10:02:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.434 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633064 s, 166 MB/s 2024-04-25T10:02:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989622 s, 106 MB/s 2024-04-25T10:02:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.453 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739678 s, 142 MB/s 2024-04-25T10:02:59.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:59.458 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:59.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.519 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686955 s, 153 MB/s 2024-04-25T10:02:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.526 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658057 s, 159 MB/s 2024-04-25T10:02:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067845 s, 155 MB/s 2024-04-25T10:02:59.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.540 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588962 s, 178 MB/s 2024-04-25T10:02:59.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653097 s, 161 MB/s 2024-04-25T10:02:59.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.556 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065477 s, 160 MB/s 2024-04-25T10:02:59.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.563 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.563 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645599 s, 162 MB/s 2024-04-25T10:02:59.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.571 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711288 s, 147 MB/s 2024-04-25T10:02:59.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.578 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.578 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656254 s, 160 MB/s 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635288 s, 165 MB/s 2024-04-25T10:02:59.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.592 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638864 s, 164 MB/s 2024-04-25T10:02:59.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706749 s, 148 MB/s 2024-04-25T10:02:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.608 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696464 s, 151 MB/s 2024-04-25T10:02:59.615 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.616 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673816 s, 156 MB/s 2024-04-25T10:02:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.622 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554951 s, 189 MB/s 2024-04-25T10:02:59.629 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.629 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.629 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567955 s, 185 MB/s 2024-04-25T10:02:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.638 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861204 s, 122 MB/s 2024-04-25T10:02:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630981 s, 166 MB/s 2024-04-25T10:02:59.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625244 s, 168 MB/s 2024-04-25T10:02:59.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655521 s, 160 MB/s 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.667 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608938 s, 172 MB/s 2024-04-25T10:02:59.675 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718307 s, 146 MB/s 2024-04-25T10:02:59.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644261 s, 163 MB/s 2024-04-25T10:02:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.689 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610933 s, 172 MB/s 2024-04-25T10:02:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865463 s, 121 MB/s 2024-04-25T10:02:59.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.707 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725295 s, 145 MB/s 2024-04-25T10:02:59.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704416 s, 149 MB/s 2024-04-25T10:02:59.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626449 s, 167 MB/s 2024-04-25T10:02:59.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565217 s, 186 MB/s 2024-04-25T10:02:59.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.735 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550485 s, 190 MB/s 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.742 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654281 s, 160 MB/s 2024-04-25T10:02:59.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073445 s, 143 MB/s 2024-04-25T10:02:59.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:59.752 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:02:59.752 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:02:59.752 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:02:59.753 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:02:59.753 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:02:59.753 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:02:59.753 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:02:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583846 s, 180 MB/s 2024-04-25T10:02:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.824 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719787 s, 146 MB/s 2024-04-25T10:02:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.832 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682513 s, 154 MB/s 2024-04-25T10:02:59.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733606 s, 143 MB/s 2024-04-25T10:02:59.848 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.848 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.848 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069016 s, 152 MB/s 2024-04-25T10:02:59.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.856 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.856 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693633 s, 151 MB/s 2024-04-25T10:02:59.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.863 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.863 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661979 s, 158 MB/s 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815258 s, 129 MB/s 2024-04-25T10:02:59.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.880 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695993 s, 151 MB/s 2024-04-25T10:02:59.887 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.888 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679725 s, 154 MB/s 2024-04-25T10:02:59.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.895 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642319 s, 163 MB/s 2024-04-25T10:02:59.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.903 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686445 s, 153 MB/s 2024-04-25T10:02:59.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071878 s, 146 MB/s 2024-04-25T10:02:59.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.922 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652784 s, 161 MB/s 2024-04-25T10:02:59.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.925 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622842 s, 168 MB/s 2024-04-25T10:02:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700484 s, 150 MB/s 2024-04-25T10:02:59.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.941 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661819 s, 158 MB/s 2024-04-25T10:02:59.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.948 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636225 s, 165 MB/s 2024-04-25T10:02:59.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681669 s, 154 MB/s 2024-04-25T10:02:59.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664494 s, 158 MB/s 2024-04-25T10:02:59.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.971 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703897 s, 149 MB/s 2024-04-25T10:02:59.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575864 s, 182 MB/s 2024-04-25T10:02:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.984 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577662 s, 182 MB/s 2024-04-25T10:02:59.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:02:59.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:02:59.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688335 s, 152 MB/s 2024-04-25T10:03:00.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.001 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736968 s, 142 MB/s 2024-04-25T10:03:00.008 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.009 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714834 s, 147 MB/s 2024-04-25T10:03:00.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006049 s, 173 MB/s 2024-04-25T10:03:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661876 s, 158 MB/s 2024-04-25T10:03:00.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.031 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683667 s, 153 MB/s 2024-04-25T10:03:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731807 s, 143 MB/s 2024-04-25T10:03:00.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.047 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639399 s, 164 MB/s 2024-04-25T10:03:00.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.055 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080929 s, 130 MB/s 2024-04-25T10:03:00.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.056 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:00.057 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:00.057 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:03:00.057 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:00.057 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:00.057 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:00.057 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:00.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749069 s, 140 MB/s 2024-04-25T10:03:00.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638508 s, 164 MB/s 2024-04-25T10:03:00.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671401 s, 156 MB/s 2024-04-25T10:03:00.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595072 s, 176 MB/s 2024-04-25T10:03:00.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636841 s, 165 MB/s 2024-04-25T10:03:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.157 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575168 s, 182 MB/s 2024-04-25T10:03:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712013 s, 147 MB/s 2024-04-25T10:03:00.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.172 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.173 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659352 s, 159 MB/s 2024-04-25T10:03:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722831 s, 145 MB/s 2024-04-25T10:03:00.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.192 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103525 s, 101 MB/s 2024-04-25T10:03:00.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.199 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633879 s, 165 MB/s 2024-04-25T10:03:00.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.207 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711289 s, 147 MB/s 2024-04-25T10:03:00.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.215 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735354 s, 143 MB/s 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563007 s, 186 MB/s 2024-04-25T10:03:00.229 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.230 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671589 s, 156 MB/s 2024-04-25T10:03:00.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.237 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.237 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642211 s, 163 MB/s 2024-04-25T10:03:00.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738317 s, 142 MB/s 2024-04-25T10:03:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625224 s, 168 MB/s 2024-04-25T10:03:00.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804698 s, 130 MB/s 2024-04-25T10:03:00.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691812 s, 152 MB/s 2024-04-25T10:03:00.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.276 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655309 s, 160 MB/s 2024-04-25T10:03:00.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.285 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.285 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731594 s, 143 MB/s 2024-04-25T10:03:00.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616271 s, 170 MB/s 2024-04-25T10:03:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.301 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864092 s, 121 MB/s 2024-04-25T10:03:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737582 s, 142 MB/s 2024-04-25T10:03:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.317 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651249 s, 161 MB/s 2024-04-25T10:03:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.324 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657907 s, 159 MB/s 2024-04-25T10:03:00.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.334 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062211 s, 169 MB/s 2024-04-25T10:03:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718844 s, 146 MB/s 2024-04-25T10:03:00.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.347 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635462 s, 165 MB/s 2024-04-25T10:03:00.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.354 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634596 s, 165 MB/s 2024-04-25T10:03:00.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.362 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667917 s, 157 MB/s 2024-04-25T10:03:00.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:00.363 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:00.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637928 s, 164 MB/s 2024-04-25T10:03:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.436 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713468 s, 147 MB/s 2024-04-25T10:03:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.442 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005849 s, 179 MB/s 2024-04-25T10:03:00.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.450 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651379 s, 161 MB/s 2024-04-25T10:03:00.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932941 s, 112 MB/s 2024-04-25T10:03:00.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619761 s, 169 MB/s 2024-04-25T10:03:00.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640436 s, 164 MB/s 2024-04-25T10:03:00.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.481 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578805 s, 181 MB/s 2024-04-25T10:03:00.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.488 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630144 s, 166 MB/s 2024-04-25T10:03:00.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.496 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.496 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066038 s, 159 MB/s 2024-04-25T10:03:00.502 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.502 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.502 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564818 s, 186 MB/s 2024-04-25T10:03:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.509 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618943 s, 169 MB/s 2024-04-25T10:03:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643409 s, 163 MB/s 2024-04-25T10:03:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739007 s, 142 MB/s 2024-04-25T10:03:00.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.532 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627032 s, 167 MB/s 2024-04-25T10:03:00.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.540 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712616 s, 147 MB/s 2024-04-25T10:03:00.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635125 s, 165 MB/s 2024-04-25T10:03:00.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.555 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637229 s, 165 MB/s 2024-04-25T10:03:00.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.563 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693792 s, 151 MB/s 2024-04-25T10:03:00.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.569 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057912 s, 181 MB/s 2024-04-25T10:03:00.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.577 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685763 s, 153 MB/s 2024-04-25T10:03:00.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713404 s, 147 MB/s 2024-04-25T10:03:00.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673979 s, 156 MB/s 2024-04-25T10:03:00.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070033 s, 150 MB/s 2024-04-25T10:03:00.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092149 s, 114 MB/s 2024-04-25T10:03:00.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709832 s, 148 MB/s 2024-04-25T10:03:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.625 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602099 s, 174 MB/s 2024-04-25T10:03:00.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677826 s, 155 MB/s 2024-04-25T10:03:00.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.641 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715728 s, 147 MB/s 2024-04-25T10:03:00.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.649 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688308 s, 152 MB/s 2024-04-25T10:03:00.656 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.657 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688248 s, 152 MB/s 2024-04-25T10:03:00.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.664 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638609 s, 164 MB/s 2024-04-25T10:03:00.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.665 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:00.665 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:00.665 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:03:00.665 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:00.665 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:00.665 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:00.666 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632995 s, 166 MB/s 2024-04-25T10:03:00.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056122 s, 187 MB/s 2024-04-25T10:03:00.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678766 s, 154 MB/s 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716286 s, 146 MB/s 2024-04-25T10:03:00.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.760 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669911 s, 157 MB/s 2024-04-25T10:03:00.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725456 s, 145 MB/s 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671507 s, 156 MB/s 2024-04-25T10:03:00.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.783 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653671 s, 160 MB/s 2024-04-25T10:03:00.790 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.791 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662524 s, 158 MB/s 2024-04-25T10:03:00.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.799 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.799 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723414 s, 145 MB/s 2024-04-25T10:03:00.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.806 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068497 s, 153 MB/s 2024-04-25T10:03:00.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.813 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611706 s, 171 MB/s 2024-04-25T10:03:00.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.820 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058285 s, 180 MB/s 2024-04-25T10:03:00.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.827 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568451 s, 184 MB/s 2024-04-25T10:03:00.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.834 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640156 s, 164 MB/s 2024-04-25T10:03:00.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737866 s, 142 MB/s 2024-04-25T10:03:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659239 s, 159 MB/s 2024-04-25T10:03:00.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.858 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740512 s, 142 MB/s 2024-04-25T10:03:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.866 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673961 s, 156 MB/s 2024-04-25T10:03:00.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.872 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.873 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581423 s, 180 MB/s 2024-04-25T10:03:00.879 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.880 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637344 s, 165 MB/s 2024-04-25T10:03:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.888 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696424 s, 151 MB/s 2024-04-25T10:03:00.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.895 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061492 s, 171 MB/s 2024-04-25T10:03:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.903 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721338 s, 145 MB/s 2024-04-25T10:03:00.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667112 s, 157 MB/s 2024-04-25T10:03:00.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.918 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716081 s, 146 MB/s 2024-04-25T10:03:00.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.925 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577913 s, 181 MB/s 2024-04-25T10:03:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.933 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668834 s, 157 MB/s 2024-04-25T10:03:00.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.941 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697728 s, 150 MB/s 2024-04-25T10:03:00.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746483 s, 140 MB/s 2024-04-25T10:03:00.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635302 s, 165 MB/s 2024-04-25T10:03:00.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:00.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:00.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631243 s, 166 MB/s 2024-04-25T10:03:00.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:00.964 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:00.964 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:00.965 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:03:00.965 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:00.965 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:00.965 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:00.965 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.029 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669108 s, 157 MB/s 2024-04-25T10:03:01.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730714 s, 144 MB/s 2024-04-25T10:03:01.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.043 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556907 s, 188 MB/s 2024-04-25T10:03:01.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.051 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680096 s, 154 MB/s 2024-04-25T10:03:01.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.058 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636658 s, 165 MB/s 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661408 s, 159 MB/s 2024-04-25T10:03:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642347 s, 163 MB/s 2024-04-25T10:03:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645058 s, 163 MB/s 2024-04-25T10:03:01.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.088 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069106 s, 152 MB/s 2024-04-25T10:03:01.095 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.095 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.095 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586003 s, 179 MB/s 2024-04-25T10:03:01.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.102 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633266 s, 166 MB/s 2024-04-25T10:03:01.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.110 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704104 s, 149 MB/s 2024-04-25T10:03:01.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.117 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578068 s, 181 MB/s 2024-04-25T10:03:01.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.124 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628797 s, 167 MB/s 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640012 s, 164 MB/s 2024-04-25T10:03:01.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596481 s, 176 MB/s 2024-04-25T10:03:01.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.146 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713486 s, 147 MB/s 2024-04-25T10:03:01.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.155 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.155 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671629 s, 156 MB/s 2024-04-25T10:03:01.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.162 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731133 s, 143 MB/s 2024-04-25T10:03:01.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557582 s, 188 MB/s 2024-04-25T10:03:01.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.176 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639304 s, 164 MB/s 2024-04-25T10:03:01.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642149 s, 163 MB/s 2024-04-25T10:03:01.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652681 s, 161 MB/s 2024-04-25T10:03:01.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710047 s, 148 MB/s 2024-04-25T10:03:01.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645105 s, 163 MB/s 2024-04-25T10:03:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.213 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638858 s, 164 MB/s 2024-04-25T10:03:01.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.220 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609119 s, 172 MB/s 2024-04-25T10:03:01.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551547 s, 190 MB/s 2024-04-25T10:03:01.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.234 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631221 s, 166 MB/s 2024-04-25T10:03:01.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.241 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.241 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664318 s, 158 MB/s 2024-04-25T10:03:01.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573726 s, 183 MB/s 2024-04-25T10:03:01.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583976 s, 180 MB/s 2024-04-25T10:03:01.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:01.256 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.321 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659211 s, 159 MB/s 2024-04-25T10:03:01.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641992 s, 163 MB/s 2024-04-25T10:03:01.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.335 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664133 s, 158 MB/s 2024-04-25T10:03:01.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073802 s, 142 MB/s 2024-04-25T10:03:01.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.351 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647623 s, 162 MB/s 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690519 s, 152 MB/s 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622759 s, 168 MB/s 2024-04-25T10:03:01.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.373 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618497 s, 170 MB/s 2024-04-25T10:03:01.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637596 s, 164 MB/s 2024-04-25T10:03:01.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673857 s, 156 MB/s 2024-04-25T10:03:01.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063523 s, 165 MB/s 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633725 s, 165 MB/s 2024-04-25T10:03:01.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.410 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749627 s, 140 MB/s 2024-04-25T10:03:01.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.418 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.418 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671419 s, 156 MB/s 2024-04-25T10:03:01.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.425 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629213 s, 167 MB/s 2024-04-25T10:03:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.433 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715833 s, 146 MB/s 2024-04-25T10:03:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.441 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661715 s, 158 MB/s 2024-04-25T10:03:01.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577452 s, 182 MB/s 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669143 s, 157 MB/s 2024-04-25T10:03:01.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612559 s, 171 MB/s 2024-04-25T10:03:01.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653413 s, 160 MB/s 2024-04-25T10:03:01.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.477 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668453 s, 157 MB/s 2024-04-25T10:03:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719049 s, 146 MB/s 2024-04-25T10:03:01.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.493 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.493 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723296 s, 145 MB/s 2024-04-25T10:03:01.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.501 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.501 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743602 s, 141 MB/s 2024-04-25T10:03:01.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.509 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668261 s, 157 MB/s 2024-04-25T10:03:01.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740924 s, 142 MB/s 2024-04-25T10:03:01.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.526 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796414 s, 132 MB/s 2024-04-25T10:03:01.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.533 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635869 s, 165 MB/s 2024-04-25T10:03:01.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675303 s, 155 MB/s 2024-04-25T10:03:01.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.549 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733533 s, 143 MB/s 2024-04-25T10:03:01.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.556 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063561 s, 165 MB/s 2024-04-25T10:03:01.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:01.558 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:01.622 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.622 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.622 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648699 s, 162 MB/s 2024-04-25T10:03:01.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.630 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.630 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734839 s, 143 MB/s 2024-04-25T10:03:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.637 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.638 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646389 s, 162 MB/s 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727195 s, 144 MB/s 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597522 s, 175 MB/s 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670873 s, 156 MB/s 2024-04-25T10:03:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754168 s, 139 MB/s 2024-04-25T10:03:01.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653075 s, 161 MB/s 2024-04-25T10:03:01.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.683 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650521 s, 161 MB/s 2024-04-25T10:03:01.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.691 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633456 s, 166 MB/s 2024-04-25T10:03:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.698 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660616 s, 159 MB/s 2024-04-25T10:03:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.705 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619021 s, 169 MB/s 2024-04-25T10:03:01.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677085 s, 155 MB/s 2024-04-25T10:03:01.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.721 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733311 s, 143 MB/s 2024-04-25T10:03:01.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063164 s, 166 MB/s 2024-04-25T10:03:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.736 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697147 s, 150 MB/s 2024-04-25T10:03:01.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.743 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571428 s, 184 MB/s 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.750 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006623 s, 158 MB/s 2024-04-25T10:03:01.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.757 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061257 s, 171 MB/s 2024-04-25T10:03:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564426 s, 186 MB/s 2024-04-25T10:03:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.770 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577556 s, 182 MB/s 2024-04-25T10:03:01.777 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641024 s, 164 MB/s 2024-04-25T10:03:01.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.785 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691384 s, 152 MB/s 2024-04-25T10:03:01.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066494 s, 158 MB/s 2024-04-25T10:03:01.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642123 s, 163 MB/s 2024-04-25T10:03:01.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639292 s, 164 MB/s 2024-04-25T10:03:01.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.816 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.816 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698656 s, 150 MB/s 2024-04-25T10:03:01.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.822 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632569 s, 166 MB/s 2024-04-25T10:03:01.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.829 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057314 s, 183 MB/s 2024-04-25T10:03:01.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.837 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695465 s, 151 MB/s 2024-04-25T10:03:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.844 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647614 s, 162 MB/s 2024-04-25T10:03:01.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.851 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581953 s, 180 MB/s 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:01.852 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:01.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.916 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587159 s, 179 MB/s 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556947 s, 188 MB/s 2024-04-25T10:03:01.929 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.930 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668022 s, 157 MB/s 2024-04-25T10:03:01.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.937 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664315 s, 158 MB/s 2024-04-25T10:03:01.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.944 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566351 s, 185 MB/s 2024-04-25T10:03:01.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.952 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682106 s, 154 MB/s 2024-04-25T10:03:01.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643257 s, 163 MB/s 2024-04-25T10:03:01.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.966 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.966 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629061 s, 167 MB/s 2024-04-25T10:03:01.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638242 s, 164 MB/s 2024-04-25T10:03:01.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.981 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683417 s, 153 MB/s 2024-04-25T10:03:01.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590945 s, 177 MB/s 2024-04-25T10:03:01.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:01.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:01.995 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639958 s, 164 MB/s 2024-04-25T10:03:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.003 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630913 s, 166 MB/s 2024-04-25T10:03:02.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.010 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662295 s, 158 MB/s 2024-04-25T10:03:02.017 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.017 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.018 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658142 s, 159 MB/s 2024-04-25T10:03:02.025 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.025 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.025 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662417 s, 158 MB/s 2024-04-25T10:03:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.032 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661172 s, 159 MB/s 2024-04-25T10:03:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.039 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576004 s, 182 MB/s 2024-04-25T10:03:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.047 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006589 s, 159 MB/s 2024-04-25T10:03:02.054 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.054 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.054 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656929 s, 160 MB/s 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.062 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738576 s, 142 MB/s 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638062 s, 164 MB/s 2024-04-25T10:03:02.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721327 s, 145 MB/s 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669477 s, 157 MB/s 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622171 s, 169 MB/s 2024-04-25T10:03:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.100 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064716 s, 162 MB/s 2024-04-25T10:03:02.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.107 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630067 s, 166 MB/s 2024-04-25T10:03:02.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711456 s, 147 MB/s 2024-04-25T10:03:02.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609472 s, 172 MB/s 2024-04-25T10:03:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.130 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664083 s, 158 MB/s 2024-04-25T10:03:02.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728812 s, 144 MB/s 2024-04-25T10:03:02.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598971 s, 175 MB/s 2024-04-25T10:03:02.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:02.146 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:02.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.211 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642374 s, 163 MB/s 2024-04-25T10:03:02.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623018 s, 168 MB/s 2024-04-25T10:03:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.225 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636052 s, 165 MB/s 2024-04-25T10:03:02.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.233 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.233 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706588 s, 148 MB/s 2024-04-25T10:03:02.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563308 s, 186 MB/s 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544962 s, 192 MB/s 2024-04-25T10:03:02.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722883 s, 145 MB/s 2024-04-25T10:03:02.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.262 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701313 s, 150 MB/s 2024-04-25T10:03:02.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698145 s, 150 MB/s 2024-04-25T10:03:02.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.277 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641071 s, 164 MB/s 2024-04-25T10:03:02.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.284 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563121 s, 186 MB/s 2024-04-25T10:03:02.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.290 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561233 s, 187 MB/s 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654882 s, 160 MB/s 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651672 s, 161 MB/s 2024-04-25T10:03:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727249 s, 144 MB/s 2024-04-25T10:03:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670821 s, 156 MB/s 2024-04-25T10:03:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632994 s, 166 MB/s 2024-04-25T10:03:02.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704587 s, 149 MB/s 2024-04-25T10:03:02.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.344 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006591 s, 159 MB/s 2024-04-25T10:03:02.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.352 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699597 s, 150 MB/s 2024-04-25T10:03:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647516 s, 162 MB/s 2024-04-25T10:03:02.365 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575595 s, 182 MB/s 2024-04-25T10:03:02.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.373 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666749 s, 157 MB/s 2024-04-25T10:03:02.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.381 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654195 s, 160 MB/s 2024-04-25T10:03:02.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.389 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737087 s, 142 MB/s 2024-04-25T10:03:02.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.397 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.397 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007211 s, 145 MB/s 2024-04-25T10:03:02.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736149 s, 142 MB/s 2024-04-25T10:03:02.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.413 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633714 s, 165 MB/s 2024-04-25T10:03:02.420 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.421 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717051 s, 146 MB/s 2024-04-25T10:03:02.429 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.429 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.429 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715145 s, 147 MB/s 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.436 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652197 s, 161 MB/s 2024-04-25T10:03:02.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.444 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.444 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701698 s, 149 MB/s 2024-04-25T10:03:02.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:02.445 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:02.445 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:02.445 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:03:02.445 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:02.446 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:02.446 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:02.446 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:02.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.509 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555033 s, 189 MB/s 2024-04-25T10:03:02.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716208 s, 146 MB/s 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060415 s, 174 MB/s 2024-04-25T10:03:02.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.532 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641804 s, 163 MB/s 2024-04-25T10:03:02.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.539 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.539 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610982 s, 172 MB/s 2024-04-25T10:03:02.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.545 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569242 s, 184 MB/s 2024-04-25T10:03:02.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.553 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067362 s, 156 MB/s 2024-04-25T10:03:02.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.561 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746364 s, 140 MB/s 2024-04-25T10:03:02.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.568 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570934 s, 184 MB/s 2024-04-25T10:03:02.575 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.575 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.576 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658102 s, 159 MB/s 2024-04-25T10:03:02.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.583 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.583 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635416 s, 165 MB/s 2024-04-25T10:03:02.589 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.590 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599366 s, 175 MB/s 2024-04-25T10:03:02.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.597 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638526 s, 164 MB/s 2024-04-25T10:03:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632276 s, 166 MB/s 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640916 s, 164 MB/s 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662807 s, 158 MB/s 2024-04-25T10:03:02.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.625 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561573 s, 187 MB/s 2024-04-25T10:03:02.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638992 s, 164 MB/s 2024-04-25T10:03:02.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621635 s, 169 MB/s 2024-04-25T10:03:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631794 s, 166 MB/s 2024-04-25T10:03:02.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.655 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701602 s, 149 MB/s 2024-04-25T10:03:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.662 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651502 s, 161 MB/s 2024-04-25T10:03:02.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.670 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724955 s, 145 MB/s 2024-04-25T10:03:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578498 s, 181 MB/s 2024-04-25T10:03:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619735 s, 169 MB/s 2024-04-25T10:03:02.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.691 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634459 s, 165 MB/s 2024-04-25T10:03:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.698 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586092 s, 179 MB/s 2024-04-25T10:03:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.705 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605381 s, 173 MB/s 2024-04-25T10:03:02.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708198 s, 148 MB/s 2024-04-25T10:03:02.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.719 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548339 s, 191 MB/s 2024-04-25T10:03:02.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.727 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653055 s, 161 MB/s 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627553 s, 167 MB/s 2024-04-25T10:03:02.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:02.735 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:02.735 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:02.735 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:03:02.736 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:02.736 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:02.736 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:02.736 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.801 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774193 s, 135 MB/s 2024-04-25T10:03:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.808 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062321 s, 168 MB/s 2024-04-25T10:03:02.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.815 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584989 s, 179 MB/s 2024-04-25T10:03:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704076 s, 149 MB/s 2024-04-25T10:03:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.829 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578019 s, 181 MB/s 2024-04-25T10:03:02.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.837 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615982 s, 170 MB/s 2024-04-25T10:03:02.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.844 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710512 s, 148 MB/s 2024-04-25T10:03:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670946 s, 156 MB/s 2024-04-25T10:03:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606133 s, 173 MB/s 2024-04-25T10:03:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586815 s, 179 MB/s 2024-04-25T10:03:02.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.873 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666704 s, 157 MB/s 2024-04-25T10:03:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691379 s, 152 MB/s 2024-04-25T10:03:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702627 s, 149 MB/s 2024-04-25T10:03:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696187 s, 151 MB/s 2024-04-25T10:03:02.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.904 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605633 s, 173 MB/s 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631504 s, 166 MB/s 2024-04-25T10:03:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.918 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632597 s, 166 MB/s 2024-04-25T10:03:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.925 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585498 s, 179 MB/s 2024-04-25T10:03:02.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.932 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061926 s, 169 MB/s 2024-04-25T10:03:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.940 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635052 s, 165 MB/s 2024-04-25T10:03:02.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638232 s, 164 MB/s 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645677 s, 162 MB/s 2024-04-25T10:03:02.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564179 s, 186 MB/s 2024-04-25T10:03:02.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623211 s, 168 MB/s 2024-04-25T10:03:02.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567921 s, 185 MB/s 2024-04-25T10:03:02.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.982 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007106 s, 148 MB/s 2024-04-25T10:03:02.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.990 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.990 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627098 s, 167 MB/s 2024-04-25T10:03:02.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:02.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:02.997 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631353 s, 166 MB/s 2024-04-25T10:03:03.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.004 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596694 s, 176 MB/s 2024-04-25T10:03:03.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.012 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697082 s, 150 MB/s 2024-04-25T10:03:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636938 s, 165 MB/s 2024-04-25T10:03:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.026 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671318 s, 156 MB/s 2024-04-25T10:03:03.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:03.028 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:03.091 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.091 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.091 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619552 s, 169 MB/s 2024-04-25T10:03:03.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.098 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.098 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595711 s, 176 MB/s 2024-04-25T10:03:03.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.104 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582692 s, 180 MB/s 2024-04-25T10:03:03.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756171 s, 139 MB/s 2024-04-25T10:03:03.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.120 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065579 s, 160 MB/s 2024-04-25T10:03:03.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657673 s, 159 MB/s 2024-04-25T10:03:03.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.135 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647408 s, 162 MB/s 2024-04-25T10:03:03.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.142 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567755 s, 185 MB/s 2024-04-25T10:03:03.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.149 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633196 s, 166 MB/s 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764974 s, 137 MB/s 2024-04-25T10:03:03.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679642 s, 154 MB/s 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.173 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649942 s, 161 MB/s 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.179 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612819 s, 171 MB/s 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.186 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579001 s, 181 MB/s 2024-04-25T10:03:03.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.194 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641916 s, 163 MB/s 2024-04-25T10:03:03.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.200 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586173 s, 179 MB/s 2024-04-25T10:03:03.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.208 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647345 s, 162 MB/s 2024-04-25T10:03:03.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.215 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667025 s, 157 MB/s 2024-04-25T10:03:03.223 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.223 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.223 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730792 s, 143 MB/s 2024-04-25T10:03:03.231 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.231 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.231 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064288 s, 163 MB/s 2024-04-25T10:03:03.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.239 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737139 s, 142 MB/s 2024-04-25T10:03:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653738 s, 160 MB/s 2024-04-25T10:03:03.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630365 s, 166 MB/s 2024-04-25T10:03:03.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.260 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585214 s, 179 MB/s 2024-04-25T10:03:03.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656387 s, 160 MB/s 2024-04-25T10:03:03.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.276 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724217 s, 145 MB/s 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.284 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703625 s, 149 MB/s 2024-04-25T10:03:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.291 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655531 s, 160 MB/s 2024-04-25T10:03:03.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658553 s, 159 MB/s 2024-04-25T10:03:03.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.306 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648152 s, 162 MB/s 2024-04-25T10:03:03.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640457 s, 164 MB/s 2024-04-25T10:03:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.320 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581315 s, 180 MB/s 2024-04-25T10:03:03.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:03.321 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:03.321 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:03.321 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:03:03.321 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:03.321 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:03.321 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:03.322 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:03.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.385 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063219 s, 166 MB/s 2024-04-25T10:03:03.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659831 s, 159 MB/s 2024-04-25T10:03:03.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732024 s, 143 MB/s 2024-04-25T10:03:03.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.408 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638044 s, 164 MB/s 2024-04-25T10:03:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.416 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715704 s, 147 MB/s 2024-04-25T10:03:03.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.423 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640496 s, 164 MB/s 2024-04-25T10:03:03.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651536 s, 161 MB/s 2024-04-25T10:03:03.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.438 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638104 s, 164 MB/s 2024-04-25T10:03:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629711 s, 167 MB/s 2024-04-25T10:03:03.452 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.453 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662702 s, 158 MB/s 2024-04-25T10:03:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662321 s, 158 MB/s 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.468 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640367 s, 164 MB/s 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.475 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653792 s, 160 MB/s 2024-04-25T10:03:03.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.483 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720052 s, 146 MB/s 2024-04-25T10:03:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067339 s, 156 MB/s 2024-04-25T10:03:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668427 s, 157 MB/s 2024-04-25T10:03:03.504 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586356 s, 179 MB/s 2024-04-25T10:03:03.512 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702667 s, 149 MB/s 2024-04-25T10:03:03.520 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.520 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.520 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651337 s, 161 MB/s 2024-04-25T10:03:03.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.527 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626304 s, 167 MB/s 2024-04-25T10:03:03.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608809 s, 172 MB/s 2024-04-25T10:03:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569183 s, 184 MB/s 2024-04-25T10:03:03.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.548 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637869 s, 164 MB/s 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628606 s, 167 MB/s 2024-04-25T10:03:03.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.563 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622078 s, 169 MB/s 2024-04-25T10:03:03.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.570 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.570 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645411 s, 162 MB/s 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.577 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656415 s, 160 MB/s 2024-04-25T10:03:03.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.585 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.585 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072635 s, 144 MB/s 2024-04-25T10:03:03.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715088 s, 147 MB/s 2024-04-25T10:03:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.600 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626456 s, 167 MB/s 2024-04-25T10:03:03.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.608 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.608 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642077 s, 163 MB/s 2024-04-25T10:03:03.615 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.616 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715892 s, 146 MB/s 2024-04-25T10:03:03.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:03.617 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:03.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.682 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.682 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717032 s, 146 MB/s 2024-04-25T10:03:03.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.690 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743818 s, 141 MB/s 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063853 s, 164 MB/s 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059704 s, 176 MB/s 2024-04-25T10:03:03.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.711 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.711 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572003 s, 183 MB/s 2024-04-25T10:03:03.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.719 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.719 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709656 s, 148 MB/s 2024-04-25T10:03:03.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.726 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669971 s, 157 MB/s 2024-04-25T10:03:03.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665698 s, 158 MB/s 2024-04-25T10:03:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.741 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631303 s, 166 MB/s 2024-04-25T10:03:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.749 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716024 s, 146 MB/s 2024-04-25T10:03:03.756 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.757 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683923 s, 153 MB/s 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657902 s, 159 MB/s 2024-04-25T10:03:03.771 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.771 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.771 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629152 s, 167 MB/s 2024-04-25T10:03:03.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.778 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.778 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634636 s, 165 MB/s 2024-04-25T10:03:03.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660011 s, 159 MB/s 2024-04-25T10:03:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.793 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063501 s, 165 MB/s 2024-04-25T10:03:03.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629723 s, 167 MB/s 2024-04-25T10:03:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.808 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626221 s, 167 MB/s 2024-04-25T10:03:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649925 s, 161 MB/s 2024-04-25T10:03:03.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.822 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.822 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625939 s, 168 MB/s 2024-04-25T10:03:03.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.829 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634843 s, 165 MB/s 2024-04-25T10:03:03.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.837 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647718 s, 162 MB/s 2024-04-25T10:03:03.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.844 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642562 s, 163 MB/s 2024-04-25T10:03:03.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722045 s, 145 MB/s 2024-04-25T10:03:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653244 s, 161 MB/s 2024-04-25T10:03:03.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.867 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648898 s, 162 MB/s 2024-04-25T10:03:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733815 s, 143 MB/s 2024-04-25T10:03:03.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.882 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.882 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660003 s, 159 MB/s 2024-04-25T10:03:03.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592994 s, 177 MB/s 2024-04-25T10:03:03.896 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.896 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.896 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056797 s, 185 MB/s 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.903 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637186 s, 165 MB/s 2024-04-25T10:03:03.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.910 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591678 s, 177 MB/s 2024-04-25T10:03:03.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:03.911 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:03.911 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:03.911 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:03:03.911 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:03.911 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:03.911 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:03.912 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:03.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727142 s, 144 MB/s 2024-04-25T10:03:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670636 s, 156 MB/s 2024-04-25T10:03:03.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:03.992 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:03.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682799 s, 154 MB/s 2024-04-25T10:03:04.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728703 s, 144 MB/s 2024-04-25T10:03:04.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.007 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643488 s, 163 MB/s 2024-04-25T10:03:04.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.016 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.016 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735398 s, 143 MB/s 2024-04-25T10:03:04.024 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.024 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.024 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756703 s, 139 MB/s 2024-04-25T10:03:04.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.031 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563565 s, 186 MB/s 2024-04-25T10:03:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.038 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681387 s, 154 MB/s 2024-04-25T10:03:04.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.046 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672481 s, 156 MB/s 2024-04-25T10:03:04.053 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.053 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.053 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618453 s, 170 MB/s 2024-04-25T10:03:04.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.060 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.060 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636592 s, 165 MB/s 2024-04-25T10:03:04.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.067 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621346 s, 169 MB/s 2024-04-25T10:03:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.075 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654908 s, 160 MB/s 2024-04-25T10:03:04.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.082 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068225 s, 154 MB/s 2024-04-25T10:03:04.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674587 s, 155 MB/s 2024-04-25T10:03:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.097 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647711 s, 162 MB/s 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064952 s, 161 MB/s 2024-04-25T10:03:04.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.112 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.112 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641108 s, 164 MB/s 2024-04-25T10:03:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.120 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710685 s, 148 MB/s 2024-04-25T10:03:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.127 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635781 s, 165 MB/s 2024-04-25T10:03:04.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.135 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.135 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673851 s, 156 MB/s 2024-04-25T10:03:04.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705651 s, 149 MB/s 2024-04-25T10:03:04.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571835 s, 183 MB/s 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.157 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694692 s, 151 MB/s 2024-04-25T10:03:04.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675421 s, 155 MB/s 2024-04-25T10:03:04.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.173 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746188 s, 141 MB/s 2024-04-25T10:03:04.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666649 s, 157 MB/s 2024-04-25T10:03:04.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.187 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.187 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556898 s, 188 MB/s 2024-04-25T10:03:04.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.195 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693497 s, 151 MB/s 2024-04-25T10:03:04.202 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.202 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.202 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062744 s, 167 MB/s 2024-04-25T10:03:04.209 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.210 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.210 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639504 s, 164 MB/s 2024-04-25T10:03:04.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:04.211 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:04.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.274 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647421 s, 162 MB/s 2024-04-25T10:03:04.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.282 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631268 s, 166 MB/s 2024-04-25T10:03:04.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.288 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590324 s, 178 MB/s 2024-04-25T10:03:04.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.296 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.296 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062415 s, 168 MB/s 2024-04-25T10:03:04.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567322 s, 185 MB/s 2024-04-25T10:03:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.310 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626669 s, 167 MB/s 2024-04-25T10:03:04.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.318 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074947 s, 140 MB/s 2024-04-25T10:03:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.325 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063171 s, 166 MB/s 2024-04-25T10:03:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.332 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654608 s, 160 MB/s 2024-04-25T10:03:04.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558367 s, 188 MB/s 2024-04-25T10:03:04.347 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.347 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.347 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701946 s, 149 MB/s 2024-04-25T10:03:04.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.355 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727973 s, 144 MB/s 2024-04-25T10:03:04.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.362 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646472 s, 162 MB/s 2024-04-25T10:03:04.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.370 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.370 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663282 s, 158 MB/s 2024-04-25T10:03:04.377 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.378 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701221 s, 150 MB/s 2024-04-25T10:03:04.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.385 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648295 s, 162 MB/s 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657323 s, 160 MB/s 2024-04-25T10:03:04.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.400 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.400 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623555 s, 168 MB/s 2024-04-25T10:03:04.407 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.407 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.407 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637675 s, 164 MB/s 2024-04-25T10:03:04.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.415 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715386 s, 147 MB/s 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.423 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714846 s, 147 MB/s 2024-04-25T10:03:04.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686176 s, 153 MB/s 2024-04-25T10:03:04.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.438 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630624 s, 166 MB/s 2024-04-25T10:03:04.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624308 s, 168 MB/s 2024-04-25T10:03:04.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.453 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667275 s, 157 MB/s 2024-04-25T10:03:04.459 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.460 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.460 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597809 s, 175 MB/s 2024-04-25T10:03:04.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.468 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699146 s, 150 MB/s 2024-04-25T10:03:04.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726084 s, 144 MB/s 2024-04-25T10:03:04.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.483 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649937 s, 161 MB/s 2024-04-25T10:03:04.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677156 s, 155 MB/s 2024-04-25T10:03:04.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.497 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620568 s, 169 MB/s 2024-04-25T10:03:04.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714388 s, 147 MB/s 2024-04-25T10:03:04.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:04.507 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:04.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.572 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007135 s, 147 MB/s 2024-04-25T10:03:04.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.580 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663788 s, 158 MB/s 2024-04-25T10:03:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.587 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693128 s, 151 MB/s 2024-04-25T10:03:04.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738121 s, 142 MB/s 2024-04-25T10:03:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647915 s, 162 MB/s 2024-04-25T10:03:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581312 s, 180 MB/s 2024-04-25T10:03:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639811 s, 164 MB/s 2024-04-25T10:03:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627618 s, 167 MB/s 2024-04-25T10:03:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.631 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617527 s, 170 MB/s 2024-04-25T10:03:04.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672243 s, 156 MB/s 2024-04-25T10:03:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063745 s, 164 MB/s 2024-04-25T10:03:04.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.654 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718972 s, 146 MB/s 2024-04-25T10:03:04.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.662 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688627 s, 152 MB/s 2024-04-25T10:03:04.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627563 s, 167 MB/s 2024-04-25T10:03:04.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718217 s, 146 MB/s 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645365 s, 162 MB/s 2024-04-25T10:03:04.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.692 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644442 s, 163 MB/s 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658477 s, 159 MB/s 2024-04-25T10:03:04.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632036 s, 166 MB/s 2024-04-25T10:03:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615413 s, 170 MB/s 2024-04-25T10:03:04.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.720 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.720 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608798 s, 172 MB/s 2024-04-25T10:03:04.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.727 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.727 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600776 s, 175 MB/s 2024-04-25T10:03:04.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.735 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712014 s, 147 MB/s 2024-04-25T10:03:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.743 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725259 s, 145 MB/s 2024-04-25T10:03:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722211 s, 145 MB/s 2024-04-25T10:03:04.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.758 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596266 s, 176 MB/s 2024-04-25T10:03:04.765 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.765 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.765 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618782 s, 169 MB/s 2024-04-25T10:03:04.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.773 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664323 s, 158 MB/s 2024-04-25T10:03:04.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634419 s, 165 MB/s 2024-04-25T10:03:04.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.787 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.787 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643087 s, 163 MB/s 2024-04-25T10:03:04.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.795 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732957 s, 143 MB/s 2024-04-25T10:03:04.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.803 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.803 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651634 s, 161 MB/s 2024-04-25T10:03:04.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:04.804 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:04.804 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:04.804 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:03:04.804 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:04.804 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:04.804 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:04.805 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:04.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.868 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.868 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571982 s, 183 MB/s 2024-04-25T10:03:04.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743473 s, 141 MB/s 2024-04-25T10:03:04.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.884 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706696 s, 148 MB/s 2024-04-25T10:03:04.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.890 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00543236 s, 193 MB/s 2024-04-25T10:03:04.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.899 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743382 s, 141 MB/s 2024-04-25T10:03:04.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.905 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.905 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573045 s, 183 MB/s 2024-04-25T10:03:04.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732562 s, 143 MB/s 2024-04-25T10:03:04.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.921 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.921 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712787 s, 147 MB/s 2024-04-25T10:03:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.928 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058595 s, 179 MB/s 2024-04-25T10:03:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.935 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634664 s, 165 MB/s 2024-04-25T10:03:04.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.944 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749855 s, 140 MB/s 2024-04-25T10:03:04.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.951 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616319 s, 170 MB/s 2024-04-25T10:03:04.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.958 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.958 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560855 s, 187 MB/s 2024-04-25T10:03:04.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.965 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.965 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629368 s, 167 MB/s 2024-04-25T10:03:04.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.972 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645443 s, 162 MB/s 2024-04-25T10:03:04.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.980 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719153 s, 146 MB/s 2024-04-25T10:03:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724862 s, 145 MB/s 2024-04-25T10:03:04.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:04.996 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:04.996 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646199 s, 162 MB/s 2024-04-25T10:03:05.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.003 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623727 s, 168 MB/s 2024-04-25T10:03:05.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.011 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763039 s, 137 MB/s 2024-04-25T10:03:05.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.018 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623999 s, 168 MB/s 2024-04-25T10:03:05.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715197 s, 147 MB/s 2024-04-25T10:03:05.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632748 s, 166 MB/s 2024-04-25T10:03:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667872 s, 157 MB/s 2024-04-25T10:03:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671702 s, 156 MB/s 2024-04-25T10:03:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664935 s, 158 MB/s 2024-04-25T10:03:05.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.064 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653337 s, 160 MB/s 2024-04-25T10:03:05.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.070 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.070 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577888 s, 181 MB/s 2024-04-25T10:03:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.078 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640576 s, 164 MB/s 2024-04-25T10:03:05.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682955 s, 154 MB/s 2024-04-25T10:03:05.093 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.094 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.094 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730947 s, 143 MB/s 2024-04-25T10:03:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.101 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626545 s, 167 MB/s 2024-04-25T10:03:05.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:05.102 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:05.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645717 s, 162 MB/s 2024-04-25T10:03:05.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.175 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713643 s, 147 MB/s 2024-04-25T10:03:05.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.182 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630387 s, 166 MB/s 2024-04-25T10:03:05.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.189 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632034 s, 166 MB/s 2024-04-25T10:03:05.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.198 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777914 s, 135 MB/s 2024-04-25T10:03:05.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663425 s, 158 MB/s 2024-04-25T10:03:05.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.213 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627527 s, 167 MB/s 2024-04-25T10:03:05.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.220 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.220 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610162 s, 172 MB/s 2024-04-25T10:03:05.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.227 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661665 s, 158 MB/s 2024-04-25T10:03:05.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715574 s, 147 MB/s 2024-04-25T10:03:05.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.243 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.243 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721758 s, 145 MB/s 2024-04-25T10:03:05.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.251 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619673 s, 169 MB/s 2024-04-25T10:03:05.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.259 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071883 s, 146 MB/s 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.266 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610766 s, 172 MB/s 2024-04-25T10:03:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.274 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715571 s, 147 MB/s 2024-04-25T10:03:05.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.281 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643007 s, 163 MB/s 2024-04-25T10:03:05.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.288 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647676 s, 162 MB/s 2024-04-25T10:03:05.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588938 s, 178 MB/s 2024-04-25T10:03:05.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.303 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.303 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071113 s, 147 MB/s 2024-04-25T10:03:05.310 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.310 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.310 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629618 s, 167 MB/s 2024-04-25T10:03:05.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.318 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.318 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704007 s, 149 MB/s 2024-04-25T10:03:05.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.326 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.326 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066393 s, 158 MB/s 2024-04-25T10:03:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.333 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067192 s, 156 MB/s 2024-04-25T10:03:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.341 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667306 s, 157 MB/s 2024-04-25T10:03:05.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.348 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616073 s, 170 MB/s 2024-04-25T10:03:05.356 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.356 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.356 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735645 s, 143 MB/s 2024-04-25T10:03:05.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630541 s, 166 MB/s 2024-04-25T10:03:05.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.371 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658378 s, 159 MB/s 2024-04-25T10:03:05.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.378 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593127 s, 177 MB/s 2024-04-25T10:03:05.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.386 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734557 s, 143 MB/s 2024-04-25T10:03:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709279 s, 148 MB/s 2024-04-25T10:03:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624119 s, 168 MB/s 2024-04-25T10:03:05.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:05.402 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:03:05.402 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:03:05.402 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:03:05.402 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:03:05.402 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:03:05.402 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:03:05.403 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:03:05.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.467 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.467 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705182 s, 149 MB/s 2024-04-25T10:03:05.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.474 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.474 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619359 s, 169 MB/s 2024-04-25T10:03:05.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.481 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.482 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635442 s, 165 MB/s 2024-04-25T10:03:05.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.488 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.489 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621422 s, 169 MB/s 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584562 s, 179 MB/s 2024-04-25T10:03:05.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072574 s, 144 MB/s 2024-04-25T10:03:05.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.511 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.511 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665219 s, 158 MB/s 2024-04-25T10:03:05.519 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.519 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.519 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719452 s, 146 MB/s 2024-04-25T10:03:05.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.526 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.526 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582736 s, 180 MB/s 2024-04-25T10:03:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.532 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571561 s, 183 MB/s 2024-04-25T10:03:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.540 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067109 s, 156 MB/s 2024-04-25T10:03:05.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.548 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648845 s, 162 MB/s 2024-04-25T10:03:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.554 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605474 s, 173 MB/s 2024-04-25T10:03:05.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.561 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610973 s, 172 MB/s 2024-04-25T10:03:05.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.568 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558791 s, 188 MB/s 2024-04-25T10:03:05.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.576 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.576 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071128 s, 147 MB/s 2024-04-25T10:03:05.583 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.583 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.583 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607192 s, 173 MB/s 2024-04-25T10:03:05.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.591 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702846 s, 149 MB/s 2024-04-25T10:03:05.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.597 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.597 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563469 s, 186 MB/s 2024-04-25T10:03:05.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.604 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568069 s, 185 MB/s 2024-04-25T10:03:05.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.612 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.612 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662621 s, 158 MB/s 2024-04-25T10:03:05.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.619 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659116 s, 159 MB/s 2024-04-25T10:03:05.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.626 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576526 s, 182 MB/s 2024-04-25T10:03:05.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634317 s, 165 MB/s 2024-04-25T10:03:05.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643596 s, 163 MB/s 2024-04-25T10:03:05.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.649 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719225 s, 146 MB/s 2024-04-25T10:03:05.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.655 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615264 s, 170 MB/s 2024-04-25T10:03:05.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064404 s, 163 MB/s 2024-04-25T10:03:05.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.671 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.671 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732384 s, 143 MB/s 2024-04-25T10:03:05.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.678 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.678 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633066 s, 166 MB/s 2024-04-25T10:03:05.685 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.686 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656224 s, 160 MB/s 2024-04-25T10:03:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:03:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:03:05.693 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641681 s, 163 MB/s 2024-04-25T10:03:05.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:05.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:03:05.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.831+0000 7fa407493640 1 Processor -- start 2024-04-25T10:03:05.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.831+0000 7fa407493640 1 -- start start 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa407493640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa40015a9f0 0x7fa4000c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa407493640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa4000ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa407493640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4000c6940 con 0x7fa4000cd520 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa407493640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4000b0020 con 0x7fa40015a9f0 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa407493640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000cb5a0 con 0x7fa4000cdbf0 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa40522f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa40015a9f0 0x7fa4000c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa405a30640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa4000cd520 0x7fa4000ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33690/0 (socket says 172.21.15.148:33690) 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa405a30640 1 -- 172.21.15.148:0/4147789405 learned_addr learned my addr 172.21.15.148:0/4147789405 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.832+0000 7fa404a2e640 1 --2- 172.21.15.148:0/4147789405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa4000ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.833+0000 7fa40522f640 1 -- 172.21.15.148:0/4147789405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 msgr2=0x7fa4000ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.833+0000 7fa40522f640 1 --2- 172.21.15.148:0/4147789405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa4000ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.833+0000 7fa40522f640 1 -- 172.21.15.148:0/4147789405 >> v1:172.21.15.148:6789/0 conn(0x7fa4000cd520 legacy=0x7fa4000ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.833+0000 7fa3f7fff640 1 -- 172.21.15.148:0/4147789405 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130670483 0 0) 0x7fa3f8001490 con 0x7fa4000cd520 2024-04-25T10:03:05.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.833+0000 7fa40522f640 1 -- 172.21.15.148:0/4147789405 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40015add0 con 0x7fa40015a9f0 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.833+0000 7fa40522f640 1 --2- 172.21.15.148:0/4147789405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40015a9f0 0x7fa4000c6010 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0097a0 tx=0x7fa3fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b1c89b2d0d9d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa3f7fff640 1 -- 172.21.15.148:0/4147789405 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc037070 con 0x7fa40015a9f0 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa3f7fff640 1 -- 172.21.15.148:0/4147789405 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc032440 con 0x7fa40015a9f0 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa3f7fff640 1 -- 172.21.15.148:0/4147789405 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc032810 con 0x7fa40015a9f0 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 -- 172.21.15.148:0/4147789405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40015a9f0 msgr2=0x7fa4000c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 --2- 172.21.15.148:0/4147789405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40015a9f0 0x7fa4000c6010 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0097a0 tx=0x7fa3fc02a180 comp rx=0 tx=0).stop 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 -- 172.21.15.148:0/4147789405 shutdown_connections 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 --2- 172.21.15.148:0/4147789405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa4000ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 --2- 172.21.15.148:0/4147789405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa40015a9f0 0x7fa4000c6010 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 -- 172.21.15.148:0/4147789405 >> 172.21.15.148:0/4147789405 conn(0x7fa4000b2390 msgr2=0x7fa4000b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 -- 172.21.15.148:0/4147789405 shutdown_connections 2024-04-25T10:03:05.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.834+0000 7fa407493640 1 -- 172.21.15.148:0/4147789405 wait complete. 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.836+0000 7fa407493640 1 Processor -- start 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.836+0000 7fa407493640 1 -- start start 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa407493640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 0x7fa4000aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa407493640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa40014ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa407493640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40015a9f0 0x7fa4000c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa407493640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000cb230 con 0x7fa4000cd520 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa407493640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000caf50 con 0x7fa40015a9f0 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa407493640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000ce070 con 0x7fa4000cdbf0 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 0x7fa4000aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 0x7fa4000aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38698/0 (socket says 172.21.15.148:38698) 2024-04-25T10:03:05.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 -- 172.21.15.148:0/3736082269 learned_addr learned my addr 172.21.15.148:0/3736082269 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa40014ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa405a30640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40015a9f0 0x7fa4000c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 -- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 msgr2=0x7fa40014ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa40014ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 -- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40015a9f0 msgr2=0x7fa4000c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40015a9f0 0x7fa4000c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 -- 172.21.15.148:0/3736082269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4000b5010 con 0x7fa4000cd520 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.837+0000 7fa40522f640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 0x7fa4000aeb20 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc035040 tx=0x7fa3fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.838+0000 7fa3f5ffb640 1 -- 172.21.15.148:0/3736082269 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc037040 con 0x7fa4000cd520 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.838+0000 7fa3f5ffb640 1 -- 172.21.15.148:0/3736082269 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc030500 con 0x7fa4000cd520 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.838+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4000c9020 con 0x7fa4000cd520 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.838+0000 7fa3f5ffb640 1 -- 172.21.15.148:0/3736082269 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc030910 con 0x7fa4000cd520 2024-04-25T10:03:05.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.838+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4000c95e0 con 0x7fa4000cd520 2024-04-25T10:03:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.839+0000 7fa3f5ffb640 1 -- 172.21.15.148:0/3736082269 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3fc044020 con 0x7fa4000cd520 2024-04-25T10:03:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.840+0000 7fa3f5ffb640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e406b4d0 0x7fa3e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.840+0000 7fa3f5ffb640 1 -- 172.21.15.148:0/3736082269 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3fc0aa510 con 0x7fa4000cd520 2024-04-25T10:03:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.840+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e406b4d0 0x7fa3e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.840+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e406b4d0 0x7fa3e406d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa4000aff60 tx=0x7fa3f0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.841+0000 7fa40a192740 1 Processor -- start 2024-04-25T10:03:05.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.841+0000 7fa40a192740 1 -- start start 2024-04-25T10:03:05.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175b9ff0 0x55a8175ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40a192740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 0x55a817591260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40a192740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8174f6000 con 0x55a8175d9350 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81749ef20 con 0x55a8175943a0 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8174f20a0 con 0x55a8175b9ff0 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 0x55a817591260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa40522f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175b9ff0 0x55a8175ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 0x55a817591260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38712/0 (socket says 172.21.15.148:38712) 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 -- 172.21.15.148:0/3033210575 learned_addr learned my addr 172.21.15.148:0/3033210575 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 -- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175b9ff0 msgr2=0x55a8175ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175b9ff0 0x55a8175ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 -- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 msgr2=0x55a8175d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 -- 172.21.15.148:0/3033210575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8172d6ba0 con 0x55a8175d9350 2024-04-25T10:03:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.842+0000 7fa405a30640 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 0x55a817591260 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001bb0 tx=0x7fa3f8002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a860ed1d455d1b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa3eaffd640 1 -- 172.21.15.148:0/3033210575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80077f0 con 0x55a8175d9350 2024-04-25T10:03:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa3eaffd640 1 -- 172.21.15.148:0/3033210575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8018070 con 0x55a8175d9350 2024-04-25T10:03:05.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa3eaffd640 1 -- 172.21.15.148:0/3033210575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80133f0 con 0x55a8175d9350 2024-04-25T10:03:05.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 -- 172.21.15.148:0/3033210575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 msgr2=0x55a817591260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 0x55a817591260 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001bb0 tx=0x7fa3f8002070 comp rx=0 tx=0).stop 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 -- 172.21.15.148:0/3033210575 shutdown_connections 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175d9350 0x55a817591260 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 --2- 172.21.15.148:0/3033210575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175b9ff0 0x55a8175ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 -- 172.21.15.148:0/3033210575 >> 172.21.15.148:0/3033210575 conn(0x55a81755a050 msgr2=0x55a8175d7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 -- 172.21.15.148:0/3033210575 shutdown_connections 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.843+0000 7fa40a192740 1 -- 172.21.15.148:0/3033210575 wait complete. 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.844+0000 7fa40a192740 1 -- start start 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.844+0000 7fa40a192740 1 Processor -- start 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a817591b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a817592050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81758e800 con 0x55a8175b9ff0 2024-04-25T10:03:05.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a817494610 con 0x55a8175943a0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a817593330 con 0x55a8175a71e0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40a192740 10 client.0 _ll_register_callbacks cb 0x55a8174acf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a817591b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a817591b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38716/0 (socket says 172.21.15.148:38716) 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 -- 172.21.15.148:0/3149484144 learned_addr learned my addr 172.21.15.148:0/3149484144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa405a30640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a817592050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa40522f640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 -- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 msgr2=0x55a817592050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a817592050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 -- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 msgr2=0x55a8175b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.845+0000 7fa404a2e640 1 -- 172.21.15.148:0/3149484144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8171ca490 con 0x55a8175b9ff0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a817591b10 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa400157640 tx=0x7fa3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17ad4c228648fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa3dffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:05.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa3dffff640 1 -- 172.21.15.148:0/3149484144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0027070 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa3dffff640 1 -- 172.21.15.148:0/3149484144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3f0014440 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3f8018070 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa3dffff640 1 -- 172.21.15.148:0/3149484144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f0032050 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 10 client.9522 fetch_fsmap learned FSMap version 74 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55a817337330 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa3dffff640 1 -- 172.21.15.148:0/3149484144 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8018070 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa3dffff640 1 -- 172.21.15.148:0/3149484144 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa3f0035020 con 0x55a8175b9ff0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 10 client.9522 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 10 client.9522 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 20 client.9522 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175af6d0 0x55a8175afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 4 client.9522 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a817503540 con 0x55a8175af6d0 2024-04-25T10:03:05.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.846+0000 7fa40522f640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175af6d0 0x55a8175afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa3dffff640 10 client.9522 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40522f640 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175af6d0 0x55a8175afad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e3b4c40f53ea54 server_cookie=7f41175a17c5da70 in_seq=0 out_seq=0 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa3dffff640 1 -- 172.21.15.148:0/3149484144 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a817503540 con 0x55a8175af6d0 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa3dffff640 10 client.9522 handle_command_reply: tid=0 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 client.9522 shutdown 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 msgr2=0x55a817591b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a817591b10 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa400157640 tx=0x7fa3f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 shutdown_connections 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175af6d0 0x55a8175afad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a817592050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a817591b10 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa3dffff640 0 client.9522 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 --2- 172.21.15.148:0/3149484144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a8175b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.847+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 >> 172.21.15.148:0/3149484144 conn(0x55a81755a050 msgr2=0x55a8175db200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.848+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 shutdown_connections 2024-04-25T10:03:05.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.848+0000 7fa40a192740 1 -- 172.21.15.148:0/3149484144 wait complete. 2024-04-25T10:03:05.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.848+0000 7fa40a192740 20 client.9522 trim_cache size 0 max 16384 2024-04-25T10:03:05.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 Processor -- start 2024-04-25T10:03:05.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 -- start start 2024-04-25T10:03:05.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a817593050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8175b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a8175cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4001fc190 con 0x55a8175b9ff0 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40006b7e0 con 0x55a8175943a0 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40006abe0 con 0x55a8175a71e0 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa405a30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a8175cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa405a30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a8175cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38720/0 (socket says 172.21.15.148:38720) 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.863+0000 7fa405a30640 1 -- 172.21.15.148:0/3169056929 learned_addr learned my addr 172.21.15.148:0/3169056929 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa405a30640 1 -- 172.21.15.148:0/3169056929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 msgr2=0x55a8175b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa405a30640 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8175b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa405a30640 1 -- 172.21.15.148:0/3169056929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 msgr2=0x55a817593050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa405a30640 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a817593050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa405a30640 1 -- 172.21.15.148:0/3169056929 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8171a3610 con 0x55a8175b9ff0 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa405a30640 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a8175cda40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001b80 tx=0x7fa3f8009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9b241e1e56d9f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa3ea7fc640 1 -- 172.21.15.148:0/3169056929 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8007930 con 0x55a8175b9ff0 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa3ea7fc640 1 -- 172.21.15.148:0/3169056929 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8018030 con 0x55a8175b9ff0 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa3ea7fc640 1 -- 172.21.15.148:0/3169056929 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8028460 con 0x55a8175b9ff0 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 -- 172.21.15.148:0/3169056929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 msgr2=0x55a8175cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a8175cda40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001b80 tx=0x7fa3f8009030 comp rx=0 tx=0).stop 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 -- 172.21.15.148:0/3169056929 shutdown_connections 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175b9ff0 0x55a8175cda40 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8175b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 --2- 172.21.15.148:0/3169056929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175943a0 0x55a817593050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 -- 172.21.15.148:0/3169056929 >> 172.21.15.148:0/3169056929 conn(0x55a81755a050 msgr2=0x55a8175ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.864+0000 7fa40a192740 1 -- 172.21.15.148:0/3169056929 shutdown_connections 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.865+0000 7fa40a192740 1 -- 172.21.15.148:0/3169056929 wait complete. 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.865+0000 7fa40a192740 1 -- start start 2024-04-25T10:03:05.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.865+0000 7fa40a192740 1 Processor -- start 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 0x55a8175fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8176469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175b9ff0 0x55a8175adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8175c0d20 con 0x55a8175943a0 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40006b1e0 con 0x55a8175b9ff0 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8175ce710 con 0x55a8175a71e0 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40a192740 10 client.0 _ll_register_callbacks cb 0x7fa3f8004ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 0x55a8175fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 0x55a8175fb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38730/0 (socket says 172.21.15.148:38730) 2024-04-25T10:03:05.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 -- 172.21.15.148:0/3579685989 learned_addr learned my addr 172.21.15.148:0/3579685989 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa405a30640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175b9ff0 0x55a8175adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8176469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 -- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 msgr2=0x55a8176469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8176469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 -- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175b9ff0 msgr2=0x55a8175adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175b9ff0 0x55a8175adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 -- 172.21.15.148:0/3579685989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8175ac5c0 con 0x55a8175943a0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa40522f640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 0x55a8175fb5f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0a8850 tx=0x7fa3fc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311031fef844a656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.866+0000 7fa3dffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa3dffff640 1 -- 172.21.15.148:0/3579685989 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc075220 con 0x55a8175943a0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa3dffff640 1 -- 172.21.15.148:0/3579685989 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3fc080d80 con 0x55a8175943a0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3f8018030 con 0x55a8175943a0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa3dffff640 1 -- 172.21.15.148:0/3579685989 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3fc0753c0 con 0x55a8175943a0 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 10 client.9528 fetch_fsmap learned FSMap version 74 2024-04-25T10:03:05.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55a81764b2e0 con 0x55a8175943a0 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa3dffff640 1 -- 172.21.15.148:0/3579685989 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8018030 con 0x55a8175943a0 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa3dffff640 1 -- 172.21.15.148:0/3579685989 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa3fc03a070 con 0x55a8175943a0 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 10 client.9528 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 10 client.9528 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 20 client.9528 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175ae1e0 0x55a8175f5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 4 client.9528 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55a8174fd890 con 0x55a8175ae1e0 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175ae1e0 0x55a8175f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa3dffff640 10 client.9528 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:05.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.867+0000 7fa404a2e640 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175ae1e0 0x55a8175f5540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b4ebe1f1042f97 server_cookie=e2aee017033c464e in_seq=0 out_seq=0 2024-04-25T10:03:05.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa3dffff640 1 -- 172.21.15.148:0/3579685989 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a8174fd890 con 0x55a8175ae1e0 2024-04-25T10:03:05.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa3dffff640 10 client.9528 handle_command_reply: tid=0 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 client.9528 shutdown 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 msgr2=0x55a8175fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 0x55a8175fb5f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0a8850 tx=0x7fa3fc009190 comp rx=0 tx=0).stop 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 shutdown_connections 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55a8175ae1e0 0x55a8175f5540 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8175b9ff0 0x55a8175adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a8175a71e0 0x55a8176469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa3dffff640 0 client.9528 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 --2- 172.21.15.148:0/3579685989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a8175943a0 0x55a8175fb5f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.871+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 >> 172.21.15.148:0/3579685989 conn(0x55a81755a050 msgr2=0x55a8175cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 shutdown_connections 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa40a192740 1 -- 172.21.15.148:0/3579685989 wait complete. 2024-04-25T10:03:05.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa40a192740 20 client.9528 trim_cache size 0 max 16384 2024-04-25T10:03:05.879 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:03:05.879 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:03:05.879 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "7baef3be-4ae9-493f-a68c-921c1cd79b0d", 2024-04-25T10:03:05.879 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:03:05.879 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e406b4d0 msgr2=0x7fa3e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e406b4d0 0x7fa3e406d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa4000aff60 tx=0x7fa3f0009190 comp rx=0 tx=0).stop 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 msgr2=0x7fa4000aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 0x7fa4000aeb20 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc035040 tx=0x7fa3fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 shutdown_connections 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e406b4d0 0x7fa3e406d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40015a9f0 0x7fa4000c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4000cdbf0 0x7fa40014ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 --2- 172.21.15.148:0/3736082269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4000cd520 0x7fa4000aeb20 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 >> 172.21.15.148:0/3736082269 conn(0x7fa4000b2390 msgr2=0x7fa400159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.872+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 shutdown_connections 2024-04-25T10:03:05.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:05.873+0000 7fa407493640 1 -- 172.21.15.148:0/3736082269 wait complete. 2024-04-25T10:03:05.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:05.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.009+0000 7f9112e3f640 1 Processor -- start 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.009+0000 7f9112e3f640 1 -- start start 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f9112e3f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f910c0ced80 0x7f910c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f9112e3f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c15e000 con 0x7f910c161710 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f9112e3f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c0d1de0 con 0x7f910c15dc20 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f9112e3f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c161c60 con 0x7f910c0ced80 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f910bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f910c0ced80 0x7f910c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f91113dc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f910c161710 0x7f910c0d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33706/0 (socket says 172.21.15.148:33706) 2024-04-25T10:03:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f91113dc640 1 -- 172.21.15.148:0/1758537158 learned_addr learned my addr 172.21.15.148:0/1758537158 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f910bfff640 1 -- 172.21.15.148:0/1758537158 >> v1:172.21.15.188:6789/0 conn(0x7f910c15dc20 legacy=0x7f910c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f910bfff640 1 -- 172.21.15.148:0/1758537158 >> v1:172.21.15.148:6789/0 conn(0x7f910c161710 legacy=0x7f910c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.010+0000 7f910bfff640 1 -- 172.21.15.148:0/1758537158 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c161f40 con 0x7f910c0ced80 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.011+0000 7f910bfff640 1 --2- 172.21.15.148:0/1758537158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c0d22e0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f91000097a0 tx=0x7f910002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08ff28652ba02be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.011+0000 7f910b7fe640 1 -- 172.21.15.148:0/1758537158 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100037070 con 0x7f910c0ced80 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.011+0000 7f910b7fe640 1 -- 172.21.15.148:0/1758537158 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100032440 con 0x7f910c0ced80 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.011+0000 7f910b7fe640 1 -- 172.21.15.148:0/1758537158 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100032810 con 0x7f910c0ced80 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 -- 172.21.15.148:0/1758537158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 msgr2=0x7f910c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1758537158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c0d22e0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f91000097a0 tx=0x7f910002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 -- 172.21.15.148:0/1758537158 shutdown_connections 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1758537158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c0d22e0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 -- 172.21.15.148:0/1758537158 >> 172.21.15.148:0/1758537158 conn(0x7f910c0d0420 msgr2=0x7f910c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 -- 172.21.15.148:0/1758537158 shutdown_connections 2024-04-25T10:03:06.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.012+0000 7f9112e3f640 1 -- 172.21.15.148:0/1758537158 wait complete. 2024-04-25T10:03:06.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.013+0000 7f9112e3f640 1 Processor -- start 2024-04-25T10:03:06.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.013+0000 7f9112e3f640 1 -- start start 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9112e3f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9112e3f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f910c15dc20 0x7f910c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9112e3f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f910c161710 0x7f910c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9112e3f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0b5e50 con 0x7f910c0ced80 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9112e3f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c162500 con 0x7f910c161710 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9112e3f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0b5fd0 con 0x7f910c15dc20 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c15a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38752/0 (socket says 172.21.15.148:38752) 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 -- 172.21.15.148:0/1134833925 learned_addr learned my addr 172.21.15.148:0/1134833925 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f91113dc640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f910c161710 0x7f910c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f910bfff640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f910c15dc20 0x7f910c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 -- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f910c15dc20 msgr2=0x7f910c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f910c15dc20 0x7f910c0c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 -- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f910c161710 msgr2=0x7f910c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f910c161710 0x7f910c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f9110bdb640 1 -- 172.21.15.148:0/1134833925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c1e7760 con 0x7f910c0ced80 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.014+0000 7f91113dc640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f910c161710 0x7f910c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.015+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c15a2a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c430 tx=0x7f90fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.015+0000 7f91097fa640 1 -- 172.21.15.148:0/1134833925 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc011790 con 0x7f910c0ced80 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.015+0000 7f91097fa640 1 -- 172.21.15.148:0/1134833925 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc017070 con 0x7f910c0ced80 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.015+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c1e7a40 con 0x7f910c0ced80 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.015+0000 7f91097fa640 1 -- 172.21.15.148:0/1134833925 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc011bb0 con 0x7f910c0ced80 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.015+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c1e8000 con 0x7f910c0ced80 2024-04-25T10:03:06.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.017+0000 7f91097fa640 1 -- 172.21.15.148:0/1134833925 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90fc01c460 con 0x7f910c0ced80 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.017+0000 7f91097fa640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90f806b4d0 0x7f90f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.017+0000 7f91097fa640 1 -- 172.21.15.148:0/1134833925 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90fc08a280 con 0x7f910c0ced80 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.017+0000 7f910bfff640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90f806b4d0 0x7f90f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.018+0000 7f910bfff640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90f806b4d0 0x7f90f806d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9100034000 tx=0x7f910002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.018+0000 7f9115c1d740 1 Processor -- start 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.018+0000 7f9115c1d740 1 -- start start 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf656ca0 0x55eacf658e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf657080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 0x55eacf681be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9115c1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacf5b9b90 con 0x55eacf681760 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacf562e40 con 0x55eacf693370 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eacf5b5c30 con 0x55eacf656ca0 2024-04-25T10:03:06.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 0x55eacf681be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 0x55eacf681be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38764/0 (socket says 172.21.15.148:38764) 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf656ca0 0x55eacf658e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 -- 172.21.15.148:0/1998217252 learned_addr learned my addr 172.21.15.148:0/1998217252 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f910bfff640 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf657080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 -- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf656ca0 msgr2=0x55eacf658e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf656ca0 0x55eacf658e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 -- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 msgr2=0x55eacf657080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf657080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 -- 172.21.15.148:0/1998217252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eacf3987d0 con 0x55eacf681760 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f91113dc640 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 0x55eacf681be0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9104002150 tx=0x7f910400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f9ff161dcfbb5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f90f27fc640 1 -- 172.21.15.148:0/1998217252 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104018070 con 0x55eacf681760 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f90f27fc640 1 -- 172.21.15.148:0/1998217252 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104013670 con 0x55eacf681760 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.019+0000 7f90f27fc640 1 -- 172.21.15.148:0/1998217252 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104013a40 con 0x55eacf681760 2024-04-25T10:03:06.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 -- 172.21.15.148:0/1998217252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 msgr2=0x55eacf681be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 0x55eacf681be0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9104002150 tx=0x7f910400d520 comp rx=0 tx=0).stop 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 -- 172.21.15.148:0/1998217252 shutdown_connections 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf681760 0x55eacf681be0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf657080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1998217252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf656ca0 0x55eacf658e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 -- 172.21.15.148:0/1998217252 >> 172.21.15.148:0/1998217252 conn(0x55eacf61e870 msgr2=0x55eacf69c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 -- 172.21.15.148:0/1998217252 shutdown_connections 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.020+0000 7f9115c1d740 1 -- 172.21.15.148:0/1998217252 wait complete. 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.021+0000 7f9115c1d740 1 -- start start 2024-04-25T10:03:06.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.021+0000 7f9115c1d740 1 Processor -- start 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.021+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf695c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.021+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf6940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf6ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacf5763e0 con 0x55eacf656ca0 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacf67f2b0 con 0x55eacf681760 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eacf696170 con 0x55eacf693370 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 10 client.0 _ll_register_callbacks cb 0x55eacf575db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf695c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf695c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38776/0 (socket says 172.21.15.148:38776) 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 -- 172.21.15.148:0/1371458324 learned_addr learned my addr 172.21.15.148:0/1371458324 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f91113dc640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf6ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f910bfff640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf6940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 -- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 msgr2=0x55eacf6ac530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf6ac530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 -- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 msgr2=0x55eacf6940d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf6940d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 -- 172.21.15.148:0/1371458324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eacf28e0f0 con 0x55eacf656ca0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf695c30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f90fc058570 tx=0x7f90fc0586f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f9689dd8a486b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f90eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f90eb7fe640 1 -- 172.21.15.148:0/1371458324 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc086060 con 0x55eacf656ca0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f90eb7fe640 1 -- 172.21.15.148:0/1371458324 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90fc054ee0 con 0x55eacf656ca0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9104013670 con 0x55eacf656ca0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f90eb7fe640 1 -- 172.21.15.148:0/1371458324 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90fc070070 con 0x55eacf656ca0 2024-04-25T10:03:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 10 client.9543 fetch_fsmap learned FSMap version 74 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55eacf3f8f60 con 0x55eacf656ca0 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.022+0000 7f90eb7fe640 1 -- 172.21.15.148:0/1371458324 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104013670 con 0x55eacf656ca0 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f90eb7fe640 1 -- 172.21.15.148:0/1371458324 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f90fc070210 con 0x55eacf656ca0 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 10 client.9543 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 10 client.9543 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 20 client.9543 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf6aca70 0x55eacf696910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 4 client.9543 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eacf5c70d0 con 0x55eacf6aca70 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f910bfff640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf6aca70 0x55eacf696910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f90eb7fe640 10 client.9543 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f910bfff640 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf6aca70 0x55eacf696910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f38eb7326f298d server_cookie=9a54be3e79db636a in_seq=0 out_seq=0 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f90eb7fe640 1 -- 172.21.15.148:0/1371458324 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eacf5c70d0 con 0x55eacf6aca70 2024-04-25T10:03:06.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f90eb7fe640 10 client.9543 handle_command_reply: tid=0 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.023+0000 7f9115c1d740 1 client.9543 shutdown 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 msgr2=0x55eacf695c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf695c30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f90fc058570 tx=0x7f90fc0586f0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 shutdown_connections 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf6aca70 0x55eacf696910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf6ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf6940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1371458324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf695c30 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f90eb7fe640 0 client.9543 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 >> 172.21.15.148:0/1371458324 conn(0x55eacf61e870 msgr2=0x55eacf67e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 shutdown_connections 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 1 -- 172.21.15.148:0/1371458324 wait complete. 2024-04-25T10:03:06.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.024+0000 7f9115c1d740 20 client.9543 trim_cache size 0 max 16384 2024-04-25T10:03:06.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.038+0000 7f9115c1d740 1 Processor -- start 2024-04-25T10:03:06.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.038+0000 7f9115c1d740 1 -- start start 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf692980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf681760 0x55eacf6bf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf69cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9115c1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c200450 con 0x55eacf656ca0 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c1ff600 con 0x55eacf693370 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c8380 con 0x55eacf681760 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf692980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf692980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38792/0 (socket says 172.21.15.148:38792) 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f9110bdb640 1 -- 172.21.15.148:0/1108669165 learned_addr learned my addr 172.21.15.148:0/1108669165 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f910bfff640 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf681760 0x55eacf6bf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.039+0000 7f91113dc640 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf69cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9110bdb640 1 -- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf681760 msgr2=0x55eacf6bf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf681760 0x55eacf6bf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9110bdb640 1 -- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 msgr2=0x55eacf69cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf69cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9110bdb640 1 -- 172.21.15.148:0/1108669165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eacf2671b0 con 0x55eacf656ca0 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9110bdb640 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf692980 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00cdf0 tx=0x7f90fc05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3e5e2a8a38be72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f90f1ffb640 1 -- 172.21.15.148:0/1108669165 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc07c070 con 0x55eacf656ca0 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f90f1ffb640 1 -- 172.21.15.148:0/1108669165 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc058f70 con 0x55eacf656ca0 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f90f1ffb640 1 -- 172.21.15.148:0/1108669165 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc086050 con 0x55eacf656ca0 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 -- 172.21.15.148:0/1108669165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 msgr2=0x55eacf692980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf692980 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00cdf0 tx=0x7f90fc05a000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 -- 172.21.15.148:0/1108669165 shutdown_connections 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf693370 0x55eacf69cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf681760 0x55eacf6bf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 --2- 172.21.15.148:0/1108669165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf692980 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 -- 172.21.15.148:0/1108669165 >> 172.21.15.148:0/1108669165 conn(0x55eacf61e870 msgr2=0x55eacf697f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 -- 172.21.15.148:0/1108669165 shutdown_connections 2024-04-25T10:03:06.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.040+0000 7f9115c1d740 1 -- 172.21.15.148:0/1108669165 wait complete. 2024-04-25T10:03:06.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.041+0000 7f9115c1d740 1 -- start start 2024-04-25T10:03:06.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.041+0000 7f9115c1d740 1 Processor -- start 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf709260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf683b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf684090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c0c8980 con 0x55eacf656ca0 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eacf6931f0 con 0x55eacf681760 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eacf680050 con 0x55eacf693370 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9115c1d740 10 client.0 _ll_register_callbacks cb 0x7f90fc0582a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf709260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf709260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38798/0 (socket says 172.21.15.148:38798) 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 -- 172.21.15.148:0/105331807 learned_addr learned my addr 172.21.15.148:0/105331807 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f910bfff640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf683b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f91113dc640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf684090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 -- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 msgr2=0x55eacf684090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf684090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 -- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 msgr2=0x55eacf683b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf683b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 -- 172.21.15.148:0/105331807 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eacf684cd0 con 0x55eacf656ca0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f91113dc640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf684090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f9110bdb640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf709260 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f90fc058570 tx=0x7f90fc003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f6234e7b3ad76f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f90eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.042+0000 7f90eb7fe640 1 -- 172.21.15.148:0/105331807 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc07c070 con 0x55eacf656ca0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f90eb7fe640 1 -- 172.21.15.148:0/105331807 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90fc060af0 con 0x55eacf656ca0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90fc058f70 con 0x55eacf656ca0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f90eb7fe640 1 -- 172.21.15.148:0/105331807 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90fc067050 con 0x55eacf656ca0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 10 client.9549 fetch_fsmap learned FSMap version 74 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55eacf684cd0 con 0x55eacf656ca0 2024-04-25T10:03:06.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f90eb7fe640 1 -- 172.21.15.148:0/105331807 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc058f70 con 0x55eacf656ca0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f90eb7fe640 1 -- 172.21.15.148:0/105331807 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f90fc06a020 con 0x55eacf656ca0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 10 client.9549 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 10 client.9549 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 20 client.9549 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf7097a0 0x55eacf6804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 4 client.9549 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55eacf5c1420 con 0x55eacf7097a0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f910bfff640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf7097a0 0x55eacf6804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f90eb7fe640 10 client.9549 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.043+0000 7f910bfff640 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf7097a0 0x55eacf6804e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88a244dc885d727 server_cookie=37e4c248b78c40bc in_seq=0 out_seq=0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f90eb7fe640 1 -- 172.21.15.148:0/105331807 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55eacf5c1420 con 0x55eacf7097a0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f90eb7fe640 10 client.9549 handle_command_reply: tid=0 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 client.9549 shutdown 2024-04-25T10:03:06.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 msgr2=0x55eacf709260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf709260 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f90fc058570 tx=0x7f90fc003960 comp rx=0 tx=0).stop 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 shutdown_connections 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x55eacf7097a0 0x55eacf6804e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eacf693370 0x55eacf684090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacf681760 0x55eacf683b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f90eb7fe640 0 client.9549 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 --2- 172.21.15.148:0/105331807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eacf656ca0 0x55eacf709260 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 >> 172.21.15.148:0/105331807 conn(0x55eacf61e870 msgr2=0x55eacf696eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 shutdown_connections 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 1 -- 172.21.15.148:0/105331807 wait complete. 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9115c1d740 20 client.9549 trim_cache size 0 max 16384 2024-04-25T10:03:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:03:06.056 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0 2024-04-25T10:03:06.056 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90f806b4d0 msgr2=0x7f90f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90f806b4d0 0x7f90f806d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9100034000 tx=0x7f910002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 msgr2=0x7f910c15a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c15a2a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c430 tx=0x7f90fc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 shutdown_connections 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.044+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f90f806b4d0 0x7f90f806d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.045+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f910c161710 0x7f910c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.045+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f910c15dc20 0x7f910c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.045+0000 7f9112e3f640 1 --2- 172.21.15.148:0/1134833925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f910c0ced80 0x7f910c15a2a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.045+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 >> 172.21.15.148:0/1134833925 conn(0x7f910c0d0420 msgr2=0x7f910c15c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.045+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 shutdown_connections 2024-04-25T10:03:06.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.045+0000 7f9112e3f640 1 -- 172.21.15.148:0/1134833925 wait complete. 2024-04-25T10:03:06.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:06.060 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T10:03:06.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.155+0000 7f73397f4640 1 Processor -- start 2024-04-25T10:03:06.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.155+0000 7f73397f4640 1 -- start start 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73397f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73397f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733406cda0 con 0x7f7334161710 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73397f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733415e000 con 0x7f73340d32f0 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73397f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340d1de0 con 0x7f733415dc20 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f732bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7334161710 0x7f733406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33720/0 (socket says 172.21.15.148:33720) 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f732bfff640 1 -- 172.21.15.148:0/2415173591 learned_addr learned my addr 172.21.15.148:0/2415173591 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f7332ffd640 1 --2- 172.21.15.148:0/2415173591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644712591 0 0) 0x7f733406cda0 con 0x7f7334161710 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7320003520 con 0x7f7334161710 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194623497 0 0) 0x7f733415e000 con 0x7f73340d32f0 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f733406cda0 con 0x7f73340d32f0 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f7332ffd640 1 -- 172.21.15.148:0/2415173591 >> v1:172.21.15.188:6789/0 conn(0x7f73340d32f0 legacy=0x7f733406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f7332ffd640 1 -- 172.21.15.148:0/2415173591 >> v1:172.21.15.148:6789/0 conn(0x7f7334161710 legacy=0x7f733406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.156+0000 7f7332ffd640 1 -- 172.21.15.148:0/2415173591 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334161c60 con 0x7f733415dc20 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.157+0000 7f7332ffd640 1 --2- 172.21.15.148:0/2415173591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340ce670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097d0 tx=0x7f731c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16770934e81f2104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.157+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c037070 con 0x7f733415dc20 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.157+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c032440 con 0x7f733415dc20 2024-04-25T10:03:06.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.157+0000 7f73327fc640 1 -- 172.21.15.148:0/2415173591 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0327f0 con 0x7f733415dc20 2024-04-25T10:03:06.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 -- 172.21.15.148:0/2415173591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 msgr2=0x7f73340ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 --2- 172.21.15.148:0/2415173591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340ce670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097d0 tx=0x7f731c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 -- 172.21.15.148:0/2415173591 shutdown_connections 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 --2- 172.21.15.148:0/2415173591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340ce670 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 -- 172.21.15.148:0/2415173591 >> 172.21.15.148:0/2415173591 conn(0x7f73340d0420 msgr2=0x7f73340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 -- 172.21.15.148:0/2415173591 shutdown_connections 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.158+0000 7f73397f4640 1 -- 172.21.15.148:0/2415173591 wait complete. 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 Processor -- start 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 -- start start 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340d32f0 0x7f73340c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 0x7f73340b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7334162500 con 0x7f7334161710 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.159+0000 7f73397f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7334162220 con 0x7f73340d32f0 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73397f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73340b5e50 con 0x7f733415dc20 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 0x7f73340b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 0x7f73340b86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38806/0 (socket says 172.21.15.148:38806) 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 -- 172.21.15.148:0/2876871797 learned_addr learned my addr 172.21.15.148:0/2876871797 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f7332ffd640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340d32f0 0x7f73340c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f732bfff640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 -- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 msgr2=0x7f73340b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 -- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340d32f0 msgr2=0x7f73340c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340d32f0 0x7f73340c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 -- 172.21.15.148:0/2876871797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340ca2f0 con 0x7f7334161710 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73337fe640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 0x7f73340b86e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c730 tx=0x7f732c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f732b7fe640 1 -- 172.21.15.148:0/2876871797 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c003b70 con 0x7f7334161710 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f732b7fe640 1 -- 172.21.15.148:0/2876871797 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c003d10 con 0x7f7334161710 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733415a760 con 0x7f7334161710 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f732b7fe640 1 -- 172.21.15.148:0/2876871797 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c005310 con 0x7f7334161710 2024-04-25T10:03:06.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.160+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7334146de0 con 0x7f7334161710 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.162+0000 7f732b7fe640 1 -- 172.21.15.148:0/2876871797 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f732c0054b0 con 0x7f7334161710 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.162+0000 7f732b7fe640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.162+0000 7f732b7fe640 1 -- 172.21.15.148:0/2876871797 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f732c08a0d0 con 0x7f7334161710 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.162+0000 7f7332ffd640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.162+0000 7f7332ffd640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f731406b4d0 0x7f731406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f731c034000 tx=0x7f731c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.163+0000 7f733c5cb740 1 Processor -- start 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.163+0000 7f733c5cb740 1 -- start start 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f55915320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55915860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f558e5770 0x564f558e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f733c5cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5584eda0 con 0x564f55912ef0 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f557f7b80 con 0x564f558e5770 2024-04-25T10:03:06.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f5584ae40 con 0x564f55912b10 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55915860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55915860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38810/0 (socket says 172.21.15.148:38810) 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 -- 172.21.15.148:0/807342352 learned_addr learned my addr 172.21.15.148:0/807342352 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f73337fe640 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f558e5770 0x564f558e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f7332ffd640 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f55915320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 -- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 msgr2=0x564f55915320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f55915320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 -- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f558e5770 msgr2=0x564f558e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f558e5770 0x564f558e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 -- 172.21.15.148:0/807342352 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f55801f10 con 0x564f55912ef0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.164+0000 7f732bfff640 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55915860 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7324006ee0 tx=0x7f7324007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5d74c899ca11af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f7328ff9640 1 -- 172.21.15.148:0/807342352 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324001760 con 0x564f55912ef0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f7328ff9640 1 -- 172.21.15.148:0/807342352 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7324018070 con 0x564f55912ef0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f7328ff9640 1 -- 172.21.15.148:0/807342352 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324001b60 con 0x564f55912ef0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 -- 172.21.15.148:0/807342352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 msgr2=0x564f55915860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55915860 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7324006ee0 tx=0x7f7324007410 comp rx=0 tx=0).stop 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 -- 172.21.15.148:0/807342352 shutdown_connections 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f558e5770 0x564f558e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55915860 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 --2- 172.21.15.148:0/807342352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f55915320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 -- 172.21.15.148:0/807342352 >> 172.21.15.148:0/807342352 conn(0x564f558e79e0 msgr2=0x564f55930320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 -- 172.21.15.148:0/807342352 shutdown_connections 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.165+0000 7f733c5cb740 1 -- 172.21.15.148:0/807342352 wait complete. 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.166+0000 7f733c5cb740 1 -- start start 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.166+0000 7f733c5cb740 1 Processor -- start 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f559349f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912ef0 0x564f55934f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 0x564f55910e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f558eab90 con 0x564f559109f0 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f557ed4b0 con 0x564f55912ef0 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f558eae70 con 0x564f55912b10 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 10 client.0 _ll_register_callbacks cb 0x564f5582d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 0x564f55910e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 0x564f55910e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38826/0 (socket says 172.21.15.148:38826) 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 -- 172.21.15.148:0/1559025802 learned_addr learned my addr 172.21.15.148:0/1559025802 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f732bfff640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912ef0 0x564f55934f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f7332ffd640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f559349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 -- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 msgr2=0x564f559349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f559349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 -- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912ef0 msgr2=0x564f55934f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912ef0 0x564f55934f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 -- 172.21.15.148:0/1559025802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f5562d7d0 con 0x564f559109f0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f73337fe640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 0x564f55910e70 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f732c00a430 tx=0x7f732c05c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b622eac88451749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f7311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f7311ffb640 1 -- 172.21.15.148:0/1559025802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c054e50 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f7311ffb640 1 -- 172.21.15.148:0/1559025802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f732c057c30 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.167+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7324018070 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f7311ffb640 1 -- 172.21.15.148:0/1559025802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f732c054ff0 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 10 client.9561 fetch_fsmap learned FSMap version 74 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x564f555633f0 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f7311ffb640 1 -- 172.21.15.148:0/1559025802 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7324018070 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f7311ffb640 1 -- 172.21.15.148:0/1559025802 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f732c054530 con 0x564f559109f0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 10 client.9561 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 10 client.9561 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 20 client.9561 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f55932d70 0x564f55933190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 4 client.9561 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f5585c2e0 con 0x564f55932d70 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f7332ffd640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f55932d70 0x564f55933190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f7332ffd640 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f55932d70 0x564f55933190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d93598df3a9ba5 server_cookie=20942ef8bedf620c in_seq=0 out_seq=0 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.168+0000 7f7311ffb640 10 client.9561 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f7311ffb640 1 -- 172.21.15.148:0/1559025802 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f5585c2e0 con 0x564f55932d70 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f7311ffb640 10 client.9561 handle_command_reply: tid=0 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 client.9561 shutdown 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 msgr2=0x564f55910e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 0x564f55910e70 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f732c00a430 tx=0x7f732c05c940 comp rx=0 tx=0).stop 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 shutdown_connections 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f55932d70 0x564f55933190 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f559109f0 0x564f55910e70 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f7311ffb640 0 client.9561 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912ef0 0x564f55934f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 --2- 172.21.15.148:0/1559025802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f55912b10 0x564f559349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 >> 172.21.15.148:0/1559025802 conn(0x564f558e5770 msgr2=0x564f5592ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 shutdown_connections 2024-04-25T10:03:06.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 1 -- 172.21.15.148:0/1559025802 wait complete. 2024-04-25T10:03:06.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.169+0000 7f733c5cb740 20 client.9561 trim_cache size 0 max 16384 2024-04-25T10:03:06.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 Processor -- start 2024-04-25T10:03:06.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 -- start start 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f55951a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f558e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73342003d0 con 0x564f55912ef0 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73341ff530 con 0x564f55912b10 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.184+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7334148440 con 0x564f559109f0 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f558e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f558e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38842/0 (socket says 172.21.15.148:38842) 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 -- 172.21.15.148:0/2522784622 learned_addr learned my addr 172.21.15.148:0/2522784622 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f7332ffd640 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f55951a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 -- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 msgr2=0x564f55951a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f55951a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f732bfff640 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 -- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 msgr2=0x564f558e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 -- 172.21.15.148:0/2522784622 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f555230f0 con 0x564f55912ef0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f73337fe640 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f558e4b40 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f732c005010 tx=0x7f732c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d94f5ad2ac8de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f7313fff640 1 -- 172.21.15.148:0/2522784622 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c054e50 con 0x564f55912ef0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f7313fff640 1 -- 172.21.15.148:0/2522784622 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c0701c0 con 0x564f55912ef0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.185+0000 7f7313fff640 1 -- 172.21.15.148:0/2522784622 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c054a80 con 0x564f55912ef0 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 -- 172.21.15.148:0/2522784622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 msgr2=0x564f558e4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f558e4b40 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f732c005010 tx=0x7f732c01d000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 -- 172.21.15.148:0/2522784622 shutdown_connections 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f558e4b40 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 --2- 172.21.15.148:0/2522784622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f55951a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 -- 172.21.15.148:0/2522784622 >> 172.21.15.148:0/2522784622 conn(0x564f558e5770 msgr2=0x564f559366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 -- 172.21.15.148:0/2522784622 shutdown_connections 2024-04-25T10:03:06.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 -- 172.21.15.148:0/2522784622 wait complete. 2024-04-25T10:03:06.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 -- start start 2024-04-25T10:03:06.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.186+0000 7f733c5cb740 1 Processor -- start 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f558e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55931450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f55851240 con 0x564f55912ef0 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73341fef60 con 0x564f55912b10 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f55932b20 con 0x564f559109f0 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f733c5cb740 10 client.0 _ll_register_callbacks cb 0x7f7324006980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55931450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55931450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38846/0 (socket says 172.21.15.148:38846) 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 -- 172.21.15.148:0/3915563998 learned_addr learned my addr 172.21.15.148:0/3915563998 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f732bfff640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f7332ffd640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f558e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 -- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 msgr2=0x564f558e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f558e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 -- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 msgr2=0x564f558e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.187+0000 7f73337fe640 1 -- 172.21.15.148:0/3915563998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f559332d0 con 0x564f55912ef0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f73337fe640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55931450 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f732c0589a0 tx=0x7f732c0883b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3830a57f241418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 1 -- 172.21.15.148:0/3915563998 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c054a80 con 0x564f55912ef0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 1 -- 172.21.15.148:0/3915563998 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f732c057c30 con 0x564f55912ef0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f732c0701c0 con 0x564f55912ef0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 1 -- 172.21.15.148:0/3915563998 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f732c054e50 con 0x564f55912ef0 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 10 client.9567 fetch_fsmap learned FSMap version 74 2024-04-25T10:03:06.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x564f559332d0 con 0x564f55912ef0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 1 -- 172.21.15.148:0/3915563998 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c0701c0 con 0x564f55912ef0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 1 -- 172.21.15.148:0/3915563998 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f732c07e020 con 0x564f55912ef0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 10 client.9567 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 10 client.9567 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 20 client.9567 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f558e10d0 0x564f55931990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 4 client.9567 mds_command: new command op to 9351 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 --> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f55856630 con 0x564f558e10d0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7332ffd640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f558e10d0 0x564f55931990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7311ffb640 10 client.9567 ms_handle_connect on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.188+0000 7f7332ffd640 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f558e10d0 0x564f55931990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c6c1c8549e59f8 server_cookie=aa23d68d0528e778 in_seq=0 out_seq=0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f7311ffb640 1 -- 172.21.15.148:0/3915563998 <== mds.0 v2:172.21.15.148:6836/1740205817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x564f55856630 con 0x564f558e10d0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f7311ffb640 10 client.9567 handle_command_reply: tid=0 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 client.9567 shutdown 2024-04-25T10:03:06.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 msgr2=0x564f55931450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55931450 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f732c0589a0 tx=0x7f732c0883b0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 shutdown_connections 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:6836/1740205817,v1:172.21.15.148:6837/1740205817] conn(0x564f558e10d0 0x564f55931990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f55912ef0 0x564f55931450 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f55912b10 0x564f558e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f7311ffb640 0 client.9567 ms_handle_reset on v2:172.21.15.148:6836/1740205817 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 --2- 172.21.15.148:0/3915563998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564f559109f0 0x564f558e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 >> 172.21.15.148:0/3915563998 conn(0x564f558e5770 msgr2=0x564f558eae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 shutdown_connections 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 1 -- 172.21.15.148:0/3915563998 wait complete. 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f733c5cb740 20 client.9567 trim_cache size 0 max 16384 2024-04-25T10:03:06.200 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: "7baef3be-4ae9-493f-a68c-921c1cd79b0d": { 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "7baef3be-4ae9-493f-a68c-921c1cd79b0d", 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:06.201 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:03:06.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f731406b4d0 msgr2=0x7f731406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f731406b4d0 0x7f731406d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f731c034000 tx=0x7f731c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 msgr2=0x7f73340b86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 0x7f73340b86e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c730 tx=0x7f732c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 shutdown_connections 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f731406b4d0 0x7f731406d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334161710 0x7f73340b86e0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f733415dc20 0x7f73340b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 --2- 172.21.15.148:0/2876871797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73340d32f0 0x7f73340c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 >> 172.21.15.148:0/2876871797 conn(0x7f73340d0420 msgr2=0x7f73340b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 shutdown_connections 2024-04-25T10:03:06.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.189+0000 7f73397f4640 1 -- 172.21.15.148:0/2876871797 wait complete. 2024-04-25T10:03:06.205 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.297+0000 7f6e97d6a640 1 Processor -- start 2024-04-25T10:03:06.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.297+0000 7f6e97d6a640 1 -- start start 2024-04-25T10:03:06.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e97d6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e900d0170 0x7f6e900ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e97d6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 0x7f6e9006c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e97d6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e900c8030 con 0x7f6e9015da70 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e97d6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e900ceb30 con 0x7f6e900d0170 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e97d6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900c8310 con 0x7f6e900ce700 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e95b06640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6e9015da70 0x7f6e900c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33734/0 (socket says 172.21.15.148:33734) 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e95b06640 1 -- 172.21.15.148:0/3557685638 learned_addr learned my addr 172.21.15.148:0/3557685638 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e95305640 1 --2- 172.21.15.148:0/3557685638 >> v2:172.21.15.188:3300/0 conn(0x7f6e900d0170 0x7f6e900ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e96307640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 0x7f6e9006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566563386 0 0) 0x7f6e900c8030 con 0x7f6e9015da70 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c003520 con 0x7f6e9015da70 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 945015135 0 0) 0x7f6e7c003520 con 0x7f6e9015da70 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 msgr2=0x7f6e9006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 0x7f6e9006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 >> v2:172.21.15.188:3300/0 conn(0x7f6e900d0170 msgr2=0x7f6e900ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 --2- 172.21.15.148:0/3557685638 >> v2:172.21.15.188:3300/0 conn(0x7f6e900d0170 0x7f6e900ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900c85f0 con 0x7f6e9015da70 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6e8c001690 con 0x7f6e9015da70 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.298+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 >> v1:172.21.15.148:6789/0 conn(0x7f6e9015da70 legacy=0x7f6e900c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e94b04640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c0039b0 0x7f6e7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e94b04640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7c004800 0x7f6e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e94b04640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 0x7f6e7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900ceb30 con 0x7f6e7c004800 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e900c8310 con 0x7f6e7c0039b0 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00b9c0 con 0x7f6e900ce700 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 --2- 172.21.15.148:0/3557685638 >> v2:172.21.15.188:3300/0 conn(0x7f6e900d0170 0x7f6e900ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95b06640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c0039b0 0x7f6e7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7c004800 0x7f6e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e96307640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 0x7f6e7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 -- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 msgr2=0x7f6e7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e900ce700 0x7f6e7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 -- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c0039b0 msgr2=0x7f6e7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c0039b0 0x7f6e7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.299+0000 7f6e95305640 1 -- 172.21.15.148:0/3557685638 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c003520 con 0x7f6e7c004800 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e95305640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7c004800 0x7f6e7c004c80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009c70 tx=0x7f6e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd63521891ee1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80009840 con 0x7f6e7c004800 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80004020 con 0x7f6e7c004800 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e94b04640 1 -- 172.21.15.148:0/3557685638 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e800043d0 con 0x7f6e7c004800 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3557685638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7c004800 msgr2=0x7f6e7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3557685638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7c004800 0x7f6e7c004c80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009c70 tx=0x7f6e7c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e96307640 1 -- 172.21.15.148:0/3557685638 reap_dead start 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3557685638 shutdown_connections 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3557685638 >> 172.21.15.148:0/3557685638 conn(0x7f6e900b33e0 msgr2=0x7f6e900b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3557685638 shutdown_connections 2024-04-25T10:03:06.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.300+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3557685638 wait complete. 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 Processor -- start 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 -- start start 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c004800 0x7f6e9015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e900b6150 0x7f6e900b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 0x7f6e90154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9015de50 con 0x7f6e901544b0 2024-04-25T10:03:06.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9006be10 con 0x7f6e900b6150 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e97d6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9006c330 con 0x7f6e7c004800 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e96307640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 0x7f6e90154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e96307640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 0x7f6e90154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38876/0 (socket says 172.21.15.148:38876) 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e96307640 1 -- 172.21.15.148:0/3082825560 learned_addr learned my addr 172.21.15.148:0/3082825560 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.302+0000 7f6e95b06640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c004800 0x7f6e9015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e95305640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e900b6150 0x7f6e900b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e96307640 1 -- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c004800 msgr2=0x7f6e9015b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e96307640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c004800 0x7f6e9015b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e96307640 1 -- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e900b6150 msgr2=0x7f6e900b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e96307640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e900b6150 0x7f6e900b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e96307640 1 -- 172.21.15.148:0/3082825560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900b6c10 con 0x7f6e901544b0 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e96307640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 0x7f6e90154930 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6e9006b8b0 tx=0x7f6e8800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800fde0 con 0x7f6e901544b0 2024-04-25T10:03:06.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8801d070 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e900b6ef0 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88015670 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.303+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e900b74b0 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.304+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e90073990 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.305+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e88015810 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.305+0000 7f6e867fc640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7406b4d0 0x7f6e7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.305+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e8808aba0 con 0x7f6e901544b0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.305+0000 7f6e95b06640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7406b4d0 0x7f6e7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.305+0000 7f6e95b06640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7406b4d0 0x7f6e7406d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c000c00 tx=0x7f6e8c001e20 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.309+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e88010040 con 0x7f6e901544b0 2024-04-25T10:03:06.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.515+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e90154e70 con 0x7f6e901544b0 2024-04-25T10:03:06.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.515+0000 7f6e867fc640 1 -- 172.21.15.148:0/3082825560 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f6e880591b0 con 0x7f6e901544b0 2024-04-25T10:03:06.520 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:06.520 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:06.521 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.517+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7406b4d0 msgr2=0x7f6e7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.517+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7406b4d0 0x7f6e7406d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c000c00 tx=0x7f6e8c001e20 comp rx=0 tx=0).stop 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.517+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 msgr2=0x7f6e90154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.517+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 0x7f6e90154930 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f6e9006b8b0 tx=0x7f6e8800f580 comp rx=0 tx=0).stop 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 shutdown_connections 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e7406b4d0 0x7f6e7406d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e901544b0 0x7f6e90154930 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e900b6150 0x7f6e900b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 --2- 172.21.15.148:0/3082825560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c004800 0x7f6e9015b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 >> 172.21.15.148:0/3082825560 conn(0x7f6e900b33e0 msgr2=0x7f6e9006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 shutdown_connections 2024-04-25T10:03:06.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.518+0000 7f6e97d6a640 1 -- 172.21.15.148:0/3082825560 wait complete. 2024-04-25T10:03:06.533 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:03:06.533 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:06.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.625+0000 7f44f1e6b640 1 Processor -- start 2024-04-25T10:03:06.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.625+0000 7f44f1e6b640 1 -- start start 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.625+0000 7f44f1e6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.625+0000 7f44f1e6b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec06c220 con 0x7f44ec06ce10 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.625+0000 7f44f1e6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec0d0080 con 0x7f44ec09a450 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.625+0000 7f44f1e6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec09a8b0 con 0x7f44ec15dc70 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44ebfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f44ec06ce10 0x7f44ec0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33738/0 (socket says 172.21.15.148:33738) 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44ebfff640 1 -- 172.21.15.148:0/675231584 learned_addr learned my addr 172.21.15.148:0/675231584 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44eb7fe640 1 --2- 172.21.15.148:0/675231584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44ea7fc640 1 -- 172.21.15.148:0/675231584 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226165834 0 0) 0x7f44ec06c220 con 0x7f44ec06ce10 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44ea7fc640 1 -- 172.21.15.148:0/675231584 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d4003520 con 0x7f44ec06ce10 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44ea7fc640 1 -- 172.21.15.148:0/675231584 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3785190661 0 0) 0x7f44d4003520 con 0x7f44ec06ce10 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44eb7fe640 1 -- 172.21.15.148:0/675231584 >> v1:172.21.15.188:6789/0 conn(0x7f44ec09a450 legacy=0x7f44ec06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44eb7fe640 1 -- 172.21.15.148:0/675231584 >> v1:172.21.15.148:6789/0 conn(0x7f44ec06ce10 legacy=0x7f44ec0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.626+0000 7f44eb7fe640 1 -- 172.21.15.148:0/675231584 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec0d0360 con 0x7f44ec15dc70 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.627+0000 7f44eb7fe640 1 --2- 172.21.15.148:0/675231584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec0d1120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44e00097d0 tx=0x7f44e002a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157c5a1bd80eb7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.627+0000 7f44ea7fc640 1 -- 172.21.15.148:0/675231584 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0037070 con 0x7f44ec15dc70 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.627+0000 7f44ea7fc640 1 -- 172.21.15.148:0/675231584 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e002ab40 con 0x7f44ec15dc70 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.627+0000 7f44ea7fc640 1 -- 172.21.15.148:0/675231584 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e40019f0 con 0x7f44ec15dc70 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.627+0000 7f44f1e6b640 1 -- 172.21.15.148:0/675231584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 msgr2=0x7f44ec0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.627+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/675231584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec0d1120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f44e00097d0 tx=0x7f44e002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.628+0000 7f44f1e6b640 1 -- 172.21.15.148:0/675231584 shutdown_connections 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.628+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/675231584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec0d1120 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.628+0000 7f44f1e6b640 1 -- 172.21.15.148:0/675231584 >> 172.21.15.148:0/675231584 conn(0x7f44ec0b3270 msgr2=0x7f44ec0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.628+0000 7f44f1e6b640 1 -- 172.21.15.148:0/675231584 shutdown_connections 2024-04-25T10:03:06.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.628+0000 7f44f1e6b640 1 -- 172.21.15.148:0/675231584 wait complete. 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.629+0000 7f44f1e6b640 1 Processor -- start 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.629+0000 7f44f1e6b640 1 -- start start 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.629+0000 7f44f1e6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44ec06ce10 0x7f44ec0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.629+0000 7f44f1e6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 0x7f44ec09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.629+0000 7f44f1e6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.629+0000 7f44f1e6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0d0920 con 0x7f44ec09a450 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44f1e6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0d0c90 con 0x7f44ec06ce10 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44f1e6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec15e050 con 0x7f44ec15dc70 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 0x7f44ec09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 0x7f44ec09b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38884/0 (socket says 172.21.15.148:38884) 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 -- 172.21.15.148:0/2305048828 learned_addr learned my addr 172.21.15.148:0/2305048828 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eb7fe640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44ec06ce10 0x7f44ec0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44ebfff640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 -- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 msgr2=0x7f44ec1f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec1f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 -- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44ec06ce10 msgr2=0x7f44ec0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44ec06ce10 0x7f44ec0c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 -- 172.21.15.148:0/2305048828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec1522f0 con 0x7f44ec09a450 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44eaffd640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 0x7f44ec09b5a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00c570 tx=0x7f44dc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc005480 con 0x7f44ec09a450 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc0035f0 con 0x7f44ec09a450 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec1f72b0 con 0x7f44ec09a450 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc0039c0 con 0x7f44ec09a450 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.630+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec1f7870 con 0x7f44ec09a450 2024-04-25T10:03:06.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.631+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec073990 con 0x7f44ec09a450 2024-04-25T10:03:06.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.632+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44dc003b60 con 0x7f44ec09a450 2024-04-25T10:03:06.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.632+0000 7f44d3fff640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44d806b670 0x7f44d806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.632+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44dc089ef0 con 0x7f44ec09a450 2024-04-25T10:03:06.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.632+0000 7f44eb7fe640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44d806b670 0x7f44d806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.633+0000 7f44eb7fe640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44d806b670 0x7f44d806db30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f44e0034000 tx=0x7f44e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.636+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44ec1522f0 con 0x7f44ec09a450 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.873+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f44ec0b40a0 con 0x7f44ec09a450 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.874+0000 7f44d3fff640 1 -- 172.21.15.148:0/2305048828 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f44dc058620 con 0x7f44ec09a450 2024-04-25T10:03:06.879 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:06.879 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":852,"data_bytes":870763554,"bytes_used":2057330688,"bytes_avail":771036782592,"bytes_total":773094113280,"write_bytes_sec":72510538,"read_op_per_sec":0,"write_op_per_sec":71},"fsmap":{"epoch":74,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9351}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:02:33.121464+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44d806b670 msgr2=0x7f44d806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44d806b670 0x7f44d806db30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f44e0034000 tx=0x7f44e002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 msgr2=0x7f44ec09b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 0x7f44ec09b5a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00c570 tx=0x7f44dc00ca30 comp rx=0 tx=0).stop 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 shutdown_connections 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44d806b670 0x7f44d806db30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44ec15dc70 0x7f44ec1f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44ec09a450 0x7f44ec09b5a0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 --2- 172.21.15.148:0/2305048828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44ec06ce10 0x7f44ec0c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 >> 172.21.15.148:0/2305048828 conn(0x7f44ec0b3270 msgr2=0x7f44ec070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 shutdown_connections 2024-04-25T10:03:06.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:06.876+0000 7f44f1e6b640 1 -- 172.21.15.148:0/2305048828 wait complete. 2024-04-25T10:03:06.891 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:03:07.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:08.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.996+0000 7f1b9ee9a640 1 Processor -- start 2024-04-25T10:03:08.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.996+0000 7f1b9ee9a640 1 -- start start 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9ee9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b980ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9ee9a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b980d0b10 0x7f1b980d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9ee9a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b980cec00 con 0x7f1b980b5f00 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9ee9a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980ce930 con 0x7f1b980d0b10 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9ee9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b6450 con 0x7f1b9815db60 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b980d0b10 0x7f1b980d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1b980d0b10 0x7f1b980d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38900/0 (socket says 172.21.15.148:38900) 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 -- 172.21.15.148:0/594197561 learned_addr learned my addr 172.21.15.148:0/594197561 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 -- 172.21.15.148:0/594197561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 msgr2=0x7f1b980ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 --2- 172.21.15.148:0/594197561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b980ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 -- 172.21.15.148:0/594197561 >> v1:172.21.15.188:6789/0 conn(0x7f1b980b5f00 legacy=0x7f1b980d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.997+0000 7f1b9d437640 1 -- 172.21.15.148:0/594197561 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980ceee0 con 0x7f1b980d0b10 2024-04-25T10:03:08.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.998+0000 7f1b9d437640 1 --2- 172.21.15.148:0/594197561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b980d2270 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009e50 tx=0x7f1b9002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cd8b0d4994dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.998+0000 7f1b977fe640 1 -- 172.21.15.148:0/594197561 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90036070 con 0x7f1b980d0b10 2024-04-25T10:03:08.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.998+0000 7f1b977fe640 1 -- 172.21.15.148:0/594197561 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b90031600 con 0x7f1b980d0b10 2024-04-25T10:03:08.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.998+0000 7f1b977fe640 1 -- 172.21.15.148:0/594197561 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900319d0 con 0x7f1b980d0b10 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/594197561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 msgr2=0x7f1b980d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/594197561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b980d2270 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009e50 tx=0x7f1b9002a630 comp rx=0 tx=0).stop 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/594197561 shutdown_connections 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/594197561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b980d2270 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/594197561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b980ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/594197561 >> 172.21.15.148:0/594197561 conn(0x7f1b98061600 msgr2=0x7f1b98061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/594197561 shutdown_connections 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:07.999+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/594197561 wait complete. 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.000+0000 7f1b9ee9a640 1 Processor -- start 2024-04-25T10:03:08.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.000+0000 7f1b9ee9a640 1 -- start start 2024-04-25T10:03:08.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9ee9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b980b5f00 0x7f1b980cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9ee9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b981488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9ee9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b981f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9ee9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9815df40 con 0x7f1b980d0b10 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9ee9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980d1d50 con 0x7f1b980b5f00 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9ee9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9815e0c0 con 0x7f1b9815db60 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b981488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b981488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38906/0 (socket says 172.21.15.148:38906) 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 -- 172.21.15.148:0/3755693998 learned_addr learned my addr 172.21.15.148:0/3755693998 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9d437640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b981f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b9cc36640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b980b5f00 0x7f1b980cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 -- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 msgr2=0x7f1b981f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b981f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 -- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b980b5f00 msgr2=0x7f1b980cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b980b5f00 0x7f1b980cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 -- 172.21.15.148:0/3755693998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b98146010 con 0x7f1b980d0b10 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.001+0000 7f1b97fff640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b981488a0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400c400 tx=0x7f1b8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.002+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84011790 con 0x7f1b980d0b10 2024-04-25T10:03:08.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.002+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b84017070 con 0x7f1b980d0b10 2024-04-25T10:03:08.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.002+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84011bb0 con 0x7f1b980d0b10 2024-04-25T10:03:08.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.002+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b981462f0 con 0x7f1b980d0b10 2024-04-25T10:03:08.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.002+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b98148de0 con 0x7f1b980d0b10 2024-04-25T10:03:08.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.003+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98073270 con 0x7f1b980d0b10 2024-04-25T10:03:08.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.003+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b8401c460 con 0x7f1b980d0b10 2024-04-25T10:03:08.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.003+0000 7f1b957fa640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b7c06b4d0 0x7f1b7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.004+0000 7f1b9cc36640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b7c06b4d0 0x7f1b7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.004+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b8408a110 con 0x7f1b980d0b10 2024-04-25T10:03:08.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.004+0000 7f1b9cc36640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b7c06b4d0 0x7f1b7c06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0098e0 tx=0x7f1b8c009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.008+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b840587b0 con 0x7f1b980d0b10 2024-04-25T10:03:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.215+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b98073990 con 0x7f1b980d0b10 2024-04-25T10:03:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.216+0000 7f1b957fa640 1 -- 172.21.15.148:0/3755693998 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f1b8405c850 con 0x7f1b980d0b10 2024-04-25T10:03:08.221 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:08.221 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.217+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b7c06b4d0 msgr2=0x7f1b7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.217+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b7c06b4d0 0x7f1b7c06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0098e0 tx=0x7f1b8c009060 comp rx=0 tx=0).stop 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.217+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 msgr2=0x7f1b981488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.217+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b981488a0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400c400 tx=0x7f1b8400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 shutdown_connections 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1b7c06b4d0 0x7f1b7c06d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b9815db60 0x7f1b981f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b980d0b10 0x7f1b981488a0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 --2- 172.21.15.148:0/3755693998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b980b5f00 0x7f1b980cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 >> 172.21.15.148:0/3755693998 conn(0x7f1b98061600 msgr2=0x7f1b980cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 shutdown_connections 2024-04-25T10:03:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.218+0000 7f1b9ee9a640 1 -- 172.21.15.148:0/3755693998 wait complete. 2024-04-25T10:03:08.233 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:03:08.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.324+0000 7fced99db640 1 Processor -- start 2024-04-25T10:03:08.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.324+0000 7fced99db640 1 -- start start 2024-04-25T10:03:08.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fced99db640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fced406b5f0 0x7fced40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fced99db640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fced40cffb0 0x7fced415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fced99db640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced415f710 con 0x7fced415b980 2024-04-25T10:03:08.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fced99db640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced415f9b0 con 0x7fced406b5f0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fced99db640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced415fc90 con 0x7fced40cffb0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fcecbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fced406b5f0 0x7fced40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fcecbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fced406b5f0 0x7fced40cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38912/0 (socket says 172.21.15.148:38912) 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.325+0000 7fcecbfff640 1 -- 172.21.15.148:0/107744647 learned_addr learned my addr 172.21.15.148:0/107744647 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fced37fe640 1 --2- 172.21.15.148:0/107744647 >> v2:172.21.15.188:3300/0 conn(0x7fced40cffb0 0x7fced415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fcecbfff640 1 -- 172.21.15.148:0/107744647 >> v2:172.21.15.188:3300/0 conn(0x7fced40cffb0 msgr2=0x7fced415c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fcecbfff640 1 --2- 172.21.15.148:0/107744647 >> v2:172.21.15.188:3300/0 conn(0x7fced40cffb0 0x7fced415c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fcecbfff640 1 -- 172.21.15.148:0/107744647 >> v1:172.21.15.188:6789/0 conn(0x7fced415b980 legacy=0x7fced40cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fcecbfff640 1 -- 172.21.15.148:0/107744647 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced415ff70 con 0x7fced406b5f0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fcecbfff640 1 --2- 172.21.15.148:0/107744647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced406b5f0 0x7fced40cc300 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fced40cc840 tx=0x7fcebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6ad3cd7744f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.326+0000 7fced27fc640 1 -- 172.21.15.148:0/107744647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc037070 con 0x7fced406b5f0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced27fc640 1 -- 172.21.15.148:0/107744647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc032440 con 0x7fced406b5f0 2024-04-25T10:03:08.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced27fc640 1 -- 172.21.15.148:0/107744647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc032810 con 0x7fced406b5f0 2024-04-25T10:03:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 -- 172.21.15.148:0/107744647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced406b5f0 msgr2=0x7fced40cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 --2- 172.21.15.148:0/107744647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced406b5f0 0x7fced40cc300 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fced40cc840 tx=0x7fcebc02a270 comp rx=0 tx=0).stop 2024-04-25T10:03:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 -- 172.21.15.148:0/107744647 shutdown_connections 2024-04-25T10:03:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 --2- 172.21.15.148:0/107744647 >> v2:172.21.15.188:3300/0 conn(0x7fced40cffb0 0x7fced415c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 --2- 172.21.15.148:0/107744647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced406b5f0 0x7fced40cc300 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 -- 172.21.15.148:0/107744647 >> 172.21.15.148:0/107744647 conn(0x7fced40b2220 msgr2=0x7fced40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 -- 172.21.15.148:0/107744647 shutdown_connections 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.327+0000 7fced99db640 1 -- 172.21.15.148:0/107744647 wait complete. 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.328+0000 7fced99db640 1 Processor -- start 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.328+0000 7fced99db640 1 -- start start 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced99db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fced406b5f0 0x7fced414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced99db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 0x7fced40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced99db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fced415b980 0x7fced415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced99db640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced40cf550 con 0x7fced40cffb0 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced99db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced415bd60 con 0x7fced406b5f0 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced99db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced40b2d40 con 0x7fced415b980 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 0x7fced40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 0x7fced40cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38924/0 (socket says 172.21.15.148:38924) 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 -- 172.21.15.148:0/2004028544 learned_addr learned my addr 172.21.15.148:0/2004028544 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced37fe640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fced415b980 0x7fced415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced2ffd640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fced406b5f0 0x7fced414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 -- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fced415b980 msgr2=0x7fced415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fced415b980 0x7fced415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 -- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fced406b5f0 msgr2=0x7fced414dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fced406b5f0 0x7fced414dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 -- 172.21.15.148:0/2004028544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40b36c0 con 0x7fced40cffb0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fced37fe640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fced415b980 0x7fced415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.329+0000 7fcecbfff640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 0x7fced40cdb50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000c00 tx=0x7fcebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.330+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc041020 con 0x7fced40cffb0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.330+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc0306c0 con 0x7fced40cffb0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.330+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced40b39a0 con 0x7fced40cffb0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.330+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc037040 con 0x7fced40cffb0 2024-04-25T10:03:08.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.330+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced40b3f60 con 0x7fced40cffb0 2024-04-25T10:03:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.331+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced4073990 con 0x7fced40cffb0 2024-04-25T10:03:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.331+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcebc044020 con 0x7fced40cffb0 2024-04-25T10:03:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.332+0000 7fced0ff9640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fceb406b4d0 0x7fceb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.332+0000 7fced2ffd640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fceb406b4d0 0x7fceb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.332+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcebc0aab90 con 0x7fced40cffb0 2024-04-25T10:03:08.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.332+0000 7fced2ffd640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fceb406b4d0 0x7fceb406d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcec4007fd0 tx=0x7fcec4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.336+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcebc04b030 con 0x7fced40cffb0 2024-04-25T10:03:08.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.570+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fced415f450 con 0x7fced40cffb0 2024-04-25T10:03:08.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.570+0000 7fced0ff9640 1 -- 172.21.15.148:0/2004028544 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1445 (secure 0 0 0) 0x7fcebc079880 con 0x7fced40cffb0 2024-04-25T10:03:08.576 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:08.576 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":549,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":984,"data_bytes":1009175586,"bytes_used":2314379264,"bytes_avail":770779734016,"bytes_total":773094113280,"write_bytes_sec":84041726,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":74,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9351}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:02:33.121464+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:08.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fceb406b4d0 msgr2=0x7fceb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fceb406b4d0 0x7fceb406d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcec4007fd0 tx=0x7fcec4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 msgr2=0x7fced40cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 0x7fced40cdb50 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fcebc000c00 tx=0x7fcebc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 shutdown_connections 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fceb406b4d0 0x7fceb406d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fced415b980 0x7fced415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fced40cffb0 0x7fced40cdb50 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 --2- 172.21.15.148:0/2004028544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fced406b5f0 0x7fced414dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 >> 172.21.15.148:0/2004028544 conn(0x7fced40b2220 msgr2=0x7fced40aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 shutdown_connections 2024-04-25T10:03:08.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:08.572+0000 7fced99db640 1 -- 172.21.15.148:0/2004028544 wait complete. 2024-04-25T10:03:08.588 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:03:09.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:09.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.685+0000 7f6a61815640 1 Processor -- start 2024-04-25T10:03:09.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.685+0000 7f6a61815640 1 -- start start 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a61815640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 0x7f6a5c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a61815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b92a0 0x7f6a5c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a61815640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c0b97a0 con 0x7f6a5c0b2cb0 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a61815640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c15dfc0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a61815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d1a00 con 0x7f6a5c0b92a0 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5affd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 0x7f6a5c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b92a0 0x7f6a5c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5affd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 0x7f6a5c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:39386/0 (socket says 172.21.15.148:39386) 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a527fc640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f6a5c0b2cb0 0x7f6a5c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:39236/0 (socket says 172.21.15.148:39236) 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5affd640 1 -- 172.21.15.148:0/474003902 learned_addr learned my addr 172.21.15.148:0/474003902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/474003902 >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 msgr2=0x7f6a5c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5b7fe640 1 --2- 172.21.15.148:0/474003902 >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 0x7f6a5c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/474003902 >> v1:172.21.15.188:6789/0 conn(0x7f6a5c0b2cb0 legacy=0x7f6a5c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5a7fc640 1 -- 172.21.15.148:0/474003902 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044253731 0 0) 0x7f6a5c0b97a0 con 0x7f6a5c0b2cb0 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/474003902 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0d1c50 con 0x7f6a5c0b92a0 2024-04-25T10:03:09.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.686+0000 7f6a5affd640 1 --2- 172.21.15.148:0/474003902 >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 0x7f6a5c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.687+0000 7f6a5b7fe640 1 --2- 172.21.15.148:0/474003902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b92a0 0x7f6a5c0d0e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009d90 tx=0x7f6a5402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5199628413c16074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.687+0000 7f6a5a7fc640 1 -- 172.21.15.148:0/474003902 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007740 con 0x7f6a5c0b92a0 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.687+0000 7f6a5a7fc640 1 -- 172.21.15.148:0/474003902 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540078e0 con 0x7f6a5c0b92a0 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.687+0000 7f6a5a7fc640 1 -- 172.21.15.148:0/474003902 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007cb0 con 0x7f6a5c0b92a0 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 -- 172.21.15.148:0/474003902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b92a0 msgr2=0x7f6a5c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 --2- 172.21.15.148:0/474003902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b92a0 0x7f6a5c0d0e50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009d90 tx=0x7f6a5402a660 comp rx=0 tx=0).stop 2024-04-25T10:03:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 -- 172.21.15.148:0/474003902 shutdown_connections 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 --2- 172.21.15.148:0/474003902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b92a0 0x7f6a5c0d0e50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 --2- 172.21.15.148:0/474003902 >> v2:172.21.15.188:3300/0 conn(0x7f6a5c15dbe0 0x7f6a5c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 -- 172.21.15.148:0/474003902 >> 172.21.15.148:0/474003902 conn(0x7f6a5c061a20 msgr2=0x7f6a5c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 -- 172.21.15.148:0/474003902 shutdown_connections 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.688+0000 7f6a61815640 1 -- 172.21.15.148:0/474003902 wait complete. 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.689+0000 7f6a61815640 1 Processor -- start 2024-04-25T10:03:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.689+0000 7f6a61815640 1 -- start start 2024-04-25T10:03:09.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a61815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b2cb0 0x7f6a5c0b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a61815640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a5c0b92a0 0x7f6a5c1f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a61815640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 0x7f6a5c1f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a61815640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0b6ab0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a61815640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d2210 con 0x7f6a5c0b92a0 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a61815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0b6c30 con 0x7f6a5c0b2cb0 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 0x7f6a5c1f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 0x7f6a5c1f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38928/0 (socket says 172.21.15.148:38928) 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/578840836 learned_addr learned my addr 172.21.15.148:0/578840836 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a527fc640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a5c0b92a0 0x7f6a5c1f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5affd640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b2cb0 0x7f6a5c0b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b2cb0 msgr2=0x7f6a5c0b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b2cb0 0x7f6a5c0b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a5c0b92a0 msgr2=0x7f6a5c1f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a5c0b92a0 0x7f6a5c1f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a5b7fe640 1 -- 172.21.15.148:0/578840836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c15f340 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.690+0000 7f6a527fc640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a5c0b92a0 0x7f6a5c1f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.691+0000 7f6a5b7fe640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 0x7f6a5c1f67f0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009eb0 tx=0x7f6a54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.691+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54040070 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.691+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54031c40 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.691+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54009450 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.691+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c15f620 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.691+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c15fbe0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.692+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c15fde0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.692+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a54039040 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.692+0000 7f6a58ff9640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.693+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a540ab5c0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.693+0000 7f6a5affd640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.693+0000 7f6a5affd640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0b9270 tx=0x7f6a4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.697+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a5c15fde0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.901+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a5c073990 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.901+0000 7f6a58ff9640 1 -- 172.21.15.148:0/578840836 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f6a5407a2b0 con 0x7f6a5c15dbe0 2024-04-25T10:03:09.906 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:09.907 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:09.907 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.903+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a3c06b4d0 msgr2=0x7f6a3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.903+0000 7f6a61815640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0b9270 tx=0x7f6a4c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.903+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 msgr2=0x7f6a5c1f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.903+0000 7f6a61815640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 0x7f6a5c1f67f0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009eb0 tx=0x7f6a54004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 shutdown_connections 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a5c15dbe0 0x7f6a5c1f67f0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a5c0b92a0 0x7f6a5c1f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 --2- 172.21.15.148:0/578840836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a5c0b2cb0 0x7f6a5c0b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 >> 172.21.15.148:0/578840836 conn(0x7f6a5c061a20 msgr2=0x7f6a5c0c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 shutdown_connections 2024-04-25T10:03:09.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:09.904+0000 7f6a61815640 1 -- 172.21.15.148:0/578840836 wait complete. 2024-04-25T10:03:09.919 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:03:09.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:10.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 Processor -- start 2024-04-25T10:03:10.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 -- start start 2024-04-25T10:03:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f243c15da70 0x7f243c0d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f243c0c7730 0x7f243c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c0c8030 con 0x7f243c0ce700 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c0ceb30 con 0x7f243c0c7730 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.010+0000 7f2441f30640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c0c8310 con 0x7f243c15da70 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f243c0c7730 0x7f243c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f243c0c7730 0x7f243c0ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38934/0 (socket says 172.21.15.148:38934) 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 -- 172.21.15.148:0/223151273 learned_addr learned my addr 172.21.15.148:0/223151273 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243b7fe640 1 --2- 172.21.15.148:0/223151273 >> v2:172.21.15.188:3300/0 conn(0x7f243c15da70 0x7f243c0d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 -- 172.21.15.148:0/223151273 >> v2:172.21.15.188:3300/0 conn(0x7f243c15da70 msgr2=0x7f243c0d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 --2- 172.21.15.148:0/223151273 >> v2:172.21.15.188:3300/0 conn(0x7f243c15da70 0x7f243c0d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 -- 172.21.15.148:0/223151273 >> v1:172.21.15.188:6789/0 conn(0x7f243c0ce700 legacy=0x7f243c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.011+0000 7f243affd640 1 -- 172.21.15.148:0/223151273 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0c85f0 con 0x7f243c0c7730 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f243affd640 1 --2- 172.21.15.148:0/223151273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0c7730 0x7f243c0ce1c0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f243c06b900 tx=0x7f243002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364c521f4016969a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f243a7fc640 1 -- 172.21.15.148:0/223151273 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430037070 con 0x7f243c0c7730 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f243a7fc640 1 -- 172.21.15.148:0/223151273 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430032440 con 0x7f243c0c7730 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f243a7fc640 1 -- 172.21.15.148:0/223151273 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430032810 con 0x7f243c0c7730 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 -- 172.21.15.148:0/223151273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0c7730 msgr2=0x7f243c0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 --2- 172.21.15.148:0/223151273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0c7730 0x7f243c0ce1c0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f243c06b900 tx=0x7f243002a230 comp rx=0 tx=0).stop 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 -- 172.21.15.148:0/223151273 shutdown_connections 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 --2- 172.21.15.148:0/223151273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0c7730 0x7f243c0ce1c0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 --2- 172.21.15.148:0/223151273 >> v2:172.21.15.188:3300/0 conn(0x7f243c15da70 0x7f243c0d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 -- 172.21.15.148:0/223151273 >> 172.21.15.148:0/223151273 conn(0x7f243c0b33e0 msgr2=0x7f243c0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 -- 172.21.15.148:0/223151273 shutdown_connections 2024-04-25T10:03:10.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.012+0000 7f2441f30640 1 -- 172.21.15.148:0/223151273 wait complete. 2024-04-25T10:03:10.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.013+0000 7f2441f30640 1 Processor -- start 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 -- start start 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0c7730 0x7f243c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 0x7f243c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c15da70 0x7f243c1f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c15de50 con 0x7f243c0ce700 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c06be10 con 0x7f243c15da70 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f2441f30640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c7bc0 con 0x7f243c0c7730 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f243affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 0x7f243c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f243affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 0x7f243c1609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38940/0 (socket says 172.21.15.148:38940) 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.014+0000 7f243affd640 1 -- 172.21.15.148:0/1573995824 learned_addr learned my addr 172.21.15.148:0/1573995824 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243affd640 1 -- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0c7730 msgr2=0x7f243c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243affd640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0c7730 0x7f243c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243affd640 1 -- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c15da70 msgr2=0x7f243c1f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243bfff640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c15da70 0x7f243c1f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243affd640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c15da70 0x7f243c1f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243affd640 1 -- 172.21.15.148:0/1573995824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0b7180 con 0x7f243c0ce700 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f243affd640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 0x7f243c1609e0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2430000c00 tx=0x7f24300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430041020 con 0x7f243c0ce700 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300306c0 con 0x7f243c0ce700 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430037040 con 0x7f243c0ce700 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c15e6d0 con 0x7f243c0ce700 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.015+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c1f6d70 con 0x7f243c0ce700 2024-04-25T10:03:10.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.016+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c06de80 con 0x7f243c0ce700 2024-04-25T10:03:10.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.016+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2430044020 con 0x7f243c0ce700 2024-04-25T10:03:10.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.017+0000 7f2423fff640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f242806b4d0 0x7f242806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.017+0000 7f243b7fe640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f242806b4d0 0x7f242806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.017+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24300aab30 con 0x7f243c0ce700 2024-04-25T10:03:10.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.017+0000 7f243b7fe640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f242806b4d0 0x7f242806d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f242c006fd0 tx=0x7f242c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.021+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f243004b030 con 0x7f243c0ce700 2024-04-25T10:03:10.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.258+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f243c0cd6b0 con 0x7f243c0ce700 2024-04-25T10:03:10.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.258+0000 7f2423fff640 1 -- 172.21.15.148:0/1573995824 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1445 (secure 0 0 0) 0x7f24300798b0 con 0x7f243c0ce700 2024-04-25T10:03:10.263 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:10.263 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":551,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":984,"data_bytes":1009175586,"bytes_used":2314379264,"bytes_avail":770779734016,"bytes_total":773094113280,"write_bytes_sec":84041489,"read_op_per_sec":0,"write_op_per_sec":82},"fsmap":{"epoch":74,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9351}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:02:33.121464+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f242806b4d0 msgr2=0x7f242806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f242806b4d0 0x7f242806d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f242c006fd0 tx=0x7f242c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 msgr2=0x7f243c1609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 0x7f243c1609e0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2430000c00 tx=0x7f24300046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 shutdown_connections 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f242806b4d0 0x7f242806d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f243c15da70 0x7f243c1f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f243c0ce700 0x7f243c1609e0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 --2- 172.21.15.148:0/1573995824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f243c0c7730 0x7f243c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 >> 172.21.15.148:0/1573995824 conn(0x7f243c0b33e0 msgr2=0x7f243c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 shutdown_connections 2024-04-25T10:03:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:10.260+0000 7f2441f30640 1 -- 172.21.15.148:0/1573995824 wait complete. 2024-04-25T10:03:10.275 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:03:11.277 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:11.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.381+0000 7f0af3129640 1 Processor -- start 2024-04-25T10:03:11.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.381+0000 7f0af3129640 1 -- start start 2024-04-25T10:03:11.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af3129640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec06abe0 0x7f0aec06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af3129640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0aec15d370 0x7f0aec15d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af3129640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aec1626e0 con 0x7f0aec0ce330 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af3129640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aec06b500 con 0x7f0aec15d370 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af3129640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec070270 con 0x7f0aec06abe0 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af16c6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0aec15d370 0x7f0aec15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0aebfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0aec0ce330 0x7f0aec06fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33746/0 (socket says 172.21.15.148:33746) 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0af16c6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0aec15d370 0x7f0aec15d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38954/0 (socket says 172.21.15.148:38954) 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.382+0000 7f0aebfff640 1 -- 172.21.15.148:0/742145716 learned_addr learned my addr 172.21.15.148:0/742145716 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af16c6640 1 -- 172.21.15.148:0/742145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec06abe0 msgr2=0x7f0aec06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af0ec5640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec06abe0 0x7f0aec06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af16c6640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec06abe0 0x7f0aec06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0aeb7fe640 1 -- 172.21.15.148:0/742145716 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292588805 0 0) 0x7f0aec1626e0 con 0x7f0aec0ce330 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af16c6640 1 -- 172.21.15.148:0/742145716 >> v1:172.21.15.148:6789/0 conn(0x7f0aec0ce330 legacy=0x7f0aec06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af16c6640 1 -- 172.21.15.148:0/742145716 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec0cd270 con 0x7f0aec15d370 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af0ec5640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec06abe0 0x7f0aec06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0af16c6640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec15d370 0x7f0aec15d800 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009d90 tx=0x7f0ae402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cc6a29f16cf6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.383+0000 7f0aeb7fe640 1 -- 172.21.15.148:0/742145716 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4031600 con 0x7f0aec15d370 2024-04-25T10:03:11.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0aeb7fe640 1 -- 172.21.15.148:0/742145716 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae40317a0 con 0x7f0aec15d370 2024-04-25T10:03:11.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0aeb7fe640 1 -- 172.21.15.148:0/742145716 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4031b70 con 0x7f0aec15d370 2024-04-25T10:03:11.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 -- 172.21.15.148:0/742145716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec15d370 msgr2=0x7f0aec15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec15d370 0x7f0aec15d800 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009d90 tx=0x7f0ae402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 -- 172.21.15.148:0/742145716 shutdown_connections 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec15d370 0x7f0aec15d800 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 --2- 172.21.15.148:0/742145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec06abe0 0x7f0aec06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 -- 172.21.15.148:0/742145716 >> 172.21.15.148:0/742145716 conn(0x7f0aec0b2220 msgr2=0x7f0aec0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 -- 172.21.15.148:0/742145716 shutdown_connections 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.384+0000 7f0af3129640 1 -- 172.21.15.148:0/742145716 wait complete. 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.385+0000 7f0af3129640 1 Processor -- start 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.385+0000 7f0af3129640 1 -- start start 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0af3129640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aec06abe0 0x7f0aec0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0af3129640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 0x7f0aec157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0af3129640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec15d370 0x7f0aec0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0af3129640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec0cdb10 con 0x7f0aec0ce330 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0af3129640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec0cd830 con 0x7f0aec06abe0 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0af3129640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec161780 con 0x7f0aec15d370 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.386+0000 7f0aebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 0x7f0aec157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 0x7f0aec157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38960/0 (socket says 172.21.15.148:38960) 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 -- 172.21.15.148:0/2738834334 learned_addr learned my addr 172.21.15.148:0/2738834334 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0af0ec5640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aec06abe0 0x7f0aec0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0af16c6640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec15d370 0x7f0aec0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 -- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec15d370 msgr2=0x7f0aec0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec15d370 0x7f0aec0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 -- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aec06abe0 msgr2=0x7f0aec0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aec06abe0 0x7f0aec0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 -- 172.21.15.148:0/2738834334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec1ea0a0 con 0x7f0aec0ce330 2024-04-25T10:03:11.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0af16c6640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec15d370 0x7f0aec0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0aebfff640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 0x7f0aec157fb0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c4d0 tx=0x7f0ae000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0018070 con 0x7f0aec0ce330 2024-04-25T10:03:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0007500 con 0x7f0aec0ce330 2024-04-25T10:03:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aec1ea380 con 0x7f0aec0ce330 2024-04-25T10:03:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0004a20 con 0x7f0aec0ce330 2024-04-25T10:03:11.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.387+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aec159010 con 0x7f0aec0ce330 2024-04-25T10:03:11.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.388+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aec06e050 con 0x7f0aec0ce330 2024-04-25T10:03:11.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.389+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ae001b070 con 0x7f0aec0ce330 2024-04-25T10:03:11.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.389+0000 7f0ae97fa640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0ad806b4d0 0x7f0ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.389+0000 7f0af0ec5640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0ad806b4d0 0x7f0ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.389+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ae0089600 con 0x7f0aec0ce330 2024-04-25T10:03:11.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.390+0000 7f0af0ec5640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0ad806b4d0 0x7f0ad806d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00aac0 tx=0x7f0adc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.394+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ae0057ca0 con 0x7f0aec0ce330 2024-04-25T10:03:11.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.603+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0aec0b6670 con 0x7f0aec0ce330 2024-04-25T10:03:11.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.604+0000 7f0ae97fa640 1 -- 172.21.15.148:0/2738834334 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f0ae005bd40 con 0x7f0aec0ce330 2024-04-25T10:03:11.609 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:11.609 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:11.609 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:11.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.605+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0ad806b4d0 msgr2=0x7f0ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.605+0000 7f0af3129640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0ad806b4d0 0x7f0ad806d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00aac0 tx=0x7f0adc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.605+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 msgr2=0x7f0aec157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.605+0000 7f0af3129640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 0x7f0aec157fb0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c4d0 tx=0x7f0ae000c990 comp rx=0 tx=0).stop 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 shutdown_connections 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0ad806b4d0 0x7f0ad806d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aec15d370 0x7f0aec0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aec0ce330 0x7f0aec157fb0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 --2- 172.21.15.148:0/2738834334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aec06abe0 0x7f0aec0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 >> 172.21.15.148:0/2738834334 conn(0x7f0aec0b2220 msgr2=0x7f0aec0ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 shutdown_connections 2024-04-25T10:03:11.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.606+0000 7f0af3129640 1 -- 172.21.15.148:0/2738834334 wait complete. 2024-04-25T10:03:11.621 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:03:11.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:11.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 Processor -- start 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 -- start start 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff1a415dbe0 0x7ff1a40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40b92a0 0x7ff1a40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a40b97a0 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a415dfc0 con 0x7ff1a415dbe0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.713+0000 7ff1a9ae2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40d1a00 con 0x7ff1a40b92a0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff1a415dbe0 0x7ff1a40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff19bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff1a40b2cb0 0x7ff1a40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33750/0 (socket says 172.21.15.148:33750) 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff19bfff640 1 -- 172.21.15.148:0/3997381464 learned_addr learned my addr 172.21.15.148:0/3997381464 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3997381464 >> v2:172.21.15.148:3300/0 conn(0x7ff1a415dbe0 0x7ff1a40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38974/0 (socket says 172.21.15.148:38974) 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3997381464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40b92a0 msgr2=0x7ff1a40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3997381464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40b92a0 0x7ff1a40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3997381464 >> v1:172.21.15.148:6789/0 conn(0x7ff1a40b2cb0 legacy=0x7ff1a40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a37fe640 1 -- 172.21.15.148:0/3997381464 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40d1c50 con 0x7ff1a415dbe0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.714+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3997381464 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257506485 0 0) 0x7ff1a40b97a0 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/3997381464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a415dbe0 0x7ff1a40d0910 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097a0 tx=0x7ff19402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cedd43a1059a7bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3997381464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194037070 con 0x7ff1a415dbe0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3997381464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194032440 con 0x7ff1a415dbe0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a2ffd640 1 -- 172.21.15.148:0/3997381464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194032810 con 0x7ff1a415dbe0 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/3997381464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a415dbe0 msgr2=0x7ff1a40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/3997381464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a415dbe0 0x7ff1a40d0910 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff1940097a0 tx=0x7ff19402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/3997381464 shutdown_connections 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/3997381464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a40b92a0 0x7ff1a40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/3997381464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a415dbe0 0x7ff1a40d0910 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/3997381464 >> 172.21.15.148:0/3997381464 conn(0x7ff1a4061a20 msgr2=0x7ff1a4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/3997381464 shutdown_connections 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.715+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/3997381464 wait complete. 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.716+0000 7ff1a9ae2640 1 Processor -- start 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.716+0000 7ff1a9ae2640 1 -- start start 2024-04-25T10:03:11.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a9ae2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 0x7ff1a40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a9ae2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a40b92a0 0x7ff1a40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a9ae2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415dbe0 0x7ff1a41f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a9ae2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40b6ab0 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a9ae2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40d2210 con 0x7ff1a40b92a0 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a9ae2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40b6c30 con 0x7ff1a415dbe0 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 0x7ff1a40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 0x7ff1a40b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:38982/0 (socket says 172.21.15.148:38982) 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 -- 172.21.15.148:0/701848383 learned_addr learned my addr 172.21.15.148:0/701848383 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 -- 172.21.15.148:0/701848383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415dbe0 msgr2=0x7ff1a41f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415dbe0 0x7ff1a41f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 -- 172.21.15.148:0/701848383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a40b92a0 msgr2=0x7ff1a40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a40b92a0 0x7ff1a40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.717+0000 7ff1a37fe640 1 -- 172.21.15.148:0/701848383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40b4c80 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.718+0000 7ff1a37fe640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 0x7ff1a40b4200 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff1940098c0 tx=0x7ff1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.718+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194041070 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.718+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19403c870 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.718+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194037040 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.718+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a40b4f30 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.718+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a41f4ad0 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.719+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4073990 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.719+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff194044070 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.720+0000 7ff1a0ff9640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff18406b4d0 0x7ff18406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.720+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1940ab070 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.720+0000 7ff19bfff640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff18406b4d0 0x7ff18406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.720+0000 7ff19bfff640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff18406b4d0 0x7ff18406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff18c007fd0 tx=0x7ff18c007920 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.724+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff194079f10 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.962+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff1a40b27b0 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.963+0000 7ff1a0ff9640 1 -- 172.21.15.148:0/701848383 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7ff194079810 con 0x7ff1a40b2cb0 2024-04-25T10:03:11.968 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:11.968 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":553,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2569732096,"bytes_avail":770524381184,"bytes_total":773094113280,"write_bytes_sec":89657305,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":74,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9351}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T10:03:11.128718+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff18406b4d0 msgr2=0x7ff18406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff18406b4d0 0x7ff18406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff18c007fd0 tx=0x7ff18c007920 comp rx=0 tx=0).stop 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 msgr2=0x7ff1a40b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 0x7ff1a40b4200 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff1940098c0 tx=0x7ff1940046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 shutdown_connections 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff18406b4d0 0x7ff18406d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a415dbe0 0x7ff1a41f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a40b92a0 0x7ff1a40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 --2- 172.21.15.148:0/701848383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1a40b2cb0 0x7ff1a40b4200 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 >> 172.21.15.148:0/701848383 conn(0x7ff1a4061a20 msgr2=0x7ff1a40d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 shutdown_connections 2024-04-25T10:03:11.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:11.965+0000 7ff1a9ae2640 1 -- 172.21.15.148:0/701848383 wait complete. 2024-04-25T10:03:11.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:03:11.981 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:11.981 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:11.993 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:11.993 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'b', 'd', 'c'} 2024-04-25T10:03:11.994 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T10:03:12.138 INFO:teuthology.orchestra.run.smithi148.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T10:03:12.138 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T10:03:12.138 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:12.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.231+0000 7ff9fd037640 1 Processor -- start 2024-04-25T10:03:12.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.231+0000 7ff9fd037640 1 -- start start 2024-04-25T10:03:12.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9fd037640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9f8154ed0 0x7ff9f80aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9fd037640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f815ec60 con 0x7ff9f8071ac0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9fd037640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f815ef40 con 0x7ff9f806c1d0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9fd037640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f806db90 con 0x7ff9f8154ed0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9effff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff9f8071ac0 0x7ff9f80ad410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33754/0 (socket says 172.21.15.148:33754) 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9effff640 1 -- 172.21.15.148:0/1040744674 learned_addr learned my addr 172.21.15.148:0/1040744674 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f6d9d640 1 --2- 172.21.15.148:0/1040744674 >> v2:172.21.15.188:3300/0 conn(0x7ff9f8154ed0 0x7ff9f80aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245293305 0 0) 0x7ff9f815ec60 con 0x7ff9f8071ac0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e4003520 con 0x7ff9f8071ac0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3049999573 0 0) 0x7ff9e4003520 con 0x7ff9f8071ac0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 >> v2:172.21.15.188:3300/0 conn(0x7ff9f8154ed0 msgr2=0x7ff9f80aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 --2- 172.21.15.148:0/1040744674 >> v2:172.21.15.188:3300/0 conn(0x7ff9f8154ed0 0x7ff9f80aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 >> v1:172.21.15.188:6789/0 conn(0x7ff9f806c1d0 legacy=0x7ff9f806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f806de70 con 0x7ff9f8071ac0 2024-04-25T10:03:12.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff9e0001670 con 0x7ff9f8071ac0 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 >> v1:172.21.15.148:6789/0 conn(0x7ff9f8071ac0 legacy=0x7ff9f80ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9e40039b0 0x7ff9e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9e4004800 0x7ff9e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9e40051c0 0x7ff9e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f806db90 con 0x7ff9e40051c0 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e400c160 con 0x7ff9e4004800 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.232+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e400c360 con 0x7ff9e40039b0 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9e40051c0 0x7ff9e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f6d9d640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9e40039b0 0x7ff9e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9effff640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9e4004800 0x7ff9e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 -- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9e40039b0 msgr2=0x7ff9e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9e40039b0 0x7ff9e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 -- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9e4004800 msgr2=0x7ff9e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9e4004800 0x7ff9e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 -- 172.21.15.148:0/1040744674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e4003520 con 0x7ff9e40051c0 2024-04-25T10:03:12.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.233+0000 7ff9f759e640 1 -- 172.21.15.148:0/1040744674 reap_dead start 2024-04-25T10:03:12.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9f759e640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9e40051c0 0x7ff9e400b9c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0001e50 tx=0x7ff9e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ba1d7db2e463a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0001a70 con 0x7ff9e40051c0 2024-04-25T10:03:12.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0003970 con 0x7ff9e40051c0 2024-04-25T10:03:12.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9f659c640 1 -- 172.21.15.148:0/1040744674 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0003d60 con 0x7ff9e40051c0 2024-04-25T10:03:12.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 -- 172.21.15.148:0/1040744674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9e40051c0 msgr2=0x7ff9e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9e40051c0 0x7ff9e400b9c0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0001e50 tx=0x7ff9e4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:12.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 -- 172.21.15.148:0/1040744674 shutdown_connections 2024-04-25T10:03:12.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 --2- 172.21.15.148:0/1040744674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9e40051c0 0x7ff9e400b9c0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 -- 172.21.15.148:0/1040744674 >> 172.21.15.148:0/1040744674 conn(0x7ff9f80af670 msgr2=0x7ff9f80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 -- 172.21.15.148:0/1040744674 shutdown_connections 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.234+0000 7ff9fd037640 1 -- 172.21.15.148:0/1040744674 wait complete. 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 Processor -- start 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 -- start start 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9f814ea00 0x7ff9f80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9f81f3340 0x7ff9f80b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 0x7ff9f8153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80ad260 con 0x7ff9f8152eb0 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80ad0e0 con 0x7ff9f814ea00 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9fd037640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80ad3e0 con 0x7ff9f81f3340 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9f759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 0x7ff9f8153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9f759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 0x7ff9f8153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39002/0 (socket says 172.21.15.148:39002) 2024-04-25T10:03:12.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.236+0000 7ff9f759e640 1 -- 172.21.15.148:0/2057222878 learned_addr learned my addr 172.21.15.148:0/2057222878 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f6d9d640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9f814ea00 0x7ff9f80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f759e640 1 -- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9f81f3340 msgr2=0x7ff9f80b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9effff640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9f81f3340 0x7ff9f80b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f759e640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9f81f3340 0x7ff9f80b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f759e640 1 -- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9f814ea00 msgr2=0x7ff9f80b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f759e640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9f814ea00 0x7ff9f80b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f759e640 1 -- 172.21.15.148:0/2057222878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80b5380 con 0x7ff9f8152eb0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f6d9d640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9f814ea00 0x7ff9f80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9f759e640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 0x7ff9f8153330 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff9f002a920 tx=0x7ff9f00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0001a70 con 0x7ff9f8152eb0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f002ac70 con 0x7ff9f8152eb0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f81564b0 con 0x7ff9f8152eb0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0006d70 con 0x7ff9f8152eb0 2024-04-25T10:03:12.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.237+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f8156a70 con 0x7ff9f8152eb0 2024-04-25T10:03:12.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.238+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f80b2b10 con 0x7ff9f8152eb0 2024-04-25T10:03:12.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.238+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9f0007030 con 0x7ff9f8152eb0 2024-04-25T10:03:12.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.239+0000 7ff9ef7fe640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9d806b4d0 0x7ff9d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.239+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9f00a9f20 con 0x7ff9f8152eb0 2024-04-25T10:03:12.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.239+0000 7ff9f6d9d640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9d806b4d0 0x7ff9d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.239+0000 7ff9f6d9d640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9d806b4d0 0x7ff9d806d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009a40 tx=0x7ff9e8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.243+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9f0043060 con 0x7ff9f8152eb0 2024-04-25T10:03:12.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.450+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9f8073990 con 0x7ff9f8152eb0 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.450+0000 7ff9ef7fe640 1 -- 172.21.15.148:0/2057222878 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ff9f0078c00 con 0x7ff9f8152eb0 2024-04-25T10:03:12.455 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:12.456 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:12.456 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9d806b4d0 msgr2=0x7ff9d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9d806b4d0 0x7ff9d806d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009a40 tx=0x7ff9e8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 msgr2=0x7ff9f8153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 0x7ff9f8153330 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff9f002a920 tx=0x7ff9f00014a0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 shutdown_connections 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff9d806b4d0 0x7ff9d806d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9f8152eb0 0x7ff9f8153330 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9f81f3340 0x7ff9f80b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 --2- 172.21.15.148:0/2057222878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9f814ea00 0x7ff9f80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 >> 172.21.15.148:0/2057222878 conn(0x7ff9e40051c0 msgr2=0x7ff9f814f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 shutdown_connections 2024-04-25T10:03:12.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:12.452+0000 7ff9fd037640 1 -- 172.21.15.148:0/2057222878 wait complete. 2024-04-25T10:03:12.469 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:03:17.470 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:17.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.573+0000 7f92e5dbe640 1 Processor -- start 2024-04-25T10:03:17.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.573+0000 7f92e5dbe640 1 -- start start 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92e5dbe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 0x7f92e00c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92e5dbe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92e00d1dd0 0x7f92e00ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92e5dbe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e015dfe0 con 0x7f92e00ceb20 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92e5dbe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e00d00c0 con 0x7f92e00d1dd0 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92e5dbe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00d03a0 con 0x7f92e015dc00 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92deffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f92e00ceb20 0x7f92e00d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47764/0 (socket says 172.21.15.148:47764) 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92deffd640 1 -- 172.21.15.148:0/2263724742 learned_addr learned my addr 172.21.15.148:0/2263724742 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:17.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553295268 0 0) 0x7f92d4001110 con 0x7f92e00ceb20 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c8003520 con 0x7f92e00ceb20 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92df7fe640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 0x7f92e00c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92dffff640 1 --2- 172.21.15.148:0/2263724742 >> v2:172.21.15.188:3300/0 conn(0x7f92e00d1dd0 0x7f92e00ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865959730 0 0) 0x7f92c8003520 con 0x7f92e00ceb20 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 msgr2=0x7f92e00c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 0x7f92e00c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 >> v2:172.21.15.188:3300/0 conn(0x7f92e00d1dd0 msgr2=0x7f92e00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 --2- 172.21.15.148:0/2263724742 >> v2:172.21.15.188:3300/0 conn(0x7f92e00d1dd0 0x7f92e00ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00cc440 con 0x7f92e00ceb20 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.574+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f92d4001840 con 0x7f92e00ceb20 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 >> v1:172.21.15.148:6789/0 conn(0x7f92e00ceb20 legacy=0x7f92e00d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c80039b0 0x7f92c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 0x7f92c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00d00c0 con 0x7f92c8004800 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00d03a0 con 0x7f92c80039b0 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c800b9c0 con 0x7f92e015dc00 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92deffd640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92df7fe640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c80039b0 0x7f92c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92dffff640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 0x7f92c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92deffd640 1 -- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 msgr2=0x7f92c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92deffd640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e015dc00 0x7f92c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92deffd640 1 -- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c80039b0 msgr2=0x7f92c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92deffd640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92c80039b0 0x7f92c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.575+0000 7f92deffd640 1 -- 172.21.15.148:0/2263724742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8003520 con 0x7f92c8004800 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92deffd640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92c8004c80 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c4c0 tx=0x7f92c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f90be14d13e3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4032020 con 0x7f92c8004800 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4005ad0 con 0x7f92c8004800 2024-04-25T10:03:17.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92de7fc640 1 -- 172.21.15.148:0/2263724742 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400c0e0 con 0x7f92c8004800 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92e5dbe640 1 -- 172.21.15.148:0/2263724742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 msgr2=0x7f92c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/2263724742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92c8004c80 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c4c0 tx=0x7f92c8005830 comp rx=0 tx=0).stop 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92dffff640 1 -- 172.21.15.148:0/2263724742 reap_dead start 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92e5dbe640 1 -- 172.21.15.148:0/2263724742 shutdown_connections 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92e5dbe640 1 -- 172.21.15.148:0/2263724742 >> 172.21.15.148:0/2263724742 conn(0x7f92e006f5d0 msgr2=0x7f92e006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92e5dbe640 1 -- 172.21.15.148:0/2263724742 shutdown_connections 2024-04-25T10:03:17.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.576+0000 7f92e5dbe640 1 -- 172.21.15.148:0/2263724742 wait complete. 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.578+0000 7f92e5dbe640 1 Processor -- start 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.578+0000 7f92e5dbe640 1 -- start start 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92e00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e00b6fc0 0x7f92e0152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e01f43b0 0x7f92e00b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00cfd90 con 0x7f92c8004800 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00cc130 con 0x7f92e00b6fc0 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00cdd70 con 0x7f92e01f43b0 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92e00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92e00b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41412/0 (socket says 172.21.15.148:41412) 2024-04-25T10:03:17.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 -- 172.21.15.148:0/1135090760 learned_addr learned my addr 172.21.15.148:0/1135090760 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92dffff640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e01f43b0 0x7f92e00b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92deffd640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e00b6fc0 0x7f92e0152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 -- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e01f43b0 msgr2=0x7f92e00b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e01f43b0 0x7f92e00b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 -- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e00b6fc0 msgr2=0x7f92e0152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e00b6fc0 0x7f92e0152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 -- 172.21.15.148:0/1135090760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0155790 con 0x7f92c8004800 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92df7fe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92e00b4630 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f92d0011910 tx=0x7f92d0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000faf0 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d000fc90 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e0154300 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0014440 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.579+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e0153d40 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.580+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e0073990 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.581+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92d0012040 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.581+0000 7f92c7fff640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f92cc06b500 0x7f92cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.581+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92d008a840 con 0x7f92c8004800 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.581+0000 7f92deffd640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f92cc06b500 0x7f92cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.582+0000 7f92deffd640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f92cc06b500 0x7f92cc06d9c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c490 tx=0x7f92d402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.586+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92d0019030 con 0x7f92c8004800 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.793+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92e00b7660 con 0x7f92c8004800 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.793+0000 7f92c7fff640 1 -- 172.21.15.148:0/1135090760 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f92d00595c0 con 0x7f92c8004800 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:17.800 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f92cc06b500 msgr2=0x7f92cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f92cc06b500 0x7f92cc06d9c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c490 tx=0x7f92d402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 msgr2=0x7f92e00b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92e00b4630 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f92d0011910 tx=0x7f92d0011dd0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 shutdown_connections 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f92cc06b500 0x7f92cc06d9c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92e01f43b0 0x7f92e00b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92e00b6fc0 0x7f92e0152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.795+0000 7f92e5dbe640 1 --2- 172.21.15.148:0/1135090760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92c8004800 0x7f92e00b4630 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.796+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 >> 172.21.15.148:0/1135090760 conn(0x7f92e006f5d0 msgr2=0x7f92e00c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.796+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 shutdown_connections 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:17.796+0000 7f92e5dbe640 1 -- 172.21.15.148:0/1135090760 wait complete. 2024-04-25T10:03:17.813 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:03:22.814 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:22.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.918+0000 7f540d05d640 1 Processor -- start 2024-04-25T10:03:22.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540d05d640 1 -- start start 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540d05d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f540815dc90 0x7f54081625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540d05d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54080d1810 0x7f54080d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540d05d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54080cce50 con 0x7f54080cc8e0 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540d05d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54080d21c0 con 0x7f54080d1810 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540d05d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5408161af0 con 0x7f540815dc90 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54080d1810 0x7f54080d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.919+0000 7f540759e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f54080d1810 0x7f54080d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41424/0 (socket says 172.21.15.148:41424) 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.920+0000 7f540759e640 1 -- 172.21.15.148:0/116357896 learned_addr learned my addr 172.21.15.148:0/116357896 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:22.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.920+0000 7f540759e640 1 -- 172.21.15.148:0/116357896 >> v2:172.21.15.188:3300/0 conn(0x7f540815dc90 msgr2=0x7f54081625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.920+0000 7f540759e640 1 --2- 172.21.15.148:0/116357896 >> v2:172.21.15.188:3300/0 conn(0x7f540815dc90 0x7f54081625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.920+0000 7f540759e640 1 -- 172.21.15.148:0/116357896 >> v1:172.21.15.188:6789/0 conn(0x7f54080cc8e0 legacy=0x7f540815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.920+0000 7f540759e640 1 -- 172.21.15.148:0/116357896 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408161dd0 con 0x7f54080d1810 2024-04-25T10:03:22.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.920+0000 7f540759e640 1 --2- 172.21.15.148:0/116357896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080d1810 0x7f54080d1c80 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f540806b820 tx=0x7f540002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ba406d2ccfb39a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f5405d9b640 1 -- 172.21.15.148:0/116357896 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400036070 con 0x7f54080d1810 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f5405d9b640 1 -- 172.21.15.148:0/116357896 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400031600 con 0x7f54080d1810 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f5405d9b640 1 -- 172.21.15.148:0/116357896 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54000319d0 con 0x7f54080d1810 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 -- 172.21.15.148:0/116357896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080d1810 msgr2=0x7f54080d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 --2- 172.21.15.148:0/116357896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080d1810 0x7f54080d1c80 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f540806b820 tx=0x7f540002a660 comp rx=0 tx=0).stop 2024-04-25T10:03:22.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 -- 172.21.15.148:0/116357896 shutdown_connections 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 --2- 172.21.15.148:0/116357896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080d1810 0x7f54080d1c80 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 --2- 172.21.15.148:0/116357896 >> v2:172.21.15.188:3300/0 conn(0x7f540815dc90 0x7f54081625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 -- 172.21.15.148:0/116357896 >> 172.21.15.148:0/116357896 conn(0x7f540806f590 msgr2=0x7f540806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 -- 172.21.15.148:0/116357896 shutdown_connections 2024-04-25T10:03:22.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.921+0000 7f540d05d640 1 -- 172.21.15.148:0/116357896 wait complete. 2024-04-25T10:03:22.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.923+0000 7f540d05d640 1 Processor -- start 2024-04-25T10:03:22.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.923+0000 7f540d05d640 1 -- start start 2024-04-25T10:03:22.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54080cc8e0 0x7f540814bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54080d1810 0x7f54080ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 0x7f54080aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540815e070 con 0x7f540815dc90 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080cd130 con 0x7f54080cc8e0 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540815e1f0 con 0x7f54080d1810 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 0x7f54080aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 0x7f54080aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41426/0 (socket says 172.21.15.148:41426) 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 -- 172.21.15.148:0/1447719617 learned_addr learned my addr 172.21.15.148:0/1447719617 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 -- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54080d1810 msgr2=0x7f54080ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f5406d9d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54080cc8e0 0x7f540814bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540659c640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54080d1810 0x7f54080ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54080d1810 0x7f54080ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 -- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54080cc8e0 msgr2=0x7f540814bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54080cc8e0 0x7f540814bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 -- 172.21.15.148:0/1447719617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080acb40 con 0x7f540815dc90 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f5406d9d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54080cc8e0 0x7f540814bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540659c640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54080d1810 0x7f54080ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540759e640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 0x7f54080aff40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5400009f40 tx=0x7f5400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400041020 con 0x7f540815dc90 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54000077a0 con 0x7f540815dc90 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54080ace20 con 0x7f540815dc90 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5400036040 con 0x7f540815dc90 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.924+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54080ad3e0 con 0x7f540815dc90 2024-04-25T10:03:22.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.925+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540806de50 con 0x7f540815dc90 2024-04-25T10:03:22.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.926+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5400044020 con 0x7f540815dc90 2024-04-25T10:03:22.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.926+0000 7f53ff7fe640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e806b4d0 0x7f53e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.926+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54000ab090 con 0x7f540815dc90 2024-04-25T10:03:22.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.926+0000 7f5406d9d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e806b4d0 0x7f53e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.927+0000 7f5406d9d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e806b4d0 0x7f53e806d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f53f000a870 tx=0x7f53f0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:22.931+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f540004b030 con 0x7f540815dc90 2024-04-25T10:03:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.136+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5408073990 con 0x7f540815dc90 2024-04-25T10:03:23.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.136+0000 7f53ff7fe640 1 -- 172.21.15.148:0/1447719617 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f5400079e40 con 0x7f540815dc90 2024-04-25T10:03:23.142 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:23.142 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:23.142 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:23.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.138+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e806b4d0 msgr2=0x7f53e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e806b4d0 0x7f53e806d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f53f000a870 tx=0x7f53f0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:23.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 msgr2=0x7f54080aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 0x7f54080aff40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5400009f40 tx=0x7f5400004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 shutdown_connections 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53e806b4d0 0x7f53e806d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f540815dc90 0x7f54080aff40 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54080d1810 0x7f54080ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 --2- 172.21.15.148:0/1447719617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54080cc8e0 0x7f540814bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 >> 172.21.15.148:0/1447719617 conn(0x7f540806f590 msgr2=0x7f540815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 shutdown_connections 2024-04-25T10:03:23.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:23.139+0000 7f540d05d640 1 -- 172.21.15.148:0/1447719617 wait complete. 2024-04-25T10:03:23.155 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:03:25.170 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:25.173 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:28.156 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.260+0000 7ff1076bd640 1 Processor -- start 2024-04-25T10:03:28.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.260+0000 7ff1076bd640 1 -- start start 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.261+0000 7ff1076bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 0x7ff10006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.261+0000 7ff1076bd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1000ace70 con 0x7ff1000720f0 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.261+0000 7ff1076bd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1000b4390 con 0x7ff1000b3410 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.261+0000 7ff1076bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10006df30 con 0x7ff1000b3f00 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.261+0000 7ff104c58640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff1000720f0 0x7ff10006e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:38682/0 (socket says 172.21.15.148:38682) 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.261+0000 7ff104c58640 1 -- 172.21.15.148:0/2661754186 learned_addr learned my addr 172.21.15.148:0/2661754186 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394806308 0 0) 0x7ff0f0001110 con 0x7ff1000720f0 2024-04-25T10:03:28.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ec003520 con 0x7ff1000720f0 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 0x7ff10006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1250735942 0 0) 0x7ff0ec003520 con 0x7ff1000720f0 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 msgr2=0x7ff10006ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 0x7ff10006ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 >> v1:172.21.15.188:6789/0 conn(0x7ff1000b3410 legacy=0x7ff1000b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000726d0 con 0x7ff1000720f0 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff0f0001840 con 0x7ff1000720f0 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 >> v1:172.21.15.148:6789/0 conn(0x7ff1000720f0 legacy=0x7ff10006e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec0039e0 0x7ff0ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 0x7ff0ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0ec004d40 0x7ff0ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10006df30 con 0x7ff0ec004d40 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00bb20 con 0x7ff0ec0039e0 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec00bd20 con 0x7ff1000b3f00 2024-04-25T10:03:28.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0ec004d40 0x7ff0ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105459640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec0039e0 0x7ff0ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff104c58640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 0x7ff0ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 -- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 msgr2=0x7ff0ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1000b3f00 0x7ff0ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 -- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec0039e0 msgr2=0x7ff0ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec0039e0 0x7ff0ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.262+0000 7ff105c5a640 1 -- 172.21.15.148:0/2661754186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec003520 con 0x7ff0ec004d40 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.263+0000 7ff105c5a640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0ec004d40 0x7ff0ec0051c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ac0 tx=0x7ff0ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bc1ec9e02614a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.263+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8036070 con 0x7ff0ec004d40 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.263+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8031400 con 0x7ff0ec004d40 2024-04-25T10:03:28.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.263+0000 7ff0f7fff640 1 -- 172.21.15.148:0/2661754186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8009690 con 0x7ff0ec004d40 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff1076bd640 1 -- 172.21.15.148:0/2661754186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0ec004d40 msgr2=0x7ff0ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff1076bd640 1 --2- 172.21.15.148:0/2661754186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0ec004d40 0x7ff0ec0051c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009ac0 tx=0x7ff0ec005830 comp rx=0 tx=0).stop 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff105c5a640 1 -- 172.21.15.148:0/2661754186 reap_dead start 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff1076bd640 1 -- 172.21.15.148:0/2661754186 shutdown_connections 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff1076bd640 1 -- 172.21.15.148:0/2661754186 >> 172.21.15.148:0/2661754186 conn(0x7ff1000cfae0 msgr2=0x7ff1000cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff1076bd640 1 -- 172.21.15.148:0/2661754186 shutdown_connections 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.264+0000 7ff1076bd640 1 -- 172.21.15.148:0/2661754186 wait complete. 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.265+0000 7ff1076bd640 1 Processor -- start 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.265+0000 7ff1076bd640 1 -- start start 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff1076bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec004d40 0x7ff10013c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff1076bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 0x7ff100145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff1076bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100160d90 0x7ff100161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff1076bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100147890 con 0x7ff100145300 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff1076bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10006dbc0 con 0x7ff0ec004d40 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff1076bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff100147230 con 0x7ff100160d90 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 0x7ff100145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 0x7ff100145760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33468/0 (socket says 172.21.15.148:33468) 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 -- 172.21.15.148:0/10415001 learned_addr learned my addr 172.21.15.148:0/10415001 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff105c5a640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100160d90 0x7ff100161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff105459640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec004d40 0x7ff10013c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 -- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100160d90 msgr2=0x7ff100161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100160d90 0x7ff100161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 -- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec004d40 msgr2=0x7ff10013c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec004d40 0x7ff10013c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.266+0000 7ff104c58640 1 -- 172.21.15.148:0/10415001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10013cb10 con 0x7ff100145300 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.267+0000 7ff104c58640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 0x7ff100145760 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0007290 tx=0x7ff0f0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.267+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000f960 con 0x7ff100145300 2024-04-25T10:03:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.267+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f0018070 con 0x7ff100145300 2024-04-25T10:03:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.267+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff100161ac0 con 0x7ff100145300 2024-04-25T10:03:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.267+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000fd20 con 0x7ff100145300 2024-04-25T10:03:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.267+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff100162080 con 0x7ff100145300 2024-04-25T10:03:28.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.268+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff100073990 con 0x7ff100145300 2024-04-25T10:03:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.268+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0f001c410 con 0x7ff100145300 2024-04-25T10:03:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.269+0000 7ff0f5ffb640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0e406b500 0x7ff0e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.269+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0f0089ef0 con 0x7ff100145300 2024-04-25T10:03:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.269+0000 7ff105459640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0e406b500 0x7ff0e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.269+0000 7ff105459640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0e406b500 0x7ff0e406d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001ee0 tx=0x7ff0fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.273+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff10013cb10 con 0x7ff100145300 2024-04-25T10:03:28.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.477+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1000d0800 con 0x7ff100145300 2024-04-25T10:03:28.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.477+0000 7ff0f5ffb640 1 -- 172.21.15.148:0/10415001 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ff1000ace70 con 0x7ff100145300 2024-04-25T10:03:28.483 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:28.483 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:02:51.165263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9351},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9351":{"gid":9351,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1740205817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1740205817},{"type":"v1","addr":"172.21.15.148:6837","nonce":1740205817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:28.483 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 74 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0e406b500 msgr2=0x7ff0e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0e406b500 0x7ff0e406d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001ee0 tx=0x7ff0fc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 msgr2=0x7ff100145760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 0x7ff100145760 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0007290 tx=0x7ff0f0007750 comp rx=0 tx=0).stop 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 shutdown_connections 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff0e406b500 0x7ff0e406d9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100160d90 0x7ff100161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff100145300 0x7ff100145760 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 --2- 172.21.15.148:0/10415001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0ec004d40 0x7ff10013c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 >> 172.21.15.148:0/10415001 conn(0x7ff1000cfae0 msgr2=0x7ff1000cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 shutdown_connections 2024-04-25T10:03:28.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:28.479+0000 7ff1076bd640 1 -- 172.21.15.148:0/10415001 wait complete. 2024-04-25T10:03:28.495 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:03:33.496 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.596+0000 7f44711b7640 1 Processor -- start 2024-04-25T10:03:33.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.596+0000 7f44711b7640 1 -- start start 2024-04-25T10:03:33.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f44711b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 0x7f446c0d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f44711b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c06fe00 con 0x7f446c158220 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f44711b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c0700e0 con 0x7f446c0af720 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f44711b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b3670 con 0x7f446c0afba0 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f446ad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f446c158220 0x7f446c15ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49584/0 (socket says 172.21.15.148:49584) 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f446ad9d640 1 -- 172.21.15.148:0/1123719090 learned_addr learned my addr 172.21.15.148:0/1123719090 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449786805 0 0) 0x7f446c06fe00 con 0x7f446c158220 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f446b59e640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 0x7f446c0d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4454003520 con 0x7f446c158220 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 406510687 0 0) 0x7f4454003520 con 0x7f446c158220 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 msgr2=0x7f446c0d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 0x7f446c0d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 >> v1:172.21.15.188:6789/0 conn(0x7f446c0af720 legacy=0x7f446c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0703c0 con 0x7f446c158220 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f445c001670 con 0x7f446c158220 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 >> v1:172.21.15.148:6789/0 conn(0x7f446c158220 legacy=0x7f446c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f4454003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4454004800 0x7f4454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 0x7f44540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445400ba70 con 0x7f44540039e0 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445400bbf0 con 0x7f4454004800 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.597+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445400bdf0 con 0x7f446c0afba0 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f4454003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446a59c640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4454004800 0x7f4454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446b59e640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 0x7f44540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 -- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 msgr2=0x7f44540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0afba0 0x7f44540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 -- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4454004800 msgr2=0x7f4454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4454004800 0x7f4454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 -- 172.21.15.148:0/1123719090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454003520 con 0x7f44540039e0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.598+0000 7f446ad9d640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f4454003e00 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f445c001eb0 tx=0x7f4454005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de044b005fa263de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c007d30 con 0x7f44540039e0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c005af0 con 0x7f44540039e0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f4469d9b640 1 -- 172.21.15.148:0/1123719090 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c007070 con 0x7f44540039e0 2024-04-25T10:03:33.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f44711b7640 1 -- 172.21.15.148:0/1123719090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 msgr2=0x7f4454003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f44711b7640 1 --2- 172.21.15.148:0/1123719090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f4454003e00 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f445c001eb0 tx=0x7f4454005830 comp rx=0 tx=0).stop 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f446b59e640 1 -- 172.21.15.148:0/1123719090 reap_dead start 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f44711b7640 1 -- 172.21.15.148:0/1123719090 shutdown_connections 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f44711b7640 1 -- 172.21.15.148:0/1123719090 >> 172.21.15.148:0/1123719090 conn(0x7f446c0b2370 msgr2=0x7f446c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f44711b7640 1 -- 172.21.15.148:0/1123719090 shutdown_connections 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.599+0000 7f44711b7640 1 -- 172.21.15.148:0/1123719090 wait complete. 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 Processor -- start 2024-04-25T10:03:33.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 -- start start 2024-04-25T10:03:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f446c0c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0cc890 0x7f446c154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c1587e0 0x7f446c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0b0010 con 0x7f44540039e0 2024-04-25T10:03:33.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c070a10 con 0x7f446c1587e0 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.601+0000 7f44711b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c158600 con 0x7f446c0cc890 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f446c0c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f446c0c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45440/0 (socket says 172.21.15.148:45440) 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 -- 172.21.15.148:0/3221453796 learned_addr learned my addr 172.21.15.148:0/3221453796 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446b59e640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c1587e0 0x7f446c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446a59c640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0cc890 0x7f446c154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 -- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0cc890 msgr2=0x7f446c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0cc890 0x7f446c154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 -- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c1587e0 msgr2=0x7f446c15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c1587e0 0x7f446c15c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 -- 172.21.15.148:0/3221453796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c159150 con 0x7f44540039e0 2024-04-25T10:03:33.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f446ad9d640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f446c0c99f0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c300 tx=0x7f445c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c041020 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c001430 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c15f330 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c007070 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.602+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c1552e0 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.603+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c073990 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.604+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f445c044020 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.604+0000 7f44537fe640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f445806b4d0 0x7f445806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.604+0000 7f446a59c640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f445806b4d0 0x7f445806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.604+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f445c0aa680 con 0x7f44540039e0 2024-04-25T10:03:33.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.605+0000 7f446a59c640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f445806b4d0 0x7f445806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f446c15efa0 tx=0x7f4460009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.609+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f445c007dd0 con 0x7f44540039e0 2024-04-25T10:03:33.777 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-04-25T10:03:33.772+0000 7fae35649640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.815+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f446c162500 con 0x7f44540039e0 2024-04-25T10:03:33.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.815+0000 7f44537fe640 1 -- 172.21.15.148:0/3221453796 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f445c0793b0 con 0x7f44540039e0 2024-04-25T10:03:33.820 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:33.821 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:03:30.801958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9327":{"gid":9327,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:33.821 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f445806b4d0 msgr2=0x7f445806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f445806b4d0 0x7f445806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f446c15efa0 tx=0x7f4460009040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 msgr2=0x7f446c0c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f446c0c99f0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c300 tx=0x7f445c001a40 comp rx=0 tx=0).stop 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 shutdown_connections 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f445806b4d0 0x7f445806d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f446c1587e0 0x7f446c15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f446c0cc890 0x7f446c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 --2- 172.21.15.148:0/3221453796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44540039e0 0x7f446c0c99f0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 >> 172.21.15.148:0/3221453796 conn(0x7f446c0b2370 msgr2=0x7f446c161c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 shutdown_connections 2024-04-25T10:03:33.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.817+0000 7f44711b7640 1 -- 172.21.15.148:0/3221453796 wait complete. 2024-04-25T10:03:33.833 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T10:03:33.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:33.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.926+0000 7fc7d34b3640 1 Processor -- start 2024-04-25T10:03:33.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.926+0000 7fc7d34b3640 1 -- start start 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d34b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7cc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d34b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc15f710 con 0x7fc7cc0cffb0 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d34b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc15f9b0 con 0x7fc7cc15b980 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d34b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15fc90 con 0x7fc7cc06b5f0 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d1a50640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc7cc0cffb0 0x7fc7cc15c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49586/0 (socket says 172.21.15.148:49586) 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d1a50640 1 -- 172.21.15.148:0/2074581449 learned_addr learned my addr 172.21.15.148:0/2074581449 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616269834 0 0) 0x7fc7cc15f710 con 0x7fc7cc0cffb0 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b8003520 con 0x7fc7cc0cffb0 2024-04-25T10:03:33.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.927+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7cc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2537455525 0 0) 0x7fc7b8003520 con 0x7fc7cc0cffb0 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 msgr2=0x7fc7cc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7cc0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 >> v1:172.21.15.188:6789/0 conn(0x7fc7cc15b980 legacy=0x7fc7cc0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc15ff70 con 0x7fc7cc0cffb0 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc7c40019f0 con 0x7fc7cc0cffb0 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 >> v1:172.21.15.148:6789/0 conn(0x7fc7cc0cffb0 legacy=0x7fc7cc15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b80039e0 0x7fc7b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15fc90 con 0x7fc7b8004800 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b800ba70 con 0x7fc7b80039e0 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b800bc70 con 0x7fc7cc06b5f0 2024-04-25T10:03:33.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d124f640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b80039e0 0x7fc7b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d1a50640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d0a4e640 1 -- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 msgr2=0x7fc7b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d0a4e640 1 -- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b80039e0 msgr2=0x7fc7b8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b80039e0 0x7fc7b8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d0a4e640 1 -- 172.21.15.148:0/2074581449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8003520 con 0x7fc7b8004800 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d1a50640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc06b5f0 0x7fc7b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.928+0000 7fc7d124f640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7b80039e0 0x7fc7b8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.929+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7b8004c80 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0d1720 tx=0x7fc7b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22950278bf57071a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.929+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc036070 con 0x7fc7b8004800 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.929+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc031400 con 0x7fc7b8004800 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.929+0000 7fc7c3fff640 1 -- 172.21.15.148:0/2074581449 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc009690 con 0x7fc7b8004800 2024-04-25T10:03:33.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d34b3640 1 -- 172.21.15.148:0/2074581449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 msgr2=0x7fc7b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/2074581449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7b8004c80 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0d1720 tx=0x7fc7b8005830 comp rx=0 tx=0).stop 2024-04-25T10:03:33.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d1a50640 1 -- 172.21.15.148:0/2074581449 reap_dead start 2024-04-25T10:03:33.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d34b3640 1 -- 172.21.15.148:0/2074581449 shutdown_connections 2024-04-25T10:03:33.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d34b3640 1 -- 172.21.15.148:0/2074581449 >> 172.21.15.148:0/2074581449 conn(0x7fc7cc0b2220 msgr2=0x7fc7cc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d34b3640 1 -- 172.21.15.148:0/2074581449 shutdown_connections 2024-04-25T10:03:33.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.930+0000 7fc7d34b3640 1 -- 172.21.15.148:0/2074581449 wait complete. 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.931+0000 7fc7d34b3640 1 Processor -- start 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.931+0000 7fc7d34b3640 1 -- start start 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7cc14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc09e170 0x7fc7cc14bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7cc14c230 0x7fc7cc09bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0cf550 con 0x7fc7b8004800 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15bd60 con 0x7fc7cc14c230 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0b2d40 con 0x7fc7cc09e170 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7cc14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7cc14a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45462/0 (socket says 172.21.15.148:45462) 2024-04-25T10:03:33.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 -- 172.21.15.148:0/654403903 learned_addr learned my addr 172.21.15.148:0/654403903 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc09e170 0x7fc7cc14bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d1a50640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7cc14c230 0x7fc7cc09bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 -- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc09e170 msgr2=0x7fc7cc14bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc09e170 0x7fc7cc14bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 -- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7cc14c230 msgr2=0x7fc7cc09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7cc14c230 0x7fc7cc09bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 -- 172.21.15.148:0/654403903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc15aa70 con 0x7fc7b8004800 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc09e170 0x7fc7cc14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d1a50640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7cc14c230 0x7fc7cc09bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d124f640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7cc14a670 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800e600 tx=0x7fc7c800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80052e0 con 0x7fc7b8004800 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c8018070 con 0x7fc7b8004800 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc14ac50 con 0x7fc7b8004800 2024-04-25T10:03:33.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8003810 con 0x7fc7b8004800 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.932+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc14b210 con 0x7fc7b8004800 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.933+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc073990 con 0x7fc7b8004800 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.934+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7c80039b0 con 0x7fc7b8004800 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.934+0000 7fc7c1ffb640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7b006b500 0x7fc7b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.934+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc7c808a0e0 con 0x7fc7b8004800 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.934+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7b006b500 0x7fc7b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.935+0000 7fc7d0a4e640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7b006b500 0x7fc7b006d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc033000 tx=0x7fc7bc008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:33.939+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7cc15aa70 con 0x7fc7b8004800 2024-04-25T10:03:34.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.146+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7cc0b6670 con 0x7fc7b8004800 2024-04-25T10:03:34.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.146+0000 7fc7c1ffb640 1 -- 172.21.15.148:0/654403903 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fc7c8058e10 con 0x7fc7b8004800 2024-04-25T10:03:34.151 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:34.151 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:03:30.801958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9327":{"gid":9327,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:34.152 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2024-04-25T10:03:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7b006b500 msgr2=0x7fc7b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7b006b500 0x7fc7b006d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc033000 tx=0x7fc7bc008030 comp rx=0 tx=0).stop 2024-04-25T10:03:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 msgr2=0x7fc7cc14a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7cc14a670 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800e600 tx=0x7fc7c800eac0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 shutdown_connections 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc7b006b500 0x7fc7b006d9c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7cc14c230 0x7fc7cc09bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7cc09e170 0x7fc7cc14bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 --2- 172.21.15.148:0/654403903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7b8004800 0x7fc7cc14a670 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 >> 172.21.15.148:0/654403903 conn(0x7fc7cc0b2220 msgr2=0x7fc7cc0b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 shutdown_connections 2024-04-25T10:03:34.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.148+0000 7fc7d34b3640 1 -- 172.21.15.148:0/654403903 wait complete. 2024-04-25T10:03:34.164 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:03:34.164 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.255+0000 7f18794e4640 1 Processor -- start 2024-04-25T10:03:34.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.255+0000 7f18794e4640 1 -- start start 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18794e4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18740b2cb0 0x7f18740d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18794e4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18740d1020 0x7f18740d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18794e4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18740b97a0 con 0x7f187415dbe0 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18794e4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187415dfc0 con 0x7f18740b2cb0 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18794e4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18740d1a00 con 0x7f18740d1020 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18727fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f18740b2cb0 0x7f18740d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f1872ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f187415dbe0 0x7f18740d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49600/0 (socket says 172.21.15.148:49600) 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f1872ffd640 1 -- 172.21.15.148:0/3193482603 learned_addr learned my addr 172.21.15.148:0/3193482603 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18727fc640 1 -- 172.21.15.148:0/3193482603 >> v2:172.21.15.188:3300/0 conn(0x7f18740d1020 msgr2=0x7f18740d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18727fc640 1 --2- 172.21.15.148:0/3193482603 >> v2:172.21.15.188:3300/0 conn(0x7f18740d1020 0x7f18740d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18727fc640 1 -- 172.21.15.148:0/3193482603 >> v1:172.21.15.148:6789/0 conn(0x7f187415dbe0 legacy=0x7f18740d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f18727fc640 1 -- 172.21.15.148:0/3193482603 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18740d1c50 con 0x7f18740b2cb0 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.256+0000 7f1871ffb640 1 -- 172.21.15.148:0/3193482603 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517334071 0 0) 0x7f18740b97a0 con 0x7f187415dbe0 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.257+0000 7f18727fc640 1 --2- 172.21.15.148:0/3193482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740b2cb0 0x7f18740d2730 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f18740cf570 tx=0x7f186802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a003df27bc52693e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.257+0000 7f1871ffb640 1 -- 172.21.15.148:0/3193482603 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868037070 con 0x7f18740b2cb0 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.257+0000 7f1871ffb640 1 -- 172.21.15.148:0/3193482603 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1868032440 con 0x7f18740b2cb0 2024-04-25T10:03:34.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.257+0000 7f1871ffb640 1 -- 172.21.15.148:0/3193482603 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868032810 con 0x7f18740b2cb0 2024-04-25T10:03:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 -- 172.21.15.148:0/3193482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740b2cb0 msgr2=0x7f18740d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 --2- 172.21.15.148:0/3193482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740b2cb0 0x7f18740d2730 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f18740cf570 tx=0x7f186802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 -- 172.21.15.148:0/3193482603 shutdown_connections 2024-04-25T10:03:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 --2- 172.21.15.148:0/3193482603 >> v2:172.21.15.188:3300/0 conn(0x7f18740d1020 0x7f18740d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 --2- 172.21.15.148:0/3193482603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740b2cb0 0x7f18740d2730 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 -- 172.21.15.148:0/3193482603 >> 172.21.15.148:0/3193482603 conn(0x7f1874061a20 msgr2=0x7f1874061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 -- 172.21.15.148:0/3193482603 shutdown_connections 2024-04-25T10:03:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.258+0000 7f18794e4640 1 -- 172.21.15.148:0/3193482603 wait complete. 2024-04-25T10:03:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.259+0000 7f18794e4640 1 Processor -- start 2024-04-25T10:03:34.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.259+0000 7f18794e4640 1 -- start start 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18794e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18740b2cb0 0x7f1874151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18794e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 0x7f187415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18794e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f187415dbe0 0x7f187415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18794e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18740b92a0 con 0x7f18740d1020 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18794e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18740d2210 con 0x7f187415dbe0 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18794e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18740b9420 con 0x7f18740b2cb0 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 0x7f187415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 0x7f187415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45478/0 (socket says 172.21.15.148:45478) 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 -- 172.21.15.148:0/2132589133 learned_addr learned my addr 172.21.15.148:0/2132589133 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f1872ffd640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18740b2cb0 0x7f1874151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18737fe640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f187415dbe0 0x7f187415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 -- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18740b2cb0 msgr2=0x7f1874151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18740b2cb0 0x7f1874151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 -- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f187415dbe0 msgr2=0x7f187415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f187415dbe0 0x7f187415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 -- 172.21.15.148:0/2132589133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18740b5060 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18737fe640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f187415dbe0 0x7f187415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f18727fc640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 0x7f187415b1e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1868000c00 tx=0x7f18680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.260+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868041070 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.261+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186803c950 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.261+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18740b5340 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.261+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868037040 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.261+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18740b5900 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.262+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187406cfc0 con 0x7f18740d1020 2024-04-25T10:03:34.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.262+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1868044070 con 0x7f18740d1020 2024-04-25T10:03:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.262+0000 7f185b7fe640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f186006b4d0 0x7f186006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.263+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f18680ab1c0 con 0x7f18740d1020 2024-04-25T10:03:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.263+0000 7f1872ffd640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f186006b4d0 0x7f186006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.263+0000 7f1872ffd640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f186006b4d0 0x7f186006d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f18640012e0 tx=0x7f1864001630 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.267+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f187406cfc0 con 0x7f18740d1020 2024-04-25T10:03:34.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.503+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f18740b76f0 con 0x7f18740d1020 2024-04-25T10:03:34.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.503+0000 7f185b7fe640 1 -- 172.21.15.148:0/2132589133 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1442 (secure 0 0 0) 0x7f18680799c0 con 0x7f18740d1020 2024-04-25T10:03:34.509 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:34.509 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":575,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2569863168,"bytes_avail":770524250112,"bytes_total":773094113280,"read_bytes_sec":230752,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9327}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:03:29.133238+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f186006b4d0 msgr2=0x7f186006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f186006b4d0 0x7f186006d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f18640012e0 tx=0x7f1864001630 comp rx=0 tx=0).stop 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 msgr2=0x7f187415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 0x7f187415b1e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1868000c00 tx=0x7f18680046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 shutdown_connections 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f186006b4d0 0x7f186006d990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f187415dbe0 0x7f187415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18740d1020 0x7f187415b1e0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 --2- 172.21.15.148:0/2132589133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18740b2cb0 0x7f1874151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 >> 172.21.15.148:0/2132589133 conn(0x7f1874061a20 msgr2=0x7f1874158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 shutdown_connections 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.505+0000 7f18794e4640 1 -- 172.21.15.148:0/2132589133 wait complete. 2024-04-25T10:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:03:34.522 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:34.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:34.563 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:34.604 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:34.604 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:03:34.604 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:34.619 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:03:34.619 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:34.675 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:34.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.809+0000 7f7367c38640 1 Processor -- start 2024-04-25T10:03:34.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.810+0000 7f7367c38640 1 -- start start 2024-04-25T10:03:34.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.810+0000 7f7367c38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f736015dbe0 0x7f73600d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.810+0000 7f7367c38640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600b6220 con 0x7f73600b2cb0 2024-04-25T10:03:34.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.810+0000 7f7367c38640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600b97a0 con 0x7f73600b92a0 2024-04-25T10:03:34.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.810+0000 7f7367c38640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736015dfc0 con 0x7f736015dbe0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.810+0000 7f73651d3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f73600b2cb0 0x7f73600b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49602/0 (socket says 172.21.15.148:49602) 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73651d3640 1 -- 172.21.15.148:0/1946041298 learned_addr learned my addr 172.21.15.148:0/1946041298 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668154110 0 0) 0x7f73600b6220 con 0x7f73600b2cb0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7348003520 con 0x7f73600b2cb0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73659d4640 1 --2- 172.21.15.148:0/1946041298 >> v2:172.21.15.188:3300/0 conn(0x7f736015dbe0 0x7f73600d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4114057020 0 0) 0x7f7348003520 con 0x7f73600b2cb0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 >> v2:172.21.15.188:3300/0 conn(0x7f736015dbe0 msgr2=0x7f73600d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 --2- 172.21.15.148:0/1946041298 >> v2:172.21.15.188:3300/0 conn(0x7f736015dbe0 0x7f73600d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 >> v1:172.21.15.188:6789/0 conn(0x7f73600b92a0 legacy=0x7f73600b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600d1c50 con 0x7f73600b2cb0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7354001670 con 0x7f73600b2cb0 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 >> v1:172.21.15.148:6789/0 conn(0x7f73600b2cb0 legacy=0x7f73600b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480039b0 0x7f7348003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7348004800 0x7f7348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73480052e0 0x7f734800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734800c160 con 0x7f73480052e0 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734800c310 con 0x7f73480039b0 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734800c510 con 0x7f7348004800 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73661d5640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73480052e0 0x7f734800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73659d4640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480039b0 0x7f7348003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73651d3640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7348004800 0x7f7348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73661d5640 1 -- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7348004800 msgr2=0x7f7348004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73661d5640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7348004800 0x7f7348004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73661d5640 1 -- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480039b0 msgr2=0x7f7348003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73661d5640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73480039b0 0x7f7348003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73661d5640 1 -- 172.21.15.148:0/1946041298 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7348003520 con 0x7f73480052e0 2024-04-25T10:03:34.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.811+0000 7f73651d3640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7348004800 0x7f7348004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:34.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.812+0000 7f73661d5640 1 -- 172.21.15.148:0/1946041298 reap_dead start 2024-04-25T10:03:34.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.812+0000 7f73661d5640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73480052e0 0x7f734800b9c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f73600b96d0 tx=0x7f7348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b74f449e8d231dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.812+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358003980 con 0x7f73480052e0 2024-04-25T10:03:34.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.812+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358007240 con 0x7f73480052e0 2024-04-25T10:03:34.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.812+0000 7f73649d2640 1 -- 172.21.15.148:0/1946041298 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358007630 con 0x7f73480052e0 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 -- 172.21.15.148:0/1946041298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73480052e0 msgr2=0x7f734800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73480052e0 0x7f734800b9c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f73600b96d0 tx=0x7f7348005830 comp rx=0 tx=0).stop 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 -- 172.21.15.148:0/1946041298 shutdown_connections 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 --2- 172.21.15.148:0/1946041298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73480052e0 0x7f734800b9c0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 -- 172.21.15.148:0/1946041298 >> 172.21.15.148:0/1946041298 conn(0x7f7360061a20 msgr2=0x7f7360061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 -- 172.21.15.148:0/1946041298 shutdown_connections 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.813+0000 7f7367c38640 1 -- 172.21.15.148:0/1946041298 wait complete. 2024-04-25T10:03:34.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.814+0000 7f7367c38640 1 Processor -- start 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.814+0000 7f7367c38640 1 -- start start 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f7367c38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73480052e0 0x7f73601f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f7367c38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73601f4130 0x7f73601f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f7367c38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 0x7f73600b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f7367c38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600d2210 con 0x7f736015efc0 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f7367c38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600d1a00 con 0x7f73601f4130 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f7367c38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600b6ab0 con 0x7f73480052e0 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 0x7f73600b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 0x7f73600b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45500/0 (socket says 172.21.15.148:45500) 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 -- 172.21.15.148:0/3010033690 learned_addr learned my addr 172.21.15.148:0/3010033690 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73651d3640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73601f4130 0x7f73601f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73659d4640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73480052e0 0x7f73601f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 -- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73480052e0 msgr2=0x7f73601f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73480052e0 0x7f73601f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 -- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73601f4130 msgr2=0x7f73601f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73601f4130 0x7f73601f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 -- 172.21.15.148:0/3010033690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736015f960 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.815+0000 7f73661d5640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 0x7f73600b4200 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f7358000c00 tx=0x7f73580036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.816+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735802a7a0 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.816+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358042070 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.816+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73600cc480 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.816+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735802ab40 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.816+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73601f4a90 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.817+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7360073990 con 0x7f736015efc0 2024-04-25T10:03:34.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.817+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f735802ace0 con 0x7f736015efc0 2024-04-25T10:03:34.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.817+0000 7f735e7fc640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f734c06b4d0 0x7f734c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.818+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f73580aa260 con 0x7f736015efc0 2024-04-25T10:03:34.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.818+0000 7f73659d4640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f734c06b4d0 0x7f734c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.818+0000 7f73659d4640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f734c06b4d0 0x7f734c06d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7350003b80 tx=0x7f7350009760 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.822+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7358078940 con 0x7f736015efc0 2024-04-25T10:03:34.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.991+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f736015c250 con 0x7f736015efc0 2024-04-25T10:03:34.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.992+0000 7f735e7fc640 1 -- 172.21.15.148:0/3010033690 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17079 (secure 0 0 0) 0x7f735807c9e0 con 0x7f736015efc0 2024-04-25T10:03:34.997 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:34.997 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:28.777320+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/1740205817":"2024-04-26T10:03:28.777252+0000","172.21.15.148:6836/1740205817":"2024-04-26T10:03:28.777252+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f734c06b4d0 msgr2=0x7f734c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f734c06b4d0 0x7f734c06d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7350003b80 tx=0x7f7350009760 comp rx=0 tx=0).stop 2024-04-25T10:03:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 msgr2=0x7f73600b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 0x7f73600b4200 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f7358000c00 tx=0x7f73580036f0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 shutdown_connections 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f734c06b4d0 0x7f734c06d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f736015efc0 0x7f73600b4200 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73601f4130 0x7f73601f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 --2- 172.21.15.148:0/3010033690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73480052e0 0x7f73601f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 >> 172.21.15.148:0/3010033690 conn(0x7f7360061a20 msgr2=0x7f736015b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 shutdown_connections 2024-04-25T10:03:35.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:34.994+0000 7f7367c38640 1 -- 172.21.15.148:0/3010033690 wait complete. 2024-04-25T10:03:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:47.359949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:48.130345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:35.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.106+0000 7f49998fc640 1 Processor -- start 2024-04-25T10:03:35.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.106+0000 7f49998fc640 1 -- start start 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.106+0000 7f49998fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49941616f0 0x7f4994161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.106+0000 7f49998fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499406c8d0 con 0x7f49940ce0a0 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.106+0000 7f49998fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49940c6960 con 0x7f49941610f0 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.106+0000 7f49998fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49941614d0 con 0x7f49941616f0 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49937fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49940ce0a0 0x7f49940d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49608/0 (socket says 172.21.15.148:49608) 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49937fe640 1 -- 172.21.15.148:0/160947594 learned_addr learned my addr 172.21.15.148:0/160947594 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f498bfff640 1 --2- 172.21.15.148:0/160947594 >> v2:172.21.15.188:3300/0 conn(0x7f49941616f0 0x7f4994161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633584301 0 0) 0x7f499406c8d0 con 0x7f49940ce0a0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4980003520 con 0x7f49940ce0a0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3217659667 0 0) 0x7f4980003520 con 0x7f49940ce0a0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 >> v2:172.21.15.188:3300/0 conn(0x7f49941616f0 msgr2=0x7f4994161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 --2- 172.21.15.148:0/160947594 >> v2:172.21.15.188:3300/0 conn(0x7f49941616f0 0x7f4994161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 >> v1:172.21.15.188:6789/0 conn(0x7f49941610f0 legacy=0x7f499406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994162260 con 0x7f49940ce0a0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f498c0019f0 con 0x7f49940ce0a0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 >> v1:172.21.15.148:6789/0 conn(0x7f49940ce0a0 legacy=0x7f49940d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49800039b0 0x7f4980003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4980004800 0x7f4980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800051c0 0x7f498000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941614d0 con 0x7f49800051c0 2024-04-25T10:03:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498000c060 con 0x7f49800039b0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498000c260 con 0x7f4980004800 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f49937fe640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800051c0 0x7f498000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f4992ffd640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49800039b0 0x7f4980003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.107+0000 7f498bfff640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4980004800 0x7f4980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 -- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4980004800 msgr2=0x7f4980004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4980004800 0x7f4980004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 -- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49800039b0 msgr2=0x7f4980003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49800039b0 0x7f4980003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 -- 172.21.15.148:0/160947594 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980003520 con 0x7f49800051c0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 -- 172.21.15.148:0/160947594 reap_dead start 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49937fe640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800051c0 0x7f498000b9c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f498c00a860 tx=0x7f4980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c4a9502b803f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c00a480 con 0x7f49800051c0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c02f070 con 0x7f49800051c0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.108+0000 7f49927fc640 1 -- 172.21.15.148:0/160947594 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c02bca0 con 0x7f49800051c0 2024-04-25T10:03:35.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 -- 172.21.15.148:0/160947594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800051c0 msgr2=0x7f498000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800051c0 0x7f498000b9c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f498c00a860 tx=0x7f4980005830 comp rx=0 tx=0).stop 2024-04-25T10:03:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 -- 172.21.15.148:0/160947594 shutdown_connections 2024-04-25T10:03:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 --2- 172.21.15.148:0/160947594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800051c0 0x7f498000b9c0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 -- 172.21.15.148:0/160947594 >> 172.21.15.148:0/160947594 conn(0x7f499406f500 msgr2=0x7f499406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 -- 172.21.15.148:0/160947594 shutdown_connections 2024-04-25T10:03:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.109+0000 7f49998fc640 1 -- 172.21.15.148:0/160947594 wait complete. 2024-04-25T10:03:35.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.110+0000 7f49998fc640 1 Processor -- start 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.110+0000 7f49998fc640 1 -- start start 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49998fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49800051c0 0x7f4994147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49998fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 0x7f4994156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49998fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4994149120 0x7f49941495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49998fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940cfce0 con 0x7f49940af400 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49998fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940cfa00 con 0x7f4994149120 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49998fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940cf690 con 0x7f49800051c0 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 0x7f4994156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 0x7f4994156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45512/0 (socket says 172.21.15.148:45512) 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 -- 172.21.15.148:0/1858174817 learned_addr learned my addr 172.21.15.148:0/1858174817 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49937fe640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4994149120 0x7f49941495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f4992ffd640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49800051c0 0x7f4994147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 -- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49800051c0 msgr2=0x7f4994147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49800051c0 0x7f4994147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 -- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4994149120 msgr2=0x7f49941495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4994149120 0x7f49941495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 -- 172.21.15.148:0/1858174817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49940af880 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f49937fe640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4994149120 0x7f49941495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f4992ffd640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49800051c0 0x7f4994147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.111+0000 7f498bfff640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 0x7f4994156f10 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f497c0118e0 tx=0x7f497c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.112+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00f9d0 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.112+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c00fb70 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.112+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49941481a0 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.112+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c014440 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.112+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994148760 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.113+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49940b0b90 con 0x7f49940af400 2024-04-25T10:03:35.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.113+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f497c012040 con 0x7f49940af400 2024-04-25T10:03:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.113+0000 7f4990ff9640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f497406b500 0x7f497406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.114+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f497c08a8d0 con 0x7f49940af400 2024-04-25T10:03:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.114+0000 7f4992ffd640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f497406b500 0x7f497406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.114+0000 7f4992ffd640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f497406b500 0x7f497406d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f4984009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.118+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49940b0b90 con 0x7f49940af400 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.289+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499406c8b0 con 0x7f49940af400 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.290+0000 7f4990ff9640 1 -- 172.21.15.148:0/1858174817 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f499406c8b0 con 0x7f49940af400 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f497406b500 msgr2=0x7f497406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f497406b500 0x7f497406d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4984000c00 tx=0x7f4984009040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 msgr2=0x7f4994156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 0x7f4994156f10 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f497c0118e0 tx=0x7f497c011da0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 shutdown_connections 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f497406b500 0x7f497406d9c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4994149120 0x7f49941495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49940af400 0x7f4994156f10 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 --2- 172.21.15.148:0/1858174817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49800051c0 0x7f4994147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 >> 172.21.15.148:0/1858174817 conn(0x7f499406f500 msgr2=0x7f49940b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 shutdown_connections 2024-04-25T10:03:35.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.292+0000 7f49998fc640 1 -- 172.21.15.148:0/1858174817 wait complete. 2024-04-25T10:03:35.308 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.403+0000 7f590f50a640 1 Processor -- start 2024-04-25T10:03:35.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.403+0000 7f590f50a640 1 -- start start 2024-04-25T10:03:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590f50a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5908144d20 0x7f59080ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590f50a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080ceba0 con 0x7f59080714d0 2024-04-25T10:03:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590f50a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59080cee80 con 0x7f59081454b0 2024-04-25T10:03:35.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590f50a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080b4830 con 0x7f5908144d20 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590d2a6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5908144d20 0x7f59080ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590daa7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f59080714d0 0x7f59080b4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49620/0 (socket says 172.21.15.148:49620) 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590daa7640 1 -- 172.21.15.148:0/2647331826 learned_addr learned my addr 172.21.15.148:0/2647331826 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590d2a6640 1 -- 172.21.15.148:0/2647331826 >> v1:172.21.15.188:6789/0 conn(0x7f59081454b0 legacy=0x7f5908070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590d2a6640 1 -- 172.21.15.148:0/2647331826 >> v1:172.21.15.148:6789/0 conn(0x7f59080714d0 legacy=0x7f59080b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f590d2a6640 1 -- 172.21.15.148:0/2647331826 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080cf160 con 0x7f5908144d20 2024-04-25T10:03:35.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.404+0000 7f58fffff640 1 -- 172.21.15.148:0/2647331826 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721663419 0 0) 0x7f59080ceba0 con 0x7f59080714d0 2024-04-25T10:03:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.405+0000 7f590d2a6640 1 --2- 172.21.15.148:0/2647331826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5908144d20 0x7f59080ce350 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f59080b49b0 tx=0x7f590402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3093029744fed55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.405+0000 7f58fffff640 1 -- 172.21.15.148:0/2647331826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904037070 con 0x7f5908144d20 2024-04-25T10:03:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.405+0000 7f58fffff640 1 -- 172.21.15.148:0/2647331826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904032440 con 0x7f5908144d20 2024-04-25T10:03:35.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.405+0000 7f58fffff640 1 -- 172.21.15.148:0/2647331826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904032810 con 0x7f5908144d20 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.405+0000 7f590f50a640 1 -- 172.21.15.148:0/2647331826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5908144d20 msgr2=0x7f59080ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.405+0000 7f590f50a640 1 --2- 172.21.15.148:0/2647331826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5908144d20 0x7f59080ce350 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f59080b49b0 tx=0x7f590402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.406+0000 7f590f50a640 1 -- 172.21.15.148:0/2647331826 shutdown_connections 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.406+0000 7f590f50a640 1 --2- 172.21.15.148:0/2647331826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5908144d20 0x7f59080ce350 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.406+0000 7f590f50a640 1 -- 172.21.15.148:0/2647331826 >> 172.21.15.148:0/2647331826 conn(0x7f59080b3530 msgr2=0x7f59080b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.406+0000 7f590f50a640 1 -- 172.21.15.148:0/2647331826 shutdown_connections 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.406+0000 7f590f50a640 1 -- 172.21.15.148:0/2647331826 wait complete. 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 Processor -- start 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 -- start start 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59080714d0 0x7f59080c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5908144d20 0x7f59081e0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 0x7f59080c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908145100 con 0x7f59081454b0 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080ce890 con 0x7f5908144d20 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.407+0000 7f590f50a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5908145280 con 0x7f59080714d0 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 0x7f59080c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 0x7f59080c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45534/0 (socket says 172.21.15.148:45534) 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 -- 172.21.15.148:0/1831871732 learned_addr learned my addr 172.21.15.148:0/1831871732 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590caa5640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5908144d20 0x7f59081e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590d2a6640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59080714d0 0x7f59080c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 -- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59080714d0 msgr2=0x7f59080c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59080714d0 0x7f59080c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 -- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5908144d20 msgr2=0x7f59081e0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5908144d20 0x7f59081e0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 -- 172.21.15.148:0/1831871732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590812eb80 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590caa5640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5908144d20 0x7f59081e0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590daa7640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 0x7f59080c7bb0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f590000c7a0 tx=0x7f590000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900018070 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590000a640 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590812ee60 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590000aab0 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.408+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590812f420 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.409+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590812f620 con 0x7f59081454b0 2024-04-25T10:03:35.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.409+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f590001b070 con 0x7f59081454b0 2024-04-25T10:03:35.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.410+0000 7f58fdffb640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.410+0000 7f590d2a6640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.410+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5900089990 con 0x7f59081454b0 2024-04-25T10:03:35.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.410+0000 7f590d2a6640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58ec06b4d0 0x7f58ec06d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.414+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f590812f620 con 0x7f59081454b0 2024-04-25T10:03:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.582+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5908073a00 con 0x7f59081454b0 2024-04-25T10:03:35.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.582+0000 7f58fdffb640 1 -- 172.21.15.148:0/1831871732 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59000586c0 con 0x7f59081454b0 2024-04-25T10:03:35.587 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:03:35.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58ec06b4d0 msgr2=0x7f58ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58ec06b4d0 0x7f58ec06d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:35.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 msgr2=0x7f59080c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 0x7f59080c7bb0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f590000c7a0 tx=0x7f590000cc60 comp rx=0 tx=0).stop 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 shutdown_connections 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f58ec06b4d0 0x7f58ec06d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59081454b0 0x7f59080c7bb0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5908144d20 0x7f59081e0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 --2- 172.21.15.148:0/1831871732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59080714d0 0x7f59080c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 >> 172.21.15.148:0/1831871732 conn(0x7f59080b3530 msgr2=0x7f59081442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 shutdown_connections 2024-04-25T10:03:35.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.584+0000 7f590f50a640 1 -- 172.21.15.148:0/1831871732 wait complete. 2024-04-25T10:03:35.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:35.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.695+0000 7f75ae168640 1 Processor -- start 2024-04-25T10:03:35.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.695+0000 7f75ae168640 1 -- start start 2024-04-25T10:03:35.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75ae168640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a8161240 0x7f75a80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75ae168640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75a80b54a0 0x7f75a80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75ae168640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a80cf1d0 con 0x7f75a8161890 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75ae168640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a80b7120 con 0x7f75a80b54a0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75ae168640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8161620 con 0x7f75a8161240 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75a7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75a80b54a0 0x7f75a80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75a7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75a80b54a0 0x7f75a80b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45540/0 (socket says 172.21.15.148:45540) 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.696+0000 7f75a7fff640 1 -- 172.21.15.148:0/2429495417 learned_addr learned my addr 172.21.15.148:0/2429495417 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a77fe640 1 --2- 172.21.15.148:0/2429495417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a8161240 0x7f75a80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a7fff640 1 -- 172.21.15.148:0/2429495417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a8161240 msgr2=0x7f75a80b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a7fff640 1 --2- 172.21.15.148:0/2429495417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a8161240 0x7f75a80b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a7fff640 1 -- 172.21.15.148:0/2429495417 >> v1:172.21.15.188:6789/0 conn(0x7f75a8161890 legacy=0x7f75a80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a7fff640 1 -- 172.21.15.148:0/2429495417 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8161e00 con 0x7f75a80b54a0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a7fff640 1 --2- 172.21.15.148:0/2429495417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a80b54a0 0x7f75a80b5930 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5f0f490e513e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a67fc640 1 -- 172.21.15.148:0/2429495417 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0037070 con 0x7f75a80b54a0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a67fc640 1 -- 172.21.15.148:0/2429495417 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a0032440 con 0x7f75a80b54a0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.697+0000 7f75a67fc640 1 -- 172.21.15.148:0/2429495417 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0032810 con 0x7f75a80b54a0 2024-04-25T10:03:35.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 -- 172.21.15.148:0/2429495417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a80b54a0 msgr2=0x7f75a80b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 --2- 172.21.15.148:0/2429495417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a80b54a0 0x7f75a80b5930 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009870 tx=0x7f75a002a210 comp rx=0 tx=0).stop 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 -- 172.21.15.148:0/2429495417 shutdown_connections 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 --2- 172.21.15.148:0/2429495417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a80b54a0 0x7f75a80b5930 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 --2- 172.21.15.148:0/2429495417 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a8161240 0x7f75a80b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 -- 172.21.15.148:0/2429495417 >> 172.21.15.148:0/2429495417 conn(0x7f75a80d04a0 msgr2=0x7f75a80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 -- 172.21.15.148:0/2429495417 shutdown_connections 2024-04-25T10:03:35.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.698+0000 7f75ae168640 1 -- 172.21.15.148:0/2429495417 wait complete. 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.699+0000 7f75ae168640 1 Processor -- start 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.699+0000 7f75ae168640 1 -- start start 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75ae168640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a80b54a0 0x7f75a815a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75ae168640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 0x7f75a81545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75ae168640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a8161890 0x7f75a8154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75ae168640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a81626a0 con 0x7f75a8161240 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75ae168640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a81623c0 con 0x7f75a8161890 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75ae168640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80d15f0 con 0x7f75a80b54a0 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75a6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 0x7f75a81545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75a6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 0x7f75a81545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45556/0 (socket says 172.21.15.148:45556) 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75a6ffd640 1 -- 172.21.15.148:0/4294026220 learned_addr learned my addr 172.21.15.148:0/4294026220 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:35.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75a7fff640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a8161890 0x7f75a8154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.700+0000 7f75a77fe640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a80b54a0 0x7f75a815a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a6ffd640 1 -- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a80b54a0 msgr2=0x7f75a815a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a6ffd640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a80b54a0 0x7f75a815a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a6ffd640 1 -- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a8161890 msgr2=0x7f75a8154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a6ffd640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a8161890 0x7f75a8154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a6ffd640 1 -- 172.21.15.148:0/4294026220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80cb890 con 0x7f75a8161240 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a7fff640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a8161890 0x7f75a8154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a77fe640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a80b54a0 0x7f75a815a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75a6ffd640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 0x7f75a81545e0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f759c00b3f0 tx=0x7f759c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c015020 con 0x7f75a8161240 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c011830 con 0x7f75a8161240 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a80cbb70 con 0x7f75a8161240 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c011c20 con 0x7f75a8161240 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.701+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a80c8090 con 0x7f75a8161240 2024-04-25T10:03:35.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.702+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a80c8290 con 0x7f75a8161240 2024-04-25T10:03:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.702+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f759c018020 con 0x7f75a8161240 2024-04-25T10:03:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.703+0000 7f758ffff640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f759406b4d0 0x7f759406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.703+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f759c08a020 con 0x7f75a8161240 2024-04-25T10:03:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.703+0000 7f75a77fe640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f759406b4d0 0x7f759406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.703+0000 7f75a77fe640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f759406b4d0 0x7f759406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7598007f50 tx=0x7f75980074a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.707+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75a80c8290 con 0x7f75a8161240 2024-04-25T10:03:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.879+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75a8073a00 con 0x7f75a8161240 2024-04-25T10:03:35.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.879+0000 7f758ffff640 1 -- 172.21.15.148:0/4294026220 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7f759c01f030 con 0x7f75a8161240 2024-04-25T10:03:35.885 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:03:35.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f759406b4d0 msgr2=0x7f759406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f759406b4d0 0x7f759406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7598007f50 tx=0x7f75980074a0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 msgr2=0x7f75a81545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 0x7f75a81545e0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f759c00b3f0 tx=0x7f759c00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 shutdown_connections 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f759406b4d0 0x7f759406d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a8161890 0x7f75a8154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75a8161240 0x7f75a81545e0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 --2- 172.21.15.148:0/4294026220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a80b54a0 0x7f75a815a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 >> 172.21.15.148:0/4294026220 conn(0x7f75a80d04a0 msgr2=0x7f75a815b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 shutdown_connections 2024-04-25T10:03:35.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.881+0000 7f75ae168640 1 -- 172.21.15.148:0/4294026220 wait complete. 2024-04-25T10:03:35.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:35.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.990+0000 7fd75d9c8640 1 Processor -- start 2024-04-25T10:03:35.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.990+0000 7fd75d9c8640 1 -- start start 2024-04-25T10:03:35.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd75d9c8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd75815dc20 0x7fd7580ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd75d9c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd75d9c8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75815e000 con 0x7fd758161710 2024-04-25T10:03:35.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd75d9c8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7580d1de0 con 0x7fd75815dc20 2024-04-25T10:03:35.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd75d9c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd758161c60 con 0x7fd7580d32f0 2024-04-25T10:03:35.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd756ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd75815dc20 0x7fd7580ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd756ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd75815dc20 0x7fd7580ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45562/0 (socket says 172.21.15.148:45562) 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.991+0000 7fd756ffd640 1 -- 172.21.15.148:0/2083595475 learned_addr learned my addr 172.21.15.148:0/2083595475 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.992+0000 7fd756ffd640 1 -- 172.21.15.148:0/2083595475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 msgr2=0x7fd7580cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.992+0000 7fd7577fe640 1 --2- 172.21.15.148:0/2083595475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.992+0000 7fd756ffd640 1 --2- 172.21.15.148:0/2083595475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.992+0000 7fd756ffd640 1 -- 172.21.15.148:0/2083595475 >> v1:172.21.15.188:6789/0 conn(0x7fd758161710 legacy=0x7fd75806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.992+0000 7fd756ffd640 1 -- 172.21.15.148:0/2083595475 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758161f40 con 0x7fd75815dc20 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.992+0000 7fd756ffd640 1 --2- 172.21.15.148:0/2083595475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580ce670 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd748009e50 tx=0x7fd74802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6f02b41a7c9a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd755ffb640 1 -- 172.21.15.148:0/2083595475 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748036070 con 0x7fd75815dc20 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd755ffb640 1 -- 172.21.15.148:0/2083595475 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748031600 con 0x7fd75815dc20 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd755ffb640 1 -- 172.21.15.148:0/2083595475 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480319d0 con 0x7fd75815dc20 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 -- 172.21.15.148:0/2083595475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 msgr2=0x7fd7580ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/2083595475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580ce670 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd748009e50 tx=0x7fd74802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 -- 172.21.15.148:0/2083595475 shutdown_connections 2024-04-25T10:03:36.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/2083595475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/2083595475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580ce670 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 -- 172.21.15.148:0/2083595475 >> 172.21.15.148:0/2083595475 conn(0x7fd7580d0420 msgr2=0x7fd7580d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 -- 172.21.15.148:0/2083595475 shutdown_connections 2024-04-25T10:03:36.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.993+0000 7fd75d9c8640 1 -- 172.21.15.148:0/2083595475 wait complete. 2024-04-25T10:03:36.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.994+0000 7fd75d9c8640 1 Processor -- start 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 -- start start 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd758161710 0x7fd75815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580b5e50 con 0x7fd75815dc20 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758162500 con 0x7fd758161710 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd75d9c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7580b5fd0 con 0x7fd7580d32f0 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd7567fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd7567fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45570/0 (socket says 172.21.15.148:45570) 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.995+0000 7fd7567fc640 1 -- 172.21.15.148:0/4176152046 learned_addr learned my addr 172.21.15.148:0/4176152046 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd756ffd640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7567fc640 1 -- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 msgr2=0x7fd7580d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7567fc640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580d3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7567fc640 1 -- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd758161710 msgr2=0x7fd75815ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7577fe640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd758161710 0x7fd75815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7567fc640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd758161710 0x7fd75815ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7567fc640 1 -- 172.21.15.148:0/4176152046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7581602d0 con 0x7fd75815dc20 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7577fe640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd758161710 0x7fd75815ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd7567fc640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580d3e70 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b3f0 tx=0x7fd74c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c015020 con 0x7fd75815dc20 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74c011830 con 0x7fd75815dc20 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758160570 con 0x7fd75815dc20 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c011c20 con 0x7fd75815dc20 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.996+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758160b30 con 0x7fd75815dc20 2024-04-25T10:03:36.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.997+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7580b9330 con 0x7fd75815dc20 2024-04-25T10:03:36.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.997+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd74c018020 con 0x7fd75815dc20 2024-04-25T10:03:36.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.998+0000 7fd73ffff640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd74406b4d0 0x7fd74406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.998+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd74c089f90 con 0x7fd75815dc20 2024-04-25T10:03:36.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.998+0000 7fd756ffd640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd74406b4d0 0x7fd74406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:35.998+0000 7fd756ffd640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd74406b4d0 0x7fd74406d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd748033000 tx=0x7fd748009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.002+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7580b9330 con 0x7fd75815dc20 2024-04-25T10:03:36.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.182+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd7580cf940 con 0x7fd75815dc20 2024-04-25T10:03:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.182+0000 7fd73ffff640 1 -- 172.21.15.148:0/4176152046 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v134) v1 ==== 80+0+124 (secure 0 0 0) 0x7fd7580cf940 con 0x7fd75815dc20 2024-04-25T10:03:36.188 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148:6837/1740205817 2024-04-26T10:03:28.777252+0000 2024-04-25T10:03:36.188 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148:6836/1740205817 2024-04-26T10:03:28.777252+0000 2024-04-25T10:03:36.188 INFO:teuthology.orchestra.run.smithi148.stderr:listed 2 entries 2024-04-25T10:03:36.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd74406b4d0 msgr2=0x7fd74406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd74406b4d0 0x7fd74406d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd748033000 tx=0x7fd748009000 comp rx=0 tx=0).stop 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 msgr2=0x7fd7580d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580d3e70 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd74c00b3f0 tx=0x7fd74c00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 shutdown_connections 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd74406b4d0 0x7fd74406d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd758161710 0x7fd75815ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd75815dc20 0x7fd7580d3e70 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 --2- 172.21.15.148:0/4176152046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7580d32f0 0x7fd7580d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.184+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 >> 172.21.15.148:0/4176152046 conn(0x7fd7580d0420 msgr2=0x7fd75815bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.185+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 shutdown_connections 2024-04-25T10:03:36.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.185+0000 7fd75d9c8640 1 -- 172.21.15.148:0/4176152046 wait complete. 2024-04-25T10:03:36.200 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:36.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:36.228 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:03:36.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:36.228 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:36.283 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:03:36.283 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:36.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:36.302 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:03:36.302 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:36.339 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:36.339 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:03:36.339 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:36.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:36.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:36.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.472+0000 7f6365a5d640 1 Processor -- start 2024-04-25T10:03:36.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.472+0000 7f6365a5d640 1 -- start start 2024-04-25T10:03:36.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f6365a5d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63600d2e90 0x7f63600d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f6365a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f63600b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f6365a5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63600b9940 con 0x7f636015dd30 2024-04-25T10:03:36.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f6365a5d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63600d0d40 con 0x7f63600d2e90 2024-04-25T10:03:36.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f6365a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600d1df0 con 0x7f63600d0900 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635e7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63600d2e90 0x7f63600d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f636015dd30 0x7f63600b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49634/0 (socket says 172.21.15.148:49634) 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635effd640 1 -- 172.21.15.148:0/2348564824 learned_addr learned my addr 172.21.15.148:0/2348564824 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635e7fc640 1 -- 172.21.15.148:0/2348564824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 msgr2=0x7f63600b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635e7fc640 1 --2- 172.21.15.148:0/2348564824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f63600b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635e7fc640 1 -- 172.21.15.148:0/2348564824 >> v1:172.21.15.148:6789/0 conn(0x7f636015dd30 legacy=0x7f63600b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635e7fc640 1 -- 172.21.15.148:0/2348564824 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600d20d0 con 0x7f63600d2e90 2024-04-25T10:03:36.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.473+0000 7f635dffb640 1 -- 172.21.15.148:0/2348564824 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632781500 0 0) 0x7f63600b9940 con 0x7f636015dd30 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.474+0000 7f635e7fc640 1 --2- 172.21.15.148:0/2348564824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63600d2e90 0x7f63600d03c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f6354009790 tx=0x7f635402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a63a11ddc6729c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.474+0000 7f635dffb640 1 -- 172.21.15.148:0/2348564824 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354037070 con 0x7f63600d2e90 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.474+0000 7f635dffb640 1 -- 172.21.15.148:0/2348564824 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635402ae50 con 0x7f63600d2e90 2024-04-25T10:03:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.474+0000 7f635dffb640 1 -- 172.21.15.148:0/2348564824 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354032660 con 0x7f63600d2e90 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 -- 172.21.15.148:0/2348564824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63600d2e90 msgr2=0x7f63600d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 --2- 172.21.15.148:0/2348564824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63600d2e90 0x7f63600d03c0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f6354009790 tx=0x7f635402a100 comp rx=0 tx=0).stop 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 -- 172.21.15.148:0/2348564824 shutdown_connections 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 --2- 172.21.15.148:0/2348564824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f63600b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 --2- 172.21.15.148:0/2348564824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63600d2e90 0x7f63600d03c0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 -- 172.21.15.148:0/2348564824 >> 172.21.15.148:0/2348564824 conn(0x7f6360061c30 msgr2=0x7f636006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 -- 172.21.15.148:0/2348564824 shutdown_connections 2024-04-25T10:03:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.475+0000 7f6365a5d640 1 -- 172.21.15.148:0/2348564824 wait complete. 2024-04-25T10:03:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.476+0000 7f6365a5d640 1 Processor -- start 2024-04-25T10:03:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.476+0000 7f6365a5d640 1 -- start start 2024-04-25T10:03:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f6360151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63600d2e90 0x7f63600cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 0x7f6360160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600d2970 con 0x7f636015dd30 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600d2690 con 0x7f63600d2e90 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63600d1a10 con 0x7f63600d0900 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 0x7f6360160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 0x7f6360160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45596/0 (socket says 172.21.15.148:45596) 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 -- 172.21.15.148:0/1515764673 learned_addr learned my addr 172.21.15.148:0/1515764673 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635effd640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f6360151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635e7fc640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63600d2e90 0x7f63600cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 -- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 msgr2=0x7f6360151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f6360151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 -- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63600d2e90 msgr2=0x7f63600cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63600d2e90 0x7f63600cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 -- 172.21.15.148:0/1515764673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600b3f00 con 0x7f636015dd30 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635e7fc640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63600d2e90 0x7f63600cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f635f7fe640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 0x7f6360160050 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f635800b970 tx=0x7f635800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358019070 con 0x7f636015dd30 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635800cb60 con 0x7f636015dd30 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.477+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63600cb8c0 con 0x7f636015dd30 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.478+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358014440 con 0x7f636015dd30 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.478+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63600cbe80 con 0x7f636015dd30 2024-04-25T10:03:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.479+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6360073990 con 0x7f636015dd30 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.479+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f635801c070 con 0x7f636015dd30 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.479+0000 7f6347fff640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b4d0 0x7f634c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.480+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f635808a100 con 0x7f636015dd30 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.480+0000 7f635effd640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b4d0 0x7f634c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.480+0000 7f635effd640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b4d0 0x7f634c06d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6350000f60 tx=0x7f6350007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.484+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6358058da0 con 0x7f636015dd30 2024-04-25T10:03:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.690+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63600b6aa0 con 0x7f636015dd30 2024-04-25T10:03:36.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.690+0000 7f6347fff640 1 -- 172.21.15.148:0/1515764673 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f63580586a0 con 0x7f636015dd30 2024-04-25T10:03:36.696 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:36.696 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:03:30.801958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9327":{"gid":9327,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:36.696 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b4d0 msgr2=0x7f634c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b4d0 0x7f634c06d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6350000f60 tx=0x7f6350007c00 comp rx=0 tx=0).stop 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 msgr2=0x7f6360160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 0x7f6360160050 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f635800b970 tx=0x7f635800be30 comp rx=0 tx=0).stop 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 shutdown_connections 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f634c06b4d0 0x7f634c06d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f636015dd30 0x7f6360160050 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63600d2e90 0x7f63600cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 --2- 172.21.15.148:0/1515764673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63600d0900 0x7f6360151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.692+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 >> 172.21.15.148:0/1515764673 conn(0x7f6360061c30 msgr2=0x7f63600c84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.693+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 shutdown_connections 2024-04-25T10:03:36.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.693+0000 7f6365a5d640 1 -- 172.21.15.148:0/1515764673 wait complete. 2024-04-25T10:03:36.709 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:36.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.801+0000 7f9275ab6640 1 Processor -- start 2024-04-25T10:03:36.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.801+0000 7f9275ab6640 1 -- start start 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f9275ab6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92700d1020 0x7f92700d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f9275ab6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92700b97a0 con 0x7f927015dbe0 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f9275ab6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92700d1c50 con 0x7f92700b2cb0 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f9275ab6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927015dfc0 con 0x7f92700d1020 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f926f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f927015dbe0 0x7f92700d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49638/0 (socket says 172.21.15.148:49638) 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f926f7fe640 1 -- 172.21.15.148:0/3712882153 learned_addr learned my addr 172.21.15.148:0/3712882153 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.802+0000 7f926ffff640 1 --2- 172.21.15.148:0/3712882153 >> v2:172.21.15.148:3300/0 conn(0x7f92700d1020 0x7f92700d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.803+0000 7f926ffff640 1 -- 172.21.15.148:0/3712882153 >> v1:172.21.15.188:6789/0 conn(0x7f92700b2cb0 legacy=0x7f92700d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.803+0000 7f926ffff640 1 -- 172.21.15.148:0/3712882153 >> v1:172.21.15.148:6789/0 conn(0x7f927015dbe0 legacy=0x7f92700d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.803+0000 7f926ffff640 1 -- 172.21.15.148:0/3712882153 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92700d1f30 con 0x7f92700d1020 2024-04-25T10:03:36.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.803+0000 7f926ffff640 1 --2- 172.21.15.148:0/3712882153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f92700d2e40 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f9268009d90 tx=0x7f926802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d61d9f110cff41a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.804+0000 7f926e7fc640 1 -- 172.21.15.148:0/3712882153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268007740 con 0x7f92700d1020 2024-04-25T10:03:36.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.804+0000 7f926e7fc640 1 -- 172.21.15.148:0/3712882153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92680078e0 con 0x7f92700d1020 2024-04-25T10:03:36.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.804+0000 7f926e7fc640 1 -- 172.21.15.148:0/3712882153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268007cb0 con 0x7f92700d1020 2024-04-25T10:03:36.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 -- 172.21.15.148:0/3712882153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 msgr2=0x7f92700d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 --2- 172.21.15.148:0/3712882153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f92700d2e40 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f9268009d90 tx=0x7f926802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 -- 172.21.15.148:0/3712882153 shutdown_connections 2024-04-25T10:03:36.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 --2- 172.21.15.148:0/3712882153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f92700d2e40 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 -- 172.21.15.148:0/3712882153 >> 172.21.15.148:0/3712882153 conn(0x7f9270061a20 msgr2=0x7f9270061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 -- 172.21.15.148:0/3712882153 shutdown_connections 2024-04-25T10:03:36.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.805+0000 7f9275ab6640 1 -- 172.21.15.148:0/3712882153 wait complete. 2024-04-25T10:03:36.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.806+0000 7f9275ab6640 1 Processor -- start 2024-04-25T10:03:36.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.806+0000 7f9275ab6640 1 -- start start 2024-04-25T10:03:36.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f9275ab6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92700b2cb0 0x7f927014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f9275ab6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f927015ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f9275ab6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f927015dbe0 0x7f92700ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f9275ab6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92700b92a0 con 0x7f92700d1020 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f9275ab6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92700d1a00 con 0x7f92700b2cb0 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f9275ab6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92700b9420 con 0x7f927015dbe0 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f927015ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f927015ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45604/0 (socket says 172.21.15.148:45604) 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 -- 172.21.15.148:0/907289921 learned_addr learned my addr 172.21.15.148:0/907289921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926f7fe640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92700b2cb0 0x7f927014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926ffff640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f927015dbe0 0x7f92700ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 -- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f927015dbe0 msgr2=0x7f92700ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f927015dbe0 0x7f92700ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 -- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92700b2cb0 msgr2=0x7f927014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92700b2cb0 0x7f927014cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.807+0000 7f926effd640 1 -- 172.21.15.148:0/907289921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270160f30 con 0x7f92700d1020 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.808+0000 7f926effd640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f927015ead0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f926400c440 tx=0x7f9264001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.808+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640052d0 con 0x7f92700d1020 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.808+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264005470 con 0x7f92700d1020 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.808+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9270160c50 con 0x7f92700d1020 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.808+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264007260 con 0x7f92700d1020 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.808+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9270152d40 con 0x7f92700d1020 2024-04-25T10:03:36.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.809+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9270073990 con 0x7f92700d1020 2024-04-25T10:03:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.809+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9264007400 con 0x7f92700d1020 2024-04-25T10:03:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.810+0000 7f9257fff640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f925c06b500 0x7f925c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.810+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f92640897c0 con 0x7f92700d1020 2024-04-25T10:03:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.810+0000 7f926f7fe640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f925c06b500 0x7f925c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.810+0000 7f926f7fe640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f925c06b500 0x7f925c06d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9260005880 tx=0x7f9260009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.814+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9264057e10 con 0x7f92700d1020 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.985+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f927006b080 con 0x7f92700d1020 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.986+0000 7f9257fff640 1 -- 172.21.15.148:0/907289921 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17079 (secure 0 0 0) 0x7f926405beb0 con 0x7f92700d1020 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:28.777320+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/1740205817":"2024-04-26T10:03:28.777252+0000","172.21.15.148:6836/1740205817":"2024-04-26T10:03:28.777252+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f925c06b500 msgr2=0x7f925c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f925c06b500 0x7f925c06d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9260005880 tx=0x7f9260009040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 msgr2=0x7f927015ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f927015ead0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f926400c440 tx=0x7f9264001e70 comp rx=0 tx=0).stop 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 shutdown_connections 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f925c06b500 0x7f925c06d9c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f927015dbe0 0x7f92700ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92700d1020 0x7f927015ead0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 --2- 172.21.15.148:0/907289921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92700b2cb0 0x7f927014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 >> 172.21.15.148:0/907289921 conn(0x7f9270061a20 msgr2=0x7f92700d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 shutdown_connections 2024-04-25T10:03:36.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:36.988+0000 7f9275ab6640 1 -- 172.21.15.148:0/907289921 wait complete. 2024-04-25T10:03:37.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:47.359949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:48.130345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:37.005 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.099+0000 7f87788c4640 1 Processor -- start 2024-04-25T10:03:37.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.099+0000 7f87788c4640 1 -- start start 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f87788c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 0x7f87740d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f87788c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f87740d08e0 0x7f87740b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f87788c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87740d2420 con 0x7f877415dac0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f87788c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87740d3030 con 0x7f87740d08e0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f87788c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740d2700 con 0x7f87740d34c0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f877259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f877415dac0 0x7f87740d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49644/0 (socket says 172.21.15.148:49644) 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.100+0000 7f877259c640 1 -- 172.21.15.148:0/2606092196 learned_addr learned my addr 172.21.15.148:0/2606092196 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f8772d9d640 1 --2- 172.21.15.148:0/2606092196 >> v2:172.21.15.188:3300/0 conn(0x7f87740d08e0 0x7f87740b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f8771d9b640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 0x7f87740d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210384091 0 0) 0x7f87740d2420 con 0x7f877415dac0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f875c003520 con 0x7f877415dac0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1589686569 0 0) 0x7f875c003520 con 0x7f877415dac0 2024-04-25T10:03:37.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 msgr2=0x7f87740d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 0x7f87740d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 >> v2:172.21.15.188:3300/0 conn(0x7f87740d08e0 msgr2=0x7f87740b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 --2- 172.21.15.148:0/2606092196 >> v2:172.21.15.188:3300/0 conn(0x7f87740d08e0 0x7f87740b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87740d29e0 con 0x7f877415dac0 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f87640019f0 con 0x7f877415dac0 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f8772d9d640 1 --2- 172.21.15.148:0/2606092196 >> v2:172.21.15.188:3300/0 conn(0x7f87740d08e0 0x7f87740b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 >> v1:172.21.15.148:6789/0 conn(0x7f877415dac0 legacy=0x7f87740d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f875c0039b0 0x7f875c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 0x7f875c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740d3030 con 0x7f875c0039b0 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740d2700 con 0x7f875c004800 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875c00b9c0 con 0x7f87740d34c0 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877259c640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f875c0039b0 0x7f875c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f8772d9d640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 0x7f875c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f8771d9b640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877259c640 1 -- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 msgr2=0x7f875c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877259c640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740d34c0 0x7f875c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877259c640 1 -- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f875c004800 msgr2=0x7f875c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877259c640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f875c004800 0x7f875c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.101+0000 7f877259c640 1 -- 172.21.15.148:0/2606092196 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c003520 con 0x7f875c0039b0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.102+0000 7f877259c640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f875c0039b0 0x7f875c003dd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f876400a880 tx=0x7f875c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26334d374b582161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.102+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400a4a0 con 0x7f875c0039b0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.102+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876402f070 con 0x7f875c0039b0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.102+0000 7f877159a640 1 -- 172.21.15.148:0/2606092196 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876402bd00 con 0x7f875c0039b0 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f87788c4640 1 -- 172.21.15.148:0/2606092196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f875c0039b0 msgr2=0x7f875c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f87788c4640 1 --2- 172.21.15.148:0/2606092196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f875c0039b0 0x7f875c003dd0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f876400a880 tx=0x7f875c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f8772d9d640 1 -- 172.21.15.148:0/2606092196 reap_dead start 2024-04-25T10:03:37.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f87788c4640 1 -- 172.21.15.148:0/2606092196 shutdown_connections 2024-04-25T10:03:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f87788c4640 1 -- 172.21.15.148:0/2606092196 >> 172.21.15.148:0/2606092196 conn(0x7f8774061c30 msgr2=0x7f877406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f87788c4640 1 -- 172.21.15.148:0/2606092196 shutdown_connections 2024-04-25T10:03:37.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.103+0000 7f87788c4640 1 -- 172.21.15.148:0/2606092196 wait complete. 2024-04-25T10:03:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.104+0000 7f87788c4640 1 Processor -- start 2024-04-25T10:03:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.104+0000 7f87788c4640 1 -- start start 2024-04-25T10:03:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f875c0039b0 0x7f8774152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87740cbd90 0x7f87740cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 0x7f877415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877415dea0 con 0x7f87740cc730 2024-04-25T10:03:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740d2100 con 0x7f87740cbd90 2024-04-25T10:03:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740b97d0 con 0x7f875c0039b0 2024-04-25T10:03:37.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 0x7f877415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 0x7f877415f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45616/0 (socket says 172.21.15.148:45616) 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 -- 172.21.15.148:0/2494786982 learned_addr learned my addr 172.21.15.148:0/2494786982 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f877259c640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f875c0039b0 0x7f8774152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8771d9b640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87740cbd90 0x7f87740cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 -- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f875c0039b0 msgr2=0x7f8774152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f875c0039b0 0x7f8774152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 -- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87740cbd90 msgr2=0x7f87740cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87740cbd90 0x7f87740cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 -- 172.21.15.148:0/2494786982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8774152970 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f877259c640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f875c0039b0 0x7f8774152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8772d9d640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 0x7f877415f630 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f87740b6550 tx=0x7f876c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c004440 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c01c070 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87740ccff0 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.105+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c003830 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.106+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87740cd5b0 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.107+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87740cd7b0 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.107+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f876c0039d0 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.107+0000 7f8762ffd640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f875406b4d0 0x7f875406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.107+0000 7f877259c640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f875406b4d0 0x7f875406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.107+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f876c08a0b0 con 0x7f87740cc730 2024-04-25T10:03:37.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.108+0000 7f877259c640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f875406b4d0 0x7f875406d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8764033000 tx=0x7f8764007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.112+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87740cd7b0 con 0x7f87740cc730 2024-04-25T10:03:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.280+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8774073a00 con 0x7f87740cc730 2024-04-25T10:03:37.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.280+0000 7f8762ffd640 1 -- 172.21.15.148:0/2494786982 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f876c025020 con 0x7f87740cc730 2024-04-25T10:03:37.286 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.282+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f875406b4d0 msgr2=0x7f875406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.282+0000 7f87788c4640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f875406b4d0 0x7f875406d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8764033000 tx=0x7f8764007000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 msgr2=0x7f877415f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 0x7f877415f630 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f87740b6550 tx=0x7f876c011a80 comp rx=0 tx=0).stop 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 shutdown_connections 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f875406b4d0 0x7f875406d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87740cc730 0x7f877415f630 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87740cbd90 0x7f87740cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 --2- 172.21.15.148:0/2494786982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f875c0039b0 0x7f8774152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 >> 172.21.15.148:0/2494786982 conn(0x7f8774061c30 msgr2=0x7f87740b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 shutdown_connections 2024-04-25T10:03:37.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.283+0000 7f87788c4640 1 -- 172.21.15.148:0/2494786982 wait complete. 2024-04-25T10:03:37.299 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:37.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.393+0000 7f6d30716640 1 Processor -- start 2024-04-25T10:03:37.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.393+0000 7f6d30716640 1 -- start start 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.394+0000 7f6d30716640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d280d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.394+0000 7f6d30716640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d2815e000 con 0x7f6d2815dc20 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.394+0000 7f6d30716640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d280d1de0 con 0x7f6d280ced80 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.394+0000 7f6d30716640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28161c60 con 0x7f6d28161710 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.394+0000 7f6d2e4b2640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6d2815dc20 0x7f6d280ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49646/0 (socket says 172.21.15.148:49646) 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.394+0000 7f6d2e4b2640 1 -- 172.21.15.148:0/1372517487 learned_addr learned my addr 172.21.15.148:0/1372517487 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:37.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186189663 0 0) 0x7f6d2815e000 con 0x7f6d2815dc20 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d14003520 con 0x7f6d2815dc20 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2ecb3640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d280d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 216485387 0 0) 0x7f6d14003520 con 0x7f6d2815dc20 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 msgr2=0x7f6d280d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d280d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 >> v1:172.21.15.188:6789/0 conn(0x7f6d280ced80 legacy=0x7f6d280d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d28161f40 con 0x7f6d2815dc20 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6d240019f0 con 0x7f6d2815dc20 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 >> v1:172.21.15.148:6789/0 conn(0x7f6d2815dc20 legacy=0x7f6d280ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d140039e0 0x7f6d14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28161c60 con 0x7f6d14004800 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1400ba70 con 0x7f6d140039e0 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1400bc70 con 0x7f6d28161710 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2e4b2640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d140039e0 0x7f6d14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2ecb3640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2dcb1640 1 -- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 msgr2=0x7f6d140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2dcb1640 1 -- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d140039e0 msgr2=0x7f6d14003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d140039e0 0x7f6d14003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2dcb1640 1 -- 172.21.15.148:0/1372517487 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d14003520 con 0x7f6d14004800 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.395+0000 7f6d2ecb3640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d28161710 0x7f6d140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:37.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.396+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d14004c80 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f6d18001720 tx=0x7f6d14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57f0ba7f9b31dec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.396+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18001370 con 0x7f6d14004800 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.396+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d180052d0 con 0x7f6d14004800 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.396+0000 7f6d2d4b0640 1 -- 172.21.15.148:0/1372517487 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1802ab90 con 0x7f6d14004800 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d30716640 1 -- 172.21.15.148:0/1372517487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 msgr2=0x7f6d14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d30716640 1 --2- 172.21.15.148:0/1372517487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d14004c80 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f6d18001720 tx=0x7f6d14005830 comp rx=0 tx=0).stop 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d2ecb3640 1 -- 172.21.15.148:0/1372517487 reap_dead start 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d30716640 1 -- 172.21.15.148:0/1372517487 shutdown_connections 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d30716640 1 -- 172.21.15.148:0/1372517487 >> 172.21.15.148:0/1372517487 conn(0x7f6d280d0420 msgr2=0x7f6d280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d30716640 1 -- 172.21.15.148:0/1372517487 shutdown_connections 2024-04-25T10:03:37.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.397+0000 7f6d30716640 1 -- 172.21.15.148:0/1372517487 wait complete. 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.398+0000 7f6d30716640 1 Processor -- start 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.398+0000 7f6d30716640 1 -- start start 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d280aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d280ccb10 0x7f6d280cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d281603e0 0x7f6d28160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280b5e50 con 0x7f6d14004800 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28162500 con 0x7f6d281603e0 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280b5fd0 con 0x7f6d280ccb10 2024-04-25T10:03:37.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d280aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d280aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45634/0 (socket says 172.21.15.148:45634) 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 -- 172.21.15.148:0/3589076079 learned_addr learned my addr 172.21.15.148:0/3589076079 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2ecb3640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d281603e0 0x7f6d28160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d280ccb10 0x7f6d280cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 -- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d280ccb10 msgr2=0x7f6d280cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d280ccb10 0x7f6d280cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 -- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d281603e0 msgr2=0x7f6d28160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d281603e0 0x7f6d28160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 -- 172.21.15.148:0/3589076079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280aaab0 con 0x7f6d14004800 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d2e4b2640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d280aa380 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f6d240077f0 tx=0x7f6d2400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24017020 con 0x7f6d14004800 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d240133f0 con 0x7f6d14004800 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d280aad90 con 0x7f6d14004800 2024-04-25T10:03:37.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.399+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d240137c0 con 0x7f6d14004800 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.400+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d280ccf90 con 0x7f6d14004800 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.401+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d28073a00 con 0x7f6d14004800 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.401+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d2401a020 con 0x7f6d14004800 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.401+0000 7f6d1effd640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d0c06b500 0x7f6d0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.401+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6d2408aea0 con 0x7f6d14004800 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.401+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d0c06b500 0x7f6d0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.402+0000 7f6d2dcb1640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d0c06b500 0x7f6d0c06d9c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d18032000 tx=0x7f6d1802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.405+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d24021020 con 0x7f6d14004800 2024-04-25T10:03:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.573+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d280d16b0 con 0x7f6d14004800 2024-04-25T10:03:37.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.574+0000 7f6d1effd640 1 -- 172.21.15.148:0/3589076079 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d24059b40 con 0x7f6d14004800 2024-04-25T10:03:37.579 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:03:37.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d0c06b500 msgr2=0x7f6d0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d0c06b500 0x7f6d0c06d9c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d18032000 tx=0x7f6d1802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 msgr2=0x7f6d280aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d280aa380 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f6d240077f0 tx=0x7f6d2400a710 comp rx=0 tx=0).stop 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 shutdown_connections 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6d0c06b500 0x7f6d0c06d9c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d281603e0 0x7f6d28160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d280ccb10 0x7f6d280cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 --2- 172.21.15.148:0/3589076079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d14004800 0x7f6d280aa380 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 >> 172.21.15.148:0/3589076079 conn(0x7f6d280d0420 msgr2=0x7f6d280d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 shutdown_connections 2024-04-25T10:03:37.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.576+0000 7f6d30716640 1 -- 172.21.15.148:0/3589076079 wait complete. 2024-04-25T10:03:37.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:37.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.686+0000 7f48d8310640 1 Processor -- start 2024-04-25T10:03:37.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.686+0000 7f48d8310640 1 -- start start 2024-04-25T10:03:37.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.687+0000 7f48d8310640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d006abe0 0x7f48d006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.687+0000 7f48d8310640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f48d015dd80 0x7f48d00b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.687+0000 7f48d8310640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48d00ad570 0x7f48d006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.687+0000 7f48d8310640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d006e410 con 0x7f48d00ad570 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.687+0000 7f48d8310640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d006b500 con 0x7f48d015dd80 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.687+0000 7f48d8310640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00b3930 con 0x7f48d006abe0 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48d00ad570 0x7f48d006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48d00ad570 0x7f48d006ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45644/0 (socket says 172.21.15.148:45644) 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 -- 172.21.15.148:0/1889645096 learned_addr learned my addr 172.21.15.148:0/1889645096 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d60ac640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d006abe0 0x7f48d006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d58ab640 1 --2- 172.21.15.148:0/1889645096 >> v2:172.21.15.188:3300/0 conn(0x7f48d015dd80 0x7f48d00b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 -- 172.21.15.148:0/1889645096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d006abe0 msgr2=0x7f48d006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d006abe0 0x7f48d006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 -- 172.21.15.148:0/1889645096 >> v2:172.21.15.188:3300/0 conn(0x7f48d015dd80 msgr2=0x7f48d00b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 --2- 172.21.15.148:0/1889645096 >> v2:172.21.15.188:3300/0 conn(0x7f48d015dd80 0x7f48d00b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d68ad640 1 -- 172.21.15.148:0/1889645096 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00b3c10 con 0x7f48d00ad570 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d60ac640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d006abe0 0x7f48d006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.688+0000 7f48d58ab640 1 --2- 172.21.15.148:0/1889645096 >> v2:172.21.15.188:3300/0 conn(0x7f48d015dd80 0x7f48d00b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d68ad640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00ad570 0x7f48d006ee60 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f48d00d1820 tx=0x7f48c802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d22ae530b3d394e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d50aa640 1 -- 172.21.15.148:0/1889645096 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8036070 con 0x7f48d00ad570 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d50aa640 1 -- 172.21.15.148:0/1889645096 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8031600 con 0x7f48d00ad570 2024-04-25T10:03:37.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d50aa640 1 -- 172.21.15.148:0/1889645096 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80319d0 con 0x7f48d00ad570 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 -- 172.21.15.148:0/1889645096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00ad570 msgr2=0x7f48d006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00ad570 0x7f48d006ee60 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f48d00d1820 tx=0x7f48c802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 -- 172.21.15.148:0/1889645096 shutdown_connections 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d00ad570 0x7f48d006ee60 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 --2- 172.21.15.148:0/1889645096 >> v2:172.21.15.188:3300/0 conn(0x7f48d015dd80 0x7f48d00b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 --2- 172.21.15.148:0/1889645096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d006abe0 0x7f48d006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 -- 172.21.15.148:0/1889645096 >> 172.21.15.148:0/1889645096 conn(0x7f48d00b5af0 msgr2=0x7f48d00b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 -- 172.21.15.148:0/1889645096 shutdown_connections 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.689+0000 7f48d8310640 1 -- 172.21.15.148:0/1889645096 wait complete. 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 Processor -- start 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 -- start start 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 0x7f48d01e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d00ad570 0x7f48d0153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d015dd80 0x7f48d01f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00b57d0 con 0x7f48d006abe0 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d006e940 con 0x7f48d015dd80 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d8310640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0070280 con 0x7f48d00ad570 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d60ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 0x7f48d01e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d60ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 0x7f48d01e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45660/0 (socket says 172.21.15.148:45660) 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.691+0000 7f48d60ac640 1 -- 172.21.15.148:0/3049080671 learned_addr learned my addr 172.21.15.148:0/3049080671 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:37.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d68ad640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d015dd80 0x7f48d01f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d60ac640 1 -- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d00ad570 msgr2=0x7f48d0153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d58ab640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d00ad570 0x7f48d0153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d60ac640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d00ad570 0x7f48d0153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d60ac640 1 -- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d015dd80 msgr2=0x7f48d01f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d60ac640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d015dd80 0x7f48d01f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d60ac640 1 -- 172.21.15.148:0/3049080671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d01537a0 con 0x7f48d006abe0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d68ad640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d015dd80 0x7f48d01f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d60ac640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 0x7f48d01e6100 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00c7f0 tx=0x7f48cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc015070 con 0x7f48d006abe0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc007820 con 0x7f48d006abe0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc007c10 con 0x7f48d006abe0 2024-04-25T10:03:37.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d01e6870 con 0x7f48d006abe0 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.692+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d01e6e30 con 0x7f48d006abe0 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.693+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0073a00 con 0x7f48d006abe0 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.694+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48cc018070 con 0x7f48d006abe0 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.694+0000 7f48c6ffd640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48b406b4d0 0x7f48b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.694+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f48cc08a480 con 0x7f48d006abe0 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.694+0000 7f48d58ab640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48b406b4d0 0x7f48b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.695+0000 7f48d58ab640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48b406b4d0 0x7f48b406d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f48d00ae6a0 tx=0x7f48c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.698+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48cc059130 con 0x7f48d006abe0 2024-04-25T10:03:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.872+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48d00d13d0 con 0x7f48d006abe0 2024-04-25T10:03:37.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.872+0000 7f48c6ffd640 1 -- 172.21.15.148:0/3049080671 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48cc058a30 con 0x7f48d006abe0 2024-04-25T10:03:37.877 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:03:37.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48b406b4d0 msgr2=0x7f48b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48b406b4d0 0x7f48b406d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f48d00ae6a0 tx=0x7f48c0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 msgr2=0x7f48d01e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 0x7f48d01e6100 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00c7f0 tx=0x7f48cc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 shutdown_connections 2024-04-25T10:03:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48b406b4d0 0x7f48b406d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d015dd80 0x7f48d01f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48d00ad570 0x7f48d0153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 --2- 172.21.15.148:0/3049080671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48d006abe0 0x7f48d01e6100 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 >> 172.21.15.148:0/3049080671 conn(0x7f48d00b5af0 msgr2=0x7f48d015c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 shutdown_connections 2024-04-25T10:03:37.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.874+0000 7f48d8310640 1 -- 172.21.15.148:0/3049080671 wait complete. 2024-04-25T10:03:37.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:37.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.983+0000 7f54cd08f640 1 Processor -- start 2024-04-25T10:03:37.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.983+0000 7f54cd08f640 1 -- start start 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.984+0000 7f54cd08f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 0x7f54c80d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.984+0000 7f54cd08f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c806fe00 con 0x7f54c8158220 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.984+0000 7f54cd08f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c80700e0 con 0x7f54c80af720 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.984+0000 7f54cd08f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c80b3670 con 0x7f54c80afba0 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.984+0000 7f54c6d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f54c8158220 0x7f54c815ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49660/0 (socket says 172.21.15.148:49660) 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.984+0000 7f54c6d9d640 1 -- 172.21.15.148:0/449597297 learned_addr learned my addr 172.21.15.148:0/449597297 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:37.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c759e640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 0x7f54c80d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180547540 0 0) 0x7f54c806fe00 con 0x7f54c8158220 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54b4003520 con 0x7f54c8158220 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1672277793 0 0) 0x7f54b4003520 con 0x7f54c8158220 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 msgr2=0x7f54c80d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 0x7f54c80d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 >> v1:172.21.15.188:6789/0 conn(0x7f54c80af720 legacy=0x7f54c80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80703c0 con 0x7f54c8158220 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f54b0001670 con 0x7f54c8158220 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 >> v1:172.21.15.148:6789/0 conn(0x7f54c8158220 legacy=0x7f54c815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 0x7f54b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b4003f60 0x7f54b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b4004d40 0x7f54b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b400ba70 con 0x7f54b4003f60 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b400bc20 con 0x7f54b4004d40 2024-04-25T10:03:37.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b400be20 con 0x7f54c80afba0 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c659c640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b4003f60 0x7f54b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c6d9d640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 0x7f54b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c759e640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b4004d40 0x7f54b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c659c640 1 -- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 msgr2=0x7f54b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c659c640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54c80afba0 0x7f54b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c659c640 1 -- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b4004d40 msgr2=0x7f54b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c659c640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b4004d40 0x7f54b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c659c640 1 -- 172.21.15.148:0/449597297 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4003520 con 0x7f54b4003f60 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.985+0000 7f54c759e640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54b4004d40 0x7f54b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.986+0000 7f54c659c640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b4003f60 0x7f54b4004800 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001750 tx=0x7f54b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61851d43ce75ac49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.986+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8001370 con 0x7f54b4003f60 2024-04-25T10:03:37.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.986+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b80052d0 con 0x7f54b4003f60 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.986+0000 7f54c5d9b640 1 -- 172.21.15.148:0/449597297 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b802abe0 con 0x7f54b4003f60 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54cd08f640 1 -- 172.21.15.148:0/449597297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b4003f60 msgr2=0x7f54b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54cd08f640 1 --2- 172.21.15.148:0/449597297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54b4003f60 0x7f54b4004800 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001750 tx=0x7f54b4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54c759e640 1 -- 172.21.15.148:0/449597297 reap_dead start 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54cd08f640 1 -- 172.21.15.148:0/449597297 shutdown_connections 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54cd08f640 1 -- 172.21.15.148:0/449597297 >> 172.21.15.148:0/449597297 conn(0x7f54c80b2370 msgr2=0x7f54c80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54cd08f640 1 -- 172.21.15.148:0/449597297 shutdown_connections 2024-04-25T10:03:37.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.987+0000 7f54cd08f640 1 -- 172.21.15.148:0/449597297 wait complete. 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.988+0000 7f54cd08f640 1 Processor -- start 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.988+0000 7f54cd08f640 1 -- start start 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54cd08f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b4003f60 0x7f54c8159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54cd08f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 0x7f54c80ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54cd08f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c80ceed0 0x7f54c81f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54cd08f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c80b0010 con 0x7f54c814c0f0 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54cd08f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8070a10 con 0x7f54c80ceed0 2024-04-25T10:03:37.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54cd08f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8158600 con 0x7f54b4003f60 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 0x7f54c80ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 0x7f54c80ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45674/0 (socket says 172.21.15.148:45674) 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 -- 172.21.15.148:0/409922599 learned_addr learned my addr 172.21.15.148:0/409922599 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c6d9d640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b4003f60 0x7f54c8159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c759e640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c80ceed0 0x7f54c81f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 -- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b4003f60 msgr2=0x7f54c8159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b4003f60 0x7f54c8159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 -- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c80ceed0 msgr2=0x7f54c81f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c80ceed0 0x7f54c81f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.989+0000 7f54c659c640 1 -- 172.21.15.148:0/409922599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c815a3d0 con 0x7f54c814c0f0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.990+0000 7f54c659c640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 0x7f54c80ce990 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005040 tx=0x7f54b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.990+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8040020 con 0x7f54c814c0f0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.990+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8007680 con 0x7f54c814c0f0 2024-04-25T10:03:37.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.990+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8007a30 con 0x7f54c814c0f0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.990+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c815a6b0 con 0x7f54c814c0f0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.990+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c80c9e00 con 0x7f54c814c0f0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.991+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c8073990 con 0x7f54c814c0f0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.991+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54b8043020 con 0x7f54c814c0f0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.992+0000 7f54bf7fe640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a806b500 0x7f54a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.992+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f54b80aa300 con 0x7f54c814c0f0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.992+0000 7f54c6d9d640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a806b500 0x7f54a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.992+0000 7f54c6d9d640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a806b500 0x7f54a806d9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f54b0007fd0 tx=0x7f54b0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:37.996+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54b804a030 con 0x7f54c814c0f0 2024-04-25T10:03:38.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.202+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54c8156430 con 0x7f54c814c0f0 2024-04-25T10:03:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.203+0000 7f54bf7fe640 1 -- 172.21.15.148:0/409922599 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f54b8079030 con 0x7f54c814c0f0 2024-04-25T10:03:38.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:38.208 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:03:30.801958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9327":{"gid":9327,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:38.208 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a806b500 msgr2=0x7f54a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a806b500 0x7f54a806d9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f54b0007fd0 tx=0x7f54b0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 msgr2=0x7f54c80ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 0x7f54c80ce990 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005040 tx=0x7f54b8001120 comp rx=0 tx=0).stop 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 shutdown_connections 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54a806b500 0x7f54a806d9c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54c80ceed0 0x7f54c81f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54c814c0f0 0x7f54c80ce990 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 --2- 172.21.15.148:0/409922599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54b4003f60 0x7f54c8159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 >> 172.21.15.148:0/409922599 conn(0x7f54c80b2370 msgr2=0x7f54c8157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 shutdown_connections 2024-04-25T10:03:38.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.205+0000 7f54cd08f640 1 -- 172.21.15.148:0/409922599 wait complete. 2024-04-25T10:03:38.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:38.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.314+0000 7f3add95c640 1 Processor -- start 2024-04-25T10:03:38.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.314+0000 7f3add95c640 1 -- start start 2024-04-25T10:03:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3add95c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ad80b2cb0 0x7f3ad80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3add95c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80d1020 0x7f3ad80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3add95c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80b97a0 con 0x7f3ad815dbe0 2024-04-25T10:03:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3add95c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad815dfc0 con 0x7f3ad80b2cb0 2024-04-25T10:03:38.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3add95c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80d1a00 con 0x7f3ad80d1020 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3ad6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3ad815dbe0 0x7f3ad80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49662/0 (socket says 172.21.15.148:49662) 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3ad6ffd640 1 -- 172.21.15.148:0/3896146629 learned_addr learned my addr 172.21.15.148:0/3896146629 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3acffff640 1 --2- 172.21.15.148:0/3896146629 >> v2:172.21.15.188:3300/0 conn(0x7f3ad80b2cb0 0x7f3ad80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3ad77fe640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80d1020 0x7f3ad80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3ad67fc640 1 -- 172.21.15.148:0/3896146629 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171939003 0 0) 0x7f3ad80b97a0 con 0x7f3ad815dbe0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3ad67fc640 1 -- 172.21.15.148:0/3896146629 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac4003520 con 0x7f3ad815dbe0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3acffff640 1 -- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80d1020 msgr2=0x7f3ad80d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3acffff640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80d1020 0x7f3ad80d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3acffff640 1 -- 172.21.15.148:0/3896146629 >> v1:172.21.15.148:6789/0 conn(0x7f3ad815dbe0 legacy=0x7f3ad80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3acffff640 1 -- 172.21.15.148:0/3896146629 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80d1c50 con 0x7f3ad80b2cb0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.315+0000 7f3ad77fe640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80d1020 0x7f3ad80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.316+0000 7f3acffff640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad80b2cb0 0x7f3ad80d2730 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00097a0 tx=0x7f3ac002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=351b9038f4c59540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.316+0000 7f3ad67fc640 1 -- 172.21.15.148:0/3896146629 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0037070 con 0x7f3ad80b2cb0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.316+0000 7f3ad67fc640 1 -- 172.21.15.148:0/3896146629 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0032440 con 0x7f3ad80b2cb0 2024-04-25T10:03:38.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.316+0000 7f3ad67fc640 1 -- 172.21.15.148:0/3896146629 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00327f0 con 0x7f3ad80b2cb0 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 -- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad80b2cb0 msgr2=0x7f3ad80d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad80b2cb0 0x7f3ad80d2730 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00097a0 tx=0x7f3ac002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 -- 172.21.15.148:0/3896146629 shutdown_connections 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80d1020 0x7f3ad80d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 --2- 172.21.15.148:0/3896146629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad80b2cb0 0x7f3ad80d2730 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 -- 172.21.15.148:0/3896146629 >> 172.21.15.148:0/3896146629 conn(0x7f3ad8061a20 msgr2=0x7f3ad8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 -- 172.21.15.148:0/3896146629 shutdown_connections 2024-04-25T10:03:38.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.317+0000 7f3add95c640 1 -- 172.21.15.148:0/3896146629 wait complete. 2024-04-25T10:03:38.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.318+0000 7f3add95c640 1 Processor -- start 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.318+0000 7f3add95c640 1 -- start start 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3add95c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80b2cb0 0x7f3ad8146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3add95c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 0x7f3ad80b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3add95c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad815dbe0 0x7f3ad815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3add95c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b92a0 con 0x7f3ad80d1020 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3add95c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80d2210 con 0x7f3ad815dbe0 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3add95c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b9420 con 0x7f3ad80b2cb0 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 0x7f3ad80b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 0x7f3ad80b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45688/0 (socket says 172.21.15.148:45688) 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 -- 172.21.15.148:0/4084978401 learned_addr learned my addr 172.21.15.148:0/4084978401 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:38.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3ad6ffd640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80b2cb0 0x7f3ad8146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 -- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80b2cb0 msgr2=0x7f3ad8146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80b2cb0 0x7f3ad8146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 -- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad815dbe0 msgr2=0x7f3ad815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad815dbe0 0x7f3ad815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 -- 172.21.15.148:0/4084978401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80b5d30 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3acffff640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 0x7f3ad80b52c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3ac002a1c0 tx=0x7f3ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0037040 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.319+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0042070 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.320+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad80b6010 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.320+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac003b930 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.320+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad815ed50 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.320+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad8073990 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.321+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ac003bad0 con 0x7f3ad80d1020 2024-04-25T10:03:38.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.321+0000 7f3ad4ff9640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab806b4d0 0x7f3ab806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.321+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3ac00aabf0 con 0x7f3ad80d1020 2024-04-25T10:03:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.321+0000 7f3ad6ffd640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab806b4d0 0x7f3ab806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.322+0000 7f3ad6ffd640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab806b4d0 0x7f3ab806d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8000f60 tx=0x7f3ac8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.326+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ac0079800 con 0x7f3ad80d1020 2024-04-25T10:03:38.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.506+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ad815c800 con 0x7f3ad80d1020 2024-04-25T10:03:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.507+0000 7f3ad4ff9640 1 -- 172.21.15.148:0/4084978401 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17079 (secure 0 0 0) 0x7f3ac0079100 con 0x7f3ad80d1020 2024-04-25T10:03:38.513 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:38.513 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:28.777320+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/1740205817":"2024-04-26T10:03:28.777252+0000","172.21.15.148:6836/1740205817":"2024-04-26T10:03:28.777252+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab806b4d0 msgr2=0x7f3ab806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab806b4d0 0x7f3ab806d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8000f60 tx=0x7f3ac8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 msgr2=0x7f3ad80b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 0x7f3ad80b52c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3ac002a1c0 tx=0x7f3ac00046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 shutdown_connections 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3ab806b4d0 0x7f3ab806d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ad815dbe0 0x7f3ad815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad80d1020 0x7f3ad80b52c0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 --2- 172.21.15.148:0/4084978401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ad80b2cb0 0x7f3ad8146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 >> 172.21.15.148:0/4084978401 conn(0x7f3ad8061a20 msgr2=0x7f3ad80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 shutdown_connections 2024-04-25T10:03:38.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.510+0000 7f3add95c640 1 -- 172.21.15.148:0/4084978401 wait complete. 2024-04-25T10:03:38.526 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:38.526 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:38.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.618+0000 7fdc9ffe0640 1 Processor -- start 2024-04-25T10:03:38.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.618+0000 7fdc9ffe0640 1 -- start start 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9ffe0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdc980ce920 0x7fdc980cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9ffe0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc98144b30 con 0x7fdc9806d290 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9ffe0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc980712d0 con 0x7fdc98144750 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9ffe0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980715b0 con 0x7fdc980ce920 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9d57b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdc980ce920 0x7fdc980cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9e57d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdc9806d290 0x7fdc9806d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49670/0 (socket says 172.21.15.148:49670) 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9e57d640 1 -- 172.21.15.148:0/3145468203 learned_addr learned my addr 172.21.15.148:0/3145468203 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9d57b640 1 -- 172.21.15.148:0/3145468203 >> v1:172.21.15.188:6789/0 conn(0x7fdc98144750 legacy=0x7fdc9806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9d57b640 1 -- 172.21.15.148:0/3145468203 >> v1:172.21.15.148:6789/0 conn(0x7fdc9806d290 legacy=0x7fdc9806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9d57b640 1 -- 172.21.15.148:0/3145468203 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98071890 con 0x7fdc980ce920 2024-04-25T10:03:38.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.619+0000 7fdc9cd7a640 1 -- 172.21.15.148:0/3145468203 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443322004 0 0) 0x7fdc98144b30 con 0x7fdc9806d290 2024-04-25T10:03:38.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9d57b640 1 --2- 172.21.15.148:0/3145468203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc980cedb0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097a0 tx=0x7fdc8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2191e5045c5de4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9cd7a640 1 -- 172.21.15.148:0/3145468203 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88037070 con 0x7fdc980ce920 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9cd7a640 1 -- 172.21.15.148:0/3145468203 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88032440 con 0x7fdc980ce920 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9cd7a640 1 -- 172.21.15.148:0/3145468203 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88032810 con 0x7fdc980ce920 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/3145468203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 msgr2=0x7fdc980cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/3145468203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc980cedb0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097a0 tx=0x7fdc8802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/3145468203 shutdown_connections 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/3145468203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc980cedb0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.620+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/3145468203 >> 172.21.15.148:0/3145468203 conn(0x7fdc9806abe0 msgr2=0x7fdc9806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.621+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/3145468203 shutdown_connections 2024-04-25T10:03:38.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.621+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/3145468203 wait complete. 2024-04-25T10:03:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 Processor -- start 2024-04-25T10:03:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 -- start start 2024-04-25T10:03:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9806d290 0x7fdc980b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc9815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc98144750 0x7fdc9813c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9806f020 con 0x7fdc980ce920 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9806ece0 con 0x7fdc98144750 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9ffe0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc98145450 con 0x7fdc9806d290 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9d57b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc9815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9d57b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc9815d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45710/0 (socket says 172.21.15.148:45710) 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.622+0000 7fdc9d57b640 1 -- 172.21.15.148:0/1011113763 learned_addr learned my addr 172.21.15.148:0/1011113763 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9dd7c640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9806d290 0x7fdc980b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9e57d640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc98144750 0x7fdc9813c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9d57b640 1 -- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9806d290 msgr2=0x7fdc980b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9d57b640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9806d290 0x7fdc980b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9d57b640 1 -- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc98144750 msgr2=0x7fdc9813c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9d57b640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc98144750 0x7fdc9813c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9d57b640 1 -- 172.21.15.148:0/1011113763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9815dd90 con 0x7fdc980ce920 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9dd7c640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9806d290 0x7fdc980b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9e57d640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc98144750 0x7fdc9813c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9d57b640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc9815d2b0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097a0 tx=0x7fdc880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88041070 con 0x7fdc980ce920 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8803c870 con 0x7fdc980ce920 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9815e070 con 0x7fdc980ce920 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88037040 con 0x7fdc980ce920 2024-04-25T10:03:38.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.623+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9815e910 con 0x7fdc980ce920 2024-04-25T10:03:38.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.624+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9815e240 con 0x7fdc980ce920 2024-04-25T10:03:38.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.625+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc88044070 con 0x7fdc980ce920 2024-04-25T10:03:38.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.625+0000 7fdc8e7fc640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.625+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdc880aaf60 con 0x7fdc980ce920 2024-04-25T10:03:38.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.625+0000 7fdc9dd7c640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.625+0000 7fdc9dd7c640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdc94007fd0 tx=0x7fdc94009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.629+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc88079c00 con 0x7fdc980ce920 2024-04-25T10:03:38.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.840+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc98073990 con 0x7fdc980ce920 2024-04-25T10:03:38.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.840+0000 7fdc8e7fc640 1 -- 172.21.15.148:0/1011113763 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fdc88079500 con 0x7fdc980ce920 2024-04-25T10:03:38.845 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:38.845 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:03:30.801958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9327":{"gid":9327,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:38.846 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2024-04-25T10:03:38.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc7c06b4d0 msgr2=0x7fdc7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdc94007fd0 tx=0x7fdc94009040 comp rx=0 tx=0).stop 2024-04-25T10:03:38.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 msgr2=0x7fdc9815d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc9815d2b0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fdc880097a0 tx=0x7fdc880046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 shutdown_connections 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc7c06b4d0 0x7fdc7c06d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc98144750 0x7fdc9813c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980ce920 0x7fdc9815d2b0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 --2- 172.21.15.148:0/1011113763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9806d290 0x7fdc980b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 >> 172.21.15.148:0/1011113763 conn(0x7fdc9806abe0 msgr2=0x7fdc9806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 shutdown_connections 2024-04-25T10:03:38.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.842+0000 7fdc9ffe0640 1 -- 172.21.15.148:0/1011113763 wait complete. 2024-04-25T10:03:38.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.950+0000 7f5b54a1f640 1 Processor -- start 2024-04-25T10:03:38.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.950+0000 7f5b54a1f640 1 -- start start 2024-04-25T10:03:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b54a1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b5015b980 0x7f5b500cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b54a1f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b5006b5f0 0x7f5b500cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b54a1f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5015f710 con 0x7f5b500b2d40 2024-04-25T10:03:38.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b54a1f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5015f9b0 con 0x7f5b5006b5f0 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b54a1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5015fc90 con 0x7f5b5015b980 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b4dd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5b5006b5f0 0x7f5b500cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b4ed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5b500b2d40 0x7f5b500ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49684/0 (socket says 172.21.15.148:49684) 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.951+0000 7f5b4ed9d640 1 -- 172.21.15.148:0/3442465330 learned_addr learned my addr 172.21.15.148:0/3442465330 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4e59c640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b5015b980 0x7f5b500cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4dd9b640 1 -- 172.21.15.148:0/3442465330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b5015b980 msgr2=0x7f5b500cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4dd9b640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b5015b980 0x7f5b500cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4dd9b640 1 -- 172.21.15.148:0/3442465330 >> v1:172.21.15.148:6789/0 conn(0x7f5b500b2d40 legacy=0x7f5b500ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4d59a640 1 -- 172.21.15.148:0/3442465330 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4913446 0 0) 0x7f5b5015f710 con 0x7f5b500b2d40 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4dd9b640 1 -- 172.21.15.148:0/3442465330 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5015ff70 con 0x7f5b5006b5f0 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4e59c640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b5015b980 0x7f5b500cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4dd9b640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500cf8a0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f5b440097a0 tx=0x7f5b4402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd4cac61746932f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4d59a640 1 -- 172.21.15.148:0/3442465330 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44037070 con 0x7f5b5006b5f0 2024-04-25T10:03:38.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4d59a640 1 -- 172.21.15.148:0/3442465330 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b44032440 con 0x7f5b5006b5f0 2024-04-25T10:03:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.952+0000 7f5b4d59a640 1 -- 172.21.15.148:0/3442465330 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44032810 con 0x7f5b5006b5f0 2024-04-25T10:03:38.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 -- 172.21.15.148:0/3442465330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 msgr2=0x7f5b500cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500cf8a0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f5b440097a0 tx=0x7f5b4402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 -- 172.21.15.148:0/3442465330 shutdown_connections 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500cf8a0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/3442465330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b5015b980 0x7f5b500cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 -- 172.21.15.148:0/3442465330 >> 172.21.15.148:0/3442465330 conn(0x7f5b500b2220 msgr2=0x7f5b500b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 -- 172.21.15.148:0/3442465330 shutdown_connections 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.953+0000 7f5b54a1f640 1 -- 172.21.15.148:0/3442465330 wait complete. 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.954+0000 7f5b54a1f640 1 Processor -- start 2024-04-25T10:03:38.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 -- start start 2024-04-25T10:03:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b500b2d40 0x7f5b50159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b5015b980 0x7f5b500c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500cf550 con 0x7f5b5006b5f0 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5015bd60 con 0x7f5b5015b980 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b54a1f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500cfde0 con 0x7f5b500b2d40 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b4e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45734/0 (socket says 172.21.15.148:45734) 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b4e59c640 1 -- 172.21.15.148:0/1187698950 learned_addr learned my addr 172.21.15.148:0/1187698950 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b4ed9d640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b5015b980 0x7f5b500c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.955+0000 7f5b4dd9b640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b500b2d40 0x7f5b50159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b4e59c640 1 -- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b500b2d40 msgr2=0x7f5b50159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b4e59c640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b500b2d40 0x7f5b50159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b4e59c640 1 -- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b5015b980 msgr2=0x7f5b500c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b4e59c640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b5015b980 0x7f5b500c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b4e59c640 1 -- 172.21.15.148:0/1187698950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b500b50f0 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b4e59c640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500b4a10 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000c820 tx=0x7f5b4000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40015070 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b40007a30 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b500b53d0 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40007de0 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.956+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b500b5990 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.957+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b500d01f0 con 0x7f5b5006b5f0 2024-04-25T10:03:38.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.957+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b40018070 con 0x7f5b5006b5f0 2024-04-25T10:03:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.958+0000 7f5b3effd640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b3006b4d0 0x7f5b3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.958+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5b4008a3a0 con 0x7f5b5006b5f0 2024-04-25T10:03:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.958+0000 7f5b4dd9b640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b3006b4d0 0x7f5b3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.958+0000 7f5b4dd9b640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b3006b4d0 0x7f5b3006d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5b44034000 tx=0x7f5b4402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:38.962+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b500d01f0 con 0x7f5b5006b5f0 2024-04-25T10:03:39.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.167+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b500685b0 con 0x7f5b5006b5f0 2024-04-25T10:03:39.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.168+0000 7f5b3effd640 1 -- 172.21.15.148:0/1187698950 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f5b500685b0 con 0x7f5b5006b5f0 2024-04-25T10:03:39.173 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:39.173 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:49.163477+0000","modified":"2024-04-25T10:03:30.801958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9327},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9327":{"gid":9327,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6835/2921516856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2921516856},{"type":"v1","addr":"172.21.15.148:6835","nonce":2921516856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:03:39.173 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 78 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b3006b4d0 msgr2=0x7f5b3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b3006b4d0 0x7f5b3006d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5b44034000 tx=0x7f5b4402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 msgr2=0x7f5b500b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500b4a10 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000c820 tx=0x7f5b4000cce0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 shutdown_connections 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b3006b4d0 0x7f5b3006d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b5015b980 0x7f5b500c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b500b2d40 0x7f5b50159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 --2- 172.21.15.148:0/1187698950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b5006b5f0 0x7f5b500b4a10 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 >> 172.21.15.148:0/1187698950 conn(0x7f5b500b2220 msgr2=0x7f5b500b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 shutdown_connections 2024-04-25T10:03:39.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.170+0000 7f5b54a1f640 1 -- 172.21.15.148:0/1187698950 wait complete. 2024-04-25T10:03:39.186 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:39.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.277+0000 7f93a87b7640 1 Processor -- start 2024-04-25T10:03:39.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.277+0000 7f93a87b7640 1 -- start start 2024-04-25T10:03:39.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a87b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93a00cc8e0 0x7f93a01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a87b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a00cce50 con 0x7f93a00d1810 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a87b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a0161af0 con 0x7f93a015dc90 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a87b7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a0161dd0 con 0x7f93a00cc8e0 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a5d52640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93a00cc8e0 0x7f93a01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a6d54640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f93a00d1810 0x7f93a00cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49690/0 (socket says 172.21.15.148:49690) 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a6d54640 1 -- 172.21.15.148:0/420062875 learned_addr learned my addr 172.21.15.148:0/420062875 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a5d52640 1 -- 172.21.15.148:0/420062875 >> v1:172.21.15.188:6789/0 conn(0x7f93a015dc90 legacy=0x7f93a015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a5d52640 1 -- 172.21.15.148:0/420062875 >> v1:172.21.15.148:6789/0 conn(0x7f93a00d1810 legacy=0x7f93a00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a5d52640 1 -- 172.21.15.148:0/420062875 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a01620b0 con 0x7f93a00cc8e0 2024-04-25T10:03:39.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.278+0000 7f93a5551640 1 -- 172.21.15.148:0/420062875 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434625743 0 0) 0x7f93a00cce50 con 0x7f93a00d1810 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.279+0000 7f93a5d52640 1 --2- 172.21.15.148:0/420062875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a00cc8e0 0x7f93a01625d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f93900097a0 tx=0x7f939002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e53bd603e59925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.279+0000 7f93a5551640 1 -- 172.21.15.148:0/420062875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390037070 con 0x7f93a00cc8e0 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.279+0000 7f93a5551640 1 -- 172.21.15.148:0/420062875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939002ae50 con 0x7f93a00cc8e0 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.279+0000 7f93a5551640 1 -- 172.21.15.148:0/420062875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390032660 con 0x7f93a00cc8e0 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 -- 172.21.15.148:0/420062875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a00cc8e0 msgr2=0x7f93a01625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 --2- 172.21.15.148:0/420062875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a00cc8e0 0x7f93a01625d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f93900097a0 tx=0x7f939002a140 comp rx=0 tx=0).stop 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 -- 172.21.15.148:0/420062875 shutdown_connections 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 --2- 172.21.15.148:0/420062875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a00cc8e0 0x7f93a01625d0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 -- 172.21.15.148:0/420062875 >> 172.21.15.148:0/420062875 conn(0x7f93a006f590 msgr2=0x7f93a006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 -- 172.21.15.148:0/420062875 shutdown_connections 2024-04-25T10:03:39.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.280+0000 7f93a87b7640 1 -- 172.21.15.148:0/420062875 wait complete. 2024-04-25T10:03:39.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.281+0000 7f93a87b7640 1 Processor -- start 2024-04-25T10:03:39.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.281+0000 7f93a87b7640 1 -- start start 2024-04-25T10:03:39.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93a00cc8e0 0x7f93a00c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93a00d1810 0x7f93a00c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 0x7f93a00b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015e1f0 con 0x7f93a015dc90 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015e070 con 0x7f93a00d1810 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00d1c80 con 0x7f93a00cc8e0 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 0x7f93a00b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 0x7f93a00b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45762/0 (socket says 172.21.15.148:45762) 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 -- 172.21.15.148:0/2698441930 learned_addr learned my addr 172.21.15.148:0/2698441930 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a5d52640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93a00d1810 0x7f93a00c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6553640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93a00cc8e0 0x7f93a00c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 -- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93a00cc8e0 msgr2=0x7f93a00c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93a00cc8e0 0x7f93a00c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 -- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93a00d1810 msgr2=0x7f93a00c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93a00d1810 0x7f93a00c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 -- 172.21.15.148:0/2698441930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a00b5530 con 0x7f93a015dc90 2024-04-25T10:03:39.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a5d52640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93a00d1810 0x7f93a00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6553640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93a00cc8e0 0x7f93a00c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a6d54640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 0x7f93a00b5d10 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f939800c3f0 tx=0x7f939800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93980074e0 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93980047e0 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a00cb9d0 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398004bd0 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.282+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a00cbf90 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.284+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0073990 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.284+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9398011070 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.284+0000 7f9396ffd640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f938406b4d0 0x7f938406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.284+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9398089f90 con 0x7f93a015dc90 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.284+0000 7f93a6553640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f938406b4d0 0x7f938406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.285+0000 7f93a6553640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f938406b4d0 0x7f938406d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f939c007e90 tx=0x7f939c0073a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.288+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9398058c30 con 0x7f93a015dc90 2024-04-25T10:03:39.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.460+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93a00cf6b0 con 0x7f93a015dc90 2024-04-25T10:03:39.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.462+0000 7f9396ffd640 1 -- 172.21.15.148:0/2698441930 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+17079 (secure 0 0 0) 0x7f9398058530 con 0x7f93a015dc90 2024-04-25T10:03:39.467 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:39.467 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":134,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:28.777320+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:47.359949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:48.130345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/1740205817":"2024-04-26T10:03:28.777252+0000","172.21.15.148:6836/1740205817":"2024-04-26T10:03:28.777252+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:39.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f938406b4d0 msgr2=0x7f938406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f938406b4d0 0x7f938406d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f939c007e90 tx=0x7f939c0073a0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 msgr2=0x7f93a00b5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 0x7f93a00b5d10 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f939800c3f0 tx=0x7f939800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 shutdown_connections 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f938406b4d0 0x7f938406d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93a015dc90 0x7f93a00b5d10 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93a00d1810 0x7f93a00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 --2- 172.21.15.148:0/2698441930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93a00cc8e0 0x7f93a00c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 >> 172.21.15.148:0/2698441930 conn(0x7f93a006f590 msgr2=0x7f93a00cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 shutdown_connections 2024-04-25T10:03:39.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.464+0000 7f93a87b7640 1 -- 172.21.15.148:0/2698441930 wait complete. 2024-04-25T10:03:39.480 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:39.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.573+0000 7f249959e640 1 Processor -- start 2024-04-25T10:03:39.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.573+0000 7f249959e640 1 -- start start 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f249959e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f249415dc20 0x7f24940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f249959e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24940d32f0 0x7f24940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f249959e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249415e000 con 0x7f2494161710 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f249959e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24940d1de0 con 0x7f249415dc20 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f249959e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2494161c60 con 0x7f24940d32f0 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f249415dc20 0x7f24940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f249415dc20 0x7f24940ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45764/0 (socket says 172.21.15.148:45764) 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 -- 172.21.15.148:0/1897883525 learned_addr learned my addr 172.21.15.148:0/1897883525 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f24937fe640 1 --2- 172.21.15.148:0/1897883525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24940d32f0 0x7f24940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 -- 172.21.15.148:0/1897883525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24940d32f0 msgr2=0x7f24940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:39.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 --2- 172.21.15.148:0/1897883525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24940d32f0 0x7f24940cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 -- 172.21.15.148:0/1897883525 >> v1:172.21.15.188:6789/0 conn(0x7f2494161710 legacy=0x7f249406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.574+0000 7f2492ffd640 1 -- 172.21.15.148:0/1897883525 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494161f40 con 0x7f249415dc20 2024-04-25T10:03:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.575+0000 7f2492ffd640 1 --2- 172.21.15.148:0/1897883525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f249415dc20 0x7f24940ce670 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f2484009870 tx=0x7f248402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5780bd242c01ba68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.575+0000 7f24927fc640 1 -- 172.21.15.148:0/1897883525 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484037070 con 0x7f249415dc20 2024-04-25T10:03:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.575+0000 7f24927fc640 1 -- 172.21.15.148:0/1897883525 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2484032440 con 0x7f249415dc20 2024-04-25T10:03:39.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.575+0000 7f24927fc640 1 -- 172.21.15.148:0/1897883525 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484032810 con 0x7f249415dc20 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.575+0000 7f249959e640 1 -- 172.21.15.148:0/1897883525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f249415dc20 msgr2=0x7f24940ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.575+0000 7f249959e640 1 --2- 172.21.15.148:0/1897883525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f249415dc20 0x7f24940ce670 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f2484009870 tx=0x7f248402a210 comp rx=0 tx=0).stop 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.576+0000 7f249959e640 1 -- 172.21.15.148:0/1897883525 shutdown_connections 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.576+0000 7f249959e640 1 --2- 172.21.15.148:0/1897883525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24940d32f0 0x7f24940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.576+0000 7f249959e640 1 --2- 172.21.15.148:0/1897883525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f249415dc20 0x7f24940ce670 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.576+0000 7f249959e640 1 -- 172.21.15.148:0/1897883525 >> 172.21.15.148:0/1897883525 conn(0x7f24940d0420 msgr2=0x7f24940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.576+0000 7f249959e640 1 -- 172.21.15.148:0/1897883525 shutdown_connections 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.576+0000 7f249959e640 1 -- 172.21.15.148:0/1897883525 wait complete. 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.577+0000 7f249959e640 1 Processor -- start 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.577+0000 7f249959e640 1 -- start start 2024-04-25T10:03:39.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 0x7f24940cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249415dc20 0x7f2494160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2494161710 0x7f2494158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24940b5e50 con 0x7f24940d32f0 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494162500 con 0x7f249415dc20 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24940b5fd0 con 0x7f2494161710 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 0x7f24940cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 0x7f24940cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45776/0 (socket says 172.21.15.148:45776) 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 -- 172.21.15.148:0/3999985661 learned_addr learned my addr 172.21.15.148:0/3999985661 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f248a7fc640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249415dc20 0x7f2494160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f24937fe640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2494161710 0x7f2494158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 -- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2494161710 msgr2=0x7f2494158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2494161710 0x7f2494158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 -- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249415dc20 msgr2=0x7f2494160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249415dc20 0x7f2494160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 -- 172.21.15.148:0/3999985661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249415f7f0 con 0x7f24940d32f0 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f248a7fc640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249415dc20 0x7f2494160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f24937fe640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2494161710 0x7f2494158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f2492ffd640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 0x7f24940cd240 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f2484009870 tx=0x7f24840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484041020 con 0x7f24940d32f0 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24840306c0 con 0x7f24940d32f0 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484037040 con 0x7f24940d32f0 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24940cb7b0 con 0x7f24940d32f0 2024-04-25T10:03:39.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.578+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24940cc050 con 0x7f24940d32f0 2024-04-25T10:03:39.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.579+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2494073a00 con 0x7f24940d32f0 2024-04-25T10:03:39.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.580+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2484044020 con 0x7f24940d32f0 2024-04-25T10:03:39.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.580+0000 7f248bfff640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b500 0x7f247406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.580+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f24840aab30 con 0x7f24940d32f0 2024-04-25T10:03:39.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.580+0000 7f248a7fc640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b500 0x7f247406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.581+0000 7f248a7fc640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b500 0x7f247406d9c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f24940d45b0 tx=0x7f247c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.584+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f248404b030 con 0x7f24940d32f0 2024-04-25T10:03:39.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.791+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f24941f3e30 con 0x7f24940d32f0 2024-04-25T10:03:39.796 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:03:39.791+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.793+0000 7f248bfff640 1 -- 172.21.15.148:0/3999985661 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2484079860 con 0x7f24940d32f0 2024-04-25T10:03:39.797 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b500 msgr2=0x7f247406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b500 0x7f247406d9c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f24940d45b0 tx=0x7f247c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 msgr2=0x7f24940cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 0x7f24940cd240 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f2484009870 tx=0x7f24840046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 shutdown_connections 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b500 0x7f247406d9c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2494161710 0x7f2494158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249415dc20 0x7f2494160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 --2- 172.21.15.148:0/3999985661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24940d32f0 0x7f24940cd240 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 >> 172.21.15.148:0/3999985661 conn(0x7f24940d0420 msgr2=0x7f24940b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 shutdown_connections 2024-04-25T10:03:39.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.795+0000 7f249959e640 1 -- 172.21.15.148:0/3999985661 wait complete. 2024-04-25T10:03:39.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:39.837 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:03:39.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.908+0000 7ff6deda5640 1 Processor -- start 2024-04-25T10:03:39.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.908+0000 7ff6deda5640 1 -- start start 2024-04-25T10:03:39.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6deda5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6d80b38d0 0x7ff6d80b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6deda5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6d806bd20 0x7ff6d806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6deda5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d815e020 con 0x7ff6d806e360 2024-04-25T10:03:39.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6deda5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d806b890 con 0x7ff6d80b38d0 2024-04-25T10:03:39.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6deda5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d806ac80 con 0x7ff6d806bd20 2024-04-25T10:03:39.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6cffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff6d80b38d0 0x7ff6d80b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6dcb41640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff6d806e360 0x7ff6d806e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49706/0 (socket says 172.21.15.148:49706) 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6dcb41640 1 -- 172.21.15.148:0/2021338607 learned_addr learned my addr 172.21.15.148:0/2021338607 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6dd342640 1 --2- 172.21.15.148:0/2021338607 >> v2:172.21.15.188:3300/0 conn(0x7ff6d806bd20 0x7ff6d806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6cffff640 1 -- 172.21.15.148:0/2021338607 >> v2:172.21.15.188:3300/0 conn(0x7ff6d806bd20 msgr2=0x7ff6d806c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6cffff640 1 --2- 172.21.15.148:0/2021338607 >> v2:172.21.15.188:3300/0 conn(0x7ff6d806bd20 0x7ff6d806c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6cffff640 1 -- 172.21.15.148:0/2021338607 >> v1:172.21.15.148:6789/0 conn(0x7ff6d806e360 legacy=0x7ff6d806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6cffff640 1 -- 172.21.15.148:0/2021338607 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d806af60 con 0x7ff6d80b38d0 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6cf7fe640 1 -- 172.21.15.148:0/2021338607 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741168194 0 0) 0x7ff6d815e020 con 0x7ff6d806e360 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.909+0000 7ff6dd342640 1 --2- 172.21.15.148:0/2021338607 >> v2:172.21.15.188:3300/0 conn(0x7ff6d806bd20 0x7ff6d806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.910+0000 7ff6cffff640 1 --2- 172.21.15.148:0/2021338607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d80b38d0 0x7ff6d80b3d10 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80d0a70 tx=0x7ff6c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109bc31281d92d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.910+0000 7ff6cf7fe640 1 -- 172.21.15.148:0/2021338607 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8037070 con 0x7ff6d80b38d0 2024-04-25T10:03:39.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.910+0000 7ff6cf7fe640 1 -- 172.21.15.148:0/2021338607 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c8032440 con 0x7ff6d80b38d0 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.910+0000 7ff6cf7fe640 1 -- 172.21.15.148:0/2021338607 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8032810 con 0x7ff6d80b38d0 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 -- 172.21.15.148:0/2021338607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d80b38d0 msgr2=0x7ff6d80b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 --2- 172.21.15.148:0/2021338607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d80b38d0 0x7ff6d80b3d10 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80d0a70 tx=0x7ff6c802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 -- 172.21.15.148:0/2021338607 shutdown_connections 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 --2- 172.21.15.148:0/2021338607 >> v2:172.21.15.188:3300/0 conn(0x7ff6d806bd20 0x7ff6d806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 --2- 172.21.15.148:0/2021338607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d80b38d0 0x7ff6d80b3d10 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 -- 172.21.15.148:0/2021338607 >> 172.21.15.148:0/2021338607 conn(0x7ff6d80cfec0 msgr2=0x7ff6d80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 -- 172.21.15.148:0/2021338607 shutdown_connections 2024-04-25T10:03:39.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.911+0000 7ff6deda5640 1 -- 172.21.15.148:0/2021338607 wait complete. 2024-04-25T10:03:39.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.912+0000 7ff6deda5640 1 Processor -- start 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.912+0000 7ff6deda5640 1 -- start start 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 0x7ff6d814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d806e360 0x7ff6d80c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d80b38d0 0x7ff6d80ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d806ee50 con 0x7ff6d806bd20 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d806b520 con 0x7ff6d806e360 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80b2fd0 con 0x7ff6d80b38d0 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 0x7ff6d814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 0x7ff6d814e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45812/0 (socket says 172.21.15.148:45812) 2024-04-25T10:03:39.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 -- 172.21.15.148:0/437516818 learned_addr learned my addr 172.21.15.148:0/437516818 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 -- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d80b38d0 msgr2=0x7ff6d80ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d80b38d0 0x7ff6d80ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 -- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d806e360 msgr2=0x7ff6d80c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6cffff640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d806e360 0x7ff6d80c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d806e360 0x7ff6d80c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 -- 172.21.15.148:0/437516818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d80af460 con 0x7ff6d806bd20 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6cffff640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d806e360 0x7ff6d80c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6dcb41640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 0x7ff6d814e170 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c4d0 tx=0x7ff6d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4018070 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d4007500 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d80af740 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.913+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4004a20 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.914+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d80afd00 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.915+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8073a00 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.915+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6d401b070 con 0x7ff6d806bd20 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.915+0000 7ff6cd7fa640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff6c406b500 0x7ff6c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.916+0000 7ff6cffff640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff6c406b500 0x7ff6c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.916+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff6d4089720 con 0x7ff6d806bd20 2024-04-25T10:03:39.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.916+0000 7ff6cffff640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff6c406b500 0x7ff6c406d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8034000 tx=0x7ff6c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:39.920+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6d4058360 con 0x7ff6d806bd20 2024-04-25T10:03:40.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.131+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6d80ce1e0 con 0x7ff6d806bd20 2024-04-25T10:03:40.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.820+0000 7ff6cd7fa640 1 -- 172.21.15.148:0/437516818 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff6d4057c60 con 0x7ff6d806bd20 2024-04-25T10:03:40.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff6c406b500 msgr2=0x7ff6c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff6c406b500 0x7ff6c406d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8034000 tx=0x7ff6c802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 msgr2=0x7ff6d814e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 0x7ff6d814e170 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c4d0 tx=0x7ff6d400c990 comp rx=0 tx=0).stop 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 shutdown_connections 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff6c406b500 0x7ff6c406d9c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6d80b38d0 0x7ff6d80ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6d806e360 0x7ff6d80c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 --2- 172.21.15.148:0/437516818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6d806bd20 0x7ff6d814e170 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 >> 172.21.15.148:0/437516818 conn(0x7ff6d80cfec0 msgr2=0x7ff6d806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 shutdown_connections 2024-04-25T10:03:40.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.822+0000 7ff6deda5640 1 -- 172.21.15.148:0/437516818 wait complete. 2024-04-25T10:03:40.840 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:40.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.979+0000 7f6230083640 1 Processor -- start 2024-04-25T10:03:40.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.979+0000 7f6230083640 1 -- start start 2024-04-25T10:03:40.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f6230083640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f622815dc20 0x7f62280ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f6230083640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6228161710 0x7f62280cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f6230083640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f622806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f6230083640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622815e000 con 0x7f622815dc20 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f6230083640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62280d1de0 con 0x7f6228161710 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f6230083640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6228161c60 con 0x7f62280b9330 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f622de1f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f622815dc20 0x7f62280ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f622de1f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f622815dc20 0x7f62280ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45824/0 (socket says 172.21.15.148:45824) 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.980+0000 7f622de1f640 1 -- 172.21.15.148:0/543112230 learned_addr learned my addr 172.21.15.148:0/543112230 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622de1f640 1 -- 172.21.15.148:0/543112230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 msgr2=0x7f622806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622de1f640 1 --2- 172.21.15.148:0/543112230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f622806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622de1f640 1 -- 172.21.15.148:0/543112230 >> v2:172.21.15.188:3300/0 conn(0x7f6228161710 msgr2=0x7f62280cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622d61e640 1 --2- 172.21.15.148:0/543112230 >> v2:172.21.15.188:3300/0 conn(0x7f6228161710 0x7f62280cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622de1f640 1 --2- 172.21.15.148:0/543112230 >> v2:172.21.15.188:3300/0 conn(0x7f6228161710 0x7f62280cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622de1f640 1 -- 172.21.15.148:0/543112230 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228161f40 con 0x7f622815dc20 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622de1f640 1 --2- 172.21.15.148:0/543112230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f62280ce670 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f622806da90 tx=0x7f622402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1ca49068814799 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.981+0000 7f622ce1d640 1 -- 172.21.15.148:0/543112230 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224036070 con 0x7f622815dc20 2024-04-25T10:03:40.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f622ce1d640 1 -- 172.21.15.148:0/543112230 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224031600 con 0x7f622815dc20 2024-04-25T10:03:40.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f622ce1d640 1 -- 172.21.15.148:0/543112230 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240319d0 con 0x7f622815dc20 2024-04-25T10:03:40.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 -- 172.21.15.148:0/543112230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 msgr2=0x7f62280ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 --2- 172.21.15.148:0/543112230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f62280ce670 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f622806da90 tx=0x7f622402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 -- 172.21.15.148:0/543112230 shutdown_connections 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 --2- 172.21.15.148:0/543112230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f622806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 --2- 172.21.15.148:0/543112230 >> v2:172.21.15.188:3300/0 conn(0x7f6228161710 0x7f62280cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 --2- 172.21.15.148:0/543112230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f62280ce670 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 -- 172.21.15.148:0/543112230 >> 172.21.15.148:0/543112230 conn(0x7f62280d0420 msgr2=0x7f62280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 -- 172.21.15.148:0/543112230 shutdown_connections 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.982+0000 7f6230083640 1 -- 172.21.15.148:0/543112230 wait complete. 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.983+0000 7f6230083640 1 Processor -- start 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.983+0000 7f6230083640 1 -- start start 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f6230083640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f62280cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f6230083640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f622815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f6230083640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228161710 0x7f62280b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f6230083640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280b5e50 con 0x7f622815dc20 2024-04-25T10:03:40.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f6230083640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6228162500 con 0x7f6228161710 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f6230083640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62280b5fd0 con 0x7f62280b9330 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f622815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f622815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45838/0 (socket says 172.21.15.148:45838) 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 -- 172.21.15.148:0/2107024561 learned_addr learned my addr 172.21.15.148:0/2107024561 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622de1f640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f62280cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622e620640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228161710 0x7f62280b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 -- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 msgr2=0x7f62280cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f62280cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 -- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228161710 msgr2=0x7f62280b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228161710 0x7f62280b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622d61e640 1 -- 172.21.15.148:0/2107024561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62281f7900 con 0x7f622815dc20 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.984+0000 7f622e620640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228161710 0x7f62280b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.985+0000 7f622d61e640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f622815ceb0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f621800c7f0 tx=0x7f621800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.985+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218015070 con 0x7f622815dc20 2024-04-25T10:03:40.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.985+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218004220 con 0x7f622815dc20 2024-04-25T10:03:40.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.985+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218007620 con 0x7f622815dc20 2024-04-25T10:03:40.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.985+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62280cbbf0 con 0x7f622815dc20 2024-04-25T10:03:40.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.985+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62280cc1b0 con 0x7f622815dc20 2024-04-25T10:03:40.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.986+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6228073a00 con 0x7f622815dc20 2024-04-25T10:03:40.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.986+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62180079d0 con 0x7f622815dc20 2024-04-25T10:03:40.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.987+0000 7f621e7fc640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f620c06b500 0x7f620c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.987+0000 7f622de1f640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f620c06b500 0x7f620c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.987+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f621808a2f0 con 0x7f622815dc20 2024-04-25T10:03:40.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.987+0000 7f622de1f640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f620c06b500 0x7f620c06d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6224033000 tx=0x7f6224009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:40.991+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6218058970 con 0x7f622815dc20 2024-04-25T10:03:41.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.165+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f62280cc380 con 0x7f622815dc20 2024-04-25T10:03:41.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.831+0000 7f621e7fc640 1 -- 172.21.15.148:0/2107024561 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) v1 ==== 181+0+0 (secure 0 0 0) 0x7f621805ca10 con 0x7f622815dc20 2024-04-25T10:03:41.836 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.834+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f620c06b500 msgr2=0x7f620c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.834+0000 7f6230083640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f620c06b500 0x7f620c06d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6224033000 tx=0x7f6224009000 comp rx=0 tx=0).stop 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.834+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 msgr2=0x7f622815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.834+0000 7f6230083640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f622815ceb0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f621800c7f0 tx=0x7f621800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 shutdown_connections 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f620c06b500 0x7f620c06d9c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6228161710 0x7f62280b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f622815dc20 0x7f622815ceb0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 --2- 172.21.15.148:0/2107024561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62280b9330 0x7f62280cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 >> 172.21.15.148:0/2107024561 conn(0x7f62280d0420 msgr2=0x7f62280c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 shutdown_connections 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.835+0000 7f6230083640 1 -- 172.21.15.148:0/2107024561 wait complete. 2024-04-25T10:03:41.853 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 Processor -- start 2024-04-25T10:03:41.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 -- start start 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbd0070dc0 0x7fbbd006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd00711a0 0x7fbbd00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd00d1770 con 0x7fbbd00b4830 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd01622b0 con 0x7fbbd0070dc0 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.951+0000 7fbbd6777640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd00b10f0 con 0x7fbbd00711a0 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbd0070dc0 0x7fbbd006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbbd0070dc0 0x7fbbd006ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45844/0 (socket says 172.21.15.148:45844) 2024-04-25T10:03:41.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 -- 172.21.15.148:0/526390362 learned_addr learned my addr 172.21.15.148:0/526390362 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:41.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcf7fe640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd00711a0 0x7fbbd00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 -- 172.21.15.148:0/526390362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd00711a0 msgr2=0x7fbbd00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd00711a0 0x7fbbd00b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 -- 172.21.15.148:0/526390362 >> v1:172.21.15.188:6789/0 conn(0x7fbbd00b4830 legacy=0x7fbbd00b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcffff640 1 -- 172.21.15.148:0/526390362 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd0162740 con 0x7fbbd0070dc0 2024-04-25T10:03:41.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.952+0000 7fbbcf7fe640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd00711a0 0x7fbbd00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbcffff640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd0070dc0 0x7fbbd006ccd0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4863a1116839d469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbceffd640 1 -- 172.21.15.148:0/526390362 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0037070 con 0x7fbbd0070dc0 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbceffd640 1 -- 172.21.15.148:0/526390362 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc0032440 con 0x7fbbd0070dc0 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbceffd640 1 -- 172.21.15.148:0/526390362 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0032810 con 0x7fbbd0070dc0 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 -- 172.21.15.148:0/526390362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd0070dc0 msgr2=0x7fbbd006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd0070dc0 0x7fbbd006ccd0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc002a130 comp rx=0 tx=0).stop 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 -- 172.21.15.148:0/526390362 shutdown_connections 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd00711a0 0x7fbbd00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 --2- 172.21.15.148:0/526390362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd0070dc0 0x7fbbd006ccd0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 -- 172.21.15.148:0/526390362 >> 172.21.15.148:0/526390362 conn(0x7fbbd00b3530 msgr2=0x7fbbd00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 -- 172.21.15.148:0/526390362 shutdown_connections 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.953+0000 7fbbd6777640 1 -- 172.21.15.148:0/526390362 wait complete. 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.954+0000 7fbbd6777640 1 Processor -- start 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 -- start start 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd0070dc0 0x7fbbd0152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 0x7fbbd0154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd00b4830 0x7fbbd01552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd0161f40 con 0x7fbbd00711a0 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd0161c60 con 0x7fbbd00b4830 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbd6777640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd0071600 con 0x7fbbd0070dc0 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.955+0000 7fbbcf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 0x7fbbd0154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 0x7fbbd0154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45856/0 (socket says 172.21.15.148:45856) 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 -- 172.21.15.148:0/3271298253 learned_addr learned my addr 172.21.15.148:0/3271298253 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcffff640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd0070dc0 0x7fbbd0152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbd4d14640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd00b4830 0x7fbbd01552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 -- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd0070dc0 msgr2=0x7fbbd0152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd0070dc0 0x7fbbd0152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 -- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd00b4830 msgr2=0x7fbbd01552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd00b4830 0x7fbbd01552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 -- 172.21.15.148:0/3271298253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd015fbb0 con 0x7fbbd00711a0 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbd4d14640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd00b4830 0x7fbbd01552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbcf7fe640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 0x7fbbd0154d70 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400c7f0 tx=0x7fbbc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4015070 con 0x7fbbd00711a0 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc4007a30 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4007e00 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd015fe90 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.956+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd00cd840 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.957+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd01603b0 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.958+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbc4018070 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.958+0000 7fbbccff9640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbbc06b4d0 0x7fbbbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.958+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fbbc408a1a0 con 0x7fbbd00711a0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.958+0000 7fbbcffff640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbbc06b4d0 0x7fbbbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.959+0000 7fbbcffff640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbbc06b4d0 0x7fbbbc06d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0034000 tx=0x7fbbc002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:41.962+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbc4059030 con 0x7fbbd00711a0 2024-04-25T10:03:42.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.134+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbbd0073a00 con 0x7fbbd00711a0 2024-04-25T10:03:42.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.838+0000 7fbbccff9640 1 -- 172.21.15.148:0/3271298253 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbbc4058930 con 0x7fbbd00711a0 2024-04-25T10:03:42.843 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:42.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbbc06b4d0 msgr2=0x7fbbbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbbc06b4d0 0x7fbbbc06d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0034000 tx=0x7fbbc002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:42.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 msgr2=0x7fbbd0154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 0x7fbbd0154d70 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400c7f0 tx=0x7fbbc400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 shutdown_connections 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbbbc06b4d0 0x7fbbbc06d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbd00b4830 0x7fbbd01552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbd00711a0 0x7fbbd0154d70 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 --2- 172.21.15.148:0/3271298253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbd0070dc0 0x7fbbd0152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.841+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 >> 172.21.15.148:0/3271298253 conn(0x7fbbd00b3530 msgr2=0x7fbbd015d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.842+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 shutdown_connections 2024-04-25T10:03:42.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.843+0000 7fbbd6777640 1 -- 172.21.15.148:0/3271298253 wait complete. 2024-04-25T10:03:42.865 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:03:42.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.968+0000 7f268b088640 1 Processor -- start 2024-04-25T10:03:42.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.968+0000 7f268b088640 1 -- start start 2024-04-25T10:03:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.968+0000 7f268b088640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26840ce230 0x7f26840cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f268b088640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2684072280 con 0x7f26840cc4f0 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f268b088640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26840714c0 con 0x7f268415b270 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f268b088640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2684071270 con 0x7f26840ce230 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f2683fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f26840ce230 0x7f26840cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f2688e24640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f26840cc4f0 0x7f26840cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49718/0 (socket says 172.21.15.148:49718) 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f2688e24640 1 -- 172.21.15.148:0/2622118547 learned_addr learned my addr 172.21.15.148:0/2622118547 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f2683fff640 1 -- 172.21.15.148:0/2622118547 >> v1:172.21.15.188:6789/0 conn(0x7f268415b270 legacy=0x7f268415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f2683fff640 1 -- 172.21.15.148:0/2622118547 >> v1:172.21.15.148:6789/0 conn(0x7f26840cc4f0 legacy=0x7f26840cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f2683fff640 1 -- 172.21.15.148:0/2622118547 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840717a0 con 0x7f26840ce230 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.969+0000 7f26837fe640 1 -- 172.21.15.148:0/2622118547 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038188843 0 0) 0x7f2684072280 con 0x7f26840cc4f0 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f2683fff640 1 --2- 172.21.15.148:0/2622118547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f26840cc8d0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f26700097a0 tx=0x7f267002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70a75f943689629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f26837fe640 1 -- 172.21.15.148:0/2622118547 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670037070 con 0x7f26840ce230 2024-04-25T10:03:42.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f26837fe640 1 -- 172.21.15.148:0/2622118547 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670032440 con 0x7f26840ce230 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f26837fe640 1 -- 172.21.15.148:0/2622118547 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670032810 con 0x7f26840ce230 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 -- 172.21.15.148:0/2622118547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 msgr2=0x7f26840cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 --2- 172.21.15.148:0/2622118547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f26840cc8d0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f26700097a0 tx=0x7f267002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 -- 172.21.15.148:0/2622118547 shutdown_connections 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 --2- 172.21.15.148:0/2622118547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f26840cc8d0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 -- 172.21.15.148:0/2622118547 >> 172.21.15.148:0/2622118547 conn(0x7f268406abe0 msgr2=0x7f268406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 -- 172.21.15.148:0/2622118547 shutdown_connections 2024-04-25T10:03:42.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.970+0000 7f268b088640 1 -- 172.21.15.148:0/2622118547 wait complete. 2024-04-25T10:03:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.971+0000 7f268b088640 1 Processor -- start 2024-04-25T10:03:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 -- start start 2024-04-25T10:03:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26840cc4f0 0x7f268415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f268414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268415b270 0x7f2684156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268406e8a0 con 0x7f26840ce230 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2684071d60 con 0x7f268415b270 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f268b088640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26840ae500 con 0x7f26840cc4f0 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.972+0000 7f2683fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f268414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f268414e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45872/0 (socket says 172.21.15.148:45872) 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 -- 172.21.15.148:0/2733083996 learned_addr learned my addr 172.21.15.148:0/2733083996 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2688e24640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26840cc4f0 0x7f268415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2689625640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268415b270 0x7f2684156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 -- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26840cc4f0 msgr2=0x7f268415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26840cc4f0 0x7f268415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 -- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268415b270 msgr2=0x7f2684156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268415b270 0x7f2684156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 -- 172.21.15.148:0/2733083996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684158f40 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f2683fff640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f268414e060 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f26700098c0 tx=0x7f26700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670041070 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267003c870 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26840b1130 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670037040 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.973+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26840b16f0 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.974+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26841598e0 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.975+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2670044070 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.975+0000 7f26817fa640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f266806b4d0 0x7f266806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.975+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f26700aac30 con 0x7f26840ce230 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.975+0000 7f2688e24640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f266806b4d0 0x7f266806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.975+0000 7f2688e24640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f266806b4d0 0x7f266806d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2678000f60 tx=0x7f2678007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:42.979+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26841598e0 con 0x7f26840ce230 2024-04-25T10:03:43.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.153+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2684073a00 con 0x7f26840ce230 2024-04-25T10:03:43.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.156+0000 7f26817fa640 1 -- 172.21.15.148:0/2733083996 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2670079cf0 con 0x7f26840ce230 2024-04-25T10:03:43.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f266806b4d0 msgr2=0x7f266806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f266806b4d0 0x7f266806d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2678000f60 tx=0x7f2678007c00 comp rx=0 tx=0).stop 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 msgr2=0x7f268414e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f268414e060 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f26700098c0 tx=0x7f26700046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 shutdown_connections 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f266806b4d0 0x7f266806d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f268415b270 0x7f2684156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26840ce230 0x7f268414e060 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 --2- 172.21.15.148:0/2733083996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26840cc4f0 0x7f268415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 >> 172.21.15.148:0/2733083996 conn(0x7f268406abe0 msgr2=0x7f26841591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 shutdown_connections 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.158+0000 7f268b088640 1 -- 172.21.15.148:0/2733083996 wait complete. 2024-04-25T10:03:43.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:03:43.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 Processor -- start 2024-04-25T10:03:43.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 -- start start 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00d415dc90 0x7f00d41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00d40cc8e0 0x7f00d415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d415f390 con 0x7f00d40d1810 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d40cce50 con 0x7f00d40cc8e0 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.281+0000 7f00db01c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d4161af0 con 0x7f00d415dc90 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00d40cc8e0 0x7f00d415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f00d40cc8e0 0x7f00d415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45880/0 (socket says 172.21.15.148:45880) 2024-04-25T10:03:43.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 -- 172.21.15.148:0/2398861446 learned_addr learned my addr 172.21.15.148:0/2398861446 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:43.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 -- 172.21.15.148:0/2398861446 >> v2:172.21.15.188:3300/0 conn(0x7f00d415dc90 msgr2=0x7f00d41625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 --2- 172.21.15.148:0/2398861446 >> v2:172.21.15.188:3300/0 conn(0x7f00d415dc90 0x7f00d41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 -- 172.21.15.148:0/2398861446 >> v1:172.21.15.188:6789/0 conn(0x7f00d40d1810 legacy=0x7f00d40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.282+0000 7f00d3fff640 1 -- 172.21.15.148:0/2398861446 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d4161dd0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00d3fff640 1 --2- 172.21.15.148:0/2398861446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d415ee50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f00d406b820 tx=0x7f00c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3e4be95edc37bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00d37fe640 1 -- 172.21.15.148:0/2398861446 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4037070 con 0x7f00d40cc8e0 2024-04-25T10:03:43.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00d37fe640 1 -- 172.21.15.148:0/2398861446 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c4032440 con 0x7f00d40cc8e0 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00d37fe640 1 -- 172.21.15.148:0/2398861446 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4032810 con 0x7f00d40cc8e0 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 -- 172.21.15.148:0/2398861446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 msgr2=0x7f00d415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 --2- 172.21.15.148:0/2398861446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d415ee50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f00d406b820 tx=0x7f00c402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 -- 172.21.15.148:0/2398861446 shutdown_connections 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 --2- 172.21.15.148:0/2398861446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d415ee50 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 --2- 172.21.15.148:0/2398861446 >> v2:172.21.15.188:3300/0 conn(0x7f00d415dc90 0x7f00d41625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 -- 172.21.15.148:0/2398861446 >> 172.21.15.148:0/2398861446 conn(0x7f00d406f590 msgr2=0x7f00d406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 -- 172.21.15.148:0/2398861446 shutdown_connections 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.283+0000 7f00db01c640 1 -- 172.21.15.148:0/2398861446 wait complete. 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.284+0000 7f00db01c640 1 Processor -- start 2024-04-25T10:03:43.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.284+0000 7f00db01c640 1 -- start start 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00db01c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00db01c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d40d1810 0x7f00d414f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00db01c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415dc90 0x7f00d40ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00db01c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415e070 con 0x7f00d40cc8e0 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00db01c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40cd130 con 0x7f00d40d1810 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00db01c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415e1f0 con 0x7f00d415dc90 2024-04-25T10:03:43.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00d8db8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00d8db8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d414c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45894/0 (socket says 172.21.15.148:45894) 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.285+0000 7f00d8db8640 1 -- 172.21.15.148:0/1158046315 learned_addr learned my addr 172.21.15.148:0/1158046315 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d3fff640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d40d1810 0x7f00d414f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d95b9640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415dc90 0x7f00d40ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d8db8640 1 -- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415dc90 msgr2=0x7f00d40ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d8db8640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415dc90 0x7f00d40ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d8db8640 1 -- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d40d1810 msgr2=0x7f00d414f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d8db8640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d40d1810 0x7f00d414f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d8db8640 1 -- 172.21.15.148:0/1158046315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d414f9e0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d3fff640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d40d1810 0x7f00d414f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d95b9640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415dc90 0x7f00d40ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d8db8640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d414c310 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00ba50 tx=0x7f00cc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc019070 con 0x7f00d40cc8e0 2024-04-25T10:03:43.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc00cba0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc014440 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d414cdc0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.286+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d414d380 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.287+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d4073990 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.287+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00cc01c070 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.288+0000 7f00d17fa640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f00b406b500 0x7f00b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.288+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f00cc089e20 con 0x7f00d40cc8e0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.288+0000 7f00d3fff640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f00b406b500 0x7f00b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.288+0000 7f00d3fff640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f00b406b500 0x7f00b406d9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f00c4034000 tx=0x7f00c402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.292+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00cc058dc0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.527+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f00d40cf6b0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.848+0000 7f00d17fa640 1 -- 172.21.15.148:0/1158046315 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v458) v1 ==== 166+0+0 (secure 0 0 0) 0x7f00cc0586c0 con 0x7f00d40cc8e0 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f00b406b500 msgr2=0x7f00b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f00b406b500 0x7f00b406d9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f00c4034000 tx=0x7f00c402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 msgr2=0x7f00d414c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d414c310 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00ba50 tx=0x7f00cc00bf10 comp rx=0 tx=0).stop 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 shutdown_connections 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f00b406b500 0x7f00b406d9c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415dc90 0x7f00d40ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d40d1810 0x7f00d414f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 --2- 172.21.15.148:0/1158046315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00d40cc8e0 0x7f00d414c310 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 >> 172.21.15.148:0/1158046315 conn(0x7f00d406f590 msgr2=0x7f00d40b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 shutdown_connections 2024-04-25T10:03:43.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.850+0000 7f00db01c640 1 -- 172.21.15.148:0/1158046315 wait complete. 2024-04-25T10:03:43.867 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:03:43.867 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:03:43.868 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:43.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.960+0000 7f858db53640 1 Processor -- start 2024-04-25T10:03:43.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.960+0000 7f858db53640 1 -- start start 2024-04-25T10:03:43.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f858db53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85880cfec0 0x7f85880cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f858db53640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8588071480 con 0x7f85880ce3c0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f858db53640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8588071760 con 0x7f8588154ee0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f858db53640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880ce830 con 0x7f85880cfec0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f8587fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f85880ce3c0 0x7f858806d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35938/0 (socket says 172.21.15.148:35938) 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f8587fff640 1 -- 172.21.15.148:0/4132249047 learned_addr learned my addr 172.21.15.148:0/4132249047 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965102091 0 0) 0x7f8588071480 con 0x7f85880ce3c0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8570003520 con 0x7f85880ce3c0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f8586ffd640 1 --2- 172.21.15.148:0/4132249047 >> v2:172.21.15.188:3300/0 conn(0x7f85880cfec0 0x7f85880cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178009520 0 0) 0x7f8570003520 con 0x7f85880ce3c0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 >> v2:172.21.15.188:3300/0 conn(0x7f85880cfec0 msgr2=0x7f85880cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 --2- 172.21.15.148:0/4132249047 >> v2:172.21.15.188:3300/0 conn(0x7f85880cfec0 0x7f85880cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 >> v1:172.21.15.188:6789/0 conn(0x7f8588154ee0 legacy=0x7f85880cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588071a40 con 0x7f85880ce3c0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f85800019f0 con 0x7f85880ce3c0 2024-04-25T10:03:43.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 >> v1:172.21.15.148:6789/0 conn(0x7f85880ce3c0 legacy=0x7f858806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8570004800 0x7f8570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85700051c0 0x7f857000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85880ce830 con 0x7f85700039b0 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857000c060 con 0x7f8570004800 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857000c260 con 0x7f85700051c0 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.961+0000 7f85877fe640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f8586ffd640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8570004800 0x7f8570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f8587fff640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85700051c0 0x7f857000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f85877fe640 1 -- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85700051c0 msgr2=0x7f857000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f85877fe640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85700051c0 0x7f857000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f85877fe640 1 -- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8570004800 msgr2=0x7f8570004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f85877fe640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8570004800 0x7f8570004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f85877fe640 1 -- 172.21.15.148:0/4132249047 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8570003520 con 0x7f85700039b0 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f8587fff640 1 -- 172.21.15.148:0/4132249047 reap_dead start 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.962+0000 7f85877fe640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f8578001750 tx=0x7f8570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7798ac6bb29c282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578001370 con 0x7f85700039b0 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85780052d0 con 0x7f85700039b0 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f85867fc640 1 -- 172.21.15.148:0/4132249047 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857802abe0 con 0x7f85700039b0 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 -- 172.21.15.148:0/4132249047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 msgr2=0x7f8570003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f8578001750 tx=0x7f8570005830 comp rx=0 tx=0).stop 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 -- 172.21.15.148:0/4132249047 shutdown_connections 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 --2- 172.21.15.148:0/4132249047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f8570003dd0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 -- 172.21.15.148:0/4132249047 >> 172.21.15.148:0/4132249047 conn(0x7f858806abe0 msgr2=0x7f858806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 -- 172.21.15.148:0/4132249047 shutdown_connections 2024-04-25T10:03:43.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.963+0000 7f858db53640 1 -- 172.21.15.148:0/4132249047 wait complete. 2024-04-25T10:03:43.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.964+0000 7f858db53640 1 Processor -- start 2024-04-25T10:03:43.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.964+0000 7f858db53640 1 -- start start 2024-04-25T10:03:43.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f858db53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f858813f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f858db53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858814ebd0 0x7f858813bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f858db53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85881531a0 0x7f8588153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f858db53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588071230 con 0x7f85700039b0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f858db53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85881552c0 con 0x7f858814ebd0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f858db53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8588155440 con 0x7f85881531a0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f858813f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f858813f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46204/0 (socket says 172.21.15.148:46204) 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 -- 172.21.15.148:0/1081712051 learned_addr learned my addr 172.21.15.148:0/1081712051 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f8587fff640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85881531a0 0x7f8588153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f8586ffd640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858814ebd0 0x7f858813bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 -- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85881531a0 msgr2=0x7f8588153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85881531a0 0x7f8588153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 -- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858814ebd0 msgr2=0x7f858813bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858814ebd0 0x7f858813bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.965+0000 7f85877fe640 1 -- 172.21.15.148:0/1081712051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858813f7f0 con 0x7f85700039b0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.966+0000 7f85877fe640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f858813f2b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8578005040 tx=0x7f8578001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.966+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857803f030 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.966+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578005480 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.966+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858814f170 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.966+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578037040 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.966+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858814f730 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.967+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588073990 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.967+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8578042070 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.968+0000 7f856ffff640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f857406b4d0 0x7f857406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.968+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f85780a96b0 con 0x7f85700039b0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.968+0000 7f8586ffd640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f857406b4d0 0x7f857406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.968+0000 7f8586ffd640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f857406b4d0 0x7f857406d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f85880b1f30 tx=0x7f857c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:43.972+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f858813f7f0 con 0x7f85700039b0 2024-04-25T10:03:44.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.145+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f858806d360 con 0x7f85700039b0 2024-04-25T10:03:44.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.146+0000 7f856ffff640 1 -- 172.21.15.148:0/1081712051 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+13485 (secure 0 0 0) 0x7f85780785c0 con 0x7f85700039b0 2024-04-25T10:03:44.151 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:44.151 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":137,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:42.832809+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6835/2921516856":"2024-04-26T10:03:39.792366+0000","172.21.15.148:6837/1740205817":"2024-04-26T10:03:28.777252+0000","172.21.15.148:6834/2921516856":"2024-04-26T10:03:39.792366+0000","172.21.15.148:6836/1740205817":"2024-04-26T10:03:28.777252+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:44.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f857406b4d0 msgr2=0x7f857406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f857406b4d0 0x7f857406d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f85880b1f30 tx=0x7f857c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:44.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 msgr2=0x7f858813f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f858813f2b0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8578005040 tx=0x7f8578001120 comp rx=0 tx=0).stop 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 shutdown_connections 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f857406b4d0 0x7f857406d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85881531a0 0x7f8588153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f858814ebd0 0x7f858813bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 --2- 172.21.15.148:0/1081712051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85700039b0 0x7f858813f2b0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 >> 172.21.15.148:0/1081712051 conn(0x7f858806abe0 msgr2=0x7f85880ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 shutdown_connections 2024-04-25T10:03:44.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.148+0000 7f858db53640 1 -- 172.21.15.148:0/1081712051 wait complete. 2024-04-25T10:03:44.165 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:44.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:44.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.259+0000 7fe4f6c9e640 1 Processor -- start 2024-04-25T10:03:44.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.259+0000 7fe4f6c9e640 1 -- start start 2024-04-25T10:03:44.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f6c9e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4f015b7d0 0x7fe4f00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f6c9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4f006c9e0 0x7fe4f00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f6c9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f006cf50 con 0x7fe4f00b5620 2024-04-25T10:03:44.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f6c9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00ce530 con 0x7fe4f006c9e0 2024-04-25T10:03:44.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f6c9e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00ce810 con 0x7fe4f015b7d0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4f006c9e0 0x7fe4f00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe4f006c9e0 0x7fe4f00b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46216/0 (socket says 172.21.15.148:46216) 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 -- 172.21.15.148:0/1301862333 learned_addr learned my addr 172.21.15.148:0/1301862333 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f4a3a640 1 --2- 172.21.15.148:0/1301862333 >> v2:172.21.15.188:3300/0 conn(0x7fe4f015b7d0 0x7fe4f00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 -- 172.21.15.148:0/1301862333 >> v2:172.21.15.188:3300/0 conn(0x7fe4f015b7d0 msgr2=0x7fe4f00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 --2- 172.21.15.148:0/1301862333 >> v2:172.21.15.188:3300/0 conn(0x7fe4f015b7d0 0x7fe4f00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 -- 172.21.15.148:0/1301862333 >> v1:172.21.15.188:6789/0 conn(0x7fe4f00b5620 legacy=0x7fe4f006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.260+0000 7fe4f523b640 1 -- 172.21.15.148:0/1301862333 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00ceaf0 con 0x7fe4f006c9e0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.261+0000 7fe4f523b640 1 --2- 172.21.15.148:0/1301862333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f006c9e0 0x7fe4f00b5ab0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fe4f008c0d0 tx=0x7fe4e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d262a1aa49406314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.261+0000 7fe4ef7fe640 1 -- 172.21.15.148:0/1301862333 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8036070 con 0x7fe4f006c9e0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.261+0000 7fe4ef7fe640 1 -- 172.21.15.148:0/1301862333 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e8031600 con 0x7fe4f006c9e0 2024-04-25T10:03:44.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.261+0000 7fe4ef7fe640 1 -- 172.21.15.148:0/1301862333 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80319d0 con 0x7fe4f006c9e0 2024-04-25T10:03:44.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/1301862333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f006c9e0 msgr2=0x7fe4f00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/1301862333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f006c9e0 0x7fe4f00b5ab0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fe4f008c0d0 tx=0x7fe4e802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/1301862333 shutdown_connections 2024-04-25T10:03:44.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/1301862333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f006c9e0 0x7fe4f00b5ab0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/1301862333 >> v2:172.21.15.188:3300/0 conn(0x7fe4f015b7d0 0x7fe4f00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/1301862333 >> 172.21.15.148:0/1301862333 conn(0x7fe4f00d0800 msgr2=0x7fe4f00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/1301862333 shutdown_connections 2024-04-25T10:03:44.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.262+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/1301862333 wait complete. 2024-04-25T10:03:44.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.263+0000 7fe4f6c9e640 1 Processor -- start 2024-04-25T10:03:44.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.263+0000 7fe4f6c9e640 1 -- start start 2024-04-25T10:03:44.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f6c9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f006c9e0 0x7fe4f00c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f6c9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 0x7fe4f00b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f6c9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f015b7d0 0x7fe4f00b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f6c9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00cf6d0 con 0x7fe4f00b5620 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f6c9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f00cf550 con 0x7fe4f015b7d0 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f6c9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f015bbb0 con 0x7fe4f006c9e0 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 0x7fe4f00b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 0x7fe4f00b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46230/0 (socket says 172.21.15.148:46230) 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 -- 172.21.15.148:0/2199000536 learned_addr learned my addr 172.21.15.148:0/2199000536 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f4a3a640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f006c9e0 0x7fe4f00c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f523b640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f015b7d0 0x7fe4f00b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 -- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f006c9e0 msgr2=0x7fe4f00c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f006c9e0 0x7fe4f00c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 -- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f015b7d0 msgr2=0x7fe4f00b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f015b7d0 0x7fe4f00b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4effff640 1 -- 172.21.15.148:0/2199000536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f00b76e0 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f4a3a640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f006c9e0 0x7fe4f00c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.264+0000 7fe4f523b640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f015b7d0 0x7fe4f00b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.265+0000 7fe4effff640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 0x7fe4f00b8780 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400c430 tx=0x7fe4e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.265+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4011790 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.265+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4017070 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.265+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f00b79c0 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.265+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4011bb0 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.265+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f00b7f80 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.266+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0073a00 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.266+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4e401c460 con 0x7fe4f00b5620 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.266+0000 7fe4ed7fa640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.267+0000 7fe4f4a3a640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.267+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe4e4089f40 con 0x7fe4f00b5620 2024-04-25T10:03:44.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.267+0000 7fe4f4a3a640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006fd0 tx=0x7fe4e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.271+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4e4058f70 con 0x7fe4f00b5620 2024-04-25T10:03:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.441+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4f015b0f0 con 0x7fe4f00b5620 2024-04-25T10:03:44.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.442+0000 7fe4ed7fa640 1 -- 172.21.15.148:0/2199000536 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4e4058870 con 0x7fe4f00b5620 2024-04-25T10:03:44.447 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.443+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4dc06b4d0 msgr2=0x7fe4dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.443+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006fd0 tx=0x7fe4e0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.443+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 msgr2=0x7fe4f00b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.443+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 0x7fe4f00b8780 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400c430 tx=0x7fe4e400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 shutdown_connections 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe4dc06b4d0 0x7fe4dc06d990 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4f015b7d0 0x7fe4f00b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4f00b5620 0x7fe4f00b8780 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 --2- 172.21.15.148:0/2199000536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4f006c9e0 0x7fe4f00c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 >> 172.21.15.148:0/2199000536 conn(0x7fe4f00d0800 msgr2=0x7fe4f015e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 shutdown_connections 2024-04-25T10:03:44.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.444+0000 7fe4f6c9e640 1 -- 172.21.15.148:0/2199000536 wait complete. 2024-04-25T10:03:44.460 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:03:44.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.552+0000 7f96d301f640 1 Processor -- start 2024-04-25T10:03:44.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.552+0000 7f96d301f640 1 -- start start 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96d301f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f96cc0cc8e0 0x7f96cc1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96d301f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96cc0cce50 con 0x7f96cc15dc90 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96d301f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96cc161af0 con 0x7f96cc0d1810 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96d301f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc161dd0 con 0x7f96cc0cc8e0 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96d0dbb640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f96cc15dc90 0x7f96cc15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35954/0 (socket says 172.21.15.148:35954) 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96d0dbb640 1 -- 172.21.15.148:0/4005828535 learned_addr learned my addr 172.21.15.148:0/4005828535 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:44.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96cbfff640 1 --2- 172.21.15.148:0/4005828535 >> v2:172.21.15.148:3300/0 conn(0x7f96cc0cc8e0 0x7f96cc1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96cb7fe640 1 -- 172.21.15.148:0/4005828535 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171577410 0 0) 0x7f96cc0cce50 con 0x7f96cc15dc90 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96cb7fe640 1 -- 172.21.15.148:0/4005828535 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b4003520 con 0x7f96cc15dc90 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96cbfff640 1 -- 172.21.15.148:0/4005828535 >> v1:172.21.15.188:6789/0 conn(0x7f96cc0d1810 legacy=0x7f96cc0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96cbfff640 1 -- 172.21.15.148:0/4005828535 >> v1:172.21.15.148:6789/0 conn(0x7f96cc15dc90 legacy=0x7f96cc15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.553+0000 7f96cbfff640 1 -- 172.21.15.148:0/4005828535 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc1620b0 con 0x7f96cc0cc8e0 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.554+0000 7f96cbfff640 1 --2- 172.21.15.148:0/4005828535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0cc8e0 0x7f96cc1625d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f96c00097a0 tx=0x7f96c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737d99d5a80876ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.554+0000 7f96cb7fe640 1 -- 172.21.15.148:0/4005828535 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0037070 con 0x7f96cc0cc8e0 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.554+0000 7f96cb7fe640 1 -- 172.21.15.148:0/4005828535 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0032440 con 0x7f96cc0cc8e0 2024-04-25T10:03:44.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.554+0000 7f96cb7fe640 1 -- 172.21.15.148:0/4005828535 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0032810 con 0x7f96cc0cc8e0 2024-04-25T10:03:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 -- 172.21.15.148:0/4005828535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0cc8e0 msgr2=0x7f96cc1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 --2- 172.21.15.148:0/4005828535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0cc8e0 0x7f96cc1625d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f96c00097a0 tx=0x7f96c002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 -- 172.21.15.148:0/4005828535 shutdown_connections 2024-04-25T10:03:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 --2- 172.21.15.148:0/4005828535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0cc8e0 0x7f96cc1625d0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 -- 172.21.15.148:0/4005828535 >> 172.21.15.148:0/4005828535 conn(0x7f96cc06f590 msgr2=0x7f96cc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 -- 172.21.15.148:0/4005828535 shutdown_connections 2024-04-25T10:03:44.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.555+0000 7f96d301f640 1 -- 172.21.15.148:0/4005828535 wait complete. 2024-04-25T10:03:44.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.556+0000 7f96d301f640 1 Processor -- start 2024-04-25T10:03:44.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.556+0000 7f96d301f640 1 -- start start 2024-04-25T10:03:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d301f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96cc0cc8e0 0x7f96cc0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d301f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 0x7f96cc155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d301f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96cc15dc90 0x7f96cc0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d301f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc15e1f0 con 0x7f96cc0d1810 2024-04-25T10:03:44.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d301f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc15e070 con 0x7f96cc15dc90 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d301f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc0d1c80 con 0x7f96cc0cc8e0 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 0x7f96cc155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 0x7f96cc155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46248/0 (socket says 172.21.15.148:46248) 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 -- 172.21.15.148:0/3821380733 learned_addr learned my addr 172.21.15.148:0/3821380733 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d15bc640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96cc15dc90 0x7f96cc0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d0dbb640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96cc0cc8e0 0x7f96cc0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 -- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96cc0cc8e0 msgr2=0x7f96cc0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96cc0cc8e0 0x7f96cc0cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 -- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96cc15dc90 msgr2=0x7f96cc0c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96cc15dc90 0x7f96cc0c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 -- 172.21.15.148:0/3821380733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc0c8950 con 0x7f96cc0d1810 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96d15bc640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96cc15dc90 0x7f96cc0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.557+0000 7f96cbfff640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 0x7f96cc155dd0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f96c0035040 tx=0x7f96c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.558+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0037040 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.558+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0030500 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.558+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0030910 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.558+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc157400 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.558+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc1579c0 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.558+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc073990 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.559+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96c0044020 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.559+0000 7f96c97fa640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96b806b4d0 0x7f96b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.560+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f96c00aa3e0 con 0x7f96cc0d1810 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.560+0000 7f96d0dbb640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96b806b4d0 0x7f96b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.560+0000 7f96d0dbb640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96b806b4d0 0x7f96b806d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0011e0 tx=0x7f96bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.564+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96c0079380 con 0x7f96cc0d1810 2024-04-25T10:03:44.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.798+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f96cc0cf6b0 con 0x7f96cc0d1810 2024-04-25T10:03:44.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.850+0000 7f96c97fa640 1 -- 172.21.15.148:0/3821380733 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v459) v1 ==== 162+0+0 (secure 0 0 0) 0x7f96c0078c80 con 0x7f96cc0d1810 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96b806b4d0 msgr2=0x7f96b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96b806b4d0 0x7f96b806d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0011e0 tx=0x7f96bc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 msgr2=0x7f96cc155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 0x7f96cc155dd0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f96c0035040 tx=0x7f96c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 shutdown_connections 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96b806b4d0 0x7f96b806d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96cc15dc90 0x7f96cc0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96cc0d1810 0x7f96cc155dd0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 --2- 172.21.15.148:0/3821380733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96cc0cc8e0 0x7f96cc0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 >> 172.21.15.148:0/3821380733 conn(0x7f96cc06f590 msgr2=0x7f96cc0b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 shutdown_connections 2024-04-25T10:03:44.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.853+0000 7f96d301f640 1 -- 172.21.15.148:0/3821380733 wait complete. 2024-04-25T10:03:44.869 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:03:44.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.965+0000 7fc958153640 1 Processor -- start 2024-04-25T10:03:44.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.965+0000 7fc958153640 1 -- start start 2024-04-25T10:03:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc958153640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc950161240 0x7fc9500b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc958153640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500cf1d0 con 0x7fc950161890 2024-04-25T10:03:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc958153640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc950161620 con 0x7fc9500b54a0 2024-04-25T10:03:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc958153640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500b7120 con 0x7fc950161240 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc9556ee640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc950161890 0x7fc9500ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35956/0 (socket says 172.21.15.148:35956) 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc9556ee640 1 -- 172.21.15.148:0/1847980658 learned_addr learned my addr 172.21.15.148:0/1847980658 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc955eef640 1 --2- 172.21.15.148:0/1847980658 >> v2:172.21.15.188:3300/0 conn(0x7fc950161240 0x7fc9500b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155836461 0 0) 0x7fc9500cf1d0 con 0x7fc950161890 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc93c003520 con 0x7fc950161890 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 113295343 0 0) 0x7fc93c003520 con 0x7fc950161890 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 >> v2:172.21.15.188:3300/0 conn(0x7fc950161240 msgr2=0x7fc9500b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 --2- 172.21.15.148:0/1847980658 >> v2:172.21.15.188:3300/0 conn(0x7fc950161240 0x7fc9500b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 >> v1:172.21.15.188:6789/0 conn(0x7fc9500b54a0 legacy=0x7fc9500d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950161e00 con 0x7fc950161890 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc940001670 con 0x7fc950161890 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 >> v1:172.21.15.148:6789/0 conn(0x7fc950161890 legacy=0x7fc9500ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc93c0039b0 0x7fc93c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004800 0x7fc93c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c0051c0 0x7fc93c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00c030 con 0x7fc93c0039b0 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00c1e0 con 0x7fc93c004800 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00c3e0 con 0x7fc93c0051c0 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.966+0000 7fc955eef640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc93c0039b0 0x7fc93c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc9556ee640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004800 0x7fc93c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc9566f0640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c0051c0 0x7fc93c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc955eef640 1 -- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c0051c0 msgr2=0x7fc93c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc955eef640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc93c0051c0 0x7fc93c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc955eef640 1 -- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004800 msgr2=0x7fc93c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc955eef640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004800 0x7fc93c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc9566f0640 1 -- 172.21.15.148:0/1847980658 reap_dead start 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc955eef640 1 -- 172.21.15.148:0/1847980658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c003520 con 0x7fc93c0039b0 2024-04-25T10:03:44.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc9556ee640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c004800 0x7fc93c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc955eef640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc93c0039b0 0x7fc93c003dd0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009ac0 tx=0x7fc93c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f8c36907e8f587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c036070 con 0x7fc93c0039b0 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c031400 con 0x7fc93c0039b0 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.967+0000 7fc954eed640 1 -- 172.21.15.148:0/1847980658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c009690 con 0x7fc93c0039b0 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 -- 172.21.15.148:0/1847980658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc93c0039b0 msgr2=0x7fc93c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc93c0039b0 0x7fc93c003dd0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009ac0 tx=0x7fc93c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 -- 172.21.15.148:0/1847980658 shutdown_connections 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 --2- 172.21.15.148:0/1847980658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc93c0039b0 0x7fc93c003dd0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 -- 172.21.15.148:0/1847980658 >> 172.21.15.148:0/1847980658 conn(0x7fc9500d04a0 msgr2=0x7fc9500d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 -- 172.21.15.148:0/1847980658 shutdown_connections 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.968+0000 7fc958153640 1 -- 172.21.15.148:0/1847980658 wait complete. 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.969+0000 7fc958153640 1 Processor -- start 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.969+0000 7fc958153640 1 -- start start 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc958153640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c0039b0 0x7fc9500c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc958153640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc95015b230 0x7fc95015e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc958153640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 0x7fc95015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc958153640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501626a0 con 0x7fc95015e750 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc958153640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501623c0 con 0x7fc93c0039b0 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc958153640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500d15f0 con 0x7fc95015b230 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 0x7fc95015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 0x7fc95015fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46266/0 (socket says 172.21.15.148:46266) 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 -- 172.21.15.148:0/2119714257 learned_addr learned my addr 172.21.15.148:0/2119714257 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc955eef640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c0039b0 0x7fc9500c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9556ee640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc95015b230 0x7fc95015e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 -- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc95015b230 msgr2=0x7fc95015e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc95015b230 0x7fc95015e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 -- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c0039b0 msgr2=0x7fc9500c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c0039b0 0x7fc9500c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.970+0000 7fc9566f0640 1 -- 172.21.15.148:0/2119714257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500c8360 con 0x7fc95015e750 2024-04-25T10:03:44.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.971+0000 7fc9566f0640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 0x7fc95015fee0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc948007fd0 tx=0x7fc94800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.971+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948003970 con 0x7fc95015e750 2024-04-25T10:03:44.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.971+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948018070 con 0x7fc95015e750 2024-04-25T10:03:44.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.971+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948003d70 con 0x7fc95015e750 2024-04-25T10:03:44.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.971+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9500c8640 con 0x7fc95015e750 2024-04-25T10:03:44.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.971+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9500c8c00 con 0x7fc95015e750 2024-04-25T10:03:44.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.972+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950073a00 con 0x7fc95015e750 2024-04-25T10:03:44.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.972+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9480050d0 con 0x7fc95015e750 2024-04-25T10:03:44.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.972+0000 7fc9467fc640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.973+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc948089ae0 con 0x7fc95015e750 2024-04-25T10:03:44.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.973+0000 7fc955eef640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.973+0000 7fc955eef640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc93406b4d0 0x7fc93406d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc94c033000 tx=0x7fc94c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:44.977+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9480584c0 con 0x7fc95015e750 2024-04-25T10:03:45.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.149+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc950068690 con 0x7fc95015e750 2024-04-25T10:03:45.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.155+0000 7fc9467fc640 1 -- 172.21.15.148:0/2119714257 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc94805c560 con 0x7fc95015e750 2024-04-25T10:03:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc93406b4d0 msgr2=0x7fc93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc93406b4d0 0x7fc93406d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc94c033000 tx=0x7fc94c008030 comp rx=0 tx=0).stop 2024-04-25T10:03:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 msgr2=0x7fc95015fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 0x7fc95015fee0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc948007fd0 tx=0x7fc94800ec10 comp rx=0 tx=0).stop 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 shutdown_connections 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc93406b4d0 0x7fc93406d990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95015e750 0x7fc95015fee0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc95015b230 0x7fc95015e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 --2- 172.21.15.148:0/2119714257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc93c0039b0 0x7fc9500c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 >> 172.21.15.148:0/2119714257 conn(0x7fc9500d04a0 msgr2=0x7fc95015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 shutdown_connections 2024-04-25T10:03:45.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.157+0000 7fc958153640 1 -- 172.21.15.148:0/2119714257 wait complete. 2024-04-25T10:03:45.173 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:45.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:45.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:45.251 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:45.251 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:03:45.251 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:45.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:45.267 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:03:45.267 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:45.307 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:45.307 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:03:45.308 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:45.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:03:45.323 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:03:45.323 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:45.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:03:45.364 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:03:45.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:45.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.457+0000 7fdcae56a640 1 Processor -- start 2024-04-25T10:03:45.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.457+0000 7fdcae56a640 1 -- start start 2024-04-25T10:03:45.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcae56a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca815bd30 0x7fdca80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcae56a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdca80cfe40 0x7fdca80d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcae56a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdca80d1930 0x7fdca80d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcae56a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca80d0350 con 0x7fdca80d1930 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcae56a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca80d0630 con 0x7fdca80cfe40 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcae56a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80d0910 con 0x7fdca815bd30 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdca80d1930 0x7fdca80d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdca80d1930 0x7fdca80d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46270/0 (socket says 172.21.15.148:46270) 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 -- 172.21.15.148:0/360761430 learned_addr learned my addr 172.21.15.148:0/360761430 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdca7fff640 1 --2- 172.21.15.148:0/360761430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca815bd30 0x7fdca80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdca77fe640 1 --2- 172.21.15.148:0/360761430 >> v2:172.21.15.188:3300/0 conn(0x7fdca80cfe40 0x7fdca80d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 -- 172.21.15.148:0/360761430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca815bd30 msgr2=0x7fdca80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 --2- 172.21.15.148:0/360761430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca815bd30 0x7fdca80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 -- 172.21.15.148:0/360761430 >> v2:172.21.15.188:3300/0 conn(0x7fdca80cfe40 msgr2=0x7fdca80d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 --2- 172.21.15.148:0/360761430 >> v2:172.21.15.188:3300/0 conn(0x7fdca80cfe40 0x7fdca80d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.458+0000 7fdcacb07640 1 -- 172.21.15.148:0/360761430 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca80d0bf0 con 0x7fdca80d1930 2024-04-25T10:03:45.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.459+0000 7fdca77fe640 1 --2- 172.21.15.148:0/360761430 >> v2:172.21.15.188:3300/0 conn(0x7fdca80cfe40 0x7fdca80d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:45.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.459+0000 7fdcacb07640 1 --2- 172.21.15.148:0/360761430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca80d1d40 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fdca80cf390 tx=0x7fdca002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d485cf15e7ccec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.459+0000 7fdca6ffd640 1 -- 172.21.15.148:0/360761430 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0036070 con 0x7fdca80d1930 2024-04-25T10:03:45.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.459+0000 7fdca6ffd640 1 -- 172.21.15.148:0/360761430 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0031600 con 0x7fdca80d1930 2024-04-25T10:03:45.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.459+0000 7fdca6ffd640 1 -- 172.21.15.148:0/360761430 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00319d0 con 0x7fdca80d1930 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 -- 172.21.15.148:0/360761430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 msgr2=0x7fdca80d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 --2- 172.21.15.148:0/360761430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca80d1d40 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fdca80cf390 tx=0x7fdca002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 -- 172.21.15.148:0/360761430 shutdown_connections 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 --2- 172.21.15.148:0/360761430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca80d1d40 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 --2- 172.21.15.148:0/360761430 >> v2:172.21.15.188:3300/0 conn(0x7fdca80cfe40 0x7fdca80d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 --2- 172.21.15.148:0/360761430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca815bd30 0x7fdca80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 -- 172.21.15.148:0/360761430 >> 172.21.15.148:0/360761430 conn(0x7fdca806f590 msgr2=0x7fdca806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 -- 172.21.15.148:0/360761430 shutdown_connections 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.460+0000 7fdcae56a640 1 -- 172.21.15.148:0/360761430 wait complete. 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.461+0000 7fdcae56a640 1 Processor -- start 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.461+0000 7fdcae56a640 1 -- start start 2024-04-25T10:03:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcae56a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca80cfe40 0x7fdca81444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcae56a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca81547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcae56a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca815bd30 0x7fdca80b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcae56a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b73b0 con 0x7fdca80d1930 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcae56a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca815c110 con 0x7fdca815bd30 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcae56a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80b7530 con 0x7fdca80cfe40 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca81547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca81547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46280/0 (socket says 172.21.15.148:46280) 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 -- 172.21.15.148:0/2612652443 learned_addr learned my addr 172.21.15.148:0/2612652443 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca7fff640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca80cfe40 0x7fdca81444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdcacb07640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca815bd30 0x7fdca80b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 -- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca80cfe40 msgr2=0x7fdca81444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca80cfe40 0x7fdca81444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 -- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca815bd30 msgr2=0x7fdca80b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca815bd30 0x7fdca80b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.462+0000 7fdca77fe640 1 -- 172.21.15.148:0/2612652443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca81449f0 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.463+0000 7fdca77fe640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca81547a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d7d0 tx=0x7fdc9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.463+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c004030 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.463+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c0041d0 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.463+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca8153700 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.463+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0096a0 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.463+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca8153fa0 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.464+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8073990 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.464+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc9c009840 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.465+0000 7fdca4ff9640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9406b4d0 0x7fdc9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.465+0000 7fdca7fff640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9406b4d0 0x7fdc9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.465+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdc9c08a690 con 0x7fdca80d1930 2024-04-25T10:03:45.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.465+0000 7fdca7fff640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9406b4d0 0x7fdc9406d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006fd0 tx=0x7fdc98008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.469+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdca81449f0 con 0x7fdca80d1930 2024-04-25T10:03:45.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.686+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdca8154ce0 con 0x7fdca80d1930 2024-04-25T10:03:45.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.687+0000 7fdca4ff9640 1 -- 172.21.15.148:0/2612652443 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2911 (secure 0 0 0) 0x7fdc9c059070 con 0x7fdca80d1930 2024-04-25T10:03:45.692 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:45.692 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7708,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/273480267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":273480267},{"type":"v1","addr":"172.21.15.188:6834","nonce":273480267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":7711,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1434339580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1434339580},{"type":"v1","addr":"172.21.15.188:6835","nonce":1434339580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70},{"gid":9780,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/409675945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":409675945},{"type":"v1","addr":"172.21.15.148:6835","nonce":409675945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[]} 2024-04-25T10:03:45.693 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 80 2024-04-25T10:03:45.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9406b4d0 msgr2=0x7fdc9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9406b4d0 0x7fdc9406d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdc98006fd0 tx=0x7fdc98008040 comp rx=0 tx=0).stop 2024-04-25T10:03:45.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 msgr2=0x7fdca81547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca81547a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00d7d0 tx=0x7fdc9c00dc90 comp rx=0 tx=0).stop 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 shutdown_connections 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc9406b4d0 0x7fdc9406d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdca815bd30 0x7fdca80b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdca80d1930 0x7fdca81547a0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 --2- 172.21.15.148:0/2612652443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdca80cfe40 0x7fdca81444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 >> 172.21.15.148:0/2612652443 conn(0x7fdca806f590 msgr2=0x7fdca80b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 shutdown_connections 2024-04-25T10:03:45.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.689+0000 7fdcae56a640 1 -- 172.21.15.148:0/2612652443 wait complete. 2024-04-25T10:03:45.705 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:03:45.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:03:45.706 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:03:45.706 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:03:45.706 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:45.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:45.707 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:03:45.707 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:03:45.707 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:45.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:45.708 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:03:45.708 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:03:45.708 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:45.708 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:45.715 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:45.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:03:45.716 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:03:45.716 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:45.719 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:45.719 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:03:45.720 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:45.720 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:03:45.758 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:03:45.758 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:45.765 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:03:45.766 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:45.766 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:03:45.766 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:45.767 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:03:45.788 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:03:45.831 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:03:45.841 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:03:45.842 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:03:45.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.872+0000 7f3c81edd640 1 Processor -- start 2024-04-25T10:03:45.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.872+0000 7f3c81edd640 1 -- start start 2024-04-25T10:03:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c81edd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c81edd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c7c161710 0x7f3c7c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7affd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7b7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3c7c15dc20 0x7f3c7c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:49784/0 (socket says 172.21.15.148:49784) 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7b7fe640 1 -- 172.21.15.148:0/3031199462 learned_addr learned my addr 172.21.15.148:0/3031199462 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c15e000 con 0x7f3c7c15dc20 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7bfff640 1 --2- 172.21.15.148:0/3031199462 >> v2:172.21.15.188:3300/0 conn(0x7f3c7c161710 0x7f3c7c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d1de0 con 0x7f3c7c161710 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c161c60 con 0x7f3c7c0ced80 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7a7fc640 1 -- 172.21.15.148:0/3031199462 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745483898 0 0) 0x7f3c7c15e000 con 0x7f3c7c15dc20 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7a7fc640 1 -- 172.21.15.148:0/3031199462 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c64003520 con 0x7f3c7c15dc20 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7bfff640 1 -- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 msgr2=0x7f3c7c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7bfff640 1 --2- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7bfff640 1 -- 172.21.15.148:0/3031199462 >> v1:172.21.15.188:6789/0 conn(0x7f3c7c15dc20 legacy=0x7f3c7c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7bfff640 1 -- 172.21.15.148:0/3031199462 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c161f40 con 0x7f3c7c161710 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.873+0000 7f3c7affd640 1 --2- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.874+0000 7f3c7bfff640 1 --2- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c161710 0x7f3c7c0d2820 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009d90 tx=0x7f3c7402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bbc103300a88ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.874+0000 7f3c7a7fc640 1 -- 172.21.15.148:0/3031199462 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74007740 con 0x7f3c7c161710 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.874+0000 7f3c7a7fc640 1 -- 172.21.15.148:0/3031199462 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c740078e0 con 0x7f3c7c161710 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.874+0000 7f3c7a7fc640 1 -- 172.21.15.148:0/3031199462 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74007cb0 con 0x7f3c7c161710 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c161710 msgr2=0x7f3c7c0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 --2- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c161710 0x7f3c7c0d2820 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009d90 tx=0x7f3c7402a660 comp rx=0 tx=0).stop 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 shutdown_connections 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 --2- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c161710 0x7f3c7c0d2820 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 --2- 172.21.15.148:0/3031199462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 >> 172.21.15.148:0/3031199462 conn(0x7f3c7c0d0420 msgr2=0x7f3c7c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 shutdown_connections 2024-04-25T10:03:45.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.875+0000 7f3c81edd640 1 -- 172.21.15.148:0/3031199462 wait complete. 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.877+0000 7f3c81edd640 1 Processor -- start 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.877+0000 7f3c81edd640 1 -- start start 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c81edd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c81edd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c15dc20 0x7f3c7c1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c81edd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c7c161710 0x7f3c7c1f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c81edd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0b5e50 con 0x7f3c7c161710 2024-04-25T10:03:45.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c81edd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c162500 con 0x7f3c7c15dc20 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c81edd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0b5fd0 con 0x7f3c7c0ced80 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c7c161710 0x7f3c7c1f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c7c161710 0x7f3c7c1f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46310/0 (socket says 172.21.15.148:46310) 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7bfff640 1 -- 172.21.15.148:0/1263341967 learned_addr learned my addr 172.21.15.148:0/1263341967 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7affd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c15dc20 0x7f3c7c1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7affd640 1 -- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 msgr2=0x7f3c7c0aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7affd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0aa820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7affd640 1 -- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c7c161710 msgr2=0x7f3c7c1f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7affd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c7c161710 0x7f3c7c1f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.878+0000 7f3c7affd640 1 -- 172.21.15.148:0/1263341967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0cd390 con 0x7f3c7c15dc20 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.879+0000 7f3c7affd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c15dc20 0x7f3c7c1f7010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000e870 tx=0x7f3c7000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.879+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70017070 con 0x7f3c7c15dc20 2024-04-25T10:03:45.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.879+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7000c940 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.879+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c15e6f0 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.879+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7000cd10 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.879+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c15ecb0 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.880+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c073a00 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.880+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c7001a070 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.881+0000 7f3c63fff640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c6806b500 0x7f3c6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.881+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3c7008a140 con 0x7f3c7c15dc20 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.881+0000 7f3c7b7fe640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c6806b500 0x7f3c6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.881+0000 7f3c7b7fe640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c6806b500 0x7f3c6806d9c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0011e0 tx=0x7f3c6c007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:45.885+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c70058b20 con 0x7f3c7c15dc20 2024-04-25T10:03:46.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.067+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3c7c0d16b0 con 0x7f3c7c15dc20 2024-04-25T10:03:46.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.866+0000 7f3c63fff640 1 -- 172.21.15.148:0/1263341967 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v141) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3c7005cbc0 con 0x7f3c7c15dc20 2024-04-25T10:03:46.871 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:03:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c6806b500 msgr2=0x7f3c6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c6806b500 0x7f3c6806d9c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0011e0 tx=0x7f3c6c007d30 comp rx=0 tx=0).stop 2024-04-25T10:03:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c15dc20 msgr2=0x7f3c7c1f7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c15dc20 0x7f3c7c1f7010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000e870 tx=0x7f3c7000ed30 comp rx=0 tx=0).stop 2024-04-25T10:03:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 shutdown_connections 2024-04-25T10:03:46.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3c6806b500 0x7f3c6806d9c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c7c161710 0x7f3c7c1f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c7c15dc20 0x7f3c7c1f7010 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 --2- 172.21.15.148:0/1263341967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c7c0ced80 0x7f3c7c0aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 >> 172.21.15.148:0/1263341967 conn(0x7f3c7c0d0420 msgr2=0x7f3c7c0b89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 shutdown_connections 2024-04-25T10:03:46.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:46.869+0000 7f3c81edd640 1 -- 172.21.15.148:0/1263341967 wait complete. 2024-04-25T10:03:46.888 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:03:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.026+0000 7fd5fffe2640 1 Processor -- start 2024-04-25T10:03:47.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.026+0000 7fd5fffe2640 1 -- start start 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fffe2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 0x7fd5f80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fffe2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5f80ce590 0x7fd5f806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fffe2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f8071480 con 0x7fd5f80cfcf0 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fffe2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f8072090 con 0x7fd5f80ce590 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fffe2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80d0150 con 0x7fd5f8154ee0 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fdd7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 0x7fd5f80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fe57f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5f80ce590 0x7fd5f806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fdd7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 0x7fd5f80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:33092/0 (socket says 172.21.15.148:33092) 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fdd7e640 1 -- 172.21.15.148:0/4013827925 learned_addr learned my addr 172.21.15.148:0/4013827925 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.027+0000 7fd5fd57d640 1 --1- 172.21.15.148:0/4013827925 >> v1:172.21.15.188:6789/0 conn(0x7fd5f80cfcf0 0x7fd5f80cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:49786/0 (socket says 172.21.15.148:49786) 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.028+0000 7fd5fe57f640 1 -- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 msgr2=0x7fd5f80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.028+0000 7fd5fe57f640 1 --2- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 0x7fd5f80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.028+0000 7fd5fe57f640 1 -- 172.21.15.148:0/4013827925 >> v1:172.21.15.188:6789/0 conn(0x7fd5f80cfcf0 legacy=0x7fd5f80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.028+0000 7fd5fe57f640 1 -- 172.21.15.148:0/4013827925 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8071760 con 0x7fd5f80ce590 2024-04-25T10:03:47.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.028+0000 7fd5fdd7e640 1 --2- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 0x7fd5f80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:47.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.028+0000 7fd5fe57f640 1 --2- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f806d6f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009d90 tx=0x7fd5f002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd777241a40240bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fcd7c640 1 -- 172.21.15.148:0/4013827925 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0007740 con 0x7fd5f80ce590 2024-04-25T10:03:47.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fcd7c640 1 -- 172.21.15.148:0/4013827925 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f00078e0 con 0x7fd5f80ce590 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fcd7c640 1 -- 172.21.15.148:0/4013827925 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0007cb0 con 0x7fd5f80ce590 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 -- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 msgr2=0x7fd5f806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f806d6f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009d90 tx=0x7fd5f002a660 comp rx=0 tx=0).stop 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 -- 172.21.15.148:0/4013827925 shutdown_connections 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f806d6f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/4013827925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f8154ee0 0x7fd5f80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 -- 172.21.15.148:0/4013827925 >> 172.21.15.148:0/4013827925 conn(0x7fd5f806abe0 msgr2=0x7fd5f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 -- 172.21.15.148:0/4013827925 shutdown_connections 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.029+0000 7fd5fffe2640 1 -- 172.21.15.148:0/4013827925 wait complete. 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.030+0000 7fd5fffe2640 1 Processor -- start 2024-04-25T10:03:47.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 -- start start 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f8143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f80cfcf0 0x7fd5f8150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 0x7fd5f8150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8071d20 con 0x7fd5f8154ee0 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8071230 con 0x7fd5f80ce590 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fffe2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f81552c0 con 0x7fd5f80cfcf0 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fe57f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 0x7fd5f8150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fe57f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 0x7fd5f8150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46318/0 (socket says 172.21.15.148:46318) 2024-04-25T10:03:47.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.031+0000 7fd5fe57f640 1 -- 172.21.15.148:0/2562781473 learned_addr learned my addr 172.21.15.148:0/2562781473 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fdd7e640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f8143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fd57d640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f80cfcf0 0x7fd5f8150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fe57f640 1 -- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f80cfcf0 msgr2=0x7fd5f8150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fe57f640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f80cfcf0 0x7fd5f8150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fe57f640 1 -- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 msgr2=0x7fd5f8143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fe57f640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f8143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fe57f640 1 -- 172.21.15.148:0/2562781473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f814f560 con 0x7fd5f8154ee0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fdd7e640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f8143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fd57d640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f80cfcf0 0x7fd5f8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fe57f640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 0x7fd5f8150da0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009eb0 tx=0x7fd5f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0040070 con 0x7fd5f8154ee0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f0031c60 con 0x7fd5f8154ee0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0009450 con 0x7fd5f8154ee0 2024-04-25T10:03:47.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f814f840 con 0x7fd5f8154ee0 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.032+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f814fe00 con 0x7fd5f8154ee0 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.033+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f8073990 con 0x7fd5f8154ee0 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.033+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5f0039040 con 0x7fd5f8154ee0 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.034+0000 7fd5ee7fc640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5dc06b4d0 0x7fd5dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.034+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd5f00ab200 con 0x7fd5f8154ee0 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.034+0000 7fd5fdd7e640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5dc06b4d0 0x7fd5dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.034+0000 7fd5fdd7e640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5dc06b4d0 0x7fd5dc06d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009a20 tx=0x7fd5f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.038+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5f0039330 con 0x7fd5f8154ee0 2024-04-25T10:03:47.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.276+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd5f806d360 con 0x7fd5f8154ee0 2024-04-25T10:03:47.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.277+0000 7fd5ee7fc640 1 -- 172.21.15.148:0/2562781473 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd5f007a1c0 con 0x7fd5f8154ee0 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:03:47.282 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.283 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.285 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.286 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:03:47.287 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.288 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:47.289 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.290 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.291 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.293 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.294 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:47.295 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:03:47.295 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:03:47.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5dc06b4d0 msgr2=0x7fd5dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5dc06b4d0 0x7fd5dc06d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009a20 tx=0x7fd5f4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:47.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 msgr2=0x7fd5f8150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 0x7fd5f8150da0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009eb0 tx=0x7fd5f0004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:47.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 shutdown_connections 2024-04-25T10:03:47.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5dc06b4d0 0x7fd5dc06d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5f8154ee0 0x7fd5f8150da0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5f80cfcf0 0x7fd5f8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 --2- 172.21.15.148:0/2562781473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5f80ce590 0x7fd5f8143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.278+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 >> 172.21.15.148:0/2562781473 conn(0x7fd5f806abe0 msgr2=0x7fd5f815d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.279+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 shutdown_connections 2024-04-25T10:03:47.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.279+0000 7fd5fffe2640 1 -- 172.21.15.148:0/2562781473 wait complete. 2024-04-25T10:03:47.297 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:47.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.389+0000 7fef547f9640 1 Processor -- start 2024-04-25T10:03:47.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.389+0000 7fef547f9640 1 -- start start 2024-04-25T10:03:47.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef547f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef4c158220 0x7fef4c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef547f9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef4c0af720 0x7fef4c0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef547f9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c15f180 con 0x7fef4c1617b0 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef547f9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c0b3670 con 0x7fef4c158220 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef547f9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c070a10 con 0x7fef4c0af720 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef52595640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef4c158220 0x7fef4c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef52d96640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef4c1617b0 0x7fef4c161be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35970/0 (socket says 172.21.15.148:35970) 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef52595640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fef4c158220 0x7fef4c15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46322/0 (socket says 172.21.15.148:46322) 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.390+0000 7fef52d96640 1 -- 172.21.15.148:0/2535003751 learned_addr learned my addr 172.21.15.148:0/2535003751 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef51d94640 1 --2- 172.21.15.148:0/2535003751 >> v2:172.21.15.188:3300/0 conn(0x7fef4c0af720 0x7fef4c0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef52595640 1 -- 172.21.15.148:0/2535003751 >> v2:172.21.15.188:3300/0 conn(0x7fef4c0af720 msgr2=0x7fef4c0afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef52595640 1 --2- 172.21.15.148:0/2535003751 >> v2:172.21.15.188:3300/0 conn(0x7fef4c0af720 0x7fef4c0afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef52595640 1 -- 172.21.15.148:0/2535003751 >> v1:172.21.15.148:6789/0 conn(0x7fef4c1617b0 legacy=0x7fef4c161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef51593640 1 -- 172.21.15.148:0/2535003751 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277008603 0 0) 0x7fef44001490 con 0x7fef4c1617b0 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef52595640 1 -- 172.21.15.148:0/2535003751 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c06fe00 con 0x7fef4c158220 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef51d94640 1 --2- 172.21.15.148:0/2535003751 >> v2:172.21.15.188:3300/0 conn(0x7fef4c0af720 0x7fef4c0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef52595640 1 --2- 172.21.15.148:0/2535003751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c158220 0x7fef4c15ec40 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0cfc70 tx=0x7fef4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4db3b1fd6a3f27d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.391+0000 7fef51593640 1 -- 172.21.15.148:0/2535003751 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48037070 con 0x7fef4c158220 2024-04-25T10:03:47.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef51593640 1 -- 172.21.15.148:0/2535003751 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef48032440 con 0x7fef4c158220 2024-04-25T10:03:47.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef51593640 1 -- 172.21.15.148:0/2535003751 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48032810 con 0x7fef4c158220 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 -- 172.21.15.148:0/2535003751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c158220 msgr2=0x7fef4c15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 --2- 172.21.15.148:0/2535003751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c158220 0x7fef4c15ec40 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0cfc70 tx=0x7fef4802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 -- 172.21.15.148:0/2535003751 shutdown_connections 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 --2- 172.21.15.148:0/2535003751 >> v2:172.21.15.188:3300/0 conn(0x7fef4c0af720 0x7fef4c0afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 --2- 172.21.15.148:0/2535003751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c158220 0x7fef4c15ec40 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 -- 172.21.15.148:0/2535003751 >> 172.21.15.148:0/2535003751 conn(0x7fef4c0b2370 msgr2=0x7fef4c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 -- 172.21.15.148:0/2535003751 shutdown_connections 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.392+0000 7fef547f9640 1 -- 172.21.15.148:0/2535003751 wait complete. 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.393+0000 7fef547f9640 1 Processor -- start 2024-04-25T10:03:47.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.393+0000 7fef547f9640 1 -- start start 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef547f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef4c0af720 0x7fef4c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef547f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef4c158220 0x7fef4c1f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef547f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 0x7fef4c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef547f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0706a0 con 0x7fef4c1617b0 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef547f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0703c0 con 0x7fef4c158220 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef547f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c158600 con 0x7fef4c0af720 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 0x7fef4c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 0x7fef4c0d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46334/0 (socket says 172.21.15.148:46334) 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 -- 172.21.15.148:0/2402363243 learned_addr learned my addr 172.21.15.148:0/2402363243 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52595640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef4c0af720 0x7fef4c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef51d94640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef4c158220 0x7fef4c1f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 -- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef4c0af720 msgr2=0x7fef4c153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef4c0af720 0x7fef4c153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 -- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef4c158220 msgr2=0x7fef4c1f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef4c158220 0x7fef4c1f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.394+0000 7fef52d96640 1 -- 172.21.15.148:0/2402363243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c159af0 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.395+0000 7fef52d96640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 0x7fef4c0d02f0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fef4400c800 tx=0x7fef4400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.395+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44018070 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.395+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4400a640 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.395+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c153df0 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.395+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4400aa70 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.395+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c1543b0 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.396+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c073990 con 0x7fef4c1617b0 2024-04-25T10:03:47.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.396+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef4401b070 con 0x7fef4c1617b0 2024-04-25T10:03:47.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.397+0000 7fef42ffd640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef3006b4d0 0x7fef3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.397+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fef44089670 con 0x7fef4c1617b0 2024-04-25T10:03:47.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.397+0000 7fef52595640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef3006b4d0 0x7fef3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.397+0000 7fef52595640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef3006b4d0 0x7fef3006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fef48034000 tx=0x7fef4802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.401+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef4c159af0 con 0x7fef4c1617b0 2024-04-25T10:03:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.573+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef4c162200 con 0x7fef4c1617b0 2024-04-25T10:03:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.574+0000 7fef42ffd640 1 -- 172.21.15.148:0/2402363243 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+13222 (secure 0 0 0) 0x7fef4c15f180 con 0x7fef4c1617b0 2024-04-25T10:03:47.579 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:47.579 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":141,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:46.861876+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef3006b4d0 msgr2=0x7fef3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef3006b4d0 0x7fef3006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fef48034000 tx=0x7fef4802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 msgr2=0x7fef4c0d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 0x7fef4c0d02f0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fef4400c800 tx=0x7fef4400ccc0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 shutdown_connections 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef3006b4d0 0x7fef3006d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef4c1617b0 0x7fef4c0d02f0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef4c158220 0x7fef4c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 --2- 172.21.15.148:0/2402363243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef4c0af720 0x7fef4c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 >> 172.21.15.148:0/2402363243 conn(0x7fef4c0b2370 msgr2=0x7fef4c1574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 shutdown_connections 2024-04-25T10:03:47.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.576+0000 7fef547f9640 1 -- 172.21.15.148:0/2402363243 wait complete. 2024-04-25T10:03:47.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:47.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:47.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.686+0000 7f360250a640 1 Processor -- start 2024-04-25T10:03:47.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.686+0000 7f360250a640 1 -- start start 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f360250a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35fc162140 0x7f35fc0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f360250a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35fc0cd540 0x7f35fc0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f360250a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc15e3a0 con 0x7f35fc0d0a00 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f360250a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc1625c0 con 0x7f35fc0cd540 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f360250a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0cd9d0 con 0x7f35fc162140 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35fc0cd540 0x7f35fc0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f35fc0cd540 0x7f35fc0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46348/0 (socket says 172.21.15.148:46348) 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 -- 172.21.15.148:0/1448797252 learned_addr learned my addr 172.21.15.148:0/1448797252 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f35fb7fe640 1 --2- 172.21.15.148:0/1448797252 >> v2:172.21.15.188:3300/0 conn(0x7f35fc162140 0x7f35fc0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 -- 172.21.15.148:0/1448797252 >> v2:172.21.15.188:3300/0 conn(0x7f35fc162140 msgr2=0x7f35fc0cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 --2- 172.21.15.148:0/1448797252 >> v2:172.21.15.188:3300/0 conn(0x7f35fc162140 0x7f35fc0cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 -- 172.21.15.148:0/1448797252 >> v1:172.21.15.188:6789/0 conn(0x7f35fc0d0a00 legacy=0x7f35fc0d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.687+0000 7f3600aa7640 1 -- 172.21.15.148:0/1448797252 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0cdc70 con 0x7f35fc0cd540 2024-04-25T10:03:47.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.688+0000 7f3600aa7640 1 --2- 172.21.15.148:0/1448797252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc0cd540 0x7f35fc0caad0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06ded0 tx=0x7f35f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f25c432b451e5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.688+0000 7f35faffd640 1 -- 172.21.15.148:0/1448797252 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4036070 con 0x7f35fc0cd540 2024-04-25T10:03:47.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.688+0000 7f35faffd640 1 -- 172.21.15.148:0/1448797252 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4031600 con 0x7f35fc0cd540 2024-04-25T10:03:47.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.688+0000 7f35faffd640 1 -- 172.21.15.148:0/1448797252 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40319d0 con 0x7f35fc0cd540 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 -- 172.21.15.148:0/1448797252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc0cd540 msgr2=0x7f35fc0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 --2- 172.21.15.148:0/1448797252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc0cd540 0x7f35fc0caad0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f35fc06ded0 tx=0x7f35f402a660 comp rx=0 tx=0).stop 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 -- 172.21.15.148:0/1448797252 shutdown_connections 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 --2- 172.21.15.148:0/1448797252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc0cd540 0x7f35fc0caad0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 --2- 172.21.15.148:0/1448797252 >> v2:172.21.15.188:3300/0 conn(0x7f35fc162140 0x7f35fc0cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 -- 172.21.15.148:0/1448797252 >> 172.21.15.148:0/1448797252 conn(0x7f35fc0d17c0 msgr2=0x7f35fc0d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 -- 172.21.15.148:0/1448797252 shutdown_connections 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.689+0000 7f360250a640 1 -- 172.21.15.148:0/1448797252 wait complete. 2024-04-25T10:03:47.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.690+0000 7f360250a640 1 Processor -- start 2024-04-25T10:03:47.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.690+0000 7f360250a640 1 -- start start 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35fc0cd540 0x7f35fc0a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35fc0d0a00 0x7f35fc0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 0x7f35fc143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc161940 con 0x7f35fc162140 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc161660 con 0x7f35fc0cd540 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc161c20 con 0x7f35fc0d0a00 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 0x7f35fc143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 0x7f35fc143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46358/0 (socket says 172.21.15.148:46358) 2024-04-25T10:03:47.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 -- 172.21.15.148:0/1320084276 learned_addr learned my addr 172.21.15.148:0/1320084276 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35fbfff640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35fc0cd540 0x7f35fc0a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 -- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35fc0d0a00 msgr2=0x7f35fc0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35fb7fe640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35fc0d0a00 0x7f35fc0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35fc0d0a00 0x7f35fc0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 -- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35fc0cd540 msgr2=0x7f35fc0a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35fc0cd540 0x7f35fc0a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 -- 172.21.15.148:0/1320084276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0b0ae0 con 0x7f35fc162140 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35fbfff640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35fc0cd540 0x7f35fc0a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35fb7fe640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35fc0d0a00 0x7f35fc0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f3600aa7640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 0x7f35fc143200 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009f40 tx=0x7f35f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f403f020 con 0x7f35fc162140 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4037040 con 0x7f35fc162140 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc0a8c10 con 0x7f35fc162140 2024-04-25T10:03:47.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.691+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4036070 con 0x7f35fc162140 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.692+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc0a91d0 con 0x7f35fc162140 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.693+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc073a00 con 0x7f35fc162140 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.693+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35f4042020 con 0x7f35fc162140 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.693+0000 7f35f8ff9640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35e806b4d0 0x7f35e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.693+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f35f40aa580 con 0x7f35fc162140 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.693+0000 7f35fbfff640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35e806b4d0 0x7f35e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.694+0000 7f35fbfff640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35e806b4d0 0x7f35e806d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f35ec007fd0 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.698+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35f4049030 con 0x7f35fc162140 2024-04-25T10:03:47.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.869+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35fc071d60 con 0x7f35fc162140 2024-04-25T10:03:47.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.869+0000 7f35f8ff9640 1 -- 172.21.15.148:0/1320084276 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35f4079000 con 0x7f35fc162140 2024-04-25T10:03:47.874 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:03:47.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35e806b4d0 msgr2=0x7f35e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35e806b4d0 0x7f35e806d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f35ec007fd0 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 msgr2=0x7f35fc143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 0x7f35fc143200 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009f40 tx=0x7f35f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 shutdown_connections 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35e806b4d0 0x7f35e806d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35fc162140 0x7f35fc143200 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35fc0d0a00 0x7f35fc0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 --2- 172.21.15.148:0/1320084276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35fc0cd540 0x7f35fc0a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 >> 172.21.15.148:0/1320084276 conn(0x7f35fc0d17c0 msgr2=0x7f35fc0cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 shutdown_connections 2024-04-25T10:03:47.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.871+0000 7f360250a640 1 -- 172.21.15.148:0/1320084276 wait complete. 2024-04-25T10:03:47.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:03:47.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.979+0000 7f0e391cf640 1 Processor -- start 2024-04-25T10:03:47.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.979+0000 7f0e391cf640 1 -- start start 2024-04-25T10:03:47.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e391cf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e340d1120 0x7f0e3409a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e391cf640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3406c220 con 0x7f0e3415dc70 2024-04-25T10:03:47.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e391cf640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e340d0080 con 0x7f0e3406ce10 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e391cf640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e340d0c90 con 0x7f0e340d1120 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e32d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0e3415dc70 0x7f0e3406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35982/0 (socket says 172.21.15.148:35982) 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e32d9d640 1 -- 172.21.15.148:0/3382243377 learned_addr learned my addr 172.21.15.148:0/3382243377 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e3359e640 1 --2- 172.21.15.148:0/3382243377 >> v2:172.21.15.148:3300/0 conn(0x7f0e340d1120 0x7f0e3409a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.980+0000 7f0e3359e640 1 -- 172.21.15.148:0/3382243377 >> v1:172.21.15.188:6789/0 conn(0x7f0e3406ce10 legacy=0x7f0e340cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e3359e640 1 -- 172.21.15.148:0/3382243377 >> v1:172.21.15.148:6789/0 conn(0x7f0e3415dc70 legacy=0x7f0e3406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e3359e640 1 -- 172.21.15.148:0/3382243377 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340d0360 con 0x7f0e340d1120 2024-04-25T10:03:47.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e31d9b640 1 -- 172.21.15.148:0/3382243377 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873888550 0 0) 0x7f0e3406c220 con 0x7f0e3415dc70 2024-04-25T10:03:47.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e3359e640 1 --2- 172.21.15.148:0/3382243377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e340d1120 0x7f0e3409a450 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0097a0 tx=0x7f0e2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2f6139c2743100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e31d9b640 1 -- 172.21.15.148:0/3382243377 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c037070 con 0x7f0e340d1120 2024-04-25T10:03:47.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e31d9b640 1 -- 172.21.15.148:0/3382243377 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c032440 con 0x7f0e340d1120 2024-04-25T10:03:47.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.981+0000 7f0e31d9b640 1 -- 172.21.15.148:0/3382243377 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c032810 con 0x7f0e340d1120 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 -- 172.21.15.148:0/3382243377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e340d1120 msgr2=0x7f0e3409a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 --2- 172.21.15.148:0/3382243377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e340d1120 0x7f0e3409a450 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0097a0 tx=0x7f0e2c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 -- 172.21.15.148:0/3382243377 shutdown_connections 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 --2- 172.21.15.148:0/3382243377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e340d1120 0x7f0e3409a450 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 -- 172.21.15.148:0/3382243377 >> 172.21.15.148:0/3382243377 conn(0x7f0e340b3270 msgr2=0x7f0e340b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 -- 172.21.15.148:0/3382243377 shutdown_connections 2024-04-25T10:03:47.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.982+0000 7f0e391cf640 1 -- 172.21.15.148:0/3382243377 wait complete. 2024-04-25T10:03:47.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.983+0000 7f0e391cf640 1 Processor -- start 2024-04-25T10:03:47.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.983+0000 7f0e391cf640 1 -- start start 2024-04-25T10:03:47.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e391cf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e3406ce10 0x7f0e34158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e391cf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e340d1120 0x7f0e340c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e391cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 0x7f0e340ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e391cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3415e050 con 0x7f0e3415dc70 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e391cf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340d0920 con 0x7f0e3406ce10 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e391cf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340cf9c0 con 0x7f0e340d1120 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e3359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 0x7f0e340ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e3359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 0x7f0e340ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46378/0 (socket says 172.21.15.148:46378) 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e3359e640 1 -- 172.21.15.148:0/2356781127 learned_addr learned my addr 172.21.15.148:0/2356781127 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e3259c640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e340d1120 0x7f0e340c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.984+0000 7f0e32d9d640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e3406ce10 0x7f0e34158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e3359e640 1 -- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e340d1120 msgr2=0x7f0e340c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e3359e640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e340d1120 0x7f0e340c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e3359e640 1 -- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e3406ce10 msgr2=0x7f0e34158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e3359e640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e3406ce10 0x7f0e34158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e3359e640 1 -- 172.21.15.148:0/2356781127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340b47a0 con 0x7f0e3415dc70 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e32d9d640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e3406ce10 0x7f0e34158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e3359e640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 0x7f0e340ca530 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0098c0 tx=0x7f0e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c041070 con 0x7f0e3415dc70 2024-04-25T10:03:47.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c03c870 con 0x7f0e3415dc70 2024-04-25T10:03:47.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c037040 con 0x7f0e3415dc70 2024-04-25T10:03:47.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e340b4a80 con 0x7f0e3415dc70 2024-04-25T10:03:47.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.985+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e340b5040 con 0x7f0e3415dc70 2024-04-25T10:03:47.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.986+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e34073990 con 0x7f0e3415dc70 2024-04-25T10:03:47.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.986+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e2c044070 con 0x7f0e3415dc70 2024-04-25T10:03:47.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.987+0000 7f0e1b7fe640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e2006b500 0x7f0e2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.987+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0e2c0aaaf0 con 0x7f0e3415dc70 2024-04-25T10:03:47.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.987+0000 7f0e32d9d640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e2006b500 0x7f0e2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.987+0000 7f0e32d9d640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e2006b500 0x7f0e2006d9c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0e24007fd0 tx=0x7f0e24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:47.991+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e2c079c50 con 0x7f0e3415dc70 2024-04-25T10:03:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.163+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0e340cfdc0 con 0x7f0e3415dc70 2024-04-25T10:03:48.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.163+0000 7f0e1b7fe640 1 -- 172.21.15.148:0/2356781127 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0e2c079550 con 0x7f0e3415dc70 2024-04-25T10:03:48.168 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:48.168 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:03:48.168 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:03:48.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:03:48.169 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:03:48.169 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:03:48.169 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e2006b500 msgr2=0x7f0e2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e2006b500 0x7f0e2006d9c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0e24007fd0 tx=0x7f0e24009040 comp rx=0 tx=0).stop 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 msgr2=0x7f0e340ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 0x7f0e340ca530 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0098c0 tx=0x7f0e2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 shutdown_connections 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e2006b500 0x7f0e2006d9c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e3415dc70 0x7f0e340ca530 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e340d1120 0x7f0e340c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 --2- 172.21.15.148:0/2356781127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e3406ce10 0x7f0e34158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 >> 172.21.15.148:0/2356781127 conn(0x7f0e340b3270 msgr2=0x7f0e340ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 shutdown_connections 2024-04-25T10:03:48.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.165+0000 7f0e391cf640 1 -- 172.21.15.148:0/2356781127 wait complete. 2024-04-25T10:03:48.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:03:48.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:03:48.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.277+0000 7f6ca8ee1640 1 Processor -- start 2024-04-25T10:03:48.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.277+0000 7f6ca8ee1640 1 -- start start 2024-04-25T10:03:48.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca8ee1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ca40d3320 0x7f6ca41689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca8ee1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca4161d70 con 0x7f6ca415da90 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca8ee1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca4162040 con 0x7f6ca40b9750 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca8ee1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca4162600 con 0x7f6ca40d3320 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca2d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ca40d3320 0x7f6ca41689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6ca415da90 0x7f6ca4161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35998/0 (socket says 172.21.15.148:35998) 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.278+0000 7f6ca259c640 1 -- 172.21.15.148:0/1647450058 learned_addr learned my addr 172.21.15.148:0/1647450058 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca2d9d640 1 -- 172.21.15.148:0/1647450058 >> v1:172.21.15.188:6789/0 conn(0x7f6ca40b9750 legacy=0x7f6ca40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca2d9d640 1 -- 172.21.15.148:0/1647450058 >> v1:172.21.15.148:6789/0 conn(0x7f6ca415da90 legacy=0x7f6ca4161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca159a640 1 -- 172.21.15.148:0/1647450058 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526457667 0 0) 0x7f6ca4161d70 con 0x7f6ca415da90 2024-04-25T10:03:48.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca2d9d640 1 -- 172.21.15.148:0/1647450058 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4162320 con 0x7f6ca40d3320 2024-04-25T10:03:48.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca2d9d640 1 --2- 172.21.15.148:0/1647450058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca41689f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0097a0 tx=0x7f6c9c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8555a95be06f1b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca159a640 1 -- 172.21.15.148:0/1647450058 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c037070 con 0x7f6ca40d3320 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca159a640 1 -- 172.21.15.148:0/1647450058 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c02ae50 con 0x7f6ca40d3320 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.279+0000 7f6ca159a640 1 -- 172.21.15.148:0/1647450058 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c032660 con 0x7f6ca40d3320 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/1647450058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 msgr2=0x7f6ca41689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/1647450058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca41689f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0097a0 tx=0x7f6c9c02a140 comp rx=0 tx=0).stop 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/1647450058 shutdown_connections 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/1647450058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca41689f0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/1647450058 >> 172.21.15.148:0/1647450058 conn(0x7f6ca4061a20 msgr2=0x7f6ca4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/1647450058 shutdown_connections 2024-04-25T10:03:48.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.280+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/1647450058 wait complete. 2024-04-25T10:03:48.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.281+0000 7f6ca8ee1640 1 Processor -- start 2024-04-25T10:03:48.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.281+0000 7f6ca8ee1640 1 -- start start 2024-04-25T10:03:48.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca8ee1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca40cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca8ee1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca415da90 0x7f6ca4155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca8ee1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca415e100 0x7f6ca406c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca8ee1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca415de70 con 0x7f6ca40d3320 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca8ee1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40d1df0 con 0x7f6ca415da90 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca8ee1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40ced80 con 0x7f6ca415e100 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca40cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca40cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46410/0 (socket says 172.21.15.148:46410) 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 -- 172.21.15.148:0/112629061 learned_addr learned my addr 172.21.15.148:0/112629061 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca1d9b640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca415da90 0x7f6ca4155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca2d9d640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca415e100 0x7f6ca406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 -- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca415e100 msgr2=0x7f6ca406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca415e100 0x7f6ca406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 -- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca415da90 msgr2=0x7f6ca4155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca415da90 0x7f6ca4155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.282+0000 7f6ca259c640 1 -- 172.21.15.148:0/112629061 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca40cb820 con 0x7f6ca40d3320 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.283+0000 7f6ca259c640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca40cc520 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c7c0 tx=0x7f6c9400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.283+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94018070 con 0x7f6ca40d3320 2024-04-25T10:03:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.283+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9400a640 con 0x7f6ca40d3320 2024-04-25T10:03:48.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.283+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400aa30 con 0x7f6ca40d3320 2024-04-25T10:03:48.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.283+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca40c76e0 con 0x7f6ca40d3320 2024-04-25T10:03:48.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.283+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca40c7ca0 con 0x7f6ca40d3320 2024-04-25T10:03:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.284+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca40c7ea0 con 0x7f6ca40d3320 2024-04-25T10:03:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.284+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c9401b070 con 0x7f6ca40d3320 2024-04-25T10:03:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.284+0000 7f6c92ffd640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c8406b4d0 0x7f6c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.285+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6c94089700 con 0x7f6ca40d3320 2024-04-25T10:03:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.285+0000 7f6ca1d9b640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c8406b4d0 0x7f6c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.285+0000 7f6ca1d9b640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c8406b4d0 0x7f6c8406d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6ca41f6d60 tx=0x7f6c98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.289+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c940586b0 con 0x7f6ca40d3320 2024-04-25T10:03:48.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.461+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6ca4073a00 con 0x7f6ca40d3320 2024-04-25T10:03:48.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.872+0000 7f6c92ffd640 1 -- 172.21.15.148:0/112629061 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c94057fb0 con 0x7f6ca40d3320 2024-04-25T10:03:48.877 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c8406b4d0 msgr2=0x7f6c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c8406b4d0 0x7f6c8406d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6ca41f6d60 tx=0x7f6c98009040 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 msgr2=0x7f6ca40cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca40cc520 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c7c0 tx=0x7f6c9400cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 shutdown_connections 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c8406b4d0 0x7f6c8406d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ca415e100 0x7f6ca406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ca415da90 0x7f6ca4155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 --2- 172.21.15.148:0/112629061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ca40d3320 0x7f6ca40cc520 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.875+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 >> 172.21.15.148:0/112629061 conn(0x7f6ca4061a20 msgr2=0x7f6ca40ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.876+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 shutdown_connections 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.876+0000 7f6ca8ee1640 1 -- 172.21.15.148:0/112629061 wait complete. 2024-04-25T10:03:48.894 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:03:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.996+0000 7f495d4ac640 1 Processor -- start 2024-04-25T10:03:49.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.996+0000 7f495d4ac640 1 -- start start 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f495d4ac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f495815dc20 0x7f49580ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f495d4ac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4958161710 0x7f49580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f495d4ac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495815e000 con 0x7f49580b9330 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f495d4ac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49580d1de0 con 0x7f495815dc20 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f495d4ac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4958161c60 con 0x7f4958161710 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f4956ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f495815dc20 0x7f49580ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f49577fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49580b9330 0x7f495806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36004/0 (socket says 172.21.15.148:36004) 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f4956ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f495815dc20 0x7f49580ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46426/0 (socket says 172.21.15.148:46426) 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f49577fe640 1 -- 172.21.15.148:0/2554308570 learned_addr learned my addr 172.21.15.148:0/2554308570 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f494ffff640 1 --2- 172.21.15.148:0/2554308570 >> v2:172.21.15.188:3300/0 conn(0x7f4958161710 0x7f49580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f4956ffd640 1 -- 172.21.15.148:0/2554308570 >> v2:172.21.15.188:3300/0 conn(0x7f4958161710 msgr2=0x7f49580cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f4956ffd640 1 --2- 172.21.15.148:0/2554308570 >> v2:172.21.15.188:3300/0 conn(0x7f4958161710 0x7f49580cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f49567fc640 1 -- 172.21.15.148:0/2554308570 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45697838 0 0) 0x7f495815e000 con 0x7f49580b9330 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f4956ffd640 1 -- 172.21.15.148:0/2554308570 >> v1:172.21.15.148:6789/0 conn(0x7f49580b9330 legacy=0x7f495806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f4956ffd640 1 -- 172.21.15.148:0/2554308570 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4958161f40 con 0x7f495815dc20 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.997+0000 7f494ffff640 1 --2- 172.21.15.148:0/2554308570 >> v2:172.21.15.188:3300/0 conn(0x7f4958161710 0x7f49580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.998+0000 7f4956ffd640 1 --2- 172.21.15.148:0/2554308570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f495815dc20 0x7f49580ce670 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f495806da90 tx=0x7f494802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97f373338081ada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.998+0000 7f49567fc640 1 -- 172.21.15.148:0/2554308570 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948037070 con 0x7f495815dc20 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.998+0000 7f49567fc640 1 -- 172.21.15.148:0/2554308570 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948032440 con 0x7f495815dc20 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.998+0000 7f49567fc640 1 -- 172.21.15.148:0/2554308570 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948032810 con 0x7f495815dc20 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 -- 172.21.15.148:0/2554308570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f495815dc20 msgr2=0x7f49580ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 --2- 172.21.15.148:0/2554308570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f495815dc20 0x7f49580ce670 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f495806da90 tx=0x7f494802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 -- 172.21.15.148:0/2554308570 shutdown_connections 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 --2- 172.21.15.148:0/2554308570 >> v2:172.21.15.188:3300/0 conn(0x7f4958161710 0x7f49580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 --2- 172.21.15.148:0/2554308570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f495815dc20 0x7f49580ce670 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 -- 172.21.15.148:0/2554308570 >> 172.21.15.148:0/2554308570 conn(0x7f49580d0420 msgr2=0x7f49580d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:49.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 -- 172.21.15.148:0/2554308570 shutdown_connections 2024-04-25T10:03:49.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:48.999+0000 7f495d4ac640 1 -- 172.21.15.148:0/2554308570 wait complete. 2024-04-25T10:03:49.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 Processor -- start 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 -- start start 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49580b9330 0x7f4958153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f495815dc20 0x7f49580acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 0x7f49580ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49580b5e50 con 0x7f4958161710 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958162500 con 0x7f49580b9330 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.000+0000 7f495d4ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49580b5fd0 con 0x7f495815dc20 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 0x7f49580ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 0x7f49580ad320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46432/0 (socket says 172.21.15.148:46432) 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 -- 172.21.15.148:0/3144627661 learned_addr learned my addr 172.21.15.148:0/3144627661 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 -- 172.21.15.148:0/3144627661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f495815dc20 msgr2=0x7f49580acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f495815dc20 0x7f49580acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 -- 172.21.15.148:0/3144627661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49580b9330 msgr2=0x7f4958153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49580b9330 0x7f4958153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 -- 172.21.15.148:0/3144627661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4958150620 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f49577fe640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 0x7f49580ad320 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f495000c780 tx=0x7f495000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950018070 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495000ce30 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495000a840 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49580aa5b0 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.001+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49580aab70 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.002+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4958073a00 con 0x7f4958161710 2024-04-25T10:03:49.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.003+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f495000a9e0 con 0x7f4958161710 2024-04-25T10:03:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.003+0000 7f494f7fe640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f493806b4d0 0x7f493806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.003+0000 7f4956ffd640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f493806b4d0 0x7f493806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.003+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f4950089750 con 0x7f4958161710 2024-04-25T10:03:49.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.004+0000 7f4956ffd640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f493806b4d0 0x7f493806d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4948034000 tx=0x7f494802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.008+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4958150620 con 0x7f4958161710 2024-04-25T10:03:49.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.180+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f49580d4560 con 0x7f4958161710 2024-04-25T10:03:49.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.877+0000 7f494f7fe640 1 -- 172.21.15.148:0/3144627661 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) v1 ==== 170+0+0 (secure 0 0 0) 0x7f49500586b0 con 0x7f4958161710 2024-04-25T10:03:49.883 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:03:49.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f493806b4d0 msgr2=0x7f493806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f493806b4d0 0x7f493806d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4948034000 tx=0x7f494802c000 comp rx=0 tx=0).stop 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 msgr2=0x7f49580ad320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 0x7f49580ad320 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f495000c780 tx=0x7f495000cc40 comp rx=0 tx=0).stop 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 shutdown_connections 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f493806b4d0 0x7f493806d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4958161710 0x7f49580ad320 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f495815dc20 0x7f49580acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 --2- 172.21.15.148:0/3144627661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49580b9330 0x7f4958153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 >> 172.21.15.148:0/3144627661 conn(0x7f49580d0420 msgr2=0x7f49580d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 shutdown_connections 2024-04-25T10:03:49.886 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:49.880+0000 7f495d4ac640 1 -- 172.21.15.148:0/3144627661 wait complete. 2024-04-25T10:03:49.905 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:03:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.002+0000 7f992ad89640 1 Processor -- start 2024-04-25T10:03:50.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.002+0000 7f992ad89640 1 -- start start 2024-04-25T10:03:50.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f992ad89640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f992406ccd0 0x7f99240b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f992ad89640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99240d1770 con 0x7f9924070dc0 2024-04-25T10:03:50.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f992ad89640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9924162740 con 0x7f992415deb0 2024-04-25T10:03:50.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f992ad89640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99241622b0 con 0x7f992406ccd0 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f9923fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f992406ccd0 0x7f99240b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f9928b25640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9924070dc0 0x7f99240711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36010/0 (socket says 172.21.15.148:36010) 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f9928b25640 1 -- 172.21.15.148:0/979005321 learned_addr learned my addr 172.21.15.148:0/979005321 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f9923fff640 1 -- 172.21.15.148:0/979005321 >> v1:172.21.15.188:6789/0 conn(0x7f992415deb0 legacy=0x7f992415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f9923fff640 1 -- 172.21.15.148:0/979005321 >> v1:172.21.15.148:6789/0 conn(0x7f9924070dc0 legacy=0x7f99240711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f9923fff640 1 -- 172.21.15.148:0/979005321 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924161980 con 0x7f992406ccd0 2024-04-25T10:03:50.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.003+0000 7f99237fe640 1 -- 172.21.15.148:0/979005321 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216533932 0 0) 0x7f99240d1770 con 0x7f9924070dc0 2024-04-25T10:03:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.004+0000 7f9923fff640 1 --2- 172.21.15.148:0/979005321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f99240b07a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f99180097a0 tx=0x7f991802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d80f20e8325d0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.004+0000 7f99237fe640 1 -- 172.21.15.148:0/979005321 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918037070 con 0x7f992406ccd0 2024-04-25T10:03:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.004+0000 7f99237fe640 1 -- 172.21.15.148:0/979005321 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918032440 con 0x7f992406ccd0 2024-04-25T10:03:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.004+0000 7f99237fe640 1 -- 172.21.15.148:0/979005321 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991802ae60 con 0x7f992406ccd0 2024-04-25T10:03:50.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 -- 172.21.15.148:0/979005321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 msgr2=0x7f99240b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 --2- 172.21.15.148:0/979005321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f99240b07a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f99180097a0 tx=0x7f991802a110 comp rx=0 tx=0).stop 2024-04-25T10:03:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 -- 172.21.15.148:0/979005321 shutdown_connections 2024-04-25T10:03:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 --2- 172.21.15.148:0/979005321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f99240b07a0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 -- 172.21.15.148:0/979005321 >> 172.21.15.148:0/979005321 conn(0x7f99240b3530 msgr2=0x7f99240b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 -- 172.21.15.148:0/979005321 shutdown_connections 2024-04-25T10:03:50.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.005+0000 7f992ad89640 1 -- 172.21.15.148:0/979005321 wait complete. 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.006+0000 7f992ad89640 1 Processor -- start 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.006+0000 7f992ad89640 1 -- start start 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f992ad89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f992414ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f992ad89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9924070dc0 0x7f99241604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f992ad89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f992415deb0 0x7f99240cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f992ad89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9924161f40 con 0x7f992406ccd0 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f992ad89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b10f0 con 0x7f9924070dc0 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f992ad89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b5840 con 0x7f992415deb0 2024-04-25T10:03:50.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f992414ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f992414ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46438/0 (socket says 172.21.15.148:46438) 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 -- 172.21.15.148:0/3159431631 learned_addr learned my addr 172.21.15.148:0/3159431631 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9923fff640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9924070dc0 0x7f99241604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9929326640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f992415deb0 0x7f99240cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 -- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f992415deb0 msgr2=0x7f99240cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f992415deb0 0x7f99240cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 -- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9924070dc0 msgr2=0x7f99241604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9924070dc0 0x7f99241604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9928b25640 1 -- 172.21.15.148:0/3159431631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240cb280 con 0x7f992406ccd0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.007+0000 7f9923fff640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9924070dc0 0x7f99241604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.008+0000 7f9928b25640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f992414ec30 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f991400c4a0 tx=0x7f991400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.008+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914018070 con 0x7f992406ccd0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.008+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99140074e0 con 0x7f992406ccd0 2024-04-25T10:03:50.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.008+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992415f410 con 0x7f992406ccd0 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.008+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914004a20 con 0x7f992406ccd0 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.008+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992415f9d0 con 0x7f992406ccd0 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.009+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99241600f0 con 0x7f992406ccd0 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.009+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991401b070 con 0x7f992406ccd0 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.009+0000 7f99217fa640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f991006b4d0 0x7f991006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.010+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9914089610 con 0x7f992406ccd0 2024-04-25T10:03:50.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.010+0000 7f9923fff640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f991006b4d0 0x7f991006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.010+0000 7f9923fff640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f991006b4d0 0x7f991006d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9918034000 tx=0x7f991802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.014+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9914058360 con 0x7f992406ccd0 2024-04-25T10:03:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.221+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f99240b2510 con 0x7f992406ccd0 2024-04-25T10:03:50.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.879+0000 7f99217fa640 1 -- 172.21.15.148:0/3159431631 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9914057c60 con 0x7f992406ccd0 2024-04-25T10:03:50.884 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.883+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f991006b4d0 msgr2=0x7f991006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.883+0000 7f992ad89640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f991006b4d0 0x7f991006d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9918034000 tx=0x7f991802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.883+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 msgr2=0x7f992414ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.883+0000 7f992ad89640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f992414ec30 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f991400c4a0 tx=0x7f991400c960 comp rx=0 tx=0).stop 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.884+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 shutdown_connections 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.884+0000 7f992ad89640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f991006b4d0 0x7f991006d990 secure :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9918034000 tx=0x7f991802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.884+0000 7f992ad89640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f992415deb0 0x7f99240cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.884+0000 7f992ad89640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9924070dc0 0x7f99241604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.884+0000 7f992ad89640 1 --2- 172.21.15.148:0/3159431631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f992406ccd0 0x7f992414ec30 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.884+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 >> 172.21.15.148:0/3159431631 conn(0x7f99240b3530 msgr2=0x7f99240b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.885+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 shutdown_connections 2024-04-25T10:03:50.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:50.886+0000 7f992ad89640 1 -- 172.21.15.148:0/3159431631 wait complete. 2024-04-25T10:03:50.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:51.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.015+0000 7ffb19a4a640 1 Processor -- start 2024-04-25T10:03:51.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.015+0000 7ffb19a4a640 1 -- start start 2024-04-25T10:03:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.016+0000 7ffb19a4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 0x7ffb1409a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.016+0000 7ffb19a4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1406c220 con 0x7ffb1406ce10 2024-04-25T10:03:51.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb19a4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140d0080 con 0x7ffb1415dc70 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb19a4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140d0c90 con 0x7ffb140d1120 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb127fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffb1406ce10 0x7ffb140cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36020/0 (socket says 172.21.15.148:36020) 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb127fc640 1 -- 172.21.15.148:0/1507081038 learned_addr learned my addr 172.21.15.148:0/1507081038 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 0x7ffb1409a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560705890 0 0) 0x7ffb1406c220 con 0x7ffb1406ce10 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc003520 con 0x7ffb1406ce10 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 179231229 0 0) 0x7ffafc003520 con 0x7ffb1406ce10 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 msgr2=0x7ffb1409a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 0x7ffb1409a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 >> v1:172.21.15.188:6789/0 conn(0x7ffb1415dc70 legacy=0x7ffb1406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140d0360 con 0x7ffb1406ce10 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ffb08001670 con 0x7ffb1406ce10 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 >> v1:172.21.15.148:6789/0 conn(0x7ffb1406ce10 legacy=0x7ffb140cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 0x7ffafc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffafc003f60 0x7ffafc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140d0c90 con 0x7ffafc004d40 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00baa0 con 0x7ffafc003f60 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00bca0 con 0x7ffb140d1120 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb127fc640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffafc003f60 0x7ffafc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.017+0000 7ffb12ffd640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 0x7ffafc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb137fe640 1 -- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 msgr2=0x7ffafc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb140d1120 0x7ffafc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb137fe640 1 -- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffafc003f60 msgr2=0x7ffafc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffafc003f60 0x7ffafc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb137fe640 1 -- 172.21.15.148:0/1507081038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc003520 con 0x7ffafc004d40 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009ac0 tx=0x7ffafc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbf8ad8c96f0218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c036070 con 0x7ffafc004d40 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c031400 con 0x7ffafc004d40 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.018+0000 7ffb11ffb640 1 -- 172.21.15.148:0/1507081038 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c009690 con 0x7ffafc004d40 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1507081038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc004d40 msgr2=0x7ffafc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1507081038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009ac0 tx=0x7ffafc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb137fe640 1 -- 172.21.15.148:0/1507081038 reap_dead start 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1507081038 shutdown_connections 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1507081038 >> 172.21.15.148:0/1507081038 conn(0x7ffb140b3270 msgr2=0x7ffb140b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1507081038 shutdown_connections 2024-04-25T10:03:51.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.019+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1507081038 wait complete. 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.020+0000 7ffb19a4a640 1 Processor -- start 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.020+0000 7ffb19a4a640 1 -- start start 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb19a4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb141f2e60 0x7ffb140ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb19a4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb141f3240 0x7ffb140b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb19a4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 0x7ffb140b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb19a4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415e050 con 0x7ffb1409b610 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb19a4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140d0920 con 0x7ffb141f3240 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb19a4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1409a450 con 0x7ffb141f2e60 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 0x7ffb140b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 0x7ffb140b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46452/0 (socket says 172.21.15.148:46452) 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 -- 172.21.15.148:0/1151931267 learned_addr learned my addr 172.21.15.148:0/1151931267 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb12ffd640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb141f2e60 0x7ffb140ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb127fc640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb141f3240 0x7ffb140b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 -- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb141f2e60 msgr2=0x7ffb140ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb141f2e60 0x7ffb140ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 -- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb141f3240 msgr2=0x7ffb140b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb141f3240 0x7ffb140b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb137fe640 1 -- 172.21.15.148:0/1151931267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1409f3f0 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.021+0000 7ffb127fc640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb141f3240 0x7ffb140b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.022+0000 7ffb137fe640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 0x7ffb140b6980 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c02abb0 tx=0x7ffb0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.022+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c041070 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.022+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c037040 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.022+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140b59a0 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.022+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c036070 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.022+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140b5c40 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.023+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb14073990 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.023+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb0c044070 con 0x7ffb1409b610 2024-04-25T10:03:51.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.024+0000 7ffafbfff640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb0006b4d0 0x7ffb0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.024+0000 7ffb12ffd640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb0006b4d0 0x7ffb0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.024+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb0c0a98b0 con 0x7ffb1409b610 2024-04-25T10:03:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.024+0000 7ffb12ffd640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb0006b4d0 0x7ffb0006d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ffb04001ee0 tx=0x7ffb04009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.028+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb0c077f50 con 0x7ffb1409b610 2024-04-25T10:03:51.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.199+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb140b40a0 con 0x7ffb1409b610 2024-04-25T10:03:51.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.201+0000 7ffafbfff640 1 -- 172.21.15.148:0/1151931267 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16953 (secure 0 0 0) 0x7ffb0c07bff0 con 0x7ffb1409b610 2024-04-25T10:03:51.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:51.206 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":144,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:50.222811+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.202+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb0006b4d0 msgr2=0x7ffb0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.202+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb0006b4d0 0x7ffb0006d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ffb04001ee0 tx=0x7ffb04009040 comp rx=0 tx=0).stop 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 msgr2=0x7ffb140b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 0x7ffb140b6980 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c02abb0 tx=0x7ffb0c004730 comp rx=0 tx=0).stop 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 shutdown_connections 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb0006b4d0 0x7ffb0006d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb1409b610 0x7ffb140b6980 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.208 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb141f3240 0x7ffb140b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 --2- 172.21.15.148:0/1151931267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb141f2e60 0x7ffb140ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 >> 172.21.15.148:0/1151931267 conn(0x7ffb140b3270 msgr2=0x7ffb140cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:51.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 shutdown_connections 2024-04-25T10:03:51.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.203+0000 7ffb19a4a640 1 -- 172.21.15.148:0/1151931267 wait complete. 2024-04-25T10:03:51.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:51.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.311+0000 7fcea1522640 1 Processor -- start 2024-04-25T10:03:51.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.311+0000 7fcea1522640 1 -- start start 2024-04-25T10:03:51.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fcea1522640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce9c15b9a0 0x7fce9c072c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fcea1522640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fcea1522640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce9c06b120 0x7fce9c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fcea1522640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9c160530 con 0x7fce9c06b120 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fcea1522640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9c0731d0 con 0x7fce9c15b9a0 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fcea1522640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c15f8b0 con 0x7fce9c0cf780 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fce9b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce9c06b120 0x7fce9c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fce9b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fce9c06b120 0x7fce9c0cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46466/0 (socket says 172.21.15.148:46466) 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fce9b7fe640 1 -- 172.21.15.148:0/2288487832 learned_addr learned my addr 172.21.15.148:0/2288487832 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fce9affd640 1 --2- 172.21.15.148:0/2288487832 >> v2:172.21.15.188:3300/0 conn(0x7fce9c15b9a0 0x7fce9c072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.312+0000 7fce9a7fc640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9b7fe640 1 -- 172.21.15.148:0/2288487832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 msgr2=0x7fce9c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9b7fe640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9b7fe640 1 -- 172.21.15.148:0/2288487832 >> v2:172.21.15.188:3300/0 conn(0x7fce9c15b9a0 msgr2=0x7fce9c072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9b7fe640 1 --2- 172.21.15.148:0/2288487832 >> v2:172.21.15.188:3300/0 conn(0x7fce9c15b9a0 0x7fce9c072c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9b7fe640 1 -- 172.21.15.148:0/2288487832 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c15fb90 con 0x7fce9c06b120 2024-04-25T10:03:51.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9a7fc640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce9b7fe640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c06b120 0x7fce9c0cd3a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fce9c06baa0 tx=0x7fce9402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08ac7de0c68290b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.313+0000 7fce99ffb640 1 -- 172.21.15.148:0/2288487832 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94037070 con 0x7fce9c06b120 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fce99ffb640 1 -- 172.21.15.148:0/2288487832 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce94032440 con 0x7fce9c06b120 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fce99ffb640 1 -- 172.21.15.148:0/2288487832 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94032810 con 0x7fce9c06b120 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 -- 172.21.15.148:0/2288487832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c06b120 msgr2=0x7fce9c0cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c06b120 0x7fce9c0cd3a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fce9c06baa0 tx=0x7fce9402a140 comp rx=0 tx=0).stop 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 -- 172.21.15.148:0/2288487832 shutdown_connections 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c06b120 0x7fce9c0cd3a0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 --2- 172.21.15.148:0/2288487832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 --2- 172.21.15.148:0/2288487832 >> v2:172.21.15.188:3300/0 conn(0x7fce9c15b9a0 0x7fce9c072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 -- 172.21.15.148:0/2288487832 >> 172.21.15.148:0/2288487832 conn(0x7fce9c0b23e0 msgr2=0x7fce9c0b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:51.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 -- 172.21.15.148:0/2288487832 shutdown_connections 2024-04-25T10:03:51.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.314+0000 7fcea1522640 1 -- 172.21.15.148:0/2288487832 wait complete. 2024-04-25T10:03:51.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.315+0000 7fcea1522640 1 Processor -- start 2024-04-25T10:03:51.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.315+0000 7fcea1522640 1 -- start start 2024-04-25T10:03:51.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fcea1522640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce9c06b120 0x7fce9c1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fcea1522640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fcea1522640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 0x7fce9c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fcea1522640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c15f560 con 0x7fce9c15b9a0 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fcea1522640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c160150 con 0x7fce9c06b120 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fcea1522640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c070b70 con 0x7fce9c0cf780 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 0x7fce9c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 0x7fce9c158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46474/0 (socket says 172.21.15.148:46474) 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 -- 172.21.15.148:0/3237766935 learned_addr learned my addr 172.21.15.148:0/3237766935 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:51.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9affd640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce9c06b120 0x7fce9c1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9a7fc640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 -- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 msgr2=0x7fce9c1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c1583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 -- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce9c06b120 msgr2=0x7fce9c1505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce9c06b120 0x7fce9c1505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.316+0000 7fce9b7fe640 1 -- 172.21.15.148:0/3237766935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c155ea0 con 0x7fce9c15b9a0 2024-04-25T10:03:51.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.317+0000 7fce9b7fe640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 0x7fce9c158900 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fce940090b0 tx=0x7fce940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.317+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94041020 con 0x7fce9c15b9a0 2024-04-25T10:03:51.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.317+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9403d640 con 0x7fce9c15b9a0 2024-04-25T10:03:51.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.317+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c156180 con 0x7fce9c15b9a0 2024-04-25T10:03:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.317+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94037040 con 0x7fce9c15b9a0 2024-04-25T10:03:51.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.317+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c156740 con 0x7fce9c15b9a0 2024-04-25T10:03:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.318+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9c073990 con 0x7fce9c15b9a0 2024-04-25T10:03:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.318+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce94044020 con 0x7fce9c15b9a0 2024-04-25T10:03:51.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.318+0000 7fce837fe640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce8806b4d0 0x7fce8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.319+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce940aaab0 con 0x7fce9c15b9a0 2024-04-25T10:03:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.319+0000 7fce9affd640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce8806b4d0 0x7fce8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.319+0000 7fce9affd640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce8806b4d0 0x7fce8806d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007380 tx=0x7fce8c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.323+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce9404b030 con 0x7fce9c15b9a0 2024-04-25T10:03:51.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.493+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce9c1f12e0 con 0x7fce9c15b9a0 2024-04-25T10:03:51.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.494+0000 7fce837fe640 1 -- 172.21.15.148:0/3237766935 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16953 (secure 0 0 0) 0x7fce940797b0 con 0x7fce9c15b9a0 2024-04-25T10:03:51.499 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:51.499 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":144,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:50.222811+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:51.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce8806b4d0 msgr2=0x7fce8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce8806b4d0 0x7fce8806d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fce8c007380 tx=0x7fce8c00b040 comp rx=0 tx=0).stop 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 msgr2=0x7fce9c158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 0x7fce9c158900 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fce940090b0 tx=0x7fce940046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 shutdown_connections 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fce8806b4d0 0x7fce8806d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce9c15b9a0 0x7fce9c158900 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce9c0cf780 0x7fce9c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 --2- 172.21.15.148:0/3237766935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce9c06b120 0x7fce9c1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 >> 172.21.15.148:0/3237766935 conn(0x7fce9c0b23e0 msgr2=0x7fce9c0b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:51.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 shutdown_connections 2024-04-25T10:03:51.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.496+0000 7fcea1522640 1 -- 172.21.15.148:0/3237766935 wait complete. 2024-04-25T10:03:51.513 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:03:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 Processor -- start 2024-04-25T10:03:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 -- start start 2024-04-25T10:03:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbdd00b5620 0x7fbdd00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 0x7fbdd006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd006cc80 con 0x7fbdd015b7d0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd006cf50 con 0x7fbdd00b5620 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdd546f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd00ce530 con 0x7fbdd006c2d0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.608+0000 7fbdceffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbdd015b7d0 0x7fbdd00cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36026/0 (socket says 172.21.15.148:36026) 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdceffd640 1 -- 172.21.15.148:0/4229991608 learned_addr learned my addr 172.21.15.148:0/4229991608 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/4229991608 >> v2:172.21.15.188:3300/0 conn(0x7fbdd00b5620 0x7fbdd00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcf7fe640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 0x7fbdd006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041369307 0 0) 0x7fbdd006cc80 con 0x7fbdd015b7d0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdbc003520 con 0x7fbdd015b7d0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845057911 0 0) 0x7fbdbc003520 con 0x7fbdd015b7d0 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 msgr2=0x7fbdd006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 0x7fbdd006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 >> v2:172.21.15.188:3300/0 conn(0x7fbdd00b5620 msgr2=0x7fbdd00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 --2- 172.21.15.148:0/4229991608 >> v2:172.21.15.188:3300/0 conn(0x7fbdd00b5620 0x7fbdd00b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00ce810 con 0x7fbdd015b7d0 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fbdc80019f0 con 0x7fbdd015b7d0 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/4229991608 >> v2:172.21.15.188:3300/0 conn(0x7fbdd00b5620 0x7fbdd00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 >> v1:172.21.15.148:6789/0 conn(0x7fbdd015b7d0 legacy=0x7fbdd00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 0x7fbdbc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdbc003f30 0x7fbdbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdbc004d40 0x7fbdbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd006cf50 con 0x7fbdbc003f30 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd00ce530 con 0x7fbdbc004d40 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc00b9c0 con 0x7fbdd006c2d0 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdbc003f30 0x7fbdbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdceffd640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 0x7fbdbc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdcf7fe640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdbc004d40 0x7fbdbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 -- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 msgr2=0x7fbdbc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdd006c2d0 0x7fbdbc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 -- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdbc004d40 msgr2=0x7fbdbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdbc004d40 0x7fbdbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.609+0000 7fbdce7fc640 1 -- 172.21.15.148:0/4229991608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdbc003520 con 0x7fbdbc003f30 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.610+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdbc003f30 0x7fbdbc004800 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009c90 tx=0x7fbdbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1878ea82f32fe72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.610+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8036070 con 0x7fbdbc003f30 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.610+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8031400 con 0x7fbdbc003f30 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.610+0000 7fbdcdffb640 1 -- 172.21.15.148:0/4229991608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8009860 con 0x7fbdbc003f30 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdd546f640 1 -- 172.21.15.148:0/4229991608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdbc003f30 msgr2=0x7fbdbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdd546f640 1 --2- 172.21.15.148:0/4229991608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdbc003f30 0x7fbdbc004800 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009c90 tx=0x7fbdbc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdcf7fe640 1 -- 172.21.15.148:0/4229991608 reap_dead start 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdd546f640 1 -- 172.21.15.148:0/4229991608 shutdown_connections 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdd546f640 1 -- 172.21.15.148:0/4229991608 >> 172.21.15.148:0/4229991608 conn(0x7fbdd00d0800 msgr2=0x7fbdd00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdd546f640 1 -- 172.21.15.148:0/4229991608 shutdown_connections 2024-04-25T10:03:51.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.611+0000 7fbdd546f640 1 -- 172.21.15.148:0/4229991608 wait complete. 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.612+0000 7fbdd546f640 1 Processor -- start 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.612+0000 7fbdd546f640 1 -- start start 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdd546f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdbc003f30 0x7fbdd015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdd546f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 0x7fbdd014add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdd546f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdd00cb0f0 0x7fbdd014b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdd546f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd015bbb0 con 0x7fbdd00cac70 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdd546f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd006d230 con 0x7fbdd00cb0f0 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdd546f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd015bd30 con 0x7fbdbc003f30 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 0x7fbdd014add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 0x7fbdd014add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46498/0 (socket says 172.21.15.148:46498) 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 -- 172.21.15.148:0/2166127121 learned_addr learned my addr 172.21.15.148:0/2166127121 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:51.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdceffd640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdbc003f30 0x7fbdd015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdcf7fe640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdd00cb0f0 0x7fbdd014b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 -- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdbc003f30 msgr2=0x7fbdd015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdbc003f30 0x7fbdd015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 -- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdd00cb0f0 msgr2=0x7fbdd014b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.613+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdd00cb0f0 0x7fbdd014b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdce7fc640 1 -- 172.21.15.148:0/2166127121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00c8660 con 0x7fbdd00cac70 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdcf7fe640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdd00cb0f0 0x7fbdd014b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdce7fc640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 0x7fbdd014add0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8000c00 tx=0x7fbdb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8040070 con 0x7fbdd00cac70 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb803b680 con 0x7fbdd00cac70 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd00cb6a0 con 0x7fbdd00cac70 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8036040 con 0x7fbdd00cac70 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.614+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd00c8900 con 0x7fbdd00cac70 2024-04-25T10:03:51.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.615+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd0073a00 con 0x7fbdd00cac70 2024-04-25T10:03:51.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.615+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdb8043070 con 0x7fbdd00cac70 2024-04-25T10:03:51.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.616+0000 7fbdc77fe640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbdb006b500 0x7fbdb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:51.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.616+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbdb80aaaf0 con 0x7fbdd00cac70 2024-04-25T10:03:51.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.616+0000 7fbdceffd640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbdb006b500 0x7fbdb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:51.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.616+0000 7fbdceffd640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbdb006b500 0x7fbdb006d9c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0160e40 tx=0x7fbdc8006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:51.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.620+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdb8079190 con 0x7fbdd00cac70 2024-04-25T10:03:51.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.826+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbdd015eda0 con 0x7fbdd00cac70 2024-04-25T10:03:51.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.894+0000 7fbdc77fe640 1 -- 172.21.15.148:0/2166127121 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbdb807d230 con 0x7fbdd00cac70 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbdb006b500 msgr2=0x7fbdb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbdb006b500 0x7fbdb006d9c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0160e40 tx=0x7fbdc8006040 comp rx=0 tx=0).stop 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 msgr2=0x7fbdd014add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 0x7fbdd014add0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8000c00 tx=0x7fbdb8004730 comp rx=0 tx=0).stop 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 shutdown_connections 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbdb006b500 0x7fbdb006d9c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdd00cb0f0 0x7fbdd014b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdd00cac70 0x7fbdd014add0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 --2- 172.21.15.148:0/2166127121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdbc003f30 0x7fbdd015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:51.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 >> 172.21.15.148:0/2166127121 conn(0x7fbdd00d0800 msgr2=0x7fbdd015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:51.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 shutdown_connections 2024-04-25T10:03:51.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:51.897+0000 7fbdd546f640 1 -- 172.21.15.148:0/2166127121 wait complete. 2024-04-25T10:03:51.913 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:03:51.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:03:52.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.008+0000 7f1d0079d640 1 Processor -- start 2024-04-25T10:03:52.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.008+0000 7f1d0079d640 1 -- start start 2024-04-25T10:03:52.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1d0079d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1d0079d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cf80d08e0 0x7f1cf80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1d0079d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf80d2420 con 0x7f1cf815dac0 2024-04-25T10:03:52.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1d0079d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf80d3030 con 0x7f1cf80d08e0 2024-04-25T10:03:52.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1d0079d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80d2700 con 0x7f1cf80d34c0 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cf80d08e0 0x7f1cf80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f1cf80d08e0 0x7f1cf80b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46506/0 (socket says 172.21.15.148:46506) 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 -- 172.21.15.148:0/2544042888 learned_addr learned my addr 172.21.15.148:0/2544042888 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfdd38640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 -- 172.21.15.148:0/2544042888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 msgr2=0x7f1cf80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf80d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 -- 172.21.15.148:0/2544042888 >> v1:172.21.15.188:6789/0 conn(0x7f1cf815dac0 legacy=0x7f1cf80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfed3a640 1 -- 172.21.15.148:0/2544042888 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80d29e0 con 0x7f1cf80d08e0 2024-04-25T10:03:52.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.009+0000 7f1cfdd38640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf80d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:52.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.010+0000 7f1cfed3a640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf80d08e0 0x7f1cf80b6990 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009e50 tx=0x7f1cf002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16507f56a46e6b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.010+0000 7f1cfd537640 1 -- 172.21.15.148:0/2544042888 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0036070 con 0x7f1cf80d08e0 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.010+0000 7f1cfd537640 1 -- 172.21.15.148:0/2544042888 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0031600 con 0x7f1cf80d08e0 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.010+0000 7f1cfd537640 1 -- 172.21.15.148:0/2544042888 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00319d0 con 0x7f1cf80d08e0 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.010+0000 7f1d0079d640 1 -- 172.21.15.148:0/2544042888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf80d08e0 msgr2=0x7f1cf80b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.010+0000 7f1d0079d640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf80d08e0 0x7f1cf80b6990 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009e50 tx=0x7f1cf002a750 comp rx=0 tx=0).stop 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.011+0000 7f1d0079d640 1 -- 172.21.15.148:0/2544042888 shutdown_connections 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.011+0000 7f1d0079d640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf80d08e0 0x7f1cf80b6990 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.011+0000 7f1d0079d640 1 --2- 172.21.15.148:0/2544042888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf80d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.011+0000 7f1d0079d640 1 -- 172.21.15.148:0/2544042888 >> 172.21.15.148:0/2544042888 conn(0x7f1cf8061c30 msgr2=0x7f1cf806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.011+0000 7f1d0079d640 1 -- 172.21.15.148:0/2544042888 shutdown_connections 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.011+0000 7f1d0079d640 1 -- 172.21.15.148:0/2544042888 wait complete. 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 Processor -- start 2024-04-25T10:03:52.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 -- start start 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cf80d08e0 0x7f1cf8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 0x7f1cf815e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf815dea0 con 0x7f1cf815dac0 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.012+0000 7f1d0079d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80d2100 con 0x7f1cf80d08e0 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1d0079d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80b97d0 con 0x7f1cf80d34c0 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 0x7f1cf815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 0x7f1cf815e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46514/0 (socket says 172.21.15.148:46514) 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 -- 172.21.15.148:0/3765750644 learned_addr learned my addr 172.21.15.148:0/3765750644 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfe539640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cf80d08e0 0x7f1cf8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfdd38640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 -- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 msgr2=0x7f1cf815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf815e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 -- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cf80d08e0 msgr2=0x7f1cf8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cf80d08e0 0x7f1cf8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 -- 172.21.15.148:0/3765750644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf815edf0 con 0x7f1cf815dac0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfe539640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cf80d08e0 0x7f1cf8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1cfed3a640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 0x7f1cf815e560 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009f70 tx=0x7f1cf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0036050 con 0x7f1cf815dac0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf0042070 con 0x7f1cf815dac0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0009510 con 0x7f1cf815dac0 2024-04-25T10:03:52.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80b4e00 con 0x7f1cf815dac0 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.013+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf80b53c0 con 0x7f1cf815dac0 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.014+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf8073a00 con 0x7f1cf815dac0 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.015+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cf0007bd0 con 0x7f1cf815dac0 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.015+0000 7f1ceeffd640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cdc06b4d0 0x7f1cdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.015+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cf00ab2c0 con 0x7f1cf815dac0 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.015+0000 7f1cfe539640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cdc06b4d0 0x7f1cdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.015+0000 7f1cfe539640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cdc06b4d0 0x7f1cdc06d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80b6020 tx=0x7f1cf4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.019+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cf007a040 con 0x7f1cf815dac0 2024-04-25T10:03:52.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.224+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1cf815ae50 con 0x7f1cf815dac0 2024-04-25T10:03:52.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.900+0000 7f1ceeffd640 1 -- 172.21.15.148:0/3765750644 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1cf0079940 con 0x7f1cf815dac0 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cdc06b4d0 msgr2=0x7f1cdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cdc06b4d0 0x7f1cdc06d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80b6020 tx=0x7f1cf4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 msgr2=0x7f1cf815e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 0x7f1cf815e560 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009f70 tx=0x7f1cf0004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 shutdown_connections 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cdc06b4d0 0x7f1cdc06d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cf815dac0 0x7f1cf815e560 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cf80d34c0 0x7f1cf815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 --2- 172.21.15.148:0/3765750644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cf80d08e0 0x7f1cf8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 >> 172.21.15.148:0/3765750644 conn(0x7f1cf8061c30 msgr2=0x7f1cf80c7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:52.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 shutdown_connections 2024-04-25T10:03:52.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:52.902+0000 7f1d0079d640 1 -- 172.21.15.148:0/3765750644 wait complete. 2024-04-25T10:03:52.920 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:03:52.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:53.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.015+0000 7fc1642dd640 1 Processor -- start 2024-04-25T10:03:53.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.015+0000 7fc1642dd640 1 -- start start 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc1642dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc15c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc1642dd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c0cce50 con 0x7fc15c0d1810 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc1642dd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15c161af0 con 0x7fc15c0cc8e0 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc1642dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c161dd0 con 0x7fc15c15dc90 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc16287a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc15c0d1810 0x7fc15c0cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36042/0 (socket says 172.21.15.148:36042) 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc16287a640 1 -- 172.21.15.148:0/2720275253 learned_addr learned my addr 172.21.15.148:0/2720275253 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc162079640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc15c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956781493 0 0) 0x7fc15c0cce50 con 0x7fc15c0d1810 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc148003520 con 0x7fc15c0d1810 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2753388537 0 0) 0x7fc148003520 con 0x7fc15c0d1810 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 msgr2=0x7fc15c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc15c1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 >> v1:172.21.15.188:6789/0 conn(0x7fc15c0cc8e0 legacy=0x7fc15c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c1620b0 con 0x7fc15c0d1810 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fc158001670 con 0x7fc15c0d1810 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.016+0000 7fc162079640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc15c1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 >> v1:172.21.15.148:6789/0 conn(0x7fc15c0d1810 legacy=0x7fc15c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1480039e0 0x7fc148003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc148004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c161dd0 con 0x7fc148004d40 2024-04-25T10:03:53.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14800baa0 con 0x7fc1480039e0 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14800bca0 con 0x7fc15c15dc90 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc16287a640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc162079640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1480039e0 0x7fc148003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc161878640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc148004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc16287a640 1 -- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 msgr2=0x7fc148004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc16287a640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c15dc90 0x7fc148004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc16287a640 1 -- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1480039e0 msgr2=0x7fc148003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc16287a640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1480039e0 0x7fc148003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.017+0000 7fc16287a640 1 -- 172.21.15.148:0/2720275253 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148003520 con 0x7fc148004d40 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc16287a640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc15800c240 tx=0x7fc148005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2d273a4a220f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158007cf0 con 0x7fc148004d40 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1580058d0 con 0x7fc148004d40 2024-04-25T10:03:53.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc161077640 1 -- 172.21.15.148:0/2720275253 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158005ce0 con 0x7fc148004d40 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc1642dd640 1 -- 172.21.15.148:0/2720275253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc148004d40 msgr2=0x7fc1480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc1642dd640 1 --2- 172.21.15.148:0/2720275253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc15800c240 tx=0x7fc148005830 comp rx=0 tx=0).stop 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc16287a640 1 -- 172.21.15.148:0/2720275253 reap_dead start 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc1642dd640 1 -- 172.21.15.148:0/2720275253 shutdown_connections 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc1642dd640 1 -- 172.21.15.148:0/2720275253 >> 172.21.15.148:0/2720275253 conn(0x7fc15c06f590 msgr2=0x7fc15c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc1642dd640 1 -- 172.21.15.148:0/2720275253 shutdown_connections 2024-04-25T10:03:53.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.018+0000 7fc1642dd640 1 -- 172.21.15.148:0/2720275253 wait complete. 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 Processor -- start 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 -- start start 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc148004d40 0x7fc15c1e8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 0x7fc15c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c0cb270 0x7fc15c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c15e1f0 con 0x7fc15c0c8950 2024-04-25T10:03:53.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c15e070 con 0x7fc148004d40 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc1642dd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c1625d0 con 0x7fc15c0cb270 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc161878640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 0x7fc15c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc161878640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 0x7fc15c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46536/0 (socket says 172.21.15.148:46536) 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.020+0000 7fc161878640 1 -- 172.21.15.148:0/3891517462 learned_addr learned my addr 172.21.15.148:0/3891517462 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc162079640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc148004d40 0x7fc15c1e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc16287a640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c0cb270 0x7fc15c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc161878640 1 -- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c0cb270 msgr2=0x7fc15c0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc161878640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c0cb270 0x7fc15c0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc161878640 1 -- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc148004d40 msgr2=0x7fc15c1e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc161878640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc148004d40 0x7fc15c1e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc161878640 1 -- 172.21.15.148:0/3891517462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c1ea110 con 0x7fc15c0c8950 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc162079640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc148004d40 0x7fc15c1e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc16287a640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c0cb270 0x7fc15c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:53.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc161878640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 0x7fc15c0c8dd0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00f850 tx=0x7fc14c00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00fe40 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c018070 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c1e9070 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00d6d0 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.021+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c1e9630 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.022+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c073990 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.022+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc14c00d870 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.023+0000 7fc152ffd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc14006b4d0 0x7fc14006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.023+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc14c08ab10 con 0x7fc15c0c8950 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.023+0000 7fc162079640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc14006b4d0 0x7fc14006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.023+0000 7fc162079640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc14006b4d0 0x7fc14006d990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc154007440 tx=0x7fc15400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.027+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc14c058ed0 con 0x7fc15c0c8950 2024-04-25T10:03:53.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.230+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc15c0b3dc0 con 0x7fc15c0c8950 2024-04-25T10:03:53.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.231+0000 7fc152ffd640 1 -- 172.21.15.148:0/3891517462 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4954 (secure 0 0 0) 0x7fc14c0587d0 con 0x7fc15c0c8950 2024-04-25T10:03:53.236 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:53.236 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9855,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:03:52.226024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:03:53.236 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 84 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc14006b4d0 msgr2=0x7fc14006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc14006b4d0 0x7fc14006d990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc154007440 tx=0x7fc15400b040 comp rx=0 tx=0).stop 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 msgr2=0x7fc15c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 0x7fc15c0c8dd0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00f850 tx=0x7fc14c00fd10 comp rx=0 tx=0).stop 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 shutdown_connections 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc14006b4d0 0x7fc14006d990 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc15c0cb270 0x7fc15c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc15c0c8950 0x7fc15c0c8dd0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 --2- 172.21.15.148:0/3891517462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc148004d40 0x7fc15c1e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 >> 172.21.15.148:0/3891517462 conn(0x7fc15c06f590 msgr2=0x7fc15c0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 shutdown_connections 2024-04-25T10:03:53.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.233+0000 7fc1642dd640 1 -- 172.21.15.148:0/3891517462 wait complete. 2024-04-25T10:03:53.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:53.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.341+0000 7f6e64bbd640 1 Processor -- start 2024-04-25T10:03:53.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.341+0000 7f6e64bbd640 1 -- start start 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e64bbd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e600b3410 0x7f6e600720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e64bbd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e6006e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e64bbd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e600ace70 con 0x7f6e600b37f0 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e64bbd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6006df30 con 0x7f6e600b3410 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e64bbd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e600b43b0 con 0x7f6e6006e3c0 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5e59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6e600b3410 0x7f6e600720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e57fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6e600b37f0 0x7f6e600b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36052/0 (socket says 172.21.15.148:36052) 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e57fff640 1 -- 172.21.15.148:0/3663154684 learned_addr learned my addr 172.21.15.148:0/3663154684 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5ed9d640 1 --2- 172.21.15.148:0/3663154684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e6006e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5e59c640 1 -- 172.21.15.148:0/3663154684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 msgr2=0x7f6e6006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5e59c640 1 --2- 172.21.15.148:0/3663154684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e6006e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5e59c640 1 -- 172.21.15.148:0/3663154684 >> v1:172.21.15.148:6789/0 conn(0x7f6e600b37f0 legacy=0x7f6e600b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5e59c640 1 -- 172.21.15.148:0/3663154684 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600726d0 con 0x7f6e600b3410 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.342+0000 7f6e5dd9b640 1 -- 172.21.15.148:0/3663154684 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547768270 0 0) 0x7f6e48001110 con 0x7f6e600b37f0 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.343+0000 7f6e5e59c640 1 --2- 172.21.15.148:0/3663154684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b3410 0x7f6e600720f0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f6e500097a0 tx=0x7f6e5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252f441e6f85f5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.343+0000 7f6e5dd9b640 1 -- 172.21.15.148:0/3663154684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50037070 con 0x7f6e600b3410 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.343+0000 7f6e5dd9b640 1 -- 172.21.15.148:0/3663154684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50032440 con 0x7f6e600b3410 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.343+0000 7f6e5dd9b640 1 -- 172.21.15.148:0/3663154684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50032810 con 0x7f6e600b3410 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.343+0000 7f6e64bbd640 1 -- 172.21.15.148:0/3663154684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b3410 msgr2=0x7f6e600720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/3663154684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b3410 0x7f6e600720f0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f6e500097a0 tx=0x7f6e5002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 -- 172.21.15.148:0/3663154684 shutdown_connections 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/3663154684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e6006e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/3663154684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b3410 0x7f6e600720f0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 -- 172.21.15.148:0/3663154684 >> 172.21.15.148:0/3663154684 conn(0x7f6e600cfae0 msgr2=0x7f6e600cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 -- 172.21.15.148:0/3663154684 shutdown_connections 2024-04-25T10:03:53.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.344+0000 7f6e64bbd640 1 -- 172.21.15.148:0/3663154684 wait complete. 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.345+0000 7f6e64bbd640 1 Processor -- start 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.345+0000 7f6e64bbd640 1 -- start start 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e601532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e600b3410 0x7f6e60145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 0x7f6e6015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e60147890 con 0x7f6e600b37f0 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6006dbc0 con 0x7f6e600b3410 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e60147230 con 0x7f6e6006e3c0 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 0x7f6e6015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 0x7f6e6015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46552/0 (socket says 172.21.15.148:46552) 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 -- 172.21.15.148:0/1975653378 learned_addr learned my addr 172.21.15.148:0/1975653378 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5e59c640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e601532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 -- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 msgr2=0x7f6e601532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e601532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 -- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e600b3410 msgr2=0x7f6e60145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e600b3410 0x7f6e60145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 -- 172.21.15.148:0/1975653378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e601447e0 con 0x7f6e600b37f0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e5ed9d640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 0x7f6e6015e370 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009dd0 tx=0x7f6e5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58018020 con 0x7f6e600b37f0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e58014480 con 0x7f6e600b37f0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e60144ac0 con 0x7f6e600b37f0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58014870 con 0x7f6e600b37f0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.346+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e60145080 con 0x7f6e600b37f0 2024-04-25T10:03:53.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.347+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e600aca50 con 0x7f6e600b37f0 2024-04-25T10:03:53.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.348+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e58005d10 con 0x7f6e600b37f0 2024-04-25T10:03:53.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.348+0000 7f6e56ffd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.348+0000 7f6e5e59c640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.348+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e5808af40 con 0x7f6e600b37f0 2024-04-25T10:03:53.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.349+0000 7f6e5e59c640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e4006b4d0 0x7f6e4006d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6e50034000 tx=0x7f6e5002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.353+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e600aca50 con 0x7f6e600b37f0 2024-04-25T10:03:53.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.525+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e60073990 con 0x7f6e600b37f0 2024-04-25T10:03:53.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.527+0000 7f6e56ffd640 1 -- 172.21.15.148:0/1975653378 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f6e58022070 con 0x7f6e600b37f0 2024-04-25T10:03:53.532 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:53.532 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":145,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:03:51.879918+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e4006b4d0 msgr2=0x7f6e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e4006b4d0 0x7f6e4006d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6e50034000 tx=0x7f6e5002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 msgr2=0x7f6e6015e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 0x7f6e6015e370 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009dd0 tx=0x7f6e5800c490 comp rx=0 tx=0).stop 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 shutdown_connections 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e600b37f0 0x7f6e6015e370 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e600b3410 0x7f6e60145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 --2- 172.21.15.148:0/1975653378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e6006e3c0 0x7f6e601532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 >> 172.21.15.148:0/1975653378 conn(0x7f6e600cfae0 msgr2=0x7f6e600716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 shutdown_connections 2024-04-25T10:03:53.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.529+0000 7f6e64bbd640 1 -- 172.21.15.148:0/1975653378 wait complete. 2024-04-25T10:03:53.545 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:03:53.545 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.637+0000 7ff398ee1640 1 Processor -- start 2024-04-25T10:03:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.637+0000 7ff398ee1640 1 -- start start 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff398ee1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3940721d0 0x7ff39406f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff398ee1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff39406d9f0 0x7ff39415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff398ee1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39406c730 con 0x7ff39415bb60 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff398ee1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39406b690 con 0x7ff3940721d0 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff398ee1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39406b970 con 0x7ff39406d9f0 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3940721d0 0x7ff39406f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff3940721d0 0x7ff39406f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33640/0 (socket says 172.21.15.148:33640) 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 -- 172.21.15.148:0/883039442 learned_addr learned my addr 172.21.15.148:0/883039442 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff39259c640 1 --1- 172.21.15.148:0/883039442 >> v1:172.21.15.148:6789/0 conn(0x7ff39415bb60 0x7ff394071ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34928/0 (socket says 172.21.15.148:34928) 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff392d9d640 1 --2- 172.21.15.148:0/883039442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff39406d9f0 0x7ff39415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 -- 172.21.15.148:0/883039442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff39406d9f0 msgr2=0x7ff39415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 --2- 172.21.15.148:0/883039442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff39406d9f0 0x7ff39415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 -- 172.21.15.148:0/883039442 >> v1:172.21.15.148:6789/0 conn(0x7ff39415bb60 legacy=0x7ff394071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff391d9b640 1 -- 172.21.15.148:0/883039442 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39406bc50 con 0x7ff3940721d0 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.638+0000 7ff39159a640 1 -- 172.21.15.148:0/883039442 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574790897 0 0) 0x7ff39406c730 con 0x7ff39415bb60 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.639+0000 7ff391d9b640 1 --2- 172.21.15.148:0/883039442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3940721d0 0x7ff39406f550 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff388009870 tx=0x7ff38802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c68928c393aae18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.639+0000 7ff39159a640 1 -- 172.21.15.148:0/883039442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388037070 con 0x7ff3940721d0 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.639+0000 7ff39159a640 1 -- 172.21.15.148:0/883039442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388032440 con 0x7ff3940721d0 2024-04-25T10:03:53.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.639+0000 7ff39159a640 1 -- 172.21.15.148:0/883039442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388032810 con 0x7ff3940721d0 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 -- 172.21.15.148:0/883039442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3940721d0 msgr2=0x7ff39406f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 --2- 172.21.15.148:0/883039442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3940721d0 0x7ff39406f550 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff388009870 tx=0x7ff38802a270 comp rx=0 tx=0).stop 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 -- 172.21.15.148:0/883039442 shutdown_connections 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 --2- 172.21.15.148:0/883039442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff39406d9f0 0x7ff39415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 --2- 172.21.15.148:0/883039442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3940721d0 0x7ff39406f550 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 -- 172.21.15.148:0/883039442 >> 172.21.15.148:0/883039442 conn(0x7ff3940af670 msgr2=0x7ff3940b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 -- 172.21.15.148:0/883039442 shutdown_connections 2024-04-25T10:03:53.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.640+0000 7ff398ee1640 1 -- 172.21.15.148:0/883039442 wait complete. 2024-04-25T10:03:53.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.641+0000 7ff398ee1640 1 Processor -- start 2024-04-25T10:03:53.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.641+0000 7ff398ee1640 1 -- start start 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940721d0 0x7ff3941561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 0x7ff3941408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff39415bf40 0x7ff394153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940727f0 con 0x7ff39415bb60 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39406c210 con 0x7ff39415bf40 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff394072970 con 0x7ff3940721d0 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 0x7ff3941408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 0x7ff3941408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33642/0 (socket says 172.21.15.148:33642) 2024-04-25T10:03:53.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 -- 172.21.15.148:0/2706824055 learned_addr learned my addr 172.21.15.148:0/2706824055 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 -- 172.21.15.148:0/2706824055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940721d0 msgr2=0x7ff3941561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940721d0 0x7ff3941561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 -- 172.21.15.148:0/2706824055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff39415bf40 msgr2=0x7ff394153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff39415bf40 0x7ff394153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 -- 172.21.15.148:0/2706824055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39415c720 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff391d9b640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 0x7ff3941408b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff3880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388041020 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3880306c0 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.642+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39415ca00 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.643+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388037040 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.643+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39415d290 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.643+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff394073990 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.644+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff388044020 con 0x7ff39415bb60 2024-04-25T10:03:53.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.644+0000 7ff382ffd640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff37406b500 0x7ff37406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.644+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff3880aab60 con 0x7ff39415bb60 2024-04-25T10:03:53.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.644+0000 7ff39259c640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff37406b500 0x7ff37406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.645+0000 7ff39259c640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff37406b500 0x7ff37406d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff384007fd0 tx=0x7ff384007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.648+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff38804b030 con 0x7ff39415bb60 2024-04-25T10:03:53.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.817+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff39415cf20 con 0x7ff37406b500 2024-04-25T10:03:53.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.822+0000 7ff382ffd640 1 -- 172.21.15.148:0/2706824055 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7ff3840070d0 con 0x7ff37406b500 2024-04-25T10:03:53.830 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:53.832 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff37406b500 msgr2=0x7ff37406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff37406b500 0x7ff37406d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff384007fd0 tx=0x7ff384007960 comp rx=0 tx=0).stop 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 msgr2=0x7ff3941408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 0x7ff3941408b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff3880046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 shutdown_connections 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff37406b500 0x7ff37406d9c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff39415bf40 0x7ff394153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39415bb60 0x7ff3941408b0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 --2- 172.21.15.148:0/2706824055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3940721d0 0x7ff3941561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 >> 172.21.15.148:0/2706824055 conn(0x7ff39406d9f0 msgr2=0x7ff3941583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 shutdown_connections 2024-04-25T10:03:53.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.825+0000 7ff398ee1640 1 -- 172.21.15.148:0/2706824055 wait complete. 2024-04-25T10:03:53.842 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-04-25T10:03:53.137713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417248,"kb_used_data":15092,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754557472,"statfs":{"total":773094113280,"available":772666851328,"internally_reserved":0,"allocated":15454208,"data_stored":7820067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.264036"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889090+0000","last_change":"2024-04-25T10:03:49.889090+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889090+0000","last_undegraded":"2024-04-25T10:03:49.889090+0000","last_fullsized":"2024-04-25T10:03:49.889090+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884569+0000","last_change":"2024-04-25T10:03:49.884569+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884569+0000","last_undegraded":"2024-04-25T10:03:49.884569+0000","last_fullsized":"2024-04-25T10:03:49.884569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885623+0000","last_change":"2024-04-25T10:03:49.885623+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885623+0000","last_undegraded":"2024-04-25T10:03:49.885623+0000","last_fullsized":"2024-04-25T10:03:49.885623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890463+0000","last_change":"2024-04-25T10:03:49.890463+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890463+0000","last_undegraded":"2024-04-25T10:03:49.890463+0000","last_fullsized":"2024-04-25T10:03:49.890463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886184+0000","last_change":"2024-04-25T10:03:49.886184+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886184+0000","last_undegraded":"2024-04-25T10:03:49.886184+0000","last_fullsized":"2024-04-25T10:03:49.886184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886898+0000","last_change":"2024-04-25T10:03:49.886898+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886898+0000","last_undegraded":"2024-04-25T10:03:49.886898+0000","last_fullsized":"2024-04-25T10:03:49.886898+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889594+0000","last_change":"2024-04-25T10:03:49.889594+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889594+0000","last_undegraded":"2024-04-25T10:03:49.889594+0000","last_fullsized":"2024-04-25T10:03:49.889594+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890128+0000","last_change":"2024-04-25T10:03:49.890128+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890128+0000","last_undegraded":"2024-04-25T10:03:49.890128+0000","last_fullsized":"2024-04-25T10:03:49.890128+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886447+0000","last_change":"2024-04-25T10:03:49.886447+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886447+0000","last_undegraded":"2024-04-25T10:03:49.886447+0000","last_fullsized":"2024-04-25T10:03:49.886447+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887235+0000","last_change":"2024-04-25T10:03:49.887235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887235+0000","last_undegraded":"2024-04-25T10:03:49.887235+0000","last_fullsized":"2024-04-25T10:03:49.887235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890445+0000","last_change":"2024-04-25T10:03:49.890445+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890445+0000","last_undegraded":"2024-04-25T10:03:49.890445+0000","last_fullsized":"2024-04-25T10:03:49.890445+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889960+0000","last_change":"2024-04-25T10:03:49.889960+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889960+0000","last_undegraded":"2024-04-25T10:03:49.889960+0000","last_fullsized":"2024-04-25T10:03:49.889960+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889706+0000","last_change":"2024-04-25T10:03:49.889706+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889706+0000","last_undegraded":"2024-04-25T10:03:49.889706+0000","last_fullsized":"2024-04-25T10:03:49.889706+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891818+0000","last_change":"2024-04-25T10:03:49.891818+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891818+0000","last_undegraded":"2024-04-25T10:03:49.891818+0000","last_fullsized":"2024-04-25T10:03:49.891818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887036+0000","last_change":"2024-04-25T10:03:49.887036+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887036+0000","last_undegraded":"2024-04-25T10:03:49.887036+0000","last_fullsized":"2024-04-25T10:03:49.887036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889684+0000","last_change":"2024-04-25T10:03:49.889684+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889684+0000","last_undegraded":"2024-04-25T10:03:49.889684+0000","last_fullsized":"2024-04-25T10:03:49.889684+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891988+0000","last_change":"2024-04-25T10:03:49.891988+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891988+0000","last_undegraded":"2024-04-25T10:03:49.891988+0000","last_fullsized":"2024-04-25T10:03:49.891988+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891421+0000","last_change":"2024-04-25T10:03:49.891421+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891421+0000","last_undegraded":"2024-04-25T10:03:49.891421+0000","last_fullsized":"2024-04-25T10:03:49.891421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888004+0000","last_change":"2024-04-25T10:03:49.888004+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888004+0000","last_undegraded":"2024-04-25T10:03:49.888004+0000","last_fullsized":"2024-04-25T10:03:49.888004+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884946+0000","last_change":"2024-04-25T10:03:49.884946+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884946+0000","last_undegraded":"2024-04-25T10:03:49.884946+0000","last_fullsized":"2024-04-25T10:03:49.884946+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886836+0000","last_change":"2024-04-25T10:03:49.886836+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886836+0000","last_undegraded":"2024-04-25T10:03:49.886836+0000","last_fullsized":"2024-04-25T10:03:49.886836+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888666+0000","last_change":"2024-04-25T10:03:49.888666+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888666+0000","last_undegraded":"2024-04-25T10:03:49.888666+0000","last_fullsized":"2024-04-25T10:03:49.888666+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887306+0000","last_change":"2024-04-25T10:03:49.887306+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887306+0000","last_undegraded":"2024-04-25T10:03:49.887306+0000","last_fullsized":"2024-04-25T10:03:49.887306+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889841+0000","last_change":"2024-04-25T10:03:49.889841+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889841+0000","last_undegraded":"2024-04-25T10:03:49.889841+0000","last_fullsized":"2024-04-25T10:03:49.889841+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888457+0000","last_change":"2024-04-25T10:03:49.888457+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888457+0000","last_undegraded":"2024-04-25T10:03:49.888457+0000","last_fullsized":"2024-04-25T10:03:49.888457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887310+0000","last_change":"2024-04-25T10:03:49.887310+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887310+0000","last_undegraded":"2024-04-25T10:03:49.887310+0000","last_fullsized":"2024-04-25T10:03:49.887310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885978+0000","last_change":"2024-04-25T10:03:49.885978+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885978+0000","last_undegraded":"2024-04-25T10:03:49.885978+0000","last_fullsized":"2024-04-25T10:03:49.885978+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891147+0000","last_change":"2024-04-25T10:03:49.891147+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891147+0000","last_undegraded":"2024-04-25T10:03:49.891147+0000","last_fullsized":"2024-04-25T10:03:49.891147+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884069+0000","last_change":"2024-04-25T10:03:49.884069+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884069+0000","last_undegraded":"2024-04-25T10:03:49.884069+0000","last_fullsized":"2024-04-25T10:03:49.884069+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.892310+0000","last_change":"2024-04-25T10:03:49.892310+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.892310+0000","last_undegraded":"2024-04-25T10:03:49.892310+0000","last_fullsized":"2024-04-25T10:03:49.892310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885408+0000","last_change":"2024-04-25T10:03:49.885408+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885408+0000","last_undegraded":"2024-04-25T10:03:49.885408+0000","last_fullsized":"2024-04-25T10:03:49.885408+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889491+0000","last_change":"2024-04-25T10:03:49.889491+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889491+0000","last_undegraded":"2024-04-25T10:03:49.889491+0000","last_fullsized":"2024-04-25T10:03:49.889491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886031+0000","last_change":"2024-04-25T10:03:49.886031+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886031+0000","last_undegraded":"2024-04-25T10:03:49.886031+0000","last_fullsized":"2024-04-25T10:03:49.886031+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893210+0000","last_change":"2024-04-25T10:03:49.891438+0000","last_active":"2024-04-25T10:03:49.893210+0000","last_peered":"2024-04-25T10:03:49.893210+0000","last_clean":"2024-04-25T10:03:49.893210+0000","last_became_active":"2024-04-25T10:03:49.890515+0000","last_became_peered":"2024-04-25T10:03:49.890515+0000","last_unstale":"2024-04-25T10:03:49.893210+0000","last_undegraded":"2024-04-25T10:03:49.893210+0000","last_fullsized":"2024-04-25T10:03:49.893210+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885234+0000","last_change":"2024-04-25T10:03:49.885234+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885234+0000","last_undegraded":"2024-04-25T10:03:49.885234+0000","last_fullsized":"2024-04-25T10:03:49.885234+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900272+0000","last_change":"2024-04-25T10:03:49.898107+0000","last_active":"2024-04-25T10:03:49.900272+0000","last_peered":"2024-04-25T10:03:49.900272+0000","last_clean":"2024-04-25T10:03:49.900272+0000","last_became_active":"2024-04-25T10:03:49.897243+0000","last_became_peered":"2024-04-25T10:03:49.897243+0000","last_unstale":"2024-04-25T10:03:49.900272+0000","last_undegraded":"2024-04-25T10:03:49.900272+0000","last_fullsized":"2024-04-25T10:03:49.900272+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885440+0000","last_change":"2024-04-25T10:03:49.885440+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885440+0000","last_undegraded":"2024-04-25T10:03:49.885440+0000","last_fullsized":"2024-04-25T10:03:49.885440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898592+0000","last_change":"2024-04-25T10:03:49.896386+0000","last_active":"2024-04-25T10:03:49.898592+0000","last_peered":"2024-04-25T10:03:49.898592+0000","last_clean":"2024-04-25T10:03:49.898592+0000","last_became_active":"2024-04-25T10:03:49.895902+0000","last_became_peered":"2024-04-25T10:03:49.895902+0000","last_unstale":"2024-04-25T10:03:49.898592+0000","last_undegraded":"2024-04-25T10:03:49.898592+0000","last_fullsized":"2024-04-25T10:03:49.898592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.352713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886573+0000","last_change":"2024-04-25T10:03:49.886573+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886573+0000","last_undegraded":"2024-04-25T10:03:49.886573+0000","last_fullsized":"2024-04-25T10:03:49.886573+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897594+0000","last_change":"2024-04-25T10:03:49.896831+0000","last_active":"2024-04-25T10:03:49.897594+0000","last_peered":"2024-04-25T10:03:49.897594+0000","last_clean":"2024-04-25T10:03:49.897594+0000","last_became_active":"2024-04-25T10:03:49.896566+0000","last_became_peered":"2024-04-25T10:03:49.896566+0000","last_unstale":"2024-04-25T10:03:49.897594+0000","last_undegraded":"2024-04-25T10:03:49.897594+0000","last_fullsized":"2024-04-25T10:03:49.897594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884360+0000","last_change":"2024-04-25T10:03:49.884360+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884360+0000","last_undegraded":"2024-04-25T10:03:49.884360+0000","last_fullsized":"2024-04-25T10:03:49.884360+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896832+0000","last_change":"2024-04-25T10:03:49.894730+0000","last_active":"2024-04-25T10:03:49.896832+0000","last_peered":"2024-04-25T10:03:49.896832+0000","last_clean":"2024-04-25T10:03:49.896832+0000","last_became_active":"2024-04-25T10:03:49.894297+0000","last_became_peered":"2024-04-25T10:03:49.894297+0000","last_unstale":"2024-04-25T10:03:49.896832+0000","last_undegraded":"2024-04-25T10:03:49.896832+0000","last_fullsized":"2024-04-25T10:03:49.896832+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:31.343353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.892113+0000","last_change":"2024-04-25T10:03:49.892113+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.892113+0000","last_undegraded":"2024-04-25T10:03:49.892113+0000","last_fullsized":"2024-04-25T10:03:49.892113+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902592+0000","last_change":"2024-04-25T10:03:49.902295+0000","last_active":"2024-04-25T10:03:49.902592+0000","last_peered":"2024-04-25T10:03:49.902592+0000","last_clean":"2024-04-25T10:03:49.902592+0000","last_became_active":"2024-04-25T10:03:49.902003+0000","last_became_peered":"2024-04-25T10:03:49.902003+0000","last_unstale":"2024-04-25T10:03:49.902592+0000","last_undegraded":"2024-04-25T10:03:49.902592+0000","last_fullsized":"2024-04-25T10:03:49.902592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:10.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885895+0000","last_change":"2024-04-25T10:03:49.885895+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885895+0000","last_undegraded":"2024-04-25T10:03:49.885895+0000","last_fullsized":"2024-04-25T10:03:49.885895+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901072+0000","last_change":"2024-04-25T10:03:49.899529+0000","last_active":"2024-04-25T10:03:49.901072+0000","last_peered":"2024-04-25T10:03:49.901072+0000","last_clean":"2024-04-25T10:03:49.901072+0000","last_became_active":"2024-04-25T10:03:49.896941+0000","last_became_peered":"2024-04-25T10:03:49.896941+0000","last_unstale":"2024-04-25T10:03:49.901072+0000","last_undegraded":"2024-04-25T10:03:49.901072+0000","last_fullsized":"2024-04-25T10:03:49.901072+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.283728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885817+0000","last_change":"2024-04-25T10:03:49.885817+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885817+0000","last_undegraded":"2024-04-25T10:03:49.885817+0000","last_fullsized":"2024-04-25T10:03:49.885817+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.894253+0000","last_change":"2024-04-25T10:03:49.893772+0000","last_active":"2024-04-25T10:03:49.894253+0000","last_peered":"2024-04-25T10:03:49.894253+0000","last_clean":"2024-04-25T10:03:49.894253+0000","last_became_active":"2024-04-25T10:03:49.893508+0000","last_became_peered":"2024-04-25T10:03:49.893508+0000","last_unstale":"2024-04-25T10:03:49.894253+0000","last_undegraded":"2024-04-25T10:03:49.894253+0000","last_fullsized":"2024-04-25T10:03:49.894253+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884845+0000","last_change":"2024-04-25T10:03:49.884845+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884845+0000","last_undegraded":"2024-04-25T10:03:49.884845+0000","last_fullsized":"2024-04-25T10:03:49.884845+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898121+0000","last_change":"2024-04-25T10:03:49.896956+0000","last_active":"2024-04-25T10:03:49.898121+0000","last_peered":"2024-04-25T10:03:49.898121+0000","last_clean":"2024-04-25T10:03:49.898121+0000","last_became_active":"2024-04-25T10:03:49.896167+0000","last_became_peered":"2024-04-25T10:03:49.896167+0000","last_unstale":"2024-04-25T10:03:49.898121+0000","last_undegraded":"2024-04-25T10:03:49.898121+0000","last_fullsized":"2024-04-25T10:03:49.898121+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890014+0000","last_change":"2024-04-25T10:03:49.890014+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890014+0000","last_undegraded":"2024-04-25T10:03:49.890014+0000","last_fullsized":"2024-04-25T10:03:49.890014+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901024+0000","last_change":"2024-04-25T10:03:49.898465+0000","last_active":"2024-04-25T10:03:49.901024+0000","last_peered":"2024-04-25T10:03:49.901024+0000","last_clean":"2024-04-25T10:03:49.901024+0000","last_became_active":"2024-04-25T10:03:49.898198+0000","last_became_peered":"2024-04-25T10:03:49.898198+0000","last_unstale":"2024-04-25T10:03:49.901024+0000","last_undegraded":"2024-04-25T10:03:49.901024+0000","last_fullsized":"2024-04-25T10:03:49.901024+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891028+0000","last_change":"2024-04-25T10:03:49.891028+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891028+0000","last_undegraded":"2024-04-25T10:03:49.891028+0000","last_fullsized":"2024-04-25T10:03:49.891028+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898187+0000","last_change":"2024-04-25T10:03:49.897101+0000","last_active":"2024-04-25T10:03:49.898187+0000","last_peered":"2024-04-25T10:03:49.898187+0000","last_clean":"2024-04-25T10:03:49.898187+0000","last_became_active":"2024-04-25T10:03:49.896790+0000","last_became_peered":"2024-04-25T10:03:49.896790+0000","last_unstale":"2024-04-25T10:03:49.898187+0000","last_undegraded":"2024-04-25T10:03:49.898187+0000","last_fullsized":"2024-04-25T10:03:49.898187+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:00.424778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888660+0000","last_change":"2024-04-25T10:03:49.888660+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888660+0000","last_undegraded":"2024-04-25T10:03:49.888660+0000","last_fullsized":"2024-04-25T10:03:49.888660+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893179+0000","last_change":"2024-04-25T10:03:49.892221+0000","last_active":"2024-04-25T10:03:49.893179+0000","last_peered":"2024-04-25T10:03:49.893179+0000","last_clean":"2024-04-25T10:03:49.893179+0000","last_became_active":"2024-04-25T10:03:49.891975+0000","last_became_peered":"2024-04-25T10:03:49.891975+0000","last_unstale":"2024-04-25T10:03:49.893179+0000","last_undegraded":"2024-04-25T10:03:49.893179+0000","last_fullsized":"2024-04-25T10:03:49.893179+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886658+0000","last_change":"2024-04-25T10:03:49.886658+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886658+0000","last_undegraded":"2024-04-25T10:03:49.886658+0000","last_fullsized":"2024-04-25T10:03:49.886658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902045+0000","last_change":"2024-04-25T10:03:49.901671+0000","last_active":"2024-04-25T10:03:49.902045+0000","last_peered":"2024-04-25T10:03:49.902045+0000","last_clean":"2024-04-25T10:03:49.902045+0000","last_became_active":"2024-04-25T10:03:49.901399+0000","last_became_peered":"2024-04-25T10:03:49.901399+0000","last_unstale":"2024-04-25T10:03:49.902045+0000","last_undegraded":"2024-04-25T10:03:49.902045+0000","last_fullsized":"2024-04-25T10:03:49.902045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:57.162299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886235+0000","last_change":"2024-04-25T10:03:49.886235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886235+0000","last_undegraded":"2024-04-25T10:03:49.886235+0000","last_fullsized":"2024-04-25T10:03:49.886235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901156+0000","last_change":"2024-04-25T10:03:49.900536+0000","last_active":"2024-04-25T10:03:49.901156+0000","last_peered":"2024-04-25T10:03:49.901156+0000","last_clean":"2024-04-25T10:03:49.901156+0000","last_became_active":"2024-04-25T10:03:49.900264+0000","last_became_peered":"2024-04-25T10:03:49.900264+0000","last_unstale":"2024-04-25T10:03:49.901156+0000","last_undegraded":"2024-04-25T10:03:49.901156+0000","last_fullsized":"2024-04-25T10:03:49.901156+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:08.595482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887864+0000","last_change":"2024-04-25T10:03:49.887864+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887864+0000","last_undegraded":"2024-04-25T10:03:49.887864+0000","last_fullsized":"2024-04-25T10:03:49.887864+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900875+0000","last_change":"2024-04-25T10:03:49.897285+0000","last_active":"2024-04-25T10:03:49.900875+0000","last_peered":"2024-04-25T10:03:49.900875+0000","last_clean":"2024-04-25T10:03:49.900875+0000","last_became_active":"2024-04-25T10:03:49.896833+0000","last_became_peered":"2024-04-25T10:03:49.896833+0000","last_unstale":"2024-04-25T10:03:49.900875+0000","last_undegraded":"2024-04-25T10:03:49.900875+0000","last_fullsized":"2024-04-25T10:03:49.900875+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885591+0000","last_change":"2024-04-25T10:03:49.885591+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885591+0000","last_undegraded":"2024-04-25T10:03:49.885591+0000","last_fullsized":"2024-04-25T10:03:49.885591+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.899696+0000","last_change":"2024-04-25T10:03:49.899336+0000","last_active":"2024-04-25T10:03:49.899696+0000","last_peered":"2024-04-25T10:03:49.899696+0000","last_clean":"2024-04-25T10:03:49.899696+0000","last_became_active":"2024-04-25T10:03:49.899096+0000","last_became_peered":"2024-04-25T10:03:49.899096+0000","last_unstale":"2024-04-25T10:03:49.899696+0000","last_undegraded":"2024-04-25T10:03:49.899696+0000","last_fullsized":"2024-04-25T10:03:49.899696+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:40.405411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886122+0000","last_change":"2024-04-25T10:03:49.886122+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886122+0000","last_undegraded":"2024-04-25T10:03:49.886122+0000","last_fullsized":"2024-04-25T10:03:49.886122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900925+0000","last_change":"2024-04-25T10:03:49.898990+0000","last_active":"2024-04-25T10:03:49.900925+0000","last_peered":"2024-04-25T10:03:49.900925+0000","last_clean":"2024-04-25T10:03:49.900925+0000","last_became_active":"2024-04-25T10:03:49.898720+0000","last_became_peered":"2024-04-25T10:03:49.898720+0000","last_unstale":"2024-04-25T10:03:49.900925+0000","last_undegraded":"2024-04-25T10:03:49.900925+0000","last_fullsized":"2024-04-25T10:03:49.900925+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886777+0000","last_change":"2024-04-25T10:03:49.886777+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886777+0000","last_undegraded":"2024-04-25T10:03:49.886777+0000","last_fullsized":"2024-04-25T10:03:49.886777+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901586+0000","last_change":"2024-04-25T10:03:49.901276+0000","last_active":"2024-04-25T10:03:49.901586+0000","last_peered":"2024-04-25T10:03:49.901586+0000","last_clean":"2024-04-25T10:03:49.901586+0000","last_became_active":"2024-04-25T10:03:49.900977+0000","last_became_peered":"2024-04-25T10:03:49.900977+0000","last_unstale":"2024-04-25T10:03:49.901586+0000","last_undegraded":"2024-04-25T10:03:49.901586+0000","last_fullsized":"2024-04-25T10:03:49.901586+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887729+0000","last_change":"2024-04-25T10:03:49.887729+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887729+0000","last_undegraded":"2024-04-25T10:03:49.887729+0000","last_fullsized":"2024-04-25T10:03:49.887729+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.899920+0000","last_change":"2024-04-25T10:03:49.898957+0000","last_active":"2024-04-25T10:03:49.899920+0000","last_peered":"2024-04-25T10:03:49.899920+0000","last_clean":"2024-04-25T10:03:49.899920+0000","last_became_active":"2024-04-25T10:03:49.898681+0000","last_became_peered":"2024-04-25T10:03:49.898681+0000","last_unstale":"2024-04-25T10:03:49.899920+0000","last_undegraded":"2024-04-25T10:03:49.899920+0000","last_fullsized":"2024-04-25T10:03:49.899920+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.487258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886494+0000","last_change":"2024-04-25T10:03:49.886494+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886494+0000","last_undegraded":"2024-04-25T10:03:49.886494+0000","last_fullsized":"2024-04-25T10:03:49.886494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896899+0000","last_change":"2024-04-25T10:03:49.895700+0000","last_active":"2024-04-25T10:03:49.896899+0000","last_peered":"2024-04-25T10:03:49.896899+0000","last_clean":"2024-04-25T10:03:49.896899+0000","last_became_active":"2024-04-25T10:03:49.895008+0000","last_became_peered":"2024-04-25T10:03:49.895008+0000","last_unstale":"2024-04-25T10:03:49.896899+0000","last_undegraded":"2024-04-25T10:03:49.896899+0000","last_fullsized":"2024-04-25T10:03:49.896899+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:03.925332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886272+0000","last_change":"2024-04-25T10:03:49.886272+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886272+0000","last_undegraded":"2024-04-25T10:03:49.886272+0000","last_fullsized":"2024-04-25T10:03:49.886272+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898332+0000","last_change":"2024-04-25T10:03:49.895688+0000","last_active":"2024-04-25T10:03:49.898332+0000","last_peered":"2024-04-25T10:03:49.898332+0000","last_clean":"2024-04-25T10:03:49.898332+0000","last_became_active":"2024-04-25T10:03:49.895423+0000","last_became_peered":"2024-04-25T10:03:49.895423+0000","last_unstale":"2024-04-25T10:03:49.898332+0000","last_undegraded":"2024-04-25T10:03:49.898332+0000","last_fullsized":"2024-04-25T10:03:49.898332+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.994391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888604+0000","last_change":"2024-04-25T10:03:49.888604+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888604+0000","last_undegraded":"2024-04-25T10:03:49.888604+0000","last_fullsized":"2024-04-25T10:03:49.888604+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896673+0000","last_change":"2024-04-25T10:03:49.895117+0000","last_active":"2024-04-25T10:03:49.896673+0000","last_peered":"2024-04-25T10:03:49.896673+0000","last_clean":"2024-04-25T10:03:49.896673+0000","last_became_active":"2024-04-25T10:03:49.894870+0000","last_became_peered":"2024-04-25T10:03:49.894870+0000","last_unstale":"2024-04-25T10:03:49.896673+0000","last_undegraded":"2024-04-25T10:03:49.896673+0000","last_fullsized":"2024-04-25T10:03:49.896673+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:30.835851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887449+0000","last_change":"2024-04-25T10:03:49.887449+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887449+0000","last_undegraded":"2024-04-25T10:03:49.887449+0000","last_fullsized":"2024-04-25T10:03:49.887449+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896859+0000","last_change":"2024-04-25T10:03:49.895442+0000","last_active":"2024-04-25T10:03:49.896859+0000","last_peered":"2024-04-25T10:03:49.896859+0000","last_clean":"2024-04-25T10:03:49.896859+0000","last_became_active":"2024-04-25T10:03:49.895113+0000","last_became_peered":"2024-04-25T10:03:49.895113+0000","last_unstale":"2024-04-25T10:03:49.896859+0000","last_undegraded":"2024-04-25T10:03:49.896859+0000","last_fullsized":"2024-04-25T10:03:49.896859+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:30.467108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887060+0000","last_change":"2024-04-25T10:03:49.887060+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887060+0000","last_undegraded":"2024-04-25T10:03:49.887060+0000","last_fullsized":"2024-04-25T10:03:49.887060+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900209+0000","last_change":"2024-04-25T10:03:49.898517+0000","last_active":"2024-04-25T10:03:49.900209+0000","last_peered":"2024-04-25T10:03:49.900209+0000","last_clean":"2024-04-25T10:03:49.900209+0000","last_became_active":"2024-04-25T10:03:49.898253+0000","last_became_peered":"2024-04-25T10:03:49.898253+0000","last_unstale":"2024-04-25T10:03:49.900209+0000","last_undegraded":"2024-04-25T10:03:49.900209+0000","last_fullsized":"2024-04-25T10:03:49.900209+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:01.257780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.893030+0000","last_change":"2024-04-25T10:03:49.893030+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.893030+0000","last_undegraded":"2024-04-25T10:03:49.893030+0000","last_fullsized":"2024-04-25T10:03:49.893030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901045+0000","last_change":"2024-04-25T10:03:49.899974+0000","last_active":"2024-04-25T10:03:49.901045+0000","last_peered":"2024-04-25T10:03:49.901045+0000","last_clean":"2024-04-25T10:03:49.901045+0000","last_became_active":"2024-04-25T10:03:49.899694+0000","last_became_peered":"2024-04-25T10:03:49.899694+0000","last_unstale":"2024-04-25T10:03:49.901045+0000","last_undegraded":"2024-04-25T10:03:49.901045+0000","last_fullsized":"2024-04-25T10:03:49.901045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.975577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886295+0000","last_change":"2024-04-25T10:03:49.886295+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886295+0000","last_undegraded":"2024-04-25T10:03:49.886295+0000","last_fullsized":"2024-04-25T10:03:49.886295+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896582+0000","last_change":"2024-04-25T10:03:49.894817+0000","last_active":"2024-04-25T10:03:49.896582+0000","last_peered":"2024-04-25T10:03:49.896582+0000","last_clean":"2024-04-25T10:03:49.896582+0000","last_became_active":"2024-04-25T10:03:49.894481+0000","last_became_peered":"2024-04-25T10:03:49.894481+0000","last_unstale":"2024-04-25T10:03:49.896582+0000","last_undegraded":"2024-04-25T10:03:49.896582+0000","last_fullsized":"2024-04-25T10:03:49.896582+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:06.966070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885867+0000","last_change":"2024-04-25T10:03:49.885867+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885867+0000","last_undegraded":"2024-04-25T10:03:49.885867+0000","last_fullsized":"2024-04-25T10:03:49.885867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897801+0000","last_change":"2024-04-25T10:03:49.897166+0000","last_active":"2024-04-25T10:03:49.897801+0000","last_peered":"2024-04-25T10:03:49.897801+0000","last_clean":"2024-04-25T10:03:49.897801+0000","last_became_active":"2024-04-25T10:03:49.896766+0000","last_became_peered":"2024-04-25T10:03:49.896766+0000","last_unstale":"2024-04-25T10:03:49.897801+0000","last_undegraded":"2024-04-25T10:03:49.897801+0000","last_fullsized":"2024-04-25T10:03:49.897801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:45.056797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885865+0000","last_change":"2024-04-25T10:03:49.885865+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885865+0000","last_undegraded":"2024-04-25T10:03:49.885865+0000","last_fullsized":"2024-04-25T10:03:49.885865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893982+0000","last_change":"2024-04-25T10:03:49.893523+0000","last_active":"2024-04-25T10:03:49.893982+0000","last_peered":"2024-04-25T10:03:49.893982+0000","last_clean":"2024-04-25T10:03:49.893982+0000","last_became_active":"2024-04-25T10:03:49.893233+0000","last_became_peered":"2024-04-25T10:03:49.893233+0000","last_unstale":"2024-04-25T10:03:49.893982+0000","last_undegraded":"2024-04-25T10:03:49.893982+0000","last_fullsized":"2024-04-25T10:03:49.893982+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891494+0000","last_change":"2024-04-25T10:03:49.891494+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891494+0000","last_undegraded":"2024-04-25T10:03:49.891494+0000","last_fullsized":"2024-04-25T10:03:49.891494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898782+0000","last_change":"2024-04-25T10:03:49.898200+0000","last_active":"2024-04-25T10:03:49.898782+0000","last_peered":"2024-04-25T10:03:49.898782+0000","last_clean":"2024-04-25T10:03:49.898782+0000","last_became_active":"2024-04-25T10:03:49.897929+0000","last_became_peered":"2024-04-25T10:03:49.897929+0000","last_unstale":"2024-04-25T10:03:49.898782+0000","last_undegraded":"2024-04-25T10:03:49.898782+0000","last_fullsized":"2024-04-25T10:03:49.898782+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:56.315225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885421+0000","last_change":"2024-04-25T10:03:49.885421+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885421+0000","last_undegraded":"2024-04-25T10:03:49.885421+0000","last_fullsized":"2024-04-25T10:03:49.885421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901223+0000","last_change":"2024-04-25T10:03:49.900360+0000","last_active":"2024-04-25T10:03:49.901223+0000","last_peered":"2024-04-25T10:03:49.901223+0000","last_clean":"2024-04-25T10:03:49.901223+0000","last_became_active":"2024-04-25T10:03:49.898980+0000","last_became_peered":"2024-04-25T10:03:49.898980+0000","last_unstale":"2024-04-25T10:03:49.901223+0000","last_undegraded":"2024-04-25T10:03:49.901223+0000","last_fullsized":"2024-04-25T10:03:49.901223+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:41.805235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885077+0000","last_change":"2024-04-25T10:03:49.885077+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885077+0000","last_undegraded":"2024-04-25T10:03:49.885077+0000","last_fullsized":"2024-04-25T10:03:49.885077+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898078+0000","last_change":"2024-04-25T10:03:49.896765+0000","last_active":"2024-04-25T10:03:49.898078+0000","last_peered":"2024-04-25T10:03:49.898078+0000","last_clean":"2024-04-25T10:03:49.898078+0000","last_became_active":"2024-04-25T10:03:49.896465+0000","last_became_peered":"2024-04-25T10:03:49.896465+0000","last_unstale":"2024-04-25T10:03:49.898078+0000","last_undegraded":"2024-04-25T10:03:49.898078+0000","last_fullsized":"2024-04-25T10:03:49.898078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:40.225994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.883729+0000","last_change":"2024-04-25T10:03:49.883729+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.883729+0000","last_undegraded":"2024-04-25T10:03:49.883729+0000","last_fullsized":"2024-04-25T10:03:49.883729+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896875+0000","last_change":"2024-04-25T10:03:49.895770+0000","last_active":"2024-04-25T10:03:49.896875+0000","last_peered":"2024-04-25T10:03:49.896875+0000","last_clean":"2024-04-25T10:03:49.896875+0000","last_became_active":"2024-04-25T10:03:49.895492+0000","last_became_peered":"2024-04-25T10:03:49.895492+0000","last_unstale":"2024-04-25T10:03:49.896875+0000","last_undegraded":"2024-04-25T10:03:49.896875+0000","last_fullsized":"2024-04-25T10:03:49.896875+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:18.825257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":396,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.886178+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:03:49.886178+0000","last_peered":"2024-04-25T10:03:49.886178+0000","last_clean":"2024-04-25T10:03:49.886178+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:03:49.886178+0000","last_undegraded":"2024-04-25T10:03:49.886178+0000","last_fullsized":"2024-04-25T10:03:49.886178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2375680,"data_stored":1419451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46812,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325028,"statfs":{"total":96636764160,"available":96588828672,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59304,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312536,"statfs":{"total":96636764160,"available":96576036864,"internally_reserved":0,"allocated":1794048,"data_stored":835750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2375680,"data_stored":1419451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:53.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:53.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.975+0000 7efe80f63640 1 Processor -- start 2024-04-25T10:03:53.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.975+0000 7efe80f63640 1 -- start start 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe80f63640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe7c15dd80 0x7efe7c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe80f63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c06f550 0x7efe7c071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe80f63640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe7c072000 0x7efe7c0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe80f63640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c06f9b0 con 0x7efe7c072000 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe80f63640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7c072d70 con 0x7efe7c15dd80 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe80f63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c06db20 con 0x7efe7c06f550 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe7c072000 0x7efe7c0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efe7c072000 0x7efe7c0aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33646/0 (socket says 172.21.15.148:33646) 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 -- 172.21.15.148:0/3996949526 learned_addr learned my addr 172.21.15.148:0/3996949526 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe79d9b640 1 --2- 172.21.15.148:0/3996949526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c06f550 0x7efe7c071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7a59c640 1 --2- 172.21.15.148:0/3996949526 >> v2:172.21.15.188:3300/0 conn(0x7efe7c15dd80 0x7efe7c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 -- 172.21.15.148:0/3996949526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c06f550 msgr2=0x7efe7c071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 --2- 172.21.15.148:0/3996949526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c06f550 0x7efe7c071ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 -- 172.21.15.148:0/3996949526 >> v2:172.21.15.188:3300/0 conn(0x7efe7c15dd80 msgr2=0x7efe7c06c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 --2- 172.21.15.148:0/3996949526 >> v2:172.21.15.188:3300/0 conn(0x7efe7c15dd80 0x7efe7c06c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7ad9d640 1 -- 172.21.15.148:0/3996949526 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c06de00 con 0x7efe7c072000 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.976+0000 7efe7a59c640 1 --2- 172.21.15.148:0/3996949526 >> v2:172.21.15.188:3300/0 conn(0x7efe7c15dd80 0x7efe7c06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.977+0000 7efe7ad9d640 1 --2- 172.21.15.148:0/3996949526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c072000 0x7efe7c0aced0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7efe7c09de50 tx=0x7efe6c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a5009358fe0118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.977+0000 7efe7959a640 1 -- 172.21.15.148:0/3996949526 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037070 con 0x7efe7c072000 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.977+0000 7efe7959a640 1 -- 172.21.15.148:0/3996949526 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c02ae00 con 0x7efe7c072000 2024-04-25T10:03:53.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.977+0000 7efe7959a640 1 -- 172.21.15.148:0/3996949526 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c032660 con 0x7efe7c072000 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.977+0000 7efe80f63640 1 -- 172.21.15.148:0/3996949526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c072000 msgr2=0x7efe7c0aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.977+0000 7efe80f63640 1 --2- 172.21.15.148:0/3996949526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c072000 0x7efe7c0aced0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7efe7c09de50 tx=0x7efe6c02a0b0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 -- 172.21.15.148:0/3996949526 shutdown_connections 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 --2- 172.21.15.148:0/3996949526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c072000 0x7efe7c0aced0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 --2- 172.21.15.148:0/3996949526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c06f550 0x7efe7c071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 --2- 172.21.15.148:0/3996949526 >> v2:172.21.15.188:3300/0 conn(0x7efe7c15dd80 0x7efe7c06c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 -- 172.21.15.148:0/3996949526 >> 172.21.15.148:0/3996949526 conn(0x7efe7c0af670 msgr2=0x7efe7c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 -- 172.21.15.148:0/3996949526 shutdown_connections 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.978+0000 7efe80f63640 1 -- 172.21.15.148:0/3996949526 wait complete. 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.979+0000 7efe80f63640 1 Processor -- start 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.979+0000 7efe80f63640 1 -- start start 2024-04-25T10:03:53.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe7c072000 0x7efe7c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c15dd80 0x7efe7c1e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 0x7efe7c1e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0ad410 con 0x7efe7c1e5850 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c06ba70 con 0x7efe7c072000 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c0ad590 con 0x7efe7c15dd80 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 0x7efe7c1e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 0x7efe7c1e5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33652/0 (socket says 172.21.15.148:33652) 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 -- 172.21.15.148:0/625269437 learned_addr learned my addr 172.21.15.148:0/625269437 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7a59c640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe7c072000 0x7efe7c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe79d9b640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c15dd80 0x7efe7c1e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 -- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c15dd80 msgr2=0x7efe7c1e2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c15dd80 0x7efe7c1e2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 -- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe7c072000 msgr2=0x7efe7c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe7c072000 0x7efe7c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 -- 172.21.15.148:0/625269437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c154e10 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe7ad9d640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 0x7efe7c1e5d00 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0090b0 tx=0x7efe6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c042070 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c03e680 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037040 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7c156cd0 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.980+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7c157290 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.981+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7c073990 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.982+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe6c03a040 con 0x7efe7c1e5850 2024-04-25T10:03:53.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.982+0000 7efe6affd640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b4d0 0x7efe5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.982+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe6c0aa3a0 con 0x7efe7c1e5850 2024-04-25T10:03:53.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.982+0000 7efe7a59c640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b4d0 0x7efe5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.983+0000 7efe7a59c640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b4d0 0x7efe5c06d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efe74007510 tx=0x7efe740074a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:53.986+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe6c079090 con 0x7efe7c1e5850 2024-04-25T10:03:54.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.156+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe7c06caa0 con 0x7efe5c06b4d0 2024-04-25T10:03:54.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.162+0000 7efe6affd640 1 -- 172.21.15.148:0/625269437 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7efe7c06caa0 con 0x7efe5c06b4d0 2024-04-25T10:03:54.176 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:54.177 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.164+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b4d0 msgr2=0x7efe5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.164+0000 7efe80f63640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b4d0 0x7efe5c06d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efe74007510 tx=0x7efe740074a0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.164+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 msgr2=0x7efe7c1e5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.164+0000 7efe80f63640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 0x7efe7c1e5d00 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0090b0 tx=0x7efe6c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 shutdown_connections 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b4d0 0x7efe5c06d990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe7c1e5850 0x7efe7c1e5d00 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7c15dd80 0x7efe7c1e2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 --2- 172.21.15.148:0/625269437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe7c072000 0x7efe7c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 >> 172.21.15.148:0/625269437 conn(0x7efe7c06f550 msgr2=0x7efe7c15cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 shutdown_connections 2024-04-25T10:03:54.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.165+0000 7efe80f63640 1 -- 172.21.15.148:0/625269437 wait complete. 2024-04-25T10:03:54.180 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-04-25T10:03:53.137713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417248,"kb_used_data":15092,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754557472,"statfs":{"total":773094113280,"available":772666851328,"internally_reserved":0,"allocated":15454208,"data_stored":7820067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.264036"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889090+0000","last_change":"2024-04-25T10:03:49.889090+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889090+0000","last_undegraded":"2024-04-25T10:03:49.889090+0000","last_fullsized":"2024-04-25T10:03:49.889090+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884569+0000","last_change":"2024-04-25T10:03:49.884569+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884569+0000","last_undegraded":"2024-04-25T10:03:49.884569+0000","last_fullsized":"2024-04-25T10:03:49.884569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885623+0000","last_change":"2024-04-25T10:03:49.885623+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885623+0000","last_undegraded":"2024-04-25T10:03:49.885623+0000","last_fullsized":"2024-04-25T10:03:49.885623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890463+0000","last_change":"2024-04-25T10:03:49.890463+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890463+0000","last_undegraded":"2024-04-25T10:03:49.890463+0000","last_fullsized":"2024-04-25T10:03:49.890463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886184+0000","last_change":"2024-04-25T10:03:49.886184+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886184+0000","last_undegraded":"2024-04-25T10:03:49.886184+0000","last_fullsized":"2024-04-25T10:03:49.886184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886898+0000","last_change":"2024-04-25T10:03:49.886898+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886898+0000","last_undegraded":"2024-04-25T10:03:49.886898+0000","last_fullsized":"2024-04-25T10:03:49.886898+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889594+0000","last_change":"2024-04-25T10:03:49.889594+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889594+0000","last_undegraded":"2024-04-25T10:03:49.889594+0000","last_fullsized":"2024-04-25T10:03:49.889594+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890128+0000","last_change":"2024-04-25T10:03:49.890128+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890128+0000","last_undegraded":"2024-04-25T10:03:49.890128+0000","last_fullsized":"2024-04-25T10:03:49.890128+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886447+0000","last_change":"2024-04-25T10:03:49.886447+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886447+0000","last_undegraded":"2024-04-25T10:03:49.886447+0000","last_fullsized":"2024-04-25T10:03:49.886447+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887235+0000","last_change":"2024-04-25T10:03:49.887235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887235+0000","last_undegraded":"2024-04-25T10:03:49.887235+0000","last_fullsized":"2024-04-25T10:03:49.887235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890445+0000","last_change":"2024-04-25T10:03:49.890445+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890445+0000","last_undegraded":"2024-04-25T10:03:49.890445+0000","last_fullsized":"2024-04-25T10:03:49.890445+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889960+0000","last_change":"2024-04-25T10:03:49.889960+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889960+0000","last_undegraded":"2024-04-25T10:03:49.889960+0000","last_fullsized":"2024-04-25T10:03:49.889960+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889706+0000","last_change":"2024-04-25T10:03:49.889706+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889706+0000","last_undegraded":"2024-04-25T10:03:49.889706+0000","last_fullsized":"2024-04-25T10:03:49.889706+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891818+0000","last_change":"2024-04-25T10:03:49.891818+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891818+0000","last_undegraded":"2024-04-25T10:03:49.891818+0000","last_fullsized":"2024-04-25T10:03:49.891818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887036+0000","last_change":"2024-04-25T10:03:49.887036+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887036+0000","last_undegraded":"2024-04-25T10:03:49.887036+0000","last_fullsized":"2024-04-25T10:03:49.887036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889684+0000","last_change":"2024-04-25T10:03:49.889684+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889684+0000","last_undegraded":"2024-04-25T10:03:49.889684+0000","last_fullsized":"2024-04-25T10:03:49.889684+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891988+0000","last_change":"2024-04-25T10:03:49.891988+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891988+0000","last_undegraded":"2024-04-25T10:03:49.891988+0000","last_fullsized":"2024-04-25T10:03:49.891988+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891421+0000","last_change":"2024-04-25T10:03:49.891421+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891421+0000","last_undegraded":"2024-04-25T10:03:49.891421+0000","last_fullsized":"2024-04-25T10:03:49.891421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888004+0000","last_change":"2024-04-25T10:03:49.888004+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888004+0000","last_undegraded":"2024-04-25T10:03:49.888004+0000","last_fullsized":"2024-04-25T10:03:49.888004+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884946+0000","last_change":"2024-04-25T10:03:49.884946+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884946+0000","last_undegraded":"2024-04-25T10:03:49.884946+0000","last_fullsized":"2024-04-25T10:03:49.884946+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886836+0000","last_change":"2024-04-25T10:03:49.886836+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886836+0000","last_undegraded":"2024-04-25T10:03:49.886836+0000","last_fullsized":"2024-04-25T10:03:49.886836+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888666+0000","last_change":"2024-04-25T10:03:49.888666+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888666+0000","last_undegraded":"2024-04-25T10:03:49.888666+0000","last_fullsized":"2024-04-25T10:03:49.888666+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887306+0000","last_change":"2024-04-25T10:03:49.887306+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887306+0000","last_undegraded":"2024-04-25T10:03:49.887306+0000","last_fullsized":"2024-04-25T10:03:49.887306+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889841+0000","last_change":"2024-04-25T10:03:49.889841+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889841+0000","last_undegraded":"2024-04-25T10:03:49.889841+0000","last_fullsized":"2024-04-25T10:03:49.889841+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888457+0000","last_change":"2024-04-25T10:03:49.888457+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888457+0000","last_undegraded":"2024-04-25T10:03:49.888457+0000","last_fullsized":"2024-04-25T10:03:49.888457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887310+0000","last_change":"2024-04-25T10:03:49.887310+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887310+0000","last_undegraded":"2024-04-25T10:03:49.887310+0000","last_fullsized":"2024-04-25T10:03:49.887310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885978+0000","last_change":"2024-04-25T10:03:49.885978+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885978+0000","last_undegraded":"2024-04-25T10:03:49.885978+0000","last_fullsized":"2024-04-25T10:03:49.885978+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891147+0000","last_change":"2024-04-25T10:03:49.891147+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891147+0000","last_undegraded":"2024-04-25T10:03:49.891147+0000","last_fullsized":"2024-04-25T10:03:49.891147+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884069+0000","last_change":"2024-04-25T10:03:49.884069+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884069+0000","last_undegraded":"2024-04-25T10:03:49.884069+0000","last_fullsized":"2024-04-25T10:03:49.884069+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.892310+0000","last_change":"2024-04-25T10:03:49.892310+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.892310+0000","last_undegraded":"2024-04-25T10:03:49.892310+0000","last_fullsized":"2024-04-25T10:03:49.892310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885408+0000","last_change":"2024-04-25T10:03:49.885408+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885408+0000","last_undegraded":"2024-04-25T10:03:49.885408+0000","last_fullsized":"2024-04-25T10:03:49.885408+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889491+0000","last_change":"2024-04-25T10:03:49.889491+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889491+0000","last_undegraded":"2024-04-25T10:03:49.889491+0000","last_fullsized":"2024-04-25T10:03:49.889491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886031+0000","last_change":"2024-04-25T10:03:49.886031+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886031+0000","last_undegraded":"2024-04-25T10:03:49.886031+0000","last_fullsized":"2024-04-25T10:03:49.886031+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893210+0000","last_change":"2024-04-25T10:03:49.891438+0000","last_active":"2024-04-25T10:03:49.893210+0000","last_peered":"2024-04-25T10:03:49.893210+0000","last_clean":"2024-04-25T10:03:49.893210+0000","last_became_active":"2024-04-25T10:03:49.890515+0000","last_became_peered":"2024-04-25T10:03:49.890515+0000","last_unstale":"2024-04-25T10:03:49.893210+0000","last_undegraded":"2024-04-25T10:03:49.893210+0000","last_fullsized":"2024-04-25T10:03:49.893210+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885234+0000","last_change":"2024-04-25T10:03:49.885234+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885234+0000","last_undegraded":"2024-04-25T10:03:49.885234+0000","last_fullsized":"2024-04-25T10:03:49.885234+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900272+0000","last_change":"2024-04-25T10:03:49.898107+0000","last_active":"2024-04-25T10:03:49.900272+0000","last_peered":"2024-04-25T10:03:49.900272+0000","last_clean":"2024-04-25T10:03:49.900272+0000","last_became_active":"2024-04-25T10:03:49.897243+0000","last_became_peered":"2024-04-25T10:03:49.897243+0000","last_unstale":"2024-04-25T10:03:49.900272+0000","last_undegraded":"2024-04-25T10:03:49.900272+0000","last_fullsized":"2024-04-25T10:03:49.900272+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885440+0000","last_change":"2024-04-25T10:03:49.885440+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885440+0000","last_undegraded":"2024-04-25T10:03:49.885440+0000","last_fullsized":"2024-04-25T10:03:49.885440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898592+0000","last_change":"2024-04-25T10:03:49.896386+0000","last_active":"2024-04-25T10:03:49.898592+0000","last_peered":"2024-04-25T10:03:49.898592+0000","last_clean":"2024-04-25T10:03:49.898592+0000","last_became_active":"2024-04-25T10:03:49.895902+0000","last_became_peered":"2024-04-25T10:03:49.895902+0000","last_unstale":"2024-04-25T10:03:49.898592+0000","last_undegraded":"2024-04-25T10:03:49.898592+0000","last_fullsized":"2024-04-25T10:03:49.898592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.352713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886573+0000","last_change":"2024-04-25T10:03:49.886573+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886573+0000","last_undegraded":"2024-04-25T10:03:49.886573+0000","last_fullsized":"2024-04-25T10:03:49.886573+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897594+0000","last_change":"2024-04-25T10:03:49.896831+0000","last_active":"2024-04-25T10:03:49.897594+0000","last_peered":"2024-04-25T10:03:49.897594+0000","last_clean":"2024-04-25T10:03:49.897594+0000","last_became_active":"2024-04-25T10:03:49.896566+0000","last_became_peered":"2024-04-25T10:03:49.896566+0000","last_unstale":"2024-04-25T10:03:49.897594+0000","last_undegraded":"2024-04-25T10:03:49.897594+0000","last_fullsized":"2024-04-25T10:03:49.897594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884360+0000","last_change":"2024-04-25T10:03:49.884360+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884360+0000","last_undegraded":"2024-04-25T10:03:49.884360+0000","last_fullsized":"2024-04-25T10:03:49.884360+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896832+0000","last_change":"2024-04-25T10:03:49.894730+0000","last_active":"2024-04-25T10:03:49.896832+0000","last_peered":"2024-04-25T10:03:49.896832+0000","last_clean":"2024-04-25T10:03:49.896832+0000","last_became_active":"2024-04-25T10:03:49.894297+0000","last_became_peered":"2024-04-25T10:03:49.894297+0000","last_unstale":"2024-04-25T10:03:49.896832+0000","last_undegraded":"2024-04-25T10:03:49.896832+0000","last_fullsized":"2024-04-25T10:03:49.896832+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:31.343353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.892113+0000","last_change":"2024-04-25T10:03:49.892113+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.892113+0000","last_undegraded":"2024-04-25T10:03:49.892113+0000","last_fullsized":"2024-04-25T10:03:49.892113+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902592+0000","last_change":"2024-04-25T10:03:49.902295+0000","last_active":"2024-04-25T10:03:49.902592+0000","last_peered":"2024-04-25T10:03:49.902592+0000","last_clean":"2024-04-25T10:03:49.902592+0000","last_became_active":"2024-04-25T10:03:49.902003+0000","last_became_peered":"2024-04-25T10:03:49.902003+0000","last_unstale":"2024-04-25T10:03:49.902592+0000","last_undegraded":"2024-04-25T10:03:49.902592+0000","last_fullsized":"2024-04-25T10:03:49.902592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:10.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885895+0000","last_change":"2024-04-25T10:03:49.885895+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885895+0000","last_undegraded":"2024-04-25T10:03:49.885895+0000","last_fullsized":"2024-04-25T10:03:49.885895+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901072+0000","last_change":"2024-04-25T10:03:49.899529+0000","last_active":"2024-04-25T10:03:49.901072+0000","last_peered":"2024-04-25T10:03:49.901072+0000","last_clean":"2024-04-25T10:03:49.901072+0000","last_became_active":"2024-04-25T10:03:49.896941+0000","last_became_peered":"2024-04-25T10:03:49.896941+0000","last_unstale":"2024-04-25T10:03:49.901072+0000","last_undegraded":"2024-04-25T10:03:49.901072+0000","last_fullsized":"2024-04-25T10:03:49.901072+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.283728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885817+0000","last_change":"2024-04-25T10:03:49.885817+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885817+0000","last_undegraded":"2024-04-25T10:03:49.885817+0000","last_fullsized":"2024-04-25T10:03:49.885817+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.894253+0000","last_change":"2024-04-25T10:03:49.893772+0000","last_active":"2024-04-25T10:03:49.894253+0000","last_peered":"2024-04-25T10:03:49.894253+0000","last_clean":"2024-04-25T10:03:49.894253+0000","last_became_active":"2024-04-25T10:03:49.893508+0000","last_became_peered":"2024-04-25T10:03:49.893508+0000","last_unstale":"2024-04-25T10:03:49.894253+0000","last_undegraded":"2024-04-25T10:03:49.894253+0000","last_fullsized":"2024-04-25T10:03:49.894253+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884845+0000","last_change":"2024-04-25T10:03:49.884845+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884845+0000","last_undegraded":"2024-04-25T10:03:49.884845+0000","last_fullsized":"2024-04-25T10:03:49.884845+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898121+0000","last_change":"2024-04-25T10:03:49.896956+0000","last_active":"2024-04-25T10:03:49.898121+0000","last_peered":"2024-04-25T10:03:49.898121+0000","last_clean":"2024-04-25T10:03:49.898121+0000","last_became_active":"2024-04-25T10:03:49.896167+0000","last_became_peered":"2024-04-25T10:03:49.896167+0000","last_unstale":"2024-04-25T10:03:49.898121+0000","last_undegraded":"2024-04-25T10:03:49.898121+0000","last_fullsized":"2024-04-25T10:03:49.898121+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890014+0000","last_change":"2024-04-25T10:03:49.890014+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890014+0000","last_undegraded":"2024-04-25T10:03:49.890014+0000","last_fullsized":"2024-04-25T10:03:49.890014+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901024+0000","last_change":"2024-04-25T10:03:49.898465+0000","last_active":"2024-04-25T10:03:49.901024+0000","last_peered":"2024-04-25T10:03:49.901024+0000","last_clean":"2024-04-25T10:03:49.901024+0000","last_became_active":"2024-04-25T10:03:49.898198+0000","last_became_peered":"2024-04-25T10:03:49.898198+0000","last_unstale":"2024-04-25T10:03:49.901024+0000","last_undegraded":"2024-04-25T10:03:49.901024+0000","last_fullsized":"2024-04-25T10:03:49.901024+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891028+0000","last_change":"2024-04-25T10:03:49.891028+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891028+0000","last_undegraded":"2024-04-25T10:03:49.891028+0000","last_fullsized":"2024-04-25T10:03:49.891028+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898187+0000","last_change":"2024-04-25T10:03:49.897101+0000","last_active":"2024-04-25T10:03:49.898187+0000","last_peered":"2024-04-25T10:03:49.898187+0000","last_clean":"2024-04-25T10:03:49.898187+0000","last_became_active":"2024-04-25T10:03:49.896790+0000","last_became_peered":"2024-04-25T10:03:49.896790+0000","last_unstale":"2024-04-25T10:03:49.898187+0000","last_undegraded":"2024-04-25T10:03:49.898187+0000","last_fullsized":"2024-04-25T10:03:49.898187+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:00.424778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888660+0000","last_change":"2024-04-25T10:03:49.888660+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888660+0000","last_undegraded":"2024-04-25T10:03:49.888660+0000","last_fullsized":"2024-04-25T10:03:49.888660+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893179+0000","last_change":"2024-04-25T10:03:49.892221+0000","last_active":"2024-04-25T10:03:49.893179+0000","last_peered":"2024-04-25T10:03:49.893179+0000","last_clean":"2024-04-25T10:03:49.893179+0000","last_became_active":"2024-04-25T10:03:49.891975+0000","last_became_peered":"2024-04-25T10:03:49.891975+0000","last_unstale":"2024-04-25T10:03:49.893179+0000","last_undegraded":"2024-04-25T10:03:49.893179+0000","last_fullsized":"2024-04-25T10:03:49.893179+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886658+0000","last_change":"2024-04-25T10:03:49.886658+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886658+0000","last_undegraded":"2024-04-25T10:03:49.886658+0000","last_fullsized":"2024-04-25T10:03:49.886658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902045+0000","last_change":"2024-04-25T10:03:49.901671+0000","last_active":"2024-04-25T10:03:49.902045+0000","last_peered":"2024-04-25T10:03:49.902045+0000","last_clean":"2024-04-25T10:03:49.902045+0000","last_became_active":"2024-04-25T10:03:49.901399+0000","last_became_peered":"2024-04-25T10:03:49.901399+0000","last_unstale":"2024-04-25T10:03:49.902045+0000","last_undegraded":"2024-04-25T10:03:49.902045+0000","last_fullsized":"2024-04-25T10:03:49.902045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:57.162299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886235+0000","last_change":"2024-04-25T10:03:49.886235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886235+0000","last_undegraded":"2024-04-25T10:03:49.886235+0000","last_fullsized":"2024-04-25T10:03:49.886235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901156+0000","last_change":"2024-04-25T10:03:49.900536+0000","last_active":"2024-04-25T10:03:49.901156+0000","last_peered":"2024-04-25T10:03:49.901156+0000","last_clean":"2024-04-25T10:03:49.901156+0000","last_became_active":"2024-04-25T10:03:49.900264+0000","last_became_peered":"2024-04-25T10:03:49.900264+0000","last_unstale":"2024-04-25T10:03:49.901156+0000","last_undegraded":"2024-04-25T10:03:49.901156+0000","last_fullsized":"2024-04-25T10:03:49.901156+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:08.595482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887864+0000","last_change":"2024-04-25T10:03:49.887864+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887864+0000","last_undegraded":"2024-04-25T10:03:49.887864+0000","last_fullsized":"2024-04-25T10:03:49.887864+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900875+0000","last_change":"2024-04-25T10:03:49.897285+0000","last_active":"2024-04-25T10:03:49.900875+0000","last_peered":"2024-04-25T10:03:49.900875+0000","last_clean":"2024-04-25T10:03:49.900875+0000","last_became_active":"2024-04-25T10:03:49.896833+0000","last_became_peered":"2024-04-25T10:03:49.896833+0000","last_unstale":"2024-04-25T10:03:49.900875+0000","last_undegraded":"2024-04-25T10:03:49.900875+0000","last_fullsized":"2024-04-25T10:03:49.900875+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885591+0000","last_change":"2024-04-25T10:03:49.885591+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885591+0000","last_undegraded":"2024-04-25T10:03:49.885591+0000","last_fullsized":"2024-04-25T10:03:49.885591+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.899696+0000","last_change":"2024-04-25T10:03:49.899336+0000","last_active":"2024-04-25T10:03:49.899696+0000","last_peered":"2024-04-25T10:03:49.899696+0000","last_clean":"2024-04-25T10:03:49.899696+0000","last_became_active":"2024-04-25T10:03:49.899096+0000","last_became_peered":"2024-04-25T10:03:49.899096+0000","last_unstale":"2024-04-25T10:03:49.899696+0000","last_undegraded":"2024-04-25T10:03:49.899696+0000","last_fullsized":"2024-04-25T10:03:49.899696+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:40.405411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886122+0000","last_change":"2024-04-25T10:03:49.886122+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886122+0000","last_undegraded":"2024-04-25T10:03:49.886122+0000","last_fullsized":"2024-04-25T10:03:49.886122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900925+0000","last_change":"2024-04-25T10:03:49.898990+0000","last_active":"2024-04-25T10:03:49.900925+0000","last_peered":"2024-04-25T10:03:49.900925+0000","last_clean":"2024-04-25T10:03:49.900925+0000","last_became_active":"2024-04-25T10:03:49.898720+0000","last_became_peered":"2024-04-25T10:03:49.898720+0000","last_unstale":"2024-04-25T10:03:49.900925+0000","last_undegraded":"2024-04-25T10:03:49.900925+0000","last_fullsized":"2024-04-25T10:03:49.900925+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886777+0000","last_change":"2024-04-25T10:03:49.886777+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886777+0000","last_undegraded":"2024-04-25T10:03:49.886777+0000","last_fullsized":"2024-04-25T10:03:49.886777+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901586+0000","last_change":"2024-04-25T10:03:49.901276+0000","last_active":"2024-04-25T10:03:49.901586+0000","last_peered":"2024-04-25T10:03:49.901586+0000","last_clean":"2024-04-25T10:03:49.901586+0000","last_became_active":"2024-04-25T10:03:49.900977+0000","last_became_peered":"2024-04-25T10:03:49.900977+0000","last_unstale":"2024-04-25T10:03:49.901586+0000","last_undegraded":"2024-04-25T10:03:49.901586+0000","last_fullsized":"2024-04-25T10:03:49.901586+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887729+0000","last_change":"2024-04-25T10:03:49.887729+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887729+0000","last_undegraded":"2024-04-25T10:03:49.887729+0000","last_fullsized":"2024-04-25T10:03:49.887729+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.899920+0000","last_change":"2024-04-25T10:03:49.898957+0000","last_active":"2024-04-25T10:03:49.899920+0000","last_peered":"2024-04-25T10:03:49.899920+0000","last_clean":"2024-04-25T10:03:49.899920+0000","last_became_active":"2024-04-25T10:03:49.898681+0000","last_became_peered":"2024-04-25T10:03:49.898681+0000","last_unstale":"2024-04-25T10:03:49.899920+0000","last_undegraded":"2024-04-25T10:03:49.899920+0000","last_fullsized":"2024-04-25T10:03:49.899920+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.487258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886494+0000","last_change":"2024-04-25T10:03:49.886494+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886494+0000","last_undegraded":"2024-04-25T10:03:49.886494+0000","last_fullsized":"2024-04-25T10:03:49.886494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896899+0000","last_change":"2024-04-25T10:03:49.895700+0000","last_active":"2024-04-25T10:03:49.896899+0000","last_peered":"2024-04-25T10:03:49.896899+0000","last_clean":"2024-04-25T10:03:49.896899+0000","last_became_active":"2024-04-25T10:03:49.895008+0000","last_became_peered":"2024-04-25T10:03:49.895008+0000","last_unstale":"2024-04-25T10:03:49.896899+0000","last_undegraded":"2024-04-25T10:03:49.896899+0000","last_fullsized":"2024-04-25T10:03:49.896899+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:03.925332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886272+0000","last_change":"2024-04-25T10:03:49.886272+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886272+0000","last_undegraded":"2024-04-25T10:03:49.886272+0000","last_fullsized":"2024-04-25T10:03:49.886272+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898332+0000","last_change":"2024-04-25T10:03:49.895688+0000","last_active":"2024-04-25T10:03:49.898332+0000","last_peered":"2024-04-25T10:03:49.898332+0000","last_clean":"2024-04-25T10:03:49.898332+0000","last_became_active":"2024-04-25T10:03:49.895423+0000","last_became_peered":"2024-04-25T10:03:49.895423+0000","last_unstale":"2024-04-25T10:03:49.898332+0000","last_undegraded":"2024-04-25T10:03:49.898332+0000","last_fullsized":"2024-04-25T10:03:49.898332+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.994391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888604+0000","last_change":"2024-04-25T10:03:49.888604+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888604+0000","last_undegraded":"2024-04-25T10:03:49.888604+0000","last_fullsized":"2024-04-25T10:03:49.888604+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896673+0000","last_change":"2024-04-25T10:03:49.895117+0000","last_active":"2024-04-25T10:03:49.896673+0000","last_peered":"2024-04-25T10:03:49.896673+0000","last_clean":"2024-04-25T10:03:49.896673+0000","last_became_active":"2024-04-25T10:03:49.894870+0000","last_became_peered":"2024-04-25T10:03:49.894870+0000","last_unstale":"2024-04-25T10:03:49.896673+0000","last_undegraded":"2024-04-25T10:03:49.896673+0000","last_fullsized":"2024-04-25T10:03:49.896673+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:30.835851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887449+0000","last_change":"2024-04-25T10:03:49.887449+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887449+0000","last_undegraded":"2024-04-25T10:03:49.887449+0000","last_fullsized":"2024-04-25T10:03:49.887449+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896859+0000","last_change":"2024-04-25T10:03:49.895442+0000","last_active":"2024-04-25T10:03:49.896859+0000","last_peered":"2024-04-25T10:03:49.896859+0000","last_clean":"2024-04-25T10:03:49.896859+0000","last_became_active":"2024-04-25T10:03:49.895113+0000","last_became_peered":"2024-04-25T10:03:49.895113+0000","last_unstale":"2024-04-25T10:03:49.896859+0000","last_undegraded":"2024-04-25T10:03:49.896859+0000","last_fullsized":"2024-04-25T10:03:49.896859+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:30.467108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887060+0000","last_change":"2024-04-25T10:03:49.887060+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887060+0000","last_undegraded":"2024-04-25T10:03:49.887060+0000","last_fullsized":"2024-04-25T10:03:49.887060+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900209+0000","last_change":"2024-04-25T10:03:49.898517+0000","last_active":"2024-04-25T10:03:49.900209+0000","last_peered":"2024-04-25T10:03:49.900209+0000","last_clean":"2024-04-25T10:03:49.900209+0000","last_became_active":"2024-04-25T10:03:49.898253+0000","last_became_peered":"2024-04-25T10:03:49.898253+0000","last_unstale":"2024-04-25T10:03:49.900209+0000","last_undegraded":"2024-04-25T10:03:49.900209+0000","last_fullsized":"2024-04-25T10:03:49.900209+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:01.257780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.893030+0000","last_change":"2024-04-25T10:03:49.893030+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.893030+0000","last_undegraded":"2024-04-25T10:03:49.893030+0000","last_fullsized":"2024-04-25T10:03:49.893030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901045+0000","last_change":"2024-04-25T10:03:49.899974+0000","last_active":"2024-04-25T10:03:49.901045+0000","last_peered":"2024-04-25T10:03:49.901045+0000","last_clean":"2024-04-25T10:03:49.901045+0000","last_became_active":"2024-04-25T10:03:49.899694+0000","last_became_peered":"2024-04-25T10:03:49.899694+0000","last_unstale":"2024-04-25T10:03:49.901045+0000","last_undegraded":"2024-04-25T10:03:49.901045+0000","last_fullsized":"2024-04-25T10:03:49.901045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.975577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886295+0000","last_change":"2024-04-25T10:03:49.886295+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886295+0000","last_undegraded":"2024-04-25T10:03:49.886295+0000","last_fullsized":"2024-04-25T10:03:49.886295+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896582+0000","last_change":"2024-04-25T10:03:49.894817+0000","last_active":"2024-04-25T10:03:49.896582+0000","last_peered":"2024-04-25T10:03:49.896582+0000","last_clean":"2024-04-25T10:03:49.896582+0000","last_became_active":"2024-04-25T10:03:49.894481+0000","last_became_peered":"2024-04-25T10:03:49.894481+0000","last_unstale":"2024-04-25T10:03:49.896582+0000","last_undegraded":"2024-04-25T10:03:49.896582+0000","last_fullsized":"2024-04-25T10:03:49.896582+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:06.966070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885867+0000","last_change":"2024-04-25T10:03:49.885867+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885867+0000","last_undegraded":"2024-04-25T10:03:49.885867+0000","last_fullsized":"2024-04-25T10:03:49.885867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897801+0000","last_change":"2024-04-25T10:03:49.897166+0000","last_active":"2024-04-25T10:03:49.897801+0000","last_peered":"2024-04-25T10:03:49.897801+0000","last_clean":"2024-04-25T10:03:49.897801+0000","last_became_active":"2024-04-25T10:03:49.896766+0000","last_became_peered":"2024-04-25T10:03:49.896766+0000","last_unstale":"2024-04-25T10:03:49.897801+0000","last_undegraded":"2024-04-25T10:03:49.897801+0000","last_fullsized":"2024-04-25T10:03:49.897801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:45.056797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885865+0000","last_change":"2024-04-25T10:03:49.885865+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885865+0000","last_undegraded":"2024-04-25T10:03:49.885865+0000","last_fullsized":"2024-04-25T10:03:49.885865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893982+0000","last_change":"2024-04-25T10:03:49.893523+0000","last_active":"2024-04-25T10:03:49.893982+0000","last_peered":"2024-04-25T10:03:49.893982+0000","last_clean":"2024-04-25T10:03:49.893982+0000","last_became_active":"2024-04-25T10:03:49.893233+0000","last_became_peered":"2024-04-25T10:03:49.893233+0000","last_unstale":"2024-04-25T10:03:49.893982+0000","last_undegraded":"2024-04-25T10:03:49.893982+0000","last_fullsized":"2024-04-25T10:03:49.893982+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891494+0000","last_change":"2024-04-25T10:03:49.891494+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891494+0000","last_undegraded":"2024-04-25T10:03:49.891494+0000","last_fullsized":"2024-04-25T10:03:49.891494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898782+0000","last_change":"2024-04-25T10:03:49.898200+0000","last_active":"2024-04-25T10:03:49.898782+0000","last_peered":"2024-04-25T10:03:49.898782+0000","last_clean":"2024-04-25T10:03:49.898782+0000","last_became_active":"2024-04-25T10:03:49.897929+0000","last_became_peered":"2024-04-25T10:03:49.897929+0000","last_unstale":"2024-04-25T10:03:49.898782+0000","last_undegraded":"2024-04-25T10:03:49.898782+0000","last_fullsized":"2024-04-25T10:03:49.898782+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:56.315225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885421+0000","last_change":"2024-04-25T10:03:49.885421+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885421+0000","last_undegraded":"2024-04-25T10:03:49.885421+0000","last_fullsized":"2024-04-25T10:03:49.885421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901223+0000","last_change":"2024-04-25T10:03:49.900360+0000","last_active":"2024-04-25T10:03:49.901223+0000","last_peered":"2024-04-25T10:03:49.901223+0000","last_clean":"2024-04-25T10:03:49.901223+0000","last_became_active":"2024-04-25T10:03:49.898980+0000","last_became_peered":"2024-04-25T10:03:49.898980+0000","last_unstale":"2024-04-25T10:03:49.901223+0000","last_undegraded":"2024-04-25T10:03:49.901223+0000","last_fullsized":"2024-04-25T10:03:49.901223+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:41.805235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885077+0000","last_change":"2024-04-25T10:03:49.885077+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885077+0000","last_undegraded":"2024-04-25T10:03:49.885077+0000","last_fullsized":"2024-04-25T10:03:49.885077+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898078+0000","last_change":"2024-04-25T10:03:49.896765+0000","last_active":"2024-04-25T10:03:49.898078+0000","last_peered":"2024-04-25T10:03:49.898078+0000","last_clean":"2024-04-25T10:03:49.898078+0000","last_became_active":"2024-04-25T10:03:49.896465+0000","last_became_peered":"2024-04-25T10:03:49.896465+0000","last_unstale":"2024-04-25T10:03:49.898078+0000","last_undegraded":"2024-04-25T10:03:49.898078+0000","last_fullsized":"2024-04-25T10:03:49.898078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:40.225994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.883729+0000","last_change":"2024-04-25T10:03:49.883729+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.883729+0000","last_undegraded":"2024-04-25T10:03:49.883729+0000","last_fullsized":"2024-04-25T10:03:49.883729+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896875+0000","last_change":"2024-04-25T10:03:49.895770+0000","last_active":"2024-04-25T10:03:49.896875+0000","last_peered":"2024-04-25T10:03:49.896875+0000","last_clean":"2024-04-25T10:03:49.896875+0000","last_became_active":"2024-04-25T10:03:49.895492+0000","last_became_peered":"2024-04-25T10:03:49.895492+0000","last_unstale":"2024-04-25T10:03:49.896875+0000","last_undegraded":"2024-04-25T10:03:49.896875+0000","last_fullsized":"2024-04-25T10:03:49.896875+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:18.825257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":396,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.886178+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:03:49.886178+0000","last_peered":"2024-04-25T10:03:49.886178+0000","last_clean":"2024-04-25T10:03:49.886178+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:03:49.886178+0000","last_undegraded":"2024-04-25T10:03:49.886178+0000","last_fullsized":"2024-04-25T10:03:49.886178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2375680,"data_stored":1419451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46812,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325028,"statfs":{"total":96636764160,"available":96588828672,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59304,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312536,"statfs":{"total":96636764160,"available":96576036864,"internally_reserved":0,"allocated":1794048,"data_stored":835750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2375680,"data_stored":1419451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:54.186 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:54.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.312+0000 7fa533563640 1 Processor -- start 2024-04-25T10:03:54.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.313+0000 7fa533563640 1 -- start start 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.313+0000 7fa533563640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa52c0cc8e0 0x7fa52c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.313+0000 7fa533563640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa52c0d1810 0x7fa52c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.313+0000 7fa533563640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0cce50 con 0x7fa52c15dc90 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.313+0000 7fa533563640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0d21c0 con 0x7fa52c0d1810 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.313+0000 7fa533563640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c161af0 con 0x7fa52c0cc8e0 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa531b00640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa52c0d1810 0x7fa52c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa5312ff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa52c15dc90 0x7fa52c15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34936/0 (socket says 172.21.15.148:34936) 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa5312ff640 1 -- 172.21.15.148:0/753673428 learned_addr learned my addr 172.21.15.148:0/753673428 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:54.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa530afe640 1 --2- 172.21.15.148:0/753673428 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0cc8e0 0x7fa52c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa531b00640 1 -- 172.21.15.148:0/753673428 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0cc8e0 msgr2=0x7fa52c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa531b00640 1 --2- 172.21.15.148:0/753673428 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0cc8e0 0x7fa52c1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa531b00640 1 -- 172.21.15.148:0/753673428 >> v1:172.21.15.148:6789/0 conn(0x7fa52c15dc90 legacy=0x7fa52c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa523fff640 1 -- 172.21.15.148:0/753673428 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934611326 0 0) 0x7fa52c0cce50 con 0x7fa52c15dc90 2024-04-25T10:03:54.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.314+0000 7fa531b00640 1 -- 172.21.15.148:0/753673428 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c161dd0 con 0x7fa52c0d1810 2024-04-25T10:03:54.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa531b00640 1 --2- 172.21.15.148:0/753673428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d1810 0x7fa52c0d1c80 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0c88a0 tx=0x7fa52402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8609d71147e832f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa523fff640 1 -- 172.21.15.148:0/753673428 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524007740 con 0x7fa52c0d1810 2024-04-25T10:03:54.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa523fff640 1 -- 172.21.15.148:0/753673428 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5240078e0 con 0x7fa52c0d1810 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa523fff640 1 -- 172.21.15.148:0/753673428 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524007cb0 con 0x7fa52c0d1810 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 -- 172.21.15.148:0/753673428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d1810 msgr2=0x7fa52c0d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 --2- 172.21.15.148:0/753673428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d1810 0x7fa52c0d1c80 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0c88a0 tx=0x7fa52402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 -- 172.21.15.148:0/753673428 shutdown_connections 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 --2- 172.21.15.148:0/753673428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c0d1810 0x7fa52c0d1c80 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 --2- 172.21.15.148:0/753673428 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0cc8e0 0x7fa52c1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 -- 172.21.15.148:0/753673428 >> 172.21.15.148:0/753673428 conn(0x7fa52c06f590 msgr2=0x7fa52c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 -- 172.21.15.148:0/753673428 shutdown_connections 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.315+0000 7fa533563640 1 -- 172.21.15.148:0/753673428 wait complete. 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 Processor -- start 2024-04-25T10:03:54.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 -- start start 2024-04-25T10:03:54.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0cc8e0 0x7fa52c156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0d1810 0x7fa52c0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 0x7fa52c0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15e070 con 0x7fa52c15dc90 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0cd130 con 0x7fa52c0cc8e0 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa533563640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15e1f0 con 0x7fa52c0d1810 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa531b00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 0x7fa52c0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa531b00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 0x7fa52c0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33666/0 (socket says 172.21.15.148:33666) 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.317+0000 7fa531b00640 1 -- 172.21.15.148:0/1798994214 learned_addr learned my addr 172.21.15.148:0/1798994214 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa530afe640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0d1810 0x7fa52c0c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa5312ff640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0cc8e0 0x7fa52c156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa531b00640 1 -- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0d1810 msgr2=0x7fa52c0c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa531b00640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0d1810 0x7fa52c0c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa531b00640 1 -- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0cc8e0 msgr2=0x7fa52c156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa531b00640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0cc8e0 0x7fa52c156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa531b00640 1 -- 172.21.15.148:0/1798994214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c1f8180 con 0x7fa52c15dc90 2024-04-25T10:03:54.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa531b00640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 0x7fa52c0c5390 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fa52402abe0 tx=0x7fa524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524040070 con 0x7fa52c15dc90 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa524031c40 con 0x7fa52c15dc90 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0c8ec0 con 0x7fa52c15dc90 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524009450 con 0x7fa52c15dc90 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.318+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c0c9480 con 0x7fa52c15dc90 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.319+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c068470 con 0x7fa52c15dc90 2024-04-25T10:03:54.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.320+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa524039040 con 0x7fa52c15dc90 2024-04-25T10:03:54.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.320+0000 7fa521ffb640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa51006b4d0 0x7fa51006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.320+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5240ab570 con 0x7fa52c15dc90 2024-04-25T10:03:54.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.320+0000 7fa5312ff640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa51006b4d0 0x7fa51006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.321+0000 7fa5312ff640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa51006b4d0 0x7fa51006d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa528000f60 tx=0x7fa528009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.324+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa52c068470 con 0x7fa52c15dc90 2024-04-25T10:03:54.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.490+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa52c073990 con 0x7fa51006b4d0 2024-04-25T10:03:54.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.495+0000 7fa521ffb640 1 -- 172.21.15.148:0/1798994214 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7fa52c073990 con 0x7fa51006b4d0 2024-04-25T10:03:54.503 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:54.504 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:03:54.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.498+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa51006b4d0 msgr2=0x7fa51006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.498+0000 7fa533563640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa51006b4d0 0x7fa51006d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa528000f60 tx=0x7fa528009040 comp rx=0 tx=0).stop 2024-04-25T10:03:54.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.498+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 msgr2=0x7fa52c0c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.498+0000 7fa533563640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 0x7fa52c0c5390 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fa52402abe0 tx=0x7fa524004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:54.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 shutdown_connections 2024-04-25T10:03:54.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa51006b4d0 0x7fa51006d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa52c15dc90 0x7fa52c0c5390 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0d1810 0x7fa52c0c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 --2- 172.21.15.148:0/1798994214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0cc8e0 0x7fa52c156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 >> 172.21.15.148:0/1798994214 conn(0x7fa52c06f590 msgr2=0x7fa52c0cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 shutdown_connections 2024-04-25T10:03:54.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.499+0000 7fa533563640 1 -- 172.21.15.148:0/1798994214 wait complete. 2024-04-25T10:03:54.515 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-04-25T10:03:53.137713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417248,"kb_used_data":15092,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754557472,"statfs":{"total":773094113280,"available":772666851328,"internally_reserved":0,"allocated":15454208,"data_stored":7820067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.264036"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889090+0000","last_change":"2024-04-25T10:03:49.889090+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889090+0000","last_undegraded":"2024-04-25T10:03:49.889090+0000","last_fullsized":"2024-04-25T10:03:49.889090+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884569+0000","last_change":"2024-04-25T10:03:49.884569+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884569+0000","last_undegraded":"2024-04-25T10:03:49.884569+0000","last_fullsized":"2024-04-25T10:03:49.884569+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885623+0000","last_change":"2024-04-25T10:03:49.885623+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885623+0000","last_undegraded":"2024-04-25T10:03:49.885623+0000","last_fullsized":"2024-04-25T10:03:49.885623+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890463+0000","last_change":"2024-04-25T10:03:49.890463+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890463+0000","last_undegraded":"2024-04-25T10:03:49.890463+0000","last_fullsized":"2024-04-25T10:03:49.890463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886184+0000","last_change":"2024-04-25T10:03:49.886184+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886184+0000","last_undegraded":"2024-04-25T10:03:49.886184+0000","last_fullsized":"2024-04-25T10:03:49.886184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886898+0000","last_change":"2024-04-25T10:03:49.886898+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886898+0000","last_undegraded":"2024-04-25T10:03:49.886898+0000","last_fullsized":"2024-04-25T10:03:49.886898+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889594+0000","last_change":"2024-04-25T10:03:49.889594+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889594+0000","last_undegraded":"2024-04-25T10:03:49.889594+0000","last_fullsized":"2024-04-25T10:03:49.889594+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890128+0000","last_change":"2024-04-25T10:03:49.890128+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890128+0000","last_undegraded":"2024-04-25T10:03:49.890128+0000","last_fullsized":"2024-04-25T10:03:49.890128+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886447+0000","last_change":"2024-04-25T10:03:49.886447+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886447+0000","last_undegraded":"2024-04-25T10:03:49.886447+0000","last_fullsized":"2024-04-25T10:03:49.886447+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887235+0000","last_change":"2024-04-25T10:03:49.887235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887235+0000","last_undegraded":"2024-04-25T10:03:49.887235+0000","last_fullsized":"2024-04-25T10:03:49.887235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890445+0000","last_change":"2024-04-25T10:03:49.890445+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890445+0000","last_undegraded":"2024-04-25T10:03:49.890445+0000","last_fullsized":"2024-04-25T10:03:49.890445+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889960+0000","last_change":"2024-04-25T10:03:49.889960+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889960+0000","last_undegraded":"2024-04-25T10:03:49.889960+0000","last_fullsized":"2024-04-25T10:03:49.889960+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889706+0000","last_change":"2024-04-25T10:03:49.889706+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889706+0000","last_undegraded":"2024-04-25T10:03:49.889706+0000","last_fullsized":"2024-04-25T10:03:49.889706+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891818+0000","last_change":"2024-04-25T10:03:49.891818+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891818+0000","last_undegraded":"2024-04-25T10:03:49.891818+0000","last_fullsized":"2024-04-25T10:03:49.891818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887036+0000","last_change":"2024-04-25T10:03:49.887036+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887036+0000","last_undegraded":"2024-04-25T10:03:49.887036+0000","last_fullsized":"2024-04-25T10:03:49.887036+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889684+0000","last_change":"2024-04-25T10:03:49.889684+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889684+0000","last_undegraded":"2024-04-25T10:03:49.889684+0000","last_fullsized":"2024-04-25T10:03:49.889684+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891988+0000","last_change":"2024-04-25T10:03:49.891988+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891988+0000","last_undegraded":"2024-04-25T10:03:49.891988+0000","last_fullsized":"2024-04-25T10:03:49.891988+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891421+0000","last_change":"2024-04-25T10:03:49.891421+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891421+0000","last_undegraded":"2024-04-25T10:03:49.891421+0000","last_fullsized":"2024-04-25T10:03:49.891421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888004+0000","last_change":"2024-04-25T10:03:49.888004+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888004+0000","last_undegraded":"2024-04-25T10:03:49.888004+0000","last_fullsized":"2024-04-25T10:03:49.888004+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884946+0000","last_change":"2024-04-25T10:03:49.884946+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884946+0000","last_undegraded":"2024-04-25T10:03:49.884946+0000","last_fullsized":"2024-04-25T10:03:49.884946+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886836+0000","last_change":"2024-04-25T10:03:49.886836+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886836+0000","last_undegraded":"2024-04-25T10:03:49.886836+0000","last_fullsized":"2024-04-25T10:03:49.886836+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888666+0000","last_change":"2024-04-25T10:03:49.888666+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888666+0000","last_undegraded":"2024-04-25T10:03:49.888666+0000","last_fullsized":"2024-04-25T10:03:49.888666+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887306+0000","last_change":"2024-04-25T10:03:49.887306+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887306+0000","last_undegraded":"2024-04-25T10:03:49.887306+0000","last_fullsized":"2024-04-25T10:03:49.887306+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889841+0000","last_change":"2024-04-25T10:03:49.889841+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889841+0000","last_undegraded":"2024-04-25T10:03:49.889841+0000","last_fullsized":"2024-04-25T10:03:49.889841+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888457+0000","last_change":"2024-04-25T10:03:49.888457+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888457+0000","last_undegraded":"2024-04-25T10:03:49.888457+0000","last_fullsized":"2024-04-25T10:03:49.888457+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887310+0000","last_change":"2024-04-25T10:03:49.887310+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887310+0000","last_undegraded":"2024-04-25T10:03:49.887310+0000","last_fullsized":"2024-04-25T10:03:49.887310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885978+0000","last_change":"2024-04-25T10:03:49.885978+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885978+0000","last_undegraded":"2024-04-25T10:03:49.885978+0000","last_fullsized":"2024-04-25T10:03:49.885978+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891147+0000","last_change":"2024-04-25T10:03:49.891147+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891147+0000","last_undegraded":"2024-04-25T10:03:49.891147+0000","last_fullsized":"2024-04-25T10:03:49.891147+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884069+0000","last_change":"2024-04-25T10:03:49.884069+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884069+0000","last_undegraded":"2024-04-25T10:03:49.884069+0000","last_fullsized":"2024-04-25T10:03:49.884069+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.892310+0000","last_change":"2024-04-25T10:03:49.892310+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.892310+0000","last_undegraded":"2024-04-25T10:03:49.892310+0000","last_fullsized":"2024-04-25T10:03:49.892310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885408+0000","last_change":"2024-04-25T10:03:49.885408+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885408+0000","last_undegraded":"2024-04-25T10:03:49.885408+0000","last_fullsized":"2024-04-25T10:03:49.885408+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889491+0000","last_change":"2024-04-25T10:03:49.889491+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889491+0000","last_undegraded":"2024-04-25T10:03:49.889491+0000","last_fullsized":"2024-04-25T10:03:49.889491+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886031+0000","last_change":"2024-04-25T10:03:49.886031+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886031+0000","last_undegraded":"2024-04-25T10:03:49.886031+0000","last_fullsized":"2024-04-25T10:03:49.886031+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893210+0000","last_change":"2024-04-25T10:03:49.891438+0000","last_active":"2024-04-25T10:03:49.893210+0000","last_peered":"2024-04-25T10:03:49.893210+0000","last_clean":"2024-04-25T10:03:49.893210+0000","last_became_active":"2024-04-25T10:03:49.890515+0000","last_became_peered":"2024-04-25T10:03:49.890515+0000","last_unstale":"2024-04-25T10:03:49.893210+0000","last_undegraded":"2024-04-25T10:03:49.893210+0000","last_fullsized":"2024-04-25T10:03:49.893210+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885234+0000","last_change":"2024-04-25T10:03:49.885234+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885234+0000","last_undegraded":"2024-04-25T10:03:49.885234+0000","last_fullsized":"2024-04-25T10:03:49.885234+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900272+0000","last_change":"2024-04-25T10:03:49.898107+0000","last_active":"2024-04-25T10:03:49.900272+0000","last_peered":"2024-04-25T10:03:49.900272+0000","last_clean":"2024-04-25T10:03:49.900272+0000","last_became_active":"2024-04-25T10:03:49.897243+0000","last_became_peered":"2024-04-25T10:03:49.897243+0000","last_unstale":"2024-04-25T10:03:49.900272+0000","last_undegraded":"2024-04-25T10:03:49.900272+0000","last_fullsized":"2024-04-25T10:03:49.900272+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885440+0000","last_change":"2024-04-25T10:03:49.885440+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885440+0000","last_undegraded":"2024-04-25T10:03:49.885440+0000","last_fullsized":"2024-04-25T10:03:49.885440+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898592+0000","last_change":"2024-04-25T10:03:49.896386+0000","last_active":"2024-04-25T10:03:49.898592+0000","last_peered":"2024-04-25T10:03:49.898592+0000","last_clean":"2024-04-25T10:03:49.898592+0000","last_became_active":"2024-04-25T10:03:49.895902+0000","last_became_peered":"2024-04-25T10:03:49.895902+0000","last_unstale":"2024-04-25T10:03:49.898592+0000","last_undegraded":"2024-04-25T10:03:49.898592+0000","last_fullsized":"2024-04-25T10:03:49.898592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.352713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886573+0000","last_change":"2024-04-25T10:03:49.886573+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886573+0000","last_undegraded":"2024-04-25T10:03:49.886573+0000","last_fullsized":"2024-04-25T10:03:49.886573+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897594+0000","last_change":"2024-04-25T10:03:49.896831+0000","last_active":"2024-04-25T10:03:49.897594+0000","last_peered":"2024-04-25T10:03:49.897594+0000","last_clean":"2024-04-25T10:03:49.897594+0000","last_became_active":"2024-04-25T10:03:49.896566+0000","last_became_peered":"2024-04-25T10:03:49.896566+0000","last_unstale":"2024-04-25T10:03:49.897594+0000","last_undegraded":"2024-04-25T10:03:49.897594+0000","last_fullsized":"2024-04-25T10:03:49.897594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884360+0000","last_change":"2024-04-25T10:03:49.884360+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884360+0000","last_undegraded":"2024-04-25T10:03:49.884360+0000","last_fullsized":"2024-04-25T10:03:49.884360+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896832+0000","last_change":"2024-04-25T10:03:49.894730+0000","last_active":"2024-04-25T10:03:49.896832+0000","last_peered":"2024-04-25T10:03:49.896832+0000","last_clean":"2024-04-25T10:03:49.896832+0000","last_became_active":"2024-04-25T10:03:49.894297+0000","last_became_peered":"2024-04-25T10:03:49.894297+0000","last_unstale":"2024-04-25T10:03:49.896832+0000","last_undegraded":"2024-04-25T10:03:49.896832+0000","last_fullsized":"2024-04-25T10:03:49.896832+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:31.343353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.892113+0000","last_change":"2024-04-25T10:03:49.892113+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.892113+0000","last_undegraded":"2024-04-25T10:03:49.892113+0000","last_fullsized":"2024-04-25T10:03:49.892113+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902592+0000","last_change":"2024-04-25T10:03:49.902295+0000","last_active":"2024-04-25T10:03:49.902592+0000","last_peered":"2024-04-25T10:03:49.902592+0000","last_clean":"2024-04-25T10:03:49.902592+0000","last_became_active":"2024-04-25T10:03:49.902003+0000","last_became_peered":"2024-04-25T10:03:49.902003+0000","last_unstale":"2024-04-25T10:03:49.902592+0000","last_undegraded":"2024-04-25T10:03:49.902592+0000","last_fullsized":"2024-04-25T10:03:49.902592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:10.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885895+0000","last_change":"2024-04-25T10:03:49.885895+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885895+0000","last_undegraded":"2024-04-25T10:03:49.885895+0000","last_fullsized":"2024-04-25T10:03:49.885895+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901072+0000","last_change":"2024-04-25T10:03:49.899529+0000","last_active":"2024-04-25T10:03:49.901072+0000","last_peered":"2024-04-25T10:03:49.901072+0000","last_clean":"2024-04-25T10:03:49.901072+0000","last_became_active":"2024-04-25T10:03:49.896941+0000","last_became_peered":"2024-04-25T10:03:49.896941+0000","last_unstale":"2024-04-25T10:03:49.901072+0000","last_undegraded":"2024-04-25T10:03:49.901072+0000","last_fullsized":"2024-04-25T10:03:49.901072+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.283728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885817+0000","last_change":"2024-04-25T10:03:49.885817+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885817+0000","last_undegraded":"2024-04-25T10:03:49.885817+0000","last_fullsized":"2024-04-25T10:03:49.885817+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.894253+0000","last_change":"2024-04-25T10:03:49.893772+0000","last_active":"2024-04-25T10:03:49.894253+0000","last_peered":"2024-04-25T10:03:49.894253+0000","last_clean":"2024-04-25T10:03:49.894253+0000","last_became_active":"2024-04-25T10:03:49.893508+0000","last_became_peered":"2024-04-25T10:03:49.893508+0000","last_unstale":"2024-04-25T10:03:49.894253+0000","last_undegraded":"2024-04-25T10:03:49.894253+0000","last_fullsized":"2024-04-25T10:03:49.894253+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.884845+0000","last_change":"2024-04-25T10:03:49.884845+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.884845+0000","last_undegraded":"2024-04-25T10:03:49.884845+0000","last_fullsized":"2024-04-25T10:03:49.884845+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898121+0000","last_change":"2024-04-25T10:03:49.896956+0000","last_active":"2024-04-25T10:03:49.898121+0000","last_peered":"2024-04-25T10:03:49.898121+0000","last_clean":"2024-04-25T10:03:49.898121+0000","last_became_active":"2024-04-25T10:03:49.896167+0000","last_became_peered":"2024-04-25T10:03:49.896167+0000","last_unstale":"2024-04-25T10:03:49.898121+0000","last_undegraded":"2024-04-25T10:03:49.898121+0000","last_fullsized":"2024-04-25T10:03:49.898121+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.890014+0000","last_change":"2024-04-25T10:03:49.890014+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.890014+0000","last_undegraded":"2024-04-25T10:03:49.890014+0000","last_fullsized":"2024-04-25T10:03:49.890014+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901024+0000","last_change":"2024-04-25T10:03:49.898465+0000","last_active":"2024-04-25T10:03:49.901024+0000","last_peered":"2024-04-25T10:03:49.901024+0000","last_clean":"2024-04-25T10:03:49.901024+0000","last_became_active":"2024-04-25T10:03:49.898198+0000","last_became_peered":"2024-04-25T10:03:49.898198+0000","last_unstale":"2024-04-25T10:03:49.901024+0000","last_undegraded":"2024-04-25T10:03:49.901024+0000","last_fullsized":"2024-04-25T10:03:49.901024+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891028+0000","last_change":"2024-04-25T10:03:49.891028+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891028+0000","last_undegraded":"2024-04-25T10:03:49.891028+0000","last_fullsized":"2024-04-25T10:03:49.891028+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898187+0000","last_change":"2024-04-25T10:03:49.897101+0000","last_active":"2024-04-25T10:03:49.898187+0000","last_peered":"2024-04-25T10:03:49.898187+0000","last_clean":"2024-04-25T10:03:49.898187+0000","last_became_active":"2024-04-25T10:03:49.896790+0000","last_became_peered":"2024-04-25T10:03:49.896790+0000","last_unstale":"2024-04-25T10:03:49.898187+0000","last_undegraded":"2024-04-25T10:03:49.898187+0000","last_fullsized":"2024-04-25T10:03:49.898187+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:00.424778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888660+0000","last_change":"2024-04-25T10:03:49.888660+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888660+0000","last_undegraded":"2024-04-25T10:03:49.888660+0000","last_fullsized":"2024-04-25T10:03:49.888660+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893179+0000","last_change":"2024-04-25T10:03:49.892221+0000","last_active":"2024-04-25T10:03:49.893179+0000","last_peered":"2024-04-25T10:03:49.893179+0000","last_clean":"2024-04-25T10:03:49.893179+0000","last_became_active":"2024-04-25T10:03:49.891975+0000","last_became_peered":"2024-04-25T10:03:49.891975+0000","last_unstale":"2024-04-25T10:03:49.893179+0000","last_undegraded":"2024-04-25T10:03:49.893179+0000","last_fullsized":"2024-04-25T10:03:49.893179+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886658+0000","last_change":"2024-04-25T10:03:49.886658+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886658+0000","last_undegraded":"2024-04-25T10:03:49.886658+0000","last_fullsized":"2024-04-25T10:03:49.886658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902045+0000","last_change":"2024-04-25T10:03:49.901671+0000","last_active":"2024-04-25T10:03:49.902045+0000","last_peered":"2024-04-25T10:03:49.902045+0000","last_clean":"2024-04-25T10:03:49.902045+0000","last_became_active":"2024-04-25T10:03:49.901399+0000","last_became_peered":"2024-04-25T10:03:49.901399+0000","last_unstale":"2024-04-25T10:03:49.902045+0000","last_undegraded":"2024-04-25T10:03:49.902045+0000","last_fullsized":"2024-04-25T10:03:49.902045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:57.162299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886235+0000","last_change":"2024-04-25T10:03:49.886235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886235+0000","last_undegraded":"2024-04-25T10:03:49.886235+0000","last_fullsized":"2024-04-25T10:03:49.886235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901156+0000","last_change":"2024-04-25T10:03:49.900536+0000","last_active":"2024-04-25T10:03:49.901156+0000","last_peered":"2024-04-25T10:03:49.901156+0000","last_clean":"2024-04-25T10:03:49.901156+0000","last_became_active":"2024-04-25T10:03:49.900264+0000","last_became_peered":"2024-04-25T10:03:49.900264+0000","last_unstale":"2024-04-25T10:03:49.901156+0000","last_undegraded":"2024-04-25T10:03:49.901156+0000","last_fullsized":"2024-04-25T10:03:49.901156+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:08.595482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887864+0000","last_change":"2024-04-25T10:03:49.887864+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887864+0000","last_undegraded":"2024-04-25T10:03:49.887864+0000","last_fullsized":"2024-04-25T10:03:49.887864+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900875+0000","last_change":"2024-04-25T10:03:49.897285+0000","last_active":"2024-04-25T10:03:49.900875+0000","last_peered":"2024-04-25T10:03:49.900875+0000","last_clean":"2024-04-25T10:03:49.900875+0000","last_became_active":"2024-04-25T10:03:49.896833+0000","last_became_peered":"2024-04-25T10:03:49.896833+0000","last_unstale":"2024-04-25T10:03:49.900875+0000","last_undegraded":"2024-04-25T10:03:49.900875+0000","last_fullsized":"2024-04-25T10:03:49.900875+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885591+0000","last_change":"2024-04-25T10:03:49.885591+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885591+0000","last_undegraded":"2024-04-25T10:03:49.885591+0000","last_fullsized":"2024-04-25T10:03:49.885591+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.899696+0000","last_change":"2024-04-25T10:03:49.899336+0000","last_active":"2024-04-25T10:03:49.899696+0000","last_peered":"2024-04-25T10:03:49.899696+0000","last_clean":"2024-04-25T10:03:49.899696+0000","last_became_active":"2024-04-25T10:03:49.899096+0000","last_became_peered":"2024-04-25T10:03:49.899096+0000","last_unstale":"2024-04-25T10:03:49.899696+0000","last_undegraded":"2024-04-25T10:03:49.899696+0000","last_fullsized":"2024-04-25T10:03:49.899696+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:40.405411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886122+0000","last_change":"2024-04-25T10:03:49.886122+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886122+0000","last_undegraded":"2024-04-25T10:03:49.886122+0000","last_fullsized":"2024-04-25T10:03:49.886122+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900925+0000","last_change":"2024-04-25T10:03:49.898990+0000","last_active":"2024-04-25T10:03:49.900925+0000","last_peered":"2024-04-25T10:03:49.900925+0000","last_clean":"2024-04-25T10:03:49.900925+0000","last_became_active":"2024-04-25T10:03:49.898720+0000","last_became_peered":"2024-04-25T10:03:49.898720+0000","last_unstale":"2024-04-25T10:03:49.900925+0000","last_undegraded":"2024-04-25T10:03:49.900925+0000","last_fullsized":"2024-04-25T10:03:49.900925+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886777+0000","last_change":"2024-04-25T10:03:49.886777+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886777+0000","last_undegraded":"2024-04-25T10:03:49.886777+0000","last_fullsized":"2024-04-25T10:03:49.886777+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901586+0000","last_change":"2024-04-25T10:03:49.901276+0000","last_active":"2024-04-25T10:03:49.901586+0000","last_peered":"2024-04-25T10:03:49.901586+0000","last_clean":"2024-04-25T10:03:49.901586+0000","last_became_active":"2024-04-25T10:03:49.900977+0000","last_became_peered":"2024-04-25T10:03:49.900977+0000","last_unstale":"2024-04-25T10:03:49.901586+0000","last_undegraded":"2024-04-25T10:03:49.901586+0000","last_fullsized":"2024-04-25T10:03:49.901586+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887729+0000","last_change":"2024-04-25T10:03:49.887729+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887729+0000","last_undegraded":"2024-04-25T10:03:49.887729+0000","last_fullsized":"2024-04-25T10:03:49.887729+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.899920+0000","last_change":"2024-04-25T10:03:49.898957+0000","last_active":"2024-04-25T10:03:49.899920+0000","last_peered":"2024-04-25T10:03:49.899920+0000","last_clean":"2024-04-25T10:03:49.899920+0000","last_became_active":"2024-04-25T10:03:49.898681+0000","last_became_peered":"2024-04-25T10:03:49.898681+0000","last_unstale":"2024-04-25T10:03:49.899920+0000","last_undegraded":"2024-04-25T10:03:49.899920+0000","last_fullsized":"2024-04-25T10:03:49.899920+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.487258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886494+0000","last_change":"2024-04-25T10:03:49.886494+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886494+0000","last_undegraded":"2024-04-25T10:03:49.886494+0000","last_fullsized":"2024-04-25T10:03:49.886494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896899+0000","last_change":"2024-04-25T10:03:49.895700+0000","last_active":"2024-04-25T10:03:49.896899+0000","last_peered":"2024-04-25T10:03:49.896899+0000","last_clean":"2024-04-25T10:03:49.896899+0000","last_became_active":"2024-04-25T10:03:49.895008+0000","last_became_peered":"2024-04-25T10:03:49.895008+0000","last_unstale":"2024-04-25T10:03:49.896899+0000","last_undegraded":"2024-04-25T10:03:49.896899+0000","last_fullsized":"2024-04-25T10:03:49.896899+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:03.925332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886272+0000","last_change":"2024-04-25T10:03:49.886272+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886272+0000","last_undegraded":"2024-04-25T10:03:49.886272+0000","last_fullsized":"2024-04-25T10:03:49.886272+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898332+0000","last_change":"2024-04-25T10:03:49.895688+0000","last_active":"2024-04-25T10:03:49.898332+0000","last_peered":"2024-04-25T10:03:49.898332+0000","last_clean":"2024-04-25T10:03:49.898332+0000","last_became_active":"2024-04-25T10:03:49.895423+0000","last_became_peered":"2024-04-25T10:03:49.895423+0000","last_unstale":"2024-04-25T10:03:49.898332+0000","last_undegraded":"2024-04-25T10:03:49.898332+0000","last_fullsized":"2024-04-25T10:03:49.898332+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.994391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.888604+0000","last_change":"2024-04-25T10:03:49.888604+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.888604+0000","last_undegraded":"2024-04-25T10:03:49.888604+0000","last_fullsized":"2024-04-25T10:03:49.888604+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896673+0000","last_change":"2024-04-25T10:03:49.895117+0000","last_active":"2024-04-25T10:03:49.896673+0000","last_peered":"2024-04-25T10:03:49.896673+0000","last_clean":"2024-04-25T10:03:49.896673+0000","last_became_active":"2024-04-25T10:03:49.894870+0000","last_became_peered":"2024-04-25T10:03:49.894870+0000","last_unstale":"2024-04-25T10:03:49.896673+0000","last_undegraded":"2024-04-25T10:03:49.896673+0000","last_fullsized":"2024-04-25T10:03:49.896673+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:30.835851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887449+0000","last_change":"2024-04-25T10:03:49.887449+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887449+0000","last_undegraded":"2024-04-25T10:03:49.887449+0000","last_fullsized":"2024-04-25T10:03:49.887449+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896859+0000","last_change":"2024-04-25T10:03:49.895442+0000","last_active":"2024-04-25T10:03:49.896859+0000","last_peered":"2024-04-25T10:03:49.896859+0000","last_clean":"2024-04-25T10:03:49.896859+0000","last_became_active":"2024-04-25T10:03:49.895113+0000","last_became_peered":"2024-04-25T10:03:49.895113+0000","last_unstale":"2024-04-25T10:03:49.896859+0000","last_undegraded":"2024-04-25T10:03:49.896859+0000","last_fullsized":"2024-04-25T10:03:49.896859+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:30.467108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887060+0000","last_change":"2024-04-25T10:03:49.887060+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887060+0000","last_undegraded":"2024-04-25T10:03:49.887060+0000","last_fullsized":"2024-04-25T10:03:49.887060+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.900209+0000","last_change":"2024-04-25T10:03:49.898517+0000","last_active":"2024-04-25T10:03:49.900209+0000","last_peered":"2024-04-25T10:03:49.900209+0000","last_clean":"2024-04-25T10:03:49.900209+0000","last_became_active":"2024-04-25T10:03:49.898253+0000","last_became_peered":"2024-04-25T10:03:49.898253+0000","last_unstale":"2024-04-25T10:03:49.900209+0000","last_undegraded":"2024-04-25T10:03:49.900209+0000","last_fullsized":"2024-04-25T10:03:49.900209+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:01.257780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.893030+0000","last_change":"2024-04-25T10:03:49.893030+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.893030+0000","last_undegraded":"2024-04-25T10:03:49.893030+0000","last_fullsized":"2024-04-25T10:03:49.893030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901045+0000","last_change":"2024-04-25T10:03:49.899974+0000","last_active":"2024-04-25T10:03:49.901045+0000","last_peered":"2024-04-25T10:03:49.901045+0000","last_clean":"2024-04-25T10:03:49.901045+0000","last_became_active":"2024-04-25T10:03:49.899694+0000","last_became_peered":"2024-04-25T10:03:49.899694+0000","last_unstale":"2024-04-25T10:03:49.901045+0000","last_undegraded":"2024-04-25T10:03:49.901045+0000","last_fullsized":"2024-04-25T10:03:49.901045+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.975577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886295+0000","last_change":"2024-04-25T10:03:49.886295+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886295+0000","last_undegraded":"2024-04-25T10:03:49.886295+0000","last_fullsized":"2024-04-25T10:03:49.886295+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896582+0000","last_change":"2024-04-25T10:03:49.894817+0000","last_active":"2024-04-25T10:03:49.896582+0000","last_peered":"2024-04-25T10:03:49.896582+0000","last_clean":"2024-04-25T10:03:49.896582+0000","last_became_active":"2024-04-25T10:03:49.894481+0000","last_became_peered":"2024-04-25T10:03:49.894481+0000","last_unstale":"2024-04-25T10:03:49.896582+0000","last_undegraded":"2024-04-25T10:03:49.896582+0000","last_fullsized":"2024-04-25T10:03:49.896582+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:06.966070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885867+0000","last_change":"2024-04-25T10:03:49.885867+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885867+0000","last_undegraded":"2024-04-25T10:03:49.885867+0000","last_fullsized":"2024-04-25T10:03:49.885867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897801+0000","last_change":"2024-04-25T10:03:49.897166+0000","last_active":"2024-04-25T10:03:49.897801+0000","last_peered":"2024-04-25T10:03:49.897801+0000","last_clean":"2024-04-25T10:03:49.897801+0000","last_became_active":"2024-04-25T10:03:49.896766+0000","last_became_peered":"2024-04-25T10:03:49.896766+0000","last_unstale":"2024-04-25T10:03:49.897801+0000","last_undegraded":"2024-04-25T10:03:49.897801+0000","last_fullsized":"2024-04-25T10:03:49.897801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:45.056797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885865+0000","last_change":"2024-04-25T10:03:49.885865+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885865+0000","last_undegraded":"2024-04-25T10:03:49.885865+0000","last_fullsized":"2024-04-25T10:03:49.885865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893982+0000","last_change":"2024-04-25T10:03:49.893523+0000","last_active":"2024-04-25T10:03:49.893982+0000","last_peered":"2024-04-25T10:03:49.893982+0000","last_clean":"2024-04-25T10:03:49.893982+0000","last_became_active":"2024-04-25T10:03:49.893233+0000","last_became_peered":"2024-04-25T10:03:49.893233+0000","last_unstale":"2024-04-25T10:03:49.893982+0000","last_undegraded":"2024-04-25T10:03:49.893982+0000","last_fullsized":"2024-04-25T10:03:49.893982+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.891494+0000","last_change":"2024-04-25T10:03:49.891494+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.891494+0000","last_undegraded":"2024-04-25T10:03:49.891494+0000","last_fullsized":"2024-04-25T10:03:49.891494+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898782+0000","last_change":"2024-04-25T10:03:49.898200+0000","last_active":"2024-04-25T10:03:49.898782+0000","last_peered":"2024-04-25T10:03:49.898782+0000","last_clean":"2024-04-25T10:03:49.898782+0000","last_became_active":"2024-04-25T10:03:49.897929+0000","last_became_peered":"2024-04-25T10:03:49.897929+0000","last_unstale":"2024-04-25T10:03:49.898782+0000","last_undegraded":"2024-04-25T10:03:49.898782+0000","last_fullsized":"2024-04-25T10:03:49.898782+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:56.315225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885421+0000","last_change":"2024-04-25T10:03:49.885421+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885421+0000","last_undegraded":"2024-04-25T10:03:49.885421+0000","last_fullsized":"2024-04-25T10:03:49.885421+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.901223+0000","last_change":"2024-04-25T10:03:49.900360+0000","last_active":"2024-04-25T10:03:49.901223+0000","last_peered":"2024-04-25T10:03:49.901223+0000","last_clean":"2024-04-25T10:03:49.901223+0000","last_became_active":"2024-04-25T10:03:49.898980+0000","last_became_peered":"2024-04-25T10:03:49.898980+0000","last_unstale":"2024-04-25T10:03:49.901223+0000","last_undegraded":"2024-04-25T10:03:49.901223+0000","last_fullsized":"2024-04-25T10:03:49.901223+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:41.805235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885077+0000","last_change":"2024-04-25T10:03:49.885077+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885077+0000","last_undegraded":"2024-04-25T10:03:49.885077+0000","last_fullsized":"2024-04-25T10:03:49.885077+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.898078+0000","last_change":"2024-04-25T10:03:49.896765+0000","last_active":"2024-04-25T10:03:49.898078+0000","last_peered":"2024-04-25T10:03:49.898078+0000","last_clean":"2024-04-25T10:03:49.898078+0000","last_became_active":"2024-04-25T10:03:49.896465+0000","last_became_peered":"2024-04-25T10:03:49.896465+0000","last_unstale":"2024-04-25T10:03:49.898078+0000","last_undegraded":"2024-04-25T10:03:49.898078+0000","last_fullsized":"2024-04-25T10:03:49.898078+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:40.225994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.883729+0000","last_change":"2024-04-25T10:03:49.883729+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.883729+0000","last_undegraded":"2024-04-25T10:03:49.883729+0000","last_fullsized":"2024-04-25T10:03:49.883729+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.896875+0000","last_change":"2024-04-25T10:03:49.895770+0000","last_active":"2024-04-25T10:03:49.896875+0000","last_peered":"2024-04-25T10:03:49.896875+0000","last_clean":"2024-04-25T10:03:49.896875+0000","last_became_active":"2024-04-25T10:03:49.895492+0000","last_became_peered":"2024-04-25T10:03:49.895492+0000","last_unstale":"2024-04-25T10:03:49.896875+0000","last_undegraded":"2024-04-25T10:03:49.896875+0000","last_fullsized":"2024-04-25T10:03:49.896875+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:18.825257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":396,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.886178+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:03:49.886178+0000","last_peered":"2024-04-25T10:03:49.886178+0000","last_clean":"2024-04-25T10:03:49.886178+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:03:49.886178+0000","last_undegraded":"2024-04-25T10:03:49.886178+0000","last_fullsized":"2024-04-25T10:03:49.886178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2375680,"data_stored":1419451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46812,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325028,"statfs":{"total":96636764160,"available":96588828672,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55128,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316712,"statfs":{"total":96636764160,"available":96580313088,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59304,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312536,"statfs":{"total":96636764160,"available":96576036864,"internally_reserved":0,"allocated":1794048,"data_stored":835750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50968,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320872,"statfs":{"total":96636764160,"available":96584572928,"internally_reserved":0,"allocated":1781760,"data_stored":829083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2375680,"data_stored":1419451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889090+0000', 'last_change': '2024-04-25T10:03:49.889090+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889090+0000', 'last_undegraded': '2024-04-25T10:03:49.889090+0000', 'last_fullsized': '2024-04-25T10:03:49.889090+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.884569+0000', 'last_change': '2024-04-25T10:03:49.884569+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.884569+0000', 'last_undegraded': '2024-04-25T10:03:49.884569+0000', 'last_fullsized': '2024-04-25T10:03:49.884569+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885623+0000', 'last_change': '2024-04-25T10:03:49.885623+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885623+0000', 'last_undegraded': '2024-04-25T10:03:49.885623+0000', 'last_fullsized': '2024-04-25T10:03:49.885623+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.890463+0000', 'last_change': '2024-04-25T10:03:49.890463+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.890463+0000', 'last_undegraded': '2024-04-25T10:03:49.890463+0000', 'last_fullsized': '2024-04-25T10:03:49.890463+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886184+0000', 'last_change': '2024-04-25T10:03:49.886184+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886184+0000', 'last_undegraded': '2024-04-25T10:03:49.886184+0000', 'last_fullsized': '2024-04-25T10:03:49.886184+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886898+0000', 'last_change': '2024-04-25T10:03:49.886898+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886898+0000', 'last_undegraded': '2024-04-25T10:03:49.886898+0000', 'last_fullsized': '2024-04-25T10:03:49.886898+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889594+0000', 'last_change': '2024-04-25T10:03:49.889594+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889594+0000', 'last_undegraded': '2024-04-25T10:03:49.889594+0000', 'last_fullsized': '2024-04-25T10:03:49.889594+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.890128+0000', 'last_change': '2024-04-25T10:03:49.890128+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.890128+0000', 'last_undegraded': '2024-04-25T10:03:49.890128+0000', 'last_fullsized': '2024-04-25T10:03:49.890128+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886447+0000', 'last_change': '2024-04-25T10:03:49.886447+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886447+0000', 'last_undegraded': '2024-04-25T10:03:49.886447+0000', 'last_fullsized': '2024-04-25T10:03:49.886447+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887235+0000', 'last_change': '2024-04-25T10:03:49.887235+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887235+0000', 'last_undegraded': '2024-04-25T10:03:49.887235+0000', 'last_fullsized': '2024-04-25T10:03:49.887235+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.890445+0000', 'last_change': '2024-04-25T10:03:49.890445+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.890445+0000', 'last_undegraded': '2024-04-25T10:03:49.890445+0000', 'last_fullsized': '2024-04-25T10:03:49.890445+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889960+0000', 'last_change': '2024-04-25T10:03:49.889960+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889960+0000', 'last_undegraded': '2024-04-25T10:03:49.889960+0000', 'last_fullsized': '2024-04-25T10:03:49.889960+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889706+0000', 'last_change': '2024-04-25T10:03:49.889706+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889706+0000', 'last_undegraded': '2024-04-25T10:03:49.889706+0000', 'last_fullsized': '2024-04-25T10:03:49.889706+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2024-04-25T10:03:54.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.891818+0000', 'last_change': '2024-04-25T10:03:49.891818+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.891818+0000', 'last_undegraded': '2024-04-25T10:03:49.891818+0000', 'last_fullsized': '2024-04-25T10:03:49.891818+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887036+0000', 'last_change': '2024-04-25T10:03:49.887036+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887036+0000', 'last_undegraded': '2024-04-25T10:03:49.887036+0000', 'last_fullsized': '2024-04-25T10:03:49.887036+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889684+0000', 'last_change': '2024-04-25T10:03:49.889684+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889684+0000', 'last_undegraded': '2024-04-25T10:03:49.889684+0000', 'last_fullsized': '2024-04-25T10:03:49.889684+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.891988+0000', 'last_change': '2024-04-25T10:03:49.891988+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.891988+0000', 'last_undegraded': '2024-04-25T10:03:49.891988+0000', 'last_fullsized': '2024-04-25T10:03:49.891988+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.891421+0000', 'last_change': '2024-04-25T10:03:49.891421+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.891421+0000', 'last_undegraded': '2024-04-25T10:03:49.891421+0000', 'last_fullsized': '2024-04-25T10:03:49.891421+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.888004+0000', 'last_change': '2024-04-25T10:03:49.888004+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.888004+0000', 'last_undegraded': '2024-04-25T10:03:49.888004+0000', 'last_fullsized': '2024-04-25T10:03:49.888004+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:54.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.884946+0000', 'last_change': '2024-04-25T10:03:49.884946+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.884946+0000', 'last_undegraded': '2024-04-25T10:03:49.884946+0000', 'last_fullsized': '2024-04-25T10:03:49.884946+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886836+0000', 'last_change': '2024-04-25T10:03:49.886836+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886836+0000', 'last_undegraded': '2024-04-25T10:03:49.886836+0000', 'last_fullsized': '2024-04-25T10:03:49.886836+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.888666+0000', 'last_change': '2024-04-25T10:03:49.888666+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.888666+0000', 'last_undegraded': '2024-04-25T10:03:49.888666+0000', 'last_fullsized': '2024-04-25T10:03:49.888666+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887306+0000', 'last_change': '2024-04-25T10:03:49.887306+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887306+0000', 'last_undegraded': '2024-04-25T10:03:49.887306+0000', 'last_fullsized': '2024-04-25T10:03:49.887306+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889841+0000', 'last_change': '2024-04-25T10:03:49.889841+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889841+0000', 'last_undegraded': '2024-04-25T10:03:49.889841+0000', 'last_fullsized': '2024-04-25T10:03:49.889841+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2024-04-25T10:03:54.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.888457+0000', 'last_change': '2024-04-25T10:03:49.888457+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.888457+0000', 'last_undegraded': '2024-04-25T10:03:49.888457+0000', 'last_fullsized': '2024-04-25T10:03:49.888457+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887310+0000', 'last_change': '2024-04-25T10:03:49.887310+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887310+0000', 'last_undegraded': '2024-04-25T10:03:49.887310+0000', 'last_fullsized': '2024-04-25T10:03:49.887310+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885978+0000', 'last_change': '2024-04-25T10:03:49.885978+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885978+0000', 'last_undegraded': '2024-04-25T10:03:49.885978+0000', 'last_fullsized': '2024-04-25T10:03:49.885978+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.891147+0000', 'last_change': '2024-04-25T10:03:49.891147+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.891147+0000', 'last_undegraded': '2024-04-25T10:03:49.891147+0000', 'last_fullsized': '2024-04-25T10:03:49.891147+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.884069+0000', 'last_change': '2024-04-25T10:03:49.884069+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.884069+0000', 'last_undegraded': '2024-04-25T10:03:49.884069+0000', 'last_fullsized': '2024-04-25T10:03:49.884069+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2024-04-25T10:03:54.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.892310+0000', 'last_change': '2024-04-25T10:03:49.892310+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.892310+0000', 'last_undegraded': '2024-04-25T10:03:49.892310+0000', 'last_fullsized': '2024-04-25T10:03:49.892310+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885408+0000', 'last_change': '2024-04-25T10:03:49.885408+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885408+0000', 'last_undegraded': '2024-04-25T10:03:49.885408+0000', 'last_fullsized': '2024-04-25T10:03:49.885408+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.889491+0000', 'last_change': '2024-04-25T10:03:49.889491+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.889491+0000', 'last_undegraded': '2024-04-25T10:03:49.889491+0000', 'last_fullsized': '2024-04-25T10:03:49.889491+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886031+0000', 'last_change': '2024-04-25T10:03:49.886031+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886031+0000', 'last_undegraded': '2024-04-25T10:03:49.886031+0000', 'last_fullsized': '2024-04-25T10:03:49.886031+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885234+0000', 'last_change': '2024-04-25T10:03:49.885234+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885234+0000', 'last_undegraded': '2024-04-25T10:03:49.885234+0000', 'last_fullsized': '2024-04-25T10:03:49.885234+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885440+0000', 'last_change': '2024-04-25T10:03:49.885440+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885440+0000', 'last_undegraded': '2024-04-25T10:03:49.885440+0000', 'last_fullsized': '2024-04-25T10:03:49.885440+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886573+0000', 'last_change': '2024-04-25T10:03:49.886573+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886573+0000', 'last_undegraded': '2024-04-25T10:03:49.886573+0000', 'last_fullsized': '2024-04-25T10:03:49.886573+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:54.525 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.884360+0000', 'last_change': '2024-04-25T10:03:49.884360+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.884360+0000', 'last_undegraded': '2024-04-25T10:03:49.884360+0000', 'last_fullsized': '2024-04-25T10:03:49.884360+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.892113+0000', 'last_change': '2024-04-25T10:03:49.892113+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.892113+0000', 'last_undegraded': '2024-04-25T10:03:49.892113+0000', 'last_fullsized': '2024-04-25T10:03:49.892113+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885895+0000', 'last_change': '2024-04-25T10:03:49.885895+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885895+0000', 'last_undegraded': '2024-04-25T10:03:49.885895+0000', 'last_fullsized': '2024-04-25T10:03:49.885895+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885817+0000', 'last_change': '2024-04-25T10:03:49.885817+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885817+0000', 'last_undegraded': '2024-04-25T10:03:49.885817+0000', 'last_fullsized': '2024-04-25T10:03:49.885817+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.884845+0000', 'last_change': '2024-04-25T10:03:49.884845+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.884845+0000', 'last_undegraded': '2024-04-25T10:03:49.884845+0000', 'last_fullsized': '2024-04-25T10:03:49.884845+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2024-04-25T10:03:54.526 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.890014+0000', 'last_change': '2024-04-25T10:03:49.890014+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.890014+0000', 'last_undegraded': '2024-04-25T10:03:49.890014+0000', 'last_fullsized': '2024-04-25T10:03:49.890014+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.891028+0000', 'last_change': '2024-04-25T10:03:49.891028+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.891028+0000', 'last_undegraded': '2024-04-25T10:03:49.891028+0000', 'last_fullsized': '2024-04-25T10:03:49.891028+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.888660+0000', 'last_change': '2024-04-25T10:03:49.888660+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.888660+0000', 'last_undegraded': '2024-04-25T10:03:49.888660+0000', 'last_fullsized': '2024-04-25T10:03:49.888660+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886658+0000', 'last_change': '2024-04-25T10:03:49.886658+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886658+0000', 'last_undegraded': '2024-04-25T10:03:49.886658+0000', 'last_fullsized': '2024-04-25T10:03:49.886658+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886235+0000', 'last_change': '2024-04-25T10:03:49.886235+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886235+0000', 'last_undegraded': '2024-04-25T10:03:49.886235+0000', 'last_fullsized': '2024-04-25T10:03:49.886235+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887864+0000', 'last_change': '2024-04-25T10:03:49.887864+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887864+0000', 'last_undegraded': '2024-04-25T10:03:49.887864+0000', 'last_fullsized': '2024-04-25T10:03:49.887864+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.527 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885591+0000', 'last_change': '2024-04-25T10:03:49.885591+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885591+0000', 'last_undegraded': '2024-04-25T10:03:49.885591+0000', 'last_fullsized': '2024-04-25T10:03:49.885591+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886122+0000', 'last_change': '2024-04-25T10:03:49.886122+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886122+0000', 'last_undegraded': '2024-04-25T10:03:49.886122+0000', 'last_fullsized': '2024-04-25T10:03:49.886122+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886777+0000', 'last_change': '2024-04-25T10:03:49.886777+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886777+0000', 'last_undegraded': '2024-04-25T10:03:49.886777+0000', 'last_fullsized': '2024-04-25T10:03:49.886777+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887729+0000', 'last_change': '2024-04-25T10:03:49.887729+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887729+0000', 'last_undegraded': '2024-04-25T10:03:49.887729+0000', 'last_fullsized': '2024-04-25T10:03:49.887729+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886494+0000', 'last_change': '2024-04-25T10:03:49.886494+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886494+0000', 'last_undegraded': '2024-04-25T10:03:49.886494+0000', 'last_fullsized': '2024-04-25T10:03:49.886494+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2024-04-25T10:03:54.528 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886272+0000', 'last_change': '2024-04-25T10:03:49.886272+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886272+0000', 'last_undegraded': '2024-04-25T10:03:49.886272+0000', 'last_fullsized': '2024-04-25T10:03:49.886272+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.888604+0000', 'last_change': '2024-04-25T10:03:49.888604+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.888604+0000', 'last_undegraded': '2024-04-25T10:03:49.888604+0000', 'last_fullsized': '2024-04-25T10:03:49.888604+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887449+0000', 'last_change': '2024-04-25T10:03:49.887449+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887449+0000', 'last_undegraded': '2024-04-25T10:03:49.887449+0000', 'last_fullsized': '2024-04-25T10:03:49.887449+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.887060+0000', 'last_change': '2024-04-25T10:03:49.887060+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.887060+0000', 'last_undegraded': '2024-04-25T10:03:49.887060+0000', 'last_fullsized': '2024-04-25T10:03:49.887060+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.893030+0000', 'last_change': '2024-04-25T10:03:49.893030+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.893030+0000', 'last_undegraded': '2024-04-25T10:03:49.893030+0000', 'last_fullsized': '2024-04-25T10:03:49.893030+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.886295+0000', 'last_change': '2024-04-25T10:03:49.886295+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.886295+0000', 'last_undegraded': '2024-04-25T10:03:49.886295+0000', 'last_fullsized': '2024-04-25T10:03:49.886295+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2024-04-25T10:03:54.529 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885867+0000', 'last_change': '2024-04-25T10:03:49.885867+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885867+0000', 'last_undegraded': '2024-04-25T10:03:49.885867+0000', 'last_fullsized': '2024-04-25T10:03:49.885867+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885865+0000', 'last_change': '2024-04-25T10:03:49.885865+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885865+0000', 'last_undegraded': '2024-04-25T10:03:49.885865+0000', 'last_fullsized': '2024-04-25T10:03:49.885865+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.891494+0000', 'last_change': '2024-04-25T10:03:49.891494+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.891494+0000', 'last_undegraded': '2024-04-25T10:03:49.891494+0000', 'last_fullsized': '2024-04-25T10:03:49.891494+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885421+0000', 'last_change': '2024-04-25T10:03:49.885421+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885421+0000', 'last_undegraded': '2024-04-25T10:03:49.885421+0000', 'last_fullsized': '2024-04-25T10:03:49.885421+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2024-04-25T10:03:54.530 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.885077+0000', 'last_change': '2024-04-25T10:03:49.885077+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.885077+0000', 'last_undegraded': '2024-04-25T10:03:49.885077+0000', 'last_fullsized': '2024-04-25T10:03:49.885077+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.531 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2024-04-25T10:03:54.531 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:49.883729+0000', 'last_change': '2024-04-25T10:03:49.883729+0000', 'last_active': '2024-04-25T10:03:49.873764+0000', 'last_peered': '2024-04-25T10:03:49.873764+0000', 'last_clean': '2024-04-25T10:03:49.873764+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:49.883729+0000', 'last_undegraded': '2024-04-25T10:03:49.883729+0000', 'last_fullsized': '2024-04-25T10:03:49.883729+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:49.873764+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:03:54.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:54.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.646+0000 7f2b927b7640 1 Processor -- start 2024-04-25T10:03:54.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.646+0000 7f2b927b7640 1 -- start start 2024-04-25T10:03:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b927b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b8c15b9c0 0x7f2b8c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b927b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c072d30 0x7f2b8c06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b927b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c0cfa60 con 0x7f2b8c06abe0 2024-04-25T10:03:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b927b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c0731a0 con 0x7f2b8c15b9c0 2024-04-25T10:03:54.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b927b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0cfd40 con 0x7f2b8c072d30 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2b8c06abe0 0x7f2b8c072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34948/0 (socket says 172.21.15.148:34948) 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8b7fe640 1 -- 172.21.15.148:0/1094359987 learned_addr learned my addr 172.21.15.148:0/1094359987 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8bfff640 1 --2- 172.21.15.148:0/1094359987 >> v2:172.21.15.188:3300/0 conn(0x7f2b8c15b9c0 0x7f2b8c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b90d54640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c072d30 0x7f2b8c06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8affd640 1 -- 172.21.15.148:0/1094359987 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440528067 0 0) 0x7f2b8c0cfa60 con 0x7f2b8c06abe0 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8affd640 1 -- 172.21.15.148:0/1094359987 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b74003520 con 0x7f2b8c06abe0 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8bfff640 1 -- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c072d30 msgr2=0x7f2b8c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8affd640 1 -- 172.21.15.148:0/1094359987 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3714191685 0 0) 0x7f2b74003520 con 0x7f2b8c06abe0 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8bfff640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c072d30 0x7f2b8c06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8bfff640 1 -- 172.21.15.148:0/1094359987 >> v1:172.21.15.148:6789/0 conn(0x7f2b8c06abe0 legacy=0x7f2b8c072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b8bfff640 1 -- 172.21.15.148:0/1094359987 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0d0020 con 0x7f2b8c15b9c0 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.647+0000 7f2b90d54640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c072d30 0x7f2b8c06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:54.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.648+0000 7f2b8bfff640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c15b9c0 0x7f2b8c15bda0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0097a0 tx=0x7f2b7c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd47929e96bb326e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.648+0000 7f2b8affd640 1 -- 172.21.15.148:0/1094359987 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c037070 con 0x7f2b8c15b9c0 2024-04-25T10:03:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.648+0000 7f2b8affd640 1 -- 172.21.15.148:0/1094359987 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c02aba0 con 0x7f2b8c15b9c0 2024-04-25T10:03:54.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.648+0000 7f2b8affd640 1 -- 172.21.15.148:0/1094359987 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80001690 con 0x7f2b8c15b9c0 2024-04-25T10:03:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 -- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c15b9c0 msgr2=0x7f2b8c15bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c15b9c0 0x7f2b8c15bda0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0097a0 tx=0x7f2b7c02a110 comp rx=0 tx=0).stop 2024-04-25T10:03:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 -- 172.21.15.148:0/1094359987 shutdown_connections 2024-04-25T10:03:54.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c072d30 0x7f2b8c06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 --2- 172.21.15.148:0/1094359987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c15b9c0 0x7f2b8c15bda0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 -- 172.21.15.148:0/1094359987 >> 172.21.15.148:0/1094359987 conn(0x7f2b8c0b2370 msgr2=0x7f2b8c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 -- 172.21.15.148:0/1094359987 shutdown_connections 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.649+0000 7f2b927b7640 1 -- 172.21.15.148:0/1094359987 wait complete. 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.650+0000 7f2b927b7640 1 Processor -- start 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.650+0000 7f2b927b7640 1 -- start start 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b927b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c06abe0 0x7f2b8c0c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b927b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 0x7f2b8c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b927b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c15b9c0 0x7f2b8c14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b927b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15c2e0 con 0x7f2b8c072d30 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b927b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0cf710 con 0x7f2b8c06abe0 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b927b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0d0820 con 0x7f2b8c15b9c0 2024-04-25T10:03:54.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 0x7f2b8c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 0x7f2b8c0ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33674/0 (socket says 172.21.15.148:33674) 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 -- 172.21.15.148:0/3761571420 learned_addr learned my addr 172.21.15.148:0/3761571420 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8bfff640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c06abe0 0x7f2b8c0c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b90d54640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c15b9c0 0x7f2b8c14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 -- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c15b9c0 msgr2=0x7f2b8c14dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c15b9c0 0x7f2b8c14dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 -- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c06abe0 msgr2=0x7f2b8c0c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.651+0000 7f2b8b7fe640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c06abe0 0x7f2b8c0c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b8b7fe640 1 -- 172.21.15.148:0/3761571420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c159920 con 0x7f2b8c072d30 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b8b7fe640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 0x7f2b8c0ca930 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000c750 tx=0x7f2b8000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80018070 con 0x7f2b8c072d30 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b800072d0 con 0x7f2b8c072d30 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c0af3e0 con 0x7f2b8c072d30 2024-04-25T10:03:54.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b800047e0 con 0x7f2b8c072d30 2024-04-25T10:03:54.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.652+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c0b46b0 con 0x7f2b8c072d30 2024-04-25T10:03:54.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.653+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c073990 con 0x7f2b8c072d30 2024-04-25T10:03:54.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.653+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b8001b070 con 0x7f2b8c072d30 2024-04-25T10:03:54.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.654+0000 7f2b88ff9640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b7806b4d0 0x7f2b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.654+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b80089710 con 0x7f2b8c072d30 2024-04-25T10:03:54.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.654+0000 7f2b8bfff640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b7806b4d0 0x7f2b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.654+0000 7f2b8bfff640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b7806b4d0 0x7f2b7806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c034000 tx=0x7f2b7c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.658+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b80057e40 con 0x7f2b8c072d30 2024-04-25T10:03:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.894+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2b8c06b070 con 0x7f2b8c072d30 2024-04-25T10:03:54.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.895+0000 7f2b88ff9640 1 -- 172.21.15.148:0/3761571420 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1199 (secure 0 0 0) 0x7f2b8005bee0 con 0x7f2b8c072d30 2024-04-25T10:03:54.900 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":595,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":145,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":427261952,"bytes_avail":772666851328,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":84,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9861}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:03:39.135397+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b7806b4d0 msgr2=0x7f2b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b7806b4d0 0x7f2b7806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c034000 tx=0x7f2b7c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 msgr2=0x7f2b8c0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 0x7f2b8c0ca930 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000c750 tx=0x7f2b8000cb50 comp rx=0 tx=0).stop 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 shutdown_connections 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2b7806b4d0 0x7f2b7806d990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8c15b9c0 0x7f2b8c14dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b8c072d30 0x7f2b8c0ca930 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 --2- 172.21.15.148:0/3761571420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c06abe0 0x7f2b8c0c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 >> 172.21.15.148:0/3761571420 conn(0x7f2b8c0b2370 msgr2=0x7f2b8c0d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 shutdown_connections 2024-04-25T10:03:54.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:54.897+0000 7f2b927b7640 1 -- 172.21.15.148:0/3761571420 wait complete. 2024-04-25T10:03:54.913 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:03:54.913 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.005+0000 7f061565a640 1 Processor -- start 2024-04-25T10:03:55.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.005+0000 7f061565a640 1 -- start start 2024-04-25T10:03:55.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f061565a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f061015db60 0x7f06100ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f061565a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06100b5f00 0x7f06100d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f061565a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f06100d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f061565a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100ce930 con 0x7f06100b5f00 2024-04-25T10:03:55.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f061565a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100b6450 con 0x7f061015db60 2024-04-25T10:03:55.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f061565a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100cec00 con 0x7f061015df40 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f060e7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06100b5f00 0x7f06100d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f060e7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06100b5f00 0x7f06100d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33678/0 (socket says 172.21.15.148:33678) 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f060e7fc640 1 -- 172.21.15.148:0/3258235348 learned_addr learned my addr 172.21.15.148:0/3258235348 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.006+0000 7f060effd640 1 --2- 172.21.15.148:0/3258235348 >> v2:172.21.15.188:3300/0 conn(0x7f061015db60 0x7f06100ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060f7fe640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f06100d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060e7fc640 1 -- 172.21.15.148:0/3258235348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 msgr2=0x7f06100d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060e7fc640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f06100d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060e7fc640 1 -- 172.21.15.148:0/3258235348 >> v2:172.21.15.188:3300/0 conn(0x7f061015db60 msgr2=0x7f06100ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060e7fc640 1 --2- 172.21.15.148:0/3258235348 >> v2:172.21.15.188:3300/0 conn(0x7f061015db60 0x7f06100ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060e7fc640 1 -- 172.21.15.148:0/3258235348 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100ceee0 con 0x7f06100b5f00 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060f7fe640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f06100d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060e7fc640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100d0400 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f0610062440 tx=0x7f060402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103252b1eedd4fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060dffb640 1 -- 172.21.15.148:0/3258235348 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604037070 con 0x7f06100b5f00 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060dffb640 1 -- 172.21.15.148:0/3258235348 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604032440 con 0x7f06100b5f00 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.007+0000 7f060dffb640 1 -- 172.21.15.148:0/3258235348 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604032810 con 0x7f06100b5f00 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 -- 172.21.15.148:0/3258235348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 msgr2=0x7f06100d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100d0400 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f0610062440 tx=0x7f060402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 -- 172.21.15.148:0/3258235348 shutdown_connections 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f06100d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 --2- 172.21.15.148:0/3258235348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100d0400 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 --2- 172.21.15.148:0/3258235348 >> v2:172.21.15.188:3300/0 conn(0x7f061015db60 0x7f06100ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 -- 172.21.15.148:0/3258235348 >> 172.21.15.148:0/3258235348 conn(0x7f0610061600 msgr2=0x7f0610061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 -- 172.21.15.148:0/3258235348 shutdown_connections 2024-04-25T10:03:55.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.008+0000 7f061565a640 1 -- 172.21.15.148:0/3258235348 wait complete. 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.009+0000 7f061565a640 1 Processor -- start 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.009+0000 7f061565a640 1 -- start start 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f061565a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f061565a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f061015db60 0x7f061015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f061565a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f0610151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f061565a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610161950 con 0x7f06100b5f00 2024-04-25T10:03:55.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f061565a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100d1d50 con 0x7f061015db60 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f061565a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610161ad0 con 0x7f061015df40 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33694/0 (socket says 172.21.15.148:33694) 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 -- 172.21.15.148:0/1290911836 learned_addr learned my addr 172.21.15.148:0/1290911836 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060e7fc640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f061015db60 0x7f061015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060f7fe640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f0610151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 -- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 msgr2=0x7f0610151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f0610151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 -- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f061015db60 msgr2=0x7f061015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f061015db60 0x7f061015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060effd640 1 -- 172.21.15.148:0/1290911836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100c7b20 con 0x7f06100b5f00 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.010+0000 7f060e7fc640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f061015db60 0x7f061015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.011+0000 7f060effd640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100c8bc0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f0600007e60 tx=0x7f060000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:55.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.011+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600014620 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.011+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600019070 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.011+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100c7e00 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.011+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600014a00 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.011+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100c83c0 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.012+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0610073270 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.012+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0600014ba0 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.013+0000 7f05f77fe640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05fc06b4d0 0x7f05fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.013+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f060008ac70 con 0x7f06100b5f00 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.013+0000 7f060e7fc640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05fc06b4d0 0x7f05fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.013+0000 7f060e7fc640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05fc06b4d0 0x7f05fc06d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0604034000 tx=0x7f060402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:55.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.017+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0600059960 con 0x7f06100b5f00 2024-04-25T10:03:55.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.185+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06100c8840 con 0x7f05fc06b4d0 2024-04-25T10:03:55.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.191+0000 7f05f77fe640 1 -- 172.21.15.148:0/1290911836 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251478 (secure 0 0 0) 0x7f06100c8840 con 0x7f05fc06b4d0 2024-04-25T10:03:55.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:55.201 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:03:55.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05fc06b4d0 msgr2=0x7f05fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05fc06b4d0 0x7f05fc06d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0604034000 tx=0x7f060402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:55.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 msgr2=0x7f06100c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100c8bc0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f0600007e60 tx=0x7f060000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 shutdown_connections 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05fc06b4d0 0x7f05fc06d990 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015df40 0x7f0610151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f061015db60 0x7f061015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 --2- 172.21.15.148:0/1290911836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b5f00 0x7f06100c8bc0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 >> 172.21.15.148:0/1290911836 conn(0x7f0610061600 msgr2=0x7f06100b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 shutdown_connections 2024-04-25T10:03:55.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:55.194+0000 7f061565a640 1 -- 172.21.15.148:0/1290911836 wait complete. 2024-04-25T10:03:55.211 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-04-25T10:03:55.138022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417576,"kb_used_data":15400,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754557144,"statfs":{"total":773094113280,"available":772666515456,"internally_reserved":0,"allocated":15769600,"data_stored":7967953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001012"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606928+0000","last_change":"2024-04-25T10:03:50.905492+0000","last_active":"2024-04-25T10:03:52.606928+0000","last_peered":"2024-04-25T10:03:52.606928+0000","last_clean":"2024-04-25T10:03:52.606928+0000","last_became_active":"2024-04-25T10:03:50.905219+0000","last_became_peered":"2024-04-25T10:03:50.905219+0000","last_unstale":"2024-04-25T10:03:52.606928+0000","last_undegraded":"2024-04-25T10:03:52.606928+0000","last_fullsized":"2024-04-25T10:03:52.606928+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:50.515584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883264+0000","last_change":"2024-04-25T10:03:50.895420+0000","last_active":"2024-04-25T10:03:51.883264+0000","last_peered":"2024-04-25T10:03:51.883264+0000","last_clean":"2024-04-25T10:03:51.883264+0000","last_became_active":"2024-04-25T10:03:50.895108+0000","last_became_peered":"2024-04-25T10:03:50.895108+0000","last_unstale":"2024-04-25T10:03:51.883264+0000","last_undegraded":"2024-04-25T10:03:51.883264+0000","last_fullsized":"2024-04-25T10:03:51.883264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:50.515584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883377+0000","last_change":"2024-04-25T10:03:50.893021+0000","last_active":"2024-04-25T10:03:51.883377+0000","last_peered":"2024-04-25T10:03:51.883377+0000","last_clean":"2024-04-25T10:03:51.883377+0000","last_became_active":"2024-04-25T10:03:50.892759+0000","last_became_peered":"2024-04-25T10:03:50.892759+0000","last_unstale":"2024-04-25T10:03:51.883377+0000","last_undegraded":"2024-04-25T10:03:51.883377+0000","last_fullsized":"2024-04-25T10:03:51.883377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606272+0000","last_change":"2024-04-25T10:03:50.899244+0000","last_active":"2024-04-25T10:03:52.606272+0000","last_peered":"2024-04-25T10:03:52.606272+0000","last_clean":"2024-04-25T10:03:52.606272+0000","last_became_active":"2024-04-25T10:03:50.898961+0000","last_became_peered":"2024-04-25T10:03:50.898961+0000","last_unstale":"2024-04-25T10:03:52.606272+0000","last_undegraded":"2024-04-25T10:03:52.606272+0000","last_fullsized":"2024-04-25T10:03:52.606272+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:41.231201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886184+0000","last_change":"2024-04-25T10:03:49.886184+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886184+0000","last_undegraded":"2024-04-25T10:03:49.886184+0000","last_fullsized":"2024-04-25T10:03:49.886184+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886898+0000","last_change":"2024-04-25T10:03:49.886898+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886898+0000","last_undegraded":"2024-04-25T10:03:49.886898+0000","last_fullsized":"2024-04-25T10:03:49.886898+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607444+0000","last_change":"2024-04-25T10:03:50.897311+0000","last_active":"2024-04-25T10:03:52.607444+0000","last_peered":"2024-04-25T10:03:52.607444+0000","last_clean":"2024-04-25T10:03:52.607444+0000","last_became_active":"2024-04-25T10:03:50.893752+0000","last_became_peered":"2024-04-25T10:03:50.893752+0000","last_unstale":"2024-04-25T10:03:52.607444+0000","last_undegraded":"2024-04-25T10:03:52.607444+0000","last_fullsized":"2024-04-25T10:03:52.607444+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:11.198548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886753+0000","last_change":"2024-04-25T10:03:50.903258+0000","last_active":"2024-04-25T10:03:51.886753+0000","last_peered":"2024-04-25T10:03:51.886753+0000","last_clean":"2024-04-25T10:03:51.886753+0000","last_became_active":"2024-04-25T10:03:50.900168+0000","last_became_peered":"2024-04-25T10:03:50.900168+0000","last_unstale":"2024-04-25T10:03:51.886753+0000","last_undegraded":"2024-04-25T10:03:51.886753+0000","last_fullsized":"2024-04-25T10:03:51.886753+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:23.357921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886447+0000","last_change":"2024-04-25T10:03:49.886447+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886447+0000","last_undegraded":"2024-04-25T10:03:49.886447+0000","last_fullsized":"2024-04-25T10:03:49.886447+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887235+0000","last_change":"2024-04-25T10:03:49.887235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887235+0000","last_undegraded":"2024-04-25T10:03:49.887235+0000","last_fullsized":"2024-04-25T10:03:49.887235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606271+0000","last_change":"2024-04-25T10:03:50.902053+0000","last_active":"2024-04-25T10:03:52.606271+0000","last_peered":"2024-04-25T10:03:52.606271+0000","last_clean":"2024-04-25T10:03:52.606271+0000","last_became_active":"2024-04-25T10:03:50.901690+0000","last_became_peered":"2024-04-25T10:03:50.901690+0000","last_unstale":"2024-04-25T10:03:52.606271+0000","last_undegraded":"2024-04-25T10:03:52.606271+0000","last_fullsized":"2024-04-25T10:03:52.606271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:17.479698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606276+0000","last_change":"2024-04-25T10:03:50.907519+0000","last_active":"2024-04-25T10:03:52.606276+0000","last_peered":"2024-04-25T10:03:52.606276+0000","last_clean":"2024-04-25T10:03:52.606276+0000","last_became_active":"2024-04-25T10:03:50.907256+0000","last_became_peered":"2024-04-25T10:03:50.907256+0000","last_unstale":"2024-04-25T10:03:52.606276+0000","last_undegraded":"2024-04-25T10:03:52.606276+0000","last_fullsized":"2024-04-25T10:03:52.606276+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:46.062004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889706+0000","last_change":"2024-04-25T10:03:49.889706+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889706+0000","last_undegraded":"2024-04-25T10:03:49.889706+0000","last_fullsized":"2024-04-25T10:03:49.889706+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606406+0000","last_change":"2024-04-25T10:03:50.909162+0000","last_active":"2024-04-25T10:03:52.606406+0000","last_peered":"2024-04-25T10:03:52.606406+0000","last_clean":"2024-04-25T10:03:52.606406+0000","last_became_active":"2024-04-25T10:03:50.908890+0000","last_became_peered":"2024-04-25T10:03:50.908890+0000","last_unstale":"2024-04-25T10:03:52.606406+0000","last_undegraded":"2024-04-25T10:03:52.606406+0000","last_fullsized":"2024-04-25T10:03:52.606406+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:11.681942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883264+0000","last_change":"2024-04-25T10:03:50.900082+0000","last_active":"2024-04-25T10:03:51.883264+0000","last_peered":"2024-04-25T10:03:51.883264+0000","last_clean":"2024-04-25T10:03:51.883264+0000","last_became_active":"2024-04-25T10:03:50.899803+0000","last_became_peered":"2024-04-25T10:03:50.899803+0000","last_unstale":"2024-04-25T10:03:51.883264+0000","last_undegraded":"2024-04-25T10:03:51.883264+0000","last_fullsized":"2024-04-25T10:03:51.883264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:53.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.889684+0000","last_change":"2024-04-25T10:03:49.889684+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.889684+0000","last_undegraded":"2024-04-25T10:03:49.889684+0000","last_fullsized":"2024-04-25T10:03:49.889684+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607095+0000","last_change":"2024-04-25T10:03:50.900657+0000","last_active":"2024-04-25T10:03:52.607095+0000","last_peered":"2024-04-25T10:03:52.607095+0000","last_clean":"2024-04-25T10:03:52.607095+0000","last_became_active":"2024-04-25T10:03:50.900355+0000","last_became_peered":"2024-04-25T10:03:50.900355+0000","last_unstale":"2024-04-25T10:03:52.607095+0000","last_undegraded":"2024-04-25T10:03:52.607095+0000","last_fullsized":"2024-04-25T10:03:52.607095+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.949477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605782+0000","last_change":"2024-04-25T10:03:50.896062+0000","last_active":"2024-04-25T10:03:52.605782+0000","last_peered":"2024-04-25T10:03:52.605782+0000","last_clean":"2024-04-25T10:03:52.605782+0000","last_became_active":"2024-04-25T10:03:50.895590+0000","last_became_peered":"2024-04-25T10:03:50.895590+0000","last_unstale":"2024-04-25T10:03:52.605782+0000","last_undegraded":"2024-04-25T10:03:52.605782+0000","last_fullsized":"2024-04-25T10:03:52.605782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:45.778284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606831+0000","last_change":"2024-04-25T10:03:50.897717+0000","last_active":"2024-04-25T10:03:52.606831+0000","last_peered":"2024-04-25T10:03:52.606831+0000","last_clean":"2024-04-25T10:03:52.606831+0000","last_became_active":"2024-04-25T10:03:50.897142+0000","last_became_peered":"2024-04-25T10:03:50.897142+0000","last_unstale":"2024-04-25T10:03:52.606831+0000","last_undegraded":"2024-04-25T10:03:52.606831+0000","last_fullsized":"2024-04-25T10:03:52.606831+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:13.082455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605650+0000","last_change":"2024-04-25T10:03:50.908907+0000","last_active":"2024-04-25T10:03:52.605650+0000","last_peered":"2024-04-25T10:03:52.605650+0000","last_clean":"2024-04-25T10:03:52.605650+0000","last_became_active":"2024-04-25T10:03:50.908266+0000","last_became_peered":"2024-04-25T10:03:50.908266+0000","last_unstale":"2024-04-25T10:03:52.605650+0000","last_undegraded":"2024-04-25T10:03:52.605650+0000","last_fullsized":"2024-04-25T10:03:52.605650+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:08.033891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886836+0000","last_change":"2024-04-25T10:03:49.886836+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886836+0000","last_undegraded":"2024-04-25T10:03:49.886836+0000","last_fullsized":"2024-04-25T10:03:49.886836+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606801+0000","last_change":"2024-04-25T10:03:50.898017+0000","last_active":"2024-04-25T10:03:52.606801+0000","last_peered":"2024-04-25T10:03:52.606801+0000","last_clean":"2024-04-25T10:03:52.606801+0000","last_became_active":"2024-04-25T10:03:50.897747+0000","last_became_peered":"2024-04-25T10:03:50.897747+0000","last_unstale":"2024-04-25T10:03:52.606801+0000","last_undegraded":"2024-04-25T10:03:52.606801+0000","last_fullsized":"2024-04-25T10:03:52.606801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886714+0000","last_change":"2024-04-25T10:03:50.897929+0000","last_active":"2024-04-25T10:03:51.886714+0000","last_peered":"2024-04-25T10:03:51.886714+0000","last_clean":"2024-04-25T10:03:51.886714+0000","last_became_active":"2024-04-25T10:03:50.895163+0000","last_became_peered":"2024-04-25T10:03:50.895163+0000","last_unstale":"2024-04-25T10:03:51.886714+0000","last_undegraded":"2024-04-25T10:03:51.886714+0000","last_fullsized":"2024-04-25T10:03:51.886714+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:51.005859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606320+0000","last_change":"2024-04-25T10:03:50.905962+0000","last_active":"2024-04-25T10:03:52.606320+0000","last_peered":"2024-04-25T10:03:52.606320+0000","last_clean":"2024-04-25T10:03:52.606320+0000","last_became_active":"2024-04-25T10:03:50.905684+0000","last_became_peered":"2024-04-25T10:03:50.905684+0000","last_unstale":"2024-04-25T10:03:52.606320+0000","last_undegraded":"2024-04-25T10:03:52.606320+0000","last_fullsized":"2024-04-25T10:03:52.606320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.222552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607043+0000","last_change":"2024-04-25T10:03:50.896576+0000","last_active":"2024-04-25T10:03:52.607043+0000","last_peered":"2024-04-25T10:03:52.607043+0000","last_clean":"2024-04-25T10:03:52.607043+0000","last_became_active":"2024-04-25T10:03:50.893344+0000","last_became_peered":"2024-04-25T10:03:50.893344+0000","last_unstale":"2024-04-25T10:03:52.607043+0000","last_undegraded":"2024-04-25T10:03:52.607043+0000","last_fullsized":"2024-04-25T10:03:52.607043+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.716636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607042+0000","last_change":"2024-04-25T10:03:50.900898+0000","last_active":"2024-04-25T10:03:52.607042+0000","last_peered":"2024-04-25T10:03:52.607042+0000","last_clean":"2024-04-25T10:03:52.607042+0000","last_became_active":"2024-04-25T10:03:50.898774+0000","last_became_peered":"2024-04-25T10:03:50.898774+0000","last_unstale":"2024-04-25T10:03:52.607042+0000","last_undegraded":"2024-04-25T10:03:52.607042+0000","last_fullsized":"2024-04-25T10:03:52.607042+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:36.335317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885978+0000","last_change":"2024-04-25T10:03:49.885978+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885978+0000","last_undegraded":"2024-04-25T10:03:49.885978+0000","last_fullsized":"2024-04-25T10:03:49.885978+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606687+0000","last_change":"2024-04-25T10:03:50.897308+0000","last_active":"2024-04-25T10:03:52.606687+0000","last_peered":"2024-04-25T10:03:52.606687+0000","last_clean":"2024-04-25T10:03:52.606687+0000","last_became_active":"2024-04-25T10:03:50.896847+0000","last_became_peered":"2024-04-25T10:03:50.896847+0000","last_unstale":"2024-04-25T10:03:52.606687+0000","last_undegraded":"2024-04-25T10:03:52.606687+0000","last_fullsized":"2024-04-25T10:03:52.606687+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.338275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887497+0000","last_change":"2024-04-25T10:03:50.902013+0000","last_active":"2024-04-25T10:03:51.887497+0000","last_peered":"2024-04-25T10:03:51.887497+0000","last_clean":"2024-04-25T10:03:51.887497+0000","last_became_active":"2024-04-25T10:03:50.901755+0000","last_became_peered":"2024-04-25T10:03:50.901755+0000","last_unstale":"2024-04-25T10:03:51.887497+0000","last_undegraded":"2024-04-25T10:03:51.887497+0000","last_fullsized":"2024-04-25T10:03:51.887497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.949477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606584+0000","last_change":"2024-04-25T10:03:50.903762+0000","last_active":"2024-04-25T10:03:52.606584+0000","last_peered":"2024-04-25T10:03:52.606584+0000","last_clean":"2024-04-25T10:03:52.606584+0000","last_became_active":"2024-04-25T10:03:50.902357+0000","last_became_peered":"2024-04-25T10:03:50.902357+0000","last_unstale":"2024-04-25T10:03:52.606584+0000","last_undegraded":"2024-04-25T10:03:52.606584+0000","last_fullsized":"2024-04-25T10:03:52.606584+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:21.956097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885408+0000","last_change":"2024-04-25T10:03:49.885408+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885408+0000","last_undegraded":"2024-04-25T10:03:49.885408+0000","last_fullsized":"2024-04-25T10:03:49.885408+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886707+0000","last_change":"2024-04-25T10:03:50.898505+0000","last_active":"2024-04-25T10:03:51.886707+0000","last_peered":"2024-04-25T10:03:51.886707+0000","last_clean":"2024-04-25T10:03:51.886707+0000","last_became_active":"2024-04-25T10:03:50.898162+0000","last_became_peered":"2024-04-25T10:03:50.898162+0000","last_unstale":"2024-04-25T10:03:51.886707+0000","last_undegraded":"2024-04-25T10:03:51.886707+0000","last_fullsized":"2024-04-25T10:03:51.886707+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.679413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883264+0000","last_change":"2024-04-25T10:03:50.890540+0000","last_active":"2024-04-25T10:03:51.883264+0000","last_peered":"2024-04-25T10:03:51.883264+0000","last_clean":"2024-04-25T10:03:51.883264+0000","last_became_active":"2024-04-25T10:03:50.889486+0000","last_became_peered":"2024-04-25T10:03:50.889486+0000","last_unstale":"2024-04-25T10:03:51.883264+0000","last_undegraded":"2024-04-25T10:03:51.883264+0000","last_fullsized":"2024-04-25T10:03:51.883264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893210+0000","last_change":"2024-04-25T10:03:49.891438+0000","last_active":"2024-04-25T10:03:49.893210+0000","last_peered":"2024-04-25T10:03:49.893210+0000","last_clean":"2024-04-25T10:03:49.893210+0000","last_became_active":"2024-04-25T10:03:49.890515+0000","last_became_peered":"2024-04-25T10:03:49.890515+0000","last_unstale":"2024-04-25T10:03:49.893210+0000","last_undegraded":"2024-04-25T10:03:49.893210+0000","last_fullsized":"2024-04-25T10:03:49.893210+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883365+0000","last_change":"2024-04-25T10:03:50.903692+0000","last_active":"2024-04-25T10:03:51.883365+0000","last_peered":"2024-04-25T10:03:51.883365+0000","last_clean":"2024-04-25T10:03:51.883365+0000","last_became_active":"2024-04-25T10:03:50.903240+0000","last_became_peered":"2024-04-25T10:03:50.903240+0000","last_unstale":"2024-04-25T10:03:51.883365+0000","last_undegraded":"2024-04-25T10:03:51.883365+0000","last_fullsized":"2024-04-25T10:03:51.883365+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.716636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606246+0000","last_change":"2024-04-25T10:03:49.898107+0000","last_active":"2024-04-25T10:03:52.606246+0000","last_peered":"2024-04-25T10:03:52.606246+0000","last_clean":"2024-04-25T10:03:52.606246+0000","last_became_active":"2024-04-25T10:03:49.897243+0000","last_became_peered":"2024-04-25T10:03:49.897243+0000","last_unstale":"2024-04-25T10:03:52.606246+0000","last_undegraded":"2024-04-25T10:03:52.606246+0000","last_fullsized":"2024-04-25T10:03:52.606246+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:23.747311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887201+0000","last_change":"2024-04-25T10:03:50.901337+0000","last_active":"2024-04-25T10:03:51.887201+0000","last_peered":"2024-04-25T10:03:51.887201+0000","last_clean":"2024-04-25T10:03:51.887201+0000","last_became_active":"2024-04-25T10:03:50.900872+0000","last_became_peered":"2024-04-25T10:03:50.900872+0000","last_unstale":"2024-04-25T10:03:51.887201+0000","last_undegraded":"2024-04-25T10:03:51.887201+0000","last_fullsized":"2024-04-25T10:03:51.887201+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:11.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607082+0000","last_change":"2024-04-25T10:03:49.896386+0000","last_active":"2024-04-25T10:03:52.607082+0000","last_peered":"2024-04-25T10:03:52.607082+0000","last_clean":"2024-04-25T10:03:52.607082+0000","last_became_active":"2024-04-25T10:03:49.895902+0000","last_became_peered":"2024-04-25T10:03:49.895902+0000","last_unstale":"2024-04-25T10:03:52.607082+0000","last_undegraded":"2024-04-25T10:03:52.607082+0000","last_fullsized":"2024-04-25T10:03:52.607082+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:26.944270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606419+0000","last_change":"2024-04-25T10:03:50.904288+0000","last_active":"2024-04-25T10:03:52.606419+0000","last_peered":"2024-04-25T10:03:52.606419+0000","last_clean":"2024-04-25T10:03:52.606419+0000","last_became_active":"2024-04-25T10:03:50.903879+0000","last_became_peered":"2024-04-25T10:03:50.903879+0000","last_unstale":"2024-04-25T10:03:52.606419+0000","last_undegraded":"2024-04-25T10:03:52.606419+0000","last_fullsized":"2024-04-25T10:03:52.606419+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883308+0000","last_change":"2024-04-25T10:03:49.896831+0000","last_active":"2024-04-25T10:03:51.883308+0000","last_peered":"2024-04-25T10:03:51.883308+0000","last_clean":"2024-04-25T10:03:51.883308+0000","last_became_active":"2024-04-25T10:03:49.896566+0000","last_became_peered":"2024-04-25T10:03:49.896566+0000","last_unstale":"2024-04-25T10:03:51.883308+0000","last_undegraded":"2024-04-25T10:03:51.883308+0000","last_fullsized":"2024-04-25T10:03:51.883308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605454+0000","last_change":"2024-04-25T10:03:50.898122+0000","last_active":"2024-04-25T10:03:52.605454+0000","last_peered":"2024-04-25T10:03:52.605454+0000","last_clean":"2024-04-25T10:03:52.605454+0000","last_became_active":"2024-04-25T10:03:50.897828+0000","last_became_peered":"2024-04-25T10:03:50.897828+0000","last_unstale":"2024-04-25T10:03:52.605454+0000","last_undegraded":"2024-04-25T10:03:52.605454+0000","last_fullsized":"2024-04-25T10:03:52.605454+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:53.111183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887450+0000","last_change":"2024-04-25T10:03:49.894730+0000","last_active":"2024-04-25T10:03:51.887450+0000","last_peered":"2024-04-25T10:03:51.887450+0000","last_clean":"2024-04-25T10:03:51.887450+0000","last_became_active":"2024-04-25T10:03:49.894297+0000","last_became_peered":"2024-04-25T10:03:49.894297+0000","last_unstale":"2024-04-25T10:03:51.887450+0000","last_undegraded":"2024-04-25T10:03:51.887450+0000","last_fullsized":"2024-04-25T10:03:51.887450+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:01.257780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606703+0000","last_change":"2024-04-25T10:03:50.900134+0000","last_active":"2024-04-25T10:03:52.606703+0000","last_peered":"2024-04-25T10:03:52.606703+0000","last_clean":"2024-04-25T10:03:52.606703+0000","last_became_active":"2024-04-25T10:03:50.899842+0000","last_became_peered":"2024-04-25T10:03:50.899842+0000","last_unstale":"2024-04-25T10:03:52.606703+0000","last_undegraded":"2024-04-25T10:03:52.606703+0000","last_fullsized":"2024-04-25T10:03:52.606703+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:11.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.902592+0000","last_change":"2024-04-25T10:03:49.902295+0000","last_active":"2024-04-25T10:03:49.902592+0000","last_peered":"2024-04-25T10:03:49.902592+0000","last_clean":"2024-04-25T10:03:49.902592+0000","last_became_active":"2024-04-25T10:03:49.902003+0000","last_became_peered":"2024-04-25T10:03:49.902003+0000","last_unstale":"2024-04-25T10:03:49.902592+0000","last_undegraded":"2024-04-25T10:03:49.902592+0000","last_fullsized":"2024-04-25T10:03:49.902592+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:10.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883081+0000","last_change":"2024-04-25T10:03:50.890023+0000","last_active":"2024-04-25T10:03:51.883081+0000","last_peered":"2024-04-25T10:03:51.883081+0000","last_clean":"2024-04-25T10:03:51.883081+0000","last_became_active":"2024-04-25T10:03:50.889713+0000","last_became_peered":"2024-04-25T10:03:50.889713+0000","last_unstale":"2024-04-25T10:03:51.883081+0000","last_undegraded":"2024-04-25T10:03:51.883081+0000","last_fullsized":"2024-04-25T10:03:51.883081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.949477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607624+0000","last_change":"2024-04-25T10:03:49.899529+0000","last_active":"2024-04-25T10:03:52.607624+0000","last_peered":"2024-04-25T10:03:52.607624+0000","last_clean":"2024-04-25T10:03:52.607624+0000","last_became_active":"2024-04-25T10:03:49.896941+0000","last_became_peered":"2024-04-25T10:03:49.896941+0000","last_unstale":"2024-04-25T10:03:52.607624+0000","last_undegraded":"2024-04-25T10:03:52.607624+0000","last_fullsized":"2024-04-25T10:03:52.607624+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:40.225994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607819+0000","last_change":"2024-04-25T10:03:50.908162+0000","last_active":"2024-04-25T10:03:52.607819+0000","last_peered":"2024-04-25T10:03:52.607819+0000","last_clean":"2024-04-25T10:03:52.607819+0000","last_became_active":"2024-04-25T10:03:50.907834+0000","last_became_peered":"2024-04-25T10:03:50.907834+0000","last_unstale":"2024-04-25T10:03:52.607819+0000","last_undegraded":"2024-04-25T10:03:52.607819+0000","last_fullsized":"2024-04-25T10:03:52.607819+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:54.432830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.894253+0000","last_change":"2024-04-25T10:03:49.893772+0000","last_active":"2024-04-25T10:03:49.894253+0000","last_peered":"2024-04-25T10:03:49.894253+0000","last_clean":"2024-04-25T10:03:49.894253+0000","last_became_active":"2024-04-25T10:03:49.893508+0000","last_became_peered":"2024-04-25T10:03:49.893508+0000","last_unstale":"2024-04-25T10:03:49.894253+0000","last_undegraded":"2024-04-25T10:03:49.894253+0000","last_fullsized":"2024-04-25T10:03:49.894253+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.861808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887202+0000","last_change":"2024-04-25T10:03:50.891822+0000","last_active":"2024-04-25T10:03:51.887202+0000","last_peered":"2024-04-25T10:03:51.887202+0000","last_clean":"2024-04-25T10:03:51.887202+0000","last_became_active":"2024-04-25T10:03:50.891513+0000","last_became_peered":"2024-04-25T10:03:50.891513+0000","last_unstale":"2024-04-25T10:03:51.887202+0000","last_undegraded":"2024-04-25T10:03:51.887202+0000","last_fullsized":"2024-04-25T10:03:51.887202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.867015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606594+0000","last_change":"2024-04-25T10:03:49.896956+0000","last_active":"2024-04-25T10:03:52.606594+0000","last_peered":"2024-04-25T10:03:52.606594+0000","last_clean":"2024-04-25T10:03:52.606594+0000","last_became_active":"2024-04-25T10:03:49.896167+0000","last_became_peered":"2024-04-25T10:03:49.896167+0000","last_unstale":"2024-04-25T10:03:52.606594+0000","last_undegraded":"2024-04-25T10:03:52.606594+0000","last_fullsized":"2024-04-25T10:03:52.606594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:10.861239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606427+0000","last_change":"2024-04-25T10:03:50.900737+0000","last_active":"2024-04-25T10:03:52.606427+0000","last_peered":"2024-04-25T10:03:52.606427+0000","last_clean":"2024-04-25T10:03:52.606427+0000","last_became_active":"2024-04-25T10:03:50.900455+0000","last_became_peered":"2024-04-25T10:03:50.900455+0000","last_unstale":"2024-04-25T10:03:52.606427+0000","last_undegraded":"2024-04-25T10:03:52.606427+0000","last_fullsized":"2024-04-25T10:03:52.606427+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:42.273961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607319+0000","last_change":"2024-04-25T10:03:49.898465+0000","last_active":"2024-04-25T10:03:52.607319+0000","last_peered":"2024-04-25T10:03:52.607319+0000","last_clean":"2024-04-25T10:03:52.607319+0000","last_became_active":"2024-04-25T10:03:49.898198+0000","last_became_peered":"2024-04-25T10:03:49.898198+0000","last_unstale":"2024-04-25T10:03:52.607319+0000","last_undegraded":"2024-04-25T10:03:52.607319+0000","last_fullsized":"2024-04-25T10:03:52.607319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:50.000651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605928+0000","last_change":"2024-04-25T10:03:50.907140+0000","last_active":"2024-04-25T10:03:52.605928+0000","last_peered":"2024-04-25T10:03:52.605928+0000","last_clean":"2024-04-25T10:03:52.605928+0000","last_became_active":"2024-04-25T10:03:50.906824+0000","last_became_peered":"2024-04-25T10:03:50.906824+0000","last_unstale":"2024-04-25T10:03:52.605928+0000","last_undegraded":"2024-04-25T10:03:52.605928+0000","last_fullsized":"2024-04-25T10:03:52.605928+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:08.352576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"144'3","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606518+0000","last_change":"2024-04-25T10:03:49.897101+0000","last_active":"2024-04-25T10:03:52.606518+0000","last_peered":"2024-04-25T10:03:52.606518+0000","last_clean":"2024-04-25T10:03:52.606518+0000","last_became_active":"2024-04-25T10:03:49.896790+0000","last_became_peered":"2024-04-25T10:03:49.896790+0000","last_unstale":"2024-04-25T10:03:52.606518+0000","last_undegraded":"2024-04-25T10:03:52.606518+0000","last_fullsized":"2024-04-25T10:03:52.606518+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:27.534285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886894+0000","last_change":"2024-04-25T10:03:50.892831+0000","last_active":"2024-04-25T10:03:51.886894+0000","last_peered":"2024-04-25T10:03:51.886894+0000","last_clean":"2024-04-25T10:03:51.886894+0000","last_became_active":"2024-04-25T10:03:50.892584+0000","last_became_peered":"2024-04-25T10:03:50.892584+0000","last_unstale":"2024-04-25T10:03:51.886894+0000","last_undegraded":"2024-04-25T10:03:51.886894+0000","last_fullsized":"2024-04-25T10:03:51.886894+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:41.231201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.893179+0000","last_change":"2024-04-25T10:03:49.892221+0000","last_active":"2024-04-25T10:03:49.893179+0000","last_peered":"2024-04-25T10:03:49.893179+0000","last_clean":"2024-04-25T10:03:49.893179+0000","last_became_active":"2024-04-25T10:03:49.891975+0000","last_became_peered":"2024-04-25T10:03:49.891975+0000","last_unstale":"2024-04-25T10:03:49.893179+0000","last_undegraded":"2024-04-25T10:03:49.893179+0000","last_fullsized":"2024-04-25T10:03:49.893179+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:10.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883081+0000","last_change":"2024-04-25T10:03:50.903374+0000","last_active":"2024-04-25T10:03:51.883081+0000","last_peered":"2024-04-25T10:03:51.883081+0000","last_clean":"2024-04-25T10:03:51.883081+0000","last_became_active":"2024-04-25T10:03:50.902904+0000","last_became_peered":"2024-04-25T10:03:50.902904+0000","last_unstale":"2024-04-25T10:03:51.883081+0000","last_undegraded":"2024-04-25T10:03:51.883081+0000","last_fullsized":"2024-04-25T10:03:51.883081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.752518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606349+0000","last_change":"2024-04-25T10:03:49.901671+0000","last_active":"2024-04-25T10:03:52.606349+0000","last_peered":"2024-04-25T10:03:52.606349+0000","last_clean":"2024-04-25T10:03:52.606349+0000","last_became_active":"2024-04-25T10:03:49.901399+0000","last_became_peered":"2024-04-25T10:03:49.901399+0000","last_unstale":"2024-04-25T10:03:52.606349+0000","last_undegraded":"2024-04-25T10:03:52.606349+0000","last_fullsized":"2024-04-25T10:03:52.606349+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.886235+0000","last_change":"2024-04-25T10:03:49.886235+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.886235+0000","last_undegraded":"2024-04-25T10:03:49.886235+0000","last_fullsized":"2024-04-25T10:03:49.886235+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606806+0000","last_change":"2024-04-25T10:03:49.900536+0000","last_active":"2024-04-25T10:03:52.606806+0000","last_peered":"2024-04-25T10:03:52.606806+0000","last_clean":"2024-04-25T10:03:52.606806+0000","last_became_active":"2024-04-25T10:03:49.900264+0000","last_became_peered":"2024-04-25T10:03:49.900264+0000","last_unstale":"2024-04-25T10:03:52.606806+0000","last_undegraded":"2024-04-25T10:03:52.606806+0000","last_fullsized":"2024-04-25T10:03:52.606806+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883110+0000","last_change":"2024-04-25T10:03:50.905600+0000","last_active":"2024-04-25T10:03:51.883110+0000","last_peered":"2024-04-25T10:03:51.883110+0000","last_clean":"2024-04-25T10:03:51.883110+0000","last_became_active":"2024-04-25T10:03:50.905258+0000","last_became_peered":"2024-04-25T10:03:50.905258+0000","last_unstale":"2024-04-25T10:03:51.883110+0000","last_undegraded":"2024-04-25T10:03:51.883110+0000","last_fullsized":"2024-04-25T10:03:51.883110+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:03.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"144'8","reported_seq":22,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605942+0000","last_change":"2024-04-25T10:03:49.897285+0000","last_active":"2024-04-25T10:03:52.605942+0000","last_peered":"2024-04-25T10:03:52.605942+0000","last_clean":"2024-04-25T10:03:52.605942+0000","last_became_active":"2024-04-25T10:03:49.896833+0000","last_became_peered":"2024-04-25T10:03:49.896833+0000","last_unstale":"2024-04-25T10:03:52.605942+0000","last_undegraded":"2024-04-25T10:03:52.605942+0000","last_fullsized":"2024-04-25T10:03:52.605942+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:40.674205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607565+0000","last_change":"2024-04-25T10:03:50.903914+0000","last_active":"2024-04-25T10:03:52.607565+0000","last_peered":"2024-04-25T10:03:52.607565+0000","last_clean":"2024-04-25T10:03:52.607565+0000","last_became_active":"2024-04-25T10:03:50.903629+0000","last_became_peered":"2024-04-25T10:03:50.903629+0000","last_unstale":"2024-04-25T10:03:52.607565+0000","last_undegraded":"2024-04-25T10:03:52.607565+0000","last_fullsized":"2024-04-25T10:03:52.607565+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:54.432830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606080+0000","last_change":"2024-04-25T10:03:49.899336+0000","last_active":"2024-04-25T10:03:52.606080+0000","last_peered":"2024-04-25T10:03:52.606080+0000","last_clean":"2024-04-25T10:03:52.606080+0000","last_became_active":"2024-04-25T10:03:49.899096+0000","last_became_peered":"2024-04-25T10:03:49.899096+0000","last_unstale":"2024-04-25T10:03:52.606080+0000","last_undegraded":"2024-04-25T10:03:52.606080+0000","last_fullsized":"2024-04-25T10:03:52.606080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:49.112970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606989+0000","last_change":"2024-04-25T10:03:50.905181+0000","last_active":"2024-04-25T10:03:52.606989+0000","last_peered":"2024-04-25T10:03:52.606989+0000","last_clean":"2024-04-25T10:03:52.606989+0000","last_became_active":"2024-04-25T10:03:50.904575+0000","last_became_peered":"2024-04-25T10:03:50.904575+0000","last_unstale":"2024-04-25T10:03:52.606989+0000","last_undegraded":"2024-04-25T10:03:52.606989+0000","last_fullsized":"2024-04-25T10:03:52.606989+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:11.198548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606384+0000","last_change":"2024-04-25T10:03:49.898990+0000","last_active":"2024-04-25T10:03:52.606384+0000","last_peered":"2024-04-25T10:03:52.606384+0000","last_clean":"2024-04-25T10:03:52.606384+0000","last_became_active":"2024-04-25T10:03:49.898720+0000","last_became_peered":"2024-04-25T10:03:49.898720+0000","last_unstale":"2024-04-25T10:03:52.606384+0000","last_undegraded":"2024-04-25T10:03:52.606384+0000","last_fullsized":"2024-04-25T10:03:52.606384+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:10.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607120+0000","last_change":"2024-04-25T10:03:50.908185+0000","last_active":"2024-04-25T10:03:52.607120+0000","last_peered":"2024-04-25T10:03:52.607120+0000","last_clean":"2024-04-25T10:03:52.607120+0000","last_became_active":"2024-04-25T10:03:50.907912+0000","last_became_peered":"2024-04-25T10:03:50.907912+0000","last_unstale":"2024-04-25T10:03:52.607120+0000","last_undegraded":"2024-04-25T10:03:52.607120+0000","last_fullsized":"2024-04-25T10:03:52.607120+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:53.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883420+0000","last_change":"2024-04-25T10:03:49.901276+0000","last_active":"2024-04-25T10:03:51.883420+0000","last_peered":"2024-04-25T10:03:51.883420+0000","last_clean":"2024-04-25T10:03:51.883420+0000","last_became_active":"2024-04-25T10:03:49.900977+0000","last_became_peered":"2024-04-25T10:03:49.900977+0000","last_unstale":"2024-04-25T10:03:51.883420+0000","last_undegraded":"2024-04-25T10:03:51.883420+0000","last_fullsized":"2024-04-25T10:03:51.883420+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.283728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607115+0000","last_change":"2024-04-25T10:03:50.902808+0000","last_active":"2024-04-25T10:03:52.607115+0000","last_peered":"2024-04-25T10:03:52.607115+0000","last_clean":"2024-04-25T10:03:52.607115+0000","last_became_active":"2024-04-25T10:03:50.902105+0000","last_became_peered":"2024-04-25T10:03:50.902105+0000","last_unstale":"2024-04-25T10:03:52.607115+0000","last_undegraded":"2024-04-25T10:03:52.607115+0000","last_fullsized":"2024-04-25T10:03:52.607115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606534+0000","last_change":"2024-04-25T10:03:49.898957+0000","last_active":"2024-04-25T10:03:52.606534+0000","last_peered":"2024-04-25T10:03:52.606534+0000","last_clean":"2024-04-25T10:03:52.606534+0000","last_became_active":"2024-04-25T10:03:49.898681+0000","last_became_peered":"2024-04-25T10:03:49.898681+0000","last_unstale":"2024-04-25T10:03:52.606534+0000","last_undegraded":"2024-04-25T10:03:52.606534+0000","last_fullsized":"2024-04-25T10:03:52.606534+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:06.966070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606206+0000","last_change":"2024-04-25T10:03:50.896762+0000","last_active":"2024-04-25T10:03:52.606206+0000","last_peered":"2024-04-25T10:03:52.606206+0000","last_clean":"2024-04-25T10:03:52.606206+0000","last_became_active":"2024-04-25T10:03:50.896491+0000","last_became_peered":"2024-04-25T10:03:50.896491+0000","last_unstale":"2024-04-25T10:03:52.606206+0000","last_undegraded":"2024-04-25T10:03:52.606206+0000","last_fullsized":"2024-04-25T10:03:52.606206+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:53.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605608+0000","last_change":"2024-04-25T10:03:49.895700+0000","last_active":"2024-04-25T10:03:52.605608+0000","last_peered":"2024-04-25T10:03:52.605608+0000","last_clean":"2024-04-25T10:03:52.605608+0000","last_became_active":"2024-04-25T10:03:49.895008+0000","last_became_peered":"2024-04-25T10:03:49.895008+0000","last_unstale":"2024-04-25T10:03:52.605608+0000","last_undegraded":"2024-04-25T10:03:52.605608+0000","last_fullsized":"2024-04-25T10:03:52.605608+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:56.862010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883265+0000","last_change":"2024-04-25T10:03:50.899317+0000","last_active":"2024-04-25T10:03:51.883265+0000","last_peered":"2024-04-25T10:03:51.883265+0000","last_clean":"2024-04-25T10:03:51.883265+0000","last_became_active":"2024-04-25T10:03:50.899023+0000","last_became_peered":"2024-04-25T10:03:50.899023+0000","last_unstale":"2024-04-25T10:03:51.883265+0000","last_undegraded":"2024-04-25T10:03:51.883265+0000","last_fullsized":"2024-04-25T10:03:51.883265+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:15.590822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886906+0000","last_change":"2024-04-25T10:03:49.895688+0000","last_active":"2024-04-25T10:03:51.886906+0000","last_peered":"2024-04-25T10:03:51.886906+0000","last_clean":"2024-04-25T10:03:51.886906+0000","last_became_active":"2024-04-25T10:03:49.895423+0000","last_became_peered":"2024-04-25T10:03:49.895423+0000","last_unstale":"2024-04-25T10:03:51.886906+0000","last_undegraded":"2024-04-25T10:03:51.886906+0000","last_fullsized":"2024-04-25T10:03:51.886906+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:56.051644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606308+0000","last_change":"2024-04-25T10:03:50.899825+0000","last_active":"2024-04-25T10:03:52.606308+0000","last_peered":"2024-04-25T10:03:52.606308+0000","last_clean":"2024-04-25T10:03:52.606308+0000","last_became_active":"2024-04-25T10:03:50.899548+0000","last_became_peered":"2024-04-25T10:03:50.899548+0000","last_unstale":"2024-04-25T10:03:52.606308+0000","last_undegraded":"2024-04-25T10:03:52.606308+0000","last_fullsized":"2024-04-25T10:03:52.606308+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:36.335317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605369+0000","last_change":"2024-04-25T10:03:49.895117+0000","last_active":"2024-04-25T10:03:52.605369+0000","last_peered":"2024-04-25T10:03:52.605369+0000","last_clean":"2024-04-25T10:03:52.605369+0000","last_became_active":"2024-04-25T10:03:49.894870+0000","last_became_peered":"2024-04-25T10:03:49.894870+0000","last_unstale":"2024-04-25T10:03:52.605369+0000","last_undegraded":"2024-04-25T10:03:52.605369+0000","last_fullsized":"2024-04-25T10:03:52.605369+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:39.270697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883101+0000","last_change":"2024-04-25T10:03:50.904119+0000","last_active":"2024-04-25T10:03:51.883101+0000","last_peered":"2024-04-25T10:03:51.883101+0000","last_clean":"2024-04-25T10:03:51.883101+0000","last_became_active":"2024-04-25T10:03:50.903818+0000","last_became_peered":"2024-04-25T10:03:50.903818+0000","last_unstale":"2024-04-25T10:03:51.883101+0000","last_undegraded":"2024-04-25T10:03:51.883101+0000","last_fullsized":"2024-04-25T10:03:51.883101+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.866446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887179+0000","last_change":"2024-04-25T10:03:49.895442+0000","last_active":"2024-04-25T10:03:51.887179+0000","last_peered":"2024-04-25T10:03:51.887179+0000","last_clean":"2024-04-25T10:03:51.887179+0000","last_became_active":"2024-04-25T10:03:49.895113+0000","last_became_peered":"2024-04-25T10:03:49.895113+0000","last_unstale":"2024-04-25T10:03:51.887179+0000","last_undegraded":"2024-04-25T10:03:51.887179+0000","last_fullsized":"2024-04-25T10:03:51.887179+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.487258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.887060+0000","last_change":"2024-04-25T10:03:49.887060+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.887060+0000","last_undegraded":"2024-04-25T10:03:49.887060+0000","last_fullsized":"2024-04-25T10:03:49.887060+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607196+0000","last_change":"2024-04-25T10:03:49.898517+0000","last_active":"2024-04-25T10:03:52.607196+0000","last_peered":"2024-04-25T10:03:52.607196+0000","last_clean":"2024-04-25T10:03:52.607196+0000","last_became_active":"2024-04-25T10:03:49.898253+0000","last_became_peered":"2024-04-25T10:03:49.898253+0000","last_unstale":"2024-04-25T10:03:52.607196+0000","last_undegraded":"2024-04-25T10:03:52.607196+0000","last_fullsized":"2024-04-25T10:03:52.607196+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606431+0000","last_change":"2024-04-25T10:03:50.908336+0000","last_active":"2024-04-25T10:03:52.606431+0000","last_peered":"2024-04-25T10:03:52.606431+0000","last_clean":"2024-04-25T10:03:52.606431+0000","last_became_active":"2024-04-25T10:03:50.908035+0000","last_became_peered":"2024-04-25T10:03:50.908035+0000","last_unstale":"2024-04-25T10:03:52.606431+0000","last_undegraded":"2024-04-25T10:03:52.606431+0000","last_fullsized":"2024-04-25T10:03:52.606431+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.908343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"145'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607395+0000","last_change":"2024-04-25T10:03:49.899974+0000","last_active":"2024-04-25T10:03:52.607395+0000","last_peered":"2024-04-25T10:03:52.607395+0000","last_clean":"2024-04-25T10:03:52.607395+0000","last_became_active":"2024-04-25T10:03:49.899694+0000","last_became_peered":"2024-04-25T10:03:49.899694+0000","last_unstale":"2024-04-25T10:03:52.607395+0000","last_undegraded":"2024-04-25T10:03:52.607395+0000","last_fullsized":"2024-04-25T10:03:52.607395+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.352713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607287+0000","last_change":"2024-04-25T10:03:50.906673+0000","last_active":"2024-04-25T10:03:52.607287+0000","last_peered":"2024-04-25T10:03:52.607287+0000","last_clean":"2024-04-25T10:03:52.607287+0000","last_became_active":"2024-04-25T10:03:50.906372+0000","last_became_peered":"2024-04-25T10:03:50.906372+0000","last_unstale":"2024-04-25T10:03:52.607287+0000","last_undegraded":"2024-04-25T10:03:52.607287+0000","last_fullsized":"2024-04-25T10:03:52.607287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:13.082455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883464+0000","last_change":"2024-04-25T10:03:49.894817+0000","last_active":"2024-04-25T10:03:51.883464+0000","last_peered":"2024-04-25T10:03:51.883464+0000","last_clean":"2024-04-25T10:03:51.883464+0000","last_became_active":"2024-04-25T10:03:49.894481+0000","last_became_peered":"2024-04-25T10:03:49.894481+0000","last_unstale":"2024-04-25T10:03:51.883464+0000","last_undegraded":"2024-04-25T10:03:51.883464+0000","last_fullsized":"2024-04-25T10:03:51.883464+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.975577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T10:03:49.885867+0000","last_change":"2024-04-25T10:03:49.885867+0000","last_active":"2024-04-25T10:03:49.873764+0000","last_peered":"2024-04-25T10:03:49.873764+0000","last_clean":"2024-04-25T10:03:49.873764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:49.885867+0000","last_undegraded":"2024-04-25T10:03:49.885867+0000","last_fullsized":"2024-04-25T10:03:49.885867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T10:03:49.897801+0000","last_change":"2024-04-25T10:03:49.897166+0000","last_active":"2024-04-25T10:03:49.897801+0000","last_peered":"2024-04-25T10:03:49.897801+0000","last_clean":"2024-04-25T10:03:49.897801+0000","last_became_active":"2024-04-25T10:03:49.896766+0000","last_became_peered":"2024-04-25T10:03:49.896766+0000","last_unstale":"2024-04-25T10:03:49.897801+0000","last_undegraded":"2024-04-25T10:03:49.897801+0000","last_fullsized":"2024-04-25T10:03:49.897801+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:45.056797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887080+0000","last_change":"2024-04-25T10:03:50.898146+0000","last_active":"2024-04-25T10:03:51.887080+0000","last_peered":"2024-04-25T10:03:51.887080+0000","last_clean":"2024-04-25T10:03:51.887080+0000","last_became_active":"2024-04-25T10:03:50.895759+0000","last_became_peered":"2024-04-25T10:03:50.895759+0000","last_unstale":"2024-04-25T10:03:51.887080+0000","last_undegraded":"2024-04-25T10:03:51.887080+0000","last_fullsized":"2024-04-25T10:03:51.887080+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.908343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607808+0000","last_change":"2024-04-25T10:03:49.893523+0000","last_active":"2024-04-25T10:03:52.607808+0000","last_peered":"2024-04-25T10:03:52.607808+0000","last_clean":"2024-04-25T10:03:52.607808+0000","last_became_active":"2024-04-25T10:03:49.893233+0000","last_became_peered":"2024-04-25T10:03:49.893233+0000","last_unstale":"2024-04-25T10:03:52.607808+0000","last_undegraded":"2024-04-25T10:03:52.607808+0000","last_fullsized":"2024-04-25T10:03:52.607808+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:09.313997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606768+0000","last_change":"2024-04-25T10:03:50.903635+0000","last_active":"2024-04-25T10:03:52.606768+0000","last_peered":"2024-04-25T10:03:52.606768+0000","last_clean":"2024-04-25T10:03:52.606768+0000","last_became_active":"2024-04-25T10:03:50.903373+0000","last_became_peered":"2024-04-25T10:03:50.903373+0000","last_unstale":"2024-04-25T10:03:52.606768+0000","last_undegraded":"2024-04-25T10:03:52.606768+0000","last_fullsized":"2024-04-25T10:03:52.606768+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.867015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606578+0000","last_change":"2024-04-25T10:03:49.898200+0000","last_active":"2024-04-25T10:03:52.606578+0000","last_peered":"2024-04-25T10:03:52.606578+0000","last_clean":"2024-04-25T10:03:52.606578+0000","last_became_active":"2024-04-25T10:03:49.897929+0000","last_became_peered":"2024-04-25T10:03:49.897929+0000","last_unstale":"2024-04-25T10:03:52.606578+0000","last_undegraded":"2024-04-25T10:03:52.606578+0000","last_fullsized":"2024-04-25T10:03:52.606578+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.342597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883127+0000","last_change":"2024-04-25T10:03:50.886604+0000","last_active":"2024-04-25T10:03:51.883127+0000","last_peered":"2024-04-25T10:03:51.883127+0000","last_clean":"2024-04-25T10:03:51.883127+0000","last_became_active":"2024-04-25T10:03:50.886318+0000","last_became_peered":"2024-04-25T10:03:50.886318+0000","last_unstale":"2024-04-25T10:03:51.883127+0000","last_undegraded":"2024-04-25T10:03:51.883127+0000","last_fullsized":"2024-04-25T10:03:51.883127+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:08.917053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607706+0000","last_change":"2024-04-25T10:03:49.900360+0000","last_active":"2024-04-25T10:03:52.607706+0000","last_peered":"2024-04-25T10:03:52.607706+0000","last_clean":"2024-04-25T10:03:52.607706+0000","last_became_active":"2024-04-25T10:03:49.898980+0000","last_became_peered":"2024-04-25T10:03:49.898980+0000","last_unstale":"2024-04-25T10:03:52.607706+0000","last_undegraded":"2024-04-25T10:03:52.607706+0000","last_fullsized":"2024-04-25T10:03:52.607706+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887301+0000","last_change":"2024-04-25T10:03:50.902623+0000","last_active":"2024-04-25T10:03:51.887301+0000","last_peered":"2024-04-25T10:03:51.887301+0000","last_clean":"2024-04-25T10:03:51.887301+0000","last_became_active":"2024-04-25T10:03:50.901922+0000","last_became_peered":"2024-04-25T10:03:50.901922+0000","last_unstale":"2024-04-25T10:03:51.887301+0000","last_undegraded":"2024-04-25T10:03:51.887301+0000","last_fullsized":"2024-04-25T10:03:51.887301+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:11.681942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606109+0000","last_change":"2024-04-25T10:03:49.896765+0000","last_active":"2024-04-25T10:03:52.606109+0000","last_peered":"2024-04-25T10:03:52.606109+0000","last_clean":"2024-04-25T10:03:52.606109+0000","last_became_active":"2024-04-25T10:03:49.896465+0000","last_became_peered":"2024-04-25T10:03:49.896465+0000","last_unstale":"2024-04-25T10:03:52.606109+0000","last_undegraded":"2024-04-25T10:03:52.606109+0000","last_fullsized":"2024-04-25T10:03:52.606109+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:45.056797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887243+0000","last_change":"2024-04-25T10:03:50.889603+0000","last_active":"2024-04-25T10:03:51.887243+0000","last_peered":"2024-04-25T10:03:51.887243+0000","last_clean":"2024-04-25T10:03:51.887243+0000","last_became_active":"2024-04-25T10:03:50.889294+0000","last_became_peered":"2024-04-25T10:03:50.889294+0000","last_unstale":"2024-04-25T10:03:51.887243+0000","last_undegraded":"2024-04-25T10:03:51.887243+0000","last_fullsized":"2024-04-25T10:03:51.887243+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:46.062004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"144'6","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605745+0000","last_change":"2024-04-25T10:03:49.895770+0000","last_active":"2024-04-25T10:03:52.605745+0000","last_peered":"2024-04-25T10:03:52.605745+0000","last_clean":"2024-04-25T10:03:52.605745+0000","last_became_active":"2024-04-25T10:03:49.895492+0000","last_became_peered":"2024-04-25T10:03:49.895492+0000","last_unstale":"2024-04-25T10:03:52.605745+0000","last_undegraded":"2024-04-25T10:03:52.605745+0000","last_fullsized":"2024-04-25T10:03:52.605745+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:00.748519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":400,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606521+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:03:52.606521+0000","last_peered":"2024-04-25T10:03:52.606521+0000","last_clean":"2024-04-25T10:03:52.606521+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:03:52.606521+0000","last_undegraded":"2024-04-25T10:03:52.606521+0000","last_fullsized":"2024-04-25T10:03:52.606521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51008,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320832,"statfs":{"total":96636764160,"available":96584531968,"internally_reserved":0,"allocated":1822720,"data_stored":849679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46848,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324992,"statfs":{"total":96636764160,"available":96588791808,"internally_reserved":0,"allocated":1818624,"data_stored":849609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1830912,"data_stored":851221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59304,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312536,"statfs":{"total":96636764160,"available":96576036864,"internally_reserved":0,"allocated":1794048,"data_stored":835750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51020,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320820,"statfs":{"total":96636764160,"available":96584519680,"internally_reserved":0,"allocated":1826816,"data_stored":849721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51012,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320828,"statfs":{"total":96636764160,"available":96584527872,"internally_reserved":0,"allocated":1822720,"data_stored":849679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51612,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320228,"statfs":{"total":96636764160,"available":96583913472,"internally_reserved":0,"allocated":2437120,"data_stored":1442295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:55.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:55.304 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:56.165 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-04-25T10:03:56.160+0000 7f9fb1e82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:58.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:58.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.320+0000 7f2aa37e0640 1 Processor -- start 2024-04-25T10:03:58.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.320+0000 7f2aa37e0640 1 -- start start 2024-04-25T10:03:58.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa37e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c154ee0 0x7f2a9c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa37e0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2a9c0ce590 0x7f2a9c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa37e0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c071480 con 0x7f2a9c0cfcf0 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa37e0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c072090 con 0x7f2a9c0ce590 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa37e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0d0150 con 0x7f2a9c154ee0 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa1d7d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2a9c0ce590 0x7f2a9c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa0d7b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2a9c0cfcf0 0x7f2a9c0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34964/0 (socket says 172.21.15.148:34964) 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa0d7b640 1 -- 172.21.15.148:0/1325584828 learned_addr learned my addr 172.21.15.148:0/1325584828 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.321+0000 7f2aa157c640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c154ee0 0x7f2a9c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2aa1d7d640 1 -- 172.21.15.148:0/1325584828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c154ee0 msgr2=0x7f2a9c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2aa1d7d640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c154ee0 0x7f2a9c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2aa1d7d640 1 -- 172.21.15.148:0/1325584828 >> v1:172.21.15.148:6789/0 conn(0x7f2a9c0cfcf0 legacy=0x7f2a9c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2aa1d7d640 1 -- 172.21.15.148:0/1325584828 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c071760 con 0x7f2a9c0ce590 2024-04-25T10:03:58.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2aa157c640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c154ee0 0x7f2a9c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2aa1d7d640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c06d6f0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097a0 tx=0x7f2a9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df0263e4b026a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.322+0000 7f2a93fff640 1 -- 172.21.15.148:0/1325584828 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94037070 con 0x7f2a9c0ce590 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2a93fff640 1 -- 172.21.15.148:0/1325584828 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94032440 con 0x7f2a9c0ce590 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2a93fff640 1 -- 172.21.15.148:0/1325584828 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94032810 con 0x7f2a9c0ce590 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 -- 172.21.15.148:0/1325584828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 msgr2=0x7f2a9c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c06d6f0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f2a940097a0 tx=0x7f2a9402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 -- 172.21.15.148:0/1325584828 shutdown_connections 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c06d6f0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/1325584828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c154ee0 0x7f2a9c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 -- 172.21.15.148:0/1325584828 >> 172.21.15.148:0/1325584828 conn(0x7f2a9c06abe0 msgr2=0x7f2a9c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 -- 172.21.15.148:0/1325584828 shutdown_connections 2024-04-25T10:03:58.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.323+0000 7f2aa37e0640 1 -- 172.21.15.148:0/1325584828 wait complete. 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.324+0000 7f2aa37e0640 1 Processor -- start 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.324+0000 7f2aa37e0640 1 -- start start 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa37e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa37e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c0cfcf0 0x7f2a9c158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa37e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c154ee0 0x7f2a9c0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa37e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c071d20 con 0x7f2a9c0ce590 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa37e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c071230 con 0x7f2a9c154ee0 2024-04-25T10:03:58.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa37e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c1552c0 con 0x7f2a9c0cfcf0 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c14a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33716/0 (socket says 172.21.15.148:33716) 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 -- 172.21.15.148:0/600782372 learned_addr learned my addr 172.21.15.148:0/600782372 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 -- 172.21.15.148:0/600782372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c0cfcf0 msgr2=0x7f2a9c158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c0cfcf0 0x7f2a9c158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 -- 172.21.15.148:0/600782372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c154ee0 msgr2=0x7f2a9c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c154ee0 0x7f2a9c0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.325+0000 7f2aa157c640 1 -- 172.21.15.148:0/600782372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c159100 con 0x7f2a9c0ce590 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.326+0000 7f2aa157c640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c14a740 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f2a98007ef0 tx=0x7f2a9800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.326+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9801a070 con 0x7f2a9c0ce590 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.326+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98003b90 con 0x7f2a9c0ce590 2024-04-25T10:03:58.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.326+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98004030 con 0x7f2a9c0ce590 2024-04-25T10:03:58.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.326+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c150700 con 0x7f2a9c0ce590 2024-04-25T10:03:58.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.326+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c150cc0 con 0x7f2a9c0ce590 2024-04-25T10:03:58.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.327+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c073990 con 0x7f2a9c0ce590 2024-04-25T10:03:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.328+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a9801d070 con 0x7f2a9c0ce590 2024-04-25T10:03:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.328+0000 7f2a91ffb640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a8006b4d0 0x7f2a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.328+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a9808a4c0 con 0x7f2a9c0ce590 2024-04-25T10:03:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.328+0000 7f2aa0d7b640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a8006b4d0 0x7f2a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.328+0000 7f2aa0d7b640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a8006b4d0 0x7f2a8006d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000f60 tx=0x7f2a8c007a80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.332+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a98010070 con 0x7f2a9c0ce590 2024-04-25T10:03:58.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.499+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a9c0ce350 con 0x7f2a8006b4d0 2024-04-25T10:03:58.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.505+0000 7f2a91ffb640 1 -- 172.21.15.148:0/600782372 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252484 (secure 0 0 0) 0x7f2a9c0ce350 con 0x7f2a8006b4d0 2024-04-25T10:03:58.516 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:58.518 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:03:58.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a8006b4d0 msgr2=0x7f2a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a8006b4d0 0x7f2a8006d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c000f60 tx=0x7f2a8c007a80 comp rx=0 tx=0).stop 2024-04-25T10:03:58.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 msgr2=0x7f2a9c14a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.521 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c14a740 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f2a98007ef0 tx=0x7f2a9800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 shutdown_connections 2024-04-25T10:03:58.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a8006b4d0 0x7f2a8006d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a9c154ee0 0x7f2a9c0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a9c0cfcf0 0x7f2a9c158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 --2- 172.21.15.148:0/600782372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a9c0ce590 0x7f2a9c14a740 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 >> 172.21.15.148:0/600782372 conn(0x7f2a9c06abe0 msgr2=0x7f2a9c06d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 shutdown_connections 2024-04-25T10:03:58.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.508+0000 7f2aa37e0640 1 -- 172.21.15.148:0/600782372 wait complete. 2024-04-25T10:03:58.525 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-25T10:03:57.138481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417608,"kb_used_data":15432,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754557112,"statfs":{"total":773094113280,"available":772666482688,"internally_reserved":0,"allocated":15802368,"data_stored":7982428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.259173"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606928+0000","last_change":"2024-04-25T10:03:50.905492+0000","last_active":"2024-04-25T10:03:52.606928+0000","last_peered":"2024-04-25T10:03:52.606928+0000","last_clean":"2024-04-25T10:03:52.606928+0000","last_became_active":"2024-04-25T10:03:50.905219+0000","last_became_peered":"2024-04-25T10:03:50.905219+0000","last_unstale":"2024-04-25T10:03:52.606928+0000","last_undegraded":"2024-04-25T10:03:52.606928+0000","last_fullsized":"2024-04-25T10:03:52.606928+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:50.515584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883264+0000","last_change":"2024-04-25T10:03:50.895420+0000","last_active":"2024-04-25T10:03:51.883264+0000","last_peered":"2024-04-25T10:03:51.883264+0000","last_clean":"2024-04-25T10:03:51.883264+0000","last_became_active":"2024-04-25T10:03:50.895108+0000","last_became_peered":"2024-04-25T10:03:50.895108+0000","last_unstale":"2024-04-25T10:03:51.883264+0000","last_undegraded":"2024-04-25T10:03:51.883264+0000","last_fullsized":"2024-04-25T10:03:51.883264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:50.515584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883377+0000","last_change":"2024-04-25T10:03:50.893021+0000","last_active":"2024-04-25T10:03:51.883377+0000","last_peered":"2024-04-25T10:03:51.883377+0000","last_clean":"2024-04-25T10:03:51.883377+0000","last_became_active":"2024-04-25T10:03:50.892759+0000","last_became_peered":"2024-04-25T10:03:50.892759+0000","last_unstale":"2024-04-25T10:03:51.883377+0000","last_undegraded":"2024-04-25T10:03:51.883377+0000","last_fullsized":"2024-04-25T10:03:51.883377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606272+0000","last_change":"2024-04-25T10:03:50.899244+0000","last_active":"2024-04-25T10:03:52.606272+0000","last_peered":"2024-04-25T10:03:52.606272+0000","last_clean":"2024-04-25T10:03:52.606272+0000","last_became_active":"2024-04-25T10:03:50.898961+0000","last_became_peered":"2024-04-25T10:03:50.898961+0000","last_unstale":"2024-04-25T10:03:52.606272+0000","last_undegraded":"2024-04-25T10:03:52.606272+0000","last_fullsized":"2024-04-25T10:03:52.606272+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:41.231201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606259+0000","last_change":"2024-04-25T10:03:50.896392+0000","last_active":"2024-04-25T10:03:52.606259+0000","last_peered":"2024-04-25T10:03:52.606259+0000","last_clean":"2024-04-25T10:03:52.606259+0000","last_became_active":"2024-04-25T10:03:50.896125+0000","last_became_peered":"2024-04-25T10:03:50.896125+0000","last_unstale":"2024-04-25T10:03:52.606259+0000","last_undegraded":"2024-04-25T10:03:52.606259+0000","last_fullsized":"2024-04-25T10:03:52.606259+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:08.033891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605775+0000","last_change":"2024-04-25T10:03:50.899238+0000","last_active":"2024-04-25T10:03:52.605775+0000","last_peered":"2024-04-25T10:03:52.605775+0000","last_clean":"2024-04-25T10:03:52.605775+0000","last_became_active":"2024-04-25T10:03:50.898968+0000","last_became_peered":"2024-04-25T10:03:50.898968+0000","last_unstale":"2024-04-25T10:03:52.605775+0000","last_undegraded":"2024-04-25T10:03:52.605775+0000","last_fullsized":"2024-04-25T10:03:52.605775+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:09.948098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607444+0000","last_change":"2024-04-25T10:03:50.897311+0000","last_active":"2024-04-25T10:03:52.607444+0000","last_peered":"2024-04-25T10:03:52.607444+0000","last_clean":"2024-04-25T10:03:52.607444+0000","last_became_active":"2024-04-25T10:03:50.893752+0000","last_became_peered":"2024-04-25T10:03:50.893752+0000","last_unstale":"2024-04-25T10:03:52.607444+0000","last_undegraded":"2024-04-25T10:03:52.607444+0000","last_fullsized":"2024-04-25T10:03:52.607444+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:11.198548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886753+0000","last_change":"2024-04-25T10:03:50.903258+0000","last_active":"2024-04-25T10:03:51.886753+0000","last_peered":"2024-04-25T10:03:51.886753+0000","last_clean":"2024-04-25T10:03:51.886753+0000","last_became_active":"2024-04-25T10:03:50.900168+0000","last_became_peered":"2024-04-25T10:03:50.900168+0000","last_unstale":"2024-04-25T10:03:51.886753+0000","last_undegraded":"2024-04-25T10:03:51.886753+0000","last_fullsized":"2024-04-25T10:03:51.886753+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:23.357921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606271+0000","last_change":"2024-04-25T10:03:50.903127+0000","last_active":"2024-04-25T10:03:52.606271+0000","last_peered":"2024-04-25T10:03:52.606271+0000","last_clean":"2024-04-25T10:03:52.606271+0000","last_became_active":"2024-04-25T10:03:50.901740+0000","last_became_peered":"2024-04-25T10:03:50.901740+0000","last_unstale":"2024-04-25T10:03:52.606271+0000","last_undegraded":"2024-04-25T10:03:52.606271+0000","last_fullsized":"2024-04-25T10:03:52.606271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:35.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606108+0000","last_change":"2024-04-25T10:03:50.909226+0000","last_active":"2024-04-25T10:03:52.606108+0000","last_peered":"2024-04-25T10:03:52.606108+0000","last_clean":"2024-04-25T10:03:52.606108+0000","last_became_active":"2024-04-25T10:03:50.908980+0000","last_became_peered":"2024-04-25T10:03:50.908980+0000","last_unstale":"2024-04-25T10:03:52.606108+0000","last_undegraded":"2024-04-25T10:03:52.606108+0000","last_fullsized":"2024-04-25T10:03:52.606108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.999312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606271+0000","last_change":"2024-04-25T10:03:50.902053+0000","last_active":"2024-04-25T10:03:52.606271+0000","last_peered":"2024-04-25T10:03:52.606271+0000","last_clean":"2024-04-25T10:03:52.606271+0000","last_became_active":"2024-04-25T10:03:50.901690+0000","last_became_peered":"2024-04-25T10:03:50.901690+0000","last_unstale":"2024-04-25T10:03:52.606271+0000","last_undegraded":"2024-04-25T10:03:52.606271+0000","last_fullsized":"2024-04-25T10:03:52.606271+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:17.479698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606276+0000","last_change":"2024-04-25T10:03:50.907519+0000","last_active":"2024-04-25T10:03:52.606276+0000","last_peered":"2024-04-25T10:03:52.606276+0000","last_clean":"2024-04-25T10:03:52.606276+0000","last_became_active":"2024-04-25T10:03:50.907256+0000","last_became_peered":"2024-04-25T10:03:50.907256+0000","last_unstale":"2024-04-25T10:03:52.606276+0000","last_undegraded":"2024-04-25T10:03:52.606276+0000","last_fullsized":"2024-04-25T10:03:52.606276+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:46.062004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606106+0000","last_change":"2024-04-25T10:03:50.908985+0000","last_active":"2024-04-25T10:03:52.606106+0000","last_peered":"2024-04-25T10:03:52.606106+0000","last_clean":"2024-04-25T10:03:52.606106+0000","last_became_active":"2024-04-25T10:03:50.908443+0000","last_became_peered":"2024-04-25T10:03:50.908443+0000","last_unstale":"2024-04-25T10:03:52.606106+0000","last_undegraded":"2024-04-25T10:03:52.606106+0000","last_fullsized":"2024-04-25T10:03:52.606106+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:53.111183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606406+0000","last_change":"2024-04-25T10:03:50.909162+0000","last_active":"2024-04-25T10:03:52.606406+0000","last_peered":"2024-04-25T10:03:52.606406+0000","last_clean":"2024-04-25T10:03:52.606406+0000","last_became_active":"2024-04-25T10:03:50.908890+0000","last_became_peered":"2024-04-25T10:03:50.908890+0000","last_unstale":"2024-04-25T10:03:52.606406+0000","last_undegraded":"2024-04-25T10:03:52.606406+0000","last_fullsized":"2024-04-25T10:03:52.606406+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:11.681942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883264+0000","last_change":"2024-04-25T10:03:50.900082+0000","last_active":"2024-04-25T10:03:51.883264+0000","last_peered":"2024-04-25T10:03:51.883264+0000","last_clean":"2024-04-25T10:03:51.883264+0000","last_became_active":"2024-04-25T10:03:50.899803+0000","last_became_peered":"2024-04-25T10:03:50.899803+0000","last_unstale":"2024-04-25T10:03:51.883264+0000","last_undegraded":"2024-04-25T10:03:51.883264+0000","last_fullsized":"2024-04-25T10:03:51.883264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:53.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605780+0000","last_change":"2024-04-25T10:03:50.910486+0000","last_active":"2024-04-25T10:03:52.605780+0000","last_peered":"2024-04-25T10:03:52.605780+0000","last_clean":"2024-04-25T10:03:52.605780+0000","last_became_active":"2024-04-25T10:03:50.909314+0000","last_became_peered":"2024-04-25T10:03:50.909314+0000","last_unstale":"2024-04-25T10:03:52.605780+0000","last_undegraded":"2024-04-25T10:03:52.605780+0000","last_fullsized":"2024-04-25T10:03:52.605780+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:57.867218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607095+0000","last_change":"2024-04-25T10:03:50.900657+0000","last_active":"2024-04-25T10:03:52.607095+0000","last_peered":"2024-04-25T10:03:52.607095+0000","last_clean":"2024-04-25T10:03:52.607095+0000","last_became_active":"2024-04-25T10:03:50.900355+0000","last_became_peered":"2024-04-25T10:03:50.900355+0000","last_unstale":"2024-04-25T10:03:52.607095+0000","last_undegraded":"2024-04-25T10:03:52.607095+0000","last_fullsized":"2024-04-25T10:03:52.607095+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.949477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605782+0000","last_change":"2024-04-25T10:03:50.896062+0000","last_active":"2024-04-25T10:03:52.605782+0000","last_peered":"2024-04-25T10:03:52.605782+0000","last_clean":"2024-04-25T10:03:52.605782+0000","last_became_active":"2024-04-25T10:03:50.895590+0000","last_became_peered":"2024-04-25T10:03:50.895590+0000","last_unstale":"2024-04-25T10:03:52.605782+0000","last_undegraded":"2024-04-25T10:03:52.605782+0000","last_fullsized":"2024-04-25T10:03:52.605782+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:45.778284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606831+0000","last_change":"2024-04-25T10:03:50.897717+0000","last_active":"2024-04-25T10:03:52.606831+0000","last_peered":"2024-04-25T10:03:52.606831+0000","last_clean":"2024-04-25T10:03:52.606831+0000","last_became_active":"2024-04-25T10:03:50.897142+0000","last_became_peered":"2024-04-25T10:03:50.897142+0000","last_unstale":"2024-04-25T10:03:52.606831+0000","last_undegraded":"2024-04-25T10:03:52.606831+0000","last_fullsized":"2024-04-25T10:03:52.606831+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:13.082455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605650+0000","last_change":"2024-04-25T10:03:50.908907+0000","last_active":"2024-04-25T10:03:52.605650+0000","last_peered":"2024-04-25T10:03:52.605650+0000","last_clean":"2024-04-25T10:03:52.605650+0000","last_became_active":"2024-04-25T10:03:50.908266+0000","last_became_peered":"2024-04-25T10:03:50.908266+0000","last_unstale":"2024-04-25T10:03:52.605650+0000","last_undegraded":"2024-04-25T10:03:52.605650+0000","last_fullsized":"2024-04-25T10:03:52.605650+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:08.033891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606110+0000","last_change":"2024-04-25T10:03:50.903009+0000","last_active":"2024-04-25T10:03:52.606110+0000","last_peered":"2024-04-25T10:03:52.606110+0000","last_clean":"2024-04-25T10:03:52.606110+0000","last_became_active":"2024-04-25T10:03:50.902547+0000","last_became_peered":"2024-04-25T10:03:50.902547+0000","last_unstale":"2024-04-25T10:03:52.606110+0000","last_undegraded":"2024-04-25T10:03:52.606110+0000","last_fullsized":"2024-04-25T10:03:52.606110+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:13.760061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606801+0000","last_change":"2024-04-25T10:03:50.898017+0000","last_active":"2024-04-25T10:03:52.606801+0000","last_peered":"2024-04-25T10:03:52.606801+0000","last_clean":"2024-04-25T10:03:52.606801+0000","last_became_active":"2024-04-25T10:03:50.897747+0000","last_became_peered":"2024-04-25T10:03:50.897747+0000","last_unstale":"2024-04-25T10:03:52.606801+0000","last_undegraded":"2024-04-25T10:03:52.606801+0000","last_fullsized":"2024-04-25T10:03:52.606801+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886714+0000","last_change":"2024-04-25T10:03:50.897929+0000","last_active":"2024-04-25T10:03:51.886714+0000","last_peered":"2024-04-25T10:03:51.886714+0000","last_clean":"2024-04-25T10:03:51.886714+0000","last_became_active":"2024-04-25T10:03:50.895163+0000","last_became_peered":"2024-04-25T10:03:50.895163+0000","last_unstale":"2024-04-25T10:03:51.886714+0000","last_undegraded":"2024-04-25T10:03:51.886714+0000","last_fullsized":"2024-04-25T10:03:51.886714+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:51.005859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606320+0000","last_change":"2024-04-25T10:03:50.905962+0000","last_active":"2024-04-25T10:03:52.606320+0000","last_peered":"2024-04-25T10:03:52.606320+0000","last_clean":"2024-04-25T10:03:52.606320+0000","last_became_active":"2024-04-25T10:03:50.905684+0000","last_became_peered":"2024-04-25T10:03:50.905684+0000","last_unstale":"2024-04-25T10:03:52.606320+0000","last_undegraded":"2024-04-25T10:03:52.606320+0000","last_fullsized":"2024-04-25T10:03:52.606320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:02.222552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607043+0000","last_change":"2024-04-25T10:03:50.896576+0000","last_active":"2024-04-25T10:03:52.607043+0000","last_peered":"2024-04-25T10:03:52.607043+0000","last_clean":"2024-04-25T10:03:52.607043+0000","last_became_active":"2024-04-25T10:03:50.893344+0000","last_became_peered":"2024-04-25T10:03:50.893344+0000","last_unstale":"2024-04-25T10:03:52.607043+0000","last_undegraded":"2024-04-25T10:03:52.607043+0000","last_fullsized":"2024-04-25T10:03:52.607043+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.716636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607042+0000","last_change":"2024-04-25T10:03:50.900898+0000","last_active":"2024-04-25T10:03:52.607042+0000","last_peered":"2024-04-25T10:03:52.607042+0000","last_clean":"2024-04-25T10:03:52.607042+0000","last_became_active":"2024-04-25T10:03:50.898774+0000","last_became_peered":"2024-04-25T10:03:50.898774+0000","last_unstale":"2024-04-25T10:03:52.607042+0000","last_undegraded":"2024-04-25T10:03:52.607042+0000","last_fullsized":"2024-04-25T10:03:52.607042+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:36.335317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606040+0000","last_change":"2024-04-25T10:03:50.902119+0000","last_active":"2024-04-25T10:03:52.606040+0000","last_peered":"2024-04-25T10:03:52.606040+0000","last_clean":"2024-04-25T10:03:52.606040+0000","last_became_active":"2024-04-25T10:03:50.901869+0000","last_became_peered":"2024-04-25T10:03:50.901869+0000","last_unstale":"2024-04-25T10:03:52.606040+0000","last_undegraded":"2024-04-25T10:03:52.606040+0000","last_fullsized":"2024-04-25T10:03:52.606040+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:01.753727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606687+0000","last_change":"2024-04-25T10:03:50.897308+0000","last_active":"2024-04-25T10:03:52.606687+0000","last_peered":"2024-04-25T10:03:52.606687+0000","last_clean":"2024-04-25T10:03:52.606687+0000","last_became_active":"2024-04-25T10:03:50.896847+0000","last_became_peered":"2024-04-25T10:03:50.896847+0000","last_unstale":"2024-04-25T10:03:52.606687+0000","last_undegraded":"2024-04-25T10:03:52.606687+0000","last_fullsized":"2024-04-25T10:03:52.606687+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:07.338275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887497+0000","last_change":"2024-04-25T10:03:50.902013+0000","last_active":"2024-04-25T10:03:51.887497+0000","last_peered":"2024-04-25T10:03:51.887497+0000","last_clean":"2024-04-25T10:03:51.887497+0000","last_became_active":"2024-04-25T10:03:50.901755+0000","last_became_peered":"2024-04-25T10:03:50.901755+0000","last_unstale":"2024-04-25T10:03:51.887497+0000","last_undegraded":"2024-04-25T10:03:51.887497+0000","last_fullsized":"2024-04-25T10:03:51.887497+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.949477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606584+0000","last_change":"2024-04-25T10:03:50.903762+0000","last_active":"2024-04-25T10:03:52.606584+0000","last_peered":"2024-04-25T10:03:52.606584+0000","last_clean":"2024-04-25T10:03:52.606584+0000","last_became_active":"2024-04-25T10:03:50.902357+0000","last_became_peered":"2024-04-25T10:03:50.902357+0000","last_unstale":"2024-04-25T10:03:52.606584+0000","last_undegraded":"2024-04-25T10:03:52.606584+0000","last_fullsized":"2024-04-25T10:03:52.606584+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:21.956097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605671+0000","last_change":"2024-04-25T10:03:50.900081+0000","last_active":"2024-04-25T10:03:52.605671+0000","last_peered":"2024-04-25T10:03:52.605671+0000","last_clean":"2024-04-25T10:03:52.605671+0000","last_became_active":"2024-04-25T10:03:50.899810+0000","last_became_peered":"2024-04-25T10:03:50.899810+0000","last_unstale":"2024-04-25T10:03:52.605671+0000","last_undegraded":"2024-04-25T10:03:52.605671+0000","last_fullsized":"2024-04-25T10:03:52.605671+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:19.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886707+0000","last_change":"2024-04-25T10:03:50.898505+0000","last_active":"2024-04-25T10:03:51.886707+0000","last_peered":"2024-04-25T10:03:51.886707+0000","last_clean":"2024-04-25T10:03:51.886707+0000","last_became_active":"2024-04-25T10:03:50.898162+0000","last_became_peered":"2024-04-25T10:03:50.898162+0000","last_unstale":"2024-04-25T10:03:51.886707+0000","last_undegraded":"2024-04-25T10:03:51.886707+0000","last_fullsized":"2024-04-25T10:03:51.886707+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:41.679413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883264+0000","last_change":"2024-04-25T10:03:50.890540+0000","last_active":"2024-04-25T10:03:51.883264+0000","last_peered":"2024-04-25T10:03:51.883264+0000","last_clean":"2024-04-25T10:03:51.883264+0000","last_became_active":"2024-04-25T10:03:50.889486+0000","last_became_peered":"2024-04-25T10:03:50.889486+0000","last_unstale":"2024-04-25T10:03:51.883264+0000","last_undegraded":"2024-04-25T10:03:51.883264+0000","last_fullsized":"2024-04-25T10:03:51.883264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"144'4","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606517+0000","last_change":"2024-04-25T10:03:49.891438+0000","last_active":"2024-04-25T10:03:52.606517+0000","last_peered":"2024-04-25T10:03:52.606517+0000","last_clean":"2024-04-25T10:03:52.606517+0000","last_became_active":"2024-04-25T10:03:49.890515+0000","last_became_peered":"2024-04-25T10:03:49.890515+0000","last_unstale":"2024-04-25T10:03:52.606517+0000","last_undegraded":"2024-04-25T10:03:52.606517+0000","last_fullsized":"2024-04-25T10:03:52.606517+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:14.585614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883365+0000","last_change":"2024-04-25T10:03:50.903692+0000","last_active":"2024-04-25T10:03:51.883365+0000","last_peered":"2024-04-25T10:03:51.883365+0000","last_clean":"2024-04-25T10:03:51.883365+0000","last_became_active":"2024-04-25T10:03:50.903240+0000","last_became_peered":"2024-04-25T10:03:50.903240+0000","last_unstale":"2024-04-25T10:03:51.883365+0000","last_undegraded":"2024-04-25T10:03:51.883365+0000","last_fullsized":"2024-04-25T10:03:51.883365+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:36.716636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606246+0000","last_change":"2024-04-25T10:03:49.898107+0000","last_active":"2024-04-25T10:03:52.606246+0000","last_peered":"2024-04-25T10:03:52.606246+0000","last_clean":"2024-04-25T10:03:52.606246+0000","last_became_active":"2024-04-25T10:03:49.897243+0000","last_became_peered":"2024-04-25T10:03:49.897243+0000","last_unstale":"2024-04-25T10:03:52.606246+0000","last_undegraded":"2024-04-25T10:03:52.606246+0000","last_fullsized":"2024-04-25T10:03:52.606246+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:23.747311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887201+0000","last_change":"2024-04-25T10:03:50.901337+0000","last_active":"2024-04-25T10:03:51.887201+0000","last_peered":"2024-04-25T10:03:51.887201+0000","last_clean":"2024-04-25T10:03:51.887201+0000","last_became_active":"2024-04-25T10:03:50.900872+0000","last_became_peered":"2024-04-25T10:03:50.900872+0000","last_unstale":"2024-04-25T10:03:51.887201+0000","last_undegraded":"2024-04-25T10:03:51.887201+0000","last_fullsized":"2024-04-25T10:03:51.887201+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:11.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607082+0000","last_change":"2024-04-25T10:03:49.896386+0000","last_active":"2024-04-25T10:03:52.607082+0000","last_peered":"2024-04-25T10:03:52.607082+0000","last_clean":"2024-04-25T10:03:52.607082+0000","last_became_active":"2024-04-25T10:03:49.895902+0000","last_became_peered":"2024-04-25T10:03:49.895902+0000","last_unstale":"2024-04-25T10:03:52.607082+0000","last_undegraded":"2024-04-25T10:03:52.607082+0000","last_fullsized":"2024-04-25T10:03:52.607082+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:26.944270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606419+0000","last_change":"2024-04-25T10:03:50.904288+0000","last_active":"2024-04-25T10:03:52.606419+0000","last_peered":"2024-04-25T10:03:52.606419+0000","last_clean":"2024-04-25T10:03:52.606419+0000","last_became_active":"2024-04-25T10:03:50.903879+0000","last_became_peered":"2024-04-25T10:03:50.903879+0000","last_unstale":"2024-04-25T10:03:52.606419+0000","last_undegraded":"2024-04-25T10:03:52.606419+0000","last_fullsized":"2024-04-25T10:03:52.606419+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883308+0000","last_change":"2024-04-25T10:03:49.896831+0000","last_active":"2024-04-25T10:03:51.883308+0000","last_peered":"2024-04-25T10:03:51.883308+0000","last_clean":"2024-04-25T10:03:51.883308+0000","last_became_active":"2024-04-25T10:03:49.896566+0000","last_became_peered":"2024-04-25T10:03:49.896566+0000","last_unstale":"2024-04-25T10:03:51.883308+0000","last_undegraded":"2024-04-25T10:03:51.883308+0000","last_fullsized":"2024-04-25T10:03:51.883308+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605454+0000","last_change":"2024-04-25T10:03:50.898122+0000","last_active":"2024-04-25T10:03:52.605454+0000","last_peered":"2024-04-25T10:03:52.605454+0000","last_clean":"2024-04-25T10:03:52.605454+0000","last_became_active":"2024-04-25T10:03:50.897828+0000","last_became_peered":"2024-04-25T10:03:50.897828+0000","last_unstale":"2024-04-25T10:03:52.605454+0000","last_undegraded":"2024-04-25T10:03:52.605454+0000","last_fullsized":"2024-04-25T10:03:52.605454+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:53.111183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887450+0000","last_change":"2024-04-25T10:03:49.894730+0000","last_active":"2024-04-25T10:03:51.887450+0000","last_peered":"2024-04-25T10:03:51.887450+0000","last_clean":"2024-04-25T10:03:51.887450+0000","last_became_active":"2024-04-25T10:03:49.894297+0000","last_became_peered":"2024-04-25T10:03:49.894297+0000","last_unstale":"2024-04-25T10:03:51.887450+0000","last_undegraded":"2024-04-25T10:03:51.887450+0000","last_fullsized":"2024-04-25T10:03:51.887450+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:01.257780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606703+0000","last_change":"2024-04-25T10:03:50.900134+0000","last_active":"2024-04-25T10:03:52.606703+0000","last_peered":"2024-04-25T10:03:52.606703+0000","last_clean":"2024-04-25T10:03:52.606703+0000","last_became_active":"2024-04-25T10:03:50.899842+0000","last_became_peered":"2024-04-25T10:03:50.899842+0000","last_unstale":"2024-04-25T10:03:52.606703+0000","last_undegraded":"2024-04-25T10:03:52.606703+0000","last_fullsized":"2024-04-25T10:03:52.606703+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:11.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605880+0000","last_change":"2024-04-25T10:03:49.902295+0000","last_active":"2024-04-25T10:03:52.605880+0000","last_peered":"2024-04-25T10:03:52.605880+0000","last_clean":"2024-04-25T10:03:52.605880+0000","last_became_active":"2024-04-25T10:03:49.902003+0000","last_became_peered":"2024-04-25T10:03:49.902003+0000","last_unstale":"2024-04-25T10:03:52.605880+0000","last_undegraded":"2024-04-25T10:03:52.605880+0000","last_fullsized":"2024-04-25T10:03:52.605880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:23.747311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883081+0000","last_change":"2024-04-25T10:03:50.890023+0000","last_active":"2024-04-25T10:03:51.883081+0000","last_peered":"2024-04-25T10:03:51.883081+0000","last_clean":"2024-04-25T10:03:51.883081+0000","last_became_active":"2024-04-25T10:03:50.889713+0000","last_became_peered":"2024-04-25T10:03:50.889713+0000","last_unstale":"2024-04-25T10:03:51.883081+0000","last_undegraded":"2024-04-25T10:03:51.883081+0000","last_fullsized":"2024-04-25T10:03:51.883081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.949477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607624+0000","last_change":"2024-04-25T10:03:49.899529+0000","last_active":"2024-04-25T10:03:52.607624+0000","last_peered":"2024-04-25T10:03:52.607624+0000","last_clean":"2024-04-25T10:03:52.607624+0000","last_became_active":"2024-04-25T10:03:49.896941+0000","last_became_peered":"2024-04-25T10:03:49.896941+0000","last_unstale":"2024-04-25T10:03:52.607624+0000","last_undegraded":"2024-04-25T10:03:52.607624+0000","last_fullsized":"2024-04-25T10:03:52.607624+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:40.225994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607819+0000","last_change":"2024-04-25T10:03:50.908162+0000","last_active":"2024-04-25T10:03:52.607819+0000","last_peered":"2024-04-25T10:03:52.607819+0000","last_clean":"2024-04-25T10:03:52.607819+0000","last_became_active":"2024-04-25T10:03:50.907834+0000","last_became_peered":"2024-04-25T10:03:50.907834+0000","last_unstale":"2024-04-25T10:03:52.607819+0000","last_undegraded":"2024-04-25T10:03:52.607819+0000","last_fullsized":"2024-04-25T10:03:52.607819+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:54.432830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605900+0000","last_change":"2024-04-25T10:03:49.893772+0000","last_active":"2024-04-25T10:03:52.605900+0000","last_peered":"2024-04-25T10:03:52.605900+0000","last_clean":"2024-04-25T10:03:52.605900+0000","last_became_active":"2024-04-25T10:03:49.893508+0000","last_became_peered":"2024-04-25T10:03:49.893508+0000","last_unstale":"2024-04-25T10:03:52.605900+0000","last_undegraded":"2024-04-25T10:03:52.605900+0000","last_fullsized":"2024-04-25T10:03:52.605900+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:10.861239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887202+0000","last_change":"2024-04-25T10:03:50.891822+0000","last_active":"2024-04-25T10:03:51.887202+0000","last_peered":"2024-04-25T10:03:51.887202+0000","last_clean":"2024-04-25T10:03:51.887202+0000","last_became_active":"2024-04-25T10:03:50.891513+0000","last_became_peered":"2024-04-25T10:03:50.891513+0000","last_unstale":"2024-04-25T10:03:51.887202+0000","last_undegraded":"2024-04-25T10:03:51.887202+0000","last_fullsized":"2024-04-25T10:03:51.887202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.867015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606594+0000","last_change":"2024-04-25T10:03:49.896956+0000","last_active":"2024-04-25T10:03:52.606594+0000","last_peered":"2024-04-25T10:03:52.606594+0000","last_clean":"2024-04-25T10:03:52.606594+0000","last_became_active":"2024-04-25T10:03:49.896167+0000","last_became_peered":"2024-04-25T10:03:49.896167+0000","last_unstale":"2024-04-25T10:03:52.606594+0000","last_undegraded":"2024-04-25T10:03:52.606594+0000","last_fullsized":"2024-04-25T10:03:52.606594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:10.861239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606427+0000","last_change":"2024-04-25T10:03:50.900737+0000","last_active":"2024-04-25T10:03:52.606427+0000","last_peered":"2024-04-25T10:03:52.606427+0000","last_clean":"2024-04-25T10:03:52.606427+0000","last_became_active":"2024-04-25T10:03:50.900455+0000","last_became_peered":"2024-04-25T10:03:50.900455+0000","last_unstale":"2024-04-25T10:03:52.606427+0000","last_undegraded":"2024-04-25T10:03:52.606427+0000","last_fullsized":"2024-04-25T10:03:52.606427+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:42.273961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607319+0000","last_change":"2024-04-25T10:03:49.898465+0000","last_active":"2024-04-25T10:03:52.607319+0000","last_peered":"2024-04-25T10:03:52.607319+0000","last_clean":"2024-04-25T10:03:52.607319+0000","last_became_active":"2024-04-25T10:03:49.898198+0000","last_became_peered":"2024-04-25T10:03:49.898198+0000","last_unstale":"2024-04-25T10:03:52.607319+0000","last_undegraded":"2024-04-25T10:03:52.607319+0000","last_fullsized":"2024-04-25T10:03:52.607319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:50.000651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605928+0000","last_change":"2024-04-25T10:03:50.907140+0000","last_active":"2024-04-25T10:03:52.605928+0000","last_peered":"2024-04-25T10:03:52.605928+0000","last_clean":"2024-04-25T10:03:52.605928+0000","last_became_active":"2024-04-25T10:03:50.906824+0000","last_became_peered":"2024-04-25T10:03:50.906824+0000","last_unstale":"2024-04-25T10:03:52.605928+0000","last_undegraded":"2024-04-25T10:03:52.605928+0000","last_fullsized":"2024-04-25T10:03:52.605928+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:08.352576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"144'3","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606518+0000","last_change":"2024-04-25T10:03:49.897101+0000","last_active":"2024-04-25T10:03:52.606518+0000","last_peered":"2024-04-25T10:03:52.606518+0000","last_clean":"2024-04-25T10:03:52.606518+0000","last_became_active":"2024-04-25T10:03:49.896790+0000","last_became_peered":"2024-04-25T10:03:49.896790+0000","last_unstale":"2024-04-25T10:03:52.606518+0000","last_undegraded":"2024-04-25T10:03:52.606518+0000","last_fullsized":"2024-04-25T10:03:52.606518+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:27.534285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886894+0000","last_change":"2024-04-25T10:03:50.892831+0000","last_active":"2024-04-25T10:03:51.886894+0000","last_peered":"2024-04-25T10:03:51.886894+0000","last_clean":"2024-04-25T10:03:51.886894+0000","last_became_active":"2024-04-25T10:03:50.892584+0000","last_became_peered":"2024-04-25T10:03:50.892584+0000","last_unstale":"2024-04-25T10:03:51.886894+0000","last_undegraded":"2024-04-25T10:03:51.886894+0000","last_fullsized":"2024-04-25T10:03:51.886894+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:41.231201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606226+0000","last_change":"2024-04-25T10:03:49.892221+0000","last_active":"2024-04-25T10:03:52.606226+0000","last_peered":"2024-04-25T10:03:52.606226+0000","last_clean":"2024-04-25T10:03:52.606226+0000","last_became_active":"2024-04-25T10:03:49.891975+0000","last_became_peered":"2024-04-25T10:03:49.891975+0000","last_unstale":"2024-04-25T10:03:52.606226+0000","last_undegraded":"2024-04-25T10:03:52.606226+0000","last_fullsized":"2024-04-25T10:03:52.606226+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:07.911845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883081+0000","last_change":"2024-04-25T10:03:50.903374+0000","last_active":"2024-04-25T10:03:51.883081+0000","last_peered":"2024-04-25T10:03:51.883081+0000","last_clean":"2024-04-25T10:03:51.883081+0000","last_became_active":"2024-04-25T10:03:50.902904+0000","last_became_peered":"2024-04-25T10:03:50.902904+0000","last_unstale":"2024-04-25T10:03:51.883081+0000","last_undegraded":"2024-04-25T10:03:51.883081+0000","last_fullsized":"2024-04-25T10:03:51.883081+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:24.752518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606349+0000","last_change":"2024-04-25T10:03:49.901671+0000","last_active":"2024-04-25T10:03:52.606349+0000","last_peered":"2024-04-25T10:03:52.606349+0000","last_clean":"2024-04-25T10:03:52.606349+0000","last_became_active":"2024-04-25T10:03:49.901399+0000","last_became_peered":"2024-04-25T10:03:49.901399+0000","last_unstale":"2024-04-25T10:03:52.606349+0000","last_undegraded":"2024-04-25T10:03:52.606349+0000","last_fullsized":"2024-04-25T10:03:52.606349+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.511087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605950+0000","last_change":"2024-04-25T10:03:50.901205+0000","last_active":"2024-04-25T10:03:52.605950+0000","last_peered":"2024-04-25T10:03:52.605950+0000","last_clean":"2024-04-25T10:03:52.605950+0000","last_became_active":"2024-04-25T10:03:50.900930+0000","last_became_peered":"2024-04-25T10:03:50.900930+0000","last_unstale":"2024-04-25T10:03:52.605950+0000","last_undegraded":"2024-04-25T10:03:52.605950+0000","last_fullsized":"2024-04-25T10:03:52.605950+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:40.275905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606806+0000","last_change":"2024-04-25T10:03:49.900536+0000","last_active":"2024-04-25T10:03:52.606806+0000","last_peered":"2024-04-25T10:03:52.606806+0000","last_clean":"2024-04-25T10:03:52.606806+0000","last_became_active":"2024-04-25T10:03:49.900264+0000","last_became_peered":"2024-04-25T10:03:49.900264+0000","last_unstale":"2024-04-25T10:03:52.606806+0000","last_undegraded":"2024-04-25T10:03:52.606806+0000","last_fullsized":"2024-04-25T10:03:52.606806+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:59.366267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883110+0000","last_change":"2024-04-25T10:03:50.905600+0000","last_active":"2024-04-25T10:03:51.883110+0000","last_peered":"2024-04-25T10:03:51.883110+0000","last_clean":"2024-04-25T10:03:51.883110+0000","last_became_active":"2024-04-25T10:03:50.905258+0000","last_became_peered":"2024-04-25T10:03:50.905258+0000","last_unstale":"2024-04-25T10:03:51.883110+0000","last_undegraded":"2024-04-25T10:03:51.883110+0000","last_fullsized":"2024-04-25T10:03:51.883110+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:03.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"144'8","reported_seq":22,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605942+0000","last_change":"2024-04-25T10:03:49.897285+0000","last_active":"2024-04-25T10:03:52.605942+0000","last_peered":"2024-04-25T10:03:52.605942+0000","last_clean":"2024-04-25T10:03:52.605942+0000","last_became_active":"2024-04-25T10:03:49.896833+0000","last_became_peered":"2024-04-25T10:03:49.896833+0000","last_unstale":"2024-04-25T10:03:52.605942+0000","last_undegraded":"2024-04-25T10:03:52.605942+0000","last_fullsized":"2024-04-25T10:03:52.605942+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:40.674205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607565+0000","last_change":"2024-04-25T10:03:50.903914+0000","last_active":"2024-04-25T10:03:52.607565+0000","last_peered":"2024-04-25T10:03:52.607565+0000","last_clean":"2024-04-25T10:03:52.607565+0000","last_became_active":"2024-04-25T10:03:50.903629+0000","last_became_peered":"2024-04-25T10:03:50.903629+0000","last_unstale":"2024-04-25T10:03:52.607565+0000","last_undegraded":"2024-04-25T10:03:52.607565+0000","last_fullsized":"2024-04-25T10:03:52.607565+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:54.432830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606080+0000","last_change":"2024-04-25T10:03:49.899336+0000","last_active":"2024-04-25T10:03:52.606080+0000","last_peered":"2024-04-25T10:03:52.606080+0000","last_clean":"2024-04-25T10:03:52.606080+0000","last_became_active":"2024-04-25T10:03:49.899096+0000","last_became_peered":"2024-04-25T10:03:49.899096+0000","last_unstale":"2024-04-25T10:03:52.606080+0000","last_undegraded":"2024-04-25T10:03:52.606080+0000","last_fullsized":"2024-04-25T10:03:52.606080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:49.112970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606989+0000","last_change":"2024-04-25T10:03:50.905181+0000","last_active":"2024-04-25T10:03:52.606989+0000","last_peered":"2024-04-25T10:03:52.606989+0000","last_clean":"2024-04-25T10:03:52.606989+0000","last_became_active":"2024-04-25T10:03:50.904575+0000","last_became_peered":"2024-04-25T10:03:50.904575+0000","last_unstale":"2024-04-25T10:03:52.606989+0000","last_undegraded":"2024-04-25T10:03:52.606989+0000","last_fullsized":"2024-04-25T10:03:52.606989+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:11.198548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606384+0000","last_change":"2024-04-25T10:03:49.898990+0000","last_active":"2024-04-25T10:03:52.606384+0000","last_peered":"2024-04-25T10:03:52.606384+0000","last_clean":"2024-04-25T10:03:52.606384+0000","last_became_active":"2024-04-25T10:03:49.898720+0000","last_became_peered":"2024-04-25T10:03:49.898720+0000","last_unstale":"2024-04-25T10:03:52.606384+0000","last_undegraded":"2024-04-25T10:03:52.606384+0000","last_fullsized":"2024-04-25T10:03:52.606384+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:10.676734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607120+0000","last_change":"2024-04-25T10:03:50.908185+0000","last_active":"2024-04-25T10:03:52.607120+0000","last_peered":"2024-04-25T10:03:52.607120+0000","last_clean":"2024-04-25T10:03:52.607120+0000","last_became_active":"2024-04-25T10:03:50.907912+0000","last_became_peered":"2024-04-25T10:03:50.907912+0000","last_unstale":"2024-04-25T10:03:52.607120+0000","last_undegraded":"2024-04-25T10:03:52.607120+0000","last_fullsized":"2024-04-25T10:03:52.607120+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:53.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883420+0000","last_change":"2024-04-25T10:03:49.901276+0000","last_active":"2024-04-25T10:03:51.883420+0000","last_peered":"2024-04-25T10:03:51.883420+0000","last_clean":"2024-04-25T10:03:51.883420+0000","last_became_active":"2024-04-25T10:03:49.900977+0000","last_became_peered":"2024-04-25T10:03:49.900977+0000","last_unstale":"2024-04-25T10:03:51.883420+0000","last_undegraded":"2024-04-25T10:03:51.883420+0000","last_fullsized":"2024-04-25T10:03:51.883420+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:08.283728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607115+0000","last_change":"2024-04-25T10:03:50.902808+0000","last_active":"2024-04-25T10:03:52.607115+0000","last_peered":"2024-04-25T10:03:52.607115+0000","last_clean":"2024-04-25T10:03:52.607115+0000","last_became_active":"2024-04-25T10:03:50.902105+0000","last_became_peered":"2024-04-25T10:03:50.902105+0000","last_unstale":"2024-04-25T10:03:52.607115+0000","last_undegraded":"2024-04-25T10:03:52.607115+0000","last_fullsized":"2024-04-25T10:03:52.607115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:02.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606534+0000","last_change":"2024-04-25T10:03:49.898957+0000","last_active":"2024-04-25T10:03:52.606534+0000","last_peered":"2024-04-25T10:03:52.606534+0000","last_clean":"2024-04-25T10:03:52.606534+0000","last_became_active":"2024-04-25T10:03:49.898681+0000","last_became_peered":"2024-04-25T10:03:49.898681+0000","last_unstale":"2024-04-25T10:03:52.606534+0000","last_undegraded":"2024-04-25T10:03:52.606534+0000","last_fullsized":"2024-04-25T10:03:52.606534+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:06.966070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606206+0000","last_change":"2024-04-25T10:03:50.896762+0000","last_active":"2024-04-25T10:03:52.606206+0000","last_peered":"2024-04-25T10:03:52.606206+0000","last_clean":"2024-04-25T10:03:52.606206+0000","last_became_active":"2024-04-25T10:03:50.896491+0000","last_became_peered":"2024-04-25T10:03:50.896491+0000","last_unstale":"2024-04-25T10:03:52.606206+0000","last_undegraded":"2024-04-25T10:03:52.606206+0000","last_fullsized":"2024-04-25T10:03:52.606206+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:53.355529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605608+0000","last_change":"2024-04-25T10:03:49.895700+0000","last_active":"2024-04-25T10:03:52.605608+0000","last_peered":"2024-04-25T10:03:52.605608+0000","last_clean":"2024-04-25T10:03:52.605608+0000","last_became_active":"2024-04-25T10:03:49.895008+0000","last_became_peered":"2024-04-25T10:03:49.895008+0000","last_unstale":"2024-04-25T10:03:52.605608+0000","last_undegraded":"2024-04-25T10:03:52.605608+0000","last_fullsized":"2024-04-25T10:03:52.605608+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:56.862010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883265+0000","last_change":"2024-04-25T10:03:50.899317+0000","last_active":"2024-04-25T10:03:51.883265+0000","last_peered":"2024-04-25T10:03:51.883265+0000","last_clean":"2024-04-25T10:03:51.883265+0000","last_became_active":"2024-04-25T10:03:50.899023+0000","last_became_peered":"2024-04-25T10:03:50.899023+0000","last_unstale":"2024-04-25T10:03:51.883265+0000","last_undegraded":"2024-04-25T10:03:51.883265+0000","last_fullsized":"2024-04-25T10:03:51.883265+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:15.590822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.886906+0000","last_change":"2024-04-25T10:03:49.895688+0000","last_active":"2024-04-25T10:03:51.886906+0000","last_peered":"2024-04-25T10:03:51.886906+0000","last_clean":"2024-04-25T10:03:51.886906+0000","last_became_active":"2024-04-25T10:03:49.895423+0000","last_became_peered":"2024-04-25T10:03:49.895423+0000","last_unstale":"2024-04-25T10:03:51.886906+0000","last_undegraded":"2024-04-25T10:03:51.886906+0000","last_fullsized":"2024-04-25T10:03:51.886906+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:56.051644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606308+0000","last_change":"2024-04-25T10:03:50.899825+0000","last_active":"2024-04-25T10:03:52.606308+0000","last_peered":"2024-04-25T10:03:52.606308+0000","last_clean":"2024-04-25T10:03:52.606308+0000","last_became_active":"2024-04-25T10:03:50.899548+0000","last_became_peered":"2024-04-25T10:03:50.899548+0000","last_unstale":"2024-04-25T10:03:52.606308+0000","last_undegraded":"2024-04-25T10:03:52.606308+0000","last_fullsized":"2024-04-25T10:03:52.606308+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:36.335317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605369+0000","last_change":"2024-04-25T10:03:49.895117+0000","last_active":"2024-04-25T10:03:52.605369+0000","last_peered":"2024-04-25T10:03:52.605369+0000","last_clean":"2024-04-25T10:03:52.605369+0000","last_became_active":"2024-04-25T10:03:49.894870+0000","last_became_peered":"2024-04-25T10:03:49.894870+0000","last_unstale":"2024-04-25T10:03:52.605369+0000","last_undegraded":"2024-04-25T10:03:52.605369+0000","last_fullsized":"2024-04-25T10:03:52.605369+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:39.270697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883101+0000","last_change":"2024-04-25T10:03:50.904119+0000","last_active":"2024-04-25T10:03:51.883101+0000","last_peered":"2024-04-25T10:03:51.883101+0000","last_clean":"2024-04-25T10:03:51.883101+0000","last_became_active":"2024-04-25T10:03:50.903818+0000","last_became_peered":"2024-04-25T10:03:50.903818+0000","last_unstale":"2024-04-25T10:03:51.883101+0000","last_undegraded":"2024-04-25T10:03:51.883101+0000","last_fullsized":"2024-04-25T10:03:51.883101+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.866446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887179+0000","last_change":"2024-04-25T10:03:49.895442+0000","last_active":"2024-04-25T10:03:51.887179+0000","last_peered":"2024-04-25T10:03:51.887179+0000","last_clean":"2024-04-25T10:03:51.887179+0000","last_became_active":"2024-04-25T10:03:49.895113+0000","last_became_peered":"2024-04-25T10:03:49.895113+0000","last_unstale":"2024-04-25T10:03:51.887179+0000","last_undegraded":"2024-04-25T10:03:51.887179+0000","last_fullsized":"2024-04-25T10:03:51.887179+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:48.487258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606580+0000","last_change":"2024-04-25T10:03:50.906274+0000","last_active":"2024-04-25T10:03:52.606580+0000","last_peered":"2024-04-25T10:03:52.606580+0000","last_clean":"2024-04-25T10:03:52.606580+0000","last_became_active":"2024-04-25T10:03:50.905500+0000","last_became_peered":"2024-04-25T10:03:50.905500+0000","last_unstale":"2024-04-25T10:03:52.606580+0000","last_undegraded":"2024-04-25T10:03:52.606580+0000","last_fullsized":"2024-04-25T10:03:52.606580+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:04.222131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607196+0000","last_change":"2024-04-25T10:03:49.898517+0000","last_active":"2024-04-25T10:03:52.607196+0000","last_peered":"2024-04-25T10:03:52.607196+0000","last_clean":"2024-04-25T10:03:52.607196+0000","last_became_active":"2024-04-25T10:03:49.898253+0000","last_became_peered":"2024-04-25T10:03:49.898253+0000","last_unstale":"2024-04-25T10:03:52.607196+0000","last_undegraded":"2024-04-25T10:03:52.607196+0000","last_fullsized":"2024-04-25T10:03:52.607196+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:36.201396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606431+0000","last_change":"2024-04-25T10:03:50.908336+0000","last_active":"2024-04-25T10:03:52.606431+0000","last_peered":"2024-04-25T10:03:52.606431+0000","last_clean":"2024-04-25T10:03:52.606431+0000","last_became_active":"2024-04-25T10:03:50.908035+0000","last_became_peered":"2024-04-25T10:03:50.908035+0000","last_unstale":"2024-04-25T10:03:52.606431+0000","last_undegraded":"2024-04-25T10:03:52.606431+0000","last_fullsized":"2024-04-25T10:03:52.606431+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.908343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"145'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607395+0000","last_change":"2024-04-25T10:03:49.899974+0000","last_active":"2024-04-25T10:03:52.607395+0000","last_peered":"2024-04-25T10:03:52.607395+0000","last_clean":"2024-04-25T10:03:52.607395+0000","last_became_active":"2024-04-25T10:03:49.899694+0000","last_became_peered":"2024-04-25T10:03:49.899694+0000","last_unstale":"2024-04-25T10:03:52.607395+0000","last_undegraded":"2024-04-25T10:03:52.607395+0000","last_fullsized":"2024-04-25T10:03:52.607395+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.352713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607287+0000","last_change":"2024-04-25T10:03:50.906673+0000","last_active":"2024-04-25T10:03:52.607287+0000","last_peered":"2024-04-25T10:03:52.607287+0000","last_clean":"2024-04-25T10:03:52.607287+0000","last_became_active":"2024-04-25T10:03:50.906372+0000","last_became_peered":"2024-04-25T10:03:50.906372+0000","last_unstale":"2024-04-25T10:03:52.607287+0000","last_undegraded":"2024-04-25T10:03:52.607287+0000","last_fullsized":"2024-04-25T10:03:52.607287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:13.082455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883464+0000","last_change":"2024-04-25T10:03:49.894817+0000","last_active":"2024-04-25T10:03:51.883464+0000","last_peered":"2024-04-25T10:03:51.883464+0000","last_clean":"2024-04-25T10:03:51.883464+0000","last_became_active":"2024-04-25T10:03:49.894481+0000","last_became_peered":"2024-04-25T10:03:49.894481+0000","last_unstale":"2024-04-25T10:03:51.883464+0000","last_undegraded":"2024-04-25T10:03:51.883464+0000","last_fullsized":"2024-04-25T10:03:51.883464+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:26.975577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606426+0000","last_change":"2024-04-25T10:03:50.903618+0000","last_active":"2024-04-25T10:03:52.606426+0000","last_peered":"2024-04-25T10:03:52.606426+0000","last_clean":"2024-04-25T10:03:52.606426+0000","last_became_active":"2024-04-25T10:03:50.903345+0000","last_became_peered":"2024-04-25T10:03:50.903345+0000","last_unstale":"2024-04-25T10:03:52.606426+0000","last_undegraded":"2024-04-25T10:03:52.606426+0000","last_fullsized":"2024-04-25T10:03:52.606426+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:34.104434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606693+0000","last_change":"2024-04-25T10:03:49.897166+0000","last_active":"2024-04-25T10:03:52.606693+0000","last_peered":"2024-04-25T10:03:52.606693+0000","last_clean":"2024-04-25T10:03:52.606693+0000","last_became_active":"2024-04-25T10:03:49.896766+0000","last_became_peered":"2024-04-25T10:03:49.896766+0000","last_unstale":"2024-04-25T10:03:52.606693+0000","last_undegraded":"2024-04-25T10:03:52.606693+0000","last_fullsized":"2024-04-25T10:03:52.606693+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:02.692459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887080+0000","last_change":"2024-04-25T10:03:50.898146+0000","last_active":"2024-04-25T10:03:51.887080+0000","last_peered":"2024-04-25T10:03:51.887080+0000","last_clean":"2024-04-25T10:03:51.887080+0000","last_became_active":"2024-04-25T10:03:50.895759+0000","last_became_peered":"2024-04-25T10:03:50.895759+0000","last_unstale":"2024-04-25T10:03:51.887080+0000","last_undegraded":"2024-04-25T10:03:51.887080+0000","last_fullsized":"2024-04-25T10:03:51.887080+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.908343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607808+0000","last_change":"2024-04-25T10:03:49.893523+0000","last_active":"2024-04-25T10:03:52.607808+0000","last_peered":"2024-04-25T10:03:52.607808+0000","last_clean":"2024-04-25T10:03:52.607808+0000","last_became_active":"2024-04-25T10:03:49.893233+0000","last_became_peered":"2024-04-25T10:03:49.893233+0000","last_unstale":"2024-04-25T10:03:52.607808+0000","last_undegraded":"2024-04-25T10:03:52.607808+0000","last_fullsized":"2024-04-25T10:03:52.607808+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:09.313997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606768+0000","last_change":"2024-04-25T10:03:50.903635+0000","last_active":"2024-04-25T10:03:52.606768+0000","last_peered":"2024-04-25T10:03:52.606768+0000","last_clean":"2024-04-25T10:03:52.606768+0000","last_became_active":"2024-04-25T10:03:50.903373+0000","last_became_peered":"2024-04-25T10:03:50.903373+0000","last_unstale":"2024-04-25T10:03:52.606768+0000","last_undegraded":"2024-04-25T10:03:52.606768+0000","last_fullsized":"2024-04-25T10:03:52.606768+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.867015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606578+0000","last_change":"2024-04-25T10:03:49.898200+0000","last_active":"2024-04-25T10:03:52.606578+0000","last_peered":"2024-04-25T10:03:52.606578+0000","last_clean":"2024-04-25T10:03:52.606578+0000","last_became_active":"2024-04-25T10:03:49.897929+0000","last_became_peered":"2024-04-25T10:03:49.897929+0000","last_unstale":"2024-04-25T10:03:52.606578+0000","last_undegraded":"2024-04-25T10:03:52.606578+0000","last_fullsized":"2024-04-25T10:03:52.606578+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.342597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.883127+0000","last_change":"2024-04-25T10:03:50.886604+0000","last_active":"2024-04-25T10:03:51.883127+0000","last_peered":"2024-04-25T10:03:51.883127+0000","last_clean":"2024-04-25T10:03:51.883127+0000","last_became_active":"2024-04-25T10:03:50.886318+0000","last_became_peered":"2024-04-25T10:03:50.886318+0000","last_unstale":"2024-04-25T10:03:51.883127+0000","last_undegraded":"2024-04-25T10:03:51.883127+0000","last_fullsized":"2024-04-25T10:03:51.883127+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:08.917053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.607706+0000","last_change":"2024-04-25T10:03:49.900360+0000","last_active":"2024-04-25T10:03:52.607706+0000","last_peered":"2024-04-25T10:03:52.607706+0000","last_clean":"2024-04-25T10:03:52.607706+0000","last_became_active":"2024-04-25T10:03:49.898980+0000","last_became_peered":"2024-04-25T10:03:49.898980+0000","last_unstale":"2024-04-25T10:03:52.607706+0000","last_undegraded":"2024-04-25T10:03:52.607706+0000","last_fullsized":"2024-04-25T10:03:52.607706+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:57.903136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887301+0000","last_change":"2024-04-25T10:03:50.902623+0000","last_active":"2024-04-25T10:03:51.887301+0000","last_peered":"2024-04-25T10:03:51.887301+0000","last_clean":"2024-04-25T10:03:51.887301+0000","last_became_active":"2024-04-25T10:03:50.901922+0000","last_became_peered":"2024-04-25T10:03:50.901922+0000","last_unstale":"2024-04-25T10:03:51.887301+0000","last_undegraded":"2024-04-25T10:03:51.887301+0000","last_fullsized":"2024-04-25T10:03:51.887301+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:11.681942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606109+0000","last_change":"2024-04-25T10:03:49.896765+0000","last_active":"2024-04-25T10:03:52.606109+0000","last_peered":"2024-04-25T10:03:52.606109+0000","last_clean":"2024-04-25T10:03:52.606109+0000","last_became_active":"2024-04-25T10:03:49.896465+0000","last_became_peered":"2024-04-25T10:03:49.896465+0000","last_unstale":"2024-04-25T10:03:52.606109+0000","last_undegraded":"2024-04-25T10:03:52.606109+0000","last_fullsized":"2024-04-25T10:03:52.606109+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:45.056797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:51.887243+0000","last_change":"2024-04-25T10:03:50.889603+0000","last_active":"2024-04-25T10:03:51.887243+0000","last_peered":"2024-04-25T10:03:51.887243+0000","last_clean":"2024-04-25T10:03:51.887243+0000","last_became_active":"2024-04-25T10:03:50.889294+0000","last_became_peered":"2024-04-25T10:03:50.889294+0000","last_unstale":"2024-04-25T10:03:51.887243+0000","last_undegraded":"2024-04-25T10:03:51.887243+0000","last_fullsized":"2024-04-25T10:03:51.887243+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:49.873764+0000","last_clean_scrub_stamp":"2024-04-25T10:03:49.873764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:46.062004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"144'6","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.605745+0000","last_change":"2024-04-25T10:03:49.895770+0000","last_active":"2024-04-25T10:03:52.605745+0000","last_peered":"2024-04-25T10:03:52.605745+0000","last_clean":"2024-04-25T10:03:52.605745+0000","last_became_active":"2024-04-25T10:03:49.895492+0000","last_became_peered":"2024-04-25T10:03:49.895492+0000","last_unstale":"2024-04-25T10:03:52.605745+0000","last_undegraded":"2024-04-25T10:03:52.605745+0000","last_fullsized":"2024-04-25T10:03:52.605745+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:48.868557+0000","last_clean_scrub_stamp":"2024-04-25T10:03:48.868557+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:00.748519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":400,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:03:52.606521+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:03:52.606521+0000","last_peered":"2024-04-25T10:03:52.606521+0000","last_clean":"2024-04-25T10:03:52.606521+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:03:52.606521+0000","last_undegraded":"2024-04-25T10:03:52.606521+0000","last_fullsized":"2024-04-25T10:03:52.606521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51592,"kb_used_data":2360,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320248,"statfs":{"total":96636764160,"available":96583933952,"internally_reserved":0,"allocated":2416640,"data_stored":1439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51008,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320832,"statfs":{"total":96636764160,"available":96584531968,"internally_reserved":0,"allocated":1822720,"data_stored":849679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46848,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324992,"statfs":{"total":96636764160,"available":96588791808,"internally_reserved":0,"allocated":1818624,"data_stored":849609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1830912,"data_stored":851221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59336,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312504,"statfs":{"total":96636764160,"available":96576004096,"internally_reserved":0,"allocated":1826816,"data_stored":850225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51020,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320820,"statfs":{"total":96636764160,"available":96584519680,"internally_reserved":0,"allocated":1826816,"data_stored":849721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51012,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320828,"statfs":{"total":96636764160,"available":96584527872,"internally_reserved":0,"allocated":1822720,"data_stored":849679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51612,"kb_used_data":2380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320228,"statfs":{"total":96636764160,"available":96583913472,"internally_reserved":0,"allocated":2437120,"data_stored":1442295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:58.531 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:03:58.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:03:58.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.665+0000 7fcc71fb0640 1 Processor -- start 2024-04-25T10:03:58.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.665+0000 7fcc71fb0640 1 -- start start 2024-04-25T10:03:58.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc71fb0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc6c0cc4f0 0x7fcc6c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc71fb0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc71fb0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c072280 con 0x7fcc6c15b270 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc71fb0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c0ce4c0 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc71fb0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c071270 con 0x7fcc6c0cdb20 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc6b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc6c0cc4f0 0x7fcc6c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc6bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcc6c15b270 0x7fcc6c15b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34968/0 (socket says 172.21.15.148:34968) 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc6bfff640 1 -- 172.21.15.148:0/2921521636 learned_addr learned my addr 172.21.15.148:0/2921521636 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.666+0000 7fcc6b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc6c0cc4f0 0x7fcc6c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33718/0 (socket says 172.21.15.148:33718) 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6affd640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/2921521636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 msgr2=0x7fcc6c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6b7fe640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/2921521636 >> v1:172.21.15.148:6789/0 conn(0x7fcc6c15b270 legacy=0x7fcc6c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/2921521636 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0714c0 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6a7fc640 1 -- 172.21.15.148:0/2921521636 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738723545 0 0) 0x7fcc6c072280 con 0x7fcc6c15b270 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6affd640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6b7fe640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c0cc8d0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00ba40 tx=0x7fcc5c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acacf1e4e9c8d982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6a7fc640 1 -- 172.21.15.148:0/2921521636 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c032750 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6a7fc640 1 -- 172.21.15.148:0/2921521636 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c030050 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.667+0000 7fcc6a7fc640 1 -- 172.21.15.148:0/2921521636 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c032b10 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 -- 172.21.15.148:0/2921521636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 msgr2=0x7fcc6c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c0cc8d0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00ba40 tx=0x7fcc5c02c850 comp rx=0 tx=0).stop 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 -- 172.21.15.148:0/2921521636 shutdown_connections 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/2921521636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c0cc8d0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 -- 172.21.15.148:0/2921521636 >> 172.21.15.148:0/2921521636 conn(0x7fcc6c06abe0 msgr2=0x7fcc6c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 -- 172.21.15.148:0/2921521636 shutdown_connections 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.668+0000 7fcc71fb0640 1 -- 172.21.15.148:0/2921521636 wait complete. 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.669+0000 7fcc71fb0640 1 Processor -- start 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.669+0000 7fcc71fb0640 1 -- start start 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc71fb0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc71fb0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc71fb0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6c15b270 0x7fcc6c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc71fb0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c071a80 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc71fb0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c15bd90 con 0x7fcc6c15b270 2024-04-25T10:03:58.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc71fb0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c071d60 con 0x7fcc6c0cdb20 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33720/0 (socket says 172.21.15.148:33720) 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/1150855073 learned_addr learned my addr 172.21.15.148:0/1150855073 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6bfff640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6c15b270 0x7fcc6c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6affd640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 msgr2=0x7fcc6c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6c15b270 msgr2=0x7fcc6c06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6c15b270 0x7fcc6c06d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6b7fe640 1 -- 172.21.15.148:0/1150855073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c154820 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.670+0000 7fcc6affd640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.671+0000 7fcc6b7fe640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c148560 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c02ce40 tx=0x7fcc5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.671+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c034c00 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.671+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c042070 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.671+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c159c80 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.671+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c03d460 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.671+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c0c8ae0 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.672+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c06f5a0 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.672+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc5c03d600 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.673+0000 7fcc53fff640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.673+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc5c0ab060 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.673+0000 7fcc6affd640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.673+0000 7fcc6affd640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c148c00 tx=0x7fcc60008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:58.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.677+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc6c06f5a0 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.917+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcc6c073a00 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.919+0000 7fcc53fff640 1 -- 172.21.15.148:0/1150855073 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7fcc6c154820 con 0x7fcc6c0cc4f0 2024-04-25T10:03:58.925 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:03:58.925 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:03:58.925 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:03:58.925 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:03:58.925 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:58.925 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:03:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 msgr2=0x7fcc5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c148c00 tx=0x7fcc60008040 comp rx=0 tx=0).stop 2024-04-25T10:03:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 msgr2=0x7fcc6c148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c148560 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c02ce40 tx=0x7fcc5c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 shutdown_connections 2024-04-25T10:03:58.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6c15b270 0x7fcc6c06d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6c0cdb20 0x7fcc6c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 --2- 172.21.15.148:0/1150855073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc6c0cc4f0 0x7fcc6c148560 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 >> 172.21.15.148:0/1150855073 conn(0x7fcc6c06abe0 msgr2=0x7fcc6c15b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 shutdown_connections 2024-04-25T10:03:58.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:58.921+0000 7fcc71fb0640 1 -- 172.21.15.148:0/1150855073 wait complete. 2024-04-25T10:03:58.938 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:59.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.030+0000 7ff33a441640 1 Processor -- start 2024-04-25T10:03:59.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.030+0000 7ff33a441640 1 -- start start 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff33a441640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff33a441640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff334099a40 0x7ff334099ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff33a441640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340cc3c0 con 0x7ff3340b0b90 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff33a441640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3340cccb0 con 0x7ff334099a40 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff33a441640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33415f400 con 0x7ff33415f020 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff3389de640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff334099a40 0x7ff334099ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff3337fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff3340b0b90 0x7ff3340b10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34970/0 (socket says 172.21.15.148:34970) 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.031+0000 7ff3337fe640 1 -- 172.21.15.148:0/1975825016 learned_addr learned my addr 172.21.15.148:0/1975825016 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff333fff640 1 --2- 172.21.15.148:0/1975825016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff3389de640 1 -- 172.21.15.148:0/1975825016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 msgr2=0x7ff3340cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff3389de640 1 --2- 172.21.15.148:0/1975825016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff3389de640 1 -- 172.21.15.148:0/1975825016 >> v1:172.21.15.148:6789/0 conn(0x7ff3340b0b90 legacy=0x7ff3340b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff3389de640 1 -- 172.21.15.148:0/1975825016 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340d0550 con 0x7ff334099a40 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff3389de640 1 --2- 172.21.15.148:0/1975825016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff334099ed0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009d90 tx=0x7ff32c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d45f3ddc2047e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff332ffd640 1 -- 172.21.15.148:0/1975825016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c007740 con 0x7ff334099a40 2024-04-25T10:03:59.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.032+0000 7ff332ffd640 1 -- 172.21.15.148:0/1975825016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0078e0 con 0x7ff334099a40 2024-04-25T10:03:59.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff332ffd640 1 -- 172.21.15.148:0/1975825016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c007cb0 con 0x7ff334099a40 2024-04-25T10:03:59.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 -- 172.21.15.148:0/1975825016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 msgr2=0x7ff334099ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 --2- 172.21.15.148:0/1975825016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff334099ed0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009d90 tx=0x7ff32c02a660 comp rx=0 tx=0).stop 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 -- 172.21.15.148:0/1975825016 shutdown_connections 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 --2- 172.21.15.148:0/1975825016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff334099ed0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 --2- 172.21.15.148:0/1975825016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 -- 172.21.15.148:0/1975825016 >> 172.21.15.148:0/1975825016 conn(0x7ff3340b3310 msgr2=0x7ff3340b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 -- 172.21.15.148:0/1975825016 shutdown_connections 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.033+0000 7ff33a441640 1 -- 172.21.15.148:0/1975825016 wait complete. 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.034+0000 7ff33a441640 1 Processor -- start 2024-04-25T10:03:59.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 -- start start 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff3340afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3340b0b90 0x7ff3340b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340cc940 con 0x7ff334099a40 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340cc660 con 0x7ff3340b0b90 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff33a441640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340cd120 con 0x7ff33415f020 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff333fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff3340afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff333fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff3340afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33748/0 (socket says 172.21.15.148:33748) 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.035+0000 7ff333fff640 1 -- 172.21.15.148:0/1130541538 learned_addr learned my addr 172.21.15.148:0/1130541538 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff3337fe640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3340b0b90 0x7ff3340b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff3389de640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff333fff640 1 -- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 msgr2=0x7ff3340b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff333fff640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff333fff640 1 -- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3340b0b90 msgr2=0x7ff3340b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff333fff640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3340b0b90 0x7ff3340b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff333fff640 1 -- 172.21.15.148:0/1130541538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33414a840 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff3389de640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff333fff640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff3340afc40 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff32400d5f0 tx=0x7ff32400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240125f0 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff324019070 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33414ab20 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240129d0 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.036+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33414b0e0 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.037+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff334073990 con 0x7ff334099a40 2024-04-25T10:03:59.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.037+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff324012b70 con 0x7ff334099a40 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.038+0000 7ff330ff9640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.038+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff32408a6f0 con 0x7ff334099a40 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.038+0000 7ff3337fe640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.038+0000 7ff3337fe640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff32006b4d0 0x7ff32006d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff33406d980 tx=0x7ff328009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.042+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff324059470 con 0x7ff334099a40 2024-04-25T10:03:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.250+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3340b77a0 con 0x7ff334099a40 2024-04-25T10:03:59.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.250+0000 7ff330ff9640 1 -- 172.21.15.148:0/1130541538 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4954 (secure 0 0 0) 0x7ff324058d70 con 0x7ff334099a40 2024-04-25T10:03:59.257 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:03:59.257 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":84,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":9855,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:03:52.226024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:03:59.257 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 84 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff32006b4d0 msgr2=0x7ff32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff32006b4d0 0x7ff32006d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff33406d980 tx=0x7ff328009040 comp rx=0 tx=0).stop 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 msgr2=0x7ff3340afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff3340afc40 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff32400d5f0 tx=0x7ff32400dab0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 shutdown_connections 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33415f020 0x7ff3340b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3340b0b90 0x7ff3340b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 --2- 172.21.15.148:0/1130541538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff334099a40 0x7ff3340afc40 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.252+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 >> 172.21.15.148:0/1130541538 conn(0x7ff3340b3310 msgr2=0x7ff334070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.253+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 shutdown_connections 2024-04-25T10:03:59.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.253+0000 7ff33a441640 1 -- 172.21.15.148:0/1130541538 wait complete. 2024-04-25T10:03:59.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:03:50.223124+0000', 'modified': '2024-04-25T10:03:52.226024+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9861': {'gid': 9861, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/3188696943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3188696943}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3188696943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:59.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:59.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:03:59.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.767+0000 7ff648088640 1 Processor -- start 2024-04-25T10:03:59.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.767+0000 7ff648088640 1 -- start start 2024-04-25T10:03:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.768+0000 7ff648088640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff640161710 0x7ff6400d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.768+0000 7ff648088640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64015e000 con 0x7ff6400ced80 2024-04-25T10:03:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.768+0000 7ff648088640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6400d1de0 con 0x7ff64015dc20 2024-04-25T10:03:59.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.768+0000 7ff648088640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff640161c60 con 0x7ff640161710 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.768+0000 7ff645623640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff6400ced80 0x7ff6400d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34980/0 (socket says 172.21.15.148:34980) 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.768+0000 7ff645623640 1 -- 172.21.15.148:0/2926878839 learned_addr learned my addr 172.21.15.148:0/2926878839 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 --2- 172.21.15.148:0/2926878839 >> v2:172.21.15.188:3300/0 conn(0x7ff640161710 0x7ff6400d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232979158 0 0) 0x7ff64015e000 con 0x7ff6400ced80 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62c003520 con 0x7ff6400ced80 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3651592686 0 0) 0x7ff62c003520 con 0x7ff6400ced80 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 >> v2:172.21.15.188:3300/0 conn(0x7ff640161710 msgr2=0x7ff6400d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 --2- 172.21.15.148:0/2926878839 >> v2:172.21.15.188:3300/0 conn(0x7ff640161710 0x7ff6400d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 >> v1:172.21.15.188:6789/0 conn(0x7ff64015dc20 legacy=0x7ff6400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff640161f40 con 0x7ff6400ced80 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff630001670 con 0x7ff6400ced80 2024-04-25T10:03:59.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 >> v1:172.21.15.148:6789/0 conn(0x7ff6400ced80 legacy=0x7ff6400d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff62c0039b0 0x7ff62c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff62c004800 0x7ff62c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff62c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640161c60 con 0x7ff62c0051c0 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00c060 con 0x7ff62c004800 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c00c260 con 0x7ff62c0039b0 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff62c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff645623640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff62c004800 0x7ff62c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff645e24640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff62c0039b0 0x7ff62c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 -- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff62c0039b0 msgr2=0x7ff62c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff62c0039b0 0x7ff62c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 -- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff62c004800 msgr2=0x7ff62c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff62c004800 0x7ff62c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.769+0000 7ff646625640 1 -- 172.21.15.148:0/2926878839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c003520 con 0x7ff62c0051c0 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.770+0000 7ff646625640 1 -- 172.21.15.148:0/2926878839 reap_dead start 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.770+0000 7ff646625640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff62c00b9c0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7ff638009ac0 tx=0x7ff62c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd8a8706ad2a03c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.770+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638009690 con 0x7ff62c0051c0 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.770+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638004020 con 0x7ff62c0051c0 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.770+0000 7ff644e22640 1 -- 172.21.15.148:0/2926878839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6380043d0 con 0x7ff62c0051c0 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 -- 172.21.15.148:0/2926878839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 msgr2=0x7ff62c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff62c00b9c0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7ff638009ac0 tx=0x7ff62c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 -- 172.21.15.148:0/2926878839 shutdown_connections 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 --2- 172.21.15.148:0/2926878839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff62c00b9c0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 -- 172.21.15.148:0/2926878839 >> 172.21.15.148:0/2926878839 conn(0x7ff6400d0420 msgr2=0x7ff6400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 -- 172.21.15.148:0/2926878839 shutdown_connections 2024-04-25T10:03:59.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.771+0000 7ff648088640 1 -- 172.21.15.148:0/2926878839 wait complete. 2024-04-25T10:03:59.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.772+0000 7ff648088640 1 Processor -- start 2024-04-25T10:03:59.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.772+0000 7ff648088640 1 -- start start 2024-04-25T10:03:59.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff640154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400c7300 0x7ff6400c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400c7cc0 0x7ff6400c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400b5e50 con 0x7ff62c0051c0 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640162500 con 0x7ff6400c7300 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400b5fd0 con 0x7ff6400c7cc0 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff640154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff640154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33770/0 (socket says 172.21.15.148:33770) 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 -- 172.21.15.148:0/4082281310 learned_addr learned my addr 172.21.15.148:0/4082281310 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 -- 172.21.15.148:0/4082281310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400c7cc0 msgr2=0x7ff6400c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400c7cc0 0x7ff6400c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 -- 172.21.15.148:0/4082281310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400c7300 msgr2=0x7ff6400c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400c7300 0x7ff6400c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 -- 172.21.15.148:0/4082281310 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400c8140 con 0x7ff62c0051c0 2024-04-25T10:03:59.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff645e24640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff640154f50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00f960 tx=0x7ff63c00fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff6367fc640 1 -- 172.21.15.148:0/4082281310 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c003b80 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff6367fc640 1 -- 172.21.15.148:0/4082281310 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c018070 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff6367fc640 1 -- 172.21.15.148:0/4082281310 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00d7f0 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff640157a40 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.773+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff640158000 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.775+0000 7ff6367fc640 1 -- 172.21.15.148:0/4082281310 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff63c00d990 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.775+0000 7ff6367fc640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff62406b500 0x7ff62406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.775+0000 7ff6367fc640 1 -- 172.21.15.148:0/4082281310 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff63c08a460 con 0x7ff62c0051c0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.775+0000 7ff645623640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff62406b500 0x7ff62406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.776+0000 7ff645623640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff62406b500 0x7ff62406d9c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff640067ec0 tx=0x7ff630000fb0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.776+0000 7ff64ad86740 1 Processor -- start 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.776+0000 7ff64ad86740 1 -- start start 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabacfd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabacd2f10 0x55aabad19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff64ad86740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabac35a10 con 0x55aabacfd230 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aababdeb30 con 0x55aabacd2f10 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aabac31c50 con 0x55aabad1aad0 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabacfd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabacfd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33786/0 (socket says 172.21.15.148:33786) 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 -- 172.21.15.148:0/2662570376 learned_addr learned my addr 172.21.15.148:0/2662570376 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff646625640 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabacd2f10 0x55aabad19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645623640 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 -- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 msgr2=0x55aabad1b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad1b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 -- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabacd2f10 msgr2=0x55aabad19190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabacd2f10 0x55aabad19190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.777+0000 7ff645e24640 1 -- 172.21.15.148:0/2662570376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aababf3e10 con 0x55aabacfd230 2024-04-25T10:03:59.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff645e24640 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabacfd610 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff63c060c30 tx=0x7ff63c007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f655f12faf4716b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff6237fe640 1 -- 172.21.15.148:0/2662570376 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c070020 con 0x55aabacfd230 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff6237fe640 1 -- 172.21.15.148:0/2662570376 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c05cba0 con 0x55aabacfd230 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff6237fe640 1 -- 172.21.15.148:0/2662570376 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c061160 con 0x55aabacfd230 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 -- 172.21.15.148:0/2662570376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 msgr2=0x55aabacfd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabacfd610 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff63c060c30 tx=0x7ff63c007d00 comp rx=0 tx=0).stop 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 -- 172.21.15.148:0/2662570376 shutdown_connections 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabacd2f10 0x55aabad19190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad1b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2662570376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabacfd610 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 -- 172.21.15.148:0/2662570376 >> 172.21.15.148:0/2662570376 conn(0x55aabaccf840 msgr2=0x55aabad18040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 -- 172.21.15.148:0/2662570376 shutdown_connections 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.778+0000 7ff64ad86740 1 -- 172.21.15.148:0/2662570376 wait complete. 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.779+0000 7ff64ad86740 1 -- start start 2024-04-25T10:03:59.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.779+0000 7ff64ad86740 1 Processor -- start 2024-04-25T10:03:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad1aad0 0x55aabad28380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad2c610 0x55aabacfbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabacd3cb0 con 0x55aabacfd230 2024-04-25T10:03:59.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aababf20c0 con 0x55aabad1aad0 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aabaccd770 con 0x55aabad2c610 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff64ad86740 10 client.0 _ll_register_callbacks cb 0x55aababe5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33794/0 (socket says 172.21.15.148:33794) 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 -- 172.21.15.148:0/2820118736 learned_addr learned my addr 172.21.15.148:0/2820118736 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff646625640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad2c610 0x55aabacfbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645623640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad1aad0 0x55aabad28380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 -- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad2c610 msgr2=0x55aabacfbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad2c610 0x55aabacfbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 -- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad1aad0 msgr2=0x55aabad28380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad1aad0 0x55aabad28380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 -- 172.21.15.148:0/2820118736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaba94a2d0 con 0x55aabacfd230 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff645e24640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16fd0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff63c088850 tx=0x7ff63c05b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d63ba1dc7178d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.780+0000 7ff620ff9640 1 -- 172.21.15.148:0/2820118736 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c074070 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff620ff9640 1 -- 172.21.15.148:0/2820118736 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff63c060cd0 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff63c05cba0 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff620ff9640 1 -- 172.21.15.148:0/2820118736 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63c070040 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 10 client.10044 fetch_fsmap learned FSMap version 84 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55aabaa146b0 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff620ff9640 1 -- 172.21.15.148:0/2820118736 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c05cba0 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff620ff9640 1 -- 172.21.15.148:0/2820118736 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff63c077370 con 0x55aabacfd230 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 10 client.10044 fetch_fsmap finished waiting for FSMap version 84 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 10 client.10044 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 20 client.10044 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacf96a0 0x55aabacf9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 4 client.10044 mds_command: new command op to 9861 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:59.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 --> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aabac94920 con 0x55aabacf96a0 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff645623640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacf96a0 0x55aabacf9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff645623640 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacf96a0 0x55aabacf9a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8604fea44bc1dc6b server_cookie=15a8d51127d5a1c8 in_seq=0 out_seq=0 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.781+0000 7ff620ff9640 10 client.10044 ms_handle_connect on v2:172.21.15.148:6836/3188696943 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff620ff9640 1 -- 172.21.15.148:0/2820118736 <== mds.0 v2:172.21.15.148:6836/3188696943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aabac94920 con 0x55aabacf96a0 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff620ff9640 10 client.10044 handle_command_reply: tid=0 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 client.10044 shutdown 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 msgr2=0x55aabad16fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16fd0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff63c088850 tx=0x7ff63c05b070 comp rx=0 tx=0).stop 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 shutdown_connections 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacf96a0 0x55aabacf9a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad2c610 0x55aabacfbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad1aad0 0x55aabad28380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff620ff9640 0 client.10044 ms_handle_reset on v2:172.21.15.148:6836/3188696943 2024-04-25T10:03:59.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 --2- 172.21.15.148:0/2820118736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16fd0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 >> 172.21.15.148:0/2820118736 conn(0x55aabacd2f10 msgr2=0x55aabad1b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 shutdown_connections 2024-04-25T10:03:59.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 1 -- 172.21.15.148:0/2820118736 wait complete. 2024-04-25T10:03:59.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.782+0000 7ff64ad86740 20 client.10044 trim_cache size 0 max 16384 2024-04-25T10:03:59.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.798+0000 7ff64ad86740 1 Processor -- start 2024-04-25T10:03:59.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.798+0000 7ff64ad86740 1 -- start start 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad16f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 0x55aabacf9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff64ad86740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401ffe20 con 0x55aabacfd230 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64015f530 con 0x55aabad2c610 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401ff820 con 0x55aabad1aad0 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33810/0 (socket says 172.21.15.148:33810) 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 -- 172.21.15.148:0/3505378812 learned_addr learned my addr 172.21.15.148:0/3505378812 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 -- 172.21.15.148:0/3505378812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 msgr2=0x55aabad16f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad16f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 -- 172.21.15.148:0/3505378812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 msgr2=0x55aabacf9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 0x55aabacf9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 -- 172.21.15.148:0/3505378812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aabacfcd70 con 0x55aabacfd230 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff645e24640 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16a20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0553d0 tx=0x7ff63c071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337bd09dbd8bdfc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff622ffd640 1 -- 172.21.15.148:0/3505378812 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c074070 con 0x55aabacfd230 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff622ffd640 1 -- 172.21.15.148:0/3505378812 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c056e10 con 0x55aabacfd230 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.799+0000 7ff622ffd640 1 -- 172.21.15.148:0/3505378812 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c054cf0 con 0x55aabacfd230 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 -- 172.21.15.148:0/3505378812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 msgr2=0x55aabad16a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16a20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0553d0 tx=0x7ff63c071000 comp rx=0 tx=0).stop 2024-04-25T10:03:59.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 -- 172.21.15.148:0/3505378812 shutdown_connections 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 0x55aabacf9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabad1aad0 0x55aabad16f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3505378812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabacfd230 0x55aabad16a20 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 -- 172.21.15.148:0/3505378812 >> 172.21.15.148:0/3505378812 conn(0x55aabacd2f10 msgr2=0x55aabacfff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 -- 172.21.15.148:0/3505378812 shutdown_connections 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 -- 172.21.15.148:0/3505378812 wait complete. 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 -- start start 2024-04-25T10:03:59.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.800+0000 7ff64ad86740 1 Processor -- start 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabacfd230 0x55aabacd2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 0x55aabad2e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 0x55aabad2ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabacd3cb0 con 0x55aabad1aad0 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aabaccd770 con 0x55aabad2c610 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aababf20c0 con 0x55aabacfd230 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff64ad86740 10 client.0 _ll_register_callbacks cb 0x55aabaccdc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff645623640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 0x55aabad2e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff645623640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 0x55aabad2e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33814/0 (socket says 172.21.15.148:33814) 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.801+0000 7ff645623640 1 -- 172.21.15.148:0/3468514646 learned_addr learned my addr 172.21.15.148:0/3468514646 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645623640 1 -- 172.21.15.148:0/3468514646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabacfd230 msgr2=0x55aabacd2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645623640 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabacfd230 0x55aabacd2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645623640 1 -- 172.21.15.148:0/3468514646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 msgr2=0x55aabad2ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645623640 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 0x55aabad2ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645623640 1 -- 172.21.15.148:0/3468514646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aabad2ecf0 con 0x55aabad1aad0 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645623640 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 0x55aabad2e7b0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x55aabacfb4e0 tx=0x7ff6300247e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608d96065cb41b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff620ff9640 1 -- 172.21.15.148:0/3468514646 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630019070 con 0x55aabad1aad0 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff620ff9640 1 -- 172.21.15.148:0/3468514646 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6300260b0 con 0x55aabad1aad0 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aabac42f50 con 0x55aabad1aad0 2024-04-25T10:03:59.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff620ff9640 1 -- 172.21.15.148:0/3468514646 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff630035050 con 0x55aabad1aad0 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 10 client.10050 fetch_fsmap learned FSMap version 84 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55aabad2ef00 con 0x55aabad1aad0 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff620ff9640 1 -- 172.21.15.148:0/3468514646 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aabac42f50 con 0x55aabad1aad0 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff620ff9640 1 -- 172.21.15.148:0/3468514646 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff630038020 con 0x55aabad1aad0 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 10 client.10050 fetch_fsmap finished waiting for FSMap version 84 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 10 client.10050 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 20 client.10050 populate_metadata read hostname 'smithi148' 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacfa110 0x55aabad1aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 4 client.10050 mds_command: new command op to 9861 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 --> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aabac3d2a0 con 0x55aabacfa110 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.802+0000 7ff645e24640 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacfa110 0x55aabad1aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff645e24640 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacfa110 0x55aabad1aeb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcae67c2c9859619 server_cookie=bb24923d4f6a8b6e in_seq=0 out_seq=0 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff620ff9640 10 client.10050 ms_handle_connect on v2:172.21.15.148:6836/3188696943 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff620ff9640 1 -- 172.21.15.148:0/3468514646 <== mds.0 v2:172.21.15.148:6836/3188696943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55aabac3d2a0 con 0x55aabacfa110 2024-04-25T10:03:59.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff620ff9640 10 client.10050 handle_command_reply: tid=0 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 client.10050 shutdown 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 msgr2=0x55aabad2e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 0x55aabad2e7b0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x55aabacfb4e0 tx=0x7ff6300247e0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 shutdown_connections 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55aabacfa110 0x55aabad1aeb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aabad2c610 0x55aabad2ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aabad1aad0 0x55aabad2e7b0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff620ff9640 0 client.10050 ms_handle_reset on v2:172.21.15.148:6836/3188696943 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 --2- 172.21.15.148:0/3468514646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55aabacfd230 0x55aabacd2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 >> 172.21.15.148:0/3468514646 conn(0x55aabacd2f10 msgr2=0x55aabad00eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 shutdown_connections 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 1 -- 172.21.15.148:0/3468514646 wait complete. 2024-04-25T10:03:59.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.803+0000 7ff64ad86740 20 client.10050 trim_cache size 0 max 16384 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 9861, 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 8.9245753029999992, 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 84, 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 145, 2024-04-25T10:03:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 145, 2024-04-25T10:03:59.819 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 13.976737524000001 2024-04-25T10:03:59.819 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:03:59.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff62406b500 msgr2=0x7ff62406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff62406b500 0x7ff62406d9c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff640067ec0 tx=0x7ff630000fb0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 msgr2=0x7ff640154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff640154f50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00f960 tx=0x7ff63c00fe20 comp rx=0 tx=0).stop 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 shutdown_connections 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff62406b500 0x7ff62406d9c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400c7cc0 0x7ff6400c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400c7300 0x7ff6400c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 --2- 172.21.15.148:0/4082281310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff62c0051c0 0x7ff640154f50 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 >> 172.21.15.148:0/4082281310 conn(0x7ff6400d0420 msgr2=0x7ff64006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 shutdown_connections 2024-04-25T10:03:59.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.804+0000 7ff648088640 1 -- 172.21.15.148:0/4082281310 wait complete. 2024-04-25T10:03:59.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:59.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.912+0000 7fb0b7b14640 1 Processor -- start 2024-04-25T10:03:59.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b7b14640 1 -- start start 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b7b14640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0b00af720 0x7fb0b00afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b7b14640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb0b00d02f0 0x7fb0b00d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b7b14640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b006fe00 con 0x7fb0b0158220 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b7b14640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b00b3670 con 0x7fb0b00af720 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b7b14640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b0070a10 con 0x7fb0b00d02f0 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b50af640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0b00af720 0x7fb0b00afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b58b0640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb0b0158220 0x7fb0b015ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:34990/0 (socket says 172.21.15.148:34990) 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b58b0640 1 -- 172.21.15.148:0/3058133960 learned_addr learned my addr 172.21.15.148:0/3058133960 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:03:59.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.913+0000 7fb0b50af640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb0b00af720 0x7fb0b00afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33816/0 (socket says 172.21.15.148:33816) 2024-04-25T10:03:59.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.914+0000 7fb0b50af640 1 -- 172.21.15.148:0/3058133960 >> v2:172.21.15.188:3300/0 conn(0x7fb0b00d02f0 msgr2=0x7fb0b00d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:59.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.914+0000 7fb0b50af640 1 --2- 172.21.15.148:0/3058133960 >> v2:172.21.15.188:3300/0 conn(0x7fb0b00d02f0 0x7fb0b00d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.914+0000 7fb0b50af640 1 -- 172.21.15.148:0/3058133960 >> v1:172.21.15.148:6789/0 conn(0x7fb0b0158220 legacy=0x7fb0b015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.914+0000 7fb0b50af640 1 -- 172.21.15.148:0/3058133960 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00700e0 con 0x7fb0b00af720 2024-04-25T10:03:59.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.914+0000 7fb0b48ae640 1 -- 172.21.15.148:0/3058133960 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057094209 0 0) 0x7fb0b006fe00 con 0x7fb0b0158220 2024-04-25T10:03:59.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.914+0000 7fb0b50af640 1 --2- 172.21.15.148:0/3058133960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b00af720 0x7fb0b00afba0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00726d0 tx=0x7fb0a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821670fe1650f467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b48ae640 1 -- 172.21.15.148:0/3058133960 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0037070 con 0x7fb0b00af720 2024-04-25T10:03:59.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b48ae640 1 -- 172.21.15.148:0/3058133960 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0032440 con 0x7fb0b00af720 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b48ae640 1 -- 172.21.15.148:0/3058133960 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0032810 con 0x7fb0b00af720 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 -- 172.21.15.148:0/3058133960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b00af720 msgr2=0x7fb0b00afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/3058133960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b00af720 0x7fb0b00afba0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00726d0 tx=0x7fb0a002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 -- 172.21.15.148:0/3058133960 shutdown_connections 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/3058133960 >> v2:172.21.15.188:3300/0 conn(0x7fb0b00d02f0 0x7fb0b00d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/3058133960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b00af720 0x7fb0b00afba0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 -- 172.21.15.148:0/3058133960 >> 172.21.15.148:0/3058133960 conn(0x7fb0b00b2370 msgr2=0x7fb0b00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 -- 172.21.15.148:0/3058133960 shutdown_connections 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.915+0000 7fb0b7b14640 1 -- 172.21.15.148:0/3058133960 wait complete. 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.916+0000 7fb0b7b14640 1 Processor -- start 2024-04-25T10:03:59.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.916+0000 7fb0b7b14640 1 -- start start 2024-04-25T10:03:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b7b14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b00af720 0x7fb0b014b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b7b14640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b00d02f0 0x7fb0b015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b7b14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 0x7fb0b015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b7b14640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b0158600 con 0x7fb0b0158220 2024-04-25T10:03:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b7b14640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b00706a0 con 0x7fb0b00af720 2024-04-25T10:03:59.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b7b14640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b0072960 con 0x7fb0b00d02f0 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b60b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 0x7fb0b015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b60b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 0x7fb0b015d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33830/0 (socket says 172.21.15.148:33830) 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b60b1640 1 -- 172.21.15.148:0/298396222 learned_addr learned my addr 172.21.15.148:0/298396222 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b58b0640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b00af720 0x7fb0b014b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.917+0000 7fb0b50af640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b00d02f0 0x7fb0b015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b60b1640 1 -- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b00d02f0 msgr2=0x7fb0b015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b60b1640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b00d02f0 0x7fb0b015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b60b1640 1 -- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b00af720 msgr2=0x7fb0b014b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b60b1640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b00af720 0x7fb0b014b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b60b1640 1 -- 172.21.15.148:0/298396222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b0158820 con 0x7fb0b0158220 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b60b1640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 0x7fb0b015d6e0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b440 tx=0x7fb0a800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8015020 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8011830 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b0158b00 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8011c20 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.918+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b01590c0 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.919+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b0157d60 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.919+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0a8018020 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.920+0000 7fb0a67fc640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb09406b4d0 0x7fb09406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.920+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb0a808a220 con 0x7fb0b0158220 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.920+0000 7fb0b58b0640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb09406b4d0 0x7fb09406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:59.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.920+0000 7fb0b58b0640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb09406b4d0 0x7fb09406d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0012e0 tx=0x7fb0ac007ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:59.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:03:59.924+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0a801f020 con 0x7fb0b0158220 2024-04-25T10:04:00.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.129+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0b0073990 con 0x7fb0b0158220 2024-04-25T10:04:00.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.129+0000 7fb0a67fc640 1 -- 172.21.15.148:0/298396222 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0a80172c0 con 0x7fb0b0158220 2024-04-25T10:04:00.136 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb09406b4d0 msgr2=0x7fb09406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb09406b4d0 0x7fb09406d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0012e0 tx=0x7fb0ac007ed0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 msgr2=0x7fb0b015d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 0x7fb0b015d6e0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800b440 tx=0x7fb0a800b900 comp rx=0 tx=0).stop 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 shutdown_connections 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb09406b4d0 0x7fb09406d990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0b0158220 0x7fb0b015d6e0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0b00d02f0 0x7fb0b015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 --2- 172.21.15.148:0/298396222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b00af720 0x7fb0b014b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:00.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 >> 172.21.15.148:0/298396222 conn(0x7fb0b00b2370 msgr2=0x7fb0b00cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:00.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 shutdown_connections 2024-04-25T10:04:00.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:00.131+0000 7fb0b7b14640 1 -- 172.21.15.148:0/298396222 wait complete. 2024-04-25T10:04:00.148 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:04:00.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:04:00.149 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:00.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.149 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:04:00.164 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:04:00.165 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42195sec preferred_lft 42195sec 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:04:00.166 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:00.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.168 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:04:00.227 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:04:00.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.228 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:04:00.284 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:04:00.285 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:04:00.285 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:04:00.285 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:04:00.285 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:04:00.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.285 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:04:00.285 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:00.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:04:00.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:04:00.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:04:00.286 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:00.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.369 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:04:00.424 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:04:00.424 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:04:00.424 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:00.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.424 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:04:00.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:00.488 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:00.543 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:00.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:00.544 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:00.598 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:04:00.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.598 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:00.681 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:00.681 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:00.682 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:04:00.689 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:00.689 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:00.689 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:04:00.689 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:04:00.690 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:04:00.694 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:04:00.695 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:04:00.695 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:04:00.695 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:04:00.695 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:04:00.695 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:04:00.695 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:04:00.696 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:04:00.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.697 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:00.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.761 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:04:00.823 INFO:teuthology.orchestra.run.smithi148.stdout:client8107 2024-04-25T10:04:00.823 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:00.823 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:00.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:00.878 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:00.878 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:00.879 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:04:00.879 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:00.879 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:00.879 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:00.879 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:00.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774943 s, 135 MB/s 2024-04-25T10:04:00.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871129 s, 120 MB/s 2024-04-25T10:04:00.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.964 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712778 s, 147 MB/s 2024-04-25T10:04:00.972 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.973 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.973 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754076 s, 139 MB/s 2024-04-25T10:04:00.980 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.981 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711825 s, 147 MB/s 2024-04-25T10:04:00.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068109 s, 154 MB/s 2024-04-25T10:04:00.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:00.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:00.995 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594455 s, 176 MB/s 2024-04-25T10:04:01.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.003 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.003 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701897 s, 149 MB/s 2024-04-25T10:04:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.011 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728123 s, 144 MB/s 2024-04-25T10:04:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.020 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779828 s, 134 MB/s 2024-04-25T10:04:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648253 s, 162 MB/s 2024-04-25T10:04:01.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.036 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797264 s, 132 MB/s 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647078 s, 162 MB/s 2024-04-25T10:04:01.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582271 s, 180 MB/s 2024-04-25T10:04:01.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712597 s, 147 MB/s 2024-04-25T10:04:01.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.067 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708708 s, 148 MB/s 2024-04-25T10:04:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.075 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751947 s, 139 MB/s 2024-04-25T10:04:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.083 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698565 s, 150 MB/s 2024-04-25T10:04:01.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665798 s, 157 MB/s 2024-04-25T10:04:01.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.097 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.097 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607847 s, 173 MB/s 2024-04-25T10:04:01.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793431 s, 132 MB/s 2024-04-25T10:04:01.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.114 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068425 s, 153 MB/s 2024-04-25T10:04:01.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599681 s, 175 MB/s 2024-04-25T10:04:01.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647574 s, 162 MB/s 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708995 s, 148 MB/s 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657049 s, 160 MB/s 2024-04-25T10:04:01.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.151 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.151 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680732 s, 154 MB/s 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568587 s, 184 MB/s 2024-04-25T10:04:01.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.166 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736663 s, 142 MB/s 2024-04-25T10:04:01.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688362 s, 152 MB/s 2024-04-25T10:04:01.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.181 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.181 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649091 s, 162 MB/s 2024-04-25T10:04:01.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.189 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.189 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667827 s, 157 MB/s 2024-04-25T10:04:01.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:01.190 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.255 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073448 s, 143 MB/s 2024-04-25T10:04:01.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.262 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.262 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669847 s, 157 MB/s 2024-04-25T10:04:01.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724056 s, 145 MB/s 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.279 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754557 s, 139 MB/s 2024-04-25T10:04:01.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.286 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650708 s, 161 MB/s 2024-04-25T10:04:01.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.293 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.293 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637751 s, 164 MB/s 2024-04-25T10:04:01.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.301 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006275 s, 167 MB/s 2024-04-25T10:04:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.308 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642371 s, 163 MB/s 2024-04-25T10:04:01.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664569 s, 158 MB/s 2024-04-25T10:04:01.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.324 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744958 s, 141 MB/s 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814209 s, 129 MB/s 2024-04-25T10:04:01.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.340 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.340 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620412 s, 169 MB/s 2024-04-25T10:04:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.348 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731904 s, 143 MB/s 2024-04-25T10:04:01.354 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.355 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581508 s, 180 MB/s 2024-04-25T10:04:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766296 s, 137 MB/s 2024-04-25T10:04:01.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.371 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721463 s, 145 MB/s 2024-04-25T10:04:01.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.379 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625323 s, 168 MB/s 2024-04-25T10:04:01.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.386 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.386 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636258 s, 165 MB/s 2024-04-25T10:04:01.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633664 s, 165 MB/s 2024-04-25T10:04:01.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692088 s, 152 MB/s 2024-04-25T10:04:01.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732724 s, 143 MB/s 2024-04-25T10:04:01.416 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.416 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.416 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617482 s, 170 MB/s 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068675 s, 153 MB/s 2024-04-25T10:04:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730416 s, 144 MB/s 2024-04-25T10:04:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.440 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669539 s, 157 MB/s 2024-04-25T10:04:01.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669191 s, 157 MB/s 2024-04-25T10:04:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728925 s, 144 MB/s 2024-04-25T10:04:01.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614476 s, 171 MB/s 2024-04-25T10:04:01.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.470 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.470 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067806 s, 155 MB/s 2024-04-25T10:04:01.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.477 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570193 s, 184 MB/s 2024-04-25T10:04:01.484 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.485 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709014 s, 148 MB/s 2024-04-25T10:04:01.492 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.492 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.492 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069028 s, 152 MB/s 2024-04-25T10:04:01.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.493 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:01.494 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:01.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.558 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.558 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653601 s, 160 MB/s 2024-04-25T10:04:01.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.565 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649811 s, 161 MB/s 2024-04-25T10:04:01.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.572 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.572 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659098 s, 159 MB/s 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.580 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658986 s, 159 MB/s 2024-04-25T10:04:01.586 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.587 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609177 s, 172 MB/s 2024-04-25T10:04:01.594 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.594 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.594 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602035 s, 174 MB/s 2024-04-25T10:04:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.602 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741332 s, 141 MB/s 2024-04-25T10:04:01.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.610 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.610 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744118 s, 141 MB/s 2024-04-25T10:04:01.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604806 s, 173 MB/s 2024-04-25T10:04:01.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.625 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661213 s, 159 MB/s 2024-04-25T10:04:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.633 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727996 s, 144 MB/s 2024-04-25T10:04:01.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634533 s, 165 MB/s 2024-04-25T10:04:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.648 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643779 s, 163 MB/s 2024-04-25T10:04:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.654 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570772 s, 184 MB/s 2024-04-25T10:04:01.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.662 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660991 s, 159 MB/s 2024-04-25T10:04:01.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608777 s, 172 MB/s 2024-04-25T10:04:01.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656847 s, 160 MB/s 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734115 s, 143 MB/s 2024-04-25T10:04:01.692 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.692 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.692 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687279 s, 153 MB/s 2024-04-25T10:04:01.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633375 s, 166 MB/s 2024-04-25T10:04:01.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.708 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761802 s, 138 MB/s 2024-04-25T10:04:01.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.716 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711953 s, 147 MB/s 2024-04-25T10:04:01.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551667 s, 190 MB/s 2024-04-25T10:04:01.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629791 s, 166 MB/s 2024-04-25T10:04:01.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639197 s, 164 MB/s 2024-04-25T10:04:01.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006418 s, 163 MB/s 2024-04-25T10:04:01.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590302 s, 178 MB/s 2024-04-25T10:04:01.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661353 s, 159 MB/s 2024-04-25T10:04:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621743 s, 169 MB/s 2024-04-25T10:04:01.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.773 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615089 s, 170 MB/s 2024-04-25T10:04:01.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640387 s, 164 MB/s 2024-04-25T10:04:01.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.787 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580043 s, 181 MB/s 2024-04-25T10:04:01.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:01.788 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:01.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716687 s, 146 MB/s 2024-04-25T10:04:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736967 s, 142 MB/s 2024-04-25T10:04:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708398 s, 148 MB/s 2024-04-25T10:04:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061067 s, 172 MB/s 2024-04-25T10:04:01.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.884 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.884 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733541 s, 143 MB/s 2024-04-25T10:04:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.892 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706744 s, 148 MB/s 2024-04-25T10:04:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.899 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648351 s, 162 MB/s 2024-04-25T10:04:01.908 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.908 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.908 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763104 s, 137 MB/s 2024-04-25T10:04:01.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.915 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645762 s, 162 MB/s 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.923 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645244 s, 163 MB/s 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.931 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734064 s, 143 MB/s 2024-04-25T10:04:01.939 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.939 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.939 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722002 s, 145 MB/s 2024-04-25T10:04:01.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.946 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643034 s, 163 MB/s 2024-04-25T10:04:01.953 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.954 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621613 s, 169 MB/s 2024-04-25T10:04:01.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.961 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611406 s, 172 MB/s 2024-04-25T10:04:01.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682049 s, 154 MB/s 2024-04-25T10:04:01.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.975 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607122 s, 173 MB/s 2024-04-25T10:04:01.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.982 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.983 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632365 s, 166 MB/s 2024-04-25T10:04:01.990 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.990 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.990 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658481 s, 159 MB/s 2024-04-25T10:04:01.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:01.998 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679812 s, 154 MB/s 2024-04-25T10:04:02.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.006 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076799 s, 137 MB/s 2024-04-25T10:04:02.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.014 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.014 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658972 s, 159 MB/s 2024-04-25T10:04:02.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.021 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644608 s, 163 MB/s 2024-04-25T10:04:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.027 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055525 s, 189 MB/s 2024-04-25T10:04:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.035 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064393 s, 163 MB/s 2024-04-25T10:04:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.043 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751649 s, 140 MB/s 2024-04-25T10:04:02.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.051 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741277 s, 141 MB/s 2024-04-25T10:04:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652825 s, 161 MB/s 2024-04-25T10:04:02.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.067 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.067 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735049 s, 143 MB/s 2024-04-25T10:04:02.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.074 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.074 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642027 s, 163 MB/s 2024-04-25T10:04:02.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.082 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.082 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705232 s, 149 MB/s 2024-04-25T10:04:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652951 s, 161 MB/s 2024-04-25T10:04:02.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:02.091 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:02.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.155 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.155 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637366 s, 165 MB/s 2024-04-25T10:04:02.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.162 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618429 s, 170 MB/s 2024-04-25T10:04:02.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.169 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061271 s, 171 MB/s 2024-04-25T10:04:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.177 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702805 s, 149 MB/s 2024-04-25T10:04:02.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639267 s, 164 MB/s 2024-04-25T10:04:02.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.192 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.192 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745931 s, 141 MB/s 2024-04-25T10:04:02.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.199 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642119 s, 163 MB/s 2024-04-25T10:04:02.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.207 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656067 s, 160 MB/s 2024-04-25T10:04:02.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688029 s, 152 MB/s 2024-04-25T10:04:02.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.222 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.222 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655837 s, 160 MB/s 2024-04-25T10:04:02.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.230 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734119 s, 143 MB/s 2024-04-25T10:04:02.238 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.238 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.238 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737576 s, 142 MB/s 2024-04-25T10:04:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637683 s, 164 MB/s 2024-04-25T10:04:02.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.253 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644928 s, 163 MB/s 2024-04-25T10:04:02.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.262 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776302 s, 135 MB/s 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075106 s, 140 MB/s 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.277 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578105 s, 181 MB/s 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.284 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006347 s, 165 MB/s 2024-04-25T10:04:02.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709368 s, 148 MB/s 2024-04-25T10:04:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656665 s, 160 MB/s 2024-04-25T10:04:02.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.307 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.307 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730776 s, 143 MB/s 2024-04-25T10:04:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.315 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710204 s, 148 MB/s 2024-04-25T10:04:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579935 s, 181 MB/s 2024-04-25T10:04:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569485 s, 184 MB/s 2024-04-25T10:04:02.335 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607172 s, 173 MB/s 2024-04-25T10:04:02.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669366 s, 157 MB/s 2024-04-25T10:04:02.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.351 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663959 s, 158 MB/s 2024-04-25T10:04:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.359 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719021 s, 146 MB/s 2024-04-25T10:04:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.367 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744966 s, 141 MB/s 2024-04-25T10:04:02.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.375 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006546 s, 160 MB/s 2024-04-25T10:04:02.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.381 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.382 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582809 s, 180 MB/s 2024-04-25T10:04:02.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.388 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.388 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555828 s, 189 MB/s 2024-04-25T10:04:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:02.389 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:02.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735855 s, 142 MB/s 2024-04-25T10:04:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737041 s, 142 MB/s 2024-04-25T10:04:02.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576663 s, 182 MB/s 2024-04-25T10:04:02.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626699 s, 167 MB/s 2024-04-25T10:04:02.485 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.485 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.485 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657633 s, 159 MB/s 2024-04-25T10:04:02.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.490 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627131 s, 167 MB/s 2024-04-25T10:04:02.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655326 s, 160 MB/s 2024-04-25T10:04:02.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759663 s, 138 MB/s 2024-04-25T10:04:02.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607812 s, 173 MB/s 2024-04-25T10:04:02.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719313 s, 146 MB/s 2024-04-25T10:04:02.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628359 s, 167 MB/s 2024-04-25T10:04:02.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595899 s, 176 MB/s 2024-04-25T10:04:02.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.542 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059569 s, 176 MB/s 2024-04-25T10:04:02.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.549 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643597 s, 163 MB/s 2024-04-25T10:04:02.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.556 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.556 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578642 s, 181 MB/s 2024-04-25T10:04:02.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720909 s, 145 MB/s 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777334 s, 135 MB/s 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636389 s, 165 MB/s 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638465 s, 164 MB/s 2024-04-25T10:04:02.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748376 s, 140 MB/s 2024-04-25T10:04:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617482 s, 170 MB/s 2024-04-25T10:04:02.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766931 s, 137 MB/s 2024-04-25T10:04:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.619 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006626 s, 158 MB/s 2024-04-25T10:04:02.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.626 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.627 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649653 s, 161 MB/s 2024-04-25T10:04:02.633 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.634 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670554 s, 156 MB/s 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.641 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616983 s, 170 MB/s 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.648 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567452 s, 185 MB/s 2024-04-25T10:04:02.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.656 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742801 s, 141 MB/s 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653692 s, 160 MB/s 2024-04-25T10:04:02.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.670 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560755 s, 187 MB/s 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641607 s, 163 MB/s 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633036 s, 166 MB/s 2024-04-25T10:04:02.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:02.685 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:02.685 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:02.686 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:04:02.686 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:02.686 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:02.686 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:02.686 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:02.749 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.750 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600611 s, 175 MB/s 2024-04-25T10:04:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.757 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644672 s, 163 MB/s 2024-04-25T10:04:02.764 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.764 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.764 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606497 s, 173 MB/s 2024-04-25T10:04:02.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.772 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.772 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729186 s, 144 MB/s 2024-04-25T10:04:02.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.779 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.779 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060885 s, 172 MB/s 2024-04-25T10:04:02.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.786 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.786 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060206 s, 174 MB/s 2024-04-25T10:04:02.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.792 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.792 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558614 s, 188 MB/s 2024-04-25T10:04:02.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.800 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.800 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682922 s, 154 MB/s 2024-04-25T10:04:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.807 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061586 s, 170 MB/s 2024-04-25T10:04:02.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.815 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.815 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006774 s, 155 MB/s 2024-04-25T10:04:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.823 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738184 s, 142 MB/s 2024-04-25T10:04:02.830 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.830 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.830 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579023 s, 181 MB/s 2024-04-25T10:04:02.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682361 s, 154 MB/s 2024-04-25T10:04:02.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627598 s, 167 MB/s 2024-04-25T10:04:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061571 s, 170 MB/s 2024-04-25T10:04:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655143 s, 160 MB/s 2024-04-25T10:04:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.867 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635934 s, 165 MB/s 2024-04-25T10:04:02.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.874 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604634 s, 173 MB/s 2024-04-25T10:04:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627743 s, 167 MB/s 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703957 s, 149 MB/s 2024-04-25T10:04:02.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.899 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903268 s, 116 MB/s 2024-04-25T10:04:02.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.907 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.907 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772594 s, 136 MB/s 2024-04-25T10:04:02.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.915 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.915 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072867 s, 144 MB/s 2024-04-25T10:04:02.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.924 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.924 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737894 s, 142 MB/s 2024-04-25T10:04:02.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.931 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.931 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626591 s, 167 MB/s 2024-04-25T10:04:02.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.938 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.939 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670697 s, 156 MB/s 2024-04-25T10:04:02.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.946 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625324 s, 168 MB/s 2024-04-25T10:04:02.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.953 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574288 s, 183 MB/s 2024-04-25T10:04:02.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.960 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.960 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664877 s, 158 MB/s 2024-04-25T10:04:02.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.967 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066233 s, 158 MB/s 2024-04-25T10:04:02.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.974 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.974 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602412 s, 174 MB/s 2024-04-25T10:04:02.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:02.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:02.981 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061681 s, 170 MB/s 2024-04-25T10:04:02.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:02.983 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:03.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.048 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765423 s, 137 MB/s 2024-04-25T10:04:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.055 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559232 s, 188 MB/s 2024-04-25T10:04:03.063 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.064 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.064 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775964 s, 135 MB/s 2024-04-25T10:04:03.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.071 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698683 s, 150 MB/s 2024-04-25T10:04:03.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.079 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.079 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072068 s, 145 MB/s 2024-04-25T10:04:03.087 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.088 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730115 s, 144 MB/s 2024-04-25T10:04:03.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.096 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720334 s, 146 MB/s 2024-04-25T10:04:03.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.103 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570836 s, 184 MB/s 2024-04-25T10:04:03.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.110 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653806 s, 160 MB/s 2024-04-25T10:04:03.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.117 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646264 s, 162 MB/s 2024-04-25T10:04:03.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.124 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651544 s, 161 MB/s 2024-04-25T10:04:03.132 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.132 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.132 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638883 s, 164 MB/s 2024-04-25T10:04:03.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580458 s, 181 MB/s 2024-04-25T10:04:03.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.146 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006334 s, 166 MB/s 2024-04-25T10:04:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638271 s, 164 MB/s 2024-04-25T10:04:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621162 s, 169 MB/s 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738482 s, 142 MB/s 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635717 s, 165 MB/s 2024-04-25T10:04:03.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674886 s, 155 MB/s 2024-04-25T10:04:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607291 s, 173 MB/s 2024-04-25T10:04:03.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.196 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.196 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554718 s, 189 MB/s 2024-04-25T10:04:03.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.204 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065967 s, 159 MB/s 2024-04-25T10:04:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.211 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651129 s, 161 MB/s 2024-04-25T10:04:03.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564458 s, 186 MB/s 2024-04-25T10:04:03.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.225 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.225 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660694 s, 159 MB/s 2024-04-25T10:04:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.232 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590471 s, 178 MB/s 2024-04-25T10:04:03.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.239 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.239 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631915 s, 166 MB/s 2024-04-25T10:04:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.247 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636573 s, 165 MB/s 2024-04-25T10:04:03.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.253 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.253 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586517 s, 179 MB/s 2024-04-25T10:04:03.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647715 s, 162 MB/s 2024-04-25T10:04:03.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066851 s, 157 MB/s 2024-04-25T10:04:03.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.276 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063673 s, 165 MB/s 2024-04-25T10:04:03.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:03.277 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:03.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.342 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.342 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754653 s, 139 MB/s 2024-04-25T10:04:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.350 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705815 s, 149 MB/s 2024-04-25T10:04:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.357 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591708 s, 177 MB/s 2024-04-25T10:04:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.364 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652951 s, 161 MB/s 2024-04-25T10:04:03.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.371 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.371 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615308 s, 170 MB/s 2024-04-25T10:04:03.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.378 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.378 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558884 s, 188 MB/s 2024-04-25T10:04:03.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.385 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.385 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065471 s, 160 MB/s 2024-04-25T10:04:03.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.393 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.393 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658067 s, 159 MB/s 2024-04-25T10:04:03.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.399 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.399 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566223 s, 185 MB/s 2024-04-25T10:04:03.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.406 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.406 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618265 s, 170 MB/s 2024-04-25T10:04:03.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.413 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.413 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611627 s, 171 MB/s 2024-04-25T10:04:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.421 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640307 s, 164 MB/s 2024-04-25T10:04:03.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.428 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.428 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626229 s, 167 MB/s 2024-04-25T10:04:03.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.435 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641552 s, 163 MB/s 2024-04-25T10:04:03.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.442 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.442 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635943 s, 165 MB/s 2024-04-25T10:04:03.449 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.449 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.449 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616193 s, 170 MB/s 2024-04-25T10:04:03.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.457 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658483 s, 159 MB/s 2024-04-25T10:04:03.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.464 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635405 s, 165 MB/s 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624395 s, 168 MB/s 2024-04-25T10:04:03.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.479 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.479 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744408 s, 141 MB/s 2024-04-25T10:04:03.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682888 s, 154 MB/s 2024-04-25T10:04:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.494 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618808 s, 169 MB/s 2024-04-25T10:04:03.502 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.502 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.502 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730436 s, 144 MB/s 2024-04-25T10:04:03.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.509 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.510 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063815 s, 164 MB/s 2024-04-25T10:04:03.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.518 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.518 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743217 s, 141 MB/s 2024-04-25T10:04:03.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660142 s, 159 MB/s 2024-04-25T10:04:03.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.534 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072592 s, 144 MB/s 2024-04-25T10:04:03.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064146 s, 163 MB/s 2024-04-25T10:04:03.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.549 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728176 s, 144 MB/s 2024-04-25T10:04:03.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713081 s, 147 MB/s 2024-04-25T10:04:03.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638927 s, 164 MB/s 2024-04-25T10:04:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720376 s, 146 MB/s 2024-04-25T10:04:03.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:03.575 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:03.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.640 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.640 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755924 s, 139 MB/s 2024-04-25T10:04:03.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.648 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.648 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660321 s, 159 MB/s 2024-04-25T10:04:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594547 s, 176 MB/s 2024-04-25T10:04:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.662 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637347 s, 165 MB/s 2024-04-25T10:04:03.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637383 s, 165 MB/s 2024-04-25T10:04:03.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.677 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.677 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679835 s, 154 MB/s 2024-04-25T10:04:03.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633587 s, 165 MB/s 2024-04-25T10:04:03.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.692 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671299 s, 156 MB/s 2024-04-25T10:04:03.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628315 s, 167 MB/s 2024-04-25T10:04:03.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662434 s, 158 MB/s 2024-04-25T10:04:03.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.713 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619923 s, 169 MB/s 2024-04-25T10:04:03.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736716 s, 142 MB/s 2024-04-25T10:04:03.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.729 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631857 s, 166 MB/s 2024-04-25T10:04:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746185 s, 141 MB/s 2024-04-25T10:04:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718665 s, 146 MB/s 2024-04-25T10:04:03.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.753 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648996 s, 162 MB/s 2024-04-25T10:04:03.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.760 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700562 s, 150 MB/s 2024-04-25T10:04:03.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.769 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.769 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751894 s, 139 MB/s 2024-04-25T10:04:03.776 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.776 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.776 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639322 s, 164 MB/s 2024-04-25T10:04:03.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.783 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.784 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662621 s, 158 MB/s 2024-04-25T10:04:03.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.791 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634214 s, 165 MB/s 2024-04-25T10:04:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.798 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617154 s, 170 MB/s 2024-04-25T10:04:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.805 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629805 s, 166 MB/s 2024-04-25T10:04:03.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.812 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631612 s, 166 MB/s 2024-04-25T10:04:03.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.819 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606348 s, 173 MB/s 2024-04-25T10:04:03.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.826 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.826 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574899 s, 182 MB/s 2024-04-25T10:04:03.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.833 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.833 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606833 s, 173 MB/s 2024-04-25T10:04:03.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617373 s, 170 MB/s 2024-04-25T10:04:03.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.848 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664289 s, 158 MB/s 2024-04-25T10:04:03.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.854 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.854 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005831 s, 180 MB/s 2024-04-25T10:04:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.862 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712458 s, 147 MB/s 2024-04-25T10:04:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629704 s, 167 MB/s 2024-04-25T10:04:03.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:03.871 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:03.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.936 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637125 s, 165 MB/s 2024-04-25T10:04:03.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.942 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.942 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654431 s, 160 MB/s 2024-04-25T10:04:03.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.949 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581256 s, 180 MB/s 2024-04-25T10:04:03.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.956 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.956 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061766 s, 170 MB/s 2024-04-25T10:04:03.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622389 s, 168 MB/s 2024-04-25T10:04:03.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.970 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570791 s, 184 MB/s 2024-04-25T10:04:03.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066099 s, 159 MB/s 2024-04-25T10:04:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.984 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060575 s, 173 MB/s 2024-04-25T10:04:03.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.991 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630195 s, 166 MB/s 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:03.998 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645746 s, 162 MB/s 2024-04-25T10:04:04.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.006 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657651 s, 159 MB/s 2024-04-25T10:04:04.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.013 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637553 s, 164 MB/s 2024-04-25T10:04:04.021 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731364 s, 143 MB/s 2024-04-25T10:04:04.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604533 s, 173 MB/s 2024-04-25T10:04:04.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.036 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646234 s, 162 MB/s 2024-04-25T10:04:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.042 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590598 s, 178 MB/s 2024-04-25T10:04:04.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.050 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706191 s, 148 MB/s 2024-04-25T10:04:04.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.058 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636257 s, 165 MB/s 2024-04-25T10:04:04.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752494 s, 139 MB/s 2024-04-25T10:04:04.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635802 s, 165 MB/s 2024-04-25T10:04:04.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.081 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.081 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636305 s, 165 MB/s 2024-04-25T10:04:04.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.089 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710538 s, 148 MB/s 2024-04-25T10:04:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.096 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647061 s, 162 MB/s 2024-04-25T10:04:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.103 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623562 s, 168 MB/s 2024-04-25T10:04:04.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.111 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710438 s, 148 MB/s 2024-04-25T10:04:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.118 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625989 s, 168 MB/s 2024-04-25T10:04:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.126 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662897 s, 158 MB/s 2024-04-25T10:04:04.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.134 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739771 s, 142 MB/s 2024-04-25T10:04:04.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.142 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722788 s, 145 MB/s 2024-04-25T10:04:04.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.149 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063568 s, 165 MB/s 2024-04-25T10:04:04.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.157 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.157 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648217 s, 162 MB/s 2024-04-25T10:04:04.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.165 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.165 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073931 s, 142 MB/s 2024-04-25T10:04:04.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:04.166 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:04.166 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:04.166 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:04:04.167 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:04.167 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:04.167 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:04.167 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:04.230 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.231 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.231 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590275 s, 178 MB/s 2024-04-25T10:04:04.238 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.238 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.238 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653627 s, 160 MB/s 2024-04-25T10:04:04.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.245 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.245 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615746 s, 170 MB/s 2024-04-25T10:04:04.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.252 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.252 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661209 s, 159 MB/s 2024-04-25T10:04:04.260 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.261 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730976 s, 143 MB/s 2024-04-25T10:04:04.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.268 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643814 s, 163 MB/s 2024-04-25T10:04:04.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.275 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.275 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647366 s, 162 MB/s 2024-04-25T10:04:04.283 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.283 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.283 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064504 s, 163 MB/s 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656784 s, 160 MB/s 2024-04-25T10:04:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.298 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751692 s, 139 MB/s 2024-04-25T10:04:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.306 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701306 s, 150 MB/s 2024-04-25T10:04:04.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.313 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.313 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056521 s, 186 MB/s 2024-04-25T10:04:04.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.320 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.320 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613545 s, 171 MB/s 2024-04-25T10:04:04.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743718 s, 141 MB/s 2024-04-25T10:04:04.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729064 s, 144 MB/s 2024-04-25T10:04:04.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.344 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.344 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718356 s, 146 MB/s 2024-04-25T10:04:04.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.352 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655552 s, 160 MB/s 2024-04-25T10:04:04.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055525 s, 189 MB/s 2024-04-25T10:04:04.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691425 s, 152 MB/s 2024-04-25T10:04:04.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.373 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630442 s, 166 MB/s 2024-04-25T10:04:04.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575324 s, 182 MB/s 2024-04-25T10:04:04.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622598 s, 168 MB/s 2024-04-25T10:04:04.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634996 s, 165 MB/s 2024-04-25T10:04:04.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.401 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.401 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582653 s, 180 MB/s 2024-04-25T10:04:04.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.408 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.408 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640205 s, 164 MB/s 2024-04-25T10:04:04.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.415 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.415 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062275 s, 168 MB/s 2024-04-25T10:04:04.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756038 s, 139 MB/s 2024-04-25T10:04:04.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627524 s, 167 MB/s 2024-04-25T10:04:04.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716346 s, 146 MB/s 2024-04-25T10:04:04.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723033 s, 145 MB/s 2024-04-25T10:04:04.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590506 s, 178 MB/s 2024-04-25T10:04:04.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.461 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612271 s, 171 MB/s 2024-04-25T10:04:04.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:04.462 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:04.463 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:04.463 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:04:04.463 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:04.463 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:04.463 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:04.463 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:04.527 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717192 s, 146 MB/s 2024-04-25T10:04:04.534 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.535 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619441 s, 169 MB/s 2024-04-25T10:04:04.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.542 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.542 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612241 s, 171 MB/s 2024-04-25T10:04:04.549 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721762 s, 145 MB/s 2024-04-25T10:04:04.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629988 s, 166 MB/s 2024-04-25T10:04:04.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.565 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.565 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713883 s, 147 MB/s 2024-04-25T10:04:04.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661906 s, 158 MB/s 2024-04-25T10:04:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.581 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757431 s, 138 MB/s 2024-04-25T10:04:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617517 s, 170 MB/s 2024-04-25T10:04:04.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.595 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.595 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596993 s, 176 MB/s 2024-04-25T10:04:04.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.602 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.602 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652522 s, 161 MB/s 2024-04-25T10:04:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609316 s, 172 MB/s 2024-04-25T10:04:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736386 s, 142 MB/s 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.625 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654541 s, 160 MB/s 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628873 s, 167 MB/s 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.639 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006165 s, 170 MB/s 2024-04-25T10:04:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.646 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632796 s, 166 MB/s 2024-04-25T10:04:04.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.653 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.653 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565738 s, 185 MB/s 2024-04-25T10:04:04.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.659 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.659 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548559 s, 191 MB/s 2024-04-25T10:04:04.665 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.666 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566502 s, 185 MB/s 2024-04-25T10:04:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.673 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638194 s, 164 MB/s 2024-04-25T10:04:04.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.681 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.681 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726654 s, 144 MB/s 2024-04-25T10:04:04.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.689 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.689 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720591 s, 146 MB/s 2024-04-25T10:04:04.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.696 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.696 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621433 s, 169 MB/s 2024-04-25T10:04:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.703 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613229 s, 171 MB/s 2024-04-25T10:04:04.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.710 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.710 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604417 s, 173 MB/s 2024-04-25T10:04:04.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.718 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.718 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702016 s, 149 MB/s 2024-04-25T10:04:04.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.726 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.726 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700781 s, 150 MB/s 2024-04-25T10:04:04.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.734 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.734 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731752 s, 143 MB/s 2024-04-25T10:04:04.741 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.742 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637628 s, 164 MB/s 2024-04-25T10:04:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.750 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744487 s, 141 MB/s 2024-04-25T10:04:04.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.757 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.757 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621559 s, 169 MB/s 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:04.758 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:04.758 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:04.759 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:04:04.759 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:04.759 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:04.759 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:04.759 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:04.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.823 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.824 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643275 s, 163 MB/s 2024-04-25T10:04:04.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.831 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683197 s, 153 MB/s 2024-04-25T10:04:04.837 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.838 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.838 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600685 s, 175 MB/s 2024-04-25T10:04:04.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.845 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.845 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613299 s, 171 MB/s 2024-04-25T10:04:04.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.852 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555615 s, 189 MB/s 2024-04-25T10:04:04.860 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.860 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.860 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753502 s, 139 MB/s 2024-04-25T10:04:04.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.867 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599324 s, 175 MB/s 2024-04-25T10:04:04.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.874 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636527 s, 165 MB/s 2024-04-25T10:04:04.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632869 s, 166 MB/s 2024-04-25T10:04:04.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747563 s, 140 MB/s 2024-04-25T10:04:04.896 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.896 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628565 s, 167 MB/s 2024-04-25T10:04:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.904 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607822 s, 173 MB/s 2024-04-25T10:04:04.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652905 s, 161 MB/s 2024-04-25T10:04:04.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.919 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748355 s, 140 MB/s 2024-04-25T10:04:04.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.927 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634324 s, 165 MB/s 2024-04-25T10:04:04.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.934 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060525 s, 173 MB/s 2024-04-25T10:04:04.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.944 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.944 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929784 s, 113 MB/s 2024-04-25T10:04:04.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.952 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.952 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072104 s, 145 MB/s 2024-04-25T10:04:04.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.961 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.961 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751727 s, 139 MB/s 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620323 s, 169 MB/s 2024-04-25T10:04:04.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.975 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.975 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606451 s, 173 MB/s 2024-04-25T10:04:04.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.981 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.981 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552796 s, 190 MB/s 2024-04-25T10:04:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.989 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676291 s, 155 MB/s 2024-04-25T10:04:04.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:04.997 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:04.997 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687393 s, 153 MB/s 2024-04-25T10:04:05.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.004 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.004 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610551 s, 172 MB/s 2024-04-25T10:04:05.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.012 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067535 s, 155 MB/s 2024-04-25T10:04:05.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631449 s, 166 MB/s 2024-04-25T10:04:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.027 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716824 s, 146 MB/s 2024-04-25T10:04:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651827 s, 161 MB/s 2024-04-25T10:04:05.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.042 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648504 s, 162 MB/s 2024-04-25T10:04:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619151 s, 169 MB/s 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633312 s, 166 MB/s 2024-04-25T10:04:05.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:05.057 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:05.057 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:05.057 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:04:05.057 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:05.057 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:05.058 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:05.058 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749155 s, 140 MB/s 2024-04-25T10:04:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752721 s, 139 MB/s 2024-04-25T10:04:05.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.139 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703119 s, 149 MB/s 2024-04-25T10:04:05.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.147 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073834 s, 142 MB/s 2024-04-25T10:04:05.155 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.155 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.155 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665112 s, 158 MB/s 2024-04-25T10:04:05.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.162 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.163 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627563 s, 167 MB/s 2024-04-25T10:04:05.967 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.968 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.968 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.804764 s, 1.3 MB/s 2024-04-25T10:04:05.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.977 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818143 s, 128 MB/s 2024-04-25T10:04:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:05.988 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103624 s, 101 MB/s 2024-04-25T10:04:05.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:05.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101947 s, 103 MB/s 2024-04-25T10:04:06.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.010 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.011 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100543 s, 104 MB/s 2024-04-25T10:04:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.022 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107051 s, 98.0 MB/s 2024-04-25T10:04:06.032 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.033 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917746 s, 114 MB/s 2024-04-25T10:04:06.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.041 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782505 s, 134 MB/s 2024-04-25T10:04:06.050 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.052 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079825 s, 131 MB/s 2024-04-25T10:04:06.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.059 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716898 s, 146 MB/s 2024-04-25T10:04:06.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.066 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.066 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592439 s, 177 MB/s 2024-04-25T10:04:06.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.073 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.073 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671742 s, 156 MB/s 2024-04-25T10:04:06.081 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.081 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.081 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695121 s, 151 MB/s 2024-04-25T10:04:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.089 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708465 s, 148 MB/s 2024-04-25T10:04:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.096 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651558 s, 161 MB/s 2024-04-25T10:04:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.104 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670888 s, 156 MB/s 2024-04-25T10:04:06.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.111 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.111 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643832 s, 163 MB/s 2024-04-25T10:04:06.118 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.118 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.118 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061745 s, 170 MB/s 2024-04-25T10:04:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.128 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077999 s, 134 MB/s 2024-04-25T10:04:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.134 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645701 s, 162 MB/s 2024-04-25T10:04:06.141 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.141 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.142 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627831 s, 167 MB/s 2024-04-25T10:04:06.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.150 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.150 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804878 s, 130 MB/s 2024-04-25T10:04:06.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.158 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071273 s, 147 MB/s 2024-04-25T10:04:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.167 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780555 s, 134 MB/s 2024-04-25T10:04:06.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.174 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635564 s, 165 MB/s 2024-04-25T10:04:06.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.182 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.182 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687948 s, 152 MB/s 2024-04-25T10:04:06.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:06.183 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:06.183 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:06.184 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:04:06.184 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:06.184 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:06.184 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:06.184 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.248 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655492 s, 160 MB/s 2024-04-25T10:04:06.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.258 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744242 s, 141 MB/s 2024-04-25T10:04:06.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.263 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.263 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595795 s, 176 MB/s 2024-04-25T10:04:06.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.270 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.271 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643571 s, 163 MB/s 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.277 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561126 s, 187 MB/s 2024-04-25T10:04:06.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.284 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.285 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643601 s, 163 MB/s 2024-04-25T10:04:06.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.292 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.292 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644601 s, 163 MB/s 2024-04-25T10:04:06.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.299 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.299 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654519 s, 160 MB/s 2024-04-25T10:04:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.306 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568586 s, 184 MB/s 2024-04-25T10:04:06.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.314 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.314 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723541 s, 145 MB/s 2024-04-25T10:04:06.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687043 s, 153 MB/s 2024-04-25T10:04:06.329 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.330 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688616 s, 152 MB/s 2024-04-25T10:04:06.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.338 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.338 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716637 s, 146 MB/s 2024-04-25T10:04:06.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682512 s, 154 MB/s 2024-04-25T10:04:06.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.353 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.353 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602458 s, 174 MB/s 2024-04-25T10:04:06.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.361 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.361 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759531 s, 138 MB/s 2024-04-25T10:04:06.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.368 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.368 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652813 s, 161 MB/s 2024-04-25T10:04:06.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.375 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.375 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593214 s, 177 MB/s 2024-04-25T10:04:06.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.382 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.382 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585953 s, 179 MB/s 2024-04-25T10:04:06.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.389 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.389 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618036 s, 170 MB/s 2024-04-25T10:04:06.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.395 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577284 s, 182 MB/s 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639749 s, 164 MB/s 2024-04-25T10:04:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.410 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616473 s, 170 MB/s 2024-04-25T10:04:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629755 s, 167 MB/s 2024-04-25T10:04:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.425 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694091 s, 151 MB/s 2024-04-25T10:04:06.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.433 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.433 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746134 s, 141 MB/s 2024-04-25T10:04:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.441 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636597 s, 165 MB/s 2024-04-25T10:04:06.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.448 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.448 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643387 s, 163 MB/s 2024-04-25T10:04:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713796 s, 147 MB/s 2024-04-25T10:04:06.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.463 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.463 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573222 s, 183 MB/s 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581451 s, 180 MB/s 2024-04-25T10:04:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666179 s, 157 MB/s 2024-04-25T10:04:06.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:06.478 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:06.478 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:06.479 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:04:06.479 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:06.479 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:06.479 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:06.479 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:06.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.541 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.542 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589337 s, 178 MB/s 2024-04-25T10:04:06.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.550 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.550 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734078 s, 143 MB/s 2024-04-25T10:04:06.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.557 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.557 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637824 s, 164 MB/s 2024-04-25T10:04:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.564 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609946 s, 172 MB/s 2024-04-25T10:04:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.571 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635596 s, 165 MB/s 2024-04-25T10:04:06.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.579 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.579 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680179 s, 154 MB/s 2024-04-25T10:04:06.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.587 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715852 s, 146 MB/s 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581201 s, 180 MB/s 2024-04-25T10:04:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.601 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065607 s, 160 MB/s 2024-04-25T10:04:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.609 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746178 s, 141 MB/s 2024-04-25T10:04:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680359 s, 154 MB/s 2024-04-25T10:04:06.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064085 s, 164 MB/s 2024-04-25T10:04:06.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.632 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.632 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644614 s, 163 MB/s 2024-04-25T10:04:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.639 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643617 s, 163 MB/s 2024-04-25T10:04:06.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.647 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.647 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768922 s, 136 MB/s 2024-04-25T10:04:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.655 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630937 s, 166 MB/s 2024-04-25T10:04:06.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.661 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.661 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575553 s, 182 MB/s 2024-04-25T10:04:06.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.669 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.669 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652805 s, 161 MB/s 2024-04-25T10:04:06.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615856 s, 170 MB/s 2024-04-25T10:04:06.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744824 s, 141 MB/s 2024-04-25T10:04:06.691 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.692 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.692 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006613 s, 159 MB/s 2024-04-25T10:04:06.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.699 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647041 s, 162 MB/s 2024-04-25T10:04:06.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.706 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631549 s, 166 MB/s 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.714 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638879 s, 164 MB/s 2024-04-25T10:04:06.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072525 s, 145 MB/s 2024-04-25T10:04:06.729 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699901 s, 150 MB/s 2024-04-25T10:04:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662815 s, 158 MB/s 2024-04-25T10:04:06.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619185 s, 169 MB/s 2024-04-25T10:04:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625601 s, 168 MB/s 2024-04-25T10:04:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637335 s, 165 MB/s 2024-04-25T10:04:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651646 s, 161 MB/s 2024-04-25T10:04:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646481 s, 162 MB/s 2024-04-25T10:04:06.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:06.775 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:06.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.840 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676814 s, 155 MB/s 2024-04-25T10:04:06.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.846 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.847 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638296 s, 164 MB/s 2024-04-25T10:04:06.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.853 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.853 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598825 s, 175 MB/s 2024-04-25T10:04:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.861 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711111 s, 147 MB/s 2024-04-25T10:04:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.869 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711401 s, 147 MB/s 2024-04-25T10:04:06.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.876 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.876 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589791 s, 178 MB/s 2024-04-25T10:04:06.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.883 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.883 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630127 s, 166 MB/s 2024-04-25T10:04:06.890 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.891 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.891 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650537 s, 161 MB/s 2024-04-25T10:04:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.898 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659162 s, 159 MB/s 2024-04-25T10:04:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659342 s, 159 MB/s 2024-04-25T10:04:06.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681454 s, 154 MB/s 2024-04-25T10:04:06.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567697 s, 185 MB/s 2024-04-25T10:04:06.927 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.928 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.928 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677703 s, 155 MB/s 2024-04-25T10:04:06.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.935 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.935 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688564 s, 152 MB/s 2024-04-25T10:04:06.943 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.943 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.943 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702626 s, 149 MB/s 2024-04-25T10:04:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.950 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645301 s, 162 MB/s 2024-04-25T10:04:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.957 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585382 s, 179 MB/s 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.964 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.965 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640497 s, 164 MB/s 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.971 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583653 s, 180 MB/s 2024-04-25T10:04:06.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570354 s, 184 MB/s 2024-04-25T10:04:06.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635679 s, 165 MB/s 2024-04-25T10:04:06.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.992 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564404 s, 186 MB/s 2024-04-25T10:04:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:06.999 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626374 s, 167 MB/s 2024-04-25T10:04:07.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.006 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.006 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679462 s, 154 MB/s 2024-04-25T10:04:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.013 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555956 s, 189 MB/s 2024-04-25T10:04:07.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056528 s, 185 MB/s 2024-04-25T10:04:07.027 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.028 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.028 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728516 s, 144 MB/s 2024-04-25T10:04:07.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.035 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.035 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570001 s, 184 MB/s 2024-04-25T10:04:07.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.041 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.042 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646911 s, 162 MB/s 2024-04-25T10:04:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640625 s, 164 MB/s 2024-04-25T10:04:07.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.058 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764695 s, 137 MB/s 2024-04-25T10:04:07.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.065 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714175 s, 147 MB/s 2024-04-25T10:04:07.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:07.066 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:07.067 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:07.067 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:04:07.067 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:07.067 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:07.067 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:07.067 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:07.132 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.133 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735894 s, 142 MB/s 2024-04-25T10:04:07.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.139 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590691 s, 178 MB/s 2024-04-25T10:04:07.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.147 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673104 s, 156 MB/s 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623133 s, 168 MB/s 2024-04-25T10:04:07.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.161 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619158 s, 169 MB/s 2024-04-25T10:04:07.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.169 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.169 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063671 s, 165 MB/s 2024-04-25T10:04:07.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.176 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647511 s, 162 MB/s 2024-04-25T10:04:07.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655399 s, 160 MB/s 2024-04-25T10:04:07.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580921 s, 181 MB/s 2024-04-25T10:04:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.197 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629113 s, 167 MB/s 2024-04-25T10:04:07.204 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.205 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662436 s, 158 MB/s 2024-04-25T10:04:07.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.212 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.212 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623266 s, 168 MB/s 2024-04-25T10:04:07.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.219 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.219 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644046 s, 163 MB/s 2024-04-25T10:04:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.226 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650643 s, 161 MB/s 2024-04-25T10:04:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.234 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638472 s, 164 MB/s 2024-04-25T10:04:07.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739041 s, 142 MB/s 2024-04-25T10:04:07.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.249 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620549 s, 169 MB/s 2024-04-25T10:04:07.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.256 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.256 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632926 s, 166 MB/s 2024-04-25T10:04:07.264 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.264 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.264 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695552 s, 151 MB/s 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711493 s, 147 MB/s 2024-04-25T10:04:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.281 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690098 s, 152 MB/s 2024-04-25T10:04:07.286 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.287 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.287 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587032 s, 179 MB/s 2024-04-25T10:04:07.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.294 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.294 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660344 s, 159 MB/s 2024-04-25T10:04:07.301 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639486 s, 164 MB/s 2024-04-25T10:04:07.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.310 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.310 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738143 s, 142 MB/s 2024-04-25T10:04:07.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.317 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631459 s, 166 MB/s 2024-04-25T10:04:07.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.323 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.324 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575333 s, 182 MB/s 2024-04-25T10:04:07.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.332 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.332 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736918 s, 142 MB/s 2024-04-25T10:04:07.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.339 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.339 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629648 s, 167 MB/s 2024-04-25T10:04:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.346 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653676 s, 160 MB/s 2024-04-25T10:04:07.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.355 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.355 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756194 s, 139 MB/s 2024-04-25T10:04:07.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.363 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.363 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750334 s, 140 MB/s 2024-04-25T10:04:07.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:07.364 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:07.364 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:07.365 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:04:07.365 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:07.365 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:07.365 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:07.365 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:07.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.430 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.430 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740307 s, 142 MB/s 2024-04-25T10:04:07.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.438 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699236 s, 150 MB/s 2024-04-25T10:04:07.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.445 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.445 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672892 s, 156 MB/s 2024-04-25T10:04:07.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.454 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731773 s, 143 MB/s 2024-04-25T10:04:07.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.462 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.462 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715618 s, 147 MB/s 2024-04-25T10:04:07.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590989 s, 177 MB/s 2024-04-25T10:04:07.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627824 s, 167 MB/s 2024-04-25T10:04:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.482 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057196 s, 183 MB/s 2024-04-25T10:04:07.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.490 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718962 s, 146 MB/s 2024-04-25T10:04:07.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.498 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.498 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649358 s, 161 MB/s 2024-04-25T10:04:07.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.505 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.505 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619091 s, 169 MB/s 2024-04-25T10:04:07.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.513 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671314 s, 156 MB/s 2024-04-25T10:04:07.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748969 s, 140 MB/s 2024-04-25T10:04:07.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.529 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.529 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704109 s, 149 MB/s 2024-04-25T10:04:07.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.536 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063356 s, 166 MB/s 2024-04-25T10:04:07.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.544 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.544 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725937 s, 144 MB/s 2024-04-25T10:04:07.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.552 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.552 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665806 s, 157 MB/s 2024-04-25T10:04:07.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.559 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.559 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653162 s, 161 MB/s 2024-04-25T10:04:07.566 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.567 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.567 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669446 s, 157 MB/s 2024-04-25T10:04:07.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.573 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.573 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580929 s, 180 MB/s 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757685 s, 138 MB/s 2024-04-25T10:04:07.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.588 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.588 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558345 s, 188 MB/s 2024-04-25T10:04:07.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.597 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695016 s, 151 MB/s 2024-04-25T10:04:07.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.604 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.604 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716193 s, 146 MB/s 2024-04-25T10:04:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.612 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715104 s, 147 MB/s 2024-04-25T10:04:07.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.619 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.620 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629102 s, 167 MB/s 2024-04-25T10:04:07.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.628 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719242 s, 146 MB/s 2024-04-25T10:04:07.634 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.635 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613564 s, 171 MB/s 2024-04-25T10:04:07.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.642 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622199 s, 169 MB/s 2024-04-25T10:04:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.649 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631795 s, 166 MB/s 2024-04-25T10:04:07.655 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.656 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.656 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584714 s, 179 MB/s 2024-04-25T10:04:07.662 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.663 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.663 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602378 s, 174 MB/s 2024-04-25T10:04:07.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.728 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707265 s, 148 MB/s 2024-04-25T10:04:07.735 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.736 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675572 s, 155 MB/s 2024-04-25T10:04:07.743 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.743 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706085 s, 149 MB/s 2024-04-25T10:04:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.752 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730921 s, 143 MB/s 2024-04-25T10:04:07.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.760 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.760 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747378 s, 140 MB/s 2024-04-25T10:04:07.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.768 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.768 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703885 s, 149 MB/s 2024-04-25T10:04:07.776 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.776 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.776 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710145 s, 148 MB/s 2024-04-25T10:04:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.784 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693293 s, 151 MB/s 2024-04-25T10:04:07.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.791 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.791 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641293 s, 164 MB/s 2024-04-25T10:04:07.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.798 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.798 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624747 s, 168 MB/s 2024-04-25T10:04:07.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626303 s, 167 MB/s 2024-04-25T10:04:07.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.814 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.814 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723534 s, 145 MB/s 2024-04-25T10:04:07.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663598 s, 158 MB/s 2024-04-25T10:04:07.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.828 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584605 s, 179 MB/s 2024-04-25T10:04:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.835 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635276 s, 165 MB/s 2024-04-25T10:04:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625094 s, 168 MB/s 2024-04-25T10:04:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.850 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655537 s, 160 MB/s 2024-04-25T10:04:07.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.858 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075382 s, 139 MB/s 2024-04-25T10:04:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.866 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662833 s, 158 MB/s 2024-04-25T10:04:07.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.873 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620671 s, 169 MB/s 2024-04-25T10:04:07.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.880 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662142 s, 158 MB/s 2024-04-25T10:04:07.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.888 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652159 s, 161 MB/s 2024-04-25T10:04:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.894 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.895 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606795 s, 173 MB/s 2024-04-25T10:04:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.902 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631166 s, 166 MB/s 2024-04-25T10:04:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634721 s, 165 MB/s 2024-04-25T10:04:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.916 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581701 s, 180 MB/s 2024-04-25T10:04:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639501 s, 164 MB/s 2024-04-25T10:04:07.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.930 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.930 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613678 s, 171 MB/s 2024-04-25T10:04:07.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.937 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.937 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609058 s, 172 MB/s 2024-04-25T10:04:07.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.945 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670314 s, 156 MB/s 2024-04-25T10:04:07.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.951 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.951 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576018 s, 182 MB/s 2024-04-25T10:04:07.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:07.959 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:07.959 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648564 s, 162 MB/s 2024-04-25T10:04:07.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:07.960 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:07.960 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:07.960 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:04:07.961 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:07.961 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:07.961 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:07.961 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:08.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.026 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731993 s, 143 MB/s 2024-04-25T10:04:08.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.034 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.034 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731797 s, 143 MB/s 2024-04-25T10:04:08.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.042 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.042 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709846 s, 148 MB/s 2024-04-25T10:04:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.049 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622318 s, 168 MB/s 2024-04-25T10:04:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775116 s, 135 MB/s 2024-04-25T10:04:08.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.065 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.065 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642321 s, 163 MB/s 2024-04-25T10:04:08.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.072 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.072 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603611 s, 174 MB/s 2024-04-25T10:04:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729829 s, 144 MB/s 2024-04-25T10:04:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.089 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740146 s, 142 MB/s 2024-04-25T10:04:08.095 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.095 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.095 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571063 s, 184 MB/s 2024-04-25T10:04:08.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.102 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.102 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632485 s, 166 MB/s 2024-04-25T10:04:08.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.110 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.110 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631228 s, 166 MB/s 2024-04-25T10:04:08.116 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.117 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.117 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560571 s, 187 MB/s 2024-04-25T10:04:08.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.123 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593254 s, 177 MB/s 2024-04-25T10:04:08.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.131 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.131 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659085 s, 159 MB/s 2024-04-25T10:04:08.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.139 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074839 s, 140 MB/s 2024-04-25T10:04:08.146 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.147 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.147 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711434 s, 147 MB/s 2024-04-25T10:04:08.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.154 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.154 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647944 s, 162 MB/s 2024-04-25T10:04:08.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.161 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617703 s, 170 MB/s 2024-04-25T10:04:08.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605921 s, 173 MB/s 2024-04-25T10:04:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065644 s, 160 MB/s 2024-04-25T10:04:08.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.183 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.183 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649005 s, 162 MB/s 2024-04-25T10:04:08.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.190 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.190 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648111 s, 162 MB/s 2024-04-25T10:04:08.198 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728301 s, 144 MB/s 2024-04-25T10:04:08.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.206 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065356 s, 160 MB/s 2024-04-25T10:04:08.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.213 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.213 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646172 s, 162 MB/s 2024-04-25T10:04:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614807 s, 171 MB/s 2024-04-25T10:04:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641792 s, 163 MB/s 2024-04-25T10:04:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629543 s, 167 MB/s 2024-04-25T10:04:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705326 s, 149 MB/s 2024-04-25T10:04:08.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.249 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.249 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636453 s, 165 MB/s 2024-04-25T10:04:08.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.257 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.257 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650559 s, 161 MB/s 2024-04-25T10:04:08.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:08.258 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.322 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641378 s, 163 MB/s 2024-04-25T10:04:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.328 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549842 s, 191 MB/s 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703586 s, 149 MB/s 2024-04-25T10:04:08.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594732 s, 176 MB/s 2024-04-25T10:04:08.350 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.351 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.351 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688468 s, 152 MB/s 2024-04-25T10:04:08.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655134 s, 160 MB/s 2024-04-25T10:04:08.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636183 s, 165 MB/s 2024-04-25T10:04:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.372 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613047 s, 171 MB/s 2024-04-25T10:04:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701334 s, 150 MB/s 2024-04-25T10:04:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625083 s, 168 MB/s 2024-04-25T10:04:08.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.395 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640831 s, 164 MB/s 2024-04-25T10:04:08.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.403 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727587 s, 144 MB/s 2024-04-25T10:04:08.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.410 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.410 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649153 s, 162 MB/s 2024-04-25T10:04:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.417 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629642 s, 167 MB/s 2024-04-25T10:04:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634627 s, 165 MB/s 2024-04-25T10:04:08.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.431 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563405 s, 186 MB/s 2024-04-25T10:04:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730138 s, 144 MB/s 2024-04-25T10:04:08.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.447 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.447 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718003 s, 146 MB/s 2024-04-25T10:04:08.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.454 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.455 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638878 s, 164 MB/s 2024-04-25T10:04:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.461 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579299 s, 181 MB/s 2024-04-25T10:04:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.469 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645636 s, 162 MB/s 2024-04-25T10:04:08.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.476 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628477 s, 167 MB/s 2024-04-25T10:04:08.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.483 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.484 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699434 s, 150 MB/s 2024-04-25T10:04:08.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.491 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.491 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676309 s, 155 MB/s 2024-04-25T10:04:08.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.499 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.499 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702183 s, 149 MB/s 2024-04-25T10:04:08.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.506 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.506 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627926 s, 167 MB/s 2024-04-25T10:04:08.513 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.514 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648607 s, 162 MB/s 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.521 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640809 s, 164 MB/s 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.528 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676887 s, 155 MB/s 2024-04-25T10:04:08.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.536 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.536 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642419 s, 163 MB/s 2024-04-25T10:04:08.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.543 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.543 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660816 s, 159 MB/s 2024-04-25T10:04:08.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.551 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.552 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718227 s, 146 MB/s 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:08.553 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:08.616 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.617 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643557 s, 163 MB/s 2024-04-25T10:04:08.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.624 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.624 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657238 s, 160 MB/s 2024-04-25T10:04:08.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.631 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.631 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640082 s, 164 MB/s 2024-04-25T10:04:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.638 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591969 s, 177 MB/s 2024-04-25T10:04:08.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.645 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.645 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637178 s, 165 MB/s 2024-04-25T10:04:08.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.652 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.652 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633846 s, 165 MB/s 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670464 s, 156 MB/s 2024-04-25T10:04:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.668 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741851 s, 141 MB/s 2024-04-25T10:04:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.676 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704269 s, 149 MB/s 2024-04-25T10:04:08.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.683 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.683 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635629 s, 165 MB/s 2024-04-25T10:04:08.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.690 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.690 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584704 s, 179 MB/s 2024-04-25T10:04:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.698 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710002 s, 148 MB/s 2024-04-25T10:04:08.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.705 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.706 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064561 s, 162 MB/s 2024-04-25T10:04:08.713 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.714 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.714 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718816 s, 146 MB/s 2024-04-25T10:04:08.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.721 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.722 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703756 s, 149 MB/s 2024-04-25T10:04:08.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789188 s, 133 MB/s 2024-04-25T10:04:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.738 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649771 s, 161 MB/s 2024-04-25T10:04:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.745 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597731 s, 175 MB/s 2024-04-25T10:04:08.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.751 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606553 s, 173 MB/s 2024-04-25T10:04:08.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.758 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.758 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058542 s, 179 MB/s 2024-04-25T10:04:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676602 s, 155 MB/s 2024-04-25T10:04:08.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.773 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.773 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577839 s, 181 MB/s 2024-04-25T10:04:08.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.780 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.780 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651986 s, 161 MB/s 2024-04-25T10:04:08.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.788 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710555 s, 148 MB/s 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609545 s, 172 MB/s 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629561 s, 167 MB/s 2024-04-25T10:04:08.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.810 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.810 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740957 s, 142 MB/s 2024-04-25T10:04:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649441 s, 161 MB/s 2024-04-25T10:04:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565099 s, 186 MB/s 2024-04-25T10:04:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.832 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622441 s, 168 MB/s 2024-04-25T10:04:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631285 s, 166 MB/s 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752887 s, 139 MB/s 2024-04-25T10:04:08.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:08.848 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:08.848 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:08.848 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:04:08.848 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:08.848 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:08.849 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:08.849 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:08.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.912 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.912 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058985 s, 178 MB/s 2024-04-25T10:04:08.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.919 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.919 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580321 s, 181 MB/s 2024-04-25T10:04:08.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.926 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.926 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064538 s, 162 MB/s 2024-04-25T10:04:08.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.933 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.933 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556448 s, 188 MB/s 2024-04-25T10:04:08.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.940 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.940 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649909 s, 161 MB/s 2024-04-25T10:04:08.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.948 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.948 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670178 s, 156 MB/s 2024-04-25T10:04:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656956 s, 160 MB/s 2024-04-25T10:04:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.963 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746359 s, 140 MB/s 2024-04-25T10:04:08.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.970 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.971 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061729 s, 170 MB/s 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.978 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666623 s, 157 MB/s 2024-04-25T10:04:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.985 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651879 s, 161 MB/s 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666603 s, 157 MB/s 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649961 s, 161 MB/s 2024-04-25T10:04:09.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.007 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.007 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616955 s, 170 MB/s 2024-04-25T10:04:09.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.015 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.015 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068538 s, 153 MB/s 2024-04-25T10:04:09.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.023 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722766 s, 145 MB/s 2024-04-25T10:04:09.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.031 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.031 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740814 s, 142 MB/s 2024-04-25T10:04:09.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.039 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.039 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640354 s, 164 MB/s 2024-04-25T10:04:09.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.046 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608982 s, 172 MB/s 2024-04-25T10:04:09.054 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.054 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.054 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723014 s, 145 MB/s 2024-04-25T10:04:09.061 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.061 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.061 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006308 s, 166 MB/s 2024-04-25T10:04:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.069 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696185 s, 151 MB/s 2024-04-25T10:04:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.077 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750146 s, 140 MB/s 2024-04-25T10:04:09.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.085 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.085 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713052 s, 147 MB/s 2024-04-25T10:04:09.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.092 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585663 s, 179 MB/s 2024-04-25T10:04:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.099 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591304 s, 177 MB/s 2024-04-25T10:04:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646229 s, 162 MB/s 2024-04-25T10:04:09.114 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699202 s, 150 MB/s 2024-04-25T10:04:09.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.122 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652686 s, 161 MB/s 2024-04-25T10:04:09.128 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631504 s, 166 MB/s 2024-04-25T10:04:09.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632825 s, 166 MB/s 2024-04-25T10:04:09.142 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.143 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.143 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617478 s, 170 MB/s 2024-04-25T10:04:09.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:09.144 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:09.145 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:09.145 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:04:09.145 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:09.145 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:09.145 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:09.145 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607952 s, 172 MB/s 2024-04-25T10:04:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.216 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702783 s, 149 MB/s 2024-04-25T10:04:09.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.224 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.224 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676792 s, 155 MB/s 2024-04-25T10:04:09.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.232 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.232 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725075 s, 145 MB/s 2024-04-25T10:04:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712056 s, 147 MB/s 2024-04-25T10:04:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599316 s, 175 MB/s 2024-04-25T10:04:09.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.254 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.254 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638148 s, 164 MB/s 2024-04-25T10:04:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.261 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.262 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627592 s, 167 MB/s 2024-04-25T10:04:09.268 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.269 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654414 s, 160 MB/s 2024-04-25T10:04:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.276 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634616 s, 165 MB/s 2024-04-25T10:04:09.283 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.283 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.283 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647057 s, 162 MB/s 2024-04-25T10:04:09.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.290 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.290 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623394 s, 168 MB/s 2024-04-25T10:04:09.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.297 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.298 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647996 s, 162 MB/s 2024-04-25T10:04:09.304 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.305 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.305 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593757 s, 177 MB/s 2024-04-25T10:04:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.312 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652961 s, 161 MB/s 2024-04-25T10:04:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.319 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633342 s, 166 MB/s 2024-04-25T10:04:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.327 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745164 s, 141 MB/s 2024-04-25T10:04:09.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.334 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.334 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616702 s, 170 MB/s 2024-04-25T10:04:09.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.343 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.343 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075825 s, 138 MB/s 2024-04-25T10:04:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.352 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738035 s, 142 MB/s 2024-04-25T10:04:09.357 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.358 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058303 s, 180 MB/s 2024-04-25T10:04:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704715 s, 149 MB/s 2024-04-25T10:04:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637604 s, 164 MB/s 2024-04-25T10:04:09.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.380 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.380 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624358 s, 168 MB/s 2024-04-25T10:04:09.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.387 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.387 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057124 s, 184 MB/s 2024-04-25T10:04:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.394 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621607 s, 169 MB/s 2024-04-25T10:04:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.402 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722335 s, 145 MB/s 2024-04-25T10:04:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.409 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583762 s, 180 MB/s 2024-04-25T10:04:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.416 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618725 s, 169 MB/s 2024-04-25T10:04:09.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.424 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.424 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717103 s, 146 MB/s 2024-04-25T10:04:09.431 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.432 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.432 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666729 s, 157 MB/s 2024-04-25T10:04:09.438 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.439 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.439 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635447 s, 165 MB/s 2024-04-25T10:04:09.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:09.440 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:09.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.503 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.504 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639411 s, 164 MB/s 2024-04-25T10:04:09.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.510 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586309 s, 179 MB/s 2024-04-25T10:04:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058716 s, 179 MB/s 2024-04-25T10:04:09.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.525 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717795 s, 146 MB/s 2024-04-25T10:04:09.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.533 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.533 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695475 s, 151 MB/s 2024-04-25T10:04:09.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.540 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.540 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647285 s, 162 MB/s 2024-04-25T10:04:09.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.547 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.547 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618883 s, 169 MB/s 2024-04-25T10:04:09.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.555 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.556 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737112 s, 142 MB/s 2024-04-25T10:04:09.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.562 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.563 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575911 s, 182 MB/s 2024-04-25T10:04:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621836 s, 169 MB/s 2024-04-25T10:04:09.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.577 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.577 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006677 s, 157 MB/s 2024-04-25T10:04:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588093 s, 178 MB/s 2024-04-25T10:04:09.591 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.592 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.592 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694965 s, 151 MB/s 2024-04-25T10:04:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.599 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640839 s, 164 MB/s 2024-04-25T10:04:09.606 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.606 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.606 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627218 s, 167 MB/s 2024-04-25T10:04:09.613 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.613 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.613 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590173 s, 178 MB/s 2024-04-25T10:04:09.620 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.620 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.620 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637637 s, 164 MB/s 2024-04-25T10:04:09.627 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.628 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.628 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649472 s, 161 MB/s 2024-04-25T10:04:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.635 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623447 s, 168 MB/s 2024-04-25T10:04:09.642 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.643 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699405 s, 150 MB/s 2024-04-25T10:04:09.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.650 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633415 s, 166 MB/s 2024-04-25T10:04:09.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.657 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.657 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654322 s, 160 MB/s 2024-04-25T10:04:09.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.664 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.664 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628598 s, 167 MB/s 2024-04-25T10:04:09.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.672 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.672 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709817 s, 148 MB/s 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628419 s, 167 MB/s 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621688 s, 169 MB/s 2024-04-25T10:04:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653032 s, 161 MB/s 2024-04-25T10:04:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601463 s, 174 MB/s 2024-04-25T10:04:09.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694294 s, 151 MB/s 2024-04-25T10:04:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.716 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.717 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694769 s, 151 MB/s 2024-04-25T10:04:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669196 s, 157 MB/s 2024-04-25T10:04:09.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.732 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.732 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656819 s, 160 MB/s 2024-04-25T10:04:09.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:09.733 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:09.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.797 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626746 s, 167 MB/s 2024-04-25T10:04:09.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.805 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720303 s, 146 MB/s 2024-04-25T10:04:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.813 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704689 s, 149 MB/s 2024-04-25T10:04:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.821 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731435 s, 143 MB/s 2024-04-25T10:04:09.828 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.829 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640609 s, 164 MB/s 2024-04-25T10:04:09.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.836 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.836 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641902 s, 163 MB/s 2024-04-25T10:04:09.843 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.844 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664976 s, 158 MB/s 2024-04-25T10:04:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.850 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588988 s, 178 MB/s 2024-04-25T10:04:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713824 s, 147 MB/s 2024-04-25T10:04:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072297 s, 145 MB/s 2024-04-25T10:04:09.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.874 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.874 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670371 s, 156 MB/s 2024-04-25T10:04:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610555 s, 172 MB/s 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.888 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595683 s, 176 MB/s 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.895 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.896 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664818 s, 158 MB/s 2024-04-25T10:04:09.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.903 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.903 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621639 s, 169 MB/s 2024-04-25T10:04:09.910 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.911 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716444 s, 146 MB/s 2024-04-25T10:04:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.918 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640414 s, 164 MB/s 2024-04-25T10:04:09.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.925 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.926 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628711 s, 167 MB/s 2024-04-25T10:04:09.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.932 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.933 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575552 s, 182 MB/s 2024-04-25T10:04:09.939 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.939 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.939 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620585 s, 169 MB/s 2024-04-25T10:04:09.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.947 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.947 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710981 s, 147 MB/s 2024-04-25T10:04:09.954 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.955 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.955 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665717 s, 158 MB/s 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646519 s, 162 MB/s 2024-04-25T10:04:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.969 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064937 s, 161 MB/s 2024-04-25T10:04:09.977 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.979 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.979 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618573 s, 170 MB/s 2024-04-25T10:04:09.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.984 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.984 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070381 s, 149 MB/s 2024-04-25T10:04:09.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.991 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.991 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638405 s, 164 MB/s 2024-04-25T10:04:09.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:09.998 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633315 s, 166 MB/s 2024-04-25T10:04:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.005 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057759 s, 182 MB/s 2024-04-25T10:04:10.012 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.013 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.013 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651363 s, 161 MB/s 2024-04-25T10:04:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060812 s, 172 MB/s 2024-04-25T10:04:10.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.026 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.026 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622347 s, 168 MB/s 2024-04-25T10:04:10.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:10.028 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.092 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062703 s, 167 MB/s 2024-04-25T10:04:10.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.100 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.100 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711691 s, 147 MB/s 2024-04-25T10:04:10.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.107 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.107 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065506 s, 160 MB/s 2024-04-25T10:04:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.115 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639499 s, 164 MB/s 2024-04-25T10:04:10.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.122 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.123 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738111 s, 142 MB/s 2024-04-25T10:04:10.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.130 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.130 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706939 s, 148 MB/s 2024-04-25T10:04:10.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.138 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.138 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658117 s, 159 MB/s 2024-04-25T10:04:10.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.145 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.145 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630972 s, 166 MB/s 2024-04-25T10:04:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725689 s, 144 MB/s 2024-04-25T10:04:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673229 s, 156 MB/s 2024-04-25T10:04:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.168 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637103 s, 165 MB/s 2024-04-25T10:04:10.176 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.177 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.177 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738136 s, 142 MB/s 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658125 s, 159 MB/s 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729201 s, 144 MB/s 2024-04-25T10:04:10.199 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.200 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.200 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634018 s, 165 MB/s 2024-04-25T10:04:10.206 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.207 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.207 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646177 s, 162 MB/s 2024-04-25T10:04:10.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.214 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.214 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640282 s, 164 MB/s 2024-04-25T10:04:10.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.221 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.221 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597953 s, 175 MB/s 2024-04-25T10:04:10.227 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.228 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.228 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619078 s, 169 MB/s 2024-04-25T10:04:10.235 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.236 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712849 s, 147 MB/s 2024-04-25T10:04:10.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.242 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.243 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611081 s, 172 MB/s 2024-04-25T10:04:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.250 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065934 s, 159 MB/s 2024-04-25T10:04:10.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.258 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.258 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071367 s, 147 MB/s 2024-04-25T10:04:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.266 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720949 s, 145 MB/s 2024-04-25T10:04:10.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.273 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.273 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641188 s, 164 MB/s 2024-04-25T10:04:10.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.280 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.280 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615209 s, 170 MB/s 2024-04-25T10:04:10.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.288 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.288 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646407 s, 162 MB/s 2024-04-25T10:04:10.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.295 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.295 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636635 s, 165 MB/s 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596556 s, 176 MB/s 2024-04-25T10:04:10.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.309 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.309 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623766 s, 168 MB/s 2024-04-25T10:04:10.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.317 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.317 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714169 s, 147 MB/s 2024-04-25T10:04:10.324 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.325 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.325 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652322 s, 161 MB/s 2024-04-25T10:04:10.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:10.326 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.391 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718169 s, 146 MB/s 2024-04-25T10:04:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.398 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650922 s, 161 MB/s 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635253 s, 165 MB/s 2024-04-25T10:04:10.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.412 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.412 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609633 s, 172 MB/s 2024-04-25T10:04:10.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.419 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.419 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611585 s, 171 MB/s 2024-04-25T10:04:10.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.427 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.427 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718655 s, 146 MB/s 2024-04-25T10:04:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.435 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688048 s, 152 MB/s 2024-04-25T10:04:10.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.443 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.443 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729642 s, 144 MB/s 2024-04-25T10:04:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.450 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632623 s, 166 MB/s 2024-04-25T10:04:10.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.457 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.457 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548739 s, 191 MB/s 2024-04-25T10:04:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.464 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620358 s, 169 MB/s 2024-04-25T10:04:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.472 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700847 s, 150 MB/s 2024-04-25T10:04:10.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.480 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.480 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704839 s, 149 MB/s 2024-04-25T10:04:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.487 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647019 s, 162 MB/s 2024-04-25T10:04:10.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.495 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.495 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709221 s, 148 MB/s 2024-04-25T10:04:10.502 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.502 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.502 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644624 s, 163 MB/s 2024-04-25T10:04:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.510 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715862 s, 146 MB/s 2024-04-25T10:04:10.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.517 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.517 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592479 s, 177 MB/s 2024-04-25T10:04:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.524 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.525 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643008 s, 163 MB/s 2024-04-25T10:04:10.531 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.531 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.531 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577995 s, 181 MB/s 2024-04-25T10:04:10.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.538 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.538 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618593 s, 170 MB/s 2024-04-25T10:04:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.545 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604963 s, 173 MB/s 2024-04-25T10:04:10.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.553 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.553 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697888 s, 150 MB/s 2024-04-25T10:04:10.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.561 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.561 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720839 s, 145 MB/s 2024-04-25T10:04:10.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.568 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.568 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582857 s, 180 MB/s 2024-04-25T10:04:10.575 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.575 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.575 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616681 s, 170 MB/s 2024-04-25T10:04:10.582 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.583 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.583 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683437 s, 153 MB/s 2024-04-25T10:04:10.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.590 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.590 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628472 s, 167 MB/s 2024-04-25T10:04:10.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.596 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.596 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562146 s, 187 MB/s 2024-04-25T10:04:10.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.603 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.604 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628533 s, 167 MB/s 2024-04-25T10:04:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.611 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642461 s, 163 MB/s 2024-04-25T10:04:10.617 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.618 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.618 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606578 s, 173 MB/s 2024-04-25T10:04:10.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:10.619 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.684 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653673 s, 160 MB/s 2024-04-25T10:04:10.692 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.692 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.692 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740529 s, 142 MB/s 2024-04-25T10:04:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.699 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.700 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671466 s, 156 MB/s 2024-04-25T10:04:10.707 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.708 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744668 s, 141 MB/s 2024-04-25T10:04:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.715 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645977 s, 162 MB/s 2024-04-25T10:04:10.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.722 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.723 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668715 s, 157 MB/s 2024-04-25T10:04:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.730 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669786 s, 157 MB/s 2024-04-25T10:04:10.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.737 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.737 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636035 s, 165 MB/s 2024-04-25T10:04:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563964 s, 186 MB/s 2024-04-25T10:04:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.752 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666246 s, 157 MB/s 2024-04-25T10:04:10.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.759 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.759 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628514 s, 167 MB/s 2024-04-25T10:04:10.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.766 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.766 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669755 s, 157 MB/s 2024-04-25T10:04:10.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.774 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.774 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654932 s, 160 MB/s 2024-04-25T10:04:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.781 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617298 s, 170 MB/s 2024-04-25T10:04:10.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.788 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.788 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685319 s, 153 MB/s 2024-04-25T10:04:10.796 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.797 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716114 s, 146 MB/s 2024-04-25T10:04:10.804 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.805 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.806 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711925 s, 147 MB/s 2024-04-25T10:04:10.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.812 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.812 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678316 s, 155 MB/s 2024-04-25T10:04:10.819 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.820 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.820 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648139 s, 162 MB/s 2024-04-25T10:04:10.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.827 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.827 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648781 s, 162 MB/s 2024-04-25T10:04:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.834 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621117 s, 169 MB/s 2024-04-25T10:04:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.842 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743484 s, 141 MB/s 2024-04-25T10:04:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066659 s, 157 MB/s 2024-04-25T10:04:10.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.858 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.858 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683984 s, 153 MB/s 2024-04-25T10:04:10.864 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.865 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.865 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061197 s, 171 MB/s 2024-04-25T10:04:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.873 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756902 s, 139 MB/s 2024-04-25T10:04:10.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.880 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.881 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063482 s, 165 MB/s 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745442 s, 141 MB/s 2024-04-25T10:04:10.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.897 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.897 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731305 s, 143 MB/s 2024-04-25T10:04:10.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.906 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.906 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757317 s, 138 MB/s 2024-04-25T10:04:10.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.913 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.913 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621552 s, 169 MB/s 2024-04-25T10:04:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.920 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661578 s, 158 MB/s 2024-04-25T10:04:10.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:10.921 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:04:10.922 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:10.922 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:04:10.922 DEBUG:teuthology.orchestra.run.smithi148:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:04:10.922 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:04:10.922 DEBUG:teuthology.orchestra.run.smithi148:> done 2024-04-25T10:04:10.922 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:10.987 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.987 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.987 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655373 s, 160 MB/s 2024-04-25T10:04:10.994 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:10.995 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659701 s, 159 MB/s 2024-04-25T10:04:11.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.002 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.002 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063791 s, 164 MB/s 2024-04-25T10:04:11.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.009 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.009 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635434 s, 165 MB/s 2024-04-25T10:04:11.017 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.017 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.017 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730991 s, 143 MB/s 2024-04-25T10:04:11.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.023 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.024 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576458 s, 182 MB/s 2024-04-25T10:04:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.030 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577221 s, 182 MB/s 2024-04-25T10:04:11.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.037 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.037 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609355 s, 172 MB/s 2024-04-25T10:04:11.045 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.046 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.046 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731304 s, 143 MB/s 2024-04-25T10:04:11.053 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.053 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.053 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703317 s, 149 MB/s 2024-04-25T10:04:11.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.062 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.062 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735979 s, 142 MB/s 2024-04-25T10:04:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632512 s, 166 MB/s 2024-04-25T10:04:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644867 s, 163 MB/s 2024-04-25T10:04:11.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.083 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.084 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649667 s, 161 MB/s 2024-04-25T10:04:11.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.090 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.090 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614252 s, 171 MB/s 2024-04-25T10:04:11.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.099 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.099 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720257 s, 146 MB/s 2024-04-25T10:04:11.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.106 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.106 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630744 s, 166 MB/s 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624225 s, 168 MB/s 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735445 s, 143 MB/s 2024-04-25T10:04:11.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.129 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.129 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718565 s, 146 MB/s 2024-04-25T10:04:11.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.136 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.136 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634166 s, 165 MB/s 2024-04-25T10:04:11.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.144 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.144 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629423 s, 167 MB/s 2024-04-25T10:04:11.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.152 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.152 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734446 s, 143 MB/s 2024-04-25T10:04:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.159 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589413 s, 178 MB/s 2024-04-25T10:04:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.166 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627357 s, 167 MB/s 2024-04-25T10:04:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.173 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.174 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658106 s, 159 MB/s 2024-04-25T10:04:11.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.180 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.180 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590644 s, 178 MB/s 2024-04-25T10:04:11.188 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.188 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.188 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649191 s, 162 MB/s 2024-04-25T10:04:11.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.195 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.195 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570814 s, 184 MB/s 2024-04-25T10:04:11.202 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.203 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.203 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736957 s, 142 MB/s 2024-04-25T10:04:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.211 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720243 s, 146 MB/s 2024-04-25T10:04:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-25T10:04:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-25T10:04:11.218 INFO:teuthology.orchestra.run.smithi148.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593423 s, 177 MB/s 2024-04-25T10:04:11.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T10:04:11.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.355+0000 7f326c730640 1 Processor -- start 2024-04-25T10:04:11.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.355+0000 7f326c730640 1 -- start start 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f326c730640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32640cd8a0 0x7f326415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f326c730640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326415e6e0 con 0x7f32640cd4c0 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f326c730640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326415e9c0 con 0x7f3264150b90 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f326c730640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326415a9c0 con 0x7f32640cd8a0 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f326a4cc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f32640cd4c0 0x7f326415dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51270/0 (socket says 172.21.15.148:51270) 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f326a4cc640 1 -- 172.21.15.148:0/222547700 learned_addr learned my addr 172.21.15.148:0/222547700 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f3269ccb640 1 --2- 172.21.15.148:0/222547700 >> v2:172.21.15.188:3300/0 conn(0x7f32640cd8a0 0x7f326415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185923649 0 0) 0x7f326415e6e0 con 0x7f32640cd4c0 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.356+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3250005730 con 0x7f32640cd4c0 2024-04-25T10:04:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.357+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383155543 0 0) 0x7f326415e9c0 con 0x7f3264150b90 2024-04-25T10:04:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.357+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f326415e6e0 con 0x7f3264150b90 2024-04-25T10:04:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.357+0000 7f3269ccb640 1 -- 172.21.15.148:0/222547700 >> v1:172.21.15.188:6789/0 conn(0x7f3264150b90 legacy=0x7f3264071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.357+0000 7f3269ccb640 1 -- 172.21.15.148:0/222547700 >> v1:172.21.15.148:6789/0 conn(0x7f32640cd4c0 legacy=0x7f326415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.357+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 116349636 0 0) 0x7f3250005730 con 0x7f32640cd4c0 2024-04-25T10:04:11.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.357+0000 7f3269ccb640 1 -- 172.21.15.148:0/222547700 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326415aca0 con 0x7f32640cd8a0 2024-04-25T10:04:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f3269ccb640 1 --2- 172.21.15.148:0/222547700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32640cd8a0 0x7f326415b1c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f32540097a0 tx=0x7f325402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e2abe1d37a584a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254037070 con 0x7f32640cd8a0 2024-04-25T10:04:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325402ac50 con 0x7f32640cd8a0 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f32694ca640 1 -- 172.21.15.148:0/222547700 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260001670 con 0x7f32640cd8a0 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f326c730640 1 -- 172.21.15.148:0/222547700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32640cd8a0 msgr2=0x7f326415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f326c730640 1 --2- 172.21.15.148:0/222547700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32640cd8a0 0x7f326415b1c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f32540097a0 tx=0x7f325402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f326c730640 1 -- 172.21.15.148:0/222547700 shutdown_connections 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f326c730640 1 --2- 172.21.15.148:0/222547700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32640cd8a0 0x7f326415b1c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.358+0000 7f326c730640 1 -- 172.21.15.148:0/222547700 >> 172.21.15.148:0/222547700 conn(0x7f326406abe0 msgr2=0x7f326406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.359+0000 7f326c730640 1 -- 172.21.15.148:0/222547700 shutdown_connections 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.359+0000 7f326c730640 1 -- 172.21.15.148:0/222547700 wait complete. 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 Processor -- start 2024-04-25T10:04:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 -- start start 2024-04-25T10:04:11.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 0x7f32641570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32640cd8a0 0x7f3264151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3264150b90 0x7f3264142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326406eca0 con 0x7f32640cd4c0 2024-04-25T10:04:11.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326415e370 con 0x7f3264150b90 2024-04-25T10:04:11.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326c730640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326406ee20 con 0x7f32640cd8a0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326a4cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 0x7f32641570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326a4cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 0x7f32641570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58028/0 (socket says 172.21.15.148:58028) 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.360+0000 7f326a4cc640 1 -- 172.21.15.148:0/1732285462 learned_addr learned my addr 172.21.15.148:0/1732285462 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326accd640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3264150b90 0x7f3264142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f3269ccb640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32640cd8a0 0x7f3264151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326a4cc640 1 -- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32640cd8a0 msgr2=0x7f3264151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326a4cc640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32640cd8a0 0x7f3264151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326a4cc640 1 -- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3264150b90 msgr2=0x7f3264142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326a4cc640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3264150b90 0x7f3264142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326a4cc640 1 -- 172.21.15.148:0/1732285462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264152080 con 0x7f32640cd4c0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326a4cc640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 0x7f32641570e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f326000c750 tx=0x7f326000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260017070 con 0x7f32640cd4c0 2024-04-25T10:04:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3260005ad0 con 0x7f32640cd4c0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3264151ac0 con 0x7f32640cd4c0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260007070 con 0x7f32640cd4c0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.361+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32641523a0 con 0x7f32640cd4c0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.362+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264073a00 con 0x7f32640cd4c0 2024-04-25T10:04:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.362+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f326001a070 con 0x7f32640cd4c0 2024-04-25T10:04:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.363+0000 7f325affd640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f324806b4d0 0x7f324806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.363+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3260089880 con 0x7f32640cd4c0 2024-04-25T10:04:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.363+0000 7f3269ccb640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f324806b4d0 0x7f324806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.363+0000 7f3269ccb640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f324806b4d0 0x7f324806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3254034000 tx=0x7f325402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.367+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3260057ed0 con 0x7f32640cd4c0 2024-04-25T10:04:11.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.572+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f32640ccb10 con 0x7f32640cd4c0 2024-04-25T10:04:11.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.959+0000 7f325affd640 1 -- 172.21.15.148:0/1732285462 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v85) v1 ==== 105+0+0 (secure 0 0 0) 0x7f326005bf70 con 0x7f32640cd4c0 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f324806b4d0 msgr2=0x7f324806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f324806b4d0 0x7f324806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3254034000 tx=0x7f325402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 msgr2=0x7f32641570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 0x7f32641570e0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f326000c750 tx=0x7f326000cc10 comp rx=0 tx=0).stop 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 shutdown_connections 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f324806b4d0 0x7f324806d990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3264150b90 0x7f3264142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32640cd8a0 0x7f3264151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 --2- 172.21.15.148:0/1732285462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32640cd4c0 0x7f32641570e0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.961+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 >> 172.21.15.148:0/1732285462 conn(0x7f326406abe0 msgr2=0x7f3264159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.962+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 shutdown_connections 2024-04-25T10:04:11.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:11.962+0000 7f326c730640 1 -- 172.21.15.148:0/1732285462 wait complete. 2024-04-25T10:04:11.978 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:04:12.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.073+0000 7f22d02f7640 1 Processor -- start 2024-04-25T10:04:12.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.074+0000 7f22d02f7640 1 -- start start 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.074+0000 7f22d02f7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22c815dc20 0x7f22c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.074+0000 7f22d02f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.074+0000 7f22d02f7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c815e000 con 0x7f22c80b9330 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.074+0000 7f22d02f7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80d1de0 con 0x7f22c815dc20 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.074+0000 7f22d02f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8161c60 con 0x7f22c8161710 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22c815dc20 0x7f22c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22c815dc20 0x7f22c80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58034/0 (socket says 172.21.15.148:58034) 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 -- 172.21.15.148:0/1342086846 learned_addr learned my addr 172.21.15.148:0/1342086846 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 -- 172.21.15.148:0/1342086846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 msgr2=0x7f22c80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22cd892640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 -- 172.21.15.148:0/1342086846 >> v1:172.21.15.188:6789/0 conn(0x7f22c80b9330 legacy=0x7f22c806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22ce093640 1 -- 172.21.15.148:0/1342086846 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8161f40 con 0x7f22c815dc20 2024-04-25T10:04:12.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.075+0000 7f22cd892640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:12.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22ce093640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c815dc20 0x7f22c80ce670 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009e50 tx=0x7f22c402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beab9844ec8823b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22cd091640 1 -- 172.21.15.148:0/1342086846 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4036070 con 0x7f22c815dc20 2024-04-25T10:04:12.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22cd091640 1 -- 172.21.15.148:0/1342086846 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4031600 con 0x7f22c815dc20 2024-04-25T10:04:12.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22cd091640 1 -- 172.21.15.148:0/1342086846 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c40319d0 con 0x7f22c815dc20 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 -- 172.21.15.148:0/1342086846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c815dc20 msgr2=0x7f22c80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c815dc20 0x7f22c80ce670 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009e50 tx=0x7f22c402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 -- 172.21.15.148:0/1342086846 shutdown_connections 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 --2- 172.21.15.148:0/1342086846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c815dc20 0x7f22c80ce670 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 -- 172.21.15.148:0/1342086846 >> 172.21.15.148:0/1342086846 conn(0x7f22c80d0420 msgr2=0x7f22c80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 -- 172.21.15.148:0/1342086846 shutdown_connections 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.076+0000 7f22d02f7640 1 -- 172.21.15.148:0/1342086846 wait complete. 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 Processor -- start 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 -- start start 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 0x7f22c8158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c815dc20 0x7f22c8160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80b5e50 con 0x7f22c80b9330 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8162500 con 0x7f22c815dc20 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22d02f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80b5fd0 con 0x7f22c8161710 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22ce093640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 0x7f22c8158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22ce093640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 0x7f22c8158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58044/0 (socket says 172.21.15.148:58044) 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.078+0000 7f22ce093640 1 -- 172.21.15.148:0/4096366573 learned_addr learned my addr 172.21.15.148:0/4096366573 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:12.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce894640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22cd892640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c815dc20 0x7f22c8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce093640 1 -- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 msgr2=0x7f22c80cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce093640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce093640 1 -- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c815dc20 msgr2=0x7f22c8160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce093640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c815dc20 0x7f22c8160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce093640 1 -- 172.21.15.148:0/4096366573 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80cab80 con 0x7f22c80b9330 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce894640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22cd892640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c815dc20 0x7f22c8160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22ce093640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 0x7f22c8158e90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034040 tx=0x7f22c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4036050 con 0x7f22c80b9330 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4042070 con 0x7f22c80b9330 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c80cae60 con 0x7f22c80b9330 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4009510 con 0x7f22c80b9330 2024-04-25T10:04:12.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.079+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80cb420 con 0x7f22c80b9330 2024-04-25T10:04:12.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.080+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8073a00 con 0x7f22c80b9330 2024-04-25T10:04:12.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.080+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22c4007c50 con 0x7f22c80b9330 2024-04-25T10:04:12.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.081+0000 7f22beffd640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22ac06b500 0x7f22ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.081+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f22c40ab3b0 con 0x7f22c80b9330 2024-04-25T10:04:12.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.081+0000 7f22cd892640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22ac06b500 0x7f22ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.081+0000 7f22cd892640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22ac06b500 0x7f22ac06d9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.085+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22c4079fd0 con 0x7f22c80b9330 2024-04-25T10:04:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.294+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f22c815c6f0 con 0x7f22c80b9330 2024-04-25T10:04:12.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.960+0000 7f22beffd640 1 -- 172.21.15.148:0/4096366573 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v86) v1 ==== 121+0+0 (secure 0 0 0) 0x7f22c40798d0 con 0x7f22c80b9330 2024-04-25T10:04:12.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22ac06b500 msgr2=0x7f22ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22ac06b500 0x7f22ac06d9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006fd0 tx=0x7f22b8008040 comp rx=0 tx=0).stop 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 msgr2=0x7f22c8158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 0x7f22c8158e90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034040 tx=0x7f22c4004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 shutdown_connections 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22ac06b500 0x7f22ac06d9c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c8161710 0x7f22c80cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c815dc20 0x7f22c8160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 --2- 172.21.15.148:0/4096366573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22c80b9330 0x7f22c8158e90 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 >> 172.21.15.148:0/4096366573 conn(0x7f22c80d0420 msgr2=0x7f22c815bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 shutdown_connections 2024-04-25T10:04:12.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:12.963+0000 7f22d02f7640 1 -- 172.21.15.148:0/4096366573 wait complete. 2024-04-25T10:04:12.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:13.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 Processor -- start 2024-04-25T10:04:13.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 -- start start 2024-04-25T10:04:13.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd8154ee0 0x7febd80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febd80ce590 0x7febd806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd8071480 con 0x7febd80cfcf0 2024-04-25T10:04:13.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd8072090 con 0x7febd80ce590 2024-04-25T10:04:13.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.076+0000 7febdf55c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd80d0150 con 0x7febd8154ee0 2024-04-25T10:04:13.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febddaf9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febd80ce590 0x7febd806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febdcaf7640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7febd80cfcf0 0x7febd80cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:41494/0 (socket says 172.21.15.148:41494) 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febdd2f8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd8154ee0 0x7febd80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febdcaf7640 1 -- 172.21.15.148:0/944415426 learned_addr learned my addr 172.21.15.148:0/944415426 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febd7fff640 1 -- 172.21.15.148:0/944415426 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504981764 0 0) 0x7febd8071480 con 0x7febd80cfcf0 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febd7fff640 1 -- 172.21.15.148:0/944415426 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febc0003520 con 0x7febd80cfcf0 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.077+0000 7febddaf9640 1 -- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd8154ee0 msgr2=0x7febd80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febddaf9640 1 --2- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd8154ee0 0x7febd80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febddaf9640 1 -- 172.21.15.148:0/944415426 >> v1:172.21.15.188:6789/0 conn(0x7febd80cfcf0 legacy=0x7febd80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febddaf9640 1 -- 172.21.15.148:0/944415426 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8071760 con 0x7febd80ce590 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febdd2f8640 1 --2- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd8154ee0 0x7febd80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febddaf9640 1 --2- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd80ce590 0x7febd806d6f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7febc80097a0 tx=0x7febc802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d6671b1cbef29b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febd7fff640 1 -- 172.21.15.148:0/944415426 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8037070 con 0x7febd80ce590 2024-04-25T10:04:13.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febd7fff640 1 -- 172.21.15.148:0/944415426 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc8032440 con 0x7febd80ce590 2024-04-25T10:04:13.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.078+0000 7febd7fff640 1 -- 172.21.15.148:0/944415426 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8032810 con 0x7febd80ce590 2024-04-25T10:04:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 -- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd80ce590 msgr2=0x7febd806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 --2- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd80ce590 0x7febd806d6f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7febc80097a0 tx=0x7febc802a180 comp rx=0 tx=0).stop 2024-04-25T10:04:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 -- 172.21.15.148:0/944415426 shutdown_connections 2024-04-25T10:04:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 --2- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd80ce590 0x7febd806d6f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 --2- 172.21.15.148:0/944415426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd8154ee0 0x7febd80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 -- 172.21.15.148:0/944415426 >> 172.21.15.148:0/944415426 conn(0x7febd806abe0 msgr2=0x7febd806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 -- 172.21.15.148:0/944415426 shutdown_connections 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.079+0000 7febdf55c640 1 -- 172.21.15.148:0/944415426 wait complete. 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.080+0000 7febdf55c640 1 Processor -- start 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.080+0000 7febdf55c640 1 -- start start 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdf55c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd80ce590 0x7febd80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdf55c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 0x7febd8150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdf55c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd8154ee0 0x7febd8145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdf55c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd8071d20 con 0x7febd80cfcf0 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdf55c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd8071230 con 0x7febd8154ee0 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdf55c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd81552c0 con 0x7febd80ce590 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 0x7febd8150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 0x7febd8150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58056/0 (socket says 172.21.15.148:58056) 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 -- 172.21.15.148:0/1487425221 learned_addr learned my addr 172.21.15.148:0/1487425221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdd2f8640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd80ce590 0x7febd80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febddaf9640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd8154ee0 0x7febd8145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 -- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd80ce590 msgr2=0x7febd80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd80ce590 0x7febd80b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 -- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd8154ee0 msgr2=0x7febd8145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd8154ee0 0x7febd8145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 -- 172.21.15.148:0/1487425221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8150cb0 con 0x7febd80cfcf0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febddaf9640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd8154ee0 0x7febd8145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.081+0000 7febdcaf7640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 0x7febd8150770 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7febcc00cb70 tx=0x7febcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.082+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc017070 con 0x7febd80cfcf0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.082+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00ad40 con 0x7febd80cfcf0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.082+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd81518c0 con 0x7febd80cfcf0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.082+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc012620 con 0x7febd80cfcf0 2024-04-25T10:04:13.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.082+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd80afd60 con 0x7febd80cfcf0 2024-04-25T10:04:13.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.083+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd8151a90 con 0x7febd80cfcf0 2024-04-25T10:04:13.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.083+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febcc01a070 con 0x7febd80cfcf0 2024-04-25T10:04:13.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.084+0000 7febd5ffb640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febc406b4d0 0x7febc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.084+0000 7febdd2f8640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febc406b4d0 0x7febc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.084+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7febcc08a250 con 0x7febd80cfcf0 2024-04-25T10:04:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.084+0000 7febdd2f8640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febc406b4d0 0x7febc406d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7febd80ccdf0 tx=0x7febd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.088+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febcc058ef0 con 0x7febd80cfcf0 2024-04-25T10:04:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.294+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd806d360 con 0x7febd80cfcf0 2024-04-25T10:04:13.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.295+0000 7febd5ffb640 1 -- 172.21.15.148:0/1487425221 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4961 (secure 0 0 0) 0x7febcc0587f0 con 0x7febd80cfcf0 2024-04-25T10:04:13.301 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:13.301 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:04:12.963932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9855":{"gid":9855,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:04:13.301 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 87 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febc406b4d0 msgr2=0x7febc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febc406b4d0 0x7febc406d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7febd80ccdf0 tx=0x7febd0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 msgr2=0x7febd8150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 0x7febd8150770 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7febcc00cb70 tx=0x7febcc00a430 comp rx=0 tx=0).stop 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 shutdown_connections 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7febc406b4d0 0x7febc406d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd8154ee0 0x7febd8145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febd80cfcf0 0x7febd8150770 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 --2- 172.21.15.148:0/1487425221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd80ce590 0x7febd80b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 >> 172.21.15.148:0/1487425221 conn(0x7febd806abe0 msgr2=0x7febd815d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 shutdown_connections 2024-04-25T10:04:13.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.297+0000 7febdf55c640 1 -- 172.21.15.148:0/1487425221 wait complete. 2024-04-25T10:04:13.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:03:50.223124+0000', 'modified': '2024-04-25T10:04:12.963932+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9855': {'gid': 9855, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/1405903723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 1405903723}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 1405903723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9861': {'gid': 9861, 'name': 'c', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/3188696943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3188696943}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3188696943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:04:13.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:04:13.716 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:04:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.812+0000 7fa30c691640 1 Processor -- start 2024-04-25T10:04:13.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.812+0000 7fa30c691640 1 -- start start 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa30c691640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3041454b0 0x7fa304070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa30c691640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3040ceba0 con 0x7fa304144d20 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa30c691640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3040cee80 con 0x7fa304071300 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa30c691640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040b4830 con 0x7fa3041454b0 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa309c2c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3041454b0 0x7fa304070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa309c2c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3041454b0 0x7fa304070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56686/0 (socket says 172.21.15.148:56686) 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa30a42d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa304144d20 0x7fa3040ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42036/0 (socket says 172.21.15.148:42036) 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa309c2c640 1 -- 172.21.15.148:0/1170350003 learned_addr learned my addr 172.21.15.148:0/1170350003 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa309c2c640 1 -- 172.21.15.148:0/1170350003 >> v1:172.21.15.188:6789/0 conn(0x7fa304071300 legacy=0x7fa3040b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa309c2c640 1 -- 172.21.15.148:0/1170350003 >> v1:172.21.15.148:6789/0 conn(0x7fa304144d20 legacy=0x7fa3040ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.813+0000 7fa309c2c640 1 -- 172.21.15.148:0/1170350003 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040cf160 con 0x7fa3041454b0 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.814+0000 7fa309c2c640 1 --2- 172.21.15.148:0/1170350003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3041454b0 0x7fa304070dc0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fa3040b49b0 tx=0x7fa2f402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972d746ff5bdade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.814+0000 7fa30942b640 1 -- 172.21.15.148:0/1170350003 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4037070 con 0x7fa3041454b0 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.814+0000 7fa30942b640 1 -- 172.21.15.148:0/1170350003 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f4032440 con 0x7fa3041454b0 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.814+0000 7fa30942b640 1 -- 172.21.15.148:0/1170350003 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4032810 con 0x7fa3041454b0 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.814+0000 7fa30c691640 1 -- 172.21.15.148:0/1170350003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3041454b0 msgr2=0x7fa304070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.814+0000 7fa30c691640 1 --2- 172.21.15.148:0/1170350003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3041454b0 0x7fa304070dc0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fa3040b49b0 tx=0x7fa2f402a290 comp rx=0 tx=0).stop 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.815+0000 7fa30c691640 1 -- 172.21.15.148:0/1170350003 shutdown_connections 2024-04-25T10:04:13.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.815+0000 7fa30c691640 1 --2- 172.21.15.148:0/1170350003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3041454b0 0x7fa304070dc0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.815+0000 7fa30c691640 1 -- 172.21.15.148:0/1170350003 >> 172.21.15.148:0/1170350003 conn(0x7fa3040b3530 msgr2=0x7fa3040b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.815+0000 7fa30c691640 1 -- 172.21.15.148:0/1170350003 shutdown_connections 2024-04-25T10:04:13.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.815+0000 7fa30c691640 1 -- 172.21.15.148:0/1170350003 wait complete. 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 Processor -- start 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 -- start start 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 0x7fa30413c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa304144d20 0x7fa304143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3041454b0 0x7fa30413a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304145100 con 0x7fa304071300 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304071730 con 0x7fa304144d20 2024-04-25T10:04:13.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.816+0000 7fa30c691640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa304145280 con 0x7fa3041454b0 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 0x7fa30413c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 0x7fa30413c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56702/0 (socket says 172.21.15.148:56702) 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 -- 172.21.15.148:0/734799717 learned_addr learned my addr 172.21.15.148:0/734799717 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 -- 172.21.15.148:0/734799717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3041454b0 msgr2=0x7fa30413a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3041454b0 0x7fa30413a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 -- 172.21.15.148:0/734799717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa304144d20 msgr2=0x7fa304143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa304144d20 0x7fa304143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 -- 172.21.15.148:0/734799717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3041f7900 con 0x7fa304071300 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30a42d640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 0x7fa30413c210 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fa30000c3c0 tx=0x7fa30000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa2faffd640 1 -- 172.21.15.148:0/734799717 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000074e0 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa2faffd640 1 -- 172.21.15.148:0/734799717 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300007680 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3041f7be0 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa2faffd640 1 -- 172.21.15.148:0/734799717 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300004a00 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.817+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041f8480 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.818+0000 7fa2faffd640 1 -- 172.21.15.148:0/734799717 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa30001a020 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.819+0000 7fa2faffd640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2e806b4d0 0x7fa2e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.819+0000 7fa2faffd640 1 -- 172.21.15.148:0/734799717 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa300089ac0 con 0x7fa304071300 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.819+0000 7fa309c2c640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2e806b4d0 0x7fa2e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.819+0000 7fa309c2c640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2e806b4d0 0x7fa2e806d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4034000 tx=0x7fa2f402c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.820+0000 7fa30f38b740 1 Processor -- start 2024-04-25T10:04:13.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.820+0000 7fa30f38b740 1 -- start start 2024-04-25T10:04:13.834 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a282f10 0x55c67a2c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30f38b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67a1e5a10 con 0x55c67a2ad230 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67a18eb30 con 0x55c67a282f10 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67a1e1c50 con 0x55c67a2caad0 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56708/0 (socket says 172.21.15.148:56708) 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 -- 172.21.15.148:0/2220343702 learned_addr learned my addr 172.21.15.148:0/2220343702 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 -- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 msgr2=0x55c67a2cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a282f10 0x55c67a2c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.835 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 -- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a282f10 msgr2=0x55c67a2c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a282f10 0x55c67a2c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 -- 172.21.15.148:0/2220343702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c67a1a3e10 con 0x55c67a2ad230 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a282f10 0x55c67a2c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2ad610 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fa300000fb0 tx=0x7fa300057b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4e8ca5f6aff46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa2e7fff640 1 -- 172.21.15.148:0/2220343702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300019da0 con 0x55c67a2ad230 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa2e7fff640 1 -- 172.21.15.148:0/2220343702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300054760 con 0x55c67a2ad230 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.821+0000 7fa2e7fff640 1 -- 172.21.15.148:0/2220343702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300058590 con 0x55c67a2ad230 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 -- 172.21.15.148:0/2220343702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 msgr2=0x55c67a2ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.836 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2ad610 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7fa300000fb0 tx=0x7fa300057b80 comp rx=0 tx=0).stop 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 -- 172.21.15.148:0/2220343702 shutdown_connections 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a282f10 0x55c67a2c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2220343702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2ad610 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 -- 172.21.15.148:0/2220343702 >> 172.21.15.148:0/2220343702 conn(0x55c67a27f840 msgr2=0x55c67a2c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 -- 172.21.15.148:0/2220343702 shutdown_connections 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.822+0000 7fa30f38b740 1 -- 172.21.15.148:0/2220343702 wait complete. 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.823+0000 7fa30f38b740 1 -- start start 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.823+0000 7fa30f38b740 1 Processor -- start 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.837 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a2c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67a283cb0 con 0x55c67a2a8d60 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67a1a20c0 con 0x55c67a2caad0 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67a27d770 con 0x55c67a2ad230 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 10 client.0 _ll_register_callbacks cb 0x55c67a195ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a2c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a2c0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56718/0 (socket says 172.21.15.148:56718) 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 -- 172.21.15.148:0/2919738310 learned_addr learned my addr 172.21.15.148:0/2919738310 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 -- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 msgr2=0x55c67a2b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 -- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 msgr2=0x55c67a2d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 -- 172.21.15.148:0/2919738310 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c679efa2d0 con 0x55c67a2a8d60 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a2c0670 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c920 tx=0x7fa2fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c887d41d7b78a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa2e57fa640 1 -- 172.21.15.148:0/2919738310 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc008040 con 0x55c67a2a8d60 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa2e57fa640 1 -- 172.21.15.148:0/2919738310 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa2fc00a9e0 con 0x55c67a2a8d60 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa300054760 con 0x55c67a2a8d60 2024-04-25T10:04:13.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa2e57fa640 1 -- 172.21.15.148:0/2919738310 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2fc00abe0 con 0x55c67a2a8d60 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 10 client.10089 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.824+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55c679fc46b0 con 0x55c67a2a8d60 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa2e57fa640 1 -- 172.21.15.148:0/2919738310 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300054760 con 0x55c67a2a8d60 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa2e57fa640 1 -- 172.21.15.148:0/2919738310 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fa2fc00e040 con 0x55c67a2a8d60 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30f38b740 10 client.10089 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30f38b740 10 client.10089 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30f38b740 20 client.10089 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2ae2a0 0x55c67a2e6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30f38b740 4 client.10089 mds_command: new command op to 9861 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 --> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c67a244920 con 0x55c67a2ae2a0 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2ae2a0 0x55c67a2e6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa2e57fa640 10 client.10089 ms_handle_connect on v2:172.21.15.148:6836/3188696943 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.825+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2ae2a0 0x55c67a2e6330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2846d913e5c9966 server_cookie=2fc61b172f214df3 in_seq=0 out_seq=0 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa2e57fa640 1 -- 172.21.15.148:0/2919738310 <== mds.0 v2:172.21.15.148:6836/3188696943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c67a244920 con 0x55c67a2ae2a0 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa2e57fa640 10 client.10089 handle_command_reply: tid=0 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 client.10089 shutdown 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 msgr2=0x55c67a2c0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a2c0670 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c920 tx=0x7fa2fc00cde0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 shutdown_connections 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2ae2a0 0x55c67a2e6330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a2c0670 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2919738310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa2e57fa640 0 client.10089 ms_handle_reset on v2:172.21.15.148:6836/3188696943 2024-04-25T10:04:13.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 >> 172.21.15.148:0/2919738310 conn(0x55c67a282f10 msgr2=0x55c67a2cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 shutdown_connections 2024-04-25T10:04:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 1 -- 172.21.15.148:0/2919738310 wait complete. 2024-04-25T10:04:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.826+0000 7fa30f38b740 20 client.10089 trim_cache size 0 max 16384 2024-04-25T10:04:13.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 Processor -- start 2024-04-25T10:04:13.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 -- start start 2024-04-25T10:04:13.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2a8d60 0x55c67a2eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304200710 con 0x55c67a2ad230 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041542e0 con 0x55c67a2a8d60 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.842+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041548e0 con 0x55c67a2caad0 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2c0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56724/0 (socket says 172.21.15.148:56724) 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 -- 172.21.15.148:0/2450436179 learned_addr learned my addr 172.21.15.148:0/2450436179 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2a8d60 0x55c67a2eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 -- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 msgr2=0x55c67a2cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 -- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2a8d60 msgr2=0x55c67a2eabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2a8d60 0x55c67a2eabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 -- 172.21.15.148:0/2450436179 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c67a2cafc0 con 0x55c67a2ad230 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa30a42d640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2a8d60 0x55c67a2eabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa309c2c640 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2c0650 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4035040 tx=0x7fa2f4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc9b37769266300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa2e77fe640 1 -- 172.21.15.148:0/2450436179 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f404d070 con 0x55c67a2ad230 2024-04-25T10:04:13.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa2e77fe640 1 -- 172.21.15.148:0/2450436179 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f4041040 con 0x55c67a2ad230 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.843+0000 7fa2e77fe640 1 -- 172.21.15.148:0/2450436179 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4037040 con 0x55c67a2ad230 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 -- 172.21.15.148:0/2450436179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 msgr2=0x55c67a2c0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2c0650 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4035040 tx=0x7fa2f4008030 comp rx=0 tx=0).stop 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 -- 172.21.15.148:0/2450436179 shutdown_connections 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2caad0 0x55c67a2cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2ad230 0x55c67a2c0650 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 --2- 172.21.15.148:0/2450436179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2a8d60 0x55c67a2eabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 -- 172.21.15.148:0/2450436179 >> 172.21.15.148:0/2450436179 conn(0x55c67a282f10 msgr2=0x55c67a2ab150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 -- 172.21.15.148:0/2450436179 shutdown_connections 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 -- 172.21.15.148:0/2450436179 wait complete. 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 -- start start 2024-04-25T10:04:13.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.844+0000 7fa30f38b740 1 Processor -- start 2024-04-25T10:04:13.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a316050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2ea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67a2ae120 con 0x55c67a2a8d60 2024-04-25T10:04:13.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c67a2c11a0 con 0x55c67a2caad0 2024-04-25T10:04:13.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c67a2ae690 con 0x55c67a2ad230 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30f38b740 10 client.0 _ll_register_callbacks cb 0x55c67a27dc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a316050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a316050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56732/0 (socket says 172.21.15.148:56732) 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 -- 172.21.15.148:0/3608249670 learned_addr learned my addr 172.21.15.148:0/3608249670 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30ac2e640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa309c2c640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 -- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 msgr2=0x55c67a2af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 -- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 msgr2=0x55c67a2ea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2ea710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.845+0000 7fa30a42d640 1 -- 172.21.15.148:0/3608249670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c67a336bd0 con 0x55c67a2a8d60 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30a42d640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a316050 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fa30000cd80 tx=0x7fa300000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e7dfe83c9d7a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:13.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa2e57fa640 1 -- 172.21.15.148:0/3608249670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300054760 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa2e57fa640 1 -- 172.21.15.148:0/3608249670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa300019da0 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2f4041040 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa2e57fa640 1 -- 172.21.15.148:0/3608249670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3000587f0 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 10 client.10095 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55c67a2c6050 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa2e57fa640 1 -- 172.21.15.148:0/3608249670 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f4041040 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa2e57fa640 1 -- 172.21.15.148:0/3608249670 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fa300058350 con 0x55c67a2a8d60 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 10 client.10095 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 10 client.10095 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 20 client.10095 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2be090 0x55c67a2be470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 4 client.10095 mds_command: new command op to 9861 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 --> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c67a1f2f50 con 0x55c67a2be090 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.846+0000 7fa309c2c640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2be090 0x55c67a2be470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa2e57fa640 10 client.10095 ms_handle_connect on v2:172.21.15.148:6836/3188696943 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa309c2c640 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2be090 0x55c67a2be470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32de145ad89d2ac8 server_cookie=1b6239672dc4ec29 in_seq=0 out_seq=0 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa2e57fa640 1 -- 172.21.15.148:0/3608249670 <== mds.0 v2:172.21.15.148:6836/3188696943 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c67a1f2f50 con 0x55c67a2be090 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa2e57fa640 10 client.10095 handle_command_reply: tid=0 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 client.10095 shutdown 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 msgr2=0x55c67a316050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a316050 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fa30000cd80 tx=0x7fa300000bc0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 shutdown_connections 2024-04-25T10:04:13.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:6836/3188696943,v1:172.21.15.148:6837/3188696943] conn(0x55c67a2be090 0x55c67a2be470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c67a2caad0 0x55c67a2ea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c67a2ad230 0x55c67a2af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa2e57fa640 0 client.10095 ms_handle_reset on v2:172.21.15.148:6836/3188696943 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 --2- 172.21.15.148:0/3608249670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c67a2a8d60 0x55c67a316050 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 >> 172.21.15.148:0/3608249670 conn(0x55c67a282f10 msgr2=0x55c67a2a8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 shutdown_connections 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 1 -- 172.21.15.148:0/3608249670 wait complete. 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30f38b740 20 client.10095 trim_cache size 0 max 16384 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:04:13.860 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 9861, 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 22.967416358000001, 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 87, 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 147, 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 147, 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 28.019577646999998 2024-04-25T10:04:13.861 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:04:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2e806b4d0 msgr2=0x7fa2e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30c691640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2e806b4d0 0x7fa2e806d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4034000 tx=0x7fa2f402c000 comp rx=0 tx=0).stop 2024-04-25T10:04:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 msgr2=0x7fa30413c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.847+0000 7fa30c691640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 0x7fa30413c210 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fa30000c3c0 tx=0x7fa30000c880 comp rx=0 tx=0).stop 2024-04-25T10:04:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 shutdown_connections 2024-04-25T10:04:13.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa2e806b4d0 0x7fa2e806d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3041454b0 0x7fa30413a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa304144d20 0x7fa304143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 --2- 172.21.15.148:0/734799717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa304071300 0x7fa30413c210 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 >> 172.21.15.148:0/734799717 conn(0x7fa3040b3530 msgr2=0x7fa3040c6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 shutdown_connections 2024-04-25T10:04:13.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.848+0000 7fa30c691640 1 -- 172.21.15.148:0/734799717 wait complete. 2024-04-25T10:04:13.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start abs_test_path recursive 2024-04-25T10:04:13.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.961+0000 7f6ad7bb7640 1 Processor -- start 2024-04-25T10:04:13.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.961+0000 7f6ad7bb7640 1 -- start start 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad7bb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad7bb7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ad00ad570 0x7f6ad006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad7bb7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad006b500 con 0x7f6ad015dd80 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad7bb7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad006e410 con 0x7f6ad00ad570 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad7bb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad00b3930 con 0x7f6ad006abe0 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad5152640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6ad015dd80 0x7f6ad00b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42044/0 (socket says 172.21.15.148:42044) 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad5152640 1 -- 172.21.15.148:0/855790234 learned_addr learned my addr 172.21.15.148:0/855790234 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad6154640 1 --2- 172.21.15.148:0/855790234 >> v2:172.21.15.148:3300/0 conn(0x7f6ad00ad570 0x7f6ad006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad4951640 1 -- 172.21.15.148:0/855790234 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243322988 0 0) 0x7f6ad006b500 con 0x7f6ad015dd80 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad5953640 1 --2- 172.21.15.148:0/855790234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad4951640 1 -- 172.21.15.148:0/855790234 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6abc003520 con 0x7f6ad015dd80 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad6154640 1 -- 172.21.15.148:0/855790234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 msgr2=0x7f6ad006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad6154640 1 --2- 172.21.15.148:0/855790234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad6154640 1 -- 172.21.15.148:0/855790234 >> v1:172.21.15.148:6789/0 conn(0x7f6ad015dd80 legacy=0x7f6ad00b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.962+0000 7f6ad6154640 1 -- 172.21.15.148:0/855790234 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad00b3c10 con 0x7f6ad00ad570 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.963+0000 7f6ad6154640 1 --2- 172.21.15.148:0/855790234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad00ad570 0x7f6ad006ee60 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009d90 tx=0x7f6ac802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0eb99da53ee6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.963+0000 7f6ad4951640 1 -- 172.21.15.148:0/855790234 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8007740 con 0x7f6ad00ad570 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.963+0000 7f6ad4951640 1 -- 172.21.15.148:0/855790234 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac80078e0 con 0x7f6ad00ad570 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.963+0000 7f6ad4951640 1 -- 172.21.15.148:0/855790234 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8007cb0 con 0x7f6ad00ad570 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/855790234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad00ad570 msgr2=0x7f6ad006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 --2- 172.21.15.148:0/855790234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad00ad570 0x7f6ad006ee60 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009d90 tx=0x7f6ac802a660 comp rx=0 tx=0).stop 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/855790234 shutdown_connections 2024-04-25T10:04:13.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 --2- 172.21.15.148:0/855790234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad00ad570 0x7f6ad006ee60 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 --2- 172.21.15.148:0/855790234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/855790234 >> 172.21.15.148:0/855790234 conn(0x7f6ad00b5af0 msgr2=0x7f6ad00b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/855790234 shutdown_connections 2024-04-25T10:04:13.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.964+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/855790234 wait complete. 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.965+0000 7f6ad7bb7640 1 Processor -- start 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.965+0000 7f6ad7bb7640 1 -- start start 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad7bb7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad014b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad7bb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00ad570 0x7f6ad014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad7bb7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad015dd80 0x7f6ad015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad7bb7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0070280 con 0x7f6ad015dd80 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad7bb7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad006e940 con 0x7f6ad00ad570 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad7bb7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0070400 con 0x7f6ad006abe0 2024-04-25T10:04:13.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad015dd80 0x7f6ad015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad015dd80 0x7f6ad015f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56758/0 (socket says 172.21.15.148:56758) 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 -- 172.21.15.148:0/2060083100 learned_addr learned my addr 172.21.15.148:0/2060083100 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad5953640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad014b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad5152640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00ad570 0x7f6ad014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 -- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 msgr2=0x7f6ad014b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad014b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 -- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00ad570 msgr2=0x7f6ad014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00ad570 0x7f6ad014e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 -- 172.21.15.148:0/2060083100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad0157e90 con 0x7f6ad015dd80 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad5152640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad00ad570 0x7f6ad014e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad5953640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad006abe0 0x7f6ad014b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.966+0000 7f6ad6154640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad015dd80 0x7f6ad015f8d0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f6ac802ac50 tx=0x7f6ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.968+0000 7f6ac67fc640 1 -- 172.21.15.148:0/2060083100 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8040070 con 0x7f6ad015dd80 2024-04-25T10:04:13.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.968+0000 7f6ac67fc640 1 -- 172.21.15.148:0/2060083100 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8031bc0 con 0x7f6ad015dd80 2024-04-25T10:04:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.968+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/2060083100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad0158130 con 0x7f6ad015dd80 2024-04-25T10:04:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.968+0000 7f6ac67fc640 1 -- 172.21.15.148:0/2060083100 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8009450 con 0x7f6ad015dd80 2024-04-25T10:04:13.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.968+0000 7f6ad7bb7640 1 -- 172.21.15.148:0/2060083100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad01586f0 con 0x7f6ad015dd80 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.969+0000 7f6ac67fc640 1 -- 172.21.15.148:0/2060083100 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ac8039040 con 0x7f6ad015dd80 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.970+0000 7f6ac67fc640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ab406b4d0 0x7f6ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.970+0000 7f6ac67fc640 1 -- 172.21.15.148:0/2060083100 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6ac80abc50 con 0x7f6ad015dd80 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.970+0000 7f6ad5953640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ab406b4d0 0x7f6ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.970+0000 7f6ad5953640 1 --2- 172.21.15.148:0/2060083100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ab406b4d0 0x7f6ab406d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006fd0 tx=0x7f6acc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.971+0000 7f6ada996740 1 Processor -- start 2024-04-25T10:04:13.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.971+0000 7f6ada996740 1 -- start start 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140feb97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ada996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140feb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140fe89c40 0x56140fe8d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ada996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140fdf2b20 con 0x56140feb73c0 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140fd9be10 con 0x56140feb6fe0 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56140fdeebc0 con 0x56140fe89c40 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140feb9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140feb9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56772/0 (socket says 172.21.15.148:56772) 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 -- 172.21.15.148:0/914222402 learned_addr learned my addr 172.21.15.148:0/914222402 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5953640 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140feb97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad6154640 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140fe89c40 0x56140fe8d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 -- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140fe89c40 msgr2=0x56140fe8d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140fe89c40 0x56140fe8d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 -- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 msgr2=0x56140feb97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140feb97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 -- 172.21.15.148:0/914222402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56140fbd37e0 con 0x56140feb73c0 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.972+0000 7f6ad5152640 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140feb9d30 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0000f60 tx=0x7f6ac0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4860965a58dcd4da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ab37fe640 1 -- 172.21.15.148:0/914222402 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0018070 con 0x56140feb73c0 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ab37fe640 1 -- 172.21.15.148:0/914222402 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac0005cc0 con 0x56140feb73c0 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ab37fe640 1 -- 172.21.15.148:0/914222402 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac00072f0 con 0x56140feb73c0 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 -- 172.21.15.148:0/914222402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 msgr2=0x56140feb9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140feb9d30 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0000f60 tx=0x7f6ac0001b10 comp rx=0 tx=0).stop 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 -- 172.21.15.148:0/914222402 shutdown_connections 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140fe89c40 0x56140fe8d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140feb9d30 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 --2- 172.21.15.148:0/914222402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140feb97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 -- 172.21.15.148:0/914222402 >> 172.21.15.148:0/914222402 conn(0x56140fe8beb0 msgr2=0x56140fed47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 -- 172.21.15.148:0/914222402 shutdown_connections 2024-04-25T10:04:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.973+0000 7f6ada996740 1 -- 172.21.15.148:0/914222402 wait complete. 2024-04-25T10:04:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.974+0000 7f6ada996740 1 -- start start 2024-04-25T10:04:13.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.974+0000 7f6ada996740 1 Processor -- start 2024-04-25T10:04:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb6fe0 0x56140fecf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fed8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fed0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140fe8f060 con 0x56140feb73c0 2024-04-25T10:04:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140fd91540 con 0x56140feb1ca0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56140fe8f340 con 0x56140feb6fe0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ada996740 10 client.0 _ll_register_callbacks cb 0x56140fb06bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fed8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fed8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56774/0 (socket says 172.21.15.148:56774) 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 -- 172.21.15.148:0/3861729662 learned_addr learned my addr 172.21.15.148:0/3861729662 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5953640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb6fe0 0x56140fecf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad6154640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fed0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 -- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb6fe0 msgr2=0x56140fecf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb6fe0 0x56140fecf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 -- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 msgr2=0x56140fed0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fed0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 -- 172.21.15.148:0/3861729662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56140fc33f70 con 0x56140feb73c0 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad6154640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fed0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:13.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ad5152640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fed8270 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x56140fe91540 tx=0x7f6ac0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cd2c1448dff5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.975+0000 7f6ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/3861729662 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0023020 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ac0005cc0 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/3861729662 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6ac0007690 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/3861729662 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ac002d050 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 10 client.10110 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x56140fac70d0 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/3861729662 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac0005cc0 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/3861729662 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f6ac0030020 con 0x56140feb73c0 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 10 client.10110 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 10 client.10110 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 20 client.10110 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140feee320 0x56140fef24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 4 client.10110 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56140fe00060 con 0x56140feee320 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ad5953640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140feee320 0x56140fef24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ab0ff9640 10 client.10110 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.976+0000 7f6ad5953640 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140feee320 0x56140fef24e0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=60504b776dd5606f server_cookie=e6b21d03d458ec48 in_seq=0 out_seq=0 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/3861729662 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56140fe00060 con 0x56140feee320 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ab0ff9640 10 client.10110 handle_command_reply: tid=0 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 client.10110 shutdown 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 msgr2=0x56140fed8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fed8270 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x56140fe91540 tx=0x7f6ac0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 shutdown_connections 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140feee320 0x56140fef24e0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fed0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ab0ff9640 0 client.10110 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fed8270 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 --2- 172.21.15.148:0/3861729662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb6fe0 0x56140fecf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 >> 172.21.15.148:0/3861729662 conn(0x56140fe89c40 msgr2=0x56140fe8aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 shutdown_connections 2024-04-25T10:04:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 1 -- 172.21.15.148:0/3861729662 wait complete. 2024-04-25T10:04:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.977+0000 7f6ada996740 20 client.10110 trim_cache size 0 max 16384 2024-04-25T10:04:13.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.992+0000 7f6ada996740 1 Processor -- start 2024-04-25T10:04:13.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 -- start start 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb1ca0 0x56140fe8e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140fe8a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fe8ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad0200510 con 0x56140feb73c0 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad015ed80 con 0x56140feb6fe0 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad01ff0f0 con 0x56140feb1ca0 2024-04-25T10:04:14.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ad6154640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fe8ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ad6154640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fe8ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56778/0 (socket says 172.21.15.148:56778) 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ad6154640 1 -- 172.21.15.148:0/898427480 learned_addr learned my addr 172.21.15.148:0/898427480 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.993+0000 7f6ad5953640 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb1ca0 0x56140fe8e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad5152640 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140fe8a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad6154640 1 -- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb1ca0 msgr2=0x56140fe8e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad6154640 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb1ca0 0x56140fe8e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad6154640 1 -- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 msgr2=0x56140fe8a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad6154640 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140fe8a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad6154640 1 -- 172.21.15.148:0/898427480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56140fdaef30 con 0x56140feb73c0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad5953640 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb1ca0 0x56140fe8e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ad6154640 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fe8ad40 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f6ac807a250 tx=0x7f6ac8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94515f648358643b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ab2ffd640 1 -- 172.21.15.148:0/898427480 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac80a7070 con 0x56140feb73c0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ab2ffd640 1 -- 172.21.15.148:0/898427480 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8082a20 con 0x56140feb73c0 2024-04-25T10:04:14.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ab2ffd640 1 -- 172.21.15.148:0/898427480 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8078610 con 0x56140feb73c0 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 -- 172.21.15.148:0/898427480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 msgr2=0x56140fe8ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fe8ad40 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f6ac807a250 tx=0x7f6ac8009750 comp rx=0 tx=0).stop 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 -- 172.21.15.148:0/898427480 shutdown_connections 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb73c0 0x56140fe8ad40 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb6fe0 0x56140fe8a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 --2- 172.21.15.148:0/898427480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb1ca0 0x56140fe8e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 -- 172.21.15.148:0/898427480 >> 172.21.15.148:0/898427480 conn(0x56140fe89c40 msgr2=0x56140fed4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 -- 172.21.15.148:0/898427480 shutdown_connections 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.994+0000 7f6ada996740 1 -- 172.21.15.148:0/898427480 wait complete. 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.995+0000 7f6ada996740 1 -- start start 2024-04-25T10:04:14.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.995+0000 7f6ada996740 1 Processor -- start 2024-04-25T10:04:14.007 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:04:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fecedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 0x56140fecf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb73c0 0x56140ff43200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140feee1a0 con 0x56140feb6fe0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56140fed1910 con 0x56140feb1ca0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56140feee7a0 con 0x56140feb73c0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ada996740 10 client.0 _ll_register_callbacks cb 0x7f6ac0007cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 0x56140fecf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 0x56140fecf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56794/0 (socket says 172.21.15.148:56794) 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 -- 172.21.15.148:0/4258591343 learned_addr learned my addr 172.21.15.148:0/4258591343 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5953640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fecedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad6154640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb73c0 0x56140ff43200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 -- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb73c0 msgr2=0x56140ff43200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb73c0 0x56140ff43200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 -- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 msgr2=0x56140fecedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fecedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad5152640 1 -- 172.21.15.148:0/4258591343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56140ff40ae0 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.996+0000 7f6ad6154640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb73c0 0x56140ff43200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ad5152640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 0x56140fecf310 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0005880 tx=0x7f6ac0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673603bb7987b8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/4258591343 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0023040 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/4258591343 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6ac00283f0 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ac8082a20 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/4258591343 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ac0018070 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 10 client.10116 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x56140ff40000 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/4258591343 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8082a20 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/4258591343 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f6ac0030040 con 0x56140feb6fe0 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 10 client.10116 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 10 client.10116 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 20 client.10116 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140ff43970 0x56140fecd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 4 client.10116 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}) v1 -- 0x56140fdfa3b0 con 0x56140ff43970 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.997+0000 7f6ad5953640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140ff43970 0x56140fecd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ab0ff9640 10 client.10116 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ad5953640 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140ff43970 0x56140fecd490 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=b863f1e971b13453 server_cookie=e6d567e46dddb477 in_seq=0 out_seq=0 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ab0ff9640 1 -- 172.21.15.148:0/4258591343 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x56140fdfa3b0 con 0x56140ff43970 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ab0ff9640 10 client.10116 handle_command_reply: tid=0 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 client.10116 shutdown 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 msgr2=0x56140fecf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 0x56140fecf310 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0005880 tx=0x7f6ac0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 shutdown_connections 2024-04-25T10:04:14.010 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x56140ff43970 0x56140fecd490 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56140feb73c0 0x56140ff43200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56140feb6fe0 0x56140fecf310 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ab0ff9640 0 client.10116 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 --2- 172.21.15.148:0/4258591343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56140feb1ca0 0x56140fecedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 >> 172.21.15.148:0/4258591343 conn(0x56140fe89c40 msgr2=0x56140fe90d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 shutdown_connections 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 1 -- 172.21.15.148:0/4258591343 wait complete. 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:13.998+0000 7f6ada996740 20 client.10116 trim_cache size 0 max 16384 2024-04-25T10:04:14.011 INFO:teuthology.orchestra.run.smithi148.stderr:Error EINVAL: MDS is not active 2024-04-25T10:04:14.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2024-04-25T10:04:14.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.106+0000 7fc0f8a2a640 1 Processor -- start 2024-04-25T10:04:14.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.106+0000 7fc0f8a2a640 1 -- start start 2024-04-25T10:04:14.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f8a2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0f415dc20 0x7fc0f40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f8a2a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0f4161710 0x7fc0f40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f8a2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f40b9330 0x7fc0f406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f8a2a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f415e000 con 0x7fc0f4161710 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f8a2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f40d1de0 con 0x7fc0f415dc20 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f8a2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f4161c60 con 0x7fc0f40b9330 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f1d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0f4161710 0x7fc0f40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f1d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc0f4161710 0x7fc0f40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56802/0 (socket says 172.21.15.148:56802) 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/1349943372 learned_addr learned my addr 172.21.15.148:0/1349943372 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/1349943372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f40b9330 0x7fc0f406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.107+0000 7fc0f259c640 1 --2- 172.21.15.148:0/1349943372 >> v2:172.21.15.188:3300/0 conn(0x7fc0f415dc20 0x7fc0f40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/1349943372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f40b9330 msgr2=0x7fc0f406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/1349943372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f40b9330 0x7fc0f406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/1349943372 >> v2:172.21.15.188:3300/0 conn(0x7fc0f415dc20 msgr2=0x7fc0f40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/1349943372 >> v2:172.21.15.188:3300/0 conn(0x7fc0f415dc20 0x7fc0f40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/1349943372 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4161f40 con 0x7fc0f4161710 2024-04-25T10:04:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/1349943372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f40cebb0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40d1520 tx=0x7fc0e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c008e5841007b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f159a640 1 -- 172.21.15.148:0/1349943372 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8037070 con 0x7fc0f4161710 2024-04-25T10:04:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.108+0000 7fc0f159a640 1 -- 172.21.15.148:0/1349943372 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e8032440 con 0x7fc0f4161710 2024-04-25T10:04:14.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f159a640 1 -- 172.21.15.148:0/1349943372 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8032810 con 0x7fc0f4161710 2024-04-25T10:04:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1349943372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 msgr2=0x7fc0f40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 --2- 172.21.15.148:0/1349943372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f40cebb0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40d1520 tx=0x7fc0e802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1349943372 shutdown_connections 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 --2- 172.21.15.148:0/1349943372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f40b9330 0x7fc0f406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 --2- 172.21.15.148:0/1349943372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f40cebb0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 --2- 172.21.15.148:0/1349943372 >> v2:172.21.15.188:3300/0 conn(0x7fc0f415dc20 0x7fc0f40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1349943372 >> 172.21.15.148:0/1349943372 conn(0x7fc0f40d0420 msgr2=0x7fc0f40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1349943372 shutdown_connections 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.109+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1349943372 wait complete. 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.110+0000 7fc0f8a2a640 1 Processor -- start 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.110+0000 7fc0f8a2a640 1 -- start start 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f8a2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f40b9330 0x7fc0f41e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f8a2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f415dc20 0x7fc0f40c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f8a2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f41475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f8a2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40b5e50 con 0x7fc0f4161710 2024-04-25T10:04:14.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f8a2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f4162500 con 0x7fc0f40b9330 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f8a2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40b5fd0 con 0x7fc0f415dc20 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f41475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f41475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56818/0 (socket says 172.21.15.148:56818) 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.111+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/1599148830 learned_addr learned my addr 172.21.15.148:0/1599148830 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f259c640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f40b9330 0x7fc0f41e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/1599148830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f415dc20 msgr2=0x7fc0f40c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f415dc20 0x7fc0f40c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0f415dc20 0x7fc0f40c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/1599148830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f40b9330 msgr2=0x7fc0f41e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0f40b9330 0x7fc0f41e9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/1599148830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40caec0 con 0x7fc0f4161710 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0f4161710 0x7fc0f41475b0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00e5a0 tx=0x7fc0ec00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0e2ffd640 1 -- 172.21.15.148:0/1599148830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec017070 con 0x7fc0f4161710 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0e2ffd640 1 -- 172.21.15.148:0/1599148830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec013680 con 0x7fc0f4161710 2024-04-25T10:04:14.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0e2ffd640 1 -- 172.21.15.148:0/1599148830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec013a70 con 0x7fc0f4161710 2024-04-25T10:04:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1599148830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f40cb1a0 con 0x7fc0f4161710 2024-04-25T10:04:14.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.112+0000 7fc0f8a2a640 1 -- 172.21.15.148:0/1599148830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f41e92f0 con 0x7fc0f4161710 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.113+0000 7fc0e2ffd640 1 -- 172.21.15.148:0/1599148830 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0ec01a070 con 0x7fc0f4161710 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.114+0000 7fc0e2ffd640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc0d406b4d0 0x7fc0d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.114+0000 7fc0f259c640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc0d406b4d0 0x7fc0d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.114+0000 7fc0e2ffd640 1 -- 172.21.15.148:0/1599148830 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc0ec08a150 con 0x7fc0f4161710 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.114+0000 7fc0f259c640 1 --2- 172.21.15.148:0/1599148830 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc0d406b4d0 0x7fc0d406d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4007380 tx=0x7fc0e400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.115+0000 7fc0fb807740 1 Processor -- start 2024-04-25T10:04:14.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.115+0000 7fc0fb807740 1 -- start start 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272d9c40 0x55e227293220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272908c0 0x55e2272925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0fb807740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2271f69b0 con 0x55e2272be200 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22719fc60 con 0x55e2272d9c40 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2271f2a50 con 0x55e2272908c0 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272be5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56820/0 (socket says 172.21.15.148:56820) 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 -- 172.21.15.148:0/3510407081 learned_addr learned my addr 172.21.15.148:0/3510407081 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272908c0 0x55e2272925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272d9c40 0x55e227293220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 -- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272908c0 msgr2=0x55e2272925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272908c0 0x55e2272925e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 -- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272d9c40 msgr2=0x55e227293220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272d9c40 0x55e227293220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 -- 172.21.15.148:0/3510407081 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e227193b30 con 0x55e2272be200 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0f259c640 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272be5e0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009e20 tx=0x7fc0e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5e8ceac974c393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0d3fff640 1 -- 172.21.15.148:0/3510407081 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4026070 con 0x55e2272be200 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0d3fff640 1 -- 172.21.15.148:0/3510407081 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40149a0 con 0x55e2272be200 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.116+0000 7fc0d3fff640 1 -- 172.21.15.148:0/3510407081 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4014d90 con 0x55e2272be200 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 -- 172.21.15.148:0/3510407081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 msgr2=0x55e2272be5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272be5e0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009e20 tx=0x7fc0e4012000 comp rx=0 tx=0).stop 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 -- 172.21.15.148:0/3510407081 shutdown_connections 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272908c0 0x55e2272925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272d9c40 0x55e227293220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3510407081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272be5e0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 -- 172.21.15.148:0/3510407081 >> 172.21.15.148:0/3510407081 conn(0x55e22725b270 msgr2=0x55e2272d9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 -- 172.21.15.148:0/3510407081 shutdown_connections 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.117+0000 7fc0fb807740 1 -- 172.21.15.148:0/3510407081 wait complete. 2024-04-25T10:04:14.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.118+0000 7fc0fb807740 1 -- start start 2024-04-25T10:04:14.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.118+0000 7fc0fb807740 1 Processor -- start 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272ecb20 0x55e2272c1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e227293760 con 0x55e2272be200 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2271b3200 con 0x55e2272b7390 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e22728fca0 con 0x55e2272ecb20 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 10 client.0 _ll_register_callbacks cb 0x55e2271b2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56836/0 (socket says 172.21.15.148:56836) 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 -- 172.21.15.148:0/3022512887 learned_addr learned my addr 172.21.15.148:0/3022512887 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272ecb20 0x55e2272c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 -- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272ecb20 msgr2=0x55e2272c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272ecb20 0x55e2272c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 -- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 msgr2=0x55e2272ec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272ec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 -- 172.21.15.148:0/3022512887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e226fd5500 con 0x55e2272be200 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272ecb20 0x55e2272c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0f259c640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272b6e50 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4024040 tx=0x7fc0e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b8d99514c67afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0d17fa640 1 -- 172.21.15.148:0/3022512887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4030020 con 0x55e2272be200 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0d17fa640 1 -- 172.21.15.148:0/3022512887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc0e402c420 con 0x55e2272be200 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.119+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0e40149a0 con 0x55e2272be200 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0d17fa640 1 -- 172.21.15.148:0/3022512887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e403a050 con 0x55e2272be200 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 10 client.10128 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55e227035c90 con 0x55e2272be200 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0d17fa640 1 -- 172.21.15.148:0/3022512887 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40149a0 con 0x55e2272be200 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0d17fa640 1 -- 172.21.15.148:0/3022512887 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fc0e403d020 con 0x55e2272be200 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 10 client.10128 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 10 client.10128 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 20 client.10128 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272dba50 0x55e2272a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 4 client.10128 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e227203ef0 con 0x55e2272dba50 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272dba50 0x55e2272a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0d17fa640 10 client.10128 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.120+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272dba50 0x55e2272a9260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=bdce180cd3e46aa4 server_cookie=367981afec80df3 in_seq=0 out_seq=0 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0d17fa640 1 -- 172.21.15.148:0/3022512887 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e227203ef0 con 0x55e2272dba50 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0d17fa640 10 client.10128 handle_command_reply: tid=0 2024-04-25T10:04:14.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 client.10128 shutdown 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 msgr2=0x55e2272b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272b6e50 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4024040 tx=0x7fc0e4004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 shutdown_connections 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272dba50 0x55e2272a9260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272ecb20 0x55e2272c1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0d17fa640 0 client.10128 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3022512887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272b6e50 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 >> 172.21.15.148:0/3022512887 conn(0x55e22725b270 msgr2=0x55e22728efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 shutdown_connections 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 1 -- 172.21.15.148:0/3022512887 wait complete. 2024-04-25T10:04:14.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.121+0000 7fc0fb807740 20 client.10128 trim_cache size 0 max 16384 2024-04-25T10:04:14.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.136+0000 7fc0fb807740 1 Processor -- start 2024-04-25T10:04:14.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.136+0000 7fc0fb807740 1 -- start start 2024-04-25T10:04:14.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272be200 0x55e2272d8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 0x55e2272b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0fb807740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f4200650 con 0x55e2272ecb20 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f414a490 con 0x55e2272b7390 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f41ff270 con 0x55e2272be200 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 0x55e2272b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 0x55e2272b7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56840/0 (socket says 172.21.15.148:56840) 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/3239729305 learned_addr learned my addr 172.21.15.148:0/3239729305 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/3239729305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272be200 msgr2=0x55e2272d8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272be200 0x55e2272d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/3239729305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 msgr2=0x55e2272e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 -- 172.21.15.148:0/3239729305 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e226ecae20 con 0x55e2272ecb20 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 0x55e2272b7aa0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00efd0 tx=0x7fc0ec058180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6c8fec710a6c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0d37fe640 1 -- 172.21.15.148:0/3239729305 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec086070 con 0x55e2272ecb20 2024-04-25T10:04:14.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0d37fe640 1 -- 172.21.15.148:0/3239729305 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec070210 con 0x55e2272ecb20 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.137+0000 7fc0d37fe640 1 -- 172.21.15.148:0/3239729305 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec070070 con 0x55e2272ecb20 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 -- 172.21.15.148:0/3239729305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 msgr2=0x55e2272b7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 0x55e2272b7aa0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00efd0 tx=0x7fc0ec058180 comp rx=0 tx=0).stop 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 -- 172.21.15.148:0/3239729305 shutdown_connections 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272ecb20 0x55e2272b7aa0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272be200 0x55e2272d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 --2- 172.21.15.148:0/3239729305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272b7390 0x55e2272e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 -- 172.21.15.148:0/3239729305 >> 172.21.15.148:0/3239729305 conn(0x55e22725b270 msgr2=0x55e227291cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 -- 172.21.15.148:0/3239729305 shutdown_connections 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 -- 172.21.15.148:0/3239729305 wait complete. 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 -- start start 2024-04-25T10:04:14.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.138+0000 7fc0fb807740 1 Processor -- start 2024-04-25T10:04:14.150 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272b7390 0x55e227343340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272ecb20 0x55e2272a9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2272aa3d0 con 0x55e2272be200 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2272a8a60 con 0x55e2272ecb20 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2272bb750 con 0x55e2272b7390 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0fb807740 10 client.0 _ll_register_callbacks cb 0x7fc0e4024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0f1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0f1d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272e8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56842/0 (socket says 172.21.15.148:56842) 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/2137038813 learned_addr learned my addr 172.21.15.148:0/2137038813 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.139+0000 7fc0f259c640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272b7390 0x55e227343340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272ecb20 0x55e2272a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272b7390 msgr2=0x55e227343340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272b7390 0x55e227343340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272ecb20 msgr2=0x55e2272a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272ecb20 0x55e2272a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f1d9b640 1 -- 172.21.15.148:0/2137038813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2271b1c40 con 0x55e2272be200 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f2d9d640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272ecb20 0x55e2272a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0f1d9b640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272e8390 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009960 tx=0x7fc0e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f63938de74cafac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0d17fa640 1 -- 172.21.15.148:0/2137038813 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e804a060 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0d17fa640 1 -- 172.21.15.148:0/2137038813 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc0e8045b80 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0ec070210 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0d17fa640 1 -- 172.21.15.148:0/2137038813 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0e8050050 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 10 client.10134 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55e2271bd980 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0d17fa640 1 -- 172.21.15.148:0/2137038813 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec070210 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0d17fa640 1 -- 172.21.15.148:0/2137038813 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fc0e8053020 con 0x55e2272be200 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 10 client.10134 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 10 client.10134 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 20 client.10134 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272c1610 0x55e2272da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 4 client.10134 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.140+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55e2271fe240 con 0x55e2272c1610 2024-04-25T10:04:14.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0f259c640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272c1610 0x55e2272da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0d17fa640 10 client.10134 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0f259c640 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272c1610 0x55e2272da140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=51bdd7774623c97d server_cookie=b3d7e18374d081e in_seq=0 out_seq=0 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0d17fa640 1 -- 172.21.15.148:0/2137038813 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x55e2271fe240 con 0x55e2272c1610 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0d17fa640 10 client.10134 handle_command_reply: tid=0 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 client.10134 shutdown 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 msgr2=0x55e2272e8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272e8390 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009960 tx=0x7fc0e8004730 comp rx=0 tx=0).stop 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 shutdown_connections 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x55e2272c1610 0x55e2272da140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2272ecb20 0x55e2272a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e2272be200 0x55e2272e8390 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 --2- 172.21.15.148:0/2137038813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2272b7390 0x55e227343340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0d17fa640 0 client.10134 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 >> 172.21.15.148:0/2137038813 conn(0x55e22725b270 msgr2=0x55e2272f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 shutdown_connections 2024-04-25T10:04:14.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 1 -- 172.21.15.148:0/2137038813 wait complete. 2024-04-25T10:04:14.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.141+0000 7fc0fb807740 20 client.10134 trim_cache size 0 max 16384 2024-04-25T10:04:14.154 INFO:teuthology.orchestra.run.smithi148.stderr:Error EINVAL: MDS is not active 2024-04-25T10:04:14.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2024-04-25T10:04:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.248+0000 7fd873e53640 1 Processor -- start 2024-04-25T10:04:14.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.248+0000 7fd873e53640 1 -- start start 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd873e53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd86c1454b0 0x7fd86c070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd873e53640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c0ceba0 con 0x7fd86c071300 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd873e53640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c0cee80 con 0x7fd86c144d20 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd873e53640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86c0b4830 con 0x7fd86c1454b0 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd8723f0640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd86c071300 0x7fd86c0b4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42048/0 (socket says 172.21.15.148:42048) 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd8723f0640 1 -- 172.21.15.148:0/1240270202 learned_addr learned my addr 172.21.15.148:0/1240270202 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd8713ee640 1 --2- 172.21.15.148:0/1240270202 >> v2:172.21.15.188:3300/0 conn(0x7fd86c1454b0 0x7fd86c070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314146246 0 0) 0x7fd86c0ceba0 con 0x7fd86c071300 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.249+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd854003520 con 0x7fd86c071300 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3016724376 0 0) 0x7fd854003520 con 0x7fd86c071300 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 >> v2:172.21.15.188:3300/0 conn(0x7fd86c1454b0 msgr2=0x7fd86c070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 --2- 172.21.15.148:0/1240270202 >> v2:172.21.15.188:3300/0 conn(0x7fd86c1454b0 0x7fd86c070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 >> v1:172.21.15.188:6789/0 conn(0x7fd86c144d20 legacy=0x7fd86c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c0cf160 con 0x7fd86c071300 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd864001670 con 0x7fd86c071300 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 >> v1:172.21.15.148:6789/0 conn(0x7fd86c071300 legacy=0x7fd86c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8540039b0 0x7fd854003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd854004800 0x7fd854004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8540051c0 0x7fd85400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c0b4830 con 0x7fd8540039b0 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85400c090 con 0x7fd854004800 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85400c290 con 0x7fd8540051c0 2024-04-25T10:04:14.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd871bef640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8540039b0 0x7fd854003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd8713ee640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd854004800 0x7fd854004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd8723f0640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8540051c0 0x7fd85400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd871bef640 1 -- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8540051c0 msgr2=0x7fd85400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd871bef640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8540051c0 0x7fd85400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd871bef640 1 -- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd854004800 msgr2=0x7fd854004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd871bef640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd854004800 0x7fd854004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd871bef640 1 -- 172.21.15.148:0/1240270202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd854003520 con 0x7fd8540039b0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.250+0000 7fd8723f0640 1 -- 172.21.15.148:0/1240270202 reap_dead start 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.251+0000 7fd871bef640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8540039b0 0x7fd854003df0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001e50 tx=0x7fd854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf5cc125c7de6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.251+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c001a70 con 0x7fd8540039b0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.251+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c02ae40 con 0x7fd8540039b0 2024-04-25T10:04:14.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.251+0000 7fd870bed640 1 -- 172.21.15.148:0/1240270202 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c003bb0 con 0x7fd8540039b0 2024-04-25T10:04:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.251+0000 7fd873e53640 1 -- 172.21.15.148:0/1240270202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8540039b0 msgr2=0x7fd854003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.251+0000 7fd873e53640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8540039b0 0x7fd854003df0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001e50 tx=0x7fd854005830 comp rx=0 tx=0).stop 2024-04-25T10:04:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.252+0000 7fd873e53640 1 -- 172.21.15.148:0/1240270202 shutdown_connections 2024-04-25T10:04:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.252+0000 7fd873e53640 1 --2- 172.21.15.148:0/1240270202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8540039b0 0x7fd854003df0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.252+0000 7fd873e53640 1 -- 172.21.15.148:0/1240270202 >> 172.21.15.148:0/1240270202 conn(0x7fd86c0b3530 msgr2=0x7fd86c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.252+0000 7fd873e53640 1 -- 172.21.15.148:0/1240270202 shutdown_connections 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.252+0000 7fd873e53640 1 -- 172.21.15.148:0/1240270202 wait complete. 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 Processor -- start 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 -- start start 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8540039b0 0x7fd86c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c1f7b00 0x7fd86c1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86c1f33e0 0x7fd86c0b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c145100 con 0x7fd86c1f33e0 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c071730 con 0x7fd8540039b0 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.253+0000 7fd873e53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c145280 con 0x7fd86c1f7b00 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86c1f33e0 0x7fd86c0b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86c1f33e0 0x7fd86c0b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56848/0 (socket says 172.21.15.148:56848) 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 -- 172.21.15.148:0/2747598825 learned_addr learned my addr 172.21.15.148:0/2747598825 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd871bef640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8540039b0 0x7fd86c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8713ee640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c1f7b00 0x7fd86c1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 -- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c1f7b00 msgr2=0x7fd86c1f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86c1f7b00 0x7fd86c1f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 -- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8540039b0 msgr2=0x7fd86c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8540039b0 0x7fd86c0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 -- 172.21.15.148:0/2747598825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c0cd750 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd871bef640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8540039b0 0x7fd86c0cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd8723f0640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86c1f33e0 0x7fd86c0b5710 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fd86c145890 tx=0x7fd86400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd86a7fc640 1 -- 172.21.15.148:0/2747598825 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864019070 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd86a7fc640 1 -- 172.21.15.148:0/2747598825 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864010510 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd873e53640 1 -- 172.21.15.148:0/2747598825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c13c350 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd86a7fc640 1 -- 172.21.15.148:0/2747598825 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864010900 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.254+0000 7fd873e53640 1 -- 172.21.15.148:0/2747598825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c0cd470 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.256+0000 7fd86a7fc640 1 -- 172.21.15.148:0/2747598825 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd864010aa0 con 0x7fd86c1f33e0 2024-04-25T10:04:14.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.256+0000 7fd86a7fc640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd85806b500 0x7fd85806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.256+0000 7fd86a7fc640 1 -- 172.21.15.148:0/2747598825 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd8640897c0 con 0x7fd86c1f33e0 2024-04-25T10:04:14.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.256+0000 7fd871bef640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd85806b500 0x7fd85806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.257+0000 7fd871bef640 1 --2- 172.21.15.148:0/2747598825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd85806b500 0x7fd85806d9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001e50 tx=0x7fd85c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.257+0000 7fd876c2a740 1 Processor -- start 2024-04-25T10:04:14.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.257+0000 7fd876c2a740 1 -- start start 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d3408c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340a7c40 0x561d34061220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3405e8c0 0x561d340605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd876c2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d33fc49b0 con 0x561d3408c200 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d33f6dc60 con 0x561d340a7c40 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d33fc0a50 con 0x561d3405e8c0 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d3408c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d3408c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56860/0 (socket says 172.21.15.148:56860) 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 -- 172.21.15.148:0/663499207 learned_addr learned my addr 172.21.15.148:0/663499207 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd8723f0640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3405e8c0 0x561d340605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd8713ee640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340a7c40 0x561d34061220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 -- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3405e8c0 msgr2=0x561d340605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3405e8c0 0x561d340605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 -- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340a7c40 msgr2=0x561d34061220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340a7c40 0x561d34061220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd871bef640 1 -- 172.21.15.148:0/663499207 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d33f61b30 con 0x561d3408c200 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd8723f0640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3405e8c0 0x561d340605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.258+0000 7fd8713ee640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340a7c40 0x561d34061220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd871bef640 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d3408c5e0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001f40 tx=0x7fd85c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d658edb97764fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd8537fe640 1 -- 172.21.15.148:0/663499207 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c04b070 con 0x561d3408c200 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd8537fe640 1 -- 172.21.15.148:0/663499207 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c0055d0 con 0x561d3408c200 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd8537fe640 1 -- 172.21.15.148:0/663499207 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c041040 con 0x561d3408c200 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd876c2a740 1 -- 172.21.15.148:0/663499207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 msgr2=0x561d3408c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd876c2a740 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d3408c5e0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001f40 tx=0x7fd85c02c000 comp rx=0 tx=0).stop 2024-04-25T10:04:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.259+0000 7fd876c2a740 1 -- 172.21.15.148:0/663499207 shutdown_connections 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3405e8c0 0x561d340605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340a7c40 0x561d34061220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 --2- 172.21.15.148:0/663499207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d3408c5e0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 -- 172.21.15.148:0/663499207 >> 172.21.15.148:0/663499207 conn(0x561d34029270 msgr2=0x561d340a7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 -- 172.21.15.148:0/663499207 shutdown_connections 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 -- 172.21.15.148:0/663499207 wait complete. 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 -- start start 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.260+0000 7fd876c2a740 1 Processor -- start 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d34084e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3408f4e0 0x561d340b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d340b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d34061760 con 0x561d3408c200 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d33f81200 con 0x561d340b7790 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d3405dca0 con 0x561d3408f4e0 2024-04-25T10:04:14.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd876c2a740 10 client.0 _ll_register_callbacks cb 0x561d33f80bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d34084e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d34084e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56868/0 (socket says 172.21.15.148:56868) 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 -- 172.21.15.148:0/2750789440 learned_addr learned my addr 172.21.15.148:0/2750789440 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 -- 172.21.15.148:0/2750789440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3408f4e0 msgr2=0x561d340b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3408f4e0 0x561d340b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 -- 172.21.15.148:0/2750789440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 msgr2=0x561d340b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d340b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.261+0000 7fd871bef640 1 -- 172.21.15.148:0/2750789440 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d33da3500 con 0x561d3408c200 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd871bef640 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d34084e50 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001f70 tx=0x7fd85c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d9767b407fdb8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd850ff9640 1 -- 172.21.15.148:0/2750789440 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c04b040 con 0x561d3408c200 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd850ff9640 1 -- 172.21.15.148:0/2750789440 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd85c052cb0 con 0x561d3408c200 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd85c0055d0 con 0x561d3408c200 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd850ff9640 1 -- 172.21.15.148:0/2750789440 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd85c037040 con 0x561d3408c200 2024-04-25T10:04:14.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 10 client.10146 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x561d33e03c90 con 0x561d3408c200 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd850ff9640 1 -- 172.21.15.148:0/2750789440 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c0055d0 con 0x561d3408c200 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd850ff9640 1 -- 172.21.15.148:0/2750789440 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fd85c006c10 con 0x561d3408c200 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 10 client.10146 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 10 client.10146 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 20 client.10146 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340c2ec0 0x561d340c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 4 client.10146 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d33fd1ef0 con 0x561d340c2ec0 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.262+0000 7fd8713ee640 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340c2ec0 0x561d340c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd8713ee640 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340c2ec0 0x561d340c52b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=a34015d27bb3bafe server_cookie=21d5da8636a462bd in_seq=0 out_seq=0 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd850ff9640 10 client.10146 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd850ff9640 1 -- 172.21.15.148:0/2750789440 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d33fd1ef0 con 0x561d340c2ec0 2024-04-25T10:04:14.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd850ff9640 10 client.10146 handle_command_reply: tid=0 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 client.10146 shutdown 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 msgr2=0x561d34084e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d34084e50 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001f70 tx=0x7fd85c001510 comp rx=0 tx=0).stop 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 shutdown_connections 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340c2ec0 0x561d340c52b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d340b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d3408f4e0 0x561d340b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd850ff9640 0 client.10146 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 --2- 172.21.15.148:0/2750789440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408c200 0x561d34084e50 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 >> 172.21.15.148:0/2750789440 conn(0x561d34029270 msgr2=0x561d3405cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 shutdown_connections 2024-04-25T10:04:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 1 -- 172.21.15.148:0/2750789440 wait complete. 2024-04-25T10:04:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.263+0000 7fd876c2a740 20 client.10146 trim_cache size 0 max 16384 2024-04-25T10:04:14.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.278+0000 7fd876c2a740 1 Processor -- start 2024-04-25T10:04:14.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.278+0000 7fd876c2a740 1 -- start start 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d3408f4e0 0x561d3405d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d340b7790 0x561d3408e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 0x561d3408eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd876c2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c200c10 con 0x561d34085c60 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c1ff280 con 0x561d3408f4e0 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c1ff820 con 0x561d340b7790 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 0x561d3408eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 0x561d3408eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56882/0 (socket says 172.21.15.148:56882) 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 -- 172.21.15.148:0/3271520646 learned_addr learned my addr 172.21.15.148:0/3271520646 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8713ee640 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d340b7790 0x561d3408e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd871bef640 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d3408f4e0 0x561d3405d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 -- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d340b7790 msgr2=0x561d3408e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d340b7790 0x561d3408e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 -- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d3408f4e0 msgr2=0x561d3405d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d3408f4e0 0x561d3405d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 -- 172.21.15.148:0/3271520646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d340851c0 con 0x561d34085c60 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd8723f0640 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 0x561d3408eea0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fd86405ffd0 tx=0x7fd86405e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dee8a5177e6814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd852ffd640 1 -- 172.21.15.148:0/3271520646 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86405ce60 con 0x561d34085c60 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd852ffd640 1 -- 172.21.15.148:0/3271520646 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864060500 con 0x561d34085c60 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.279+0000 7fd852ffd640 1 -- 172.21.15.148:0/3271520646 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864010d90 con 0x561d34085c60 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 -- 172.21.15.148:0/3271520646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 msgr2=0x561d3408eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 0x561d3408eea0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fd86405ffd0 tx=0x7fd86405e040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 -- 172.21.15.148:0/3271520646 shutdown_connections 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d34085c60 0x561d3408eea0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d340b7790 0x561d3408e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 --2- 172.21.15.148:0/3271520646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d3408f4e0 0x561d3405d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 -- 172.21.15.148:0/3271520646 >> 172.21.15.148:0/3271520646 conn(0x561d34029270 msgr2=0x561d340a7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 -- 172.21.15.148:0/3271520646 shutdown_connections 2024-04-25T10:04:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 -- 172.21.15.148:0/3271520646 wait complete. 2024-04-25T10:04:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.280+0000 7fd876c2a740 1 -- start start 2024-04-25T10:04:14.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 Processor -- start 2024-04-25T10:04:14.292 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:04:14.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d34085c60 0x561d34111c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 0x561d3408e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d3408ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d340a9910 con 0x561d3408f4e0 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d34061760 con 0x561d340b7790 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.281+0000 7fd876c2a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d340a9f50 con 0x561d34085c60 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd876c2a740 10 client.0 _ll_register_callbacks cb 0x7fd85c049070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 0x561d3408e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 0x561d3408e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56892/0 (socket says 172.21.15.148:56892) 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 -- 172.21.15.148:0/1024080624 learned_addr learned my addr 172.21.15.148:0/1024080624 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd871bef640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d34085c60 0x561d34111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8723f0640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d3408ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 -- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d34085c60 msgr2=0x561d34111c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d34085c60 0x561d34111c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 -- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 msgr2=0x561d3408ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d3408ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 -- 172.21.15.148:0/1024080624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d3408c950 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8723f0640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d3408ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd8713ee640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 0x561d3408e820 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0cc540 tx=0x7fd86001ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10de165685a18ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd850ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd850ff9640 1 -- 172.21.15.148:0/1024080624 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86001d350 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd850ff9640 1 -- 172.21.15.148:0/1024080624 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd86001d4f0 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd864060500 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd850ff9640 1 -- 172.21.15.148:0/1024080624 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd86002c050 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd876c2a740 10 client.10152 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x561d33c98e20 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.282+0000 7fd850ff9640 1 -- 172.21.15.148:0/1024080624 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864060500 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd850ff9640 1 -- 172.21.15.148:0/1024080624 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fd86002f020 con 0x561d3408f4e0 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 10 client.10152 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 10 client.10152 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 20 client.10152 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340b5ee0 0x561d3409cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 4 client.10152 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x561d33fcc240 con 0x561d340b5ee0 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd871bef640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340b5ee0 0x561d3409cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd850ff9640 10 client.10152 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd871bef640 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340b5ee0 0x561d3409cb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=6d768848cc0278a1 server_cookie=cd2ae1bd3fec5a92 in_seq=0 out_seq=0 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd850ff9640 1 -- 172.21.15.148:0/1024080624 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x561d33fcc240 con 0x561d340b5ee0 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd850ff9640 10 client.10152 handle_command_reply: tid=0 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 client.10152 shutdown 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 msgr2=0x561d3408e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 0x561d3408e820 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0cc540 tx=0x7fd86001ded0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 shutdown_connections 2024-04-25T10:04:14.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x561d340b5ee0 0x561d3409cb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d340b7790 0x561d3408ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d3408f4e0 0x561d3408e820 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 --2- 172.21.15.148:0/1024080624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561d34085c60 0x561d34111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd850ff9640 0 client.10152 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 >> 172.21.15.148:0/1024080624 conn(0x561d34029270 msgr2=0x561d34088ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 shutdown_connections 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.283+0000 7fd876c2a740 1 -- 172.21.15.148:0/1024080624 wait complete. 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.284+0000 7fd876c2a740 20 client.10152 trim_cache size 0 max 16384 2024-04-25T10:04:14.296 INFO:teuthology.orchestra.run.smithi148.stderr:Error EINVAL: MDS is not active 2024-04-25T10:04:14.296 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2024-04-25T10:04:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.390+0000 7f742fdf4640 1 Processor -- start 2024-04-25T10:04:14.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.390+0000 7f742fdf4640 1 -- start start 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742fdf4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74281452a0 0x7f742806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742fdf4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74280cccc0 con 0x7f7428144d50 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742fdf4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74280cc140 con 0x7f74281446d0 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742fdf4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74280b2b20 con 0x7f74281452a0 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742d38f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7428144d50 0x7f742806d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42062/0 (socket says 172.21.15.148:42062) 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742d38f640 1 -- 172.21.15.148:0/3466791950 learned_addr learned my addr 172.21.15.148:0/3466791950 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742e391640 1 --2- 172.21.15.148:0/3466791950 >> v2:172.21.15.188:3300/0 conn(0x7f74281452a0 0x7f742806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742cb8e640 1 -- 172.21.15.148:0/3466791950 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676621502 0 0) 0x7f74280cccc0 con 0x7f7428144d50 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.391+0000 7f742cb8e640 1 -- 172.21.15.148:0/3466791950 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7410003520 con 0x7f7428144d50 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.392+0000 7f742cb8e640 1 -- 172.21.15.148:0/3466791950 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3303755131 0 0) 0x7f7410003520 con 0x7f7428144d50 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.392+0000 7f742e391640 1 -- 172.21.15.148:0/3466791950 >> v1:172.21.15.188:6789/0 conn(0x7f74281446d0 legacy=0x7f74280b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.392+0000 7f742e391640 1 -- 172.21.15.148:0/3466791950 >> v1:172.21.15.148:6789/0 conn(0x7f7428144d50 legacy=0x7f742806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.392+0000 7f742e391640 1 -- 172.21.15.148:0/3466791950 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280cc420 con 0x7f74281452a0 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.392+0000 7f742e391640 1 --2- 172.21.15.148:0/3466791950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74281452a0 0x7f742806dfa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f74200097a0 tx=0x7f742002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ef691f67ffd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742cb8e640 1 -- 172.21.15.148:0/3466791950 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420037070 con 0x7f74281452a0 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742cb8e640 1 -- 172.21.15.148:0/3466791950 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742002ac50 con 0x7f74281452a0 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742cb8e640 1 -- 172.21.15.148:0/3466791950 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c001670 con 0x7f74281452a0 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 -- 172.21.15.148:0/3466791950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74281452a0 msgr2=0x7f742806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 --2- 172.21.15.148:0/3466791950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74281452a0 0x7f742806dfa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f74200097a0 tx=0x7f742002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 -- 172.21.15.148:0/3466791950 shutdown_connections 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 --2- 172.21.15.148:0/3466791950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74281452a0 0x7f742806dfa0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 -- 172.21.15.148:0/3466791950 >> 172.21.15.148:0/3466791950 conn(0x7f742806abe0 msgr2=0x7f742806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 -- 172.21.15.148:0/3466791950 shutdown_connections 2024-04-25T10:04:14.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.393+0000 7f742fdf4640 1 -- 172.21.15.148:0/3466791950 wait complete. 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.394+0000 7f742fdf4640 1 Processor -- start 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.394+0000 7f742fdf4640 1 -- start start 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742fdf4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74281446d0 0x7f74280b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742fdf4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7428144d50 0x7f74280afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742fdf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74281452a0 0x7f74280c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742fdf4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74280cc9e0 con 0x7f74281452a0 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742fdf4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742806d400 con 0x7f7428144d50 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742fdf4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7428144ab0 con 0x7f74281446d0 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74281452a0 0x7f74280c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74281452a0 0x7f74280c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56908/0 (socket says 172.21.15.148:56908) 2024-04-25T10:04:14.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 -- 172.21.15.148:0/3483452955 learned_addr learned my addr 172.21.15.148:0/3483452955 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742d38f640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7428144d50 0x7f74280afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 -- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74281446d0 msgr2=0x7f74280b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742db90640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74281446d0 0x7f74280b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74281446d0 0x7f74280b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 -- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7428144d50 msgr2=0x7f74280afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7428144d50 0x7f74280afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742e391640 1 -- 172.21.15.148:0/3483452955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74280b04a0 con 0x7f74281452a0 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.395+0000 7f742d38f640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7428144d50 0x7f74280afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.396+0000 7f742e391640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74281452a0 0x7f74280c68b0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f7420035040 tx=0x7f74200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.396+0000 7f74267fc640 1 -- 172.21.15.148:0/3483452955 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420041020 con 0x7f74281452a0 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.396+0000 7f74267fc640 1 -- 172.21.15.148:0/3483452955 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74200044c0 con 0x7f74281452a0 2024-04-25T10:04:14.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.396+0000 7f742fdf4640 1 -- 172.21.15.148:0/3483452955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74280b0780 con 0x7f74281452a0 2024-04-25T10:04:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.396+0000 7f74267fc640 1 -- 172.21.15.148:0/3483452955 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420037040 con 0x7f74281452a0 2024-04-25T10:04:14.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.396+0000 7f742fdf4640 1 -- 172.21.15.148:0/3483452955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74280b0d40 con 0x7f74281452a0 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.397+0000 7f74267fc640 1 -- 172.21.15.148:0/3483452955 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7420044020 con 0x7f74281452a0 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.398+0000 7f74267fc640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f741406b4d0 0x7f741406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.398+0000 7f74267fc640 1 -- 172.21.15.148:0/3483452955 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f74200aa7a0 con 0x7f74281452a0 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.398+0000 7f742db90640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f741406b4d0 0x7f741406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.398+0000 7f742db90640 1 --2- 172.21.15.148:0/3483452955 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f741406b4d0 0x7f741406d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7418007fd0 tx=0x7f7418009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.399+0000 7f7432bcb740 1 Processor -- start 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.399+0000 7f7432bcb740 1 -- start start 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 0x556b4a61ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a647760 0x556b4a647be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f7432bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a57fb90 con 0x556b4a61cca0 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a528e40 con 0x556b4a659370 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b4a57bc30 con 0x556b4a647760 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 0x556b4a61ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 0x556b4a61ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56912/0 (socket says 172.21.15.148:56912) 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 -- 172.21.15.148:0/3803080639 learned_addr learned my addr 172.21.15.148:0/3803080639 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 -- 172.21.15.148:0/3803080639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a647760 msgr2=0x556b4a647be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a647760 0x556b4a647be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 -- 172.21.15.148:0/3803080639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 msgr2=0x556b4a61d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 -- 172.21.15.148:0/3803080639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b4a35e7d0 con 0x556b4a61cca0 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f742db90640 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 0x556b4a61ee00 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f7418003e50 tx=0x7f7418010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970e5dbb04202f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f740f7fe640 1 -- 172.21.15.148:0/3803080639 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418019030 con 0x556b4a61cca0 2024-04-25T10:04:14.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f740f7fe640 1 -- 172.21.15.148:0/3803080639 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74180053d0 con 0x556b4a61cca0 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.400+0000 7f740f7fe640 1 -- 172.21.15.148:0/3803080639 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180057a0 con 0x556b4a61cca0 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 -- 172.21.15.148:0/3803080639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 msgr2=0x556b4a61ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 0x556b4a61ee00 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f7418003e50 tx=0x7f7418010000 comp rx=0 tx=0).stop 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 -- 172.21.15.148:0/3803080639 shutdown_connections 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a647760 0x556b4a647be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3803080639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a61cca0 0x556b4a61ee00 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 -- 172.21.15.148:0/3803080639 >> 172.21.15.148:0/3803080639 conn(0x556b4a5e4870 msgr2=0x556b4a662760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 -- 172.21.15.148:0/3803080639 shutdown_connections 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.401+0000 7f7432bcb740 1 -- 172.21.15.148:0/3803080639 wait complete. 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.402+0000 7f7432bcb740 1 -- start start 2024-04-25T10:04:14.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.402+0000 7f7432bcb740 1 Processor -- start 2024-04-25T10:04:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a637000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a676690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a644500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a53c3e0 con 0x556b4a647760 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a6452b0 con 0x556b4a659370 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b4a665060 con 0x556b4a61cca0 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 10 client.0 _ll_register_callbacks cb 0x556b4a53bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a676690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a676690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56928/0 (socket says 172.21.15.148:56928) 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 -- 172.21.15.148:0/2788447139 learned_addr learned my addr 172.21.15.148:0/2788447139 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742db90640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a637000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742e391640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a644500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 -- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 msgr2=0x556b4a637000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a637000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 -- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 msgr2=0x556b4a644500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a644500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 -- 172.21.15.148:0/2788447139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b4a2540f0 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742e391640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a644500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f742d38f640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a676690 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f741c007f60 tx=0x7f741c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6081ed20e1e7d821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f740cff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f740cff9640 1 -- 172.21.15.148:0/2788447139 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c018060 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f740cff9640 1 -- 172.21.15.148:0/2788447139 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f741c005ad0 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74180053d0 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f740cff9640 1 -- 172.21.15.148:0/2788447139 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f741c01e050 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 10 client.10164 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.403+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x556b4a3bef60 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f740cff9640 1 -- 172.21.15.148:0/2788447139 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74180053d0 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f740cff9640 1 -- 172.21.15.148:0/2788447139 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f741c021020 con 0x556b4a647760 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f7432bcb740 10 client.10164 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f7432bcb740 10 client.10164 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f7432bcb740 20 client.10164 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f7432bcb740 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a663d20 0x556b4a664100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f7432bcb740 4 client.10164 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b4a58d0d0 con 0x556b4a663d20 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f742db90640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a663d20 0x556b4a664100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f740cff9640 10 client.10164 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.404+0000 7f742db90640 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a663d20 0x556b4a664100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=6897b12fdb8639c2 server_cookie=75bc471d60d2680a in_seq=0 out_seq=0 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f740cff9640 1 -- 172.21.15.148:0/2788447139 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b4a58d0d0 con 0x556b4a663d20 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f740cff9640 10 client.10164 handle_command_reply: tid=0 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 client.10164 shutdown 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 msgr2=0x556b4a676690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a676690 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f741c007f60 tx=0x7f741c007f90 comp rx=0 tx=0).stop 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 shutdown_connections 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a663d20 0x556b4a664100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a644500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a676690 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 --2- 172.21.15.148:0/2788447139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a637000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f740cff9640 0 client.10164 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 >> 172.21.15.148:0/2788447139 conn(0x556b4a5e4870 msgr2=0x556b4a6429d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 shutdown_connections 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 1 -- 172.21.15.148:0/2788447139 wait complete. 2024-04-25T10:04:14.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.405+0000 7f7432bcb740 20 client.10164 trim_cache size 0 max 16384 2024-04-25T10:04:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.420+0000 7f7432bcb740 1 Processor -- start 2024-04-25T10:04:14.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.420+0000 7f7432bcb740 1 -- start start 2024-04-25T10:04:14.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a644370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a641940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a641e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f7432bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74281fff00 con 0x556b4a647760 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7428200500 con 0x556b4a659370 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74280c5ae0 con 0x556b4a61cca0 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a641940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a641940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56938/0 (socket says 172.21.15.148:56938) 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 -- 172.21.15.148:0/3433131323 learned_addr learned my addr 172.21.15.148:0/3433131323 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742e391640 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a641e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742db90640 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a644370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 -- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 msgr2=0x556b4a644370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a644370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 -- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 msgr2=0x556b4a641e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a641e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 -- 172.21.15.148:0/3433131323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b4a22d1b0 con 0x556b4a647760 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f742d38f640 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a641940 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f741c001990 tx=0x7f741c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164a842e932edcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f740effd640 1 -- 172.21.15.148:0/3433131323 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c005ad0 con 0x556b4a647760 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f740effd640 1 -- 172.21.15.148:0/3433131323 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c026070 con 0x556b4a647760 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.421+0000 7f740effd640 1 -- 172.21.15.148:0/3433131323 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c007360 con 0x556b4a647760 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 -- 172.21.15.148:0/3433131323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 msgr2=0x556b4a641940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a641940 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f741c001990 tx=0x7f741c00b000 comp rx=0 tx=0).stop 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 -- 172.21.15.148:0/3433131323 shutdown_connections 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a641e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a641940 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 --2- 172.21.15.148:0/3433131323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a644370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 -- 172.21.15.148:0/3433131323 >> 172.21.15.148:0/3433131323 conn(0x556b4a5e4870 msgr2=0x556b4a685230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 -- 172.21.15.148:0/3433131323 shutdown_connections 2024-04-25T10:04:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 -- 172.21.15.148:0/3433131323 wait complete. 2024-04-25T10:04:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 -- start start 2024-04-25T10:04:14.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.422+0000 7f7432bcb740 1 Processor -- start 2024-04-25T10:04:14.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:04:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a685740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a6355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a676bd0 con 0x556b4a647760 2024-04-25T10:04:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74281ff140 con 0x556b4a659370 2024-04-25T10:04:14.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b4a647e60 con 0x556b4a61cca0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f7432bcb740 10 client.0 _ll_register_callbacks cb 0x7f741c001b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f742d38f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a6355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f742d38f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a6355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56950/0 (socket says 172.21.15.148:56950) 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.423+0000 7f742d38f640 1 -- 172.21.15.148:0/304120500 learned_addr learned my addr 172.21.15.148:0/304120500 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742e391640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742db90640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a685740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742d38f640 1 -- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 msgr2=0x556b4a685740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742d38f640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a685740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742d38f640 1 -- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 msgr2=0x556b4a61b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742d38f640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742d38f640 1 -- 172.21.15.148:0/304120500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b4a635b20 con 0x556b4a647760 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f742d38f640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a6355e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f741c00dbf0 tx=0x7f741c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aa6cb25ae5878e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f740cff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:14.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f740cff9640 1 -- 172.21.15.148:0/304120500 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c007530 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f740cff9640 1 -- 172.21.15.148:0/304120500 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f741c0076d0 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f741c026070 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f740cff9640 1 -- 172.21.15.148:0/304120500 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f741c03c050 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 10 client.10170 fetch_fsmap learned FSMap version 87 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x556b4a635d30 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f740cff9640 1 -- 172.21.15.148:0/304120500 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c026070 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f740cff9640 1 -- 172.21.15.148:0/304120500 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 87) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f741c03f020 con 0x556b4a647760 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 10 client.10170 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 10 client.10170 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 20 client.10170 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a641dc0 0x556b4a65b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 4 client.10170 mds_command: new command op to 9855 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:04:14.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.424+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 --> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x556b4a587420 con 0x556b4a641dc0 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f742db90640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a641dc0 0x556b4a65b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f740cff9640 10 client.10170 ms_handle_connect on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f742db90640 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a641dc0 0x556b4a65b760 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9855 client_cookie=8d68a65fed659458 server_cookie=3d37f1a152d23f47 in_seq=0 out_seq=0 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f740cff9640 1 -- 172.21.15.148:0/304120500 <== mds.9855 v2:172.21.15.148:6834/1405903723 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x556b4a587420 con 0x556b4a641dc0 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f740cff9640 10 client.10170 handle_command_reply: tid=0 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 client.10170 shutdown 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 msgr2=0x556b4a6355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a6355e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f741c00dbf0 tx=0x7f741c0057f0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 shutdown_connections 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:6834/1405903723,v1:172.21.15.148:6835/1405903723] conn(0x556b4a641dc0 0x556b4a65b760 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4a659370 0x556b4a61b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f740cff9640 0 client.10170 ms_handle_reset on v2:172.21.15.148:6834/1405903723 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556b4a647760 0x556b4a6355e0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 --2- 172.21.15.148:0/304120500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556b4a61cca0 0x556b4a685740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 >> 172.21.15.148:0/304120500 conn(0x556b4a5e4870 msgr2=0x556b4a666500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 shutdown_connections 2024-04-25T10:04:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 1 -- 172.21.15.148:0/304120500 wait complete. 2024-04-25T10:04:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.425+0000 7f7432bcb740 20 client.10170 trim_cache size 0 max 16384 2024-04-25T10:04:14.438 INFO:teuthology.orchestra.run.smithi148.stderr:Error EINVAL: MDS is not active 2024-04-25T10:04:14.438 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:14.438 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:14.474 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:14.521 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:14.521 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:14.521 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:14.537 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:14.537 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:14.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:14.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.724+0000 7f2e1ba5d640 1 Processor -- start 2024-04-25T10:04:14.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.724+0000 7f2e1ba5d640 1 -- start start 2024-04-25T10:04:14.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e1ba5d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e140d2e90 0x7f2e140d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e1ba5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e140b9940 con 0x7f2e1415dd30 2024-04-25T10:04:14.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e1ba5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e140d1df0 con 0x7f2e140d0900 2024-04-25T10:04:14.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e1ba5d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e140d0d40 con 0x7f2e140d2e90 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e18ff8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e140d2e90 0x7f2e140d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e18ff8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2e140d2e90 0x7f2e140d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56956/0 (socket says 172.21.15.148:56956) 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1503503190 learned_addr learned my addr 172.21.15.148:0/1503503190 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e197f9640 1 --1- 172.21.15.148:0/1503503190 >> v1:172.21.15.148:6789/0 conn(0x7f2e1415dd30 0x7f2e140b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42064/0 (socket says 172.21.15.148:42064) 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1503503190 >> v1:172.21.15.188:6789/0 conn(0x7f2e140d0900 legacy=0x7f2e140b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1503503190 >> v1:172.21.15.148:6789/0 conn(0x7f2e1415dd30 legacy=0x7f2e140b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.725+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1503503190 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140d20d0 con 0x7f2e140d2e90 2024-04-25T10:04:14.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.726+0000 7f2e18ff8640 1 --2- 172.21.15.148:0/1503503190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140d03c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009850 tx=0x7f2e0402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73934ec9c9e7e330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.726+0000 7f2e0bfff640 1 -- 172.21.15.148:0/1503503190 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04004030 con 0x7f2e140d2e90 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.726+0000 7f2e0bfff640 1 -- 172.21.15.148:0/1503503190 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e040041d0 con 0x7f2e140d2e90 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.726+0000 7f2e0bfff640 1 -- 172.21.15.148:0/1503503190 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04003610 con 0x7f2e140d2e90 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1503503190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 msgr2=0x7f2e140d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1503503190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140d03c0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009850 tx=0x7f2e0402a160 comp rx=0 tx=0).stop 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1503503190 shutdown_connections 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1503503190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140d03c0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1503503190 >> 172.21.15.148:0/1503503190 conn(0x7f2e14061c30 msgr2=0x7f2e1406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1503503190 shutdown_connections 2024-04-25T10:04:14.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.727+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1503503190 wait complete. 2024-04-25T10:04:14.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.728+0000 7f2e1ba5d640 1 Processor -- start 2024-04-25T10:04:14.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.728+0000 7f2e1ba5d640 1 -- start start 2024-04-25T10:04:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.728+0000 7f2e1ba5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e140d0900 0x7f2e14156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.728+0000 7f2e1ba5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e1ba5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e1415dd30 0x7f2e140b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e1ba5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140d2970 con 0x7f2e140d2e90 2024-04-25T10:04:14.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e1ba5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140d2690 con 0x7f2e140d0900 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e1ba5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e140d1a10 con 0x7f2e1415dd30 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56964/0 (socket says 172.21.15.148:56964) 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1648361917 learned_addr learned my addr 172.21.15.148:0/1648361917 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1648361917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e1415dd30 msgr2=0x7f2e140b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e1415dd30 0x7f2e140b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1648361917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e140d0900 msgr2=0x7f2e14156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e140d0900 0x7f2e14156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 -- 172.21.15.148:0/1648361917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140b61f0 con 0x7f2e140d2e90 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e18ff8640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140b5aa0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f2e04034040 tx=0x7f2e040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04041020 con 0x7f2e140d2e90 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e04003c50 con 0x7f2e140d2e90 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1415f4a0 con 0x7f2e140d2e90 2024-04-25T10:04:14.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04006040 con 0x7f2e140d2e90 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.729+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e140b42c0 con 0x7f2e140d2e90 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.730+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e14073990 con 0x7f2e140d2e90 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.731+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e04044020 con 0x7f2e140d2e90 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.731+0000 7f2e09ffb640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2df806b4d0 0x7f2df806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.731+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2e040ab150 con 0x7f2e140d2e90 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.731+0000 7f2e197f9640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2df806b4d0 0x7f2df806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.732+0000 7f2e197f9640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2df806b4d0 0x7f2df806d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2e10000f60 tx=0x7f2e10007a00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.736+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e0404b030 con 0x7f2e140d2e90 2024-04-25T10:04:14.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.912+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e1415fc10 con 0x7f2e140d2e90 2024-04-25T10:04:14.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.913+0000 7f2e09ffb640 1 -- 172.21.15.148:0/1648361917 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f2e04079df0 con 0x7f2e140d2e90 2024-04-25T10:04:14.919 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:14.919 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":147,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:05.942649+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.2b","mappings":[{"from":3,"to":0}]},{"pgid":"22.36","mappings":[{"from":3,"to":1}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2df806b4d0 msgr2=0x7f2df806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2df806b4d0 0x7f2df806d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2e10000f60 tx=0x7f2e10007a00 comp rx=0 tx=0).stop 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 msgr2=0x7f2e140b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140b5aa0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f2e04034040 tx=0x7f2e040046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 shutdown_connections 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2df806b4d0 0x7f2df806d990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e1415dd30 0x7f2e140b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e140d2e90 0x7f2e140b5aa0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 --2- 172.21.15.148:0/1648361917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e140d0900 0x7f2e14156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 >> 172.21.15.148:0/1648361917 conn(0x7f2e14061c30 msgr2=0x7f2e141e6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 shutdown_connections 2024-04-25T10:04:14.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:14.916+0000 7f2e1ba5d640 1 -- 172.21.15.148:0/1648361917 wait complete. 2024-04-25T10:04:14.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:48.480288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:49.199618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:14.932 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.028+0000 7f1179170640 1 Processor -- start 2024-04-25T10:04:15.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.028+0000 7f1179170640 1 -- start start 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f1179170640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1174070dc0 0x7f117406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f1179170640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f117415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f1179170640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11740d1770 con 0x7f11740711a0 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f1179170640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11740b0eb0 con 0x7f1174070dc0 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f1179170640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11741622b0 con 0x7f117415deb0 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f117415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f1172d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1174070dc0 0x7f117406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f117415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:45548/0 (socket says 172.21.15.148:45548) 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 -- 172.21.15.148:0/3526798075 learned_addr learned my addr 172.21.15.148:0/3526798075 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 -- 172.21.15.148:0/3526798075 >> v2:172.21.15.188:3300/0 conn(0x7f1174070dc0 msgr2=0x7f117406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 --2- 172.21.15.148:0/3526798075 >> v2:172.21.15.188:3300/0 conn(0x7f1174070dc0 0x7f117406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 -- 172.21.15.148:0/3526798075 >> v1:172.21.15.188:6789/0 conn(0x7f11740711a0 legacy=0x7f11740b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.029+0000 7f117359e640 1 -- 172.21.15.148:0/3526798075 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174162740 con 0x7f117415deb0 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.030+0000 7f117359e640 1 --2- 172.21.15.148:0/3526798075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f117415e340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1164009870 tx=0x7f116402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1efeeb1e6800f3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.030+0000 7f1171d9b640 1 -- 172.21.15.148:0/3526798075 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164037070 con 0x7f117415deb0 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.030+0000 7f1171d9b640 1 -- 172.21.15.148:0/3526798075 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164032440 con 0x7f117415deb0 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.030+0000 7f1171d9b640 1 -- 172.21.15.148:0/3526798075 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164032810 con 0x7f117415deb0 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 -- 172.21.15.148:0/3526798075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 msgr2=0x7f117415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 --2- 172.21.15.148:0/3526798075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f117415e340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1164009870 tx=0x7f116402a130 comp rx=0 tx=0).stop 2024-04-25T10:04:15.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 -- 172.21.15.148:0/3526798075 shutdown_connections 2024-04-25T10:04:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 --2- 172.21.15.148:0/3526798075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f117415e340 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 --2- 172.21.15.148:0/3526798075 >> v2:172.21.15.188:3300/0 conn(0x7f1174070dc0 0x7f117406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 -- 172.21.15.148:0/3526798075 >> 172.21.15.148:0/3526798075 conn(0x7f11740b3530 msgr2=0x7f11740b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 -- 172.21.15.148:0/3526798075 shutdown_connections 2024-04-25T10:04:15.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.031+0000 7f1179170640 1 -- 172.21.15.148:0/3526798075 wait complete. 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.032+0000 7f1179170640 1 Processor -- start 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.032+0000 7f1179170640 1 -- start start 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1179170640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 0x7f11740cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1179170640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11740711a0 0x7f1174154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1179170640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f11741f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1179170640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1174161c60 con 0x7f1174070dc0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1179170640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740b10f0 con 0x7f11740711a0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1179170640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1174161f40 con 0x7f117415deb0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1172d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 0x7f11740cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1172d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 0x7f11740cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56968/0 (socket says 172.21.15.148:56968) 2024-04-25T10:04:15.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f1172d9d640 1 -- 172.21.15.148:0/3195371284 learned_addr learned my addr 172.21.15.148:0/3195371284 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f117359e640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f11741f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.033+0000 7f117259c640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11740711a0 0x7f1174154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1172d9d640 1 -- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 msgr2=0x7f11741f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1172d9d640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f11741f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1172d9d640 1 -- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11740711a0 msgr2=0x7f1174154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1172d9d640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11740711a0 0x7f1174154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1172d9d640 1 -- 172.21.15.148:0/3195371284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740cbb00 con 0x7f1174070dc0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f117359e640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f11741f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1172d9d640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 0x7f11740cb5c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f116c00e9b0 tx=0x7f116c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c017070 con 0x7f1174070dc0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c00c9a0 con 0x7f1174070dc0 2024-04-25T10:04:15.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1174155360 con 0x7f1174070dc0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00cd90 con 0x7f1174070dc0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.034+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11740cc060 con 0x7f1174070dc0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.035+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1174073a00 con 0x7f1174070dc0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.035+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f116c01a070 con 0x7f1174070dc0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.036+0000 7f115b7fe640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f116006b4d0 0x7f116006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.036+0000 7f117259c640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f116006b4d0 0x7f116006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.036+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f116c08a3a0 con 0x7f1174070dc0 2024-04-25T10:04:15.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.036+0000 7f117259c640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f116006b4d0 0x7f116006d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f11740d2580 tx=0x7f1168009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.040+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11740cbb00 con 0x7f1174070dc0 2024-04-25T10:04:15.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.213+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11740b6510 con 0x7f1174070dc0 2024-04-25T10:04:15.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.213+0000 7f115b7fe640 1 -- 172.21.15.148:0/3195371284 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7f116c059040 con 0x7f1174070dc0 2024-04-25T10:04:15.219 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:04:15.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f116006b4d0 msgr2=0x7f116006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f116006b4d0 0x7f116006d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f11740d2580 tx=0x7f1168009040 comp rx=0 tx=0).stop 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 msgr2=0x7f11740cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 0x7f11740cb5c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f116c00e9b0 tx=0x7f116c00ee70 comp rx=0 tx=0).stop 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 shutdown_connections 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f116006b4d0 0x7f116006d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117415deb0 0x7f11741f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11740711a0 0x7f1174154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 --2- 172.21.15.148:0/3195371284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1174070dc0 0x7f11740cb5c0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 >> 172.21.15.148:0/3195371284 conn(0x7f11740b3530 msgr2=0x7f11740d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 shutdown_connections 2024-04-25T10:04:15.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.215+0000 7f1179170640 1 -- 172.21.15.148:0/3195371284 wait complete. 2024-04-25T10:04:15.231 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:04:15.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.326+0000 7feef365d640 1 Processor -- start 2024-04-25T10:04:15.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.327+0000 7feef365d640 1 -- start start 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.327+0000 7feef365d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feeec144d20 0x7feeec0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.327+0000 7feef365d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feeec1454b0 0x7feeec070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.327+0000 7feef365d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec0ceba0 con 0x7feeec071300 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.327+0000 7feef365d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0b4830 con 0x7feeec144d20 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.327+0000 7feef365d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec071730 con 0x7feeec1454b0 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feeec144d20 0x7feeec0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feeec144d20 0x7feeec0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56974/0 (socket says 172.21.15.148:56974) 2024-04-25T10:04:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 -- 172.21.15.148:0/2922109453 learned_addr learned my addr 172.21.15.148:0/2922109453 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef0bf8640 1 --2- 172.21.15.148:0/2922109453 >> v2:172.21.15.188:3300/0 conn(0x7feeec1454b0 0x7feeec070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 -- 172.21.15.148:0/2922109453 >> v2:172.21.15.188:3300/0 conn(0x7feeec1454b0 msgr2=0x7feeec070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 --2- 172.21.15.148:0/2922109453 >> v2:172.21.15.188:3300/0 conn(0x7feeec1454b0 0x7feeec070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 -- 172.21.15.148:0/2922109453 >> v1:172.21.15.188:6789/0 conn(0x7feeec071300 legacy=0x7feeec15f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.328+0000 7feef13f9640 1 -- 172.21.15.148:0/2922109453 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0cee80 con 0x7feeec144d20 2024-04-25T10:04:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef13f9640 1 --2- 172.21.15.148:0/2922109453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec144d20 0x7feeec0ce350 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7feeec145940 tx=0x7feee802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c992d649681ced00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feee3fff640 1 -- 172.21.15.148:0/2922109453 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8037070 con 0x7feeec144d20 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feee3fff640 1 -- 172.21.15.148:0/2922109453 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8032440 con 0x7feeec144d20 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feee3fff640 1 -- 172.21.15.148:0/2922109453 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8032810 con 0x7feeec144d20 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef365d640 1 -- 172.21.15.148:0/2922109453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec144d20 msgr2=0x7feeec0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef365d640 1 --2- 172.21.15.148:0/2922109453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec144d20 0x7feeec0ce350 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7feeec145940 tx=0x7feee802a270 comp rx=0 tx=0).stop 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef365d640 1 -- 172.21.15.148:0/2922109453 shutdown_connections 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef365d640 1 --2- 172.21.15.148:0/2922109453 >> v2:172.21.15.188:3300/0 conn(0x7feeec1454b0 0x7feeec070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef365d640 1 --2- 172.21.15.148:0/2922109453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec144d20 0x7feeec0ce350 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.329+0000 7feef365d640 1 -- 172.21.15.148:0/2922109453 >> 172.21.15.148:0/2922109453 conn(0x7feeec0b3530 msgr2=0x7feeec0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.330+0000 7feef365d640 1 -- 172.21.15.148:0/2922109453 shutdown_connections 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.330+0000 7feef365d640 1 -- 172.21.15.148:0/2922109453 wait complete. 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 Processor -- start 2024-04-25T10:04:15.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 -- start start 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 0x7feeec0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec144d20 0x7feeec0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec1454b0 0x7feeec0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0ce890 con 0x7feeec071300 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0cf440 con 0x7feeec1454b0 2024-04-25T10:04:15.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef365d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec145100 con 0x7feeec144d20 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef13f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 0x7feeec0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef13f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 0x7feeec0c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56982/0 (socket says 172.21.15.148:56982) 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.331+0000 7feef13f9640 1 -- 172.21.15.148:0/3031299362 learned_addr learned my addr 172.21.15.148:0/3031299362 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef0bf8640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec144d20 0x7feeec0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef1bfa640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec1454b0 0x7feeec0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef13f9640 1 -- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec144d20 msgr2=0x7feeec0ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef13f9640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec144d20 0x7feeec0ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef13f9640 1 -- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec1454b0 msgr2=0x7feeec0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef13f9640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec1454b0 0x7feeec0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef13f9640 1 -- 172.21.15.148:0/3031299362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec14f640 con 0x7feeec071300 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef0bf8640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec144d20 0x7feeec0ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef1bfa640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec1454b0 0x7feeec0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef13f9640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 0x7feeec0c9a50 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7feee802a2b0 tx=0x7feee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8037040 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8042070 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec14f8e0 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8030880 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.332+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec14fea0 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.333+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec073a00 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.334+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feee8030a20 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.334+0000 7feee1ffb640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.334+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7feee80aabf0 con 0x7feeec071300 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.334+0000 7feef0bf8640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.334+0000 7feef0bf8640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7feeec140e10 tx=0x7feedc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.338+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feee8079920 con 0x7feeec071300 2024-04-25T10:04:15.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.506+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feeec0b0f50 con 0x7feeec071300 2024-04-25T10:04:15.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.506+0000 7feee1ffb640 1 -- 172.21.15.148:0/3031299362 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+11 (secure 0 0 0) 0x7feee8079220 con 0x7feeec071300 2024-04-25T10:04:15.511 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:04:15.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 msgr2=0x7feed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7feeec140e10 tx=0x7feedc008040 comp rx=0 tx=0).stop 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 msgr2=0x7feeec0c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 0x7feeec0c9a50 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7feee802a2b0 tx=0x7feee80046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 shutdown_connections 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feed006b4d0 0x7feed006d990 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeec1454b0 0x7feeec0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeec144d20 0x7feeec0ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 --2- 172.21.15.148:0/3031299362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeec071300 0x7feeec0c9a50 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 >> 172.21.15.148:0/3031299362 conn(0x7feeec0b3530 msgr2=0x7feeec0c67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 shutdown_connections 2024-04-25T10:04:15.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.508+0000 7feef365d640 1 -- 172.21.15.148:0/3031299362 wait complete. 2024-04-25T10:04:15.524 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:04:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.619+0000 7f666f7f9640 1 Processor -- start 2024-04-25T10:04:15.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.619+0000 7f666f7f9640 1 -- start start 2024-04-25T10:04:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666f7f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f666806c330 0x7f66680b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666f7f9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680d1d50 con 0x7f666806c7e0 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666f7f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680d2020 con 0x7f6668158230 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666f7f9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6668158610 con 0x7f666806c330 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666cd94640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f666806c330 0x7f66680b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666dd96640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f666806c7e0 0x7f666806f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42068/0 (socket says 172.21.15.148:42068) 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666cd94640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f666806c330 0x7f66680b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56984/0 (socket says 172.21.15.148:56984) 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666dd96640 1 -- 172.21.15.148:0/1782985696 learned_addr learned my addr 172.21.15.148:0/1782985696 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666cd94640 1 -- 172.21.15.148:0/1782985696 >> v1:172.21.15.188:6789/0 conn(0x7f6668158230 legacy=0x7f66680d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666cd94640 1 -- 172.21.15.148:0/1782985696 >> v1:172.21.15.148:6789/0 conn(0x7f666806c7e0 legacy=0x7f666806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f665ffff640 1 -- 172.21.15.148:0/1782985696 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375312369 0 0) 0x7f66680d1d50 con 0x7f666806c7e0 2024-04-25T10:04:15.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.620+0000 7f666cd94640 1 -- 172.21.15.148:0/1782985696 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680d2300 con 0x7f666806c330 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.621+0000 7f666cd94640 1 --2- 172.21.15.148:0/1782985696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680b4e60 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f6658009790 tx=0x7f665802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae8f5d47d25791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.621+0000 7f665ffff640 1 -- 172.21.15.148:0/1782985696 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658037070 con 0x7f666806c330 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.621+0000 7f665ffff640 1 -- 172.21.15.148:0/1782985696 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658032440 con 0x7f666806c330 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.621+0000 7f665ffff640 1 -- 172.21.15.148:0/1782985696 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658032810 con 0x7f666806c330 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 -- 172.21.15.148:0/1782985696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 msgr2=0x7f66680b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 --2- 172.21.15.148:0/1782985696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680b4e60 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f6658009790 tx=0x7f665802a210 comp rx=0 tx=0).stop 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 -- 172.21.15.148:0/1782985696 shutdown_connections 2024-04-25T10:04:15.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 --2- 172.21.15.148:0/1782985696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680b4e60 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 -- 172.21.15.148:0/1782985696 >> 172.21.15.148:0/1782985696 conn(0x7f6668061600 msgr2=0x7f66680619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 -- 172.21.15.148:0/1782985696 shutdown_connections 2024-04-25T10:04:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.622+0000 7f666f7f9640 1 -- 172.21.15.148:0/1782985696 wait complete. 2024-04-25T10:04:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.623+0000 7f666f7f9640 1 Processor -- start 2024-04-25T10:04:15.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.623+0000 7f666f7f9640 1 -- start start 2024-04-25T10:04:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f666806c7e0 0x7f666814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6668158230 0x7f66680cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66680d25e0 con 0x7f666806c330 2024-04-25T10:04:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666806bea0 con 0x7f666806c7e0 2024-04-25T10:04:15.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666815e320 con 0x7f6668158230 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:56996/0 (socket says 172.21.15.148:56996) 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 -- 172.21.15.148:0/873446786 learned_addr learned my addr 172.21.15.148:0/873446786 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666dd96640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6668158230 0x7f66680cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666cd94640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f666806c7e0 0x7f666814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 -- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6668158230 msgr2=0x7f66680cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6668158230 0x7f66680cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 -- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f666806c7e0 msgr2=0x7f666814c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f666806c7e0 0x7f666814c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 -- 172.21.15.148:0/873446786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680cc720 con 0x7f666806c330 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666d595640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680caa60 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f666400c480 tx=0x7f666400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640052d0 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6664005470 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664012870 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.624+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666814d900 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.625+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66680cb420 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.626+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66680b0e10 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.626+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6664012a10 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.626+0000 7f665dffb640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f664c06b4d0 0x7f664c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.627+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6664089fe0 con 0x7f666806c330 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.627+0000 7f666cd94640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f664c06b4d0 0x7f664c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.627+0000 7f666cd94640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f664c06b4d0 0x7f664c06d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6658034000 tx=0x7f665802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.631+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66680b0e10 con 0x7f666806c330 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.802+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6668073a00 con 0x7f666806c330 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.802+0000 7f665dffb640 1 -- 172.21.15.148:0/873446786 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66680cc720 con 0x7f666806c330 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:04:15.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f664c06b4d0 msgr2=0x7f664c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f664c06b4d0 0x7f664c06d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6658034000 tx=0x7f665802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:15.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 msgr2=0x7f66680caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680caa60 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f666400c480 tx=0x7f666400c940 comp rx=0 tx=0).stop 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 shutdown_connections 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f664c06b4d0 0x7f664c06d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6668158230 0x7f66680cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f666806c7e0 0x7f666814c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 --2- 172.21.15.148:0/873446786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f666806c330 0x7f66680caa60 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 >> 172.21.15.148:0/873446786 conn(0x7f6668061600 msgr2=0x7f66680b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 shutdown_connections 2024-04-25T10:04:15.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.804+0000 7f666f7f9640 1 -- 172.21.15.148:0/873446786 wait complete. 2024-04-25T10:04:15.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:04:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.914+0000 7fdd9c313640 1 Processor -- start 2024-04-25T10:04:15.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.914+0000 7fdd9c313640 1 -- start start 2024-04-25T10:04:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9c313640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd940cfa40 0x7fdd940cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9c313640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940c75e0 con 0x7fdd9415bb50 2024-04-25T10:04:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9c313640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940c78c0 con 0x7fdd940cf600 2024-04-25T10:04:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9c313640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd940c8680 con 0x7fdd940cfa40 2024-04-25T10:04:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9a0af640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fdd9415bb50 0x7fdd940c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42076/0 (socket says 172.21.15.148:42076) 2024-04-25T10:04:15.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9a8b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdd940cfa40 0x7fdd940cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.915+0000 7fdd9a0af640 1 -- 172.21.15.148:0/2627497178 learned_addr learned my addr 172.21.15.148:0/2627497178 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd9a8b0640 1 -- 172.21.15.148:0/2627497178 >> v1:172.21.15.188:6789/0 conn(0x7fdd940cf600 legacy=0x7fdd940cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd9a8b0640 1 -- 172.21.15.148:0/2627497178 >> v1:172.21.15.148:6789/0 conn(0x7fdd9415bb50 legacy=0x7fdd940c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd9a8b0640 1 -- 172.21.15.148:0/2627497178 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940c7ba0 con 0x7fdd940cfa40 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd9a8b0640 1 --2- 172.21.15.148:0/2627497178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cfa40 0x7fdd940cc630 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0097a0 tx=0x7fdd8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9069440d9c6e8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd990ad640 1 -- 172.21.15.148:0/2627497178 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c037070 con 0x7fdd940cfa40 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd990ad640 1 -- 172.21.15.148:0/2627497178 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c032440 con 0x7fdd940cfa40 2024-04-25T10:04:15.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.916+0000 7fdd990ad640 1 -- 172.21.15.148:0/2627497178 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c032810 con 0x7fdd940cfa40 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 -- 172.21.15.148:0/2627497178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cfa40 msgr2=0x7fdd940cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 --2- 172.21.15.148:0/2627497178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cfa40 0x7fdd940cc630 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0097a0 tx=0x7fdd8c02a270 comp rx=0 tx=0).stop 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 -- 172.21.15.148:0/2627497178 shutdown_connections 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 --2- 172.21.15.148:0/2627497178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cfa40 0x7fdd940cc630 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 -- 172.21.15.148:0/2627497178 >> 172.21.15.148:0/2627497178 conn(0x7fdd9406f500 msgr2=0x7fdd9406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 -- 172.21.15.148:0/2627497178 shutdown_connections 2024-04-25T10:04:15.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.917+0000 7fdd9c313640 1 -- 172.21.15.148:0/2627497178 wait complete. 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.918+0000 7fdd9c313640 1 Processor -- start 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.918+0000 7fdd9c313640 1 -- start start 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9c313640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 0x7fdd9414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9c313640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd940cfa40 0x7fdd940b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9c313640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9415bb50 0x7fdd940b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9c313640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940c72a0 con 0x7fdd940cf600 2024-04-25T10:04:15.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9c313640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940c8160 con 0x7fdd940cfa40 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9c313640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940cfe90 con 0x7fdd9415bb50 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 0x7fdd9414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 0x7fdd9414f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57006/0 (socket says 172.21.15.148:57006) 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 -- 172.21.15.148:0/4232456283 learned_addr learned my addr 172.21.15.148:0/4232456283 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd998ae640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd940cfa40 0x7fdd940b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a8b0640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9415bb50 0x7fdd940b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 -- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9415bb50 msgr2=0x7fdd940b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9415bb50 0x7fdd940b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 -- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd940cfa40 msgr2=0x7fdd940b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd940cfa40 0x7fdd940b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 -- 172.21.15.148:0/4232456283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940b6260 con 0x7fdd940cf600 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.919+0000 7fdd9a0af640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 0x7fdd9414f3b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ca70 tx=0x7fdd8400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.920+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84017070 con 0x7fdd940cf600 2024-04-25T10:04:15.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.920+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400abd0 con 0x7fdd940cf600 2024-04-25T10:04:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.920+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd940b6540 con 0x7fdd940cf600 2024-04-25T10:04:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.920+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84012400 con 0x7fdd940cf600 2024-04-25T10:04:15.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.920+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd940b6b00 con 0x7fdd940cf600 2024-04-25T10:04:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.921+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94073a00 con 0x7fdd940cf600 2024-04-25T10:04:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.921+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd8401a070 con 0x7fdd940cf600 2024-04-25T10:04:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.921+0000 7fdd92ffd640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd8006b4d0 0x7fdd8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.921+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fdd8408a070 con 0x7fdd940cf600 2024-04-25T10:04:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.921+0000 7fdd998ae640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd8006b4d0 0x7fdd8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.922+0000 7fdd998ae640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd8006b4d0 0x7fdd8006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdd940b5490 tx=0x7fdd88009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:15.925+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd84058da0 con 0x7fdd940cf600 2024-04-25T10:04:16.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.110+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd94159140 con 0x7fdd940cf600 2024-04-25T10:04:16.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.110+0000 7fdd92ffd640 1 -- 172.21.15.148:0/4232456283 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v147) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd840586a0 con 0x7fdd940cf600 2024-04-25T10:04:16.115 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd8006b4d0 msgr2=0x7fdd8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd8006b4d0 0x7fdd8006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdd940b5490 tx=0x7fdd88009040 comp rx=0 tx=0).stop 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 msgr2=0x7fdd9414f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 0x7fdd9414f3b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ca70 tx=0x7fdd8400cf30 comp rx=0 tx=0).stop 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 shutdown_connections 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdd8006b4d0 0x7fdd8006d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd9415bb50 0x7fdd940b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd940cfa40 0x7fdd940b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 --2- 172.21.15.148:0/4232456283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd940cf600 0x7fdd9414f3b0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 >> 172.21.15.148:0/4232456283 conn(0x7fdd9406f500 msgr2=0x7fdd940d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 shutdown_connections 2024-04-25T10:04:16.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.112+0000 7fdd9c313640 1 -- 172.21.15.148:0/4232456283 wait complete. 2024-04-25T10:04:16.128 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:04:16.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:16.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:16.732 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:16.732 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:04:16.748 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:04:16.748 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:16.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:16.765 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:16.765 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:16.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:16.803 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:04:16.803 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:16.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:16.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:16.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.934+0000 7fde574fc640 1 Processor -- start 2024-04-25T10:04:16.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.935+0000 7fde574fc640 1 -- start start 2024-04-25T10:04:16.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.935+0000 7fde574fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde5015dc90 0x7fde501625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.935+0000 7fde574fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 0x7fde500d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.935+0000 7fde574fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500cce50 con 0x7fde500cc8e0 2024-04-25T10:04:16.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.935+0000 7fde574fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500d21c0 con 0x7fde5015dc90 2024-04-25T10:04:16.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.935+0000 7fde574fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde50161af0 con 0x7fde500d1810 2024-04-25T10:04:16.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55a99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 0x7fde500d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.941 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde5015dc90 0x7fde501625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde5015dc90 0x7fde501625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:37112/0 (socket says 172.21.15.148:37112) 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55a99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 0x7fde500d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:45582/0 (socket says 172.21.15.148:45582) 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 -- 172.21.15.148:0/2862853197 learned_addr learned my addr 172.21.15.148:0/2862853197 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde54a97640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fde500cc8e0 0x7fde5015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34414/0 (socket says 172.21.15.148:34414) 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 -- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 msgr2=0x7fde500d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 --2- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 0x7fde500d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 -- 172.21.15.148:0/2862853197 >> v1:172.21.15.188:6789/0 conn(0x7fde500cc8e0 legacy=0x7fde5015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55298640 1 -- 172.21.15.148:0/2862853197 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50161dd0 con 0x7fde5015dc90 2024-04-25T10:04:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.936+0000 7fde55a99640 1 --2- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 0x7fde500d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.937+0000 7fde55298640 1 --2- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde501625d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fde48009e50 tx=0x7fde4802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d23d699cb2cf04e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.937+0000 7fde47fff640 1 -- 172.21.15.148:0/2862853197 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48036070 con 0x7fde5015dc90 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.937+0000 7fde47fff640 1 -- 172.21.15.148:0/2862853197 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde48031600 con 0x7fde5015dc90 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.937+0000 7fde47fff640 1 -- 172.21.15.148:0/2862853197 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde480319d0 con 0x7fde5015dc90 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 -- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 msgr2=0x7fde501625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 --2- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde501625d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fde48009e50 tx=0x7fde4802a680 comp rx=0 tx=0).stop 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 -- 172.21.15.148:0/2862853197 shutdown_connections 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 --2- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500d1810 0x7fde500d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 --2- 172.21.15.148:0/2862853197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde501625d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 -- 172.21.15.148:0/2862853197 >> 172.21.15.148:0/2862853197 conn(0x7fde5006f590 msgr2=0x7fde5006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 -- 172.21.15.148:0/2862853197 shutdown_connections 2024-04-25T10:04:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.938+0000 7fde574fc640 1 -- 172.21.15.148:0/2862853197 wait complete. 2024-04-25T10:04:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.939+0000 7fde574fc640 1 Processor -- start 2024-04-25T10:04:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.939+0000 7fde574fc640 1 -- start start 2024-04-25T10:04:16.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde574fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500cc8e0 0x7fde500c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde574fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 0x7fde500c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde574fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde50148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde574fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015e070 con 0x7fde500d1810 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde574fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500cd130 con 0x7fde5015dc90 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde574fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015e1f0 con 0x7fde500cc8e0 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 0x7fde500c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 0x7fde500c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57018/0 (socket says 172.21.15.148:57018) 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 -- 172.21.15.148:0/58043647 learned_addr learned my addr 172.21.15.148:0/58043647 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde55298640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500cc8e0 0x7fde500c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde55a99640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde50148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 -- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500cc8e0 msgr2=0x7fde500c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500cc8e0 0x7fde500c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 -- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 msgr2=0x7fde50148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde50148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 -- 172.21.15.148:0/58043647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500c6cc0 con 0x7fde500d1810 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde54a97640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 0x7fde500c7d60 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fde4000c3c0 tx=0x7fde4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.940+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde400074e0 con 0x7fde500d1810 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.941+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40007680 con 0x7fde500d1810 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.941+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40004a00 con 0x7fde500d1810 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.941+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde500c6fa0 con 0x7fde500d1810 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.941+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde500c7560 con 0x7fde500d1810 2024-04-25T10:04:16.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.942+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50073990 con 0x7fde500d1810 2024-04-25T10:04:16.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.942+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde4001a020 con 0x7fde500d1810 2024-04-25T10:04:16.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.942+0000 7fde45ffb640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde3406b4d0 0x7fde3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.943+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fde40089940 con 0x7fde500d1810 2024-04-25T10:04:16.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.943+0000 7fde55298640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde3406b4d0 0x7fde3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.943+0000 7fde55298640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde3406b4d0 0x7fde3406d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fde48033000 tx=0x7fde48009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:16.947+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde40058020 con 0x7fde500d1810 2024-04-25T10:04:17.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.183+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde500c5130 con 0x7fde500d1810 2024-04-25T10:04:17.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.183+0000 7fde45ffb640 1 -- 172.21.15.148:0/58043647 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4961 (secure 0 0 0) 0x7fde4005c0c0 con 0x7fde500d1810 2024-04-25T10:04:17.188 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:17.189 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:04:12.963932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9855":{"gid":9855,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:04:17.189 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 87 2024-04-25T10:04:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde3406b4d0 msgr2=0x7fde3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde3406b4d0 0x7fde3406d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fde48033000 tx=0x7fde48009000 comp rx=0 tx=0).stop 2024-04-25T10:04:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 msgr2=0x7fde500c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 0x7fde500c7d60 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fde4000c3c0 tx=0x7fde4000c880 comp rx=0 tx=0).stop 2024-04-25T10:04:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 shutdown_connections 2024-04-25T10:04:17.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde3406b4d0 0x7fde3406d990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde5015dc90 0x7fde50148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde500d1810 0x7fde500c7d60 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 --2- 172.21.15.148:0/58043647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde500cc8e0 0x7fde500c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 >> 172.21.15.148:0/58043647 conn(0x7fde5006f590 msgr2=0x7fde500cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 shutdown_connections 2024-04-25T10:04:17.192 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.186+0000 7fde574fc640 1 -- 172.21.15.148:0/58043647 wait complete. 2024-04-25T10:04:17.202 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:17.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.294+0000 7f6a11122640 1 Processor -- start 2024-04-25T10:04:17.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.294+0000 7f6a11122640 1 -- start start 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a11122640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a0c15dc70 0x7f6a0c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a11122640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0c06c220 con 0x7f6a0c09a450 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a11122640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0c0d0080 con 0x7f6a0c06ce10 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a11122640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d0c90 con 0x7f6a0c15dc70 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a03fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6a0c09a450 0x7f6a0c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42078/0 (socket says 172.21.15.148:42078) 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a0ad9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a0c15dc70 0x7f6a0c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.295+0000 7f6a03fff640 1 -- 172.21.15.148:0/1095842312 learned_addr learned my addr 172.21.15.148:0/1095842312 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.296+0000 7f6a0a59c640 1 -- 172.21.15.148:0/1095842312 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498201319 0 0) 0x7f6a0c06c220 con 0x7f6a0c09a450 2024-04-25T10:04:17.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.296+0000 7f6a0a59c640 1 -- 172.21.15.148:0/1095842312 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f8003520 con 0x7f6a0c09a450 2024-04-25T10:04:17.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.296+0000 7f6a0ad9d640 1 -- 172.21.15.148:0/1095842312 >> v1:172.21.15.188:6789/0 conn(0x7f6a0c06ce10 legacy=0x7f6a0c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.296+0000 7f6a0ad9d640 1 -- 172.21.15.148:0/1095842312 >> v1:172.21.15.148:6789/0 conn(0x7f6a0c09a450 legacy=0x7f6a0c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.296+0000 7f6a0ad9d640 1 -- 172.21.15.148:0/1095842312 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c0d0360 con 0x7f6a0c15dc70 2024-04-25T10:04:17.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.296+0000 7f6a0ad9d640 1 --2- 172.21.15.148:0/1095842312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c0d1120 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0097a0 tx=0x7f69fc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b120ee150d5c53f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a0a59c640 1 -- 172.21.15.148:0/1095842312 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037070 con 0x7f6a0c15dc70 2024-04-25T10:04:17.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a0a59c640 1 -- 172.21.15.148:0/1095842312 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc032440 con 0x7f6a0c15dc70 2024-04-25T10:04:17.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a0a59c640 1 -- 172.21.15.148:0/1095842312 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc032810 con 0x7f6a0c15dc70 2024-04-25T10:04:17.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 -- 172.21.15.148:0/1095842312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c15dc70 msgr2=0x7f6a0c0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 --2- 172.21.15.148:0/1095842312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c0d1120 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0097a0 tx=0x7f69fc02a230 comp rx=0 tx=0).stop 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 -- 172.21.15.148:0/1095842312 shutdown_connections 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 --2- 172.21.15.148:0/1095842312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c0d1120 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 -- 172.21.15.148:0/1095842312 >> 172.21.15.148:0/1095842312 conn(0x7f6a0c0b3270 msgr2=0x7f6a0c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 -- 172.21.15.148:0/1095842312 shutdown_connections 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.297+0000 7f6a11122640 1 -- 172.21.15.148:0/1095842312 wait complete. 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.298+0000 7f6a11122640 1 Processor -- start 2024-04-25T10:04:17.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.298+0000 7f6a11122640 1 -- start start 2024-04-25T10:04:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a11122640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0c06ce10 0x7f6a0c09b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a11122640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 0x7f6a0c09e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a11122640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c1f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a11122640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c15e050 con 0x7f6a0c09a450 2024-04-25T10:04:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a11122640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d0920 con 0x7f6a0c15dc70 2024-04-25T10:04:17.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a11122640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0cf9c0 con 0x7f6a0c06ce10 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 0x7f6a0c09e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a03fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 0x7f6a0c09e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57042/0 (socket says 172.21.15.148:57042) 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a03fff640 1 -- 172.21.15.148:0/3699039486 learned_addr learned my addr 172.21.15.148:0/3699039486 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a03fff640 1 -- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0c06ce10 msgr2=0x7f6a0c09b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a03fff640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0c06ce10 0x7f6a0c09b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a03fff640 1 -- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c15dc70 msgr2=0x7f6a0c1f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.299+0000 7f6a0b59e640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c1f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a03fff640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c1f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a03fff640 1 -- 172.21.15.148:0/3699039486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c09ed50 con 0x7f6a0c09a450 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a0b59e640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c1f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a03fff640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 0x7f6a0c09e520 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f69f400c500 tx=0x7f69f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4017070 con 0x7f6a0c09a450 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f4005ce0 con 0x7f6a0c09a450 2024-04-25T10:04:17.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c09f030 con 0x7f6a0c09a450 2024-04-25T10:04:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4007330 con 0x7f6a0c09a450 2024-04-25T10:04:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.300+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c1f6ea0 con 0x7f6a0c09a450 2024-04-25T10:04:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.301+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c073990 con 0x7f6a0c09a450 2024-04-25T10:04:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.301+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69f401a070 con 0x7f6a0c09a450 2024-04-25T10:04:17.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.302+0000 7f6a037fe640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69ec06b4d0 0x7f69ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.302+0000 7f6a0ad9d640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69ec06b4d0 0x7f69ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.302+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f69f4089810 con 0x7f6a0c09a450 2024-04-25T10:04:17.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.302+0000 7f6a0ad9d640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69ec06b4d0 0x7f69ec06d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f69fc034000 tx=0x7f69fc02c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.306+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69f4058600 con 0x7f6a0c09a450 2024-04-25T10:04:17.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.478+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a0c0b40a0 con 0x7f6a0c09a450 2024-04-25T10:04:17.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.479+0000 7f6a037fe640 1 -- 172.21.15.148:0/3699039486 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f69f4057f00 con 0x7f6a0c09a450 2024-04-25T10:04:17.485 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:17.485 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":147,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:05.942649+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.2b","mappings":[{"from":3,"to":0}]},{"pgid":"22.36","mappings":[{"from":3,"to":1}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69ec06b4d0 msgr2=0x7f69ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69ec06b4d0 0x7f69ec06d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f69fc034000 tx=0x7f69fc02c000 comp rx=0 tx=0).stop 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 msgr2=0x7f6a0c09e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 0x7f6a0c09e520 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f69f400c500 tx=0x7f69f400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 shutdown_connections 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69ec06b4d0 0x7f69ec06d990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c15dc70 0x7f6a0c1f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a0c09a450 0x7f6a0c09e520 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 --2- 172.21.15.148:0/3699039486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0c06ce10 0x7f6a0c09b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 >> 172.21.15.148:0/3699039486 conn(0x7f6a0c0b3270 msgr2=0x7f6a0c0cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 shutdown_connections 2024-04-25T10:04:17.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.481+0000 7f6a11122640 1 -- 172.21.15.148:0/3699039486 wait complete. 2024-04-25T10:04:17.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:48.480288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:49.199618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:17.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:17.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.593+0000 7fa85c238640 1 Processor -- start 2024-04-25T10:04:17.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.593+0000 7fa85c238640 1 -- start start 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa85c238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 0x7fa8540afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa85c238640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa85406abe0 0x7fa85406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa85c238640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8540b5770 con 0x7fa8540b52e0 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa85c238640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8540b3590 con 0x7fa85406abe0 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa85c238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540b37e0 con 0x7fa8541453f0 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa8597d3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa8540b52e0 0x7fa8540b42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42080/0 (socket says 172.21.15.148:42080) 2024-04-25T10:04:17.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.594+0000 7fa8597d3640 1 -- 172.21.15.148:0/1127341956 learned_addr learned my addr 172.21.15.148:0/1127341956 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 0x7fa8540afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa85a7d5640 1 --2- 172.21.15.148:0/1127341956 >> v2:172.21.15.188:3300/0 conn(0x7fa85406abe0 0x7fa85406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174593533 0 0) 0x7fa8540b5770 con 0x7fa8540b52e0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa840003520 con 0x7fa8540b52e0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842888388 0 0) 0x7fa840003520 con 0x7fa8540b52e0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 msgr2=0x7fa8540afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 0x7fa8540afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 >> v2:172.21.15.188:3300/0 conn(0x7fa85406abe0 msgr2=0x7fa85406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 --2- 172.21.15.148:0/1127341956 >> v2:172.21.15.188:3300/0 conn(0x7fa85406abe0 0x7fa85406b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540b3ac0 con 0x7fa8540b52e0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa844001670 con 0x7fa8540b52e0 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 >> v1:172.21.15.148:6789/0 conn(0x7fa8540b52e0 legacy=0x7fa8540b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8400039b0 0x7fa840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 0x7fa840004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa840004d40 0x7fa8400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540b3590 con 0x7fa8400039b0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540b37e0 con 0x7fa840004d40 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000b9c0 con 0x7fa8541453f0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8400039b0 0x7fa840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa8597d3640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 0x7fa840004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa85a7d5640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa840004d40 0x7fa8400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 -- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 msgr2=0x7fa840004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8541453f0 0x7fa840004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 -- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa840004d40 msgr2=0x7fa8400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa840004d40 0x7fa8400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.595+0000 7fa859fd4640 1 -- 172.21.15.148:0/1127341956 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840003520 con 0x7fa8400039b0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.596+0000 7fa859fd4640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8400039b0 0x7fa840003dd0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fa850009ac0 tx=0x7fa840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14037d6f9259a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.596+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850036070 con 0x7fa8400039b0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.596+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa850031400 con 0x7fa8400039b0 2024-04-25T10:04:17.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.596+0000 7fa858fd2640 1 -- 172.21.15.148:0/1127341956 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850009690 con 0x7fa8400039b0 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85c238640 1 -- 172.21.15.148:0/1127341956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8400039b0 msgr2=0x7fa840003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85c238640 1 --2- 172.21.15.148:0/1127341956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8400039b0 0x7fa840003dd0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fa850009ac0 tx=0x7fa840005830 comp rx=0 tx=0).stop 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85a7d5640 1 -- 172.21.15.148:0/1127341956 reap_dead start 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85c238640 1 -- 172.21.15.148:0/1127341956 shutdown_connections 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85c238640 1 -- 172.21.15.148:0/1127341956 >> 172.21.15.148:0/1127341956 conn(0x7fa8540b5af0 msgr2=0x7fa8540b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85c238640 1 -- 172.21.15.148:0/1127341956 shutdown_connections 2024-04-25T10:04:17.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.597+0000 7fa85c238640 1 -- 172.21.15.148:0/1127341956 wait complete. 2024-04-25T10:04:17.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.598+0000 7fa85c238640 1 Processor -- start 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.598+0000 7fa85c238640 1 -- start start 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8400039b0 0x7fa854140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 0x7fa85413d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85413f360 0x7fa85413f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540afa50 con 0x7fa85413cf70 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540b49d0 con 0x7fa8400039b0 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540afbd0 con 0x7fa85413f360 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 0x7fa85413d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 0x7fa85413d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57072/0 (socket says 172.21.15.148:57072) 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 -- 172.21.15.148:0/2955256036 learned_addr learned my addr 172.21.15.148:0/2955256036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa859fd4640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8400039b0 0x7fa854140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85a7d5640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85413f360 0x7fa85413f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 -- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85413f360 msgr2=0x7fa85413f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85413f360 0x7fa85413f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 -- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8400039b0 msgr2=0x7fa854140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8400039b0 0x7fa854140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 -- 172.21.15.148:0/2955256036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85413dc60 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa859fd4640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8400039b0 0x7fa854140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa8597d3640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 0x7fa85413d3f0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e920 tx=0x7fa84400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440074e0 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8440047e0 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85413df40 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.599+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844004bd0 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.600+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85413ee40 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.601+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85406cc80 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.601+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa84400d790 con 0x7fa85413cf70 2024-04-25T10:04:17.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.601+0000 7fa84a7fc640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa83806b4d0 0x7fa83806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.601+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa84408a230 con 0x7fa85413cf70 2024-04-25T10:04:17.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.601+0000 7fa859fd4640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa83806b4d0 0x7fa83806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.602+0000 7fa859fd4640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa83806b4d0 0x7fa83806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa850033000 tx=0x7fa850008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.605+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa844013070 con 0x7fa85413cf70 2024-04-25T10:04:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.777+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa85413e460 con 0x7fa85413cf70 2024-04-25T10:04:17.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.777+0000 7fa84a7fc640 1 -- 172.21.15.148:0/2955256036 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa85413e460 con 0x7fa85413cf70 2024-04-25T10:04:17.783 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa83806b4d0 msgr2=0x7fa83806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa83806b4d0 0x7fa83806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa850033000 tx=0x7fa850008030 comp rx=0 tx=0).stop 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 msgr2=0x7fa85413d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 0x7fa85413d3f0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e920 tx=0x7fa84400ede0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 shutdown_connections 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa83806b4d0 0x7fa83806d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85413f360 0x7fa85413f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa85413cf70 0x7fa85413d3f0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 --2- 172.21.15.148:0/2955256036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8400039b0 0x7fa854140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.779+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 >> 172.21.15.148:0/2955256036 conn(0x7fa8540b5af0 msgr2=0x7fa854074060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.780+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 shutdown_connections 2024-04-25T10:04:17.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.780+0000 7fa85c238640 1 -- 172.21.15.148:0/2955256036 wait complete. 2024-04-25T10:04:17.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:04:17.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.890+0000 7f3827bc8640 1 Processor -- start 2024-04-25T10:04:17.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.890+0000 7f3827bc8640 1 -- start start 2024-04-25T10:04:17.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3827bc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38200cc4f0 0x7f38200cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3827bc8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38200cdb20 0x7f38200cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3827bc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f382015b270 0x7f382015b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3827bc8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38200ce4c0 con 0x7f38200cdb20 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3827bc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3820071270 con 0x7f382015b270 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3827bc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3820072280 con 0x7f38200cc4f0 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38200cdb20 0x7f38200cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38200cdb20 0x7f38200cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57078/0 (socket says 172.21.15.148:57078) 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 -- 172.21.15.148:0/1063072959 learned_addr learned my addr 172.21.15.148:0/1063072959 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825964640 1 --2- 172.21.15.148:0/1063072959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38200cc4f0 0x7f38200cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3826165640 1 --2- 172.21.15.148:0/1063072959 >> v2:172.21.15.188:3300/0 conn(0x7f382015b270 0x7f382015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 -- 172.21.15.148:0/1063072959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38200cc4f0 msgr2=0x7f38200cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 --2- 172.21.15.148:0/1063072959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38200cc4f0 0x7f38200cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 -- 172.21.15.148:0/1063072959 >> v2:172.21.15.188:3300/0 conn(0x7f382015b270 msgr2=0x7f382015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 --2- 172.21.15.148:0/1063072959 >> v2:172.21.15.188:3300/0 conn(0x7f382015b270 0x7f382015b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.891+0000 7f3825163640 1 -- 172.21.15.148:0/1063072959 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200714c0 con 0x7f38200cdb20 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.892+0000 7f3825163640 1 --2- 172.21.15.148:0/1063072959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cdb20 0x7f38200cdf80 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f3820158aa0 tx=0x7f381002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498f6e00a8c15fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.892+0000 7f3824962640 1 -- 172.21.15.148:0/1063072959 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810032790 con 0x7f38200cdb20 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.892+0000 7f3824962640 1 -- 172.21.15.148:0/1063072959 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3810038070 con 0x7f38200cdb20 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.892+0000 7f3824962640 1 -- 172.21.15.148:0/1063072959 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810032b50 con 0x7f38200cdb20 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 -- 172.21.15.148:0/1063072959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cdb20 msgr2=0x7f38200cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 --2- 172.21.15.148:0/1063072959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cdb20 0x7f38200cdf80 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f3820158aa0 tx=0x7f381002c890 comp rx=0 tx=0).stop 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 -- 172.21.15.148:0/1063072959 shutdown_connections 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 --2- 172.21.15.148:0/1063072959 >> v2:172.21.15.188:3300/0 conn(0x7f382015b270 0x7f382015b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 --2- 172.21.15.148:0/1063072959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cdb20 0x7f38200cdf80 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 --2- 172.21.15.148:0/1063072959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38200cc4f0 0x7f38200cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 -- 172.21.15.148:0/1063072959 >> 172.21.15.148:0/1063072959 conn(0x7f382006abe0 msgr2=0x7f382006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 -- 172.21.15.148:0/1063072959 shutdown_connections 2024-04-25T10:04:17.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.893+0000 7f3827bc8640 1 -- 172.21.15.148:0/1063072959 wait complete. 2024-04-25T10:04:17.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.894+0000 7f3827bc8640 1 Processor -- start 2024-04-25T10:04:17.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.894+0000 7f3827bc8640 1 -- start start 2024-04-25T10:04:17.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 0x7f382013cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cdb20 0x7f3820154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b270 0x7f38201545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820071d60 con 0x7f38200cc4f0 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820071a80 con 0x7f38200cdb20 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382015bbc0 con 0x7f382015b270 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 0x7f382013cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 0x7f382013cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57080/0 (socket says 172.21.15.148:57080) 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 -- 172.21.15.148:0/2590590846 learned_addr learned my addr 172.21.15.148:0/2590590846 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825163640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cdb20 0x7f3820154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3826165640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b270 0x7f38201545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 -- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b270 msgr2=0x7f38201545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b270 0x7f38201545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 -- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cdb20 msgr2=0x7f3820154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cdb20 0x7f3820154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 -- 172.21.15.148:0/2590590846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3820154be0 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825163640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cdb20 0x7f3820154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3825964640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 0x7f382013cc90 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c5a0 tx=0x7f381c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c015070 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c004220 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3820154ec0 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c003610 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.895+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382013d1d0 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.896+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3820073a00 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.897+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f381c0037b0 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.897+0000 7f38167fc640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f380406b500 0x7f380406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.897+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f381c08a140 con 0x7f38200cc4f0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.897+0000 7f3825163640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f380406b500 0x7f380406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.898+0000 7f3825163640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f380406b500 0x7f380406d9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3810037000 tx=0x7f381003e040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:17.901+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f381c058de0 con 0x7f38200cc4f0 2024-04-25T10:04:18.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.072+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f382015ca10 con 0x7f38200cc4f0 2024-04-25T10:04:18.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.072+0000 7f38167fc640 1 -- 172.21.15.148:0/2590590846 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+11 (secure 0 0 0) 0x7f381c0586e0 con 0x7f38200cc4f0 2024-04-25T10:04:18.077 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:04:18.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f380406b500 msgr2=0x7f380406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f380406b500 0x7f380406d9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3810037000 tx=0x7f381003e040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 msgr2=0x7f382013cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 0x7f382013cc90 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c5a0 tx=0x7f381c00ca60 comp rx=0 tx=0).stop 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 shutdown_connections 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f380406b500 0x7f380406d9c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b270 0x7f38201545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cdb20 0x7f3820154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 --2- 172.21.15.148:0/2590590846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38200cc4f0 0x7f382013cc90 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 >> 172.21.15.148:0/2590590846 conn(0x7f382006abe0 msgr2=0x7f38200cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 shutdown_connections 2024-04-25T10:04:18.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.074+0000 7f3827bc8640 1 -- 172.21.15.148:0/2590590846 wait complete. 2024-04-25T10:04:18.090 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:04:18.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.184+0000 7f5727ab3640 1 Processor -- start 2024-04-25T10:04:18.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.185+0000 7f5727ab3640 1 -- start start 2024-04-25T10:04:18.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.185+0000 7f5727ab3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5720161710 0x7f57200d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.185+0000 7f5727ab3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572015e000 con 0x7f572015dc20 2024-04-25T10:04:18.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.185+0000 7f5727ab3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57200d1de0 con 0x7f57200ced80 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.185+0000 7f5727ab3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5720161c60 con 0x7f5720161710 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572584f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f572015dc20 0x7f57200ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42084/0 (socket says 172.21.15.148:42084) 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572584f640 1 -- 172.21.15.148:0/1263098843 learned_addr learned my addr 172.21.15.148:0/1263098843 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f5726050640 1 --2- 172.21.15.148:0/1263098843 >> v2:172.21.15.188:3300/0 conn(0x7f5720161710 0x7f57200d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190579637 0 0) 0x7f572015e000 con 0x7f572015dc20 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f570c003520 con 0x7f572015dc20 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263768966 0 0) 0x7f570c003520 con 0x7f572015dc20 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 >> v2:172.21.15.188:3300/0 conn(0x7f5720161710 msgr2=0x7f57200d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 --2- 172.21.15.148:0/1263098843 >> v2:172.21.15.188:3300/0 conn(0x7f5720161710 0x7f57200d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 >> v1:172.21.15.188:6789/0 conn(0x7f57200ced80 legacy=0x7f57200d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720161f40 con 0x7f572015dc20 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f571c001670 con 0x7f572015dc20 2024-04-25T10:04:18.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 >> v1:172.21.15.148:6789/0 conn(0x7f572015dc20 legacy=0x7f57200ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0039b0 0x7f570c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c004800 0x7f570c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0051c0 0x7f570c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5720161c60 con 0x7f570c0051c0 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c00c060 con 0x7f570c004800 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c00c260 con 0x7f570c0039b0 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f5726050640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0051c0 0x7f570c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572504e640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c004800 0x7f570c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.186+0000 7f572584f640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0039b0 0x7f570c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 -- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0039b0 msgr2=0x7f570c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0039b0 0x7f570c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 -- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c004800 msgr2=0x7f570c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f570c004800 0x7f570c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 -- 172.21.15.148:0/1263098843 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c003520 con 0x7f570c0051c0 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 -- 172.21.15.148:0/1263098843 reap_dead start 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f5726050640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0051c0 0x7f570c00b9c0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f5718009fc0 tx=0x7f570c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f857687f03c10295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718035020 con 0x7f570c0051c0 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5718003b90 con 0x7f570c0051c0 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.187+0000 7f572484d640 1 -- 172.21.15.148:0/1263098843 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718009c10 con 0x7f570c0051c0 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 -- 172.21.15.148:0/1263098843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0051c0 msgr2=0x7f570c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0051c0 0x7f570c00b9c0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f5718009fc0 tx=0x7f570c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 -- 172.21.15.148:0/1263098843 shutdown_connections 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 --2- 172.21.15.148:0/1263098843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f570c0051c0 0x7f570c00b9c0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 -- 172.21.15.148:0/1263098843 >> 172.21.15.148:0/1263098843 conn(0x7f57200d0420 msgr2=0x7f57200d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 -- 172.21.15.148:0/1263098843 shutdown_connections 2024-04-25T10:04:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.188+0000 7f5727ab3640 1 -- 172.21.15.148:0/1263098843 wait complete. 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.189+0000 7f5727ab3640 1 Processor -- start 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.189+0000 7f5727ab3640 1 -- start start 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5727ab3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0051c0 0x7f5720155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5727ab3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5720160ce0 0x7f57200c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5727ab3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 0x7f5720157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5727ab3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200b5e50 con 0x7f5720157260 2024-04-25T10:04:18.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5727ab3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5720162500 con 0x7f5720160ce0 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5727ab3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57200b5fd0 con 0x7f570c0051c0 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 0x7f5720157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 0x7f5720157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57108/0 (socket says 172.21.15.148:57108) 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 -- 172.21.15.148:0/3220755335 learned_addr learned my addr 172.21.15.148:0/3220755335 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f572584f640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0051c0 0x7f5720155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f572504e640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5720160ce0 0x7f57200c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 -- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0051c0 msgr2=0x7f5720155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0051c0 0x7f5720155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 -- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5720160ce0 msgr2=0x7f57200c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5720160ce0 0x7f57200c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f5726050640 1 -- 172.21.15.148:0/3220755335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720155fe0 con 0x7f5720157260 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f572504e640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5720160ce0 0x7f57200c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.190+0000 7f572584f640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0051c0 0x7f5720155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:18.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.191+0000 7f5726050640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 0x7f5720157710 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f571802af10 tx=0x7f5718009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.191+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571803f030 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.191+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5718003990 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.191+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57201562c0 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.191+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718035070 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.191+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5720155cd0 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.192+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5720073a00 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.192+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5718042030 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.192+0000 7f57167fc640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f570406b500 0x7f570406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.192+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f57180aaa50 con 0x7f5720157260 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.193+0000 7f572584f640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f570406b500 0x7f570406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.193+0000 7f572584f640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f570406b500 0x7f570406d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f57201f6f50 tx=0x7f571c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.197+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5718079130 con 0x7f5720157260 2024-04-25T10:04:18.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.367+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57200d16b0 con 0x7f5720157260 2024-04-25T10:04:18.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.367+0000 7f57167fc640 1 -- 172.21.15.148:0/3220755335 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+11 (secure 0 0 0) 0x7f571807d1d0 con 0x7f5720157260 2024-04-25T10:04:18.373 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f570406b500 msgr2=0x7f570406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f570406b500 0x7f570406d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f57201f6f50 tx=0x7f571c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 msgr2=0x7f5720157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 0x7f5720157710 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f571802af10 tx=0x7f5718009750 comp rx=0 tx=0).stop 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 shutdown_connections 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f570406b500 0x7f570406d9c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5720157260 0x7f5720157710 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5720160ce0 0x7f57200c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 --2- 172.21.15.148:0/3220755335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f570c0051c0 0x7f5720155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.369+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 >> 172.21.15.148:0/3220755335 conn(0x7f57200d0420 msgr2=0x7f57200b8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.370+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 shutdown_connections 2024-04-25T10:04:18.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.370+0000 7f5727ab3640 1 -- 172.21.15.148:0/3220755335 wait complete. 2024-04-25T10:04:18.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:18.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.477+0000 7fd6c62ee640 1 Processor -- start 2024-04-25T10:04:18.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.478+0000 7fd6c62ee640 1 -- start start 2024-04-25T10:04:18.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.478+0000 7fd6c62ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c0158220 0x7fd6c015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.478+0000 7fd6c62ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6c00af720 0x7fd6c00afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.478+0000 7fd6c62ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c015f180 con 0x7fd6c01617b0 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.478+0000 7fd6c62ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c00b3670 con 0x7fd6c00af720 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.478+0000 7fd6c62ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0070a10 con 0x7fd6c0158220 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c0158220 0x7fd6c015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bf7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6c00af720 0x7fd6c00afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6c488b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd6c01617b0 0x7fd6c0161c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34444/0 (socket says 172.21.15.148:34444) 2024-04-25T10:04:18.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6c488b640 1 -- 172.21.15.148:0/597550165 learned_addr learned my addr 172.21.15.148:0/597550165 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6beffd640 1 -- 172.21.15.148:0/597550165 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792585626 0 0) 0x7fd6b8001490 con 0x7fd6c01617b0 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6beffd640 1 -- 172.21.15.148:0/597550165 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6a8003520 con 0x7fd6c01617b0 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bffff640 1 -- 172.21.15.148:0/597550165 >> v2:172.21.15.188:3300/0 conn(0x7fd6c00af720 msgr2=0x7fd6c00afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bffff640 1 --2- 172.21.15.148:0/597550165 >> v2:172.21.15.188:3300/0 conn(0x7fd6c00af720 0x7fd6c00afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bffff640 1 -- 172.21.15.148:0/597550165 >> v1:172.21.15.188:6789/0 conn(0x7fd6c01617b0 legacy=0x7fd6c0161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bffff640 1 -- 172.21.15.148:0/597550165 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c006fe00 con 0x7fd6c0158220 2024-04-25T10:04:18.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.479+0000 7fd6bf7fe640 1 --2- 172.21.15.148:0/597550165 >> v2:172.21.15.188:3300/0 conn(0x7fd6c00af720 0x7fd6c00afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.480+0000 7fd6bffff640 1 --2- 172.21.15.148:0/597550165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c0158220 0x7fd6c015ec40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40097a0 tx=0x7fd6b402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3b5e6734795497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.480+0000 7fd6beffd640 1 -- 172.21.15.148:0/597550165 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4037070 con 0x7fd6c0158220 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.480+0000 7fd6beffd640 1 -- 172.21.15.148:0/597550165 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b4032440 con 0x7fd6c0158220 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.480+0000 7fd6beffd640 1 -- 172.21.15.148:0/597550165 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40327f0 con 0x7fd6c0158220 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 -- 172.21.15.148:0/597550165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c0158220 msgr2=0x7fd6c015ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/597550165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c0158220 0x7fd6c015ec40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40097a0 tx=0x7fd6b402a180 comp rx=0 tx=0).stop 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 -- 172.21.15.148:0/597550165 shutdown_connections 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/597550165 >> v2:172.21.15.188:3300/0 conn(0x7fd6c00af720 0x7fd6c00afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/597550165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c0158220 0x7fd6c015ec40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 -- 172.21.15.148:0/597550165 >> 172.21.15.148:0/597550165 conn(0x7fd6c00b2370 msgr2=0x7fd6c00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 -- 172.21.15.148:0/597550165 shutdown_connections 2024-04-25T10:04:18.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.481+0000 7fd6c62ee640 1 -- 172.21.15.148:0/597550165 wait complete. 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.482+0000 7fd6c62ee640 1 Processor -- start 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.482+0000 7fd6c62ee640 1 -- start start 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c62ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c00af720 0x7fd6c00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c62ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 0x7fd6c00cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c62ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01617b0 0x7fd6c014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c62ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00706a0 con 0x7fd6c0158220 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c62ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00703c0 con 0x7fd6c01617b0 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c62ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0158600 con 0x7fd6c00af720 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 0x7fd6c00cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 0x7fd6c00cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57124/0 (socket says 172.21.15.148:57124) 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 -- 172.21.15.148:0/914038771 learned_addr learned my addr 172.21.15.148:0/914038771 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bffff640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c00af720 0x7fd6c00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6c488b640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01617b0 0x7fd6c014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 -- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c00af720 msgr2=0x7fd6c00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c00af720 0x7fd6c00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 -- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01617b0 msgr2=0x7fd6c014bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01617b0 0x7fd6c014bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 -- 172.21.15.148:0/914038771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c0159b70 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bf7fe640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 0x7fd6c00cc5f0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000ebe0 tx=0x7fd6b000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0017070 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000cbd0 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.483+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0013470 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.484+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0159e50 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.484+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c015a410 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.485+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c0073990 con 0x7fd6c0158220 2024-04-25T10:04:18.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.485+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6b001a070 con 0x7fd6c0158220 2024-04-25T10:04:18.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.485+0000 7fd6bcff9640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6ac06b500 0x7fd6ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.485+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd6b008a4f0 con 0x7fd6c0158220 2024-04-25T10:04:18.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.485+0000 7fd6bffff640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6ac06b500 0x7fd6ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.486+0000 7fd6bffff640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6ac06b500 0x7fd6ac06d9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4034000 tx=0x7fd6b402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.490+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6b0058bd0 con 0x7fd6c0158220 2024-04-25T10:04:18.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.695+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6c015cda0 con 0x7fd6c0158220 2024-04-25T10:04:18.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.695+0000 7fd6bcff9640 1 -- 172.21.15.148:0/914038771 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4961 (secure 0 0 0) 0x7fd6b005cc70 con 0x7fd6c0158220 2024-04-25T10:04:18.701 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:18.701 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:04:12.963932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9855":{"gid":9855,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:04:18.701 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 87 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6ac06b500 msgr2=0x7fd6ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6ac06b500 0x7fd6ac06d9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4034000 tx=0x7fd6b402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 msgr2=0x7fd6c00cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 0x7fd6c00cc5f0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000ebe0 tx=0x7fd6b000c520 comp rx=0 tx=0).stop 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 shutdown_connections 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd6ac06b500 0x7fd6ac06d9c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01617b0 0x7fd6c014bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6c0158220 0x7fd6c00cc5f0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 --2- 172.21.15.148:0/914038771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6c00af720 0x7fd6c00c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 >> 172.21.15.148:0/914038771 conn(0x7fd6c00b2370 msgr2=0x7fd6c006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 shutdown_connections 2024-04-25T10:04:18.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.697+0000 7fd6c62ee640 1 -- 172.21.15.148:0/914038771 wait complete. 2024-04-25T10:04:18.714 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:18.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.807+0000 7f2f0f85d640 1 Processor -- start 2024-04-25T10:04:18.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.807+0000 7f2f0f85d640 1 -- start start 2024-04-25T10:04:18.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0f85d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f0815da70 0x7f2f080d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0f85d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2f080c7730 0x7f2f080ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0f85d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f080ce700 0x7f2f08070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0f85d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080ceb30 con 0x7f2f0815da70 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0f85d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f080c8030 con 0x7f2f080c7730 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0f85d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c8310 con 0x7f2f080ce700 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f0815da70 0x7f2f080d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f2f0815da70 0x7f2f080d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57126/0 (socket says 172.21.15.148:57126) 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 -- 172.21.15.148:0/3182476628 learned_addr learned my addr 172.21.15.148:0/3182476628 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0cdf8640 1 --2- 172.21.15.148:0/3182476628 >> v2:172.21.15.188:3300/0 conn(0x7f2f080c7730 0x7f2f080ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0ddfa640 1 --2- 172.21.15.148:0/3182476628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f080ce700 0x7f2f08070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 -- 172.21.15.148:0/3182476628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f080ce700 msgr2=0x7f2f08070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3182476628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f080ce700 0x7f2f08070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 -- 172.21.15.148:0/3182476628 >> v2:172.21.15.188:3300/0 conn(0x7f2f080c7730 msgr2=0x7f2f080ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3182476628 >> v2:172.21.15.188:3300/0 conn(0x7f2f080c7730 0x7f2f080ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.808+0000 7f2f0d5f9640 1 -- 172.21.15.148:0/3182476628 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080c85f0 con 0x7f2f0815da70 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.809+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3182476628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f0815da70 0x7f2f080d0170 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f2f080b1040 tx=0x7f2f0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4a34667c454f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.809+0000 7f2efffff640 1 -- 172.21.15.148:0/3182476628 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04037070 con 0x7f2f0815da70 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.809+0000 7f2efffff640 1 -- 172.21.15.148:0/3182476628 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f04032440 con 0x7f2f0815da70 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.809+0000 7f2efffff640 1 -- 172.21.15.148:0/3182476628 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04032810 con 0x7f2f0815da70 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3182476628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f0815da70 msgr2=0x7f2f080d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3182476628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f0815da70 0x7f2f080d0170 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f2f080b1040 tx=0x7f2f0402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3182476628 shutdown_connections 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3182476628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f080ce700 0x7f2f08070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3182476628 >> v2:172.21.15.188:3300/0 conn(0x7f2f080c7730 0x7f2f080ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3182476628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f0815da70 0x7f2f080d0170 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3182476628 >> 172.21.15.148:0/3182476628 conn(0x7f2f080b33e0 msgr2=0x7f2f080b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3182476628 shutdown_connections 2024-04-25T10:04:18.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.810+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3182476628 wait complete. 2024-04-25T10:04:18.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.811+0000 7f2f0f85d640 1 Processor -- start 2024-04-25T10:04:18.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.811+0000 7f2f0f85d640 1 -- start start 2024-04-25T10:04:18.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f080c7730 0x7f2f081f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 0x7f2f08160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0815da70 0x7f2f081593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0815de50 con 0x7f2f080ce700 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0806be10 con 0x7f2f080c7730 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f080c7bc0 con 0x7f2f0815da70 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 0x7f2f08160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 0x7f2f08160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57130/0 (socket says 172.21.15.148:57130) 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 -- 172.21.15.148:0/3219045970 learned_addr learned my addr 172.21.15.148:0/3219045970 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f080c7730 0x7f2f081f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0ddfa640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0815da70 0x7f2f081593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 -- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0815da70 msgr2=0x7f2f081593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0815da70 0x7f2f081593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 -- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f080c7730 msgr2=0x7f2f081f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f080c7730 0x7f2f081f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 -- 172.21.15.148:0/3219045970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f081612f0 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f080c7730 0x7f2f081f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0ddfa640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0815da70 0x7f2f081593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0cdf8640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 0x7f2f08160db0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d800 tx=0x7f2ef800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800b6c0 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef8019070 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800ba80 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.812+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0815e340 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.813+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0815e900 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.814+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0815f010 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.814+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ef800bc20 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.814+0000 7f2efdffb640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2eec06b4d0 0x7f2eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.814+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2ef808a830 con 0x7f2f080ce700 2024-04-25T10:04:18.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.815+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2eec06b4d0 0x7f2eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.815+0000 7f2f0d5f9640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2eec06b4d0 0x7f2eec06d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034000 tx=0x7f2f0402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.819+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ef8059560 con 0x7f2f080ce700 2024-04-25T10:04:18.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.990+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f081601e0 con 0x7f2f080ce700 2024-04-25T10:04:18.997 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.991+0000 7f2efdffb640 1 -- 172.21.15.148:0/3219045970 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f2ef8058e60 con 0x7f2f080ce700 2024-04-25T10:04:18.997 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:18.997 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":147,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:05.942649+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.2b","mappings":[{"from":3,"to":0}]},{"pgid":"22.36","mappings":[{"from":3,"to":1}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2eec06b4d0 msgr2=0x7f2eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2eec06b4d0 0x7f2eec06d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034000 tx=0x7f2f0402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 msgr2=0x7f2f08160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 0x7f2f08160db0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d800 tx=0x7f2ef800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 shutdown_connections 2024-04-25T10:04:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2eec06b4d0 0x7f2eec06d990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0815da70 0x7f2f081593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f080ce700 0x7f2f08160db0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 --2- 172.21.15.148:0/3219045970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f080c7730 0x7f2f081f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 >> 172.21.15.148:0/3219045970 conn(0x7f2f080b33e0 msgr2=0x7f2f080cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 shutdown_connections 2024-04-25T10:04:19.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:18.994+0000 7f2f0f85d640 1 -- 172.21.15.148:0/3219045970 wait complete. 2024-04-25T10:04:19.010 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:04:19.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:19.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.104+0000 7fd7deb22640 1 Processor -- start 2024-04-25T10:04:19.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.104+0000 7fd7deb22640 1 -- start start 2024-04-25T10:04:19.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.104+0000 7fd7deb22640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7d815dd80 0x7fd7d815e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.104+0000 7fd7deb22640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d80cfaf0 con 0x7fd7d806caf0 2024-04-25T10:04:19.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.104+0000 7fd7deb22640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d815e8e0 con 0x7fd7d8161800 2024-04-25T10:04:19.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.104+0000 7fd7deb22640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d815e6a0 con 0x7fd7d815dd80 2024-04-25T10:04:19.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.105+0000 7fd7dc8be640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7d815dd80 0x7fd7d815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.105+0000 7fd7dd0bf640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd7d806caf0 0x7fd7d80cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42100/0 (socket says 172.21.15.148:42100) 2024-04-25T10:04:19.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.105+0000 7fd7dd0bf640 1 -- 172.21.15.148:0/1954552913 learned_addr learned my addr 172.21.15.148:0/1954552913 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:19.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.105+0000 7fd7dc8be640 1 -- 172.21.15.148:0/1954552913 >> v1:172.21.15.188:6789/0 conn(0x7fd7d8161800 legacy=0x7fd7d806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.105+0000 7fd7dc8be640 1 -- 172.21.15.148:0/1954552913 >> v1:172.21.15.148:6789/0 conn(0x7fd7d806caf0 legacy=0x7fd7d80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.105+0000 7fd7dc8be640 1 -- 172.21.15.148:0/1954552913 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d815ebc0 con 0x7fd7d815dd80 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7dc8be640 1 --2- 172.21.15.148:0/1954552913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d815e160 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0097a0 tx=0x7fd7cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e9b6e69c277730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7d77fe640 1 -- 172.21.15.148:0/1954552913 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc037070 con 0x7fd7d815dd80 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7d77fe640 1 -- 172.21.15.148:0/1954552913 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc032440 con 0x7fd7d815dd80 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7d77fe640 1 -- 172.21.15.148:0/1954552913 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc032810 con 0x7fd7d815dd80 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 -- 172.21.15.148:0/1954552913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 msgr2=0x7fd7d815e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 --2- 172.21.15.148:0/1954552913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d815e160 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0097a0 tx=0x7fd7cc02a230 comp rx=0 tx=0).stop 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 -- 172.21.15.148:0/1954552913 shutdown_connections 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 --2- 172.21.15.148:0/1954552913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d815e160 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 -- 172.21.15.148:0/1954552913 >> 172.21.15.148:0/1954552913 conn(0x7fd7d80b3310 msgr2=0x7fd7d80b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 -- 172.21.15.148:0/1954552913 shutdown_connections 2024-04-25T10:04:19.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.106+0000 7fd7deb22640 1 -- 172.21.15.148:0/1954552913 wait complete. 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.107+0000 7fd7deb22640 1 Processor -- start 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.107+0000 7fd7deb22640 1 -- start start 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7deb22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d806caf0 0x7fd7d8151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7deb22640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7deb22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7d8161800 0x7fd7d80c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7deb22640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d806c0c0 con 0x7fd7d815dd80 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7deb22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d815f180 con 0x7fd7d806caf0 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7deb22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80d06e0 con 0x7fd7d8161800 2024-04-25T10:04:19.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d80c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57148/0 (socket says 172.21.15.148:57148) 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.108+0000 7fd7d7fff640 1 -- 172.21.15.148:0/516525471 learned_addr learned my addr 172.21.15.148:0/516525471 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7dc8be640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d806caf0 0x7fd7d8151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7dd0bf640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7d8161800 0x7fd7d80c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d7fff640 1 -- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7d8161800 msgr2=0x7fd7d80c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d7fff640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7d8161800 0x7fd7d80c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d7fff640 1 -- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d806caf0 msgr2=0x7fd7d8151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d7fff640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d806caf0 0x7fd7d8151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d7fff640 1 -- 172.21.15.148:0/516525471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80c9b00 con 0x7fd7d815dd80 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7dc8be640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d806caf0 0x7fd7d8151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d7fff640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d80c88f0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c3c0 tx=0x7fd7c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40052d0 con 0x7fd7d815dd80 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4005490 con 0x7fd7d815dd80 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d80c9260 con 0x7fd7d815dd80 2024-04-25T10:04:19.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4007260 con 0x7fd7d815dd80 2024-04-25T10:04:19.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.109+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d80c9820 con 0x7fd7d815dd80 2024-04-25T10:04:19.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.110+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d8073990 con 0x7fd7d815dd80 2024-04-25T10:04:19.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.110+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7c4007400 con 0x7fd7d815dd80 2024-04-25T10:04:19.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.111+0000 7fd7d57fa640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.111+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd7c4089830 con 0x7fd7d815dd80 2024-04-25T10:04:19.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.111+0000 7fd7dc8be640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.111+0000 7fd7dc8be640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc034000 tx=0x7fd7cc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.115+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7c4058560 con 0x7fd7d815dd80 2024-04-25T10:04:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.323+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7d809a2a0 con 0x7fd7d815dd80 2024-04-25T10:04:19.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.323+0000 7fd7d57fa640 1 -- 172.21.15.148:0/516525471 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4961 (secure 0 0 0) 0x7fd7c4057e60 con 0x7fd7d815dd80 2024-04-25T10:04:19.329 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:19.329 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:04:12.963932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9855":{"gid":9855,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:04:19.329 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 87 2024-04-25T10:04:19.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd7bc06b4d0 msgr2=0x7fd7bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc034000 tx=0x7fd7cc02e000 comp rx=0 tx=0).stop 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 msgr2=0x7fd7d80c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d80c88f0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c3c0 tx=0x7fd7c400c880 comp rx=0 tx=0).stop 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 shutdown_connections 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7d8161800 0x7fd7d80c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7d815dd80 0x7fd7d80c88f0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 --2- 172.21.15.148:0/516525471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d806caf0 0x7fd7d8151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 >> 172.21.15.148:0/516525471 conn(0x7fd7d80b3310 msgr2=0x7fd7d80d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 shutdown_connections 2024-04-25T10:04:19.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.325+0000 7fd7deb22640 1 -- 172.21.15.148:0/516525471 wait complete. 2024-04-25T10:04:19.342 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.433+0000 7f1820bb6640 1 Processor -- start 2024-04-25T10:04:19.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.433+0000 7f1820bb6640 1 -- start start 2024-04-25T10:04:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1820bb6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f181c15db70 0x7f181c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1820bb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b5c30 0x7f181c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1820bb6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181c162690 con 0x7f181c0b6cb0 2024-04-25T10:04:19.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1820bb6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181c0ceee0 con 0x7f181c15db70 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1820bb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181c0d1d10 con 0x7f181c0b5c30 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f181c15db70 0x7f181c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1819d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f181c0b6cb0 0x7f181c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42112/0 (socket says 172.21.15.148:42112) 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181a59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f181c15db70 0x7f181c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57160/0 (socket says 172.21.15.148:57160) 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f1819d9b640 1 -- 172.21.15.148:0/3307852763 learned_addr learned my addr 172.21.15.148:0/3307852763 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181ad9d640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b5c30 0x7f181c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181a59c640 1 -- 172.21.15.148:0/3307852763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b5c30 msgr2=0x7f181c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181a59c640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b5c30 0x7f181c0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181a59c640 1 -- 172.21.15.148:0/3307852763 >> v1:172.21.15.148:6789/0 conn(0x7f181c0b6cb0 legacy=0x7f181c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181959a640 1 -- 172.21.15.148:0/3307852763 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330717160 0 0) 0x7f181c162690 con 0x7f181c0b6cb0 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.434+0000 7f181a59c640 1 -- 172.21.15.148:0/3307852763 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0d1ff0 con 0x7f181c15db70 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.435+0000 7f181ad9d640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b5c30 0x7f181c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.435+0000 7f181a59c640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c15db70 0x7f181c162150 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f180c0097a0 tx=0x7f180c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf169d090f1ab81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.435+0000 7f181959a640 1 -- 172.21.15.148:0/3307852763 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c037070 con 0x7f181c15db70 2024-04-25T10:04:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.435+0000 7f181959a640 1 -- 172.21.15.148:0/3307852763 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c032440 con 0x7f181c15db70 2024-04-25T10:04:19.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.435+0000 7f181959a640 1 -- 172.21.15.148:0/3307852763 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c032810 con 0x7f181c15db70 2024-04-25T10:04:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 -- 172.21.15.148:0/3307852763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c15db70 msgr2=0x7f181c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c15db70 0x7f181c162150 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f180c0097a0 tx=0x7f180c02a180 comp rx=0 tx=0).stop 2024-04-25T10:04:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 -- 172.21.15.148:0/3307852763 shutdown_connections 2024-04-25T10:04:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b5c30 0x7f181c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 --2- 172.21.15.148:0/3307852763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c15db70 0x7f181c162150 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 -- 172.21.15.148:0/3307852763 >> 172.21.15.148:0/3307852763 conn(0x7f181c0d0520 msgr2=0x7f181c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 -- 172.21.15.148:0/3307852763 shutdown_connections 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.436+0000 7f1820bb6640 1 -- 172.21.15.148:0/3307852763 wait complete. 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.437+0000 7f1820bb6640 1 Processor -- start 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.437+0000 7f1820bb6640 1 -- start start 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1820bb6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 0x7f181c161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1820bb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b6cb0 0x7f181c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1820bb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f181c15db70 0x7f181c147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1820bb6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c0ce530 con 0x7f181c0b5c30 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1820bb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c15df50 con 0x7f181c15db70 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1820bb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181c0cf1c0 con 0x7f181c0b6cb0 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 0x7f181c161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 0x7f181c161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57170/0 (socket says 172.21.15.148:57170) 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 -- 172.21.15.148:0/2634348167 learned_addr learned my addr 172.21.15.148:0/2634348167 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:19.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 -- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b6cb0 msgr2=0x7f181c155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181ad9d640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f181c15db70 0x7f181c147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f1819d9b640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b6cb0 0x7f181c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b6cb0 0x7f181c155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 -- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f181c15db70 msgr2=0x7f181c147cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f181c15db70 0x7f181c147cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.438+0000 7f181a59c640 1 -- 172.21.15.148:0/2634348167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c148230 con 0x7f181c0b5c30 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.439+0000 7f181a59c640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 0x7f181c161100 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f180c000c00 tx=0x7f180c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.439+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c037040 con 0x7f181c0b5c30 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.439+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c042070 con 0x7f181c0b5c30 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.439+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c160060 con 0x7f181c0b5c30 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.439+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c030bb0 con 0x7f181c0b5c30 2024-04-25T10:04:19.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.439+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c160620 con 0x7f181c0b5c30 2024-04-25T10:04:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.440+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c073990 con 0x7f181c0b5c30 2024-04-25T10:04:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.440+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f180c047460 con 0x7f181c0b5c30 2024-04-25T10:04:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.440+0000 7f180affd640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f17fc06b500 0x7f17fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.441+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f180c0aaf00 con 0x7f181c0b5c30 2024-04-25T10:04:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.441+0000 7f1819d9b640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f17fc06b500 0x7f17fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.441+0000 7f1819d9b640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f17fc06b500 0x7f17fc06d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1810007fd0 tx=0x7f1810009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.445+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f180c032e20 con 0x7f181c0b5c30 2024-04-25T10:04:19.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.648+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f181c0d1820 con 0x7f181c0b5c30 2024-04-25T10:04:19.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.649+0000 7f180affd640 1 -- 172.21.15.148:0/2634348167 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4961 (secure 0 0 0) 0x7f180c079c30 con 0x7f181c0b5c30 2024-04-25T10:04:19.654 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:19.654 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:03:50.223124+0000","modified":"2024-04-25T10:04:12.963932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9855":{"gid":9855,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/1405903723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1405903723},{"type":"v1","addr":"172.21.15.148:6835","nonce":1405903723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9861":{"gid":9861,"name":"c","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3188696943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3188696943},{"type":"v1","addr":"172.21.15.148:6837","nonce":3188696943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:04:19.654 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 87 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f17fc06b500 msgr2=0x7f17fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f17fc06b500 0x7f17fc06d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1810007fd0 tx=0x7f1810009040 comp rx=0 tx=0).stop 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 msgr2=0x7f181c161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 0x7f181c161100 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f180c000c00 tx=0x7f180c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 shutdown_connections 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f17fc06b500 0x7f17fc06d9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f181c15db70 0x7f181c147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f181c0b6cb0 0x7f181c155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 --2- 172.21.15.148:0/2634348167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f181c0b5c30 0x7f181c161100 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 >> 172.21.15.148:0/2634348167 conn(0x7f181c0d0520 msgr2=0x7f181c06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 shutdown_connections 2024-04-25T10:04:19.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.651+0000 7f1820bb6640 1 -- 172.21.15.148:0/2634348167 wait complete. 2024-04-25T10:04:19.667 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:19.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.758+0000 7f0614e9c640 1 Processor -- start 2024-04-25T10:04:19.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.758+0000 7f0614e9c640 1 -- start start 2024-04-25T10:04:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.759+0000 7f0614e9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f06100d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.759+0000 7f0614e9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06100b92a0 0x7f06100d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.759+0000 7f0614e9c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06100b97a0 con 0x7f06100b2cb0 2024-04-25T10:04:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.759+0000 7f0614e9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061015dfc0 con 0x7f06100b92a0 2024-04-25T10:04:19.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.759+0000 7f0614e9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100d1a00 con 0x7f061015dbe0 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f06100d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06100b92a0 0x7f06100d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060dd9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f06100b2cb0 0x7f06100b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34456/0 (socket says 172.21.15.148:34456) 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06100b92a0 0x7f06100d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:37244/0 (socket says 172.21.15.148:37244) 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f06100d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:45678/0 (socket says 172.21.15.148:45678) 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060dd9b640 1 -- 172.21.15.148:0/3492865150 learned_addr learned my addr 172.21.15.148:0/3492865150 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060ed9d640 1 -- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 msgr2=0x7f06100d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060ed9d640 1 --2- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f06100d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060ed9d640 1 -- 172.21.15.148:0/3492865150 >> v1:172.21.15.188:6789/0 conn(0x7f06100b2cb0 legacy=0x7f06100b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060ed9d640 1 -- 172.21.15.148:0/3492865150 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100d1c50 con 0x7f06100b92a0 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060d59a640 1 -- 172.21.15.148:0/3492865150 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359465734 0 0) 0x7f06100b97a0 con 0x7f06100b2cb0 2024-04-25T10:04:19.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.760+0000 7f060e59c640 1 --2- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f06100d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.761+0000 7f060ed9d640 1 --2- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100d0e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0608009d90 tx=0x7f060802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd69987c69b7625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.761+0000 7f060d59a640 1 -- 172.21.15.148:0/3492865150 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608007740 con 0x7f06100b92a0 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.761+0000 7f060d59a640 1 -- 172.21.15.148:0/3492865150 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06080078e0 con 0x7f06100b92a0 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.761+0000 7f060d59a640 1 -- 172.21.15.148:0/3492865150 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608007cb0 con 0x7f06100b92a0 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.761+0000 7f0614e9c640 1 -- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 msgr2=0x7f06100d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.761+0000 7f0614e9c640 1 --2- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100d0e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0608009d90 tx=0x7f060802a660 comp rx=0 tx=0).stop 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.762+0000 7f0614e9c640 1 -- 172.21.15.148:0/3492865150 shutdown_connections 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.762+0000 7f0614e9c640 1 --2- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100d0e50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.762+0000 7f0614e9c640 1 --2- 172.21.15.148:0/3492865150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f06100d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.762+0000 7f0614e9c640 1 -- 172.21.15.148:0/3492865150 >> 172.21.15.148:0/3492865150 conn(0x7f0610061a20 msgr2=0x7f0610061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.762+0000 7f0614e9c640 1 -- 172.21.15.148:0/3492865150 shutdown_connections 2024-04-25T10:04:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.762+0000 7f0614e9c640 1 -- 172.21.15.148:0/3492865150 wait complete. 2024-04-25T10:04:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.763+0000 7f0614e9c640 1 Processor -- start 2024-04-25T10:04:19.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.763+0000 7f0614e9c640 1 -- start start 2024-04-25T10:04:19.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 0x7f0610153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f0610158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100b6ab0 con 0x7f06100b2cb0 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100d2210 con 0x7f06100b92a0 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100b6c30 con 0x7f061015dbe0 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 0x7f0610153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 0x7f0610153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57174/0 (socket says 172.21.15.148:57174) 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 -- 172.21.15.148:0/1584616421 learned_addr learned my addr 172.21.15.148:0/1584616421 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060ed9d640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f0610158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060dd9b640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 -- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 msgr2=0x7f0610158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f0610158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 -- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 msgr2=0x7f06100cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 -- 172.21.15.148:0/1584616421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610154340 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060ed9d640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f0610158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060dd9b640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f060e59c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 0x7f0610153e00 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f060000e960 tx=0x7f060000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600017070 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060000ca00 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100cc4f0 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060000cdd0 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.764+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100ccab0 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.765+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100cd1c0 con 0x7f06100b2cb0 2024-04-25T10:04:19.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.766+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f060001a070 con 0x7f06100b2cb0 2024-04-25T10:04:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.766+0000 7f05feffd640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05f006b500 0x7f05f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.767+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f060008a240 con 0x7f06100b2cb0 2024-04-25T10:04:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.767+0000 7f060dd9b640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05f006b500 0x7f05f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.767+0000 7f060dd9b640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05f006b500 0x7f05f006d9c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f061006cde0 tx=0x7f0604009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.771+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0600058f70 con 0x7f06100b2cb0 2024-04-25T10:04:19.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.942+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0610073990 con 0x7f06100b2cb0 2024-04-25T10:04:19.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.943+0000 7f05feffd640 1 -- 172.21.15.148:0/1584616421 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+17086 (secure 0 0 0) 0x7f0600058870 con 0x7f06100b2cb0 2024-04-25T10:04:19.948 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:19.948 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":147,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:05.942649+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:48.480288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:49.199618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.2b","mappings":[{"from":3,"to":0}]},{"pgid":"22.36","mappings":[{"from":3,"to":1}]},{"pgid":"22.3b","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:19.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05f006b500 msgr2=0x7f05f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05f006b500 0x7f05f006d9c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f061006cde0 tx=0x7f0604009040 comp rx=0 tx=0).stop 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 msgr2=0x7f0610153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 0x7f0610153e00 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f060000e960 tx=0x7f060000ee20 comp rx=0 tx=0).stop 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 shutdown_connections 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f05f006b500 0x7f05f006d9c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015dbe0 0x7f0610158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100b92a0 0x7f06100cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 --2- 172.21.15.148:0/1584616421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06100b2cb0 0x7f0610153e00 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 >> 172.21.15.148:0/1584616421 conn(0x7f0610061a20 msgr2=0x7f061015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 shutdown_connections 2024-04-25T10:04:19.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:19.945+0000 7f0614e9c640 1 -- 172.21.15.148:0/1584616421 wait complete. 2024-04-25T10:04:19.962 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:04:20.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 Processor -- start 2024-04-25T10:04:20.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 -- start start 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34040cadb0 0x7f34040cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34040ccfe0 0x7f34040cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34040afc80 con 0x7f340415de80 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040cb240 con 0x7f34040cadb0 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.057+0000 7f340aa9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040ce4d0 con 0x7f34040ccfe0 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34040cadb0 0x7f34040cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f34040cadb0 0x7f34040cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57176/0 (socket says 172.21.15.148:57176) 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 -- 172.21.15.148:0/3458158052 learned_addr learned my addr 172.21.15.148:0/3458158052 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f340903b640 1 --2- 172.21.15.148:0/3458158052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34040ccfe0 0x7f34040cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 -- 172.21.15.148:0/3458158052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34040ccfe0 msgr2=0x7f34040cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 --2- 172.21.15.148:0/3458158052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34040ccfe0 0x7f34040cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.064 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 -- 172.21.15.148:0/3458158052 >> v1:172.21.15.188:6789/0 conn(0x7f340415de80 legacy=0x7f34040cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.058+0000 7f33fbfff640 1 -- 172.21.15.148:0/3458158052 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040ce7b0 con 0x7f34040cadb0 2024-04-25T10:04:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f33fbfff640 1 --2- 172.21.15.148:0/3458158052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040cadb0 0x7f34040cf6c0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009870 tx=0x7f33f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831b75e2f3454cff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f33fb7fe640 1 -- 172.21.15.148:0/3458158052 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4037070 con 0x7f34040cadb0 2024-04-25T10:04:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f33fb7fe640 1 -- 172.21.15.148:0/3458158052 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4032440 con 0x7f34040cadb0 2024-04-25T10:04:20.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f33fb7fe640 1 -- 172.21.15.148:0/3458158052 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4032810 con 0x7f34040cadb0 2024-04-25T10:04:20.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 -- 172.21.15.148:0/3458158052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040cadb0 msgr2=0x7f34040cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3458158052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040cadb0 0x7f34040cf6c0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009870 tx=0x7f33f402a270 comp rx=0 tx=0).stop 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 -- 172.21.15.148:0/3458158052 shutdown_connections 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3458158052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040cadb0 0x7f34040cf6c0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3458158052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34040ccfe0 0x7f34040cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 -- 172.21.15.148:0/3458158052 >> 172.21.15.148:0/3458158052 conn(0x7f34040b2390 msgr2=0x7f34040b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 -- 172.21.15.148:0/3458158052 shutdown_connections 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.059+0000 7f340aa9e640 1 -- 172.21.15.148:0/3458158052 wait complete. 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.060+0000 7f340aa9e640 1 Processor -- start 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.060+0000 7f340aa9e640 1 -- start start 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340aa9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34040cadb0 0x7f34040b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340aa9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 0x7f34040c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340aa9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340415de80 0x7f3404151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340aa9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340415e260 con 0x7f34040ccfe0 2024-04-25T10:04:20.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340aa9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34040cfc00 con 0x7f34040cadb0 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340aa9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040b2eb0 con 0x7f340415de80 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 0x7f34040c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 0x7f34040c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57180/0 (socket says 172.21.15.148:57180) 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f33fbfff640 1 -- 172.21.15.148:0/3899814217 learned_addr learned my addr 172.21.15.148:0/3899814217 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340883a640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34040cadb0 0x7f34040b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.061+0000 7f340903b640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340415de80 0x7f3404151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33fbfff640 1 -- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340415de80 msgr2=0x7f3404151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33fbfff640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340415de80 0x7f3404151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33fbfff640 1 -- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34040cadb0 msgr2=0x7f34040b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33fbfff640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34040cadb0 0x7f34040b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33fbfff640 1 -- 172.21.15.148:0/3899814217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040c85b0 con 0x7f34040ccfe0 2024-04-25T10:04:20.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33fbfff640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 0x7f34040c9e90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f33f402a2b0 tx=0x7f33f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4041020 con 0x7f34040ccfe0 2024-04-25T10:04:20.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f40306c0 con 0x7f34040ccfe0 2024-04-25T10:04:20.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4037040 con 0x7f34040ccfe0 2024-04-25T10:04:20.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34040c8890 con 0x7f34040ccfe0 2024-04-25T10:04:20.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.062+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34040c8e50 con 0x7f34040ccfe0 2024-04-25T10:04:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.063+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3404073a00 con 0x7f34040ccfe0 2024-04-25T10:04:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.064+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33f4044020 con 0x7f34040ccfe0 2024-04-25T10:04:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.064+0000 7f33f97fa640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33f006b500 0x7f33f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.064+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f33f40aaaf0 con 0x7f34040ccfe0 2024-04-25T10:04:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.064+0000 7f340883a640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33f006b500 0x7f33f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.064+0000 7f340883a640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33f006b500 0x7f33f006d9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f34000016e0 tx=0x7f3400009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.068+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33f404b030 con 0x7f34040ccfe0 2024-04-25T10:04:20.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.276+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3404070100 con 0x7f34040ccfe0 2024-04-25T10:04:20.282 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:04:20.277+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:04:20.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.278+0000 7f33f97fa640 1 -- 172.21.15.148:0/3899814217 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7f33f40790b0 con 0x7f34040ccfe0 2024-04-25T10:04:20.283 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.281+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33f006b500 msgr2=0x7f33f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.281+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33f006b500 0x7f33f006d9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f34000016e0 tx=0x7f3400009040 comp rx=0 tx=0).stop 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 msgr2=0x7f34040c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 0x7f34040c9e90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f33f402a2b0 tx=0x7f33f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 shutdown_connections 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f33f006b500 0x7f33f006d9c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340415de80 0x7f3404151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34040ccfe0 0x7f34040c9e90 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 --2- 172.21.15.148:0/3899814217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34040cadb0 0x7f34040b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 >> 172.21.15.148:0/3899814217 conn(0x7f34040b2390 msgr2=0x7f340406f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 shutdown_connections 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.282+0000 7f340aa9e640 1 -- 172.21.15.148:0/3899814217 wait complete. 2024-04-25T10:04:20.300 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:04:20.323 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:04:20.324 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:04:20.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.395+0000 7fba9816a640 1 Processor -- start 2024-04-25T10:04:20.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.396+0000 7fba9816a640 1 -- start start 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.396+0000 7fba9816a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba900ccdd0 0x7fba900cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.396+0000 7fba9816a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba900cdcd0 0x7fba900cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.396+0000 7fba9816a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba900cb190 con 0x7fba9015bb20 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.396+0000 7fba9816a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba900d0450 con 0x7fba900ccdd0 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.396+0000 7fba9816a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba900cb470 con 0x7fba900cdcd0 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba900ccdd0 0x7fba900cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fba900ccdd0 0x7fba900cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57200/0 (socket says 172.21.15.148:57200) 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 -- 172.21.15.148:0/1852165087 learned_addr learned my addr 172.21.15.148:0/1852165087 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba96707640 1 --2- 172.21.15.148:0/1852165087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba900cdcd0 0x7fba900cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 -- 172.21.15.148:0/1852165087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba900cdcd0 msgr2=0x7fba900cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 --2- 172.21.15.148:0/1852165087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba900cdcd0 0x7fba900cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 -- 172.21.15.148:0/1852165087 >> v1:172.21.15.188:6789/0 conn(0x7fba9015bb20 legacy=0x7fba900cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.397+0000 7fba95705640 1 -- 172.21.15.148:0/1852165087 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900cb750 con 0x7fba900ccdd0 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba95705640 1 --2- 172.21.15.148:0/1852165087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900ccdd0 0x7fba900cf9d0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fba80009870 tx=0x7fba8002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d94ca7a50a33e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba94f04640 1 -- 172.21.15.148:0/1852165087 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80037070 con 0x7fba900ccdd0 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba94f04640 1 -- 172.21.15.148:0/1852165087 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba80032440 con 0x7fba900ccdd0 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba94f04640 1 -- 172.21.15.148:0/1852165087 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80032810 con 0x7fba900ccdd0 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba9816a640 1 -- 172.21.15.148:0/1852165087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900ccdd0 msgr2=0x7fba900cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba9816a640 1 --2- 172.21.15.148:0/1852165087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900ccdd0 0x7fba900cf9d0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fba80009870 tx=0x7fba8002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba9816a640 1 -- 172.21.15.148:0/1852165087 shutdown_connections 2024-04-25T10:04:20.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba9816a640 1 --2- 172.21.15.148:0/1852165087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900ccdd0 0x7fba900cf9d0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba9816a640 1 --2- 172.21.15.148:0/1852165087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba900cdcd0 0x7fba900cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.398+0000 7fba9816a640 1 -- 172.21.15.148:0/1852165087 >> 172.21.15.148:0/1852165087 conn(0x7fba900b2390 msgr2=0x7fba900b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.399+0000 7fba9816a640 1 -- 172.21.15.148:0/1852165087 shutdown_connections 2024-04-25T10:04:20.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.399+0000 7fba9816a640 1 -- 172.21.15.148:0/1852165087 wait complete. 2024-04-25T10:04:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 Processor -- start 2024-04-25T10:04:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 -- start start 2024-04-25T10:04:20.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba900ccdd0 0x7fba9014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 0x7fba90149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba9015bb20 0x7fba9014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9015bf00 con 0x7fba900cdcd0 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba900cbd10 con 0x7fba900ccdd0 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba9816a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9015c080 con 0x7fba9015bb20 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba95705640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 0x7fba90149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.400+0000 7fba95705640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 0x7fba90149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57206/0 (socket says 172.21.15.148:57206) 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 -- 172.21.15.148:0/68267422 learned_addr learned my addr 172.21.15.148:0/68267422 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95f06640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba900ccdd0 0x7fba9014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba96707640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba9015bb20 0x7fba9014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 -- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba9015bb20 msgr2=0x7fba9014bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba9015bb20 0x7fba9014bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 -- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba900ccdd0 msgr2=0x7fba9014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba900ccdd0 0x7fba9014f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 -- 172.21.15.148:0/68267422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900b4ee0 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95f06640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba900ccdd0 0x7fba9014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba95705640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 0x7fba90149820 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fba80000c00 tx=0x7fba800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80037040 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba80042070 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba90159e20 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80030880 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.401+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba90156e80 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.402+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba90073a00 con 0x7fba900cdcd0 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.403+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba8002ad10 con 0x7fba900cdcd0 2024-04-25T10:04:20.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.403+0000 7fba867fc640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba7406b4d0 0x7fba7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.403+0000 7fba95f06640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba7406b4d0 0x7fba7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.403+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fba800aa910 con 0x7fba900cdcd0 2024-04-25T10:04:20.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.403+0000 7fba95f06640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba7406b4d0 0x7fba7406d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fba900af5a0 tx=0x7fba88009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.407+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba800795e0 con 0x7fba900cdcd0 2024-04-25T10:04:20.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:20.613+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fba900ae3b0 con 0x7fba900cdcd0 2024-04-25T10:04:21.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.280+0000 7fba867fc640 1 -- 172.21.15.148:0/68267422 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba80078ee0 con 0x7fba900cdcd0 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba7406b4d0 msgr2=0x7fba7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba7406b4d0 0x7fba7406d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fba900af5a0 tx=0x7fba88009040 comp rx=0 tx=0).stop 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 msgr2=0x7fba90149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 0x7fba90149820 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fba80000c00 tx=0x7fba800046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 shutdown_connections 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fba7406b4d0 0x7fba7406d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba9015bb20 0x7fba9014bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba900cdcd0 0x7fba90149820 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 --2- 172.21.15.148:0/68267422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba900ccdd0 0x7fba9014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.282+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 >> 172.21.15.148:0/68267422 conn(0x7fba900b2390 msgr2=0x7fba9015ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.283+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 shutdown_connections 2024-04-25T10:04:21.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.283+0000 7fba9816a640 1 -- 172.21.15.148:0/68267422 wait complete. 2024-04-25T10:04:21.301 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:04:21.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.442+0000 7f601ac9e640 1 Processor -- start 2024-04-25T10:04:21.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.442+0000 7f601ac9e640 1 -- start start 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f601ac9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60141616f0 0x7f6014161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f601ac9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60140ce0a0 0x7f60140ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f601ac9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601406c8d0 con 0x7f60141610f0 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f601ac9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60141614d0 con 0x7f60141616f0 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f601ac9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60140c6960 con 0x7f60140ce0a0 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f6013fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60141616f0 0x7f6014161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f6013fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f60141616f0 0x7f6014161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57214/0 (socket says 172.21.15.148:57214) 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.443+0000 7f6013fff640 1 -- 172.21.15.148:0/2471506707 learned_addr learned my addr 172.21.15.148:0/2471506707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f601923b640 1 --2- 172.21.15.148:0/2471506707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60140ce0a0 0x7f60140ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f6013fff640 1 -- 172.21.15.148:0/2471506707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60140ce0a0 msgr2=0x7f60140ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f6013fff640 1 --2- 172.21.15.148:0/2471506707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60140ce0a0 0x7f60140ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f6013fff640 1 -- 172.21.15.148:0/2471506707 >> v1:172.21.15.188:6789/0 conn(0x7f60141610f0 legacy=0x7f601406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f6013fff640 1 -- 172.21.15.148:0/2471506707 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6014162260 con 0x7f60141616f0 2024-04-25T10:04:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f6013fff640 1 --2- 172.21.15.148:0/2471506707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60141616f0 0x7f6014161b50 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f6008009870 tx=0x7f600802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c690d557830fa02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.444+0000 7f60137fe640 1 -- 172.21.15.148:0/2471506707 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008037070 con 0x7f60141616f0 2024-04-25T10:04:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f60137fe640 1 -- 172.21.15.148:0/2471506707 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6008032440 con 0x7f60141616f0 2024-04-25T10:04:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f60137fe640 1 -- 172.21.15.148:0/2471506707 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008032810 con 0x7f60141616f0 2024-04-25T10:04:21.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 -- 172.21.15.148:0/2471506707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60141616f0 msgr2=0x7f6014161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 --2- 172.21.15.148:0/2471506707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60141616f0 0x7f6014161b50 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f6008009870 tx=0x7f600802a270 comp rx=0 tx=0).stop 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 -- 172.21.15.148:0/2471506707 shutdown_connections 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 --2- 172.21.15.148:0/2471506707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60141616f0 0x7f6014161b50 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 --2- 172.21.15.148:0/2471506707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60140ce0a0 0x7f60140ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 -- 172.21.15.148:0/2471506707 >> 172.21.15.148:0/2471506707 conn(0x7f601406f500 msgr2=0x7f601406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 -- 172.21.15.148:0/2471506707 shutdown_connections 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.445+0000 7f601ac9e640 1 -- 172.21.15.148:0/2471506707 wait complete. 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.446+0000 7f601ac9e640 1 Processor -- start 2024-04-25T10:04:21.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.446+0000 7f601ac9e640 1 -- start start 2024-04-25T10:04:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601ac9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 0x7f601415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601ac9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60141610f0 0x7f6014152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601ac9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60141616f0 0x7f60140b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601ac9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60140cfce0 con 0x7f60140ce0a0 2024-04-25T10:04:21.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601ac9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60140cfa00 con 0x7f60141610f0 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601ac9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60140cf690 con 0x7f60141616f0 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 0x7f601415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 0x7f601415a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57224/0 (socket says 172.21.15.148:57224) 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 -- 172.21.15.148:0/774561553 learned_addr learned my addr 172.21.15.148:0/774561553 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f601923b640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60141616f0 0x7f60140b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6013fff640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60141610f0 0x7f6014152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 -- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60141616f0 msgr2=0x7f60140b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60141616f0 0x7f60140b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 -- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60141610f0 msgr2=0x7f6014152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60141610f0 0x7f6014152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.447+0000 7f6018a3a640 1 -- 172.21.15.148:0/774561553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6014153d90 con 0x7f60140ce0a0 2024-04-25T10:04:21.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.448+0000 7f6018a3a640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 0x7f601415a580 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f600400c400 tx=0x7f600400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.448+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040052d0 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.448+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004005490 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.448+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6014154070 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.448+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004007260 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.448+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6014154630 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.449+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60140b79f0 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.449+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6004007400 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.450+0000 7f60117fa640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f600006b4d0 0x7f600006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.450+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f6004089900 con 0x7f60140ce0a0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.450+0000 7f6013fff640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f600006b4d0 0x7f600006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.450+0000 7f6013fff640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f600006b4d0 0x7f600006d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6008034000 tx=0x7f600802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.454+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60140b79f0 con 0x7f60140ce0a0 2024-04-25T10:04:21.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:21.622+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6014073a00 con 0x7f60140ce0a0 2024-04-25T10:04:22.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.281+0000 7f60117fa640 1 -- 172.21.15.148:0/774561553 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v149) v1 ==== 181+0+0 (secure 0 0 0) 0x7f60040585d0 con 0x7f60140ce0a0 2024-04-25T10:04:22.287 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:04:22.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f600006b4d0 msgr2=0x7f600006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f600006b4d0 0x7f600006d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6008034000 tx=0x7f600802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:22.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 msgr2=0x7f601415a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 0x7f601415a580 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f600400c400 tx=0x7f600400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 shutdown_connections 2024-04-25T10:04:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f600006b4d0 0x7f600006d990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60141616f0 0x7f60140b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60141610f0 0x7f6014152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 --2- 172.21.15.148:0/774561553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60140ce0a0 0x7f601415a580 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.284+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 >> 172.21.15.148:0/774561553 conn(0x7f601406f500 msgr2=0x7f60140b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.285+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 shutdown_connections 2024-04-25T10:04:22.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.285+0000 7f601ac9e640 1 -- 172.21.15.148:0/774561553 wait complete. 2024-04-25T10:04:22.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:04:22.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.404+0000 7f594d24a640 1 Processor -- start 2024-04-25T10:04:22.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.404+0000 7f594d24a640 1 -- start start 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594d24a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f594815dac0 0x7f59480d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594d24a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59480d1f30 0x7f59480d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594d24a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59480d2420 con 0x7f59480d34c0 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594d24a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59480d3030 con 0x7f594815dac0 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594d24a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59480d2700 con 0x7f59480d1f30 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594759e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f59480d34c0 0x7f59480b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42128/0 (socket says 172.21.15.148:42128) 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594759e640 1 -- 172.21.15.148:0/514535198 learned_addr learned my addr 172.21.15.148:0/514535198 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:22.413 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f5946d9d640 1 --2- 172.21.15.148:0/514535198 >> v2:172.21.15.188:3300/0 conn(0x7f594815dac0 0x7f59480d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f594659c640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59480d1f30 0x7f59480d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746902638 0 0) 0x7f59480d2420 con 0x7f59480d34c0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.405+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5930003520 con 0x7f59480d34c0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100925876 0 0) 0x7f5930003520 con 0x7f59480d34c0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59480d1f30 msgr2=0x7f59480d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59480d1f30 0x7f59480d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 >> v2:172.21.15.188:3300/0 conn(0x7f594815dac0 msgr2=0x7f59480d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 --2- 172.21.15.148:0/514535198 >> v2:172.21.15.188:3300/0 conn(0x7f594815dac0 0x7f59480d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480d29e0 con 0x7f59480d34c0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f59400019f0 con 0x7f59480d34c0 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 --2- 172.21.15.148:0/514535198 >> v2:172.21.15.188:3300/0 conn(0x7f594815dac0 0x7f59480d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 >> v1:172.21.15.148:6789/0 conn(0x7f59480d34c0 legacy=0x7f59480b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59300039b0 0x7f5930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5930004800 0x7f5930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300051c0 0x7f593000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59480d3030 con 0x7f59300039b0 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59480d2700 con 0x7f5930004800 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593000bf00 con 0x7f59300051c0 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f594659c640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59480d1f30 0x7f59480d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59300039b0 0x7f5930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f594759e640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300051c0 0x7f593000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f594659c640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5930004800 0x7f5930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 -- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300051c0 msgr2=0x7f593000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300051c0 0x7f593000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 -- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5930004800 msgr2=0x7f5930004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5930004800 0x7f5930004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f5946d9d640 1 -- 172.21.15.148:0/514535198 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5930003520 con 0x7f59300039b0 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.406+0000 7f594659c640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5930004800 0x7f5930004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f5946d9d640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59300039b0 0x7f5930003dd0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f5938009c90 tx=0x7f5930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e192228efbf0e3da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938036070 con 0x7f59300039b0 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5938031400 con 0x7f59300039b0 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f5945d9b640 1 -- 172.21.15.148:0/514535198 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938009860 con 0x7f59300039b0 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594d24a640 1 -- 172.21.15.148:0/514535198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59300039b0 msgr2=0x7f5930003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594d24a640 1 --2- 172.21.15.148:0/514535198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59300039b0 0x7f5930003dd0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f5938009c90 tx=0x7f5930005830 comp rx=0 tx=0).stop 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594759e640 1 -- 172.21.15.148:0/514535198 reap_dead start 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594d24a640 1 -- 172.21.15.148:0/514535198 shutdown_connections 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594d24a640 1 -- 172.21.15.148:0/514535198 >> 172.21.15.148:0/514535198 conn(0x7f5948061c30 msgr2=0x7f594806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594d24a640 1 -- 172.21.15.148:0/514535198 shutdown_connections 2024-04-25T10:04:22.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.407+0000 7f594d24a640 1 -- 172.21.15.148:0/514535198 wait complete. 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.409+0000 7f594d24a640 1 Processor -- start 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.409+0000 7f594d24a640 1 -- start start 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594d24a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300039b0 0x7f59480c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594d24a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 0x7f5948153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594d24a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5948153020 0x7f594815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594d24a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59480b97d0 con 0x7f5948152bc0 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594d24a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594815dea0 con 0x7f5948153020 2024-04-25T10:04:22.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594d24a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59480b9950 con 0x7f59300039b0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 0x7f5948153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 0x7f5948153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57232/0 (socket says 172.21.15.148:57232) 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594659c640 1 -- 172.21.15.148:0/1142124767 learned_addr learned my addr 172.21.15.148:0/1142124767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594759e640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5948153020 0x7f594815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f5946d9d640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300039b0 0x7f59480c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.410+0000 7f594659c640 1 -- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300039b0 msgr2=0x7f59480c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594659c640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300039b0 0x7f59480c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594659c640 1 -- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5948153020 msgr2=0x7f594815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594659c640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5948153020 0x7f594815ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594659c640 1 -- 172.21.15.148:0/1142124767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594815ee90 con 0x7f5948152bc0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594659c640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 0x7f5948153dd0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f593c010930 tx=0x7f593c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00f040 con 0x7f5948152bc0 2024-04-25T10:04:22.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c005b90 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594815f170 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00e8e0 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.411+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594815f730 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.412+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5948073a00 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.412+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f593c00ea80 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.413+0000 7f592f7fe640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f593406b4d0 0x7f593406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.413+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5836+0+0 (secure 0 0 0) 0x7f593c08a590 con 0x7f5948152bc0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.413+0000 7f5946d9d640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f593406b4d0 0x7f593406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.413+0000 7f5946d9d640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f593406b4d0 0x7f593406d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5938033000 tx=0x7f5938008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.417+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f593c018070 con 0x7f5948152bc0 2024-04-25T10:04:22.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:22.587+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f594806d0a0 con 0x7f5948152bc0 2024-04-25T10:04:23.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.283+0000 7f592f7fe640 1 -- 172.21.15.148:0/1142124767 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v150) v1 ==== 169+0+0 (secure 0 0 0) 0x7f593c058e00 con 0x7f5948152bc0 2024-04-25T10:04:23.288 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:04:23.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f593406b4d0 msgr2=0x7f593406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f593406b4d0 0x7f593406d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5938033000 tx=0x7f5938008030 comp rx=0 tx=0).stop 2024-04-25T10:04:23.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 msgr2=0x7f5948153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 0x7f5948153dd0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f593c010930 tx=0x7f593c010d30 comp rx=0 tx=0).stop 2024-04-25T10:04:23.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 shutdown_connections 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f593406b4d0 0x7f593406d990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5948153020 0x7f594815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5948152bc0 0x7f5948153dd0 secure :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f593c010930 tx=0x7f593c010d30 comp rx=0 tx=0).stop 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 --2- 172.21.15.148:0/1142124767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59300039b0 0x7f59480c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.287+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 >> 172.21.15.148:0/1142124767 conn(0x7f5948061c30 msgr2=0x7f59480b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.288+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 shutdown_connections 2024-04-25T10:04:23.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.288+0000 7f594d24a640 1 -- 172.21.15.148:0/1142124767 wait complete. 2024-04-25T10:04:23.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:04:23.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.413+0000 7f77400ee640 1 Processor -- start 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.413+0000 7f77400ee640 1 -- start start 2024-04-25T10:04:23.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f77400ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f773815b7d0 0x7f77380cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f77400ee640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77380b5620 0x7f77380b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f77400ee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773806cf50 con 0x7f773806c2d0 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f77400ee640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77380b5fc0 con 0x7f77380b5620 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f77400ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380ce530 con 0x7f773815b7d0 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f773d689640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f77380b5620 0x7f77380b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f773e68b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f773806c2d0 0x7f773806c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42130/0 (socket says 172.21.15.148:42130) 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.414+0000 7f773e68b640 1 -- 172.21.15.148:0/3403396117 learned_addr learned my addr 172.21.15.148:0/3403396117 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773de8a640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f773815b7d0 0x7f77380cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773d689640 1 -- 172.21.15.148:0/3403396117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f773815b7d0 msgr2=0x7f77380cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773d689640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f773815b7d0 0x7f77380cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773d689640 1 -- 172.21.15.148:0/3403396117 >> v1:172.21.15.148:6789/0 conn(0x7f773806c2d0 legacy=0x7f773806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773ce88640 1 -- 172.21.15.148:0/3403396117 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762527194 0 0) 0x7f773806cf50 con 0x7f773806c2d0 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773d689640 1 -- 172.21.15.148:0/3403396117 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77380ce810 con 0x7f77380b5620 2024-04-25T10:04:23.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773de8a640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f773815b7d0 0x7f77380cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.415+0000 7f773d689640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77380b5620 0x7f77380b5a80 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f77280097a0 tx=0x7f772802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f26e487a866874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f773ce88640 1 -- 172.21.15.148:0/3403396117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728037070 con 0x7f77380b5620 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f773ce88640 1 -- 172.21.15.148:0/3403396117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728032440 con 0x7f77380b5620 2024-04-25T10:04:23.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f773ce88640 1 -- 172.21.15.148:0/3403396117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728032810 con 0x7f77380b5620 2024-04-25T10:04:23.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 -- 172.21.15.148:0/3403396117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77380b5620 msgr2=0x7f77380b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77380b5620 0x7f77380b5a80 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f77280097a0 tx=0x7f772802a180 comp rx=0 tx=0).stop 2024-04-25T10:04:23.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 -- 172.21.15.148:0/3403396117 shutdown_connections 2024-04-25T10:04:23.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f77380b5620 0x7f77380b5a80 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 --2- 172.21.15.148:0/3403396117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f773815b7d0 0x7f77380cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 -- 172.21.15.148:0/3403396117 >> 172.21.15.148:0/3403396117 conn(0x7f77380d0800 msgr2=0x7f77380d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 -- 172.21.15.148:0/3403396117 shutdown_connections 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.416+0000 7f77400ee640 1 -- 172.21.15.148:0/3403396117 wait complete. 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.417+0000 7f77400ee640 1 Processor -- start 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.417+0000 7f77400ee640 1 -- start start 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f77400ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f773806c2d0 0x7f7738150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f77400ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77380b5620 0x7f773815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f77400ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 0x7f77380b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f77400ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77380cf550 con 0x7f773815b7d0 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f77400ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773806d230 con 0x7f773806c2d0 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f77400ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77380cf6d0 con 0x7f77380b5620 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f773e68b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 0x7f77380b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f773e68b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 0x7f77380b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:57252/0 (socket says 172.21.15.148:57252) 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f773e68b640 1 -- 172.21.15.148:0/1020252610 learned_addr learned my addr 172.21.15.148:0/1020252610 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f773d689640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77380b5620 0x7f773815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.418+0000 7f773de8a640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f773806c2d0 0x7f7738150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f773e68b640 1 -- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77380b5620 msgr2=0x7f773815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f773e68b640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77380b5620 0x7f773815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f773e68b640 1 -- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f773806c2d0 msgr2=0x7f7738150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f773e68b640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f773806c2d0 0x7f7738150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f773e68b640 1 -- 172.21.15.148:0/1020252610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77381f4520 con 0x7f773815b7d0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f773e68b640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 0x7f77380b8050 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f773000c800 tx=0x7f773000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730018070 con 0x7f773815b7d0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7730004d80 con 0x7f773815b7d0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77381f4800 con 0x7f773815b7d0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000a880 con 0x7f773815b7d0 2024-04-25T10:04:23.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.419+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77381f4dc0 con 0x7f773815b7d0 2024-04-25T10:04:23.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.420+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7738073a00 con 0x7f773815b7d0 2024-04-25T10:04:23.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.420+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f773001b070 con 0x7f773815b7d0 2024-04-25T10:04:23.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.421+0000 7f772e7fc640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f771c06b4d0 0x7f771c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.427 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.421+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7730089560 con 0x7f773815b7d0 2024-04-25T10:04:23.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.421+0000 7f773de8a640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f771c06b4d0 0x7f771c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.421+0000 7f773de8a640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f771c06b4d0 0x7f771c06d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f773815ab10 tx=0x7f7734008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.425+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7730057fc0 con 0x7f773815b7d0 2024-04-25T10:04:23.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.597+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f773806b210 con 0x7f773815b7d0 2024-04-25T10:04:23.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.600+0000 7f772e7fc640 1 -- 172.21.15.148:0/1020252610 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f773005c060 con 0x7f773815b7d0 2024-04-25T10:04:23.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f771c06b4d0 msgr2=0x7f771c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f771c06b4d0 0x7f771c06d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f773815ab10 tx=0x7f7734008040 comp rx=0 tx=0).stop 2024-04-25T10:04:23.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 msgr2=0x7f77380b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 0x7f77380b8050 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f773000c800 tx=0x7f773000ccc0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 shutdown_connections 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f771c06b4d0 0x7f771c06d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f773815b7d0 0x7f77380b8050 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77380b5620 0x7f773815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 --2- 172.21.15.148:0/1020252610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f773806c2d0 0x7f7738150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 >> 172.21.15.148:0/1020252610 conn(0x7f77380d0800 msgr2=0x7f77380d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 shutdown_connections 2024-04-25T10:04:23.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.603+0000 7f77400ee640 1 -- 172.21.15.148:0/1020252610 wait complete. 2024-04-25T10:04:23.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:04:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.712+0000 7fa8c7493640 1 Processor -- start 2024-04-25T10:04:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.712+0000 7fa8c7493640 1 -- start start 2024-04-25T10:04:23.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c7493640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8c00b2cb0 0x7fa8c00d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c7493640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00b97a0 con 0x7fa8c00d1020 2024-04-25T10:04:23.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c7493640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00d1c50 con 0x7fa8c015dbe0 2024-04-25T10:04:23.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c7493640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c015dfc0 con 0x7fa8c00b2cb0 2024-04-25T10:04:23.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c5a30640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa8c00d1020 0x7fa8c00d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49452/0 (socket says 172.21.15.148:49452) 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c5a30640 1 -- 172.21.15.148:0/2433629746 learned_addr learned my addr 172.21.15.148:0/2433629746 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.713+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/2433629746 >> v2:172.21.15.188:3300/0 conn(0x7fa8c00b2cb0 0x7fa8c00d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256715895 0 0) 0x7fa8c00b97a0 con 0x7fa8c00d1020 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8ac003520 con 0x7fa8c00d1020 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889904939 0 0) 0x7fa8ac003520 con 0x7fa8c00d1020 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 >> v2:172.21.15.188:3300/0 conn(0x7fa8c00b2cb0 msgr2=0x7fa8c00d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 --2- 172.21.15.148:0/2433629746 >> v2:172.21.15.188:3300/0 conn(0x7fa8c00b2cb0 0x7fa8c00d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 >> v1:172.21.15.188:6789/0 conn(0x7fa8c015dbe0 legacy=0x7fa8c00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00d1f30 con 0x7fa8c00d1020 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa8b8001670 con 0x7fa8c00d1020 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 >> v1:172.21.15.148:6789/0 conn(0x7fa8c00d1020 legacy=0x7fa8c00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac0039b0 0x7fa8ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ac0051c0 0x7fa8ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac00c030 con 0x7fa8ac004800 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac00c1e0 con 0x7fa8ac0051c0 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac00c3e0 con 0x7fa8ac0039b0 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c5a30640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ac0051c0 0x7fa8ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c522f640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac0039b0 0x7fa8ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac0039b0 msgr2=0x7fa8ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac0039b0 0x7fa8ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ac0051c0 msgr2=0x7fa8ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ac0051c0 0x7fa8ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c5a30640 1 -- 172.21.15.148:0/2433629746 reap_dead start 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/2433629746 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac003520 con 0x7fa8ac004800 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.714+0000 7fa8c522f640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac0039b0 0x7fa8ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.715+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c80 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009a90 tx=0x7fa8ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151140246acf8f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.715+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0036070 con 0x7fa8ac004800 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.715+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0031400 con 0x7fa8ac004800 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.715+0000 7fa8b7fff640 1 -- 172.21.15.148:0/2433629746 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0009690 con 0x7fa8ac004800 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 -- 172.21.15.148:0/2433629746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac004800 msgr2=0x7fa8ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c80 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009a90 tx=0x7fa8ac005830 comp rx=0 tx=0).stop 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 -- 172.21.15.148:0/2433629746 shutdown_connections 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 --2- 172.21.15.148:0/2433629746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8ac004800 0x7fa8ac004c80 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 -- 172.21.15.148:0/2433629746 >> 172.21.15.148:0/2433629746 conn(0x7fa8c0061a20 msgr2=0x7fa8c0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 -- 172.21.15.148:0/2433629746 shutdown_connections 2024-04-25T10:04:23.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.716+0000 7fa8c7493640 1 -- 172.21.15.148:0/2433629746 wait complete. 2024-04-25T10:04:23.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.717+0000 7fa8c7493640 1 Processor -- start 2024-04-25T10:04:23.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.717+0000 7fa8c7493640 1 -- start start 2024-04-25T10:04:23.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac004800 0x7fa8c00cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 0x7fa8c01f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c00b8320 0x7fa8c00b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00b92a0 con 0x7fa8c00caa50 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00d1a00 con 0x7fa8c00b8320 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00b9420 con 0x7fa8ac004800 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 0x7fa8c01f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 0x7fa8c01f6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46036/0 (socket says 172.21.15.148:46036) 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/3808978119 learned_addr learned my addr 172.21.15.148:0/3808978119 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c522f640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac004800 0x7fa8c00cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c5a30640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c00b8320 0x7fa8c00b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac004800 msgr2=0x7fa8c00cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac004800 0x7fa8c00cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c00b8320 msgr2=0x7fa8c00b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c00b8320 0x7fa8c00b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 -- 172.21.15.148:0/3808978119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00cb230 con 0x7fa8c00caa50 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c522f640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac004800 0x7fa8c00cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c5a30640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c00b8320 0x7fa8c00b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c4a2e640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 0x7fa8c01f6470 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009bb0 tx=0x7fa8b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b003f020 con 0x7fa8c00caa50 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0037040 con 0x7fa8c00caa50 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c00cb510 con 0x7fa8c00caa50 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.718+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0036070 con 0x7fa8c00caa50 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.719+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c00cbad0 con 0x7fa8c00caa50 2024-04-25T10:04:23.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.719+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c0073990 con 0x7fa8c00caa50 2024-04-25T10:04:23.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.720+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8b0042020 con 0x7fa8c00caa50 2024-04-25T10:04:23.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.720+0000 7fa8b5ffb640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8a406b4d0 0x7fa8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.720+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa8b00aa120 con 0x7fa8c00caa50 2024-04-25T10:04:23.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.720+0000 7fa8c522f640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8a406b4d0 0x7fa8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.721+0000 7fa8c522f640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8a406b4d0 0x7fa8a406d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc007fd0 tx=0x7fa8bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.724+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8b0049030 con 0x7fa8c00caa50 2024-04-25T10:04:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:23.959+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7fa8c0067f30 con 0x7fa8c00caa50 2024-04-25T10:04:24.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.284+0000 7fa8b5ffb640 1 -- 172.21.15.148:0/3808978119 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v493) v1 ==== 159+0+0 (secure 0 0 0) 0x7fa8b00791d0 con 0x7fa8c00caa50 2024-04-25T10:04:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8a406b4d0 msgr2=0x7fa8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8a406b4d0 0x7fa8a406d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc007fd0 tx=0x7fa8bc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 msgr2=0x7fa8c01f6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 0x7fa8c01f6470 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009bb0 tx=0x7fa8b0004730 comp rx=0 tx=0).stop 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 shutdown_connections 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8a406b4d0 0x7fa8a406d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c00b8320 0x7fa8c00b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c00caa50 0x7fa8c01f6470 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 --2- 172.21.15.148:0/3808978119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ac004800 0x7fa8c00cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 >> 172.21.15.148:0/3808978119 conn(0x7fa8c0061a20 msgr2=0x7fa8c00b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 shutdown_connections 2024-04-25T10:04:24.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.286+0000 7fa8c7493640 1 -- 172.21.15.148:0/3808978119 wait complete. 2024-04-25T10:04:24.303 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:04:24.303 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:04:24.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:24.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.423+0000 7f52963e7640 1 Processor -- start 2024-04-25T10:04:24.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.423+0000 7f52963e7640 1 -- start start 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f52963e7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f529015dbe0 0x7f52900d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f52963e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52900b92a0 0x7f52900d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f52963e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900b97a0 con 0x7f52900b2cb0 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f52963e7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529015dfc0 con 0x7f529015dbe0 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f52963e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52900d1a00 con 0x7f52900b92a0 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f529015dbe0 0x7f52900d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f529015dbe0 0x7f52900d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46044/0 (socket says 172.21.15.148:46044) 2024-04-25T10:04:24.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 -- 172.21.15.148:0/4235613573 learned_addr learned my addr 172.21.15.148:0/4235613573 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f5294984640 1 --2- 172.21.15.148:0/4235613573 >> v2:172.21.15.188:3300/0 conn(0x7f52900b92a0 0x7f52900d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 -- 172.21.15.148:0/4235613573 >> v2:172.21.15.188:3300/0 conn(0x7f52900b92a0 msgr2=0x7f52900d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 --2- 172.21.15.148:0/4235613573 >> v2:172.21.15.188:3300/0 conn(0x7f52900b92a0 0x7f52900d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 -- 172.21.15.148:0/4235613573 >> v1:172.21.15.188:6789/0 conn(0x7f52900b2cb0 legacy=0x7f52900b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f528ffff640 1 -- 172.21.15.148:0/4235613573 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900d1c50 con 0x7f529015dbe0 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.424+0000 7f5294984640 1 --2- 172.21.15.148:0/4235613573 >> v2:172.21.15.188:3300/0 conn(0x7f52900b92a0 0x7f52900d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.425+0000 7f528ffff640 1 --2- 172.21.15.148:0/4235613573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529015dbe0 0x7f52900d0910 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f52900cfb90 tx=0x7f528002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd42924c464432f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.425+0000 7f528effd640 1 -- 172.21.15.148:0/4235613573 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280037070 con 0x7f529015dbe0 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.425+0000 7f528effd640 1 -- 172.21.15.148:0/4235613573 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280032440 con 0x7f529015dbe0 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.425+0000 7f528effd640 1 -- 172.21.15.148:0/4235613573 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280032810 con 0x7f529015dbe0 2024-04-25T10:04:24.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.425+0000 7f52963e7640 1 -- 172.21.15.148:0/4235613573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529015dbe0 msgr2=0x7f52900d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.425+0000 7f52963e7640 1 --2- 172.21.15.148:0/4235613573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529015dbe0 0x7f52900d0910 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f52900cfb90 tx=0x7f528002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.426+0000 7f52963e7640 1 -- 172.21.15.148:0/4235613573 shutdown_connections 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.426+0000 7f52963e7640 1 --2- 172.21.15.148:0/4235613573 >> v2:172.21.15.188:3300/0 conn(0x7f52900b92a0 0x7f52900d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.426+0000 7f52963e7640 1 --2- 172.21.15.148:0/4235613573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f529015dbe0 0x7f52900d0910 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.426+0000 7f52963e7640 1 -- 172.21.15.148:0/4235613573 >> 172.21.15.148:0/4235613573 conn(0x7f5290061a20 msgr2=0x7f5290061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.426+0000 7f52963e7640 1 -- 172.21.15.148:0/4235613573 shutdown_connections 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.426+0000 7f52963e7640 1 -- 172.21.15.148:0/4235613573 wait complete. 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.427+0000 7f52963e7640 1 Processor -- start 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.427+0000 7f52963e7640 1 -- start start 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52900b2cb0 0x7f52901f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 0x7f52900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529015dbe0 0x7f52901f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900b6ab0 con 0x7f52900b92a0 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900d2210 con 0x7f529015dbe0 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900b6c30 con 0x7f52900b2cb0 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 0x7f52900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 0x7f52900b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46050/0 (socket says 172.21.15.148:46050) 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 -- 172.21.15.148:0/2595582626 learned_addr learned my addr 172.21.15.148:0/2595582626 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528ffff640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52900b2cb0 0x7f52901f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f5294984640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529015dbe0 0x7f52901f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 -- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52900b2cb0 msgr2=0x7f52901f0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52900b2cb0 0x7f52901f0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 -- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529015dbe0 msgr2=0x7f52901f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529015dbe0 0x7f52901f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 -- 172.21.15.148:0/2595582626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52901f6f40 con 0x7f52900b92a0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528f7fe640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 0x7f52900b4e50 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f528400c400 tx=0x7f528400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284011790 con 0x7f52900b92a0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284017070 con 0x7f52900b92a0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52901f7220 con 0x7f52900b92a0 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284011bb0 con 0x7f52900b92a0 2024-04-25T10:04:24.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.428+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52901f77e0 con 0x7f52900b92a0 2024-04-25T10:04:24.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.429+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5290073990 con 0x7f52900b92a0 2024-04-25T10:04:24.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.430+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f528401c460 con 0x7f52900b92a0 2024-04-25T10:04:24.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.430+0000 7f528cff9640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f527c06b4d0 0x7f527c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.430+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5284089f00 con 0x7f52900b92a0 2024-04-25T10:04:24.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.430+0000 7f528ffff640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f527c06b4d0 0x7f527c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.431+0000 7f528ffff640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f527c06b4d0 0x7f527c06d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5280034000 tx=0x7f528002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.434+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5284058f80 con 0x7f52900b92a0 2024-04-25T10:04:24.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.605+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5290160b60 con 0x7f52900b92a0 2024-04-25T10:04:24.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.606+0000 7f528cff9640 1 -- 172.21.15.148:0/2595582626 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+13353 (secure 0 0 0) 0x7f5284058880 con 0x7f52900b92a0 2024-04-25T10:04:24.611 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:24.612 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":150,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:23.282443+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/3188696943":"2024-04-26T10:04:20.277764+0000","172.21.15.148:6836/3188696943":"2024-04-26T10:04:20.277764+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f527c06b4d0 msgr2=0x7f527c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f527c06b4d0 0x7f527c06d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5280034000 tx=0x7f528002e000 comp rx=0 tx=0).stop 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 msgr2=0x7f52900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 0x7f52900b4e50 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f528400c400 tx=0x7f528400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 shutdown_connections 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f527c06b4d0 0x7f527c06d990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f529015dbe0 0x7f52901f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52900b92a0 0x7f52900b4e50 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 --2- 172.21.15.148:0/2595582626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52900b2cb0 0x7f52901f0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 >> 172.21.15.148:0/2595582626 conn(0x7f5290061a20 msgr2=0x7f5290152840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 shutdown_connections 2024-04-25T10:04:24.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.608+0000 7f52963e7640 1 -- 172.21.15.148:0/2595582626 wait complete. 2024-04-25T10:04:24.625 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:24.625 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:24.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.720+0000 7fc863ed8640 1 Processor -- start 2024-04-25T10:04:24.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc863ed8640 1 -- start start 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc863ed8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc85c161240 0x7fc85c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc863ed8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc85c161890 0x7fc85c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc863ed8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c161620 con 0x7fc85c0b54a0 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc863ed8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0b7120 con 0x7fc85c161240 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc863ed8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c161e00 con 0x7fc85c161890 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc861c74640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc85c161240 0x7fc85c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc861c74640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc85c161240 0x7fc85c0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46064/0 (socket says 172.21.15.148:46064) 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.721+0000 7fc861c74640 1 -- 172.21.15.148:0/3413888677 learned_addr learned my addr 172.21.15.148:0/3413888677 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc861473640 1 --2- 172.21.15.148:0/3413888677 >> v2:172.21.15.188:3300/0 conn(0x7fc85c161890 0x7fc85c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc861c74640 1 -- 172.21.15.148:0/3413888677 >> v2:172.21.15.188:3300/0 conn(0x7fc85c161890 msgr2=0x7fc85c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc861c74640 1 --2- 172.21.15.148:0/3413888677 >> v2:172.21.15.188:3300/0 conn(0x7fc85c161890 0x7fc85c0ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc861c74640 1 -- 172.21.15.148:0/3413888677 >> v1:172.21.15.188:6789/0 conn(0x7fc85c0b54a0 legacy=0x7fc85c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc861c74640 1 -- 172.21.15.148:0/3413888677 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c1620e0 con 0x7fc85c161240 2024-04-25T10:04:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc861c74640 1 --2- 172.21.15.148:0/3413888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c0b6be0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fc85c08c330 tx=0x7fc85802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d058a4c5960d7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.722+0000 7fc860c72640 1 -- 172.21.15.148:0/3413888677 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858036070 con 0x7fc85c161240 2024-04-25T10:04:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc860c72640 1 -- 172.21.15.148:0/3413888677 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858031600 con 0x7fc85c161240 2024-04-25T10:04:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc860c72640 1 -- 172.21.15.148:0/3413888677 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8580319d0 con 0x7fc85c161240 2024-04-25T10:04:24.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 -- 172.21.15.148:0/3413888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 msgr2=0x7fc85c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 --2- 172.21.15.148:0/3413888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c0b6be0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fc85c08c330 tx=0x7fc85802a730 comp rx=0 tx=0).stop 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 -- 172.21.15.148:0/3413888677 shutdown_connections 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 --2- 172.21.15.148:0/3413888677 >> v2:172.21.15.188:3300/0 conn(0x7fc85c161890 0x7fc85c0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 --2- 172.21.15.148:0/3413888677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c0b6be0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 -- 172.21.15.148:0/3413888677 >> 172.21.15.148:0/3413888677 conn(0x7fc85c0d04a0 msgr2=0x7fc85c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 -- 172.21.15.148:0/3413888677 shutdown_connections 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.723+0000 7fc863ed8640 1 -- 172.21.15.148:0/3413888677 wait complete. 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.724+0000 7fc863ed8640 1 Processor -- start 2024-04-25T10:04:24.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.724+0000 7fc863ed8640 1 -- start start 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc863ed8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0b54a0 0x7fc85c1e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc863ed8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc863ed8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161890 0x7fc85c0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc863ed8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0d15f0 con 0x7fc85c161240 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc863ed8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1626a0 con 0x7fc85c161890 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc863ed8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0cf000 con 0x7fc85c0b54a0 2024-04-25T10:04:24.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46076/0 (socket says 172.21.15.148:46076) 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 -- 172.21.15.148:0/1938695727 learned_addr learned my addr 172.21.15.148:0/1938695727 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc862475640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161890 0x7fc85c0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861c74640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0b54a0 0x7fc85c1e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 -- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0b54a0 msgr2=0x7fc85c1e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0b54a0 0x7fc85c1e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 -- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161890 msgr2=0x7fc85c0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161890 0x7fc85c0ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.725+0000 7fc861473640 1 -- 172.21.15.148:0/1938695727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0cd1b0 con 0x7fc85c161240 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.726+0000 7fc861473640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c151ce0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00d5f0 tx=0x7fc84c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.726+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0125f0 con 0x7fc85c161240 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.726+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c019070 con 0x7fc85c161240 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.726+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c1e8660 con 0x7fc85c161240 2024-04-25T10:04:24.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.726+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0129d0 con 0x7fc85c161240 2024-04-25T10:04:24.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.726+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1e8c20 con 0x7fc85c161240 2024-04-25T10:04:24.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.727+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c073a00 con 0x7fc85c161240 2024-04-25T10:04:24.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.727+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc84c012b70 con 0x7fc85c161240 2024-04-25T10:04:24.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.728+0000 7fc8527fc640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.728+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc84c08a300 con 0x7fc85c161240 2024-04-25T10:04:24.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.728+0000 7fc861c74640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.728+0000 7fc861c74640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc858033000 tx=0x7fc858009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.732+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc84c058ca0 con 0x7fc85c161240 2024-04-25T10:04:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.902+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc85c06d240 con 0x7fc85c161240 2024-04-25T10:04:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.903+0000 7fc8527fc640 1 -- 172.21.15.148:0/1938695727 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc84c05cd40 con 0x7fc85c161240 2024-04-25T10:04:24.908 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:04:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 msgr2=0x7fc84006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc858033000 tx=0x7fc858009000 comp rx=0 tx=0).stop 2024-04-25T10:04:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 msgr2=0x7fc85c151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c151ce0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00d5f0 tx=0x7fc84c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 shutdown_connections 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c161890 0x7fc85c0ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c161240 0x7fc85c151ce0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 --2- 172.21.15.148:0/1938695727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0b54a0 0x7fc85c1e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 >> 172.21.15.148:0/1938695727 conn(0x7fc85c0d04a0 msgr2=0x7fc85c0b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 shutdown_connections 2024-04-25T10:04:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:24.905+0000 7fc863ed8640 1 -- 172.21.15.148:0/1938695727 wait complete. 2024-04-25T10:04:24.921 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:04:25.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.013+0000 7fd44642a640 1 Processor -- start 2024-04-25T10:04:25.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.013+0000 7fd44642a640 1 -- start start 2024-04-25T10:04:25.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd44642a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd44015bd30 0x7fd4400cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd44642a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400cfe40 0x7fd4400d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd44642a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4400d1930 0x7fd4400d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd44642a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4400d0350 con 0x7fd44015bd30 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd44642a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4400d0630 con 0x7fd4400d1930 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd44642a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400d0910 con 0x7fd4400cfe40 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd43ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd44015bd30 0x7fd4400cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd43ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd44015bd30 0x7fd4400cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46080/0 (socket says 172.21.15.148:46080) 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd43ffff640 1 -- 172.21.15.148:0/1696790439 learned_addr learned my addr 172.21.15.148:0/1696790439 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd4449c7640 1 --2- 172.21.15.148:0/1696790439 >> v2:172.21.15.188:3300/0 conn(0x7fd4400d1930 0x7fd4400d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.014+0000 7fd43f7fe640 1 --2- 172.21.15.148:0/1696790439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400cfe40 0x7fd4400d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43ffff640 1 -- 172.21.15.148:0/1696790439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400cfe40 msgr2=0x7fd4400d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43ffff640 1 --2- 172.21.15.148:0/1696790439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400cfe40 0x7fd4400d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43ffff640 1 -- 172.21.15.148:0/1696790439 >> v2:172.21.15.188:3300/0 conn(0x7fd4400d1930 msgr2=0x7fd4400d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43ffff640 1 --2- 172.21.15.148:0/1696790439 >> v2:172.21.15.188:3300/0 conn(0x7fd4400d1930 0x7fd4400d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43ffff640 1 -- 172.21.15.148:0/1696790439 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400d0bf0 con 0x7fd44015bd30 2024-04-25T10:04:25.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43ffff640 1 --2- 172.21.15.148:0/1696790439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd44015bd30 0x7fd4400cf900 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fd44015f7f0 tx=0x7fd43002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1856b6bbe89360f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43effd640 1 -- 172.21.15.148:0/1696790439 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430037070 con 0x7fd44015bd30 2024-04-25T10:04:25.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43effd640 1 -- 172.21.15.148:0/1696790439 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430032440 con 0x7fd44015bd30 2024-04-25T10:04:25.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.015+0000 7fd43effd640 1 -- 172.21.15.148:0/1696790439 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430032810 con 0x7fd44015bd30 2024-04-25T10:04:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 -- 172.21.15.148:0/1696790439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd44015bd30 msgr2=0x7fd4400cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 --2- 172.21.15.148:0/1696790439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd44015bd30 0x7fd4400cf900 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fd44015f7f0 tx=0x7fd43002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 -- 172.21.15.148:0/1696790439 shutdown_connections 2024-04-25T10:04:25.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 --2- 172.21.15.148:0/1696790439 >> v2:172.21.15.188:3300/0 conn(0x7fd4400d1930 0x7fd4400d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 --2- 172.21.15.148:0/1696790439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400cfe40 0x7fd4400d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 --2- 172.21.15.148:0/1696790439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd44015bd30 0x7fd4400cf900 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 -- 172.21.15.148:0/1696790439 >> 172.21.15.148:0/1696790439 conn(0x7fd44006f590 msgr2=0x7fd44006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 -- 172.21.15.148:0/1696790439 shutdown_connections 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.016+0000 7fd44642a640 1 -- 172.21.15.148:0/1696790439 wait complete. 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.017+0000 7fd44642a640 1 Processor -- start 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.017+0000 7fd44642a640 1 -- start start 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd44642a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 0x7fd4400c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd44642a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400d1930 0x7fd4400cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd44642a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd44015bd30 0x7fd44006ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd44642a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400b73b0 con 0x7fd4400cfe40 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd44642a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44015c110 con 0x7fd44015bd30 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd44642a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400b7530 con 0x7fd4400d1930 2024-04-25T10:04:25.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 0x7fd4400c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 0x7fd4400c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46084/0 (socket says 172.21.15.148:46084) 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43ffff640 1 -- 172.21.15.148:0/1702894887 learned_addr learned my addr 172.21.15.148:0/1702894887 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd4449c7640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd44015bd30 0x7fd44006ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43f7fe640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400d1930 0x7fd4400cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43ffff640 1 -- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400d1930 msgr2=0x7fd4400cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43ffff640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400d1930 0x7fd4400cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.018+0000 7fd43ffff640 1 -- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd44015bd30 msgr2=0x7fd44006ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43ffff640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd44015bd30 0x7fd44006ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43ffff640 1 -- 172.21.15.148:0/1702894887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44006bd90 con 0x7fd4400cfe40 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd4449c7640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd44015bd30 0x7fd44006ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43f7fe640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400d1930 0x7fd4400cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43ffff640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 0x7fd4400c7490 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fd43002a220 tx=0x7fd4300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430037040 con 0x7fd4400cfe40 2024-04-25T10:04:25.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430042070 con 0x7fd4400cfe40 2024-04-25T10:04:25.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44006c070 con 0x7fd4400cfe40 2024-04-25T10:04:25.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430030880 con 0x7fd4400cfe40 2024-04-25T10:04:25.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.019+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44006c630 con 0x7fd4400cfe40 2024-04-25T10:04:25.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.020+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd440073990 con 0x7fd4400cfe40 2024-04-25T10:04:25.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.020+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd430030a20 con 0x7fd4400cfe40 2024-04-25T10:04:25.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.021+0000 7fd43cff9640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd42c06b4d0 0x7fd42c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.021+0000 7fd43f7fe640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd42c06b4d0 0x7fd42c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.021+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd4300aa8d0 con 0x7fd4400cfe40 2024-04-25T10:04:25.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.021+0000 7fd43f7fe640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd42c06b4d0 0x7fd42c06d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd434006fd0 tx=0x7fd434008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.025+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4300799e0 con 0x7fd4400cfe40 2024-04-25T10:04:25.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.262+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7fd4400b2730 con 0x7fd4400cfe40 2024-04-25T10:04:25.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.288+0000 7fd43cff9640 1 -- 172.21.15.148:0/1702894887 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v494) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd4300792e0 con 0x7fd4400cfe40 2024-04-25T10:04:25.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd42c06b4d0 msgr2=0x7fd42c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd42c06b4d0 0x7fd42c06d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd434006fd0 tx=0x7fd434008040 comp rx=0 tx=0).stop 2024-04-25T10:04:25.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 msgr2=0x7fd4400c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 0x7fd4400c7490 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fd43002a220 tx=0x7fd4300046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 shutdown_connections 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd42c06b4d0 0x7fd42c06d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd44015bd30 0x7fd44006ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400d1930 0x7fd4400cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 --2- 172.21.15.148:0/1702894887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4400cfe40 0x7fd4400c7490 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 >> 172.21.15.148:0/1702894887 conn(0x7fd44006f590 msgr2=0x7fd4400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 shutdown_connections 2024-04-25T10:04:25.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.291+0000 7fd44642a640 1 -- 172.21.15.148:0/1702894887 wait complete. 2024-04-25T10:04:25.307 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:04:25.369 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:25.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:25.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.404+0000 7f734bf34640 1 Processor -- start 2024-04-25T10:04:25.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.404+0000 7f734bf34640 1 -- start start 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734bf34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f73440c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734bf34640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73440ca5a0 0x7f734406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734bf34640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f734406c130 0x7f73440cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734bf34640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73440cfa70 con 0x7f73440ca5a0 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734bf34640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73440cc2e0 con 0x7f734406c130 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734bf34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73440cc5c0 con 0x7f734415aa10 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73440ca5a0 0x7f734406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f73440ca5a0 0x7f734406bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46096/0 (socket says 172.21.15.148:46096) 2024-04-25T10:04:25.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 -- 172.21.15.148:0/3656592322 learned_addr learned my addr 172.21.15.148:0/3656592322 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f7349cd0640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f73440c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734a4d1640 1 --2- 172.21.15.148:0/3656592322 >> v2:172.21.15.188:3300/0 conn(0x7f734406c130 0x7f73440cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 -- 172.21.15.148:0/3656592322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 msgr2=0x7f73440c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f73440c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 -- 172.21.15.148:0/3656592322 >> v2:172.21.15.188:3300/0 conn(0x7f734406c130 msgr2=0x7f73440cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 --2- 172.21.15.148:0/3656592322 >> v2:172.21.15.188:3300/0 conn(0x7f734406c130 0x7f73440cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f73494cf640 1 -- 172.21.15.148:0/3656592322 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440cf420 con 0x7f73440ca5a0 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f734a4d1640 1 --2- 172.21.15.148:0/3656592322 >> v2:172.21.15.188:3300/0 conn(0x7f734406c130 0x7f73440cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.405+0000 7f7349cd0640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f73440c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f73494cf640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73440ca5a0 0x7f734406bbf0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f73440d0f30 tx=0x7f733402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c72ae14a852a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f7348cce640 1 -- 172.21.15.148:0/3656592322 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334037070 con 0x7f73440ca5a0 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f7348cce640 1 -- 172.21.15.148:0/3656592322 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334032440 con 0x7f73440ca5a0 2024-04-25T10:04:25.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f7348cce640 1 -- 172.21.15.148:0/3656592322 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334032810 con 0x7f73440ca5a0 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 -- 172.21.15.148:0/3656592322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73440ca5a0 msgr2=0x7f734406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73440ca5a0 0x7f734406bbf0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f73440d0f30 tx=0x7f733402a140 comp rx=0 tx=0).stop 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 -- 172.21.15.148:0/3656592322 shutdown_connections 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 --2- 172.21.15.148:0/3656592322 >> v2:172.21.15.188:3300/0 conn(0x7f734406c130 0x7f73440cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73440ca5a0 0x7f734406bbf0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 --2- 172.21.15.148:0/3656592322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f73440c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.406+0000 7f734bf34640 1 -- 172.21.15.148:0/3656592322 >> 172.21.15.148:0/3656592322 conn(0x7f7344071af0 msgr2=0x7f7344071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.407+0000 7f734bf34640 1 -- 172.21.15.148:0/3656592322 shutdown_connections 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.407+0000 7f734bf34640 1 -- 172.21.15.148:0/3656592322 wait complete. 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 Processor -- start 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 -- start start 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 0x7f73440c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73440ca5a0 0x7f73440c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f734409bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73440cbff0 con 0x7f734406c130 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73440cbe70 con 0x7f73440ca5a0 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f734bf34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73440cdf60 con 0x7f734415aa10 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f7349cd0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 0x7f73440c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f7349cd0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 0x7f73440c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46098/0 (socket says 172.21.15.148:46098) 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.408+0000 7f7349cd0640 1 -- 172.21.15.148:0/1241832665 learned_addr learned my addr 172.21.15.148:0/1241832665 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f734a4d1640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f734409bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f73494cf640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73440ca5a0 0x7f73440c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f7349cd0640 1 -- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 msgr2=0x7f734409bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f7349cd0640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f734409bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f7349cd0640 1 -- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73440ca5a0 msgr2=0x7f73440c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f7349cd0640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73440ca5a0 0x7f73440c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f7349cd0640 1 -- 172.21.15.148:0/1241832665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440c99c0 con 0x7f734406c130 2024-04-25T10:04:25.418 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f73494cf640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73440ca5a0 0x7f73440c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f7349cd0640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 0x7f73440c5710 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f734000c820 tx=0x7f734000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340015070 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7340007a30 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73440c83e0 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340007de0 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.409+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73440c89a0 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.410+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7344073a00 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.410+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7340018070 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.411+0000 7f733a7fc640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.411+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7340089eb0 con 0x7f734406c130 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.411+0000 7f73494cf640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.419 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.411+0000 7f73494cf640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732806b4d0 0x7f732806d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7334034000 tx=0x7f733402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.415+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7340058f30 con 0x7f734406c130 2024-04-25T10:04:25.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.588+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f73440b1880 con 0x7f734406c130 2024-04-25T10:04:25.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.592+0000 7f733a7fc640 1 -- 172.21.15.148:0/1241832665 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7340058830 con 0x7f734406c130 2024-04-25T10:04:25.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732806b4d0 msgr2=0x7f732806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732806b4d0 0x7f732806d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7334034000 tx=0x7f733402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:25.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 msgr2=0x7f73440c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 0x7f73440c5710 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f734000c820 tx=0x7f734000cce0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 shutdown_connections 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f734415aa10 0x7f734409bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73440ca5a0 0x7f73440c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 --2- 172.21.15.148:0/1241832665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f734406c130 0x7f73440c5710 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 >> 172.21.15.148:0/1241832665 conn(0x7f7344071af0 msgr2=0x7f73440ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 shutdown_connections 2024-04-25T10:04:25.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.594+0000 7f734bf34640 1 -- 172.21.15.148:0/1241832665 wait complete. 2024-04-25T10:04:25.610 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:25.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:25.646 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:25.695 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:25.695 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:25.695 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:25.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:25.710 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:04:25.710 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:25.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:25.728 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:25.728 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:25.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:25.766 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:04:25.766 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:25.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:25.785 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:04:25.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:25.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.898+0000 7f0e1f9ce640 1 Processor -- start 2024-04-25T10:04:25.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.898+0000 7f0e1f9ce640 1 -- start start 2024-04-25T10:04:25.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.898+0000 7f0e1f9ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1815dc00 0x7f0e180c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1f9ce640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e180ceb20 0x7f0e180d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1f9ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e180d1c00 0x7f0e180ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1f9ce640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e180d2060 con 0x7f0e180ceb20 2024-04-25T10:04:25.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1f9ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1815dfe0 con 0x7f0e180d1c00 2024-04-25T10:04:25.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1f9ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180d00c0 con 0x7f0e1815dc00 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e180ceb20 0x7f0e180d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e180ceb20 0x7f0e180d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46110/0 (socket says 172.21.15.148:46110) 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 -- 172.21.15.148:0/3449607425 learned_addr learned my addr 172.21.15.148:0/3449607425 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1df6b640 1 --2- 172.21.15.148:0/3449607425 >> v2:172.21.15.188:3300/0 conn(0x7f0e180d1c00 0x7f0e180ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 -- 172.21.15.148:0/3449607425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1815dc00 msgr2=0x7f0e180c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 --2- 172.21.15.148:0/3449607425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1815dc00 0x7f0e180c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 -- 172.21.15.148:0/3449607425 >> v2:172.21.15.188:3300/0 conn(0x7f0e180d1c00 msgr2=0x7f0e180ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 --2- 172.21.15.148:0/3449607425 >> v2:172.21.15.188:3300/0 conn(0x7f0e180d1c00 0x7f0e180ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.899+0000 7f0e1cf69640 1 -- 172.21.15.148:0/3449607425 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180d03a0 con 0x7f0e180ceb20 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1cf69640 1 --2- 172.21.15.148:0/3449607425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180d16c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f0e180b7a10 tx=0x7f0e0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44068154fe480ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e0ffff640 1 -- 172.21.15.148:0/3449607425 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08037070 con 0x7f0e180ceb20 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e0ffff640 1 -- 172.21.15.148:0/3449607425 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08032440 con 0x7f0e180ceb20 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e0ffff640 1 -- 172.21.15.148:0/3449607425 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08032810 con 0x7f0e180ceb20 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/3449607425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 msgr2=0x7f0e180d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/3449607425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180d16c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f0e180b7a10 tx=0x7f0e0802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/3449607425 shutdown_connections 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/3449607425 >> v2:172.21.15.188:3300/0 conn(0x7f0e180d1c00 0x7f0e180ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/3449607425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180d16c0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/3449607425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e1815dc00 0x7f0e180c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/3449607425 >> 172.21.15.148:0/3449607425 conn(0x7f0e1806f5d0 msgr2=0x7f0e1806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/3449607425 shutdown_connections 2024-04-25T10:04:25.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.900+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/3449607425 wait complete. 2024-04-25T10:04:25.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 Processor -- start 2024-04-25T10:04:25.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 -- start start 2024-04-25T10:04:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e180d1c00 0x7f0e18154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1815dc00 0x7f0e18158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180cc130 con 0x7f0e180ceb20 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180cc720 con 0x7f0e1815dc00 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1f9ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180cfd90 con 0x7f0e180d1c00 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1d76a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1d76a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46122/0 (socket says 172.21.15.148:46122) 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1d76a640 1 -- 172.21.15.148:0/2748504170 learned_addr learned my addr 172.21.15.148:0/2748504170 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.902+0000 7f0e1df6b640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1815dc00 0x7f0e18158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1cf69640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e180d1c00 0x7f0e18154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1d76a640 1 -- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e180d1c00 msgr2=0x7f0e18154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1d76a640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e180d1c00 0x7f0e18154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1d76a640 1 -- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1815dc00 msgr2=0x7f0e18158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1d76a640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1815dc00 0x7f0e18158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1d76a640 1 -- 172.21.15.148:0/2748504170 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180b53f0 con 0x7f0e180ceb20 2024-04-25T10:04:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1df6b640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1815dc00 0x7f0e18158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1d76a640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180b4580 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000b4a0 tx=0x7f0e1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10014070 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e10010610 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e18154c60 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10010a00 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.903+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e180b4ac0 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.904+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e18073990 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.905+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e10017070 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.905+0000 7f0e0dffb640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0dfc06b4d0 0x7f0dfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.905+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0e10089a40 con 0x7f0e180ceb20 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.905+0000 7f0e1cf69640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0dfc06b4d0 0x7f0dfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.905+0000 7f0e1cf69640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0dfc06b4d0 0x7f0dfc06d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e08034000 tx=0x7f0e0802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:25.909+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e10058ac0 con 0x7f0e180ceb20 2024-04-25T10:04:26.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.118+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e180d1650 con 0x7f0e180ceb20 2024-04-25T10:04:26.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.119+0000 7f0e0dffb640 1 -- 172.21.15.148:0/2748504170 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3710 (secure 0 0 0) 0x7f0e10089e20 con 0x7f0e180ceb20 2024-04-25T10:04:26.124 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:26.124 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8023,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3070224908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3070224908},{"type":"v1","addr":"172.21.15.188:6834","nonce":3070224908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3008962231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3008962231},{"type":"v1","addr":"172.21.15.188:6835","nonce":3008962231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":10284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/1178901140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1178901140},{"type":"v1","addr":"172.21.15.148:6835","nonce":1178901140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":10287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/877386480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":877386480},{"type":"v1","addr":"172.21.15.148:6837","nonce":877386480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[]} 2024-04-25T10:04:26.125 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 89 2024-04-25T10:04:26.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0dfc06b4d0 msgr2=0x7f0dfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0dfc06b4d0 0x7f0dfc06d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e08034000 tx=0x7f0e0802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 msgr2=0x7f0e180b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180b4580 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000b4a0 tx=0x7f0e1000b960 comp rx=0 tx=0).stop 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 shutdown_connections 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0dfc06b4d0 0x7f0dfc06d990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e1815dc00 0x7f0e18158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e180d1c00 0x7f0e18154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 --2- 172.21.15.148:0/2748504170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e180ceb20 0x7f0e180b4580 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 >> 172.21.15.148:0/2748504170 conn(0x7f0e1806f5d0 msgr2=0x7f0e180c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 shutdown_connections 2024-04-25T10:04:26.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.121+0000 7f0e1f9ce640 1 -- 172.21.15.148:0/2748504170 wait complete. 2024-04-25T10:04:26.137 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:04:26.137 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:04:26.138 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:04:26.138 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:26.138 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:04:26.138 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:04:26.138 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:04:26.138 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:26.139 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:04:26.139 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:04:26.139 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:04:26.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:26.139 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:04:26.139 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:04:26.139 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:04:26.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:26.147 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:04:26.148 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:04:26.148 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:04:26.149 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:04:26.151 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:04:26.151 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:04:26.152 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:04:26.152 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:04:26.190 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:04:26.191 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:26.191 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:04:26.192 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:26.196 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:04:26.196 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:26.197 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:04:26.197 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:26.197 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:04:26.263 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:04:26.263 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:04:26.276 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:04:26.277 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:04:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.295+0000 7f0aff763640 1 Processor -- start 2024-04-25T10:04:26.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.295+0000 7f0aff763640 1 -- start start 2024-04-25T10:04:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0aff763640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0af8099cb0 0x7f0af80b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0aff763640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af815f3d0 0x7f0af80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0aff763640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80cc5e0 con 0x7f0af80cfd30 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0aff763640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80cc3b0 con 0x7f0af8099cb0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0aff763640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80d0210 con 0x7f0af815f3d0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0afd4ff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0af8099cb0 0x7f0af80b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0afd4ff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0af8099cb0 0x7f0af80b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46164/0 (socket says 172.21.15.148:46164) 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.296+0000 7f0afd4ff640 1 -- 172.21.15.148:0/3845243827 learned_addr learned my addr 172.21.15.148:0/3845243827 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0afdd00640 1 --2- 172.21.15.148:0/3845243827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af815f3d0 0x7f0af80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0afd4ff640 1 -- 172.21.15.148:0/3845243827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af815f3d0 msgr2=0x7f0af80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0afd4ff640 1 --2- 172.21.15.148:0/3845243827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af815f3d0 0x7f0af80ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0afd4ff640 1 -- 172.21.15.148:0/3845243827 >> v1:172.21.15.188:6789/0 conn(0x7f0af80cfd30 legacy=0x7f0af815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0afd4ff640 1 -- 172.21.15.148:0/3845243827 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80d04f0 con 0x7f0af8099cb0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0afd4ff640 1 --2- 172.21.15.148:0/3845243827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af8099cb0 0x7f0af80b27d0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009870 tx=0x7f0af402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3b3382de7cb258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.297+0000 7f0aeffff640 1 -- 172.21.15.148:0/3845243827 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4037070 con 0x7f0af8099cb0 2024-04-25T10:04:26.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aeffff640 1 -- 172.21.15.148:0/3845243827 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af4032440 con 0x7f0af8099cb0 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aeffff640 1 -- 172.21.15.148:0/3845243827 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4032810 con 0x7f0af8099cb0 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 -- 172.21.15.148:0/3845243827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af8099cb0 msgr2=0x7f0af80b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 --2- 172.21.15.148:0/3845243827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af8099cb0 0x7f0af80b27d0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009870 tx=0x7f0af402a270 comp rx=0 tx=0).stop 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 -- 172.21.15.148:0/3845243827 shutdown_connections 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 --2- 172.21.15.148:0/3845243827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af815f3d0 0x7f0af80ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 --2- 172.21.15.148:0/3845243827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af8099cb0 0x7f0af80b27d0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 -- 172.21.15.148:0/3845243827 >> 172.21.15.148:0/3845243827 conn(0x7f0af80b3290 msgr2=0x7f0af80b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 -- 172.21.15.148:0/3845243827 shutdown_connections 2024-04-25T10:04:26.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.298+0000 7f0aff763640 1 -- 172.21.15.148:0/3845243827 wait complete. 2024-04-25T10:04:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 Processor -- start 2024-04-25T10:04:26.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 -- start start 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af8099cb0 0x7f0af80cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cfd30 0x7f0af80b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 0x7f0af80ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c52f0 con 0x7f0af815f3d0 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80d07d0 con 0x7f0af80cfd30 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.300+0000 7f0aff763640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c5470 con 0x7f0af8099cb0 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 0x7f0af80ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 0x7f0af80ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46178/0 (socket says 172.21.15.148:46178) 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 -- 172.21.15.148:0/3713240721 learned_addr learned my addr 172.21.15.148:0/3713240721 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afd4ff640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af8099cb0 0x7f0af80cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afccfe640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cfd30 0x7f0af80b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 -- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af8099cb0 msgr2=0x7f0af80cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af8099cb0 0x7f0af80cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 -- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cfd30 msgr2=0x7f0af80b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cfd30 0x7f0af80b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 -- 172.21.15.148:0/3713240721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80b4af0 con 0x7f0af815f3d0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0afdd00640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 0x7f0af80ca340 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f0af0007e60 tx=0x7f0af000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0014420 con 0x7f0af815f3d0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0019070 con 0x7f0af815f3d0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af80b4e00 con 0x7f0af815f3d0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0014820 con 0x7f0af815f3d0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.301+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af80b5640 con 0x7f0af815f3d0 2024-04-25T10:04:26.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.303+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8073a00 con 0x7f0af815f3d0 2024-04-25T10:04:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.303+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0af00149c0 con 0x7f0af815f3d0 2024-04-25T10:04:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.303+0000 7f0aedffb640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0adc06b4d0 0x7f0adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.303+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0af008a7a0 con 0x7f0af815f3d0 2024-04-25T10:04:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.303+0000 7f0afd4ff640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0adc06b4d0 0x7f0adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.304+0000 7f0afd4ff640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0adc06b4d0 0x7f0adc06d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0af4034000 tx=0x7f0af402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.311+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0af80b4e00 con 0x7f0af815f3d0 2024-04-25T10:04:26.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:26.495+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0af8067df0 con 0x7f0af815f3d0 2024-04-25T10:04:27.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.305+0000 7f0aedffb640 1 -- 172.21.15.148:0/3713240721 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v155) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0af0059140 con 0x7f0af815f3d0 2024-04-25T10:04:27.311 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:04:27.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0adc06b4d0 msgr2=0x7f0adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0adc06b4d0 0x7f0adc06d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0af4034000 tx=0x7f0af402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 msgr2=0x7f0af80ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 0x7f0af80ca340 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f0af0007e60 tx=0x7f0af000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 shutdown_connections 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0adc06b4d0 0x7f0adc06d990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0af815f3d0 0x7f0af80ca340 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cfd30 0x7f0af80b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 --2- 172.21.15.148:0/3713240721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af8099cb0 0x7f0af80cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 >> 172.21.15.148:0/3713240721 conn(0x7f0af80b3290 msgr2=0x7f0af809a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 shutdown_connections 2024-04-25T10:04:27.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.308+0000 7f0aff763640 1 -- 172.21.15.148:0/3713240721 wait complete. 2024-04-25T10:04:27.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:04:27.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.419+0000 7efe76c93640 1 Processor -- start 2024-04-25T10:04:27.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.419+0000 7efe76c93640 1 -- start start 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe76c93640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 0x7efe700b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe76c93640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700d1df0 con 0x7efe700d0ad0 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe76c93640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe700d20d0 con 0x7efe700b9770 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe76c93640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700d23b0 con 0x7efe7015dd30 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe75230640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efe700d0ad0 0x7efe700b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49470/0 (socket says 172.21.15.148:49470) 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe75230640 1 -- 172.21.15.148:0/2244674108 learned_addr learned my addr 172.21.15.148:0/2244674108 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 0x7efe700b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017350420 0 0) 0x7efe700d1df0 con 0x7efe700d0ad0 2024-04-25T10:04:27.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.420+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe58003520 con 0x7efe700d0ad0 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1976050151 0 0) 0x7efe58003520 con 0x7efe700d0ad0 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 msgr2=0x7efe700b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 0x7efe700b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 >> v1:172.21.15.188:6789/0 conn(0x7efe700b9770 legacy=0x7efe700d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700d2690 con 0x7efe700d0ad0 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7efe680019f0 con 0x7efe700d0ad0 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 >> v1:172.21.15.148:6789/0 conn(0x7efe700d0ad0 legacy=0x7efe700b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 0x7efe58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58004d40 0x7efe580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700d23b0 con 0x7efe580039e0 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5800ba70 con 0x7efe58004d40 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5800bc70 con 0x7efe7015dd30 2024-04-25T10:04:27.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe75230640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58004d40 0x7efe580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe6ffff640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 0x7efe58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe74a2f640 1 -- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 msgr2=0x7efe58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015dd30 0x7efe58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe74a2f640 1 -- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58004d40 msgr2=0x7efe580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe58004d40 0x7efe580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.421+0000 7efe74a2f640 1 -- 172.21.15.148:0/2244674108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe58003520 con 0x7efe580039e0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe58003dc0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7efe60009ac0 tx=0x7efe58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b6e52f6a29084c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60036070 con 0x7efe580039e0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60031400 con 0x7efe580039e0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe6f7fe640 1 -- 172.21.15.148:0/2244674108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60009690 con 0x7efe580039e0 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe76c93640 1 -- 172.21.15.148:0/2244674108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 msgr2=0x7efe58003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe76c93640 1 --2- 172.21.15.148:0/2244674108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe58003dc0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7efe60009ac0 tx=0x7efe58005830 comp rx=0 tx=0).stop 2024-04-25T10:04:27.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe75230640 1 -- 172.21.15.148:0/2244674108 reap_dead start 2024-04-25T10:04:27.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe76c93640 1 -- 172.21.15.148:0/2244674108 shutdown_connections 2024-04-25T10:04:27.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.422+0000 7efe76c93640 1 -- 172.21.15.148:0/2244674108 >> 172.21.15.148:0/2244674108 conn(0x7efe70061c30 msgr2=0x7efe7006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.423+0000 7efe76c93640 1 -- 172.21.15.148:0/2244674108 shutdown_connections 2024-04-25T10:04:27.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.423+0000 7efe76c93640 1 -- 172.21.15.148:0/2244674108 wait complete. 2024-04-25T10:04:27.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.424+0000 7efe76c93640 1 Processor -- start 2024-04-25T10:04:27.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.424+0000 7efe76c93640 1 -- start start 2024-04-25T10:04:27.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.424+0000 7efe76c93640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe700b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.424+0000 7efe76c93640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015bf60 0x7efe700aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe76c93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe700ac640 0x7efe7015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe76c93640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700d1b90 con 0x7efe580039e0 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe76c93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe700d1a10 con 0x7efe700ac640 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe76c93640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe700d2e90 con 0x7efe7015bf60 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe700b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe700b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46202/0 (socket says 172.21.15.148:46202) 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 -- 172.21.15.148:0/2605287977 learned_addr learned my addr 172.21.15.148:0/2605287977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 -- 172.21.15.148:0/2605287977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015bf60 msgr2=0x7efe700aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015bf60 0x7efe700aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 -- 172.21.15.148:0/2605287977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe700ac640 msgr2=0x7efe7015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe700ac640 0x7efe7015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 -- 172.21.15.148:0/2605287977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700ab5a0 con 0x7efe580039e0 2024-04-25T10:04:27.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe74a2f640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe700b5b40 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7efe60034040 tx=0x7efe60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60040020 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe60003bc0 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe700ab880 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60036040 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.425+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe700abe40 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.426+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe70073990 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.427+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe60043020 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.427+0000 7efe6d7fa640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b500 0x7efe5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.427+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efe600aa0b0 con 0x7efe580039e0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.427+0000 7efe6ffff640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b500 0x7efe5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.428+0000 7efe6ffff640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b500 0x7efe5c06d9c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efe700ca3c0 tx=0x7efe64009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.431+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe6004a070 con 0x7efe580039e0 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.666+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efe700ab2a0 con 0x7efe580039e0 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.666+0000 7efe6d7fa640 1 -- 172.21.15.148:0/2605287977 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efe600044e0 con 0x7efe580039e0 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.674 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.675 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.676 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:04:27.677 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.678 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.679 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:04:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:04:27.681 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.682 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.683 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:27.684 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:04:27.685 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.668+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b500 msgr2=0x7efe5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.668+0000 7efe76c93640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b500 0x7efe5c06d9c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efe700ca3c0 tx=0x7efe64009040 comp rx=0 tx=0).stop 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.668+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 msgr2=0x7efe700b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.668+0000 7efe76c93640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe700b5b40 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7efe60034040 tx=0x7efe60004730 comp rx=0 tx=0).stop 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 shutdown_connections 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe5c06b500 0x7efe5c06d9c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe700ac640 0x7efe7015f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe7015bf60 0x7efe700aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 --2- 172.21.15.148:0/2605287977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe580039e0 0x7efe700b5b40 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 >> 172.21.15.148:0/2605287977 conn(0x7efe70061c30 msgr2=0x7efe701606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 shutdown_connections 2024-04-25T10:04:27.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.669+0000 7efe76c93640 1 -- 172.21.15.148:0/2605287977 wait complete. 2024-04-25T10:04:27.688 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:27.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.781+0000 7efc49c7c640 1 Processor -- start 2024-04-25T10:04:27.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.781+0000 7efc49c7c640 1 -- start start 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc49c7c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efc440ce920 0x7efc440cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc49c7c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc4406d290 0x7efc4406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc49c7c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc44144b30 con 0x7efc44144750 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc49c7c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc440712d0 con 0x7efc4406d290 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc49c7c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc440715b0 con 0x7efc440ce920 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc4406d290 0x7efc4406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7efc4406d290 0x7efc4406d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46210/0 (socket says 172.21.15.148:46210) 2024-04-25T10:04:27.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 -- 172.21.15.148:0/3375657574 learned_addr learned my addr 172.21.15.148:0/3375657574 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc42ffd640 1 --2- 172.21.15.148:0/3375657574 >> v2:172.21.15.188:3300/0 conn(0x7efc440ce920 0x7efc440cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 -- 172.21.15.148:0/3375657574 >> v2:172.21.15.188:3300/0 conn(0x7efc440ce920 msgr2=0x7efc440cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 --2- 172.21.15.148:0/3375657574 >> v2:172.21.15.188:3300/0 conn(0x7efc440ce920 0x7efc440cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 -- 172.21.15.148:0/3375657574 >> v1:172.21.15.188:6789/0 conn(0x7efc44144750 legacy=0x7efc4406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.782+0000 7efc43fff640 1 -- 172.21.15.148:0/3375657574 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44071890 con 0x7efc4406d290 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.783+0000 7efc43fff640 1 --2- 172.21.15.148:0/3375657574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc4406d290 0x7efc4406d6c0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7efc441606b0 tx=0x7efc3c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b41d90cf342718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.783+0000 7efc427fc640 1 -- 172.21.15.148:0/3375657574 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c036070 con 0x7efc4406d290 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.783+0000 7efc427fc640 1 -- 172.21.15.148:0/3375657574 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c031600 con 0x7efc4406d290 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.783+0000 7efc427fc640 1 -- 172.21.15.148:0/3375657574 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0319d0 con 0x7efc4406d290 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 -- 172.21.15.148:0/3375657574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc4406d290 msgr2=0x7efc4406d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 --2- 172.21.15.148:0/3375657574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc4406d290 0x7efc4406d6c0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7efc441606b0 tx=0x7efc3c02a730 comp rx=0 tx=0).stop 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 -- 172.21.15.148:0/3375657574 shutdown_connections 2024-04-25T10:04:27.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 --2- 172.21.15.148:0/3375657574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc4406d290 0x7efc4406d6c0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 --2- 172.21.15.148:0/3375657574 >> v2:172.21.15.188:3300/0 conn(0x7efc440ce920 0x7efc440cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 -- 172.21.15.148:0/3375657574 >> 172.21.15.148:0/3375657574 conn(0x7efc4406abe0 msgr2=0x7efc4406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 -- 172.21.15.148:0/3375657574 shutdown_connections 2024-04-25T10:04:27.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.784+0000 7efc49c7c640 1 -- 172.21.15.148:0/3375657574 wait complete. 2024-04-25T10:04:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.785+0000 7efc49c7c640 1 Processor -- start 2024-04-25T10:04:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.785+0000 7efc49c7c640 1 -- start start 2024-04-25T10:04:27.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc4406d290 0x7efc441ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 0x7efc440b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc44144750 0x7efc440b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4406f020 con 0x7efc440ce920 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4406ece0 con 0x7efc44144750 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc44145450 con 0x7efc4406d290 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 0x7efc440b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 0x7efc440b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46218/0 (socket says 172.21.15.148:46218) 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 -- 172.21.15.148:0/1183571619 learned_addr learned my addr 172.21.15.148:0/1183571619 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc43fff640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc44144750 0x7efc440b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc437fe640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc4406d290 0x7efc441ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 -- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc4406d290 msgr2=0x7efc441ee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc4406d290 0x7efc441ee9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 -- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc44144750 msgr2=0x7efc440b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc44144750 0x7efc440b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 -- 172.21.15.148:0/1183571619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc441f23e0 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc43fff640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc44144750 0x7efc440b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc437fe640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc4406d290 0x7efc441ee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc42ffd640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 0x7efc440b1460 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d5f0 tx=0x7efc3800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380125f0 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38019070 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc380129d0 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc44141af0 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.786+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc441420b0 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.787+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc44073990 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.788+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc38012b70 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.788+0000 7efc2bfff640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3006b4d0 0x7efc3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.788+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efc3808a410 con 0x7efc440ce920 2024-04-25T10:04:27.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.788+0000 7efc437fe640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3006b4d0 0x7efc3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.789+0000 7efc437fe640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3006b4d0 0x7efc3006d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7efc4406e170 tx=0x7efc34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.793+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc38059450 con 0x7efc440ce920 2024-04-25T10:04:27.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.965+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc440d0510 con 0x7efc440ce920 2024-04-25T10:04:27.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.966+0000 7efc2bfff640 1 -- 172.21.15.148:0/1183571619 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+13222 (secure 0 0 0) 0x7efc38058d50 con 0x7efc440ce920 2024-04-25T10:04:27.971 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:27.971 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":155,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:27.303461+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:27.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3006b4d0 msgr2=0x7efc3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3006b4d0 0x7efc3006d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7efc4406e170 tx=0x7efc34009040 comp rx=0 tx=0).stop 2024-04-25T10:04:27.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 msgr2=0x7efc440b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 0x7efc440b1460 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7efc3800d5f0 tx=0x7efc3800dab0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 shutdown_connections 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3006b4d0 0x7efc3006d990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc44144750 0x7efc440b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc440ce920 0x7efc440b1460 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 --2- 172.21.15.148:0/1183571619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc4406d290 0x7efc441ee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 >> 172.21.15.148:0/1183571619 conn(0x7efc4406abe0 msgr2=0x7efc4406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 shutdown_connections 2024-04-25T10:04:27.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:27.968+0000 7efc49c7c640 1 -- 172.21.15.148:0/1183571619 wait complete. 2024-04-25T10:04:27.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:27.988 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:28.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.082+0000 7f933c491640 1 Processor -- start 2024-04-25T10:04:28.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.082+0000 7f933c491640 1 -- start start 2024-04-25T10:04:28.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933c491640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933415dc20 0x7f93340ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933c491640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9334161710 0x7f93340cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933c491640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93340b9330 0x7f933406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933c491640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933415e000 con 0x7f93340b9330 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933c491640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340d1de0 con 0x7f9334161710 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933c491640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9334161c60 con 0x7f933415dc20 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933aa2e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93340b9330 0x7f933406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933aa2e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f93340b9330 0x7f933406bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46234/0 (socket says 172.21.15.148:46234) 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.083+0000 7f933aa2e640 1 -- 172.21.15.148:0/520642877 learned_addr learned my addr 172.21.15.148:0/520642877 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933a22d640 1 --2- 172.21.15.148:0/520642877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933415dc20 0x7f93340ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f9339a2c640 1 --2- 172.21.15.148:0/520642877 >> v2:172.21.15.188:3300/0 conn(0x7f9334161710 0x7f93340cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933aa2e640 1 -- 172.21.15.148:0/520642877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933415dc20 msgr2=0x7f93340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933aa2e640 1 --2- 172.21.15.148:0/520642877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933415dc20 0x7f93340ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933aa2e640 1 -- 172.21.15.148:0/520642877 >> v2:172.21.15.188:3300/0 conn(0x7f9334161710 msgr2=0x7f93340cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933aa2e640 1 --2- 172.21.15.148:0/520642877 >> v2:172.21.15.188:3300/0 conn(0x7f9334161710 0x7f93340cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933aa2e640 1 -- 172.21.15.148:0/520642877 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334161f40 con 0x7f93340b9330 2024-04-25T10:04:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.084+0000 7f933aa2e640 1 --2- 172.21.15.148:0/520642877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93340b9330 0x7f933406bf80 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f933406d9a0 tx=0x7f932c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8a074cb7a4fec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933922b640 1 -- 172.21.15.148:0/520642877 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c037070 con 0x7f93340b9330 2024-04-25T10:04:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933922b640 1 -- 172.21.15.148:0/520642877 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c032440 con 0x7f93340b9330 2024-04-25T10:04:28.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933922b640 1 -- 172.21.15.148:0/520642877 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c032810 con 0x7f93340b9330 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 -- 172.21.15.148:0/520642877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93340b9330 msgr2=0x7f933406bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 --2- 172.21.15.148:0/520642877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93340b9330 0x7f933406bf80 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f933406d9a0 tx=0x7f932c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 -- 172.21.15.148:0/520642877 shutdown_connections 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 --2- 172.21.15.148:0/520642877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93340b9330 0x7f933406bf80 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 --2- 172.21.15.148:0/520642877 >> v2:172.21.15.188:3300/0 conn(0x7f9334161710 0x7f93340cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 --2- 172.21.15.148:0/520642877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f933415dc20 0x7f93340ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 -- 172.21.15.148:0/520642877 >> 172.21.15.148:0/520642877 conn(0x7f93340d0420 msgr2=0x7f93340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 -- 172.21.15.148:0/520642877 shutdown_connections 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.085+0000 7f933c491640 1 -- 172.21.15.148:0/520642877 wait complete. 2024-04-25T10:04:28.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.086+0000 7f933c491640 1 Processor -- start 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.086+0000 7f933c491640 1 -- start start 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933c491640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93340b9330 0x7f93341f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933c491640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 0x7f93341f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933c491640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9334161710 0x7f933415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933c491640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b5e50 con 0x7f933415dc20 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933c491640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334162500 con 0x7f93340b9330 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933c491640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b5fd0 con 0x7f9334161710 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 0x7f93341f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 0x7f93341f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46244/0 (socket says 172.21.15.148:46244) 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 -- 172.21.15.148:0/2771332216 learned_addr learned my addr 172.21.15.148:0/2771332216 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933aa2e640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9334161710 0x7f933415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f933a22d640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93340b9330 0x7f93341f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 -- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9334161710 msgr2=0x7f933415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9334161710 0x7f933415cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 -- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93340b9330 msgr2=0x7f93341f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93340b9330 0x7f93341f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.087+0000 7f9339a2c640 1 -- 172.21.15.148:0/2771332216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340aa380 con 0x7f933415dc20 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.088+0000 7f9339a2c640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 0x7f93341f3620 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f932400e870 tx=0x7f932400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.088+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324017070 con 0x7f933415dc20 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.088+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400c8e0 con 0x7f933415dc20 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.088+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93341f3ec0 con 0x7f933415dc20 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.088+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400ccd0 con 0x7f933415dc20 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.088+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340aa620 con 0x7f933415dc20 2024-04-25T10:04:28.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.089+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93341f42b0 con 0x7f933415dc20 2024-04-25T10:04:28.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.089+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f932401a070 con 0x7f933415dc20 2024-04-25T10:04:28.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.090+0000 7f932affd640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931806b4d0 0x7f931806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.090+0000 7f933a22d640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931806b4d0 0x7f931806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.090+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f932408a0e0 con 0x7f933415dc20 2024-04-25T10:04:28.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.090+0000 7f933a22d640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931806b4d0 0x7f931806d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9330011200 tx=0x7f933000d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.094+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9324058b60 con 0x7f933415dc20 2024-04-25T10:04:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.266+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9334073a00 con 0x7f933415dc20 2024-04-25T10:04:28.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.266+0000 7f932affd640 1 -- 172.21.15.148:0/2771332216 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f932405cc00 con 0x7f933415dc20 2024-04-25T10:04:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:04:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931806b4d0 msgr2=0x7f931806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931806b4d0 0x7f931806d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9330011200 tx=0x7f933000d040 comp rx=0 tx=0).stop 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 msgr2=0x7f93341f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 0x7f93341f3620 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f932400e870 tx=0x7f932400ed30 comp rx=0 tx=0).stop 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 shutdown_connections 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f931806b4d0 0x7f931806d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9334161710 0x7f933415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f933415dc20 0x7f93341f3620 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 --2- 172.21.15.148:0/2771332216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93340b9330 0x7f93341f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 >> 172.21.15.148:0/2771332216 conn(0x7f93340d0420 msgr2=0x7f93340d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 shutdown_connections 2024-04-25T10:04:28.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.268+0000 7f933c491640 1 -- 172.21.15.148:0/2771332216 wait complete. 2024-04-25T10:04:28.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:04:28.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.376+0000 7f38f60ac640 1 Processor -- start 2024-04-25T10:04:28.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.376+0000 7f38f60ac640 1 -- start start 2024-04-25T10:04:28.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.377+0000 7f38f60ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 0x7f38f00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.377+0000 7f38f60ac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00d0350 con 0x7f38f00cfe40 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.377+0000 7f38f60ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00d0630 con 0x7f38f00d1b00 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.377+0000 7f38f60ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00d0910 con 0x7f38f015bd30 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.377+0000 7f38eeffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f38f00cfe40 0x7f38f00d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49474/0 (socket says 172.21.15.148:49474) 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.377+0000 7f38eeffd640 1 -- 172.21.15.148:0/2356665997 learned_addr learned my addr 172.21.15.148:0/2356665997 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ef7fe640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 0x7f38f00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413441941 0 0) 0x7f38f00d0350 con 0x7f38f00cfe40 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003520 con 0x7f38f00cfe40 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364303303 0 0) 0x7f38f00d0630 con 0x7f38f00d1b00 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f00d0350 con 0x7f38f00d1b00 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435327405 0 0) 0x7f38d8003520 con 0x7f38f00cfe40 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 msgr2=0x7f38f00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:28.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 0x7f38f00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 >> v1:172.21.15.188:6789/0 conn(0x7f38f00d1b00 legacy=0x7f38f00b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f00d0bf0 con 0x7f38f00cfe40 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f38e40015f0 con 0x7f38f00cfe40 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 >> v1:172.21.15.148:6789/0 conn(0x7f38f00cfe40 legacy=0x7f38f00d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 0x7f38d8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d8004120 0x7f38d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004d40 0x7f38d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00d0910 con 0x7f38d8004120 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800bb00 con 0x7f38d8004d40 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d800bd00 con 0x7f38f015bd30 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38eeffd640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d8004120 0x7f38d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38ef7fe640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 0x7f38d8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38effff640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004d40 0x7f38d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38eeffd640 1 -- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 msgr2=0x7f38d8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:28.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38eeffd640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f015bd30 0x7f38d8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38eeffd640 1 -- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004d40 msgr2=0x7f38d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38eeffd640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004d40 0x7f38d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38eeffd640 1 -- 172.21.15.148:0/2356665997 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d8003520 con 0x7f38d8004120 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.378+0000 7f38effff640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004d40 0x7f38d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.379+0000 7f38eeffd640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d8004120 0x7f38d8004800 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f38e4007f10 tx=0x7f38d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31229bc569b7cd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.379+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4007b60 con 0x7f38d8004120 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.379+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e4005bb0 con 0x7f38d8004120 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.379+0000 7f38ee7fc640 1 -- 172.21.15.148:0/2356665997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4007070 con 0x7f38d8004120 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38f60ac640 1 -- 172.21.15.148:0/2356665997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d8004120 msgr2=0x7f38d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38f60ac640 1 --2- 172.21.15.148:0/2356665997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38d8004120 0x7f38d8004800 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f38e4007f10 tx=0x7f38d8005830 comp rx=0 tx=0).stop 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38effff640 1 -- 172.21.15.148:0/2356665997 reap_dead start 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38f60ac640 1 -- 172.21.15.148:0/2356665997 shutdown_connections 2024-04-25T10:04:28.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38f60ac640 1 -- 172.21.15.148:0/2356665997 >> 172.21.15.148:0/2356665997 conn(0x7f38f006f590 msgr2=0x7f38f006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38f60ac640 1 -- 172.21.15.148:0/2356665997 shutdown_connections 2024-04-25T10:04:28.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.380+0000 7f38f60ac640 1 -- 172.21.15.148:0/2356665997 wait complete. 2024-04-25T10:04:28.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.381+0000 7f38f60ac640 1 Processor -- start 2024-04-25T10:04:28.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.381+0000 7f38f60ac640 1 -- start start 2024-04-25T10:04:28.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38f60ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004120 0x7f38f01e3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38f60ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f014aab0 0x7f38f01e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38f60ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 0x7f38f01e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38f60ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00b73b0 con 0x7f38f01e6f90 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38f60ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f015c110 con 0x7f38d8004120 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38f60ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00b7530 con 0x7f38f014aab0 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 0x7f38f01e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 0x7f38f01e5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46262/0 (socket says 172.21.15.148:46262) 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 -- 172.21.15.148:0/3082996839 learned_addr learned my addr 172.21.15.148:0/3082996839 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38eeffd640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f014aab0 0x7f38f01e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38ef7fe640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004120 0x7f38f01e3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 -- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f014aab0 msgr2=0x7f38f01e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f014aab0 0x7f38f01e4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 -- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004120 msgr2=0x7f38f01e3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004120 0x7f38f01e3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 -- 172.21.15.148:0/3082996839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f01e4bc0 con 0x7f38f01e6f90 2024-04-25T10:04:28.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38ef7fe640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004120 0x7f38f01e3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38effff640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 0x7f38f01e5c60 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f38f00b1390 tx=0x7f38e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8006b50 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.382+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e8006cf0 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.383+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8003bb0 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.383+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f01e4ea0 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.383+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f01e5460 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.383+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0073990 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.384+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38e8005070 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.384+0000 7f38d7fff640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38dc06b4d0 0x7f38dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.384+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f38e8089a60 con 0x7f38f01e6f90 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.384+0000 7f38ef7fe640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38dc06b4d0 0x7f38dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.385+0000 7f38ef7fe640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38dc06b4d0 0x7f38dc06d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009ae0 tx=0x7f38e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.389+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38e8058aa0 con 0x7f38f01e6f90 2024-04-25T10:04:28.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.560+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f38f00b5650 con 0x7f38f01e6f90 2024-04-25T10:04:28.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.560+0000 7f38d7fff640 1 -- 172.21.15.148:0/3082996839 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v155) v1 ==== 84+0+65 (secure 0 0 0) 0x7f38e80583a0 con 0x7f38f01e6f90 2024-04-25T10:04:28.565 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:28.566 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:04:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:04:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:04:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:04:28.566 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:28.566 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:04:28.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38dc06b4d0 msgr2=0x7f38dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38dc06b4d0 0x7f38dc06d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009ae0 tx=0x7f38e0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:28.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 msgr2=0x7f38f01e5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 0x7f38f01e5c60 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f38f00b1390 tx=0x7f38e800eb90 comp rx=0 tx=0).stop 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 shutdown_connections 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f38dc06b4d0 0x7f38dc06d990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38f01e6f90 0x7f38f01e5c60 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38f014aab0 0x7f38f01e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 --2- 172.21.15.148:0/3082996839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38d8004120 0x7f38f01e3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 >> 172.21.15.148:0/3082996839 conn(0x7f38f006f590 msgr2=0x7f38f00b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 shutdown_connections 2024-04-25T10:04:28.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.562+0000 7f38f60ac640 1 -- 172.21.15.148:0/3082996839 wait complete. 2024-04-25T10:04:28.578 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:04:28.578 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:04:28.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.674+0000 7f6625a4a640 1 Processor -- start 2024-04-25T10:04:28.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.674+0000 7f6625a4a640 1 -- start start 2024-04-25T10:04:28.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f6625a4a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66200b3d10 0x7f66200acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f6625a4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662015e020 con 0x7f662006e360 2024-04-25T10:04:28.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f6625a4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662006ac80 con 0x7f66200b38d0 2024-04-25T10:04:28.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f6625a4a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66200b4180 con 0x7f66200b3d10 2024-04-25T10:04:28.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f661effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f662006e360 0x7f662006e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49486/0 (socket says 172.21.15.148:49486) 2024-04-25T10:04:28.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f661effd640 1 -- 172.21.15.148:0/2886635055 learned_addr learned my addr 172.21.15.148:0/2886635055 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.675+0000 7f661f7fe640 1 --2- 172.21.15.148:0/2886635055 >> v2:172.21.15.188:3300/0 conn(0x7f66200b3d10 0x7f66200acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.676+0000 7f661dffb640 1 -- 172.21.15.148:0/2886635055 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946127283 0 0) 0x7f662015e020 con 0x7f662006e360 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.676+0000 7f661dffb640 1 -- 172.21.15.148:0/2886635055 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6608003520 con 0x7f662006e360 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.676+0000 7f661dffb640 1 -- 172.21.15.148:0/2886635055 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 114001735 0 0) 0x7f6608003520 con 0x7f662006e360 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.676+0000 7f661f7fe640 1 -- 172.21.15.148:0/2886635055 >> v1:172.21.15.188:6789/0 conn(0x7f66200b38d0 legacy=0x7f662006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.676+0000 7f661f7fe640 1 -- 172.21.15.148:0/2886635055 >> v1:172.21.15.148:6789/0 conn(0x7f662006e360 legacy=0x7f662006e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.676+0000 7f661f7fe640 1 -- 172.21.15.148:0/2886635055 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662006af60 con 0x7f66200b3d10 2024-04-25T10:04:28.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f661f7fe640 1 --2- 172.21.15.148:0/2886635055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200acea0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f66180097a0 tx=0x7f661802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb81456f59e8ee41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f661dffb640 1 -- 172.21.15.148:0/2886635055 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618037070 con 0x7f66200b3d10 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f661dffb640 1 -- 172.21.15.148:0/2886635055 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661802ace0 con 0x7f66200b3d10 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f661dffb640 1 -- 172.21.15.148:0/2886635055 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66100019f0 con 0x7f66200b3d10 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 -- 172.21.15.148:0/2886635055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 msgr2=0x7f66200acea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 --2- 172.21.15.148:0/2886635055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200acea0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f66180097a0 tx=0x7f661802a250 comp rx=0 tx=0).stop 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 -- 172.21.15.148:0/2886635055 shutdown_connections 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 --2- 172.21.15.148:0/2886635055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200acea0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 -- 172.21.15.148:0/2886635055 >> 172.21.15.148:0/2886635055 conn(0x7f66200cfec0 msgr2=0x7f66200d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 -- 172.21.15.148:0/2886635055 shutdown_connections 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.677+0000 7f6625a4a640 1 -- 172.21.15.148:0/2886635055 wait complete. 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.678+0000 7f6625a4a640 1 Processor -- start 2024-04-25T10:04:28.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 -- start start 2024-04-25T10:04:28.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 0x7f6620151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200b38d0 0x7f66200af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662006b520 con 0x7f662006e360 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662006b890 con 0x7f66200b3d10 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f6625a4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662006ee50 con 0x7f66200b38d0 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f661effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 0x7f6620151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f661effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 0x7f6620151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46268/0 (socket says 172.21.15.148:46268) 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.679+0000 7f661effd640 1 -- 172.21.15.148:0/556457100 learned_addr learned my addr 172.21.15.148:0/556457100 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661f7fe640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661e7fc640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200b38d0 0x7f66200af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661effd640 1 -- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200b38d0 msgr2=0x7f66200af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661effd640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200b38d0 0x7f66200af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661effd640 1 -- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 msgr2=0x7f66200afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661effd640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661effd640 1 -- 172.21.15.148:0/556457100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200b0380 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661e7fc640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200b38d0 0x7f66200af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f661effd640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 0x7f6620151af0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f661000cc90 tx=0x7f661000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610017070 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661000ce40 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610012640 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66200b0660 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.680+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66200c9bf0 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.681+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66200b5170 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.682+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f661001a070 con 0x7f662006e360 2024-04-25T10:04:28.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.682+0000 7f6607fff640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f660c06b4d0 0x7f660c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.682+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6610089f90 con 0x7f662006e360 2024-04-25T10:04:28.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.682+0000 7f661e7fc640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f660c06b4d0 0x7f660c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.682+0000 7f661e7fc640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f660c06b4d0 0x7f660c06d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f662015cc80 tx=0x7f6614009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.686+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66200b5170 con 0x7f662006e360 2024-04-25T10:04:28.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:28.857+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6620073a00 con 0x7f662006e360 2024-04-25T10:04:29.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.311+0000 7f6607fff640 1 -- 172.21.15.148:0/556457100 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v156) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6610058fd0 con 0x7f662006e360 2024-04-25T10:04:29.316 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.313+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f660c06b4d0 msgr2=0x7f660c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.313+0000 7f6625a4a640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f660c06b4d0 0x7f660c06d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f662015cc80 tx=0x7f6614009040 comp rx=0 tx=0).stop 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.313+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 msgr2=0x7f6620151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.313+0000 7f6625a4a640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 0x7f6620151af0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f661000cc90 tx=0x7f661000a430 comp rx=0 tx=0).stop 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 shutdown_connections 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f660c06b4d0 0x7f660c06d990 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200b3d10 0x7f66200afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200b38d0 0x7f66200af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 --2- 172.21.15.148:0/556457100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f662006e360 0x7f6620151af0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 >> 172.21.15.148:0/556457100 conn(0x7f66200cfec0 msgr2=0x7f66200cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 shutdown_connections 2024-04-25T10:04:29.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.314+0000 7f6625a4a640 1 -- 172.21.15.148:0/556457100 wait complete. 2024-04-25T10:04:29.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:04:29.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.434+0000 7f0ff80b3640 1 Processor -- start 2024-04-25T10:04:29.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.434+0000 7f0ff80b3640 1 -- start start 2024-04-25T10:04:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff80b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff015dcd0 0x7f0ff006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff80b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0ff006b8c0 0x7f0ff00b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff80b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ff00b49f0 0x7f0ff006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff80b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00b3450 con 0x7f0ff00b49f0 2024-04-25T10:04:29.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff80b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00b36f0 con 0x7f0ff006b8c0 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff80b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00b39d0 con 0x7f0ff015dcd0 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ff00b49f0 0x7f0ff006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0ff00b49f0 0x7f0ff006e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46284/0 (socket says 172.21.15.148:46284) 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 -- 172.21.15.148:0/2940605720 learned_addr learned my addr 172.21.15.148:0/2940605720 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 -- 172.21.15.148:0/2940605720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff015dcd0 msgr2=0x7f0ff006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff5e4f640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff015dcd0 0x7f0ff006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff564e640 1 --2- 172.21.15.148:0/2940605720 >> v2:172.21.15.188:3300/0 conn(0x7f0ff006b8c0 0x7f0ff00b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff015dcd0 0x7f0ff006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 -- 172.21.15.148:0/2940605720 >> v2:172.21.15.188:3300/0 conn(0x7f0ff006b8c0 msgr2=0x7f0ff00b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 --2- 172.21.15.148:0/2940605720 >> v2:172.21.15.188:3300/0 conn(0x7f0ff006b8c0 0x7f0ff00b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff6650640 1 -- 172.21.15.148:0/2940605720 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00b3cb0 con 0x7f0ff00b49f0 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff5e4f640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff015dcd0 0x7f0ff006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.435+0000 7f0ff564e640 1 --2- 172.21.15.148:0/2940605720 >> v2:172.21.15.188:3300/0 conn(0x7f0ff006b8c0 0x7f0ff00b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.436+0000 7f0ff6650640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff00b49f0 0x7f0ff006e2f0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b2850 tx=0x7f0fe802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5653ba084c677f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.436+0000 7f0ff4e4d640 1 -- 172.21.15.148:0/2940605720 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8036070 con 0x7f0ff00b49f0 2024-04-25T10:04:29.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.436+0000 7f0ff4e4d640 1 -- 172.21.15.148:0/2940605720 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8031600 con 0x7f0ff00b49f0 2024-04-25T10:04:29.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.436+0000 7f0ff4e4d640 1 -- 172.21.15.148:0/2940605720 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80319d0 con 0x7f0ff00b49f0 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 -- 172.21.15.148:0/2940605720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff00b49f0 msgr2=0x7f0ff006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff00b49f0 0x7f0ff006e2f0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b2850 tx=0x7f0fe802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 -- 172.21.15.148:0/2940605720 shutdown_connections 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff00b49f0 0x7f0ff006e2f0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/2940605720 >> v2:172.21.15.188:3300/0 conn(0x7f0ff006b8c0 0x7f0ff00b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/2940605720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff015dcd0 0x7f0ff006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 -- 172.21.15.148:0/2940605720 >> 172.21.15.148:0/2940605720 conn(0x7f0ff006abe0 msgr2=0x7f0ff006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 -- 172.21.15.148:0/2940605720 shutdown_connections 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.437+0000 7f0ff80b3640 1 -- 172.21.15.148:0/2940605720 wait complete. 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.438+0000 7f0ff80b3640 1 Processor -- start 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.438+0000 7f0ff80b3640 1 -- start start 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff006b8c0 0x7f0ff0151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff00b49f0 0x7f0ff00ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 0x7f0ff00cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff006dda0 con 0x7f0ff015dcd0 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff006dc20 con 0x7f0ff006b8c0 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff006e830 con 0x7f0ff00b49f0 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 0x7f0ff00cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 0x7f0ff00cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46286/0 (socket says 172.21.15.148:46286) 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 -- 172.21.15.148:0/424126142 learned_addr learned my addr 172.21.15.148:0/424126142 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff5e4f640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff006b8c0 0x7f0ff0151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff564e640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff00b49f0 0x7f0ff00ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 -- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff00b49f0 msgr2=0x7f0ff00ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff00b49f0 0x7f0ff00ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 -- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff006b8c0 msgr2=0x7f0ff0151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff006b8c0 0x7f0ff0151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 -- 172.21.15.148:0/424126142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0160820 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff5e4f640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff006b8c0 0x7f0ff0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff564e640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff00b49f0 0x7f0ff00ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff6650640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 0x7f0ff00cf2a0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009f40 tx=0x7f0fe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8041020 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8007820 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8036040 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.439+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff0160b00 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.440+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff00ccc50 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.440+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff0073a00 con 0x7f0ff015dcd0 2024-04-25T10:04:29.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.441+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fe8044020 con 0x7f0ff015dcd0 2024-04-25T10:04:29.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.441+0000 7f0fe67fc640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0fd406b4d0 0x7f0fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.442+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0fe80ab090 con 0x7f0ff015dcd0 2024-04-25T10:04:29.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.442+0000 7f0ff5e4f640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0fd406b4d0 0x7f0fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.442+0000 7f0ff5e4f640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0fd406b4d0 0x7f0fd406d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0fec006fd0 tx=0x7f0fec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.446+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fe804b030 con 0x7f0ff015dcd0 2024-04-25T10:04:29.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:29.616+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0ff00685d0 con 0x7f0ff015dcd0 2024-04-25T10:04:30.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.314+0000 7f0fe67fc640 1 -- 172.21.15.148:0/424126142 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v157) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0fe8079f60 con 0x7f0ff015dcd0 2024-04-25T10:04:30.319 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:04:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.317+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0fd406b4d0 msgr2=0x7f0fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.317+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0fd406b4d0 0x7f0fd406d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f0fec006fd0 tx=0x7f0fec008040 comp rx=0 tx=0).stop 2024-04-25T10:04:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.317+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 msgr2=0x7f0ff00cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.317+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 0x7f0ff00cf2a0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009f40 tx=0x7f0fe8004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 shutdown_connections 2024-04-25T10:04:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0fd406b4d0 0x7f0fd406d990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ff015dcd0 0x7f0ff00cf2a0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ff00b49f0 0x7f0ff00ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 --2- 172.21.15.148:0/424126142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ff006b8c0 0x7f0ff0151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 >> 172.21.15.148:0/424126142 conn(0x7f0ff006abe0 msgr2=0x7f0ff00acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 shutdown_connections 2024-04-25T10:04:30.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.318+0000 7f0ff80b3640 1 -- 172.21.15.148:0/424126142 wait complete. 2024-04-25T10:04:30.351 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:04:30.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.447+0000 7f6642ecb640 1 Processor -- start 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.447+0000 7f6642ecb640 1 -- start start 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6642ecb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f663c15dac0 0x7f663c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6642ecb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f663c0d1f30 0x7f663c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6642ecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6642ecb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0d3030 con 0x7f663c0d1f30 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6642ecb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0d2420 con 0x7f663c15dac0 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6642ecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0d2700 con 0x7f663c0d34c0 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f663bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f663c0d1f30 0x7f663c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f663bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f663c0d1f30 0x7f663c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46294/0 (socket says 172.21.15.148:46294) 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f663bfff640 1 -- 172.21.15.148:0/666018273 learned_addr learned my addr 172.21.15.148:0/666018273 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6641468640 1 --2- 172.21.15.148:0/666018273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.448+0000 7f6640c67640 1 --2- 172.21.15.148:0/666018273 >> v2:172.21.15.188:3300/0 conn(0x7f663c15dac0 0x7f663c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663bfff640 1 -- 172.21.15.148:0/666018273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 msgr2=0x7f663c0d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663bfff640 1 --2- 172.21.15.148:0/666018273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663bfff640 1 -- 172.21.15.148:0/666018273 >> v2:172.21.15.188:3300/0 conn(0x7f663c15dac0 msgr2=0x7f663c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663bfff640 1 --2- 172.21.15.148:0/666018273 >> v2:172.21.15.188:3300/0 conn(0x7f663c15dac0 0x7f663c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663bfff640 1 -- 172.21.15.148:0/666018273 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0d29e0 con 0x7f663c0d1f30 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f6640c67640 1 --2- 172.21.15.148:0/666018273 >> v2:172.21.15.188:3300/0 conn(0x7f663c15dac0 0x7f663c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663bfff640 1 --2- 172.21.15.148:0/666018273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c0d1f30 0x7f663c0d03a0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f663c0d2910 tx=0x7f663002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb37634b75f6384 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663b7fe640 1 -- 172.21.15.148:0/666018273 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630004030 con 0x7f663c0d1f30 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663b7fe640 1 -- 172.21.15.148:0/666018273 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66300041d0 con 0x7f663c0d1f30 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.449+0000 7f663b7fe640 1 -- 172.21.15.148:0/666018273 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630003610 con 0x7f663c0d1f30 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 -- 172.21.15.148:0/666018273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c0d1f30 msgr2=0x7f663c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 --2- 172.21.15.148:0/666018273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c0d1f30 0x7f663c0d03a0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f663c0d2910 tx=0x7f663002a250 comp rx=0 tx=0).stop 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 -- 172.21.15.148:0/666018273 shutdown_connections 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 --2- 172.21.15.148:0/666018273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 --2- 172.21.15.148:0/666018273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c0d1f30 0x7f663c0d03a0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 --2- 172.21.15.148:0/666018273 >> v2:172.21.15.188:3300/0 conn(0x7f663c15dac0 0x7f663c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 -- 172.21.15.148:0/666018273 >> 172.21.15.148:0/666018273 conn(0x7f663c061c30 msgr2=0x7f663c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 -- 172.21.15.148:0/666018273 shutdown_connections 2024-04-25T10:04:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.450+0000 7f6642ecb640 1 -- 172.21.15.148:0/666018273 wait complete. 2024-04-25T10:04:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.451+0000 7f6642ecb640 1 Processor -- start 2024-04-25T10:04:30.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.451+0000 7f6642ecb640 1 -- start start 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6642ecb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c0d1f30 0x7f663c0b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6642ecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6642ecb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 0x7f663c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6642ecb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b97d0 con 0x7f663c15dac0 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6642ecb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c15dea0 con 0x7f663c0d1f30 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6642ecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b9950 con 0x7f663c0d34c0 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 0x7f663c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 0x7f663c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46296/0 (socket says 172.21.15.148:46296) 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 -- 172.21.15.148:0/2677340790 learned_addr learned my addr 172.21.15.148:0/2677340790 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f663bfff640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6640c67640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c0d1f30 0x7f663c0b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 -- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 msgr2=0x7f663c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 -- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c0d1f30 msgr2=0x7f663c0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c0d1f30 0x7f663c0b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.452+0000 7f6641468640 1 -- 172.21.15.148:0/2677340790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c1f3550 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.453+0000 7f6641468640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 0x7f663c0cafe0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f6634007e40 tx=0x7f663400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.453+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634014620 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.453+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634019070 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.453+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c0b45b0 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.453+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634014a20 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.453+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c0b4b70 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.454+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c073a00 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.455+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6634014bc0 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.455+0000 7f66397fa640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f662806b4d0 0x7f662806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.455+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f663408aa80 con 0x7f663c15dac0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.455+0000 7f6640c67640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f662806b4d0 0x7f662806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.455+0000 7f6640c67640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f662806b4d0 0x7f662806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f663c1f6bf0 tx=0x7f662c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.459+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6634059210 con 0x7f663c15dac0 2024-04-25T10:04:30.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:30.665+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f663c0ca190 con 0x7f663c15dac0 2024-04-25T10:04:31.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.348+0000 7f66397fa640 1 -- 172.21.15.148:0/2677340790 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v91) v1 ==== 174+0+0 (secure 0 0 0) 0x7f663405d2b0 con 0x7f663c15dac0 2024-04-25T10:04:31.357 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 23 and data pool 24 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.352+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f662806b4d0 msgr2=0x7f662806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.352+0000 7f6642ecb640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f662806b4d0 0x7f662806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f663c1f6bf0 tx=0x7f662c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.352+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 msgr2=0x7f663c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.352+0000 7f6642ecb640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 0x7f663c0cafe0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f6634007e40 tx=0x7f663400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 shutdown_connections 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f662806b4d0 0x7f662806d990 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f663c15dac0 0x7f663c0cafe0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0d34c0 0x7f663c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 --2- 172.21.15.148:0/2677340790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c0d1f30 0x7f663c0b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 >> 172.21.15.148:0/2677340790 conn(0x7f663c061c30 msgr2=0x7f663c0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 shutdown_connections 2024-04-25T10:04:31.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.353+0000 7f6642ecb640 1 -- 172.21.15.148:0/2677340790 wait complete. 2024-04-25T10:04:31.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:31.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.469+0000 7ff6122b3640 1 Processor -- start 2024-04-25T10:04:31.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.469+0000 7ff6122b3640 1 -- start start 2024-04-25T10:04:31.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff6122b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c0ce920 0x7ff60c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff6122b3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff60c06d290 0x7ff60c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff6122b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c144b30 con 0x7ff60c144750 2024-04-25T10:04:31.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff6122b3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c0712d0 con 0x7ff60c06d290 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff6122b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0715b0 con 0x7ff60c0ce920 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff60c06d290 0x7ff60c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff60c06d290 0x7ff60c06d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46312/0 (socket says 172.21.15.148:46312) 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 -- 172.21.15.148:0/3770495069 learned_addr learned my addr 172.21.15.148:0/3770495069 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff60b7fe640 1 --2- 172.21.15.148:0/3770495069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c0ce920 0x7ff60c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 -- 172.21.15.148:0/3770495069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c0ce920 msgr2=0x7ff60c0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 --2- 172.21.15.148:0/3770495069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c0ce920 0x7ff60c0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 -- 172.21.15.148:0/3770495069 >> v1:172.21.15.188:6789/0 conn(0x7ff60c144750 legacy=0x7ff60c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:31.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.470+0000 7ff610850640 1 -- 172.21.15.148:0/3770495069 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c071890 con 0x7ff60c06d290 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff610850640 1 --2- 172.21.15.148:0/3770495069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c06d290 0x7ff60c06d720 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7ff604009e50 tx=0x7ff60402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817f35de07587c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff60affd640 1 -- 172.21.15.148:0/3770495069 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604036070 con 0x7ff60c06d290 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff60affd640 1 -- 172.21.15.148:0/3770495069 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604031600 con 0x7ff60c06d290 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff60affd640 1 -- 172.21.15.148:0/3770495069 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040319d0 con 0x7ff60c06d290 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff6122b3640 1 -- 172.21.15.148:0/3770495069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c06d290 msgr2=0x7ff60c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff6122b3640 1 --2- 172.21.15.148:0/3770495069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c06d290 0x7ff60c06d720 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7ff604009e50 tx=0x7ff60402a710 comp rx=0 tx=0).stop 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff6122b3640 1 -- 172.21.15.148:0/3770495069 shutdown_connections 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff6122b3640 1 --2- 172.21.15.148:0/3770495069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c06d290 0x7ff60c06d720 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.471+0000 7ff6122b3640 1 --2- 172.21.15.148:0/3770495069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c0ce920 0x7ff60c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.472+0000 7ff6122b3640 1 -- 172.21.15.148:0/3770495069 >> 172.21.15.148:0/3770495069 conn(0x7ff60c06abe0 msgr2=0x7ff60c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.472+0000 7ff6122b3640 1 -- 172.21.15.148:0/3770495069 shutdown_connections 2024-04-25T10:04:31.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.472+0000 7ff6122b3640 1 -- 172.21.15.148:0/3770495069 wait complete. 2024-04-25T10:04:31.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.473+0000 7ff6122b3640 1 Processor -- start 2024-04-25T10:04:31.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.473+0000 7ff6122b3640 1 -- start start 2024-04-25T10:04:31.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.473+0000 7ff6122b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c06d290 0x7ff60c1f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 0x7ff60c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60c144750 0x7ff60c13f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c06f020 con 0x7ff60c0ce920 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c06ece0 con 0x7ff60c144750 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c145450 con 0x7ff60c06d290 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 0x7ff60c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 0x7ff60c0b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46314/0 (socket says 172.21.15.148:46314) 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 -- 172.21.15.148:0/862615118 learned_addr learned my addr 172.21.15.148:0/862615118 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff610850640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60c144750 0x7ff60c13f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60bfff640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c06d290 0x7ff60c1f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 -- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c06d290 msgr2=0x7ff60c1f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c06d290 0x7ff60c1f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 -- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60c144750 msgr2=0x7ff60c13f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60c144750 0x7ff60c13f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 -- 172.21.15.148:0/862615118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c1f7790 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff60b7fe640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 0x7ff60c0b2e20 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7ff60000d800 tx=0x7ff60000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000b6c0 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600019070 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1f7a70 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000ba80 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.474+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c0b0310 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.475+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c073990 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.476+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff60000bc20 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.476+0000 7ff608ff9640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5f806b4d0 0x7ff5f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.476+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff60008a870 con 0x7ff60c0ce920 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.476+0000 7ff60bfff640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5f806b4d0 0x7ff5f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.477+0000 7ff60bfff640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5f806b4d0 0x7ff5f806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc001640 tx=0x7ff5fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.480+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff600059560 con 0x7ff60c0ce920 2024-04-25T10:04:31.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.651+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff60c13bba0 con 0x7ff60c0ce920 2024-04-25T10:04:31.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.652+0000 7ff608ff9640 1 -- 172.21.15.148:0/862615118 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16945 (secure 0 0 0) 0x7ff600058e60 con 0x7ff60c0ce920 2024-04-25T10:04:31.658 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:31.658 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":158,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:30.666915+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5f806b4d0 msgr2=0x7ff5f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5f806b4d0 0x7ff5f806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc001640 tx=0x7ff5fc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 msgr2=0x7ff60c0b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 0x7ff60c0b2e20 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7ff60000d800 tx=0x7ff60000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 shutdown_connections 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5f806b4d0 0x7ff5f806d990 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60c144750 0x7ff60c13f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff60c0ce920 0x7ff60c0b2e20 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 --2- 172.21.15.148:0/862615118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff60c06d290 0x7ff60c1f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 >> 172.21.15.148:0/862615118 conn(0x7ff60c06abe0 msgr2=0x7ff60c140250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 shutdown_connections 2024-04-25T10:04:31.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.654+0000 7ff6122b3640 1 -- 172.21.15.148:0/862615118 wait complete. 2024-04-25T10:04:31.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:31.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.764+0000 7f7e6306b640 1 Processor -- start 2024-04-25T10:04:31.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.764+0000 7f7e6306b640 1 -- start start 2024-04-25T10:04:31.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e6306b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c13d190 0x7f7e5c0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e6306b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7e5c15b320 0x7f7e5c0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e6306b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c071ed0 con 0x7f7e5c071230 2024-04-25T10:04:31.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e6306b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c13d570 con 0x7f7e5c15b320 2024-04-25T10:04:31.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e6306b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c15a840 con 0x7f7e5c13d190 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7e5c15b320 0x7f7e5c0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7e5c15b320 0x7f7e5c0cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46320/0 (socket says 172.21.15.148:46320) 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e61608640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7e5c071230 0x7f7e5c0716c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49492/0 (socket says 172.21.15.148:49492) 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3865125655 learned_addr learned my addr 172.21.15.148:0/3865125655 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e60e07640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c13d190 0x7f7e5c0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3865125655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c13d190 msgr2=0x7f7e5c0cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c13d190 0x7f7e5c0cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3865125655 >> v1:172.21.15.148:6789/0 conn(0x7f7e5c071230 legacy=0x7f7e5c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3865125655 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c15ab20 con 0x7f7e5c15b320 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e5b7fe640 1 -- 172.21.15.148:0/3865125655 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385853651 0 0) 0x7f7e5c071ed0 con 0x7f7e5c071230 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.765+0000 7f7e60e07640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c13d190 0x7f7e5c0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.766+0000 7f7e5bfff640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c15b320 0x7f7e5c0cc4f0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800bb00 tx=0x7f7e4802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b74ee9ac5e8b4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.766+0000 7f7e5b7fe640 1 -- 172.21.15.148:0/3865125655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4803a070 con 0x7f7e5c15b320 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.766+0000 7f7e5b7fe640 1 -- 172.21.15.148:0/3865125655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e48032480 con 0x7f7e5c15b320 2024-04-25T10:04:31.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.766+0000 7f7e5b7fe640 1 -- 172.21.15.148:0/3865125655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48032850 con 0x7f7e5c15b320 2024-04-25T10:04:31.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 -- 172.21.15.148:0/3865125655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c15b320 msgr2=0x7f7e5c0cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c15b320 0x7f7e5c0cc4f0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800bb00 tx=0x7f7e4802c650 comp rx=0 tx=0).stop 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 -- 172.21.15.148:0/3865125655 shutdown_connections 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c15b320 0x7f7e5c0cc4f0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3865125655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c13d190 0x7f7e5c0cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 -- 172.21.15.148:0/3865125655 >> 172.21.15.148:0/3865125655 conn(0x7f7e5c06abe0 msgr2=0x7f7e5c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 -- 172.21.15.148:0/3865125655 shutdown_connections 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.767+0000 7f7e6306b640 1 -- 172.21.15.148:0/3865125655 wait complete. 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.768+0000 7f7e6306b640 1 Processor -- start 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.768+0000 7f7e6306b640 1 -- start start 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e5c071230 0x7f7e5c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 0x7f7e5c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c15b320 0x7f7e5c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0cf7a0 con 0x7f7e5c13d190 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0cf620 con 0x7f7e5c071230 2024-04-25T10:04:31.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0cf920 con 0x7f7e5c15b320 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 0x7f7e5c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 0x7f7e5c0b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46328/0 (socket says 172.21.15.148:46328) 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3395872985 learned_addr learned my addr 172.21.15.148:0/3395872985 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3395872985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c15b320 msgr2=0x7f7e5c0b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c15b320 0x7f7e5c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3395872985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e5c071230 msgr2=0x7f7e5c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e5c071230 0x7f7e5c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 -- 172.21.15.148:0/3395872985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c1f0300 con 0x7f7e5c13d190 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e5bfff640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 0x7f7e5c0b3a40 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800bc20 tx=0x7f7e480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48044020 con 0x7f7e5c13d190 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4803f670 con 0x7f7e5c13d190 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c1f05e0 con 0x7f7e5c13d190 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4803a040 con 0x7f7e5c13d190 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.769+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c0af5a0 con 0x7f7e5c13d190 2024-04-25T10:04:31.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.770+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c073990 con 0x7f7e5c13d190 2024-04-25T10:04:31.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.771+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e48047020 con 0x7f7e5c13d190 2024-04-25T10:04:31.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.771+0000 7f7e597fa640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e4006b4d0 0x7f7e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.771+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e480acf90 con 0x7f7e5c13d190 2024-04-25T10:04:31.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.771+0000 7f7e60e07640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e4006b4d0 0x7f7e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.772+0000 7f7e60e07640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e4006b4d0 0x7f7e4006d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006e80 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.776+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e4804e030 con 0x7f7e5c13d190 2024-04-25T10:04:31.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.948+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e5c0cfeb0 con 0x7f7e5c13d190 2024-04-25T10:04:31.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.949+0000 7f7e597fa640 1 -- 172.21.15.148:0/3395872985 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f7e4807bda0 con 0x7f7e5c13d190 2024-04-25T10:04:31.954 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:31.954 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":158,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:30.666915+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:31.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e4006b4d0 msgr2=0x7f7e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e4006b4d0 0x7f7e4006d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006e80 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2024-04-25T10:04:31.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 msgr2=0x7f7e5c0b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 0x7f7e5c0b3a40 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800bc20 tx=0x7f7e480046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 shutdown_connections 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7e4006b4d0 0x7f7e4006d990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e5c15b320 0x7f7e5c0b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e5c13d190 0x7f7e5c0b3a40 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 --2- 172.21.15.148:0/3395872985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e5c071230 0x7f7e5c0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 >> 172.21.15.148:0/3395872985 conn(0x7f7e5c06abe0 msgr2=0x7f7e5c13c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 shutdown_connections 2024-04-25T10:04:31.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:31.951+0000 7f7e6306b640 1 -- 172.21.15.148:0/3395872985 wait complete. 2024-04-25T10:04:31.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:04:32.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.063+0000 7fb91c21f640 1 Processor -- start 2024-04-25T10:04:32.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.063+0000 7fb91c21f640 1 -- start start 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb91c21f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb91406ccd0 0x7fb9140b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb91c21f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9140d1770 con 0x7fb91415deb0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb91c21f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb914162740 con 0x7fb914070dc0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb91c21f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9141622b0 con 0x7fb91406ccd0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb91a7bc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb91415deb0 0x7fb91415e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49498/0 (socket says 172.21.15.148:49498) 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb91a7bc640 1 -- 172.21.15.148:0/4273750665 learned_addr learned my addr 172.21.15.148:0/4273750665 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb9197ba640 1 --2- 172.21.15.148:0/4273750665 >> v2:172.21.15.188:3300/0 conn(0x7fb91406ccd0 0x7fb9140b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180667686 0 0) 0x7fb9140d1770 con 0x7fb91415deb0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.064+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb900003520 con 0x7fb91415deb0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 371620109 0 0) 0x7fb900003520 con 0x7fb91415deb0 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 >> v2:172.21.15.188:3300/0 conn(0x7fb91406ccd0 msgr2=0x7fb9140b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 --2- 172.21.15.148:0/4273750665 >> v2:172.21.15.188:3300/0 conn(0x7fb91406ccd0 0x7fb9140b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 >> v1:172.21.15.188:6789/0 conn(0x7fb914070dc0 legacy=0x7fb9140711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb914161980 con 0x7fb91415deb0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb90c0019f0 con 0x7fb91415deb0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 >> v1:172.21.15.148:6789/0 conn(0x7fb91415deb0 legacy=0x7fb91415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9000039b0 0x7fb900003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb900004800 0x7fb900004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9000052e0 0x7fb90000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141622b0 con 0x7fb9000039b0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90000c1b0 con 0x7fb900004800 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90000c3b0 con 0x7fb9000052e0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb919fbb640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9000039b0 0x7fb900003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb9197ba640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb900004800 0x7fb900004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb91a7bc640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9000052e0 0x7fb90000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb919fbb640 1 -- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9000052e0 msgr2=0x7fb90000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb919fbb640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9000052e0 0x7fb90000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb919fbb640 1 -- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb900004800 msgr2=0x7fb900004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:32.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb919fbb640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb900004800 0x7fb900004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb919fbb640 1 -- 172.21.15.148:0/4273750665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900003520 con 0x7fb9000039b0 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.065+0000 7fb91a7bc640 1 -- 172.21.15.148:0/4273750665 reap_dead start 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.066+0000 7fb919fbb640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9000039b0 0x7fb900003dd0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fb910001720 tx=0x7fb900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaf8c8915abc9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.066+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910001370 con 0x7fb9000039b0 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.066+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9100052d0 con 0x7fb9000039b0 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.066+0000 7fb918fb9640 1 -- 172.21.15.148:0/4273750665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91002ab70 con 0x7fb9000039b0 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 -- 172.21.15.148:0/4273750665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9000039b0 msgr2=0x7fb900003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9000039b0 0x7fb900003dd0 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fb910001720 tx=0x7fb900005830 comp rx=0 tx=0).stop 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 -- 172.21.15.148:0/4273750665 shutdown_connections 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 --2- 172.21.15.148:0/4273750665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9000039b0 0x7fb900003dd0 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 -- 172.21.15.148:0/4273750665 >> 172.21.15.148:0/4273750665 conn(0x7fb9140b3530 msgr2=0x7fb9140b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 -- 172.21.15.148:0/4273750665 shutdown_connections 2024-04-25T10:04:32.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.067+0000 7fb91c21f640 1 -- 172.21.15.148:0/4273750665 wait complete. 2024-04-25T10:04:32.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.068+0000 7fb91c21f640 1 Processor -- start 2024-04-25T10:04:32.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.068+0000 7fb91c21f640 1 -- start start 2024-04-25T10:04:32.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9000039b0 0x7fb9141f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 0x7fb91415ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb91414f4b0 0x7fb91415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb914161f40 con 0x7fb91414fa30 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140b10f0 con 0x7fb9000039b0 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140711a0 con 0x7fb91414f4b0 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 0x7fb91415ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 0x7fb91415ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46344/0 (socket says 172.21.15.148:46344) 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 -- 172.21.15.148:0/2547218900 learned_addr learned my addr 172.21.15.148:0/2547218900 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb919fbb640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9000039b0 0x7fb9141f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91a7bc640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb91414f4b0 0x7fb91415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 -- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb91414f4b0 msgr2=0x7fb91415b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb91414f4b0 0x7fb91415b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 -- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9000039b0 msgr2=0x7fb9141f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9000039b0 0x7fb9141f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 -- 172.21.15.148:0/2547218900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140cc850 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91a7bc640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb91414f4b0 0x7fb91415b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb919fbb640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9000039b0 0x7fb9141f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb9197ba640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 0x7fb91415ae00 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fb904011aa0 tx=0x7fb904011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400fb80 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90400fd20 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9140cd8b0 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904014660 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.069+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9140ccdd0 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.070+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb914073a00 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.071+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb904012040 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.071+0000 7fb90a7fc640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.071+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb90408a9e0 con 0x7fb91414fa30 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.071+0000 7fb919fbb640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.071+0000 7fb919fbb640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f806b4d0 0x7fb8f806d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb910032000 tx=0x7fb91002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:32.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.075+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb904019030 con 0x7fb91414fa30 2024-04-25T10:04:32.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.281+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb9140cf880 con 0x7fb91414fa30 2024-04-25T10:04:32.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.344+0000 7fb90a7fc640 1 -- 172.21.15.148:0/2547218900 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v92) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb9040596d0 con 0x7fb91414fa30 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f806b4d0 msgr2=0x7fb8f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f806b4d0 0x7fb8f806d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb910032000 tx=0x7fb91002e000 comp rx=0 tx=0).stop 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 msgr2=0x7fb91415ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 0x7fb91415ae00 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fb904011aa0 tx=0x7fb904011f60 comp rx=0 tx=0).stop 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 shutdown_connections 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb91414f4b0 0x7fb91415b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb91414fa30 0x7fb91415ae00 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 --2- 172.21.15.148:0/2547218900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9000039b0 0x7fb9141f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 >> 172.21.15.148:0/2547218900 conn(0x7fb9140b3530 msgr2=0x7fb9140cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 shutdown_connections 2024-04-25T10:04:32.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.346+0000 7fb91c21f640 1 -- 172.21.15.148:0/2547218900 wait complete. 2024-04-25T10:04:32.363 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:04:32.364 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:04:32.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.458+0000 7f43cb5d5640 1 Processor -- start 2024-04-25T10:04:32.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.458+0000 7f43cb5d5640 1 -- start start 2024-04-25T10:04:32.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.459+0000 7f43cb5d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43c415bb50 0x7f43c40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.459+0000 7f43cb5d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c40c70d0 0x7f43c40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.459+0000 7f43cb5d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c40c75e0 con 0x7f43c40cfb40 2024-04-25T10:04:32.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.459+0000 7f43cb5d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c40c8680 con 0x7f43c415bb50 2024-04-25T10:04:32.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.459+0000 7f43cb5d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c78c0 con 0x7f43c40c70d0 2024-04-25T10:04:32.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9371640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43c415bb50 0x7f43c40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c8b70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c40c70d0 0x7f43c40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9b72640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f43c40cfb40 0x7f43c4070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:51598/0 (socket says 172.21.15.148:51598) 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9b72640 1 -- 172.21.15.148:0/1240946723 learned_addr learned my addr 172.21.15.148:0/1240946723 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9371640 1 -- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c40c70d0 msgr2=0x7f43c40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9371640 1 --2- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c40c70d0 0x7f43c40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9371640 1 -- 172.21.15.148:0/1240946723 >> v1:172.21.15.188:6789/0 conn(0x7f43c40cfb40 legacy=0x7f43c4070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c9371640 1 -- 172.21.15.148:0/1240946723 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40c7ba0 con 0x7f43c415bb50 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43bbfff640 1 -- 172.21.15.148:0/1240946723 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092590398 0 0) 0x7f43c40c75e0 con 0x7f43c40cfb40 2024-04-25T10:04:32.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.460+0000 7f43c8b70640 1 --2- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c40c70d0 0x7f43c40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.461+0000 7f43c9371640 1 --2- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c415bb50 0x7f43c40c6b90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009d90 tx=0x7f43c002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08683e25c62cfd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.461+0000 7f43bbfff640 1 -- 172.21.15.148:0/1240946723 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0007740 con 0x7f43c415bb50 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.461+0000 7f43bbfff640 1 -- 172.21.15.148:0/1240946723 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c00078e0 con 0x7f43c415bb50 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.461+0000 7f43bbfff640 1 -- 172.21.15.148:0/1240946723 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0007cb0 con 0x7f43c415bb50 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.461+0000 7f43cb5d5640 1 -- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c415bb50 msgr2=0x7f43c40c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.461+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c415bb50 0x7f43c40c6b90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009d90 tx=0x7f43c002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.462+0000 7f43cb5d5640 1 -- 172.21.15.148:0/1240946723 shutdown_connections 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.462+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c40c70d0 0x7f43c40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.462+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/1240946723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c415bb50 0x7f43c40c6b90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.462+0000 7f43cb5d5640 1 -- 172.21.15.148:0/1240946723 >> 172.21.15.148:0/1240946723 conn(0x7f43c406f500 msgr2=0x7f43c406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.462+0000 7f43cb5d5640 1 -- 172.21.15.148:0/1240946723 shutdown_connections 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.462+0000 7f43cb5d5640 1 -- 172.21.15.148:0/1240946723 wait complete. 2024-04-25T10:04:32.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 Processor -- start 2024-04-25T10:04:32.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 -- start start 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c40c70d0 0x7f43c415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 0x7f43c40b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c415bb50 0x7f43c40b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c415f6d0 con 0x7f43c40cfb40 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c40c8160 con 0x7f43c40c70d0 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.463+0000 7f43cb5d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c415f850 con 0x7f43c415bb50 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 0x7f43c40b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 0x7f43c40b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:46354/0 (socket says 172.21.15.148:46354) 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 -- 172.21.15.148:0/317026809 learned_addr learned my addr 172.21.15.148:0/317026809 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:32.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c9371640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c40c70d0 0x7f43c415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c9b72640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c415bb50 0x7f43c40b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 -- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c415bb50 msgr2=0x7f43c40b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c415bb50 0x7f43c40b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 -- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c40c70d0 msgr2=0x7f43c415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c40c70d0 0x7f43c415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 -- 172.21.15.148:0/317026809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c406da20 con 0x7f43c40cfb40 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c9b72640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c415bb50 0x7f43c40b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43c8b70640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 0x7f43c40b00b0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f43b400e910 tx=0x7f43b400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4017070 con 0x7f43c40cfb40 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b400ca00 con 0x7f43c40cfb40 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400cdb0 con 0x7f43c40cfb40 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.464+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c40c9a70 con 0x7f43c40cfb40 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.465+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c406d660 con 0x7f43c40cfb40 2024-04-25T10:04:32.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.465+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c4073a00 con 0x7f43c40cfb40 2024-04-25T10:04:32.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.466+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43b401a070 con 0x7f43c40cfb40 2024-04-25T10:04:32.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.466+0000 7f43b9ffb640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f43a806b4d0 0x7f43a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:32.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.466+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43b408a220 con 0x7f43c40cfb40 2024-04-25T10:04:32.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.466+0000 7f43c9371640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f43a806b4d0 0x7f43a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:32.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.467+0000 7f43c9371640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f43a806b4d0 0x7f43a806d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f43c0034000 tx=0x7f43c002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:32.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.471+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43b4058fa0 con 0x7f43c40cfb40 2024-04-25T10:04:32.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:32.674+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f43c40cf150 con 0x7f43c40cfb40 2024-04-25T10:04:33.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.355+0000 7f43b9ffb640 1 -- 172.21.15.148:0/317026809 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v93) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43b40588a0 con 0x7f43c40cfb40 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f43a806b4d0 msgr2=0x7f43a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f43a806b4d0 0x7f43a806d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f43c0034000 tx=0x7f43c002f000 comp rx=0 tx=0).stop 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 msgr2=0x7f43c40b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 0x7f43c40b00b0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f43b400e910 tx=0x7f43b400edd0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 shutdown_connections 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f43a806b4d0 0x7f43a806d990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43c415bb50 0x7f43c40b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43c40cfb40 0x7f43c40b00b0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 --2- 172.21.15.148:0/317026809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43c40c70d0 0x7f43c415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.357+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 >> 172.21.15.148:0/317026809 conn(0x7f43c406f500 msgr2=0x7f43c40cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.358+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 shutdown_connections 2024-04-25T10:04:33.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.358+0000 7f43cb5d5640 1 -- 172.21.15.148:0/317026809 wait complete. 2024-04-25T10:04:33.375 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:04:33.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:33.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.471+0000 7fa3b9d09640 1 Processor -- start 2024-04-25T10:04:33.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.471+0000 7fa3b9d09640 1 -- start start 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b9d09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 0x7fa3b40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b9d09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b406c220 con 0x7fa3b409a450 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b9d09640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b40d0080 con 0x7fa3b406ce10 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b9d09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b409a8b0 con 0x7fa3b415dc70 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa3b409a450 0x7fa3b406c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49504/0 (socket says 172.21.15.148:49504) 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b2ffd640 1 -- 172.21.15.148:0/2550837538 learned_addr learned my addr 172.21.15.148:0/2550837538 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843466676 0 0) 0x7fa3b406c220 con 0x7fa3b409a450 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 0x7fa3b40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.472+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa39c003520 con 0x7fa3b409a450 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2563298985 0 0) 0x7fa39c003520 con 0x7fa3b409a450 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 msgr2=0x7fa3b40d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 0x7fa3b40d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 >> v1:172.21.15.188:6789/0 conn(0x7fa3b406ce10 legacy=0x7fa3b40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b40d0360 con 0x7fa3b409a450 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa3a8001670 con 0x7fa3b409a450 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 >> v1:172.21.15.148:6789/0 conn(0x7fa3b409a450 legacy=0x7fa3b406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa39c0039e0 0x7fa39c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 0x7fa39c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39c004d40 0x7fa39c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b409a8b0 con 0x7fa39c0039e0 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c00ba70 con 0x7fa39c004d40 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c00bc70 con 0x7fa3b415dc70 2024-04-25T10:04:33.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa39c0039e0 0x7fa39c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b2ffd640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 0x7fa39c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b3fff640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39c004d40 0x7fa39c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b37fe640 1 -- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 msgr2=0x7fa39c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3b415dc70 0x7fa39c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b37fe640 1 -- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39c004d40 msgr2=0x7fa39c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39c004d40 0x7fa39c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b37fe640 1 -- 172.21.15.148:0/2550837538 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c003520 con 0x7fa39c0039e0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.473+0000 7fa3b3fff640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa39c004d40 0x7fa39c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.474+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa39c0039e0 0x7fa39c003e00 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009f90 tx=0x7fa39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f820f84b989a38fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.474+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4009c10 con 0x7fa39c0039e0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.474+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a4036070 con 0x7fa39c0039e0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.474+0000 7fa3b27fc640 1 -- 172.21.15.148:0/2550837538 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4031850 con 0x7fa39c0039e0 2024-04-25T10:04:33.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.474+0000 7fa3b9d09640 1 -- 172.21.15.148:0/2550837538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa39c0039e0 msgr2=0x7fa39c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.474+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/2550837538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa39c0039e0 0x7fa39c003e00 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009f90 tx=0x7fa39c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.475+0000 7fa3b3fff640 1 -- 172.21.15.148:0/2550837538 reap_dead start 2024-04-25T10:04:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.475+0000 7fa3b9d09640 1 -- 172.21.15.148:0/2550837538 shutdown_connections 2024-04-25T10:04:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.475+0000 7fa3b9d09640 1 -- 172.21.15.148:0/2550837538 >> 172.21.15.148:0/2550837538 conn(0x7fa3b40b3270 msgr2=0x7fa3b40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.475+0000 7fa3b9d09640 1 -- 172.21.15.148:0/2550837538 shutdown_connections 2024-04-25T10:04:33.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.475+0000 7fa3b9d09640 1 -- 172.21.15.148:0/2550837538 wait complete. 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.476+0000 7fa3b9d09640 1 Processor -- start 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.476+0000 7fa3b9d09640 1 -- start start 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa39c0039e0 0x7fa3b40c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 0x7fa3b41f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3b40c7d80 0x7fa3b40cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40d0920 con 0x7fa3b4158870 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40d0c90 con 0x7fa3b40c7d80 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b415e050 con 0x7fa39c0039e0 2024-04-25T10:04:33.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 0x7fa3b41f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 0x7fa3b41f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41252/0 (socket says 172.21.15.148:41252) 2024-04-25T10:04:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 -- 172.21.15.148:0/979288555 learned_addr learned my addr 172.21.15.148:0/979288555 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b3fff640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3b40c7d80 0x7fa3b40cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa39c0039e0 0x7fa3b40c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 -- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa39c0039e0 msgr2=0x7fa3b40c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa39c0039e0 0x7fa3b40c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 -- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3b40c7d80 msgr2=0x7fa3b40cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3b40c7d80 0x7fa3b40cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 -- 172.21.15.148:0/979288555 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b41f7af0 con 0x7fa3b4158870 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b2ffd640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 0x7fa3b41f75b0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80072c0 tx=0x7fa3a80076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800f960 con 0x7fa3b4158870 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a8018070 con 0x7fa3b4158870 2024-04-25T10:04:33.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b40c5f50 con 0x7fa3b4158870 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800fd60 con 0x7fa3b4158870 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.477+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b41f6db0 con 0x7fa3b4158870 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.478+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b4073990 con 0x7fa3b4158870 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.479+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3a801d460 con 0x7fa3b4158870 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.479+0000 7fa39bfff640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a006b4d0 0x7fa3a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.479+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3a808a240 con 0x7fa3b4158870 2024-04-25T10:04:33.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.479+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a006b4d0 0x7fa3a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.480+0000 7fa3b37fe640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a006b4d0 0x7fa3a006d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4035000 tx=0x7fa3a402f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:33.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.483+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3a8058fc0 con 0x7fa3b4158870 2024-04-25T10:04:33.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.687+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3b40b40a0 con 0x7fa3b4158870 2024-04-25T10:04:33.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.688+0000 7fa39bfff640 1 -- 172.21.15.148:0/979288555 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fa3a80588c0 con 0x7fa3b4158870 2024-04-25T10:04:33.693 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:33.693 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:30.667288+0000","modified":"2024-04-25T10:04:32.676174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10365":{"gid":10365,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3286540466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3286540466},{"type":"v1","addr":"172.21.15.148:6837","nonce":3286540466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:04:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 93 2024-04-25T10:04:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a006b4d0 msgr2=0x7fa3a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a006b4d0 0x7fa3a006d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4035000 tx=0x7fa3a402f000 comp rx=0 tx=0).stop 2024-04-25T10:04:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 msgr2=0x7fa3b41f75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 0x7fa3b41f75b0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80072c0 tx=0x7fa3a80076c0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 shutdown_connections 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3a006b4d0 0x7fa3a006d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3b40c7d80 0x7fa3b40cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3b4158870 0x7fa3b41f75b0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 --2- 172.21.15.148:0/979288555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa39c0039e0 0x7fa3b40c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 >> 172.21.15.148:0/979288555 conn(0x7fa3b40b3270 msgr2=0x7fa3b40b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 shutdown_connections 2024-04-25T10:04:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.690+0000 7fa3b9d09640 1 -- 172.21.15.148:0/979288555 wait complete. 2024-04-25T10:04:33.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.799+0000 7fe983ec4640 1 Processor -- start 2024-04-25T10:04:33.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.799+0000 7fe983ec4640 1 -- start start 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe983ec4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe97c144750 0x7fe97c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe983ec4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe97c0cf030 0x7fe97c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe983ec4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97c144b30 con 0x7fe97c06eb10 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe983ec4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c0712d0 con 0x7fe97c144750 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe983ec4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c0715b0 con 0x7fe97c0cf030 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe97c144750 0x7fe97c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe97c144750 0x7fe97c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41254/0 (socket says 172.21.15.148:41254) 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 -- 172.21.15.148:0/449164953 learned_addr learned my addr 172.21.15.148:0/449164953 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe982461640 1 --2- 172.21.15.148:0/449164953 >> v2:172.21.15.188:3300/0 conn(0x7fe97c0cf030 0x7fe97c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 -- 172.21.15.148:0/449164953 >> v2:172.21.15.188:3300/0 conn(0x7fe97c0cf030 msgr2=0x7fe97c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:33.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 --2- 172.21.15.148:0/449164953 >> v2:172.21.15.188:3300/0 conn(0x7fe97c0cf030 0x7fe97c06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 -- 172.21.15.148:0/449164953 >> v1:172.21.15.188:6789/0 conn(0x7fe97c06eb10 legacy=0x7fe97c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe981c60640 1 -- 172.21.15.148:0/449164953 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c071890 con 0x7fe97c144750 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.800+0000 7fe982461640 1 --2- 172.21.15.148:0/449164953 >> v2:172.21.15.188:3300/0 conn(0x7fe97c0cf030 0x7fe97c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe981c60640 1 --2- 172.21.15.148:0/449164953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c144750 0x7fe97c06e5d0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fe97c06f720 tx=0x7fe97802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8103351b6f1c7aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe980c5e640 1 -- 172.21.15.148:0/449164953 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978037070 con 0x7fe97c144750 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe980c5e640 1 -- 172.21.15.148:0/449164953 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978032440 con 0x7fe97c144750 2024-04-25T10:04:33.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe980c5e640 1 -- 172.21.15.148:0/449164953 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978032810 con 0x7fe97c144750 2024-04-25T10:04:33.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 -- 172.21.15.148:0/449164953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c144750 msgr2=0x7fe97c06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 --2- 172.21.15.148:0/449164953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c144750 0x7fe97c06e5d0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fe97c06f720 tx=0x7fe97802a270 comp rx=0 tx=0).stop 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 -- 172.21.15.148:0/449164953 shutdown_connections 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 --2- 172.21.15.148:0/449164953 >> v2:172.21.15.188:3300/0 conn(0x7fe97c0cf030 0x7fe97c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 --2- 172.21.15.148:0/449164953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c144750 0x7fe97c06e5d0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 -- 172.21.15.148:0/449164953 >> 172.21.15.148:0/449164953 conn(0x7fe97c06abe0 msgr2=0x7fe97c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.801+0000 7fe983ec4640 1 -- 172.21.15.148:0/449164953 shutdown_connections 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.802+0000 7fe983ec4640 1 -- 172.21.15.148:0/449164953 wait complete. 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 Processor -- start 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 -- start start 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 0x7fe97c1f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97c0cf030 0x7fe97c13c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97c144750 0x7fe97c140d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1455d0 con 0x7fe97c06eb10 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c145450 con 0x7fe97c144750 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe983ec4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c15c000 con 0x7fe97c0cf030 2024-04-25T10:04:33.812 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe981c60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 0x7fe97c1f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe981c60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 0x7fe97c1f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41264/0 (socket says 172.21.15.148:41264) 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.803+0000 7fe981c60640 1 -- 172.21.15.148:0/4199561919 learned_addr learned my addr 172.21.15.148:0/4199561919 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe981c60640 1 -- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97c0cf030 msgr2=0x7fe97c13c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe982461640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97c144750 0x7fe97c140d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe98145f640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97c0cf030 0x7fe97c13c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe981c60640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97c0cf030 0x7fe97c13c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe981c60640 1 -- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97c144750 msgr2=0x7fe97c140d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe981c60640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97c144750 0x7fe97c140d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe981c60640 1 -- 172.21.15.148:0/4199561919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c13ce60 con 0x7fe97c06eb10 2024-04-25T10:04:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe98145f640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97c0cf030 0x7fe97c13c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe982461640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97c144750 0x7fe97c140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe981c60640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 0x7fe97c1f4a30 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fe978000c00 tx=0x7fe9780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:33.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978041020 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780306c0 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c13d140 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978037040 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.804+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c15e2b0 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.805+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c073990 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.806+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe978044020 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.806+0000 7fe96e7fc640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96006b500 0x7fe96006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.806+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9780aab50 con 0x7fe97c06eb10 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.806+0000 7fe98145f640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96006b500 0x7fe96006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:33.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.806+0000 7fe98145f640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96006b500 0x7fe96006d9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe97c15f020 tx=0x7fe970009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:33.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.810+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe97804b030 con 0x7fe97c06eb10 2024-04-25T10:04:33.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.981+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe97c0b06e0 con 0x7fe97c06eb10 2024-04-25T10:04:33.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.982+0000 7fe96e7fc640 1 -- 172.21.15.148:0/4199561919 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16932 (secure 0 0 0) 0x7fe97807f020 con 0x7fe97c06eb10 2024-04-25T10:04:33.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:33.988 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":159,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:32.326979+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96006b500 msgr2=0x7fe96006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96006b500 0x7fe96006d9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe97c15f020 tx=0x7fe970009040 comp rx=0 tx=0).stop 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 msgr2=0x7fe97c1f4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 0x7fe97c1f4a30 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fe978000c00 tx=0x7fe9780046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 shutdown_connections 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe96006b500 0x7fe96006d9c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97c144750 0x7fe97c140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe97c0cf030 0x7fe97c13c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 --2- 172.21.15.148:0/4199561919 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe97c06eb10 0x7fe97c1f4a30 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 >> 172.21.15.148:0/4199561919 conn(0x7fe97c06abe0 msgr2=0x7fe97c0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:33.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 shutdown_connections 2024-04-25T10:04:33.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:33.984+0000 7fe983ec4640 1 -- 172.21.15.148:0/4199561919 wait complete. 2024-04-25T10:04:34.001 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:04:34.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.094+0000 7ffad7fab640 1 Processor -- start 2024-04-25T10:04:34.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.094+0000 7ffad7fab640 1 -- start start 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad7fab640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffad015ef00 0x7ffad00d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad7fab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffad00cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad7fab640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad00c7970 con 0x7ffad00d1ca0 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad7fab640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad00d2110 con 0x7ffad015ef00 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad7fab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015f2e0 con 0x7ffad00c7310 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad5546640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffad00d1ca0 0x7ffad00c6c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36894/0 (socket says 172.21.15.148:36894) 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad5546640 1 -- 172.21.15.148:0/1807079138 learned_addr learned my addr 172.21.15.148:0/1807079138 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad5d47640 1 --2- 172.21.15.148:0/1807079138 >> v2:172.21.15.188:3300/0 conn(0x7ffad015ef00 0x7ffad00d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad6548640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffad00cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931305546 0 0) 0x7ffad00c7970 con 0x7ffad00d1ca0 2024-04-25T10:04:34.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffabc003520 con 0x7ffad00d1ca0 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2845503947 0 0) 0x7ffabc003520 con 0x7ffad00d1ca0 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 msgr2=0x7ffad00cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffad00cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 >> v2:172.21.15.188:3300/0 conn(0x7ffad015ef00 msgr2=0x7ffad00d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 --2- 172.21.15.148:0/1807079138 >> v2:172.21.15.188:3300/0 conn(0x7ffad015ef00 0x7ffad00d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00c7c50 con 0x7ffad00d1ca0 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ffac0001690 con 0x7ffad00d1ca0 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad6548640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffad00cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 >> v1:172.21.15.148:6789/0 conn(0x7ffad00d1ca0 legacy=0x7ffad00c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc0039e0 0x7ffabc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc004800 0x7ffabc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffabc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00d2110 con 0x7ffabc004800 2024-04-25T10:04:34.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad015f2e0 con 0x7ffabc0039e0 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.095+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc00b9c0 con 0x7ffad00c7310 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5546640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc004800 0x7ffabc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5d47640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc0039e0 0x7ffabc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad6548640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffabc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5546640 1 -- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 msgr2=0x7ffabc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5546640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad00c7310 0x7ffabc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5546640 1 -- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc0039e0 msgr2=0x7ffabc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5546640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc0039e0 0x7ffabc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.096+0000 7ffad5546640 1 -- 172.21.15.148:0/1807079138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc003520 con 0x7ffabc004800 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad5546640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc004800 0x7ffabc004c80 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7ffac0007fd0 tx=0x7ffabc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cf0078cf14680d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.105 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0007d50 con 0x7ffabc004800 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac00058d0 con 0x7ffabc004800 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad4d45640 1 -- 172.21.15.148:0/1807079138 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0005ca0 con 0x7ffabc004800 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad7fab640 1 -- 172.21.15.148:0/1807079138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc004800 msgr2=0x7ffabc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad7fab640 1 --2- 172.21.15.148:0/1807079138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffabc004800 0x7ffabc004c80 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7ffac0007fd0 tx=0x7ffabc005830 comp rx=0 tx=0).stop 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad6548640 1 -- 172.21.15.148:0/1807079138 reap_dead start 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad7fab640 1 -- 172.21.15.148:0/1807079138 shutdown_connections 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad7fab640 1 -- 172.21.15.148:0/1807079138 >> 172.21.15.148:0/1807079138 conn(0x7ffad006f5d0 msgr2=0x7ffad006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad7fab640 1 -- 172.21.15.148:0/1807079138 shutdown_connections 2024-04-25T10:04:34.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.097+0000 7ffad7fab640 1 -- 172.21.15.148:0/1807079138 wait complete. 2024-04-25T10:04:34.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.098+0000 7ffad7fab640 1 Processor -- start 2024-04-25T10:04:34.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.098+0000 7ffad7fab640 1 -- start start 2024-04-25T10:04:34.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad7fab640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc004800 0x7ffad00b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad7fab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 0x7ffad01529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad7fab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad0154500 0x7ffad01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad7fab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00c84f0 con 0x7ffad00b6520 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad7fab640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00c8210 con 0x7ffabc004800 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad7fab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00cc270 con 0x7ffad0154500 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 0x7ffad01529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 0x7ffad01529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41282/0 (socket says 172.21.15.148:41282) 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 -- 172.21.15.148:0/3842282625 learned_addr learned my addr 172.21.15.148:0/3842282625 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 -- 172.21.15.148:0/3842282625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad0154500 msgr2=0x7ffad01589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad0154500 0x7ffad01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 -- 172.21.15.148:0/3842282625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc004800 msgr2=0x7ffad00b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.099+0000 7ffad5546640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc004800 0x7ffad00b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffad5546640 1 -- 172.21.15.148:0/3842282625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad0153460 con 0x7ffad00b6520 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffad5546640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 0x7ffad01529f0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c380 tx=0x7ffac0001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0038040 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0042070 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004c00 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad0153740 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.100+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad0153fe0 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.101+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad0153b30 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.101+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffac0007070 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.102+0000 7ffac67fc640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffab406b4d0 0x7ffab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.102+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffac00a9fc0 con 0x7ffad00b6520 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.102+0000 7ffad5d47640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffab406b4d0 0x7ffab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.102+0000 7ffad5d47640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffab406b4d0 0x7ffab406d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006b10 tx=0x7ffacc006aa0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.106+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffac0078cb0 con 0x7ffad00b6520 2024-04-25T10:04:34.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.274+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffad0153910 con 0x7ffab406b4d0 2024-04-25T10:04:34.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.279+0000 7ffac67fc640 1 -- 172.21.15.148:0/3842282625 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240037 (secure 0 0 0) 0x7ffad0153910 con 0x7ffab406b4d0 2024-04-25T10:04:34.287 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:34.288 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:04:34.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffab406b4d0 msgr2=0x7ffab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffab406b4d0 0x7ffab406d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006b10 tx=0x7ffacc006aa0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 msgr2=0x7ffad01529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 0x7ffad01529f0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c380 tx=0x7ffac0001a60 comp rx=0 tx=0).stop 2024-04-25T10:04:34.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 shutdown_connections 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffab406b4d0 0x7ffab406d990 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffad0154500 0x7ffad01589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffad00b6520 0x7ffad01529f0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 --2- 172.21.15.148:0/3842282625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffabc004800 0x7ffad00b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 >> 172.21.15.148:0/3842282625 conn(0x7ffad006f5d0 msgr2=0x7ffad00c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 shutdown_connections 2024-04-25T10:04:34.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.282+0000 7ffad7fab640 1 -- 172.21.15.148:0/3842282625 wait complete. 2024-04-25T10:04:34.298 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-04-25T10:04:33.145804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":435284,"kb_used_data":16476,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754539436,"statfs":{"total":773094113280,"available":772648382464,"internally_reserved":0,"allocated":16871424,"data_stored":8497775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000657"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320899+0000","last_change":"2024-04-25T10:04:29.320899+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320899+0000","last_undegraded":"2024-04-25T10:04:29.320899+0000","last_fullsized":"2024-04-25T10:04:29.320899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322613+0000","last_change":"2024-04-25T10:04:29.322613+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322613+0000","last_undegraded":"2024-04-25T10:04:29.322613+0000","last_fullsized":"2024-04-25T10:04:29.322613+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322737+0000","last_change":"2024-04-25T10:04:29.322737+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322737+0000","last_undegraded":"2024-04-25T10:04:29.322737+0000","last_fullsized":"2024-04-25T10:04:29.322737+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325911+0000","last_change":"2024-04-25T10:04:29.325911+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325911+0000","last_undegraded":"2024-04-25T10:04:29.325911+0000","last_fullsized":"2024-04-25T10:04:29.325911+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326815+0000","last_change":"2024-04-25T10:04:29.326815+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326815+0000","last_undegraded":"2024-04-25T10:04:29.326815+0000","last_fullsized":"2024-04-25T10:04:29.326815+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322345+0000","last_change":"2024-04-25T10:04:29.322345+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322345+0000","last_undegraded":"2024-04-25T10:04:29.322345+0000","last_fullsized":"2024-04-25T10:04:29.322345+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325920+0000","last_change":"2024-04-25T10:04:29.325920+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325920+0000","last_undegraded":"2024-04-25T10:04:29.325920+0000","last_fullsized":"2024-04-25T10:04:29.325920+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326409+0000","last_change":"2024-04-25T10:04:29.326409+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326409+0000","last_undegraded":"2024-04-25T10:04:29.326409+0000","last_fullsized":"2024-04-25T10:04:29.326409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323262+0000","last_change":"2024-04-25T10:04:29.323262+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323262+0000","last_undegraded":"2024-04-25T10:04:29.323262+0000","last_fullsized":"2024-04-25T10:04:29.323262+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320899+0000","last_change":"2024-04-25T10:04:29.320899+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320899+0000","last_undegraded":"2024-04-25T10:04:29.320899+0000","last_fullsized":"2024-04-25T10:04:29.320899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.324410+0000","last_change":"2024-04-25T10:04:29.324410+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.324410+0000","last_undegraded":"2024-04-25T10:04:29.324410+0000","last_fullsized":"2024-04-25T10:04:29.324410+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323263+0000","last_change":"2024-04-25T10:04:29.323263+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323263+0000","last_undegraded":"2024-04-25T10:04:29.323263+0000","last_fullsized":"2024-04-25T10:04:29.323263+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322977+0000","last_change":"2024-04-25T10:04:29.322977+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322977+0000","last_undegraded":"2024-04-25T10:04:29.322977+0000","last_fullsized":"2024-04-25T10:04:29.322977+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323238+0000","last_change":"2024-04-25T10:04:29.323238+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323238+0000","last_undegraded":"2024-04-25T10:04:29.323238+0000","last_fullsized":"2024-04-25T10:04:29.323238+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325955+0000","last_change":"2024-04-25T10:04:29.325955+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325955+0000","last_undegraded":"2024-04-25T10:04:29.325955+0000","last_fullsized":"2024-04-25T10:04:29.325955+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322343+0000","last_change":"2024-04-25T10:04:29.322343+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322343+0000","last_undegraded":"2024-04-25T10:04:29.322343+0000","last_fullsized":"2024-04-25T10:04:29.322343+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326694+0000","last_change":"2024-04-25T10:04:29.326694+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326694+0000","last_undegraded":"2024-04-25T10:04:29.326694+0000","last_fullsized":"2024-04-25T10:04:29.326694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322638+0000","last_change":"2024-04-25T10:04:29.322638+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322638+0000","last_undegraded":"2024-04-25T10:04:29.322638+0000","last_fullsized":"2024-04-25T10:04:29.322638+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326671+0000","last_change":"2024-04-25T10:04:29.326671+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326671+0000","last_undegraded":"2024-04-25T10:04:29.326671+0000","last_fullsized":"2024-04-25T10:04:29.326671+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322923+0000","last_change":"2024-04-25T10:04:29.322923+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322923+0000","last_undegraded":"2024-04-25T10:04:29.322923+0000","last_fullsized":"2024-04-25T10:04:29.322923+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322748+0000","last_change":"2024-04-25T10:04:29.322748+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322748+0000","last_undegraded":"2024-04-25T10:04:29.322748+0000","last_fullsized":"2024-04-25T10:04:29.322748+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325493+0000","last_change":"2024-04-25T10:04:29.325493+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325493+0000","last_undegraded":"2024-04-25T10:04:29.325493+0000","last_fullsized":"2024-04-25T10:04:29.325493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.315825+0000","last_change":"2024-04-25T10:04:29.315825+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.315825+0000","last_undegraded":"2024-04-25T10:04:29.315825+0000","last_fullsized":"2024-04-25T10:04:29.315825+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322387+0000","last_change":"2024-04-25T10:04:29.322387+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322387+0000","last_undegraded":"2024-04-25T10:04:29.322387+0000","last_fullsized":"2024-04-25T10:04:29.322387+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322377+0000","last_change":"2024-04-25T10:04:29.322377+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322377+0000","last_undegraded":"2024-04-25T10:04:29.322377+0000","last_fullsized":"2024-04-25T10:04:29.322377+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322495+0000","last_change":"2024-04-25T10:04:29.322495+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322495+0000","last_undegraded":"2024-04-25T10:04:29.322495+0000","last_fullsized":"2024-04-25T10:04:29.322495+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325459+0000","last_change":"2024-04-25T10:04:29.325459+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325459+0000","last_undegraded":"2024-04-25T10:04:29.325459+0000","last_fullsized":"2024-04-25T10:04:29.325459+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320900+0000","last_change":"2024-04-25T10:04:29.320900+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320900+0000","last_undegraded":"2024-04-25T10:04:29.320900+0000","last_fullsized":"2024-04-25T10:04:29.320900+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322338+0000","last_change":"2024-04-25T10:04:29.322338+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322338+0000","last_undegraded":"2024-04-25T10:04:29.322338+0000","last_fullsized":"2024-04-25T10:04:29.322338+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322708+0000","last_change":"2024-04-25T10:04:29.322708+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322708+0000","last_undegraded":"2024-04-25T10:04:29.322708+0000","last_fullsized":"2024-04-25T10:04:29.322708+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322341+0000","last_change":"2024-04-25T10:04:29.322341+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322341+0000","last_undegraded":"2024-04-25T10:04:29.322341+0000","last_fullsized":"2024-04-25T10:04:29.322341+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.324372+0000","last_change":"2024-04-25T10:04:29.324372+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.324372+0000","last_undegraded":"2024-04-25T10:04:29.324372+0000","last_fullsized":"2024-04-25T10:04:29.324372+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":420,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:04:29.315173+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:04:29.315173+0000","last_peered":"2024-04-25T10:04:29.315173+0000","last_clean":"2024-04-25T10:04:29.315173+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:04:29.315173+0000","last_undegraded":"2024-04-25T10:04:29.315173+0000","last_fullsized":"2024-04-25T10:04:29.315173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51728,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320112,"statfs":{"total":96636764160,"available":96583794688,"internally_reserved":0,"allocated":2555904,"data_stored":1505774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55308,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316532,"statfs":{"total":96636764160,"available":96580128768,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59468,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312372,"statfs":{"total":96636764160,"available":96575868928,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55308,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316532,"statfs":{"total":96636764160,"available":96580128768,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2543616,"data_stored":1499565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:34.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.431+0000 7f8032438640 1 Processor -- start 2024-04-25T10:04:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.431+0000 7f8032438640 1 -- start start 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f8032438640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f802c15dbe0 0x7f802c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f8032438640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f802c0b2cb0 0x7f802c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f8032438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b92a0 0x7f802c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f8032438640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c15dfc0 con 0x7f802c0b2cb0 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f8032438640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0d1a00 con 0x7f802c15dbe0 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f8032438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b97a0 con 0x7f802c0b92a0 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f802b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f802c0b2cb0 0x7f802c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f802b7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f802c0b2cb0 0x7f802c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41294/0 (socket says 172.21.15.148:41294) 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f802b7fe640 1 -- 172.21.15.148:0/1883728850 learned_addr learned my addr 172.21.15.148:0/1883728850 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.432+0000 7f80309d5640 1 --2- 172.21.15.148:0/1883728850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b92a0 0x7f802c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802bfff640 1 --2- 172.21.15.148:0/1883728850 >> v2:172.21.15.188:3300/0 conn(0x7f802c15dbe0 0x7f802c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802b7fe640 1 -- 172.21.15.148:0/1883728850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b92a0 msgr2=0x7f802c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802b7fe640 1 --2- 172.21.15.148:0/1883728850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b92a0 0x7f802c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802b7fe640 1 -- 172.21.15.148:0/1883728850 >> v2:172.21.15.188:3300/0 conn(0x7f802c15dbe0 msgr2=0x7f802c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802b7fe640 1 --2- 172.21.15.148:0/1883728850 >> v2:172.21.15.188:3300/0 conn(0x7f802c15dbe0 0x7f802c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802b7fe640 1 -- 172.21.15.148:0/1883728850 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0d1c50 con 0x7f802c0b2cb0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802b7fe640 1 --2- 172.21.15.148:0/1883728850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b2cb0 0x7f802c0d0e50 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f802c06c140 tx=0x7f802002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fdbd4be13a133e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.433+0000 7f802affd640 1 -- 172.21.15.148:0/1883728850 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020037070 con 0x7f802c0b2cb0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f802affd640 1 -- 172.21.15.148:0/1883728850 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020032440 con 0x7f802c0b2cb0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f802affd640 1 -- 172.21.15.148:0/1883728850 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020032810 con 0x7f802c0b2cb0 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 -- 172.21.15.148:0/1883728850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b2cb0 msgr2=0x7f802c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 --2- 172.21.15.148:0/1883728850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b2cb0 0x7f802c0d0e50 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f802c06c140 tx=0x7f802002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 -- 172.21.15.148:0/1883728850 shutdown_connections 2024-04-25T10:04:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 --2- 172.21.15.148:0/1883728850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b92a0 0x7f802c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 --2- 172.21.15.148:0/1883728850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b2cb0 0x7f802c0d0e50 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 --2- 172.21.15.148:0/1883728850 >> v2:172.21.15.188:3300/0 conn(0x7f802c15dbe0 0x7f802c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 -- 172.21.15.148:0/1883728850 >> 172.21.15.148:0/1883728850 conn(0x7f802c061a20 msgr2=0x7f802c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 -- 172.21.15.148:0/1883728850 shutdown_connections 2024-04-25T10:04:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.434+0000 7f8032438640 1 -- 172.21.15.148:0/1883728850 wait complete. 2024-04-25T10:04:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.435+0000 7f8032438640 1 Processor -- start 2024-04-25T10:04:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 -- start start 2024-04-25T10:04:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b2cb0 0x7f802c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 0x7f802c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c15dbe0 0x7f802c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b6ab0 con 0x7f802c0b92a0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0d2210 con 0x7f802c15dbe0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f8032438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b6c30 con 0x7f802c0b2cb0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f802b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 0x7f802c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f802b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 0x7f802c15c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41308/0 (socket says 172.21.15.148:41308) 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.436+0000 7f802b7fe640 1 -- 172.21.15.148:0/2263136188 learned_addr learned my addr 172.21.15.148:0/2263136188 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802bfff640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b2cb0 0x7f802c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802b7fe640 1 -- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b2cb0 msgr2=0x7f802c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f80309d5640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c15dbe0 0x7f802c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802b7fe640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b2cb0 0x7f802c0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802b7fe640 1 -- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c15dbe0 msgr2=0x7f802c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802b7fe640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c15dbe0 0x7f802c15cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802b7fe640 1 -- 172.21.15.148:0/2263136188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0cabf0 con 0x7f802c0b92a0 2024-04-25T10:04:34.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f802b7fe640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 0x7f802c15c890 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f8020000c00 tx=0x7f80200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020041020 con 0x7f802c0b92a0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802003d620 con 0x7f802c0b92a0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0caed0 con 0x7f802c0b92a0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020037040 con 0x7f802c0b92a0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.437+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0cb490 con 0x7f802c0b92a0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.438+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c073990 con 0x7f802c0b92a0 2024-04-25T10:04:34.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.438+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8020044020 con 0x7f802c0b92a0 2024-04-25T10:04:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.439+0000 7f8028ff9640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.439+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80200aa990 con 0x7f802c0b92a0 2024-04-25T10:04:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.439+0000 7f802bfff640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.439+0000 7f802bfff640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f801806b4d0 0x7f801806d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f801c0098f0 tx=0x7f801c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.443+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f802004b030 con 0x7f802c0b92a0 2024-04-25T10:04:34.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.610+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f802c0b7960 con 0x7f801806b4d0 2024-04-25T10:04:34.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.616+0000 7f8028ff9640 1 -- 172.21.15.148:0/2263136188 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240037 (secure 0 0 0) 0x7f802c0b7960 con 0x7f801806b4d0 2024-04-25T10:04:34.623 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:34.624 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.618+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f801806b4d0 msgr2=0x7f801806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f801806b4d0 0x7f801806d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f801c0098f0 tx=0x7f801c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 msgr2=0x7f802c15c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 0x7f802c15c890 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f8020000c00 tx=0x7f80200046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 shutdown_connections 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f801806b4d0 0x7f801806d990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c15dbe0 0x7f802c15cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f802c0b92a0 0x7f802c15c890 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 --2- 172.21.15.148:0/2263136188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c0b2cb0 0x7f802c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 >> 172.21.15.148:0/2263136188 conn(0x7f802c061a20 msgr2=0x7f802c0b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:34.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 shutdown_connections 2024-04-25T10:04:34.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.619+0000 7f8032438640 1 -- 172.21.15.148:0/2263136188 wait complete. 2024-04-25T10:04:34.635 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-04-25T10:04:33.145804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":435284,"kb_used_data":16476,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754539436,"statfs":{"total":773094113280,"available":772648382464,"internally_reserved":0,"allocated":16871424,"data_stored":8497775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000657"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320899+0000","last_change":"2024-04-25T10:04:29.320899+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320899+0000","last_undegraded":"2024-04-25T10:04:29.320899+0000","last_fullsized":"2024-04-25T10:04:29.320899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322613+0000","last_change":"2024-04-25T10:04:29.322613+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322613+0000","last_undegraded":"2024-04-25T10:04:29.322613+0000","last_fullsized":"2024-04-25T10:04:29.322613+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322737+0000","last_change":"2024-04-25T10:04:29.322737+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322737+0000","last_undegraded":"2024-04-25T10:04:29.322737+0000","last_fullsized":"2024-04-25T10:04:29.322737+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325911+0000","last_change":"2024-04-25T10:04:29.325911+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325911+0000","last_undegraded":"2024-04-25T10:04:29.325911+0000","last_fullsized":"2024-04-25T10:04:29.325911+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326815+0000","last_change":"2024-04-25T10:04:29.326815+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326815+0000","last_undegraded":"2024-04-25T10:04:29.326815+0000","last_fullsized":"2024-04-25T10:04:29.326815+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322345+0000","last_change":"2024-04-25T10:04:29.322345+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322345+0000","last_undegraded":"2024-04-25T10:04:29.322345+0000","last_fullsized":"2024-04-25T10:04:29.322345+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325920+0000","last_change":"2024-04-25T10:04:29.325920+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325920+0000","last_undegraded":"2024-04-25T10:04:29.325920+0000","last_fullsized":"2024-04-25T10:04:29.325920+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326409+0000","last_change":"2024-04-25T10:04:29.326409+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326409+0000","last_undegraded":"2024-04-25T10:04:29.326409+0000","last_fullsized":"2024-04-25T10:04:29.326409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323262+0000","last_change":"2024-04-25T10:04:29.323262+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323262+0000","last_undegraded":"2024-04-25T10:04:29.323262+0000","last_fullsized":"2024-04-25T10:04:29.323262+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320899+0000","last_change":"2024-04-25T10:04:29.320899+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320899+0000","last_undegraded":"2024-04-25T10:04:29.320899+0000","last_fullsized":"2024-04-25T10:04:29.320899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.324410+0000","last_change":"2024-04-25T10:04:29.324410+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.324410+0000","last_undegraded":"2024-04-25T10:04:29.324410+0000","last_fullsized":"2024-04-25T10:04:29.324410+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323263+0000","last_change":"2024-04-25T10:04:29.323263+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323263+0000","last_undegraded":"2024-04-25T10:04:29.323263+0000","last_fullsized":"2024-04-25T10:04:29.323263+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322977+0000","last_change":"2024-04-25T10:04:29.322977+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322977+0000","last_undegraded":"2024-04-25T10:04:29.322977+0000","last_fullsized":"2024-04-25T10:04:29.322977+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323238+0000","last_change":"2024-04-25T10:04:29.323238+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323238+0000","last_undegraded":"2024-04-25T10:04:29.323238+0000","last_fullsized":"2024-04-25T10:04:29.323238+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325955+0000","last_change":"2024-04-25T10:04:29.325955+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325955+0000","last_undegraded":"2024-04-25T10:04:29.325955+0000","last_fullsized":"2024-04-25T10:04:29.325955+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322343+0000","last_change":"2024-04-25T10:04:29.322343+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322343+0000","last_undegraded":"2024-04-25T10:04:29.322343+0000","last_fullsized":"2024-04-25T10:04:29.322343+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326694+0000","last_change":"2024-04-25T10:04:29.326694+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326694+0000","last_undegraded":"2024-04-25T10:04:29.326694+0000","last_fullsized":"2024-04-25T10:04:29.326694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322638+0000","last_change":"2024-04-25T10:04:29.322638+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322638+0000","last_undegraded":"2024-04-25T10:04:29.322638+0000","last_fullsized":"2024-04-25T10:04:29.322638+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326671+0000","last_change":"2024-04-25T10:04:29.326671+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326671+0000","last_undegraded":"2024-04-25T10:04:29.326671+0000","last_fullsized":"2024-04-25T10:04:29.326671+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322923+0000","last_change":"2024-04-25T10:04:29.322923+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322923+0000","last_undegraded":"2024-04-25T10:04:29.322923+0000","last_fullsized":"2024-04-25T10:04:29.322923+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322748+0000","last_change":"2024-04-25T10:04:29.322748+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322748+0000","last_undegraded":"2024-04-25T10:04:29.322748+0000","last_fullsized":"2024-04-25T10:04:29.322748+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325493+0000","last_change":"2024-04-25T10:04:29.325493+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325493+0000","last_undegraded":"2024-04-25T10:04:29.325493+0000","last_fullsized":"2024-04-25T10:04:29.325493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.315825+0000","last_change":"2024-04-25T10:04:29.315825+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.315825+0000","last_undegraded":"2024-04-25T10:04:29.315825+0000","last_fullsized":"2024-04-25T10:04:29.315825+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322387+0000","last_change":"2024-04-25T10:04:29.322387+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322387+0000","last_undegraded":"2024-04-25T10:04:29.322387+0000","last_fullsized":"2024-04-25T10:04:29.322387+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322377+0000","last_change":"2024-04-25T10:04:29.322377+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322377+0000","last_undegraded":"2024-04-25T10:04:29.322377+0000","last_fullsized":"2024-04-25T10:04:29.322377+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322495+0000","last_change":"2024-04-25T10:04:29.322495+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322495+0000","last_undegraded":"2024-04-25T10:04:29.322495+0000","last_fullsized":"2024-04-25T10:04:29.322495+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325459+0000","last_change":"2024-04-25T10:04:29.325459+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325459+0000","last_undegraded":"2024-04-25T10:04:29.325459+0000","last_fullsized":"2024-04-25T10:04:29.325459+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320900+0000","last_change":"2024-04-25T10:04:29.320900+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320900+0000","last_undegraded":"2024-04-25T10:04:29.320900+0000","last_fullsized":"2024-04-25T10:04:29.320900+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322338+0000","last_change":"2024-04-25T10:04:29.322338+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322338+0000","last_undegraded":"2024-04-25T10:04:29.322338+0000","last_fullsized":"2024-04-25T10:04:29.322338+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322708+0000","last_change":"2024-04-25T10:04:29.322708+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322708+0000","last_undegraded":"2024-04-25T10:04:29.322708+0000","last_fullsized":"2024-04-25T10:04:29.322708+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322341+0000","last_change":"2024-04-25T10:04:29.322341+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322341+0000","last_undegraded":"2024-04-25T10:04:29.322341+0000","last_fullsized":"2024-04-25T10:04:29.322341+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.324372+0000","last_change":"2024-04-25T10:04:29.324372+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.324372+0000","last_undegraded":"2024-04-25T10:04:29.324372+0000","last_fullsized":"2024-04-25T10:04:29.324372+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":420,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:04:29.315173+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:04:29.315173+0000","last_peered":"2024-04-25T10:04:29.315173+0000","last_clean":"2024-04-25T10:04:29.315173+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:04:29.315173+0000","last_undegraded":"2024-04-25T10:04:29.315173+0000","last_fullsized":"2024-04-25T10:04:29.315173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51728,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320112,"statfs":{"total":96636764160,"available":96583794688,"internally_reserved":0,"allocated":2555904,"data_stored":1505774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55308,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316532,"statfs":{"total":96636764160,"available":96580128768,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59468,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312372,"statfs":{"total":96636764160,"available":96575868928,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55308,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316532,"statfs":{"total":96636764160,"available":96580128768,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2543616,"data_stored":1499565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:34.638 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.766+0000 7f0b139ab640 1 Processor -- start 2024-04-25T10:04:34.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.767+0000 7f0b139ab640 1 -- start start 2024-04-25T10:04:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.767+0000 7f0b139ab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 0x7f0b0c0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.767+0000 7f0b139ab640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b0c071ac0 0x7f0b0c0ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.767+0000 7f0b139ab640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c15ec60 con 0x7f0b0c06c1d0 2024-04-25T10:04:34.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.767+0000 7f0b139ab640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c15ef40 con 0x7f0b0c071ac0 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.767+0000 7f0b139ab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c06db90 con 0x7f0b0c154ed0 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b10f46640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b0c071ac0 0x7f0b0c0ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b11747640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 0x7f0b0c0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b11f48640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0b0c06c1d0 0x7f0b0c1617d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:55726/0 (socket says 172.21.15.148:55726) 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b11747640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 0x7f0b0c0aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:59880/0 (socket says 172.21.15.148:59880) 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b11f48640 1 -- 172.21.15.148:0/1127848025 learned_addr learned my addr 172.21.15.148:0/1127848025 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b10f46640 1 -- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 msgr2=0x7f0b0c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b10f46640 1 --2- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 0x7f0b0c0aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b10f46640 1 -- 172.21.15.148:0/1127848025 >> v1:172.21.15.188:6789/0 conn(0x7f0b0c06c1d0 legacy=0x7f0b0c1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b03fff640 1 -- 172.21.15.148:0/1127848025 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019800984 0 0) 0x7f0b0c15ec60 con 0x7f0b0c06c1d0 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b10f46640 1 -- 172.21.15.148:0/1127848025 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c06de70 con 0x7f0b0c071ac0 2024-04-25T10:04:34.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.768+0000 7f0b11747640 1 --2- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 0x7f0b0c0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.769+0000 7f0b10f46640 1 --2- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c071ac0 0x7f0b0c0ad410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0097a0 tx=0x7f0afc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e4fc4b954df7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.769+0000 7f0b03fff640 1 -- 172.21.15.148:0/1127848025 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc037070 con 0x7f0b0c071ac0 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.769+0000 7f0b03fff640 1 -- 172.21.15.148:0/1127848025 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0afc032440 con 0x7f0b0c071ac0 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.769+0000 7f0b03fff640 1 -- 172.21.15.148:0/1127848025 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc032810 con 0x7f0b0c071ac0 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.769+0000 7f0b139ab640 1 -- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c071ac0 msgr2=0x7f0b0c0ad410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.769+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c071ac0 0x7f0b0c0ad410 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0097a0 tx=0x7f0afc02a180 comp rx=0 tx=0).stop 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.770+0000 7f0b139ab640 1 -- 172.21.15.148:0/1127848025 shutdown_connections 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.770+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c071ac0 0x7f0b0c0ad410 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.770+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1127848025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c154ed0 0x7f0b0c0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.770+0000 7f0b139ab640 1 -- 172.21.15.148:0/1127848025 >> 172.21.15.148:0/1127848025 conn(0x7f0b0c0af670 msgr2=0x7f0b0c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.770+0000 7f0b139ab640 1 -- 172.21.15.148:0/1127848025 shutdown_connections 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.770+0000 7f0b139ab640 1 -- 172.21.15.148:0/1127848025 wait complete. 2024-04-25T10:04:34.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 Processor -- start 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 -- start start 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c071ac0 0x7f0b0c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c154ed0 0x7f0b0c1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 0x7f0b0c0b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0add20 con 0x7f0b0c156540 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0adba0 con 0x7f0b0c154ed0 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.771+0000 7f0b139ab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c161ee0 con 0x7f0b0c071ac0 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 0x7f0b0c0b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 0x7f0b0c0b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41324/0 (socket says 172.21.15.148:41324) 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 -- 172.21.15.148:0/1092022506 learned_addr learned my addr 172.21.15.148:0/1092022506 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11747640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c071ac0 0x7f0b0c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b10f46640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c154ed0 0x7f0b0c1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 -- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c071ac0 msgr2=0x7f0b0c155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c071ac0 0x7f0b0c155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 -- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c154ed0 msgr2=0x7f0b0c1448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c154ed0 0x7f0b0c1448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 -- 172.21.15.148:0/1092022506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c1f3e90 con 0x7f0b0c156540 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b11f48640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 0x7f0b0c0b0730 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400c3e0 tx=0x7f0b0400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040074e0 con 0x7f0b0c156540 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b040047e0 con 0x7f0b0c156540 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1f4170 con 0x7f0b0c156540 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.772+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04004bd0 con 0x7f0b0c156540 2024-04-25T10:04:34.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.773+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1f4730 con 0x7f0b0c156540 2024-04-25T10:04:34.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.773+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c0b2b10 con 0x7f0b0c156540 2024-04-25T10:04:34.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.774+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b04011070 con 0x7f0b0c156540 2024-04-25T10:04:34.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.774+0000 7f0b01ffb640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0af006b4d0 0x7f0af006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:34.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.774+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b04089f70 con 0x7f0b0c156540 2024-04-25T10:04:34.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.774+0000 7f0b11747640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0af006b4d0 0x7f0af006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:34.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.774+0000 7f0b11747640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0af006b4d0 0x7f0af006d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0cbfa0 tx=0x7f0b0800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:34.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.778+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b0400e280 con 0x7f0b0c156540 2024-04-25T10:04:34.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.945+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b0c073990 con 0x7f0af006b4d0 2024-04-25T10:04:34.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.950+0000 7f0b01ffb640 1 -- 172.21.15.148:0/1092022506 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240037 (secure 0 0 0) 0x7f0b0c073990 con 0x7f0af006b4d0 2024-04-25T10:04:34.957 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:34.958 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:04:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0af006b4d0 msgr2=0x7f0af006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0af006b4d0 0x7f0af006d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0cbfa0 tx=0x7f0b0800b040 comp rx=0 tx=0).stop 2024-04-25T10:04:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 msgr2=0x7f0b0c0b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 0x7f0b0c0b0730 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400c3e0 tx=0x7f0b0400c8a0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 shutdown_connections 2024-04-25T10:04:34.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0af006b4d0 0x7f0af006d990 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0b0c156540 0x7f0b0c0b0730 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b0c154ed0 0x7f0b0c1448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 --2- 172.21.15.148:0/1092022506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b0c071ac0 0x7f0b0c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 >> 172.21.15.148:0/1092022506 conn(0x7f0b0c06c1d0 msgr2=0x7f0b0c0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 shutdown_connections 2024-04-25T10:04:34.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:34.953+0000 7f0b139ab640 1 -- 172.21.15.148:0/1092022506 wait complete. 2024-04-25T10:04:34.969 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-04-25T10:04:33.145804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":435284,"kb_used_data":16476,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754539436,"statfs":{"total":773094113280,"available":772648382464,"internally_reserved":0,"allocated":16871424,"data_stored":8497775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000657"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320899+0000","last_change":"2024-04-25T10:04:29.320899+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320899+0000","last_undegraded":"2024-04-25T10:04:29.320899+0000","last_fullsized":"2024-04-25T10:04:29.320899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322613+0000","last_change":"2024-04-25T10:04:29.322613+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322613+0000","last_undegraded":"2024-04-25T10:04:29.322613+0000","last_fullsized":"2024-04-25T10:04:29.322613+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322737+0000","last_change":"2024-04-25T10:04:29.322737+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322737+0000","last_undegraded":"2024-04-25T10:04:29.322737+0000","last_fullsized":"2024-04-25T10:04:29.322737+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325911+0000","last_change":"2024-04-25T10:04:29.325911+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325911+0000","last_undegraded":"2024-04-25T10:04:29.325911+0000","last_fullsized":"2024-04-25T10:04:29.325911+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326815+0000","last_change":"2024-04-25T10:04:29.326815+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326815+0000","last_undegraded":"2024-04-25T10:04:29.326815+0000","last_fullsized":"2024-04-25T10:04:29.326815+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322345+0000","last_change":"2024-04-25T10:04:29.322345+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322345+0000","last_undegraded":"2024-04-25T10:04:29.322345+0000","last_fullsized":"2024-04-25T10:04:29.322345+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325920+0000","last_change":"2024-04-25T10:04:29.325920+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325920+0000","last_undegraded":"2024-04-25T10:04:29.325920+0000","last_fullsized":"2024-04-25T10:04:29.325920+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326409+0000","last_change":"2024-04-25T10:04:29.326409+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326409+0000","last_undegraded":"2024-04-25T10:04:29.326409+0000","last_fullsized":"2024-04-25T10:04:29.326409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323262+0000","last_change":"2024-04-25T10:04:29.323262+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323262+0000","last_undegraded":"2024-04-25T10:04:29.323262+0000","last_fullsized":"2024-04-25T10:04:29.323262+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320899+0000","last_change":"2024-04-25T10:04:29.320899+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320899+0000","last_undegraded":"2024-04-25T10:04:29.320899+0000","last_fullsized":"2024-04-25T10:04:29.320899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.324410+0000","last_change":"2024-04-25T10:04:29.324410+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.324410+0000","last_undegraded":"2024-04-25T10:04:29.324410+0000","last_fullsized":"2024-04-25T10:04:29.324410+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323263+0000","last_change":"2024-04-25T10:04:29.323263+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323263+0000","last_undegraded":"2024-04-25T10:04:29.323263+0000","last_fullsized":"2024-04-25T10:04:29.323263+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322977+0000","last_change":"2024-04-25T10:04:29.322977+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322977+0000","last_undegraded":"2024-04-25T10:04:29.322977+0000","last_fullsized":"2024-04-25T10:04:29.322977+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.323238+0000","last_change":"2024-04-25T10:04:29.323238+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.323238+0000","last_undegraded":"2024-04-25T10:04:29.323238+0000","last_fullsized":"2024-04-25T10:04:29.323238+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325955+0000","last_change":"2024-04-25T10:04:29.325955+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325955+0000","last_undegraded":"2024-04-25T10:04:29.325955+0000","last_fullsized":"2024-04-25T10:04:29.325955+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322343+0000","last_change":"2024-04-25T10:04:29.322343+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322343+0000","last_undegraded":"2024-04-25T10:04:29.322343+0000","last_fullsized":"2024-04-25T10:04:29.322343+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326694+0000","last_change":"2024-04-25T10:04:29.326694+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326694+0000","last_undegraded":"2024-04-25T10:04:29.326694+0000","last_fullsized":"2024-04-25T10:04:29.326694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322638+0000","last_change":"2024-04-25T10:04:29.322638+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322638+0000","last_undegraded":"2024-04-25T10:04:29.322638+0000","last_fullsized":"2024-04-25T10:04:29.322638+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.326671+0000","last_change":"2024-04-25T10:04:29.326671+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.326671+0000","last_undegraded":"2024-04-25T10:04:29.326671+0000","last_fullsized":"2024-04-25T10:04:29.326671+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322923+0000","last_change":"2024-04-25T10:04:29.322923+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322923+0000","last_undegraded":"2024-04-25T10:04:29.322923+0000","last_fullsized":"2024-04-25T10:04:29.322923+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322748+0000","last_change":"2024-04-25T10:04:29.322748+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322748+0000","last_undegraded":"2024-04-25T10:04:29.322748+0000","last_fullsized":"2024-04-25T10:04:29.322748+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325493+0000","last_change":"2024-04-25T10:04:29.325493+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325493+0000","last_undegraded":"2024-04-25T10:04:29.325493+0000","last_fullsized":"2024-04-25T10:04:29.325493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.315825+0000","last_change":"2024-04-25T10:04:29.315825+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.315825+0000","last_undegraded":"2024-04-25T10:04:29.315825+0000","last_fullsized":"2024-04-25T10:04:29.315825+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322387+0000","last_change":"2024-04-25T10:04:29.322387+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322387+0000","last_undegraded":"2024-04-25T10:04:29.322387+0000","last_fullsized":"2024-04-25T10:04:29.322387+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322377+0000","last_change":"2024-04-25T10:04:29.322377+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322377+0000","last_undegraded":"2024-04-25T10:04:29.322377+0000","last_fullsized":"2024-04-25T10:04:29.322377+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322495+0000","last_change":"2024-04-25T10:04:29.322495+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322495+0000","last_undegraded":"2024-04-25T10:04:29.322495+0000","last_fullsized":"2024-04-25T10:04:29.322495+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.325459+0000","last_change":"2024-04-25T10:04:29.325459+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.325459+0000","last_undegraded":"2024-04-25T10:04:29.325459+0000","last_fullsized":"2024-04-25T10:04:29.325459+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.320900+0000","last_change":"2024-04-25T10:04:29.320900+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.320900+0000","last_undegraded":"2024-04-25T10:04:29.320900+0000","last_fullsized":"2024-04-25T10:04:29.320900+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322338+0000","last_change":"2024-04-25T10:04:29.322338+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322338+0000","last_undegraded":"2024-04-25T10:04:29.322338+0000","last_fullsized":"2024-04-25T10:04:29.322338+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322708+0000","last_change":"2024-04-25T10:04:29.322708+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322708+0000","last_undegraded":"2024-04-25T10:04:29.322708+0000","last_fullsized":"2024-04-25T10:04:29.322708+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.322341+0000","last_change":"2024-04-25T10:04:29.322341+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.322341+0000","last_undegraded":"2024-04-25T10:04:29.322341+0000","last_fullsized":"2024-04-25T10:04:29.322341+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:30.312402+0000","last_change":"2024-04-25T10:04:30.312402+0000","last_active":"2024-04-25T10:04:30.312402+0000","last_peered":"2024-04-25T10:04:30.312402+0000","last_clean":"2024-04-25T10:04:30.312402+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:30.312402+0000","last_undegraded":"2024-04-25T10:04:30.312402+0000","last_fullsized":"2024-04-25T10:04:30.312402+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":1,"reported_epoch":156,"state":"creating+peering","last_fresh":"2024-04-25T10:04:29.324372+0000","last_change":"2024-04-25T10:04:29.324372+0000","last_active":"2024-04-25T10:04:29.308107+0000","last_peered":"2024-04-25T10:04:29.308107+0000","last_clean":"2024-04-25T10:04:29.308107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:29.324372+0000","last_undegraded":"2024-04-25T10:04:29.324372+0000","last_fullsized":"2024-04-25T10:04:29.324372+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":420,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T10:04:29.315173+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:04:29.315173+0000","last_peered":"2024-04-25T10:04:29.315173+0000","last_clean":"2024-04-25T10:04:29.315173+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:04:29.315173+0000","last_undegraded":"2024-04-25T10:04:29.315173+0000","last_fullsized":"2024-04-25T10:04:29.315173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51728,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320112,"statfs":{"total":96636764160,"available":96583794688,"internally_reserved":0,"allocated":2555904,"data_stored":1505774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51148,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320692,"statfs":{"total":96636764160,"available":96584388608,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55308,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316532,"statfs":{"total":96636764160,"available":96580128768,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59468,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312372,"statfs":{"total":96636764160,"available":96575868928,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55308,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316532,"statfs":{"total":96636764160,"available":96580128768,"internally_reserved":0,"allocated":1961984,"data_stored":915406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55872,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315968,"statfs":{"total":96636764160,"available":96579551232,"internally_reserved":0,"allocated":2543616,"data_stored":1499565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:34.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3f is not active+clean 2024-04-25T10:04:34.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3e is not active+clean 2024-04-25T10:04:34.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3d is not active+clean 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3c is not active+clean 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3b is not active+clean 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3a is not active+clean 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.39 is not active+clean 2024-04-25T10:04:34.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.38 is not active+clean 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.37 is not active+clean 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.36 is not active+clean 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.35 is not active+clean 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.34 is not active+clean 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.33 is not active+clean 2024-04-25T10:04:34.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.32 is not active+clean 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.31 is not active+clean 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.30 is not active+clean 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2f is not active+clean 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2e is not active+clean 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2d is not active+clean 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2c is not active+clean 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2b is not active+clean 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2a is not active+clean 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.29 is not active+clean 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.28 is not active+clean 2024-04-25T10:04:34.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.27 is not active+clean 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.26 is not active+clean 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.25 is not active+clean 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.24 is not active+clean 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.23 is not active+clean 2024-04-25T10:04:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.22 is not active+clean 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.21 is not active+clean 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.20 is not active+clean 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1f is not active+clean 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.320899+0000', 'last_change': '2024-04-25T10:04:29.320899+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.320899+0000', 'last_undegraded': '2024-04-25T10:04:29.320899+0000', 'last_fullsized': '2024-04-25T10:04:29.320899+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2024-04-25T10:04:34.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1e is not active+clean 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322613+0000', 'last_change': '2024-04-25T10:04:29.322613+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322613+0000', 'last_undegraded': '2024-04-25T10:04:29.322613+0000', 'last_fullsized': '2024-04-25T10:04:29.322613+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1d is not active+clean 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322737+0000', 'last_change': '2024-04-25T10:04:29.322737+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322737+0000', 'last_undegraded': '2024-04-25T10:04:29.322737+0000', 'last_fullsized': '2024-04-25T10:04:29.322737+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1c is not active+clean 2024-04-25T10:04:34.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.325911+0000', 'last_change': '2024-04-25T10:04:29.325911+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.325911+0000', 'last_undegraded': '2024-04-25T10:04:29.325911+0000', 'last_fullsized': '2024-04-25T10:04:29.325911+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1b is not active+clean 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.326815+0000', 'last_change': '2024-04-25T10:04:29.326815+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.326815+0000', 'last_undegraded': '2024-04-25T10:04:29.326815+0000', 'last_fullsized': '2024-04-25T10:04:29.326815+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1a is not active+clean 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322345+0000', 'last_change': '2024-04-25T10:04:29.322345+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322345+0000', 'last_undegraded': '2024-04-25T10:04:29.322345+0000', 'last_fullsized': '2024-04-25T10:04:29.322345+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.19 is not active+clean 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.325920+0000', 'last_change': '2024-04-25T10:04:29.325920+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.325920+0000', 'last_undegraded': '2024-04-25T10:04:29.325920+0000', 'last_fullsized': '2024-04-25T10:04:29.325920+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.18 is not active+clean 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.326409+0000', 'last_change': '2024-04-25T10:04:29.326409+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.326409+0000', 'last_undegraded': '2024-04-25T10:04:29.326409+0000', 'last_fullsized': '2024-04-25T10:04:29.326409+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.17 is not active+clean 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.323262+0000', 'last_change': '2024-04-25T10:04:29.323262+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.323262+0000', 'last_undegraded': '2024-04-25T10:04:29.323262+0000', 'last_fullsized': '2024-04-25T10:04:29.323262+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2024-04-25T10:04:34.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.15 is not active+clean 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.320899+0000', 'last_change': '2024-04-25T10:04:29.320899+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.320899+0000', 'last_undegraded': '2024-04-25T10:04:29.320899+0000', 'last_fullsized': '2024-04-25T10:04:29.320899+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.14 is not active+clean 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.324410+0000', 'last_change': '2024-04-25T10:04:29.324410+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.324410+0000', 'last_undegraded': '2024-04-25T10:04:29.324410+0000', 'last_fullsized': '2024-04-25T10:04:29.324410+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1c is not active+clean 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.13 is not active+clean 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.323263+0000', 'last_change': '2024-04-25T10:04:29.323263+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.323263+0000', 'last_undegraded': '2024-04-25T10:04:29.323263+0000', 'last_fullsized': '2024-04-25T10:04:29.323263+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2024-04-25T10:04:34.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.12 is not active+clean 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322977+0000', 'last_change': '2024-04-25T10:04:29.322977+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322977+0000', 'last_undegraded': '2024-04-25T10:04:29.322977+0000', 'last_fullsized': '2024-04-25T10:04:29.322977+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.11 is not active+clean 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.323238+0000', 'last_change': '2024-04-25T10:04:29.323238+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.323238+0000', 'last_undegraded': '2024-04-25T10:04:29.323238+0000', 'last_fullsized': '2024-04-25T10:04:29.323238+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.10 is not active+clean 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.325955+0000', 'last_change': '2024-04-25T10:04:29.325955+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.325955+0000', 'last_undegraded': '2024-04-25T10:04:29.325955+0000', 'last_fullsized': '2024-04-25T10:04:29.325955+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:34.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.f is not active+clean 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322343+0000', 'last_change': '2024-04-25T10:04:29.322343+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322343+0000', 'last_undegraded': '2024-04-25T10:04:29.322343+0000', 'last_fullsized': '2024-04-25T10:04:29.322343+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1 is not active+clean 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.e is not active+clean 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.326694+0000', 'last_change': '2024-04-25T10:04:29.326694+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.326694+0000', 'last_undegraded': '2024-04-25T10:04:29.326694+0000', 'last_fullsized': '2024-04-25T10:04:29.326694+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.d is not active+clean 2024-04-25T10:04:34.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322638+0000', 'last_change': '2024-04-25T10:04:29.322638+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322638+0000', 'last_undegraded': '2024-04-25T10:04:29.322638+0000', 'last_fullsized': '2024-04-25T10:04:29.322638+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.c is not active+clean 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.326671+0000', 'last_change': '2024-04-25T10:04:29.326671+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.326671+0000', 'last_undegraded': '2024-04-25T10:04:29.326671+0000', 'last_fullsized': '2024-04-25T10:04:29.326671+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.b is not active+clean 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322923+0000', 'last_change': '2024-04-25T10:04:29.322923+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322923+0000', 'last_undegraded': '2024-04-25T10:04:29.322923+0000', 'last_fullsized': '2024-04-25T10:04:29.322923+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.a is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322748+0000', 'last_change': '2024-04-25T10:04:29.322748+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322748+0000', 'last_undegraded': '2024-04-25T10:04:29.322748+0000', 'last_fullsized': '2024-04-25T10:04:29.322748+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.9 is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.325493+0000', 'last_change': '2024-04-25T10:04:29.325493+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.325493+0000', 'last_undegraded': '2024-04-25T10:04:29.325493+0000', 'last_fullsized': '2024-04-25T10:04:29.325493+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.8 is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.315825+0000', 'last_change': '2024-04-25T10:04:29.315825+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.315825+0000', 'last_undegraded': '2024-04-25T10:04:29.315825+0000', 'last_fullsized': '2024-04-25T10:04:29.315825+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.7 is not active+clean 2024-04-25T10:04:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322387+0000', 'last_change': '2024-04-25T10:04:29.322387+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322387+0000', 'last_undegraded': '2024-04-25T10:04:29.322387+0000', 'last_fullsized': '2024-04-25T10:04:29.322387+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.6 is not active+clean 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322377+0000', 'last_change': '2024-04-25T10:04:29.322377+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322377+0000', 'last_undegraded': '2024-04-25T10:04:29.322377+0000', 'last_fullsized': '2024-04-25T10:04:29.322377+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.5 is not active+clean 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322495+0000', 'last_change': '2024-04-25T10:04:29.322495+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322495+0000', 'last_undegraded': '2024-04-25T10:04:29.322495+0000', 'last_fullsized': '2024-04-25T10:04:29.322495+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.4 is not active+clean 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.325459+0000', 'last_change': '2024-04-25T10:04:29.325459+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.325459+0000', 'last_undegraded': '2024-04-25T10:04:29.325459+0000', 'last_fullsized': '2024-04-25T10:04:29.325459+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.c is not active+clean 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3 is not active+clean 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.320900+0000', 'last_change': '2024-04-25T10:04:29.320900+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.320900+0000', 'last_undegraded': '2024-04-25T10:04:29.320900+0000', 'last_fullsized': '2024-04-25T10:04:29.320900+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2 is not active+clean 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322338+0000', 'last_change': '2024-04-25T10:04:29.322338+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322338+0000', 'last_undegraded': '2024-04-25T10:04:29.322338+0000', 'last_fullsized': '2024-04-25T10:04:29.322338+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2024-04-25T10:04:34.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1 is not active+clean 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322708+0000', 'last_change': '2024-04-25T10:04:29.322708+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322708+0000', 'last_undegraded': '2024-04-25T10:04:29.322708+0000', 'last_fullsized': '2024-04-25T10:04:29.322708+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.0 is not active+clean 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.322341+0000', 'last_change': '2024-04-25T10:04:29.322341+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.322341+0000', 'last_undegraded': '2024-04-25T10:04:29.322341+0000', 'last_fullsized': '2024-04-25T10:04:29.322341+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:30.312402+0000', 'last_change': '2024-04-25T10:04:30.312402+0000', 'last_active': '2024-04-25T10:04:30.312402+0000', 'last_peered': '2024-04-25T10:04:30.312402+0000', 'last_clean': '2024-04-25T10:04:30.312402+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:30.312402+0000', 'last_undegraded': '2024-04-25T10:04:30.312402+0000', 'last_fullsized': '2024-04-25T10:04:30.312402+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:30.312402+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.16 is not active+clean 2024-04-25T10:04:34.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 156, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:04:29.324372+0000', 'last_change': '2024-04-25T10:04:29.324372+0000', 'last_active': '2024-04-25T10:04:29.308107+0000', 'last_peered': '2024-04-25T10:04:29.308107+0000', 'last_clean': '2024-04-25T10:04:29.308107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:29.324372+0000', 'last_undegraded': '2024-04-25T10:04:29.324372+0000', 'last_fullsized': '2024-04-25T10:04:29.324372+0000', 'mapping_epoch': 156, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 156, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:29.308107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:04:34.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:04:35.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.100+0000 7f3df1e41640 1 Processor -- start 2024-04-25T10:04:35.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3df1e41640 1 -- start start 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3df1e41640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3dec154ee0 0x7f3dec0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3df1e41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3df1e41640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec071480 con 0x7f3dec0cfcf0 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3df1e41640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dec072090 con 0x7f3dec154ee0 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3df1e41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec071230 con 0x7f3dec0ce590 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3deb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3dec154ee0 0x7f3dec0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.101+0000 7f3deaffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3dec0cfcf0 0x7f3dec0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36896/0 (socket says 172.21.15.148:36896) 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deb7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3dec154ee0 0x7f3dec0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41330/0 (socket says 172.21.15.148:41330) 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deaffd640 1 -- 172.21.15.148:0/1786093083 learned_addr learned my addr 172.21.15.148:0/1786093083 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3debfff640 1 --2- 172.21.15.148:0/1786093083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deb7fe640 1 -- 172.21.15.148:0/1786093083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 msgr2=0x7f3dec06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deb7fe640 1 --2- 172.21.15.148:0/1786093083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deb7fe640 1 -- 172.21.15.148:0/1786093083 >> v1:172.21.15.148:6789/0 conn(0x7f3dec0cfcf0 legacy=0x7f3dec0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deb7fe640 1 -- 172.21.15.148:0/1786093083 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec071760 con 0x7f3dec154ee0 2024-04-25T10:04:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.102+0000 7f3deb7fe640 1 --2- 172.21.15.148:0/1786093083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec0cf7b0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009d90 tx=0x7f3ddc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9183bc9ef99c43c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3dea7fc640 1 -- 172.21.15.148:0/1786093083 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc007740 con 0x7f3dec154ee0 2024-04-25T10:04:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3dea7fc640 1 -- 172.21.15.148:0/1786093083 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0078e0 con 0x7f3dec154ee0 2024-04-25T10:04:35.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3dea7fc640 1 -- 172.21.15.148:0/1786093083 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc007cb0 con 0x7f3dec154ee0 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 -- 172.21.15.148:0/1786093083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 msgr2=0x7f3dec0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 --2- 172.21.15.148:0/1786093083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec0cf7b0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009d90 tx=0x7f3ddc02a660 comp rx=0 tx=0).stop 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 -- 172.21.15.148:0/1786093083 shutdown_connections 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 --2- 172.21.15.148:0/1786093083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 --2- 172.21.15.148:0/1786093083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec0cf7b0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 -- 172.21.15.148:0/1786093083 >> 172.21.15.148:0/1786093083 conn(0x7f3dec06abe0 msgr2=0x7f3dec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 -- 172.21.15.148:0/1786093083 shutdown_connections 2024-04-25T10:04:35.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.103+0000 7f3df1e41640 1 -- 172.21.15.148:0/1786093083 wait complete. 2024-04-25T10:04:35.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 Processor -- start 2024-04-25T10:04:35.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 -- start start 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec1ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dec0cfcf0 0x7f3dec1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec1ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1552c0 con 0x7f3dec154ee0 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec071d20 con 0x7f3dec0cfcf0 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.105+0000 7f3df1e41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec155440 con 0x7f3dec0ce590 2024-04-25T10:04:35.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec1ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec1ee8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41342/0 (socket says 172.21.15.148:41342) 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 -- 172.21.15.148:0/3543535556 learned_addr learned my addr 172.21.15.148:0/3543535556 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3deb7fe640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec1ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 -- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 msgr2=0x7f3dec1ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec1ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 -- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dec0cfcf0 msgr2=0x7f3dec1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dec0cfcf0 0x7f3dec1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 -- 172.21.15.148:0/3543535556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec1f2540 con 0x7f3dec154ee0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3debfff640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec1ee8f0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d800 tx=0x7f3de400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400b6c0 con 0x7f3dec154ee0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de4019070 con 0x7f3dec154ee0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec1f2ac0 con 0x7f3dec154ee0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400bac0 con 0x7f3dec154ee0 2024-04-25T10:04:35.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.106+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec1f3080 con 0x7f3dec154ee0 2024-04-25T10:04:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.107+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec073990 con 0x7f3dec154ee0 2024-04-25T10:04:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.108+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3de400bc60 con 0x7f3dec154ee0 2024-04-25T10:04:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.108+0000 7f3de3fff640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dcc06b4d0 0x7f3dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.108+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3de408a870 con 0x7f3dec154ee0 2024-04-25T10:04:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.108+0000 7f3deb7fe640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dcc06b4d0 0x7f3dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.109+0000 7f3deb7fe640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dcc06b4d0 0x7f3dcc06d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc034000 tx=0x7f3ddc033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.112+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3dec1f2540 con 0x7f3dec154ee0 2024-04-25T10:04:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.351+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3dec06d360 con 0x7f3dec154ee0 2024-04-25T10:04:35.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.351+0000 7f3de3fff640 1 -- 172.21.15.148:0/3543535556 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1404 (secure 0 0 0) 0x7f3de4059560 con 0x7f3dec154ee0 2024-04-25T10:04:35.357 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:35.357 INFO:teuthology.orchestra.run.smithi148.stdout:{"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":159,"num_osds":8,"num_up_osds":8,"osd_up_since":1714038849,"num_in_osds":8,"osd_in_since":1714038841,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":445730816,"bytes_avail":772648382464,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":93,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"c","status":"up:active","gid":10365}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:04:21.143500+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dcc06b4d0 msgr2=0x7f3dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dcc06b4d0 0x7f3dcc06d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc034000 tx=0x7f3ddc033040 comp rx=0 tx=0).stop 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 msgr2=0x7f3dec1ee8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec1ee8f0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d800 tx=0x7f3de400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 shutdown_connections 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3dcc06b4d0 0x7f3dcc06d990 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dec154ee0 0x7f3dec1ee8f0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dec0cfcf0 0x7f3dec1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 --2- 172.21.15.148:0/3543535556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dec0ce590 0x7f3dec1ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 >> 172.21.15.148:0/3543535556 conn(0x7f3dec06abe0 msgr2=0x7f3dec15dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 shutdown_connections 2024-04-25T10:04:35.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.353+0000 7f3df1e41640 1 -- 172.21.15.148:0/3543535556 wait complete. 2024-04-25T10:04:35.370 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:04:35.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:35.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.461+0000 7f3876128640 1 Processor -- start 2024-04-25T10:04:35.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f3876128640 1 -- start start 2024-04-25T10:04:35.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f3876128640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387015dbe0 0x7f38700d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f3876128640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38700b92a0 0x7f38700d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f3876128640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38700b97a0 con 0x7f38700b2cb0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f3876128640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387015dfc0 con 0x7f38700b92a0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f3876128640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700d1a00 con 0x7f387015dbe0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f386ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38700b92a0 0x7f38700d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f386ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f38700b92a0 0x7f38700d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41348/0 (socket says 172.21.15.148:41348) 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.462+0000 7f386ffff640 1 -- 172.21.15.148:0/2706208143 learned_addr learned my addr 172.21.15.148:0/2706208143 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386f7fe640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387015dbe0 0x7f38700d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386ffff640 1 -- 172.21.15.148:0/2706208143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387015dbe0 msgr2=0x7f38700d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386ffff640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387015dbe0 0x7f38700d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386ffff640 1 -- 172.21.15.148:0/2706208143 >> v1:172.21.15.188:6789/0 conn(0x7f38700b2cb0 legacy=0x7f38700b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386ffff640 1 -- 172.21.15.148:0/2706208143 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700d1c50 con 0x7f38700b92a0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386f7fe640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387015dbe0 0x7f38700d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.463+0000 7f386ffff640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38700b92a0 0x7f38700d0e50 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f3868009e50 tx=0x7f386802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603a64093213308d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f386e7fc640 1 -- 172.21.15.148:0/2706208143 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868036070 con 0x7f38700b92a0 2024-04-25T10:04:35.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f386e7fc640 1 -- 172.21.15.148:0/2706208143 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3868031600 con 0x7f38700b92a0 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f386e7fc640 1 -- 172.21.15.148:0/2706208143 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38680319d0 con 0x7f38700b92a0 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 -- 172.21.15.148:0/2706208143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38700b92a0 msgr2=0x7f38700d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38700b92a0 0x7f38700d0e50 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f3868009e50 tx=0x7f386802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 -- 172.21.15.148:0/2706208143 shutdown_connections 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38700b92a0 0x7f38700d0e50 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 --2- 172.21.15.148:0/2706208143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387015dbe0 0x7f38700d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 -- 172.21.15.148:0/2706208143 >> 172.21.15.148:0/2706208143 conn(0x7f3870061a20 msgr2=0x7f3870061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 -- 172.21.15.148:0/2706208143 shutdown_connections 2024-04-25T10:04:35.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.464+0000 7f3876128640 1 -- 172.21.15.148:0/2706208143 wait complete. 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.465+0000 7f3876128640 1 Processor -- start 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 -- start start 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38700b2cb0 0x7f3870152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38700b92a0 0x7f3870154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 0x7f38701549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700b6ab0 con 0x7f387015dbe0 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700d2210 con 0x7f38700b2cb0 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f3876128640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700b6c30 con 0x7f38700b92a0 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f386ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 0x7f38701549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f386ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 0x7f38701549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41352/0 (socket says 172.21.15.148:41352) 2024-04-25T10:04:35.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.466+0000 7f386ffff640 1 -- 172.21.15.148:0/3331867250 learned_addr learned my addr 172.21.15.148:0/3331867250 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f386ffff640 1 -- 172.21.15.148:0/3331867250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38700b92a0 msgr2=0x7f3870154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f386ffff640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38700b92a0 0x7f3870154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f386ffff640 1 -- 172.21.15.148:0/3331867250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38700b2cb0 msgr2=0x7f3870152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f386ffff640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38700b2cb0 0x7f3870152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f386ffff640 1 -- 172.21.15.148:0/3331867250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700b3800 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f386ffff640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 0x7f38701549b0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f3868009e50 tx=0x7f3868004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868036050 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3868042070 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38700b3ae0 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868009510 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.467+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38700b40a0 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.468+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3870073990 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.468+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3868007bd0 con 0x7f387015dbe0 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.469+0000 7f3857fff640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f385c06b4d0 0x7f385c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.469+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38680ab2c0 con 0x7f387015dbe0 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.469+0000 7f386f7fe640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f385c06b4d0 0x7f385c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.469+0000 7f386f7fe640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f385c06b4d0 0x7f385c06d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3860006fd0 tx=0x7f3860008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.473+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f386807a040 con 0x7f387015dbe0 2024-04-25T10:04:35.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.640+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3870154ef0 con 0x7f385c06b4d0 2024-04-25T10:04:35.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.645+0000 7f3857fff640 1 -- 172.21.15.148:0/3331867250 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f3870154ef0 con 0x7f385c06b4d0 2024-04-25T10:04:35.653 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:35.654 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:04:35.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f385c06b4d0 msgr2=0x7f385c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f385c06b4d0 0x7f385c06d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3860006fd0 tx=0x7f3860008040 comp rx=0 tx=0).stop 2024-04-25T10:04:35.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 msgr2=0x7f38701549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 0x7f38701549b0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f3868009e50 tx=0x7f3868004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 shutdown_connections 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f385c06b4d0 0x7f385c06d990 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f387015dbe0 0x7f38701549b0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38700b92a0 0x7f3870154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 --2- 172.21.15.148:0/3331867250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38700b2cb0 0x7f3870152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.648+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 >> 172.21.15.148:0/3331867250 conn(0x7f3870061a20 msgr2=0x7f38700b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.649+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 shutdown_connections 2024-04-25T10:04:35.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:35.649+0000 7f3876128640 1 -- 172.21.15.148:0/3331867250 wait complete. 2024-04-25T10:04:35.665 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-04-25T10:04:35.146136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439820,"kb_used_data":16840,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534900,"statfs":{"total":773094113280,"available":772643737600,"internally_reserved":0,"allocated":17244160,"data_stored":8672948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.833324"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334159+0000","last_change":"2024-04-25T10:04:31.336044+0000","last_active":"2024-04-25T10:04:32.334159+0000","last_peered":"2024-04-25T10:04:32.334159+0000","last_clean":"2024-04-25T10:04:32.334159+0000","last_became_active":"2024-04-25T10:04:31.335749+0000","last_became_peered":"2024-04-25T10:04:31.335749+0000","last_unstale":"2024-04-25T10:04:32.334159+0000","last_undegraded":"2024-04-25T10:04:32.334159+0000","last_fullsized":"2024-04-25T10:04:32.334159+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.543072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411338+0000","last_change":"2024-04-25T10:04:31.348819+0000","last_active":"2024-04-25T10:04:33.411338+0000","last_peered":"2024-04-25T10:04:33.411338+0000","last_clean":"2024-04-25T10:04:33.411338+0000","last_became_active":"2024-04-25T10:04:31.348571+0000","last_became_peered":"2024-04-25T10:04:31.348571+0000","last_unstale":"2024-04-25T10:04:33.411338+0000","last_undegraded":"2024-04-25T10:04:33.411338+0000","last_fullsized":"2024-04-25T10:04:33.411338+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:09.930071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334005+0000","last_change":"2024-04-25T10:04:31.345946+0000","last_active":"2024-04-25T10:04:32.334005+0000","last_peered":"2024-04-25T10:04:32.334005+0000","last_clean":"2024-04-25T10:04:32.334005+0000","last_became_active":"2024-04-25T10:04:31.345698+0000","last_became_peered":"2024-04-25T10:04:31.345698+0000","last_unstale":"2024-04-25T10:04:32.334005+0000","last_undegraded":"2024-04-25T10:04:32.334005+0000","last_fullsized":"2024-04-25T10:04:32.334005+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:48.472529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334155+0000","last_change":"2024-04-25T10:04:31.344886+0000","last_active":"2024-04-25T10:04:32.334155+0000","last_peered":"2024-04-25T10:04:32.334155+0000","last_clean":"2024-04-25T10:04:32.334155+0000","last_became_active":"2024-04-25T10:04:31.344576+0000","last_became_peered":"2024-04-25T10:04:31.344576+0000","last_unstale":"2024-04-25T10:04:32.334155+0000","last_undegraded":"2024-04-25T10:04:32.334155+0000","last_fullsized":"2024-04-25T10:04:32.334155+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.660769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331390+0000","last_change":"2024-04-25T10:04:31.352554+0000","last_active":"2024-04-25T10:04:32.331390+0000","last_peered":"2024-04-25T10:04:32.331390+0000","last_clean":"2024-04-25T10:04:32.331390+0000","last_became_active":"2024-04-25T10:04:31.352243+0000","last_became_peered":"2024-04-25T10:04:31.352243+0000","last_unstale":"2024-04-25T10:04:32.331390+0000","last_undegraded":"2024-04-25T10:04:32.331390+0000","last_fullsized":"2024-04-25T10:04:32.331390+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:13.406702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410545+0000","last_change":"2024-04-25T10:04:31.336152+0000","last_active":"2024-04-25T10:04:33.410545+0000","last_peered":"2024-04-25T10:04:33.410545+0000","last_clean":"2024-04-25T10:04:33.410545+0000","last_became_active":"2024-04-25T10:04:31.335695+0000","last_became_peered":"2024-04-25T10:04:31.335695+0000","last_unstale":"2024-04-25T10:04:33.410545+0000","last_undegraded":"2024-04-25T10:04:33.410545+0000","last_fullsized":"2024-04-25T10:04:33.410545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:52.255652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357297+0000","last_change":"2024-04-25T10:04:31.342792+0000","last_active":"2024-04-25T10:04:32.357297+0000","last_peered":"2024-04-25T10:04:32.357297+0000","last_clean":"2024-04-25T10:04:32.357297+0000","last_became_active":"2024-04-25T10:04:31.342524+0000","last_became_peered":"2024-04-25T10:04:31.342524+0000","last_unstale":"2024-04-25T10:04:32.357297+0000","last_undegraded":"2024-04-25T10:04:32.357297+0000","last_fullsized":"2024-04-25T10:04:32.357297+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:09.930071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.356999+0000","last_change":"2024-04-25T10:04:31.336921+0000","last_active":"2024-04-25T10:04:32.356999+0000","last_peered":"2024-04-25T10:04:32.356999+0000","last_clean":"2024-04-25T10:04:32.356999+0000","last_became_active":"2024-04-25T10:04:31.336293+0000","last_became_peered":"2024-04-25T10:04:31.336293+0000","last_unstale":"2024-04-25T10:04:32.356999+0000","last_undegraded":"2024-04-25T10:04:32.356999+0000","last_fullsized":"2024-04-25T10:04:32.356999+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:13.580436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357211+0000","last_change":"2024-04-25T10:04:31.337363+0000","last_active":"2024-04-25T10:04:32.357211+0000","last_peered":"2024-04-25T10:04:32.357211+0000","last_clean":"2024-04-25T10:04:32.357211+0000","last_became_active":"2024-04-25T10:04:31.337084+0000","last_became_peered":"2024-04-25T10:04:31.337084+0000","last_unstale":"2024-04-25T10:04:32.357211+0000","last_undegraded":"2024-04-25T10:04:32.357211+0000","last_fullsized":"2024-04-25T10:04:32.357211+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357321+0000","last_change":"2024-04-25T10:04:31.346554+0000","last_active":"2024-04-25T10:04:32.357321+0000","last_peered":"2024-04-25T10:04:32.357321+0000","last_clean":"2024-04-25T10:04:32.357321+0000","last_became_active":"2024-04-25T10:04:31.346281+0000","last_became_peered":"2024-04-25T10:04:31.346281+0000","last_unstale":"2024-04-25T10:04:32.357321+0000","last_undegraded":"2024-04-25T10:04:32.357321+0000","last_fullsized":"2024-04-25T10:04:32.357321+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410902+0000","last_change":"2024-04-25T10:04:31.347754+0000","last_active":"2024-04-25T10:04:33.410902+0000","last_peered":"2024-04-25T10:04:33.410902+0000","last_clean":"2024-04-25T10:04:33.410902+0000","last_became_active":"2024-04-25T10:04:31.347433+0000","last_became_peered":"2024-04-25T10:04:31.347433+0000","last_unstale":"2024-04-25T10:04:33.410902+0000","last_undegraded":"2024-04-25T10:04:33.410902+0000","last_fullsized":"2024-04-25T10:04:33.410902+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410867+0000","last_change":"2024-04-25T10:04:31.337133+0000","last_active":"2024-04-25T10:04:33.410867+0000","last_peered":"2024-04-25T10:04:33.410867+0000","last_clean":"2024-04-25T10:04:33.410867+0000","last_became_active":"2024-04-25T10:04:31.336195+0000","last_became_peered":"2024-04-25T10:04:31.336195+0000","last_unstale":"2024-04-25T10:04:33.410867+0000","last_undegraded":"2024-04-25T10:04:33.410867+0000","last_fullsized":"2024-04-25T10:04:33.410867+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.448151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355335+0000","last_change":"2024-04-25T10:04:31.344786+0000","last_active":"2024-04-25T10:04:32.355335+0000","last_peered":"2024-04-25T10:04:32.355335+0000","last_clean":"2024-04-25T10:04:32.355335+0000","last_became_active":"2024-04-25T10:04:31.344287+0000","last_became_peered":"2024-04-25T10:04:31.344287+0000","last_unstale":"2024-04-25T10:04:32.355335+0000","last_undegraded":"2024-04-25T10:04:32.355335+0000","last_fullsized":"2024-04-25T10:04:32.355335+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:42.362812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331010+0000","last_change":"2024-04-25T10:04:31.343226+0000","last_active":"2024-04-25T10:04:32.331010+0000","last_peered":"2024-04-25T10:04:32.331010+0000","last_clean":"2024-04-25T10:04:32.331010+0000","last_became_active":"2024-04-25T10:04:31.342925+0000","last_became_peered":"2024-04-25T10:04:31.342925+0000","last_unstale":"2024-04-25T10:04:32.331010+0000","last_undegraded":"2024-04-25T10:04:32.331010+0000","last_fullsized":"2024-04-25T10:04:32.331010+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:17.535351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410432+0000","last_change":"2024-04-25T10:04:31.347121+0000","last_active":"2024-04-25T10:04:33.410432+0000","last_peered":"2024-04-25T10:04:33.410432+0000","last_clean":"2024-04-25T10:04:33.410432+0000","last_became_active":"2024-04-25T10:04:31.346846+0000","last_became_peered":"2024-04-25T10:04:31.346846+0000","last_unstale":"2024-04-25T10:04:33.410432+0000","last_undegraded":"2024-04-25T10:04:33.410432+0000","last_fullsized":"2024-04-25T10:04:33.410432+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:29.431821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333680+0000","last_change":"2024-04-25T10:04:31.345041+0000","last_active":"2024-04-25T10:04:32.333680+0000","last_peered":"2024-04-25T10:04:32.333680+0000","last_clean":"2024-04-25T10:04:32.333680+0000","last_became_active":"2024-04-25T10:04:31.344756+0000","last_became_peered":"2024-04-25T10:04:31.344756+0000","last_unstale":"2024-04-25T10:04:32.333680+0000","last_undegraded":"2024-04-25T10:04:32.333680+0000","last_fullsized":"2024-04-25T10:04:32.333680+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:54.198699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357421+0000","last_change":"2024-04-25T10:04:31.348783+0000","last_active":"2024-04-25T10:04:32.357421+0000","last_peered":"2024-04-25T10:04:32.357421+0000","last_clean":"2024-04-25T10:04:32.357421+0000","last_became_active":"2024-04-25T10:04:31.348458+0000","last_became_peered":"2024-04-25T10:04:31.348458+0000","last_unstale":"2024-04-25T10:04:32.357421+0000","last_undegraded":"2024-04-25T10:04:32.357421+0000","last_fullsized":"2024-04-25T10:04:32.357421+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:00.332191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411368+0000","last_change":"2024-04-25T10:04:31.340150+0000","last_active":"2024-04-25T10:04:33.411368+0000","last_peered":"2024-04-25T10:04:33.411368+0000","last_clean":"2024-04-25T10:04:33.411368+0000","last_became_active":"2024-04-25T10:04:31.339875+0000","last_became_peered":"2024-04-25T10:04:31.339875+0000","last_unstale":"2024-04-25T10:04:33.411368+0000","last_undegraded":"2024-04-25T10:04:33.411368+0000","last_fullsized":"2024-04-25T10:04:33.411368+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.305856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411144+0000","last_change":"2024-04-25T10:04:31.346786+0000","last_active":"2024-04-25T10:04:33.411144+0000","last_peered":"2024-04-25T10:04:33.411144+0000","last_clean":"2024-04-25T10:04:33.411144+0000","last_became_active":"2024-04-25T10:04:31.346469+0000","last_became_peered":"2024-04-25T10:04:31.346469+0000","last_unstale":"2024-04-25T10:04:33.411144+0000","last_undegraded":"2024-04-25T10:04:33.411144+0000","last_fullsized":"2024-04-25T10:04:33.411144+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:09.930071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412764+0000","last_change":"2024-04-25T10:04:31.348231+0000","last_active":"2024-04-25T10:04:32.412764+0000","last_peered":"2024-04-25T10:04:32.412764+0000","last_clean":"2024-04-25T10:04:32.412764+0000","last_became_active":"2024-04-25T10:04:31.347954+0000","last_became_peered":"2024-04-25T10:04:31.347954+0000","last_unstale":"2024-04-25T10:04:32.412764+0000","last_undegraded":"2024-04-25T10:04:32.412764+0000","last_fullsized":"2024-04-25T10:04:32.412764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.645241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355542+0000","last_change":"2024-04-25T10:04:31.350292+0000","last_active":"2024-04-25T10:04:32.355542+0000","last_peered":"2024-04-25T10:04:32.355542+0000","last_clean":"2024-04-25T10:04:32.355542+0000","last_became_active":"2024-04-25T10:04:31.350018+0000","last_became_peered":"2024-04-25T10:04:31.350018+0000","last_unstale":"2024-04-25T10:04:32.355542+0000","last_undegraded":"2024-04-25T10:04:32.355542+0000","last_fullsized":"2024-04-25T10:04:32.355542+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:21.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410743+0000","last_change":"2024-04-25T10:04:31.346930+0000","last_active":"2024-04-25T10:04:33.410743+0000","last_peered":"2024-04-25T10:04:33.410743+0000","last_clean":"2024-04-25T10:04:33.410743+0000","last_became_active":"2024-04-25T10:04:31.346607+0000","last_became_peered":"2024-04-25T10:04:31.346607+0000","last_unstale":"2024-04-25T10:04:33.410743+0000","last_undegraded":"2024-04-25T10:04:33.410743+0000","last_fullsized":"2024-04-25T10:04:33.410743+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:29.431821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.330905+0000","last_change":"2024-04-25T10:04:31.345657+0000","last_active":"2024-04-25T10:04:32.330905+0000","last_peered":"2024-04-25T10:04:32.330905+0000","last_clean":"2024-04-25T10:04:32.330905+0000","last_became_active":"2024-04-25T10:04:31.345314+0000","last_became_peered":"2024-04-25T10:04:31.345314+0000","last_unstale":"2024-04-25T10:04:32.330905+0000","last_undegraded":"2024-04-25T10:04:32.330905+0000","last_fullsized":"2024-04-25T10:04:32.330905+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:08.132401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333894+0000","last_change":"2024-04-25T10:04:31.345395+0000","last_active":"2024-04-25T10:04:32.333894+0000","last_peered":"2024-04-25T10:04:32.333894+0000","last_clean":"2024-04-25T10:04:32.333894+0000","last_became_active":"2024-04-25T10:04:31.345077+0000","last_became_peered":"2024-04-25T10:04:31.345077+0000","last_unstale":"2024-04-25T10:04:32.333894+0000","last_undegraded":"2024-04-25T10:04:32.333894+0000","last_fullsized":"2024-04-25T10:04:32.333894+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:33.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355643+0000","last_change":"2024-04-25T10:04:31.347607+0000","last_active":"2024-04-25T10:04:32.355643+0000","last_peered":"2024-04-25T10:04:32.355643+0000","last_clean":"2024-04-25T10:04:32.355643+0000","last_became_active":"2024-04-25T10:04:31.347335+0000","last_became_peered":"2024-04-25T10:04:31.347335+0000","last_unstale":"2024-04-25T10:04:32.355643+0000","last_undegraded":"2024-04-25T10:04:32.355643+0000","last_fullsized":"2024-04-25T10:04:32.355643+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:53.054606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412863+0000","last_change":"2024-04-25T10:04:31.345513+0000","last_active":"2024-04-25T10:04:32.412863+0000","last_peered":"2024-04-25T10:04:32.412863+0000","last_clean":"2024-04-25T10:04:32.412863+0000","last_became_active":"2024-04-25T10:04:31.345173+0000","last_became_peered":"2024-04-25T10:04:31.345173+0000","last_unstale":"2024-04-25T10:04:32.412863+0000","last_undegraded":"2024-04-25T10:04:32.412863+0000","last_fullsized":"2024-04-25T10:04:32.412863+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:40.810112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410676+0000","last_change":"2024-04-25T10:04:31.347391+0000","last_active":"2024-04-25T10:04:33.410676+0000","last_peered":"2024-04-25T10:04:33.410676+0000","last_clean":"2024-04-25T10:04:33.410676+0000","last_became_active":"2024-04-25T10:04:31.347108+0000","last_became_peered":"2024-04-25T10:04:31.347108+0000","last_unstale":"2024-04-25T10:04:33.410676+0000","last_undegraded":"2024-04-25T10:04:33.410676+0000","last_fullsized":"2024-04-25T10:04:33.410676+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:58.889086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355543+0000","last_change":"2024-04-25T10:04:31.352074+0000","last_active":"2024-04-25T10:04:32.355543+0000","last_peered":"2024-04-25T10:04:32.355543+0000","last_clean":"2024-04-25T10:04:32.355543+0000","last_became_active":"2024-04-25T10:04:31.351809+0000","last_became_peered":"2024-04-25T10:04:31.351809+0000","last_unstale":"2024-04-25T10:04:32.355543+0000","last_undegraded":"2024-04-25T10:04:32.355543+0000","last_fullsized":"2024-04-25T10:04:32.355543+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:45.835666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411066+0000","last_change":"2024-04-25T10:04:31.340746+0000","last_active":"2024-04-25T10:04:33.411066+0000","last_peered":"2024-04-25T10:04:33.411066+0000","last_clean":"2024-04-25T10:04:33.411066+0000","last_became_active":"2024-04-25T10:04:31.340457+0000","last_became_peered":"2024-04-25T10:04:31.340457+0000","last_unstale":"2024-04-25T10:04:33.411066+0000","last_undegraded":"2024-04-25T10:04:33.411066+0000","last_fullsized":"2024-04-25T10:04:33.411066+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:55.547730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411112+0000","last_change":"2024-04-25T10:04:31.349990+0000","last_active":"2024-04-25T10:04:33.411112+0000","last_peered":"2024-04-25T10:04:33.411112+0000","last_clean":"2024-04-25T10:04:33.411112+0000","last_became_active":"2024-04-25T10:04:31.349412+0000","last_became_peered":"2024-04-25T10:04:31.349412+0000","last_unstale":"2024-04-25T10:04:33.411112+0000","last_undegraded":"2024-04-25T10:04:33.411112+0000","last_fullsized":"2024-04-25T10:04:33.411112+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.269103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410624+0000","last_change":"2024-04-25T10:04:31.344992+0000","last_active":"2024-04-25T10:04:33.410624+0000","last_peered":"2024-04-25T10:04:33.410624+0000","last_clean":"2024-04-25T10:04:33.410624+0000","last_became_active":"2024-04-25T10:04:31.344734+0000","last_became_peered":"2024-04-25T10:04:31.344734+0000","last_unstale":"2024-04-25T10:04:33.410624+0000","last_undegraded":"2024-04-25T10:04:33.410624+0000","last_fullsized":"2024-04-25T10:04:33.410624+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.448151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410432+0000","last_change":"2024-04-25T10:04:31.340905+0000","last_active":"2024-04-25T10:04:33.410432+0000","last_peered":"2024-04-25T10:04:33.410432+0000","last_clean":"2024-04-25T10:04:33.410432+0000","last_became_active":"2024-04-25T10:04:31.340420+0000","last_became_peered":"2024-04-25T10:04:31.340420+0000","last_unstale":"2024-04-25T10:04:33.410432+0000","last_undegraded":"2024-04-25T10:04:33.410432+0000","last_fullsized":"2024-04-25T10:04:33.410432+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:12.279696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411497+0000","last_change":"2024-04-25T10:04:31.338175+0000","last_active":"2024-04-25T10:04:33.411497+0000","last_peered":"2024-04-25T10:04:33.411497+0000","last_clean":"2024-04-25T10:04:33.411497+0000","last_became_active":"2024-04-25T10:04:31.337918+0000","last_became_peered":"2024-04-25T10:04:31.337918+0000","last_unstale":"2024-04-25T10:04:33.411497+0000","last_undegraded":"2024-04-25T10:04:33.411497+0000","last_fullsized":"2024-04-25T10:04:33.411497+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:00.332191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334234+0000","last_change":"2024-04-25T10:04:30.320042+0000","last_active":"2024-04-25T10:04:32.334234+0000","last_peered":"2024-04-25T10:04:32.334234+0000","last_clean":"2024-04-25T10:04:32.334234+0000","last_became_active":"2024-04-25T10:04:30.319750+0000","last_became_peered":"2024-04-25T10:04:30.319750+0000","last_unstale":"2024-04-25T10:04:32.334234+0000","last_undegraded":"2024-04-25T10:04:32.334234+0000","last_fullsized":"2024-04-25T10:04:32.334234+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:39.849636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355231+0000","last_change":"2024-04-25T10:04:31.344644+0000","last_active":"2024-04-25T10:04:32.355231+0000","last_peered":"2024-04-25T10:04:32.355231+0000","last_clean":"2024-04-25T10:04:32.355231+0000","last_became_active":"2024-04-25T10:04:31.344368+0000","last_became_peered":"2024-04-25T10:04:31.344368+0000","last_unstale":"2024-04-25T10:04:32.355231+0000","last_undegraded":"2024-04-25T10:04:32.355231+0000","last_fullsized":"2024-04-25T10:04:32.355231+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:05.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331094+0000","last_change":"2024-04-25T10:04:30.334823+0000","last_active":"2024-04-25T10:04:32.331094+0000","last_peered":"2024-04-25T10:04:32.331094+0000","last_clean":"2024-04-25T10:04:32.331094+0000","last_became_active":"2024-04-25T10:04:30.334535+0000","last_became_peered":"2024-04-25T10:04:30.334535+0000","last_unstale":"2024-04-25T10:04:32.331094+0000","last_undegraded":"2024-04-25T10:04:32.331094+0000","last_fullsized":"2024-04-25T10:04:32.331094+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:14.617748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331194+0000","last_change":"2024-04-25T10:04:31.342584+0000","last_active":"2024-04-25T10:04:32.331194+0000","last_peered":"2024-04-25T10:04:32.331194+0000","last_clean":"2024-04-25T10:04:32.331194+0000","last_became_active":"2024-04-25T10:04:31.342308+0000","last_became_peered":"2024-04-25T10:04:31.342308+0000","last_unstale":"2024-04-25T10:04:32.331194+0000","last_undegraded":"2024-04-25T10:04:32.331194+0000","last_fullsized":"2024-04-25T10:04:32.331194+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:57.359501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411186+0000","last_change":"2024-04-25T10:04:30.329348+0000","last_active":"2024-04-25T10:04:33.411186+0000","last_peered":"2024-04-25T10:04:33.411186+0000","last_clean":"2024-04-25T10:04:33.411186+0000","last_became_active":"2024-04-25T10:04:30.328793+0000","last_became_peered":"2024-04-25T10:04:30.328793+0000","last_unstale":"2024-04-25T10:04:33.411186+0000","last_undegraded":"2024-04-25T10:04:33.411186+0000","last_fullsized":"2024-04-25T10:04:33.411186+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:51.614469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411206+0000","last_change":"2024-04-25T10:04:31.345031+0000","last_active":"2024-04-25T10:04:33.411206+0000","last_peered":"2024-04-25T10:04:33.411206+0000","last_clean":"2024-04-25T10:04:33.411206+0000","last_became_active":"2024-04-25T10:04:31.344742+0000","last_became_peered":"2024-04-25T10:04:31.344742+0000","last_unstale":"2024-04-25T10:04:33.411206+0000","last_undegraded":"2024-04-25T10:04:33.411206+0000","last_fullsized":"2024-04-25T10:04:33.411206+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:18.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412601+0000","last_change":"2024-04-25T10:04:30.334748+0000","last_active":"2024-04-25T10:04:32.412601+0000","last_peered":"2024-04-25T10:04:32.412601+0000","last_clean":"2024-04-25T10:04:32.412601+0000","last_became_active":"2024-04-25T10:04:30.334301+0000","last_became_peered":"2024-04-25T10:04:30.334301+0000","last_unstale":"2024-04-25T10:04:32.412601+0000","last_undegraded":"2024-04-25T10:04:32.412601+0000","last_fullsized":"2024-04-25T10:04:32.412601+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:29.492631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357338+0000","last_change":"2024-04-25T10:04:31.338459+0000","last_active":"2024-04-25T10:04:32.357338+0000","last_peered":"2024-04-25T10:04:32.357338+0000","last_clean":"2024-04-25T10:04:32.357338+0000","last_became_active":"2024-04-25T10:04:31.338210+0000","last_became_peered":"2024-04-25T10:04:31.338210+0000","last_unstale":"2024-04-25T10:04:32.357338+0000","last_undegraded":"2024-04-25T10:04:32.357338+0000","last_fullsized":"2024-04-25T10:04:32.357338+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:18.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357299+0000","last_change":"2024-04-25T10:04:30.338677+0000","last_active":"2024-04-25T10:04:32.357299+0000","last_peered":"2024-04-25T10:04:32.357299+0000","last_clean":"2024-04-25T10:04:32.357299+0000","last_became_active":"2024-04-25T10:04:30.338427+0000","last_became_peered":"2024-04-25T10:04:30.338427+0000","last_unstale":"2024-04-25T10:04:32.357299+0000","last_undegraded":"2024-04-25T10:04:32.357299+0000","last_fullsized":"2024-04-25T10:04:32.357299+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:18.535207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412884+0000","last_change":"2024-04-25T10:04:31.337628+0000","last_active":"2024-04-25T10:04:32.412884+0000","last_peered":"2024-04-25T10:04:32.412884+0000","last_clean":"2024-04-25T10:04:32.412884+0000","last_became_active":"2024-04-25T10:04:31.337381+0000","last_became_peered":"2024-04-25T10:04:31.337381+0000","last_unstale":"2024-04-25T10:04:32.412884+0000","last_undegraded":"2024-04-25T10:04:32.412884+0000","last_fullsized":"2024-04-25T10:04:32.412884+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.727573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331090+0000","last_change":"2024-04-25T10:04:30.330388+0000","last_active":"2024-04-25T10:04:32.331090+0000","last_peered":"2024-04-25T10:04:32.331090+0000","last_clean":"2024-04-25T10:04:32.331090+0000","last_became_active":"2024-04-25T10:04:30.330137+0000","last_became_peered":"2024-04-25T10:04:30.330137+0000","last_unstale":"2024-04-25T10:04:32.331090+0000","last_undegraded":"2024-04-25T10:04:32.331090+0000","last_fullsized":"2024-04-25T10:04:32.331090+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:38.969075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412929+0000","last_change":"2024-04-25T10:04:31.338813+0000","last_active":"2024-04-25T10:04:32.412929+0000","last_peered":"2024-04-25T10:04:32.412929+0000","last_clean":"2024-04-25T10:04:32.412929+0000","last_became_active":"2024-04-25T10:04:31.338329+0000","last_became_peered":"2024-04-25T10:04:31.338329+0000","last_unstale":"2024-04-25T10:04:32.412929+0000","last_undegraded":"2024-04-25T10:04:32.412929+0000","last_fullsized":"2024-04-25T10:04:32.412929+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:50.757842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412516+0000","last_change":"2024-04-25T10:04:30.322317+0000","last_active":"2024-04-25T10:04:32.412516+0000","last_peered":"2024-04-25T10:04:32.412516+0000","last_clean":"2024-04-25T10:04:32.412516+0000","last_became_active":"2024-04-25T10:04:30.321997+0000","last_became_peered":"2024-04-25T10:04:30.321997+0000","last_unstale":"2024-04-25T10:04:32.412516+0000","last_undegraded":"2024-04-25T10:04:32.412516+0000","last_fullsized":"2024-04-25T10:04:32.412516+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:13.472641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334334+0000","last_change":"2024-04-25T10:04:31.348567+0000","last_active":"2024-04-25T10:04:32.334334+0000","last_peered":"2024-04-25T10:04:32.334334+0000","last_clean":"2024-04-25T10:04:32.334334+0000","last_became_active":"2024-04-25T10:04:31.348232+0000","last_became_peered":"2024-04-25T10:04:31.348232+0000","last_unstale":"2024-04-25T10:04:32.334334+0000","last_undegraded":"2024-04-25T10:04:32.334334+0000","last_fullsized":"2024-04-25T10:04:32.334334+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:37.874922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357178+0000","last_change":"2024-04-25T10:04:30.333941+0000","last_active":"2024-04-25T10:04:32.357178+0000","last_peered":"2024-04-25T10:04:32.357178+0000","last_clean":"2024-04-25T10:04:32.357178+0000","last_became_active":"2024-04-25T10:04:30.333598+0000","last_became_peered":"2024-04-25T10:04:30.333598+0000","last_unstale":"2024-04-25T10:04:32.357178+0000","last_undegraded":"2024-04-25T10:04:32.357178+0000","last_fullsized":"2024-04-25T10:04:32.357178+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.656473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411429+0000","last_change":"2024-04-25T10:04:31.345733+0000","last_active":"2024-04-25T10:04:33.411429+0000","last_peered":"2024-04-25T10:04:33.411429+0000","last_clean":"2024-04-25T10:04:33.411429+0000","last_became_active":"2024-04-25T10:04:31.345461+0000","last_became_peered":"2024-04-25T10:04:31.345461+0000","last_unstale":"2024-04-25T10:04:33.411429+0000","last_undegraded":"2024-04-25T10:04:33.411429+0000","last_fullsized":"2024-04-25T10:04:33.411429+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.498360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355534+0000","last_change":"2024-04-25T10:04:30.331610+0000","last_active":"2024-04-25T10:04:32.355534+0000","last_peered":"2024-04-25T10:04:32.355534+0000","last_clean":"2024-04-25T10:04:32.355534+0000","last_became_active":"2024-04-25T10:04:30.331347+0000","last_became_peered":"2024-04-25T10:04:30.331347+0000","last_unstale":"2024-04-25T10:04:32.355534+0000","last_undegraded":"2024-04-25T10:04:32.355534+0000","last_fullsized":"2024-04-25T10:04:32.355534+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:34.619632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355501+0000","last_change":"2024-04-25T10:04:31.339104+0000","last_active":"2024-04-25T10:04:32.355501+0000","last_peered":"2024-04-25T10:04:32.355501+0000","last_clean":"2024-04-25T10:04:32.355501+0000","last_became_active":"2024-04-25T10:04:31.337317+0000","last_became_peered":"2024-04-25T10:04:31.337317+0000","last_unstale":"2024-04-25T10:04:32.355501+0000","last_undegraded":"2024-04-25T10:04:32.355501+0000","last_fullsized":"2024-04-25T10:04:32.355501+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:24.477923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":17,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333862+0000","last_change":"2024-04-25T10:04:30.328347+0000","last_active":"2024-04-25T10:04:32.333862+0000","last_peered":"2024-04-25T10:04:32.333862+0000","last_clean":"2024-04-25T10:04:32.333862+0000","last_became_active":"2024-04-25T10:04:30.328046+0000","last_became_peered":"2024-04-25T10:04:30.328046+0000","last_unstale":"2024-04-25T10:04:32.333862+0000","last_undegraded":"2024-04-25T10:04:32.333862+0000","last_fullsized":"2024-04-25T10:04:32.333862+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:50.632890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412807+0000","last_change":"2024-04-25T10:04:31.352533+0000","last_active":"2024-04-25T10:04:32.412807+0000","last_peered":"2024-04-25T10:04:32.412807+0000","last_clean":"2024-04-25T10:04:32.412807+0000","last_became_active":"2024-04-25T10:04:31.352218+0000","last_became_peered":"2024-04-25T10:04:31.352218+0000","last_unstale":"2024-04-25T10:04:32.412807+0000","last_undegraded":"2024-04-25T10:04:32.412807+0000","last_fullsized":"2024-04-25T10:04:32.412807+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:50.954932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"158'6","reported_seq":20,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355599+0000","last_change":"2024-04-25T10:04:30.332873+0000","last_active":"2024-04-25T10:04:32.355599+0000","last_peered":"2024-04-25T10:04:32.355599+0000","last_clean":"2024-04-25T10:04:32.355599+0000","last_became_active":"2024-04-25T10:04:30.332601+0000","last_became_peered":"2024-04-25T10:04:30.332601+0000","last_unstale":"2024-04-25T10:04:32.355599+0000","last_undegraded":"2024-04-25T10:04:32.355599+0000","last_fullsized":"2024-04-25T10:04:32.355599+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.735149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411280+0000","last_change":"2024-04-25T10:04:31.340618+0000","last_active":"2024-04-25T10:04:33.411280+0000","last_peered":"2024-04-25T10:04:33.411280+0000","last_clean":"2024-04-25T10:04:33.411280+0000","last_became_active":"2024-04-25T10:04:31.339372+0000","last_became_peered":"2024-04-25T10:04:31.339372+0000","last_unstale":"2024-04-25T10:04:33.411280+0000","last_undegraded":"2024-04-25T10:04:33.411280+0000","last_fullsized":"2024-04-25T10:04:33.411280+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:50.386736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331368+0000","last_change":"2024-04-25T10:04:30.331087+0000","last_active":"2024-04-25T10:04:32.331368+0000","last_peered":"2024-04-25T10:04:32.331368+0000","last_clean":"2024-04-25T10:04:32.331368+0000","last_became_active":"2024-04-25T10:04:30.330823+0000","last_became_peered":"2024-04-25T10:04:30.330823+0000","last_unstale":"2024-04-25T10:04:32.331368+0000","last_undegraded":"2024-04-25T10:04:32.331368+0000","last_fullsized":"2024-04-25T10:04:32.331368+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:25.494064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357398+0000","last_change":"2024-04-25T10:04:31.345933+0000","last_active":"2024-04-25T10:04:32.357398+0000","last_peered":"2024-04-25T10:04:32.357398+0000","last_clean":"2024-04-25T10:04:32.357398+0000","last_became_active":"2024-04-25T10:04:31.345656+0000","last_became_peered":"2024-04-25T10:04:31.345656+0000","last_unstale":"2024-04-25T10:04:32.357398+0000","last_undegraded":"2024-04-25T10:04:32.357398+0000","last_fullsized":"2024-04-25T10:04:32.357398+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.498360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355253+0000","last_change":"2024-04-25T10:04:30.326763+0000","last_active":"2024-04-25T10:04:32.355253+0000","last_peered":"2024-04-25T10:04:32.355253+0000","last_clean":"2024-04-25T10:04:32.355253+0000","last_became_active":"2024-04-25T10:04:30.326309+0000","last_became_peered":"2024-04-25T10:04:30.326309+0000","last_unstale":"2024-04-25T10:04:32.355253+0000","last_undegraded":"2024-04-25T10:04:32.355253+0000","last_fullsized":"2024-04-25T10:04:32.355253+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:14.617748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411275+0000","last_change":"2024-04-25T10:04:31.347835+0000","last_active":"2024-04-25T10:04:33.411275+0000","last_peered":"2024-04-25T10:04:33.411275+0000","last_clean":"2024-04-25T10:04:33.411275+0000","last_became_active":"2024-04-25T10:04:31.347455+0000","last_became_peered":"2024-04-25T10:04:31.347455+0000","last_unstale":"2024-04-25T10:04:33.411275+0000","last_undegraded":"2024-04-25T10:04:33.411275+0000","last_fullsized":"2024-04-25T10:04:33.411275+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410884+0000","last_change":"2024-04-25T10:04:30.332586+0000","last_active":"2024-04-25T10:04:33.410884+0000","last_peered":"2024-04-25T10:04:33.410884+0000","last_clean":"2024-04-25T10:04:33.410884+0000","last_became_active":"2024-04-25T10:04:30.332228+0000","last_became_peered":"2024-04-25T10:04:30.332228+0000","last_unstale":"2024-04-25T10:04:33.410884+0000","last_undegraded":"2024-04-25T10:04:33.410884+0000","last_fullsized":"2024-04-25T10:04:33.410884+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:18.535207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412894+0000","last_change":"2024-04-25T10:04:31.344684+0000","last_active":"2024-04-25T10:04:32.412894+0000","last_peered":"2024-04-25T10:04:32.412894+0000","last_clean":"2024-04-25T10:04:32.412894+0000","last_became_active":"2024-04-25T10:04:31.344419+0000","last_became_peered":"2024-04-25T10:04:31.344419+0000","last_unstale":"2024-04-25T10:04:32.412894+0000","last_undegraded":"2024-04-25T10:04:32.412894+0000","last_fullsized":"2024-04-25T10:04:32.412894+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.419422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"158'8","reported_seq":22,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357105+0000","last_change":"2024-04-25T10:04:30.334000+0000","last_active":"2024-04-25T10:04:32.357105+0000","last_peered":"2024-04-25T10:04:32.357105+0000","last_clean":"2024-04-25T10:04:32.357105+0000","last_became_active":"2024-04-25T10:04:30.333633+0000","last_became_peered":"2024-04-25T10:04:30.333633+0000","last_unstale":"2024-04-25T10:04:32.357105+0000","last_undegraded":"2024-04-25T10:04:32.357105+0000","last_fullsized":"2024-04-25T10:04:32.357105+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.870627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.356999+0000","last_change":"2024-04-25T10:04:31.338320+0000","last_active":"2024-04-25T10:04:32.356999+0000","last_peered":"2024-04-25T10:04:32.356999+0000","last_clean":"2024-04-25T10:04:32.356999+0000","last_became_active":"2024-04-25T10:04:31.338004+0000","last_became_peered":"2024-04-25T10:04:31.338004+0000","last_unstale":"2024-04-25T10:04:32.356999+0000","last_undegraded":"2024-04-25T10:04:32.356999+0000","last_fullsized":"2024-04-25T10:04:32.356999+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.448151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355633+0000","last_change":"2024-04-25T10:04:30.321626+0000","last_active":"2024-04-25T10:04:32.355633+0000","last_peered":"2024-04-25T10:04:32.355633+0000","last_clean":"2024-04-25T10:04:32.355633+0000","last_became_active":"2024-04-25T10:04:30.321309+0000","last_became_peered":"2024-04-25T10:04:30.321309+0000","last_unstale":"2024-04-25T10:04:32.355633+0000","last_undegraded":"2024-04-25T10:04:32.355633+0000","last_fullsized":"2024-04-25T10:04:32.355633+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:23.214192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411081+0000","last_change":"2024-04-25T10:04:31.342568+0000","last_active":"2024-04-25T10:04:33.411081+0000","last_peered":"2024-04-25T10:04:33.411081+0000","last_clean":"2024-04-25T10:04:33.411081+0000","last_became_active":"2024-04-25T10:04:31.342305+0000","last_became_peered":"2024-04-25T10:04:31.342305+0000","last_unstale":"2024-04-25T10:04:33.411081+0000","last_undegraded":"2024-04-25T10:04:33.411081+0000","last_fullsized":"2024-04-25T10:04:33.411081+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:45.369177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357441+0000","last_change":"2024-04-25T10:04:30.333730+0000","last_active":"2024-04-25T10:04:32.357441+0000","last_peered":"2024-04-25T10:04:32.357441+0000","last_clean":"2024-04-25T10:04:32.357441+0000","last_became_active":"2024-04-25T10:04:30.333378+0000","last_became_peered":"2024-04-25T10:04:30.333378+0000","last_unstale":"2024-04-25T10:04:32.357441+0000","last_undegraded":"2024-04-25T10:04:32.357441+0000","last_fullsized":"2024-04-25T10:04:32.357441+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:58.863328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410915+0000","last_change":"2024-04-25T10:04:31.344033+0000","last_active":"2024-04-25T10:04:33.410915+0000","last_peered":"2024-04-25T10:04:33.410915+0000","last_clean":"2024-04-25T10:04:33.410915+0000","last_became_active":"2024-04-25T10:04:31.343747+0000","last_became_peered":"2024-04-25T10:04:31.343747+0000","last_unstale":"2024-04-25T10:04:33.410915+0000","last_undegraded":"2024-04-25T10:04:33.410915+0000","last_fullsized":"2024-04-25T10:04:33.410915+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:55.547730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331204+0000","last_change":"2024-04-25T10:04:30.324053+0000","last_active":"2024-04-25T10:04:32.331204+0000","last_peered":"2024-04-25T10:04:32.331204+0000","last_clean":"2024-04-25T10:04:32.331204+0000","last_became_active":"2024-04-25T10:04:30.323799+0000","last_became_peered":"2024-04-25T10:04:30.323799+0000","last_unstale":"2024-04-25T10:04:32.331204+0000","last_undegraded":"2024-04-25T10:04:32.331204+0000","last_fullsized":"2024-04-25T10:04:32.331204+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.543434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410961+0000","last_change":"2024-04-25T10:04:31.354111+0000","last_active":"2024-04-25T10:04:33.410961+0000","last_peered":"2024-04-25T10:04:33.410961+0000","last_clean":"2024-04-25T10:04:33.410961+0000","last_became_active":"2024-04-25T10:04:31.353888+0000","last_became_peered":"2024-04-25T10:04:31.353888+0000","last_unstale":"2024-04-25T10:04:33.410961+0000","last_undegraded":"2024-04-25T10:04:33.410961+0000","last_fullsized":"2024-04-25T10:04:33.410961+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:18.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357110+0000","last_change":"2024-04-25T10:04:30.335564+0000","last_active":"2024-04-25T10:04:32.357110+0000","last_peered":"2024-04-25T10:04:32.357110+0000","last_clean":"2024-04-25T10:04:32.357110+0000","last_became_active":"2024-04-25T10:04:30.335292+0000","last_became_peered":"2024-04-25T10:04:30.335292+0000","last_unstale":"2024-04-25T10:04:32.357110+0000","last_undegraded":"2024-04-25T10:04:32.357110+0000","last_fullsized":"2024-04-25T10:04:32.357110+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:51.614469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355558+0000","last_change":"2024-04-25T10:04:31.351313+0000","last_active":"2024-04-25T10:04:32.355558+0000","last_peered":"2024-04-25T10:04:32.355558+0000","last_clean":"2024-04-25T10:04:32.355558+0000","last_became_active":"2024-04-25T10:04:31.350825+0000","last_became_peered":"2024-04-25T10:04:31.350825+0000","last_unstale":"2024-04-25T10:04:32.355558+0000","last_undegraded":"2024-04-25T10:04:32.355558+0000","last_fullsized":"2024-04-25T10:04:32.355558+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:36.485307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331435+0000","last_change":"2024-04-25T10:04:30.328432+0000","last_active":"2024-04-25T10:04:32.331435+0000","last_peered":"2024-04-25T10:04:32.331435+0000","last_clean":"2024-04-25T10:04:32.331435+0000","last_became_active":"2024-04-25T10:04:30.327752+0000","last_became_peered":"2024-04-25T10:04:30.327752+0000","last_unstale":"2024-04-25T10:04:32.331435+0000","last_undegraded":"2024-04-25T10:04:32.331435+0000","last_fullsized":"2024-04-25T10:04:32.331435+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331155+0000","last_change":"2024-04-25T10:04:31.347847+0000","last_active":"2024-04-25T10:04:32.331155+0000","last_peered":"2024-04-25T10:04:32.331155+0000","last_clean":"2024-04-25T10:04:32.331155+0000","last_became_active":"2024-04-25T10:04:31.345601+0000","last_became_peered":"2024-04-25T10:04:31.345601+0000","last_unstale":"2024-04-25T10:04:32.331155+0000","last_undegraded":"2024-04-25T10:04:32.331155+0000","last_fullsized":"2024-04-25T10:04:32.331155+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:37.276457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410810+0000","last_change":"2024-04-25T10:04:30.336366+0000","last_active":"2024-04-25T10:04:33.410810+0000","last_peered":"2024-04-25T10:04:33.410810+0000","last_clean":"2024-04-25T10:04:33.410810+0000","last_became_active":"2024-04-25T10:04:30.336105+0000","last_became_peered":"2024-04-25T10:04:30.336105+0000","last_unstale":"2024-04-25T10:04:33.410810+0000","last_undegraded":"2024-04-25T10:04:33.410810+0000","last_fullsized":"2024-04-25T10:04:33.410810+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.870627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331210+0000","last_change":"2024-04-25T10:04:31.346249+0000","last_active":"2024-04-25T10:04:32.331210+0000","last_peered":"2024-04-25T10:04:32.331210+0000","last_clean":"2024-04-25T10:04:32.331210+0000","last_became_active":"2024-04-25T10:04:31.345995+0000","last_became_peered":"2024-04-25T10:04:31.345995+0000","last_unstale":"2024-04-25T10:04:32.331210+0000","last_undegraded":"2024-04-25T10:04:32.331210+0000","last_fullsized":"2024-04-25T10:04:32.331210+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.090553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412584+0000","last_change":"2024-04-25T10:04:30.335680+0000","last_active":"2024-04-25T10:04:32.412584+0000","last_peered":"2024-04-25T10:04:32.412584+0000","last_clean":"2024-04-25T10:04:32.412584+0000","last_became_active":"2024-04-25T10:04:30.335429+0000","last_became_peered":"2024-04-25T10:04:30.335429+0000","last_unstale":"2024-04-25T10:04:32.412584+0000","last_undegraded":"2024-04-25T10:04:32.412584+0000","last_fullsized":"2024-04-25T10:04:32.412584+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:33.032462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410998+0000","last_change":"2024-04-25T10:04:31.342014+0000","last_active":"2024-04-25T10:04:33.410998+0000","last_peered":"2024-04-25T10:04:33.410998+0000","last_clean":"2024-04-25T10:04:33.410998+0000","last_became_active":"2024-04-25T10:04:31.341742+0000","last_became_peered":"2024-04-25T10:04:31.341742+0000","last_unstale":"2024-04-25T10:04:33.410998+0000","last_undegraded":"2024-04-25T10:04:33.410998+0000","last_fullsized":"2024-04-25T10:04:33.410998+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411552+0000","last_change":"2024-04-25T10:04:30.327551+0000","last_active":"2024-04-25T10:04:33.411552+0000","last_peered":"2024-04-25T10:04:33.411552+0000","last_clean":"2024-04-25T10:04:33.411552+0000","last_became_active":"2024-04-25T10:04:30.327245+0000","last_became_peered":"2024-04-25T10:04:30.327245+0000","last_unstale":"2024-04-25T10:04:33.411552+0000","last_undegraded":"2024-04-25T10:04:33.411552+0000","last_fullsized":"2024-04-25T10:04:33.411552+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:13.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410462+0000","last_change":"2024-04-25T10:04:31.343068+0000","last_active":"2024-04-25T10:04:33.410462+0000","last_peered":"2024-04-25T10:04:33.410462+0000","last_clean":"2024-04-25T10:04:33.410462+0000","last_became_active":"2024-04-25T10:04:31.342804+0000","last_became_peered":"2024-04-25T10:04:31.342804+0000","last_unstale":"2024-04-25T10:04:33.410462+0000","last_undegraded":"2024-04-25T10:04:33.410462+0000","last_fullsized":"2024-04-25T10:04:33.410462+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:13.580436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331100+0000","last_change":"2024-04-25T10:04:30.329132+0000","last_active":"2024-04-25T10:04:32.331100+0000","last_peered":"2024-04-25T10:04:32.331100+0000","last_clean":"2024-04-25T10:04:32.331100+0000","last_became_active":"2024-04-25T10:04:30.328879+0000","last_became_peered":"2024-04-25T10:04:30.328879+0000","last_unstale":"2024-04-25T10:04:32.331100+0000","last_undegraded":"2024-04-25T10:04:32.331100+0000","last_fullsized":"2024-04-25T10:04:32.331100+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:17.500610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357441+0000","last_change":"2024-04-25T10:04:31.342101+0000","last_active":"2024-04-25T10:04:32.357441+0000","last_peered":"2024-04-25T10:04:32.357441+0000","last_clean":"2024-04-25T10:04:32.357441+0000","last_became_active":"2024-04-25T10:04:31.341847+0000","last_became_peered":"2024-04-25T10:04:31.341847+0000","last_unstale":"2024-04-25T10:04:32.357441+0000","last_undegraded":"2024-04-25T10:04:32.357441+0000","last_fullsized":"2024-04-25T10:04:32.357441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:27.502026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"159'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355551+0000","last_change":"2024-04-25T10:04:30.334642+0000","last_active":"2024-04-25T10:04:32.355551+0000","last_peered":"2024-04-25T10:04:32.355551+0000","last_clean":"2024-04-25T10:04:32.355551+0000","last_became_active":"2024-04-25T10:04:30.334411+0000","last_became_peered":"2024-04-25T10:04:30.334411+0000","last_unstale":"2024-04-25T10:04:32.355551+0000","last_undegraded":"2024-04-25T10:04:32.355551+0000","last_fullsized":"2024-04-25T10:04:32.355551+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412923+0000","last_change":"2024-04-25T10:04:31.340066+0000","last_active":"2024-04-25T10:04:32.412923+0000","last_peered":"2024-04-25T10:04:32.412923+0000","last_clean":"2024-04-25T10:04:32.412923+0000","last_became_active":"2024-04-25T10:04:31.339657+0000","last_became_peered":"2024-04-25T10:04:31.339657+0000","last_unstale":"2024-04-25T10:04:32.412923+0000","last_undegraded":"2024-04-25T10:04:32.412923+0000","last_fullsized":"2024-04-25T10:04:32.412923+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:23.249081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355337+0000","last_change":"2024-04-25T10:04:30.332838+0000","last_active":"2024-04-25T10:04:32.355337+0000","last_peered":"2024-04-25T10:04:32.355337+0000","last_clean":"2024-04-25T10:04:32.355337+0000","last_became_active":"2024-04-25T10:04:30.332568+0000","last_became_peered":"2024-04-25T10:04:30.332568+0000","last_unstale":"2024-04-25T10:04:32.355337+0000","last_undegraded":"2024-04-25T10:04:32.355337+0000","last_fullsized":"2024-04-25T10:04:32.355337+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:08.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357240+0000","last_change":"2024-04-25T10:04:31.342113+0000","last_active":"2024-04-25T10:04:32.357240+0000","last_peered":"2024-04-25T10:04:32.357240+0000","last_clean":"2024-04-25T10:04:32.357240+0000","last_became_active":"2024-04-25T10:04:31.339474+0000","last_became_peered":"2024-04-25T10:04:31.339474+0000","last_unstale":"2024-04-25T10:04:32.357240+0000","last_undegraded":"2024-04-25T10:04:32.357240+0000","last_fullsized":"2024-04-25T10:04:32.357240+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:55.547730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412715+0000","last_change":"2024-04-25T10:04:30.336384+0000","last_active":"2024-04-25T10:04:32.412715+0000","last_peered":"2024-04-25T10:04:32.412715+0000","last_clean":"2024-04-25T10:04:32.412715+0000","last_became_active":"2024-04-25T10:04:30.336066+0000","last_became_peered":"2024-04-25T10:04:30.336066+0000","last_unstale":"2024-04-25T10:04:32.412715+0000","last_undegraded":"2024-04-25T10:04:32.412715+0000","last_fullsized":"2024-04-25T10:04:32.412715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:05.056296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334160+0000","last_change":"2024-04-25T10:04:31.344710+0000","last_active":"2024-04-25T10:04:32.334160+0000","last_peered":"2024-04-25T10:04:32.334160+0000","last_clean":"2024-04-25T10:04:32.334160+0000","last_became_active":"2024-04-25T10:04:31.343961+0000","last_became_peered":"2024-04-25T10:04:31.343961+0000","last_unstale":"2024-04-25T10:04:32.334160+0000","last_undegraded":"2024-04-25T10:04:32.334160+0000","last_fullsized":"2024-04-25T10:04:32.334160+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:16.066745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333715+0000","last_change":"2024-04-25T10:04:30.323701+0000","last_active":"2024-04-25T10:04:32.333715+0000","last_peered":"2024-04-25T10:04:32.333715+0000","last_clean":"2024-04-25T10:04:32.333715+0000","last_became_active":"2024-04-25T10:04:30.323240+0000","last_became_peered":"2024-04-25T10:04:30.323240+0000","last_unstale":"2024-04-25T10:04:32.333715+0000","last_undegraded":"2024-04-25T10:04:32.333715+0000","last_fullsized":"2024-04-25T10:04:32.333715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:33.867172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410786+0000","last_change":"2024-04-25T10:04:31.340949+0000","last_active":"2024-04-25T10:04:33.410786+0000","last_peered":"2024-04-25T10:04:33.410786+0000","last_clean":"2024-04-25T10:04:33.410786+0000","last_became_active":"2024-04-25T10:04:31.340582+0000","last_became_peered":"2024-04-25T10:04:31.340582+0000","last_unstale":"2024-04-25T10:04:33.410786+0000","last_undegraded":"2024-04-25T10:04:33.410786+0000","last_fullsized":"2024-04-25T10:04:33.410786+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:13.580436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355446+0000","last_change":"2024-04-25T10:04:30.326550+0000","last_active":"2024-04-25T10:04:32.355446+0000","last_peered":"2024-04-25T10:04:32.355446+0000","last_clean":"2024-04-25T10:04:32.355446+0000","last_became_active":"2024-04-25T10:04:30.325966+0000","last_became_peered":"2024-04-25T10:04:30.325966+0000","last_unstale":"2024-04-25T10:04:32.355446+0000","last_undegraded":"2024-04-25T10:04:32.355446+0000","last_fullsized":"2024-04-25T10:04:32.355446+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:25.494064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355454+0000","last_change":"2024-04-25T10:04:31.349087+0000","last_active":"2024-04-25T10:04:32.355454+0000","last_peered":"2024-04-25T10:04:32.355454+0000","last_clean":"2024-04-25T10:04:32.355454+0000","last_became_active":"2024-04-25T10:04:31.348812+0000","last_became_peered":"2024-04-25T10:04:31.348812+0000","last_unstale":"2024-04-25T10:04:32.355454+0000","last_undegraded":"2024-04-25T10:04:32.355454+0000","last_fullsized":"2024-04-25T10:04:32.355454+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:14.542078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410991+0000","last_change":"2024-04-25T10:04:30.334944+0000","last_active":"2024-04-25T10:04:33.410991+0000","last_peered":"2024-04-25T10:04:33.410991+0000","last_clean":"2024-04-25T10:04:33.410991+0000","last_became_active":"2024-04-25T10:04:30.334680+0000","last_became_peered":"2024-04-25T10:04:30.334680+0000","last_unstale":"2024-04-25T10:04:33.410991+0000","last_undegraded":"2024-04-25T10:04:33.410991+0000","last_fullsized":"2024-04-25T10:04:33.410991+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:58.863328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412954+0000","last_change":"2024-04-25T10:04:31.347735+0000","last_active":"2024-04-25T10:04:32.412954+0000","last_peered":"2024-04-25T10:04:32.412954+0000","last_clean":"2024-04-25T10:04:32.412954+0000","last_became_active":"2024-04-25T10:04:31.347443+0000","last_became_peered":"2024-04-25T10:04:31.347443+0000","last_unstale":"2024-04-25T10:04:32.412954+0000","last_undegraded":"2024-04-25T10:04:32.412954+0000","last_fullsized":"2024-04-25T10:04:32.412954+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.330773+0000","last_change":"2024-04-25T10:04:30.328739+0000","last_active":"2024-04-25T10:04:32.330773+0000","last_peered":"2024-04-25T10:04:32.330773+0000","last_clean":"2024-04-25T10:04:32.330773+0000","last_became_active":"2024-04-25T10:04:30.328488+0000","last_became_peered":"2024-04-25T10:04:30.328488+0000","last_unstale":"2024-04-25T10:04:32.330773+0000","last_undegraded":"2024-04-25T10:04:32.330773+0000","last_fullsized":"2024-04-25T10:04:32.330773+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:08.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411405+0000","last_change":"2024-04-25T10:04:31.336228+0000","last_active":"2024-04-25T10:04:33.411405+0000","last_peered":"2024-04-25T10:04:33.411405+0000","last_clean":"2024-04-25T10:04:33.411405+0000","last_became_active":"2024-04-25T10:04:31.335914+0000","last_became_peered":"2024-04-25T10:04:31.335914+0000","last_unstale":"2024-04-25T10:04:33.411405+0000","last_undegraded":"2024-04-25T10:04:33.411405+0000","last_fullsized":"2024-04-25T10:04:33.411405+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355405+0000","last_change":"2024-04-25T10:04:30.334713+0000","last_active":"2024-04-25T10:04:32.355405+0000","last_peered":"2024-04-25T10:04:32.355405+0000","last_clean":"2024-04-25T10:04:32.355405+0000","last_became_active":"2024-04-25T10:04:30.334490+0000","last_became_peered":"2024-04-25T10:04:30.334490+0000","last_unstale":"2024-04-25T10:04:32.355405+0000","last_undegraded":"2024-04-25T10:04:32.355405+0000","last_fullsized":"2024-04-25T10:04:32.355405+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:26.497730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":426,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411559+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:04:33.411559+0000","last_peered":"2024-04-25T10:04:33.411559+0000","last_clean":"2024-04-25T10:04:33.411559+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:04:33.411559+0000","last_undegraded":"2024-04-25T10:04:33.411559+0000","last_fullsized":"2024-04-25T10:04:33.411559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2592768,"data_stored":1526292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51184,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320656,"statfs":{"total":96636764160,"available":96584351744,"internally_reserved":0,"allocated":1998848,"data_stored":935924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51188,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320652,"statfs":{"total":96636764160,"available":96584347648,"internally_reserved":0,"allocated":2007040,"data_stored":936048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59524,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312316,"statfs":{"total":96636764160,"available":96575811584,"internally_reserved":0,"allocated":2015232,"data_stored":936688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59528,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312312,"statfs":{"total":96636764160,"available":96575807488,"internally_reserved":0,"allocated":2019328,"data_stored":938140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55340,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316500,"statfs":{"total":96636764160,"available":96580096000,"internally_reserved":0,"allocated":1998848,"data_stored":935924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2011136,"data_stored":937536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55936,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315904,"statfs":{"total":96636764160,"available":96579485696,"internally_reserved":0,"allocated":2600960,"data_stored":1526396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:36.603 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-04-25T10:04:36.598+0000 7f2517f15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:38.670 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.773+0000 7f1a2e691640 1 Processor -- start 2024-04-25T10:04:38.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.773+0000 7f1a2e691640 1 -- start start 2024-04-25T10:04:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a2e691640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 0x7f1a280d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a2e691640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2815dfe0 con 0x7f1a2815dc00 2024-04-25T10:04:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a2e691640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a280d00c0 con 0x7f1a280d1c00 2024-04-25T10:04:38.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a2e691640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280d2060 con 0x7f1a280c6df0 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a27fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1a2815dc00 0x7f1a280ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36912/0 (socket says 172.21.15.148:36912) 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a27fff640 1 -- 172.21.15.148:0/109487444 learned_addr learned my addr 172.21.15.148:0/109487444 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a277fe640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 0x7f1a280d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017035014 0 0) 0x7f1a18001110 con 0x7f1a2815dc00 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.774+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a10003520 con 0x7f1a2815dc00 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791934903 0 0) 0x7f1a280d00c0 con 0x7f1a280d1c00 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a18001110 con 0x7f1a280d1c00 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2839151193 0 0) 0x7f1a10003520 con 0x7f1a2815dc00 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 msgr2=0x7f1a280d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 0x7f1a280d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 >> v1:172.21.15.188:6789/0 conn(0x7f1a280d1c00 legacy=0x7f1a280ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a280d03a0 con 0x7f1a2815dc00 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f1a18001840 con 0x7f1a2815dc00 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 >> v1:172.21.15.148:6789/0 conn(0x7f1a2815dc00 legacy=0x7f1a280ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a10003b70 0x7f1a10003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 0x7f1a10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10004d40 0x7f1a100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280d2060 con 0x7f1a10003b70 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1000ba70 con 0x7f1a10004d40 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1000bc70 con 0x7f1a280c6df0 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a27fff640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a10003b70 0x7f1a10003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a277fe640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 0x7f1a10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a2cc2e640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10004d40 0x7f1a100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a27fff640 1 -- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 msgr2=0x7f1a10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a27fff640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280c6df0 0x7f1a10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a27fff640 1 -- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10004d40 msgr2=0x7f1a100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:38.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a27fff640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10004d40 0x7f1a100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.775+0000 7f1a27fff640 1 -- 172.21.15.148:0/109487444 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10003520 con 0x7f1a10003b70 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.776+0000 7f1a27fff640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a10003b70 0x7f1a10003f70 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c2a0 tx=0x7f1a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f94ea9f351c8730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.776+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18036070 con 0x7f1a10003b70 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.776+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a18005ad0 con 0x7f1a10003b70 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.776+0000 7f1a26ffd640 1 -- 172.21.15.148:0/109487444 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18007de0 con 0x7f1a10003b70 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2e691640 1 -- 172.21.15.148:0/109487444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a10003b70 msgr2=0x7f1a10003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2e691640 1 --2- 172.21.15.148:0/109487444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a10003b70 0x7f1a10003f70 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c2a0 tx=0x7f1a10005830 comp rx=0 tx=0).stop 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2cc2e640 1 -- 172.21.15.148:0/109487444 reap_dead start 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2e691640 1 -- 172.21.15.148:0/109487444 shutdown_connections 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2e691640 1 -- 172.21.15.148:0/109487444 >> 172.21.15.148:0/109487444 conn(0x7f1a2806f5d0 msgr2=0x7f1a2806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2e691640 1 -- 172.21.15.148:0/109487444 shutdown_connections 2024-04-25T10:04:38.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.777+0000 7f1a2e691640 1 -- 172.21.15.148:0/109487444 wait complete. 2024-04-25T10:04:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.778+0000 7f1a2e691640 1 Processor -- start 2024-04-25T10:04:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.778+0000 7f1a2e691640 1 -- start start 2024-04-25T10:04:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2e691640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10003b70 0x7f1a28159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2e691640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 0x7f1a28156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2e691640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a28161620 0x7f1a28161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2e691640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cc130 con 0x7f1a281573e0 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2e691640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cc720 con 0x7f1a10003b70 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2e691640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280cfd90 con 0x7f1a28161620 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 0x7f1a28156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 0x7f1a28156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41372/0 (socket says 172.21.15.148:41372) 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 -- 172.21.15.148:0/3959696643 learned_addr learned my addr 172.21.15.148:0/3959696643 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a27fff640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10003b70 0x7f1a28159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a2cc2e640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a28161620 0x7f1a28161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 -- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a28161620 msgr2=0x7f1a28161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a28161620 0x7f1a28161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 -- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10003b70 msgr2=0x7f1a28159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10003b70 0x7f1a28159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 -- 172.21.15.148:0/3959696643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28159f50 con 0x7f1a281573e0 2024-04-25T10:04:38.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a27fff640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10003b70 0x7f1a28159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.779+0000 7f1a277fe640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 0x7f1a28156b70 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0119d0 tx=0x7f1a1c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.780+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00fb50 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.780+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c00fcf0 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.780+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2815a1d0 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.780+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00fea0 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.780+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2815a790 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.781+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a28073990 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.781+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a1c012040 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.782+0000 7f1a24ff9640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a1406b500 0x7f1a1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.782+0000 7f1a27fff640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a1406b500 0x7f1a1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.782+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a1c08a9d0 con 0x7f1a281573e0 2024-04-25T10:04:38.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.782+0000 7f1a27fff640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a1406b500 0x7f1a1406d9c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1a18033000 tx=0x7f1a1802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:38.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.786+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a28159f50 con 0x7f1a281573e0 2024-04-25T10:04:38.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.956+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a280c66c0 con 0x7f1a1406b500 2024-04-25T10:04:38.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.961+0000 7f1a24ff9640 1 -- 172.21.15.148:0/3959696643 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f1a280c66c0 con 0x7f1a1406b500 2024-04-25T10:04:38.970 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:38.971 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a1406b500 msgr2=0x7f1a1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a1406b500 0x7f1a1406d9c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1a18033000 tx=0x7f1a1802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 msgr2=0x7f1a28156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 0x7f1a28156b70 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0119d0 tx=0x7f1a1c011e90 comp rx=0 tx=0).stop 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 shutdown_connections 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a1406b500 0x7f1a1406d9c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a28161620 0x7f1a28161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a281573e0 0x7f1a28156b70 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 --2- 172.21.15.148:0/3959696643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a10003b70 0x7f1a28159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 >> 172.21.15.148:0/3959696643 conn(0x7f1a2806f5d0 msgr2=0x7f1a280b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:38.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 shutdown_connections 2024-04-25T10:04:38.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:38.965+0000 7f1a2e691640 1 -- 172.21.15.148:0/3959696643 wait complete. 2024-04-25T10:04:38.982 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-25T10:04:37.146607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439820,"kb_used_data":16840,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754534900,"statfs":{"total":773094113280,"available":772643737600,"internally_reserved":0,"allocated":17244160,"data_stored":8672948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.831124"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334159+0000","last_change":"2024-04-25T10:04:31.336044+0000","last_active":"2024-04-25T10:04:32.334159+0000","last_peered":"2024-04-25T10:04:32.334159+0000","last_clean":"2024-04-25T10:04:32.334159+0000","last_became_active":"2024-04-25T10:04:31.335749+0000","last_became_peered":"2024-04-25T10:04:31.335749+0000","last_unstale":"2024-04-25T10:04:32.334159+0000","last_undegraded":"2024-04-25T10:04:32.334159+0000","last_fullsized":"2024-04-25T10:04:32.334159+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.543072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411338+0000","last_change":"2024-04-25T10:04:31.348819+0000","last_active":"2024-04-25T10:04:33.411338+0000","last_peered":"2024-04-25T10:04:33.411338+0000","last_clean":"2024-04-25T10:04:33.411338+0000","last_became_active":"2024-04-25T10:04:31.348571+0000","last_became_peered":"2024-04-25T10:04:31.348571+0000","last_unstale":"2024-04-25T10:04:33.411338+0000","last_undegraded":"2024-04-25T10:04:33.411338+0000","last_fullsized":"2024-04-25T10:04:33.411338+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:09.930071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334005+0000","last_change":"2024-04-25T10:04:31.345946+0000","last_active":"2024-04-25T10:04:32.334005+0000","last_peered":"2024-04-25T10:04:32.334005+0000","last_clean":"2024-04-25T10:04:32.334005+0000","last_became_active":"2024-04-25T10:04:31.345698+0000","last_became_peered":"2024-04-25T10:04:31.345698+0000","last_unstale":"2024-04-25T10:04:32.334005+0000","last_undegraded":"2024-04-25T10:04:32.334005+0000","last_fullsized":"2024-04-25T10:04:32.334005+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:48.472529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334155+0000","last_change":"2024-04-25T10:04:31.344886+0000","last_active":"2024-04-25T10:04:32.334155+0000","last_peered":"2024-04-25T10:04:32.334155+0000","last_clean":"2024-04-25T10:04:32.334155+0000","last_became_active":"2024-04-25T10:04:31.344576+0000","last_became_peered":"2024-04-25T10:04:31.344576+0000","last_unstale":"2024-04-25T10:04:32.334155+0000","last_undegraded":"2024-04-25T10:04:32.334155+0000","last_fullsized":"2024-04-25T10:04:32.334155+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:44.660769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331390+0000","last_change":"2024-04-25T10:04:31.352554+0000","last_active":"2024-04-25T10:04:32.331390+0000","last_peered":"2024-04-25T10:04:32.331390+0000","last_clean":"2024-04-25T10:04:32.331390+0000","last_became_active":"2024-04-25T10:04:31.352243+0000","last_became_peered":"2024-04-25T10:04:31.352243+0000","last_unstale":"2024-04-25T10:04:32.331390+0000","last_undegraded":"2024-04-25T10:04:32.331390+0000","last_fullsized":"2024-04-25T10:04:32.331390+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:13.406702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410545+0000","last_change":"2024-04-25T10:04:31.336152+0000","last_active":"2024-04-25T10:04:33.410545+0000","last_peered":"2024-04-25T10:04:33.410545+0000","last_clean":"2024-04-25T10:04:33.410545+0000","last_became_active":"2024-04-25T10:04:31.335695+0000","last_became_peered":"2024-04-25T10:04:31.335695+0000","last_unstale":"2024-04-25T10:04:33.410545+0000","last_undegraded":"2024-04-25T10:04:33.410545+0000","last_fullsized":"2024-04-25T10:04:33.410545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:52.255652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357297+0000","last_change":"2024-04-25T10:04:31.342792+0000","last_active":"2024-04-25T10:04:32.357297+0000","last_peered":"2024-04-25T10:04:32.357297+0000","last_clean":"2024-04-25T10:04:32.357297+0000","last_became_active":"2024-04-25T10:04:31.342524+0000","last_became_peered":"2024-04-25T10:04:31.342524+0000","last_unstale":"2024-04-25T10:04:32.357297+0000","last_undegraded":"2024-04-25T10:04:32.357297+0000","last_fullsized":"2024-04-25T10:04:32.357297+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:09.930071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.356999+0000","last_change":"2024-04-25T10:04:31.336921+0000","last_active":"2024-04-25T10:04:32.356999+0000","last_peered":"2024-04-25T10:04:32.356999+0000","last_clean":"2024-04-25T10:04:32.356999+0000","last_became_active":"2024-04-25T10:04:31.336293+0000","last_became_peered":"2024-04-25T10:04:31.336293+0000","last_unstale":"2024-04-25T10:04:32.356999+0000","last_undegraded":"2024-04-25T10:04:32.356999+0000","last_fullsized":"2024-04-25T10:04:32.356999+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:13.580436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357211+0000","last_change":"2024-04-25T10:04:31.337363+0000","last_active":"2024-04-25T10:04:32.357211+0000","last_peered":"2024-04-25T10:04:32.357211+0000","last_clean":"2024-04-25T10:04:32.357211+0000","last_became_active":"2024-04-25T10:04:31.337084+0000","last_became_peered":"2024-04-25T10:04:31.337084+0000","last_unstale":"2024-04-25T10:04:32.357211+0000","last_undegraded":"2024-04-25T10:04:32.357211+0000","last_fullsized":"2024-04-25T10:04:32.357211+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357321+0000","last_change":"2024-04-25T10:04:31.346554+0000","last_active":"2024-04-25T10:04:32.357321+0000","last_peered":"2024-04-25T10:04:32.357321+0000","last_clean":"2024-04-25T10:04:32.357321+0000","last_became_active":"2024-04-25T10:04:31.346281+0000","last_became_peered":"2024-04-25T10:04:31.346281+0000","last_unstale":"2024-04-25T10:04:32.357321+0000","last_undegraded":"2024-04-25T10:04:32.357321+0000","last_fullsized":"2024-04-25T10:04:32.357321+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410902+0000","last_change":"2024-04-25T10:04:31.347754+0000","last_active":"2024-04-25T10:04:33.410902+0000","last_peered":"2024-04-25T10:04:33.410902+0000","last_clean":"2024-04-25T10:04:33.410902+0000","last_became_active":"2024-04-25T10:04:31.347433+0000","last_became_peered":"2024-04-25T10:04:31.347433+0000","last_unstale":"2024-04-25T10:04:33.410902+0000","last_undegraded":"2024-04-25T10:04:33.410902+0000","last_fullsized":"2024-04-25T10:04:33.410902+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410867+0000","last_change":"2024-04-25T10:04:31.337133+0000","last_active":"2024-04-25T10:04:33.410867+0000","last_peered":"2024-04-25T10:04:33.410867+0000","last_clean":"2024-04-25T10:04:33.410867+0000","last_became_active":"2024-04-25T10:04:31.336195+0000","last_became_peered":"2024-04-25T10:04:31.336195+0000","last_unstale":"2024-04-25T10:04:33.410867+0000","last_undegraded":"2024-04-25T10:04:33.410867+0000","last_fullsized":"2024-04-25T10:04:33.410867+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.448151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355335+0000","last_change":"2024-04-25T10:04:31.344786+0000","last_active":"2024-04-25T10:04:32.355335+0000","last_peered":"2024-04-25T10:04:32.355335+0000","last_clean":"2024-04-25T10:04:32.355335+0000","last_became_active":"2024-04-25T10:04:31.344287+0000","last_became_peered":"2024-04-25T10:04:31.344287+0000","last_unstale":"2024-04-25T10:04:32.355335+0000","last_undegraded":"2024-04-25T10:04:32.355335+0000","last_fullsized":"2024-04-25T10:04:32.355335+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:42.362812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331010+0000","last_change":"2024-04-25T10:04:31.343226+0000","last_active":"2024-04-25T10:04:32.331010+0000","last_peered":"2024-04-25T10:04:32.331010+0000","last_clean":"2024-04-25T10:04:32.331010+0000","last_became_active":"2024-04-25T10:04:31.342925+0000","last_became_peered":"2024-04-25T10:04:31.342925+0000","last_unstale":"2024-04-25T10:04:32.331010+0000","last_undegraded":"2024-04-25T10:04:32.331010+0000","last_fullsized":"2024-04-25T10:04:32.331010+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:17.535351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410432+0000","last_change":"2024-04-25T10:04:31.347121+0000","last_active":"2024-04-25T10:04:33.410432+0000","last_peered":"2024-04-25T10:04:33.410432+0000","last_clean":"2024-04-25T10:04:33.410432+0000","last_became_active":"2024-04-25T10:04:31.346846+0000","last_became_peered":"2024-04-25T10:04:31.346846+0000","last_unstale":"2024-04-25T10:04:33.410432+0000","last_undegraded":"2024-04-25T10:04:33.410432+0000","last_fullsized":"2024-04-25T10:04:33.410432+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:29.431821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333680+0000","last_change":"2024-04-25T10:04:31.345041+0000","last_active":"2024-04-25T10:04:32.333680+0000","last_peered":"2024-04-25T10:04:32.333680+0000","last_clean":"2024-04-25T10:04:32.333680+0000","last_became_active":"2024-04-25T10:04:31.344756+0000","last_became_peered":"2024-04-25T10:04:31.344756+0000","last_unstale":"2024-04-25T10:04:32.333680+0000","last_undegraded":"2024-04-25T10:04:32.333680+0000","last_fullsized":"2024-04-25T10:04:32.333680+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:54.198699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357421+0000","last_change":"2024-04-25T10:04:31.348783+0000","last_active":"2024-04-25T10:04:32.357421+0000","last_peered":"2024-04-25T10:04:32.357421+0000","last_clean":"2024-04-25T10:04:32.357421+0000","last_became_active":"2024-04-25T10:04:31.348458+0000","last_became_peered":"2024-04-25T10:04:31.348458+0000","last_unstale":"2024-04-25T10:04:32.357421+0000","last_undegraded":"2024-04-25T10:04:32.357421+0000","last_fullsized":"2024-04-25T10:04:32.357421+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:00.332191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411368+0000","last_change":"2024-04-25T10:04:31.340150+0000","last_active":"2024-04-25T10:04:33.411368+0000","last_peered":"2024-04-25T10:04:33.411368+0000","last_clean":"2024-04-25T10:04:33.411368+0000","last_became_active":"2024-04-25T10:04:31.339875+0000","last_became_peered":"2024-04-25T10:04:31.339875+0000","last_unstale":"2024-04-25T10:04:33.411368+0000","last_undegraded":"2024-04-25T10:04:33.411368+0000","last_fullsized":"2024-04-25T10:04:33.411368+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:38.305856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411144+0000","last_change":"2024-04-25T10:04:31.346786+0000","last_active":"2024-04-25T10:04:33.411144+0000","last_peered":"2024-04-25T10:04:33.411144+0000","last_clean":"2024-04-25T10:04:33.411144+0000","last_became_active":"2024-04-25T10:04:31.346469+0000","last_became_peered":"2024-04-25T10:04:31.346469+0000","last_unstale":"2024-04-25T10:04:33.411144+0000","last_undegraded":"2024-04-25T10:04:33.411144+0000","last_fullsized":"2024-04-25T10:04:33.411144+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:09.930071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412764+0000","last_change":"2024-04-25T10:04:31.348231+0000","last_active":"2024-04-25T10:04:32.412764+0000","last_peered":"2024-04-25T10:04:32.412764+0000","last_clean":"2024-04-25T10:04:32.412764+0000","last_became_active":"2024-04-25T10:04:31.347954+0000","last_became_peered":"2024-04-25T10:04:31.347954+0000","last_unstale":"2024-04-25T10:04:32.412764+0000","last_undegraded":"2024-04-25T10:04:32.412764+0000","last_fullsized":"2024-04-25T10:04:32.412764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:17.645241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355542+0000","last_change":"2024-04-25T10:04:31.350292+0000","last_active":"2024-04-25T10:04:32.355542+0000","last_peered":"2024-04-25T10:04:32.355542+0000","last_clean":"2024-04-25T10:04:32.355542+0000","last_became_active":"2024-04-25T10:04:31.350018+0000","last_became_peered":"2024-04-25T10:04:31.350018+0000","last_unstale":"2024-04-25T10:04:32.355542+0000","last_undegraded":"2024-04-25T10:04:32.355542+0000","last_fullsized":"2024-04-25T10:04:32.355542+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:21.746394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410743+0000","last_change":"2024-04-25T10:04:31.346930+0000","last_active":"2024-04-25T10:04:33.410743+0000","last_peered":"2024-04-25T10:04:33.410743+0000","last_clean":"2024-04-25T10:04:33.410743+0000","last_became_active":"2024-04-25T10:04:31.346607+0000","last_became_peered":"2024-04-25T10:04:31.346607+0000","last_unstale":"2024-04-25T10:04:33.410743+0000","last_undegraded":"2024-04-25T10:04:33.410743+0000","last_fullsized":"2024-04-25T10:04:33.410743+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:29.431821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.330905+0000","last_change":"2024-04-25T10:04:31.345657+0000","last_active":"2024-04-25T10:04:32.330905+0000","last_peered":"2024-04-25T10:04:32.330905+0000","last_clean":"2024-04-25T10:04:32.330905+0000","last_became_active":"2024-04-25T10:04:31.345314+0000","last_became_peered":"2024-04-25T10:04:31.345314+0000","last_unstale":"2024-04-25T10:04:32.330905+0000","last_undegraded":"2024-04-25T10:04:32.330905+0000","last_fullsized":"2024-04-25T10:04:32.330905+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:08.132401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333894+0000","last_change":"2024-04-25T10:04:31.345395+0000","last_active":"2024-04-25T10:04:32.333894+0000","last_peered":"2024-04-25T10:04:32.333894+0000","last_clean":"2024-04-25T10:04:32.333894+0000","last_became_active":"2024-04-25T10:04:31.345077+0000","last_became_peered":"2024-04-25T10:04:31.345077+0000","last_unstale":"2024-04-25T10:04:32.333894+0000","last_undegraded":"2024-04-25T10:04:32.333894+0000","last_fullsized":"2024-04-25T10:04:32.333894+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:33.549821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355643+0000","last_change":"2024-04-25T10:04:31.347607+0000","last_active":"2024-04-25T10:04:32.355643+0000","last_peered":"2024-04-25T10:04:32.355643+0000","last_clean":"2024-04-25T10:04:32.355643+0000","last_became_active":"2024-04-25T10:04:31.347335+0000","last_became_peered":"2024-04-25T10:04:31.347335+0000","last_unstale":"2024-04-25T10:04:32.355643+0000","last_undegraded":"2024-04-25T10:04:32.355643+0000","last_fullsized":"2024-04-25T10:04:32.355643+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:53.054606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412863+0000","last_change":"2024-04-25T10:04:31.345513+0000","last_active":"2024-04-25T10:04:32.412863+0000","last_peered":"2024-04-25T10:04:32.412863+0000","last_clean":"2024-04-25T10:04:32.412863+0000","last_became_active":"2024-04-25T10:04:31.345173+0000","last_became_peered":"2024-04-25T10:04:31.345173+0000","last_unstale":"2024-04-25T10:04:32.412863+0000","last_undegraded":"2024-04-25T10:04:32.412863+0000","last_fullsized":"2024-04-25T10:04:32.412863+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:40.810112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410676+0000","last_change":"2024-04-25T10:04:31.347391+0000","last_active":"2024-04-25T10:04:33.410676+0000","last_peered":"2024-04-25T10:04:33.410676+0000","last_clean":"2024-04-25T10:04:33.410676+0000","last_became_active":"2024-04-25T10:04:31.347108+0000","last_became_peered":"2024-04-25T10:04:31.347108+0000","last_unstale":"2024-04-25T10:04:33.410676+0000","last_undegraded":"2024-04-25T10:04:33.410676+0000","last_fullsized":"2024-04-25T10:04:33.410676+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:58.889086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355543+0000","last_change":"2024-04-25T10:04:31.352074+0000","last_active":"2024-04-25T10:04:32.355543+0000","last_peered":"2024-04-25T10:04:32.355543+0000","last_clean":"2024-04-25T10:04:32.355543+0000","last_became_active":"2024-04-25T10:04:31.351809+0000","last_became_peered":"2024-04-25T10:04:31.351809+0000","last_unstale":"2024-04-25T10:04:32.355543+0000","last_undegraded":"2024-04-25T10:04:32.355543+0000","last_fullsized":"2024-04-25T10:04:32.355543+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:45.835666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411066+0000","last_change":"2024-04-25T10:04:31.340746+0000","last_active":"2024-04-25T10:04:33.411066+0000","last_peered":"2024-04-25T10:04:33.411066+0000","last_clean":"2024-04-25T10:04:33.411066+0000","last_became_active":"2024-04-25T10:04:31.340457+0000","last_became_peered":"2024-04-25T10:04:31.340457+0000","last_unstale":"2024-04-25T10:04:33.411066+0000","last_undegraded":"2024-04-25T10:04:33.411066+0000","last_fullsized":"2024-04-25T10:04:33.411066+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:55.547730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411112+0000","last_change":"2024-04-25T10:04:31.349990+0000","last_active":"2024-04-25T10:04:33.411112+0000","last_peered":"2024-04-25T10:04:33.411112+0000","last_clean":"2024-04-25T10:04:33.411112+0000","last_became_active":"2024-04-25T10:04:31.349412+0000","last_became_peered":"2024-04-25T10:04:31.349412+0000","last_unstale":"2024-04-25T10:04:33.411112+0000","last_undegraded":"2024-04-25T10:04:33.411112+0000","last_fullsized":"2024-04-25T10:04:33.411112+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:00.269103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410624+0000","last_change":"2024-04-25T10:04:31.344992+0000","last_active":"2024-04-25T10:04:33.410624+0000","last_peered":"2024-04-25T10:04:33.410624+0000","last_clean":"2024-04-25T10:04:33.410624+0000","last_became_active":"2024-04-25T10:04:31.344734+0000","last_became_peered":"2024-04-25T10:04:31.344734+0000","last_unstale":"2024-04-25T10:04:33.410624+0000","last_undegraded":"2024-04-25T10:04:33.410624+0000","last_fullsized":"2024-04-25T10:04:33.410624+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.448151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410432+0000","last_change":"2024-04-25T10:04:31.340905+0000","last_active":"2024-04-25T10:04:33.410432+0000","last_peered":"2024-04-25T10:04:33.410432+0000","last_clean":"2024-04-25T10:04:33.410432+0000","last_became_active":"2024-04-25T10:04:31.340420+0000","last_became_peered":"2024-04-25T10:04:31.340420+0000","last_unstale":"2024-04-25T10:04:33.410432+0000","last_undegraded":"2024-04-25T10:04:33.410432+0000","last_fullsized":"2024-04-25T10:04:33.410432+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:12.279696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411497+0000","last_change":"2024-04-25T10:04:31.338175+0000","last_active":"2024-04-25T10:04:33.411497+0000","last_peered":"2024-04-25T10:04:33.411497+0000","last_clean":"2024-04-25T10:04:33.411497+0000","last_became_active":"2024-04-25T10:04:31.337918+0000","last_became_peered":"2024-04-25T10:04:31.337918+0000","last_unstale":"2024-04-25T10:04:33.411497+0000","last_undegraded":"2024-04-25T10:04:33.411497+0000","last_fullsized":"2024-04-25T10:04:33.411497+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:00.332191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334234+0000","last_change":"2024-04-25T10:04:30.320042+0000","last_active":"2024-04-25T10:04:32.334234+0000","last_peered":"2024-04-25T10:04:32.334234+0000","last_clean":"2024-04-25T10:04:32.334234+0000","last_became_active":"2024-04-25T10:04:30.319750+0000","last_became_peered":"2024-04-25T10:04:30.319750+0000","last_unstale":"2024-04-25T10:04:32.334234+0000","last_undegraded":"2024-04-25T10:04:32.334234+0000","last_fullsized":"2024-04-25T10:04:32.334234+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:39.849636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355231+0000","last_change":"2024-04-25T10:04:31.344644+0000","last_active":"2024-04-25T10:04:32.355231+0000","last_peered":"2024-04-25T10:04:32.355231+0000","last_clean":"2024-04-25T10:04:32.355231+0000","last_became_active":"2024-04-25T10:04:31.344368+0000","last_became_peered":"2024-04-25T10:04:31.344368+0000","last_unstale":"2024-04-25T10:04:32.355231+0000","last_undegraded":"2024-04-25T10:04:32.355231+0000","last_fullsized":"2024-04-25T10:04:32.355231+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:05.629870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331094+0000","last_change":"2024-04-25T10:04:30.334823+0000","last_active":"2024-04-25T10:04:32.331094+0000","last_peered":"2024-04-25T10:04:32.331094+0000","last_clean":"2024-04-25T10:04:32.331094+0000","last_became_active":"2024-04-25T10:04:30.334535+0000","last_became_peered":"2024-04-25T10:04:30.334535+0000","last_unstale":"2024-04-25T10:04:32.331094+0000","last_undegraded":"2024-04-25T10:04:32.331094+0000","last_fullsized":"2024-04-25T10:04:32.331094+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:14.617748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331194+0000","last_change":"2024-04-25T10:04:31.342584+0000","last_active":"2024-04-25T10:04:32.331194+0000","last_peered":"2024-04-25T10:04:32.331194+0000","last_clean":"2024-04-25T10:04:32.331194+0000","last_became_active":"2024-04-25T10:04:31.342308+0000","last_became_peered":"2024-04-25T10:04:31.342308+0000","last_unstale":"2024-04-25T10:04:32.331194+0000","last_undegraded":"2024-04-25T10:04:32.331194+0000","last_fullsized":"2024-04-25T10:04:32.331194+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:57.359501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411186+0000","last_change":"2024-04-25T10:04:30.329348+0000","last_active":"2024-04-25T10:04:33.411186+0000","last_peered":"2024-04-25T10:04:33.411186+0000","last_clean":"2024-04-25T10:04:33.411186+0000","last_became_active":"2024-04-25T10:04:30.328793+0000","last_became_peered":"2024-04-25T10:04:30.328793+0000","last_unstale":"2024-04-25T10:04:33.411186+0000","last_undegraded":"2024-04-25T10:04:33.411186+0000","last_fullsized":"2024-04-25T10:04:33.411186+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:51.614469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411206+0000","last_change":"2024-04-25T10:04:31.345031+0000","last_active":"2024-04-25T10:04:33.411206+0000","last_peered":"2024-04-25T10:04:33.411206+0000","last_clean":"2024-04-25T10:04:33.411206+0000","last_became_active":"2024-04-25T10:04:31.344742+0000","last_became_peered":"2024-04-25T10:04:31.344742+0000","last_unstale":"2024-04-25T10:04:33.411206+0000","last_undegraded":"2024-04-25T10:04:33.411206+0000","last_fullsized":"2024-04-25T10:04:33.411206+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:18.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412601+0000","last_change":"2024-04-25T10:04:30.334748+0000","last_active":"2024-04-25T10:04:32.412601+0000","last_peered":"2024-04-25T10:04:32.412601+0000","last_clean":"2024-04-25T10:04:32.412601+0000","last_became_active":"2024-04-25T10:04:30.334301+0000","last_became_peered":"2024-04-25T10:04:30.334301+0000","last_unstale":"2024-04-25T10:04:32.412601+0000","last_undegraded":"2024-04-25T10:04:32.412601+0000","last_fullsized":"2024-04-25T10:04:32.412601+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:29.492631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357338+0000","last_change":"2024-04-25T10:04:31.338459+0000","last_active":"2024-04-25T10:04:32.357338+0000","last_peered":"2024-04-25T10:04:32.357338+0000","last_clean":"2024-04-25T10:04:32.357338+0000","last_became_active":"2024-04-25T10:04:31.338210+0000","last_became_peered":"2024-04-25T10:04:31.338210+0000","last_unstale":"2024-04-25T10:04:32.357338+0000","last_undegraded":"2024-04-25T10:04:32.357338+0000","last_fullsized":"2024-04-25T10:04:32.357338+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:18.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357299+0000","last_change":"2024-04-25T10:04:30.338677+0000","last_active":"2024-04-25T10:04:32.357299+0000","last_peered":"2024-04-25T10:04:32.357299+0000","last_clean":"2024-04-25T10:04:32.357299+0000","last_became_active":"2024-04-25T10:04:30.338427+0000","last_became_peered":"2024-04-25T10:04:30.338427+0000","last_unstale":"2024-04-25T10:04:32.357299+0000","last_undegraded":"2024-04-25T10:04:32.357299+0000","last_fullsized":"2024-04-25T10:04:32.357299+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:18.535207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412884+0000","last_change":"2024-04-25T10:04:31.337628+0000","last_active":"2024-04-25T10:04:32.412884+0000","last_peered":"2024-04-25T10:04:32.412884+0000","last_clean":"2024-04-25T10:04:32.412884+0000","last_became_active":"2024-04-25T10:04:31.337381+0000","last_became_peered":"2024-04-25T10:04:31.337381+0000","last_unstale":"2024-04-25T10:04:32.412884+0000","last_undegraded":"2024-04-25T10:04:32.412884+0000","last_fullsized":"2024-04-25T10:04:32.412884+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:49.727573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331090+0000","last_change":"2024-04-25T10:04:30.330388+0000","last_active":"2024-04-25T10:04:32.331090+0000","last_peered":"2024-04-25T10:04:32.331090+0000","last_clean":"2024-04-25T10:04:32.331090+0000","last_became_active":"2024-04-25T10:04:30.330137+0000","last_became_peered":"2024-04-25T10:04:30.330137+0000","last_unstale":"2024-04-25T10:04:32.331090+0000","last_undegraded":"2024-04-25T10:04:32.331090+0000","last_fullsized":"2024-04-25T10:04:32.331090+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:38.969075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412929+0000","last_change":"2024-04-25T10:04:31.338813+0000","last_active":"2024-04-25T10:04:32.412929+0000","last_peered":"2024-04-25T10:04:32.412929+0000","last_clean":"2024-04-25T10:04:32.412929+0000","last_became_active":"2024-04-25T10:04:31.338329+0000","last_became_peered":"2024-04-25T10:04:31.338329+0000","last_unstale":"2024-04-25T10:04:32.412929+0000","last_undegraded":"2024-04-25T10:04:32.412929+0000","last_fullsized":"2024-04-25T10:04:32.412929+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:50.757842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412516+0000","last_change":"2024-04-25T10:04:30.322317+0000","last_active":"2024-04-25T10:04:32.412516+0000","last_peered":"2024-04-25T10:04:32.412516+0000","last_clean":"2024-04-25T10:04:32.412516+0000","last_became_active":"2024-04-25T10:04:30.321997+0000","last_became_peered":"2024-04-25T10:04:30.321997+0000","last_unstale":"2024-04-25T10:04:32.412516+0000","last_undegraded":"2024-04-25T10:04:32.412516+0000","last_fullsized":"2024-04-25T10:04:32.412516+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:13.472641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334334+0000","last_change":"2024-04-25T10:04:31.348567+0000","last_active":"2024-04-25T10:04:32.334334+0000","last_peered":"2024-04-25T10:04:32.334334+0000","last_clean":"2024-04-25T10:04:32.334334+0000","last_became_active":"2024-04-25T10:04:31.348232+0000","last_became_peered":"2024-04-25T10:04:31.348232+0000","last_unstale":"2024-04-25T10:04:32.334334+0000","last_undegraded":"2024-04-25T10:04:32.334334+0000","last_fullsized":"2024-04-25T10:04:32.334334+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:37.874922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357178+0000","last_change":"2024-04-25T10:04:30.333941+0000","last_active":"2024-04-25T10:04:32.357178+0000","last_peered":"2024-04-25T10:04:32.357178+0000","last_clean":"2024-04-25T10:04:32.357178+0000","last_became_active":"2024-04-25T10:04:30.333598+0000","last_became_peered":"2024-04-25T10:04:30.333598+0000","last_unstale":"2024-04-25T10:04:32.357178+0000","last_undegraded":"2024-04-25T10:04:32.357178+0000","last_fullsized":"2024-04-25T10:04:32.357178+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.656473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411429+0000","last_change":"2024-04-25T10:04:31.345733+0000","last_active":"2024-04-25T10:04:33.411429+0000","last_peered":"2024-04-25T10:04:33.411429+0000","last_clean":"2024-04-25T10:04:33.411429+0000","last_became_active":"2024-04-25T10:04:31.345461+0000","last_became_peered":"2024-04-25T10:04:31.345461+0000","last_unstale":"2024-04-25T10:04:33.411429+0000","last_undegraded":"2024-04-25T10:04:33.411429+0000","last_fullsized":"2024-04-25T10:04:33.411429+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.498360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355534+0000","last_change":"2024-04-25T10:04:30.331610+0000","last_active":"2024-04-25T10:04:32.355534+0000","last_peered":"2024-04-25T10:04:32.355534+0000","last_clean":"2024-04-25T10:04:32.355534+0000","last_became_active":"2024-04-25T10:04:30.331347+0000","last_became_peered":"2024-04-25T10:04:30.331347+0000","last_unstale":"2024-04-25T10:04:32.355534+0000","last_undegraded":"2024-04-25T10:04:32.355534+0000","last_fullsized":"2024-04-25T10:04:32.355534+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:34.619632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355501+0000","last_change":"2024-04-25T10:04:31.339104+0000","last_active":"2024-04-25T10:04:32.355501+0000","last_peered":"2024-04-25T10:04:32.355501+0000","last_clean":"2024-04-25T10:04:32.355501+0000","last_became_active":"2024-04-25T10:04:31.337317+0000","last_became_peered":"2024-04-25T10:04:31.337317+0000","last_unstale":"2024-04-25T10:04:32.355501+0000","last_undegraded":"2024-04-25T10:04:32.355501+0000","last_fullsized":"2024-04-25T10:04:32.355501+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:24.477923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":17,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333862+0000","last_change":"2024-04-25T10:04:30.328347+0000","last_active":"2024-04-25T10:04:32.333862+0000","last_peered":"2024-04-25T10:04:32.333862+0000","last_clean":"2024-04-25T10:04:32.333862+0000","last_became_active":"2024-04-25T10:04:30.328046+0000","last_became_peered":"2024-04-25T10:04:30.328046+0000","last_unstale":"2024-04-25T10:04:32.333862+0000","last_undegraded":"2024-04-25T10:04:32.333862+0000","last_fullsized":"2024-04-25T10:04:32.333862+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:50.632890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412807+0000","last_change":"2024-04-25T10:04:31.352533+0000","last_active":"2024-04-25T10:04:32.412807+0000","last_peered":"2024-04-25T10:04:32.412807+0000","last_clean":"2024-04-25T10:04:32.412807+0000","last_became_active":"2024-04-25T10:04:31.352218+0000","last_became_peered":"2024-04-25T10:04:31.352218+0000","last_unstale":"2024-04-25T10:04:32.412807+0000","last_undegraded":"2024-04-25T10:04:32.412807+0000","last_fullsized":"2024-04-25T10:04:32.412807+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:50.954932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"158'6","reported_seq":20,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355599+0000","last_change":"2024-04-25T10:04:30.332873+0000","last_active":"2024-04-25T10:04:32.355599+0000","last_peered":"2024-04-25T10:04:32.355599+0000","last_clean":"2024-04-25T10:04:32.355599+0000","last_became_active":"2024-04-25T10:04:30.332601+0000","last_became_peered":"2024-04-25T10:04:30.332601+0000","last_unstale":"2024-04-25T10:04:32.355599+0000","last_undegraded":"2024-04-25T10:04:32.355599+0000","last_fullsized":"2024-04-25T10:04:32.355599+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:29.735149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411280+0000","last_change":"2024-04-25T10:04:31.340618+0000","last_active":"2024-04-25T10:04:33.411280+0000","last_peered":"2024-04-25T10:04:33.411280+0000","last_clean":"2024-04-25T10:04:33.411280+0000","last_became_active":"2024-04-25T10:04:31.339372+0000","last_became_peered":"2024-04-25T10:04:31.339372+0000","last_unstale":"2024-04-25T10:04:33.411280+0000","last_undegraded":"2024-04-25T10:04:33.411280+0000","last_fullsized":"2024-04-25T10:04:33.411280+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:50.386736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331368+0000","last_change":"2024-04-25T10:04:30.331087+0000","last_active":"2024-04-25T10:04:32.331368+0000","last_peered":"2024-04-25T10:04:32.331368+0000","last_clean":"2024-04-25T10:04:32.331368+0000","last_became_active":"2024-04-25T10:04:30.330823+0000","last_became_peered":"2024-04-25T10:04:30.330823+0000","last_unstale":"2024-04-25T10:04:32.331368+0000","last_undegraded":"2024-04-25T10:04:32.331368+0000","last_fullsized":"2024-04-25T10:04:32.331368+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:25.494064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357398+0000","last_change":"2024-04-25T10:04:31.345933+0000","last_active":"2024-04-25T10:04:32.357398+0000","last_peered":"2024-04-25T10:04:32.357398+0000","last_clean":"2024-04-25T10:04:32.357398+0000","last_became_active":"2024-04-25T10:04:31.345656+0000","last_became_peered":"2024-04-25T10:04:31.345656+0000","last_unstale":"2024-04-25T10:04:32.357398+0000","last_undegraded":"2024-04-25T10:04:32.357398+0000","last_fullsized":"2024-04-25T10:04:32.357398+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:26.498360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355253+0000","last_change":"2024-04-25T10:04:30.326763+0000","last_active":"2024-04-25T10:04:32.355253+0000","last_peered":"2024-04-25T10:04:32.355253+0000","last_clean":"2024-04-25T10:04:32.355253+0000","last_became_active":"2024-04-25T10:04:30.326309+0000","last_became_peered":"2024-04-25T10:04:30.326309+0000","last_unstale":"2024-04-25T10:04:32.355253+0000","last_undegraded":"2024-04-25T10:04:32.355253+0000","last_fullsized":"2024-04-25T10:04:32.355253+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:14.617748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411275+0000","last_change":"2024-04-25T10:04:31.347835+0000","last_active":"2024-04-25T10:04:33.411275+0000","last_peered":"2024-04-25T10:04:33.411275+0000","last_clean":"2024-04-25T10:04:33.411275+0000","last_became_active":"2024-04-25T10:04:31.347455+0000","last_became_peered":"2024-04-25T10:04:31.347455+0000","last_unstale":"2024-04-25T10:04:33.411275+0000","last_undegraded":"2024-04-25T10:04:33.411275+0000","last_fullsized":"2024-04-25T10:04:33.411275+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410884+0000","last_change":"2024-04-25T10:04:30.332586+0000","last_active":"2024-04-25T10:04:33.410884+0000","last_peered":"2024-04-25T10:04:33.410884+0000","last_clean":"2024-04-25T10:04:33.410884+0000","last_became_active":"2024-04-25T10:04:30.332228+0000","last_became_peered":"2024-04-25T10:04:30.332228+0000","last_unstale":"2024-04-25T10:04:33.410884+0000","last_undegraded":"2024-04-25T10:04:33.410884+0000","last_fullsized":"2024-04-25T10:04:33.410884+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:18.535207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412894+0000","last_change":"2024-04-25T10:04:31.344684+0000","last_active":"2024-04-25T10:04:32.412894+0000","last_peered":"2024-04-25T10:04:32.412894+0000","last_clean":"2024-04-25T10:04:32.412894+0000","last_became_active":"2024-04-25T10:04:31.344419+0000","last_became_peered":"2024-04-25T10:04:31.344419+0000","last_unstale":"2024-04-25T10:04:32.412894+0000","last_undegraded":"2024-04-25T10:04:32.412894+0000","last_fullsized":"2024-04-25T10:04:32.412894+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.419422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"158'8","reported_seq":22,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357105+0000","last_change":"2024-04-25T10:04:30.334000+0000","last_active":"2024-04-25T10:04:32.357105+0000","last_peered":"2024-04-25T10:04:32.357105+0000","last_clean":"2024-04-25T10:04:32.357105+0000","last_became_active":"2024-04-25T10:04:30.333633+0000","last_became_peered":"2024-04-25T10:04:30.333633+0000","last_unstale":"2024-04-25T10:04:32.357105+0000","last_undegraded":"2024-04-25T10:04:32.357105+0000","last_fullsized":"2024-04-25T10:04:32.357105+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.870627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.356999+0000","last_change":"2024-04-25T10:04:31.338320+0000","last_active":"2024-04-25T10:04:32.356999+0000","last_peered":"2024-04-25T10:04:32.356999+0000","last_clean":"2024-04-25T10:04:32.356999+0000","last_became_active":"2024-04-25T10:04:31.338004+0000","last_became_peered":"2024-04-25T10:04:31.338004+0000","last_unstale":"2024-04-25T10:04:32.356999+0000","last_undegraded":"2024-04-25T10:04:32.356999+0000","last_fullsized":"2024-04-25T10:04:32.356999+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:03.448151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355633+0000","last_change":"2024-04-25T10:04:30.321626+0000","last_active":"2024-04-25T10:04:32.355633+0000","last_peered":"2024-04-25T10:04:32.355633+0000","last_clean":"2024-04-25T10:04:32.355633+0000","last_became_active":"2024-04-25T10:04:30.321309+0000","last_became_peered":"2024-04-25T10:04:30.321309+0000","last_unstale":"2024-04-25T10:04:32.355633+0000","last_undegraded":"2024-04-25T10:04:32.355633+0000","last_fullsized":"2024-04-25T10:04:32.355633+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:23.214192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411081+0000","last_change":"2024-04-25T10:04:31.342568+0000","last_active":"2024-04-25T10:04:33.411081+0000","last_peered":"2024-04-25T10:04:33.411081+0000","last_clean":"2024-04-25T10:04:33.411081+0000","last_became_active":"2024-04-25T10:04:31.342305+0000","last_became_peered":"2024-04-25T10:04:31.342305+0000","last_unstale":"2024-04-25T10:04:33.411081+0000","last_undegraded":"2024-04-25T10:04:33.411081+0000","last_fullsized":"2024-04-25T10:04:33.411081+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:45.369177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357441+0000","last_change":"2024-04-25T10:04:30.333730+0000","last_active":"2024-04-25T10:04:32.357441+0000","last_peered":"2024-04-25T10:04:32.357441+0000","last_clean":"2024-04-25T10:04:32.357441+0000","last_became_active":"2024-04-25T10:04:30.333378+0000","last_became_peered":"2024-04-25T10:04:30.333378+0000","last_unstale":"2024-04-25T10:04:32.357441+0000","last_undegraded":"2024-04-25T10:04:32.357441+0000","last_fullsized":"2024-04-25T10:04:32.357441+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:58.863328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410915+0000","last_change":"2024-04-25T10:04:31.344033+0000","last_active":"2024-04-25T10:04:33.410915+0000","last_peered":"2024-04-25T10:04:33.410915+0000","last_clean":"2024-04-25T10:04:33.410915+0000","last_became_active":"2024-04-25T10:04:31.343747+0000","last_became_peered":"2024-04-25T10:04:31.343747+0000","last_unstale":"2024-04-25T10:04:33.410915+0000","last_undegraded":"2024-04-25T10:04:33.410915+0000","last_fullsized":"2024-04-25T10:04:33.410915+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:55.547730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331204+0000","last_change":"2024-04-25T10:04:30.324053+0000","last_active":"2024-04-25T10:04:32.331204+0000","last_peered":"2024-04-25T10:04:32.331204+0000","last_clean":"2024-04-25T10:04:32.331204+0000","last_became_active":"2024-04-25T10:04:30.323799+0000","last_became_peered":"2024-04-25T10:04:30.323799+0000","last_unstale":"2024-04-25T10:04:32.331204+0000","last_undegraded":"2024-04-25T10:04:32.331204+0000","last_fullsized":"2024-04-25T10:04:32.331204+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:54.543434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410961+0000","last_change":"2024-04-25T10:04:31.354111+0000","last_active":"2024-04-25T10:04:33.410961+0000","last_peered":"2024-04-25T10:04:33.410961+0000","last_clean":"2024-04-25T10:04:33.410961+0000","last_became_active":"2024-04-25T10:04:31.353888+0000","last_became_peered":"2024-04-25T10:04:31.353888+0000","last_unstale":"2024-04-25T10:04:33.410961+0000","last_undegraded":"2024-04-25T10:04:33.410961+0000","last_fullsized":"2024-04-25T10:04:33.410961+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:18.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357110+0000","last_change":"2024-04-25T10:04:30.335564+0000","last_active":"2024-04-25T10:04:32.357110+0000","last_peered":"2024-04-25T10:04:32.357110+0000","last_clean":"2024-04-25T10:04:32.357110+0000","last_became_active":"2024-04-25T10:04:30.335292+0000","last_became_peered":"2024-04-25T10:04:30.335292+0000","last_unstale":"2024-04-25T10:04:32.357110+0000","last_undegraded":"2024-04-25T10:04:32.357110+0000","last_fullsized":"2024-04-25T10:04:32.357110+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:51.614469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355558+0000","last_change":"2024-04-25T10:04:31.351313+0000","last_active":"2024-04-25T10:04:32.355558+0000","last_peered":"2024-04-25T10:04:32.355558+0000","last_clean":"2024-04-25T10:04:32.355558+0000","last_became_active":"2024-04-25T10:04:31.350825+0000","last_became_peered":"2024-04-25T10:04:31.350825+0000","last_unstale":"2024-04-25T10:04:32.355558+0000","last_undegraded":"2024-04-25T10:04:32.355558+0000","last_fullsized":"2024-04-25T10:04:32.355558+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:36.485307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331435+0000","last_change":"2024-04-25T10:04:30.328432+0000","last_active":"2024-04-25T10:04:32.331435+0000","last_peered":"2024-04-25T10:04:32.331435+0000","last_clean":"2024-04-25T10:04:32.331435+0000","last_became_active":"2024-04-25T10:04:30.327752+0000","last_became_peered":"2024-04-25T10:04:30.327752+0000","last_unstale":"2024-04-25T10:04:32.331435+0000","last_undegraded":"2024-04-25T10:04:32.331435+0000","last_fullsized":"2024-04-25T10:04:32.331435+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331155+0000","last_change":"2024-04-25T10:04:31.347847+0000","last_active":"2024-04-25T10:04:32.331155+0000","last_peered":"2024-04-25T10:04:32.331155+0000","last_clean":"2024-04-25T10:04:32.331155+0000","last_became_active":"2024-04-25T10:04:31.345601+0000","last_became_peered":"2024-04-25T10:04:31.345601+0000","last_unstale":"2024-04-25T10:04:32.331155+0000","last_undegraded":"2024-04-25T10:04:32.331155+0000","last_fullsized":"2024-04-25T10:04:32.331155+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:37.276457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410810+0000","last_change":"2024-04-25T10:04:30.336366+0000","last_active":"2024-04-25T10:04:33.410810+0000","last_peered":"2024-04-25T10:04:33.410810+0000","last_clean":"2024-04-25T10:04:33.410810+0000","last_became_active":"2024-04-25T10:04:30.336105+0000","last_became_peered":"2024-04-25T10:04:30.336105+0000","last_unstale":"2024-04-25T10:04:33.410810+0000","last_undegraded":"2024-04-25T10:04:33.410810+0000","last_fullsized":"2024-04-25T10:04:33.410810+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.870627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331210+0000","last_change":"2024-04-25T10:04:31.346249+0000","last_active":"2024-04-25T10:04:32.331210+0000","last_peered":"2024-04-25T10:04:32.331210+0000","last_clean":"2024-04-25T10:04:32.331210+0000","last_became_active":"2024-04-25T10:04:31.345995+0000","last_became_peered":"2024-04-25T10:04:31.345995+0000","last_unstale":"2024-04-25T10:04:32.331210+0000","last_undegraded":"2024-04-25T10:04:32.331210+0000","last_fullsized":"2024-04-25T10:04:32.331210+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.090553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412584+0000","last_change":"2024-04-25T10:04:30.335680+0000","last_active":"2024-04-25T10:04:32.412584+0000","last_peered":"2024-04-25T10:04:32.412584+0000","last_clean":"2024-04-25T10:04:32.412584+0000","last_became_active":"2024-04-25T10:04:30.335429+0000","last_became_peered":"2024-04-25T10:04:30.335429+0000","last_unstale":"2024-04-25T10:04:32.412584+0000","last_undegraded":"2024-04-25T10:04:32.412584+0000","last_fullsized":"2024-04-25T10:04:32.412584+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:33.032462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410998+0000","last_change":"2024-04-25T10:04:31.342014+0000","last_active":"2024-04-25T10:04:33.410998+0000","last_peered":"2024-04-25T10:04:33.410998+0000","last_clean":"2024-04-25T10:04:33.410998+0000","last_became_active":"2024-04-25T10:04:31.341742+0000","last_became_peered":"2024-04-25T10:04:31.341742+0000","last_unstale":"2024-04-25T10:04:33.410998+0000","last_undegraded":"2024-04-25T10:04:33.410998+0000","last_fullsized":"2024-04-25T10:04:33.410998+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:39.973371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411552+0000","last_change":"2024-04-25T10:04:30.327551+0000","last_active":"2024-04-25T10:04:33.411552+0000","last_peered":"2024-04-25T10:04:33.411552+0000","last_clean":"2024-04-25T10:04:33.411552+0000","last_became_active":"2024-04-25T10:04:30.327245+0000","last_became_peered":"2024-04-25T10:04:30.327245+0000","last_unstale":"2024-04-25T10:04:33.411552+0000","last_undegraded":"2024-04-25T10:04:33.411552+0000","last_fullsized":"2024-04-25T10:04:33.411552+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:13.538776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410462+0000","last_change":"2024-04-25T10:04:31.343068+0000","last_active":"2024-04-25T10:04:33.410462+0000","last_peered":"2024-04-25T10:04:33.410462+0000","last_clean":"2024-04-25T10:04:33.410462+0000","last_became_active":"2024-04-25T10:04:31.342804+0000","last_became_peered":"2024-04-25T10:04:31.342804+0000","last_unstale":"2024-04-25T10:04:33.410462+0000","last_undegraded":"2024-04-25T10:04:33.410462+0000","last_fullsized":"2024-04-25T10:04:33.410462+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:13.580436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.331100+0000","last_change":"2024-04-25T10:04:30.329132+0000","last_active":"2024-04-25T10:04:32.331100+0000","last_peered":"2024-04-25T10:04:32.331100+0000","last_clean":"2024-04-25T10:04:32.331100+0000","last_became_active":"2024-04-25T10:04:30.328879+0000","last_became_peered":"2024-04-25T10:04:30.328879+0000","last_unstale":"2024-04-25T10:04:32.331100+0000","last_undegraded":"2024-04-25T10:04:32.331100+0000","last_fullsized":"2024-04-25T10:04:32.331100+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:17.500610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357441+0000","last_change":"2024-04-25T10:04:31.342101+0000","last_active":"2024-04-25T10:04:32.357441+0000","last_peered":"2024-04-25T10:04:32.357441+0000","last_clean":"2024-04-25T10:04:32.357441+0000","last_became_active":"2024-04-25T10:04:31.341847+0000","last_became_peered":"2024-04-25T10:04:31.341847+0000","last_unstale":"2024-04-25T10:04:32.357441+0000","last_undegraded":"2024-04-25T10:04:32.357441+0000","last_fullsized":"2024-04-25T10:04:32.357441+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:27.502026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"159'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355551+0000","last_change":"2024-04-25T10:04:30.334642+0000","last_active":"2024-04-25T10:04:32.355551+0000","last_peered":"2024-04-25T10:04:32.355551+0000","last_clean":"2024-04-25T10:04:32.355551+0000","last_became_active":"2024-04-25T10:04:30.334411+0000","last_became_peered":"2024-04-25T10:04:30.334411+0000","last_unstale":"2024-04-25T10:04:32.355551+0000","last_undegraded":"2024-04-25T10:04:32.355551+0000","last_fullsized":"2024-04-25T10:04:32.355551+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.327895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412923+0000","last_change":"2024-04-25T10:04:31.340066+0000","last_active":"2024-04-25T10:04:32.412923+0000","last_peered":"2024-04-25T10:04:32.412923+0000","last_clean":"2024-04-25T10:04:32.412923+0000","last_became_active":"2024-04-25T10:04:31.339657+0000","last_became_peered":"2024-04-25T10:04:31.339657+0000","last_unstale":"2024-04-25T10:04:32.412923+0000","last_undegraded":"2024-04-25T10:04:32.412923+0000","last_fullsized":"2024-04-25T10:04:32.412923+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:23.249081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355337+0000","last_change":"2024-04-25T10:04:30.332838+0000","last_active":"2024-04-25T10:04:32.355337+0000","last_peered":"2024-04-25T10:04:32.355337+0000","last_clean":"2024-04-25T10:04:32.355337+0000","last_became_active":"2024-04-25T10:04:30.332568+0000","last_became_peered":"2024-04-25T10:04:30.332568+0000","last_unstale":"2024-04-25T10:04:32.355337+0000","last_undegraded":"2024-04-25T10:04:32.355337+0000","last_fullsized":"2024-04-25T10:04:32.355337+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:08.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.357240+0000","last_change":"2024-04-25T10:04:31.342113+0000","last_active":"2024-04-25T10:04:32.357240+0000","last_peered":"2024-04-25T10:04:32.357240+0000","last_clean":"2024-04-25T10:04:32.357240+0000","last_became_active":"2024-04-25T10:04:31.339474+0000","last_became_peered":"2024-04-25T10:04:31.339474+0000","last_unstale":"2024-04-25T10:04:32.357240+0000","last_undegraded":"2024-04-25T10:04:32.357240+0000","last_fullsized":"2024-04-25T10:04:32.357240+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:55.547730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412715+0000","last_change":"2024-04-25T10:04:30.336384+0000","last_active":"2024-04-25T10:04:32.412715+0000","last_peered":"2024-04-25T10:04:32.412715+0000","last_clean":"2024-04-25T10:04:32.412715+0000","last_became_active":"2024-04-25T10:04:30.336066+0000","last_became_peered":"2024-04-25T10:04:30.336066+0000","last_unstale":"2024-04-25T10:04:32.412715+0000","last_undegraded":"2024-04-25T10:04:32.412715+0000","last_fullsized":"2024-04-25T10:04:32.412715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:05.056296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.334160+0000","last_change":"2024-04-25T10:04:31.344710+0000","last_active":"2024-04-25T10:04:32.334160+0000","last_peered":"2024-04-25T10:04:32.334160+0000","last_clean":"2024-04-25T10:04:32.334160+0000","last_became_active":"2024-04-25T10:04:31.343961+0000","last_became_peered":"2024-04-25T10:04:31.343961+0000","last_unstale":"2024-04-25T10:04:32.334160+0000","last_undegraded":"2024-04-25T10:04:32.334160+0000","last_fullsized":"2024-04-25T10:04:32.334160+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:16.066745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.333715+0000","last_change":"2024-04-25T10:04:30.323701+0000","last_active":"2024-04-25T10:04:32.333715+0000","last_peered":"2024-04-25T10:04:32.333715+0000","last_clean":"2024-04-25T10:04:32.333715+0000","last_became_active":"2024-04-25T10:04:30.323240+0000","last_became_peered":"2024-04-25T10:04:30.323240+0000","last_unstale":"2024-04-25T10:04:32.333715+0000","last_undegraded":"2024-04-25T10:04:32.333715+0000","last_fullsized":"2024-04-25T10:04:32.333715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:33.867172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410786+0000","last_change":"2024-04-25T10:04:31.340949+0000","last_active":"2024-04-25T10:04:33.410786+0000","last_peered":"2024-04-25T10:04:33.410786+0000","last_clean":"2024-04-25T10:04:33.410786+0000","last_became_active":"2024-04-25T10:04:31.340582+0000","last_became_peered":"2024-04-25T10:04:31.340582+0000","last_unstale":"2024-04-25T10:04:33.410786+0000","last_undegraded":"2024-04-25T10:04:33.410786+0000","last_fullsized":"2024-04-25T10:04:33.410786+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:13.580436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355446+0000","last_change":"2024-04-25T10:04:30.326550+0000","last_active":"2024-04-25T10:04:32.355446+0000","last_peered":"2024-04-25T10:04:32.355446+0000","last_clean":"2024-04-25T10:04:32.355446+0000","last_became_active":"2024-04-25T10:04:30.325966+0000","last_became_peered":"2024-04-25T10:04:30.325966+0000","last_unstale":"2024-04-25T10:04:32.355446+0000","last_undegraded":"2024-04-25T10:04:32.355446+0000","last_fullsized":"2024-04-25T10:04:32.355446+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:25.494064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355454+0000","last_change":"2024-04-25T10:04:31.349087+0000","last_active":"2024-04-25T10:04:32.355454+0000","last_peered":"2024-04-25T10:04:32.355454+0000","last_clean":"2024-04-25T10:04:32.355454+0000","last_became_active":"2024-04-25T10:04:31.348812+0000","last_became_peered":"2024-04-25T10:04:31.348812+0000","last_unstale":"2024-04-25T10:04:32.355454+0000","last_undegraded":"2024-04-25T10:04:32.355454+0000","last_fullsized":"2024-04-25T10:04:32.355454+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:14.542078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.410991+0000","last_change":"2024-04-25T10:04:30.334944+0000","last_active":"2024-04-25T10:04:33.410991+0000","last_peered":"2024-04-25T10:04:33.410991+0000","last_clean":"2024-04-25T10:04:33.410991+0000","last_became_active":"2024-04-25T10:04:30.334680+0000","last_became_peered":"2024-04-25T10:04:30.334680+0000","last_unstale":"2024-04-25T10:04:33.410991+0000","last_undegraded":"2024-04-25T10:04:33.410991+0000","last_fullsized":"2024-04-25T10:04:33.410991+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:58.863328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.412954+0000","last_change":"2024-04-25T10:04:31.347735+0000","last_active":"2024-04-25T10:04:32.412954+0000","last_peered":"2024-04-25T10:04:32.412954+0000","last_clean":"2024-04-25T10:04:32.412954+0000","last_became_active":"2024-04-25T10:04:31.347443+0000","last_became_peered":"2024-04-25T10:04:31.347443+0000","last_unstale":"2024-04-25T10:04:32.412954+0000","last_undegraded":"2024-04-25T10:04:32.412954+0000","last_fullsized":"2024-04-25T10:04:32.412954+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:22.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.330773+0000","last_change":"2024-04-25T10:04:30.328739+0000","last_active":"2024-04-25T10:04:32.330773+0000","last_peered":"2024-04-25T10:04:32.330773+0000","last_clean":"2024-04-25T10:04:32.330773+0000","last_became_active":"2024-04-25T10:04:30.328488+0000","last_became_peered":"2024-04-25T10:04:30.328488+0000","last_unstale":"2024-04-25T10:04:32.330773+0000","last_undegraded":"2024-04-25T10:04:32.330773+0000","last_fullsized":"2024-04-25T10:04:32.330773+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:08.925775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411405+0000","last_change":"2024-04-25T10:04:31.336228+0000","last_active":"2024-04-25T10:04:33.411405+0000","last_peered":"2024-04-25T10:04:33.411405+0000","last_clean":"2024-04-25T10:04:33.411405+0000","last_became_active":"2024-04-25T10:04:31.335914+0000","last_became_peered":"2024-04-25T10:04:31.335914+0000","last_unstale":"2024-04-25T10:04:33.411405+0000","last_undegraded":"2024-04-25T10:04:33.411405+0000","last_fullsized":"2024-04-25T10:04:33.411405+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:30.312402+0000","last_clean_scrub_stamp":"2024-04-25T10:04:30.312402+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:15.622043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:32.355405+0000","last_change":"2024-04-25T10:04:30.334713+0000","last_active":"2024-04-25T10:04:32.355405+0000","last_peered":"2024-04-25T10:04:32.355405+0000","last_clean":"2024-04-25T10:04:32.355405+0000","last_became_active":"2024-04-25T10:04:30.334490+0000","last_became_peered":"2024-04-25T10:04:30.334490+0000","last_unstale":"2024-04-25T10:04:32.355405+0000","last_undegraded":"2024-04-25T10:04:32.355405+0000","last_fullsized":"2024-04-25T10:04:32.355405+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:29.308107+0000","last_clean_scrub_stamp":"2024-04-25T10:04:29.308107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:26.497730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":426,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:04:33.411559+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:04:33.411559+0000","last_peered":"2024-04-25T10:04:33.411559+0000","last_clean":"2024-04-25T10:04:33.411559+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:04:33.411559+0000","last_undegraded":"2024-04-25T10:04:33.411559+0000","last_fullsized":"2024-04-25T10:04:33.411559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2592768,"data_stored":1526292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51184,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320656,"statfs":{"total":96636764160,"available":96584351744,"internally_reserved":0,"allocated":1998848,"data_stored":935924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51188,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320652,"statfs":{"total":96636764160,"available":96584347648,"internally_reserved":0,"allocated":2007040,"data_stored":936048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59524,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312316,"statfs":{"total":96636764160,"available":96575811584,"internally_reserved":0,"allocated":2015232,"data_stored":936688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574977,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59528,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312312,"statfs":{"total":96636764160,"available":96575807488,"internally_reserved":0,"allocated":2019328,"data_stored":938140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55340,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316500,"statfs":{"total":96636764160,"available":96580096000,"internally_reserved":0,"allocated":1998848,"data_stored":935924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55360,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316480,"statfs":{"total":96636764160,"available":96580075520,"internally_reserved":0,"allocated":2011136,"data_stored":937536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55936,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315904,"statfs":{"total":96636764160,"available":96579485696,"internally_reserved":0,"allocated":2600960,"data_stored":1526396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:38.986 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:04:38.986 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:04:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.120+0000 7fb3eeb5c640 1 Processor -- start 2024-04-25T10:04:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.120+0000 7fb3eeb5c640 1 -- start start 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3eeb5c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3e806ccd0 0x7fb3e80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3eeb5c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80d1770 con 0x7fb3e815deb0 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3eeb5c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e8162740 con 0x7fb3e8070dc0 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3eeb5c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e81622b0 con 0x7fb3e806ccd0 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3ed0f9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb3e815deb0 0x7fb3e815e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36928/0 (socket says 172.21.15.148:36928) 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3ed0f9640 1 -- 172.21.15.148:0/3431724191 learned_addr learned my addr 172.21.15.148:0/3431724191 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e7fff640 1 --2- 172.21.15.148:0/3431724191 >> v2:172.21.15.188:3300/0 conn(0x7fb3e806ccd0 0x7fb3e80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435455970 0 0) 0x7fb3e80d1770 con 0x7fb3e815deb0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d8003520 con 0x7fb3e815deb0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121230190 0 0) 0x7fb3e8162740 con 0x7fb3e8070dc0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e80d1770 con 0x7fb3e8070dc0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3686191063 0 0) 0x7fb3d8003520 con 0x7fb3e815deb0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e7fff640 1 -- 172.21.15.148:0/3431724191 >> v1:172.21.15.188:6789/0 conn(0x7fb3e8070dc0 legacy=0x7fb3e80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e7fff640 1 -- 172.21.15.148:0/3431724191 >> v1:172.21.15.148:6789/0 conn(0x7fb3e815deb0 legacy=0x7fb3e815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.121+0000 7fb3e7fff640 1 -- 172.21.15.148:0/3431724191 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e8161980 con 0x7fb3e806ccd0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.122+0000 7fb3e7fff640 1 --2- 172.21.15.148:0/3431724191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e806ccd0 0x7fb3e80b07a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40097a0 tx=0x7fb3d402a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99913673b2d98299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.122+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4037070 con 0x7fb3e806ccd0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.122+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d402ab30 con 0x7fb3e806ccd0 2024-04-25T10:04:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.122+0000 7fb3e77fe640 1 -- 172.21.15.148:0/3431724191 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00019f0 con 0x7fb3e806ccd0 2024-04-25T10:04:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/3431724191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e806ccd0 msgr2=0x7fb3e80b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/3431724191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e806ccd0 0x7fb3e80b07a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40097a0 tx=0x7fb3d402a0a0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/3431724191 shutdown_connections 2024-04-25T10:04:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/3431724191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e806ccd0 0x7fb3e80b07a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/3431724191 >> 172.21.15.148:0/3431724191 conn(0x7fb3e80b3530 msgr2=0x7fb3e80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/3431724191 shutdown_connections 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.123+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/3431724191 wait complete. 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.124+0000 7fb3eeb5c640 1 Processor -- start 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.124+0000 7fb3eeb5c640 1 -- start start 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3eeb5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3e806ccd0 0x7fb3e8150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3eeb5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 0x7fb3e80d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3eeb5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e815deb0 0x7fb3e80d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3eeb5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e8161f40 con 0x7fb3e8070dc0 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3eeb5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80b10f0 con 0x7fb3e815deb0 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3eeb5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80b5840 con 0x7fb3e806ccd0 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 0x7fb3e80d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 0x7fb3e80d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41378/0 (socket says 172.21.15.148:41378) 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 -- 172.21.15.148:0/2153117480 learned_addr learned my addr 172.21.15.148:0/2153117480 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3ed0f9640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e815deb0 0x7fb3e80d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3ec8f8640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3e806ccd0 0x7fb3e8150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 -- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3e806ccd0 msgr2=0x7fb3e8150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3e806ccd0 0x7fb3e8150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 -- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e815deb0 msgr2=0x7fb3e80d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e815deb0 0x7fb3e80d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.125+0000 7fb3e7fff640 1 -- 172.21.15.148:0/2153117480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e8161170 con 0x7fb3e8070dc0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.126+0000 7fb3e7fff640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 0x7fb3e80d0510 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4035040 tx=0x7fb3d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.126+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4037050 con 0x7fb3e8070dc0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.126+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4043070 con 0x7fb3e8070dc0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.126+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e81600d0 con 0x7fb3e8070dc0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.126+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d403ca30 con 0x7fb3e8070dc0 2024-04-25T10:04:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.126+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e8160690 con 0x7fb3e8070dc0 2024-04-25T10:04:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.127+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e8073a00 con 0x7fb3e8070dc0 2024-04-25T10:04:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.127+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3d403cbd0 con 0x7fb3e8070dc0 2024-04-25T10:04:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.128+0000 7fb3e57fa640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb3cc06b4d0 0x7fb3cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.128+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3d40aad20 con 0x7fb3e8070dc0 2024-04-25T10:04:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.128+0000 7fb3ec8f8640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb3cc06b4d0 0x7fb3cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.128+0000 7fb3ec8f8640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb3cc06b4d0 0x7fb3cc06d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0011e0 tx=0x7fb3dc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.132+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3d4079570 con 0x7fb3e8070dc0 2024-04-25T10:04:39.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.374+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb3e806e570 con 0x7fb3e8070dc0 2024-04-25T10:04:39.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.376+0000 7fb3e57fa640 1 -- 172.21.15.148:0/2153117480 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb3d407d610 con 0x7fb3e8070dc0 2024-04-25T10:04:39.382 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:04:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:04:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:04:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:04:39.383 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:39.383 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb3cc06b4d0 msgr2=0x7fb3cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb3cc06b4d0 0x7fb3cc06d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0011e0 tx=0x7fb3dc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 msgr2=0x7fb3e80d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 0x7fb3e80d0510 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4035040 tx=0x7fb3d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 shutdown_connections 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb3cc06b4d0 0x7fb3cc06d990 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3e815deb0 0x7fb3e80d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3e8070dc0 0x7fb3e80d0510 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 --2- 172.21.15.148:0/2153117480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3e806ccd0 0x7fb3e8150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 >> 172.21.15.148:0/2153117480 conn(0x7fb3e80b3530 msgr2=0x7fb3e80cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 shutdown_connections 2024-04-25T10:04:39.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.378+0000 7fb3eeb5c640 1 -- 172.21.15.148:0/2153117480 wait complete. 2024-04-25T10:04:39.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:39.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.486+0000 7f9dcf6c4640 1 Processor -- start 2024-04-25T10:04:39.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 -- start start 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dc813d190 0x7f9dc80cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc815b320 0x7f9dc80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dc8071230 0x7f9dc80716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc813d570 con 0x7f9dc813d190 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc8071c00 con 0x7f9dc8071230 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.487+0000 7f9dcf6c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8071ed0 con 0x7f9dc815b320 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dc813d190 0x7f9dc80cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9dc813d190 0x7f9dc80cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41392/0 (socket says 172.21.15.148:41392) 2024-04-25T10:04:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 -- 172.21.15.148:0/3184258329 learned_addr learned my addr 172.21.15.148:0/3184258329 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 -- 172.21.15.148:0/3184258329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc815b320 msgr2=0x7f9dc80cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 --2- 172.21.15.148:0/3184258329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc815b320 0x7f9dc80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 -- 172.21.15.148:0/3184258329 >> v2:172.21.15.188:3300/0 conn(0x7f9dc8071230 msgr2=0x7f9dc80716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 --2- 172.21.15.148:0/3184258329 >> v2:172.21.15.188:3300/0 conn(0x7f9dc8071230 0x7f9dc80716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.488+0000 7f9dcd460640 1 -- 172.21.15.148:0/3184258329 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc815a840 con 0x7f9dc813d190 2024-04-25T10:04:39.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcd460640 1 --2- 172.21.15.148:0/3184258329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc813d190 0x7f9dc80cbfb0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f9dc806f320 tx=0x7f9db802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a809fed3b058276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dbffff640 1 -- 172.21.15.148:0/3184258329 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8037070 con 0x7f9dc813d190 2024-04-25T10:04:39.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dbffff640 1 -- 172.21.15.148:0/3184258329 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db8032440 con 0x7f9dc813d190 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dbffff640 1 -- 172.21.15.148:0/3184258329 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8032810 con 0x7f9dc813d190 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/3184258329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc813d190 msgr2=0x7f9dc80cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/3184258329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc813d190 0x7f9dc80cbfb0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f9dc806f320 tx=0x7f9db802a140 comp rx=0 tx=0).stop 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/3184258329 shutdown_connections 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/3184258329 >> v2:172.21.15.188:3300/0 conn(0x7f9dc8071230 0x7f9dc80716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/3184258329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc815b320 0x7f9dc80cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/3184258329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc813d190 0x7f9dc80cbfb0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/3184258329 >> 172.21.15.148:0/3184258329 conn(0x7f9dc806abe0 msgr2=0x7f9dc806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/3184258329 shutdown_connections 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.489+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/3184258329 wait complete. 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.490+0000 7f9dcf6c4640 1 Processor -- start 2024-04-25T10:04:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 -- start start 2024-04-25T10:04:39.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8071230 0x7f9dc80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc813d190 0x7f9dc81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 0x7f9dc8150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cf620 con 0x7f9dc815b320 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815ae00 con 0x7f9dc8071230 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcf6c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cf7a0 con 0x7f9dc813d190 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcdc61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 0x7f9dc8150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcdc61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 0x7f9dc8150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41396/0 (socket says 172.21.15.148:41396) 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcdc61640 1 -- 172.21.15.148:0/697083016 learned_addr learned my addr 172.21.15.148:0/697083016 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dccc5f640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc813d190 0x7f9dc81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.491+0000 7f9dcd460640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8071230 0x7f9dc80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcdc61640 1 -- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc813d190 msgr2=0x7f9dc81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcdc61640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc813d190 0x7f9dc81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcdc61640 1 -- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8071230 msgr2=0x7f9dc80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcdc61640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8071230 0x7f9dc80b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcdc61640 1 -- 172.21.15.148:0/697083016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80af630 con 0x7f9dc815b320 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcd460640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8071230 0x7f9dc80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcdc61640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 0x7f9dc8150010 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b970 tx=0x7f9dc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0019070 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc000cb60 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc80af910 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0014440 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.492+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc80afed0 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.493+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc8073990 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.493+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dc001c070 con 0x7f9dc815b320 2024-04-25T10:04:39.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.494+0000 7f9dbdffb640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dac06b4d0 0x7f9dac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.494+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9dc008a000 con 0x7f9dc815b320 2024-04-25T10:04:39.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.494+0000 7f9dcd460640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dac06b4d0 0x7f9dac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.494+0000 7f9dcd460640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dac06b4d0 0x7f9dac06d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9db8034000 tx=0x7f9db802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.498+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dc0058d80 con 0x7f9dc815b320 2024-04-25T10:04:39.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.702+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc81590e0 con 0x7f9dc815b320 2024-04-25T10:04:39.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.702+0000 7f9dbdffb640 1 -- 172.21.15.148:0/697083016 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f9dc0058680 con 0x7f9dc815b320 2024-04-25T10:04:39.709 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:39.709 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:30.667288+0000","modified":"2024-04-25T10:04:32.676174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10365":{"gid":10365,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3286540466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3286540466},{"type":"v1","addr":"172.21.15.148:6837","nonce":3286540466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:04:39.709 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 93 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dac06b4d0 msgr2=0x7f9dac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dac06b4d0 0x7f9dac06d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9db8034000 tx=0x7f9db802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 msgr2=0x7f9dc8150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 0x7f9dc8150010 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b970 tx=0x7f9dc000be30 comp rx=0 tx=0).stop 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 shutdown_connections 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9dac06b4d0 0x7f9dac06d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc815b320 0x7f9dc8150010 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc813d190 0x7f9dc81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 --2- 172.21.15.148:0/697083016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8071230 0x7f9dc80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 >> 172.21.15.148:0/697083016 conn(0x7f9dc806abe0 msgr2=0x7f9dc806ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 shutdown_connections 2024-04-25T10:04:39.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:39.704+0000 7f9dcf6c4640 1 -- 172.21.15.148:0/697083016 wait complete. 2024-04-25T10:04:40.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:04:30.667288+0000', 'modified': '2024-04-25T10:04:32.676174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10365}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10365': {'gid': 10365, 'name': 'c', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/3286540466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 3286540466}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3286540466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:04:40.124 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:04:40.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:04:40.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.219+0000 7fa6cb45d640 1 Processor -- start 2024-04-25T10:04:40.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.219+0000 7fa6cb45d640 1 -- start start 2024-04-25T10:04:40.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6cb45d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6c4144d20 0x7fa6c40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6cb45d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c41454b0 0x7fa6c4070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6cb45d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6c4071300 0x7fa6c415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6cb45d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c4071750 con 0x7fa6c4144d20 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6cb45d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c40b4830 con 0x7fa6c4071300 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6cb45d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40ceba0 con 0x7fa6c41454b0 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6c4144d20 0x7fa6c40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6c4144d20 0x7fa6c40ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41406/0 (socket says 172.21.15.148:41406) 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 -- 172.21.15.148:0/799129244 learned_addr learned my addr 172.21.15.148:0/799129244 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c99fa640 1 --2- 172.21.15.148:0/799129244 >> v2:172.21.15.188:3300/0 conn(0x7fa6c4071300 0x7fa6c415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/799129244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c41454b0 0x7fa6c4070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 -- 172.21.15.148:0/799129244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c41454b0 msgr2=0x7fa6c4070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/799129244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c41454b0 0x7fa6c4070dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 -- 172.21.15.148:0/799129244 >> v2:172.21.15.188:3300/0 conn(0x7fa6c4071300 msgr2=0x7fa6c415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/799129244 >> v2:172.21.15.188:3300/0 conn(0x7fa6c4071300 0x7fa6c415f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c91f9640 1 -- 172.21.15.148:0/799129244 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40cee80 con 0x7fa6c4144d20 2024-04-25T10:04:40.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.220+0000 7fa6c99fa640 1 --2- 172.21.15.148:0/799129244 >> v2:172.21.15.188:3300/0 conn(0x7fa6c4071300 0x7fa6c415f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.221+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/799129244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ce350 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40b45d0 tx=0x7fa6c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97db238e888c9a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.221+0000 7fa6bbfff640 1 -- 172.21.15.148:0/799129244 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0036070 con 0x7fa6c4144d20 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.221+0000 7fa6bbfff640 1 -- 172.21.15.148:0/799129244 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c0031600 con 0x7fa6c4144d20 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.221+0000 7fa6bbfff640 1 -- 172.21.15.148:0/799129244 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00319d0 con 0x7fa6c4144d20 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 -- 172.21.15.148:0/799129244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 msgr2=0x7fa6c40ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/799129244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ce350 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40b45d0 tx=0x7fa6c002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 -- 172.21.15.148:0/799129244 shutdown_connections 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/799129244 >> v2:172.21.15.188:3300/0 conn(0x7fa6c4071300 0x7fa6c415f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/799129244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c41454b0 0x7fa6c4070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/799129244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ce350 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 -- 172.21.15.148:0/799129244 >> 172.21.15.148:0/799129244 conn(0x7fa6c40b3530 msgr2=0x7fa6c40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 -- 172.21.15.148:0/799129244 shutdown_connections 2024-04-25T10:04:40.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.222+0000 7fa6cb45d640 1 -- 172.21.15.148:0/799129244 wait complete. 2024-04-25T10:04:40.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.223+0000 7fa6cb45d640 1 Processor -- start 2024-04-25T10:04:40.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.223+0000 7fa6cb45d640 1 -- start start 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6cb45d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c4071300 0x7fa6c41f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6cb45d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6cb45d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6c41454b0 0x7fa6c41d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6cb45d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40ce890 con 0x7fa6c4144d20 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6cb45d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40cf440 con 0x7fa6c41454b0 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6cb45d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4145100 con 0x7fa6c4071300 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41420/0 (socket says 172.21.15.148:41420) 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 -- 172.21.15.148:0/664901351 learned_addr learned my addr 172.21.15.148:0/664901351 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c4071300 0x7fa6c41f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c99fa640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6c41454b0 0x7fa6c41d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 -- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c4071300 msgr2=0x7fa6c41f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c4071300 0x7fa6c41f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 -- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6c41454b0 msgr2=0x7fa6c41d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6c41454b0 0x7fa6c41d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 -- 172.21.15.148:0/664901351 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c41f3c90 con 0x7fa6c4144d20 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c99fa640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6c41454b0 0x7fa6c41d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ccb80 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c7f0 tx=0x7fa6b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6b9ffb640 1 -- 172.21.15.148:0/664901351 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4015070 con 0x7fa6c4144d20 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.224+0000 7fa6b9ffb640 1 -- 172.21.15.148:0/664901351 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4004240 con 0x7fa6c4144d20 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.225+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40b58e0 con 0x7fa6c4144d20 2024-04-25T10:04:40.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.225+0000 7fa6b9ffb640 1 -- 172.21.15.148:0/664901351 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4007620 con 0x7fa6c4144d20 2024-04-25T10:04:40.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.225+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40b5ea0 con 0x7fa6c4144d20 2024-04-25T10:04:40.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.226+0000 7fa6b9ffb640 1 -- 172.21.15.148:0/664901351 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6b40079d0 con 0x7fa6c4144d20 2024-04-25T10:04:40.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.226+0000 7fa6b9ffb640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa6a806b4d0 0x7fa6a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.226+0000 7fa6b9ffb640 1 -- 172.21.15.148:0/664901351 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6b408a340 con 0x7fa6c4144d20 2024-04-25T10:04:40.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.226+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa6a806b4d0 0x7fa6a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.227+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa6a806b4d0 0x7fa6a806d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0033000 tx=0x7fa6c0009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.227+0000 7fa6ce163740 1 Processor -- start 2024-04-25T10:04:40.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.227+0000 7fa6ce163740 1 -- start start 2024-04-25T10:04:40.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e2a07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 0x5566e2a25020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e29dcf10 0x5566e2a23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6ce163740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566e293fa10 con 0x5566e2a24ad0 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566e28e8b30 con 0x5566e2a07230 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566e293bc50 con 0x5566e29dcf10 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 0x5566e2a25020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 0x5566e2a25020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41422/0 (socket says 172.21.15.148:41422) 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 -- 172.21.15.148:0/3619153267 learned_addr learned my addr 172.21.15.148:0/3619153267 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e2a07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c99fa640 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e29dcf10 0x5566e2a23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 -- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e29dcf10 msgr2=0x5566e2a23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e29dcf10 0x5566e2a23190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 -- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 msgr2=0x5566e2a07610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e2a07610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.228+0000 7fa6c89f8640 1 -- 172.21.15.148:0/3619153267 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566e28fde10 con 0x5566e2a24ad0 2024-04-25T10:04:40.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 0x5566e2a25020 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4060b10 tx=0x7fa6b405cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c45be0e7c706f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6aeffd640 1 -- 172.21.15.148:0/3619153267 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b401e040 con 0x5566e2a24ad0 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6aeffd640 1 -- 172.21.15.148:0/3619153267 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4055050 con 0x5566e2a24ad0 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6aeffd640 1 -- 172.21.15.148:0/3619153267 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4057e30 con 0x5566e2a24ad0 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 -- 172.21.15.148:0/3619153267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 msgr2=0x5566e2a25020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 0x5566e2a25020 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4060b10 tx=0x7fa6b405cc00 comp rx=0 tx=0).stop 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 -- 172.21.15.148:0/3619153267 shutdown_connections 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e29dcf10 0x5566e2a23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a24ad0 0x5566e2a25020 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 --2- 172.21.15.148:0/3619153267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e2a07610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 -- 172.21.15.148:0/3619153267 >> 172.21.15.148:0/3619153267 conn(0x5566e29d9840 msgr2=0x5566e2a22040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 -- 172.21.15.148:0/3619153267 shutdown_connections 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.229+0000 7fa6ce163740 1 -- 172.21.15.148:0/3619153267 wait complete. 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.230+0000 7fa6ce163740 1 -- start start 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.230+0000 7fa6ce163740 1 Processor -- start 2024-04-25T10:04:40.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e29f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e29f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566e29ddcb0 con 0x5566e2a07230 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566e28fc0c0 con 0x5566e2a24ad0 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566e29d7770 con 0x5566e2a081e0 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6ce163740 10 client.0 _ll_register_callbacks cb 0x5566e28efec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e29f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e29f0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41430/0 (socket says 172.21.15.148:41430) 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 -- 172.21.15.148:0/2400728130 learned_addr learned my addr 172.21.15.148:0/2400728130 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c99fa640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e29f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 -- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 msgr2=0x5566e2a08660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a08660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 -- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 msgr2=0x5566e29f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e29f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.231+0000 7fa6c91f9640 1 -- 172.21.15.148:0/2400728130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566e26542d0 con 0x5566e2a07230 2024-04-25T10:04:40.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e29f0e60 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000f80 tx=0x7fa6c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09fdb2edb612af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6a3fff640 1 -- 172.21.15.148:0/2400728130 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0036040 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6a3fff640 1 -- 172.21.15.148:0/2400728130 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6c003cc50 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6b4055050 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6a3fff640 1 -- 172.21.15.148:0/2400728130 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c003ce50 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 10 client.10524 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x5566e271e6b0 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6a3fff640 1 -- 172.21.15.148:0/2400728130 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4055050 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6a3fff640 1 -- 172.21.15.148:0/2400728130 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa6c0008040 con 0x5566e2a07230 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 10 client.10524 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 10 client.10524 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 20 client.10524 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29d6ac0 0x5566e29d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 4 client.10524 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:40.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566e299e920 con 0x5566e29d6ac0 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.232+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29d6ac0 0x5566e29d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29d6ac0 0x5566e29d6ea0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50ac485d7b12e39 server_cookie=9acebddd0aed6338 in_seq=0 out_seq=0 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6a3fff640 10 client.10524 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6a3fff640 1 -- 172.21.15.148:0/2400728130 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566e299e920 con 0x5566e29d6ac0 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6a3fff640 10 client.10524 handle_command_reply: tid=0 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 client.10524 shutdown 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 msgr2=0x5566e29f0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e29f0e60 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0000f80 tx=0x7fa6c0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 shutdown_connections 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29d6ac0 0x5566e29d6ea0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a08660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e29f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6a3fff640 0 client.10524 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 --2- 172.21.15.148:0/2400728130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e29f0e60 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 >> 172.21.15.148:0/2400728130 conn(0x5566e29dcf10 msgr2=0x5566e2a21750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 shutdown_connections 2024-04-25T10:04:40.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 1 -- 172.21.15.148:0/2400728130 wait complete. 2024-04-25T10:04:40.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.233+0000 7fa6ce163740 20 client.10524 trim_cache size 0 max 16384 2024-04-25T10:04:40.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.249+0000 7fa6ce163740 1 Processor -- start 2024-04-25T10:04:40.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.249+0000 7fa6ce163740 1 -- start start 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.249+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e29dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.249+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 0x5566e29dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a24ad0 0x5566e2a34780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ce163740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c42003a0 con 0x5566e2a081e0 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41ff5b0 con 0x5566e2a07230 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c406c920 con 0x5566e2a24ad0 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 0x5566e29dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 0x5566e29dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41432/0 (socket says 172.21.15.148:41432) 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 -- 172.21.15.148:0/4012385058 learned_addr learned my addr 172.21.15.148:0/4012385058 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 -- 172.21.15.148:0/4012385058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a24ad0 msgr2=0x5566e2a34780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a24ad0 0x5566e2a34780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 -- 172.21.15.148:0/4012385058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 msgr2=0x5566e29dd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e29dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 -- 172.21.15.148:0/4012385058 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566e29f2330 con 0x5566e2a081e0 2024-04-25T10:04:40.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 0x5566e29dd830 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c7c0 tx=0x7fa6b408a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb4d24e07e46e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ae7fc640 1 -- 172.21.15.148:0/4012385058 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b401e040 con 0x5566e2a081e0 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ae7fc640 1 -- 172.21.15.148:0/4012385058 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40589e0 con 0x5566e2a081e0 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.250+0000 7fa6ae7fc640 1 -- 172.21.15.148:0/4012385058 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40587b0 con 0x5566e2a081e0 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 -- 172.21.15.148:0/4012385058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 msgr2=0x5566e29dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 0x5566e29dd830 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c7c0 tx=0x7fa6b408a000 comp rx=0 tx=0).stop 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 -- 172.21.15.148:0/4012385058 shutdown_connections 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a24ad0 0x5566e2a34780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a081e0 0x5566e29dd830 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 --2- 172.21.15.148:0/4012385058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a07230 0x5566e29dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 -- 172.21.15.148:0/4012385058 >> 172.21.15.148:0/4012385058 conn(0x5566e29dcf10 msgr2=0x5566e2a241a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 -- 172.21.15.148:0/4012385058 shutdown_connections 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 -- 172.21.15.148:0/4012385058 wait complete. 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 -- start start 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.251+0000 7fa6ce163740 1 Processor -- start 2024-04-25T10:04:40.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e2a08fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a064f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e2a34780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566e29f1a40 con 0x5566e2a07230 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c406cf20 con 0x5566e2a24ad0 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566e29f20a0 con 0x5566e2a081e0 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6ce163740 10 client.0 _ll_register_callbacks cb 0x5566e29d7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e2a08fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e2a08fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41438/0 (socket says 172.21.15.148:41438) 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 -- 172.21.15.148:0/1069888809 learned_addr learned my addr 172.21.15.148:0/1069888809 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 -- 172.21.15.148:0/1069888809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 msgr2=0x5566e2a064f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a064f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 -- 172.21.15.148:0/1069888809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 msgr2=0x5566e2a34780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.252+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e2a34780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6c91f9640 1 -- 172.21.15.148:0/1069888809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566e2a243f0 con 0x5566e2a07230 2024-04-25T10:04:40.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6c91f9640 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e2a08fc0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0047a10 tx=0x7fa6c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6b1b2a7bde60a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6a3fff640 1 -- 172.21.15.148:0/1069888809 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c005e030 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6a3fff640 1 -- 172.21.15.148:0/1069888809 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6c004e040 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6b40589e0 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6a3fff640 1 -- 172.21.15.148:0/1069888809 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c0066050 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 10 client.10530 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x5566e2a06a30 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6a3fff640 1 -- 172.21.15.148:0/1069888809 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40589e0 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6a3fff640 1 -- 172.21.15.148:0/1069888809 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa6c0069020 con 0x5566e2a07230 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 10 client.10530 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 10 client.10530 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 20 client.10530 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29eefe0 0x5566e29ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 4 client.10530 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:04:40.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5566e29d8660 con 0x5566e29eefe0 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.253+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29eefe0 0x5566e29ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6a3fff640 10 client.10530 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6c89f8640 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29eefe0 0x5566e29ef3c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc655f94d554185 server_cookie=c1467c4f6fffb293 in_seq=0 out_seq=0 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6a3fff640 1 -- 172.21.15.148:0/1069888809 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5566e29d8660 con 0x5566e29eefe0 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6a3fff640 10 client.10530 handle_command_reply: tid=0 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 client.10530 shutdown 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 msgr2=0x5566e2a08fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e2a08fc0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0047a10 tx=0x7fa6c0004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 shutdown_connections 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x5566e29eefe0 0x5566e29ef3c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566e2a24ad0 0x5566e2a34780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5566e2a081e0 0x5566e2a064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 --2- 172.21.15.148:0/1069888809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566e2a07230 0x5566e2a08fc0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6a3fff640 0 client.10530 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 >> 172.21.15.148:0/1069888809 conn(0x5566e29dcf10 msgr2=0x5566e29dd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 shutdown_connections 2024-04-25T10:04:40.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 1 -- 172.21.15.148:0/1069888809 wait complete. 2024-04-25T10:04:40.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.254+0000 7fa6ce163740 20 client.10530 trim_cache size 0 max 16384 2024-04-25T10:04:40.268 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:04:40.268 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:04:40.268 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:04:40.268 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 10365, 2024-04-25T10:04:40.268 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 8.9108246359999992, 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 93, 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 159, 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 159, 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 13.995390026999999 2024-04-25T10:04:40.269 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa6a806b4d0 msgr2=0x7fa6a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa6a806b4d0 0x7fa6a806d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0033000 tx=0x7fa6c0009000 comp rx=0 tx=0).stop 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 msgr2=0x7fa6c40ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ccb80 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c7f0 tx=0x7fa6b400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 shutdown_connections 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa6a806b4d0 0x7fa6a806d990 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6c41454b0 0x7fa6c41d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6c4144d20 0x7fa6c40ccb80 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 --2- 172.21.15.148:0/664901351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6c4071300 0x7fa6c41f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 >> 172.21.15.148:0/664901351 conn(0x7fa6c40b3530 msgr2=0x7fa6c4144310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 shutdown_connections 2024-04-25T10:04:40.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.255+0000 7fa6cb45d640 1 -- 172.21.15.148:0/664901351 wait complete. 2024-04-25T10:04:40.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:04:40.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.363+0000 7f2005a6f640 1 Processor -- start 2024-04-25T10:04:40.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.363+0000 7f2005a6f640 1 -- start start 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f2005a6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20000b5c30 0x7f2000168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f2005a6f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000ceee0 con 0x7f20000b6cb0 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f2005a6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000d1d10 con 0x7f200015db70 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f2005a6f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20000d3800 con 0x7f20000b5c30 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f1ffffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20000b5c30 0x7f2000168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f1ffffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20000b5c30 0x7f2000168ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41442/0 (socket says 172.21.15.148:41442) 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.364+0000 7f1ffffff640 1 -- 172.21.15.148:0/3142875325 learned_addr learned my addr 172.21.15.148:0/3142875325 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffffff640 1 -- 172.21.15.148:0/3142875325 >> v1:172.21.15.188:6789/0 conn(0x7f200015db70 legacy=0x7f2000162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffffff640 1 -- 172.21.15.148:0/3142875325 >> v1:172.21.15.148:6789/0 conn(0x7f20000b6cb0 legacy=0x7f20000b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffffff640 1 -- 172.21.15.148:0/3142875325 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000d1ff0 con 0x7f20000b5c30 2024-04-25T10:04:40.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffffff640 1 --2- 172.21.15.148:0/3142875325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b5c30 0x7f2000168ad0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f20000d3980 tx=0x7f1ff802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164b321eec62b27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffe7fc640 1 -- 172.21.15.148:0/3142875325 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8036070 con 0x7f20000b5c30 2024-04-25T10:04:40.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffe7fc640 1 -- 172.21.15.148:0/3142875325 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff8031600 con 0x7f20000b5c30 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.365+0000 7f1ffe7fc640 1 -- 172.21.15.148:0/3142875325 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80319d0 con 0x7f20000b5c30 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 -- 172.21.15.148:0/3142875325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b5c30 msgr2=0x7f2000168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 --2- 172.21.15.148:0/3142875325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b5c30 0x7f2000168ad0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f20000d3980 tx=0x7f1ff802a750 comp rx=0 tx=0).stop 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 -- 172.21.15.148:0/3142875325 shutdown_connections 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 --2- 172.21.15.148:0/3142875325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b5c30 0x7f2000168ad0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 -- 172.21.15.148:0/3142875325 >> 172.21.15.148:0/3142875325 conn(0x7f20000d0520 msgr2=0x7f20000d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 -- 172.21.15.148:0/3142875325 shutdown_connections 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.366+0000 7f2005a6f640 1 -- 172.21.15.148:0/3142875325 wait complete. 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.367+0000 7f2005a6f640 1 Processor -- start 2024-04-25T10:04:40.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.367+0000 7f2005a6f640 1 -- start start 2024-04-25T10:04:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f2005a6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20000b5c30 0x7f20001f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f2005a6f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 0x7f200015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f2005a6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f200015db70 0x7f200015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f2005a6f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000ce530 con 0x7f20000b6cb0 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f2005a6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200015df50 con 0x7f20000b5c30 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f2005a6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000cf1c0 con 0x7f200015db70 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f1ffeffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 0x7f200015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f1ffeffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 0x7f200015e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41444/0 (socket says 172.21.15.148:41444) 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f1ffeffd640 1 -- 172.21.15.148:0/2119502488 learned_addr learned my addr 172.21.15.148:0/2119502488 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f1fff7fe640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20000b5c30 0x7f20001f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.368+0000 7f1ffffff640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f200015db70 0x7f200015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffeffd640 1 -- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f200015db70 msgr2=0x7f200015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffeffd640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f200015db70 0x7f200015e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffeffd640 1 -- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20000b5c30 msgr2=0x7f20001f7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffeffd640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20000b5c30 0x7f20001f7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffeffd640 1 -- 172.21.15.148:0/2119502488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000cad10 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffffff640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f200015db70 0x7f200015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1ffeffd640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 0x7f200015e160 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c400 tx=0x7f1ff400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4017070 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff40052d0 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4003830 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20000caff0 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.369+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20000cd850 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.370+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2000073990 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.370+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff401a070 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.371+0000 7f1fe7fff640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fec06b500 0x7f1fec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.371+0000 7f1fff7fe640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fec06b500 0x7f1fec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.371+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ff4089680 con 0x7f20000b6cb0 2024-04-25T10:04:40.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.371+0000 7f1fff7fe640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fec06b500 0x7f1fec06d9c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.377+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff4057d20 con 0x7f20000b6cb0 2024-04-25T10:04:40.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.583+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20000d1820 con 0x7f20000b6cb0 2024-04-25T10:04:40.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.583+0000 7f1fe7fff640 1 -- 172.21.15.148:0/2119502488 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ff405bdc0 con 0x7f20000b6cb0 2024-04-25T10:04:40.589 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fec06b500 msgr2=0x7f1fec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fec06b500 0x7f1fec06d9c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0009060 comp rx=0 tx=0).stop 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 msgr2=0x7f200015e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 0x7f200015e160 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c400 tx=0x7f1ff400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 shutdown_connections 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fec06b500 0x7f1fec06d9c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f200015db70 0x7f200015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20000b6cb0 0x7f200015e160 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 --2- 172.21.15.148:0/2119502488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20000b5c30 0x7f20001f7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 >> 172.21.15.148:0/2119502488 conn(0x7f20000d0520 msgr2=0x7f200015cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 shutdown_connections 2024-04-25T10:04:40.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:40.585+0000 7f2005a6f640 1 -- 172.21.15.148:0/2119502488 wait complete. 2024-04-25T10:04:40.603 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:04:40.603 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:04:40.603 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:04:40.603 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:04:40.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:40.604 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:04:40.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:04:40.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:04:40.604 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:40.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.605 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:04:40.621 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:04:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:04:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:04:40.621 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:40.622 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:04:40.623 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:40.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.624 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:04:40.683 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:04:40.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.685 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:04:40.740 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:04:40.740 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:04:40.740 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:04:40.740 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:04:40.741 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:04:40.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.741 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:04:40.741 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:04:40.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:04:40.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:04:40.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:04:40.741 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.824 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:04:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:04:40.880 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:04:40.880 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:40.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.880 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:04:40.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:40.943 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:40.998 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:40.999 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:40.999 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:41.053 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:04:41.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:41.053 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:41.134 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:41.135 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:41.135 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:04:41.143 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:41.143 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:41.143 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:04:41.143 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:04:41.143 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:04:41.147 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:41.147 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:04:41.148 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:04:41.149 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:04:41.150 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:04:41.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:41.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:41.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:41.215 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:04:41.272 INFO:teuthology.orchestra.run.smithi148.stdout:client7406 2024-04-25T10:04:41.272 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:04:41.272 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:41.327 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:04:41.328 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:04:41.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 Processor -- start 2024-04-25T10:04:41.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 -- start start 2024-04-25T10:04:41.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67e00ced80 0x7f67e00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 0x7f67e00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e015e000 con 0x7f67e015dc20 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00d1de0 con 0x7f67e00ced80 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.461+0000 7f67e6b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0161c60 con 0x7f67e0161710 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67e48b9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f67e015dc20 0x7f67e00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:36950/0 (socket says 172.21.15.148:36950) 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67e48b9640 1 -- 172.21.15.148:0/452737428 learned_addr learned my addr 172.21.15.148:0/452737428 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528584144 0 0) 0x7f67e015e000 con 0x7f67e015dc20 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c8003520 con 0x7f67e015dc20 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67e50ba640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 0x7f67e00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d7fff640 1 --2- 172.21.15.148:0/452737428 >> v2:172.21.15.188:3300/0 conn(0x7f67e00ced80 0x7f67e00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330394331 0 0) 0x7f67c8003520 con 0x7f67e015dc20 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 msgr2=0x7f67e00d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 0x7f67e00d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 >> v2:172.21.15.188:3300/0 conn(0x7f67e00ced80 msgr2=0x7f67e00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 --2- 172.21.15.148:0/452737428 >> v2:172.21.15.188:3300/0 conn(0x7f67e00ced80 0x7f67e00d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0161f40 con 0x7f67e015dc20 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f67dc001670 con 0x7f67e015dc20 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 >> v1:172.21.15.148:6789/0 conn(0x7f67e015dc20 legacy=0x7f67e00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 0x7f67c80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c8003fe0 0x7f67c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00d1de0 con 0x7f67c8004d40 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0161c60 con 0x7f67c8003fe0 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c800bac0 con 0x7f67e0161710 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67e50ba640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67e48b9640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 0x7f67c80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.462+0000 7f67d7fff640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c8003fe0 0x7f67c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67e50ba640 1 -- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 msgr2=0x7f67c80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67e50ba640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e0161710 0x7f67c80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67e50ba640 1 -- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c8003fe0 msgr2=0x7f67c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67e50ba640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67c8003fe0 0x7f67c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67e50ba640 1 -- 172.21.15.148:0/452737428 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8003520 con 0x7f67c8004d40 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67e50ba640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f67d8007360 tx=0x7f67c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc4ad10936d9268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8009be0 con 0x7f67c8004d40 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8036600 con 0x7f67c8004d40 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.463+0000 7f67d77fe640 1 -- 172.21.15.148:0/452737428 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80369d0 con 0x7f67c8004d40 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e6b1d640 1 -- 172.21.15.148:0/452737428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 msgr2=0x7f67c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/452737428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67c80051c0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f67d8007360 tx=0x7f67c8005830 comp rx=0 tx=0).stop 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e50ba640 1 -- 172.21.15.148:0/452737428 reap_dead start 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e6b1d640 1 -- 172.21.15.148:0/452737428 shutdown_connections 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e6b1d640 1 -- 172.21.15.148:0/452737428 >> 172.21.15.148:0/452737428 conn(0x7f67e00d0420 msgr2=0x7f67e00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e6b1d640 1 -- 172.21.15.148:0/452737428 shutdown_connections 2024-04-25T10:04:41.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.464+0000 7f67e6b1d640 1 -- 172.21.15.148:0/452737428 wait complete. 2024-04-25T10:04:41.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.465+0000 7f67e6b1d640 1 Processor -- start 2024-04-25T10:04:41.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.465+0000 7f67e6b1d640 1 -- start start 2024-04-25T10:04:41.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e6b1d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67e014ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e6b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e00cc490 0x7f67e00aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e6b1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67e00ab0b0 0x7f67e015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e6b1d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00b5e50 con 0x7f67c8004d40 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e6b1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0162500 con 0x7f67e00ab0b0 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e6b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00b5fd0 con 0x7f67e00cc490 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67e014ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67e014ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41464/0 (socket says 172.21.15.148:41464) 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 -- 172.21.15.148:0/2954216055 learned_addr learned my addr 172.21.15.148:0/2954216055 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e00cc490 0x7f67e00aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67e00ab0b0 0x7f67e015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 -- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e00cc490 msgr2=0x7f67e00aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e00cc490 0x7f67e00aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 -- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67e00ab0b0 msgr2=0x7f67e015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67e00ab0b0 0x7f67e015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.466+0000 7f67e48b9640 1 -- 172.21.15.148:0/2954216055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00cc920 con 0x7f67c8004d40 2024-04-25T10:04:41.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.467+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67e014ff20 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f67dc007310 tx=0x7f67dc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.467+0000 7f67d57fa640 1 -- 172.21.15.148:0/2954216055 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00f940 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.467+0000 7f67d57fa640 1 -- 172.21.15.148:0/2954216055 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc018070 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.467+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e00ccc00 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.467+0000 7f67d57fa640 1 -- 172.21.15.148:0/2954216055 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00fd40 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.467+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e00cd1c0 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.469+0000 7f67d57fa640 1 -- 172.21.15.148:0/2954216055 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67dc01d460 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.469+0000 7f67d57fa640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67cc06b500 0x7f67cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.469+0000 7f67d57fa640 1 -- 172.21.15.148:0/2954216055 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67dc08a2c0 con 0x7f67c8004d40 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.469+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67cc06b500 0x7f67cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.470+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67cc06b500 0x7f67cc06d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f67e015a2d0 tx=0x7f67d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.470+0000 7f67e981b740 1 Processor -- start 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.470+0000 7f67e981b740 1 -- start start 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da21232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da21237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e981b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 0x562da20f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e981b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da205ce00 con 0x562da20f3700 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da2005d20 con 0x562da2120e80 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da2058ea0 con 0x562da2120aa0 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 0x562da20f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 0x562da20f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41470/0 (socket says 172.21.15.148:41470) 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 -- 172.21.15.148:0/2127667423 learned_addr learned my addr 172.21.15.148:0/2127667423 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da21237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da21232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 -- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 msgr2=0x562da21232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da21232b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 -- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 msgr2=0x562da21237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da21237f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 -- 172.21.15.148:0/2127667423 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da2027df0 con 0x562da20f3700 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 0x562da20f6f10 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f67d8007480 tx=0x7f67d802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155a007fba45fc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67c67fc640 1 -- 172.21.15.148:0/2127667423 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8047070 con 0x562da20f3700 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67c67fc640 1 -- 172.21.15.148:0/2127667423 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8003d50 con 0x562da20f3700 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.471+0000 7f67c67fc640 1 -- 172.21.15.148:0/2127667423 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004240 con 0x562da20f3700 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 -- 172.21.15.148:0/2127667423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 msgr2=0x562da20f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 0x562da20f6f10 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f67d8007480 tx=0x7f67d802f000 comp rx=0 tx=0).stop 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 -- 172.21.15.148:0/2127667423 shutdown_connections 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da20f3700 0x562da20f6f10 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da21237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 --2- 172.21.15.148:0/2127667423 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da21232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 -- 172.21.15.148:0/2127667423 >> 172.21.15.148:0/2127667423 conn(0x562da20f5970 msgr2=0x562da213e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 -- 172.21.15.148:0/2127667423 shutdown_connections 2024-04-25T10:04:41.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 -- 172.21.15.148:0/2127667423 wait complete. 2024-04-25T10:04:41.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.472+0000 7f67e981b740 1 -- start start 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 Processor -- start 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da2139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2141f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2119660 0x562da2119b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da20f8b20 con 0x562da2120e80 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da1ffb590 con 0x562da2119660 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da20f8e00 con 0x562da2120aa0 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.473+0000 7f67e981b740 10 client.0 _ll_register_callbacks cb 0x562da2007e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41482/0 (socket says 172.21.15.148:41482) 2024-04-25T10:04:41.487 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 -- 172.21.15.148:0/2938042186 learned_addr learned my addr 172.21.15.148:0/2938042186 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da2139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2119660 0x562da2119b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 -- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 msgr2=0x562da2139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da2139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 -- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2119660 msgr2=0x562da2119b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2119660 0x562da2119b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 -- 172.21.15.148:0/2938042186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562da1d713f0 con 0x562da2120e80 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2119660 0x562da2119b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2141f20 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f67d00156a0 tx=0x7f67d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dafdfb83457231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67bf7fe640 1 -- 172.21.15.148:0/2938042186 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0023700 con 0x562da2120e80 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67bf7fe640 1 -- 172.21.15.148:0/2938042186 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67d002cc40 con 0x562da2120e80 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67d8003d50 con 0x562da2120e80 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67bf7fe640 1 -- 172.21.15.148:0/2938042186 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d002ce40 con 0x562da2120e80 2024-04-25T10:04:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67e981b740 10 client.10551 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x562da1e3b7d0 con 0x562da2120e80 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.474+0000 7f67bf7fe640 1 -- 172.21.15.148:0/2938042186 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8003d50 con 0x562da2120e80 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67bf7fe640 1 -- 172.21.15.148:0/2938042186 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67d00238a0 con 0x562da2120e80 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e981b740 10 client.10551 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e981b740 10 client.10551 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e981b740 20 client.10551 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e981b740 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da213d650 0x562da213f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e981b740 4 client.10551 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562da206a340 con 0x562da213d650 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da213d650 0x562da213f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67bf7fe640 10 client.10551 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.475+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da213d650 0x562da213f3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9dc148dd85aff7 server_cookie=1bc51f89e61f1a07 in_seq=0 out_seq=0 2024-04-25T10:04:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67bf7fe640 1 -- 172.21.15.148:0/2938042186 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562da206a340 con 0x562da213d650 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67bf7fe640 10 client.10551 handle_command_reply: tid=0 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 client.10551 shutdown 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 msgr2=0x562da2141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2141f20 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f67d00156a0 tx=0x7f67d0004730 comp rx=0 tx=0).stop 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 shutdown_connections 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da213d650 0x562da213f3e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2119660 0x562da2119b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2141f20 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 --2- 172.21.15.148:0/2938042186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da2139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67bf7fe640 0 client.10551 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 >> 172.21.15.148:0/2938042186 conn(0x562da20f3700 msgr2=0x562da211b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 shutdown_connections 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 1 -- 172.21.15.148:0/2938042186 wait complete. 2024-04-25T10:04:41.490 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.476+0000 7f67e981b740 20 client.10551 trim_cache size 0 max 16384 2024-04-25T10:04:41.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.487+0000 7f67e981b740 1 Processor -- start 2024-04-25T10:04:41.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 -- start start 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2119660 0x562da20f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120aa0 0x562da2123690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0200a60 con 0x562da2120e80 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01ffc60 con 0x562da2120aa0 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01ff090 con 0x562da2119660 2024-04-25T10:04:41.496 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e50ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e50ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41488/0 (socket says 172.21.15.148:41488) 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.488+0000 7f67e50ba640 1 -- 172.21.15.148:0/2809582446 learned_addr learned my addr 172.21.15.148:0/2809582446 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e50ba640 1 -- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2119660 msgr2=0x562da20f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e48b9640 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2119660 0x562da20f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120aa0 0x562da2123690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2119660 0x562da20f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e50ba640 1 -- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120aa0 msgr2=0x562da2123690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120aa0 0x562da2123690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e50ba640 1 -- 172.21.15.148:0/2809582446 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da2137da0 con 0x562da2120e80 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67d7fff640 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120aa0 0x562da2123690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e50ba640 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2160bb0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f67d8007480 tx=0x7f67d8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75db3122bb82de41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67c5ffb640 1 -- 172.21.15.148:0/2809582446 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8047040 con 0x562da2120e80 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67c5ffb640 1 -- 172.21.15.148:0/2809582446 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8042540 con 0x562da2120e80 2024-04-25T10:04:41.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67c5ffb640 1 -- 172.21.15.148:0/2809582446 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8042930 con 0x562da2120e80 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 -- 172.21.15.148:0/2809582446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 msgr2=0x562da2160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2160bb0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f67d8007480 tx=0x7f67d8030000 comp rx=0 tx=0).stop 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 -- 172.21.15.148:0/2809582446 shutdown_connections 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2120e80 0x562da2160bb0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120aa0 0x562da2123690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 --2- 172.21.15.148:0/2809582446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2119660 0x562da20f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 -- 172.21.15.148:0/2809582446 >> 172.21.15.148:0/2809582446 conn(0x562da20f3700 msgr2=0x562da20f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.489+0000 7f67e981b740 1 -- 172.21.15.148:0/2809582446 shutdown_connections 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.490+0000 7f67e981b740 1 -- 172.21.15.148:0/2809582446 wait complete. 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.490+0000 7f67e981b740 1 -- start start 2024-04-25T10:04:41.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.490+0000 7f67e981b740 1 Processor -- start 2024-04-25T10:04:41.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 0x562da213b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da213bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da214cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da206a340 con 0x562da2119660 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01ff660 con 0x562da2120e80 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562da20faa00 con 0x562da2120aa0 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e981b740 10 client.0 _ll_register_callbacks cb 0x7f67c80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 0x562da213b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 0x562da213b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41500/0 (socket says 172.21.15.148:41500) 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 -- 172.21.15.148:0/1690684511 learned_addr learned my addr 172.21.15.148:0/1690684511 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e50ba640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da214cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67d7fff640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da213bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 -- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 msgr2=0x562da213bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da213bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 -- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 msgr2=0x562da214cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da214cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 -- 172.21.15.148:0/1690684511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562da211f000 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67d7fff640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da213bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67e48b9640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 0x562da213b760 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f67dc054dc0 tx=0x7f67dc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5ecd3fc273ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.491+0000 7f67bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67bf7fe640 1 -- 172.21.15.148:0/1690684511 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc056c60 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67bf7fe640 1 -- 172.21.15.148:0/1690684511 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67dc054fd0 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67d8042540 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67bf7fe640 1 -- 172.21.15.148:0/1690684511 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67dc075050 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 10 client.10557 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x562da213f9e0 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67bf7fe640 1 -- 172.21.15.148:0/1690684511 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8042540 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67bf7fe640 1 -- 172.21.15.148:0/1690684511 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67dc0772e0 con 0x562da2119660 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 10 client.10557 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 10 client.10557 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:04:41.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 20 client.10557 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da20ef1b0 0x562da214d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 4 client.10557 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562da2064690 con 0x562da20ef1b0 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67d7fff640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da20ef1b0 0x562da214d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.492+0000 7f67bf7fe640 10 client.10557 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67d7fff640 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da20ef1b0 0x562da214d140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba9e79eb0a7253e server_cookie=4c45fb5edbdd9b4c in_seq=0 out_seq=0 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67bf7fe640 1 -- 172.21.15.148:0/1690684511 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562da2064690 con 0x562da20ef1b0 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67bf7fe640 10 client.10557 handle_command_reply: tid=0 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 client.10557 shutdown 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 msgr2=0x562da213b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 0x562da213b760 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f67dc054dc0 tx=0x7f67dc015040 comp rx=0 tx=0).stop 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 shutdown_connections 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562da20ef1b0 0x562da214d140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562da2120e80 0x562da214cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562da2120aa0 0x562da213bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67bf7fe640 0 client.10557 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 --2- 172.21.15.148:0/1690684511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da2119660 0x562da213b760 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 >> 172.21.15.148:0/1690684511 conn(0x562da20f3700 msgr2=0x562da20f26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 shutdown_connections 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 1 -- 172.21.15.148:0/1690684511 wait complete. 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e981b740 20 client.10557 trim_cache size 0 max 16384 2024-04-25T10:04:41.506 INFO:teuthology.orchestra.run.smithi148.stdout:[] 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67cc06b500 msgr2=0x7f67cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67cc06b500 0x7f67cc06d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f67e015a2d0 tx=0x7f67d0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.493+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 msgr2=0x7f67e014ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67e014ff20 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f67dc007310 tx=0x7f67dc0077d0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 shutdown_connections 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67cc06b500 0x7f67cc06d9c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67e00ab0b0 0x7f67e015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67e00cc490 0x7f67e00aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 --2- 172.21.15.148:0/2954216055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c8004d40 0x7f67e014ff20 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 >> 172.21.15.148:0/2954216055 conn(0x7f67e00d0420 msgr2=0x7f67e00b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 shutdown_connections 2024-04-25T10:04:41.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.494+0000 7f67e6b1d640 1 -- 172.21.15.148:0/2954216055 wait complete. 2024-04-25T10:04:41.511 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:04:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.603+0000 7fef03b70640 1 Processor -- start 2024-04-25T10:04:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.603+0000 7fef03b70640 1 -- start start 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef03b70640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feefc154ee0 0x7feefc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef03b70640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feefc0cfcf0 0x7feefc0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef03b70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc0ce3c0 0x7feefc15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef03b70640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0ce830 con 0x7feefc154ee0 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef03b70640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc072090 con 0x7feefc0cfcf0 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef03b70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc071230 con 0x7feefc0ce3c0 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feefc154ee0 0x7feefc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feefc154ee0 0x7feefc0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41514/0 (socket says 172.21.15.148:41514) 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 -- 172.21.15.148:0/362372085 learned_addr learned my addr 172.21.15.148:0/362372085 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0110b640 1 --2- 172.21.15.148:0/362372085 >> v2:172.21.15.188:3300/0 conn(0x7feefc0cfcf0 0x7feefc0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0210d640 1 --2- 172.21.15.148:0/362372085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc0ce3c0 0x7feefc15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 -- 172.21.15.148:0/362372085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc0ce3c0 msgr2=0x7feefc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 --2- 172.21.15.148:0/362372085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc0ce3c0 0x7feefc15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 -- 172.21.15.148:0/362372085 >> v2:172.21.15.188:3300/0 conn(0x7feefc0cfcf0 msgr2=0x7feefc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 --2- 172.21.15.148:0/362372085 >> v2:172.21.15.188:3300/0 conn(0x7feefc0cfcf0 0x7feefc0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.604+0000 7fef0190c640 1 -- 172.21.15.148:0/362372085 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc071480 con 0x7feefc154ee0 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.605+0000 7fef0190c640 1 --2- 172.21.15.148:0/362372085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc154ee0 0x7feefc0cf7b0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7feefc0cf560 tx=0x7feef802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9866a7ec95c3f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.605+0000 7fef0090a640 1 -- 172.21.15.148:0/362372085 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8037070 con 0x7feefc154ee0 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.605+0000 7fef0090a640 1 -- 172.21.15.148:0/362372085 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef8032440 con 0x7feefc154ee0 2024-04-25T10:04:41.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.605+0000 7fef0090a640 1 -- 172.21.15.148:0/362372085 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8032810 con 0x7feefc154ee0 2024-04-25T10:04:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 -- 172.21.15.148:0/362372085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc154ee0 msgr2=0x7feefc0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 --2- 172.21.15.148:0/362372085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc154ee0 0x7feefc0cf7b0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7feefc0cf560 tx=0x7feef802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 -- 172.21.15.148:0/362372085 shutdown_connections 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 --2- 172.21.15.148:0/362372085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc0ce3c0 0x7feefc15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 --2- 172.21.15.148:0/362372085 >> v2:172.21.15.188:3300/0 conn(0x7feefc0cfcf0 0x7feefc0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 --2- 172.21.15.148:0/362372085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc154ee0 0x7feefc0cf7b0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 -- 172.21.15.148:0/362372085 >> 172.21.15.148:0/362372085 conn(0x7feefc06abe0 msgr2=0x7feefc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 -- 172.21.15.148:0/362372085 shutdown_connections 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.606+0000 7fef03b70640 1 -- 172.21.15.148:0/362372085 wait complete. 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.607+0000 7fef03b70640 1 Processor -- start 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.607+0000 7fef03b70640 1 -- start start 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 0x7feefc1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feefc0cfcf0 0x7feefc1417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc154ee0 0x7feefc13fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc071d20 con 0x7feefc0ce3c0 2024-04-25T10:04:41.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc071a40 con 0x7feefc0cfcf0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1552c0 con 0x7feefc154ee0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 0x7feefc1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 0x7feefc1562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41520/0 (socket says 172.21.15.148:41520) 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 -- 172.21.15.148:0/3236857084 learned_addr learned my addr 172.21.15.148:0/3236857084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0110b640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feefc0cfcf0 0x7feefc1417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0210d640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc154ee0 0x7feefc13fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 -- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc154ee0 msgr2=0x7feefc13fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc154ee0 0x7feefc13fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 -- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feefc0cfcf0 msgr2=0x7feefc1417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feefc0cfcf0 0x7feefc1417f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 -- 172.21.15.148:0/3236857084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc140750 con 0x7feefc0ce3c0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef0190c640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 0x7feefc1562d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7feef8000c00 tx=0x7feef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8041020 con 0x7feefc0ce3c0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef803d620 con 0x7feefc0ce3c0 2024-04-25T10:04:41.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc140a30 con 0x7feefc0ce3c0 2024-04-25T10:04:41.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8037040 con 0x7feefc0ce3c0 2024-04-25T10:04:41.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.608+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc140ff0 con 0x7feefc0ce3c0 2024-04-25T10:04:41.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.609+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc073990 con 0x7feefc0ce3c0 2024-04-25T10:04:41.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.610+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feef8044020 con 0x7feefc0ce3c0 2024-04-25T10:04:41.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.610+0000 7feef27fc640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feee006b4d0 0x7feee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.610+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feef80aa980 con 0x7feefc0ce3c0 2024-04-25T10:04:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.610+0000 7fef0110b640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feee006b4d0 0x7feee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.611+0000 7fef0110b640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feee006b4d0 0x7feee006d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feefc06b5b0 tx=0x7feeec008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.615+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feef804b030 con 0x7feefc0ce3c0 2024-04-25T10:04:41.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.823+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feefc06b300 con 0x7feefc0ce3c0 2024-04-25T10:04:41.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.823+0000 7feef27fc640 1 -- 172.21.15.148:0/3236857084 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7feef8079700 con 0x7feefc0ce3c0 2024-04-25T10:04:41.829 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:04:41.830 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:04:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feee006b4d0 msgr2=0x7feee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feee006b4d0 0x7feee006d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feefc06b5b0 tx=0x7feeec008040 comp rx=0 tx=0).stop 2024-04-25T10:04:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 msgr2=0x7feefc1562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 0x7feefc1562d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7feef8000c00 tx=0x7feef80046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 shutdown_connections 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feee006b4d0 0x7feee006d990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feefc154ee0 0x7feefc13fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feefc0cfcf0 0x7feefc1417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 --2- 172.21.15.148:0/3236857084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feefc0ce3c0 0x7feefc1562d0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 >> 172.21.15.148:0/3236857084 conn(0x7feefc06abe0 msgr2=0x7feefc153520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 shutdown_connections 2024-04-25T10:04:41.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.825+0000 7fef03b70640 1 -- 172.21.15.148:0/3236857084 wait complete. 2024-04-25T10:04:41.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:41.842 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start '~mdsdir' recursive 2024-04-25T10:04:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.937+0000 7f77928f7640 1 Processor -- start 2024-04-25T10:04:41.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.937+0000 7f77928f7640 1 -- start start 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.937+0000 7f77928f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f778c070dc0 0x7f778c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.937+0000 7f77928f7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f778c15deb0 0x7f778c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.937+0000 7f77928f7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778c0d1770 con 0x7f778c0711a0 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f77928f7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c0b0eb0 con 0x7f778c15deb0 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f77928f7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c1622b0 con 0x7f778c070dc0 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f778c15deb0 0x7f778c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f778c15deb0 0x7f778c15e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41526/0 (socket says 172.21.15.148:41526) 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 -- 172.21.15.148:0/1245527780 learned_addr learned my addr 172.21.15.148:0/1245527780 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f778bfff640 1 --2- 172.21.15.148:0/1245527780 >> v2:172.21.15.188:3300/0 conn(0x7f778c070dc0 0x7f778c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 -- 172.21.15.148:0/1245527780 >> v2:172.21.15.188:3300/0 conn(0x7f778c070dc0 msgr2=0x7f778c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 --2- 172.21.15.148:0/1245527780 >> v2:172.21.15.188:3300/0 conn(0x7f778c070dc0 0x7f778c06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 -- 172.21.15.148:0/1245527780 >> v1:172.21.15.188:6789/0 conn(0x7f778c0711a0 legacy=0x7f778c0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.938+0000 7f7790e94640 1 -- 172.21.15.148:0/1245527780 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c162740 con 0x7f778c15deb0 2024-04-25T10:04:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f7790e94640 1 --2- 172.21.15.148:0/1245527780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c15e300 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f778c070a00 tx=0x7f778402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e244e968ab6bfb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f778affd640 1 -- 172.21.15.148:0/1245527780 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784036070 con 0x7f778c15deb0 2024-04-25T10:04:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f778affd640 1 -- 172.21.15.148:0/1245527780 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7784031600 con 0x7f778c15deb0 2024-04-25T10:04:41.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f778affd640 1 -- 172.21.15.148:0/1245527780 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77840319d0 con 0x7f778c15deb0 2024-04-25T10:04:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 -- 172.21.15.148:0/1245527780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 msgr2=0x7f778c15e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 --2- 172.21.15.148:0/1245527780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c15e300 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f778c070a00 tx=0x7f778402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 -- 172.21.15.148:0/1245527780 shutdown_connections 2024-04-25T10:04:41.948 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 --2- 172.21.15.148:0/1245527780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c15e300 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 --2- 172.21.15.148:0/1245527780 >> v2:172.21.15.188:3300/0 conn(0x7f778c070dc0 0x7f778c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 -- 172.21.15.148:0/1245527780 >> 172.21.15.148:0/1245527780 conn(0x7f778c0b3530 msgr2=0x7f778c0b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 -- 172.21.15.148:0/1245527780 shutdown_connections 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.939+0000 7f77928f7640 1 -- 172.21.15.148:0/1245527780 wait complete. 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.940+0000 7f77928f7640 1 Processor -- start 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 -- start start 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c070dc0 0x7f778c15f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0711a0 0x7f778c160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c161c60 con 0x7f778c15deb0 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0b10f0 con 0x7f778c070dc0 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c161f40 con 0x7f778c0711a0 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c0cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41536/0 (socket says 172.21.15.148:41536) 2024-04-25T10:04:41.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 -- 172.21.15.148:0/2048079295 learned_addr learned my addr 172.21.15.148:0/2048079295 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 -- 172.21.15.148:0/2048079295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0711a0 msgr2=0x7f778c160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0711a0 0x7f778c160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 -- 172.21.15.148:0/2048079295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c070dc0 msgr2=0x7f778c15f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c070dc0 0x7f778c15f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 -- 172.21.15.148:0/2048079295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c15fca0 con 0x7f778c15deb0 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7790e94640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c0cfe10 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f7784009f40 tx=0x7f7784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7788ff9640 1 -- 172.21.15.148:0/2048079295 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784036050 con 0x7f778c15deb0 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7788ff9640 1 -- 172.21.15.148:0/2048079295 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7784042070 con 0x7f778c15deb0 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c15ff80 con 0x7f778c15deb0 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f7788ff9640 1 -- 172.21.15.148:0/2048079295 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784009510 con 0x7f778c15deb0 2024-04-25T10:04:41.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.942+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c160540 con 0x7f778c15deb0 2024-04-25T10:04:41.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.943+0000 7f7788ff9640 1 -- 172.21.15.148:0/2048079295 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7784007bd0 con 0x7f778c15deb0 2024-04-25T10:04:41.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.944+0000 7f7788ff9640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b4d0 0x7f777806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.944+0000 7f7788ff9640 1 -- 172.21.15.148:0/2048079295 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f77840ab3f0 con 0x7f778c15deb0 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.944+0000 7f778bfff640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b4d0 0x7f777806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.944+0000 7f778bfff640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b4d0 0x7f777806d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7780006fd0 tx=0x7f7780008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.945+0000 7f77955f0740 1 Processor -- start 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.945+0000 7f77955f0740 1 -- start start 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.945+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec054460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f77955f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec024200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec024740 0x55d9ec04c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f77955f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ebf8d0a0 con 0x55d9ec054aa0 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ebf35d40 con 0x55d9ec024740 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9ebf89140 con 0x55d9ec054080 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec024200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec024200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41544/0 (socket says 172.21.15.148:41544) 2024-04-25T10:04:41.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 -- 172.21.15.148:0/1713991658 learned_addr learned my addr 172.21.15.148:0/1713991658 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f7790e94640 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec024740 0x55d9ec04c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 -- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 msgr2=0x55d9ec054460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778bfff640 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec054460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec054460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 -- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec024740 msgr2=0x55d9ec04c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec024740 0x55d9ec04c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 -- 172.21.15.148:0/1713991658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ebd6da20 con 0x55d9ec054aa0 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f7790e94640 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec024740 0x55d9ec04c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec024200 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f777c001630 tx=0x7f777c001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22a6d129617776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f7771ffb640 1 -- 172.21.15.148:0/1713991658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c017070 con 0x55d9ec054aa0 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f7771ffb640 1 -- 172.21.15.148:0/1713991658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00c6c0 con 0x55d9ec054aa0 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.946+0000 7f7771ffb640 1 -- 172.21.15.148:0/1713991658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c001d40 con 0x55d9ec054aa0 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 -- 172.21.15.148:0/1713991658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 msgr2=0x55d9ec024200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec024200 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f777c001630 tx=0x7f777c001af0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 -- 172.21.15.148:0/1713991658 shutdown_connections 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec024740 0x55d9ec04c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec024200 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 --2- 172.21.15.148:0/1713991658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec054460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 -- 172.21.15.148:0/1713991658 >> 172.21.15.148:0/1713991658 conn(0x55d9ec026470 msgr2=0x55d9ec06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 -- 172.21.15.148:0/1713991658 shutdown_connections 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 -- 172.21.15.148:0/1713991658 wait complete. 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.947+0000 7f77955f0740 1 -- start start 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 Processor -- start 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec042620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec047770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec028f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ec0507d0 con 0x55d9ec054aa0 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ebf2b3d0 con 0x55d9ec047cb0 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9ec050a70 con 0x55d9ec054080 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f77955f0740 10 client.0 _ll_register_callbacks cb 0x55d9ebca0e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.948+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec047770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec047770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41558/0 (socket says 172.21.15.148:41558) 2024-04-25T10:04:41.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 -- 172.21.15.148:0/1264126973 learned_addr learned my addr 172.21.15.148:0/1264126973 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f7790e94640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec028f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778bfff640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec042620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 -- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 msgr2=0x55d9ec042620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec042620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 -- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 msgr2=0x55d9ec028f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec028f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 -- 172.21.15.148:0/1264126973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9ebc61310 con 0x55d9ec054aa0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f778b7fe640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec047770 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f777c00c4b0 tx=0x7f777c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241da1c59a5a40f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f776affd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f776affd640 1 -- 172.21.15.148:0/1264126973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c017030 con 0x55d9ec054aa0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f776affd640 1 -- 172.21.15.148:0/1264126973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777c018070 con 0x55d9ec054aa0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f777c00c6c0 con 0x55d9ec054aa0 2024-04-25T10:04:41.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f776affd640 1 -- 172.21.15.148:0/1264126973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f777c02a050 con 0x55d9ec054aa0 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f77955f0740 10 client.10575 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55d9ebdce1b0 con 0x55d9ec054aa0 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f776affd640 1 -- 172.21.15.148:0/1264126973 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00c6c0 con 0x55d9ec054aa0 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f776affd640 1 -- 172.21.15.148:0/1264126973 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f777c01d070 con 0x55d9ec054aa0 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f77955f0740 10 client.10575 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.949+0000 7f77955f0740 10 client.10575 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f77955f0740 20 client.10575 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f77955f0740 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec044180 0x55d9ec044590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f77955f0740 4 client.10575 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9ebf9a5e0 con 0x55d9ec044180 2024-04-25T10:04:41.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f778bfff640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec044180 0x55d9ec044590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f776affd640 10 client.10575 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f778bfff640 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec044180 0x55d9ec044590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5acc63d8d630dc0 server_cookie=b83030c966e800de in_seq=0 out_seq=0 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f776affd640 1 -- 172.21.15.148:0/1264126973 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9ebf9a5e0 con 0x55d9ec044180 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f776affd640 10 client.10575 handle_command_reply: tid=0 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.950+0000 7f77955f0740 1 client.10575 shutdown 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 msgr2=0x55d9ec047770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec047770 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f777c00c4b0 tx=0x7f777c001090 comp rx=0 tx=0).stop 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 shutdown_connections 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec044180 0x55d9ec044590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec028f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054aa0 0x55d9ec047770 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f776affd640 0 client.10575 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 --2- 172.21.15.148:0/1264126973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054080 0x55d9ec042620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 >> 172.21.15.148:0/1264126973 conn(0x55d9ec024740 msgr2=0x55d9ec0722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 shutdown_connections 2024-04-25T10:04:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 1 -- 172.21.15.148:0/1264126973 wait complete. 2024-04-25T10:04:41.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.951+0000 7f77955f0740 20 client.10575 trim_cache size 0 max 16384 2024-04-25T10:04:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.966+0000 7f77955f0740 1 Processor -- start 2024-04-25T10:04:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.966+0000 7f77955f0740 1 -- start start 2024-04-25T10:04:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.966+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec047cb0 0x55d9ec0d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77955f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec051ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec054aa0 0x55d9ec052000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77955f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ec027380 con 0x55d9ec054080 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c200440 con 0x55d9ec054aa0 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1ff680 con 0x55d9ec047cb0 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec051ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec051ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41560/0 (socket says 172.21.15.148:41560) 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 -- 172.21.15.148:0/944591758 learned_addr learned my addr 172.21.15.148:0/944591758 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778bfff640 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec047cb0 0x55d9ec0d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f7790e94640 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec054aa0 0x55d9ec052000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 -- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec047cb0 msgr2=0x55d9ec0d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec047cb0 0x55d9ec0d39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 -- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec054aa0 msgr2=0x55d9ec052000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec054aa0 0x55d9ec052000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 -- 172.21.15.148:0/944591758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ec0292e0 con 0x55d9ec054080 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f7790e94640 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec054aa0 0x55d9ec052000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f778b7fe640 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec051ac0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f777c001df0 tx=0x7f777c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125831eb6f0a450b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77717fa640 1 -- 172.21.15.148:0/944591758 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c017070 con 0x55d9ec054080 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77717fa640 1 -- 172.21.15.148:0/944591758 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00c6c0 con 0x55d9ec054080 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.967+0000 7f77717fa640 1 -- 172.21.15.148:0/944591758 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c007220 con 0x55d9ec054080 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 -- 172.21.15.148:0/944591758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 msgr2=0x55d9ec051ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec051ac0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f777c001df0 tx=0x7f777c01d000 comp rx=0 tx=0).stop 2024-04-25T10:04:41.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 -- 172.21.15.148:0/944591758 shutdown_connections 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec054aa0 0x55d9ec052000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec051ac0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 --2- 172.21.15.148:0/944591758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec047cb0 0x55d9ec0d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 -- 172.21.15.148:0/944591758 >> 172.21.15.148:0/944591758 conn(0x55d9ec024740 msgr2=0x55d9ec027df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 -- 172.21.15.148:0/944591758 shutdown_connections 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 -- 172.21.15.148:0/944591758 wait complete. 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.968+0000 7f77955f0740 1 -- start start 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 Processor -- start 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec088560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec088aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054aa0 0x55d9ec042430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c15ecd0 con 0x55d9ec054080 2024-04-25T10:04:41.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c15e6d0 con 0x55d9ec047cb0 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9ec043540 con 0x55d9ec054aa0 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f77955f0740 10 client.0 _ll_register_callbacks cb 0x7f777c007c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec088aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f778b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec088aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41564/0 (socket says 172.21.15.148:41564) 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.969+0000 7f778b7fe640 1 -- 172.21.15.148:0/4060370898 learned_addr learned my addr 172.21.15.148:0/4060370898 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778b7fe640 1 -- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054aa0 msgr2=0x55d9ec042430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f7790e94640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054aa0 0x55d9ec042430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778bfff640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec088560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778b7fe640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054aa0 0x55d9ec042430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778b7fe640 1 -- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 msgr2=0x55d9ec088560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:41.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778b7fe640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec088560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778b7fe640 1 -- 172.21.15.148:0/4060370898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9ebc3a490 con 0x55d9ec054080 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778bfff640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec088560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778b7fe640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec088aa0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f777c001c20 tx=0x7f777c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d2e8e444275d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f776affd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f776affd640 1 -- 172.21.15.148:0/4060370898 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c03b060 con 0x55d9ec054080 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f776affd640 1 -- 172.21.15.148:0/4060370898 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777c0075e0 con 0x55d9ec054080 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f777c00c6c0 con 0x55d9ec054080 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f776affd640 1 -- 172.21.15.148:0/4060370898 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f777c041050 con 0x55d9ec054080 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 10 client.10581 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55d9ec0d0490 con 0x55d9ec054080 2024-04-25T10:04:41.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f776affd640 1 -- 172.21.15.148:0/4060370898 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c00c6c0 con 0x55d9ec054080 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f776affd640 1 -- 172.21.15.148:0/4060370898 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f777c044020 con 0x55d9ec054080 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 10 client.10581 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 10 client.10581 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 20 client.10581 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec0c9d40 0x55d9ec042970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 4 client.10581 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55d9ebf94930 con 0x55d9ec0c9d40 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.970+0000 7f778bfff640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec0c9d40 0x55d9ec042970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f776affd640 10 client.10581 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f778bfff640 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec0c9d40 0x55d9ec042970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f416be68321ce15 server_cookie=3fb66269bf15754b in_seq=0 out_seq=0 2024-04-25T10:04:41.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f776affd640 1 -- 172.21.15.148:0/4060370898 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d9ebf94930 con 0x55d9ec0c9d40 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f776affd640 10 client.10581 handle_command_reply: tid=0 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f77955f0740 1 client.10581 shutdown 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 msgr2=0x55d9ec088aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f77955f0740 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec088aa0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f777c001c20 tx=0x7f777c007df0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 shutdown_connections 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.971+0000 7f77955f0740 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55d9ec0c9d40 0x55d9ec042970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d9ec054aa0 0x55d9ec042430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d9ec054080 0x55d9ec088aa0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 1 --2- 172.21.15.148:0/4060370898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9ec047cb0 0x55d9ec088560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 >> 172.21.15.148:0/4060370898 conn(0x55d9ec024740 msgr2=0x55d9ec051a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f776affd640 0 client.10581 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:41.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 shutdown_connections 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 1 -- 172.21.15.148:0/4060370898 wait complete. 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77955f0740 20 client.10581 trim_cache size 0 max 16384 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "704e8500-6e8b-4b7d-9a85-297e2d5761e1", 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:04:41.988 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:04:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b4d0 msgr2=0x7f777806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b4d0 0x7f777806d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7780006fd0 tx=0x7f7780008040 comp rx=0 tx=0).stop 2024-04-25T10:04:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 msgr2=0x7f778c0cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c0cfe10 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f7784009f40 tx=0x7f7784004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:41.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 shutdown_connections 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f777806b4d0 0x7f777806d990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f778c15deb0 0x7f778c0cfe10 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f778c0711a0 0x7f778c160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 --2- 172.21.15.148:0/2048079295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f778c070dc0 0x7f778c15f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 >> 172.21.15.148:0/2048079295 conn(0x7f778c0b3530 msgr2=0x7f778c06ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 shutdown_connections 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:41.972+0000 7f77928f7640 1 -- 172.21.15.148:0/2048079295 wait complete. 2024-04-25T10:04:41.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:41.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:04:42.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 Processor -- start 2024-04-25T10:04:42.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 -- start start 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf1015dc20 0x7fcf100ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf100d32f0 0x7fcf100cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1015e000 con 0x7fcf10161710 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf100d1de0 con 0x7fcf1015dc20 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1859e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10161c60 con 0x7fcf100d32f0 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1633a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf1015dc20 0x7fcf100ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1633a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf1015dc20 0x7fcf100ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41580/0 (socket says 172.21.15.148:41580) 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.085+0000 7fcf1633a640 1 -- 172.21.15.148:0/1204286520 learned_addr learned my addr 172.21.15.148:0/1204286520 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf100d32f0 0x7fcf100cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf1633a640 1 -- 172.21.15.148:0/1204286520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf100d32f0 msgr2=0x7fcf100cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf1633a640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf100d32f0 0x7fcf100cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf1633a640 1 -- 172.21.15.148:0/1204286520 >> v1:172.21.15.188:6789/0 conn(0x7fcf10161710 legacy=0x7fcf1006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf1633a640 1 -- 172.21.15.148:0/1204286520 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10161f40 con 0x7fcf1015dc20 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf100d32f0 0x7fcf100cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf1633a640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf1015dc20 0x7fcf100ce670 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009e50 tx=0x7fcf0c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffe94b365b23c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.086+0000 7fcf15338640 1 -- 172.21.15.148:0/1204286520 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c036070 con 0x7fcf1015dc20 2024-04-25T10:04:42.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf15338640 1 -- 172.21.15.148:0/1204286520 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c031600 con 0x7fcf1015dc20 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf15338640 1 -- 172.21.15.148:0/1204286520 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0319d0 con 0x7fcf1015dc20 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 -- 172.21.15.148:0/1204286520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf1015dc20 msgr2=0x7fcf100ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf1015dc20 0x7fcf100ce670 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009e50 tx=0x7fcf0c02a710 comp rx=0 tx=0).stop 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 -- 172.21.15.148:0/1204286520 shutdown_connections 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf100d32f0 0x7fcf100cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 --2- 172.21.15.148:0/1204286520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf1015dc20 0x7fcf100ce670 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 -- 172.21.15.148:0/1204286520 >> 172.21.15.148:0/1204286520 conn(0x7fcf100d0420 msgr2=0x7fcf100d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 -- 172.21.15.148:0/1204286520 shutdown_connections 2024-04-25T10:04:42.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.087+0000 7fcf1859e640 1 -- 172.21.15.148:0/1204286520 wait complete. 2024-04-25T10:04:42.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.088+0000 7fcf1859e640 1 Processor -- start 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.088+0000 7fcf1859e640 1 -- start start 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1859e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 0x7fcf101f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1859e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf1015dc20 0x7fcf101f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1859e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf10161710 0x7fcf101e9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1859e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100b5e50 con 0x7fcf100d32f0 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1859e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10162500 con 0x7fcf10161710 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1859e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100b5fd0 con 0x7fcf1015dc20 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 0x7fcf101f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 0x7fcf101f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41590/0 (socket says 172.21.15.148:41590) 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 -- 172.21.15.148:0/915050084 learned_addr learned my addr 172.21.15.148:0/915050084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:42.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf15b39640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf1015dc20 0x7fcf101f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf10161710 0x7fcf101e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 -- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf1015dc20 msgr2=0x7fcf101f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf1015dc20 0x7fcf101f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 -- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf10161710 msgr2=0x7fcf101e9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf10161710 0x7fcf101e9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.089+0000 7fcf1633a640 1 -- 172.21.15.148:0/915050084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1015e270 con 0x7fcf100d32f0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.090+0000 7fcf1633a640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 0x7fcf101f3aa0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009f70 tx=0x7fcf0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.090+0000 7fcf06ffd640 1 -- 172.21.15.148:0/915050084 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c03f020 con 0x7fcf100d32f0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.090+0000 7fcf06ffd640 1 -- 172.21.15.148:0/915050084 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c037040 con 0x7fcf100d32f0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.090+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf101f6dd0 con 0x7fcf100d32f0 2024-04-25T10:04:42.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.090+0000 7fcf06ffd640 1 -- 172.21.15.148:0/915050084 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c036070 con 0x7fcf100d32f0 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.090+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1015e7c0 con 0x7fcf100d32f0 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.091+0000 7fcf06ffd640 1 -- 172.21.15.148:0/915050084 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf0c042020 con 0x7fcf100d32f0 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.092+0000 7fcf06ffd640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcef406b500 0x7fcef406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.092+0000 7fcf06ffd640 1 -- 172.21.15.148:0/915050084 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf0c0aa7d0 con 0x7fcf100d32f0 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.092+0000 7fcf15b39640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcef406b500 0x7fcef406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.092+0000 7fcf15b39640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcef406b500 0x7fcef406d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcf00007fd0 tx=0x7fcf00009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.093+0000 7fcf1b37e740 1 Processor -- start 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.093+0000 7fcf1b37e740 1 -- start start 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e7006a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70067ef0 0x560e7006a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e7003a770 0x560e7003df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ffa3da0 con 0x560e70067b10 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ff4cb80 con 0x560e70067ef0 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e6ff9fe40 con 0x560e7003a770 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e7006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e7006a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41594/0 (socket says 172.21.15.148:41594) 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 -- 172.21.15.148:0/927843782 learned_addr learned my addr 172.21.15.148:0/927843782 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf15b39640 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70067ef0 0x560e7006a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e7003a770 0x560e7003df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 -- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e7003a770 msgr2=0x560e7003df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e7003a770 0x560e7003df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 -- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70067ef0 msgr2=0x560e7006a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70067ef0 0x560e7006a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 -- 172.21.15.148:0/927843782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e6ff56f10 con 0x560e70067b10 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf15b39640 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70067ef0 0x560e7006a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcf1633a640 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e7006a320 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c034040 tx=0x7fcf0c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05a81cc2a979aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcef3fff640 1 -- 172.21.15.148:0/927843782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c078e50 con 0x560e70067b10 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcef3fff640 1 -- 172.21.15.148:0/927843782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c07b070 con 0x560e70067b10 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.094+0000 7fcef3fff640 1 -- 172.21.15.148:0/927843782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c07cfa0 con 0x560e70067b10 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 -- 172.21.15.148:0/927843782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 msgr2=0x560e7006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e7006a320 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c034040 tx=0x7fcf0c009810 comp rx=0 tx=0).stop 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 -- 172.21.15.148:0/927843782 shutdown_connections 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e7003a770 0x560e7003df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70067ef0 0x560e7006a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/927843782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e7006a320 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 -- 172.21.15.148:0/927843782 >> 172.21.15.148:0/927843782 conn(0x560e7003c9e0 msgr2=0x560e70085320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 -- 172.21.15.148:0/927843782 shutdown_connections 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.095+0000 7fcf1b37e740 1 -- 172.21.15.148:0/927843782 wait complete. 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.096+0000 7fcf1b37e740 1 -- start start 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.096+0000 7fcf1b37e740 1 Processor -- start 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e7005eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e70086450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70062490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ffa3da0 con 0x560e70067ef0 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ff424b0 con 0x560e70062010 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e7003fb90 con 0x560e70067b10 2024-04-25T10:04:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 10 client.0 _ll_register_callbacks cb 0x560e6ff82180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e70086450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e70086450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41596/0 (socket says 172.21.15.148:41596) 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 -- 172.21.15.148:0/95177063 learned_addr learned my addr 172.21.15.148:0/95177063 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1633a640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e7005eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70062490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 -- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 msgr2=0x560e7005eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e7005eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 -- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 msgr2=0x560e70062490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70062490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 -- 172.21.15.148:0/95177063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e6fd827d0 con 0x560e70067ef0 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70062490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:42.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf15b39640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e70086450 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fcf00003920 tx=0x7fcf00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88b5d3128fc0f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcef17fa640 1 -- 172.21.15.148:0/95177063 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00027060 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcef17fa640 1 -- 172.21.15.148:0/95177063 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf00010040 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.097+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf0c07b070 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcef17fa640 1 -- 172.21.15.148:0/95177063 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf00015110 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 10 client.10593 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x560e6fcb83f0 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcef17fa640 1 -- 172.21.15.148:0/95177063 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c07b070 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcef17fa640 1 -- 172.21.15.148:0/95177063 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf00015bd0 con 0x560e70067ef0 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 10 client.10593 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 10 client.10593 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 20 client.10593 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:42.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e70080fa0 0x560e700a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 4 client.10593 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e6ffb12e0 con 0x560e70080fa0 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1633a640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e70080fa0 0x560e700a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcef17fa640 10 client.10593 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.098+0000 7fcf1633a640 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e70080fa0 0x560e700a24f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473ced3645008347 server_cookie=365a874c0967bb05 in_seq=0 out_seq=0 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcef17fa640 1 -- 172.21.15.148:0/95177063 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e6ffb12e0 con 0x560e70080fa0 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcef17fa640 10 client.10593 handle_command_reply: tid=0 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 client.10593 shutdown 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 msgr2=0x560e70086450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e70086450 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fcf00003920 tx=0x7fcf00000bc0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 shutdown_connections 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e70080fa0 0x560e700a24f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70062490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e70086450 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcef17fa640 0 client.10593 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/95177063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e7005eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 >> 172.21.15.148:0/95177063 conn(0x560e7003a770 msgr2=0x560e7006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 shutdown_connections 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 1 -- 172.21.15.148:0/95177063 wait complete. 2024-04-25T10:04:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.099+0000 7fcf1b37e740 20 client.10593 trim_cache size 0 max 16384 2024-04-25T10:04:42.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.114+0000 7fcf1b37e740 1 Processor -- start 2024-04-25T10:04:42.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.114+0000 7fcf1b37e740 1 -- start start 2024-04-25T10:04:42.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70056190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e70035fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e7003fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10200760 con 0x560e70067ef0 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf101ff900 con 0x560e70062010 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1015f950 con 0x560e70067b10 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e7003fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e7003fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41598/0 (socket says 172.21.15.148:41598) 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 -- 172.21.15.148:0/3383715606 learned_addr learned my addr 172.21.15.148:0/3383715606 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:42.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 -- 172.21.15.148:0/3383715606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 msgr2=0x560e70035fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e70035fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 -- 172.21.15.148:0/3383715606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 msgr2=0x560e70056190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70056190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 -- 172.21.15.148:0/3383715606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e70062800 con 0x560e70067ef0 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcf16b3b640 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e7003fc20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fcf08003720 tx=0x7fcf08003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c45150f65c059d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcef37fe640 1 -- 172.21.15.148:0/3383715606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0801a030 con 0x560e70067ef0 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.115+0000 7fcef37fe640 1 -- 172.21.15.148:0/3383715606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf08003e30 con 0x560e70067ef0 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcef37fe640 1 -- 172.21.15.148:0/3383715606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08013840 con 0x560e70067ef0 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 -- 172.21.15.148:0/3383715606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 msgr2=0x560e7003fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e7003fc20 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fcf08003720 tx=0x7fcf08003be0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 -- 172.21.15.148:0/3383715606 shutdown_connections 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067ef0 0x560e7003fc20 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067b10 0x560e70035fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/3383715606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e70056190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 -- 172.21.15.148:0/3383715606 >> 172.21.15.148:0/3383715606 conn(0x560e7003a770 msgr2=0x560e70041920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 -- 172.21.15.148:0/3383715606 shutdown_connections 2024-04-25T10:04:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.116+0000 7fcf1b37e740 1 -- 172.21.15.148:0/3383715606 wait complete. 2024-04-25T10:04:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.117+0000 7fcf1b37e740 1 -- start start 2024-04-25T10:04:42.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.117+0000 7fcf1b37e740 1 Processor -- start 2024-04-25T10:04:42.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.117+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e700603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.117+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e70060920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.117+0000 7fcf1b37e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067ef0 0x560e7006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.117+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf101ff3a0 con 0x560e70067b10 2024-04-25T10:04:42.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7006dfa0 con 0x560e70062010 2024-04-25T10:04:42.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1b37e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e700398b0 con 0x560e70067ef0 2024-04-25T10:04:42.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1b37e740 10 client.0 _ll_register_callbacks cb 0x7fcf0c0092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:42.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e70060920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e70060920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:41602/0 (socket says 172.21.15.148:41602) 2024-04-25T10:04:42.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 -- 172.21.15.148:0/2527118957 learned_addr learned my addr 172.21.15.148:0/2527118957 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:04:42.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 -- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067ef0 msgr2=0x560e7006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067ef0 0x560e7006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1633a640 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e700603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 -- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 msgr2=0x560e700603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e700603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 -- 172.21.15.148:0/2527118957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e6fc780f0 con 0x560e70067b10 2024-04-25T10:04:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf15b39640 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e70060920 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005620 tx=0x7fcf00000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9bf258e61bf36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:42.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:04:42.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcef17fa640 1 -- 172.21.15.148:0/2527118957 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00015810 con 0x560e70067b10 2024-04-25T10:04:42.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcef17fa640 1 -- 172.21.15.148:0/2527118957 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf0001e070 con 0x560e70067b10 2024-04-25T10:04:42.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf08003e30 con 0x560e70067b10 2024-04-25T10:04:42.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcef17fa640 1 -- 172.21.15.148:0/2527118957 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0003a050 con 0x560e70067b10 2024-04-25T10:04:42.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1b37e740 10 client.10599 fetch_fsmap learned FSMap version 93 2024-04-25T10:04:42.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.118+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x560e700a7190 con 0x560e70067b10 2024-04-25T10:04:42.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcef17fa640 1 -- 172.21.15.148:0/2527118957 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf08003e30 con 0x560e70067b10 2024-04-25T10:04:42.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcef17fa640 1 -- 172.21.15.148:0/2527118957 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf0003d020 con 0x560e70067b10 2024-04-25T10:04:42.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 10 client.10599 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:04:42.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 10 client.10599 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:04:42.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 20 client.10599 populate_metadata read hostname 'smithi148' 2024-04-25T10:04:42.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e7007b8c0 0x560e7006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:42.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 4 client.10599 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:04:42.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e6ffab630 con 0x560e7007b8c0 2024-04-25T10:04:42.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1633a640 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e7007b8c0 0x560e7006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:42.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcef17fa640 10 client.10599 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:42.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1633a640 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e7007b8c0 0x560e7006d4d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682f87e6f04a30e3 server_cookie=a6c1508d201c27ff in_seq=0 out_seq=0 2024-04-25T10:04:42.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcef17fa640 1 -- 172.21.15.148:0/2527118957 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x560e6ffab630 con 0x560e7007b8c0 2024-04-25T10:04:42.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcef17fa640 10 client.10599 handle_command_reply: tid=0 2024-04-25T10:04:42.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 1 client.10599 shutdown 2024-04-25T10:04:42.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 msgr2=0x560e70060920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.119+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e70060920 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005620 tx=0x7fcf00000ed0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 shutdown_connections 2024-04-25T10:04:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x560e7007b8c0 0x560e7006d4d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560e70067ef0 0x560e7006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560e70067b10 0x560e70060920 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 --2- 172.21.15.148:0/2527118957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e70062010 0x560e700603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcef17fa640 0 client.10599 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:04:42.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 >> 172.21.15.148:0/2527118957 conn(0x560e7003a770 msgr2=0x560e700865e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:42.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 shutdown_connections 2024-04-25T10:04:42.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 1 -- 172.21.15.148:0/2527118957 wait complete. 2024-04-25T10:04:42.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1b37e740 20 client.10599 trim_cache size 0 max 16384 2024-04-25T10:04:42.145 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:04:42.145 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:04:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:04:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "704e8500-6e8b-4b7d-9a85-297e2d5761e1": { 2024-04-25T10:04:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "~mds0", 2024-04-25T10:04:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "704e8500-6e8b-4b7d-9a85-297e2d5761e1", 2024-04-25T10:04:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive" 2024-04-25T10:04:42.146 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:42.147 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:04:42.147 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:04:42.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcef406b500 msgr2=0x7fcef406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcef406b500 0x7fcef406d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fcf00007fd0 tx=0x7fcf00009040 comp rx=0 tx=0).stop 2024-04-25T10:04:42.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 msgr2=0x7fcf101f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:42.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 0x7fcf101f3aa0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009f70 tx=0x7fcf0c004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:42.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 shutdown_connections 2024-04-25T10:04:42.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcef406b500 0x7fcef406d9c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf10161710 0x7fcf101e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf1015dc20 0x7fcf101f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 --2- 172.21.15.148:0/915050084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf100d32f0 0x7fcf101f3aa0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:42.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 >> 172.21.15.148:0/915050084 conn(0x7fcf100d0420 msgr2=0x7fcf100d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:42.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 shutdown_connections 2024-04-25T10:04:42.152 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:04:42.120+0000 7fcf1859e640 1 -- 172.21.15.148:0/915050084 wait complete. 2024-04-25T10:04:42.152 INFO:tasks.cephfs.filesystem:scrub status for tag:704e8500-6e8b-4b7d-9a85-297e2d5761e1 - {'path': '~mds0', 'tag': '704e8500-6e8b-4b7d-9a85-297e2d5761e1', 'options': 'recursive'} 2024-04-25T10:04:55.514 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:55.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:12.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:12.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:05:12.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.260+0000 7fd827f7e640 1 Processor -- start 2024-04-25T10:05:12.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.260+0000 7fd827f7e640 1 -- start start 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd827f7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82015dcd0 0x7fd82006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd827f7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd82006e2f0 0x7fd82006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd827f7e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8200b3450 con 0x7fd82006b8c0 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd827f7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8200b36f0 con 0x7fd82006e2f0 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd827f7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200b39d0 con 0x7fd82015dcd0 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd82651b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd82006e2f0 0x7fd82006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd82651b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd82006e2f0 0x7fd82006e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47726/0 (socket says 172.21.15.148:47726) 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd82651b640 1 -- 172.21.15.148:0/816330144 learned_addr learned my addr 172.21.15.148:0/816330144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd825d1a640 1 --2- 172.21.15.148:0/816330144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82015dcd0 0x7fd82006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.261+0000 7fd82651b640 1 -- 172.21.15.148:0/816330144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82015dcd0 msgr2=0x7fd82006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd82651b640 1 --2- 172.21.15.148:0/816330144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82015dcd0 0x7fd82006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd82651b640 1 -- 172.21.15.148:0/816330144 >> v1:172.21.15.188:6789/0 conn(0x7fd82006b8c0 legacy=0x7fd8200b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd82651b640 1 -- 172.21.15.148:0/816330144 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200b3cb0 con 0x7fd82006e2f0 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd82651b640 1 --2- 172.21.15.148:0/816330144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82006e780 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fd818009870 tx=0x7fd81802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9179dda96da85b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd824d18640 1 -- 172.21.15.148:0/816330144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818037070 con 0x7fd82006e2f0 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd824d18640 1 -- 172.21.15.148:0/816330144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818032440 con 0x7fd82006e2f0 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.262+0000 7fd824d18640 1 -- 172.21.15.148:0/816330144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818032810 con 0x7fd82006e2f0 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 -- 172.21.15.148:0/816330144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 msgr2=0x7fd82006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 --2- 172.21.15.148:0/816330144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82006e780 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fd818009870 tx=0x7fd81802a210 comp rx=0 tx=0).stop 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 -- 172.21.15.148:0/816330144 shutdown_connections 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 --2- 172.21.15.148:0/816330144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82006e780 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 --2- 172.21.15.148:0/816330144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82015dcd0 0x7fd82006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 -- 172.21.15.148:0/816330144 >> 172.21.15.148:0/816330144 conn(0x7fd82006abe0 msgr2=0x7fd82006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 -- 172.21.15.148:0/816330144 shutdown_connections 2024-04-25T10:05:12.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.263+0000 7fd827f7e640 1 -- 172.21.15.148:0/816330144 wait complete. 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.264+0000 7fd827f7e640 1 Processor -- start 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.264+0000 7fd827f7e640 1 -- start start 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd827f7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82006b8c0 0x7fd82015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd827f7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd827f7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82015dcd0 0x7fd8200cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd827f7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82006dc20 con 0x7fd82006e2f0 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd827f7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200b4bc0 con 0x7fd82015dcd0 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd827f7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82006dda0 con 0x7fd82006b8c0 2024-04-25T10:05:12.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47736/0 (socket says 172.21.15.148:47736) 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 -- 172.21.15.148:0/3089273046 learned_addr learned my addr 172.21.15.148:0/3089273046 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825d1a640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82006b8c0 0x7fd82015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd82651b640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82015dcd0 0x7fd8200cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 -- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82006b8c0 msgr2=0x7fd82015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82006b8c0 0x7fd82015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 -- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82015dcd0 msgr2=0x7fd8200cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82015dcd0 0x7fd8200cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd825519640 1 -- 172.21.15.148:0/3089273046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200cf8b0 con 0x7fd82006e2f0 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.265+0000 7fd82651b640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82015dcd0 0x7fd8200cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.266+0000 7fd825519640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82015cad0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd81000c430 tx=0x7fd81000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.266+0000 7fd8167fc640 1 -- 172.21.15.148:0/3089273046 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011790 con 0x7fd82006e2f0 2024-04-25T10:05:12.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.266+0000 7fd8167fc640 1 -- 172.21.15.148:0/3089273046 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810017070 con 0x7fd82006e2f0 2024-04-25T10:05:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.266+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82015ba30 con 0x7fd82006e2f0 2024-04-25T10:05:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.266+0000 7fd8167fc640 1 -- 172.21.15.148:0/3089273046 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011b70 con 0x7fd82006e2f0 2024-04-25T10:05:12.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.266+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82015bff0 con 0x7fd82006e2f0 2024-04-25T10:05:12.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.267+0000 7fd8167fc640 1 -- 172.21.15.148:0/3089273046 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd810011d10 con 0x7fd82006e2f0 2024-04-25T10:05:12.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.268+0000 7fd8167fc640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd80406b4d0 0x7fd80406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.268+0000 7fd8167fc640 1 -- 172.21.15.148:0/3089273046 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd81008a100 con 0x7fd82006e2f0 2024-04-25T10:05:12.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.268+0000 7fd825d1a640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd80406b4d0 0x7fd80406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.268+0000 7fd825d1a640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd80406b4d0 0x7fd80406d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0099c0 tx=0x7fd81c003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.269+0000 7fd82a922740 1 Processor -- start 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.269+0000 7fd82a922740 1 -- start start 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563ee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82a922740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d563ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563be770 0x562d563c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82a922740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d56327da0 con 0x562d563ebef0 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d562d0b80 con 0x562d563be770 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d56323e40 con 0x562d563ebb10 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d563ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d563ee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47738/0 (socket says 172.21.15.148:47738) 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 -- 172.21.15.148:0/229275162 learned_addr learned my addr 172.21.15.148:0/229275162 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825d1a640 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd82651b640 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563be770 0x562d563c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 -- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 msgr2=0x562d563ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563ee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 -- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563be770 msgr2=0x562d563c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563be770 0x562d563c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 -- 172.21.15.148:0/229275162 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d562daf10 con 0x562d563ebef0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd825519640 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d563ee860 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fd8100037a0 tx=0x7fd810058fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5626334a734b5200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd8037fe640 1 -- 172.21.15.148:0/229275162 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810070070 con 0x562d563ebef0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd8037fe640 1 -- 172.21.15.148:0/229275162 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810060e90 con 0x562d563ebef0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.270+0000 7fd8037fe640 1 -- 172.21.15.148:0/229275162 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810070210 con 0x562d563ebef0 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 -- 172.21.15.148:0/229275162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 msgr2=0x562d563ee860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d563ee860 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fd8100037a0 tx=0x7fd810058fc0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 -- 172.21.15.148:0/229275162 shutdown_connections 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563be770 0x562d563c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d563ee860 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 --2- 172.21.15.148:0/229275162 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563ee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 -- 172.21.15.148:0/229275162 >> 172.21.15.148:0/229275162 conn(0x562d563c09e0 msgr2=0x562d56409320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 -- 172.21.15.148:0/229275162 shutdown_connections 2024-04-25T10:05:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.271+0000 7fd82a922740 1 -- 172.21.15.148:0/229275162 wait complete. 2024-04-25T10:05:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.272+0000 7fd82a922740 1 -- start start 2024-04-25T10:05:12.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.272+0000 7fd82a922740 1 Processor -- start 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d563eac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56419ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d5640d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d563c3b90 con 0x562d563ebef0 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d562c64b0 con 0x562d563ebb10 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d563c3e70 con 0x562d5640fbc0 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 10 client.0 _ll_register_callbacks cb 0x562d56306180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56419ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56419ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47740/0 (socket says 172.21.15.148:47740) 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 -- 172.21.15.148:0/1289420244 learned_addr learned my addr 172.21.15.148:0/1289420244 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82651b640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d5640d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d563eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 -- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 msgr2=0x562d5640d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d5640d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 -- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 msgr2=0x562d563eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d563eac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 -- 172.21.15.148:0/1289420244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d561067d0 con 0x562d563ebef0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd825519640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56419ad0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fd810058940 tx=0x7fd8100036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29ee7e9529efef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd800ff9640 1 -- 172.21.15.148:0/1289420244 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810085030 con 0x562d563ebef0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd800ff9640 1 -- 172.21.15.148:0/1289420244 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd810071d50 con 0x562d563ebef0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.273+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd810060e90 con 0x562d563ebef0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd800ff9640 1 -- 172.21.15.148:0/1289420244 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd810070040 con 0x562d563ebef0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 10 client.10611 fetch_fsmap learned FSMap version 93 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x562d5603c3f0 con 0x562d563ebef0 2024-04-25T10:05:12.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd800ff9640 1 -- 172.21.15.148:0/1289420244 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810060e90 con 0x562d563ebef0 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd800ff9640 1 -- 172.21.15.148:0/1289420244 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd810054df0 con 0x562d563ebef0 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 10 client.10611 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 10 client.10611 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 20 client.10611 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d5640b970 0x562d5640bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 4 client.10611 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d563352e0 con 0x562d5640b970 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d5640b970 0x562d5640bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d5640b970 0x562d5640bd50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b0adb041f3e079 server_cookie=31a5fce4ca64ff6c in_seq=0 out_seq=0 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.274+0000 7fd800ff9640 10 client.10611 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd800ff9640 1 -- 172.21.15.148:0/1289420244 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d563352e0 con 0x562d5640b970 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd800ff9640 10 client.10611 handle_command_reply: tid=0 2024-04-25T10:05:12.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 client.10611 shutdown 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 msgr2=0x562d56419ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56419ad0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fd810058940 tx=0x7fd8100036f0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 shutdown_connections 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d5640b970 0x562d5640bd50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d5640d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd800ff9640 0 client.10611 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56419ad0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 --2- 172.21.15.148:0/1289420244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d563eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 >> 172.21.15.148:0/1289420244 conn(0x562d563be770 msgr2=0x562d563ee2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 shutdown_connections 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 1 -- 172.21.15.148:0/1289420244 wait complete. 2024-04-25T10:05:12.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.275+0000 7fd82a922740 20 client.10611 trim_cache size 0 max 16384 2024-04-25T10:05:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.290+0000 7fd82a922740 1 Processor -- start 2024-04-25T10:05:12.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.290+0000 7fd82a922740 1 -- start start 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebef0 0x562d56411050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82a922740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 0x562d56411590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82a922740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820200550 con 0x562d5640fbc0 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201ff730 con 0x562d563ebef0 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201f5b20 con 0x562d563ebb10 2024-04-25T10:05:12.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 0x562d56411590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 0x562d56411590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47748/0 (socket says 172.21.15.148:47748) 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 -- 172.21.15.148:0/2722357728 learned_addr learned my addr 172.21.15.148:0/2722357728 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 -- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 msgr2=0x562d563bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd825d1a640 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd825519640 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebef0 0x562d56411050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563bec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 -- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebef0 msgr2=0x562d56411050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebef0 0x562d56411050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.291+0000 7fd82651b640 1 -- 172.21.15.148:0/2722357728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d5640c0c0 con 0x562d5640fbc0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82651b640 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 0x562d56411590 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fd818009990 tx=0x7fd81802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5749dbfe86ee3333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd802ffd640 1 -- 172.21.15.148:0/2722357728 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81804b070 con 0x562d5640fbc0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd802ffd640 1 -- 172.21.15.148:0/2722357728 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818032600 con 0x562d5640fbc0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd802ffd640 1 -- 172.21.15.148:0/2722357728 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818037040 con 0x562d5640fbc0 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 -- 172.21.15.148:0/2722357728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 msgr2=0x562d56411590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 0x562d56411590 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fd818009990 tx=0x7fd81802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 -- 172.21.15.148:0/2722357728 shutdown_connections 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d5640fbc0 0x562d56411590 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebef0 0x562d56411050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 --2- 172.21.15.148:0/2722357728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d563ebb10 0x562d563bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 -- 172.21.15.148:0/2722357728 >> 172.21.15.148:0/2722357728 conn(0x562d563be770 msgr2=0x562d563c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 -- 172.21.15.148:0/2722357728 shutdown_connections 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.292+0000 7fd82a922740 1 -- 172.21.15.148:0/2722357728 wait complete. 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.293+0000 7fd82a922740 1 -- start start 2024-04-25T10:05:12.300 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.293+0000 7fd82a922740 1 Processor -- start 2024-04-25T10:05:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d564765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56408eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d564093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d563eaac0 con 0x562d563ebef0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201f49a0 con 0x562d563ebb10 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d56411a40 con 0x562d5640fbc0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82a922740 10 client.0 _ll_register_callbacks cb 0x7fd810058550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56408eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56408eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47754/0 (socket says 172.21.15.148:47754) 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 -- 172.21.15.148:0/1469669062 learned_addr learned my addr 172.21.15.148:0/1469669062 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d564765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd82651b640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d564093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 -- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 msgr2=0x562d564093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d564093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 -- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 msgr2=0x562d564765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d564765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 -- 172.21.15.148:0/1469669062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d55ffc0f0 con 0x562d563ebef0 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d564765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:12.305 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd825519640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56408eb0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fd810056a30 tx=0x7fd81008a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210246f334f78066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.294+0000 7fd800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 1 -- 172.21.15.148:0/1469669062 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810085030 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 1 -- 172.21.15.148:0/1469669062 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd810060c10 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd818032600 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 1 -- 172.21.15.148:0/1469669062 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd810060e90 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 10 client.10617 fetch_fsmap learned FSMap version 93 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x562d56411760 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 1 -- 172.21.15.148:0/1469669062 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818032600 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 1 -- 172.21.15.148:0/1469669062 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd810057bd0 con 0x562d563ebef0 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 10 client.10617 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 10 client.10617 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 20 client.10617 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d563be1f0 0x562d56419050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 4 client.10617 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d5632f630 con 0x562d563be1f0 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d563be1f0 0x562d56419050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 10 client.10617 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd825d1a640 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d563be1f0 0x562d56419050 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e3a7dea8b81824 server_cookie=c98ba2f736c4df9f in_seq=0 out_seq=0 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 1 -- 172.21.15.148:0/1469669062 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562d5632f630 con 0x562d563be1f0 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.295+0000 7fd800ff9640 10 client.10617 handle_command_reply: tid=0 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 client.10617 shutdown 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 msgr2=0x562d56408eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56408eb0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fd810056a30 tx=0x7fd81008a050 comp rx=0 tx=0).stop 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 shutdown_connections 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x562d563be1f0 0x562d56419050 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562d5640fbc0 0x562d564093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562d563ebef0 0x562d56408eb0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd800ff9640 0 client.10617 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 --2- 172.21.15.148:0/1469669062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562d563ebb10 0x562d564765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 >> 172.21.15.148:0/1469669062 conn(0x562d563be770 msgr2=0x562d563f19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 shutdown_connections 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 1 -- 172.21.15.148:0/1469669062 wait complete. 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd82a922740 20 client.10617 trim_cache size 0 max 16384 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:05:12.308 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:05:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd80406b4d0 msgr2=0x7fd80406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd80406b4d0 0x7fd80406d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0099c0 tx=0x7fd81c003990 comp rx=0 tx=0).stop 2024-04-25T10:05:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 msgr2=0x7fd82015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82015cad0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd81000c430 tx=0x7fd81000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 shutdown_connections 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd80406b4d0 0x7fd80406d990 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82015dcd0 0x7fd8200cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd82006e2f0 0x7fd82015cad0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 --2- 172.21.15.148:0/3089273046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82006b8c0 0x7fd82015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 >> 172.21.15.148:0/3089273046 conn(0x7fd82006abe0 msgr2=0x7fd8200ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.296+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 shutdown_connections 2024-04-25T10:05:12.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.297+0000 7fd827f7e640 1 -- 172.21.15.148:0/3089273046 wait complete. 2024-04-25T10:05:12.312 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:05:12.313 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:05:12.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:05:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.407+0000 7f53dee9a640 1 Processor -- start 2024-04-25T10:05:12.412 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.407+0000 7f53dee9a640 1 -- start start 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dee9a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53d8162140 0x7f53d80cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dee9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f53d80cd540 0x7f53d80caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dee9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d815e3a0 con 0x7f53d80d0a00 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dee9a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d81625c0 con 0x7f53d8162140 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dee9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d80cd9d0 con 0x7f53d80cd540 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53d8162140 0x7f53d80cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f53d8162140 0x7f53d80cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47762/0 (socket says 172.21.15.148:47762) 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 -- 172.21.15.148:0/522274689 learned_addr learned my addr 172.21.15.148:0/522274689 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dcc36640 1 --1- 172.21.15.148:0/522274689 >> v1:172.21.15.148:6789/0 conn(0x7f53d80d0a00 0x7f53d80d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:53550/0 (socket says 172.21.15.148:53550) 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53dd437640 1 --2- 172.21.15.148:0/522274689 >> v2:172.21.15.188:3300/0 conn(0x7f53d80cd540 0x7f53d80caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 -- 172.21.15.148:0/522274689 >> v2:172.21.15.188:3300/0 conn(0x7f53d80cd540 msgr2=0x7f53d80caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 --2- 172.21.15.148:0/522274689 >> v2:172.21.15.188:3300/0 conn(0x7f53d80cd540 0x7f53d80caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 -- 172.21.15.148:0/522274689 >> v1:172.21.15.148:6789/0 conn(0x7f53d80d0a00 legacy=0x7f53d80d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d77fe640 1 -- 172.21.15.148:0/522274689 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834896776 0 0) 0x7f53d815e3a0 con 0x7f53d80d0a00 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.408+0000 7f53d7fff640 1 -- 172.21.15.148:0/522274689 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d80cdc70 con 0x7f53d8162140 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53d7fff640 1 --2- 172.21.15.148:0/522274689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80cd000 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f53d815f1c0 tx=0x7f53c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a13c41aa712ea65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53d77fe640 1 -- 172.21.15.148:0/522274689 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037070 con 0x7f53d8162140 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53d77fe640 1 -- 172.21.15.148:0/522274689 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8032440 con 0x7f53d8162140 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53d77fe640 1 -- 172.21.15.148:0/522274689 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8032810 con 0x7f53d8162140 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 -- 172.21.15.148:0/522274689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 msgr2=0x7f53d80cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 --2- 172.21.15.148:0/522274689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80cd000 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f53d815f1c0 tx=0x7f53c802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 -- 172.21.15.148:0/522274689 shutdown_connections 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 --2- 172.21.15.148:0/522274689 >> v2:172.21.15.188:3300/0 conn(0x7f53d80cd540 0x7f53d80caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 --2- 172.21.15.148:0/522274689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80cd000 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 -- 172.21.15.148:0/522274689 >> 172.21.15.148:0/522274689 conn(0x7f53d80d17c0 msgr2=0x7f53d80d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 -- 172.21.15.148:0/522274689 shutdown_connections 2024-04-25T10:05:12.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.409+0000 7f53dee9a640 1 -- 172.21.15.148:0/522274689 wait complete. 2024-04-25T10:05:12.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 Processor -- start 2024-04-25T10:05:12.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 -- start start 2024-04-25T10:05:12.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d80cd540 0x7f53d815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d80d0a00 0x7f53d80a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.420 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8161940 con 0x7f53d8162140 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8161660 con 0x7f53d80cd540 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.411+0000 7f53dee9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8161c20 con 0x7f53d80d0a00 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80a8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47766/0 (socket says 172.21.15.148:47766) 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 -- 172.21.15.148:0/1213314654 learned_addr learned my addr 172.21.15.148:0/1213314654 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dcc36640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d80cd540 0x7f53d815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53d7fff640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d80d0a00 0x7f53d80a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 -- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d80d0a00 msgr2=0x7f53d80a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d80d0a00 0x7f53d80a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 -- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d80cd540 msgr2=0x7f53d815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d80cd540 0x7f53d815d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 -- 172.21.15.148:0/1213314654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d8132ef0 con 0x7f53d8162140 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dd437640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80a8ac0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f53c000d800 tx=0x7f53c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.421 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53d57fa640 1 -- 172.21.15.148:0/1213314654 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000b6c0 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53d57fa640 1 -- 172.21.15.148:0/1213314654 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0019070 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d806d320 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53d57fa640 1 -- 172.21.15.148:0/1213314654 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000bac0 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.412+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d8133490 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.414+0000 7f53d57fa640 1 -- 172.21.15.148:0/1213314654 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53c000bc60 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.414+0000 7f53d57fa640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53b806b4d0 0x7f53b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.414+0000 7f53dcc36640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53b806b4d0 0x7f53b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.414+0000 7f53d57fa640 1 -- 172.21.15.148:0/1213314654 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53c008a600 con 0x7f53d8162140 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.414+0000 7f53dcc36640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53b806b4d0 0x7f53b806d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f53d00012e0 tx=0x7f53d0001700 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.415+0000 7f53e1b9a740 1 Processor -- start 2024-04-25T10:05:12.422 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.415+0000 7f53e1b9a740 1 -- start start 2024-04-25T10:05:12.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada689e700 0x55ada68a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada6807e00 con 0x55ada68cbe80 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada67b0d20 con 0x55ada689e700 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada6803ea0 con 0x55ada68cbaa0 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47774/0 (socket says 172.21.15.148:47774) 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 -- 172.21.15.148:0/1349074036 learned_addr learned my addr 172.21.15.148:0/1349074036 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53dcc36640 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53dd437640 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada689e700 0x55ada68a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 -- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 msgr2=0x55ada68ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 -- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada689e700 msgr2=0x55ada68a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada689e700 0x55ada68a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 -- 172.21.15.148:0/1349074036 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada67d2df0 con 0x55ada68cbe80 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53d7fff640 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ce7f0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f53c8035040 tx=0x7f53c802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246452435b18b8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53ce7fc640 1 -- 172.21.15.148:0/1349074036 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037040 con 0x55ada68cbe80 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53ce7fc640 1 -- 172.21.15.148:0/1349074036 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c80327a0 con 0x55ada68cbe80 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.416+0000 7f53ce7fc640 1 -- 172.21.15.148:0/1349074036 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8032b90 con 0x55ada68cbe80 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 -- 172.21.15.148:0/1349074036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 msgr2=0x55ada68ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ce7f0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f53c8035040 tx=0x7f53c802c000 comp rx=0 tx=0).stop 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 -- 172.21.15.148:0/1349074036 shutdown_connections 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada689e700 0x55ada68a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ce7f0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/1349074036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.432 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 -- 172.21.15.148:0/1349074036 >> 172.21.15.148:0/1349074036 conn(0x55ada68a0970 msgr2=0x55ada68e92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 -- 172.21.15.148:0/1349074036 shutdown_connections 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 -- 172.21.15.148:0/1349074036 wait complete. 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.417+0000 7f53e1b9a740 1 -- start start 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 Processor -- start 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68cbaa0 0x55ada68caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbe80 0x55ada68f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 0x55ada68eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada68a3b20 con 0x55ada68ecaa0 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada67a6590 con 0x55ada68cbaa0 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.418+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada68a3e00 con 0x55ada68cbe80 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53e1b9a740 10 client.0 _ll_register_callbacks cb 0x55ada67b2e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 0x55ada68eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 0x55ada68eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47782/0 (socket says 172.21.15.148:47782) 2024-04-25T10:05:12.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 -- 172.21.15.148:0/3991543414 learned_addr learned my addr 172.21.15.148:0/3991543414 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53d7fff640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbe80 0x55ada68f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68cbaa0 0x55ada68caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 -- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbe80 msgr2=0x55ada68f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbe80 0x55ada68f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 -- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68cbaa0 msgr2=0x55ada68caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68cbaa0 0x55ada68caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 -- 172.21.15.148:0/3991543414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ada651c3f0 con 0x55ada68ecaa0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53dd437640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 0x55ada68eb430 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c005cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e88a798c1441e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53b77fe640 1 -- 172.21.15.148:0/3991543414 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0070020 con 0x55ada68ecaa0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53b77fe640 1 -- 172.21.15.148:0/3991543414 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53c00553a0 con 0x55ada68ecaa0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53c80327a0 con 0x55ada68ecaa0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53b77fe640 1 -- 172.21.15.148:0/3991543414 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53c0058180 con 0x55ada68ecaa0 2024-04-25T10:05:12.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53e1b9a740 10 client.10632 fetch_fsmap learned FSMap version 93 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55ada65e67d0 con 0x55ada68ecaa0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53b77fe640 1 -- 172.21.15.148:0/3991543414 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c80327a0 con 0x55ada68ecaa0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.419+0000 7f53b77fe640 1 -- 172.21.15.148:0/3991543414 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53c0013070 con 0x55ada68ecaa0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 10 client.10632 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 10 client.10632 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 20 client.10632 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ee0b0 0x55ada68fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 4 client.10632 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ada6815340 con 0x55ada68ee0b0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ee0b0 0x55ada68fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53b77fe640 10 client.10632 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ee0b0 0x55ada68fc900 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a21b0782b0bbd53 server_cookie=78075ef1f7b90b51 in_seq=0 out_seq=0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53b77fe640 1 -- 172.21.15.148:0/3991543414 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ada6815340 con 0x55ada68ee0b0 2024-04-25T10:05:12.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53b77fe640 10 client.10632 handle_command_reply: tid=0 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.420+0000 7f53e1b9a740 1 client.10632 shutdown 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 msgr2=0x55ada68eb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 0x55ada68eb430 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c005cdd0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 shutdown_connections 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ee0b0 0x55ada68fc900 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68ecaa0 0x55ada68eb430 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbe80 0x55ada68f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3991543414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68cbaa0 0x55ada68caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 >> 172.21.15.148:0/3991543414 conn(0x55ada689e700 msgr2=0x55ada68e6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53b77fe640 0 client.10632 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 shutdown_connections 2024-04-25T10:05:12.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3991543414 wait complete. 2024-04-25T10:05:12.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.421+0000 7f53e1b9a740 20 client.10632 trim_cache size 0 max 16384 2024-04-25T10:05:12.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 Processor -- start 2024-04-25T10:05:12.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 -- start start 2024-04-25T10:05:12.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8200240 con 0x55ada68cbe80 2024-04-25T10:05:12.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81ff400 con 0x55ada68ecaa0 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.433+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81345a0 con 0x55ada68cbaa0 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47790/0 (socket says 172.21.15.148:47790) 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 -- 172.21.15.148:0/266539090 learned_addr learned my addr 172.21.15.148:0/266539090 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 -- 172.21.15.148:0/266539090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 msgr2=0x55ada68e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 -- 172.21.15.148:0/266539090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 msgr2=0x55ada68ed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 -- 172.21.15.148:0/266539090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ada68ef1d0 con 0x55ada68cbe80 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53d7fff640 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed090 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f53c802a7d0 tx=0x7f53c8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a08eee87912d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53cdffb640 1 -- 172.21.15.148:0/266539090 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037040 con 0x55ada68cbe80 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53cdffb640 1 -- 172.21.15.148:0/266539090 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8003610 con 0x55ada68cbe80 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.434+0000 7f53cdffb640 1 -- 172.21.15.148:0/266539090 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80039e0 con 0x55ada68cbe80 2024-04-25T10:05:12.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 -- 172.21.15.148:0/266539090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 msgr2=0x55ada68ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed090 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f53c802a7d0 tx=0x7f53c8008030 comp rx=0 tx=0).stop 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 -- 172.21.15.148:0/266539090 shutdown_connections 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68ed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed090 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/266539090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 -- 172.21.15.148:0/266539090 >> 172.21.15.148:0/266539090 conn(0x55ada689e700 msgr2=0x55ada68cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 -- 172.21.15.148:0/266539090 shutdown_connections 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 -- 172.21.15.148:0/266539090 wait complete. 2024-04-25T10:05:12.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.435+0000 7f53e1b9a740 1 -- start start 2024-04-25T10:05:12.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 Processor -- start 2024-04-25T10:05:12.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ada68ca170 con 0x55ada68cbe80 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8134ba0 con 0x55ada68ecaa0 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ada690bd90 con 0x55ada68cbaa0 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53e1b9a740 10 client.0 _ll_register_callbacks cb 0x7f53c80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47792/0 (socket says 172.21.15.148:47792) 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.436+0000 7f53d7fff640 1 -- 172.21.15.148:0/3384689918 learned_addr learned my addr 172.21.15.148:0/3384689918 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53d7fff640 1 -- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 msgr2=0x55ada68ecea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53dd437640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53d7fff640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ecea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53d7fff640 1 -- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 msgr2=0x55ada68fb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53d7fff640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68fb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53d7fff640 1 -- 172.21.15.148:0/3384689918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ada68eff80 con 0x55ada68cbe80 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53dd437640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53d7fff640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed3e0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f53c8006c80 tx=0x7f53c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95612cc8e80431df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53b77fe640 1 -- 172.21.15.148:0/3384689918 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037040 con 0x55ada68cbe80 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53b77fe640 1 -- 172.21.15.148:0/3384689918 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53c80039d0 con 0x55ada68cbe80 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53c8003610 con 0x55ada68cbe80 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53b77fe640 1 -- 172.21.15.148:0/3384689918 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53c8058050 con 0x55ada68cbe80 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 10 client.10638 fetch_fsmap learned FSMap version 93 2024-04-25T10:05:12.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x55ada68d2750 con 0x55ada68cbe80 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53b77fe640 1 -- 172.21.15.148:0/3384689918 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8003610 con 0x55ada68cbe80 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53b77fe640 1 -- 172.21.15.148:0/3384689918 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 93) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53c805b020 con 0x55ada68cbe80 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 10 client.10638 fetch_fsmap finished waiting for FSMap version 93 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 10 client.10638 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.c 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 20 client.10638 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ce270 0x55ada68fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 4 client.10638 mds_command: new command op to 10365 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 --> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ada68a4170 con 0x55ada68ce270 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.437+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ce270 0x55ada68fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53b77fe640 10 client.10638 ms_handle_connect on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53dcc36640 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ce270 0x55ada68fb8e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985a340632b4c533 server_cookie=9d8507129b15319d in_seq=0 out_seq=0 2024-04-25T10:05:12.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53b77fe640 1 -- 172.21.15.148:0/3384689918 <== mds.0 v2:172.21.15.148:6835/3286540466 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ada68a4170 con 0x55ada68ce270 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53b77fe640 10 client.10638 handle_command_reply: tid=0 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 client.10638 shutdown 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 msgr2=0x55ada68ed3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed3e0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f53c8006c80 tx=0x7f53c8004730 comp rx=0 tx=0).stop 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 shutdown_connections 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:6835/3286540466,v1:172.21.15.148:6837/3286540466] conn(0x55ada68ce270 0x55ada68fb8e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ada68ecaa0 0x55ada68fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53b77fe640 0 client.10638 ms_handle_reset on v2:172.21.15.148:6835/3286540466 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ada68cbe80 0x55ada68ed3e0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 --2- 172.21.15.148:0/3384689918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ada68cbaa0 0x55ada68ecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 >> 172.21.15.148:0/3384689918 conn(0x55ada689e700 msgr2=0x55ada68cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 shutdown_connections 2024-04-25T10:05:12.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 1 -- 172.21.15.148:0/3384689918 wait complete. 2024-04-25T10:05:12.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.438+0000 7f53e1b9a740 20 client.10638 trim_cache size 0 max 16384 2024-04-25T10:05:12.451 INFO:teuthology.orchestra.run.smithi148.stdout:[] 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53b806b4d0 msgr2=0x7f53b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53b806b4d0 0x7f53b806d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f53d00012e0 tx=0x7f53d0001700 comp rx=0 tx=0).stop 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 msgr2=0x7f53d80a8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80a8ac0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f53c000d800 tx=0x7f53c000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 shutdown_connections 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f53b806b4d0 0x7f53b806d990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d8162140 0x7f53d80a8ac0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d80d0a00 0x7f53d80a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 --2- 172.21.15.148:0/1213314654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d80cd540 0x7f53d815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 >> 172.21.15.148:0/1213314654 conn(0x7f53d80d17c0 msgr2=0x7f53d8070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 shutdown_connections 2024-04-25T10:05:12.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.439+0000 7f53dee9a640 1 -- 172.21.15.148:0/1213314654 wait complete. 2024-04-25T10:05:12.455 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:05:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.545+0000 7f574a2fc640 1 Processor -- start 2024-04-25T10:05:12.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.545+0000 7f574a2fc640 1 -- start start 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.546+0000 7f574a2fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f574415b9c0 0x7f574415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.546+0000 7f574a2fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5744073150 con 0x7f574406abe0 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.546+0000 7f574a2fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57440cfa60 con 0x7f5744072d30 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.546+0000 7f574a2fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57440cfd40 con 0x7f574415b9c0 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.546+0000 7f57437fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f574406abe0 0x7f5744072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:53558/0 (socket says 172.21.15.148:53558) 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.546+0000 7f57437fe640 1 -- 172.21.15.148:0/901041213 learned_addr learned my addr 172.21.15.148:0/901041213 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812592411 0 0) 0x7f5744073150 con 0x7f574406abe0 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f572c003520 con 0x7f574406abe0 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 --2- 172.21.15.148:0/901041213 >> v2:172.21.15.188:3300/0 conn(0x7f574415b9c0 0x7f574415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2284698572 0 0) 0x7f572c003520 con 0x7f574406abe0 2024-04-25T10:05:12.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 >> v2:172.21.15.188:3300/0 conn(0x7f574415b9c0 msgr2=0x7f574415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 --2- 172.21.15.148:0/901041213 >> v2:172.21.15.188:3300/0 conn(0x7f574415b9c0 0x7f574415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 >> v1:172.21.15.188:6789/0 conn(0x7f5744072d30 legacy=0x7f57440d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440d0020 con 0x7f574406abe0 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5738001670 con 0x7f574406abe0 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 >> v1:172.21.15.148:6789/0 conn(0x7f574406abe0 legacy=0x7f5744072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f572c0039b0 0x7f572c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f572c004800 0x7f572c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0051c0 0x7f572c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440cfd40 con 0x7f572c0039b0 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c00c060 con 0x7f572c004800 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c00c260 con 0x7f572c0051c0 2024-04-25T10:05:12.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f572c0039b0 0x7f572c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f57437fe640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f572c004800 0x7f572c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5748899640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0051c0 0x7f572c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 -- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0051c0 msgr2=0x7f572c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0051c0 0x7f572c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 -- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f572c004800 msgr2=0x7f572c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f572c004800 0x7f572c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5743fff640 1 -- 172.21.15.148:0/901041213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c003520 con 0x7f572c0039b0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.547+0000 7f5748899640 1 -- 172.21.15.148:0/901041213 reap_dead start 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.548+0000 7f5743fff640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f572c0039b0 0x7f572c003dd0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f5734009ac0 tx=0x7f572c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9a848c3680503c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.548+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734036070 con 0x7f572c0039b0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.548+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734031400 con 0x7f572c0039b0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.548+0000 7f5742ffd640 1 -- 172.21.15.148:0/901041213 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734009690 con 0x7f572c0039b0 2024-04-25T10:05:12.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 -- 172.21.15.148:0/901041213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f572c0039b0 msgr2=0x7f572c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f572c0039b0 0x7f572c003dd0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f5734009ac0 tx=0x7f572c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 -- 172.21.15.148:0/901041213 shutdown_connections 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 --2- 172.21.15.148:0/901041213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f572c0039b0 0x7f572c003dd0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 -- 172.21.15.148:0/901041213 >> 172.21.15.148:0/901041213 conn(0x7f57440b2370 msgr2=0x7f57440b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 -- 172.21.15.148:0/901041213 shutdown_connections 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.549+0000 7f574a2fc640 1 -- 172.21.15.148:0/901041213 wait complete. 2024-04-25T10:05:12.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.550+0000 7f574a2fc640 1 Processor -- start 2024-04-25T10:05:12.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.550+0000 7f574a2fc640 1 -- start start 2024-04-25T10:05:12.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0039b0 0x7f5744153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 0x7f57440caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574414f3d0 0x7f574414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574406b070 con 0x7f57440c99f0 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440cf710 con 0x7f574414f3d0 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574406b1f0 con 0x7f572c0039b0 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 0x7f57440caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 0x7f57440caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47822/0 (socket says 172.21.15.148:47822) 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 -- 172.21.15.148:0/2752734633 learned_addr learned my addr 172.21.15.148:0/2752734633 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 -- 172.21.15.148:0/2752734633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0039b0 msgr2=0x7f5744153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0039b0 0x7f5744153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 -- 172.21.15.148:0/2752734633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574414f3d0 msgr2=0x7f574414f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574414f3d0 0x7f574414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 -- 172.21.15.148:0/2752734633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440c9ed0 con 0x7f57440c99f0 2024-04-25T10:05:12.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f57437fe640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 0x7f57440caf70 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f573800e7b0 tx=0x7f573800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800c870 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738018070 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57440ca1b0 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800cc50 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.551+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57440ca770 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.552+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5744073990 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.553+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f573801d460 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.553+0000 7f5740ff9640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573006b4d0 0x7f573006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.553+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f573808a300 con 0x7f57440c99f0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.553+0000 7f5743fff640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573006b4d0 0x7f573006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.554+0000 7f5743fff640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573006b4d0 0x7f573006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5734033000 tx=0x7f573402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.558+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5738058fe0 con 0x7f57440c99f0 2024-04-25T10:05:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.765+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f57440b1fb0 con 0x7f57440c99f0 2024-04-25T10:05:12.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.766+0000 7f5740ff9640 1 -- 172.21.15.148:0/2752734633 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f57380588e0 con 0x7f57440c99f0 2024-04-25T10:05:12.771 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:12.771 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:05:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573006b4d0 msgr2=0x7f573006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573006b4d0 0x7f573006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5734033000 tx=0x7f573402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 msgr2=0x7f57440caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 0x7f57440caf70 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f573800e7b0 tx=0x7f573800ec70 comp rx=0 tx=0).stop 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 shutdown_connections 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f573006b4d0 0x7f573006d990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f574414f3d0 0x7f574414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57440c99f0 0x7f57440caf70 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 --2- 172.21.15.148:0/2752734633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f572c0039b0 0x7f5744153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 >> 172.21.15.148:0/2752734633 conn(0x7f57440b2370 msgr2=0x7f5744072620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 shutdown_connections 2024-04-25T10:05:12.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:12.768+0000 7f574a2fc640 1 -- 172.21.15.148:0/2752734633 wait complete. 2024-04-25T10:05:12.784 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:12.784 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:12.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:12.878 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:12.878 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:12.878 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.894 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:12.894 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.081+0000 7f9dad75c640 1 Processor -- start 2024-04-25T10:05:13.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.081+0000 7f9dad75c640 1 -- start start 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.082+0000 7f9dad75c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9da815f5f0 0x7f9da815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.082+0000 7f9dad75c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80cd710 con 0x7f9da81602e0 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.082+0000 7f9dad75c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da8099b70 con 0x7f9da815ba00 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.082+0000 7f9dad75c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da815ff80 con 0x7f9da815f5f0 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.082+0000 7f9da67fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9da81602e0 0x7f9da80ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:53568/0 (socket says 172.21.15.148:53568) 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.082+0000 7f9da67fc640 1 -- 172.21.15.148:0/2665807109 learned_addr learned my addr 172.21.15.148:0/2665807109 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74227291 0 0) 0x7f9d9c001110 con 0x7f9da81602e0 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da77fe640 1 --2- 172.21.15.148:0/2665807109 >> v2:172.21.15.188:3300/0 conn(0x7f9da815f5f0 0x7f9da815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d90003520 con 0x7f9da81602e0 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852697240 0 0) 0x7f9d90003520 con 0x7f9da81602e0 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 >> v2:172.21.15.188:3300/0 conn(0x7f9da815f5f0 msgr2=0x7f9da815fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 --2- 172.21.15.148:0/2665807109 >> v2:172.21.15.188:3300/0 conn(0x7f9da815f5f0 0x7f9da815fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 >> v1:172.21.15.188:6789/0 conn(0x7f9da815ba00 legacy=0x7f9da809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da8099e50 con 0x7f9da81602e0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9d9c001840 con 0x7f9da81602e0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 >> v1:172.21.15.148:6789/0 conn(0x7f9da81602e0 legacy=0x7f9da80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d900039b0 0x7f9d90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d90004800 0x7f9d90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900052e0 0x7f9d9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da815ff80 con 0x7f9d900039b0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9000c1b0 con 0x7f9d90004800 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9000c3b0 con 0x7f9d900052e0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d900039b0 0x7f9d90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da67fc640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d90004800 0x7f9d90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da77fe640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900052e0 0x7f9d9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da6ffd640 1 -- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900052e0 msgr2=0x7f9d9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d900052e0 0x7f9d9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da6ffd640 1 -- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d90004800 msgr2=0x7f9d90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d90004800 0x7f9d90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da6ffd640 1 -- 172.21.15.148:0/2665807109 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90003520 con 0x7f9d900039b0 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da77fe640 1 -- 172.21.15.148:0/2665807109 reap_dead start 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.083+0000 7f9da67fc640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d90004800 0x7f9d90004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.084+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d900039b0 0x7f9d90003dd0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001e20 tx=0x7f9d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a227c275b80309fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.084+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0001a70 con 0x7f9d900039b0 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.084+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da002adf0 con 0x7f9d900039b0 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.084+0000 7f9da5ffb640 1 -- 172.21.15.148:0/2665807109 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0003bb0 con 0x7f9d900039b0 2024-04-25T10:05:13.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 -- 172.21.15.148:0/2665807109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d900039b0 msgr2=0x7f9d90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d900039b0 0x7f9d90003dd0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001e20 tx=0x7f9d90005830 comp rx=0 tx=0).stop 2024-04-25T10:05:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 -- 172.21.15.148:0/2665807109 shutdown_connections 2024-04-25T10:05:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 --2- 172.21.15.148:0/2665807109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d900039b0 0x7f9d90003dd0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 -- 172.21.15.148:0/2665807109 >> 172.21.15.148:0/2665807109 conn(0x7f9da80b3270 msgr2=0x7f9da80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 -- 172.21.15.148:0/2665807109 shutdown_connections 2024-04-25T10:05:13.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.085+0000 7f9dad75c640 1 -- 172.21.15.148:0/2665807109 wait complete. 2024-04-25T10:05:13.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.086+0000 7f9dad75c640 1 Processor -- start 2024-04-25T10:05:13.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.086+0000 7f9dad75c640 1 -- start start 2024-04-25T10:05:13.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9dad75c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d900039b0 0x7f9da806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9dad75c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 0x7f9da8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9dad75c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157810 0x7f9da8158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9dad75c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da815bde0 con 0x7f9da8157390 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9dad75c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da815fa40 con 0x7f9d900039b0 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9dad75c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8158cf0 con 0x7f9da8157810 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 0x7f9da8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 0x7f9da8157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47842/0 (socket says 172.21.15.148:47842) 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 -- 172.21.15.148:0/4138803474 learned_addr learned my addr 172.21.15.148:0/4138803474 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:13.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da77fe640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157810 0x7f9da8158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d900039b0 0x7f9da806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 -- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157810 msgr2=0x7f9da8158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157810 0x7f9da8158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 -- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d900039b0 msgr2=0x7f9da806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d900039b0 0x7f9da806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 -- 172.21.15.148:0/4138803474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da8158fd0 con 0x7f9da8157390 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da77fe640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157810 0x7f9da8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d900039b0 0x7f9da806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9da67fc640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 0x7f9da8157cc0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c005d60 tx=0x7f9d9c007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.087+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00e740 con 0x7f9da8157390 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.088+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c018070 con 0x7f9da8157390 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.088+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da81592b0 con 0x7f9da8157390 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.088+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00eae0 con 0x7f9da8157390 2024-04-25T10:05:13.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.088+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da815c3d0 con 0x7f9da8157390 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.088+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da8073990 con 0x7f9da8157390 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.089+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d9c00ec80 con 0x7f9da8157390 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.089+0000 7f9d8f7fe640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d9406b4d0 0x7f9d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.089+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d9c08a0d0 con 0x7f9da8157390 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.089+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d9406b4d0 0x7f9d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.090+0000 7f9da6ffd640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d9406b4d0 0x7f9d9406d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9da0032000 tx=0x7f9da002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.094+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9da80cd710 con 0x7f9da8157390 2024-04-25T10:05:13.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.266+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da80cb9a0 con 0x7f9da8157390 2024-04-25T10:05:13.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.267+0000 7f9d8f7fe640 1 -- 172.21.15.148:0/4138803474 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f9d9c058e50 con 0x7f9da8157390 2024-04-25T10:05:13.273 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:13.273 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":159,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:32.326979+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d9406b4d0 msgr2=0x7f9d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d9406b4d0 0x7f9d9406d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9da0032000 tx=0x7f9da002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:13.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 msgr2=0x7f9da8157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 0x7f9da8157cc0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c005d60 tx=0x7f9d9c007420 comp rx=0 tx=0).stop 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 shutdown_connections 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d9406b4d0 0x7f9d9406d990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da8157810 0x7f9da8158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9da8157390 0x7f9da8157cc0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 --2- 172.21.15.148:0/4138803474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d900039b0 0x7f9da806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 >> 172.21.15.148:0/4138803474 conn(0x7f9da80b3270 msgr2=0x7f9da80cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 shutdown_connections 2024-04-25T10:05:13.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.269+0000 7f9dad75c640 1 -- 172.21.15.148:0/4138803474 wait complete. 2024-04-25T10:05:13.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:28.876622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:29.635420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:13.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:13.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.382+0000 7f5482dc4640 1 Processor -- start 2024-04-25T10:05:13.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.382+0000 7f5482dc4640 1 -- start start 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5482dc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0d0ab0 0x7f547c168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5482dc4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c0d2420 con 0x7f547c15dac0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5482dc4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c0d2700 con 0x7f547c0d34c0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5482dc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0d3030 con 0x7f547c0d0ab0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5480b60640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f547c15dac0 0x7f547c0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:53580/0 (socket says 172.21.15.148:53580) 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5480b60640 1 -- 172.21.15.148:0/4251164194 learned_addr learned my addr 172.21.15.148:0/4251164194 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5481361640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0d0ab0 0x7f547c168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097874402 0 0) 0x7f547c0d2420 con 0x7f547c15dac0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c003520 con 0x7f547c15dac0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 560493122 0 0) 0x7f546c003520 con 0x7f547c15dac0 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0d0ab0 msgr2=0x7f547c168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0d0ab0 0x7f547c168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 >> v1:172.21.15.188:6789/0 conn(0x7f547c0d34c0 legacy=0x7f547c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0d29e0 con 0x7f547c15dac0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5470001670 con 0x7f547c15dac0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 >> v1:172.21.15.148:6789/0 conn(0x7f547c15dac0 legacy=0x7f547c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f546c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0043b0 0x7f546c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546c005080 0x7f546c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0d3030 con 0x7f546c0039e0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c00bf00 con 0x7f546c0043b0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c00c0b0 con 0x7f546c005080 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5481361640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0d0ab0 0x7f547c168670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.383+0000 7f5480b60640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f546c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f547bfff640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0043b0 0x7f546c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5480b60640 1 -- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546c005080 msgr2=0x7f546c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5481361640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546c005080 0x7f546c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5480b60640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546c005080 0x7f546c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5480b60640 1 -- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0043b0 msgr2=0x7f546c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5480b60640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0043b0 0x7f546c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5480b60640 1 -- 172.21.15.148:0/4251164194 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c003520 con 0x7f546c0039e0 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5481361640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f546c005080 0x7f546c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f5480b60640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f546c003de0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f547000c240 tx=0x7f546c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47b217d6dbe5138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.384+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470007d10 con 0x7f546c0039e0 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470005af0 con 0x7f546c0039e0 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f547b7fe640 1 -- 172.21.15.148:0/4251164194 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470007070 con 0x7f546c0039e0 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5482dc4640 1 -- 172.21.15.148:0/4251164194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 msgr2=0x7f546c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5482dc4640 1 --2- 172.21.15.148:0/4251164194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f546c003de0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f547000c240 tx=0x7f546c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5481361640 1 -- 172.21.15.148:0/4251164194 reap_dead start 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5482dc4640 1 -- 172.21.15.148:0/4251164194 shutdown_connections 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5482dc4640 1 -- 172.21.15.148:0/4251164194 >> 172.21.15.148:0/4251164194 conn(0x7f547c061c30 msgr2=0x7f547c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5482dc4640 1 -- 172.21.15.148:0/4251164194 shutdown_connections 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.385+0000 7f5482dc4640 1 -- 172.21.15.148:0/4251164194 wait complete. 2024-04-25T10:05:13.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.386+0000 7f5482dc4640 1 Processor -- start 2024-04-25T10:05:13.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.386+0000 7f5482dc4640 1 -- start start 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5482dc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f547c1f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5482dc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0b69b0 0x7f547c0b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5482dc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f547c15e090 0x7f547c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5482dc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c15dea0 con 0x7f546c0039e0 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5482dc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0d2100 con 0x7f547c15e090 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5482dc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0b97d0 con 0x7f547c0b69b0 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f547c1f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f547c1f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47862/0 (socket says 172.21.15.148:47862) 2024-04-25T10:05:13.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 -- 172.21.15.148:0/905537142 learned_addr learned my addr 172.21.15.148:0/905537142 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 -- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0b69b0 msgr2=0x7f547c0b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5481361640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f547c15e090 0x7f547c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f547bfff640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0b69b0 0x7f547c0b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0b69b0 0x7f547c0b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 -- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f547c15e090 msgr2=0x7f547c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f547c15e090 0x7f547c15e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.387+0000 7f5480b60640 1 -- 172.21.15.148:0/905537142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c1f7b80 con 0x7f546c0039e0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.388+0000 7f5480b60640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f547c1f3030 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f547000c240 tx=0x7f547c0d28d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.388+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470040070 con 0x7f546c0039e0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.388+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470005930 con 0x7f546c0039e0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.388+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c15ee00 con 0x7f546c0039e0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.388+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470001630 con 0x7f546c0039e0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.388+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c1f70a0 con 0x7f546c0039e0 2024-04-25T10:05:13.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.389+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c073a00 con 0x7f546c0039e0 2024-04-25T10:05:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.389+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5470043070 con 0x7f546c0039e0 2024-04-25T10:05:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.390+0000 7f54797fa640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f546006b4d0 0x7f546006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.390+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54700aa060 con 0x7f546c0039e0 2024-04-25T10:05:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.390+0000 7f547bfff640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f546006b4d0 0x7f546006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.390+0000 7f547bfff640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f546006b4d0 0x7f546006d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5468007fd0 tx=0x7f5468007b00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.394+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f547004a030 con 0x7f546c0039e0 2024-04-25T10:05:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.564+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f547c0b3670 con 0x7f546c0039e0 2024-04-25T10:05:13.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.564+0000 7f54797fa640 1 -- 172.21.15.148:0/905537142 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5470078de0 con 0x7f546c0039e0 2024-04-25T10:05:13.570 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:05:13.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f546006b4d0 msgr2=0x7f546006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f546006b4d0 0x7f546006d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5468007fd0 tx=0x7f5468007b00 comp rx=0 tx=0).stop 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 msgr2=0x7f547c1f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f547c1f3030 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f547000c240 tx=0x7f547c0d28d0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 shutdown_connections 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f546006b4d0 0x7f546006d990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f547c15e090 0x7f547c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f547c0b69b0 0x7f547c0b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 --2- 172.21.15.148:0/905537142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f546c0039e0 0x7f547c1f3030 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 >> 172.21.15.148:0/905537142 conn(0x7f547c061c30 msgr2=0x7f547c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 shutdown_connections 2024-04-25T10:05:13.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.566+0000 7f5482dc4640 1 -- 172.21.15.148:0/905537142 wait complete. 2024-04-25T10:05:13.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:13.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.678+0000 7f30db31d640 1 Processor -- start 2024-04-25T10:05:13.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.678+0000 7f30db31d640 1 -- start start 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30db31d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30d4144d50 0x7f30d406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30db31d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 0x7f30d406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30db31d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d40cccc0 con 0x7f30d41446d0 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30db31d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40b2b20 con 0x7f30d4144d50 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30db31d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d406d400 con 0x7f30d41452a0 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30d90b9640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f30d41446d0 0x7f30d40b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47038/0 (socket says 172.21.15.148:47038) 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30d90b9640 1 -- 172.21.15.148:0/4102063473 learned_addr learned my addr 172.21.15.148:0/4102063473 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481683929 0 0) 0x7f30d40cccc0 con 0x7f30d41446d0 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4102063473 >> v2:172.21.15.188:3300/0 conn(0x7f30d4144d50 0x7f30d406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30c0003520 con 0x7f30d41446d0 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30d98ba640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 0x7f30d406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947496662 0 0) 0x7f30c0003520 con 0x7f30d41446d0 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 msgr2=0x7f30d406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 0x7f30d406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 >> v2:172.21.15.188:3300/0 conn(0x7f30d4144d50 msgr2=0x7f30d406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 --2- 172.21.15.148:0/4102063473 >> v2:172.21.15.188:3300/0 conn(0x7f30d4144d50 0x7f30d406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40cc140 con 0x7f30d41446d0 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f30d0001670 con 0x7f30d41446d0 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 >> v1:172.21.15.148:6789/0 conn(0x7f30d41446d0 legacy=0x7f30d40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 0x7f30c00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c0004d40 0x7f30c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40b2b20 con 0x7f30c0003f30 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d406d400 con 0x7f30c0004d40 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.679+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c000b9c0 con 0x7f30d41452a0 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d90b9640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 0x7f30c00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d98ba640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c0004d40 0x7f30c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 -- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 msgr2=0x7f30c00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d41452a0 0x7f30c00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 -- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c0004d40 msgr2=0x7f30c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30c0004d40 0x7f30c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 -- 172.21.15.148:0/4102063473 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c0003520 con 0x7f30c0003f30 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.680+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30c0004800 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009ac0 tx=0x7f30c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32c05eed1c5c910 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4036070 con 0x7f30c0003f30 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c4031400 con 0x7f30c0003f30 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30cbfff640 1 -- 172.21.15.148:0/4102063473 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4009690 con 0x7f30c0003f30 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30db31d640 1 -- 172.21.15.148:0/4102063473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 msgr2=0x7f30c0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30db31d640 1 --2- 172.21.15.148:0/4102063473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30c0004800 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009ac0 tx=0x7f30c0005830 comp rx=0 tx=0).stop 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30d98ba640 1 -- 172.21.15.148:0/4102063473 reap_dead start 2024-04-25T10:05:13.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30db31d640 1 -- 172.21.15.148:0/4102063473 shutdown_connections 2024-04-25T10:05:13.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30db31d640 1 -- 172.21.15.148:0/4102063473 >> 172.21.15.148:0/4102063473 conn(0x7f30d406abe0 msgr2=0x7f30d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30db31d640 1 -- 172.21.15.148:0/4102063473 shutdown_connections 2024-04-25T10:05:13.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.681+0000 7f30db31d640 1 -- 172.21.15.148:0/4102063473 wait complete. 2024-04-25T10:05:13.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.682+0000 7f30db31d640 1 Processor -- start 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.682+0000 7f30db31d640 1 -- start start 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30db31d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30d4159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30db31d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d40cebd0 0x7f30d40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30db31d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d40adbb0 0x7f30d41e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30db31d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40cc9e0 con 0x7f30c0003f30 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30db31d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40cc700 con 0x7f30d40cebd0 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30db31d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4144ab0 con 0x7f30d40adbb0 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30d90b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30d4159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30d90b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30d4159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47190/0 (socket says 172.21.15.148:47190) 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30d90b9640 1 -- 172.21.15.148:0/4061760726 learned_addr learned my addr 172.21.15.148:0/4061760726 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30d98ba640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d40adbb0 0x7f30d41e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.683+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d40cebd0 0x7f30d40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d90b9640 1 -- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d40adbb0 msgr2=0x7f30d41e0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d90b9640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d40adbb0 0x7f30d41e0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d90b9640 1 -- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d40cebd0 msgr2=0x7f30d40cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d90b9640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d40cebd0 0x7f30d40cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d90b9640 1 -- 172.21.15.148:0/4061760726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40cdb30 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d98ba640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d40adbb0 0x7f30d41e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30d90b9640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30d4159720 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f30d0007340 tx=0x7f30d0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d001a070 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d000f940 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000fd10 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d40cde10 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.684+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40ce3d0 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.685+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d4073a00 con 0x7f30c0003f30 2024-04-25T10:05:13.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.685+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30d001d070 con 0x7f30c0003f30 2024-04-25T10:05:13.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.686+0000 7f30c9ffb640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.686+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30d0089e10 con 0x7f30c0003f30 2024-04-25T10:05:13.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.686+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.686+0000 7f30d88b8640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30b806b4d0 0x7f30b806d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f30c4033000 tx=0x7f30c4008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.690+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30d0058b00 con 0x7f30c0003f30 2024-04-25T10:05:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.861+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30d41434e0 con 0x7f30c0003f30 2024-04-25T10:05:13.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.862+0000 7f30c9ffb640 1 -- 172.21.15.148:0/4061760726 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30d0058400 con 0x7f30c0003f30 2024-04-25T10:05:13.867 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30b806b4d0 msgr2=0x7f30b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30b806b4d0 0x7f30b806d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f30c4033000 tx=0x7f30c4008030 comp rx=0 tx=0).stop 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 msgr2=0x7f30d4159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30d4159720 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f30d0007340 tx=0x7f30d0007800 comp rx=0 tx=0).stop 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 shutdown_connections 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f30b806b4d0 0x7f30b806d990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30d40adbb0 0x7f30d41e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d40cebd0 0x7f30d40cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 --2- 172.21.15.148:0/4061760726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30c0003f30 0x7f30d4159720 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 >> 172.21.15.148:0/4061760726 conn(0x7f30d406abe0 msgr2=0x7f30d4143700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 shutdown_connections 2024-04-25T10:05:13.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.864+0000 7f30db31d640 1 -- 172.21.15.148:0/4061760726 wait complete. 2024-04-25T10:05:13.880 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.975+0000 7f163531d640 1 Processor -- start 2024-04-25T10:05:13.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.975+0000 7f163531d640 1 -- start start 2024-04-25T10:05:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f163531d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f163015dc20 0x7f16300ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f163531d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16300d32f0 0x7f16300cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f163531d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163015e000 con 0x7f1630161710 2024-04-25T10:05:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f163531d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16300d1de0 con 0x7f16300d32f0 2024-04-25T10:05:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f163531d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1630161c60 con 0x7f163015dc20 2024-04-25T10:05:13.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16300d32f0 0x7f16300cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f16300d32f0 0x7f16300cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47200/0 (socket says 172.21.15.148:47200) 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 -- 172.21.15.148:0/803883081 learned_addr learned my addr 172.21.15.148:0/803883081 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162effd640 1 --2- 172.21.15.148:0/803883081 >> v2:172.21.15.188:3300/0 conn(0x7f163015dc20 0x7f16300ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 -- 172.21.15.148:0/803883081 >> v2:172.21.15.188:3300/0 conn(0x7f163015dc20 msgr2=0x7f16300ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 --2- 172.21.15.148:0/803883081 >> v2:172.21.15.188:3300/0 conn(0x7f163015dc20 0x7f16300ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 -- 172.21.15.148:0/803883081 >> v1:172.21.15.188:6789/0 conn(0x7f1630161710 legacy=0x7f163006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.976+0000 7f162f7fe640 1 -- 172.21.15.148:0/803883081 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1630161f40 con 0x7f16300d32f0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f162f7fe640 1 --2- 172.21.15.148:0/803883081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f16300cebb0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f16300d1520 tx=0x7f162802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ede1e6515c421c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f162e7fc640 1 -- 172.21.15.148:0/803883081 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628036070 con 0x7f16300d32f0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f162e7fc640 1 -- 172.21.15.148:0/803883081 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628031600 con 0x7f16300d32f0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f162e7fc640 1 -- 172.21.15.148:0/803883081 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280319d0 con 0x7f16300d32f0 2024-04-25T10:05:13.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f163531d640 1 -- 172.21.15.148:0/803883081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 msgr2=0x7f16300cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f163531d640 1 --2- 172.21.15.148:0/803883081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f16300cebb0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f16300d1520 tx=0x7f162802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f163531d640 1 -- 172.21.15.148:0/803883081 shutdown_connections 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f163531d640 1 --2- 172.21.15.148:0/803883081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f16300cebb0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f163531d640 1 --2- 172.21.15.148:0/803883081 >> v2:172.21.15.188:3300/0 conn(0x7f163015dc20 0x7f16300ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.977+0000 7f163531d640 1 -- 172.21.15.148:0/803883081 >> 172.21.15.148:0/803883081 conn(0x7f16300d0420 msgr2=0x7f16300d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.978+0000 7f163531d640 1 -- 172.21.15.148:0/803883081 shutdown_connections 2024-04-25T10:05:13.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.978+0000 7f163531d640 1 -- 172.21.15.148:0/803883081 wait complete. 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 Processor -- start 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 -- start start 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f1630158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015dc20 0x7f16300b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1630161710 0x7f16300b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b5e50 con 0x7f16300d32f0 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630162500 con 0x7f1630161710 2024-04-25T10:05:13.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f163531d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b5fd0 con 0x7f163015dc20 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f162effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f1630158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f162effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f1630158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47212/0 (socket says 172.21.15.148:47212) 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.979+0000 7f162effd640 1 -- 172.21.15.148:0/1964797973 learned_addr learned my addr 172.21.15.148:0/1964797973 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f162effd640 1 -- 172.21.15.148:0/1964797973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015dc20 msgr2=0x7f16300b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f162effd640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015dc20 0x7f16300b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f162effd640 1 -- 172.21.15.148:0/1964797973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1630161710 msgr2=0x7f16300b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f162effd640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1630161710 0x7f16300b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f162effd640 1 -- 172.21.15.148:0/1964797973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16301f4430 con 0x7f16300d32f0 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f162effd640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f1630158c60 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f162000c820 tx=0x7f162000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620015070 con 0x7f16300d32f0 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620004020 con 0x7f16300d32f0 2024-04-25T10:05:13.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16301f4710 con 0x7f16300d32f0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16200043f0 con 0x7f16300d32f0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.980+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16301f4cd0 con 0x7f16300d32f0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.981+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16301f51c0 con 0x7f16300d32f0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.981+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1620007830 con 0x7f16300d32f0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.982+0000 7f16277fe640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f161006b4d0 0x7f161006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.982+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f162008a120 con 0x7f16300d32f0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.982+0000 7f1627fff640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f161006b4d0 0x7f161006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.982+0000 7f1627fff640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f161006b4d0 0x7f161006d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1618006fd0 tx=0x7f1618006bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.991 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:13.986+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1620058ea0 con 0x7f16300d32f0 2024-04-25T10:05:14.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.156+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1630073a00 con 0x7f16300d32f0 2024-04-25T10:05:14.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.156+0000 7f16277fe640 1 -- 172.21.15.148:0/1964797973 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16200587a0 con 0x7f16300d32f0 2024-04-25T10:05:14.162 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:05:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f161006b4d0 msgr2=0x7f161006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f161006b4d0 0x7f161006d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1618006fd0 tx=0x7f1618006bc0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 msgr2=0x7f1630158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f1630158c60 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f162000c820 tx=0x7f162000cce0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 shutdown_connections 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f161006b4d0 0x7f161006d990 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1630161710 0x7f16300b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015dc20 0x7f16300b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 --2- 172.21.15.148:0/1964797973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16300d32f0 0x7f1630158c60 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 >> 172.21.15.148:0/1964797973 conn(0x7f16300d0420 msgr2=0x7f163015c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 shutdown_connections 2024-04-25T10:05:14.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.158+0000 7f163531d640 1 -- 172.21.15.148:0/1964797973 wait complete. 2024-04-25T10:05:14.174 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.267+0000 7f8625122640 1 Processor -- start 2024-04-25T10:05:14.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.267+0000 7f8625122640 1 -- start start 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f8625122640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86201446d0 0x7f86200b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f8625122640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86200cccc0 con 0x7f8620144d50 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f8625122640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86200d0460 con 0x7f862006e8a0 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f8625122640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200b2b20 con 0x7f86201446d0 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f8617fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8620144d50 0x7f86200cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47052/0 (socket says 172.21.15.148:47052) 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f8617fff640 1 -- 172.21.15.148:0/967583024 learned_addr learned my addr 172.21.15.148:0/967583024 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861ed9d640 1 --2- 172.21.15.148:0/967583024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86201446d0 0x7f86200b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802414982 0 0) 0x7f86200cccc0 con 0x7f8620144d50 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f860c003520 con 0x7f8620144d50 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273800702 0 0) 0x7f86200d0460 con 0x7f862006e8a0 2024-04-25T10:05:14.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86200cccc0 con 0x7f862006e8a0 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89481244 0 0) 0x7f860c003520 con 0x7f8620144d50 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861ed9d640 1 -- 172.21.15.148:0/967583024 >> v1:172.21.15.188:6789/0 conn(0x7f862006e8a0 legacy=0x7f86200cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861ed9d640 1 -- 172.21.15.148:0/967583024 >> v1:172.21.15.148:6789/0 conn(0x7f8620144d50 legacy=0x7f86200cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.268+0000 7f861ed9d640 1 -- 172.21.15.148:0/967583024 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200cc140 con 0x7f86201446d0 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.269+0000 7f861ed9d640 1 --2- 172.21.15.148:0/967583024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86201446d0 0x7f86200b3f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f86100097d0 tx=0x7f861002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41389edc53a9c917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.269+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610037070 con 0x7f86201446d0 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.269+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861002ac30 con 0x7f86201446d0 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.269+0000 7f861e59c640 1 -- 172.21.15.148:0/967583024 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86080015f0 con 0x7f86201446d0 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 -- 172.21.15.148:0/967583024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86201446d0 msgr2=0x7f86200b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 --2- 172.21.15.148:0/967583024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86201446d0 0x7f86200b3f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f86100097d0 tx=0x7f861002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 -- 172.21.15.148:0/967583024 shutdown_connections 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 --2- 172.21.15.148:0/967583024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86201446d0 0x7f86200b3f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 -- 172.21.15.148:0/967583024 >> 172.21.15.148:0/967583024 conn(0x7f862006abe0 msgr2=0x7f862006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 -- 172.21.15.148:0/967583024 shutdown_connections 2024-04-25T10:05:14.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.270+0000 7f8625122640 1 -- 172.21.15.148:0/967583024 wait complete. 2024-04-25T10:05:14.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.271+0000 7f8625122640 1 Processor -- start 2024-04-25T10:05:14.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.271+0000 7f8625122640 1 -- start start 2024-04-25T10:05:14.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8625122640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f862006e8a0 0x7f862015ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8625122640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 0x7f86200c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8625122640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8620144d50 0x7f86200c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8625122640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86200cc700 con 0x7f86201446d0 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8625122640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862006d400 con 0x7f8620144d50 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8625122640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86200cc9e0 con 0x7f862006e8a0 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 0x7f86200c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 0x7f86200c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47222/0 (socket says 172.21.15.148:47222) 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 -- 172.21.15.148:0/2193633204 learned_addr learned my addr 172.21.15.148:0/2193633204 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f861ed9d640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f862006e8a0 0x7f862015ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f861f59e640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8620144d50 0x7f86200c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 -- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f862006e8a0 msgr2=0x7f862015ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f862006e8a0 0x7f862015ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 -- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8620144d50 msgr2=0x7f86200c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8620144d50 0x7f86200c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 -- 172.21.15.148:0/2193633204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200c5590 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.272+0000 7f8617fff640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 0x7f86200c6230 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f860800c550 tx=0x7f860800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.273+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608005ce0 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.273+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8608007070 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.273+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86200c5870 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.273+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608007460 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.273+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86200c5e30 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.274+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8620073a00 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.274+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8608007600 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.275+0000 7f86177fe640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f860006b4d0 0x7f860006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.275+0000 7f861ed9d640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f860006b4d0 0x7f860006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.275+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f860808a030 con 0x7f86201446d0 2024-04-25T10:05:14.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.275+0000 7f861ed9d640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f860006b4d0 0x7f860006d990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8610034000 tx=0x7f861002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.279+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86080586e0 con 0x7f86201446d0 2024-04-25T10:05:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.462+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8620143370 con 0x7f86201446d0 2024-04-25T10:05:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.463+0000 7f86177fe640 1 -- 172.21.15.148:0/2193633204 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v159) v1 ==== 80+0+0 (secure 0 0 0) 0x7f860805c780 con 0x7f86201446d0 2024-04-25T10:05:14.468 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:05:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f860006b4d0 msgr2=0x7f860006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f860006b4d0 0x7f860006d990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8610034000 tx=0x7f861002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 msgr2=0x7f86200c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 0x7f86200c6230 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f860800c550 tx=0x7f860800ca10 comp rx=0 tx=0).stop 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 shutdown_connections 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f860006b4d0 0x7f860006d990 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8620144d50 0x7f86200c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86201446d0 0x7f86200c6230 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 --2- 172.21.15.148:0/2193633204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f862006e8a0 0x7f862015ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 >> 172.21.15.148:0/2193633204 conn(0x7f862006abe0 msgr2=0x7f862006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 shutdown_connections 2024-04-25T10:05:14.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.465+0000 7f8625122640 1 -- 172.21.15.148:0/2193633204 wait complete. 2024-04-25T10:05:14.481 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:05:14.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:14.481 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:14.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:05:14.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:14.507 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:14.562 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:05:14.562 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:14.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:14.581 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:14.581 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:14.617 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:14.617 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:05:14.617 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:14.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:14.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:14.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.749+0000 7fa45589a640 1 Processor -- start 2024-04-25T10:05:14.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.749+0000 7fa45589a640 1 -- start start 2024-04-25T10:05:14.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa45589a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4500d1020 0x7fa4500d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa45589a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500b97a0 con 0x7fa4500b2cb0 2024-04-25T10:05:14.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa45589a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500d1c50 con 0x7fa45015dbe0 2024-04-25T10:05:14.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa45589a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45015dfc0 con 0x7fa4500d1020 2024-04-25T10:05:14.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44e7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa4500b2cb0 0x7fa4500d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47058/0 (socket says 172.21.15.148:47058) 2024-04-25T10:05:14.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44e7fc640 1 -- 172.21.15.148:0/1021878536 learned_addr learned my addr 172.21.15.148:0/1021878536 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153090706 0 0) 0x7fa4500b97a0 con 0x7fa4500b2cb0 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa438003520 con 0x7fa4500b2cb0 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44f7fe640 1 --2- 172.21.15.148:0/1021878536 >> v2:172.21.15.188:3300/0 conn(0x7fa4500d1020 0x7fa4500d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2782963582 0 0) 0x7fa438003520 con 0x7fa4500b2cb0 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 >> v2:172.21.15.188:3300/0 conn(0x7fa4500d1020 msgr2=0x7fa4500d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44dffb640 1 --2- 172.21.15.148:0/1021878536 >> v2:172.21.15.188:3300/0 conn(0x7fa4500d1020 0x7fa4500d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.750+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 >> v1:172.21.15.188:6789/0 conn(0x7fa45015dbe0 legacy=0x7fa4500d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500d1f30 con 0x7fa4500b2cb0 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa444001670 con 0x7fa4500b2cb0 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 >> v1:172.21.15.148:6789/0 conn(0x7fa4500b2cb0 legacy=0x7fa4500d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa438004800 0x7fa438004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4380051c0 0x7fa43800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43800c0e0 con 0x7fa4380039b0 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43800c290 con 0x7fa438004800 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43800c490 con 0x7fa4380051c0 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44f7fe640 1 --2- 172.21.15.148:0/1021878536 >> v2:172.21.15.188:3300/0 conn(0x7fa4500d1020 0x7fa4500d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44effd640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44e7fc640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa438004800 0x7fa438004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44f7fe640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4380051c0 0x7fa43800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44effd640 1 -- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4380051c0 msgr2=0x7fa43800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44effd640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4380051c0 0x7fa43800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44effd640 1 -- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa438004800 msgr2=0x7fa438004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44effd640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa438004800 0x7fa438004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44effd640 1 -- 172.21.15.148:0/1021878536 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438003520 con 0x7fa4380039b0 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44e7fc640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa438004800 0x7fa438004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44f7fe640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4380051c0 0x7fa43800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.751+0000 7fa44f7fe640 1 -- 172.21.15.148:0/1021878536 reap_dead start 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa44effd640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fa440001750 tx=0x7fa438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4d3ca0005e503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440001370 con 0x7fa4380039b0 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4400052d0 con 0x7fa4380039b0 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa44dffb640 1 -- 172.21.15.148:0/1021878536 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44002abc0 con 0x7fa4380039b0 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 -- 172.21.15.148:0/1021878536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4380039b0 msgr2=0x7fa438003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fa440001750 tx=0x7fa438005830 comp rx=0 tx=0).stop 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 -- 172.21.15.148:0/1021878536 shutdown_connections 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 --2- 172.21.15.148:0/1021878536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4380039b0 0x7fa438003dd0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 -- 172.21.15.148:0/1021878536 >> 172.21.15.148:0/1021878536 conn(0x7fa450061a20 msgr2=0x7fa450061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 -- 172.21.15.148:0/1021878536 shutdown_connections 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.752+0000 7fa45589a640 1 -- 172.21.15.148:0/1021878536 wait complete. 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.754+0000 7fa45589a640 1 Processor -- start 2024-04-25T10:05:14.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.754+0000 7fa45589a640 1 -- start start 2024-04-25T10:05:14.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.754+0000 7fa45589a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4380039b0 0x7fa4500b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.754+0000 7fa45589a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 0x7fa450142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.754+0000 7fa45589a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa450142960 0x7fa450142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa45589a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500b92a0 con 0x7fa4500b8740 2024-04-25T10:05:14.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa45589a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500d1a00 con 0x7fa4380039b0 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa45589a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500b9420 con 0x7fa450142960 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 0x7fa450142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 0x7fa450142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47246/0 (socket says 172.21.15.148:47246) 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 -- 172.21.15.148:0/1418288945 learned_addr learned my addr 172.21.15.148:0/1418288945 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44f7fe640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa450142960 0x7fa450142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44effd640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4380039b0 0x7fa4500b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 -- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa450142960 msgr2=0x7fa450142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa450142960 0x7fa450142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 -- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4380039b0 msgr2=0x7fa4500b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4380039b0 0x7fa4500b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 -- 172.21.15.148:0/1418288945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450157430 con 0x7fa4500b8740 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44effd640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4380039b0 0x7fa4500b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44f7fe640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa450142960 0x7fa450142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa44e7fc640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 0x7fa450142420 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e880 tx=0x7fa44400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400c870 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444018070 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa450156b90 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400cc10 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.755+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa450157150 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.756+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450073990 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.757+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa44401d460 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.757+0000 7fa437fff640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.757+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa44408a1f0 con 0x7fa4500b8740 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.757+0000 7fa44effd640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.758+0000 7fa44effd640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa43c06b4d0 0x7fa43c06d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa440032000 tx=0x7fa44002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.761+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa444058ee0 con 0x7fa4500b8740 2024-04-25T10:05:14.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.974+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4500b9060 con 0x7fa4500b8740 2024-04-25T10:05:14.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.975+0000 7fa437fff640 1 -- 172.21.15.148:0/1418288945 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fa4440587e0 con 0x7fa4500b8740 2024-04-25T10:05:14.980 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:14.981 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:30.667288+0000","modified":"2024-04-25T10:04:32.676174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10365":{"gid":10365,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3286540466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3286540466},{"type":"v1","addr":"172.21.15.148:6837","nonce":3286540466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:05:14.981 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 93 2024-04-25T10:05:14.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa43c06b4d0 msgr2=0x7fa43c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa43c06b4d0 0x7fa43c06d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa440032000 tx=0x7fa44002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 msgr2=0x7fa450142420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 0x7fa450142420 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e880 tx=0x7fa44400ed40 comp rx=0 tx=0).stop 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 shutdown_connections 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa450142960 0x7fa450142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4500b8740 0x7fa450142420 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 --2- 172.21.15.148:0/1418288945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4380039b0 0x7fa4500b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 >> 172.21.15.148:0/1418288945 conn(0x7fa450061a20 msgr2=0x7fa450157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 shutdown_connections 2024-04-25T10:05:14.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:14.977+0000 7fa45589a640 1 -- 172.21.15.148:0/1418288945 wait complete. 2024-04-25T10:05:14.993 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.085+0000 7f18e245d640 1 Processor -- start 2024-04-25T10:05:15.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.085+0000 7f18e245d640 1 -- start start 2024-04-25T10:05:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.086+0000 7f18e245d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18dc0cc8e0 0x7f18dc1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.086+0000 7f18e245d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.086+0000 7f18e245d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0cce50 con 0x7f18dc15dc90 2024-04-25T10:05:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.086+0000 7f18e245d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0d21c0 con 0x7f18dc0cc8e0 2024-04-25T10:05:15.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.086+0000 7f18e245d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc161af0 con 0x7f18dc0d1810 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18e09fa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18db7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18dc0cc8e0 0x7f18dc1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18dbfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f18dc15dc90 0x7f18dc15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43386/0 (socket says 172.21.15.148:43386) 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18dbfff640 1 -- 172.21.15.148:0/3553786050 learned_addr learned my addr 172.21.15.148:0/3553786050 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18daffd640 1 -- 172.21.15.148:0/3553786050 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829062208 0 0) 0x7f18dc0cce50 con 0x7f18dc15dc90 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18daffd640 1 -- 172.21.15.148:0/3553786050 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c4003520 con 0x7f18dc15dc90 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18e09fa640 1 -- 172.21.15.148:0/3553786050 >> v2:172.21.15.188:3300/0 conn(0x7f18dc0cc8e0 msgr2=0x7f18dc1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18e09fa640 1 --2- 172.21.15.148:0/3553786050 >> v2:172.21.15.188:3300/0 conn(0x7f18dc0cc8e0 0x7f18dc1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18e09fa640 1 -- 172.21.15.148:0/3553786050 >> v1:172.21.15.188:6789/0 conn(0x7f18dc15dc90 legacy=0x7f18dc15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.087+0000 7f18e09fa640 1 -- 172.21.15.148:0/3553786050 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc161dd0 con 0x7f18dc0d1810 2024-04-25T10:05:15.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.088+0000 7f18e09fa640 1 --2- 172.21.15.148:0/3553786050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc0d1c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d90 tx=0x7f18d402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb426d09a090c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.088+0000 7f18daffd640 1 -- 172.21.15.148:0/3553786050 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4031600 con 0x7f18dc0d1810 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.088+0000 7f18daffd640 1 -- 172.21.15.148:0/3553786050 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d40317a0 con 0x7f18dc0d1810 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.088+0000 7f18daffd640 1 -- 172.21.15.148:0/3553786050 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4031b50 con 0x7f18dc0d1810 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 -- 172.21.15.148:0/3553786050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 msgr2=0x7f18dc0d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 --2- 172.21.15.148:0/3553786050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc0d1c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d90 tx=0x7f18d402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 -- 172.21.15.148:0/3553786050 shutdown_connections 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 --2- 172.21.15.148:0/3553786050 >> v2:172.21.15.188:3300/0 conn(0x7f18dc0cc8e0 0x7f18dc1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 --2- 172.21.15.148:0/3553786050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc0d1c80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 -- 172.21.15.148:0/3553786050 >> 172.21.15.148:0/3553786050 conn(0x7f18dc06f590 msgr2=0x7f18dc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 -- 172.21.15.148:0/3553786050 shutdown_connections 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.089+0000 7f18e245d640 1 -- 172.21.15.148:0/3553786050 wait complete. 2024-04-25T10:05:15.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.090+0000 7f18e245d640 1 Processor -- start 2024-04-25T10:05:15.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.090+0000 7f18e245d640 1 -- start start 2024-04-25T10:05:15.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc0cc8e0 0x7f18dc0b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 0x7f18dc0c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc15e070 con 0x7f18dc15dc90 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0cd130 con 0x7f18dc0cc8e0 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc15e1f0 con 0x7f18dc0d1810 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 0x7f18dc0c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 0x7f18dc0c9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47260/0 (socket says 172.21.15.148:47260) 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 -- 172.21.15.148:0/3184792156 learned_addr learned my addr 172.21.15.148:0/3184792156 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18db7fe640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18dbfff640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc0cc8e0 0x7f18dc0b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.098 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 -- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 msgr2=0x7f18dc14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc14fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 -- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc0cc8e0 msgr2=0x7f18dc0b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc0cc8e0 0x7f18dc0b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 -- 172.21.15.148:0/3184792156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0ca020 con 0x7f18dc15dc90 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e09fa640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 0x7f18dc0c9640 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d90 tx=0x7f18d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4040070 con 0x7f18dc15dc90 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4004260 con 0x7f18dc15dc90 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4009450 con 0x7f18dc15dc90 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.091+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc0b46f0 con 0x7f18dc15dc90 2024-04-25T10:05:15.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.092+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc0b4cb0 con 0x7f18dc15dc90 2024-04-25T10:05:15.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.093+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc073990 con 0x7f18dc15dc90 2024-04-25T10:05:15.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.093+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18d4039040 con 0x7f18dc15dc90 2024-04-25T10:05:15.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.093+0000 7f18d8ff9640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f18c806b4d0 0x7f18c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.093+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18d40ab370 con 0x7f18dc15dc90 2024-04-25T10:05:15.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.093+0000 7f18dbfff640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f18c806b4d0 0x7f18c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.094+0000 7f18dbfff640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f18c806b4d0 0x7f18c806d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005880 tx=0x7f18cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.098+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18d4079ad0 con 0x7f18dc15dc90 2024-04-25T10:05:15.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.271+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18dc0b4e80 con 0x7f18dc15dc90 2024-04-25T10:05:15.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.272+0000 7f18d8ff9640 1 -- 172.21.15.148:0/3184792156 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f18d407db70 con 0x7f18dc15dc90 2024-04-25T10:05:15.278 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:15.278 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":159,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:32.326979+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f18c806b4d0 msgr2=0x7f18c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f18c806b4d0 0x7f18c806d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f18cc005880 tx=0x7f18cc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 msgr2=0x7f18dc0c9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 0x7f18dc0c9640 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d90 tx=0x7f18d4004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 shutdown_connections 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f18c806b4d0 0x7f18c806d990 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18dc15dc90 0x7f18dc0c9640 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18dc0d1810 0x7f18dc14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 --2- 172.21.15.148:0/3184792156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18dc0cc8e0 0x7f18dc0b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 >> 172.21.15.148:0/3184792156 conn(0x7f18dc06f590 msgr2=0x7f18dc0ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 shutdown_connections 2024-04-25T10:05:15.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.274+0000 7f18e245d640 1 -- 172.21.15.148:0/3184792156 wait complete. 2024-04-25T10:05:15.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:28.876622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:29.635420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:15.292 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:15.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.386+0000 7f76684d1640 1 Processor -- start 2024-04-25T10:05:15.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.386+0000 7f76684d1640 1 -- start start 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f76684d1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f766015dac0 0x7f76600d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f76684d1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76600d2420 con 0x7f76600d0ab0 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f76684d1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76600d2700 con 0x7f76600d1f30 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f76684d1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76600d3030 con 0x7f766015dac0 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f7666a6e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f76600d0ab0 0x7f76600b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47064/0 (socket says 172.21.15.148:47064) 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f7666a6e640 1 -- 172.21.15.148:0/4068236647 learned_addr learned my addr 172.21.15.148:0/4068236647 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:15.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f766626d640 1 --2- 172.21.15.148:0/4068236647 >> v2:172.21.15.188:3300/0 conn(0x7f766015dac0 0x7f76600d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856572155 0 0) 0x7f76600d2420 con 0x7f76600d0ab0 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.387+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c003520 con 0x7f76600d0ab0 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108562066 0 0) 0x7f76600d2700 con 0x7f76600d1f30 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76600d2420 con 0x7f76600d1f30 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2821820810 0 0) 0x7f764c003520 con 0x7f76600d0ab0 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 >> v2:172.21.15.188:3300/0 conn(0x7f766015dac0 msgr2=0x7f76600d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 --2- 172.21.15.148:0/4068236647 >> v2:172.21.15.188:3300/0 conn(0x7f766015dac0 0x7f76600d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 >> v1:172.21.15.188:6789/0 conn(0x7f76600d1f30 legacy=0x7f76600d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600d29e0 con 0x7f76600d0ab0 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7658001970 con 0x7f76600d0ab0 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 >> v1:172.21.15.148:6789/0 conn(0x7f76600d0ab0 legacy=0x7f76600b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c003b90 0x7f764c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f764c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0051c0 0x7f764c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76600d3030 con 0x7f764c004800 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00bfe0 con 0x7f764c003b90 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c00c1e0 con 0x7f764c0051c0 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7665a6c640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f764c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f766626d640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c003b90 0x7f764c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7666a6e640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0051c0 0x7f764c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7665a6c640 1 -- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0051c0 msgr2=0x7f764c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7665a6c640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0051c0 0x7f764c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7665a6c640 1 -- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c003b90 msgr2=0x7f764c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7665a6c640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f764c003b90 0x7f764c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7665a6c640 1 -- 172.21.15.148:0/4068236647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c003520 con 0x7f764c004800 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7666a6e640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f764c0051c0 0x7f764c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.388+0000 7f7666a6e640 1 -- 172.21.15.148:0/4068236647 reap_dead start 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.389+0000 7f7665a6c640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f764c004c80 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f76500018f0 tx=0x7f764c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d139033fa727d5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.389+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650001540 con 0x7f764c004800 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.389+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765002ac40 con 0x7f764c004800 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.389+0000 7f766526b640 1 -- 172.21.15.148:0/4068236647 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500052f0 con 0x7f764c004800 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.389+0000 7f76684d1640 1 -- 172.21.15.148:0/4068236647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 msgr2=0x7f764c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.389+0000 7f76684d1640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f764c004c80 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f76500018f0 tx=0x7f764c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.390+0000 7f76684d1640 1 -- 172.21.15.148:0/4068236647 shutdown_connections 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.390+0000 7f76684d1640 1 --2- 172.21.15.148:0/4068236647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f764c004c80 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.390+0000 7f76684d1640 1 -- 172.21.15.148:0/4068236647 >> 172.21.15.148:0/4068236647 conn(0x7f7660061c30 msgr2=0x7f766006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.390+0000 7f76684d1640 1 -- 172.21.15.148:0/4068236647 shutdown_connections 2024-04-25T10:05:15.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.390+0000 7f76684d1640 1 -- 172.21.15.148:0/4068236647 wait complete. 2024-04-25T10:05:15.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 Processor -- start 2024-04-25T10:05:15.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 -- start start 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f7660155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660156ba0 0x7f76600cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76601f7930 0x7f76600c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76600b97d0 con 0x7f764c004800 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766015dea0 con 0x7f76601f7930 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.391+0000 7f76684d1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76600b9950 con 0x7f7660156ba0 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f7660155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f7660155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47276/0 (socket says 172.21.15.148:47276) 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 -- 172.21.15.148:0/237121092 learned_addr learned my addr 172.21.15.148:0/237121092 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f7665a6c640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660156ba0 0x7f76600cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f7666a6e640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76601f7930 0x7f76600c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 -- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660156ba0 msgr2=0x7f76600cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660156ba0 0x7f76600cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 -- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76601f7930 msgr2=0x7f76600c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76601f7930 0x7f76600c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 -- 172.21.15.148:0/237121092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660154c60 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f7666a6e640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76601f7930 0x7f76600c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f766626d640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f7660155db0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f765c011aa0 tx=0x7f765c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00fb80 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c00fd20 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c014660 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7660154f40 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.392+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76601f7130 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.393+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7660073a00 con 0x7f764c004800 2024-04-25T10:05:15.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.394+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f765c012040 con 0x7f764c004800 2024-04-25T10:05:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.394+0000 7f7656ffd640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f764406b4d0 0x7f764406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.394+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f765c08a910 con 0x7f764c004800 2024-04-25T10:05:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.394+0000 7f7665a6c640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f764406b4d0 0x7f764406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.394+0000 7f7665a6c640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f764406b4d0 0x7f764406d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7650032000 tx=0x7f7650009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.398+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f765c019030 con 0x7f764c004800 2024-04-25T10:05:15.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.570+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76600d05f0 con 0x7f764c004800 2024-04-25T10:05:15.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.571+0000 7f7656ffd640 1 -- 172.21.15.148:0/237121092 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f765c059690 con 0x7f764c004800 2024-04-25T10:05:15.576 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:05:15.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.572+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f764406b4d0 msgr2=0x7f764406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.572+0000 7f76684d1640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f764406b4d0 0x7f764406d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7650032000 tx=0x7f7650009030 comp rx=0 tx=0).stop 2024-04-25T10:05:15.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.572+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 msgr2=0x7f7660155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.572+0000 7f76684d1640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f7660155db0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f765c011aa0 tx=0x7f765c011f60 comp rx=0 tx=0).stop 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 shutdown_connections 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f764406b4d0 0x7f764406d990 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76601f7930 0x7f76600c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7660156ba0 0x7f76600cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 --2- 172.21.15.148:0/237121092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f764c004800 0x7f7660155db0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 >> 172.21.15.148:0/237121092 conn(0x7f7660061c30 msgr2=0x7f76600ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 shutdown_connections 2024-04-25T10:05:15.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.573+0000 7f76684d1640 1 -- 172.21.15.148:0/237121092 wait complete. 2024-04-25T10:05:15.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.684+0000 7f4975a7d640 1 Processor -- start 2024-04-25T10:05:15.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.684+0000 7f4975a7d640 1 -- start start 2024-04-25T10:05:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.685+0000 7f4975a7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f497006c330 0x7f49700b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.685+0000 7f4975a7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 0x7f49700b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.685+0000 7f4975a7d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49700d1d50 con 0x7f4970158230 2024-04-25T10:05:15.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.685+0000 7f4975a7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4970158610 con 0x7f497006c330 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.685+0000 7f4975a7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497006bea0 con 0x7f497006c7e0 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.685+0000 7f496ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 0x7f49700b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496effd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f497006c330 0x7f49700b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 0x7f49700b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:40216/0 (socket says 172.21.15.148:40216) 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496ffff640 1 -- 172.21.15.148:0/1997049707 learned_addr learned my addr 172.21.15.148:0/1997049707 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496f7fe640 1 --1- 172.21.15.148:0/1997049707 >> v1:172.21.15.188:6789/0 conn(0x7f4970158230 0x7f49700d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43398/0 (socket says 172.21.15.148:43398) 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496ffff640 1 -- 172.21.15.148:0/1997049707 >> v2:172.21.15.188:3300/0 conn(0x7f497006c330 msgr2=0x7f49700b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496ffff640 1 --2- 172.21.15.148:0/1997049707 >> v2:172.21.15.188:3300/0 conn(0x7f497006c330 0x7f49700b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496ffff640 1 -- 172.21.15.148:0/1997049707 >> v1:172.21.15.188:6789/0 conn(0x7f4970158230 legacy=0x7f49700d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496ffff640 1 -- 172.21.15.148:0/1997049707 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700d2020 con 0x7f497006c7e0 2024-04-25T10:05:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.686+0000 7f496effd640 1 --2- 172.21.15.148:0/1997049707 >> v2:172.21.15.188:3300/0 conn(0x7f497006c330 0x7f49700b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f496ffff640 1 --2- 172.21.15.148:0/1997049707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 0x7f49700b53a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4968009e50 tx=0x7f496802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f7860a2126e975 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f496e7fc640 1 -- 172.21.15.148:0/1997049707 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968036070 con 0x7f497006c7e0 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f496e7fc640 1 -- 172.21.15.148:0/1997049707 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968031600 con 0x7f497006c7e0 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f496e7fc640 1 -- 172.21.15.148:0/1997049707 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680319d0 con 0x7f497006c7e0 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 -- 172.21.15.148:0/1997049707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 msgr2=0x7f49700b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 --2- 172.21.15.148:0/1997049707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 0x7f49700b53a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4968009e50 tx=0x7f496802a710 comp rx=0 tx=0).stop 2024-04-25T10:05:15.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 -- 172.21.15.148:0/1997049707 shutdown_connections 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 --2- 172.21.15.148:0/1997049707 >> v2:172.21.15.188:3300/0 conn(0x7f497006c330 0x7f49700b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 --2- 172.21.15.148:0/1997049707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c7e0 0x7f49700b53a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 -- 172.21.15.148:0/1997049707 >> 172.21.15.148:0/1997049707 conn(0x7f4970061600 msgr2=0x7f49700619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 -- 172.21.15.148:0/1997049707 shutdown_connections 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.687+0000 7f4975a7d640 1 -- 172.21.15.148:0/1997049707 wait complete. 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.689+0000 7f4975a7d640 1 Processor -- start 2024-04-25T10:05:15.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.689+0000 7f4975a7d640 1 -- start start 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.689+0000 7f4975a7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c330 0x7f497015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.689+0000 7f4975a7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f497006c7e0 0x7f49700d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.689+0000 7f4975a7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 0x7f497014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4975a7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700d25e0 con 0x7f4970158230 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4975a7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497006bb30 con 0x7f497006c7e0 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4975a7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497015e320 con 0x7f497006c330 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 0x7f497014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 0x7f497014ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47292/0 (socket says 172.21.15.148:47292) 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 -- 172.21.15.148:0/2212800958 learned_addr learned my addr 172.21.15.148:0/2212800958 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496f7fe640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c330 0x7f497015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496effd640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f497006c7e0 0x7f49700d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 -- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c330 msgr2=0x7f497015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c330 0x7f497015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 -- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f497006c7e0 msgr2=0x7f49700d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f497006c7e0 0x7f49700d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 -- 172.21.15.148:0/2212800958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970158d70 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f496ffff640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 0x7f497014ec10 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f4968034040 tx=0x7f4968004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968036050 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968042070 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4970159050 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968009510 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.690+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970159610 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.691+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497006de50 con 0x7f4970158230 2024-04-25T10:05:15.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.692+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4968007c70 con 0x7f4970158230 2024-04-25T10:05:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.692+0000 7f4957fff640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495c06b4d0 0x7f495c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.692+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49680ab1d0 con 0x7f4970158230 2024-04-25T10:05:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.692+0000 7f496f7fe640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495c06b4d0 0x7f495c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.693+0000 7f496f7fe640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495c06b4d0 0x7f495c06d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4960000f60 tx=0x7f4960007cc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.696+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4968045360 con 0x7f4970158230 2024-04-25T10:05:15.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.867+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f497015bf60 con 0x7f4970158230 2024-04-25T10:05:15.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.867+0000 7f4957fff640 1 -- 172.21.15.148:0/2212800958 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4968079ed0 con 0x7f4970158230 2024-04-25T10:05:15.873 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495c06b4d0 msgr2=0x7f495c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495c06b4d0 0x7f495c06d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4960000f60 tx=0x7f4960007cc0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 msgr2=0x7f497014ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 0x7f497014ec10 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f4968034040 tx=0x7f4968004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 shutdown_connections 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f495c06b4d0 0x7f495c06d990 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4970158230 0x7f497014ec10 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f497006c7e0 0x7f49700d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 --2- 172.21.15.148:0/2212800958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f497006c330 0x7f497015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 >> 172.21.15.148:0/2212800958 conn(0x7f4970061600 msgr2=0x7f49700ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 shutdown_connections 2024-04-25T10:05:15.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.869+0000 7f4975a7d640 1 -- 172.21.15.148:0/2212800958 wait complete. 2024-04-25T10:05:15.885 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:15.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.979+0000 7f5f321ce640 1 Processor -- start 2024-04-25T10:05:15.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.979+0000 7f5f321ce640 1 -- start start 2024-04-25T10:05:15.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f321ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c15bb20 0x7f5f2c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f321ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0d00e0 0x7f5f2c0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f321ce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0cb190 con 0x7f5f2c0ccdd0 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f321ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cb470 con 0x7f5f2c0d00e0 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f321ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cb750 con 0x7f5f2c15bb20 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f22ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5f2c0ccdd0 0x7f5f2c0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47066/0 (socket says 172.21.15.148:47066) 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f22ffd640 1 -- 172.21.15.148:0/2033010311 learned_addr learned my addr 172.21.15.148:0/2033010311 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.980+0000 7f5f2b7fe640 1 --2- 172.21.15.148:0/2033010311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c15bb20 0x7f5f2c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2bfff640 1 --2- 172.21.15.148:0/2033010311 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0d00e0 0x7f5f2c0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2affd640 1 -- 172.21.15.148:0/2033010311 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130396784 0 0) 0x7f5f2c0cb190 con 0x7f5f2c0ccdd0 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2affd640 1 -- 172.21.15.148:0/2033010311 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f18003520 con 0x7f5f2c0ccdd0 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2b7fe640 1 -- 172.21.15.148:0/2033010311 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0d00e0 msgr2=0x7f5f2c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2b7fe640 1 --2- 172.21.15.148:0/2033010311 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0d00e0 0x7f5f2c0ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2b7fe640 1 -- 172.21.15.148:0/2033010311 >> v1:172.21.15.148:6789/0 conn(0x7f5f2c0ccdd0 legacy=0x7f5f2c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2b7fe640 1 -- 172.21.15.148:0/2033010311 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0cba30 con 0x7f5f2c15bb20 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.981+0000 7f5f2bfff640 1 --2- 172.21.15.148:0/2033010311 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0d00e0 0x7f5f2c0ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f2b7fe640 1 --2- 172.21.15.148:0/2033010311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c15bb20 0x7f5f2c0cc230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0097a0 tx=0x7f5f1c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93858ef9fcff4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f2affd640 1 -- 172.21.15.148:0/2033010311 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c037070 con 0x7f5f2c15bb20 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f2affd640 1 -- 172.21.15.148:0/2033010311 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c02ae50 con 0x7f5f2c15bb20 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f2affd640 1 -- 172.21.15.148:0/2033010311 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c032640 con 0x7f5f2c15bb20 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 -- 172.21.15.148:0/2033010311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c15bb20 msgr2=0x7f5f2c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2033010311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c15bb20 0x7f5f2c0cc230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0097a0 tx=0x7f5f1c02a100 comp rx=0 tx=0).stop 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 -- 172.21.15.148:0/2033010311 shutdown_connections 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2033010311 >> v2:172.21.15.188:3300/0 conn(0x7f5f2c0d00e0 0x7f5f2c0ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2033010311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c15bb20 0x7f5f2c0cc230 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 -- 172.21.15.148:0/2033010311 >> 172.21.15.148:0/2033010311 conn(0x7f5f2c0b2390 msgr2=0x7f5f2c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 -- 172.21.15.148:0/2033010311 shutdown_connections 2024-04-25T10:05:15.990 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.982+0000 7f5f321ce640 1 -- 172.21.15.148:0/2033010311 wait complete. 2024-04-25T10:05:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.983+0000 7f5f321ce640 1 Processor -- start 2024-04-25T10:05:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 -- start start 2024-04-25T10:05:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0ccdd0 0x7f5f2c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0d00e0 0x7f5f2c0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.992 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 0x7f5f2c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c15bf00 con 0x7f5f2c15bb20 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cc770 con 0x7f5f2c0ccdd0 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f321ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c15c080 con 0x7f5f2c0d00e0 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.984+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 0x7f5f2c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 0x7f5f2c0c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47294/0 (socket says 172.21.15.148:47294) 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 -- 172.21.15.148:0/2775739268 learned_addr learned my addr 172.21.15.148:0/2775739268 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f22ffd640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0d00e0 0x7f5f2c0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2b7fe640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0ccdd0 0x7f5f2c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 -- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0d00e0 msgr2=0x7f5f2c0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0d00e0 0x7f5f2c0c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 -- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0ccdd0 msgr2=0x7f5f2c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0ccdd0 0x7f5f2c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 -- 172.21.15.148:0/2775739268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c159690 con 0x7f5f2c15bb20 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f2bfff640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 0x7f5f2c0c9d40 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009f00 tx=0x7f5f2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.993 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24018030 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2400ce50 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c159970 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003bb0 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.985+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c159f30 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.986+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c073a00 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.987+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f2401b070 con 0x7f5f2c15bb20 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.987+0000 7f5f28ff9640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.987+0000 7f5f2b7fe640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.994 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.987+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f2408acf0 con 0x7f5f2c15bb20 2024-04-25T10:05:15.995 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.988+0000 7f5f2b7fe640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c034000 tx=0x7f5f1c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:15.991+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f2408f070 con 0x7f5f2c15bb20 2024-04-25T10:05:16.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.161+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f2c0686d0 con 0x7f5f2c15bb20 2024-04-25T10:05:16.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.161+0000 7f5f28ff9640 1 -- 172.21.15.148:0/2775739268 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f24010040 con 0x7f5f2c15bb20 2024-04-25T10:05:16.167 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0c06b500 msgr2=0x7f5f0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c034000 tx=0x7f5f1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 msgr2=0x7f5f2c0c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 0x7f5f2c0c9d40 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009f00 tx=0x7f5f2400c490 comp rx=0 tx=0).stop 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 shutdown_connections 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f2c15bb20 0x7f5f2c0c9d40 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2c0d00e0 0x7f5f2c0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 --2- 172.21.15.148:0/2775739268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f2c0ccdd0 0x7f5f2c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 >> 172.21.15.148:0/2775739268 conn(0x7f5f2c0b2390 msgr2=0x7f5f2c06f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 shutdown_connections 2024-04-25T10:05:16.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.163+0000 7f5f321ce640 1 -- 172.21.15.148:0/2775739268 wait complete. 2024-04-25T10:05:16.180 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.272+0000 7f5badf0a640 1 Processor -- start 2024-04-25T10:05:16.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.272+0000 7f5badf0a640 1 -- start start 2024-04-25T10:05:16.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5badf0a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5ba815dd30 0x7f5ba80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5badf0a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5ba80d0ad0 0x7f5ba80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5badf0a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba80d1df0 con 0x7f5ba80b9770 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5badf0a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80d20d0 con 0x7f5ba815dd30 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5badf0a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba80d23b0 con 0x7f5ba80d0ad0 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5ba815dd30 0x7f5ba80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5ba815dd30 0x7f5ba80b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47298/0 (socket says 172.21.15.148:47298) 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 -- 172.21.15.148:0/964092928 learned_addr learned my addr 172.21.15.148:0/964092928 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba7fff640 1 --2- 172.21.15.148:0/964092928 >> v2:172.21.15.188:3300/0 conn(0x7f5ba80d0ad0 0x7f5ba80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 -- 172.21.15.148:0/964092928 >> v2:172.21.15.188:3300/0 conn(0x7f5ba80d0ad0 msgr2=0x7f5ba80b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 --2- 172.21.15.148:0/964092928 >> v2:172.21.15.188:3300/0 conn(0x7f5ba80d0ad0 0x7f5ba80b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 -- 172.21.15.148:0/964092928 >> v1:172.21.15.188:6789/0 conn(0x7f5ba80b9770 legacy=0x7f5ba80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.273+0000 7f5ba77fe640 1 -- 172.21.15.148:0/964092928 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80d2690 con 0x7f5ba815dd30 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.274+0000 7f5ba77fe640 1 --2- 172.21.15.148:0/964092928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba815dd30 0x7f5ba80b9230 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80ce200 tx=0x7f5b9002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc79304a7ef8060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.274+0000 7f5ba67fc640 1 -- 172.21.15.148:0/964092928 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90037070 con 0x7f5ba815dd30 2024-04-25T10:05:16.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.274+0000 7f5ba67fc640 1 -- 172.21.15.148:0/964092928 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b90032440 con 0x7f5ba815dd30 2024-04-25T10:05:16.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.274+0000 7f5ba67fc640 1 -- 172.21.15.148:0/964092928 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90032810 con 0x7f5ba815dd30 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 -- 172.21.15.148:0/964092928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba815dd30 msgr2=0x7f5ba80b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 --2- 172.21.15.148:0/964092928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba815dd30 0x7f5ba80b9230 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80ce200 tx=0x7f5b9002a150 comp rx=0 tx=0).stop 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 -- 172.21.15.148:0/964092928 shutdown_connections 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 --2- 172.21.15.148:0/964092928 >> v2:172.21.15.188:3300/0 conn(0x7f5ba80d0ad0 0x7f5ba80b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 --2- 172.21.15.148:0/964092928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba815dd30 0x7f5ba80b9230 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 -- 172.21.15.148:0/964092928 >> 172.21.15.148:0/964092928 conn(0x7f5ba8061c30 msgr2=0x7f5ba806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 -- 172.21.15.148:0/964092928 shutdown_connections 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.275+0000 7f5badf0a640 1 -- 172.21.15.148:0/964092928 wait complete. 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.276+0000 7f5badf0a640 1 Processor -- start 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.276+0000 7f5badf0a640 1 -- start start 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5badf0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 0x7f5ba814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5badf0a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ba80d0ad0 0x7f5ba815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5badf0a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ba815dd30 0x7f5ba815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5badf0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80d1b90 con 0x7f5ba80b9770 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5badf0a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80d1a10 con 0x7f5ba80d0ad0 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5badf0a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba80d2e90 con 0x7f5ba815dd30 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 0x7f5ba814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 0x7f5ba814dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47302/0 (socket says 172.21.15.148:47302) 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 -- 172.21.15.148:0/2000036887 learned_addr learned my addr 172.21.15.148:0/2000036887 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba7fff640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ba815dd30 0x7f5ba815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba6ffd640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ba80d0ad0 0x7f5ba815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 -- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ba815dd30 msgr2=0x7f5ba815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ba815dd30 0x7f5ba815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 -- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ba80d0ad0 msgr2=0x7f5ba815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ba80d0ad0 0x7f5ba815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.277+0000 7f5ba77fe640 1 -- 172.21.15.148:0/2000036887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8159b30 con 0x7f5ba80b9770 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.278+0000 7f5ba77fe640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 0x7f5ba814dbc0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f5b9002a190 tx=0x7f5b900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.278+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90041020 con 0x7f5ba80b9770 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.278+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b900306c0 con 0x7f5ba80b9770 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.278+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba8159e10 con 0x7f5ba80b9770 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.278+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90037040 con 0x7f5ba80b9770 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.278+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba815f5c0 con 0x7f5ba80b9770 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.279+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8073990 con 0x7f5ba80b9770 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.279+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b90044020 con 0x7f5ba80b9770 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.279+0000 7f5b9ffff640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b8806b4d0 0x7f5b8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.280+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b900aab30 con 0x7f5ba80b9770 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.280+0000 7f5ba6ffd640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b8806b4d0 0x7f5b8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.280+0000 7f5ba6ffd640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b8806b4d0 0x7f5b8806d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5ba81f7020 tx=0x7f5b98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.284+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b9004b030 con 0x7f5ba80b9770 2024-04-25T10:05:16.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.496+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ba80b8ae0 con 0x7f5ba80b9770 2024-04-25T10:05:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.496+0000 7f5b9ffff640 1 -- 172.21.15.148:0/2000036887 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f5b900798b0 con 0x7f5ba80b9770 2024-04-25T10:05:16.502 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:16.502 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:30.667288+0000","modified":"2024-04-25T10:04:32.676174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10365":{"gid":10365,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3286540466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3286540466},{"type":"v1","addr":"172.21.15.148:6837","nonce":3286540466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:05:16.502 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 93 2024-04-25T10:05:16.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.498+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b8806b4d0 msgr2=0x7f5b8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.498+0000 7f5badf0a640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b8806b4d0 0x7f5b8806d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5ba81f7020 tx=0x7f5b98009040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.498+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 msgr2=0x7f5ba814dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.498+0000 7f5badf0a640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 0x7f5ba814dbc0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f5b9002a190 tx=0x7f5b900046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 shutdown_connections 2024-04-25T10:05:16.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5b8806b4d0 0x7f5b8806d990 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ba815dd30 0x7f5ba815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ba80d0ad0 0x7f5ba815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 --2- 172.21.15.148:0/2000036887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ba80b9770 0x7f5ba814dbc0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 >> 172.21.15.148:0/2000036887 conn(0x7f5ba8061c30 msgr2=0x7f5ba80d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 shutdown_connections 2024-04-25T10:05:16.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.499+0000 7f5badf0a640 1 -- 172.21.15.148:0/2000036887 wait complete. 2024-04-25T10:05:16.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.606+0000 7fcc04a12640 1 Processor -- start 2024-04-25T10:05:16.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.607+0000 7fcc04a12640 1 -- start start 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.607+0000 7fcc04a12640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc0015dc90 0x7fcc001625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.607+0000 7fcc04a12640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc000cce50 con 0x7fcc000d1810 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.607+0000 7fcc04a12640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc00161af0 con 0x7fcc000cc8e0 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.607+0000 7fcc04a12640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc00161dd0 con 0x7fcc0015dc90 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfe59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc0015dc90 0x7fcc001625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfe59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcc0015dc90 0x7fcc001625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47306/0 (socket says 172.21.15.148:47306) 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfed9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcc000d1810 0x7fcc000cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47068/0 (socket says 172.21.15.148:47068) 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1738708431 learned_addr learned my addr 172.21.15.148:0/1738708431 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:16.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1738708431 >> v1:172.21.15.188:6789/0 conn(0x7fcc000cc8e0 legacy=0x7fcc0015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1738708431 >> v1:172.21.15.148:6789/0 conn(0x7fcc000d1810 legacy=0x7fcc000cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1738708431 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc001620b0 con 0x7fcc0015dc90 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.608+0000 7fcbfd59a640 1 -- 172.21.15.148:0/1738708431 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030428347 0 0) 0x7fcc000cce50 con 0x7fcc000d1810 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcbfe59c640 1 --2- 172.21.15.148:0/1738708431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc0015dc90 0x7fcc001625d0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5dc7803033d1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcbfd59a640 1 -- 172.21.15.148:0/1738708431 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037070 con 0x7fcc0015dc90 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcbfd59a640 1 -- 172.21.15.148:0/1738708431 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8032440 con 0x7fcc0015dc90 2024-04-25T10:05:16.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcbfd59a640 1 -- 172.21.15.148:0/1738708431 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8032810 con 0x7fcc0015dc90 2024-04-25T10:05:16.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 -- 172.21.15.148:0/1738708431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc0015dc90 msgr2=0x7fcc001625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1738708431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc0015dc90 0x7fcc001625d0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe802a210 comp rx=0 tx=0).stop 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 -- 172.21.15.148:0/1738708431 shutdown_connections 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1738708431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc0015dc90 0x7fcc001625d0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 -- 172.21.15.148:0/1738708431 >> 172.21.15.148:0/1738708431 conn(0x7fcc0006f590 msgr2=0x7fcc0006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 -- 172.21.15.148:0/1738708431 shutdown_connections 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.609+0000 7fcc04a12640 1 -- 172.21.15.148:0/1738708431 wait complete. 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 Processor -- start 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 -- start start 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 0x7fcc00155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc000d1810 0x7fcc0015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0015dc90 0x7fcc00160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0015e1f0 con 0x7fcc000cc8e0 2024-04-25T10:05:16.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0015e070 con 0x7fcc000d1810 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcc04a12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000d1c80 con 0x7fcc0015dc90 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcbfe59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 0x7fcc00155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcbfe59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 0x7fcc00155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47312/0 (socket says 172.21.15.148:47312) 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.611+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1878824578 learned_addr learned my addr 172.21.15.148:0/1878824578 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0015dc90 msgr2=0x7fcc00160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfed9d640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0015dc90 0x7fcc00160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfdd9b640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc000d1810 0x7fcc0015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfe59c640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0015dc90 0x7fcc00160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc000d1810 msgr2=0x7fcc0015fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfe59c640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc000d1810 0x7fcc0015fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfe59c640 1 -- 172.21.15.148:0/1878824578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00157360 con 0x7fcc000cc8e0 2024-04-25T10:05:16.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfed9d640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0015dc90 0x7fcc00160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfdd9b640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc000d1810 0x7fcc0015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbfe59c640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 0x7fcc00155910 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8041070 con 0x7fcc000cc8e0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe803c870 con 0x7fcc000cc8e0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc00157640 con 0x7fcc000cc8e0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037040 con 0x7fcc000cc8e0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.612+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc000c6aa0 con 0x7fcc000cc8e0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.613+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc00161720 con 0x7fcc000cc8e0 2024-04-25T10:05:16.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.613+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbe8044070 con 0x7fcc000cc8e0 2024-04-25T10:05:16.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.614+0000 7fcbf6ffd640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbe006b4d0 0x7fcbe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.614+0000 7fcbfdd9b640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbe006b4d0 0x7fcbe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.614+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcbe80aae60 con 0x7fcc000cc8e0 2024-04-25T10:05:16.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.614+0000 7fcbfdd9b640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbe006b4d0 0x7fcbe006d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0007fd0 tx=0x7fcbf0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.618+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc00161720 con 0x7fcc000cc8e0 2024-04-25T10:05:16.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.788+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc00073990 con 0x7fcc000cc8e0 2024-04-25T10:05:16.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.790+0000 7fcbf6ffd640 1 -- 172.21.15.148:0/1878824578 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16932 (secure 0 0 0) 0x7fcbe8079be0 con 0x7fcc000cc8e0 2024-04-25T10:05:16.795 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:16.795 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":159,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:32.326979+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbe006b4d0 msgr2=0x7fcbe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbe006b4d0 0x7fcbe006d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0007fd0 tx=0x7fcbf0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 msgr2=0x7fcc00155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 0x7fcc00155910 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009870 tx=0x7fcbe80046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 shutdown_connections 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcbe006b4d0 0x7fcbe006d990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc0015dc90 0x7fcc00160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc000d1810 0x7fcc0015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.791+0000 7fcc04a12640 1 --2- 172.21.15.148:0/1878824578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc000cc8e0 0x7fcc00155910 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.792+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 >> 172.21.15.148:0/1878824578 conn(0x7fcc0006f590 msgr2=0x7fcc000ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.792+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 shutdown_connections 2024-04-25T10:05:16.798 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.792+0000 7fcc04a12640 1 -- 172.21.15.148:0/1878824578 wait complete. 2024-04-25T10:05:16.808 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:16.808 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.901+0000 7fbed553b640 1 Processor -- start 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.901+0000 7fbed553b640 1 -- start start 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbed553b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbed015dbe0 0x7fbed00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbed553b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbed00b2cb0 0x7fbed00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbed553b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed00b97a0 con 0x7fbed00b92a0 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbed553b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed015dfc0 con 0x7fbed015dbe0 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbed553b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed00d1a00 con 0x7fbed00b2cb0 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbeceffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbed015dbe0 0x7fbed00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbecf7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbed00b92a0 0x7fbed00b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47082/0 (socket says 172.21.15.148:47082) 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbecf7fe640 1 -- 172.21.15.148:0/3285708979 learned_addr learned my addr 172.21.15.148:0/3285708979 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:16.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbece7fc640 1 --2- 172.21.15.148:0/3285708979 >> v2:172.21.15.188:3300/0 conn(0x7fbed00b2cb0 0x7fbed00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbeceffd640 1 -- 172.21.15.148:0/3285708979 >> v2:172.21.15.188:3300/0 conn(0x7fbed00b2cb0 msgr2=0x7fbed00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbeceffd640 1 --2- 172.21.15.148:0/3285708979 >> v2:172.21.15.188:3300/0 conn(0x7fbed00b2cb0 0x7fbed00d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbeceffd640 1 -- 172.21.15.148:0/3285708979 >> v1:172.21.15.148:6789/0 conn(0x7fbed00b92a0 legacy=0x7fbed00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbeceffd640 1 -- 172.21.15.148:0/3285708979 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00d1c50 con 0x7fbed015dbe0 2024-04-25T10:05:16.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.902+0000 7fbecdffb640 1 -- 172.21.15.148:0/3285708979 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182397282 0 0) 0x7fbed00b97a0 con 0x7fbed00b92a0 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.903+0000 7fbeceffd640 1 --2- 172.21.15.148:0/3285708979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed015dbe0 0x7fbed00d0910 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fbed006cff0 tx=0x7fbec002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fab1af7965db0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.903+0000 7fbecdffb640 1 -- 172.21.15.148:0/3285708979 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0037070 con 0x7fbed015dbe0 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.903+0000 7fbecdffb640 1 -- 172.21.15.148:0/3285708979 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0032440 con 0x7fbed015dbe0 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.903+0000 7fbecdffb640 1 -- 172.21.15.148:0/3285708979 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec002ae60 con 0x7fbed015dbe0 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 -- 172.21.15.148:0/3285708979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed015dbe0 msgr2=0x7fbed00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 --2- 172.21.15.148:0/3285708979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed015dbe0 0x7fbed00d0910 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fbed006cff0 tx=0x7fbec002a110 comp rx=0 tx=0).stop 2024-04-25T10:05:16.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 -- 172.21.15.148:0/3285708979 shutdown_connections 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 --2- 172.21.15.148:0/3285708979 >> v2:172.21.15.188:3300/0 conn(0x7fbed00b2cb0 0x7fbed00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 --2- 172.21.15.148:0/3285708979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed015dbe0 0x7fbed00d0910 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 -- 172.21.15.148:0/3285708979 >> 172.21.15.148:0/3285708979 conn(0x7fbed0061a20 msgr2=0x7fbed0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 -- 172.21.15.148:0/3285708979 shutdown_connections 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.904+0000 7fbed553b640 1 -- 172.21.15.148:0/3285708979 wait complete. 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.905+0000 7fbed553b640 1 Processor -- start 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.905+0000 7fbed553b640 1 -- start start 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 0x7fbed00b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbed00b92a0 0x7fbed00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbed015dbe0 0x7fbed0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00b6ab0 con 0x7fbed00b2cb0 2024-04-25T10:05:16.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00d2210 con 0x7fbed015dbe0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00b6c30 con 0x7fbed00b92a0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 0x7fbed00b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 0x7fbed00b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47322/0 (socket says 172.21.15.148:47322) 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 -- 172.21.15.148:0/3493362144 learned_addr learned my addr 172.21.15.148:0/3493362144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 -- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbed00b92a0 msgr2=0x7fbed00cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbece7fc640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbed00b92a0 0x7fbed00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbed00b92a0 0x7fbed00cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbecf7fe640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbed015dbe0 0x7fbed0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 -- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbed015dbe0 msgr2=0x7fbed0151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbed015dbe0 0x7fbed0151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 -- 172.21.15.148:0/3493362144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00b5810 con 0x7fbed00b2cb0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeceffd640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 0x7fbed00b52d0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009890 tx=0x7fbec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0037040 con 0x7fbed00b2cb0 2024-04-25T10:05:16.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0042070 con 0x7fbed00b2cb0 2024-04-25T10:05:16.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0030bd0 con 0x7fbed00b2cb0 2024-04-25T10:05:16.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed00cb920 con 0x7fbed00b2cb0 2024-04-25T10:05:16.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.906+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed00b4510 con 0x7fbed00b2cb0 2024-04-25T10:05:16.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.907+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed0073990 con 0x7fbed00b2cb0 2024-04-25T10:05:16.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.908+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbec0047460 con 0x7fbed00b2cb0 2024-04-25T10:05:16.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.908+0000 7fbeb77fe640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbebc06b4d0 0x7fbebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.908+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbec00aaf10 con 0x7fbed00b2cb0 2024-04-25T10:05:16.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.908+0000 7fbece7fc640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbebc06b4d0 0x7fbebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.909+0000 7fbece7fc640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbebc06b4d0 0x7fbebc06d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006fd0 tx=0x7fbec4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:16.912+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbec0032e20 con 0x7fbed00b2cb0 2024-04-25T10:05:17.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.116+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbed015ad90 con 0x7fbed00b2cb0 2024-04-25T10:05:17.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.116+0000 7fbeb77fe640 1 -- 172.21.15.148:0/3493362144 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fbec0045360 con 0x7fbed00b2cb0 2024-04-25T10:05:17.122 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:17.122 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:30.667288+0000","modified":"2024-04-25T10:04:32.676174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10365":{"gid":10365,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3286540466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3286540466},{"type":"v1","addr":"172.21.15.148:6837","nonce":3286540466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:05:17.122 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 93 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbebc06b4d0 msgr2=0x7fbebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbebc06b4d0 0x7fbebc06d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbec4006fd0 tx=0x7fbec4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 msgr2=0x7fbed00b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 0x7fbed00b52d0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009890 tx=0x7fbec00046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 shutdown_connections 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbebc06b4d0 0x7fbebc06d990 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbed015dbe0 0x7fbed0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbed00b92a0 0x7fbed00cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 --2- 172.21.15.148:0/3493362144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbed00b2cb0 0x7fbed00b52d0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 >> 172.21.15.148:0/3493362144 conn(0x7fbed0061a20 msgr2=0x7fbed00d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 shutdown_connections 2024-04-25T10:05:17.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.118+0000 7fbed553b640 1 -- 172.21.15.148:0/3493362144 wait complete. 2024-04-25T10:05:17.135 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:17.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.227+0000 7f24303d5640 1 Processor -- start 2024-04-25T10:05:17.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.227+0000 7f24303d5640 1 -- start start 2024-04-25T10:05:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f24303d5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f242815dc90 0x7f24281625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f24303d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280cce50 con 0x7f24280d1810 2024-04-25T10:05:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f24303d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2428161af0 con 0x7f24280cc8e0 2024-04-25T10:05:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f24303d5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2428161dd0 con 0x7f242815dc90 2024-04-25T10:05:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f242e972640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f24280d1810 0x7f24280cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47096/0 (socket says 172.21.15.148:47096) 2024-04-25T10:05:17.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f242e972640 1 -- 172.21.15.148:0/401343243 learned_addr learned my addr 172.21.15.148:0/401343243 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f242e171640 1 --2- 172.21.15.148:0/401343243 >> v2:172.21.15.188:3300/0 conn(0x7f242815dc90 0x7f24281625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235851005 0 0) 0x7f24280cce50 con 0x7f24280d1810 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.228+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2414003520 con 0x7f24280d1810 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123240410 0 0) 0x7f2414003520 con 0x7f24280d1810 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 >> v2:172.21.15.188:3300/0 conn(0x7f242815dc90 msgr2=0x7f24281625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 --2- 172.21.15.148:0/401343243 >> v2:172.21.15.188:3300/0 conn(0x7f242815dc90 0x7f24281625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 >> v1:172.21.15.188:6789/0 conn(0x7f24280cc8e0 legacy=0x7f242815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24281620b0 con 0x7f24280d1810 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f24200019f0 con 0x7f24280d1810 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 >> v1:172.21.15.148:6789/0 conn(0x7f24280d1810 legacy=0x7f24280cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24140039b0 0x7f2414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2414004800 0x7f2414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24140051c0 0x7f241400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428161dd0 con 0x7f2414004800 2024-04-25T10:05:17.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241400c160 con 0x7f24140039b0 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241400c360 con 0x7f24140051c0 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d970640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2414004800 0x7f2414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242e171640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24140039b0 0x7f2414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242e972640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24140051c0 0x7f241400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d970640 1 -- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24140051c0 msgr2=0x7f241400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d970640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24140051c0 0x7f241400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d970640 1 -- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24140039b0 msgr2=0x7f2414003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d970640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24140039b0 0x7f2414003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242d970640 1 -- 172.21.15.148:0/401343243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414003520 con 0x7f2414004800 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.229+0000 7f242e972640 1 -- 172.21.15.148:0/401343243 reap_dead start 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.230+0000 7f242d970640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2414004800 0x7f2414004c80 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f2418001750 tx=0x7f2414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ff565aad00ea38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.230+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418001370 con 0x7f2414004800 2024-04-25T10:05:17.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.230+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24180052d0 con 0x7f2414004800 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.230+0000 7f242d16f640 1 -- 172.21.15.148:0/401343243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241802abc0 con 0x7f2414004800 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.230+0000 7f24303d5640 1 -- 172.21.15.148:0/401343243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2414004800 msgr2=0x7f2414004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.230+0000 7f24303d5640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2414004800 0x7f2414004c80 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f2418001750 tx=0x7f2414005830 comp rx=0 tx=0).stop 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.231+0000 7f24303d5640 1 -- 172.21.15.148:0/401343243 shutdown_connections 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.231+0000 7f24303d5640 1 --2- 172.21.15.148:0/401343243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2414004800 0x7f2414004c80 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.231+0000 7f24303d5640 1 -- 172.21.15.148:0/401343243 >> 172.21.15.148:0/401343243 conn(0x7f242806f590 msgr2=0x7f242806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.231+0000 7f24303d5640 1 -- 172.21.15.148:0/401343243 shutdown_connections 2024-04-25T10:05:17.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.231+0000 7f24303d5640 1 -- 172.21.15.148:0/401343243 wait complete. 2024-04-25T10:05:17.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.232+0000 7f24303d5640 1 Processor -- start 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.232+0000 7f24303d5640 1 -- start start 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.232+0000 7f24303d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2414004800 0x7f2428158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.232+0000 7f24303d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 0x7f24280c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.232+0000 7f24303d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24280cfde0 0x7f24280d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f24303d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242815e1f0 con 0x7f24280c7e70 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f24303d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242815e070 con 0x7f2414004800 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f24303d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24281625d0 con 0x7f24280cfde0 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 0x7f24280c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 0x7f24280c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47344/0 (socket says 172.21.15.148:47344) 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 -- 172.21.15.148:0/3468068684 learned_addr learned my addr 172.21.15.148:0/3468068684 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 -- 172.21.15.148:0/3468068684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24280cfde0 msgr2=0x7f24280d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24280cfde0 0x7f24280d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 -- 172.21.15.148:0/3468068684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2414004800 msgr2=0x7f2428158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2414004800 0x7f2428158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 -- 172.21.15.148:0/3468068684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280c8350 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f242d970640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 0x7f24280c8950 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f2418001870 tx=0x7f2418001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241803f030 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418005480 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418037040 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280cbb70 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.233+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428160d90 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.234+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428073990 con 0x7f24280c7e70 2024-04-25T10:05:17.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.235+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2418042070 con 0x7f24280c7e70 2024-04-25T10:05:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.235+0000 7f241effd640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.235+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24180a9750 con 0x7f24280c7e70 2024-04-25T10:05:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.235+0000 7f242e171640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.235+0000 7f242e171640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2424006e80 tx=0x7f2424006df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.239+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f241804a020 con 0x7f24280c7e70 2024-04-25T10:05:17.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.448+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24280c5ba0 con 0x7f24280c7e70 2024-04-25T10:05:17.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.449+0000 7f241effd640 1 -- 172.21.15.148:0/3468068684 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f24180783b0 con 0x7f24280c7e70 2024-04-25T10:05:17.454 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:17.454 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":93,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:30.667288+0000","modified":"2024-04-25T10:04:32.676174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10365":{"gid":10365,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/3286540466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3286540466},{"type":"v1","addr":"172.21.15.148:6837","nonce":3286540466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:05:17.454 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 93 2024-04-25T10:05:17.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f240c06b4d0 msgr2=0x7f240c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2424006e80 tx=0x7f2424006df0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 msgr2=0x7f24280c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 0x7f24280c8950 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f2418001870 tx=0x7f2418001120 comp rx=0 tx=0).stop 2024-04-25T10:05:17.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 shutdown_connections 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24280cfde0 0x7f24280d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24280c7e70 0x7f24280c8950 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 --2- 172.21.15.148:0/3468068684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2414004800 0x7f2428158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 >> 172.21.15.148:0/3468068684 conn(0x7f242806f590 msgr2=0x7f242806c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 shutdown_connections 2024-04-25T10:05:17.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.451+0000 7f24303d5640 1 -- 172.21.15.148:0/3468068684 wait complete. 2024-04-25T10:05:17.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:17.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.558+0000 7f5dc4293640 1 Processor -- start 2024-04-25T10:05:17.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.558+0000 7f5dc4293640 1 -- start start 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc4293640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dbc15dc00 0x7f5dbc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc4293640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc4293640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc15dfe0 con 0x7f5dbc0ceb20 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc4293640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc0d00c0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc4293640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0d03a0 con 0x7f5dbc0d1dd0 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc202f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dbc15dc00 0x7f5dbc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc2830640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc182e640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5dbc0ceb20 0x7f5dbc0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43430/0 (socket says 172.21.15.148:43430) 2024-04-25T10:05:17.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.559+0000 7f5dc182e640 1 -- 172.21.15.148:0/3784525831 learned_addr learned my addr 172.21.15.148:0/3784525831 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:17.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.560+0000 7f5dc102d640 1 -- 172.21.15.148:0/3784525831 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794554894 0 0) 0x7f5dac001110 con 0x7f5dbc0ceb20 2024-04-25T10:05:17.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.560+0000 7f5dc2830640 1 -- 172.21.15.148:0/3784525831 >> v2:172.21.15.188:3300/0 conn(0x7f5dbc15dc00 msgr2=0x7f5dbc0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.560+0000 7f5dc2830640 1 --2- 172.21.15.148:0/3784525831 >> v2:172.21.15.188:3300/0 conn(0x7f5dbc15dc00 0x7f5dbc0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.560+0000 7f5dc2830640 1 -- 172.21.15.148:0/3784525831 >> v1:172.21.15.188:6789/0 conn(0x7f5dbc0ceb20 legacy=0x7f5dbc0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.560+0000 7f5dc2830640 1 -- 172.21.15.148:0/3784525831 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0cc440 con 0x7f5dbc0d1dd0 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.560+0000 7f5dc2830640 1 --2- 172.21.15.148:0/3784525831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0ccc40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5db40097a0 tx=0x7f5db402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42801976461008eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc102d640 1 -- 172.21.15.148:0/3784525831 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4037070 con 0x7f5dbc0d1dd0 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc102d640 1 -- 172.21.15.148:0/3784525831 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4032440 con 0x7f5dbc0d1dd0 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc102d640 1 -- 172.21.15.148:0/3784525831 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4032810 con 0x7f5dbc0d1dd0 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 -- 172.21.15.148:0/3784525831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0d1dd0 msgr2=0x7f5dbc0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 --2- 172.21.15.148:0/3784525831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0ccc40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5db40097a0 tx=0x7f5db402a180 comp rx=0 tx=0).stop 2024-04-25T10:05:17.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 -- 172.21.15.148:0/3784525831 shutdown_connections 2024-04-25T10:05:17.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 --2- 172.21.15.148:0/3784525831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0ccc40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 --2- 172.21.15.148:0/3784525831 >> v2:172.21.15.188:3300/0 conn(0x7f5dbc15dc00 0x7f5dbc0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 -- 172.21.15.148:0/3784525831 >> 172.21.15.148:0/3784525831 conn(0x7f5dbc06f5d0 msgr2=0x7f5dbc06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 -- 172.21.15.148:0/3784525831 shutdown_connections 2024-04-25T10:05:17.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.561+0000 7f5dc4293640 1 -- 172.21.15.148:0/3784525831 wait complete. 2024-04-25T10:05:17.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.562+0000 7f5dc4293640 1 Processor -- start 2024-04-25T10:05:17.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.562+0000 7f5dc4293640 1 -- start start 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc4293640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0ceb20 0x7f5dbc1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc4293640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc4293640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 0x7f5dbc0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc4293640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cfd90 con 0x7f5dbc15dc00 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc4293640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cc130 con 0x7f5dbc0d1dd0 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc4293640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cdd70 con 0x7f5dbc0ceb20 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc2830640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 0x7f5dbc0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc2830640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 0x7f5dbc0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47358/0 (socket says 172.21.15.148:47358) 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc2830640 1 -- 172.21.15.148:0/2225704456 learned_addr learned my addr 172.21.15.148:0/2225704456 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc202f640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0ceb20 0x7f5dbc1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.563+0000 7f5dc182e640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc2830640 1 -- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0ceb20 msgr2=0x7f5dbc1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc2830640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0ceb20 0x7f5dbc1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc2830640 1 -- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc0d1dd0 msgr2=0x7f5dbc0b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc2830640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc2830640 1 -- 172.21.15.148:0/2225704456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0c8df0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc202f640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0ceb20 0x7f5dbc1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc182e640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc2830640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 0x7f5dbc0b1a60 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f5db4035040 tx=0x7f5db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4037040 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4042070 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc0c90d0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40308a0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.564+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0c9690 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.565+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc073990 con 0x7f5dbc15dc00 2024-04-25T10:05:17.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.565+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5db4030a40 con 0x7f5dbc15dc00 2024-04-25T10:05:17.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.566+0000 7f5db2ffd640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5da006b4d0 0x7f5da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.566+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5db40aaca0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.566+0000 7f5dc202f640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5da006b4d0 0x7f5da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.566+0000 7f5dc202f640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5da006b4d0 0x7f5da006d990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009fd0 tx=0x7f5db800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.570+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5db4079ab0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.739+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dbc0c66c0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.746 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.740+0000 7f5db2ffd640 1 -- 172.21.15.148:0/2225704456 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f5db40793b0 con 0x7f5dbc15dc00 2024-04-25T10:05:17.747 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:17.749 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":159,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:04:32.326979+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:28.876622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:29.635420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5da006b4d0 msgr2=0x7f5da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5da006b4d0 0x7f5da006d990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009fd0 tx=0x7f5db800b040 comp rx=0 tx=0).stop 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 msgr2=0x7f5dbc0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 0x7f5dbc0b1a60 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f5db4035040 tx=0x7f5db40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 shutdown_connections 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5da006b4d0 0x7f5da006d990 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5dbc15dc00 0x7f5dbc0b1a60 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dbc0d1dd0 0x7f5dbc0b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 --2- 172.21.15.148:0/2225704456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dbc0ceb20 0x7f5dbc1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 >> 172.21.15.148:0/2225704456 conn(0x7f5dbc06f5d0 msgr2=0x7f5dbc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 shutdown_connections 2024-04-25T10:05:17.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.742+0000 7f5dc4293640 1 -- 172.21.15.148:0/2225704456 wait complete. 2024-04-25T10:05:17.760 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.852+0000 7efe8804a640 1 Processor -- start 2024-04-25T10:05:17.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.852+0000 7efe8804a640 1 -- start start 2024-04-25T10:05:17.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe8804a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 0x7efe800b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe8804a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe8015deb0 0x7efe8015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe8804a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe800d1770 con 0x7efe80070dc0 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe8804a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe801622b0 con 0x7efe8015deb0 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe8804a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe800b10f0 con 0x7efe8006ccd0 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 0x7efe800b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe865e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe8015deb0 0x7efe8015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 0x7efe800b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:40306/0 (socket says 172.21.15.148:40306) 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe865e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe8015deb0 0x7efe8015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:34826/0 (socket says 172.21.15.148:34826) 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 -- 172.21.15.148:0/512421900 learned_addr learned my addr 172.21.15.148:0/512421900 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe85de6640 1 --1- 172.21.15.148:0/512421900 >> v1:172.21.15.188:6789/0 conn(0x7efe80070dc0 0x7efe800711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:43434/0 (socket says 172.21.15.148:43434) 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 -- 172.21.15.148:0/512421900 >> v2:172.21.15.188:3300/0 conn(0x7efe8015deb0 msgr2=0x7efe8015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 --2- 172.21.15.148:0/512421900 >> v2:172.21.15.188:3300/0 conn(0x7efe8015deb0 0x7efe8015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 -- 172.21.15.148:0/512421900 >> v1:172.21.15.188:6789/0 conn(0x7efe80070dc0 legacy=0x7efe800711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.853+0000 7efe855e5640 1 -- 172.21.15.148:0/512421900 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80162740 con 0x7efe8006ccd0 2024-04-25T10:05:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.854+0000 7efe855e5640 1 --2- 172.21.15.148:0/512421900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 0x7efe800b07a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe70009870 tx=0x7efe7002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9b7ba7bf3a2d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.854+0000 7efe84de4640 1 -- 172.21.15.148:0/512421900 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70037070 con 0x7efe8006ccd0 2024-04-25T10:05:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.854+0000 7efe84de4640 1 -- 172.21.15.148:0/512421900 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70032440 con 0x7efe8006ccd0 2024-04-25T10:05:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.854+0000 7efe84de4640 1 -- 172.21.15.148:0/512421900 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70032810 con 0x7efe8006ccd0 2024-04-25T10:05:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 -- 172.21.15.148:0/512421900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 msgr2=0x7efe800b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 --2- 172.21.15.148:0/512421900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 0x7efe800b07a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe70009870 tx=0x7efe7002a130 comp rx=0 tx=0).stop 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 -- 172.21.15.148:0/512421900 shutdown_connections 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 --2- 172.21.15.148:0/512421900 >> v2:172.21.15.188:3300/0 conn(0x7efe8015deb0 0x7efe8015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 --2- 172.21.15.148:0/512421900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8006ccd0 0x7efe800b07a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 -- 172.21.15.148:0/512421900 >> 172.21.15.148:0/512421900 conn(0x7efe800b3530 msgr2=0x7efe800b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 -- 172.21.15.148:0/512421900 shutdown_connections 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.855+0000 7efe8804a640 1 -- 172.21.15.148:0/512421900 wait complete. 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.856+0000 7efe8804a640 1 Processor -- start 2024-04-25T10:05:17.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.856+0000 7efe8804a640 1 -- start start 2024-04-25T10:05:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe8804a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 0x7efe800cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe8804a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe80070dc0 0x7efe80147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe8804a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8015deb0 0x7efe80148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe8804a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe80161f40 con 0x7efe8006ccd0 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe8804a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe80161c60 con 0x7efe80070dc0 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe8804a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe800b5840 con 0x7efe8015deb0 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 0x7efe800cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 0x7efe800cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47374/0 (socket says 172.21.15.148:47374) 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 -- 172.21.15.148:0/2300522968 learned_addr learned my addr 172.21.15.148:0/2300522968 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe865e7640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8015deb0 0x7efe80148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe855e5640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe80070dc0 0x7efe80147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 -- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8015deb0 msgr2=0x7efe80148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8015deb0 0x7efe80148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 -- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe80070dc0 msgr2=0x7efe80147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe80070dc0 0x7efe80147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 -- 172.21.15.148:0/2300522968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe800cbbc0 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe85de6640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 0x7efe800cd0e0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00c800 tx=0x7efe7c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.857+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c017070 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.858+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c00a640 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.858+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe800cbea0 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.858+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00aa30 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.858+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe801537c0 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.858+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe800cc290 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.859+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe7c01a070 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.859+0000 7efe767fc640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe6406b4d0 0x7efe6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.859+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe7c089960 con 0x7efe8006ccd0 2024-04-25T10:05:17.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.859+0000 7efe855e5640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe6406b4d0 0x7efe6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.860+0000 7efe855e5640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe6406b4d0 0x7efe6406d990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efe70034000 tx=0x7efe7002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:17.863+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe7c058000 con 0x7efe8006ccd0 2024-04-25T10:05:18.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.072+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efe800cc070 con 0x7efe8006ccd0 2024-04-25T10:05:18.079 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:05:18.073+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:18.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.075+0000 7efe767fc640 1 -- 172.21.15.148:0/2300522968 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) v1 ==== 161+0+0 (secure 0 0 0) 0x7efe7c05c0a0 con 0x7efe8006ccd0 2024-04-25T10:05:18.081 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe6406b4d0 msgr2=0x7efe6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe6406b4d0 0x7efe6406d990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efe70034000 tx=0x7efe7002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 msgr2=0x7efe800cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 0x7efe800cd0e0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00c800 tx=0x7efe7c00ccc0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 shutdown_connections 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efe6406b4d0 0x7efe6406d990 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8015deb0 0x7efe80148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe80070dc0 0x7efe80147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 --2- 172.21.15.148:0/2300522968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe8006ccd0 0x7efe800cd0e0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 >> 172.21.15.148:0/2300522968 conn(0x7efe800b3530 msgr2=0x7efe8015d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 shutdown_connections 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.077+0000 7efe8804a640 1 -- 172.21.15.148:0/2300522968 wait complete. 2024-04-25T10:05:18.084 INFO:tasks.ceph.mds.c.smithi148.stderr: -2399> 2024-04-25T10:04:36.598+0000 7f2517f15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:18.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:18.111 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:05:18.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.188+0000 7fc9787ce640 1 Processor -- start 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.188+0000 7fc9787ce640 1 -- start start 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc9787ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9700b5620 0x7fc9700b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc9787ce640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc97006c2d0 0x7fc97006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc9787ce640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97006cc80 con 0x7fc97015b7d0 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc9787ce640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97006cf50 con 0x7fc97006c2d0 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc9787ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9700ce530 con 0x7fc9700b5620 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc97006c2d0 0x7fc97006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc97006c2d0 0x7fc97006c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47398/0 (socket says 172.21.15.148:47398) 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc97656a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc97015b7d0 0x7fc9700cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47098/0 (socket says 172.21.15.148:47098) 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 -- 172.21.15.148:0/1206314475 learned_addr learned my addr 172.21.15.148:0/1206314475 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc975d69640 1 --2- 172.21.15.148:0/1206314475 >> v2:172.21.15.188:3300/0 conn(0x7fc9700b5620 0x7fc9700b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 -- 172.21.15.148:0/1206314475 >> v2:172.21.15.188:3300/0 conn(0x7fc9700b5620 msgr2=0x7fc9700b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 --2- 172.21.15.148:0/1206314475 >> v2:172.21.15.188:3300/0 conn(0x7fc9700b5620 0x7fc9700b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 -- 172.21.15.148:0/1206314475 >> v1:172.21.15.148:6789/0 conn(0x7fc97015b7d0 legacy=0x7fc9700cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.189+0000 7fc976d6b640 1 -- 172.21.15.148:0/1206314475 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700ce810 con 0x7fc97006c2d0 2024-04-25T10:05:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.190+0000 7fc976d6b640 1 --2- 172.21.15.148:0/1206314475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97006c740 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fc9700ce180 tx=0x7fc96802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9352f496892619aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.190+0000 7fc975568640 1 -- 172.21.15.148:0/1206314475 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968036070 con 0x7fc97006c2d0 2024-04-25T10:05:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.190+0000 7fc975568640 1 -- 172.21.15.148:0/1206314475 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968031600 con 0x7fc97006c2d0 2024-04-25T10:05:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.190+0000 7fc975568640 1 -- 172.21.15.148:0/1206314475 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9680319d0 con 0x7fc97006c2d0 2024-04-25T10:05:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 -- 172.21.15.148:0/1206314475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 msgr2=0x7fc97006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 --2- 172.21.15.148:0/1206314475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97006c740 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fc9700ce180 tx=0x7fc96802a730 comp rx=0 tx=0).stop 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 -- 172.21.15.148:0/1206314475 shutdown_connections 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 --2- 172.21.15.148:0/1206314475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97006c740 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 --2- 172.21.15.148:0/1206314475 >> v2:172.21.15.188:3300/0 conn(0x7fc9700b5620 0x7fc9700b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 -- 172.21.15.148:0/1206314475 >> 172.21.15.148:0/1206314475 conn(0x7fc9700d0800 msgr2=0x7fc9700d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 -- 172.21.15.148:0/1206314475 shutdown_connections 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.191+0000 7fc9787ce640 1 -- 172.21.15.148:0/1206314475 wait complete. 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.192+0000 7fc9787ce640 1 Processor -- start 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.192+0000 7fc9787ce640 1 -- start start 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.192+0000 7fc9787ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97014b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.192+0000 7fc9787ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9700b5620 0x7fc9701f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.192+0000 7fc9787ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97015b7d0 0x7fc9700cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc9787ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97015bbb0 con 0x7fc97006c2d0 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc9787ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97006d230 con 0x7fc97015b7d0 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc9787ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97015bd30 con 0x7fc9700b5620 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97014b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97014b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47414/0 (socket says 172.21.15.148:47414) 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 -- 172.21.15.148:0/2232731695 learned_addr learned my addr 172.21.15.148:0/2232731695 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc975d69640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9700b5620 0x7fc9701f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc976d6b640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97015b7d0 0x7fc9700cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 -- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9700b5620 msgr2=0x7fc9701f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9700b5620 0x7fc9701f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 -- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97015b7d0 msgr2=0x7fc9700cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97015b7d0 0x7fc9700cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 -- 172.21.15.148:0/2232731695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9701f4430 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc97656a640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97014b750 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c3c0 tx=0x7fc96c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0074e0 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c007680 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c004a00 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97014acb0 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.193+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97014af50 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.194+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc970073a00 con 0x7fc97006c2d0 2024-04-25T10:05:18.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.195+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc96c01a020 con 0x7fc97006c2d0 2024-04-25T10:05:18.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.195+0000 7fc966ffd640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc95406b4d0 0x7fc95406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.195+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc96c089970 con 0x7fc97006c2d0 2024-04-25T10:05:18.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.195+0000 7fc975d69640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc95406b4d0 0x7fc95406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.205 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.196+0000 7fc975d69640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc95406b4d0 0x7fc95406d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc9700d1f60 tx=0x7fc960008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.199+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9701f4430 con 0x7fc97006c2d0 2024-04-25T10:05:18.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:18.409+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc97015d9d0 con 0x7fc97006c2d0 2024-04-25T10:05:19.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.076+0000 7fc966ffd640 1 -- 172.21.15.148:0/2232731695 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc96c058050 con 0x7fc97006c2d0 2024-04-25T10:05:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc95406b4d0 msgr2=0x7fc95406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc95406b4d0 0x7fc95406d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc9700d1f60 tx=0x7fc960008040 comp rx=0 tx=0).stop 2024-04-25T10:05:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 msgr2=0x7fc97014b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97014b750 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c3c0 tx=0x7fc96c00c880 comp rx=0 tx=0).stop 2024-04-25T10:05:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 shutdown_connections 2024-04-25T10:05:19.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc95406b4d0 0x7fc95406d990 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97015b7d0 0x7fc9700cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9700b5620 0x7fc9701f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 --2- 172.21.15.148:0/2232731695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc97006c2d0 0x7fc97014b750 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 >> 172.21.15.148:0/2232731695 conn(0x7fc9700d0800 msgr2=0x7fc97015afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 shutdown_connections 2024-04-25T10:05:19.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.079+0000 7fc9787ce640 1 -- 172.21.15.148:0/2232731695 wait complete. 2024-04-25T10:05:19.099 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:19.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.239+0000 7f57b115c640 1 Processor -- start 2024-04-25T10:05:19.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.239+0000 7f57b115c640 1 -- start start 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57b115c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57ac15a9f0 0x7f57ac0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57b115c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 0x7f57ac0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57b115c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ac0c6940 con 0x7f57ac0cd520 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57b115c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac0b0020 con 0x7f57ac15a9f0 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57b115c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cb5a0 con 0x7f57ac0cdbf0 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57ab59e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f57ac0cd520 0x7f57ac0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47108/0 (socket says 172.21.15.148:47108) 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.240+0000 7f57ab59e640 1 -- 172.21.15.148:0/1479066706 learned_addr learned my addr 172.21.15.148:0/1479066706 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aad9d640 1 --2- 172.21.15.148:0/1479066706 >> v2:172.21.15.188:3300/0 conn(0x7f57ac15a9f0 0x7f57ac0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 0x7f57ac0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401904692 0 0) 0x7f57a4001490 con 0x7f57ac0cd520 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5798003520 con 0x7f57ac0cd520 2024-04-25T10:05:19.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4265180500 0 0) 0x7f5798003520 con 0x7f57ac0cd520 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 msgr2=0x7f57ac0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 0x7f57ac0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 >> v2:172.21.15.188:3300/0 conn(0x7f57ac15a9f0 msgr2=0x7f57ac0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 --2- 172.21.15.148:0/1479066706 >> v2:172.21.15.188:3300/0 conn(0x7f57ac15a9f0 0x7f57ac0c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac15add0 con 0x7f57ac0cd520 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f57a4001bc0 con 0x7f57ac0cd520 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 >> v1:172.21.15.148:6789/0 conn(0x7f57ac0cd520 legacy=0x7f57ac0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 0x7f57980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5798003f30 0x7f5798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798004d40 0x7f57980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cb5a0 con 0x7f5798003f30 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579800b9c0 con 0x7f5798004d40 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579800bb40 con 0x7f57ac0cdbf0 2024-04-25T10:05:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5798003f30 0x7f5798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aad9d640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 0x7f57980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57ab59e640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798004d40 0x7f57980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 -- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 msgr2=0x7f57980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac0cdbf0 0x7f57980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 -- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798004d40 msgr2=0x7f57980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798004d40 0x7f57980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57aa59c640 1 -- 172.21.15.148:0/1479066706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5798003520 con 0x7f5798003f30 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.241+0000 7f57ab59e640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798004d40 0x7f57980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.242+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5798003f30 0x7f5798004800 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f5794009ac0 tx=0x7f5798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86bde05a1ce5c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.242+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794036070 con 0x7f5798003f30 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.242+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794031400 con 0x7f5798003f30 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.242+0000 7f57a9d9b640 1 -- 172.21.15.148:0/1479066706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794009690 con 0x7f5798003f30 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57b115c640 1 -- 172.21.15.148:0/1479066706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5798003f30 msgr2=0x7f5798004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57b115c640 1 --2- 172.21.15.148:0/1479066706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5798003f30 0x7f5798004800 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f5794009ac0 tx=0x7f5798005830 comp rx=0 tx=0).stop 2024-04-25T10:05:19.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57ab59e640 1 -- 172.21.15.148:0/1479066706 reap_dead start 2024-04-25T10:05:19.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57b115c640 1 -- 172.21.15.148:0/1479066706 shutdown_connections 2024-04-25T10:05:19.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57b115c640 1 -- 172.21.15.148:0/1479066706 >> 172.21.15.148:0/1479066706 conn(0x7f57ac0b2390 msgr2=0x7f57ac0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57b115c640 1 -- 172.21.15.148:0/1479066706 shutdown_connections 2024-04-25T10:05:19.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.243+0000 7f57b115c640 1 -- 172.21.15.148:0/1479066706 wait complete. 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.244+0000 7f57b115c640 1 Processor -- start 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.244+0000 7f57b115c640 1 -- start start 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57b115c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798003f30 0x7f57ac0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57b115c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 0x7f57ac14d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57b115c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac1f1590 0x7f57ac15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57b115c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0cb230 con 0x7f57ac0cf760 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57b115c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0caf50 con 0x7f5798003f30 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57b115c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0ce070 con 0x7f57ac1f1590 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 0x7f57ac14d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 0x7f57ac14d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47428/0 (socket says 172.21.15.148:47428) 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 -- 172.21.15.148:0/1338779835 learned_addr learned my addr 172.21.15.148:0/1338779835 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57ab59e640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac1f1590 0x7f57ac15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aad9d640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798003f30 0x7f57ac0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 -- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac1f1590 msgr2=0x7f57ac15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac1f1590 0x7f57ac15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 -- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798003f30 msgr2=0x7f57ac0ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798003f30 0x7f57ac0ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aa59c640 1 -- 172.21.15.148:0/1338779835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0b0ef0 con 0x7f57ac0cf760 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.245+0000 7f57aad9d640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798003f30 0x7f57ac0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.246+0000 7f57aa59c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 0x7f57ac14d2b0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f5794034040 tx=0x7f5794004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.246+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794040020 con 0x7f57ac0cf760 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.246+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794003c60 con 0x7f57ac0cf760 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.246+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac14d7f0 con 0x7f57ac0cf760 2024-04-25T10:05:19.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.246+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794036040 con 0x7f57ac0cf760 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.246+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac0c7cb0 con 0x7f57ac0cf760 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.247+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac06cc80 con 0x7f57ac0cf760 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.247+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5794043020 con 0x7f57ac0cf760 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.248+0000 7f57a37fe640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b500 0x7f578c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.248+0000 7f57aad9d640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b500 0x7f578c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.248+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f57940aa420 con 0x7f57ac0cf760 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.248+0000 7f57aad9d640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b500 0x7f578c06d9c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f579c009c40 tx=0x7f579c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.252+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f579404a070 con 0x7f57ac0cf760 2024-04-25T10:05:19.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:19.427+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f57ac0739e0 con 0x7f57ac0cf760 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.084+0000 7f57a37fe640 1 -- 172.21.15.148:0/1338779835 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v161) v1 ==== 181+0+0 (secure 0 0 0) 0x7f57940044e0 con 0x7f57ac0cf760 2024-04-25T10:05:20.090 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b500 msgr2=0x7f578c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b500 0x7f578c06d9c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f579c009c40 tx=0x7f579c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 msgr2=0x7f57ac14d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 0x7f57ac14d2b0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f5794034040 tx=0x7f5794004730 comp rx=0 tx=0).stop 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 shutdown_connections 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f578c06b500 0x7f578c06d9c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57ac1f1590 0x7f57ac15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57ac0cf760 0x7f57ac14d2b0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 --2- 172.21.15.148:0/1338779835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5798003f30 0x7f57ac0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.088+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 >> 172.21.15.148:0/1338779835 conn(0x7f57ac0b2390 msgr2=0x7f57ac1590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.089+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 shutdown_connections 2024-04-25T10:05:20.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.089+0000 7f57b115c640 1 -- 172.21.15.148:0/1338779835 wait complete. 2024-04-25T10:05:20.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:20.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.209+0000 7f99ce368640 1 Processor -- start 2024-04-25T10:05:20.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.209+0000 7f99ce368640 1 -- start start 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99ce368640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99c80d0ab0 0x7f99c8168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99ce368640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c80d2420 con 0x7f99c815dac0 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99ce368640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c80d2700 con 0x7f99c80d34c0 2024-04-25T10:05:20.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99ce368640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c80d3030 con 0x7f99c80d0ab0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99cc905640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99c80d0ab0 0x7f99c8168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99c7fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f99c815dac0 0x7f99c80d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47110/0 (socket says 172.21.15.148:47110) 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99c7fff640 1 -- 172.21.15.148:0/1920296055 learned_addr learned my addr 172.21.15.148:0/1920296055 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99cc905640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f99c80d0ab0 0x7f99c8168670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47442/0 (socket says 172.21.15.148:47442) 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99cc905640 1 -- 172.21.15.148:0/1920296055 >> v1:172.21.15.188:6789/0 conn(0x7f99c80d34c0 legacy=0x7f99c80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99cc905640 1 -- 172.21.15.148:0/1920296055 >> v1:172.21.15.148:6789/0 conn(0x7f99c815dac0 legacy=0x7f99c80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99cc905640 1 -- 172.21.15.148:0/1920296055 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80d29e0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.210+0000 7f99c6ffd640 1 -- 172.21.15.148:0/1920296055 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633435805 0 0) 0x7f99c80d2420 con 0x7f99c815dac0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.211+0000 7f99cc905640 1 --2- 172.21.15.148:0/1920296055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c8168670 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f99c80d31b0 tx=0x7f99c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d3230d0ff9265d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.211+0000 7f99c6ffd640 1 -- 172.21.15.148:0/1920296055 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0007740 con 0x7f99c80d0ab0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.211+0000 7f99c6ffd640 1 -- 172.21.15.148:0/1920296055 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c00078e0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.211+0000 7f99c6ffd640 1 -- 172.21.15.148:0/1920296055 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0007cb0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.211+0000 7f99ce368640 1 -- 172.21.15.148:0/1920296055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 msgr2=0x7f99c8168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.211+0000 7f99ce368640 1 --2- 172.21.15.148:0/1920296055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c8168670 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f99c80d31b0 tx=0x7f99c002a730 comp rx=0 tx=0).stop 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.212+0000 7f99ce368640 1 -- 172.21.15.148:0/1920296055 shutdown_connections 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.212+0000 7f99ce368640 1 --2- 172.21.15.148:0/1920296055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c8168670 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.212+0000 7f99ce368640 1 -- 172.21.15.148:0/1920296055 >> 172.21.15.148:0/1920296055 conn(0x7f99c8061c30 msgr2=0x7f99c806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.212+0000 7f99ce368640 1 -- 172.21.15.148:0/1920296055 shutdown_connections 2024-04-25T10:05:20.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.212+0000 7f99ce368640 1 -- 172.21.15.148:0/1920296055 wait complete. 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.213+0000 7f99ce368640 1 Processor -- start 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.213+0000 7f99ce368640 1 -- start start 2024-04-25T10:05:20.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80d34c0 0x7f99c8160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c815dac0 0x7f99c80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c815dea0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80d2100 con 0x7f99c815dac0 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80b97d0 con 0x7f99c80d34c0 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c80b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47452/0 (socket says 172.21.15.148:47452) 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 -- 172.21.15.148:0/2225797457 learned_addr learned my addr 172.21.15.148:0/2225797457 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99cc905640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c815dac0 0x7f99c80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c77fe640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80d34c0 0x7f99c8160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 -- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80d34c0 msgr2=0x7f99c8160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80d34c0 0x7f99c8160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 -- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c815dac0 msgr2=0x7f99c80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c815dac0 0x7f99c80b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 -- 172.21.15.148:0/2225797457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c80b4280 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c7fff640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c80b4e40 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c430 tx=0x7f99b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80074e0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80047e0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c815fd10 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004bd0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.214+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81602d0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.215+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c806c8e0 con 0x7f99c80d0ab0 2024-04-25T10:05:20.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.216+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99b8011070 con 0x7f99c80d0ab0 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.216+0000 7f99c4ff9640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f99b406b500 0x7f99b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.216+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f99b8089e20 con 0x7f99c80d0ab0 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.216+0000 7f99c77fe640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f99b406b500 0x7f99b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.217+0000 7f99c77fe640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f99b406b500 0x7f99b406d9c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f99c8152970 tx=0x7f99bc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.220+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99b800e280 con 0x7f99c80d0ab0 2024-04-25T10:05:20.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:20.390+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99c80739e0 con 0x7f99c80d0ab0 2024-04-25T10:05:21.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.085+0000 7f99c4ff9640 1 -- 172.21.15.148:0/2225797457 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v162) v1 ==== 169+0+0 (secure 0 0 0) 0x7f99b8058c70 con 0x7f99c80d0ab0 2024-04-25T10:05:21.092 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:21.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f99b406b500 msgr2=0x7f99b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f99b406b500 0x7f99b406d9c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f99c8152970 tx=0x7f99bc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 msgr2=0x7f99c80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c80b4e40 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c430 tx=0x7f99b800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 shutdown_connections 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f99b406b500 0x7f99b406d9c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99c815dac0 0x7f99c80b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c80d34c0 0x7f99c8160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 --2- 172.21.15.148:0/2225797457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99c80d0ab0 0x7f99c80b4e40 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 >> 172.21.15.148:0/2225797457 conn(0x7f99c8061c30 msgr2=0x7f99c80c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 shutdown_connections 2024-04-25T10:05:21.095 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.088+0000 7f99ce368640 1 -- 172.21.15.148:0/2225797457 wait complete. 2024-04-25T10:05:21.107 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:05:21.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.203+0000 7f568d609640 1 Processor -- start 2024-04-25T10:05:21.210 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.203+0000 7f568d609640 1 -- start start 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f568d609640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f568815dc20 0x7f56880ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f568d609640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568806cda0 con 0x7f5688161710 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f568d609640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568815e000 con 0x7f56880d32f0 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f568d609640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880d1de0 con 0x7f568815dc20 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f56867fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5688161710 0x7f568806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47124/0 (socket says 172.21.15.148:47124) 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f56867fc640 1 -- 172.21.15.148:0/1842585 learned_addr learned my addr 172.21.15.148:0/1842585 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5686ffd640 1 --2- 172.21.15.148:0/1842585 >> v2:172.21.15.188:3300/0 conn(0x7f568815dc20 0x7f56880ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328454059 0 0) 0x7f568806cda0 con 0x7f5688161710 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5670003520 con 0x7f5688161710 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860755007 0 0) 0x7f5670003520 con 0x7f5688161710 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 >> v2:172.21.15.188:3300/0 conn(0x7f568815dc20 msgr2=0x7f56880ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 --2- 172.21.15.148:0/1842585 >> v2:172.21.15.188:3300/0 conn(0x7f568815dc20 0x7f56880ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 >> v1:172.21.15.188:6789/0 conn(0x7f56880d32f0 legacy=0x7f568806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688161c60 con 0x7f5688161710 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f567c001670 con 0x7f5688161710 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.204+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 >> v1:172.21.15.148:6789/0 conn(0x7f5688161710 legacy=0x7f568806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5685ffb640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56700039b0 0x7f5670003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5685ffb640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5670004800 0x7f5670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5685ffb640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700051c0 0x7f567000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880d1de0 con 0x7f56700039b0 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567000c060 con 0x7f5670004800 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567000c260 con 0x7f56700051c0 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5686ffd640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56700039b0 0x7f5670003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f56867fc640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5670004800 0x7f5670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f56877fe640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700051c0 0x7f567000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5686ffd640 1 -- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700051c0 msgr2=0x7f567000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5686ffd640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56700051c0 0x7f567000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5686ffd640 1 -- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5670004800 msgr2=0x7f5670004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5686ffd640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5670004800 0x7f5670004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f5686ffd640 1 -- 172.21.15.148:0/1842585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670003520 con 0x7f56700039b0 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.205+0000 7f56877fe640 1 -- 172.21.15.148:0/1842585 reap_dead start 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f5686ffd640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56700039b0 0x7f5670003dd0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f5678009fc0 tx=0x7f5670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389fc5a9a055de86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678009c10 con 0x7f56700039b0 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678036070 con 0x7f56700039b0 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f5685ffb640 1 -- 172.21.15.148:0/1842585 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678031880 con 0x7f56700039b0 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 -- 172.21.15.148:0/1842585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56700039b0 msgr2=0x7f5670003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56700039b0 0x7f5670003dd0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f5678009fc0 tx=0x7f5670005830 comp rx=0 tx=0).stop 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 -- 172.21.15.148:0/1842585 shutdown_connections 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 --2- 172.21.15.148:0/1842585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56700039b0 0x7f5670003dd0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 -- 172.21.15.148:0/1842585 >> 172.21.15.148:0/1842585 conn(0x7f56880d0420 msgr2=0x7f56880d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 -- 172.21.15.148:0/1842585 shutdown_connections 2024-04-25T10:05:21.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.206+0000 7f568d609640 1 -- 172.21.15.148:0/1842585 wait complete. 2024-04-25T10:05:21.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 Processor -- start 2024-04-25T10:05:21.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 -- start start 2024-04-25T10:05:21.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56700039b0 0x7f56880b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5688154dd0 0x7f56880b8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 0x7f56881f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688162500 con 0x7f56881f7850 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5688162220 con 0x7f56700039b0 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f568d609640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b5e50 con 0x7f5688154dd0 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f56877fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 0x7f56881f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f56877fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 0x7f56881f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47476/0 (socket says 172.21.15.148:47476) 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.208+0000 7f56877fe640 1 -- 172.21.15.148:0/594834540 learned_addr learned my addr 172.21.15.148:0/594834540 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f5686ffd640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56700039b0 0x7f56880b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56867fc640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5688154dd0 0x7f56880b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56877fe640 1 -- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5688154dd0 msgr2=0x7f56880b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56877fe640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5688154dd0 0x7f56880b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56877fe640 1 -- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56700039b0 msgr2=0x7f56880b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56877fe640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56700039b0 0x7f56880b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56877fe640 1 -- 172.21.15.148:0/594834540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56881609c0 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56867fc640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5688154dd0 0x7f56880b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f5686ffd640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56700039b0 0x7f56880b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f56877fe640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 0x7f56881f7cd0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f56880b6570 tx=0x7f568000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680019070 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56800050d0 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5688160ca0 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56800035f0 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.209+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56880c7380 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.210+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56880d16b0 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.210+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f568001c070 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.211+0000 7f566f7fe640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b500 0x7f567406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.211+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5680089190 con 0x7f56881f7850 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.211+0000 7f5686ffd640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b500 0x7f567406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.211+0000 7f5686ffd640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b500 0x7f567406d9c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5678035000 tx=0x7f567802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.215+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5680058180 con 0x7f56881f7850 2024-04-25T10:05:21.408 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.402+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f56880739e0 con 0x7f56881f7850 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.405+0000 7f566f7fe640 1 -- 172.21.15.148:0/594834540 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5680057a80 con 0x7f56881f7850 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b500 msgr2=0x7f567406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b500 0x7f567406d9c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5678035000 tx=0x7f567802f000 comp rx=0 tx=0).stop 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 msgr2=0x7f56881f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 0x7f56881f7cd0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f56880b6570 tx=0x7f568000e5f0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 shutdown_connections 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f567406b500 0x7f567406d9c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56881f7850 0x7f56881f7cd0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5688154dd0 0x7f56880b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 --2- 172.21.15.148:0/594834540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56700039b0 0x7f56880b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 >> 172.21.15.148:0/594834540 conn(0x7f56880d0420 msgr2=0x7f56880d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 shutdown_connections 2024-04-25T10:05:21.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.407+0000 7f568d609640 1 -- 172.21.15.148:0/594834540 wait complete. 2024-04-25T10:05:21.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:05:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.516+0000 7fee5cd3b640 1 Processor -- start 2024-04-25T10:05:21.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.516+0000 7fee5cd3b640 1 -- start start 2024-04-25T10:05:21.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee5cd3b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee580ce330 0x7fee5806fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee5cd3b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee581626e0 con 0x7fee580700a0 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee5cd3b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee580cd270 con 0x7fee5806abe0 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee5cd3b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee580cd550 con 0x7fee580ce330 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee55d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fee580ce330 0x7fee5806fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee56d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fee580700a0 0x7fee5815d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47138/0 (socket says 172.21.15.148:47138) 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee56d9d640 1 -- 172.21.15.148:0/1116923103 learned_addr learned my addr 172.21.15.148:0/1116923103 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee55d9b640 1 -- 172.21.15.148:0/1116923103 >> v1:172.21.15.188:6789/0 conn(0x7fee5806abe0 legacy=0x7fee5806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee55d9b640 1 -- 172.21.15.148:0/1116923103 >> v1:172.21.15.148:6789/0 conn(0x7fee580700a0 legacy=0x7fee5815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.517+0000 7fee55d9b640 1 -- 172.21.15.148:0/1116923103 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580cd830 con 0x7fee580ce330 2024-04-25T10:05:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.518+0000 7fee55d9b640 1 --2- 172.21.15.148:0/1116923103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee580ce330 0x7fee5806fb60 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0097a0 tx=0x7fee4c02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4616351d9fc1fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.518+0000 7fee5559a640 1 -- 172.21.15.148:0/1116923103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c037070 con 0x7fee580ce330 2024-04-25T10:05:21.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.518+0000 7fee5559a640 1 -- 172.21.15.148:0/1116923103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c02acd0 con 0x7fee580ce330 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.518+0000 7fee5559a640 1 -- 172.21.15.148:0/1116923103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c032660 con 0x7fee580ce330 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.518+0000 7fee5cd3b640 1 -- 172.21.15.148:0/1116923103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee580ce330 msgr2=0x7fee5806fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.518+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/1116923103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee580ce330 0x7fee5806fb60 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0097a0 tx=0x7fee4c02a040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.519+0000 7fee5cd3b640 1 -- 172.21.15.148:0/1116923103 shutdown_connections 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.519+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/1116923103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee580ce330 0x7fee5806fb60 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.519+0000 7fee5cd3b640 1 -- 172.21.15.148:0/1116923103 >> 172.21.15.148:0/1116923103 conn(0x7fee580b2220 msgr2=0x7fee580b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.519+0000 7fee5cd3b640 1 -- 172.21.15.148:0/1116923103 shutdown_connections 2024-04-25T10:05:21.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.519+0000 7fee5cd3b640 1 -- 172.21.15.148:0/1116923103 wait complete. 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.520+0000 7fee5cd3b640 1 Processor -- start 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.520+0000 7fee5cd3b640 1 -- start start 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 0x7fee580b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee580700a0 0x7fee580b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee580ce330 0x7fee5814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5815ea90 con 0x7fee5806abe0 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58161780 con 0x7fee580700a0 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5815ec10 con 0x7fee580ce330 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 0x7fee580b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 0x7fee580b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47498/0 (socket says 172.21.15.148:47498) 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 -- 172.21.15.148:0/3282206316 learned_addr learned my addr 172.21.15.148:0/3282206316 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee56d9d640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee580ce330 0x7fee5814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee55d9b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee580700a0 0x7fee580b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 -- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee580ce330 msgr2=0x7fee5814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee580ce330 0x7fee5814c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 -- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee580700a0 msgr2=0x7fee580b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee580700a0 0x7fee580b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 -- 172.21.15.148:0/3282206316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5814e6c0 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee55d9b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee580700a0 0x7fee580b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5659c640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 0x7fee580b3bb0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c460 tx=0x7fee4800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48016020 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48012870 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581f7ef0 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48012c60 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.521+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581f84b0 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.522+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee580afe50 con 0x7fee5806abe0 2024-04-25T10:05:21.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.523+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee48019020 con 0x7fee5806abe0 2024-04-25T10:05:21.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.523+0000 7fee46ffd640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee3806b4d0 0x7fee3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.523+0000 7fee55d9b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee3806b4d0 0x7fee3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.523+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fee48089db0 con 0x7fee5806abe0 2024-04-25T10:05:21.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.524+0000 7fee55d9b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee3806b4d0 0x7fee3806d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fee4c034000 tx=0x7fee4c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.527+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee48020030 con 0x7fee5806abe0 2024-04-25T10:05:21.768 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:21.762+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7fee580b2ff0 con 0x7fee5806abe0 2024-04-25T10:05:22.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.090+0000 7fee46ffd640 1 -- 172.21.15.148:0/3282206316 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v534) v1 ==== 140+0+0 (secure 0 0 0) 0x7fee48058da0 con 0x7fee5806abe0 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.092+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee3806b4d0 msgr2=0x7fee3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.092+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee3806b4d0 0x7fee3806d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fee4c034000 tx=0x7fee4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.092+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 msgr2=0x7fee580b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.092+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 0x7fee580b3bb0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c460 tx=0x7fee4800c920 comp rx=0 tx=0).stop 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 shutdown_connections 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fee3806b4d0 0x7fee3806d990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee580ce330 0x7fee5814c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee580700a0 0x7fee580b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 --2- 172.21.15.148:0/3282206316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee5806abe0 0x7fee580b3bb0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 >> 172.21.15.148:0/3282206316 conn(0x7fee580b2220 msgr2=0x7fee580ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 shutdown_connections 2024-04-25T10:05:22.100 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.093+0000 7fee5cd3b640 1 -- 172.21.15.148:0/3282206316 wait complete. 2024-04-25T10:05:22.110 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:05:22.111 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:05:22.111 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:22.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.202+0000 7fd07ab7e640 1 Processor -- start 2024-04-25T10:05:22.209 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.202+0000 7fd07ab7e640 1 -- start start 2024-04-25T10:05:22.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07ab7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0740af720 0x7fd0740afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07ab7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740d02f0 0x7fd0740d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07ab7e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07406fe00 con 0x7fd074158220 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07ab7e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0740b3670 con 0x7fd0740af720 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07ab7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074070a10 con 0x7fd0740d02f0 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd073fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd0740af720 0x7fd0740afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07891a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd074158220 0x7fd07415ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47142/0 (socket says 172.21.15.148:47142) 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.203+0000 7fd07891a640 1 -- 172.21.15.148:0/2337319580 learned_addr learned my addr 172.21.15.148:0/2337319580 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd073fff640 1 -- 172.21.15.148:0/2337319580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740d02f0 msgr2=0x7fd0740d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd073fff640 1 --2- 172.21.15.148:0/2337319580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740d02f0 0x7fd0740d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd073fff640 1 -- 172.21.15.148:0/2337319580 >> v1:172.21.15.148:6789/0 conn(0x7fd074158220 legacy=0x7fd07415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd073fff640 1 -- 172.21.15.148:0/2337319580 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740700e0 con 0x7fd0740af720 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd0737fe640 1 -- 172.21.15.148:0/2337319580 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363020158 0 0) 0x7fd07406fe00 con 0x7fd074158220 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd073fff640 1 --2- 172.21.15.148:0/2337319580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0740af720 0x7fd0740afba0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fd0680097a0 tx=0x7fd06802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4447edc18b760b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd0737fe640 1 -- 172.21.15.148:0/2337319580 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068037070 con 0x7fd0740af720 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd0737fe640 1 -- 172.21.15.148:0/2337319580 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068032440 con 0x7fd0740af720 2024-04-25T10:05:22.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.204+0000 7fd0737fe640 1 -- 172.21.15.148:0/2337319580 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068032810 con 0x7fd0740af720 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2337319580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0740af720 msgr2=0x7fd0740afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2337319580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0740af720 0x7fd0740afba0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fd0680097a0 tx=0x7fd06802a180 comp rx=0 tx=0).stop 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2337319580 shutdown_connections 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2337319580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740d02f0 0x7fd0740d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2337319580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0740af720 0x7fd0740afba0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2337319580 >> 172.21.15.148:0/2337319580 conn(0x7fd0740b2370 msgr2=0x7fd0740b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2337319580 shutdown_connections 2024-04-25T10:05:22.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.205+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2337319580 wait complete. 2024-04-25T10:05:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.206+0000 7fd07ab7e640 1 Processor -- start 2024-04-25T10:05:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.206+0000 7fd07ab7e640 1 -- start start 2024-04-25T10:05:22.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07ab7e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740af720 0x7fd0741445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07ab7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0740d02f0 0x7fd07414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07ab7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 0x7fd07415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07ab7e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd074158600 con 0x7fd074158220 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07ab7e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740706a0 con 0x7fd0740d02f0 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07ab7e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074072960 con 0x7fd0740af720 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 0x7fd07415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 0x7fd07415aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47506/0 (socket says 172.21.15.148:47506) 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 -- 172.21.15.148:0/2162518612 learned_addr learned my addr 172.21.15.148:0/2162518612 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07891a640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740af720 0x7fd0741445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd073fff640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0740d02f0 0x7fd07414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 -- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740af720 msgr2=0x7fd0741445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740af720 0x7fd0741445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 -- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0740d02f0 msgr2=0x7fd07414a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0740d02f0 0x7fd07414a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd07911b640 1 -- 172.21.15.148:0/2162518612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07414aac0 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.207+0000 7fd073fff640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0740d02f0 0x7fd07414a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.208+0000 7fd07911b640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 0x7fd07415aa10 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b970 tx=0x7fd06c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.208+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c019070 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.208+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c00cb60 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.208+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd074159970 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.208+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014440 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.208+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd074159f30 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.209+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd074073990 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.209+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd06c01c070 con 0x7fd074158220 2024-04-25T10:05:22.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.210+0000 7fd0717fa640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd06006b4d0 0x7fd06006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.210+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd06c089d00 con 0x7fd074158220 2024-04-25T10:05:22.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.210+0000 7fd07891a640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd06006b4d0 0x7fd06006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.210+0000 7fd07891a640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd06006b4d0 0x7fd06006d990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd064000f60 tx=0x7fd064007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.214+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd07414aac0 con 0x7fd074158220 2024-04-25T10:05:22.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.384+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0740b2b90 con 0x7fd074158220 2024-04-25T10:05:22.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.385+0000 7fd0717fa640 1 -- 172.21.15.148:0/2162518612 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+13353 (secure 0 0 0) 0x7fd06c0586a0 con 0x7fd074158220 2024-04-25T10:05:22.391 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:22.391 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":162,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:05:21.086148+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/3286540466":"2024-04-26T10:05:18.074954+0000","172.21.15.148:6835/3286540466":"2024-04-26T10:05:18.074954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd06006b4d0 msgr2=0x7fd06006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd06006b4d0 0x7fd06006d990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd064000f60 tx=0x7fd064007d50 comp rx=0 tx=0).stop 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 msgr2=0x7fd07415aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 0x7fd07415aa10 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00b970 tx=0x7fd06c00be30 comp rx=0 tx=0).stop 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 shutdown_connections 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd06006b4d0 0x7fd06006d990 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd074158220 0x7fd07415aa10 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0740d02f0 0x7fd07414a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 --2- 172.21.15.148:0/2162518612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0740af720 0x7fd0741445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 >> 172.21.15.148:0/2162518612 conn(0x7fd0740b2370 msgr2=0x7fd074156140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 shutdown_connections 2024-04-25T10:05:22.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.387+0000 7fd07ab7e640 1 -- 172.21.15.148:0/2162518612 wait complete. 2024-04-25T10:05:22.405 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:22.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:22.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.498+0000 7f92abb34640 1 Processor -- start 2024-04-25T10:05:22.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.498+0000 7f92abb34640 1 -- start start 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92abb34640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92a4155f80 0x7f92a406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92abb34640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92a406fa00 0x7f92a40ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92abb34640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40b4d80 con 0x7f92a40ce730 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92abb34640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40b5390 con 0x7f92a406fa00 2024-04-25T10:05:22.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92abb34640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40d07f0 con 0x7f92a4155f80 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a90cf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f92a406fa00 0x7f92a40ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92aa0d1640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f92a40ce730 0x7f92a415f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47150/0 (socket says 172.21.15.148:47150) 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92aa0d1640 1 -- 172.21.15.148:0/4179190444 learned_addr learned my addr 172.21.15.148:0/4179190444 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a98d0640 1 --2- 172.21.15.148:0/4179190444 >> v2:172.21.15.188:3300/0 conn(0x7f92a4155f80 0x7f92a406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a90cf640 1 -- 172.21.15.148:0/4179190444 >> v2:172.21.15.188:3300/0 conn(0x7f92a4155f80 msgr2=0x7f92a406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a90cf640 1 --2- 172.21.15.148:0/4179190444 >> v2:172.21.15.188:3300/0 conn(0x7f92a4155f80 0x7f92a406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a90cf640 1 -- 172.21.15.148:0/4179190444 >> v1:172.21.15.148:6789/0 conn(0x7f92a40ce730 legacy=0x7f92a415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a90cf640 1 -- 172.21.15.148:0/4179190444 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40cef70 con 0x7f92a406fa00 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a88ce640 1 -- 172.21.15.148:0/4179190444 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744789666 0 0) 0x7f92a40b4d80 con 0x7f92a40ce730 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.499+0000 7f92a98d0640 1 --2- 172.21.15.148:0/4179190444 >> v2:172.21.15.188:3300/0 conn(0x7f92a4155f80 0x7f92a406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.500+0000 7f92a90cf640 1 --2- 172.21.15.148:0/4179190444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a406fa00 0x7f92a40ce1f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f92a40d2800 tx=0x7f929402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb177eb81aa6de87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.500+0000 7f92a88ce640 1 -- 172.21.15.148:0/4179190444 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294037070 con 0x7f92a406fa00 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.500+0000 7f92a88ce640 1 -- 172.21.15.148:0/4179190444 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9294032440 con 0x7f92a406fa00 2024-04-25T10:05:22.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.500+0000 7f92a88ce640 1 -- 172.21.15.148:0/4179190444 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294032810 con 0x7f92a406fa00 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 -- 172.21.15.148:0/4179190444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a406fa00 msgr2=0x7f92a40ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 --2- 172.21.15.148:0/4179190444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a406fa00 0x7f92a40ce1f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f92a40d2800 tx=0x7f929402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 -- 172.21.15.148:0/4179190444 shutdown_connections 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 --2- 172.21.15.148:0/4179190444 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a406fa00 0x7f92a40ce1f0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 --2- 172.21.15.148:0/4179190444 >> v2:172.21.15.188:3300/0 conn(0x7f92a4155f80 0x7f92a406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 -- 172.21.15.148:0/4179190444 >> 172.21.15.148:0/4179190444 conn(0x7f92a4061600 msgr2=0x7f92a40619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 -- 172.21.15.148:0/4179190444 shutdown_connections 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.501+0000 7f92abb34640 1 -- 172.21.15.148:0/4179190444 wait complete. 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.502+0000 7f92abb34640 1 Processor -- start 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.502+0000 7f92abb34640 1 -- start start 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a406fa00 0x7f92a40b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce730 0x7f92a414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 0x7f92a414b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40cf250 con 0x7f92a4155f80 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a4156360 con 0x7f92a40ce730 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40cf3d0 con 0x7f92a406fa00 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 0x7f92a414b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 0x7f92a414b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47524/0 (socket says 172.21.15.148:47524) 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 -- 172.21.15.148:0/2592182420 learned_addr learned my addr 172.21.15.148:0/2592182420 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:22.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92a98d0640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a406fa00 0x7f92a40b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 -- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a406fa00 msgr2=0x7f92a40b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92a90cf640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce730 0x7f92a414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a406fa00 0x7f92a40b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 -- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce730 msgr2=0x7f92a414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce730 0x7f92a414a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 -- 172.21.15.148:0/2592182420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40b1870 con 0x7f92a4155f80 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92aa0d1640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 0x7f92a414b110 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f929c00caf0 tx=0x7f929c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00aca0 con 0x7f92a4155f80 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00ae40 con 0x7f92a4155f80 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c012640 con 0x7f92a4155f80 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40b1b50 con 0x7f92a4155f80 2024-04-25T10:05:22.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.503+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a40b2110 con 0x7f92a4155f80 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.504+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a4073a00 con 0x7f92a4155f80 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.505+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f929c0128a0 con 0x7f92a4155f80 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.505+0000 7f929a7fc640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f928806b4d0 0x7f928806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.505+0000 7f92a98d0640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f928806b4d0 0x7f928806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.506+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f929c08a6d0 con 0x7f92a4155f80 2024-04-25T10:05:22.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.506+0000 7f92a98d0640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f928806b4d0 0x7f928806d990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f92a00096d0 tx=0x7f92a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.510+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f929c059750 con 0x7f92a4155f80 2024-04-25T10:05:22.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.678+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92a40b1030 con 0x7f92a4155f80 2024-04-25T10:05:22.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.678+0000 7f929a7fc640 1 -- 172.21.15.148:0/2592182420 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7f929c059050 con 0x7f92a4155f80 2024-04-25T10:05:22.685 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.680+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f928806b4d0 msgr2=0x7f928806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.680+0000 7f92abb34640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f928806b4d0 0x7f928806d990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f92a00096d0 tx=0x7f92a0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.680+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 msgr2=0x7f92a414b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.680+0000 7f92abb34640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 0x7f92a414b110 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f929c00caf0 tx=0x7f929c00a060 comp rx=0 tx=0).stop 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 shutdown_connections 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f928806b4d0 0x7f928806d990 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92a4155f80 0x7f92a414b110 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92a40ce730 0x7f92a414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 --2- 172.21.15.148:0/2592182420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92a406fa00 0x7f92a40b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 >> 172.21.15.148:0/2592182420 conn(0x7f92a4061600 msgr2=0x7f92a40b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 shutdown_connections 2024-04-25T10:05:22.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.681+0000 7f92abb34640 1 -- 172.21.15.148:0/2592182420 wait complete. 2024-04-25T10:05:22.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:05:22.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.789+0000 7f5432114640 1 Processor -- start 2024-04-25T10:05:22.796 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.789+0000 7f5432114640 1 -- start start 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f5432114640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c1602e0 0x7f542c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f5432114640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f542c15f5f0 0x7f542c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f5432114640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0cd710 con 0x7f542c15ba00 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f5432114640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c099b70 con 0x7f542c15f5f0 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f5432114640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c099e50 con 0x7f542c1602e0 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f542bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f542c15f5f0 0x7f542c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f542bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f542c15f5f0 0x7f542c15faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47538/0 (socket says 172.21.15.148:47538) 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f542b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f542c15ba00 0x7f542c09a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47154/0 (socket says 172.21.15.148:47154) 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.790+0000 7f542bfff640 1 -- 172.21.15.148:0/49135254 learned_addr learned my addr 172.21.15.148:0/49135254 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:22.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542affd640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c1602e0 0x7f542c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542bfff640 1 -- 172.21.15.148:0/49135254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c1602e0 msgr2=0x7f542c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542bfff640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c1602e0 0x7f542c0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542bfff640 1 -- 172.21.15.148:0/49135254 >> v1:172.21.15.148:6789/0 conn(0x7f542c15ba00 legacy=0x7f542c09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542a7fc640 1 -- 172.21.15.148:0/49135254 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678146061 0 0) 0x7f541c001110 con 0x7f542c15ba00 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542bfff640 1 -- 172.21.15.148:0/49135254 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c09a130 con 0x7f542c15f5f0 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542affd640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c1602e0 0x7f542c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.791+0000 7f542bfff640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c15f5f0 0x7f542c15faa0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f5424009e50 tx=0x7f542402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e693b9aebebbb697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f542a7fc640 1 -- 172.21.15.148:0/49135254 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424036070 con 0x7f542c15f5f0 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f542a7fc640 1 -- 172.21.15.148:0/49135254 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424031600 con 0x7f542c15f5f0 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f542a7fc640 1 -- 172.21.15.148:0/49135254 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54240319d0 con 0x7f542c15f5f0 2024-04-25T10:05:22.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 -- 172.21.15.148:0/49135254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c15f5f0 msgr2=0x7f542c15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c15f5f0 0x7f542c15faa0 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f5424009e50 tx=0x7f542402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 -- 172.21.15.148:0/49135254 shutdown_connections 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c15f5f0 0x7f542c15faa0 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 --2- 172.21.15.148:0/49135254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c1602e0 0x7f542c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 -- 172.21.15.148:0/49135254 >> 172.21.15.148:0/49135254 conn(0x7f542c0b3270 msgr2=0x7f542c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 -- 172.21.15.148:0/49135254 shutdown_connections 2024-04-25T10:05:22.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.792+0000 7f5432114640 1 -- 172.21.15.148:0/49135254 wait complete. 2024-04-25T10:05:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.793+0000 7f5432114640 1 Processor -- start 2024-04-25T10:05:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.793+0000 7f5432114640 1 -- start start 2024-04-25T10:05:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f5432114640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c15ba00 0x7f542c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f5432114640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c15f5f0 0x7f542c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f5432114640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 0x7f542c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f5432114640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0cd460 con 0x7f542c1602e0 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f5432114640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0cd2e0 con 0x7f542c15f5f0 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f5432114640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0c52d0 con 0x7f542c15ba00 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f542bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 0x7f542c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f542bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 0x7f542c06f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47546/0 (socket says 172.21.15.148:47546) 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.794+0000 7f542bfff640 1 -- 172.21.15.148:0/1828217875 learned_addr learned my addr 172.21.15.148:0/1828217875 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542b7fe640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c15ba00 0x7f542c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542affd640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c15f5f0 0x7f542c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542bfff640 1 -- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c15ba00 msgr2=0x7f542c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542bfff640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c15ba00 0x7f542c158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542bfff640 1 -- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c15f5f0 msgr2=0x7f542c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542bfff640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c15f5f0 0x7f542c06f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542bfff640 1 -- 172.21.15.148:0/1828217875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0cb810 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542b7fe640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c15ba00 0x7f542c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542affd640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c15f5f0 0x7f542c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f542bfff640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 0x7f542c06f9e0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5424034040 tx=0x7f5424004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424041020 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424037040 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c159010 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424036050 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.795+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c0b5260 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.796+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c0b55a0 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.796+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5424007c30 con 0x7f542c1602e0 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.797+0000 7f5413fff640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541806b4d0 0x7f541806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.797+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f54240aaa50 con 0x7f542c1602e0 2024-04-25T10:05:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.797+0000 7f542b7fe640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541806b4d0 0x7f541806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.797+0000 7f542b7fe640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541806b4d0 0x7f541806d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f541c0013b0 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:22.802+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f542c0b55a0 con 0x7f542c1602e0 2024-04-25T10:05:23.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.041+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7f542c0b40a0 con 0x7f542c1602e0 2024-04-25T10:05:23.097 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.091+0000 7f5413fff640 1 -- 172.21.15.148:0/1828217875 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v535) v1 ==== 136+0+0 (secure 0 0 0) 0x7f542404b060 con 0x7f542c1602e0 2024-04-25T10:05:23.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541806b4d0 msgr2=0x7f541806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541806b4d0 0x7f541806d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f541c0013b0 tx=0x7f541c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 msgr2=0x7f542c06f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 0x7f542c06f9e0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5424034040 tx=0x7f5424004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:23.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 shutdown_connections 2024-04-25T10:05:23.101 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f541806b4d0 0x7f541806d990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f542c1602e0 0x7f542c06f9e0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f542c15f5f0 0x7f542c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 --2- 172.21.15.148:0/1828217875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f542c15ba00 0x7f542c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 >> 172.21.15.148:0/1828217875 conn(0x7f542c0b3270 msgr2=0x7f542c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 shutdown_connections 2024-04-25T10:05:23.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.095+0000 7f5432114640 1 -- 172.21.15.148:0/1828217875 wait complete. 2024-04-25T10:05:23.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.207+0000 7f99628e1640 1 Processor -- start 2024-04-25T10:05:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.207+0000 7f99628e1640 1 -- start start 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f99628e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f995c15b7d0 0x7f995c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f99628e1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f995c06c9e0 0x7f995c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f99628e1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995c06cf50 con 0x7f995c0b5620 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f99628e1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995c0ce530 con 0x7f995c06c9e0 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f99628e1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995c0ce810 con 0x7f995c15b7d0 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f9960e7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f995c06c9e0 0x7f995c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f995b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f995c0b5620 0x7f995c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47158/0 (socket says 172.21.15.148:47158) 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f9960e7e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f995c06c9e0 0x7f995c0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47562/0 (socket says 172.21.15.148:47562) 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f995b7fe640 1 -- 172.21.15.148:0/325910433 learned_addr learned my addr 172.21.15.148:0/325910433 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f9960e7e640 1 -- 172.21.15.148:0/325910433 >> v2:172.21.15.188:3300/0 conn(0x7f995c15b7d0 msgr2=0x7f995c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f995bfff640 1 --2- 172.21.15.148:0/325910433 >> v2:172.21.15.188:3300/0 conn(0x7f995c15b7d0 0x7f995c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f9960e7e640 1 --2- 172.21.15.148:0/325910433 >> v2:172.21.15.188:3300/0 conn(0x7f995c15b7d0 0x7f995c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f995affd640 1 -- 172.21.15.148:0/325910433 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851282632 0 0) 0x7f995c06cf50 con 0x7f995c0b5620 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f9960e7e640 1 -- 172.21.15.148:0/325910433 >> v1:172.21.15.148:6789/0 conn(0x7f995c0b5620 legacy=0x7f995c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f9960e7e640 1 -- 172.21.15.148:0/325910433 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0ceaf0 con 0x7f995c06c9e0 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.208+0000 7f995bfff640 1 --2- 172.21.15.148:0/325910433 >> v2:172.21.15.188:3300/0 conn(0x7f995c15b7d0 0x7f995c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.209+0000 7f9960e7e640 1 --2- 172.21.15.148:0/325910433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c06c9e0 0x7f995c0b5ab0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f995c08c0d0 tx=0x7f995402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d98c50bbb3cca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.209+0000 7f995affd640 1 -- 172.21.15.148:0/325910433 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954007740 con 0x7f995c06c9e0 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.209+0000 7f995affd640 1 -- 172.21.15.148:0/325910433 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99540078e0 con 0x7f995c06c9e0 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.209+0000 7f995affd640 1 -- 172.21.15.148:0/325910433 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954007cb0 con 0x7f995c06c9e0 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 -- 172.21.15.148:0/325910433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c06c9e0 msgr2=0x7f995c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 --2- 172.21.15.148:0/325910433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c06c9e0 0x7f995c0b5ab0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7f995c08c0d0 tx=0x7f995402a680 comp rx=0 tx=0).stop 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 -- 172.21.15.148:0/325910433 shutdown_connections 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 --2- 172.21.15.148:0/325910433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c06c9e0 0x7f995c0b5ab0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 --2- 172.21.15.148:0/325910433 >> v2:172.21.15.188:3300/0 conn(0x7f995c15b7d0 0x7f995c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 -- 172.21.15.148:0/325910433 >> 172.21.15.148:0/325910433 conn(0x7f995c0d0800 msgr2=0x7f995c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 -- 172.21.15.148:0/325910433 shutdown_connections 2024-04-25T10:05:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.210+0000 7f99628e1640 1 -- 172.21.15.148:0/325910433 wait complete. 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.211+0000 7f99628e1640 1 Processor -- start 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.211+0000 7f99628e1640 1 -- start start 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f99628e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f995c06c9e0 0x7f995c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f99628e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f995c0b5620 0x7f995c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f99628e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 0x7f995c1f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f99628e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0cf6d0 con 0x7f995c15b7d0 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f99628e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0cf550 con 0x7f995c06c9e0 2024-04-25T10:05:23.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f99628e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c15bbb0 con 0x7f995c0b5620 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 0x7f995c1f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 0x7f995c1f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47566/0 (socket says 172.21.15.148:47566) 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 -- 172.21.15.148:0/4247899767 learned_addr learned my addr 172.21.15.148:0/4247899767 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f995bfff640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f995c06c9e0 0x7f995c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 -- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f995c0b5620 msgr2=0x7f995c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f995b7fe640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f995c0b5620 0x7f995c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f995c0b5620 0x7f995c0b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 -- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f995c06c9e0 msgr2=0x7f995c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f995c06c9e0 0x7f995c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 -- 172.21.15.148:0/4247899767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0b3070 con 0x7f995c15b7d0 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f995bfff640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f995c06c9e0 0x7f995c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.212+0000 7f9960e7e640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 0x7f995c1f6a50 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f995402ac70 tx=0x7f9954004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.213+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954040070 con 0x7f995c15b7d0 2024-04-25T10:05:23.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.213+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954031bc0 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.213+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c1f77d0 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.213+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954045460 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.213+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c0cd680 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.214+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c073a00 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.214+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9954039040 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.215+0000 7f9958ff9640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.215+0000 7f995bfff640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.215+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f99540ab9f0 con 0x7f995c15b7d0 2024-04-25T10:05:23.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.215+0000 7f995bfff640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f994806b4d0 0x7f994806d990 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f994c006fd0 tx=0x7f994c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.219+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f995407aaa0 con 0x7f995c15b7d0 2024-04-25T10:05:23.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.389+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f995c0b3240 con 0x7f995c15b7d0 2024-04-25T10:05:23.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.392+0000 7f9958ff9640 1 -- 172.21.15.148:0/4247899767 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f995407a3a0 con 0x7f995c15b7d0 2024-04-25T10:05:23.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f994806b4d0 msgr2=0x7f994806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f994806b4d0 0x7f994806d990 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f994c006fd0 tx=0x7f994c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 msgr2=0x7f995c1f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 0x7f995c1f6a50 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7f995402ac70 tx=0x7f9954004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 shutdown_connections 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f994806b4d0 0x7f994806d990 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f995c15b7d0 0x7f995c1f6a50 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f995c0b5620 0x7f995c0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 --2- 172.21.15.148:0/4247899767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f995c06c9e0 0x7f995c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 >> 172.21.15.148:0/4247899767 conn(0x7f995c0d0800 msgr2=0x7f995c160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 shutdown_connections 2024-04-25T10:05:23.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.395+0000 7f99628e1640 1 -- 172.21.15.148:0/4247899767 wait complete. 2024-04-25T10:05:23.411 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:23.412 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:23.450 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:23.494 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:23.494 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:23.494 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:23.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:23.511 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:05:23.511 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:23.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:23.552 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:23.552 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:23.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:23.568 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:05:23.568 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:23.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:23.611 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:23.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:23.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.703+0000 7fa665ae1640 1 Processor -- start 2024-04-25T10:05:23.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.703+0000 7fa665ae1640 1 -- start start 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.703+0000 7fa665ae1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6600cc8e0 0x7fa6601625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.703+0000 7fa665ae1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6600cce50 con 0x7fa6600d1810 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.703+0000 7fa665ae1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa660161af0 con 0x7fa66015dc90 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.703+0000 7fa665ae1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa660161dd0 con 0x7fa6600cc8e0 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa6600cc8e0 0x7fa6601625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa6600d1810 0x7fa6600cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55668/0 (socket says 172.21.15.148:55668) 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65ffff640 1 -- 172.21.15.148:0/486932664 learned_addr learned my addr 172.21.15.148:0/486932664 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:23.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65effd640 1 --2- 172.21.15.148:0/486932664 >> v2:172.21.15.148:3300/0 conn(0x7fa6600cc8e0 0x7fa6601625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48260/0 (socket says 172.21.15.148:48260) 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65effd640 1 -- 172.21.15.148:0/486932664 >> v1:172.21.15.188:6789/0 conn(0x7fa66015dc90 legacy=0x7fa66015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65effd640 1 -- 172.21.15.148:0/486932664 >> v1:172.21.15.148:6789/0 conn(0x7fa6600d1810 legacy=0x7fa6600cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65e7fc640 1 -- 172.21.15.148:0/486932664 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371830205 0 0) 0x7fa6600cce50 con 0x7fa6600d1810 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.704+0000 7fa65effd640 1 -- 172.21.15.148:0/486932664 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6601620b0 con 0x7fa6600cc8e0 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa65effd640 1 --2- 172.21.15.148:0/486932664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600cc8e0 0x7fa6601625d0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa6540097a0 tx=0x7fa65402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a862a7f028771549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa65e7fc640 1 -- 172.21.15.148:0/486932664 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654037070 con 0x7fa6600cc8e0 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa65e7fc640 1 -- 172.21.15.148:0/486932664 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa654032440 con 0x7fa6600cc8e0 2024-04-25T10:05:23.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa65e7fc640 1 -- 172.21.15.148:0/486932664 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654032810 con 0x7fa6600cc8e0 2024-04-25T10:05:23.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 -- 172.21.15.148:0/486932664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600cc8e0 msgr2=0x7fa6601625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 --2- 172.21.15.148:0/486932664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600cc8e0 0x7fa6601625d0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa6540097a0 tx=0x7fa65402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 -- 172.21.15.148:0/486932664 shutdown_connections 2024-04-25T10:05:23.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 --2- 172.21.15.148:0/486932664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600cc8e0 0x7fa6601625d0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 -- 172.21.15.148:0/486932664 >> 172.21.15.148:0/486932664 conn(0x7fa66006f590 msgr2=0x7fa66006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 -- 172.21.15.148:0/486932664 shutdown_connections 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.705+0000 7fa665ae1640 1 -- 172.21.15.148:0/486932664 wait complete. 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.706+0000 7fa665ae1640 1 Processor -- start 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 -- start start 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6600cc8e0 0x7fa6600c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 0x7fa6600c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa66015dc90 0x7fa6600b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015e1f0 con 0x7fa6600d1810 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015e070 con 0x7fa66015dc90 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa665ae1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600d1c80 con 0x7fa6600cc8e0 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa65effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 0x7fa6600c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa65effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 0x7fa6600c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48274/0 (socket says 172.21.15.148:48274) 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.707+0000 7fa65effd640 1 -- 172.21.15.148:0/289751432 learned_addr learned my addr 172.21.15.148:0/289751432 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65ffff640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa66015dc90 0x7fa6600b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65effd640 1 -- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6600cc8e0 msgr2=0x7fa6600c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65f7fe640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6600cc8e0 0x7fa6600c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65effd640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6600cc8e0 0x7fa6600c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65effd640 1 -- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa66015dc90 msgr2=0x7fa6600b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65effd640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa66015dc90 0x7fa6600b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65effd640 1 -- 172.21.15.148:0/289751432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600b5530 con 0x7fa6600d1810 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65ffff640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa66015dc90 0x7fa6600b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65f7fe640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6600cc8e0 0x7fa6600c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa65effd640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 0x7fa6600c8bc0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fa6540098c0 tx=0x7fa6540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654037040 con 0x7fa6600d1810 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa654042070 con 0x7fa6600d1810 2024-04-25T10:05:23.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6600cb9d0 con 0x7fa6600d1810 2024-04-25T10:05:23.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654030a70 con 0x7fa6600d1810 2024-04-25T10:05:23.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.708+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6600cbf90 con 0x7fa6600d1810 2024-04-25T10:05:23.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.709+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa660073990 con 0x7fa6600d1810 2024-04-25T10:05:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.710+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa654030c10 con 0x7fa6600d1810 2024-04-25T10:05:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.710+0000 7fa647fff640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa64c06b4d0 0x7fa64c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.710+0000 7fa65f7fe640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa64c06b4d0 0x7fa64c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.710+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa6540aab20 con 0x7fa6600d1810 2024-04-25T10:05:23.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.710+0000 7fa65f7fe640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa64c06b4d0 0x7fa64c06d990 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa650000f60 tx=0x7fa650009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.714+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa654079b10 con 0x7fa6600d1810 2024-04-25T10:05:23.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.921+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6600cf6b0 con 0x7fa6600d1810 2024-04-25T10:05:23.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.921+0000 7fa647fff640 1 -- 172.21.15.148:0/289751432 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+3707 (secure 0 0 0) 0x7fa654079410 con 0x7fa6600d1810 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":95,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7313,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/62552984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":62552984},{"type":"v1","addr":"172.21.15.188:6834","nonce":62552984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":7316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1207475566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1207475566},{"type":"v1","addr":"172.21.15.188:6835","nonce":1207475566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10362,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/4040763396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4040763396},{"type":"v1","addr":"172.21.15.148:6836","nonce":4040763396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":90},{"gid":10740,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4209248359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":4209248359},{"type":"v1","addr":"172.21.15.148:6837","nonce":4209248359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[]} 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 95 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa64c06b4d0 msgr2=0x7fa64c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa64c06b4d0 0x7fa64c06d990 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa650000f60 tx=0x7fa650009040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 msgr2=0x7fa6600c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 0x7fa6600c8bc0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fa6540098c0 tx=0x7fa6540046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 shutdown_connections 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa64c06b4d0 0x7fa64c06d990 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa66015dc90 0x7fa6600b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6600d1810 0x7fa6600c8bc0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 --2- 172.21.15.148:0/289751432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6600cc8e0 0x7fa6600c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 >> 172.21.15.148:0/289751432 conn(0x7fa66006f590 msgr2=0x7fa6600cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 shutdown_connections 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:23.923+0000 7fa665ae1640 1 -- 172.21.15.148:0/289751432 wait complete. 2024-04-25T10:05:23.940 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:23.940 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:23.940 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:23.940 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:23.940 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:23.940 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:23.941 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:23.941 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:23.941 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:23.941 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:23.941 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:23.941 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:23.941 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:23.941 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:23.942 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:23.942 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:23.949 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:23.950 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:23.951 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:23.951 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:23.953 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:23.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:23.954 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:05:23.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:23.992 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:23.993 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:23.993 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:23.994 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:23.996 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:23.996 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:23.996 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:23.997 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:23.997 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:05:24.066 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:05:24.067 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:05:24.075 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:05:24.076 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:05:24.076 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-25T10:05:24.071+0000 7ff677037640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.094+0000 7f4dd0e6a640 1 Processor -- start 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.094+0000 7f4dd0e6a640 1 -- start start 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dd0e6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4dcc15bb20 0x7f4dcc0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dd0e6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dcc0cb190 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dd0e6a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dcc0cb470 con 0x7f4dcc0d00e0 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dd0e6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dcc0cb750 con 0x7f4dcc15bb20 2024-04-25T10:05:24.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dc3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4dcc0ccdd0 0x7f4dcc0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55684/0 (socket says 172.21.15.148:55684) 2024-04-25T10:05:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dc3fff640 1 -- 172.21.15.148:0/2013295677 learned_addr learned my addr 172.21.15.148:0/2013295677 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dca59c640 1 --2- 172.21.15.148:0/2013295677 >> v2:172.21.15.188:3300/0 conn(0x7f4dcc15bb20 0x7f4dcc0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dca59c640 1 -- 172.21.15.148:0/2013295677 >> v1:172.21.15.188:6789/0 conn(0x7f4dcc0d00e0 legacy=0x7f4dcc0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dca59c640 1 -- 172.21.15.148:0/2013295677 >> v1:172.21.15.148:6789/0 conn(0x7f4dcc0ccdd0 legacy=0x7f4dcc0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.095+0000 7f4dca59c640 1 -- 172.21.15.148:0/2013295677 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0cba30 con 0x7f4dcc15bb20 2024-04-25T10:05:24.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.096+0000 7f4dca59c640 1 --2- 172.21.15.148:0/2013295677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc0cc230 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0097a0 tx=0x7f4dbc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71cac657281d971d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.101+0000 7f4dc9d9b640 1 -- 172.21.15.148:0/2013295677 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc037070 con 0x7f4dcc15bb20 2024-04-25T10:05:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.101+0000 7f4dc9d9b640 1 -- 172.21.15.148:0/2013295677 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc032440 con 0x7f4dcc15bb20 2024-04-25T10:05:24.106 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.101+0000 7f4dc9d9b640 1 -- 172.21.15.148:0/2013295677 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc032810 con 0x7f4dcc15bb20 2024-04-25T10:05:24.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2013295677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 msgr2=0x7f4dcc0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2013295677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc0cc230 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0097a0 tx=0x7f4dbc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.107 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2013295677 shutdown_connections 2024-04-25T10:05:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2013295677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc0cc230 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2013295677 >> 172.21.15.148:0/2013295677 conn(0x7f4dcc0b2390 msgr2=0x7f4dcc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2013295677 shutdown_connections 2024-04-25T10:05:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.102+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2013295677 wait complete. 2024-04-25T10:05:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.103+0000 7f4dd0e6a640 1 Processor -- start 2024-04-25T10:05:24.108 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.103+0000 7f4dd0e6a640 1 -- start start 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dd0e6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 0x7f4dcc1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dd0e6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc0d00e0 0x7f4dcc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dd0e6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dd0e6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc15bf00 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dd0e6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0cc770 con 0x7f4dcc15bb20 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dd0e6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc15c080 con 0x7f4dcc0d00e0 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 0x7f4dcc1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 0x7f4dcc1f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48322/0 (socket says 172.21.15.148:48322) 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 -- 172.21.15.148:0/2967380029 learned_addr learned my addr 172.21.15.148:0/2967380029 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:24.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dc3fff640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc0d00e0 0x7f4dcc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dcad9d640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 -- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc0d00e0 msgr2=0x7f4dcc0b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc0d00e0 0x7f4dcc0b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 -- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 msgr2=0x7f4dcc150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.104+0000 7f4dca59c640 1 -- 172.21.15.148:0/2967380029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0b4cb0 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.105+0000 7f4dca59c640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 0x7f4dcc1f5a60 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc02a790 tx=0x7f4dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.106+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc040020 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.106+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc038040 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.106+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc0b4f90 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.106+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc037070 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.106+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc0b5550 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.107+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc067be0 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.107+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4dbc043020 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.108+0000 7f4dc2ffd640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4dac06b4d0 0x7f4dac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.108+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4dbc0aa3a0 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.108+0000 7f4dc3fff640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4dac06b4d0 0x7f4dac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.108+0000 7f4dc3fff640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4dac06b4d0 0x7f4dac06d990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0b0cc0 tx=0x7f4db4007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.112+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4dcc067be0 con 0x7f4dcc0ccdd0 2024-04-25T10:05:24.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:24.299+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4dcc073a00 con 0x7f4dcc0ccdd0 2024-04-25T10:05:25.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.117+0000 7f4dc2ffd640 1 -- 172.21.15.148:0/2967380029 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v167) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4dbc04a030 con 0x7f4dcc0ccdd0 2024-04-25T10:05:25.123 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4dac06b4d0 msgr2=0x7f4dac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4dac06b4d0 0x7f4dac06d990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc0b0cc0 tx=0x7f4db4007e80 comp rx=0 tx=0).stop 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 msgr2=0x7f4dcc1f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 0x7f4dcc1f5a60 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc02a790 tx=0x7f4dbc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 shutdown_connections 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4dac06b4d0 0x7f4dac06d990 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dcc15bb20 0x7f4dcc150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dcc0d00e0 0x7f4dcc0b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 --2- 172.21.15.148:0/2967380029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dcc0ccdd0 0x7f4dcc1f5a60 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 >> 172.21.15.148:0/2967380029 conn(0x7f4dcc0b2390 msgr2=0x7f4dcc0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 shutdown_connections 2024-04-25T10:05:25.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.120+0000 7f4dd0e6a640 1 -- 172.21.15.148:0/2967380029 wait complete. 2024-04-25T10:05:25.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:05:25.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.238+0000 7fd9df91d640 1 Processor -- start 2024-04-25T10:05:25.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.238+0000 7fd9df91d640 1 -- start start 2024-04-25T10:05:25.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9df91d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9d815dc00 0x7fd9d80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9df91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 0x7fd9d80d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9df91d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d815dfe0 con 0x7fd9d80d1c00 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9df91d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d80d2060 con 0x7fd9d815dc00 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9df91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80d00c0 con 0x7fd9d80ceb20 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9ddeba640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9d80d1c00 0x7fd9d80ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55690/0 (socket says 172.21.15.148:55690) 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9ddeba640 1 -- 172.21.15.148:0/2428330111 learned_addr learned my addr 172.21.15.148:0/2428330111 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9dceb8640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 0x7fd9d80d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.239+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909404129 0 0) 0x7fd9d0001110 con 0x7fd9d80d1c00 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9dd6b9640 1 --2- 172.21.15.148:0/2428330111 >> v2:172.21.15.188:3300/0 conn(0x7fd9d815dc00 0x7fd9d80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c4003520 con 0x7fd9d80d1c00 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3605212551 0 0) 0x7fd9c4003520 con 0x7fd9d80d1c00 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 msgr2=0x7fd9d80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 0x7fd9d80d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 >> v2:172.21.15.188:3300/0 conn(0x7fd9d815dc00 msgr2=0x7fd9d80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 --2- 172.21.15.148:0/2428330111 >> v2:172.21.15.188:3300/0 conn(0x7fd9d815dc00 0x7fd9d80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80d03a0 con 0x7fd9d80d1c00 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd9d0001840 con 0x7fd9d80d1c00 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 >> v1:172.21.15.148:6789/0 conn(0x7fd9d80d1c00 legacy=0x7fd9d80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9c40039b0 0x7fd9c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 0x7fd9c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80d2060 con 0x7fd9c4004d40 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80d00c0 con 0x7fd9c40039b0 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400b9c0 con 0x7fd9d80ceb20 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9ddeba640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9dd6b9640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9c40039b0 0x7fd9c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9dceb8640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 0x7fd9c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9ddeba640 1 -- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 msgr2=0x7fd9c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9ddeba640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d80ceb20 0x7fd9c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9ddeba640 1 -- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9c40039b0 msgr2=0x7fd9c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9ddeba640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9c40039b0 0x7fd9c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.240+0000 7fd9ddeba640 1 -- 172.21.15.148:0/2428330111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4003520 con 0x7fd9c4004d40 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.241+0000 7fd9ddeba640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9c40051c0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c490 tx=0x7fd9c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8039b8e29a7feb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.241+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0032020 con 0x7fd9c4004d40 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.241+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d0005ad0 con 0x7fd9c4004d40 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.241+0000 7fd9cffff640 1 -- 172.21.15.148:0/2428330111 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000c0e0 con 0x7fd9c4004d40 2024-04-25T10:05:25.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9df91d640 1 -- 172.21.15.148:0/2428330111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 msgr2=0x7fd9c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9df91d640 1 --2- 172.21.15.148:0/2428330111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9c40051c0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c490 tx=0x7fd9c4005830 comp rx=0 tx=0).stop 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9ddeba640 1 -- 172.21.15.148:0/2428330111 reap_dead start 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9df91d640 1 -- 172.21.15.148:0/2428330111 shutdown_connections 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9df91d640 1 -- 172.21.15.148:0/2428330111 >> 172.21.15.148:0/2428330111 conn(0x7fd9d806f5d0 msgr2=0x7fd9d806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9df91d640 1 -- 172.21.15.148:0/2428330111 shutdown_connections 2024-04-25T10:05:25.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.242+0000 7fd9df91d640 1 -- 172.21.15.148:0/2428330111 wait complete. 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.243+0000 7fd9df91d640 1 Processor -- start 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.243+0000 7fd9df91d640 1 -- start start 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9d80b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d815af20 0x7fd9d8158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d815c680 0x7fd9d815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80cc130 con 0x7fd9c4004d40 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80cc720 con 0x7fd9d815af20 2024-04-25T10:05:25.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80cfd90 con 0x7fd9d815c680 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9d80b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9d80b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48334/0 (socket says 172.21.15.148:48334) 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 -- 172.21.15.148:0/873693334 learned_addr learned my addr 172.21.15.148:0/873693334 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dceb8640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d815af20 0x7fd9d8158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9ddeba640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d815c680 0x7fd9d815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 -- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d815c680 msgr2=0x7fd9d815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d815c680 0x7fd9d815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 -- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d815af20 msgr2=0x7fd9d8158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d815af20 0x7fd9d8158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 -- 172.21.15.148:0/873693334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d815b790 con 0x7fd9c4004d40 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9dd6b9640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9d80b5680 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009fc0 tx=0x7fd9d400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400fea0 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4015630 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.244+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d815ba70 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.245+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4015a20 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.245+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d815c030 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.246+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d8073990 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.246+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9d4015bc0 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.247+0000 7fd9cdffb640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9bc06b500 0x7fd9bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.247+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd9d408b220 con 0x7fd9c4004d40 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.247+0000 7fd9dceb8640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9bc06b500 0x7fd9bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.247+0000 7fd9dceb8640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9bc06b500 0x7fd9bc06d9c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80b46d0 tx=0x7fd9c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.251+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9d4019030 con 0x7fd9c4004d40 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.485+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd9d80cf750 con 0x7fd9c4004d40 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.486+0000 7fd9cdffb640 1 -- 172.21.15.148:0/873693334 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd9d4010040 con 0x7fd9c4004d40 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.491 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:05:25.492 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:05:25.493 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.494 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:05:25.495 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.496 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.497 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.498 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.499 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.500 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:05:25.501 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.502 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.503 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:25.504 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:05:25.505 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:05:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9bc06b500 msgr2=0x7fd9bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9bc06b500 0x7fd9bc06d9c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80b46d0 tx=0x7fd9c8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 msgr2=0x7fd9d80b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9d80b5680 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009fc0 tx=0x7fd9d400f580 comp rx=0 tx=0).stop 2024-04-25T10:05:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 shutdown_connections 2024-04-25T10:05:25.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9bc06b500 0x7fd9bc06d9c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d815c680 0x7fd9d815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d815af20 0x7fd9d8158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 --2- 172.21.15.148:0/873693334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9c4004d40 0x7fd9d80b5680 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 >> 172.21.15.148:0/873693334 conn(0x7fd9d806f5d0 msgr2=0x7fd9d80b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 shutdown_connections 2024-04-25T10:05:25.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.488+0000 7fd9df91d640 1 -- 172.21.15.148:0/873693334 wait complete. 2024-04-25T10:05:25.507 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:25.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.598+0000 7fe254390640 1 Processor -- start 2024-04-25T10:05:25.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.598+0000 7fe254390640 1 -- start start 2024-04-25T10:05:25.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.599+0000 7fe254390640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe24c15dc00 0x7fe24c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.599+0000 7fe254390640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.599+0000 7fe254390640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c15dfe0 con 0x7fe24c0ceb20 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.599+0000 7fe254390640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0d00c0 con 0x7fe24c15dc00 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.599+0000 7fe254390640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0d03a0 con 0x7fe24c0d1dd0 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25212c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe24c15dc00 0x7fe24c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25292d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25192b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe24c0ceb20 0x7fe24c0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:47944/0 (socket says 172.21.15.148:47944) 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25192b640 1 -- 172.21.15.148:0/3789189588 learned_addr learned my addr 172.21.15.148:0/3789189588 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25212c640 1 -- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 msgr2=0x7fe24c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25212c640 1 --2- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25212c640 1 -- 172.21.15.148:0/3789189588 >> v1:172.21.15.188:6789/0 conn(0x7fe24c0ceb20 legacy=0x7fe24c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25212c640 1 -- 172.21.15.148:0/3789189588 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0cc440 con 0x7fe24c15dc00 2024-04-25T10:05:25.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25112a640 1 -- 172.21.15.148:0/3789189588 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1407670 0 0) 0x7fe248001110 con 0x7fe24c0ceb20 2024-04-25T10:05:25.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.600+0000 7fe25292d640 1 --2- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:25.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.601+0000 7fe25212c640 1 --2- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c15dc00 0x7fe24c0c6df0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0097a0 tx=0x7fe23c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854b8d74313bd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.601+0000 7fe25112a640 1 -- 172.21.15.148:0/3789189588 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c037070 con 0x7fe24c15dc00 2024-04-25T10:05:25.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.601+0000 7fe25112a640 1 -- 172.21.15.148:0/3789189588 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c032440 con 0x7fe24c15dc00 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.601+0000 7fe25112a640 1 -- 172.21.15.148:0/3789189588 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c032810 con 0x7fe24c15dc00 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.601+0000 7fe254390640 1 -- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c15dc00 msgr2=0x7fe24c0c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.601+0000 7fe254390640 1 --2- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c15dc00 0x7fe24c0c6df0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0097a0 tx=0x7fe23c02a180 comp rx=0 tx=0).stop 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.602+0000 7fe254390640 1 -- 172.21.15.148:0/3789189588 shutdown_connections 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.602+0000 7fe254390640 1 --2- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.602+0000 7fe254390640 1 --2- 172.21.15.148:0/3789189588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c15dc00 0x7fe24c0c6df0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.602+0000 7fe254390640 1 -- 172.21.15.148:0/3789189588 >> 172.21.15.148:0/3789189588 conn(0x7fe24c06f5d0 msgr2=0x7fe24c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.602+0000 7fe254390640 1 -- 172.21.15.148:0/3789189588 shutdown_connections 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.602+0000 7fe254390640 1 -- 172.21.15.148:0/3789189588 wait complete. 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 Processor -- start 2024-04-25T10:05:25.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 -- start start 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c0ceb20 0x7fe24c0b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 0x7fe24c1f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0cfd90 con 0x7fe24c15dc00 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0cc130 con 0x7fe24c0ceb20 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.603+0000 7fe254390640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0cdd70 con 0x7fe24c0d1dd0 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 0x7fe24c1f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 0x7fe24c1f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48340/0 (socket says 172.21.15.148:48340) 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 -- 172.21.15.148:0/926727984 learned_addr learned my addr 172.21.15.148:0/926727984 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25192b640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25212c640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c0ceb20 0x7fe24c0b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 -- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 msgr2=0x7fe24c0b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 -- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c0ceb20 msgr2=0x7fe24c0b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c0ceb20 0x7fe24c0b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 -- 172.21.15.148:0/926727984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c0b6250 con 0x7fe24c15dc00 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe25292d640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 0x7fe24c1f3510 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fe24400efb0 tx=0x7fe24400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244018020 con 0x7fe24c15dc00 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe244014480 con 0x7fe24c15dc00 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c0b6500 con 0x7fe24c15dc00 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244014850 con 0x7fe24c15dc00 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.604+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c0b6ac0 con 0x7fe24c15dc00 2024-04-25T10:05:25.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.605+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c159f00 con 0x7fe24c15dc00 2024-04-25T10:05:25.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.606+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe244005d10 con 0x7fe24c15dc00 2024-04-25T10:05:25.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.606+0000 7fe242ffd640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.606+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe24408ac30 con 0x7fe24c15dc00 2024-04-25T10:05:25.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.606+0000 7fe25212c640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.607+0000 7fe25212c640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe23006b4d0 0x7fe23006d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe23c034000 tx=0x7fe23c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.610+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe24c159f00 con 0x7fe24c15dc00 2024-04-25T10:05:25.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:25.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:25.790 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.785+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe24c0c66c0 con 0x7fe24c15dc00 2024-04-25T10:05:25.791 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.786+0000 7fe242ffd640 1 -- 172.21.15.148:0/926727984 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+13222 (secure 0 0 0) 0x7fe244022030 con 0x7fe24c15dc00 2024-04-25T10:05:25.791 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:25.791 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":167,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:05:25.116765+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:25.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe23006b4d0 msgr2=0x7fe23006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe23006b4d0 0x7fe23006d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe23c034000 tx=0x7fe23c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 msgr2=0x7fe24c1f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 0x7fe24c1f3510 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7fe24400efb0 tx=0x7fe24400c490 comp rx=0 tx=0).stop 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 shutdown_connections 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe23006b4d0 0x7fe23006d990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe24c15dc00 0x7fe24c1f3510 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe24c0d1dd0 0x7fe24c0b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 --2- 172.21.15.148:0/926727984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe24c0ceb20 0x7fe24c0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 >> 172.21.15.148:0/926727984 conn(0x7fe24c06f5d0 msgr2=0x7fe24c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 shutdown_connections 2024-04-25T10:05:25.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.788+0000 7fe254390640 1 -- 172.21.15.148:0/926727984 wait complete. 2024-04-25T10:05:25.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:25.805 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.900+0000 7fe953fd5640 1 Processor -- start 2024-04-25T10:05:25.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.900+0000 7fe953fd5640 1 -- start start 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe953fd5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe94c0cc4f0 0x7fe94c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe953fd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe94c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe953fd5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c072280 con 0x7fe94c0cdb20 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe953fd5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94c071270 con 0x7fe94c0cc4f0 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe953fd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0714c0 con 0x7fe94c15b980 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe951570640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe94c0cdb20 0x7fe94c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55698/0 (socket says 172.21.15.148:55698) 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe951570640 1 -- 172.21.15.148:0/722235307 learned_addr learned my addr 172.21.15.148:0/722235307 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:25.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe951d71640 1 --2- 172.21.15.148:0/722235307 >> v2:172.21.15.188:3300/0 conn(0x7fe94c0cc4f0 0x7fe94c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe952572640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe94c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805928559 0 0) 0x7fe94c072280 con 0x7fe94c0cdb20 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.901+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe938003520 con 0x7fe94c0cdb20 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2440112015 0 0) 0x7fe938003520 con 0x7fe94c0cdb20 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 msgr2=0x7fe94c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe94c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 >> v2:172.21.15.188:3300/0 conn(0x7fe94c0cc4f0 msgr2=0x7fe94c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 --2- 172.21.15.148:0/722235307 >> v2:172.21.15.188:3300/0 conn(0x7fe94c0cc4f0 0x7fe94c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c0717a0 con 0x7fe94c0cdb20 2024-04-25T10:05:25.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe952572640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe94c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe93c001690 con 0x7fe94c0cdb20 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 >> v1:172.21.15.148:6789/0 conn(0x7fe94c0cdb20 legacy=0x7fe94c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe9380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c071270 con 0x7fe938004800 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0714c0 con 0x7fe9380039b0 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93800b9c0 con 0x7fe94c15b980 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951570640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe952572640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe9380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951d71640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951570640 1 -- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 msgr2=0x7fe9380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951570640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c15b980 0x7fe9380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951570640 1 -- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9380039b0 msgr2=0x7fe938003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951570640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951570640 1 -- 172.21.15.148:0/722235307 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938003520 con 0x7fe938004800 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.902+0000 7fe951d71640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe951570640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe938004c80 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c210 tx=0x7fe938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68f66cba710e36c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c007d60 con 0x7fe938004800 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c005af0 con 0x7fe938004800 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe950d6f640 1 -- 172.21.15.148:0/722235307 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c007070 con 0x7fe938004800 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe953fd5640 1 -- 172.21.15.148:0/722235307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 msgr2=0x7fe938004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe953fd5640 1 --2- 172.21.15.148:0/722235307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe938004c80 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c210 tx=0x7fe938005830 comp rx=0 tx=0).stop 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe952572640 1 -- 172.21.15.148:0/722235307 reap_dead start 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe953fd5640 1 -- 172.21.15.148:0/722235307 shutdown_connections 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.903+0000 7fe953fd5640 1 -- 172.21.15.148:0/722235307 >> 172.21.15.148:0/722235307 conn(0x7fe94c06abe0 msgr2=0x7fe94c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.904+0000 7fe953fd5640 1 -- 172.21.15.148:0/722235307 shutdown_connections 2024-04-25T10:05:25.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.904+0000 7fe953fd5640 1 -- 172.21.15.148:0/722235307 wait complete. 2024-04-25T10:05:25.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 Processor -- start 2024-04-25T10:05:25.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 -- start start 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe94c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c14beb0 0x7fe94c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe94c14c310 0x7fe94c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0cdfb0 con 0x7fe938004800 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c071d60 con 0x7fe94c14c310 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe953fd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0ce130 con 0x7fe94c14beb0 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.905+0000 7fe951d71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe94c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe94c146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48358/0 (socket says 172.21.15.148:48358) 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 -- 172.21.15.148:0/2819070088 learned_addr learned my addr 172.21.15.148:0/2819070088 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951570640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c14beb0 0x7fe94c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe952572640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe94c14c310 0x7fe94c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 -- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c14beb0 msgr2=0x7fe94c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c14beb0 0x7fe94c1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 -- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe94c14c310 msgr2=0x7fe94c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe94c14c310 0x7fe94c159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 -- 172.21.15.148:0/2819070088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c149750 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe951d71640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe94c146110 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fe948011540 tx=0x7fe948011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948015870 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948015a10 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c149a30 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948015dc0 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.906+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c14d6e0 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.907+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c149f50 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.908+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe94800f460 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.908+0000 7fe9427fc640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.908+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe94808a8f0 con 0x7fe938004800 2024-04-25T10:05:25.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.908+0000 7fe951570640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.909+0000 7fe951570640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe93c033000 tx=0x7fe93c009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:25.912+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe948019030 con 0x7fe938004800 2024-04-25T10:05:26.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.084+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe94c073a00 con 0x7fe938004800 2024-04-25T10:05:26.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.085+0000 7fe9427fc640 1 -- 172.21.15.148:0/2819070088 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9480599c0 con 0x7fe938004800 2024-04-25T10:05:26.090 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 msgr2=0x7fe93006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe93c033000 tx=0x7fe93c009030 comp rx=0 tx=0).stop 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 msgr2=0x7fe94c146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe94c146110 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fe948011540 tx=0x7fe948011a00 comp rx=0 tx=0).stop 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 shutdown_connections 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe93006b4d0 0x7fe93006d990 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe94c14c310 0x7fe94c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe94c14beb0 0x7fe94c1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 --2- 172.21.15.148:0/2819070088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe938004800 0x7fe94c146110 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 >> 172.21.15.148:0/2819070088 conn(0x7fe94c06abe0 msgr2=0x7fe94c0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 shutdown_connections 2024-04-25T10:05:26.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.087+0000 7fe953fd5640 1 -- 172.21.15.148:0/2819070088 wait complete. 2024-04-25T10:05:26.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:26.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.196+0000 7fc79055c640 1 Processor -- start 2024-04-25T10:05:26.211 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.196+0000 7fc79055c640 1 -- start start 2024-04-25T10:05:26.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc79055c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc78815dd30 0x7fc7880b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc79055c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7880d0ad0 0x7fc7880b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc79055c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7880d1df0 con 0x7fc7880b9770 2024-04-25T10:05:26.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc79055c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7880d20d0 con 0x7fc78815dd30 2024-04-25T10:05:26.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc79055c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880d23b0 con 0x7fc7880d0ad0 2024-04-25T10:05:26.212 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc78815dd30 0x7fc7880b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc78815dd30 0x7fc7880b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48362/0 (socket says 172.21.15.148:48362) 2024-04-25T10:05:26.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 -- 172.21.15.148:0/2645092717 learned_addr learned my addr 172.21.15.148:0/2645092717 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:26.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78eaf9640 1 --2- 172.21.15.148:0/2645092717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7880d0ad0 0x7fc7880b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 -- 172.21.15.148:0/2645092717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7880d0ad0 msgr2=0x7fc7880b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 --2- 172.21.15.148:0/2645092717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7880d0ad0 0x7fc7880b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 -- 172.21.15.148:0/2645092717 >> v1:172.21.15.148:6789/0 conn(0x7fc7880b9770 legacy=0x7fc7880d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.197+0000 7fc78e2f8640 1 -- 172.21.15.148:0/2645092717 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880d2690 con 0x7fc78815dd30 2024-04-25T10:05:26.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.198+0000 7fc78e2f8640 1 --2- 172.21.15.148:0/2645092717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc78815dd30 0x7fc7880b9230 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fc784009870 tx=0x7fc78402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c2f86377787c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.198+0000 7fc78d2f6640 1 -- 172.21.15.148:0/2645092717 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784037070 con 0x7fc78815dd30 2024-04-25T10:05:26.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.198+0000 7fc78d2f6640 1 -- 172.21.15.148:0/2645092717 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784032440 con 0x7fc78815dd30 2024-04-25T10:05:26.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.198+0000 7fc78d2f6640 1 -- 172.21.15.148:0/2645092717 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784032810 con 0x7fc78815dd30 2024-04-25T10:05:26.214 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.198+0000 7fc79055c640 1 -- 172.21.15.148:0/2645092717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc78815dd30 msgr2=0x7fc7880b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.198+0000 7fc79055c640 1 --2- 172.21.15.148:0/2645092717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc78815dd30 0x7fc7880b9230 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7fc784009870 tx=0x7fc78402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.199+0000 7fc79055c640 1 -- 172.21.15.148:0/2645092717 shutdown_connections 2024-04-25T10:05:26.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.199+0000 7fc79055c640 1 --2- 172.21.15.148:0/2645092717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7880d0ad0 0x7fc7880b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.199+0000 7fc79055c640 1 --2- 172.21.15.148:0/2645092717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc78815dd30 0x7fc7880b9230 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.199+0000 7fc79055c640 1 -- 172.21.15.148:0/2645092717 >> 172.21.15.148:0/2645092717 conn(0x7fc788061c30 msgr2=0x7fc78806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.199+0000 7fc79055c640 1 -- 172.21.15.148:0/2645092717 shutdown_connections 2024-04-25T10:05:26.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.199+0000 7fc79055c640 1 -- 172.21.15.148:0/2645092717 wait complete. 2024-04-25T10:05:26.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 Processor -- start 2024-04-25T10:05:26.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 -- start start 2024-04-25T10:05:26.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7880b9770 0x7fc78815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 0x7fc788156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc78815dd30 0x7fc788154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880d1b90 con 0x7fc7880d0ad0 2024-04-25T10:05:26.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880d1a10 con 0x7fc7880b9770 2024-04-25T10:05:26.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.200+0000 7fc79055c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880d2e90 con 0x7fc78815dd30 2024-04-25T10:05:26.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 0x7fc788156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 0x7fc788156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48372/0 (socket says 172.21.15.148:48372) 2024-04-25T10:05:26.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 -- 172.21.15.148:0/757766022 learned_addr learned my addr 172.21.15.148:0/757766022 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:26.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 -- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc78815dd30 msgr2=0x7fc788154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78eaf9640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc78815dd30 0x7fc788154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78e2f8640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7880b9770 0x7fc78815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc78815dd30 0x7fc788154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 -- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7880b9770 msgr2=0x7fc78815f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7880b9770 0x7fc78815f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 -- 172.21.15.148:0/757766022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78815ff20 con 0x7fc7880d0ad0 2024-04-25T10:05:26.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc78daf7640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 0x7fc788156730 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fc77800c460 tx=0x7fc77800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778017070 con 0x7fc7880d0ad0 2024-04-25T10:05:26.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7780035f0 con 0x7fc7880d0ad0 2024-04-25T10:05:26.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc788160200 con 0x7fc7880d0ad0 2024-04-25T10:05:26.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7780039e0 con 0x7fc7880d0ad0 2024-04-25T10:05:26.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.201+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc788160aa0 con 0x7fc7880d0ad0 2024-04-25T10:05:26.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.202+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc788160ca0 con 0x7fc7880d0ad0 2024-04-25T10:05:26.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.203+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc77801a070 con 0x7fc7880d0ad0 2024-04-25T10:05:26.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.203+0000 7fc77effd640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.203+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc778089270 con 0x7fc7880d0ad0 2024-04-25T10:05:26.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.203+0000 7fc78e2f8640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.203+0000 7fc78e2f8640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc76c06b4d0 0x7fc76c06d990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc784034000 tx=0x7fc78402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.207+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc788160ca0 con 0x7fc7880d0ad0 2024-04-25T10:05:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.379+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc788073990 con 0x7fc7880d0ad0 2024-04-25T10:05:26.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.379+0000 7fc77effd640 1 -- 172.21.15.148:0/757766022 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v167) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc778058340 con 0x7fc7880d0ad0 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:26.385 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:05:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc76c06b4d0 msgr2=0x7fc76c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc76c06b4d0 0x7fc76c06d990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc784034000 tx=0x7fc78402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 msgr2=0x7fc788156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 0x7fc788156730 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fc77800c460 tx=0x7fc77800c920 comp rx=0 tx=0).stop 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 shutdown_connections 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc78815dd30 0x7fc788154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880d0ad0 0x7fc788156730 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 --2- 172.21.15.148:0/757766022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7880b9770 0x7fc78815f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 >> 172.21.15.148:0/757766022 conn(0x7fc788061c30 msgr2=0x7fc7880c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 shutdown_connections 2024-04-25T10:05:26.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.381+0000 7fc79055c640 1 -- 172.21.15.148:0/757766022 wait complete. 2024-04-25T10:05:26.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:26.397 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:26.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.494+0000 7fd02677e640 1 Processor -- start 2024-04-25T10:05:26.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.494+0000 7fd02677e640 1 -- start start 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd02677e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd0200b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd02677e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd02015deb0 0x7fd02015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd02677e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0200d1770 con 0x7fd020070dc0 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd02677e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0201622b0 con 0x7fd02015deb0 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd02677e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200b10f0 con 0x7fd02006ccd0 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd02015deb0 0x7fd02015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd02015deb0 0x7fd02015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48386/0 (socket says 172.21.15.148:48386) 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 -- 172.21.15.148:0/3403295330 learned_addr learned my addr 172.21.15.148:0/3403295330 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd01f7fe640 1 --2- 172.21.15.148:0/3403295330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd0200b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 -- 172.21.15.148:0/3403295330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 msgr2=0x7fd0200b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 --2- 172.21.15.148:0/3403295330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd0200b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 -- 172.21.15.148:0/3403295330 >> v1:172.21.15.188:6789/0 conn(0x7fd020070dc0 legacy=0x7fd0200711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.495+0000 7fd024d1b640 1 -- 172.21.15.148:0/3403295330 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd020162740 con 0x7fd02015deb0 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.496+0000 7fd024d1b640 1 --2- 172.21.15.148:0/3403295330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd02015e340 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e50 tx=0x7fd01802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99940b70a6a88abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.496+0000 7fd01effd640 1 -- 172.21.15.148:0/3403295330 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018036070 con 0x7fd02015deb0 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.496+0000 7fd01effd640 1 -- 172.21.15.148:0/3403295330 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018031600 con 0x7fd02015deb0 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.496+0000 7fd01effd640 1 -- 172.21.15.148:0/3403295330 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180319d0 con 0x7fd02015deb0 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 -- 172.21.15.148:0/3403295330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 msgr2=0x7fd02015e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 --2- 172.21.15.148:0/3403295330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd02015e340 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e50 tx=0x7fd01802a5b0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 -- 172.21.15.148:0/3403295330 shutdown_connections 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 --2- 172.21.15.148:0/3403295330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd02015e340 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 --2- 172.21.15.148:0/3403295330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd0200b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 -- 172.21.15.148:0/3403295330 >> 172.21.15.148:0/3403295330 conn(0x7fd0200b3530 msgr2=0x7fd0200b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 -- 172.21.15.148:0/3403295330 shutdown_connections 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.497+0000 7fd02677e640 1 -- 172.21.15.148:0/3403295330 wait complete. 2024-04-25T10:05:26.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.498+0000 7fd02677e640 1 Processor -- start 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.498+0000 7fd02677e640 1 -- start start 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd02014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd020070dc0 0x7fd0201f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd0200cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd020161f40 con 0x7fd02015deb0 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd020161c60 con 0x7fd020070dc0 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200b5840 con 0x7fd02006ccd0 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd0200cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd0200cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48400/0 (socket says 172.21.15.148:48400) 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 -- 172.21.15.148:0/2276901068 learned_addr learned my addr 172.21.15.148:0/2276901068 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 -- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 msgr2=0x7fd02014e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd02014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 -- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd020070dc0 msgr2=0x7fd0201f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd01f7fe640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd020070dc0 0x7fd0201f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd020070dc0 0x7fd0201f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 -- 172.21.15.148:0/2276901068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02015e360 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd024d1b640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd0200cb580 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e50 tx=0x7fd018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018041020 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018007780 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018036040 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02015e640 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.499+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02015ec00 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.500+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd020073a00 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.501+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd018044020 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.501+0000 7fd01cff9640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd00c06b4d0 0x7fd00c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.501+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd0180aae10 con 0x7fd02015deb0 2024-04-25T10:05:26.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.501+0000 7fd01ffff640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd00c06b4d0 0x7fd00c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.502+0000 7fd01ffff640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd00c06b4d0 0x7fd00c06d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd0201f6c90 tx=0x7fd0100079b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.505+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd01804b030 con 0x7fd02015deb0 2024-04-25T10:05:26.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:26.673+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd02015ba90 con 0x7fd02015deb0 2024-04-25T10:05:27.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.131+0000 7fd01cff9640 1 -- 172.21.15.148:0/2276901068 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v168) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd018079e50 con 0x7fd02015deb0 2024-04-25T10:05:27.137 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd00c06b4d0 msgr2=0x7fd00c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd00c06b4d0 0x7fd00c06d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd0201f6c90 tx=0x7fd0100079b0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 msgr2=0x7fd0200cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd0200cb580 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7fd018009e50 tx=0x7fd018004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 shutdown_connections 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd00c06b4d0 0x7fd00c06d990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd02015deb0 0x7fd0200cb580 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd020070dc0 0x7fd0201f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 --2- 172.21.15.148:0/2276901068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd02006ccd0 0x7fd02014e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 >> 172.21.15.148:0/2276901068 conn(0x7fd0200b3530 msgr2=0x7fd0200ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 shutdown_connections 2024-04-25T10:05:27.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.133+0000 7fd02677e640 1 -- 172.21.15.148:0/2276901068 wait complete. 2024-04-25T10:05:27.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:27.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.253+0000 7fe49af6a640 1 Processor -- start 2024-04-25T10:05:27.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.253+0000 7fe49af6a640 1 -- start start 2024-04-25T10:05:27.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe49af6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe49406c330 0x7fe4940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe49af6a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe49406c7e0 0x7fe4940b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe49af6a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4940d1d50 con 0x7fe494158230 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe49af6a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe494158610 con 0x7fe49406c7e0 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe49af6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49406bea0 con 0x7fe49406c330 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe49406c7e0 0x7fe4940b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe49406c7e0 0x7fe4940b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48414/0 (socket says 172.21.15.148:48414) 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 -- 172.21.15.148:0/2510930595 learned_addr learned my addr 172.21.15.148:0/2510930595 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe493fff640 1 --2- 172.21.15.148:0/2510930595 >> v2:172.21.15.188:3300/0 conn(0x7fe49406c330 0x7fe4940b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 -- 172.21.15.148:0/2510930595 >> v2:172.21.15.188:3300/0 conn(0x7fe49406c330 msgr2=0x7fe4940b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 --2- 172.21.15.148:0/2510930595 >> v2:172.21.15.188:3300/0 conn(0x7fe49406c330 0x7fe4940b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 -- 172.21.15.148:0/2510930595 >> v1:172.21.15.188:6789/0 conn(0x7fe494158230 legacy=0x7fe4940d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.254+0000 7fe499507640 1 -- 172.21.15.148:0/2510930595 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940d2020 con 0x7fe49406c7e0 2024-04-25T10:05:27.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.255+0000 7fe499507640 1 --2- 172.21.15.148:0/2510930595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c7e0 0x7fe4940b53a0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe4940b4800 tx=0x7fe48c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e3de3ca297df34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.255+0000 7fe4937fe640 1 -- 172.21.15.148:0/2510930595 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c036070 con 0x7fe49406c7e0 2024-04-25T10:05:27.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.255+0000 7fe4937fe640 1 -- 172.21.15.148:0/2510930595 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c031600 con 0x7fe49406c7e0 2024-04-25T10:05:27.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.255+0000 7fe4937fe640 1 -- 172.21.15.148:0/2510930595 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0319d0 con 0x7fe49406c7e0 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 -- 172.21.15.148:0/2510930595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c7e0 msgr2=0x7fe4940b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 --2- 172.21.15.148:0/2510930595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c7e0 0x7fe4940b53a0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fe4940b4800 tx=0x7fe48c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 -- 172.21.15.148:0/2510930595 shutdown_connections 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 --2- 172.21.15.148:0/2510930595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c7e0 0x7fe4940b53a0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 --2- 172.21.15.148:0/2510930595 >> v2:172.21.15.188:3300/0 conn(0x7fe49406c330 0x7fe4940b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 -- 172.21.15.148:0/2510930595 >> 172.21.15.148:0/2510930595 conn(0x7fe494061600 msgr2=0x7fe4940619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 -- 172.21.15.148:0/2510930595 shutdown_connections 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.256+0000 7fe49af6a640 1 -- 172.21.15.148:0/2510930595 wait complete. 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.257+0000 7fe49af6a640 1 Processor -- start 2024-04-25T10:05:27.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.257+0000 7fe49af6a640 1 -- start start 2024-04-25T10:05:27.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe49af6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 0x7fe494149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe49af6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe49406c7e0 0x7fe4940cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe49af6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe494158230 0x7fe4940cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe49af6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940d25e0 con 0x7fe49406c330 2024-04-25T10:05:27.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe49af6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49406bb30 con 0x7fe494158230 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe49af6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49415e320 con 0x7fe49406c7e0 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 0x7fe494149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 0x7fe494149520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48424/0 (socket says 172.21.15.148:48424) 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 -- 172.21.15.148:0/1574386506 learned_addr learned my addr 172.21.15.148:0/1574386506 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe499507640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe494158230 0x7fe4940cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe493fff640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe49406c7e0 0x7fe4940cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 -- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe49406c7e0 msgr2=0x7fe4940cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe49406c7e0 0x7fe4940cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 -- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe494158230 msgr2=0x7fe4940cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe494158230 0x7fe4940cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.258+0000 7fe498d06640 1 -- 172.21.15.148:0/1574386506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494149a60 con 0x7fe49406c330 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.259+0000 7fe498d06640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 0x7fe494149520 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c430 tx=0x7fe48800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.259+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488011790 con 0x7fe49406c330 2024-04-25T10:05:27.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.259+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe488017070 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.259+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488011b70 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.259+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4941e4280 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.259+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4940cce90 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.260+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49406de50 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.260+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe488011d10 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.261+0000 7fe4917fa640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe47806b4d0 0x7fe47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.261+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe488089f70 con 0x7fe49406c330 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.261+0000 7fe493fff640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe47806b4d0 0x7fe47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.261+0000 7fe493fff640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe47806b4d0 0x7fe47806d990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe4941f7340 tx=0x7fe480008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.265+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe488058760 con 0x7fe49406c330 2024-04-25T10:05:27.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:27.438+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe49415cda0 con 0x7fe49406c330 2024-04-25T10:05:28.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.139+0000 7fe4917fa640 1 -- 172.21.15.148:0/1574386506 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v169) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe48805c800 con 0x7fe49406c330 2024-04-25T10:05:28.144 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.141+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe47806b4d0 msgr2=0x7fe47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.141+0000 7fe49af6a640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe47806b4d0 0x7fe47806d990 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe4941f7340 tx=0x7fe480008040 comp rx=0 tx=0).stop 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.141+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 msgr2=0x7fe494149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.141+0000 7fe49af6a640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 0x7fe494149520 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c430 tx=0x7fe48800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 shutdown_connections 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe47806b4d0 0x7fe47806d990 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe494158230 0x7fe4940cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe49406c7e0 0x7fe4940cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 --2- 172.21.15.148:0/1574386506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe49406c330 0x7fe494149520 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.147 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 >> 172.21.15.148:0/1574386506 conn(0x7fe494061600 msgr2=0x7fe4940b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 shutdown_connections 2024-04-25T10:05:28.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.142+0000 7fe49af6a640 1 -- 172.21.15.148:0/1574386506 wait complete. 2024-04-25T10:05:28.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:28.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.274+0000 7f3f602f5640 1 Processor -- start 2024-04-25T10:05:28.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.274+0000 7f3f602f5640 1 -- start start 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f602f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f5815bb50 0x7f3f580c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f602f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3f580cfd10 0x7f3f580cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f602f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580c75e0 con 0x7f3f580c70d0 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f602f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f580c8680 con 0x7f3f5815bb50 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f602f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f580c78c0 con 0x7f3f580cfd10 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f5e091640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f5815bb50 0x7f3f580c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f5e091640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3f5815bb50 0x7f3f580c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48426/0 (socket says 172.21.15.148:48426) 2024-04-25T10:05:28.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f5e091640 1 -- 172.21.15.148:0/2500181121 learned_addr learned my addr 172.21.15.148:0/2500181121 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.275+0000 7f3f5d890640 1 --1- 172.21.15.148:0/2500181121 >> v1:172.21.15.148:6789/0 conn(0x7f3f580c70d0 0x7f3f580cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55714/0 (socket says 172.21.15.148:55714) 2024-04-25T10:05:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.276+0000 7f3f5e892640 1 --2- 172.21.15.148:0/2500181121 >> v2:172.21.15.188:3300/0 conn(0x7f3f580cfd10 0x7f3f580cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.276+0000 7f3f5e091640 1 -- 172.21.15.148:0/2500181121 >> v2:172.21.15.188:3300/0 conn(0x7f3f580cfd10 msgr2=0x7f3f580cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.276+0000 7f3f5e091640 1 --2- 172.21.15.148:0/2500181121 >> v2:172.21.15.188:3300/0 conn(0x7f3f580cfd10 0x7f3f580cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.276+0000 7f3f5e091640 1 -- 172.21.15.148:0/2500181121 >> v1:172.21.15.148:6789/0 conn(0x7f3f580c70d0 legacy=0x7f3f580cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.282 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.276+0000 7f3f5e091640 1 -- 172.21.15.148:0/2500181121 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580c7ba0 con 0x7f3f5815bb50 2024-04-25T10:05:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.276+0000 7f3f5e091640 1 --2- 172.21.15.148:0/2500181121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f5815bb50 0x7f3f580c6b90 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f3f58071f40 tx=0x7f3f5402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5a3f176062b5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f5d08f640 1 -- 172.21.15.148:0/2500181121 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54037070 con 0x7f3f5815bb50 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f5d08f640 1 -- 172.21.15.148:0/2500181121 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5402ae50 con 0x7f3f5815bb50 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f5d08f640 1 -- 172.21.15.148:0/2500181121 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54032860 con 0x7f3f5815bb50 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 -- 172.21.15.148:0/2500181121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f5815bb50 msgr2=0x7f3f580c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 --2- 172.21.15.148:0/2500181121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f5815bb50 0x7f3f580c6b90 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f3f58071f40 tx=0x7f3f5402a300 comp rx=0 tx=0).stop 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 -- 172.21.15.148:0/2500181121 shutdown_connections 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 --2- 172.21.15.148:0/2500181121 >> v2:172.21.15.188:3300/0 conn(0x7f3f580cfd10 0x7f3f580cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 --2- 172.21.15.148:0/2500181121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f5815bb50 0x7f3f580c6b90 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 -- 172.21.15.148:0/2500181121 >> 172.21.15.148:0/2500181121 conn(0x7f3f5806f500 msgr2=0x7f3f5806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 -- 172.21.15.148:0/2500181121 shutdown_connections 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.277+0000 7f3f602f5640 1 -- 172.21.15.148:0/2500181121 wait complete. 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.278+0000 7f3f602f5640 1 Processor -- start 2024-04-25T10:05:28.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.278+0000 7f3f602f5640 1 -- start start 2024-04-25T10:05:28.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f602f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 0x7f3f5815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f602f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f580cfd10 0x7f3f5814b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f602f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5815bb50 0x7f3f581590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f602f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5815f6d0 con 0x7f3f580c70d0 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f602f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580c8160 con 0x7f3f5815bb50 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f602f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5815f850 con 0x7f3f580cfd10 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 0x7f3f5815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 0x7f3f5815ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48430/0 (socket says 172.21.15.148:48430) 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 -- 172.21.15.148:0/455971750 learned_addr learned my addr 172.21.15.148:0/455971750 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e892640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5815bb50 0x7f3f581590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5d890640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f580cfd10 0x7f3f5814b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 -- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f580cfd10 msgr2=0x7f3f5814b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f580cfd10 0x7f3f5814b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 -- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5815bb50 msgr2=0x7f3f581590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.279+0000 7f3f5e091640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5815bb50 0x7f3f581590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f5e091640 1 -- 172.21.15.148:0/455971750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58158030 con 0x7f3f580c70d0 2024-04-25T10:05:28.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f5e892640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5815bb50 0x7f3f581590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f5e091640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 0x7f3f5815ab50 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c00 tx=0x7f3f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54041070 con 0x7f3f580c70d0 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5403c670 con 0x7f3f580c70d0 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58158310 con 0x7f3f580c70d0 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54037040 con 0x7f3f580c70d0 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.280+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f581588d0 con 0x7f3f580c70d0 2024-04-25T10:05:28.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.281+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f58073a00 con 0x7f3f580c70d0 2024-04-25T10:05:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.282+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f54044070 con 0x7f3f580c70d0 2024-04-25T10:05:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.282+0000 7f3f4effd640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3c06b4d0 0x7f3f3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.282+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3f540ab0c0 con 0x7f3f580c70d0 2024-04-25T10:05:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.282+0000 7f3f5d890640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3c06b4d0 0x7f3f3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.282+0000 7f3f5d890640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3c06b4d0 0x7f3f3c06d990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3f581f4ee0 tx=0x7f3f48007cc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.286+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f54079ef0 con 0x7f3f580c70d0 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:28.493+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3f580cf150 con 0x7f3f580c70d0 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.140+0000 7f3f4effd640 1 -- 172.21.15.148:0/455971750 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v97) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3f540797f0 con 0x7f3f580c70d0 2024-04-25T10:05:29.146 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 25 and data pool 26 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.143+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3c06b4d0 msgr2=0x7f3f3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.143+0000 7f3f602f5640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3c06b4d0 0x7f3f3c06d990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3f581f4ee0 tx=0x7f3f48007cc0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.143+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 msgr2=0x7f3f5815ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.143+0000 7f3f602f5640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 0x7f3f5815ab50 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c00 tx=0x7f3f540046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.147+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 shutdown_connections 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.147+0000 7f3f602f5640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3f3c06b4d0 0x7f3f3c06d990 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.147+0000 7f3f602f5640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f5815bb50 0x7f3f581590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.147+0000 7f3f602f5640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f580cfd10 0x7f3f5814b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.147+0000 7f3f602f5640 1 --2- 172.21.15.148:0/455971750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f580c70d0 0x7f3f5815ab50 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.147+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 >> 172.21.15.148:0/455971750 conn(0x7f3f5806f500 msgr2=0x7f3f580b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.149+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 shutdown_connections 2024-04-25T10:05:29.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.150+0000 7f3f602f5640 1 -- 172.21.15.148:0/455971750 wait complete. 2024-04-25T10:05:29.183 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:29.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.276+0000 7fccd3c7d640 1 Processor -- start 2024-04-25T10:05:29.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.276+0000 7fccd3c7d640 1 -- start start 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd3c7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcccc0d1f20 0x7fcccc1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd3c7d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc0cce50 con 0x7fcccc0cc8e0 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd3c7d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc161af0 con 0x7fcccc15dc90 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd3c7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcccc161dd0 con 0x7fcccc0d1f20 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd1218640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fcccc0cc8e0 0x7fcccc0d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55730/0 (socket says 172.21.15.148:55730) 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd1218640 1 -- 172.21.15.148:0/2470217598 learned_addr learned my addr 172.21.15.148:0/2470217598 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd221a640 1 --2- 172.21.15.148:0/2470217598 >> v2:172.21.15.188:3300/0 conn(0x7fcccc0d1f20 0x7fcccc1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306271724 0 0) 0x7fcccc0cce50 con 0x7fcccc0cc8e0 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb8003520 con 0x7fcccc0cc8e0 2024-04-25T10:05:29.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404698271 0 0) 0x7fccb8003520 con 0x7fcccc0cc8e0 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 >> v2:172.21.15.188:3300/0 conn(0x7fcccc0d1f20 msgr2=0x7fcccc1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 --2- 172.21.15.148:0/2470217598 >> v2:172.21.15.188:3300/0 conn(0x7fcccc0d1f20 0x7fcccc1625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 >> v1:172.21.15.188:6789/0 conn(0x7fcccc15dc90 legacy=0x7fcccc15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc1620b0 con 0x7fcccc0cc8e0 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fccbc001670 con 0x7fcccc0cc8e0 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 >> v1:172.21.15.148:6789/0 conn(0x7fcccc0cc8e0 legacy=0x7fcccc0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb80039b0 0x7fccb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.277+0000 7fccd0a17640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccb8004800 0x7fccb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd0a17640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80051c0 0x7fccb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc161dd0 con 0x7fccb80039b0 2024-04-25T10:05:29.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb800c160 con 0x7fccb8004800 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb800c360 con 0x7fccb80051c0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1a19640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb80039b0 0x7fccb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1218640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccb8004800 0x7fccb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd221a640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80051c0 0x7fccb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1a19640 1 -- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80051c0 msgr2=0x7fccb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1a19640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80051c0 0x7fccb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1a19640 1 -- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccb8004800 msgr2=0x7fccb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1a19640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fccb8004800 0x7fccb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd1a19640 1 -- 172.21.15.148:0/2470217598 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8003520 con 0x7fccb80039b0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.278+0000 7fccd221a640 1 -- 172.21.15.148:0/2470217598 reap_dead start 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd1a19640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb80039b0 0x7fccb8003dd0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fccc8001750 tx=0x7fccb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f7665efce6f024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8001370 con 0x7fccb80039b0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc80052d0 con 0x7fccb80039b0 2024-04-25T10:05:29.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd0a17640 1 -- 172.21.15.148:0/2470217598 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc802abe0 con 0x7fccb80039b0 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 -- 172.21.15.148:0/2470217598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb80039b0 msgr2=0x7fccb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb80039b0 0x7fccb8003dd0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fccc8001750 tx=0x7fccb8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 -- 172.21.15.148:0/2470217598 shutdown_connections 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/2470217598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb80039b0 0x7fccb8003dd0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 -- 172.21.15.148:0/2470217598 >> 172.21.15.148:0/2470217598 conn(0x7fcccc06f590 msgr2=0x7fcccc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 -- 172.21.15.148:0/2470217598 shutdown_connections 2024-04-25T10:05:29.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.279+0000 7fccd3c7d640 1 -- 172.21.15.148:0/2470217598 wait complete. 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.280+0000 7fccd3c7d640 1 Processor -- start 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 -- start start 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80039b0 0x7fcccc1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 0x7fcccc1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcccc0b65d0 0x7fcccc0b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc15e1f0 con 0x7fcccc156890 2024-04-25T10:05:29.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc15e070 con 0x7fcccc0b65d0 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd3c7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc1625d0 con 0x7fccb80039b0 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd1218640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 0x7fcccc1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd1218640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 0x7fcccc1556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48450/0 (socket says 172.21.15.148:48450) 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd1218640 1 -- 172.21.15.148:0/1587836124 learned_addr learned my addr 172.21.15.148:0/1587836124 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd1a19640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80039b0 0x7fcccc1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.281+0000 7fccd221a640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcccc0b65d0 0x7fcccc0b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1218640 1 -- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80039b0 msgr2=0x7fcccc1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1218640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80039b0 0x7fcccc1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1218640 1 -- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcccc0b65d0 msgr2=0x7fcccc0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1218640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcccc0b65d0 0x7fcccc0b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1218640 1 -- 172.21.15.148:0/1587836124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc157070 con 0x7fcccc156890 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1a19640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80039b0 0x7fcccc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:29.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd221a640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcccc0b65d0 0x7fcccc0b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd1218640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 0x7fcccc1556a0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fccbc007370 tx=0x7fccbc007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00f940 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc018070 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc157350 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00fd20 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.282+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc0c5390 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.283+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc06b040 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.283+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccbc01d460 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.284+0000 7fccc27fc640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fccb006b4d0 0x7fccb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.284+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccbc08a3b0 con 0x7fcccc156890 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.284+0000 7fccd1a19640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fccb006b4d0 0x7fccb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.284+0000 7fccd1a19640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fccb006b4d0 0x7fccb006d990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fccc8032000 tx=0x7fccc802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.288+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcccc06b040 con 0x7fcccc156890 2024-04-25T10:05:29.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.457+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcccc0cf6b0 con 0x7fcccc156890 2024-04-25T10:05:29.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.458+0000 7fccc27fc640 1 -- 172.21.15.148:0/1587836124 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16949 (secure 0 0 0) 0x7fccbc059130 con 0x7fcccc156890 2024-04-25T10:05:29.464 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:29.464 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":170,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:05:28.494260+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:29.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fccb006b4d0 msgr2=0x7fccb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fccb006b4d0 0x7fccb006d990 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fccc8032000 tx=0x7fccc802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:29.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 msgr2=0x7fcccc1556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 0x7fcccc1556a0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fccbc007370 tx=0x7fccbc007830 comp rx=0 tx=0).stop 2024-04-25T10:05:29.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 shutdown_connections 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fccb006b4d0 0x7fccb006d990 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcccc0b65d0 0x7fcccc0b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc156890 0x7fcccc1556a0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 --2- 172.21.15.148:0/1587836124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fccb80039b0 0x7fcccc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.460+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 >> 172.21.15.148:0/1587836124 conn(0x7fcccc06f590 msgr2=0x7fcccc0b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.461+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 shutdown_connections 2024-04-25T10:05:29.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.461+0000 7fccd3c7d640 1 -- 172.21.15.148:0/1587836124 wait complete. 2024-04-25T10:05:29.477 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:29.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.570+0000 7fe6f5334640 1 Processor -- start 2024-04-25T10:05:29.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.570+0000 7fe6f5334640 1 -- start start 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6f5334640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f015dbe0 0x7fe6f00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6f5334640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6f00b2cb0 0x7fe6f00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6f5334640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe6f00b92a0 0x7fe6f00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6f5334640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f015dfc0 con 0x7fe6f00b92a0 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6f5334640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f00d1a00 con 0x7fe6f00b2cb0 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6f5334640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00b97a0 con 0x7fe6f015dbe0 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe6f00b92a0 0x7fe6f00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe6f00b92a0 0x7fe6f00b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48462/0 (socket says 172.21.15.148:48462) 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/3213102982 learned_addr learned my addr 172.21.15.148:0/3213102982 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ee7fc640 1 --2- 172.21.15.148:0/3213102982 >> v2:172.21.15.188:3300/0 conn(0x7fe6f00b2cb0 0x7fe6f00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6eeffd640 1 --2- 172.21.15.148:0/3213102982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f015dbe0 0x7fe6f00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/3213102982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f015dbe0 msgr2=0x7fe6f00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 --2- 172.21.15.148:0/3213102982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f015dbe0 0x7fe6f00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/3213102982 >> v2:172.21.15.188:3300/0 conn(0x7fe6f00b2cb0 msgr2=0x7fe6f00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 --2- 172.21.15.148:0/3213102982 >> v2:172.21.15.188:3300/0 conn(0x7fe6f00b2cb0 0x7fe6f00d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/3213102982 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f00d1c50 con 0x7fe6f00b92a0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.571+0000 7fe6ee7fc640 1 --2- 172.21.15.148:0/3213102982 >> v2:172.21.15.188:3300/0 conn(0x7fe6f00b2cb0 0x7fe6f00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.572+0000 7fe6ef7fe640 1 --2- 172.21.15.148:0/3213102982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f00b92a0 0x7fe6f00b5400 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00b64d0 tx=0x7fe6e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5be3e7e4315ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.572+0000 7fe6edffb640 1 -- 172.21.15.148:0/3213102982 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8037070 con 0x7fe6f00b92a0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.572+0000 7fe6edffb640 1 -- 172.21.15.148:0/3213102982 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8032440 con 0x7fe6f00b92a0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.572+0000 7fe6edffb640 1 -- 172.21.15.148:0/3213102982 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8032810 con 0x7fe6f00b92a0 2024-04-25T10:05:29.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 -- 172.21.15.148:0/3213102982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f00b92a0 msgr2=0x7fe6f00b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 --2- 172.21.15.148:0/3213102982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f00b92a0 0x7fe6f00b5400 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00b64d0 tx=0x7fe6e802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 -- 172.21.15.148:0/3213102982 shutdown_connections 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 --2- 172.21.15.148:0/3213102982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f00b92a0 0x7fe6f00b5400 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 --2- 172.21.15.148:0/3213102982 >> v2:172.21.15.188:3300/0 conn(0x7fe6f00b2cb0 0x7fe6f00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 --2- 172.21.15.148:0/3213102982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f015dbe0 0x7fe6f00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 -- 172.21.15.148:0/3213102982 >> 172.21.15.148:0/3213102982 conn(0x7fe6f0061a20 msgr2=0x7fe6f0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 -- 172.21.15.148:0/3213102982 shutdown_connections 2024-04-25T10:05:29.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.573+0000 7fe6f5334640 1 -- 172.21.15.148:0/3213102982 wait complete. 2024-04-25T10:05:29.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.574+0000 7fe6f5334640 1 Processor -- start 2024-04-25T10:05:29.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.574+0000 7fe6f5334640 1 -- start start 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6f5334640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6f00b2cb0 0x7fe6f00b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6f5334640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f00b92a0 0x7fe6f015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6f5334640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 0x7fe6f015c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6f5334640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00b6ab0 con 0x7fe6f015dbe0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6f5334640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00d2210 con 0x7fe6f00b2cb0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6f5334640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00b6c30 con 0x7fe6f00b92a0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 0x7fe6f015c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 0x7fe6f015c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48474/0 (socket says 172.21.15.148:48474) 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/2242701921 learned_addr learned my addr 172.21.15.148:0/2242701921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6eeffd640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6f00b2cb0 0x7fe6f00b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ee7fc640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f00b92a0 0x7fe6f015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f00b92a0 msgr2=0x7fe6f015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f00b92a0 0x7fe6f015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6f00b2cb0 msgr2=0x7fe6f00b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6f00b2cb0 0x7fe6f00b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 -- 172.21.15.148:0/2242701921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f01600c0 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.575+0000 7fe6ef7fe640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 0x7fe6f015c1d0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe6e802a220 tx=0x7fe6e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.576+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8037040 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.576+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8042070 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.576+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f00cc990 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.576+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8030920 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.576+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f00ccf50 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.577+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f0073990 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.577+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6e8030ac0 con 0x7fe6f015dbe0 2024-04-25T10:05:29.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.578+0000 7fe6d77fe640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe6dc06b4d0 0x7fe6dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.578+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6e80aaae0 con 0x7fe6f015dbe0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.578+0000 7fe6eeffd640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe6dc06b4d0 0x7fe6dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.578+0000 7fe6eeffd640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe6dc06b4d0 0x7fe6dc06d990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00039e0 tx=0x7fe6e0009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.582+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6e8079180 con 0x7fe6f015dbe0 2024-04-25T10:05:29.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.751+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6f0159ca0 con 0x7fe6f015dbe0 2024-04-25T10:05:29.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.752+0000 7fe6d77fe640 1 -- 172.21.15.148:0/2242701921 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16949 (secure 0 0 0) 0x7fe6e807d220 con 0x7fe6f015dbe0 2024-04-25T10:05:29.757 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:29.758 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":170,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:05:28.494260+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:29.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe6dc06b4d0 msgr2=0x7fe6dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe6dc06b4d0 0x7fe6dc06d990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00039e0 tx=0x7fe6e0009830 comp rx=0 tx=0).stop 2024-04-25T10:05:29.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 msgr2=0x7fe6f015c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 0x7fe6f015c1d0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe6e802a220 tx=0x7fe6e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 shutdown_connections 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe6dc06b4d0 0x7fe6dc06d990 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6f015dbe0 0x7fe6f015c1d0 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f00b92a0 0x7fe6f015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 --2- 172.21.15.148:0/2242701921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6f00b2cb0 0x7fe6f00b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 >> 172.21.15.148:0/2242701921 conn(0x7fe6f0061a20 msgr2=0x7fe6f00b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 shutdown_connections 2024-04-25T10:05:29.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.754+0000 7fe6f5334640 1 -- 172.21.15.148:0/2242701921 wait complete. 2024-04-25T10:05:29.771 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:29.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.867+0000 7f87e23b0640 1 Processor -- start 2024-04-25T10:05:29.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.867+0000 7f87e23b0640 1 -- start start 2024-04-25T10:05:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.867+0000 7f87e23b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87dc1446d0 0x7f87dc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.867+0000 7f87e23b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f87dc06e8a0 0x7f87dc0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.867+0000 7f87e23b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc0cccc0 con 0x7f87dc144d50 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87e23b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0b2b20 con 0x7f87dc1446d0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87e23b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc06d400 con 0x7f87dc06e8a0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87dc1446d0 0x7f87dc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f87dc1446d0 0x7f87dc0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48482/0 (socket says 172.21.15.148:48482) 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 -- 172.21.15.148:0/3417064734 learned_addr learned my addr 172.21.15.148:0/3417064734 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87e094d640 1 --2- 172.21.15.148:0/3417064734 >> v2:172.21.15.188:3300/0 conn(0x7f87dc06e8a0 0x7f87dc0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 -- 172.21.15.148:0/3417064734 >> v2:172.21.15.188:3300/0 conn(0x7f87dc06e8a0 msgr2=0x7f87dc0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 --2- 172.21.15.148:0/3417064734 >> v2:172.21.15.188:3300/0 conn(0x7f87dc06e8a0 0x7f87dc0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 -- 172.21.15.148:0/3417064734 >> v1:172.21.15.148:6789/0 conn(0x7f87dc144d50 legacy=0x7f87dc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.868+0000 7f87dbfff640 1 -- 172.21.15.148:0/3417064734 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0cc140 con 0x7f87dc1446d0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87dbfff640 1 --2- 172.21.15.148:0/3417064734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc1446d0 0x7f87dc0b3f20 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f87dc071200 tx=0x7f87cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2035701aaa8333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87daffd640 1 -- 172.21.15.148:0/3417064734 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc037070 con 0x7f87dc1446d0 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87daffd640 1 -- 172.21.15.148:0/3417064734 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc032440 con 0x7f87dc1446d0 2024-04-25T10:05:29.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87daffd640 1 -- 172.21.15.148:0/3417064734 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc032810 con 0x7f87dc1446d0 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 -- 172.21.15.148:0/3417064734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc1446d0 msgr2=0x7f87dc0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3417064734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc1446d0 0x7f87dc0b3f20 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f87dc071200 tx=0x7f87cc02a270 comp rx=0 tx=0).stop 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 -- 172.21.15.148:0/3417064734 shutdown_connections 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3417064734 >> v2:172.21.15.188:3300/0 conn(0x7f87dc06e8a0 0x7f87dc0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3417064734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc1446d0 0x7f87dc0b3f20 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 -- 172.21.15.148:0/3417064734 >> 172.21.15.148:0/3417064734 conn(0x7f87dc06abe0 msgr2=0x7f87dc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 -- 172.21.15.148:0/3417064734 shutdown_connections 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.869+0000 7f87e23b0640 1 -- 172.21.15.148:0/3417064734 wait complete. 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.870+0000 7f87e23b0640 1 Processor -- start 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 -- start start 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 0x7f87dc0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc1446d0 0x7f87dc0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc144d50 0x7f87dc0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0cc700 con 0x7f87dc06e8a0 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0b49a0 con 0x7f87dc144d50 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87e23b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0cc9e0 con 0x7f87dc1446d0 2024-04-25T10:05:29.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.871+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 0x7f87dc0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 0x7f87dc0cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48492/0 (socket says 172.21.15.148:48492) 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 -- 172.21.15.148:0/3783272584 learned_addr learned my addr 172.21.15.148:0/3783272584 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 -- 172.21.15.148:0/3783272584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc1446d0 msgr2=0x7f87dc0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc1446d0 0x7f87dc0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 -- 172.21.15.148:0/3783272584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc144d50 msgr2=0x7f87dc0ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc144d50 0x7f87dc0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 -- 172.21.15.148:0/3783272584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0cd8d0 con 0x7f87dc06e8a0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87dbfff640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 0x7f87dc0cd2e0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f87cc000c00 tx=0x7f87cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc041020 con 0x7f87dc06e8a0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0306c0 con 0x7f87dc06e8a0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc037040 con 0x7f87dc06e8a0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0cdbb0 con 0x7f87dc06e8a0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.872+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0ce170 con 0x7f87dc06e8a0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.873+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc073a00 con 0x7f87dc06e8a0 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.873+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87cc044020 con 0x7f87dc06e8a0 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.874+0000 7f87d8ff9640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.874+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87cc0aab50 con 0x7f87dc06e8a0 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.874+0000 7f87db7fe640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.874+0000 7f87db7fe640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f87c806b4d0 0x7f87c806d990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f87d0000f60 tx=0x7f87d00079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:29.878+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87cc04b030 con 0x7f87dc06e8a0 2024-04-25T10:05:30.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.082+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f87dc141ca0 con 0x7f87dc06e8a0 2024-04-25T10:05:30.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.155+0000 7f87d8ff9640 1 -- 172.21.15.148:0/3783272584 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f87cc0798d0 con 0x7f87dc06e8a0 2024-04-25T10:05:30.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f87c806b4d0 msgr2=0x7f87c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f87c806b4d0 0x7f87c806d990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f87d0000f60 tx=0x7f87d00079d0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 msgr2=0x7f87dc0cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 0x7f87dc0cd2e0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f87cc000c00 tx=0x7f87cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 shutdown_connections 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f87c806b4d0 0x7f87c806d990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87dc144d50 0x7f87dc0ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87dc1446d0 0x7f87dc0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 --2- 172.21.15.148:0/3783272584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87dc06e8a0 0x7f87dc0cd2e0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 >> 172.21.15.148:0/3783272584 conn(0x7f87dc06abe0 msgr2=0x7f87dc071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 shutdown_connections 2024-04-25T10:05:30.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.158+0000 7f87e23b0640 1 -- 172.21.15.148:0/3783272584 wait complete. 2024-04-25T10:05:30.174 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:30.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.269+0000 7f9ba04f5640 1 Processor -- start 2024-04-25T10:05:30.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.269+0000 7f9ba04f5640 1 -- start start 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9ba04f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b98161240 0x7f9b980b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9ba04f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9b98161890 0x7f9b980ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9ba04f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b98161620 con 0x7f9b980b54a0 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9ba04f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b980b7120 con 0x7f9b98161240 2024-04-25T10:05:30.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9ba04f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b98161e00 con 0x7f9b98161890 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9b9e291640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b98161240 0x7f9b980b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.270+0000 7f9b9e291640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9b98161240 0x7f9b980b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48506/0 (socket says 172.21.15.148:48506) 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.271+0000 7f9b9e291640 1 -- 172.21.15.148:0/338204580 learned_addr learned my addr 172.21.15.148:0/338204580 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.271+0000 7f9b9da90640 1 --2- 172.21.15.148:0/338204580 >> v2:172.21.15.188:3300/0 conn(0x7f9b98161890 0x7f9b980ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.271+0000 7f9b9e291640 1 -- 172.21.15.148:0/338204580 >> v2:172.21.15.188:3300/0 conn(0x7f9b98161890 msgr2=0x7f9b980ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.271+0000 7f9b9e291640 1 --2- 172.21.15.148:0/338204580 >> v2:172.21.15.188:3300/0 conn(0x7f9b98161890 0x7f9b980ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.271+0000 7f9b9e291640 1 -- 172.21.15.148:0/338204580 >> v1:172.21.15.148:6789/0 conn(0x7f9b980b54a0 legacy=0x7f9b980d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.271+0000 7f9b9e291640 1 -- 172.21.15.148:0/338204580 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b981620e0 con 0x7f9b98161240 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9b9e291640 1 --2- 172.21.15.148:0/338204580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b98161240 0x7f9b980b6be0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f9b9808c330 tx=0x7f9b9402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeda20b229f86c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9b9d28f640 1 -- 172.21.15.148:0/338204580 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94036070 con 0x7f9b98161240 2024-04-25T10:05:30.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9b9d28f640 1 -- 172.21.15.148:0/338204580 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b94031600 con 0x7f9b98161240 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9b9d28f640 1 -- 172.21.15.148:0/338204580 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940319d0 con 0x7f9b98161240 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 -- 172.21.15.148:0/338204580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b98161240 msgr2=0x7f9b980b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/338204580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b98161240 0x7f9b980b6be0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f9b9808c330 tx=0x7f9b9402a730 comp rx=0 tx=0).stop 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 -- 172.21.15.148:0/338204580 shutdown_connections 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/338204580 >> v2:172.21.15.188:3300/0 conn(0x7f9b98161890 0x7f9b980ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/338204580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b98161240 0x7f9b980b6be0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 -- 172.21.15.148:0/338204580 >> 172.21.15.148:0/338204580 conn(0x7f9b980d04a0 msgr2=0x7f9b980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 -- 172.21.15.148:0/338204580 shutdown_connections 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.272+0000 7f9ba04f5640 1 -- 172.21.15.148:0/338204580 wait complete. 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.273+0000 7f9ba04f5640 1 Processor -- start 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 -- start start 2024-04-25T10:05:30.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 0x7f9b981e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b98161240 0x7f9b98151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b98161890 0x7f9b980ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980d15f0 con 0x7f9b980b54a0 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981626a0 con 0x7f9b98161240 2024-04-25T10:05:30.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9ba04f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980cf000 con 0x7f9b98161890 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9b9e291640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 0x7f9b981e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9b9e291640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 0x7f9b981e8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48510/0 (socket says 172.21.15.148:48510) 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9b9e291640 1 -- 172.21.15.148:0/1896508464 learned_addr learned my addr 172.21.15.148:0/1896508464 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.274+0000 7f9b9da90640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b98161240 0x7f9b98151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9ea92640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b98161890 0x7f9b980ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9e291640 1 -- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b98161890 msgr2=0x7f9b980ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9e291640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b98161890 0x7f9b980ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9e291640 1 -- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b98161240 msgr2=0x7f9b98151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9e291640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b98161240 0x7f9b98151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9e291640 1 -- 172.21.15.148:0/1896508464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980cd1b0 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9da90640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b98161240 0x7f9b98151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9ea92640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b98161890 0x7f9b980ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b9e291640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 0x7f9b981e8050 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f9b94000f80 tx=0x7f9b94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94036050 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b94042070 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b981e8660 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94009510 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.275+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b981e8c20 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.276+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b98073a00 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.276+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b94007c70 con 0x7f9b980b54a0 2024-04-25T10:05:30.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.277+0000 7f9b8effd640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.277+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b940ab210 con 0x7f9b980b54a0 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.277+0000 7f9b9da90640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.277+0000 7f9b9da90640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9b981e8590 tx=0x7f9b88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.281+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b9407a020 con 0x7f9b980b54a0 2024-04-25T10:05:30.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:30.486+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9b9806d240 con 0x7f9b980b54a0 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.164+0000 7f9b8effd640 1 -- 172.21.15.148:0/1896508464 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9b94079920 con 0x7f9b980b54a0 2024-04-25T10:05:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b7c06b4d0 msgr2=0x7f9b7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9b981e8590 tx=0x7f9b88008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 msgr2=0x7f9b981e8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 0x7f9b981e8050 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f9b94000f80 tx=0x7f9b94004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 shutdown_connections 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9b7c06b4d0 0x7f9b7c06d990 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b98161890 0x7f9b980ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b98161240 0x7f9b98151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 --2- 172.21.15.148:0/1896508464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b980b54a0 0x7f9b981e8050 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 >> 172.21.15.148:0/1896508464 conn(0x7f9b980d04a0 msgr2=0x7f9b980b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 shutdown_connections 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.166+0000 7f9ba04f5640 1 -- 172.21.15.148:0/1896508464 wait complete. 2024-04-25T10:05:31.183 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:31.183 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.276+0000 7f03f5838640 1 Processor -- start 2024-04-25T10:05:31.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.276+0000 7f03f5838640 1 -- start start 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03f5838640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03f015dbe0 0x7f03f00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03f5838640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b2cb0 0x7f03f00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03f5838640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f03f00b92a0 0x7f03f00b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03f5838640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f015dfc0 con 0x7f03f015dbe0 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03f5838640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f00d1a00 con 0x7f03f00b92a0 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03f5838640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00b97a0 con 0x7f03f00b2cb0 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03f015dbe0 0x7f03f00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03f015dbe0 0x7f03f00d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48516/0 (socket says 172.21.15.148:48516) 2024-04-25T10:05:31.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 -- 172.21.15.148:0/3436746514 learned_addr learned my addr 172.21.15.148:0/3436746514 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 -- 172.21.15.148:0/3436746514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b2cb0 msgr2=0x7f03f00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 --2- 172.21.15.148:0/3436746514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b2cb0 0x7f03f00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03ef7fe640 1 --2- 172.21.15.148:0/3436746514 >> v2:172.21.15.188:3300/0 conn(0x7f03f00b92a0 0x7f03f00b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 -- 172.21.15.148:0/3436746514 >> v2:172.21.15.188:3300/0 conn(0x7f03f00b92a0 msgr2=0x7f03f00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 --2- 172.21.15.148:0/3436746514 >> v2:172.21.15.188:3300/0 conn(0x7f03f00b92a0 0x7f03f00b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.277+0000 7f03eeffd640 1 -- 172.21.15.148:0/3436746514 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00d1c50 con 0x7f03f015dbe0 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.278+0000 7f03eeffd640 1 --2- 172.21.15.148:0/3436746514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00d0910 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f03f006c640 tx=0x7f03e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c7fc7d440596e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.278+0000 7f03edffb640 1 -- 172.21.15.148:0/3436746514 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0037070 con 0x7f03f015dbe0 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.278+0000 7f03edffb640 1 -- 172.21.15.148:0/3436746514 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0032440 con 0x7f03f015dbe0 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.278+0000 7f03edffb640 1 -- 172.21.15.148:0/3436746514 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032810 con 0x7f03f015dbe0 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 -- 172.21.15.148:0/3436746514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 msgr2=0x7f03f00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 --2- 172.21.15.148:0/3436746514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00d0910 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f03f006c640 tx=0x7f03e002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 -- 172.21.15.148:0/3436746514 shutdown_connections 2024-04-25T10:05:31.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 --2- 172.21.15.148:0/3436746514 >> v2:172.21.15.188:3300/0 conn(0x7f03f00b92a0 0x7f03f00b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 --2- 172.21.15.148:0/3436746514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b2cb0 0x7f03f00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 --2- 172.21.15.148:0/3436746514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00d0910 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 -- 172.21.15.148:0/3436746514 >> 172.21.15.148:0/3436746514 conn(0x7f03f0061a20 msgr2=0x7f03f0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 -- 172.21.15.148:0/3436746514 shutdown_connections 2024-04-25T10:05:31.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.279+0000 7f03f5838640 1 -- 172.21.15.148:0/3436746514 wait complete. 2024-04-25T10:05:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.280+0000 7f03f5838640 1 Processor -- start 2024-04-25T10:05:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.280+0000 7f03f5838640 1 -- start start 2024-04-25T10:05:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03f5838640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03f00b2cb0 0x7f03f00b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03f5838640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b92a0 0x7f03f015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03f5838640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03f5838640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00b6ab0 con 0x7f03f015dbe0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03f5838640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00d2210 con 0x7f03f00b2cb0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03f5838640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00b6c30 con 0x7f03f00b92a0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48532/0 (socket says 172.21.15.148:48532) 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 -- 172.21.15.148:0/2080449291 learned_addr learned my addr 172.21.15.148:0/2080449291 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ee7fc640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b92a0 0x7f03f015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03eeffd640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03f00b2cb0 0x7f03f00b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 -- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b92a0 msgr2=0x7f03f015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b92a0 0x7f03f015f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 -- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03f00b2cb0 msgr2=0x7f03f00b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03f00b2cb0 0x7f03f00b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 -- 172.21.15.148:0/2080449291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00b3530 con 0x7f03f015dbe0 2024-04-25T10:05:31.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03ef7fe640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00b5a90 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f03e800cd90 tx=0x7f03e800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8003b90 con 0x7f03f015dbe0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.281+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8003d50 con 0x7f03f015dbe0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.282+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f00b3830 con 0x7f03f015dbe0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.282+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8004260 con 0x7f03f015dbe0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.282+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f00b4070 con 0x7f03f015dbe0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.283+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0073990 con 0x7f03f015dbe0 2024-04-25T10:05:31.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.283+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03e8004400 con 0x7f03f015dbe0 2024-04-25T10:05:31.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.283+0000 7f03d7fff640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03dc06b4d0 0x7f03dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.284+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03e808ac40 con 0x7f03f015dbe0 2024-04-25T10:05:31.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.284+0000 7f03eeffd640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03dc06b4d0 0x7f03dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.284+0000 7f03eeffd640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03dc06b4d0 0x7f03dc06d990 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f03e0034000 tx=0x7f03e002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.288+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03f00b3830 con 0x7f03f015dbe0 2024-04-25T10:05:31.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.497+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03f00b4240 con 0x7f03f015dbe0 2024-04-25T10:05:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.497+0000 7f03d7fff640 1 -- 172.21.15.148:0/2080449291 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f03e80599c0 con 0x7f03f015dbe0 2024-04-25T10:05:31.503 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:31.503 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:05:31.503 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03dc06b4d0 msgr2=0x7f03dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03dc06b4d0 0x7f03dc06d990 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f03e0034000 tx=0x7f03e002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 msgr2=0x7f03f00b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00b5a90 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f03e800cd90 tx=0x7f03e800a660 comp rx=0 tx=0).stop 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 shutdown_connections 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f03dc06b4d0 0x7f03dc06d990 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03f015dbe0 0x7f03f00b5a90 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03f00b92a0 0x7f03f015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 --2- 172.21.15.148:0/2080449291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03f00b2cb0 0x7f03f00b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 >> 172.21.15.148:0/2080449291 conn(0x7f03f0061a20 msgr2=0x7f03f00b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 shutdown_connections 2024-04-25T10:05:31.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.499+0000 7f03f5838640 1 -- 172.21.15.148:0/2080449291 wait complete. 2024-04-25T10:05:31.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:31.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.608+0000 7feb45683640 1 Processor -- start 2024-04-25T10:05:31.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.608+0000 7feb45683640 1 -- start start 2024-04-25T10:05:31.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb45683640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4015dd30 0x7feb400b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb45683640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb400b9770 0x7feb400d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb45683640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb400d2e90 0x7feb400d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb45683640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb400d1df0 con 0x7feb400d2e90 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb45683640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb400d20d0 con 0x7feb400b9770 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb45683640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400d23b0 con 0x7feb4015dd30 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb400d2e90 0x7feb400d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb400d2e90 0x7feb400d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48536/0 (socket says 172.21.15.148:48536) 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 -- 172.21.15.148:0/1641734415 learned_addr learned my addr 172.21.15.148:0/1641734415 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3e7fc640 1 --2- 172.21.15.148:0/1641734415 >> v2:172.21.15.188:3300/0 conn(0x7feb400b9770 0x7feb400d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3effd640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4015dd30 0x7feb400b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 -- 172.21.15.148:0/1641734415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4015dd30 msgr2=0x7feb400b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4015dd30 0x7feb400b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 -- 172.21.15.148:0/1641734415 >> v2:172.21.15.188:3300/0 conn(0x7feb400b9770 msgr2=0x7feb400d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 --2- 172.21.15.148:0/1641734415 >> v2:172.21.15.188:3300/0 conn(0x7feb400b9770 0x7feb400d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3f7fe640 1 -- 172.21.15.148:0/1641734415 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400d2690 con 0x7feb400d2e90 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.609+0000 7feb3effd640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4015dd30 0x7feb400b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.610+0000 7feb3f7fe640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb400d0900 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7feb400d49c0 tx=0x7feb3802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362e0c66b9a905a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.610+0000 7feb3dffb640 1 -- 172.21.15.148:0/1641734415 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38036070 con 0x7feb400d2e90 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.610+0000 7feb3dffb640 1 -- 172.21.15.148:0/1641734415 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38031600 con 0x7feb400d2e90 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.610+0000 7feb3dffb640 1 -- 172.21.15.148:0/1641734415 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb380319d0 con 0x7feb400d2e90 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 -- 172.21.15.148:0/1641734415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 msgr2=0x7feb400d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb400d0900 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7feb400d49c0 tx=0x7feb3802a610 comp rx=0 tx=0).stop 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 -- 172.21.15.148:0/1641734415 shutdown_connections 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb400d0900 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 --2- 172.21.15.148:0/1641734415 >> v2:172.21.15.188:3300/0 conn(0x7feb400b9770 0x7feb400d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 --2- 172.21.15.148:0/1641734415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4015dd30 0x7feb400b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 -- 172.21.15.148:0/1641734415 >> 172.21.15.148:0/1641734415 conn(0x7feb40061c30 msgr2=0x7feb4006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 -- 172.21.15.148:0/1641734415 shutdown_connections 2024-04-25T10:05:31.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.611+0000 7feb45683640 1 -- 172.21.15.148:0/1641734415 wait complete. 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.612+0000 7feb45683640 1 Processor -- start 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.612+0000 7feb45683640 1 -- start start 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb400b9770 0x7feb4014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb4014bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4015dd30 0x7feb40160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400d1b90 con 0x7feb400d2e90 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb400d1a10 con 0x7feb4015dd30 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb400b5430 con 0x7feb400b9770 2024-04-25T10:05:31.624 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb4014bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb4014bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48538/0 (socket says 172.21.15.148:48538) 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 -- 172.21.15.148:0/2203908537 learned_addr learned my addr 172.21.15.148:0/2203908537 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3f7fe640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4015dd30 0x7feb40160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3effd640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb400b9770 0x7feb4014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 -- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb400b9770 msgr2=0x7feb4014e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb400b9770 0x7feb4014e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 -- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4015dd30 msgr2=0x7feb40160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4015dd30 0x7feb40160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 -- 172.21.15.148:0/2203908537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4015e110 con 0x7feb400d2e90 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3effd640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb400b9770 0x7feb4014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb3e7fc640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb4014bca0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7feb3400d830 tx=0x7feb3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400b6c0 con 0x7feb400d2e90 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34019070 con 0x7feb400d2e90 2024-04-25T10:05:31.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4014eed0 con 0x7feb400d2e90 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.613+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400baa0 con 0x7feb400d2e90 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.614+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4014f490 con 0x7feb400d2e90 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.615+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4015e500 con 0x7feb400d2e90 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.615+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb3401e460 con 0x7feb400d2e90 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.615+0000 7feb277fe640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb2c06b4d0 0x7feb2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.615+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb3408a940 con 0x7feb400d2e90 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.615+0000 7feb3effd640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb2c06b4d0 0x7feb2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.616+0000 7feb3effd640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb2c06b4d0 0x7feb2c06d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7feb400cd150 tx=0x7feb30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.619+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb340596c0 con 0x7feb400d2e90 2024-04-25T10:05:31.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.794+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb40073990 con 0x7feb400d2e90 2024-04-25T10:05:31.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.795+0000 7feb277fe640 1 -- 172.21.15.148:0/2203908537 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16936 (secure 0 0 0) 0x7feb34058fc0 con 0x7feb400d2e90 2024-04-25T10:05:31.801 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:31.802 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":171,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:05:30.141480+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb2c06b4d0 msgr2=0x7feb2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb2c06b4d0 0x7feb2c06d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7feb400cd150 tx=0x7feb30008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 msgr2=0x7feb4014bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb4014bca0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7feb3400d830 tx=0x7feb3400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 shutdown_connections 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb2c06b4d0 0x7feb2c06d990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4015dd30 0x7feb40160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb400d2e90 0x7feb4014bca0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 --2- 172.21.15.148:0/2203908537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb400b9770 0x7feb4014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.797+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 >> 172.21.15.148:0/2203908537 conn(0x7feb40061c30 msgr2=0x7feb4015d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.798+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 shutdown_connections 2024-04-25T10:05:31.804 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.798+0000 7feb45683640 1 -- 172.21.15.148:0/2203908537 wait complete. 2024-04-25T10:05:31.815 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:31.815 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:31.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.908+0000 7fead3542640 1 Processor -- start 2024-04-25T10:05:31.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.908+0000 7fead3542640 1 -- start start 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead3542640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feacc0cfec0 0x7feacc0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead3542640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc071480 con 0x7feacc154ee0 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead3542640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feacc071760 con 0x7feacc0ce3c0 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead3542640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feacc0ce830 con 0x7feacc0cfec0 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead12de640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feacc154ee0 0x7feacc0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55756/0 (socket says 172.21.15.148:55756) 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead12de640 1 -- 172.21.15.148:0/271699986 learned_addr learned my addr 172.21.15.148:0/271699986 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998175114 0 0) 0x7feacc071480 con 0x7feacc154ee0 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab8003520 con 0x7feacc154ee0 2024-04-25T10:05:31.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead0add640 1 --2- 172.21.15.148:0/271699986 >> v2:172.21.15.188:3300/0 conn(0x7feacc0cfec0 0x7feacc0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642097417 0 0) 0x7feab8003520 con 0x7feacc154ee0 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 >> v2:172.21.15.188:3300/0 conn(0x7feacc0cfec0 msgr2=0x7feacc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 --2- 172.21.15.148:0/271699986 >> v2:172.21.15.188:3300/0 conn(0x7feacc0cfec0 0x7feacc0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 >> v1:172.21.15.188:6789/0 conn(0x7feacc0ce3c0 legacy=0x7feacc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc071a40 con 0x7feacc154ee0 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead0add640 1 --2- 172.21.15.148:0/271699986 >> v2:172.21.15.188:3300/0 conn(0x7feacc0cfec0 0x7feacc0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7feac8001670 con 0x7feacc154ee0 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 >> v1:172.21.15.148:6789/0 conn(0x7feacc154ee0 legacy=0x7feacc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab80039b0 0x7feab8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab80051c0 0x7feab800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc0ce830 con 0x7feab8004800 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab800c110 con 0x7feab80051c0 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800e520 con 0x7feab80039b0 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead0add640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead12de640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab80039b0 0x7feab8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.909+0000 7fead1adf640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab80051c0 0x7feab800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead0add640 1 -- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab80039b0 msgr2=0x7feab8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead0add640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab80039b0 0x7feab8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead0add640 1 -- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab80051c0 msgr2=0x7feab800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead0add640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab80051c0 0x7feab800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead1adf640 1 -- 172.21.15.148:0/271699986 reap_dead start 2024-04-25T10:05:31.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead0add640 1 -- 172.21.15.148:0/271699986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8003520 con 0x7feab8004800 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7fead0add640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feab8004c80 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7feabc0098a0 tx=0x7feab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d92e14f55e62d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.910+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc00f040 con 0x7feab8004800 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc02ace0 con 0x7feab8004800 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7feac3fff640 1 -- 172.21.15.148:0/271699986 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc004250 con 0x7feab8004800 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 -- 172.21.15.148:0/271699986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 msgr2=0x7feab8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feab8004c80 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7feabc0098a0 tx=0x7feab8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 -- 172.21.15.148:0/271699986 shutdown_connections 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 --2- 172.21.15.148:0/271699986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feab8004c80 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 -- 172.21.15.148:0/271699986 >> 172.21.15.148:0/271699986 conn(0x7feacc06abe0 msgr2=0x7feacc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 -- 172.21.15.148:0/271699986 shutdown_connections 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.911+0000 7fead3542640 1 -- 172.21.15.148:0/271699986 wait complete. 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.912+0000 7fead3542640 1 Processor -- start 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.912+0000 7fead3542640 1 -- start start 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead3542640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feacc14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead3542640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feacc0adde0 0x7feacc1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead3542640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feacc14e3a0 0x7feacc159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead3542640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc071230 con 0x7feab8004800 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead3542640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc1552c0 con 0x7feacc14e3a0 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead3542640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feacc155440 con 0x7feacc0adde0 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feacc14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feacc14f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48556/0 (socket says 172.21.15.148:48556) 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 -- 172.21.15.148:0/4044659851 learned_addr learned my addr 172.21.15.148:0/4044659851 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:31.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead0add640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feacc0adde0 0x7feacc1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead1adf640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feacc14e3a0 0x7feacc159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 -- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feacc0adde0 msgr2=0x7feacc1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feacc0adde0 0x7feacc1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 -- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feacc14e3a0 msgr2=0x7feacc159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.913+0000 7fead12de640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feacc14e3a0 0x7feacc159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7fead12de640 1 -- 172.21.15.148:0/4044659851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc0afcc0 con 0x7feab8004800 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7fead12de640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feacc14f430 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7feac80072c0 tx=0x7feac8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800f960 con 0x7feab8004800 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac8018070 con 0x7feab8004800 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac800fd80 con 0x7feab8004800 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc0ae2e0 con 0x7feab8004800 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.914+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc1e3f10 con 0x7feab8004800 2024-04-25T10:05:31.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.915+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc073990 con 0x7feab8004800 2024-04-25T10:05:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.915+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feac801d460 con 0x7feab8004800 2024-04-25T10:05:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.916+0000 7feac1ffb640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab006b500 0x7feab006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.916+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feac808a270 con 0x7feab8004800 2024-04-25T10:05:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.916+0000 7fead0add640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab006b500 0x7feab006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.916+0000 7fead0add640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab006b500 0x7feab006d9c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7feabc033000 tx=0x7feabc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:31.920+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feac8058910 con 0x7feab8004800 2024-04-25T10:05:32.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.088+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feacc14e810 con 0x7feab006b500 2024-04-25T10:05:32.099 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.094+0000 7feac1ffb640 1 -- 172.21.15.148:0/4044659851 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7feacc14e810 con 0x7feab006b500 2024-04-25T10:05:32.104 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:32.107 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.096+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab006b500 msgr2=0x7feab006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.096+0000 7fead3542640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab006b500 0x7feab006d9c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7feabc033000 tx=0x7feabc02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.096+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 msgr2=0x7feacc14f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.096+0000 7fead3542640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feacc14f430 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7feac80072c0 tx=0x7feac8007780 comp rx=0 tx=0).stop 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 shutdown_connections 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feab006b500 0x7feab006d9c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feacc14e3a0 0x7feacc159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feacc0adde0 0x7feacc1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 --2- 172.21.15.148:0/4044659851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feab8004800 0x7feacc14f430 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 >> 172.21.15.148:0/4044659851 conn(0x7feacc06abe0 msgr2=0x7feacc0b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 shutdown_connections 2024-04-25T10:05:32.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.097+0000 7fead3542640 1 -- 172.21.15.148:0/4044659851 wait complete. 2024-04-25T10:05:32.113 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-04-25T10:05:31.158596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440952,"kb_used_data":17960,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754533768,"statfs":{"total":773094113280,"available":772642578432,"internally_reserved":0,"allocated":18391040,"data_stored":9205164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025625"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.156552+0000","last_change":"2024-04-25T10:05:29.154087+0000","last_active":"2024-04-25T10:05:29.156552+0000","last_peered":"2024-04-25T10:05:29.156552+0000","last_clean":"2024-04-25T10:05:29.156552+0000","last_became_active":"2024-04-25T10:05:29.153599+0000","last_became_peered":"2024-04-25T10:05:29.153599+0000","last_unstale":"2024-04-25T10:05:29.156552+0000","last_undegraded":"2024-04-25T10:05:29.156552+0000","last_fullsized":"2024-04-25T10:05:29.156552+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.164447+0000","last_change":"2024-04-25T10:05:29.158279+0000","last_active":"2024-04-25T10:05:29.164447+0000","last_peered":"2024-04-25T10:05:29.164447+0000","last_clean":"2024-04-25T10:05:29.164447+0000","last_became_active":"2024-04-25T10:05:29.158024+0000","last_became_peered":"2024-04-25T10:05:29.158024+0000","last_unstale":"2024-04-25T10:05:29.164447+0000","last_undegraded":"2024-04-25T10:05:29.164447+0000","last_fullsized":"2024-04-25T10:05:29.164447+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:25.308183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166797+0000","last_change":"2024-04-25T10:05:29.163993+0000","last_active":"2024-04-25T10:05:29.166797+0000","last_peered":"2024-04-25T10:05:29.166797+0000","last_clean":"2024-04-25T10:05:29.166797+0000","last_became_active":"2024-04-25T10:05:29.163652+0000","last_became_peered":"2024-04-25T10:05:29.163652+0000","last_unstale":"2024-04-25T10:05:29.166797+0000","last_undegraded":"2024-04-25T10:05:29.166797+0000","last_fullsized":"2024-04-25T10:05:29.166797+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:37.793621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163799+0000","last_change":"2024-04-25T10:05:29.161112+0000","last_active":"2024-04-25T10:05:29.163799+0000","last_peered":"2024-04-25T10:05:29.163799+0000","last_clean":"2024-04-25T10:05:29.163799+0000","last_became_active":"2024-04-25T10:05:29.160861+0000","last_became_peered":"2024-04-25T10:05:29.160861+0000","last_unstale":"2024-04-25T10:05:29.163799+0000","last_undegraded":"2024-04-25T10:05:29.163799+0000","last_fullsized":"2024-04-25T10:05:29.163799+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:07.408468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144711+0000","last_change":"2024-04-25T10:05:29.157549+0000","last_active":"2024-04-25T10:05:30.144711+0000","last_peered":"2024-04-25T10:05:30.144711+0000","last_clean":"2024-04-25T10:05:30.144711+0000","last_became_active":"2024-04-25T10:05:29.156314+0000","last_became_peered":"2024-04-25T10:05:29.156314+0000","last_unstale":"2024-04-25T10:05:30.144711+0000","last_undegraded":"2024-04-25T10:05:30.144711+0000","last_fullsized":"2024-04-25T10:05:30.144711+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:35.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144471+0000","last_change":"2024-04-25T10:05:29.161933+0000","last_active":"2024-04-25T10:05:30.144471+0000","last_peered":"2024-04-25T10:05:30.144471+0000","last_clean":"2024-04-25T10:05:30.144471+0000","last_became_active":"2024-04-25T10:05:29.161530+0000","last_became_peered":"2024-04-25T10:05:29.161530+0000","last_unstale":"2024-04-25T10:05:30.144471+0000","last_undegraded":"2024-04-25T10:05:30.144471+0000","last_fullsized":"2024-04-25T10:05:30.144471+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:50.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.160581+0000","last_change":"2024-04-25T10:05:29.159168+0000","last_active":"2024-04-25T10:05:29.160581+0000","last_peered":"2024-04-25T10:05:29.160581+0000","last_clean":"2024-04-25T10:05:29.160581+0000","last_became_active":"2024-04-25T10:05:29.158800+0000","last_became_peered":"2024-04-25T10:05:29.158800+0000","last_unstale":"2024-04-25T10:05:29.160581+0000","last_undegraded":"2024-04-25T10:05:29.160581+0000","last_fullsized":"2024-04-25T10:05:29.160581+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:14.594205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173189+0000","last_change":"2024-04-25T10:05:29.171224+0000","last_active":"2024-04-25T10:05:29.173189+0000","last_peered":"2024-04-25T10:05:29.173189+0000","last_clean":"2024-04-25T10:05:29.173189+0000","last_became_active":"2024-04-25T10:05:29.170912+0000","last_became_peered":"2024-04-25T10:05:29.170912+0000","last_unstale":"2024-04-25T10:05:29.173189+0000","last_undegraded":"2024-04-25T10:05:29.173189+0000","last_fullsized":"2024-04-25T10:05:29.173189+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:12.203089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.155137+0000","last_change":"2024-04-25T10:05:29.153879+0000","last_active":"2024-04-25T10:05:29.155137+0000","last_peered":"2024-04-25T10:05:29.155137+0000","last_clean":"2024-04-25T10:05:29.155137+0000","last_became_active":"2024-04-25T10:05:29.153246+0000","last_became_peered":"2024-04-25T10:05:29.153246+0000","last_unstale":"2024-04-25T10:05:29.155137+0000","last_undegraded":"2024-04-25T10:05:29.155137+0000","last_fullsized":"2024-04-25T10:05:29.155137+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:42.808918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144672+0000","last_change":"2024-04-25T10:05:29.164386+0000","last_active":"2024-04-25T10:05:30.144672+0000","last_peered":"2024-04-25T10:05:30.144672+0000","last_clean":"2024-04-25T10:05:30.144672+0000","last_became_active":"2024-04-25T10:05:29.164124+0000","last_became_peered":"2024-04-25T10:05:29.164124+0000","last_unstale":"2024-04-25T10:05:30.144672+0000","last_undegraded":"2024-04-25T10:05:30.144672+0000","last_fullsized":"2024-04-25T10:05:30.144672+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:46.181959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163143+0000","last_change":"2024-04-25T10:05:29.161316+0000","last_active":"2024-04-25T10:05:29.163143+0000","last_peered":"2024-04-25T10:05:29.163143+0000","last_clean":"2024-04-25T10:05:29.163143+0000","last_became_active":"2024-04-25T10:05:29.161046+0000","last_became_peered":"2024-04-25T10:05:29.161046+0000","last_unstale":"2024-04-25T10:05:29.163143+0000","last_undegraded":"2024-04-25T10:05:29.163143+0000","last_fullsized":"2024-04-25T10:05:29.163143+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:24.010605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157848+0000","last_change":"2024-04-25T10:05:29.156268+0000","last_active":"2024-04-25T10:05:29.157848+0000","last_peered":"2024-04-25T10:05:29.157848+0000","last_clean":"2024-04-25T10:05:29.157848+0000","last_became_active":"2024-04-25T10:05:29.155963+0000","last_became_peered":"2024-04-25T10:05:29.155963+0000","last_unstale":"2024-04-25T10:05:29.157848+0000","last_undegraded":"2024-04-25T10:05:29.157848+0000","last_fullsized":"2024-04-25T10:05:29.157848+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:01.586423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174377+0000","last_change":"2024-04-25T10:05:29.172710+0000","last_active":"2024-04-25T10:05:29.174377+0000","last_peered":"2024-04-25T10:05:29.174377+0000","last_clean":"2024-04-25T10:05:29.174377+0000","last_became_active":"2024-04-25T10:05:29.172441+0000","last_became_peered":"2024-04-25T10:05:29.172441+0000","last_unstale":"2024-04-25T10:05:29.174377+0000","last_undegraded":"2024-04-25T10:05:29.174377+0000","last_fullsized":"2024-04-25T10:05:29.174377+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:35.096707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163352+0000","last_change":"2024-04-25T10:05:29.161614+0000","last_active":"2024-04-25T10:05:29.163352+0000","last_peered":"2024-04-25T10:05:29.163352+0000","last_clean":"2024-04-25T10:05:29.163352+0000","last_became_active":"2024-04-25T10:05:29.161368+0000","last_became_peered":"2024-04-25T10:05:29.161368+0000","last_unstale":"2024-04-25T10:05:29.163352+0000","last_undegraded":"2024-04-25T10:05:29.163352+0000","last_fullsized":"2024-04-25T10:05:29.163352+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.674181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166957+0000","last_change":"2024-04-25T10:05:29.163172+0000","last_active":"2024-04-25T10:05:29.166957+0000","last_peered":"2024-04-25T10:05:29.166957+0000","last_clean":"2024-04-25T10:05:29.166957+0000","last_became_active":"2024-04-25T10:05:29.162891+0000","last_became_peered":"2024-04-25T10:05:29.162891+0000","last_unstale":"2024-04-25T10:05:29.166957+0000","last_undegraded":"2024-04-25T10:05:29.166957+0000","last_fullsized":"2024-04-25T10:05:29.166957+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:31.890534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144575+0000","last_change":"2024-04-25T10:05:29.158837+0000","last_active":"2024-04-25T10:05:30.144575+0000","last_peered":"2024-04-25T10:05:30.144575+0000","last_clean":"2024-04-25T10:05:30.144575+0000","last_became_active":"2024-04-25T10:05:29.157286+0000","last_became_peered":"2024-04-25T10:05:29.157286+0000","last_unstale":"2024-04-25T10:05:30.144575+0000","last_undegraded":"2024-04-25T10:05:30.144575+0000","last_fullsized":"2024-04-25T10:05:30.144575+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:59.322410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144783+0000","last_change":"2024-04-25T10:05:29.159522+0000","last_active":"2024-04-25T10:05:30.144783+0000","last_peered":"2024-04-25T10:05:30.144783+0000","last_clean":"2024-04-25T10:05:30.144783+0000","last_became_active":"2024-04-25T10:05:29.156706+0000","last_became_peered":"2024-04-25T10:05:29.156706+0000","last_unstale":"2024-04-25T10:05:30.144783+0000","last_undegraded":"2024-04-25T10:05:30.144783+0000","last_fullsized":"2024-04-25T10:05:30.144783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:00.088588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.167369+0000","last_change":"2024-04-25T10:05:29.165602+0000","last_active":"2024-04-25T10:05:29.167369+0000","last_peered":"2024-04-25T10:05:29.167369+0000","last_clean":"2024-04-25T10:05:29.167369+0000","last_became_active":"2024-04-25T10:05:29.165302+0000","last_became_peered":"2024-04-25T10:05:29.165302+0000","last_unstale":"2024-04-25T10:05:29.167369+0000","last_undegraded":"2024-04-25T10:05:29.167369+0000","last_fullsized":"2024-04-25T10:05:29.167369+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:51.604284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170688+0000","last_change":"2024-04-25T10:05:29.169091+0000","last_active":"2024-04-25T10:05:29.170688+0000","last_peered":"2024-04-25T10:05:29.170688+0000","last_clean":"2024-04-25T10:05:29.170688+0000","last_became_active":"2024-04-25T10:05:29.167208+0000","last_became_peered":"2024-04-25T10:05:29.167208+0000","last_unstale":"2024-04-25T10:05:29.170688+0000","last_undegraded":"2024-04-25T10:05:29.170688+0000","last_fullsized":"2024-04-25T10:05:29.170688+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:12.790345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144530+0000","last_change":"2024-04-25T10:05:29.159795+0000","last_active":"2024-04-25T10:05:30.144530+0000","last_peered":"2024-04-25T10:05:30.144530+0000","last_clean":"2024-04-25T10:05:30.144530+0000","last_became_active":"2024-04-25T10:05:29.159547+0000","last_became_peered":"2024-04-25T10:05:29.159547+0000","last_unstale":"2024-04-25T10:05:30.144530+0000","last_undegraded":"2024-04-25T10:05:30.144530+0000","last_fullsized":"2024-04-25T10:05:30.144530+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:06.640485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.167031+0000","last_change":"2024-04-25T10:05:29.163954+0000","last_active":"2024-04-25T10:05:29.167031+0000","last_peered":"2024-04-25T10:05:29.167031+0000","last_clean":"2024-04-25T10:05:29.167031+0000","last_became_active":"2024-04-25T10:05:29.162312+0000","last_became_peered":"2024-04-25T10:05:29.162312+0000","last_unstale":"2024-04-25T10:05:29.167031+0000","last_undegraded":"2024-04-25T10:05:29.167031+0000","last_fullsized":"2024-04-25T10:05:29.167031+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:25.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157719+0000","last_change":"2024-04-25T10:05:29.156719+0000","last_active":"2024-04-25T10:05:29.157719+0000","last_peered":"2024-04-25T10:05:29.157719+0000","last_clean":"2024-04-25T10:05:29.157719+0000","last_became_active":"2024-04-25T10:05:29.153417+0000","last_became_peered":"2024-04-25T10:05:29.153417+0000","last_unstale":"2024-04-25T10:05:29.157719+0000","last_undegraded":"2024-04-25T10:05:29.157719+0000","last_fullsized":"2024-04-25T10:05:29.157719+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:08.389066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.160835+0000","last_change":"2024-04-25T10:05:29.158313+0000","last_active":"2024-04-25T10:05:29.160835+0000","last_peered":"2024-04-25T10:05:29.160835+0000","last_clean":"2024-04-25T10:05:29.160835+0000","last_became_active":"2024-04-25T10:05:29.158040+0000","last_became_peered":"2024-04-25T10:05:29.158040+0000","last_unstale":"2024-04-25T10:05:29.160835+0000","last_undegraded":"2024-04-25T10:05:29.160835+0000","last_fullsized":"2024-04-25T10:05:29.160835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.686063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157180+0000","last_change":"2024-04-25T10:05:29.153968+0000","last_active":"2024-04-25T10:05:29.157180+0000","last_peered":"2024-04-25T10:05:29.157180+0000","last_clean":"2024-04-25T10:05:29.157180+0000","last_became_active":"2024-04-25T10:05:29.153714+0000","last_became_peered":"2024-04-25T10:05:29.153714+0000","last_unstale":"2024-04-25T10:05:29.157180+0000","last_undegraded":"2024-04-25T10:05:29.157180+0000","last_fullsized":"2024-04-25T10:05:29.157180+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:12.203089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.161049+0000","last_change":"2024-04-25T10:05:29.160521+0000","last_active":"2024-04-25T10:05:29.161049+0000","last_peered":"2024-04-25T10:05:29.161049+0000","last_clean":"2024-04-25T10:05:29.161049+0000","last_became_active":"2024-04-25T10:05:29.160251+0000","last_became_peered":"2024-04-25T10:05:29.160251+0000","last_unstale":"2024-04-25T10:05:29.161049+0000","last_undegraded":"2024-04-25T10:05:29.161049+0000","last_fullsized":"2024-04-25T10:05:29.161049+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:49.457436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166593+0000","last_change":"2024-04-25T10:05:29.165928+0000","last_active":"2024-04-25T10:05:29.166593+0000","last_peered":"2024-04-25T10:05:29.166593+0000","last_clean":"2024-04-25T10:05:29.166593+0000","last_became_active":"2024-04-25T10:05:29.165488+0000","last_became_peered":"2024-04-25T10:05:29.165488+0000","last_unstale":"2024-04-25T10:05:29.166593+0000","last_undegraded":"2024-04-25T10:05:29.166593+0000","last_fullsized":"2024-04-25T10:05:29.166593+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163991+0000","last_change":"2024-04-25T10:05:29.162139+0000","last_active":"2024-04-25T10:05:29.163991+0000","last_peered":"2024-04-25T10:05:29.163991+0000","last_clean":"2024-04-25T10:05:29.163991+0000","last_became_active":"2024-04-25T10:05:29.161845+0000","last_became_peered":"2024-04-25T10:05:29.161845+0000","last_unstale":"2024-04-25T10:05:29.163991+0000","last_undegraded":"2024-04-25T10:05:29.163991+0000","last_fullsized":"2024-04-25T10:05:29.163991+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:28.347011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144670+0000","last_change":"2024-04-25T10:05:29.166217+0000","last_active":"2024-04-25T10:05:30.144670+0000","last_peered":"2024-04-25T10:05:30.144670+0000","last_clean":"2024-04-25T10:05:30.144670+0000","last_became_active":"2024-04-25T10:05:29.165944+0000","last_became_peered":"2024-04-25T10:05:29.165944+0000","last_unstale":"2024-04-25T10:05:30.144670+0000","last_undegraded":"2024-04-25T10:05:30.144670+0000","last_fullsized":"2024-04-25T10:05:30.144670+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:48.900730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.171617+0000","last_change":"2024-04-25T10:05:29.169638+0000","last_active":"2024-04-25T10:05:29.171617+0000","last_peered":"2024-04-25T10:05:29.171617+0000","last_clean":"2024-04-25T10:05:29.171617+0000","last_became_active":"2024-04-25T10:05:29.169346+0000","last_became_peered":"2024-04-25T10:05:29.169346+0000","last_unstale":"2024-04-25T10:05:29.171617+0000","last_undegraded":"2024-04-25T10:05:29.171617+0000","last_fullsized":"2024-04-25T10:05:29.171617+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:28.118560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.165286+0000","last_change":"2024-04-25T10:05:29.164340+0000","last_active":"2024-04-25T10:05:29.165286+0000","last_peered":"2024-04-25T10:05:29.165286+0000","last_clean":"2024-04-25T10:05:29.165286+0000","last_became_active":"2024-04-25T10:05:29.164078+0000","last_became_peered":"2024-04-25T10:05:29.164078+0000","last_unstale":"2024-04-25T10:05:29.165286+0000","last_undegraded":"2024-04-25T10:05:29.165286+0000","last_fullsized":"2024-04-25T10:05:29.165286+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:12.203089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.155368+0000","last_change":"2024-04-25T10:05:29.154076+0000","last_active":"2024-04-25T10:05:29.155368+0000","last_peered":"2024-04-25T10:05:29.155368+0000","last_clean":"2024-04-25T10:05:29.155368+0000","last_became_active":"2024-04-25T10:05:29.153786+0000","last_became_peered":"2024-04-25T10:05:29.153786+0000","last_unstale":"2024-04-25T10:05:29.155368+0000","last_undegraded":"2024-04-25T10:05:29.155368+0000","last_fullsized":"2024-04-25T10:05:29.155368+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:41.163694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144405+0000","last_change":"2024-04-25T10:05:29.160527+0000","last_active":"2024-04-25T10:05:30.144405+0000","last_peered":"2024-04-25T10:05:30.144405+0000","last_clean":"2024-04-25T10:05:30.144405+0000","last_became_active":"2024-04-25T10:05:29.160268+0000","last_became_peered":"2024-04-25T10:05:29.160268+0000","last_unstale":"2024-04-25T10:05:30.144405+0000","last_undegraded":"2024-04-25T10:05:30.144405+0000","last_fullsized":"2024-04-25T10:05:30.144405+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:09.037951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.162174+0000","last_change":"2024-04-25T10:05:29.161075+0000","last_active":"2024-04-25T10:05:29.162174+0000","last_peered":"2024-04-25T10:05:29.162174+0000","last_clean":"2024-04-25T10:05:29.162174+0000","last_became_active":"2024-04-25T10:05:29.160577+0000","last_became_peered":"2024-04-25T10:05:29.160577+0000","last_unstale":"2024-04-25T10:05:29.162174+0000","last_undegraded":"2024-04-25T10:05:29.162174+0000","last_fullsized":"2024-04-25T10:05:29.162174+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.573591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"170'4","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172372+0000","last_change":"2024-04-25T10:05:28.154845+0000","last_active":"2024-04-25T10:05:29.172372+0000","last_peered":"2024-04-25T10:05:29.172372+0000","last_clean":"2024-04-25T10:05:29.172372+0000","last_became_active":"2024-04-25T10:05:28.154486+0000","last_became_peered":"2024-04-25T10:05:28.154486+0000","last_unstale":"2024-04-25T10:05:29.172372+0000","last_undegraded":"2024-04-25T10:05:29.172372+0000","last_fullsized":"2024-04-25T10:05:29.172372+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:16.351218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163328+0000","last_change":"2024-04-25T10:05:29.161089+0000","last_active":"2024-04-25T10:05:29.163328+0000","last_peered":"2024-04-25T10:05:29.163328+0000","last_clean":"2024-04-25T10:05:29.163328+0000","last_became_active":"2024-04-25T10:05:29.160837+0000","last_became_peered":"2024-04-25T10:05:29.160837+0000","last_unstale":"2024-04-25T10:05:29.163328+0000","last_undegraded":"2024-04-25T10:05:29.163328+0000","last_fullsized":"2024-04-25T10:05:29.163328+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:32.691718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144802+0000","last_change":"2024-04-25T10:05:28.155931+0000","last_active":"2024-04-25T10:05:30.144802+0000","last_peered":"2024-04-25T10:05:30.144802+0000","last_clean":"2024-04-25T10:05:30.144802+0000","last_became_active":"2024-04-25T10:05:28.155637+0000","last_became_peered":"2024-04-25T10:05:28.155637+0000","last_unstale":"2024-04-25T10:05:30.144802+0000","last_undegraded":"2024-04-25T10:05:30.144802+0000","last_fullsized":"2024-04-25T10:05:30.144802+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:14.968782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.164672+0000","last_change":"2024-04-25T10:05:29.164055+0000","last_active":"2024-04-25T10:05:29.164672+0000","last_peered":"2024-04-25T10:05:29.164672+0000","last_clean":"2024-04-25T10:05:29.164672+0000","last_became_active":"2024-04-25T10:05:29.163427+0000","last_became_peered":"2024-04-25T10:05:29.163427+0000","last_unstale":"2024-04-25T10:05:29.164672+0000","last_undegraded":"2024-04-25T10:05:29.164672+0000","last_fullsized":"2024-04-25T10:05:29.164672+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:42.808918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.168241+0000","last_change":"2024-04-25T10:05:28.156908+0000","last_active":"2024-04-25T10:05:29.168241+0000","last_peered":"2024-04-25T10:05:29.168241+0000","last_clean":"2024-04-25T10:05:29.168241+0000","last_became_active":"2024-04-25T10:05:28.156603+0000","last_became_peered":"2024-04-25T10:05:28.156603+0000","last_unstale":"2024-04-25T10:05:29.168241+0000","last_undegraded":"2024-04-25T10:05:29.168241+0000","last_fullsized":"2024-04-25T10:05:29.168241+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:49.869162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170810+0000","last_change":"2024-04-25T10:05:29.168030+0000","last_active":"2024-04-25T10:05:29.170810+0000","last_peered":"2024-04-25T10:05:29.170810+0000","last_clean":"2024-04-25T10:05:29.170810+0000","last_became_active":"2024-04-25T10:05:29.167757+0000","last_became_peered":"2024-04-25T10:05:29.167757+0000","last_unstale":"2024-04-25T10:05:29.170810+0000","last_undegraded":"2024-04-25T10:05:29.170810+0000","last_fullsized":"2024-04-25T10:05:29.170810+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:53.367980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173190+0000","last_change":"2024-04-25T10:05:28.154255+0000","last_active":"2024-04-25T10:05:29.173190+0000","last_peered":"2024-04-25T10:05:29.173190+0000","last_clean":"2024-04-25T10:05:29.173190+0000","last_became_active":"2024-04-25T10:05:28.153588+0000","last_became_peered":"2024-04-25T10:05:28.153588+0000","last_unstale":"2024-04-25T10:05:29.173190+0000","last_undegraded":"2024-04-25T10:05:29.173190+0000","last_fullsized":"2024-04-25T10:05:29.173190+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:57.146747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.171023+0000","last_change":"2024-04-25T10:05:29.170461+0000","last_active":"2024-04-25T10:05:29.171023+0000","last_peered":"2024-04-25T10:05:29.171023+0000","last_clean":"2024-04-25T10:05:29.171023+0000","last_became_active":"2024-04-25T10:05:29.170162+0000","last_became_peered":"2024-04-25T10:05:29.170162+0000","last_unstale":"2024-04-25T10:05:29.171023+0000","last_undegraded":"2024-04-25T10:05:29.171023+0000","last_fullsized":"2024-04-25T10:05:29.171023+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:05.952651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144885+0000","last_change":"2024-04-25T10:05:28.156281+0000","last_active":"2024-04-25T10:05:30.144885+0000","last_peered":"2024-04-25T10:05:30.144885+0000","last_clean":"2024-04-25T10:05:30.144885+0000","last_became_active":"2024-04-25T10:05:28.155999+0000","last_became_peered":"2024-04-25T10:05:28.155999+0000","last_unstale":"2024-04-25T10:05:30.144885+0000","last_undegraded":"2024-04-25T10:05:30.144885+0000","last_fullsized":"2024-04-25T10:05:30.144885+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:26.596777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157196+0000","last_change":"2024-04-25T10:05:29.156214+0000","last_active":"2024-04-25T10:05:29.157196+0000","last_peered":"2024-04-25T10:05:29.157196+0000","last_clean":"2024-04-25T10:05:29.157196+0000","last_became_active":"2024-04-25T10:05:29.153121+0000","last_became_peered":"2024-04-25T10:05:29.153121+0000","last_unstale":"2024-04-25T10:05:29.157196+0000","last_undegraded":"2024-04-25T10:05:29.157196+0000","last_fullsized":"2024-04-25T10:05:29.157196+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:28.118560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173766+0000","last_change":"2024-04-25T10:05:28.165373+0000","last_active":"2024-04-25T10:05:29.173766+0000","last_peered":"2024-04-25T10:05:29.173766+0000","last_clean":"2024-04-25T10:05:29.173766+0000","last_became_active":"2024-04-25T10:05:28.165083+0000","last_became_peered":"2024-04-25T10:05:28.165083+0000","last_unstale":"2024-04-25T10:05:29.173766+0000","last_undegraded":"2024-04-25T10:05:29.173766+0000","last_fullsized":"2024-04-25T10:05:29.173766+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:16.351054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.167146+0000","last_change":"2024-04-25T10:05:29.162980+0000","last_active":"2024-04-25T10:05:29.167146+0000","last_peered":"2024-04-25T10:05:29.167146+0000","last_clean":"2024-04-25T10:05:29.167146+0000","last_became_active":"2024-04-25T10:05:29.162665+0000","last_became_peered":"2024-04-25T10:05:29.162665+0000","last_unstale":"2024-04-25T10:05:29.167146+0000","last_undegraded":"2024-04-25T10:05:29.167146+0000","last_fullsized":"2024-04-25T10:05:29.167146+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:13.700571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.159290+0000","last_change":"2024-04-25T10:05:28.155936+0000","last_active":"2024-04-25T10:05:29.159290+0000","last_peered":"2024-04-25T10:05:29.159290+0000","last_clean":"2024-04-25T10:05:29.159290+0000","last_became_active":"2024-04-25T10:05:28.155589+0000","last_became_peered":"2024-04-25T10:05:28.155589+0000","last_unstale":"2024-04-25T10:05:29.159290+0000","last_undegraded":"2024-04-25T10:05:29.159290+0000","last_fullsized":"2024-04-25T10:05:29.159290+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:23.312915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163976+0000","last_change":"2024-04-25T10:05:29.162037+0000","last_active":"2024-04-25T10:05:29.163976+0000","last_peered":"2024-04-25T10:05:29.163976+0000","last_clean":"2024-04-25T10:05:29.163976+0000","last_became_active":"2024-04-25T10:05:29.161445+0000","last_became_peered":"2024-04-25T10:05:29.161445+0000","last_unstale":"2024-04-25T10:05:29.163976+0000","last_undegraded":"2024-04-25T10:05:29.163976+0000","last_fullsized":"2024-04-25T10:05:29.163976+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:24.010605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144719+0000","last_change":"2024-04-25T10:05:28.160663+0000","last_active":"2024-04-25T10:05:30.144719+0000","last_peered":"2024-04-25T10:05:30.144719+0000","last_clean":"2024-04-25T10:05:30.144719+0000","last_became_active":"2024-04-25T10:05:28.157679+0000","last_became_peered":"2024-04-25T10:05:28.157679+0000","last_unstale":"2024-04-25T10:05:30.144719+0000","last_undegraded":"2024-04-25T10:05:30.144719+0000","last_fullsized":"2024-04-25T10:05:30.144719+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:00.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144763+0000","last_change":"2024-04-25T10:05:29.160476+0000","last_active":"2024-04-25T10:05:30.144763+0000","last_peered":"2024-04-25T10:05:30.144763+0000","last_clean":"2024-04-25T10:05:30.144763+0000","last_became_active":"2024-04-25T10:05:29.160070+0000","last_became_peered":"2024-04-25T10:05:29.160070+0000","last_unstale":"2024-04-25T10:05:30.144763+0000","last_undegraded":"2024-04-25T10:05:30.144763+0000","last_fullsized":"2024-04-25T10:05:30.144763+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:05.677474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.144789+0000","last_change":"2024-04-25T10:05:28.165152+0000","last_active":"2024-04-25T10:05:29.144789+0000","last_peered":"2024-04-25T10:05:29.144789+0000","last_clean":"2024-04-25T10:05:29.144789+0000","last_became_active":"2024-04-25T10:05:28.164888+0000","last_became_peered":"2024-04-25T10:05:28.164888+0000","last_unstale":"2024-04-25T10:05:29.144789+0000","last_undegraded":"2024-04-25T10:05:29.144789+0000","last_fullsized":"2024-04-25T10:05:29.144789+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:23.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170659+0000","last_change":"2024-04-25T10:05:29.167230+0000","last_active":"2024-04-25T10:05:29.170659+0000","last_peered":"2024-04-25T10:05:29.170659+0000","last_clean":"2024-04-25T10:05:29.170659+0000","last_became_active":"2024-04-25T10:05:29.166188+0000","last_became_peered":"2024-04-25T10:05:29.166188+0000","last_unstale":"2024-04-25T10:05:29.170659+0000","last_undegraded":"2024-04-25T10:05:29.170659+0000","last_fullsized":"2024-04-25T10:05:29.170659+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:07.793081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"170'3","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173771+0000","last_change":"2024-04-25T10:05:28.165707+0000","last_active":"2024-04-25T10:05:29.173771+0000","last_peered":"2024-04-25T10:05:29.173771+0000","last_clean":"2024-04-25T10:05:29.173771+0000","last_became_active":"2024-04-25T10:05:28.165440+0000","last_became_peered":"2024-04-25T10:05:28.165440+0000","last_unstale":"2024-04-25T10:05:29.173771+0000","last_undegraded":"2024-04-25T10:05:29.173771+0000","last_fullsized":"2024-04-25T10:05:29.173771+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:13.986487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.169157+0000","last_change":"2024-04-25T10:05:29.165141+0000","last_active":"2024-04-25T10:05:29.169157+0000","last_peered":"2024-04-25T10:05:29.169157+0000","last_clean":"2024-04-25T10:05:29.169157+0000","last_became_active":"2024-04-25T10:05:29.164890+0000","last_became_peered":"2024-04-25T10:05:29.164890+0000","last_unstale":"2024-04-25T10:05:29.169157+0000","last_undegraded":"2024-04-25T10:05:29.169157+0000","last_fullsized":"2024-04-25T10:05:29.169157+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:37.172203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"170'6","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174650+0000","last_change":"2024-04-25T10:05:28.154486+0000","last_active":"2024-04-25T10:05:29.174650+0000","last_peered":"2024-04-25T10:05:29.174650+0000","last_clean":"2024-04-25T10:05:29.174650+0000","last_became_active":"2024-04-25T10:05:28.153981+0000","last_became_peered":"2024-04-25T10:05:28.153981+0000","last_unstale":"2024-04-25T10:05:29.174650+0000","last_undegraded":"2024-04-25T10:05:29.174650+0000","last_fullsized":"2024-04-25T10:05:29.174650+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:32.438483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.160611+0000","last_change":"2024-04-25T10:05:29.159247+0000","last_active":"2024-04-25T10:05:29.160611+0000","last_peered":"2024-04-25T10:05:29.160611+0000","last_clean":"2024-04-25T10:05:29.160611+0000","last_became_active":"2024-04-25T10:05:29.158949+0000","last_became_peered":"2024-04-25T10:05:29.158949+0000","last_unstale":"2024-04-25T10:05:29.160611+0000","last_undegraded":"2024-04-25T10:05:29.160611+0000","last_fullsized":"2024-04-25T10:05:29.160611+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:51.341343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172388+0000","last_change":"2024-04-25T10:05:28.158539+0000","last_active":"2024-04-25T10:05:29.172388+0000","last_peered":"2024-04-25T10:05:29.172388+0000","last_clean":"2024-04-25T10:05:29.172388+0000","last_became_active":"2024-04-25T10:05:28.158269+0000","last_became_peered":"2024-04-25T10:05:28.158269+0000","last_unstale":"2024-04-25T10:05:29.172388+0000","last_undegraded":"2024-04-25T10:05:29.172388+0000","last_fullsized":"2024-04-25T10:05:29.172388+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:21.033044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144384+0000","last_change":"2024-04-25T10:05:29.168323+0000","last_active":"2024-04-25T10:05:30.144384+0000","last_peered":"2024-04-25T10:05:30.144384+0000","last_clean":"2024-04-25T10:05:30.144384+0000","last_became_active":"2024-04-25T10:05:29.168071+0000","last_became_peered":"2024-04-25T10:05:29.168071+0000","last_unstale":"2024-04-25T10:05:30.144384+0000","last_undegraded":"2024-04-25T10:05:30.144384+0000","last_fullsized":"2024-04-25T10:05:30.144384+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:47.860042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.144731+0000","last_change":"2024-04-25T10:05:28.159751+0000","last_active":"2024-04-25T10:05:29.144731+0000","last_peered":"2024-04-25T10:05:29.144731+0000","last_clean":"2024-04-25T10:05:29.144731+0000","last_became_active":"2024-04-25T10:05:28.159478+0000","last_became_peered":"2024-04-25T10:05:28.159478+0000","last_unstale":"2024-04-25T10:05:29.144731+0000","last_undegraded":"2024-04-25T10:05:29.144731+0000","last_fullsized":"2024-04-25T10:05:29.144731+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:16.567897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.162611+0000","last_change":"2024-04-25T10:05:29.160006+0000","last_active":"2024-04-25T10:05:29.162611+0000","last_peered":"2024-04-25T10:05:29.162611+0000","last_clean":"2024-04-25T10:05:29.162611+0000","last_became_active":"2024-04-25T10:05:29.159699+0000","last_became_peered":"2024-04-25T10:05:29.159699+0000","last_unstale":"2024-04-25T10:05:29.162611+0000","last_undegraded":"2024-04-25T10:05:29.162611+0000","last_fullsized":"2024-04-25T10:05:29.162611+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:25.322276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.148653+0000","last_change":"2024-04-25T10:05:28.155082+0000","last_active":"2024-04-25T10:05:29.148653+0000","last_peered":"2024-04-25T10:05:29.148653+0000","last_clean":"2024-04-25T10:05:29.148653+0000","last_became_active":"2024-04-25T10:05:28.154148+0000","last_became_peered":"2024-04-25T10:05:28.154148+0000","last_unstale":"2024-04-25T10:05:29.148653+0000","last_undegraded":"2024-04-25T10:05:29.148653+0000","last_fullsized":"2024-04-25T10:05:29.148653+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:59.445538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174578+0000","last_change":"2024-04-25T10:05:29.173526+0000","last_active":"2024-04-25T10:05:29.174578+0000","last_peered":"2024-04-25T10:05:29.174578+0000","last_clean":"2024-04-25T10:05:29.174578+0000","last_became_active":"2024-04-25T10:05:29.173255+0000","last_became_peered":"2024-04-25T10:05:29.173255+0000","last_unstale":"2024-04-25T10:05:29.174578+0000","last_undegraded":"2024-04-25T10:05:29.174578+0000","last_fullsized":"2024-04-25T10:05:29.174578+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:55.179751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"170'8","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166934+0000","last_change":"2024-04-25T10:05:28.164222+0000","last_active":"2024-04-25T10:05:29.166934+0000","last_peered":"2024-04-25T10:05:29.166934+0000","last_clean":"2024-04-25T10:05:29.166934+0000","last_became_active":"2024-04-25T10:05:28.163913+0000","last_became_peered":"2024-04-25T10:05:28.163913+0000","last_unstale":"2024-04-25T10:05:29.166934+0000","last_undegraded":"2024-04-25T10:05:29.166934+0000","last_fullsized":"2024-04-25T10:05:29.166934+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.680369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166156+0000","last_change":"2024-04-25T10:05:29.165424+0000","last_active":"2024-04-25T10:05:29.166156+0000","last_peered":"2024-04-25T10:05:29.166156+0000","last_clean":"2024-04-25T10:05:29.166156+0000","last_became_active":"2024-04-25T10:05:29.165145+0000","last_became_peered":"2024-04-25T10:05:29.165145+0000","last_unstale":"2024-04-25T10:05:29.166156+0000","last_undegraded":"2024-04-25T10:05:29.166156+0000","last_fullsized":"2024-04-25T10:05:29.166156+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.089353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.168581+0000","last_change":"2024-04-25T10:05:28.164314+0000","last_active":"2024-04-25T10:05:29.168581+0000","last_peered":"2024-04-25T10:05:29.168581+0000","last_clean":"2024-04-25T10:05:29.168581+0000","last_became_active":"2024-04-25T10:05:28.163980+0000","last_became_peered":"2024-04-25T10:05:28.163980+0000","last_unstale":"2024-04-25T10:05:29.168581+0000","last_undegraded":"2024-04-25T10:05:29.168581+0000","last_fullsized":"2024-04-25T10:05:29.168581+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:16.567897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144766+0000","last_change":"2024-04-25T10:05:29.164386+0000","last_active":"2024-04-25T10:05:30.144766+0000","last_peered":"2024-04-25T10:05:30.144766+0000","last_clean":"2024-04-25T10:05:30.144766+0000","last_became_active":"2024-04-25T10:05:29.164097+0000","last_became_peered":"2024-04-25T10:05:29.164097+0000","last_unstale":"2024-04-25T10:05:30.144766+0000","last_undegraded":"2024-04-25T10:05:30.144766+0000","last_fullsized":"2024-04-25T10:05:30.144766+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.124971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172741+0000","last_change":"2024-04-25T10:05:28.154484+0000","last_active":"2024-04-25T10:05:29.172741+0000","last_peered":"2024-04-25T10:05:29.172741+0000","last_clean":"2024-04-25T10:05:29.172741+0000","last_became_active":"2024-04-25T10:05:28.154200+0000","last_became_peered":"2024-04-25T10:05:28.154200+0000","last_unstale":"2024-04-25T10:05:29.172741+0000","last_undegraded":"2024-04-25T10:05:29.172741+0000","last_fullsized":"2024-04-25T10:05:29.172741+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:14.546367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.164651+0000","last_change":"2024-04-25T10:05:29.163798+0000","last_active":"2024-04-25T10:05:29.164651+0000","last_peered":"2024-04-25T10:05:29.164651+0000","last_clean":"2024-04-25T10:05:29.164651+0000","last_became_active":"2024-04-25T10:05:29.163385+0000","last_became_peered":"2024-04-25T10:05:29.163385+0000","last_unstale":"2024-04-25T10:05:29.164651+0000","last_undegraded":"2024-04-25T10:05:29.164651+0000","last_fullsized":"2024-04-25T10:05:29.164651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:28.347011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172367+0000","last_change":"2024-04-25T10:05:28.159022+0000","last_active":"2024-04-25T10:05:29.172367+0000","last_peered":"2024-04-25T10:05:29.172367+0000","last_clean":"2024-04-25T10:05:29.172367+0000","last_became_active":"2024-04-25T10:05:28.158532+0000","last_became_peered":"2024-04-25T10:05:28.158532+0000","last_unstale":"2024-04-25T10:05:29.172367+0000","last_undegraded":"2024-04-25T10:05:29.172367+0000","last_fullsized":"2024-04-25T10:05:29.172367+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:03.037687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172282+0000","last_change":"2024-04-25T10:05:29.170688+0000","last_active":"2024-04-25T10:05:29.172282+0000","last_peered":"2024-04-25T10:05:29.172282+0000","last_clean":"2024-04-25T10:05:29.172282+0000","last_became_active":"2024-04-25T10:05:29.170406+0000","last_became_peered":"2024-04-25T10:05:29.170406+0000","last_unstale":"2024-04-25T10:05:29.172282+0000","last_undegraded":"2024-04-25T10:05:29.172282+0000","last_fullsized":"2024-04-25T10:05:29.172282+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.392878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149198+0000","last_change":"2024-04-25T10:05:28.157518+0000","last_active":"2024-04-25T10:05:29.149198+0000","last_peered":"2024-04-25T10:05:29.149198+0000","last_clean":"2024-04-25T10:05:29.149198+0000","last_became_active":"2024-04-25T10:05:28.157178+0000","last_became_peered":"2024-04-25T10:05:28.157178+0000","last_unstale":"2024-04-25T10:05:29.149198+0000","last_undegraded":"2024-04-25T10:05:29.149198+0000","last_fullsized":"2024-04-25T10:05:29.149198+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:33.299863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170881+0000","last_change":"2024-04-25T10:05:29.169743+0000","last_active":"2024-04-25T10:05:29.170881+0000","last_peered":"2024-04-25T10:05:29.170881+0000","last_clean":"2024-04-25T10:05:29.170881+0000","last_became_active":"2024-04-25T10:05:29.167474+0000","last_became_peered":"2024-04-25T10:05:29.167474+0000","last_unstale":"2024-04-25T10:05:29.170881+0000","last_undegraded":"2024-04-25T10:05:29.170881+0000","last_fullsized":"2024-04-25T10:05:29.170881+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:11.550963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.171559+0000","last_change":"2024-04-25T10:05:28.146888+0000","last_active":"2024-04-25T10:05:29.171559+0000","last_peered":"2024-04-25T10:05:29.171559+0000","last_clean":"2024-04-25T10:05:29.171559+0000","last_became_active":"2024-04-25T10:05:28.146571+0000","last_became_peered":"2024-04-25T10:05:28.146571+0000","last_unstale":"2024-04-25T10:05:29.171559+0000","last_undegraded":"2024-04-25T10:05:29.171559+0000","last_fullsized":"2024-04-25T10:05:29.171559+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:27.554001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.162668+0000","last_change":"2024-04-25T10:05:29.160016+0000","last_active":"2024-04-25T10:05:29.162668+0000","last_peered":"2024-04-25T10:05:29.162668+0000","last_clean":"2024-04-25T10:05:29.162668+0000","last_became_active":"2024-04-25T10:05:29.159731+0000","last_became_peered":"2024-04-25T10:05:29.159731+0000","last_unstale":"2024-04-25T10:05:29.162668+0000","last_undegraded":"2024-04-25T10:05:29.162668+0000","last_fullsized":"2024-04-25T10:05:29.162668+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:21.767972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.148841+0000","last_change":"2024-04-25T10:05:28.157297+0000","last_active":"2024-04-25T10:05:29.148841+0000","last_peered":"2024-04-25T10:05:29.148841+0000","last_clean":"2024-04-25T10:05:29.148841+0000","last_became_active":"2024-04-25T10:05:28.157004+0000","last_became_peered":"2024-04-25T10:05:28.157004+0000","last_unstale":"2024-04-25T10:05:29.148841+0000","last_undegraded":"2024-04-25T10:05:29.148841+0000","last_fullsized":"2024-04-25T10:05:29.148841+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:26.596777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.153393+0000","last_change":"2024-04-25T10:05:29.151726+0000","last_active":"2024-04-25T10:05:29.153393+0000","last_peered":"2024-04-25T10:05:29.153393+0000","last_clean":"2024-04-25T10:05:29.153393+0000","last_became_active":"2024-04-25T10:05:29.151426+0000","last_became_peered":"2024-04-25T10:05:29.151426+0000","last_unstale":"2024-04-25T10:05:29.153393+0000","last_undegraded":"2024-04-25T10:05:29.153393+0000","last_fullsized":"2024-04-25T10:05:29.153393+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:15.974476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.150402+0000","last_change":"2024-04-25T10:05:28.163435+0000","last_active":"2024-04-25T10:05:29.150402+0000","last_peered":"2024-04-25T10:05:29.150402+0000","last_clean":"2024-04-25T10:05:29.150402+0000","last_became_active":"2024-04-25T10:05:28.163028+0000","last_became_peered":"2024-04-25T10:05:28.163028+0000","last_unstale":"2024-04-25T10:05:29.150402+0000","last_undegraded":"2024-04-25T10:05:29.150402+0000","last_fullsized":"2024-04-25T10:05:29.150402+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:13.969830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144507+0000","last_change":"2024-04-25T10:05:29.157515+0000","last_active":"2024-04-25T10:05:30.144507+0000","last_peered":"2024-04-25T10:05:30.144507+0000","last_clean":"2024-04-25T10:05:30.144507+0000","last_became_active":"2024-04-25T10:05:29.156294+0000","last_became_peered":"2024-04-25T10:05:29.156294+0000","last_unstale":"2024-04-25T10:05:30.144507+0000","last_undegraded":"2024-04-25T10:05:30.144507+0000","last_fullsized":"2024-04-25T10:05:30.144507+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:51.617924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149293+0000","last_change":"2024-04-25T10:05:28.150517+0000","last_active":"2024-04-25T10:05:29.149293+0000","last_peered":"2024-04-25T10:05:29.149293+0000","last_clean":"2024-04-25T10:05:29.149293+0000","last_became_active":"2024-04-25T10:05:28.150236+0000","last_became_peered":"2024-04-25T10:05:28.150236+0000","last_unstale":"2024-04-25T10:05:29.149293+0000","last_undegraded":"2024-04-25T10:05:29.149293+0000","last_fullsized":"2024-04-25T10:05:29.149293+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:49.070208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.165600+0000","last_change":"2024-04-25T10:05:29.164872+0000","last_active":"2024-04-25T10:05:29.165600+0000","last_peered":"2024-04-25T10:05:29.165600+0000","last_clean":"2024-04-25T10:05:29.165600+0000","last_became_active":"2024-04-25T10:05:29.164596+0000","last_became_peered":"2024-04-25T10:05:29.164596+0000","last_unstale":"2024-04-25T10:05:29.165600+0000","last_undegraded":"2024-04-25T10:05:29.165600+0000","last_fullsized":"2024-04-25T10:05:29.165600+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:10.099946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173362+0000","last_change":"2024-04-25T10:05:28.158956+0000","last_active":"2024-04-25T10:05:29.173362+0000","last_peered":"2024-04-25T10:05:29.173362+0000","last_clean":"2024-04-25T10:05:29.173362+0000","last_became_active":"2024-04-25T10:05:28.157449+0000","last_became_peered":"2024-04-25T10:05:28.157449+0000","last_unstale":"2024-04-25T10:05:29.173362+0000","last_undegraded":"2024-04-25T10:05:29.173362+0000","last_fullsized":"2024-04-25T10:05:29.173362+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:41.033606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.154149+0000","last_change":"2024-04-25T10:05:29.152897+0000","last_active":"2024-04-25T10:05:29.154149+0000","last_peered":"2024-04-25T10:05:29.154149+0000","last_clean":"2024-04-25T10:05:29.154149+0000","last_became_active":"2024-04-25T10:05:29.152603+0000","last_became_peered":"2024-04-25T10:05:29.152603+0000","last_unstale":"2024-04-25T10:05:29.154149+0000","last_undegraded":"2024-04-25T10:05:29.154149+0000","last_fullsized":"2024-04-25T10:05:29.154149+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:28.347011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149640+0000","last_change":"2024-04-25T10:05:28.162650+0000","last_active":"2024-04-25T10:05:29.149640+0000","last_peered":"2024-04-25T10:05:29.149640+0000","last_clean":"2024-04-25T10:05:29.149640+0000","last_became_active":"2024-04-25T10:05:28.162365+0000","last_became_peered":"2024-04-25T10:05:28.162365+0000","last_unstale":"2024-04-25T10:05:29.149640+0000","last_undegraded":"2024-04-25T10:05:29.149640+0000","last_fullsized":"2024-04-25T10:05:29.149640+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:27.768777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170674+0000","last_change":"2024-04-25T10:05:29.167420+0000","last_active":"2024-04-25T10:05:29.170674+0000","last_peered":"2024-04-25T10:05:29.170674+0000","last_clean":"2024-04-25T10:05:29.170674+0000","last_became_active":"2024-04-25T10:05:29.167122+0000","last_became_peered":"2024-04-25T10:05:29.167122+0000","last_unstale":"2024-04-25T10:05:29.170674+0000","last_undegraded":"2024-04-25T10:05:29.170674+0000","last_fullsized":"2024-04-25T10:05:29.170674+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:42.808918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144864+0000","last_change":"2024-04-25T10:05:28.160758+0000","last_active":"2024-04-25T10:05:30.144864+0000","last_peered":"2024-04-25T10:05:30.144864+0000","last_clean":"2024-04-25T10:05:30.144864+0000","last_became_active":"2024-04-25T10:05:28.157631+0000","last_became_peered":"2024-04-25T10:05:28.157631+0000","last_unstale":"2024-04-25T10:05:30.144864+0000","last_undegraded":"2024-04-25T10:05:30.144864+0000","last_fullsized":"2024-04-25T10:05:30.144864+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:16.351054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.165583+0000","last_change":"2024-04-25T10:05:29.165025+0000","last_active":"2024-04-25T10:05:29.165583+0000","last_peered":"2024-04-25T10:05:29.165583+0000","last_clean":"2024-04-25T10:05:29.165583+0000","last_became_active":"2024-04-25T10:05:29.164759+0000","last_became_peered":"2024-04-25T10:05:29.164759+0000","last_unstale":"2024-04-25T10:05:29.165583+0000","last_undegraded":"2024-04-25T10:05:29.165583+0000","last_fullsized":"2024-04-25T10:05:29.165583+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.189427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144788+0000","last_change":"2024-04-25T10:05:28.162242+0000","last_active":"2024-04-25T10:05:30.144788+0000","last_peered":"2024-04-25T10:05:30.144788+0000","last_clean":"2024-04-25T10:05:30.144788+0000","last_became_active":"2024-04-25T10:05:28.161956+0000","last_became_peered":"2024-04-25T10:05:28.161956+0000","last_unstale":"2024-04-25T10:05:30.144788+0000","last_undegraded":"2024-04-25T10:05:30.144788+0000","last_fullsized":"2024-04-25T10:05:30.144788+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:20.762278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163705+0000","last_change":"2024-04-25T10:05:29.162783+0000","last_active":"2024-04-25T10:05:29.163705+0000","last_peered":"2024-04-25T10:05:29.163705+0000","last_clean":"2024-04-25T10:05:29.163705+0000","last_became_active":"2024-04-25T10:05:29.162519+0000","last_became_peered":"2024-04-25T10:05:29.162519+0000","last_unstale":"2024-04-25T10:05:29.163705+0000","last_undegraded":"2024-04-25T10:05:29.163705+0000","last_fullsized":"2024-04-25T10:05:29.163705+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:01.268401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.148129+0000","last_change":"2024-04-25T10:05:28.157279+0000","last_active":"2024-04-25T10:05:29.148129+0000","last_peered":"2024-04-25T10:05:29.148129+0000","last_clean":"2024-04-25T10:05:29.148129+0000","last_became_active":"2024-04-25T10:05:28.156947+0000","last_became_peered":"2024-04-25T10:05:28.156947+0000","last_unstale":"2024-04-25T10:05:29.148129+0000","last_undegraded":"2024-04-25T10:05:29.148129+0000","last_fullsized":"2024-04-25T10:05:29.148129+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:49.869162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172853+0000","last_change":"2024-04-25T10:05:29.172036+0000","last_active":"2024-04-25T10:05:29.172853+0000","last_peered":"2024-04-25T10:05:29.172853+0000","last_clean":"2024-04-25T10:05:29.172853+0000","last_became_active":"2024-04-25T10:05:29.171748+0000","last_became_peered":"2024-04-25T10:05:29.171748+0000","last_unstale":"2024-04-25T10:05:29.172853+0000","last_undegraded":"2024-04-25T10:05:29.172853+0000","last_fullsized":"2024-04-25T10:05:29.172853+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:15.355601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.147467+0000","last_change":"2024-04-25T10:05:28.164714+0000","last_active":"2024-04-25T10:05:29.147467+0000","last_peered":"2024-04-25T10:05:29.147467+0000","last_clean":"2024-04-25T10:05:29.147467+0000","last_became_active":"2024-04-25T10:05:28.164365+0000","last_became_peered":"2024-04-25T10:05:28.164365+0000","last_unstale":"2024-04-25T10:05:29.147467+0000","last_undegraded":"2024-04-25T10:05:29.147467+0000","last_fullsized":"2024-04-25T10:05:29.147467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:33.299863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.158588+0000","last_change":"2024-04-25T10:05:29.157545+0000","last_active":"2024-04-25T10:05:29.158588+0000","last_peered":"2024-04-25T10:05:29.158588+0000","last_clean":"2024-04-25T10:05:29.158588+0000","last_became_active":"2024-04-25T10:05:29.157250+0000","last_became_peered":"2024-04-25T10:05:29.157250+0000","last_unstale":"2024-04-25T10:05:29.158588+0000","last_undegraded":"2024-04-25T10:05:29.158588+0000","last_fullsized":"2024-04-25T10:05:29.158588+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:37.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.150552+0000","last_change":"2024-04-25T10:05:28.155301+0000","last_active":"2024-04-25T10:05:29.150552+0000","last_peered":"2024-04-25T10:05:29.150552+0000","last_clean":"2024-04-25T10:05:29.150552+0000","last_became_active":"2024-04-25T10:05:28.155007+0000","last_became_peered":"2024-04-25T10:05:28.155007+0000","last_unstale":"2024-04-25T10:05:29.150552+0000","last_undegraded":"2024-04-25T10:05:29.150552+0000","last_fullsized":"2024-04-25T10:05:29.150552+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:30.608723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174117+0000","last_change":"2024-04-25T10:05:29.173025+0000","last_active":"2024-04-25T10:05:29.174117+0000","last_peered":"2024-04-25T10:05:29.174117+0000","last_clean":"2024-04-25T10:05:29.174117+0000","last_became_active":"2024-04-25T10:05:29.172737+0000","last_became_peered":"2024-04-25T10:05:29.172737+0000","last_unstale":"2024-04-25T10:05:29.174117+0000","last_undegraded":"2024-04-25T10:05:29.174117+0000","last_fullsized":"2024-04-25T10:05:29.174117+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149442+0000","last_change":"2024-04-25T10:05:28.160219+0000","last_active":"2024-04-25T10:05:29.149442+0000","last_peered":"2024-04-25T10:05:29.149442+0000","last_clean":"2024-04-25T10:05:29.149442+0000","last_became_active":"2024-04-25T10:05:28.159864+0000","last_became_peered":"2024-04-25T10:05:28.159864+0000","last_unstale":"2024-04-25T10:05:29.149442+0000","last_undegraded":"2024-04-25T10:05:29.149442+0000","last_fullsized":"2024-04-25T10:05:29.149442+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:55.703642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163733+0000","last_change":"2024-04-25T10:05:29.161688+0000","last_active":"2024-04-25T10:05:29.163733+0000","last_peered":"2024-04-25T10:05:29.163733+0000","last_clean":"2024-04-25T10:05:29.163733+0000","last_became_active":"2024-04-25T10:05:29.160865+0000","last_became_peered":"2024-04-25T10:05:29.160865+0000","last_unstale":"2024-04-25T10:05:29.163733+0000","last_undegraded":"2024-04-25T10:05:29.163733+0000","last_fullsized":"2024-04-25T10:05:29.163733+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:11.400686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.147205+0000","last_change":"2024-04-25T10:05:28.153587+0000","last_active":"2024-04-25T10:05:29.147205+0000","last_peered":"2024-04-25T10:05:29.147205+0000","last_clean":"2024-04-25T10:05:29.147205+0000","last_became_active":"2024-04-25T10:05:28.153076+0000","last_became_peered":"2024-04-25T10:05:28.153076+0000","last_unstale":"2024-04-25T10:05:29.147205+0000","last_undegraded":"2024-04-25T10:05:29.147205+0000","last_fullsized":"2024-04-25T10:05:29.147205+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:57.768217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":446,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149080+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:05:29.149080+0000","last_peered":"2024-04-25T10:05:29.149080+0000","last_clean":"2024-04-25T10:05:29.149080+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:05:29.149080+0000","last_undegraded":"2024-04-25T10:05:29.149080+0000","last_fullsized":"2024-04-25T10:05:29.149080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51904,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319936,"statfs":{"total":96636764160,"available":96583614464,"internally_reserved":0,"allocated":2736128,"data_stored":1592819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2146304,"data_stored":1002521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51340,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320500,"statfs":{"total":96636764160,"available":96584192000,"internally_reserved":0,"allocated":2158592,"data_stored":1004085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2158592,"data_stored":1004097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59660,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312180,"statfs":{"total":96636764160,"available":96575672320,"internally_reserved":0,"allocated":2150400,"data_stored":1002611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55484,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316356,"statfs":{"total":96636764160,"available":96579948544,"internally_reserved":0,"allocated":2146304,"data_stored":1003033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2142208,"data_stored":1002451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56088,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315752,"statfs":{"total":96636764160,"available":96579330048,"internally_reserved":0,"allocated":2752512,"data_stored":1593547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:32.118 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.246+0000 7fd077ecb640 1 Processor -- start 2024-04-25T10:05:32.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.246+0000 7fd077ecb640 1 -- start start 2024-04-25T10:05:32.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd077ecb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd070155fe0 0x7fd07015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd077ecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 0x7fd070071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd077ecb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07006de40 con 0x7fd070072000 2024-04-25T10:05:32.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd077ecb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd070072470 con 0x7fd070155fe0 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd077ecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700ad0e0 con 0x7fd0700adbc0 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd076468640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd070072000 0x7fd07006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55764/0 (socket says 172.21.15.148:55764) 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.247+0000 7fd076468640 1 -- 172.21.15.148:0/186522975 learned_addr learned my addr 172.21.15.148:0/186522975 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771500330 0 0) 0x7fd07006de40 con 0x7fd070072000 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 --2- 172.21.15.148:0/186522975 >> v2:172.21.15.188:3300/0 conn(0x7fd070155fe0 0x7fd07015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075466640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 0x7fd070071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd05c003520 con 0x7fd070072000 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926060792 0 0) 0x7fd05c003520 con 0x7fd070072000 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 msgr2=0x7fd070071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 0x7fd070071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 >> v2:172.21.15.188:3300/0 conn(0x7fd070155fe0 msgr2=0x7fd07015c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 --2- 172.21.15.148:0/186522975 >> v2:172.21.15.188:3300/0 conn(0x7fd070155fe0 0x7fd07015c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0700ad3c0 con 0x7fd070072000 2024-04-25T10:05:32.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fd0680019f0 con 0x7fd070072000 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 >> v1:172.21.15.148:6789/0 conn(0x7fd070072000 legacy=0x7fd07006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd05c0039b0 0x7fd05c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 0x7fd05c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd070072470 con 0x7fd05c0039b0 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700ad0e0 con 0x7fd05c004d40 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c00b9c0 con 0x7fd0700adbc0 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd05c0039b0 0x7fd05c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075466640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 0x7fd05c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd076468640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 -- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 msgr2=0x7fd05c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0700adbc0 0x7fd05c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 -- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd05c004d40 msgr2=0x7fd05c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd075c67640 1 -- 172.21.15.148:0/186522975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c003520 con 0x7fd05c0039b0 2024-04-25T10:05:32.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.248+0000 7fd076468640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:32.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.249+0000 7fd075c67640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd05c0039b0 0x7fd05c003dd0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009af0 tx=0x7fd05c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495246d6cc098f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.249+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c036070 con 0x7fd05c0039b0 2024-04-25T10:05:32.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.249+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c031400 con 0x7fd05c0039b0 2024-04-25T10:05:32.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.249+0000 7fd074c65640 1 -- 172.21.15.148:0/186522975 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c009690 con 0x7fd05c0039b0 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd077ecb640 1 -- 172.21.15.148:0/186522975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd05c0039b0 msgr2=0x7fd05c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd077ecb640 1 --2- 172.21.15.148:0/186522975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd05c0039b0 0x7fd05c003dd0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009af0 tx=0x7fd05c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd076468640 1 -- 172.21.15.148:0/186522975 reap_dead start 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd077ecb640 1 -- 172.21.15.148:0/186522975 shutdown_connections 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd077ecb640 1 -- 172.21.15.148:0/186522975 >> 172.21.15.148:0/186522975 conn(0x7fd0700af670 msgr2=0x7fd0700b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd077ecb640 1 -- 172.21.15.148:0/186522975 shutdown_connections 2024-04-25T10:05:32.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.250+0000 7fd077ecb640 1 -- 172.21.15.148:0/186522975 wait complete. 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.251+0000 7fd077ecb640 1 Processor -- start 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.251+0000 7fd077ecb640 1 -- start start 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd07014c620 0x7fd070099fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 0x7fd07009c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd07009ce50 0x7fd07014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07006f880 con 0x7fd07014cdd0 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0701563c0 con 0x7fd07014c620 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07006fa00 con 0x7fd07009ce50 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 0x7fd07009c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 0x7fd07009c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48576/0 (socket says 172.21.15.148:48576) 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 -- 172.21.15.148:0/1859819215 learned_addr learned my addr 172.21.15.148:0/1859819215 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075c67640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd07014c620 0x7fd070099fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd076468640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd07009ce50 0x7fd07014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 -- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd07009ce50 msgr2=0x7fd07014f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd07009ce50 0x7fd07014f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 -- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd07014c620 msgr2=0x7fd070099fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd07014c620 0x7fd070099fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 -- 172.21.15.148:0/1859819215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07009a4e0 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd075466640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 0x7fd07009c910 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fd0600119a0 tx=0x7fd060011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06000faf0 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06000fc90 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07009a7b0 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060014440 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.252+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07009ad70 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.253+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07009d260 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.254+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd060012040 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.254+0000 7fd0667fc640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd05406b4d0 0x7fd05406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.254+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd06008aa70 con 0x7fd07014cdd0 2024-04-25T10:05:32.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.254+0000 7fd075c67640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd05406b4d0 0x7fd05406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.255+0000 7fd075c67640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd05406b4d0 0x7fd05406d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd06c033000 tx=0x7fd06c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.258+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd07009d260 con 0x7fd07014cdd0 2024-04-25T10:05:32.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.426+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd070073990 con 0x7fd05406b4d0 2024-04-25T10:05:32.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.432+0000 7fd0667fc640 1 -- 172.21.15.148:0/1859819215 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7fd070073990 con 0x7fd05406b4d0 2024-04-25T10:05:32.440 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:32.441 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd05406b4d0 msgr2=0x7fd05406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd05406b4d0 0x7fd05406d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd06c033000 tx=0x7fd06c008030 comp rx=0 tx=0).stop 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 msgr2=0x7fd07009c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 0x7fd07009c910 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fd0600119a0 tx=0x7fd060011e60 comp rx=0 tx=0).stop 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 shutdown_connections 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd05406b4d0 0x7fd05406d990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd07009ce50 0x7fd07014f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07014cdd0 0x7fd07009c910 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 --2- 172.21.15.148:0/1859819215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd07014c620 0x7fd070099fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 >> 172.21.15.148:0/1859819215 conn(0x7fd05c0039b0 msgr2=0x7fd07014daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 shutdown_connections 2024-04-25T10:05:32.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.435+0000 7fd077ecb640 1 -- 172.21.15.148:0/1859819215 wait complete. 2024-04-25T10:05:32.451 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-04-25T10:05:31.158596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440952,"kb_used_data":17960,"kb_used_omap":12,"kb_used_meta":422835,"kb_avail":754533768,"statfs":{"total":773094113280,"available":772642578432,"internally_reserved":0,"allocated":18391040,"data_stored":9205164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":432983600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025625"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.156552+0000","last_change":"2024-04-25T10:05:29.154087+0000","last_active":"2024-04-25T10:05:29.156552+0000","last_peered":"2024-04-25T10:05:29.156552+0000","last_clean":"2024-04-25T10:05:29.156552+0000","last_became_active":"2024-04-25T10:05:29.153599+0000","last_became_peered":"2024-04-25T10:05:29.153599+0000","last_unstale":"2024-04-25T10:05:29.156552+0000","last_undegraded":"2024-04-25T10:05:29.156552+0000","last_fullsized":"2024-04-25T10:05:29.156552+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.164447+0000","last_change":"2024-04-25T10:05:29.158279+0000","last_active":"2024-04-25T10:05:29.164447+0000","last_peered":"2024-04-25T10:05:29.164447+0000","last_clean":"2024-04-25T10:05:29.164447+0000","last_became_active":"2024-04-25T10:05:29.158024+0000","last_became_peered":"2024-04-25T10:05:29.158024+0000","last_unstale":"2024-04-25T10:05:29.164447+0000","last_undegraded":"2024-04-25T10:05:29.164447+0000","last_fullsized":"2024-04-25T10:05:29.164447+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:25.308183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166797+0000","last_change":"2024-04-25T10:05:29.163993+0000","last_active":"2024-04-25T10:05:29.166797+0000","last_peered":"2024-04-25T10:05:29.166797+0000","last_clean":"2024-04-25T10:05:29.166797+0000","last_became_active":"2024-04-25T10:05:29.163652+0000","last_became_peered":"2024-04-25T10:05:29.163652+0000","last_unstale":"2024-04-25T10:05:29.166797+0000","last_undegraded":"2024-04-25T10:05:29.166797+0000","last_fullsized":"2024-04-25T10:05:29.166797+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:37.793621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163799+0000","last_change":"2024-04-25T10:05:29.161112+0000","last_active":"2024-04-25T10:05:29.163799+0000","last_peered":"2024-04-25T10:05:29.163799+0000","last_clean":"2024-04-25T10:05:29.163799+0000","last_became_active":"2024-04-25T10:05:29.160861+0000","last_became_peered":"2024-04-25T10:05:29.160861+0000","last_unstale":"2024-04-25T10:05:29.163799+0000","last_undegraded":"2024-04-25T10:05:29.163799+0000","last_fullsized":"2024-04-25T10:05:29.163799+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:07.408468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144711+0000","last_change":"2024-04-25T10:05:29.157549+0000","last_active":"2024-04-25T10:05:30.144711+0000","last_peered":"2024-04-25T10:05:30.144711+0000","last_clean":"2024-04-25T10:05:30.144711+0000","last_became_active":"2024-04-25T10:05:29.156314+0000","last_became_peered":"2024-04-25T10:05:29.156314+0000","last_unstale":"2024-04-25T10:05:30.144711+0000","last_undegraded":"2024-04-25T10:05:30.144711+0000","last_fullsized":"2024-04-25T10:05:30.144711+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:35.036235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144471+0000","last_change":"2024-04-25T10:05:29.161933+0000","last_active":"2024-04-25T10:05:30.144471+0000","last_peered":"2024-04-25T10:05:30.144471+0000","last_clean":"2024-04-25T10:05:30.144471+0000","last_became_active":"2024-04-25T10:05:29.161530+0000","last_became_peered":"2024-04-25T10:05:29.161530+0000","last_unstale":"2024-04-25T10:05:30.144471+0000","last_undegraded":"2024-04-25T10:05:30.144471+0000","last_fullsized":"2024-04-25T10:05:30.144471+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:50.282879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.160581+0000","last_change":"2024-04-25T10:05:29.159168+0000","last_active":"2024-04-25T10:05:29.160581+0000","last_peered":"2024-04-25T10:05:29.160581+0000","last_clean":"2024-04-25T10:05:29.160581+0000","last_became_active":"2024-04-25T10:05:29.158800+0000","last_became_peered":"2024-04-25T10:05:29.158800+0000","last_unstale":"2024-04-25T10:05:29.160581+0000","last_undegraded":"2024-04-25T10:05:29.160581+0000","last_fullsized":"2024-04-25T10:05:29.160581+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:14.594205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173189+0000","last_change":"2024-04-25T10:05:29.171224+0000","last_active":"2024-04-25T10:05:29.173189+0000","last_peered":"2024-04-25T10:05:29.173189+0000","last_clean":"2024-04-25T10:05:29.173189+0000","last_became_active":"2024-04-25T10:05:29.170912+0000","last_became_peered":"2024-04-25T10:05:29.170912+0000","last_unstale":"2024-04-25T10:05:29.173189+0000","last_undegraded":"2024-04-25T10:05:29.173189+0000","last_fullsized":"2024-04-25T10:05:29.173189+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:12.203089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.155137+0000","last_change":"2024-04-25T10:05:29.153879+0000","last_active":"2024-04-25T10:05:29.155137+0000","last_peered":"2024-04-25T10:05:29.155137+0000","last_clean":"2024-04-25T10:05:29.155137+0000","last_became_active":"2024-04-25T10:05:29.153246+0000","last_became_peered":"2024-04-25T10:05:29.153246+0000","last_unstale":"2024-04-25T10:05:29.155137+0000","last_undegraded":"2024-04-25T10:05:29.155137+0000","last_fullsized":"2024-04-25T10:05:29.155137+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:42.808918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144672+0000","last_change":"2024-04-25T10:05:29.164386+0000","last_active":"2024-04-25T10:05:30.144672+0000","last_peered":"2024-04-25T10:05:30.144672+0000","last_clean":"2024-04-25T10:05:30.144672+0000","last_became_active":"2024-04-25T10:05:29.164124+0000","last_became_peered":"2024-04-25T10:05:29.164124+0000","last_unstale":"2024-04-25T10:05:30.144672+0000","last_undegraded":"2024-04-25T10:05:30.144672+0000","last_fullsized":"2024-04-25T10:05:30.144672+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:46.181959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163143+0000","last_change":"2024-04-25T10:05:29.161316+0000","last_active":"2024-04-25T10:05:29.163143+0000","last_peered":"2024-04-25T10:05:29.163143+0000","last_clean":"2024-04-25T10:05:29.163143+0000","last_became_active":"2024-04-25T10:05:29.161046+0000","last_became_peered":"2024-04-25T10:05:29.161046+0000","last_unstale":"2024-04-25T10:05:29.163143+0000","last_undegraded":"2024-04-25T10:05:29.163143+0000","last_fullsized":"2024-04-25T10:05:29.163143+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:24.010605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157848+0000","last_change":"2024-04-25T10:05:29.156268+0000","last_active":"2024-04-25T10:05:29.157848+0000","last_peered":"2024-04-25T10:05:29.157848+0000","last_clean":"2024-04-25T10:05:29.157848+0000","last_became_active":"2024-04-25T10:05:29.155963+0000","last_became_peered":"2024-04-25T10:05:29.155963+0000","last_unstale":"2024-04-25T10:05:29.157848+0000","last_undegraded":"2024-04-25T10:05:29.157848+0000","last_fullsized":"2024-04-25T10:05:29.157848+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:01.586423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174377+0000","last_change":"2024-04-25T10:05:29.172710+0000","last_active":"2024-04-25T10:05:29.174377+0000","last_peered":"2024-04-25T10:05:29.174377+0000","last_clean":"2024-04-25T10:05:29.174377+0000","last_became_active":"2024-04-25T10:05:29.172441+0000","last_became_peered":"2024-04-25T10:05:29.172441+0000","last_unstale":"2024-04-25T10:05:29.174377+0000","last_undegraded":"2024-04-25T10:05:29.174377+0000","last_fullsized":"2024-04-25T10:05:29.174377+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:35.096707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163352+0000","last_change":"2024-04-25T10:05:29.161614+0000","last_active":"2024-04-25T10:05:29.163352+0000","last_peered":"2024-04-25T10:05:29.163352+0000","last_clean":"2024-04-25T10:05:29.163352+0000","last_became_active":"2024-04-25T10:05:29.161368+0000","last_became_peered":"2024-04-25T10:05:29.161368+0000","last_unstale":"2024-04-25T10:05:29.163352+0000","last_undegraded":"2024-04-25T10:05:29.163352+0000","last_fullsized":"2024-04-25T10:05:29.163352+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:38.674181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166957+0000","last_change":"2024-04-25T10:05:29.163172+0000","last_active":"2024-04-25T10:05:29.166957+0000","last_peered":"2024-04-25T10:05:29.166957+0000","last_clean":"2024-04-25T10:05:29.166957+0000","last_became_active":"2024-04-25T10:05:29.162891+0000","last_became_peered":"2024-04-25T10:05:29.162891+0000","last_unstale":"2024-04-25T10:05:29.166957+0000","last_undegraded":"2024-04-25T10:05:29.166957+0000","last_fullsized":"2024-04-25T10:05:29.166957+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:31.890534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144575+0000","last_change":"2024-04-25T10:05:29.158837+0000","last_active":"2024-04-25T10:05:30.144575+0000","last_peered":"2024-04-25T10:05:30.144575+0000","last_clean":"2024-04-25T10:05:30.144575+0000","last_became_active":"2024-04-25T10:05:29.157286+0000","last_became_peered":"2024-04-25T10:05:29.157286+0000","last_unstale":"2024-04-25T10:05:30.144575+0000","last_undegraded":"2024-04-25T10:05:30.144575+0000","last_fullsized":"2024-04-25T10:05:30.144575+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:59.322410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144783+0000","last_change":"2024-04-25T10:05:29.159522+0000","last_active":"2024-04-25T10:05:30.144783+0000","last_peered":"2024-04-25T10:05:30.144783+0000","last_clean":"2024-04-25T10:05:30.144783+0000","last_became_active":"2024-04-25T10:05:29.156706+0000","last_became_peered":"2024-04-25T10:05:29.156706+0000","last_unstale":"2024-04-25T10:05:30.144783+0000","last_undegraded":"2024-04-25T10:05:30.144783+0000","last_fullsized":"2024-04-25T10:05:30.144783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:00.088588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.167369+0000","last_change":"2024-04-25T10:05:29.165602+0000","last_active":"2024-04-25T10:05:29.167369+0000","last_peered":"2024-04-25T10:05:29.167369+0000","last_clean":"2024-04-25T10:05:29.167369+0000","last_became_active":"2024-04-25T10:05:29.165302+0000","last_became_peered":"2024-04-25T10:05:29.165302+0000","last_unstale":"2024-04-25T10:05:29.167369+0000","last_undegraded":"2024-04-25T10:05:29.167369+0000","last_fullsized":"2024-04-25T10:05:29.167369+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:51.604284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170688+0000","last_change":"2024-04-25T10:05:29.169091+0000","last_active":"2024-04-25T10:05:29.170688+0000","last_peered":"2024-04-25T10:05:29.170688+0000","last_clean":"2024-04-25T10:05:29.170688+0000","last_became_active":"2024-04-25T10:05:29.167208+0000","last_became_peered":"2024-04-25T10:05:29.167208+0000","last_unstale":"2024-04-25T10:05:29.170688+0000","last_undegraded":"2024-04-25T10:05:29.170688+0000","last_fullsized":"2024-04-25T10:05:29.170688+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:12.790345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144530+0000","last_change":"2024-04-25T10:05:29.159795+0000","last_active":"2024-04-25T10:05:30.144530+0000","last_peered":"2024-04-25T10:05:30.144530+0000","last_clean":"2024-04-25T10:05:30.144530+0000","last_became_active":"2024-04-25T10:05:29.159547+0000","last_became_peered":"2024-04-25T10:05:29.159547+0000","last_unstale":"2024-04-25T10:05:30.144530+0000","last_undegraded":"2024-04-25T10:05:30.144530+0000","last_fullsized":"2024-04-25T10:05:30.144530+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:06.640485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.167031+0000","last_change":"2024-04-25T10:05:29.163954+0000","last_active":"2024-04-25T10:05:29.167031+0000","last_peered":"2024-04-25T10:05:29.167031+0000","last_clean":"2024-04-25T10:05:29.167031+0000","last_became_active":"2024-04-25T10:05:29.162312+0000","last_became_peered":"2024-04-25T10:05:29.162312+0000","last_unstale":"2024-04-25T10:05:29.167031+0000","last_undegraded":"2024-04-25T10:05:29.167031+0000","last_fullsized":"2024-04-25T10:05:29.167031+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:25.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157719+0000","last_change":"2024-04-25T10:05:29.156719+0000","last_active":"2024-04-25T10:05:29.157719+0000","last_peered":"2024-04-25T10:05:29.157719+0000","last_clean":"2024-04-25T10:05:29.157719+0000","last_became_active":"2024-04-25T10:05:29.153417+0000","last_became_peered":"2024-04-25T10:05:29.153417+0000","last_unstale":"2024-04-25T10:05:29.157719+0000","last_undegraded":"2024-04-25T10:05:29.157719+0000","last_fullsized":"2024-04-25T10:05:29.157719+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:08.389066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.160835+0000","last_change":"2024-04-25T10:05:29.158313+0000","last_active":"2024-04-25T10:05:29.160835+0000","last_peered":"2024-04-25T10:05:29.160835+0000","last_clean":"2024-04-25T10:05:29.160835+0000","last_became_active":"2024-04-25T10:05:29.158040+0000","last_became_peered":"2024-04-25T10:05:29.158040+0000","last_unstale":"2024-04-25T10:05:29.160835+0000","last_undegraded":"2024-04-25T10:05:29.160835+0000","last_fullsized":"2024-04-25T10:05:29.160835+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:18.686063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157180+0000","last_change":"2024-04-25T10:05:29.153968+0000","last_active":"2024-04-25T10:05:29.157180+0000","last_peered":"2024-04-25T10:05:29.157180+0000","last_clean":"2024-04-25T10:05:29.157180+0000","last_became_active":"2024-04-25T10:05:29.153714+0000","last_became_peered":"2024-04-25T10:05:29.153714+0000","last_unstale":"2024-04-25T10:05:29.157180+0000","last_undegraded":"2024-04-25T10:05:29.157180+0000","last_fullsized":"2024-04-25T10:05:29.157180+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:12.203089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.161049+0000","last_change":"2024-04-25T10:05:29.160521+0000","last_active":"2024-04-25T10:05:29.161049+0000","last_peered":"2024-04-25T10:05:29.161049+0000","last_clean":"2024-04-25T10:05:29.161049+0000","last_became_active":"2024-04-25T10:05:29.160251+0000","last_became_peered":"2024-04-25T10:05:29.160251+0000","last_unstale":"2024-04-25T10:05:29.161049+0000","last_undegraded":"2024-04-25T10:05:29.161049+0000","last_fullsized":"2024-04-25T10:05:29.161049+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:49.457436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166593+0000","last_change":"2024-04-25T10:05:29.165928+0000","last_active":"2024-04-25T10:05:29.166593+0000","last_peered":"2024-04-25T10:05:29.166593+0000","last_clean":"2024-04-25T10:05:29.166593+0000","last_became_active":"2024-04-25T10:05:29.165488+0000","last_became_peered":"2024-04-25T10:05:29.165488+0000","last_unstale":"2024-04-25T10:05:29.166593+0000","last_undegraded":"2024-04-25T10:05:29.166593+0000","last_fullsized":"2024-04-25T10:05:29.166593+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163991+0000","last_change":"2024-04-25T10:05:29.162139+0000","last_active":"2024-04-25T10:05:29.163991+0000","last_peered":"2024-04-25T10:05:29.163991+0000","last_clean":"2024-04-25T10:05:29.163991+0000","last_became_active":"2024-04-25T10:05:29.161845+0000","last_became_peered":"2024-04-25T10:05:29.161845+0000","last_unstale":"2024-04-25T10:05:29.163991+0000","last_undegraded":"2024-04-25T10:05:29.163991+0000","last_fullsized":"2024-04-25T10:05:29.163991+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:28.347011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144670+0000","last_change":"2024-04-25T10:05:29.166217+0000","last_active":"2024-04-25T10:05:30.144670+0000","last_peered":"2024-04-25T10:05:30.144670+0000","last_clean":"2024-04-25T10:05:30.144670+0000","last_became_active":"2024-04-25T10:05:29.165944+0000","last_became_peered":"2024-04-25T10:05:29.165944+0000","last_unstale":"2024-04-25T10:05:30.144670+0000","last_undegraded":"2024-04-25T10:05:30.144670+0000","last_fullsized":"2024-04-25T10:05:30.144670+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:48.900730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.171617+0000","last_change":"2024-04-25T10:05:29.169638+0000","last_active":"2024-04-25T10:05:29.171617+0000","last_peered":"2024-04-25T10:05:29.171617+0000","last_clean":"2024-04-25T10:05:29.171617+0000","last_became_active":"2024-04-25T10:05:29.169346+0000","last_became_peered":"2024-04-25T10:05:29.169346+0000","last_unstale":"2024-04-25T10:05:29.171617+0000","last_undegraded":"2024-04-25T10:05:29.171617+0000","last_fullsized":"2024-04-25T10:05:29.171617+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:28.118560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.165286+0000","last_change":"2024-04-25T10:05:29.164340+0000","last_active":"2024-04-25T10:05:29.165286+0000","last_peered":"2024-04-25T10:05:29.165286+0000","last_clean":"2024-04-25T10:05:29.165286+0000","last_became_active":"2024-04-25T10:05:29.164078+0000","last_became_peered":"2024-04-25T10:05:29.164078+0000","last_unstale":"2024-04-25T10:05:29.165286+0000","last_undegraded":"2024-04-25T10:05:29.165286+0000","last_fullsized":"2024-04-25T10:05:29.165286+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:12.203089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.155368+0000","last_change":"2024-04-25T10:05:29.154076+0000","last_active":"2024-04-25T10:05:29.155368+0000","last_peered":"2024-04-25T10:05:29.155368+0000","last_clean":"2024-04-25T10:05:29.155368+0000","last_became_active":"2024-04-25T10:05:29.153786+0000","last_became_peered":"2024-04-25T10:05:29.153786+0000","last_unstale":"2024-04-25T10:05:29.155368+0000","last_undegraded":"2024-04-25T10:05:29.155368+0000","last_fullsized":"2024-04-25T10:05:29.155368+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:41.163694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144405+0000","last_change":"2024-04-25T10:05:29.160527+0000","last_active":"2024-04-25T10:05:30.144405+0000","last_peered":"2024-04-25T10:05:30.144405+0000","last_clean":"2024-04-25T10:05:30.144405+0000","last_became_active":"2024-04-25T10:05:29.160268+0000","last_became_peered":"2024-04-25T10:05:29.160268+0000","last_unstale":"2024-04-25T10:05:30.144405+0000","last_undegraded":"2024-04-25T10:05:30.144405+0000","last_fullsized":"2024-04-25T10:05:30.144405+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:09.037951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.162174+0000","last_change":"2024-04-25T10:05:29.161075+0000","last_active":"2024-04-25T10:05:29.162174+0000","last_peered":"2024-04-25T10:05:29.162174+0000","last_clean":"2024-04-25T10:05:29.162174+0000","last_became_active":"2024-04-25T10:05:29.160577+0000","last_became_peered":"2024-04-25T10:05:29.160577+0000","last_unstale":"2024-04-25T10:05:29.162174+0000","last_undegraded":"2024-04-25T10:05:29.162174+0000","last_fullsized":"2024-04-25T10:05:29.162174+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:17.573591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"170'4","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172372+0000","last_change":"2024-04-25T10:05:28.154845+0000","last_active":"2024-04-25T10:05:29.172372+0000","last_peered":"2024-04-25T10:05:29.172372+0000","last_clean":"2024-04-25T10:05:29.172372+0000","last_became_active":"2024-04-25T10:05:28.154486+0000","last_became_peered":"2024-04-25T10:05:28.154486+0000","last_unstale":"2024-04-25T10:05:29.172372+0000","last_undegraded":"2024-04-25T10:05:29.172372+0000","last_fullsized":"2024-04-25T10:05:29.172372+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:16.351218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163328+0000","last_change":"2024-04-25T10:05:29.161089+0000","last_active":"2024-04-25T10:05:29.163328+0000","last_peered":"2024-04-25T10:05:29.163328+0000","last_clean":"2024-04-25T10:05:29.163328+0000","last_became_active":"2024-04-25T10:05:29.160837+0000","last_became_peered":"2024-04-25T10:05:29.160837+0000","last_unstale":"2024-04-25T10:05:29.163328+0000","last_undegraded":"2024-04-25T10:05:29.163328+0000","last_fullsized":"2024-04-25T10:05:29.163328+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:32.691718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144802+0000","last_change":"2024-04-25T10:05:28.155931+0000","last_active":"2024-04-25T10:05:30.144802+0000","last_peered":"2024-04-25T10:05:30.144802+0000","last_clean":"2024-04-25T10:05:30.144802+0000","last_became_active":"2024-04-25T10:05:28.155637+0000","last_became_peered":"2024-04-25T10:05:28.155637+0000","last_unstale":"2024-04-25T10:05:30.144802+0000","last_undegraded":"2024-04-25T10:05:30.144802+0000","last_fullsized":"2024-04-25T10:05:30.144802+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:14.968782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.164672+0000","last_change":"2024-04-25T10:05:29.164055+0000","last_active":"2024-04-25T10:05:29.164672+0000","last_peered":"2024-04-25T10:05:29.164672+0000","last_clean":"2024-04-25T10:05:29.164672+0000","last_became_active":"2024-04-25T10:05:29.163427+0000","last_became_peered":"2024-04-25T10:05:29.163427+0000","last_unstale":"2024-04-25T10:05:29.164672+0000","last_undegraded":"2024-04-25T10:05:29.164672+0000","last_fullsized":"2024-04-25T10:05:29.164672+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:42.808918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.168241+0000","last_change":"2024-04-25T10:05:28.156908+0000","last_active":"2024-04-25T10:05:29.168241+0000","last_peered":"2024-04-25T10:05:29.168241+0000","last_clean":"2024-04-25T10:05:29.168241+0000","last_became_active":"2024-04-25T10:05:28.156603+0000","last_became_peered":"2024-04-25T10:05:28.156603+0000","last_unstale":"2024-04-25T10:05:29.168241+0000","last_undegraded":"2024-04-25T10:05:29.168241+0000","last_fullsized":"2024-04-25T10:05:29.168241+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:49.869162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170810+0000","last_change":"2024-04-25T10:05:29.168030+0000","last_active":"2024-04-25T10:05:29.170810+0000","last_peered":"2024-04-25T10:05:29.170810+0000","last_clean":"2024-04-25T10:05:29.170810+0000","last_became_active":"2024-04-25T10:05:29.167757+0000","last_became_peered":"2024-04-25T10:05:29.167757+0000","last_unstale":"2024-04-25T10:05:29.170810+0000","last_undegraded":"2024-04-25T10:05:29.170810+0000","last_fullsized":"2024-04-25T10:05:29.170810+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:53.367980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173190+0000","last_change":"2024-04-25T10:05:28.154255+0000","last_active":"2024-04-25T10:05:29.173190+0000","last_peered":"2024-04-25T10:05:29.173190+0000","last_clean":"2024-04-25T10:05:29.173190+0000","last_became_active":"2024-04-25T10:05:28.153588+0000","last_became_peered":"2024-04-25T10:05:28.153588+0000","last_unstale":"2024-04-25T10:05:29.173190+0000","last_undegraded":"2024-04-25T10:05:29.173190+0000","last_fullsized":"2024-04-25T10:05:29.173190+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:57.146747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.171023+0000","last_change":"2024-04-25T10:05:29.170461+0000","last_active":"2024-04-25T10:05:29.171023+0000","last_peered":"2024-04-25T10:05:29.171023+0000","last_clean":"2024-04-25T10:05:29.171023+0000","last_became_active":"2024-04-25T10:05:29.170162+0000","last_became_peered":"2024-04-25T10:05:29.170162+0000","last_unstale":"2024-04-25T10:05:29.171023+0000","last_undegraded":"2024-04-25T10:05:29.171023+0000","last_fullsized":"2024-04-25T10:05:29.171023+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:05.952651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144885+0000","last_change":"2024-04-25T10:05:28.156281+0000","last_active":"2024-04-25T10:05:30.144885+0000","last_peered":"2024-04-25T10:05:30.144885+0000","last_clean":"2024-04-25T10:05:30.144885+0000","last_became_active":"2024-04-25T10:05:28.155999+0000","last_became_peered":"2024-04-25T10:05:28.155999+0000","last_unstale":"2024-04-25T10:05:30.144885+0000","last_undegraded":"2024-04-25T10:05:30.144885+0000","last_fullsized":"2024-04-25T10:05:30.144885+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:26.596777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.157196+0000","last_change":"2024-04-25T10:05:29.156214+0000","last_active":"2024-04-25T10:05:29.157196+0000","last_peered":"2024-04-25T10:05:29.157196+0000","last_clean":"2024-04-25T10:05:29.157196+0000","last_became_active":"2024-04-25T10:05:29.153121+0000","last_became_peered":"2024-04-25T10:05:29.153121+0000","last_unstale":"2024-04-25T10:05:29.157196+0000","last_undegraded":"2024-04-25T10:05:29.157196+0000","last_fullsized":"2024-04-25T10:05:29.157196+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:28.118560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173766+0000","last_change":"2024-04-25T10:05:28.165373+0000","last_active":"2024-04-25T10:05:29.173766+0000","last_peered":"2024-04-25T10:05:29.173766+0000","last_clean":"2024-04-25T10:05:29.173766+0000","last_became_active":"2024-04-25T10:05:28.165083+0000","last_became_peered":"2024-04-25T10:05:28.165083+0000","last_unstale":"2024-04-25T10:05:29.173766+0000","last_undegraded":"2024-04-25T10:05:29.173766+0000","last_fullsized":"2024-04-25T10:05:29.173766+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:16.351054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.167146+0000","last_change":"2024-04-25T10:05:29.162980+0000","last_active":"2024-04-25T10:05:29.167146+0000","last_peered":"2024-04-25T10:05:29.167146+0000","last_clean":"2024-04-25T10:05:29.167146+0000","last_became_active":"2024-04-25T10:05:29.162665+0000","last_became_peered":"2024-04-25T10:05:29.162665+0000","last_unstale":"2024-04-25T10:05:29.167146+0000","last_undegraded":"2024-04-25T10:05:29.167146+0000","last_fullsized":"2024-04-25T10:05:29.167146+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:13.700571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.159290+0000","last_change":"2024-04-25T10:05:28.155936+0000","last_active":"2024-04-25T10:05:29.159290+0000","last_peered":"2024-04-25T10:05:29.159290+0000","last_clean":"2024-04-25T10:05:29.159290+0000","last_became_active":"2024-04-25T10:05:28.155589+0000","last_became_peered":"2024-04-25T10:05:28.155589+0000","last_unstale":"2024-04-25T10:05:29.159290+0000","last_undegraded":"2024-04-25T10:05:29.159290+0000","last_fullsized":"2024-04-25T10:05:29.159290+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:23.312915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163976+0000","last_change":"2024-04-25T10:05:29.162037+0000","last_active":"2024-04-25T10:05:29.163976+0000","last_peered":"2024-04-25T10:05:29.163976+0000","last_clean":"2024-04-25T10:05:29.163976+0000","last_became_active":"2024-04-25T10:05:29.161445+0000","last_became_peered":"2024-04-25T10:05:29.161445+0000","last_unstale":"2024-04-25T10:05:29.163976+0000","last_undegraded":"2024-04-25T10:05:29.163976+0000","last_fullsized":"2024-04-25T10:05:29.163976+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:24.010605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144719+0000","last_change":"2024-04-25T10:05:28.160663+0000","last_active":"2024-04-25T10:05:30.144719+0000","last_peered":"2024-04-25T10:05:30.144719+0000","last_clean":"2024-04-25T10:05:30.144719+0000","last_became_active":"2024-04-25T10:05:28.157679+0000","last_became_peered":"2024-04-25T10:05:28.157679+0000","last_unstale":"2024-04-25T10:05:30.144719+0000","last_undegraded":"2024-04-25T10:05:30.144719+0000","last_fullsized":"2024-04-25T10:05:30.144719+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:00.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144763+0000","last_change":"2024-04-25T10:05:29.160476+0000","last_active":"2024-04-25T10:05:30.144763+0000","last_peered":"2024-04-25T10:05:30.144763+0000","last_clean":"2024-04-25T10:05:30.144763+0000","last_became_active":"2024-04-25T10:05:29.160070+0000","last_became_peered":"2024-04-25T10:05:29.160070+0000","last_unstale":"2024-04-25T10:05:30.144763+0000","last_undegraded":"2024-04-25T10:05:30.144763+0000","last_fullsized":"2024-04-25T10:05:30.144763+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:05.677474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.144789+0000","last_change":"2024-04-25T10:05:28.165152+0000","last_active":"2024-04-25T10:05:29.144789+0000","last_peered":"2024-04-25T10:05:29.144789+0000","last_clean":"2024-04-25T10:05:29.144789+0000","last_became_active":"2024-04-25T10:05:28.164888+0000","last_became_peered":"2024-04-25T10:05:28.164888+0000","last_unstale":"2024-04-25T10:05:29.144789+0000","last_undegraded":"2024-04-25T10:05:29.144789+0000","last_fullsized":"2024-04-25T10:05:29.144789+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:23.004911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170659+0000","last_change":"2024-04-25T10:05:29.167230+0000","last_active":"2024-04-25T10:05:29.170659+0000","last_peered":"2024-04-25T10:05:29.170659+0000","last_clean":"2024-04-25T10:05:29.170659+0000","last_became_active":"2024-04-25T10:05:29.166188+0000","last_became_peered":"2024-04-25T10:05:29.166188+0000","last_unstale":"2024-04-25T10:05:29.170659+0000","last_undegraded":"2024-04-25T10:05:29.170659+0000","last_fullsized":"2024-04-25T10:05:29.170659+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:07.793081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"170'3","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173771+0000","last_change":"2024-04-25T10:05:28.165707+0000","last_active":"2024-04-25T10:05:29.173771+0000","last_peered":"2024-04-25T10:05:29.173771+0000","last_clean":"2024-04-25T10:05:29.173771+0000","last_became_active":"2024-04-25T10:05:28.165440+0000","last_became_peered":"2024-04-25T10:05:28.165440+0000","last_unstale":"2024-04-25T10:05:29.173771+0000","last_undegraded":"2024-04-25T10:05:29.173771+0000","last_fullsized":"2024-04-25T10:05:29.173771+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:13.986487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.169157+0000","last_change":"2024-04-25T10:05:29.165141+0000","last_active":"2024-04-25T10:05:29.169157+0000","last_peered":"2024-04-25T10:05:29.169157+0000","last_clean":"2024-04-25T10:05:29.169157+0000","last_became_active":"2024-04-25T10:05:29.164890+0000","last_became_peered":"2024-04-25T10:05:29.164890+0000","last_unstale":"2024-04-25T10:05:29.169157+0000","last_undegraded":"2024-04-25T10:05:29.169157+0000","last_fullsized":"2024-04-25T10:05:29.169157+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:37.172203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"170'6","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174650+0000","last_change":"2024-04-25T10:05:28.154486+0000","last_active":"2024-04-25T10:05:29.174650+0000","last_peered":"2024-04-25T10:05:29.174650+0000","last_clean":"2024-04-25T10:05:29.174650+0000","last_became_active":"2024-04-25T10:05:28.153981+0000","last_became_peered":"2024-04-25T10:05:28.153981+0000","last_unstale":"2024-04-25T10:05:29.174650+0000","last_undegraded":"2024-04-25T10:05:29.174650+0000","last_fullsized":"2024-04-25T10:05:29.174650+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:32.438483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.160611+0000","last_change":"2024-04-25T10:05:29.159247+0000","last_active":"2024-04-25T10:05:29.160611+0000","last_peered":"2024-04-25T10:05:29.160611+0000","last_clean":"2024-04-25T10:05:29.160611+0000","last_became_active":"2024-04-25T10:05:29.158949+0000","last_became_peered":"2024-04-25T10:05:29.158949+0000","last_unstale":"2024-04-25T10:05:29.160611+0000","last_undegraded":"2024-04-25T10:05:29.160611+0000","last_fullsized":"2024-04-25T10:05:29.160611+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:51.341343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172388+0000","last_change":"2024-04-25T10:05:28.158539+0000","last_active":"2024-04-25T10:05:29.172388+0000","last_peered":"2024-04-25T10:05:29.172388+0000","last_clean":"2024-04-25T10:05:29.172388+0000","last_became_active":"2024-04-25T10:05:28.158269+0000","last_became_peered":"2024-04-25T10:05:28.158269+0000","last_unstale":"2024-04-25T10:05:29.172388+0000","last_undegraded":"2024-04-25T10:05:29.172388+0000","last_fullsized":"2024-04-25T10:05:29.172388+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:21.033044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144384+0000","last_change":"2024-04-25T10:05:29.168323+0000","last_active":"2024-04-25T10:05:30.144384+0000","last_peered":"2024-04-25T10:05:30.144384+0000","last_clean":"2024-04-25T10:05:30.144384+0000","last_became_active":"2024-04-25T10:05:29.168071+0000","last_became_peered":"2024-04-25T10:05:29.168071+0000","last_unstale":"2024-04-25T10:05:30.144384+0000","last_undegraded":"2024-04-25T10:05:30.144384+0000","last_fullsized":"2024-04-25T10:05:30.144384+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:47.860042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.144731+0000","last_change":"2024-04-25T10:05:28.159751+0000","last_active":"2024-04-25T10:05:29.144731+0000","last_peered":"2024-04-25T10:05:29.144731+0000","last_clean":"2024-04-25T10:05:29.144731+0000","last_became_active":"2024-04-25T10:05:28.159478+0000","last_became_peered":"2024-04-25T10:05:28.159478+0000","last_unstale":"2024-04-25T10:05:29.144731+0000","last_undegraded":"2024-04-25T10:05:29.144731+0000","last_fullsized":"2024-04-25T10:05:29.144731+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:16.567897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.162611+0000","last_change":"2024-04-25T10:05:29.160006+0000","last_active":"2024-04-25T10:05:29.162611+0000","last_peered":"2024-04-25T10:05:29.162611+0000","last_clean":"2024-04-25T10:05:29.162611+0000","last_became_active":"2024-04-25T10:05:29.159699+0000","last_became_peered":"2024-04-25T10:05:29.159699+0000","last_unstale":"2024-04-25T10:05:29.162611+0000","last_undegraded":"2024-04-25T10:05:29.162611+0000","last_fullsized":"2024-04-25T10:05:29.162611+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:25.322276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.148653+0000","last_change":"2024-04-25T10:05:28.155082+0000","last_active":"2024-04-25T10:05:29.148653+0000","last_peered":"2024-04-25T10:05:29.148653+0000","last_clean":"2024-04-25T10:05:29.148653+0000","last_became_active":"2024-04-25T10:05:28.154148+0000","last_became_peered":"2024-04-25T10:05:28.154148+0000","last_unstale":"2024-04-25T10:05:29.148653+0000","last_undegraded":"2024-04-25T10:05:29.148653+0000","last_fullsized":"2024-04-25T10:05:29.148653+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:59.445538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174578+0000","last_change":"2024-04-25T10:05:29.173526+0000","last_active":"2024-04-25T10:05:29.174578+0000","last_peered":"2024-04-25T10:05:29.174578+0000","last_clean":"2024-04-25T10:05:29.174578+0000","last_became_active":"2024-04-25T10:05:29.173255+0000","last_became_peered":"2024-04-25T10:05:29.173255+0000","last_unstale":"2024-04-25T10:05:29.174578+0000","last_undegraded":"2024-04-25T10:05:29.174578+0000","last_fullsized":"2024-04-25T10:05:29.174578+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:55.179751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"170'8","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166934+0000","last_change":"2024-04-25T10:05:28.164222+0000","last_active":"2024-04-25T10:05:29.166934+0000","last_peered":"2024-04-25T10:05:29.166934+0000","last_clean":"2024-04-25T10:05:29.166934+0000","last_became_active":"2024-04-25T10:05:28.163913+0000","last_became_peered":"2024-04-25T10:05:28.163913+0000","last_unstale":"2024-04-25T10:05:29.166934+0000","last_undegraded":"2024-04-25T10:05:29.166934+0000","last_fullsized":"2024-04-25T10:05:29.166934+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:17.680369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.166156+0000","last_change":"2024-04-25T10:05:29.165424+0000","last_active":"2024-04-25T10:05:29.166156+0000","last_peered":"2024-04-25T10:05:29.166156+0000","last_clean":"2024-04-25T10:05:29.166156+0000","last_became_active":"2024-04-25T10:05:29.165145+0000","last_became_peered":"2024-04-25T10:05:29.165145+0000","last_unstale":"2024-04-25T10:05:29.166156+0000","last_undegraded":"2024-04-25T10:05:29.166156+0000","last_fullsized":"2024-04-25T10:05:29.166156+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.089353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.168581+0000","last_change":"2024-04-25T10:05:28.164314+0000","last_active":"2024-04-25T10:05:29.168581+0000","last_peered":"2024-04-25T10:05:29.168581+0000","last_clean":"2024-04-25T10:05:29.168581+0000","last_became_active":"2024-04-25T10:05:28.163980+0000","last_became_peered":"2024-04-25T10:05:28.163980+0000","last_unstale":"2024-04-25T10:05:29.168581+0000","last_undegraded":"2024-04-25T10:05:29.168581+0000","last_fullsized":"2024-04-25T10:05:29.168581+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:16.567897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144766+0000","last_change":"2024-04-25T10:05:29.164386+0000","last_active":"2024-04-25T10:05:30.144766+0000","last_peered":"2024-04-25T10:05:30.144766+0000","last_clean":"2024-04-25T10:05:30.144766+0000","last_became_active":"2024-04-25T10:05:29.164097+0000","last_became_peered":"2024-04-25T10:05:29.164097+0000","last_unstale":"2024-04-25T10:05:30.144766+0000","last_undegraded":"2024-04-25T10:05:30.144766+0000","last_fullsized":"2024-04-25T10:05:30.144766+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:38.124971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172741+0000","last_change":"2024-04-25T10:05:28.154484+0000","last_active":"2024-04-25T10:05:29.172741+0000","last_peered":"2024-04-25T10:05:29.172741+0000","last_clean":"2024-04-25T10:05:29.172741+0000","last_became_active":"2024-04-25T10:05:28.154200+0000","last_became_peered":"2024-04-25T10:05:28.154200+0000","last_unstale":"2024-04-25T10:05:29.172741+0000","last_undegraded":"2024-04-25T10:05:29.172741+0000","last_fullsized":"2024-04-25T10:05:29.172741+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:14.546367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.164651+0000","last_change":"2024-04-25T10:05:29.163798+0000","last_active":"2024-04-25T10:05:29.164651+0000","last_peered":"2024-04-25T10:05:29.164651+0000","last_clean":"2024-04-25T10:05:29.164651+0000","last_became_active":"2024-04-25T10:05:29.163385+0000","last_became_peered":"2024-04-25T10:05:29.163385+0000","last_unstale":"2024-04-25T10:05:29.164651+0000","last_undegraded":"2024-04-25T10:05:29.164651+0000","last_fullsized":"2024-04-25T10:05:29.164651+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:28.347011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172367+0000","last_change":"2024-04-25T10:05:28.159022+0000","last_active":"2024-04-25T10:05:29.172367+0000","last_peered":"2024-04-25T10:05:29.172367+0000","last_clean":"2024-04-25T10:05:29.172367+0000","last_became_active":"2024-04-25T10:05:28.158532+0000","last_became_peered":"2024-04-25T10:05:28.158532+0000","last_unstale":"2024-04-25T10:05:29.172367+0000","last_undegraded":"2024-04-25T10:05:29.172367+0000","last_fullsized":"2024-04-25T10:05:29.172367+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:03.037687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172282+0000","last_change":"2024-04-25T10:05:29.170688+0000","last_active":"2024-04-25T10:05:29.172282+0000","last_peered":"2024-04-25T10:05:29.172282+0000","last_clean":"2024-04-25T10:05:29.172282+0000","last_became_active":"2024-04-25T10:05:29.170406+0000","last_became_peered":"2024-04-25T10:05:29.170406+0000","last_unstale":"2024-04-25T10:05:29.172282+0000","last_undegraded":"2024-04-25T10:05:29.172282+0000","last_fullsized":"2024-04-25T10:05:29.172282+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:10.392878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149198+0000","last_change":"2024-04-25T10:05:28.157518+0000","last_active":"2024-04-25T10:05:29.149198+0000","last_peered":"2024-04-25T10:05:29.149198+0000","last_clean":"2024-04-25T10:05:29.149198+0000","last_became_active":"2024-04-25T10:05:28.157178+0000","last_became_peered":"2024-04-25T10:05:28.157178+0000","last_unstale":"2024-04-25T10:05:29.149198+0000","last_undegraded":"2024-04-25T10:05:29.149198+0000","last_fullsized":"2024-04-25T10:05:29.149198+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:33.299863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170881+0000","last_change":"2024-04-25T10:05:29.169743+0000","last_active":"2024-04-25T10:05:29.170881+0000","last_peered":"2024-04-25T10:05:29.170881+0000","last_clean":"2024-04-25T10:05:29.170881+0000","last_became_active":"2024-04-25T10:05:29.167474+0000","last_became_peered":"2024-04-25T10:05:29.167474+0000","last_unstale":"2024-04-25T10:05:29.170881+0000","last_undegraded":"2024-04-25T10:05:29.170881+0000","last_fullsized":"2024-04-25T10:05:29.170881+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:11.550963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.171559+0000","last_change":"2024-04-25T10:05:28.146888+0000","last_active":"2024-04-25T10:05:29.171559+0000","last_peered":"2024-04-25T10:05:29.171559+0000","last_clean":"2024-04-25T10:05:29.171559+0000","last_became_active":"2024-04-25T10:05:28.146571+0000","last_became_peered":"2024-04-25T10:05:28.146571+0000","last_unstale":"2024-04-25T10:05:29.171559+0000","last_undegraded":"2024-04-25T10:05:29.171559+0000","last_fullsized":"2024-04-25T10:05:29.171559+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:27.554001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.162668+0000","last_change":"2024-04-25T10:05:29.160016+0000","last_active":"2024-04-25T10:05:29.162668+0000","last_peered":"2024-04-25T10:05:29.162668+0000","last_clean":"2024-04-25T10:05:29.162668+0000","last_became_active":"2024-04-25T10:05:29.159731+0000","last_became_peered":"2024-04-25T10:05:29.159731+0000","last_unstale":"2024-04-25T10:05:29.162668+0000","last_undegraded":"2024-04-25T10:05:29.162668+0000","last_fullsized":"2024-04-25T10:05:29.162668+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:21.767972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.148841+0000","last_change":"2024-04-25T10:05:28.157297+0000","last_active":"2024-04-25T10:05:29.148841+0000","last_peered":"2024-04-25T10:05:29.148841+0000","last_clean":"2024-04-25T10:05:29.148841+0000","last_became_active":"2024-04-25T10:05:28.157004+0000","last_became_peered":"2024-04-25T10:05:28.157004+0000","last_unstale":"2024-04-25T10:05:29.148841+0000","last_undegraded":"2024-04-25T10:05:29.148841+0000","last_fullsized":"2024-04-25T10:05:29.148841+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:26.596777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.153393+0000","last_change":"2024-04-25T10:05:29.151726+0000","last_active":"2024-04-25T10:05:29.153393+0000","last_peered":"2024-04-25T10:05:29.153393+0000","last_clean":"2024-04-25T10:05:29.153393+0000","last_became_active":"2024-04-25T10:05:29.151426+0000","last_became_peered":"2024-04-25T10:05:29.151426+0000","last_unstale":"2024-04-25T10:05:29.153393+0000","last_undegraded":"2024-04-25T10:05:29.153393+0000","last_fullsized":"2024-04-25T10:05:29.153393+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:15.974476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.150402+0000","last_change":"2024-04-25T10:05:28.163435+0000","last_active":"2024-04-25T10:05:29.150402+0000","last_peered":"2024-04-25T10:05:29.150402+0000","last_clean":"2024-04-25T10:05:29.150402+0000","last_became_active":"2024-04-25T10:05:28.163028+0000","last_became_peered":"2024-04-25T10:05:28.163028+0000","last_unstale":"2024-04-25T10:05:29.150402+0000","last_undegraded":"2024-04-25T10:05:29.150402+0000","last_fullsized":"2024-04-25T10:05:29.150402+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:13.969830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144507+0000","last_change":"2024-04-25T10:05:29.157515+0000","last_active":"2024-04-25T10:05:30.144507+0000","last_peered":"2024-04-25T10:05:30.144507+0000","last_clean":"2024-04-25T10:05:30.144507+0000","last_became_active":"2024-04-25T10:05:29.156294+0000","last_became_peered":"2024-04-25T10:05:29.156294+0000","last_unstale":"2024-04-25T10:05:30.144507+0000","last_undegraded":"2024-04-25T10:05:30.144507+0000","last_fullsized":"2024-04-25T10:05:30.144507+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:51.617924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149293+0000","last_change":"2024-04-25T10:05:28.150517+0000","last_active":"2024-04-25T10:05:29.149293+0000","last_peered":"2024-04-25T10:05:29.149293+0000","last_clean":"2024-04-25T10:05:29.149293+0000","last_became_active":"2024-04-25T10:05:28.150236+0000","last_became_peered":"2024-04-25T10:05:28.150236+0000","last_unstale":"2024-04-25T10:05:29.149293+0000","last_undegraded":"2024-04-25T10:05:29.149293+0000","last_fullsized":"2024-04-25T10:05:29.149293+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:49.070208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.165600+0000","last_change":"2024-04-25T10:05:29.164872+0000","last_active":"2024-04-25T10:05:29.165600+0000","last_peered":"2024-04-25T10:05:29.165600+0000","last_clean":"2024-04-25T10:05:29.165600+0000","last_became_active":"2024-04-25T10:05:29.164596+0000","last_became_peered":"2024-04-25T10:05:29.164596+0000","last_unstale":"2024-04-25T10:05:29.165600+0000","last_undegraded":"2024-04-25T10:05:29.165600+0000","last_fullsized":"2024-04-25T10:05:29.165600+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:10.099946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.173362+0000","last_change":"2024-04-25T10:05:28.158956+0000","last_active":"2024-04-25T10:05:29.173362+0000","last_peered":"2024-04-25T10:05:29.173362+0000","last_clean":"2024-04-25T10:05:29.173362+0000","last_became_active":"2024-04-25T10:05:28.157449+0000","last_became_peered":"2024-04-25T10:05:28.157449+0000","last_unstale":"2024-04-25T10:05:29.173362+0000","last_undegraded":"2024-04-25T10:05:29.173362+0000","last_fullsized":"2024-04-25T10:05:29.173362+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:41.033606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.154149+0000","last_change":"2024-04-25T10:05:29.152897+0000","last_active":"2024-04-25T10:05:29.154149+0000","last_peered":"2024-04-25T10:05:29.154149+0000","last_clean":"2024-04-25T10:05:29.154149+0000","last_became_active":"2024-04-25T10:05:29.152603+0000","last_became_peered":"2024-04-25T10:05:29.152603+0000","last_unstale":"2024-04-25T10:05:29.154149+0000","last_undegraded":"2024-04-25T10:05:29.154149+0000","last_fullsized":"2024-04-25T10:05:29.154149+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:28.347011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149640+0000","last_change":"2024-04-25T10:05:28.162650+0000","last_active":"2024-04-25T10:05:29.149640+0000","last_peered":"2024-04-25T10:05:29.149640+0000","last_clean":"2024-04-25T10:05:29.149640+0000","last_became_active":"2024-04-25T10:05:28.162365+0000","last_became_peered":"2024-04-25T10:05:28.162365+0000","last_unstale":"2024-04-25T10:05:29.149640+0000","last_undegraded":"2024-04-25T10:05:29.149640+0000","last_fullsized":"2024-04-25T10:05:29.149640+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:27.768777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.170674+0000","last_change":"2024-04-25T10:05:29.167420+0000","last_active":"2024-04-25T10:05:29.170674+0000","last_peered":"2024-04-25T10:05:29.170674+0000","last_clean":"2024-04-25T10:05:29.170674+0000","last_became_active":"2024-04-25T10:05:29.167122+0000","last_became_peered":"2024-04-25T10:05:29.167122+0000","last_unstale":"2024-04-25T10:05:29.170674+0000","last_undegraded":"2024-04-25T10:05:29.170674+0000","last_fullsized":"2024-04-25T10:05:29.170674+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:42.808918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144864+0000","last_change":"2024-04-25T10:05:28.160758+0000","last_active":"2024-04-25T10:05:30.144864+0000","last_peered":"2024-04-25T10:05:30.144864+0000","last_clean":"2024-04-25T10:05:30.144864+0000","last_became_active":"2024-04-25T10:05:28.157631+0000","last_became_peered":"2024-04-25T10:05:28.157631+0000","last_unstale":"2024-04-25T10:05:30.144864+0000","last_undegraded":"2024-04-25T10:05:30.144864+0000","last_fullsized":"2024-04-25T10:05:30.144864+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:16.351054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.165583+0000","last_change":"2024-04-25T10:05:29.165025+0000","last_active":"2024-04-25T10:05:29.165583+0000","last_peered":"2024-04-25T10:05:29.165583+0000","last_clean":"2024-04-25T10:05:29.165583+0000","last_became_active":"2024-04-25T10:05:29.164759+0000","last_became_peered":"2024-04-25T10:05:29.164759+0000","last_unstale":"2024-04-25T10:05:29.165583+0000","last_undegraded":"2024-04-25T10:05:29.165583+0000","last_fullsized":"2024-04-25T10:05:29.165583+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.189427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:05:30.144788+0000","last_change":"2024-04-25T10:05:28.162242+0000","last_active":"2024-04-25T10:05:30.144788+0000","last_peered":"2024-04-25T10:05:30.144788+0000","last_clean":"2024-04-25T10:05:30.144788+0000","last_became_active":"2024-04-25T10:05:28.161956+0000","last_became_peered":"2024-04-25T10:05:28.161956+0000","last_unstale":"2024-04-25T10:05:30.144788+0000","last_undegraded":"2024-04-25T10:05:30.144788+0000","last_fullsized":"2024-04-25T10:05:30.144788+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:20.762278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163705+0000","last_change":"2024-04-25T10:05:29.162783+0000","last_active":"2024-04-25T10:05:29.163705+0000","last_peered":"2024-04-25T10:05:29.163705+0000","last_clean":"2024-04-25T10:05:29.163705+0000","last_became_active":"2024-04-25T10:05:29.162519+0000","last_became_peered":"2024-04-25T10:05:29.162519+0000","last_unstale":"2024-04-25T10:05:29.163705+0000","last_undegraded":"2024-04-25T10:05:29.163705+0000","last_fullsized":"2024-04-25T10:05:29.163705+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:01.268401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.148129+0000","last_change":"2024-04-25T10:05:28.157279+0000","last_active":"2024-04-25T10:05:29.148129+0000","last_peered":"2024-04-25T10:05:29.148129+0000","last_clean":"2024-04-25T10:05:29.148129+0000","last_became_active":"2024-04-25T10:05:28.156947+0000","last_became_peered":"2024-04-25T10:05:28.156947+0000","last_unstale":"2024-04-25T10:05:29.148129+0000","last_undegraded":"2024-04-25T10:05:29.148129+0000","last_fullsized":"2024-04-25T10:05:29.148129+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:49.869162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.172853+0000","last_change":"2024-04-25T10:05:29.172036+0000","last_active":"2024-04-25T10:05:29.172853+0000","last_peered":"2024-04-25T10:05:29.172853+0000","last_clean":"2024-04-25T10:05:29.172853+0000","last_became_active":"2024-04-25T10:05:29.171748+0000","last_became_peered":"2024-04-25T10:05:29.171748+0000","last_unstale":"2024-04-25T10:05:29.172853+0000","last_undegraded":"2024-04-25T10:05:29.172853+0000","last_fullsized":"2024-04-25T10:05:29.172853+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:15.355601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.147467+0000","last_change":"2024-04-25T10:05:28.164714+0000","last_active":"2024-04-25T10:05:29.147467+0000","last_peered":"2024-04-25T10:05:29.147467+0000","last_clean":"2024-04-25T10:05:29.147467+0000","last_became_active":"2024-04-25T10:05:28.164365+0000","last_became_peered":"2024-04-25T10:05:28.164365+0000","last_unstale":"2024-04-25T10:05:29.147467+0000","last_undegraded":"2024-04-25T10:05:29.147467+0000","last_fullsized":"2024-04-25T10:05:29.147467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:33.299863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.158588+0000","last_change":"2024-04-25T10:05:29.157545+0000","last_active":"2024-04-25T10:05:29.158588+0000","last_peered":"2024-04-25T10:05:29.158588+0000","last_clean":"2024-04-25T10:05:29.158588+0000","last_became_active":"2024-04-25T10:05:29.157250+0000","last_became_peered":"2024-04-25T10:05:29.157250+0000","last_unstale":"2024-04-25T10:05:29.158588+0000","last_undegraded":"2024-04-25T10:05:29.158588+0000","last_fullsized":"2024-04-25T10:05:29.158588+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:37.984469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.150552+0000","last_change":"2024-04-25T10:05:28.155301+0000","last_active":"2024-04-25T10:05:29.150552+0000","last_peered":"2024-04-25T10:05:29.150552+0000","last_clean":"2024-04-25T10:05:29.150552+0000","last_became_active":"2024-04-25T10:05:28.155007+0000","last_became_peered":"2024-04-25T10:05:28.155007+0000","last_unstale":"2024-04-25T10:05:29.150552+0000","last_undegraded":"2024-04-25T10:05:29.150552+0000","last_fullsized":"2024-04-25T10:05:29.150552+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:30.608723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.174117+0000","last_change":"2024-04-25T10:05:29.173025+0000","last_active":"2024-04-25T10:05:29.174117+0000","last_peered":"2024-04-25T10:05:29.174117+0000","last_clean":"2024-04-25T10:05:29.174117+0000","last_became_active":"2024-04-25T10:05:29.172737+0000","last_became_peered":"2024-04-25T10:05:29.172737+0000","last_unstale":"2024-04-25T10:05:29.174117+0000","last_undegraded":"2024-04-25T10:05:29.174117+0000","last_fullsized":"2024-04-25T10:05:29.174117+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:07.964660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149442+0000","last_change":"2024-04-25T10:05:28.160219+0000","last_active":"2024-04-25T10:05:29.149442+0000","last_peered":"2024-04-25T10:05:29.149442+0000","last_clean":"2024-04-25T10:05:29.149442+0000","last_became_active":"2024-04-25T10:05:28.159864+0000","last_became_peered":"2024-04-25T10:05:28.159864+0000","last_unstale":"2024-04-25T10:05:29.149442+0000","last_undegraded":"2024-04-25T10:05:29.149442+0000","last_fullsized":"2024-04-25T10:05:29.149442+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:55.703642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.163733+0000","last_change":"2024-04-25T10:05:29.161688+0000","last_active":"2024-04-25T10:05:29.163733+0000","last_peered":"2024-04-25T10:05:29.163733+0000","last_clean":"2024-04-25T10:05:29.163733+0000","last_became_active":"2024-04-25T10:05:29.160865+0000","last_became_peered":"2024-04-25T10:05:29.160865+0000","last_unstale":"2024-04-25T10:05:29.163733+0000","last_undegraded":"2024-04-25T10:05:29.163733+0000","last_fullsized":"2024-04-25T10:05:29.163733+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:28.132652+0000","last_clean_scrub_stamp":"2024-04-25T10:05:28.132652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:11.400686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.147205+0000","last_change":"2024-04-25T10:05:28.153587+0000","last_active":"2024-04-25T10:05:29.147205+0000","last_peered":"2024-04-25T10:05:29.147205+0000","last_clean":"2024-04-25T10:05:29.147205+0000","last_became_active":"2024-04-25T10:05:28.153076+0000","last_became_peered":"2024-04-25T10:05:28.153076+0000","last_unstale":"2024-04-25T10:05:29.147205+0000","last_undegraded":"2024-04-25T10:05:29.147205+0000","last_fullsized":"2024-04-25T10:05:29.147205+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:27.126958+0000","last_clean_scrub_stamp":"2024-04-25T10:05:27.126958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:57.768217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":446,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T10:05:29.149080+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:05:29.149080+0000","last_peered":"2024-04-25T10:05:29.149080+0000","last_clean":"2024-04-25T10:05:29.149080+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:05:29.149080+0000","last_undegraded":"2024-04-25T10:05:29.149080+0000","last_fullsized":"2024-04-25T10:05:29.149080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51904,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319936,"statfs":{"total":96636764160,"available":96583614464,"internally_reserved":0,"allocated":2736128,"data_stored":1592819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2146304,"data_stored":1002521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51340,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320500,"statfs":{"total":96636764160,"available":96584192000,"internally_reserved":0,"allocated":2158592,"data_stored":1004085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2158592,"data_stored":1004097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59660,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312180,"statfs":{"total":96636764160,"available":96575672320,"internally_reserved":0,"allocated":2150400,"data_stored":1002611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55484,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316356,"statfs":{"total":96636764160,"available":96579948544,"internally_reserved":0,"allocated":2146304,"data_stored":1003033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2142208,"data_stored":1002451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56088,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315752,"statfs":{"total":96636764160,"available":96579330048,"internally_reserved":0,"allocated":2752512,"data_stored":1593547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:32.455 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:32.456 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:05:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.591+0000 7f4d11f09640 1 Processor -- start 2024-04-25T10:05:32.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.591+0000 7f4d11f09640 1 -- start start 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d11f09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4d0c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d11f09640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0c0d2420 con 0x7f4d0c0d08e0 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d11f09640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0c0d2700 con 0x7f4d0c15dac0 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d11f09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0d3030 con 0x7f4d0c0d34c0 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4d0c0d08e0 0x7f4d0c0b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55768/0 (socket says 172.21.15.148:55768) 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0bfff640 1 -- 172.21.15.148:0/3879539910 learned_addr learned my addr 172.21.15.148:0/3879539910 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0affd640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4d0c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080544187 0 0) 0x7f4d0c0d2420 con 0x7f4d0c0d08e0 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf4003520 con 0x7f4d0c0d08e0 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355090114 0 0) 0x7f4cf4003520 con 0x7f4d0c0d08e0 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 msgr2=0x7f4d0c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4d0c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 >> v1:172.21.15.188:6789/0 conn(0x7f4d0c15dac0 legacy=0x7f4d0c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0d29e0 con 0x7f4d0c0d08e0 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4d040019f0 con 0x7f4d0c0d08e0 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0affd640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4d0c0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 >> v1:172.21.15.148:6789/0 conn(0x7f4d0c0d08e0 legacy=0x7f4d0c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf40039e0 0x7f4cf4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4004800 0x7f4cf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4cf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0d3030 con 0x7f4cf4004800 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf400bb00 con 0x7f4cf40039e0 2024-04-25T10:05:32.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf400bcd0 con 0x7f4d0c0d34c0 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0affd640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4004800 0x7f4cf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0bfff640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4cf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.592+0000 7f4d0b7fe640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf40039e0 0x7f4cf4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0affd640 1 -- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 msgr2=0x7f4cf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0affd640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c0d34c0 0x7f4cf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0affd640 1 -- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf40039e0 msgr2=0x7f4cf4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0affd640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf40039e0 0x7f4cf4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0affd640 1 -- 172.21.15.148:0/3879539910 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4003520 con 0x7f4cf4004800 2024-04-25T10:05:32.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0b7fe640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf40039e0 0x7f4cf4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0affd640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4004800 0x7f4cf4004c80 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009c50 tx=0x7f4cf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ab9a13dd8c404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00009850 con 0x7f4cf4004800 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00004020 con 0x7f4cf4004800 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.593+0000 7f4d0a7fc640 1 -- 172.21.15.148:0/3879539910 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000043f0 con 0x7f4cf4004800 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d11f09640 1 -- 172.21.15.148:0/3879539910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4004800 msgr2=0x7f4cf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d11f09640 1 --2- 172.21.15.148:0/3879539910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf4004800 0x7f4cf4004c80 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009c50 tx=0x7f4cf4005830 comp rx=0 tx=0).stop 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d0bfff640 1 -- 172.21.15.148:0/3879539910 reap_dead start 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d11f09640 1 -- 172.21.15.148:0/3879539910 shutdown_connections 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d11f09640 1 -- 172.21.15.148:0/3879539910 >> 172.21.15.148:0/3879539910 conn(0x7f4d0c061c30 msgr2=0x7f4d0c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d11f09640 1 -- 172.21.15.148:0/3879539910 shutdown_connections 2024-04-25T10:05:32.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.594+0000 7f4d11f09640 1 -- 172.21.15.148:0/3879539910 wait complete. 2024-04-25T10:05:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.595+0000 7f4d11f09640 1 Processor -- start 2024-04-25T10:05:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.595+0000 7f4d11f09640 1 -- start start 2024-04-25T10:05:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d11f09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf4004800 0x7f4d0c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d11f09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 0x7f4d0c153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d11f09640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c1f4140 0x7f4d0c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d11f09640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c15dea0 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d11f09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0d2100 con 0x7f4cf4004800 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d11f09640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0b97d0 con 0x7f4d0c1f4140 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d0affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 0x7f4d0c153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d0affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 0x7f4d0c153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48600/0 (socket says 172.21.15.148:48600) 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d0affd640 1 -- 172.21.15.148:0/280049652 learned_addr learned my addr 172.21.15.148:0/280049652 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d0b7fe640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf4004800 0x7f4d0c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.596+0000 7f4d0bfff640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c1f4140 0x7f4d0c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d0affd640 1 -- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c1f4140 msgr2=0x7f4d0c1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d0affd640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c1f4140 0x7f4d0c1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d0affd640 1 -- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf4004800 msgr2=0x7f4d0c1f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d0affd640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf4004800 0x7f4d0c1f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d0affd640 1 -- 172.21.15.148:0/280049652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c1560a0 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d0affd640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 0x7f4d0c153e60 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009c50 tx=0x7f4d00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00041070 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00009380 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c156380 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00006450 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.597+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c156940 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.598+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c073a00 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.598+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d00044070 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.599+0000 7f4cf3fff640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cf806b500 0x7f4cf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.599+0000 7f4d0b7fe640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cf806b500 0x7f4cf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.599+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d000ab100 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.599+0000 7f4d0b7fe640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cf806b500 0x7f4cf806d9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc001750 tx=0x7f4cfc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.603+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d00079950 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.839+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4d0c0b3560 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.841+0000 7f4cf3fff640 1 -- 172.21.15.148:0/280049652 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4d0007d9f0 con 0x7f4d0c1f3ce0 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:05:32.846 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:32.847 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cf806b500 msgr2=0x7f4cf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cf806b500 0x7f4cf806d9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc001750 tx=0x7f4cfc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 msgr2=0x7f4d0c153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 0x7f4d0c153e60 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009c50 tx=0x7f4d00004730 comp rx=0 tx=0).stop 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 shutdown_connections 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4cf806b500 0x7f4cf806d9c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d0c1f4140 0x7f4d0c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d0c1f3ce0 0x7f4d0c153e60 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 --2- 172.21.15.148:0/280049652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4cf4004800 0x7f4d0c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 >> 172.21.15.148:0/280049652 conn(0x7f4d0c061c30 msgr2=0x7f4d0c0b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 shutdown_connections 2024-04-25T10:05:32.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.843+0000 7f4d11f09640 1 -- 172.21.15.148:0/280049652 wait complete. 2024-04-25T10:05:32.860 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:32.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.952+0000 7ff06ec4d640 1 Processor -- start 2024-04-25T10:05:32.958 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.952+0000 7ff06ec4d640 1 -- start start 2024-04-25T10:05:32.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06ec4d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff06815dbe0 0x7ff0680d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06ec4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b92a0 0x7ff0680d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06ec4d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0680b97a0 con 0x7ff0680b2cb0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06ec4d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06815dfc0 con 0x7ff06815dbe0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06ec4d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680d1a00 con 0x7ff0680b92a0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06c9e9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff06815dbe0 0x7ff0680d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06c9e9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff06815dbe0 0x7ff0680d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48612/0 (socket says 172.21.15.148:48612) 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06c9e9640 1 -- 172.21.15.148:0/3423622085 learned_addr learned my addr 172.21.15.148:0/3423622085 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.953+0000 7ff06d1ea640 1 --2- 172.21.15.148:0/3423622085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b92a0 0x7ff0680d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.954+0000 7ff06c9e9640 1 -- 172.21.15.148:0/3423622085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b92a0 msgr2=0x7ff0680d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.954+0000 7ff06c9e9640 1 --2- 172.21.15.148:0/3423622085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b92a0 0x7ff0680d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.954+0000 7ff06c9e9640 1 -- 172.21.15.148:0/3423622085 >> v1:172.21.15.188:6789/0 conn(0x7ff0680b2cb0 legacy=0x7ff0680b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.954+0000 7ff06c9e9640 1 -- 172.21.15.148:0/3423622085 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0680d1c50 con 0x7ff06815dbe0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.954+0000 7ff06c9e9640 1 --2- 172.21.15.148:0/3423622085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff06815dbe0 0x7ff0680d0910 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7ff058009870 tx=0x7ff05802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5432d3f0597c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.954+0000 7ff0677fe640 1 -- 172.21.15.148:0/3423622085 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058037070 con 0x7ff06815dbe0 2024-04-25T10:05:32.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff0677fe640 1 -- 172.21.15.148:0/3423622085 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff058032440 con 0x7ff06815dbe0 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff0677fe640 1 -- 172.21.15.148:0/3423622085 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058032810 con 0x7ff06815dbe0 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3423622085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff06815dbe0 msgr2=0x7ff0680d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3423622085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff06815dbe0 0x7ff0680d0910 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7ff058009870 tx=0x7ff05802a270 comp rx=0 tx=0).stop 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3423622085 shutdown_connections 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3423622085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b92a0 0x7ff0680d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3423622085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff06815dbe0 0x7ff0680d0910 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3423622085 >> 172.21.15.148:0/3423622085 conn(0x7ff068061a20 msgr2=0x7ff068061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3423622085 shutdown_connections 2024-04-25T10:05:32.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.955+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3423622085 wait complete. 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.956+0000 7ff06ec4d640 1 Processor -- start 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.956+0000 7ff06ec4d640 1 -- start start 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06ec4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b2cb0 0x7ff0680c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06ec4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 0x7ff06815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06ec4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06815dbe0 0x7ff068160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06ec4d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680b6ab0 con 0x7ff0680b92a0 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06ec4d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680d2210 con 0x7ff06815dbe0 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06ec4d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680b6c30 con 0x7ff0680b2cb0 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff067fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 0x7ff06815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff067fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 0x7ff06815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:48620/0 (socket says 172.21.15.148:48620) 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff067fff640 1 -- 172.21.15.148:0/3700010151 learned_addr learned my addr 172.21.15.148:0/3700010151 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06d1ea640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06815dbe0 0x7ff068160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff06c9e9640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b2cb0 0x7ff0680c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff067fff640 1 -- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b2cb0 msgr2=0x7ff0680c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.957+0000 7ff067fff640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b2cb0 0x7ff0680c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff067fff640 1 -- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06815dbe0 msgr2=0x7ff068160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff067fff640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06815dbe0 0x7ff068160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff067fff640 1 -- 172.21.15.148:0/3700010151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0680cb940 con 0x7ff0680b92a0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff067fff640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 0x7ff06815fcf0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c440 tx=0x7ff05c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0052d0 con 0x7ff0680b92a0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c005470 con 0x7ff0680b92a0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0680cbc20 con 0x7ff0680b92a0 2024-04-25T10:05:32.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c007260 con 0x7ff0680b92a0 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.958+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0680cc1e0 con 0x7ff0680b92a0 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.959+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff068073990 con 0x7ff0680b92a0 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.959+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff05c007400 con 0x7ff0680b92a0 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.960+0000 7ff0657fa640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff05406b4d0 0x7ff05406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.960+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff05c089790 con 0x7ff0680b92a0 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.960+0000 7ff06c9e9640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff05406b4d0 0x7ff05406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.960+0000 7ff06c9e9640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff05406b4d0 0x7ff05406d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff058034000 tx=0x7ff05802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:32.964+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff05c057ec0 con 0x7ff0680b92a0 2024-04-25T10:05:33.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.173+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff06806b080 con 0x7ff0680b92a0 2024-04-25T10:05:33.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.173+0000 7ff0657fa640 1 -- 172.21.15.148:0/3700010151 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7ff05c05bf60 con 0x7ff0680b92a0 2024-04-25T10:05:33.179 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:33.179 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:05:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff05406b4d0 msgr2=0x7ff05406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff05406b4d0 0x7ff05406d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff058034000 tx=0x7ff05802e000 comp rx=0 tx=0).stop 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 msgr2=0x7ff06815fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 0x7ff06815fcf0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c440 tx=0x7ff05c001e70 comp rx=0 tx=0).stop 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 shutdown_connections 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff05406b4d0 0x7ff05406d990 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff06815dbe0 0x7ff068160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0680b92a0 0x7ff06815fcf0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 --2- 172.21.15.148:0/3700010151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0680b2cb0 0x7ff0680c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.175+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 >> 172.21.15.148:0/3700010151 conn(0x7ff068061a20 msgr2=0x7ff0680b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.176+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 shutdown_connections 2024-04-25T10:05:33.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.176+0000 7ff06ec4d640 1 -- 172.21.15.148:0/3700010151 wait complete. 2024-04-25T10:05:33.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:28.494582+0000', 'modified': '2024-04-25T10:05:30.487395+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10845': {'gid': 10845, 'name': 'b', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/906811420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 906811420}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 906811420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:33.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:33.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:05:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.690+0000 7ff07f03f640 1 Processor -- start 2024-04-25T10:05:33.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.690+0000 7ff07f03f640 1 -- start start 2024-04-25T10:05:33.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07f03f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0780d34c0 0x7ff0780d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07f03f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0780d08e0 0x7ff0780b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07f03f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0780d2420 con 0x7ff07815dac0 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07f03f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0780d0d50 con 0x7ff0780d08e0 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07f03f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0780d3030 con 0x7ff0780d34c0 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07d5dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0780d08e0 0x7ff0780b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07d5dc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff0780d08e0 0x7ff0780b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58408/0 (socket says 172.21.15.148:58408) 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07d5dc640 1 -- 172.21.15.148:0/3570595482 learned_addr learned my addr 172.21.15.148:0/3570595482 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff077fff640 1 --2- 172.21.15.148:0/3570595482 >> v2:172.21.15.188:3300/0 conn(0x7ff0780d34c0 0x7ff0780d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07d5dc640 1 -- 172.21.15.148:0/3570595482 >> v2:172.21.15.188:3300/0 conn(0x7ff0780d34c0 msgr2=0x7ff0780d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/3570595482 >> v2:172.21.15.188:3300/0 conn(0x7ff0780d34c0 0x7ff0780d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.691+0000 7ff07d5dc640 1 -- 172.21.15.148:0/3570595482 >> v1:172.21.15.188:6789/0 conn(0x7ff07815dac0 legacy=0x7ff0780d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.692+0000 7ff07d5dc640 1 -- 172.21.15.148:0/3570595482 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0780d2700 con 0x7ff0780d08e0 2024-04-25T10:05:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.692+0000 7ff077fff640 1 --2- 172.21.15.148:0/3570595482 >> v2:172.21.15.188:3300/0 conn(0x7ff0780d34c0 0x7ff0780d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.692+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/3570595482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780b6990 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7ff078161590 tx=0x7ff07002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77565be4affe81da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.692+0000 7ff0777fe640 1 -- 172.21.15.148:0/3570595482 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070036070 con 0x7ff0780d08e0 2024-04-25T10:05:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.692+0000 7ff0777fe640 1 -- 172.21.15.148:0/3570595482 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff070031600 con 0x7ff0780d08e0 2024-04-25T10:05:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.692+0000 7ff0777fe640 1 -- 172.21.15.148:0/3570595482 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700319d0 con 0x7ff0780d08e0 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 -- 172.21.15.148:0/3570595482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 msgr2=0x7ff0780b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 --2- 172.21.15.148:0/3570595482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780b6990 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7ff078161590 tx=0x7ff07002a780 comp rx=0 tx=0).stop 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 -- 172.21.15.148:0/3570595482 shutdown_connections 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 --2- 172.21.15.148:0/3570595482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780b6990 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 --2- 172.21.15.148:0/3570595482 >> v2:172.21.15.188:3300/0 conn(0x7ff0780d34c0 0x7ff0780d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 -- 172.21.15.148:0/3570595482 >> 172.21.15.148:0/3570595482 conn(0x7ff078061c30 msgr2=0x7ff07806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 -- 172.21.15.148:0/3570595482 shutdown_connections 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.693+0000 7ff07f03f640 1 -- 172.21.15.148:0/3570595482 wait complete. 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.694+0000 7ff07f03f640 1 Processor -- start 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.694+0000 7ff07f03f640 1 -- start start 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07f03f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07f03f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0780d34c0 0x7ff0780cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07f03f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07815dac0 0x7ff07814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07f03f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780d2100 con 0x7ff0780d08e0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07f03f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780d2cc0 con 0x7ff07815dac0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07f03f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815dea0 con 0x7ff0780d34c0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58414/0 (socket says 172.21.15.148:58414) 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 -- 172.21.15.148:0/2776871209 learned_addr learned my addr 172.21.15.148:0/2776871209 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07815dac0 0x7ff07814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff077fff640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0780d34c0 0x7ff0780cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 -- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0780d34c0 msgr2=0x7ff0780cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0780d34c0 0x7ff0780cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 -- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07815dac0 msgr2=0x7ff07814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07815dac0 0x7ff07814cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07cddb640 1 -- 172.21.15.148:0/2776871209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07814d8f0 con 0x7ff0780d08e0 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.695+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07815dac0 0x7ff07814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:33.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.696+0000 7ff07cddb640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780caaa0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b420 tx=0x7ff06800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.696+0000 7ff0757fa640 1 -- 172.21.15.148:0/2776871209 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068011830 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.696+0000 7ff0757fa640 1 -- 172.21.15.148:0/2776871209 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068016070 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.696+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07814dbd0 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.696+0000 7ff0757fa640 1 -- 172.21.15.148:0/2776871209 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068011c10 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.696+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07814e190 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.697+0000 7ff0757fa640 1 -- 172.21.15.148:0/2776871209 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff06801b460 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.697+0000 7ff0757fa640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff06406b4d0 0x7ff06406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.698+0000 7ff0757fa640 1 -- 172.21.15.148:0/2776871209 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff068089eb0 con 0x7ff0780d08e0 2024-04-25T10:05:33.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.698+0000 7ff077fff640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff06406b4d0 0x7ff06406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.698+0000 7ff077fff640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff06406b4d0 0x7ff06406d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff06c006fd0 tx=0x7ff06c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.698+0000 7ff081e1d740 1 Processor -- start 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.698+0000 7ff081e1d740 1 -- start start 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f81610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f56f10 0x5628d0f9d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff081e1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d0eb9a10 con 0x5628d0f81230 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d0e62b30 con 0x5628d0f56f10 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628d0eb5c50 con 0x5628d0f9ead0 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f81610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f81610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58424/0 (socket says 172.21.15.148:58424) 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 -- 172.21.15.148:0/2812036689 learned_addr learned my addr 172.21.15.148:0/2812036689 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff077fff640 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f56f10 0x5628d0f9d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 -- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 msgr2=0x5628d0f9f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 -- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f56f10 msgr2=0x5628d0f9d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f56f10 0x5628d0f9d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.699+0000 7ff07cddb640 1 -- 172.21.15.148:0/2812036689 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628d0e77e10 con 0x5628d0f81230 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff07cddb640 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f81610 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7ff068060680 tx=0x7ff068058560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c390ee9db2aae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff05e7fc640 1 -- 172.21.15.148:0/2812036689 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0680567a0 con 0x5628d0f81230 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff05e7fc640 1 -- 172.21.15.148:0/2812036689 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068054b10 con 0x5628d0f81230 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff05e7fc640 1 -- 172.21.15.148:0/2812036689 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06805c390 con 0x5628d0f81230 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 -- 172.21.15.148:0/2812036689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 msgr2=0x5628d0f81610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f81610 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7ff068060680 tx=0x7ff068058560 comp rx=0 tx=0).stop 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 -- 172.21.15.148:0/2812036689 shutdown_connections 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f56f10 0x5628d0f9d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 --2- 172.21.15.148:0/2812036689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f81610 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 -- 172.21.15.148:0/2812036689 >> 172.21.15.148:0/2812036689 conn(0x5628d0f53840 msgr2=0x5628d0f9c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 -- 172.21.15.148:0/2812036689 shutdown_connections 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.700+0000 7ff081e1d740 1 -- 172.21.15.148:0/2812036689 wait complete. 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.701+0000 7ff081e1d740 1 -- start start 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.701+0000 7ff081e1d740 1 Processor -- start 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f84a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f6f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d0f57cb0 con 0x5628d0f81230 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d0e760c0 con 0x5628d0f9ead0 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628d0f51770 con 0x5628d0f6f8a0 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff081e1d740 10 client.0 _ll_register_callbacks cb 0x5628d0e69ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f84a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f84a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58430/0 (socket says 172.21.15.148:58430) 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 -- 172.21.15.148:0/176135792 learned_addr learned my addr 172.21.15.148:0/176135792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff077fff640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f6f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 -- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 msgr2=0x5628d0f6bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 -- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 msgr2=0x5628d0f6f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f6f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 -- 172.21.15.148:0/176135792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628d0bce2d0 con 0x5628d0f81230 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff07cddb640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f84a90 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7ff06805b040 tx=0x7ff06800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d97d6dce0d8eb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff0577fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.702+0000 7ff0577fe640 1 -- 172.21.15.148:0/176135792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0680770b0 con 0x5628d0f81230 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff0577fe640 1 -- 172.21.15.148:0/176135792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff06805c370 con 0x5628d0f81230 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff068054b10 con 0x5628d0f81230 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff0577fe640 1 -- 172.21.15.148:0/176135792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff068081bb0 con 0x5628d0f81230 2024-04-25T10:05:33.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 10 client.10962 fetch_fsmap learned FSMap version 99 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5628d0c986b0 con 0x5628d0f81230 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff0577fe640 1 -- 172.21.15.148:0/176135792 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068054b10 con 0x5628d0f81230 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff0577fe640 1 -- 172.21.15.148:0/176135792 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff0680578f0 con 0x5628d0f81230 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 10 client.10962 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 10 client.10962 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 20 client.10962 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d0fafc50 0x5628d0fb0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 4 client.10962 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628d0f18920 con 0x5628d0fafc50 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.703+0000 7ff077fff640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d0fafc50 0x5628d0fb0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.704+0000 7ff0577fe640 10 client.10962 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.704+0000 7ff077fff640 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d0fafc50 0x5628d0fb0030 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3268d430371a85 server_cookie=295a3b372062f09d in_seq=0 out_seq=0 2024-04-25T10:05:33.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.704+0000 7ff0577fe640 1 -- 172.21.15.148:0/176135792 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628d0f18920 con 0x5628d0fafc50 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.704+0000 7ff0577fe640 10 client.10962 handle_command_reply: tid=0 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.704+0000 7ff081e1d740 1 client.10962 shutdown 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 msgr2=0x5628d0f84a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f84a90 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7ff06805b040 tx=0x7ff06800bf90 comp rx=0 tx=0).stop 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 shutdown_connections 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d0fafc50 0x5628d0fb0030 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f6f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 --2- 172.21.15.148:0/176135792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0f84a90 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff0577fe640 0 client.10962 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 >> 172.21.15.148:0/176135792 conn(0x5628d0f56f10 msgr2=0x5628d0f9f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 shutdown_connections 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 1 -- 172.21.15.148:0/176135792 wait complete. 2024-04-25T10:05:33.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.705+0000 7ff081e1d740 20 client.10962 trim_cache size 0 max 16384 2024-04-25T10:05:33.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.720+0000 7ff081e1d740 1 Processor -- start 2024-04-25T10:05:33.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.720+0000 7ff081e1d740 1 -- start start 2024-04-25T10:05:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 0x5628d0f72180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f81230 0x5628d0fb7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff081e1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0781ff730 con 0x5628d0f6f8a0 2024-04-25T10:05:33.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07814f540 con 0x5628d0f81230 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07814fb40 con 0x5628d0f9ead0 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 0x5628d0f72180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 0x5628d0f72180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58440/0 (socket says 172.21.15.148:58440) 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 -- 172.21.15.148:0/136986820 learned_addr learned my addr 172.21.15.148:0/136986820 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff077fff640 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f81230 0x5628d0fb7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 -- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 msgr2=0x5628d0f9f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 -- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f81230 msgr2=0x5628d0fb7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f81230 0x5628d0fb7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07cddb640 1 -- 172.21.15.148:0/136986820 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628d0f587f0 con 0x5628d0f6f8a0 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.721+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff07cddb640 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 0x5628d0f72180 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7ff068013010 tx=0x7ff06806e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a888abcf276f50e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff05dffb640 1 -- 172.21.15.148:0/136986820 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068077060 con 0x5628d0f6f8a0 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff05dffb640 1 -- 172.21.15.148:0/136986820 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0680578f0 con 0x5628d0f6f8a0 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff05dffb640 1 -- 172.21.15.148:0/136986820 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06805c170 con 0x5628d0f6f8a0 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 -- 172.21.15.148:0/136986820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 msgr2=0x5628d0f72180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 0x5628d0f72180 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7ff068013010 tx=0x7ff06806e040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 -- 172.21.15.148:0/136986820 shutdown_connections 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f81230 0x5628d0fb7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 --2- 172.21.15.148:0/136986820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f6f8a0 0x5628d0f72180 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 -- 172.21.15.148:0/136986820 >> 172.21.15.148:0/136986820 conn(0x5628d0f56f10 msgr2=0x5628d0f572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 -- 172.21.15.148:0/136986820 shutdown_connections 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.722+0000 7ff081e1d740 1 -- 172.21.15.148:0/136986820 wait complete. 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.723+0000 7ff081e1d740 1 -- start start 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.723+0000 7ff081e1d740 1 Processor -- start 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0fb6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d0f57cb0 con 0x5628d0f81230 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628d0f51770 con 0x5628d0f9ead0 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628d0e760c0 con 0x5628d0f6f8a0 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff081e1d740 10 client.0 _ll_register_callbacks cb 0x5628d0f51c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:33.737 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0fb6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0fb6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58454/0 (socket says 172.21.15.148:58454) 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 -- 172.21.15.148:0/3712872718 learned_addr learned my addr 172.21.15.148:0/3712872718 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff07cddb640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 -- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 msgr2=0x5628d0f6ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 -- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 msgr2=0x5628d0f9f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 -- 172.21.15.148:0/3712872718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628d0fb6ff0 con 0x5628d0f81230 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff07d5dc640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff077fff640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0fb6ab0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00fa60 tx=0x7ff06c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5b2489540e725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff0577fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:33.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff0577fe640 1 -- 172.21.15.148:0/3712872718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c01b040 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.724+0000 7ff0577fe640 1 -- 172.21.15.148:0/3712872718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff06c02ec30 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0680578f0 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff0577fe640 1 -- 172.21.15.148:0/3712872718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff06c02ee30 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 10 client.10968 fetch_fsmap learned FSMap version 99 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5628d10077d0 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff0577fe640 1 -- 172.21.15.148:0/3712872718 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0680578f0 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff0577fe640 1 -- 172.21.15.148:0/3712872718 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff06c035050 con 0x5628d0f81230 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 10 client.10968 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 10 client.10968 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 20 client.10968 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:33.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d1007280 0x5628d0f504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 4 client.10968 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5628d0ec6f50 con 0x5628d1007280 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff07cddb640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d1007280 0x5628d0f504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff0577fe640 10 client.10968 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.725+0000 7ff07cddb640 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d1007280 0x5628d0f504a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2278c1b44207c0b0 server_cookie=a18cd5b544141838 in_seq=0 out_seq=0 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff0577fe640 1 -- 172.21.15.148:0/3712872718 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5628d0ec6f50 con 0x5628d1007280 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff0577fe640 10 client.10968 handle_command_reply: tid=0 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 client.10968 shutdown 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 msgr2=0x5628d0fb6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0fb6ab0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00fa60 tx=0x7ff06c004730 comp rx=0 tx=0).stop 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 shutdown_connections 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5628d1007280 0x5628d0f504a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5628d0f9ead0 0x5628d0f9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628d0f81230 0x5628d0fb6ab0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff0577fe640 0 client.10968 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 --2- 172.21.15.148:0/3712872718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5628d0f6f8a0 0x5628d0f6ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 >> 172.21.15.148:0/3712872718 conn(0x5628d0f56f10 msgr2=0x5628d0f9c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 shutdown_connections 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 1 -- 172.21.15.148:0/3712872718 wait complete. 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff081e1d740 20 client.10968 trim_cache size 0 max 16384 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:05:33.741 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 10845, 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.5846862100000001, 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 99, 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 171, 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 171, 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 9.6536324350000005 2024-04-25T10:05:33.742 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff06406b4d0 msgr2=0x7ff06406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff07f03f640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff06406b4d0 0x7ff06406d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff06c006fd0 tx=0x7ff06c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 msgr2=0x7ff0780caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.726+0000 7ff07f03f640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780caaa0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b420 tx=0x7ff06800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 shutdown_connections 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff06406b4d0 0x7ff06406d990 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07815dac0 0x7ff07814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0780d34c0 0x7ff0780cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 --2- 172.21.15.148:0/2776871209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0780d08e0 0x7ff0780caaa0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 >> 172.21.15.148:0/2776871209 conn(0x7ff078061c30 msgr2=0x7ff0780b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 shutdown_connections 2024-04-25T10:05:33.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.727+0000 7ff07f03f640 1 -- 172.21.15.148:0/2776871209 wait complete. 2024-04-25T10:05:33.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 Processor -- start 2024-04-25T10:05:33.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 -- start start 2024-04-25T10:05:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f31e015dc00 0x7f31e00c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e00d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31e00d1c00 0x7f31e00ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e00d2060 con 0x7f31e00d1c00 2024-04-25T10:05:33.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e015dfe0 con 0x7f31e015dc00 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.837+0000 7f31e4cf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e00d00c0 con 0x7f31e00ceb20 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31e00d1c00 0x7f31e00ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f31e00d1c00 0x7f31e00ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58462/0 (socket says 172.21.15.148:58462) 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 -- 172.21.15.148:0/917045669 learned_addr learned my addr 172.21.15.148:0/917045669 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31d7fff640 1 --2- 172.21.15.148:0/917045669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e00d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31de59c640 1 --2- 172.21.15.148:0/917045669 >> v2:172.21.15.188:3300/0 conn(0x7f31e015dc00 0x7f31e00c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 -- 172.21.15.148:0/917045669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 msgr2=0x7f31e00d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 --2- 172.21.15.148:0/917045669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e00d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 -- 172.21.15.148:0/917045669 >> v2:172.21.15.188:3300/0 conn(0x7f31e015dc00 msgr2=0x7f31e00c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 --2- 172.21.15.148:0/917045669 >> v2:172.21.15.188:3300/0 conn(0x7f31e015dc00 0x7f31e00c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31ded9d640 1 -- 172.21.15.148:0/917045669 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e00d03a0 con 0x7f31e00d1c00 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.838+0000 7f31de59c640 1 --2- 172.21.15.148:0/917045669 >> v2:172.21.15.188:3300/0 conn(0x7f31e015dc00 0x7f31e00c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:33.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31ded9d640 1 --2- 172.21.15.148:0/917045669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e00d1c00 0x7f31e00ccc40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f31e006efe0 tx=0x7f31d802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb7d5cc938d2c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31ddd9b640 1 -- 172.21.15.148:0/917045669 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8036070 con 0x7f31e00d1c00 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31ddd9b640 1 -- 172.21.15.148:0/917045669 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8031600 con 0x7f31e00d1c00 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31ddd9b640 1 -- 172.21.15.148:0/917045669 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d80319d0 con 0x7f31e00d1c00 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 -- 172.21.15.148:0/917045669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e00d1c00 msgr2=0x7f31e00ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/917045669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e00d1c00 0x7f31e00ccc40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f31e006efe0 tx=0x7f31d802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 -- 172.21.15.148:0/917045669 shutdown_connections 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/917045669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e00d1c00 0x7f31e00ccc40 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/917045669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e00d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/917045669 >> v2:172.21.15.188:3300/0 conn(0x7f31e015dc00 0x7f31e00c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 -- 172.21.15.148:0/917045669 >> 172.21.15.148:0/917045669 conn(0x7f31e006f5d0 msgr2=0x7f31e006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 -- 172.21.15.148:0/917045669 shutdown_connections 2024-04-25T10:05:33.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.839+0000 7f31e4cf5640 1 -- 172.21.15.148:0/917045669 wait complete. 2024-04-25T10:05:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.841+0000 7f31e4cf5640 1 Processor -- start 2024-04-25T10:05:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.841+0000 7f31e4cf5640 1 -- start start 2024-04-25T10:05:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.841+0000 7f31e4cf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e01f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.841+0000 7f31e4cf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e00d1c00 0x7f31e0142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.841+0000 7f31e4cf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 0x7f31e0144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.841+0000 7f31e4cf5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e00cc130 con 0x7f31e015dc00 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31e4cf5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e00cc720 con 0x7f31e00d1c00 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31e4cf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e00cfd90 con 0x7f31e00ceb20 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 0x7f31e0144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 0x7f31e0144dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58466/0 (socket says 172.21.15.148:58466) 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 -- 172.21.15.148:0/3610122211 learned_addr learned my addr 172.21.15.148:0/3610122211 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31de59c640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e01f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31d7fff640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e00d1c00 0x7f31e0142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 -- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 msgr2=0x7f31e01f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e01f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 -- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e00d1c00 msgr2=0x7f31e0142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e00d1c00 0x7f31e0142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 -- 172.21.15.148:0/3610122211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0145810 con 0x7f31e015dc00 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.842+0000 7f31ded9d640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 0x7f31e0144dc0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f31d8034040 tx=0x7f31d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.844+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8036050 con 0x7f31e015dc00 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.844+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8042070 con 0x7f31e015dc00 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.844+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e0145af0 con 0x7f31e015dc00 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.844+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8009510 con 0x7f31e015dc00 2024-04-25T10:05:33.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.844+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e01f45e0 con 0x7f31e015dc00 2024-04-25T10:05:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.845+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e01f36b0 con 0x7f31e015dc00 2024-04-25T10:05:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.845+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31d8007bd0 con 0x7f31e015dc00 2024-04-25T10:05:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.845+0000 7f31d6ffd640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f31c006b4d0 0x7f31c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.845+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31d80ab150 con 0x7f31e015dc00 2024-04-25T10:05:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.846+0000 7f31de59c640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f31c006b4d0 0x7f31c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.846+0000 7f31de59c640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f31c006b4d0 0x7f31c006d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f31d0006fd0 tx=0x7f31d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:33.850+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31d8079770 con 0x7f31e015dc00 2024-04-25T10:05:34.062 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.057+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31e00c66c0 con 0x7f31e015dc00 2024-04-25T10:05:34.063 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.057+0000 7f31d6ffd640 1 -- 172.21.15.148:0/3610122211 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31d807d810 con 0x7f31e015dc00 2024-04-25T10:05:34.063 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:34.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.059+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f31c006b4d0 msgr2=0x7f31c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.059+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f31c006b4d0 0x7f31c006d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f31d0006fd0 tx=0x7f31d0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:34.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.059+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 msgr2=0x7f31e0144dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.059+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 0x7f31e0144dc0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f31d8034040 tx=0x7f31d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:34.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 shutdown_connections 2024-04-25T10:05:34.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f31c006b4d0 0x7f31c006d990 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e015dc00 0x7f31e0144dc0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e00d1c00 0x7f31e0142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 --2- 172.21.15.148:0/3610122211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31e00ceb20 0x7f31e01f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 >> 172.21.15.148:0/3610122211 conn(0x7f31e006f5d0 msgr2=0x7f31e00cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 shutdown_connections 2024-04-25T10:05:34.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:34.060+0000 7f31e4cf5640 1 -- 172.21.15.148:0/3610122211 wait complete. 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:05:34.076 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:34.077 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:34.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.077 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.092 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42101sec preferred_lft 42101sec 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:34.093 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:05:34.094 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:34.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.095 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:34.156 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:05:34.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.157 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:05:34.214 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:05:34.215 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:34.215 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:34.215 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:34.215 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:34.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.216 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:05:34.216 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:05:34.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:34.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:34.217 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:34.217 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:05:34.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.300 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:05:34.314 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:05:34.314 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:34.314 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:34.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:05:34.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:34.378 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:34.404 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-25T10:05:34.399+0000 7ff67102b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:34.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:34.433 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:34.487 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:05:34.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:34.569 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:34.569 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:34.569 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:05:34.577 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:34.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:34.577 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:05:34.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:05:34.577 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:34.582 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:05:34.583 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:05:34.584 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:05:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:34.584 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:34.585 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:34.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.585 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:34.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.650 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:34.711 INFO:teuthology.orchestra.run.smithi148.stdout:client10977 2024-04-25T10:05:34.712 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:34.712 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:34.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.767 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:05:34.827 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:05:34.827 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:34.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:34.882 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:05:34.973 INFO:teuthology.orchestra.run.smithi148.stderr:6+0 records in 2024-04-25T10:05:34.973 INFO:teuthology.orchestra.run.smithi148.stderr:6+0 records out 2024-04-25T10:05:34.974 INFO:teuthology.orchestra.run.smithi148.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0346887 s, 181 MB/s 2024-04-25T10:05:34.975 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:34.978 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.016 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-25T10:05:35.017 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.018 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:35.019 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:35.020 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-25T10:05:35.021 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.022 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:35.023 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.024 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: mds cache max size = 1000 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = True 2024-04-25T10:05:35.025 INFO:teuthology.orchestra.run.smithi148.stdout: mds log max segments = 1 2024-04-25T10:05:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-04-25T10:05:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = False 2024-04-25T10:05:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:05:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.026 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.026 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-25T10:05:35.027 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:35.028 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-25T10:05:35.029 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.030 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:35.031 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:05:35.032 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-25T10:05:35.033 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:35.034 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-25T10:05:35.035 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-25T10:05:35.036 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: mds cache max size = 1000 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: mds log max segments = 1 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = False 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.037 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.040 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:35.081 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.120 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.121 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.122 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:35.123 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.124 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-04-25T10:05:35.125 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.126 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.127 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:35.128 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:35.129 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-04-25T10:05:35.130 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: mds cache max size = 1000 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = False 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: mds log max segments = 1 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = False 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.131 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.342 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-25T10:05:35.343 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-25T10:05:35.344 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:05:35.345 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-25T10:05:35.346 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-25T10:05:35.347 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.348 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:05:35.349 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-25T10:05:35.350 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-25T10:05:35.351 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: mds cache max size = 1000 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = False 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: mds log max segments = 1 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = False 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:05:35.352 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> import json 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> import sys 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2024-04-25T10:05:35.354 DEBUG:teuthology.orchestra.run.smithi148:> sys.exit(e.errno) 2024-04-25T10:05:35.355 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:05:35.355 DEBUG:teuthology.orchestra.run.smithi148:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:05:35.355 DEBUG:teuthology.orchestra.run.smithi148:> print(json.dumps( 2024-04-25T10:05:35.355 DEBUG:teuthology.orchestra.run.smithi148:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:05:35.355 DEBUG:teuthology.orchestra.run.smithi148:> indent=2)) 2024-04-25T10:05:35.355 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_mode": 33188, 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_ino": 1099511627777, 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_dev": 43, 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_nlink": 1, 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_uid": 1000, 2024-04-25T10:05:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "st_gid": 1316, 2024-04-25T10:05:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "st_size": 6291456, 2024-04-25T10:05:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "st_atime": 1714039534.933581, 2024-04-25T10:05:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "st_mtime": 1714039534.9515808, 2024-04-25T10:05:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "st_ctime": 1714039534.9515808 2024-04-25T10:05:35.423 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:05:35.424 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:35.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:35.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.563+0000 7f7dc03f5640 1 Processor -- start 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.563+0000 7f7dc03f5640 1 -- start start 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dc03f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7db815dbe0 0x7f7db80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dc03f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7db80b92a0 0x7f7db80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dc03f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db80b97a0 con 0x7f7db80b2cb0 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dc03f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db815dfc0 con 0x7f7db80b92a0 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dc03f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db80d1a00 con 0x7f7db815dbe0 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dbe992640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7db80b92a0 0x7f7db80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dbd990640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7db80b2cb0 0x7f7db80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42862/0 (socket says 172.21.15.148:42862) 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.564+0000 7f7dbd990640 1 -- 172.21.15.148:0/4053563930 learned_addr learned my addr 172.21.15.148:0/4053563930 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:35.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe191640 1 --2- 172.21.15.148:0/4053563930 >> v2:172.21.15.188:3300/0 conn(0x7f7db815dbe0 0x7f7db80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe992640 1 -- 172.21.15.148:0/4053563930 >> v2:172.21.15.188:3300/0 conn(0x7f7db815dbe0 msgr2=0x7f7db80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe992640 1 --2- 172.21.15.148:0/4053563930 >> v2:172.21.15.188:3300/0 conn(0x7f7db815dbe0 0x7f7db80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe992640 1 -- 172.21.15.148:0/4053563930 >> v1:172.21.15.148:6789/0 conn(0x7f7db80b2cb0 legacy=0x7f7db80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe992640 1 -- 172.21.15.148:0/4053563930 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80d1c50 con 0x7f7db80b92a0 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe191640 1 --2- 172.21.15.148:0/4053563930 >> v2:172.21.15.188:3300/0 conn(0x7f7db815dbe0 0x7f7db80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.565+0000 7f7dbe992640 1 --2- 172.21.15.148:0/4053563930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db80d0e50 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f7db806c140 tx=0x7f7db002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e1958aa0bcbcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dbd18f640 1 -- 172.21.15.148:0/4053563930 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db0037070 con 0x7f7db80b92a0 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dbd18f640 1 -- 172.21.15.148:0/4053563930 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db0032440 con 0x7f7db80b92a0 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dbd18f640 1 -- 172.21.15.148:0/4053563930 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db002ae60 con 0x7f7db80b92a0 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 -- 172.21.15.148:0/4053563930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 msgr2=0x7f7db80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/4053563930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db80d0e50 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f7db806c140 tx=0x7f7db002a110 comp rx=0 tx=0).stop 2024-04-25T10:05:35.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 -- 172.21.15.148:0/4053563930 shutdown_connections 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/4053563930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db80d0e50 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/4053563930 >> v2:172.21.15.188:3300/0 conn(0x7f7db815dbe0 0x7f7db80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 -- 172.21.15.148:0/4053563930 >> 172.21.15.148:0/4053563930 conn(0x7f7db8061a20 msgr2=0x7f7db8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 -- 172.21.15.148:0/4053563930 shutdown_connections 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.566+0000 7f7dc03f5640 1 -- 172.21.15.148:0/4053563930 wait complete. 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.567+0000 7f7dc03f5640 1 Processor -- start 2024-04-25T10:05:35.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.567+0000 7f7dc03f5640 1 -- start start 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dc03f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7db80b2cb0 0x7f7db8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dc03f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db8155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dc03f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7db815dbe0 0x7f7db8156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dc03f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80b6ab0 con 0x7f7db80b92a0 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dc03f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80d2210 con 0x7f7db80b2cb0 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dc03f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db80b6c30 con 0x7f7db815dbe0 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dbd990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db8155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dbd990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db8155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58476/0 (socket says 172.21.15.148:58476) 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dbd990640 1 -- 172.21.15.148:0/68151005 learned_addr learned my addr 172.21.15.148:0/68151005 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dbe191640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7db80b2cb0 0x7f7db8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.568+0000 7f7dbe992640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7db815dbe0 0x7f7db8156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dbd990640 1 -- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7db815dbe0 msgr2=0x7f7db8156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dbd990640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7db815dbe0 0x7f7db8156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dbd990640 1 -- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7db80b2cb0 msgr2=0x7f7db8159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dbd990640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7db80b2cb0 0x7f7db8159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dbd990640 1 -- 172.21.15.148:0/68151005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80cc1c0 con 0x7f7db80b92a0 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dbd990640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db8155b50 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7da800c470 tx=0x7f7da8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017070 con 0x7f7db80b92a0 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da8005ce0 con 0x7f7db80b92a0 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db80cc4a0 con 0x7f7db80b92a0 2024-04-25T10:05:35.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80072f0 con 0x7f7db80b92a0 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.569+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db80cca60 con 0x7f7db80b92a0 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.570+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db8073990 con 0x7f7db80b92a0 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.570+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7da801a070 con 0x7f7db80b92a0 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.571+0000 7f7daeffd640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.571+0000 7f7dbe191640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.571+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7da8089710 con 0x7f7db80b92a0 2024-04-25T10:05:35.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.571+0000 7f7dbe191640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7db4007510 tx=0x7f7db40074a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.575+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7da8058490 con 0x7f7db80b92a0 2024-04-25T10:05:35.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.779+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7db806c6d0 con 0x7f7db80b92a0 2024-04-25T10:05:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.779+0000 7f7daeffd640 1 -- 172.21.15.148:0/68151005 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f7da8057d90 con 0x7f7db80b92a0 2024-04-25T10:05:35.785 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:05:35.785 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:05:35.785 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d9c06b4d0 msgr2=0x7f7d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7db4007510 tx=0x7f7db40074a0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 msgr2=0x7f7db8155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db8155b50 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f7da800c470 tx=0x7f7da8001e70 comp rx=0 tx=0).stop 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 shutdown_connections 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d9c06b4d0 0x7f7d9c06d990 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7db815dbe0 0x7f7db8156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db80b92a0 0x7f7db8155b50 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 --2- 172.21.15.148:0/68151005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7db80b2cb0 0x7f7db8159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 >> 172.21.15.148:0/68151005 conn(0x7f7db8061a20 msgr2=0x7f7db80b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 shutdown_connections 2024-04-25T10:05:35.788 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.781+0000 7f7dc03f5640 1 -- 172.21.15.148:0/68151005 wait complete. 2024-04-25T10:05:35.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:05:35.921 INFO:teuthology.orchestra.run.smithi188.stdout:{"message":"","return_code":0} 2024-04-25T10:05:35.921 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:05:35.922 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T10:05:35.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 Processor -- start 2024-04-25T10:05:35.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 -- start start 2024-04-25T10:05:35.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264f74d40 0x563264ff20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 0x5632650017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264fff010 0x563264fff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563264f59480 con 0x563264ffec30 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563264f591e0 con 0x563264f74d40 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.960+0000 7fea80914f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563264f59720 con 0x563264fff010 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 0x5632650017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 0x5632650017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58496/0 (socket says 172.21.15.148:58496) 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 -- 172.21.15.148:0/4189030409 learned_addr learned my addr 172.21.15.148:0/4189030409 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7e415640 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264f74d40 0x563264ff20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7ec16640 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264fff010 0x563264fff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 -- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264fff010 msgr2=0x563264fff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264fff010 0x563264fff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 -- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264f74d40 msgr2=0x563264ff20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264f74d40 0x563264ff20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 -- 172.21.15.148:0/4189030409 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563264e6bde0 con 0x563264ffec30 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7e415640 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264f74d40 0x563264ff20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7dc14640 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 0x5632650017b0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fea74009600 tx=0x7fea74009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8498415abd768736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7cc12640 1 -- 172.21.15.148:0/4189030409 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74017070 con 0x563264ffec30 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7cc12640 1 -- 172.21.15.148:0/4189030409 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740066b0 con 0x563264ffec30 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea7cc12640 1 -- 172.21.15.148:0/4189030409 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74006a60 con 0x563264ffec30 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea80914f40 1 -- 172.21.15.148:0/4189030409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 msgr2=0x5632650017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea80914f40 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 0x5632650017b0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fea74009600 tx=0x7fea74009ac0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.961+0000 7fea80914f40 1 -- 172.21.15.148:0/4189030409 shutdown_connections 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264fff010 0x563264fff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264ffec30 0x5632650017b0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 --2- 172.21.15.148:0/4189030409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264f74d40 0x563264ff20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 -- 172.21.15.148:0/4189030409 >> 172.21.15.148:0/4189030409 conn(0x563264f299b0 msgr2=0x563264f28690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 -- 172.21.15.148:0/4189030409 shutdown_connections 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 -- 172.21.15.148:0/4189030409 wait complete. 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 Processor -- start 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.962+0000 7fea80914f40 1 -- start start 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea80914f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264f74d40 0x563264ff05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea80914f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264ffec30 0x563264fff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea80914f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264fff010 0x563264ff5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea80914f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x563265001320 con 0x563264f74d40 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea80914f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563264ff5210 con 0x563264fff010 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea80914f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563265000d10 con 0x563264ffec30 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264f74d40 0x563264ff05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264f74d40 0x563264ff05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58508/0 (socket says 172.21.15.148:58508) 2024-04-25T10:05:35.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 -- 172.21.15.148:0/1787553023 learned_addr learned my addr 172.21.15.148:0/1787553023 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7ec16640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264fff010 0x563264ff5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7dc14640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264ffec30 0x563264fff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 -- 172.21.15.148:0/1787553023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264ffec30 msgr2=0x563264fff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563264ffec30 0x563264fff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 -- 172.21.15.148:0/1787553023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264fff010 msgr2=0x563264ff5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563264fff010 0x563264ff5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 -- 172.21.15.148:0/1787553023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563264e57c50 con 0x563264f74d40 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.963+0000 7fea7e415640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264f74d40 0x563264ff05c0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fea70048250 tx=0x7fea70003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.964+0000 7fea6f7fe640 1 -- 172.21.15.148:0/1787553023 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70067070 con 0x563264f74d40 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.964+0000 7fea6f7fe640 1 -- 172.21.15.148:0/1787553023 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70057550 con 0x563264f74d40 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.964+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563264ff1310 con 0x563264f74d40 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.964+0000 7fea6f7fe640 1 -- 172.21.15.148:0/1787553023 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70057900 con 0x563264f74d40 2024-04-25T10:05:35.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.964+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563264ff0c50 con 0x563264f74d40 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.965+0000 7fea6f7fe640 1 -- 172.21.15.148:0/1787553023 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea7006a070 con 0x563264f74d40 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.965+0000 7fea6f7fe640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea60069ee0 0x7fea6006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.965+0000 7fea6f7fe640 1 -- 172.21.15.148:0/1787553023 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fea700cfdf0 con 0x563264f74d40 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.965+0000 7fea7dc14640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea60069ee0 0x7fea6006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.966+0000 7fea80914f40 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x563264fffb20 0x563264f745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.966+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 --> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171) v9 -- 0x563264ff5b80 con 0x563264fffb20 2024-04-25T10:05:35.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.966+0000 7fea7ec16640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x563264fffb20 0x563264f745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.966+0000 7fea7dc14640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea60069ee0 0x7fea6006c3a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x563264ffdd30 tx=0x7fea7400f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.966+0000 7fea7ec16640 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x563264fffb20 0x563264f745e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea7ec16640 1 -- 172.21.15.148:0/1787553023 <== osd.2 v2:172.21.15.148:6808/1311261585 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v171'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fea78007790 con 0x563264fffb20 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x563264fffb20 msgr2=0x563264f745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x563264fffb20 0x563264f745e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea60069ee0 msgr2=0x7fea6006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fea60069ee0 0x7fea6006c3a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x563264ffdd30 tx=0x7fea7400f000 comp rx=0 tx=0).stop 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264f74d40 msgr2=0x563264ff05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 --2- 172.21.15.148:0/1787553023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563264f74d40 0x563264ff05c0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fea70048250 tx=0x7fea70003330 comp rx=0 tx=0).stop 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea7ec16640 1 -- 172.21.15.148:0/1787553023 reap_dead start 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 shutdown_connections 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 >> 172.21.15.148:0/1787553023 conn(0x563264f299b0 msgr2=0x563264f2d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 shutdown_connections 2024-04-25T10:05:35.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:35.967+0000 7fea80914f40 1 -- 172.21.15.148:0/1787553023 wait complete. 2024-04-25T10:05:35.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:35.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start / recursive,repair 2024-04-25T10:05:36.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.116+0000 7f0e4925d640 1 Processor -- start 2024-04-25T10:05:36.122 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.116+0000 7f0e4925d640 1 -- start start 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4925d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e441446d0 0x7f0e440b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4925d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e4406e8a0 0x7f0e440b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4925d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e440cccc0 con 0x7f0e44144d50 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4925d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e440b2b20 con 0x7f0e4406e8a0 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4925d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4406d400 con 0x7f0e441446d0 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e4406e8a0 0x7f0e440b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4359e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f0e4406e8a0 0x7f0e440b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58510/0 (socket says 172.21.15.148:58510) 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e4359e640 1 -- 172.21.15.148:0/1348990876 learned_addr learned my addr 172.21.15.148:0/1348990876 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.117+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1348990876 >> v2:172.21.15.188:3300/0 conn(0x7f0e441446d0 0x7f0e440b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e4359e640 1 -- 172.21.15.148:0/1348990876 >> v2:172.21.15.188:3300/0 conn(0x7f0e441446d0 msgr2=0x7f0e440b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1348990876 >> v2:172.21.15.188:3300/0 conn(0x7f0e441446d0 0x7f0e440b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e4359e640 1 -- 172.21.15.148:0/1348990876 >> v1:172.21.15.188:6789/0 conn(0x7f0e44144d50 legacy=0x7f0e440cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e4359e640 1 -- 172.21.15.148:0/1348990876 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440cc140 con 0x7f0e4406e8a0 2024-04-25T10:05:36.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1348990876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e4406e8a0 0x7f0e440b4460 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f0e440cf560 tx=0x7f0e3402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e779ba5e33a472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e41d9b640 1 -- 172.21.15.148:0/1348990876 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34036070 con 0x7f0e4406e8a0 2024-04-25T10:05:36.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e41d9b640 1 -- 172.21.15.148:0/1348990876 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e34031600 con 0x7f0e4406e8a0 2024-04-25T10:05:36.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.118+0000 7f0e41d9b640 1 -- 172.21.15.148:0/1348990876 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340319d0 con 0x7f0e4406e8a0 2024-04-25T10:05:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 -- 172.21.15.148:0/1348990876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e4406e8a0 msgr2=0x7f0e440b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 --2- 172.21.15.148:0/1348990876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e4406e8a0 0x7f0e440b4460 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f0e440cf560 tx=0x7f0e3402a730 comp rx=0 tx=0).stop 2024-04-25T10:05:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 -- 172.21.15.148:0/1348990876 shutdown_connections 2024-04-25T10:05:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 --2- 172.21.15.148:0/1348990876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e4406e8a0 0x7f0e440b4460 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 --2- 172.21.15.148:0/1348990876 >> v2:172.21.15.188:3300/0 conn(0x7f0e441446d0 0x7f0e440b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 -- 172.21.15.148:0/1348990876 >> 172.21.15.148:0/1348990876 conn(0x7f0e4406abe0 msgr2=0x7f0e4406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 -- 172.21.15.148:0/1348990876 shutdown_connections 2024-04-25T10:05:36.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.119+0000 7f0e4925d640 1 -- 172.21.15.148:0/1348990876 wait complete. 2024-04-25T10:05:36.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.120+0000 7f0e4925d640 1 Processor -- start 2024-04-25T10:05:36.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 -- start start 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4406e8a0 0x7f0e4415b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e441446d0 0x7f0e441526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 0x7f0e440a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440cc700 con 0x7f0e44144d50 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b49a0 con 0x7f0e4406e8a0 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4925d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440cc9e0 con 0x7f0e441446d0 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 0x7f0e440a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 0x7f0e440a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58516/0 (socket says 172.21.15.148:58516) 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.121+0000 7f0e4359e640 1 -- 172.21.15.148:0/4122450499 learned_addr learned my addr 172.21.15.148:0/4122450499 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4259c640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e441446d0 0x7f0e441526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4406e8a0 0x7f0e4415b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4359e640 1 -- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e441446d0 msgr2=0x7f0e441526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4359e640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e441446d0 0x7f0e441526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4359e640 1 -- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4406e8a0 msgr2=0x7f0e4415b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4359e640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4406e8a0 0x7f0e4415b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4359e640 1 -- 172.21.15.148:0/4122450499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440aa130 con 0x7f0e44144d50 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4406e8a0 0x7f0e4415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4359e640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 0x7f0e440a9bf0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009f40 tx=0x7f0e34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/4122450499 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34036050 con 0x7f0e44144d50 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/4122450499 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e34042070 con 0x7f0e44144d50 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e440aa3b0 con 0x7f0e44144d50 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/4122450499 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34009510 con 0x7f0e44144d50 2024-04-25T10:05:36.131 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.122+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e440aa970 con 0x7f0e44144d50 2024-04-25T10:05:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.123+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/4122450499 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e34007bf0 con 0x7f0e44144d50 2024-04-25T10:05:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.124+0000 7f0e2b7fe640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e3006b4d0 0x7f0e3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.124+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/4122450499 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e340ab150 con 0x7f0e44144d50 2024-04-25T10:05:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.124+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e3006b4d0 0x7f0e3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.124+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e3006b4d0 0x7f0e3006d990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009ac0 tx=0x7f0e3c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.125+0000 7f0e4bbfd740 1 Processor -- start 2024-04-25T10:05:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.125+0000 7f0e4bbfd740 1 -- start start 2024-04-25T10:05:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af3cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af37830 0x55c09af3b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 0x55c09af3c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09aea0ac0 con 0x55c09af3bcf0 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09ae49d80 con 0x55c09af3cbb0 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09ae9cb60 con 0x55c09af37830 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 0x55c09af3c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 0x55c09af3c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58528/0 (socket says 172.21.15.148:58528) 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 -- 172.21.15.148:0/2275522449 learned_addr learned my addr 172.21.15.148:0/2275522449 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af3cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4259c640 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af37830 0x55c09af3b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 -- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af37830 msgr2=0x55c09af3b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af37830 0x55c09af3b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 -- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 msgr2=0x55c09af3cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af3cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e4359e640 1 -- 172.21.15.148:0/2275522449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c09ac7d7e0 con 0x55c09af3bcf0 2024-04-25T10:05:36.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.126+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af3cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4359e640 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 0x55c09af3c130 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f0e34075ba0 tx=0x7f0e34079280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98d2e0b6161fd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e28ff9640 1 -- 172.21.15.148:0/2275522449 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34031600 con 0x55c09af3bcf0 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e28ff9640 1 -- 172.21.15.148:0/2275522449 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340a2e50 con 0x55c09af3bcf0 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e28ff9640 1 -- 172.21.15.148:0/2275522449 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34079e40 con 0x55c09af3bcf0 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/2275522449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 msgr2=0x55c09af3c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 0x55c09af3c130 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f0e34075ba0 tx=0x7f0e34079280 comp rx=0 tx=0).stop 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/2275522449 shutdown_connections 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3bcf0 0x55c09af3c130 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af37830 0x55c09af3b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/2275522449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af3cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/2275522449 >> 172.21.15.148:0/2275522449 conn(0x55c09af39cb0 msgr2=0x55c09af82770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/2275522449 shutdown_connections 2024-04-25T10:05:36.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.127+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/2275522449 wait complete. 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.128+0000 7f0e4bbfd740 1 -- start start 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.128+0000 7f0e4bbfd740 1 Processor -- start 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af65fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af85ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09aea0ac0 con 0x55c09af85a50 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09ae3f530 con 0x55c09af3cbb0 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09af68740 con 0x55c09af3bcf0 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4bbfd740 10 client.0 _ll_register_callbacks cb 0x55c09ae97130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af85ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af85ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58540/0 (socket says 172.21.15.148:58540) 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 -- 172.21.15.148:0/1942605618 learned_addr learned my addr 172.21.15.148:0/1942605618 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4259c640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af65fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 -- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 msgr2=0x55c09af59c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af59c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 -- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 msgr2=0x55c09af65fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af65fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 -- 172.21.15.148:0/1942605618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c09acddf70 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af85ed0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f0e3408f040 tx=0x7f0e34009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80c1c954624423a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.129+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1942605618 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34096060 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1942605618 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e34078b90 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e340a2e50 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1942605618 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e340a8070 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 10 client.11007 fetch_fsmap learned FSMap version 99 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55c09ab75170 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1942605618 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340a2e50 con 0x55c09af85a50 2024-04-25T10:05:36.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1942605618 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e34075490 con 0x55c09af85a50 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 10 client.11007 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 10 client.11007 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 20 client.11007 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af57f40 0x55c09af56500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 4 client.11007 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c09aeae000 con 0x55c09af57f40 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af57f40 0x55c09af56500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e21ffb640 10 client.11007 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.130+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af57f40 0x55c09af56500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c5fbe747250d03 server_cookie=6077cd3130a55d4 in_seq=0 out_seq=0 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1942605618 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c09aeae000 con 0x55c09af57f40 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e21ffb640 10 client.11007 handle_command_reply: tid=0 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 client.11007 shutdown 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 msgr2=0x55c09af85ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af85ed0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f0e3408f040 tx=0x7f0e34009810 comp rx=0 tx=0).stop 2024-04-25T10:05:36.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 shutdown_connections 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af57f40 0x55c09af56500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af85ed0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af65fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e21ffb640 0 client.11007 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1942605618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af59c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 >> 172.21.15.148:0/1942605618 conn(0x55c09af37830 msgr2=0x55c09af3d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 shutdown_connections 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1942605618 wait complete. 2024-04-25T10:05:36.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.131+0000 7f0e4bbfd740 20 client.11007 trim_cache size 0 max 16384 2024-04-25T10:05:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.146+0000 7f0e4bbfd740 1 Processor -- start 2024-04-25T10:05:36.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.146+0000 7f0e4bbfd740 1 -- start start 2024-04-25T10:05:36.160 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-04-25T10:05:36.152+0000 7ff66d023640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:05:34.949580+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={10977=0-12582912@1} caps={10977=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=10977 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5634df989600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af95980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 0x55c09af8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af85a50 0x55c09af9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441fe470 con 0x55c09af3cbb0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441fd630 con 0x55c09af85a50 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440a8fa0 con 0x55c09af3bcf0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 0x55c09af8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 0x55c09af8fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58550/0 (socket says 172.21.15.148:58550) 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 -- 172.21.15.148:0/1900346931 learned_addr learned my addr 172.21.15.148:0/1900346931 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af95980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af85a50 0x55c09af9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 -- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 msgr2=0x55c09af95980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af95980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 -- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af85a50 msgr2=0x55c09af9d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af85a50 0x55c09af9d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 -- 172.21.15.148:0/1900346931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c09af3c9e0 con 0x55c09af3cbb0 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af85a50 0x55c09af9d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.147+0000 7f0e4259c640 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 0x55c09af8fad0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0e38001490 tx=0x7f0e38001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4e230e6a895c09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e23fff640 1 -- 172.21.15.148:0/1900346931 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38005320 con 0x55c09af3cbb0 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e23fff640 1 -- 172.21.15.148:0/1900346931 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3801b020 con 0x55c09af3cbb0 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e23fff640 1 -- 172.21.15.148:0/1900346931 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38007050 con 0x55c09af3cbb0 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1900346931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 msgr2=0x55c09af8fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 0x55c09af8fad0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0e38001490 tx=0x7f0e38001950 comp rx=0 tx=0).stop 2024-04-25T10:05:36.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1900346931 shutdown_connections 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af85a50 0x55c09af9d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af3cbb0 0x55c09af8fad0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1900346931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09af95980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1900346931 >> 172.21.15.148:0/1900346931 conn(0x55c09af37830 msgr2=0x55c09af5e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1900346931 shutdown_connections 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.148+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1900346931 wait complete. 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.149+0000 7f0e4bbfd740 1 -- start start 2024-04-25T10:05:36.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.149+0000 7f0e4bbfd740 1 Processor -- start 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.149+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09afe2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.149+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af5e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.149+0000 7f0e4bbfd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af5ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441fd060 con 0x55c09af85a50 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09aea2fd0 con 0x55c09af3cbb0 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c09af92700 con 0x55c09af3bcf0 2024-04-25T10:05:36.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 10 client.0 _ll_register_callbacks cb 0x7f0e34007ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af5ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af5ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58558/0 (socket says 172.21.15.148:58558) 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e42d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09afe2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 -- 172.21.15.148:0/1747582703 learned_addr learned my addr 172.21.15.148:0/1747582703 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4259c640 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af5e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 -- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 msgr2=0x55c09afe2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09afe2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 -- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 msgr2=0x55c09af5e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af5e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 -- 172.21.15.148:0/1747582703 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c09af673d0 con 0x55c09af85a50 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4359e640 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af5ef00 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009f10 tx=0x7f0e34007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9759e5ccc0c294f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1747582703 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34098350 con 0x55c09af85a50 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1747582703 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e34078600 con 0x55c09af85a50 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e3801b020 con 0x55c09af85a50 2024-04-25T10:05:36.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1747582703 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e340790a0 con 0x55c09af85a50 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 10 client.11013 fetch_fsmap learned FSMap version 99 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55c09af673d0 con 0x55c09af85a50 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1747582703 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3801b020 con 0x55c09af85a50 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.150+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1747582703 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e34078820 con 0x55c09af85a50 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e4bbfd740 10 client.11013 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e4bbfd740 10 client.11013 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T10:05:36.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e4bbfd740 20 client.11013 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af67610 0x55c09af91820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e4bbfd740 4 client.11013 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55c09af85200 con 0x55c09af67610 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af67610 0x55c09af91820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e42d9d640 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af67610 0x55c09af91820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ec8048ba05df1a server_cookie=f3f5adb73cece9ec in_seq=0 out_seq=0 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.151+0000 7f0e21ffb640 10 client.11013 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e21ffb640 1 -- 172.21.15.148:0/1747582703 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c09af85200 con 0x55c09af67610 2024-04-25T10:05:36.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e21ffb640 10 client.11013 handle_command_reply: tid=0 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 client.11013 shutdown 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 msgr2=0x55c09af5ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af5ef00 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009f10 tx=0x7f0e34007f90 comp rx=0 tx=0).stop 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 shutdown_connections 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55c09af67610 0x55c09af91820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c09af85a50 0x55c09af5ef00 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.170 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af3cbb0 0x55c09af5e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 --2- 172.21.15.148:0/1747582703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c09af3bcf0 0x55c09afe2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e21ffb640 0 client.11013 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 >> 172.21.15.148:0/1747582703 conn(0x55c09af37830 msgr2=0x55c09af3d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 shutdown_connections 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 1 -- 172.21.15.148:0/1747582703 wait complete. 2024-04-25T10:05:36.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.152+0000 7f0e4bbfd740 20 client.11013 trim_cache size 0 max 16384 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "6bc85a2f-adc0-4852-8dc8-b671a6270fd1", 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:05:36.172 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:05:36.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e3006b4d0 msgr2=0x7f0e3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e3006b4d0 0x7f0e3006d990 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009ac0 tx=0x7f0e3c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 msgr2=0x7f0e440a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 0x7f0e440a9bf0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009f40 tx=0x7f0e34004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 shutdown_connections 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e3006b4d0 0x7f0e3006d990 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e44144d50 0x7f0e440a9bf0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e441446d0 0x7f0e441526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 --2- 172.21.15.148:0/4122450499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4406e8a0 0x7f0e4415b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 >> 172.21.15.148:0/4122450499 conn(0x7f0e4406abe0 msgr2=0x7f0e44071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 shutdown_connections 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.153+0000 7f0e4925d640 1 -- 172.21.15.148:0/4122450499 wait complete. 2024-04-25T10:05:36.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:36.177 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:05:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.271+0000 7f9c2eab6640 1 Processor -- start 2024-04-25T10:05:36.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.271+0000 7f9c2eab6640 1 -- start start 2024-04-25T10:05:36.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2eab6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c280cd4c0 0x7f9c280cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2eab6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c2815b1c0 0x7f9c2815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2eab6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2815e6e0 con 0x7f9c2815e1a0 2024-04-25T10:05:36.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2eab6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2815b620 con 0x7f9c2815b1c0 2024-04-25T10:05:36.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2eab6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2815e9c0 con 0x7f9c280cd4c0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2d053640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9c2815e1a0 0x7f9c28150b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:42864/0 (socket says 172.21.15.148:42864) 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2d053640 1 -- 172.21.15.148:0/1626415623 learned_addr learned my addr 172.21.15.148:0/1626415623 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1626415623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c280cd4c0 0x7f9c280cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1626415623 >> v2:172.21.15.188:3300/0 conn(0x7f9c2815b1c0 0x7f9c2815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c277fe640 1 -- 172.21.15.148:0/1626415623 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507757903 0 0) 0x7f9c2815e6e0 con 0x7f9c2815e1a0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c277fe640 1 -- 172.21.15.148:0/1626415623 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c18003520 con 0x7f9c2815e1a0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c277fe640 1 -- 172.21.15.148:0/1626415623 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571340001 0 0) 0x7f9c18003520 con 0x7f9c2815e1a0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2c852640 1 -- 172.21.15.148:0/1626415623 >> v2:172.21.15.188:3300/0 conn(0x7f9c2815b1c0 msgr2=0x7f9c2815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1626415623 >> v2:172.21.15.188:3300/0 conn(0x7f9c2815b1c0 0x7f9c2815dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2c852640 1 -- 172.21.15.148:0/1626415623 >> v1:172.21.15.148:6789/0 conn(0x7f9c2815e1a0 legacy=0x7f9c28150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c2c852640 1 -- 172.21.15.148:0/1626415623 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2815a9c0 con 0x7f9c280cd4c0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.272+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1626415623 >> v2:172.21.15.188:3300/0 conn(0x7f9c2815b1c0 0x7f9c2815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.273+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1626415623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c280cd4c0 0x7f9c280cd8a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0097a0 tx=0x7f9c1c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd0e6b598733b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.273+0000 7f9c277fe640 1 -- 172.21.15.148:0/1626415623 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c037070 con 0x7f9c280cd4c0 2024-04-25T10:05:36.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.273+0000 7f9c277fe640 1 -- 172.21.15.148:0/1626415623 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c02aba0 con 0x7f9c280cd4c0 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.273+0000 7f9c277fe640 1 -- 172.21.15.148:0/1626415623 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c200019f0 con 0x7f9c280cd4c0 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 -- 172.21.15.148:0/1626415623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c280cd4c0 msgr2=0x7f9c280cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/1626415623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c280cd4c0 0x7f9c280cd8a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0097a0 tx=0x7f9c1c02a110 comp rx=0 tx=0).stop 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 -- 172.21.15.148:0/1626415623 shutdown_connections 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/1626415623 >> v2:172.21.15.188:3300/0 conn(0x7f9c2815b1c0 0x7f9c2815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/1626415623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c280cd4c0 0x7f9c280cd8a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 -- 172.21.15.148:0/1626415623 >> 172.21.15.148:0/1626415623 conn(0x7f9c2806abe0 msgr2=0x7f9c2806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 -- 172.21.15.148:0/1626415623 shutdown_connections 2024-04-25T10:05:36.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.274+0000 7f9c2eab6640 1 -- 172.21.15.148:0/1626415623 wait complete. 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.275+0000 7f9c2eab6640 1 Processor -- start 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.275+0000 7f9c2eab6640 1 -- start start 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2eab6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c280cd4c0 0x7f9c280ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2eab6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2815b1c0 0x7f9c281f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2eab6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 0x7f9c281f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2eab6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2806eca0 con 0x7f9c2815e1a0 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2eab6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2815a670 con 0x7f9c280cd4c0 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2eab6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2806ee20 con 0x7f9c2815b1c0 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 0x7f9c281f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 0x7f9c281f4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58574/0 (socket says 172.21.15.148:58574) 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2815b1c0 0x7f9c281f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 -- 172.21.15.148:0/3378281104 learned_addr learned my addr 172.21.15.148:0/3378281104 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2c852640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c280cd4c0 0x7f9c280ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 -- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2815b1c0 msgr2=0x7f9c281f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2815b1c0 0x7f9c281f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 -- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c280cd4c0 msgr2=0x7f9c280ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c280cd4c0 0x7f9c280ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 -- 172.21.15.148:0/3378281104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c281f4780 con 0x7f9c2815e1a0 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c27fff640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2815b1c0 0x7f9c281f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.276+0000 7f9c2d053640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 0x7f9c281f4240 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000cc00 tx=0x7f9c2000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.277+0000 7f9c257fa640 1 -- 172.21.15.148:0/3378281104 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20017070 con 0x7f9c2815e1a0 2024-04-25T10:05:36.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.277+0000 7f9c257fa640 1 -- 172.21.15.148:0/3378281104 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2000ce10 con 0x7f9c2815e1a0 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.277+0000 7f9c257fa640 1 -- 172.21.15.148:0/3378281104 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20012640 con 0x7f9c2815e1a0 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.277+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c280ac950 con 0x7f9c2815e1a0 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.277+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c280acf10 con 0x7f9c2815e1a0 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.278+0000 7f9c257fa640 1 -- 172.21.15.148:0/3378281104 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c2001a070 con 0x7f9c2815e1a0 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.278+0000 7f9c257fa640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0c06b500 0x7f9c0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.279+0000 7f9c257fa640 1 -- 172.21.15.148:0/3378281104 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c2008a120 con 0x7f9c2815e1a0 2024-04-25T10:05:36.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.279+0000 7f9c2c852640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0c06b500 0x7f9c0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.279+0000 7f9c2c852640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0c06b500 0x7f9c0c06d9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c034000 tx=0x7f9c1c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.279+0000 7f9c317bf740 1 Processor -- start 2024-04-25T10:05:36.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.279+0000 7f9c317bf740 1 -- start start 2024-04-25T10:05:36.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c40320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3def0 0x5594a8c40860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c10770 0x5594a8c13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c317bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a8b79da0 con 0x5594a8c3db10 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a8b22b80 con 0x5594a8c10770 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a8b75e40 con 0x5594a8c3def0 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c40320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c40320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58586/0 (socket says 172.21.15.148:58586) 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 -- 172.21.15.148:0/3634978554 learned_addr learned my addr 172.21.15.148:0/3634978554 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2d053640 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c10770 0x5594a8c13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c27fff640 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3def0 0x5594a8c40860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 -- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3def0 msgr2=0x5594a8c40860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3def0 0x5594a8c40860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 -- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c10770 msgr2=0x5594a8c13f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c10770 0x5594a8c13f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.280+0000 7f9c2c852640 1 -- 172.21.15.148:0/3634978554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a8b2cf10 con 0x5594a8c3db10 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c2c852640 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c40320 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0097a0 tx=0x7f9c1c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d951fb35c546462 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c127fc640 1 -- 172.21.15.148:0/3634978554 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c04f070 con 0x5594a8c3db10 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c127fc640 1 -- 172.21.15.148:0/3634978554 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c038040 con 0x5594a8c3db10 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c127fc640 1 -- 172.21.15.148:0/3634978554 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c037040 con 0x5594a8c3db10 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 -- 172.21.15.148:0/3634978554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 msgr2=0x5594a8c40320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c40320 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0097a0 tx=0x7f9c1c02c000 comp rx=0 tx=0).stop 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 -- 172.21.15.148:0/3634978554 shutdown_connections 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c10770 0x5594a8c13f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3def0 0x5594a8c40860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 --2- 172.21.15.148:0/3634978554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c40320 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 -- 172.21.15.148:0/3634978554 >> 172.21.15.148:0/3634978554 conn(0x5594a8c129e0 msgr2=0x5594a8c5b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 -- 172.21.15.148:0/3634978554 shutdown_connections 2024-04-25T10:05:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.281+0000 7f9c317bf740 1 -- 172.21.15.148:0/3634978554 wait complete. 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.282+0000 7f9c317bf740 1 -- start start 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.282+0000 7f9c317bf740 1 Processor -- start 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3db10 0x5594a8c5d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c3f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c6b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a8c15b90 con 0x5594a8c3def0 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a8b184b0 con 0x5594a8c3db10 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a8c15e70 con 0x5594a8c441c0 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c317bf740 10 client.0 _ll_register_callbacks cb 0x5594a8b58180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c3f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c2c852640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3db10 0x5594a8c5d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c3f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58596/0 (socket says 172.21.15.148:58596) 2024-04-25T10:05:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 -- 172.21.15.148:0/2577817943 learned_addr learned my addr 172.21.15.148:0/2577817943 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c2d053640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c6b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 -- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 msgr2=0x5594a8c6b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c6b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 -- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3db10 msgr2=0x5594a8c5d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3db10 0x5594a8c5d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c27fff640 1 -- 172.21.15.148:0/2577817943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594a89587d0 con 0x5594a8c3def0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.283+0000 7f9c2c852640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3db10 0x5594a8c5d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c27fff640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c3f7b0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f9c14010cd0 tx=0x7f9c14016580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d90f4b799568bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c077fe640 1 -- 172.21.15.148:0/2577817943 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14004230 con 0x5594a8c3def0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c077fe640 1 -- 172.21.15.148:0/2577817943 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c140043d0 con 0x5594a8c3def0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c1c038040 con 0x5594a8c3def0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c077fe640 1 -- 172.21.15.148:0/2577817943 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c14028050 con 0x5594a8c3def0 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 10 client.11025 fetch_fsmap learned FSMap version 99 2024-04-25T10:05:36.297 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5594a888e3f0 con 0x5594a8c3def0 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c077fe640 1 -- 172.21.15.148:0/2577817943 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c038040 con 0x5594a8c3def0 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c077fe640 1 -- 172.21.15.148:0/2577817943 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9c1402b020 con 0x5594a8c3def0 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 10 client.11025 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 10 client.11025 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 20 client.11025 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c5c450 0x5594a8c5c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 4 client.11025 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594a8b872e0 con 0x5594a8c5c450 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.284+0000 7f9c2c852640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c5c450 0x5594a8c5c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.285+0000 7f9c077fe640 10 client.11025 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.285+0000 7f9c2c852640 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c5c450 0x5594a8c5c880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e32411a51d8b7f8 server_cookie=b91a9b0f1f0c8c0b in_seq=0 out_seq=0 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.285+0000 7f9c077fe640 1 -- 172.21.15.148:0/2577817943 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5594a8b872e0 con 0x5594a8c5c450 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.285+0000 7f9c077fe640 10 client.11025 handle_command_reply: tid=0 2024-04-25T10:05:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.285+0000 7f9c317bf740 1 client.11025 shutdown 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 msgr2=0x5594a8c3f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c3f7b0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f9c14010cd0 tx=0x7f9c14016580 comp rx=0 tx=0).stop 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 shutdown_connections 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c5c450 0x5594a8c5c880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c6b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c3f7b0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 --2- 172.21.15.148:0/2577817943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3db10 0x5594a8c5d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c077fe640 0 client.11025 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 >> 172.21.15.148:0/2577817943 conn(0x5594a8c10770 msgr2=0x5594a8c15330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 shutdown_connections 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 1 -- 172.21.15.148:0/2577817943 wait complete. 2024-04-25T10:05:36.299 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.286+0000 7f9c317bf740 20 client.11025 trim_cache size 0 max 16384 2024-04-25T10:05:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 Processor -- start 2024-04-25T10:05:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 -- start start 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c3f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3def0 0x5594a8c3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c3c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281fbce0 con 0x5594a8c3db10 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281e5ba0 con 0x5594a8c3def0 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280a8db0 con 0x5594a8c441c0 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.301+0000 7f9c2c852640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c3f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c3f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58606/0 (socket says 172.21.15.148:58606) 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 -- 172.21.15.148:0/1110213349 learned_addr learned my addr 172.21.15.148:0/1110213349 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.309 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2d053640 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c3c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3def0 0x5594a8c3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 -- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 msgr2=0x5594a8c3c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c3c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 -- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3def0 msgr2=0x5594a8c3c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3def0 0x5594a8c3c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 -- 172.21.15.148:0/1110213349 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594a8c6b3e0 con 0x5594a8c3db10 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3def0 0x5594a8c3c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c3f7b0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0098c0 tx=0x7f9c1c038040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587f51823b84b899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c11ffb640 1 -- 172.21.15.148:0/1110213349 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c04f040 con 0x5594a8c3db10 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c11ffb640 1 -- 172.21.15.148:0/1110213349 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0042c0 con 0x5594a8c3db10 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.302+0000 7f9c11ffb640 1 -- 172.21.15.148:0/1110213349 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c053a90 con 0x5594a8c3db10 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 -- 172.21.15.148:0/1110213349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 msgr2=0x5594a8c3f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.310 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c3f7b0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c0098c0 tx=0x7f9c1c038040 comp rx=0 tx=0).stop 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 -- 172.21.15.148:0/1110213349 shutdown_connections 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c441c0 0x5594a8c3c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c3def0 0x5594a8c3c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1110213349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3db10 0x5594a8c3f7b0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 -- 172.21.15.148:0/1110213349 >> 172.21.15.148:0/1110213349 conn(0x5594a8c10770 msgr2=0x5594a8c15180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 -- 172.21.15.148:0/1110213349 shutdown_connections 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 -- 172.21.15.148:0/1110213349 wait complete. 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 -- start start 2024-04-25T10:05:36.311 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.303+0000 7f9c317bf740 1 Processor -- start 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3db10 0x5594a8c0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c5d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c441c0 0x5594a8cc4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c281e55a0 con 0x5594a8c3def0 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594a8c10ec0 con 0x5594a8c441c0 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594a8c0fe50 con 0x5594a8c3db10 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c317bf740 10 client.0 _ll_register_callbacks cb 0x7f9c180036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c5d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c5d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58618/0 (socket says 172.21.15.148:58618) 2024-04-25T10:05:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 -- 172.21.15.148:0/1595021884 learned_addr learned my addr 172.21.15.148:0/1595021884 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c2d053640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c441c0 0x5594a8cc4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3db10 0x5594a8c0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 -- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3db10 msgr2=0x5594a8c0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3db10 0x5594a8c0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 -- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c441c0 msgr2=0x5594a8cc4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c441c0 0x5594a8cc4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.304+0000 7f9c27fff640 1 -- 172.21.15.148:0/1595021884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594a884e0f0 con 0x5594a8c3def0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c27fff640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c5d6c0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f9c140165c0 tx=0x7f9c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759b89dc9683701e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c077fe640 1 -- 172.21.15.148:0/1595021884 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14032070 con 0x5594a8c3def0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c077fe640 1 -- 172.21.15.148:0/1595021884 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c14036400 con 0x5594a8c3def0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c1c0042c0 con 0x5594a8c3def0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c077fe640 1 -- 172.21.15.148:0/1595021884 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c14040050 con 0x5594a8c3def0 2024-04-25T10:05:36.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 10 client.11031 fetch_fsmap learned FSMap version 99 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5594a8c0fb90 con 0x5594a8c3def0 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c077fe640 1 -- 172.21.15.148:0/1595021884 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0042c0 con 0x5594a8c3def0 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c077fe640 1 -- 172.21.15.148:0/1595021884 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9c14043020 con 0x5594a8c3def0 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 10 client.11031 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 10 client.11031 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 20 client.11031 populate_metadata read hostname 'smithi148' 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c74d00 0x5594a8c75110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 4 client.11031 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5594a8b81630 con 0x5594a8c74d00 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.305+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c74d00 0x5594a8c75110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c077fe640 10 client.11031 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c2c852640 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c74d00 0x5594a8c75110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95e1b8d5d50be72 server_cookie=3b7385954cc1a30 in_seq=0 out_seq=0 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c077fe640 1 -- 172.21.15.148:0/1595021884 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5594a8b81630 con 0x5594a8c74d00 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c077fe640 10 client.11031 handle_command_reply: tid=0 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 client.11031 shutdown 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 msgr2=0x5594a8c5d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c5d6c0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f9c140165c0 tx=0x7f9c140046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 shutdown_connections 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x5594a8c74d00 0x5594a8c75110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5594a8c441c0 0x5594a8cc4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594a8c3def0 0x5594a8c5d6c0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c077fe640 0 client.11031 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 --2- 172.21.15.148:0/1595021884 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5594a8c3db10 0x5594a8c0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 >> 172.21.15.148:0/1595021884 conn(0x5594a8c10770 msgr2=0x5594a8c15da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 shutdown_connections 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 1 -- 172.21.15.148:0/1595021884 wait complete. 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.306+0000 7f9c317bf740 20 client.11031 trim_cache size 0 max 16384 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: "6bc85a2f-adc0-4852-8dc8-b671a6270fd1": { 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "6bc85a2f-adc0-4852-8dc8-b671a6270fd1", 2024-04-25T10:05:36.319 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive,repair" 2024-04-25T10:05:36.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:36.320 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:05:36.320 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:05:36.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0c06b500 msgr2=0x7f9c0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0c06b500 0x7f9c0c06d9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c034000 tx=0x7f9c1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 msgr2=0x7f9c281f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 0x7f9c281f4240 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000cc00 tx=0x7f9c2000a430 comp rx=0 tx=0).stop 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 shutdown_connections 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c0c06b500 0x7f9c0c06d9c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c2815e1a0 0x7f9c281f4240 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c2815b1c0 0x7f9c281f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 --2- 172.21.15.148:0/3378281104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c280cd4c0 0x7f9c280ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 >> 172.21.15.148:0/3378281104 conn(0x7f9c2806abe0 msgr2=0x7f9c2814fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 shutdown_connections 2024-04-25T10:05:36.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:05:36.307+0000 7f9c2eab6640 1 -- 172.21.15.148:0/3378281104 wait complete. 2024-04-25T10:05:36.323 INFO:tasks.cephfs.filesystem:scrub status for tag:6bc85a2f-adc0-4852-8dc8-b671a6270fd1 - {'path': '/', 'tag': '6bc85a2f-adc0-4852-8dc8-b671a6270fd1', 'options': 'recursive,repair'} 2024-04-25T10:05:55.720 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:55.724 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:06.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:06.324 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:06:06.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.429+0000 7f9bcfe9a640 1 Processor -- start 2024-04-25T10:06:06.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.429+0000 7f9bcfe9a640 1 -- start start 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bcfe9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 0x7f9bc80ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bcfe9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9bc81454b0 0x7f9bc8070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bcfe9a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc80ceba0 con 0x7f9bc8071300 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bcfe9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc8071750 con 0x7f9bc81454b0 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bcfe9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b4830 con 0x7f9bc8144d20 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bce437640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9bc8071300 0x7f9bc815f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45212/0 (socket says 172.21.15.148:45212) 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.430+0000 7f9bce437640 1 -- 172.21.15.148:0/749075639 learned_addr learned my addr 172.21.15.148:0/749075639 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 --2- 172.21.15.148:0/749075639 >> v2:172.21.15.188:3300/0 conn(0x7f9bc81454b0 0x7f9bc8070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 0x7f9bc80ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206342834 0 0) 0x7f9bc80ceba0 con 0x7f9bc8071300 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb4003520 con 0x7f9bc8071300 2024-04-25T10:06:06.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981088715 0 0) 0x7f9bb4003520 con 0x7f9bc8071300 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 msgr2=0x7f9bc80ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 0x7f9bc80ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 >> v2:172.21.15.188:3300/0 conn(0x7f9bc81454b0 msgr2=0x7f9bc8070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 --2- 172.21.15.148:0/749075639 >> v2:172.21.15.188:3300/0 conn(0x7f9bc81454b0 0x7f9bc8070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80cee80 con 0x7f9bc8071300 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9bc00019f0 con 0x7f9bc8071300 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 >> v1:172.21.15.148:6789/0 conn(0x7f9bc8071300 legacy=0x7f9bc815f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 0x7f9bb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4003f30 0x7f9bb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8071750 con 0x7f9bb4003f30 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b4830 con 0x7f9bb4004d40 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb400b9c0 con 0x7f9bc8144d20 2024-04-25T10:06:06.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4003f30 0x7f9bb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 0x7f9bb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bce437640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 -- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 msgr2=0x7f9bb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bc8144d20 0x7f9bb40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 -- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bb4004d40 msgr2=0x7f9bb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bb4004d40 0x7f9bb40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.431+0000 7f9bcd435640 1 -- 172.21.15.148:0/749075639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4003520 con 0x7f9bb4003f30 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.432+0000 7f9bcd435640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4003f30 0x7f9bb4004800 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009ac0 tx=0x7f9bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5318dba1f814d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.432+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8036070 con 0x7f9bb4003f30 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.432+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8031400 con 0x7f9bb4003f30 2024-04-25T10:06:06.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.432+0000 7f9bccc34640 1 -- 172.21.15.148:0/749075639 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8009690 con 0x7f9bb4003f30 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/749075639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4003f30 msgr2=0x7f9bb4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/749075639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb4003f30 0x7f9bb4004800 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009ac0 tx=0x7f9bb4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bce437640 1 -- 172.21.15.148:0/749075639 reap_dead start 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/749075639 shutdown_connections 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/749075639 >> 172.21.15.148:0/749075639 conn(0x7f9bc80b3530 msgr2=0x7f9bc80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/749075639 shutdown_connections 2024-04-25T10:06:06.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.433+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/749075639 wait complete. 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.434+0000 7f9bcfe9a640 1 Processor -- start 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.434+0000 7f9bcfe9a640 1 -- start start 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcfe9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bb4003f30 0x7f9bc81e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcfe9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 0x7f9bc813f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcfe9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc80c9d00 0x7f9bc80ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcfe9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8145100 con 0x7f9bc813efd0 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcfe9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80cf440 con 0x7f9bc80c9d00 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcfe9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8145280 con 0x7f9bb4003f30 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 0x7f9bc813f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 0x7f9bc813f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60680/0 (socket says 172.21.15.148:60680) 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 -- 172.21.15.148:0/1338018984 learned_addr learned my addr 172.21.15.148:0/1338018984 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bce437640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc80c9d00 0x7f9bc80ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bb4003f30 0x7f9bc81e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 -- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bb4003f30 msgr2=0x7f9bc81e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bb4003f30 0x7f9bc81e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 -- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc80c9d00 msgr2=0x7f9bc80ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc80c9d00 0x7f9bc80ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.435+0000 7f9bcd435640 1 -- 172.21.15.148:0/1338018984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc80caa60 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.436+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 0x7f9bc813f450 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8034040 tx=0x7f9bb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.436+0000 7f9bbe7fc640 1 -- 172.21.15.148:0/1338018984 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8040020 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.436+0000 7f9bbe7fc640 1 -- 172.21.15.148:0/1338018984 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8003c60 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.436+0000 7f9bbe7fc640 1 -- 172.21.15.148:0/1338018984 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8036040 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.436+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc813f990 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.436+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc80cbc90 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.437+0000 7f9bbe7fc640 1 -- 172.21.15.148:0/1338018984 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bb8043020 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.438+0000 7f9bbe7fc640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bac06b4d0 0x7f9bac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.438+0000 7f9bbe7fc640 1 -- 172.21.15.148:0/1338018984 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f9bb80aa4e0 con 0x7f9bc813efd0 2024-04-25T10:06:06.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.438+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bac06b4d0 0x7f9bac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.438+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bac06b4d0 0x7f9bac06d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009ae0 tx=0x7f9bc4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.439+0000 7f9bd2b98740 1 Processor -- start 2024-04-25T10:06:06.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.439+0000 7f9bd2b98740 1 -- start start 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df5e510 0x55b31df5e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df2e770 0x55b31df347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df7a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b31de97da0 con 0x55b31df7a470 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b31de40b80 con 0x55b31df2e770 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b31de93e40 con 0x55b31df5e510 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df7a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df7a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60692/0 (socket says 172.21.15.148:60692) 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 -- 172.21.15.148:0/1025701477 learned_addr learned my addr 172.21.15.148:0/1025701477 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:06.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df5e510 0x55b31df5e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df2e770 0x55b31df347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 -- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df5e510 msgr2=0x55b31df5e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df5e510 0x55b31df5e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 -- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df2e770 msgr2=0x55b31df347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df2e770 0x55b31df347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 -- 172.21.15.148:0/1025701477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b31de4af10 con 0x55b31df7a470 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bce437640 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df7a8f0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80b0960 tx=0x7f9bc000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a573ba500bd6bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bab7fe640 1 -- 172.21.15.148:0/1025701477 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc001b070 con 0x55b31df7a470 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bab7fe640 1 -- 172.21.15.148:0/1025701477 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0015440 con 0x55b31df7a470 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.440+0000 7f9bab7fe640 1 -- 172.21.15.148:0/1025701477 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00157f0 con 0x55b31df7a470 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1025701477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 msgr2=0x55b31df7a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df7a8f0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80b0960 tx=0x7f9bc000a060 comp rx=0 tx=0).stop 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1025701477 shutdown_connections 2024-04-25T10:06:06.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df7a8f0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df2e770 0x55b31df347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1025701477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df5e510 0x55b31df5e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1025701477 >> 172.21.15.148:0/1025701477 conn(0x55b31df309e0 msgr2=0x55b31df79320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1025701477 shutdown_connections 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.441+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1025701477 wait complete. 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.442+0000 7f9bd2b98740 1 -- start start 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.442+0000 7f9bd2b98740 1 Processor -- start 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 0x55b31df7bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df7a470 0x55b31df90770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df4a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b31df34ce0 con 0x55b31df5e510 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b31de364b0 con 0x55b31df7a470 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b31df34310 con 0x55b31df8de70 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 10 client.0 _ll_register_callbacks cb 0x55b31de76180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:06.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 0x55b31df7bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 0x55b31df7bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60698/0 (socket says 172.21.15.148:60698) 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 -- 172.21.15.148:0/438212443 learned_addr learned my addr 172.21.15.148:0/438212443 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcd435640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df7a470 0x55b31df90770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bce437640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df4a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 -- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 msgr2=0x55b31df4a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df4a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 -- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df7a470 msgr2=0x55b31df90770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df7a470 0x55b31df90770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 -- 172.21.15.148:0/438212443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b31dc767d0 con 0x55b31df5e510 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bce437640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df4a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 0x55b31df7bcc0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4003640 tx=0x7f9bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23978b3ad60dd637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9ba8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/438212443 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc401b070 con 0x55b31df5e510 2024-04-25T10:06:06.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/438212443 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f9bc4015770 con 0x55b31df5e510 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.443+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bc0015440 con 0x55b31df5e510 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/438212443 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bc4034050 con 0x55b31df5e510 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 10 client.11043 fetch_fsmap learned FSMap version 99 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55b31dbac3f0 con 0x55b31df5e510 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/438212443 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0015440 con 0x55b31df5e510 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/438212443 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9bc4015970 con 0x55b31df5e510 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 10 client.11043 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 10 client.11043 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 20 client.11043 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df8e340 0x55b31df8e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 4 client.11043 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b31dea52e0 con 0x55b31df8e340 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.444+0000 7f9bcd435640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df8e340 0x55b31df8e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bcd435640 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df8e340 0x55b31df8e750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3049e7f4470fdf02 server_cookie=6d6edf565e578b00 in_seq=0 out_seq=0 2024-04-25T10:06:06.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9ba8ff9640 10 client.11043 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/438212443 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b31dea52e0 con 0x55b31df8e340 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9ba8ff9640 10 client.11043 handle_command_reply: tid=0 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 client.11043 shutdown 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 msgr2=0x55b31df7bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 0x55b31df7bcc0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4003640 tx=0x7f9bc40046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 shutdown_connections 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df8e340 0x55b31df8e750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df4a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9ba8ff9640 0 client.11043 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df7a470 0x55b31df90770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/438212443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df5e510 0x55b31df7bcc0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 >> 172.21.15.148:0/438212443 conn(0x55b31df2e770 msgr2=0x55b31df7c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.445+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 shutdown_connections 2024-04-25T10:06:06.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.446+0000 7f9bd2b98740 1 -- 172.21.15.148:0/438212443 wait complete. 2024-04-25T10:06:06.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.446+0000 7f9bd2b98740 20 client.11043 trim_cache size 0 max 16384 2024-04-25T10:06:06.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.461+0000 7f9bd2b98740 1 Processor -- start 2024-04-25T10:06:06.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.461+0000 7f9bd2b98740 1 -- start start 2024-04-25T10:06:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31df796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc81fca50 con 0x55b31df7a470 2024-04-25T10:06:06.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc806d900 con 0x55b31df5e510 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b6300 con 0x55b31df8de70 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df890a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60708/0 (socket says 172.21.15.148:60708) 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 -- 172.21.15.148:0/1822800883 learned_addr learned my addr 172.21.15.148:0/1822800883 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31df796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bce437640 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 -- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 msgr2=0x55b31df895e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df895e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 -- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 msgr2=0x55b31df796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31df796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 -- 172.21.15.148:0/1822800883 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b31db6c0f0 con 0x55b31df7a470 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bce437640 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df895e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9bcd435640 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df890a0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80789f0 tx=0x7f9bb80a8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adf56eb8e09ffca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9baaffd640 1 -- 172.21.15.148:0/1822800883 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8095070 con 0x55b31df7a470 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9baaffd640 1 -- 172.21.15.148:0/1822800883 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8081170 con 0x55b31df7a470 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.462+0000 7f9baaffd640 1 -- 172.21.15.148:0/1822800883 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb80a7030 con 0x55b31df7a470 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1822800883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 msgr2=0x55b31df890a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df890a0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f9bb80789f0 tx=0x7f9bb80a8620 comp rx=0 tx=0).stop 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1822800883 shutdown_connections 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df895e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df890a0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/1822800883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31df796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1822800883 >> 172.21.15.148:0/1822800883 conn(0x55b31df2e770 msgr2=0x55b31df346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1822800883 shutdown_connections 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 -- 172.21.15.148:0/1822800883 wait complete. 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.463+0000 7f9bd2b98740 1 -- start start 2024-04-25T10:06:06.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 Processor -- start 2024-04-25T10:06:06.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31dfebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df59400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b31df7fb50 con 0x55b31df7a470 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc806d300 con 0x55b31df5e510 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.464+0000 7f9bd2b98740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b31df291a0 con 0x55b31df8de70 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bd2b98740 10 client.0 _ll_register_callbacks cb 0x7f9bc0001910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df9fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60710/0 (socket says 172.21.15.148:60710) 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 -- 172.21.15.148:0/3999470558 learned_addr learned my addr 172.21.15.148:0/3999470558 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bce437640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df59400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31dfebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 -- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 msgr2=0x55b31df59400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df59400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 -- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 msgr2=0x55b31dfebe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31dfebe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 -- 172.21.15.148:0/3999470558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b31df2dab0 con 0x55b31df7a470 2024-04-25T10:06:06.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bcd435640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df9fc20 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8000c00 tx=0x7f9bc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdeb2bc35d6314b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9ba8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/3999470558 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8095050 con 0x55b31df7a470 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/3999470558 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f9bb80750d0 con 0x55b31df7a470 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bb8081170 con 0x55b31df7a470 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/3999470558 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bb8078cc0 con 0x55b31df7a470 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bd2b98740 10 client.11049 fetch_fsmap learned FSMap version 99 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55b31df324f0 con 0x55b31df7a470 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.465+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/3999470558 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8081170 con 0x55b31df7a470 2024-04-25T10:06:06.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/3999470558 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9bb8080ce0 con 0x55b31df7a470 2024-04-25T10:06:06.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 10 client.11049 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T10:06:06.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 10 client.11049 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.b 2024-04-25T10:06:06.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 20 client.11049 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:06.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df31f80 0x55b31df59940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 4 client.11049 mds_command: new command op to 10845 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 --> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b31df80e10 con 0x55b31df31f80 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df31f80 0x55b31df59940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9ba8ff9640 10 client.11049 ms_handle_connect on v2:172.21.15.188:6833/906811420 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bcdc36640 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df31f80 0x55b31df59940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea44daecf415f72 server_cookie=f24c7227a8870097 in_seq=0 out_seq=0 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9ba8ff9640 1 -- 172.21.15.148:0/3999470558 <== mds.0 v2:172.21.15.188:6833/906811420 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b31df80e10 con 0x55b31df31f80 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9ba8ff9640 10 client.11049 handle_command_reply: tid=0 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 1 client.11049 shutdown 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.466+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 msgr2=0x55b31df9fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df9fc20 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8000c00 tx=0x7f9bc4017000 comp rx=0 tx=0).stop 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 shutdown_connections 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:6833/906811420,v1:172.21.15.188:6835/906811420] conn(0x55b31df31f80 0x55b31df59940 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b31df8de70 0x55b31df59400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31df7a470 0x55b31df9fc20 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9ba8ff9640 0 client.11049 ms_handle_reset on v2:172.21.15.188:6833/906811420 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 --2- 172.21.15.148:0/3999470558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b31df5e510 0x55b31dfebe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 >> 172.21.15.148:0/3999470558 conn(0x55b31df2e770 msgr2=0x55b31df346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 shutdown_connections 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 1 -- 172.21.15.148:0/3999470558 wait complete. 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bd2b98740 20 client.11049 trim_cache size 0 max 16384 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:06.480 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:06:06.481 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:06:06.481 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:06.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bac06b4d0 msgr2=0x7f9bac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bac06b4d0 0x7f9bac06d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009ae0 tx=0x7f9bc4009190 comp rx=0 tx=0).stop 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 msgr2=0x7f9bc813f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 0x7f9bc813f450 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8034040 tx=0x7f9bb8004730 comp rx=0 tx=0).stop 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 shutdown_connections 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9bac06b4d0 0x7f9bac06d990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bc80c9d00 0x7f9bc80ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bc813efd0 0x7f9bc813f450 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 --2- 172.21.15.148:0/1338018984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bb4003f30 0x7f9bc81e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 >> 172.21.15.148:0/1338018984 conn(0x7f9bc80b3530 msgr2=0x7f9bc80b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 shutdown_connections 2024-04-25T10:06:06.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.467+0000 7f9bcfe9a640 1 -- 172.21.15.148:0/1338018984 wait complete. 2024-04-25T10:06:06.484 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:06.484 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T10:06:06.484 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> import stat 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> import json 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> import sys 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> except OSError as e: 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> sys.exit(e.errno) 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> print(json.dumps( 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> indent=2)) 2024-04-25T10:06:06.485 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi148.stdout: "st_mode": 33188, 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi148.stdout: "st_ino": 1099511627777, 2024-04-25T10:06:06.553 INFO:teuthology.orchestra.run.smithi148.stdout: "st_dev": 43, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_nlink": 1, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_uid": 1000, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_gid": 1316, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_size": 6291456, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_atime": 1714039534.933581, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_mtime": 1714039534.9515808, 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout: "st_ctime": 1714039534.9515808 2024-04-25T10:06:06.554 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:06.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:06.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:06.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.694+0000 7f6a0d05d640 1 Processor -- start 2024-04-25T10:06:06.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.694+0000 7f6a0d05d640 1 -- start start 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0d05d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a080cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0d05d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a0815df50 0x7f6a080cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0d05d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a08071650 con 0x7f6a080cec70 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0d05d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a080cbd80 con 0x7f6a0815df50 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0d05d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08161910 con 0x7f6a0815db70 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f69fffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a0815df50 0x7f6a080cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a06d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a080cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0759e640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f6a080cec70 0x7f6a080cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:45838/0 (socket says 172.21.15.148:45838) 2024-04-25T10:06:06.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f6a0759e640 1 -- 172.21.15.148:0/152847731 learned_addr learned my addr 172.21.15.148:0/152847731 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:06.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f69fffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a0815df50 0x7f6a080cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:38186/0 (socket says 172.21.15.148:38186) 2024-04-25T10:06:06.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f69fffff640 1 -- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 msgr2=0x7f6a080cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f69fffff640 1 --2- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a080cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f69fffff640 1 -- 172.21.15.148:0/152847731 >> v1:172.21.15.188:6789/0 conn(0x7f6a080cec70 legacy=0x7f6a080cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.695+0000 7f69fffff640 1 -- 172.21.15.148:0/152847731 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080cf230 con 0x7f6a0815df50 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.696+0000 7f69fffff640 1 --2- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a080cf510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f69f00097a0 tx=0x7f69f002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5ab2293164d556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.696+0000 7f6a0659c640 1 -- 172.21.15.148:0/152847731 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0037070 con 0x7f6a0815df50 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.696+0000 7f6a0659c640 1 -- 172.21.15.148:0/152847731 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0032440 con 0x7f6a0815df50 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.696+0000 7f6a0659c640 1 -- 172.21.15.148:0/152847731 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0032810 con 0x7f6a0815df50 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 -- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 msgr2=0x7f6a080cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a080cf510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f69f00097a0 tx=0x7f69f002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:06.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 -- 172.21.15.148:0/152847731 shutdown_connections 2024-04-25T10:06:06.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a080cf510 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/152847731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a080cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 -- 172.21.15.148:0/152847731 >> 172.21.15.148:0/152847731 conn(0x7f6a0806abe0 msgr2=0x7f6a0806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 -- 172.21.15.148:0/152847731 shutdown_connections 2024-04-25T10:06:06.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.697+0000 7f6a0d05d640 1 -- 172.21.15.148:0/152847731 wait complete. 2024-04-25T10:06:06.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.698+0000 7f6a0d05d640 1 Processor -- start 2024-04-25T10:06:06.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.698+0000 7f6a0d05d640 1 -- start start 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0d05d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 0x7f6a08142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0d05d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a0814e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0d05d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a0814e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0d05d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08162120 con 0x7f6a080cec70 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0d05d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08161e40 con 0x7f6a0815df50 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0d05d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a08162640 con 0x7f6a0815db70 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 0x7f6a08142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 0x7f6a08142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60724/0 (socket says 172.21.15.148:60724) 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 -- 172.21.15.148:0/967570795 learned_addr learned my addr 172.21.15.148:0/967570795 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a0759e640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a0814e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f69fffff640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a0814e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 -- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 msgr2=0x7f6a0814e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a0814e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 -- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 msgr2=0x7f6a0814e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a0814e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f6a06d9d640 1 -- 172.21.15.148:0/967570795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0813f950 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.699+0000 7f69fffff640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a0814e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.700+0000 7f6a06d9d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 0x7f6a08142510 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f69f800e9b0 tx=0x7f69f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.700+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8017070 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.700+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f800c9a0 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.700+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0813fc30 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.700+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800cd90 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.700+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a081401f0 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.701+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a08073990 con 0x7f6a080cec70 2024-04-25T10:06:06.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.701+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69f801a070 con 0x7f6a080cec70 2024-04-25T10:06:06.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.702+0000 7f69ff7fe640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69e806b4d0 0x7f69e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.702+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f69f808a800 con 0x7f6a080cec70 2024-04-25T10:06:06.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.702+0000 7f69fffff640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69e806b4d0 0x7f69e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.702+0000 7f69fffff640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69e806b4d0 0x7f69e806d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f69f0034000 tx=0x7f69f002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.706+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69f8059460 con 0x7f6a080cec70 2024-04-25T10:06:06.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.914+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a080d0250 con 0x7f6a080cec70 2024-04-25T10:06:06.920 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.915+0000 7f69ff7fe640 1 -- 172.21.15.148:0/967570795 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f69f8058d60 con 0x7f6a080cec70 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:06:06.921 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69e806b4d0 msgr2=0x7f69e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69e806b4d0 0x7f69e806d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f69f0034000 tx=0x7f69f002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 msgr2=0x7f6a08142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 0x7f6a08142510 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f69f800e9b0 tx=0x7f69f800ee70 comp rx=0 tx=0).stop 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 shutdown_connections 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69e806b4d0 0x7f69e806d990 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0815df50 0x7f6a0814e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0815db70 0x7f6a0814e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 --2- 172.21.15.148:0/967570795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a080cec70 0x7f6a08142510 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 >> 172.21.15.148:0/967570795 conn(0x7f6a0806abe0 msgr2=0x7f6a0815be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 shutdown_connections 2024-04-25T10:06:06.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:06.917+0000 7f6a0d05d640 1 -- 172.21.15.148:0/967570795 wait complete. 2024-04-25T10:06:06.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:06:07.060 INFO:teuthology.orchestra.run.smithi188.stdout:{"message":"","return_code":0} 2024-04-25T10:06:07.061 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:06:07.061 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:06:07.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.098+0000 7f7b10106f40 1 Processor -- start 2024-04-25T10:06:07.104 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.098+0000 7f7b10106f40 1 -- start start 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b10106f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76ed8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b10106f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 0x556e76ecda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b10106f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed8b80 0x556e76e47510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b10106f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e76da5df0 con 0x556e76ed60a0 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b10106f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e76da9d50 con 0x556e76ed8b80 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b10106f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e76dff9a0 con 0x556e76dfcfb0 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 0x556e76ecda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 0x556e76ecda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60742/0 (socket says 172.21.15.148:60742) 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 -- 172.21.15.148:0/3484547895 learned_addr learned my addr 172.21.15.148:0/3484547895 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0e408640 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed8b80 0x556e76e47510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0dc07640 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76ed8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 -- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 msgr2=0x556e76ed8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76ed8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 -- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed8b80 msgr2=0x556e76e47510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed8b80 0x556e76e47510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 -- 172.21.15.148:0/3484547895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e76e2c300 con 0x556e76ed60a0 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7b0d406640 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 0x556e76ecda90 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f7b04001d80 tx=0x7f7b04001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7869cf84f690f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7afffff640 1 -- 172.21.15.148:0/3484547895 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400bd70 con 0x556e76ed60a0 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7afffff640 1 -- 172.21.15.148:0/3484547895 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0400f400 con 0x556e76ed60a0 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.099+0000 7f7afffff640 1 -- 172.21.15.148:0/3484547895 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0400f7f0 con 0x556e76ed60a0 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 -- 172.21.15.148:0/3484547895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 msgr2=0x556e76ecda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 0x556e76ecda90 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f7b04001d80 tx=0x7f7b04001e70 comp rx=0 tx=0).stop 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 -- 172.21.15.148:0/3484547895 shutdown_connections 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed8b80 0x556e76e47510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.111 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76ed60a0 0x556e76ecda90 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 --2- 172.21.15.148:0/3484547895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76ed8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 -- 172.21.15.148:0/3484547895 >> 172.21.15.148:0/3484547895 conn(0x556e76dfd9b0 msgr2=0x556e76dfc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 -- 172.21.15.148:0/3484547895 shutdown_connections 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.100+0000 7f7b10106f40 1 -- 172.21.15.148:0/3484547895 wait complete. 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 Processor -- start 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 -- start start 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76e47700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed60a0 0x556e76ed2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76e47c40 0x556e76ed3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e76e2c5e0 con 0x556e76e47c40 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e76e2c0b0 con 0x556e76ed60a0 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b10106f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e76e43770 con 0x556e76dfcfb0 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b0e408640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76e47c40 0x556e76ed3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b0e408640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76e47c40 0x556e76ed3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60748/0 (socket says 172.21.15.148:60748) 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b0e408640 1 -- 172.21.15.148:0/1371563127 learned_addr learned my addr 172.21.15.148:0/1371563127 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b0dc07640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76e47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.101+0000 7f7b0d406640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed60a0 0x556e76ed2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b0e408640 1 -- 172.21.15.148:0/1371563127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 msgr2=0x556e76e47700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b0e408640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556e76dfcfb0 0x556e76e47700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b0e408640 1 -- 172.21.15.148:0/1371563127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed60a0 msgr2=0x556e76ed2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b0e408640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e76ed60a0 0x556e76ed2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b0e408640 1 -- 172.21.15.148:0/1371563127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e76ecb430 con 0x556e76e47c40 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b0e408640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76e47c40 0x556e76ed3260 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800d440 tx=0x7f7b0800d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7afeffd640 1 -- 172.21.15.148:0/1371563127 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08018070 con 0x556e76e47c40 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7afeffd640 1 -- 172.21.15.148:0/1371563127 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b080060d0 con 0x556e76e47c40 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556e76ed52b0 con 0x556e76e47c40 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7afeffd640 1 -- 172.21.15.148:0/1371563127 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08004520 con 0x556e76e47c40 2024-04-25T10:06:07.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.102+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e76ec95b0 con 0x556e76e47c40 2024-04-25T10:06:07.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.103+0000 7f7afeffd640 1 -- 172.21.15.148:0/1371563127 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b0801b070 con 0x556e76e47c40 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7afeffd640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7af0069ee0 0x7f7af006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7afeffd640 1 -- 172.21.15.148:0/1371563127 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7b080886a0 con 0x556e76e47c40 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7b0dc07640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7af0069ee0 0x7f7af006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7b10106f40 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x556e76e457e0 0x556e76ec9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 --> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e173) v9 -- 0x556e76eca010 con 0x556e76e457e0 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7b0d406640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x556e76e457e0 0x556e76ec9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7b0dc07640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7af0069ee0 0x7f7af006c3a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7b00057000 tx=0x7f7b00052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.104+0000 7f7b0d406640 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x556e76e457e0 0x556e76ec9ad0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b0d406640 1 -- 172.21.15.148:0/1371563127 <== osd.2 v2:172.21.15.148:6808/1311261585 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f7b0400f5b0 con 0x556e76e457e0 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x556e76e457e0 msgr2=0x556e76ec9ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x556e76e457e0 0x556e76ec9ad0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7af0069ee0 msgr2=0x7f7af006c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7af0069ee0 0x7f7af006c3a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7b00057000 tx=0x7f7b00052000 comp rx=0 tx=0).stop 2024-04-25T10:06:07.116 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76e47c40 msgr2=0x556e76ed3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 --2- 172.21.15.148:0/1371563127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e76e47c40 0x556e76ed3260 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800d440 tx=0x7f7b0800d840 comp rx=0 tx=0).stop 2024-04-25T10:06:07.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b0e408640 1 -- 172.21.15.148:0/1371563127 reap_dead start 2024-04-25T10:06:07.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 shutdown_connections 2024-04-25T10:06:07.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 >> 172.21.15.148:0/1371563127 conn(0x556e76dfd9b0 msgr2=0x556e76ecb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 shutdown_connections 2024-04-25T10:06:07.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.105+0000 7f7b10106f40 1 -- 172.21.15.148:0/1371563127 wait complete. 2024-04-25T10:06:07.118 INFO:teuthology.orchestra.run.smithi148.stdout:Ysixmegssubdir  2024-04-25T10:06:07.118 DEBUG:teuthology.orchestra.run.smithi148:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 1099511627777, 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi148.stdout: "ancestors": [ 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi148.stdout: "dirino": 1099511627776, 2024-04-25T10:06:07.232 INFO:teuthology.orchestra.run.smithi148.stdout: "dname": "sixmegs", 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: "version": 8 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: "dirino": 1, 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: "dname": "subdir", 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: "version": 11 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: ], 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: "pool": 26, 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: "old_pools": [] 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:07.233 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:07.250 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:07.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:07.330 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:07.363 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:07.363 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:07.363 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:07.385 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:07.385 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:07.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:07.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.571+0000 7fc68a8fc640 1 Processor -- start 2024-04-25T10:06:07.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.571+0000 7fc68a8fc640 1 -- start start 2024-04-25T10:06:07.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc68a8fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc6840c6df0 0x7fc6840d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc68a8fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6840d1c00 0x7fc6840ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc68a8fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68415dfe0 con 0x7fc68415dc00 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc68a8fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840d2090 con 0x7fc6840c6df0 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc68a8fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840d00c0 con 0x7fc6840d1c00 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc6837fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc6840c6df0 0x7fc6840d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc683fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc68415dc00 0x7fc6840ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45224/0 (socket says 172.21.15.148:45224) 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc683fff640 1 -- 172.21.15.148:0/116605251 learned_addr learned my addr 172.21.15.148:0/116605251 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc688e99640 1 --2- 172.21.15.148:0/116605251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6840d1c00 0x7fc6840ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc6837fe640 1 -- 172.21.15.148:0/116605251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6840d1c00 msgr2=0x7fc6840ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc6837fe640 1 --2- 172.21.15.148:0/116605251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6840d1c00 0x7fc6840ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc6837fe640 1 -- 172.21.15.148:0/116605251 >> v1:172.21.15.148:6789/0 conn(0x7fc68415dc00 legacy=0x7fc6840ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc6837fe640 1 -- 172.21.15.148:0/116605251 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840d03a0 con 0x7fc6840c6df0 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.572+0000 7fc682ffd640 1 -- 172.21.15.148:0/116605251 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480971336 0 0) 0x7fc674001110 con 0x7fc68415dc00 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.573+0000 7fc6837fe640 1 --2- 172.21.15.148:0/116605251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc6840d16c0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097a0 tx=0x7fc67802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f189fb196d6a101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.573+0000 7fc682ffd640 1 -- 172.21.15.148:0/116605251 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678037070 con 0x7fc6840c6df0 2024-04-25T10:06:07.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.573+0000 7fc682ffd640 1 -- 172.21.15.148:0/116605251 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678032440 con 0x7fc6840c6df0 2024-04-25T10:06:07.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.573+0000 7fc682ffd640 1 -- 172.21.15.148:0/116605251 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678032810 con 0x7fc6840c6df0 2024-04-25T10:06:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 -- 172.21.15.148:0/116605251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 msgr2=0x7fc6840d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/116605251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc6840d16c0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097a0 tx=0x7fc67802a180 comp rx=0 tx=0).stop 2024-04-25T10:06:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 -- 172.21.15.148:0/116605251 shutdown_connections 2024-04-25T10:06:07.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/116605251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6840d1c00 0x7fc6840ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/116605251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc6840d16c0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 -- 172.21.15.148:0/116605251 >> 172.21.15.148:0/116605251 conn(0x7fc68406f5d0 msgr2=0x7fc68406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 -- 172.21.15.148:0/116605251 shutdown_connections 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.574+0000 7fc68a8fc640 1 -- 172.21.15.148:0/116605251 wait complete. 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.575+0000 7fc68a8fc640 1 Processor -- start 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.575+0000 7fc68a8fc640 1 -- start start 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc68406d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6840d1c00 0x7fc6840b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc68415dc00 0x7fc684146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cc130 con 0x7fc6840c6df0 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cc720 con 0x7fc6840d1c00 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840cfd90 con 0x7fc68415dc00 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc68406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc68406d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60772/0 (socket says 172.21.15.148:60772) 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 -- 172.21.15.148:0/3164233639 learned_addr learned my addr 172.21.15.148:0/3164233639 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc6837fe640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6840d1c00 0x7fc6840b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc688e99640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc68415dc00 0x7fc684146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 -- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc68415dc00 msgr2=0x7fc684146340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc68415dc00 0x7fc684146340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 -- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6840d1c00 msgr2=0x7fc6840b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6840d1c00 0x7fc6840b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 -- 172.21.15.148:0/3164233639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840b7340 con 0x7fc6840c6df0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc683fff640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc68406d090 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c430 tx=0x7fc67400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740074e0 con 0x7fc6840c6df0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6740047e0 con 0x7fc6840c6df0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6840b7060 con 0x7fc6840c6df0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674004bd0 con 0x7fc6840c6df0 2024-04-25T10:06:07.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.576+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68406c890 con 0x7fc6840c6df0 2024-04-25T10:06:07.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.577+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc684073990 con 0x7fc6840c6df0 2024-04-25T10:06:07.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.578+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc674011070 con 0x7fc6840c6df0 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.578+0000 7fc680ff9640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc67006b4d0 0x7fc67006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.578+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fc67408a090 con 0x7fc6840c6df0 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.578+0000 7fc6837fe640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc67006b4d0 0x7fc67006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.579+0000 7fc6837fe640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc67006b4d0 0x7fc67006d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc678034000 tx=0x7fc67802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.583+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc68415dfe0 con 0x7fc6840c6df0 2024-04-25T10:06:07.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.752+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6840c66c0 con 0x7fc6840c6df0 2024-04-25T10:06:07.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.753+0000 7fc680ff9640 1 -- 172.21.15.148:0/3164233639 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17232 (secure 0 0 0) 0x7fc674058cf0 con 0x7fc6840c6df0 2024-04-25T10:06:07.759 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:07.759 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":173,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:06.807921+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.2a","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc67006b4d0 msgr2=0x7fc67006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc67006b4d0 0x7fc67006d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc678034000 tx=0x7fc67802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 msgr2=0x7fc68406d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc68406d090 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c430 tx=0x7fc67400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 shutdown_connections 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc67006b4d0 0x7fc67006d990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc68415dc00 0x7fc684146340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6840d1c00 0x7fc6840b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 --2- 172.21.15.148:0/3164233639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6840c6df0 0x7fc68406d090 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 >> 172.21.15.148:0/3164233639 conn(0x7fc68406f5d0 msgr2=0x7fc6840b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 shutdown_connections 2024-04-25T10:06:07.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.755+0000 7fc68a8fc640 1 -- 172.21.15.148:0/3164233639 wait complete. 2024-04-25T10:06:07.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:26.693014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:27.457155+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:07.773 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:07.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.868+0000 7f045cba5640 1 Processor -- start 2024-04-25T10:06:07.874 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.868+0000 7f045cba5640 1 -- start start 2024-04-25T10:06:07.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f045cba5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f045815eec0 0x7f04581689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f045cba5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04580d1770 con 0x7f0458070dc0 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f045cba5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0458162740 con 0x7f045806ccd0 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f045cba5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04580b0eb0 con 0x7f045815eec0 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f0456d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f045815eec0 0x7f04581689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f045659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0458070dc0 0x7f04580711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45234/0 (socket says 172.21.15.148:45234) 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f045659c640 1 -- 172.21.15.148:0/2167284724 learned_addr learned my addr 172.21.15.148:0/2167284724 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f0456d9d640 1 -- 172.21.15.148:0/2167284724 >> v1:172.21.15.188:6789/0 conn(0x7f045806ccd0 legacy=0x7f04580b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f0456d9d640 1 -- 172.21.15.148:0/2167284724 >> v1:172.21.15.148:6789/0 conn(0x7f0458070dc0 legacy=0x7f04580711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.869+0000 7f0456d9d640 1 -- 172.21.15.148:0/2167284724 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458161980 con 0x7f045815eec0 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.870+0000 7f0456d9d640 1 --2- 172.21.15.148:0/2167284724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045815eec0 0x7f04581689d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f04500097a0 tx=0x7f045002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bacd1f6cf7c303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.870+0000 7f045559a640 1 -- 172.21.15.148:0/2167284724 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450037070 con 0x7f045815eec0 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.870+0000 7f045559a640 1 -- 172.21.15.148:0/2167284724 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045002adc0 con 0x7f045815eec0 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.870+0000 7f045559a640 1 -- 172.21.15.148:0/2167284724 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450032660 con 0x7f045815eec0 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 -- 172.21.15.148:0/2167284724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045815eec0 msgr2=0x7f04581689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 --2- 172.21.15.148:0/2167284724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045815eec0 0x7f04581689d0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f04500097a0 tx=0x7f045002a0b0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 -- 172.21.15.148:0/2167284724 shutdown_connections 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 --2- 172.21.15.148:0/2167284724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f045815eec0 0x7f04581689d0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 -- 172.21.15.148:0/2167284724 >> 172.21.15.148:0/2167284724 conn(0x7f04580b3530 msgr2=0x7f04580b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 -- 172.21.15.148:0/2167284724 shutdown_connections 2024-04-25T10:06:07.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.871+0000 7f045cba5640 1 -- 172.21.15.148:0/2167284724 wait complete. 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.872+0000 7f045cba5640 1 Processor -- start 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.872+0000 7f045cba5640 1 -- start start 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045cba5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f045806ccd0 0x7f04580b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045cba5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 0x7f04581510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045cba5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f045815eec0 0x7f0458151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045cba5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580b10f0 con 0x7f0458070dc0 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045cba5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581622b0 con 0x7f045806ccd0 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045cba5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0458161f40 con 0x7f045815eec0 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 0x7f04581510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 0x7f04581510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60800/0 (socket says 172.21.15.148:60800) 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 -- 172.21.15.148:0/2202332895 learned_addr learned my addr 172.21.15.148:0/2202332895 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:07.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f045659c640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f045806ccd0 0x7f04580b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0456d9d640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f045815eec0 0x7f0458151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 -- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f045815eec0 msgr2=0x7f0458151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f045815eec0 0x7f0458151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 -- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f045806ccd0 msgr2=0x7f04580b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f045806ccd0 0x7f04580b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 -- 172.21.15.148:0/2202332895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04580a9520 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0456d9d640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f045815eec0 0x7f0458151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.873+0000 7f0455d9b640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 0x7f04581510c0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f044c00c430 tx=0x7f044c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.874+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c011790 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.874+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c017070 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.874+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04580a8760 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.874+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c011bb0 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.874+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04580a8d20 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.875+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045806da80 con 0x7f0458070dc0 2024-04-25T10:06:07.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.875+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f044c01c460 con 0x7f0458070dc0 2024-04-25T10:06:07.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.876+0000 7f0446ffd640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.876+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f044c08a270 con 0x7f0458070dc0 2024-04-25T10:06:07.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.876+0000 7f045659c640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.876+0000 7f045659c640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f04480012e0 tx=0x7f0448009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:07.880+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f045806da80 con 0x7f0458070dc0 2024-04-25T10:06:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.049+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0458073a00 con 0x7f0458070dc0 2024-04-25T10:06:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.049+0000 7f0446ffd640 1 -- 172.21.15.148:0/2202332895 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7f044c058f60 con 0x7f0458070dc0 2024-04-25T10:06:08.055 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f043806b4d0 msgr2=0x7f043806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f04480012e0 tx=0x7f0448009040 comp rx=0 tx=0).stop 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 msgr2=0x7f04581510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 0x7f04581510c0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f044c00c430 tx=0x7f044c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 shutdown_connections 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f045815eec0 0x7f0458151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0458070dc0 0x7f04581510c0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 --2- 172.21.15.148:0/2202332895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f045806ccd0 0x7f04580b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 >> 172.21.15.148:0/2202332895 conn(0x7f04580b3530 msgr2=0x7f045815e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 shutdown_connections 2024-04-25T10:06:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.051+0000 7f045cba5640 1 -- 172.21.15.148:0/2202332895 wait complete. 2024-04-25T10:06:08.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:08.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.163+0000 7ff6071f4640 1 Processor -- start 2024-04-25T10:06:08.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.163+0000 7ff6071f4640 1 -- start start 2024-04-25T10:06:08.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.164+0000 7ff6071f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7ff60015dc20 0x7ff6000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.164+0000 7ff6071f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6000d32f0 0x7ff6000cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.164+0000 7ff6071f4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60015e000 con 0x7ff600161710 2024-04-25T10:06:08.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.164+0000 7ff6071f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000d1de0 con 0x7ff60015dc20 2024-04-25T10:06:08.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.164+0000 7ff6071f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff600161c60 con 0x7ff6000d32f0 2024-04-25T10:06:08.171 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.164+0000 7ff5fffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff600161710 0x7ff60006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45250/0 (socket says 172.21.15.148:45250) 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff5fffff640 1 -- 172.21.15.148:0/3254755384 learned_addr learned my addr 172.21.15.148:0/3254755384 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff604f90640 1 --2- 172.21.15.148:0/3254755384 >> v2:172.21.15.148:3300/0 conn(0x7ff60015dc20 0x7ff6000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff605791640 1 --2- 172.21.15.148:0/3254755384 >> v2:172.21.15.188:3300/0 conn(0x7ff6000d32f0 0x7ff6000cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff5ff7fe640 1 -- 172.21.15.148:0/3254755384 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583897061 0 0) 0x7ff60015e000 con 0x7ff600161710 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff5ff7fe640 1 -- 172.21.15.148:0/3254755384 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0003520 con 0x7ff600161710 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff605791640 1 -- 172.21.15.148:0/3254755384 >> v2:172.21.15.148:3300/0 conn(0x7ff60015dc20 msgr2=0x7ff6000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff605791640 1 --2- 172.21.15.148:0/3254755384 >> v2:172.21.15.148:3300/0 conn(0x7ff60015dc20 0x7ff6000ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff605791640 1 -- 172.21.15.148:0/3254755384 >> v1:172.21.15.148:6789/0 conn(0x7ff600161710 legacy=0x7ff60006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff605791640 1 -- 172.21.15.148:0/3254755384 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600161f40 con 0x7ff6000d32f0 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.165+0000 7ff604f90640 1 --2- 172.21.15.148:0/3254755384 >> v2:172.21.15.148:3300/0 conn(0x7ff60015dc20 0x7ff6000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff605791640 1 --2- 172.21.15.148:0/3254755384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6000d32f0 0x7ff6000cebb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b6910 tx=0x7ff5f802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f1ceb0caabafe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff5ff7fe640 1 -- 172.21.15.148:0/3254755384 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8007740 con 0x7ff6000d32f0 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff5ff7fe640 1 -- 172.21.15.148:0/3254755384 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f80078e0 con 0x7ff6000d32f0 2024-04-25T10:06:08.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff5ff7fe640 1 -- 172.21.15.148:0/3254755384 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8007cb0 con 0x7ff6000d32f0 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 -- 172.21.15.148:0/3254755384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6000d32f0 msgr2=0x7ff6000cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 --2- 172.21.15.148:0/3254755384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6000d32f0 0x7ff6000cebb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b6910 tx=0x7ff5f802a680 comp rx=0 tx=0).stop 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 -- 172.21.15.148:0/3254755384 shutdown_connections 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 --2- 172.21.15.148:0/3254755384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6000d32f0 0x7ff6000cebb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 --2- 172.21.15.148:0/3254755384 >> v2:172.21.15.148:3300/0 conn(0x7ff60015dc20 0x7ff6000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 -- 172.21.15.148:0/3254755384 >> 172.21.15.148:0/3254755384 conn(0x7ff6000d0420 msgr2=0x7ff6000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 -- 172.21.15.148:0/3254755384 shutdown_connections 2024-04-25T10:06:08.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.166+0000 7ff6071f4640 1 -- 172.21.15.148:0/3254755384 wait complete. 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.168+0000 7ff6071f4640 1 Processor -- start 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.168+0000 7ff6071f4640 1 -- start start 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.168+0000 7ff6071f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 0x7ff6001f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60015dc20 0x7ff6001f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600161710 0x7ff6000cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b5e50 con 0x7ff6000d32f0 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600162500 con 0x7ff60015dc20 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b5fd0 con 0x7ff600161710 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 0x7ff6001f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 0x7ff6001f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60814/0 (socket says 172.21.15.148:60814) 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 -- 172.21.15.148:0/2800537388 learned_addr learned my addr 172.21.15.148:0/2800537388 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff605791640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600161710 0x7ff6000cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff5fffff640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60015dc20 0x7ff6001f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 -- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600161710 msgr2=0x7ff6000cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600161710 0x7ff6000cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 -- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60015dc20 msgr2=0x7ff6001f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60015dc20 0x7ff6001f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 -- 172.21.15.148:0/2800537388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6001f3bc0 con 0x7ff6000d32f0 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff604f90640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 0x7ff6001f3680 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e990 tx=0x7ff5f400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4017070 con 0x7ff6000d32f0 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f400c9e0 con 0x7ff6000d32f0 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6000cc380 con 0x7ff6000d32f0 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400cdb0 con 0x7ff6000d32f0 2024-04-25T10:06:08.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.169+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6000cc940 con 0x7ff6000d32f0 2024-04-25T10:06:08.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.170+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6000cf660 con 0x7ff6000d32f0 2024-04-25T10:06:08.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.171+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5f401a070 con 0x7ff6000d32f0 2024-04-25T10:06:08.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.171+0000 7ff5fd7fa640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.171+0000 7ff5fffff640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.171+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7ff5f408a250 con 0x7ff6000d32f0 2024-04-25T10:06:08.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.172+0000 7ff5fffff640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff6001f7d90 tx=0x7ff5ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.175+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6000cf660 con 0x7ff6000d32f0 2024-04-25T10:06:08.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.345+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6000cf380 con 0x7ff6000d32f0 2024-04-25T10:06:08.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.345+0000 7ff5fd7fa640 1 -- 172.21.15.148:0/2800537388 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6000cf380 con 0x7ff6000d32f0 2024-04-25T10:06:08.351 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:06:08.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e406b4d0 msgr2=0x7ff5e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff6001f7d90 tx=0x7ff5ec009040 comp rx=0 tx=0).stop 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 msgr2=0x7ff6001f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 0x7ff6001f3680 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400e990 tx=0x7ff5f400ee50 comp rx=0 tx=0).stop 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 shutdown_connections 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600161710 0x7ff6000cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff60015dc20 0x7ff6001f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 --2- 172.21.15.148:0/2800537388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6000d32f0 0x7ff6001f3680 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 >> 172.21.15.148:0/2800537388 conn(0x7ff6000d0420 msgr2=0x7ff60015bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 shutdown_connections 2024-04-25T10:06:08.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.347+0000 7ff6071f4640 1 -- 172.21.15.148:0/2800537388 wait complete. 2024-04-25T10:06:08.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:08.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.458+0000 7f7111e4d640 1 Processor -- start 2024-04-25T10:06:08.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.458+0000 7f7111e4d640 1 -- start start 2024-04-25T10:06:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f7111e4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c0cd8a0 0x7f710c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f7111e4d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710c15e6e0 con 0x7f710c150b90 2024-04-25T10:06:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f7111e4d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710c15e9c0 con 0x7f710c0cd4c0 2024-04-25T10:06:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f7111e4d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15a9c0 con 0x7f710c0cd8a0 2024-04-25T10:06:08.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f710bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f710c150b90 0x7f710c071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45260/0 (socket says 172.21.15.148:45260) 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f710bfff640 1 -- 172.21.15.148:0/615936803 learned_addr learned my addr 172.21.15.148:0/615936803 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f710affd640 1 --2- 172.21.15.148:0/615936803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c0cd8a0 0x7f710c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160053879 0 0) 0x7f710c15e6e0 con 0x7f710c150b90 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.459+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f4003520 con 0x7f710c150b90 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523152294 0 0) 0x7f710c15e9c0 con 0x7f710c0cd4c0 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f710c15e6e0 con 0x7f710c0cd4c0 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1348637632 0 0) 0x7f70f4003520 con 0x7f710c150b90 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710affd640 1 -- 172.21.15.148:0/615936803 >> v1:172.21.15.188:6789/0 conn(0x7f710c0cd4c0 legacy=0x7f710c15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710affd640 1 -- 172.21.15.148:0/615936803 >> v1:172.21.15.148:6789/0 conn(0x7f710c150b90 legacy=0x7f710c071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710affd640 1 -- 172.21.15.148:0/615936803 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c15aca0 con 0x7f710c0cd8a0 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.460+0000 7f710affd640 1 --2- 172.21.15.148:0/615936803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c0cd8a0 0x7f710c15b1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f71000097d0 tx=0x7f710002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fee28fd59d9433 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100037070 con 0x7f710c0cd8a0 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710002ac30 con 0x7f710c0cd8a0 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f710a7fc640 1 -- 172.21.15.148:0/615936803 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040019f0 con 0x7f710c0cd8a0 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 -- 172.21.15.148:0/615936803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c0cd8a0 msgr2=0x7f710c15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 --2- 172.21.15.148:0/615936803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c0cd8a0 0x7f710c15b1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f71000097d0 tx=0x7f710002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 -- 172.21.15.148:0/615936803 shutdown_connections 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 --2- 172.21.15.148:0/615936803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c0cd8a0 0x7f710c15b1c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 -- 172.21.15.148:0/615936803 >> 172.21.15.148:0/615936803 conn(0x7f710c06abe0 msgr2=0x7f710c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 -- 172.21.15.148:0/615936803 shutdown_connections 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.461+0000 7f7111e4d640 1 -- 172.21.15.148:0/615936803 wait complete. 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.462+0000 7f7111e4d640 1 Processor -- start 2024-04-25T10:06:08.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.462+0000 7f7111e4d640 1 -- start start 2024-04-25T10:06:08.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f7111e4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f710c0cd4c0 0x7f710c13eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f7111e4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 0x7f710c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f7111e4d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c150b90 0x7f710c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f7111e4d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c06eca0 con 0x7f710c0cd8a0 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f7111e4d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15e370 con 0x7f710c0cd4c0 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f7111e4d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c06ee20 con 0x7f710c150b90 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f710affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 0x7f710c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f710affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 0x7f710c0b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60828/0 (socket says 172.21.15.148:60828) 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f710affd640 1 -- 172.21.15.148:0/2037879908 learned_addr learned my addr 172.21.15.148:0/2037879908 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f710b7fe640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f710c0cd4c0 0x7f710c13eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.463+0000 7f710bfff640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c150b90 0x7f710c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f710affd640 1 -- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c150b90 msgr2=0x7f710c0b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f710affd640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c150b90 0x7f710c0b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f710affd640 1 -- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f710c0cd4c0 msgr2=0x7f710c13eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f710affd640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f710c0cd4c0 0x7f710c13eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f710affd640 1 -- 172.21.15.148:0/2037879908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c157090 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f710affd640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 0x7f710c0b0be0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f7100035040 tx=0x7f71000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100041020 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710003d640 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100037040 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c157370 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.464+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c157930 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.465+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c073a00 con 0x7f710c0cd8a0 2024-04-25T10:06:08.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.465+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7100044020 con 0x7f710c0cd8a0 2024-04-25T10:06:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.466+0000 7f70f3fff640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70f806b4d0 0x7f70f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.466+0000 7f710b7fe640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70f806b4d0 0x7f70f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.466+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f71000aa800 con 0x7f710c0cd8a0 2024-04-25T10:06:08.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.466+0000 7f710b7fe640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70f806b4d0 0x7f70f806d990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f70fc007fd0 tx=0x7f70fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.470+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f710004b030 con 0x7f710c0cd8a0 2024-04-25T10:06:08.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.639+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f710c0ccb10 con 0x7f710c0cd8a0 2024-04-25T10:06:08.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.639+0000 7f70f3fff640 1 -- 172.21.15.148:0/2037879908 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7100078e10 con 0x7f710c0cd8a0 2024-04-25T10:06:08.645 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70f806b4d0 msgr2=0x7f70f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70f806b4d0 0x7f70f806d990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f70fc007fd0 tx=0x7f70fc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 msgr2=0x7f710c0b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 0x7f710c0b0be0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f7100035040 tx=0x7f71000046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 shutdown_connections 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70f806b4d0 0x7f70f806d990 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f710c150b90 0x7f710c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f710c0cd8a0 0x7f710c0b0be0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 --2- 172.21.15.148:0/2037879908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f710c0cd4c0 0x7f710c13eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 >> 172.21.15.148:0/2037879908 conn(0x7f710c06abe0 msgr2=0x7f710c15be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 shutdown_connections 2024-04-25T10:06:08.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.641+0000 7f7111e4d640 1 -- 172.21.15.148:0/2037879908 wait complete. 2024-04-25T10:06:08.658 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:08.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.749+0000 7ffbd7122640 1 Processor -- start 2024-04-25T10:06:08.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.749+0000 7ffbd7122640 1 -- start start 2024-04-25T10:06:08.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.750+0000 7ffbd7122640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffbd015bb50 0x7ffbd00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.750+0000 7ffbd7122640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd00c75e0 con 0x7ffbd00cfd10 2024-04-25T10:06:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.750+0000 7ffbd7122640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd00c78c0 con 0x7ffbd00c70d0 2024-04-25T10:06:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.750+0000 7ffbd7122640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd00c8680 con 0x7ffbd015bb50 2024-04-25T10:06:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.750+0000 7ffbd56bf640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffbd00cfd10 0x7ffbd00cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45264/0 (socket says 172.21.15.148:45264) 2024-04-25T10:06:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.750+0000 7ffbd56bf640 1 -- 172.21.15.148:0/4095338992 learned_addr learned my addr 172.21.15.148:0/4095338992 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:08.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915967901 0 0) 0x7ffbd00c75e0 con 0x7ffbd00cfd10 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb8003520 con 0x7ffbd00cfd10 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/4095338992 >> v2:172.21.15.188:3300/0 conn(0x7ffbd015bb50 0x7ffbd00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270210453 0 0) 0x7ffbb8003520 con 0x7ffbd00cfd10 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 >> v2:172.21.15.188:3300/0 conn(0x7ffbd015bb50 msgr2=0x7ffbd00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 --2- 172.21.15.148:0/4095338992 >> v2:172.21.15.188:3300/0 conn(0x7ffbd015bb50 0x7ffbd00c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 >> v1:172.21.15.188:6789/0 conn(0x7ffbd00c70d0 legacy=0x7ffbd00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00c7ba0 con 0x7ffbd00cfd10 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ffbc80019f0 con 0x7ffbd00cfd10 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 >> v1:172.21.15.148:6789/0 conn(0x7ffbd00cfd10 legacy=0x7ffbd00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb80039b0 0x7ffbb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb8004800 0x7ffbb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80051c0 0x7ffbb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd00c8680 con 0x7ffbb80039b0 2024-04-25T10:06:08.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb800c070 con 0x7ffbb8004800 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb800c270 con 0x7ffbb80051c0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb80039b0 0x7ffbb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbcffff640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb8004800 0x7ffbb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd56bf640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80051c0 0x7ffbb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 -- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80051c0 msgr2=0x7ffbb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80051c0 0x7ffbb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 -- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb8004800 msgr2=0x7ffbb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbb8004800 0x7ffbb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd4ebe640 1 -- 172.21.15.148:0/4095338992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb8003520 con 0x7ffbb80039b0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd56bf640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80051c0 0x7ffbb800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.751+0000 7ffbd56bf640 1 -- 172.21.15.148:0/4095338992 reap_dead start 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.752+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb80039b0 0x7ffbb8003dd0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009ac0 tx=0x7ffbb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadfaf995b870b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.752+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0036070 con 0x7ffbb80039b0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.752+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0031400 con 0x7ffbb80039b0 2024-04-25T10:06:08.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.752+0000 7ffbcf7fe640 1 -- 172.21.15.148:0/4095338992 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0009690 con 0x7ffbb80039b0 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 -- 172.21.15.148:0/4095338992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb80039b0 msgr2=0x7ffbb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb80039b0 0x7ffbb8003dd0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009ac0 tx=0x7ffbb8005830 comp rx=0 tx=0).stop 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 -- 172.21.15.148:0/4095338992 shutdown_connections 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 --2- 172.21.15.148:0/4095338992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbb80039b0 0x7ffbb8003dd0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 -- 172.21.15.148:0/4095338992 >> 172.21.15.148:0/4095338992 conn(0x7ffbd006f500 msgr2=0x7ffbd006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 -- 172.21.15.148:0/4095338992 shutdown_connections 2024-04-25T10:06:08.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.753+0000 7ffbd7122640 1 -- 172.21.15.148:0/4095338992 wait complete. 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.754+0000 7ffbd7122640 1 Processor -- start 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.754+0000 7ffbd7122640 1 -- start start 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd7122640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80039b0 0x7ffbd00b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd7122640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd01544a0 0x7ffbd014a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd7122640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 0x7ffbd00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd7122640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015f6d0 con 0x7ffbd01f16f0 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd7122640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd00c8160 con 0x7ffbd01544a0 2024-04-25T10:06:08.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd7122640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015f850 con 0x7ffbb80039b0 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 0x7ffbd00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 0x7ffbd00c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60836/0 (socket says 172.21.15.148:60836) 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 -- 172.21.15.148:0/2703263912 learned_addr learned my addr 172.21.15.148:0/2703263912 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbcffff640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd01544a0 0x7ffbd014a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80039b0 0x7ffbd00b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 -- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80039b0 msgr2=0x7ffbd00b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80039b0 0x7ffbd00b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 -- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd01544a0 msgr2=0x7ffbd014a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd01544a0 0x7ffbd014a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 -- 172.21.15.148:0/2703263912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd015d200 con 0x7ffbd01f16f0 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbcffff640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd01544a0 0x7ffbd014a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80039b0 0x7ffbd00b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbd56bf640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 0x7ffbd00c9ee0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00b3400 tx=0x7ffbc800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.755+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800eb30 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.756+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc800ecd0 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.756+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd015d470 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.756+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80146a0 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.756+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd015da30 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.757+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd0073a00 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.757+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbc8014840 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.757+0000 7ffbcd7fa640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.758+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7ffbc808a3e0 con 0x7ffbd01f16f0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.758+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.758+0000 7ffbd4ebe640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0033000 tx=0x7ffbc0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.767 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.762+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbc8059040 con 0x7ffbd01f16f0 2024-04-25T10:06:08.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.946+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffbd00cf150 con 0x7ffbd01f16f0 2024-04-25T10:06:08.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.946+0000 7ffbcd7fa640 1 -- 172.21.15.148:0/2703263912 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v173) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffbc8058940 con 0x7ffbd01f16f0 2024-04-25T10:06:08.952 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 msgr2=0x7ffbbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0033000 tx=0x7ffbc0008030 comp rx=0 tx=0).stop 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 msgr2=0x7ffbd00c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 0x7ffbd00c9ee0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00b3400 tx=0x7ffbc800e530 comp rx=0 tx=0).stop 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 shutdown_connections 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffbbc06b4d0 0x7ffbbc06d990 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbd01f16f0 0x7ffbd00c9ee0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbd01544a0 0x7ffbd014a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 --2- 172.21.15.148:0/2703263912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbb80039b0 0x7ffbd00b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.948+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 >> 172.21.15.148:0/2703263912 conn(0x7ffbd006f500 msgr2=0x7ffbd0070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.949+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 shutdown_connections 2024-04-25T10:06:08.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:08.949+0000 7ffbd7122640 1 -- 172.21.15.148:0/2703263912 wait complete. 2024-04-25T10:06:08.965 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:06:08.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:08.965 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:09.010 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:06:09.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:09.011 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:09.034 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:09.034 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:09.052 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:09.052 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:09.052 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:09.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:09.090 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:09.090 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:09.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:09.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.224+0000 7f983ea4a640 1 Processor -- start 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.224+0000 7f983ea4a640 1 -- start start 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f983ea4a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f983815da70 0x7f9838161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f983ea4a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9838161b80 0x7f98380d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f983ea4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9838162020 con 0x7f98380b9730 2024-04-25T10:06:09.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f983ea4a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98381625e0 con 0x7f9838161b80 2024-04-25T10:06:09.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f983ea4a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9838162300 con 0x7f983815da70 2024-04-25T10:06:09.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f98377fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f98380b9730 0x7f98380ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45278/0 (socket says 172.21.15.148:45278) 2024-04-25T10:06:09.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f983cfe7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9838161b80 0x7f98380d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.225+0000 7f98377fe640 1 -- 172.21.15.148:0/662811854 learned_addr learned my addr 172.21.15.148:0/662811854 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:09.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f9837fff640 1 --2- 172.21.15.148:0/662811854 >> v2:172.21.15.188:3300/0 conn(0x7f983815da70 0x7f9838161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f983cfe7640 1 -- 172.21.15.148:0/662811854 >> v2:172.21.15.188:3300/0 conn(0x7f983815da70 msgr2=0x7f9838161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f983cfe7640 1 --2- 172.21.15.148:0/662811854 >> v2:172.21.15.188:3300/0 conn(0x7f983815da70 0x7f9838161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f983cfe7640 1 -- 172.21.15.148:0/662811854 >> v1:172.21.15.148:6789/0 conn(0x7f98380b9730 legacy=0x7f98380ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f983cfe7640 1 -- 172.21.15.148:0/662811854 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380d1af0 con 0x7f9838161b80 2024-04-25T10:06:09.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f983cfe7640 1 --2- 172.21.15.148:0/662811854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838161b80 0x7f98380d22f0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f98380b61a0 tx=0x7f983002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f69c957b69da3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.226+0000 7f9836ffd640 1 -- 172.21.15.148:0/662811854 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830007740 con 0x7f9838161b80 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f9836ffd640 1 -- 172.21.15.148:0/662811854 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98300078e0 con 0x7f9838161b80 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f9836ffd640 1 -- 172.21.15.148:0/662811854 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830007cb0 con 0x7f9838161b80 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 -- 172.21.15.148:0/662811854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838161b80 msgr2=0x7f98380d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 --2- 172.21.15.148:0/662811854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838161b80 0x7f98380d22f0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f98380b61a0 tx=0x7f983002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 -- 172.21.15.148:0/662811854 shutdown_connections 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 --2- 172.21.15.148:0/662811854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9838161b80 0x7f98380d22f0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 --2- 172.21.15.148:0/662811854 >> v2:172.21.15.188:3300/0 conn(0x7f983815da70 0x7f9838161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 -- 172.21.15.148:0/662811854 >> 172.21.15.148:0/662811854 conn(0x7f9838061a20 msgr2=0x7f9838061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 -- 172.21.15.148:0/662811854 shutdown_connections 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.227+0000 7f983ea4a640 1 -- 172.21.15.148:0/662811854 wait complete. 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.228+0000 7f983ea4a640 1 Processor -- start 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 -- start start 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f983815da70 0x7f98380caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9838161b80 0x7f983814cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 0x7f983814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380ced60 con 0x7f983815b150 2024-04-25T10:06:09.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983815de50 con 0x7f983815da70 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983ea4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380ceee0 con 0x7f9838161b80 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983cfe7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 0x7f983814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983cfe7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 0x7f983814d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60846/0 (socket says 172.21.15.148:60846) 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.229+0000 7f983cfe7640 1 -- 172.21.15.148:0/3641937756 learned_addr learned my addr 172.21.15.148:0/3641937756 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f9837fff640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f983815da70 0x7f98380caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f98377fe640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9838161b80 0x7f983814cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983cfe7640 1 -- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9838161b80 msgr2=0x7f983814cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983cfe7640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9838161b80 0x7f983814cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983cfe7640 1 -- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f983815da70 msgr2=0x7f98380caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983cfe7640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f983815da70 0x7f98380caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983cfe7640 1 -- 172.21.15.148:0/3641937756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380caf90 con 0x7f983815b150 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f98377fe640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9838161b80 0x7f983814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983cfe7640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 0x7f983814d5d0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f983002abe0 tx=0x7f9830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983003f070 con 0x7f983815b150 2024-04-25T10:06:09.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9830031bc0 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830044460 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983815f4c0 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.230+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983815fa80 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.231+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838073990 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.231+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9830007790 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.232+0000 7f9834ff9640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f982406b500 0x7f982406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.232+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f98300abbf0 con 0x7f983815b150 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.232+0000 7f9837fff640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f982406b500 0x7f982406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.232+0000 7f9837fff640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f982406b500 0x7f982406d9c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f982c006fd0 tx=0x7f982c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.236+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f983007a850 con 0x7f983815b150 2024-04-25T10:06:09.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.442+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98380b2940 con 0x7f983815b150 2024-04-25T10:06:09.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.442+0000 7f9834ff9640 1 -- 172.21.15.148:0/3641937756 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f983007a150 con 0x7f983815b150 2024-04-25T10:06:09.448 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:09.448 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:06:09.449 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:06:09.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f982406b500 msgr2=0x7f982406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f982406b500 0x7f982406d9c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f982c006fd0 tx=0x7f982c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:09.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 msgr2=0x7f983814d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 0x7f983814d5d0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f983002abe0 tx=0x7f9830004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 shutdown_connections 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f982406b500 0x7f982406d9c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f983815b150 0x7f983814d5d0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9838161b80 0x7f983814cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 --2- 172.21.15.148:0/3641937756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f983815da70 0x7f98380caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.444+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 >> 172.21.15.148:0/3641937756 conn(0x7f9838061a20 msgr2=0x7f98380b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.445+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 shutdown_connections 2024-04-25T10:06:09.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.445+0000 7f983ea4a640 1 -- 172.21.15.148:0/3641937756 wait complete. 2024-04-25T10:06:09.461 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:09.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.552+0000 7f01af01b640 1 Processor -- start 2024-04-25T10:06:09.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.552+0000 7f01af01b640 1 -- start start 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01af01b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01a80af720 0x7f01a80afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01af01b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a80d02f0 0x7f01a80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01af01b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a806fe00 con 0x7f01a8158220 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01af01b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80b3670 con 0x7f01a80d02f0 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01af01b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a8070a10 con 0x7f01a80af720 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01ad5b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a80d02f0 0x7f01a80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01ad5b8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f01a80d02f0 0x7f01a80d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60850/0 (socket says 172.21.15.148:60850) 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.553+0000 7f01ad5b8640 1 -- 172.21.15.148:0/621375180 learned_addr learned my addr 172.21.15.148:0/621375180 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.554+0000 7f01a7fff640 1 --2- 172.21.15.148:0/621375180 >> v2:172.21.15.188:3300/0 conn(0x7f01a80af720 0x7f01a80afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.554+0000 7f01ad5b8640 1 -- 172.21.15.148:0/621375180 >> v2:172.21.15.188:3300/0 conn(0x7f01a80af720 msgr2=0x7f01a80afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.554+0000 7f01ad5b8640 1 --2- 172.21.15.148:0/621375180 >> v2:172.21.15.188:3300/0 conn(0x7f01a80af720 0x7f01a80afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.554+0000 7f01ad5b8640 1 -- 172.21.15.148:0/621375180 >> v1:172.21.15.188:6789/0 conn(0x7f01a8158220 legacy=0x7f01a815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.554+0000 7f01ad5b8640 1 -- 172.21.15.148:0/621375180 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80700e0 con 0x7f01a80d02f0 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.554+0000 7f01ad5b8640 1 --2- 172.21.15.148:0/621375180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80d0720 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f01a80cfc70 tx=0x7f01a002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b1c418c6e6440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01a77fe640 1 -- 172.21.15.148:0/621375180 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0036070 con 0x7f01a80d02f0 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01a77fe640 1 -- 172.21.15.148:0/621375180 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0031600 con 0x7f01a80d02f0 2024-04-25T10:06:09.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01a77fe640 1 -- 172.21.15.148:0/621375180 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a00319d0 con 0x7f01a80d02f0 2024-04-25T10:06:09.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 -- 172.21.15.148:0/621375180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 msgr2=0x7f01a80d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 --2- 172.21.15.148:0/621375180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80d0720 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7f01a80cfc70 tx=0x7f01a002a730 comp rx=0 tx=0).stop 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 -- 172.21.15.148:0/621375180 shutdown_connections 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 --2- 172.21.15.148:0/621375180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80d0720 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 --2- 172.21.15.148:0/621375180 >> v2:172.21.15.188:3300/0 conn(0x7f01a80af720 0x7f01a80afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 -- 172.21.15.148:0/621375180 >> 172.21.15.148:0/621375180 conn(0x7f01a80b2370 msgr2=0x7f01a80b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 -- 172.21.15.148:0/621375180 shutdown_connections 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.555+0000 7f01af01b640 1 -- 172.21.15.148:0/621375180 wait complete. 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.556+0000 7f01af01b640 1 Processor -- start 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.556+0000 7f01af01b640 1 -- start start 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01af01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01a80af720 0x7f01a815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01af01b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01af01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01a8158220 0x7f01a814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01af01b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8158600 con 0x7f01a80d02f0 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01af01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80706a0 con 0x7f01a80af720 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01af01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8072960 con 0x7f01a8158220 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01a7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01a7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60852/0 (socket says 172.21.15.148:60852) 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.557+0000 7f01a7fff640 1 -- 172.21.15.148:0/713590449 learned_addr learned my addr 172.21.15.148:0/713590449 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01acdb7640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01a80af720 0x7f01a815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01ad5b8640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01a8158220 0x7f01a814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a7fff640 1 -- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01a8158220 msgr2=0x7f01a814ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a7fff640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01a8158220 0x7f01a814ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a7fff640 1 -- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01a80af720 msgr2=0x7f01a815ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a7fff640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01a80af720 0x7f01a815ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a7fff640 1 -- 172.21.15.148:0/713590449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a81606b0 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a7fff640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80b1290 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f019c00e870 tx=0x7f019c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c017070 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c00c940 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a8158820 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c00cd30 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.558+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a8158de0 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.559+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a8073990 con 0x7f01a80d02f0 2024-04-25T10:06:09.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.559+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f019c01a070 con 0x7f01a80d02f0 2024-04-25T10:06:09.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.560+0000 7f01a57fa640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f019406b500 0x7f019406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.560+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f019c08a590 con 0x7f01a80d02f0 2024-04-25T10:06:09.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.560+0000 7f01acdb7640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f019406b500 0x7f019406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.560+0000 7f01acdb7640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f019406b500 0x7f019406d9c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0198001780 tx=0x7f0198009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.564+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f019c058ba0 con 0x7f01a80d02f0 2024-04-25T10:06:09.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.734+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01a8156150 con 0x7f01a80d02f0 2024-04-25T10:06:09.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.735+0000 7f01a57fa640 1 -- 172.21.15.148:0/713590449 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17232 (secure 0 0 0) 0x7f019c05cc40 con 0x7f01a80d02f0 2024-04-25T10:06:09.741 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:09.741 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":173,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:06.807921+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.2a","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f019406b500 msgr2=0x7f019406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f019406b500 0x7f019406d9c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0198001780 tx=0x7f0198009040 comp rx=0 tx=0).stop 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 msgr2=0x7f01a80b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80b1290 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f019c00e870 tx=0x7f019c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 shutdown_connections 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f019406b500 0x7f019406d9c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01a8158220 0x7f01a814ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01a80d02f0 0x7f01a80b1290 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 --2- 172.21.15.148:0/713590449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01a80af720 0x7f01a815ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 >> 172.21.15.148:0/713590449 conn(0x7f01a80b2370 msgr2=0x7f01a815c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 shutdown_connections 2024-04-25T10:06:09.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.737+0000 7f01af01b640 1 -- 172.21.15.148:0/713590449 wait complete. 2024-04-25T10:06:09.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:26.693014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:27.457155+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:09.754 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.850+0000 7fe55ef70640 1 Processor -- start 2024-04-25T10:06:09.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.850+0000 7fe55ef70640 1 -- start start 2024-04-25T10:06:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55ef70640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe55806c330 0x7fe5580b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55ef70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55ef70640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5580d1d50 con 0x7fe558158230 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55ef70640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe558158610 con 0x7fe55806c330 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55ef70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55806bea0 con 0x7fe55806c7e0 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe557fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe55806c330 0x7fe5580b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55d50d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55cd0c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe558158230 0x7fe5580d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:45894/0 (socket says 172.21.15.148:45894) 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55d50d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:54068/0 (socket says 172.21.15.148:54068) 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe557fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe55806c330 0x7fe5580b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:38342/0 (socket says 172.21.15.148:38342) 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.851+0000 7fe55cd0c640 1 -- 172.21.15.148:0/3585253434 learned_addr learned my addr 172.21.15.148:0/3585253434 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.852+0000 7fe557fff640 1 -- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 msgr2=0x7fe5580b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.852+0000 7fe557fff640 1 --2- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.852+0000 7fe557fff640 1 -- 172.21.15.148:0/3585253434 >> v1:172.21.15.188:6789/0 conn(0x7fe558158230 legacy=0x7fe5580d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.852+0000 7fe557fff640 1 -- 172.21.15.148:0/3585253434 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5580d2020 con 0x7fe55806c330 2024-04-25T10:06:09.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.852+0000 7fe5577fe640 1 -- 172.21.15.148:0/3585253434 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768831449 0 0) 0x7fe5580d1d50 con 0x7fe558158230 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.852+0000 7fe557fff640 1 --2- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe55806c330 0x7fe5580b4e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009790 tx=0x7fe54c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44da9f17469c0edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe5577fe640 1 -- 172.21.15.148:0/3585253434 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c037070 con 0x7fe55806c330 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe5577fe640 1 -- 172.21.15.148:0/3585253434 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c032440 con 0x7fe55806c330 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe5577fe640 1 -- 172.21.15.148:0/3585253434 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c032810 con 0x7fe55806c330 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 -- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe55806c330 msgr2=0x7fe5580b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 --2- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe55806c330 0x7fe5580b4e60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009790 tx=0x7fe54c02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 -- 172.21.15.148:0/3585253434 shutdown_connections 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 --2- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe55806c330 0x7fe5580b4e60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 --2- 172.21.15.148:0/3585253434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 -- 172.21.15.148:0/3585253434 >> 172.21.15.148:0/3585253434 conn(0x7fe558061600 msgr2=0x7fe5580619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 -- 172.21.15.148:0/3585253434 shutdown_connections 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.853+0000 7fe55ef70640 1 -- 172.21.15.148:0/3585253434 wait complete. 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.854+0000 7fe55ef70640 1 Processor -- start 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.854+0000 7fe55ef70640 1 -- start start 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55ef70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 0x7fe5580cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55ef70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55ef70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe558158230 0x7fe5580cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55ef70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580d25e0 con 0x7fe55806c330 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55ef70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55806bb30 con 0x7fe558158230 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55ef70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55815e320 con 0x7fe55806c7e0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55cd0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 0x7fe5580cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55cd0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 0x7fe5580cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60854/0 (socket says 172.21.15.148:60854) 2024-04-25T10:06:09.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.855+0000 7fe55cd0c640 1 -- 172.21.15.148:0/270578665 learned_addr learned my addr 172.21.15.148:0/270578665 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55d50d640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe558158230 0x7fe5580cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe557fff640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55cd0c640 1 -- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 msgr2=0x7fe5580caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55cd0c640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55cd0c640 1 -- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe558158230 msgr2=0x7fe5580cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55cd0c640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe558158230 0x7fe5580cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55cd0c640 1 -- 172.21.15.148:0/270578665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5580cb5d0 con 0x7fe55806c330 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55cd0c640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 0x7fe5580cc0a0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c4d0 tx=0x7fe54800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548018070 con 0x7fe55806c330 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548007500 con 0x7fe55806c330 2024-04-25T10:06:09.865 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe558158850 con 0x7fe55806c330 2024-04-25T10:06:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548004a20 con 0x7fe55806c330 2024-04-25T10:06:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.856+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe558158e10 con 0x7fe55806c330 2024-04-25T10:06:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.857+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55806de50 con 0x7fe55806c330 2024-04-25T10:06:09.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.858+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe54801b070 con 0x7fe55806c330 2024-04-25T10:06:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.858+0000 7fe5557fa640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.858+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fe5480896d0 con 0x7fe55806c330 2024-04-25T10:06:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.858+0000 7fe557fff640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.859+0000 7fe557fff640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:09.862+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe548057ce0 con 0x7fe55806c330 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.032+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5580b5fb0 con 0x7fe55806c330 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.032+0000 7fe5557fa640 1 -- 172.21.15.148:0/270578665 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe54805bd80 con 0x7fe55806c330 2024-04-25T10:06:10.038 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 msgr2=0x7fe54406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fe54c034000 tx=0x7fe54c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 msgr2=0x7fe5580cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 0x7fe5580cc0a0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c4d0 tx=0x7fe54800c990 comp rx=0 tx=0).stop 2024-04-25T10:06:10.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 shutdown_connections 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe54406b4d0 0x7fe54406d990 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe558158230 0x7fe5580cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe55806c7e0 0x7fe5580caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 --2- 172.21.15.148:0/270578665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe55806c330 0x7fe5580cc0a0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 >> 172.21.15.148:0/270578665 conn(0x7fe558061600 msgr2=0x7fe5580b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 shutdown_connections 2024-04-25T10:06:10.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.034+0000 7fe55ef70640 1 -- 172.21.15.148:0/270578665 wait complete. 2024-04-25T10:06:10.051 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:10.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.146+0000 7f7d2c65a640 1 Processor -- start 2024-04-25T10:06:10.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.146+0000 7f7d2c65a640 1 -- start start 2024-04-25T10:06:10.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2c65a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d24155f80 0x7f7d2406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2c65a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d2406fa00 0x7f7d240ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2c65a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d240b4d80 con 0x7f7d240ce730 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2c65a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d240b5390 con 0x7f7d24155f80 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2c65a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240d07f0 con 0x7f7d2406fa00 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2abf7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7d240ce730 0x7f7d2415f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45280/0 (socket says 172.21.15.148:45280) 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2abf7640 1 -- 172.21.15.148:0/2471701444 learned_addr learned my addr 172.21.15.148:0/2471701444 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2a3f6640 1 --2- 172.21.15.148:0/2471701444 >> v2:172.21.15.188:3300/0 conn(0x7f7d24155f80 0x7f7d2406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d29bf5640 1 --2- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d2406fa00 0x7f7d240ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d293f4640 1 -- 172.21.15.148:0/2471701444 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858200867 0 0) 0x7f7d240b4d80 con 0x7f7d240ce730 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d293f4640 1 -- 172.21.15.148:0/2471701444 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d10003520 con 0x7f7d240ce730 2024-04-25T10:06:10.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d2406fa00 msgr2=0x7f7d240ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2a3f6640 1 --2- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d2406fa00 0x7f7d240ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/2471701444 >> v1:172.21.15.148:6789/0 conn(0x7f7d240ce730 legacy=0x7f7d2415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.147+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/2471701444 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240cef70 con 0x7f7d24155f80 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.148+0000 7f7d2a3f6640 1 --2- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d2406f4c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7d200097a0 tx=0x7f7d2002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b543884bf45259a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.148+0000 7f7d293f4640 1 -- 172.21.15.148:0/2471701444 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20037070 con 0x7f7d24155f80 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.148+0000 7f7d293f4640 1 -- 172.21.15.148:0/2471701444 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20032440 con 0x7f7d24155f80 2024-04-25T10:06:10.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.148+0000 7f7d293f4640 1 -- 172.21.15.148:0/2471701444 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d200327f0 con 0x7f7d24155f80 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 -- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 msgr2=0x7f7d2406f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d2406f4c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7d200097a0 tx=0x7f7d2002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 -- 172.21.15.148:0/2471701444 shutdown_connections 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d2406fa00 0x7f7d240ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/2471701444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d2406f4c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 -- 172.21.15.148:0/2471701444 >> 172.21.15.148:0/2471701444 conn(0x7f7d24061600 msgr2=0x7f7d240619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 -- 172.21.15.148:0/2471701444 shutdown_connections 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.149+0000 7f7d2c65a640 1 -- 172.21.15.148:0/2471701444 wait complete. 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.150+0000 7f7d2c65a640 1 Processor -- start 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.150+0000 7f7d2c65a640 1 -- start start 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 0x7f7d2406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d240ce730 0x7f7d2406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d241420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240cf250 con 0x7f7d2406fa00 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24156360 con 0x7f7d24155f80 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d240cf3d0 con 0x7f7d240ce730 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 0x7f7d2406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 0x7f7d2406d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60856/0 (socket says 172.21.15.148:60856) 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/3650458303 learned_addr learned my addr 172.21.15.148:0/3650458303 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2abf7640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d241420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d29bf5640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d240ce730 0x7f7d2406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d240ce730 msgr2=0x7f7d2406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d240ce730 0x7f7d2406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 msgr2=0x7f7d241420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d241420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 -- 172.21.15.148:0/3650458303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2406c460 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2a3f6640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 0x7f7d2406d9a0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f7d2002a1c0 tx=0x7f7d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d20037040 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d20042070 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2003b930 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2406c740 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.151+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2406cd00 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.152+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24073a00 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.153+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d2003bad0 con 0x7f7d2406fa00 2024-04-25T10:06:10.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.153+0000 7f7d16ffd640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0806b500 0x7f7d0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.153+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7d200aac60 con 0x7f7d2406fa00 2024-04-25T10:06:10.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.153+0000 7f7d29bf5640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0806b500 0x7f7d0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.154+0000 7f7d29bf5640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0806b500 0x7f7d0806d9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7d24155690 tx=0x7f7d18008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.157+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d200798c0 con 0x7f7d2406fa00 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.326+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d240b1030 con 0x7f7d2406fa00 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.326+0000 7f7d16ffd640 1 -- 172.21.15.148:0/3650458303 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v173) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d200791c0 con 0x7f7d2406fa00 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0806b500 msgr2=0x7f7d0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0806b500 0x7f7d0806d9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7d24155690 tx=0x7f7d18008040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 msgr2=0x7f7d2406d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 0x7f7d2406d9a0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f7d2002a1c0 tx=0x7f7d200046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 shutdown_connections 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0806b500 0x7f7d0806d9c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d24155f80 0x7f7d241420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d240ce730 0x7f7d2406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 --2- 172.21.15.148:0/3650458303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d2406fa00 0x7f7d2406d9a0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 >> 172.21.15.148:0/3650458303 conn(0x7f7d24061600 msgr2=0x7f7d240b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 shutdown_connections 2024-04-25T10:06:10.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.328+0000 7f7d2c65a640 1 -- 172.21.15.148:0/3650458303 wait complete. 2024-04-25T10:06:10.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:10.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.439+0000 7f700bff9640 1 Processor -- start 2024-04-25T10:06:10.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700bff9640 1 -- start start 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700bff9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f700415de80 0x7f70040cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700bff9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70040ccfe0 0x7f70040cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700bff9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040afc80 con 0x7f70040cefb0 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700bff9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70040ce4d0 con 0x7f70040ccfe0 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700bff9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70040ce7b0 con 0x7f700415de80 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.440+0000 7f700a596640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70040ccfe0 0x7f70040cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f70040ccfe0 0x7f70040cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60860/0 (socket says 172.21.15.148:60860) 2024-04-25T10:06:10.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 -- 172.21.15.148:0/1578204769 learned_addr learned my addr 172.21.15.148:0/1578204769 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 -- 172.21.15.148:0/1578204769 >> v2:172.21.15.188:3300/0 conn(0x7f700415de80 msgr2=0x7f70040cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 --2- 172.21.15.148:0/1578204769 >> v2:172.21.15.188:3300/0 conn(0x7f700415de80 0x7f70040cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 -- 172.21.15.148:0/1578204769 >> v1:172.21.15.188:6789/0 conn(0x7f70040cefb0 legacy=0x7f70040cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 -- 172.21.15.148:0/1578204769 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040cea90 con 0x7f70040ccfe0 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.441+0000 7f700a596640 1 --2- 172.21.15.148:0/1578204769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70040ccfe0 0x7f70040cd410 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f70040d1800 tx=0x7f6ffc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96c29d412aef261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f7008d93640 1 -- 172.21.15.148:0/1578204769 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc037070 con 0x7f70040ccfe0 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f7008d93640 1 -- 172.21.15.148:0/1578204769 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc032440 con 0x7f70040ccfe0 2024-04-25T10:06:10.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f7008d93640 1 -- 172.21.15.148:0/1578204769 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc032810 con 0x7f70040ccfe0 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 -- 172.21.15.148:0/1578204769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70040ccfe0 msgr2=0x7f70040cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 --2- 172.21.15.148:0/1578204769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70040ccfe0 0x7f70040cd410 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f70040d1800 tx=0x7f6ffc02a270 comp rx=0 tx=0).stop 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 -- 172.21.15.148:0/1578204769 shutdown_connections 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 --2- 172.21.15.148:0/1578204769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70040ccfe0 0x7f70040cd410 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 --2- 172.21.15.148:0/1578204769 >> v2:172.21.15.188:3300/0 conn(0x7f700415de80 0x7f70040cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 -- 172.21.15.148:0/1578204769 >> 172.21.15.148:0/1578204769 conn(0x7f70040b2390 msgr2=0x7f70040b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 -- 172.21.15.148:0/1578204769 shutdown_connections 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.442+0000 7f700bff9640 1 -- 172.21.15.148:0/1578204769 wait complete. 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.443+0000 7f700bff9640 1 Processor -- start 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.443+0000 7f700bff9640 1 -- start start 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700bff9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70040ccfe0 0x7f700414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700bff9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70040cefb0 0x7f70040b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700bff9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 0x7f70040b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700bff9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70040cfb50 con 0x7f700415de80 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700bff9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70040b2eb0 con 0x7f70040cefb0 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700bff9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040c8d50 con 0x7f70040ccfe0 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 0x7f70040b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 0x7f70040b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60868/0 (socket says 172.21.15.148:60868) 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 -- 172.21.15.148:0/364073838 learned_addr learned my addr 172.21.15.148:0/364073838 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 -- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70040ccfe0 msgr2=0x7f700414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f7009594640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70040cefb0 0x7f70040b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f7009d95640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70040ccfe0 0x7f700414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70040ccfe0 0x7f700414e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 -- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70040cefb0 msgr2=0x7f70040b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70040cefb0 0x7f70040b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f700a596640 1 -- 172.21.15.148:0/364073838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70041f4190 con 0x7f700415de80 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.444+0000 7f7009d95640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70040ccfe0 0x7f700414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.445+0000 7f700a596640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 0x7f70040b0ea0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0090b0 tx=0x7f6ffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.445+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc041020 con 0x7f700415de80 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.445+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc030990 con 0x7f700415de80 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.445+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70041f4470 con 0x7f700415de80 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.445+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc037040 con 0x7f700415de80 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.445+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700414ead0 con 0x7f700415de80 2024-04-25T10:06:10.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.446+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70041f4860 con 0x7f700415de80 2024-04-25T10:06:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.446+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ffc044020 con 0x7f700415de80 2024-04-25T10:06:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.446+0000 7f6ffa7fc640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fe806b4d0 0x7f6fe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.447+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f6ffc0aad20 con 0x7f700415de80 2024-04-25T10:06:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.447+0000 7f7009d95640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fe806b4d0 0x7f6fe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.447+0000 7f7009d95640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fe806b4d0 0x7f6fe806d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f70040af460 tx=0x7f70000073e0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.451+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ffc04b030 con 0x7f700415de80 2024-04-25T10:06:10.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.622+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70041f4640 con 0x7f700415de80 2024-04-25T10:06:10.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.622+0000 7f6ffa7fc640 1 -- 172.21.15.148:0/364073838 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v173) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ffc079a10 con 0x7f700415de80 2024-04-25T10:06:10.628 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fe806b4d0 msgr2=0x7f6fe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fe806b4d0 0x7f6fe806d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f70040af460 tx=0x7f70000073e0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 msgr2=0x7f70040b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 0x7f70040b0ea0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0090b0 tx=0x7f6ffc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 shutdown_connections 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6fe806b4d0 0x7f6fe806d990 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f700415de80 0x7f70040b0ea0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70040cefb0 0x7f70040b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 --2- 172.21.15.148:0/364073838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70040ccfe0 0x7f700414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 >> 172.21.15.148:0/364073838 conn(0x7f70040b2390 msgr2=0x7f70040cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 shutdown_connections 2024-04-25T10:06:10.631 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.624+0000 7f700bff9640 1 -- 172.21.15.148:0/364073838 wait complete. 2024-04-25T10:06:10.641 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.732+0000 7f9430fd5640 1 Processor -- start 2024-04-25T10:06:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.732+0000 7f9430fd5640 1 -- start start 2024-04-25T10:06:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.733+0000 7f9430fd5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f942c15bd30 0x7f942c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.733+0000 7f9430fd5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0d0350 con 0x7f942c0d1b00 2024-04-25T10:06:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.733+0000 7f9430fd5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c0d0630 con 0x7f942c0cfe40 2024-04-25T10:06:10.742 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.733+0000 7f9430fd5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0d0910 con 0x7f942c15bd30 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942ad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f942c0d1b00 0x7f942c0b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45292/0 (socket says 172.21.15.148:45292) 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942ad9d640 1 -- 172.21.15.148:0/4242766099 learned_addr learned my addr 172.21.15.148:0/4242766099 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942a59c640 1 --2- 172.21.15.148:0/4242766099 >> v2:172.21.15.188:3300/0 conn(0x7f942c15bd30 0x7f942c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592460944 0 0) 0x7f942c0d0350 con 0x7f942c0d1b00 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9414003520 con 0x7f942c0d1b00 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283259246 0 0) 0x7f9414003520 con 0x7f942c0d1b00 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 >> v2:172.21.15.188:3300/0 conn(0x7f942c15bd30 msgr2=0x7f942c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 --2- 172.21.15.148:0/4242766099 >> v2:172.21.15.188:3300/0 conn(0x7f942c15bd30 0x7f942c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 >> v1:172.21.15.188:6789/0 conn(0x7f942c0cfe40 legacy=0x7f942c0d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0d0bf0 con 0x7f942c0d1b00 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f941c0019f0 con 0x7f942c0d1b00 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 >> v1:172.21.15.148:6789/0 conn(0x7f942c0d1b00 legacy=0x7f942c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f9414003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9414004800 0x7f9414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.743 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94140051c0 0x7f941400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0d0910 con 0x7f94140039b0 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941400c070 con 0x7f94140051c0 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941400c270 con 0x7f9414004800 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942a59c640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f9414003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f942ad9d640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94140051c0 0x7f941400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.734+0000 7f9429d9b640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9414004800 0x7f9414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942a59c640 1 -- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9414004800 msgr2=0x7f9414004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942a59c640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9414004800 0x7f9414004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942a59c640 1 -- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94140051c0 msgr2=0x7f941400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942a59c640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94140051c0 0x7f941400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942a59c640 1 -- 172.21.15.148:0/4242766099 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414003520 con 0x7f94140039b0 2024-04-25T10:06:10.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942ad9d640 1 -- 172.21.15.148:0/4242766099 reap_dead start 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942a59c640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f9414003dd0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f9424009ac0 tx=0x7f9414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deee43e05da6656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424036070 con 0x7f94140039b0 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424031400 con 0x7f94140039b0 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.735+0000 7f942959a640 1 -- 172.21.15.148:0/4242766099 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424009690 con 0x7f94140039b0 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 -- 172.21.15.148:0/4242766099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 msgr2=0x7f9414003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f9414003dd0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f9424009ac0 tx=0x7f9414005830 comp rx=0 tx=0).stop 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 -- 172.21.15.148:0/4242766099 shutdown_connections 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 --2- 172.21.15.148:0/4242766099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f9414003dd0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 -- 172.21.15.148:0/4242766099 >> 172.21.15.148:0/4242766099 conn(0x7f942c06f590 msgr2=0x7f942c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 -- 172.21.15.148:0/4242766099 shutdown_connections 2024-04-25T10:06:10.745 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.736+0000 7f9430fd5640 1 -- 172.21.15.148:0/4242766099 wait complete. 2024-04-25T10:06:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.737+0000 7f9430fd5640 1 Processor -- start 2024-04-25T10:06:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.737+0000 7f9430fd5640 1 -- start start 2024-04-25T10:06:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9430fd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f942c0a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9430fd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f942c0b0090 0x7f942c1f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9430fd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f942c0aa460 0x7f942c0aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9430fd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0b73b0 con 0x7f94140039b0 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9430fd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15c110 con 0x7f942c0aa460 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9430fd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0b7530 con 0x7f942c0b0090 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f942c0a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f942c0a8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60898/0 (socket says 172.21.15.148:60898) 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 -- 172.21.15.148:0/2074991640 learned_addr learned my addr 172.21.15.148:0/2074991640 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942ad9d640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f942c0aa460 0x7f942c0aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f9429d9b640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f942c0b0090 0x7f942c1f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 -- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f942c0b0090 msgr2=0x7f942c1f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f942c0b0090 0x7f942c1f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 -- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f942c0aa460 msgr2=0x7f942c0aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.738+0000 7f942a59c640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f942c0aa460 0x7f942c0aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f942a59c640 1 -- 172.21.15.148:0/2074991640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c1f2770 con 0x7f94140039b0 2024-04-25T10:06:10.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f942ad9d640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f942c0aa460 0x7f942c0aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f942a59c640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f942c0a8520 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f9424034040 tx=0x7f9424004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942403f020 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424037040 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c0a8b00 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424036070 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.739+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c0a90c0 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.740+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c073990 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.740+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9424042020 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.741+0000 7f941affd640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940c06b4d0 0x7f940c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.741+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f94240aa410 con 0x7f94140039b0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.741+0000 7f9429d9b640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940c06b4d0 0x7f940c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.741+0000 7f9429d9b640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940c06b4d0 0x7f940c06d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9420000c00 tx=0x7f9420009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.745+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9424049030 con 0x7f94140039b0 2024-04-25T10:06:10.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.951+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f942c0cb760 con 0x7f94140039b0 2024-04-25T10:06:10.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.951+0000 7f941affd640 1 -- 172.21.15.148:0/2074991640 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7f9424079130 con 0x7f94140039b0 2024-04-25T10:06:10.957 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:10.957 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:06:10.957 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940c06b4d0 msgr2=0x7f940c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940c06b4d0 0x7f940c06d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9420000c00 tx=0x7f9420009040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 msgr2=0x7f942c0a8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f942c0a8520 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f9424034040 tx=0x7f9424004730 comp rx=0 tx=0).stop 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 shutdown_connections 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f940c06b4d0 0x7f940c06d990 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f942c0aa460 0x7f942c0aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f942c0b0090 0x7f942c1f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 --2- 172.21.15.148:0/2074991640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94140039b0 0x7f942c0a8520 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 >> 172.21.15.148:0/2074991640 conn(0x7f942c06f590 msgr2=0x7f942c0b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 shutdown_connections 2024-04-25T10:06:10.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:10.953+0000 7f9430fd5640 1 -- 172.21.15.148:0/2074991640 wait complete. 2024-04-25T10:06:10.970 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:11.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.061+0000 7f8abe1f5640 1 Processor -- start 2024-04-25T10:06:11.067 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.061+0000 7f8abe1f5640 1 -- start start 2024-04-25T10:06:11.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8abe1f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ab80b5c30 0x7f8ab8168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8abe1f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80ceee0 con 0x7f8ab80b6cb0 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8abe1f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80d1d10 con 0x7f8ab815db70 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8abe1f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab80d3800 con 0x7f8ab80b5c30 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8ab6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ab80b6cb0 0x7f8ab80b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45304/0 (socket says 172.21.15.148:45304) 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8ab6ffd640 1 -- 172.21.15.148:0/3850247662 learned_addr learned my addr 172.21.15.148:0/3850247662 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8ab7fff640 1 --2- 172.21.15.148:0/3850247662 >> v2:172.21.15.148:3300/0 conn(0x7f8ab80b5c30 0x7f8ab8168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8ab7fff640 1 -- 172.21.15.148:0/3850247662 >> v1:172.21.15.188:6789/0 conn(0x7f8ab815db70 legacy=0x7f8ab8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8ab7fff640 1 -- 172.21.15.148:0/3850247662 >> v1:172.21.15.148:6789/0 conn(0x7f8ab80b6cb0 legacy=0x7f8ab80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.062+0000 7f8ab7fff640 1 -- 172.21.15.148:0/3850247662 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80d1ff0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.063+0000 7f8ab7fff640 1 --2- 172.21.15.148:0/3850247662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab8168ad0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80d3980 tx=0x7f8ab002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722e9440d77a03a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.063+0000 7f8ab67fc640 1 -- 172.21.15.148:0/3850247662 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0007740 con 0x7f8ab80b5c30 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.063+0000 7f8ab67fc640 1 -- 172.21.15.148:0/3850247662 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab00078e0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.063+0000 7f8ab67fc640 1 -- 172.21.15.148:0/3850247662 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0007cb0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3850247662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 msgr2=0x7f8ab8168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3850247662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab8168ad0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80d3980 tx=0x7f8ab002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.070 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3850247662 shutdown_connections 2024-04-25T10:06:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3850247662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab8168ad0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3850247662 >> 172.21.15.148:0/3850247662 conn(0x7f8ab80d0520 msgr2=0x7f8ab80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3850247662 shutdown_connections 2024-04-25T10:06:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.064+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3850247662 wait complete. 2024-04-25T10:06:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.065+0000 7f8abe1f5640 1 Processor -- start 2024-04-25T10:06:11.071 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.065+0000 7f8abe1f5640 1 -- start start 2024-04-25T10:06:11.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab80aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b6cb0 0x7f8ab80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab815db70 0x7f8ab81f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80ce530 con 0x7f8ab80b5c30 2024-04-25T10:06:11.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab815df50 con 0x7f8ab815db70 2024-04-25T10:06:11.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80cf1c0 con 0x7f8ab80b6cb0 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab80aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab80aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60910/0 (socket says 172.21.15.148:60910) 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 -- 172.21.15.148:0/3517796745 learned_addr learned my addr 172.21.15.148:0/3517796745 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab7fff640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab815db70 0x7f8ab81f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab6ffd640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b6cb0 0x7f8ab80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 -- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b6cb0 msgr2=0x7f8ab80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b6cb0 0x7f8ab80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 -- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab815db70 msgr2=0x7f8ab81f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab815db70 0x7f8ab81f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 -- 172.21.15.148:0/3517796745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80aa9e0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab7fff640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab815db70 0x7f8ab81f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab6ffd640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b6cb0 0x7f8ab80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:11.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8ab77fe640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab80aa350 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c3c0 tx=0x7f8aa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80052d0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa8005490 con 0x7f8ab80b5c30 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab80cc1d0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.066+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8007260 con 0x7f8ab80b5c30 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.067+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab80cc790 con 0x7f8ab80b5c30 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.068+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab8073990 con 0x7f8ab80b5c30 2024-04-25T10:06:11.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.068+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8aa8007400 con 0x7f8ab80b5c30 2024-04-25T10:06:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.068+0000 7f8a9ffff640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8aa406b4d0 0x7f8aa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.069+0000 7f8ab6ffd640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8aa406b4d0 0x7f8aa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.069+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f8aa80898f0 con 0x7f8ab80b5c30 2024-04-25T10:06:11.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.069+0000 7f8ab6ffd640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8aa406b4d0 0x7f8aa406d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8ab815bb10 tx=0x7f8aac009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.073+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8aa8058560 con 0x7f8ab80b5c30 2024-04-25T10:06:11.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.244+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ab80b9630 con 0x7f8ab80b5c30 2024-04-25T10:06:11.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.245+0000 7f8a9ffff640 1 -- 172.21.15.148:0/3517796745 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17232 (secure 0 0 0) 0x7f8aa8057e60 con 0x7f8ab80b5c30 2024-04-25T10:06:11.251 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:11.251 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":173,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:06.807921+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.2a","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:11.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8aa406b4d0 msgr2=0x7f8aa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8aa406b4d0 0x7f8aa406d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8ab815bb10 tx=0x7f8aac009040 comp rx=0 tx=0).stop 2024-04-25T10:06:11.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 msgr2=0x7f8ab80aa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab80aa350 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c3c0 tx=0x7f8aa800c880 comp rx=0 tx=0).stop 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 shutdown_connections 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8aa406b4d0 0x7f8aa406d990 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ab815db70 0x7f8ab81f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ab80b6cb0 0x7f8ab80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 --2- 172.21.15.148:0/3517796745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab80b5c30 0x7f8ab80aa350 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 >> 172.21.15.148:0/3517796745 conn(0x7f8ab80d0520 msgr2=0x7f8ab80b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.247+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 shutdown_connections 2024-04-25T10:06:11.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.248+0000 7f8abe1f5640 1 -- 172.21.15.148:0/3517796745 wait complete. 2024-04-25T10:06:11.264 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:11.264 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 Processor -- start 2024-04-25T10:06:11.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 -- start start 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8c413d190 0x7fe8c40cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8c4071940 0x7fe8c40cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c4071ed0 con 0x7fe8c415b320 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c413d570 con 0x7fe8c413d190 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.357+0000 7fe8cb110640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c415a840 con 0x7fe8c4071940 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8c413d190 0x7fe8c40cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe8c413d190 0x7fe8c40cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60924/0 (socket says 172.21.15.148:60924) 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 -- 172.21.15.148:0/4014397591 learned_addr learned my addr 172.21.15.148:0/4014397591 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c96ad640 1 --2- 172.21.15.148:0/4014397591 >> v2:172.21.15.188:3300/0 conn(0x7fe8c4071940 0x7fe8c40cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 -- 172.21.15.148:0/4014397591 >> v2:172.21.15.188:3300/0 conn(0x7fe8c4071940 msgr2=0x7fe8c40cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 --2- 172.21.15.148:0/4014397591 >> v2:172.21.15.188:3300/0 conn(0x7fe8c4071940 0x7fe8c40cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 -- 172.21.15.148:0/4014397591 >> v1:172.21.15.148:6789/0 conn(0x7fe8c415b320 legacy=0x7fe8c4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.358+0000 7fe8c8eac640 1 -- 172.21.15.148:0/4014397591 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c415ab20 con 0x7fe8c413d190 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8c8eac640 1 --2- 172.21.15.148:0/4014397591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c413d190 0x7fe8c40cbfb0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40ccd10 tx=0x7fe8b802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a3227d33cf9b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8c37fe640 1 -- 172.21.15.148:0/4014397591 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8033640 con 0x7fe8c413d190 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8c37fe640 1 -- 172.21.15.148:0/4014397591 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8038070 con 0x7fe8c413d190 2024-04-25T10:06:11.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8c37fe640 1 -- 172.21.15.148:0/4014397591 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8033a00 con 0x7fe8c413d190 2024-04-25T10:06:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 -- 172.21.15.148:0/4014397591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c413d190 msgr2=0x7fe8c40cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 --2- 172.21.15.148:0/4014397591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c413d190 0x7fe8c40cbfb0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40ccd10 tx=0x7fe8b802c8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 -- 172.21.15.148:0/4014397591 shutdown_connections 2024-04-25T10:06:11.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 --2- 172.21.15.148:0/4014397591 >> v2:172.21.15.188:3300/0 conn(0x7fe8c4071940 0x7fe8c40cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 --2- 172.21.15.148:0/4014397591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c413d190 0x7fe8c40cbfb0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 -- 172.21.15.148:0/4014397591 >> 172.21.15.148:0/4014397591 conn(0x7fe8c406abe0 msgr2=0x7fe8c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 -- 172.21.15.148:0/4014397591 shutdown_connections 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.359+0000 7fe8cb110640 1 -- 172.21.15.148:0/4014397591 wait complete. 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.360+0000 7fe8cb110640 1 Processor -- start 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.360+0000 7fe8cb110640 1 -- start start 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8cb110640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8c4071940 0x7fe8c40af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8cb110640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8c413d190 0x7fe8c413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8cb110640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 0x7fe8c413c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8cb110640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40cf7a0 con 0x7fe8c415b320 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8cb110640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40cf620 con 0x7fe8c4071940 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8cb110640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40cf920 con 0x7fe8c413d190 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8c96ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 0x7fe8c413c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8c96ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 0x7fe8c413c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60930/0 (socket says 172.21.15.148:60930) 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.361+0000 7fe8c96ad640 1 -- 172.21.15.148:0/261300759 learned_addr learned my addr 172.21.15.148:0/261300759 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c96ad640 1 -- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8c413d190 msgr2=0x7fe8c413bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c96ad640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8c413d190 0x7fe8c413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c96ad640 1 -- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8c4071940 msgr2=0x7fe8c40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c8eac640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8c4071940 0x7fe8c40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c96ad640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8c4071940 0x7fe8c40af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c96ad640 1 -- 172.21.15.148:0/261300759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40b4a30 con 0x7fe8c415b320 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c8eac640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8c4071940 0x7fe8c40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c96ad640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 0x7fe8c413c090 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00cd70 tx=0x7fe8bc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc003b90 con 0x7fe8c415b320 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc003d30 con 0x7fe8c415b320 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40afe00 con 0x7fe8c415b320 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc004240 con 0x7fe8c415b320 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.362+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c40b03c0 con 0x7fe8c415b320 2024-04-25T10:06:11.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.363+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c4073990 con 0x7fe8c415b320 2024-04-25T10:06:11.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.364+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8bc0043e0 con 0x7fe8c415b320 2024-04-25T10:06:11.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.364+0000 7fe8c17fa640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe8a806b4d0 0x7fe8a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.364+0000 7fe8c8eac640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe8a806b4d0 0x7fe8a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.364+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fe8bc08aef0 con 0x7fe8c415b320 2024-04-25T10:06:11.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.365+0000 7fe8c8eac640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe8a806b4d0 0x7fe8a806d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8037000 tx=0x7fe8b8032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.368+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8bc059be0 con 0x7fe8c415b320 2024-04-25T10:06:11.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.574+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8c40cfeb0 con 0x7fe8c415b320 2024-04-25T10:06:11.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.575+0000 7fe8c17fa640 1 -- 172.21.15.148:0/261300759 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fe8bc0594e0 con 0x7fe8c415b320 2024-04-25T10:06:11.580 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:11.581 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:06:11.581 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:06:11.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe8a806b4d0 msgr2=0x7fe8a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe8a806b4d0 0x7fe8a806d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8037000 tx=0x7fe8b8032000 comp rx=0 tx=0).stop 2024-04-25T10:06:11.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 msgr2=0x7fe8c413c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 0x7fe8c413c090 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00cd70 tx=0x7fe8bc00a660 comp rx=0 tx=0).stop 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 shutdown_connections 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe8a806b4d0 0x7fe8a806d990 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8c415b320 0x7fe8c413c090 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8c413d190 0x7fe8c413bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 --2- 172.21.15.148:0/261300759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8c4071940 0x7fe8c40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 >> 172.21.15.148:0/261300759 conn(0x7fe8c406abe0 msgr2=0x7fe8c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 shutdown_connections 2024-04-25T10:06:11.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.577+0000 7fe8cb110640 1 -- 172.21.15.148:0/261300759 wait complete. 2024-04-25T10:06:11.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:11.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.686+0000 7fa487c07640 1 Processor -- start 2024-04-25T10:06:11.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.686+0000 7fa487c07640 1 -- start start 2024-04-25T10:06:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa487c07640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa48006abe0 0x7fa48006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa487c07640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4800ce330 0x7fa48006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa487c07640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4801626e0 con 0x7fa4800700a0 2024-04-25T10:06:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa487c07640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48006b500 con 0x7fa48006abe0 2024-04-25T10:06:11.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa487c07640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4800cd270 con 0x7fa4800ce330 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa48006abe0 0x7fa48006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa48006abe0 0x7fa48006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60942/0 (socket says 172.21.15.148:60942) 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 -- 172.21.15.148:0/1276519116 learned_addr learned my addr 172.21.15.148:0/1276519116 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4851a2640 1 --2- 172.21.15.148:0/1276519116 >> v2:172.21.15.188:3300/0 conn(0x7fa4800ce330 0x7fa48006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 -- 172.21.15.148:0/1276519116 >> v2:172.21.15.188:3300/0 conn(0x7fa4800ce330 msgr2=0x7fa48006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 --2- 172.21.15.148:0/1276519116 >> v2:172.21.15.188:3300/0 conn(0x7fa4800ce330 0x7fa48006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 -- 172.21.15.148:0/1276519116 >> v1:172.21.15.188:6789/0 conn(0x7fa4800700a0 legacy=0x7fa48006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.687+0000 7fa4859a3640 1 -- 172.21.15.148:0/1276519116 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800cd550 con 0x7fa48006abe0 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa4859a3640 1 --2- 172.21.15.148:0/1276519116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa48006abe0 0x7fa48006afc0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fa48006bbf0 tx=0x7fa47c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff68d27e78076c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa4849a1640 1 -- 172.21.15.148:0/1276519116 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037070 con 0x7fa48006abe0 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa4849a1640 1 -- 172.21.15.148:0/1276519116 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47c032440 con 0x7fa48006abe0 2024-04-25T10:06:11.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa4849a1640 1 -- 172.21.15.148:0/1276519116 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c032810 con 0x7fa48006abe0 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa487c07640 1 -- 172.21.15.148:0/1276519116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa48006abe0 msgr2=0x7fa48006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa487c07640 1 --2- 172.21.15.148:0/1276519116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa48006abe0 0x7fa48006afc0 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fa48006bbf0 tx=0x7fa47c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa487c07640 1 -- 172.21.15.148:0/1276519116 shutdown_connections 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa487c07640 1 --2- 172.21.15.148:0/1276519116 >> v2:172.21.15.188:3300/0 conn(0x7fa4800ce330 0x7fa48006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa487c07640 1 --2- 172.21.15.148:0/1276519116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa48006abe0 0x7fa48006afc0 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.688+0000 7fa487c07640 1 -- 172.21.15.148:0/1276519116 >> 172.21.15.148:0/1276519116 conn(0x7fa4800b2220 msgr2=0x7fa4800b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.689+0000 7fa487c07640 1 -- 172.21.15.148:0/1276519116 shutdown_connections 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.689+0000 7fa487c07640 1 -- 172.21.15.148:0/1276519116 wait complete. 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 Processor -- start 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 -- start start 2024-04-25T10:06:11.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48006abe0 0x7fa4800b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4800700a0 0x7fa4801f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 0x7fa48015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa480161780 con 0x7fa4800ce330 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800cdb10 con 0x7fa48006abe0 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa487c07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48006eb00 con 0x7fa4800700a0 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa4861a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 0x7fa48015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa4861a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 0x7fa48015cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60950/0 (socket says 172.21.15.148:60950) 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.690+0000 7fa4861a4640 1 -- 172.21.15.148:0/810129141 learned_addr learned my addr 172.21.15.148:0/810129141 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4861a4640 1 -- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4800700a0 msgr2=0x7fa4801f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4859a3640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48006abe0 0x7fa4800b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4851a2640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4800700a0 0x7fa4801f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4861a4640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4800700a0 0x7fa4801f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4861a4640 1 -- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48006abe0 msgr2=0x7fa4800b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4861a4640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48006abe0 0x7fa4800b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4861a4640 1 -- 172.21.15.148:0/810129141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800b37f0 con 0x7fa4800ce330 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4861a4640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 0x7fa48015cd60 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fa47800c780 tx=0x7fa47800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa478004b60 con 0x7fa4800ce330 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa478004d00 con 0x7fa4800ce330 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800aa90 con 0x7fa4800ce330 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4800b3ad0 con 0x7fa4800ce330 2024-04-25T10:06:11.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.691+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4800b4090 con 0x7fa4800ce330 2024-04-25T10:06:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.692+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa480073990 con 0x7fa4800ce330 2024-04-25T10:06:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.692+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa47800ac30 con 0x7fa4800ce330 2024-04-25T10:06:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.693+0000 7fa4767fc640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa46406b4d0 0x7fa46406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.693+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fa47808a700 con 0x7fa4800ce330 2024-04-25T10:06:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.693+0000 7fa4859a3640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa46406b4d0 0x7fa46406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.693+0000 7fa4859a3640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa46406b4d0 0x7fa46406d990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa47c034000 tx=0x7fa47c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.697+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa478020070 con 0x7fa4800ce330 2024-04-25T10:06:11.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.904+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4800afe50 con 0x7fa4800ce330 2024-04-25T10:06:11.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.904+0000 7fa4767fc640 1 -- 172.21.15.148:0/810129141 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4955 (secure 0 0 0) 0x7fa478059360 con 0x7fa4800ce330 2024-04-25T10:06:11.910 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:11.910 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:28.494582+0000","modified":"2024-04-25T10:05:30.487395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10845":{"gid":10845,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/906811420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":906811420},{"type":"v1","addr":"172.21.15.188:6835","nonce":906811420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:06:11.911 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 99 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa46406b4d0 msgr2=0x7fa46406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa46406b4d0 0x7fa46406d990 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa47c034000 tx=0x7fa47c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 msgr2=0x7fa48015cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 0x7fa48015cd60 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fa47800c780 tx=0x7fa47800cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 shutdown_connections 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa46406b4d0 0x7fa46406d990 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4800ce330 0x7fa48015cd60 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4800700a0 0x7fa4801f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 --2- 172.21.15.148:0/810129141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa48006abe0 0x7fa4800b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 >> 172.21.15.148:0/810129141 conn(0x7fa4800b2220 msgr2=0x7fa4800ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 shutdown_connections 2024-04-25T10:06:11.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:11.906+0000 7fa487c07640 1 -- 172.21.15.148:0/810129141 wait complete. 2024-04-25T10:06:11.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:12.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.014+0000 7f510f23a640 1 Processor -- start 2024-04-25T10:06:12.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.014+0000 7f510f23a640 1 -- start start 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f510f23a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51080b2cb0 0x7f51080d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f510f23a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080b97a0 con 0x7f51080d1020 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f510f23a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080d1c50 con 0x7f510815dbe0 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f510f23a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510815dfc0 con 0x7f51080b2cb0 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f5107fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51080b2cb0 0x7f51080d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f5107fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f51080b2cb0 0x7f51080d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60952/0 (socket says 172.21.15.148:60952) 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f510d7d7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f51080d1020 0x7f51080d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45322/0 (socket says 172.21.15.148:45322) 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.015+0000 7f5107fff640 1 -- 172.21.15.148:0/2319716596 learned_addr learned my addr 172.21.15.148:0/2319716596 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f5107fff640 1 -- 172.21.15.148:0/2319716596 >> v1:172.21.15.188:6789/0 conn(0x7f510815dbe0 legacy=0x7f51080d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f5107fff640 1 -- 172.21.15.148:0/2319716596 >> v1:172.21.15.148:6789/0 conn(0x7f51080d1020 legacy=0x7f51080d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f5107fff640 1 -- 172.21.15.148:0/2319716596 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080d1f30 con 0x7f51080b2cb0 2024-04-25T10:06:12.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f5107fff640 1 --2- 172.21.15.148:0/2319716596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51080b2cb0 0x7f51080d2730 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009870 tx=0x7f50fc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7113ae19c1f223e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f51077fe640 1 -- 172.21.15.148:0/2319716596 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc037070 con 0x7f51080b2cb0 2024-04-25T10:06:12.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f51077fe640 1 -- 172.21.15.148:0/2319716596 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc02ae50 con 0x7f51080b2cb0 2024-04-25T10:06:12.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.016+0000 7f51077fe640 1 -- 172.21.15.148:0/2319716596 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032860 con 0x7f51080b2cb0 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 -- 172.21.15.148:0/2319716596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51080b2cb0 msgr2=0x7f51080d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 --2- 172.21.15.148:0/2319716596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51080b2cb0 0x7f51080d2730 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009870 tx=0x7f50fc02a300 comp rx=0 tx=0).stop 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 -- 172.21.15.148:0/2319716596 shutdown_connections 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 --2- 172.21.15.148:0/2319716596 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51080b2cb0 0x7f51080d2730 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 -- 172.21.15.148:0/2319716596 >> 172.21.15.148:0/2319716596 conn(0x7f5108061a20 msgr2=0x7f5108061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 -- 172.21.15.148:0/2319716596 shutdown_connections 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.017+0000 7f510f23a640 1 -- 172.21.15.148:0/2319716596 wait complete. 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.018+0000 7f510f23a640 1 Processor -- start 2024-04-25T10:06:12.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.018+0000 7f510f23a640 1 -- start start 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510f23a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51080b2cb0 0x7f5108150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510f23a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51080d1020 0x7f51080b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510f23a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 0x7f51080b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510f23a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b92a0 con 0x7f510815dbe0 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510f23a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080d1a00 con 0x7f51080b2cb0 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510f23a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b9420 con 0x7f51080d1020 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 0x7f51080b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 0x7f51080b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60956/0 (socket says 172.21.15.148:60956) 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 -- 172.21.15.148:0/3457635909 learned_addr learned my addr 172.21.15.148:0/3457635909 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f5107fff640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51080d1020 0x7f51080b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510cfd6640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51080b2cb0 0x7f5108150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 -- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51080d1020 msgr2=0x7f51080b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51080d1020 0x7f51080b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 -- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51080b2cb0 msgr2=0x7f5108150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51080b2cb0 0x7f5108150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.019+0000 7f510d7d7640 1 -- 172.21.15.148:0/3457635909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510815e7e0 con 0x7f510815dbe0 2024-04-25T10:06:12.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f510cfd6640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51080b2cb0 0x7f5108150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f510d7d7640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 0x7f51080b4d70 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f510000c7b0 tx=0x7f510000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100017070 con 0x7f510815dbe0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000a640 con 0x7f510815dbe0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000a9f0 con 0x7f510815dbe0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510815eac0 con 0x7f510815dbe0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.020+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510815f080 con 0x7f510815dbe0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.021+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5108073990 con 0x7f510815dbe0 2024-04-25T10:06:12.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.021+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f510001a070 con 0x7f510815dbe0 2024-04-25T10:06:12.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.022+0000 7f51057fa640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50f406b4d0 0x7f50f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.022+0000 7f510cfd6640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50f406b4d0 0x7f50f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.022+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f5100089a20 con 0x7f510815dbe0 2024-04-25T10:06:12.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.022+0000 7f510cfd6640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50f406b4d0 0x7f50f406d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001660 tx=0x7f50f8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.026+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5100057fa0 con 0x7f510815dbe0 2024-04-25T10:06:12.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.198+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f510815d190 con 0x7f510815dbe0 2024-04-25T10:06:12.204 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.199+0000 7f51057fa640 1 -- 172.21.15.148:0/3457635909 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17232 (secure 0 0 0) 0x7f510005c040 con 0x7f510815dbe0 2024-04-25T10:06:12.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:12.205 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":173,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:06.807921+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:26.693014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:27.457155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.3","mappings":[{"from":1,"to":6}]},{"pgid":"26.b","mappings":[{"from":3,"to":0}]},{"pgid":"26.13","mappings":[{"from":1,"to":6}]},{"pgid":"26.2a","mappings":[{"from":1,"to":0}]},{"pgid":"26.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:12.206 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50f406b4d0 msgr2=0x7f50f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50f406b4d0 0x7f50f406d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001660 tx=0x7f50f8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 msgr2=0x7f51080b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 0x7f51080b4d70 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f510000c7b0 tx=0x7f510000cc70 comp rx=0 tx=0).stop 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 shutdown_connections 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f50f406b4d0 0x7f50f406d990 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510815dbe0 0x7f51080b4d70 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51080d1020 0x7f51080b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 --2- 172.21.15.148:0/3457635909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51080b2cb0 0x7f5108150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 >> 172.21.15.148:0/3457635909 conn(0x7f5108061a20 msgr2=0x7f510815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 shutdown_connections 2024-04-25T10:06:12.207 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.201+0000 7f510f23a640 1 -- 172.21.15.148:0/3457635909 wait complete. 2024-04-25T10:06:12.218 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:12.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.309+0000 7fde1dfc4640 1 Processor -- start 2024-04-25T10:06:12.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.309+0000 7fde1dfc4640 1 -- start start 2024-04-25T10:06:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.310+0000 7fde1dfc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde180ced80 0x7fde180d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.310+0000 7fde1dfc4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde18161710 0x7fde180d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.310+0000 7fde1dfc4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1815e000 con 0x7fde1815dc20 2024-04-25T10:06:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.310+0000 7fde1dfc4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde180d1de0 con 0x7fde18161710 2024-04-25T10:06:12.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.310+0000 7fde1dfc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde18161c60 con 0x7fde180ced80 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde177fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fde1815dc20 0x7fde180ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:45952/0 (socket says 172.21.15.148:45952) 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde17fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde18161710 0x7fde180d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde177fe640 1 -- 172.21.15.148:0/1201882013 learned_addr learned my addr 172.21.15.148:0/1201882013 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde16ffd640 1 --2- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde180ced80 0x7fde180d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde17fff640 1 -- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde180ced80 msgr2=0x7fde180d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde17fff640 1 --2- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde180ced80 0x7fde180d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde17fff640 1 -- 172.21.15.148:0/1201882013 >> v1:172.21.15.188:6789/0 conn(0x7fde1815dc20 legacy=0x7fde180ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.311+0000 7fde17fff640 1 -- 172.21.15.148:0/1201882013 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18161f40 con 0x7fde18161710 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.312+0000 7fde17fff640 1 --2- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde18161710 0x7fde180d2820 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde100097a0 tx=0x7fde1002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d527845ad63fc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.312+0000 7fde167fc640 1 -- 172.21.15.148:0/1201882013 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10037070 con 0x7fde18161710 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.312+0000 7fde167fc640 1 -- 172.21.15.148:0/1201882013 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10032440 con 0x7fde18161710 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.312+0000 7fde167fc640 1 -- 172.21.15.148:0/1201882013 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10032810 con 0x7fde18161710 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde18161710 msgr2=0x7fde180d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde18161710 0x7fde180d2820 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde100097a0 tx=0x7fde1002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1201882013 shutdown_connections 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde18161710 0x7fde180d2820 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1201882013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde180ced80 0x7fde180d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1201882013 >> 172.21.15.148:0/1201882013 conn(0x7fde180d0420 msgr2=0x7fde180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1201882013 shutdown_connections 2024-04-25T10:06:12.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.313+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1201882013 wait complete. 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.314+0000 7fde1dfc4640 1 Processor -- start 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.314+0000 7fde1dfc4640 1 -- start start 2024-04-25T10:06:12.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde1dfc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 0x7fde1815ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde1dfc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1815dc20 0x7fde1815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde1dfc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde18161710 0x7fde180d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde1dfc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180b5e50 con 0x7fde180ced80 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde1dfc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18162500 con 0x7fde1815dc20 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde1dfc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde180b5fd0 con 0x7fde18161710 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 0x7fde1815ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 0x7fde1815ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60972/0 (socket says 172.21.15.148:60972) 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 -- 172.21.15.148:0/1874392921 learned_addr learned my addr 172.21.15.148:0/1874392921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde16ffd640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1815dc20 0x7fde1815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde17fff640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde18161710 0x7fde180d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 -- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde18161710 msgr2=0x7fde180d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde18161710 0x7fde180d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 -- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1815dc20 msgr2=0x7fde1815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1815dc20 0x7fde1815c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.315+0000 7fde177fe640 1 -- 172.21.15.148:0/1874392921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde181e9270 con 0x7fde180ced80 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.316+0000 7fde177fe640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 0x7fde1815ef00 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fde0800caa0 tx=0x7fde0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.316+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08017070 con 0x7fde180ced80 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.316+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800ac00 con 0x7fde180ced80 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.316+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1815e420 con 0x7fde180ced80 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.316+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08012400 con 0x7fde180ced80 2024-04-25T10:06:12.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.316+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1815e9e0 con 0x7fde180ced80 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.317+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1815e230 con 0x7fde180ced80 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.317+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde0801a070 con 0x7fde180ced80 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.318+0000 7fddfffff640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde0406b4d0 0x7fde0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.318+0000 7fde16ffd640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde0406b4d0 0x7fde0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.318+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fde0808a000 con 0x7fde180ced80 2024-04-25T10:06:12.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.318+0000 7fde16ffd640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde0406b4d0 0x7fde0406d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fde180ca4f0 tx=0x7fde0c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.322+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde080586a0 con 0x7fde180ced80 2024-04-25T10:06:12.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.527+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fde180d16b0 con 0x7fde180ced80 2024-04-25T10:06:12.533 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:06:12.528+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:12.535 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.529+0000 7fddfffff640 1 -- 172.21.15.148:0/1874392921 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7fde0805c740 con 0x7fde180ced80 2024-04-25T10:06:12.535 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde0406b4d0 msgr2=0x7fde0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde0406b4d0 0x7fde0406d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fde180ca4f0 tx=0x7fde0c009190 comp rx=0 tx=0).stop 2024-04-25T10:06:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 msgr2=0x7fde1815ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 0x7fde1815ef00 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fde0800caa0 tx=0x7fde0800cf60 comp rx=0 tx=0).stop 2024-04-25T10:06:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 shutdown_connections 2024-04-25T10:06:12.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fde0406b4d0 0x7fde0406d990 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde18161710 0x7fde180d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde1815dc20 0x7fde1815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 --2- 172.21.15.148:0/1874392921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde180ced80 0x7fde1815ef00 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 >> 172.21.15.148:0/1874392921 conn(0x7fde180d0420 msgr2=0x7fde180cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 shutdown_connections 2024-04-25T10:06:12.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.531+0000 7fde1dfc4640 1 -- 172.21.15.148:0/1874392921 wait complete. 2024-04-25T10:06:12.540 INFO:tasks.ceph.mds.b.smithi188.stderr: -7335> 2024-04-25T10:05:24.071+0000 7ff677037640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:06:12.542 INFO:tasks.ceph.mds.b.smithi188.stderr: -6055> 2024-04-25T10:05:34.399+0000 7ff67102b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:12.545 INFO:tasks.ceph.mds.b.smithi188.stderr: -3277> 2024-04-25T10:05:36.152+0000 7ff66d023640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:05:34.949580+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={10977=0-12582912@1} caps={10977=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=10977 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x5634df989600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:06:12.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:12.573 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:06:12.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.641+0000 7f4bdd86f640 1 Processor -- start 2024-04-25T10:06:12.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.641+0000 7f4bdd86f640 1 -- start start 2024-04-25T10:06:12.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bdd86f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 0x7f4bd80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bdd86f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4bd80d0ab0 0x7f4bd80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bdd86f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80d2420 con 0x7f4bd80d1f30 2024-04-25T10:06:12.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bdd86f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd80d3030 con 0x7f4bd80d0ab0 2024-04-25T10:06:12.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bdd86f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80d2700 con 0x7f4bd815dac0 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd67fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4bd80d1f30 0x7f4bd80d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45334/0 (socket says 172.21.15.148:45334) 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd67fc640 1 -- 172.21.15.148:0/1092576684 learned_addr learned my addr 172.21.15.148:0/1092576684 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961908266 0 0) 0x7f4bd80d2420 con 0x7f4bd80d1f30 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc0003520 con 0x7f4bd80d1f30 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 0x7f4bd80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd77fe640 1 --2- 172.21.15.148:0/1092576684 >> v2:172.21.15.188:3300/0 conn(0x7f4bd80d0ab0 0x7f4bd80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.642+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110195764 0 0) 0x7f4bc0003520 con 0x7f4bd80d1f30 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 msgr2=0x7f4bd80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 0x7f4bd80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 >> v2:172.21.15.188:3300/0 conn(0x7f4bd80d0ab0 msgr2=0x7f4bd80b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 --2- 172.21.15.148:0/1092576684 >> v2:172.21.15.188:3300/0 conn(0x7f4bd80d0ab0 0x7f4bd80b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80d29e0 con 0x7f4bd80d1f30 2024-04-25T10:06:12.651 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4bcc0017f0 con 0x7f4bd80d1f30 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd77fe640 1 --2- 172.21.15.148:0/1092576684 >> v2:172.21.15.188:3300/0 conn(0x7f4bd80d0ab0 0x7f4bd80b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 >> v1:172.21.15.148:6789/0 conn(0x7f4bd80d1f30 legacy=0x7f4bd80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc00039b0 0x7f4bc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 0x7f4bc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0004d40 0x7f4bc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80d3030 con 0x7f4bc00039b0 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80d2700 con 0x7f4bc0004d40 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc000ba80 con 0x7f4bd815dac0 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc00039b0 0x7f4bc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd77fe640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0004d40 0x7f4bc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd67fc640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 0x7f4bc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd6ffd640 1 -- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 msgr2=0x7f4bc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd815dac0 0x7f4bc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd6ffd640 1 -- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0004d40 msgr2=0x7f4bc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0004d40 0x7f4bc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.643+0000 7f4bd6ffd640 1 -- 172.21.15.148:0/1092576684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0003520 con 0x7f4bc00039b0 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc00039b0 0x7f4bc0003dd0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009fc0 tx=0x7f4bc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e212c55b746e5755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8009c10 con 0x7f4bc00039b0 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc8036070 con 0x7f4bc00039b0 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bd5ffb640 1 -- 172.21.15.148:0/1092576684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8031880 con 0x7f4bc00039b0 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bdd86f640 1 -- 172.21.15.148:0/1092576684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc00039b0 msgr2=0x7f4bc0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/1092576684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bc00039b0 0x7f4bc0003dd0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009fc0 tx=0x7f4bc0005830 comp rx=0 tx=0).stop 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bd77fe640 1 -- 172.21.15.148:0/1092576684 reap_dead start 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bdd86f640 1 -- 172.21.15.148:0/1092576684 shutdown_connections 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bdd86f640 1 -- 172.21.15.148:0/1092576684 >> 172.21.15.148:0/1092576684 conn(0x7f4bd8061c30 msgr2=0x7f4bd806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bdd86f640 1 -- 172.21.15.148:0/1092576684 shutdown_connections 2024-04-25T10:06:12.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.644+0000 7f4bdd86f640 1 -- 172.21.15.148:0/1092576684 wait complete. 2024-04-25T10:06:12.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.646+0000 7f4bdd86f640 1 Processor -- start 2024-04-25T10:06:12.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.646+0000 7f4bdd86f640 1 -- start start 2024-04-25T10:06:12.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.646+0000 7f4bdd86f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc00039b0 0x7f4bd815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.646+0000 7f4bdd86f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd8143d20 0x7f4bd81441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bdd86f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 0x7f4bd8160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bdd86f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80b97d0 con 0x7f4bd81601d0 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bdd86f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd815dea0 con 0x7f4bc00039b0 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bdd86f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80b9950 con 0x7f4bd8143d20 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 0x7f4bd8160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 0x7f4bd8160650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60980/0 (socket says 172.21.15.148:60980) 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 -- 172.21.15.148:0/2141743168 learned_addr learned my addr 172.21.15.148:0/2141743168 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd67fc640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd8143d20 0x7f4bd81441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc00039b0 0x7f4bd815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 -- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd8143d20 msgr2=0x7f4bd81441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd8143d20 0x7f4bd81441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 -- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc00039b0 msgr2=0x7f4bd815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc00039b0 0x7f4bd815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 -- 172.21.15.148:0/2141743168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80c9f60 con 0x7f4bd81601d0 2024-04-25T10:06:12.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bd77fe640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 0x7f4bd8160650 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0011490 tx=0x7f4bd0011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd001d070 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd0004250 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd8156860 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000f040 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.647+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd81534c0 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.648+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd8073a00 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.649+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bd0020070 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.649+0000 7f4bbffff640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4bc406b4d0 0x7f4bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.649+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6332+0+0 (secure 0 0 0) 0x7f4bd0089770 con 0x7f4bd81601d0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.649+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4bc406b4d0 0x7f4bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.650+0000 7f4bd6ffd640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4bc406b4d0 0x7f4bc406d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8035000 tx=0x7f4bc802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.654+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bd0057d30 con 0x7f4bd81601d0 2024-04-25T10:06:12.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:12.861+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4bd80b3570 con 0x7f4bd81601d0 2024-04-25T10:06:13.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.531+0000 7f4bbffff640 1 -- 172.21.15.148:0/2141743168 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bd005bdd0 con 0x7f4bd81601d0 2024-04-25T10:06:13.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4bc406b4d0 msgr2=0x7f4bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4bc406b4d0 0x7f4bc406d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8035000 tx=0x7f4bc802f000 comp rx=0 tx=0).stop 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 msgr2=0x7f4bd8160650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 0x7f4bd8160650 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0011490 tx=0x7f4bd0011950 comp rx=0 tx=0).stop 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 shutdown_connections 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4bc406b4d0 0x7f4bc406d990 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bd81601d0 0x7f4bd8160650 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd8143d20 0x7f4bd81441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 --2- 172.21.15.148:0/2141743168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc00039b0 0x7f4bd815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 >> 172.21.15.148:0/2141743168 conn(0x7f4bd8061c30 msgr2=0x7f4bd80d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 shutdown_connections 2024-04-25T10:06:13.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.534+0000 7f4bdd86f640 1 -- 172.21.15.148:0/2141743168 wait complete. 2024-04-25T10:06:13.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:13.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.655+0000 7f70da438640 1 Processor -- start 2024-04-25T10:06:13.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.655+0000 7f70da438640 1 -- start start 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.655+0000 7f70da438640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70d40ce230 0x7f70d40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.655+0000 7f70da438640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d4072280 con 0x7f70d40cc4f0 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.655+0000 7f70da438640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d40714c0 con 0x7f70d415b270 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.655+0000 7f70da438640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d4071270 con 0x7f70d40ce230 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f70d40cc4f0 0x7f70d40cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55726/0 (socket says 172.21.15.148:55726) 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d3fff640 1 -- 172.21.15.148:0/673782625 learned_addr learned my addr 172.21.15.148:0/673782625 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d37fe640 1 --2- 172.21.15.148:0/673782625 >> v2:172.21.15.188:3300/0 conn(0x7f70d40ce230 0x7f70d40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171576346 0 0) 0x7f70d4072280 con 0x7f70d40cc4f0 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70bc003520 con 0x7f70d40cc4f0 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067794519 0 0) 0x7f70bc003520 con 0x7f70d40cc4f0 2024-04-25T10:06:13.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 >> v2:172.21.15.188:3300/0 conn(0x7f70d40ce230 msgr2=0x7f70d40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 --2- 172.21.15.148:0/673782625 >> v2:172.21.15.188:3300/0 conn(0x7f70d40ce230 0x7f70d40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 >> v1:172.21.15.188:6789/0 conn(0x7f70d415b270 legacy=0x7f70d415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d40717a0 con 0x7f70d40cc4f0 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f70c4001670 con 0x7f70d40cc4f0 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 >> v1:172.21.15.148:6789/0 conn(0x7f70d40cc4f0 legacy=0x7f70d40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0039b0 0x7f70bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc0051c0 0x7f70bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4071270 con 0x7f70bc004800 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00c060 con 0x7f70bc0051c0 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc00c260 con 0x7f70bc0039b0 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d37fe640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d89d5640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc0051c0 0x7f70bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.656+0000 7f70d3fff640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0039b0 0x7f70bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d37fe640 1 -- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0039b0 msgr2=0x7f70bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d37fe640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70bc0039b0 0x7f70bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d37fe640 1 -- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc0051c0 msgr2=0x7f70bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d37fe640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc0051c0 0x7f70bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d37fe640 1 -- 172.21.15.148:0/673782625 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc003520 con 0x7f70bc004800 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d89d5640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70bc0051c0 0x7f70bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d89d5640 1 -- 172.21.15.148:0/673782625 reap_dead start 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d37fe640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70bc004c80 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f70d40cd090 tx=0x7f70bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50d0d16837b12e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8040070 con 0x7f70bc004800 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8033670 con 0x7f70bc004800 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.657+0000 7f70d2ffd640 1 -- 172.21.15.148:0/673782625 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8009690 con 0x7f70bc004800 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 -- 172.21.15.148:0/673782625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 msgr2=0x7f70bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70bc004c80 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f70d40cd090 tx=0x7f70bc005830 comp rx=0 tx=0).stop 2024-04-25T10:06:13.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 -- 172.21.15.148:0/673782625 shutdown_connections 2024-04-25T10:06:13.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 --2- 172.21.15.148:0/673782625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70bc004c80 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 -- 172.21.15.148:0/673782625 >> 172.21.15.148:0/673782625 conn(0x7f70d406abe0 msgr2=0x7f70d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 -- 172.21.15.148:0/673782625 shutdown_connections 2024-04-25T10:06:13.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.658+0000 7f70da438640 1 -- 172.21.15.148:0/673782625 wait complete. 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.659+0000 7f70da438640 1 Processor -- start 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.659+0000 7f70da438640 1 -- start start 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70da438640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70d414fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70da438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d4150610 0x7f70d40c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70da438640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c4c10 0x7f70d41575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70da438640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40cc8d0 con 0x7f70bc004800 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70da438640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4071d60 con 0x7f70d40c4c10 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70da438640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40cca50 con 0x7f70d4150610 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70d414fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70d414fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42634/0 (socket says 172.21.15.148:42634) 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 -- 172.21.15.148:0/4168973585 learned_addr learned my addr 172.21.15.148:0/4168973585 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:13.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d89d5640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c4c10 0x7f70d41575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d37fe640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d4150610 0x7f70d40c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 -- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d4150610 msgr2=0x7f70d40c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d4150610 0x7f70d40c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 -- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c4c10 msgr2=0x7f70d41575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c4c10 0x7f70d41575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d3fff640 1 -- 172.21.15.148:0/4168973585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4156b30 con 0x7f70bc004800 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d89d5640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c4c10 0x7f70d41575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.660+0000 7f70d37fe640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d4150610 0x7f70d40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.661+0000 7f70d3fff640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70d414fdd0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f70c4001b80 tx=0x7f70c4007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.661+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4015070 con 0x7f70bc004800 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.661+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c400fcf0 con 0x7f70bc004800 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.661+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d4155e10 con 0x7f70bc004800 2024-04-25T10:06:13.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.661+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c400fea0 con 0x7f70bc004800 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.661+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d4156df0 con 0x7f70bc004800 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.662+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d40c5020 con 0x7f70bc004800 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.662+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70c4018070 con 0x7f70bc004800 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.663+0000 7f70d0ff9640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70c006b4d0 0x7f70c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.663+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6332+0+0 (secure 0 0 0) 0x7f70c408ab00 con 0x7f70bc004800 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.663+0000 7f70d37fe640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70c006b4d0 0x7f70c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.663+0000 7f70d37fe640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70c006b4d0 0x7f70c006d990 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f70c803d000 tx=0x7f70c8037000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.667+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70d40c5020 con 0x7f70bc004800 2024-04-25T10:06:13.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:13.836+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f70d4073a00 con 0x7f70bc004800 2024-04-25T10:06:14.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.532+0000 7f70d0ff9640 1 -- 172.21.15.148:0/4168973585 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f70c40597a0 con 0x7f70bc004800 2024-04-25T10:06:14.538 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:14.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70c006b4d0 msgr2=0x7f70c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70c006b4d0 0x7f70c006d990 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f70c803d000 tx=0x7f70c8037000 comp rx=0 tx=0).stop 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 msgr2=0x7f70d414fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70d414fdd0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f70c4001b80 tx=0x7f70c4007710 comp rx=0 tx=0).stop 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 shutdown_connections 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70c006b4d0 0x7f70c006d990 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70d40c4c10 0x7f70d41575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70d4150610 0x7f70d40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 --2- 172.21.15.148:0/4168973585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70bc004800 0x7f70d414fdd0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 >> 172.21.15.148:0/4168973585 conn(0x7f70d406abe0 msgr2=0x7f70d40b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 shutdown_connections 2024-04-25T10:06:14.542 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.535+0000 7f70da438640 1 -- 172.21.15.148:0/4168973585 wait complete. 2024-04-25T10:06:14.552 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.649+0000 7f67aafb6640 1 Processor -- start 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.649+0000 7f67aafb6640 1 -- start start 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67aafb6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67a40714d0 0x7f67a4166980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67aafb6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a40ceba0 con 0x7f67a4144d20 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67aafb6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a40cee80 con 0x7f67a41454b0 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67aafb6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40b4830 con 0x7f67a40714d0 2024-04-25T10:06:14.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67a8d52640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f67a4144d20 0x7f67a40ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55738/0 (socket says 172.21.15.148:55738) 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67a8d52640 1 -- 172.21.15.148:0/1519178734 learned_addr learned my addr 172.21.15.148:0/1519178734 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198838282 0 0) 0x7f67a40ceba0 con 0x7f67a4144d20 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f678c003520 con 0x7f67a4144d20 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.650+0000 7f67a9553640 1 --2- 172.21.15.148:0/1519178734 >> v2:172.21.15.188:3300/0 conn(0x7f67a40714d0 0x7f67a4166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 683152498 0 0) 0x7f678c003520 con 0x7f67a4144d20 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 >> v2:172.21.15.188:3300/0 conn(0x7f67a40714d0 msgr2=0x7f67a4166980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 --2- 172.21.15.148:0/1519178734 >> v2:172.21.15.188:3300/0 conn(0x7f67a40714d0 0x7f67a4166980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 >> v1:172.21.15.188:6789/0 conn(0x7f67a41454b0 legacy=0x7f67a4070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40cf160 con 0x7f67a4144d20 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6794001670 con 0x7f67a4144d20 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 >> v1:172.21.15.148:6789/0 conn(0x7f67a4144d20 legacy=0x7f67a40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f678c0039b0 0x7f678c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c004800 0x7f678c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f678c0051c0 0x7f678c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b4830 con 0x7f678c0051c0 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c00c090 con 0x7f678c0039b0 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c00c290 con 0x7f678c004800 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a9553640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f678c0051c0 0x7f678c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a8d52640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f678c0039b0 0x7f678c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a3fff640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c004800 0x7f678c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a9553640 1 -- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c004800 msgr2=0x7f678c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a9553640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c004800 0x7f678c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a9553640 1 -- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f678c0039b0 msgr2=0x7f678c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a9553640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f678c0039b0 0x7f678c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.651+0000 7f67a9553640 1 -- 172.21.15.148:0/1519178734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c003520 con 0x7f678c0051c0 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67a9553640 1 -- 172.21.15.148:0/1519178734 reap_dead start 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67a9553640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f678c0051c0 0x7f678c00b9c0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f679c009fc0 tx=0x7f678c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558069a86d312282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c009c10 con 0x7f678c0051c0 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c003990 con 0x7f678c0051c0 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67a37fe640 1 -- 172.21.15.148:0/1519178734 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c02d070 con 0x7f678c0051c0 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 -- 172.21.15.148:0/1519178734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f678c0051c0 msgr2=0x7f678c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f678c0051c0 0x7f678c00b9c0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f679c009fc0 tx=0x7f678c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 -- 172.21.15.148:0/1519178734 shutdown_connections 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 --2- 172.21.15.148:0/1519178734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f678c0051c0 0x7f678c00b9c0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 -- 172.21.15.148:0/1519178734 >> 172.21.15.148:0/1519178734 conn(0x7f67a40b3530 msgr2=0x7f67a40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 -- 172.21.15.148:0/1519178734 shutdown_connections 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.652+0000 7f67aafb6640 1 -- 172.21.15.148:0/1519178734 wait complete. 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 Processor -- start 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 -- start start 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c0051c0 0x7f67a413df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a41e3e80 0x7f67a41e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 0x7f67a41e4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4145280 con 0x7f67a41e28a0 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4145100 con 0x7f67a41e3e80 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.654+0000 7f67aafb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4070dc0 con 0x7f678c0051c0 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 0x7f67a41e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 0x7f67a41e4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42652/0 (socket says 172.21.15.148:42652) 2024-04-25T10:06:14.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 -- 172.21.15.148:0/520293953 learned_addr learned my addr 172.21.15.148:0/520293953 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a3fff640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a41e3e80 0x7f67a41e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a8d52640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c0051c0 0x7f67a413df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 -- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c0051c0 msgr2=0x7f67a413df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c0051c0 0x7f67a413df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 -- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a41e3e80 msgr2=0x7f67a41e2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a41e3e80 0x7f67a41e2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 -- 172.21.15.148:0/520293953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a41e2de0 con 0x7f67a41e28a0 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a8d52640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c0051c0 0x7f67a413df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a3fff640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a41e3e80 0x7f67a41e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:14.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a9553640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 0x7f67a41e4340 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f679c02af10 tx=0x7f679c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c009c10 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c044070 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a41e30c0 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c03c880 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.655+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a41e3680 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.656+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4073a00 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.657+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f679c038070 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.657+0000 7f67a17fa640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f679006b500 0x7f679006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.657+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5894+0+0 (secure 0 0 0) 0x7f679c0ab0a0 con 0x7f67a41e28a0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.657+0000 7f67a8d52640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f679006b500 0x7f679006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.658+0000 7f67a8d52640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f679006b500 0x7f679006d9c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f67940015d0 tx=0x7f6794009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.661+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f679c079f80 con 0x7f67a41e28a0 2024-04-25T10:06:14.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:14.833+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67a40b0f50 con 0x7f67a41e28a0 2024-04-25T10:06:15.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.534+0000 7f67a17fa640 1 -- 172.21.15.148:0/520293953 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f679c079880 con 0x7f67a41e28a0 2024-04-25T10:06:15.540 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.538+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f679006b500 msgr2=0x7f679006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.538+0000 7f67aafb6640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f679006b500 0x7f679006d9c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f67940015d0 tx=0x7f6794009040 comp rx=0 tx=0).stop 2024-04-25T10:06:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.538+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 msgr2=0x7f67a41e4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.538+0000 7f67aafb6640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 0x7f67a41e4340 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f679c02af10 tx=0x7f679c009750 comp rx=0 tx=0).stop 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.538+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 shutdown_connections 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.538+0000 7f67aafb6640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f679006b500 0x7f679006d9c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.539+0000 7f67aafb6640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67a41e28a0 0x7f67a41e4340 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.539+0000 7f67aafb6640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67a41e3e80 0x7f67a41e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.539+0000 7f67aafb6640 1 --2- 172.21.15.148:0/520293953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f678c0051c0 0x7f67a413df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.539+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 >> 172.21.15.148:0/520293953 conn(0x7f67a40b3530 msgr2=0x7f67a40b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.539+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 shutdown_connections 2024-04-25T10:06:15.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.539+0000 7f67aafb6640 1 -- 172.21.15.148:0/520293953 wait complete. 2024-04-25T10:06:15.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:15.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.664+0000 7fdcf070a640 1 Processor -- start 2024-04-25T10:06:15.669 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.664+0000 7fdcf070a640 1 -- start start 2024-04-25T10:06:15.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcf070a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdce806abe0 0x7fdce806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcf070a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdce815dd80 0x7fdce80b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcf070a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce806b500 con 0x7fdce80ad570 2024-04-25T10:06:15.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcf070a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce806e410 con 0x7fdce815dd80 2024-04-25T10:06:15.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcf070a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce80b3930 con 0x7fdce806abe0 2024-04-25T10:06:15.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdce815dd80 0x7fdce80b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdce815dd80 0x7fdce80b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42654/0 (socket says 172.21.15.148:42654) 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 -- 172.21.15.148:0/658314639 learned_addr learned my addr 172.21.15.148:0/658314639 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 -- 172.21.15.148:0/658314639 >> v2:172.21.15.188:3300/0 conn(0x7fdce806abe0 msgr2=0x7fdce806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 --2- 172.21.15.148:0/658314639 >> v2:172.21.15.188:3300/0 conn(0x7fdce806abe0 0x7fdce806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 -- 172.21.15.148:0/658314639 >> v1:172.21.15.188:6789/0 conn(0x7fdce80ad570 legacy=0x7fdce806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.665+0000 7fdcedca5640 1 -- 172.21.15.148:0/658314639 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80b3c10 con 0x7fdce815dd80 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.666+0000 7fdcedca5640 1 --2- 172.21.15.148:0/658314639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce815dd80 0x7fdce80b5290 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fdce806e590 tx=0x7fdcd802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6c689f86e299a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.666+0000 7fdced4a4640 1 -- 172.21.15.148:0/658314639 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8037070 con 0x7fdce815dd80 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.666+0000 7fdced4a4640 1 -- 172.21.15.148:0/658314639 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd8032440 con 0x7fdce815dd80 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.666+0000 7fdced4a4640 1 -- 172.21.15.148:0/658314639 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8032810 con 0x7fdce815dd80 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.666+0000 7fdcf070a640 1 -- 172.21.15.148:0/658314639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce815dd80 msgr2=0x7fdce80b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.666+0000 7fdcf070a640 1 --2- 172.21.15.148:0/658314639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce815dd80 0x7fdce80b5290 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fdce806e590 tx=0x7fdcd802a210 comp rx=0 tx=0).stop 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.667+0000 7fdcf070a640 1 -- 172.21.15.148:0/658314639 shutdown_connections 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.667+0000 7fdcf070a640 1 --2- 172.21.15.148:0/658314639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce815dd80 0x7fdce80b5290 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.667+0000 7fdcf070a640 1 --2- 172.21.15.148:0/658314639 >> v2:172.21.15.188:3300/0 conn(0x7fdce806abe0 0x7fdce806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.667+0000 7fdcf070a640 1 -- 172.21.15.148:0/658314639 >> 172.21.15.148:0/658314639 conn(0x7fdce80b5af0 msgr2=0x7fdce80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.667+0000 7fdcf070a640 1 -- 172.21.15.148:0/658314639 shutdown_connections 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.667+0000 7fdcf070a640 1 -- 172.21.15.148:0/658314639 wait complete. 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 Processor -- start 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 -- start start 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdce806abe0 0x7fdce8160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 0x7fdce814a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce815dd80 0x7fdce81e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80b57d0 con 0x7fdce80ad570 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce806e940 con 0x7fdce815dd80 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcf070a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8070280 con 0x7fdce806abe0 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.668+0000 7fdcedca5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 0x7fdce814a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 0x7fdce814a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42658/0 (socket says 172.21.15.148:42658) 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 -- 172.21.15.148:0/2382156695 learned_addr learned my addr 172.21.15.148:0/2382156695 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcee4a6640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdce806abe0 0x7fdce8160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 -- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdce806abe0 msgr2=0x7fdce8160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdceeca7640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce815dd80 0x7fdce81e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdce806abe0 0x7fdce8160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 -- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce815dd80 msgr2=0x7fdce81e85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce815dd80 0x7fdce81e85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 -- 172.21.15.148:0/2382156695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8160fa0 con 0x7fdce80ad570 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdceeca7640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce815dd80 0x7fdce81e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcedca5640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 0x7fdce814a0a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000c00 tx=0x7fdcd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8041070 con 0x7fdce80ad570 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd803c870 con 0x7fdce80ad570 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8037040 con 0x7fdce80ad570 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce814a5e0 con 0x7fdce80ad570 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.669+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce81e7060 con 0x7fdce80ad570 2024-04-25T10:06:15.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.670+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce8073a00 con 0x7fdce80ad570 2024-04-25T10:06:15.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.671+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcd8044070 con 0x7fdce80ad570 2024-04-25T10:06:15.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.671+0000 7fdcdeffd640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdccc06b4d0 0x7fdccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.671+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdcd80aaba0 con 0x7fdce80ad570 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.671+0000 7fdcee4a6640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdccc06b4d0 0x7fdccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.671+0000 7fdcee4a6640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdccc06b4d0 0x7fdccc06d990 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009890 tx=0x7fdce4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.680 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.675+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcd8079d40 con 0x7fdce80ad570 2024-04-25T10:06:15.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.844+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdce80b0b90 con 0x7fdce80ad570 2024-04-25T10:06:15.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.849+0000 7fdcdeffd640 1 -- 172.21.15.148:0/2382156695 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdcd8079640 con 0x7fdce80ad570 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdccc06b4d0 msgr2=0x7fdccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdccc06b4d0 0x7fdccc06d990 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009890 tx=0x7fdce4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 msgr2=0x7fdce814a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 0x7fdce814a0a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8000c00 tx=0x7fdcd80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 shutdown_connections 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdccc06b4d0 0x7fdccc06d990 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce815dd80 0x7fdce81e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce80ad570 0x7fdce814a0a0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 --2- 172.21.15.148:0/2382156695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdce806abe0 0x7fdce8160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 >> 172.21.15.148:0/2382156695 conn(0x7fdce80b5af0 msgr2=0x7fdce80ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 shutdown_connections 2024-04-25T10:06:15.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.851+0000 7fdcf070a640 1 -- 172.21.15.148:0/2382156695 wait complete. 2024-04-25T10:06:15.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:06:15.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.959+0000 7fd9f4adb640 1 Processor -- start 2024-04-25T10:06:15.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.959+0000 7fd9f4adb640 1 -- start start 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9f4adb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f00cfec0 0x7fd9f00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9f4adb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f0071480 con 0x7fd9f0154ee0 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9f4adb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f0071760 con 0x7fd9f00ce3c0 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9f4adb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f00ce830 con 0x7fd9f00cfec0 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9edd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f00cfec0 0x7fd9f00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9ee59c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd9f0154ee0 0x7fd9f00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55754/0 (socket says 172.21.15.148:55754) 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9edd9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd9f00cfec0 0x7fd9f00cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42660/0 (socket says 172.21.15.148:42660) 2024-04-25T10:06:15.966 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9ee59c640 1 -- 172.21.15.148:0/2605946030 learned_addr learned my addr 172.21.15.148:0/2605946030 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:15.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9edd9b640 1 -- 172.21.15.148:0/2605946030 >> v1:172.21.15.188:6789/0 conn(0x7fd9f00ce3c0 legacy=0x7fd9f006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9edd9b640 1 -- 172.21.15.148:0/2605946030 >> v1:172.21.15.148:6789/0 conn(0x7fd9f0154ee0 legacy=0x7fd9f00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.960+0000 7fd9edd9b640 1 -- 172.21.15.148:0/2605946030 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0071a40 con 0x7fd9f00cfec0 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.961+0000 7fd9edd9b640 1 --2- 172.21.15.148:0/2605946030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f00cfec0 0x7fd9f00cde80 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097a0 tx=0x7fd9e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d8fc0aadb338c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.961+0000 7fd9ed59a640 1 -- 172.21.15.148:0/2605946030 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4037070 con 0x7fd9f00cfec0 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.961+0000 7fd9ed59a640 1 -- 172.21.15.148:0/2605946030 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e4032440 con 0x7fd9f00cfec0 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.961+0000 7fd9ed59a640 1 -- 172.21.15.148:0/2605946030 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4032810 con 0x7fd9f00cfec0 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 -- 172.21.15.148:0/2605946030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f00cfec0 msgr2=0x7fd9f00cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/2605946030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f00cfec0 0x7fd9f00cde80 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097a0 tx=0x7fd9e402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 -- 172.21.15.148:0/2605946030 shutdown_connections 2024-04-25T10:06:15.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/2605946030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f00cfec0 0x7fd9f00cde80 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 -- 172.21.15.148:0/2605946030 >> 172.21.15.148:0/2605946030 conn(0x7fd9f006abe0 msgr2=0x7fd9f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 -- 172.21.15.148:0/2605946030 shutdown_connections 2024-04-25T10:06:15.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.962+0000 7fd9f4adb640 1 -- 172.21.15.148:0/2605946030 wait complete. 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.963+0000 7fd9f4adb640 1 Processor -- start 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.963+0000 7fd9f4adb640 1 -- start start 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9f4adb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f00ce3c0 0x7fd9f0156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9f4adb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f00cfec0 0x7fd9f01f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9f4adb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 0x7fd9f0147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9f4adb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f0071230 con 0x7fd9f0154ee0 2024-04-25T10:06:15.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9f4adb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f01552c0 con 0x7fd9f00ce3c0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9f4adb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f0155440 con 0x7fd9f00cfec0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9eed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 0x7fd9f0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9eed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 0x7fd9f0147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42668/0 (socket says 172.21.15.148:42668) 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9eed9d640 1 -- 172.21.15.148:0/1189177361 learned_addr learned my addr 172.21.15.148:0/1189177361 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9ee59c640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f00ce3c0 0x7fd9f0156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.964+0000 7fd9edd9b640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f00cfec0 0x7fd9f01f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9eed9d640 1 -- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f00cfec0 msgr2=0x7fd9f01f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9eed9d640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f00cfec0 0x7fd9f01f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9eed9d640 1 -- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f00ce3c0 msgr2=0x7fd9f0156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9eed9d640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f00ce3c0 0x7fd9f0156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9eed9d640 1 -- 172.21.15.148:0/1189177361 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0147770 con 0x7fd9f0154ee0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9ee59c640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f00ce3c0 0x7fd9f0156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9eed9d640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 0x7fd9f0147230 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c440 tx=0x7fd9e800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8011790 con 0x7fd9f0154ee0 2024-04-25T10:06:15.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e8017070 con 0x7fd9f0154ee0 2024-04-25T10:06:15.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f0157080 con 0x7fd9f0154ee0 2024-04-25T10:06:15.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8011b70 con 0x7fd9f0154ee0 2024-04-25T10:06:15.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.965+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f0157640 con 0x7fd9f0154ee0 2024-04-25T10:06:15.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.966+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f0073990 con 0x7fd9f0154ee0 2024-04-25T10:06:15.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.966+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9e8011d10 con 0x7fd9f0154ee0 2024-04-25T10:06:15.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.967+0000 7fd9deffd640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d006b4d0 0x7fd9d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.967+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd9e8089e00 con 0x7fd9f0154ee0 2024-04-25T10:06:15.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.967+0000 7fd9ee59c640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d006b4d0 0x7fd9d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.967+0000 7fd9ee59c640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d006b4d0 0x7fd9d006d990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00012e0 tx=0x7fd9e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:15.971+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9f0147770 con 0x7fd9f0154ee0 2024-04-25T10:06:16.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.213+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7fd9f006d360 con 0x7fd9f0154ee0 2024-04-25T10:06:16.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.535+0000 7fd9deffd640 1 -- 172.21.15.148:0/1189177361 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v573) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd9e8058e80 con 0x7fd9f0154ee0 2024-04-25T10:06:16.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d006b4d0 msgr2=0x7fd9d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d006b4d0 0x7fd9d006d990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00012e0 tx=0x7fd9e0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 msgr2=0x7fd9f0147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 0x7fd9f0147230 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800c440 tx=0x7fd9e800c900 comp rx=0 tx=0).stop 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 shutdown_connections 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd9d006b4d0 0x7fd9d006d990 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f0154ee0 0x7fd9f0147230 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f00cfec0 0x7fd9f01f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 --2- 172.21.15.148:0/1189177361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f00ce3c0 0x7fd9f0156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 >> 172.21.15.148:0/1189177361 conn(0x7fd9f006abe0 msgr2=0x7fd9f00b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 shutdown_connections 2024-04-25T10:06:16.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.537+0000 7fd9f4adb640 1 -- 172.21.15.148:0/1189177361 wait complete. 2024-04-25T10:06:16.555 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:06:16.555 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:06:16.556 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:16.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 Processor -- start 2024-04-25T10:06:16.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 -- start start 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 0x7f24881625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24880d1810 0x7f24880d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880cce50 con 0x7f24880cc8e0 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24880d2180 con 0x7f24880d1810 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.648+0000 7f248e81f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2488161af0 con 0x7f248815dc90 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f24877fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f24880cc8e0 0x7f248815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55770/0 (socket says 172.21.15.148:55770) 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f24877fe640 1 -- 172.21.15.148:0/2664449792 learned_addr learned my addr 172.21.15.148:0/2664449792 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771813297 0 0) 0x7f24880cce50 con 0x7f24880cc8e0 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2470003520 con 0x7f24880cc8e0 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2487fff640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 0x7f24881625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f248cdbc640 1 --2- 172.21.15.148:0/2664449792 >> v2:172.21.15.188:3300/0 conn(0x7f24880d1810 0x7f24880d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3127033332 0 0) 0x7f2470003520 con 0x7f24880cc8e0 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 msgr2=0x7f24881625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 0x7f24881625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 >> v2:172.21.15.188:3300/0 conn(0x7f24880d1810 msgr2=0x7f24880d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 --2- 172.21.15.148:0/2664449792 >> v2:172.21.15.188:3300/0 conn(0x7f24880d1810 0x7f24880d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2488161dd0 con 0x7f24880cc8e0 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f247c001670 con 0x7f24880cc8e0 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f248cdbc640 1 --2- 172.21.15.148:0/2664449792 >> v2:172.21.15.188:3300/0 conn(0x7f24880d1810 0x7f24880d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 >> v1:172.21.15.148:6789/0 conn(0x7f24880cc8e0 legacy=0x7f248815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 0x7f24700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470003f80 0x7f2470004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2470004d40 0x7f24700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880d2180 con 0x7f2470004d40 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488161af0 con 0x7f2470003f80 2024-04-25T10:06:16.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247000bb20 con 0x7f248815dc90 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f248cdbc640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2470004d40 0x7f24700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f2487fff640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 0x7f24700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.649+0000 7f24877fe640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470003f80 0x7f2470004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f248cdbc640 1 -- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 msgr2=0x7f24700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f248cdbc640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248815dc90 0x7f24700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f248cdbc640 1 -- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470003f80 msgr2=0x7f2470004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f248cdbc640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470003f80 0x7f2470004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f248cdbc640 1 -- 172.21.15.148:0/2664449792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470003520 con 0x7f2470004d40 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f24877fe640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470003f80 0x7f2470004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f248cdbc640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2470004d40 0x7f24700051c0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f24800073e0 tx=0x7f2470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410fc814e1f39cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480009c80 con 0x7f2470004d40 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248002c680 con 0x7f2470004d40 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.650+0000 7f2486ffd640 1 -- 172.21.15.148:0/2664449792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248002ca30 con 0x7f2470004d40 2024-04-25T10:06:16.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248e81f640 1 -- 172.21.15.148:0/2664449792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2470004d40 msgr2=0x7f24700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248e81f640 1 --2- 172.21.15.148:0/2664449792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2470004d40 0x7f24700051c0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f24800073e0 tx=0x7f2470005830 comp rx=0 tx=0).stop 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248cdbc640 1 -- 172.21.15.148:0/2664449792 reap_dead start 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248e81f640 1 -- 172.21.15.148:0/2664449792 shutdown_connections 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248e81f640 1 -- 172.21.15.148:0/2664449792 >> 172.21.15.148:0/2664449792 conn(0x7f248806f590 msgr2=0x7f248806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248e81f640 1 -- 172.21.15.148:0/2664449792 shutdown_connections 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.651+0000 7f248e81f640 1 -- 172.21.15.148:0/2664449792 wait complete. 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.652+0000 7f248e81f640 1 Processor -- start 2024-04-25T10:06:16.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.652+0000 7f248e81f640 1 -- start start 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248e81f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470004d40 0x7f24881556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248e81f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 0x7f24881f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248e81f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24880b4b60 0x7f248815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248e81f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248815e070 con 0x7f24880b6c30 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248e81f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880cd130 con 0x7f2470004d40 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248e81f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248815e1f0 con 0x7f24880b4b60 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 0x7f24881f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 0x7f24881f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42696/0 (socket says 172.21.15.148:42696) 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 -- 172.21.15.148:0/477964227 learned_addr learned my addr 172.21.15.148:0/477964227 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f2487fff640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470004d40 0x7f24881556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f248cdbc640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24880b4b60 0x7f248815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 -- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24880b4b60 msgr2=0x7f248815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24880b4b60 0x7f248815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 -- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470004d40 msgr2=0x7f24881556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470004d40 0x7f24881556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f24877fe640 1 -- 172.21.15.148:0/477964227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880b5420 con 0x7f24880b6c30 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.653+0000 7f2487fff640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470004d40 0x7f24881556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.654+0000 7f24877fe640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 0x7f24881f3fc0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e7d0 tx=0x7f247c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.654+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00c870 con 0x7f24880b6c30 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.654+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c018070 con 0x7f24880b6c30 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.654+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248815fb30 con 0x7f24880b6c30 2024-04-25T10:06:16.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.654+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00cc50 con 0x7f24880b6c30 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.654+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881600f0 con 0x7f24880b6c30 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.655+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488073990 con 0x7f24880b6c30 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.655+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f247c01d460 con 0x7f24880b6c30 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.656+0000 7f2484ff9640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b4d0 0x7f247406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.656+0000 7f2487fff640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b4d0 0x7f247406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.656+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f247c089fa0 con 0x7f24880b6c30 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.656+0000 7f2487fff640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b4d0 0x7f247406d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2478009ab0 tx=0x7f2478008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.660+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f247c058f90 con 0x7f24880b6c30 2024-04-25T10:06:16.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.832+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24880cf6b0 con 0x7f24880b6c30 2024-04-25T10:06:16.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.833+0000 7f2484ff9640 1 -- 172.21.15.148:0/477964227 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+13351 (secure 0 0 0) 0x7f247c058890 con 0x7f24880b6c30 2024-04-25T10:06:16.839 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:16.839 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":176,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:15.533440+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/906811420":"2024-04-26T10:06:12.528881+0000","172.21.15.188:6833/906811420":"2024-04-26T10:06:12.528881+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b4d0 msgr2=0x7f247406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b4d0 0x7f247406d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2478009ab0 tx=0x7f2478008040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 msgr2=0x7f24881f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 0x7f24881f3fc0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e7d0 tx=0x7f247c00ec90 comp rx=0 tx=0).stop 2024-04-25T10:06:16.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 shutdown_connections 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f247406b4d0 0x7f247406d990 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24880b4b60 0x7f248815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24880b6c30 0x7f24881f3fc0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 --2- 172.21.15.148:0/477964227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2470004d40 0x7f24881556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 >> 172.21.15.148:0/477964227 conn(0x7f248806f590 msgr2=0x7f248806bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 shutdown_connections 2024-04-25T10:06:16.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.835+0000 7f248e81f640 1 -- 172.21.15.148:0/477964227 wait complete. 2024-04-25T10:06:16.852 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:16.852 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:16.953 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.946+0000 7fa98806b640 1 Processor -- start 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.947+0000 7fa98806b640 1 -- start start 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.947+0000 7fa98806b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 0x7fa9800cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.947+0000 7fa98806b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 0x7fa9800cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.947+0000 7fa98806b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9800c75e0 con 0x7fa98015bb50 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.947+0000 7fa98806b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9800c8680 con 0x7fa9800cbf20 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.947+0000 7fa98806b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800cff80 con 0x7fa9800cf600 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa986608640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 0x7fa9800cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985606640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 0x7fa9800cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa986608640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 0x7fa9800cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:50416/0 (socket says 172.21.15.148:50416) 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985e07640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa98015bb50 0x7fa9800c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:33434/0 (socket says 172.21.15.148:33434) 2024-04-25T10:06:16.954 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985606640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 0x7fa9800cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:57104/0 (socket says 172.21.15.148:57104) 2024-04-25T10:06:16.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa986608640 1 -- 172.21.15.148:0/3667108473 learned_addr learned my addr 172.21.15.148:0/3667108473 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:16.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985606640 1 -- 172.21.15.148:0/3667108473 >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 msgr2=0x7fa9800cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985606640 1 --2- 172.21.15.148:0/3667108473 >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 0x7fa9800cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985606640 1 -- 172.21.15.148:0/3667108473 >> v1:172.21.15.188:6789/0 conn(0x7fa98015bb50 legacy=0x7fa9800c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa985606640 1 -- 172.21.15.148:0/3667108473 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800c78c0 con 0x7fa9800cf600 2024-04-25T10:06:16.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.948+0000 7fa986608640 1 --2- 172.21.15.148:0/3667108473 >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 0x7fa9800cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:16.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.949+0000 7fa985606640 1 --2- 172.21.15.148:0/3667108473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 0x7fa9800cfa40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa9700097a0 tx=0x7fa97002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a27f09d7fae8fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.949+0000 7fa984e05640 1 -- 172.21.15.148:0/3667108473 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970037070 con 0x7fa9800cf600 2024-04-25T10:06:16.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.949+0000 7fa984e05640 1 -- 172.21.15.148:0/3667108473 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa970032440 con 0x7fa9800cf600 2024-04-25T10:06:16.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.949+0000 7fa984e05640 1 -- 172.21.15.148:0/3667108473 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970032810 con 0x7fa9800cf600 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 -- 172.21.15.148:0/3667108473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 msgr2=0x7fa9800cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 --2- 172.21.15.148:0/3667108473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 0x7fa9800cfa40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa9700097a0 tx=0x7fa97002a210 comp rx=0 tx=0).stop 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 -- 172.21.15.148:0/3667108473 shutdown_connections 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 --2- 172.21.15.148:0/3667108473 >> v2:172.21.15.188:3300/0 conn(0x7fa9800cbf20 0x7fa9800cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 --2- 172.21.15.148:0/3667108473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9800cf600 0x7fa9800cfa40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 -- 172.21.15.148:0/3667108473 >> 172.21.15.148:0/3667108473 conn(0x7fa98006f500 msgr2=0x7fa98006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 -- 172.21.15.148:0/3667108473 shutdown_connections 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.950+0000 7fa98806b640 1 -- 172.21.15.148:0/3667108473 wait complete. 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.951+0000 7fa98806b640 1 Processor -- start 2024-04-25T10:06:16.957 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.951+0000 7fa98806b640 1 -- start start 2024-04-25T10:06:16.959 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9800cbf20 0x7fa9800b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 0x7fa9801e7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015bb50 0x7fa98006d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c8160 con 0x7fa9800cf600 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c7e80 con 0x7fa9800cbf20 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9800c72a0 con 0x7fa98015bb50 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 0x7fa9801e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 0x7fa9801e7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42698/0 (socket says 172.21.15.148:42698) 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 -- 172.21.15.148:0/4243565238 learned_addr learned my addr 172.21.15.148:0/4243565238 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa986608640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015bb50 0x7fa98006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985e07640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9800cbf20 0x7fa9800b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 -- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015bb50 msgr2=0x7fa98006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015bb50 0x7fa98006d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 -- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9800cbf20 msgr2=0x7fa9800b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9800cbf20 0x7fa9800b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 -- 172.21.15.148:0/4243565238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9801e66f0 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985e07640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9800cbf20 0x7fa9800b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa985606640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 0x7fa9801e7790 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa970000c00 tx=0x7fa9700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970041070 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9801e69d0 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97003c890 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa970037040 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.952+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9801e6f90 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.953+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa980071ce0 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.954+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa970044070 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.954+0000 7fa9767fc640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa96406b4d0 0x7fa96406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.954+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa9700aab60 con 0x7fa9800cf600 2024-04-25T10:06:16.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.954+0000 7fa985e07640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa96406b4d0 0x7fa96406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.955+0000 7fa985e07640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa96406b4d0 0x7fa96406d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa98006bdb0 tx=0x7fa97c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:16.958+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa970079b60 con 0x7fa9800cf600 2024-04-25T10:06:17.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.133+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9800b0d60 con 0x7fa9800cf600 2024-04-25T10:06:17.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.133+0000 7fa9767fc640 1 -- 172.21.15.148:0/4243565238 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa970079460 con 0x7fa9800cf600 2024-04-25T10:06:17.139 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa96406b4d0 msgr2=0x7fa96406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa96406b4d0 0x7fa96406d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa98006bdb0 tx=0x7fa97c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 msgr2=0x7fa9801e7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 0x7fa9801e7790 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fa970000c00 tx=0x7fa9700046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 shutdown_connections 2024-04-25T10:06:17.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa96406b4d0 0x7fa96406d990 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa98015bb50 0x7fa98006d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9800cf600 0x7fa9801e7790 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 --2- 172.21.15.148:0/4243565238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9800cbf20 0x7fa9800b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 >> 172.21.15.148:0/4243565238 conn(0x7fa98006f500 msgr2=0x7fa9800cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 shutdown_connections 2024-04-25T10:06:17.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.135+0000 7fa98806b640 1 -- 172.21.15.148:0/4243565238 wait complete. 2024-04-25T10:06:17.152 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:06:17.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.244+0000 7f79ea8d2640 1 Processor -- start 2024-04-25T10:06:17.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.244+0000 7f79ea8d2640 1 -- start start 2024-04-25T10:06:17.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79ea8d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e415dc90 0x7f79e41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79ea8d2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f79e40cc8e0 0x7f79e415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79ea8d2640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79e40d1810 0x7f79e40d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79ea8d2640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e40d2180 con 0x7f79e40d1810 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79ea8d2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e415f390 con 0x7f79e40cc8e0 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79ea8d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40cce50 con 0x7f79e415dc90 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79e40d1810 0x7f79e40d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f79e40d1810 0x7f79e40d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42714/0 (socket says 172.21.15.148:42714) 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 -- 172.21.15.148:0/3014518784 learned_addr learned my addr 172.21.15.148:0/3014518784 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e3fff640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e415dc90 0x7f79e41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e37fe640 1 --2- 172.21.15.148:0/3014518784 >> v2:172.21.15.188:3300/0 conn(0x7f79e40cc8e0 0x7f79e415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 -- 172.21.15.148:0/3014518784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e415dc90 msgr2=0x7f79e41625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e415dc90 0x7f79e41625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 -- 172.21.15.148:0/3014518784 >> v2:172.21.15.188:3300/0 conn(0x7f79e40cc8e0 msgr2=0x7f79e415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 --2- 172.21.15.148:0/3014518784 >> v2:172.21.15.188:3300/0 conn(0x7f79e40cc8e0 0x7f79e415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e8e6f640 1 -- 172.21.15.148:0/3014518784 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e4161af0 con 0x7f79e40d1810 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.245+0000 7f79e3fff640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e415dc90 0x7f79e41625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.246+0000 7f79e8e6f640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40d1c40 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f79e40cf390 tx=0x7f79dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c248fe53861fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.246+0000 7f79e2ffd640 1 -- 172.21.15.148:0/3014518784 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc036070 con 0x7f79e40d1810 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.246+0000 7f79e2ffd640 1 -- 172.21.15.148:0/3014518784 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc031600 con 0x7f79e40d1810 2024-04-25T10:06:17.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.246+0000 7f79e2ffd640 1 -- 172.21.15.148:0/3014518784 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0319d0 con 0x7f79e40d1810 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 -- 172.21.15.148:0/3014518784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 msgr2=0x7f79e40d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40d1c40 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f79e40cf390 tx=0x7f79dc02a610 comp rx=0 tx=0).stop 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 -- 172.21.15.148:0/3014518784 shutdown_connections 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40d1c40 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/3014518784 >> v2:172.21.15.188:3300/0 conn(0x7f79e40cc8e0 0x7f79e415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/3014518784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e415dc90 0x7f79e41625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 -- 172.21.15.148:0/3014518784 >> 172.21.15.148:0/3014518784 conn(0x7f79e406f590 msgr2=0x7f79e406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 -- 172.21.15.148:0/3014518784 shutdown_connections 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.247+0000 7f79ea8d2640 1 -- 172.21.15.148:0/3014518784 wait complete. 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.248+0000 7f79ea8d2640 1 Processor -- start 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.248+0000 7f79ea8d2640 1 -- start start 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79ea8d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e40cc8e0 0x7f79e40acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79ea8d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79ea8d2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79e415dc90 0x7f79e40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79ea8d2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40cd130 con 0x7f79e40d1810 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79ea8d2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e41620b0 con 0x7f79e415dc90 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79ea8d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415e070 con 0x7f79e40cc8e0 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42724/0 (socket says 172.21.15.148:42724) 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 -- 172.21.15.148:0/2760729432 learned_addr learned my addr 172.21.15.148:0/2760729432 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:17.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e3fff640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e40cc8e0 0x7f79e40acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e8e6f640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79e415dc90 0x7f79e40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 -- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e40cc8e0 msgr2=0x7f79e40acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e40cc8e0 0x7f79e40acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 -- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79e415dc90 msgr2=0x7f79e40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79e415dc90 0x7f79e40cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.249+0000 7f79e37fe640 1 -- 172.21.15.148:0/2760729432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40c98a0 con 0x7f79e40d1810 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.250+0000 7f79e37fe640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40c9360 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f79d800d830 tx=0x7f79d800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.250+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8018020 con 0x7f79e40d1810 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.250+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d80096a0 con 0x7f79e40d1810 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.250+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e40cbda0 con 0x7f79e40d1810 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.250+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8009a70 con 0x7f79e40d1810 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.250+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e414c710 con 0x7f79e40d1810 2024-04-25T10:06:17.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.251+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e4073990 con 0x7f79e40d1810 2024-04-25T10:06:17.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.251+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79d801b020 con 0x7f79e40d1810 2024-04-25T10:06:17.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.251+0000 7f79e0ff9640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79d006b4d0 0x7f79d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.252+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f79d808a140 con 0x7f79e40d1810 2024-04-25T10:06:17.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.252+0000 7f79e3fff640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79d006b4d0 0x7f79d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.252+0000 7f79e3fff640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79d006b4d0 0x7f79d006d990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006fd0 tx=0x7f79d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.256+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79d8022030 con 0x7f79e40d1810 2024-04-25T10:06:17.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.492+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f79e40cf6b0 con 0x7f79e40d1810 2024-04-25T10:06:17.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.542+0000 7f79e0ff9640 1 -- 172.21.15.148:0/2760729432 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v574) v1 ==== 150+0+0 (secure 0 0 0) 0x7f79d8058ae0 con 0x7f79e40d1810 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79d006b4d0 msgr2=0x7f79d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79d006b4d0 0x7f79d006d990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f79d4006fd0 tx=0x7f79d4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 msgr2=0x7f79e40c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40c9360 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f79d800d830 tx=0x7f79d800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 shutdown_connections 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f79d006b4d0 0x7f79d006d990 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79e415dc90 0x7f79e40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79e40d1810 0x7f79e40c9360 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 --2- 172.21.15.148:0/2760729432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79e40cc8e0 0x7f79e40acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 >> 172.21.15.148:0/2760729432 conn(0x7f79e406f590 msgr2=0x7f79e406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 shutdown_connections 2024-04-25T10:06:17.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.544+0000 7f79ea8d2640 1 -- 172.21.15.148:0/2760729432 wait complete. 2024-04-25T10:06:17.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:06:17.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.655+0000 7f0e3195c640 1 Processor -- start 2024-04-25T10:06:17.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.655+0000 7f0e3195c640 1 -- start start 2024-04-25T10:06:17.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e3195c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 0x7f0e2c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e3195c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e2c161710 0x7f0e2c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e3195c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c15e000 con 0x7f0e2c0b9330 2024-04-25T10:06:17.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e3195c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0d1de0 con 0x7f0e2c161710 2024-04-25T10:06:17.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e3195c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c161c60 con 0x7f0e2c15dc20 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e2b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f0e2c0b9330 0x7f0e2c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:55780/0 (socket says 172.21.15.148:55780) 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/944203105 learned_addr learned my addr 172.21.15.148:0/944203105 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e2affd640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 0x7f0e2c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e2a7fc640 1 --2- 172.21.15.148:0/944203105 >> v2:172.21.15.188:3300/0 conn(0x7f0e2c161710 0x7f0e2c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210229762 0 0) 0x7f0e2c15e000 con 0x7f0e2c0b9330 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e14003520 con 0x7f0e2c0b9330 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044447759 0 0) 0x7f0e14003520 con 0x7f0e2c0b9330 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 msgr2=0x7f0e2c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 0x7f0e2c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 >> v2:172.21.15.188:3300/0 conn(0x7f0e2c161710 msgr2=0x7f0e2c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 --2- 172.21.15.148:0/944203105 >> v2:172.21.15.188:3300/0 conn(0x7f0e2c161710 0x7f0e2c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c161f40 con 0x7f0e2c0b9330 2024-04-25T10:06:17.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.656+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f0e24001670 con 0x7f0e2c0b9330 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 >> v1:172.21.15.148:6789/0 conn(0x7f0e2c0b9330 legacy=0x7f0e2c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 0x7f0e140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e14003f30 0x7f0e14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0d1de0 con 0x7f0e14004d40 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c161c60 con 0x7f0e14003f30 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1400b9c0 con 0x7f0e2c15dc20 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2b7fe640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2affd640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 0x7f0e140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2a7fc640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e14003f30 0x7f0e14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 msgr2=0x7f0e140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2b7fe640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e2c15dc20 0x7f0e140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e14003f30 msgr2=0x7f0e14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2b7fe640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e14003f30 0x7f0e14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.657+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/944203105 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14003520 con 0x7f0e14004d40 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e2b7fe640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f0e24007fd0 tx=0x7f0e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c305c29c5242ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007d50 con 0x7f0e14004d40 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24005ad0 con 0x7f0e14004d40 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e29ffb640 1 -- 172.21.15.148:0/944203105 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007070 con 0x7f0e14004d40 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e3195c640 1 -- 172.21.15.148:0/944203105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14004d40 msgr2=0x7f0e140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e3195c640 1 --2- 172.21.15.148:0/944203105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e14004d40 0x7f0e140051c0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f0e24007fd0 tx=0x7f0e14005830 comp rx=0 tx=0).stop 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e2b7fe640 1 -- 172.21.15.148:0/944203105 reap_dead start 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e3195c640 1 -- 172.21.15.148:0/944203105 shutdown_connections 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e3195c640 1 -- 172.21.15.148:0/944203105 >> 172.21.15.148:0/944203105 conn(0x7f0e2c0d0420 msgr2=0x7f0e2c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e3195c640 1 -- 172.21.15.148:0/944203105 shutdown_connections 2024-04-25T10:06:17.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.658+0000 7f0e3195c640 1 -- 172.21.15.148:0/944203105 wait complete. 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.659+0000 7f0e3195c640 1 Processor -- start 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 -- start start 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e14004d40 0x7f0e2c1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 0x7f0e2c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e2c0cb840 0x7f0e2c0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0b5e50 con 0x7f0e2c0caa70 2024-04-25T10:06:17.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c162500 con 0x7f0e2c0cb840 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e3195c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0b5fd0 con 0x7f0e14004d40 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e2a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 0x7f0e2c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e2a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 0x7f0e2c0cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42736/0 (socket says 172.21.15.148:42736) 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.660+0000 7f0e2a7fc640 1 -- 172.21.15.148:0/1591946527 learned_addr learned my addr 172.21.15.148:0/1591946527 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2b7fe640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e2c0cb840 0x7f0e2c0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2a7fc640 1 -- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e14004d40 msgr2=0x7f0e2c1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2a7fc640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e14004d40 0x7f0e2c1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2a7fc640 1 -- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e2c0cb840 msgr2=0x7f0e2c0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2a7fc640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e2c0cb840 0x7f0e2c0cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2a7fc640 1 -- 172.21.15.148:0/1591946527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c1f3620 con 0x7f0e2c0caa70 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e2a7fc640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 0x7f0e2c0cda60 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f0e20011960 tx=0x7f0e20011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000faf0 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000fc90 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c0c80a0 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20014440 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.661+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c0c8660 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.662+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c073a00 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.663+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e20012040 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.663+0000 7f0e13fff640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e1806b4d0 0x7f0e1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.663+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0e2008a7c0 con 0x7f0e2c0caa70 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.663+0000 7f0e2affd640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e1806b4d0 0x7f0e1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.663+0000 7f0e2affd640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e1806b4d0 0x7f0e1806d990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009fc0 tx=0x7f0e1c009750 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.667+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e20019030 con 0x7f0e2c0caa70 2024-04-25T10:06:17.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.835+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0e2c0d16b0 con 0x7f0e2c0caa70 2024-04-25T10:06:17.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.840+0000 7f0e13fff640 1 -- 172.21.15.148:0/1591946527 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0e20059160 con 0x7f0e2c0caa70 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e1806b4d0 msgr2=0x7f0e1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e1806b4d0 0x7f0e1806d990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009fc0 tx=0x7f0e1c009750 comp rx=0 tx=0).stop 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 msgr2=0x7f0e2c0cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 0x7f0e2c0cda60 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f0e20011960 tx=0x7f0e20011e20 comp rx=0 tx=0).stop 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 shutdown_connections 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f0e1806b4d0 0x7f0e1806d990 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e2c0cb840 0x7f0e2c0cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e2c0caa70 0x7f0e2c0cda60 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 --2- 172.21.15.148:0/1591946527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e14004d40 0x7f0e2c1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 >> 172.21.15.148:0/1591946527 conn(0x7f0e2c0d0420 msgr2=0x7f0e2c0b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 shutdown_connections 2024-04-25T10:06:17.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:17.842+0000 7f0e3195c640 1 -- 172.21.15.148:0/1591946527 wait complete. 2024-04-25T10:06:17.859 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:17.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:17.902 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:17.964 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:17.964 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:17.965 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:17.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:17.980 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:17.980 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:17.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:17.999 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:17.999 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:18.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:18.035 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:18.035 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:18.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:18.056 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:18.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:18.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.166+0000 7f632fdc8640 1 Processor -- start 2024-04-25T10:06:18.172 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632fdc8640 1 -- start start 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632fdc8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f63280b2cb0 0x7f63280d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632fdc8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63280d1020 0x7f63280d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632fdc8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63280b97a0 con 0x7f632815dbe0 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632fdc8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632815dfc0 con 0x7f63280d1020 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632fdc8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280d1a00 con 0x7f63280b2cb0 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.167+0000 7f632e365640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63280d1020 0x7f63280d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.174 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f63280d1020 0x7f63280d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42750/0 (socket says 172.21.15.148:42750) 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 -- 172.21.15.148:0/94118996 learned_addr learned my addr 172.21.15.148:0/94118996 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632d363640 1 --2- 172.21.15.148:0/94118996 >> v2:172.21.15.188:3300/0 conn(0x7f63280b2cb0 0x7f63280d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 -- 172.21.15.148:0/94118996 >> v2:172.21.15.188:3300/0 conn(0x7f63280b2cb0 msgr2=0x7f63280d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 --2- 172.21.15.148:0/94118996 >> v2:172.21.15.188:3300/0 conn(0x7f63280b2cb0 0x7f63280d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 -- 172.21.15.148:0/94118996 >> v1:172.21.15.188:6789/0 conn(0x7f632815dbe0 legacy=0x7f63280d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 -- 172.21.15.148:0/94118996 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280d1c50 con 0x7f63280d1020 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632d363640 1 --2- 172.21.15.148:0/94118996 >> v2:172.21.15.188:3300/0 conn(0x7f63280b2cb0 0x7f63280d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.168+0000 7f632e365640 1 --2- 172.21.15.148:0/94118996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280d1020 0x7f63280d2c70 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f632806cff0 tx=0x7f632002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b795d804bee88f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632cb62640 1 -- 172.21.15.148:0/94118996 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320037070 con 0x7f63280d1020 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632cb62640 1 -- 172.21.15.148:0/94118996 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320032440 con 0x7f63280d1020 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632cb62640 1 -- 172.21.15.148:0/94118996 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320032810 con 0x7f63280d1020 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 -- 172.21.15.148:0/94118996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280d1020 msgr2=0x7f63280d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 --2- 172.21.15.148:0/94118996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280d1020 0x7f63280d2c70 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f632806cff0 tx=0x7f632002a230 comp rx=0 tx=0).stop 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 -- 172.21.15.148:0/94118996 shutdown_connections 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 --2- 172.21.15.148:0/94118996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280d1020 0x7f63280d2c70 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 --2- 172.21.15.148:0/94118996 >> v2:172.21.15.188:3300/0 conn(0x7f63280b2cb0 0x7f63280d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 -- 172.21.15.148:0/94118996 >> 172.21.15.148:0/94118996 conn(0x7f6328061a20 msgr2=0x7f6328061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 -- 172.21.15.148:0/94118996 shutdown_connections 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.169+0000 7f632fdc8640 1 -- 172.21.15.148:0/94118996 wait complete. 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.170+0000 7f632fdc8640 1 Processor -- start 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.170+0000 7f632fdc8640 1 -- start start 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632fdc8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 0x7f6328160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632fdc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280d1020 0x7f63281f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632fdc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632815dbe0 0x7f63281f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632fdc8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280b92a0 con 0x7f63280b2cb0 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632fdc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280d2210 con 0x7f632815dbe0 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632fdc8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280b9420 con 0x7f63280d1020 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632db64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 0x7f6328160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632db64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 0x7f6328160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42762/0 (socket says 172.21.15.148:42762) 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.171+0000 7f632db64640 1 -- 172.21.15.148:0/1486105649 learned_addr learned my addr 172.21.15.148:0/1486105649 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632db64640 1 -- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280d1020 msgr2=0x7f63281f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632db64640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280d1020 0x7f63281f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632db64640 1 -- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632815dbe0 msgr2=0x7f63281f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632e365640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632815dbe0 0x7f63281f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632db64640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632815dbe0 0x7f63281f49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632db64640 1 -- 172.21.15.148:0/1486105649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280b43a0 con 0x7f63280b2cb0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632db64640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 0x7f6328160bd0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f632400ba50 tx=0x7f632400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324019070 con 0x7f63280b2cb0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632400cc20 con 0x7f63280b2cb0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632815e280 con 0x7f63280b2cb0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324014680 con 0x7f63280b2cb0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.172+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632815e840 con 0x7f63280b2cb0 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.173+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6328073990 con 0x7f63280b2cb0 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.173+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f632401c070 con 0x7f63280b2cb0 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.174+0000 7f631e7fc640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f630c06b4d0 0x7f630c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.174+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6324089cf0 con 0x7f63280b2cb0 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.174+0000 7f632d363640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f630c06b4d0 0x7f630c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.174+0000 7f632d363640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f630c06b4d0 0x7f630c06d990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6318006fd0 tx=0x7f6318008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.178+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6324058700 con 0x7f63280b2cb0 2024-04-25T10:06:18.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.384+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f632815ea10 con 0x7f63280b2cb0 2024-04-25T10:06:18.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.384+0000 7f631e7fc640 1 -- 172.21.15.148:0/1486105649 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+3711 (secure 0 0 0) 0x7f632405c7a0 con 0x7f63280b2cb0 2024-04-25T10:06:18.390 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:18.390 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":101,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4115242822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4115242822},{"type":"v1","addr":"172.21.15.188:6834","nonce":4115242822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":7754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/339626102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":339626102},{"type":"v1","addr":"172.21.15.188:6835","nonce":339626102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3878650074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3878650074},{"type":"v1","addr":"172.21.15.148:6836","nonce":3878650074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":10833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3746563003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3746563003},{"type":"v1","addr":"172.21.15.148:6837","nonce":3746563003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[]} 2024-04-25T10:06:18.390 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 101 2024-04-25T10:06:18.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f630c06b4d0 msgr2=0x7f630c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f630c06b4d0 0x7f630c06d990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6318006fd0 tx=0x7f6318008040 comp rx=0 tx=0).stop 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 msgr2=0x7f6328160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 0x7f6328160bd0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f632400ba50 tx=0x7f632400bf10 comp rx=0 tx=0).stop 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 shutdown_connections 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f630c06b4d0 0x7f630c06d990 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f632815dbe0 0x7f63281f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63280d1020 0x7f63281f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 --2- 172.21.15.148:0/1486105649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63280b2cb0 0x7f6328160bd0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 >> 172.21.15.148:0/1486105649 conn(0x7f6328061a20 msgr2=0x7f63281594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 shutdown_connections 2024-04-25T10:06:18.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:18.386+0000 7f632fdc8640 1 -- 172.21.15.148:0/1486105649 wait complete. 2024-04-25T10:06:18.403 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:18.403 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:18.403 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:18.403 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:18.403 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:18.403 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:18.404 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:18.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:18.404 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:18.404 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:18.404 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:18.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:18.404 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:18.405 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:18.405 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:18.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:18.413 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:18.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:18.416 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:18.416 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:18.417 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:18.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:18.457 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:18.457 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:18.459 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:18.459 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:18.460 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:18.460 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:18.528 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:06:18.537 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:06:18.538 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:06:24.414 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:24.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:24.417 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:24.418 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:24.418 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:24.501 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:06:24.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.523+0000 7f87312fc640 1 Processor -- start 2024-04-25T10:06:24.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.523+0000 7f87312fc640 1 -- start start 2024-04-25T10:06:24.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f87312fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c161710 0x7f872c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f87312fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c15e000 con 0x7f872c0ced80 2024-04-25T10:06:24.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f87312fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c0d1de0 con 0x7f872c15dc20 2024-04-25T10:06:24.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f87312fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c161c60 con 0x7f872c161710 2024-04-25T10:06:24.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f8723fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f872c0ced80 0x7f872c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45558/0 (socket says 172.21.15.148:45558) 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f8723fff640 1 -- 172.21.15.148:0/28826439 learned_addr learned my addr 172.21.15.148:0/28826439 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f872b7fe640 1 --2- 172.21.15.148:0/28826439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c161710 0x7f872c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579350202 0 0) 0x7f872c15e000 con 0x7f872c0ced80 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8718003520 con 0x7f872c0ced80 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474161066 0 0) 0x7f872c0d1de0 con 0x7f872c15dc20 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.524+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f872c15e000 con 0x7f872c15dc20 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.525+0000 7f872b7fe640 1 -- 172.21.15.148:0/28826439 >> v1:172.21.15.188:6789/0 conn(0x7f872c15dc20 legacy=0x7f872c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.525+0000 7f872b7fe640 1 -- 172.21.15.148:0/28826439 >> v1:172.21.15.148:6789/0 conn(0x7f872c0ced80 legacy=0x7f872c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.525+0000 7f872b7fe640 1 -- 172.21.15.148:0/28826439 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c161f40 con 0x7f872c161710 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.525+0000 7f872b7fe640 1 --2- 172.21.15.148:0/28826439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c161710 0x7f872c0d29f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8724009dc0 tx=0x7f872402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a740653a5058e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.525+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724007740 con 0x7f872c161710 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87240078e0 con 0x7f872c161710 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f872a7fc640 1 -- 172.21.15.148:0/28826439 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724007c90 con 0x7f872c161710 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 -- 172.21.15.148:0/28826439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c161710 msgr2=0x7f872c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 --2- 172.21.15.148:0/28826439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c161710 0x7f872c0d29f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8724009dc0 tx=0x7f872402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 -- 172.21.15.148:0/28826439 shutdown_connections 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 --2- 172.21.15.148:0/28826439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c161710 0x7f872c0d29f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 -- 172.21.15.148:0/28826439 >> 172.21.15.148:0/28826439 conn(0x7f872c0d0420 msgr2=0x7f872c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 -- 172.21.15.148:0/28826439 shutdown_connections 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.526+0000 7f87312fc640 1 -- 172.21.15.148:0/28826439 wait complete. 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.527+0000 7f87312fc640 1 Processor -- start 2024-04-25T10:06:24.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.527+0000 7f87312fc640 1 -- start start 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f87312fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f872c0ced80 0x7f872c0b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f87312fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c15dc20 0x7f872c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f87312fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 0x7f872c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f87312fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0b5e50 con 0x7f872c161710 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f87312fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c162500 con 0x7f872c0ced80 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f87312fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0b5fd0 con 0x7f872c15dc20 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 0x7f872c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f872b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 0x7f872c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40548/0 (socket says 172.21.15.148:40548) 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f872b7fe640 1 -- 172.21.15.148:0/1992612660 learned_addr learned my addr 172.21.15.148:0/1992612660 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f8723fff640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c15dc20 0x7f872c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.536 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.528+0000 7f872affd640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f872c0ced80 0x7f872c0b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f872b7fe640 1 -- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c15dc20 msgr2=0x7f872c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f872b7fe640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c15dc20 0x7f872c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f872b7fe640 1 -- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f872c0ced80 msgr2=0x7f872c0b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f872b7fe640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f872c0ced80 0x7f872c0b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f872b7fe640 1 -- 172.21.15.148:0/1992612660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c15ec90 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f872b7fe640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 0x7f872c15e750 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f8724009eb0 tx=0x7f8724004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724040070 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8724031c70 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c15ef40 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724009450 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.529+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c0c7fa0 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.530+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872c0b4ff0 con 0x7f872c161710 2024-04-25T10:06:24.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.530+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8724039040 con 0x7f872c161710 2024-04-25T10:06:24.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.531+0000 7f87237fe640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.531+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f87240ab650 con 0x7f872c161710 2024-04-25T10:06:24.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.531+0000 7f872affd640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.531+0000 7f872affd640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f871c007fd0 tx=0x7f871c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.535+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f872c0b4ff0 con 0x7f872c161710 2024-04-25T10:06:24.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:24.716+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f872c073a00 con 0x7f872c161710 2024-04-25T10:06:25.810 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:25.813 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:25.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.853+0000 7f87237fe640 1 -- 172.21.15.148:0/1992612660 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8724031e20 con 0x7f872c161710 2024-04-25T10:06:25.859 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 msgr2=0x7f870c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f871c007fd0 tx=0x7f871c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 msgr2=0x7f872c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 0x7f872c15e750 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f8724009eb0 tx=0x7f8724004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 shutdown_connections 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f872c161710 0x7f872c15e750 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f872c15dc20 0x7f872c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 --2- 172.21.15.148:0/1992612660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f872c0ced80 0x7f872c0b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 >> 172.21.15.148:0/1992612660 conn(0x7f872c0d0420 msgr2=0x7f872c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.863 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 shutdown_connections 2024-04-25T10:06:25.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:25.857+0000 7f87312fc640 1 -- 172.21.15.148:0/1992612660 wait complete. 2024-04-25T10:06:25.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:26.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.017+0000 7f44f042a640 1 Processor -- start 2024-04-25T10:06:26.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.017+0000 7f44f042a640 1 -- start start 2024-04-25T10:06:26.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44f042a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44e8154ee0 0x7f44e80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44f042a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e8071480 con 0x7f44e80ce590 2024-04-25T10:06:26.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44f042a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e8071760 con 0x7f44e80cfcf0 2024-04-25T10:06:26.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44f042a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e8072090 con 0x7f44e8154ee0 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44ee9c7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f44e80ce590 0x7f44e806d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45562/0 (socket says 172.21.15.148:45562) 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44ee9c7640 1 -- 172.21.15.148:0/2470256842 learned_addr learned my addr 172.21.15.148:0/2470256842 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384832007 0 0) 0x7f44e8071480 con 0x7f44e80ce590 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d4003520 con 0x7f44e80ce590 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.018+0000 7f44ee1c6640 1 --2- 172.21.15.148:0/2470256842 >> v2:172.21.15.188:3300/0 conn(0x7f44e8154ee0 0x7f44e80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2682961851 0 0) 0x7f44d4003520 con 0x7f44e80ce590 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 >> v2:172.21.15.188:3300/0 conn(0x7f44e8154ee0 msgr2=0x7f44e80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 --2- 172.21.15.148:0/2470256842 >> v2:172.21.15.188:3300/0 conn(0x7f44e8154ee0 0x7f44e80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 >> v1:172.21.15.188:6789/0 conn(0x7f44e80cfcf0 legacy=0x7f44e80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e8071a40 con 0x7f44e80ce590 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f44e00019f0 con 0x7f44e80ce590 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 >> v1:172.21.15.148:6789/0 conn(0x7f44e80ce590 legacy=0x7f44e806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d40039b0 0x7f44d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44d4004800 0x7f44d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d40051c0 0x7f44d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d400c0e0 con 0x7f44d4004800 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d400c290 con 0x7f44d40039b0 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d400e6a0 con 0x7f44d40051c0 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44d4004800 0x7f44d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ee1c6640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d40039b0 0x7f44d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ee9c7640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d40051c0 0x7f44d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed9c5640 1 -- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d40051c0 msgr2=0x7f44d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d40051c0 0x7f44d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed9c5640 1 -- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d40039b0 msgr2=0x7f44d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d40039b0 0x7f44d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ed9c5640 1 -- 172.21.15.148:0/2470256842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4003520 con 0x7f44d4004800 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ee9c7640 1 -- 172.21.15.148:0/2470256842 reap_dead start 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.019+0000 7f44ee1c6640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44d40039b0 0x7f44d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.020+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44d4004800 0x7f44d4004c80 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f44d8001750 tx=0x7f44d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2121f5fff564b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.020+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8001370 con 0x7f44d4004800 2024-04-25T10:06:26.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.020+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d80052d0 con 0x7f44d4004800 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.020+0000 7f44ed1c4640 1 -- 172.21.15.148:0/2470256842 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d802abc0 con 0x7f44d4004800 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.020+0000 7f44f042a640 1 -- 172.21.15.148:0/2470256842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44d4004800 msgr2=0x7f44d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.020+0000 7f44f042a640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44d4004800 0x7f44d4004c80 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f44d8001750 tx=0x7f44d4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.021+0000 7f44f042a640 1 -- 172.21.15.148:0/2470256842 shutdown_connections 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.021+0000 7f44f042a640 1 --2- 172.21.15.148:0/2470256842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44d4004800 0x7f44d4004c80 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.021+0000 7f44f042a640 1 -- 172.21.15.148:0/2470256842 >> 172.21.15.148:0/2470256842 conn(0x7f44e806abe0 msgr2=0x7f44e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.021+0000 7f44f042a640 1 -- 172.21.15.148:0/2470256842 shutdown_connections 2024-04-25T10:06:26.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.021+0000 7f44f042a640 1 -- 172.21.15.148:0/2470256842 wait complete. 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.022+0000 7f44f042a640 1 Processor -- start 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.022+0000 7f44f042a640 1 -- start start 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44f042a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d4004800 0x7f44e8153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44f042a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 0x7f44e81f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44f042a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e81f33b0 0x7f44e80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44f042a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81552c0 con 0x7f44e81f0630 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44f042a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8071230 con 0x7f44e81f33b0 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44f042a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8155440 con 0x7f44d4004800 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44ee1c6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d4004800 0x7f44e8153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44ed9c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 0x7f44e81f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44ed9c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 0x7f44e81f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40564/0 (socket says 172.21.15.148:40564) 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44ed9c5640 1 -- 172.21.15.148:0/1438417745 learned_addr learned my addr 172.21.15.148:0/1438417745 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44ee1c6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d4004800 0x7f44e8153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55132/0 (socket says 172.21.15.148:55132) 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.023+0000 7f44ee9c7640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e81f33b0 0x7f44e80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44ed9c5640 1 -- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d4004800 msgr2=0x7f44e8153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d4004800 0x7f44e8153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44ed9c5640 1 -- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e81f33b0 msgr2=0x7f44e80ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e81f33b0 0x7f44e80ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44ed9c5640 1 -- 172.21.15.148:0/1438417745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e81f3830 con 0x7f44e81f0630 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44ed9c5640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 0x7f44e81f2e70 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005040 tx=0x7f44d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8037040 con 0x7f44e81f0630 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d8040070 con 0x7f44e81f0630 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81f3b10 con 0x7f44e81f0630 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8007a30 con 0x7f44e81f0630 2024-04-25T10:06:26.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.024+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e8156a70 con 0x7f44e81f0630 2024-04-25T10:06:26.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.025+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e8073990 con 0x7f44e81f0630 2024-04-25T10:06:26.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.025+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44d80052d0 con 0x7f44e81f0630 2024-04-25T10:06:26.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.026+0000 7f44deffd640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.026+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f44d80a9be0 con 0x7f44e81f0630 2024-04-25T10:06:26.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.026+0000 7f44ee1c6640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.026+0000 7f44ee1c6640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44cc06b4d0 0x7f44cc06d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f44e80692f0 tx=0x7f44e4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.030+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44d80785d0 con 0x7f44e81f0630 2024-04-25T10:06:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.268+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f44e806d360 con 0x7f44e81f0630 2024-04-25T10:06:26.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.269+0000 7f44deffd640 1 -- 172.21.15.148:0/1438417745 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f44d807c670 con 0x7f44e81f0630 2024-04-25T10:06:26.274 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:26.274 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.275 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:26.276 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:06:26.277 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.278 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.279 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:06:26.280 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.281 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.282 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.283 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:06:26.284 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.285 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:06:26.286 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:26.287 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:06:26.287 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44cc06b4d0 msgr2=0x7f44cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44cc06b4d0 0x7f44cc06d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f44e80692f0 tx=0x7f44e4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:26.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 msgr2=0x7f44e81f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 0x7f44e81f2e70 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005040 tx=0x7f44d8001120 comp rx=0 tx=0).stop 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 shutdown_connections 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e81f33b0 0x7f44e80ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e81f0630 0x7f44e81f2e70 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 --2- 172.21.15.148:0/1438417745 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44d4004800 0x7f44e8153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 >> 172.21.15.148:0/1438417745 conn(0x7f44e806abe0 msgr2=0x7f44e80cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 shutdown_connections 2024-04-25T10:06:26.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.271+0000 7f44f042a640 1 -- 172.21.15.148:0/1438417745 wait complete. 2024-04-25T10:06:26.290 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.380+0000 7fe21c09a640 1 Processor -- start 2024-04-25T10:06:26.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.381+0000 7fe21c09a640 1 -- start start 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.381+0000 7fe21c09a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 0x7fe2140d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.381+0000 7fe21c09a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2140b6220 con 0x7fe2140b2cb0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.381+0000 7fe21c09a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2140b97a0 con 0x7fe2140b92a0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.381+0000 7fe21c09a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415dfc0 con 0x7fe21415dbe0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.381+0000 7fe219635640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe2140b2cb0 0x7fe2140b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45568/0 (socket says 172.21.15.148:45568) 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe219635640 1 -- 172.21.15.148:0/633970587 learned_addr learned my addr 172.21.15.148:0/633970587 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe219e36640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 0x7fe2140d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493517112 0 0) 0x7fe2140b6220 con 0x7fe2140b2cb0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1fc003520 con 0x7fe2140b2cb0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2824797027 0 0) 0x7fe1fc003520 con 0x7fe2140b2cb0 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 msgr2=0x7fe2140d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 0x7fe2140d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 >> v1:172.21.15.188:6789/0 conn(0x7fe2140b92a0 legacy=0x7fe2140b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140d1c50 con 0x7fe2140b2cb0 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fe208001670 con 0x7fe2140b2cb0 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 >> v1:172.21.15.148:6789/0 conn(0x7fe2140b2cb0 legacy=0x7fe2140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 0x7fe1fc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1fc003f60 0x7fe1fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1fc004d40 0x7fe1fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00ba70 con 0x7fe1fc004d40 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00bc20 con 0x7fe1fc003f60 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00be20 con 0x7fe21415dbe0 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe21a637640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1fc004d40 0x7fe1fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe219635640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1fc003f60 0x7fe1fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe219e36640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 0x7fe1fc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe21a637640 1 -- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 msgr2=0x7fe1fc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe21a637640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415dbe0 0x7fe1fc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe21a637640 1 -- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1fc003f60 msgr2=0x7fe1fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe21a637640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1fc003f60 0x7fe1fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.382+0000 7fe21a637640 1 -- 172.21.15.148:0/633970587 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc003520 con 0x7fe1fc004d40 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.383+0000 7fe21a637640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1fc004d40 0x7fe1fc0051c0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fe2140b25c0 tx=0x7fe1fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c286dd483e3b26d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.383+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204001370 con 0x7fe1fc004d40 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.383+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2040052d0 con 0x7fe1fc004d40 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.383+0000 7fe218e34640 1 -- 172.21.15.148:0/633970587 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20402ab90 con 0x7fe1fc004d40 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21c09a640 1 -- 172.21.15.148:0/633970587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1fc004d40 msgr2=0x7fe1fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21c09a640 1 --2- 172.21.15.148:0/633970587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1fc004d40 0x7fe1fc0051c0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fe2140b25c0 tx=0x7fe1fc005830 comp rx=0 tx=0).stop 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21a637640 1 -- 172.21.15.148:0/633970587 reap_dead start 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21c09a640 1 -- 172.21.15.148:0/633970587 shutdown_connections 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21c09a640 1 -- 172.21.15.148:0/633970587 >> 172.21.15.148:0/633970587 conn(0x7fe214061a20 msgr2=0x7fe214061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21c09a640 1 -- 172.21.15.148:0/633970587 shutdown_connections 2024-04-25T10:06:26.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.384+0000 7fe21c09a640 1 -- 172.21.15.148:0/633970587 wait complete. 2024-04-25T10:06:26.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.385+0000 7fe21c09a640 1 Processor -- start 2024-04-25T10:06:26.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.385+0000 7fe21c09a640 1 -- start start 2024-04-25T10:06:26.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2141f3610 0x7fe2141538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 0x7fe21415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415e500 0x7fe21415f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140d2210 con 0x7fe2141f39f0 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140d1a00 con 0x7fe2141f3610 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140b6ab0 con 0x7fe21415e500 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 0x7fe21415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 0x7fe21415dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40584/0 (socket says 172.21.15.148:40584) 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 -- 172.21.15.148:0/303007156 learned_addr learned my addr 172.21.15.148:0/303007156 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21a637640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415e500 0x7fe21415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219e36640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2141f3610 0x7fe2141538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 -- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415e500 msgr2=0x7fe21415f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415e500 0x7fe21415f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 -- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2141f3610 msgr2=0x7fe2141538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2141f3610 0x7fe2141538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 -- 172.21.15.148:0/303007156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe214155500 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe219635640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 0x7fe21415dfc0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fe208004840 tx=0x7fe208007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800f8e0 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208018070 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe214150680 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800fca0 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.386+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe214150c40 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.387+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214073990 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.388+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe20801d460 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.388+0000 7fe2127fc640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe20006b500 0x7fe20006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.388+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe20808a180 con 0x7fe2141f39f0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.388+0000 7fe219e36640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe20006b500 0x7fe20006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.389+0000 7fe219e36640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe20006b500 0x7fe20006d9c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe20c007520 tx=0x7fe20c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.392+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe214155500 con 0x7fe2141f39f0 2024-04-25T10:06:26.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.564+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe214150e10 con 0x7fe2141f39f0 2024-04-25T10:06:26.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.565+0000 7fe2127fc640 1 -- 172.21.15.148:0/303007156 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+13222 (secure 0 0 0) 0x7fe2080591c0 con 0x7fe2141f39f0 2024-04-25T10:06:26.570 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:26.570 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":181,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:25.853095+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:26.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.566+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe20006b500 msgr2=0x7fe20006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.566+0000 7fe21c09a640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe20006b500 0x7fe20006d9c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe20c007520 tx=0x7fe20c00b040 comp rx=0 tx=0).stop 2024-04-25T10:06:26.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.566+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 msgr2=0x7fe21415dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.566+0000 7fe21c09a640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 0x7fe21415dfc0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fe208004840 tx=0x7fe208007720 comp rx=0 tx=0).stop 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 shutdown_connections 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe20006b500 0x7fe20006d9c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe21415e500 0x7fe21415f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2141f39f0 0x7fe21415dfc0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 --2- 172.21.15.148:0/303007156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2141f3610 0x7fe2141538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 >> 172.21.15.148:0/303007156 conn(0x7fe214061a20 msgr2=0x7fe2140b87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 shutdown_connections 2024-04-25T10:06:26.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.567+0000 7fe21c09a640 1 -- 172.21.15.148:0/303007156 wait complete. 2024-04-25T10:06:26.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:26.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:26.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.679+0000 7f3aadc4a640 1 Processor -- start 2024-04-25T10:06:26.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.679+0000 7f3aadc4a640 1 -- start start 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aadc4a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3aa806c330 0x7f3aa80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aadc4a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80d1d50 con 0x7f3aa8158230 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aadc4a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80d2020 con 0x7f3aa806c7e0 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aadc4a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa8158610 con 0x7f3aa806c330 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aa77fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3aa8158230 0x7f3aa80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45572/0 (socket says 172.21.15.148:45572) 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aa77fe640 1 -- 172.21.15.148:0/4159052482 learned_addr learned my addr 172.21.15.148:0/4159052482 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.680+0000 7f3aa6ffd640 1 --2- 172.21.15.148:0/4159052482 >> v2:172.21.15.188:3300/0 conn(0x7f3aa806c330 0x7f3aa80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.681+0000 7f3aa67fc640 1 -- 172.21.15.148:0/4159052482 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241574966 0 0) 0x7f3aa80d1d50 con 0x7f3aa8158230 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.681+0000 7f3aa67fc640 1 -- 172.21.15.148:0/4159052482 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a90003520 con 0x7f3aa8158230 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.681+0000 7f3aa6ffd640 1 -- 172.21.15.148:0/4159052482 >> v1:172.21.15.188:6789/0 conn(0x7f3aa806c7e0 legacy=0x7f3aa806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.681+0000 7f3aa6ffd640 1 -- 172.21.15.148:0/4159052482 >> v1:172.21.15.148:6789/0 conn(0x7f3aa8158230 legacy=0x7f3aa80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.681+0000 7f3aa6ffd640 1 -- 172.21.15.148:0/4159052482 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80d2300 con 0x7f3aa806c330 2024-04-25T10:06:26.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.682+0000 7f3aa6ffd640 1 --2- 172.21.15.148:0/4159052482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c330 0x7f3aa80b4e60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009790 tx=0x7f3a9c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11dce1fd317e1e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.682+0000 7f3aa67fc640 1 -- 172.21.15.148:0/4159052482 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c037070 con 0x7f3aa806c330 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.682+0000 7f3aa67fc640 1 -- 172.21.15.148:0/4159052482 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c032440 con 0x7f3aa806c330 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.682+0000 7f3aa67fc640 1 -- 172.21.15.148:0/4159052482 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0327f0 con 0x7f3aa806c330 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 -- 172.21.15.148:0/4159052482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c330 msgr2=0x7f3aa80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/4159052482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c330 0x7f3aa80b4e60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009790 tx=0x7f3a9c02a210 comp rx=0 tx=0).stop 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 -- 172.21.15.148:0/4159052482 shutdown_connections 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/4159052482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c330 0x7f3aa80b4e60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 -- 172.21.15.148:0/4159052482 >> 172.21.15.148:0/4159052482 conn(0x7f3aa8061600 msgr2=0x7f3aa80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 -- 172.21.15.148:0/4159052482 shutdown_connections 2024-04-25T10:06:26.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.683+0000 7f3aadc4a640 1 -- 172.21.15.148:0/4159052482 wait complete. 2024-04-25T10:06:26.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.684+0000 7f3aadc4a640 1 Processor -- start 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.684+0000 7f3aadc4a640 1 -- start start 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 0x7f3aa81528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c7e0 0x7f3aa8145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aa8158230 0x7f3aa8142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80d25e0 con 0x7f3aa806c330 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa806bea0 con 0x7f3aa806c7e0 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa815e320 con 0x7f3aa8158230 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 0x7f3aa81528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 0x7f3aa81528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40598/0 (socket says 172.21.15.148:40598) 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 -- 172.21.15.148:0/1284146861 learned_addr learned my addr 172.21.15.148:0/1284146861 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa6ffd640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c7e0 0x7f3aa8145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa7fff640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aa8158230 0x7f3aa8142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 -- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aa8158230 msgr2=0x7f3aa8142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aa8158230 0x7f3aa8142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 -- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c7e0 msgr2=0x7f3aa8145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c7e0 0x7f3aa8145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 -- 172.21.15.148:0/1284146861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa8146490 con 0x7f3aa806c330 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aa77fe640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 0x7f3aa81528f0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c500 tx=0x7f3a9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98017070 con 0x7f3aa806c330 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98005ce0 con 0x7f3aa806c330 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa8152f90 con 0x7f3aa806c330 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98007330 con 0x7f3aa806c330 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.685+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa8153550 con 0x7f3aa806c330 2024-04-25T10:06:26.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.686+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa8073a00 con 0x7f3aa806c330 2024-04-25T10:06:26.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.687+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a9801a070 con 0x7f3aa806c330 2024-04-25T10:06:26.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.687+0000 7f3a8ffff640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a9406b4d0 0x7f3a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.687+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3a980893d0 con 0x7f3aa806c330 2024-04-25T10:06:26.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.687+0000 7f3aa6ffd640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a9406b4d0 0x7f3a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.688+0000 7f3aa6ffd640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a9406b4d0 0x7f3a9406d990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c034000 tx=0x7f3a9c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.691+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a98057f10 con 0x7f3aa806c330 2024-04-25T10:06:26.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.861+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3aa815c8e0 con 0x7f3aa806c330 2024-04-25T10:06:26.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.861+0000 7f3a8ffff640 1 -- 172.21.15.148:0/1284146861 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a9805bfb0 con 0x7f3aa806c330 2024-04-25T10:06:26.867 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:26.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a9406b4d0 msgr2=0x7f3a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a9406b4d0 0x7f3a9406d990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c034000 tx=0x7f3a9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 msgr2=0x7f3aa81528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 0x7f3aa81528f0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c500 tx=0x7f3a9800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 shutdown_connections 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3a9406b4d0 0x7f3a9406d990 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3aa8158230 0x7f3aa8142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3aa806c7e0 0x7f3aa8145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 --2- 172.21.15.148:0/1284146861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aa806c330 0x7f3aa81528f0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 >> 172.21.15.148:0/1284146861 conn(0x7f3aa8061600 msgr2=0x7f3aa815d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 shutdown_connections 2024-04-25T10:06:26.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.864+0000 7f3aadc4a640 1 -- 172.21.15.148:0/1284146861 wait complete. 2024-04-25T10:06:26.881 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:26.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.972+0000 7f36cf128640 1 Processor -- start 2024-04-25T10:06:26.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.973+0000 7f36cf128640 1 -- start start 2024-04-25T10:06:26.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.973+0000 7f36cf128640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36c815dd80 0x7f36c815e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.973+0000 7f36cf128640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 0x7f36c806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.973+0000 7f36cf128640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c80cfaf0 con 0x7f36c806c920 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.973+0000 7f36cf128640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c815e6a0 con 0x7f36c815dd80 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.973+0000 7f36cf128640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c806cd70 con 0x7f36c8161800 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 0x7f36c806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36ccec4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36c815dd80 0x7f36c815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36cd6c5640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f36c806c920 0x7f36c8099a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60800/0 (socket says 172.21.15.148:60800) 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 0x7f36c806c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55180/0 (socket says 172.21.15.148:55180) 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36cd6c5640 1 -- 172.21.15.148:0/3830746789 learned_addr learned my addr 172.21.15.148:0/3830746789 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36ccec4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36c815dd80 0x7f36c815e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:55050/0 (socket says 172.21.15.148:55050) 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36ccec4640 1 -- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 msgr2=0x7f36c806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36ccec4640 1 --2- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 0x7f36c806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36ccec4640 1 -- 172.21.15.148:0/3830746789 >> v1:172.21.15.188:6789/0 conn(0x7f36c806c920 legacy=0x7f36c8099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36ccec4640 1 -- 172.21.15.148:0/3830746789 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c815e8e0 con 0x7f36c815dd80 2024-04-25T10:06:26.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.974+0000 7f36c7fff640 1 --2- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 0x7f36c806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36ccec4640 1 --2- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c815dd80 0x7f36c815e160 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f36b80097a0 tx=0x7f36b802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0cf74eb64cf6331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36c77fe640 1 -- 172.21.15.148:0/3830746789 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8037070 con 0x7f36c815dd80 2024-04-25T10:06:26.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36c77fe640 1 -- 172.21.15.148:0/3830746789 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b802ad10 con 0x7f36c815dd80 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36c77fe640 1 -- 172.21.15.148:0/3830746789 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8032660 con 0x7f36c815dd80 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 -- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c815dd80 msgr2=0x7f36c815e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 --2- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c815dd80 0x7f36c815e160 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f36b80097a0 tx=0x7f36b802a080 comp rx=0 tx=0).stop 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 -- 172.21.15.148:0/3830746789 shutdown_connections 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 --2- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c8161800 0x7f36c806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 --2- 172.21.15.148:0/3830746789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c815dd80 0x7f36c815e160 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 -- 172.21.15.148:0/3830746789 >> 172.21.15.148:0/3830746789 conn(0x7f36c80b3310 msgr2=0x7f36c80b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 -- 172.21.15.148:0/3830746789 shutdown_connections 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.975+0000 7f36cf128640 1 -- 172.21.15.148:0/3830746789 wait complete. 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 Processor -- start 2024-04-25T10:06:26.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 -- start start 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 0x7f36c81f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c815dd80 0x7f36c80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c8161800 0x7f36c80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c815f180 con 0x7f36c806c920 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c815eea0 con 0x7f36c8161800 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.977+0000 7f36cf128640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c806c0c0 con 0x7f36c815dd80 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 0x7f36c81f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 0x7f36c81f3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40612/0 (socket says 172.21.15.148:40612) 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 -- 172.21.15.148:0/1562321581 learned_addr learned my addr 172.21.15.148:0/1562321581 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36c7fff640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c815dd80 0x7f36c80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36cd6c5640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c8161800 0x7f36c80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 -- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c815dd80 msgr2=0x7f36c80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c815dd80 0x7f36c80b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 -- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c8161800 msgr2=0x7f36c80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c8161800 0x7f36c80b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 -- 172.21.15.148:0/1562321581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c81602b0 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36ccec4640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 0x7f36c81f3930 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f36b8035040 tx=0x7f36b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8042070 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b8004030 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8037040 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c8160590 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.978+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c8160b50 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.979+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c8073990 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.980+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36b8045070 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.980+0000 7f36c57fa640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f36b406b4d0 0x7f36b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.980+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f36b80a9a50 con 0x7f36c806c920 2024-04-25T10:06:26.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.980+0000 7f36c7fff640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f36b406b4d0 0x7f36b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.981+0000 7f36c7fff640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f36b406b4d0 0x7f36b406d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cd570 tx=0x7f36bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:26.984+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36b8078a90 con 0x7f36c806c920 2024-04-25T10:06:27.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.160+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f36c815cbd0 con 0x7f36c806c920 2024-04-25T10:06:27.165 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.160+0000 7f36c57fa640 1 -- 172.21.15.148:0/1562321581 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v181) v1 ==== 84+0+65 (secure 0 0 0) 0x7f36b8078390 con 0x7f36c806c920 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:27.166 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f36b406b4d0 msgr2=0x7f36b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f36b406b4d0 0x7f36b406d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cd570 tx=0x7f36bc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 msgr2=0x7f36c81f3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 0x7f36c81f3930 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f36b8035040 tx=0x7f36b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 shutdown_connections 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f36b406b4d0 0x7f36b406d990 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36c8161800 0x7f36c80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36c815dd80 0x7f36c80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 --2- 172.21.15.148:0/1562321581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36c806c920 0x7f36c81f3930 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 >> 172.21.15.148:0/1562321581 conn(0x7f36c80b3310 msgr2=0x7f36c8099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 shutdown_connections 2024-04-25T10:06:27.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.162+0000 7f36cf128640 1 -- 172.21.15.148:0/1562321581 wait complete. 2024-04-25T10:06:27.179 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:27.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:27.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.275+0000 7f6ef9716640 1 Processor -- start 2024-04-25T10:06:27.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.275+0000 7f6ef9716640 1 -- start start 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef9716640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ef415dc20 0x7f6ef40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef9716640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ef40d32f0 0x7f6ef40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef9716640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef415e000 con 0x7f6ef4161710 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef9716640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef40d1de0 con 0x7f6ef40d32f0 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef9716640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef4161c60 con 0x7f6ef415dc20 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ef40d32f0 0x7f6ef40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ef40d32f0 0x7f6ef40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40616/0 (socket says 172.21.15.148:40616) 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 -- 172.21.15.148:0/3082627500 learned_addr learned my addr 172.21.15.148:0/3082627500 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef2ffd640 1 --2- 172.21.15.148:0/3082627500 >> v2:172.21.15.188:3300/0 conn(0x7f6ef415dc20 0x7f6ef40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 -- 172.21.15.148:0/3082627500 >> v2:172.21.15.188:3300/0 conn(0x7f6ef415dc20 msgr2=0x7f6ef40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 --2- 172.21.15.148:0/3082627500 >> v2:172.21.15.188:3300/0 conn(0x7f6ef415dc20 0x7f6ef40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 -- 172.21.15.148:0/3082627500 >> v1:172.21.15.188:6789/0 conn(0x7f6ef4161710 legacy=0x7f6ef406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef37fe640 1 -- 172.21.15.148:0/3082627500 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4161f40 con 0x7f6ef40d32f0 2024-04-25T10:06:27.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.276+0000 7f6ef2ffd640 1 --2- 172.21.15.148:0/3082627500 >> v2:172.21.15.188:3300/0 conn(0x7f6ef415dc20 0x7f6ef40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:27.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.277+0000 7f6ef37fe640 1 --2- 172.21.15.148:0/3082627500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cebb0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40d1520 tx=0x7f6eec02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f268df3bbd032b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.277+0000 7f6ef27fc640 1 -- 172.21.15.148:0/3082627500 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec036070 con 0x7f6ef40d32f0 2024-04-25T10:06:27.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.277+0000 7f6ef27fc640 1 -- 172.21.15.148:0/3082627500 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec031600 con 0x7f6ef40d32f0 2024-04-25T10:06:27.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.277+0000 7f6ef27fc640 1 -- 172.21.15.148:0/3082627500 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0319d0 con 0x7f6ef40d32f0 2024-04-25T10:06:27.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 -- 172.21.15.148:0/3082627500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef40d32f0 msgr2=0x7f6ef40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3082627500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cebb0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40d1520 tx=0x7f6eec02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 -- 172.21.15.148:0/3082627500 shutdown_connections 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3082627500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cebb0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3082627500 >> v2:172.21.15.188:3300/0 conn(0x7f6ef415dc20 0x7f6ef40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 -- 172.21.15.148:0/3082627500 >> 172.21.15.148:0/3082627500 conn(0x7f6ef40d0420 msgr2=0x7f6ef40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 -- 172.21.15.148:0/3082627500 shutdown_connections 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.278+0000 7f6ef9716640 1 -- 172.21.15.148:0/3082627500 wait complete. 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.279+0000 7f6ef9716640 1 Processor -- start 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.279+0000 7f6ef9716640 1 -- start start 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 0x7f6ef41f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef4161710 0x7f6ef41f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40b5e50 con 0x7f6ef415dc20 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4162500 con 0x7f6ef40d32f0 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40b5fd0 con 0x7f6ef4161710 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 0x7f6ef41f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 0x7f6ef41f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40624/0 (socket says 172.21.15.148:40624) 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 -- 172.21.15.148:0/3282615553 learned_addr learned my addr 172.21.15.148:0/3282615553 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef2ffd640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef37fe640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef4161710 0x7f6ef41f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 -- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef4161710 msgr2=0x7f6ef41f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef4161710 0x7f6ef41f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 -- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef40d32f0 msgr2=0x7f6ef40cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 -- 172.21.15.148:0/3282615553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef41f4860 con 0x7f6ef415dc20 2024-04-25T10:06:27.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef37fe640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef4161710 0x7f6ef41f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:27.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eebfff640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 0x7f6ef41f6c90 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00b3f0 tx=0x7f6edc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc015020 con 0x7f6ef415dc20 2024-04-25T10:06:27.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc011830 con 0x7f6ef415dc20 2024-04-25T10:06:27.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef415e370 con 0x7f6ef415dc20 2024-04-25T10:06:27.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc011c20 con 0x7f6ef415dc20 2024-04-25T10:06:27.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.280+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef415e930 con 0x7f6ef415dc20 2024-04-25T10:06:27.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.282+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef4073a00 con 0x7f6ef415dc20 2024-04-25T10:06:27.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.282+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6edc018020 con 0x7f6ef415dc20 2024-04-25T10:06:27.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.282+0000 7f6eeb7fe640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ed406b4d0 0x7f6ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.282+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6edc089bd0 con 0x7f6ef415dc20 2024-04-25T10:06:27.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.282+0000 7f6ef2ffd640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ed406b4d0 0x7f6ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.290 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.283+0000 7f6ef2ffd640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ed406b4d0 0x7f6ed406d990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40b67f0 tx=0x7f6ee4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.287+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6edc01f030 con 0x7f6ef415dc20 2024-04-25T10:06:27.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.459+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6ef40d16b0 con 0x7f6ef415dc20 2024-04-25T10:06:27.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.867+0000 7f6eeb7fe640 1 -- 172.21.15.148:0/3282615553 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v182) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6edc058c10 con 0x7f6ef415dc20 2024-04-25T10:06:27.874 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:27.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ed406b4d0 msgr2=0x7f6ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ed406b4d0 0x7f6ed406d990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f6ef40b67f0 tx=0x7f6ee4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:27.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 msgr2=0x7f6ef41f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 0x7f6ef41f6c90 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00b3f0 tx=0x7f6edc00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 shutdown_connections 2024-04-25T10:06:27.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ed406b4d0 0x7f6ed406d990 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ef4161710 0x7f6ef41f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ef415dc20 0x7f6ef41f6c90 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 --2- 172.21.15.148:0/3282615553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ef40d32f0 0x7f6ef40cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 >> 172.21.15.148:0/3282615553 conn(0x7f6ef40d0420 msgr2=0x7f6ef415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 shutdown_connections 2024-04-25T10:06:27.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.870+0000 7f6ef9716640 1 -- 172.21.15.148:0/3282615553 wait complete. 2024-04-25T10:06:27.890 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:27.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.990+0000 7f23a6730640 1 Processor -- start 2024-04-25T10:06:27.996 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.990+0000 7f23a6730640 1 -- start start 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f23a6730640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23a00b5c40 0x7f23a00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f23a6730640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b980 0x7f23a00ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f23a6730640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00cf320 con 0x7f23a015b5a0 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f23a6730640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00ced90 con 0x7f23a00b5c40 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f23a6730640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00cf040 con 0x7f23a015b980 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f239f7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23a00b5c40 0x7f23a00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f23a4ccd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b980 0x7f23a00ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f239ffff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f23a015b5a0 0x7f23a00b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60820/0 (socket says 172.21.15.148:60820) 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f239ffff640 1 -- 172.21.15.148:0/2851397367 learned_addr learned my addr 172.21.15.148:0/2851397367 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.991+0000 7f239effd640 1 -- 172.21.15.148:0/2851397367 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963458341 0 0) 0x7f2390001110 con 0x7f23a015b5a0 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f239effd640 1 -- 172.21.15.148:0/2851397367 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2388003520 con 0x7f23a015b5a0 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f23a4ccd640 1 -- 172.21.15.148:0/2851397367 >> v2:172.21.15.188:3300/0 conn(0x7f23a00b5c40 msgr2=0x7f23a00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f23a4ccd640 1 --2- 172.21.15.148:0/2851397367 >> v2:172.21.15.188:3300/0 conn(0x7f23a00b5c40 0x7f23a00ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f23a4ccd640 1 -- 172.21.15.148:0/2851397367 >> v1:172.21.15.188:6789/0 conn(0x7f23a015b5a0 legacy=0x7f23a00b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f23a4ccd640 1 -- 172.21.15.148:0/2851397367 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0161970 con 0x7f23a015b980 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f239f7fe640 1 --2- 172.21.15.148:0/2851397367 >> v2:172.21.15.188:3300/0 conn(0x7f23a00b5c40 0x7f23a00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.992+0000 7f23a4ccd640 1 --2- 172.21.15.148:0/2851397367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b980 0x7f23a00ce850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2398009d90 tx=0x7f239802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c172feb2c3964e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f239effd640 1 -- 172.21.15.148:0/2851397367 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398007740 con 0x7f23a015b980 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f239effd640 1 -- 172.21.15.148:0/2851397367 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23980078e0 con 0x7f23a015b980 2024-04-25T10:06:27.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f239effd640 1 -- 172.21.15.148:0/2851397367 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398007c90 con 0x7f23a015b980 2024-04-25T10:06:28.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 -- 172.21.15.148:0/2851397367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b980 msgr2=0x7f23a00ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 --2- 172.21.15.148:0/2851397367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b980 0x7f23a00ce850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2398009d90 tx=0x7f239802a660 comp rx=0 tx=0).stop 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 -- 172.21.15.148:0/2851397367 shutdown_connections 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 --2- 172.21.15.148:0/2851397367 >> v2:172.21.15.188:3300/0 conn(0x7f23a00b5c40 0x7f23a00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 --2- 172.21.15.148:0/2851397367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b980 0x7f23a00ce850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 -- 172.21.15.148:0/2851397367 >> 172.21.15.148:0/2851397367 conn(0x7f23a00d0530 msgr2=0x7f23a00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 -- 172.21.15.148:0/2851397367 shutdown_connections 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.993+0000 7f23a6730640 1 -- 172.21.15.148:0/2851397367 wait complete. 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.994+0000 7f23a6730640 1 Processor -- start 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.994+0000 7f23a6730640 1 -- start start 2024-04-25T10:06:28.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a6730640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23a00b5c40 0x7f23a00b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a6730640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b5a0 0x7f23a014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a6730640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 0x7f23a014c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a6730640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0162450 con 0x7f23a015b980 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a6730640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0161f30 con 0x7f23a00b5c40 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a6730640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01625d0 con 0x7f23a015b5a0 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a4ccd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 0x7f23a014c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a4ccd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 0x7f23a014c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40626/0 (socket says 172.21.15.148:40626) 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f23a4ccd640 1 -- 172.21.15.148:0/1999926706 learned_addr learned my addr 172.21.15.148:0/1999926706 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f239f7fe640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b5a0 0x7f23a014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.995+0000 7f239ffff640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23a00b5c40 0x7f23a00b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a4ccd640 1 -- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b5a0 msgr2=0x7f23a014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a4ccd640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b5a0 0x7f23a014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a4ccd640 1 -- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23a00b5c40 msgr2=0x7f23a00b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a4ccd640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23a00b5c40 0x7f23a00b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a4ccd640 1 -- 172.21.15.148:0/1999926706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a014db50 con 0x7f23a015b980 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a4ccd640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 0x7f23a014c3c0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f2398009eb0 tx=0x7f2398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239803f070 con 0x7f23a015b980 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398031c60 con 0x7f23a015b980 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a014de30 con 0x7f23a015b980 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398009450 con 0x7f23a015b980 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.996+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a014e6d0 con 0x7f23a015b980 2024-04-25T10:06:28.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.997+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a014e000 con 0x7f23a015b980 2024-04-25T10:06:28.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.998+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f239803b410 con 0x7f23a015b980 2024-04-25T10:06:28.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.998+0000 7f239cff9640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f238c06b4d0 0x7f238c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.998+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f23980ab2d0 con 0x7f23a015b980 2024-04-25T10:06:28.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.998+0000 7f239ffff640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f238c06b4d0 0x7f238c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:27.998+0000 7f239ffff640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f238c06b4d0 0x7f238c06d990 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f2390005880 tx=0x7f2390009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.002+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2398079be0 con 0x7f23a015b980 2024-04-25T10:06:28.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.173+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f23a0073a00 con 0x7f23a015b980 2024-04-25T10:06:28.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.870+0000 7f239cff9640 1 -- 172.21.15.148:0/1999926706 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v183) v1 ==== 170+0+0 (secure 0 0 0) 0x7f239807dc80 con 0x7f23a015b980 2024-04-25T10:06:28.876 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:06:28.878 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f238c06b4d0 msgr2=0x7f238c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f238c06b4d0 0x7f238c06d990 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f2390005880 tx=0x7f2390009040 comp rx=0 tx=0).stop 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 msgr2=0x7f23a014c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 0x7f23a014c3c0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f2398009eb0 tx=0x7f2398004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 shutdown_connections 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f238c06b4d0 0x7f238c06d990 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23a015b980 0x7f23a014c3c0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015b5a0 0x7f23a014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 --2- 172.21.15.148:0/1999926706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23a00b5c40 0x7f23a00b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 >> 172.21.15.148:0/1999926706 conn(0x7f23a00d0530 msgr2=0x7f23a00d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 shutdown_connections 2024-04-25T10:06:28.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:28.873+0000 7f23a6730640 1 -- 172.21.15.148:0/1999926706 wait complete. 2024-04-25T10:06:28.903 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:29.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.000+0000 7f8427eac640 1 Processor -- start 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.001+0000 7f8427eac640 1 -- start start 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.001+0000 7f8427eac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84200d34c0 0x7f84200d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.001+0000 7f8427eac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f84200d08e0 0x7f84200b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.001+0000 7f8427eac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84200d2420 con 0x7f842015dac0 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.001+0000 7f8427eac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200d0d50 con 0x7f84200d34c0 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8427eac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200d3030 con 0x7f84200d08e0 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425447640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f84200d34c0 0x7f84200d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425c48640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f842015dac0 0x7f84200d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45576/0 (socket says 172.21.15.148:45576) 2024-04-25T10:06:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425c48640 1 -- 172.21.15.148:0/3901169440 learned_addr learned my addr 172.21.15.148:0/3901169440 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425447640 1 -- 172.21.15.148:0/3901169440 >> v2:172.21.15.188:3300/0 conn(0x7f84200d08e0 msgr2=0x7f84200b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425447640 1 --2- 172.21.15.148:0/3901169440 >> v2:172.21.15.188:3300/0 conn(0x7f84200d08e0 0x7f84200b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425447640 1 -- 172.21.15.148:0/3901169440 >> v1:172.21.15.148:6789/0 conn(0x7f842015dac0 legacy=0x7f84200d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8425447640 1 -- 172.21.15.148:0/3901169440 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200d2700 con 0x7f84200d34c0 2024-04-25T10:06:29.009 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.002+0000 7f8424c46640 1 -- 172.21.15.148:0/3901169440 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759117017 0 0) 0x7f84200d2420 con 0x7f842015dac0 2024-04-25T10:06:29.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8425447640 1 --2- 172.21.15.148:0/3901169440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f84200d03a0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f84200d2fa0 tx=0x7f841002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1308e9d4103b6875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8424c46640 1 -- 172.21.15.148:0/3901169440 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410037070 con 0x7f84200d34c0 2024-04-25T10:06:29.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8424c46640 1 -- 172.21.15.148:0/3901169440 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8410032440 con 0x7f84200d34c0 2024-04-25T10:06:29.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8424c46640 1 -- 172.21.15.148:0/3901169440 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410032810 con 0x7f84200d34c0 2024-04-25T10:06:29.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 -- 172.21.15.148:0/3901169440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 msgr2=0x7f84200d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 --2- 172.21.15.148:0/3901169440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f84200d03a0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f84200d2fa0 tx=0x7f841002a230 comp rx=0 tx=0).stop 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 -- 172.21.15.148:0/3901169440 shutdown_connections 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 --2- 172.21.15.148:0/3901169440 >> v2:172.21.15.188:3300/0 conn(0x7f84200d08e0 0x7f84200b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 --2- 172.21.15.148:0/3901169440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f84200d03a0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 -- 172.21.15.148:0/3901169440 >> 172.21.15.148:0/3901169440 conn(0x7f8420061c30 msgr2=0x7f842006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 -- 172.21.15.148:0/3901169440 shutdown_connections 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.003+0000 7f8427eac640 1 -- 172.21.15.148:0/3901169440 wait complete. 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 Processor -- start 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 -- start start 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84200d08e0 0x7f8420159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f8420152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f842015dac0 0x7f84200caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200d2100 con 0x7f84200d34c0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200d2cc0 con 0x7f84200d08e0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8427eac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842015dea0 con 0x7f842015dac0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8425447640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f8420152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8425447640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f8420152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40644/0 (socket says 172.21.15.148:40644) 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.005+0000 7f8425447640 1 -- 172.21.15.148:0/2919246520 learned_addr learned my addr 172.21.15.148:0/2919246520 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8425447640 1 -- 172.21.15.148:0/2919246520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f842015dac0 msgr2=0x7f84200caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8425447640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f842015dac0 0x7f84200caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8425447640 1 -- 172.21.15.148:0/2919246520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84200d08e0 msgr2=0x7f8420159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8425447640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84200d08e0 0x7f8420159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8425447640 1 -- 172.21.15.148:0/2919246520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420153d00 con 0x7f84200d34c0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8425447640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f8420152f60 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f84100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410041070 con 0x7f84200d34c0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841003c950 con 0x7f84200d34c0 2024-04-25T10:06:29.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8420156770 con 0x7f84200d34c0 2024-04-25T10:06:29.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410037040 con 0x7f84200d34c0 2024-04-25T10:06:29.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.006+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84200cafe0 con 0x7f84200d34c0 2024-04-25T10:06:29.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.007+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8420073a00 con 0x7f84200d34c0 2024-04-25T10:06:29.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.007+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8410044070 con 0x7f84200d34c0 2024-04-25T10:06:29.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.008+0000 7f84167fc640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f840406b4d0 0x7f840406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.008+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f84100ab110 con 0x7f84200d34c0 2024-04-25T10:06:29.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.008+0000 7f8425c48640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f840406b4d0 0x7f840406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.017 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.008+0000 7f8425c48640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f840406b4d0 0x7f840406d990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8418007fd0 tx=0x7f8418007ce0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.012+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8410079f80 con 0x7f84200d34c0 2024-04-25T10:06:29.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.222+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f84200b5e40 con 0x7f84200d34c0 2024-04-25T10:06:29.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.890+0000 7f84167fc640 1 -- 172.21.15.148:0/2919246520 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v104) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8410079880 con 0x7f84200d34c0 2024-04-25T10:06:29.901 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 27 and data pool 28 2024-04-25T10:06:29.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f840406b4d0 msgr2=0x7f840406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f840406b4d0 0x7f840406d990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8418007fd0 tx=0x7f8418007ce0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 msgr2=0x7f8420152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f8420152f60 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f84100046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 shutdown_connections 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f840406b4d0 0x7f840406d990 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f842015dac0 0x7f84200caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84200d34c0 0x7f8420152f60 secure :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f8410000c00 tx=0x7f84100046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 --2- 172.21.15.148:0/2919246520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84200d08e0 0x7f8420159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.902+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 >> 172.21.15.148:0/2919246520 conn(0x7f8420061c30 msgr2=0x7f842006c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.903+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 shutdown_connections 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:29.904+0000 7f8427eac640 1 -- 172.21.15.148:0/2919246520 wait complete. 2024-04-25T10:06:29.931 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:30.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.025+0000 7f09cb66f640 1 Processor -- start 2024-04-25T10:06:30.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09cb66f640 1 -- start start 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09cb66f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 0x7f09c40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09cb66f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40b97a0 con 0x7f09c40b2cb0 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09cb66f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40d1c50 con 0x7f09c415dbe0 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09cb66f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c415dfc0 con 0x7f09c40d1020 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09c8c0a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f09c40b2cb0 0x7f09c40d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45592/0 (socket says 172.21.15.148:45592) 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.026+0000 7f09c8c0a640 1 -- 172.21.15.148:0/3991596071 learned_addr learned my addr 172.21.15.148:0/3991596071 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118090297 0 0) 0x7f09c40b97a0 con 0x7f09c40b2cb0 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b0003520 con 0x7f09c40b2cb0 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c9c0c640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 0x7f09c40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3601105866 0 0) 0x7f09b0003520 con 0x7f09c40b2cb0 2024-04-25T10:06:30.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 msgr2=0x7f09c40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 0x7f09c40d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 >> v1:172.21.15.188:6789/0 conn(0x7f09c415dbe0 legacy=0x7f09c40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40d1f30 con 0x7f09c40b2cb0 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f09b4001670 con 0x7f09c40b2cb0 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 >> v1:172.21.15.148:6789/0 conn(0x7f09c40b2cb0 legacy=0x7f09c40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 0x7f09b00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b0003f60 0x7f09b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0004d40 0x7f09b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b000ba70 con 0x7f09b0003f60 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b000bc20 con 0x7f09b0004d40 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b000be20 con 0x7f09c40d1020 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c8c0a640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b0003f60 0x7f09b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c9c0c640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0004d40 0x7f09b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c940b640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 0x7f09b00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c8c0a640 1 -- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 msgr2=0x7f09b00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c8c0a640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c40d1020 0x7f09b00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c8c0a640 1 -- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0004d40 msgr2=0x7f09b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c8c0a640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0004d40 0x7f09b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.027+0000 7f09c8c0a640 1 -- 172.21.15.148:0/3991596071 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0003520 con 0x7f09b0003f60 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.028+0000 7f09c8c0a640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b0003f60 0x7f09b0004800 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c210 tx=0x7f09b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d2523ef71fda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.028+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4007d10 con 0x7f09b0003f60 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.028+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b40058d0 con 0x7f09b0003f60 2024-04-25T10:06:30.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.028+0000 7f09bbfff640 1 -- 172.21.15.148:0/3991596071 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4005d40 con 0x7f09b0003f60 2024-04-25T10:06:30.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09cb66f640 1 -- 172.21.15.148:0/3991596071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b0003f60 msgr2=0x7f09b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3991596071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09b0003f60 0x7f09b0004800 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c210 tx=0x7f09b0005830 comp rx=0 tx=0).stop 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09c9c0c640 1 -- 172.21.15.148:0/3991596071 reap_dead start 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09cb66f640 1 -- 172.21.15.148:0/3991596071 shutdown_connections 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09cb66f640 1 -- 172.21.15.148:0/3991596071 >> 172.21.15.148:0/3991596071 conn(0x7f09c4061a20 msgr2=0x7f09c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09cb66f640 1 -- 172.21.15.148:0/3991596071 shutdown_connections 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.029+0000 7f09cb66f640 1 -- 172.21.15.148:0/3991596071 wait complete. 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.030+0000 7f09cb66f640 1 Processor -- start 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.030+0000 7f09cb66f640 1 -- start start 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0003f60 0x7f09c40cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c415e960 0x7f09c41f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 0x7f09c40b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40b92a0 con 0x7f09c40b4330 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40d1a00 con 0x7f09b0003f60 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40b9420 con 0x7f09c415e960 2024-04-25T10:06:30.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 0x7f09c40b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 0x7f09c40b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40652/0 (socket says 172.21.15.148:40652) 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 -- 172.21.15.148:0/3745921243 learned_addr learned my addr 172.21.15.148:0/3745921243 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c940b640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0003f60 0x7f09c40cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c8c0a640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c415e960 0x7f09c41f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 -- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c415e960 msgr2=0x7f09c41f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c415e960 0x7f09c41f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 -- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0003f60 msgr2=0x7f09c40cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0003f60 0x7f09c40cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 -- 172.21.15.148:0/3745921243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40cc6c0 con 0x7f09c40b4330 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c940b640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0003f60 0x7f09c40cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09c9c0c640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 0x7f09c40b47b0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f09c40b25a0 tx=0x7f09bc011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc019020 con 0x7f09c40b4330 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc004240 con 0x7f09c40b4330 2024-04-25T10:06:30.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.031+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c40cc9a0 con 0x7f09c40b4330 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.032+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00f710 con 0x7f09c40b4330 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.032+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c40ccf60 con 0x7f09c40b4330 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.033+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c4073990 con 0x7f09c40b4330 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.033+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09bc01c020 con 0x7f09c40b4330 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.033+0000 7f09b9ffb640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09a806b4d0 0x7f09a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.034+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09bc08a640 con 0x7f09c40b4330 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.034+0000 7f09c940b640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09a806b4d0 0x7f09a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.034+0000 7f09c940b640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09a806b4d0 0x7f09a806d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f09c00019f0 tx=0x7f09c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.038+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09bc023020 con 0x7f09c40b4330 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.210+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09c406cb00 con 0x7f09c40b4330 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.211+0000 7f09b9ffb640 1 -- 172.21.15.148:0/3745921243 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17013 (secure 0 0 0) 0x7f09bc0593c0 con 0x7f09c40b4330 2024-04-25T10:06:30.217 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:30.217 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":184,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:29.223445+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09a806b4d0 msgr2=0x7f09a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09a806b4d0 0x7f09a806d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f09c00019f0 tx=0x7f09c0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 msgr2=0x7f09c40b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 0x7f09c40b47b0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f09c40b25a0 tx=0x7f09bc011cf0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 shutdown_connections 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f09a806b4d0 0x7f09a806d990 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09c40b4330 0x7f09c40b47b0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09c415e960 0x7f09c41f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 --2- 172.21.15.148:0/3745921243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09b0003f60 0x7f09c40cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 >> 172.21.15.148:0/3745921243 conn(0x7f09c4061a20 msgr2=0x7f09c41526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 shutdown_connections 2024-04-25T10:06:30.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.213+0000 7f09cb66f640 1 -- 172.21.15.148:0/3745921243 wait complete. 2024-04-25T10:06:30.230 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 Processor -- start 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 -- start start 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ae815dbe0 0x7f7ae80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae80b97a0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae815dfc0 con 0x7f7ae815dbe0 2024-04-25T10:06:30.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.322+0000 7f7aee43f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80d1a00 con 0x7f7ae80b2cb0 2024-04-25T10:06:30.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ae815dbe0 0x7f7ae80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55272/0 (socket says 172.21.15.148:55272) 2024-04-25T10:06:30.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ae815dbe0 0x7f7ae80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:55132/0 (socket says 172.21.15.148:55132) 2024-04-25T10:06:30.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae77fe640 1 -- 172.21.15.148:0/2831614847 learned_addr learned my addr 172.21.15.148:0/2831614847 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:30.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7aec9dc640 1 --1- 172.21.15.148:0/2831614847 >> v1:172.21.15.188:6789/0 conn(0x7f7ae80b92a0 0x7f7ae80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60838/0 (socket says 172.21.15.148:60838) 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae7fff640 1 -- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 msgr2=0x7f7ae80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae7fff640 1 --2- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae7fff640 1 -- 172.21.15.148:0/2831614847 >> v1:172.21.15.188:6789/0 conn(0x7f7ae80b92a0 legacy=0x7f7ae80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae6ffd640 1 -- 172.21.15.148:0/2831614847 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77609158 0 0) 0x7f7ae80b97a0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae7fff640 1 -- 172.21.15.148:0/2831614847 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae80d1c50 con 0x7f7ae815dbe0 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.323+0000 7f7ae77fe640 1 --2- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:30.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.324+0000 7f7ae7fff640 1 --2- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae80d0910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00097a0 tx=0x7f7ad002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d226f2ecb1091c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.324+0000 7f7ae6ffd640 1 -- 172.21.15.148:0/2831614847 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0037070 con 0x7f7ae815dbe0 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.324+0000 7f7ae6ffd640 1 -- 172.21.15.148:0/2831614847 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0032440 con 0x7f7ae815dbe0 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.324+0000 7f7ae6ffd640 1 -- 172.21.15.148:0/2831614847 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0032810 con 0x7f7ae815dbe0 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 -- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 msgr2=0x7f7ae80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 --2- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae80d0910 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00097a0 tx=0x7f7ad002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 -- 172.21.15.148:0/2831614847 shutdown_connections 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 --2- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 --2- 172.21.15.148:0/2831614847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae80d0910 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 -- 172.21.15.148:0/2831614847 >> 172.21.15.148:0/2831614847 conn(0x7f7ae8061a20 msgr2=0x7f7ae8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 -- 172.21.15.148:0/2831614847 shutdown_connections 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.325+0000 7f7aee43f640 1 -- 172.21.15.148:0/2831614847 wait complete. 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.326+0000 7f7aee43f640 1 Processor -- start 2024-04-25T10:06:30.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.326+0000 7f7aee43f640 1 -- start start 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aee43f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aee43f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 0x7f7ae80b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aee43f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae815f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aee43f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80b6ab0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aee43f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80d2210 con 0x7f7ae815dbe0 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aee43f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80b6c30 con 0x7f7ae80b2cb0 2024-04-25T10:06:30.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 0x7f7ae80b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 0x7f7ae80b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40662/0 (socket says 172.21.15.148:40662) 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 -- 172.21.15.148:0/3698184957 learned_addr learned my addr 172.21.15.148:0/3698184957 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae7fff640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7aec9dc640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 -- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 msgr2=0x7f7ae80c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 -- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 msgr2=0x7f7ae815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.327+0000 7f7ae77fe640 1 -- 172.21.15.148:0/3698184957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae80b3780 con 0x7f7ae80b92a0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.328+0000 7f7ae77fe640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 0x7f7ae80b3240 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800e9b0 tx=0x7f7ad800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.328+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8017070 con 0x7f7ae80b92a0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.328+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad800c9a0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.328+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae81f6db0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.328+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad800cd50 con 0x7f7ae80b92a0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.328+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae81f7370 con 0x7f7ae80b92a0 2024-04-25T10:06:30.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.329+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae8073990 con 0x7f7ae80b92a0 2024-04-25T10:06:30.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.329+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ad801a070 con 0x7f7ae80b92a0 2024-04-25T10:06:30.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.330+0000 7f7ae4ff9640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ac806b4d0 0x7f7ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.330+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ad808a1b0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.330+0000 7f7ae7fff640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ac806b4d0 0x7f7ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.330+0000 7f7ae7fff640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ac806b4d0 0x7f7ac806d990 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0034000 tx=0x7f7ad002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.339 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.334+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ad8058850 con 0x7f7ae80b92a0 2024-04-25T10:06:30.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.503+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ae815c270 con 0x7f7ae80b92a0 2024-04-25T10:06:30.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.504+0000 7f7ae4ff9640 1 -- 172.21.15.148:0/3698184957 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17013 (secure 0 0 0) 0x7f7ad805c8f0 con 0x7f7ae80b92a0 2024-04-25T10:06:30.510 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:30.510 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":184,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:29.223445+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ac806b4d0 msgr2=0x7f7ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ac806b4d0 0x7f7ac806d990 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0034000 tx=0x7f7ad002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 msgr2=0x7f7ae80b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 0x7f7ae80b3240 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800e9b0 tx=0x7f7ad800ee70 comp rx=0 tx=0).stop 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 shutdown_connections 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7ac806b4d0 0x7f7ac806d990 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae815dbe0 0x7f7ae815f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ae80b92a0 0x7f7ae80b3240 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 --2- 172.21.15.148:0/3698184957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80b2cb0 0x7f7ae80c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 >> 172.21.15.148:0/3698184957 conn(0x7f7ae8061a20 msgr2=0x7f7ae80d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 shutdown_connections 2024-04-25T10:06:30.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.506+0000 7f7aee43f640 1 -- 172.21.15.148:0/3698184957 wait complete. 2024-04-25T10:06:30.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:30.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.617+0000 7f216cbed640 1 Processor -- start 2024-04-25T10:06:30.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.617+0000 7f216cbed640 1 -- start start 2024-04-25T10:06:30.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f216cbed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f216806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f216cbed640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21680ce7b0 0x7f2168161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f216cbed640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680c6960 con 0x7f21681616f0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f216cbed640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21681614d0 con 0x7f21680ce7b0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f216cbed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2168162260 con 0x7f21681610f0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f2165d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f21681616f0 0x7f21680ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45596/0 (socket says 172.21.15.148:45596) 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.618+0000 7f2165d9b640 1 -- 172.21.15.148:0/2181251004 learned_addr learned my addr 172.21.15.148:0/2181251004 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f2166d9d640 1 --2- 172.21.15.148:0/2181251004 >> v2:172.21.15.188:3300/0 conn(0x7f21680ce7b0 0x7f2168161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f216806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455956866 0 0) 0x7f21680c6960 con 0x7f21681616f0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2150003520 con 0x7f21681616f0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394585764 0 0) 0x7f2150003520 con 0x7f21681616f0 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 msgr2=0x7f216806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f216806c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 >> v2:172.21.15.188:3300/0 conn(0x7f21680ce7b0 msgr2=0x7f2168161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 --2- 172.21.15.148:0/2181251004 >> v2:172.21.15.188:3300/0 conn(0x7f21680ce7b0 0x7f2168161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168162540 con 0x7f21681616f0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f215c001670 con 0x7f21681616f0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 >> v1:172.21.15.148:6789/0 conn(0x7f21681616f0 legacy=0x7f21680ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500039b0 0x7f2150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f2150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2150004d40 0x7f21500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21681614d0 con 0x7f21500039b0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168162260 con 0x7f2150004d40 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215000b9c0 con 0x7f21681610f0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500039b0 0x7f2150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f2165d9b640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f2150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f2166d9d640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2150004d40 0x7f21500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 -- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 msgr2=0x7f2150004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f2150004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 -- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2150004d40 msgr2=0x7f21500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2150004d40 0x7f21500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f216659c640 1 -- 172.21.15.148:0/2181251004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2150003520 con 0x7f21500039b0 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.619+0000 7f2165d9b640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21681610f0 0x7f2150004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.620+0000 7f216659c640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500039b0 0x7f2150003dd0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f2158009fc0 tx=0x7f2150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef1fe05e6f1f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.620+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158009c10 con 0x7f21500039b0 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.620+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158036070 con 0x7f21500039b0 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.620+0000 7f216559a640 1 -- 172.21.15.148:0/2181251004 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158031880 con 0x7f21500039b0 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f216cbed640 1 -- 172.21.15.148:0/2181251004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500039b0 msgr2=0x7f2150003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f216cbed640 1 --2- 172.21.15.148:0/2181251004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500039b0 0x7f2150003dd0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f2158009fc0 tx=0x7f2150005830 comp rx=0 tx=0).stop 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f2166d9d640 1 -- 172.21.15.148:0/2181251004 reap_dead start 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f216cbed640 1 -- 172.21.15.148:0/2181251004 shutdown_connections 2024-04-25T10:06:30.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f216cbed640 1 -- 172.21.15.148:0/2181251004 >> 172.21.15.148:0/2181251004 conn(0x7f216806f500 msgr2=0x7f216806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f216cbed640 1 -- 172.21.15.148:0/2181251004 shutdown_connections 2024-04-25T10:06:30.630 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.621+0000 7f216cbed640 1 -- 172.21.15.148:0/2181251004 wait complete. 2024-04-25T10:06:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.622+0000 7f216cbed640 1 Processor -- start 2024-04-25T10:06:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.622+0000 7f216cbed640 1 -- start start 2024-04-25T10:06:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216cbed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21500039b0 0x7f2168143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.632 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216cbed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 0x7f21680b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216cbed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f216815e4d0 0x7f21680b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216cbed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21680cf690 con 0x7f216815e070 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216cbed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21680cfce0 con 0x7f216815e4d0 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216cbed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216806c1c0 con 0x7f21500039b0 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 0x7f21680b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 0x7f21680b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40678/0 (socket says 172.21.15.148:40678) 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 -- 172.21.15.148:0/2431011146 learned_addr learned my addr 172.21.15.148:0/2431011146 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f216659c640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21500039b0 0x7f2168143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2166d9d640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f216815e4d0 0x7f21680b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 -- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21500039b0 msgr2=0x7f2168143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21500039b0 0x7f2168143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 -- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f216815e4d0 msgr2=0x7f21680b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.633 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f216815e4d0 0x7f21680b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.623+0000 7f2165d9b640 1 -- 172.21.15.148:0/2431011146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216815ea50 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.624+0000 7f2165d9b640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 0x7f21680b4520 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f215c00e8f0 tx=0x7f215c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.624+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c019070 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.624+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c00c870 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.624+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216815ed30 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.624+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c00cc20 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.624+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216815f2f0 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.625+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2168073a00 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.625+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f215c01c070 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.626+0000 7f2156ffd640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f214806b4d0 0x7f214806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.626+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f215c089ca0 con 0x7f216815e070 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.626+0000 7f216659c640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f214806b4d0 0x7f214806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.634 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.626+0000 7f216659c640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f214806b4d0 0x7f214806d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2158035000 tx=0x7f215802f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.630+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f215c0582b0 con 0x7f216815e070 2024-04-25T10:06:30.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.835+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2168161f30 con 0x7f216815e070 2024-04-25T10:06:30.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.900+0000 7f2156ffd640 1 -- 172.21.15.148:0/2431011146 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v105) v1 ==== 118+0+0 (secure 0 0 0) 0x7f215c05c350 con 0x7f216815e070 2024-04-25T10:06:30.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.902+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f214806b4d0 msgr2=0x7f214806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.902+0000 7f216cbed640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f214806b4d0 0x7f214806d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2158035000 tx=0x7f215802f000 comp rx=0 tx=0).stop 2024-04-25T10:06:30.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.902+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 msgr2=0x7f21680b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.902+0000 7f216cbed640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 0x7f21680b4520 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f215c00e8f0 tx=0x7f215c00edb0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 shutdown_connections 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f214806b4d0 0x7f214806d990 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f216815e4d0 0x7f21680b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f216815e070 0x7f21680b4520 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 --2- 172.21.15.148:0/2431011146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21500039b0 0x7f2168143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 >> 172.21.15.148:0/2431011146 conn(0x7f216806f500 msgr2=0x7f21680c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 shutdown_connections 2024-04-25T10:06:30.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:30.903+0000 7f216cbed640 1 -- 172.21.15.148:0/2431011146 wait complete. 2024-04-25T10:06:30.919 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:30.919 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:31.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.013+0000 7fcf4c7f4640 1 Processor -- start 2024-04-25T10:06:31.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 -- start start 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf4415a9f0 0x7fcf440c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf440cdbf0 0x7fcf440ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cd520 0x7fcf440ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440b0020 con 0x7fcf440cdbf0 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440cb5a0 con 0x7fcf4415a9f0 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.014+0000 7fcf4c7f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440c6940 con 0x7fcf440cd520 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf440cdbf0 0x7fcf440ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcf440cdbf0 0x7fcf440ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40692/0 (socket says 172.21.15.148:40692) 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 -- 172.21.15.148:0/4239404760 learned_addr learned my addr 172.21.15.148:0/4239404760 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf4a590640 1 --2- 172.21.15.148:0/4239404760 >> v2:172.21.15.188:3300/0 conn(0x7fcf4415a9f0 0x7fcf440c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf4ad91640 1 --2- 172.21.15.148:0/4239404760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cd520 0x7fcf440ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 -- 172.21.15.148:0/4239404760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cd520 msgr2=0x7fcf440ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 --2- 172.21.15.148:0/4239404760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cd520 0x7fcf440ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 -- 172.21.15.148:0/4239404760 >> v2:172.21.15.188:3300/0 conn(0x7fcf4415a9f0 msgr2=0x7fcf440c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 --2- 172.21.15.148:0/4239404760 >> v2:172.21.15.188:3300/0 conn(0x7fcf4415a9f0 0x7fcf440c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.015+0000 7fcf49d8f640 1 -- 172.21.15.148:0/4239404760 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4415add0 con 0x7fcf440cdbf0 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf49d8f640 1 --2- 172.21.15.148:0/4239404760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cdbf0 0x7fcf440ccfe0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fcf440c77e0 tx=0x7fcf4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4521cefac8b1516b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4958e640 1 -- 172.21.15.148:0/4239404760 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40037070 con 0x7fcf440cdbf0 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4958e640 1 -- 172.21.15.148:0/4239404760 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf40032440 con 0x7fcf440cdbf0 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4958e640 1 -- 172.21.15.148:0/4239404760 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40032810 con 0x7fcf440cdbf0 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/4239404760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cdbf0 msgr2=0x7fcf440ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/4239404760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cdbf0 0x7fcf440ccfe0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fcf440c77e0 tx=0x7fcf4002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/4239404760 shutdown_connections 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/4239404760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cd520 0x7fcf440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/4239404760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cdbf0 0x7fcf440ccfe0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/4239404760 >> v2:172.21.15.188:3300/0 conn(0x7fcf4415a9f0 0x7fcf440c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/4239404760 >> 172.21.15.148:0/4239404760 conn(0x7fcf440b2390 msgr2=0x7fcf440b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/4239404760 shutdown_connections 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.016+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/4239404760 wait complete. 2024-04-25T10:06:31.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 Processor -- start 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 -- start start 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 0x7fcf440b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cdbf0 0x7fcf440c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4415a9f0 0x7fcf441f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440cb230 con 0x7fcf440cd520 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440caf50 con 0x7fcf4415a9f0 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4c7f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440ce070 con 0x7fcf440cdbf0 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4a590640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 0x7fcf440b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4a590640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 0x7fcf440b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40694/0 (socket says 172.21.15.148:40694) 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.018+0000 7fcf4a590640 1 -- 172.21.15.148:0/704678608 learned_addr learned my addr 172.21.15.148:0/704678608 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:31.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4a590640 1 -- 172.21.15.148:0/704678608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cdbf0 msgr2=0x7fcf440c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4a590640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cdbf0 0x7fcf440c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4a590640 1 -- 172.21.15.148:0/704678608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4415a9f0 msgr2=0x7fcf441f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4a590640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4415a9f0 0x7fcf441f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4a590640 1 -- 172.21.15.148:0/704678608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440c8760 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4a590640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 0x7fcf440b6010 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400d620 tx=0x7fcf3400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400b620 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34019070 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf440b40f0 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400ba00 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.019+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf440b46b0 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.020+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4406cc80 con 0x7fcf440cd520 2024-04-25T10:06:31.028 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.020+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf3400bba0 con 0x7fcf440cd520 2024-04-25T10:06:31.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.021+0000 7fcf3affd640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf2806b4d0 0x7fcf2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.021+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf3408a740 con 0x7fcf440cd520 2024-04-25T10:06:31.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.021+0000 7fcf49d8f640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf2806b4d0 0x7fcf2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.021+0000 7fcf49d8f640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf2806b4d0 0x7fcf2806d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcf40034000 tx=0x7fcf4002c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.025+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf340594c0 con 0x7fcf440cd520 2024-04-25T10:06:31.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.229+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcf440c7140 con 0x7fcf440cd520 2024-04-25T10:06:31.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.907+0000 7fcf3affd640 1 -- 172.21.15.148:0/704678608 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v106) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcf440c7140 con 0x7fcf440cd520 2024-04-25T10:06:31.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf2806b4d0 msgr2=0x7fcf2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf2806b4d0 0x7fcf2806d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcf40034000 tx=0x7fcf4002c000 comp rx=0 tx=0).stop 2024-04-25T10:06:31.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 msgr2=0x7fcf440b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 0x7fcf440b6010 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400d620 tx=0x7fcf3400dae0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 shutdown_connections 2024-04-25T10:06:31.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf2806b4d0 0x7fcf2806d990 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4415a9f0 0x7fcf441f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf440cdbf0 0x7fcf440c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 --2- 172.21.15.148:0/704678608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf440cd520 0x7fcf440b6010 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 >> 172.21.15.148:0/704678608 conn(0x7fcf440b2390 msgr2=0x7fcf4406fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 shutdown_connections 2024-04-25T10:06:31.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:31.909+0000 7fcf4c7f4640 1 -- 172.21.15.148:0/704678608 wait complete. 2024-04-25T10:06:31.928 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:31.928 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:32.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.024+0000 7f3193bf9640 1 Processor -- start 2024-04-25T10:06:32.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.024+0000 7f3193bf9640 1 -- start start 2024-04-25T10:06:32.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3193bf9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f318c15dc20 0x7f318c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3193bf9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f318c0b93a0 0x7f318c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3193bf9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3193bf9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c15e000 con 0x7f318c0b93a0 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3193bf9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c0b6bb0 con 0x7f318c15dc20 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3193bf9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0b6030 con 0x7f318c0cec20 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f318c0b93a0 0x7f318c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f318c0b93a0 0x7f318c0ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40706/0 (socket says 172.21.15.148:40706) 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 -- 172.21.15.148:0/2522628531 learned_addr learned my addr 172.21.15.148:0/2522628531 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 -- 172.21.15.148:0/2522628531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 msgr2=0x7f318c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 --2- 172.21.15.148:0/2522628531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 -- 172.21.15.148:0/2522628531 >> v2:172.21.15.188:3300/0 conn(0x7f318c15dc20 msgr2=0x7f318c1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191995640 1 --2- 172.21.15.148:0/2522628531 >> v2:172.21.15.188:3300/0 conn(0x7f318c15dc20 0x7f318c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 --2- 172.21.15.148:0/2522628531 >> v2:172.21.15.188:3300/0 conn(0x7f318c15dc20 0x7f318c1616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.025+0000 7f3191194640 1 -- 172.21.15.148:0/2522628531 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c161de0 con 0x7f318c0b93a0 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.026+0000 7f3191194640 1 --2- 172.21.15.148:0/2522628531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c0b93a0 0x7f318c0ce6e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f318c08be60 tx=0x7f317c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41bf42e7e0bba97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.026+0000 7f3190993640 1 -- 172.21.15.148:0/2522628531 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c037070 con 0x7f318c0b93a0 2024-04-25T10:06:32.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.026+0000 7f3190993640 1 -- 172.21.15.148:0/2522628531 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c032440 con 0x7f318c0b93a0 2024-04-25T10:06:32.033 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.026+0000 7f3190993640 1 -- 172.21.15.148:0/2522628531 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c032810 con 0x7f318c0b93a0 2024-04-25T10:06:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 -- 172.21.15.148:0/2522628531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c0b93a0 msgr2=0x7f318c0ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 --2- 172.21.15.148:0/2522628531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c0b93a0 0x7f318c0ce6e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f318c08be60 tx=0x7f317c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 -- 172.21.15.148:0/2522628531 shutdown_connections 2024-04-25T10:06:32.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 --2- 172.21.15.148:0/2522628531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 --2- 172.21.15.148:0/2522628531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c0b93a0 0x7f318c0ce6e0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 --2- 172.21.15.148:0/2522628531 >> v2:172.21.15.188:3300/0 conn(0x7f318c15dc20 0x7f318c1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 -- 172.21.15.148:0/2522628531 >> 172.21.15.148:0/2522628531 conn(0x7f318c0d0490 msgr2=0x7f318c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 -- 172.21.15.148:0/2522628531 shutdown_connections 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.027+0000 7f3193bf9640 1 -- 172.21.15.148:0/2522628531 wait complete. 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.028+0000 7f3193bf9640 1 Processor -- start 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.028+0000 7f3193bf9640 1 -- start start 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f318c0b93a0 0x7f318c1f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c1e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 0x7f318c1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c06c610 con 0x7f318c15dc20 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1623a0 con 0x7f318c0b93a0 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c06c790 con 0x7f318c0cec20 2024-04-25T10:06:32.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 0x7f318c1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 0x7f318c1f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40716/0 (socket says 172.21.15.148:40716) 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 -- 172.21.15.148:0/786330892 learned_addr learned my addr 172.21.15.148:0/786330892 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3191194640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c1e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3191995640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f318c0b93a0 0x7f318c1f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 -- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 msgr2=0x7f318c1e7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c1e7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 -- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f318c0b93a0 msgr2=0x7f318c1f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f318c0b93a0 0x7f318c1f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 -- 172.21.15.148:0/786330892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c1611e0 con 0x7f318c15dc20 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3192196640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 0x7f318c1f6fe0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f318400b9a0 tx=0x7f318400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184019070 con 0x7f318c15dc20 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318400cb60 con 0x7f318c15dc20 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c160f00 con 0x7f318c15dc20 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.029+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184014440 con 0x7f318c15dc20 2024-04-25T10:06:32.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.030+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c15e730 con 0x7f318c15dc20 2024-04-25T10:06:32.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.030+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c073990 con 0x7f318c15dc20 2024-04-25T10:06:32.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.031+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f318401c070 con 0x7f318c15dc20 2024-04-25T10:06:32.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.031+0000 7f31827fc640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f317006b4d0 0x7f317006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.032+0000 7f3191995640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f317006b4d0 0x7f317006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.032+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f318408a010 con 0x7f318c15dc20 2024-04-25T10:06:32.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.032+0000 7f3191995640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f317006b4d0 0x7f317006d990 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f318800aa20 tx=0x7f3188008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.036+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31840586b0 con 0x7f318c15dc20 2024-04-25T10:06:32.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.242+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f318c0d1720 con 0x7f318c15dc20 2024-04-25T10:06:32.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.242+0000 7f31827fc640 1 -- 172.21.15.148:0/786330892 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f318405c750 con 0x7f318c15dc20 2024-04-25T10:06:32.248 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:32.248 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:29.223742+0000","modified":"2024-04-25T10:06:31.231117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11268":{"gid":11268,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1743980733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1743980733},{"type":"v1","addr":"172.21.15.148:6837","nonce":1743980733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:06:32.249 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 106 2024-04-25T10:06:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f317006b4d0 msgr2=0x7f317006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f317006b4d0 0x7f317006d990 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f318800aa20 tx=0x7f3188008040 comp rx=0 tx=0).stop 2024-04-25T10:06:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 msgr2=0x7f318c1f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 0x7f318c1f6fe0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f318400b9a0 tx=0x7f318400be60 comp rx=0 tx=0).stop 2024-04-25T10:06:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 shutdown_connections 2024-04-25T10:06:32.250 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f317006b4d0 0x7f317006d990 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f318c15dc20 0x7f318c1f6fe0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f318c0cec20 0x7f318c1e7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 --2- 172.21.15.148:0/786330892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f318c0b93a0 0x7f318c1f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 >> 172.21.15.148:0/786330892 conn(0x7f318c0d0490 msgr2=0x7f318c0b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 shutdown_connections 2024-04-25T10:06:32.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.244+0000 7f3193bf9640 1 -- 172.21.15.148:0/786330892 wait complete. 2024-04-25T10:06:32.261 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:32.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.352+0000 7f48022c4640 1 Processor -- start 2024-04-25T10:06:32.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.352+0000 7f48022c4640 1 -- start start 2024-04-25T10:06:32.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f48022c4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47fc0b6cb0 0x7f47fc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f48022c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc0b5a60 0x7f47fc0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f48022c4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc0ceee0 con 0x7f47fc15db70 2024-04-25T10:06:32.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f48022c4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47fc0d1d10 con 0x7f47fc0b6cb0 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f48022c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0d1ff0 con 0x7f47fc0b5a60 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47fbfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f47fc15db70 0x7f47fc162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45610/0 (socket says 172.21.15.148:45610) 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47fbfff640 1 -- 172.21.15.148:0/3857163793 learned_addr learned my addr 172.21.15.148:0/3857163793 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47fb7fe640 1 --2- 172.21.15.148:0/3857163793 >> v2:172.21.15.188:3300/0 conn(0x7f47fc0b6cb0 0x7f47fc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f4800861640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc0b5a60 0x7f47fc0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274744883 0 0) 0x7f47fc0ceee0 con 0x7f47fc15db70 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e4003520 con 0x7f47fc15db70 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320252784 0 0) 0x7f47e4003520 con 0x7f47fc15db70 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc0b5a60 msgr2=0x7f47fc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc0b5a60 0x7f47fc0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 >> v2:172.21.15.188:3300/0 conn(0x7f47fc0b6cb0 msgr2=0x7f47fc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 --2- 172.21.15.148:0/3857163793 >> v2:172.21.15.188:3300/0 conn(0x7f47fc0b6cb0 0x7f47fc0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc0d22d0 con 0x7f47fc15db70 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f47ec001670 con 0x7f47fc15db70 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47fb7fe640 1 --2- 172.21.15.148:0/3857163793 >> v2:172.21.15.188:3300/0 conn(0x7f47fc0b6cb0 0x7f47fc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 >> v1:172.21.15.148:6789/0 conn(0x7f47fc15db70 legacy=0x7f47fc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40039b0 0x7f47e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.353+0000 7f47faffd640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e4004800 0x7f47e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f47faffd640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e40051c0 0x7f47e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0d1d10 con 0x7f47e40051c0 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0d1ff0 con 0x7f47e40039b0 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e400bf00 con 0x7f47e4004800 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc0b5a60 0x7f47fc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f47fbfff640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40039b0 0x7f47e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e40051c0 0x7f47e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f47fb7fe640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e4004800 0x7f47e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 -- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e4004800 msgr2=0x7f47e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e4004800 0x7f47e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 -- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40039b0 msgr2=0x7f47e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40039b0 0x7f47e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.354+0000 7f4800861640 1 -- 172.21.15.148:0/3857163793 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e4003520 con 0x7f47e40051c0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f4800861640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e40051c0 0x7f47e400b9c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f47f400e220 tx=0x7f47e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cbcfda844a4e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4009d40 con 0x7f47e40051c0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4035070 con 0x7f47e40051c0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f47faffd640 1 -- 172.21.15.148:0/3857163793 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f402c040 con 0x7f47e40051c0 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f48022c4640 1 -- 172.21.15.148:0/3857163793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e40051c0 msgr2=0x7f47e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f48022c4640 1 --2- 172.21.15.148:0/3857163793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47e40051c0 0x7f47e400b9c0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f47f400e220 tx=0x7f47e4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:32.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f4800861640 1 -- 172.21.15.148:0/3857163793 reap_dead start 2024-04-25T10:06:32.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f48022c4640 1 -- 172.21.15.148:0/3857163793 shutdown_connections 2024-04-25T10:06:32.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f48022c4640 1 -- 172.21.15.148:0/3857163793 >> 172.21.15.148:0/3857163793 conn(0x7f47fc0d0520 msgr2=0x7f47fc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f48022c4640 1 -- 172.21.15.148:0/3857163793 shutdown_connections 2024-04-25T10:06:32.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.355+0000 7f48022c4640 1 -- 172.21.15.148:0/3857163793 wait complete. 2024-04-25T10:06:32.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.356+0000 7f48022c4640 1 Processor -- start 2024-04-25T10:06:32.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 -- start start 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40051c0 0x7f47fc15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc15ff30 0x7f47fc0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 0x7f47fc149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0cf1c0 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0ce530 con 0x7f47e40051c0 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f48022c4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0cf340 con 0x7f47fc15ff30 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f4800861640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 0x7f47fc149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f4800861640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 0x7f47fc149020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40734/0 (socket says 172.21.15.148:40734) 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.357+0000 7f4800861640 1 -- 172.21.15.148:0/3272633819 learned_addr learned my addr 172.21.15.148:0/3272633819 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f47fbfff640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40051c0 0x7f47fc15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f4800861640 1 -- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc15ff30 msgr2=0x7f47fc0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f4800861640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc15ff30 0x7f47fc0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f4800861640 1 -- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40051c0 msgr2=0x7f47fc15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f4800861640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40051c0 0x7f47fc15f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f4800861640 1 -- 172.21.15.148:0/3272633819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc149cc0 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f4800861640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 0x7f47fc149020 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f47f400e340 tx=0x7f47f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4009d40 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4004560 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4030440 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc149fa0 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.358+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc14a560 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.359+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc073990 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.360+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47f4005d10 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.360+0000 7f47f8ff9640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47e806b500 0x7f47e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.360+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47f40ab250 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.360+0000 7f47fbfff640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47e806b500 0x7f47e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.361+0000 7f47fbfff640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47e806b500 0x7f47e806d9c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f47ec007fd0 tx=0x7f47ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.364+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47f404a060 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.534+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47fc0d1820 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.535+0000 7f47f8ff9640 1 -- 172.21.15.148:0/3272633819 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f47f4039390 con 0x7f47fc0cb0b0 2024-04-25T10:06:32.541 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:32.541 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":185,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:30.890882+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:32.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47e806b500 msgr2=0x7f47e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47e806b500 0x7f47e806d9c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f47ec007fd0 tx=0x7f47ec009040 comp rx=0 tx=0).stop 2024-04-25T10:06:32.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 msgr2=0x7f47fc149020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 0x7f47fc149020 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f47f400e340 tx=0x7f47f4009750 comp rx=0 tx=0).stop 2024-04-25T10:06:32.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 shutdown_connections 2024-04-25T10:06:32.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f47e806b500 0x7f47e806d9c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47fc0cb0b0 0x7f47fc149020 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47fc15ff30 0x7f47fc0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 --2- 172.21.15.148:0/3272633819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e40051c0 0x7f47fc15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.537+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 >> 172.21.15.148:0/3272633819 conn(0x7f47fc0d0520 msgr2=0x7f47fc15b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.538+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 shutdown_connections 2024-04-25T10:06:32.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.538+0000 7f48022c4640 1 -- 172.21.15.148:0/3272633819 wait complete. 2024-04-25T10:06:32.554 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:32.555 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:32.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.646+0000 7f1860cf7640 1 Processor -- start 2024-04-25T10:06:32.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.646+0000 7f1860cf7640 1 -- start start 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.647+0000 7f1860cf7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 0x7f185c168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.647+0000 7f1860cf7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c0adcf0 con 0x7f185c071ac0 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.647+0000 7f1860cf7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c15ec60 con 0x7f185c154ed0 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.647+0000 7f1860cf7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c15ef40 con 0x7f185c06c1d0 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f1859d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f185c071ac0 0x7f185c0ad5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:45616/0 (socket says 172.21.15.148:45616) 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f1859d9b640 1 -- 172.21.15.148:0/4059338725 learned_addr learned my addr 172.21.15.148:0/4059338725 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 0x7f185c168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929971057 0 0) 0x7f185c0adcf0 con 0x7f185c071ac0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1844003520 con 0x7f185c071ac0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671481839 0 0) 0x7f1844003520 con 0x7f185c071ac0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 msgr2=0x7f185c168930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 0x7f185c168930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 >> v1:172.21.15.188:6789/0 conn(0x7f185c154ed0 legacy=0x7f185c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c06db90 con 0x7f185c071ac0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f184c001670 con 0x7f185c071ac0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 >> v1:172.21.15.148:6789/0 conn(0x7f185c071ac0 legacy=0x7f185c0ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 0x7f18440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1844003f60 0x7f1844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1844004d40 0x7f18440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c15ef40 con 0x7f1844004d40 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184400baa0 con 0x7f1844003f60 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184400bca0 con 0x7f185c06c1d0 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1844004d40 0x7f18440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f185a59c640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 0x7f18440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.648+0000 7f1859d9b640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1844003f60 0x7f1844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185ad9d640 1 -- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 msgr2=0x7f18440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c06c1d0 0x7f18440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185ad9d640 1 -- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1844003f60 msgr2=0x7f1844004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1844003f60 0x7f1844004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185ad9d640 1 -- 172.21.15.148:0/4059338725 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844003520 con 0x7f1844004d40 2024-04-25T10:06:32.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1844004d40 0x7f18440051c0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f1854009fc0 tx=0x7f1844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0754ba667daf9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854009c10 con 0x7f1844004d40 2024-04-25T10:06:32.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1854036070 con 0x7f1844004d40 2024-04-25T10:06:32.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.649+0000 7f185959a640 1 -- 172.21.15.148:0/4059338725 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854031880 con 0x7f1844004d40 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f1860cf7640 1 -- 172.21.15.148:0/4059338725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1844004d40 msgr2=0x7f18440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4059338725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1844004d40 0x7f18440051c0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f1854009fc0 tx=0x7f1844005830 comp rx=0 tx=0).stop 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f185ad9d640 1 -- 172.21.15.148:0/4059338725 reap_dead start 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f1860cf7640 1 -- 172.21.15.148:0/4059338725 shutdown_connections 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f1860cf7640 1 -- 172.21.15.148:0/4059338725 >> 172.21.15.148:0/4059338725 conn(0x7f185c0af670 msgr2=0x7f185c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f1860cf7640 1 -- 172.21.15.148:0/4059338725 shutdown_connections 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.650+0000 7f1860cf7640 1 -- 172.21.15.148:0/4059338725 wait complete. 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.651+0000 7f1860cf7640 1 Processor -- start 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.651+0000 7f1860cf7640 1 -- start start 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1860cf7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c149bf0 0x7f185c163260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1860cf7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f185c149fd0 0x7f185c14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1860cf7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 0x7f185c14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1860cf7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c166aa0 con 0x7f185c14a450 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1860cf7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c166920 con 0x7f185c149fd0 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1860cf7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c166c20 con 0x7f185c149bf0 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f185ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 0x7f185c14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f185ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 0x7f185c14eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40752/0 (socket says 172.21.15.148:40752) 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f185ad9d640 1 -- 172.21.15.148:0/4060337192 learned_addr learned my addr 172.21.15.148:0/4060337192 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f1859d9b640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f185c149fd0 0x7f185c14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.652+0000 7f185a59c640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c149bf0 0x7f185c163260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185ad9d640 1 -- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c149bf0 msgr2=0x7f185c163260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c149bf0 0x7f185c163260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185ad9d640 1 -- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f185c149fd0 msgr2=0x7f185c14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f185c149fd0 0x7f185c14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185ad9d640 1 -- 172.21.15.148:0/4060337192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c0afff0 con 0x7f185c14a450 2024-04-25T10:06:32.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185a59c640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c149bf0 0x7f185c163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f185ad9d640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 0x7f185c14eea0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f1854033040 tx=0x7f1854009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854009c10 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1854031e10 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c0b02d0 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854003bd0 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.653+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c14b070 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.654+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c0b3c90 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.654+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1854005d10 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.655+0000 7f184affd640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f183c06b4d0 0x7f183c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.655+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18540ab7a0 con 0x7f185c14a450 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.655+0000 7f185a59c640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f183c06b4d0 0x7f183c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.655+0000 7f185a59c640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f183c06b4d0 0x7f183c06d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f18500035f0 tx=0x7f1850009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.659+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f185404a060 con 0x7f185c14a450 2024-04-25T10:06:32.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.825+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f185c073990 con 0x7f183c06b4d0 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.831+0000 7f184affd640 1 -- 172.21.15.148:0/4060337192 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252424 (secure 0 0 0) 0x7f185c073990 con 0x7f183c06b4d0 2024-04-25T10:06:32.842 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:32.843 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f183c06b4d0 msgr2=0x7f183c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f183c06b4d0 0x7f183c06d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f18500035f0 tx=0x7f1850009040 comp rx=0 tx=0).stop 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 msgr2=0x7f185c14eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 0x7f185c14eea0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f1854033040 tx=0x7f1854009750 comp rx=0 tx=0).stop 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 shutdown_connections 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f183c06b4d0 0x7f183c06d990 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f185c14a450 0x7f185c14eea0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f185c149fd0 0x7f185c14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 --2- 172.21.15.148:0/4060337192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f185c149bf0 0x7f185c163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 >> 172.21.15.148:0/4060337192 conn(0x7f1844004d40 msgr2=0x7f185c15fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 shutdown_connections 2024-04-25T10:06:32.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.834+0000 7f1860cf7640 1 -- 172.21.15.148:0/4060337192 wait complete. 2024-04-25T10:06:32.851 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-04-25T10:06:31.171769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":446340,"kb_used_data":19208,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754528380,"statfs":{"total":773094113280,"available":772637061120,"internally_reserved":0,"allocated":19668992,"data_stored":9836248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":437243440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000669"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905720+0000","last_change":"2024-04-25T10:06:29.903429+0000","last_active":"2024-04-25T10:06:29.905720+0000","last_peered":"2024-04-25T10:06:29.905720+0000","last_clean":"2024-04-25T10:06:29.905720+0000","last_became_active":"2024-04-25T10:06:29.903123+0000","last_became_peered":"2024-04-25T10:06:29.903123+0000","last_unstale":"2024-04-25T10:06:29.905720+0000","last_undegraded":"2024-04-25T10:06:29.905720+0000","last_fullsized":"2024-04-25T10:06:29.905720+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.824563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.900098+0000","last_change":"2024-04-25T10:06:29.899227+0000","last_active":"2024-04-25T10:06:29.900098+0000","last_peered":"2024-04-25T10:06:29.900098+0000","last_clean":"2024-04-25T10:06:29.900098+0000","last_became_active":"2024-04-25T10:06:29.898960+0000","last_became_peered":"2024-04-25T10:06:29.898960+0000","last_unstale":"2024-04-25T10:06:29.900098+0000","last_undegraded":"2024-04-25T10:06:29.900098+0000","last_fullsized":"2024-04-25T10:06:29.900098+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.851479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904899+0000","last_change":"2024-04-25T10:06:29.904040+0000","last_active":"2024-04-25T10:06:29.904899+0000","last_peered":"2024-04-25T10:06:29.904899+0000","last_clean":"2024-04-25T10:06:29.904899+0000","last_became_active":"2024-04-25T10:06:29.903792+0000","last_became_peered":"2024-04-25T10:06:29.903792+0000","last_unstale":"2024-04-25T10:06:29.904899+0000","last_undegraded":"2024-04-25T10:06:29.904899+0000","last_fullsized":"2024-04-25T10:06:29.904899+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:03.129065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907686+0000","last_change":"2024-04-25T10:06:29.906906+0000","last_active":"2024-04-25T10:06:29.907686+0000","last_peered":"2024-04-25T10:06:29.907686+0000","last_clean":"2024-04-25T10:06:29.907686+0000","last_became_active":"2024-04-25T10:06:29.906617+0000","last_became_peered":"2024-04-25T10:06:29.906617+0000","last_unstale":"2024-04-25T10:06:29.907686+0000","last_undegraded":"2024-04-25T10:06:29.907686+0000","last_fullsized":"2024-04-25T10:06:29.907686+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.824563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902352+0000","last_change":"2024-04-25T10:06:29.899792+0000","last_active":"2024-04-25T10:06:29.902352+0000","last_peered":"2024-04-25T10:06:29.902352+0000","last_clean":"2024-04-25T10:06:29.902352+0000","last_became_active":"2024-04-25T10:06:29.899476+0000","last_became_peered":"2024-04-25T10:06:29.899476+0000","last_unstale":"2024-04-25T10:06:29.902352+0000","last_undegraded":"2024-04-25T10:06:29.902352+0000","last_fullsized":"2024-04-25T10:06:29.902352+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:22.774713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907670+0000","last_change":"2024-04-25T10:06:29.904690+0000","last_active":"2024-04-25T10:06:29.907670+0000","last_peered":"2024-04-25T10:06:29.907670+0000","last_clean":"2024-04-25T10:06:29.907670+0000","last_became_active":"2024-04-25T10:06:29.904428+0000","last_became_peered":"2024-04-25T10:06:29.904428+0000","last_unstale":"2024-04-25T10:06:29.907670+0000","last_undegraded":"2024-04-25T10:06:29.907670+0000","last_fullsized":"2024-04-25T10:06:29.907670+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.413449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908681+0000","last_change":"2024-04-25T10:06:29.908010+0000","last_active":"2024-04-25T10:06:29.908681+0000","last_peered":"2024-04-25T10:06:29.908681+0000","last_clean":"2024-04-25T10:06:29.908681+0000","last_became_active":"2024-04-25T10:06:29.907590+0000","last_became_peered":"2024-04-25T10:06:29.907590+0000","last_unstale":"2024-04-25T10:06:29.908681+0000","last_undegraded":"2024-04-25T10:06:29.908681+0000","last_fullsized":"2024-04-25T10:06:29.908681+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:06.688626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906095+0000","last_change":"2024-04-25T10:06:29.903383+0000","last_active":"2024-04-25T10:06:29.906095+0000","last_peered":"2024-04-25T10:06:29.906095+0000","last_clean":"2024-04-25T10:06:29.906095+0000","last_became_active":"2024-04-25T10:06:29.903100+0000","last_became_peered":"2024-04-25T10:06:29.903100+0000","last_unstale":"2024-04-25T10:06:29.906095+0000","last_undegraded":"2024-04-25T10:06:29.906095+0000","last_fullsized":"2024-04-25T10:06:29.906095+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:12.939064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906560+0000","last_change":"2024-04-25T10:06:29.904887+0000","last_active":"2024-04-25T10:06:29.906560+0000","last_peered":"2024-04-25T10:06:29.906560+0000","last_clean":"2024-04-25T10:06:29.906560+0000","last_became_active":"2024-04-25T10:06:29.904608+0000","last_became_peered":"2024-04-25T10:06:29.904608+0000","last_unstale":"2024-04-25T10:06:29.906560+0000","last_undegraded":"2024-04-25T10:06:29.906560+0000","last_fullsized":"2024-04-25T10:06:29.906560+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.575608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911287+0000","last_change":"2024-04-25T10:06:29.910365+0000","last_active":"2024-04-25T10:06:29.911287+0000","last_peered":"2024-04-25T10:06:29.911287+0000","last_clean":"2024-04-25T10:06:29.911287+0000","last_became_active":"2024-04-25T10:06:29.910049+0000","last_became_peered":"2024-04-25T10:06:29.910049+0000","last_unstale":"2024-04-25T10:06:29.911287+0000","last_undegraded":"2024-04-25T10:06:29.911287+0000","last_fullsized":"2024-04-25T10:06:29.911287+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:32.626510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904851+0000","last_change":"2024-04-25T10:06:29.903207+0000","last_active":"2024-04-25T10:06:29.904851+0000","last_peered":"2024-04-25T10:06:29.904851+0000","last_clean":"2024-04-25T10:06:29.904851+0000","last_became_active":"2024-04-25T10:06:29.902590+0000","last_became_peered":"2024-04-25T10:06:29.902590+0000","last_unstale":"2024-04-25T10:06:29.904851+0000","last_undegraded":"2024-04-25T10:06:29.904851+0000","last_fullsized":"2024-04-25T10:06:29.904851+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:46.937997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908683+0000","last_change":"2024-04-25T10:06:29.908122+0000","last_active":"2024-04-25T10:06:29.908683+0000","last_peered":"2024-04-25T10:06:29.908683+0000","last_clean":"2024-04-25T10:06:29.908683+0000","last_became_active":"2024-04-25T10:06:29.907837+0000","last_became_peered":"2024-04-25T10:06:29.907837+0000","last_unstale":"2024-04-25T10:06:29.908683+0000","last_undegraded":"2024-04-25T10:06:29.908683+0000","last_fullsized":"2024-04-25T10:06:29.908683+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:16.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907075+0000","last_change":"2024-04-25T10:06:29.904154+0000","last_active":"2024-04-25T10:06:29.907075+0000","last_peered":"2024-04-25T10:06:29.907075+0000","last_clean":"2024-04-25T10:06:29.907075+0000","last_became_active":"2024-04-25T10:06:29.903583+0000","last_became_peered":"2024-04-25T10:06:29.903583+0000","last_unstale":"2024-04-25T10:06:29.907075+0000","last_undegraded":"2024-04-25T10:06:29.907075+0000","last_fullsized":"2024-04-25T10:06:29.907075+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:49.636706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905867+0000","last_change":"2024-04-25T10:06:29.900566+0000","last_active":"2024-04-25T10:06:29.905867+0000","last_peered":"2024-04-25T10:06:29.905867+0000","last_clean":"2024-04-25T10:06:29.905867+0000","last_became_active":"2024-04-25T10:06:29.899568+0000","last_became_peered":"2024-04-25T10:06:29.899568+0000","last_unstale":"2024-04-25T10:06:29.905867+0000","last_undegraded":"2024-04-25T10:06:29.905867+0000","last_fullsized":"2024-04-25T10:06:29.905867+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:18.309566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.901426+0000","last_change":"2024-04-25T10:06:29.900824+0000","last_active":"2024-04-25T10:06:29.901426+0000","last_peered":"2024-04-25T10:06:29.901426+0000","last_clean":"2024-04-25T10:06:29.901426+0000","last_became_active":"2024-04-25T10:06:29.900487+0000","last_became_peered":"2024-04-25T10:06:29.900487+0000","last_unstale":"2024-04-25T10:06:29.901426+0000","last_undegraded":"2024-04-25T10:06:29.901426+0000","last_fullsized":"2024-04-25T10:06:29.901426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:08.144443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906251+0000","last_change":"2024-04-25T10:06:29.904617+0000","last_active":"2024-04-25T10:06:29.906251+0000","last_peered":"2024-04-25T10:06:29.906251+0000","last_clean":"2024-04-25T10:06:29.906251+0000","last_became_active":"2024-04-25T10:06:29.904357+0000","last_became_peered":"2024-04-25T10:06:29.904357+0000","last_unstale":"2024-04-25T10:06:29.906251+0000","last_undegraded":"2024-04-25T10:06:29.906251+0000","last_fullsized":"2024-04-25T10:06:29.906251+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:03.315521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906543+0000","last_change":"2024-04-25T10:06:29.900585+0000","last_active":"2024-04-25T10:06:29.906543+0000","last_peered":"2024-04-25T10:06:29.906543+0000","last_clean":"2024-04-25T10:06:29.906543+0000","last_became_active":"2024-04-25T10:06:29.900336+0000","last_became_peered":"2024-04-25T10:06:29.900336+0000","last_unstale":"2024-04-25T10:06:29.906543+0000","last_undegraded":"2024-04-25T10:06:29.906543+0000","last_fullsized":"2024-04-25T10:06:29.906543+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:48.479138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905710+0000","last_change":"2024-04-25T10:06:29.899827+0000","last_active":"2024-04-25T10:06:29.905710+0000","last_peered":"2024-04-25T10:06:29.905710+0000","last_clean":"2024-04-25T10:06:29.905710+0000","last_became_active":"2024-04-25T10:06:29.899551+0000","last_became_peered":"2024-04-25T10:06:29.899551+0000","last_unstale":"2024-04-25T10:06:29.905710+0000","last_undegraded":"2024-04-25T10:06:29.905710+0000","last_fullsized":"2024-04-25T10:06:29.905710+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:59.670309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911316+0000","last_change":"2024-04-25T10:06:29.910232+0000","last_active":"2024-04-25T10:06:29.911316+0000","last_peered":"2024-04-25T10:06:29.911316+0000","last_clean":"2024-04-25T10:06:29.911316+0000","last_became_active":"2024-04-25T10:06:29.909470+0000","last_became_peered":"2024-04-25T10:06:29.909470+0000","last_unstale":"2024-04-25T10:06:29.911316+0000","last_undegraded":"2024-04-25T10:06:29.911316+0000","last_fullsized":"2024-04-25T10:06:29.911316+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.436547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907393+0000","last_change":"2024-04-25T10:06:29.905088+0000","last_active":"2024-04-25T10:06:29.907393+0000","last_peered":"2024-04-25T10:06:29.907393+0000","last_clean":"2024-04-25T10:06:29.907393+0000","last_became_active":"2024-04-25T10:06:29.901371+0000","last_became_peered":"2024-04-25T10:06:29.901371+0000","last_unstale":"2024-04-25T10:06:29.907393+0000","last_undegraded":"2024-04-25T10:06:29.907393+0000","last_fullsized":"2024-04-25T10:06:29.907393+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:08.700635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911269+0000","last_change":"2024-04-25T10:06:29.910207+0000","last_active":"2024-04-25T10:06:29.911269+0000","last_peered":"2024-04-25T10:06:29.911269+0000","last_clean":"2024-04-25T10:06:29.911269+0000","last_became_active":"2024-04-25T10:06:29.909902+0000","last_became_peered":"2024-04-25T10:06:29.909902+0000","last_unstale":"2024-04-25T10:06:29.911269+0000","last_undegraded":"2024-04-25T10:06:29.911269+0000","last_fullsized":"2024-04-25T10:06:29.911269+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:26.044159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905889+0000","last_change":"2024-04-25T10:06:29.901500+0000","last_active":"2024-04-25T10:06:29.905889+0000","last_peered":"2024-04-25T10:06:29.905889+0000","last_clean":"2024-04-25T10:06:29.905889+0000","last_became_active":"2024-04-25T10:06:29.898953+0000","last_became_peered":"2024-04-25T10:06:29.898953+0000","last_unstale":"2024-04-25T10:06:29.905889+0000","last_undegraded":"2024-04-25T10:06:29.905889+0000","last_fullsized":"2024-04-25T10:06:29.905889+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:24.746581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908441+0000","last_change":"2024-04-25T10:06:29.907492+0000","last_active":"2024-04-25T10:06:29.908441+0000","last_peered":"2024-04-25T10:06:29.908441+0000","last_clean":"2024-04-25T10:06:29.908441+0000","last_became_active":"2024-04-25T10:06:29.907093+0000","last_became_peered":"2024-04-25T10:06:29.907093+0000","last_unstale":"2024-04-25T10:06:29.908441+0000","last_undegraded":"2024-04-25T10:06:29.908441+0000","last_fullsized":"2024-04-25T10:06:29.908441+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:49.843817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.903924+0000","last_change":"2024-04-25T10:06:29.903067+0000","last_active":"2024-04-25T10:06:29.903924+0000","last_peered":"2024-04-25T10:06:29.903924+0000","last_clean":"2024-04-25T10:06:29.903924+0000","last_became_active":"2024-04-25T10:06:29.902737+0000","last_became_peered":"2024-04-25T10:06:29.902737+0000","last_unstale":"2024-04-25T10:06:29.903924+0000","last_undegraded":"2024-04-25T10:06:29.903924+0000","last_fullsized":"2024-04-25T10:06:29.903924+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:13.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906524+0000","last_change":"2024-04-25T10:06:29.899141+0000","last_active":"2024-04-25T10:06:29.906524+0000","last_peered":"2024-04-25T10:06:29.906524+0000","last_clean":"2024-04-25T10:06:29.906524+0000","last_became_active":"2024-04-25T10:06:29.898886+0000","last_became_peered":"2024-04-25T10:06:29.898886+0000","last_unstale":"2024-04-25T10:06:29.906524+0000","last_undegraded":"2024-04-25T10:06:29.906524+0000","last_fullsized":"2024-04-25T10:06:29.906524+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:48.045511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906536+0000","last_change":"2024-04-25T10:06:29.905491+0000","last_active":"2024-04-25T10:06:29.906536+0000","last_peered":"2024-04-25T10:06:29.906536+0000","last_clean":"2024-04-25T10:06:29.906536+0000","last_became_active":"2024-04-25T10:06:29.904003+0000","last_became_peered":"2024-04-25T10:06:29.904003+0000","last_unstale":"2024-04-25T10:06:29.906536+0000","last_undegraded":"2024-04-25T10:06:29.906536+0000","last_fullsized":"2024-04-25T10:06:29.906536+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:11.179506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904881+0000","last_change":"2024-04-25T10:06:29.903462+0000","last_active":"2024-04-25T10:06:29.904881+0000","last_peered":"2024-04-25T10:06:29.904881+0000","last_clean":"2024-04-25T10:06:29.904881+0000","last_became_active":"2024-04-25T10:06:29.902933+0000","last_became_peered":"2024-04-25T10:06:29.902933+0000","last_unstale":"2024-04-25T10:06:29.904881+0000","last_undegraded":"2024-04-25T10:06:29.904881+0000","last_fullsized":"2024-04-25T10:06:29.904881+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:38.297145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902019+0000","last_change":"2024-04-25T10:06:29.900913+0000","last_active":"2024-04-25T10:06:29.902019+0000","last_peered":"2024-04-25T10:06:29.902019+0000","last_clean":"2024-04-25T10:06:29.902019+0000","last_became_active":"2024-04-25T10:06:29.900660+0000","last_became_peered":"2024-04-25T10:06:29.900660+0000","last_unstale":"2024-04-25T10:06:29.902019+0000","last_undegraded":"2024-04-25T10:06:29.902019+0000","last_fullsized":"2024-04-25T10:06:29.902019+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:52.340259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905914+0000","last_change":"2024-04-25T10:06:29.905326+0000","last_active":"2024-04-25T10:06:29.905914+0000","last_peered":"2024-04-25T10:06:29.905914+0000","last_clean":"2024-04-25T10:06:29.905914+0000","last_became_active":"2024-04-25T10:06:29.905049+0000","last_became_peered":"2024-04-25T10:06:29.905049+0000","last_unstale":"2024-04-25T10:06:29.905914+0000","last_undegraded":"2024-04-25T10:06:29.905914+0000","last_fullsized":"2024-04-25T10:06:29.905914+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:19.083421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.894524+0000","last_change":"2024-04-25T10:06:29.891210+0000","last_active":"2024-04-25T10:06:29.894524+0000","last_peered":"2024-04-25T10:06:29.894524+0000","last_clean":"2024-04-25T10:06:29.894524+0000","last_became_active":"2024-04-25T10:06:29.890803+0000","last_became_peered":"2024-04-25T10:06:29.890803+0000","last_unstale":"2024-04-25T10:06:29.894524+0000","last_undegraded":"2024-04-25T10:06:29.894524+0000","last_fullsized":"2024-04-25T10:06:29.894524+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:16.710451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902364+0000","last_change":"2024-04-25T10:06:29.901240+0000","last_active":"2024-04-25T10:06:29.902364+0000","last_peered":"2024-04-25T10:06:29.902364+0000","last_clean":"2024-04-25T10:06:29.902364+0000","last_became_active":"2024-04-25T10:06:29.900984+0000","last_became_peered":"2024-04-25T10:06:29.900984+0000","last_unstale":"2024-04-25T10:06:29.902364+0000","last_undegraded":"2024-04-25T10:06:29.902364+0000","last_fullsized":"2024-04-25T10:06:29.902364+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:00.058385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.898800+0000","last_change":"2024-04-25T10:06:29.896419+0000","last_active":"2024-04-25T10:06:29.898800+0000","last_peered":"2024-04-25T10:06:29.898800+0000","last_clean":"2024-04-25T10:06:29.898800+0000","last_became_active":"2024-04-25T10:06:29.896082+0000","last_became_peered":"2024-04-25T10:06:29.896082+0000","last_unstale":"2024-04-25T10:06:29.898800+0000","last_undegraded":"2024-04-25T10:06:29.898800+0000","last_fullsized":"2024-04-25T10:06:29.898800+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:52.353899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908448+0000","last_change":"2024-04-25T10:06:29.907189+0000","last_active":"2024-04-25T10:06:29.908448+0000","last_peered":"2024-04-25T10:06:29.908448+0000","last_clean":"2024-04-25T10:06:29.908448+0000","last_became_active":"2024-04-25T10:06:29.906898+0000","last_became_peered":"2024-04-25T10:06:29.906898+0000","last_unstale":"2024-04-25T10:06:29.908448+0000","last_undegraded":"2024-04-25T10:06:29.908448+0000","last_fullsized":"2024-04-25T10:06:29.908448+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:15.706576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"184'4","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.912187+0000","last_change":"2024-04-25T10:06:28.888525+0000","last_active":"2024-04-25T10:06:29.912187+0000","last_peered":"2024-04-25T10:06:29.912187+0000","last_clean":"2024-04-25T10:06:29.912187+0000","last_became_active":"2024-04-25T10:06:28.888237+0000","last_became_peered":"2024-04-25T10:06:28.888237+0000","last_unstale":"2024-04-25T10:06:29.912187+0000","last_undegraded":"2024-04-25T10:06:29.912187+0000","last_fullsized":"2024-04-25T10:06:29.912187+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:17.088230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.899947+0000","last_change":"2024-04-25T10:06:29.898953+0000","last_active":"2024-04-25T10:06:29.899947+0000","last_peered":"2024-04-25T10:06:29.899947+0000","last_clean":"2024-04-25T10:06:29.899947+0000","last_became_active":"2024-04-25T10:06:29.898335+0000","last_became_peered":"2024-04-25T10:06:29.898335+0000","last_unstale":"2024-04-25T10:06:29.899947+0000","last_undegraded":"2024-04-25T10:06:29.899947+0000","last_fullsized":"2024-04-25T10:06:29.899947+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:46.917935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916648+0000","last_change":"2024-04-25T10:06:28.892165+0000","last_active":"2024-04-25T10:06:29.916648+0000","last_peered":"2024-04-25T10:06:29.916648+0000","last_clean":"2024-04-25T10:06:29.916648+0000","last_became_active":"2024-04-25T10:06:28.891836+0000","last_became_peered":"2024-04-25T10:06:28.891836+0000","last_unstale":"2024-04-25T10:06:29.916648+0000","last_undegraded":"2024-04-25T10:06:29.916648+0000","last_fullsized":"2024-04-25T10:06:29.916648+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:34.767717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905843+0000","last_change":"2024-04-25T10:06:29.899873+0000","last_active":"2024-04-25T10:06:29.905843+0000","last_peered":"2024-04-25T10:06:29.905843+0000","last_clean":"2024-04-25T10:06:29.905843+0000","last_became_active":"2024-04-25T10:06:29.899275+0000","last_became_peered":"2024-04-25T10:06:29.899275+0000","last_unstale":"2024-04-25T10:06:29.905843+0000","last_undegraded":"2024-04-25T10:06:29.905843+0000","last_fullsized":"2024-04-25T10:06:29.905843+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:19.422039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.914230+0000","last_change":"2024-04-25T10:06:28.890044+0000","last_active":"2024-04-25T10:06:29.914230+0000","last_peered":"2024-04-25T10:06:29.914230+0000","last_clean":"2024-04-25T10:06:29.914230+0000","last_became_active":"2024-04-25T10:06:28.889782+0000","last_became_peered":"2024-04-25T10:06:28.889782+0000","last_unstale":"2024-04-25T10:06:29.914230+0000","last_undegraded":"2024-04-25T10:06:29.914230+0000","last_fullsized":"2024-04-25T10:06:29.914230+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:40.814630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902372+0000","last_change":"2024-04-25T10:06:29.900240+0000","last_active":"2024-04-25T10:06:29.902372+0000","last_peered":"2024-04-25T10:06:29.902372+0000","last_clean":"2024-04-25T10:06:29.902372+0000","last_became_active":"2024-04-25T10:06:29.899969+0000","last_became_peered":"2024-04-25T10:06:29.899969+0000","last_unstale":"2024-04-25T10:06:29.902372+0000","last_undegraded":"2024-04-25T10:06:29.902372+0000","last_fullsized":"2024-04-25T10:06:29.902372+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:29.295670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917156+0000","last_change":"2024-04-25T10:06:28.885031+0000","last_active":"2024-04-25T10:06:29.917156+0000","last_peered":"2024-04-25T10:06:29.917156+0000","last_clean":"2024-04-25T10:06:29.917156+0000","last_became_active":"2024-04-25T10:06:28.884778+0000","last_became_peered":"2024-04-25T10:06:28.884778+0000","last_unstale":"2024-04-25T10:06:29.917156+0000","last_undegraded":"2024-04-25T10:06:29.917156+0000","last_fullsized":"2024-04-25T10:06:29.917156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:37.165444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895515+0000","last_change":"2024-04-25T10:06:29.893390+0000","last_active":"2024-04-25T10:06:29.895515+0000","last_peered":"2024-04-25T10:06:29.895515+0000","last_clean":"2024-04-25T10:06:29.895515+0000","last_became_active":"2024-04-25T10:06:29.893012+0000","last_became_peered":"2024-04-25T10:06:29.893012+0000","last_unstale":"2024-04-25T10:06:29.895515+0000","last_undegraded":"2024-04-25T10:06:29.895515+0000","last_fullsized":"2024-04-25T10:06:29.895515+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:02.322399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911068+0000","last_change":"2024-04-25T10:06:28.896108+0000","last_active":"2024-04-25T10:06:29.911068+0000","last_peered":"2024-04-25T10:06:29.911068+0000","last_clean":"2024-04-25T10:06:29.911068+0000","last_became_active":"2024-04-25T10:06:28.895802+0000","last_became_peered":"2024-04-25T10:06:28.895802+0000","last_unstale":"2024-04-25T10:06:29.911068+0000","last_undegraded":"2024-04-25T10:06:29.911068+0000","last_fullsized":"2024-04-25T10:06:29.911068+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:08.769434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905208+0000","last_change":"2024-04-25T10:06:29.902969+0000","last_active":"2024-04-25T10:06:29.905208+0000","last_peered":"2024-04-25T10:06:29.905208+0000","last_clean":"2024-04-25T10:06:29.905208+0000","last_became_active":"2024-04-25T10:06:29.902686+0000","last_became_peered":"2024-04-25T10:06:29.902686+0000","last_unstale":"2024-04-25T10:06:29.905208+0000","last_undegraded":"2024-04-25T10:06:29.905208+0000","last_fullsized":"2024-04-25T10:06:29.905208+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:08.495304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889787+0000","last_change":"2024-04-25T10:06:28.895991+0000","last_active":"2024-04-25T10:06:29.889787+0000","last_peered":"2024-04-25T10:06:29.889787+0000","last_clean":"2024-04-25T10:06:29.889787+0000","last_became_active":"2024-04-25T10:06:28.895697+0000","last_became_peered":"2024-04-25T10:06:28.895697+0000","last_unstale":"2024-04-25T10:06:29.889787+0000","last_undegraded":"2024-04-25T10:06:29.889787+0000","last_fullsized":"2024-04-25T10:06:29.889787+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:04.446442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906503+0000","last_change":"2024-04-25T10:06:29.898010+0000","last_active":"2024-04-25T10:06:29.906503+0000","last_peered":"2024-04-25T10:06:29.906503+0000","last_clean":"2024-04-25T10:06:29.906503+0000","last_became_active":"2024-04-25T10:06:29.897707+0000","last_became_peered":"2024-04-25T10:06:29.897707+0000","last_unstale":"2024-04-25T10:06:29.906503+0000","last_undegraded":"2024-04-25T10:06:29.906503+0000","last_fullsized":"2024-04-25T10:06:29.906503+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.896851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889129+0000","last_change":"2024-04-25T10:06:28.885723+0000","last_active":"2024-04-25T10:06:29.889129+0000","last_peered":"2024-04-25T10:06:29.889129+0000","last_clean":"2024-04-25T10:06:29.889129+0000","last_became_active":"2024-04-25T10:06:28.885431+0000","last_became_peered":"2024-04-25T10:06:28.885431+0000","last_unstale":"2024-04-25T10:06:29.889129+0000","last_undegraded":"2024-04-25T10:06:29.889129+0000","last_fullsized":"2024-04-25T10:06:29.889129+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:52.298185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895091+0000","last_change":"2024-04-25T10:06:29.894166+0000","last_active":"2024-04-25T10:06:29.895091+0000","last_peered":"2024-04-25T10:06:29.895091+0000","last_clean":"2024-04-25T10:06:29.895091+0000","last_became_active":"2024-04-25T10:06:29.893894+0000","last_became_peered":"2024-04-25T10:06:29.893894+0000","last_unstale":"2024-04-25T10:06:29.895091+0000","last_undegraded":"2024-04-25T10:06:29.895091+0000","last_fullsized":"2024-04-25T10:06:29.895091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:22.503947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.899073+0000","last_change":"2024-04-25T10:06:28.884408+0000","last_active":"2024-04-25T10:06:29.899073+0000","last_peered":"2024-04-25T10:06:29.899073+0000","last_clean":"2024-04-25T10:06:29.899073+0000","last_became_active":"2024-04-25T10:06:28.884145+0000","last_became_peered":"2024-04-25T10:06:28.884145+0000","last_unstale":"2024-04-25T10:06:29.899073+0000","last_undegraded":"2024-04-25T10:06:29.899073+0000","last_fullsized":"2024-04-25T10:06:29.899073+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:33.175660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905250+0000","last_change":"2024-04-25T10:06:29.902182+0000","last_active":"2024-04-25T10:06:29.905250+0000","last_peered":"2024-04-25T10:06:29.905250+0000","last_clean":"2024-04-25T10:06:29.905250+0000","last_became_active":"2024-04-25T10:06:29.901911+0000","last_became_peered":"2024-04-25T10:06:29.901911+0000","last_unstale":"2024-04-25T10:06:29.905250+0000","last_undegraded":"2024-04-25T10:06:29.905250+0000","last_fullsized":"2024-04-25T10:06:29.905250+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:23.502409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.891065+0000","last_change":"2024-04-25T10:06:28.893876+0000","last_active":"2024-04-25T10:06:29.891065+0000","last_peered":"2024-04-25T10:06:29.891065+0000","last_clean":"2024-04-25T10:06:29.891065+0000","last_became_active":"2024-04-25T10:06:28.893544+0000","last_became_peered":"2024-04-25T10:06:28.893544+0000","last_unstale":"2024-04-25T10:06:29.891065+0000","last_undegraded":"2024-04-25T10:06:29.891065+0000","last_fullsized":"2024-04-25T10:06:29.891065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:47.591524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907110+0000","last_change":"2024-04-25T10:06:29.905315+0000","last_active":"2024-04-25T10:06:29.907110+0000","last_peered":"2024-04-25T10:06:29.907110+0000","last_clean":"2024-04-25T10:06:29.907110+0000","last_became_active":"2024-04-25T10:06:29.905046+0000","last_became_peered":"2024-04-25T10:06:29.905046+0000","last_unstale":"2024-04-25T10:06:29.907110+0000","last_undegraded":"2024-04-25T10:06:29.907110+0000","last_fullsized":"2024-04-25T10:06:29.907110+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:31.258172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"184'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917759+0000","last_change":"2024-04-25T10:06:28.885646+0000","last_active":"2024-04-25T10:06:29.917759+0000","last_peered":"2024-04-25T10:06:29.917759+0000","last_clean":"2024-04-25T10:06:29.917759+0000","last_became_active":"2024-04-25T10:06:28.884509+0000","last_became_peered":"2024-04-25T10:06:28.884509+0000","last_unstale":"2024-04-25T10:06:29.917759+0000","last_undegraded":"2024-04-25T10:06:29.917759+0000","last_fullsized":"2024-04-25T10:06:29.917759+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:14.936048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.903141+0000","last_change":"2024-04-25T10:06:29.902256+0000","last_active":"2024-04-25T10:06:29.903141+0000","last_peered":"2024-04-25T10:06:29.903141+0000","last_clean":"2024-04-25T10:06:29.903141+0000","last_became_active":"2024-04-25T10:06:29.901572+0000","last_became_peered":"2024-04-25T10:06:29.901572+0000","last_unstale":"2024-04-25T10:06:29.903141+0000","last_undegraded":"2024-04-25T10:06:29.903141+0000","last_fullsized":"2024-04-25T10:06:29.903141+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:08.529056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"184'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916797+0000","last_change":"2024-04-25T10:06:28.890561+0000","last_active":"2024-04-25T10:06:29.916797+0000","last_peered":"2024-04-25T10:06:29.916797+0000","last_clean":"2024-04-25T10:06:29.916797+0000","last_became_active":"2024-04-25T10:06:28.890258+0000","last_became_peered":"2024-04-25T10:06:28.890258+0000","last_unstale":"2024-04-25T10:06:29.916797+0000","last_undegraded":"2024-04-25T10:06:29.916797+0000","last_fullsized":"2024-04-25T10:06:29.916797+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:50.014361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907134+0000","last_change":"2024-04-25T10:06:29.902810+0000","last_active":"2024-04-25T10:06:29.907134+0000","last_peered":"2024-04-25T10:06:29.907134+0000","last_clean":"2024-04-25T10:06:29.907134+0000","last_became_active":"2024-04-25T10:06:29.902545+0000","last_became_peered":"2024-04-25T10:06:29.902545+0000","last_unstale":"2024-04-25T10:06:29.907134+0000","last_undegraded":"2024-04-25T10:06:29.907134+0000","last_fullsized":"2024-04-25T10:06:29.907134+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:59.868430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916719+0000","last_change":"2024-04-25T10:06:28.892677+0000","last_active":"2024-04-25T10:06:29.916719+0000","last_peered":"2024-04-25T10:06:29.916719+0000","last_clean":"2024-04-25T10:06:29.916719+0000","last_became_active":"2024-04-25T10:06:28.892403+0000","last_became_peered":"2024-04-25T10:06:28.892403+0000","last_unstale":"2024-04-25T10:06:29.916719+0000","last_undegraded":"2024-04-25T10:06:29.916719+0000","last_fullsized":"2024-04-25T10:06:29.916719+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:09.624872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.910041+0000","last_change":"2024-04-25T10:06:29.909528+0000","last_active":"2024-04-25T10:06:29.910041+0000","last_peered":"2024-04-25T10:06:29.910041+0000","last_clean":"2024-04-25T10:06:29.910041+0000","last_became_active":"2024-04-25T10:06:29.909227+0000","last_became_peered":"2024-04-25T10:06:29.909227+0000","last_unstale":"2024-04-25T10:06:29.910041+0000","last_undegraded":"2024-04-25T10:06:29.910041+0000","last_fullsized":"2024-04-25T10:06:29.910041+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:15.570957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889306+0000","last_change":"2024-04-25T10:06:28.887255+0000","last_active":"2024-04-25T10:06:29.889306+0000","last_peered":"2024-04-25T10:06:29.889306+0000","last_clean":"2024-04-25T10:06:29.889306+0000","last_became_active":"2024-04-25T10:06:28.886991+0000","last_became_peered":"2024-04-25T10:06:28.886991+0000","last_unstale":"2024-04-25T10:06:29.889306+0000","last_undegraded":"2024-04-25T10:06:29.889306+0000","last_fullsized":"2024-04-25T10:06:29.889306+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:58.016837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906068+0000","last_change":"2024-04-25T10:06:29.904415+0000","last_active":"2024-04-25T10:06:29.906068+0000","last_peered":"2024-04-25T10:06:29.906068+0000","last_clean":"2024-04-25T10:06:29.906068+0000","last_became_active":"2024-04-25T10:06:29.904003+0000","last_became_peered":"2024-04-25T10:06:29.904003+0000","last_unstale":"2024-04-25T10:06:29.906068+0000","last_undegraded":"2024-04-25T10:06:29.906068+0000","last_fullsized":"2024-04-25T10:06:29.906068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:07.772900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889712+0000","last_change":"2024-04-25T10:06:28.892751+0000","last_active":"2024-04-25T10:06:29.889712+0000","last_peered":"2024-04-25T10:06:29.889712+0000","last_clean":"2024-04-25T10:06:29.889712+0000","last_became_active":"2024-04-25T10:06:28.892461+0000","last_became_peered":"2024-04-25T10:06:28.892461+0000","last_unstale":"2024-04-25T10:06:29.889712+0000","last_undegraded":"2024-04-25T10:06:29.889712+0000","last_fullsized":"2024-04-25T10:06:29.889712+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:02.311028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907648+0000","last_change":"2024-04-25T10:06:29.905702+0000","last_active":"2024-04-25T10:06:29.907648+0000","last_peered":"2024-04-25T10:06:29.907648+0000","last_clean":"2024-04-25T10:06:29.907648+0000","last_became_active":"2024-04-25T10:06:29.905446+0000","last_became_peered":"2024-04-25T10:06:29.905446+0000","last_unstale":"2024-04-25T10:06:29.907648+0000","last_undegraded":"2024-04-25T10:06:29.907648+0000","last_fullsized":"2024-04-25T10:06:29.907648+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.860946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"184'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.914525+0000","last_change":"2024-04-25T10:06:28.885978+0000","last_active":"2024-04-25T10:06:29.914525+0000","last_peered":"2024-04-25T10:06:29.914525+0000","last_clean":"2024-04-25T10:06:29.914525+0000","last_became_active":"2024-04-25T10:06:28.885718+0000","last_became_peered":"2024-04-25T10:06:28.885718+0000","last_unstale":"2024-04-25T10:06:29.914525+0000","last_undegraded":"2024-04-25T10:06:29.914525+0000","last_fullsized":"2024-04-25T10:06:29.914525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:13.173776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911655+0000","last_change":"2024-04-25T10:06:29.909509+0000","last_active":"2024-04-25T10:06:29.911655+0000","last_peered":"2024-04-25T10:06:29.911655+0000","last_clean":"2024-04-25T10:06:29.911655+0000","last_became_active":"2024-04-25T10:06:29.909225+0000","last_became_peered":"2024-04-25T10:06:29.909225+0000","last_unstale":"2024-04-25T10:06:29.911655+0000","last_undegraded":"2024-04-25T10:06:29.911655+0000","last_fullsized":"2024-04-25T10:06:29.911655+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:38.720444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917662+0000","last_change":"2024-04-25T10:06:28.891608+0000","last_active":"2024-04-25T10:06:29.917662+0000","last_peered":"2024-04-25T10:06:29.917662+0000","last_clean":"2024-04-25T10:06:29.917662+0000","last_became_active":"2024-04-25T10:06:28.891297+0000","last_became_peered":"2024-04-25T10:06:28.891297+0000","last_unstale":"2024-04-25T10:06:29.917662+0000","last_undegraded":"2024-04-25T10:06:29.917662+0000","last_fullsized":"2024-04-25T10:06:29.917662+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.349406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904723+0000","last_change":"2024-04-25T10:06:29.903992+0000","last_active":"2024-04-25T10:06:29.904723+0000","last_peered":"2024-04-25T10:06:29.904723+0000","last_clean":"2024-04-25T10:06:29.904723+0000","last_became_active":"2024-04-25T10:06:29.903102+0000","last_became_peered":"2024-04-25T10:06:29.903102+0000","last_unstale":"2024-04-25T10:06:29.904723+0000","last_undegraded":"2024-04-25T10:06:29.904723+0000","last_fullsized":"2024-04-25T10:06:29.904723+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:53.832878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916807+0000","last_change":"2024-04-25T10:06:28.892890+0000","last_active":"2024-04-25T10:06:29.916807+0000","last_peered":"2024-04-25T10:06:29.916807+0000","last_clean":"2024-04-25T10:06:29.916807+0000","last_became_active":"2024-04-25T10:06:28.892549+0000","last_became_peered":"2024-04-25T10:06:28.892549+0000","last_unstale":"2024-04-25T10:06:29.916807+0000","last_undegraded":"2024-04-25T10:06:29.916807+0000","last_fullsized":"2024-04-25T10:06:29.916807+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:48.632213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.909283+0000","last_change":"2024-04-25T10:06:29.908290+0000","last_active":"2024-04-25T10:06:29.909283+0000","last_peered":"2024-04-25T10:06:29.909283+0000","last_clean":"2024-04-25T10:06:29.909283+0000","last_became_active":"2024-04-25T10:06:29.907977+0000","last_became_peered":"2024-04-25T10:06:29.907977+0000","last_unstale":"2024-04-25T10:06:29.909283+0000","last_undegraded":"2024-04-25T10:06:29.909283+0000","last_fullsized":"2024-04-25T10:06:29.909283+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:50.048658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.914579+0000","last_change":"2024-04-25T10:06:28.887441+0000","last_active":"2024-04-25T10:06:29.914579+0000","last_peered":"2024-04-25T10:06:29.914579+0000","last_clean":"2024-04-25T10:06:29.914579+0000","last_became_active":"2024-04-25T10:06:28.887170+0000","last_became_peered":"2024-04-25T10:06:28.887170+0000","last_unstale":"2024-04-25T10:06:29.914579+0000","last_undegraded":"2024-04-25T10:06:29.914579+0000","last_fullsized":"2024-04-25T10:06:29.914579+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:18.078928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905215+0000","last_change":"2024-04-25T10:06:29.901576+0000","last_active":"2024-04-25T10:06:29.905215+0000","last_peered":"2024-04-25T10:06:29.905215+0000","last_clean":"2024-04-25T10:06:29.905215+0000","last_became_active":"2024-04-25T10:06:29.901219+0000","last_became_peered":"2024-04-25T10:06:29.901219+0000","last_unstale":"2024-04-25T10:06:29.905215+0000","last_undegraded":"2024-04-25T10:06:29.905215+0000","last_fullsized":"2024-04-25T10:06:29.905215+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:29.082987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.898292+0000","last_change":"2024-04-25T10:06:28.890267+0000","last_active":"2024-04-25T10:06:29.898292+0000","last_peered":"2024-04-25T10:06:29.898292+0000","last_clean":"2024-04-25T10:06:29.898292+0000","last_became_active":"2024-04-25T10:06:28.889994+0000","last_became_peered":"2024-04-25T10:06:28.889994+0000","last_unstale":"2024-04-25T10:06:29.898292+0000","last_undegraded":"2024-04-25T10:06:29.898292+0000","last_fullsized":"2024-04-25T10:06:29.898292+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.883923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.899133+0000","last_change":"2024-04-25T10:06:29.898095+0000","last_active":"2024-04-25T10:06:29.899133+0000","last_peered":"2024-04-25T10:06:29.899133+0000","last_clean":"2024-04-25T10:06:29.899133+0000","last_became_active":"2024-04-25T10:06:29.897755+0000","last_became_peered":"2024-04-25T10:06:29.897755+0000","last_unstale":"2024-04-25T10:06:29.899133+0000","last_undegraded":"2024-04-25T10:06:29.899133+0000","last_fullsized":"2024-04-25T10:06:29.899133+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:38.161734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.913373+0000","last_change":"2024-04-25T10:06:28.883492+0000","last_active":"2024-04-25T10:06:29.913373+0000","last_peered":"2024-04-25T10:06:29.913373+0000","last_clean":"2024-04-25T10:06:29.913373+0000","last_became_active":"2024-04-25T10:06:28.882461+0000","last_became_peered":"2024-04-25T10:06:28.882461+0000","last_unstale":"2024-04-25T10:06:29.913373+0000","last_undegraded":"2024-04-25T10:06:29.913373+0000","last_fullsized":"2024-04-25T10:06:29.913373+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:25.053758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.901423+0000","last_change":"2024-04-25T10:06:29.900321+0000","last_active":"2024-04-25T10:06:29.901423+0000","last_peered":"2024-04-25T10:06:29.901423+0000","last_clean":"2024-04-25T10:06:29.901423+0000","last_became_active":"2024-04-25T10:06:29.900034+0000","last_became_peered":"2024-04-25T10:06:29.900034+0000","last_unstale":"2024-04-25T10:06:29.901423+0000","last_undegraded":"2024-04-25T10:06:29.901423+0000","last_fullsized":"2024-04-25T10:06:29.901423+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:07.376460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.886288+0000","last_change":"2024-04-25T10:06:28.889177+0000","last_active":"2024-04-25T10:06:29.886288+0000","last_peered":"2024-04-25T10:06:29.886288+0000","last_clean":"2024-04-25T10:06:29.886288+0000","last_became_active":"2024-04-25T10:06:28.888917+0000","last_became_peered":"2024-04-25T10:06:28.888917+0000","last_unstale":"2024-04-25T10:06:29.886288+0000","last_undegraded":"2024-04-25T10:06:29.886288+0000","last_fullsized":"2024-04-25T10:06:29.886288+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:30.253679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907152+0000","last_change":"2024-04-25T10:06:29.906188+0000","last_active":"2024-04-25T10:06:29.907152+0000","last_peered":"2024-04-25T10:06:29.907152+0000","last_clean":"2024-04-25T10:06:29.907152+0000","last_became_active":"2024-04-25T10:06:29.905880+0000","last_became_peered":"2024-04-25T10:06:29.905880+0000","last_unstale":"2024-04-25T10:06:29.907152+0000","last_undegraded":"2024-04-25T10:06:29.907152+0000","last_fullsized":"2024-04-25T10:06:29.907152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:41.819123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.887639+0000","last_change":"2024-04-25T10:06:28.886557+0000","last_active":"2024-04-25T10:06:29.887639+0000","last_peered":"2024-04-25T10:06:29.887639+0000","last_clean":"2024-04-25T10:06:29.887639+0000","last_became_active":"2024-04-25T10:06:28.885871+0000","last_became_peered":"2024-04-25T10:06:28.885871+0000","last_unstale":"2024-04-25T10:06:29.887639+0000","last_undegraded":"2024-04-25T10:06:29.887639+0000","last_fullsized":"2024-04-25T10:06:29.887639+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:27.333954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905232+0000","last_change":"2024-04-25T10:06:29.902725+0000","last_active":"2024-04-25T10:06:29.905232+0000","last_peered":"2024-04-25T10:06:29.905232+0000","last_clean":"2024-04-25T10:06:29.905232+0000","last_became_active":"2024-04-25T10:06:29.902449+0000","last_became_peered":"2024-04-25T10:06:29.902449+0000","last_unstale":"2024-04-25T10:06:29.905232+0000","last_undegraded":"2024-04-25T10:06:29.905232+0000","last_fullsized":"2024-04-25T10:06:29.905232+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:43.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895857+0000","last_change":"2024-04-25T10:06:28.895446+0000","last_active":"2024-04-25T10:06:29.895857+0000","last_peered":"2024-04-25T10:06:29.895857+0000","last_clean":"2024-04-25T10:06:29.895857+0000","last_became_active":"2024-04-25T10:06:28.895125+0000","last_became_peered":"2024-04-25T10:06:28.895125+0000","last_unstale":"2024-04-25T10:06:29.895857+0000","last_undegraded":"2024-04-25T10:06:29.895857+0000","last_fullsized":"2024-04-25T10:06:29.895857+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:06.371967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911635+0000","last_change":"2024-04-25T10:06:29.908847+0000","last_active":"2024-04-25T10:06:29.911635+0000","last_peered":"2024-04-25T10:06:29.911635+0000","last_clean":"2024-04-25T10:06:29.911635+0000","last_became_active":"2024-04-25T10:06:29.908562+0000","last_became_peered":"2024-04-25T10:06:29.908562+0000","last_unstale":"2024-04-25T10:06:29.911635+0000","last_undegraded":"2024-04-25T10:06:29.911635+0000","last_fullsized":"2024-04-25T10:06:29.911635+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.125042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916667+0000","last_change":"2024-04-25T10:06:28.888156+0000","last_active":"2024-04-25T10:06:29.916667+0000","last_peered":"2024-04-25T10:06:29.916667+0000","last_clean":"2024-04-25T10:06:29.916667+0000","last_became_active":"2024-04-25T10:06:28.887727+0000","last_became_peered":"2024-04-25T10:06:28.887727+0000","last_unstale":"2024-04-25T10:06:29.916667+0000","last_undegraded":"2024-04-25T10:06:29.916667+0000","last_fullsized":"2024-04-25T10:06:29.916667+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:14.566464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908764+0000","last_change":"2024-04-25T10:06:29.905125+0000","last_active":"2024-04-25T10:06:29.908764+0000","last_peered":"2024-04-25T10:06:29.908764+0000","last_clean":"2024-04-25T10:06:29.908764+0000","last_became_active":"2024-04-25T10:06:29.904863+0000","last_became_peered":"2024-04-25T10:06:29.904863+0000","last_unstale":"2024-04-25T10:06:29.908764+0000","last_undegraded":"2024-04-25T10:06:29.908764+0000","last_fullsized":"2024-04-25T10:06:29.908764+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:08.126372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.893302+0000","last_change":"2024-04-25T10:06:28.884059+0000","last_active":"2024-04-25T10:06:29.893302+0000","last_peered":"2024-04-25T10:06:29.893302+0000","last_clean":"2024-04-25T10:06:29.893302+0000","last_became_active":"2024-04-25T10:06:28.883785+0000","last_became_peered":"2024-04-25T10:06:28.883785+0000","last_unstale":"2024-04-25T10:06:29.893302+0000","last_undegraded":"2024-04-25T10:06:29.893302+0000","last_fullsized":"2024-04-25T10:06:29.893302+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:34.037040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905975+0000","last_change":"2024-04-25T10:06:29.904501+0000","last_active":"2024-04-25T10:06:29.905975+0000","last_peered":"2024-04-25T10:06:29.905975+0000","last_clean":"2024-04-25T10:06:29.905975+0000","last_became_active":"2024-04-25T10:06:29.903981+0000","last_became_peered":"2024-04-25T10:06:29.903981+0000","last_unstale":"2024-04-25T10:06:29.905975+0000","last_undegraded":"2024-04-25T10:06:29.905975+0000","last_fullsized":"2024-04-25T10:06:29.905975+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:10.629365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.887257+0000","last_change":"2024-04-25T10:06:28.885036+0000","last_active":"2024-04-25T10:06:29.887257+0000","last_peered":"2024-04-25T10:06:29.887257+0000","last_clean":"2024-04-25T10:06:29.887257+0000","last_became_active":"2024-04-25T10:06:28.884772+0000","last_became_peered":"2024-04-25T10:06:28.884772+0000","last_unstale":"2024-04-25T10:06:29.887257+0000","last_undegraded":"2024-04-25T10:06:29.887257+0000","last_fullsized":"2024-04-25T10:06:29.887257+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:00.182715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904198+0000","last_change":"2024-04-25T10:06:29.902079+0000","last_active":"2024-04-25T10:06:29.904198+0000","last_peered":"2024-04-25T10:06:29.904198+0000","last_clean":"2024-04-25T10:06:29.904198+0000","last_became_active":"2024-04-25T10:06:29.901814+0000","last_became_peered":"2024-04-25T10:06:29.901814+0000","last_unstale":"2024-04-25T10:06:29.904198+0000","last_undegraded":"2024-04-25T10:06:29.904198+0000","last_fullsized":"2024-04-25T10:06:29.904198+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.413449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.890483+0000","last_change":"2024-04-25T10:06:28.889751+0000","last_active":"2024-04-25T10:06:29.890483+0000","last_peered":"2024-04-25T10:06:29.890483+0000","last_clean":"2024-04-25T10:06:29.890483+0000","last_became_active":"2024-04-25T10:06:28.889448+0000","last_became_peered":"2024-04-25T10:06:28.889448+0000","last_unstale":"2024-04-25T10:06:29.890483+0000","last_undegraded":"2024-04-25T10:06:29.890483+0000","last_fullsized":"2024-04-25T10:06:29.890483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:50.014361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.910685+0000","last_change":"2024-04-25T10:06:29.907143+0000","last_active":"2024-04-25T10:06:29.910685+0000","last_peered":"2024-04-25T10:06:29.910685+0000","last_clean":"2024-04-25T10:06:29.910685+0000","last_became_active":"2024-04-25T10:06:29.905123+0000","last_became_peered":"2024-04-25T10:06:29.905123+0000","last_unstale":"2024-04-25T10:06:29.910685+0000","last_undegraded":"2024-04-25T10:06:29.910685+0000","last_fullsized":"2024-04-25T10:06:29.910685+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:28.854535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895604+0000","last_change":"2024-04-25T10:06:28.888454+0000","last_active":"2024-04-25T10:06:29.895604+0000","last_peered":"2024-04-25T10:06:29.895604+0000","last_clean":"2024-04-25T10:06:29.895604+0000","last_became_active":"2024-04-25T10:06:28.888118+0000","last_became_peered":"2024-04-25T10:06:28.888118+0000","last_unstale":"2024-04-25T10:06:29.895604+0000","last_undegraded":"2024-04-25T10:06:29.895604+0000","last_fullsized":"2024-04-25T10:06:29.895604+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.606339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904957+0000","last_change":"2024-04-25T10:06:29.903217+0000","last_active":"2024-04-25T10:06:29.904957+0000","last_peered":"2024-04-25T10:06:29.904957+0000","last_clean":"2024-04-25T10:06:29.904957+0000","last_became_active":"2024-04-25T10:06:29.902949+0000","last_became_peered":"2024-04-25T10:06:29.902949+0000","last_unstale":"2024-04-25T10:06:29.904957+0000","last_undegraded":"2024-04-25T10:06:29.904957+0000","last_fullsized":"2024-04-25T10:06:29.904957+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:35.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.890793+0000","last_change":"2024-04-25T10:06:28.889178+0000","last_active":"2024-04-25T10:06:29.890793+0000","last_peered":"2024-04-25T10:06:29.890793+0000","last_clean":"2024-04-25T10:06:29.890793+0000","last_became_active":"2024-04-25T10:06:28.888920+0000","last_became_peered":"2024-04-25T10:06:28.888920+0000","last_unstale":"2024-04-25T10:06:29.890793+0000","last_undegraded":"2024-04-25T10:06:29.890793+0000","last_fullsized":"2024-04-25T10:06:29.890793+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:31.192187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907068+0000","last_change":"2024-04-25T10:06:29.904373+0000","last_active":"2024-04-25T10:06:29.907068+0000","last_peered":"2024-04-25T10:06:29.907068+0000","last_clean":"2024-04-25T10:06:29.907068+0000","last_became_active":"2024-04-25T10:06:29.904071+0000","last_became_peered":"2024-04-25T10:06:29.904071+0000","last_unstale":"2024-04-25T10:06:29.907068+0000","last_undegraded":"2024-04-25T10:06:29.907068+0000","last_fullsized":"2024-04-25T10:06:29.907068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:46.917935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.896723+0000","last_change":"2024-04-25T10:06:28.887755+0000","last_active":"2024-04-25T10:06:29.896723+0000","last_peered":"2024-04-25T10:06:29.896723+0000","last_clean":"2024-04-25T10:06:29.896723+0000","last_became_active":"2024-04-25T10:06:28.887477+0000","last_became_peered":"2024-04-25T10:06:28.887477+0000","last_unstale":"2024-04-25T10:06:29.896723+0000","last_undegraded":"2024-04-25T10:06:29.896723+0000","last_fullsized":"2024-04-25T10:06:29.896723+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.053892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.909689+0000","last_change":"2024-04-25T10:06:29.905746+0000","last_active":"2024-04-25T10:06:29.909689+0000","last_peered":"2024-04-25T10:06:29.909689+0000","last_clean":"2024-04-25T10:06:29.909689+0000","last_became_active":"2024-04-25T10:06:29.905446+0000","last_became_peered":"2024-04-25T10:06:29.905446+0000","last_unstale":"2024-04-25T10:06:29.909689+0000","last_undegraded":"2024-04-25T10:06:29.909689+0000","last_fullsized":"2024-04-25T10:06:29.909689+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:32.196680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.896003+0000","last_change":"2024-04-25T10:06:28.885341+0000","last_active":"2024-04-25T10:06:29.896003+0000","last_peered":"2024-04-25T10:06:29.896003+0000","last_clean":"2024-04-25T10:06:29.896003+0000","last_became_active":"2024-04-25T10:06:28.885059+0000","last_became_peered":"2024-04-25T10:06:28.885059+0000","last_unstale":"2024-04-25T10:06:29.896003+0000","last_undegraded":"2024-04-25T10:06:29.896003+0000","last_fullsized":"2024-04-25T10:06:29.896003+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:24.050092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904260+0000","last_change":"2024-04-25T10:06:29.902683+0000","last_active":"2024-04-25T10:06:29.904260+0000","last_peered":"2024-04-25T10:06:29.904260+0000","last_clean":"2024-04-25T10:06:29.904260+0000","last_became_active":"2024-04-25T10:06:29.902406+0000","last_became_peered":"2024-04-25T10:06:29.902406+0000","last_unstale":"2024-04-25T10:06:29.904260+0000","last_undegraded":"2024-04-25T10:06:29.904260+0000","last_fullsized":"2024-04-25T10:06:29.904260+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.860946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917557+0000","last_change":"2024-04-25T10:06:28.885084+0000","last_active":"2024-04-25T10:06:29.917557+0000","last_peered":"2024-04-25T10:06:29.917557+0000","last_clean":"2024-04-25T10:06:29.917557+0000","last_became_active":"2024-04-25T10:06:28.884238+0000","last_became_peered":"2024-04-25T10:06:28.884238+0000","last_unstale":"2024-04-25T10:06:29.917557+0000","last_undegraded":"2024-04-25T10:06:29.917557+0000","last_fullsized":"2024-04-25T10:06:29.917557+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:07.121879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":473,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.890009+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:06:29.890009+0000","last_peered":"2024-04-25T10:06:29.890009+0000","last_clean":"2024-04-25T10:06:29.890009+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:06:29.890009+0000","last_undegraded":"2024-04-25T10:06:29.890009+0000","last_fullsized":"2024-04-25T10:06:29.890009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2904064,"data_stored":1672307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55644,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316196,"statfs":{"total":96636764160,"available":96579784704,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59808,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59808,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55648,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316192,"statfs":{"total":96636764160,"available":96579780608,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55648,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316192,"statfs":{"total":96636764160,"available":96579780608,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56228,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315612,"statfs":{"total":96636764160,"available":96579186688,"internally_reserved":0,"allocated":2904064,"data_stored":1672307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:32.856 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 Processor -- start 2024-04-25T10:06:32.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 -- start start 2024-04-25T10:06:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ffc15ba00 0x7f1ffc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc15f5f0 0x7f1ffc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.998 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc0cd710 con 0x7f1ffc1602e0 2024-04-25T10:06:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc0cd4b0 con 0x7f1ffc15ba00 2024-04-25T10:06:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.984+0000 7f2002b53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc099b70 con 0x7f1ffc15f5f0 2024-04-25T10:06:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20010f0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc15f5f0 0x7f1ffc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ffc15ba00 0x7f1ffc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ffc15ba00 0x7f1ffc09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:55272/0 (socket says 172.21.15.148:55272) 2024-04-25T10:06:32.999 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f1ffbfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1ffc1602e0 0x7f1ffc0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:60850/0 (socket says 172.21.15.148:60850) 2024-04-25T10:06:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 -- 172.21.15.148:0/3363994438 learned_addr learned my addr 172.21.15.148:0/3363994438 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f1ffb7fe640 1 -- 172.21.15.148:0/3363994438 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392752093 0 0) 0x7f1fe4001110 con 0x7f1ffc1602e0 2024-04-25T10:06:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f1ffb7fe640 1 -- 172.21.15.148:0/3363994438 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8003520 con 0x7f1ffc1602e0 2024-04-25T10:06:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 -- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc15f5f0 msgr2=0x7f1ffc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 --2- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc15f5f0 0x7f1ffc15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.000 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 -- 172.21.15.148:0/3363994438 >> v1:172.21.15.188:6789/0 conn(0x7f1ffc1602e0 legacy=0x7f1ffc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20008ef640 1 -- 172.21.15.148:0/3363994438 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc099e50 con 0x7f1ffc15ba00 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.985+0000 7f20010f0640 1 --2- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc15f5f0 0x7f1ffc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.986+0000 7f20008ef640 1 --2- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc09a930 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009e50 tx=0x7f1ff402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84703c0bb72d9137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.986+0000 7f1ffb7fe640 1 -- 172.21.15.148:0/3363994438 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4036070 con 0x7f1ffc15ba00 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.986+0000 7f1ffb7fe640 1 -- 172.21.15.148:0/3363994438 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff4031600 con 0x7f1ffc15ba00 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.986+0000 7f1ffb7fe640 1 -- 172.21.15.148:0/3363994438 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40319b0 con 0x7f1ffc15ba00 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 -- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15ba00 msgr2=0x7f1ffc09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 --2- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc09a930 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009e50 tx=0x7f1ff402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 -- 172.21.15.148:0/3363994438 shutdown_connections 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 --2- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc15f5f0 0x7f1ffc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 --2- 172.21.15.148:0/3363994438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc09a930 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 -- 172.21.15.148:0/3363994438 >> 172.21.15.148:0/3363994438 conn(0x7f1ffc0b3270 msgr2=0x7f1ffc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 -- 172.21.15.148:0/3363994438 shutdown_connections 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.987+0000 7f2002b53640 1 -- 172.21.15.148:0/3363994438 wait complete. 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.988+0000 7f2002b53640 1 Processor -- start 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.988+0000 7f2002b53640 1 -- start start 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f2002b53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f2002b53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15f5f0 0x7f1ffc1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f2002b53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc1602e0 0x7f1ffc1e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.002 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f2002b53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc15bde0 con 0x7f1ffc15ba00 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f2002b53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc09ae70 con 0x7f1ffc15f5f0 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f2002b53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc06cc90 con 0x7f1ffc1602e0 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40758/0 (socket says 172.21.15.148:40758) 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 -- 172.21.15.148:0/1307420728 learned_addr learned my addr 172.21.15.148:0/1307420728 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20010f0640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc1602e0 0x7f1ffc1e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f1ffbfff640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15f5f0 0x7f1ffc1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 -- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc1602e0 msgr2=0x7f1ffc1e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc1602e0 0x7f1ffc1e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 -- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15f5f0 msgr2=0x7f1ffc1f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15f5f0 0x7f1ffc1f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 -- 172.21.15.148:0/1307420728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc06c0f0 con 0x7f1ffc15ba00 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.989+0000 7f20008ef640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc14fe20 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009f70 tx=0x7f1ff4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.003 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.990+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4036050 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.990+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff4042070 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.990+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4009510 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.990+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc150360 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.990+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc06c640 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.990+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc073990 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.991+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff4007c50 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.991+0000 7f1ff97fa640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.992+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ff40ab040 con 0x7f1ffc15ba00 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.992+0000 7f1ffbfff640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.992+0000 7f1ffbfff640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b7130 tx=0x7f1fe4007b30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.005 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:32.996+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ffc150360 con 0x7f1ffc15ba00 2024-04-25T10:06:33.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.163+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ffc0b40a0 con 0x7f1fdc06b4d0 2024-04-25T10:06:33.173 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.168+0000 7f1ff97fa640 1 -- 172.21.15.148:0/1307420728 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252424 (secure 0 0 0) 0x7f1ffc0b40a0 con 0x7f1fdc06b4d0 2024-04-25T10:06:33.176 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:33.177 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:06:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.171+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fdc06b4d0 msgr2=0x7f1fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.171+0000 7f2002b53640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0b7130 tx=0x7f1fe4007b30 comp rx=0 tx=0).stop 2024-04-25T10:06:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.171+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 msgr2=0x7f1ffc14fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.171+0000 7f2002b53640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc14fe20 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009f70 tx=0x7f1ff4004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 shutdown_connections 2024-04-25T10:06:33.179 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ffc1602e0 0x7f1ffc1e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ffc15f5f0 0x7f1ffc1f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 --2- 172.21.15.148:0/1307420728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ffc15ba00 0x7f1ffc14fe20 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 >> 172.21.15.148:0/1307420728 conn(0x7f1ffc0b3270 msgr2=0x7f1ffc0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 shutdown_connections 2024-04-25T10:06:33.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.172+0000 7f2002b53640 1 -- 172.21.15.148:0/1307420728 wait complete. 2024-04-25T10:06:33.188 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2024-04-25T10:06:31.171769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":446340,"kb_used_data":19208,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754528380,"statfs":{"total":773094113280,"available":772637061120,"internally_reserved":0,"allocated":19668992,"data_stored":9836248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":437243440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000669"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905720+0000","last_change":"2024-04-25T10:06:29.903429+0000","last_active":"2024-04-25T10:06:29.905720+0000","last_peered":"2024-04-25T10:06:29.905720+0000","last_clean":"2024-04-25T10:06:29.905720+0000","last_became_active":"2024-04-25T10:06:29.903123+0000","last_became_peered":"2024-04-25T10:06:29.903123+0000","last_unstale":"2024-04-25T10:06:29.905720+0000","last_undegraded":"2024-04-25T10:06:29.905720+0000","last_fullsized":"2024-04-25T10:06:29.905720+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.824563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.900098+0000","last_change":"2024-04-25T10:06:29.899227+0000","last_active":"2024-04-25T10:06:29.900098+0000","last_peered":"2024-04-25T10:06:29.900098+0000","last_clean":"2024-04-25T10:06:29.900098+0000","last_became_active":"2024-04-25T10:06:29.898960+0000","last_became_peered":"2024-04-25T10:06:29.898960+0000","last_unstale":"2024-04-25T10:06:29.900098+0000","last_undegraded":"2024-04-25T10:06:29.900098+0000","last_fullsized":"2024-04-25T10:06:29.900098+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:39.851479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904899+0000","last_change":"2024-04-25T10:06:29.904040+0000","last_active":"2024-04-25T10:06:29.904899+0000","last_peered":"2024-04-25T10:06:29.904899+0000","last_clean":"2024-04-25T10:06:29.904899+0000","last_became_active":"2024-04-25T10:06:29.903792+0000","last_became_peered":"2024-04-25T10:06:29.903792+0000","last_unstale":"2024-04-25T10:06:29.904899+0000","last_undegraded":"2024-04-25T10:06:29.904899+0000","last_fullsized":"2024-04-25T10:06:29.904899+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:03.129065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907686+0000","last_change":"2024-04-25T10:06:29.906906+0000","last_active":"2024-04-25T10:06:29.907686+0000","last_peered":"2024-04-25T10:06:29.907686+0000","last_clean":"2024-04-25T10:06:29.907686+0000","last_became_active":"2024-04-25T10:06:29.906617+0000","last_became_peered":"2024-04-25T10:06:29.906617+0000","last_unstale":"2024-04-25T10:06:29.907686+0000","last_undegraded":"2024-04-25T10:06:29.907686+0000","last_fullsized":"2024-04-25T10:06:29.907686+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.824563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902352+0000","last_change":"2024-04-25T10:06:29.899792+0000","last_active":"2024-04-25T10:06:29.902352+0000","last_peered":"2024-04-25T10:06:29.902352+0000","last_clean":"2024-04-25T10:06:29.902352+0000","last_became_active":"2024-04-25T10:06:29.899476+0000","last_became_peered":"2024-04-25T10:06:29.899476+0000","last_unstale":"2024-04-25T10:06:29.902352+0000","last_undegraded":"2024-04-25T10:06:29.902352+0000","last_fullsized":"2024-04-25T10:06:29.902352+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:22.774713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907670+0000","last_change":"2024-04-25T10:06:29.904690+0000","last_active":"2024-04-25T10:06:29.907670+0000","last_peered":"2024-04-25T10:06:29.907670+0000","last_clean":"2024-04-25T10:06:29.907670+0000","last_became_active":"2024-04-25T10:06:29.904428+0000","last_became_peered":"2024-04-25T10:06:29.904428+0000","last_unstale":"2024-04-25T10:06:29.907670+0000","last_undegraded":"2024-04-25T10:06:29.907670+0000","last_fullsized":"2024-04-25T10:06:29.907670+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.413449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908681+0000","last_change":"2024-04-25T10:06:29.908010+0000","last_active":"2024-04-25T10:06:29.908681+0000","last_peered":"2024-04-25T10:06:29.908681+0000","last_clean":"2024-04-25T10:06:29.908681+0000","last_became_active":"2024-04-25T10:06:29.907590+0000","last_became_peered":"2024-04-25T10:06:29.907590+0000","last_unstale":"2024-04-25T10:06:29.908681+0000","last_undegraded":"2024-04-25T10:06:29.908681+0000","last_fullsized":"2024-04-25T10:06:29.908681+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:06.688626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906095+0000","last_change":"2024-04-25T10:06:29.903383+0000","last_active":"2024-04-25T10:06:29.906095+0000","last_peered":"2024-04-25T10:06:29.906095+0000","last_clean":"2024-04-25T10:06:29.906095+0000","last_became_active":"2024-04-25T10:06:29.903100+0000","last_became_peered":"2024-04-25T10:06:29.903100+0000","last_unstale":"2024-04-25T10:06:29.906095+0000","last_undegraded":"2024-04-25T10:06:29.906095+0000","last_fullsized":"2024-04-25T10:06:29.906095+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:12.939064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906560+0000","last_change":"2024-04-25T10:06:29.904887+0000","last_active":"2024-04-25T10:06:29.906560+0000","last_peered":"2024-04-25T10:06:29.906560+0000","last_clean":"2024-04-25T10:06:29.906560+0000","last_became_active":"2024-04-25T10:06:29.904608+0000","last_became_peered":"2024-04-25T10:06:29.904608+0000","last_unstale":"2024-04-25T10:06:29.906560+0000","last_undegraded":"2024-04-25T10:06:29.906560+0000","last_fullsized":"2024-04-25T10:06:29.906560+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.575608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911287+0000","last_change":"2024-04-25T10:06:29.910365+0000","last_active":"2024-04-25T10:06:29.911287+0000","last_peered":"2024-04-25T10:06:29.911287+0000","last_clean":"2024-04-25T10:06:29.911287+0000","last_became_active":"2024-04-25T10:06:29.910049+0000","last_became_peered":"2024-04-25T10:06:29.910049+0000","last_unstale":"2024-04-25T10:06:29.911287+0000","last_undegraded":"2024-04-25T10:06:29.911287+0000","last_fullsized":"2024-04-25T10:06:29.911287+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:32.626510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904851+0000","last_change":"2024-04-25T10:06:29.903207+0000","last_active":"2024-04-25T10:06:29.904851+0000","last_peered":"2024-04-25T10:06:29.904851+0000","last_clean":"2024-04-25T10:06:29.904851+0000","last_became_active":"2024-04-25T10:06:29.902590+0000","last_became_peered":"2024-04-25T10:06:29.902590+0000","last_unstale":"2024-04-25T10:06:29.904851+0000","last_undegraded":"2024-04-25T10:06:29.904851+0000","last_fullsized":"2024-04-25T10:06:29.904851+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:46.937997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908683+0000","last_change":"2024-04-25T10:06:29.908122+0000","last_active":"2024-04-25T10:06:29.908683+0000","last_peered":"2024-04-25T10:06:29.908683+0000","last_clean":"2024-04-25T10:06:29.908683+0000","last_became_active":"2024-04-25T10:06:29.907837+0000","last_became_peered":"2024-04-25T10:06:29.907837+0000","last_unstale":"2024-04-25T10:06:29.908683+0000","last_undegraded":"2024-04-25T10:06:29.908683+0000","last_fullsized":"2024-04-25T10:06:29.908683+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:16.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907075+0000","last_change":"2024-04-25T10:06:29.904154+0000","last_active":"2024-04-25T10:06:29.907075+0000","last_peered":"2024-04-25T10:06:29.907075+0000","last_clean":"2024-04-25T10:06:29.907075+0000","last_became_active":"2024-04-25T10:06:29.903583+0000","last_became_peered":"2024-04-25T10:06:29.903583+0000","last_unstale":"2024-04-25T10:06:29.907075+0000","last_undegraded":"2024-04-25T10:06:29.907075+0000","last_fullsized":"2024-04-25T10:06:29.907075+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:49.636706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905867+0000","last_change":"2024-04-25T10:06:29.900566+0000","last_active":"2024-04-25T10:06:29.905867+0000","last_peered":"2024-04-25T10:06:29.905867+0000","last_clean":"2024-04-25T10:06:29.905867+0000","last_became_active":"2024-04-25T10:06:29.899568+0000","last_became_peered":"2024-04-25T10:06:29.899568+0000","last_unstale":"2024-04-25T10:06:29.905867+0000","last_undegraded":"2024-04-25T10:06:29.905867+0000","last_fullsized":"2024-04-25T10:06:29.905867+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:18.309566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.901426+0000","last_change":"2024-04-25T10:06:29.900824+0000","last_active":"2024-04-25T10:06:29.901426+0000","last_peered":"2024-04-25T10:06:29.901426+0000","last_clean":"2024-04-25T10:06:29.901426+0000","last_became_active":"2024-04-25T10:06:29.900487+0000","last_became_peered":"2024-04-25T10:06:29.900487+0000","last_unstale":"2024-04-25T10:06:29.901426+0000","last_undegraded":"2024-04-25T10:06:29.901426+0000","last_fullsized":"2024-04-25T10:06:29.901426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:08.144443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906251+0000","last_change":"2024-04-25T10:06:29.904617+0000","last_active":"2024-04-25T10:06:29.906251+0000","last_peered":"2024-04-25T10:06:29.906251+0000","last_clean":"2024-04-25T10:06:29.906251+0000","last_became_active":"2024-04-25T10:06:29.904357+0000","last_became_peered":"2024-04-25T10:06:29.904357+0000","last_unstale":"2024-04-25T10:06:29.906251+0000","last_undegraded":"2024-04-25T10:06:29.906251+0000","last_fullsized":"2024-04-25T10:06:29.906251+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:03.315521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906543+0000","last_change":"2024-04-25T10:06:29.900585+0000","last_active":"2024-04-25T10:06:29.906543+0000","last_peered":"2024-04-25T10:06:29.906543+0000","last_clean":"2024-04-25T10:06:29.906543+0000","last_became_active":"2024-04-25T10:06:29.900336+0000","last_became_peered":"2024-04-25T10:06:29.900336+0000","last_unstale":"2024-04-25T10:06:29.906543+0000","last_undegraded":"2024-04-25T10:06:29.906543+0000","last_fullsized":"2024-04-25T10:06:29.906543+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:48.479138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905710+0000","last_change":"2024-04-25T10:06:29.899827+0000","last_active":"2024-04-25T10:06:29.905710+0000","last_peered":"2024-04-25T10:06:29.905710+0000","last_clean":"2024-04-25T10:06:29.905710+0000","last_became_active":"2024-04-25T10:06:29.899551+0000","last_became_peered":"2024-04-25T10:06:29.899551+0000","last_unstale":"2024-04-25T10:06:29.905710+0000","last_undegraded":"2024-04-25T10:06:29.905710+0000","last_fullsized":"2024-04-25T10:06:29.905710+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:59.670309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911316+0000","last_change":"2024-04-25T10:06:29.910232+0000","last_active":"2024-04-25T10:06:29.911316+0000","last_peered":"2024-04-25T10:06:29.911316+0000","last_clean":"2024-04-25T10:06:29.911316+0000","last_became_active":"2024-04-25T10:06:29.909470+0000","last_became_peered":"2024-04-25T10:06:29.909470+0000","last_unstale":"2024-04-25T10:06:29.911316+0000","last_undegraded":"2024-04-25T10:06:29.911316+0000","last_fullsized":"2024-04-25T10:06:29.911316+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:14.436547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907393+0000","last_change":"2024-04-25T10:06:29.905088+0000","last_active":"2024-04-25T10:06:29.907393+0000","last_peered":"2024-04-25T10:06:29.907393+0000","last_clean":"2024-04-25T10:06:29.907393+0000","last_became_active":"2024-04-25T10:06:29.901371+0000","last_became_peered":"2024-04-25T10:06:29.901371+0000","last_unstale":"2024-04-25T10:06:29.907393+0000","last_undegraded":"2024-04-25T10:06:29.907393+0000","last_fullsized":"2024-04-25T10:06:29.907393+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:08.700635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911269+0000","last_change":"2024-04-25T10:06:29.910207+0000","last_active":"2024-04-25T10:06:29.911269+0000","last_peered":"2024-04-25T10:06:29.911269+0000","last_clean":"2024-04-25T10:06:29.911269+0000","last_became_active":"2024-04-25T10:06:29.909902+0000","last_became_peered":"2024-04-25T10:06:29.909902+0000","last_unstale":"2024-04-25T10:06:29.911269+0000","last_undegraded":"2024-04-25T10:06:29.911269+0000","last_fullsized":"2024-04-25T10:06:29.911269+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:26.044159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905889+0000","last_change":"2024-04-25T10:06:29.901500+0000","last_active":"2024-04-25T10:06:29.905889+0000","last_peered":"2024-04-25T10:06:29.905889+0000","last_clean":"2024-04-25T10:06:29.905889+0000","last_became_active":"2024-04-25T10:06:29.898953+0000","last_became_peered":"2024-04-25T10:06:29.898953+0000","last_unstale":"2024-04-25T10:06:29.905889+0000","last_undegraded":"2024-04-25T10:06:29.905889+0000","last_fullsized":"2024-04-25T10:06:29.905889+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:24.746581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908441+0000","last_change":"2024-04-25T10:06:29.907492+0000","last_active":"2024-04-25T10:06:29.908441+0000","last_peered":"2024-04-25T10:06:29.908441+0000","last_clean":"2024-04-25T10:06:29.908441+0000","last_became_active":"2024-04-25T10:06:29.907093+0000","last_became_peered":"2024-04-25T10:06:29.907093+0000","last_unstale":"2024-04-25T10:06:29.908441+0000","last_undegraded":"2024-04-25T10:06:29.908441+0000","last_fullsized":"2024-04-25T10:06:29.908441+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:49.843817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.903924+0000","last_change":"2024-04-25T10:06:29.903067+0000","last_active":"2024-04-25T10:06:29.903924+0000","last_peered":"2024-04-25T10:06:29.903924+0000","last_clean":"2024-04-25T10:06:29.903924+0000","last_became_active":"2024-04-25T10:06:29.902737+0000","last_became_peered":"2024-04-25T10:06:29.902737+0000","last_unstale":"2024-04-25T10:06:29.903924+0000","last_undegraded":"2024-04-25T10:06:29.903924+0000","last_fullsized":"2024-04-25T10:06:29.903924+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:13.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906524+0000","last_change":"2024-04-25T10:06:29.899141+0000","last_active":"2024-04-25T10:06:29.906524+0000","last_peered":"2024-04-25T10:06:29.906524+0000","last_clean":"2024-04-25T10:06:29.906524+0000","last_became_active":"2024-04-25T10:06:29.898886+0000","last_became_peered":"2024-04-25T10:06:29.898886+0000","last_unstale":"2024-04-25T10:06:29.906524+0000","last_undegraded":"2024-04-25T10:06:29.906524+0000","last_fullsized":"2024-04-25T10:06:29.906524+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:48.045511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906536+0000","last_change":"2024-04-25T10:06:29.905491+0000","last_active":"2024-04-25T10:06:29.906536+0000","last_peered":"2024-04-25T10:06:29.906536+0000","last_clean":"2024-04-25T10:06:29.906536+0000","last_became_active":"2024-04-25T10:06:29.904003+0000","last_became_peered":"2024-04-25T10:06:29.904003+0000","last_unstale":"2024-04-25T10:06:29.906536+0000","last_undegraded":"2024-04-25T10:06:29.906536+0000","last_fullsized":"2024-04-25T10:06:29.906536+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:11.179506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904881+0000","last_change":"2024-04-25T10:06:29.903462+0000","last_active":"2024-04-25T10:06:29.904881+0000","last_peered":"2024-04-25T10:06:29.904881+0000","last_clean":"2024-04-25T10:06:29.904881+0000","last_became_active":"2024-04-25T10:06:29.902933+0000","last_became_peered":"2024-04-25T10:06:29.902933+0000","last_unstale":"2024-04-25T10:06:29.904881+0000","last_undegraded":"2024-04-25T10:06:29.904881+0000","last_fullsized":"2024-04-25T10:06:29.904881+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:38.297145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902019+0000","last_change":"2024-04-25T10:06:29.900913+0000","last_active":"2024-04-25T10:06:29.902019+0000","last_peered":"2024-04-25T10:06:29.902019+0000","last_clean":"2024-04-25T10:06:29.902019+0000","last_became_active":"2024-04-25T10:06:29.900660+0000","last_became_peered":"2024-04-25T10:06:29.900660+0000","last_unstale":"2024-04-25T10:06:29.902019+0000","last_undegraded":"2024-04-25T10:06:29.902019+0000","last_fullsized":"2024-04-25T10:06:29.902019+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:52.340259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905914+0000","last_change":"2024-04-25T10:06:29.905326+0000","last_active":"2024-04-25T10:06:29.905914+0000","last_peered":"2024-04-25T10:06:29.905914+0000","last_clean":"2024-04-25T10:06:29.905914+0000","last_became_active":"2024-04-25T10:06:29.905049+0000","last_became_peered":"2024-04-25T10:06:29.905049+0000","last_unstale":"2024-04-25T10:06:29.905914+0000","last_undegraded":"2024-04-25T10:06:29.905914+0000","last_fullsized":"2024-04-25T10:06:29.905914+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:19.083421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.894524+0000","last_change":"2024-04-25T10:06:29.891210+0000","last_active":"2024-04-25T10:06:29.894524+0000","last_peered":"2024-04-25T10:06:29.894524+0000","last_clean":"2024-04-25T10:06:29.894524+0000","last_became_active":"2024-04-25T10:06:29.890803+0000","last_became_peered":"2024-04-25T10:06:29.890803+0000","last_unstale":"2024-04-25T10:06:29.894524+0000","last_undegraded":"2024-04-25T10:06:29.894524+0000","last_fullsized":"2024-04-25T10:06:29.894524+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:16.710451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902364+0000","last_change":"2024-04-25T10:06:29.901240+0000","last_active":"2024-04-25T10:06:29.902364+0000","last_peered":"2024-04-25T10:06:29.902364+0000","last_clean":"2024-04-25T10:06:29.902364+0000","last_became_active":"2024-04-25T10:06:29.900984+0000","last_became_peered":"2024-04-25T10:06:29.900984+0000","last_unstale":"2024-04-25T10:06:29.902364+0000","last_undegraded":"2024-04-25T10:06:29.902364+0000","last_fullsized":"2024-04-25T10:06:29.902364+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:00.058385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.898800+0000","last_change":"2024-04-25T10:06:29.896419+0000","last_active":"2024-04-25T10:06:29.898800+0000","last_peered":"2024-04-25T10:06:29.898800+0000","last_clean":"2024-04-25T10:06:29.898800+0000","last_became_active":"2024-04-25T10:06:29.896082+0000","last_became_peered":"2024-04-25T10:06:29.896082+0000","last_unstale":"2024-04-25T10:06:29.898800+0000","last_undegraded":"2024-04-25T10:06:29.898800+0000","last_fullsized":"2024-04-25T10:06:29.898800+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:52.353899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908448+0000","last_change":"2024-04-25T10:06:29.907189+0000","last_active":"2024-04-25T10:06:29.908448+0000","last_peered":"2024-04-25T10:06:29.908448+0000","last_clean":"2024-04-25T10:06:29.908448+0000","last_became_active":"2024-04-25T10:06:29.906898+0000","last_became_peered":"2024-04-25T10:06:29.906898+0000","last_unstale":"2024-04-25T10:06:29.908448+0000","last_undegraded":"2024-04-25T10:06:29.908448+0000","last_fullsized":"2024-04-25T10:06:29.908448+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:15.706576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"184'4","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.912187+0000","last_change":"2024-04-25T10:06:28.888525+0000","last_active":"2024-04-25T10:06:29.912187+0000","last_peered":"2024-04-25T10:06:29.912187+0000","last_clean":"2024-04-25T10:06:29.912187+0000","last_became_active":"2024-04-25T10:06:28.888237+0000","last_became_peered":"2024-04-25T10:06:28.888237+0000","last_unstale":"2024-04-25T10:06:29.912187+0000","last_undegraded":"2024-04-25T10:06:29.912187+0000","last_fullsized":"2024-04-25T10:06:29.912187+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:17.088230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.899947+0000","last_change":"2024-04-25T10:06:29.898953+0000","last_active":"2024-04-25T10:06:29.899947+0000","last_peered":"2024-04-25T10:06:29.899947+0000","last_clean":"2024-04-25T10:06:29.899947+0000","last_became_active":"2024-04-25T10:06:29.898335+0000","last_became_peered":"2024-04-25T10:06:29.898335+0000","last_unstale":"2024-04-25T10:06:29.899947+0000","last_undegraded":"2024-04-25T10:06:29.899947+0000","last_fullsized":"2024-04-25T10:06:29.899947+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:46.917935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916648+0000","last_change":"2024-04-25T10:06:28.892165+0000","last_active":"2024-04-25T10:06:29.916648+0000","last_peered":"2024-04-25T10:06:29.916648+0000","last_clean":"2024-04-25T10:06:29.916648+0000","last_became_active":"2024-04-25T10:06:28.891836+0000","last_became_peered":"2024-04-25T10:06:28.891836+0000","last_unstale":"2024-04-25T10:06:29.916648+0000","last_undegraded":"2024-04-25T10:06:29.916648+0000","last_fullsized":"2024-04-25T10:06:29.916648+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:34.767717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905843+0000","last_change":"2024-04-25T10:06:29.899873+0000","last_active":"2024-04-25T10:06:29.905843+0000","last_peered":"2024-04-25T10:06:29.905843+0000","last_clean":"2024-04-25T10:06:29.905843+0000","last_became_active":"2024-04-25T10:06:29.899275+0000","last_became_peered":"2024-04-25T10:06:29.899275+0000","last_unstale":"2024-04-25T10:06:29.905843+0000","last_undegraded":"2024-04-25T10:06:29.905843+0000","last_fullsized":"2024-04-25T10:06:29.905843+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:19.422039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.914230+0000","last_change":"2024-04-25T10:06:28.890044+0000","last_active":"2024-04-25T10:06:29.914230+0000","last_peered":"2024-04-25T10:06:29.914230+0000","last_clean":"2024-04-25T10:06:29.914230+0000","last_became_active":"2024-04-25T10:06:28.889782+0000","last_became_peered":"2024-04-25T10:06:28.889782+0000","last_unstale":"2024-04-25T10:06:29.914230+0000","last_undegraded":"2024-04-25T10:06:29.914230+0000","last_fullsized":"2024-04-25T10:06:29.914230+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:40.814630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.902372+0000","last_change":"2024-04-25T10:06:29.900240+0000","last_active":"2024-04-25T10:06:29.902372+0000","last_peered":"2024-04-25T10:06:29.902372+0000","last_clean":"2024-04-25T10:06:29.902372+0000","last_became_active":"2024-04-25T10:06:29.899969+0000","last_became_peered":"2024-04-25T10:06:29.899969+0000","last_unstale":"2024-04-25T10:06:29.902372+0000","last_undegraded":"2024-04-25T10:06:29.902372+0000","last_fullsized":"2024-04-25T10:06:29.902372+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:29.295670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917156+0000","last_change":"2024-04-25T10:06:28.885031+0000","last_active":"2024-04-25T10:06:29.917156+0000","last_peered":"2024-04-25T10:06:29.917156+0000","last_clean":"2024-04-25T10:06:29.917156+0000","last_became_active":"2024-04-25T10:06:28.884778+0000","last_became_peered":"2024-04-25T10:06:28.884778+0000","last_unstale":"2024-04-25T10:06:29.917156+0000","last_undegraded":"2024-04-25T10:06:29.917156+0000","last_fullsized":"2024-04-25T10:06:29.917156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:37.165444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895515+0000","last_change":"2024-04-25T10:06:29.893390+0000","last_active":"2024-04-25T10:06:29.895515+0000","last_peered":"2024-04-25T10:06:29.895515+0000","last_clean":"2024-04-25T10:06:29.895515+0000","last_became_active":"2024-04-25T10:06:29.893012+0000","last_became_peered":"2024-04-25T10:06:29.893012+0000","last_unstale":"2024-04-25T10:06:29.895515+0000","last_undegraded":"2024-04-25T10:06:29.895515+0000","last_fullsized":"2024-04-25T10:06:29.895515+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:02.322399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911068+0000","last_change":"2024-04-25T10:06:28.896108+0000","last_active":"2024-04-25T10:06:29.911068+0000","last_peered":"2024-04-25T10:06:29.911068+0000","last_clean":"2024-04-25T10:06:29.911068+0000","last_became_active":"2024-04-25T10:06:28.895802+0000","last_became_peered":"2024-04-25T10:06:28.895802+0000","last_unstale":"2024-04-25T10:06:29.911068+0000","last_undegraded":"2024-04-25T10:06:29.911068+0000","last_fullsized":"2024-04-25T10:06:29.911068+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:08.769434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905208+0000","last_change":"2024-04-25T10:06:29.902969+0000","last_active":"2024-04-25T10:06:29.905208+0000","last_peered":"2024-04-25T10:06:29.905208+0000","last_clean":"2024-04-25T10:06:29.905208+0000","last_became_active":"2024-04-25T10:06:29.902686+0000","last_became_peered":"2024-04-25T10:06:29.902686+0000","last_unstale":"2024-04-25T10:06:29.905208+0000","last_undegraded":"2024-04-25T10:06:29.905208+0000","last_fullsized":"2024-04-25T10:06:29.905208+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:08.495304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889787+0000","last_change":"2024-04-25T10:06:28.895991+0000","last_active":"2024-04-25T10:06:29.889787+0000","last_peered":"2024-04-25T10:06:29.889787+0000","last_clean":"2024-04-25T10:06:29.889787+0000","last_became_active":"2024-04-25T10:06:28.895697+0000","last_became_peered":"2024-04-25T10:06:28.895697+0000","last_unstale":"2024-04-25T10:06:29.889787+0000","last_undegraded":"2024-04-25T10:06:29.889787+0000","last_fullsized":"2024-04-25T10:06:29.889787+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:04.446442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906503+0000","last_change":"2024-04-25T10:06:29.898010+0000","last_active":"2024-04-25T10:06:29.906503+0000","last_peered":"2024-04-25T10:06:29.906503+0000","last_clean":"2024-04-25T10:06:29.906503+0000","last_became_active":"2024-04-25T10:06:29.897707+0000","last_became_peered":"2024-04-25T10:06:29.897707+0000","last_unstale":"2024-04-25T10:06:29.906503+0000","last_undegraded":"2024-04-25T10:06:29.906503+0000","last_fullsized":"2024-04-25T10:06:29.906503+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.896851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889129+0000","last_change":"2024-04-25T10:06:28.885723+0000","last_active":"2024-04-25T10:06:29.889129+0000","last_peered":"2024-04-25T10:06:29.889129+0000","last_clean":"2024-04-25T10:06:29.889129+0000","last_became_active":"2024-04-25T10:06:28.885431+0000","last_became_peered":"2024-04-25T10:06:28.885431+0000","last_unstale":"2024-04-25T10:06:29.889129+0000","last_undegraded":"2024-04-25T10:06:29.889129+0000","last_fullsized":"2024-04-25T10:06:29.889129+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:52.298185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895091+0000","last_change":"2024-04-25T10:06:29.894166+0000","last_active":"2024-04-25T10:06:29.895091+0000","last_peered":"2024-04-25T10:06:29.895091+0000","last_clean":"2024-04-25T10:06:29.895091+0000","last_became_active":"2024-04-25T10:06:29.893894+0000","last_became_peered":"2024-04-25T10:06:29.893894+0000","last_unstale":"2024-04-25T10:06:29.895091+0000","last_undegraded":"2024-04-25T10:06:29.895091+0000","last_fullsized":"2024-04-25T10:06:29.895091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:22.503947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.899073+0000","last_change":"2024-04-25T10:06:28.884408+0000","last_active":"2024-04-25T10:06:29.899073+0000","last_peered":"2024-04-25T10:06:29.899073+0000","last_clean":"2024-04-25T10:06:29.899073+0000","last_became_active":"2024-04-25T10:06:28.884145+0000","last_became_peered":"2024-04-25T10:06:28.884145+0000","last_unstale":"2024-04-25T10:06:29.899073+0000","last_undegraded":"2024-04-25T10:06:29.899073+0000","last_fullsized":"2024-04-25T10:06:29.899073+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:33.175660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905250+0000","last_change":"2024-04-25T10:06:29.902182+0000","last_active":"2024-04-25T10:06:29.905250+0000","last_peered":"2024-04-25T10:06:29.905250+0000","last_clean":"2024-04-25T10:06:29.905250+0000","last_became_active":"2024-04-25T10:06:29.901911+0000","last_became_peered":"2024-04-25T10:06:29.901911+0000","last_unstale":"2024-04-25T10:06:29.905250+0000","last_undegraded":"2024-04-25T10:06:29.905250+0000","last_fullsized":"2024-04-25T10:06:29.905250+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:23.502409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.891065+0000","last_change":"2024-04-25T10:06:28.893876+0000","last_active":"2024-04-25T10:06:29.891065+0000","last_peered":"2024-04-25T10:06:29.891065+0000","last_clean":"2024-04-25T10:06:29.891065+0000","last_became_active":"2024-04-25T10:06:28.893544+0000","last_became_peered":"2024-04-25T10:06:28.893544+0000","last_unstale":"2024-04-25T10:06:29.891065+0000","last_undegraded":"2024-04-25T10:06:29.891065+0000","last_fullsized":"2024-04-25T10:06:29.891065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:47.591524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907110+0000","last_change":"2024-04-25T10:06:29.905315+0000","last_active":"2024-04-25T10:06:29.907110+0000","last_peered":"2024-04-25T10:06:29.907110+0000","last_clean":"2024-04-25T10:06:29.907110+0000","last_became_active":"2024-04-25T10:06:29.905046+0000","last_became_peered":"2024-04-25T10:06:29.905046+0000","last_unstale":"2024-04-25T10:06:29.907110+0000","last_undegraded":"2024-04-25T10:06:29.907110+0000","last_fullsized":"2024-04-25T10:06:29.907110+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:31.258172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"184'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917759+0000","last_change":"2024-04-25T10:06:28.885646+0000","last_active":"2024-04-25T10:06:29.917759+0000","last_peered":"2024-04-25T10:06:29.917759+0000","last_clean":"2024-04-25T10:06:29.917759+0000","last_became_active":"2024-04-25T10:06:28.884509+0000","last_became_peered":"2024-04-25T10:06:28.884509+0000","last_unstale":"2024-04-25T10:06:29.917759+0000","last_undegraded":"2024-04-25T10:06:29.917759+0000","last_fullsized":"2024-04-25T10:06:29.917759+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:14.936048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.903141+0000","last_change":"2024-04-25T10:06:29.902256+0000","last_active":"2024-04-25T10:06:29.903141+0000","last_peered":"2024-04-25T10:06:29.903141+0000","last_clean":"2024-04-25T10:06:29.903141+0000","last_became_active":"2024-04-25T10:06:29.901572+0000","last_became_peered":"2024-04-25T10:06:29.901572+0000","last_unstale":"2024-04-25T10:06:29.903141+0000","last_undegraded":"2024-04-25T10:06:29.903141+0000","last_fullsized":"2024-04-25T10:06:29.903141+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:08.529056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"184'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916797+0000","last_change":"2024-04-25T10:06:28.890561+0000","last_active":"2024-04-25T10:06:29.916797+0000","last_peered":"2024-04-25T10:06:29.916797+0000","last_clean":"2024-04-25T10:06:29.916797+0000","last_became_active":"2024-04-25T10:06:28.890258+0000","last_became_peered":"2024-04-25T10:06:28.890258+0000","last_unstale":"2024-04-25T10:06:29.916797+0000","last_undegraded":"2024-04-25T10:06:29.916797+0000","last_fullsized":"2024-04-25T10:06:29.916797+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:50.014361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907134+0000","last_change":"2024-04-25T10:06:29.902810+0000","last_active":"2024-04-25T10:06:29.907134+0000","last_peered":"2024-04-25T10:06:29.907134+0000","last_clean":"2024-04-25T10:06:29.907134+0000","last_became_active":"2024-04-25T10:06:29.902545+0000","last_became_peered":"2024-04-25T10:06:29.902545+0000","last_unstale":"2024-04-25T10:06:29.907134+0000","last_undegraded":"2024-04-25T10:06:29.907134+0000","last_fullsized":"2024-04-25T10:06:29.907134+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:59.868430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916719+0000","last_change":"2024-04-25T10:06:28.892677+0000","last_active":"2024-04-25T10:06:29.916719+0000","last_peered":"2024-04-25T10:06:29.916719+0000","last_clean":"2024-04-25T10:06:29.916719+0000","last_became_active":"2024-04-25T10:06:28.892403+0000","last_became_peered":"2024-04-25T10:06:28.892403+0000","last_unstale":"2024-04-25T10:06:29.916719+0000","last_undegraded":"2024-04-25T10:06:29.916719+0000","last_fullsized":"2024-04-25T10:06:29.916719+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:09.624872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.910041+0000","last_change":"2024-04-25T10:06:29.909528+0000","last_active":"2024-04-25T10:06:29.910041+0000","last_peered":"2024-04-25T10:06:29.910041+0000","last_clean":"2024-04-25T10:06:29.910041+0000","last_became_active":"2024-04-25T10:06:29.909227+0000","last_became_peered":"2024-04-25T10:06:29.909227+0000","last_unstale":"2024-04-25T10:06:29.910041+0000","last_undegraded":"2024-04-25T10:06:29.910041+0000","last_fullsized":"2024-04-25T10:06:29.910041+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:15.570957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889306+0000","last_change":"2024-04-25T10:06:28.887255+0000","last_active":"2024-04-25T10:06:29.889306+0000","last_peered":"2024-04-25T10:06:29.889306+0000","last_clean":"2024-04-25T10:06:29.889306+0000","last_became_active":"2024-04-25T10:06:28.886991+0000","last_became_peered":"2024-04-25T10:06:28.886991+0000","last_unstale":"2024-04-25T10:06:29.889306+0000","last_undegraded":"2024-04-25T10:06:29.889306+0000","last_fullsized":"2024-04-25T10:06:29.889306+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:58.016837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.906068+0000","last_change":"2024-04-25T10:06:29.904415+0000","last_active":"2024-04-25T10:06:29.906068+0000","last_peered":"2024-04-25T10:06:29.906068+0000","last_clean":"2024-04-25T10:06:29.906068+0000","last_became_active":"2024-04-25T10:06:29.904003+0000","last_became_peered":"2024-04-25T10:06:29.904003+0000","last_unstale":"2024-04-25T10:06:29.906068+0000","last_undegraded":"2024-04-25T10:06:29.906068+0000","last_fullsized":"2024-04-25T10:06:29.906068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:07.772900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.889712+0000","last_change":"2024-04-25T10:06:28.892751+0000","last_active":"2024-04-25T10:06:29.889712+0000","last_peered":"2024-04-25T10:06:29.889712+0000","last_clean":"2024-04-25T10:06:29.889712+0000","last_became_active":"2024-04-25T10:06:28.892461+0000","last_became_peered":"2024-04-25T10:06:28.892461+0000","last_unstale":"2024-04-25T10:06:29.889712+0000","last_undegraded":"2024-04-25T10:06:29.889712+0000","last_fullsized":"2024-04-25T10:06:29.889712+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:02.311028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907648+0000","last_change":"2024-04-25T10:06:29.905702+0000","last_active":"2024-04-25T10:06:29.907648+0000","last_peered":"2024-04-25T10:06:29.907648+0000","last_clean":"2024-04-25T10:06:29.907648+0000","last_became_active":"2024-04-25T10:06:29.905446+0000","last_became_peered":"2024-04-25T10:06:29.905446+0000","last_unstale":"2024-04-25T10:06:29.907648+0000","last_undegraded":"2024-04-25T10:06:29.907648+0000","last_fullsized":"2024-04-25T10:06:29.907648+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.860946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"184'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.914525+0000","last_change":"2024-04-25T10:06:28.885978+0000","last_active":"2024-04-25T10:06:29.914525+0000","last_peered":"2024-04-25T10:06:29.914525+0000","last_clean":"2024-04-25T10:06:29.914525+0000","last_became_active":"2024-04-25T10:06:28.885718+0000","last_became_peered":"2024-04-25T10:06:28.885718+0000","last_unstale":"2024-04-25T10:06:29.914525+0000","last_undegraded":"2024-04-25T10:06:29.914525+0000","last_fullsized":"2024-04-25T10:06:29.914525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:13.173776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911655+0000","last_change":"2024-04-25T10:06:29.909509+0000","last_active":"2024-04-25T10:06:29.911655+0000","last_peered":"2024-04-25T10:06:29.911655+0000","last_clean":"2024-04-25T10:06:29.911655+0000","last_became_active":"2024-04-25T10:06:29.909225+0000","last_became_peered":"2024-04-25T10:06:29.909225+0000","last_unstale":"2024-04-25T10:06:29.911655+0000","last_undegraded":"2024-04-25T10:06:29.911655+0000","last_fullsized":"2024-04-25T10:06:29.911655+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:38.720444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917662+0000","last_change":"2024-04-25T10:06:28.891608+0000","last_active":"2024-04-25T10:06:29.917662+0000","last_peered":"2024-04-25T10:06:29.917662+0000","last_clean":"2024-04-25T10:06:29.917662+0000","last_became_active":"2024-04-25T10:06:28.891297+0000","last_became_peered":"2024-04-25T10:06:28.891297+0000","last_unstale":"2024-04-25T10:06:29.917662+0000","last_undegraded":"2024-04-25T10:06:29.917662+0000","last_fullsized":"2024-04-25T10:06:29.917662+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.349406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904723+0000","last_change":"2024-04-25T10:06:29.903992+0000","last_active":"2024-04-25T10:06:29.904723+0000","last_peered":"2024-04-25T10:06:29.904723+0000","last_clean":"2024-04-25T10:06:29.904723+0000","last_became_active":"2024-04-25T10:06:29.903102+0000","last_became_peered":"2024-04-25T10:06:29.903102+0000","last_unstale":"2024-04-25T10:06:29.904723+0000","last_undegraded":"2024-04-25T10:06:29.904723+0000","last_fullsized":"2024-04-25T10:06:29.904723+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:53.832878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916807+0000","last_change":"2024-04-25T10:06:28.892890+0000","last_active":"2024-04-25T10:06:29.916807+0000","last_peered":"2024-04-25T10:06:29.916807+0000","last_clean":"2024-04-25T10:06:29.916807+0000","last_became_active":"2024-04-25T10:06:28.892549+0000","last_became_peered":"2024-04-25T10:06:28.892549+0000","last_unstale":"2024-04-25T10:06:29.916807+0000","last_undegraded":"2024-04-25T10:06:29.916807+0000","last_fullsized":"2024-04-25T10:06:29.916807+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:48.632213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.909283+0000","last_change":"2024-04-25T10:06:29.908290+0000","last_active":"2024-04-25T10:06:29.909283+0000","last_peered":"2024-04-25T10:06:29.909283+0000","last_clean":"2024-04-25T10:06:29.909283+0000","last_became_active":"2024-04-25T10:06:29.907977+0000","last_became_peered":"2024-04-25T10:06:29.907977+0000","last_unstale":"2024-04-25T10:06:29.909283+0000","last_undegraded":"2024-04-25T10:06:29.909283+0000","last_fullsized":"2024-04-25T10:06:29.909283+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:50.048658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.914579+0000","last_change":"2024-04-25T10:06:28.887441+0000","last_active":"2024-04-25T10:06:29.914579+0000","last_peered":"2024-04-25T10:06:29.914579+0000","last_clean":"2024-04-25T10:06:29.914579+0000","last_became_active":"2024-04-25T10:06:28.887170+0000","last_became_peered":"2024-04-25T10:06:28.887170+0000","last_unstale":"2024-04-25T10:06:29.914579+0000","last_undegraded":"2024-04-25T10:06:29.914579+0000","last_fullsized":"2024-04-25T10:06:29.914579+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:18.078928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905215+0000","last_change":"2024-04-25T10:06:29.901576+0000","last_active":"2024-04-25T10:06:29.905215+0000","last_peered":"2024-04-25T10:06:29.905215+0000","last_clean":"2024-04-25T10:06:29.905215+0000","last_became_active":"2024-04-25T10:06:29.901219+0000","last_became_peered":"2024-04-25T10:06:29.901219+0000","last_unstale":"2024-04-25T10:06:29.905215+0000","last_undegraded":"2024-04-25T10:06:29.905215+0000","last_fullsized":"2024-04-25T10:06:29.905215+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:29.082987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.898292+0000","last_change":"2024-04-25T10:06:28.890267+0000","last_active":"2024-04-25T10:06:29.898292+0000","last_peered":"2024-04-25T10:06:29.898292+0000","last_clean":"2024-04-25T10:06:29.898292+0000","last_became_active":"2024-04-25T10:06:28.889994+0000","last_became_peered":"2024-04-25T10:06:28.889994+0000","last_unstale":"2024-04-25T10:06:29.898292+0000","last_undegraded":"2024-04-25T10:06:29.898292+0000","last_fullsized":"2024-04-25T10:06:29.898292+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:57.883923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.899133+0000","last_change":"2024-04-25T10:06:29.898095+0000","last_active":"2024-04-25T10:06:29.899133+0000","last_peered":"2024-04-25T10:06:29.899133+0000","last_clean":"2024-04-25T10:06:29.899133+0000","last_became_active":"2024-04-25T10:06:29.897755+0000","last_became_peered":"2024-04-25T10:06:29.897755+0000","last_unstale":"2024-04-25T10:06:29.899133+0000","last_undegraded":"2024-04-25T10:06:29.899133+0000","last_fullsized":"2024-04-25T10:06:29.899133+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:38.161734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.913373+0000","last_change":"2024-04-25T10:06:28.883492+0000","last_active":"2024-04-25T10:06:29.913373+0000","last_peered":"2024-04-25T10:06:29.913373+0000","last_clean":"2024-04-25T10:06:29.913373+0000","last_became_active":"2024-04-25T10:06:28.882461+0000","last_became_peered":"2024-04-25T10:06:28.882461+0000","last_unstale":"2024-04-25T10:06:29.913373+0000","last_undegraded":"2024-04-25T10:06:29.913373+0000","last_fullsized":"2024-04-25T10:06:29.913373+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:25.053758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.901423+0000","last_change":"2024-04-25T10:06:29.900321+0000","last_active":"2024-04-25T10:06:29.901423+0000","last_peered":"2024-04-25T10:06:29.901423+0000","last_clean":"2024-04-25T10:06:29.901423+0000","last_became_active":"2024-04-25T10:06:29.900034+0000","last_became_peered":"2024-04-25T10:06:29.900034+0000","last_unstale":"2024-04-25T10:06:29.901423+0000","last_undegraded":"2024-04-25T10:06:29.901423+0000","last_fullsized":"2024-04-25T10:06:29.901423+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:07.376460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.886288+0000","last_change":"2024-04-25T10:06:28.889177+0000","last_active":"2024-04-25T10:06:29.886288+0000","last_peered":"2024-04-25T10:06:29.886288+0000","last_clean":"2024-04-25T10:06:29.886288+0000","last_became_active":"2024-04-25T10:06:28.888917+0000","last_became_peered":"2024-04-25T10:06:28.888917+0000","last_unstale":"2024-04-25T10:06:29.886288+0000","last_undegraded":"2024-04-25T10:06:29.886288+0000","last_fullsized":"2024-04-25T10:06:29.886288+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:30.253679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907152+0000","last_change":"2024-04-25T10:06:29.906188+0000","last_active":"2024-04-25T10:06:29.907152+0000","last_peered":"2024-04-25T10:06:29.907152+0000","last_clean":"2024-04-25T10:06:29.907152+0000","last_became_active":"2024-04-25T10:06:29.905880+0000","last_became_peered":"2024-04-25T10:06:29.905880+0000","last_unstale":"2024-04-25T10:06:29.907152+0000","last_undegraded":"2024-04-25T10:06:29.907152+0000","last_fullsized":"2024-04-25T10:06:29.907152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:41.819123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.887639+0000","last_change":"2024-04-25T10:06:28.886557+0000","last_active":"2024-04-25T10:06:29.887639+0000","last_peered":"2024-04-25T10:06:29.887639+0000","last_clean":"2024-04-25T10:06:29.887639+0000","last_became_active":"2024-04-25T10:06:28.885871+0000","last_became_peered":"2024-04-25T10:06:28.885871+0000","last_unstale":"2024-04-25T10:06:29.887639+0000","last_undegraded":"2024-04-25T10:06:29.887639+0000","last_fullsized":"2024-04-25T10:06:29.887639+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:27.333954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905232+0000","last_change":"2024-04-25T10:06:29.902725+0000","last_active":"2024-04-25T10:06:29.905232+0000","last_peered":"2024-04-25T10:06:29.905232+0000","last_clean":"2024-04-25T10:06:29.905232+0000","last_became_active":"2024-04-25T10:06:29.902449+0000","last_became_peered":"2024-04-25T10:06:29.902449+0000","last_unstale":"2024-04-25T10:06:29.905232+0000","last_undegraded":"2024-04-25T10:06:29.905232+0000","last_fullsized":"2024-04-25T10:06:29.905232+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:43.544894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895857+0000","last_change":"2024-04-25T10:06:28.895446+0000","last_active":"2024-04-25T10:06:29.895857+0000","last_peered":"2024-04-25T10:06:29.895857+0000","last_clean":"2024-04-25T10:06:29.895857+0000","last_became_active":"2024-04-25T10:06:28.895125+0000","last_became_peered":"2024-04-25T10:06:28.895125+0000","last_unstale":"2024-04-25T10:06:29.895857+0000","last_undegraded":"2024-04-25T10:06:29.895857+0000","last_fullsized":"2024-04-25T10:06:29.895857+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:06.371967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.911635+0000","last_change":"2024-04-25T10:06:29.908847+0000","last_active":"2024-04-25T10:06:29.911635+0000","last_peered":"2024-04-25T10:06:29.911635+0000","last_clean":"2024-04-25T10:06:29.911635+0000","last_became_active":"2024-04-25T10:06:29.908562+0000","last_became_peered":"2024-04-25T10:06:29.908562+0000","last_unstale":"2024-04-25T10:06:29.911635+0000","last_undegraded":"2024-04-25T10:06:29.911635+0000","last_fullsized":"2024-04-25T10:06:29.911635+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:09.125042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.916667+0000","last_change":"2024-04-25T10:06:28.888156+0000","last_active":"2024-04-25T10:06:29.916667+0000","last_peered":"2024-04-25T10:06:29.916667+0000","last_clean":"2024-04-25T10:06:29.916667+0000","last_became_active":"2024-04-25T10:06:28.887727+0000","last_became_peered":"2024-04-25T10:06:28.887727+0000","last_unstale":"2024-04-25T10:06:29.916667+0000","last_undegraded":"2024-04-25T10:06:29.916667+0000","last_fullsized":"2024-04-25T10:06:29.916667+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:14.566464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.908764+0000","last_change":"2024-04-25T10:06:29.905125+0000","last_active":"2024-04-25T10:06:29.908764+0000","last_peered":"2024-04-25T10:06:29.908764+0000","last_clean":"2024-04-25T10:06:29.908764+0000","last_became_active":"2024-04-25T10:06:29.904863+0000","last_became_peered":"2024-04-25T10:06:29.904863+0000","last_unstale":"2024-04-25T10:06:29.908764+0000","last_undegraded":"2024-04-25T10:06:29.908764+0000","last_fullsized":"2024-04-25T10:06:29.908764+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:08.126372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.893302+0000","last_change":"2024-04-25T10:06:28.884059+0000","last_active":"2024-04-25T10:06:29.893302+0000","last_peered":"2024-04-25T10:06:29.893302+0000","last_clean":"2024-04-25T10:06:29.893302+0000","last_became_active":"2024-04-25T10:06:28.883785+0000","last_became_peered":"2024-04-25T10:06:28.883785+0000","last_unstale":"2024-04-25T10:06:29.893302+0000","last_undegraded":"2024-04-25T10:06:29.893302+0000","last_fullsized":"2024-04-25T10:06:29.893302+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:34.037040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.905975+0000","last_change":"2024-04-25T10:06:29.904501+0000","last_active":"2024-04-25T10:06:29.905975+0000","last_peered":"2024-04-25T10:06:29.905975+0000","last_clean":"2024-04-25T10:06:29.905975+0000","last_became_active":"2024-04-25T10:06:29.903981+0000","last_became_peered":"2024-04-25T10:06:29.903981+0000","last_unstale":"2024-04-25T10:06:29.905975+0000","last_undegraded":"2024-04-25T10:06:29.905975+0000","last_fullsized":"2024-04-25T10:06:29.905975+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:10.629365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.887257+0000","last_change":"2024-04-25T10:06:28.885036+0000","last_active":"2024-04-25T10:06:29.887257+0000","last_peered":"2024-04-25T10:06:29.887257+0000","last_clean":"2024-04-25T10:06:29.887257+0000","last_became_active":"2024-04-25T10:06:28.884772+0000","last_became_peered":"2024-04-25T10:06:28.884772+0000","last_unstale":"2024-04-25T10:06:29.887257+0000","last_undegraded":"2024-04-25T10:06:29.887257+0000","last_fullsized":"2024-04-25T10:06:29.887257+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:00.182715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904198+0000","last_change":"2024-04-25T10:06:29.902079+0000","last_active":"2024-04-25T10:06:29.904198+0000","last_peered":"2024-04-25T10:06:29.904198+0000","last_clean":"2024-04-25T10:06:29.904198+0000","last_became_active":"2024-04-25T10:06:29.901814+0000","last_became_peered":"2024-04-25T10:06:29.901814+0000","last_unstale":"2024-04-25T10:06:29.904198+0000","last_undegraded":"2024-04-25T10:06:29.904198+0000","last_fullsized":"2024-04-25T10:06:29.904198+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:06.413449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.890483+0000","last_change":"2024-04-25T10:06:28.889751+0000","last_active":"2024-04-25T10:06:29.890483+0000","last_peered":"2024-04-25T10:06:29.890483+0000","last_clean":"2024-04-25T10:06:29.890483+0000","last_became_active":"2024-04-25T10:06:28.889448+0000","last_became_peered":"2024-04-25T10:06:28.889448+0000","last_unstale":"2024-04-25T10:06:29.890483+0000","last_undegraded":"2024-04-25T10:06:29.890483+0000","last_fullsized":"2024-04-25T10:06:29.890483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:50.014361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.910685+0000","last_change":"2024-04-25T10:06:29.907143+0000","last_active":"2024-04-25T10:06:29.910685+0000","last_peered":"2024-04-25T10:06:29.910685+0000","last_clean":"2024-04-25T10:06:29.910685+0000","last_became_active":"2024-04-25T10:06:29.905123+0000","last_became_peered":"2024-04-25T10:06:29.905123+0000","last_unstale":"2024-04-25T10:06:29.910685+0000","last_undegraded":"2024-04-25T10:06:29.910685+0000","last_fullsized":"2024-04-25T10:06:29.910685+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:28.854535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.895604+0000","last_change":"2024-04-25T10:06:28.888454+0000","last_active":"2024-04-25T10:06:29.895604+0000","last_peered":"2024-04-25T10:06:29.895604+0000","last_clean":"2024-04-25T10:06:29.895604+0000","last_became_active":"2024-04-25T10:06:28.888118+0000","last_became_peered":"2024-04-25T10:06:28.888118+0000","last_unstale":"2024-04-25T10:06:29.895604+0000","last_undegraded":"2024-04-25T10:06:29.895604+0000","last_fullsized":"2024-04-25T10:06:29.895604+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.606339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904957+0000","last_change":"2024-04-25T10:06:29.903217+0000","last_active":"2024-04-25T10:06:29.904957+0000","last_peered":"2024-04-25T10:06:29.904957+0000","last_clean":"2024-04-25T10:06:29.904957+0000","last_became_active":"2024-04-25T10:06:29.902949+0000","last_became_peered":"2024-04-25T10:06:29.902949+0000","last_unstale":"2024-04-25T10:06:29.904957+0000","last_undegraded":"2024-04-25T10:06:29.904957+0000","last_fullsized":"2024-04-25T10:06:29.904957+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:35.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.890793+0000","last_change":"2024-04-25T10:06:28.889178+0000","last_active":"2024-04-25T10:06:29.890793+0000","last_peered":"2024-04-25T10:06:29.890793+0000","last_clean":"2024-04-25T10:06:29.890793+0000","last_became_active":"2024-04-25T10:06:28.888920+0000","last_became_peered":"2024-04-25T10:06:28.888920+0000","last_unstale":"2024-04-25T10:06:29.890793+0000","last_undegraded":"2024-04-25T10:06:29.890793+0000","last_fullsized":"2024-04-25T10:06:29.890793+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:31.192187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.907068+0000","last_change":"2024-04-25T10:06:29.904373+0000","last_active":"2024-04-25T10:06:29.907068+0000","last_peered":"2024-04-25T10:06:29.907068+0000","last_clean":"2024-04-25T10:06:29.907068+0000","last_became_active":"2024-04-25T10:06:29.904071+0000","last_became_peered":"2024-04-25T10:06:29.904071+0000","last_unstale":"2024-04-25T10:06:29.907068+0000","last_undegraded":"2024-04-25T10:06:29.907068+0000","last_fullsized":"2024-04-25T10:06:29.907068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:46.917935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.896723+0000","last_change":"2024-04-25T10:06:28.887755+0000","last_active":"2024-04-25T10:06:29.896723+0000","last_peered":"2024-04-25T10:06:29.896723+0000","last_clean":"2024-04-25T10:06:29.896723+0000","last_became_active":"2024-04-25T10:06:28.887477+0000","last_became_peered":"2024-04-25T10:06:28.887477+0000","last_unstale":"2024-04-25T10:06:29.896723+0000","last_undegraded":"2024-04-25T10:06:29.896723+0000","last_fullsized":"2024-04-25T10:06:29.896723+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.053892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.909689+0000","last_change":"2024-04-25T10:06:29.905746+0000","last_active":"2024-04-25T10:06:29.909689+0000","last_peered":"2024-04-25T10:06:29.909689+0000","last_clean":"2024-04-25T10:06:29.909689+0000","last_became_active":"2024-04-25T10:06:29.905446+0000","last_became_peered":"2024-04-25T10:06:29.905446+0000","last_unstale":"2024-04-25T10:06:29.909689+0000","last_undegraded":"2024-04-25T10:06:29.909689+0000","last_fullsized":"2024-04-25T10:06:29.909689+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:32.196680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.896003+0000","last_change":"2024-04-25T10:06:28.885341+0000","last_active":"2024-04-25T10:06:29.896003+0000","last_peered":"2024-04-25T10:06:29.896003+0000","last_clean":"2024-04-25T10:06:29.896003+0000","last_became_active":"2024-04-25T10:06:28.885059+0000","last_became_peered":"2024-04-25T10:06:28.885059+0000","last_unstale":"2024-04-25T10:06:29.896003+0000","last_undegraded":"2024-04-25T10:06:29.896003+0000","last_fullsized":"2024-04-25T10:06:29.896003+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:24.050092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.904260+0000","last_change":"2024-04-25T10:06:29.902683+0000","last_active":"2024-04-25T10:06:29.904260+0000","last_peered":"2024-04-25T10:06:29.904260+0000","last_clean":"2024-04-25T10:06:29.904260+0000","last_became_active":"2024-04-25T10:06:29.902406+0000","last_became_peered":"2024-04-25T10:06:29.902406+0000","last_unstale":"2024-04-25T10:06:29.904260+0000","last_undegraded":"2024-04-25T10:06:29.904260+0000","last_fullsized":"2024-04-25T10:06:29.904260+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:28.868628+0000","last_clean_scrub_stamp":"2024-04-25T10:06:28.868628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.860946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.917557+0000","last_change":"2024-04-25T10:06:28.885084+0000","last_active":"2024-04-25T10:06:29.917557+0000","last_peered":"2024-04-25T10:06:29.917557+0000","last_clean":"2024-04-25T10:06:29.917557+0000","last_became_active":"2024-04-25T10:06:28.884238+0000","last_became_peered":"2024-04-25T10:06:28.884238+0000","last_unstale":"2024-04-25T10:06:29.917557+0000","last_undegraded":"2024-04-25T10:06:29.917557+0000","last_fullsized":"2024-04-25T10:06:29.917557+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:27.864135+0000","last_clean_scrub_stamp":"2024-04-25T10:06:27.864135+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:07.121879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":473,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:06:29.890009+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:06:29.890009+0000","last_peered":"2024-04-25T10:06:29.890009+0000","last_clean":"2024-04-25T10:06:29.890009+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:06:29.890009+0000","last_undegraded":"2024-04-25T10:06:29.890009+0000","last_fullsized":"2024-04-25T10:06:29.890009+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2904064,"data_stored":1672307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55644,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316196,"statfs":{"total":96636764160,"available":96579784704,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59808,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59808,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55648,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316192,"statfs":{"total":96636764160,"available":96579780608,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55648,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316192,"statfs":{"total":96636764160,"available":96579780608,"internally_reserved":0,"allocated":2310144,"data_stored":1081939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56228,"kb_used_data":2836,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315612,"statfs":{"total":96636764160,"available":96579186688,"internally_reserved":0,"allocated":2904064,"data_stored":1672307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:33.191 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:33.192 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:33.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.327+0000 7f752db28640 1 Processor -- start 2024-04-25T10:06:33.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.327+0000 7f752db28640 1 -- start start 2024-04-25T10:06:33.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f752db28640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f752806abe0 0x7f752806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f752db28640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75280ad570 0x7f752806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f752db28640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752806b500 con 0x7f752815dd80 2024-04-25T10:06:33.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f752db28640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752806e410 con 0x7f75280ad570 2024-04-25T10:06:33.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f752db28640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75280b3930 con 0x7f752806abe0 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75280ad570 0x7f752806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f75280ad570 0x7f752806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40766/0 (socket says 172.21.15.148:40766) 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 -- 172.21.15.148:0/3273948338 learned_addr learned my addr 172.21.15.148:0/3273948338 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f75277fe640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f752806abe0 0x7f752806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 -- 172.21.15.148:0/3273948338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f752806abe0 msgr2=0x7f752806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f752806abe0 0x7f752806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 -- 172.21.15.148:0/3273948338 >> v1:172.21.15.188:6789/0 conn(0x7f752815dd80 legacy=0x7f75280b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f7527fff640 1 -- 172.21.15.148:0/3273948338 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280b3c10 con 0x7f75280ad570 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.328+0000 7f75277fe640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f752806abe0 0x7f752806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.329+0000 7f7527fff640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75280ad570 0x7f752806ee60 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f7520009e50 tx=0x7f752002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f284a7bc7c31bce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.329+0000 7f75267fc640 1 -- 172.21.15.148:0/3273948338 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520036070 con 0x7f75280ad570 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.329+0000 7f75267fc640 1 -- 172.21.15.148:0/3273948338 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7520031600 con 0x7f75280ad570 2024-04-25T10:06:33.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.329+0000 7f75267fc640 1 -- 172.21.15.148:0/3273948338 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75200319d0 con 0x7f75280ad570 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 -- 172.21.15.148:0/3273948338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75280ad570 msgr2=0x7f752806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75280ad570 0x7f752806ee60 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f7520009e50 tx=0x7f752002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 -- 172.21.15.148:0/3273948338 shutdown_connections 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75280ad570 0x7f752806ee60 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 --2- 172.21.15.148:0/3273948338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f752806abe0 0x7f752806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 -- 172.21.15.148:0/3273948338 >> 172.21.15.148:0/3273948338 conn(0x7f75280b5af0 msgr2=0x7f75280b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 -- 172.21.15.148:0/3273948338 shutdown_connections 2024-04-25T10:06:33.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.330+0000 7f752db28640 1 -- 172.21.15.148:0/3273948338 wait complete. 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.331+0000 7f752db28640 1 Processor -- start 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.331+0000 7f752db28640 1 -- start start 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752806abe0 0x7f7528158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75280ad570 0x7f752815f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 0x7f75281f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7528070280 con 0x7f752815dd80 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752806e940 con 0x7f752806abe0 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7528070400 con 0x7f75280ad570 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 0x7f75281f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 0x7f75281f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:40782/0 (socket says 172.21.15.148:40782) 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 -- 172.21.15.148:0/3448822737 learned_addr learned my addr 172.21.15.148:0/3448822737 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f75277fe640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752806abe0 0x7f7528158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7526ffd640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75280ad570 0x7f752815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 -- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75280ad570 msgr2=0x7f752815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75280ad570 0x7f752815f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 -- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752806abe0 msgr2=0x7f7528158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752806abe0 0x7f7528158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 -- 172.21.15.148:0/3448822737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75281f7a00 con 0x7f752815dd80 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f75277fe640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752806abe0 0x7f7528158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f7527fff640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 0x7f75281f6bb0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f7520034040 tx=0x7f7520004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520036050 con 0x7f752815dd80 2024-04-25T10:06:33.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7520042070 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520009510 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7528157df0 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.332+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75281583b0 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.333+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7528073a00 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.334+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7520007bf0 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.334+0000 7f750ffff640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751406b4d0 0x7f751406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.334+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75200ab2f0 con 0x7f752815dd80 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.334+0000 7f75277fe640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751406b4d0 0x7f751406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.335+0000 7f75277fe640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751406b4d0 0x7f751406d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f752814dfe0 tx=0x7f7518008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.338+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f752007a070 con 0x7f752815dd80 2024-04-25T10:06:33.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.573+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f752806d370 con 0x7f752815dd80 2024-04-25T10:06:33.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.576+0000 7f750ffff640 1 -- 172.21.15.148:0/3448822737 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7520079970 con 0x7f752815dd80 2024-04-25T10:06:33.581 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:06:33.581 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751406b4d0 msgr2=0x7f751406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751406b4d0 0x7f751406d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f752814dfe0 tx=0x7f7518008040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 msgr2=0x7f75281f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 0x7f75281f6bb0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f7520034040 tx=0x7f7520004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 shutdown_connections 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f751406b4d0 0x7f751406d990 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f752815dd80 0x7f75281f6bb0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75280ad570 0x7f752815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 --2- 172.21.15.148:0/3448822737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f752806abe0 0x7f7528158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 >> 172.21.15.148:0/3448822737 conn(0x7f75280b5af0 msgr2=0x7f752806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 shutdown_connections 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.578+0000 7f752db28640 1 -- 172.21.15.148:0/3448822737 wait complete. 2024-04-25T10:06:33.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:33.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.686+0000 7fe608f17640 1 Processor -- start 2024-04-25T10:06:33.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.686+0000 7fe608f17640 1 -- start start 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe608f17640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe604154ed0 0x7fe6040aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe608f17640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60415ec60 con 0x7fe60406c1d0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe608f17640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60415ef40 con 0x7fe604071ac0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe608f17640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406db90 con 0x7fe604154ed0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe602d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe60406c1d0 0x7fe60406afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33614/0 (socket says 172.21.15.148:33614) 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe602d9d640 1 -- 172.21.15.148:0/3203644454 learned_addr learned my addr 172.21.15.148:0/3203644454 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe60259c640 1 --2- 172.21.15.148:0/3203644454 >> v2:172.21.15.188:3300/0 conn(0x7fe604154ed0 0x7fe6040aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591390182 0 0) 0x7fe60415ec60 con 0x7fe60406c1d0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.687+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f0003520 con 0x7fe60406c1d0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788437848 0 0) 0x7fe60415ef40 con 0x7fe604071ac0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe60415ec60 con 0x7fe604071ac0 2024-04-25T10:06:33.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594761629 0 0) 0x7fe5f0003520 con 0x7fe60406c1d0 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe60259c640 1 -- 172.21.15.148:0/3203644454 >> v1:172.21.15.188:6789/0 conn(0x7fe604071ac0 legacy=0x7fe6040ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe60259c640 1 -- 172.21.15.148:0/3203644454 >> v1:172.21.15.148:6789/0 conn(0x7fe60406c1d0 legacy=0x7fe60406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe60259c640 1 -- 172.21.15.148:0/3203644454 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60406de70 con 0x7fe604154ed0 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.688+0000 7fe60259c640 1 --2- 172.21.15.148:0/3203644454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040aced0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40097a0 tx=0x7fe5f402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b115689d24d1650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4037070 con 0x7fe604154ed0 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f402abc0 con 0x7fe604154ed0 2024-04-25T10:06:33.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe601d9b640 1 -- 172.21.15.148:0/3203644454 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0019f0 con 0x7fe604154ed0 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 -- 172.21.15.148:0/3203644454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 msgr2=0x7fe6040aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 --2- 172.21.15.148:0/3203644454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040aced0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40097a0 tx=0x7fe5f402a130 comp rx=0 tx=0).stop 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 -- 172.21.15.148:0/3203644454 shutdown_connections 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 --2- 172.21.15.148:0/3203644454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040aced0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 -- 172.21.15.148:0/3203644454 >> 172.21.15.148:0/3203644454 conn(0x7fe6040af670 msgr2=0x7fe6040b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 -- 172.21.15.148:0/3203644454 shutdown_connections 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.689+0000 7fe608f17640 1 -- 172.21.15.148:0/3203644454 wait complete. 2024-04-25T10:06:33.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.690+0000 7fe608f17640 1 Processor -- start 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.690+0000 7fe608f17640 1 -- start start 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe608f17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 0x7fe60413d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe608f17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe608f17640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6040b11a0 0x7fe60413dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe608f17640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406b830 con 0x7fe604071ac0 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe608f17640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406b6b0 con 0x7fe604154ed0 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe608f17640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406b9b0 con 0x7fe6040b11a0 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe60259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 0x7fe60413d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe60259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 0x7fe60413d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51224/0 (socket says 172.21.15.148:51224) 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe60259c640 1 -- 172.21.15.148:0/3856316798 learned_addr learned my addr 172.21.15.148:0/3856316798 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe60259c640 1 -- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6040b11a0 msgr2=0x7fe60413dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe5fbfff640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.691+0000 7fe602d9d640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6040b11a0 0x7fe60413dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe60259c640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6040b11a0 0x7fe60413dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe60259c640 1 -- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 msgr2=0x7fe6040b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe60259c640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe60259c640 1 -- 172.21.15.148:0/3856316798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040afc00 con 0x7fe604071ac0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe5fbfff640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe60259c640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 0x7fe60413d570 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4035040 tx=0x7fe5f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4042020 con 0x7fe604071ac0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f4003a40 con 0x7fe604071ac0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6040afea0 con 0x7fe604071ac0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4037040 con 0x7fe604071ac0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.692+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6040b0460 con 0x7fe604071ac0 2024-04-25T10:06:33.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.693+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6040b2b10 con 0x7fe604071ac0 2024-04-25T10:06:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.693+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5f4045020 con 0x7fe604071ac0 2024-04-25T10:06:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.694+0000 7fe5faffd640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5e406b4d0 0x7fe5e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.694+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5f40aa900 con 0x7fe604071ac0 2024-04-25T10:06:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.694+0000 7fe5fbfff640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5e406b4d0 0x7fe5e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.694+0000 7fe5fbfff640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5e406b4d0 0x7fe5e406d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe6040b1770 tx=0x7fe5ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.698+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5f404c030 con 0x7fe604071ac0 2024-04-25T10:06:33.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.913+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6040b4cf0 con 0x7fe604071ac0 2024-04-25T10:06:33.919 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.913+0000 7fe5faffd640 1 -- 172.21.15.148:0/3856316798 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4965 (secure 0 0 0) 0x7fe5f40797a0 con 0x7fe604071ac0 2024-04-25T10:06:33.919 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:33.919 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:29.223742+0000","modified":"2024-04-25T10:06:31.231117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11268":{"gid":11268,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1743980733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1743980733},{"type":"v1","addr":"172.21.15.148:6837","nonce":1743980733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:06:33.919 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 106 2024-04-25T10:06:33.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.915+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5e406b4d0 msgr2=0x7fe5e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.915+0000 7fe608f17640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5e406b4d0 0x7fe5e406d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe6040b1770 tx=0x7fe5ec009040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.921 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.915+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 msgr2=0x7fe60413d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.915+0000 7fe608f17640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 0x7fe60413d570 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4035040 tx=0x7fe5f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 shutdown_connections 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe5e406b4d0 0x7fe5e406d990 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6040b11a0 0x7fe60413dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe604154ed0 0x7fe6040b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 --2- 172.21.15.148:0/3856316798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe604071ac0 0x7fe60413d570 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 >> 172.21.15.148:0/3856316798 conn(0x7fe60406c1d0 msgr2=0x7fe6040adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 shutdown_connections 2024-04-25T10:06:33.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:33.916+0000 7fe608f17640 1 -- 172.21.15.148:0/3856316798 wait complete. 2024-04-25T10:06:34.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:29.223742+0000', 'modified': '2024-04-25T10:06:31.231117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11268}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11268': {'gid': 11268, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.148:6837/1743980733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1743980733}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1743980733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:34.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:34.335 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:06:34.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.428+0000 7f6e80789640 1 Processor -- start 2024-04-25T10:06:34.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.428+0000 7f6e80789640 1 -- start start 2024-04-25T10:06:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e80789640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e780b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e80789640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780b3930 con 0x7f6e780ad570 2024-04-25T10:06:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e80789640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780b3c10 con 0x7f6e7806abe0 2024-04-25T10:06:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e80789640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7806e410 con 0x7f6e7815dd80 2024-04-25T10:06:34.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e7ed26640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6e780ad570 0x7f6e7806ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33630/0 (socket says 172.21.15.148:33630) 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2648932642 learned_addr learned my addr 172.21.15.148:0/2648932642 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.429+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e780b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871151467 0 0) 0x7f6e780b3930 con 0x7f6e780ad570 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64003520 con 0x7f6e780ad570 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243761355 0 0) 0x7f6e780b3c10 con 0x7f6e7806abe0 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e780b3930 con 0x7f6e7806abe0 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453930022 0 0) 0x7f6e64003520 con 0x7f6e780ad570 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 msgr2=0x7f6e780b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e780b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 >> v1:172.21.15.188:6789/0 conn(0x7f6e7806abe0 legacy=0x7f6e7806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7806e660 con 0x7f6e780ad570 2024-04-25T10:06:34.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f6e70001670 con 0x7f6e780ad570 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e780b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 >> v1:172.21.15.148:6789/0 conn(0x7f6e780ad570 legacy=0x7f6e7806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e64003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e640040b0 0x7f6e64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e64004d40 0x7f6e640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7806e410 con 0x7f6e64004d40 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400ba70 con 0x7f6e640040b0 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400bc70 con 0x7f6e7815dd80 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e64004d40 0x7f6e640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e64003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e640040b0 0x7f6e64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 msgr2=0x7f6e64003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e64003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e640040b0 msgr2=0x7f6e64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e640040b0 0x7f6e64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2648932642 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64003520 con 0x7f6e64004d40 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815dd80 0x7f6e64003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.430+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e640040b0 0x7f6e64004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.431+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e64004d40 0x7f6e640051c0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c190 tx=0x7f6e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2532b8dfb544788f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.431+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007ce0 con 0x7f6e64004d40 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.431+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70005ad0 con 0x7f6e64004d40 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.431+0000 7f6e7d523640 1 -- 172.21.15.148:0/2648932642 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007070 con 0x7f6e64004d40 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e80789640 1 -- 172.21.15.148:0/2648932642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e64004d40 msgr2=0x7f6e640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e80789640 1 --2- 172.21.15.148:0/2648932642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e64004d40 0x7f6e640051c0 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c190 tx=0x7f6e64005830 comp rx=0 tx=0).stop 2024-04-25T10:06:34.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2648932642 reap_dead start 2024-04-25T10:06:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e80789640 1 -- 172.21.15.148:0/2648932642 shutdown_connections 2024-04-25T10:06:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e80789640 1 -- 172.21.15.148:0/2648932642 >> 172.21.15.148:0/2648932642 conn(0x7f6e780b5af0 msgr2=0x7f6e780b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e80789640 1 -- 172.21.15.148:0/2648932642 shutdown_connections 2024-04-25T10:06:34.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.432+0000 7f6e80789640 1 -- 172.21.15.148:0/2648932642 wait complete. 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.433+0000 7f6e80789640 1 Processor -- start 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.433+0000 7f6e80789640 1 -- start start 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e64004d40 0x7f6e781431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 0x7f6e7809ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e781e67b0 0x7f6e781e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e78070400 con 0x7f6e7809b590 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e78070280 con 0x7f6e64004d40 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780b3590 con 0x7f6e781e67b0 2024-04-25T10:06:34.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 0x7f6e7809ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 0x7f6e7809ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51236/0 (socket says 172.21.15.148:51236) 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 -- 172.21.15.148:0/4123469175 learned_addr learned my addr 172.21.15.148:0/4123469175 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7e525640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e64004d40 0x7f6e781431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e781e67b0 0x7f6e781e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 -- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e781e67b0 msgr2=0x7f6e781e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e781e67b0 0x7f6e781e6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 -- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e64004d40 msgr2=0x7f6e781431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e64004d40 0x7f6e781431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 -- 172.21.15.148:0/4123469175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e781e7520 con 0x7f6e7809b590 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 0x7f6e7809ba10 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f6e68011560 tx=0x7f6e68011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e6effd640 1 -- 172.21.15.148:0/4123469175 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68015870 con 0x7f6e7809b590 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e6effd640 1 -- 172.21.15.148:0/4123469175 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68015a10 con 0x7f6e7809b590 2024-04-25T10:06:34.447 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e781e7800 con 0x7f6e7809b590 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e6effd640 1 -- 172.21.15.148:0/4123469175 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68015dc0 con 0x7f6e7809b590 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.434+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e781e7dc0 con 0x7f6e7809b590 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.436+0000 7f6e6effd640 1 -- 172.21.15.148:0/4123469175 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e6801d400 con 0x7f6e7809b590 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.436+0000 7f6e6effd640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.437+0000 7f6e7e525640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.437+0000 7f6e6effd640 1 -- 172.21.15.148:0/4123469175 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e6808aae0 con 0x7f6e7809b590 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.437+0000 7f6e7e525640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6e740051f0 tx=0x7f6e74009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.437+0000 7f6e83569740 1 Processor -- start 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 -- start start 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000287610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002a5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56300025cf10 0x5630002a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630001bfa10 con 0x5630002a4ad0 2024-04-25T10:06:34.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563000168b30 con 0x563000287230 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630001bbc50 con 0x56300025cf10 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e7dd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e7dd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51240/0 (socket says 172.21.15.148:51240) 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.438+0000 7f6e7dd24640 1 -- 172.21.15.148:0/183401669 learned_addr learned my addr 172.21.15.148:0/183401669 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7dd24640 1 -- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56300025cf10 msgr2=0x5630002a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7e525640 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000287610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56300025cf10 0x5630002a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56300025cf10 0x5630002a3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7dd24640 1 -- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 msgr2=0x563000287610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000287610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7dd24640 1 -- 172.21.15.148:0/183401669 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56300017de10 con 0x5630002a4ad0 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7e525640 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000287610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002a5020 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f6e68061340 tx=0x7f6e68059990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bf76ec0f3576e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e5bfff640 1 -- 172.21.15.148:0/183401669 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68057200 con 0x5630002a4ad0 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e5bfff640 1 -- 172.21.15.148:0/183401669 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68087070 con 0x5630002a4ad0 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.439+0000 7f6e5bfff640 1 -- 172.21.15.148:0/183401669 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680557d0 con 0x5630002a4ad0 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 -- 172.21.15.148:0/183401669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 msgr2=0x5630002a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002a5020 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f6e68061340 tx=0x7f6e68059990 comp rx=0 tx=0).stop 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 -- 172.21.15.148:0/183401669 shutdown_connections 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56300025cf10 0x5630002a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002a5020 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 --2- 172.21.15.148:0/183401669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000287610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 -- 172.21.15.148:0/183401669 >> 172.21.15.148:0/183401669 conn(0x563000259840 msgr2=0x5630002a2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 -- 172.21.15.148:0/183401669 shutdown_connections 2024-04-25T10:06:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 -- 172.21.15.148:0/183401669 wait complete. 2024-04-25T10:06:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 -- start start 2024-04-25T10:06:34.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.440+0000 7f6e83569740 1 Processor -- start 2024-04-25T10:06:34.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000299a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002a4ad0 0x5630002889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x563000285c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300025dcb0 con 0x563000287230 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300017c0c0 con 0x5630002a4ad0 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563000257770 con 0x5630002857e0 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e83569740 10 client.0 _ll_register_callbacks cb 0x56300016fec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e7e525640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000299a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e7e525640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000299a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51248/0 (socket says 172.21.15.148:51248) 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.441+0000 7f6e7e525640 1 -- 172.21.15.148:0/2141350094 learned_addr learned my addr 172.21.15.148:0/2141350094 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x563000285c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002a4ad0 0x5630002889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7e525640 1 -- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 msgr2=0x563000285c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x563000285c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7e525640 1 -- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002a4ad0 msgr2=0x5630002889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002a4ad0 0x5630002889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7e525640 1 -- 172.21.15.148:0/2141350094 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fffed42d0 con 0x563000287230 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002a4ad0 0x5630002889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x563000285c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000299a60 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f6e740146b0 tx=0x7f6e74007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a17f9790fb2725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e597fa640 1 -- 172.21.15.148:0/2141350094 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7401a070 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e597fa640 1 -- 172.21.15.148:0/2141350094 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e740238b0 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e68087070 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e597fa640 1 -- 172.21.15.148:0/2141350094 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e74023ab0 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 10 client.11379 fetch_fsmap learned FSMap version 106 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x562ffff9e6b0 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e597fa640 1 -- 172.21.15.148:0/2141350094 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68087070 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e597fa640 1 -- 172.21.15.148:0/2141350094 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e74020070 con 0x563000287230 2024-04-25T10:06:34.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 10 client.11379 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 10 client.11379 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 20 client.11379 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5630002bded0 0x5630002c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 4 client.11379 mds_command: new command op to 11268 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.442+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 --> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56300021e920 con 0x5630002bded0 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5630002bded0 0x5630002c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e597fa640 10 client.11379 ms_handle_connect on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5630002bded0 0x5630002c02c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f5e1d833a7297e server_cookie=e401a77ec135ef5 in_seq=0 out_seq=0 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e597fa640 1 -- 172.21.15.148:0/2141350094 <== mds.0 v2:172.21.15.148:6836/1743980733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56300021e920 con 0x5630002bded0 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e597fa640 10 client.11379 handle_command_reply: tid=0 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e83569740 1 client.11379 shutdown 2024-04-25T10:06:34.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 msgr2=0x563000299a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.443+0000 7f6e83569740 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000299a60 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f6e740146b0 tx=0x7f6e74007f90 comp rx=0 tx=0).stop 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 shutdown_connections 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5630002bded0 0x5630002c02c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x563000285c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002a4ad0 0x5630002889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e597fa640 0 client.11379 ms_handle_reset on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 --2- 172.21.15.148:0/2141350094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000299a60 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 >> 172.21.15.148:0/2141350094 conn(0x56300025cf10 msgr2=0x563000256980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 shutdown_connections 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 1 -- 172.21.15.148:0/2141350094 wait complete. 2024-04-25T10:06:34.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.444+0000 7f6e83569740 20 client.11379 trim_cache size 0 max 16384 2024-04-25T10:06:34.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.459+0000 7f6e83569740 1 Processor -- start 2024-04-25T10:06:34.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.459+0000 7f6e83569740 1 -- start start 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002857e0 0x563000283890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e83569740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000272f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002a4ad0 0x56300025d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e83569740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e78200e20 con 0x563000287230 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e78200020 con 0x5630002857e0 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e781ff420 con 0x5630002a4ad0 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000272f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000272f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51262/0 (socket says 172.21.15.148:51262) 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 -- 172.21.15.148:0/901937631 learned_addr learned my addr 172.21.15.148:0/901937631 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:34.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 -- 172.21.15.148:0/901937631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002a4ad0 msgr2=0x56300025d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002a4ad0 0x56300025d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 -- 172.21.15.148:0/901937631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002857e0 msgr2=0x563000283890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002857e0 0x563000283890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.460+0000 7f6e7dd24640 1 -- 172.21.15.148:0/901937631 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56300029a700 con 0x563000287230 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000272f20 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f6e68059320 tx=0x7f6e6805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c78b3467ec7fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e5b7fe640 1 -- 172.21.15.148:0/901937631 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6807b020 con 0x563000287230 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e5b7fe640 1 -- 172.21.15.148:0/901937631 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68087040 con 0x563000287230 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e5b7fe640 1 -- 172.21.15.148:0/901937631 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680557d0 con 0x563000287230 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 -- 172.21.15.148:0/901937631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 msgr2=0x563000272f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000272f20 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f6e68059320 tx=0x7f6e6805c000 comp rx=0 tx=0).stop 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 -- 172.21.15.148:0/901937631 shutdown_connections 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002a4ad0 0x56300025d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563000287230 0x563000272f20 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 --2- 172.21.15.148:0/901937631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5630002857e0 0x563000283890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 -- 172.21.15.148:0/901937631 >> 172.21.15.148:0/901937631 conn(0x56300025cf10 msgr2=0x563000287ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 -- 172.21.15.148:0/901937631 shutdown_connections 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.461+0000 7f6e83569740 1 -- 172.21.15.148:0/901937631 wait complete. 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.462+0000 7f6e83569740 1 -- start start 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.462+0000 7f6e83569740 1 Processor -- start 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x5630002a2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000284a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300028a7a0 con 0x5630002a4ad0 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e781ffa20 con 0x563000287230 2024-04-25T10:06:34.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56300028ace0 con 0x5630002857e0 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 10 client.0 _ll_register_callbacks cb 0x563000257c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51270/0 (socket says 172.21.15.148:51270) 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2235902383 learned_addr learned my addr 172.21.15.148:0/2235902383 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x5630002a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000284a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 msgr2=0x5630002a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x5630002a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 msgr2=0x563000284a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000284a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 -- 172.21.15.148:0/2235902383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630002855f0 con 0x5630002a4ad0 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7dd24640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000284a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x5630002a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:34.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e7ed26640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002979d0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f6e7002ada0 tx=0x7f6e700019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e677154391d89d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e597fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e597fa640 1 -- 172.21.15.148:0/2235902383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70037040 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e597fa640 1 -- 172.21.15.148:0/2235902383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e70007da0 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.463+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e68087040 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e597fa640 1 -- 172.21.15.148:0/2235902383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e70001470 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 10 client.11385 fetch_fsmap learned FSMap version 106 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x5630002a1790 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e597fa640 1 -- 172.21.15.148:0/2235902383 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68087040 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e597fa640 1 -- 172.21.15.148:0/2235902383 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e7003d040 con 0x5630002a4ad0 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 10 client.11385 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 10 client.11385 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 20 client.11385 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x563000298080 0x56300025ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 4 client.11385 mds_command: new command op to 11268 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:34.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 --> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5630001ccf50 con 0x563000298080 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x563000298080 0x56300025ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e597fa640 10 client.11385 ms_handle_connect on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e7e525640 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x563000298080 0x56300025ade0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1415ae46e6730bc7 server_cookie=8d076cde90ddb005 in_seq=0 out_seq=0 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e597fa640 1 -- 172.21.15.148:0/2235902383 <== mds.0 v2:172.21.15.148:6836/1743980733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x5630001ccf50 con 0x563000298080 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e597fa640 10 client.11385 handle_command_reply: tid=0 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.464+0000 7f6e83569740 1 client.11385 shutdown 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 msgr2=0x5630002979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002979d0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f6e7002ada0 tx=0x7f6e700019c0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 shutdown_connections 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x563000298080 0x56300025ade0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630002a4ad0 0x5630002979d0 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563000287230 0x563000284a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e597fa640 0 client.11385 ms_handle_reset on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:34.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 --2- 172.21.15.148:0/2235902383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5630002857e0 0x5630002a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 >> 172.21.15.148:0/2235902383 conn(0x56300025cf10 msgr2=0x5630002567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 shutdown_connections 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 1 -- 172.21.15.148:0/2235902383 wait complete. 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e83569740 20 client.11385 trim_cache size 0 max 16384 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 11268, 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:06:34.482 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.5742572709999996, 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 106, 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 185, 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 185, 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 9.9672566440000008 2024-04-25T10:06:34.483 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e5c06b500 msgr2=0x7f6e5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6e740051f0 tx=0x7f6e74009040 comp rx=0 tx=0).stop 2024-04-25T10:06:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 msgr2=0x7f6e7809ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 0x7f6e7809ba10 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f6e68011560 tx=0x7f6e68011a20 comp rx=0 tx=0).stop 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 shutdown_connections 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e781e67b0 0x7f6e781e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e7809b590 0x7f6e7809ba10 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 --2- 172.21.15.148:0/4123469175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e64004d40 0x7f6e781431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 >> 172.21.15.148:0/4123469175 conn(0x7f6e780b5af0 msgr2=0x7f6e7806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 shutdown_connections 2024-04-25T10:06:34.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.465+0000 7f6e80789640 1 -- 172.21.15.148:0/4123469175 wait complete. 2024-04-25T10:06:34.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:34.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.577+0000 7f5518a9e640 1 Processor -- start 2024-04-25T10:06:34.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.577+0000 7f5518a9e640 1 -- start start 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5518a9e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5514154ee0 0x7f55140cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5518a9e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f55140ce590 0x7f551406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5518a9e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5514071480 con 0x7f55140cfcf0 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5518a9e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5514072090 con 0x7f5514154ee0 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5518a9e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5514071230 con 0x7f55140ce590 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f551259c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5514154ee0 0x7f55140cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5511d9b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f55140cfcf0 0x7f55140cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33636/0 (socket says 172.21.15.148:33636) 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5511d9b640 1 -- 172.21.15.148:0/2198457363 learned_addr learned my addr 172.21.15.148:0/2198457363 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f5512d9d640 1 --2- 172.21.15.148:0/2198457363 >> v2:172.21.15.188:3300/0 conn(0x7f55140ce590 0x7f551406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f551259c640 1 -- 172.21.15.148:0/2198457363 >> v2:172.21.15.188:3300/0 conn(0x7f55140ce590 msgr2=0x7f551406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f551259c640 1 --2- 172.21.15.148:0/2198457363 >> v2:172.21.15.188:3300/0 conn(0x7f55140ce590 0x7f551406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f551259c640 1 -- 172.21.15.148:0/2198457363 >> v1:172.21.15.148:6789/0 conn(0x7f55140cfcf0 legacy=0x7f55140cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f551259c640 1 -- 172.21.15.148:0/2198457363 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514071760 con 0x7f5514154ee0 2024-04-25T10:06:34.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.578+0000 7f551159a640 1 -- 172.21.15.148:0/2198457363 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161512018 0 0) 0x7f5514071480 con 0x7f55140cfcf0 2024-04-25T10:06:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.579+0000 7f551259c640 1 --2- 172.21.15.148:0/2198457363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5514154ee0 0x7f55140cf7b0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f55140d0700 tx=0x7f550402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaed79358de2c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.579+0000 7f551159a640 1 -- 172.21.15.148:0/2198457363 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504037070 con 0x7f5514154ee0 2024-04-25T10:06:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.579+0000 7f551159a640 1 -- 172.21.15.148:0/2198457363 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504032440 con 0x7f5514154ee0 2024-04-25T10:06:34.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.579+0000 7f551159a640 1 -- 172.21.15.148:0/2198457363 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504032810 con 0x7f5514154ee0 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.579+0000 7f5518a9e640 1 -- 172.21.15.148:0/2198457363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5514154ee0 msgr2=0x7f55140cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.579+0000 7f5518a9e640 1 --2- 172.21.15.148:0/2198457363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5514154ee0 0x7f55140cf7b0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f55140d0700 tx=0x7f550402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.580+0000 7f5518a9e640 1 -- 172.21.15.148:0/2198457363 shutdown_connections 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.580+0000 7f5518a9e640 1 --2- 172.21.15.148:0/2198457363 >> v2:172.21.15.188:3300/0 conn(0x7f55140ce590 0x7f551406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.580+0000 7f5518a9e640 1 --2- 172.21.15.148:0/2198457363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5514154ee0 0x7f55140cf7b0 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.580+0000 7f5518a9e640 1 -- 172.21.15.148:0/2198457363 >> 172.21.15.148:0/2198457363 conn(0x7f551406abe0 msgr2=0x7f551406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.580+0000 7f5518a9e640 1 -- 172.21.15.148:0/2198457363 shutdown_connections 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.580+0000 7f5518a9e640 1 -- 172.21.15.148:0/2198457363 wait complete. 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.581+0000 7f5518a9e640 1 Processor -- start 2024-04-25T10:06:34.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.581+0000 7f5518a9e640 1 -- start start 2024-04-25T10:06:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 0x7f55140ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55140cfcf0 0x7f55141f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5514154ee0 0x7f5514158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55141552c0 con 0x7f55140ce590 2024-04-25T10:06:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514071d20 con 0x7f55140cfcf0 2024-04-25T10:06:34.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5514155440 con 0x7f5514154ee0 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 0x7f55140ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 0x7f55140ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51282/0 (socket says 172.21.15.148:51282) 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 -- 172.21.15.148:0/1267976312 learned_addr learned my addr 172.21.15.148:0/1267976312 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5512d9d640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5514154ee0 0x7f5514158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5511d9b640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55140cfcf0 0x7f55141f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 -- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5514154ee0 msgr2=0x7f5514158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5514154ee0 0x7f5514158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 -- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55140cfcf0 msgr2=0x7f55141f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55140cfcf0 0x7f55141f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 -- 172.21.15.148:0/1267976312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55141579e0 con 0x7f55140ce590 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f551259c640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 0x7f55140ada90 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f5504009890 tx=0x7f55040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504037040 con 0x7f55140ce590 2024-04-25T10:06:34.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504042070 con 0x7f55140ce590 2024-04-25T10:06:34.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504030bd0 con 0x7f55140ce590 2024-04-25T10:06:34.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5514157cc0 con 0x7f55140ce590 2024-04-25T10:06:34.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.582+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5514158280 con 0x7f55140ce590 2024-04-25T10:06:34.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.584+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55141584c0 con 0x7f55140ce590 2024-04-25T10:06:34.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.584+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5504047460 con 0x7f55140ce590 2024-04-25T10:06:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.584+0000 7f5502ffd640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54f406b4d0 0x7f54f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.584+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55040aafc0 con 0x7f55140ce590 2024-04-25T10:06:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.584+0000 7f5511d9b640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54f406b4d0 0x7f54f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.585+0000 7f5511d9b640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54f406b4d0 0x7f54f406d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5508007fd0 tx=0x7f5508009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.589+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5504032e20 con 0x7f55140ce590 2024-04-25T10:06:34.799 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.794+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f551406d360 con 0x7f55140ce590 2024-04-25T10:06:34.800 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.794+0000 7f5502ffd640 1 -- 172.21.15.148:0/1267976312 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5504079cb0 con 0x7f55140ce590 2024-04-25T10:06:34.800 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54f406b4d0 msgr2=0x7f54f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54f406b4d0 0x7f54f406d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f5508007fd0 tx=0x7f5508009040 comp rx=0 tx=0).stop 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 msgr2=0x7f55140ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 0x7f55140ada90 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f5504009890 tx=0x7f55040046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 shutdown_connections 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f54f406b4d0 0x7f54f406d990 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5514154ee0 0x7f5514158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.802 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55140cfcf0 0x7f55141f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 --2- 172.21.15.148:0/1267976312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55140ce590 0x7f55140ada90 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 >> 172.21.15.148:0/1267976312 conn(0x7f551406abe0 msgr2=0x7f551406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 shutdown_connections 2024-04-25T10:06:34.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:34.796+0000 7f5518a9e640 1 -- 172.21.15.148:0/1267976312 wait complete. 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:34.813 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:34.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:34.814 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:06:34.828 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:34.828 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:34.828 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:34.829 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42041sec preferred_lft 42041sec 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:34.830 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:06:34.831 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.831 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:34.831 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:34.831 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:06:34.831 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:34.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:34.831 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:34.863 INFO:tasks.ceph.mds.a.smithi148.stderr:2024-04-25T10:06:34.858+0000 7f0610096640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:34.884 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:06:34.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:34.886 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:06:34.942 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:06:34.942 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:34.942 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:06:34.942 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:06:34.942 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:34.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:34.943 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:06:34.943 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:06:34.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:06:34.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:06:34.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:06:34.943 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:06:35.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:35.026 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:06:35.081 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:35.081 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:35.081 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:35.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:35.081 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:06:35.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:35.145 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:35.200 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:35.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:35.200 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:35.254 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:06:35.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:35.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:35.336 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:35.336 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:35.336 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:06:35.344 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:35.344 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:35.344 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:06:35.345 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:06:35.345 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:06:35.354 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:06:35.356 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:35.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:35.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:35.421 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:06:35.479 INFO:teuthology.orchestra.run.smithi148.stdout:client9142 2024-04-25T10:06:35.480 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:35.480 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:35.535 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:35.535 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:35.590 DEBUG:teuthology.orchestra.run.smithi148:> stdin-killer -- python3 -c ' 2024-04-25T10:06:35.590 DEBUG:teuthology.orchestra.run.smithi148:> import os 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> import uuid 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> n = 1000 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> create_hard_links = False 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> if 0 > 0: 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> create_hard_links = True 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> dpath = os.path.dirname(path) 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> fnameprefix = os.path.basename(path) 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> try: 2024-04-25T10:06:35.591 DEBUG:teuthology.orchestra.run.smithi148:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> for i in range(n): 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> f.write(f"{i}") 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> f.flush() 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(f.fileno()) 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> os.unlink(fpath) 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(dirfd) 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> if create_hard_links: 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> for j in range(0): 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:06:35.592 DEBUG:teuthology.orchestra.run.smithi148:> if False: 2024-04-25T10:06:35.593 DEBUG:teuthology.orchestra.run.smithi148:> os.fsync(dirfd) 2024-04-25T10:06:35.593 DEBUG:teuthology.orchestra.run.smithi148:> finally: 2024-04-25T10:06:35.593 DEBUG:teuthology.orchestra.run.smithi148:> os.close(dirfd) 2024-04-25T10:06:35.593 DEBUG:teuthology.orchestra.run.smithi148:> ' 2024-04-25T10:06:36.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:36.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:36.155 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T10:06:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.289+0000 7f8ff9df4640 1 Processor -- start 2024-04-25T10:06:36.295 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.289+0000 7f8ff9df4640 1 -- start start 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff9df4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ff40d0ab0 0x7f8ff4168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff9df4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40d2420 con 0x7f8ff415dac0 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff9df4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff40d2700 con 0x7f8ff40d34c0 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff9df4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff40d3030 con 0x7f8ff40d0ab0 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8ff40d0ab0 0x7f8ff4168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff37fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8ff415dac0 0x7f8ff40d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33640/0 (socket says 172.21.15.148:33640) 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff37fe640 1 -- 172.21.15.148:0/2778235400 learned_addr learned my addr 172.21.15.148:0/2778235400 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2778235400 >> v1:172.21.15.188:6789/0 conn(0x7f8ff40d34c0 legacy=0x7f8ff40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2778235400 >> v1:172.21.15.148:6789/0 conn(0x7f8ff415dac0 legacy=0x7f8ff40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2778235400 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff40d29e0 con 0x7f8ff40d0ab0 2024-04-25T10:06:36.296 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.290+0000 7f8ff27fc640 1 -- 172.21.15.148:0/2778235400 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455347724 0 0) 0x7f8ff40d2420 con 0x7f8ff415dac0 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.291+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/2778235400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff4168670 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40d31b0 tx=0x7f8fec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c39d366401c103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.291+0000 7f8ff27fc640 1 -- 172.21.15.148:0/2778235400 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007740 con 0x7f8ff40d0ab0 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.291+0000 7f8ff27fc640 1 -- 172.21.15.148:0/2778235400 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec0078e0 con 0x7f8ff40d0ab0 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.291+0000 7f8ff27fc640 1 -- 172.21.15.148:0/2778235400 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007cb0 con 0x7f8ff40d0ab0 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 -- 172.21.15.148:0/2778235400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff40d0ab0 msgr2=0x7f8ff4168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/2778235400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff4168670 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40d31b0 tx=0x7f8fec02a730 comp rx=0 tx=0).stop 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 -- 172.21.15.148:0/2778235400 shutdown_connections 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/2778235400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff4168670 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 -- 172.21.15.148:0/2778235400 >> 172.21.15.148:0/2778235400 conn(0x7f8ff4061c30 msgr2=0x7f8ff406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 -- 172.21.15.148:0/2778235400 shutdown_connections 2024-04-25T10:06:36.298 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.292+0000 7f8ff9df4640 1 -- 172.21.15.148:0/2778235400 wait complete. 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.293+0000 7f8ff9df4640 1 Processor -- start 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.293+0000 7f8ff9df4640 1 -- start start 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ff40d34c0 0x7f8ff41f0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 0x7f8ff40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff415dea0 con 0x7f8ff415dac0 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40d2100 con 0x7f8ff40d0ab0 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40b97d0 con 0x7f8ff40d34c0 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 0x7f8ff40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 0x7f8ff40b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51306/0 (socket says 172.21.15.148:51306) 2024-04-25T10:06:36.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 -- 172.21.15.148:0/449516138 learned_addr learned my addr 172.21.15.148:0/449516138 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ff40d34c0 0x7f8ff41f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 -- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ff40d34c0 msgr2=0x7f8ff41f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ff40d34c0 0x7f8ff41f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 -- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ff40d0ab0 msgr2=0x7f8ff40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 -- 172.21.15.148:0/449516138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff41f42b0 con 0x7f8ff415dac0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ff40d34c0 0x7f8ff41f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 0x7f8ff40b77a0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f8fec02ad20 tx=0x7f8fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8fdbfff640 1 -- 172.21.15.148:0/449516138 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec040070 con 0x7f8ff415dac0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8fdbfff640 1 -- 172.21.15.148:0/449516138 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec031c40 con 0x7f8ff415dac0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff40cc8b0 con 0x7f8ff415dac0 2024-04-25T10:06:36.302 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.294+0000 7f8fdbfff640 1 -- 172.21.15.148:0/449516138 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec03c620 con 0x7f8ff415dac0 2024-04-25T10:06:36.303 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.295+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff40cd150 con 0x7f8ff415dac0 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.296+0000 7f8fdbfff640 1 -- 172.21.15.148:0/449516138 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fec039040 con 0x7f8ff415dac0 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.296+0000 7f8fdbfff640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8fe006b4d0 0x7f8fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.297+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8fe006b4d0 0x7f8fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.297+0000 7f8fdbfff640 1 -- 172.21.15.148:0/449516138 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fec0ab980 con 0x7f8ff415dac0 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.297+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8fe006b4d0 0x7f8fe006d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000f60 tx=0x7f8fe4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 Processor -- start 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 -- start start 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64b10 0x5635eec67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.306 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64ef0 0x5635eec67860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 0x5635eec3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635eeba0da0 con 0x5635eec37770 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635eeb49b80 con 0x5635eec64ef0 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.298+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635eeb9ce40 con 0x5635eec64b10 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 0x5635eec3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 0x5635eec3af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51314/0 (socket says 172.21.15.148:51314) 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2554066465 learned_addr learned my addr 172.21.15.148:0/2554066465 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64ef0 0x5635eec67860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64b10 msgr2=0x5635eec67320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64b10 0x5635eec67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64b10 0x5635eec67320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64ef0 msgr2=0x5635eec67860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64ef0 0x5635eec67860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 -- 172.21.15.148:0/2554066465 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635eeb53f10 con 0x5635eec37770 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 0x5635eec3af80 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f8fec008000 tx=0x7f8fec079a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a184df9daedc446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8fd97fa640 1 -- 172.21.15.148:0/2554066465 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0a9070 con 0x5635eec37770 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8fd97fa640 1 -- 172.21.15.148:0/2554066465 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec07e130 con 0x5635eec37770 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.299+0000 7f8fd97fa640 1 -- 172.21.15.148:0/2554066465 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec091030 con 0x5635eec37770 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2554066465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 msgr2=0x5635eec3af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 0x5635eec3af80 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f8fec008000 tx=0x7f8fec079a70 comp rx=0 tx=0).stop 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2554066465 shutdown_connections 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec37770 0x5635eec3af80 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64ef0 0x5635eec67860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2554066465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64b10 0x5635eec67320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2554066465 >> 172.21.15.148:0/2554066465 conn(0x5635eec399e0 msgr2=0x5635eec82320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2554066465 shutdown_connections 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2554066465 wait complete. 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.300+0000 7f8ffcbcf740 1 -- start start 2024-04-25T10:06:36.308 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 Processor -- start 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec85f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec63590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec5fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635eec3cb90 con 0x5635eec64b10 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635eeb3f4b0 con 0x5635eec955a0 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635eec3ce70 con 0x5635eec64ef0 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ffcbcf740 10 client.0 _ll_register_callbacks cb 0x5635eeb7f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec85f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec85f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51324/0 (socket says 172.21.15.148:51324) 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.301+0000 7f8ff37fe640 1 -- 172.21.15.148:0/320000792 learned_addr learned my addr 172.21.15.148:0/320000792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec5fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec63590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff37fe640 1 -- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 msgr2=0x5635eec63590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec63590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff37fe640 1 -- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 msgr2=0x5635eec5fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec5fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff37fe640 1 -- 172.21.15.148:0/320000792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635ee97f7d0 con 0x5635eec64b10 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec5fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec85f70 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005a70 tx=0x7f8fe4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce96a110cbf9a650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8fd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8fd27fc640 1 -- 172.21.15.148:0/320000792 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4021630 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8fd27fc640 1 -- 172.21.15.148:0/320000792 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fe4020c10 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fec07e130 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8fd27fc640 1 -- 172.21.15.148:0/320000792 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe4020e10 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 10 client.11409 fetch_fsmap learned FSMap version 106 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x5635ee8b53f0 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8fd27fc640 1 -- 172.21.15.148:0/320000792 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec07e130 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8fd27fc640 1 -- 172.21.15.148:0/320000792 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8fe4020420 con 0x5635eec64b10 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 10 client.11409 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 10 client.11409 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 20 client.11409 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec60150 0x5635eec5d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 4 client.11409 mds_command: new command op to 11268 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:36.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 --> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635eebae2e0 con 0x5635eec60150 2024-04-25T10:06:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.302+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec60150 0x5635eec5d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8fd27fc640 10 client.11409 ms_handle_connect on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec60150 0x5635eec5d1a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0798fef17d06c1 server_cookie=4e4a2e0396fe6e6a in_seq=0 out_seq=0 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8fd27fc640 1 -- 172.21.15.148:0/320000792 <== mds.0 v2:172.21.15.148:6836/1743980733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5635eebae2e0 con 0x5635eec60150 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8fd27fc640 10 client.11409 handle_command_reply: tid=0 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 client.11409 shutdown 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 msgr2=0x5635eec85f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec85f70 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005a70 tx=0x7f8fe4000ed0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 shutdown_connections 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec60150 0x5635eec5d1a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec5fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/320000792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec85f70 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 >> 172.21.15.148:0/320000792 conn(0x5635eec37770 msgr2=0x5635eec38420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.318 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.303+0000 7f8fd27fc640 0 client.11409 ms_handle_reset on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.304+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 shutdown_connections 2024-04-25T10:06:36.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.304+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/320000792 wait complete. 2024-04-25T10:06:36.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.304+0000 7f8ffcbcf740 20 client.11409 trim_cache size 0 max 16384 2024-04-25T10:06:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.318+0000 7f8ffcbcf740 1 Processor -- start 2024-04-25T10:06:36.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.318+0000 7f8ffcbcf740 1 -- start start 2024-04-25T10:06:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec64180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4200440 con 0x5635eec64b10 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff41ff5a0 con 0x5635eec955a0 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4146a80 con 0x5635eec64ef0 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec64180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ff37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec64180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51332/0 (socket says 172.21.15.148:51332) 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.319+0000 7f8ff37fe640 1 -- 172.21.15.148:0/2689913455 learned_addr learned my addr 172.21.15.148:0/2689913455 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff37fe640 1 -- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 msgr2=0x5635eec36600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff37fe640 1 -- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 msgr2=0x5635eec36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec36b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff37fe640 1 -- 172.21.15.148:0/2689913455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635ee8750f0 con 0x5635eec64b10 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec64180 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40013a0 tx=0x7f8fe401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b038d4e4057788f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8fd8ff9640 1 -- 172.21.15.148:0/2689913455 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe402f070 con 0x5635eec64b10 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8fd8ff9640 1 -- 172.21.15.148:0/2689913455 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4030040 con 0x5635eec64b10 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.320+0000 7f8fd8ff9640 1 -- 172.21.15.148:0/2689913455 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4020b70 con 0x5635eec64b10 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2689913455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 msgr2=0x5635eec64180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec64180 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40013a0 tx=0x7f8fe401d000 comp rx=0 tx=0).stop 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2689913455 shutdown_connections 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec955a0 0x5635eec36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec64ef0 0x5635eec36600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/2689913455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64b10 0x5635eec64180 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2689913455 >> 172.21.15.148:0/2689913455 conn(0x5635eec37770 msgr2=0x5635eeca4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2689913455 shutdown_connections 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/2689913455 wait complete. 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 -- start start 2024-04-25T10:06:36.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.321+0000 7f8ffcbcf740 1 Processor -- start 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64b10 0x5635eecece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 0x5635eec6a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec955a0 0x5635eec36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635eec63dc0 con 0x5635eec64ef0 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff41fefa0 con 0x5635eec64b10 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635eeb57d90 con 0x5635eec955a0 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ffcbcf740 10 client.0 _ll_register_callbacks cb 0x7f8fec009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 0x5635eec6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 0x5635eec6a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51334/0 (socket says 172.21.15.148:51334) 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 -- 172.21.15.148:0/1090814718 learned_addr learned my addr 172.21.15.148:0/1090814718 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64b10 0x5635eecece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec955a0 0x5635eec36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 -- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec955a0 msgr2=0x5635eec36600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec955a0 0x5635eec36600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 -- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64b10 msgr2=0x5635eecece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64b10 0x5635eecece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 -- 172.21.15.148:0/1090814718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635eec5fe40 con 0x5635eec64ef0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff3fff640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec955a0 0x5635eec36600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8ff2ffd640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 0x5635eec6a760 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x5635eec681c0 tx=0x7f8fe80147d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afe7e7ba84a7199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.322+0000 7f8fd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8fd27fc640 1 -- 172.21.15.148:0/1090814718 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8014e50 con 0x5635eec64ef0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8fd27fc640 1 -- 172.21.15.148:0/1090814718 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fe8006820 con 0x5635eec64ef0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fe4030040 con 0x5635eec64ef0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8fd27fc640 1 -- 172.21.15.148:0/1090814718 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe8029050 con 0x5635eec64ef0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 10 client.11415 fetch_fsmap learned FSMap version 106 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x5635eec6b1c0 con 0x5635eec64ef0 2024-04-25T10:06:36.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8fd27fc640 1 -- 172.21.15.148:0/1090814718 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4030040 con 0x5635eec64ef0 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8fd27fc640 1 -- 172.21.15.148:0/1090814718 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8fe802c020 con 0x5635eec64ef0 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 10 client.11415 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 10 client.11415 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 20 client.11415 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec3ce00 0x5635eec36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 4 client.11415 mds_command: new command op to 11268 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 --> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635eeba8630 con 0x5635eec3ce00 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec3ce00 0x5635eec36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8ff37fe640 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec3ce00 0x5635eec36b40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166e75e102408eb3 server_cookie=a4e32eef6ab29468 in_seq=0 out_seq=0 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.323+0000 7f8fd27fc640 10 client.11415 ms_handle_connect on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8fd27fc640 1 -- 172.21.15.148:0/1090814718 <== mds.0 v2:172.21.15.148:6836/1743980733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5635eeba8630 con 0x5635eec3ce00 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8fd27fc640 10 client.11415 handle_command_reply: tid=0 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 client.11415 shutdown 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 msgr2=0x5635eec6a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 0x5635eec6a760 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x5635eec681c0 tx=0x7f8fe80147d0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 shutdown_connections 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x5635eec3ce00 0x5635eec36b40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635eec955a0 0x5635eec36600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635eec64ef0 0x5635eec6a760 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 --2- 172.21.15.148:0/1090814718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635eec64b10 0x5635eecece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8fd27fc640 0 client.11415 ms_handle_reset on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 >> 172.21.15.148:0/1090814718 conn(0x5635eec37770 msgr2=0x5635eec633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 shutdown_connections 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 1 -- 172.21.15.148:0/1090814718 wait complete. 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ffcbcf740 20 client.11415 trim_cache size 0 max 16384 2024-04-25T10:06:36.335 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:06:36.336 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:06:36.336 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:36.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8fe006b4d0 msgr2=0x7f8fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8fe006b4d0 0x7f8fe006d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000f60 tx=0x7f8fe4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 msgr2=0x7f8ff40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 0x7f8ff40b77a0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f8fec02ad20 tx=0x7f8fec004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 shutdown_connections 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8fe006b4d0 0x7f8fe006d990 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ff415dac0 0x7f8ff40b77a0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ff40d34c0 0x7f8ff41f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 --2- 172.21.15.148:0/449516138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ff40d0ab0 0x7f8ff40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.324+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 >> 172.21.15.148:0/449516138 conn(0x7f8ff4061c30 msgr2=0x7f8ff40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.325+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 shutdown_connections 2024-04-25T10:06:36.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.325+0000 7f8ff9df4640 1 -- 172.21.15.148:0/449516138 wait complete. 2024-04-25T10:06:36.340 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:36.341 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:36.341 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 damage ls 2024-04-25T10:06:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.434+0000 7f113fff9640 1 Processor -- start 2024-04-25T10:06:36.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.434+0000 7f113fff9640 1 -- start start 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113fff9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380b9750 0x7f11380ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113fff9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f11380cebb0 0x7f11380d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113fff9640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1138161d70 con 0x7f113815da90 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113fff9640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1138162600 con 0x7f11380cebb0 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113fff9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11380cf020 con 0x7f11380b9750 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113e596640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f11380cebb0 0x7f11380d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113dd95640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f113815da90 0x7f1138161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33646/0 (socket says 172.21.15.148:33646) 2024-04-25T10:06:36.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113dd95640 1 -- 172.21.15.148:0/429596129 learned_addr learned my addr 172.21.15.148:0/429596129 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.435+0000 7f113e596640 1 --2- 172.21.15.148:0/429596129 >> v2:172.21.15.148:3300/0 conn(0x7f11380cebb0 0x7f11380d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51342/0 (socket says 172.21.15.148:51342) 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113d594640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380b9750 0x7f11380ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113e596640 1 -- 172.21.15.148:0/429596129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380b9750 msgr2=0x7f11380ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113e596640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380b9750 0x7f11380ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113e596640 1 -- 172.21.15.148:0/429596129 >> v1:172.21.15.148:6789/0 conn(0x7f113815da90 legacy=0x7f1138161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113cd93640 1 -- 172.21.15.148:0/429596129 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844138835 0 0) 0x7f1138161d70 con 0x7f113815da90 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113e596640 1 -- 172.21.15.148:0/429596129 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1138162040 con 0x7f11380cebb0 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113d594640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380b9750 0x7f11380ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113e596640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11380cebb0 0x7f11380d2310 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f11300097a0 tx=0x7f113002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cae8ce27c8efcd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113cd93640 1 -- 172.21.15.148:0/429596129 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130037070 con 0x7f11380cebb0 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113cd93640 1 -- 172.21.15.148:0/429596129 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113002ae50 con 0x7f11380cebb0 2024-04-25T10:06:36.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.436+0000 7f113cd93640 1 -- 172.21.15.148:0/429596129 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130032660 con 0x7f11380cebb0 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 -- 172.21.15.148:0/429596129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11380cebb0 msgr2=0x7f11380d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11380cebb0 0x7f11380d2310 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f11300097a0 tx=0x7f113002a100 comp rx=0 tx=0).stop 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 -- 172.21.15.148:0/429596129 shutdown_connections 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11380cebb0 0x7f11380d2310 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 --2- 172.21.15.148:0/429596129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380b9750 0x7f11380ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 -- 172.21.15.148:0/429596129 >> 172.21.15.148:0/429596129 conn(0x7f1138061a20 msgr2=0x7f1138061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 -- 172.21.15.148:0/429596129 shutdown_connections 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.437+0000 7f113fff9640 1 -- 172.21.15.148:0/429596129 wait complete. 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.438+0000 7f113fff9640 1 Processor -- start 2024-04-25T10:06:36.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.438+0000 7f113fff9640 1 -- start start 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113fff9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380cebb0 0x7f1138155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113fff9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 0x7f11380c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113fff9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11380d4330 0x7f11380c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113fff9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380d1df0 con 0x7f113815da90 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113fff9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380d1b10 con 0x7f11380d4330 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113fff9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113815de70 con 0x7f11380cebb0 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 0x7f11380c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 0x7f11380c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51344/0 (socket says 172.21.15.148:51344) 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 -- 172.21.15.148:0/2432463810 learned_addr learned my addr 172.21.15.148:0/2432463810 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113e596640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11380d4330 0x7f11380c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113dd95640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380cebb0 0x7f1138155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 -- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380cebb0 msgr2=0x7f1138155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380cebb0 0x7f1138155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 -- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11380d4330 msgr2=0x7f11380c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11380d4330 0x7f11380c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113d594640 1 -- 172.21.15.148:0/2432463810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380d47e0 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113e596640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11380d4330 0x7f11380c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.439+0000 7f113dd95640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380cebb0 0x7f1138155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.440+0000 7f113d594640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 0x7f11380c7eb0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f112800c7f0 tx=0x7f112800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.440+0000 7f112e7fc640 1 -- 172.21.15.148:0/2432463810 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128004230 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.440+0000 7f112e7fc640 1 -- 172.21.15.148:0/2432463810 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11280043d0 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.440+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11380c85c0 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.440+0000 7f112e7fc640 1 -- 172.21.15.148:0/2432463810 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128007a70 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.440+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11380c9080 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.441+0000 7f112e7fc640 1 -- 172.21.15.148:0/2432463810 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1128007c10 con 0x7f113815da90 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.442+0000 7f112e7fc640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.442+0000 7f112e7fc640 1 -- 172.21.15.148:0/2432463810 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f112808a410 con 0x7f113815da90 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.442+0000 7f113dd95640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.442+0000 7f113dd95640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f111c06b4d0 0x7f111c06d990 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1134007fd0 tx=0x7f1134009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 Processor -- start 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 -- start start 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e23e80 0x564e26e267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26df6700 0x564e26df9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26d5fe00 con 0x564e26e23aa0 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26d08d20 con 0x564e26e23e80 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e26d5bea0 con 0x564e26df6700 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f113dd95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f113dd95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e262b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51350/0 (socket says 172.21.15.148:51350) 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.443+0000 7f113dd95640 1 -- 172.21.15.148:0/1203640190 learned_addr learned my addr 172.21.15.148:0/1203640190 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113dd95640 1 -- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26df6700 msgr2=0x564e26df9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113e596640 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26df6700 0x564e26df9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113d594640 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e23e80 0x564e26e267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113dd95640 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26df6700 0x564e26df9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113dd95640 1 -- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e23e80 msgr2=0x564e26e267f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113dd95640 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e23e80 0x564e26e267f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113dd95640 1 -- 172.21.15.148:0/1203640190 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e26d2adf0 con 0x564e26e23aa0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113d594640 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e23e80 0x564e26e267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f113dd95640 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e262b0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fa0 tx=0x7f1134010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34de7954a0660974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f111b7fe640 1 -- 172.21.15.148:0/1203640190 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134007820 con 0x564e26e23aa0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f111b7fe640 1 -- 172.21.15.148:0/1203640190 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11340079c0 con 0x564e26e23aa0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.444+0000 7f111b7fe640 1 -- 172.21.15.148:0/1203640190 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134020650 con 0x564e26e23aa0 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 -- 172.21.15.148:0/1203640190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 msgr2=0x564e26e262b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e262b0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fa0 tx=0x7f1134010000 comp rx=0 tx=0).stop 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 -- 172.21.15.148:0/1203640190 shutdown_connections 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26df6700 0x564e26df9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e23e80 0x564e26e267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1203640190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e262b0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 -- 172.21.15.148:0/1203640190 >> 172.21.15.148:0/1203640190 conn(0x564e26df8970 msgr2=0x564e26e412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 -- 172.21.15.148:0/1203640190 shutdown_connections 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 -- 172.21.15.148:0/1203640190 wait complete. 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 -- start start 2024-04-25T10:06:36.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.445+0000 7f1142dd8740 1 Processor -- start 2024-04-25T10:06:36.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e3c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26dfbb20 con 0x564e26e23e80 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26cfe590 con 0x564e26e3be20 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e26dfbe00 con 0x564e26e23aa0 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f1142dd8740 10 client.0 _ll_register_callbacks cb 0x564e26d0ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f113d594640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e3b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f113d594640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e3b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51364/0 (socket says 172.21.15.148:51364) 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.446+0000 7f113d594640 1 -- 172.21.15.148:0/4280721968 learned_addr learned my addr 172.21.15.148:0/4280721968 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113e596640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e3c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113dd95640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113d594640 1 -- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 msgr2=0x564e26e198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113d594640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113d594640 1 -- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 msgr2=0x564e26e3c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113d594640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e3c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113d594640 1 -- 172.21.15.148:0/4280721968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e26a743f0 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113e596640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e3c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113dd95640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113d594640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e3b8e0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f1128060be0 tx=0x7f11280591c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111a818581f21af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1118ff9640 1 -- 172.21.15.148:0/4280721968 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112801e040 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1118ff9640 1 -- 172.21.15.148:0/4280721968 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1128088bd0 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11340079c0 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1118ff9640 1 -- 172.21.15.148:0/4280721968 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f112808a270 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 10 client.11430 fetch_fsmap learned FSMap version 106 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x564e26b3e7d0 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1118ff9640 1 -- 172.21.15.148:0/4280721968 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11340079c0 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1118ff9640 1 -- 172.21.15.148:0/4280721968 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1128057e50 con 0x564e26e23e80 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 10 client.11430 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:06:36.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 10 client.11430 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 20 client.11430 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e1a350 0x564e26e1a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 4 client.11430 mds_command: new command op to 11268 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 --> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e26d6d340 con 0x564e26e1a350 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.447+0000 7f113dd95640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e1a350 0x564e26e1a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1118ff9640 10 client.11430 ms_handle_connect on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f113dd95640 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e1a350 0x564e26e1a750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3df551c8745cf3 server_cookie=483b361b2dcec57e in_seq=0 out_seq=0 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1118ff9640 1 -- 172.21.15.148:0/4280721968 <== mds.0 v2:172.21.15.148:6836/1743980733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e26d6d340 con 0x564e26e1a350 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1118ff9640 10 client.11430 handle_command_reply: tid=0 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 client.11430 shutdown 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 msgr2=0x564e26e3b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e3b8e0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f1128060be0 tx=0x7f11280591c0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 shutdown_connections 2024-04-25T10:06:36.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e1a350 0x564e26e1a750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e3c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1118ff9640 0 client.11430 ms_handle_reset on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e3b8e0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 --2- 172.21.15.148:0/4280721968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.448+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 >> 172.21.15.148:0/4280721968 conn(0x564e26df6700 msgr2=0x564e26e22120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.449+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 shutdown_connections 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.449+0000 7f1142dd8740 1 -- 172.21.15.148:0/4280721968 wait complete. 2024-04-25T10:06:36.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.449+0000 7f1142dd8740 20 client.11430 trim_cache size 0 max 16384 2024-04-25T10:06:36.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.460+0000 7f1142dd8740 1 Processor -- start 2024-04-25T10:06:36.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.460+0000 7f1142dd8740 1 -- start start 2024-04-25T10:06:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e43920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e1a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f1142dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1138200390 con 0x564e26e23e80 2024-04-25T10:06:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11381ff580 con 0x564e26e3be20 2024-04-25T10:06:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11380cc430 con 0x564e26e23aa0 2024-04-25T10:06:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e1a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e1a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51376/0 (socket says 172.21.15.148:51376) 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 -- 172.21.15.148:0/1221401655 learned_addr learned my addr 172.21.15.148:0/1221401655 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113e596640 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 -- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 msgr2=0x564e26e43920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e43920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 -- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 msgr2=0x564e26e1ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e1ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113d594640 1 -- 172.21.15.148:0/1221401655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e26e26b60 con 0x564e26e23e80 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.461+0000 7f113e596640 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e1ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f113d594640 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e1a730 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f112805ccf0 tx=0x7f112801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbf1b8d245928a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f111affd640 1 -- 172.21.15.148:0/1221401655 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112801e040 con 0x564e26e23e80 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f111affd640 1 -- 172.21.15.148:0/1221401655 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128061110 con 0x564e26e23e80 2024-04-25T10:06:36.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f111affd640 1 -- 172.21.15.148:0/1221401655 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128088bd0 con 0x564e26e23e80 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 -- 172.21.15.148:0/1221401655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 msgr2=0x564e26e1a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e1a730 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f112805ccf0 tx=0x7f112801b000 comp rx=0 tx=0).stop 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 -- 172.21.15.148:0/1221401655 shutdown_connections 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e1ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23e80 0x564e26e1a730 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 --2- 172.21.15.148:0/1221401655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23aa0 0x564e26e43920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 -- 172.21.15.148:0/1221401655 >> 172.21.15.148:0/1221401655 conn(0x564e26df6700 msgr2=0x564e26e19700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 -- 172.21.15.148:0/1221401655 shutdown_connections 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.462+0000 7f1142dd8740 1 -- 172.21.15.148:0/1221401655 wait complete. 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.463+0000 7f1142dd8740 1 -- start start 2024-04-25T10:06:36.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.463+0000 7f1142dd8740 1 Processor -- start 2024-04-25T10:06:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23e80 0x564e26eabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e22d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26e3c4f0 con 0x564e26e23aa0 2024-04-25T10:06:36.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380d2ed0 con 0x564e26e3be20 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e26e27320 con 0x564e26e23e80 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 10 client.0 _ll_register_callbacks cb 0x7f112800cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51384/0 (socket says 172.21.15.148:51384) 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 -- 172.21.15.148:0/3172579183 learned_addr learned my addr 172.21.15.148:0/3172579183 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113e596640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e22d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113d594640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23e80 0x564e26eabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 -- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23e80 msgr2=0x564e26eabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23e80 0x564e26eabf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 -- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 msgr2=0x564e26e22d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e22d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 -- 172.21.15.148:0/3172579183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e26dfb520 con 0x564e26e23aa0 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f113dd95640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e377c0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f11340012e0 tx=0x7f1134001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b424e79cf537ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1118ff9640 1 -- 172.21.15.148:0/3172579183 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113402d630 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1118ff9640 1 -- 172.21.15.148:0/3172579183 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f113402d7d0 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.464+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1128061110 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1118ff9640 1 -- 172.21.15.148:0/3172579183 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f113403c050 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 10 client.11436 fetch_fsmap learned FSMap version 106 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x564e26e438f0 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1118ff9640 1 -- 172.21.15.148:0/3172579183 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128061110 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1118ff9640 1 -- 172.21.15.148:0/3172579183 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 106) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f113403f020 con 0x564e26e23aa0 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 10 client.11436 fetch_fsmap finished waiting for FSMap version 106 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 10 client.11436 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 20 client.11436 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e22750 0x564e26e23290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 4 client.11436 mds_command: new command op to 11268 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:36.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 --> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e26e27690 con 0x564e26e22750 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f113d594640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e22750 0x564e26e23290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1118ff9640 10 client.11436 ms_handle_connect on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f113d594640 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e22750 0x564e26e23290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1076e2c39867e98d server_cookie=77c8b8aed688b2b3 in_seq=0 out_seq=0 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1118ff9640 1 -- 172.21.15.148:0/3172579183 <== mds.0 v2:172.21.15.148:6836/1743980733 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e26e27690 con 0x564e26e22750 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1118ff9640 10 client.11436 handle_command_reply: tid=0 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.465+0000 7f1142dd8740 1 client.11436 shutdown 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 msgr2=0x564e26e377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e377c0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f11340012e0 tx=0x7f1134001400 comp rx=0 tx=0).stop 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 shutdown_connections 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:6836/1743980733,v1:172.21.15.148:6837/1743980733] conn(0x564e26e22750 0x564e26e23290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564e26e3be20 0x564e26e22d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564e26e23e80 0x564e26eabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 --2- 172.21.15.148:0/3172579183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564e26e23aa0 0x564e26e377c0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1118ff9640 0 client.11436 ms_handle_reset on v2:172.21.15.148:6836/1743980733 2024-04-25T10:06:36.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 >> 172.21.15.148:0/3172579183 conn(0x564e26df6700 msgr2=0x564e26e1a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 shutdown_connections 2024-04-25T10:06:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 1 -- 172.21.15.148:0/3172579183 wait complete. 2024-04-25T10:06:36.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f1142dd8740 20 client.11436 trim_cache size 0 max 16384 2024-04-25T10:06:36.484 INFO:teuthology.orchestra.run.smithi148.stdout:[] 2024-04-25T10:06:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f111c06b4d0 msgr2=0x7f111c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f111c06b4d0 0x7f111c06d990 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1134007fd0 tx=0x7f1134009040 comp rx=0 tx=0).stop 2024-04-25T10:06:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 msgr2=0x7f11380c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 0x7f11380c7eb0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f112800c7f0 tx=0x7f112800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 shutdown_connections 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11380d4330 0x7f11380c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f113815da90 0x7f11380c7eb0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 --2- 172.21.15.148:0/2432463810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11380cebb0 0x7f1138155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 >> 172.21.15.148:0/2432463810 conn(0x7f1138061a20 msgr2=0x7f11380b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 shutdown_connections 2024-04-25T10:06:36.486 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.466+0000 7f113fff9640 1 -- 172.21.15.148:0/2432463810 wait complete. 2024-04-25T10:06:36.486 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:36.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:36.523 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:36.566 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:36.567 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:36.567 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:36.582 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:36.582 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:36.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:36.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.769+0000 7f1a6c0db640 1 Processor -- start 2024-04-25T10:06:36.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.769+0000 7f1a6c0db640 1 -- start start 2024-04-25T10:06:36.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a6c0db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 0x7f1a6406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a6c0db640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a640721d0 0x7f1a640b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a6c0db640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a64072d70 con 0x7f1a6406f550 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a6c0db640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6406f9b0 con 0x7f1a640721d0 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a6c0db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6406db20 con 0x7f1a6415dd80 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a69e77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 0x7f1a6406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a6a678640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a640721d0 0x7f1a640b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a69e77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 0x7f1a6406c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:52282/0 (socket says 172.21.15.148:52282) 2024-04-25T10:06:36.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a69e77640 1 -- 172.21.15.148:0/4218880791 learned_addr learned my addr 172.21.15.148:0/4218880791 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a69e77640 1 -- 172.21.15.148:0/4218880791 >> v2:172.21.15.188:3300/0 conn(0x7f1a640721d0 msgr2=0x7f1a640b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a69e77640 1 --2- 172.21.15.148:0/4218880791 >> v2:172.21.15.188:3300/0 conn(0x7f1a640721d0 0x7f1a640b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.770+0000 7f1a69e77640 1 -- 172.21.15.148:0/4218880791 >> v1:172.21.15.188:6789/0 conn(0x7f1a6406f550 legacy=0x7f1a64071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.771+0000 7f1a69e77640 1 -- 172.21.15.148:0/4218880791 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6406de00 con 0x7f1a6415dd80 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.771+0000 7f1a6a678640 1 --2- 172.21.15.148:0/4218880791 >> v2:172.21.15.188:3300/0 conn(0x7f1a640721d0 0x7f1a640b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.771+0000 7f1a69e77640 1 --2- 172.21.15.148:0/4218880791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 0x7f1a6406c370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009870 tx=0x7f1a6002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b47b41e1412ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.771+0000 7f1a68e75640 1 -- 172.21.15.148:0/4218880791 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60037070 con 0x7f1a6415dd80 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.771+0000 7f1a68e75640 1 -- 172.21.15.148:0/4218880791 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60032440 con 0x7f1a6415dd80 2024-04-25T10:06:36.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.771+0000 7f1a68e75640 1 -- 172.21.15.148:0/4218880791 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60032810 con 0x7f1a6415dd80 2024-04-25T10:06:36.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 -- 172.21.15.148:0/4218880791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 msgr2=0x7f1a6406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/4218880791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 0x7f1a6406c370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009870 tx=0x7f1a6002a130 comp rx=0 tx=0).stop 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 -- 172.21.15.148:0/4218880791 shutdown_connections 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/4218880791 >> v2:172.21.15.188:3300/0 conn(0x7f1a640721d0 0x7f1a640b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/4218880791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a6415dd80 0x7f1a6406c370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 -- 172.21.15.148:0/4218880791 >> 172.21.15.148:0/4218880791 conn(0x7f1a640af670 msgr2=0x7f1a640b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 -- 172.21.15.148:0/4218880791 shutdown_connections 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.772+0000 7f1a6c0db640 1 -- 172.21.15.148:0/4218880791 wait complete. 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.773+0000 7f1a6c0db640 1 Processor -- start 2024-04-25T10:06:36.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.773+0000 7f1a6c0db640 1 -- start start 2024-04-25T10:06:36.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6c0db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a640721d0 0x7f1a64142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6c0db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 0x7f1a64152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6c0db640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a64152ff0 0x7f1a641e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6c0db640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640b3430 con 0x7f1a6415dd80 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6c0db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6406ba70 con 0x7f1a640721d0 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6c0db640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640b35b0 con 0x7f1a64152ff0 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 0x7f1a64152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 0x7f1a64152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51390/0 (socket says 172.21.15.148:51390) 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 -- 172.21.15.148:0/732612038 learned_addr learned my addr 172.21.15.148:0/732612038 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69e77640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a640721d0 0x7f1a64142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a6a678640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a64152ff0 0x7f1a641e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 -- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a64152ff0 msgr2=0x7f1a641e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a64152ff0 0x7f1a641e7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 -- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a640721d0 msgr2=0x7f1a64142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a640721d0 0x7f1a64142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.774+0000 7f1a69676640 1 -- 172.21.15.148:0/732612038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64142fa0 con 0x7f1a6415dd80 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.775+0000 7f1a69676640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 0x7f1a64152ab0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c400 tx=0x7f1a5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.775+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540052d0 con 0x7f1a6415dd80 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.775+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a54005490 con 0x7f1a6415dd80 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.775+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a641e8b50 con 0x7f1a6415dd80 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.775+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54007260 con 0x7f1a6415dd80 2024-04-25T10:06:36.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.775+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a641e9110 con 0x7f1a6415dd80 2024-04-25T10:06:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.776+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a64153400 con 0x7f1a6415dd80 2024-04-25T10:06:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.776+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a54007400 con 0x7f1a6415dd80 2024-04-25T10:06:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.777+0000 7f1a5a7fc640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a4806b4d0 0x7f1a4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.777+0000 7f1a69e77640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a4806b4d0 0x7f1a4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.777+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a54089860 con 0x7f1a6415dd80 2024-04-25T10:06:36.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.777+0000 7f1a69e77640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a4806b4d0 0x7f1a4806d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1a60034000 tx=0x7f1a6002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.781+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a64153400 con 0x7f1a6415dd80 2024-04-25T10:06:36.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.958+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a64073990 con 0x7f1a6415dd80 2024-04-25T10:06:36.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.959+0000 7f1a5a7fc640 1 -- 172.21.15.148:0/732612038 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f1a54057f00 con 0x7f1a6415dd80 2024-04-25T10:06:36.965 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:36.965 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":185,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:30.890882+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:36.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.961+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a4806b4d0 msgr2=0x7f1a4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.961+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a4806b4d0 0x7f1a4806d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1a60034000 tx=0x7f1a6002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:36.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.961+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 msgr2=0x7f1a64152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.961+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 0x7f1a64152ab0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c400 tx=0x7f1a5400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 shutdown_connections 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1a4806b4d0 0x7f1a4806d990 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a64152ff0 0x7f1a641e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a6415dd80 0x7f1a64152ab0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 --2- 172.21.15.148:0/732612038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a640721d0 0x7f1a64142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 >> 172.21.15.148:0/732612038 conn(0x7f1a6406f550 msgr2=0x7f1a640b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 shutdown_connections 2024-04-25T10:06:36.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:36.962+0000 7f1a6c0db640 1 -- 172.21.15.148:0/732612038 wait complete. 2024-04-25T10:06:36.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:27.479372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:28.192397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:36.979 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.074+0000 7f48f68f5640 1 Processor -- start 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.074+0000 7f48f68f5640 1 -- start start 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48f68f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f48f015dd80 0x7f48f00b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48f68f5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f00ad570 0x7f48f006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48f68f5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f00b3930 con 0x7f48f006abe0 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48f68f5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f006e410 con 0x7f48f00ad570 2024-04-25T10:06:37.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48f68f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f00b3c10 con 0x7f48f015dd80 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48f4e92640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f48f00ad570 0x7f48f006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48effff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f48f006abe0 0x7f48f006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33660/0 (socket says 172.21.15.148:33660) 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.075+0000 7f48effff640 1 -- 172.21.15.148:0/1380799858 learned_addr learned my addr 172.21.15.148:0/1380799858 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48f4e92640 1 -- 172.21.15.148:0/1380799858 >> v2:172.21.15.188:3300/0 conn(0x7f48f015dd80 msgr2=0x7f48f00b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48f4e92640 1 --2- 172.21.15.148:0/1380799858 >> v2:172.21.15.188:3300/0 conn(0x7f48f015dd80 0x7f48f00b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48f4e92640 1 -- 172.21.15.148:0/1380799858 >> v1:172.21.15.148:6789/0 conn(0x7f48f006abe0 legacy=0x7f48f006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48f4e92640 1 -- 172.21.15.148:0/1380799858 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f006e660 con 0x7f48f00ad570 2024-04-25T10:06:37.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48eeffd640 1 -- 172.21.15.148:0/1380799858 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308466635 0 0) 0x7f48f00b3930 con 0x7f48f006abe0 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48f4e92640 1 --2- 172.21.15.148:0/1380799858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f006ee60 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f48f0071360 tx=0x7f48e802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad77534fb422829a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48eeffd640 1 -- 172.21.15.148:0/1380799858 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8007740 con 0x7f48f00ad570 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48eeffd640 1 -- 172.21.15.148:0/1380799858 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e80078e0 con 0x7f48f00ad570 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.076+0000 7f48eeffd640 1 -- 172.21.15.148:0/1380799858 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8007cb0 con 0x7f48f00ad570 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 -- 172.21.15.148:0/1380799858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 msgr2=0x7f48f006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 --2- 172.21.15.148:0/1380799858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f006ee60 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f48f0071360 tx=0x7f48e802a680 comp rx=0 tx=0).stop 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 -- 172.21.15.148:0/1380799858 shutdown_connections 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 --2- 172.21.15.148:0/1380799858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f006ee60 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 --2- 172.21.15.148:0/1380799858 >> v2:172.21.15.188:3300/0 conn(0x7f48f015dd80 0x7f48f00b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 -- 172.21.15.148:0/1380799858 >> 172.21.15.148:0/1380799858 conn(0x7f48f00b5af0 msgr2=0x7f48f00b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 -- 172.21.15.148:0/1380799858 shutdown_connections 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.077+0000 7f48f68f5640 1 -- 172.21.15.148:0/1380799858 wait complete. 2024-04-25T10:06:37.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.078+0000 7f48f68f5640 1 Processor -- start 2024-04-25T10:06:37.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.078+0000 7f48f68f5640 1 -- start start 2024-04-25T10:06:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f68f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f006abe0 0x7f48f0151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f68f5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f68f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f015dd80 0x7f48f015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f68f5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f0070280 con 0x7f48f00ad570 2024-04-25T10:06:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f68f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00b57d0 con 0x7f48f006abe0 2024-04-25T10:06:37.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f68f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f0070400 con 0x7f48f015dd80 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f0157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51398/0 (socket says 172.21.15.148:51398) 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 -- 172.21.15.148:0/2222415595 learned_addr learned my addr 172.21.15.148:0/2222415595 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48effff640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f006abe0 0x7f48f0151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f4e92640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f015dd80 0x7f48f015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 -- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f015dd80 msgr2=0x7f48f015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f015dd80 0x7f48f015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 -- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f006abe0 msgr2=0x7f48f0151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f006abe0 0x7f48f0151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48ef7fe640 1 -- 172.21.15.148:0/2222415595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f014fa10 con 0x7f48f00ad570 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.079+0000 7f48f4e92640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f015dd80 0x7f48f015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.080+0000 7f48ef7fe640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f0157df0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f48e400b410 tx=0x7f48e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.080+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4015020 con 0x7f48f00ad570 2024-04-25T10:06:37.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.080+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4011850 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.080+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011c40 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.080+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f014fcf0 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.080+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f015a270 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.081+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f00685b0 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.081+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48e4018020 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.082+0000 7f48ecff9640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48dc06b4d0 0x7f48dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.082+0000 7f48effff640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48dc06b4d0 0x7f48dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.082+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f48e408a280 con 0x7f48f00ad570 2024-04-25T10:06:37.089 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.082+0000 7f48effff640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48dc06b4d0 0x7f48dc06d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f48e0000f60 tx=0x7f48e0007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.091 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.086+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48f00685b0 con 0x7f48f00ad570 2024-04-25T10:06:37.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.257+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48f00b3590 con 0x7f48f00ad570 2024-04-25T10:06:37.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.258+0000 7f48ecff9640 1 -- 172.21.15.148:0/2222415595 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48f00b3590 con 0x7f48f00ad570 2024-04-25T10:06:37.263 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:37.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.259+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48dc06b4d0 msgr2=0x7f48dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.259+0000 7f48f68f5640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48dc06b4d0 0x7f48dc06d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f48e0000f60 tx=0x7f48e0007b50 comp rx=0 tx=0).stop 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.259+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 msgr2=0x7f48f0157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.259+0000 7f48f68f5640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f0157df0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f48e400b410 tx=0x7f48e400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 shutdown_connections 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f48dc06b4d0 0x7f48dc06d990 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f015dd80 0x7f48f015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48f00ad570 0x7f48f0157df0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 --2- 172.21.15.148:0/2222415595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f006abe0 0x7f48f0151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 >> 172.21.15.148:0/2222415595 conn(0x7f48f00b5af0 msgr2=0x7f48f00b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 shutdown_connections 2024-04-25T10:06:37.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.260+0000 7f48f68f5640 1 -- 172.21.15.148:0/2222415595 wait complete. 2024-04-25T10:06:37.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:37.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.371+0000 7fbd65e5d640 1 Processor -- start 2024-04-25T10:06:37.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.371+0000 7fbd65e5d640 1 -- start start 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd65e5d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbd6006fbd0 0x7fbd600ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd65e5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd600b4d80 con 0x7fbd60155f80 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd65e5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd600cef70 con 0x7fbd600ce730 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd65e5d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600b5390 con 0x7fbd6006fbd0 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbd6006fbd0 0x7fbd600ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5effd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbd6006fbd0 0x7fbd600ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51410/0 (socket says 172.21.15.148:51410) 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5f7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fbd60155f80 0x7fbd6006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33662/0 (socket says 172.21.15.148:33662) 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5effd640 1 -- 172.21.15.148:0/2881971997 learned_addr learned my addr 172.21.15.148:0/2881971997 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5effd640 1 -- 172.21.15.148:0/2881971997 >> v1:172.21.15.188:6789/0 conn(0x7fbd600ce730 legacy=0x7fbd600b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5effd640 1 -- 172.21.15.148:0/2881971997 >> v1:172.21.15.148:6789/0 conn(0x7fbd60155f80 legacy=0x7fbd6006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5effd640 1 -- 172.21.15.148:0/2881971997 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600b5020 con 0x7fbd6006fbd0 2024-04-25T10:06:37.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.372+0000 7fbd5e7fc640 1 -- 172.21.15.148:0/2881971997 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960164619 0 0) 0x7fbd600b4d80 con 0x7fbd60155f80 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.373+0000 7fbd5effd640 1 --2- 172.21.15.148:0/2881971997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd6006fbd0 0x7fbd600ce1f0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009850 tx=0x7fbd5402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7f0d0dd46b64c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.373+0000 7fbd5e7fc640 1 -- 172.21.15.148:0/2881971997 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54004230 con 0x7fbd6006fbd0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.373+0000 7fbd5e7fc640 1 -- 172.21.15.148:0/2881971997 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd540043d0 con 0x7fbd6006fbd0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.373+0000 7fbd5e7fc640 1 -- 172.21.15.148:0/2881971997 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54003810 con 0x7fbd6006fbd0 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2881971997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd6006fbd0 msgr2=0x7fbd600ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2881971997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd6006fbd0 0x7fbd600ce1f0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009850 tx=0x7fbd5402a2e0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2881971997 shutdown_connections 2024-04-25T10:06:37.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2881971997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd6006fbd0 0x7fbd600ce1f0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2881971997 >> 172.21.15.148:0/2881971997 conn(0x7fbd60061600 msgr2=0x7fbd600619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2881971997 shutdown_connections 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.374+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2881971997 wait complete. 2024-04-25T10:06:37.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.375+0000 7fbd65e5d640 1 Processor -- start 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.375+0000 7fbd65e5d640 1 -- start start 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd6006fbd0 0x7fbd6014df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 0x7fbd6014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd60155f80 0x7fbd600b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600cf250 con 0x7fbd600ce730 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600d07f0 con 0x7fbd60155f80 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600cf3d0 con 0x7fbd6006fbd0 2024-04-25T10:06:37.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 0x7fbd6014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 0x7fbd6014b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51416/0 (socket says 172.21.15.148:51416) 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 -- 172.21.15.148:0/2707038825 learned_addr learned my addr 172.21.15.148:0/2707038825 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5ffff640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd60155f80 0x7fbd600b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5f7fe640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd6006fbd0 0x7fbd6014df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 -- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd6006fbd0 msgr2=0x7fbd6014df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd6006fbd0 0x7fbd6014df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 -- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd60155f80 msgr2=0x7fbd600b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd60155f80 0x7fbd600b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 -- 172.21.15.148:0/2707038825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6014be30 con 0x7fbd600ce730 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5f7fe640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd6006fbd0 0x7fbd6014df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd5effd640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 0x7fbd6014b8f0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54041020 con 0x7fbd600ce730 2024-04-25T10:06:37.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54003c50 con 0x7fbd600ce730 2024-04-25T10:06:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd601f5370 con 0x7fbd600ce730 2024-04-25T10:06:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54006040 con 0x7fbd600ce730 2024-04-25T10:06:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.376+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd601f5930 con 0x7fbd600ce730 2024-04-25T10:06:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.377+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd600d03d0 con 0x7fbd600ce730 2024-04-25T10:06:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.378+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd54044020 con 0x7fbd600ce730 2024-04-25T10:06:37.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.378+0000 7fbd47fff640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.378+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbd540aafb0 con 0x7fbd600ce730 2024-04-25T10:06:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.378+0000 7fbd5f7fe640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.379+0000 7fbd5f7fe640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbd50000f60 tx=0x7fbd50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.383+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd600d03d0 con 0x7fbd600ce730 2024-04-25T10:06:37.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.555+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd60073a00 con 0x7fbd600ce730 2024-04-25T10:06:37.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.555+0000 7fbd47fff640 1 -- 172.21.15.148:0/2707038825 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd5404b030 con 0x7fbd600ce730 2024-04-25T10:06:37.561 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:06:37.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.557+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd4c06b4d0 msgr2=0x7fbd4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.557+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbd50000f60 tx=0x7fbd50009040 comp rx=0 tx=0).stop 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.557+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 msgr2=0x7fbd6014b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.557+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 0x7fbd6014b8f0 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd540046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 shutdown_connections 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd60155f80 0x7fbd600b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd600ce730 0x7fbd6014b8f0 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 --2- 172.21.15.148:0/2707038825 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd6006fbd0 0x7fbd6014df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 >> 172.21.15.148:0/2707038825 conn(0x7fbd60061600 msgr2=0x7fbd600b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 shutdown_connections 2024-04-25T10:06:37.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.558+0000 7fbd65e5d640 1 -- 172.21.15.148:0/2707038825 wait complete. 2024-04-25T10:06:37.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:37.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.668+0000 7f3af6328640 1 Processor -- start 2024-04-25T10:06:37.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3af6328640 1 -- start start 2024-04-25T10:06:37.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3af6328640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3af015dac0 0x7f3af00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3af6328640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3af00d1f30 0x7f3af00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3af6328640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af00d2420 con 0x7f3af00d34c0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3af6328640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af00d3030 con 0x7f3af00d1f30 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3af6328640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af00d2700 con 0x7f3af015dac0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.669+0000 7f3aef7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3af00d1f30 0x7f3af00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3af00d1f30 0x7f3af00d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51424/0 (socket says 172.21.15.148:51424) 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 -- 172.21.15.148:0/4224537557 learned_addr learned my addr 172.21.15.148:0/4224537557 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 -- 172.21.15.148:0/4224537557 >> v2:172.21.15.188:3300/0 conn(0x7f3af015dac0 msgr2=0x7f3af00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 --2- 172.21.15.148:0/4224537557 >> v2:172.21.15.188:3300/0 conn(0x7f3af015dac0 0x7f3af00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 -- 172.21.15.148:0/4224537557 >> v1:172.21.15.188:6789/0 conn(0x7f3af00d34c0 legacy=0x7f3af00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 -- 172.21.15.148:0/4224537557 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00d29e0 con 0x7f3af00d1f30 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.670+0000 7f3aef7fe640 1 --2- 172.21.15.148:0/4224537557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d1f30 0x7f3af00d03a0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f3af00b6520 tx=0x7f3ae402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a793520ad3ba889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3aeeffd640 1 -- 172.21.15.148:0/4224537557 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004030 con 0x7f3af00d1f30 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3aeeffd640 1 -- 172.21.15.148:0/4224537557 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae40041d0 con 0x7f3af00d1f30 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3aeeffd640 1 -- 172.21.15.148:0/4224537557 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4003610 con 0x7f3af00d1f30 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 -- 172.21.15.148:0/4224537557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d1f30 msgr2=0x7f3af00d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.678 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 --2- 172.21.15.148:0/4224537557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d1f30 0x7f3af00d03a0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f3af00b6520 tx=0x7f3ae402a290 comp rx=0 tx=0).stop 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 -- 172.21.15.148:0/4224537557 shutdown_connections 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 --2- 172.21.15.148:0/4224537557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d1f30 0x7f3af00d03a0 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 --2- 172.21.15.148:0/4224537557 >> v2:172.21.15.188:3300/0 conn(0x7f3af015dac0 0x7f3af00d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 -- 172.21.15.148:0/4224537557 >> 172.21.15.148:0/4224537557 conn(0x7f3af0061c30 msgr2=0x7f3af006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 -- 172.21.15.148:0/4224537557 shutdown_connections 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.671+0000 7f3af6328640 1 -- 172.21.15.148:0/4224537557 wait complete. 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.672+0000 7f3af6328640 1 Processor -- start 2024-04-25T10:06:37.679 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 -- start start 2024-04-25T10:06:37.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af00d1f30 0x7f3af00b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 0x7f3af015fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af015dac0 0x7f3af00cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00b97d0 con 0x7f3af00d34c0 2024-04-25T10:06:37.682 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af015dea0 con 0x7f3af00d1f30 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af6328640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00b9950 con 0x7f3af015dac0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 0x7f3af015fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3aef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 0x7f3af015fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51426/0 (socket says 172.21.15.148:51426) 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3aef7fe640 1 -- 172.21.15.148:0/3420439575 learned_addr learned my addr 172.21.15.148:0/3420439575 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3aeffff640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af00d1f30 0x7f3af00b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.673+0000 7f3af48c5640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af015dac0 0x7f3af00cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aef7fe640 1 -- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af015dac0 msgr2=0x7f3af00cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aef7fe640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af015dac0 0x7f3af00cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aef7fe640 1 -- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af00d1f30 msgr2=0x7f3af00b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aef7fe640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af00d1f30 0x7f3af00b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aef7fe640 1 -- 172.21.15.148:0/3420439575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af00b39a0 con 0x7f3af00d34c0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aeffff640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af00d1f30 0x7f3af00b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aef7fe640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 0x7f3af015fa00 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4034040 tx=0x7f3ae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4041020 con 0x7f3af00d34c0 2024-04-25T10:06:37.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae4003c50 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004030 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af00b3c80 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.674+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af00b4240 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.675+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af0073a00 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.675+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ae4044020 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.676+0000 7f3aecff9640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3adc06b500 0x7f3adc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.676+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ae40ab160 con 0x7f3af00d34c0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.676+0000 7f3aeffff640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3adc06b500 0x7f3adc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.676+0000 7f3aeffff640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3adc06b500 0x7f3adc06d9c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009890 tx=0x7f3ae0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.680+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ae404b030 con 0x7f3af00d34c0 2024-04-25T10:06:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.848+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3af00b2e50 con 0x7f3af00d34c0 2024-04-25T10:06:37.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.849+0000 7f3aecff9640 1 -- 172.21.15.148:0/3420439575 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ae4079dc0 con 0x7f3af00d34c0 2024-04-25T10:06:37.854 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3adc06b500 msgr2=0x7f3adc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3adc06b500 0x7f3adc06d9c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009890 tx=0x7f3ae0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 msgr2=0x7f3af015fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 0x7f3af015fa00 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4034040 tx=0x7f3ae40046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 shutdown_connections 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3adc06b500 0x7f3adc06d9c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3af015dac0 0x7f3af00cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3af00d34c0 0x7f3af015fa00 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 --2- 172.21.15.148:0/3420439575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3af00d1f30 0x7f3af00b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 >> 172.21.15.148:0/3420439575 conn(0x7f3af0061c30 msgr2=0x7f3af00ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 shutdown_connections 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.851+0000 7f3af6328640 1 -- 172.21.15.148:0/3420439575 wait complete. 2024-04-25T10:06:37.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:37.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.965+0000 7f279944a640 1 Processor -- start 2024-04-25T10:06:37.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.965+0000 7f279944a640 1 -- start start 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f279944a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 0x7f27940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f279944a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2794161710 0x7f27940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f279944a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279415e000 con 0x7f27940b9330 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f27937fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f27940b9330 0x7f279406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33670/0 (socket says 172.21.15.148:33670) 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f279944a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27940d1de0 con 0x7f2794161710 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f27937fe640 1 -- 172.21.15.148:0/4110441002 learned_addr learned my addr 172.21.15.148:0/4110441002 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f279944a640 1 -- 172.21.15.148:0/4110441002 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2794161c60 con 0x7f279415dc20 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f278bfff640 1 --2- 172.21.15.148:0/4110441002 >> v2:172.21.15.188:3300/0 conn(0x7f2794161710 0x7f27940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f2792ffd640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 0x7f27940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219761536 0 0) 0x7f279415e000 con 0x7f27940b9330 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.966+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2780003520 con 0x7f27940b9330 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470903221 0 0) 0x7f2780003520 con 0x7f27940b9330 2024-04-25T10:06:37.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 msgr2=0x7f27940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 0x7f27940ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 >> v2:172.21.15.188:3300/0 conn(0x7f2794161710 msgr2=0x7f27940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 --2- 172.21.15.148:0/4110441002 >> v2:172.21.15.188:3300/0 conn(0x7f2794161710 0x7f27940cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794161f40 con 0x7f27940b9330 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f278c0019f0 con 0x7f27940b9330 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 >> v1:172.21.15.148:6789/0 conn(0x7f27940b9330 legacy=0x7f279406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 0x7f27800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2780003f30 0x7f2780004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940d1de0 con 0x7f2780004d40 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794161c60 con 0x7f2780003f30 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278000b9c0 con 0x7f279415dc20 2024-04-25T10:06:37.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f278bfff640 1 --2- 172.21.15.148:0/4110441002 >> v2:172.21.15.188:3300/0 conn(0x7f2794161710 0x7f27940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27937fe640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f2792ffd640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 0x7f27800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f278bfff640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2780003f30 0x7f2780004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27937fe640 1 -- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 msgr2=0x7f27800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27937fe640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279415dc20 0x7f27800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27937fe640 1 -- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2780003f30 msgr2=0x7f2780004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27937fe640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2780003f30 0x7f2780004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.967+0000 7f27937fe640 1 -- 172.21.15.148:0/4110441002 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780003520 con 0x7f2780004d40 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f27937fe640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27800051c0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f278c00a870 tx=0x7f2780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449b306f3ea9b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00a4c0 con 0x7f2780004d40 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c02f070 con 0x7f2780004d40 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f27927fc640 1 -- 172.21.15.148:0/4110441002 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c02bcb0 con 0x7f2780004d40 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f279944a640 1 -- 172.21.15.148:0/4110441002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 msgr2=0x7f27800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f279944a640 1 --2- 172.21.15.148:0/4110441002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27800051c0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f278c00a870 tx=0x7f2780005830 comp rx=0 tx=0).stop 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f27937fe640 1 -- 172.21.15.148:0/4110441002 reap_dead start 2024-04-25T10:06:37.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f279944a640 1 -- 172.21.15.148:0/4110441002 shutdown_connections 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.968+0000 7f279944a640 1 -- 172.21.15.148:0/4110441002 >> 172.21.15.148:0/4110441002 conn(0x7f27940d0420 msgr2=0x7f27940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.969+0000 7f279944a640 1 -- 172.21.15.148:0/4110441002 shutdown_connections 2024-04-25T10:06:37.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.969+0000 7f279944a640 1 -- 172.21.15.148:0/4110441002 wait complete. 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 Processor -- start 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 -- start start 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27941544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27940caa70 0x7f27940cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27940cb840 0x7f27940cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27940b5e50 con 0x7f2780004d40 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794162500 con 0x7f27940caa70 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.970+0000 7f279944a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27940b5fd0 con 0x7f27940cb840 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27941544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27941544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51450/0 (socket says 172.21.15.148:51450) 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 -- 172.21.15.148:0/3511227336 learned_addr learned my addr 172.21.15.148:0/3511227336 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f278bfff640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27940caa70 0x7f27940cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 -- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27940cb840 msgr2=0x7f27940cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27940cb840 0x7f27940cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 -- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27940caa70 msgr2=0x7f27940cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27940caa70 0x7f27940cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 -- 172.21.15.148:0/3511227336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27941f3620 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f2792ffd640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27941544b0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2784011920 tx=0x7f2784011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278400faf0 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278400fc90 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784014440 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27940c80a0 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.971+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27940c8660 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.972+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2794073a00 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.972+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2784012040 con 0x7f2780004d40 2024-04-25T10:06:37.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.973+0000 7f278b7fe640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f277406b4d0 0x7f277406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.973+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f278408a9e0 con 0x7f2780004d40 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.973+0000 7f278bfff640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f277406b4d0 0x7f277406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.973+0000 7f278bfff640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f277406b4d0 0x7f277406d990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f279415bf60 tx=0x7f277c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.983 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:37.977+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2784019030 con 0x7f2780004d40 2024-04-25T10:06:38.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.160+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27940d16b0 con 0x7f2780004d40 2024-04-25T10:06:38.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.160+0000 7f278b7fe640 1 -- 172.21.15.148:0/3511227336 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v185) v1 ==== 80+0+0 (secure 0 0 0) 0x7f27840596d0 con 0x7f2780004d40 2024-04-25T10:06:38.166 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f277406b4d0 msgr2=0x7f277406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f277406b4d0 0x7f277406d990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f279415bf60 tx=0x7f277c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 msgr2=0x7f27941544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27941544b0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2784011920 tx=0x7f2784011de0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 shutdown_connections 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f277406b4d0 0x7f277406d990 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27940cb840 0x7f27940cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27940caa70 0x7f27940cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.168 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 --2- 172.21.15.148:0/3511227336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2780004d40 0x7f27941544b0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 >> 172.21.15.148:0/3511227336 conn(0x7f27940d0420 msgr2=0x7f27940b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 shutdown_connections 2024-04-25T10:06:38.169 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:38.162+0000 7f279944a640 1 -- 172.21.15.148:0/3511227336 wait complete. 2024-04-25T10:06:38.179 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:06:38.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:38.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:38.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:06:38.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:38.918 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:38.932 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:38.932 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:38.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:38.948 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:38.948 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:38.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:38.988 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:38.988 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:39.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:39.004 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:39.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.120+0000 7f6862d42640 1 Processor -- start 2024-04-25T10:06:39.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.120+0000 7f6862d42640 1 -- start start 2024-04-25T10:06:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6862d42640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f685c15dc00 0x7f685c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6862d42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0d1dd0 0x7f685c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6862d42640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c15dfe0 con 0x7f685c0ceb20 2024-04-25T10:06:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6862d42640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c0d00c0 con 0x7f685c15dc00 2024-04-25T10:06:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6862d42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0d03a0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6860ade640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f685c15dc00 0x7f685c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f685bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f685c0ceb20 0x7f685c0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33674/0 (socket says 172.21.15.148:33674) 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f685bfff640 1 -- 172.21.15.148:0/162963379 learned_addr learned my addr 172.21.15.148:0/162963379 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f68612df640 1 --2- 172.21.15.148:0/162963379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0d1dd0 0x7f685c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6860ade640 1 -- 172.21.15.148:0/162963379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0d1dd0 msgr2=0x7f685c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6860ade640 1 --2- 172.21.15.148:0/162963379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0d1dd0 0x7f685c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6860ade640 1 -- 172.21.15.148:0/162963379 >> v1:172.21.15.148:6789/0 conn(0x7f685c0ceb20 legacy=0x7f685c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f6860ade640 1 -- 172.21.15.148:0/162963379 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0cc440 con 0x7f685c15dc00 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.121+0000 7f685b7fe640 1 -- 172.21.15.148:0/162963379 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638273743 0 0) 0x7f6848001110 con 0x7f685c0ceb20 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6860ade640 1 --2- 172.21.15.148:0/162963379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c15dc00 0x7f685c0c6df0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f68500097a0 tx=0x7f685002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d57d7158eac42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f685b7fe640 1 -- 172.21.15.148:0/162963379 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850037070 con 0x7f685c15dc00 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f685b7fe640 1 -- 172.21.15.148:0/162963379 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6850032440 con 0x7f685c15dc00 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f685b7fe640 1 -- 172.21.15.148:0/162963379 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850032810 con 0x7f685c15dc00 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 -- 172.21.15.148:0/162963379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c15dc00 msgr2=0x7f685c0c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 --2- 172.21.15.148:0/162963379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c15dc00 0x7f685c0c6df0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f68500097a0 tx=0x7f685002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:39.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 -- 172.21.15.148:0/162963379 shutdown_connections 2024-04-25T10:06:39.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 --2- 172.21.15.148:0/162963379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0d1dd0 0x7f685c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 --2- 172.21.15.148:0/162963379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c15dc00 0x7f685c0c6df0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 -- 172.21.15.148:0/162963379 >> 172.21.15.148:0/162963379 conn(0x7f685c06f5d0 msgr2=0x7f685c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.122+0000 7f6862d42640 1 -- 172.21.15.148:0/162963379 shutdown_connections 2024-04-25T10:06:39.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.123+0000 7f6862d42640 1 -- 172.21.15.148:0/162963379 wait complete. 2024-04-25T10:06:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 Processor -- start 2024-04-25T10:06:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 -- start start 2024-04-25T10:06:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0ceb20 0x7f685c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 0x7f685c1e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f685c15dc00 0x7f685c0b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0cfd90 con 0x7f685c0d1dd0 2024-04-25T10:06:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0cc130 con 0x7f685c15dc00 2024-04-25T10:06:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.124+0000 7f6862d42640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0cdd70 con 0x7f685c0ceb20 2024-04-25T10:06:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 0x7f685c1e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 0x7f685c1e8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51472/0 (socket says 172.21.15.148:51472) 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 -- 172.21.15.148:0/3591488430 learned_addr learned my addr 172.21.15.148:0/3591488430 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f6860ade640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0ceb20 0x7f685c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f68612df640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f685c15dc00 0x7f685c0b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 -- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0ceb20 msgr2=0x7f685c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0ceb20 0x7f685c158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 -- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f685c15dc00 msgr2=0x7f685c0b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f685c15dc00 0x7f685c0b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 -- 172.21.15.148:0/3591488430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c1e9520 con 0x7f685c0d1dd0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f685bfff640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 0x7f685c1e8fe0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f684800c780 tx=0x7f684800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848018070 con 0x7f685c0d1dd0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68480072d0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0b58f0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480047e0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.125+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c0b5eb0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.126+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c073990 con 0x7f685c0d1dd0 2024-04-25T10:06:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.127+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f684801b070 con 0x7f685c0d1dd0 2024-04-25T10:06:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.127+0000 7f68597fa640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f684006b500 0x7f684006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.127+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6848089670 con 0x7f685c0d1dd0 2024-04-25T10:06:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.127+0000 7f6860ade640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f684006b500 0x7f684006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.127+0000 7f6860ade640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f684006b500 0x7f684006d9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6850034000 tx=0x7f685002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.131+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f685c15dfe0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.339+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685c159eb0 con 0x7f685c0d1dd0 2024-04-25T10:06:39.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.340+0000 7f68597fa640 1 -- 172.21.15.148:0/3591488430 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f6848058360 con 0x7f685c0d1dd0 2024-04-25T10:06:39.345 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:39.345 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:29.223742+0000","modified":"2024-04-25T10:06:31.231117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11268":{"gid":11268,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1743980733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1743980733},{"type":"v1","addr":"172.21.15.148:6837","nonce":1743980733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:06:39.346 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 106 2024-04-25T10:06:39.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f684006b500 msgr2=0x7f684006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f684006b500 0x7f684006d9c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6850034000 tx=0x7f685002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 msgr2=0x7f685c1e8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 0x7f685c1e8fe0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f684800c780 tx=0x7f684800cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 shutdown_connections 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f684006b500 0x7f684006d9c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f685c15dc00 0x7f685c0b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685c0d1dd0 0x7f685c1e8fe0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 --2- 172.21.15.148:0/3591488430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f685c0ceb20 0x7f685c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 >> 172.21.15.148:0/3591488430 conn(0x7f685c06f5d0 msgr2=0x7f685c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 shutdown_connections 2024-04-25T10:06:39.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.342+0000 7f6862d42640 1 -- 172.21.15.148:0/3591488430 wait complete. 2024-04-25T10:06:39.359 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:39.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.450+0000 7fe393e12640 1 Processor -- start 2024-04-25T10:06:39.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.450+0000 7fe393e12640 1 -- start start 2024-04-25T10:06:39.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe393e12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe393e12640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c1626e0 con 0x7fe38c0ce330 2024-04-25T10:06:39.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe393e12640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c070270 con 0x7fe38c06abe0 2024-04-25T10:06:39.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe393e12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0cd270 con 0x7fe38c15e380 2024-04-25T10:06:39.457 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe3913ad640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fe38c0ce330 0x7fe38c06fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33684/0 (socket says 172.21.15.148:33684) 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe3913ad640 1 -- 172.21.15.148:0/346177485 learned_addr learned my addr 172.21.15.148:0/346177485 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.451+0000 7fe3923af640 1 --2- 172.21.15.148:0/346177485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe390bac640 1 -- 172.21.15.148:0/346177485 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198027782 0 0) 0x7fe38c1626e0 con 0x7fe38c0ce330 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe390bac640 1 -- 172.21.15.148:0/346177485 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe378003520 con 0x7fe38c0ce330 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe390bac640 1 -- 172.21.15.148:0/346177485 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931520698 0 0) 0x7fe38c070270 con 0x7fe38c06abe0 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe3923af640 1 -- 172.21.15.148:0/346177485 >> v1:172.21.15.188:6789/0 conn(0x7fe38c06abe0 legacy=0x7fe38c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe3923af640 1 -- 172.21.15.148:0/346177485 >> v1:172.21.15.148:6789/0 conn(0x7fe38c0ce330 legacy=0x7fe38c06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe3923af640 1 -- 172.21.15.148:0/346177485 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0cd550 con 0x7fe38c15e380 2024-04-25T10:06:39.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.452+0000 7fe3923af640 1 --2- 172.21.15.148:0/346177485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c15e7f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3840097d0 tx=0x7fe38402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=712369c8d3602b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe390bac640 1 -- 172.21.15.148:0/346177485 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384037070 con 0x7fe38c15e380 2024-04-25T10:06:39.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe390bac640 1 -- 172.21.15.148:0/346177485 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe384032440 con 0x7fe38c15e380 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe390bac640 1 -- 172.21.15.148:0/346177485 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3840327f0 con 0x7fe38c15e380 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 -- 172.21.15.148:0/346177485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 msgr2=0x7fe38c15e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 --2- 172.21.15.148:0/346177485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c15e7f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3840097d0 tx=0x7fe38402a180 comp rx=0 tx=0).stop 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 -- 172.21.15.148:0/346177485 shutdown_connections 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 --2- 172.21.15.148:0/346177485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c15e7f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 -- 172.21.15.148:0/346177485 >> 172.21.15.148:0/346177485 conn(0x7fe38c0b2220 msgr2=0x7fe38c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 -- 172.21.15.148:0/346177485 shutdown_connections 2024-04-25T10:06:39.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.453+0000 7fe393e12640 1 -- 172.21.15.148:0/346177485 wait complete. 2024-04-25T10:06:39.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.454+0000 7fe393e12640 1 Processor -- start 2024-04-25T10:06:39.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.454+0000 7fe393e12640 1 -- start start 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe393e12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 0x7fe38c1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe393e12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe38c0ce330 0x7fe38c15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe393e12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe393e12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c161780 con 0x7fe38c06abe0 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe393e12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0cdb10 con 0x7fe38c0ce330 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe393e12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c15ed30 con 0x7fe38c15e380 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe391bae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 0x7fe38c1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe391bae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 0x7fe38c1579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51474/0 (socket says 172.21.15.148:51474) 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe391bae640 1 -- 172.21.15.148:0/622004437 learned_addr learned my addr 172.21.15.148:0/622004437 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe3913ad640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe38c0ce330 0x7fe38c15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.455+0000 7fe3923af640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe391bae640 1 -- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 msgr2=0x7fe38c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe391bae640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe391bae640 1 -- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe38c0ce330 msgr2=0x7fe38c15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe391bae640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe38c0ce330 0x7fe38c15cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe391bae640 1 -- 172.21.15.148:0/622004437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0c7d30 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe3913ad640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe38c0ce330 0x7fe38c15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe3923af640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe391bae640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 0x7fe38c1579d0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c890 tx=0x7fe38800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388017070 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388003dc0 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c0c8010 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880052b0 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.456+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c0c88b0 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.457+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c0c81e0 con 0x7fe38c06abe0 2024-04-25T10:06:39.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.457+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe38801a070 con 0x7fe38c06abe0 2024-04-25T10:06:39.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.458+0000 7fe3827fc640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe37006b500 0x7fe37006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.458+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe388089ba0 con 0x7fe38c06abe0 2024-04-25T10:06:39.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.458+0000 7fe3913ad640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe37006b500 0x7fe37006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.458+0000 7fe3913ad640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe37006b500 0x7fe37006d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe38c162440 tx=0x7fe37c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.462+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe388058780 con 0x7fe38c06abe0 2024-04-25T10:06:39.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.634+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe38c073990 con 0x7fe38c06abe0 2024-04-25T10:06:39.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.635+0000 7fe3827fc640 1 -- 172.21.15.148:0/622004437 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17000 (secure 0 0 0) 0x7fe388058080 con 0x7fe38c06abe0 2024-04-25T10:06:39.641 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:39.641 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":185,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:30.890882+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:39.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe37006b500 msgr2=0x7fe37006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe37006b500 0x7fe37006d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe38c162440 tx=0x7fe37c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 msgr2=0x7fe38c1579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 0x7fe38c1579d0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c890 tx=0x7fe38800cd50 comp rx=0 tx=0).stop 2024-04-25T10:06:39.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 shutdown_connections 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe37006b500 0x7fe37006d9c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe38c15e380 0x7fe38c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe38c0ce330 0x7fe38c15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 --2- 172.21.15.148:0/622004437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c06abe0 0x7fe38c1579d0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 >> 172.21.15.148:0/622004437 conn(0x7fe38c0b2220 msgr2=0x7fe38c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 shutdown_connections 2024-04-25T10:06:39.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.637+0000 7fe393e12640 1 -- 172.21.15.148:0/622004437 wait complete. 2024-04-25T10:06:39.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:27.479372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:28.192397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:39.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:39.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.748+0000 7f741ade7640 1 Processor -- start 2024-04-25T10:06:39.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.748+0000 7f741ade7640 1 -- start start 2024-04-25T10:06:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f741ade7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f741415dc20 0x7f74140ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f741ade7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7414161710 0x7f74140cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f741ade7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f741415e000 con 0x7f74140b9330 2024-04-25T10:06:39.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f741ade7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74140d1de0 con 0x7f7414161710 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f741ade7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7414161c60 con 0x7f741415dc20 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f7413fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7414161710 0x7f74140cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f7419384640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f74140b9330 0x7f741406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33696/0 (socket says 172.21.15.148:33696) 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.749+0000 7f7419384640 1 -- 172.21.15.148:0/4051997149 learned_addr learned my addr 172.21.15.148:0/4051997149 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f7413fff640 1 -- 172.21.15.148:0/4051997149 >> v2:172.21.15.188:3300/0 conn(0x7f741415dc20 msgr2=0x7f74140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f7418b83640 1 --2- 172.21.15.148:0/4051997149 >> v2:172.21.15.188:3300/0 conn(0x7f741415dc20 0x7f74140ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f7413fff640 1 --2- 172.21.15.148:0/4051997149 >> v2:172.21.15.188:3300/0 conn(0x7f741415dc20 0x7f74140ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f74137fe640 1 -- 172.21.15.148:0/4051997149 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291545709 0 0) 0x7f741415e000 con 0x7f74140b9330 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f7413fff640 1 -- 172.21.15.148:0/4051997149 >> v1:172.21.15.148:6789/0 conn(0x7f74140b9330 legacy=0x7f741406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f7413fff640 1 -- 172.21.15.148:0/4051997149 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7414161f40 con 0x7f7414161710 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.750+0000 7f7413fff640 1 --2- 172.21.15.148:0/4051997149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140cebb0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f74140d1520 tx=0x7f740802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beca06ac07d30442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f74137fe640 1 -- 172.21.15.148:0/4051997149 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408037070 con 0x7f7414161710 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f74137fe640 1 -- 172.21.15.148:0/4051997149 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408032440 con 0x7f7414161710 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f74137fe640 1 -- 172.21.15.148:0/4051997149 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408032810 con 0x7f7414161710 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 -- 172.21.15.148:0/4051997149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 msgr2=0x7f74140cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 --2- 172.21.15.148:0/4051997149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140cebb0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f74140d1520 tx=0x7f740802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 -- 172.21.15.148:0/4051997149 shutdown_connections 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 --2- 172.21.15.148:0/4051997149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140cebb0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 --2- 172.21.15.148:0/4051997149 >> v2:172.21.15.188:3300/0 conn(0x7f741415dc20 0x7f74140ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 -- 172.21.15.148:0/4051997149 >> 172.21.15.148:0/4051997149 conn(0x7f74140d0420 msgr2=0x7f74140d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 -- 172.21.15.148:0/4051997149 shutdown_connections 2024-04-25T10:06:39.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.751+0000 7f741ade7640 1 -- 172.21.15.148:0/4051997149 wait complete. 2024-04-25T10:06:39.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.752+0000 7f741ade7640 1 Processor -- start 2024-04-25T10:06:39.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.752+0000 7f741ade7640 1 -- start start 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f741ade7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74140b9330 0x7f741414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f741ade7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f741415dc20 0x7f7414160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f741ade7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140b8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f741ade7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140b5e50 con 0x7f7414161710 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f741ade7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414162500 con 0x7f74140b9330 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f741ade7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140b5fd0 con 0x7f741415dc20 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f7419384640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f7419384640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51492/0 (socket says 172.21.15.148:51492) 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f7419384640 1 -- 172.21.15.148:0/1556982661 learned_addr learned my addr 172.21.15.148:0/1556982661 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f7418b83640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74140b9330 0x7f741414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.753+0000 7f7413fff640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f741415dc20 0x7f7414160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f7419384640 1 -- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f741415dc20 msgr2=0x7f7414160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f7419384640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f741415dc20 0x7f7414160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f7419384640 1 -- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74140b9330 msgr2=0x7f741414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f7419384640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74140b9330 0x7f741414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f7419384640 1 -- 172.21.15.148:0/1556982661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74141f6f40 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f7419384640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140b8220 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f740c00c780 tx=0x7f740c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c018070 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c00ce30 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c00a860 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74141f7220 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.754+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74141f77e0 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.755+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74140cf660 con 0x7f7414161710 2024-04-25T10:06:39.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.755+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f740c01b070 con 0x7f7414161710 2024-04-25T10:06:39.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.756+0000 7f74117fa640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f740006b4d0 0x7f740006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.756+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f740c089730 con 0x7f7414161710 2024-04-25T10:06:39.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.756+0000 7f7418b83640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f740006b4d0 0x7f740006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.756+0000 7f7418b83640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f740006b4d0 0x7f740006d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f740400aa20 tx=0x7f7404008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.760+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74140cf660 con 0x7f7414161710 2024-04-25T10:06:39.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.930+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7414073a00 con 0x7f7414161710 2024-04-25T10:06:39.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.931+0000 7f74117fa640 1 -- 172.21.15.148:0/1556982661 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f740c05ce20 con 0x7f7414161710 2024-04-25T10:06:39.936 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f740006b4d0 msgr2=0x7f740006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f740006b4d0 0x7f740006d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f740400aa20 tx=0x7f7404008040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 msgr2=0x7f74140b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140b8220 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f740c00c780 tx=0x7f740c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 shutdown_connections 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f740006b4d0 0x7f740006d990 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7414161710 0x7f74140b8220 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f741415dc20 0x7f7414160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 --2- 172.21.15.148:0/1556982661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74140b9330 0x7f741414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 >> 172.21.15.148:0/1556982661 conn(0x7f74140d0420 msgr2=0x7f74140b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 shutdown_connections 2024-04-25T10:06:39.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:39.933+0000 7f741ade7640 1 -- 172.21.15.148:0/1556982661 wait complete. 2024-04-25T10:06:39.949 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:40.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.043+0000 7f65cee5a640 1 Processor -- start 2024-04-25T10:06:40.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.043+0000 7f65cee5a640 1 -- start start 2024-04-25T10:06:40.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cee5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c81454a0 0x7f65c80aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cee5a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f65c815c970 0x7f65c806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cee5a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80af540 con 0x7f65c806b120 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cee5a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80af820 con 0x7f65c815c970 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cee5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80afb00 con 0x7f65c81454a0 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65c7fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f65c815c970 0x7f65c806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cd3f7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f65c806b120 0x7f65c815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33712/0 (socket says 172.21.15.148:33712) 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65cd3f7640 1 -- 172.21.15.148:0/2785343179 learned_addr learned my addr 172.21.15.148:0/2785343179 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65ccbf6640 1 --2- 172.21.15.148:0/2785343179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c81454a0 0x7f65c80aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65c7fff640 1 -- 172.21.15.148:0/2785343179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c81454a0 msgr2=0x7f65c80aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65c7fff640 1 --2- 172.21.15.148:0/2785343179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c81454a0 0x7f65c80aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65c7fff640 1 -- 172.21.15.148:0/2785343179 >> v1:172.21.15.148:6789/0 conn(0x7f65c806b120 legacy=0x7f65c815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65c7fff640 1 -- 172.21.15.148:0/2785343179 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80afde0 con 0x7f65c815c970 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.044+0000 7f65c77fe640 1 -- 172.21.15.148:0/2785343179 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977594144 0 0) 0x7f65c80af540 con 0x7f65c806b120 2024-04-25T10:06:40.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.045+0000 7f65c7fff640 1 --2- 172.21.15.148:0/2785343179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c815c970 0x7f65c806abe0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097a0 tx=0x7f65bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb945d8324e134d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.045+0000 7f65c77fe640 1 -- 172.21.15.148:0/2785343179 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc037070 con 0x7f65c815c970 2024-04-25T10:06:40.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.045+0000 7f65c77fe640 1 -- 172.21.15.148:0/2785343179 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc032440 con 0x7f65c815c970 2024-04-25T10:06:40.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.045+0000 7f65c77fe640 1 -- 172.21.15.148:0/2785343179 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc032810 con 0x7f65c815c970 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 -- 172.21.15.148:0/2785343179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c815c970 msgr2=0x7f65c806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 --2- 172.21.15.148:0/2785343179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c815c970 0x7f65c806abe0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0097a0 tx=0x7f65bc02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 -- 172.21.15.148:0/2785343179 shutdown_connections 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 --2- 172.21.15.148:0/2785343179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c815c970 0x7f65c806abe0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 --2- 172.21.15.148:0/2785343179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c81454a0 0x7f65c80aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 -- 172.21.15.148:0/2785343179 >> 172.21.15.148:0/2785343179 conn(0x7f65c80b5af0 msgr2=0x7f65c80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 -- 172.21.15.148:0/2785343179 shutdown_connections 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.046+0000 7f65cee5a640 1 -- 172.21.15.148:0/2785343179 wait complete. 2024-04-25T10:06:40.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.047+0000 7f65cee5a640 1 Processor -- start 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.047+0000 7f65cee5a640 1 -- start start 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cee5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65c806b120 0x7f65c813f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cee5a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 0x7f65c81f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cee5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c815c970 0x7f65c815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cee5a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815e620 con 0x7f65c81454a0 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cee5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815e4a0 con 0x7f65c806b120 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cee5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815e7a0 con 0x7f65c815c970 2024-04-25T10:06:40.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 0x7f65c81f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 0x7f65c81f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51514/0 (socket says 172.21.15.148:51514) 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 -- 172.21.15.148:0/3513240057 learned_addr learned my addr 172.21.15.148:0/3513240057 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cd3f7640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c815c970 0x7f65c815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65ccbf6640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65c806b120 0x7f65c813f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 -- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c815c970 msgr2=0x7f65c815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c815c970 0x7f65c815d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 -- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65c806b120 msgr2=0x7f65c813f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65c806b120 0x7f65c813f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 -- 172.21.15.148:0/3513240057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c81f2240 con 0x7f65c81454a0 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65ccbf6640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65c806b120 0x7f65c813f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65cd3f7640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c815c970 0x7f65c815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.048+0000 7f65c7fff640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 0x7f65c81f2430 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0098c0 tx=0x7f65bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.049+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc037040 con 0x7f65c81454a0 2024-04-25T10:06:40.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.049+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc030770 con 0x7f65c81454a0 2024-04-25T10:06:40.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.049+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c81f1f60 con 0x7f65c81454a0 2024-04-25T10:06:40.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.049+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc030b40 con 0x7f65c81454a0 2024-04-25T10:06:40.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.049+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c809bdc0 con 0x7f65c81454a0 2024-04-25T10:06:40.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.050+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c8073a00 con 0x7f65c81454a0 2024-04-25T10:06:40.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.050+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65bc044020 con 0x7f65c81454a0 2024-04-25T10:06:40.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.051+0000 7f65c57fa640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65b406b4d0 0x7f65b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.051+0000 7f65ccbf6640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65b406b4d0 0x7f65b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.051+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65bc0aa940 con 0x7f65c81454a0 2024-04-25T10:06:40.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.051+0000 7f65ccbf6640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65b406b4d0 0x7f65b406d990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f65c81f47c0 tx=0x7f65c00039f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.060 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.055+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65bc032e20 con 0x7f65c81454a0 2024-04-25T10:06:40.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.230+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65c806d370 con 0x7f65c81454a0 2024-04-25T10:06:40.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.230+0000 7f65c57fa640 1 -- 172.21.15.148:0/3513240057 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v185) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65bc079590 con 0x7f65c81454a0 2024-04-25T10:06:40.236 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:06:40.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.232+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65b406b4d0 msgr2=0x7f65b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.232+0000 7f65cee5a640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65b406b4d0 0x7f65b406d990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f65c81f47c0 tx=0x7f65c00039f0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.232+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 msgr2=0x7f65c81f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.232+0000 7f65cee5a640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 0x7f65c81f2430 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0098c0 tx=0x7f65bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 shutdown_connections 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f65b406b4d0 0x7f65b406d990 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65c815c970 0x7f65c815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c81454a0 0x7f65c81f2430 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 --2- 172.21.15.148:0/3513240057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65c806b120 0x7f65c813f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 >> 172.21.15.148:0/3513240057 conn(0x7f65c80b5af0 msgr2=0x7f65c80b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 shutdown_connections 2024-04-25T10:06:40.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.233+0000 7f65cee5a640 1 -- 172.21.15.148:0/3513240057 wait complete. 2024-04-25T10:06:40.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:40.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.344+0000 7f8f25d6a640 1 Processor -- start 2024-04-25T10:06:40.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.344+0000 7f8f25d6a640 1 -- start start 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f25d6a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f2006c330 0x7f8f200b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f25d6a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f200d1d50 con 0x7f8f2006c7e0 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f25d6a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f200d2020 con 0x7f8f20158230 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f25d6a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f20158610 con 0x7f8f2006c330 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8f2006c7e0 0x7f8f2006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33718/0 (socket says 172.21.15.148:33718) 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1ffff640 1 -- 172.21.15.148:0/1880944263 learned_addr learned my addr 172.21.15.148:0/1880944263 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:40.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1effd640 1 --2- 172.21.15.148:0/1880944263 >> v2:172.21.15.188:3300/0 conn(0x7f8f2006c330 0x7f8f200b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1e7fc640 1 -- 172.21.15.148:0/1880944263 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554460936 0 0) 0x7f8f200d1d50 con 0x7f8f2006c7e0 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1e7fc640 1 -- 172.21.15.148:0/1880944263 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f08003520 con 0x7f8f2006c7e0 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1e7fc640 1 -- 172.21.15.148:0/1880944263 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665616244 0 0) 0x7f8f08003520 con 0x7f8f2006c7e0 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1effd640 1 -- 172.21.15.148:0/1880944263 >> v1:172.21.15.188:6789/0 conn(0x7f8f20158230 legacy=0x7f8f200d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1effd640 1 -- 172.21.15.148:0/1880944263 >> v1:172.21.15.148:6789/0 conn(0x7f8f2006c7e0 legacy=0x7f8f2006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.352 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.345+0000 7f8f1effd640 1 -- 172.21.15.148:0/1880944263 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200d2300 con 0x7f8f2006c330 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.346+0000 7f8f1effd640 1 --2- 172.21.15.148:0/1880944263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f200b4e60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009790 tx=0x7f8f1402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53e2d1c351166d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.346+0000 7f8f1e7fc640 1 -- 172.21.15.148:0/1880944263 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14037070 con 0x7f8f2006c330 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.346+0000 7f8f1e7fc640 1 -- 172.21.15.148:0/1880944263 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1402ac30 con 0x7f8f2006c330 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.346+0000 7f8f1e7fc640 1 -- 172.21.15.148:0/1880944263 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f180019f0 con 0x7f8f2006c330 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 -- 172.21.15.148:0/1880944263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 msgr2=0x7f8f200b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/1880944263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f200b4e60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009790 tx=0x7f8f1402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 -- 172.21.15.148:0/1880944263 shutdown_connections 2024-04-25T10:06:40.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/1880944263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f200b4e60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 -- 172.21.15.148:0/1880944263 >> 172.21.15.148:0/1880944263 conn(0x7f8f20061600 msgr2=0x7f8f200619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 -- 172.21.15.148:0/1880944263 shutdown_connections 2024-04-25T10:06:40.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.347+0000 7f8f25d6a640 1 -- 172.21.15.148:0/1880944263 wait complete. 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.348+0000 7f8f25d6a640 1 Processor -- start 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.348+0000 7f8f25d6a640 1 -- start start 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f25d6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f2015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f25d6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 0x7f8f2015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f25d6a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f20158230 0x7f8f200cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f25d6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200d25e0 con 0x7f8f2006c7e0 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f25d6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2006bea0 con 0x7f8f2006c330 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f25d6a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2015e320 con 0x7f8f20158230 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 0x7f8f2015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 0x7f8f2015cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51520/0 (socket says 172.21.15.148:51520) 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 -- 172.21.15.148:0/2928746093 learned_addr learned my addr 172.21.15.148:0/2928746093 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1ffff640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f20158230 0x7f8f200cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1f7fe640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f2015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 -- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f20158230 msgr2=0x7f8f200cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f20158230 0x7f8f200cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 -- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 msgr2=0x7f8f2015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f2015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.349+0000 7f8f1effd640 1 -- 172.21.15.148:0/2928746093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200cd210 con 0x7f8f2006c7e0 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.350+0000 7f8f1effd640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 0x7f8f2015cbf0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098b0 tx=0x7f8f140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.350+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14042020 con 0x7f8f2006c7e0 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.350+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14003c40 con 0x7f8f2006c7e0 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.350+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f200cd4f0 con 0x7f8f2006c7e0 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.350+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14037040 con 0x7f8f2006c7e0 2024-04-25T10:06:40.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.350+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f201588c0 con 0x7f8f2006c7e0 2024-04-25T10:06:40.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.351+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f20073a00 con 0x7f8f2006c7e0 2024-04-25T10:06:40.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.351+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f14045020 con 0x7f8f2006c7e0 2024-04-25T10:06:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.352+0000 7f8f07fff640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.352+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f140aa8f0 con 0x7f8f2006c7e0 2024-04-25T10:06:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.352+0000 7f8f1f7fe640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.352+0000 7f8f1f7fe640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8f10000f60 tx=0x7f8f10009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.356+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f1404c030 con 0x7f8f2006c7e0 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.523+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f200b21c0 con 0x7f8f2006c7e0 2024-04-25T10:06:40.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.524+0000 7f8f07fff640 1 -- 172.21.15.148:0/2928746093 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v185) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f14079140 con 0x7f8f2006c7e0 2024-04-25T10:06:40.531 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:06:40.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f0c06b4d0 msgr2=0x7f8f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8f10000f60 tx=0x7f8f10009040 comp rx=0 tx=0).stop 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 msgr2=0x7f8f2015cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 0x7f8f2015cbf0 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f8f140098b0 tx=0x7f8f140046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 shutdown_connections 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f20158230 0x7f8f200cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f2006c7e0 0x7f8f2015cbf0 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 --2- 172.21.15.148:0/2928746093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2006c330 0x7f8f2015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 >> 172.21.15.148:0/2928746093 conn(0x7f8f20061600 msgr2=0x7f8f2015d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 shutdown_connections 2024-04-25T10:06:40.533 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.526+0000 7f8f25d6a640 1 -- 172.21.15.148:0/2928746093 wait complete. 2024-04-25T10:06:40.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:40.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.634+0000 7f9f96513640 1 Processor -- start 2024-04-25T10:06:40.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.634+0000 7f9f96513640 1 -- start start 2024-04-25T10:06:40.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f96513640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 0x7f9f900cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f96513640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f900ce590 0x7f9f9006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f96513640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f90071480 con 0x7f9f900cfcf0 2024-04-25T10:06:40.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f96513640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f90072090 con 0x7f9f900ce590 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f96513640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900d0150 con 0x7f9f90154ee0 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f8ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 0x7f9f900cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f94ab0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f900ce590 0x7f9f9006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f8ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 0x7f9f900cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:52430/0 (socket says 172.21.15.148:52430) 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f8f7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9f900cfcf0 0x7f9f900cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56410/0 (socket says 172.21.15.148:56410) 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.635+0000 7f9f8ffff640 1 -- 172.21.15.148:0/209534084 learned_addr learned my addr 172.21.15.148:0/209534084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.636+0000 7f9f8ffff640 1 -- 172.21.15.148:0/209534084 >> v2:172.21.15.188:3300/0 conn(0x7f9f900ce590 msgr2=0x7f9f9006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.636+0000 7f9f8ffff640 1 --2- 172.21.15.148:0/209534084 >> v2:172.21.15.188:3300/0 conn(0x7f9f900ce590 0x7f9f9006d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.636+0000 7f9f8ffff640 1 -- 172.21.15.148:0/209534084 >> v1:172.21.15.188:6789/0 conn(0x7f9f900cfcf0 legacy=0x7f9f900cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.636+0000 7f9f8ffff640 1 -- 172.21.15.148:0/209534084 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f90071760 con 0x7f9f90154ee0 2024-04-25T10:06:40.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.636+0000 7f9f94ab0640 1 --2- 172.21.15.148:0/209534084 >> v2:172.21.15.188:3300/0 conn(0x7f9f900ce590 0x7f9f9006d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.636+0000 7f9f8ffff640 1 --2- 172.21.15.148:0/209534084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 0x7f9f900cf7b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009870 tx=0x7f9f8002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef1cdfe4e528e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f8effd640 1 -- 172.21.15.148:0/209534084 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80037070 con 0x7f9f90154ee0 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f8effd640 1 -- 172.21.15.148:0/209534084 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f80032440 con 0x7f9f90154ee0 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f8effd640 1 -- 172.21.15.148:0/209534084 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80032810 con 0x7f9f90154ee0 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 -- 172.21.15.148:0/209534084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 msgr2=0x7f9f900cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 --2- 172.21.15.148:0/209534084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 0x7f9f900cf7b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009870 tx=0x7f9f8002a250 comp rx=0 tx=0).stop 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 -- 172.21.15.148:0/209534084 shutdown_connections 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 --2- 172.21.15.148:0/209534084 >> v2:172.21.15.188:3300/0 conn(0x7f9f900ce590 0x7f9f9006d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 --2- 172.21.15.148:0/209534084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f90154ee0 0x7f9f900cf7b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 -- 172.21.15.148:0/209534084 >> 172.21.15.148:0/209534084 conn(0x7f9f9006abe0 msgr2=0x7f9f9006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 -- 172.21.15.148:0/209534084 shutdown_connections 2024-04-25T10:06:40.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.637+0000 7f9f96513640 1 -- 172.21.15.148:0/209534084 wait complete. 2024-04-25T10:06:40.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.638+0000 7f9f96513640 1 Processor -- start 2024-04-25T10:06:40.646 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 -- start start 2024-04-25T10:06:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f900ce590 0x7f9f90146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f900cfcf0 0x7f9f900ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 0x7f9f90159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90071d20 con 0x7f9f90154ee0 2024-04-25T10:06:40.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90071230 con 0x7f9f900cfcf0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f96513640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f901552c0 con 0x7f9f900ce590 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f94ab0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 0x7f9f90159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f94ab0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 0x7f9f90159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51526/0 (socket says 172.21.15.148:51526) 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.639+0000 7f9f94ab0640 1 -- 172.21.15.148:0/2783258721 learned_addr learned my addr 172.21.15.148:0/2783258721 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f8ffff640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f900ce590 0x7f9f90146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f8f7fe640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f900cfcf0 0x7f9f900ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f94ab0640 1 -- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f900ce590 msgr2=0x7f9f90146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f94ab0640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f900ce590 0x7f9f90146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f94ab0640 1 -- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f900cfcf0 msgr2=0x7f9f900ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f94ab0640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f900cfcf0 0x7f9f900ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f94ab0640 1 -- 172.21.15.148:0/2783258721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f900ad290 con 0x7f9f90154ee0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f8ffff640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f900ce590 0x7f9f90146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f94ab0640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 0x7f9f90159be0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800efc0 tx=0x7f9f88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88018030 con 0x7f9f90154ee0 2024-04-25T10:06:40.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8800cde0 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f900ad570 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88003bb0 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.640+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f901478c0 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.641+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f90073990 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.642+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f8801b070 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.642+0000 7f9f8cff9640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f7c06b500 0x7f9f7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.642+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f8808af00 con 0x7f9f90154ee0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.642+0000 7f9f8ffff640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f7c06b500 0x7f9f7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.642+0000 7f9f8ffff640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f7c06b500 0x7f9f7c06d9c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9f80034000 tx=0x7f9f8002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.652 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.646+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f88010040 con 0x7f9f90154ee0 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.854+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f900ce0c0 con 0x7f9f90154ee0 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.855+0000 7f9f8cff9640 1 -- 172.21.15.148:0/2783258721 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f9f88012340 con 0x7f9f90154ee0 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:29.223742+0000","modified":"2024-04-25T10:06:31.231117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11268":{"gid":11268,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1743980733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1743980733},{"type":"v1","addr":"172.21.15.148:6837","nonce":1743980733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 106 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f7c06b500 msgr2=0x7f9f7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f7c06b500 0x7f9f7c06d9c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9f80034000 tx=0x7f9f8002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 msgr2=0x7f9f90159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 0x7f9f90159be0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800efc0 tx=0x7f9f88009f90 comp rx=0 tx=0).stop 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 shutdown_connections 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f7c06b500 0x7f9f7c06d9c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f90154ee0 0x7f9f90159be0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f900cfcf0 0x7f9f900ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 --2- 172.21.15.148:0/2783258721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f900ce590 0x7f9f90146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 >> 172.21.15.148:0/2783258721 conn(0x7f9f9006abe0 msgr2=0x7f9f900b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 shutdown_connections 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.857+0000 7f9f96513640 1 -- 172.21.15.148:0/2783258721 wait complete. 2024-04-25T10:06:40.875 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:40.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.966+0000 7f829d7bd640 1 Processor -- start 2024-04-25T10:06:40.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.966+0000 7f829d7bd640 1 -- start start 2024-04-25T10:06:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f829d7bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f829815f5f0 0x7f829815fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f829d7bd640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82980cd710 con 0x7f82981602e0 2024-04-25T10:06:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f829d7bd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8298099b70 con 0x7f829815ba00 2024-04-25T10:06:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f829d7bd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829815ff80 con 0x7f829815f5f0 2024-04-25T10:06:40.974 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f82977fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f829815f5f0 0x7f829815fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f82967fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f82981602e0 0x7f82980ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33728/0 (socket says 172.21.15.148:33728) 2024-04-25T10:06:40.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.967+0000 7f82967fc640 1 -- 172.21.15.148:0/480984221 learned_addr learned my addr 172.21.15.148:0/480984221 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:40.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f82977fe640 1 -- 172.21.15.148:0/480984221 >> v1:172.21.15.188:6789/0 conn(0x7f829815ba00 legacy=0x7f829809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f82977fe640 1 -- 172.21.15.148:0/480984221 >> v1:172.21.15.148:6789/0 conn(0x7f82981602e0 legacy=0x7f82980ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.975 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f82977fe640 1 -- 172.21.15.148:0/480984221 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298099e50 con 0x7f829815f5f0 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f82977fe640 1 --2- 172.21.15.148:0/480984221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815f5f0 0x7f829815fa40 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f82880097a0 tx=0x7f828802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb89ac3648f782f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f8295ffb640 1 -- 172.21.15.148:0/480984221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288037070 con 0x7f829815f5f0 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f8295ffb640 1 -- 172.21.15.148:0/480984221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828802adc0 con 0x7f829815f5f0 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.968+0000 7f8295ffb640 1 -- 172.21.15.148:0/480984221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288032660 con 0x7f829815f5f0 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 -- 172.21.15.148:0/480984221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815f5f0 msgr2=0x7f829815fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 --2- 172.21.15.148:0/480984221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815f5f0 0x7f829815fa40 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f82880097a0 tx=0x7f828802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.976 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 -- 172.21.15.148:0/480984221 shutdown_connections 2024-04-25T10:06:40.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 --2- 172.21.15.148:0/480984221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815f5f0 0x7f829815fa40 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 -- 172.21.15.148:0/480984221 >> 172.21.15.148:0/480984221 conn(0x7f82980b3270 msgr2=0x7f82980b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 -- 172.21.15.148:0/480984221 shutdown_connections 2024-04-25T10:06:40.977 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.969+0000 7f829d7bd640 1 -- 172.21.15.148:0/480984221 wait complete. 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.970+0000 7f829d7bd640 1 Processor -- start 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.970+0000 7f829d7bd640 1 -- start start 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f829d7bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 0x7f82980cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f829d7bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829815f5f0 0x7f829815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f829d7bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82981602e0 0x7f82980b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f829d7bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829815bde0 con 0x7f829815ba00 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f829d7bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980cd4b0 con 0x7f829815f5f0 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f829d7bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829815a8c0 con 0x7f82981602e0 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 0x7f82980cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.979 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 0x7f82980cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51546/0 (socket says 172.21.15.148:51546) 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 -- 172.21.15.148:0/3224243676 learned_addr learned my addr 172.21.15.148:0/3224243676 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f82967fc640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829815f5f0 0x7f829815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f82977fe640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82981602e0 0x7f82980b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 -- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82981602e0 msgr2=0x7f82980b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82981602e0 0x7f82980b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 -- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829815f5f0 msgr2=0x7f829815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829815f5f0 0x7f829815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.971+0000 7f8296ffd640 1 -- 172.21.15.148:0/3224243676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980ae670 con 0x7f829815ba00 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.972+0000 7f8296ffd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 0x7f82980cb680 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f829000b3f0 tx=0x7f829000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.972+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290015020 con 0x7f829815ba00 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.972+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290011830 con 0x7f829815ba00 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.972+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980c8d90 con 0x7f829815ba00 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.972+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290011c20 con 0x7f829815ba00 2024-04-25T10:06:40.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.972+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82980c9350 con 0x7f829815ba00 2024-04-25T10:06:40.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.973+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82980d06d0 con 0x7f829815ba00 2024-04-25T10:06:40.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.973+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8290018020 con 0x7f829815ba00 2024-04-25T10:06:40.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.974+0000 7f827f7fe640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f828406b4d0 0x7f828406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.974+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f829008a020 con 0x7f829815ba00 2024-04-25T10:06:40.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.974+0000 7f82967fc640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f828406b4d0 0x7f828406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.982 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.974+0000 7f82967fc640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f828406b4d0 0x7f828406d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f82980b7450 tx=0x7f828c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.984 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:40.978+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82980d06d0 con 0x7f829815ba00 2024-04-25T10:06:41.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.151+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82980710d0 con 0x7f829815ba00 2024-04-25T10:06:41.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.152+0000 7f827f7fe640 1 -- 172.21.15.148:0/3224243676 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17000 (secure 0 0 0) 0x7f82980710d0 con 0x7f829815ba00 2024-04-25T10:06:41.159 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:41.159 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":185,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:30.890882+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f828406b4d0 msgr2=0x7f828406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f828406b4d0 0x7f828406d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f82980b7450 tx=0x7f828c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 msgr2=0x7f82980cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 0x7f82980cb680 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f829000b3f0 tx=0x7f829000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 shutdown_connections 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f828406b4d0 0x7f828406d990 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82981602e0 0x7f82980b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829815f5f0 0x7f829815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 --2- 172.21.15.148:0/3224243676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f829815ba00 0x7f82980cb680 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 >> 172.21.15.148:0/3224243676 conn(0x7f82980b3270 msgr2=0x7f82980b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 shutdown_connections 2024-04-25T10:06:41.162 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.154+0000 7f829d7bd640 1 -- 172.21.15.148:0/3224243676 wait complete. 2024-04-25T10:06:41.172 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:41.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:41.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.263+0000 7ff59c09e640 1 Processor -- start 2024-04-25T10:06:41.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.263+0000 7ff59c09e640 1 -- start start 2024-04-25T10:06:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff59c09e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5940c6df0 0x7ff5940d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff59c09e640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59415dfe0 con 0x7ff5940d1c00 2024-04-25T10:06:41.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff59c09e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940d00c0 con 0x7ff59415dc00 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff59c09e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940d2070 con 0x7ff5940c6df0 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff59a63b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff5940d1c00 0x7ff5940ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33744/0 (socket says 172.21.15.148:33744) 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff59a63b640 1 -- 172.21.15.148:0/2468283830 learned_addr learned my addr 172.21.15.148:0/2468283830 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff599639640 1 --2- 172.21.15.148:0/2468283830 >> v2:172.21.15.188:3300/0 conn(0x7ff5940c6df0 0x7ff5940d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012807681 0 0) 0x7ff58c001490 con 0x7ff5940d1c00 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff580003520 con 0x7ff5940d1c00 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568427965 0 0) 0x7ff5940d00c0 con 0x7ff59415dc00 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff58c001490 con 0x7ff59415dc00 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338325218 0 0) 0x7ff580003520 con 0x7ff5940d1c00 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 >> v2:172.21.15.188:3300/0 conn(0x7ff5940c6df0 msgr2=0x7ff5940d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 --2- 172.21.15.148:0/2468283830 >> v2:172.21.15.188:3300/0 conn(0x7ff5940c6df0 0x7ff5940d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 >> v1:172.21.15.188:6789/0 conn(0x7ff59415dc00 legacy=0x7ff5940ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940d03a0 con 0x7ff5940d1c00 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ff58c001bc0 con 0x7ff5940d1c00 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.264+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 >> v1:172.21.15.148:6789/0 conn(0x7ff5940d1c00 legacy=0x7ff5940ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff598e38640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff580003b40 0x7ff580003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff598e38640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580004800 0x7ff580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff598e38640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5800051c0 0x7ff58000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940d2070 con 0x7ff580003b40 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58000c040 con 0x7ff580004800 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58000c240 con 0x7ff5800051c0 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599e3a640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff580003b40 0x7ff580003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599639640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580004800 0x7ff580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff59a63b640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5800051c0 0x7ff58000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599e3a640 1 -- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5800051c0 msgr2=0x7ff58000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599e3a640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5800051c0 0x7ff58000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599e3a640 1 -- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580004800 msgr2=0x7ff580004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599e3a640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580004800 0x7ff580004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff599e3a640 1 -- 172.21.15.148:0/2468283830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580003520 con 0x7ff580003b40 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.265+0000 7ff59a63b640 1 -- 172.21.15.148:0/2468283830 reap_dead start 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff599e3a640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff580003b40 0x7ff580003f40 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7ff5900018d0 tx=0x7ff580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8523613d9a9473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5900014f0 con 0x7ff580003b40 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5900052d0 con 0x7ff580003b40 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff598e38640 1 -- 172.21.15.148:0/2468283830 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59002aca0 con 0x7ff580003b40 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 -- 172.21.15.148:0/2468283830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff580003b40 msgr2=0x7ff580003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff580003b40 0x7ff580003f40 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7ff5900018d0 tx=0x7ff580005830 comp rx=0 tx=0).stop 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 -- 172.21.15.148:0/2468283830 shutdown_connections 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 --2- 172.21.15.148:0/2468283830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff580003b40 0x7ff580003f40 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 -- 172.21.15.148:0/2468283830 >> 172.21.15.148:0/2468283830 conn(0x7ff59406f5d0 msgr2=0x7ff59406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 -- 172.21.15.148:0/2468283830 shutdown_connections 2024-04-25T10:06:41.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.266+0000 7ff59c09e640 1 -- 172.21.15.148:0/2468283830 wait complete. 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 Processor -- start 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 -- start start 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580003b40 0x7ff594147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 0x7ff5940b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59414da70 0x7ff5940b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cc130 con 0x7ff59414d5f0 2024-04-25T10:06:41.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cc720 con 0x7ff580003b40 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff59c09e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cfd90 con 0x7ff59414da70 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff599639640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 0x7ff5940b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff599639640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 0x7ff5940b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51560/0 (socket says 172.21.15.148:51560) 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.268+0000 7ff599639640 1 -- 172.21.15.148:0/4286571570 learned_addr learned my addr 172.21.15.148:0/4286571570 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599e3a640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580003b40 0x7ff594147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff59a63b640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59414da70 0x7ff5940b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599639640 1 -- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59414da70 msgr2=0x7ff5940b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599639640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59414da70 0x7ff5940b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599639640 1 -- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580003b40 msgr2=0x7ff594147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599639640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580003b40 0x7ff594147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599639640 1 -- 172.21.15.148:0/4286571570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940b7190 con 0x7ff59414d5f0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff599639640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 0x7ff5940b6710 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7ff584011a70 tx=0x7ff584011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400fb50 con 0x7ff59414d5f0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58400fcf0 con 0x7ff59414d5f0 2024-04-25T10:06:41.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5940b7440 con 0x7ff59414d5f0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58400fea0 con 0x7ff59414d5f0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.269+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff594155070 con 0x7ff59414d5f0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.270+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594073990 con 0x7ff59414d5f0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.270+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff584012040 con 0x7ff59414d5f0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.271+0000 7ff58a7fc640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff57806b4d0 0x7ff57806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.271+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff58408a9a0 con 0x7ff59414d5f0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.271+0000 7ff599e3a640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff57806b4d0 0x7ff57806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.271+0000 7ff599e3a640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff57806b4d0 0x7ff57806d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff590032000 tx=0x7ff590009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.281 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.275+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff584019030 con 0x7ff59414d5f0 2024-04-25T10:06:41.488 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.482+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5940c66c0 con 0x7ff59414d5f0 2024-04-25T10:06:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.482+0000 7ff58a7fc640 1 -- 172.21.15.148:0/4286571570 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4965 (secure 0 0 0) 0x7ff584059720 con 0x7ff59414d5f0 2024-04-25T10:06:41.489 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:41.489 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:29.223742+0000","modified":"2024-04-25T10:06:31.231117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11268":{"gid":11268,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1743980733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1743980733},{"type":"v1","addr":"172.21.15.148:6837","nonce":1743980733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:06:41.489 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 106 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff57806b4d0 msgr2=0x7ff57806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff57806b4d0 0x7ff57806d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff590032000 tx=0x7ff590009030 comp rx=0 tx=0).stop 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 msgr2=0x7ff5940b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 0x7ff5940b6710 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7ff584011a70 tx=0x7ff584011f30 comp rx=0 tx=0).stop 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 shutdown_connections 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff57806b4d0 0x7ff57806d990 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff59414da70 0x7ff5940b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff59414d5f0 0x7ff5940b6710 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 --2- 172.21.15.148:0/4286571570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff580003b40 0x7ff594147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 >> 172.21.15.148:0/4286571570 conn(0x7ff59406f5d0 msgr2=0x7ff5940ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 shutdown_connections 2024-04-25T10:06:41.492 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.484+0000 7ff59c09e640 1 -- 172.21.15.148:0/4286571570 wait complete. 2024-04-25T10:06:41.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:41.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.592+0000 7f8352990640 1 Processor -- start 2024-04-25T10:06:41.599 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.592+0000 7f8352990640 1 -- start start 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f8352990640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 0x7f834c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f8352990640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f834c06d290 0x7f834c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f8352990640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834c144b30 con 0x7f834c144750 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f8352990640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834c0712d0 con 0x7f834c06d290 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f8352990640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0715b0 con 0x7f834c0ce920 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f834c144750 0x7f834c06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33748/0 (socket says 172.21.15.148:33748) 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834bfff640 1 -- 172.21.15.148:0/3126347986 learned_addr learned my addr 172.21.15.148:0/3126347986 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:41.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f8350f2d640 1 --2- 172.21.15.148:0/3126347986 >> v2:172.21.15.188:3300/0 conn(0x7f834c06d290 0x7f834c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834b7fe640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 0x7f834c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729818723 0 0) 0x7f834c144b30 con 0x7f834c144750 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8334003520 con 0x7f834c144750 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107160271 0 0) 0x7f8334003520 con 0x7f834c144750 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 msgr2=0x7f834c0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 0x7f834c0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 >> v2:172.21.15.188:3300/0 conn(0x7f834c06d290 msgr2=0x7f834c06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 --2- 172.21.15.148:0/3126347986 >> v2:172.21.15.188:3300/0 conn(0x7f834c06d290 0x7f834c06d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c071890 con 0x7f834c144750 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f833c001670 con 0x7f834c144750 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.593+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 >> v1:172.21.15.148:6789/0 conn(0x7f834c144750 legacy=0x7f834c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834affd640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83340039b0 0x7f8334003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834affd640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f8334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834affd640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 0x7f83340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0712d0 con 0x7f8334004800 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c0715b0 con 0x7f83340039b0 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833400b9c0 con 0x7f834c0ce920 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834b7fe640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f8334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834bfff640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83340039b0 0x7f8334003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f8350f2d640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 0x7f83340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834b7fe640 1 -- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 msgr2=0x7f83340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834b7fe640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c0ce920 0x7f83340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834b7fe640 1 -- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83340039b0 msgr2=0x7f8334003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834b7fe640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83340039b0 0x7f8334003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834b7fe640 1 -- 172.21.15.148:0/3126347986 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334003520 con 0x7f8334004800 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.594+0000 7f834bfff640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83340039b0 0x7f8334003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f834b7fe640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f8334004c80 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f8340009ac0 tx=0x7f8334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0fea11ab635c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340036070 con 0x7f8334004800 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8340031400 con 0x7f8334004800 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f834affd640 1 -- 172.21.15.148:0/3126347986 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340009690 con 0x7f8334004800 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8352990640 1 -- 172.21.15.148:0/3126347986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 msgr2=0x7f8334004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8352990640 1 --2- 172.21.15.148:0/3126347986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f8334004c80 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f8340009ac0 tx=0x7f8334005830 comp rx=0 tx=0).stop 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8350f2d640 1 -- 172.21.15.148:0/3126347986 reap_dead start 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8352990640 1 -- 172.21.15.148:0/3126347986 shutdown_connections 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8352990640 1 -- 172.21.15.148:0/3126347986 >> 172.21.15.148:0/3126347986 conn(0x7f834c06abe0 msgr2=0x7f834c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8352990640 1 -- 172.21.15.148:0/3126347986 shutdown_connections 2024-04-25T10:06:41.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.595+0000 7f8352990640 1 -- 172.21.15.148:0/3126347986 wait complete. 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.596+0000 7f8352990640 1 Processor -- start 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 -- start start 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f834c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c15daf0 0x7f834c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c15d0c0 0x7f834c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c145450 con 0x7f8334004800 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c06f020 con 0x7f834c15daf0 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f8352990640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c1455d0 con 0x7f834c15d0c0 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f834bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f834c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f834bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f834c0ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51582/0 (socket says 172.21.15.148:51582) 2024-04-25T10:06:41.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.597+0000 7f834bfff640 1 -- 172.21.15.148:0/1978355143 learned_addr learned my addr 172.21.15.148:0/1978355143 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834bfff640 1 -- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c15d0c0 msgr2=0x7f834c159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834bfff640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c15d0c0 0x7f834c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834bfff640 1 -- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c15daf0 msgr2=0x7f834c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834b7fe640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c15daf0 0x7f834c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834bfff640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c15daf0 0x7f834c15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834bfff640 1 -- 172.21.15.148:0/1978355143 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c15e4d0 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f834bfff640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f834c0ade50 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f833c00e770 tx=0x7f833c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00ee80 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c018070 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834c15e7b0 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00cad0 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.598+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834c15c380 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.599+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c073990 con 0x7f8334004800 2024-04-25T10:06:41.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.599+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f833c00cc70 con 0x7f8334004800 2024-04-25T10:06:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.600+0000 7f8348ff9640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f833806b500 0x7f833806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.600+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f833c08a1f0 con 0x7f8334004800 2024-04-25T10:06:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.600+0000 7f834b7fe640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f833806b500 0x7f833806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.600+0000 7f834b7fe640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f833806b500 0x7f833806d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8340033000 tx=0x7f8340008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.604+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f833c058ee0 con 0x7f8334004800 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.809+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f834c0d0510 con 0x7f8334004800 2024-04-25T10:06:41.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.809+0000 7f8348ff9640 1 -- 172.21.15.148:0/1978355143 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f833c0587e0 con 0x7f8334004800 2024-04-25T10:06:41.816 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:41.816 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":106,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:29.223742+0000","modified":"2024-04-25T10:06:31.231117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11268},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11268":{"gid":11268,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.148:6837/1743980733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1743980733},{"type":"v1","addr":"172.21.15.148:6837","nonce":1743980733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:06:41.816 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 106 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f833806b500 msgr2=0x7f833806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f833806b500 0x7f833806d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8340033000 tx=0x7f8340008030 comp rx=0 tx=0).stop 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 msgr2=0x7f834c0ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f834c0ade50 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f833c00e770 tx=0x7f833c00ec30 comp rx=0 tx=0).stop 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 shutdown_connections 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f833806b500 0x7f833806d9c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c15d0c0 0x7f834c159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c15daf0 0x7f834c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 --2- 172.21.15.148:0/1978355143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8334004800 0x7f834c0ade50 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 >> 172.21.15.148:0/1978355143 conn(0x7f834c06abe0 msgr2=0x7f834c140c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 shutdown_connections 2024-04-25T10:06:41.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.811+0000 7f8352990640 1 -- 172.21.15.148:0/1978355143 wait complete. 2024-04-25T10:06:41.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:41.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.920+0000 7fd970330640 1 Processor -- start 2024-04-25T10:06:41.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.920+0000 7fd970330640 1 -- start start 2024-04-25T10:06:41.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.921+0000 7fd970330640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 0x7fd9680ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.921+0000 7fd970330640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9680b5f00 0x7fd9680d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.921+0000 7fd970330640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9680cec00 con 0x7fd96815df40 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.921+0000 7fd970330640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9680ce930 con 0x7fd9680b5f00 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.921+0000 7fd970330640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9680b6450 con 0x7fd96815db60 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 0x7fd9680ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96d8cb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9680b5f00 0x7fd9680d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 0x7fd9680ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:52504/0 (socket says 172.21.15.148:52504) 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 -- 172.21.15.148:0/1610953858 learned_addr learned my addr 172.21.15.148:0/1610953858 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e8cd640 1 --1- 172.21.15.148:0/1610953858 >> v1:172.21.15.188:6789/0 conn(0x7fd96815df40 0x7fd968161950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56436/0 (socket says 172.21.15.148:56436) 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 -- 172.21.15.148:0/1610953858 >> v2:172.21.15.188:3300/0 conn(0x7fd9680b5f00 msgr2=0x7fd9680d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 --2- 172.21.15.148:0/1610953858 >> v2:172.21.15.188:3300/0 conn(0x7fd9680b5f00 0x7fd9680d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 -- 172.21.15.148:0/1610953858 >> v1:172.21.15.188:6789/0 conn(0x7fd96815df40 legacy=0x7fd968161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.922+0000 7fd96e0cc640 1 -- 172.21.15.148:0/1610953858 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9680ceee0 con 0x7fd96815db60 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd96e0cc640 1 --2- 172.21.15.148:0/1610953858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 0x7fd9680ce3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd964009e50 tx=0x7fd96402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962e008bf396320f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd96d0ca640 1 -- 172.21.15.148:0/1610953858 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964036070 con 0x7fd96815db60 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd96d0ca640 1 -- 172.21.15.148:0/1610953858 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964031600 con 0x7fd96815db60 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd96d0ca640 1 -- 172.21.15.148:0/1610953858 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9640319d0 con 0x7fd96815db60 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd970330640 1 -- 172.21.15.148:0/1610953858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 msgr2=0x7fd9680ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd970330640 1 --2- 172.21.15.148:0/1610953858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 0x7fd9680ce3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd964009e50 tx=0x7fd96402a680 comp rx=0 tx=0).stop 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd970330640 1 -- 172.21.15.148:0/1610953858 shutdown_connections 2024-04-25T10:06:41.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd970330640 1 --2- 172.21.15.148:0/1610953858 >> v2:172.21.15.188:3300/0 conn(0x7fd9680b5f00 0x7fd9680d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd970330640 1 --2- 172.21.15.148:0/1610953858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd96815db60 0x7fd9680ce3f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.923+0000 7fd970330640 1 -- 172.21.15.148:0/1610953858 >> 172.21.15.148:0/1610953858 conn(0x7fd968061600 msgr2=0x7fd968061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.924+0000 7fd970330640 1 -- 172.21.15.148:0/1610953858 shutdown_connections 2024-04-25T10:06:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.924+0000 7fd970330640 1 -- 172.21.15.148:0/1610953858 wait complete. 2024-04-25T10:06:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 Processor -- start 2024-04-25T10:06:41.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 -- start start 2024-04-25T10:06:41.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9680b5f00 0x7fd968150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd96815db60 0x7fd9680ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 0x7fd9680cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9680d3280 con 0x7fd96815df40 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9680d1d50 con 0x7fd96815db60 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd970330640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9680d3400 con 0x7fd9680b5f00 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.925+0000 7fd96e8cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 0x7fd9680cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 0x7fd9680cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51586/0 (socket says 172.21.15.148:51586) 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 -- 172.21.15.148:0/2871072459 learned_addr learned my addr 172.21.15.148:0/2871072459 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e0cc640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9680b5f00 0x7fd968150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96d8cb640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd96815db60 0x7fd9680ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 -- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9680b5f00 msgr2=0x7fd968150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9680b5f00 0x7fd968150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 -- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd96815db60 msgr2=0x7fd9680ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd96815db60 0x7fd9680ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 -- 172.21.15.148:0/2871072459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96815b5d0 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd96e8cd640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 0x7fd9680cb470 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fd96000c3c0 tx=0x7fd96000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9600074e0 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd960007680 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960004a00 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9680cd4f0 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.926+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9680cac70 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.927+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd968073270 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.928+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd96001a020 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.928+0000 7fd95effd640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94c06b4d0 0x7fd94c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.928+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd960089a90 con 0x7fd96815df40 2024-04-25T10:06:41.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.928+0000 7fd96e0cc640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94c06b4d0 0x7fd94c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.929+0000 7fd96e0cc640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94c06b4d0 0x7fd94c06d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd964009e50 tx=0x7fd964009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:41.932+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd960058810 con 0x7fd96815df40 2024-04-25T10:06:42.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.106+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd968073990 con 0x7fd96815df40 2024-04-25T10:06:42.114 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.108+0000 7fd95effd640 1 -- 172.21.15.148:0/2871072459 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+17000 (secure 0 0 0) 0x7fd960058110 con 0x7fd96815df40 2024-04-25T10:06:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:42.114 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":185,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:30.890882+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:27.479372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:28.192397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94c06b4d0 msgr2=0x7fd94c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94c06b4d0 0x7fd94c06d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd964009e50 tx=0x7fd964009000 comp rx=0 tx=0).stop 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 msgr2=0x7fd9680cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 0x7fd9680cb470 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7fd96000c3c0 tx=0x7fd96000c880 comp rx=0 tx=0).stop 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 shutdown_connections 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd94c06b4d0 0x7fd94c06d990 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd96815df40 0x7fd9680cb470 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd96815db60 0x7fd9680ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 --2- 172.21.15.148:0/2871072459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9680b5f00 0x7fd968150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 >> 172.21.15.148:0/2871072459 conn(0x7fd968061600 msgr2=0x7fd96815c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 shutdown_connections 2024-04-25T10:06:42.118 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.110+0000 7fd970330640 1 -- 172.21.15.148:0/2871072459 wait complete. 2024-04-25T10:06:42.128 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:42.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.219+0000 7f52e11d5640 1 Processor -- start 2024-04-25T10:06:42.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.219+0000 7f52e11d5640 1 -- start start 2024-04-25T10:06:42.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52e11d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52dc161710 0x7f52dc0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52e11d5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc15e000 con 0x7f52dc15dc20 2024-04-25T10:06:42.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52e11d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc0d1de0 con 0x7f52dc0ced80 2024-04-25T10:06:42.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52e11d5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc161c60 con 0x7f52dc161710 2024-04-25T10:06:42.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52db59e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f52dc161710 0x7f52dc0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52dad9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f52dc15dc20 0x7f52dc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:33754/0 (socket says 172.21.15.148:33754) 2024-04-25T10:06:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52dad9d640 1 -- 172.21.15.148:0/1966090762 learned_addr learned my addr 172.21.15.148:0/1966090762 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52db59e640 1 -- 172.21.15.148:0/1966090762 >> v1:172.21.15.188:6789/0 conn(0x7f52dc0ced80 legacy=0x7f52dc0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52db59e640 1 -- 172.21.15.148:0/1966090762 >> v1:172.21.15.148:6789/0 conn(0x7f52dc15dc20 legacy=0x7f52dc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52db59e640 1 -- 172.21.15.148:0/1966090762 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc161f40 con 0x7f52dc161710 2024-04-25T10:06:42.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.220+0000 7f52d9d9b640 1 -- 172.21.15.148:0/1966090762 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532071018 0 0) 0x7f52dc15e000 con 0x7f52dc15dc20 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.221+0000 7f52db59e640 1 --2- 172.21.15.148:0/1966090762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc161710 0x7f52dc0d29f0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097a0 tx=0x7f52d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ae62afa2eab73b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.221+0000 7f52d9d9b640 1 -- 172.21.15.148:0/1966090762 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4037070 con 0x7f52dc161710 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.221+0000 7f52d9d9b640 1 -- 172.21.15.148:0/1966090762 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4032440 con 0x7f52dc161710 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.221+0000 7f52d9d9b640 1 -- 172.21.15.148:0/1966090762 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4032810 con 0x7f52dc161710 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 -- 172.21.15.148:0/1966090762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc161710 msgr2=0x7f52dc0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1966090762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc161710 0x7f52dc0d29f0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f52d40097a0 tx=0x7f52d402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 -- 172.21.15.148:0/1966090762 shutdown_connections 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1966090762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc161710 0x7f52dc0d29f0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 -- 172.21.15.148:0/1966090762 >> 172.21.15.148:0/1966090762 conn(0x7f52dc0d0420 msgr2=0x7f52dc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 -- 172.21.15.148:0/1966090762 shutdown_connections 2024-04-25T10:06:42.230 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.222+0000 7f52e11d5640 1 -- 172.21.15.148:0/1966090762 wait complete. 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.223+0000 7f52e11d5640 1 Processor -- start 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.223+0000 7f52e11d5640 1 -- start start 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52e11d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0ced80 0x7f52dc0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52e11d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 0x7f52dc15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52e11d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc161710 0x7f52dc15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52e11d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0b5e50 con 0x7f52dc15dc20 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52e11d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc162500 con 0x7f52dc161710 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52e11d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0b5fd0 con 0x7f52dc0ced80 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 0x7f52dc15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 0x7f52dc15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51614/0 (socket says 172.21.15.148:51614) 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 -- 172.21.15.148:0/1958262606 learned_addr learned my addr 172.21.15.148:0/1958262606 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52dad9d640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0ced80 0x7f52dc0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52db59e640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc161710 0x7f52dc15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 -- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0ced80 msgr2=0x7f52dc0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0ced80 0x7f52dc0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 -- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc161710 msgr2=0x7f52dc15c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc161710 0x7f52dc15c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.224+0000 7f52da59c640 1 -- 172.21.15.148:0/1958262606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc1f4690 con 0x7f52dc15dc20 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.225+0000 7f52da59c640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 0x7f52dc15be10 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f52d000b3f0 tx=0x7f52d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.225+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0015020 con 0x7f52dc15dc20 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.225+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d0011830 con 0x7f52dc15dc20 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.225+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc1f38d0 con 0x7f52dc15dc20 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.225+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0011c20 con 0x7f52dc15dc20 2024-04-25T10:06:42.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.225+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc1f3e90 con 0x7f52dc15dc20 2024-04-25T10:06:42.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.226+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc1f3650 con 0x7f52dc15dc20 2024-04-25T10:06:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.226+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52d0018020 con 0x7f52dc15dc20 2024-04-25T10:06:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.227+0000 7f52c37fe640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.227+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52d0089f20 con 0x7f52dc15dc20 2024-04-25T10:06:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.227+0000 7f52dad9d640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.227+0000 7f52dad9d640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0012e0 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.231+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52dc1f3650 con 0x7f52dc15dc20 2024-04-25T10:06:42.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.437+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f52dc073a00 con 0x7f52dc15dc20 2024-04-25T10:06:42.444 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:06:42.438+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:42.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.439+0000 7f52c37fe640 1 -- 172.21.15.148:0/1958262606 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) v1 ==== 161+0+0 (secure 0 0 0) 0x7f52d001f030 con 0x7f52dc15dc20 2024-04-25T10:06:42.446 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 msgr2=0x7f52c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0012e0 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 msgr2=0x7f52dc15be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 0x7f52dc15be10 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f52d000b3f0 tx=0x7f52d000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 shutdown_connections 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc161710 0x7f52dc15c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52dc15dc20 0x7f52dc15be10 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 --2- 172.21.15.148:0/1958262606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0ced80 0x7f52dc0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 >> 172.21.15.148:0/1958262606 conn(0x7f52dc0d0420 msgr2=0x7f52dc0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 shutdown_connections 2024-04-25T10:06:42.450 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.442+0000 7f52e11d5640 1 -- 172.21.15.148:0/1958262606 wait complete. 2024-04-25T10:06:42.460 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:42.485 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:06:42.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.555+0000 7f3e3bd5c640 1 Processor -- start 2024-04-25T10:06:42.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.555+0000 7f3e3bd5c640 1 -- start start 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3bd5c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3e34070dc0 0x7f3e3406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3bd5c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3e3415deb0 0x7f3e3415e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3bd5c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e340d1770 con 0x7f3e340711a0 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3bd5c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e340b0eb0 con 0x7f3e3415deb0 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3bd5c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e341622b0 con 0x7f3e34070dc0 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3e3415deb0 0x7f3e3415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3e3415deb0 0x7f3e3415e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51626/0 (socket says 172.21.15.148:51626) 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 -- 172.21.15.148:0/1364853540 learned_addr learned my addr 172.21.15.148:0/1364853540 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e39af8640 1 --2- 172.21.15.148:0/1364853540 >> v2:172.21.15.188:3300/0 conn(0x7f3e34070dc0 0x7f3e3406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 -- 172.21.15.148:0/1364853540 >> v2:172.21.15.188:3300/0 conn(0x7f3e34070dc0 msgr2=0x7f3e3406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 --2- 172.21.15.148:0/1364853540 >> v2:172.21.15.188:3300/0 conn(0x7f3e34070dc0 0x7f3e3406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 -- 172.21.15.148:0/1364853540 >> v1:172.21.15.188:6789/0 conn(0x7f3e340711a0 legacy=0x7f3e340b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.556+0000 7f3e3a2f9640 1 -- 172.21.15.148:0/1364853540 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34162740 con 0x7f3e3415deb0 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.557+0000 7f3e3a2f9640 1 --2- 172.21.15.148:0/1364853540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3415deb0 0x7f3e3415e300 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f3e34070a00 tx=0x7f3e2c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d9b6d44f80afb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.557+0000 7f3e38af6640 1 -- 172.21.15.148:0/1364853540 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c036070 con 0x7f3e3415deb0 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.557+0000 7f3e38af6640 1 -- 172.21.15.148:0/1364853540 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c031600 con 0x7f3e3415deb0 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.557+0000 7f3e38af6640 1 -- 172.21.15.148:0/1364853540 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0319d0 con 0x7f3e3415deb0 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/1364853540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3415deb0 msgr2=0x7f3e3415e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/1364853540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3415deb0 0x7f3e3415e300 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f3e34070a00 tx=0x7f3e2c02a610 comp rx=0 tx=0).stop 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/1364853540 shutdown_connections 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/1364853540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3415deb0 0x7f3e3415e300 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/1364853540 >> v2:172.21.15.188:3300/0 conn(0x7f3e34070dc0 0x7f3e3406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/1364853540 >> 172.21.15.148:0/1364853540 conn(0x7f3e340b3530 msgr2=0x7f3e340b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/1364853540 shutdown_connections 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.558+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/1364853540 wait complete. 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.559+0000 7f3e3bd5c640 1 Processor -- start 2024-04-25T10:06:42.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.559+0000 7f3e3bd5c640 1 -- start start 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 0x7f3e3414bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e340711a0 0x7f3e340aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e3415deb0 0x7f3e34149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e34161c60 con 0x7f3e34070dc0 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e340b10f0 con 0x7f3e340711a0 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e34161f40 con 0x7f3e3415deb0 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 0x7f3e3414bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 0x7f3e3414bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51632/0 (socket says 172.21.15.148:51632) 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 -- 172.21.15.148:0/659491665 learned_addr learned my addr 172.21.15.148:0/659491665 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e392f7640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e340711a0 0x7f3e340aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3a2f9640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e3415deb0 0x7f3e34149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 -- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e3415deb0 msgr2=0x7f3e34149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e3415deb0 0x7f3e34149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 -- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e340711a0 msgr2=0x7f3e340aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e340711a0 0x7f3e340aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 -- 172.21.15.148:0/659491665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e340b0450 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3a2f9640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e3415deb0 0x7f3e34149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e392f7640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e340711a0 0x7f3e340aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e39af8640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 0x7f3e3414bec0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d830 tx=0x7f3e3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000b6c0 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30019070 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000bac0 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e341497d0 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.560+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e34149a50 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.561+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e34073a00 con 0x7f3e34070dc0 2024-04-25T10:06:42.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.562+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e3001e460 con 0x7f3e34070dc0 2024-04-25T10:06:42.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.562+0000 7f3e2a7fc640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3e1806b4d0 0x7f3e1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.562+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3e3008aaf0 con 0x7f3e34070dc0 2024-04-25T10:06:42.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.562+0000 7f3e392f7640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3e1806b4d0 0x7f3e1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.563+0000 7f3e392f7640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3e1806b4d0 0x7f3e1806d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3e341f6f30 tx=0x7f3e24009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.566+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e341497d0 con 0x7f3e34070dc0 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:42.772+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3e340b23b0 con 0x7f3e34070dc0 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.443+0000 7f3e2a7fc640 1 -- 172.21.15.148:0/659491665 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v108) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3e30059790 con 0x7f3e34070dc0 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3e1806b4d0 msgr2=0x7f3e1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3e1806b4d0 0x7f3e1806d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3e341f6f30 tx=0x7f3e24009040 comp rx=0 tx=0).stop 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 msgr2=0x7f3e3414bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 0x7f3e3414bec0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d830 tx=0x7f3e3000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 shutdown_connections 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3e1806b4d0 0x7f3e1806d990 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e3415deb0 0x7f3e34149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e340711a0 0x7f3e340aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 --2- 172.21.15.148:0/659491665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e34070dc0 0x7f3e3414bec0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 >> 172.21.15.148:0/659491665 conn(0x7f3e340b3530 msgr2=0x7f3e340b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 shutdown_connections 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.446+0000 7f3e3bd5c640 1 -- 172.21.15.148:0/659491665 wait complete. 2024-04-25T10:06:43.466 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:43.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.604+0000 7f5ab402c640 1 Processor -- start 2024-04-25T10:06:43.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.604+0000 7f5ab402c640 1 -- start start 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab402c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5aac099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab402c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5aac06bf00 0x7f5aac06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab402c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aac0d06c0 con 0x7f5aac15e0e0 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab402c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac0d1440 con 0x7f5aac06bf00 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab402c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac09a690 con 0x7f5aac15dd00 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab15c7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f5aac15e0e0 0x7f5aac099f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47738/0 (socket says 172.21.15.148:47738) 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.605+0000 7f5ab15c7640 1 -- 172.21.15.148:0/1811371018 learned_addr learned my addr 172.21.15.148:0/1811371018 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab1dc8640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5aac099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/1811371018 >> v2:172.21.15.188:3300/0 conn(0x7f5aac06bf00 0x7f5aac06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225676875 0 0) 0x7f5aac0d06c0 con 0x7f5aac15e0e0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a98003520 con 0x7f5aac15e0e0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 944143299 0 0) 0x7f5a98003520 con 0x7f5aac15e0e0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 msgr2=0x7f5aac099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5aac099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 >> v2:172.21.15.188:3300/0 conn(0x7f5aac06bf00 msgr2=0x7f5aac06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 --2- 172.21.15.148:0/1811371018 >> v2:172.21.15.188:3300/0 conn(0x7f5aac06bf00 0x7f5aac06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0d0960 con 0x7f5aac15e0e0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f5a9c001670 con 0x7f5aac15e0e0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/1811371018 >> v2:172.21.15.188:3300/0 conn(0x7f5aac06bf00 0x7f5aac06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 >> v1:172.21.15.148:6789/0 conn(0x7f5aac15e0e0 legacy=0x7f5aac099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5a980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a98003f30 0x7f5a98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a98004d40 0x7f5a980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0d1440 con 0x7f5a98004d40 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac09a690 con 0x7f5a98003f30 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9800b9c0 con 0x7f5aac15dd00 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a98004d40 0x7f5a980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab1dc8640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5a980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab15c7640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a98003f30 0x7f5a98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 -- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 msgr2=0x7f5a980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5a980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 -- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a98003f30 msgr2=0x7f5a98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a98003f30 0x7f5a98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab25c9640 1 -- 172.21.15.148:0/1811371018 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98003520 con 0x7f5a98004d40 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab15c7640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a98003f30 0x7f5a98004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.606+0000 7f5ab1dc8640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aac15dd00 0x7f5a980039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.607+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a98004d40 0x7f5a980051c0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009c90 tx=0x7f5a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f38902b783e51a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.607+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4036070 con 0x7f5a98004d40 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.607+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa4031400 con 0x7f5a98004d40 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.607+0000 7f5ab0dc6640 1 -- 172.21.15.148:0/1811371018 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4009860 con 0x7f5a98004d40 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab402c640 1 -- 172.21.15.148:0/1811371018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a98004d40 msgr2=0x7f5a980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab402c640 1 --2- 172.21.15.148:0/1811371018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a98004d40 0x7f5a980051c0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009c90 tx=0x7f5a98005830 comp rx=0 tx=0).stop 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab25c9640 1 -- 172.21.15.148:0/1811371018 reap_dead start 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab402c640 1 -- 172.21.15.148:0/1811371018 shutdown_connections 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab402c640 1 -- 172.21.15.148:0/1811371018 >> 172.21.15.148:0/1811371018 conn(0x7f5aac0b3290 msgr2=0x7f5aac0b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab402c640 1 -- 172.21.15.148:0/1811371018 shutdown_connections 2024-04-25T10:06:43.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.608+0000 7f5ab402c640 1 -- 172.21.15.148:0/1811371018 wait complete. 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.609+0000 7f5ab402c640 1 Processor -- start 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.609+0000 7f5ab402c640 1 -- start start 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab402c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a98004d40 0x7f5aac155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab402c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aac0b55a0 0x7f5aac0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab402c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 0x7f5aac0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab402c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac09a950 con 0x7f5aac0ce460 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab402c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0d0f20 con 0x7f5aac0b55a0 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab402c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0b2b40 con 0x7f5a98004d40 2024-04-25T10:06:43.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 0x7f5aac0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 0x7f5aac0ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52820/0 (socket says 172.21.15.148:52820) 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 -- 172.21.15.148:0/3014679144 learned_addr learned my addr 172.21.15.148:0/3014679144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab1dc8640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a98004d40 0x7f5aac155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab15c7640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aac0b55a0 0x7f5aac0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 -- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a98004d40 msgr2=0x7f5aac155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a98004d40 0x7f5aac155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 -- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aac0b55a0 msgr2=0x7f5aac0b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aac0b55a0 0x7f5aac0b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.610+0000 7f5ab25c9640 1 -- 172.21.15.148:0/3014679144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0c8ac0 con 0x7f5aac0ce460 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.611+0000 7f5ab25c9640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 0x7f5aac0ce910 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009db0 tx=0x7f5aa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.611+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4036070 con 0x7f5aac0ce460 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.611+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa4041070 con 0x7f5aac0ce460 2024-04-25T10:06:43.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.611+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac0c8da0 con 0x7f5aac0ce460 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.611+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4009860 con 0x7f5aac0ce460 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.611+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac0c9360 con 0x7f5aac0ce460 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.612+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac073a00 con 0x7f5aac0ce460 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.612+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5aa403b860 con 0x7f5aac0ce460 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.612+0000 7f5aa27fc640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a9006b500 0x7f5a9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.613+0000 7f5ab1dc8640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a9006b500 0x7f5a9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.613+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5aa40aa530 con 0x7f5aac0ce460 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.613+0000 7f5ab1dc8640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a9006b500 0x7f5a9006d9c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8007540 tx=0x7f5aa800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.617+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5aa4078af0 con 0x7f5aac0ce460 2024-04-25T10:06:43.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:43.787+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5aac0d05f0 con 0x7f5aac0ce460 2024-04-25T10:06:44.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.444+0000 7f5aa27fc640 1 -- 172.21.15.148:0/3014679144 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v187) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5aa407cb90 con 0x7f5aac0ce460 2024-04-25T10:06:44.451 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a9006b500 msgr2=0x7f5a9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a9006b500 0x7f5a9006d9c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8007540 tx=0x7f5aa800b040 comp rx=0 tx=0).stop 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 msgr2=0x7f5aac0ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 0x7f5aac0ce910 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009db0 tx=0x7f5aa4004730 comp rx=0 tx=0).stop 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 shutdown_connections 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5a9006b500 0x7f5a9006d9c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5aac0ce460 0x7f5aac0ce910 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aac0b55a0 0x7f5aac0b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 --2- 172.21.15.148:0/3014679144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a98004d40 0x7f5aac155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 >> 172.21.15.148:0/3014679144 conn(0x7f5aac0b3290 msgr2=0x7f5aac06fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 shutdown_connections 2024-04-25T10:06:44.455 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.447+0000 7f5ab402c640 1 -- 172.21.15.148:0/3014679144 wait complete. 2024-04-25T10:06:44.467 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.563+0000 7f9c6aa38640 1 Processor -- start 2024-04-25T10:06:44.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.563+0000 7f9c6aa38640 1 -- start start 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.563+0000 7f9c6aa38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 0x7f9c64166980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.563+0000 7f9c6aa38640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c640ceba0 con 0x7f9c64144d20 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.563+0000 7f9c6aa38640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c640cee80 con 0x7f9c641454b0 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.563+0000 7f9c6aa38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b4830 con 0x7f9c640714d0 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c63fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f9c64144d20 0x7f9c640ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47746/0 (socket says 172.21.15.148:47746) 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c63fff640 1 -- 172.21.15.148:0/2323016732 learned_addr learned my addr 172.21.15.148:0/2323016732 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191191907 0 0) 0x7f9c640ceba0 con 0x7f9c64144d20 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c68fd5640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 0x7f9c64166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.572 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c50003520 con 0x7f9c64144d20 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474540117 0 0) 0x7f9c50003520 con 0x7f9c64144d20 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 msgr2=0x7f9c64166980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 0x7f9c64166980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 >> v1:172.21.15.188:6789/0 conn(0x7f9c641454b0 legacy=0x7f9c64070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640cf160 con 0x7f9c64144d20 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f9c54001670 con 0x7f9c64144d20 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 >> v1:172.21.15.148:6789/0 conn(0x7f9c64144d20 legacy=0x7f9c640ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 0x7f9c500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c50003f60 0x7f9c50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c50004d40 0x7f9c500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b4830 con 0x7f9c50004d40 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5000baa0 con 0x7f9c50003f60 2024-04-25T10:06:44.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5000bca0 con 0x7f9c640714d0 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c68fd5640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c50004d40 0x7f9c500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c63fff640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 0x7f9c500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.564+0000 7f9c637fe640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c50003f60 0x7f9c50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.565+0000 7f9c68fd5640 1 -- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 msgr2=0x7f9c500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.565+0000 7f9c68fd5640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640714d0 0x7f9c500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.565+0000 7f9c68fd5640 1 -- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c50003f60 msgr2=0x7f9c50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.565+0000 7f9c68fd5640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c50003f60 0x7f9c50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.565+0000 7f9c68fd5640 1 -- 172.21.15.148:0/2323016732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50003520 con 0x7f9c50004d40 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.565+0000 7f9c68fd5640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c50004d40 0x7f9c500051c0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009fc0 tx=0x7f9c50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed49a6834782f066 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.566+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c040070 con 0x7f9c50004d40 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.566+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c033c30 con 0x7f9c50004d40 2024-04-25T10:06:44.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.566+0000 7f9c62ffd640 1 -- 172.21.15.148:0/2323016732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c009c10 con 0x7f9c50004d40 2024-04-25T10:06:44.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.566+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2323016732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c50004d40 msgr2=0x7f9c500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.566+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2323016732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c50004d40 0x7f9c500051c0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009fc0 tx=0x7f9c50005830 comp rx=0 tx=0).stop 2024-04-25T10:06:44.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.566+0000 7f9c68fd5640 1 -- 172.21.15.148:0/2323016732 reap_dead start 2024-04-25T10:06:44.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.567+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2323016732 shutdown_connections 2024-04-25T10:06:44.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.567+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2323016732 >> 172.21.15.148:0/2323016732 conn(0x7f9c640b3530 msgr2=0x7f9c640b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.567+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2323016732 shutdown_connections 2024-04-25T10:06:44.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.567+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2323016732 wait complete. 2024-04-25T10:06:44.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.569+0000 7f9c6aa38640 1 Processor -- start 2024-04-25T10:06:44.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.570+0000 7f9c6aa38640 1 -- start start 2024-04-25T10:06:44.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.571+0000 7f9c6aa38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 0x7f9c641f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.571+0000 7f9c6aa38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c641e2da0 0x7f9c640b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.571+0000 7f9c6aa38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640b2b40 0x7f9c6413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.571+0000 7f9c6aa38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64145280 con 0x7f9c641e2da0 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.571+0000 7f9c6aa38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64145100 con 0x7f9c6415ae80 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.571+0000 7f9c6aa38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b5cb0 con 0x7f9c640b2b40 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c63fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 0x7f9c641f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c63fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 0x7f9c641f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:49822/0 (socket says 172.21.15.148:49822) 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c63fff640 1 -- 172.21.15.148:0/2105601682 learned_addr learned my addr 172.21.15.148:0/2105601682 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c637fe640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c641e2da0 0x7f9c640b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c68fd5640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640b2b40 0x7f9c6413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c637fe640 1 -- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640b2b40 msgr2=0x7f9c6413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c637fe640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640b2b40 0x7f9c6413d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c637fe640 1 -- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 msgr2=0x7f9c641f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c637fe640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 0x7f9c641f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c637fe640 1 -- 172.21.15.148:0/2105601682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6415b260 con 0x7f9c641e2da0 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.572+0000 7f9c63fff640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 0x7f9c641f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.573+0000 7f9c637fe640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c641e2da0 0x7f9c640b2600 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00f5f0 tx=0x7f9c4c00f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.573+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0052e0 con 0x7f9c641e2da0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.573+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c018070 con 0x7f9c641e2da0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.573+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c003830 con 0x7f9c641e2da0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.573+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c641e35d0 con 0x7f9c641e2da0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.573+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c641e3b90 con 0x7f9c641e2da0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.575+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c640b2f50 con 0x7f9c641e2da0 2024-04-25T10:06:44.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.576+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c4c0039d0 con 0x7f9c641e2da0 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.576+0000 7f9c60ff9640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c4406b520 0x7f9c4406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.576+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f9c4c089e10 con 0x7f9c641e2da0 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.576+0000 7f9c63fff640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c4406b520 0x7f9c4406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.577+0000 7f9c63fff640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c4406b520 0x7f9c4406d9e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c540078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.580+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c640b2f50 con 0x7f9c641e2da0 2024-04-25T10:06:44.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:44.750+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c641e4250 con 0x7f9c641e2da0 2024-04-25T10:06:45.454 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.448+0000 7f9c60ff9640 1 -- 172.21.15.148:0/2105601682 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v188) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9c4c08d030 con 0x7f9c641e2da0 2024-04-25T10:06:45.455 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c4406b520 msgr2=0x7f9c4406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c4406b520 0x7f9c4406d9e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c540078f0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c641e2da0 msgr2=0x7f9c640b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c641e2da0 0x7f9c640b2600 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00f5f0 tx=0x7f9c4c00f9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 shutdown_connections 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9c4406b520 0x7f9c4406d9e0 secure :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005ab0 tx=0x7f9c540078f0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c640b2b40 0x7f9c6413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c641e2da0 0x7f9c640b2600 secure :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00f5f0 tx=0x7f9c4c00f9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 --2- 172.21.15.148:0/2105601682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c6415ae80 0x7f9c641f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.451+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 >> 172.21.15.148:0/2105601682 conn(0x7f9c640b3530 msgr2=0x7f9c6406c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.452+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 shutdown_connections 2024-04-25T10:06:45.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.452+0000 7f9c6aa38640 1 -- 172.21.15.148:0/2105601682 wait complete. 2024-04-25T10:06:45.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:45.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.568+0000 7f9e003e0640 1 Processor -- start 2024-04-25T10:06:45.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.568+0000 7f9e003e0640 1 -- start start 2024-04-25T10:06:45.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9e003e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9df806ccd0 0x7f9df80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9e003e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df815deb0 0x7f9df815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9e003e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df80d1770 con 0x7f9df8070dc0 2024-04-25T10:06:45.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9e003e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df81622b0 con 0x7f9df806ccd0 2024-04-25T10:06:45.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9e003e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80b10f0 con 0x7f9df815deb0 2024-04-25T10:06:45.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfd97b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9df806ccd0 0x7f9df80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe97d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df815deb0 0x7f9df815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe17c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9df8070dc0 0x7f9df80711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56062/0 (socket says 172.21.15.148:56062) 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe17c640 1 -- 172.21.15.148:0/109386657 learned_addr learned my addr 172.21.15.148:0/109386657 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfd17a640 1 -- 172.21.15.148:0/109386657 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008599452 0 0) 0x7f9df80d1770 con 0x7f9df8070dc0 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfd17a640 1 -- 172.21.15.148:0/109386657 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de4003520 con 0x7f9df8070dc0 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe97d640 1 -- 172.21.15.148:0/109386657 >> v2:172.21.15.188:3300/0 conn(0x7f9df806ccd0 msgr2=0x7f9df80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe97d640 1 --2- 172.21.15.148:0/109386657 >> v2:172.21.15.188:3300/0 conn(0x7f9df806ccd0 0x7f9df80b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe97d640 1 -- 172.21.15.148:0/109386657 >> v1:172.21.15.188:6789/0 conn(0x7f9df8070dc0 legacy=0x7f9df80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.569+0000 7f9dfe97d640 1 -- 172.21.15.148:0/109386657 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8162740 con 0x7f9df815deb0 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.570+0000 7f9dfe97d640 1 --2- 172.21.15.148:0/109386657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df815deb0 0x7f9df815e340 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009d90 tx=0x7f9df002a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5634390da81872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.570+0000 7f9dfd17a640 1 -- 172.21.15.148:0/109386657 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0007740 con 0x7f9df815deb0 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.570+0000 7f9dfd17a640 1 -- 172.21.15.148:0/109386657 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df00078e0 con 0x7f9df815deb0 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.570+0000 7f9dfd17a640 1 -- 172.21.15.148:0/109386657 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0007cb0 con 0x7f9df815deb0 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 -- 172.21.15.148:0/109386657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df815deb0 msgr2=0x7f9df815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 --2- 172.21.15.148:0/109386657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df815deb0 0x7f9df815e340 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009d90 tx=0x7f9df002a530 comp rx=0 tx=0).stop 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 -- 172.21.15.148:0/109386657 shutdown_connections 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 --2- 172.21.15.148:0/109386657 >> v2:172.21.15.188:3300/0 conn(0x7f9df806ccd0 0x7f9df80b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 --2- 172.21.15.148:0/109386657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df815deb0 0x7f9df815e340 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 -- 172.21.15.148:0/109386657 >> 172.21.15.148:0/109386657 conn(0x7f9df80b3530 msgr2=0x7f9df80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 -- 172.21.15.148:0/109386657 shutdown_connections 2024-04-25T10:06:45.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.571+0000 7f9e003e0640 1 -- 172.21.15.148:0/109386657 wait complete. 2024-04-25T10:06:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.572+0000 7f9e003e0640 1 Processor -- start 2024-04-25T10:06:45.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.572+0000 7f9e003e0640 1 -- start start 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9e003e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 0x7f9df814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9e003e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df8070dc0 0x7f9df80cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9e003e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9df815deb0 0x7f9df8148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9e003e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8161f40 con 0x7f9df806ccd0 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9e003e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8161c60 con 0x7f9df815deb0 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9e003e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80b5840 con 0x7f9df8070dc0 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 0x7f9df814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 0x7f9df814bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52842/0 (socket says 172.21.15.148:52842) 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 -- 172.21.15.148:0/3437909152 learned_addr learned my addr 172.21.15.148:0/3437909152 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfd97b640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df8070dc0 0x7f9df80cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe97d640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9df815deb0 0x7f9df8148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 -- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df8070dc0 msgr2=0x7f9df80cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df8070dc0 0x7f9df80cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 -- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9df815deb0 msgr2=0x7f9df8148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9df815deb0 0x7f9df8148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.573+0000 7f9dfe17c640 1 -- 172.21.15.148:0/3437909152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df814c090 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.574+0000 7f9dfe17c640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 0x7f9df814bb50 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f9df400c460 tx=0x7f9df400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.574+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4017070 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.574+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4005ce0 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.574+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df81f3c90 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.574+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4007290 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.574+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df81f4250 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.575+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df8073a00 con 0x7f9df806ccd0 2024-04-25T10:06:45.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.575+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9df401a070 con 0x7f9df806ccd0 2024-04-25T10:06:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.576+0000 7f9deeffd640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ddc06b4d0 0x7f9ddc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.576+0000 7f9dfd97b640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ddc06b4d0 0x7f9ddc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.576+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9df4089300 con 0x7f9df806ccd0 2024-04-25T10:06:45.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.576+0000 7f9dfd97b640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ddc06b4d0 0x7f9ddc06d990 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9df8161410 tx=0x7f9de8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.580+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9df814c090 con 0x7f9df806ccd0 2024-04-25T10:06:45.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.750+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9df80b67b0 con 0x7f9df806ccd0 2024-04-25T10:06:45.759 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.753+0000 7f9deeffd640 1 -- 172.21.15.148:0/3437909152 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9df4057d30 con 0x7f9df806ccd0 2024-04-25T10:06:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ddc06b4d0 msgr2=0x7f9ddc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ddc06b4d0 0x7f9ddc06d990 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9df8161410 tx=0x7f9de8009190 comp rx=0 tx=0).stop 2024-04-25T10:06:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 msgr2=0x7f9df814bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 0x7f9df814bb50 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f9df400c460 tx=0x7f9df400c920 comp rx=0 tx=0).stop 2024-04-25T10:06:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 shutdown_connections 2024-04-25T10:06:45.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9ddc06b4d0 0x7f9ddc06d990 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9df815deb0 0x7f9df8148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9df8070dc0 0x7f9df80cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 --2- 172.21.15.148:0/3437909152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df806ccd0 0x7f9df814bb50 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 >> 172.21.15.148:0/3437909152 conn(0x7f9df80b3530 msgr2=0x7f9df80d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 shutdown_connections 2024-04-25T10:06:45.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.755+0000 7f9e003e0640 1 -- 172.21.15.148:0/3437909152 wait complete. 2024-04-25T10:06:45.773 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:06:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.864+0000 7f3b3a8fc640 1 Processor -- start 2024-04-25T10:06:45.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.864+0000 7f3b3a8fc640 1 -- start start 2024-04-25T10:06:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b3a8fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b340d1e70 0x7f3b340c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b3a8fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b340c7140 0x7f3b340cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b3a8fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340c7970 con 0x7f3b3415ef00 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b3a8fc640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3415f2e0 con 0x7f3b340d1e70 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b3a8fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b340c7c50 con 0x7f3b340c7140 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b337fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f3b340d1e70 0x7f3b340c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b33fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f3b3415ef00 0x7f3b340d1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47760/0 (socket says 172.21.15.148:47760) 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b33fff640 1 -- 172.21.15.148:0/1018556273 learned_addr learned my addr 172.21.15.148:0/1018556273 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b38e99640 1 --2- 172.21.15.148:0/1018556273 >> v2:172.21.15.188:3300/0 conn(0x7f3b340c7140 0x7f3b340cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b337fe640 1 -- 172.21.15.148:0/1018556273 >> v2:172.21.15.188:3300/0 conn(0x7f3b340c7140 msgr2=0x7f3b340cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b337fe640 1 --2- 172.21.15.148:0/1018556273 >> v2:172.21.15.188:3300/0 conn(0x7f3b340c7140 0x7f3b340cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b337fe640 1 -- 172.21.15.148:0/1018556273 >> v1:172.21.15.148:6789/0 conn(0x7f3b3415ef00 legacy=0x7f3b340d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.865+0000 7f3b337fe640 1 -- 172.21.15.148:0/1018556273 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340c7f30 con 0x7f3b340d1e70 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.866+0000 7f3b337fe640 1 --2- 172.21.15.148:0/1018556273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340d1e70 0x7f3b340c6c00 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f3b3406bcc0 tx=0x7f3b2802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6084a256c56adbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.866+0000 7f3b32ffd640 1 -- 172.21.15.148:0/1018556273 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28037070 con 0x7f3b340d1e70 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.866+0000 7f3b32ffd640 1 -- 172.21.15.148:0/1018556273 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b28032440 con 0x7f3b340d1e70 2024-04-25T10:06:45.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.866+0000 7f3b32ffd640 1 -- 172.21.15.148:0/1018556273 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28032810 con 0x7f3b340d1e70 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/1018556273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340d1e70 msgr2=0x7f3b340c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/1018556273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340d1e70 0x7f3b340c6c00 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f3b3406bcc0 tx=0x7f3b2802a230 comp rx=0 tx=0).stop 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/1018556273 shutdown_connections 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/1018556273 >> v2:172.21.15.188:3300/0 conn(0x7f3b340c7140 0x7f3b340cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/1018556273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340d1e70 0x7f3b340c6c00 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/1018556273 >> 172.21.15.148:0/1018556273 conn(0x7f3b3406f5d0 msgr2=0x7f3b3406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/1018556273 shutdown_connections 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.867+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/1018556273 wait complete. 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.868+0000 7f3b3a8fc640 1 Processor -- start 2024-04-25T10:06:45.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.868+0000 7f3b3a8fc640 1 -- start start 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 0x7f3b340add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b340d1e70 0x7f3b341f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b3415ef00 0x7f3b341f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c75d0 con 0x7f3b340c7140 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c84f0 con 0x7f3b340d1e70 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c7750 con 0x7f3b3415ef00 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 0x7f3b340add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 0x7f3b340add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52866/0 (socket says 172.21.15.148:52866) 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 -- 172.21.15.148:0/109970618 learned_addr learned my addr 172.21.15.148:0/109970618 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b337fe640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b340d1e70 0x7f3b341f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b38e99640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b3415ef00 0x7f3b341f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 -- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b3415ef00 msgr2=0x7f3b341f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b3415ef00 0x7f3b341f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 -- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b340d1e70 msgr2=0x7f3b341f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b340d1e70 0x7f3b341f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 -- 172.21.15.148:0/109970618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340c8990 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b38e99640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b3415ef00 0x7f3b341f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b33fff640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 0x7f3b340add20 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c3f0 tx=0x7f3b2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24017070 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b24005ce0 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b240072f0 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.869+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b340c8c30 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.870+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b340c91f0 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.870+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3415c2f0 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.871+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b2401a070 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.871+0000 7f3b30ff9640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b2006b500 0x7f3b2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.871+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3b240893a0 con 0x7f3b340c7140 2024-04-25T10:06:45.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.871+0000 7f3b337fe640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b2006b500 0x7f3b2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.872+0000 7f3b337fe640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b2006b500 0x7f3b2006d9c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3b28034000 tx=0x7f3b2802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:45.876+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b3415c2f0 con 0x7f3b340c7140 2024-04-25T10:06:46.123 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.117+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f3b34073990 con 0x7f3b340c7140 2024-04-25T10:06:46.456 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.450+0000 7f3b30ff9640 1 -- 172.21.15.148:0/109970618 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v599) v1 ==== 147+0+0 (secure 0 0 0) 0x7f3b24058420 con 0x7f3b340c7140 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b2006b500 msgr2=0x7f3b2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b2006b500 0x7f3b2006d9c0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3b28034000 tx=0x7f3b2802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 msgr2=0x7f3b340add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 0x7f3b340add20 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400c3f0 tx=0x7f3b2400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 shutdown_connections 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f3b2006b500 0x7f3b2006d9c0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b3415ef00 0x7f3b341f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b340d1e70 0x7f3b341f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 --2- 172.21.15.148:0/109970618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340c7140 0x7f3b340add20 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.453+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 >> 172.21.15.148:0/109970618 conn(0x7f3b3406f5d0 msgr2=0x7f3b3415bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.454+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 shutdown_connections 2024-04-25T10:06:46.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.454+0000 7f3b3a8fc640 1 -- 172.21.15.148:0/109970618 wait complete. 2024-04-25T10:06:46.473 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:06:46.473 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:06:46.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:46.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.571+0000 7f15c561f640 1 Processor -- start 2024-04-25T10:06:46.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.571+0000 7f15c561f640 1 -- start start 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15c561f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c015a790 0x7f15c00d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15c561f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15c006fea0 0x7f15c00d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15c561f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c00b3710 con 0x7f15c00af9c0 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15c561f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c00cf0e0 con 0x7f15c006fea0 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15c561f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00cf3c0 con 0x7f15c015a790 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15be7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15c006fea0 0x7f15c00d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15be7fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f15c006fea0 0x7f15c00d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52876/0 (socket says 172.21.15.148:52876) 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15be7fc640 1 -- 172.21.15.148:0/1397231703 learned_addr learned my addr 172.21.15.148:0/1397231703 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:46.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.572+0000 7f15beffd640 1 --2- 172.21.15.148:0/1397231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c015a790 0x7f15c00d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15be7fc640 1 -- 172.21.15.148:0/1397231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c015a790 msgr2=0x7f15c00d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15be7fc640 1 --2- 172.21.15.148:0/1397231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c015a790 0x7f15c00d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15be7fc640 1 -- 172.21.15.148:0/1397231703 >> v1:172.21.15.188:6789/0 conn(0x7f15c00af9c0 legacy=0x7f15c0072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15be7fc640 1 -- 172.21.15.148:0/1397231703 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00cf6a0 con 0x7f15c006fea0 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15be7fc640 1 --2- 172.21.15.148:0/1397231703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c00d08d0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009870 tx=0x7f15b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a013f9acfe8c22c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15bdffb640 1 -- 172.21.15.148:0/1397231703 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4037070 con 0x7f15c006fea0 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15bdffb640 1 -- 172.21.15.148:0/1397231703 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4032440 con 0x7f15c006fea0 2024-04-25T10:06:46.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.573+0000 7f15bdffb640 1 -- 172.21.15.148:0/1397231703 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4032810 con 0x7f15c006fea0 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 -- 172.21.15.148:0/1397231703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 msgr2=0x7f15c00d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 --2- 172.21.15.148:0/1397231703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c00d08d0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009870 tx=0x7f15b402a270 comp rx=0 tx=0).stop 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 -- 172.21.15.148:0/1397231703 shutdown_connections 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 --2- 172.21.15.148:0/1397231703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c00d08d0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 --2- 172.21.15.148:0/1397231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c015a790 0x7f15c00d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 -- 172.21.15.148:0/1397231703 >> 172.21.15.148:0/1397231703 conn(0x7f15c00b2370 msgr2=0x7f15c00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 -- 172.21.15.148:0/1397231703 shutdown_connections 2024-04-25T10:06:46.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.574+0000 7f15c561f640 1 -- 172.21.15.148:0/1397231703 wait complete. 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.575+0000 7f15c561f640 1 Processor -- start 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.575+0000 7f15c561f640 1 -- start start 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15c561f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c015c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15c561f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c00af9c0 0x7f15c0153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15c561f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c015a790 0x7f15c0156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15c561f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c015ab70 con 0x7f15c006fea0 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15c561f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c00cf980 con 0x7f15c015a790 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15c561f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c015acf0 con 0x7f15c00af9c0 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c015c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c015c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52892/0 (socket says 172.21.15.148:52892) 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 -- 172.21.15.148:0/3468657108 learned_addr learned my addr 172.21.15.148:0/3468657108 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15be7fc640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c00af9c0 0x7f15c0153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15bf7fe640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c015a790 0x7f15c0156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 -- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c00af9c0 msgr2=0x7f15c0153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c00af9c0 0x7f15c0153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 -- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c015a790 msgr2=0x7f15c0156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c015a790 0x7f15c0156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15beffd640 1 -- 172.21.15.148:0/3468657108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c015d130 con 0x7f15c006fea0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15be7fc640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c00af9c0 0x7f15c0153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.576+0000 7f15bf7fe640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c015a790 0x7f15c0156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.577+0000 7f15beffd640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c015c7e0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f15b0007ec0 tx=0x7f15b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.577+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0014620 con 0x7f15c006fea0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.577+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0019070 con 0x7f15c006fea0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.577+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c015d410 con 0x7f15c006fea0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.577+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0014a20 con 0x7f15c006fea0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.577+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c015dcb0 con 0x7f15c006fea0 2024-04-25T10:06:46.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.578+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c015d800 con 0x7f15c006fea0 2024-04-25T10:06:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.578+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15b0014bc0 con 0x7f15c006fea0 2024-04-25T10:06:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.579+0000 7f15a77fe640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15ac06b4d0 0x7f15ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.579+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f15b008aa50 con 0x7f15c006fea0 2024-04-25T10:06:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.579+0000 7f15be7fc640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15ac06b4d0 0x7f15ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.579+0000 7f15be7fc640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15ac06b4d0 0x7f15ac06d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f15b4034000 tx=0x7f15b402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.583+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15b0059ad0 con 0x7f15c006fea0 2024-04-25T10:06:46.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.757+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15c0073990 con 0x7f15c006fea0 2024-04-25T10:06:46.763 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.757+0000 7f15a77fe640 1 -- 172.21.15.148:0/3468657108 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+13353 (secure 0 0 0) 0x7f15b00593d0 con 0x7f15c006fea0 2024-04-25T10:06:46.763 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:46.763 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":188,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:45.446713+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6837/1743980733":"2024-04-26T10:06:42.439925+0000","172.21.15.148:6836/1743980733":"2024-04-26T10:06:42.439925+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15ac06b4d0 msgr2=0x7f15ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15ac06b4d0 0x7f15ac06d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f15b4034000 tx=0x7f15b402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 msgr2=0x7f15c015c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c015c7e0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f15b0007ec0 tx=0x7f15b000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 shutdown_connections 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f15ac06b4d0 0x7f15ac06d990 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15c015a790 0x7f15c0156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15c00af9c0 0x7f15c0153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 --2- 172.21.15.148:0/3468657108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15c006fea0 0x7f15c015c7e0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 >> 172.21.15.148:0/3468657108 conn(0x7f15c00b2370 msgr2=0x7f15c0157160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 shutdown_connections 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.760+0000 7f15c561f640 1 -- 172.21.15.148:0/3468657108 wait complete. 2024-04-25T10:06:46.776 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:46.777 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.872+0000 7fa9166ac640 1 Processor -- start 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.872+0000 7fa9166ac640 1 -- start start 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa9166ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 0x7fa9100ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa9166ac640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9100c6940 con 0x7fa91015a9f0 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa9166ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91015add0 con 0x7fa9100cd520 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa9166ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100b0020 con 0x7fa9100c6010 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa90ffff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa91015a9f0 0x7fa9100cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47776/0 (socket says 172.21.15.148:47776) 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa90ffff640 1 -- 172.21.15.148:0/391272593 learned_addr learned my addr 172.21.15.148:0/391272593 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551161049 0 0) 0x7fa900001110 con 0x7fa91015a9f0 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f8003520 con 0x7fa91015a9f0 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.873+0000 7fa90f7fe640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 0x7fa9100ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756372812 0 0) 0x7fa8f8003520 con 0x7fa91015a9f0 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 msgr2=0x7fa9100ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 0x7fa9100ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 >> v1:172.21.15.188:6789/0 conn(0x7fa9100cd520 legacy=0x7fa9100cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100c6660 con 0x7fa91015a9f0 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa900001840 con 0x7fa91015a9f0 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 >> v1:172.21.15.148:6789/0 conn(0x7fa91015a9f0 legacy=0x7fa9100cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f80039e0 0x7fa8f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 0x7fa8f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f8004d40 0x7fa8f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f800bb50 con 0x7fa8f80039e0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f800bcd0 con 0x7fa8f8004d40 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f800bea0 con 0x7fa9100c6010 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90ffff640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f80039e0 0x7fa8f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90f7fe640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 0x7fa8f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa914c49640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f8004d40 0x7fa8f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90ffff640 1 -- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 msgr2=0x7fa8f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90ffff640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9100c6010 0x7fa8f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90ffff640 1 -- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f8004d40 msgr2=0x7fa8f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90ffff640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f8004d40 0x7fa8f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.874+0000 7fa90ffff640 1 -- 172.21.15.148:0/391272593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8003520 con 0x7fa8f80039e0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa90ffff640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f80039e0 0x7fa8f8003e00 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c4f0 tx=0x7fa8f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffa7d63cc94ca23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000c0e0 con 0x7fa8f80039e0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa900034070 con 0x7fa8f80039e0 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa90effd640 1 -- 172.21.15.148:0/391272593 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900005d30 con 0x7fa8f80039e0 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa9166ac640 1 -- 172.21.15.148:0/391272593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f80039e0 msgr2=0x7fa8f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa9166ac640 1 --2- 172.21.15.148:0/391272593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f80039e0 0x7fa8f8003e00 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c4f0 tx=0x7fa8f8005830 comp rx=0 tx=0).stop 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.875+0000 7fa914c49640 1 -- 172.21.15.148:0/391272593 reap_dead start 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.876+0000 7fa9166ac640 1 -- 172.21.15.148:0/391272593 shutdown_connections 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.876+0000 7fa9166ac640 1 -- 172.21.15.148:0/391272593 >> 172.21.15.148:0/391272593 conn(0x7fa9100b2390 msgr2=0x7fa9100b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.876+0000 7fa9166ac640 1 -- 172.21.15.148:0/391272593 shutdown_connections 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.876+0000 7fa9166ac640 1 -- 172.21.15.148:0/391272593 wait complete. 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.877+0000 7fa9166ac640 1 Processor -- start 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.877+0000 7fa9166ac640 1 -- start start 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8f80039e0 0x7fa91014b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 0x7fa91006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91006b120 0x7fa9100aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100cb230 con 0x7fa9100ae010 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100cb5a0 con 0x7fa91006b120 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100ccfe0 con 0x7fa8f80039e0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 0x7fa91006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 0x7fa91006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52912/0 (socket says 172.21.15.148:52912) 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 -- 172.21.15.148:0/308803611 learned_addr learned my addr 172.21.15.148:0/308803611 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 -- 172.21.15.148:0/308803611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8f80039e0 msgr2=0x7fa91014b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8f80039e0 0x7fa91014b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 -- 172.21.15.148:0/308803611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91006b120 msgr2=0x7fa9100aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91006b120 0x7fa9100aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 -- 172.21.15.148:0/308803611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91014b860 con 0x7fa9100ae010 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90f7fe640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 0x7fa91006abe0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fa90400b7f0 tx=0x7fa90400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904018070 con 0x7fa9100ae010 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9040107b0 con 0x7fa9100ae010 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa904010b80 con 0x7fa9100ae010 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100b37f0 con 0x7fa9100ae010 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.878+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100b3db0 con 0x7fa9100ae010 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.879+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91006b540 con 0x7fa9100ae010 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.880+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa90401b070 con 0x7fa9100ae010 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.880+0000 7fa90cff9640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8fc06b500 0x7fa8fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.880+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa90408a5a0 con 0x7fa9100ae010 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.880+0000 7fa90ffff640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8fc06b500 0x7fa8fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.881+0000 7fa90ffff640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8fc06b500 0x7fa8fc06d9c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa900033000 tx=0x7fa900007000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:46.884+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa904059590 con 0x7fa9100ae010 2024-04-25T10:06:47.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.063+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa91006cc80 con 0x7fa9100ae010 2024-04-25T10:06:47.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.063+0000 7fa90cff9640 1 -- 172.21.15.148:0/308803611 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v188) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa904058e90 con 0x7fa9100ae010 2024-04-25T10:06:47.069 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8fc06b500 msgr2=0x7fa8fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8fc06b500 0x7fa8fc06d9c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa900033000 tx=0x7fa900007000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 msgr2=0x7fa91006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 0x7fa91006abe0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fa90400b7f0 tx=0x7fa90400bcb0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 shutdown_connections 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa8fc06b500 0x7fa8fc06d9c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91006b120 0x7fa9100aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9100ae010 0x7fa91006abe0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 --2- 172.21.15.148:0/308803611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8f80039e0 0x7fa91014b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 >> 172.21.15.148:0/308803611 conn(0x7fa9100b2390 msgr2=0x7fa9100c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 shutdown_connections 2024-04-25T10:06:47.072 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.066+0000 7fa9166ac640 1 -- 172.21.15.148:0/308803611 wait complete. 2024-04-25T10:06:47.082 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:06:47.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.175+0000 7fd51cbf4640 1 Processor -- start 2024-04-25T10:06:47.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.175+0000 7fd51cbf4640 1 -- start start 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.176+0000 7fd51cbf4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd51815dbe0 0x7fd5180d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.176+0000 7fd51cbf4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5180b2cb0 0x7fd5180d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.176+0000 7fd51cbf4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180b97a0 con 0x7fd5180b92a0 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.176+0000 7fd51cbf4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51815dfc0 con 0x7fd5180b2cb0 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.176+0000 7fd51cbf4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180d1a00 con 0x7fd51815dbe0 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd515d9b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5180b2cb0 0x7fd5180d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd51659c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd51815dbe0 0x7fd5180d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd515d9b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5180b2cb0 0x7fd5180d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:49870/0 (socket says 172.21.15.148:49870) 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd516d9d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd5180b92a0 0x7fd5180b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:56084/0 (socket says 172.21.15.148:56084) 2024-04-25T10:06:47.183 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd515d9b640 1 -- 172.21.15.148:0/2304576315 learned_addr learned my addr 172.21.15.148:0/2304576315 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:47.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd51659c640 1 -- 172.21.15.148:0/2304576315 >> v2:172.21.15.188:3300/0 conn(0x7fd5180b2cb0 msgr2=0x7fd5180d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd51659c640 1 --2- 172.21.15.148:0/2304576315 >> v2:172.21.15.188:3300/0 conn(0x7fd5180b2cb0 0x7fd5180d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd51659c640 1 -- 172.21.15.148:0/2304576315 >> v1:172.21.15.188:6789/0 conn(0x7fd5180b92a0 legacy=0x7fd5180b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.177+0000 7fd51659c640 1 -- 172.21.15.148:0/2304576315 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180d1c50 con 0x7fd51815dbe0 2024-04-25T10:06:47.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51659c640 1 --2- 172.21.15.148:0/2304576315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd51815dbe0 0x7fd5180d0910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd510009d90 tx=0x7fd51002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500f2658e501f972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51559a640 1 -- 172.21.15.148:0/2304576315 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510007740 con 0x7fd51815dbe0 2024-04-25T10:06:47.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51559a640 1 -- 172.21.15.148:0/2304576315 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100078e0 con 0x7fd51815dbe0 2024-04-25T10:06:47.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51559a640 1 -- 172.21.15.148:0/2304576315 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510007cb0 con 0x7fd51815dbe0 2024-04-25T10:06:47.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51cbf4640 1 -- 172.21.15.148:0/2304576315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd51815dbe0 msgr2=0x7fd5180d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/2304576315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd51815dbe0 0x7fd5180d0910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd510009d90 tx=0x7fd51002a660 comp rx=0 tx=0).stop 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51cbf4640 1 -- 172.21.15.148:0/2304576315 shutdown_connections 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/2304576315 >> v2:172.21.15.188:3300/0 conn(0x7fd5180b2cb0 0x7fd5180d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/2304576315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd51815dbe0 0x7fd5180d0910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.178+0000 7fd51cbf4640 1 -- 172.21.15.148:0/2304576315 >> 172.21.15.148:0/2304576315 conn(0x7fd518061a20 msgr2=0x7fd518061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.179+0000 7fd51cbf4640 1 -- 172.21.15.148:0/2304576315 shutdown_connections 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.179+0000 7fd51cbf4640 1 -- 172.21.15.148:0/2304576315 wait complete. 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.180+0000 7fd51cbf4640 1 Processor -- start 2024-04-25T10:06:47.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.180+0000 7fd51cbf4640 1 -- start start 2024-04-25T10:06:47.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5180b2cb0 0x7fd518151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5180b92a0 0x7fd5181f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 0x7fd5181f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180b6ab0 con 0x7fd51815dbe0 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180d2210 con 0x7fd5180b92a0 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180b6c30 con 0x7fd5180b2cb0 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 0x7fd5181f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 0x7fd5181f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52922/0 (socket says 172.21.15.148:52922) 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 -- 172.21.15.148:0/1089647683 learned_addr learned my addr 172.21.15.148:0/1089647683 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51659c640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5180b2cb0 0x7fd518151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd515d9b640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5180b92a0 0x7fd5181f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 -- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5180b2cb0 msgr2=0x7fd518151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5180b2cb0 0x7fd518151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 -- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5180b92a0 msgr2=0x7fd5181f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.189 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5180b92a0 0x7fd5181f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 -- 172.21.15.148:0/1089647683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5181f48b0 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd516d9d640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 0x7fd5181f4ab0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c430 tx=0x7fd50800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080074e0 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508007680 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180caed0 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080049e0 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.181+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5180cb490 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.182+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd518073990 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.183+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd50801a020 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.183+0000 7fd506ffd640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4f806b4d0 0x7fd4f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.183+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd508089720 con 0x7fd51815dbe0 2024-04-25T10:06:47.190 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.183+0000 7fd51659c640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4f806b4d0 0x7fd4f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.184+0000 7fd51659c640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4f806b4d0 0x7fd4f806d990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd510034000 tx=0x7fd51002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.193 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.187+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5181f48b0 con 0x7fd51815dbe0 2024-04-25T10:06:47.426 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.420+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7fd5180b8e80 con 0x7fd51815dbe0 2024-04-25T10:06:47.460 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.455+0000 7fd506ffd640 1 -- 172.21.15.148:0/1089647683 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v600) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd508058030 con 0x7fd51815dbe0 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4f806b4d0 msgr2=0x7fd4f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4f806b4d0 0x7fd4f806d990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd510034000 tx=0x7fd51002f000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 msgr2=0x7fd5181f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 0x7fd5181f4ab0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c430 tx=0x7fd50800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 shutdown_connections 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd4f806b4d0 0x7fd4f806d990 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd51815dbe0 0x7fd5181f4ab0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5180b92a0 0x7fd5181f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 --2- 172.21.15.148:0/1089647683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5180b2cb0 0x7fd518151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 >> 172.21.15.148:0/1089647683 conn(0x7fd518061a20 msgr2=0x7fd518061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 shutdown_connections 2024-04-25T10:06:47.463 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.457+0000 7fd51cbf4640 1 -- 172.21.15.148:0/1089647683 wait complete. 2024-04-25T10:06:47.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:06:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.569+0000 7f9f3a16a640 1 Processor -- start 2024-04-25T10:06:47.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.569+0000 7f9f3a16a640 1 -- start start 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f3a16a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f3406abe0 0x7f9f3406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f3a16a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f340ad570 0x7f9f3406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f3a16a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3406b500 con 0x7f9f3415dd80 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f3a16a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3406e410 con 0x7f9f3406abe0 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f337fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f3406abe0 0x7f9f3406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f3a16a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f340b3930 con 0x7f9f340ad570 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f337fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f9f3406abe0 0x7f9f3406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52930/0 (socket says 172.21.15.148:52930) 2024-04-25T10:06:47.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.570+0000 7f9f337fe640 1 -- 172.21.15.148:0/771147153 learned_addr learned my addr 172.21.15.148:0/771147153 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.571+0000 7f9f337fe640 1 -- 172.21.15.148:0/771147153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f340ad570 msgr2=0x7f9f3406ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.571+0000 7f9f33fff640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f340ad570 0x7f9f3406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.577 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.571+0000 7f9f337fe640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f340ad570 0x7f9f3406ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.571+0000 7f9f337fe640 1 -- 172.21.15.148:0/771147153 >> v1:172.21.15.188:6789/0 conn(0x7f9f3415dd80 legacy=0x7f9f340b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.571+0000 7f9f337fe640 1 -- 172.21.15.148:0/771147153 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340b3c10 con 0x7f9f3406abe0 2024-04-25T10:06:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.571+0000 7f9f33fff640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f340ad570 0x7f9f3406ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.573+0000 7f9f337fe640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3406abe0 0x7f9f3406afc0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009870 tx=0x7f9f2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da42c756c3529485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.573+0000 7f9f327fc640 1 -- 172.21.15.148:0/771147153 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24037070 con 0x7f9f3406abe0 2024-04-25T10:06:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.573+0000 7f9f327fc640 1 -- 172.21.15.148:0/771147153 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f24032440 con 0x7f9f3406abe0 2024-04-25T10:06:47.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.573+0000 7f9f327fc640 1 -- 172.21.15.148:0/771147153 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24032810 con 0x7f9f3406abe0 2024-04-25T10:06:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.574+0000 7f9f3a16a640 1 -- 172.21.15.148:0/771147153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3406abe0 msgr2=0x7f9f3406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.574+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3406abe0 0x7f9f3406afc0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009870 tx=0x7f9f2402a210 comp rx=0 tx=0).stop 2024-04-25T10:06:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.574+0000 7f9f3a16a640 1 -- 172.21.15.148:0/771147153 shutdown_connections 2024-04-25T10:06:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.574+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f340ad570 0x7f9f3406ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.574+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/771147153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3406abe0 0x7f9f3406afc0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.574+0000 7f9f3a16a640 1 -- 172.21.15.148:0/771147153 >> 172.21.15.148:0/771147153 conn(0x7f9f340b5af0 msgr2=0x7f9f340b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.576+0000 7f9f3a16a640 1 -- 172.21.15.148:0/771147153 shutdown_connections 2024-04-25T10:06:47.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.576+0000 7f9f3a16a640 1 -- 172.21.15.148:0/771147153 wait complete. 2024-04-25T10:06:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.577+0000 7f9f3a16a640 1 Processor -- start 2024-04-25T10:06:47.583 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.577+0000 7f9f3a16a640 1 -- start start 2024-04-25T10:06:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f3a16a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3406abe0 0x7f9f34153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f3a16a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f340ad570 0x7f9f34153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f3a16a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 0x7f9f3415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f3a16a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34070280 con 0x7f9f3415dd80 2024-04-25T10:06:47.586 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f3a16a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3406e940 con 0x7f9f340ad570 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f3a16a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34070400 con 0x7f9f3406abe0 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 0x7f9f3415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 0x7f9f3415b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52934/0 (socket says 172.21.15.148:52934) 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 -- 172.21.15.148:0/3057744142 learned_addr learned my addr 172.21.15.148:0/3057744142 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f337fe640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3406abe0 0x7f9f34153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f32ffd640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f340ad570 0x7f9f34153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 -- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3406abe0 msgr2=0x7f9f34153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3406abe0 0x7f9f34153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 -- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f340ad570 msgr2=0x7f9f34153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f340ad570 0x7f9f34153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f33fff640 1 -- 172.21.15.148:0/3057744142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f341f8c40 con 0x7f9f3415dd80 2024-04-25T10:06:47.587 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f32ffd640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f340ad570 0x7f9f34153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.579+0000 7f9f337fe640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3406abe0 0x7f9f34153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.580+0000 7f9f33fff640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 0x7f9f3415b790 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00cef0 tx=0x7f9f2c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.580+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c01a070 con 0x7f9f3415dd80 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.580+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c00d860 con 0x7f9f3415dd80 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.580+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00dc50 con 0x7f9f3415dd80 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.580+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f34159f90 con 0x7f9f3415dd80 2024-04-25T10:06:47.588 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.580+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f34153640 con 0x7f9f3415dd80 2024-04-25T10:06:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.583+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f2c01d070 con 0x7f9f3415dd80 2024-04-25T10:06:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.583+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f34159d80 con 0x7f9f3415dd80 2024-04-25T10:06:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.584+0000 7f9f1bfff640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b5a0 0x7f9f2006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.589 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.584+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9f2c08a920 con 0x7f9f3415dd80 2024-04-25T10:06:47.590 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.584+0000 7f9f337fe640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b5a0 0x7f9f2006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.591 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.586+0000 7f9f337fe640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b5a0 0x7f9f2006da60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9f24034000 tx=0x7f9f2402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.590+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f2c059a40 con 0x7f9f3415dd80 2024-04-25T10:06:47.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.761+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9f340739e0 con 0x7f9f3415dd80 2024-04-25T10:06:47.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.764+0000 7f9f1bfff640 1 -- 172.21.15.148:0/3057744142 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f2c059340 con 0x7f9f3415dd80 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b5a0 msgr2=0x7f9f2006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b5a0 0x7f9f2006da60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9f24034000 tx=0x7f9f2402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 msgr2=0x7f9f3415b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 0x7f9f3415b790 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00cef0 tx=0x7f9f2c00a710 comp rx=0 tx=0).stop 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 shutdown_connections 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9f2006b5a0 0x7f9f2006da60 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f3415dd80 0x7f9f3415b790 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f340ad570 0x7f9f34153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 --2- 172.21.15.148:0/3057744142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f3406abe0 0x7f9f34153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 >> 172.21.15.148:0/3057744142 conn(0x7f9f340b5af0 msgr2=0x7f9f340d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 shutdown_connections 2024-04-25T10:06:47.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:47.766+0000 7f9f3a16a640 1 -- 172.21.15.148:0/3057744142 wait complete. 2024-04-25T10:06:47.783 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:47.783 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:47.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:47.865 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:47.866 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:47.866 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:47.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:47.881 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:47.881 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:47.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:47.898 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:47.898 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:47.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:47.936 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:06:47.936 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:47.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:47.955 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:47.955 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:48.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 Processor -- start 2024-04-25T10:06:48.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 -- start start 2024-04-25T10:06:48.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a8815dc70 0x7f6a880d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a8809a450 0x7f6a880cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8806c220 con 0x7f6a8806c700 2024-04-25T10:06:48.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880d0c90 con 0x7f6a8815dc70 2024-04-25T10:06:48.075 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8eaac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880d0080 con 0x7f6a8809a450 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8c848640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a8815dc70 0x7f6a880d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8c848640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6a8815dc70 0x7f6a880d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52948/0 (socket says 172.21.15.148:52948) 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.068+0000 7f6a8c848640 1 -- 172.21.15.148:0/3435322094 learned_addr learned my addr 172.21.15.148:0/3435322094 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.069+0000 7f6a87fff640 1 --2- 172.21.15.148:0/3435322094 >> v2:172.21.15.188:3300/0 conn(0x7f6a8809a450 0x7f6a880cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.069+0000 7f6a8c848640 1 -- 172.21.15.148:0/3435322094 >> v2:172.21.15.188:3300/0 conn(0x7f6a8809a450 msgr2=0x7f6a880cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.069+0000 7f6a8c848640 1 --2- 172.21.15.148:0/3435322094 >> v2:172.21.15.188:3300/0 conn(0x7f6a8809a450 0x7f6a880cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.069+0000 7f6a8c848640 1 -- 172.21.15.148:0/3435322094 >> v1:172.21.15.188:6789/0 conn(0x7f6a8806c700 legacy=0x7f6a880cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.069+0000 7f6a8c848640 1 -- 172.21.15.148:0/3435322094 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880d0360 con 0x7f6a8815dc70 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8c848640 1 --2- 172.21.15.148:0/3435322094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8815dc70 0x7f6a880d1120 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f6a880733b0 tx=0x7f6a7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c248d6d16f0ab25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a877fe640 1 -- 172.21.15.148:0/3435322094 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78037070 con 0x7f6a8815dc70 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a877fe640 1 -- 172.21.15.148:0/3435322094 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a78032440 con 0x7f6a8815dc70 2024-04-25T10:06:48.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a877fe640 1 -- 172.21.15.148:0/3435322094 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78032810 con 0x7f6a8815dc70 2024-04-25T10:06:48.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 -- 172.21.15.148:0/3435322094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8815dc70 msgr2=0x7f6a880d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/3435322094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8815dc70 0x7f6a880d1120 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f6a880733b0 tx=0x7f6a7802a230 comp rx=0 tx=0).stop 2024-04-25T10:06:48.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 -- 172.21.15.148:0/3435322094 shutdown_connections 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/3435322094 >> v2:172.21.15.188:3300/0 conn(0x7f6a8809a450 0x7f6a880cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/3435322094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8815dc70 0x7f6a880d1120 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 -- 172.21.15.148:0/3435322094 >> 172.21.15.148:0/3435322094 conn(0x7f6a880b3270 msgr2=0x7f6a880b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 -- 172.21.15.148:0/3435322094 shutdown_connections 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.070+0000 7f6a8eaac640 1 -- 172.21.15.148:0/3435322094 wait complete. 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.071+0000 7f6a8eaac640 1 Processor -- start 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.071+0000 7f6a8eaac640 1 -- start start 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8eaac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 0x7f6a880ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8eaac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8809a450 0x7f6a88160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8eaac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8815dc70 0x7f6a881f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8eaac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8815e050 con 0x7f6a8806c700 2024-04-25T10:06:48.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8eaac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880d0920 con 0x7f6a8809a450 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8eaac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8806cb90 con 0x7f6a8815dc70 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8c848640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 0x7f6a880ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8c848640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 0x7f6a880ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:52960/0 (socket says 172.21.15.148:52960) 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.072+0000 7f6a8c848640 1 -- 172.21.15.148:0/1413398442 learned_addr learned my addr 172.21.15.148:0/1413398442 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8d049640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8815dc70 0x7f6a881f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8c848640 1 -- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8815dc70 msgr2=0x7f6a881f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a87fff640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8809a450 0x7f6a88160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8c848640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8815dc70 0x7f6a881f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8c848640 1 -- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8809a450 msgr2=0x7f6a88160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8c848640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8809a450 0x7f6a88160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8c848640 1 -- 172.21.15.148:0/1413398442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a881614f0 con 0x7f6a8806c700 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8d049640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8815dc70 0x7f6a881f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:48.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a87fff640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8809a450 0x7f6a88160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8c848640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 0x7f6a880ca850 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000c00 tx=0x7f6a780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78041070 con 0x7f6a8806c700 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a880caee0 con 0x7f6a8806c700 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7803c870 con 0x7f6a8806c700 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78037040 con 0x7f6a8806c700 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.073+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a880cb4a0 con 0x7f6a8806c700 2024-04-25T10:06:48.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.074+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a88073990 con 0x7f6a8806c700 2024-04-25T10:06:48.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.075+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a78044070 con 0x7f6a8806c700 2024-04-25T10:06:48.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.075+0000 7f6a857fa640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a7406b4d0 0x7f6a7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.075+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a780aadf0 con 0x7f6a8806c700 2024-04-25T10:06:48.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.075+0000 7f6a87fff640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a7406b4d0 0x7f6a7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.076+0000 7f6a87fff640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a7406b4d0 0x7f6a7406d990 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6a880cc600 tx=0x7f6a7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.079+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a881614f0 con 0x7f6a8806c700 2024-04-25T10:06:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.283+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a880b40a0 con 0x7f6a8806c700 2024-04-25T10:06:48.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.283+0000 7f6a857fa640 1 -- 172.21.15.148:0/1413398442 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+3719 (secure 0 0 0) 0x7f6a78079f10 con 0x7f6a8806c700 2024-04-25T10:06:48.290 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:48.290 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":108,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2385986022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2385986022},{"type":"v1","addr":"172.21.15.188:6834","nonce":2385986022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3094951772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3094951772},{"type":"v1","addr":"172.21.15.148:6835","nonce":3094951772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3913236754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3913236754},{"type":"v1","addr":"172.21.15.188:6835","nonce":3913236754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":11550,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2710351743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2710351743},{"type":"v1","addr":"172.21.15.148:6837","nonce":2710351743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108}],"filesystems":[]} 2024-04-25T10:06:48.290 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 108 2024-04-25T10:06:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.285+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a7406b4d0 msgr2=0x7f6a7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.291 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.285+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a7406b4d0 0x7f6a7406d990 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6a880cc600 tx=0x7f6a7c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.285+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 msgr2=0x7f6a880ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.285+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 0x7f6a880ca850 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000c00 tx=0x7f6a780046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 shutdown_connections 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6a7406b4d0 0x7f6a7406d990 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a8815dc70 0x7f6a881f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a8809a450 0x7f6a88160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 --2- 172.21.15.148:0/1413398442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a8806c700 0x7f6a880ca850 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 >> 172.21.15.148:0/1413398442 conn(0x7f6a880b3270 msgr2=0x7f6a880b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 shutdown_connections 2024-04-25T10:06:48.292 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.286+0000 7f6a8eaac640 1 -- 172.21.15.148:0/1413398442 wait complete. 2024-04-25T10:06:48.302 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:48.303 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:48.303 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:48.303 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:48.303 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:48.303 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:48.303 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:48.304 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:48.304 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:48.304 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:48.304 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:48.304 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:48.304 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:48.304 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:48.305 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:48.305 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:48.312 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:48.312 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:48.313 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:48.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:48.316 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:48.316 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:48.317 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:48.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:48.355 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:48.356 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:48.357 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:48.357 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:48.359 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:48.359 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:48.360 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:48.360 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:48.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:48.428 INFO:tasks.ceph.mds.a.smithi148.stdout:starting mds.a at 2024-04-25T10:06:48.429 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:06:48.440 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-04-25T10:06:48.440 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:06:48.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.459+0000 7f46a58cb640 1 Processor -- start 2024-04-25T10:06:48.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.459+0000 7f46a58cb640 1 -- start start 2024-04-25T10:06:48.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f46a58cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00c6010 0x7f46a00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f46a58cb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00c6940 con 0x7f46a015a9f0 2024-04-25T10:06:48.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f46a58cb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a015add0 con 0x7f46a00cd520 2024-04-25T10:06:48.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f46a58cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00b0020 con 0x7f46a00c6010 2024-04-25T10:06:48.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f469effd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f46a015a9f0 0x7f46a00cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47800/0 (socket says 172.21.15.148:47800) 2024-04-25T10:06:48.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f469effd640 1 -- 172.21.15.148:0/854653891 learned_addr learned my addr 172.21.15.148:0/854653891 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:48.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.460+0000 7f4697fff640 1 --2- 172.21.15.148:0/854653891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00c6010 0x7f46a00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.461+0000 7f4697fff640 1 -- 172.21.15.148:0/854653891 >> v1:172.21.15.188:6789/0 conn(0x7f46a00cd520 legacy=0x7f46a00cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.461+0000 7f4697fff640 1 -- 172.21.15.148:0/854653891 >> v1:172.21.15.148:6789/0 conn(0x7f46a015a9f0 legacy=0x7f46a00cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.461+0000 7f4697fff640 1 -- 172.21.15.148:0/854653891 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00c6660 con 0x7f46a00c6010 2024-04-25T10:06:48.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.461+0000 7f4697fff640 1 --2- 172.21.15.148:0/854653891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00c6010 0x7f46a00ccfe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46880097d0 tx=0x7f468802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1abf5de4285d6ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.462+0000 7f469e7fc640 1 -- 172.21.15.148:0/854653891 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688037070 con 0x7f46a00c6010 2024-04-25T10:06:48.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.462+0000 7f469e7fc640 1 -- 172.21.15.148:0/854653891 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688032440 con 0x7f46a00c6010 2024-04-25T10:06:48.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.462+0000 7f469e7fc640 1 -- 172.21.15.148:0/854653891 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688032810 con 0x7f46a00c6010 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 -- 172.21.15.148:0/854653891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00c6010 msgr2=0x7f46a00ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 --2- 172.21.15.148:0/854653891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00c6010 0x7f46a00ccfe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46880097d0 tx=0x7f468802a210 comp rx=0 tx=0).stop 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 -- 172.21.15.148:0/854653891 shutdown_connections 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 --2- 172.21.15.148:0/854653891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00c6010 0x7f46a00ccfe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 -- 172.21.15.148:0/854653891 >> 172.21.15.148:0/854653891 conn(0x7f46a00b2390 msgr2=0x7f46a00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 -- 172.21.15.148:0/854653891 shutdown_connections 2024-04-25T10:06:48.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.463+0000 7f46a58cb640 1 -- 172.21.15.148:0/854653891 wait complete. 2024-04-25T10:06:48.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.464+0000 7f46a58cb640 1 Processor -- start 2024-04-25T10:06:48.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 -- start start 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 0x7f46a01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00cd520 0x7f46a01f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a015a9f0 0x7f46a01f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cb230 con 0x7f46a00c6010 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cb5a0 con 0x7f46a015a9f0 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f46a58cb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cc140 con 0x7f46a00cd520 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f469effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 0x7f46a01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f469effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 0x7f46a01597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53002/0 (socket says 172.21.15.148:53002) 2024-04-25T10:06:48.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f469effd640 1 -- 172.21.15.148:0/730603951 learned_addr learned my addr 172.21.15.148:0/730603951 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f469f7fe640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a015a9f0 0x7f46a01f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.465+0000 7f4697fff640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00cd520 0x7f46a01f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469effd640 1 -- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00cd520 msgr2=0x7f46a01f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469effd640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00cd520 0x7f46a01f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469effd640 1 -- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a015a9f0 msgr2=0x7f46a01f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469effd640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a015a9f0 0x7f46a01f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469effd640 1 -- 172.21.15.148:0/730603951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a0157f60 con 0x7f46a00c6010 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469effd640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 0x7f46a01597a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f469000c480 tx=0x7f469000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690017070 con 0x7f46a00c6010 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4690005ce0 con 0x7f46a00c6010 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900072f0 con 0x7f46a00c6010 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a0158200 con 0x7f46a00c6010 2024-04-25T10:06:48.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.466+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a01587c0 con 0x7f46a00c6010 2024-04-25T10:06:48.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.467+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0073a00 con 0x7f46a00c6010 2024-04-25T10:06:48.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.468+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f469001a070 con 0x7f46a00c6010 2024-04-25T10:06:48.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.468+0000 7f469cff9640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f468006b4d0 0x7f468006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.469+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4690089270 con 0x7f46a00c6010 2024-04-25T10:06:48.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.469+0000 7f4697fff640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f468006b4d0 0x7f468006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.469+0000 7f4697fff640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f468006b4d0 0x7f468006d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4688034000 tx=0x7f468802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.474+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46a00c6940 con 0x7f46a00c6010 2024-04-25T10:06:48.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:48.659+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f46a00b1540 con 0x7f46a00c6010 2024-04-25T10:06:49.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.469+0000 7f469cff9640 1 -- 172.21.15.148:0/730603951 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v193) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4690057ca0 con 0x7f46a00c6010 2024-04-25T10:06:49.475 INFO:teuthology.orchestra.run.smithi148.stderr: removed all blocklist entries 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f468006b4d0 msgr2=0x7f468006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f468006b4d0 0x7f468006d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4688034000 tx=0x7f468802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 msgr2=0x7f46a01597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 0x7f46a01597a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f469000c480 tx=0x7f469000c940 comp rx=0 tx=0).stop 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 shutdown_connections 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f468006b4d0 0x7f468006d990 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a015a9f0 0x7f46a01f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a00cd520 0x7f46a01f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 --2- 172.21.15.148:0/730603951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46a00c6010 0x7f46a01597a0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 >> 172.21.15.148:0/730603951 conn(0x7f46a00b2390 msgr2=0x7f46a00af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 shutdown_connections 2024-04-25T10:06:49.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.472+0000 7f46a58cb640 1 -- 172.21.15.148:0/730603951 wait complete. 2024-04-25T10:06:49.491 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.587+0000 7fd7b401f640 1 Processor -- start 2024-04-25T10:06:49.593 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.588+0000 7fd7b401f640 1 -- start start 2024-04-25T10:06:49.594 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.588+0000 7fd7b401f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7ac154ee0 0x7fd7ac0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.588+0000 7fd7b401f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7ac0cfcf0 0x7fd7ac0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.588+0000 7fd7b401f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac071480 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.588+0000 7fd7b401f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ac0ce830 con 0x7fd7ac0cfcf0 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.588+0000 7fd7b401f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ac072090 con 0x7fd7ac154ee0 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7ac0cfcf0 0x7fd7ac0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd7ac0cfcf0 0x7fd7ac0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53010/0 (socket says 172.21.15.148:53010) 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 -- 172.21.15.148:0/1854776221 learned_addr learned my addr 172.21.15.148:0/1854776221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b1dbb640 1 --2- 172.21.15.148:0/1854776221 >> v2:172.21.15.188:3300/0 conn(0x7fd7ac154ee0 0x7fd7ac0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 -- 172.21.15.148:0/1854776221 >> v2:172.21.15.188:3300/0 conn(0x7fd7ac154ee0 msgr2=0x7fd7ac0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 --2- 172.21.15.148:0/1854776221 >> v2:172.21.15.188:3300/0 conn(0x7fd7ac154ee0 0x7fd7ac0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 -- 172.21.15.148:0/1854776221 >> v1:172.21.15.188:6789/0 conn(0x7fd7ac0ce3c0 legacy=0x7fd7ac15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b15ba640 1 -- 172.21.15.148:0/1854776221 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac071760 con 0x7fd7ac0cfcf0 2024-04-25T10:06:49.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.589+0000 7fd7b1dbb640 1 --2- 172.21.15.148:0/1854776221 >> v2:172.21.15.188:3300/0 conn(0x7fd7ac154ee0 0x7fd7ac0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b15ba640 1 --2- 172.21.15.148:0/1854776221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac0cde80 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0d02e0 tx=0x7fd7a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e9e1de0432497d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b0db9640 1 -- 172.21.15.148:0/1854776221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8037070 con 0x7fd7ac0cfcf0 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b0db9640 1 -- 172.21.15.148:0/1854776221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a8032440 con 0x7fd7ac0cfcf0 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b0db9640 1 -- 172.21.15.148:0/1854776221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8032810 con 0x7fd7ac0cfcf0 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 -- 172.21.15.148:0/1854776221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0cfcf0 msgr2=0x7fd7ac0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 --2- 172.21.15.148:0/1854776221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac0cde80 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0d02e0 tx=0x7fd7a802a270 comp rx=0 tx=0).stop 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 -- 172.21.15.148:0/1854776221 shutdown_connections 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 --2- 172.21.15.148:0/1854776221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac0cde80 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 --2- 172.21.15.148:0/1854776221 >> v2:172.21.15.188:3300/0 conn(0x7fd7ac154ee0 0x7fd7ac0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.597 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 -- 172.21.15.148:0/1854776221 >> 172.21.15.148:0/1854776221 conn(0x7fd7ac06abe0 msgr2=0x7fd7ac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 -- 172.21.15.148:0/1854776221 shutdown_connections 2024-04-25T10:06:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.590+0000 7fd7b401f640 1 -- 172.21.15.148:0/1854776221 wait complete. 2024-04-25T10:06:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 Processor -- start 2024-04-25T10:06:49.598 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 -- start start 2024-04-25T10:06:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 0x7fd7ac0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.600 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7ac154ee0 0x7fd7ac1e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac071d20 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac071230 con 0x7fd7ac0cfcf0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b401f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac1552c0 con 0x7fd7ac154ee0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b1dbb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 0x7fd7ac0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b1dbb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 0x7fd7ac0ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53022/0 (socket says 172.21.15.148:53022) 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.592+0000 7fd7b1dbb640 1 -- 172.21.15.148:0/3295870344 learned_addr learned my addr 172.21.15.148:0/3295870344 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b15ba640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b25bc640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7ac154ee0 0x7fd7ac1e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b1dbb640 1 -- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7ac154ee0 msgr2=0x7fd7ac1e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b1dbb640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7ac154ee0 0x7fd7ac1e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b1dbb640 1 -- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7ac0cfcf0 msgr2=0x7fd7ac156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b1dbb640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b1dbb640 1 -- 172.21.15.148:0/3295870344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac1562e0 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b25bc640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7ac154ee0 0x7fd7ac1e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b1dbb640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 0x7fd7ac0ad870 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00d800 tx=0x7fd79c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c004030 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c0041d0 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac0acd90 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0096a0 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.593+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac0ad350 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.594+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac073990 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.594+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd79c009840 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.595+0000 7fd7a27fc640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd79006b4d0 0x7fd79006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.595+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd79c08a280 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.595+0000 7fd7b15ba640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd79006b4d0 0x7fd79006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.595+0000 7fd7b15ba640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd79006b4d0 0x7fd79006d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8034000 tx=0x7fd7a802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.599+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd79c0592c0 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.837+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd7ac06d360 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.838+0000 7fd7a27fc640 1 -- 172.21.15.148:0/3295870344 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd79c058bc0 con 0x7fd7ac0ce3c0 2024-04-25T10:06:49.843 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "auth_dump": [ 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.a", 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmIkBDGhAAgO1wTf/ds07wdLlmzSWgbA==", 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.844 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.b", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmvaSoJRAAi5LWADD2LLk13V7rEyP0Yg==", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.c", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmQNUVIRAApe2fXhKOePOQu37XrQVpdw==", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.845 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mds.d", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmbWl2LBAAtElsYguvkRkch/ju90uhbQ==", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.846 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.0", 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAZKCpm1UPyChAA4Ye9Yi1zlTgGWmgzmpRoDA==", 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.1", 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAcKCpmb4abCRAAJ9ou+acdwrhSCfrAEVP1wg==", 2024-04-25T10:06:49.847 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.2", 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAfKCpmisvqBxAAEFekIFCPz6+pzjEZkhpnFA==", 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.848 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.3", 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAiKCpmM7G+BRAAaPnkfYqkSrt/EOmJMH8qSA==", 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.849 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.4", 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAoKCpm4i7GDRAABUXmGmDKo5nQVYXY+jDiAw==", 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.5", 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQArKCpmVHAODBAAFu+CMVrho9oCWunHcAq0vg==", 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.850 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.6", 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAuKCpm4zZZCRAAy62O/N9vVH4Hg7CldJ9QYg==", 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.851 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "osd.7", 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAxKCpmTAK1CRAA8It1zyyX8ayw/vU34R/x2A==", 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile osd", 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.0", 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww==", 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow r", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.1", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmWO/gOBAA8L7B24fMnt4RlZy2GiT8Eg==", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.853 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.2", 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmCgzcNRAAuRmBMpp0Ljy2vDbc/X1iSw==", 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.854 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.3", 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAWKCpmRTziAhAAlsoa5ThuX5yHec73W5kGsg==", 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow", 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow r", 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow rw", 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow rwx" 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "client.admin", 2024-04-25T10:06:49.855 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAUKCpmXBatOhAAecxAyQ7nbyoIYA2/0KyNwg==", 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "mgr": "allow *", 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow *", 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.x", 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmjzI4FxAAANU0hTM3Zw6tHrkrp1OQ4A==", 2024-04-25T10:06:49.856 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "entity": "mgr.y", 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "key": "AQAVKCpmJAOSExAAJGSRpbC3LqLTjbNRGPIjkQ==", 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": { 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": "allow *", 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: "osd": "allow *" 2024-04-25T10:06:49.857 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:49.858 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-04-25T10:06:49.858 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:49.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd79006b4d0 msgr2=0x7fd79006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd79006b4d0 0x7fd79006d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8034000 tx=0x7fd7a802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 msgr2=0x7fd7ac0ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 0x7fd7ac0ad870 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00d800 tx=0x7fd79c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 shutdown_connections 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd79006b4d0 0x7fd79006d990 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7ac154ee0 0x7fd7ac1e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7ac0cfcf0 0x7fd7ac156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 --2- 172.21.15.148:0/3295870344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7ac0ce3c0 0x7fd7ac0ad870 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 >> 172.21.15.148:0/3295870344 conn(0x7fd7ac06abe0 msgr2=0x7fd7ac15d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 shutdown_connections 2024-04-25T10:06:49.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.840+0000 7fd7b401f640 1 -- 172.21.15.148:0/3295870344 wait complete. 2024-04-25T10:06:49.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.954+0000 7f8b4c7b0640 1 Processor -- start 2024-04-25T10:06:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.954+0000 7f8b4c7b0640 1 -- start start 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4c7b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b4406abe0 0x7f8b4406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4c7b0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b44072620 0x7f8b44072a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4c7b0640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b440cfa60 con 0x7f8b4415b9c0 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4c7b0640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4406b580 con 0x7f8b44072620 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4c7b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440cfd40 con 0x7f8b4406abe0 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4a54c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8b4415b9c0 0x7f8b4415bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47816/0 (socket says 172.21.15.148:47816) 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4ad4d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8b44072620 0x7f8b44072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4a54c640 1 -- 172.21.15.148:0/3288005906 learned_addr learned my addr 172.21.15.148:0/3288005906 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b49d4b640 1 --2- 172.21.15.148:0/3288005906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b4406abe0 0x7f8b4406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/3288005906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b4406abe0 msgr2=0x7f8b4406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4ad4d640 1 --2- 172.21.15.148:0/3288005906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b4406abe0 0x7f8b4406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/3288005906 >> v1:172.21.15.148:6789/0 conn(0x7f8b4415b9c0 legacy=0x7f8b4415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.955+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/3288005906 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440d0020 con 0x7f8b44072620 2024-04-25T10:06:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.956+0000 7f8b4ad4d640 1 --2- 172.21.15.148:0/3288005906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44072620 0x7f8b44072a90 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f609ec543509b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.956+0000 7f8b4954a640 1 -- 172.21.15.148:0/3288005906 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c037070 con 0x7f8b44072620 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.956+0000 7f8b4954a640 1 -- 172.21.15.148:0/3288005906 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c032440 con 0x7f8b44072620 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.956+0000 7f8b4954a640 1 -- 172.21.15.148:0/3288005906 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c032810 con 0x7f8b44072620 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/3288005906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44072620 msgr2=0x7f8b44072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/3288005906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44072620 0x7f8b44072a90 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/3288005906 shutdown_connections 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/3288005906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b44072620 0x7f8b44072a90 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/3288005906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b4406abe0 0x7f8b4406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/3288005906 >> 172.21.15.148:0/3288005906 conn(0x7f8b440b2370 msgr2=0x7f8b440b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/3288005906 shutdown_connections 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.957+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/3288005906 wait complete. 2024-04-25T10:06:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.958+0000 7f8b4c7b0640 1 Processor -- start 2024-04-25T10:06:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.958+0000 7f8b4c7b0640 1 -- start start 2024-04-25T10:06:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4c7b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b4406abe0 0x7f8b44145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4c7b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b44072620 0x7f8b440cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4c7b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 0x7f8b441585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4c7b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44072fd0 con 0x7f8b4415b9c0 2024-04-25T10:06:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4c7b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440cf710 con 0x7f8b4406abe0 2024-04-25T10:06:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4c7b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44073150 con 0x7f8b44072620 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 0x7f8b441585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 0x7f8b441585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53044/0 (socket says 172.21.15.148:53044) 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/4208812082 learned_addr learned my addr 172.21.15.148:0/4208812082 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b49d4b640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b44072620 0x7f8b440cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4a54c640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b4406abe0 0x7f8b44145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b44072620 msgr2=0x7f8b440cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b44072620 0x7f8b440cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b4406abe0 msgr2=0x7f8b44145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b4406abe0 0x7f8b44145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4ad4d640 1 -- 172.21.15.148:0/4208812082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440cb8e0 con 0x7f8b4415b9c0 2024-04-25T10:06:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.959+0000 7f8b4a54c640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b4406abe0 0x7f8b44145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.960+0000 7f8b4ad4d640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 0x7f8b441585e0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c035040 tx=0x7f8b3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.960+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c037040 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.960+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c042070 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.960+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b440cbbc0 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.960+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c030b50 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.960+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b440cc180 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.961+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b44073990 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.961+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b3c030cf0 con 0x7f8b4415b9c0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.962+0000 7f8b3affd640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.962+0000 7f8b4a54c640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.962+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8b3c0aaa10 con 0x7f8b4415b9c0 2024-04-25T10:06:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.962+0000 7f8b4a54c640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8b400012e0 tx=0x7f8b40009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:49.966+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b3c032e20 con 0x7f8b4415b9c0 2024-04-25T10:06:50.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.136+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b44146560 con 0x7f8b4415b9c0 2024-04-25T10:06:50.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.137+0000 7f8b3affd640 1 -- 172.21.15.148:0/4208812082 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+13222 (secure 0 0 0) 0x7f8b3c079ae0 con 0x7f8b4415b9c0 2024-04-25T10:06:50.143 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:50.143 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":193,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:49.468362+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8b2806b4d0 msgr2=0x7f8b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8b400012e0 tx=0x7f8b40009040 comp rx=0 tx=0).stop 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 msgr2=0x7f8b441585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 0x7f8b441585e0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c035040 tx=0x7f8b3c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 shutdown_connections 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b4415b9c0 0x7f8b441585e0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b44072620 0x7f8b440cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 --2- 172.21.15.148:0/4208812082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b4406abe0 0x7f8b44145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 >> 172.21.15.148:0/4208812082 conn(0x7f8b440b2370 msgr2=0x7f8b4415e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 shutdown_connections 2024-04-25T10:06:50.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.139+0000 7f8b4c7b0640 1 -- 172.21.15.148:0/4208812082 wait complete. 2024-04-25T10:06:50.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:50.156 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.250+0000 7f9773acb640 1 Processor -- start 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.250+0000 7f9773acb640 1 -- start start 2024-04-25T10:06:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9773acb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f976c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9773acb640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976c0d2020 con 0x7f976c0d1b80 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9773acb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976c0d2300 con 0x7f976c0b4e60 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9773acb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c158610 con 0x7f976c158230 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9771066640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f976c0d1b80 0x7f976c06c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47822/0 (socket says 172.21.15.148:47822) 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9771066640 1 -- 172.21.15.148:0/1429999914 learned_addr learned my addr 172.21.15.148:0/1429999914 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9771867640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f976c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108928866 0 0) 0x7f976c0d2020 con 0x7f976c0d1b80 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.251+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9758003520 con 0x7f976c0d1b80 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1594964085 0 0) 0x7f9758003520 con 0x7f976c0d1b80 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 msgr2=0x7f976c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f976c0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 >> v1:172.21.15.188:6789/0 conn(0x7f976c0b4e60 legacy=0x7f976c06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c06bb30 con 0x7f976c0d1b80 2024-04-25T10:06:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f975c0016c0 con 0x7f976c0d1b80 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771867640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f976c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 >> v1:172.21.15.148:6789/0 conn(0x7f976c0d1b80 legacy=0x7f976c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f97580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9758003f20 0x7f9758004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758004d40 0x7f97580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c158610 con 0x7f9758003f20 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975800ba20 con 0x7f9758004d40 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975800bc20 con 0x7f976c158230 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771066640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9758003f20 0x7f9758004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771867640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f97580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9772068640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758004d40 0x7f97580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771066640 1 -- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 msgr2=0x7f97580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771066640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c158230 0x7f97580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771066640 1 -- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758004d40 msgr2=0x7f97580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771066640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758004d40 0x7f97580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.252+0000 7f9771066640 1 -- 172.21.15.148:0/1429999914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758003520 con 0x7f9758003f20 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9771066640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9758003f20 0x7f9758004800 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c210 tx=0x7f9758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff22157630d9af98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c007d30 con 0x7f9758003f20 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c005ad0 con 0x7f9758003f20 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9770865640 1 -- 172.21.15.148:0/1429999914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c007070 con 0x7f9758003f20 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9773acb640 1 -- 172.21.15.148:0/1429999914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9758003f20 msgr2=0x7f9758004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9773acb640 1 --2- 172.21.15.148:0/1429999914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9758003f20 0x7f9758004800 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c210 tx=0x7f9758005830 comp rx=0 tx=0).stop 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9772068640 1 -- 172.21.15.148:0/1429999914 reap_dead start 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9773acb640 1 -- 172.21.15.148:0/1429999914 shutdown_connections 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.253+0000 7f9773acb640 1 -- 172.21.15.148:0/1429999914 >> 172.21.15.148:0/1429999914 conn(0x7f976c061600 msgr2=0x7f976c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.254+0000 7f9773acb640 1 -- 172.21.15.148:0/1429999914 shutdown_connections 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.254+0000 7f9773acb640 1 -- 172.21.15.148:0/1429999914 wait complete. 2024-04-25T10:06:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 Processor -- start 2024-04-25T10:06:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 -- start start 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758003f20 0x7f976c0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 0x7f976c0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c0cb180 0x7f976c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c06ca40 con 0x7f976c154210 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c06bea0 con 0x7f9758003f20 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9773acb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c15e320 con 0x7f976c0cb180 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9771066640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 0x7f976c0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9771066640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 0x7f976c0cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53068/0 (socket says 172.21.15.148:53068) 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.255+0000 7f9771066640 1 -- 172.21.15.148:0/956019490 learned_addr learned my addr 172.21.15.148:0/956019490 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771867640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758003f20 0x7f976c0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9772068640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c0cb180 0x7f976c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771066640 1 -- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c0cb180 msgr2=0x7f976c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771066640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c0cb180 0x7f976c15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771066640 1 -- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758003f20 msgr2=0x7f976c0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771066640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758003f20 0x7f976c0cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771066640 1 -- 172.21.15.148:0/956019490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c154760 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9771066640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 0x7f976c0cac40 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f975c02ad30 tx=0x7f976c06be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c041020 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c001480 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c154a40 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c007070 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.256+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c155000 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.257+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c15d430 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.257+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f975c044020 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.258+0000 7f97627fc640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f975006b4d0 0x7f975006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.258+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f975c0aa030 con 0x7f976c154210 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.258+0000 7f9771867640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f975006b4d0 0x7f975006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.258+0000 7f9771867640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f975006b4d0 0x7f975006d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9768009b10 tx=0x7f9768008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.262+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f976c15d430 con 0x7f976c154210 2024-04-25T10:06:50.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.430+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f976c073a00 con 0x7f976c154210 2024-04-25T10:06:50.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.430+0000 7f97627fc640 1 -- 172.21.15.148:0/956019490 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7f975c078fe0 con 0x7f976c154210 2024-04-25T10:06:50.436 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:06:50.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f975006b4d0 msgr2=0x7f975006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f975006b4d0 0x7f975006d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9768009b10 tx=0x7f9768008040 comp rx=0 tx=0).stop 2024-04-25T10:06:50.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 msgr2=0x7f976c0cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 0x7f976c0cac40 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f975c02ad30 tx=0x7f976c06be10 comp rx=0 tx=0).stop 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 shutdown_connections 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f975006b4d0 0x7f975006d990 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f976c0cb180 0x7f976c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f976c154210 0x7f976c0cac40 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 --2- 172.21.15.148:0/956019490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9758003f20 0x7f976c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 >> 172.21.15.148:0/956019490 conn(0x7f976c061600 msgr2=0x7f976c15e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 shutdown_connections 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.432+0000 7f9773acb640 1 -- 172.21.15.148:0/956019490 wait complete. 2024-04-25T10:06:50.449 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:50.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.541+0000 7f22f42e4640 1 Processor -- start 2024-04-25T10:06:50.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.541+0000 7f22f42e4640 1 -- start start 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f42e4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22ec164970 0x7f22ec166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f42e4640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec144b30 con 0x7f22ec0ce920 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f42e4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec0712d0 con 0x7f22ec144750 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f42e4640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec0715b0 con 0x7f22ec164970 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f2881640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f22ec164970 0x7f22ec166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f187f640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f22ec0ce920 0x7f22ec0cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47832/0 (socket says 172.21.15.148:47832) 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f187f640 1 -- 172.21.15.148:0/2273207475 learned_addr learned my addr 172.21.15.148:0/2273207475 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f2881640 1 -- 172.21.15.148:0/2273207475 >> v1:172.21.15.188:6789/0 conn(0x7f22ec144750 legacy=0x7f22ec06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f2881640 1 -- 172.21.15.148:0/2273207475 >> v1:172.21.15.148:6789/0 conn(0x7f22ec0ce920 legacy=0x7f22ec0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f2881640 1 -- 172.21.15.148:0/2273207475 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec071890 con 0x7f22ec164970 2024-04-25T10:06:50.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.542+0000 7f22f107e640 1 -- 172.21.15.148:0/2273207475 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792406327 0 0) 0x7f22ec144b30 con 0x7f22ec0ce920 2024-04-25T10:06:50.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.543+0000 7f22f2881640 1 --2- 172.21.15.148:0/2273207475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec164970 0x7f22ec166d60 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f22e40097a0 tx=0x7f22e402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c6cd481e85f25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.550 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.543+0000 7f22f107e640 1 -- 172.21.15.148:0/2273207475 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4037070 con 0x7f22ec164970 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.543+0000 7f22f107e640 1 -- 172.21.15.148:0/2273207475 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e402ae50 con 0x7f22ec164970 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.543+0000 7f22f107e640 1 -- 172.21.15.148:0/2273207475 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4032660 con 0x7f22ec164970 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 -- 172.21.15.148:0/2273207475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec164970 msgr2=0x7f22ec166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 --2- 172.21.15.148:0/2273207475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec164970 0x7f22ec166d60 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f22e40097a0 tx=0x7f22e402a140 comp rx=0 tx=0).stop 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 -- 172.21.15.148:0/2273207475 shutdown_connections 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 --2- 172.21.15.148:0/2273207475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec164970 0x7f22ec166d60 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 -- 172.21.15.148:0/2273207475 >> 172.21.15.148:0/2273207475 conn(0x7f22ec06abe0 msgr2=0x7f22ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 -- 172.21.15.148:0/2273207475 shutdown_connections 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.544+0000 7f22f42e4640 1 -- 172.21.15.148:0/2273207475 wait complete. 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.545+0000 7f22f42e4640 1 Processor -- start 2024-04-25T10:06:50.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.545+0000 7f22f42e4640 1 -- start start 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 0x7f22ec1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22ec144750 0x7f22ec15f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22ec164970 0x7f22ec0ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec145450 con 0x7f22ec0ce920 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec06f020 con 0x7f22ec144750 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec1455d0 con 0x7f22ec164970 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 0x7f22ec1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 0x7f22ec1f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53084/0 (socket says 172.21.15.148:53084) 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 -- 172.21.15.148:0/637194859 learned_addr learned my addr 172.21.15.148:0/637194859 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 -- 172.21.15.148:0/637194859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22ec164970 msgr2=0x7f22ec0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22ec164970 0x7f22ec0ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 -- 172.21.15.148:0/637194859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22ec144750 msgr2=0x7f22ec15f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22ec144750 0x7f22ec15f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 -- 172.21.15.148:0/637194859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec1f4a20 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f2080640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 0x7f22ec1f4cf0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c7c0 tx=0x7f22e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8016020 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e800aa50 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec1f5230 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.546+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8012420 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.547+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec1f4740 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.547+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0685e0 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.548+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22e8019020 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.548+0000 7f22e2ffd640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22d006b500 0x7f22d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.548+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f22e8089e70 con 0x7f22ec0ce920 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.548+0000 7f22f187f640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22d006b500 0x7f22d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.549+0000 7f22f187f640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22d006b500 0x7f22d006d9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f22dc007fd0 tx=0x7f22dc007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.558 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.552+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22ec0685e0 con 0x7f22ec0ce920 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.724+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f22ec0d0510 con 0x7f22ec0ce920 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.724+0000 7f22e2ffd640 1 -- 172.21.15.148:0/637194859 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v193) v1 ==== 84+0+65 (secure 0 0 0) 0x7f22e8020070 con 0x7f22ec0ce920 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stdout:[ 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stdout: { 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stdout: "poolnum": 1, 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi148.stdout: "poolname": ".mgr" 2024-04-25T10:06:50.731 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:06:50.731 INFO:teuthology.orchestra.run.smithi148.stdout:] 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22d006b500 msgr2=0x7f22d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22d006b500 0x7f22d006d9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f22dc007fd0 tx=0x7f22dc007960 comp rx=0 tx=0).stop 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 msgr2=0x7f22ec1f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 0x7f22ec1f4cf0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c7c0 tx=0x7f22e800cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 shutdown_connections 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f22d006b500 0x7f22d006d9c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22ec164970 0x7f22ec0ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22ec144750 0x7f22ec15f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 --2- 172.21.15.148:0/637194859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22ec0ce920 0x7f22ec1f4cf0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 >> 172.21.15.148:0/637194859 conn(0x7f22ec06abe0 msgr2=0x7f22ec1410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 shutdown_connections 2024-04-25T10:06:50.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.726+0000 7f22f42e4640 1 -- 172.21.15.148:0/637194859 wait complete. 2024-04-25T10:06:50.743 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:50.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.838+0000 7f8963309640 1 Processor -- start 2024-04-25T10:06:50.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.838+0000 7f8963309640 1 -- start start 2024-04-25T10:06:50.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f8963309640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f895c0d0ab0 0x7f895c168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f8963309640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c0d2420 con 0x7f895c0d34c0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f8963309640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895c0d2700 con 0x7f895c15dac0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f8963309640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895c0d3030 con 0x7f895c0d0ab0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f89608a4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f895c0d34c0 0x7f895c0d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47842/0 (socket says 172.21.15.148:47842) 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f89608a4640 1 -- 172.21.15.148:0/2262874578 learned_addr learned my addr 172.21.15.148:0/2262874578 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.839+0000 7f89618a6640 1 --2- 172.21.15.148:0/2262874578 >> v2:172.21.15.188:3300/0 conn(0x7f895c0d0ab0 0x7f895c168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483775973 0 0) 0x7f895c0d2420 con 0x7f895c0d34c0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8948003520 con 0x7f895c0d34c0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141848057 0 0) 0x7f8948003520 con 0x7f895c0d34c0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 >> v2:172.21.15.188:3300/0 conn(0x7f895c0d0ab0 msgr2=0x7f895c168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 --2- 172.21.15.148:0/2262874578 >> v2:172.21.15.188:3300/0 conn(0x7f895c0d0ab0 0x7f895c168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 >> v1:172.21.15.188:6789/0 conn(0x7f895c15dac0 legacy=0x7f895c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0d29e0 con 0x7f895c0d34c0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f894c0016c0 con 0x7f895c0d34c0 2024-04-25T10:06:50.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 >> v1:172.21.15.148:6789/0 conn(0x7f895c0d34c0 legacy=0x7f895c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89480039b0 0x7f8948003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8948004800 0x7f8948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89480051c0 0x7f894800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0d3030 con 0x7f8948004800 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894800c070 con 0x7f89480051c0 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894800c270 con 0x7f89480039b0 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89608a4640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8948004800 0x7f8948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89610a5640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89480039b0 0x7f8948003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89618a6640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89480051c0 0x7f894800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89608a4640 1 -- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89480039b0 msgr2=0x7f8948003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89608a4640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89480039b0 0x7f8948003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89608a4640 1 -- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89480051c0 msgr2=0x7f894800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89608a4640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89480051c0 0x7f894800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89608a4640 1 -- 172.21.15.148:0/2262874578 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8948003520 con 0x7f8948004800 2024-04-25T10:06:50.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89618a6640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89480051c0 0x7f894800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89610a5640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89480039b0 0x7f8948003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.840+0000 7f89618a6640 1 -- 172.21.15.148:0/2262874578 reap_dead start 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.841+0000 7f89608a4640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8948004800 0x7f8948004c80 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c240 tx=0x7f8948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d640601ee17ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.841+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c007ce0 con 0x7f8948004800 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.841+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c005ad0 con 0x7f8948004800 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.841+0000 7f8953fff640 1 -- 172.21.15.148:0/2262874578 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c007070 con 0x7f8948004800 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 -- 172.21.15.148:0/2262874578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8948004800 msgr2=0x7f8948004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8948004800 0x7f8948004c80 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f894c00c240 tx=0x7f8948005830 comp rx=0 tx=0).stop 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 -- 172.21.15.148:0/2262874578 shutdown_connections 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 --2- 172.21.15.148:0/2262874578 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8948004800 0x7f8948004c80 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 -- 172.21.15.148:0/2262874578 >> 172.21.15.148:0/2262874578 conn(0x7f895c061c30 msgr2=0x7f895c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 -- 172.21.15.148:0/2262874578 shutdown_connections 2024-04-25T10:06:50.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.842+0000 7f8963309640 1 -- 172.21.15.148:0/2262874578 wait complete. 2024-04-25T10:06:50.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.843+0000 7f8963309640 1 Processor -- start 2024-04-25T10:06:50.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.843+0000 7f8963309640 1 -- start start 2024-04-25T10:06:50.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8948004800 0x7f895c1f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f895c15e4f0 0x7f895c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 0x7f895c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b97d0 con 0x7f895c15ee90 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c15dea0 con 0x7f895c15e4f0 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c0b9950 con 0x7f8948004800 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 0x7f895c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 0x7f895c0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53108/0 (socket says 172.21.15.148:53108) 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 -- 172.21.15.148:0/3306651685 learned_addr learned my addr 172.21.15.148:0/3306651685 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89608a4640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f895c15e4f0 0x7f895c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 -- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8948004800 msgr2=0x7f895c1f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89610a5640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8948004800 0x7f895c1f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8948004800 0x7f895c1f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 -- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f895c15e4f0 msgr2=0x7f895c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f895c15e4f0 0x7f895c15e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 -- 172.21.15.148:0/3306651685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c15f320 con 0x7f895c15ee90 2024-04-25T10:06:50.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f89618a6640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 0x7f895c0b3f40 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f895c06cc00 tx=0x7f8954011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954003610 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895401b070 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895c1571e0 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.844+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954003a10 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.845+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895c0b3460 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.846+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895c0739e0 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.846+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8954003bb0 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.846+0000 7f8951ffb640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f894006b4d0 0x7f894006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.846+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8954089ec0 con 0x7f895c15ee90 2024-04-25T10:06:50.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.846+0000 7f89610a5640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f894006b4d0 0x7f894006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.847+0000 7f89610a5640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f894006b4d0 0x7f894006d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8958007fd0 tx=0x7f8958009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:50.851+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8954024020 con 0x7f895c15ee90 2024-04-25T10:06:51.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.021+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f895c0c8840 con 0x7f895c15ee90 2024-04-25T10:06:51.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.474+0000 7f8951ffb640 1 -- 172.21.15.148:0/3306651685 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v194) v1 ==== 138+0+0 (secure 0 0 0) 0x7f895c15f320 con 0x7f895c15ee90 2024-04-25T10:06:51.480 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f894006b4d0 msgr2=0x7f894006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f894006b4d0 0x7f894006d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8958007fd0 tx=0x7f8958009040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 msgr2=0x7f895c0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 0x7f895c0b3f40 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f895c06cc00 tx=0x7f8954011920 comp rx=0 tx=0).stop 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 shutdown_connections 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f894006b4d0 0x7f894006d990 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.483 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c15ee90 0x7f895c0b3f40 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f895c15e4f0 0x7f895c15e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 --2- 172.21.15.148:0/3306651685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8948004800 0x7f895c1f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 >> 172.21.15.148:0/3306651685 conn(0x7f895c061c30 msgr2=0x7f895c0c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 shutdown_connections 2024-04-25T10:06:51.484 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.477+0000 7f8963309640 1 -- 172.21.15.148:0/3306651685 wait complete. 2024-04-25T10:06:51.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.596+0000 7f8d77ffe640 1 Processor -- start 2024-04-25T10:06:51.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.596+0000 7f8d77ffe640 1 -- start start 2024-04-25T10:06:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d77ffe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8d70158230 0x7f8d700d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d77ffe640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700d2020 con 0x7f8d700b4e60 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d77ffe640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700d2300 con 0x7f8d700d1b80 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d77ffe640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d70158610 con 0x7f8d70158230 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d7659b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f8d700b4e60 0x7f8d7006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47858/0 (socket says 172.21.15.148:47858) 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d7659b640 1 -- 172.21.15.148:0/2499689204 learned_addr learned my addr 172.21.15.148:0/2499689204 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d75d9a640 1 --2- 172.21.15.148:0/2499689204 >> v2:172.21.15.188:3300/0 conn(0x7f8d70158230 0x7f8d700d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824818473 0 0) 0x7f8d700d2020 con 0x7f8d700b4e60 2024-04-25T10:06:51.611 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d5c003520 con 0x7f8d700b4e60 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812395472 0 0) 0x7f8d5c003520 con 0x7f8d700b4e60 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 >> v2:172.21.15.188:3300/0 conn(0x7f8d70158230 msgr2=0x7f8d700d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 --2- 172.21.15.148:0/2499689204 >> v2:172.21.15.188:3300/0 conn(0x7f8d70158230 0x7f8d700d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 >> v1:172.21.15.188:6789/0 conn(0x7f8d700d1b80 legacy=0x7f8d7006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7006bb30 con 0x7f8d700b4e60 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f8d6c001670 con 0x7f8d700b4e60 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 >> v1:172.21.15.148:6789/0 conn(0x7f8d700b4e60 legacy=0x7f8d7006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.597+0000 7f8d74d98640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d5c0039b0 0x7f8d5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d74d98640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d74d98640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70158610 con 0x7f8d5c0051c0 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c00c060 con 0x7f8d5c0039b0 2024-04-25T10:06:51.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c00c260 con 0x7f8d5c004800 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d7659b640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d75599640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d75d9a640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d5c0039b0 0x7f8d5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d7659b640 1 -- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c004800 msgr2=0x7f8d5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d7659b640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d7659b640 1 -- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d5c0039b0 msgr2=0x7f8d5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d7659b640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d5c0039b0 0x7f8d5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d7659b640 1 -- 172.21.15.148:0/2499689204 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c003520 con 0x7f8d5c0051c0 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d75599640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:51.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.598+0000 7f8d75d9a640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d5c0039b0 0x7f8d5c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d7659b640 1 -- 172.21.15.148:0/2499689204 reap_dead start 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d7659b640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c240 tx=0x7f8d5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a78d8f73e2b744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007cf0 con 0x7f8d5c0051c0 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0058d0 con 0x7f8d5c0051c0 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d74d98640 1 -- 172.21.15.148:0/2499689204 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c005ce0 con 0x7f8d5c0051c0 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 -- 172.21.15.148:0/2499689204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0051c0 msgr2=0x7f8d5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c240 tx=0x7f8d5c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 -- 172.21.15.148:0/2499689204 shutdown_connections 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/2499689204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 -- 172.21.15.148:0/2499689204 >> 172.21.15.148:0/2499689204 conn(0x7f8d70061600 msgr2=0x7f8d700619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 -- 172.21.15.148:0/2499689204 shutdown_connections 2024-04-25T10:06:51.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.599+0000 7f8d77ffe640 1 -- 172.21.15.148:0/2499689204 wait complete. 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 Processor -- start 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 -- start start 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c0051c0 0x7f8d7014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d7015a1d0 0x7f8d7015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 0x7f8d701f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015e320 con 0x7f8d701f3d20 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7006bea0 con 0x7f8d7015a1d0 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d77ffe640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7015e4a0 con 0x7f8d5c0051c0 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d7659b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 0x7f8d701f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d7659b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 0x7f8d701f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53112/0 (socket says 172.21.15.148:53112) 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.601+0000 7f8d7659b640 1 -- 172.21.15.148:0/662718347 learned_addr learned my addr 172.21.15.148:0/662718347 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d75d9a640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c0051c0 0x7f8d7014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d75599640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d7015a1d0 0x7f8d7015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d7659b640 1 -- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c0051c0 msgr2=0x7f8d7014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d7659b640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c0051c0 0x7f8d7014c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d7659b640 1 -- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d7015a1d0 msgr2=0x7f8d7015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d7659b640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d7015a1d0 0x7f8d7015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d7659b640 1 -- 172.21.15.148:0/662718347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70158950 con 0x7f8d701f3d20 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d7659b640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 0x7f8d701f41b0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c360 tx=0x7f8d7006bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007cf0 con 0x7f8d701f3d20 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c042070 con 0x7f8d701f3d20 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c001c30 con 0x7f8d701f3d20 2024-04-25T10:06:51.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d70158c30 con 0x7f8d701f3d20 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.602+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d701591f0 con 0x7f8d701f3d20 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.603+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7006de50 con 0x7f8d701f3d20 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.603+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d6c03b070 con 0x7f8d701f3d20 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.604+0000 7f8d667fc640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d5406b4d0 0x7f8d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.604+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f8d6c0a9a10 con 0x7f8d701f3d20 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.604+0000 7f8d75d9a640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d5406b4d0 0x7f8d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.604+0000 7f8d75d9a640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d5406b4d0 0x7f8d5406d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8d700cd790 tx=0x7f8d6800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.608+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d6c078a00 con 0x7f8d701f3d20 2024-04-25T10:06:51.787 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:51.781+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8d70073a00 con 0x7f8d701f3d20 2024-04-25T10:06:52.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.476+0000 7f8d667fc640 1 -- 172.21.15.148:0/662718347 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v195) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8d6c078300 con 0x7f8d701f3d20 2024-04-25T10:06:52.482 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' created 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d5406b4d0 msgr2=0x7f8d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d5406b4d0 0x7f8d5406d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8d700cd790 tx=0x7f8d6800b040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 msgr2=0x7f8d701f41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 0x7f8d701f41b0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c360 tx=0x7f8d7006bd00 comp rx=0 tx=0).stop 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 shutdown_connections 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8d5406b4d0 0x7f8d5406d990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d701f3d20 0x7f8d701f41b0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d7015a1d0 0x7f8d7015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 --2- 172.21.15.148:0/662718347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d5c0051c0 0x7f8d7014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 >> 172.21.15.148:0/662718347 conn(0x7f8d70061600 msgr2=0x7f8d700b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 shutdown_connections 2024-04-25T10:06:52.485 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.479+0000 7f8d77ffe640 1 -- 172.21.15.148:0/662718347 wait complete. 2024-04-25T10:06:52.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:52.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.604+0000 7f2e518fc640 1 Processor -- start 2024-04-25T10:06:52.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.604+0000 7f2e518fc640 1 -- start start 2024-04-25T10:06:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e518fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 0x7f2e4c0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e518fc640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c0d23a0 con 0x7f2e4c0d1ce0 2024-04-25T10:06:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e518fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c0d2680 con 0x7f2e4c15dc20 2024-04-25T10:06:52.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e518fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0d2960 con 0x7f2e4c0ced80 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e4b7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2e4c0d1ce0 0x7f2e4c0b6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:47862/0 (socket says 172.21.15.148:47862) 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e4b7fe640 1 -- 172.21.15.148:0/1459999761 learned_addr learned my addr 172.21.15.148:0/1459999761 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783375191 0 0) 0x7f2e4c0d23a0 con 0x7f2e4c0d1ce0 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e34003520 con 0x7f2e4c0d1ce0 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.605+0000 7f2e4a7fc640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 0x7f2e4c0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3197602718 0 0) 0x7f2e34003520 con 0x7f2e4c0d1ce0 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 msgr2=0x7f2e4c0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 0x7f2e4c0d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 >> v1:172.21.15.188:6789/0 conn(0x7f2e4c15dc20 legacy=0x7f2e4c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0d2c40 con 0x7f2e4c0d1ce0 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f2e440019f0 con 0x7f2e4c0d1ce0 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 >> v1:172.21.15.148:6789/0 conn(0x7f2e4c0d1ce0 legacy=0x7f2e4c0b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e340039e0 0x7f2e34003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34004800 0x7f2e34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 0x7f2e3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0d2960 con 0x7f2e34004800 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34005520 con 0x7f2e340039e0 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3400bf00 con 0x7f2e4c0ced80 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4a7fc640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34004800 0x7f2e34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4affd640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e340039e0 0x7f2e34003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4b7fe640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 0x7f2e3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4a7fc640 1 -- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 msgr2=0x7f2e3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4a7fc640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c0ced80 0x7f2e3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4a7fc640 1 -- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e340039e0 msgr2=0x7f2e34003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4a7fc640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e340039e0 0x7f2e34003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.606+0000 7f2e4a7fc640 1 -- 172.21.15.148:0/1459999761 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34003520 con 0x7f2e34004800 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e4a7fc640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34004800 0x7f2e34004c80 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000ba30 tx=0x7f2e34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce91e92860b3086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40030070 con 0x7f2e34004800 2024-04-25T10:06:52.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4002ade0 con 0x7f2e34004800 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e49ffb640 1 -- 172.21.15.148:0/1459999761 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000b600 con 0x7f2e34004800 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e518fc640 1 -- 172.21.15.148:0/1459999761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34004800 msgr2=0x7f2e34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e518fc640 1 --2- 172.21.15.148:0/1459999761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e34004800 0x7f2e34004c80 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000ba30 tx=0x7f2e34005830 comp rx=0 tx=0).stop 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e4b7fe640 1 -- 172.21.15.148:0/1459999761 reap_dead start 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e518fc640 1 -- 172.21.15.148:0/1459999761 shutdown_connections 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e518fc640 1 -- 172.21.15.148:0/1459999761 >> 172.21.15.148:0/1459999761 conn(0x7f2e4c061a20 msgr2=0x7f2e4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e518fc640 1 -- 172.21.15.148:0/1459999761 shutdown_connections 2024-04-25T10:06:52.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.607+0000 7f2e518fc640 1 -- 172.21.15.148:0/1459999761 wait complete. 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 Processor -- start 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 -- start start 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e34004800 0x7f2e4c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c15e000 0x7f2e4c0cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 0x7f2e4c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0d35c0 con 0x7f2e4c15e480 2024-04-25T10:06:52.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0d3440 con 0x7f2e34004800 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.609+0000 7f2e518fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0d3740 con 0x7f2e4c15e000 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 0x7f2e4c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 0x7f2e4c0ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53116/0 (socket says 172.21.15.148:53116) 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 -- 172.21.15.148:0/556248270 learned_addr learned my addr 172.21.15.148:0/556248270 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4affd640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e34004800 0x7f2e4c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4a7fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c15e000 0x7f2e4c0cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 -- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c15e000 msgr2=0x7f2e4c0cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c15e000 0x7f2e4c0cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 -- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e34004800 msgr2=0x7f2e4c14dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e34004800 0x7f2e4c14dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 -- 172.21.15.148:0/556248270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c14e1f0 con 0x7f2e4c15e480 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e4b7fe640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 0x7f2e4c0ccb50 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000f80 tx=0x7f2e44007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44017020 con 0x7f2e4c15e480 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4400ace0 con 0x7f2e4c15e480 2024-04-25T10:06:52.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c15e9e0 con 0x7f2e4c15e480 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44013630 con 0x7f2e4c15e480 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.610+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c1e6180 con 0x7f2e4c15e480 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.611+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c073a00 con 0x7f2e4c15e480 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.612+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e4401a020 con 0x7f2e4c15e480 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.612+0000 7f2e33fff640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e3806b4d0 0x7f2e3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.612+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f2e4408a9f0 con 0x7f2e4c15e480 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.612+0000 7f2e4affd640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e3806b4d0 0x7f2e3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.613+0000 7f2e4affd640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e3806b4d0 0x7f2e3806d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c001760 tx=0x7f2e3c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.617+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e44021020 con 0x7f2e4c15e480 2024-04-25T10:06:52.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:52.823+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2e4c0b84b0 con 0x7f2e4c15e480 2024-04-25T10:06:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.500+0000 7f2e33fff640 1 -- 172.21.15.148:0/556248270 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v110) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2e44059180 con 0x7f2e4c15e480 2024-04-25T10:06:53.506 INFO:teuthology.orchestra.run.smithi148.stderr:new fs with metadata pool 29 and data pool 30 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e3806b4d0 msgr2=0x7f2e3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e3806b4d0 0x7f2e3806d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c001760 tx=0x7f2e3c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 msgr2=0x7f2e4c0ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 0x7f2e4c0ccb50 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f2e44000f80 tx=0x7f2e44007bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 shutdown_connections 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2e3806b4d0 0x7f2e3806d990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e4c15e480 0x7f2e4c0ccb50 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e4c15e000 0x7f2e4c0cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 --2- 172.21.15.148:0/556248270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e34004800 0x7f2e4c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 >> 172.21.15.148:0/556248270 conn(0x7f2e4c061a20 msgr2=0x7f2e4c0b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 shutdown_connections 2024-04-25T10:06:53.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.506+0000 7f2e518fc640 1 -- 172.21.15.148:0/556248270 wait complete. 2024-04-25T10:06:53.528 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:53.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.621+0000 7f69dc01b640 1 Processor -- start 2024-04-25T10:06:53.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.621+0000 7f69dc01b640 1 -- start start 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69dc01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d406abe0 0x7f69d406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69dc01b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69d415d370 0x7f69d415d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69dc01b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d41626e0 con 0x7f69d40ce330 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69dc01b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d406b500 con 0x7f69d415d370 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69dc01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4070270 con 0x7f69d406abe0 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69d9db7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d406abe0 0x7f69d406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69da5b8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69d415d370 0x7f69d415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69d95b6640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f69d40ce330 0x7f69d406fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34548/0 (socket says 172.21.15.148:34548) 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69d95b6640 1 -- 172.21.15.148:0/3906036355 learned_addr learned my addr 172.21.15.148:0/3906036355 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69d8db5640 1 -- 172.21.15.148:0/3906036355 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028365881 0 0) 0x7f69d41626e0 con 0x7f69d40ce330 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69d8db5640 1 -- 172.21.15.148:0/3906036355 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c0003520 con 0x7f69d40ce330 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69da5b8640 1 -- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d406abe0 msgr2=0x7f69d406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69da5b8640 1 --2- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d406abe0 0x7f69d406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69da5b8640 1 -- 172.21.15.148:0/3906036355 >> v1:172.21.15.188:6789/0 conn(0x7f69d40ce330 legacy=0x7f69d406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69da5b8640 1 -- 172.21.15.148:0/3906036355 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d40cd270 con 0x7f69d415d370 2024-04-25T10:06:53.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.622+0000 7f69d9db7640 1 --2- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d406abe0 0x7f69d406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:53.635 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.623+0000 7f69da5b8640 1 --2- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d415d370 0x7f69d415d800 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009d90 tx=0x7f69cc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361def0c75fb39ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.623+0000 7f69d8db5640 1 -- 172.21.15.148:0/3906036355 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc031600 con 0x7f69d415d370 2024-04-25T10:06:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.623+0000 7f69d8db5640 1 -- 172.21.15.148:0/3906036355 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc0317a0 con 0x7f69d415d370 2024-04-25T10:06:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.623+0000 7f69d8db5640 1 -- 172.21.15.148:0/3906036355 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc031b50 con 0x7f69d415d370 2024-04-25T10:06:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 -- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d415d370 msgr2=0x7f69d415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.636 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 --2- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d415d370 0x7f69d415d800 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009d90 tx=0x7f69cc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 -- 172.21.15.148:0/3906036355 shutdown_connections 2024-04-25T10:06:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 --2- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d415d370 0x7f69d415d800 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 --2- 172.21.15.148:0/3906036355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d406abe0 0x7f69d406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 -- 172.21.15.148:0/3906036355 >> 172.21.15.148:0/3906036355 conn(0x7f69d40b2220 msgr2=0x7f69d40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 -- 172.21.15.148:0/3906036355 shutdown_connections 2024-04-25T10:06:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.624+0000 7f69dc01b640 1 -- 172.21.15.148:0/3906036355 wait complete. 2024-04-25T10:06:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.625+0000 7f69dc01b640 1 Processor -- start 2024-04-25T10:06:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.625+0000 7f69dc01b640 1 -- start start 2024-04-25T10:06:53.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d406abe0 0x7f69d415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d40ce330 0x7f69d4157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 0x7f69d40b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cdb10 con 0x7f69d415d370 2024-04-25T10:06:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d40cd830 con 0x7f69d406abe0 2024-04-25T10:06:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4161780 con 0x7f69d40ce330 2024-04-25T10:06:53.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 0x7f69d40b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 0x7f69d40b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33174/0 (socket says 172.21.15.148:33174) 2024-04-25T10:06:53.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 -- 172.21.15.148:0/2910867784 learned_addr learned my addr 172.21.15.148:0/2910867784 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:53.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69d9db7640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d406abe0 0x7f69d415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69d95b6640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d40ce330 0x7f69d4157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 -- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d40ce330 msgr2=0x7f69d4157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d40ce330 0x7f69d4157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 -- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d406abe0 msgr2=0x7f69d415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d406abe0 0x7f69d415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 -- 172.21.15.148:0/2910867784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4158440 con 0x7f69d415d370 2024-04-25T10:06:53.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69da5b8640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 0x7f69d40b5c30 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009eb0 tx=0x7f69cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc040070 con 0x7f69d415d370 2024-04-25T10:06:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69cc03c850 con 0x7f69d415d370 2024-04-25T10:06:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d415fd50 con 0x7f69d415d370 2024-04-25T10:06:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69cc03cc40 con 0x7f69d415d370 2024-04-25T10:06:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.626+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d40b34f0 con 0x7f69d415d370 2024-04-25T10:06:53.642 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.627+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d40afe50 con 0x7f69d415d370 2024-04-25T10:06:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.628+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69cc039040 con 0x7f69d415d370 2024-04-25T10:06:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.628+0000 7f69ca7fc640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69b806b4d0 0x7f69b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.628+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69cc0ab540 con 0x7f69d415d370 2024-04-25T10:06:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.628+0000 7f69d9db7640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69b806b4d0 0x7f69b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.629+0000 7f69d9db7640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69b806b4d0 0x7f69b806d990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f69d406fd60 tx=0x7f69d0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.632+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69cc079d10 con 0x7f69d415d370 2024-04-25T10:06:53.810 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.804+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69d40b6670 con 0x7f69d415d370 2024-04-25T10:06:53.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.805+0000 7f69ca7fc640 1 -- 172.21.15.148:0/2910867784 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f69cc07ddb0 con 0x7f69d415d370 2024-04-25T10:06:53.812 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:53.812 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":196,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:52.824561+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:53.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69b806b4d0 msgr2=0x7f69b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69b806b4d0 0x7f69b806d990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f69d406fd60 tx=0x7f69d0009190 comp rx=0 tx=0).stop 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 msgr2=0x7f69d40b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 0x7f69d40b5c30 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f69cc009eb0 tx=0x7f69cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 shutdown_connections 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f69b806b4d0 0x7f69b806d990 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69d415d370 0x7f69d40b5c30 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69d40ce330 0x7f69d4157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 --2- 172.21.15.148:0/2910867784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69d406abe0 0x7f69d415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 >> 172.21.15.148:0/2910867784 conn(0x7f69d40b2220 msgr2=0x7f69d40af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 shutdown_connections 2024-04-25T10:06:53.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.807+0000 7f69dc01b640 1 -- 172.21.15.148:0/2910867784 wait complete. 2024-04-25T10:06:53.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:53.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.919+0000 7facc94bd640 1 Processor -- start 2024-04-25T10:06:53.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.919+0000 7facc94bd640 1 -- start start 2024-04-25T10:06:53.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc94bd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7facc415da70 0x7facc4161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc94bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7facc40b9730 0x7facc40ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc94bd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc4162020 con 0x7facc4161b80 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc94bd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc41625e0 con 0x7facc40b9730 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc94bd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc4162300 con 0x7facc415da70 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7facc40b9730 0x7facc40ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7facc40b9730 0x7facc40ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33186/0 (socket says 172.21.15.148:33186) 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 -- 172.21.15.148:0/1071564840 learned_addr learned my addr 172.21.15.148:0/1071564840 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 -- 172.21.15.148:0/1071564840 >> v2:172.21.15.188:3300/0 conn(0x7facc415da70 msgr2=0x7facc4161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 --2- 172.21.15.148:0/1071564840 >> v2:172.21.15.188:3300/0 conn(0x7facc415da70 0x7facc4161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 -- 172.21.15.148:0/1071564840 >> v1:172.21.15.188:6789/0 conn(0x7facc4161b80 legacy=0x7facc406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.920+0000 7facc27fc640 1 -- 172.21.15.148:0/1071564840 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc40d1af0 con 0x7facc40b9730 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.921+0000 7facc27fc640 1 --2- 172.21.15.148:0/1071564840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc40b9730 0x7facc40ce650 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7facc40b61a0 tx=0x7facb802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d79cf83da68f53b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.921+0000 7facc1ffb640 1 -- 172.21.15.148:0/1071564840 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8037070 con 0x7facc40b9730 2024-04-25T10:06:53.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.921+0000 7facc1ffb640 1 -- 172.21.15.148:0/1071564840 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8032440 con 0x7facc40b9730 2024-04-25T10:06:53.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.921+0000 7facc1ffb640 1 -- 172.21.15.148:0/1071564840 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8032810 con 0x7facc40b9730 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 -- 172.21.15.148:0/1071564840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc40b9730 msgr2=0x7facc40ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 --2- 172.21.15.148:0/1071564840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc40b9730 0x7facc40ce650 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7facc40b61a0 tx=0x7facb802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 -- 172.21.15.148:0/1071564840 shutdown_connections 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 --2- 172.21.15.148:0/1071564840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc40b9730 0x7facc40ce650 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 --2- 172.21.15.148:0/1071564840 >> v2:172.21.15.188:3300/0 conn(0x7facc415da70 0x7facc4161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 -- 172.21.15.148:0/1071564840 >> 172.21.15.148:0/1071564840 conn(0x7facc4061a20 msgr2=0x7facc4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 -- 172.21.15.148:0/1071564840 shutdown_connections 2024-04-25T10:06:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.922+0000 7facc94bd640 1 -- 172.21.15.148:0/1071564840 wait complete. 2024-04-25T10:06:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.923+0000 7facc94bd640 1 Processor -- start 2024-04-25T10:06:53.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.923+0000 7facc94bd640 1 -- start start 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc94bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc415da70 0x7facc4159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc94bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc4161b80 0x7facc40b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc94bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 0x7facc4152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc94bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40d3300 con 0x7facc4151db0 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc94bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc415de50 con 0x7facc4161b80 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc94bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40d3480 con 0x7facc415da70 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 0x7facc4152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 0x7facc4152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33194/0 (socket says 172.21.15.148:33194) 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 -- 172.21.15.148:0/4110827583 learned_addr learned my addr 172.21.15.148:0/4110827583 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc27fc640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc4161b80 0x7facc40b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc2ffd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc415da70 0x7facc4159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 -- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc415da70 msgr2=0x7facc4159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc415da70 0x7facc4159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 -- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc4161b80 msgr2=0x7facc40b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc4161b80 0x7facc40b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 -- 172.21.15.148:0/4110827583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc415b390 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc27fc640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc4161b80 0x7facc40b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.924+0000 7facc37fe640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 0x7facc4152240 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b9c0 tx=0x7facbc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.925+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc019070 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.925+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00cba0 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.925+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40c8a50 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.925+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc014440 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.925+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc40c9010 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.926+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc4073990 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.926+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facbc01c070 con 0x7facc4151db0 2024-04-25T10:06:53.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.926+0000 7facab7fe640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7facb006b4d0 0x7facb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.927+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facbc08a1e0 con 0x7facc4151db0 2024-04-25T10:06:53.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.927+0000 7facc2ffd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7facb006b4d0 0x7facb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.927+0000 7facc2ffd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7facb006b4d0 0x7facb006d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7facb4009890 tx=0x7facb4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:53.931+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facbc058e40 con 0x7facc4151db0 2024-04-25T10:06:54.109 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.104+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facc40d1570 con 0x7facc4151db0 2024-04-25T10:06:54.110 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.105+0000 7facab7fe640 1 -- 172.21.15.148:0/4110827583 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16945 (secure 0 0 0) 0x7facbc058740 con 0x7facc4151db0 2024-04-25T10:06:54.111 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:54.111 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":196,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:52.824561+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7facb006b4d0 msgr2=0x7facb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7facb006b4d0 0x7facb006d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7facb4009890 tx=0x7facb4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 msgr2=0x7facc4152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 0x7facc4152240 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b9c0 tx=0x7facbc00be80 comp rx=0 tx=0).stop 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 shutdown_connections 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7facb006b4d0 0x7facb006d990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7facc4151db0 0x7facc4152240 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7facc4161b80 0x7facc40b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 --2- 172.21.15.148:0/4110827583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7facc415da70 0x7facc4159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 >> 172.21.15.148:0/4110827583 conn(0x7facc4061a20 msgr2=0x7facc40b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 shutdown_connections 2024-04-25T10:06:54.113 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.107+0000 7facc94bd640 1 -- 172.21.15.148:0/4110827583 wait complete. 2024-04-25T10:06:54.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.218+0000 7f59aad10640 1 Processor -- start 2024-04-25T10:06:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.218+0000 7f59aad10640 1 -- start start 2024-04-25T10:06:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59aad10640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59a415bb50 0x7f59a40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59aad10640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40c70d0 0x7f59a40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59aad10640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a40c75e0 con 0x7f59a40cfb40 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59aad10640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a40c8680 con 0x7f59a415bb50 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59aad10640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a40c78c0 con 0x7f59a40c70d0 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59a8aac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59a415bb50 0x7f59a40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59a92ad640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f59a40cfb40 0x7f59a4070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51784/0 (socket says 172.21.15.148:51784) 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59a8aac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f59a415bb50 0x7f59a40c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33198/0 (socket says 172.21.15.148:33198) 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.219+0000 7f59a92ad640 1 -- 172.21.15.148:0/1699602217 learned_addr learned my addr 172.21.15.148:0/1699602217 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a8aac640 1 -- 172.21.15.148:0/1699602217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40c70d0 msgr2=0x7f59a40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a8aac640 1 --2- 172.21.15.148:0/1699602217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40c70d0 0x7f59a40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a8aac640 1 -- 172.21.15.148:0/1699602217 >> v1:172.21.15.148:6789/0 conn(0x7f59a40cfb40 legacy=0x7f59a4070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a8aac640 1 -- 172.21.15.148:0/1699602217 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a40c7ba0 con 0x7f59a415bb50 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a8aac640 1 --2- 172.21.15.148:0/1699602217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a415bb50 0x7f59a40c6b90 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f599c009d90 tx=0x7f599c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fc7aee83e65b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a37fe640 1 -- 172.21.15.148:0/1699602217 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007740 con 0x7f59a415bb50 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a37fe640 1 -- 172.21.15.148:0/1699602217 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c0078e0 con 0x7f59a415bb50 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.220+0000 7f59a37fe640 1 -- 172.21.15.148:0/1699602217 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c007cb0 con 0x7f59a415bb50 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 -- 172.21.15.148:0/1699602217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a415bb50 msgr2=0x7f59a40c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 --2- 172.21.15.148:0/1699602217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a415bb50 0x7f59a40c6b90 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f599c009d90 tx=0x7f599c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 -- 172.21.15.148:0/1699602217 shutdown_connections 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 --2- 172.21.15.148:0/1699602217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40c70d0 0x7f59a40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 --2- 172.21.15.148:0/1699602217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a415bb50 0x7f59a40c6b90 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 -- 172.21.15.148:0/1699602217 >> 172.21.15.148:0/1699602217 conn(0x7f59a406f500 msgr2=0x7f59a406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 -- 172.21.15.148:0/1699602217 shutdown_connections 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.221+0000 7f59aad10640 1 -- 172.21.15.148:0/1699602217 wait complete. 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.222+0000 7f59aad10640 1 Processor -- start 2024-04-25T10:06:54.229 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.222+0000 7f59aad10640 1 -- start start 2024-04-25T10:06:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59aad10640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 0x7f59a41533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59aad10640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40cfb40 0x7f59a40b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59aad10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59a415bb50 0x7f59a40b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59aad10640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a415f6d0 con 0x7f59a40c70d0 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59aad10640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a40c8160 con 0x7f59a415bb50 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59aad10640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a415f850 con 0x7f59a40cfb40 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 0x7f59a41533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 0x7f59a41533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33212/0 (socket says 172.21.15.148:33212) 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 -- 172.21.15.148:0/1005063668 learned_addr learned my addr 172.21.15.148:0/1005063668 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a92ad640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59a415bb50 0x7f59a40b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a3fff640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40cfb40 0x7f59a40b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 -- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40cfb40 msgr2=0x7f59a40b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40cfb40 0x7f59a40b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 -- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59a415bb50 msgr2=0x7f59a40b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59a415bb50 0x7f59a40b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.223+0000 7f59a8aac640 1 -- 172.21.15.148:0/1005063668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a40c92a0 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.224+0000 7f59a8aac640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 0x7f59a41533e0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f599c009eb0 tx=0x7f599c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.224+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c040070 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.224+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c031bc0 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.224+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a40c9580 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.224+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c045460 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.224+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a40c9b40 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.225+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a4073a00 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.225+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f599c039040 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.226+0000 7f59a17fa640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.226+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f599c0abe70 con 0x7f59a40c70d0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.226+0000 7f59a3fff640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.226+0000 7f59a3fff640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f598406b4d0 0x7f598406d990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f59a40b5410 tx=0x7f5994008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.230+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f599c07ab60 con 0x7f59a40c70d0 2024-04-25T10:06:54.444 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.438+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f59a40cc040 con 0x7f59a40c70d0 2024-04-25T10:06:54.814 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.809+0000 7f59a17fa640 1 -- 172.21.15.148:0/1005063668 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v112) v1 ==== 118+0+0 (secure 0 0 0) 0x7f599c07a460 con 0x7f59a40c70d0 2024-04-25T10:06:54.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f598406b4d0 msgr2=0x7f598406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f598406b4d0 0x7f598406d990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f59a40b5410 tx=0x7f5994008040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 msgr2=0x7f59a41533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 0x7f59a41533e0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f599c009eb0 tx=0x7f599c004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 shutdown_connections 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59a415bb50 0x7f59a40b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59a40cfb40 0x7f59a40b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 --2- 172.21.15.148:0/1005063668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59a40c70d0 0x7f59a41533e0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 >> 172.21.15.148:0/1005063668 conn(0x7f59a406f500 msgr2=0x7f59a40b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 shutdown_connections 2024-04-25T10:06:54.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.811+0000 7f59aad10640 1 -- 172.21.15.148:0/1005063668 wait complete. 2024-04-25T10:06:54.829 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:54.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:54.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.925+0000 7fc2df913640 1 Processor -- start 2024-04-25T10:06:54.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.925+0000 7fc2df913640 1 -- start start 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2df913640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2d8158230 0x7fc2d80d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2df913640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc2d80b4e60 0x7fc2d80b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2df913640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d80d2020 con 0x7fc2d80d1b80 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2df913640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d8158610 con 0x7fc2d80b4e60 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2df913640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d80b57c0 con 0x7fc2d8158230 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc2d80b4e60 0x7fc2d80b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc2d80b4e60 0x7fc2d80b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33228/0 (socket says 172.21.15.148:33228) 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 -- 172.21.15.148:0/3322244827 learned_addr learned my addr 172.21.15.148:0/3322244827 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 -- 172.21.15.148:0/3322244827 >> v2:172.21.15.188:3300/0 conn(0x7fc2d8158230 msgr2=0x7fc2d80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 --2- 172.21.15.148:0/3322244827 >> v2:172.21.15.188:3300/0 conn(0x7fc2d8158230 0x7fc2d80d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 -- 172.21.15.148:0/3322244827 >> v1:172.21.15.188:6789/0 conn(0x7fc2d80d1b80 legacy=0x7fc2d806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.926+0000 7fc2ddeb0640 1 -- 172.21.15.148:0/3322244827 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80d2300 con 0x7fc2d80b4e60 2024-04-25T10:06:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.927+0000 7fc2ddeb0640 1 --2- 172.21.15.148:0/3322244827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80b4e60 0x7fc2d80b5280 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8070460 tx=0x7fc2d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3427265c1bf2e363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.927+0000 7fc2cffff640 1 -- 172.21.15.148:0/3322244827 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0036070 con 0x7fc2d80b4e60 2024-04-25T10:06:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.927+0000 7fc2cffff640 1 -- 172.21.15.148:0/3322244827 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0031600 con 0x7fc2d80b4e60 2024-04-25T10:06:54.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.927+0000 7fc2cffff640 1 -- 172.21.15.148:0/3322244827 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00319d0 con 0x7fc2d80b4e60 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.927+0000 7fc2df913640 1 -- 172.21.15.148:0/3322244827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80b4e60 msgr2=0x7fc2d80b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.927+0000 7fc2df913640 1 --2- 172.21.15.148:0/3322244827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80b4e60 0x7fc2d80b5280 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8070460 tx=0x7fc2d002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.928+0000 7fc2df913640 1 -- 172.21.15.148:0/3322244827 shutdown_connections 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.928+0000 7fc2df913640 1 --2- 172.21.15.148:0/3322244827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80b4e60 0x7fc2d80b5280 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.928+0000 7fc2df913640 1 --2- 172.21.15.148:0/3322244827 >> v2:172.21.15.188:3300/0 conn(0x7fc2d8158230 0x7fc2d80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.928+0000 7fc2df913640 1 -- 172.21.15.148:0/3322244827 >> 172.21.15.148:0/3322244827 conn(0x7fc2d8061600 msgr2=0x7fc2d80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.928+0000 7fc2df913640 1 -- 172.21.15.148:0/3322244827 shutdown_connections 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.928+0000 7fc2df913640 1 -- 172.21.15.148:0/3322244827 wait complete. 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.929+0000 7fc2df913640 1 Processor -- start 2024-04-25T10:06:54.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.929+0000 7fc2df913640 1 -- start start 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.929+0000 7fc2df913640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d80b4e60 0x7fc2d815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 0x7fc2d8159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2d8158230 0x7fc2d815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80d25e0 con 0x7fc2d80d1b80 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d806bea0 con 0x7fc2d80b4e60 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d806ca40 con 0x7fc2d8158230 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 0x7fc2d8159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 0x7fc2d8159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33230/0 (socket says 172.21.15.148:33230) 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 -- 172.21.15.148:0/1346214918 learned_addr learned my addr 172.21.15.148:0/1346214918 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 -- 172.21.15.148:0/1346214918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2d8158230 msgr2=0x7fc2d815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2d8158230 0x7fc2d815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 -- 172.21.15.148:0/1346214918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d80b4e60 msgr2=0x7fc2d815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d80b4e60 0x7fc2d815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 -- 172.21.15.148:0/1346214918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d815b5f0 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2dceae640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 0x7fc2d8159280 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800c4c0 tx=0x7fc2c800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8017070 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c8005480 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d815b310 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8003880 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.930+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d8154200 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.931+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d806de50 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.932+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2c801a070 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.932+0000 7fc2cdffb640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.932+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2c80894b0 con 0x7fc2d80d1b80 2024-04-25T10:06:54.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.932+0000 7fc2dd6af640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.940 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.933+0000 7fc2dd6af640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc2d814b150 tx=0x7fc2d4006bf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:54.937+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2c8057be0 con 0x7fc2d80d1b80 2024-04-25T10:06:55.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.143+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc2d8073a00 con 0x7fc2d80d1b80 2024-04-25T10:06:55.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.816+0000 7fc2cdffb640 1 -- 172.21.15.148:0/1346214918 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v113) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2c805bc80 con 0x7fc2d80d1b80 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2bc06b4d0 msgr2=0x7fc2bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc2d814b150 tx=0x7fc2d4006bf0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 msgr2=0x7fc2d8159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 0x7fc2d8159280 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800c4c0 tx=0x7fc2c800c980 comp rx=0 tx=0).stop 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 shutdown_connections 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc2bc06b4d0 0x7fc2bc06d990 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2d8158230 0x7fc2d815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2d80d1b80 0x7fc2d8159280 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 --2- 172.21.15.148:0/1346214918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d80b4e60 0x7fc2d815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 >> 172.21.15.148:0/1346214918 conn(0x7fc2d8061600 msgr2=0x7fc2d80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 shutdown_connections 2024-04-25T10:06:55.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.819+0000 7fc2df913640 1 -- 172.21.15.148:0/1346214918 wait complete. 2024-04-25T10:06:55.838 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:55.838 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:55.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:55.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:55.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 Processor -- start 2024-04-25T10:06:55.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 -- start start 2024-04-25T10:06:55.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3815dc70 0x7f8c380d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c3809a450 0x7f8c380cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c3806c700 0x7f8c3806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3809a8b0 con 0x7f8c3809a450 2024-04-25T10:06:55.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c380d0c90 con 0x7f8c3806c700 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.937+0000 7f8c3dd63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3806c220 con 0x7f8c3815dc70 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c3809a450 0x7f8c380cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f8c3809a450 0x7f8c380cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33234/0 (socket says 172.21.15.148:33234) 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2217275687 learned_addr learned my addr 172.21.15.148:0/2217275687 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c377fe640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3815dc70 0x7f8c380d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c37fff640 1 --2- 172.21.15.148:0/2217275687 >> v2:172.21.15.188:3300/0 conn(0x7f8c3806c700 0x7f8c3806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2217275687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3815dc70 msgr2=0x7f8c380d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3815dc70 0x7f8c380d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2217275687 >> v2:172.21.15.188:3300/0 conn(0x7f8c3806c700 msgr2=0x7f8c3806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 --2- 172.21.15.148:0/2217275687 >> v2:172.21.15.188:3300/0 conn(0x7f8c3806c700 0x7f8c3806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2217275687 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380d0080 con 0x7f8c3809a450 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.938+0000 7f8c377fe640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3815dc70 0x7f8c380d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c36ffd640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c380cf9c0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f8c38072d90 tx=0x7f8c2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276748d83e224227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c367fc640 1 -- 172.21.15.148:0/2217275687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c037070 con 0x7f8c3809a450 2024-04-25T10:06:55.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c367fc640 1 -- 172.21.15.148:0/2217275687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c032440 con 0x7f8c3809a450 2024-04-25T10:06:55.946 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c367fc640 1 -- 172.21.15.148:0/2217275687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c032810 con 0x7f8c3809a450 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2217275687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 msgr2=0x7f8c380cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c380cf9c0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f8c38072d90 tx=0x7f8c2c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2217275687 shutdown_connections 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2217275687 >> v2:172.21.15.188:3300/0 conn(0x7f8c3806c700 0x7f8c3806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c380cf9c0 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2217275687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3815dc70 0x7f8c380d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.939+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2217275687 >> 172.21.15.148:0/2217275687 conn(0x7f8c380b3270 msgr2=0x7f8c380b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.940+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2217275687 shutdown_connections 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.940+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2217275687 wait complete. 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.941+0000 7f8c3dd63640 1 Processor -- start 2024-04-25T10:06:55.947 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.941+0000 7f8c3dd63640 1 -- start start 2024-04-25T10:06:55.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c3dd63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3806c700 0x7f8c3815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.949 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c3dd63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c3809b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c3dd63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3815dc70 0x7f8c380c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c3dd63640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380d0920 con 0x7f8c3809a450 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c3dd63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380d0640 con 0x7f8c3815dc70 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c3dd63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3815e050 con 0x7f8c3806c700 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c3809b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c3809b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33248/0 (socket says 172.21.15.148:33248) 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2871651027 learned_addr learned my addr 172.21.15.148:0/2871651027 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c37fff640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3815dc70 0x7f8c380c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c377fe640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3806c700 0x7f8c3815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3806c700 msgr2=0x7f8c3815c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3806c700 0x7f8c3815c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3815dc70 msgr2=0x7f8c380c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3815dc70 0x7f8c380c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.950 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.942+0000 7f8c36ffd640 1 -- 172.21.15.148:0/2871651027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380c6c20 con 0x7f8c3809a450 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.943+0000 7f8c36ffd640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c3809b5a0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097c0 tx=0x7f8c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.943+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c041020 con 0x7f8c3809a450 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.943+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c0306c0 con 0x7f8c3809a450 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.943+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c380cd430 con 0x7f8c3809a450 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.943+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c037040 con 0x7f8c3809a450 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.943+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c380cdcd0 con 0x7f8c3809a450 2024-04-25T10:06:55.951 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.944+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c38073990 con 0x7f8c3809a450 2024-04-25T10:06:55.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.944+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c2c044020 con 0x7f8c3809a450 2024-04-25T10:06:55.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.945+0000 7f8c1ffff640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c2406b4d0 0x7f8c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.945+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c2c0aab20 con 0x7f8c3809a450 2024-04-25T10:06:55.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.945+0000 7f8c377fe640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c2406b4d0 0x7f8c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.952 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.945+0000 7f8c377fe640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c2406b4d0 0x7f8c2406d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8c28006fd0 tx=0x7f8c28008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.955 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:55.949+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c2c04b030 con 0x7f8c3809a450 2024-04-25T10:06:56.163 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.158+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c380cdea0 con 0x7f8c3809a450 2024-04-25T10:06:56.164 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.158+0000 7f8c1ffff640 1 -- 172.21.15.148:0/2871651027 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f8c2c079090 con 0x7f8c3809a450 2024-04-25T10:06:56.164 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:56.164 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:06:55.145145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11634":{"gid":11634,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1255014956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1255014956},{"type":"v1","addr":"172.21.15.188:6835","nonce":1255014956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:06:56.164 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 113 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c2406b4d0 msgr2=0x7f8c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c2406b4d0 0x7f8c2406d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8c28006fd0 tx=0x7f8c28008040 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 msgr2=0x7f8c3809b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c3809b5a0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097c0 tx=0x7f8c2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 shutdown_connections 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f8c2406b4d0 0x7f8c2406d990 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c3815dc70 0x7f8c380c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c3809a450 0x7f8c3809b5a0 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 --2- 172.21.15.148:0/2871651027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c3806c700 0x7f8c3815c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 >> 172.21.15.148:0/2871651027 conn(0x7f8c380b3270 msgr2=0x7f8c380cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 shutdown_connections 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.160+0000 7f8c3dd63640 1 -- 172.21.15.148:0/2871651027 wait complete. 2024-04-25T10:06:56.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:56.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.270+0000 7f2878a2c640 1 Processor -- start 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.270+0000 7f2878a2c640 1 -- start start 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2878a2c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28740720f0 0x7f28740b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2878a2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f287406e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2878a2c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28740ace70 con 0x7f28740b3410 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2878a2c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287406df30 con 0x7f28740720f0 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2878a2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28740726d0 con 0x7f287406e3c0 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2871d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f28740720f0 0x7f28740b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f287259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f28740b3410 0x7f28740b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51798/0 (socket says 172.21.15.148:51798) 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f287259c640 1 -- 172.21.15.148:0/1897735095 learned_addr learned my addr 172.21.15.148:0/1897735095 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2872d9d640 1 --2- 172.21.15.148:0/1897735095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f287406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2871d9b640 1 -- 172.21.15.148:0/1897735095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 msgr2=0x7f287406e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2871d9b640 1 --2- 172.21.15.148:0/1897735095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f287406e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2871d9b640 1 -- 172.21.15.148:0/1897735095 >> v1:172.21.15.148:6789/0 conn(0x7f28740b3410 legacy=0x7f28740b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f2871d9b640 1 -- 172.21.15.148:0/1897735095 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287406d8e0 con 0x7f28740720f0 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.271+0000 7f287159a640 1 -- 172.21.15.148:0/1897735095 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385275292 0 0) 0x7f2864001110 con 0x7f28740b3410 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.272+0000 7f2871d9b640 1 --2- 172.21.15.148:0/1897735095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740720f0 0x7f28740b3f00 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f28680097a0 tx=0x7f286802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57e9e9d41c73963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.272+0000 7f287159a640 1 -- 172.21.15.148:0/1897735095 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868037070 con 0x7f28740720f0 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.272+0000 7f287159a640 1 -- 172.21.15.148:0/1897735095 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286802ae50 con 0x7f28740720f0 2024-04-25T10:06:56.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.272+0000 7f287159a640 1 -- 172.21.15.148:0/1897735095 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868032660 con 0x7f28740720f0 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 -- 172.21.15.148:0/1897735095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740720f0 msgr2=0x7f28740b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 --2- 172.21.15.148:0/1897735095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740720f0 0x7f28740b3f00 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f28680097a0 tx=0x7f286802a100 comp rx=0 tx=0).stop 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 -- 172.21.15.148:0/1897735095 shutdown_connections 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 --2- 172.21.15.148:0/1897735095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f287406e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 --2- 172.21.15.148:0/1897735095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740720f0 0x7f28740b3f00 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 -- 172.21.15.148:0/1897735095 >> 172.21.15.148:0/1897735095 conn(0x7f28740cfae0 msgr2=0x7f28740cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 -- 172.21.15.148:0/1897735095 shutdown_connections 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.273+0000 7f2878a2c640 1 -- 172.21.15.148:0/1897735095 wait complete. 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.274+0000 7f2878a2c640 1 Processor -- start 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.274+0000 7f2878a2c640 1 -- start start 2024-04-25T10:06:56.283 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f28740c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28740720f0 0x7f2874138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 0x7f28740c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2874147230 con 0x7f28740b3410 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2874147890 con 0x7f28740720f0 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28741473b0 con 0x7f287406e3c0 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 0x7f28740c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 0x7f28740c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33262/0 (socket says 172.21.15.148:33262) 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 -- 172.21.15.148:0/306428024 learned_addr learned my addr 172.21.15.148:0/306428024 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2871d9b640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28740720f0 0x7f2874138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f287259c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f28740c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 -- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 msgr2=0x7f28740c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f28740c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 -- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28740720f0 msgr2=0x7f2874138260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28740720f0 0x7f2874138260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.284 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 -- 172.21.15.148:0/306428024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740c5df0 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2871d9b640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28740720f0 0x7f2874138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2872d9d640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 0x7f28740c6ac0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f286c00efc0 tx=0x7f286c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c018020 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c014480 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28740c60d0 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.275+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c014870 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.276+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28741f4110 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.276+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28740b5b30 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.277+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f286c005d10 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.277+0000 7f2862ffd640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f285406b4d0 0x7f285406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.277+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f286c08adf0 con 0x7f28740b3410 2024-04-25T10:06:56.285 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.277+0000 7f287259c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f285406b4d0 0x7f285406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.286 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.278+0000 7f287259c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f285406b4d0 0x7f285406d990 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2864000f60 tx=0x7f2864007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.287 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.282+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28740b5b30 con 0x7f28740b3410 2024-04-25T10:06:56.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.452+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2874146de0 con 0x7f28740b3410 2024-04-25T10:06:56.459 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.453+0000 7f2862ffd640 1 -- 172.21.15.148:0/306428024 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f2874146de0 con 0x7f28740b3410 2024-04-25T10:06:56.459 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:56.459 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":197,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:54.484614+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f285406b4d0 msgr2=0x7f285406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f285406b4d0 0x7f285406d990 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f2864000f60 tx=0x7f2864007c00 comp rx=0 tx=0).stop 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 msgr2=0x7f28740c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 0x7f28740c6ac0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f286c00efc0 tx=0x7f286c00c490 comp rx=0 tx=0).stop 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 shutdown_connections 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f285406b4d0 0x7f285406d990 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28740b3410 0x7f28740c6ac0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28740720f0 0x7f2874138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 --2- 172.21.15.148:0/306428024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f287406e3c0 0x7f28740c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 >> 172.21.15.148:0/306428024 conn(0x7f28740cfae0 msgr2=0x7f28740718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 shutdown_connections 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.455+0000 7f2878a2c640 1 -- 172.21.15.148:0/306428024 wait complete. 2024-04-25T10:06:56.472 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:56.473 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:56.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.564+0000 7fa87b1b7640 1 Processor -- start 2024-04-25T10:06:56.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.564+0000 7fa87b1b7640 1 -- start start 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.564+0000 7fa87b1b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 0x7fa8740725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.564+0000 7fa87b1b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87415be50 con 0x7fa87406f360 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.564+0000 7fa87b1b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8740717f0 con 0x7fa87415ba70 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.564+0000 7fa87b1b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa874071ad0 con 0x7fa8740aefa0 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa873fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa87406f360 0x7fa87406d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51806/0 (socket says 172.21.15.148:51806) 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa873fff640 1 -- 172.21.15.148:0/766778904 learned_addr learned my addr 172.21.15.148:0/766778904 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa879754640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 0x7fa8740725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117435142 0 0) 0x7fa85c001110 con 0x7fa87406f360 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa860003520 con 0x7fa87406f360 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530032700 0 0) 0x7fa860003520 con 0x7fa87406f360 2024-04-25T10:06:56.573 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 msgr2=0x7fa8740725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 0x7fa8740725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 >> v1:172.21.15.188:6789/0 conn(0x7fa87415ba70 legacy=0x7fa8740cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874071db0 con 0x7fa87406f360 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa85c001840 con 0x7fa87406f360 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 >> v1:172.21.15.148:6789/0 conn(0x7fa87406f360 legacy=0x7fa87406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8600039e0 0x7fa860003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa860004800 0x7fa860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 0x7fa8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874071ad0 con 0x7fa8600039e0 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86000ba70 con 0x7fa860004800 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86000bc70 con 0x7fa8740aefa0 2024-04-25T10:06:56.574 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa878f53640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8600039e0 0x7fa860003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa873fff640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa860004800 0x7fa860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.565+0000 7fa879754640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 0x7fa8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa878f53640 1 -- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 msgr2=0x7fa8600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa878f53640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8740aefa0 0x7fa8600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa878f53640 1 -- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa860004800 msgr2=0x7fa860004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa878f53640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa860004800 0x7fa860004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa878f53640 1 -- 172.21.15.148:0/766778904 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860003520 con 0x7fa8600039e0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa878f53640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8600039e0 0x7fa860003e00 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fa86c001e20 tx=0x7fa860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed3d3638e69113f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.566+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c001a70 con 0x7fa8600039e0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c02ae10 con 0x7fa8600039e0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa8737fe640 1 -- 172.21.15.148:0/766778904 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c003bb0 con 0x7fa8600039e0 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa87b1b7640 1 -- 172.21.15.148:0/766778904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8600039e0 msgr2=0x7fa860003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/766778904 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8600039e0 0x7fa860003e00 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fa86c001e20 tx=0x7fa860005830 comp rx=0 tx=0).stop 2024-04-25T10:06:56.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa879754640 1 -- 172.21.15.148:0/766778904 reap_dead start 2024-04-25T10:06:56.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa87b1b7640 1 -- 172.21.15.148:0/766778904 shutdown_connections 2024-04-25T10:06:56.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa87b1b7640 1 -- 172.21.15.148:0/766778904 >> 172.21.15.148:0/766778904 conn(0x7fa8740cfae0 msgr2=0x7fa8740cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa87b1b7640 1 -- 172.21.15.148:0/766778904 shutdown_connections 2024-04-25T10:06:56.576 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.567+0000 7fa87b1b7640 1 -- 172.21.15.148:0/766778904 wait complete. 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.568+0000 7fa87b1b7640 1 Processor -- start 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.568+0000 7fa87b1b7640 1 -- start start 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa87b1b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8740aac80 0x7fa8740acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa87b1b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 0x7fa8740ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa87b1b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa874148160 0x7fa8741485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa87b1b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740cd420 con 0x7fa8740ab060 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa87b1b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740cd7e0 con 0x7fa8740aac80 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa87b1b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740cd5a0 con 0x7fa874148160 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 0x7fa8740ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 0x7fa8740ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33276/0 (socket says 172.21.15.148:33276) 2024-04-25T10:06:56.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 -- 172.21.15.148:0/2166438531 learned_addr learned my addr 172.21.15.148:0/2166438531 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa878f53640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8740aac80 0x7fa8740acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa879754640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa874148160 0x7fa8741485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 -- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa874148160 msgr2=0x7fa8741485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa874148160 0x7fa8741485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 -- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8740aac80 msgr2=0x7fa8740acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.569+0000 7fa873fff640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8740aac80 0x7fa8740acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa873fff640 1 -- 172.21.15.148:0/2166438531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8740ad630 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa873fff640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 0x7fa8740ad020 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0072f0 tx=0x7fa85c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c01a070 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c00f860 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8740ad910 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c00fc30 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.570+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8740aded0 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.571+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa874073990 con 0x7fa8740ab060 2024-04-25T10:06:56.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.571+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa85c01d070 con 0x7fa8740ab060 2024-04-25T10:06:56.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.572+0000 7fa8717fa640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85406b4d0 0x7fa85406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.572+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa85c089d20 con 0x7fa8740ab060 2024-04-25T10:06:56.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.572+0000 7fa878f53640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85406b4d0 0x7fa85406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.572+0000 7fa878f53640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85406b4d0 0x7fa85406d990 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa86c032000 tx=0x7fa86c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.576+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa87415be50 con 0x7fa8740ab060 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.743+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8740aeb70 con 0x7fa85406b4d0 2024-04-25T10:06:56.754 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.749+0000 7fa8717fa640 1 -- 172.21.15.148:0/2166438531 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252452 (secure 0 0 0) 0x7fa8740aeb70 con 0x7fa85406b4d0 2024-04-25T10:06:56.758 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:56.759 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85406b4d0 msgr2=0x7fa85406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85406b4d0 0x7fa85406d990 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa86c032000 tx=0x7fa86c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 msgr2=0x7fa8740ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 0x7fa8740ad020 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0072f0 tx=0x7fa85c0077b0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 shutdown_connections 2024-04-25T10:06:56.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa85406b4d0 0x7fa85406d990 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa874148160 0x7fa8741485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8740ab060 0x7fa8740ad020 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 --2- 172.21.15.148:0/2166438531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8740aac80 0x7fa8740acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 >> 172.21.15.148:0/2166438531 conn(0x7fa8740cfae0 msgr2=0x7fa8740cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 shutdown_connections 2024-04-25T10:06:56.762 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.752+0000 7fa87b1b7640 1 -- 172.21.15.148:0/2166438531 wait complete. 2024-04-25T10:06:56.769 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":573,"stamp":"2024-04-25T10:06:55.176475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465732,"kb_used_data":20608,"kb_used_omap":12,"kb_used_meta":444979,"kb_avail":754508988,"statfs":{"total":773094113280,"available":772617203712,"internally_reserved":0,"allocated":21102592,"data_stored":10497540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":455659056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000803"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492999+0000","last_change":"2024-04-25T10:06:53.499533+0000","last_active":"2024-04-25T10:06:54.492999+0000","last_peered":"2024-04-25T10:06:54.492999+0000","last_clean":"2024-04-25T10:06:54.492999+0000","last_became_active":"2024-04-25T10:06:53.499259+0000","last_became_peered":"2024-04-25T10:06:53.499259+0000","last_unstale":"2024-04-25T10:06:54.492999+0000","last_undegraded":"2024-04-25T10:06:54.492999+0000","last_fullsized":"2024-04-25T10:06:54.492999+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.228046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506330+0000","last_change":"2024-04-25T10:06:53.499287+0000","last_active":"2024-04-25T10:06:54.506330+0000","last_peered":"2024-04-25T10:06:54.506330+0000","last_clean":"2024-04-25T10:06:54.506330+0000","last_became_active":"2024-04-25T10:06:53.498985+0000","last_became_peered":"2024-04-25T10:06:53.498985+0000","last_unstale":"2024-04-25T10:06:54.506330+0000","last_undegraded":"2024-04-25T10:06:54.506330+0000","last_fullsized":"2024-04-25T10:06:54.506330+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.280124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.498731+0000","last_change":"2024-04-25T10:06:53.495495+0000","last_active":"2024-04-25T10:06:53.498731+0000","last_peered":"2024-04-25T10:06:53.498731+0000","last_clean":"2024-04-25T10:06:53.498731+0000","last_became_active":"2024-04-25T10:06:53.495207+0000","last_became_peered":"2024-04-25T10:06:53.495207+0000","last_unstale":"2024-04-25T10:06:53.498731+0000","last_undegraded":"2024-04-25T10:06:53.498731+0000","last_fullsized":"2024-04-25T10:06:53.498731+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:24.372617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491485+0000","last_change":"2024-04-25T10:06:53.496243+0000","last_active":"2024-04-25T10:06:54.491485+0000","last_peered":"2024-04-25T10:06:54.491485+0000","last_clean":"2024-04-25T10:06:54.491485+0000","last_became_active":"2024-04-25T10:06:53.495981+0000","last_became_peered":"2024-04-25T10:06:53.495981+0000","last_unstale":"2024-04-25T10:06:54.491485+0000","last_undegraded":"2024-04-25T10:06:54.491485+0000","last_fullsized":"2024-04-25T10:06:54.491485+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492874+0000","last_change":"2024-04-25T10:06:53.506395+0000","last_active":"2024-04-25T10:06:54.492874+0000","last_peered":"2024-04-25T10:06:54.492874+0000","last_clean":"2024-04-25T10:06:54.492874+0000","last_became_active":"2024-04-25T10:06:53.506095+0000","last_became_peered":"2024-04-25T10:06:53.506095+0000","last_unstale":"2024-04-25T10:06:54.492874+0000","last_undegraded":"2024-04-25T10:06:54.492874+0000","last_fullsized":"2024-04-25T10:06:54.492874+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:46.781173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.504177+0000","last_change":"2024-04-25T10:06:53.501579+0000","last_active":"2024-04-25T10:06:53.504177+0000","last_peered":"2024-04-25T10:06:53.504177+0000","last_clean":"2024-04-25T10:06:53.504177+0000","last_became_active":"2024-04-25T10:06:53.501313+0000","last_became_peered":"2024-04-25T10:06:53.501313+0000","last_unstale":"2024-04-25T10:06:53.504177+0000","last_undegraded":"2024-04-25T10:06:53.504177+0000","last_fullsized":"2024-04-25T10:06:53.504177+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:03.458446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507584+0000","last_change":"2024-04-25T10:06:53.503403+0000","last_active":"2024-04-25T10:06:53.507584+0000","last_peered":"2024-04-25T10:06:53.507584+0000","last_clean":"2024-04-25T10:06:53.507584+0000","last_became_active":"2024-04-25T10:06:53.503133+0000","last_became_peered":"2024-04-25T10:06:53.503133+0000","last_unstale":"2024-04-25T10:06:53.507584+0000","last_undegraded":"2024-04-25T10:06:53.507584+0000","last_fullsized":"2024-04-25T10:06:53.507584+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:39.235821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.504044+0000","last_change":"2024-04-25T10:06:53.502226+0000","last_active":"2024-04-25T10:06:53.504044+0000","last_peered":"2024-04-25T10:06:53.504044+0000","last_clean":"2024-04-25T10:06:53.504044+0000","last_became_active":"2024-04-25T10:06:53.501942+0000","last_became_peered":"2024-04-25T10:06:53.501942+0000","last_unstale":"2024-04-25T10:06:53.504044+0000","last_undegraded":"2024-04-25T10:06:53.504044+0000","last_fullsized":"2024-04-25T10:06:53.504044+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:23.475397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829268+0000","last_change":"2024-04-25T10:06:53.506051+0000","last_active":"2024-04-25T10:06:54.829268+0000","last_peered":"2024-04-25T10:06:54.829268+0000","last_clean":"2024-04-25T10:06:54.829268+0000","last_became_active":"2024-04-25T10:06:53.505573+0000","last_became_peered":"2024-04-25T10:06:53.505573+0000","last_unstale":"2024-04-25T10:06:54.829268+0000","last_undegraded":"2024-04-25T10:06:54.829268+0000","last_fullsized":"2024-04-25T10:06:54.829268+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:41.699691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.499927+0000","last_change":"2024-04-25T10:06:53.496842+0000","last_active":"2024-04-25T10:06:53.499927+0000","last_peered":"2024-04-25T10:06:53.499927+0000","last_clean":"2024-04-25T10:06:53.499927+0000","last_became_active":"2024-04-25T10:06:53.495825+0000","last_became_peered":"2024-04-25T10:06:53.495825+0000","last_unstale":"2024-04-25T10:06:53.499927+0000","last_undegraded":"2024-04-25T10:06:53.499927+0000","last_fullsized":"2024-04-25T10:06:53.499927+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.280342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491621+0000","last_change":"2024-04-25T10:06:53.496862+0000","last_active":"2024-04-25T10:06:54.491621+0000","last_peered":"2024-04-25T10:06:54.491621+0000","last_clean":"2024-04-25T10:06:54.491621+0000","last_became_active":"2024-04-25T10:06:53.495833+0000","last_became_peered":"2024-04-25T10:06:53.495833+0000","last_unstale":"2024-04-25T10:06:54.491621+0000","last_undegraded":"2024-04-25T10:06:54.491621+0000","last_fullsized":"2024-04-25T10:06:54.491621+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:16.909645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.509860+0000","last_change":"2024-04-25T10:06:53.506846+0000","last_active":"2024-04-25T10:06:53.509860+0000","last_peered":"2024-04-25T10:06:53.509860+0000","last_clean":"2024-04-25T10:06:53.509860+0000","last_became_active":"2024-04-25T10:06:53.506472+0000","last_became_peered":"2024-04-25T10:06:53.506472+0000","last_unstale":"2024-04-25T10:06:53.509860+0000","last_undegraded":"2024-04-25T10:06:53.509860+0000","last_fullsized":"2024-04-25T10:06:53.509860+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.736033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.505793+0000","last_change":"2024-04-25T10:06:53.503669+0000","last_active":"2024-04-25T10:06:53.505793+0000","last_peered":"2024-04-25T10:06:53.505793+0000","last_clean":"2024-04-25T10:06:53.505793+0000","last_became_active":"2024-04-25T10:06:53.503350+0000","last_became_peered":"2024-04-25T10:06:53.503350+0000","last_unstale":"2024-04-25T10:06:53.505793+0000","last_undegraded":"2024-04-25T10:06:53.505793+0000","last_fullsized":"2024-04-25T10:06:53.505793+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:39.313543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508843+0000","last_change":"2024-04-25T10:06:53.503319+0000","last_active":"2024-04-25T10:06:53.508843+0000","last_peered":"2024-04-25T10:06:53.508843+0000","last_clean":"2024-04-25T10:06:53.508843+0000","last_became_active":"2024-04-25T10:06:53.503019+0000","last_became_peered":"2024-04-25T10:06:53.503019+0000","last_unstale":"2024-04-25T10:06:53.508843+0000","last_undegraded":"2024-04-25T10:06:53.508843+0000","last_fullsized":"2024-04-25T10:06:53.508843+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:47.109377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509427+0000","last_change":"2024-04-25T10:06:53.508607+0000","last_active":"2024-04-25T10:06:54.509427+0000","last_peered":"2024-04-25T10:06:54.509427+0000","last_clean":"2024-04-25T10:06:54.509427+0000","last_became_active":"2024-04-25T10:06:53.508323+0000","last_became_peered":"2024-04-25T10:06:53.508323+0000","last_unstale":"2024-04-25T10:06:54.509427+0000","last_undegraded":"2024-04-25T10:06:54.509427+0000","last_fullsized":"2024-04-25T10:06:54.509427+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:23.901917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.499800+0000","last_change":"2024-04-25T10:06:53.498592+0000","last_active":"2024-04-25T10:06:53.499800+0000","last_peered":"2024-04-25T10:06:53.499800+0000","last_clean":"2024-04-25T10:06:53.499800+0000","last_became_active":"2024-04-25T10:06:53.498309+0000","last_became_peered":"2024-04-25T10:06:53.498309+0000","last_unstale":"2024-04-25T10:06:53.499800+0000","last_undegraded":"2024-04-25T10:06:53.499800+0000","last_fullsized":"2024-04-25T10:06:53.499800+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:05.426090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517212+0000","last_change":"2024-04-25T10:06:53.509516+0000","last_active":"2024-04-25T10:06:53.517212+0000","last_peered":"2024-04-25T10:06:53.517212+0000","last_clean":"2024-04-25T10:06:53.517212+0000","last_became_active":"2024-04-25T10:06:53.509076+0000","last_became_peered":"2024-04-25T10:06:53.509076+0000","last_unstale":"2024-04-25T10:06:53.517212+0000","last_undegraded":"2024-04-25T10:06:53.517212+0000","last_fullsized":"2024-04-25T10:06:53.517212+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:31.733339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.491990+0000","last_change":"2024-04-25T10:06:53.490235+0000","last_active":"2024-04-25T10:06:53.491990+0000","last_peered":"2024-04-25T10:06:53.491990+0000","last_clean":"2024-04-25T10:06:53.491990+0000","last_became_active":"2024-04-25T10:06:53.489973+0000","last_became_peered":"2024-04-25T10:06:53.489973+0000","last_unstale":"2024-04-25T10:06:53.491990+0000","last_undegraded":"2024-04-25T10:06:53.491990+0000","last_fullsized":"2024-04-25T10:06:53.491990+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:21.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492958+0000","last_change":"2024-04-25T10:06:53.500588+0000","last_active":"2024-04-25T10:06:54.492958+0000","last_peered":"2024-04-25T10:06:54.492958+0000","last_clean":"2024-04-25T10:06:54.492958+0000","last_became_active":"2024-04-25T10:06:53.499768+0000","last_became_peered":"2024-04-25T10:06:53.499768+0000","last_unstale":"2024-04-25T10:06:54.492958+0000","last_undegraded":"2024-04-25T10:06:54.492958+0000","last_fullsized":"2024-04-25T10:06:54.492958+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:35.342883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.503779+0000","last_change":"2024-04-25T10:06:53.499873+0000","last_active":"2024-04-25T10:06:53.503779+0000","last_peered":"2024-04-25T10:06:53.503779+0000","last_clean":"2024-04-25T10:06:53.503779+0000","last_became_active":"2024-04-25T10:06:53.499612+0000","last_became_peered":"2024-04-25T10:06:53.499612+0000","last_unstale":"2024-04-25T10:06:53.503779+0000","last_undegraded":"2024-04-25T10:06:53.503779+0000","last_fullsized":"2024-04-25T10:06:53.503779+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:01.768702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.493566+0000","last_change":"2024-04-25T10:06:53.492847+0000","last_active":"2024-04-25T10:06:53.493566+0000","last_peered":"2024-04-25T10:06:53.493566+0000","last_clean":"2024-04-25T10:06:53.493566+0000","last_became_active":"2024-04-25T10:06:53.492570+0000","last_became_peered":"2024-04-25T10:06:53.492570+0000","last_unstale":"2024-04-25T10:06:53.493566+0000","last_undegraded":"2024-04-25T10:06:53.493566+0000","last_fullsized":"2024-04-25T10:06:53.493566+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:52.329847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509423+0000","last_change":"2024-04-25T10:06:53.500818+0000","last_active":"2024-04-25T10:06:54.509423+0000","last_peered":"2024-04-25T10:06:54.509423+0000","last_clean":"2024-04-25T10:06:54.509423+0000","last_became_active":"2024-04-25T10:06:53.499454+0000","last_became_peered":"2024-04-25T10:06:53.499454+0000","last_unstale":"2024-04-25T10:06:54.509423+0000","last_undegraded":"2024-04-25T10:06:54.509423+0000","last_fullsized":"2024-04-25T10:06:54.509423+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.930376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828871+0000","last_change":"2024-04-25T10:06:53.505222+0000","last_active":"2024-04-25T10:06:54.828871+0000","last_peered":"2024-04-25T10:06:54.828871+0000","last_clean":"2024-04-25T10:06:54.828871+0000","last_became_active":"2024-04-25T10:06:53.504922+0000","last_became_peered":"2024-04-25T10:06:53.504922+0000","last_unstale":"2024-04-25T10:06:54.828871+0000","last_undegraded":"2024-04-25T10:06:54.828871+0000","last_fullsized":"2024-04-25T10:06:54.828871+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:23.116854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492610+0000","last_change":"2024-04-25T10:06:53.496210+0000","last_active":"2024-04-25T10:06:54.492610+0000","last_peered":"2024-04-25T10:06:54.492610+0000","last_clean":"2024-04-25T10:06:54.492610+0000","last_became_active":"2024-04-25T10:06:53.495882+0000","last_became_peered":"2024-04-25T10:06:53.495882+0000","last_unstale":"2024-04-25T10:06:54.492610+0000","last_undegraded":"2024-04-25T10:06:54.492610+0000","last_fullsized":"2024-04-25T10:06:54.492610+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:25.914385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492990+0000","last_change":"2024-04-25T10:06:53.498931+0000","last_active":"2024-04-25T10:06:54.492990+0000","last_peered":"2024-04-25T10:06:54.492990+0000","last_clean":"2024-04-25T10:06:54.492990+0000","last_became_active":"2024-04-25T10:06:53.498666+0000","last_became_peered":"2024-04-25T10:06:53.498666+0000","last_unstale":"2024-04-25T10:06:54.492990+0000","last_undegraded":"2024-04-25T10:06:54.492990+0000","last_fullsized":"2024-04-25T10:06:54.492990+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:45.505542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507671+0000","last_change":"2024-04-25T10:06:53.504383+0000","last_active":"2024-04-25T10:06:53.507671+0000","last_peered":"2024-04-25T10:06:53.507671+0000","last_clean":"2024-04-25T10:06:53.507671+0000","last_became_active":"2024-04-25T10:06:53.504091+0000","last_became_peered":"2024-04-25T10:06:53.504091+0000","last_unstale":"2024-04-25T10:06:53.507671+0000","last_undegraded":"2024-04-25T10:06:53.507671+0000","last_fullsized":"2024-04-25T10:06:53.507671+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:32.305830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491729+0000","last_change":"2024-04-25T10:06:53.496930+0000","last_active":"2024-04-25T10:06:54.491729+0000","last_peered":"2024-04-25T10:06:54.491729+0000","last_clean":"2024-04-25T10:06:54.491729+0000","last_became_active":"2024-04-25T10:06:53.496681+0000","last_became_peered":"2024-04-25T10:06:53.496681+0000","last_unstale":"2024-04-25T10:06:54.491729+0000","last_undegraded":"2024-04-25T10:06:54.491729+0000","last_fullsized":"2024-04-25T10:06:54.491729+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:22.628297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508896+0000","last_change":"2024-04-25T10:06:53.503396+0000","last_active":"2024-04-25T10:06:53.508896+0000","last_peered":"2024-04-25T10:06:53.508896+0000","last_clean":"2024-04-25T10:06:53.508896+0000","last_became_active":"2024-04-25T10:06:53.503062+0000","last_became_peered":"2024-04-25T10:06:53.503062+0000","last_unstale":"2024-04-25T10:06:53.508896+0000","last_undegraded":"2024-04-25T10:06:53.508896+0000","last_fullsized":"2024-04-25T10:06:53.508896+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:10.544964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506353+0000","last_change":"2024-04-25T10:06:53.497618+0000","last_active":"2024-04-25T10:06:54.506353+0000","last_peered":"2024-04-25T10:06:54.506353+0000","last_clean":"2024-04-25T10:06:54.506353+0000","last_became_active":"2024-04-25T10:06:53.497363+0000","last_became_peered":"2024-04-25T10:06:53.497363+0000","last_unstale":"2024-04-25T10:06:54.506353+0000","last_undegraded":"2024-04-25T10:06:54.506353+0000","last_fullsized":"2024-04-25T10:06:54.506353+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:20.503651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508657+0000","last_change":"2024-04-25T10:06:53.505556+0000","last_active":"2024-04-25T10:06:53.508657+0000","last_peered":"2024-04-25T10:06:53.508657+0000","last_clean":"2024-04-25T10:06:53.508657+0000","last_became_active":"2024-04-25T10:06:53.505289+0000","last_became_peered":"2024-04-25T10:06:53.505289+0000","last_unstale":"2024-04-25T10:06:53.508657+0000","last_undegraded":"2024-04-25T10:06:53.508657+0000","last_fullsized":"2024-04-25T10:06:53.508657+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:55.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828930+0000","last_change":"2024-04-25T10:06:53.503147+0000","last_active":"2024-04-25T10:06:54.828930+0000","last_peered":"2024-04-25T10:06:54.828930+0000","last_clean":"2024-04-25T10:06:54.828930+0000","last_became_active":"2024-04-25T10:06:53.502658+0000","last_became_peered":"2024-04-25T10:06:53.502658+0000","last_unstale":"2024-04-25T10:06:54.828930+0000","last_undegraded":"2024-04-25T10:06:54.828930+0000","last_fullsized":"2024-04-25T10:06:54.828930+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:06.382243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508987+0000","last_change":"2024-04-25T10:06:53.506986+0000","last_active":"2024-04-25T10:06:53.508987+0000","last_peered":"2024-04-25T10:06:53.508987+0000","last_clean":"2024-04-25T10:06:53.508987+0000","last_became_active":"2024-04-25T10:06:53.506676+0000","last_became_peered":"2024-04-25T10:06:53.506676+0000","last_unstale":"2024-04-25T10:06:53.508987+0000","last_undegraded":"2024-04-25T10:06:53.508987+0000","last_fullsized":"2024-04-25T10:06:53.508987+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:04.277774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.497702+0000","last_change":"2024-04-25T10:06:53.496250+0000","last_active":"2024-04-25T10:06:53.497702+0000","last_peered":"2024-04-25T10:06:53.497702+0000","last_clean":"2024-04-25T10:06:53.497702+0000","last_became_active":"2024-04-25T10:06:53.495993+0000","last_became_peered":"2024-04-25T10:06:53.495993+0000","last_unstale":"2024-04-25T10:06:53.497702+0000","last_undegraded":"2024-04-25T10:06:53.497702+0000","last_fullsized":"2024-04-25T10:06:53.497702+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:08.628944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"196'4","reported_seq":18,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829301+0000","last_change":"2024-04-25T10:06:52.495381+0000","last_active":"2024-04-25T10:06:54.829301+0000","last_peered":"2024-04-25T10:06:54.829301+0000","last_clean":"2024-04-25T10:06:54.829301+0000","last_became_active":"2024-04-25T10:06:52.495127+0000","last_became_peered":"2024-04-25T10:06:52.495127+0000","last_unstale":"2024-04-25T10:06:54.829301+0000","last_undegraded":"2024-04-25T10:06:54.829301+0000","last_fullsized":"2024-04-25T10:06:54.829301+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:01.323500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.499613+0000","last_change":"2024-04-25T10:06:53.497909+0000","last_active":"2024-04-25T10:06:53.499613+0000","last_peered":"2024-04-25T10:06:53.499613+0000","last_clean":"2024-04-25T10:06:53.499613+0000","last_became_active":"2024-04-25T10:06:53.497629+0000","last_became_peered":"2024-04-25T10:06:53.497629+0000","last_unstale":"2024-04-25T10:06:53.499613+0000","last_undegraded":"2024-04-25T10:06:53.499613+0000","last_fullsized":"2024-04-25T10:06:53.499613+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.665724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"196'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.519561+0000","last_change":"2024-04-25T10:06:52.497276+0000","last_active":"2024-04-25T10:06:53.519561+0000","last_peered":"2024-04-25T10:06:53.519561+0000","last_clean":"2024-04-25T10:06:53.519561+0000","last_became_active":"2024-04-25T10:06:52.497015+0000","last_became_peered":"2024-04-25T10:06:52.497015+0000","last_unstale":"2024-04-25T10:06:53.519561+0000","last_undegraded":"2024-04-25T10:06:53.519561+0000","last_fullsized":"2024-04-25T10:06:53.519561+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:23.427618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508725+0000","last_change":"2024-04-25T10:06:53.501853+0000","last_active":"2024-04-25T10:06:53.508725+0000","last_peered":"2024-04-25T10:06:53.508725+0000","last_clean":"2024-04-25T10:06:53.508725+0000","last_became_active":"2024-04-25T10:06:53.501391+0000","last_became_peered":"2024-04-25T10:06:53.501391+0000","last_unstale":"2024-04-25T10:06:53.508725+0000","last_undegraded":"2024-04-25T10:06:53.508725+0000","last_fullsized":"2024-04-25T10:06:53.508725+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:22.628297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.520845+0000","last_change":"2024-04-25T10:06:52.491587+0000","last_active":"2024-04-25T10:06:53.520845+0000","last_peered":"2024-04-25T10:06:53.520845+0000","last_clean":"2024-04-25T10:06:53.520845+0000","last_became_active":"2024-04-25T10:06:52.491321+0000","last_became_peered":"2024-04-25T10:06:52.491321+0000","last_unstale":"2024-04-25T10:06:53.520845+0000","last_undegraded":"2024-04-25T10:06:53.520845+0000","last_fullsized":"2024-04-25T10:06:53.520845+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:53.861228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507126+0000","last_change":"2024-04-25T10:06:53.500731+0000","last_active":"2024-04-25T10:06:53.507126+0000","last_peered":"2024-04-25T10:06:53.507126+0000","last_clean":"2024-04-25T10:06:53.507126+0000","last_became_active":"2024-04-25T10:06:53.500462+0000","last_became_peered":"2024-04-25T10:06:53.500462+0000","last_unstale":"2024-04-25T10:06:53.507126+0000","last_undegraded":"2024-04-25T10:06:53.507126+0000","last_fullsized":"2024-04-25T10:06:53.507126+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:20.503651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"196'1","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509466+0000","last_change":"2024-04-25T10:06:52.491625+0000","last_active":"2024-04-25T10:06:54.509466+0000","last_peered":"2024-04-25T10:06:54.509466+0000","last_clean":"2024-04-25T10:06:54.509466+0000","last_became_active":"2024-04-25T10:06:52.491360+0000","last_became_peered":"2024-04-25T10:06:52.491360+0000","last_unstale":"2024-04-25T10:06:54.509466+0000","last_undegraded":"2024-04-25T10:06:54.509466+0000","last_fullsized":"2024-04-25T10:06:54.509466+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:15.607411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508749+0000","last_change":"2024-04-25T10:06:53.501691+0000","last_active":"2024-04-25T10:06:53.508749+0000","last_peered":"2024-04-25T10:06:53.508749+0000","last_clean":"2024-04-25T10:06:53.508749+0000","last_became_active":"2024-04-25T10:06:53.501427+0000","last_became_peered":"2024-04-25T10:06:53.501427+0000","last_unstale":"2024-04-25T10:06:53.508749+0000","last_undegraded":"2024-04-25T10:06:53.508749+0000","last_fullsized":"2024-04-25T10:06:53.508749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:16.909645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"196'1","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492651+0000","last_change":"2024-04-25T10:06:52.499853+0000","last_active":"2024-04-25T10:06:54.492651+0000","last_peered":"2024-04-25T10:06:54.492651+0000","last_clean":"2024-04-25T10:06:54.492651+0000","last_became_active":"2024-04-25T10:06:52.499538+0000","last_became_peered":"2024-04-25T10:06:52.499538+0000","last_unstale":"2024-04-25T10:06:54.492651+0000","last_undegraded":"2024-04-25T10:06:54.492651+0000","last_fullsized":"2024-04-25T10:06:54.492651+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:53.861228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509451+0000","last_change":"2024-04-25T10:06:53.501005+0000","last_active":"2024-04-25T10:06:54.509451+0000","last_peered":"2024-04-25T10:06:54.509451+0000","last_clean":"2024-04-25T10:06:54.509451+0000","last_became_active":"2024-04-25T10:06:53.500146+0000","last_became_peered":"2024-04-25T10:06:53.500146+0000","last_unstale":"2024-04-25T10:06:54.509451+0000","last_undegraded":"2024-04-25T10:06:54.509451+0000","last_fullsized":"2024-04-25T10:06:54.509451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:50.958579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492138+0000","last_change":"2024-04-25T10:06:52.492189+0000","last_active":"2024-04-25T10:06:54.492138+0000","last_peered":"2024-04-25T10:06:54.492138+0000","last_clean":"2024-04-25T10:06:54.492138+0000","last_became_active":"2024-04-25T10:06:52.491551+0000","last_became_peered":"2024-04-25T10:06:52.491551+0000","last_unstale":"2024-04-25T10:06:54.492138+0000","last_undegraded":"2024-04-25T10:06:54.492138+0000","last_fullsized":"2024-04-25T10:06:54.492138+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:09.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829110+0000","last_change":"2024-04-25T10:06:53.499516+0000","last_active":"2024-04-25T10:06:54.829110+0000","last_peered":"2024-04-25T10:06:54.829110+0000","last_clean":"2024-04-25T10:06:54.829110+0000","last_became_active":"2024-04-25T10:06:53.499277+0000","last_became_peered":"2024-04-25T10:06:53.499277+0000","last_unstale":"2024-04-25T10:06:54.829110+0000","last_undegraded":"2024-04-25T10:06:54.829110+0000","last_fullsized":"2024-04-25T10:06:54.829110+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:39.895005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.487705+0000","last_change":"2024-04-25T10:06:52.494250+0000","last_active":"2024-04-25T10:06:53.487705+0000","last_peered":"2024-04-25T10:06:53.487705+0000","last_clean":"2024-04-25T10:06:53.487705+0000","last_became_active":"2024-04-25T10:06:52.493981+0000","last_became_peered":"2024-04-25T10:06:52.493981+0000","last_unstale":"2024-04-25T10:06:53.487705+0000","last_undegraded":"2024-04-25T10:06:53.487705+0000","last_fullsized":"2024-04-25T10:06:53.487705+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:09.565197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509554+0000","last_change":"2024-04-25T10:06:53.495742+0000","last_active":"2024-04-25T10:06:54.509554+0000","last_peered":"2024-04-25T10:06:54.509554+0000","last_clean":"2024-04-25T10:06:54.509554+0000","last_became_active":"2024-04-25T10:06:53.494938+0000","last_became_peered":"2024-04-25T10:06:53.494938+0000","last_unstale":"2024-04-25T10:06:54.509554+0000","last_undegraded":"2024-04-25T10:06:54.509554+0000","last_fullsized":"2024-04-25T10:06:54.509554+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:43.078800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493124+0000","last_change":"2024-04-25T10:06:52.495657+0000","last_active":"2024-04-25T10:06:54.493124+0000","last_peered":"2024-04-25T10:06:54.493124+0000","last_clean":"2024-04-25T10:06:54.493124+0000","last_became_active":"2024-04-25T10:06:52.495378+0000","last_became_peered":"2024-04-25T10:06:52.495378+0000","last_unstale":"2024-04-25T10:06:54.493124+0000","last_undegraded":"2024-04-25T10:06:54.493124+0000","last_fullsized":"2024-04-25T10:06:54.493124+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:02.454534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509705+0000","last_change":"2024-04-25T10:06:53.503283+0000","last_active":"2024-04-25T10:06:54.509705+0000","last_peered":"2024-04-25T10:06:54.509705+0000","last_clean":"2024-04-25T10:06:54.509705+0000","last_became_active":"2024-04-25T10:06:53.501066+0000","last_became_peered":"2024-04-25T10:06:53.501066+0000","last_unstale":"2024-04-25T10:06:54.509705+0000","last_undegraded":"2024-04-25T10:06:54.509705+0000","last_fullsized":"2024-04-25T10:06:54.509705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:06.707170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491984+0000","last_change":"2024-04-25T10:06:52.491968+0000","last_active":"2024-04-25T10:06:54.491984+0000","last_peered":"2024-04-25T10:06:54.491984+0000","last_clean":"2024-04-25T10:06:54.491984+0000","last_became_active":"2024-04-25T10:06:52.491578+0000","last_became_peered":"2024-04-25T10:06:52.491578+0000","last_unstale":"2024-04-25T10:06:54.491984+0000","last_undegraded":"2024-04-25T10:06:54.491984+0000","last_fullsized":"2024-04-25T10:06:54.491984+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:12.239761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507480+0000","last_change":"2024-04-25T10:06:53.502547+0000","last_active":"2024-04-25T10:06:53.507480+0000","last_peered":"2024-04-25T10:06:53.507480+0000","last_clean":"2024-04-25T10:06:53.507480+0000","last_became_active":"2024-04-25T10:06:53.500862+0000","last_became_peered":"2024-04-25T10:06:53.500862+0000","last_unstale":"2024-04-25T10:06:53.507480+0000","last_undegraded":"2024-04-25T10:06:53.507480+0000","last_fullsized":"2024-04-25T10:06:53.507480+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:40.586964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"196'3","reported_seq":17,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493125+0000","last_change":"2024-04-25T10:06:52.495765+0000","last_active":"2024-04-25T10:06:54.493125+0000","last_peered":"2024-04-25T10:06:54.493125+0000","last_clean":"2024-04-25T10:06:54.493125+0000","last_became_active":"2024-04-25T10:06:52.495507+0000","last_became_peered":"2024-04-25T10:06:52.495507+0000","last_unstale":"2024-04-25T10:06:54.493125+0000","last_undegraded":"2024-04-25T10:06:54.493125+0000","last_fullsized":"2024-04-25T10:06:54.493125+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:00.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491798+0000","last_change":"2024-04-25T10:06:53.501388+0000","last_active":"2024-04-25T10:06:54.491798+0000","last_peered":"2024-04-25T10:06:54.491798+0000","last_clean":"2024-04-25T10:06:54.491798+0000","last_became_active":"2024-04-25T10:06:53.501102+0000","last_became_peered":"2024-04-25T10:06:53.501102+0000","last_unstale":"2024-04-25T10:06:54.491798+0000","last_undegraded":"2024-04-25T10:06:54.491798+0000","last_fullsized":"2024-04-25T10:06:54.491798+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:47.109377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"196'6","reported_seq":18,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.523123+0000","last_change":"2024-04-25T10:06:52.488704+0000","last_active":"2024-04-25T10:06:53.523123+0000","last_peered":"2024-04-25T10:06:53.523123+0000","last_clean":"2024-04-25T10:06:53.523123+0000","last_became_active":"2024-04-25T10:06:52.488416+0000","last_became_peered":"2024-04-25T10:06:52.488416+0000","last_unstale":"2024-04-25T10:06:53.523123+0000","last_undegraded":"2024-04-25T10:06:53.523123+0000","last_fullsized":"2024-04-25T10:06:53.523123+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:41.686476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506245+0000","last_change":"2024-04-25T10:06:53.496941+0000","last_active":"2024-04-25T10:06:54.506245+0000","last_peered":"2024-04-25T10:06:54.506245+0000","last_clean":"2024-04-25T10:06:54.506245+0000","last_became_active":"2024-04-25T10:06:53.496686+0000","last_became_peered":"2024-04-25T10:06:53.496686+0000","last_unstale":"2024-04-25T10:06:54.506245+0000","last_undegraded":"2024-04-25T10:06:54.506245+0000","last_fullsized":"2024-04-25T10:06:54.506245+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.665724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.519959+0000","last_change":"2024-04-25T10:06:52.499728+0000","last_active":"2024-04-25T10:06:53.519959+0000","last_peered":"2024-04-25T10:06:53.519959+0000","last_clean":"2024-04-25T10:06:53.519959+0000","last_became_active":"2024-04-25T10:06:52.499452+0000","last_became_peered":"2024-04-25T10:06:52.499452+0000","last_unstale":"2024-04-25T10:06:53.519959+0000","last_undegraded":"2024-04-25T10:06:53.519959+0000","last_fullsized":"2024-04-25T10:06:53.519959+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:16.435934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.509809+0000","last_change":"2024-04-25T10:06:53.506139+0000","last_active":"2024-04-25T10:06:53.509809+0000","last_peered":"2024-04-25T10:06:53.509809+0000","last_clean":"2024-04-25T10:06:53.509809+0000","last_became_active":"2024-04-25T10:06:53.505836+0000","last_became_peered":"2024-04-25T10:06:53.505836+0000","last_unstale":"2024-04-25T10:06:53.509809+0000","last_undegraded":"2024-04-25T10:06:53.509809+0000","last_fullsized":"2024-04-25T10:06:53.509809+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:01.904113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506346+0000","last_change":"2024-04-25T10:06:52.492663+0000","last_active":"2024-04-25T10:06:54.506346+0000","last_peered":"2024-04-25T10:06:54.506346+0000","last_clean":"2024-04-25T10:06:54.506346+0000","last_became_active":"2024-04-25T10:06:52.492389+0000","last_became_peered":"2024-04-25T10:06:52.492389+0000","last_unstale":"2024-04-25T10:06:54.506346+0000","last_undegraded":"2024-04-25T10:06:54.506346+0000","last_fullsized":"2024-04-25T10:06:54.506346+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.074888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.502367+0000","last_change":"2024-04-25T10:06:53.498504+0000","last_active":"2024-04-25T10:06:53.502367+0000","last_peered":"2024-04-25T10:06:53.502367+0000","last_clean":"2024-04-25T10:06:53.502367+0000","last_became_active":"2024-04-25T10:06:53.498211+0000","last_became_peered":"2024-04-25T10:06:53.498211+0000","last_unstale":"2024-04-25T10:06:53.502367+0000","last_undegraded":"2024-04-25T10:06:53.502367+0000","last_fullsized":"2024-04-25T10:06:53.502367+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:19.611795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.494562+0000","last_change":"2024-04-25T10:06:52.496638+0000","last_active":"2024-04-25T10:06:53.494562+0000","last_peered":"2024-04-25T10:06:53.494562+0000","last_clean":"2024-04-25T10:06:53.494562+0000","last_became_active":"2024-04-25T10:06:52.496343+0000","last_became_peered":"2024-04-25T10:06:52.496343+0000","last_unstale":"2024-04-25T10:06:53.494562+0000","last_undegraded":"2024-04-25T10:06:53.494562+0000","last_fullsized":"2024-04-25T10:06:53.494562+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:33.232420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509443+0000","last_change":"2024-04-25T10:06:53.497026+0000","last_active":"2024-04-25T10:06:54.509443+0000","last_peered":"2024-04-25T10:06:54.509443+0000","last_clean":"2024-04-25T10:06:54.509443+0000","last_became_active":"2024-04-25T10:06:53.495116+0000","last_became_peered":"2024-04-25T10:06:53.495116+0000","last_unstale":"2024-04-25T10:06:54.509443+0000","last_undegraded":"2024-04-25T10:06:54.509443+0000","last_fullsized":"2024-04-25T10:06:54.509443+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:35.905488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"196'8","reported_seq":20,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.521555+0000","last_change":"2024-04-25T10:06:52.494696+0000","last_active":"2024-04-25T10:06:53.521555+0000","last_peered":"2024-04-25T10:06:53.521555+0000","last_clean":"2024-04-25T10:06:53.521555+0000","last_became_active":"2024-04-25T10:06:52.494372+0000","last_became_peered":"2024-04-25T10:06:52.494372+0000","last_unstale":"2024-04-25T10:06:53.521555+0000","last_undegraded":"2024-04-25T10:06:53.521555+0000","last_fullsized":"2024-04-25T10:06:53.521555+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:29.016504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506315+0000","last_change":"2024-04-25T10:06:53.506355+0000","last_active":"2024-04-25T10:06:54.506315+0000","last_peered":"2024-04-25T10:06:54.506315+0000","last_clean":"2024-04-25T10:06:54.506315+0000","last_became_active":"2024-04-25T10:06:53.506121+0000","last_became_peered":"2024-04-25T10:06:53.506121+0000","last_unstale":"2024-04-25T10:06:54.506315+0000","last_undegraded":"2024-04-25T10:06:54.506315+0000","last_fullsized":"2024-04-25T10:06:54.506315+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:19.611795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"196'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.520292+0000","last_change":"2024-04-25T10:06:52.494770+0000","last_active":"2024-04-25T10:06:53.520292+0000","last_peered":"2024-04-25T10:06:53.520292+0000","last_clean":"2024-04-25T10:06:53.520292+0000","last_became_active":"2024-04-25T10:06:52.494415+0000","last_became_peered":"2024-04-25T10:06:52.494415+0000","last_unstale":"2024-04-25T10:06:53.520292+0000","last_undegraded":"2024-04-25T10:06:53.520292+0000","last_fullsized":"2024-04-25T10:06:53.520292+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:05.703258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493020+0000","last_change":"2024-04-25T10:06:53.501021+0000","last_active":"2024-04-25T10:06:54.493020+0000","last_peered":"2024-04-25T10:06:54.493020+0000","last_clean":"2024-04-25T10:06:54.493020+0000","last_became_active":"2024-04-25T10:06:53.500343+0000","last_became_peered":"2024-04-25T10:06:53.500343+0000","last_unstale":"2024-04-25T10:06:54.493020+0000","last_undegraded":"2024-04-25T10:06:54.493020+0000","last_fullsized":"2024-04-25T10:06:54.493020+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:14.600607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"196'2","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829126+0000","last_change":"2024-04-25T10:06:52.494499+0000","last_active":"2024-04-25T10:06:54.829126+0000","last_peered":"2024-04-25T10:06:54.829126+0000","last_clean":"2024-04-25T10:06:54.829126+0000","last_became_active":"2024-04-25T10:06:52.494253+0000","last_became_peered":"2024-04-25T10:06:52.494253+0000","last_unstale":"2024-04-25T10:06:54.829126+0000","last_undegraded":"2024-04-25T10:06:54.829126+0000","last_fullsized":"2024-04-25T10:06:54.829126+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:31.728097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.506760+0000","last_change":"2024-04-25T10:06:53.500438+0000","last_active":"2024-04-25T10:06:53.506760+0000","last_peered":"2024-04-25T10:06:53.506760+0000","last_clean":"2024-04-25T10:06:53.506760+0000","last_became_active":"2024-04-25T10:06:53.499757+0000","last_became_peered":"2024-04-25T10:06:53.499757+0000","last_unstale":"2024-04-25T10:06:53.506760+0000","last_undegraded":"2024-04-25T10:06:53.506760+0000","last_fullsized":"2024-04-25T10:06:53.506760+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.280124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"196'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.519810+0000","last_change":"2024-04-25T10:06:52.498806+0000","last_active":"2024-04-25T10:06:53.519810+0000","last_peered":"2024-04-25T10:06:53.519810+0000","last_clean":"2024-04-25T10:06:53.519810+0000","last_became_active":"2024-04-25T10:06:52.498522+0000","last_became_peered":"2024-04-25T10:06:52.498522+0000","last_unstale":"2024-04-25T10:06:53.519810+0000","last_undegraded":"2024-04-25T10:06:53.519810+0000","last_fullsized":"2024-04-25T10:06:53.519810+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:22.661441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517238+0000","last_change":"2024-04-25T10:06:53.512212+0000","last_active":"2024-04-25T10:06:53.517238+0000","last_peered":"2024-04-25T10:06:53.517238+0000","last_clean":"2024-04-25T10:06:53.517238+0000","last_became_active":"2024-04-25T10:06:53.511949+0000","last_became_peered":"2024-04-25T10:06:53.511949+0000","last_unstale":"2024-04-25T10:06:53.517238+0000","last_undegraded":"2024-04-25T10:06:53.517238+0000","last_fullsized":"2024-04-25T10:06:53.517238+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:39.177925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.496513+0000","last_change":"2024-04-25T10:06:52.493475+0000","last_active":"2024-04-25T10:06:53.496513+0000","last_peered":"2024-04-25T10:06:53.496513+0000","last_clean":"2024-04-25T10:06:53.496513+0000","last_became_active":"2024-04-25T10:06:52.493188+0000","last_became_peered":"2024-04-25T10:06:52.493188+0000","last_unstale":"2024-04-25T10:06:53.496513+0000","last_undegraded":"2024-04-25T10:06:53.496513+0000","last_fullsized":"2024-04-25T10:06:53.496513+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:58.375265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517460+0000","last_change":"2024-04-25T10:06:53.509350+0000","last_active":"2024-04-25T10:06:53.517460+0000","last_peered":"2024-04-25T10:06:53.517460+0000","last_clean":"2024-04-25T10:06:53.517460+0000","last_became_active":"2024-04-25T10:06:53.508930+0000","last_became_peered":"2024-04-25T10:06:53.508930+0000","last_unstale":"2024-04-25T10:06:53.517460+0000","last_undegraded":"2024-04-25T10:06:53.517460+0000","last_fullsized":"2024-04-25T10:06:53.517460+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:55.803648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.521951+0000","last_change":"2024-04-25T10:06:52.499604+0000","last_active":"2024-04-25T10:06:53.521951+0000","last_peered":"2024-04-25T10:06:53.521951+0000","last_clean":"2024-04-25T10:06:53.521951+0000","last_became_active":"2024-04-25T10:06:52.498958+0000","last_became_peered":"2024-04-25T10:06:52.498958+0000","last_unstale":"2024-04-25T10:06:53.521951+0000","last_undegraded":"2024-04-25T10:06:53.521951+0000","last_fullsized":"2024-04-25T10:06:53.521951+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:33.782562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492758+0000","last_change":"2024-04-25T10:06:53.496726+0000","last_active":"2024-04-25T10:06:54.492758+0000","last_peered":"2024-04-25T10:06:54.492758+0000","last_clean":"2024-04-25T10:06:54.492758+0000","last_became_active":"2024-04-25T10:06:53.496444+0000","last_became_peered":"2024-04-25T10:06:53.496444+0000","last_unstale":"2024-04-25T10:06:54.492758+0000","last_undegraded":"2024-04-25T10:06:54.492758+0000","last_fullsized":"2024-04-25T10:06:54.492758+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:11.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509517+0000","last_change":"2024-04-25T10:06:52.491358+0000","last_active":"2024-04-25T10:06:54.509517+0000","last_peered":"2024-04-25T10:06:54.509517+0000","last_clean":"2024-04-25T10:06:54.509517+0000","last_became_active":"2024-04-25T10:06:52.491003+0000","last_became_peered":"2024-04-25T10:06:52.491003+0000","last_unstale":"2024-04-25T10:06:54.509517+0000","last_undegraded":"2024-04-25T10:06:54.509517+0000","last_fullsized":"2024-04-25T10:06:54.509517+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.910473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506197+0000","last_change":"2024-04-25T10:06:53.504813+0000","last_active":"2024-04-25T10:06:54.506197+0000","last_peered":"2024-04-25T10:06:54.506197+0000","last_clean":"2024-04-25T10:06:54.506197+0000","last_became_active":"2024-04-25T10:06:53.504200+0000","last_became_peered":"2024-04-25T10:06:53.504200+0000","last_unstale":"2024-04-25T10:06:54.506197+0000","last_undegraded":"2024-04-25T10:06:54.506197+0000","last_fullsized":"2024-04-25T10:06:54.506197+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.280342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509388+0000","last_change":"2024-04-25T10:06:52.493447+0000","last_active":"2024-04-25T10:06:54.509388+0000","last_peered":"2024-04-25T10:06:54.509388+0000","last_clean":"2024-04-25T10:06:54.509388+0000","last_became_active":"2024-04-25T10:06:52.493199+0000","last_became_peered":"2024-04-25T10:06:52.493199+0000","last_unstale":"2024-04-25T10:06:54.509388+0000","last_undegraded":"2024-04-25T10:06:54.509388+0000","last_fullsized":"2024-04-25T10:06:54.509388+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:10.241193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507612+0000","last_change":"2024-04-25T10:06:53.505356+0000","last_active":"2024-04-25T10:06:53.507612+0000","last_peered":"2024-04-25T10:06:53.507612+0000","last_clean":"2024-04-25T10:06:53.507612+0000","last_became_active":"2024-04-25T10:06:53.505087+0000","last_became_peered":"2024-04-25T10:06:53.505087+0000","last_unstale":"2024-04-25T10:06:53.507612+0000","last_undegraded":"2024-04-25T10:06:53.507612+0000","last_fullsized":"2024-04-25T10:06:53.507612+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:43.873052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828826+0000","last_change":"2024-04-25T10:06:52.494978+0000","last_active":"2024-04-25T10:06:54.828826+0000","last_peered":"2024-04-25T10:06:54.828826+0000","last_clean":"2024-04-25T10:06:54.828826+0000","last_became_active":"2024-04-25T10:06:52.494629+0000","last_became_peered":"2024-04-25T10:06:52.494629+0000","last_unstale":"2024-04-25T10:06:54.828826+0000","last_undegraded":"2024-04-25T10:06:54.828826+0000","last_fullsized":"2024-04-25T10:06:54.828826+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:51.457590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506427+0000","last_change":"2024-04-25T10:06:53.498504+0000","last_active":"2024-04-25T10:06:54.506427+0000","last_peered":"2024-04-25T10:06:54.506427+0000","last_clean":"2024-04-25T10:06:54.506427+0000","last_became_active":"2024-04-25T10:06:53.498254+0000","last_became_peered":"2024-04-25T10:06:53.498254+0000","last_unstale":"2024-04-25T10:06:54.506427+0000","last_undegraded":"2024-04-25T10:06:54.506427+0000","last_fullsized":"2024-04-25T10:06:54.506427+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:40.586964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517402+0000","last_change":"2024-04-25T10:06:52.494622+0000","last_active":"2024-04-25T10:06:53.517402+0000","last_peered":"2024-04-25T10:06:53.517402+0000","last_clean":"2024-04-25T10:06:53.517402+0000","last_became_active":"2024-04-25T10:06:52.494175+0000","last_became_peered":"2024-04-25T10:06:52.494175+0000","last_unstale":"2024-04-25T10:06:53.517402+0000","last_undegraded":"2024-04-25T10:06:53.517402+0000","last_fullsized":"2024-04-25T10:06:53.517402+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:09.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493093+0000","last_change":"2024-04-25T10:06:53.500806+0000","last_active":"2024-04-25T10:06:54.493093+0000","last_peered":"2024-04-25T10:06:54.493093+0000","last_clean":"2024-04-25T10:06:54.493093+0000","last_became_active":"2024-04-25T10:06:53.500064+0000","last_became_peered":"2024-04-25T10:06:53.500064+0000","last_unstale":"2024-04-25T10:06:54.493093+0000","last_undegraded":"2024-04-25T10:06:54.493093+0000","last_fullsized":"2024-04-25T10:06:54.493093+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:48.833595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"197'1","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.510687+0000","last_change":"2024-04-25T10:06:52.492874+0000","last_active":"2024-04-25T10:06:54.510687+0000","last_peered":"2024-04-25T10:06:54.510687+0000","last_clean":"2024-04-25T10:06:54.510687+0000","last_became_active":"2024-04-25T10:06:52.492399+0000","last_became_peered":"2024-04-25T10:06:52.492399+0000","last_unstale":"2024-04-25T10:06:54.510687+0000","last_undegraded":"2024-04-25T10:06:54.510687+0000","last_fullsized":"2024-04-25T10:06:54.510687+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:54.757751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508612+0000","last_change":"2024-04-25T10:06:53.504949+0000","last_active":"2024-04-25T10:06:53.508612+0000","last_peered":"2024-04-25T10:06:53.508612+0000","last_clean":"2024-04-25T10:06:53.508612+0000","last_became_active":"2024-04-25T10:06:53.504677+0000","last_became_peered":"2024-04-25T10:06:53.504677+0000","last_unstale":"2024-04-25T10:06:53.508612+0000","last_undegraded":"2024-04-25T10:06:53.508612+0000","last_fullsized":"2024-04-25T10:06:53.508612+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:13.450784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509682+0000","last_change":"2024-04-25T10:06:52.489251+0000","last_active":"2024-04-25T10:06:54.509682+0000","last_peered":"2024-04-25T10:06:54.509682+0000","last_clean":"2024-04-25T10:06:54.509682+0000","last_became_active":"2024-04-25T10:06:52.488980+0000","last_became_peered":"2024-04-25T10:06:52.488980+0000","last_unstale":"2024-04-25T10:06:54.509682+0000","last_undegraded":"2024-04-25T10:06:54.509682+0000","last_fullsized":"2024-04-25T10:06:54.509682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.707912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829213+0000","last_change":"2024-04-25T10:06:53.505826+0000","last_active":"2024-04-25T10:06:54.829213+0000","last_peered":"2024-04-25T10:06:54.829213+0000","last_clean":"2024-04-25T10:06:54.829213+0000","last_became_active":"2024-04-25T10:06:53.505204+0000","last_became_peered":"2024-04-25T10:06:53.505204+0000","last_unstale":"2024-04-25T10:06:54.829213+0000","last_undegraded":"2024-04-25T10:06:54.829213+0000","last_fullsized":"2024-04-25T10:06:54.829213+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:24.794175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.496542+0000","last_change":"2024-04-25T10:06:52.495275+0000","last_active":"2024-04-25T10:06:53.496542+0000","last_peered":"2024-04-25T10:06:53.496542+0000","last_clean":"2024-04-25T10:06:53.496542+0000","last_became_active":"2024-04-25T10:06:52.495016+0000","last_became_peered":"2024-04-25T10:06:52.495016+0000","last_unstale":"2024-04-25T10:06:53.496542+0000","last_undegraded":"2024-04-25T10:06:53.496542+0000","last_fullsized":"2024-04-25T10:06:53.496542+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:12.239761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507856+0000","last_change":"2024-04-25T10:06:53.505732+0000","last_active":"2024-04-25T10:06:53.507856+0000","last_peered":"2024-04-25T10:06:53.507856+0000","last_clean":"2024-04-25T10:06:53.507856+0000","last_became_active":"2024-04-25T10:06:53.505352+0000","last_became_peered":"2024-04-25T10:06:53.505352+0000","last_unstale":"2024-04-25T10:06:53.507856+0000","last_undegraded":"2024-04-25T10:06:53.507856+0000","last_fullsized":"2024-04-25T10:06:53.507856+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:08.542039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.500374+0000","last_change":"2024-04-25T10:06:52.499384+0000","last_active":"2024-04-25T10:06:53.500374+0000","last_peered":"2024-04-25T10:06:53.500374+0000","last_clean":"2024-04-25T10:06:53.500374+0000","last_became_active":"2024-04-25T10:06:52.498945+0000","last_became_peered":"2024-04-25T10:06:52.498945+0000","last_unstale":"2024-04-25T10:06:53.500374+0000","last_undegraded":"2024-04-25T10:06:53.500374+0000","last_fullsized":"2024-04-25T10:06:53.500374+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.651714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.493005+0000","last_change":"2024-04-25T10:06:53.491642+0000","last_active":"2024-04-25T10:06:53.493005+0000","last_peered":"2024-04-25T10:06:53.493005+0000","last_clean":"2024-04-25T10:06:53.493005+0000","last_became_active":"2024-04-25T10:06:53.491382+0000","last_became_peered":"2024-04-25T10:06:53.491382+0000","last_unstale":"2024-04-25T10:06:53.493005+0000","last_undegraded":"2024-04-25T10:06:53.493005+0000","last_fullsized":"2024-04-25T10:06:53.493005+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:18.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491143+0000","last_change":"2024-04-25T10:06:52.491582+0000","last_active":"2024-04-25T10:06:54.491143+0000","last_peered":"2024-04-25T10:06:54.491143+0000","last_clean":"2024-04-25T10:06:54.491143+0000","last_became_active":"2024-04-25T10:06:52.491314+0000","last_became_peered":"2024-04-25T10:06:52.491314+0000","last_unstale":"2024-04-25T10:06:54.491143+0000","last_undegraded":"2024-04-25T10:06:54.491143+0000","last_fullsized":"2024-04-25T10:06:54.491143+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:29.016504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828877+0000","last_change":"2024-04-25T10:06:53.506470+0000","last_active":"2024-04-25T10:06:54.828877+0000","last_peered":"2024-04-25T10:06:54.828877+0000","last_clean":"2024-04-25T10:06:54.828877+0000","last_became_active":"2024-04-25T10:06:53.506224+0000","last_became_peered":"2024-04-25T10:06:53.506224+0000","last_unstale":"2024-04-25T10:06:54.828877+0000","last_undegraded":"2024-04-25T10:06:54.828877+0000","last_fullsized":"2024-04-25T10:06:54.828877+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:41.699855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492853+0000","last_change":"2024-04-25T10:06:52.497632+0000","last_active":"2024-04-25T10:06:54.492853+0000","last_peered":"2024-04-25T10:06:54.492853+0000","last_clean":"2024-04-25T10:06:54.492853+0000","last_became_active":"2024-04-25T10:06:52.497343+0000","last_became_peered":"2024-04-25T10:06:52.497343+0000","last_unstale":"2024-04-25T10:06:54.492853+0000","last_undegraded":"2024-04-25T10:06:54.492853+0000","last_fullsized":"2024-04-25T10:06:54.492853+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:41.686476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.515043+0000","last_change":"2024-04-25T10:06:53.508937+0000","last_active":"2024-04-25T10:06:53.515043+0000","last_peered":"2024-04-25T10:06:53.515043+0000","last_clean":"2024-04-25T10:06:53.515043+0000","last_became_active":"2024-04-25T10:06:53.508628+0000","last_became_peered":"2024-04-25T10:06:53.508628+0000","last_unstale":"2024-04-25T10:06:53.515043+0000","last_undegraded":"2024-04-25T10:06:53.515043+0000","last_fullsized":"2024-04-25T10:06:53.515043+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:26.922488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.497479+0000","last_change":"2024-04-25T10:06:52.495755+0000","last_active":"2024-04-25T10:06:53.497479+0000","last_peered":"2024-04-25T10:06:53.497479+0000","last_clean":"2024-04-25T10:06:53.497479+0000","last_became_active":"2024-04-25T10:06:52.495452+0000","last_became_peered":"2024-04-25T10:06:52.495452+0000","last_unstale":"2024-04-25T10:06:53.497479+0000","last_undegraded":"2024-04-25T10:06:53.497479+0000","last_fullsized":"2024-04-25T10:06:53.497479+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:01.464001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507640+0000","last_change":"2024-04-25T10:06:53.505744+0000","last_active":"2024-04-25T10:06:53.507640+0000","last_peered":"2024-04-25T10:06:53.507640+0000","last_clean":"2024-04-25T10:06:53.507640+0000","last_became_active":"2024-04-25T10:06:53.504889+0000","last_became_peered":"2024-04-25T10:06:53.504889+0000","last_unstale":"2024-04-25T10:06:53.507640+0000","last_undegraded":"2024-04-25T10:06:53.507640+0000","last_fullsized":"2024-04-25T10:06:53.507640+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:03.371549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"196'2","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492845+0000","last_change":"2024-04-25T10:06:52.498191+0000","last_active":"2024-04-25T10:06:54.492845+0000","last_peered":"2024-04-25T10:06:54.492845+0000","last_clean":"2024-04-25T10:06:54.492845+0000","last_became_active":"2024-04-25T10:06:52.497933+0000","last_became_peered":"2024-04-25T10:06:52.497933+0000","last_unstale":"2024-04-25T10:06:54.492845+0000","last_undegraded":"2024-04-25T10:06:54.492845+0000","last_fullsized":"2024-04-25T10:06:54.492845+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:04.422178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":496,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492734+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:06:54.492734+0000","last_peered":"2024-04-25T10:06:54.492734+0000","last_clean":"2024-04-25T10:06:54.492734+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:06:54.492734+0000","last_undegraded":"2024-04-25T10:06:54.492734+0000","last_fullsized":"2024-04-25T10:06:54.492734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52248,"kb_used_data":3020,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319592,"statfs":{"total":96636764160,"available":96583262208,"internally_reserved":0,"allocated":3092480,"data_stored":1760057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2502656,"data_stored":1164063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51672,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320168,"statfs":{"total":96636764160,"available":96583852032,"internally_reserved":0,"allocated":2494464,"data_stored":1164007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59980,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311860,"statfs":{"total":96636764160,"available":96575344640,"internally_reserved":0,"allocated":2482176,"data_stored":1162415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77964,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94293876,"statfs":{"total":96636764160,"available":96556929024,"internally_reserved":0,"allocated":2482176,"data_stored":1162347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55816,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316024,"statfs":{"total":96636764160,"available":96579608576,"internally_reserved":0,"allocated":2486272,"data_stored":1162941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55820,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316020,"statfs":{"total":96636764160,"available":96579604480,"internally_reserved":0,"allocated":2490368,"data_stored":1169017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56396,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315444,"statfs":{"total":96636764160,"available":96579014656,"internally_reserved":0,"allocated":3072000,"data_stored":1752693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:56.773 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:56.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.901+0000 7fac7a8ac640 1 Processor -- start 2024-04-25T10:06:56.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.901+0000 7fac7a8ac640 1 -- start start 2024-04-25T10:06:56.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac7a8ac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac7415dc90 0x7fac741625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac7a8ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740cc8e0 0x7fac7415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac7a8ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7415f390 con 0x7fac740d1810 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac7a8ac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740cce50 con 0x7fac7415dc90 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac7a8ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac74161af0 con 0x7fac740cc8e0 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac737fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740cc8e0 0x7fac7415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac73fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac7415dc90 0x7fac741625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac737fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740cc8e0 0x7fac7415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:50622/0 (socket says 172.21.15.148:50622) 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac73fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac7415dc90 0x7fac741625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:39648/0 (socket says 172.21.15.148:39648) 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac78e49640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fac740d1810 0x7fac740cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:34566/0 (socket says 172.21.15.148:34566) 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.902+0000 7fac737fe640 1 -- 172.21.15.148:0/1569897837 learned_addr learned my addr 172.21.15.148:0/1569897837 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.903+0000 7fac73fff640 1 -- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740cc8e0 msgr2=0x7fac7415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.903+0000 7fac73fff640 1 --2- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740cc8e0 0x7fac7415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.903+0000 7fac73fff640 1 -- 172.21.15.148:0/1569897837 >> v1:172.21.15.188:6789/0 conn(0x7fac740d1810 legacy=0x7fac740cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.903+0000 7fac73fff640 1 -- 172.21.15.148:0/1569897837 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac74161dd0 con 0x7fac7415dc90 2024-04-25T10:06:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.903+0000 7fac73fff640 1 --2- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7415dc90 0x7fac741625d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fac640097a0 tx=0x7fac6402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e372d0ef801eaf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac72ffd640 1 -- 172.21.15.148:0/1569897837 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64037070 con 0x7fac7415dc90 2024-04-25T10:06:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac72ffd640 1 -- 172.21.15.148:0/1569897837 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6402ae50 con 0x7fac7415dc90 2024-04-25T10:06:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac72ffd640 1 -- 172.21.15.148:0/1569897837 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64032660 con 0x7fac7415dc90 2024-04-25T10:06:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 -- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7415dc90 msgr2=0x7fac741625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7415dc90 0x7fac741625d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fac640097a0 tx=0x7fac6402a100 comp rx=0 tx=0).stop 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 -- 172.21.15.148:0/1569897837 shutdown_connections 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740cc8e0 0x7fac7415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/1569897837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac7415dc90 0x7fac741625d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 -- 172.21.15.148:0/1569897837 >> 172.21.15.148:0/1569897837 conn(0x7fac7406f590 msgr2=0x7fac7406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 -- 172.21.15.148:0/1569897837 shutdown_connections 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.904+0000 7fac7a8ac640 1 -- 172.21.15.148:0/1569897837 wait complete. 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.905+0000 7fac7a8ac640 1 Processor -- start 2024-04-25T10:06:56.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.905+0000 7fac7a8ac640 1 -- start start 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac7a8ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac740cc8e0 0x7fac741f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac7a8ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740d1810 0x7fac740caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac7a8ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 0x7fac741e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac7a8ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7415e070 con 0x7fac7415dc90 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac7a8ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740cd130 con 0x7fac740cc8e0 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac7a8ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7415e1f0 con 0x7fac740d1810 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac78e49640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 0x7fac741e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac78e49640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 0x7fac741e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33292/0 (socket says 172.21.15.148:33292) 2024-04-25T10:06:56.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac78e49640 1 -- 172.21.15.148:0/2398313108 learned_addr learned my addr 172.21.15.148:0/2398313108 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac73fff640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac740cc8e0 0x7fac741f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.906+0000 7fac737fe640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740d1810 0x7fac740caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac78e49640 1 -- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740d1810 msgr2=0x7fac740caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac78e49640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740d1810 0x7fac740caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac78e49640 1 -- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac740cc8e0 msgr2=0x7fac741f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac78e49640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac740cc8e0 0x7fac741f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac78e49640 1 -- 172.21.15.148:0/2398313108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740b5890 con 0x7fac7415dc90 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac78e49640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 0x7fac741e82a0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c730 tx=0x7fac6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c017070 con 0x7fac7415dc90 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c00ce40 con 0x7fac7415dc90 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7415b5f0 con 0x7fac7415dc90 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00a7e0 con 0x7fac7415dc90 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.907+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7415bbb0 con 0x7fac7415dc90 2024-04-25T10:06:56.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.908+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac74073990 con 0x7fac7415dc90 2024-04-25T10:06:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.908+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac6c01a070 con 0x7fac7415dc90 2024-04-25T10:06:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.909+0000 7fac70ff9640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac6006b4d0 0x7fac6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.909+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac6c0897d0 con 0x7fac7415dc90 2024-04-25T10:06:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.909+0000 7fac73fff640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac6006b4d0 0x7fac6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.917 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.909+0000 7fac73fff640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac6006b4d0 0x7fac6006d990 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fac64034000 tx=0x7fac6402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.918 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:56.913+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac7415f390 con 0x7fac7415dc90 2024-04-25T10:06:57.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.079+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac7406fd80 con 0x7fac6006b4d0 2024-04-25T10:06:57.090 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.085+0000 7fac70ff9640 1 -- 172.21.15.148:0/2398313108 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252452 (secure 0 0 0) 0x7fac7406fd80 con 0x7fac6006b4d0 2024-04-25T10:06:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:57.094 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac6006b4d0 msgr2=0x7fac6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac6006b4d0 0x7fac6006d990 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fac64034000 tx=0x7fac6402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 msgr2=0x7fac741e82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 0x7fac741e82a0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c730 tx=0x7fac6c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 shutdown_connections 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fac6006b4d0 0x7fac6006d990 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac7415dc90 0x7fac741e82a0 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac740d1810 0x7fac740caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 --2- 172.21.15.148:0/2398313108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac740cc8e0 0x7fac741f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 >> 172.21.15.148:0/2398313108 conn(0x7fac7406f590 msgr2=0x7fac7415ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 shutdown_connections 2024-04-25T10:06:57.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.088+0000 7fac7a8ac640 1 -- 172.21.15.148:0/2398313108 wait complete. 2024-04-25T10:06:57.105 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":573,"stamp":"2024-04-25T10:06:55.176475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":465732,"kb_used_data":20608,"kb_used_omap":12,"kb_used_meta":444979,"kb_avail":754508988,"statfs":{"total":773094113280,"available":772617203712,"internally_reserved":0,"allocated":21102592,"data_stored":10497540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":455659056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000803"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492999+0000","last_change":"2024-04-25T10:06:53.499533+0000","last_active":"2024-04-25T10:06:54.492999+0000","last_peered":"2024-04-25T10:06:54.492999+0000","last_clean":"2024-04-25T10:06:54.492999+0000","last_became_active":"2024-04-25T10:06:53.499259+0000","last_became_peered":"2024-04-25T10:06:53.499259+0000","last_unstale":"2024-04-25T10:06:54.492999+0000","last_undegraded":"2024-04-25T10:06:54.492999+0000","last_fullsized":"2024-04-25T10:06:54.492999+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:21.228046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506330+0000","last_change":"2024-04-25T10:06:53.499287+0000","last_active":"2024-04-25T10:06:54.506330+0000","last_peered":"2024-04-25T10:06:54.506330+0000","last_clean":"2024-04-25T10:06:54.506330+0000","last_became_active":"2024-04-25T10:06:53.498985+0000","last_became_peered":"2024-04-25T10:06:53.498985+0000","last_unstale":"2024-04-25T10:06:54.506330+0000","last_undegraded":"2024-04-25T10:06:54.506330+0000","last_fullsized":"2024-04-25T10:06:54.506330+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.280124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.498731+0000","last_change":"2024-04-25T10:06:53.495495+0000","last_active":"2024-04-25T10:06:53.498731+0000","last_peered":"2024-04-25T10:06:53.498731+0000","last_clean":"2024-04-25T10:06:53.498731+0000","last_became_active":"2024-04-25T10:06:53.495207+0000","last_became_peered":"2024-04-25T10:06:53.495207+0000","last_unstale":"2024-04-25T10:06:53.498731+0000","last_undegraded":"2024-04-25T10:06:53.498731+0000","last_fullsized":"2024-04-25T10:06:53.498731+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:24.372617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491485+0000","last_change":"2024-04-25T10:06:53.496243+0000","last_active":"2024-04-25T10:06:54.491485+0000","last_peered":"2024-04-25T10:06:54.491485+0000","last_clean":"2024-04-25T10:06:54.491485+0000","last_became_active":"2024-04-25T10:06:53.495981+0000","last_became_peered":"2024-04-25T10:06:53.495981+0000","last_unstale":"2024-04-25T10:06:54.491485+0000","last_undegraded":"2024-04-25T10:06:54.491485+0000","last_fullsized":"2024-04-25T10:06:54.491485+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:13.655626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492874+0000","last_change":"2024-04-25T10:06:53.506395+0000","last_active":"2024-04-25T10:06:54.492874+0000","last_peered":"2024-04-25T10:06:54.492874+0000","last_clean":"2024-04-25T10:06:54.492874+0000","last_became_active":"2024-04-25T10:06:53.506095+0000","last_became_peered":"2024-04-25T10:06:53.506095+0000","last_unstale":"2024-04-25T10:06:54.492874+0000","last_undegraded":"2024-04-25T10:06:54.492874+0000","last_fullsized":"2024-04-25T10:06:54.492874+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:46.781173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.504177+0000","last_change":"2024-04-25T10:06:53.501579+0000","last_active":"2024-04-25T10:06:53.504177+0000","last_peered":"2024-04-25T10:06:53.504177+0000","last_clean":"2024-04-25T10:06:53.504177+0000","last_became_active":"2024-04-25T10:06:53.501313+0000","last_became_peered":"2024-04-25T10:06:53.501313+0000","last_unstale":"2024-04-25T10:06:53.504177+0000","last_undegraded":"2024-04-25T10:06:53.504177+0000","last_fullsized":"2024-04-25T10:06:53.504177+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:03.458446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507584+0000","last_change":"2024-04-25T10:06:53.503403+0000","last_active":"2024-04-25T10:06:53.507584+0000","last_peered":"2024-04-25T10:06:53.507584+0000","last_clean":"2024-04-25T10:06:53.507584+0000","last_became_active":"2024-04-25T10:06:53.503133+0000","last_became_peered":"2024-04-25T10:06:53.503133+0000","last_unstale":"2024-04-25T10:06:53.507584+0000","last_undegraded":"2024-04-25T10:06:53.507584+0000","last_fullsized":"2024-04-25T10:06:53.507584+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:39.235821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.504044+0000","last_change":"2024-04-25T10:06:53.502226+0000","last_active":"2024-04-25T10:06:53.504044+0000","last_peered":"2024-04-25T10:06:53.504044+0000","last_clean":"2024-04-25T10:06:53.504044+0000","last_became_active":"2024-04-25T10:06:53.501942+0000","last_became_peered":"2024-04-25T10:06:53.501942+0000","last_unstale":"2024-04-25T10:06:53.504044+0000","last_undegraded":"2024-04-25T10:06:53.504044+0000","last_fullsized":"2024-04-25T10:06:53.504044+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:23.475397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829268+0000","last_change":"2024-04-25T10:06:53.506051+0000","last_active":"2024-04-25T10:06:54.829268+0000","last_peered":"2024-04-25T10:06:54.829268+0000","last_clean":"2024-04-25T10:06:54.829268+0000","last_became_active":"2024-04-25T10:06:53.505573+0000","last_became_peered":"2024-04-25T10:06:53.505573+0000","last_unstale":"2024-04-25T10:06:54.829268+0000","last_undegraded":"2024-04-25T10:06:54.829268+0000","last_fullsized":"2024-04-25T10:06:54.829268+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:41.699691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.499927+0000","last_change":"2024-04-25T10:06:53.496842+0000","last_active":"2024-04-25T10:06:53.499927+0000","last_peered":"2024-04-25T10:06:53.499927+0000","last_clean":"2024-04-25T10:06:53.499927+0000","last_became_active":"2024-04-25T10:06:53.495825+0000","last_became_peered":"2024-04-25T10:06:53.495825+0000","last_unstale":"2024-04-25T10:06:53.499927+0000","last_undegraded":"2024-04-25T10:06:53.499927+0000","last_fullsized":"2024-04-25T10:06:53.499927+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.280342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491621+0000","last_change":"2024-04-25T10:06:53.496862+0000","last_active":"2024-04-25T10:06:54.491621+0000","last_peered":"2024-04-25T10:06:54.491621+0000","last_clean":"2024-04-25T10:06:54.491621+0000","last_became_active":"2024-04-25T10:06:53.495833+0000","last_became_peered":"2024-04-25T10:06:53.495833+0000","last_unstale":"2024-04-25T10:06:54.491621+0000","last_undegraded":"2024-04-25T10:06:54.491621+0000","last_fullsized":"2024-04-25T10:06:54.491621+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:16.909645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.509860+0000","last_change":"2024-04-25T10:06:53.506846+0000","last_active":"2024-04-25T10:06:53.509860+0000","last_peered":"2024-04-25T10:06:53.509860+0000","last_clean":"2024-04-25T10:06:53.509860+0000","last_became_active":"2024-04-25T10:06:53.506472+0000","last_became_peered":"2024-04-25T10:06:53.506472+0000","last_unstale":"2024-04-25T10:06:53.509860+0000","last_undegraded":"2024-04-25T10:06:53.509860+0000","last_fullsized":"2024-04-25T10:06:53.509860+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.736033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.505793+0000","last_change":"2024-04-25T10:06:53.503669+0000","last_active":"2024-04-25T10:06:53.505793+0000","last_peered":"2024-04-25T10:06:53.505793+0000","last_clean":"2024-04-25T10:06:53.505793+0000","last_became_active":"2024-04-25T10:06:53.503350+0000","last_became_peered":"2024-04-25T10:06:53.503350+0000","last_unstale":"2024-04-25T10:06:53.505793+0000","last_undegraded":"2024-04-25T10:06:53.505793+0000","last_fullsized":"2024-04-25T10:06:53.505793+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:39.313543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508843+0000","last_change":"2024-04-25T10:06:53.503319+0000","last_active":"2024-04-25T10:06:53.508843+0000","last_peered":"2024-04-25T10:06:53.508843+0000","last_clean":"2024-04-25T10:06:53.508843+0000","last_became_active":"2024-04-25T10:06:53.503019+0000","last_became_peered":"2024-04-25T10:06:53.503019+0000","last_unstale":"2024-04-25T10:06:53.508843+0000","last_undegraded":"2024-04-25T10:06:53.508843+0000","last_fullsized":"2024-04-25T10:06:53.508843+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:47.109377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509427+0000","last_change":"2024-04-25T10:06:53.508607+0000","last_active":"2024-04-25T10:06:54.509427+0000","last_peered":"2024-04-25T10:06:54.509427+0000","last_clean":"2024-04-25T10:06:54.509427+0000","last_became_active":"2024-04-25T10:06:53.508323+0000","last_became_peered":"2024-04-25T10:06:53.508323+0000","last_unstale":"2024-04-25T10:06:54.509427+0000","last_undegraded":"2024-04-25T10:06:54.509427+0000","last_fullsized":"2024-04-25T10:06:54.509427+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:23.901917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.499800+0000","last_change":"2024-04-25T10:06:53.498592+0000","last_active":"2024-04-25T10:06:53.499800+0000","last_peered":"2024-04-25T10:06:53.499800+0000","last_clean":"2024-04-25T10:06:53.499800+0000","last_became_active":"2024-04-25T10:06:53.498309+0000","last_became_peered":"2024-04-25T10:06:53.498309+0000","last_unstale":"2024-04-25T10:06:53.499800+0000","last_undegraded":"2024-04-25T10:06:53.499800+0000","last_fullsized":"2024-04-25T10:06:53.499800+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:05.426090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517212+0000","last_change":"2024-04-25T10:06:53.509516+0000","last_active":"2024-04-25T10:06:53.517212+0000","last_peered":"2024-04-25T10:06:53.517212+0000","last_clean":"2024-04-25T10:06:53.517212+0000","last_became_active":"2024-04-25T10:06:53.509076+0000","last_became_peered":"2024-04-25T10:06:53.509076+0000","last_unstale":"2024-04-25T10:06:53.517212+0000","last_undegraded":"2024-04-25T10:06:53.517212+0000","last_fullsized":"2024-04-25T10:06:53.517212+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:31.733339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.491990+0000","last_change":"2024-04-25T10:06:53.490235+0000","last_active":"2024-04-25T10:06:53.491990+0000","last_peered":"2024-04-25T10:06:53.491990+0000","last_clean":"2024-04-25T10:06:53.491990+0000","last_became_active":"2024-04-25T10:06:53.489973+0000","last_became_peered":"2024-04-25T10:06:53.489973+0000","last_unstale":"2024-04-25T10:06:53.491990+0000","last_undegraded":"2024-04-25T10:06:53.491990+0000","last_fullsized":"2024-04-25T10:06:53.491990+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:21.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492958+0000","last_change":"2024-04-25T10:06:53.500588+0000","last_active":"2024-04-25T10:06:54.492958+0000","last_peered":"2024-04-25T10:06:54.492958+0000","last_clean":"2024-04-25T10:06:54.492958+0000","last_became_active":"2024-04-25T10:06:53.499768+0000","last_became_peered":"2024-04-25T10:06:53.499768+0000","last_unstale":"2024-04-25T10:06:54.492958+0000","last_undegraded":"2024-04-25T10:06:54.492958+0000","last_fullsized":"2024-04-25T10:06:54.492958+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:35.342883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.503779+0000","last_change":"2024-04-25T10:06:53.499873+0000","last_active":"2024-04-25T10:06:53.503779+0000","last_peered":"2024-04-25T10:06:53.503779+0000","last_clean":"2024-04-25T10:06:53.503779+0000","last_became_active":"2024-04-25T10:06:53.499612+0000","last_became_peered":"2024-04-25T10:06:53.499612+0000","last_unstale":"2024-04-25T10:06:53.503779+0000","last_undegraded":"2024-04-25T10:06:53.503779+0000","last_fullsized":"2024-04-25T10:06:53.503779+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:01.768702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.493566+0000","last_change":"2024-04-25T10:06:53.492847+0000","last_active":"2024-04-25T10:06:53.493566+0000","last_peered":"2024-04-25T10:06:53.493566+0000","last_clean":"2024-04-25T10:06:53.493566+0000","last_became_active":"2024-04-25T10:06:53.492570+0000","last_became_peered":"2024-04-25T10:06:53.492570+0000","last_unstale":"2024-04-25T10:06:53.493566+0000","last_undegraded":"2024-04-25T10:06:53.493566+0000","last_fullsized":"2024-04-25T10:06:53.493566+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:52.329847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509423+0000","last_change":"2024-04-25T10:06:53.500818+0000","last_active":"2024-04-25T10:06:54.509423+0000","last_peered":"2024-04-25T10:06:54.509423+0000","last_clean":"2024-04-25T10:06:54.509423+0000","last_became_active":"2024-04-25T10:06:53.499454+0000","last_became_peered":"2024-04-25T10:06:53.499454+0000","last_unstale":"2024-04-25T10:06:54.509423+0000","last_undegraded":"2024-04-25T10:06:54.509423+0000","last_fullsized":"2024-04-25T10:06:54.509423+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.930376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828871+0000","last_change":"2024-04-25T10:06:53.505222+0000","last_active":"2024-04-25T10:06:54.828871+0000","last_peered":"2024-04-25T10:06:54.828871+0000","last_clean":"2024-04-25T10:06:54.828871+0000","last_became_active":"2024-04-25T10:06:53.504922+0000","last_became_peered":"2024-04-25T10:06:53.504922+0000","last_unstale":"2024-04-25T10:06:54.828871+0000","last_undegraded":"2024-04-25T10:06:54.828871+0000","last_fullsized":"2024-04-25T10:06:54.828871+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:23.116854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492610+0000","last_change":"2024-04-25T10:06:53.496210+0000","last_active":"2024-04-25T10:06:54.492610+0000","last_peered":"2024-04-25T10:06:54.492610+0000","last_clean":"2024-04-25T10:06:54.492610+0000","last_became_active":"2024-04-25T10:06:53.495882+0000","last_became_peered":"2024-04-25T10:06:53.495882+0000","last_unstale":"2024-04-25T10:06:54.492610+0000","last_undegraded":"2024-04-25T10:06:54.492610+0000","last_fullsized":"2024-04-25T10:06:54.492610+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:25.914385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492990+0000","last_change":"2024-04-25T10:06:53.498931+0000","last_active":"2024-04-25T10:06:54.492990+0000","last_peered":"2024-04-25T10:06:54.492990+0000","last_clean":"2024-04-25T10:06:54.492990+0000","last_became_active":"2024-04-25T10:06:53.498666+0000","last_became_peered":"2024-04-25T10:06:53.498666+0000","last_unstale":"2024-04-25T10:06:54.492990+0000","last_undegraded":"2024-04-25T10:06:54.492990+0000","last_fullsized":"2024-04-25T10:06:54.492990+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:45.505542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507671+0000","last_change":"2024-04-25T10:06:53.504383+0000","last_active":"2024-04-25T10:06:53.507671+0000","last_peered":"2024-04-25T10:06:53.507671+0000","last_clean":"2024-04-25T10:06:53.507671+0000","last_became_active":"2024-04-25T10:06:53.504091+0000","last_became_peered":"2024-04-25T10:06:53.504091+0000","last_unstale":"2024-04-25T10:06:53.507671+0000","last_undegraded":"2024-04-25T10:06:53.507671+0000","last_fullsized":"2024-04-25T10:06:53.507671+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:32.305830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491729+0000","last_change":"2024-04-25T10:06:53.496930+0000","last_active":"2024-04-25T10:06:54.491729+0000","last_peered":"2024-04-25T10:06:54.491729+0000","last_clean":"2024-04-25T10:06:54.491729+0000","last_became_active":"2024-04-25T10:06:53.496681+0000","last_became_peered":"2024-04-25T10:06:53.496681+0000","last_unstale":"2024-04-25T10:06:54.491729+0000","last_undegraded":"2024-04-25T10:06:54.491729+0000","last_fullsized":"2024-04-25T10:06:54.491729+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:22.628297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508896+0000","last_change":"2024-04-25T10:06:53.503396+0000","last_active":"2024-04-25T10:06:53.508896+0000","last_peered":"2024-04-25T10:06:53.508896+0000","last_clean":"2024-04-25T10:06:53.508896+0000","last_became_active":"2024-04-25T10:06:53.503062+0000","last_became_peered":"2024-04-25T10:06:53.503062+0000","last_unstale":"2024-04-25T10:06:53.508896+0000","last_undegraded":"2024-04-25T10:06:53.508896+0000","last_fullsized":"2024-04-25T10:06:53.508896+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:10.544964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506353+0000","last_change":"2024-04-25T10:06:53.497618+0000","last_active":"2024-04-25T10:06:54.506353+0000","last_peered":"2024-04-25T10:06:54.506353+0000","last_clean":"2024-04-25T10:06:54.506353+0000","last_became_active":"2024-04-25T10:06:53.497363+0000","last_became_peered":"2024-04-25T10:06:53.497363+0000","last_unstale":"2024-04-25T10:06:54.506353+0000","last_undegraded":"2024-04-25T10:06:54.506353+0000","last_fullsized":"2024-04-25T10:06:54.506353+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:20.503651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508657+0000","last_change":"2024-04-25T10:06:53.505556+0000","last_active":"2024-04-25T10:06:53.508657+0000","last_peered":"2024-04-25T10:06:53.508657+0000","last_clean":"2024-04-25T10:06:53.508657+0000","last_became_active":"2024-04-25T10:06:53.505289+0000","last_became_peered":"2024-04-25T10:06:53.505289+0000","last_unstale":"2024-04-25T10:06:53.508657+0000","last_undegraded":"2024-04-25T10:06:53.508657+0000","last_fullsized":"2024-04-25T10:06:53.508657+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:55.431312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828930+0000","last_change":"2024-04-25T10:06:53.503147+0000","last_active":"2024-04-25T10:06:54.828930+0000","last_peered":"2024-04-25T10:06:54.828930+0000","last_clean":"2024-04-25T10:06:54.828930+0000","last_became_active":"2024-04-25T10:06:53.502658+0000","last_became_peered":"2024-04-25T10:06:53.502658+0000","last_unstale":"2024-04-25T10:06:54.828930+0000","last_undegraded":"2024-04-25T10:06:54.828930+0000","last_fullsized":"2024-04-25T10:06:54.828930+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:06.382243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508987+0000","last_change":"2024-04-25T10:06:53.506986+0000","last_active":"2024-04-25T10:06:53.508987+0000","last_peered":"2024-04-25T10:06:53.508987+0000","last_clean":"2024-04-25T10:06:53.508987+0000","last_became_active":"2024-04-25T10:06:53.506676+0000","last_became_peered":"2024-04-25T10:06:53.506676+0000","last_unstale":"2024-04-25T10:06:53.508987+0000","last_undegraded":"2024-04-25T10:06:53.508987+0000","last_fullsized":"2024-04-25T10:06:53.508987+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:04.277774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.497702+0000","last_change":"2024-04-25T10:06:53.496250+0000","last_active":"2024-04-25T10:06:53.497702+0000","last_peered":"2024-04-25T10:06:53.497702+0000","last_clean":"2024-04-25T10:06:53.497702+0000","last_became_active":"2024-04-25T10:06:53.495993+0000","last_became_peered":"2024-04-25T10:06:53.495993+0000","last_unstale":"2024-04-25T10:06:53.497702+0000","last_undegraded":"2024-04-25T10:06:53.497702+0000","last_fullsized":"2024-04-25T10:06:53.497702+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:08.628944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"196'4","reported_seq":18,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829301+0000","last_change":"2024-04-25T10:06:52.495381+0000","last_active":"2024-04-25T10:06:54.829301+0000","last_peered":"2024-04-25T10:06:54.829301+0000","last_clean":"2024-04-25T10:06:54.829301+0000","last_became_active":"2024-04-25T10:06:52.495127+0000","last_became_peered":"2024-04-25T10:06:52.495127+0000","last_unstale":"2024-04-25T10:06:54.829301+0000","last_undegraded":"2024-04-25T10:06:54.829301+0000","last_fullsized":"2024-04-25T10:06:54.829301+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:01.323500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.499613+0000","last_change":"2024-04-25T10:06:53.497909+0000","last_active":"2024-04-25T10:06:53.499613+0000","last_peered":"2024-04-25T10:06:53.499613+0000","last_clean":"2024-04-25T10:06:53.499613+0000","last_became_active":"2024-04-25T10:06:53.497629+0000","last_became_peered":"2024-04-25T10:06:53.497629+0000","last_unstale":"2024-04-25T10:06:53.499613+0000","last_undegraded":"2024-04-25T10:06:53.499613+0000","last_fullsized":"2024-04-25T10:06:53.499613+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.665724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"196'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.519561+0000","last_change":"2024-04-25T10:06:52.497276+0000","last_active":"2024-04-25T10:06:53.519561+0000","last_peered":"2024-04-25T10:06:53.519561+0000","last_clean":"2024-04-25T10:06:53.519561+0000","last_became_active":"2024-04-25T10:06:52.497015+0000","last_became_peered":"2024-04-25T10:06:52.497015+0000","last_unstale":"2024-04-25T10:06:53.519561+0000","last_undegraded":"2024-04-25T10:06:53.519561+0000","last_fullsized":"2024-04-25T10:06:53.519561+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:23.427618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508725+0000","last_change":"2024-04-25T10:06:53.501853+0000","last_active":"2024-04-25T10:06:53.508725+0000","last_peered":"2024-04-25T10:06:53.508725+0000","last_clean":"2024-04-25T10:06:53.508725+0000","last_became_active":"2024-04-25T10:06:53.501391+0000","last_became_peered":"2024-04-25T10:06:53.501391+0000","last_unstale":"2024-04-25T10:06:53.508725+0000","last_undegraded":"2024-04-25T10:06:53.508725+0000","last_fullsized":"2024-04-25T10:06:53.508725+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:22.628297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.520845+0000","last_change":"2024-04-25T10:06:52.491587+0000","last_active":"2024-04-25T10:06:53.520845+0000","last_peered":"2024-04-25T10:06:53.520845+0000","last_clean":"2024-04-25T10:06:53.520845+0000","last_became_active":"2024-04-25T10:06:52.491321+0000","last_became_peered":"2024-04-25T10:06:52.491321+0000","last_unstale":"2024-04-25T10:06:53.520845+0000","last_undegraded":"2024-04-25T10:06:53.520845+0000","last_fullsized":"2024-04-25T10:06:53.520845+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:53.861228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507126+0000","last_change":"2024-04-25T10:06:53.500731+0000","last_active":"2024-04-25T10:06:53.507126+0000","last_peered":"2024-04-25T10:06:53.507126+0000","last_clean":"2024-04-25T10:06:53.507126+0000","last_became_active":"2024-04-25T10:06:53.500462+0000","last_became_peered":"2024-04-25T10:06:53.500462+0000","last_unstale":"2024-04-25T10:06:53.507126+0000","last_undegraded":"2024-04-25T10:06:53.507126+0000","last_fullsized":"2024-04-25T10:06:53.507126+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:20.503651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"196'1","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509466+0000","last_change":"2024-04-25T10:06:52.491625+0000","last_active":"2024-04-25T10:06:54.509466+0000","last_peered":"2024-04-25T10:06:54.509466+0000","last_clean":"2024-04-25T10:06:54.509466+0000","last_became_active":"2024-04-25T10:06:52.491360+0000","last_became_peered":"2024-04-25T10:06:52.491360+0000","last_unstale":"2024-04-25T10:06:54.509466+0000","last_undegraded":"2024-04-25T10:06:54.509466+0000","last_fullsized":"2024-04-25T10:06:54.509466+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:15.607411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508749+0000","last_change":"2024-04-25T10:06:53.501691+0000","last_active":"2024-04-25T10:06:53.508749+0000","last_peered":"2024-04-25T10:06:53.508749+0000","last_clean":"2024-04-25T10:06:53.508749+0000","last_became_active":"2024-04-25T10:06:53.501427+0000","last_became_peered":"2024-04-25T10:06:53.501427+0000","last_unstale":"2024-04-25T10:06:53.508749+0000","last_undegraded":"2024-04-25T10:06:53.508749+0000","last_fullsized":"2024-04-25T10:06:53.508749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:16.909645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"196'1","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492651+0000","last_change":"2024-04-25T10:06:52.499853+0000","last_active":"2024-04-25T10:06:54.492651+0000","last_peered":"2024-04-25T10:06:54.492651+0000","last_clean":"2024-04-25T10:06:54.492651+0000","last_became_active":"2024-04-25T10:06:52.499538+0000","last_became_peered":"2024-04-25T10:06:52.499538+0000","last_unstale":"2024-04-25T10:06:54.492651+0000","last_undegraded":"2024-04-25T10:06:54.492651+0000","last_fullsized":"2024-04-25T10:06:54.492651+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:53.861228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509451+0000","last_change":"2024-04-25T10:06:53.501005+0000","last_active":"2024-04-25T10:06:54.509451+0000","last_peered":"2024-04-25T10:06:54.509451+0000","last_clean":"2024-04-25T10:06:54.509451+0000","last_became_active":"2024-04-25T10:06:53.500146+0000","last_became_peered":"2024-04-25T10:06:53.500146+0000","last_unstale":"2024-04-25T10:06:54.509451+0000","last_undegraded":"2024-04-25T10:06:54.509451+0000","last_fullsized":"2024-04-25T10:06:54.509451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:50.958579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492138+0000","last_change":"2024-04-25T10:06:52.492189+0000","last_active":"2024-04-25T10:06:54.492138+0000","last_peered":"2024-04-25T10:06:54.492138+0000","last_clean":"2024-04-25T10:06:54.492138+0000","last_became_active":"2024-04-25T10:06:52.491551+0000","last_became_peered":"2024-04-25T10:06:52.491551+0000","last_unstale":"2024-04-25T10:06:54.492138+0000","last_undegraded":"2024-04-25T10:06:54.492138+0000","last_fullsized":"2024-04-25T10:06:54.492138+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:09.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829110+0000","last_change":"2024-04-25T10:06:53.499516+0000","last_active":"2024-04-25T10:06:54.829110+0000","last_peered":"2024-04-25T10:06:54.829110+0000","last_clean":"2024-04-25T10:06:54.829110+0000","last_became_active":"2024-04-25T10:06:53.499277+0000","last_became_peered":"2024-04-25T10:06:53.499277+0000","last_unstale":"2024-04-25T10:06:54.829110+0000","last_undegraded":"2024-04-25T10:06:54.829110+0000","last_fullsized":"2024-04-25T10:06:54.829110+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:39.895005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.487705+0000","last_change":"2024-04-25T10:06:52.494250+0000","last_active":"2024-04-25T10:06:53.487705+0000","last_peered":"2024-04-25T10:06:53.487705+0000","last_clean":"2024-04-25T10:06:53.487705+0000","last_became_active":"2024-04-25T10:06:52.493981+0000","last_became_peered":"2024-04-25T10:06:52.493981+0000","last_unstale":"2024-04-25T10:06:53.487705+0000","last_undegraded":"2024-04-25T10:06:53.487705+0000","last_fullsized":"2024-04-25T10:06:53.487705+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:09.565197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509554+0000","last_change":"2024-04-25T10:06:53.495742+0000","last_active":"2024-04-25T10:06:54.509554+0000","last_peered":"2024-04-25T10:06:54.509554+0000","last_clean":"2024-04-25T10:06:54.509554+0000","last_became_active":"2024-04-25T10:06:53.494938+0000","last_became_peered":"2024-04-25T10:06:53.494938+0000","last_unstale":"2024-04-25T10:06:54.509554+0000","last_undegraded":"2024-04-25T10:06:54.509554+0000","last_fullsized":"2024-04-25T10:06:54.509554+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:43.078800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493124+0000","last_change":"2024-04-25T10:06:52.495657+0000","last_active":"2024-04-25T10:06:54.493124+0000","last_peered":"2024-04-25T10:06:54.493124+0000","last_clean":"2024-04-25T10:06:54.493124+0000","last_became_active":"2024-04-25T10:06:52.495378+0000","last_became_peered":"2024-04-25T10:06:52.495378+0000","last_unstale":"2024-04-25T10:06:54.493124+0000","last_undegraded":"2024-04-25T10:06:54.493124+0000","last_fullsized":"2024-04-25T10:06:54.493124+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:02.454534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509705+0000","last_change":"2024-04-25T10:06:53.503283+0000","last_active":"2024-04-25T10:06:54.509705+0000","last_peered":"2024-04-25T10:06:54.509705+0000","last_clean":"2024-04-25T10:06:54.509705+0000","last_became_active":"2024-04-25T10:06:53.501066+0000","last_became_peered":"2024-04-25T10:06:53.501066+0000","last_unstale":"2024-04-25T10:06:54.509705+0000","last_undegraded":"2024-04-25T10:06:54.509705+0000","last_fullsized":"2024-04-25T10:06:54.509705+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:06.707170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491984+0000","last_change":"2024-04-25T10:06:52.491968+0000","last_active":"2024-04-25T10:06:54.491984+0000","last_peered":"2024-04-25T10:06:54.491984+0000","last_clean":"2024-04-25T10:06:54.491984+0000","last_became_active":"2024-04-25T10:06:52.491578+0000","last_became_peered":"2024-04-25T10:06:52.491578+0000","last_unstale":"2024-04-25T10:06:54.491984+0000","last_undegraded":"2024-04-25T10:06:54.491984+0000","last_fullsized":"2024-04-25T10:06:54.491984+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:12.239761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507480+0000","last_change":"2024-04-25T10:06:53.502547+0000","last_active":"2024-04-25T10:06:53.507480+0000","last_peered":"2024-04-25T10:06:53.507480+0000","last_clean":"2024-04-25T10:06:53.507480+0000","last_became_active":"2024-04-25T10:06:53.500862+0000","last_became_peered":"2024-04-25T10:06:53.500862+0000","last_unstale":"2024-04-25T10:06:53.507480+0000","last_undegraded":"2024-04-25T10:06:53.507480+0000","last_fullsized":"2024-04-25T10:06:53.507480+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:40.586964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"196'3","reported_seq":17,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493125+0000","last_change":"2024-04-25T10:06:52.495765+0000","last_active":"2024-04-25T10:06:54.493125+0000","last_peered":"2024-04-25T10:06:54.493125+0000","last_clean":"2024-04-25T10:06:54.493125+0000","last_became_active":"2024-04-25T10:06:52.495507+0000","last_became_peered":"2024-04-25T10:06:52.495507+0000","last_unstale":"2024-04-25T10:06:54.493125+0000","last_undegraded":"2024-04-25T10:06:54.493125+0000","last_fullsized":"2024-04-25T10:06:54.493125+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:00.764790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491798+0000","last_change":"2024-04-25T10:06:53.501388+0000","last_active":"2024-04-25T10:06:54.491798+0000","last_peered":"2024-04-25T10:06:54.491798+0000","last_clean":"2024-04-25T10:06:54.491798+0000","last_became_active":"2024-04-25T10:06:53.501102+0000","last_became_peered":"2024-04-25T10:06:53.501102+0000","last_unstale":"2024-04-25T10:06:54.491798+0000","last_undegraded":"2024-04-25T10:06:54.491798+0000","last_fullsized":"2024-04-25T10:06:54.491798+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:47.109377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"196'6","reported_seq":18,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.523123+0000","last_change":"2024-04-25T10:06:52.488704+0000","last_active":"2024-04-25T10:06:53.523123+0000","last_peered":"2024-04-25T10:06:53.523123+0000","last_clean":"2024-04-25T10:06:53.523123+0000","last_became_active":"2024-04-25T10:06:52.488416+0000","last_became_peered":"2024-04-25T10:06:52.488416+0000","last_unstale":"2024-04-25T10:06:53.523123+0000","last_undegraded":"2024-04-25T10:06:53.523123+0000","last_fullsized":"2024-04-25T10:06:53.523123+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:41.686476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506245+0000","last_change":"2024-04-25T10:06:53.496941+0000","last_active":"2024-04-25T10:06:54.506245+0000","last_peered":"2024-04-25T10:06:54.506245+0000","last_clean":"2024-04-25T10:06:54.506245+0000","last_became_active":"2024-04-25T10:06:53.496686+0000","last_became_peered":"2024-04-25T10:06:53.496686+0000","last_unstale":"2024-04-25T10:06:54.506245+0000","last_undegraded":"2024-04-25T10:06:54.506245+0000","last_fullsized":"2024-04-25T10:06:54.506245+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.665724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.519959+0000","last_change":"2024-04-25T10:06:52.499728+0000","last_active":"2024-04-25T10:06:53.519959+0000","last_peered":"2024-04-25T10:06:53.519959+0000","last_clean":"2024-04-25T10:06:53.519959+0000","last_became_active":"2024-04-25T10:06:52.499452+0000","last_became_peered":"2024-04-25T10:06:52.499452+0000","last_unstale":"2024-04-25T10:06:53.519959+0000","last_undegraded":"2024-04-25T10:06:53.519959+0000","last_fullsized":"2024-04-25T10:06:53.519959+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:16.435934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.509809+0000","last_change":"2024-04-25T10:06:53.506139+0000","last_active":"2024-04-25T10:06:53.509809+0000","last_peered":"2024-04-25T10:06:53.509809+0000","last_clean":"2024-04-25T10:06:53.509809+0000","last_became_active":"2024-04-25T10:06:53.505836+0000","last_became_peered":"2024-04-25T10:06:53.505836+0000","last_unstale":"2024-04-25T10:06:53.509809+0000","last_undegraded":"2024-04-25T10:06:53.509809+0000","last_fullsized":"2024-04-25T10:06:53.509809+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:01.904113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506346+0000","last_change":"2024-04-25T10:06:52.492663+0000","last_active":"2024-04-25T10:06:54.506346+0000","last_peered":"2024-04-25T10:06:54.506346+0000","last_clean":"2024-04-25T10:06:54.506346+0000","last_became_active":"2024-04-25T10:06:52.492389+0000","last_became_peered":"2024-04-25T10:06:52.492389+0000","last_unstale":"2024-04-25T10:06:54.506346+0000","last_undegraded":"2024-04-25T10:06:54.506346+0000","last_fullsized":"2024-04-25T10:06:54.506346+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.074888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.502367+0000","last_change":"2024-04-25T10:06:53.498504+0000","last_active":"2024-04-25T10:06:53.502367+0000","last_peered":"2024-04-25T10:06:53.502367+0000","last_clean":"2024-04-25T10:06:53.502367+0000","last_became_active":"2024-04-25T10:06:53.498211+0000","last_became_peered":"2024-04-25T10:06:53.498211+0000","last_unstale":"2024-04-25T10:06:53.502367+0000","last_undegraded":"2024-04-25T10:06:53.502367+0000","last_fullsized":"2024-04-25T10:06:53.502367+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:19.611795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.494562+0000","last_change":"2024-04-25T10:06:52.496638+0000","last_active":"2024-04-25T10:06:53.494562+0000","last_peered":"2024-04-25T10:06:53.494562+0000","last_clean":"2024-04-25T10:06:53.494562+0000","last_became_active":"2024-04-25T10:06:52.496343+0000","last_became_peered":"2024-04-25T10:06:52.496343+0000","last_unstale":"2024-04-25T10:06:53.494562+0000","last_undegraded":"2024-04-25T10:06:53.494562+0000","last_fullsized":"2024-04-25T10:06:53.494562+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:33.232420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509443+0000","last_change":"2024-04-25T10:06:53.497026+0000","last_active":"2024-04-25T10:06:54.509443+0000","last_peered":"2024-04-25T10:06:54.509443+0000","last_clean":"2024-04-25T10:06:54.509443+0000","last_became_active":"2024-04-25T10:06:53.495116+0000","last_became_peered":"2024-04-25T10:06:53.495116+0000","last_unstale":"2024-04-25T10:06:54.509443+0000","last_undegraded":"2024-04-25T10:06:54.509443+0000","last_fullsized":"2024-04-25T10:06:54.509443+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:35.905488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"196'8","reported_seq":20,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.521555+0000","last_change":"2024-04-25T10:06:52.494696+0000","last_active":"2024-04-25T10:06:53.521555+0000","last_peered":"2024-04-25T10:06:53.521555+0000","last_clean":"2024-04-25T10:06:53.521555+0000","last_became_active":"2024-04-25T10:06:52.494372+0000","last_became_peered":"2024-04-25T10:06:52.494372+0000","last_unstale":"2024-04-25T10:06:53.521555+0000","last_undegraded":"2024-04-25T10:06:53.521555+0000","last_fullsized":"2024-04-25T10:06:53.521555+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:29.016504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506315+0000","last_change":"2024-04-25T10:06:53.506355+0000","last_active":"2024-04-25T10:06:54.506315+0000","last_peered":"2024-04-25T10:06:54.506315+0000","last_clean":"2024-04-25T10:06:54.506315+0000","last_became_active":"2024-04-25T10:06:53.506121+0000","last_became_peered":"2024-04-25T10:06:53.506121+0000","last_unstale":"2024-04-25T10:06:54.506315+0000","last_undegraded":"2024-04-25T10:06:54.506315+0000","last_fullsized":"2024-04-25T10:06:54.506315+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:19.611795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"196'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.520292+0000","last_change":"2024-04-25T10:06:52.494770+0000","last_active":"2024-04-25T10:06:53.520292+0000","last_peered":"2024-04-25T10:06:53.520292+0000","last_clean":"2024-04-25T10:06:53.520292+0000","last_became_active":"2024-04-25T10:06:52.494415+0000","last_became_peered":"2024-04-25T10:06:52.494415+0000","last_unstale":"2024-04-25T10:06:53.520292+0000","last_undegraded":"2024-04-25T10:06:53.520292+0000","last_fullsized":"2024-04-25T10:06:53.520292+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:05.703258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493020+0000","last_change":"2024-04-25T10:06:53.501021+0000","last_active":"2024-04-25T10:06:54.493020+0000","last_peered":"2024-04-25T10:06:54.493020+0000","last_clean":"2024-04-25T10:06:54.493020+0000","last_became_active":"2024-04-25T10:06:53.500343+0000","last_became_peered":"2024-04-25T10:06:53.500343+0000","last_unstale":"2024-04-25T10:06:54.493020+0000","last_undegraded":"2024-04-25T10:06:54.493020+0000","last_fullsized":"2024-04-25T10:06:54.493020+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:14.600607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"196'2","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829126+0000","last_change":"2024-04-25T10:06:52.494499+0000","last_active":"2024-04-25T10:06:54.829126+0000","last_peered":"2024-04-25T10:06:54.829126+0000","last_clean":"2024-04-25T10:06:54.829126+0000","last_became_active":"2024-04-25T10:06:52.494253+0000","last_became_peered":"2024-04-25T10:06:52.494253+0000","last_unstale":"2024-04-25T10:06:54.829126+0000","last_undegraded":"2024-04-25T10:06:54.829126+0000","last_fullsized":"2024-04-25T10:06:54.829126+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:31.728097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.506760+0000","last_change":"2024-04-25T10:06:53.500438+0000","last_active":"2024-04-25T10:06:53.506760+0000","last_peered":"2024-04-25T10:06:53.506760+0000","last_clean":"2024-04-25T10:06:53.506760+0000","last_became_active":"2024-04-25T10:06:53.499757+0000","last_became_peered":"2024-04-25T10:06:53.499757+0000","last_unstale":"2024-04-25T10:06:53.506760+0000","last_undegraded":"2024-04-25T10:06:53.506760+0000","last_fullsized":"2024-04-25T10:06:53.506760+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:36.280124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"196'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.519810+0000","last_change":"2024-04-25T10:06:52.498806+0000","last_active":"2024-04-25T10:06:53.519810+0000","last_peered":"2024-04-25T10:06:53.519810+0000","last_clean":"2024-04-25T10:06:53.519810+0000","last_became_active":"2024-04-25T10:06:52.498522+0000","last_became_peered":"2024-04-25T10:06:52.498522+0000","last_unstale":"2024-04-25T10:06:53.519810+0000","last_undegraded":"2024-04-25T10:06:53.519810+0000","last_fullsized":"2024-04-25T10:06:53.519810+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:22.661441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517238+0000","last_change":"2024-04-25T10:06:53.512212+0000","last_active":"2024-04-25T10:06:53.517238+0000","last_peered":"2024-04-25T10:06:53.517238+0000","last_clean":"2024-04-25T10:06:53.517238+0000","last_became_active":"2024-04-25T10:06:53.511949+0000","last_became_peered":"2024-04-25T10:06:53.511949+0000","last_unstale":"2024-04-25T10:06:53.517238+0000","last_undegraded":"2024-04-25T10:06:53.517238+0000","last_fullsized":"2024-04-25T10:06:53.517238+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:39.177925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.496513+0000","last_change":"2024-04-25T10:06:52.493475+0000","last_active":"2024-04-25T10:06:53.496513+0000","last_peered":"2024-04-25T10:06:53.496513+0000","last_clean":"2024-04-25T10:06:53.496513+0000","last_became_active":"2024-04-25T10:06:52.493188+0000","last_became_peered":"2024-04-25T10:06:52.493188+0000","last_unstale":"2024-04-25T10:06:53.496513+0000","last_undegraded":"2024-04-25T10:06:53.496513+0000","last_fullsized":"2024-04-25T10:06:53.496513+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:58.375265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517460+0000","last_change":"2024-04-25T10:06:53.509350+0000","last_active":"2024-04-25T10:06:53.517460+0000","last_peered":"2024-04-25T10:06:53.517460+0000","last_clean":"2024-04-25T10:06:53.517460+0000","last_became_active":"2024-04-25T10:06:53.508930+0000","last_became_peered":"2024-04-25T10:06:53.508930+0000","last_unstale":"2024-04-25T10:06:53.517460+0000","last_undegraded":"2024-04-25T10:06:53.517460+0000","last_fullsized":"2024-04-25T10:06:53.517460+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:55.803648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.521951+0000","last_change":"2024-04-25T10:06:52.499604+0000","last_active":"2024-04-25T10:06:53.521951+0000","last_peered":"2024-04-25T10:06:53.521951+0000","last_clean":"2024-04-25T10:06:53.521951+0000","last_became_active":"2024-04-25T10:06:52.498958+0000","last_became_peered":"2024-04-25T10:06:52.498958+0000","last_unstale":"2024-04-25T10:06:53.521951+0000","last_undegraded":"2024-04-25T10:06:53.521951+0000","last_fullsized":"2024-04-25T10:06:53.521951+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:33.782562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492758+0000","last_change":"2024-04-25T10:06:53.496726+0000","last_active":"2024-04-25T10:06:54.492758+0000","last_peered":"2024-04-25T10:06:54.492758+0000","last_clean":"2024-04-25T10:06:54.492758+0000","last_became_active":"2024-04-25T10:06:53.496444+0000","last_became_peered":"2024-04-25T10:06:53.496444+0000","last_unstale":"2024-04-25T10:06:54.492758+0000","last_undegraded":"2024-04-25T10:06:54.492758+0000","last_fullsized":"2024-04-25T10:06:54.492758+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:11.245105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509517+0000","last_change":"2024-04-25T10:06:52.491358+0000","last_active":"2024-04-25T10:06:54.509517+0000","last_peered":"2024-04-25T10:06:54.509517+0000","last_clean":"2024-04-25T10:06:54.509517+0000","last_became_active":"2024-04-25T10:06:52.491003+0000","last_became_peered":"2024-04-25T10:06:52.491003+0000","last_unstale":"2024-04-25T10:06:54.509517+0000","last_undegraded":"2024-04-25T10:06:54.509517+0000","last_fullsized":"2024-04-25T10:06:54.509517+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:24.910473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506197+0000","last_change":"2024-04-25T10:06:53.504813+0000","last_active":"2024-04-25T10:06:54.506197+0000","last_peered":"2024-04-25T10:06:54.506197+0000","last_clean":"2024-04-25T10:06:54.506197+0000","last_became_active":"2024-04-25T10:06:53.504200+0000","last_became_peered":"2024-04-25T10:06:53.504200+0000","last_unstale":"2024-04-25T10:06:54.506197+0000","last_undegraded":"2024-04-25T10:06:54.506197+0000","last_fullsized":"2024-04-25T10:06:54.506197+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:05.280342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509388+0000","last_change":"2024-04-25T10:06:52.493447+0000","last_active":"2024-04-25T10:06:54.509388+0000","last_peered":"2024-04-25T10:06:54.509388+0000","last_clean":"2024-04-25T10:06:54.509388+0000","last_became_active":"2024-04-25T10:06:52.493199+0000","last_became_peered":"2024-04-25T10:06:52.493199+0000","last_unstale":"2024-04-25T10:06:54.509388+0000","last_undegraded":"2024-04-25T10:06:54.509388+0000","last_fullsized":"2024-04-25T10:06:54.509388+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:10.241193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507612+0000","last_change":"2024-04-25T10:06:53.505356+0000","last_active":"2024-04-25T10:06:53.507612+0000","last_peered":"2024-04-25T10:06:53.507612+0000","last_clean":"2024-04-25T10:06:53.507612+0000","last_became_active":"2024-04-25T10:06:53.505087+0000","last_became_peered":"2024-04-25T10:06:53.505087+0000","last_unstale":"2024-04-25T10:06:53.507612+0000","last_undegraded":"2024-04-25T10:06:53.507612+0000","last_fullsized":"2024-04-25T10:06:53.507612+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:43.873052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828826+0000","last_change":"2024-04-25T10:06:52.494978+0000","last_active":"2024-04-25T10:06:54.828826+0000","last_peered":"2024-04-25T10:06:54.828826+0000","last_clean":"2024-04-25T10:06:54.828826+0000","last_became_active":"2024-04-25T10:06:52.494629+0000","last_became_peered":"2024-04-25T10:06:52.494629+0000","last_unstale":"2024-04-25T10:06:54.828826+0000","last_undegraded":"2024-04-25T10:06:54.828826+0000","last_fullsized":"2024-04-25T10:06:54.828826+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:51.457590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.506427+0000","last_change":"2024-04-25T10:06:53.498504+0000","last_active":"2024-04-25T10:06:54.506427+0000","last_peered":"2024-04-25T10:06:54.506427+0000","last_clean":"2024-04-25T10:06:54.506427+0000","last_became_active":"2024-04-25T10:06:53.498254+0000","last_became_peered":"2024-04-25T10:06:53.498254+0000","last_unstale":"2024-04-25T10:06:54.506427+0000","last_undegraded":"2024-04-25T10:06:54.506427+0000","last_fullsized":"2024-04-25T10:06:54.506427+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:40.586964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"196'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.517402+0000","last_change":"2024-04-25T10:06:52.494622+0000","last_active":"2024-04-25T10:06:53.517402+0000","last_peered":"2024-04-25T10:06:53.517402+0000","last_clean":"2024-04-25T10:06:53.517402+0000","last_became_active":"2024-04-25T10:06:52.494175+0000","last_became_peered":"2024-04-25T10:06:52.494175+0000","last_unstale":"2024-04-25T10:06:53.517402+0000","last_undegraded":"2024-04-25T10:06:53.517402+0000","last_fullsized":"2024-04-25T10:06:53.517402+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:09.520990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.493093+0000","last_change":"2024-04-25T10:06:53.500806+0000","last_active":"2024-04-25T10:06:54.493093+0000","last_peered":"2024-04-25T10:06:54.493093+0000","last_clean":"2024-04-25T10:06:54.493093+0000","last_became_active":"2024-04-25T10:06:53.500064+0000","last_became_peered":"2024-04-25T10:06:53.500064+0000","last_unstale":"2024-04-25T10:06:54.493093+0000","last_undegraded":"2024-04-25T10:06:54.493093+0000","last_fullsized":"2024-04-25T10:06:54.493093+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:48.833595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"197'1","reported_seq":15,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.510687+0000","last_change":"2024-04-25T10:06:52.492874+0000","last_active":"2024-04-25T10:06:54.510687+0000","last_peered":"2024-04-25T10:06:54.510687+0000","last_clean":"2024-04-25T10:06:54.510687+0000","last_became_active":"2024-04-25T10:06:52.492399+0000","last_became_peered":"2024-04-25T10:06:52.492399+0000","last_unstale":"2024-04-25T10:06:54.510687+0000","last_undegraded":"2024-04-25T10:06:54.510687+0000","last_fullsized":"2024-04-25T10:06:54.510687+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:54.757751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.508612+0000","last_change":"2024-04-25T10:06:53.504949+0000","last_active":"2024-04-25T10:06:53.508612+0000","last_peered":"2024-04-25T10:06:53.508612+0000","last_clean":"2024-04-25T10:06:53.508612+0000","last_became_active":"2024-04-25T10:06:53.504677+0000","last_became_peered":"2024-04-25T10:06:53.504677+0000","last_unstale":"2024-04-25T10:06:53.508612+0000","last_undegraded":"2024-04-25T10:06:53.508612+0000","last_fullsized":"2024-04-25T10:06:53.508612+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:13.450784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.509682+0000","last_change":"2024-04-25T10:06:52.489251+0000","last_active":"2024-04-25T10:06:54.509682+0000","last_peered":"2024-04-25T10:06:54.509682+0000","last_clean":"2024-04-25T10:06:54.509682+0000","last_became_active":"2024-04-25T10:06:52.488980+0000","last_became_peered":"2024-04-25T10:06:52.488980+0000","last_unstale":"2024-04-25T10:06:54.509682+0000","last_undegraded":"2024-04-25T10:06:54.509682+0000","last_fullsized":"2024-04-25T10:06:54.509682+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:27.707912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.829213+0000","last_change":"2024-04-25T10:06:53.505826+0000","last_active":"2024-04-25T10:06:54.829213+0000","last_peered":"2024-04-25T10:06:54.829213+0000","last_clean":"2024-04-25T10:06:54.829213+0000","last_became_active":"2024-04-25T10:06:53.505204+0000","last_became_peered":"2024-04-25T10:06:53.505204+0000","last_unstale":"2024-04-25T10:06:54.829213+0000","last_undegraded":"2024-04-25T10:06:54.829213+0000","last_fullsized":"2024-04-25T10:06:54.829213+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:24.794175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.496542+0000","last_change":"2024-04-25T10:06:52.495275+0000","last_active":"2024-04-25T10:06:53.496542+0000","last_peered":"2024-04-25T10:06:53.496542+0000","last_clean":"2024-04-25T10:06:53.496542+0000","last_became_active":"2024-04-25T10:06:52.495016+0000","last_became_peered":"2024-04-25T10:06:52.495016+0000","last_unstale":"2024-04-25T10:06:53.496542+0000","last_undegraded":"2024-04-25T10:06:53.496542+0000","last_fullsized":"2024-04-25T10:06:53.496542+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:12.239761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507856+0000","last_change":"2024-04-25T10:06:53.505732+0000","last_active":"2024-04-25T10:06:53.507856+0000","last_peered":"2024-04-25T10:06:53.507856+0000","last_clean":"2024-04-25T10:06:53.507856+0000","last_became_active":"2024-04-25T10:06:53.505352+0000","last_became_peered":"2024-04-25T10:06:53.505352+0000","last_unstale":"2024-04-25T10:06:53.507856+0000","last_undegraded":"2024-04-25T10:06:53.507856+0000","last_fullsized":"2024-04-25T10:06:53.507856+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:08.542039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.500374+0000","last_change":"2024-04-25T10:06:52.499384+0000","last_active":"2024-04-25T10:06:53.500374+0000","last_peered":"2024-04-25T10:06:53.500374+0000","last_clean":"2024-04-25T10:06:53.500374+0000","last_became_active":"2024-04-25T10:06:52.498945+0000","last_became_peered":"2024-04-25T10:06:52.498945+0000","last_unstale":"2024-04-25T10:06:53.500374+0000","last_undegraded":"2024-04-25T10:06:53.500374+0000","last_fullsized":"2024-04-25T10:06:53.500374+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:12.651714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.493005+0000","last_change":"2024-04-25T10:06:53.491642+0000","last_active":"2024-04-25T10:06:53.493005+0000","last_peered":"2024-04-25T10:06:53.493005+0000","last_clean":"2024-04-25T10:06:53.493005+0000","last_became_active":"2024-04-25T10:06:53.491382+0000","last_became_peered":"2024-04-25T10:06:53.491382+0000","last_unstale":"2024-04-25T10:06:53.493005+0000","last_undegraded":"2024-04-25T10:06:53.493005+0000","last_fullsized":"2024-04-25T10:06:53.493005+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:18.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.491143+0000","last_change":"2024-04-25T10:06:52.491582+0000","last_active":"2024-04-25T10:06:54.491143+0000","last_peered":"2024-04-25T10:06:54.491143+0000","last_clean":"2024-04-25T10:06:54.491143+0000","last_became_active":"2024-04-25T10:06:52.491314+0000","last_became_peered":"2024-04-25T10:06:52.491314+0000","last_unstale":"2024-04-25T10:06:54.491143+0000","last_undegraded":"2024-04-25T10:06:54.491143+0000","last_fullsized":"2024-04-25T10:06:54.491143+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:29.016504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.828877+0000","last_change":"2024-04-25T10:06:53.506470+0000","last_active":"2024-04-25T10:06:54.828877+0000","last_peered":"2024-04-25T10:06:54.828877+0000","last_clean":"2024-04-25T10:06:54.828877+0000","last_became_active":"2024-04-25T10:06:53.506224+0000","last_became_peered":"2024-04-25T10:06:53.506224+0000","last_unstale":"2024-04-25T10:06:54.828877+0000","last_undegraded":"2024-04-25T10:06:54.828877+0000","last_fullsized":"2024-04-25T10:06:54.828877+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:41.699855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492853+0000","last_change":"2024-04-25T10:06:52.497632+0000","last_active":"2024-04-25T10:06:54.492853+0000","last_peered":"2024-04-25T10:06:54.492853+0000","last_clean":"2024-04-25T10:06:54.492853+0000","last_became_active":"2024-04-25T10:06:52.497343+0000","last_became_peered":"2024-04-25T10:06:52.497343+0000","last_unstale":"2024-04-25T10:06:54.492853+0000","last_undegraded":"2024-04-25T10:06:54.492853+0000","last_fullsized":"2024-04-25T10:06:54.492853+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:41.686476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.515043+0000","last_change":"2024-04-25T10:06:53.508937+0000","last_active":"2024-04-25T10:06:53.515043+0000","last_peered":"2024-04-25T10:06:53.515043+0000","last_clean":"2024-04-25T10:06:53.515043+0000","last_became_active":"2024-04-25T10:06:53.508628+0000","last_became_peered":"2024-04-25T10:06:53.508628+0000","last_unstale":"2024-04-25T10:06:53.515043+0000","last_undegraded":"2024-04-25T10:06:53.515043+0000","last_fullsized":"2024-04-25T10:06:53.515043+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:26.922488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.497479+0000","last_change":"2024-04-25T10:06:52.495755+0000","last_active":"2024-04-25T10:06:53.497479+0000","last_peered":"2024-04-25T10:06:53.497479+0000","last_clean":"2024-04-25T10:06:53.497479+0000","last_became_active":"2024-04-25T10:06:52.495452+0000","last_became_peered":"2024-04-25T10:06:52.495452+0000","last_unstale":"2024-04-25T10:06:53.497479+0000","last_undegraded":"2024-04-25T10:06:53.497479+0000","last_fullsized":"2024-04-25T10:06:53.497479+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:01.464001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-04-25T10:06:53.507640+0000","last_change":"2024-04-25T10:06:53.505744+0000","last_active":"2024-04-25T10:06:53.507640+0000","last_peered":"2024-04-25T10:06:53.507640+0000","last_clean":"2024-04-25T10:06:53.507640+0000","last_became_active":"2024-04-25T10:06:53.504889+0000","last_became_peered":"2024-04-25T10:06:53.504889+0000","last_unstale":"2024-04-25T10:06:53.507640+0000","last_undegraded":"2024-04-25T10:06:53.507640+0000","last_fullsized":"2024-04-25T10:06:53.507640+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:52.475595+0000","last_clean_scrub_stamp":"2024-04-25T10:06:52.475595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:03.371549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"196'2","reported_seq":16,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492845+0000","last_change":"2024-04-25T10:06:52.498191+0000","last_active":"2024-04-25T10:06:54.492845+0000","last_peered":"2024-04-25T10:06:54.492845+0000","last_clean":"2024-04-25T10:06:54.492845+0000","last_became_active":"2024-04-25T10:06:52.497933+0000","last_became_peered":"2024-04-25T10:06:52.497933+0000","last_unstale":"2024-04-25T10:06:54.492845+0000","last_undegraded":"2024-04-25T10:06:54.492845+0000","last_fullsized":"2024-04-25T10:06:54.492845+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:51.471683+0000","last_clean_scrub_stamp":"2024-04-25T10:06:51.471683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:04.422178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":496,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-04-25T10:06:54.492734+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:06:54.492734+0000","last_peered":"2024-04-25T10:06:54.492734+0000","last_clean":"2024-04-25T10:06:54.492734+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:06:54.492734+0000","last_undegraded":"2024-04-25T10:06:54.492734+0000","last_fullsized":"2024-04-25T10:06:54.492734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52248,"kb_used_data":3020,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319592,"statfs":{"total":96636764160,"available":96583262208,"internally_reserved":0,"allocated":3092480,"data_stored":1760057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55836,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316004,"statfs":{"total":96636764160,"available":96579588096,"internally_reserved":0,"allocated":2502656,"data_stored":1164063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51672,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320168,"statfs":{"total":96636764160,"available":96583852032,"internally_reserved":0,"allocated":2494464,"data_stored":1164007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59980,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311860,"statfs":{"total":96636764160,"available":96575344640,"internally_reserved":0,"allocated":2482176,"data_stored":1162415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575005,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77964,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94293876,"statfs":{"total":96636764160,"available":96556929024,"internally_reserved":0,"allocated":2482176,"data_stored":1162347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55816,"kb_used_data":2428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316024,"statfs":{"total":96636764160,"available":96579608576,"internally_reserved":0,"allocated":2486272,"data_stored":1162941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55820,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316020,"statfs":{"total":96636764160,"available":96579604480,"internally_reserved":0,"allocated":2490368,"data_stored":1169017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56396,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315444,"statfs":{"total":96636764160,"available":96579014656,"internally_reserved":0,"allocated":3072000,"data_stored":1752693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:57.109 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:57.109 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:57.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.245+0000 7f19c7d4c640 1 Processor -- start 2024-04-25T10:06:57.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c7d4c640 1 -- start start 2024-04-25T10:06:57.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c7d4c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19c015dd00 0x7f19c0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c7d4c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00d06c0 con 0x7f19c015e0e0 2024-04-25T10:06:57.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c7d4c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c009a690 con 0x7f19c006bf00 2024-04-25T10:06:57.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c7d4c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c00d1440 con 0x7f19c015dd00 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c5ae8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f19c015dd00 0x7f19c0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c52e7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f19c015e0e0 0x7f19c0099f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51822/0 (socket says 172.21.15.148:51822) 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.246+0000 7f19c52e7640 1 -- 172.21.15.148:0/1948364501 learned_addr learned my addr 172.21.15.148:0/1948364501 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.247+0000 7f19c5ae8640 1 -- 172.21.15.148:0/1948364501 >> v1:172.21.15.188:6789/0 conn(0x7f19c006bf00 legacy=0x7f19c006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.247+0000 7f19c5ae8640 1 -- 172.21.15.148:0/1948364501 >> v1:172.21.15.148:6789/0 conn(0x7f19c015e0e0 legacy=0x7f19c0099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.247+0000 7f19c4ae6640 1 -- 172.21.15.148:0/1948364501 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387308039 0 0) 0x7f19c00d06c0 con 0x7f19c015e0e0 2024-04-25T10:06:57.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.247+0000 7f19c5ae8640 1 -- 172.21.15.148:0/1948364501 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00d0960 con 0x7f19c015dd00 2024-04-25T10:06:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.247+0000 7f19c5ae8640 1 --2- 172.21.15.148:0/1948364501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c0099a40 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009d90 tx=0x7f19b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4e0adebc210c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c4ae6640 1 -- 172.21.15.148:0/1948364501 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0007740 con 0x7f19c015dd00 2024-04-25T10:06:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c4ae6640 1 -- 172.21.15.148:0/1948364501 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b00078e0 con 0x7f19c015dd00 2024-04-25T10:06:57.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c4ae6640 1 -- 172.21.15.148:0/1948364501 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0007cb0 con 0x7f19c015dd00 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 -- 172.21.15.148:0/1948364501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 msgr2=0x7f19c0099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/1948364501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c0099a40 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009d90 tx=0x7f19b002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 -- 172.21.15.148:0/1948364501 shutdown_connections 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/1948364501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c0099a40 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 -- 172.21.15.148:0/1948364501 >> 172.21.15.148:0/1948364501 conn(0x7f19c00b3290 msgr2=0x7f19c00b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 -- 172.21.15.148:0/1948364501 shutdown_connections 2024-04-25T10:06:57.255 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.248+0000 7f19c7d4c640 1 -- 172.21.15.148:0/1948364501 wait complete. 2024-04-25T10:06:57.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.249+0000 7f19c7d4c640 1 Processor -- start 2024-04-25T10:06:57.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.249+0000 7f19c7d4c640 1 -- start start 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c7d4c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c006bf00 0x7f19c0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c7d4c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c7d4c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c015e0e0 0x7f19c00c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c7d4c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c009a950 con 0x7f19c015dd00 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c7d4c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00d0f20 con 0x7f19c006bf00 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c7d4c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00b2b40 con 0x7f19c015e0e0 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c52e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c52e7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c015ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33320/0 (socket says 172.21.15.148:33320) 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c52e7640 1 -- 172.21.15.148:0/2142841238 learned_addr learned my addr 172.21.15.148:0/2142841238 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c5ae8640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c006bf00 0x7f19c0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.250+0000 7f19c62e9640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c015e0e0 0x7f19c00c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c52e7640 1 -- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c015e0e0 msgr2=0x7f19c00c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c52e7640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c015e0e0 0x7f19c00c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c52e7640 1 -- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c006bf00 msgr2=0x7f19c0152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c52e7640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c006bf00 0x7f19c0152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c52e7640 1 -- 172.21.15.148:0/2142841238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c015f980 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c62e9640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c015e0e0 0x7f19c00c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c52e7640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c015ef70 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c4a0 tx=0x7f19b400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4018070 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b40074e0 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c015fc60 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4004a20 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.251+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c0160500 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.252+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c006fc30 con 0x7f19c015dd00 2024-04-25T10:06:57.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.252+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19b401b070 con 0x7f19c015dd00 2024-04-25T10:06:57.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.253+0000 7f19be7fc640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19ac06b4d0 0x7f19ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.253+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f19b4089670 con 0x7f19c015dd00 2024-04-25T10:06:57.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.253+0000 7f19c5ae8640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19ac06b4d0 0x7f19ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.260 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.253+0000 7f19c5ae8640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19ac06b4d0 0x7f19ac06d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f19b0034000 tx=0x7f19b002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.262 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.257+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19c006fc30 con 0x7f19c015dd00 2024-04-25T10:06:57.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.491+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f19c0073a00 con 0x7f19c015dd00 2024-04-25T10:06:57.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.493+0000 7f19be7fc640 1 -- 172.21.15.148:0/2142841238 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f19b40583f0 con 0x7f19c015dd00 2024-04-25T10:06:57.499 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-25T10:06:57.499 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQAVKCpmFYBwMBAAOCOlit3YO9bBKjuKhbCZww== 2024-04-25T10:06:57.499 INFO:teuthology.orchestra.run.smithi148.stdout: caps mds = "allow" 2024-04-25T10:06:57.499 INFO:teuthology.orchestra.run.smithi148.stdout: caps mon = "allow r" 2024-04-25T10:06:57.499 INFO:teuthology.orchestra.run.smithi148.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:57.500 INFO:teuthology.orchestra.run.smithi148.stderr:updated caps for client.0 2024-04-25T10:06:57.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19ac06b4d0 msgr2=0x7f19ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19ac06b4d0 0x7f19ac06d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f19b0034000 tx=0x7f19b002f000 comp rx=0 tx=0).stop 2024-04-25T10:06:57.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 msgr2=0x7f19c015ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c015ef70 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c4a0 tx=0x7f19b400c960 comp rx=0 tx=0).stop 2024-04-25T10:06:57.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 shutdown_connections 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f19ac06b4d0 0x7f19ac06d990 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19c015e0e0 0x7f19c00c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19c015dd00 0x7f19c015ef70 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 --2- 172.21.15.148:0/2142841238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19c006bf00 0x7f19c0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.495+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 >> 172.21.15.148:0/2142841238 conn(0x7f19c00b3290 msgr2=0x7f19c006f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.496+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 shutdown_connections 2024-04-25T10:06:57.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.496+0000 7f19c7d4c640 1 -- 172.21.15.148:0/2142841238 wait complete. 2024-04-25T10:06:57.512 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:57.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.604+0000 7fc17141f640 1 Processor -- start 2024-04-25T10:06:57.610 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.604+0000 7fc17141f640 1 -- start start 2024-04-25T10:06:57.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc17141f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc16c15dbe0 0x7fc16c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc17141f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0b92a0 0x7fc16c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc17141f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c0b97a0 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc17141f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c15dfc0 con 0x7fc16c15dbe0 2024-04-25T10:06:57.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc17141f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0d1a00 con 0x7fc16c0b92a0 2024-04-25T10:06:57.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16affd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc16c15dbe0 0x7fc16c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16a7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc16c0b2cb0 0x7fc16c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51832/0 (socket says 172.21.15.148:51832) 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16a7fc640 1 -- 172.21.15.148:0/485256827 learned_addr learned my addr 172.21.15.148:0/485256827 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16affd640 1 -- 172.21.15.148:0/485256827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0b92a0 msgr2=0x7fc16c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16affd640 1 --2- 172.21.15.148:0/485256827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0b92a0 0x7fc16c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16affd640 1 -- 172.21.15.148:0/485256827 >> v1:172.21.15.148:6789/0 conn(0x7fc16c0b2cb0 legacy=0x7fc16c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc16affd640 1 -- 172.21.15.148:0/485256827 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c0d1c50 con 0x7fc16c15dbe0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.605+0000 7fc169ffb640 1 -- 172.21.15.148:0/485256827 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647538393 0 0) 0x7fc16c0b97a0 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.606+0000 7fc16affd640 1 --2- 172.21.15.148:0/485256827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c15dbe0 0x7fc16c0d0910 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009d90 tx=0x7fc15c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685b1bcbc990de9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.606+0000 7fc169ffb640 1 -- 172.21.15.148:0/485256827 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c007740 con 0x7fc16c15dbe0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.606+0000 7fc169ffb640 1 -- 172.21.15.148:0/485256827 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c0078e0 con 0x7fc16c15dbe0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.606+0000 7fc169ffb640 1 -- 172.21.15.148:0/485256827 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c007cb0 con 0x7fc16c15dbe0 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 -- 172.21.15.148:0/485256827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c15dbe0 msgr2=0x7fc16c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 --2- 172.21.15.148:0/485256827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c15dbe0 0x7fc16c0d0910 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009d90 tx=0x7fc15c02a660 comp rx=0 tx=0).stop 2024-04-25T10:06:57.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 -- 172.21.15.148:0/485256827 shutdown_connections 2024-04-25T10:06:57.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 --2- 172.21.15.148:0/485256827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0b92a0 0x7fc16c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 --2- 172.21.15.148:0/485256827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c15dbe0 0x7fc16c0d0910 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 -- 172.21.15.148:0/485256827 >> 172.21.15.148:0/485256827 conn(0x7fc16c061a20 msgr2=0x7fc16c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 -- 172.21.15.148:0/485256827 shutdown_connections 2024-04-25T10:06:57.614 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.607+0000 7fc17141f640 1 -- 172.21.15.148:0/485256827 wait complete. 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.608+0000 7fc17141f640 1 Processor -- start 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.608+0000 7fc17141f640 1 -- start start 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 0x7fc16c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c0b92a0 0x7fc16c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c15dbe0 0x7fc16c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b6ab0 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0d2210 con 0x7fc16c0b92a0 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0b6c30 con 0x7fc16c15dbe0 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 0x7fc16c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 0x7fc16c0cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33336/0 (socket says 172.21.15.148:33336) 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 -- 172.21.15.148:0/2866134524 learned_addr learned my addr 172.21.15.148:0/2866134524 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:57.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16b7fe640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c15dbe0 0x7fc16c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16a7fc640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c0b92a0 0x7fc16c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 -- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c15dbe0 msgr2=0x7fc16c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c15dbe0 0x7fc16c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 -- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c0b92a0 msgr2=0x7fc16c1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c0b92a0 0x7fc16c1f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 -- 172.21.15.148:0/2866134524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c152300 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16b7fe640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c15dbe0 0x7fc16c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16a7fc640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c0b92a0 0x7fc16c1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc16affd640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 0x7fc16c0cc3c0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009d90 tx=0x7fc15c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c040070 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c031c40 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c151d70 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.609+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c03c420 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.610+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0b4c80 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.611+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c073990 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.611+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc15c039040 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.611+0000 7fc1537fe640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc15806b4d0 0x7fc15806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.611+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc15c0ab9a0 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.611+0000 7fc16a7fc640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc15806b4d0 0x7fc15806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.619 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.612+0000 7fc16a7fc640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc15806b4d0 0x7fc15806d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0ca4d0 tx=0x7fc160009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.621 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.615+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc15c07a690 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.821+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc16c152500 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.823+0000 7fc1537fe640 1 -- 172.21.15.148:0/2866134524 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4965 (secure 0 0 0) 0x7fc15c079f90 con 0x7fc16c0b2cb0 2024-04-25T10:06:57.829 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:57.829 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:06:55.145145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11634":{"gid":11634,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1255014956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1255014956},{"type":"v1","addr":"172.21.15.188:6835","nonce":1255014956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:06:57.829 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 113 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc15806b4d0 msgr2=0x7fc15806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc15806b4d0 0x7fc15806d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0ca4d0 tx=0x7fc160009040 comp rx=0 tx=0).stop 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 msgr2=0x7fc16c0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 0x7fc16c0cc3c0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009d90 tx=0x7fc15c004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 shutdown_connections 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc15806b4d0 0x7fc15806d990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c15dbe0 0x7fc16c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c0b92a0 0x7fc16c1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 --2- 172.21.15.148:0/2866134524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc16c0b2cb0 0x7fc16c0cc3c0 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 >> 172.21.15.148:0/2866134524 conn(0x7fc16c061a20 msgr2=0x7fc16c15b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 shutdown_connections 2024-04-25T10:06:57.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:57.825+0000 7fc17141f640 1 -- 172.21.15.148:0/2866134524 wait complete. 2024-04-25T10:06:58.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:52.824899+0000', 'modified': '2024-04-25T10:06:55.145145+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11634': {'gid': 11634, 'name': 'd', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/1255014956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 1255014956}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 1255014956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:58.246 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:58.246 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:06:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.340+0000 7fb66455c640 1 Processor -- start 2024-04-25T10:06:58.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.340+0000 7fb66455c640 1 -- start start 2024-04-25T10:06:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb66455c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb65c15b7d0 0x7fb65c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb66455c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c0b5620 0x7fb65c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb66455c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c06cf50 con 0x7fb65c06c2d0 2024-04-25T10:06:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb66455c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c0b5fc0 con 0x7fb65c15b7d0 2024-04-25T10:06:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb66455c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0ce530 con 0x7fb65c0b5620 2024-04-25T10:06:58.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb6622f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb65c15b7d0 0x7fb65c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb6622f8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb65c15b7d0 0x7fb65c0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33348/0 (socket says 172.21.15.148:33348) 2024-04-25T10:06:58.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.341+0000 7fb6622f8640 1 -- 172.21.15.148:0/4124702641 learned_addr learned my addr 172.21.15.148:0/4124702641 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.342+0000 7fb6622f8640 1 -- 172.21.15.148:0/4124702641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c0b5620 msgr2=0x7fb65c0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.342+0000 7fb6622f8640 1 --2- 172.21.15.148:0/4124702641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c0b5620 0x7fb65c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.342+0000 7fb6622f8640 1 -- 172.21.15.148:0/4124702641 >> v1:172.21.15.188:6789/0 conn(0x7fb65c06c2d0 legacy=0x7fb65c06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.348 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.342+0000 7fb6622f8640 1 -- 172.21.15.148:0/4124702641 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c0ce810 con 0x7fb65c15b7d0 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.342+0000 7fb6622f8640 1 --2- 172.21.15.148:0/4124702641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c15b7d0 0x7fb65c0cf010 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb65802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6061c29de30bc1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.342+0000 7fb6612f6640 1 -- 172.21.15.148:0/4124702641 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658036070 con 0x7fb65c15b7d0 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb6612f6640 1 -- 172.21.15.148:0/4124702641 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb658031600 con 0x7fb65c15b7d0 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb6612f6640 1 -- 172.21.15.148:0/4124702641 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580319d0 con 0x7fb65c15b7d0 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 -- 172.21.15.148:0/4124702641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c15b7d0 msgr2=0x7fb65c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 --2- 172.21.15.148:0/4124702641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c15b7d0 0x7fb65c0cf010 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb65802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 -- 172.21.15.148:0/4124702641 shutdown_connections 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 --2- 172.21.15.148:0/4124702641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c0b5620 0x7fb65c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 --2- 172.21.15.148:0/4124702641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c15b7d0 0x7fb65c0cf010 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 -- 172.21.15.148:0/4124702641 >> 172.21.15.148:0/4124702641 conn(0x7fb65c0d0800 msgr2=0x7fb65c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 -- 172.21.15.148:0/4124702641 shutdown_connections 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.343+0000 7fb66455c640 1 -- 172.21.15.148:0/4124702641 wait complete. 2024-04-25T10:06:58.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.344+0000 7fb66455c640 1 Processor -- start 2024-04-25T10:06:58.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.344+0000 7fb66455c640 1 -- start start 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb66455c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c06c2d0 0x7fb65c0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb66455c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 0x7fb65c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb66455c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c15b7d0 0x7fb65c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb66455c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0cf550 con 0x7fb65c0b5620 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb66455c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c06d230 con 0x7fb65c15b7d0 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb66455c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0cf6d0 con 0x7fb65c06c2d0 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb661af7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 0x7fb65c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb661af7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 0x7fb65c0cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33354/0 (socket says 172.21.15.148:33354) 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb661af7640 1 -- 172.21.15.148:0/1026234203 learned_addr learned my addr 172.21.15.148:0/1026234203 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb6622f8640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c06c2d0 0x7fb65c0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.345+0000 7fb662af9640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c15b7d0 0x7fb65c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.353 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb661af7640 1 -- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c06c2d0 msgr2=0x7fb65c0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb661af7640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c06c2d0 0x7fb65c0c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb661af7640 1 -- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c15b7d0 msgr2=0x7fb65c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb661af7640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c15b7d0 0x7fb65c15ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb661af7640 1 -- 172.21.15.148:0/1026234203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c15bc50 con 0x7fb65c0b5620 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb662af9640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c15b7d0 0x7fb65c15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb6622f8640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c06c2d0 0x7fb65c0c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb661af7640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 0x7fb65c0cb4b0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b4d0 tx=0x7fb64c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb652ffd640 1 -- 172.21.15.148:0/1026234203 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c014070 con 0x7fb65c0b5620 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb652ffd640 1 -- 172.21.15.148:0/1026234203 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c010870 con 0x7fb65c0b5620 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c15bf30 con 0x7fb65c0b5620 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb652ffd640 1 -- 172.21.15.148:0/1026234203 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c010c60 con 0x7fb65c0b5620 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.346+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c15c7d0 con 0x7fb65c0b5620 2024-04-25T10:06:58.354 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.347+0000 7fb652ffd640 1 -- 172.21.15.148:0/1026234203 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb64c017070 con 0x7fb65c0b5620 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.348+0000 7fb652ffd640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb64006b4d0 0x7fb64006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.348+0000 7fb652ffd640 1 -- 172.21.15.148:0/1026234203 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb64c08a1a0 con 0x7fb65c0b5620 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.348+0000 7fb6622f8640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb64006b4d0 0x7fb64006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.348+0000 7fb6622f8640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb64006b4d0 0x7fb64006d990 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb658033000 tx=0x7fb658009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.349+0000 7fb667264740 1 Processor -- start 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.349+0000 7fb667264740 1 -- start start 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb667264740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975b2f10 0x5605975f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb667264740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x560597515a10 con 0x5605975faad0 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605974beb30 con 0x5605975dd230 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560597511c50 con 0x5605975b2f10 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975fb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33370/0 (socket says 172.21.15.148:33370) 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 -- 172.21.15.148:0/2212217608 learned_addr learned my addr 172.21.15.148:0/2212217608 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb662af9640 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975b2f10 0x5605975f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb6622f8640 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 -- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975b2f10 msgr2=0x5605975f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975b2f10 0x5605975f9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 -- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 msgr2=0x5605975dd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975dd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 -- 172.21.15.148:0/2212217608 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605974d3e10 con 0x5605975faad0 2024-04-25T10:06:58.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb6622f8640 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.350+0000 7fb661af7640 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975fb020 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0884e0 tx=0x7fb64c05c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5809862858259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb63ffff640 1 -- 172.21.15.148:0/2212217608 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c01e070 con 0x5605975faad0 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb63ffff640 1 -- 172.21.15.148:0/2212217608 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c056830 con 0x5605975faad0 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb63ffff640 1 -- 172.21.15.148:0/2212217608 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c054e00 con 0x5605975faad0 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 -- 172.21.15.148:0/2212217608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 msgr2=0x5605975fb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975fb020 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0884e0 tx=0x7fb64c05c9a0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 -- 172.21.15.148:0/2212217608 shutdown_connections 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975b2f10 0x5605975f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975fb020 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 --2- 172.21.15.148:0/2212217608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 -- 172.21.15.148:0/2212217608 >> 172.21.15.148:0/2212217608 conn(0x5605975af840 msgr2=0x5605975f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 -- 172.21.15.148:0/2212217608 shutdown_connections 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.351+0000 7fb667264740 1 -- 172.21.15.148:0/2212217608 wait complete. 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.352+0000 7fb667264740 1 -- start start 2024-04-25T10:06:58.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.352+0000 7fb667264740 1 Processor -- start 2024-04-25T10:06:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975faad0 0x5605975d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605975b3cb0 con 0x5605975d9af0 2024-04-25T10:06:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605974d20c0 con 0x5605975faad0 2024-04-25T10:06:58.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605975ad770 con 0x5605975dd230 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb667264740 10 client.0 _ll_register_callbacks cb 0x5605974c5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975db9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33374/0 (socket says 172.21.15.148:33374) 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 -- 172.21.15.148:0/411349963 learned_addr learned my addr 172.21.15.148:0/411349963 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb6622f8640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb661af7640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975faad0 0x5605975d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 -- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 msgr2=0x5605975e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 -- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975faad0 msgr2=0x5605975d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975faad0 0x5605975d9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 -- 172.21.15.148:0/411349963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56059722a2d0 con 0x5605975d9af0 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb662af9640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975db9f0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fb654001d80 tx=0x7fb654001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d1b60ca8c8024a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.353+0000 7fb63d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb63d7fa640 1 -- 172.21.15.148:0/411349963 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6540052d0 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb63d7fa640 1 -- 172.21.15.148:0/411349963 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6540035f0 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb64c056830 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb63d7fa640 1 -- 172.21.15.148:0/411349963 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb654005470 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 10 client.11763 fetch_fsmap learned FSMap version 113 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x5605972f46b0 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb63d7fa640 1 -- 172.21.15.148:0/411349963 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c056830 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb63d7fa640 1 -- 172.21.15.148:0/411349963 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb654007020 con 0x5605975d9af0 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 10 client.11763 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 10 client.11763 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 20 client.11763 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x560597608b30 0x5605975de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 4 client.11763 mds_command: new command op to 11634 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:58.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 --> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560597574920 con 0x560597608b30 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.354+0000 7fb6622f8640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x560597608b30 0x5605975de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb6622f8640 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x560597608b30 0x5605975de320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53388e3cff23b098 server_cookie=ef333beb5c7c9c12 in_seq=0 out_seq=0 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb63d7fa640 10 client.11763 ms_handle_connect on v2:172.21.15.188:6833/1255014956 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb63d7fa640 1 -- 172.21.15.148:0/411349963 <== mds.0 v2:172.21.15.188:6833/1255014956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560597574920 con 0x560597608b30 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb63d7fa640 10 client.11763 handle_command_reply: tid=0 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb667264740 1 client.11763 shutdown 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 msgr2=0x5605975db9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.355+0000 7fb667264740 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975db9f0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fb654001d80 tx=0x7fb654001e70 comp rx=0 tx=0).stop 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 shutdown_connections 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x560597608b30 0x5605975de320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975db9f0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975faad0 0x5605975d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb63d7fa640 0 client.11763 ms_handle_reset on v2:172.21.15.188:6833/1255014956 2024-04-25T10:06:58.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 --2- 172.21.15.148:0/411349963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 >> 172.21.15.148:0/411349963 conn(0x5605975b2f10 msgr2=0x5605975ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 shutdown_connections 2024-04-25T10:06:58.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 1 -- 172.21.15.148:0/411349963 wait complete. 2024-04-25T10:06:58.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.356+0000 7fb667264740 20 client.11763 trim_cache size 0 max 16384 2024-04-25T10:06:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.371+0000 7fb667264740 1 Processor -- start 2024-04-25T10:06:58.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.371+0000 7fb667264740 1 -- start start 2024-04-25T10:06:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975d9af0 0x56059765e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb667264740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb667264740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1ff790 con 0x5605975faad0 2024-04-25T10:06:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1f7840 con 0x5605975d9af0 2024-04-25T10:06:58.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1f7e40 con 0x5605975dd230 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975f7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33384/0 (socket says 172.21.15.148:33384) 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 -- 172.21.15.148:0/149111933 learned_addr learned my addr 172.21.15.148:0/149111933 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb6622f8640 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975d9af0 0x56059765e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb661af7640 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 -- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 msgr2=0x5605975b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 -- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975d9af0 msgr2=0x56059765e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975d9af0 0x56059765e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 -- 172.21.15.148:0/149111933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605975aed00 con 0x5605975faad0 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb662af9640 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975f7e10 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fb654001950 tx=0x7fb65400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a212fff8927facf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb63f7fe640 1 -- 172.21.15.148:0/149111933 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6540052d0 con 0x5605975faad0 2024-04-25T10:06:58.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb63f7fe640 1 -- 172.21.15.148:0/149111933 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb654007670 con 0x5605975faad0 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.372+0000 7fb63f7fe640 1 -- 172.21.15.148:0/149111933 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654003810 con 0x5605975faad0 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 -- 172.21.15.148:0/149111933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 msgr2=0x5605975f7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975f7e10 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fb654001950 tx=0x7fb65400c000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 -- 172.21.15.148:0/149111933 shutdown_connections 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975faad0 0x5605975f7e10 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975dd230 0x5605975b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 --2- 172.21.15.148:0/149111933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975d9af0 0x56059765e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 -- 172.21.15.148:0/149111933 >> 172.21.15.148:0/149111933 conn(0x5605975b2f10 msgr2=0x5605975f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 -- 172.21.15.148:0/149111933 shutdown_connections 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 -- 172.21.15.148:0/149111933 wait complete. 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.373+0000 7fb667264740 1 -- start start 2024-04-25T10:06:58.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 Processor -- start 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975faf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975faad0 0x5605975ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605976089b0 con 0x5605975d9af0 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605975e0c60 con 0x5605975dd230 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560597608fb0 con 0x5605975faad0 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb667264740 10 client.0 _ll_register_callbacks cb 0x5605975adc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.374+0000 7fb6622f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975faf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33388/0 (socket says 172.21.15.148:33388) 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 -- 172.21.15.148:0/2957388452 learned_addr learned my addr 172.21.15.148:0/2957388452 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb661af7640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb662af9640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975faad0 0x5605975ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 -- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975faad0 msgr2=0x5605975ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975faad0 0x5605975ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 -- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 msgr2=0x5605975fb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975fb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 -- 172.21.15.148:0/2957388452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560597612970 con 0x5605975d9af0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb661af7640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975fb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb6622f8640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975faf80 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fb658009f70 tx=0x7fb65802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79039b37537a08eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb63d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb63d7fa640 1 -- 172.21.15.148:0/2957388452 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658036040 con 0x5605975d9af0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb63d7fa640 1 -- 172.21.15.148:0/2957388452 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb658031c40 con 0x5605975d9af0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb654007670 con 0x5605975d9af0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb63d7fa640 1 -- 172.21.15.148:0/2957388452 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb658031e40 con 0x5605975d9af0 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 10 client.11769 fetch_fsmap learned FSMap version 113 2024-04-25T10:06:58.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x5605975ee6c0 con 0x5605975d9af0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb63d7fa640 1 -- 172.21.15.148:0/2957388452 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb654007670 con 0x5605975d9af0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb63d7fa640 1 -- 172.21.15.148:0/2957388452 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb658008040 con 0x5605975d9af0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 10 client.11769 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 10 client.11769 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 20 client.11769 populate_metadata read hostname 'smithi148' 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x5605975acdc0 0x5605975f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 4 client.11769 mds_command: new command op to 11634 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.375+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 --> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560597522f50 con 0x5605975acdc0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb661af7640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x5605975acdc0 0x5605975f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb661af7640 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x5605975acdc0 0x5605975f52c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8440a29b4e18a68 server_cookie=5cb8667e29c1c1c6 in_seq=0 out_seq=0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb63d7fa640 10 client.11769 ms_handle_connect on v2:172.21.15.188:6833/1255014956 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb63d7fa640 1 -- 172.21.15.148:0/2957388452 <== mds.0 v2:172.21.15.188:6833/1255014956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x560597522f50 con 0x5605975acdc0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb63d7fa640 10 client.11769 handle_command_reply: tid=0 2024-04-25T10:06:58.388 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 client.11769 shutdown 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 msgr2=0x5605975faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975faf80 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7fb658009f70 tx=0x7fb65802c000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 shutdown_connections 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:6833/1255014956,v1:172.21.15.188:6835/1255014956] conn(0x5605975acdc0 0x5605975f52c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5605975faad0 0x5605975ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605975dd230 0x5605975fb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 --2- 172.21.15.148:0/2957388452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605975d9af0 0x5605975faf80 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb63d7fa640 0 client.11769 ms_handle_reset on v2:172.21.15.188:6833/1255014956 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 >> 172.21.15.148:0/2957388452 conn(0x5605975b2f10 msgr2=0x5605975f72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 shutdown_connections 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.376+0000 7fb667264740 1 -- 172.21.15.148:0/2957388452 wait complete. 2024-04-25T10:06:58.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb667264740 20 client.11769 trim_cache size 0 max 16384 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 11634, 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.8766085590000001, 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 113, 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 197, 2024-04-25T10:06:58.390 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 197, 2024-04-25T10:06:58.391 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 9.9398511369999998 2024-04-25T10:06:58.391 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb64006b4d0 msgr2=0x7fb64006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb64006b4d0 0x7fb64006d990 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb658033000 tx=0x7fb658009000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 msgr2=0x7fb65c0cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 0x7fb65c0cb4b0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b4d0 tx=0x7fb64c00b990 comp rx=0 tx=0).stop 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 shutdown_connections 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb64006b4d0 0x7fb64006d990 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb65c15b7d0 0x7fb65c15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb65c0b5620 0x7fb65c0cb4b0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 --2- 172.21.15.148:0/1026234203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb65c06c2d0 0x7fb65c0c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 >> 172.21.15.148:0/1026234203 conn(0x7fb65c0d0800 msgr2=0x7fb65c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.392 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 shutdown_connections 2024-04-25T10:06:58.393 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.377+0000 7fb66455c640 1 -- 172.21.15.148:0/1026234203 wait complete. 2024-04-25T10:06:58.394 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:58.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.485+0000 7f98d0509640 1 Processor -- start 2024-04-25T10:06:58.491 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.485+0000 7f98d0509640 1 -- start start 2024-04-25T10:06:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.486+0000 7f98d0509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c815ba00 0x7f98c809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.486+0000 7f98d0509640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98c815f5f0 0x7f98c815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.486+0000 7f98d0509640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c80cd710 con 0x7f98c81602e0 2024-04-25T10:06:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.486+0000 7f98d0509640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c80cd4b0 con 0x7f98c815f5f0 2024-04-25T10:06:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.486+0000 7f98d0509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8099b70 con 0x7f98c815ba00 2024-04-25T10:06:58.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98c815f5f0 0x7f98c815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f98c815f5f0 0x7f98c815faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33394/0 (socket says 172.21.15.148:33394) 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 -- 172.21.15.148:0/1109248240 learned_addr learned my addr 172.21.15.148:0/1109248240 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ce2a5640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c815ba00 0x7f98c809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 -- 172.21.15.148:0/1109248240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c815ba00 msgr2=0x7f98c809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c815ba00 0x7f98c809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 -- 172.21.15.148:0/1109248240 >> v1:172.21.15.188:6789/0 conn(0x7f98c81602e0 legacy=0x7f98c80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ceaa6640 1 -- 172.21.15.148:0/1109248240 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8099e50 con 0x7f98c815f5f0 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.487+0000 7f98ce2a5640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c815ba00 0x7f98c809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98ceaa6640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c815faa0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009870 tx=0x7f98c002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187c1ea3978ee087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98cd2a3640 1 -- 172.21.15.148:0/1109248240 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0037070 con 0x7f98c815f5f0 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98cd2a3640 1 -- 172.21.15.148:0/1109248240 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c002ae10 con 0x7f98c815f5f0 2024-04-25T10:06:58.494 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98cd2a3640 1 -- 172.21.15.148:0/1109248240 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0032660 con 0x7f98c815f5f0 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 -- 172.21.15.148:0/1109248240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 msgr2=0x7f98c815faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c815faa0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009870 tx=0x7f98c002a100 comp rx=0 tx=0).stop 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 -- 172.21.15.148:0/1109248240 shutdown_connections 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c815faa0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 --2- 172.21.15.148:0/1109248240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c815ba00 0x7f98c809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 -- 172.21.15.148:0/1109248240 >> 172.21.15.148:0/1109248240 conn(0x7f98c80b3270 msgr2=0x7f98c80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.497 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 -- 172.21.15.148:0/1109248240 shutdown_connections 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.488+0000 7f98d0509640 1 -- 172.21.15.148:0/1109248240 wait complete. 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.489+0000 7f98d0509640 1 Processor -- start 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 -- start start 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98c815ba00 0x7f98c815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c81f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c81602e0 0x7f98c81f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c815bde0 con 0x7f98c815f5f0 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c809ae70 con 0x7f98c815ba00 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98d0509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c81f2580 con 0x7f98c81602e0 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98cdaa4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c81f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98cdaa4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c81f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33408/0 (socket says 172.21.15.148:33408) 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98cdaa4640 1 -- 172.21.15.148:0/4201775873 learned_addr learned my addr 172.21.15.148:0/4201775873 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:06:58.498 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98ceaa6640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c81602e0 0x7f98c81f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.490+0000 7f98ce2a5640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98c815ba00 0x7f98c815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98cdaa4640 1 -- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c81602e0 msgr2=0x7f98c81f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98cdaa4640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c81602e0 0x7f98c81f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98cdaa4640 1 -- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98c815ba00 msgr2=0x7f98c815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98cdaa4640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98c815ba00 0x7f98c815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98cdaa4640 1 -- 172.21.15.148:0/4201775873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c81f2860 con 0x7f98c815f5f0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98cdaa4640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c81f1a60 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f98b800b3f0 tx=0x7f98b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8015020 con 0x7f98c815f5f0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b8011830 con 0x7f98c815f5f0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c80c9ff0 con 0x7f98c815f5f0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8011c20 con 0x7f98c815f5f0 2024-04-25T10:06:58.499 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.491+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c815e220 con 0x7f98c815f5f0 2024-04-25T10:06:58.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.492+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c80d1250 con 0x7f98c815f5f0 2024-04-25T10:06:58.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.492+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98b8018020 con 0x7f98c815f5f0 2024-04-25T10:06:58.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.493+0000 7f98beffd640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f98ac06b4d0 0x7f98ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.500 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.493+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98b8089e20 con 0x7f98c815f5f0 2024-04-25T10:06:58.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.493+0000 7f98ce2a5640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f98ac06b4d0 0x7f98ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.493+0000 7f98ce2a5640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f98ac06b4d0 0x7f98ac06d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f98c400d620 tx=0x7f98c4007810 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.497+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98c80d1250 con 0x7f98c815f5f0 2024-04-25T10:06:58.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.702+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98c8073990 con 0x7f98c815f5f0 2024-04-25T10:06:58.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.703+0000 7f98beffd640 1 -- 172.21.15.148:0/4201775873 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98b801f030 con 0x7f98c815f5f0 2024-04-25T10:06:58.708 INFO:teuthology.orchestra.run.smithi148.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:58.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f98ac06b4d0 msgr2=0x7f98ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f98ac06b4d0 0x7f98ac06d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f98c400d620 tx=0x7f98c4007810 comp rx=0 tx=0).stop 2024-04-25T10:06:58.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 msgr2=0x7f98c81f1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c81f1a60 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f98b800b3f0 tx=0x7f98b800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 shutdown_connections 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f98ac06b4d0 0x7f98ac06d990 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98c81602e0 0x7f98c81f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98c815f5f0 0x7f98c81f1a60 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.704+0000 7f98d0509640 1 --2- 172.21.15.148:0/4201775873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98c815ba00 0x7f98c815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.705+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 >> 172.21.15.148:0/4201775873 conn(0x7f98c80b3270 msgr2=0x7f98c80cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.705+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 shutdown_connections 2024-04-25T10:06:58.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:06:58.705+0000 7f98d0509640 1 -- 172.21.15.148:0/4201775873 wait complete. 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:06:58.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:58.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:58.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.722 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-04-25T10:06:58.737 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:58.737 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7494:166a:fa2c:beb0/64 scope link noprefixroute 2024-04-25T10:06:58.738 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::8d9:b5ff:fe99:b622/64 scope link 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ca:9b:e2:64:5f:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::c89b:e2ff:fe64:5fe3/64 scope link 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 4a:7b:7d:fe:e5:6b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:58.739 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::487b:7dff:fefe:e56b/64 scope link 2024-04-25T10:06:58.740 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:58.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.740 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:58.776 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-04-25T10:06:58.770+0000 7fb201eb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:58.799 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-25T10:06:58.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.801 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-04-25T10:06:58.857 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-04-25T10:06:58.857 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:58.857 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:06:58.857 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:06:58.857 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:58.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.858 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec bash -c ' 2024-04-25T10:06:58.858 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-04-25T10:06:58.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:06:58.858 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:06:58.859 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:06:58.859 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-04-25T10:06:58.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.941 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:06:58.996 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:58.996 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:58.996 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:58.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:58.997 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-04-25T10:06:59.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:59.062 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.117 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:59.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:59.118 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.172 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.172 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:59.253 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:59.254 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:59.254 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:06:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:06:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:06:59.262 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:invalid new device string format 2024-04-25T10:06:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-04-25T10:06:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.148:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=086ed9ea-68e4-4e62-83c5-c0821c62dd17" will pass to kernel 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:06:59.271 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:06:59.272 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:06:59.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.336 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:06:59.397 INFO:teuthology.orchestra.run.smithi148.stdout:client9397 2024-04-25T10:06:59.397 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:59.398 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.453 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T10:06:59.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.512 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T10:06:59.569 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:59.569 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.625 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:06:59.714 INFO:teuthology.orchestra.run.smithi148.stderr:6+0 records in 2024-04-25T10:06:59.714 INFO:teuthology.orchestra.run.smithi148.stderr:6+0 records out 2024-04-25T10:06:59.714 INFO:teuthology.orchestra.run.smithi148.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0342804 s, 184 MB/s 2024-04-25T10:06:59.715 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:59.715 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.731 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:06:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:6+0 records in 2024-04-25T10:06:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:6+0 records out 2024-04-25T10:06:59.818 INFO:teuthology.orchestra.run.smithi148.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0315451 s, 199 MB/s 2024-04-25T10:06:59.820 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:06:59.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:06:59.860 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-25T10:06:59.860 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-25T10:06:59.860 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:06:59.860 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-25T10:06:59.860 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.860 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:06:59.861 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.862 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-25T10:06:59.863 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-25T10:06:59.864 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.865 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-25T10:06:59.866 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.867 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:06:59.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.869 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-25T10:06:59.870 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-25T10:06:59.870 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-25T10:06:59.871 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.872 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.873 INFO:teuthology.orchestra.run.smithi148.stdout: mds cache max size = 1000 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = False 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: mds log max segments = 1 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = False 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.874 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.878 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-25T10:06:59.879 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.880 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:06:59.881 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-25T10:06:59.882 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.883 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-04-25T10:06:59.884 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:06:59.885 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.886 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.887 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-25T10:06:59.889 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: mds cache max size = 1000 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = False 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: mds log max segments = 1 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = False 2024-04-25T10:06:59.890 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:06:59.891 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.891 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.893 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:06:59.935 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.973 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-25T10:06:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.975 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.976 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-25T10:06:59.977 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.978 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 20 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 20 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 10 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-25T10:06:59.979 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.980 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.981 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: mon op complaint time = 120 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-25T10:06:59.982 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: client mount timeout = 600 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: debug client = 20 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-04-25T10:06:59.983 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds = 20 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: debug mds balancer = 20 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: mds cache max size = 1000 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug frag = True 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: mds debug scatterstat = False 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: mds log max segments = 1 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: mds op complaint time = 180 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: mds verify scatter = False 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: osd op complaint time = 180 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.984 INFO:teuthology.orchestra.run.smithi148.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-04-25T10:06:59.985 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:06:59.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-04-25T10:06:59.987 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.148,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 086ed9ea-68e4-4e62-83c5-c0821c62dd17 2024-04-25T10:06:59.988 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-04-25T10:06:59.989 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.990 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-04-25T10:06:59.991 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-04-25T10:06:59.992 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:06:59.993 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:06:59.994 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-04-25T10:06:59.995 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: mds cache max size = 1000 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = False 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: mds log max segments = 1 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = False 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-04-25T10:06:59.996 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-04-25T10:06:59.998 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:06:59.998 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:00.052 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:07:00.053 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:00.107 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:00.244 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.239+0000 7f4f9ff29640 1 Processor -- start 2024-04-25T10:07:00.245 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.239+0000 7f4f9ff29640 1 -- start start 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9ff29640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f980c6df0 0x7f4f980d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9ff29640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f980d1c00 0x7f4f980ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9ff29640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9815dfe0 con 0x7f4f9815dc00 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9ff29640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980d2070 con 0x7f4f980c6df0 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9ff29640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980d00c0 con 0x7f4f980d1c00 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9d4c4640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4f980c6df0 0x7f4f980d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9dcc5640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4f9815dc00 0x7f4f980ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51834/0 (socket says 172.21.15.148:51834) 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9dcc5640 1 -- 172.21.15.148:0/4121241955 learned_addr learned my addr 172.21.15.148:0/4121241955 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9e4c6640 1 --2- 172.21.15.148:0/4121241955 >> v2:172.21.15.188:3300/0 conn(0x7f4f980d1c00 0x7f4f980ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4121241955 >> v2:172.21.15.188:3300/0 conn(0x7f4f980d1c00 msgr2=0x7f4f980ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.246 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9d4c4640 1 --2- 172.21.15.148:0/4121241955 >> v2:172.21.15.188:3300/0 conn(0x7f4f980d1c00 0x7f4f980ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4121241955 >> v1:172.21.15.148:6789/0 conn(0x7f4f9815dc00 legacy=0x7f4f980ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.247 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.240+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4121241955 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980d03a0 con 0x7f4f980c6df0 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9d4c4640 1 --2- 172.21.15.148:0/4121241955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980c6df0 0x7f4f980d16c0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f4f980b1040 tx=0x7f4f8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98f8d0198a95276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ccc3640 1 -- 172.21.15.148:0/4121241955 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88037070 con 0x7f4f980c6df0 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ccc3640 1 -- 172.21.15.148:0/4121241955 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88032440 con 0x7f4f980c6df0 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ccc3640 1 -- 172.21.15.148:0/4121241955 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88032810 con 0x7f4f980c6df0 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4121241955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980c6df0 msgr2=0x7f4f980d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4121241955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980c6df0 0x7f4f980d16c0 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f4f980b1040 tx=0x7f4f8802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4121241955 shutdown_connections 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4121241955 >> v2:172.21.15.188:3300/0 conn(0x7f4f980d1c00 0x7f4f980ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4121241955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980c6df0 0x7f4f980d16c0 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.241+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4121241955 >> 172.21.15.148:0/4121241955 conn(0x7f4f9806f5d0 msgr2=0x7f4f9806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.242+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4121241955 shutdown_connections 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.242+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4121241955 wait complete. 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 Processor -- start 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 -- start start 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f980c6df0 0x7f4f981533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 0x7f4f980c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f9815dc00 0x7f4f981599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980cc130 con 0x7f4f980d1c00 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980cc720 con 0x7f4f980c6df0 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9ff29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980cfd90 con 0x7f4f9815dc00 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9d4c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 0x7f4f980c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9d4c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 0x7f4f980c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33424/0 (socket says 172.21.15.148:33424) 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.243+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4261998403 learned_addr learned my addr 172.21.15.148:0/4261998403 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9dcc5640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f980c6df0 0x7f4f981533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9e4c6640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f9815dc00 0x7f4f981599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f9815dc00 msgr2=0x7f4f981599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9d4c4640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f9815dc00 0x7f4f981599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f980c6df0 msgr2=0x7f4f981533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9d4c4640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f980c6df0 0x7f4f981533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9d4c4640 1 -- 172.21.15.148:0/4261998403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980b4d40 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9e4c6640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f9815dc00 0x7f4f981599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9d4c4640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 0x7f4f980c92b0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88037040 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88042070 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f981538f0 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88030bd0 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.244+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f980c9fc0 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.245+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f98073990 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.246+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f88047460 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.246+0000 7f4f8e7fc640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.246+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f880aaef0 con 0x7f4f980d1c00 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.246+0000 7f4f9dcc5640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.254 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.246+0000 7f4f9dcc5640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007fd0 tx=0x7f4f94007e80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.250+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f88032e20 con 0x7f4f980d1c00 2024-04-25T10:07:00.429 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.424+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f980d16f0 con 0x7f4f980d1c00 2024-04-25T10:07:00.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.425+0000 7f4f8e7fc640 1 -- 172.21.15.148:0/4261998403 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16932 (secure 0 0 0) 0x7f4f88079b50 con 0x7f4f980d1c00 2024-04-25T10:07:00.432 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:07:00.432 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":197,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:06:54.484614+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f7c06b4d0 msgr2=0x7f4f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007fd0 tx=0x7f4f94007e80 comp rx=0 tx=0).stop 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 msgr2=0x7f4f980c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 0x7f4f980c92b0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f880046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 shutdown_connections 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4f7c06b4d0 0x7f4f7c06d990 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.433 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f9815dc00 0x7f4f981599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f980d1c00 0x7f4f980c92b0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 --2- 172.21.15.148:0/4261998403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f980c6df0 0x7f4f981533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 >> 172.21.15.148:0/4261998403 conn(0x7f4f9806f5d0 msgr2=0x7f4f98156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 shutdown_connections 2024-04-25T10:07:00.434 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.427+0000 7f4f9ff29640 1 -- 172.21.15.148:0/4261998403 wait complete. 2024-04-25T10:07:00.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:51.040797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:51.800930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:00.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:00.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.539+0000 7f96fc35c640 1 Processor -- start 2024-04-25T10:07:00.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.540+0000 7f96fc35c640 1 -- start start 2024-04-25T10:07:00.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.540+0000 7f96fc35c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96f4158230 0x7f96f40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.540+0000 7f96fc35c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f40b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.540+0000 7f96fc35c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f40d2020 con 0x7f96f40d1b80 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.540+0000 7f96fc35c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f4158610 con 0x7f96f4158230 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.540+0000 7f96fc35c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40b57c0 con 0x7f96f40b4e60 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96f98f7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f96f40d1b80 0x7f96f406c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51848/0 (socket says 172.21.15.148:51848) 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96f98f7640 1 -- 172.21.15.148:0/1904471974 learned_addr learned my addr 172.21.15.148:0/1904471974 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/1904471974 >> v2:172.21.15.188:3300/0 conn(0x7f96f4158230 0x7f96f40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96fa8f9640 1 --2- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f40b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96f90f6640 1 -- 172.21.15.148:0/1904471974 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977336669 0 0) 0x7f96f40d2020 con 0x7f96f40d1b80 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96f90f6640 1 -- 172.21.15.148:0/1904471974 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e0003520 con 0x7f96f40d1b80 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96fa0f8640 1 -- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 msgr2=0x7f96f40b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f40b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96fa0f8640 1 -- 172.21.15.148:0/1904471974 >> v1:172.21.15.148:6789/0 conn(0x7f96f40d1b80 legacy=0x7f96f406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.541+0000 7f96fa0f8640 1 -- 172.21.15.148:0/1904471974 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40d2300 con 0x7f96f4158230 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.542+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f4158230 0x7f96f40d1640 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f96f00097a0 tx=0x7f96f002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190da329386fc1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.542+0000 7f96f90f6640 1 -- 172.21.15.148:0/1904471974 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0037070 con 0x7f96f4158230 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.542+0000 7f96f90f6640 1 -- 172.21.15.148:0/1904471974 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0032440 con 0x7f96f4158230 2024-04-25T10:07:00.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.542+0000 7f96f90f6640 1 -- 172.21.15.148:0/1904471974 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0032810 con 0x7f96f4158230 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.542+0000 7f96fc35c640 1 -- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f4158230 msgr2=0x7f96f40d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.542+0000 7f96fc35c640 1 --2- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f4158230 0x7f96f40d1640 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f96f00097a0 tx=0x7f96f002a180 comp rx=0 tx=0).stop 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.543+0000 7f96fc35c640 1 -- 172.21.15.148:0/1904471974 shutdown_connections 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.543+0000 7f96fc35c640 1 --2- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f40b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.543+0000 7f96fc35c640 1 --2- 172.21.15.148:0/1904471974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f4158230 0x7f96f40d1640 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.543+0000 7f96fc35c640 1 -- 172.21.15.148:0/1904471974 >> 172.21.15.148:0/1904471974 conn(0x7f96f4061600 msgr2=0x7f96f40619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.543+0000 7f96fc35c640 1 -- 172.21.15.148:0/1904471974 shutdown_connections 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.543+0000 7f96fc35c640 1 -- 172.21.15.148:0/1904471974 wait complete. 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 Processor -- start 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 -- start start 2024-04-25T10:07:00.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f40d1b80 0x7f96f40ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 0x7f96f40cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40d25e0 con 0x7f96f4158230 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.544+0000 7f96fc35c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f406bea0 con 0x7f96f40d1b80 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fc35c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f406ca40 con 0x7f96f40b4e60 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 0x7f96f40cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 0x7f96f40cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33440/0 (socket says 172.21.15.148:33440) 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 -- 172.21.15.148:0/4181593974 learned_addr learned my addr 172.21.15.148:0/4181593974 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96f98f7640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f40d1b80 0x7f96f40ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 -- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 msgr2=0x7f96f414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 -- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f40d1b80 msgr2=0x7f96f40ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f40d1b80 0x7f96f40ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 -- 172.21.15.148:0/4181593974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40cab00 con 0x7f96f4158230 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96f98f7640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f40d1b80 0x7f96f40ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fa8f9640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 0x7f96f40cedd0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009dd0 tx=0x7f96ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec018020 con 0x7f96f4158230 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec014480 con 0x7f96f4158230 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f40cada0 con 0x7f96f4158230 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec014870 con 0x7f96f4158230 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.545+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f40cb360 con 0x7f96f4158230 2024-04-25T10:07:00.554 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.546+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f406de50 con 0x7f96f4158230 2024-04-25T10:07:00.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.547+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96ec005d10 con 0x7f96f4158230 2024-04-25T10:07:00.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.547+0000 7f96eaffd640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96d806b4d0 0x7f96d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.556 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.547+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96ec08aea0 con 0x7f96f4158230 2024-04-25T10:07:00.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.547+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96d806b4d0 0x7f96d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.548+0000 7f96fa0f8640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96d806b4d0 0x7f96d806d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f96f0034000 tx=0x7f96f002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.557 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.551+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96ec022070 con 0x7f96f4158230 2024-04-25T10:07:00.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.722+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96f40b0f10 con 0x7f96f4158230 2024-04-25T10:07:00.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.722+0000 7f96eaffd640 1 -- 172.21.15.148:0/4181593974 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v197) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96ec010040 con 0x7f96f4158230 2024-04-25T10:07:00.729 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:07:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96d806b4d0 msgr2=0x7f96d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96d806b4d0 0x7f96d806d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f96f0034000 tx=0x7f96f002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 msgr2=0x7f96f40cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 0x7f96f40cedd0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009dd0 tx=0x7f96ec00c490 comp rx=0 tx=0).stop 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 shutdown_connections 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f96d806b4d0 0x7f96d806d990 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96f4158230 0x7f96f40cedd0 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96f40d1b80 0x7f96f40ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 --2- 172.21.15.148:0/4181593974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96f40b4e60 0x7f96f414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 >> 172.21.15.148:0/4181593974 conn(0x7f96f4061600 msgr2=0x7f96f40b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 shutdown_connections 2024-04-25T10:07:00.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.724+0000 7f96fc35c640 1 -- 172.21.15.148:0/4181593974 wait complete. 2024-04-25T10:07:00.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:00.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.836+0000 7fd332168640 1 Processor -- start 2024-04-25T10:07:00.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.836+0000 7fd332168640 1 -- start start 2024-04-25T10:07:00.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd332168640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd32c15b7d0 0x7fd32c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd332168640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c06cf50 con 0x7fd32c0b5620 2024-04-25T10:07:00.843 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd332168640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c0ce530 con 0x7fd32c06c9e0 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd332168640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0ce810 con 0x7fd32c15b7d0 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd32affd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd32c0b5620 0x7fd32c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51858/0 (socket says 172.21.15.148:51858) 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd32affd640 1 -- 172.21.15.148:0/80685207 learned_addr learned my addr 172.21.15.148:0/80685207 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd32b7fe640 1 --2- 172.21.15.148:0/80685207 >> v2:172.21.15.188:3300/0 conn(0x7fd32c15b7d0 0x7fd32c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd32a7fc640 1 -- 172.21.15.148:0/80685207 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531229351 0 0) 0x7fd32c06cf50 con 0x7fd32c0b5620 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.837+0000 7fd32a7fc640 1 -- 172.21.15.148:0/80685207 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd314003520 con 0x7fd32c0b5620 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32a7fc640 1 -- 172.21.15.148:0/80685207 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527081640 0 0) 0x7fd314003520 con 0x7fd32c0b5620 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32b7fe640 1 -- 172.21.15.148:0/80685207 >> v1:172.21.15.188:6789/0 conn(0x7fd32c06c9e0 legacy=0x7fd32c0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32b7fe640 1 -- 172.21.15.148:0/80685207 >> v1:172.21.15.148:6789/0 conn(0x7fd32c0b5620 legacy=0x7fd32c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32b7fe640 1 -- 172.21.15.148:0/80685207 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0ceaf0 con 0x7fd32c15b7d0 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32b7fe640 1 --2- 172.21.15.148:0/80685207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c15b7d0 0x7fd32c0cf010 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097a0 tx=0x7fd31c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5211018e6cfd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32a7fc640 1 -- 172.21.15.148:0/80685207 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c037070 con 0x7fd32c15b7d0 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32a7fc640 1 -- 172.21.15.148:0/80685207 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c02ac50 con 0x7fd32c15b7d0 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.838+0000 7fd32a7fc640 1 -- 172.21.15.148:0/80685207 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320001670 con 0x7fd32c15b7d0 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 -- 172.21.15.148:0/80685207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c15b7d0 msgr2=0x7fd32c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 --2- 172.21.15.148:0/80685207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c15b7d0 0x7fd32c0cf010 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0097a0 tx=0x7fd31c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 -- 172.21.15.148:0/80685207 shutdown_connections 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 --2- 172.21.15.148:0/80685207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c15b7d0 0x7fd32c0cf010 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 -- 172.21.15.148:0/80685207 >> 172.21.15.148:0/80685207 conn(0x7fd32c0d0800 msgr2=0x7fd32c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 -- 172.21.15.148:0/80685207 shutdown_connections 2024-04-25T10:07:00.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.839+0000 7fd332168640 1 -- 172.21.15.148:0/80685207 wait complete. 2024-04-25T10:07:00.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.840+0000 7fd332168640 1 Processor -- start 2024-04-25T10:07:00.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.840+0000 7fd332168640 1 -- start start 2024-04-25T10:07:00.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd332168640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c06c9e0 0x7fd32c0b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd332168640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 0x7fd32c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd332168640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32c15b7d0 0x7fd32c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd332168640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0cf6d0 con 0x7fd32c0b5620 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd332168640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0cf550 con 0x7fd32c06c9e0 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd332168640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c15bbb0 con 0x7fd32c15b7d0 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 0x7fd32c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 0x7fd32c1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33456/0 (socket says 172.21.15.148:33456) 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 -- 172.21.15.148:0/543599933 learned_addr learned my addr 172.21.15.148:0/543599933 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32b7fe640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c06c9e0 0x7fd32c0b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32bfff640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32c15b7d0 0x7fd32c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 -- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32c15b7d0 msgr2=0x7fd32c0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32c15b7d0 0x7fd32c0b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 -- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c06c9e0 msgr2=0x7fd32c0b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c06c9e0 0x7fd32c0b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.841+0000 7fd32affd640 1 -- 172.21.15.148:0/543599933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0b7ad0 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.842+0000 7fd32affd640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 0x7fd32c1592b0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c7b0 tx=0x7fd32000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.842+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320017070 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.842+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd320005ad0 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.842+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c0b7db0 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.842+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320007070 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.842+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c0b8370 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.843+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c073a00 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.843+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd32001a070 con 0x7fd32c0b5620 2024-04-25T10:07:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.844+0000 7fd313fff640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd31806b4d0 0x7fd31806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.844+0000 7fd32b7fe640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd31806b4d0 0x7fd31806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.844+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd320089860 con 0x7fd32c0b5620 2024-04-25T10:07:00.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.844+0000 7fd32b7fe640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd31806b4d0 0x7fd31806d990 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd31c034000 tx=0x7fd31c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.853 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:00.848+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd320057f00 con 0x7fd32c0b5620 2024-04-25T10:07:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.017+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd32c0b77e0 con 0x7fd32c0b5620 2024-04-25T10:07:01.023 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.017+0000 7fd313fff640 1 -- 172.21.15.148:0/543599933 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v197) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd32005bfa0 con 0x7fd32c0b5620 2024-04-25T10:07:01.023 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd31806b4d0 msgr2=0x7fd31806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd31806b4d0 0x7fd31806d990 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd31c034000 tx=0x7fd31c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 msgr2=0x7fd32c1592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 0x7fd32c1592b0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c7b0 tx=0x7fd32000cc70 comp rx=0 tx=0).stop 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 shutdown_connections 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd31806b4d0 0x7fd31806d990 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd32c15b7d0 0x7fd32c0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd32c0b5620 0x7fd32c1592b0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 --2- 172.21.15.148:0/543599933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32c06c9e0 0x7fd32c0b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 >> 172.21.15.148:0/543599933 conn(0x7fd32c0d0800 msgr2=0x7fd32c1609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 shutdown_connections 2024-04-25T10:07:01.026 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.019+0000 7fd332168640 1 -- 172.21.15.148:0/543599933 wait complete. 2024-04-25T10:07:01.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:01.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.129+0000 7f679b93b640 1 Processor -- start 2024-04-25T10:07:01.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.129+0000 7f679b93b640 1 -- start start 2024-04-25T10:07:01.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.130+0000 7f679b93b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67941610f0 0x7f679406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.130+0000 7f679b93b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67941616f0 0x7f6794161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.130+0000 7f679b93b640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67940c6960 con 0x7f67940ce0a0 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.130+0000 7f679b93b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67941614d0 con 0x7f67941610f0 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.130+0000 7f679b93b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6794162260 con 0x7f67941616f0 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.130+0000 7f67996d7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67941610f0 0x7f679406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f6799ed8640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f67940ce0a0 0x7f67940d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51872/0 (socket says 172.21.15.148:51872) 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f6799ed8640 1 -- 172.21.15.148:0/4022118782 learned_addr learned my addr 172.21.15.148:0/4022118782 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f6798ed6640 1 --2- 172.21.15.148:0/4022118782 >> v2:172.21.15.188:3300/0 conn(0x7f67941616f0 0x7f6794161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f67996d7640 1 -- 172.21.15.148:0/4022118782 >> v2:172.21.15.188:3300/0 conn(0x7f67941616f0 msgr2=0x7f6794161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f67996d7640 1 --2- 172.21.15.148:0/4022118782 >> v2:172.21.15.188:3300/0 conn(0x7f67941616f0 0x7f6794161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f67996d7640 1 -- 172.21.15.148:0/4022118782 >> v1:172.21.15.148:6789/0 conn(0x7f67940ce0a0 legacy=0x7f67940d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f678bfff640 1 -- 172.21.15.148:0/4022118782 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434489551 0 0) 0x7f67940c6960 con 0x7f67940ce0a0 2024-04-25T10:07:01.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.131+0000 7f67996d7640 1 -- 172.21.15.148:0/4022118782 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794162540 con 0x7f67941610f0 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f67996d7640 1 --2- 172.21.15.148:0/4022118782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941610f0 0x7f679406c1c0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f679406f220 tx=0x7f679002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e23564b6b6046f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f678bfff640 1 -- 172.21.15.148:0/4022118782 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790037070 con 0x7f67941610f0 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f678bfff640 1 -- 172.21.15.148:0/4022118782 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790032440 con 0x7f67941610f0 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f678bfff640 1 -- 172.21.15.148:0/4022118782 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790032810 con 0x7f67941610f0 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 -- 172.21.15.148:0/4022118782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941610f0 msgr2=0x7f679406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 --2- 172.21.15.148:0/4022118782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941610f0 0x7f679406c1c0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f679406f220 tx=0x7f679002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 -- 172.21.15.148:0/4022118782 shutdown_connections 2024-04-25T10:07:01.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 --2- 172.21.15.148:0/4022118782 >> v2:172.21.15.188:3300/0 conn(0x7f67941616f0 0x7f6794161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 --2- 172.21.15.148:0/4022118782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941610f0 0x7f679406c1c0 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 -- 172.21.15.148:0/4022118782 >> 172.21.15.148:0/4022118782 conn(0x7f679406f500 msgr2=0x7f679406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 -- 172.21.15.148:0/4022118782 shutdown_connections 2024-04-25T10:07:01.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.132+0000 7f679b93b640 1 -- 172.21.15.148:0/4022118782 wait complete. 2024-04-25T10:07:01.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.133+0000 7f679b93b640 1 Processor -- start 2024-04-25T10:07:01.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.133+0000 7f679b93b640 1 -- start start 2024-04-25T10:07:01.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f679b93b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67940ce0a0 0x7f67941f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f679b93b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67941610f0 0x7f67940ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f679b93b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 0x7f67940ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f679b93b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67940cf690 con 0x7f67941616f0 2024-04-25T10:07:01.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f679b93b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67940cfce0 con 0x7f67940ce0a0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f679b93b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679406c700 con 0x7f67941610f0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f6799ed8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 0x7f67940ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f6799ed8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 0x7f67940ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33476/0 (socket says 172.21.15.148:33476) 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.134+0000 7f6799ed8640 1 -- 172.21.15.148:0/1339495917 learned_addr learned my addr 172.21.15.148:0/1339495917 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6798ed6640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67941610f0 0x7f67940ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f67996d7640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67940ce0a0 0x7f67941f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6799ed8640 1 -- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67941610f0 msgr2=0x7f67940ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6799ed8640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67941610f0 0x7f67940ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6799ed8640 1 -- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67940ce0a0 msgr2=0x7f67941f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6799ed8640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67940ce0a0 0x7f67941f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6799ed8640 1 -- 172.21.15.148:0/1339495917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679415e0e0 con 0x7f67941616f0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6799ed8640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 0x7f67940ac920 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c7a0 tx=0x7f678c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c018070 con 0x7f67941616f0 2024-04-25T10:07:01.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c00a640 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c00aab0 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67940ae570 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.135+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67941f7610 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.136+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794073a00 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.136+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f678c01b070 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.137+0000 7f6789ffb640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f677806b500 0x7f677806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.137+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f678c0898b0 con 0x7f67941616f0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.137+0000 7f67996d7640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f677806b500 0x7f677806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.137+0000 7f67996d7640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f677806b500 0x7f677806d9c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6790034000 tx=0x7f679002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.141+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67940ae570 con 0x7f67941616f0 2024-04-25T10:07:01.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.311+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67940cf150 con 0x7f67941616f0 2024-04-25T10:07:01.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.311+0000 7f6789ffb640 1 -- 172.21.15.148:0/1339495917 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v197) v1 ==== 100+0+11 (secure 0 0 0) 0x7f678c058630 con 0x7f67941616f0 2024-04-25T10:07:01.317 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f677806b500 msgr2=0x7f677806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f677806b500 0x7f677806d9c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6790034000 tx=0x7f679002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 msgr2=0x7f67940ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 0x7f67940ac920 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c7a0 tx=0x7f678c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 shutdown_connections 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f677806b500 0x7f677806d9c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67941616f0 0x7f67940ac920 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67941610f0 0x7f67940ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 --2- 172.21.15.148:0/1339495917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67940ce0a0 0x7f67941f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 >> 172.21.15.148:0/1339495917 conn(0x7f679406f500 msgr2=0x7f67940b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 shutdown_connections 2024-04-25T10:07:01.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.313+0000 7f679b93b640 1 -- 172.21.15.148:0/1339495917 wait complete. 2024-04-25T10:07:01.330 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:01.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.422+0000 7feb60530640 1 Processor -- start 2024-04-25T10:07:01.428 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.422+0000 7feb60530640 1 -- start start 2024-04-25T10:07:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.423+0000 7feb60530640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb5815dc20 0x7feb580ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.423+0000 7feb60530640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb58161710 0x7feb580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.423+0000 7feb60530640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5815e000 con 0x7feb580b9330 2024-04-25T10:07:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.423+0000 7feb60530640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb580d1de0 con 0x7feb58161710 2024-04-25T10:07:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.423+0000 7feb60530640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb58161c60 con 0x7feb5815dc20 2024-04-25T10:07:01.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.423+0000 7feb5dacb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb58161710 0x7feb580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5eacd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7feb580b9330 0x7feb5806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51874/0 (socket says 172.21.15.148:51874) 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5eacd640 1 -- 172.21.15.148:0/125219769 learned_addr learned my addr 172.21.15.148:0/125219769 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5e2cc640 1 --2- 172.21.15.148:0/125219769 >> v2:172.21.15.188:3300/0 conn(0x7feb5815dc20 0x7feb580ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5dacb640 1 -- 172.21.15.148:0/125219769 >> v2:172.21.15.188:3300/0 conn(0x7feb5815dc20 msgr2=0x7feb580ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5dacb640 1 --2- 172.21.15.148:0/125219769 >> v2:172.21.15.188:3300/0 conn(0x7feb5815dc20 0x7feb580ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5dacb640 1 -- 172.21.15.148:0/125219769 >> v1:172.21.15.148:6789/0 conn(0x7feb580b9330 legacy=0x7feb5806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5dacb640 1 -- 172.21.15.148:0/125219769 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58161f40 con 0x7feb58161710 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5d2ca640 1 -- 172.21.15.148:0/125219769 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873839683 0 0) 0x7feb5815e000 con 0x7feb580b9330 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.424+0000 7feb5dacb640 1 --2- 172.21.15.148:0/125219769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb58161710 0x7feb580cebb0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7feb580d1520 tx=0x7feb4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd1434a13efb4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb5d2ca640 1 -- 172.21.15.148:0/125219769 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48037070 con 0x7feb58161710 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb5d2ca640 1 -- 172.21.15.148:0/125219769 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48032440 con 0x7feb58161710 2024-04-25T10:07:01.431 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb5d2ca640 1 -- 172.21.15.148:0/125219769 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48032810 con 0x7feb58161710 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 -- 172.21.15.148:0/125219769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb58161710 msgr2=0x7feb580cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 --2- 172.21.15.148:0/125219769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb58161710 0x7feb580cebb0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7feb580d1520 tx=0x7feb4802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 -- 172.21.15.148:0/125219769 shutdown_connections 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 --2- 172.21.15.148:0/125219769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb58161710 0x7feb580cebb0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 --2- 172.21.15.148:0/125219769 >> v2:172.21.15.188:3300/0 conn(0x7feb5815dc20 0x7feb580ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 -- 172.21.15.148:0/125219769 >> 172.21.15.148:0/125219769 conn(0x7feb580d0420 msgr2=0x7feb580d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 -- 172.21.15.148:0/125219769 shutdown_connections 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.425+0000 7feb60530640 1 -- 172.21.15.148:0/125219769 wait complete. 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.426+0000 7feb60530640 1 Processor -- start 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.426+0000 7feb60530640 1 -- start start 2024-04-25T10:07:01.435 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb60530640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb580b9330 0x7feb580c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb60530640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 0x7feb58156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb60530640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb58161710 0x7feb580b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb60530640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb580b5e50 con 0x7feb5815dc20 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb60530640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb58162500 con 0x7feb580b9330 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb60530640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb580b5fd0 con 0x7feb58161710 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb5dacb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 0x7feb58156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb5dacb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 0x7feb58156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33486/0 (socket says 172.21.15.148:33486) 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb5dacb640 1 -- 172.21.15.148:0/739549156 learned_addr learned my addr 172.21.15.148:0/739549156 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb5e2cc640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb580b9330 0x7feb580c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.427+0000 7feb5eacd640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb58161710 0x7feb580b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.436 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5dacb640 1 -- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb58161710 msgr2=0x7feb580b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5dacb640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb58161710 0x7feb580b7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5dacb640 1 -- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb580b9330 msgr2=0x7feb580c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5dacb640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb580b9330 0x7feb580c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5dacb640 1 -- 172.21.15.148:0/739549156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb581559d0 con 0x7feb5815dc20 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5e2cc640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb580b9330 0x7feb580c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5eacd640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb58161710 0x7feb580b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb5dacb640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 0x7feb58156cc0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7feb48000c00 tx=0x7feb480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48037050 con 0x7feb5815dc20 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48038040 con 0x7feb5815dc20 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb58155c70 con 0x7feb5815dc20 2024-04-25T10:07:01.437 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48004470 con 0x7feb5815dc20 2024-04-25T10:07:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.428+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb580c9fb0 con 0x7feb5815dc20 2024-04-25T10:07:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.429+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb58073a00 con 0x7feb5815dc20 2024-04-25T10:07:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.429+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb48030900 con 0x7feb5815dc20 2024-04-25T10:07:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.430+0000 7feb4effd640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb3c06b4d0 0x7feb3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.438 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.430+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb480aaf70 con 0x7feb5815dc20 2024-04-25T10:07:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.430+0000 7feb5e2cc640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb3c06b4d0 0x7feb3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.430+0000 7feb5e2cc640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb3c06b4d0 0x7feb3c06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feb580ce2f0 tx=0x7feb54008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.439 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.434+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb48079d80 con 0x7feb5815dc20 2024-04-25T10:07:01.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.619+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb5815c540 con 0x7feb5815dc20 2024-04-25T10:07:01.625 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.619+0000 7feb4effd640 1 -- 172.21.15.148:0/739549156 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v197) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb48079680 con 0x7feb5815dc20 2024-04-25T10:07:01.625 INFO:teuthology.orchestra.run.smithi148.stderr:listed 0 entries 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb3c06b4d0 msgr2=0x7feb3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb3c06b4d0 0x7feb3c06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feb580ce2f0 tx=0x7feb54008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 msgr2=0x7feb58156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 0x7feb58156cc0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7feb48000c00 tx=0x7feb480046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 shutdown_connections 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb3c06b4d0 0x7feb3c06d990 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb58161710 0x7feb580b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5815dc20 0x7feb58156cc0 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 --2- 172.21.15.148:0/739549156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb580b9330 0x7feb580c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 >> 172.21.15.148:0/739549156 conn(0x7feb580d0420 msgr2=0x7feb580d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 shutdown_connections 2024-04-25T10:07:01.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.622+0000 7feb60530640 1 -- 172.21.15.148:0/739549156 wait complete. 2024-04-25T10:07:01.638 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:01.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.638 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:01.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:07:01.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:01.698 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:01.718 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:01.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.852+0000 7fc863d70640 1 Processor -- start 2024-04-25T10:07:01.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.852+0000 7fc863d70640 1 -- start start 2024-04-25T10:07:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc863d70640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc85c15dc90 0x7fc85c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc863d70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0cc8e0 0x7fc85c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc863d70640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc85c0d1810 0x7fc85c0d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc863d70640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c15f390 con 0x7fc85c15dc90 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc863d70640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0cce50 con 0x7fc85c0d1810 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc863d70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c161af0 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc85c15dc90 0x7fc85c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc85c15dc90 0x7fc85c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33498/0 (socket says 172.21.15.148:33498) 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 -- 172.21.15.148:0/1270443229 learned_addr learned my addr 172.21.15.148:0/1270443229 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc86230d640 1 --2- 172.21.15.148:0/1270443229 >> v2:172.21.15.188:3300/0 conn(0x7fc85c0d1810 0x7fc85c0d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc86130b640 1 --2- 172.21.15.148:0/1270443229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0cc8e0 0x7fc85c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 -- 172.21.15.148:0/1270443229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0cc8e0 msgr2=0x7fc85c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 --2- 172.21.15.148:0/1270443229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0cc8e0 0x7fc85c15ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 -- 172.21.15.148:0/1270443229 >> v2:172.21.15.188:3300/0 conn(0x7fc85c0d1810 msgr2=0x7fc85c0d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 --2- 172.21.15.148:0/1270443229 >> v2:172.21.15.188:3300/0 conn(0x7fc85c0d1810 0x7fc85c0d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.853+0000 7fc861b0c640 1 -- 172.21.15.148:0/1270443229 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c161dd0 con 0x7fc85c15dc90 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.854+0000 7fc861b0c640 1 --2- 172.21.15.148:0/1270443229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15dc90 0x7fc85c1625d0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0d2920 tx=0x7fc85802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4735c35b18b0fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.854+0000 7fc860b0a640 1 -- 172.21.15.148:0/1270443229 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858037070 con 0x7fc85c15dc90 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.854+0000 7fc860b0a640 1 -- 172.21.15.148:0/1270443229 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858032440 con 0x7fc85c15dc90 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.854+0000 7fc860b0a640 1 -- 172.21.15.148:0/1270443229 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858032810 con 0x7fc85c15dc90 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.854+0000 7fc863d70640 1 -- 172.21.15.148:0/1270443229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15dc90 msgr2=0x7fc85c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.854+0000 7fc863d70640 1 --2- 172.21.15.148:0/1270443229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15dc90 0x7fc85c1625d0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0d2920 tx=0x7fc85802a140 comp rx=0 tx=0).stop 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 -- 172.21.15.148:0/1270443229 shutdown_connections 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 --2- 172.21.15.148:0/1270443229 >> v2:172.21.15.188:3300/0 conn(0x7fc85c0d1810 0x7fc85c0d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 --2- 172.21.15.148:0/1270443229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0cc8e0 0x7fc85c15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 --2- 172.21.15.148:0/1270443229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c15dc90 0x7fc85c1625d0 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 -- 172.21.15.148:0/1270443229 >> 172.21.15.148:0/1270443229 conn(0x7fc85c06f590 msgr2=0x7fc85c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 -- 172.21.15.148:0/1270443229 shutdown_connections 2024-04-25T10:07:01.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.855+0000 7fc863d70640 1 -- 172.21.15.148:0/1270443229 wait complete. 2024-04-25T10:07:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.856+0000 7fc863d70640 1 Processor -- start 2024-04-25T10:07:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.856+0000 7fc863d70640 1 -- start start 2024-04-25T10:07:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.856+0000 7fc863d70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 0x7fc85c0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.856+0000 7fc863d70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0d1810 0x7fc85c0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.866 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.856+0000 7fc863d70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c15dc90 0x7fc85c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc863d70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c15e070 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc863d70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0cd130 con 0x7fc85c15dc90 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc863d70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c15e1f0 con 0x7fc85c0d1810 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 0x7fc85c0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 0x7fc85c0cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33508/0 (socket says 172.21.15.148:33508) 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 -- 172.21.15.148:0/2292191493 learned_addr learned my addr 172.21.15.148:0/2292191493 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 -- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0d1810 msgr2=0x7fc85c0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc86130b640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0d1810 0x7fc85c0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0d1810 0x7fc85c0af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 -- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c15dc90 msgr2=0x7fc85c0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc86230d640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c15dc90 0x7fc85c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c15dc90 0x7fc85c0afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 -- 172.21.15.148:0/2292191493 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0b0040 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc861b0c640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 0x7fc85c0cad90 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fc85802a180 tx=0x7fc8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858041020 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8580306c0 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c0b0320 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858037040 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.857+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c0b0bc0 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.858+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c073990 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.859+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc858044020 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.859+0000 7fc8527fc640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.859+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8580aab10 con 0x7fc85c0cc8e0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.859+0000 7fc86130b640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.860+0000 7fc86130b640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc84c006fd0 tx=0x7fc84c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:01.863+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc85804b030 con 0x7fc85c0cc8e0 2024-04-25T10:07:02.073 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.067+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc85c0cf6b0 con 0x7fc85c0cc8e0 2024-04-25T10:07:02.074 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.068+0000 7fc8527fc640 1 -- 172.21.15.148:0/2292191493 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4965 (secure 0 0 0) 0x7fc858079890 con 0x7fc85c0cc8e0 2024-04-25T10:07:02.074 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:07:02.074 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":11619,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:06:55.145145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11634":{"gid":11634,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/1255014956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":1255014956},{"type":"v1","addr":"172.21.15.188:6835","nonce":1255014956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:02.074 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 113 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 msgr2=0x7fc84006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc84c006fd0 tx=0x7fc84c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 msgr2=0x7fc85c0cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 0x7fc85c0cad90 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fc85802a180 tx=0x7fc8580046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 shutdown_connections 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc84006b4d0 0x7fc84006d990 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc85c15dc90 0x7fc85c0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc85c0d1810 0x7fc85c0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 --2- 172.21.15.148:0/2292191493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc85c0cc8e0 0x7fc85c0cad90 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 >> 172.21.15.148:0/2292191493 conn(0x7fc85c06f590 msgr2=0x7fc85c06ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 shutdown_connections 2024-04-25T10:07:02.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.070+0000 7fc863d70640 1 -- 172.21.15.148:0/2292191493 wait complete. 2024-04-25T10:07:02.087 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:07:02.220 INFO:teuthology.orchestra.run.smithi188.stdout:{"message":"","return_code":0} 2024-04-25T10:07:02.220 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:02.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:02.319 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 Processor -- start 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 -- start start 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0d015a9f0 0x7fe0d00c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d00cdbf0 0x7fe0d00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0d00cd520 0x7fe0d00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00b0020 con 0x7fe0d015a9f0 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00cb5a0 con 0x7fe0d00cd520 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.313+0000 7fe0d4f53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00c6940 con 0x7fe0d00cdbf0 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0d015a9f0 0x7fe0d00c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe0d015a9f0 0x7fe0d00c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33520/0 (socket says 172.21.15.148:33520) 2024-04-25T10:07:02.320 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 -- 172.21.15.148:0/3291777650 learned_addr learned my addr 172.21.15.148:0/3291777650 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ced9d640 1 --2- 172.21.15.148:0/3291777650 >> v2:172.21.15.188:3300/0 conn(0x7fe0d00cd520 0x7fe0d00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0c7fff640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d00cdbf0 0x7fe0d00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 -- 172.21.15.148:0/3291777650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d00cdbf0 msgr2=0x7fe0d00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d00cdbf0 0x7fe0d00ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 -- 172.21.15.148:0/3291777650 >> v2:172.21.15.188:3300/0 conn(0x7fe0d00cd520 msgr2=0x7fe0d00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 --2- 172.21.15.148:0/3291777650 >> v2:172.21.15.188:3300/0 conn(0x7fe0d00cd520 0x7fe0d00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ce59c640 1 -- 172.21.15.148:0/3291777650 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d015add0 con 0x7fe0d015a9f0 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0c7fff640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d00cdbf0 0x7fe0d00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.321 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.314+0000 7fe0ced9d640 1 --2- 172.21.15.148:0/3291777650 >> v2:172.21.15.188:3300/0 conn(0x7fe0d00cd520 0x7fe0d00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0ce59c640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d015a9f0 0x7fe0d00c6010 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00b6950 tx=0x7fe0c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45523ce91f1baf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0cdd9b640 1 -- 172.21.15.148:0/3291777650 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0037070 con 0x7fe0d015a9f0 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0cdd9b640 1 -- 172.21.15.148:0/3291777650 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c0032440 con 0x7fe0d015a9f0 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0cdd9b640 1 -- 172.21.15.148:0/3291777650 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0032810 con 0x7fe0d015a9f0 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 -- 172.21.15.148:0/3291777650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d015a9f0 msgr2=0x7fe0d00c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d015a9f0 0x7fe0d00c6010 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00b6950 tx=0x7fe0c002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 -- 172.21.15.148:0/3291777650 shutdown_connections 2024-04-25T10:07:02.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/3291777650 >> v2:172.21.15.188:3300/0 conn(0x7fe0d00cd520 0x7fe0d00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d00cdbf0 0x7fe0d00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/3291777650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d015a9f0 0x7fe0d00c6010 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 -- 172.21.15.148:0/3291777650 >> 172.21.15.148:0/3291777650 conn(0x7fe0d00b2390 msgr2=0x7fe0d00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 -- 172.21.15.148:0/3291777650 shutdown_connections 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.315+0000 7fe0d4f53640 1 -- 172.21.15.148:0/3291777650 wait complete. 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 Processor -- start 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 -- start start 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 0x7fe0d0158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d00cdbf0 0x7fe0d00b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d015a9f0 0x7fe0d01f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00cb230 con 0x7fe0d00cd520 2024-04-25T10:07:02.326 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00caf50 con 0x7fe0d00cdbf0 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0d4f53640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00ce070 con 0x7fe0d015a9f0 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0ce59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 0x7fe0d0158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0ce59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 0x7fe0d0158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33528/0 (socket says 172.21.15.148:33528) 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.317+0000 7fe0ce59c640 1 -- 172.21.15.148:0/1482155118 learned_addr learned my addr 172.21.15.148:0/1482155118 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0c7fff640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d00cdbf0 0x7fe0d00b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ce59c640 1 -- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d015a9f0 msgr2=0x7fe0d01f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ced9d640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d015a9f0 0x7fe0d01f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ce59c640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d015a9f0 0x7fe0d01f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ce59c640 1 -- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d00cdbf0 msgr2=0x7fe0d00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ce59c640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d00cdbf0 0x7fe0d00b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ce59c640 1 -- 172.21.15.148:0/1482155118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d0158280 con 0x7fe0d00cd520 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ced9d640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d015a9f0 0x7fe0d01f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0c7fff640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d00cdbf0 0x7fe0d00b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0ce59c640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 0x7fe0d0158560 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fe0c002a220 tx=0x7fe0c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0041020 con 0x7fe0d00cd520 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c00306c0 con 0x7fe0d00cd520 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d00c94a0 con 0x7fe0d00cd520 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0037040 con 0x7fe0d00cd520 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.318+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d00c9a60 con 0x7fe0d00cd520 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.319+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d006cc80 con 0x7fe0d00cd520 2024-04-25T10:07:02.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.320+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0c0044020 con 0x7fe0d00cd520 2024-04-25T10:07:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.320+0000 7fe0c6ffd640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0b006b4d0 0x7fe0b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.320+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0c00aab20 con 0x7fe0d00cd520 2024-04-25T10:07:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.320+0000 7fe0c7fff640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0b006b4d0 0x7fe0b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.329 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.321+0000 7fe0c7fff640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0b006b4d0 0x7fe0b006d990 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00afb20 tx=0x7fe0b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.330 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.324+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0c004b030 con 0x7fe0d00cd520 2024-04-25T10:07:02.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.532+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe0d0073a00 con 0x7fe0d00cd520 2024-04-25T10:07:02.538 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:07:02.532+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:02.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.534+0000 7fe0c6ffd640 1 -- 172.21.15.148:0/1482155118 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe0c0079810 con 0x7fe0d00cd520 2024-04-25T10:07:02.539 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0b006b4d0 msgr2=0x7fe0b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0b006b4d0 0x7fe0b006d990 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00afb20 tx=0x7fe0b8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 msgr2=0x7fe0d0158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 0x7fe0d0158560 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fe0c002a220 tx=0x7fe0c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 shutdown_connections 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0b006b4d0 0x7fe0b006d990 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0d015a9f0 0x7fe0d01f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0d00cdbf0 0x7fe0d00b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 --2- 172.21.15.148:0/1482155118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0d00cd520 0x7fe0d0158560 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 >> 172.21.15.148:0/1482155118 conn(0x7fe0d00b2390 msgr2=0x7fe0d00cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 shutdown_connections 2024-04-25T10:07:02.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.537+0000 7fe0d4f53640 1 -- 172.21.15.148:0/1482155118 wait complete. 2024-04-25T10:07:02.544 INFO:tasks.ceph.mds.d.smithi188.stderr: -5262> 2024-04-25T10:06:58.770+0000 7fb201eb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:02.553 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T10:07:02.574 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.590+0000 7fe0f0d6ff40 1 Processor -- start 2024-04-25T10:07:02.596 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.590+0000 7fe0f0d6ff40 1 -- start start 2024-04-25T10:07:02.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0f0d6ff40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0f0d6ff40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633454cf0a0 0x5633454b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.601 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0f0d6ff40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563345440510 0x5633454409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0f0d6ff40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633453f89a0 con 0x5633453f5fb0 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0f0d6ff40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334539edf0 con 0x563345440510 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0f0d6ff40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633454411a0 con 0x5633454cf0a0 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33542/0 (socket says 172.21.15.148:33542) 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 -- 172.21.15.148:0/2617638906 learned_addr learned my addr 172.21.15.148:0/2617638906 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee06f640 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633454cf0a0 0x5633454b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ef071640 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563345440510 0x5633454409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 -- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633454cf0a0 msgr2=0x5633454b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633454cf0a0 0x5633454b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 -- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563345440510 msgr2=0x5633454409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563345440510 0x5633454409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 -- 172.21.15.148:0/2617638906 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563345324c50 con 0x5633453f5fb0 2024-04-25T10:07:02.602 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ee870640 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454c3b10 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0055040 tx=0x7fe0e0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b697aa2263e99df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ed06d640 1 -- 172.21.15.148:0/2617638906 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0069070 con 0x5633453f5fb0 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ed06d640 1 -- 172.21.15.148:0/2617638906 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e0052ad0 con 0x5633453f5fb0 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.591+0000 7fe0ed06d640 1 -- 172.21.15.148:0/2617638906 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e005b420 con 0x5633453f5fb0 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/2617638906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 msgr2=0x5633454c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454c3b10 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0055040 tx=0x7fe0e0051000 comp rx=0 tx=0).stop 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/2617638906 shutdown_connections 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563345440510 0x5633454409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5633454cf0a0 0x5633454b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/2617638906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454c3b10 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/2617638906 >> 172.21.15.148:0/2617638906 conn(0x5633453f69b0 msgr2=0x56334543b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/2617638906 shutdown_connections 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.592+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/2617638906 wait complete. 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 Processor -- start 2024-04-25T10:07:02.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 -- start start 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563345440510 0x56334543d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633454cf0a0 0x56334543d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334543c940 con 0x5633453f5fb0 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56334543c2f0 con 0x5633454cf0a0 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0f0d6ff40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633454cb250 con 0x563345440510 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0ee870640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0ee870640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33544/0 (socket says 172.21.15.148:33544) 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0ee870640 1 -- 172.21.15.148:0/815850056 learned_addr learned my addr 172.21.15.148:0/815850056 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0ee06f640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563345440510 0x56334543d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.593+0000 7fe0ef071640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633454cf0a0 0x56334543d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee870640 1 -- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563345440510 msgr2=0x56334543d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee870640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563345440510 0x56334543d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee870640 1 -- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633454cf0a0 msgr2=0x56334543d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.604 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee870640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633454cf0a0 0x56334543d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee870640 1 -- 172.21.15.148:0/815850056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633454d0a00 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee06f640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563345440510 0x56334543d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ef071640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5633454cf0a0 0x56334543d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0ee870640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454b4ac0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0045b00 tx=0x7fe0e0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0d7fff640 1 -- 172.21.15.148:0/815850056 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0069050 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0d7fff640 1 -- 172.21.15.148:0/815850056 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e006a040 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0d7fff640 1 -- 172.21.15.148:0/815850056 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0070690 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5633453f7f90 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.594+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633454b5000 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.595+0000 7fe0d7fff640 1 -- 172.21.15.148:0/815850056 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0e0059020 con 0x5633453f5fb0 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0d7fff640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0dc069eb0 0x7fe0dc06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.605 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0ee06f640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0dc069eb0 0x7fe0dc06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0d7fff640 1 -- 172.21.15.148:0/815850056 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe0e00ce3d0 con 0x5633453f5fb0 2024-04-25T10:07:02.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5633454d1e00 0x56334555e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e198) v9 -- 0x5633454c66e0 con 0x5633454d1e00 2024-04-25T10:07:02.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0ee06f640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0dc069eb0 0x7fe0dc06c370 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x56334543ddd0 tx=0x7fe0e4005940 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0ef071640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5633454d1e00 0x56334555e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.596+0000 7fe0ef071640 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5633454d1e00 0x56334555e630 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.607 INFO:teuthology.orchestra.run.smithi148.stderr:Writing to - 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.598+0000 7fe0ef071640 1 -- 172.21.15.148:0/815850056 <== osd.6 v2:172.21.15.188:6816/1362851547 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=588b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+588 (crc 0 0 0) 0x7fe0e8011040 con 0x5633454d1e00 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.598+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5633454d1e00 msgr2=0x56334555e630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.598+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x5633454d1e00 0x56334555e630 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0dc069eb0 msgr2=0x7fe0dc06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe0dc069eb0 0x7fe0dc06c370 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x56334543ddd0 tx=0x7fe0e4005940 comp rx=0 tx=0).stop 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 msgr2=0x5633454b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 --2- 172.21.15.148:0/815850056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633453f5fb0 0x5633454b4ac0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0045b00 tx=0x7fe0e0002fb0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0ef071640 1 -- 172.21.15.148:0/815850056 reap_dead start 2024-04-25T10:07:02.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 shutdown_connections 2024-04-25T10:07:02.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 >> 172.21.15.148:0/815850056 conn(0x5633453f69b0 msgr2=0x56334543b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 shutdown_connections 2024-04-25T10:07:02.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.599+0000 7fe0f0d6ff40 1 -- 172.21.15.148:0/815850056 wait complete. 2024-04-25T10:07:02.609 INFO:teuthology.orchestra.run.smithi148.stdout:i"�C+*f�:G)���$@@`��������C+*f�:G)C+*f�Q%((8`C+*f�:G)8`C+*f�:G)��������C+*f�Q%(����security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 2024-04-25T10:07:02.609 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T10:07:02.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.685+0000 7f9d2a9e6f40 1 Processor -- start 2024-04-25T10:07:02.691 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.685+0000 7f9d2a9e6f40 1 -- start start 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d2a9e6f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 0x564b716d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d2a9e6f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 0x564b717189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d2a9e6f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8d00 0x564b717a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d2a9e6f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b7171bf30 con 0x564b717a8060 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d2a9e6f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b71679df0 con 0x564b716d08a0 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d2a9e6f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b7171c1d0 con 0x564b717a8d00 2024-04-25T10:07:02.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 0x564b717189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 0x564b717189c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33564/0 (socket says 172.21.15.148:33564) 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 -- 172.21.15.148:0/2314500037 learned_addr learned my addr 172.21.15.148:0/2314500037 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 -- 172.21.15.148:0/2314500037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8d00 msgr2=0x564b717a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8d00 0x564b717a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 -- 172.21.15.148:0/2314500037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 msgr2=0x564b716d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 0x564b716d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 -- 172.21.15.148:0/2314500037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b715ffc50 con 0x564b717a8060 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d27ce6640 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 0x564b717189c0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00a440 tx=0x7f9d1c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30d29b69e29cad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d26ce4640 1 -- 172.21.15.148:0/2314500037 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0538f0 con 0x564b717a8060 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d26ce4640 1 -- 172.21.15.148:0/2314500037 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c053a90 con 0x564b717a8060 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.686+0000 7f9d26ce4640 1 -- 172.21.15.148:0/2314500037 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c053e40 con 0x564b717a8060 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/2314500037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 msgr2=0x564b717189c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 0x564b717189c0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00a440 tx=0x7f9d1c051000 comp rx=0 tx=0).stop 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/2314500037 shutdown_connections 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8d00 0x564b717a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8060 0x564b717189c0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/2314500037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 0x564b716d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/2314500037 >> 172.21.15.148:0/2314500037 conn(0x564b716d19b0 msgr2=0x564b716fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/2314500037 shutdown_connections 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/2314500037 wait complete. 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 Processor -- start 2024-04-25T10:07:02.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.687+0000 7f9d2a9e6f40 1 -- start start 2024-04-25T10:07:02.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d2a9e6f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 0x564b7179ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d2a9e6f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8060 0x564b717a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d2a9e6f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8d00 0x564b717a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d2a9e6f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b716fe890 con 0x564b717a8d00 2024-04-25T10:07:02.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d2a9e6f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b71718f00 con 0x564b716d08a0 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d2a9e6f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b716fd320 con 0x564b717a8060 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d28ce8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8d00 0x564b717a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d28ce8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8d00 0x564b717a2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33566/0 (socket says 172.21.15.148:33566) 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d28ce8640 1 -- 172.21.15.148:0/3959361771 learned_addr learned my addr 172.21.15.148:0/3959361771 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d284e7640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 0x564b7179ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.688+0000 7f9d27ce6640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8060 0x564b717a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d28ce8640 1 -- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8060 msgr2=0x564b717a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d28ce8640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564b717a8060 0x564b717a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d28ce8640 1 -- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 msgr2=0x564b7179ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d28ce8640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564b716d08a0 0x564b7179ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d28ce8640 1 -- 172.21.15.148:0/3959361771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b716d22a0 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d28ce8640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8d00 0x564b717a2630 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000af60 tx=0x7f9d2000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d25ce2640 1 -- 172.21.15.148:0/3959361771 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200067a0 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d25ce2640 1 -- 172.21.15.148:0/3959361771 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d20006940 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564b716d1fc0 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d25ce2640 1 -- 172.21.15.148:0/3959361771 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20006d30 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.689+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b717a4dd0 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.690+0000 7f9d25ce2640 1 -- 172.21.15.148:0/3959361771 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d2001c440 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d25ce2640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d14069eb0 0x7f9d1406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d25ce2640 1 -- 172.21.15.148:0/3959361771 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9d20088ed0 con 0x564b717a8d00 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d284e7640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d14069eb0 0x7f9d1406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x564b7183e730 0x564b7183ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198) v9 -- 0x564b7183f120 con 0x564b7183e730 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d27ce6640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x564b7183e730 0x564b7183ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d284e7640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d14069eb0 0x7f9d1406c370 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9d180012e0 tx=0x7f9d180069f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.691+0000 7f9d27ce6640 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x564b7183e730 0x564b7183ebe0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d27ce6640 1 -- 172.21.15.148:0/3959361771 <== osd.6 v2:172.21.15.188:6816/1362851547 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v198'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9d1c067510 con 0x564b7183e730 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x564b7183e730 msgr2=0x564b7183ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x564b7183e730 0x564b7183ebe0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d14069eb0 msgr2=0x7f9d1406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f9d14069eb0 0x7f9d1406c370 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9d180012e0 tx=0x7f9d180069f0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8d00 msgr2=0x564b717a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.692+0000 7f9d2a9e6f40 1 --2- 172.21.15.148:0/3959361771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b717a8d00 0x564b717a2630 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000af60 tx=0x7f9d2000af90 comp rx=0 tx=0).stop 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.693+0000 7f9d28ce8640 1 -- 172.21.15.148:0/3959361771 reap_dead start 2024-04-25T10:07:02.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.693+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 shutdown_connections 2024-04-25T10:07:02.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.693+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 >> 172.21.15.148:0/3959361771 conn(0x564b716d19b0 msgr2=0x564b717186b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.693+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 shutdown_connections 2024-04-25T10:07:02.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.693+0000 7f9d2a9e6f40 1 -- 172.21.15.148:0/3959361771 wait complete. 2024-04-25T10:07:02.704 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T10:07:02.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.839+0000 7ffaaf729640 1 Processor -- start 2024-04-25T10:07:02.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.839+0000 7ffaaf729640 1 -- start start 2024-04-25T10:07:02.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaaf729640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80b9750 0x7ffaa80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaaf729640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8161d70 con 0x7ffaa80cebb0 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaaf729640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa80cf020 con 0x7ffaa815da90 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaaf729640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8162600 con 0x7ffaa80b9750 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaadcc6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ffaa80cebb0 0x7ffaa80d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:51886/0 (socket says 172.21.15.148:51886) 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaadcc6640 1 -- 172.21.15.148:0/3884988863 learned_addr learned my addr 172.21.15.148:0/3884988863 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750895966 0 0) 0x7ffaa8161d70 con 0x7ffaa80cebb0 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa94003520 con 0x7ffaa80cebb0 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.840+0000 7ffaaccc4640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80b9750 0x7ffaa80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4123787746 0 0) 0x7ffa94003520 con 0x7ffaa80cebb0 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80b9750 msgr2=0x7ffaa80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80b9750 0x7ffaa80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 >> v1:172.21.15.188:6789/0 conn(0x7ffaa815da90 legacy=0x7ffaa8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8162040 con 0x7ffaa80cebb0 2024-04-25T10:07:02.849 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7ffaa00019f0 con 0x7ffaa80cebb0 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 >> v1:172.21.15.148:6789/0 conn(0x7ffaa80cebb0 legacy=0x7ffaa80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa940039e0 0x7ffa94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa94004800 0x7ffa94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa94005180 0x7ffa9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8162600 con 0x7ffa940039e0 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9400bf00 con 0x7ffa94005180 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9400c100 con 0x7ffa94004800 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa940039e0 0x7ffa94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaaccc4640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa94004800 0x7ffa94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaadcc6640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa94005180 0x7ffa9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaad4c5640 1 -- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa94004800 msgr2=0x7ffa94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa94004800 0x7ffa94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaad4c5640 1 -- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa94005180 msgr2=0x7ffa9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa94005180 0x7ffa9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.841+0000 7ffaad4c5640 1 -- 172.21.15.148:0/3884988863 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94003520 con 0x7ffa940039e0 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.842+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa940039e0 0x7ffa94003e00 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4001750 tx=0x7ffa94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d427fb284917bd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.842+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4001370 con 0x7ffa940039e0 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.842+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa40052d0 con 0x7ffa940039e0 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.842+0000 7ffa9ffff640 1 -- 172.21.15.148:0/3884988863 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa402abe0 con 0x7ffa940039e0 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaaf729640 1 -- 172.21.15.148:0/3884988863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa940039e0 msgr2=0x7ffa94003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaaf729640 1 --2- 172.21.15.148:0/3884988863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa940039e0 0x7ffa94003e00 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4001750 tx=0x7ffa94005830 comp rx=0 tx=0).stop 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaadcc6640 1 -- 172.21.15.148:0/3884988863 reap_dead start 2024-04-25T10:07:02.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaaf729640 1 -- 172.21.15.148:0/3884988863 shutdown_connections 2024-04-25T10:07:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaaf729640 1 -- 172.21.15.148:0/3884988863 >> 172.21.15.148:0/3884988863 conn(0x7ffaa8061a20 msgr2=0x7ffaa8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaaf729640 1 -- 172.21.15.148:0/3884988863 shutdown_connections 2024-04-25T10:07:02.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.843+0000 7ffaaf729640 1 -- 172.21.15.148:0/3884988863 wait complete. 2024-04-25T10:07:02.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.844+0000 7ffaaf729640 1 Processor -- start 2024-04-25T10:07:02.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.844+0000 7ffaaf729640 1 -- start start 2024-04-25T10:07:02.854 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa940039e0 0x7ffaa8157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 0x7ffaa80cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80c9ee0 0x7ffaa8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80d1df0 con 0x7ffaa80c88a0 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80d1b10 con 0x7ffa940039e0 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa815de70 con 0x7ffaa80c9ee0 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 0x7ffaa80cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 0x7ffaa80cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33586/0 (socket says 172.21.15.148:33586) 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 -- 172.21.15.148:0/1988668221 learned_addr learned my addr 172.21.15.148:0/1988668221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa940039e0 0x7ffaa8157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaadcc6640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80c9ee0 0x7ffaa8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 -- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80c9ee0 msgr2=0x7ffaa8159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80c9ee0 0x7ffaa8159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 -- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa940039e0 msgr2=0x7ffaa8157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.855 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa940039e0 0x7ffaa8157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 -- 172.21.15.148:0/1988668221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa80c8e40 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaccc4640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 0x7ffaa80cbe60 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7ffa98011930 tx=0x7ffa98011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800fa30 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9800fbd0 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa80c9120 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.845+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98014440 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.846+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa80c96e0 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.846+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8073a00 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.847+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa98012040 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.847+0000 7ffa9dffb640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.847+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ffa9808a890 con 0x7ffaa80c88a0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.847+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.848+0000 7ffaad4c5640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4032000 tx=0x7ffaa402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:02.852+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa98019030 con 0x7ffaa80c88a0 2024-04-25T10:07:03.031 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.026+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7ffaa8061e30 con 0x7ffaa80c88a0 2024-04-25T10:07:03.034 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.029+0000 7ffa9dffb640 1 -- 172.21.15.148:0/1988668221 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v35)=0 v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7ffa98058f70 con 0x7ffaa80c88a0 2024-04-25T10:07:03.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b500 msgr2=0x7ffa8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4032000 tx=0x7ffaa402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 msgr2=0x7ffaa80cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 0x7ffaa80cbe60 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7ffa98011930 tx=0x7ffa98011df0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 shutdown_connections 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffa8c06b500 0x7ffa8c06d9c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa80c9ee0 0x7ffaa8159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaa80c88a0 0x7ffaa80cbe60 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 --2- 172.21.15.148:0/1988668221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa940039e0 0x7ffaa8157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.031+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 >> 172.21.15.148:0/1988668221 conn(0x7ffaa8061a20 msgr2=0x7ffaa80d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.032+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 shutdown_connections 2024-04-25T10:07:03.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.032+0000 7ffaaf729640 1 -- 172.21.15.148:0/1988668221 wait complete. 2024-04-25T10:07:03.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:07:03.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.143+0000 7fe3b0fce640 1 Processor -- start 2024-04-25T10:07:03.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.143+0000 7fe3b0fce640 1 -- start start 2024-04-25T10:07:03.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3b0fce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe3ac0b4f10 0x7fe3ac0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3b0fce640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3ac0af5e0 0x7fe3ac06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3b0fce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ac0b52f0 con 0x7fe3ac15f630 2024-04-25T10:07:03.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3b0fce640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ac06bc40 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3b0fce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ac0adaa0 con 0x7fe3ac0b4f10 2024-04-25T10:07:03.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3ac0af5e0 0x7fe3ac06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fe3ac0af5e0 0x7fe3ac06c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33588/0 (socket says 172.21.15.148:33588) 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 -- 172.21.15.148:0/3528561876 learned_addr learned my addr 172.21.15.148:0/3528561876 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3aa59c640 1 --2- 172.21.15.148:0/3528561876 >> v2:172.21.15.188:3300/0 conn(0x7fe3ac0b4f10 0x7fe3ac0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 -- 172.21.15.148:0/3528561876 >> v2:172.21.15.188:3300/0 conn(0x7fe3ac0b4f10 msgr2=0x7fe3ac0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 --2- 172.21.15.148:0/3528561876 >> v2:172.21.15.188:3300/0 conn(0x7fe3ac0b4f10 0x7fe3ac0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 -- 172.21.15.148:0/3528561876 >> v1:172.21.15.188:6789/0 conn(0x7fe3ac15f630 legacy=0x7fe3ac0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.144+0000 7fe3a3fff640 1 -- 172.21.15.148:0/3528561876 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0add80 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3a3fff640 1 --2- 172.21.15.148:0/3528561876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac06c0d0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac072e50 tx=0x7fe39402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e52df17ac1e25b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3a9d9b640 1 -- 172.21.15.148:0/3528561876 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394037070 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3a9d9b640 1 -- 172.21.15.148:0/3528561876 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394032440 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3a9d9b640 1 -- 172.21.15.148:0/3528561876 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394032810 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3528561876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 msgr2=0x7fe3ac06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3528561876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac06c0d0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac072e50 tx=0x7fe39402a230 comp rx=0 tx=0).stop 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3528561876 shutdown_connections 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3528561876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac06c0d0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.145+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3528561876 >> v2:172.21.15.188:3300/0 conn(0x7fe3ac0b4f10 0x7fe3ac0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.146+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3528561876 >> 172.21.15.148:0/3528561876 conn(0x7fe3ac0cfec0 msgr2=0x7fe3ac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.146+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3528561876 shutdown_connections 2024-04-25T10:07:03.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.146+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3528561876 wait complete. 2024-04-25T10:07:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.147+0000 7fe3b0fce640 1 Processor -- start 2024-04-25T10:07:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.147+0000 7fe3b0fce640 1 -- start start 2024-04-25T10:07:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3ac0b4f10 0x7fe3ac158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3ac15f630 0x7fe3ac1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac0ad730 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac06b8d0 con 0x7fe3ac0b4f10 2024-04-25T10:07:03.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac15fa60 con 0x7fe3ac15f630 2024-04-25T10:07:03.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33598/0 (socket says 172.21.15.148:33598) 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 -- 172.21.15.148:0/3482394653 learned_addr learned my addr 172.21.15.148:0/3482394653 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3a3fff640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3ac0b4f10 0x7fe3ac158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aad9d640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3ac15f630 0x7fe3ac1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 -- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3ac15f630 msgr2=0x7fe3ac1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3ac15f630 0x7fe3ac1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 -- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3ac0b4f10 msgr2=0x7fe3ac158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3ac0b4f10 0x7fe3ac158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 -- 172.21.15.148:0/3482394653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac1572d0 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aad9d640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3ac15f630 0x7fe3ac1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3aa59c640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac145a30 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00d800 tx=0x7fe39c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00b6c0 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c019070 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac1575b0 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00ba80 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.148+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac157b70 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.149+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac0cfa40 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.150+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe39c00bc20 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.150+0000 7fe3a2ffd640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38c06b4d0 0x7fe38c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.150+0000 7fe3a3fff640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38c06b4d0 0x7fe38c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.150+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe39c08a8a0 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.151+0000 7fe3a3fff640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38c06b4d0 0x7fe38c06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe394034000 tx=0x7fe39402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.161 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.154+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3ac0cfa40 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.360+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe3ac0aef70 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.535+0000 7fe3a2ffd640 1 -- 172.21.15.148:0/3482394653 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v115) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe3ac0aef70 con 0x7fe3ac0af5e0 2024-04-25T10:07:03.541 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:07:03.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38c06b4d0 msgr2=0x7fe38c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38c06b4d0 0x7fe38c06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe394034000 tx=0x7fe39402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 msgr2=0x7fe3ac145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac145a30 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00d800 tx=0x7fe39c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.544 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 shutdown_connections 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fe38c06b4d0 0x7fe38c06d990 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3ac15f630 0x7fe3ac1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3ac0b4f10 0x7fe3ac158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 --2- 172.21.15.148:0/3482394653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3ac0af5e0 0x7fe3ac145a30 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 >> 172.21.15.148:0/3482394653 conn(0x7fe3ac0cfec0 msgr2=0x7fe3ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 shutdown_connections 2024-04-25T10:07:03.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.538+0000 7fe3b0fce640 1 -- 172.21.15.148:0/3482394653 wait complete. 2024-04-25T10:07:03.557 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.648+0000 7f36120ac640 1 Processor -- start 2024-04-25T10:07:03.654 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.648+0000 7f36120ac640 1 -- start start 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f36120ac640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f360c0d0170 0x7f360c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f36120ac640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f360c0ce700 0x7f360c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f36120ac640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0c8030 con 0x7f360c15da70 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f36120ac640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0ceb30 con 0x7f360c0ce700 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f36120ac640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360c0c8310 con 0x7f360c0d0170 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f360c0ce700 0x7f360c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f360c0ce700 0x7f360c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53464/0 (socket says 172.21.15.148:53464) 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 -- 172.21.15.148:0/311269320 learned_addr learned my addr 172.21.15.148:0/311269320 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360affd640 1 --2- 172.21.15.148:0/311269320 >> v2:172.21.15.188:3300/0 conn(0x7f360c0d0170 0x7f360c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 -- 172.21.15.148:0/311269320 >> v2:172.21.15.188:3300/0 conn(0x7f360c0d0170 msgr2=0x7f360c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 --2- 172.21.15.148:0/311269320 >> v2:172.21.15.188:3300/0 conn(0x7f360c0d0170 0x7f360c0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 -- 172.21.15.148:0/311269320 >> v1:172.21.15.188:6789/0 conn(0x7f360c15da70 legacy=0x7f360c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.649+0000 7f360bfff640 1 -- 172.21.15.148:0/311269320 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0c85f0 con 0x7f360c0ce700 2024-04-25T10:07:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.650+0000 7f360bfff640 1 --2- 172.21.15.148:0/311269320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0ce700 0x7f360c06c330 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f360c0b1040 tx=0x7f360402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6deaf84c02fc62f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.650+0000 7f360a7fc640 1 -- 172.21.15.148:0/311269320 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604036070 con 0x7f360c0ce700 2024-04-25T10:07:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.650+0000 7f360a7fc640 1 -- 172.21.15.148:0/311269320 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604031600 con 0x7f360c0ce700 2024-04-25T10:07:03.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.650+0000 7f360a7fc640 1 -- 172.21.15.148:0/311269320 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36040319d0 con 0x7f360c0ce700 2024-04-25T10:07:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 -- 172.21.15.148:0/311269320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0ce700 msgr2=0x7f360c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 --2- 172.21.15.148:0/311269320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0ce700 0x7f360c06c330 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f360c0b1040 tx=0x7f360402a730 comp rx=0 tx=0).stop 2024-04-25T10:07:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 -- 172.21.15.148:0/311269320 shutdown_connections 2024-04-25T10:07:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 --2- 172.21.15.148:0/311269320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0ce700 0x7f360c06c330 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 --2- 172.21.15.148:0/311269320 >> v2:172.21.15.188:3300/0 conn(0x7f360c0d0170 0x7f360c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 -- 172.21.15.148:0/311269320 >> 172.21.15.148:0/311269320 conn(0x7f360c0b33e0 msgr2=0x7f360c0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 -- 172.21.15.148:0/311269320 shutdown_connections 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.651+0000 7f36120ac640 1 -- 172.21.15.148:0/311269320 wait complete. 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.652+0000 7f36120ac640 1 Processor -- start 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.652+0000 7f36120ac640 1 -- start start 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c0ce700 0x7f360c14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 0x7f360c15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c15da70 0x7f360c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c15de50 con 0x7f360c0d0170 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c06be10 con 0x7f360c15da70 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c06c870 con 0x7f360c0ce700 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 0x7f360c15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 0x7f360c15bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53472/0 (socket says 172.21.15.148:53472) 2024-04-25T10:07:03.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 -- 172.21.15.148:0/11080397 learned_addr learned my addr 172.21.15.148:0/11080397 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360bfff640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c15da70 0x7f360c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360b7fe640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c0ce700 0x7f360c14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 -- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c0ce700 msgr2=0x7f360c14d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c0ce700 0x7f360c14d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 -- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c15da70 msgr2=0x7f360c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c15da70 0x7f360c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 -- 172.21.15.148:0/11080397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c14d8d0 con 0x7f360c0d0170 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f360affd640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 0x7f360c15bc00 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f360000d830 tx=0x7f360000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360000b6c0 con 0x7f360c0d0170 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3600019070 con 0x7f360c0d0170 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c0b4570 con 0x7f360c0d0170 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360000ba80 con 0x7f360c0d0170 2024-04-25T10:07:03.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.653+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c0b4b30 con 0x7f360c0d0170 2024-04-25T10:07:03.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.654+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c073990 con 0x7f360c0d0170 2024-04-25T10:07:03.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.655+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f360001e460 con 0x7f360c0d0170 2024-04-25T10:07:03.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.655+0000 7f35f3fff640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35f806b4d0 0x7f35f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.655+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f360008aa10 con 0x7f360c0d0170 2024-04-25T10:07:03.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.655+0000 7f360b7fe640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35f806b4d0 0x7f35f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.656+0000 7f360b7fe640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35f806b4d0 0x7f35f806d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007fd0 tx=0x7f35fc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.660+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3600059740 con 0x7f360c0d0170 2024-04-25T10:07:03.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.867+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f360c0cfc50 con 0x7f360c0d0170 2024-04-25T10:07:03.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.867+0000 7f35f3fff640 1 -- 172.21.15.148:0/11080397 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4966 (secure 0 0 0) 0x7f3600059040 con 0x7f360c0d0170 2024-04-25T10:07:03.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:07:03.873 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:03.539420+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:03.873 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 116 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35f806b4d0 msgr2=0x7f35f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35f806b4d0 0x7f35f806d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f35fc007fd0 tx=0x7f35fc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 msgr2=0x7f360c15bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 0x7f360c15bc00 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f360000d830 tx=0x7f360000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 shutdown_connections 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f35f806b4d0 0x7f35f806d990 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f360c15da70 0x7f360c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f360c0d0170 0x7f360c15bc00 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 --2- 172.21.15.148:0/11080397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f360c0ce700 0x7f360c14d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.875 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 >> 172.21.15.148:0/11080397 conn(0x7f360c0b33e0 msgr2=0x7f360c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 shutdown_connections 2024-04-25T10:07:03.876 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:03.869+0000 7f36120ac640 1 -- 172.21.15.148:0/11080397 wait complete. 2024-04-25T10:07:04.289 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 116, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:52.824899+0000', 'modified': '2024-04-25T10:07:03.539420+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11619': {'gid': 11619, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.148:6837/1465938320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1465938320}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1465938320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:04.289 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11619:up:replay 2024-04-25T10:07:05.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:05.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.394+0000 7fab2436a640 1 Processor -- start 2024-04-25T10:07:05.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.394+0000 7fab2436a640 1 -- start start 2024-04-25T10:07:05.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab2436a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab2436a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0d1dd0 0x7fab1c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab2436a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c15dfe0 con 0x7fab1c0ceb20 2024-04-25T10:07:05.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab2436a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c0d00c0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab2436a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0d03a0 con 0x7fab1c15dc00 2024-04-25T10:07:05.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0d1dd0 0x7fab1c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fab1c0d1dd0 0x7fab1c0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53486/0 (socket says 172.21.15.148:53486) 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab21905640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fab1c0ceb20 0x7fab1c0d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:49592/0 (socket says 172.21.15.148:49592) 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 -- 172.21.15.148:0/7937826 learned_addr learned my addr 172.21.15.148:0/7937826 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22106640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 -- 172.21.15.148:0/7937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 msgr2=0x7fab1c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 -- 172.21.15.148:0/7937826 >> v1:172.21.15.148:6789/0 conn(0x7fab1c0ceb20 legacy=0x7fab1c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab21104640 1 -- 172.21.15.148:0/7937826 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350919108 0 0) 0x7fab0c001110 con 0x7fab1c0ceb20 2024-04-25T10:07:05.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22907640 1 -- 172.21.15.148:0/7937826 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0cc440 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.395+0000 7fab22106640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.396+0000 7fab22907640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c0ccc40 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e50 tx=0x7fab1402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c079eaaa0a086c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.396+0000 7fab21104640 1 -- 172.21.15.148:0/7937826 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14036070 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.396+0000 7fab21104640 1 -- 172.21.15.148:0/7937826 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab14031600 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.396+0000 7fab21104640 1 -- 172.21.15.148:0/7937826 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140319d0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 -- 172.21.15.148:0/7937826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 msgr2=0x7fab1c0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c0ccc40 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fab14009e50 tx=0x7fab1402a710 comp rx=0 tx=0).stop 2024-04-25T10:07:05.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 -- 172.21.15.148:0/7937826 shutdown_connections 2024-04-25T10:07:05.409 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c0ccc40 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 --2- 172.21.15.148:0/7937826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 -- 172.21.15.148:0/7937826 >> 172.21.15.148:0/7937826 conn(0x7fab1c06f5d0 msgr2=0x7fab1c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 -- 172.21.15.148:0/7937826 shutdown_connections 2024-04-25T10:07:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.397+0000 7fab2436a640 1 -- 172.21.15.148:0/7937826 wait complete. 2024-04-25T10:07:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.398+0000 7fab2436a640 1 Processor -- start 2024-04-25T10:07:05.410 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.398+0000 7fab2436a640 1 -- start start 2024-04-25T10:07:05.414 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0ceb20 0x7fab1c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cfd90 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cc130 con 0x7fab1c0ceb20 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cdd70 con 0x7fab1c15dc00 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c1f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53488/0 (socket says 172.21.15.148:53488) 2024-04-25T10:07:05.415 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 -- 172.21.15.148:0/4281617683 learned_addr learned my addr 172.21.15.148:0/4281617683 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab22106640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0ceb20 0x7fab1c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab22907640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 -- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 msgr2=0x7fab1c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 -- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0ceb20 msgr2=0x7fab1c152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0ceb20 0x7fab1c152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 -- 172.21.15.148:0/4281617683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0b6d20 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.416 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab22106640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0ceb20 0x7fab1c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab22907640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab21905640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c1f7930 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c7b0 tx=0x7fab0c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c018070 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c0072d0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c0b7000 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0047e0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.399+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1f7410 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.401+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c073990 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.401+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab0c01b070 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.401+0000 7fab12ffd640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fab0006b4d0 0x7fab0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.401+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fab0c089700 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.401+0000 7fab22106640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fab0006b4d0 0x7fab0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.402+0000 7fab22106640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fab0006b4d0 0x7fab0006d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.417 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.406+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab1c15dfe0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.614+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab1c15afb0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.620 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.615+0000 7fab12ffd640 1 -- 172.21.15.148:0/4281617683 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4966 (secure 0 0 0) 0x7fab0c0583a0 con 0x7fab1c0d1dd0 2024-04-25T10:07:05.620 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:07:05.621 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:04.544018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:05.621 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 118 2024-04-25T10:07:05.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.616+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fab0006b4d0 msgr2=0x7fab0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.616+0000 7fab2436a640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fab0006b4d0 0x7fab0006d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fab18006fd0 tx=0x7fab18008040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.616+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 msgr2=0x7fab1c1f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.616+0000 7fab2436a640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c1f7930 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c7b0 tx=0x7fab0c00cc70 comp rx=0 tx=0).stop 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.616+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 shutdown_connections 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fab0006b4d0 0x7fab0006d990 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1c15dc00 0x7fab1c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c0d1dd0 0x7fab1c1f7930 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 --2- 172.21.15.148:0/4281617683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1c0ceb20 0x7fab1c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 >> 172.21.15.148:0/4281617683 conn(0x7fab1c06f5d0 msgr2=0x7fab1c0b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 shutdown_connections 2024-04-25T10:07:05.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:05.617+0000 7fab2436a640 1 -- 172.21.15.148:0/4281617683 wait complete. 2024-04-25T10:07:06.036 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:52.824899+0000', 'modified': '2024-04-25T10:07:04.544018+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11619': {'gid': 11619, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.148:6837/1465938320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1465938320}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1465938320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:06.036 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11619:up:rejoin 2024-04-25T10:07:07.037 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:07.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.140+0000 7fc007d16640 1 Processor -- start 2024-04-25T10:07:07.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.141+0000 7fc007d16640 1 -- start start 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.141+0000 7fc007d16640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.141+0000 7fc007d16640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0000b2cb0 0x7fc0000d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.141+0000 7fc007d16640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0000b97a0 con 0x7fc0000b92a0 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.141+0000 7fc007d16640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00015dfc0 con 0x7fc0000b2cb0 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.141+0000 7fc007d16640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000d1a00 con 0x7fc00015dbe0 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc005ab2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0052b1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0000b2cb0 0x7fc0000d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0062b3640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc0000b92a0 0x7fc0000b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:54518/0 (socket says 172.21.15.148:54518) 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0052b1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0000b2cb0 0x7fc0000d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:58002/0 (socket says 172.21.15.148:58002) 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc005ab2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:55834/0 (socket says 172.21.15.148:55834) 2024-04-25T10:07:07.148 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0062b3640 1 -- 172.21.15.148:0/108838511 learned_addr learned my addr 172.21.15.148:0/108838511 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc004ab0640 1 -- 172.21.15.148:0/108838511 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886444317 0 0) 0x7fc0000b97a0 con 0x7fc0000b92a0 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc004ab0640 1 -- 172.21.15.148:0/108838511 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe8003520 con 0x7fc0000b92a0 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0052b1640 1 -- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 msgr2=0x7fc0000d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0052b1640 1 --2- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0052b1640 1 -- 172.21.15.148:0/108838511 >> v1:172.21.15.188:6789/0 conn(0x7fc0000b92a0 legacy=0x7fc0000b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc0052b1640 1 -- 172.21.15.148:0/108838511 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000d1c50 con 0x7fc0000b2cb0 2024-04-25T10:07:07.149 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.142+0000 7fc005ab2640 1 --2- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:07.150 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.143+0000 7fc0052b1640 1 --2- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc0000d0e50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=410ee07d6b2e2476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.143+0000 7fc004ab0640 1 -- 172.21.15.148:0/108838511 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4037070 con 0x7fc0000b2cb0 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.143+0000 7fc004ab0640 1 -- 172.21.15.148:0/108838511 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4032440 con 0x7fc0000b2cb0 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.143+0000 7fc004ab0640 1 -- 172.21.15.148:0/108838511 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4032810 con 0x7fc0000b2cb0 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 -- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 msgr2=0x7fc0000d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 --2- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc0000d0e50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 -- 172.21.15.148:0/108838511 shutdown_connections 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 --2- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc0000d0e50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 --2- 172.21.15.148:0/108838511 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 -- 172.21.15.148:0/108838511 >> 172.21.15.148:0/108838511 conn(0x7fc000061a20 msgr2=0x7fc000061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 -- 172.21.15.148:0/108838511 shutdown_connections 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.144+0000 7fc007d16640 1 -- 172.21.15.148:0/108838511 wait complete. 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.145+0000 7fc007d16640 1 Processor -- start 2024-04-25T10:07:07.151 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.145+0000 7fc007d16640 1 -- start start 2024-04-25T10:07:07.153 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc007d16640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc000148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc007d16640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 0x7fc0001f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc007d16640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc007d16640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000b6ab0 con 0x7fc0000b92a0 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc007d16640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000d2210 con 0x7fc0000b2cb0 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc007d16640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000b6c30 con 0x7fc00015dbe0 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 0x7fc0001f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 0x7fc0001f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53504/0 (socket says 172.21.15.148:53504) 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 -- 172.21.15.148:0/1077718152 learned_addr learned my addr 172.21.15.148:0/1077718152 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc005ab2640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc000148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0062b3640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 -- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 msgr2=0x7fc0000cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.154 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 -- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 msgr2=0x7fc000148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc000148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 -- 172.21.15.148:0/1077718152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc000149210 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc005ab2640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc000148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fc0052b1640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 0x7fc0001f6ad0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fbff402a1c0 tx=0x7fbff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4037040 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.146+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4042070 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.147+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0001f7300 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.147+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff403b930 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.147+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0001f78c0 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.147+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000073990 con 0x7fc0000b92a0 2024-04-25T10:07:07.155 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.148+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbff403bad0 con 0x7fc0000b92a0 2024-04-25T10:07:07.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.148+0000 7fbffe7fc640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbfec06b4d0 0x7fbfec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.149+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fbff40aad10 con 0x7fc0000b92a0 2024-04-25T10:07:07.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.149+0000 7fc005ab2640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbfec06b4d0 0x7fbfec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.149+0000 7fc005ab2640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbfec06b4d0 0x7fbfec06d990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbff80074d0 tx=0x7fbff800b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.153+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbff4079a60 con 0x7fc0000b92a0 2024-04-25T10:07:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.363+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc000155350 con 0x7fc0000b92a0 2024-04-25T10:07:07.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.363+0000 7fbffe7fc640 1 -- 172.21.15.148:0/1077718152 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4966 (secure 0 0 0) 0x7fbff4079360 con 0x7fc0000b92a0 2024-04-25T10:07:07.369 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:07:07.369 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:05.546576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:07:07.370 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 119 2024-04-25T10:07:07.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbfec06b4d0 msgr2=0x7fbfec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbfec06b4d0 0x7fbfec06d990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbff80074d0 tx=0x7fbff800b040 comp rx=0 tx=0).stop 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 msgr2=0x7fc0001f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 0x7fc0001f6ad0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fbff402a1c0 tx=0x7fbff40046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 shutdown_connections 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbfec06b4d0 0x7fbfec06d990 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc00015dbe0 0x7fc0000cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.365+0000 7fc007d16640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0000b92a0 0x7fc0001f6ad0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.366+0000 7fc007d16640 1 --2- 172.21.15.148:0/1077718152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000b2cb0 0x7fc000148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.366+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 >> 172.21.15.148:0/1077718152 conn(0x7fc000061a20 msgr2=0x7fc0000b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.366+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 shutdown_connections 2024-04-25T10:07:07.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.366+0000 7fc007d16640 1 -- 172.21.15.148:0/1077718152 wait complete. 2024-04-25T10:07:07.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:52.824899+0000', 'modified': '2024-04-25T10:07:05.546576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11619': {'gid': 11619, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6837/1465938320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1465938320}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1465938320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:07.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:07.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:07:07.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.879+0000 7f445d1b7640 1 Processor -- start 2024-04-25T10:07:07.885 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.879+0000 7f445d1b7640 1 -- start start 2024-04-25T10:07:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f445d1b7640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f445806fbd0 0x7f44580ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f445d1b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44580ce730 0x7f44580b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f445d1b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44580b4d80 con 0x7f4458155f80 2024-04-25T10:07:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f445d1b7640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44580b5390 con 0x7f445806fbd0 2024-04-25T10:07:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f445d1b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44580d07f0 con 0x7f44580ce730 2024-04-25T10:07:07.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f445806fbd0 0x7f44580ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f445806fbd0 0x7f44580ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53516/0 (socket says 172.21.15.148:53516) 2024-04-25T10:07:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 -- 172.21.15.148:0/46066328 learned_addr learned my addr 172.21.15.148:0/46066328 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 -- 172.21.15.148:0/46066328 >> v2:172.21.15.188:3300/0 conn(0x7f44580ce730 msgr2=0x7f44580b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 --2- 172.21.15.148:0/46066328 >> v2:172.21.15.188:3300/0 conn(0x7f44580ce730 0x7f44580b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 -- 172.21.15.148:0/46066328 >> v1:172.21.15.188:6789/0 conn(0x7f4458155f80 legacy=0x7f445806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.890 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.880+0000 7f444ffff640 1 -- 172.21.15.148:0/46066328 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580cef70 con 0x7f445806fbd0 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.881+0000 7f444ffff640 1 --2- 172.21.15.148:0/46066328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445806fbd0 0x7f44580ce1f0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f44580625b0 tx=0x7f444002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837e3a069a306bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.881+0000 7f445659c640 1 -- 172.21.15.148:0/46066328 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440004030 con 0x7f445806fbd0 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.881+0000 7f445659c640 1 -- 172.21.15.148:0/46066328 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44400041d0 con 0x7f445806fbd0 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.881+0000 7f445659c640 1 -- 172.21.15.148:0/46066328 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440003610 con 0x7f445806fbd0 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 -- 172.21.15.148:0/46066328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445806fbd0 msgr2=0x7f44580ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 --2- 172.21.15.148:0/46066328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445806fbd0 0x7f44580ce1f0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f44580625b0 tx=0x7f444002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:07.891 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 -- 172.21.15.148:0/46066328 shutdown_connections 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 --2- 172.21.15.148:0/46066328 >> v2:172.21.15.188:3300/0 conn(0x7f44580ce730 0x7f44580b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 --2- 172.21.15.148:0/46066328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f445806fbd0 0x7f44580ce1f0 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 -- 172.21.15.148:0/46066328 >> 172.21.15.148:0/46066328 conn(0x7f4458061600 msgr2=0x7f44580619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 -- 172.21.15.148:0/46066328 shutdown_connections 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.882+0000 7f445d1b7640 1 -- 172.21.15.148:0/46066328 wait complete. 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.883+0000 7f445d1b7640 1 Processor -- start 2024-04-25T10:07:07.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.883+0000 7f445d1b7640 1 -- start start 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445d1b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f445806fbd0 0x7f445814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445d1b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 0x7f44581f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445d1b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4458155f80 0x7f4458152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445d1b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580cf250 con 0x7f44580ce730 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445d1b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4458156360 con 0x7f445806fbd0 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445d1b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44580cf3d0 con 0x7f4458155f80 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 0x7f44581f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 0x7f44581f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53526/0 (socket says 172.21.15.148:53526) 2024-04-25T10:07:07.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 -- 172.21.15.148:0/3983961763 learned_addr learned my addr 172.21.15.148:0/3983961763 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f445806fbd0 0x7f445814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 -- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4458155f80 msgr2=0x7f4458152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445759e640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4458155f80 0x7f4458152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4458155f80 0x7f4458152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 -- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f445806fbd0 msgr2=0x7f445814a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f445806fbd0 0x7f445814a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 -- 172.21.15.148:0/3983961763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44581f3320 con 0x7f44580ce730 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f445806fbd0 0x7f445814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f445759e640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4458155f80 0x7f4458152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444ffff640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 0x7f44581f2de0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f4440034040 tx=0x7f44400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.884+0000 7f444f7fe640 1 -- 172.21.15.148:0/3983961763 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440041020 con 0x7f44580ce730 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.885+0000 7f444f7fe640 1 -- 172.21.15.148:0/3983961763 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440003c50 con 0x7f44580ce730 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.885+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445814f830 con 0x7f44580ce730 2024-04-25T10:07:07.904 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.885+0000 7f444f7fe640 1 -- 172.21.15.148:0/3983961763 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440004030 con 0x7f44580ce730 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.885+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44581f36c0 con 0x7f44580ce730 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.886+0000 7f444f7fe640 1 -- 172.21.15.148:0/3983961763 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4440044020 con 0x7f44580ce730 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.886+0000 7f444f7fe640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f443806b4d0 0x7f443806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.886+0000 7f444f7fe640 1 -- 172.21.15.148:0/3983961763 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f44400ab250 con 0x7f44580ce730 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.887+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f443806b4d0 0x7f443806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.887+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f443806b4d0 0x7f443806d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f445815b5b0 tx=0x7f4448008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.887+0000 7f445ff97740 1 Processor -- start 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 -- start start 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 0x5592404d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.905 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240500bc0 0x559240503040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240503580 0x5592404d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924043ba10 con 0x5592405007e0 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592403e4b30 con 0x559240500bc0 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559240437c50 con 0x559240503580 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 0x5592404d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 0x5592404d8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53532/0 (socket says 172.21.15.148:53532) 2024-04-25T10:07:07.906 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.888+0000 7f4456d9d640 1 -- 172.21.15.148:0/4275105103 learned_addr learned my addr 172.21.15.148:0/4275105103 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f445759e640 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240503580 0x5592404d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f444ffff640 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240500bc0 0x559240503040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f4456d9d640 1 -- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240503580 msgr2=0x5592404d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f4456d9d640 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240503580 0x5592404d6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f4456d9d640 1 -- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240500bc0 msgr2=0x559240503040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f4456d9d640 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240500bc0 0x559240503040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f4456d9d640 1 -- 172.21.15.148:0/4275105103 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592403f9e10 con 0x5592405007e0 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f444ffff640 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240500bc0 0x559240503040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f4456d9d640 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 0x5592404d8e40 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f444800ca90 tx=0x7f444800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d83f274cd36b738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f444cff9640 1 -- 172.21.15.148:0/4275105103 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448023070 con 0x5592405007e0 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f444cff9640 1 -- 172.21.15.148:0/4275105103 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448013db0 con 0x5592405007e0 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.889+0000 7f444cff9640 1 -- 172.21.15.148:0/4275105103 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444800ae60 con 0x5592405007e0 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 -- 172.21.15.148:0/4275105103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 msgr2=0x5592404d8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 0x5592404d8e40 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f444800ca90 tx=0x7f444800f000 comp rx=0 tx=0).stop 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 -- 172.21.15.148:0/4275105103 shutdown_connections 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240503580 0x5592404d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240500bc0 0x559240503040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 --2- 172.21.15.148:0/4275105103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592405007e0 0x5592404d8e40 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 -- 172.21.15.148:0/4275105103 >> 172.21.15.148:0/4275105103 conn(0x5592404d5840 msgr2=0x55924051e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 -- 172.21.15.148:0/4275105103 shutdown_connections 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 -- 172.21.15.148:0/4275105103 wait complete. 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 -- start start 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.890+0000 7f445ff97740 1 Processor -- start 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 0x55924051dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592404ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5592404ffbd0 0x5592405152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592403f80c0 con 0x559240500bc0 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924051bd40 con 0x559240503580 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559240506900 con 0x5592404ffbd0 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f445ff97740 10 client.0 _ll_register_callbacks cb 0x5592403ebec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 0x55924051dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 0x55924051dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53534/0 (socket says 172.21.15.148:53534) 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.891+0000 7f4456d9d640 1 -- 172.21.15.148:0/3608332074 learned_addr learned my addr 172.21.15.148:0/3608332074 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4456d9d640 1 -- 172.21.15.148:0/3608332074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5592404ffbd0 msgr2=0x5592405152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5592404ffbd0 0x5592405152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4456d9d640 1 -- 172.21.15.148:0/3608332074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 msgr2=0x5592404ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592404ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4456d9d640 1 -- 172.21.15.148:0/3608332074 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592401502d0 con 0x559240500bc0 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4456d9d640 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 0x55924051dc30 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f444801e6b0 tx=0x7f4448013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfca108776e1834c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4435ffb640 1 -- 172.21.15.148:0/3608332074 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448023050 con 0x559240500bc0 2024-04-25T10:07:07.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4435ffb640 1 -- 172.21.15.148:0/3608332074 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f444801eaa0 con 0x559240500bc0 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4448013db0 con 0x559240500bc0 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4435ffb640 1 -- 172.21.15.148:0/3608332074 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4448037050 con 0x559240500bc0 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 10 client.11895 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55924021a6b0 con 0x559240500bc0 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4435ffb640 1 -- 172.21.15.148:0/3608332074 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448013db0 con 0x559240500bc0 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f4435ffb640 1 -- 172.21.15.148:0/3608332074 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f444803a020 con 0x559240500bc0 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 10 client.11895 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 10 client.11895 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:07.912 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 20 client.11895 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x559240505bd0 0x559240505fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 4 client.11895 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.892+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55924049a920 con 0x559240505bd0 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f444ffff640 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x559240505bd0 0x559240505fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f4435ffb640 10 client.11895 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f444ffff640 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x559240505bd0 0x559240505fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910832d11e63cf5 server_cookie=1c56accc8b7af755 in_seq=0 out_seq=0 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f4435ffb640 1 -- 172.21.15.148:0/3608332074 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55924049a920 con 0x559240505bd0 2024-04-25T10:07:07.913 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f4435ffb640 10 client.11895 handle_command_reply: tid=0 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 client.11895 shutdown 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 msgr2=0x55924051dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 0x55924051dc30 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7f444801e6b0 tx=0x7f4448013000 comp rx=0 tx=0).stop 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 shutdown_connections 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x559240505bd0 0x559240505fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5592404ffbd0 0x5592405152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592404ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f4435ffb640 0 client.11895 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:07.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 --2- 172.21.15.148:0/3608332074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559240500bc0 0x55924051dc30 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.893+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 >> 172.21.15.148:0/3608332074 conn(0x5592405007e0 msgr2=0x5592405215c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.894+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 shutdown_connections 2024-04-25T10:07:07.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.894+0000 7f445ff97740 1 -- 172.21.15.148:0/3608332074 wait complete. 2024-04-25T10:07:07.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.894+0000 7f445ff97740 20 client.11895 trim_cache size 0 max 16384 2024-04-25T10:07:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.909+0000 7f445ff97740 1 Processor -- start 2024-04-25T10:07:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.909+0000 7f445ff97740 1 -- start start 2024-04-25T10:07:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f445ff97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x559240584260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x559240521900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.923 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f445ff97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445806dc30 con 0x5592404ffbd0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445806ce80 con 0x559240503580 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44581f4a90 con 0x559240500bc0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x559240584260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x559240584260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53538/0 (socket says 172.21.15.148:53538) 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 -- 172.21.15.148:0/2827376768 learned_addr learned my addr 172.21.15.148:0/2827376768 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 -- 172.21.15.148:0/2827376768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 msgr2=0x5592405213c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 -- 172.21.15.148:0/2827376768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 msgr2=0x559240521900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x559240521900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 -- 172.21.15.148:0/2827376768 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55924052a880 con 0x5592404ffbd0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4456d9d640 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x559240584260 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f444801ef30 tx=0x7f444801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bee6ffc3662c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4437fff640 1 -- 172.21.15.148:0/2827376768 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448023040 con 0x5592404ffbd0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4437fff640 1 -- 172.21.15.148:0/2827376768 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444802f070 con 0x5592404ffbd0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.910+0000 7f4437fff640 1 -- 172.21.15.148:0/2827376768 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448013bb0 con 0x5592404ffbd0 2024-04-25T10:07:07.924 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 -- 172.21.15.148:0/2827376768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 msgr2=0x559240584260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x559240584260 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f444801ef30 tx=0x7f444801c000 comp rx=0 tx=0).stop 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 -- 172.21.15.148:0/2827376768 shutdown_connections 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x559240521900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405213c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 --2- 172.21.15.148:0/2827376768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x559240584260 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 -- 172.21.15.148:0/2827376768 >> 172.21.15.148:0/2827376768 conn(0x5592405007e0 msgr2=0x559240503a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 -- 172.21.15.148:0/2827376768 shutdown_connections 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.911+0000 7f445ff97740 1 -- 172.21.15.148:0/2827376768 wait complete. 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 -- start start 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 Processor -- start 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x55924053cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592405043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x559240506690 con 0x5592404ffbd0 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44581f5090 con 0x559240503580 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559240536a60 con 0x559240500bc0 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f445ff97740 10 client.0 _ll_register_callbacks cb 0x7f444800ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x55924053cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f4456d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x55924053cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53554/0 (socket says 172.21.15.148:53554) 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.912+0000 7f4456d9d640 1 -- 172.21.15.148:0/2903325676 learned_addr learned my addr 172.21.15.148:0/2903325676 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445759e640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592405043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f444ffff640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4456d9d640 1 -- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 msgr2=0x5592405855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4456d9d640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4456d9d640 1 -- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 msgr2=0x5592405043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4456d9d640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592405043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4456d9d640 1 -- 172.21.15.148:0/2903325676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592405148d0 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445759e640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592405043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4456d9d640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x55924053cd90 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f444800c740 tx=0x7f4448004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beac2e76b98f242c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4435ffb640 1 -- 172.21.15.148:0/2903325676 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448023040 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4435ffb640 1 -- 172.21.15.148:0/2903325676 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4448032760 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f444802f070 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4435ffb640 1 -- 172.21.15.148:0/2903325676 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f444804e050 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445ff97740 10 client.11901 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x559240504930 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4435ffb640 1 -- 172.21.15.148:0/2903325676 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444802f070 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f4435ffb640 1 -- 172.21.15.148:0/2903325676 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4448051020 con 0x5592404ffbd0 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445ff97740 10 client.11901 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.913+0000 7f445ff97740 10 client.11901 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:07.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 20 client.11901 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5592404d2cc0 0x5592404d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 4 client.11901 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559240448f50 con 0x5592404d2cc0 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f444ffff640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5592404d2cc0 0x5592404d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f4435ffb640 10 client.11901 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f444ffff640 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5592404d2cc0 0x5592404d30a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2381a7cb9c584b0b server_cookie=7d86e18ace691c2f in_seq=0 out_seq=0 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f4435ffb640 1 -- 172.21.15.148:0/2903325676 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x559240448f50 con 0x5592404d2cc0 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f4435ffb640 10 client.11901 handle_command_reply: tid=0 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 client.11901 shutdown 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 msgr2=0x55924053cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x55924053cd90 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f444800c740 tx=0x7f4448004640 comp rx=0 tx=0).stop 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 shutdown_connections 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5592404d2cc0 0x5592404d30a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559240503580 0x5592405043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f4435ffb640 0 client.11901 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559240500bc0 0x5592405855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 --2- 172.21.15.148:0/2903325676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592404ffbd0 0x55924053cd90 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 >> 172.21.15.148:0/2903325676 conn(0x5592405007e0 msgr2=0x5592404fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 shutdown_connections 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 1 -- 172.21.15.148:0/2903325676 wait complete. 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.914+0000 7f445ff97740 20 client.11901 trim_cache size 0 max 16384 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:07:07.929 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 11619, 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 4.3742355670000004, 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 119, 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 200, 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 198, 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 19.488118672999999 2024-04-25T10:07:07.930 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:07:07.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f443806b4d0 msgr2=0x7f443806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f443806b4d0 0x7f443806d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f445815b5b0 tx=0x7f4448008040 comp rx=0 tx=0).stop 2024-04-25T10:07:07.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 msgr2=0x7f44581f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 0x7f44581f2de0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f4440034040 tx=0x7f44400046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 shutdown_connections 2024-04-25T10:07:07.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f443806b4d0 0x7f443806d990 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4458155f80 0x7f4458152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44580ce730 0x7f44581f2de0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 --2- 172.21.15.148:0/3983961763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f445806fbd0 0x7f445814a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 >> 172.21.15.148:0/3983961763 conn(0x7f4458061600 msgr2=0x7f445814aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 shutdown_connections 2024-04-25T10:07:07.934 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:07.915+0000 7f445d1b7640 1 -- 172.21.15.148:0/3983961763 wait complete. 2024-04-25T10:07:07.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:07.935 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:07:08.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.030+0000 7f20caad8640 1 Processor -- start 2024-04-25T10:07:08.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.030+0000 7f20caad8640 1 -- start start 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20caad8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20caad8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20caad8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20caad8640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c415e000 con 0x7f20c4161710 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20caad8640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40d1de0 con 0x7f20c415dc20 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20caad8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4161c60 con 0x7f20c40b9330 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53562/0 (socket says 172.21.15.148:53562) 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 -- 172.21.15.148:0/3461639114 learned_addr learned my addr 172.21.15.148:0/3461639114 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c9075640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c8874640 1 --2- 172.21.15.148:0/3461639114 >> v2:172.21.15.188:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 -- 172.21.15.148:0/3461639114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 msgr2=0x7f20c406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 -- 172.21.15.148:0/3461639114 >> v2:172.21.15.188:3300/0 conn(0x7f20c415dc20 msgr2=0x7f20c40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3461639114 >> v2:172.21.15.188:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c3fff640 1 -- 172.21.15.148:0/3461639114 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4161f40 con 0x7f20c4161710 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c9075640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.031+0000 7f20c8874640 1 --2- 172.21.15.148:0/3461639114 >> v2:172.21.15.188:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.032+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4161710 0x7f20c40cebb0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f20c40d1520 tx=0x7f20b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8d2f67d614a939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.032+0000 7f20c37fe640 1 -- 172.21.15.148:0/3461639114 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0037070 con 0x7f20c4161710 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.032+0000 7f20c37fe640 1 -- 172.21.15.148:0/3461639114 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0032440 con 0x7f20c4161710 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.032+0000 7f20c37fe640 1 -- 172.21.15.148:0/3461639114 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0032810 con 0x7f20c4161710 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 -- 172.21.15.148:0/3461639114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4161710 msgr2=0x7f20c40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4161710 0x7f20c40cebb0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f20c40d1520 tx=0x7f20b002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 -- 172.21.15.148:0/3461639114 shutdown_connections 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 --2- 172.21.15.148:0/3461639114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c4161710 0x7f20c40cebb0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 --2- 172.21.15.148:0/3461639114 >> v2:172.21.15.188:3300/0 conn(0x7f20c415dc20 0x7f20c40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 -- 172.21.15.148:0/3461639114 >> 172.21.15.148:0/3461639114 conn(0x7f20c40d0420 msgr2=0x7f20c40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 -- 172.21.15.148:0/3461639114 shutdown_connections 2024-04-25T10:07:08.041 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.033+0000 7f20caad8640 1 -- 172.21.15.148:0/3461639114 wait complete. 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.034+0000 7f20caad8640 1 Processor -- start 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.034+0000 7f20caad8640 1 -- start start 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20caad8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c40cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20caad8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 0x7f20c4151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20caad8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20c4161710 0x7f20c415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20caad8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b5e50 con 0x7f20c415dc20 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20caad8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4162500 con 0x7f20c4161710 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20caad8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b5fd0 con 0x7f20c40b9330 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 0x7f20c4151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 0x7f20c4151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53578/0 (socket says 172.21.15.148:53578) 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 -- 172.21.15.148:0/3633930070 learned_addr learned my addr 172.21.15.148:0/3633930070 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c8874640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c40cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c9075640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20c4161710 0x7f20c415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 -- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 msgr2=0x7f20c40cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c40cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 -- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20c4161710 msgr2=0x7f20c415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20c4161710 0x7f20c415fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 -- 172.21.15.148:0/3633930070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40cb520 con 0x7f20c415dc20 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c8874640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c9075640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20c4161710 0x7f20c415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.035+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 0x7f20c4151a60 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f20b002a220 tx=0x7f20b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.036+0000 7f20c17fa640 1 -- 172.21.15.148:0/3633930070 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0041020 con 0x7f20c415dc20 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.036+0000 7f20c17fa640 1 -- 172.21.15.148:0/3633930070 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b00306c0 con 0x7f20c415dc20 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.036+0000 7f20c17fa640 1 -- 172.21.15.148:0/3633930070 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0037040 con 0x7f20c415dc20 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.036+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c40cb800 con 0x7f20c415dc20 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.036+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40cbdc0 con 0x7f20c415dc20 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.037+0000 7f20c17fa640 1 -- 172.21.15.148:0/3633930070 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20b0044020 con 0x7f20c415dc20 2024-04-25T10:07:08.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.038+0000 7f20c17fa640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.038+0000 7f20c17fa640 1 -- 172.21.15.148:0/3633930070 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f20b00aacc0 con 0x7f20c415dc20 2024-04-25T10:07:08.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.038+0000 7f20c8874640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.038+0000 7f20c8874640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f20a806b4d0 0x7f20a806d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f20b8006fd0 tx=0x7f20b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 Processor -- start 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 -- start start 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be4830 0x55e273be87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e273b4dac0 con 0x55e273be9bb0 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e273af6d80 con 0x55e273be8cf0 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e273b49b60 con 0x55e273be4830 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20c8874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20c8874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53584/0 (socket says 172.21.15.148:53584) 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.039+0000 7f20c8874640 1 -- 172.21.15.148:0/3893614403 learned_addr learned my addr 172.21.15.148:0/3893614403 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c3fff640 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be4830 0x55e273be87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c9075640 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c8874640 1 -- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be4830 msgr2=0x55e273be87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c8874640 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be4830 0x55e273be87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c8874640 1 -- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 msgr2=0x55e273be9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c8874640 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c8874640 1 -- 172.21.15.148:0/3893614403 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e27392a7e0 con 0x55e273be9bb0 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c9075640 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20c8874640 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be9f90 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e920 tx=0x7f20b8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30123ba039c8d0db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20ae7fc640 1 -- 172.21.15.148:0/3893614403 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8025070 con 0x55e273be9bb0 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20ae7fc640 1 -- 172.21.15.148:0/3893614403 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8015700 con 0x55e273be9bb0 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.040+0000 7f20ae7fc640 1 -- 172.21.15.148:0/3893614403 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800f040 con 0x55e273be9bb0 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 -- 172.21.15.148:0/3893614403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 msgr2=0x55e273be9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be9f90 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e920 tx=0x7f20b8011000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 -- 172.21.15.148:0/3893614403 shutdown_connections 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be4830 0x55e273be87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/3893614403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be9f90 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 -- 172.21.15.148:0/3893614403 >> 172.21.15.148:0/3893614403 conn(0x55e273be6cb0 msgr2=0x55e273c2f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 -- 172.21.15.148:0/3893614403 shutdown_connections 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 -- 172.21.15.148:0/3893614403 wait complete. 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 -- start start 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.041+0000 7f20cd7d6740 1 Processor -- start 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273c30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273c042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c04800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e273c15740 con 0x55e273be9bb0 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e273aec530 con 0x55e273c008d0 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e273c14bc0 con 0x55e273be8cf0 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20cd7d6740 10 client.0 _ll_register_callbacks cb 0x55e273b44130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:08.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273c042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273c042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53596/0 (socket says 172.21.15.148:53596) 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20c3fff640 1 -- 172.21.15.148:0/2561517838 learned_addr learned my addr 172.21.15.148:0/2561517838 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20c9075640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c04800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.042+0000 7f20c8874640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273c30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20c3fff640 1 -- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 msgr2=0x55e273c30930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20c3fff640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273c30930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20c3fff640 1 -- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 msgr2=0x55e273c04800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20c3fff640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c04800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20c3fff640 1 -- 172.21.15.148:0/2561517838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e27398af70 con 0x55e273be9bb0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20c3fff640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273c042c0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f20b00a8fa0 tx=0x7f20b0031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310e9d0e289845aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20a37fe640 1 -- 172.21.15.148:0/2561517838 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0081670 con 0x55e273be9bb0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20a37fe640 1 -- 172.21.15.148:0/2561517838 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f20b0078640 con 0x55e273be9bb0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b8015700 con 0x55e273be9bb0 2024-04-25T10:07:08.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20a37fe640 1 -- 172.21.15.148:0/2561517838 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20b007d340 con 0x55e273be9bb0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 10 client.11913 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55e273822170 con 0x55e273be9bb0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20a37fe640 1 -- 172.21.15.148:0/2561517838 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8015700 con 0x55e273be9bb0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20a37fe640 1 -- 172.21.15.148:0/2561517838 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20b003a030 con 0x55e273be9bb0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 10 client.11913 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 10 client.11913 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 20 client.11913 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c062d0 0x55e273c066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 4 client.11913 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.043+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e273b5b000 con 0x55e273c062d0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20c8874640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c062d0 0x55e273c066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20c8874640 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c062d0 0x55e273c066e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d71728ed092074 server_cookie=f1e00c6d91236a64 in_seq=0 out_seq=0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20a37fe640 10 client.11913 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20a37fe640 1 -- 172.21.15.148:0/2561517838 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e273b5b000 con 0x55e273c062d0 2024-04-25T10:07:08.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20a37fe640 10 client.11913 handle_command_reply: tid=0 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20cd7d6740 1 client.11913 shutdown 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 msgr2=0x55e273c042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.044+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273c042c0 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f20b00a8fa0 tx=0x7f20b0031f40 comp rx=0 tx=0).stop 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 shutdown_connections 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c062d0 0x55e273c066e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c04800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273c042c0 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20a37fe640 0 client.11913 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/2561517838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273c30930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 >> 172.21.15.148:0/2561517838 conn(0x55e273be4830 msgr2=0x55e273bea6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 shutdown_connections 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 1 -- 172.21.15.148:0/2561517838 wait complete. 2024-04-25T10:07:08.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.045+0000 7f20cd7d6740 20 client.11913 trim_cache size 0 max 16384 2024-04-25T10:07:08.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.060+0000 7f20cd7d6740 1 Processor -- start 2024-04-25T10:07:08.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.060+0000 7f20cd7d6740 1 -- start start 2024-04-25T10:07:08.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273c008d0 0x55e273c03e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4200870 con 0x55e273be9bb0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41ffaa0 con 0x55e273be8cf0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c415ee90 con 0x55e273c008d0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53610/0 (socket says 172.21.15.148:53610) 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 -- 172.21.15.148:0/50463646 learned_addr learned my addr 172.21.15.148:0/50463646 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c8874640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 -- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273c008d0 msgr2=0x55e273c03e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c9075640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273c008d0 0x55e273c03e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273c008d0 0x55e273c03e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 -- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 msgr2=0x55e273be90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 -- 172.21.15.148:0/50463646 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e273c043c0 con 0x55e273be9bb0 2024-04-25T10:07:08.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c9075640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273c008d0 0x55e273c03e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c8874640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20c3fff640 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be3b40 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f20b0075b70 tx=0x7f20b0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bdf55aa579ed25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20adffb640 1 -- 172.21.15.148:0/50463646 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0078b50 con 0x55e273be9bb0 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20adffb640 1 -- 172.21.15.148:0/50463646 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0081670 con 0x55e273be9bb0 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.061+0000 7f20adffb640 1 -- 172.21.15.148:0/50463646 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0078fc0 con 0x55e273be9bb0 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 -- 172.21.15.148:0/50463646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 msgr2=0x55e273be3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be3b40 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f20b0075b70 tx=0x7f20b0047000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 -- 172.21.15.148:0/50463646 shutdown_connections 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273c008d0 0x55e273c03e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273be3b40 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.078 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/50463646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273be8cf0 0x55e273be90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 -- 172.21.15.148:0/50463646 >> 172.21.15.148:0/50463646 conn(0x55e273be4830 msgr2=0x55e273bdfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 -- 172.21.15.148:0/50463646 shutdown_connections 2024-04-25T10:07:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 -- 172.21.15.148:0/50463646 wait complete. 2024-04-25T10:07:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.062+0000 7f20cd7d6740 1 -- start start 2024-04-25T10:07:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 Processor -- start 2024-04-25T10:07:08.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273bea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273bea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e273c059a0 con 0x55e273be9bb0 2024-04-25T10:07:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41ff4a0 con 0x55e273c008d0 2024-04-25T10:07:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e273c31e70 con 0x55e273be8cf0 2024-04-25T10:07:08.080 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20cd7d6740 10 client.0 _ll_register_callbacks cb 0x7f20b8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273bea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273bea5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53614/0 (socket says 172.21.15.148:53614) 2024-04-25T10:07:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.063+0000 7f20c3fff640 1 -- 172.21.15.148:0/4115634351 learned_addr learned my addr 172.21.15.148:0/4115634351 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c8874640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273bea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.081 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c9075640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c3fff640 1 -- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 msgr2=0x55e273bea0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c3fff640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273bea0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c3fff640 1 -- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 msgr2=0x55e273c419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c3fff640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c3fff640 1 -- 172.21.15.148:0/4115634351 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e273c48f20 con 0x55e273be9bb0 2024-04-25T10:07:08.082 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20c3fff640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273bea5f0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f20b0000c00 tx=0x7f20b00a8e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48a9c019dea2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20a37fe640 1 -- 172.21.15.148:0/4115634351 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0085020 con 0x55e273be9bb0 2024-04-25T10:07:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20a37fe640 1 -- 172.21.15.148:0/4115634351 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f20b0075150 con 0x55e273be9bb0 2024-04-25T10:07:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20b0081670 con 0x55e273be9bb0 2024-04-25T10:07:08.083 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20a37fe640 1 -- 172.21.15.148:0/4115634351 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20b0078fc0 con 0x55e273be9bb0 2024-04-25T10:07:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 10 client.11919 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55e273c48f20 con 0x55e273be9bb0 2024-04-25T10:07:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20a37fe640 1 -- 172.21.15.148:0/4115634351 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0081670 con 0x55e273be9bb0 2024-04-25T10:07:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20a37fe640 1 -- 172.21.15.148:0/4115634351 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20b0081b50 con 0x55e273be9bb0 2024-04-25T10:07:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 10 client.11919 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:08.084 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 10 client.11919 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:08.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 20 client.11919 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:08.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c835d0 0x55e273c41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 4 client.11919 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:07:08.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.064+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55e273b55350 con 0x55e273c835d0 2024-04-25T10:07:08.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.065+0000 7f20c8874640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c835d0 0x55e273c41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.085 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.065+0000 7f20a37fe640 10 client.11919 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.065+0000 7f20c8874640 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c835d0 0x55e273c41ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea07193b4805452 server_cookie=cc3fefbd422c65a1 in_seq=0 out_seq=0 2024-04-25T10:07:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.065+0000 7f20a37fe640 1 -- 172.21.15.148:0/4115634351 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e273b55350 con 0x55e273c835d0 2024-04-25T10:07:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.065+0000 7f20a37fe640 10 client.11919 handle_command_reply: tid=0 2024-04-25T10:07:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.065+0000 7f20cd7d6740 1 client.11919 shutdown 2024-04-25T10:07:08.086 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 msgr2=0x55e273bea5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273bea5f0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f20b0000c00 tx=0x7f20b00a8e70 comp rx=0 tx=0).stop 2024-04-25T10:07:08.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 shutdown_connections 2024-04-25T10:07:08.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55e273c835d0 0x55e273c41ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20a37fe640 0 client.11919 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.087 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e273c008d0 0x55e273c419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e273be9bb0 0x55e273bea5f0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 --2- 172.21.15.148:0/4115634351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e273be8cf0 0x55e273bea0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 >> 172.21.15.148:0/4115634351 conn(0x55e273be4830 msgr2=0x55e273c0d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 shutdown_connections 2024-04-25T10:07:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 1 -- 172.21.15.148:0/4115634351 wait complete. 2024-04-25T10:07:08.088 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20cd7d6740 20 client.11919 trim_cache size 0 max 16384 2024-04-25T10:07:08.089 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:07:08.089 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:07:08.089 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "f1c1ce9d-e3ba-468b-b589-06b7cb7206c9", 2024-04-25T10:07:08.090 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:07:08.090 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:07:08.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f20a806b4d0 msgr2=0x7f20a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20caad8640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f20a806b4d0 0x7f20a806d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f20b8006fd0 tx=0x7f20b8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:08.092 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 msgr2=0x7f20c4151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.066+0000 7f20caad8640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 0x7f20c4151a60 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f20b002a220 tx=0x7f20b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 shutdown_connections 2024-04-25T10:07:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20c4161710 0x7f20c415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20c415dc20 0x7f20c4151a60 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.093 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 --2- 172.21.15.148:0/3633930070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20c40b9330 0x7f20c40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 >> 172.21.15.148:0/3633930070 conn(0x7f20c40d0420 msgr2=0x7f20c40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 shutdown_connections 2024-04-25T10:07:08.094 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.067+0000 7f20caad8640 1 -- 172.21.15.148:0/3633930070 wait complete. 2024-04-25T10:07:08.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:08.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:07:08.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 Processor -- start 2024-04-25T10:07:08.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 -- start start 2024-04-25T10:07:08.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcfb81610f0 0x7fcfb806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcfb80ce7b0 0x7fcfb8161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb80c6960 con 0x7fcfb81616f0 2024-04-25T10:07:08.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb81614d0 con 0x7fcfb81610f0 2024-04-25T10:07:08.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.190+0000 7fcfbf15c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb8162260 con 0x7fcfb80ce7b0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcfb81610f0 0x7fcfb806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fcfb81610f0 0x7fcfb806c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53624/0 (socket says 172.21.15.148:53624) 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 -- 172.21.15.148:0/785199138 learned_addr learned my addr 172.21.15.148:0/785199138 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/785199138 >> v2:172.21.15.188:3300/0 conn(0x7fcfb80ce7b0 0x7fcfb8161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 -- 172.21.15.148:0/785199138 >> v2:172.21.15.188:3300/0 conn(0x7fcfb80ce7b0 msgr2=0x7fcfb8161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/785199138 >> v2:172.21.15.188:3300/0 conn(0x7fcfb80ce7b0 0x7fcfb8161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 -- 172.21.15.148:0/785199138 >> v1:172.21.15.188:6789/0 conn(0x7fcfb81616f0 legacy=0x7fcfb80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.191+0000 7fcfbcef8640 1 -- 172.21.15.148:0/785199138 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8162540 con 0x7fcfb81610f0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/785199138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb806c1c0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80b33e0 tx=0x7fcfb002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df2085da23ef5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfb77fe640 1 -- 172.21.15.148:0/785199138 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0036070 con 0x7fcfb81610f0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfb77fe640 1 -- 172.21.15.148:0/785199138 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0031600 con 0x7fcfb81610f0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfb77fe640 1 -- 172.21.15.148:0/785199138 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00319d0 con 0x7fcfb81610f0 2024-04-25T10:07:08.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 -- 172.21.15.148:0/785199138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 msgr2=0x7fcfb806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/785199138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb806c1c0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80b33e0 tx=0x7fcfb002a730 comp rx=0 tx=0).stop 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 -- 172.21.15.148:0/785199138 shutdown_connections 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/785199138 >> v2:172.21.15.188:3300/0 conn(0x7fcfb80ce7b0 0x7fcfb8161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/785199138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb806c1c0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 -- 172.21.15.148:0/785199138 >> 172.21.15.148:0/785199138 conn(0x7fcfb806f500 msgr2=0x7fcfb806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 -- 172.21.15.148:0/785199138 shutdown_connections 2024-04-25T10:07:08.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.192+0000 7fcfbf15c640 1 -- 172.21.15.148:0/785199138 wait complete. 2024-04-25T10:07:08.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 Processor -- start 2024-04-25T10:07:08.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 -- start start 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfb80ce7b0 0x7fcfb8149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb80ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfb81616f0 0x7fcfb80ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80cf690 con 0x7fcfb81610f0 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80cfce0 con 0x7fcfb81616f0 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.194+0000 7fcfbf15c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb806c700 con 0x7fcfb80ce7b0 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb80ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb80ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53636/0 (socket says 172.21.15.148:53636) 2024-04-25T10:07:08.216 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 -- 172.21.15.148:0/240379234 learned_addr learned my addr 172.21.15.148:0/240379234 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfb81616f0 0x7fcfb80ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfb80ce7b0 0x7fcfb8149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 -- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfb80ce7b0 msgr2=0x7fcfb8149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfb80ce7b0 0x7fcfb8149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 -- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfb81616f0 msgr2=0x7fcfb80ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfb81616f0 0x7fcfb80ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 -- 172.21.15.148:0/240379234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80adc40 con 0x7fcfb81610f0 2024-04-25T10:07:08.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfb81616f0 0x7fcfb80ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb80ad030 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c3c0 tx=0x7fcfa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb57fa640 1 -- 172.21.15.148:0/240379234 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80052d0 con 0x7fcfb81610f0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb57fa640 1 -- 172.21.15.148:0/240379234 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8005490 con 0x7fcfb81610f0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfb57fa640 1 -- 172.21.15.148:0/240379234 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8007260 con 0x7fcfb81610f0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb80adf20 con 0x7fcfb81610f0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.195+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb80ae4e0 con 0x7fcfb81610f0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.197+0000 7fcfb57fa640 1 -- 172.21.15.148:0/240379234 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfa8007400 con 0x7fcfb81610f0 2024-04-25T10:07:08.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.197+0000 7fcfb57fa640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf9806b500 0x7fcf9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.197+0000 7fcfb57fa640 1 -- 172.21.15.148:0/240379234 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcfa8089810 con 0x7fcfb81610f0 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.197+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf9806b500 0x7fcf9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.198+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf9806b500 0x7fcf9806d9c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0033000 tx=0x7fcfb0009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.198+0000 7fcfc1e59740 1 Processor -- start 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.198+0000 7fcfc1e59740 1 -- start start 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9aca5770 0x55bb9aca8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9ac0eda0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9abb7b80 con 0x55bb9aca5770 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb9ac0ae40 con 0x55bb9acd2b10 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53642/0 (socket says 172.21.15.148:53642) 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 -- 172.21.15.148:0/3333243730 learned_addr learned my addr 172.21.15.148:0/3333243730 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9aca5770 0x55bb9aca8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 -- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 msgr2=0x55bb9acd5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 -- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9aca5770 msgr2=0x55bb9aca8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9aca5770 0x55bb9aca8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 -- 172.21.15.148:0/3333243730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9abc1f10 con 0x55bb9acd2ef0 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd5860 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8007830 tx=0x7fcfa8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d039f52a2a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfae7fc640 1 -- 172.21.15.148:0/3333243730 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80560d0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfae7fc640 1 -- 172.21.15.148:0/3333243730 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa805e0c0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.199+0000 7fcfae7fc640 1 -- 172.21.15.148:0/3333243730 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8057220 con 0x55bb9acd2ef0 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3333243730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 msgr2=0x55bb9acd5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd5860 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8007830 tx=0x7fcfa8011000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3333243730 shutdown_connections 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9aca5770 0x55bb9aca8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd5860 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3333243730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3333243730 >> 172.21.15.148:0/3333243730 conn(0x55bb9aca79e0 msgr2=0x55bb9acf0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3333243730 shutdown_connections 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3333243730 wait complete. 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.200+0000 7fcfc1e59740 1 -- start start 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 Processor -- start 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acc2d50 0x55bb9acd1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9acaab90 con 0x55bb9acd2ef0 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9abad4b0 con 0x55bb9acc2d50 2024-04-25T10:07:08.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.201+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb9acaae70 con 0x55bb9acd2b10 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfc1e59740 10 client.0 _ll_register_callbacks cb 0x55bb9abed180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53656/0 (socket says 172.21.15.148:53656) 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 -- 172.21.15.148:0/1012622702 learned_addr learned my addr 172.21.15.148:0/1012622702 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acc2d50 0x55bb9acd1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 -- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 msgr2=0x55bb9acd8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 -- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acc2d50 msgr2=0x55bb9acd1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acc2d50 0x55bb9acd1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 -- 172.21.15.148:0/1012622702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb9a9ed7d0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd0af0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8058680 tx=0x7fcfa8087a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae4ada80af9f775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcf977fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcf977fe640 1 -- 172.21.15.148:0/1012622702 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8057220 con 0x55bb9acd2ef0 2024-04-25T10:07:08.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcf977fe640 1 -- 172.21.15.148:0/1012622702 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcfa80573c0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfa805e0c0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcf977fe640 1 -- 172.21.15.148:0/1012622702 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfa806fb60 con 0x55bb9acd2ef0 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfc1e59740 10 client.11931 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55bb9a9233f0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.202+0000 7fcf977fe640 1 -- 172.21.15.148:0/1012622702 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa805e0c0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcf977fe640 1 -- 172.21.15.148:0/1012622702 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcfa8053b20 con 0x55bb9acd2ef0 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfc1e59740 10 client.11931 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfc1e59740 10 client.11931 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfc1e59740 20 client.11931 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9acd7590 0x55bb9acd79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfc1e59740 4 client.11931 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:08.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb9ac1c2e0 con 0x55bb9acd7590 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9acd7590 0x55bb9acd79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcf977fe640 10 client.11931 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9acd7590 0x55bb9acd79a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927d6a8485c4f99b server_cookie=d54b9e76d9f61ead in_seq=0 out_seq=0 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.203+0000 7fcf977fe640 1 -- 172.21.15.148:0/1012622702 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb9ac1c2e0 con 0x55bb9acd7590 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcf977fe640 10 client.11931 handle_command_reply: tid=0 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 client.11931 shutdown 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 msgr2=0x55bb9acd0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd0af0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8058680 tx=0x7fcfa8087a50 comp rx=0 tx=0).stop 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 shutdown_connections 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9acd7590 0x55bb9acd79a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acc2d50 0x55bb9acd1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9acd0af0 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/1012622702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2b10 0x55bb9acd8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcf977fe640 0 client.11931 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 >> 172.21.15.148:0/1012622702 conn(0x55bb9aca5770 msgr2=0x55bb9aca6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 shutdown_connections 2024-04-25T10:07:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 1 -- 172.21.15.148:0/1012622702 wait complete. 2024-04-25T10:07:08.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.204+0000 7fcfc1e59740 20 client.11931 trim_cache size 0 max 16384 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.219+0000 7fcfc1e59740 1 Processor -- start 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.219+0000 7fcfc1e59740 1 -- start start 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 0x55bb9acf07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9acf0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2ef0 0x55bb9acf2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb82007c0 con 0x55bb9acc2d50 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81ff9d0 con 0x55bb9acd2b10 2024-04-25T10:07:08.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb80b6370 con 0x55bb9acd2ef0 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 0x55bb9acf07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 0x55bb9acf07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53668/0 (socket says 172.21.15.148:53668) 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 -- 172.21.15.148:0/3115272708 learned_addr learned my addr 172.21.15.148:0/3115272708 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 -- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2ef0 msgr2=0x55bb9acf2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2ef0 0x55bb9acf2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9acf0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 -- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 msgr2=0x55bb9acf0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9acf0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 -- 172.21.15.148:0/3115272708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb9a8e30f0 con 0x55bb9acc2d50 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9acf0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 0x55bb9acf07e0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009680 tx=0x7fcfb002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf651cbf511d903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfadffb640 1 -- 172.21.15.148:0/3115272708 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0050070 con 0x55bb9acc2d50 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfadffb640 1 -- 172.21.15.148:0/3115272708 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb004c420 con 0x55bb9acc2d50 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.220+0000 7fcfadffb640 1 -- 172.21.15.148:0/3115272708 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0036040 con 0x55bb9acc2d50 2024-04-25T10:07:08.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3115272708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 msgr2=0x55bb9acf07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 0x55bb9acf07e0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009680 tx=0x7fcfb002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3115272708 shutdown_connections 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acd2ef0 0x55bb9acf2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9acf0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3115272708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acc2d50 0x55bb9acf07e0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3115272708 >> 172.21.15.148:0/3115272708 conn(0x55bb9aca5770 msgr2=0x55bb9acd7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3115272708 shutdown_connections 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3115272708 wait complete. 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.221+0000 7fcfc1e59740 1 -- start start 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 Processor -- start 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acc2d50 0x55bb9aca4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9ad5d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9ad01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81ff3d0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.236 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb9acab070 con 0x55bb9acd2b10 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb9acd20f0 con 0x55bb9acc2d50 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.222+0000 7fcfc1e59740 10 client.0 _ll_register_callbacks cb 0x7fcfa8057ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9ad01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9ad01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:53676/0 (socket says 172.21.15.148:53676) 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 -- 172.21.15.148:0/3645740999 learned_addr learned my addr 172.21.15.148:0/3645740999 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acc2d50 0x55bb9aca4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfb7fff640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9ad5d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 -- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acc2d50 msgr2=0x55bb9aca4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acc2d50 0x55bb9aca4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 -- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 msgr2=0x55bb9ad5d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9ad5d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.237 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 -- 172.21.15.148:0/3645740999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb9acf0dd0 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfbd6f9640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9ad01fb0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009f60 tx=0x7fcfa0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4910e70ac51e7a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcf977fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcf977fe640 1 -- 172.21.15.148:0/3645740999 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000f040 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfb004c420 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcf977fe640 1 -- 172.21.15.148:0/3645740999 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcfa0004230 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcf977fe640 1 -- 172.21.15.148:0/3645740999 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfa0027050 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 10 client.11937 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55bb9acf3690 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcf977fe640 1 -- 172.21.15.148:0/3645740999 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb004c420 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcf977fe640 1 -- 172.21.15.148:0/3645740999 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcfa0003610 con 0x55bb9acd2ef0 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 10 client.11937 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 10 client.11937 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:08.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 20 client.11937 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9ad024f0 0x55bb9acd1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 4 client.11937 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.223+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb9ac16630 con 0x55bb9ad024f0 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9ad024f0 0x55bb9acd1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcf977fe640 10 client.11937 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfbcef8640 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9ad024f0 0x55bb9acd1240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffc6d564aed3dbe server_cookie=763701d851dca42 in_seq=0 out_seq=0 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcf977fe640 1 -- 172.21.15.148:0/3645740999 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55bb9ac16630 con 0x55bb9ad024f0 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcf977fe640 10 client.11937 handle_command_reply: tid=0 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 client.11937 shutdown 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 msgr2=0x55bb9ad01fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9ad01fb0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009f60 tx=0x7fcfa0009f90 comp rx=0 tx=0).stop 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 shutdown_connections 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55bb9ad024f0 0x55bb9acd1240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bb9acd2ef0 0x55bb9ad01fb0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcf977fe640 0 client.11937 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bb9acd2b10 0x55bb9ad5d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 --2- 172.21.15.148:0/3645740999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55bb9acc2d50 0x55bb9aca4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 >> 172.21.15.148:0/3645740999 conn(0x55bb9aca5770 msgr2=0x55bb9acd7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 shutdown_connections 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.224+0000 7fcfc1e59740 1 -- 172.21.15.148:0/3645740999 wait complete. 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfc1e59740 20 client.11937 trim_cache size 0 max 16384 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stdout: "f1c1ce9d-e3ba-468b-b589-06b7cb7206c9": { 2024-04-25T10:07:08.240 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-04-25T10:07:08.241 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "f1c1ce9d-e3ba-468b-b589-06b7cb7206c9", 2024-04-25T10:07:08.241 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive,repair" 2024-04-25T10:07:08.241 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:07:08.241 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:07:08.241 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:07:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf9806b500 msgr2=0x7fcf9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf9806b500 0x7fcf9806d9c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0033000 tx=0x7fcfb0009000 comp rx=0 tx=0).stop 2024-04-25T10:07:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 msgr2=0x7fcfb80ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb80ad030 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c3c0 tx=0x7fcfa800c880 comp rx=0 tx=0).stop 2024-04-25T10:07:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 shutdown_connections 2024-04-25T10:07:08.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fcf9806b500 0x7fcf9806d9c0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfb81616f0 0x7fcfb80ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfb81610f0 0x7fcfb80ad030 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 --2- 172.21.15.148:0/240379234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfb80ce7b0 0x7fcfb8149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 >> 172.21.15.148:0/240379234 conn(0x7fcfb806f500 msgr2=0x7fcfb8156620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 shutdown_connections 2024-04-25T10:07:08.243 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:08.225+0000 7fcfbf15c640 1 -- 172.21.15.148:0/240379234 wait complete. 2024-04-25T10:07:08.243 INFO:tasks.cephfs.filesystem:scrub status for tag:f1c1ce9d-e3ba-468b-b589-06b7cb7206c9 - {'path': '/', 'tag': 'f1c1ce9d-e3ba-468b-b589-06b7cb7206c9', 'options': 'recursive,repair'} 2024-04-25T10:07:08.776 INFO:tasks.ceph.mds.c.smithi148.stderr:2024-04-25T10:07:08.771+0000 7f76d0482640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:26.056 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:26.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:38.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:07:38.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.351+0000 7f6add322640 1 Processor -- start 2024-04-25T10:07:38.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.351+0000 7f6add322640 1 -- start start 2024-04-25T10:07:38.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6add322640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81616f0 0x7f6ad8161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6add322640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ad80ce0a0 0x7f6ad80ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6add322640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad806c8d0 con 0x7f6ad81610f0 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6add322640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad81614d0 con 0x7f6ad80ce0a0 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6add322640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80c6960 con 0x7f6ad81616f0 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6ad77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ad80ce0a0 0x7f6ad80ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6ad77fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6ad80ce0a0 0x7f6ad80ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36020/0 (socket says 172.21.15.148:36020) 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.352+0000 7f6ad77fe640 1 -- 172.21.15.148:0/4167481589 learned_addr learned my addr 172.21.15.148:0/4167481589 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.353+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/4167481589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81616f0 0x7f6ad8161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.353+0000 7f6ad77fe640 1 -- 172.21.15.148:0/4167481589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81616f0 msgr2=0x7f6ad8161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.353+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/4167481589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81616f0 0x7f6ad8161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.353+0000 7f6ad77fe640 1 -- 172.21.15.148:0/4167481589 >> v1:172.21.15.188:6789/0 conn(0x7f6ad81610f0 legacy=0x7f6ad806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.353+0000 7f6ad77fe640 1 -- 172.21.15.148:0/4167481589 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8162260 con 0x7f6ad80ce0a0 2024-04-25T10:07:38.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.353+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/4167481589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad80ce530 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009870 tx=0x7f6ad002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1748e323074ceaa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6ad5ffb640 1 -- 172.21.15.148:0/4167481589 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0037070 con 0x7f6ad80ce0a0 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6ad5ffb640 1 -- 172.21.15.148:0/4167481589 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad0032440 con 0x7f6ad80ce0a0 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6ad5ffb640 1 -- 172.21.15.148:0/4167481589 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0032810 con 0x7f6ad80ce0a0 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 -- 172.21.15.148:0/4167481589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad80ce0a0 msgr2=0x7f6ad80ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 --2- 172.21.15.148:0/4167481589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad80ce530 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009870 tx=0x7f6ad002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 -- 172.21.15.148:0/4167481589 shutdown_connections 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 --2- 172.21.15.148:0/4167481589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad80ce530 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 --2- 172.21.15.148:0/4167481589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81616f0 0x7f6ad8161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 -- 172.21.15.148:0/4167481589 >> 172.21.15.148:0/4167481589 conn(0x7f6ad806f500 msgr2=0x7f6ad806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 -- 172.21.15.148:0/4167481589 shutdown_connections 2024-04-25T10:07:38.362 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.354+0000 7f6add322640 1 -- 172.21.15.148:0/4167481589 wait complete. 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 Processor -- start 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 -- start start 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad81e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81610f0 0x7f6ad81597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 0x7f6ad8157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80cfce0 con 0x7f6ad81616f0 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80cfa00 con 0x7f6ad80ce0a0 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6add322640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80cf690 con 0x7f6ad81610f0 2024-04-25T10:07:38.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 0x7f6ad8157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 0x7f6ad8157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36022/0 (socket says 172.21.15.148:36022) 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.356+0000 7f6ad77fe640 1 -- 172.21.15.148:0/1617852914 learned_addr learned my addr 172.21.15.148:0/1617852914 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad81e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81610f0 0x7f6ad81597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad77fe640 1 -- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81610f0 msgr2=0x7f6ad81597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81610f0 0x7f6ad81597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad77fe640 1 -- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad80ce0a0 msgr2=0x7f6ad81e99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad81e99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad77fe640 1 -- 172.21.15.148:0/1617852914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8158a30 con 0x7f6ad81616f0 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad81e99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 0x7f6ad8157be0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009870 tx=0x7f6ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6abf7fe640 1 -- 172.21.15.148:0/1617852914 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0037040 con 0x7f6ad81616f0 2024-04-25T10:07:38.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6abf7fe640 1 -- 172.21.15.148:0/1617852914 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad0042070 con 0x7f6ad81616f0 2024-04-25T10:07:38.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad8158d10 con 0x7f6ad81616f0 2024-04-25T10:07:38.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6abf7fe640 1 -- 172.21.15.148:0/1617852914 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0030bd0 con 0x7f6ad81616f0 2024-04-25T10:07:38.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.357+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad81592d0 con 0x7f6ad81616f0 2024-04-25T10:07:38.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.359+0000 7f6abf7fe640 1 -- 172.21.15.148:0/1617852914 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ad0047460 con 0x7f6ad81616f0 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.359+0000 7f6abf7fe640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ac406b4d0 0x7f6ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.359+0000 7f6abf7fe640 1 -- 172.21.15.148:0/1617852914 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6ad00ab040 con 0x7f6ad81616f0 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.359+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ac406b4d0 0x7f6ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.359+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ac406b4d0 0x7f6ac406d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80045b0 tx=0x7f6ac8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.360+0000 7f6ae0022740 1 Processor -- start 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.360+0000 7f6ae0022740 1 -- start start 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f43c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f43c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f40c770 0x56493f40ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ae0022740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493f375da0 con 0x56493f439ef0 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493f31eb80 con 0x56493f439b10 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56493f371e40 con 0x56493f40c770 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f43c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f43c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36036/0 (socket says 172.21.15.148:36036) 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 -- 172.21.15.148:0/2257042577 learned_addr learned my addr 172.21.15.148:0/2257042577 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f43c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f40c770 0x56493f40ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 -- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f40c770 msgr2=0x56493f40ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f40c770 0x56493f40ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 -- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 msgr2=0x56493f43c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f43c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 -- 172.21.15.148:0/2257042577 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56493f328f10 con 0x56493f439ef0 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.361+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f43c860 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c950 tx=0x7f6acc00ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b08f82727c0cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6abcff9640 1 -- 172.21.15.148:0/2257042577 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc004240 con 0x56493f439ef0 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6abcff9640 1 -- 172.21.15.148:0/2257042577 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc019070 con 0x56493f439ef0 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6abcff9640 1 -- 172.21.15.148:0/2257042577 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00e810 con 0x56493f439ef0 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 -- 172.21.15.148:0/2257042577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 msgr2=0x56493f43c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f43c860 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00c950 tx=0x7f6acc00ce10 comp rx=0 tx=0).stop 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 -- 172.21.15.148:0/2257042577 shutdown_connections 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f40c770 0x56493f40ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f43c860 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2257042577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f43c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 -- 172.21.15.148:0/2257042577 >> 172.21.15.148:0/2257042577 conn(0x56493f40e9e0 msgr2=0x56493f457320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 -- 172.21.15.148:0/2257042577 shutdown_connections 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.362+0000 7f6ae0022740 1 -- 172.21.15.148:0/2257042577 wait complete. 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.363+0000 7f6ae0022740 1 -- start start 2024-04-25T10:07:38.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.363+0000 7f6ae0022740 1 Processor -- start 2024-04-25T10:07:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f4599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439ef0 0x56493f4327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f4537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493f411b90 con 0x56493f4310e0 2024-04-25T10:07:38.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493f3144b0 con 0x56493f439ef0 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56493f411e70 con 0x56493f439b10 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 10 client.0 _ll_register_callbacks cb 0x56493f354180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f4537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f4537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36044/0 (socket says 172.21.15.148:36044) 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 -- 172.21.15.148:0/3343955276 learned_addr learned my addr 172.21.15.148:0/3343955276 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f4599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439ef0 0x56493f4327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 -- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 msgr2=0x56493f4599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f4599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 -- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439ef0 msgr2=0x56493f4327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439ef0 0x56493f4327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 -- 172.21.15.148:0/3343955276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56493f1547d0 con 0x56493f4310e0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f4537c0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0079300 tx=0x7f6ad0079dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514aac5c35ca4093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/3343955276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0091180 con 0x56493f4310e0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/3343955276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6ad0045360 con 0x56493f4310e0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.364+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6acc019070 con 0x56493f4310e0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/3343955276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad0045560 con 0x56493f4310e0 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 10 client.11949 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:38.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x56493f08a3f0 con 0x56493f4310e0 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/3343955276 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc019070 con 0x56493f4310e0 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/3343955276 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ad003a030 con 0x56493f4310e0 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 10 client.11949 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 10 client.11949 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 20 client.11949 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f452ad0 0x56493f474670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 4 client.11949 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56493f3832e0 con 0x56493f452ad0 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f452ad0 0x56493f474670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f452ad0 0x56493f474670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc1959ac984420a server_cookie=7da55fa571c11f77 in_seq=0 out_seq=0 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.365+0000 7f6ab5ffb640 10 client.11949 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/3343955276 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56493f3832e0 con 0x56493f452ad0 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ab5ffb640 10 client.11949 handle_command_reply: tid=0 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 client.11949 shutdown 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 msgr2=0x56493f4537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f4537c0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0079300 tx=0x7f6ad0079dc0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 shutdown_connections 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f452ad0 0x56493f474670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f4537c0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439ef0 0x56493f4327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ab5ffb640 0 client.11949 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3343955276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f4599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 >> 172.21.15.148:0/3343955276 conn(0x56493f40c770 msgr2=0x56493f438dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 shutdown_connections 2024-04-25T10:07:38.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 1 -- 172.21.15.148:0/3343955276 wait complete. 2024-04-25T10:07:38.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.366+0000 7f6ae0022740 20 client.11949 trim_cache size 0 max 16384 2024-04-25T10:07:38.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.381+0000 7f6ae0022740 1 Processor -- start 2024-04-25T10:07:38.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.381+0000 7f6ae0022740 1 -- start start 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f470460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f4577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439ef0 0x56493f40b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ae0022740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad82007a0 con 0x56493f4310e0 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81ff9b0 con 0x56493f439b10 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80b4e30 con 0x56493f439ef0 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f470460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f470460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36056/0 (socket says 172.21.15.148:36056) 2024-04-25T10:07:38.389 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 -- 172.21.15.148:0/3041057435 learned_addr learned my addr 172.21.15.148:0/3041057435 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439ef0 0x56493f40b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f4577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 -- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439ef0 msgr2=0x56493f40b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439ef0 0x56493f40b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 -- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 msgr2=0x56493f4577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f4577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 -- 172.21.15.148:0/3041057435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56493f04a0f0 con 0x56493f4310e0 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.382+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f470460 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8007d80 tx=0x7f6ac8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345085e0d3c98f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ab7fff640 1 -- 172.21.15.148:0/3041057435 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8026590 con 0x56493f4310e0 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ab7fff640 1 -- 172.21.15.148:0/3041057435 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8026730 con 0x56493f4310e0 2024-04-25T10:07:38.390 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ab7fff640 1 -- 172.21.15.148:0/3041057435 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8007230 con 0x56493f4310e0 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 -- 172.21.15.148:0/3041057435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 msgr2=0x56493f470460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f470460 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8007d80 tx=0x7f6ac8014000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 -- 172.21.15.148:0/3041057435 shutdown_connections 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439ef0 0x56493f40b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f439b10 0x56493f4577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 --2- 172.21.15.148:0/3041057435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f4310e0 0x56493f470460 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 -- 172.21.15.148:0/3041057435 >> 172.21.15.148:0/3041057435 conn(0x56493f40c770 msgr2=0x56493f453560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 -- 172.21.15.148:0/3041057435 shutdown_connections 2024-04-25T10:07:38.391 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.383+0000 7f6ae0022740 1 -- 172.21.15.148:0/3041057435 wait complete. 2024-04-25T10:07:38.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.384+0000 7f6ae0022740 1 -- start start 2024-04-25T10:07:38.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.384+0000 7f6ae0022740 1 Processor -- start 2024-04-25T10:07:38.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.384+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f4310e0 0x56493f4c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.395 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.384+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f40b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.384+0000 7f6ae0022740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f40bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.384+0000 7f6ae0022740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad81ff5d0 con 0x56493f439ef0 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56493f32ce00 con 0x56493f4310e0 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ae0022740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56493f450ee0 con 0x56493f439b10 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ae0022740 10 client.0 _ll_register_callbacks cb 0x7f6acc00c1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f40bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f40bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36072/0 (socket says 172.21.15.148:36072) 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 -- 172.21.15.148:0/2022787314 learned_addr learned my addr 172.21.15.148:0/2022787314 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f4310e0 0x56493f4c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.396 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad67fc640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f40b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 -- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 msgr2=0x56493f40b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f40b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 -- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f4310e0 msgr2=0x56493f4c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f4310e0 0x56493f4c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 -- 172.21.15.148:0/2022787314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56493f42fc00 con 0x56493f439ef0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f4310e0 0x56493f4c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ad77fe640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f40bb40 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0075f80 tx=0x7f6ad00a9280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436bf5023b962455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/2022787314 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0096070 con 0x56493f439ef0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/2022787314 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6ad0079a70 con 0x56493f439ef0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ac8026730 con 0x56493f439ef0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/2022787314 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad0078f60 con 0x56493f439ef0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ae0022740 10 client.11955 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x56493f42fc00 con 0x56493f439ef0 2024-04-25T10:07:38.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/2022787314 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8026730 con 0x56493f439ef0 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.385+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/2022787314 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ad0078a50 con 0x56493f439ef0 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 10 client.11955 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 10 client.11955 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 20 client.11955 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f476bc0 0x56493f44ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 4 client.11955 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56493f432340 con 0x56493f476bc0 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f476bc0 0x56493f44ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ab5ffb640 10 client.11955 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ad6ffd640 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f476bc0 0x56493f44ffc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e4acabd675eea2f server_cookie=1d7b493f9c1f4c65 in_seq=0 out_seq=0 2024-04-25T10:07:38.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ab5ffb640 1 -- 172.21.15.148:0/2022787314 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56493f432340 con 0x56493f476bc0 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ab5ffb640 10 client.11955 handle_command_reply: tid=0 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 client.11955 shutdown 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 msgr2=0x56493f40bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f40bb40 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0075f80 tx=0x7f6ad00a9280 comp rx=0 tx=0).stop 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 shutdown_connections 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x56493f476bc0 0x56493f44ffc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56493f439ef0 0x56493f40bb40 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56493f439b10 0x56493f40b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ab5ffb640 0 client.11955 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 --2- 172.21.15.148:0/2022787314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56493f4310e0 0x56493f4c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 >> 172.21.15.148:0/2022787314 conn(0x56493f40c770 msgr2=0x56493f453510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 shutdown_connections 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.386+0000 7f6ae0022740 1 -- 172.21.15.148:0/2022787314 wait complete. 2024-04-25T10:07:38.399 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6ae0022740 20 client.11955 trim_cache size 0 max 16384 2024-04-25T10:07:38.400 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:07:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:07:38.400 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:07:38.400 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ac406b4d0 msgr2=0x7f6ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ac406b4d0 0x7f6ac406d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80045b0 tx=0x7f6ac8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 msgr2=0x7f6ad8157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 0x7f6ad8157be0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009870 tx=0x7f6ad00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 shutdown_connections 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6ac406b4d0 0x7f6ac406d990 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad81616f0 0x7f6ad8157be0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ad81610f0 0x7f6ad81597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 --2- 172.21.15.148:0/1617852914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ad80ce0a0 0x7f6ad81e99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 >> 172.21.15.148:0/1617852914 conn(0x7f6ad806f500 msgr2=0x7f6ad80b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 shutdown_connections 2024-04-25T10:07:38.402 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.387+0000 7f6add322640 1 -- 172.21.15.148:0/1617852914 wait complete. 2024-04-25T10:07:38.404 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:07:38.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:07:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.499+0000 7fc412c8f640 1 Processor -- start 2024-04-25T10:07:38.507 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc412c8f640 1 -- start start 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc412c8f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc40c0cc4f0 0x7fc40c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc412c8f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc40c15b980 0x7fc40c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc412c8f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40c072280 con 0x7fc40c0cdb20 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc412c8f640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c071270 con 0x7fc40c0cc4f0 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc412c8f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c15bd80 con 0x7fc40c15b980 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.500+0000 7fc410a2b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc40c0cc4f0 0x7fc40c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc40c0cc4f0 0x7fc40c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36080/0 (socket says 172.21.15.148:36080) 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc40bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc40c0cdb20 0x7fc40c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37274/0 (socket says 172.21.15.148:37274) 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 -- 172.21.15.148:0/2450015495 learned_addr learned my addr 172.21.15.148:0/2450015495 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc41122c640 1 --2- 172.21.15.148:0/2450015495 >> v2:172.21.15.188:3300/0 conn(0x7fc40c15b980 0x7fc40c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 -- 172.21.15.148:0/2450015495 >> v2:172.21.15.188:3300/0 conn(0x7fc40c15b980 msgr2=0x7fc40c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 --2- 172.21.15.148:0/2450015495 >> v2:172.21.15.188:3300/0 conn(0x7fc40c15b980 0x7fc40c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 -- 172.21.15.148:0/2450015495 >> v1:172.21.15.148:6789/0 conn(0x7fc40c0cdb20 legacy=0x7fc40c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 -- 172.21.15.148:0/2450015495 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0714c0 con 0x7fc40c0cc4f0 2024-04-25T10:07:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc40b7fe640 1 -- 172.21.15.148:0/2450015495 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418112530 0 0) 0x7fc40c072280 con 0x7fc40c0cdb20 2024-04-25T10:07:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.501+0000 7fc410a2b640 1 --2- 172.21.15.148:0/2450015495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c0cc8d0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0b4c00 tx=0x7fc3fc02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2001b8b822063151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc40b7fe640 1 -- 172.21.15.148:0/2450015495 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc036020 con 0x7fc40c0cc4f0 2024-04-25T10:07:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc40b7fe640 1 -- 172.21.15.148:0/2450015495 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc032640 con 0x7fc40c0cc4f0 2024-04-25T10:07:38.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc40b7fe640 1 -- 172.21.15.148:0/2450015495 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc032a10 con 0x7fc40c0cc4f0 2024-04-25T10:07:38.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 -- 172.21.15.148:0/2450015495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cc4f0 msgr2=0x7fc40c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 --2- 172.21.15.148:0/2450015495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c0cc8d0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0b4c00 tx=0x7fc3fc02c940 comp rx=0 tx=0).stop 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 -- 172.21.15.148:0/2450015495 shutdown_connections 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 --2- 172.21.15.148:0/2450015495 >> v2:172.21.15.188:3300/0 conn(0x7fc40c15b980 0x7fc40c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 --2- 172.21.15.148:0/2450015495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c0cc8d0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 -- 172.21.15.148:0/2450015495 >> 172.21.15.148:0/2450015495 conn(0x7fc40c06abe0 msgr2=0x7fc40c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 -- 172.21.15.148:0/2450015495 shutdown_connections 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.502+0000 7fc412c8f640 1 -- 172.21.15.148:0/2450015495 wait complete. 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.503+0000 7fc412c8f640 1 Processor -- start 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.503+0000 7fc412c8f640 1 -- start start 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc412c8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc412c8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 0x7fc40c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc412c8f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40c15b980 0x7fc40c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc412c8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c071d60 con 0x7fc40c0cdb20 2024-04-25T10:07:38.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc412c8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c071a80 con 0x7fc40c0cc4f0 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc412c8f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c0ae500 con 0x7fc40c15b980 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 0x7fc40c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 0x7fc40c0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36096/0 (socket says 172.21.15.148:36096) 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.504+0000 7fc40bfff640 1 -- 172.21.15.148:0/462745902 learned_addr learned my addr 172.21.15.148:0/462745902 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc410a2b640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc41122c640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40c15b980 0x7fc40c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc40bfff640 1 -- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40c15b980 msgr2=0x7fc40c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc40bfff640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40c15b980 0x7fc40c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc40bfff640 1 -- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40c0cc4f0 msgr2=0x7fc40c1579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc40bfff640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c1579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc40bfff640 1 -- 172.21.15.148:0/462745902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0ad930 con 0x7fc40c0cdb20 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc41122c640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40c15b980 0x7fc40c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc40bfff640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 0x7fc40c0ac8b0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fc40000c4d0 tx=0x7fc40000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc4097fa640 1 -- 172.21.15.148:0/462745902 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400018070 con 0x7fc40c0cdb20 2024-04-25T10:07:38.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc4097fa640 1 -- 172.21.15.148:0/462745902 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc400007500 con 0x7fc40c0cdb20 2024-04-25T10:07:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c0ad370 con 0x7fc40c0cdb20 2024-04-25T10:07:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc4097fa640 1 -- 172.21.15.148:0/462745902 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400004a20 con 0x7fc40c0cdb20 2024-04-25T10:07:38.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.505+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c1520d0 con 0x7fc40c0cdb20 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.507+0000 7fc4097fa640 1 -- 172.21.15.148:0/462745902 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc40001b070 con 0x7fc40c0cdb20 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.507+0000 7fc4097fa640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc3f806b4d0 0x7fc3f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.507+0000 7fc4097fa640 1 -- 172.21.15.148:0/462745902 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc4000896e0 con 0x7fc40c0cdb20 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.507+0000 7fc410a2b640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc3f806b4d0 0x7fc3f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.508+0000 7fc410a2b640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc3f806b4d0 0x7fc3f806d990 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc037000 tx=0x7fc3fc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.508+0000 7fc41598c740 1 Processor -- start 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.508+0000 7fc41598c740 1 -- start start 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8970abb0 0x55cf8970af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf89705830 0x55cf897097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41598c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf8970a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41598c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8966eac0 con 0x55cf89709cf0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf89617d80 con 0x55cf89705830 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf8966ab60 con 0x55cf8970abb0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf8970a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf8970a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36112/0 (socket says 172.21.15.148:36112) 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 -- 172.21.15.148:0/2037172079 learned_addr learned my addr 172.21.15.148:0/2037172079 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc410a2b640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8970abb0 0x55cf8970af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc40bfff640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf89705830 0x55cf897097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 -- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8970abb0 msgr2=0x55cf8970af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8970abb0 0x55cf8970af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 -- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf89705830 msgr2=0x55cf897097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf89705830 0x55cf897097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 -- 172.21.15.148:0/2037172079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf8944b7e0 con 0x55cf89709cf0 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc40bfff640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf89705830 0x55cf897097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc410a2b640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8970abb0 0x55cf8970af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc41122c640 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf8970a130 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fc4040074f0 tx=0x7fc404009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9227e9d4ac33b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc3f27fc640 1 -- 172.21.15.148:0/2037172079 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4040147e0 con 0x55cf89709cf0 2024-04-25T10:07:38.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc3f27fc640 1 -- 172.21.15.148:0/2037172079 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40401a070 con 0x55cf89709cf0 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.509+0000 7fc3f27fc640 1 -- 172.21.15.148:0/2037172079 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404014b80 con 0x55cf89709cf0 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 -- 172.21.15.148:0/2037172079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 msgr2=0x55cf8970a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf8970a130 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7fc4040074f0 tx=0x7fc404009750 comp rx=0 tx=0).stop 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 -- 172.21.15.148:0/2037172079 shutdown_connections 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf8970a130 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf89705830 0x55cf897097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 --2- 172.21.15.148:0/2037172079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8970abb0 0x55cf8970af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 -- 172.21.15.148:0/2037172079 >> 172.21.15.148:0/2037172079 conn(0x55cf89707cb0 msgr2=0x55cf89750770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 -- 172.21.15.148:0/2037172079 shutdown_connections 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.510+0000 7fc41598c740 1 -- 172.21.15.148:0/2037172079 wait complete. 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.511+0000 7fc41598c740 1 -- start start 2024-04-25T10:07:38.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.511+0000 7fc41598c740 1 Processor -- start 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf89731ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf8972b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89756700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf89736740 con 0x55cf8970abb0 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8960d530 con 0x55cf8972ba10 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf89735bc0 con 0x55cf89709cf0 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 10 client.0 _ll_register_callbacks cb 0x55cf89665130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf8972b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf8972b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36116/0 (socket says 172.21.15.148:36116) 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 -- 172.21.15.148:0/3882354394 learned_addr learned my addr 172.21.15.148:0/3882354394 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 -- 172.21.15.148:0/3882354394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 msgr2=0x55cf89731ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf89731ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 -- 172.21.15.148:0/3882354394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 msgr2=0x55cf89756700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89756700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 -- 172.21.15.148:0/3882354394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf894abf70 con 0x55cf8970abb0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc40bfff640 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf8972b370 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fc40000ce90 tx=0x7fc400057d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd03ffb5af2ff0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc3eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3882354394 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400004bc0 con 0x55cf8970abb0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc40401a070 con 0x55cf8970abb0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3882354394 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc400004d60 con 0x55cf8970abb0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3882354394 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4000570f0 con 0x55cf8970abb0 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 10 client.11970 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:38.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55cf89343170 con 0x55cf8970abb0 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.512+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3882354394 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40401a070 con 0x55cf8970abb0 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3882354394 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc400057290 con 0x55cf8970abb0 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc41598c740 10 client.11970 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc41598c740 10 client.11970 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc41598c740 20 client.11970 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc41598c740 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf89754010 0x55cf89754e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc41598c740 4 client.11970 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf8967c000 con 0x55cf89754010 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc410a2b640 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf89754010 0x55cf89754e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc3eb7fe640 10 client.11970 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc410a2b640 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf89754010 0x55cf89754e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00190a18d7aae7 server_cookie=5105c09edd3c64ab in_seq=0 out_seq=0 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/3882354394 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf8967c000 con 0x55cf89754010 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.513+0000 7fc3eb7fe640 10 client.11970 handle_command_reply: tid=0 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 client.11970 shutdown 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 msgr2=0x55cf8972b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf8972b370 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7fc40000ce90 tx=0x7fc400057d60 comp rx=0 tx=0).stop 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 shutdown_connections 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf89754010 0x55cf89754e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89756700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf8972b370 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc3eb7fe640 0 client.11970 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 --2- 172.21.15.148:0/3882354394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf89731ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 >> 172.21.15.148:0/3882354394 conn(0x55cf89705830 msgr2=0x55cf89704f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 shutdown_connections 2024-04-25T10:07:38.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 1 -- 172.21.15.148:0/3882354394 wait complete. 2024-04-25T10:07:38.532 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.514+0000 7fc41598c740 20 client.11970 trim_cache size 0 max 16384 2024-04-25T10:07:38.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 Processor -- start 2024-04-25T10:07:38.534 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 -- start start 2024-04-25T10:07:38.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf89745d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8970abb0 0x55cf8972b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.537 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8972ba10 0x55cf8972fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c200640 con 0x55cf89709cf0 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c1ff7d0 con 0x55cf8970abb0 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c1476d0 con 0x55cf8972ba10 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.529+0000 7fc410a2b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf89745d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf89745d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36120/0 (socket says 172.21.15.148:36120) 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 -- 172.21.15.148:0/3048553064 learned_addr learned my addr 172.21.15.148:0/3048553064 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc40bfff640 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8970abb0 0x55cf8972b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.538 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc41122c640 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8972ba10 0x55cf8972fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 -- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8972ba10 msgr2=0x55cf8972fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8972ba10 0x55cf8972fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 -- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8970abb0 msgr2=0x55cf8972b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8970abb0 0x55cf8972b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 -- 172.21.15.148:0/3048553064 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf89706b70 con 0x55cf89709cf0 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc40bfff640 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8970abb0 0x55cf8972b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc410a2b640 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf89745d50 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc008010 tx=0x7fc3fc031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6469d22d306de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc3f1ffb640 1 -- 172.21.15.148:0/3048553064 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc036040 con 0x55cf89709cf0 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc3f1ffb640 1 -- 172.21.15.148:0/3048553064 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc03ba70 con 0x55cf89709cf0 2024-04-25T10:07:38.539 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.530+0000 7fc3f1ffb640 1 -- 172.21.15.148:0/3048553064 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc03be20 con 0x55cf89709cf0 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 -- 172.21.15.148:0/3048553064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 msgr2=0x55cf89745d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf89745d50 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc008010 tx=0x7fc3fc031000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 -- 172.21.15.148:0/3048553064 shutdown_connections 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf8972ba10 0x55cf8972fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8970abb0 0x55cf8972b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 --2- 172.21.15.148:0/3048553064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf89709cf0 0x55cf89745d50 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 -- 172.21.15.148:0/3048553064 >> 172.21.15.148:0/3048553064 conn(0x55cf89705830 msgr2=0x55cf8976ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 -- 172.21.15.148:0/3048553064 shutdown_connections 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 -- 172.21.15.148:0/3048553064 wait complete. 2024-04-25T10:07:38.540 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 -- start start 2024-04-25T10:07:38.541 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.531+0000 7fc41598c740 1 Processor -- start 2024-04-25T10:07:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf8972c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf897555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89704e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf89707040 con 0x55cf8970abb0 2024-04-25T10:07:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c1ff1d0 con 0x55cf8972ba10 2024-04-25T10:07:38.545 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cf89701190 con 0x55cf89709cf0 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc41598c740 10 client.0 _ll_register_callbacks cb 0x7fc404009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf897555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc40bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf897555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36126/0 (socket says 172.21.15.148:36126) 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.532+0000 7fc40bfff640 1 -- 172.21.15.148:0/1853575817 learned_addr learned my addr 172.21.15.148:0/1853575817 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41122c640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89704e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc410a2b640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf8972c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc40bfff640 1 -- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 msgr2=0x55cf8972c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc40bfff640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf8972c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc40bfff640 1 -- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 msgr2=0x55cf89704e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc40bfff640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89704e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc40bfff640 1 -- 172.21.15.148:0/1853575817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf897046c0 con 0x55cf8970abb0 2024-04-25T10:07:38.546 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc410a2b640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf8972c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc40bfff640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf897555e0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7fc4000561f0 tx=0x7fc40005a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eecf73bf303ef30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc3eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/1853575817 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000583a0 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/1853575817 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc40001ed40 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3fc03ba70 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/1853575817 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc400056b60 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 10 client.11976 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55cf89705420 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/1853575817 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc03ba70 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/1853575817 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc400056d00 con 0x55cf8970abb0 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 10 client.11976 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:38.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 10 client.11976 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 20 client.11976 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf897bbaa0 0x55cf897bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 4 client.11976 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55cf89676350 con 0x55cf897bbaa0 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.533+0000 7fc410a2b640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf897bbaa0 0x55cf897bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc3eb7fe640 10 client.11976 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc410a2b640 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf897bbaa0 0x55cf897bbeb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8419a5694f10de65 server_cookie=8f9a6f6f89aca393 in_seq=0 out_seq=0 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc3eb7fe640 1 -- 172.21.15.148:0/1853575817 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cf89676350 con 0x55cf897bbaa0 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc3eb7fe640 10 client.11976 handle_command_reply: tid=0 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 client.11976 shutdown 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 msgr2=0x55cf897555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf897555e0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7fc4000561f0 tx=0x7fc40005a040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 shutdown_connections 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55cf897bbaa0 0x55cf897bbeb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cf8972ba10 0x55cf89704e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8970abb0 0x55cf897555e0 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.534+0000 7fc41598c740 1 --2- 172.21.15.148:0/1853575817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cf89709cf0 0x55cf8972c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc3eb7fe640 0 client.11976 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 >> 172.21.15.148:0/1853575817 conn(0x55cf89705830 msgr2=0x55cf8970af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 shutdown_connections 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc41598c740 1 -- 172.21.15.148:0/1853575817 wait complete. 2024-04-25T10:07:38.549 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc41598c740 20 client.11976 trim_cache size 0 max 16384 2024-04-25T10:07:38.550 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:07:38.550 INFO:teuthology.orchestra.run.smithi148.stdout: "return_code": 0, 2024-04-25T10:07:38.550 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_tag": "0e454c02-6633-48b1-8ef1-adb91c828370", 2024-04-25T10:07:38.550 INFO:teuthology.orchestra.run.smithi148.stdout: "mode": "asynchronous" 2024-04-25T10:07:38.550 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc3f806b4d0 msgr2=0x7fc3f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc3f806b4d0 0x7fc3f806d990 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc037000 tx=0x7fc3fc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 msgr2=0x7fc40c0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 0x7fc40c0ac8b0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7fc40000c4d0 tx=0x7fc40000c990 comp rx=0 tx=0).stop 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 shutdown_connections 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc3f806b4d0 0x7fc3f806d990 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40c15b980 0x7fc40c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc40c0cdb20 0x7fc40c0ac8b0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 --2- 172.21.15.148:0/462745902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40c0cc4f0 0x7fc40c1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 >> 172.21.15.148:0/462745902 conn(0x7fc40c06abe0 msgr2=0x7fc40c1597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 shutdown_connections 2024-04-25T10:07:38.553 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.535+0000 7fc412c8f640 1 -- 172.21.15.148:0/462745902 wait complete. 2024-04-25T10:07:38.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:38.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:07:38.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.647+0000 7f67fc914640 1 Processor -- start 2024-04-25T10:07:38.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.647+0000 7f67fc914640 1 -- start start 2024-04-25T10:07:38.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.648+0000 7f67fc914640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67f815de80 0x7f67f80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.655 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.648+0000 7f67fc914640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67f80cefb0 0x7f67f80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.648+0000 7f67fc914640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f80afc80 con 0x7f67f80cf980 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.648+0000 7f67fc914640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f80ce4d0 con 0x7f67f80cefb0 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.648+0000 7f67fc914640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f80ce7b0 con 0x7f67f815de80 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f5d9b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f67f80cefb0 0x7f67f80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f6d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f67f80cf980 0x7f67f806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:37278/0 (socket says 172.21.15.148:37278) 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f6d9d640 1 -- 172.21.15.148:0/4238615542 learned_addr learned my addr 172.21.15.148:0/4238615542 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f659c640 1 --2- 172.21.15.148:0/4238615542 >> v2:172.21.15.188:3300/0 conn(0x7f67f815de80 0x7f67f80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f5d9b640 1 -- 172.21.15.148:0/4238615542 >> v2:172.21.15.188:3300/0 conn(0x7f67f815de80 msgr2=0x7f67f80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/4238615542 >> v2:172.21.15.188:3300/0 conn(0x7f67f815de80 0x7f67f80cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f5d9b640 1 -- 172.21.15.148:0/4238615542 >> v1:172.21.15.148:6789/0 conn(0x7f67f80cf980 legacy=0x7f67f806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f559a640 1 -- 172.21.15.148:0/4238615542 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894144405 0 0) 0x7f67f80afc80 con 0x7f67f80cf980 2024-04-25T10:07:38.656 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f5d9b640 1 -- 172.21.15.148:0/4238615542 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80cea90 con 0x7f67f80cefb0 2024-04-25T10:07:38.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.649+0000 7f67f659c640 1 --2- 172.21.15.148:0/4238615542 >> v2:172.21.15.188:3300/0 conn(0x7f67f815de80 0x7f67f80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:38.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/4238615542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f80cf440 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f67f80d1800 tx=0x7f67ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eee40519bca5826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67f559a640 1 -- 172.21.15.148:0/4238615542 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec037070 con 0x7f67f80cefb0 2024-04-25T10:07:38.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67f559a640 1 -- 172.21.15.148:0/4238615542 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec032440 con 0x7f67f80cefb0 2024-04-25T10:07:38.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67f559a640 1 -- 172.21.15.148:0/4238615542 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec032810 con 0x7f67f80cefb0 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 -- 172.21.15.148:0/4238615542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 msgr2=0x7f67f80cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 --2- 172.21.15.148:0/4238615542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f80cf440 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f67f80d1800 tx=0x7f67ec02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 -- 172.21.15.148:0/4238615542 shutdown_connections 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 --2- 172.21.15.148:0/4238615542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f80cf440 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 --2- 172.21.15.148:0/4238615542 >> v2:172.21.15.188:3300/0 conn(0x7f67f815de80 0x7f67f80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 -- 172.21.15.148:0/4238615542 >> 172.21.15.148:0/4238615542 conn(0x7f67f80b2390 msgr2=0x7f67f80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 -- 172.21.15.148:0/4238615542 shutdown_connections 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.650+0000 7f67fc914640 1 -- 172.21.15.148:0/4238615542 wait complete. 2024-04-25T10:07:38.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.652+0000 7f67fc914640 1 Processor -- start 2024-04-25T10:07:38.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.652+0000 7f67fc914640 1 -- start start 2024-04-25T10:07:38.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f81494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67f80cf980 0x7f67f80c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67f815de80 0x7f67f80c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80b3030 con 0x7f67f80cefb0 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80b2eb0 con 0x7f67f80cf980 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80c5600 con 0x7f67f815de80 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f81494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f81494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36144/0 (socket says 172.21.15.148:36144) 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 -- 172.21.15.148:0/395352259 learned_addr learned my addr 172.21.15.148:0/395352259 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67f80cf980 0x7f67f80c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f6d9d640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67f815de80 0x7f67f80c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 -- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67f815de80 msgr2=0x7f67f80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67f815de80 0x7f67f80c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 -- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67f80cf980 msgr2=0x7f67f80c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67f80cf980 0x7f67f80c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.661 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 -- 172.21.15.148:0/395352259 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8158f00 con 0x7f67f80cefb0 2024-04-25T10:07:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67f659c640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f81494a0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f67e800ce30 tx=0x7f67e800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67e6ffd640 1 -- 172.21.15.148:0/395352259 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800f890 con 0x7f67f80cefb0 2024-04-25T10:07:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67e6ffd640 1 -- 172.21.15.148:0/395352259 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e800fa30 con 0x7f67f80cefb0 2024-04-25T10:07:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67e6ffd640 1 -- 172.21.15.148:0/395352259 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800fe00 con 0x7f67f80cefb0 2024-04-25T10:07:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f81591e0 con 0x7f67f80cefb0 2024-04-25T10:07:38.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.653+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f81597a0 con 0x7f67f80cefb0 2024-04-25T10:07:38.663 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.655+0000 7f67e6ffd640 1 -- 172.21.15.148:0/395352259 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67e8018400 con 0x7f67f80cefb0 2024-04-25T10:07:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.655+0000 7f67e6ffd640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67d806b4d0 0x7f67d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.656+0000 7f67e6ffd640 1 -- 172.21.15.148:0/395352259 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67e808b220 con 0x7f67f80cefb0 2024-04-25T10:07:38.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.656+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67d806b4d0 0x7f67d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.656+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67d806b4d0 0x7f67d806d990 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f67ec034000 tx=0x7f67ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.656+0000 7f67ff618740 1 Processor -- start 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.656+0000 7f67ff618740 1 -- start start 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67ff618740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf994770 0x5635bf997f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67ff618740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bf8fdda0 con 0x5635bf9c1ef0 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bf8a6b80 con 0x5635bf9c1b10 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635bf8f9e40 con 0x5635bf994770 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36152/0 (socket says 172.21.15.148:36152) 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 -- 172.21.15.148:0/3477977004 learned_addr learned my addr 172.21.15.148:0/3477977004 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f659c640 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f6d9d640 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf994770 0x5635bf997f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 -- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf994770 msgr2=0x5635bf997f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf994770 0x5635bf997f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 -- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 msgr2=0x5635bf9c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.657+0000 7f67f5d9b640 1 -- 172.21.15.148:0/3477977004 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635bf8b0f10 con 0x5635bf9c1ef0 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9c4860 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009560 tx=0x7f67ec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b0213463e823b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67d7fff640 1 -- 172.21.15.148:0/3477977004 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec04e070 con 0x5635bf9c1ef0 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67d7fff640 1 -- 172.21.15.148:0/3477977004 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec049470 con 0x5635bf9c1ef0 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67d7fff640 1 -- 172.21.15.148:0/3477977004 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec037040 con 0x5635bf9c1ef0 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 -- 172.21.15.148:0/3477977004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 msgr2=0x5635bf9c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9c4860 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009560 tx=0x7f67ec02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 -- 172.21.15.148:0/3477977004 shutdown_connections 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf994770 0x5635bf997f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9c4860 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 --2- 172.21.15.148:0/3477977004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.658+0000 7f67ff618740 1 -- 172.21.15.148:0/3477977004 >> 172.21.15.148:0/3477977004 conn(0x5635bf9969e0 msgr2=0x5635bf9df320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.659+0000 7f67ff618740 1 -- 172.21.15.148:0/3477977004 shutdown_connections 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.659+0000 7f67ff618740 1 -- 172.21.15.148:0/3477977004 wait complete. 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.659+0000 7f67ff618740 1 -- start start 2024-04-25T10:07:38.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.659+0000 7f67ff618740 1 Processor -- start 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bf9e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 0x5635bf998f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bf999b90 con 0x5635bf9e1f30 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bf89c4b0 con 0x5635bf9c1b10 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635bf999e70 con 0x5635bf9c1ef0 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67ff618740 10 client.0 _ll_register_callbacks cb 0x5635bf8dc180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 0x5635bf998f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 0x5635bf998f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36162/0 (socket says 172.21.15.148:36162) 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 -- 172.21.15.148:0/1298478592 learned_addr learned my addr 172.21.15.148:0/1298478592 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bf9e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f659c640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.674 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 -- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 msgr2=0x5635bf9e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bf9e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 -- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 msgr2=0x5635bf9b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.660+0000 7f67f6d9d640 1 -- 172.21.15.148:0/1298478592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635bf6dc7d0 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67f6d9d640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 0x5635bf998f90 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005cc0 tx=0x7f67f0001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fb4d54c93847c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67d57fa640 1 -- 172.21.15.148:0/1298478592 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0019060 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67d57fa640 1 -- 172.21.15.148:0/1298478592 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67f00114c0 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67ec049470 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67d57fa640 1 -- 172.21.15.148:0/1298478592 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67f001f050 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 10 client.11991 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x5635bf6123f0 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67d57fa640 1 -- 172.21.15.148:0/1298478592 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec049470 con 0x5635bf9e1f30 2024-04-25T10:07:38.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67d57fa640 1 -- 172.21.15.148:0/1298478592 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67f0022020 con 0x5635bf9e1f30 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 10 client.11991 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 10 client.11991 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 20 client.11991 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9da250 0x5635bf9da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 4 client.11991 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635bf90b2e0 con 0x5635bf9da250 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.661+0000 7f67f659c640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9da250 0x5635bf9da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67d57fa640 10 client.11991 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67f659c640 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9da250 0x5635bf9da660 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669c6f5cb63637bc server_cookie=b480a0c2ff7507b5 in_seq=0 out_seq=0 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67d57fa640 1 -- 172.21.15.148:0/1298478592 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5635bf90b2e0 con 0x5635bf9da250 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67d57fa640 10 client.11991 handle_command_reply: tid=0 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 client.11991 shutdown 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 msgr2=0x5635bf998f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.676 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 0x5635bf998f90 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005cc0 tx=0x7f67f0001680 comp rx=0 tx=0).stop 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 shutdown_connections 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9da250 0x5635bf9da660 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9e1f30 0x5635bf998f90 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bf9e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67d57fa640 0 client.11991 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 --2- 172.21.15.148:0/1298478592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 >> 172.21.15.148:0/1298478592 conn(0x5635bf994770 msgr2=0x5635bf9c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 shutdown_connections 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 1 -- 172.21.15.148:0/1298478592 wait complete. 2024-04-25T10:07:38.677 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.662+0000 7f67ff618740 20 client.11991 trim_cache size 0 max 16384 2024-04-25T10:07:38.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.677+0000 7f67ff618740 1 Processor -- start 2024-04-25T10:07:38.683 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.677+0000 7f67ff618740 1 -- start start 2024-04-25T10:07:38.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67ff618740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9e1f30 0x5635bfa49160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67ff618740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f8200640 con 0x5635bf9c1ef0 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81ff760 con 0x5635bf9c1b10 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80aeb40 con 0x5635bf9e1f30 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9937f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36168/0 (socket says 172.21.15.148:36168) 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 -- 172.21.15.148:0/773743629 learned_addr learned my addr 172.21.15.148:0/773743629 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 -- 172.21.15.148:0/773743629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9e1f30 msgr2=0x5635bfa49160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9e1f30 0x5635bfa49160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 -- 172.21.15.148:0/773743629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 msgr2=0x5635bf9c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.686 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 -- 172.21.15.148:0/773743629 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635bf5d20f0 con 0x5635bf9c1ef0 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.678+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9937f0 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009860 tx=0x7f67ec03b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ff174070a3625e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67d77fe640 1 -- 172.21.15.148:0/773743629 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec04e070 con 0x5635bf9c1ef0 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67d77fe640 1 -- 172.21.15.148:0/773743629 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec03e6d0 con 0x5635bf9c1ef0 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67d77fe640 1 -- 172.21.15.148:0/773743629 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec037040 con 0x5635bf9c1ef0 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 -- 172.21.15.148:0/773743629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 msgr2=0x5635bf9937f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9937f0 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009860 tx=0x7f67ec03b000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 -- 172.21.15.148:0/773743629 shutdown_connections 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9e1f30 0x5635bfa49160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1ef0 0x5635bf9937f0 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 --2- 172.21.15.148:0/773743629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9c1b10 0x5635bf9c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 -- 172.21.15.148:0/773743629 >> 172.21.15.148:0/773743629 conn(0x5635bf994770 msgr2=0x5635bf9b86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 -- 172.21.15.148:0/773743629 shutdown_connections 2024-04-25T10:07:38.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.679+0000 7f67ff618740 1 -- 172.21.15.148:0/773743629 wait complete. 2024-04-25T10:07:38.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.680+0000 7f67ff618740 1 -- start start 2024-04-25T10:07:38.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.680+0000 7f67ff618740 1 Processor -- start 2024-04-25T10:07:38.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 0x5635bfa48850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bfa4af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9e1f30 0x5635bf9df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635bf9dc330 con 0x5635bf9c1b10 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81ff160 con 0x5635bf9e1f30 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635bf98fd50 con 0x5635bf9c1ef0 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 10 client.0 _ll_register_callbacks cb 0x7f67ec035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 0x5635bfa48850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 0x5635bfa48850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:36180/0 (socket says 172.21.15.148:36180) 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 -- 172.21.15.148:0/1139983185 learned_addr learned my addr 172.21.15.148:0/1139983185 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:07:38.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bfa4af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f6d9d640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9e1f30 0x5635bf9df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 -- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 msgr2=0x5635bfa4af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bfa4af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 -- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9e1f30 msgr2=0x5635bf9df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9e1f30 0x5635bf9df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 -- 172.21.15.148:0/1139983185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635bf9ee590 con 0x5635bf9c1b10 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67f659c640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 0x5635bfa48850 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f67e8059450 tx=0x7f67e808b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bf9f452be86adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67d57fa640 1 -- 172.21.15.148:0/1139983185 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8086020 con 0x5635bf9c1b10 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67d57fa640 1 -- 172.21.15.148:0/1139983185 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67e8072de0 con 0x5635bf9c1b10 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.681+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67ec03e6d0 con 0x5635bf9c1b10 2024-04-25T10:07:38.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67d57fa640 1 -- 172.21.15.148:0/1139983185 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67e8082e60 con 0x5635bf9c1b10 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 10 client.11997 fetch_fsmap learned FSMap version 119 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x5635bfa4b480 con 0x5635bf9c1b10 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67d57fa640 1 -- 172.21.15.148:0/1139983185 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec03e6d0 con 0x5635bf9c1b10 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67d57fa640 1 -- 172.21.15.148:0/1139983185 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67e8020030 con 0x5635bf9c1b10 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 10 client.11997 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 10 client.11997 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 20 client.11997 populate_metadata read hostname 'smithi148' 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9b86f0 0x5635bfa4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 4 client.11997 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635bf905630 con 0x5635bf9b86f0 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9b86f0 0x5635bfa4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67d57fa640 10 client.11997 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67f5d9b640 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9b86f0 0x5635bfa4c240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d1af97ab77774a8 server_cookie=cb0f7f23a08d09f in_seq=0 out_seq=0 2024-04-25T10:07:38.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67d57fa640 1 -- 172.21.15.148:0/1139983185 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5635bf905630 con 0x5635bf9b86f0 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67d57fa640 10 client.11997 handle_command_reply: tid=0 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.682+0000 7f67ff618740 1 client.11997 shutdown 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 msgr2=0x5635bfa48850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 0x5635bfa48850 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f67e8059450 tx=0x7f67e808b040 comp rx=0 tx=0).stop 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 shutdown_connections 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x5635bf9b86f0 0x5635bfa4c240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635bf9e1f30 0x5635bf9df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5635bf9c1ef0 0x5635bfa4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67d57fa640 0 client.11997 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 --2- 172.21.15.148:0/1139983185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5635bf9c1b10 0x5635bfa48850 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 >> 172.21.15.148:0/1139983185 conn(0x5635bf994770 msgr2=0x5635bf9c0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 shutdown_connections 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 1 -- 172.21.15.148:0/1139983185 wait complete. 2024-04-25T10:07:38.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67ff618740 20 client.11997 trim_cache size 0 max 16384 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": { 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: "0e454c02-6633-48b1-8ef1-adb91c828370": { 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: "path": "/", 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: "tag": "0e454c02-6633-48b1-8ef1-adb91c828370", 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: "options": "recursive,repair" 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-04-25T10:07:38.697 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67d806b4d0 msgr2=0x7f67d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67d806b4d0 0x7f67d806d990 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f67ec034000 tx=0x7f67ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 msgr2=0x7f67f81494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f81494a0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f67e800ce30 tx=0x7f67e800a650 comp rx=0 tx=0).stop 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 shutdown_connections 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f67d806b4d0 0x7f67d806d990 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67f815de80 0x7f67f80c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67f80cf980 0x7f67f80c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 --2- 172.21.15.148:0/395352259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67f80cefb0 0x7f67f81494a0 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 >> 172.21.15.148:0/395352259 conn(0x7f67f80b2390 msgr2=0x7f67f815c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 shutdown_connections 2024-04-25T10:07:38.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:07:38.683+0000 7f67fc914640 1 -- 172.21.15.148:0/395352259 wait complete. 2024-04-25T10:07:38.701 INFO:tasks.cephfs.filesystem:scrub status for tag:0e454c02-6633-48b1-8ef1-adb91c828370 - {'path': '/', 'tag': '0e454c02-6633-48b1-8ef1-adb91c828370', 'options': 'recursive,repair'} 2024-04-25T10:07:56.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:56.215 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:08.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:08.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:08:08.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.811+0000 7f928fbdb640 1 Processor -- start 2024-04-25T10:08:08.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.811+0000 7f928fbdb640 1 -- start start 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928fbdb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f928806ccd0 0x7f92880b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928fbdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 0x7f928815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928fbdb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92880d1770 con 0x7f9288070dc0 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928fbdb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92881622b0 con 0x7f928806ccd0 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928fbdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880b10f0 con 0x7f928815deb0 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928d176640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f928806ccd0 0x7f92880b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928e178640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 0x7f928815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928e178640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 0x7f928815e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:53954/0 (socket says 172.21.15.148:53954) 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928d176640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f928806ccd0 0x7f92880b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:41978/0 (socket says 172.21.15.148:41978) 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928e178640 1 -- 172.21.15.148:0/1445525335 learned_addr learned my addr 172.21.15.148:0/1445525335 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928d176640 1 -- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 msgr2=0x7f928815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928d176640 1 --2- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 0x7f928815e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928d176640 1 -- 172.21.15.148:0/1445525335 >> v1:172.21.15.188:6789/0 conn(0x7f9288070dc0 legacy=0x7f92880711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928c975640 1 -- 172.21.15.148:0/1445525335 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471277257 0 0) 0x7f92880d1770 con 0x7f9288070dc0 2024-04-25T10:08:08.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928d176640 1 -- 172.21.15.148:0/1445525335 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288162740 con 0x7f928806ccd0 2024-04-25T10:08:08.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.812+0000 7f928e178640 1 --2- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 0x7f928815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:08:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.813+0000 7f928d176640 1 --2- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880b07a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f927c0097a0 tx=0x7f927c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16fb03e517c654c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.813+0000 7f928c975640 1 -- 172.21.15.148:0/1445525335 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c037070 con 0x7f928806ccd0 2024-04-25T10:08:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.813+0000 7f928c975640 1 -- 172.21.15.148:0/1445525335 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c02adb0 con 0x7f928806ccd0 2024-04-25T10:08:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.813+0000 7f928c975640 1 -- 172.21.15.148:0/1445525335 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c032660 con 0x7f928806ccd0 2024-04-25T10:08:08.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 -- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 msgr2=0x7f92880b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 --2- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880b07a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f927c0097a0 tx=0x7f927c02a060 comp rx=0 tx=0).stop 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 -- 172.21.15.148:0/1445525335 shutdown_connections 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 --2- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880b07a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 --2- 172.21.15.148:0/1445525335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928815deb0 0x7f928815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 -- 172.21.15.148:0/1445525335 >> 172.21.15.148:0/1445525335 conn(0x7f92880b3530 msgr2=0x7f92880b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 -- 172.21.15.148:0/1445525335 shutdown_connections 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.814+0000 7f928fbdb640 1 -- 172.21.15.148:0/1445525335 wait complete. 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.815+0000 7f928fbdb640 1 Processor -- start 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.815+0000 7f928fbdb640 1 -- start start 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928fbdb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928fbdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9288070dc0 0x7f928814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928fbdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 0x7f92880cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928fbdb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288161f40 con 0x7f928815deb0 2024-04-25T10:08:08.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928fbdb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288161c60 con 0x7f928806ccd0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928fbdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92880b5840 con 0x7f9288070dc0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 0x7f92880cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 0x7f92880cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54938/0 (socket says 172.21.15.148:54938) 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 -- 172.21.15.148:0/4113040712 learned_addr learned my addr 172.21.15.148:0/4113040712 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928d977640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928d176640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9288070dc0 0x7f928814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 -- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9288070dc0 msgr2=0x7f928814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9288070dc0 0x7f928814b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 -- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 msgr2=0x7f92880ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.816+0000 7f928e178640 1 -- 172.21.15.148:0/4113040712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928814be90 con 0x7f928815deb0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.817+0000 7f928e178640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 0x7f92880cabb0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f927800efc0 tx=0x7f927800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.817+0000 7f92867fc640 1 -- 172.21.15.148:0/4113040712 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927800cbe0 con 0x7f928815deb0 2024-04-25T10:08:08.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.817+0000 7f92867fc640 1 -- 172.21.15.148:0/4113040712 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927800cd80 con 0x7f928815deb0 2024-04-25T10:08:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.817+0000 7f92867fc640 1 -- 172.21.15.148:0/4113040712 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278003bb0 con 0x7f928815deb0 2024-04-25T10:08:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.817+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92880cd6e0 con 0x7f928815deb0 2024-04-25T10:08:08.826 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.817+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92881495e0 con 0x7f928815deb0 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.818+0000 7f92867fc640 1 -- 172.21.15.148:0/4113040712 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f927801b030 con 0x7f928815deb0 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.819+0000 7f92867fc640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f927406b500 0x7f927406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.819+0000 7f928d977640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f927406b500 0x7f927406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.819+0000 7f92867fc640 1 -- 172.21.15.148:0/4113040712 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f927808a600 con 0x7f928815deb0 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.819+0000 7f928d977640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f927406b500 0x7f927406d9c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9280000f60 tx=0x7f9280009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.820+0000 7f92929b0740 1 Processor -- start 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.820+0000 7f92929b0740 1 -- start start 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02ab10 0x55783e02d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e02d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f92929b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 0x55783e000f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f92929b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783df66da0 con 0x55783dffd770 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783df0fb80 con 0x55783e02aef0 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783df62e40 con 0x55783e02ab10 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 0x55783e000f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 0x55783e000f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54940/0 (socket says 172.21.15.148:54940) 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 -- 172.21.15.148:0/684059481 learned_addr learned my addr 172.21.15.148:0/684059481 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928d176640 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e02d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928d977640 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02ab10 0x55783e02d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 -- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02ab10 msgr2=0x55783e02d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02ab10 0x55783e02d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 -- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 msgr2=0x55783e02d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e02d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.821+0000 7f928e178640 1 -- 172.21.15.148:0/684059481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55783df19f10 con 0x55783dffd770 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f928e178640 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 0x55783e000f80 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f9278060da0 tx=0x7f9278058c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19266738ebec5ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f926f7fe640 1 -- 172.21.15.148:0/684059481 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780612d0 con 0x55783dffd770 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f926f7fe640 1 -- 172.21.15.148:0/684059481 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780092e0 con 0x55783dffd770 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f926f7fe640 1 -- 172.21.15.148:0/684059481 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92780552e0 con 0x55783dffd770 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 -- 172.21.15.148:0/684059481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 msgr2=0x55783e000f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 0x55783e000f80 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f9278060da0 tx=0x7f9278058c70 comp rx=0 tx=0).stop 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 -- 172.21.15.148:0/684059481 shutdown_connections 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783dffd770 0x55783e000f80 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e02d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 --2- 172.21.15.148:0/684059481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02ab10 0x55783e02d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 -- 172.21.15.148:0/684059481 >> 172.21.15.148:0/684059481 conn(0x55783dfff9e0 msgr2=0x55783e048320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 -- 172.21.15.148:0/684059481 shutdown_connections 2024-04-25T10:08:08.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.822+0000 7f92929b0740 1 -- 172.21.15.148:0/684059481 wait complete. 2024-04-25T10:08:08.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.823+0000 7f92929b0740 1 -- start start 2024-04-25T10:08:08.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.823+0000 7f92929b0740 1 Processor -- start 2024-04-25T10:08:08.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e0237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.838 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02aef0 0x55783e029190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e04bee0 0x55783e0296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e002b90 con 0x55783e02ab10 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783df054b0 con 0x55783e04bee0 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783e002e70 con 0x55783e02aef0 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f92929b0740 10 client.0 _ll_register_callbacks cb 0x55783df45180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e0237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e0237b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54948/0 (socket says 172.21.15.148:54948) 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 -- 172.21.15.148:0/65200972 learned_addr learned my addr 172.21.15.148:0/65200972 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d176640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02aef0 0x55783e029190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928e178640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e04bee0 0x55783e0296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 -- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02aef0 msgr2=0x55783e029190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02aef0 0x55783e029190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 -- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e04bee0 msgr2=0x55783e0296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e04bee0 0x55783e0296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.824+0000 7f928d977640 1 -- 172.21.15.148:0/65200972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55783dd457d0 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f928d977640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e0237b0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f9280014700 tx=0x7f9280001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4567903c0ebfb322 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f926cff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f926cff9640 1 -- 172.21.15.148:0/65200972 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280025020 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f926cff9640 1 -- 172.21.15.148:0/65200972 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9280013780 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92780092e0 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f926cff9640 1 -- 172.21.15.148:0/65200972 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f928002f050 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 10 client.12006 fetch_fsmap learned FSMap version 119 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55783dc7b3f0 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f926cff9640 1 -- 172.21.15.148:0/65200972 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92780092e0 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f926cff9640 1 -- 172.21.15.148:0/65200972 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9280032020 con 0x55783e02ab10 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 10 client.12006 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 10 client.12006 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:08:08.840 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 20 client.12006 populate_metadata read hostname 'smithi148' 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e0266a0 0x55783e026a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 4 client.12006 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55783df742e0 con 0x55783e0266a0 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.825+0000 7f928d176640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e0266a0 0x55783e026a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f926cff9640 10 client.12006 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f928d176640 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e0266a0 0x55783e026a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfc7b78742f056 server_cookie=9949b44a0692642b in_seq=0 out_seq=0 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f926cff9640 1 -- 172.21.15.148:0/65200972 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55783df742e0 con 0x55783e0266a0 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f926cff9640 10 client.12006 handle_command_reply: tid=0 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 client.12006 shutdown 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 msgr2=0x55783e0237b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e0237b0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f9280014700 tx=0x7f9280001080 comp rx=0 tx=0).stop 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 shutdown_connections 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e0266a0 0x55783e026a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e04bee0 0x55783e0296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f926cff9640 0 client.12006 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e02aef0 0x55783e029190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 --2- 172.21.15.148:0/65200972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e0237b0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 >> 172.21.15.148:0/65200972 conn(0x55783dffd770 msgr2=0x55783e029f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 shutdown_connections 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.826+0000 7f92929b0740 1 -- 172.21.15.148:0/65200972 wait complete. 2024-04-25T10:08:08.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.827+0000 7f92929b0740 20 client.12006 trim_cache size 0 max 16384 2024-04-25T10:08:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.841+0000 7f92929b0740 1 Processor -- start 2024-04-25T10:08:08.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.841+0000 7f92929b0740 1 -- start start 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02ab10 0x55783e022640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f92929b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 0x55783e04b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f92929b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92882005c0 con 0x55783e02aef0 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92881ff780 con 0x55783e02ab10 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92881feb80 con 0x55783e04bee0 2024-04-25T10:08:08.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f928d176640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 0x55783e04b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f928d176640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 0x55783e04b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54960/0 (socket says 172.21.15.148:54960) 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.842+0000 7f928d176640 1 -- 172.21.15.148:0/3054331117 learned_addr learned my addr 172.21.15.148:0/3054331117 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d176640 1 -- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 msgr2=0x55783e04b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d176640 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d176640 1 -- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02ab10 msgr2=0x55783e022640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d977640 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02ab10 0x55783e022640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d176640 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02ab10 0x55783e022640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d176640 1 -- 172.21.15.148:0/3054331117 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55783dc3b0f0 con 0x55783e02aef0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f928d176640 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 0x55783e04b100 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f927c006ce0 tx=0x7f927c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c384db0d008251cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f926effd640 1 -- 172.21.15.148:0/3054331117 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c037040 con 0x55783e02aef0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f926effd640 1 -- 172.21.15.148:0/3054331117 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c004300 con 0x55783e02aef0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f926effd640 1 -- 172.21.15.148:0/3054331117 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c003810 con 0x55783e02aef0 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 -- 172.21.15.148:0/3054331117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 msgr2=0x55783e04b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 0x55783e04b100 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f927c006ce0 tx=0x7f927c02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:08.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 -- 172.21.15.148:0/3054331117 shutdown_connections 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02aef0 0x55783e04b100 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 --2- 172.21.15.148:0/3054331117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02ab10 0x55783e022640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 -- 172.21.15.148:0/3054331117 >> 172.21.15.148:0/3054331117 conn(0x55783dffd770 msgr2=0x55783e02d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 -- 172.21.15.148:0/3054331117 shutdown_connections 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.843+0000 7f92929b0740 1 -- 172.21.15.148:0/3054331117 wait complete. 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.844+0000 7f92929b0740 1 -- start start 2024-04-25T10:08:08.852 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.844+0000 7f92929b0740 1 Processor -- start 2024-04-25T10:08:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e03dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e0489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92881ff180 con 0x55783e02ab10 2024-04-25T10:08:08.856 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55783e06adf0 con 0x55783e02aef0 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55783e03e6c0 con 0x55783e04bee0 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f92929b0740 10 client.0 _ll_register_callbacks cb 0x7f9278009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e03dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e03dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54976/0 (socket says 172.21.15.148:54976) 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 -- 172.21.15.148:0/3712310840 learned_addr learned my addr 172.21.15.148:0/3712310840 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d176640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e0489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928e178640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 -- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 msgr2=0x55783e04b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 -- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 msgr2=0x55783e0489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e0489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 -- 172.21.15.148:0/3712310840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55783e03e1f0 con 0x55783e02ab10 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928e178640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:08.857 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f928d977640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e03dcb0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f9280001570 tx=0x7f9280014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b671120898ee23e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.845+0000 7f926cff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f926cff9640 1 -- 172.21.15.148:0/3712310840 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280025070 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f926cff9640 1 -- 172.21.15.148:0/3712310840 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9280013b40 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f927c004300 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f926cff9640 1 -- 172.21.15.148:0/3712310840 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9280042050 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 10 client.12012 fetch_fsmap learned FSMap version 119 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55783e04a1d0 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f926cff9640 1 -- 172.21.15.148:0/3712310840 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c004300 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f926cff9640 1 -- 172.21.15.148:0/3712310840 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9280045020 con 0x55783e02ab10 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 10 client.12012 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 10 client.12012 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 20 client.12012 populate_metadata read hostname 'smithi148' 2024-04-25T10:08:08.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e048f20 0x55783e04b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 4 client.12012 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55783e04aa20 con 0x55783e048f20 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f928d176640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e048f20 0x55783e04b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f926cff9640 10 client.12012 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.846+0000 7f928d176640 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e048f20 0x55783e04b5f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a79402612e030c1f server_cookie=9bd7a54665eb4547 in_seq=0 out_seq=0 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f926cff9640 1 -- 172.21.15.148:0/3712310840 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55783e04aa20 con 0x55783e048f20 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f926cff9640 10 client.12012 handle_command_reply: tid=0 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 client.12012 shutdown 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 msgr2=0x55783e03dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e03dcb0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f9280001570 tx=0x7f9280014000 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 shutdown_connections 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55783e048f20 0x55783e04b5f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55783e04bee0 0x55783e04b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55783e02aef0 0x55783e0489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f926cff9640 0 client.12012 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 --2- 172.21.15.148:0/3712310840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55783e02ab10 0x55783e03dcb0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 >> 172.21.15.148:0/3712310840 conn(0x55783dffd770 msgr2=0x55783e0255f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 shutdown_connections 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 1 -- 172.21.15.148:0/3712310840 wait complete. 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f92929b0740 20 client.12012 trim_cache size 0 max 16384 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:08.860 INFO:teuthology.orchestra.run.smithi148.stdout: "status": "no active scrubs running", 2024-04-25T10:08:08.861 INFO:teuthology.orchestra.run.smithi148.stdout: "scrubs": {} 2024-04-25T10:08:08.861 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f927406b500 msgr2=0x7f927406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.861 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f928fbdb640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f927406b500 0x7f927406d9c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f9280000f60 tx=0x7f9280009040 comp rx=0 tx=0).stop 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 msgr2=0x7f92880cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f928fbdb640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 0x7f92880cabb0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f927800efc0 tx=0x7f927800c490 comp rx=0 tx=0).stop 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 shutdown_connections 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.847+0000 7f928fbdb640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f927406b500 0x7f927406d9c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.848+0000 7f928fbdb640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f928815deb0 0x7f92880cabb0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.848+0000 7f928fbdb640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9288070dc0 0x7f928814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.848+0000 7f928fbdb640 1 --2- 172.21.15.148:0/4113040712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928806ccd0 0x7f92880ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.848+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 >> 172.21.15.148:0/4113040712 conn(0x7f92880b3530 msgr2=0x7f92880d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.848+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 shutdown_connections 2024-04-25T10:08:08.862 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:08.848+0000 7f928fbdb640 1 -- 172.21.15.148:0/4113040712 wait complete. 2024-04-25T10:08:08.864 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:08:09.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:09.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.360+0000 7fbb02532640 1 Processor -- start 2024-04-25T10:08:09.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.360+0000 7fbb02532640 1 -- start start 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.360+0000 7fbb02532640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbafc0ce8d0 0x7fbafc168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.360+0000 7fbb02532640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc0c8030 con 0x7fbafc15da70 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.360+0000 7fbb02532640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc0c8310 con 0x7fbafc0d0170 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.360+0000 7fbb02532640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc0c85f0 con 0x7fbafc0ce8d0 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbb00acf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbafc0ce8d0 0x7fbafc168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbb00acf640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbafc0ce8d0 0x7fbafc168690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:54992/0 (socket says 172.21.15.148:54992) 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbb00acf640 1 -- 172.21.15.148:0/136862276 learned_addr learned my addr 172.21.15.148:0/136862276 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbafbfff640 1 --1- 172.21.15.148:0/136862276 >> v1:172.21.15.148:6789/0 conn(0x7fbafc15da70 0x7fbafc0c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52238/0 (socket says 172.21.15.148:52238) 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbb00acf640 1 -- 172.21.15.148:0/136862276 >> v1:172.21.15.188:6789/0 conn(0x7fbafc0d0170 legacy=0x7fbafc0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbb00acf640 1 -- 172.21.15.148:0/136862276 >> v1:172.21.15.148:6789/0 conn(0x7fbafc15da70 legacy=0x7fbafc0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbb00acf640 1 -- 172.21.15.148:0/136862276 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc06bb30 con 0x7fbafc0ce8d0 2024-04-25T10:08:09.368 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.361+0000 7fbafaffd640 1 -- 172.21.15.148:0/136862276 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471215018 0 0) 0x7fbafc0c8030 con 0x7fbafc15da70 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb00acf640 1 --2- 172.21.15.148:0/136862276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc168690 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009e50 tx=0x7fbaf402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedc8f39c90f89db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbafaffd640 1 -- 172.21.15.148:0/136862276 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4036070 con 0x7fbafc0ce8d0 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbafaffd640 1 -- 172.21.15.148:0/136862276 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4031600 con 0x7fbafc0ce8d0 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbafaffd640 1 -- 172.21.15.148:0/136862276 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40319f0 con 0x7fbafc0ce8d0 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 -- 172.21.15.148:0/136862276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc0ce8d0 msgr2=0x7fbafc168690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 --2- 172.21.15.148:0/136862276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc168690 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009e50 tx=0x7fbaf402a7c0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.369 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 -- 172.21.15.148:0/136862276 shutdown_connections 2024-04-25T10:08:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 --2- 172.21.15.148:0/136862276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc168690 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 -- 172.21.15.148:0/136862276 >> 172.21.15.148:0/136862276 conn(0x7fbafc0b33e0 msgr2=0x7fbafc0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 -- 172.21.15.148:0/136862276 shutdown_connections 2024-04-25T10:08:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.362+0000 7fbb02532640 1 -- 172.21.15.148:0/136862276 wait complete. 2024-04-25T10:08:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.363+0000 7fbb02532640 1 Processor -- start 2024-04-25T10:08:09.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 -- start start 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbafc0d0170 0x7fbafc14af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 0x7fbafc147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc06d340 con 0x7fbafc15da70 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc15de50 con 0x7fbafc0ce8d0 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb02532640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc06d4c0 con 0x7fbafc0d0170 2024-04-25T10:08:09.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb00acf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 0x7fbafc147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb00acf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 0x7fbafc147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55002/0 (socket says 172.21.15.148:55002) 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.364+0000 7fbb00acf640 1 -- 172.21.15.148:0/3526414942 learned_addr learned my addr 172.21.15.148:0/3526414942 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbafbfff640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbafb7fe640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbafc0d0170 0x7fbafc14af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb00acf640 1 -- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbafc0d0170 msgr2=0x7fbafc14af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb00acf640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbafc0d0170 0x7fbafc14af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb00acf640 1 -- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbafc0ce8d0 msgr2=0x7fbafc160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb00acf640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb00acf640 1 -- 172.21.15.148:0/3526414942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc160e90 con 0x7fbafc15da70 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb00acf640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 0x7fbafc147950 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7fbaf402a800 tx=0x7fbaf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4036050 con 0x7fbafc15da70 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4042070 con 0x7fbafc15da70 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc148540 con 0x7fbafc15da70 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4009510 con 0x7fbafc15da70 2024-04-25T10:08:09.373 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.365+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc148b00 con 0x7fbafc15da70 2024-04-25T10:08:09.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.366+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc073990 con 0x7fbafc15da70 2024-04-25T10:08:09.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.366+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaf4004490 con 0x7fbafc15da70 2024-04-25T10:08:09.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.367+0000 7fbaf8ff9640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbae806b4d0 0x7fbae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.367+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fbaf40ab360 con 0x7fbafc15da70 2024-04-25T10:08:09.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.367+0000 7fbafbfff640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbae806b4d0 0x7fbae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.367+0000 7fbafbfff640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbae806b4d0 0x7fbae806d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbaec000f60 tx=0x7fbaec007b90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.371+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaf407a020 con 0x7fbafc15da70 2024-04-25T10:08:09.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.575+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbafc0cdf50 con 0x7fbafc15da70 2024-04-25T10:08:09.582 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.576+0000 7fbaf8ff9640 1 -- 172.21.15.148:0/3526414942 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4966 (secure 0 0 0) 0x7fbaf4079920 con 0x7fbafc15da70 2024-04-25T10:08:09.582 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:09.582 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:05.546576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:08:09.582 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 119 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbae806b4d0 msgr2=0x7fbae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbae806b4d0 0x7fbae806d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbaec000f60 tx=0x7fbaec007b90 comp rx=0 tx=0).stop 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 msgr2=0x7fbafc147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 0x7fbafc147950 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7fbaf402a800 tx=0x7fbaf4004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 shutdown_connections 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbae806b4d0 0x7fbae806d990 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbafc15da70 0x7fbafc147950 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.584 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbafc0d0170 0x7fbafc14af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 --2- 172.21.15.148:0/3526414942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbafc0ce8d0 0x7fbafc160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 >> 172.21.15.148:0/3526414942 conn(0x7fbafc0b33e0 msgr2=0x7fbafc0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 shutdown_connections 2024-04-25T10:08:09.585 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.578+0000 7fbb02532640 1 -- 172.21.15.148:0/3526414942 wait complete. 2024-04-25T10:08:09.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:52.824899+0000', 'modified': '2024-04-25T10:07:05.546576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11619': {'gid': 11619, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6837/1465938320', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 1465938320}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1465938320}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:08:09.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:08:09.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:08:09.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.687+0000 7fd86dfc1640 1 Processor -- start 2024-04-25T10:08:09.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.688+0000 7fd86dfc1640 1 -- start start 2024-04-25T10:08:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.688+0000 7fd86dfc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 0x7fd8680cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.688+0000 7fd86dfc1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8680ccfe0 0x7fd8680cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.688+0000 7fd86dfc1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8680afc80 con 0x7fd86815de80 2024-04-25T10:08:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.688+0000 7fd86dfc1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680cb210 con 0x7fd8680ccfe0 2024-04-25T10:08:09.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.688+0000 7fd86dfc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680ce4d0 con 0x7fd8680cadb0 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 0x7fd8680cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd867fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8680ccfe0 0x7fd8680cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 0x7fd8680cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:53996/0 (socket says 172.21.15.148:53996) 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd867fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8680ccfe0 0x7fd8680cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:42040/0 (socket says 172.21.15.148:42040) 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd85ffff640 1 -- 172.21.15.148:0/3641586084 learned_addr learned my addr 172.21.15.148:0/3641586084 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd8677fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd86815de80 0x7fd8680cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:36810/0 (socket says 172.21.15.148:36810) 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd866ffd640 1 -- 172.21.15.148:0/3641586084 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802577711 0 0) 0x7fd8680afc80 con 0x7fd86815de80 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd866ffd640 1 -- 172.21.15.148:0/3641586084 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd854003520 con 0x7fd86815de80 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd867fff640 1 -- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 msgr2=0x7fd8680cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd867fff640 1 --2- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 0x7fd8680cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd867fff640 1 -- 172.21.15.148:0/3641586084 >> v1:172.21.15.188:6789/0 conn(0x7fd86815de80 legacy=0x7fd8680cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd867fff640 1 -- 172.21.15.148:0/3641586084 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680ce7b0 con 0x7fd8680ccfe0 2024-04-25T10:08:09.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.689+0000 7fd85ffff640 1 --2- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 0x7fd8680cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.690+0000 7fd867fff640 1 --2- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680cd470 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd860009d90 tx=0x7fd86002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec49418fc9bd267c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.690+0000 7fd866ffd640 1 -- 172.21.15.148:0/3641586084 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860007740 con 0x7fd8680ccfe0 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.690+0000 7fd866ffd640 1 -- 172.21.15.148:0/3641586084 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8600078e0 con 0x7fd8680ccfe0 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.690+0000 7fd866ffd640 1 -- 172.21.15.148:0/3641586084 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860007c90 con 0x7fd8680ccfe0 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 msgr2=0x7fd8680cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680cd470 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd860009d90 tx=0x7fd86002a660 comp rx=0 tx=0).stop 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3641586084 shutdown_connections 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680cd470 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3641586084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8680cadb0 0x7fd8680cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3641586084 >> 172.21.15.148:0/3641586084 conn(0x7fd8680b2390 msgr2=0x7fd8680b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3641586084 shutdown_connections 2024-04-25T10:08:09.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.691+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3641586084 wait complete. 2024-04-25T10:08:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.692+0000 7fd86dfc1640 1 Processor -- start 2024-04-25T10:08:09.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.692+0000 7fd86dfc1640 1 -- start start 2024-04-25T10:08:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 0x7fd8681569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86815de80 0x7fd8680b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86815e260 con 0x7fd8680cadb0 2024-04-25T10:08:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680cfc00 con 0x7fd8680ccfe0 2024-04-25T10:08:09.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b2eb0 con 0x7fd86815de80 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 0x7fd8681569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 0x7fd8681569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55018/0 (socket says 172.21.15.148:55018) 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 -- 172.21.15.148:0/3822029221 learned_addr learned my addr 172.21.15.148:0/3822029221 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd85ffff640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd867fff640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86815de80 0x7fd8680b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 -- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86815de80 msgr2=0x7fd8680b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86815de80 0x7fd8680b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 -- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 msgr2=0x7fd8680b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 -- 172.21.15.148:0/3822029221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680c8520 con 0x7fd8680cadb0 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd8677fe640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 0x7fd8681569c0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c390 tx=0x7fd85800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd864ff9640 1 -- 172.21.15.148:0/3822029221 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580074e0 con 0x7fd8680cadb0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd864ff9640 1 -- 172.21.15.148:0/3822029221 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8580047e0 con 0x7fd8680cadb0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8680c8800 con 0x7fd8680cadb0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.693+0000 7fd864ff9640 1 -- 172.21.15.148:0/3822029221 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858004bd0 con 0x7fd8680cadb0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.694+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8680c8ff0 con 0x7fd8680cadb0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.695+0000 7fd864ff9640 1 -- 172.21.15.148:0/3822029221 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd85801a020 con 0x7fd8680cadb0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.696+0000 7fd864ff9640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd84806b500 0x7fd84806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.696+0000 7fd85ffff640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd84806b500 0x7fd84806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.703 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.696+0000 7fd864ff9640 1 -- 172.21.15.148:0/3822029221 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd858089cc0 con 0x7fd8680cadb0 2024-04-25T10:08:09.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.696+0000 7fd85ffff640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd84806b500 0x7fd84806d9c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd850009630 tx=0x7fd850008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.696+0000 7fd870da1740 1 Processor -- start 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 -- start start 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 0x55ab2cdf7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2ce14ad0 0x55ab2ce15020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2cdccf10 0x55ab2ce13190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2cd2fa10 con 0x55ab2cdf7230 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2ccd8b30 con 0x55ab2cdccf10 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab2cd2bc50 con 0x55ab2ce14ad0 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 0x55ab2cdf7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 0x55ab2cdf7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55030/0 (socket says 172.21.15.148:55030) 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.697+0000 7fd8677fe640 1 -- 172.21.15.148:0/1572375353 learned_addr learned my addr 172.21.15.148:0/1572375353 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd85ffff640 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2ce14ad0 0x55ab2ce15020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.710 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd867fff640 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2cdccf10 0x55ab2ce13190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd8677fe640 1 -- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2ce14ad0 msgr2=0x55ab2ce15020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd8677fe640 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2ce14ad0 0x55ab2ce15020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd8677fe640 1 -- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2cdccf10 msgr2=0x55ab2ce13190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd8677fe640 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2cdccf10 0x55ab2ce13190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd8677fe640 1 -- 172.21.15.148:0/1572375353 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab2ccede10 con 0x55ab2cdf7230 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd867fff640 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2cdccf10 0x55ab2ce13190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd8677fe640 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 0x55ab2cdf7610 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7fd858060330 tx=0x7fd858058210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a117976da3a2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd85d7fa640 1 -- 172.21.15.148:0/1572375353 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85806e070 con 0x55ab2cdf7230 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd85d7fa640 1 -- 172.21.15.148:0/1572375353 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858056450 con 0x55ab2cdf7230 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.698+0000 7fd85d7fa640 1 -- 172.21.15.148:0/1572375353 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580583b0 con 0x55ab2cdf7230 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 -- 172.21.15.148:0/1572375353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 msgr2=0x55ab2cdf7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 0x55ab2cdf7610 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7fd858060330 tx=0x7fd858058210 comp rx=0 tx=0).stop 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 -- 172.21.15.148:0/1572375353 shutdown_connections 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2cdccf10 0x55ab2ce13190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2ce14ad0 0x55ab2ce15020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 --2- 172.21.15.148:0/1572375353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2cdf7230 0x55ab2cdf7610 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 -- 172.21.15.148:0/1572375353 >> 172.21.15.148:0/1572375353 conn(0x55ab2cdc9840 msgr2=0x55ab2ce12040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 -- 172.21.15.148:0/1572375353 shutdown_connections 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 -- 172.21.15.148:0/1572375353 wait complete. 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 -- start start 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.699+0000 7fd870da1740 1 Processor -- start 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce22380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 0x55ab2cdf5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2cdcdcb0 con 0x55ab2ce26610 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2ccec0c0 con 0x55ab2ce14ad0 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab2cdc7770 con 0x55ab2cdf7230 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd870da1740 10 client.0 _ll_register_callbacks cb 0x55ab2ccdfec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd867fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 0x55ab2cdf5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd867fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 0x55ab2cdf5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55036/0 (socket says 172.21.15.148:55036) 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd867fff640 1 -- 172.21.15.148:0/2036411802 learned_addr learned my addr 172.21.15.148:0/2036411802 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd85ffff640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce22380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.700+0000 7fd8677fe640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd867fff640 1 -- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 msgr2=0x55ab2ce10fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd867fff640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd867fff640 1 -- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce14ad0 msgr2=0x55ab2ce22380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd867fff640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce22380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd867fff640 1 -- 172.21.15.148:0/2036411802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab2ca442d0 con 0x55ab2ce26610 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd867fff640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 0x55ab2cdf5de0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fd860009e80 tx=0x7fd860004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cae5465ff95fd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8467fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8467fc640 1 -- 172.21.15.148:0/2036411802 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860009450 con 0x55ab2ce26610 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8467fc640 1 -- 172.21.15.148:0/2036411802 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd86003b410 con 0x55ab2ce26610 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd858056450 con 0x55ab2ce26610 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8467fc640 1 -- 172.21.15.148:0/2036411802 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd86004d050 con 0x55ab2ce26610 2024-04-25T10:08:09.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 10 client.12030 fetch_fsmap learned FSMap version 119 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55ab2cb0e6b0 con 0x55ab2ce26610 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8467fc640 1 -- 172.21.15.148:0/2036411802 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858056450 con 0x55ab2ce26610 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8467fc640 1 -- 172.21.15.148:0/2036411802 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd860050020 con 0x55ab2ce26610 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 10 client.12030 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 10 client.12030 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 20 client.12030 populate_metadata read hostname 'smithi148' 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2cdf8070 0x55ab2cdf3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 4 client.12030 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab2cd8e920 con 0x55ab2cdf8070 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.701+0000 7fd8677fe640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2cdf8070 0x55ab2cdf3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd8467fc640 10 client.12030 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd8677fe640 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2cdf8070 0x55ab2cdf3c50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db65922f652e01c4 server_cookie=da80306bc1d8b3d6 in_seq=0 out_seq=0 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd8467fc640 1 -- 172.21.15.148:0/2036411802 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab2cd8e920 con 0x55ab2cdf8070 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd8467fc640 10 client.12030 handle_command_reply: tid=0 2024-04-25T10:08:09.714 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 client.12030 shutdown 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 msgr2=0x55ab2cdf5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 0x55ab2cdf5de0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fd860009e80 tx=0x7fd860004ab0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 shutdown_connections 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2cdf8070 0x55ab2cdf3c50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce26610 0x55ab2cdf5de0 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce22380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 --2- 172.21.15.148:0/2036411802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 >> 172.21.15.148:0/2036411802 conn(0x55ab2cdccf10 msgr2=0x55ab2ce155c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.702+0000 7fd8467fc640 0 client.12030 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.703+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 shutdown_connections 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.703+0000 7fd870da1740 1 -- 172.21.15.148:0/2036411802 wait complete. 2024-04-25T10:08:09.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.703+0000 7fd870da1740 20 client.12030 trim_cache size 0 max 16384 2024-04-25T10:08:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.718+0000 7fd870da1740 1 Processor -- start 2024-04-25T10:08:09.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.718+0000 7fd870da1740 1 -- start start 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd870da1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2cdf5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2cdcade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd870da1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681ff900 con 0x55ab2ce14ad0 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680ca150 con 0x55ab2ce26610 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681ff410 con 0x55ab2cdf7230 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2cdf5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2cdf5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55048/0 (socket says 172.21.15.148:55048) 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 -- 172.21.15.148:0/1693234845 learned_addr learned my addr 172.21.15.148:0/1693234845 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd8677fe640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd867fff640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2cdcade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 -- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 msgr2=0x55ab2ce10c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 -- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 msgr2=0x55ab2cdcade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2cdcade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.719+0000 7fd85ffff640 1 -- 172.21.15.148:0/1693234845 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab2cdc3550 con 0x55ab2ce14ad0 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd8677fe640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd867fff640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2cdcade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd85ffff640 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2cdf5630 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fd8681f8140 tx=0x7fd850013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d218eb497bc4c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd85cff9640 1 -- 172.21.15.148:0/1693234845 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85001c030 con 0x55ab2ce14ad0 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd85cff9640 1 -- 172.21.15.148:0/1693234845 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85001d040 con 0x55ab2ce14ad0 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd85cff9640 1 -- 172.21.15.148:0/1693234845 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000f040 con 0x55ab2ce14ad0 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd870da1740 1 -- 172.21.15.148:0/1693234845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 msgr2=0x55ab2cdf5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd870da1740 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2cdf5630 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fd8681f8140 tx=0x7fd850013000 comp rx=0 tx=0).stop 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd870da1740 1 -- 172.21.15.148:0/1693234845 shutdown_connections 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd870da1740 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2cdcade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd870da1740 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2cdf5630 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.720+0000 7fd870da1740 1 --2- 172.21.15.148:0/1693234845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2ce10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.721+0000 7fd870da1740 1 -- 172.21.15.148:0/1693234845 >> 172.21.15.148:0/1693234845 conn(0x55ab2cdccf10 msgr2=0x55ab2cdf3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.721+0000 7fd870da1740 1 -- 172.21.15.148:0/1693234845 shutdown_connections 2024-04-25T10:08:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.721+0000 7fd870da1740 1 -- 172.21.15.148:0/1693234845 wait complete. 2024-04-25T10:08:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.721+0000 7fd870da1740 1 -- start start 2024-04-25T10:08:09.729 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.721+0000 7fd870da1740 1 Processor -- start 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2cdf3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2ce2be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2cdf7ef0 con 0x55ab2ce14ad0 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab2ce260f0 con 0x55ab2ce26610 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab2cdf84f0 con 0x55ab2cdf7230 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd870da1740 10 client.0 _ll_register_callbacks cb 0x55ab2cdc7c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55058/0 (socket says 172.21.15.148:55058) 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 -- 172.21.15.148:0/2832197839 learned_addr learned my addr 172.21.15.148:0/2832197839 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd8677fe640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2cdf3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.733 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd867fff640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2ce2be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 -- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 msgr2=0x55ab2cdf3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2cdf3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 -- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 msgr2=0x55ab2ce2be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2ce2be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.722+0000 7fd85ffff640 1 -- 172.21.15.148:0/2832197839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab2cdf4510 con 0x55ab2ce14ad0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd85ffff640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce741f0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fd850008010 tx=0x7fd8500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7b5e7a3105d615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8467fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8467fc640 1 -- 172.21.15.148:0/2832197839 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85001c040 con 0x55ab2ce14ad0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd85001d040 con 0x55ab2ce14ad0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8467fc640 1 -- 172.21.15.148:0/2832197839 <== mon.0 v2:172.21.15.148:3300/0 2 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd850033400 con 0x55ab2ce14ad0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8467fc640 1 -- 172.21.15.148:0/2832197839 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=119) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd85003b050 con 0x55ab2ce14ad0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 10 client.12036 fetch_fsmap learned FSMap version 119 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=119,monmap=2+}) v3 -- 0x55ab2ce5fb90 con 0x55ab2ce14ad0 2024-04-25T10:08:09.734 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8467fc640 1 -- 172.21.15.148:0/2832197839 <== mon.0 v2:172.21.15.148:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85001d040 con 0x55ab2ce14ad0 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8467fc640 1 -- 172.21.15.148:0/2832197839 <== mon.0 v2:172.21.15.148:3300/0 5 ==== fsmap(e 119) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd85003e020 con 0x55ab2ce14ad0 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 10 client.12036 fetch_fsmap finished waiting for FSMap version 119 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 10 client.12036 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 20 client.12036 populate_metadata read hostname 'smithi148' 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2ce62600 0x55ab2ce12fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 4 client.12036 mds_command: new command op to 11619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 --> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ab2cd3cf50 con 0x55ab2ce62600 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.723+0000 7fd8677fe640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2ce62600 0x55ab2ce12fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd8467fc640 10 client.12036 ms_handle_connect on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd8677fe640 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2ce62600 0x55ab2ce12fa0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11fe5bc25caf2ee server_cookie=fd15aa6b3e7c1b50 in_seq=0 out_seq=0 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd8467fc640 1 -- 172.21.15.148:0/2832197839 <== mds.0 v2:172.21.15.148:6836/1465938320 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55ab2cd3cf50 con 0x55ab2ce62600 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd8467fc640 10 client.12036 handle_command_reply: tid=0 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 client.12036 shutdown 2024-04-25T10:08:09.735 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 msgr2=0x55ab2ce741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce741f0 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fd850008010 tx=0x7fd8500046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 shutdown_connections 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:6836/1465938320,v1:172.21.15.148:6837/1465938320] conn(0x55ab2ce62600 0x55ab2ce12fa0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ab2ce26610 0x55ab2ce2be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd8467fc640 0 client.12036 ms_handle_reset on v2:172.21.15.148:6836/1465938320 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ab2ce14ad0 0x55ab2ce741f0 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 --2- 172.21.15.148:0/2832197839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ab2cdf7230 0x55ab2cdf3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 >> 172.21.15.148:0/2832197839 conn(0x55ab2cdccf10 msgr2=0x55ab2cdf5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 shutdown_connections 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 1 -- 172.21.15.148:0/2832197839 wait complete. 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.724+0000 7fd870da1740 20 client.12036 trim_cache size 0 max 16384 2024-04-25T10:08:09.736 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "cluster_fsid": "086ed9ea-68e4-4e62-83c5-c0821c62dd17", 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "whoami": 0, 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "id": 11619, 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "want_state": "up:active", 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "state": "up:active", 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "fs_name": "cephfs", 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "rank_uptime": 66.183946124000002, 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "mdsmap_epoch": 119, 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch": 200, 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_epoch_barrier": 198, 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout: "uptime": 81.297829586999995 2024-04-25T10:08:09.737 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:09.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd84806b500 msgr2=0x7fd84806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd84806b500 0x7fd84806d9c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd850009630 tx=0x7fd850008040 comp rx=0 tx=0).stop 2024-04-25T10:08:09.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 msgr2=0x7fd8681569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.738 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 0x7fd8681569c0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c390 tx=0x7fd85800c850 comp rx=0 tx=0).stop 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 shutdown_connections 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd84806b500 0x7fd84806d9c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86815de80 0x7fd8680b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8680ccfe0 0x7fd8680b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 --2- 172.21.15.148:0/3822029221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8680cadb0 0x7fd8681569c0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 >> 172.21.15.148:0/3822029221 conn(0x7fd8680b2390 msgr2=0x7fd86806fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 shutdown_connections 2024-04-25T10:08:09.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:09.725+0000 7fd86dfc1640 1 -- 172.21.15.148:0/3822029221 wait complete. 2024-04-25T10:08:09.741 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:09.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:09.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:09.818 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:09.818 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:08:09.818 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:09.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:09.834 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:08:09.834 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:09.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:09.877 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:08:09.877 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:09.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:09.892 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:08:09.892 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:09.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:09.934 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:10.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.026+0000 7f76be132640 1 Processor -- start 2024-04-25T10:08:10.032 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.026+0000 7f76be132640 1 -- start start 2024-04-25T10:08:10.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76be132640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 0x7f76b80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76be132640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76b80d1dd0 0x7f76b80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76be132640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b815dfe0 con 0x7f76b80ceb20 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76be132640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b80d00c0 con 0x7f76b80d1dd0 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76be132640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80d03a0 con 0x7f76b815dc00 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b6ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f76b80ceb20 0x7f76b80d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52246/0 (socket says 172.21.15.148:52246) 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b6ffd640 1 -- 172.21.15.148:0/1263981799 learned_addr learned my addr 172.21.15.148:0/1263981799 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747464100 0 0) 0x7f76a8001110 con 0x7f76b80ceb20 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a0003520 con 0x7f76b80ceb20 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b7fff640 1 --2- 172.21.15.148:0/1263981799 >> v2:172.21.15.188:3300/0 conn(0x7f76b80d1dd0 0x7f76b80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b77fe640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 0x7f76b80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2342237506 0 0) 0x7f76a0003520 con 0x7f76b80ceb20 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 msgr2=0x7f76b80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 0x7f76b80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 >> v2:172.21.15.188:3300/0 conn(0x7f76b80d1dd0 msgr2=0x7f76b80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 --2- 172.21.15.148:0/1263981799 >> v2:172.21.15.188:3300/0 conn(0x7f76b80d1dd0 0x7f76b80ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80cc440 con 0x7f76b80ceb20 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f76a8001840 con 0x7f76b80ceb20 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 >> v1:172.21.15.148:6789/0 conn(0x7f76b80ceb20 legacy=0x7f76b80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a00039b0 0x7f76a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76a0004800 0x7f76a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 0x7f76a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80d00c0 con 0x7f76a00039b0 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80d03a0 con 0x7f76a0004800 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.027+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a000b9c0 con 0x7f76b815dc00 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a00039b0 0x7f76a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b7fff640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 0x7f76a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b6ffd640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76a0004800 0x7f76a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 -- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 msgr2=0x7f76a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76b815dc00 0x7f76a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 -- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76a0004800 msgr2=0x7f76a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76a0004800 0x7f76a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 -- 172.21.15.148:0/1263981799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0003520 con 0x7f76a00039b0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b6ffd640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76a0004800 0x7f76a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.028+0000 7f76b77fe640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a00039b0 0x7f76a0003dd0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009ac0 tx=0x7f76a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681369b804b76a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac036070 con 0x7f76a00039b0 2024-04-25T10:08:10.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac031400 con 0x7f76a00039b0 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76b67fc640 1 -- 172.21.15.148:0/1263981799 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac009690 con 0x7f76a00039b0 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76be132640 1 -- 172.21.15.148:0/1263981799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a00039b0 msgr2=0x7f76a0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76be132640 1 --2- 172.21.15.148:0/1263981799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76a00039b0 0x7f76a0003dd0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009ac0 tx=0x7f76a0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76b7fff640 1 -- 172.21.15.148:0/1263981799 reap_dead start 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76be132640 1 -- 172.21.15.148:0/1263981799 shutdown_connections 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76be132640 1 -- 172.21.15.148:0/1263981799 >> 172.21.15.148:0/1263981799 conn(0x7f76b806f5d0 msgr2=0x7f76b806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76be132640 1 -- 172.21.15.148:0/1263981799 shutdown_connections 2024-04-25T10:08:10.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.029+0000 7f76be132640 1 -- 172.21.15.148:0/1263981799 wait complete. 2024-04-25T10:08:10.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.030+0000 7f76be132640 1 Processor -- start 2024-04-25T10:08:10.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 -- start start 2024-04-25T10:08:10.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76a00039b0 0x7f76b80b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.042 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b80b6ef0 0x7f76b81f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 0x7f76b8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80cfd90 con 0x7f76b81f66e0 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80cc130 con 0x7f76b80b6ef0 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76be132640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80cdd70 con 0x7f76a00039b0 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 0x7f76b8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 0x7f76b8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55080/0 (socket says 172.21.15.148:55080) 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76b7fff640 1 -- 172.21.15.148:0/2719830286 learned_addr learned my addr 172.21.15.148:0/2719830286 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76b77fe640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76a00039b0 0x7f76b80b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.031+0000 7f76b6ffd640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b80b6ef0 0x7f76b81f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b7fff640 1 -- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76a00039b0 msgr2=0x7f76b80b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b7fff640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76a00039b0 0x7f76b80b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b7fff640 1 -- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b80b6ef0 msgr2=0x7f76b81f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b7fff640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b80b6ef0 0x7f76b81f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b7fff640 1 -- 172.21.15.148:0/2719830286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80c9940 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b77fe640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76a00039b0 0x7f76b80b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b6ffd640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b80b6ef0 0x7f76b81f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76b7fff640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 0x7f76b8156010 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f76b8070530 tx=0x7f76b000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0015430 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b001d070 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0015830 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b80b6200 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.032+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b80c9ee0 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.033+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b8153430 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.033+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76b00159d0 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.034+0000 7f769ffff640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76a406b4d0 0x7f76a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.034+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f76b008adc0 con 0x7f76b81f66e0 2024-04-25T10:08:10.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.034+0000 7f76b77fe640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76a406b4d0 0x7f76a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.034+0000 7f76b77fe640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76a406b4d0 0x7f76a406d990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f76ac033000 tx=0x7f76ac008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.038+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76b0010040 con 0x7f76b81f66e0 2024-04-25T10:08:10.252 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.246+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76b8073990 con 0x7f76b81f66e0 2024-04-25T10:08:10.253 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.246+0000 7f769ffff640 1 -- 172.21.15.148:0/2719830286 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4966 (secure 0 0 0) 0x7f76b0059310 con 0x7f76b81f66e0 2024-04-25T10:08:10.254 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:10.254 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:05.546576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:08:10.255 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 119 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76a406b4d0 msgr2=0x7f76a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76a406b4d0 0x7f76a406d990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f76ac033000 tx=0x7f76ac008030 comp rx=0 tx=0).stop 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 msgr2=0x7f76b8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 0x7f76b8156010 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f76b8070530 tx=0x7f76b000f580 comp rx=0 tx=0).stop 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 shutdown_connections 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f76a406b4d0 0x7f76a406d990 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b81f66e0 0x7f76b8156010 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76b80b6ef0 0x7f76b81f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 --2- 172.21.15.148:0/2719830286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76a00039b0 0x7f76b80b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.248+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 >> 172.21.15.148:0/2719830286 conn(0x7f76b806f5d0 msgr2=0x7f76b80ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.249+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 shutdown_connections 2024-04-25T10:08:10.257 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.249+0000 7f76be132640 1 -- 172.21.15.148:0/2719830286 wait complete. 2024-04-25T10:08:10.266 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:10.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 Processor -- start 2024-04-25T10:08:10.363 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 -- start start 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43240b3410 0x7f43240720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f43240b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240ace70 con 0x7f43240b41b0 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432406df30 con 0x7f43240b3410 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.357+0000 7f432a3ed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240726d0 con 0x7f43240b37f0 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43240b3410 0x7f43240720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f43240b3410 0x7f43240720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55090/0 (socket says 172.21.15.148:55090) 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f432898a640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f43240b41b0 0x7f4324161870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52262/0 (socket says 172.21.15.148:52262) 2024-04-25T10:08:10.364 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 -- 172.21.15.148:0/2176871710 learned_addr learned my addr 172.21.15.148:0/2176871710 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f43237fe640 1 --2- 172.21.15.148:0/2176871710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f43240b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 -- 172.21.15.148:0/2176871710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 msgr2=0x7f43240b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 --2- 172.21.15.148:0/2176871710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f43240b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 -- 172.21.15.148:0/2176871710 >> v1:172.21.15.148:6789/0 conn(0x7f43240b41b0 legacy=0x7f4324161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.358+0000 7f4323fff640 1 -- 172.21.15.148:0/2176871710 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432406d8e0 con 0x7f43240b3410 2024-04-25T10:08:10.365 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f4323fff640 1 --2- 172.21.15.148:0/2176871710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f43240720f0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f431402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71850138e014422a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f4322ffd640 1 -- 172.21.15.148:0/2176871710 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314037070 con 0x7f43240b3410 2024-04-25T10:08:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f4322ffd640 1 -- 172.21.15.148:0/2176871710 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314032440 con 0x7f43240b3410 2024-04-25T10:08:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f4322ffd640 1 -- 172.21.15.148:0/2176871710 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314032810 con 0x7f43240b3410 2024-04-25T10:08:10.366 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 -- 172.21.15.148:0/2176871710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 msgr2=0x7f43240720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 --2- 172.21.15.148:0/2176871710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f43240720f0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f431402a270 comp rx=0 tx=0).stop 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 -- 172.21.15.148:0/2176871710 shutdown_connections 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 --2- 172.21.15.148:0/2176871710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f43240b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 --2- 172.21.15.148:0/2176871710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f43240720f0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 -- 172.21.15.148:0/2176871710 >> 172.21.15.148:0/2176871710 conn(0x7f43240cfae0 msgr2=0x7f43240cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 -- 172.21.15.148:0/2176871710 shutdown_connections 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.359+0000 7f432a3ed640 1 -- 172.21.15.148:0/2176871710 wait complete. 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.360+0000 7f432a3ed640 1 Processor -- start 2024-04-25T10:08:10.367 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 -- start start 2024-04-25T10:08:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f432415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f4324145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240b41b0 0x7f432413dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.370 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432406f3d0 con 0x7f43240b3410 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324147890 con 0x7f43240b41b0 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f432a3ed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432406f550 con 0x7f43240b37f0 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f4323fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f432415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f4323fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f432415f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55092/0 (socket says 172.21.15.148:55092) 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.361+0000 7f4323fff640 1 -- 172.21.15.148:0/959707687 learned_addr learned my addr 172.21.15.148:0/959707687 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f43237fe640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f4324145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f432898a640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240b41b0 0x7f432413dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4323fff640 1 -- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 msgr2=0x7f4324145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4323fff640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f4324145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4323fff640 1 -- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240b41b0 msgr2=0x7f432413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4323fff640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240b41b0 0x7f432413dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4323fff640 1 -- 172.21.15.148:0/959707687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43241433d0 con 0x7f43240b3410 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f432898a640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240b41b0 0x7f432413dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:10.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4323fff640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f432415f7c0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f43140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314041070 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431403c870 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4324143670 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314037040 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.362+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324143c30 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.363+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324073990 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.364+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4314044070 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.364+0000 7f4320ff9640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f431006b4d0 0x7f431006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.364+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f43140ab2b0 con 0x7f43240b3410 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.364+0000 7f43237fe640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f431006b4d0 0x7f431006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.372 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.364+0000 7f43237fe640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f431006b4d0 0x7f431006d990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f43241423e0 tx=0x7f4318008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.374 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.368+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4314079f60 con 0x7f43240b3410 2024-04-25T10:08:10.547 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.541+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43240b3f90 con 0x7f43240b3410 2024-04-25T10:08:10.548 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.542+0000 7f4320ff9640 1 -- 172.21.15.148:0/959707687 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17260 (secure 0 0 0) 0x7f4314079860 con 0x7f43240b3410 2024-04-25T10:08:10.549 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:10.549 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":200,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:07:07.548514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.14","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/1255014956":"2024-04-26T10:07:02.533393+0000","172.21.15.188:6833/1255014956":"2024-04-26T10:07:02.533393+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f431006b4d0 msgr2=0x7f431006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f431006b4d0 0x7f431006d990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f43241423e0 tx=0x7f4318008040 comp rx=0 tx=0).stop 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 msgr2=0x7f432415f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f432415f7c0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f4314009870 tx=0x7f43140046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 shutdown_connections 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f431006b4d0 0x7f431006d990 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43240b41b0 0x7f432413dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43240b37f0 0x7f4324145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 --2- 172.21.15.148:0/959707687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43240b3410 0x7f432415f7c0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 >> 172.21.15.148:0/959707687 conn(0x7f43240cfae0 msgr2=0x7f43240b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 shutdown_connections 2024-04-25T10:08:10.552 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.545+0000 7f432a3ed640 1 -- 172.21.15.148:0/959707687 wait complete. 2024-04-25T10:08:10.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T09:54:11.048758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:51.040797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:51.800930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:10.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:10.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.656+0000 7f5f19fdb640 1 Processor -- start 2024-04-25T10:08:10.662 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.656+0000 7f5f19fdb640 1 -- start start 2024-04-25T10:08:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f19fdb640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f14070dc0 0x7f5f1406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f19fdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f140711a0 0x7f5f140b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f19fdb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f140d1770 con 0x7f5f140b4830 2024-04-25T10:08:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f19fdb640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f141622b0 con 0x7f5f14070dc0 2024-04-25T10:08:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f19fdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140b10f0 con 0x7f5f140711a0 2024-04-25T10:08:10.664 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f137fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f14070dc0 0x7f5f1406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f137fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5f14070dc0 0x7f5f1406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55094/0 (socket says 172.21.15.148:55094) 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f137fe640 1 -- 172.21.15.148:0/1595070521 learned_addr learned my addr 172.21.15.148:0/1595070521 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.657+0000 7f5f12ffd640 1 --2- 172.21.15.148:0/1595070521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f140711a0 0x7f5f140b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.658+0000 7f5f137fe640 1 -- 172.21.15.148:0/1595070521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f140711a0 msgr2=0x7f5f140b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.658+0000 7f5f137fe640 1 --2- 172.21.15.148:0/1595070521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f140711a0 0x7f5f140b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.658+0000 7f5f137fe640 1 -- 172.21.15.148:0/1595070521 >> v1:172.21.15.188:6789/0 conn(0x7f5f140b4830 legacy=0x7f5f140b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.658+0000 7f5f137fe640 1 -- 172.21.15.148:0/1595070521 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14162740 con 0x7f5f14070dc0 2024-04-25T10:08:10.665 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.658+0000 7f5f137fe640 1 --2- 172.21.15.148:0/1595070521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f14070dc0 0x7f5f1406ccd0 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009870 tx=0x7f5f0402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85282e2a0d3f2a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.660+0000 7f5f127fc640 1 -- 172.21.15.148:0/1595070521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04037070 con 0x7f5f14070dc0 2024-04-25T10:08:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.660+0000 7f5f127fc640 1 -- 172.21.15.148:0/1595070521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f04032440 con 0x7f5f14070dc0 2024-04-25T10:08:10.666 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.660+0000 7f5f127fc640 1 -- 172.21.15.148:0/1595070521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04032810 con 0x7f5f14070dc0 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 -- 172.21.15.148:0/1595070521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f14070dc0 msgr2=0x7f5f1406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/1595070521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f14070dc0 0x7f5f1406ccd0 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009870 tx=0x7f5f0402a130 comp rx=0 tx=0).stop 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 -- 172.21.15.148:0/1595070521 shutdown_connections 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/1595070521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f140711a0 0x7f5f140b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/1595070521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f14070dc0 0x7f5f1406ccd0 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 -- 172.21.15.148:0/1595070521 >> 172.21.15.148:0/1595070521 conn(0x7f5f140b3530 msgr2=0x7f5f140b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 -- 172.21.15.148:0/1595070521 shutdown_connections 2024-04-25T10:08:10.667 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.661+0000 7f5f19fdb640 1 -- 172.21.15.148:0/1595070521 wait complete. 2024-04-25T10:08:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.662+0000 7f5f19fdb640 1 Processor -- start 2024-04-25T10:08:10.668 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.662+0000 7f5f19fdb640 1 -- start start 2024-04-25T10:08:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f19fdb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f14070dc0 0x7f5f14153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f19fdb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140711a0 0x7f5f14158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f19fdb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 0x7f5f14158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f19fdb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14161f40 con 0x7f5f140b4830 2024-04-25T10:08:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f19fdb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14161c60 con 0x7f5f140711a0 2024-04-25T10:08:10.670 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f19fdb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14071600 con 0x7f5f14070dc0 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 0x7f5f14158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 0x7f5f14158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55104/0 (socket says 172.21.15.148:55104) 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 -- 172.21.15.148:0/881268407 learned_addr learned my addr 172.21.15.148:0/881268407 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f137fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f14070dc0 0x7f5f14153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f12ffd640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140711a0 0x7f5f14158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 -- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f14070dc0 msgr2=0x7f5f14153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f14070dc0 0x7f5f14153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 -- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140711a0 msgr2=0x7f5f14158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140711a0 0x7f5f14158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 -- 172.21.15.148:0/881268407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f141590e0 con 0x7f5f140b4830 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.663+0000 7f5f13fff640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 0x7f5f14158ba0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c6b0 tx=0x7f5f0c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.664+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c003b70 con 0x7f5f140b4830 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.664+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c003d30 con 0x7f5f140b4830 2024-04-25T10:08:10.671 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.664+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f140cac50 con 0x7f5f140b4830 2024-04-25T10:08:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.664+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c005310 con 0x7f5f140b4830 2024-04-25T10:08:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.664+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f140cb210 con 0x7f5f140b4830 2024-04-25T10:08:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.665+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f14073a00 con 0x7f5f140b4830 2024-04-25T10:08:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.665+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f0c0054b0 con 0x7f5f140b4830 2024-04-25T10:08:10.672 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.665+0000 7f5efbfff640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0006b4d0 0x7f5f0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.666+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5f0c08a170 con 0x7f5f140b4830 2024-04-25T10:08:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.666+0000 7f5f137fe640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0006b4d0 0x7f5f0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.673 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.666+0000 7f5f137fe640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0006b4d0 0x7f5f0006d990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5f04034000 tx=0x7f5f0402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.675 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.670+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f0c058750 con 0x7f5f140b4830 2024-04-25T10:08:10.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.842+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f1415da00 con 0x7f5f140b4830 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.842+0000 7f5efbfff640 1 -- 172.21.15.148:0/881268407 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f0c05c7f0 con 0x7f5f140b4830 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 1 2024-04-25T10:08:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0006b4d0 msgr2=0x7f5f0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0006b4d0 0x7f5f0006d990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5f04034000 tx=0x7f5f0402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 msgr2=0x7f5f14158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 0x7f5f14158ba0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c6b0 tx=0x7f5f0c00cb70 comp rx=0 tx=0).stop 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 shutdown_connections 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5f0006b4d0 0x7f5f0006d990 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f140b4830 0x7f5f14158ba0 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140711a0 0x7f5f14158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 --2- 172.21.15.148:0/881268407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f14070dc0 0x7f5f14153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 >> 172.21.15.148:0/881268407 conn(0x7f5f140b3530 msgr2=0x7f5f1415d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 shutdown_connections 2024-04-25T10:08:10.851 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.844+0000 7f5f19fdb640 1 -- 172.21.15.148:0/881268407 wait complete. 2024-04-25T10:08:10.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.954+0000 7f136c67c640 1 Processor -- start 2024-04-25T10:08:10.960 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.954+0000 7f136c67c640 1 -- start start 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f136c67c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13640cadb0 0x7f13640cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f136c67c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13640ccfe0 0x7f13640cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f136c67c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13640afc80 con 0x7f136415de80 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f136c67c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13640cb240 con 0x7f13640cadb0 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f136c67c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13640ce4d0 con 0x7f13640ccfe0 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f1369c17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13640cadb0 0x7f13640cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f1369c17640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f13640cadb0 0x7f13640cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55108/0 (socket says 172.21.15.148:55108) 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.955+0000 7f1369c17640 1 -- 172.21.15.148:0/1949289538 learned_addr learned my addr 172.21.15.148:0/1949289538 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.956+0000 7f136ac19640 1 --2- 172.21.15.148:0/1949289538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13640ccfe0 0x7f13640cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.956+0000 7f1369c17640 1 -- 172.21.15.148:0/1949289538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13640ccfe0 msgr2=0x7f13640cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.956+0000 7f1369c17640 1 --2- 172.21.15.148:0/1949289538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13640ccfe0 0x7f13640cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.962 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.956+0000 7f1369c17640 1 -- 172.21.15.148:0/1949289538 >> v1:172.21.15.188:6789/0 conn(0x7f136415de80 legacy=0x7f13640cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.963 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.956+0000 7f1369c17640 1 -- 172.21.15.148:0/1949289538 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13640ce7b0 con 0x7f13640cadb0 2024-04-25T10:08:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.956+0000 7f1369c17640 1 --2- 172.21.15.148:0/1949289538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640cadb0 0x7f13640cf6c0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f1354009870 tx=0x7f135402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a742395b3c97f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f1369416640 1 -- 172.21.15.148:0/1949289538 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354037070 con 0x7f13640cadb0 2024-04-25T10:08:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f1369416640 1 -- 172.21.15.148:0/1949289538 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1354032440 con 0x7f13640cadb0 2024-04-25T10:08:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f1369416640 1 -- 172.21.15.148:0/1949289538 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354032810 con 0x7f13640cadb0 2024-04-25T10:08:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 -- 172.21.15.148:0/1949289538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640cadb0 msgr2=0x7f13640cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.964 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 --2- 172.21.15.148:0/1949289538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640cadb0 0x7f13640cf6c0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f1354009870 tx=0x7f135402a270 comp rx=0 tx=0).stop 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 -- 172.21.15.148:0/1949289538 shutdown_connections 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 --2- 172.21.15.148:0/1949289538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640cadb0 0x7f13640cf6c0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 --2- 172.21.15.148:0/1949289538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13640ccfe0 0x7f13640cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 -- 172.21.15.148:0/1949289538 >> 172.21.15.148:0/1949289538 conn(0x7f13640b2390 msgr2=0x7f13640b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 -- 172.21.15.148:0/1949289538 shutdown_connections 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.957+0000 7f136c67c640 1 -- 172.21.15.148:0/1949289538 wait complete. 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.958+0000 7f136c67c640 1 Processor -- start 2024-04-25T10:08:10.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.958+0000 7f136c67c640 1 -- start start 2024-04-25T10:08:10.967 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136c67c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13640cadb0 0x7f1364152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136c67c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 0x7f136415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136c67c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f136415de80 0x7f136415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136c67c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136415e260 con 0x7f13640ccfe0 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136c67c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13640cfc00 con 0x7f13640cadb0 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136c67c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13640b2eb0 con 0x7f136415de80 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 0x7f136415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 0x7f136415cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55110/0 (socket says 172.21.15.148:55110) 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 -- 172.21.15.148:0/2314595747 learned_addr learned my addr 172.21.15.148:0/2314595747 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136a418640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13640cadb0 0x7f1364152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f136ac19640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f136415de80 0x7f136415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 -- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f136415de80 msgr2=0x7f136415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f136415de80 0x7f136415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 -- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13640cadb0 msgr2=0x7f1364152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.968 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13640cadb0 0x7f1364152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.959+0000 7f1369c17640 1 -- 172.21.15.148:0/2314595747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13641f38e0 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.960+0000 7f1369c17640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 0x7f136415cbe0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f13540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.960+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354041020 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.960+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13540306c0 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.960+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13641f3ba0 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.960+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354037040 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.960+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13641f4160 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.961+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1364073a00 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.961+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1354044020 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.962+0000 7f135affd640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f134806b4d0 0x7f134806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.962+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f13540aac20 con 0x7f13640ccfe0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.962+0000 7f136a418640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f134806b4d0 0x7f134806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.969 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.962+0000 7f136a418640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f134806b4d0 0x7f134806d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1360007fd0 tx=0x7f1360009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.971 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:10.966+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f135404b030 con 0x7f13640ccfe0 2024-04-25T10:08:11.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.137+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13640d1470 con 0x7f13640ccfe0 2024-04-25T10:08:11.143 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.137+0000 7f135affd640 1 -- 172.21.15.148:0/2314595747 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v200) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1354079830 con 0x7f13640ccfe0 2024-04-25T10:08:11.143 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 32 2024-04-25T10:08:11.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f134806b4d0 msgr2=0x7f134806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.145 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f134806b4d0 0x7f134806d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1360007fd0 tx=0x7f1360009040 comp rx=0 tx=0).stop 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 msgr2=0x7f136415cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 0x7f136415cbe0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f1354000c00 tx=0x7f13540046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 shutdown_connections 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f134806b4d0 0x7f134806d990 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f136415de80 0x7f136415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13640ccfe0 0x7f136415cbe0 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 --2- 172.21.15.148:0/2314595747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13640cadb0 0x7f1364152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 >> 172.21.15.148:0/2314595747 conn(0x7f13640b2390 msgr2=0x7f13640b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 shutdown_connections 2024-04-25T10:08:11.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.139+0000 7f136c67c640 1 -- 172.21.15.148:0/2314595747 wait complete. 2024-04-25T10:08:11.156 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:11.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.250+0000 7fc35b110640 1 Processor -- start 2024-04-25T10:08:11.256 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.250+0000 7fc35b110640 1 -- start start 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc35b110640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3540cd4c0 0x7fc3540cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc35b110640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc35415b1c0 0x7fc35415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc35b110640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc35415e1a0 0x7fc354150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc35b110640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35415b620 con 0x7fc35415b1c0 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc35b110640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35415e6e0 con 0x7fc35415e1a0 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc35b110640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415e9c0 con 0x7fc3540cd4c0 2024-04-25T10:08:11.258 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc353fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc35415b1c0 0x7fc35415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc353fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fc35415b1c0 0x7fc35415dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55114/0 (socket says 172.21.15.148:55114) 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc353fff640 1 -- 172.21.15.148:0/2088126921 learned_addr learned my addr 172.21.15.148:0/2088126921 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc3596ad640 1 --2- 172.21.15.148:0/2088126921 >> v2:172.21.15.188:3300/0 conn(0x7fc35415e1a0 0x7fc354150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.251+0000 7fc358eac640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3540cd4c0 0x7fc3540cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc353fff640 1 -- 172.21.15.148:0/2088126921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3540cd4c0 msgr2=0x7fc3540cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc353fff640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3540cd4c0 0x7fc3540cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc353fff640 1 -- 172.21.15.148:0/2088126921 >> v2:172.21.15.188:3300/0 conn(0x7fc35415e1a0 msgr2=0x7fc354150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc353fff640 1 --2- 172.21.15.148:0/2088126921 >> v2:172.21.15.188:3300/0 conn(0x7fc35415e1a0 0x7fc354150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc353fff640 1 -- 172.21.15.148:0/2088126921 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35415a9c0 con 0x7fc35415b1c0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc358eac640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3540cd4c0 0x7fc3540cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc353fff640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415b1c0 0x7fc35415dc60 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fc3540af3a0 tx=0x7fc34802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2967869df46ec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc3537fe640 1 -- 172.21.15.148:0/2088126921 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348037070 con 0x7fc35415b1c0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc3537fe640 1 -- 172.21.15.148:0/2088126921 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc348032440 con 0x7fc35415b1c0 2024-04-25T10:08:11.259 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.252+0000 7fc3537fe640 1 -- 172.21.15.148:0/2088126921 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348032810 con 0x7fc35415b1c0 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 -- 172.21.15.148:0/2088126921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415b1c0 msgr2=0x7fc35415dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415b1c0 0x7fc35415dc60 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fc3540af3a0 tx=0x7fc34802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 -- 172.21.15.148:0/2088126921 shutdown_connections 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 --2- 172.21.15.148:0/2088126921 >> v2:172.21.15.188:3300/0 conn(0x7fc35415e1a0 0x7fc354150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415b1c0 0x7fc35415dc60 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 --2- 172.21.15.148:0/2088126921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3540cd4c0 0x7fc3540cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 -- 172.21.15.148:0/2088126921 >> 172.21.15.148:0/2088126921 conn(0x7fc35406abe0 msgr2=0x7fc35406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 -- 172.21.15.148:0/2088126921 shutdown_connections 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.253+0000 7fc35b110640 1 -- 172.21.15.148:0/2088126921 wait complete. 2024-04-25T10:08:11.265 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.254+0000 7fc35b110640 1 Processor -- start 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.254+0000 7fc35b110640 1 -- start start 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc35b110640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540cd4c0 0x7fc354141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc35b110640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc35415b1c0 0x7fc3541f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc35b110640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 0x7fc354154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc35b110640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35406eca0 con 0x7fc35415e1a0 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc35b110640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415a670 con 0x7fc3540cd4c0 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc35b110640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35406ee20 con 0x7fc35415b1c0 2024-04-25T10:08:11.266 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 0x7fc354154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 0x7fc354154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55118/0 (socket says 172.21.15.148:55118) 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 -- 172.21.15.148:0/795158383 learned_addr learned my addr 172.21.15.148:0/795158383 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc353fff640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc35415b1c0 0x7fc3541f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc358eac640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540cd4c0 0x7fc354141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 -- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc35415b1c0 msgr2=0x7fc3541f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc35415b1c0 0x7fc3541f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 -- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540cd4c0 msgr2=0x7fc354141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540cd4c0 0x7fc354141150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.255+0000 7fc3596ad640 1 -- 172.21.15.148:0/795158383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3541f20a0 con 0x7fc35415e1a0 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.256+0000 7fc3596ad640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 0x7fc354154ca0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00efc0 tx=0x7fc34c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.256+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c018020 con 0x7fc35415e1a0 2024-04-25T10:08:11.267 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.256+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c014480 con 0x7fc35415e1a0 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.256+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3541f23e0 con 0x7fc35415e1a0 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.256+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c009e60 con 0x7fc35415e1a0 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.256+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc354141690 con 0x7fc35415e1a0 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.257+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3540b2300 con 0x7fc35415e1a0 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.257+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc34c005d10 con 0x7fc35415e1a0 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.258+0000 7fc3517fa640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc34006b4d0 0x7fc34006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.268 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.258+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc34c08b190 con 0x7fc35415e1a0 2024-04-25T10:08:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.258+0000 7fc358eac640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc34006b4d0 0x7fc34006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.258+0000 7fc358eac640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc34006b4d0 0x7fc34006d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc344006fd0 tx=0x7fc344008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.262+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3540b2300 con 0x7fc35415e1a0 2024-04-25T10:08:11.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.434+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc35414e7f0 con 0x7fc35415e1a0 2024-04-25T10:08:11.440 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.434+0000 7fc3517fa640 1 -- 172.21.15.148:0/795158383 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v200) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc35414e7f0 con 0x7fc35415e1a0 2024-04-25T10:08:11.440 INFO:teuthology.orchestra.run.smithi148.stdout:pg_num: 64 2024-04-25T10:08:11.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc34006b4d0 msgr2=0x7fc34006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc34006b4d0 0x7fc34006d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc344006fd0 tx=0x7fc344008040 comp rx=0 tx=0).stop 2024-04-25T10:08:11.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 msgr2=0x7fc354154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 0x7fc354154ca0 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00efc0 tx=0x7fc34c00c490 comp rx=0 tx=0).stop 2024-04-25T10:08:11.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 shutdown_connections 2024-04-25T10:08:11.442 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc34006b4d0 0x7fc34006d990 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc35415e1a0 0x7fc354154ca0 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc35415b1c0 0x7fc3541f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 --2- 172.21.15.148:0/795158383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540cd4c0 0x7fc354141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 >> 172.21.15.148:0/795158383 conn(0x7fc35406abe0 msgr2=0x7fc3540b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 shutdown_connections 2024-04-25T10:08:11.443 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.436+0000 7fc35b110640 1 -- 172.21.15.148:0/795158383 wait complete. 2024-04-25T10:08:11.454 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:11.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.545+0000 7efc5c06f640 1 Processor -- start 2024-04-25T10:08:11.551 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.545+0000 7efc5c06f640 1 -- start start 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc5c06f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efc540cc8e0 0x7efc541625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc5c06f640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc540cce50 con 0x7efc5415dc90 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc5c06f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc54161af0 con 0x7efc540d1810 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc5c06f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc54161dd0 con 0x7efc540cc8e0 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc59e0b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7efc5415dc90 0x7efc5415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52276/0 (socket says 172.21.15.148:52276) 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc59e0b640 1 -- 172.21.15.148:0/2863891966 learned_addr learned my addr 172.21.15.148:0/2863891966 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:11.559 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc5960a640 1 --2- 172.21.15.148:0/2863891966 >> v2:172.21.15.188:3300/0 conn(0x7efc540cc8e0 0x7efc541625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580515661 0 0) 0x7efc540cce50 con 0x7efc5415dc90 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc40003520 con 0x7efc5415dc90 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17263249 0 0) 0x7efc40003520 con 0x7efc5415dc90 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 >> v2:172.21.15.188:3300/0 conn(0x7efc540cc8e0 msgr2=0x7efc541625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 --2- 172.21.15.148:0/2863891966 >> v2:172.21.15.188:3300/0 conn(0x7efc540cc8e0 0x7efc541625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 >> v1:172.21.15.188:6789/0 conn(0x7efc540d1810 legacy=0x7efc540cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc541620b0 con 0x7efc5415dc90 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7efc50001670 con 0x7efc5415dc90 2024-04-25T10:08:11.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 >> v1:172.21.15.148:6789/0 conn(0x7efc5415dc90 legacy=0x7efc5415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc400039b0 0x7efc40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc400043b0 0x7efc40004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc40005120 0x7efc4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54161dd0 con 0x7efc400039b0 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4000c060 con 0x7efc400043b0 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.546+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4000c260 con 0x7efc40005120 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc400039b0 0x7efc40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc5a60c640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc40005120 0x7efc4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc5960a640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc400043b0 0x7efc40004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 -- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc40005120 msgr2=0x7efc4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc40005120 0x7efc4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 -- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc400043b0 msgr2=0x7efc40004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc400043b0 0x7efc40004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 -- 172.21.15.148:0/2863891966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40003520 con 0x7efc400039b0 2024-04-25T10:08:11.562 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc5a60c640 1 -- 172.21.15.148:0/2863891966 reap_dead start 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.547+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc400039b0 0x7efc40003d90 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c240 tx=0x7efc40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a42637590163d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50007d10 con 0x7efc400039b0 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50005ad0 con 0x7efc400039b0 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc58e09640 1 -- 172.21.15.148:0/2863891966 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50007070 con 0x7efc400039b0 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 -- 172.21.15.148:0/2863891966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc400039b0 msgr2=0x7efc40003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc400039b0 0x7efc40003d90 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c240 tx=0x7efc40005830 comp rx=0 tx=0).stop 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 -- 172.21.15.148:0/2863891966 shutdown_connections 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2863891966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc400039b0 0x7efc40003d90 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 -- 172.21.15.148:0/2863891966 >> 172.21.15.148:0/2863891966 conn(0x7efc5406f590 msgr2=0x7efc5406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 -- 172.21.15.148:0/2863891966 shutdown_connections 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.548+0000 7efc5c06f640 1 -- 172.21.15.148:0/2863891966 wait complete. 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.549+0000 7efc5c06f640 1 Processor -- start 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.549+0000 7efc5c06f640 1 -- start start 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5c06f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc400039b0 0x7efc5415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5c06f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc540b71d0 0x7efc540cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5c06f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 0x7efc540ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5c06f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415e1f0 con 0x7efc541f7730 2024-04-25T10:08:11.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5c06f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415e070 con 0x7efc540b71d0 2024-04-25T10:08:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5c06f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc541625d0 con 0x7efc400039b0 2024-04-25T10:08:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5a60c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 0x7efc540ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5a60c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 0x7efc540ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55134/0 (socket says 172.21.15.148:55134) 2024-04-25T10:08:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5a60c640 1 -- 172.21.15.148:0/2555476564 learned_addr learned my addr 172.21.15.148:0/2555476564 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc400039b0 0x7efc5415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.550+0000 7efc5960a640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc540b71d0 0x7efc540cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5a60c640 1 -- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc400039b0 msgr2=0x7efc5415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5a60c640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc400039b0 0x7efc5415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5a60c640 1 -- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc540b71d0 msgr2=0x7efc540cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5a60c640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc540b71d0 0x7efc540cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5a60c640 1 -- 172.21.15.148:0/2555476564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540c8d80 con 0x7efc541f7730 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5960a640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc540b71d0 0x7efc540cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:11.566 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5a60c640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 0x7efc540ce440 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7efc540cdb80 tx=0x7efc4c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c003970 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c018070 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc540c9030 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c003d70 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.551+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc540c95f0 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.552+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc54073990 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.552+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc4c0050d0 con 0x7efc541f7730 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.553+0000 7efc4a7fc640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3806b500 0x7efc3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.553+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7efc4c08a060 con 0x7efc541f7730 2024-04-25T10:08:11.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.553+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3806b500 0x7efc3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.553+0000 7efc59e0b640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3806b500 0x7efc3806d9c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efc50033000 tx=0x7efc50009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.557+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc4c058d20 con 0x7efc541f7730 2024-04-25T10:08:11.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.767+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc540cf6b0 con 0x7efc541f7730 2024-04-25T10:08:11.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.767+0000 7efc4a7fc640 1 -- 172.21.15.148:0/2555476564 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4966 (secure 0 0 0) 0x7efc4c058620 con 0x7efc541f7730 2024-04-25T10:08:11.774 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:11.774 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:05.546576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:08:11.774 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 119 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3806b500 msgr2=0x7efc3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3806b500 0x7efc3806d9c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efc50033000 tx=0x7efc50009030 comp rx=0 tx=0).stop 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 msgr2=0x7efc540ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 0x7efc540ce440 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7efc540cdb80 tx=0x7efc4c00eb90 comp rx=0 tx=0).stop 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 shutdown_connections 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7efc3806b500 0x7efc3806d9c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc541f7730 0x7efc540ce440 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc540b71d0 0x7efc540cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 --2- 172.21.15.148:0/2555476564 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc400039b0 0x7efc5415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.769+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 >> 172.21.15.148:0/2555476564 conn(0x7efc5406f590 msgr2=0x7efc540cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.770+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 shutdown_connections 2024-04-25T10:08:11.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.770+0000 7efc5c06f640 1 -- 172.21.15.148:0/2555476564 wait complete. 2024-04-25T10:08:11.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.878+0000 7fb14a7f4640 1 Processor -- start 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.878+0000 7fb14a7f4640 1 -- start start 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb14a7f4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 0x7fb1440cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb14a7f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb144071940 0x7fb1440cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb14a7f4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb144071ed0 con 0x7fb14415b320 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb14a7f4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14413d570 con 0x7fb14413d190 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb14a7f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14415a840 con 0x7fb144071940 2024-04-25T10:08:11.887 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb143fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 0x7fb1440cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb148d91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb144071940 0x7fb1440cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb1437fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb14415b320 0x7fb144071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:36846/0 (socket says 172.21.15.148:36846) 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb143fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 0x7fb1440cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:42202/0 (socket says 172.21.15.148:42202) 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb143fff640 1 -- 172.21.15.148:0/3252520083 learned_addr learned my addr 172.21.15.148:0/3252520083 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb142ffd640 1 -- 172.21.15.148:0/3252520083 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793342919 0 0) 0x7fb144071ed0 con 0x7fb14415b320 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb142ffd640 1 -- 172.21.15.148:0/3252520083 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb12c003520 con 0x7fb14415b320 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb148d91640 1 -- 172.21.15.148:0/3252520083 >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 msgr2=0x7fb1440cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb148d91640 1 --2- 172.21.15.148:0/3252520083 >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 0x7fb1440cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb148d91640 1 -- 172.21.15.148:0/3252520083 >> v1:172.21.15.188:6789/0 conn(0x7fb14415b320 legacy=0x7fb144071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb148d91640 1 -- 172.21.15.148:0/3252520083 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14415ab20 con 0x7fb144071940 2024-04-25T10:08:11.888 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.879+0000 7fb143fff640 1 --2- 172.21.15.148:0/3252520083 >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 0x7fb1440cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.880+0000 7fb148d91640 1 --2- 172.21.15.148:0/3252520083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb144071940 0x7fb1440cc4f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00bfa0 tx=0x7fb13c02c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76972e79a4b0a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.880+0000 7fb142ffd640 1 -- 172.21.15.148:0/3252520083 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c038070 con 0x7fb144071940 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.880+0000 7fb142ffd640 1 -- 172.21.15.148:0/3252520083 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c009540 con 0x7fb144071940 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.880+0000 7fb142ffd640 1 -- 172.21.15.148:0/3252520083 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c0098f0 con 0x7fb144071940 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 -- 172.21.15.148:0/3252520083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb144071940 msgr2=0x7fb1440cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/3252520083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb144071940 0x7fb1440cc4f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00bfa0 tx=0x7fb13c02c7e0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 -- 172.21.15.148:0/3252520083 shutdown_connections 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/3252520083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb144071940 0x7fb1440cc4f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/3252520083 >> v2:172.21.15.188:3300/0 conn(0x7fb14413d190 0x7fb1440cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 -- 172.21.15.148:0/3252520083 >> 172.21.15.148:0/3252520083 conn(0x7fb14406abe0 msgr2=0x7fb14406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 -- 172.21.15.148:0/3252520083 shutdown_connections 2024-04-25T10:08:11.889 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.881+0000 7fb14a7f4640 1 -- 172.21.15.148:0/3252520083 wait complete. 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.882+0000 7fb14a7f4640 1 Processor -- start 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.882+0000 7fb14a7f4640 1 -- start start 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb14a7f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb144071940 0x7fb1441e8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb14a7f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 0x7fb144157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb14a7f4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb14415b320 0x7fb1440b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb14a7f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440cf7a0 con 0x7fb14413d190 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb14a7f4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440cf620 con 0x7fb144071940 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb14a7f4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440cf920 con 0x7fb14415b320 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 0x7fb144157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 0x7fb144157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55140/0 (socket says 172.21.15.148:55140) 2024-04-25T10:08:11.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 -- 172.21.15.148:0/367962869 learned_addr learned my addr 172.21.15.148:0/367962869 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb143fff640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb144071940 0x7fb1441e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb148d91640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb14415b320 0x7fb1440b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 -- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb14415b320 msgr2=0x7fb1440b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb14415b320 0x7fb1440b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 -- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb144071940 msgr2=0x7fb1441e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.883+0000 7fb1437fe640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb144071940 0x7fb1441e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb1437fe640 1 -- 172.21.15.148:0/367962869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144156350 con 0x7fb14413d190 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb1437fe640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 0x7fb144157690 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c3c0 tx=0x7fb13800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1380074e0 con 0x7fb14413d190 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1380047e0 con 0x7fb14413d190 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138004bd0 con 0x7fb14413d190 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1441e8970 con 0x7fb14413d190 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.884+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1441568d0 con 0x7fb14413d190 2024-04-25T10:08:11.896 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.885+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144073990 con 0x7fb14413d190 2024-04-25T10:08:11.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.885+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb13801a020 con 0x7fb14413d190 2024-04-25T10:08:11.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.886+0000 7fb140ff9640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb13006b4d0 0x7fb13006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.886+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb138089af0 con 0x7fb14413d190 2024-04-25T10:08:11.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.886+0000 7fb143fff640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb13006b4d0 0x7fb13006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.886+0000 7fb143fff640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb13006b4d0 0x7fb13006d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb134006ca0 tx=0x7fb134009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.897 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:11.890+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb138058190 con 0x7fb14413d190 2024-04-25T10:08:12.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.059+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1440cfeb0 con 0x7fb14413d190 2024-04-25T10:08:12.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.060+0000 7fb140ff9640 1 -- 172.21.15.148:0/367962869 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17260 (secure 0 0 0) 0x7fb13805c230 con 0x7fb14413d190 2024-04-25T10:08:12.066 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:12.066 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":200,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:07:07.548514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.14","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/1255014956":"2024-04-26T10:07:02.533393+0000","172.21.15.188:6833/1255014956":"2024-04-26T10:07:02.533393+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:12.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb13006b4d0 msgr2=0x7fb13006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb13006b4d0 0x7fb13006d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb134006ca0 tx=0x7fb134009240 comp rx=0 tx=0).stop 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 msgr2=0x7fb144157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 0x7fb144157690 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fb13800c3c0 tx=0x7fb13800c880 comp rx=0 tx=0).stop 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 shutdown_connections 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb13006b4d0 0x7fb13006d990 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb14415b320 0x7fb1440b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb14413d190 0x7fb144157690 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 --2- 172.21.15.148:0/367962869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb144071940 0x7fb1441e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 >> 172.21.15.148:0/367962869 conn(0x7fb14406abe0 msgr2=0x7fb144155000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.062+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 shutdown_connections 2024-04-25T10:08:12.069 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.063+0000 7fb14a7f4640 1 -- 172.21.15.148:0/367962869 wait complete. 2024-04-25T10:08:12.080 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:12.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.171+0000 7fc9655c8640 1 Processor -- start 2024-04-25T10:08:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.171+0000 7fc9655c8640 1 -- start start 2024-04-25T10:08:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc9655c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc960144750 0x7fc96006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc9655c8640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc960144b30 con 0x7fc96006eb10 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc9655c8640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9600712d0 con 0x7fc9600cf030 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc9655c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600715b0 con 0x7fc960144750 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95e7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fc96006eb10 0x7fc9600ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52284/0 (socket says 172.21.15.148:52284) 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95e7fc640 1 -- 172.21.15.148:0/2228508130 learned_addr learned my addr 172.21.15.148:0/2228508130 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95effd640 1 --2- 172.21.15.148:0/2228508130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc960144750 0x7fc96006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555653594 0 0) 0x7fc960144b30 con 0x7fc96006eb10 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc948003520 con 0x7fc96006eb10 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606628126 0 0) 0x7fc9600712d0 con 0x7fc9600cf030 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc960144b30 con 0x7fc9600cf030 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95effd640 1 -- 172.21.15.148:0/2228508130 >> v1:172.21.15.188:6789/0 conn(0x7fc9600cf030 legacy=0x7fc96006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95effd640 1 -- 172.21.15.148:0/2228508130 >> v1:172.21.15.148:6789/0 conn(0x7fc96006eb10 legacy=0x7fc9600ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.172+0000 7fc95effd640 1 -- 172.21.15.148:0/2228508130 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960071890 con 0x7fc960144750 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.173+0000 7fc95effd640 1 --2- 172.21.15.148:0/2228508130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc960144750 0x7fc96006e5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc950009dc0 tx=0x7fc95002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf853dc18b20f1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.173+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95002ae60 con 0x7fc960144750 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.173+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950007740 con 0x7fc960144750 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.173+0000 7fc95dffb640 1 -- 172.21.15.148:0/2228508130 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950007b10 con 0x7fc960144750 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 -- 172.21.15.148:0/2228508130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc960144750 msgr2=0x7fc96006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 --2- 172.21.15.148:0/2228508130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc960144750 0x7fc96006e5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc950009dc0 tx=0x7fc95002a640 comp rx=0 tx=0).stop 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 -- 172.21.15.148:0/2228508130 shutdown_connections 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 --2- 172.21.15.148:0/2228508130 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc960144750 0x7fc96006e5d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 -- 172.21.15.148:0/2228508130 >> 172.21.15.148:0/2228508130 conn(0x7fc96006abe0 msgr2=0x7fc96006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 -- 172.21.15.148:0/2228508130 shutdown_connections 2024-04-25T10:08:12.182 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.174+0000 7fc9655c8640 1 -- 172.21.15.148:0/2228508130 wait complete. 2024-04-25T10:08:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.175+0000 7fc9655c8640 1 Processor -- start 2024-04-25T10:08:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.175+0000 7fc9655c8640 1 -- start start 2024-04-25T10:08:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc9655c8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc96006eb10 0x7fc9601e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc9655c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9600cf030 0x7fc9600abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc9655c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 0x7fc9600ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc9655c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9601455d0 con 0x7fc960144750 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc9655c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc960145450 con 0x7fc9600cf030 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc9655c8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96015c000 con 0x7fc96006eb10 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 0x7fc9600ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 0x7fc9600ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55156/0 (socket says 172.21.15.148:55156) 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 -- 172.21.15.148:0/582488732 learned_addr learned my addr 172.21.15.148:0/582488732 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95effd640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc96006eb10 0x7fc9601e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95e7fc640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9600cf030 0x7fc9600abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 -- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc96006eb10 msgr2=0x7fc9601e34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc96006eb10 0x7fc9601e34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 -- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9600cf030 msgr2=0x7fc9600abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9600cf030 0x7fc9600abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.176+0000 7fc95f7fe640 1 -- 172.21.15.148:0/582488732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600aadf0 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.177+0000 7fc95f7fe640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 0x7fc9600ada50 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fc958001eb0 tx=0x7fc95800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.177+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958017070 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.177+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9580052d0 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.177+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9580037f0 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.177+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600ab0d0 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.177+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9600ab690 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.178+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc960073990 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.178+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc95801a070 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.179+0000 7fc9477fe640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94c06b4d0 0x7fc94c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.179+0000 7fc95effd640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94c06b4d0 0x7fc94c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.179+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc958089650 con 0x7fc960144750 2024-04-25T10:08:12.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.179+0000 7fc95effd640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94c06b4d0 0x7fc94c06d990 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc950034000 tx=0x7fc95002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.188 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.183+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc958057cc0 con 0x7fc960144750 2024-04-25T10:08:12.397 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.392+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9600cd9e0 con 0x7fc960144750 2024-04-25T10:08:12.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.392+0000 7fc9477fe640 1 -- 172.21.15.148:0/582488732 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4966 (secure 0 0 0) 0x7fc95805bd60 con 0x7fc960144750 2024-04-25T10:08:12.398 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:12.398 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:05.546576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:08:12.398 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 119 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94c06b4d0 msgr2=0x7fc94c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94c06b4d0 0x7fc94c06d990 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc950034000 tx=0x7fc95002f000 comp rx=0 tx=0).stop 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 msgr2=0x7fc9600ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 0x7fc9600ada50 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fc958001eb0 tx=0x7fc95800c820 comp rx=0 tx=0).stop 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 shutdown_connections 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fc94c06b4d0 0x7fc94c06d990 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.400 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc960144750 0x7fc9600ada50 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9600cf030 0x7fc9600abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 --2- 172.21.15.148:0/582488732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc96006eb10 0x7fc9601e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 >> 172.21.15.148:0/582488732 conn(0x7fc96006abe0 msgr2=0x7fc960140ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 shutdown_connections 2024-04-25T10:08:12.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.394+0000 7fc9655c8640 1 -- 172.21.15.148:0/582488732 wait complete. 2024-04-25T10:08:12.411 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:12.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.503+0000 7f08a664a640 1 Processor -- start 2024-04-25T10:08:12.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.503+0000 7f08a664a640 1 -- start start 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.503+0000 7f08a664a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f08a015ba00 0x7f08a009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.503+0000 7f08a664a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f08a01602e0 0x7f08a00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f08a664a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a00cd710 con 0x7f08a015f5f0 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f08a664a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a0099b70 con 0x7f08a01602e0 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f08a664a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a0099e50 con 0x7f08a015ba00 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f08a01602e0 0x7f08a00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f08a01602e0 0x7f08a00ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55160/0 (socket says 172.21.15.148:55160) 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 -- 172.21.15.148:0/2725391385 learned_addr learned my addr 172.21.15.148:0/2725391385 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089ffff640 1 --2- 172.21.15.148:0/2725391385 >> v2:172.21.15.188:3300/0 conn(0x7f08a015ba00 0x7f08a009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 -- 172.21.15.148:0/2725391385 >> v2:172.21.15.188:3300/0 conn(0x7f08a015ba00 msgr2=0x7f08a009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:12.510 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 --2- 172.21.15.148:0/2725391385 >> v2:172.21.15.188:3300/0 conn(0x7f08a015ba00 0x7f08a009a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 -- 172.21.15.148:0/2725391385 >> v1:172.21.15.188:6789/0 conn(0x7f08a015f5f0 legacy=0x7f08a00c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:12.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089f7fe640 1 -- 172.21.15.148:0/2725391385 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a009a130 con 0x7f08a01602e0 2024-04-25T10:08:12.511 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.504+0000 7f089ffff640 1 --2- 172.21.15.148:0/2725391385 >> v2:172.21.15.188:3300/0 conn(0x7f08a015ba00 0x7f08a009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:12.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f089f7fe640 1 --2- 172.21.15.148:0/2725391385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a01602e0 0x7f08a00ccda0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b4370 tx=0x7f089002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a012186f87c11f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.512 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f089effd640 1 -- 172.21.15.148:0/2725391385 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890037070 con 0x7f08a01602e0 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f089effd640 1 -- 172.21.15.148:0/2725391385 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089002ae20 con 0x7f08a01602e0 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f089effd640 1 -- 172.21.15.148:0/2725391385 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890032660 con 0x7f08a01602e0 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 -- 172.21.15.148:0/2725391385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a01602e0 msgr2=0x7f08a00ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 --2- 172.21.15.148:0/2725391385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a01602e0 0x7f08a00ccda0 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b4370 tx=0x7f089002a110 comp rx=0 tx=0).stop 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 -- 172.21.15.148:0/2725391385 shutdown_connections 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 --2- 172.21.15.148:0/2725391385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a01602e0 0x7f08a00ccda0 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 --2- 172.21.15.148:0/2725391385 >> v2:172.21.15.188:3300/0 conn(0x7f08a015ba00 0x7f08a009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 -- 172.21.15.148:0/2725391385 >> 172.21.15.148:0/2725391385 conn(0x7f08a00b3270 msgr2=0x7f08a00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 -- 172.21.15.148:0/2725391385 shutdown_connections 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.505+0000 7f08a664a640 1 -- 172.21.15.148:0/2725391385 wait complete. 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 Processor -- start 2024-04-25T10:08:12.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 -- start start 2024-04-25T10:08:12.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 0x7f08a0152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a015f5f0 0x7f08a015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a01602e0 0x7f08a01f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a015bde0 con 0x7f08a015ba00 2024-04-25T10:08:12.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a00cd2e0 con 0x7f08a01602e0 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f08a664a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a015eee0 con 0x7f08a015f5f0 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f089ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 0x7f08a0152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f089ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 0x7f08a0152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55176/0 (socket says 172.21.15.148:55176) 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.507+0000 7f089ffff640 1 -- 172.21.15.148:0/2178492521 learned_addr learned my addr 172.21.15.148:0/2178492521 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089f7fe640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a015f5f0 0x7f08a015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f08a4be7640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a01602e0 0x7f08a01f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089ffff640 1 -- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a015f5f0 msgr2=0x7f08a015e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089ffff640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a015f5f0 0x7f08a015e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089ffff640 1 -- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a01602e0 msgr2=0x7f08a01f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089ffff640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a01602e0 0x7f08a01f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089ffff640 1 -- 172.21.15.148:0/2178492521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a01f5b70 con 0x7f08a015ba00 2024-04-25T10:08:12.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089f7fe640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a015f5f0 0x7f08a015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f08a4be7640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a01602e0 0x7f08a01f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089ffff640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 0x7f08a0152900 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f089400c7f0 tx=0x7f089400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894015070 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0894007820 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a01f5e50 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894007bd0 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.508+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a015e1a0 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.509+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a0073990 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.510+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0894018070 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.510+0000 7f089cff9640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f088c06b4d0 0x7f088c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.510+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f089408a310 con 0x7f08a015ba00 2024-04-25T10:08:12.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.510+0000 7f089f7fe640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f088c06b4d0 0x7f088c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.510+0000 7f089f7fe640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f088c06b4d0 0x7f088c06d990 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0890034000 tx=0x7f089002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.514+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0894058f40 con 0x7f08a015ba00 2024-04-25T10:08:12.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.722+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a00b40a0 con 0x7f08a015ba00 2024-04-25T10:08:12.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.723+0000 7f089cff9640 1 -- 172.21.15.148:0/2178492521 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4966 (secure 0 0 0) 0x7f0894058840 con 0x7f08a015ba00 2024-04-25T10:08:12.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:12.729 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":119,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:52.824899+0000","modified":"2024-04-25T10:07:05.546576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11619":{"gid":11619,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/1465938320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":1465938320},{"type":"v1","addr":"172.21.15.148:6837","nonce":1465938320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:08:12.729 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 119 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f088c06b4d0 msgr2=0x7f088c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f088c06b4d0 0x7f088c06d990 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0890034000 tx=0x7f089002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 msgr2=0x7f08a0152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 0x7f08a0152900 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f089400c7f0 tx=0x7f089400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 shutdown_connections 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f088c06b4d0 0x7f088c06d990 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a01602e0 0x7f08a01f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08a015f5f0 0x7f08a015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 --2- 172.21.15.148:0/2178492521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08a015ba00 0x7f08a0152900 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 >> 172.21.15.148:0/2178492521 conn(0x7f08a00b3270 msgr2=0x7f08a00cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 shutdown_connections 2024-04-25T10:08:12.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.725+0000 7f08a664a640 1 -- 172.21.15.148:0/2178492521 wait complete. 2024-04-25T10:08:12.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:12.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.833+0000 7f64c7cb3640 1 Processor -- start 2024-04-25T10:08:12.839 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.833+0000 7f64c7cb3640 1 -- start start 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c7cb3640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64c015dd30 0x7f64c00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c7cb3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f64c00b9770 0x7f64c00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c7cb3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c00d1df0 con 0x7f64c00d2e90 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c7cb3640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c00b7a50 con 0x7f64c015dd30 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c7cb3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c00d20d0 con 0x7f64c00b9770 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c5a4f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64c015dd30 0x7f64c00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c6250640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f64c00d2e90 0x7f64c00b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:52300/0 (socket says 172.21.15.148:52300) 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c5a4f640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f64c015dd30 0x7f64c00b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55188/0 (socket says 172.21.15.148:55188) 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c6250640 1 -- 172.21.15.148:0/1280245836 learned_addr learned my addr 172.21.15.148:0/1280245836 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c524e640 1 --2- 172.21.15.148:0/1280245836 >> v2:172.21.15.188:3300/0 conn(0x7f64c00b9770 0x7f64c00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c5a4f640 1 -- 172.21.15.148:0/1280245836 >> v2:172.21.15.188:3300/0 conn(0x7f64c00b9770 msgr2=0x7f64c00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:12.841 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c5a4f640 1 --2- 172.21.15.148:0/1280245836 >> v2:172.21.15.188:3300/0 conn(0x7f64c00b9770 0x7f64c00d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c5a4f640 1 -- 172.21.15.148:0/1280245836 >> v1:172.21.15.148:6789/0 conn(0x7f64c00d2e90 legacy=0x7f64c00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c5a4f640 1 -- 172.21.15.148:0/1280245836 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00d23b0 con 0x7f64c015dd30 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c4a4d640 1 -- 172.21.15.148:0/1280245836 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642863946 0 0) 0x7f64c00d1df0 con 0x7f64c00d2e90 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.834+0000 7f64c524e640 1 --2- 172.21.15.148:0/1280245836 >> v2:172.21.15.188:3300/0 conn(0x7f64c00b9770 0x7f64c00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.835+0000 7f64c5a4f640 1 --2- 172.21.15.148:0/1280245836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00b9230 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f64c00b6540 tx=0x7f64bc02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4109aab43f26bd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.835+0000 7f64c4a4d640 1 -- 172.21.15.148:0/1280245836 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc037070 con 0x7f64c015dd30 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.835+0000 7f64c4a4d640 1 -- 172.21.15.148:0/1280245836 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc02add0 con 0x7f64c015dd30 2024-04-25T10:08:12.842 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.835+0000 7f64c4a4d640 1 -- 172.21.15.148:0/1280245836 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc032660 con 0x7f64c015dd30 2024-04-25T10:08:12.844 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1280245836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 msgr2=0x7f64c00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1280245836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00b9230 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7f64c00b6540 tx=0x7f64bc02a080 comp rx=0 tx=0).stop 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1280245836 shutdown_connections 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1280245836 >> v2:172.21.15.188:3300/0 conn(0x7f64c00b9770 0x7f64c00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1280245836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00b9230 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1280245836 >> 172.21.15.148:0/1280245836 conn(0x7f64c0061c30 msgr2=0x7f64c006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1280245836 shutdown_connections 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.836+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1280245836 wait complete. 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.837+0000 7f64c7cb3640 1 Processor -- start 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.837+0000 7f64c7cb3640 1 -- start start 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c00b9770 0x7f64c00c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c00d2e90 0x7f64c00ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.845 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00d1a10 con 0x7f64c015dd30 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00d2970 con 0x7f64c00d2e90 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c00d1b90 con 0x7f64c00b9770 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55204/0 (socket says 172.21.15.148:55204) 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 -- 172.21.15.148:0/1383031755 learned_addr learned my addr 172.21.15.148:0/1383031755 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c5a4f640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c00b9770 0x7f64c00c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c524e640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c00d2e90 0x7f64c00ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 -- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c00b9770 msgr2=0x7f64c00c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c00b9770 0x7f64c00c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 -- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c00d2e90 msgr2=0x7f64c00ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c00d2e90 0x7f64c00ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 -- 172.21.15.148:0/1383031755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c01f75d0 con 0x7f64c015dd30 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c6250640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00cd280 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c7a0 tx=0x7f64b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.846 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8018070 con 0x7f64c015dd30 2024-04-25T10:08:12.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b800a640 con 0x7f64c015dd30 2024-04-25T10:08:12.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c01f7870 con 0x7f64c015dd30 2024-04-25T10:08:12.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b800aab0 con 0x7f64c015dd30 2024-04-25T10:08:12.847 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.838+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c01f7e30 con 0x7f64c015dd30 2024-04-25T10:08:12.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.839+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c0073990 con 0x7f64c015dd30 2024-04-25T10:08:12.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.840+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64b801b070 con 0x7f64c015dd30 2024-04-25T10:08:12.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.840+0000 7f64b67fc640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64a406b4d0 0x7f64a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:12.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.840+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f64b80899e0 con 0x7f64c015dd30 2024-04-25T10:08:12.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.840+0000 7f64c5a4f640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64a406b4d0 0x7f64a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:12.848 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.841+0000 7f64c5a4f640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64a406b4d0 0x7f64a406d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f64bc034000 tx=0x7f64bc02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:12.850 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:12.844+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64b80586a0 con 0x7f64c015dd30 2024-04-25T10:08:13.020 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.014+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64c00b8750 con 0x7f64c015dd30 2024-04-25T10:08:13.021 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.016+0000 7f64b67fc640 1 -- 172.21.15.148:0/1383031755 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17260 (secure 0 0 0) 0x7f64b8057fa0 con 0x7f64c015dd30 2024-04-25T10:08:13.022 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:13.022 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":200,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:07:07.548514+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:51.040797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:51.800930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.7","mappings":[{"from":6,"to":0}]},{"pgid":"29.14","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/1255014956":"2024-04-26T10:07:02.533393+0000","172.21.15.188:6833/1255014956":"2024-04-26T10:07:02.533393+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:13.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64a406b4d0 msgr2=0x7f64a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64a406b4d0 0x7f64a406d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f64bc034000 tx=0x7f64bc02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:13.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 msgr2=0x7f64c00cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.024 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00cd280 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c7a0 tx=0x7f64b800cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 shutdown_connections 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f64a406b4d0 0x7f64a406d990 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64c015dd30 0x7f64c00cd280 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64c00d2e90 0x7f64c00ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 --2- 172.21.15.148:0/1383031755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64c00b9770 0x7f64c00c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 >> 172.21.15.148:0/1383031755 conn(0x7f64c0061c30 msgr2=0x7f64c00d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 shutdown_connections 2024-04-25T10:08:13.025 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.018+0000 7f64c7cb3640 1 -- 172.21.15.148:0/1383031755 wait complete. 2024-04-25T10:08:13.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:08:13.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.127+0000 7f5d3dc6b640 1 Processor -- start 2024-04-25T10:08:13.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.127+0000 7f5d3dc6b640 1 -- start start 2024-04-25T10:08:13.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d3dc6b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d3815b1c0 0x7f5d380cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d3dc6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d380cd950 0x7f5d38150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d3dc6b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3815b5a0 con 0x7f5d380cd4c0 2024-04-25T10:08:13.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d3dc6b640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3815e6e0 con 0x7f5d3815b1c0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d3dc6b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3815e9c0 con 0x7f5d380cd950 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d377fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d3815b1c0 0x7f5d380cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d377fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f5d3815b1c0 0x7f5d380cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55216/0 (socket says 172.21.15.148:55216) 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.128+0000 7f5d377fe640 1 -- 172.21.15.148:0/3770154965 learned_addr learned my addr 172.21.15.148:0/3770154965 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d37fff640 1 --2- 172.21.15.148:0/3770154965 >> v2:172.21.15.188:3300/0 conn(0x7f5d380cd950 0x7f5d38150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d377fe640 1 -- 172.21.15.148:0/3770154965 >> v2:172.21.15.188:3300/0 conn(0x7f5d380cd950 msgr2=0x7f5d38150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d377fe640 1 --2- 172.21.15.148:0/3770154965 >> v2:172.21.15.188:3300/0 conn(0x7f5d380cd950 0x7f5d38150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d377fe640 1 -- 172.21.15.148:0/3770154965 >> v1:172.21.15.188:6789/0 conn(0x7f5d380cd4c0 legacy=0x7f5d3815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d377fe640 1 -- 172.21.15.148:0/3770154965 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3815a9c0 con 0x7f5d3815b1c0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d37fff640 1 --2- 172.21.15.148:0/3770154965 >> v2:172.21.15.188:3300/0 conn(0x7f5d380cd950 0x7f5d38150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.129+0000 7f5d377fe640 1 --2- 172.21.15.148:0/3770154965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3815b1c0 0x7f5d380cc180 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f5d380d03c0 tx=0x7f5d2802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa304e64ea1713f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d367fc640 1 -- 172.21.15.148:0/3770154965 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28037070 con 0x7f5d3815b1c0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d367fc640 1 -- 172.21.15.148:0/3770154965 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28032440 con 0x7f5d3815b1c0 2024-04-25T10:08:13.136 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d367fc640 1 -- 172.21.15.148:0/3770154965 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28032810 con 0x7f5d3815b1c0 2024-04-25T10:08:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3770154965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3815b1c0 msgr2=0x7f5d380cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3770154965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3815b1c0 0x7f5d380cc180 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f5d380d03c0 tx=0x7f5d2802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3770154965 shutdown_connections 2024-04-25T10:08:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3770154965 >> v2:172.21.15.188:3300/0 conn(0x7f5d380cd950 0x7f5d38150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3770154965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3815b1c0 0x7f5d380cc180 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3770154965 >> 172.21.15.148:0/3770154965 conn(0x7f5d3806abe0 msgr2=0x7f5d3806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3770154965 shutdown_connections 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.130+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3770154965 wait complete. 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.131+0000 7f5d3dc6b640 1 Processor -- start 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 -- start start 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 0x7f5d38153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d380cd950 0x7f5d3814c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d3815b1c0 0x7f5d3814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3815e370 con 0x7f5d380cd4c0 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3815a670 con 0x7f5d380cd950 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d3dc6b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3806eca0 con 0x7f5d3815b1c0 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 0x7f5d38153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 0x7f5d38153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55224/0 (socket says 172.21.15.148:55224) 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.132+0000 7f5d377fe640 1 -- 172.21.15.148:0/3789861899 learned_addr learned my addr 172.21.15.148:0/3789861899 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:13.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d37fff640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d3815b1c0 0x7f5d3814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d36ffd640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d380cd950 0x7f5d3814c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d377fe640 1 -- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d3815b1c0 msgr2=0x7f5d3814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d377fe640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d3815b1c0 0x7f5d3814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d377fe640 1 -- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d380cd950 msgr2=0x7f5d3814c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d377fe640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d380cd950 0x7f5d3814c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d377fe640 1 -- 172.21.15.148:0/3789861899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d381514a0 con 0x7f5d380cd4c0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d377fe640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 0x7f5d38153b10 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000c00 tx=0x7f5d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2803f070 con 0x7f5d380cd4c0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28004250 con 0x7f5d380cd4c0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28003610 con 0x7f5d380cd4c0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d38151780 con 0x7f5d380cd4c0 2024-04-25T10:08:13.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.133+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d38151d40 con 0x7f5d380cd4c0 2024-04-25T10:08:13.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.134+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d380aed30 con 0x7f5d380cd4c0 2024-04-25T10:08:13.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.134+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d280037b0 con 0x7f5d380cd4c0 2024-04-25T10:08:13.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.135+0000 7f5d1ffff640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d2406b500 0x7f5d2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.135+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5d280ac690 con 0x7f5d380cd4c0 2024-04-25T10:08:13.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.135+0000 7f5d36ffd640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d2406b500 0x7f5d2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.142 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.135+0000 7f5d36ffd640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d2406b500 0x7f5d2406d9c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5d380cca00 tx=0x7f5d2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.144 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.139+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d380aed30 con 0x7f5d380cd4c0 2024-04-25T10:08:13.355 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.349+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5d380ccb10 con 0x7f5d380cd4c0 2024-04-25T10:08:13.355 INFO:tasks.ceph.mon.a.smithi148.stderr:2024-04-25T10:08:13.349+0000 7fb158f72640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:08:13.356 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.351+0000 7f5d1ffff640 1 -- 172.21.15.148:0/3789861899 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v120) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5d2807b350 con 0x7f5d380cd4c0 2024-04-25T10:08:13.357 INFO:teuthology.orchestra.run.smithi148.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:08:13.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d2406b500 msgr2=0x7f5d2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d2406b500 0x7f5d2406d9c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5d380cca00 tx=0x7f5d2c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:13.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 msgr2=0x7f5d38153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 0x7f5d38153b10 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000c00 tx=0x7f5d280046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 shutdown_connections 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f5d2406b500 0x7f5d2406d9c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d3815b1c0 0x7f5d3814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d380cd950 0x7f5d3814c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 --2- 172.21.15.148:0/3789861899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d380cd4c0 0x7f5d38153b10 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.354+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 >> 172.21.15.148:0/3789861899 conn(0x7f5d3806abe0 msgr2=0x7f5d3815ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.355+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 shutdown_connections 2024-04-25T10:08:13.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.355+0000 7f5d3dc6b640 1 -- 172.21.15.148:0/3789861899 wait complete. 2024-04-25T10:08:13.363 INFO:tasks.ceph.mds.c.smithi148.stderr: -2375> 2024-04-25T10:07:08.771+0000 7f76d0482640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:13.372 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:08:13.389 INFO:tasks.ceph.mds.c.smithi148.stdout:starting mds.c at 2024-04-25T10:08:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.465+0000 7f62e4f5c640 1 Processor -- start 2024-04-25T10:08:13.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.465+0000 7f62e4f5c640 1 -- start start 2024-04-25T10:08:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62e4f5c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f62e0155f80 0x7f62e006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62e4f5c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62e00ce900 0x7f62e00b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62e4f5c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00b4d80 con 0x7f62e006fa00 2024-04-25T10:08:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62e4f5c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00ced30 con 0x7f62e00ce900 2024-04-25T10:08:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62e4f5c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00b5390 con 0x7f62e0155f80 2024-04-25T10:08:13.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62e00ce900 0x7f62e00b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f62e00ce900 0x7f62e00b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55264/0 (socket says 172.21.15.148:55264) 2024-04-25T10:08:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 -- 172.21.15.148:0/3999111163 learned_addr learned my addr 172.21.15.148:0/3999111163 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 -- 172.21.15.148:0/3999111163 >> v2:172.21.15.188:3300/0 conn(0x7f62e0155f80 msgr2=0x7f62e006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 --2- 172.21.15.148:0/3999111163 >> v2:172.21.15.188:3300/0 conn(0x7f62e0155f80 0x7f62e006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 -- 172.21.15.148:0/3999111163 >> v1:172.21.15.188:6789/0 conn(0x7f62e006fa00 legacy=0x7f62e00ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.466+0000 7f62ded9d640 1 -- 172.21.15.148:0/3999111163 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00cef70 con 0x7f62e00ce900 2024-04-25T10:08:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.467+0000 7f62ded9d640 1 --2- 172.21.15.148:0/3999111163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e00ce900 0x7f62e00b5820 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f62e00d2860 tx=0x7f62d802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6101166218c64a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.467+0000 7f62ddd9b640 1 -- 172.21.15.148:0/3999111163 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8036070 con 0x7f62e00ce900 2024-04-25T10:08:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.467+0000 7f62ddd9b640 1 -- 172.21.15.148:0/3999111163 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8031600 con 0x7f62e00ce900 2024-04-25T10:08:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.467+0000 7f62ddd9b640 1 -- 172.21.15.148:0/3999111163 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80319d0 con 0x7f62e00ce900 2024-04-25T10:08:13.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 -- 172.21.15.148:0/3999111163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e00ce900 msgr2=0x7f62e00b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/3999111163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e00ce900 0x7f62e00b5820 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f62e00d2860 tx=0x7f62d802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 -- 172.21.15.148:0/3999111163 shutdown_connections 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/3999111163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e00ce900 0x7f62e00b5820 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/3999111163 >> v2:172.21.15.188:3300/0 conn(0x7f62e0155f80 0x7f62e006f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 -- 172.21.15.148:0/3999111163 >> 172.21.15.148:0/3999111163 conn(0x7f62e0061600 msgr2=0x7f62e00619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 -- 172.21.15.148:0/3999111163 shutdown_connections 2024-04-25T10:08:13.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.468+0000 7f62e4f5c640 1 -- 172.21.15.148:0/3999111163 wait complete. 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.469+0000 7f62e4f5c640 1 Processor -- start 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.469+0000 7f62e4f5c640 1 -- start start 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62e006fa00 0x7f62e014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62e00ce900 0x7f62e014bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 0x7f62e00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e0156360 con 0x7f62e0155f80 2024-04-25T10:08:13.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00d07f0 con 0x7f62e00ce900 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00cf250 con 0x7f62e006fa00 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 0x7f62e00b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 0x7f62e00b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:55274/0 (socket says 172.21.15.148:55274) 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 -- 172.21.15.148:0/1551951832 learned_addr learned my addr 172.21.15.148:0/1551951832 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 -- 172.21.15.148:0/1551951832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62e006fa00 msgr2=0x7f62e014a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62e006fa00 0x7f62e014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 -- 172.21.15.148:0/1551951832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62e00ce900 msgr2=0x7f62e014bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62e00ce900 0x7f62e014bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 -- 172.21.15.148:0/1551951832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e0151530 con 0x7f62e0155f80 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62ded9d640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 0x7f62e00b22a0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009f10 tx=0x7f62d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8041020 con 0x7f62e0155f80 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d8007780 con 0x7f62e0155f80 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e0151810 con 0x7f62e0155f80 2024-04-25T10:08:13.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8036040 con 0x7f62e0155f80 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.470+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e014a6c0 con 0x7f62e0155f80 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.471+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e00ce3f0 con 0x7f62e0155f80 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.472+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62d8044020 con 0x7f62e0155f80 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.472+0000 7f62d6ffd640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62c006b4d0 0x7f62c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.472+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f62d80ab2f0 con 0x7f62e0155f80 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.472+0000 7f62de59c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62c006b4d0 0x7f62c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.472+0000 7f62de59c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62c006b4d0 0x7f62c006d990 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009670 tx=0x7f62d0006bf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.482 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.476+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62e00ce3f0 con 0x7f62e0155f80 2024-04-25T10:08:13.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:13.690+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f62e0073a00 con 0x7f62e0155f80 2024-04-25T10:08:14.358 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.352+0000 7f62d6ffd640 1 -- 172.21.15.148:0/1551951832 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v121) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62d804b030 con 0x7f62e0155f80 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.354+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62c006b4d0 msgr2=0x7f62c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.354+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62c006b4d0 0x7f62c006d990 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009670 tx=0x7f62d0006bf0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.354+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 msgr2=0x7f62e00b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.354+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 0x7f62e00b22a0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009f10 tx=0x7f62d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 shutdown_connections 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f62c006b4d0 0x7f62c006d990 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e0155f80 0x7f62e00b22a0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62e00ce900 0x7f62e014bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 --2- 172.21.15.148:0/1551951832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62e006fa00 0x7f62e014a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 >> 172.21.15.148:0/1551951832 conn(0x7f62e0061600 msgr2=0x7f62e00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 shutdown_connections 2024-04-25T10:08:14.361 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.355+0000 7f62e4f5c640 1 -- 172.21.15.148:0/1551951832 wait complete. 2024-04-25T10:08:14.376 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:08:14.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.513+0000 7fde107ce640 1 Processor -- start 2024-04-25T10:08:14.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.513+0000 7fde107ce640 1 -- start start 2024-04-25T10:08:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.514+0000 7fde107ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde080c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.514+0000 7fde107ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde080cd6f0 0x7fde080cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.514+0000 7fde107ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde080c6940 con 0x7fde080cdbf0 2024-04-25T10:08:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.514+0000 7fde107ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde080b0020 con 0x7fde080cd6f0 2024-04-25T10:08:14.520 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.514+0000 7fde107ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080cb5a0 con 0x7fde0815a9f0 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde080cd6f0 0x7fde080cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0e56a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde080c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde080cd6f0 0x7fde080cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:47244/0 (socket says 172.21.15.148:47244) 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0e56a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde080c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:40740/0 (socket says 172.21.15.148:40740) 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 -- 172.21.15.148:0/3944708644 learned_addr learned my addr 172.21.15.148:0/3944708644 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0dd69640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fde080cdbf0 0x7fde080ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53438/0 (socket says 172.21.15.148:53438) 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 -- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 msgr2=0x7fde080c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 --2- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde080c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 -- 172.21.15.148:0/3944708644 >> v1:172.21.15.188:6789/0 conn(0x7fde080cdbf0 legacy=0x7fde080ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0ed6b640 1 -- 172.21.15.148:0/3944708644 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0815add0 con 0x7fde080cd6f0 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0d568640 1 -- 172.21.15.148:0/3944708644 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164219006 0 0) 0x7fddf8001110 con 0x7fde080cdbf0 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.515+0000 7fde0e56a640 1 --2- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde080c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.516+0000 7fde0ed6b640 1 --2- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cd6f0 0x7fde080cba30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fde0000bb10 tx=0x7fde0002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53809986f1f8cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.516+0000 7fde0d568640 1 -- 172.21.15.148:0/3944708644 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00036020 con 0x7fde080cd6f0 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.516+0000 7fde0d568640 1 -- 172.21.15.148:0/3944708644 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00032820 con 0x7fde080cd6f0 2024-04-25T10:08:14.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.516+0000 7fde0d568640 1 -- 172.21.15.148:0/3944708644 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00032bf0 con 0x7fde080cd6f0 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.516+0000 7fde107ce640 1 -- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cd6f0 msgr2=0x7fde080cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.516+0000 7fde107ce640 1 --2- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cd6f0 0x7fde080cba30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fde0000bb10 tx=0x7fde0002c920 comp rx=0 tx=0).stop 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.517+0000 7fde107ce640 1 -- 172.21.15.148:0/3944708644 shutdown_connections 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.517+0000 7fde107ce640 1 --2- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cd6f0 0x7fde080cba30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.517+0000 7fde107ce640 1 --2- 172.21.15.148:0/3944708644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde080c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.517+0000 7fde107ce640 1 -- 172.21.15.148:0/3944708644 >> 172.21.15.148:0/3944708644 conn(0x7fde080b2390 msgr2=0x7fde080b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.517+0000 7fde107ce640 1 -- 172.21.15.148:0/3944708644 shutdown_connections 2024-04-25T10:08:14.526 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.517+0000 7fde107ce640 1 -- 172.21.15.148:0/3944708644 wait complete. 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 Processor -- start 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 -- start start 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 0x7fde08145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cdbf0 0x7fde08148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde08148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080cb230 con 0x7fde080cd6f0 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde080caf50 con 0x7fde080cdbf0 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.518+0000 7fde107ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080ce070 con 0x7fde0815a9f0 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 0x7fde08145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 0x7fde08145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60286/0 (socket says 172.21.15.148:60286) 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 -- 172.21.15.148:0/1392292374 learned_addr learned my addr 172.21.15.148:0/1392292374 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0ed6b640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde08148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0dd69640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cdbf0 0x7fde08148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.527 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 -- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 msgr2=0x7fde08148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde08148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 -- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cdbf0 msgr2=0x7fde08148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cdbf0 0x7fde08148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 -- 172.21.15.148:0/1392292374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde080b38a0 con 0x7fde080cd6f0 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde0e56a640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 0x7fde08145b50 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fde0400efc0 tx=0x7fde0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04018020 con 0x7fde080cd6f0 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04014680 con 0x7fde080cd6f0 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde080ae1b0 con 0x7fde080cd6f0 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04014a50 con 0x7fde080cd6f0 2024-04-25T10:08:14.528 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.519+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde080ae770 con 0x7fde080cd6f0 2024-04-25T10:08:14.529 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.520+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0806cc80 con 0x7fde080cd6f0 2024-04-25T10:08:14.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.521+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde04005d10 con 0x7fde080cd6f0 2024-04-25T10:08:14.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.521+0000 7fddfeffd640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddec06b500 0x7fddec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.521+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fde0408b100 con 0x7fde080cd6f0 2024-04-25T10:08:14.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.521+0000 7fde0dd69640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddec06b500 0x7fddec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.530 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.522+0000 7fde0dd69640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddec06b500 0x7fddec06d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fde0806f710 tx=0x7fddf8007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.531 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.525+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde04022030 con 0x7fde080cd6f0 2024-04-25T10:08:14.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:14.698+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde080adfc0 con 0x7fde080cd6f0 2024-04-25T10:08:15.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.682+0000 7fddfeffd640 1 -- 172.21.15.148:0/1392292374 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7fde04010040 con 0x7fde080cd6f0 2024-04-25T10:08:15.689 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.685+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddec06b500 msgr2=0x7fddec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.685+0000 7fde107ce640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddec06b500 0x7fddec06d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fde0806f710 tx=0x7fddf8007b50 comp rx=0 tx=0).stop 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.685+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 msgr2=0x7fde08145b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.685+0000 7fde107ce640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 0x7fde08145b50 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7fde0400efc0 tx=0x7fde0400c490 comp rx=0 tx=0).stop 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 shutdown_connections 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fddec06b500 0x7fddec06d9c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde0815a9f0 0x7fde08148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde080cdbf0 0x7fde08148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 --2- 172.21.15.148:0/1392292374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde080cd6f0 0x7fde08145b50 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 >> 172.21.15.148:0/1392292374 conn(0x7fde080b2390 msgr2=0x7fde0806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 shutdown_connections 2024-04-25T10:08:15.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.686+0000 7fde107ce640 1 -- 172.21.15.148:0/1392292374 wait complete. 2024-04-25T10:08:15.705 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.807+0000 7ffb458e1640 1 Processor -- start 2024-04-25T10:08:15.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.807+0000 7ffb458e1640 1 -- start start 2024-04-25T10:08:15.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb458e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb401616f0 0x7ffb40161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb458e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400ce0a0 0x7ffb400ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb458e1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4006c8d0 con 0x7ffb401610f0 2024-04-25T10:08:15.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb458e1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb401614d0 con 0x7ffb401616f0 2024-04-25T10:08:15.815 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb458e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400c6960 con 0x7ffb400ce0a0 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb3e7fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffb401616f0 0x7ffb40161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb3f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400ce0a0 0x7ffb400ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.808+0000 7ffb3effd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ffb401610f0 0x7ffb4006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53452/0 (socket says 172.21.15.148:53452) 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3effd640 1 -- 172.21.15.148:0/859168983 learned_addr learned my addr 172.21.15.148:0/859168983 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3dffb640 1 -- 172.21.15.148:0/859168983 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889532490 0 0) 0x7ffb4006c8d0 con 0x7ffb401610f0 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3dffb640 1 -- 172.21.15.148:0/859168983 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb28003520 con 0x7ffb401610f0 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3f7fe640 1 -- 172.21.15.148:0/859168983 >> v2:172.21.15.188:3300/0 conn(0x7ffb401616f0 msgr2=0x7ffb40161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3f7fe640 1 --2- 172.21.15.148:0/859168983 >> v2:172.21.15.188:3300/0 conn(0x7ffb401616f0 0x7ffb40161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3f7fe640 1 -- 172.21.15.148:0/859168983 >> v1:172.21.15.188:6789/0 conn(0x7ffb401610f0 legacy=0x7ffb4006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.809+0000 7ffb3f7fe640 1 -- 172.21.15.148:0/859168983 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40162260 con 0x7ffb400ce0a0 2024-04-25T10:08:15.816 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb3f7fe640 1 --2- 172.21.15.148:0/859168983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400ce0a0 0x7ffb400ce530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffb300097a0 tx=0x7ffb3002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2c2d1229d7fc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb3dffb640 1 -- 172.21.15.148:0/859168983 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30037070 con 0x7ffb400ce0a0 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb3dffb640 1 -- 172.21.15.148:0/859168983 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30032440 con 0x7ffb400ce0a0 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb3dffb640 1 -- 172.21.15.148:0/859168983 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb300327f0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 -- 172.21.15.148:0/859168983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400ce0a0 msgr2=0x7ffb400ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 --2- 172.21.15.148:0/859168983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400ce0a0 0x7ffb400ce530 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffb300097a0 tx=0x7ffb3002a180 comp rx=0 tx=0).stop 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 -- 172.21.15.148:0/859168983 shutdown_connections 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 --2- 172.21.15.148:0/859168983 >> v2:172.21.15.188:3300/0 conn(0x7ffb401616f0 0x7ffb40161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 --2- 172.21.15.148:0/859168983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb400ce0a0 0x7ffb400ce530 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 -- 172.21.15.148:0/859168983 >> 172.21.15.148:0/859168983 conn(0x7ffb4006f500 msgr2=0x7ffb4006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 -- 172.21.15.148:0/859168983 shutdown_connections 2024-04-25T10:08:15.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.810+0000 7ffb458e1640 1 -- 172.21.15.148:0/859168983 wait complete. 2024-04-25T10:08:15.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 Processor -- start 2024-04-25T10:08:15.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 -- start start 2024-04-25T10:08:15.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 0x7ffb4015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb401610f0 0x7ffb4014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb401616f0 0x7ffb4015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cfce0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cfa00 con 0x7ffb401610f0 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.812+0000 7ffb458e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cf690 con 0x7ffb401616f0 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 0x7ffb4015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 0x7ffb4015a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60296/0 (socket says 172.21.15.148:60296) 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 -- 172.21.15.148:0/2328713358 learned_addr learned my addr 172.21.15.148:0/2328713358 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3e7fc640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb401610f0 0x7ffb4014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3f7fe640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb401616f0 0x7ffb4015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 -- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb401616f0 msgr2=0x7ffb4015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb401616f0 0x7ffb4015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 -- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb401610f0 msgr2=0x7ffb4014d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb401610f0 0x7ffb4014d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 -- 172.21.15.148:0/2328713358 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4014d6a0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb3effd640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 0x7ffb4015a8a0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800cb40 tx=0x7ffb3800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38017070 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3800aca0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4015d9f0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38012620 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.813+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4015dfb0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.814+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40073a00 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.815+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb3801a070 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.815+0000 7ffb27fff640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb2c06b4d0 0x7ffb2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.815+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5835+0+0 (secure 0 0 0) 0x7ffb38089fa0 con 0x7ffb400ce0a0 2024-04-25T10:08:15.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.815+0000 7ffb3e7fc640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb2c06b4d0 0x7ffb2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.816+0000 7ffb3e7fc640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb2c06b4d0 0x7ffb2c06d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ffb401572f0 tx=0x7ffb34009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.825 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:15.820+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb380587e0 con 0x7ffb400ce0a0 2024-04-25T10:08:16.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.001+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb400cf150 con 0x7ffb400ce0a0 2024-04-25T10:08:16.690 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.684+0000 7ffb27fff640 1 -- 172.21.15.148:0/2328713358 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v203) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffb3805c880 con 0x7ffb400ce0a0 2024-04-25T10:08:16.690 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:16.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb2c06b4d0 msgr2=0x7ffb2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb2c06b4d0 0x7ffb2c06d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ffb401572f0 tx=0x7ffb34009190 comp rx=0 tx=0).stop 2024-04-25T10:08:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 msgr2=0x7ffb4015a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 0x7ffb4015a8a0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800cb40 tx=0x7ffb3800a060 comp rx=0 tx=0).stop 2024-04-25T10:08:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 shutdown_connections 2024-04-25T10:08:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ffb2c06b4d0 0x7ffb2c06d990 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb401616f0 0x7ffb4015ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.102 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb401610f0 0x7ffb4014d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 --2- 172.21.15.148:0/2328713358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffb400ce0a0 0x7ffb4015a8a0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 >> 172.21.15.148:0/2328713358 conn(0x7ffb4006f500 msgr2=0x7ffb400b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 shutdown_connections 2024-04-25T10:08:17.103 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:16.688+0000 7ffb458e1640 1 -- 172.21.15.148:0/2328713358 wait complete. 2024-04-25T10:08:17.104 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T10:08:17.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.207+0000 7fa9d96b3640 1 Processor -- start 2024-04-25T10:08:17.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.207+0000 7fa9d96b3640 1 -- start start 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d96b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 0x7fa9d4072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d96b3640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d4072ca0 con 0x7fa9d415b990 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d96b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d415c260 con 0x7fa9d415bd70 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d96b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415c540 con 0x7fa9d406c300 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d2ffd640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa9d415b990 0x7fa9d406bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44358/0 (socket says 172.21.15.148:44358) 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d2ffd640 1 -- 172.21.15.148:0/2906787254 learned_addr learned my addr 172.21.15.148:0/2906787254 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.208+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773895165 0 0) 0x7fa9d4072ca0 con 0x7fa9d415b990 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9bc003520 con 0x7fa9d415b990 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d27fc640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 0x7fa9d4072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540982164 0 0) 0x7fa9bc003520 con 0x7fa9d415b990 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 msgr2=0x7fa9d4072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:17.217 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 0x7fa9d4072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 >> v1:172.21.15.188:6789/0 conn(0x7fa9d415bd70 legacy=0x7fa9d40acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40cb9d0 con 0x7fa9d415b990 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fa9c40019f0 con 0x7fa9d415b990 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 >> v1:172.21.15.148:6789/0 conn(0x7fa9d415b990 legacy=0x7fa9d406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc0039e0 0x7fa9bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 0x7fa9bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d415c540 con 0x7fa9bc004d40 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc00ba70 con 0x7fa9bc0039e0 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc00bc70 con 0x7fa9d406c300 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d37fe640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d27fc640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 0x7fa9bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d2ffd640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc0039e0 0x7fa9bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d37fe640 1 -- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 msgr2=0x7fa9bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d37fe640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d406c300 0x7fa9bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d37fe640 1 -- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc0039e0 msgr2=0x7fa9bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d37fe640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc0039e0 0x7fa9bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.209+0000 7fa9d37fe640 1 -- 172.21.15.148:0/2906787254 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc003520 con 0x7fa9bc004d40 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.210+0000 7fa9d37fe640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9bc0051c0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40718d0 tx=0x7fa9bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869b16dac588c046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.210+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc001370 con 0x7fa9bc004d40 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.210+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0052d0 con 0x7fa9bc004d40 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.210+0000 7fa9d1ffb640 1 -- 172.21.15.148:0/2906787254 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc02ab90 con 0x7fa9bc004d40 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d96b3640 1 -- 172.21.15.148:0/2906787254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 msgr2=0x7fa9bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/2906787254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9bc0051c0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40718d0 tx=0x7fa9bc005830 comp rx=0 tx=0).stop 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d37fe640 1 -- 172.21.15.148:0/2906787254 reap_dead start 2024-04-25T10:08:17.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d96b3640 1 -- 172.21.15.148:0/2906787254 shutdown_connections 2024-04-25T10:08:17.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d96b3640 1 -- 172.21.15.148:0/2906787254 >> 172.21.15.148:0/2906787254 conn(0x7fa9d4071af0 msgr2=0x7fa9d4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d96b3640 1 -- 172.21.15.148:0/2906787254 shutdown_connections 2024-04-25T10:08:17.220 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.211+0000 7fa9d96b3640 1 -- 172.21.15.148:0/2906787254 wait complete. 2024-04-25T10:08:17.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.212+0000 7fa9d96b3640 1 Processor -- start 2024-04-25T10:08:17.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.212+0000 7fa9d96b3640 1 -- start start 2024-04-25T10:08:17.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d96b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9d414f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d96b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d40c85f0 0x7fa9d415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d96b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d40b4de0 0x7fa9d415f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d96b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40af250 con 0x7fa9bc004d40 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d96b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40af0d0 con 0x7fa9d40c85f0 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d96b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40af3d0 con 0x7fa9d40b4de0 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9d414f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9d414f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60322/0 (socket says 172.21.15.148:60322) 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 -- 172.21.15.148:0/1167848898 learned_addr learned my addr 172.21.15.148:0/1167848898 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d27fc640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d40c85f0 0x7fa9d415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d37fe640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d40b4de0 0x7fa9d415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 -- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d40b4de0 msgr2=0x7fa9d415f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d40b4de0 0x7fa9d415f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 -- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d40c85f0 msgr2=0x7fa9d415eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d40c85f0 0x7fa9d415eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 -- 172.21.15.148:0/1167848898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40c90b0 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d27fc640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d40c85f0 0x7fa9d415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.213+0000 7fa9d2ffd640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9d414f8f0 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40079a0 tx=0x7fa9c400a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.214+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4017020 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.214+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c4007a90 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.214+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4013610 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.214+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40c8af0 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.214+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d415cff0 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.215+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d40c6d20 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.215+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9c401a020 con 0x7fa9bc004d40 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.216+0000 7fa9bb7fe640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c006b4d0 0x7fa9c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.216+0000 7fa9d27fc640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c006b4d0 0x7fa9c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.216+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa9c408af20 con 0x7fa9bc004d40 2024-04-25T10:08:17.225 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.216+0000 7fa9d27fc640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c006b4d0 0x7fa9c006d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa9d415b2f0 tx=0x7fa9c8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.220+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9c4021020 con 0x7fa9bc004d40 2024-04-25T10:08:17.398 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.392+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7fa9d4073a00 con 0x7fa9bc004d40 2024-04-25T10:08:17.401 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.395+0000 7fa9bb7fe640 1 -- 172.21.15.148:0/1167848898 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9c4059ec0 con 0x7fa9bc004d40 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c006b4d0 msgr2=0x7fa9c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c006b4d0 0x7fa9c006d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa9d415b2f0 tx=0x7fa9c8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 msgr2=0x7fa9d414f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9d414f8f0 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40079a0 tx=0x7fa9c400a730 comp rx=0 tx=0).stop 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 shutdown_connections 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa9c006b4d0 0x7fa9c006d990 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9d40b4de0 0x7fa9d415f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d40c85f0 0x7fa9d415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 --2- 172.21.15.148:0/1167848898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc004d40 0x7fa9d414f8f0 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 >> 172.21.15.148:0/1167848898 conn(0x7fa9d4071af0 msgr2=0x7fa9d40b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 shutdown_connections 2024-04-25T10:08:17.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.398+0000 7fa9d96b3640 1 -- 172.21.15.148:0/1167848898 wait complete. 2024-04-25T10:08:17.415 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:08:17.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.508+0000 7f7d2765a640 1 Processor -- start 2024-04-25T10:08:17.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.509+0000 7f7d2765a640 1 -- start start 2024-04-25T10:08:17.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.509+0000 7f7d2765a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d20161890 0x7f7d200b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.509+0000 7f7d2765a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d200b54a0 0x7f7d200b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.509+0000 7f7d2765a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d200cf1d0 con 0x7f7d20161240 2024-04-25T10:08:17.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.509+0000 7f7d2765a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d200b7120 con 0x7f7d20161890 2024-04-25T10:08:17.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.509+0000 7f7d2765a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d200b5e30 con 0x7f7d200b54a0 2024-04-25T10:08:17.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d24bf5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f7d20161890 0x7f7d200b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d253f6640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f7d20161240 0x7f7d200ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44374/0 (socket says 172.21.15.148:44374) 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d253f6640 1 -- 172.21.15.148:0/68341449 learned_addr learned my addr 172.21.15.148:0/68341449 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d25bf7640 1 --2- 172.21.15.148:0/68341449 >> v2:172.21.15.188:3300/0 conn(0x7f7d200b54a0 0x7f7d200b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d24bf5640 1 -- 172.21.15.148:0/68341449 >> v2:172.21.15.188:3300/0 conn(0x7f7d200b54a0 msgr2=0x7f7d200b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d24bf5640 1 --2- 172.21.15.148:0/68341449 >> v2:172.21.15.188:3300/0 conn(0x7f7d200b54a0 0x7f7d200b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d24bf5640 1 -- 172.21.15.148:0/68341449 >> v1:172.21.15.148:6789/0 conn(0x7f7d20161240 legacy=0x7f7d200ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.510+0000 7f7d24bf5640 1 -- 172.21.15.148:0/68341449 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20161620 con 0x7f7d20161890 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d24bf5640 1 --2- 172.21.15.148:0/68341449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d20161890 0x7f7d200b6be0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f7d2008ce10 tx=0x7f7d1002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38efba2721b996f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d17fff640 1 -- 172.21.15.148:0/68341449 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10037070 con 0x7f7d20161890 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d17fff640 1 -- 172.21.15.148:0/68341449 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1002ad30 con 0x7f7d20161890 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d17fff640 1 -- 172.21.15.148:0/68341449 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10032660 con 0x7f7d20161890 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 -- 172.21.15.148:0/68341449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d20161890 msgr2=0x7f7d200b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 --2- 172.21.15.148:0/68341449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d20161890 0x7f7d200b6be0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f7d2008ce10 tx=0x7f7d1002a0a0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.518 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 -- 172.21.15.148:0/68341449 shutdown_connections 2024-04-25T10:08:17.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 --2- 172.21.15.148:0/68341449 >> v2:172.21.15.188:3300/0 conn(0x7f7d200b54a0 0x7f7d200b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 --2- 172.21.15.148:0/68341449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d20161890 0x7f7d200b6be0 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 -- 172.21.15.148:0/68341449 >> 172.21.15.148:0/68341449 conn(0x7f7d200d04a0 msgr2=0x7f7d200d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 -- 172.21.15.148:0/68341449 shutdown_connections 2024-04-25T10:08:17.519 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.511+0000 7f7d2765a640 1 -- 172.21.15.148:0/68341449 wait complete. 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.512+0000 7f7d2765a640 1 Processor -- start 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 -- start start 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 0x7f7d200cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d20161240 0x7f7d20150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20161890 0x7f7d200d2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d201623c0 con 0x7f7d200b54a0 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d201620e0 con 0x7f7d20161890 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d2765a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d201626a0 con 0x7f7d20161240 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d253f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 0x7f7d200cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.522 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d253f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 0x7f7d200cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60350/0 (socket says 172.21.15.148:60350) 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.513+0000 7f7d253f6640 1 -- 172.21.15.148:0/2222360420 learned_addr learned my addr 172.21.15.148:0/2222360420 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d25bf7640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20161890 0x7f7d200d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d253f6640 1 -- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d20161240 msgr2=0x7f7d20150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d24bf5640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d20161240 0x7f7d20150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d253f6640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d20161240 0x7f7d20150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d253f6640 1 -- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20161890 msgr2=0x7f7d200d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d253f6640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20161890 0x7f7d200d2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d253f6640 1 -- 172.21.15.148:0/2222360420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200cc4e0 con 0x7f7d200b54a0 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d25bf7640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20161890 0x7f7d200d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d24bf5640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d20161240 0x7f7d20150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d253f6640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 0x7f7d200cd580 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c780 tx=0x7f7d1c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c018070 con 0x7f7d200b54a0 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c0072f0 con 0x7f7d200b54a0 2024-04-25T10:08:17.523 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0047e0 con 0x7f7d200b54a0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d200cc7c0 con 0x7f7d200b54a0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.514+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d200ccd80 con 0x7f7d200b54a0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.515+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d20067be0 con 0x7f7d200b54a0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.515+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d1c004980 con 0x7f7d200b54a0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.516+0000 7f7d15ffb640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0406b4d0 0x7f7d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.516+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7d1c089540 con 0x7f7d200b54a0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.516+0000 7f7d24bf5640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0406b4d0 0x7f7d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.524 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.516+0000 7f7d24bf5640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0406b4d0 0x7f7d0406d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7d10034000 tx=0x7f7d1002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.525 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.520+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d20067be0 con 0x7f7d200b54a0 2024-04-25T10:08:17.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.693+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7d20160380 con 0x7f7d200b54a0 2024-04-25T10:08:17.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.703+0000 7f7d15ffb640 1 -- 172.21.15.148:0/2222360420 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d1c004c70 con 0x7f7d200b54a0 2024-04-25T10:08:17.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0406b4d0 msgr2=0x7f7d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0406b4d0 0x7f7d0406d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7d10034000 tx=0x7f7d1002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 msgr2=0x7f7d200cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 0x7f7d200cd580 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00c780 tx=0x7f7d1c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 shutdown_connections 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f7d0406b4d0 0x7f7d0406d990 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d20161890 0x7f7d200d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d20161240 0x7f7d20150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 --2- 172.21.15.148:0/2222360420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d200b54a0 0x7f7d200cd580 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 >> 172.21.15.148:0/2222360420 conn(0x7f7d200d04a0 msgr2=0x7f7d200d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 shutdown_connections 2024-04-25T10:08:17.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.705+0000 7f7d2765a640 1 -- 172.21.15.148:0/2222360420 wait complete. 2024-04-25T10:08:17.722 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:08:17.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.813+0000 7f44accf5640 1 Processor -- start 2024-04-25T10:08:17.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.814+0000 7f44accf5640 1 -- start start 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.814+0000 7f44accf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80cfec0 0x7f44a80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.814+0000 7f44accf5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44a80ce3c0 0x7f44a806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.814+0000 7f44accf5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a8071480 con 0x7f44a8154ee0 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.814+0000 7f44accf5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a8072090 con 0x7f44a80ce3c0 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.814+0000 7f44accf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8071230 con 0x7f44a80cfec0 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a6d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44a80ce3c0 0x7f44a806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a5d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80cfec0 0x7f44a80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a6d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44a80ce3c0 0x7f44a806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:47322/0 (socket says 172.21.15.148:47322) 2024-04-25T10:08:17.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a6d9d640 1 -- 172.21.15.148:0/217172350 learned_addr learned my addr 172.21.15.148:0/217172350 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a659c640 1 --1- 172.21.15.148:0/217172350 >> v1:172.21.15.188:6789/0 conn(0x7f44a8154ee0 0x7f44a80cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.148:53476/0 (socket says 172.21.15.148:53476) 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a559a640 1 -- 172.21.15.148:0/217172350 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266056649 0 0) 0x7f44a8071480 con 0x7f44a8154ee0 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a559a640 1 -- 172.21.15.148:0/217172350 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4494003520 con 0x7f44a8154ee0 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a5d9b640 1 -- 172.21.15.148:0/217172350 >> v2:172.21.15.188:3300/0 conn(0x7f44a80ce3c0 msgr2=0x7f44a806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a5d9b640 1 --2- 172.21.15.148:0/217172350 >> v2:172.21.15.188:3300/0 conn(0x7f44a80ce3c0 0x7f44a806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a5d9b640 1 -- 172.21.15.148:0/217172350 >> v1:172.21.15.188:6789/0 conn(0x7f44a8154ee0 legacy=0x7f44a80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.815+0000 7f44a5d9b640 1 -- 172.21.15.148:0/217172350 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a8071760 con 0x7f44a80cfec0 2024-04-25T10:08:17.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.816+0000 7f44a5d9b640 1 --2- 172.21.15.148:0/217172350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80cfec0 0x7f44a80cde80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44900097a0 tx=0x7f449002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c26fc27f6739db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.816+0000 7f44a559a640 1 -- 172.21.15.148:0/217172350 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490037070 con 0x7f44a80cfec0 2024-04-25T10:08:17.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.816+0000 7f44a559a640 1 -- 172.21.15.148:0/217172350 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4490032440 con 0x7f44a80cfec0 2024-04-25T10:08:17.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.816+0000 7f44a559a640 1 -- 172.21.15.148:0/217172350 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490032810 con 0x7f44a80cfec0 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 -- 172.21.15.148:0/217172350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80cfec0 msgr2=0x7f44a80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 --2- 172.21.15.148:0/217172350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80cfec0 0x7f44a80cde80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44900097a0 tx=0x7f449002a180 comp rx=0 tx=0).stop 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 -- 172.21.15.148:0/217172350 shutdown_connections 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 --2- 172.21.15.148:0/217172350 >> v2:172.21.15.188:3300/0 conn(0x7f44a80ce3c0 0x7f44a806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 --2- 172.21.15.148:0/217172350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80cfec0 0x7f44a80cde80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 -- 172.21.15.148:0/217172350 >> 172.21.15.148:0/217172350 conn(0x7f44a806abe0 msgr2=0x7f44a806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 -- 172.21.15.148:0/217172350 shutdown_connections 2024-04-25T10:08:17.824 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.817+0000 7f44accf5640 1 -- 172.21.15.148:0/217172350 wait complete. 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.818+0000 7f44accf5640 1 Processor -- start 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.818+0000 7f44accf5640 1 -- start start 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44accf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a80ce3c0 0x7f44a8154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44accf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 0x7f44a80aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44accf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a8154ee0 0x7f44a80aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44accf5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8071d20 con 0x7f44a80cfec0 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44accf5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a80ce800 con 0x7f44a80ce3c0 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44accf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a81552c0 con 0x7f44a8154ee0 2024-04-25T10:08:17.827 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 0x7f44a80aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 0x7f44a80aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60352/0 (socket says 172.21.15.148:60352) 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 -- 172.21.15.148:0/2586354734 learned_addr learned my addr 172.21.15.148:0/2586354734 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a659c640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a80ce3c0 0x7f44a8154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a6d9d640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a8154ee0 0x7f44a80aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 -- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a8154ee0 msgr2=0x7f44a80aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a8154ee0 0x7f44a80aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 -- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a80ce3c0 msgr2=0x7f44a8154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a80ce3c0 0x7f44a8154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 -- 172.21.15.148:0/2586354734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80ad840 con 0x7f44a80cfec0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.819+0000 7f44a5d9b640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 0x7f44a80aa9b0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f4490035040 tx=0x7f44900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.820+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490041030 con 0x7f44a80cfec0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.820+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449003d4e0 con 0x7f44a80cfec0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.820+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a80adb20 con 0x7f44a80cfec0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.820+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490037040 con 0x7f44a80cfec0 2024-04-25T10:08:17.828 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.820+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a80ae0e0 con 0x7f44a80cfec0 2024-04-25T10:08:17.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.821+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a8073990 con 0x7f44a80cfec0 2024-04-25T10:08:17.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.821+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4490044070 con 0x7f44a80cfec0 2024-04-25T10:08:17.829 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.821+0000 7f449effd640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f448806b4d0 0x7f448806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:17.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.822+0000 7f44a659c640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f448806b4d0 0x7f448806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:17.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.822+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f44900aa700 con 0x7f44a80cfec0 2024-04-25T10:08:17.830 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.822+0000 7f44a659c640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f448806b4d0 0x7f448806d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f44a00012e0 tx=0x7f44a0007ed0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:17.831 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:17.826+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44900797d0 con 0x7f44a80cfec0 2024-04-25T10:08:18.065 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.059+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f44a806d360 con 0x7f44a80cfec0 2024-04-25T10:08:18.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.703+0000 7f449effd640 1 -- 172.21.15.148:0/2586354734 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v662) v1 ==== 133+0+0 (secure 0 0 0) 0x7f44900790d0 con 0x7f44a80cfec0 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.705+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f448806b4d0 msgr2=0x7f448806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.705+0000 7f44accf5640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f448806b4d0 0x7f448806d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f44a00012e0 tx=0x7f44a0007ed0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.705+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 msgr2=0x7f44a80aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.705+0000 7f44accf5640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 0x7f44a80aa9b0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f4490035040 tx=0x7f44900046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 shutdown_connections 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f448806b4d0 0x7f448806d990 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a8154ee0 0x7f44a80aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44a80cfec0 0x7f44a80aa9b0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 --2- 172.21.15.148:0/2586354734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a80ce3c0 0x7f44a8154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 >> 172.21.15.148:0/2586354734 conn(0x7f44a806abe0 msgr2=0x7f44a80b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 shutdown_connections 2024-04-25T10:08:18.713 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:18.706+0000 7f44accf5640 1 -- 172.21.15.148:0/2586354734 wait complete. 2024-04-25T10:08:18.724 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:08:18.724 INFO:tasks.cephfs_test_runner: 2024-04-25T10:08:18.724 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:08:18.724 INFO:tasks.cephfs_test_runner:Ran 13 tests in 789.559s 2024-04-25T10:08:18.725 INFO:tasks.cephfs_test_runner: 2024-04-25T10:08:18.725 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:08:18.725 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:08:18.734 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:08:18.777 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:08:18.777 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:08:18.778 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:18.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:18.794 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-25T10:08:18.794 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:18.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056109,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20503 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497696k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:18.851 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:08:18.851 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:18.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:18.870 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-25T10:08:18.870 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:18.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2479 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:18.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:18.928 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:18.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:18.944 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:19.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.009 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:19.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.074 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-04-25T10:08:19.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.170 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-04-25T10:08:19.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.184 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-04-25T10:08:19.251 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-04-25T10:08:19.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:19.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.253 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-25T10:08:19.272 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:08:19.272 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:08:19.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:08:19.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.340 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:08:19.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.407 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-04-25T10:08:19.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.493 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-04-25T10:08:19.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:19.509 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-04-25T10:08:19.575 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-04-25T10:08:19.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:19.577 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:08:19.587 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:08:19.587 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.679+0000 7fbb8805d640 1 Processor -- start 2024-04-25T10:08:19.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.679+0000 7fbb8805d640 1 -- start start 2024-04-25T10:08:19.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb8805d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800b2cb0 0x7fbb800d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.687 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb8805d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb800d1020 0x7fbb800d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb8805d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800b97a0 con 0x7fbb8015dbe0 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb8805d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8015dfc0 con 0x7fbb800d1020 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb8805d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800d1a00 con 0x7fbb800b2cb0 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb865fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb800d1020 0x7fbb800d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb865fa640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb800d1020 0x7fbb800d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60362/0 (socket says 172.21.15.148:60362) 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.680+0000 7fbb865fa640 1 -- 172.21.15.148:0/1910326082 learned_addr learned my addr 172.21.15.148:0/1910326082 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.681+0000 7fbb855f8640 1 --2- 172.21.15.148:0/1910326082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800b2cb0 0x7fbb800d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.681+0000 7fbb865fa640 1 -- 172.21.15.148:0/1910326082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800b2cb0 msgr2=0x7fbb800d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.681+0000 7fbb865fa640 1 --2- 172.21.15.148:0/1910326082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800b2cb0 0x7fbb800d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.681+0000 7fbb865fa640 1 -- 172.21.15.148:0/1910326082 >> v1:172.21.15.188:6789/0 conn(0x7fbb8015dbe0 legacy=0x7fbb800d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.681+0000 7fbb865fa640 1 -- 172.21.15.148:0/1910326082 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800d1c50 con 0x7fbb800d1020 2024-04-25T10:08:19.688 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.681+0000 7fbb865fa640 1 --2- 172.21.15.148:0/1910326082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800d1020 0x7fbb800d2c70 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009870 tx=0x7fbb7802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febb744b8eb65966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb84df7640 1 -- 172.21.15.148:0/1910326082 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037070 con 0x7fbb800d1020 2024-04-25T10:08:19.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb84df7640 1 -- 172.21.15.148:0/1910326082 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78032440 con 0x7fbb800d1020 2024-04-25T10:08:19.689 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb84df7640 1 -- 172.21.15.148:0/1910326082 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78032810 con 0x7fbb800d1020 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 -- 172.21.15.148:0/1910326082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800d1020 msgr2=0x7fbb800d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 --2- 172.21.15.148:0/1910326082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800d1020 0x7fbb800d2c70 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009870 tx=0x7fbb7802a210 comp rx=0 tx=0).stop 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 -- 172.21.15.148:0/1910326082 shutdown_connections 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 --2- 172.21.15.148:0/1910326082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800d1020 0x7fbb800d2c70 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 --2- 172.21.15.148:0/1910326082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800b2cb0 0x7fbb800d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 -- 172.21.15.148:0/1910326082 >> 172.21.15.148:0/1910326082 conn(0x7fbb80061a20 msgr2=0x7fbb80061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 -- 172.21.15.148:0/1910326082 shutdown_connections 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.682+0000 7fbb8805d640 1 -- 172.21.15.148:0/1910326082 wait complete. 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.683+0000 7fbb8805d640 1 Processor -- start 2024-04-25T10:08:19.692 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.683+0000 7fbb8805d640 1 -- start start 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb8805d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 0x7fbb801519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb8805d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1020 0x7fbb80160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb8805d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8015dbe0 0x7fbb800caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb8805d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800b92a0 con 0x7fbb800b2cb0 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb8805d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800d2210 con 0x7fbb8015dbe0 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb8805d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800b9420 con 0x7fbb800d1020 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb85df9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 0x7fbb801519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb85df9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 0x7fbb801519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60364/0 (socket says 172.21.15.148:60364) 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb85df9640 1 -- 172.21.15.148:0/2380938649 learned_addr learned my addr 172.21.15.148:0/2380938649 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb855f8640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1020 0x7fbb80160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.684+0000 7fbb865fa640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8015dbe0 0x7fbb800caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.693 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb85df9640 1 -- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1020 msgr2=0x7fbb80160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb85df9640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1020 0x7fbb80160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb85df9640 1 -- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8015dbe0 msgr2=0x7fbb800caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb85df9640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8015dbe0 0x7fbb800caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb85df9640 1 -- 172.21.15.148:0/2380938649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800cbda0 con 0x7fbb800b2cb0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb85df9640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 0x7fbb801519b0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00ba50 tx=0x7fbb7c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c019070 con 0x7fbb800b2cb0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00cc10 con 0x7fbb800b2cb0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c014440 con 0x7fbb800b2cb0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb800ccda0 con 0x7fbb800b2cb0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.685+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb800cd360 con 0x7fbb800b2cb0 2024-04-25T10:08:19.694 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.686+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb80073990 con 0x7fbb800b2cb0 2024-04-25T10:08:19.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.686+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb7c01c070 con 0x7fbb800b2cb0 2024-04-25T10:08:19.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.687+0000 7fbb767fc640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb6406b4d0 0x7fbb6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.687+0000 7fbb855f8640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb6406b4d0 0x7fbb6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.687+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbb7c089de0 con 0x7fbb800b2cb0 2024-04-25T10:08:19.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.687+0000 7fbb855f8640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb6406b4d0 0x7fbb6406d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbb801609b0 tx=0x7fbb70008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.697 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.691+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb7c058730 con 0x7fbb800b2cb0 2024-04-25T10:08:19.867 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.861+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb800b62e0 con 0x7fbb6406b4d0 2024-04-25T10:08:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.862+0000 7fbb767fc640 1 -- 172.21.15.148:0/2380938649 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13424 (secure 0 0 0) 0x7fbb7000f950 con 0x7fbb6406b4d0 2024-04-25T10:08:19.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:19.868 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:08:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb6406b4d0 msgr2=0x7fbb6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb6406b4d0 0x7fbb6406d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbb801609b0 tx=0x7fbb70008040 comp rx=0 tx=0).stop 2024-04-25T10:08:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 msgr2=0x7fbb801519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 0x7fbb801519b0 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00ba50 tx=0x7fbb7c00bf10 comp rx=0 tx=0).stop 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 shutdown_connections 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb6406b4d0 0x7fbb6406d990 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb8015dbe0 0x7fbb800caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb800d1020 0x7fbb80160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 --2- 172.21.15.148:0/2380938649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb800b2cb0 0x7fbb801519b0 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 >> 172.21.15.148:0/2380938649 conn(0x7fbb80061a20 msgr2=0x7fbb800ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 shutdown_connections 2024-04-25T10:08:19.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.864+0000 7fbb8805d640 1 -- 172.21.15.148:0/2380938649 wait complete. 2024-04-25T10:08:19.881 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-04-25T10:08:19.196935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494960,"kb_used_data":45712,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754479760,"statfs":{"total":773094113280,"available":772587274240,"internally_reserved":0,"allocated":46809088,"data_stored":35992012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":459918896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-35082,"num_keys_recovered":-1,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001106"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":505,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.570026+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:08:13.570026+0000","last_peered":"2024-04-25T10:08:13.570026+0000","last_clean":"2024-04-25T10:08:13.570026+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:08:13.570026+0000","last_undegraded":"2024-04-25T10:08:13.570026+0000","last_fullsized":"2024-04-25T10:08:13.570026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":7164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":7335936,"data_stored":5980663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55880,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2551808,"data_stored":1195461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55856,"kb_used_data":6628,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315984,"statfs":{"total":96636764160,"available":96579567616,"internally_reserved":0,"allocated":6787072,"data_stored":5430329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60032,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311808,"statfs":{"total":96636764160,"available":96575291392,"internally_reserved":0,"allocated":2543616,"data_stored":1195409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575021,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88256,"kb_used_data":12724,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94283584,"statfs":{"total":96636764160,"available":96546390016,"internally_reserved":0,"allocated":13029376,"data_stored":11681101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57924,"kb_used_data":4536,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313916,"statfs":{"total":96636764160,"available":96577449984,"internally_reserved":0,"allocated":4644864,"data_stored":3293103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57916,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313924,"statfs":{"total":96636764160,"available":96577458176,"internally_reserved":0,"allocated":4636672,"data_stored":3292471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":5156,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":5279744,"data_stored":3923475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:19.881 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.972+0000 7f6c82fb6640 1 Processor -- start 2024-04-25T10:08:19.978 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.972+0000 7f6c82fb6640 1 -- start start 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c82fb6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c7c0d2e90 0x7f6c7c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c82fb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c82fb6640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c0b9940 con 0x7f6c7c15dd30 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c82fb6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0d0d40 con 0x7f6c7c0d2e90 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c82fb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0d1df0 con 0x7f6c7c0d0900 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c80d52640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f6c7c15dd30 0x7f6c7c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44386/0 (socket says 172.21.15.148:44386) 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c7bfff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f6c7c0d2e90 0x7f6c7c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c80d52640 1 -- 172.21.15.148:0/3409208315 learned_addr learned my addr 172.21.15.148:0/3409208315 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:19.980 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c7bfff640 1 -- 172.21.15.148:0/3409208315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 msgr2=0x7f6c7c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c7bfff640 1 --2- 172.21.15.148:0/3409208315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c7bfff640 1 -- 172.21.15.148:0/3409208315 >> v1:172.21.15.148:6789/0 conn(0x7f6c7c15dd30 legacy=0x7f6c7c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c7bfff640 1 -- 172.21.15.148:0/3409208315 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c0d20d0 con 0x7f6c7c0d2e90 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.973+0000 7f6c7b7fe640 1 -- 172.21.15.148:0/3409208315 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498277374 0 0) 0x7f6c7c0b9940 con 0x7f6c7c15dd30 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.974+0000 7f6c7bfff640 1 --2- 172.21.15.148:0/3409208315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c0d03c0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009790 tx=0x7f6c7002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a497ee2ad2cd4dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.974+0000 7f6c7b7fe640 1 -- 172.21.15.148:0/3409208315 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70037070 con 0x7f6c7c0d2e90 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.974+0000 7f6c7b7fe640 1 -- 172.21.15.148:0/3409208315 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7002ae50 con 0x7f6c7c0d2e90 2024-04-25T10:08:19.981 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.974+0000 7f6c7b7fe640 1 -- 172.21.15.148:0/3409208315 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70032660 con 0x7f6c7c0d2e90 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 -- 172.21.15.148:0/3409208315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c0d2e90 msgr2=0x7f6c7c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/3409208315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c0d03c0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009790 tx=0x7f6c7002a100 comp rx=0 tx=0).stop 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 -- 172.21.15.148:0/3409208315 shutdown_connections 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/3409208315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/3409208315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c0d03c0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 -- 172.21.15.148:0/3409208315 >> 172.21.15.148:0/3409208315 conn(0x7f6c7c061c30 msgr2=0x7f6c7c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 -- 172.21.15.148:0/3409208315 shutdown_connections 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.975+0000 7f6c82fb6640 1 -- 172.21.15.148:0/3409208315 wait complete. 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.976+0000 7f6c82fb6640 1 Processor -- start 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.976+0000 7f6c82fb6640 1 -- start start 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c82fb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c82fb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.985 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c82fb6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 0x7f6c7c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c82fb6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0d2970 con 0x7f6c7c15dd30 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c82fb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0d2690 con 0x7f6c7c0d2e90 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c82fb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0d1a10 con 0x7f6c7c0d0900 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 0x7f6c7c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 0x7f6c7c0b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60382/0 (socket says 172.21.15.148:60382) 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 -- 172.21.15.148:0/315346551 learned_addr learned my addr 172.21.15.148:0/315346551 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c80d52640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c7bfff640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 -- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 msgr2=0x7f6c7c0b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 -- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c0d2e90 msgr2=0x7f6c7c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.986 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c15e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 -- 172.21.15.148:0/315346551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c15ebd0 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c80d52640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c7bfff640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.977+0000 7f6c81553640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 0x7f6c7c0b3150 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400b9a0 tx=0x7f6c7400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.978+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74019070 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.978+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7400cbc0 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.978+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c15eeb0 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.978+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74014440 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.978+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c15f470 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.979+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c073990 con 0x7f6c7c15dd30 2024-04-25T10:08:19.987 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.979+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c7401c070 con 0x7f6c7c15dd30 2024-04-25T10:08:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.979+0000 7f6c797fa640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c6806b4d0 0x7f6c6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.980+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6c74089e60 con 0x7f6c7c15dd30 2024-04-25T10:08:19.988 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.980+0000 7f6c80d52640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c6806b4d0 0x7f6c6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.980+0000 7f6c80d52640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c6806b4d0 0x7f6c6806d990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c000f60 tx=0x7f6c6c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.989 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:19.984+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c74058e00 con 0x7f6c7c15dd30 2024-04-25T10:08:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.151+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c7c160d50 con 0x7f6c6806b4d0 2024-04-25T10:08:20.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.151+0000 7f6c797fa640 1 -- 172.21.15.148:0/315346551 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13424 (secure 0 0 0) 0x7f6c7c160d50 con 0x7f6c6806b4d0 2024-04-25T10:08:20.157 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:20.158 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:08:20.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c6806b4d0 msgr2=0x7f6c6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c6806b4d0 0x7f6c6806d990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c000f60 tx=0x7f6c6c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 msgr2=0x7f6c7c0b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 0x7f6c7c0b3150 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400b9a0 tx=0x7f6c7400be60 comp rx=0 tx=0).stop 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 shutdown_connections 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f6c6806b4d0 0x7f6c6806d990 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c7c15dd30 0x7f6c7c0b3150 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c0d2e90 0x7f6c7c15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 --2- 172.21.15.148:0/315346551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0d0900 0x7f6c7c0b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 >> 172.21.15.148:0/315346551 conn(0x7f6c7c061c30 msgr2=0x7f6c7c0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 shutdown_connections 2024-04-25T10:08:20.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.153+0000 7f6c82fb6640 1 -- 172.21.15.148:0/315346551 wait complete. 2024-04-25T10:08:20.170 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-04-25T10:08:19.196935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494960,"kb_used_data":45712,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754479760,"statfs":{"total":773094113280,"available":772587274240,"internally_reserved":0,"allocated":46809088,"data_stored":35992012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":459918896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-35082,"num_keys_recovered":-1,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001106"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":505,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.570026+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:08:13.570026+0000","last_peered":"2024-04-25T10:08:13.570026+0000","last_clean":"2024-04-25T10:08:13.570026+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:08:13.570026+0000","last_undegraded":"2024-04-25T10:08:13.570026+0000","last_fullsized":"2024-04-25T10:08:13.570026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":7164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":7335936,"data_stored":5980663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55880,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2551808,"data_stored":1195461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55856,"kb_used_data":6628,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315984,"statfs":{"total":96636764160,"available":96579567616,"internally_reserved":0,"allocated":6787072,"data_stored":5430329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60032,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311808,"statfs":{"total":96636764160,"available":96575291392,"internally_reserved":0,"allocated":2543616,"data_stored":1195409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575021,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88256,"kb_used_data":12724,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94283584,"statfs":{"total":96636764160,"available":96546390016,"internally_reserved":0,"allocated":13029376,"data_stored":11681101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57924,"kb_used_data":4536,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313916,"statfs":{"total":96636764160,"available":96577449984,"internally_reserved":0,"allocated":4644864,"data_stored":3293103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57916,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313924,"statfs":{"total":96636764160,"available":96577458176,"internally_reserved":0,"allocated":4636672,"data_stored":3292471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":5156,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":5279744,"data_stored":3923475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:20.171 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:08:20.171 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:20.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.262+0000 7fdc9ddd5640 1 Processor -- start 2024-04-25T10:08:20.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.262+0000 7fdc9ddd5640 1 -- start start 2024-04-25T10:08:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc9ddd5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdc9815dc90 0x7fdc981625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc9ddd5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc980cc8e0 0x7fdc9815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc9ddd5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9815f390 con 0x7fdc980d1810 2024-04-25T10:08:20.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc9ddd5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980cce50 con 0x7fdc9815dc90 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc9ddd5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc98161af0 con 0x7fdc980cc8e0 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc977fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdc9815dc90 0x7fdc981625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc977fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fdc9815dc90 0x7fdc981625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60386/0 (socket says 172.21.15.148:60386) 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc977fe640 1 -- 172.21.15.148:0/4191878256 learned_addr learned my addr 172.21.15.148:0/4191878256 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc977fe640 1 -- 172.21.15.148:0/4191878256 >> v2:172.21.15.188:3300/0 conn(0x7fdc980cc8e0 msgr2=0x7fdc9815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.263+0000 7fdc96ffd640 1 --2- 172.21.15.148:0/4191878256 >> v2:172.21.15.188:3300/0 conn(0x7fdc980cc8e0 0x7fdc9815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc977fe640 1 --2- 172.21.15.148:0/4191878256 >> v2:172.21.15.188:3300/0 conn(0x7fdc980cc8e0 0x7fdc9815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc977fe640 1 -- 172.21.15.148:0/4191878256 >> v1:172.21.15.188:6789/0 conn(0x7fdc980d1810 legacy=0x7fdc980cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc977fe640 1 -- 172.21.15.148:0/4191878256 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98161dd0 con 0x7fdc9815dc90 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc96ffd640 1 --2- 172.21.15.148:0/4191878256 >> v2:172.21.15.188:3300/0 conn(0x7fdc980cc8e0 0x7fdc9815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc977fe640 1 --2- 172.21.15.148:0/4191878256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc981625d0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7fdc980c88a0 tx=0x7fdc8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214e223a1c1a383a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc967fc640 1 -- 172.21.15.148:0/4191878256 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88037070 con 0x7fdc9815dc90 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc967fc640 1 -- 172.21.15.148:0/4191878256 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88032440 con 0x7fdc9815dc90 2024-04-25T10:08:20.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.264+0000 7fdc967fc640 1 -- 172.21.15.148:0/4191878256 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88032810 con 0x7fdc9815dc90 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4191878256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 msgr2=0x7fdc981625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4191878256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc981625d0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7fdc980c88a0 tx=0x7fdc8802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4191878256 shutdown_connections 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4191878256 >> v2:172.21.15.188:3300/0 conn(0x7fdc980cc8e0 0x7fdc9815ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4191878256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc981625d0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4191878256 >> 172.21.15.148:0/4191878256 conn(0x7fdc9806f590 msgr2=0x7fdc9806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4191878256 shutdown_connections 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.265+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4191878256 wait complete. 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.266+0000 7fdc9ddd5640 1 Processor -- start 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.266+0000 7fdc9ddd5640 1 -- start start 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc9ddd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc980cc8e0 0x7fdc980b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc9ddd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc980d1810 0x7fdc98155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc9ddd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc980cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc9ddd5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9815e070 con 0x7fdc9815dc90 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc9ddd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980cd130 con 0x7fdc980cc8e0 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc9ddd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9815e1f0 con 0x7fdc980d1810 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc980cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc980cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60400/0 (socket says 172.21.15.148:60400) 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 -- 172.21.15.148:0/4209965624 learned_addr learned my addr 172.21.15.148:0/4209965624 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc96ffd640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc980d1810 0x7fdc98155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc977fe640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc980cc8e0 0x7fdc980b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 -- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc980d1810 msgr2=0x7fdc98155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc980d1810 0x7fdc98155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 -- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc980cc8e0 msgr2=0x7fdc980b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc980cc8e0 0x7fdc980b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.267+0000 7fdc97fff640 1 -- 172.21.15.148:0/4209965624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980b71d0 con 0x7fdc9815dc90 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.268+0000 7fdc97fff640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc980cb390 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c780 tx=0x7fdc9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.268+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90017070 con 0x7fdc9815dc90 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.268+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90003da0 con 0x7fdc9815dc90 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.268+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc980b5a30 con 0x7fdc9815dc90 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.268+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900052b0 con 0x7fdc9815dc90 2024-04-25T10:08:20.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.268+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980b5ff0 con 0x7fdc9815dc90 2024-04-25T10:08:20.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.269+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9806b0a0 con 0x7fdc9815dc90 2024-04-25T10:08:20.279 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.269+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc9001a070 con 0x7fdc9815dc90 2024-04-25T10:08:20.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.270+0000 7fdc7ffff640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc8406b500 0x7fdc8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.270+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdc900896c0 con 0x7fdc9815dc90 2024-04-25T10:08:20.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.270+0000 7fdc977fe640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc8406b500 0x7fdc8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.270+0000 7fdc977fe640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc8406b500 0x7fdc8406d9c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdc88034000 tx=0x7fdc8802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.280 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.274+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc9806b0a0 con 0x7fdc9815dc90 2024-04-25T10:08:20.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.443+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc98073990 con 0x7fdc8406b500 2024-04-25T10:08:20.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.443+0000 7fdc7ffff640 1 -- 172.21.15.148:0/4209965624 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13424 (secure 0 0 0) 0x7fdc88045480 con 0x7fdc8406b500 2024-04-25T10:08:20.449 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:20.450 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc8406b500 msgr2=0x7fdc8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc8406b500 0x7fdc8406d9c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fdc88034000 tx=0x7fdc8802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 msgr2=0x7fdc980cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc980cb390 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c780 tx=0x7fdc9000cc40 comp rx=0 tx=0).stop 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 shutdown_connections 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fdc8406b500 0x7fdc8406d9c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc9815dc90 0x7fdc980cb390 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc980d1810 0x7fdc98155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 --2- 172.21.15.148:0/4209965624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc980cc8e0 0x7fdc980b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.445+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 >> 172.21.15.148:0/4209965624 conn(0x7fdc9806f590 msgr2=0x7fdc980b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.446+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 shutdown_connections 2024-04-25T10:08:20.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.446+0000 7fdc9ddd5640 1 -- 172.21.15.148:0/4209965624 wait complete. 2024-04-25T10:08:20.462 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":621,"stamp":"2024-04-25T10:08:19.196935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494960,"kb_used_data":45712,"kb_used_omap":12,"kb_used_meta":449139,"kb_avail":754479760,"statfs":{"total":773094113280,"available":772587274240,"internally_reserved":0,"allocated":46809088,"data_stored":35992012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":459918896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624362,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-97,"num_write_kb":-12362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6,"num_bytes_recovered":-35082,"num_keys_recovered":-1,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001106"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":505,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:08:13.570026+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:08:13.570026+0000","last_peered":"2024-04-25T10:08:13.570026+0000","last_clean":"2024-04-25T10:08:13.570026+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:08:13.570026+0000","last_undegraded":"2024-04-25T10:08:13.570026+0000","last_fullsized":"2024-04-25T10:08:13.570026+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60552,"kb_used_data":7164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94311288,"statfs":{"total":96636764160,"available":96574758912,"internally_reserved":0,"allocated":7335936,"data_stored":5980663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55880,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2551808,"data_stored":1195461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575021,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55856,"kb_used_data":6628,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315984,"statfs":{"total":96636764160,"available":96579567616,"internally_reserved":0,"allocated":6787072,"data_stored":5430329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60032,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311808,"statfs":{"total":96636764160,"available":96575291392,"internally_reserved":0,"allocated":2543616,"data_stored":1195409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575021,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88256,"kb_used_data":12724,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94283584,"statfs":{"total":96636764160,"available":96546390016,"internally_reserved":0,"allocated":13029376,"data_stored":11681101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57924,"kb_used_data":4536,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313916,"statfs":{"total":96636764160,"available":96577449984,"internally_reserved":0,"allocated":4644864,"data_stored":3293103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57916,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313924,"statfs":{"total":96636764160,"available":96577458176,"internally_reserved":0,"allocated":4636672,"data_stored":3292471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58544,"kb_used_data":5156,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313296,"statfs":{"total":96636764160,"available":96576815104,"internally_reserved":0,"allocated":5279744,"data_stored":3923475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:20.463 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:20.560 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.554+0000 7feb890bd640 1 Processor -- start 2024-04-25T10:08:20.561 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.554+0000 7feb890bd640 1 -- start start 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb890bd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb8415dbe0 0x7feb840d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb890bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b92a0 0x7feb840d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb890bd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb840b97a0 con 0x7feb840b2cb0 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb890bd640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8415dfc0 con 0x7feb8415dbe0 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb890bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb840d1a00 con 0x7feb840b92a0 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb82d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb8415dbe0 0x7feb840d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb82d9d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7feb8415dbe0 0x7feb840d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60410/0 (socket says 172.21.15.148:60410) 2024-04-25T10:08:20.563 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.555+0000 7feb82d9d640 1 -- 172.21.15.148:0/2616526684 learned_addr learned my addr 172.21.15.148:0/2616526684 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb82d9d640 1 -- 172.21.15.148:0/2616526684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b92a0 msgr2=0x7feb840d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb8359e640 1 --2- 172.21.15.148:0/2616526684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b92a0 0x7feb840d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb82d9d640 1 --2- 172.21.15.148:0/2616526684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b92a0 0x7feb840d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb82d9d640 1 -- 172.21.15.148:0/2616526684 >> v1:172.21.15.188:6789/0 conn(0x7feb840b2cb0 legacy=0x7feb840b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb82d9d640 1 -- 172.21.15.148:0/2616526684 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840d1c50 con 0x7feb8415dbe0 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb82d9d640 1 --2- 172.21.15.148:0/2616526684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb840d0910 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7feb74009870 tx=0x7feb7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c77c8b46655b6b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb8259c640 1 -- 172.21.15.148:0/2616526684 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74037070 con 0x7feb8415dbe0 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb8259c640 1 -- 172.21.15.148:0/2616526684 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb74032440 con 0x7feb8415dbe0 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.556+0000 7feb8259c640 1 -- 172.21.15.148:0/2616526684 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74032810 con 0x7feb8415dbe0 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 -- 172.21.15.148:0/2616526684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 msgr2=0x7feb840d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 --2- 172.21.15.148:0/2616526684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb840d0910 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7feb74009870 tx=0x7feb7402a210 comp rx=0 tx=0).stop 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 -- 172.21.15.148:0/2616526684 shutdown_connections 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 --2- 172.21.15.148:0/2616526684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b92a0 0x7feb840d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.564 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 --2- 172.21.15.148:0/2616526684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb840d0910 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 -- 172.21.15.148:0/2616526684 >> 172.21.15.148:0/2616526684 conn(0x7feb84061a20 msgr2=0x7feb84061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 -- 172.21.15.148:0/2616526684 shutdown_connections 2024-04-25T10:08:20.565 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.557+0000 7feb890bd640 1 -- 172.21.15.148:0/2616526684 wait complete. 2024-04-25T10:08:20.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.558+0000 7feb890bd640 1 Processor -- start 2024-04-25T10:08:20.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.558+0000 7feb890bd640 1 -- start start 2024-04-25T10:08:20.567 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb890bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b2cb0 0x7feb840c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb890bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb840b92a0 0x7feb8415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb890bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb8415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb890bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb840b6ab0 con 0x7feb8415dbe0 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb890bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb840d2210 con 0x7feb840b92a0 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb890bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb840b6c30 con 0x7feb840b2cb0 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb8415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb8415f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60424/0 (socket says 172.21.15.148:60424) 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 -- 172.21.15.148:0/61113144 learned_addr learned my addr 172.21.15.148:0/61113144 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb82d9d640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b2cb0 0x7feb840c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb7bfff640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb840b92a0 0x7feb8415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 -- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b2cb0 msgr2=0x7feb840c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b2cb0 0x7feb840c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 -- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb840b92a0 msgr2=0x7feb8415f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.568 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.559+0000 7feb8359e640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb840b92a0 0x7feb8415f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb8359e640 1 -- 172.21.15.148:0/61113144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840b3f80 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb8359e640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb8415f810 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00cd70 tx=0x7feb7c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c003b90 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c003d30 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb840b4260 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c004240 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.560+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb840b4820 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.561+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb84073990 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.561+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb7c0043e0 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.562+0000 7feb7b7fe640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb6406b4d0 0x7feb6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.562+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7feb7c08aa30 con 0x7feb8415dbe0 2024-04-25T10:08:20.569 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.562+0000 7feb82d9d640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb6406b4d0 0x7feb6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.562+0000 7feb82d9d640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb6406b4d0 0x7feb6406d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7feb74034000 tx=0x7feb7402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.571 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.566+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb7c059410 con 0x7feb8415dbe0 2024-04-25T10:08:20.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.738+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb84155300 con 0x7feb8415dbe0 2024-04-25T10:08:20.744 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.739+0000 7feb7b7fe640 1 -- 172.21.15.148:0/61113144 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+13485 (secure 0 0 0) 0x7feb7c05d4b0 con 0x7feb8415dbe0 2024-04-25T10:08:20.744 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:20.745 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":203,"fsid":"086ed9ea-68e4-4e62-83c5-c0821c62dd17","created":"2024-04-25T09:53:58.918637+0000","modified":"2024-04-25T10:08:16.683569+0000","last_up_change":"2024-04-25T09:54:09.969496+0000","last_in_change":"2024-04-25T09:54:01.064025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T09:54:11.048758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"99e54694-af1a-42d6-bdcf-6f34b9ecb4d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6817","nonce":2808668116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6820","nonce":2808668116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6828","nonce":2808668116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2808668116},{"type":"v1","addr":"172.21.15.148:6824","nonce":2808668116}]},"public_addr":"172.21.15.148:6817/2808668116","cluster_addr":"172.21.15.148:6820/2808668116","heartbeat_back_addr":"172.21.15.148:6828/2808668116","heartbeat_front_addr":"172.21.15.148:6824/2808668116","state":["exists","up"]},{"osd":1,"uuid":"833ab044-9d43-4c94-88b5-db5debf42a89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6801","nonce":3547190582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6803","nonce":3547190582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6807","nonce":3547190582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3547190582},{"type":"v1","addr":"172.21.15.148:6805","nonce":3547190582}]},"public_addr":"172.21.15.148:6801/3547190582","cluster_addr":"172.21.15.148:6803/3547190582","heartbeat_back_addr":"172.21.15.148:6807/3547190582","heartbeat_front_addr":"172.21.15.148:6805/3547190582","state":["exists","up"]},{"osd":2,"uuid":"976d340c-b8e6-4f86-805f-7402637d96e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6809","nonce":1311261585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6811","nonce":1311261585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6815","nonce":1311261585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":1311261585},{"type":"v1","addr":"172.21.15.148:6813","nonce":1311261585}]},"public_addr":"172.21.15.148:6809/1311261585","cluster_addr":"172.21.15.148:6811/1311261585","heartbeat_back_addr":"172.21.15.148:6815/1311261585","heartbeat_front_addr":"172.21.15.148:6813/1311261585","state":["exists","up"]},{"osd":3,"uuid":"0b8d5dda-2380-4190-95e8-3084f6a71da4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6819","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6821","nonce":1795140808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6823","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6825","nonce":1795140808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6831","nonce":1795140808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6827","nonce":1795140808},{"type":"v1","addr":"172.21.15.148:6829","nonce":1795140808}]},"public_addr":"172.21.15.148:6821/1795140808","cluster_addr":"172.21.15.148:6825/1795140808","heartbeat_back_addr":"172.21.15.148:6831/1795140808","heartbeat_front_addr":"172.21.15.148:6829/1795140808","state":["exists","up"]},{"osd":4,"uuid":"05f1febd-ccd2-4ec3-9e25-e8cbf5569152","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6801","nonce":3303846211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6803","nonce":3303846211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6807","nonce":3303846211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3303846211},{"type":"v1","addr":"172.21.15.188:6805","nonce":3303846211}]},"public_addr":"172.21.15.188:6801/3303846211","cluster_addr":"172.21.15.188:6803/3303846211","heartbeat_back_addr":"172.21.15.188:6807/3303846211","heartbeat_front_addr":"172.21.15.188:6805/3303846211","state":["exists","up"]},{"osd":5,"uuid":"4f810a22-a237-42ea-af28-df38627332fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6809","nonce":3945194105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6811","nonce":3945194105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6815","nonce":3945194105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":3945194105},{"type":"v1","addr":"172.21.15.188:6813","nonce":3945194105}]},"public_addr":"172.21.15.188:6809/3945194105","cluster_addr":"172.21.15.188:6811/3945194105","heartbeat_back_addr":"172.21.15.188:6815/3945194105","heartbeat_front_addr":"172.21.15.188:6813/3945194105","state":["exists","up"]},{"osd":6,"uuid":"9dbe47fa-4206-44be-8ac0-908c472bff8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6817","nonce":1362851547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6819","nonce":1362851547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6823","nonce":1362851547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":1362851547},{"type":"v1","addr":"172.21.15.188:6821","nonce":1362851547}]},"public_addr":"172.21.15.188:6817/1362851547","cluster_addr":"172.21.15.188:6819/1362851547","heartbeat_back_addr":"172.21.15.188:6823/1362851547","heartbeat_front_addr":"172.21.15.188:6821/1362851547","state":["exists","up"]},{"osd":7,"uuid":"8be29075-43eb-44ff-8f48-96316cff73f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6825","nonce":678316061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6827","nonce":678316061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6831","nonce":678316061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":678316061},{"type":"v1","addr":"172.21.15.188:6829","nonce":678316061}]},"public_addr":"172.21.15.188:6825/678316061","cluster_addr":"172.21.15.188:6827/678316061","heartbeat_back_addr":"172.21.15.188:6831/678316061","heartbeat_front_addr":"172.21.15.188:6829/678316061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.246700+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.236735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.266503+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.186269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.178741+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.282927+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.287926+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T09:54:06.269466+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/1465938320":"2024-04-26T10:08:13.350440+0000","172.21.15.188:6835/1255014956":"2024-04-26T10:07:02.533393+0000","172.21.15.148:6837/1465938320":"2024-04-26T10:08:13.350440+0000","172.21.15.188:6833/1255014956":"2024-04-26T10:07:02.533393+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.740+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb6406b4d0 msgr2=0x7feb6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.740+0000 7feb890bd640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb6406b4d0 0x7feb6406d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7feb74034000 tx=0x7feb7402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.740+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 msgr2=0x7feb8415f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.740+0000 7feb890bd640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb8415f810 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00cd70 tx=0x7feb7c00a660 comp rx=0 tx=0).stop 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 shutdown_connections 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7feb6406b4d0 0x7feb6406d990 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb8415dbe0 0x7feb8415f810 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb840b92a0 0x7feb8415f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 --2- 172.21.15.148:0/61113144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb840b2cb0 0x7feb840c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 >> 172.21.15.148:0/61113144 conn(0x7feb84061a20 msgr2=0x7feb840d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 shutdown_connections 2024-04-25T10:08:20.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:20.741+0000 7feb890bd640 1 -- 172.21.15.148:0/61113144 wait complete. 2024-04-25T10:08:21.758 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:08:21.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:21.869 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.863+0000 7fd4230b3640 1 Processor -- start 2024-04-25T10:08:21.870 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.864+0000 7fd4230b3640 1 -- start start 2024-04-25T10:08:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.864+0000 7fd4230b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.864+0000 7fd4230b3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd41c15b270 0x7fd41c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.871 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.864+0000 7fd4230b3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c072280 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.864+0000 7fd4230b3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c071270 con 0x7fd41c15b270 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.864+0000 7fd4230b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0714c0 con 0x7fd41c0ce230 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd421650640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd41c15b270 0x7fd41c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:40860/0 (socket says 172.21.15.148:40860) 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 -- 172.21.15.148:0/3315515120 learned_addr learned my addr 172.21.15.148:0/3315515120 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41b7fe640 1 -- 172.21.15.148:0/3315515120 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705281607 0 0) 0x7fd41c072280 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 -- 172.21.15.148:0/3315515120 >> v2:172.21.15.188:3300/0 conn(0x7fd41c15b270 msgr2=0x7fd41c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 --2- 172.21.15.148:0/3315515120 >> v2:172.21.15.188:3300/0 conn(0x7fd41c15b270 0x7fd41c15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 -- 172.21.15.148:0/3315515120 >> v1:172.21.15.188:6789/0 conn(0x7fd41c0cc4f0 legacy=0x7fd41c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.872 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.865+0000 7fd41bfff640 1 -- 172.21.15.148:0/3315515120 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0717a0 con 0x7fd41c0ce230 2024-04-25T10:08:21.873 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.866+0000 7fd41bfff640 1 --2- 172.21.15.148:0/3315515120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0cc8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd408009870 tx=0x7fd40802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f48be49c4914f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.866+0000 7fd41b7fe640 1 -- 172.21.15.148:0/3315515120 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408037070 con 0x7fd41c0ce230 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.866+0000 7fd41b7fe640 1 -- 172.21.15.148:0/3315515120 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408032440 con 0x7fd41c0ce230 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.866+0000 7fd41b7fe640 1 -- 172.21.15.148:0/3315515120 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408032810 con 0x7fd41c0ce230 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 -- 172.21.15.148:0/3315515120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 msgr2=0x7fd41c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 --2- 172.21.15.148:0/3315515120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0cc8d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd408009870 tx=0x7fd40802a270 comp rx=0 tx=0).stop 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 -- 172.21.15.148:0/3315515120 shutdown_connections 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 --2- 172.21.15.148:0/3315515120 >> v2:172.21.15.188:3300/0 conn(0x7fd41c15b270 0x7fd41c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 --2- 172.21.15.148:0/3315515120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0cc8d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 -- 172.21.15.148:0/3315515120 >> 172.21.15.148:0/3315515120 conn(0x7fd41c06abe0 msgr2=0x7fd41c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 -- 172.21.15.148:0/3315515120 shutdown_connections 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.867+0000 7fd4230b3640 1 -- 172.21.15.148:0/3315515120 wait complete. 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.868+0000 7fd4230b3640 1 Processor -- start 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.868+0000 7fd4230b3640 1 -- start start 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd4230b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c0cc4f0 0x7fd41c1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd4230b3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd4230b3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c15b270 0x7fd41c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd4230b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c15bc00 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd4230b3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c071d60 con 0x7fd41c15b270 2024-04-25T10:08:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd4230b3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c15bd80 con 0x7fd41c0ce230 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c0cc4f0 0x7fd41c1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c0cc4f0 0x7fd41c1572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60432/0 (socket says 172.21.15.148:60432) 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 -- 172.21.15.148:0/3481096894 learned_addr learned my addr 172.21.15.148:0/3481096894 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd421650640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c15b270 0x7fd41c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd41bfff640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 -- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 msgr2=0x7fd41c0c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd41c0ce230 0x7fd41c0c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.881 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 -- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c15b270 msgr2=0x7fd41c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c15b270 0x7fd41c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 -- 172.21.15.148:0/3481096894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c151fa0 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd421650640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd41c15b270 0x7fd41c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.869+0000 7fd420e4f640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c0cc4f0 0x7fd41c1572c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c440 tx=0x7fd41000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.870+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410017070 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.870+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410005480 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.870+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c152280 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.870+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410003830 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.870+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c152840 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.870+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd41c073a00 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.882 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.871+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd41001a070 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.871+0000 7fd4197fa640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40006b4d0 0x7fd40006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd410089330 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd41bfff640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40006b4d0 0x7fd40006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd4197fa640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7fd400071a10 0x7fd400073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 --> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd400074520 con 0x7fd400071a10 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd410090050 con 0x7fd41c0cc4f0 2024-04-25T10:08:21.883 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd421650640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7fd400071a10 0x7fd400073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd41bfff640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40006b4d0 0x7fd40006d990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd408034000 tx=0x7fd40802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.872+0000 7fd421650640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7fd400071a10 0x7fd400073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.884 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.873+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== osd.0 v2:172.21.15.148:6816/2808668116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd400074520 con 0x7fd400071a10 2024-04-25T10:08:21.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.886+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 --> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd41c06d3a0 con 0x7fd400071a10 2024-04-25T10:08:21.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.886+0000 7fd4197fa640 1 -- 172.21.15.148:0/3481096894 <== osd.0 v2:172.21.15.148:6816/2808668116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd41c06d3a0 con 0x7fd400071a10 2024-04-25T10:08:21.892 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:21.893 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:21.893 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7fd400071a10 msgr2=0x7fd400073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6816/2808668116,v1:172.21.15.148:6817/2808668116] conn(0x7fd400071a10 0x7fd400073e50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40006b4d0 msgr2=0x7fd40006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd40006b4d0 0x7fd40006d990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd408034000 tx=0x7fd40802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c0cc4f0 msgr2=0x7fd41c1572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 --2- 172.21.15.148:0/3481096894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41c0cc4f0 0x7fd41c1572c0 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fd41000c440 tx=0x7fd41000c900 comp rx=0 tx=0).stop 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd421650640 1 -- 172.21.15.148:0/3481096894 reap_dead start 2024-04-25T10:08:21.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 shutdown_connections 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 >> 172.21.15.148:0/3481096894 conn(0x7fd41c06abe0 msgr2=0x7fd41c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 shutdown_connections 2024-04-25T10:08:21.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.887+0000 7fd4230b3640 1 -- 172.21.15.148:0/3481096894 wait complete. 2024-04-25T10:08:21.904 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:08:22.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.998+0000 7fbb57ab5640 1 Processor -- start 2024-04-25T10:08:22.004 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.998+0000 7fbb57ab5640 1 -- start start 2024-04-25T10:08:22.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb57ab5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb500cadb0 0x7fbb500cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb57ab5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb500cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb57ab5640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb500afc80 con 0x7fbb5015de80 2024-04-25T10:08:22.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb57ab5640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb500cb240 con 0x7fbb500cadb0 2024-04-25T10:08:22.006 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb57ab5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500ce4d0 con 0x7fbb500ccfe0 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb55050640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb500cadb0 0x7fbb500cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb55050640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fbb500cadb0 0x7fbb500cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60440/0 (socket says 172.21.15.148:60440) 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:21.999+0000 7fbb55050640 1 -- 172.21.15.148:0/1002956530 learned_addr learned my addr 172.21.15.148:0/1002956530 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb56052640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb500cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb55050640 1 -- 172.21.15.148:0/1002956530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 msgr2=0x7fbb500cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb55050640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb500cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb55050640 1 -- 172.21.15.148:0/1002956530 >> v1:172.21.15.148:6789/0 conn(0x7fbb5015de80 legacy=0x7fbb500cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb55050640 1 -- 172.21.15.148:0/1002956530 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500ce7b0 con 0x7fbb500cadb0 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb56052640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb500cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.000+0000 7fbb55050640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500cf6c0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009870 tx=0x7fbb4002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d8215e958db94b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.007 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb5484f640 1 -- 172.21.15.148:0/1002956530 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40037070 con 0x7fbb500cadb0 2024-04-25T10:08:22.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb5484f640 1 -- 172.21.15.148:0/1002956530 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb40032440 con 0x7fbb500cadb0 2024-04-25T10:08:22.008 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb5484f640 1 -- 172.21.15.148:0/1002956530 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40032810 con 0x7fbb500cadb0 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 -- 172.21.15.148:0/1002956530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 msgr2=0x7fbb500cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500cf6c0 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009870 tx=0x7fbb4002a250 comp rx=0 tx=0).stop 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 -- 172.21.15.148:0/1002956530 shutdown_connections 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb500cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/1002956530 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500cf6c0 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 -- 172.21.15.148:0/1002956530 >> 172.21.15.148:0/1002956530 conn(0x7fbb500b2390 msgr2=0x7fbb500b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.012 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 -- 172.21.15.148:0/1002956530 shutdown_connections 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.001+0000 7fbb57ab5640 1 -- 172.21.15.148:0/1002956530 wait complete. 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 Processor -- start 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 -- start start 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb501f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5015de80 0x7fbb500b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5015e260 con 0x7fbb500cadb0 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500cfc00 con 0x7fbb5015de80 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb57ab5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500b2eb0 con 0x7fbb500ccfe0 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb55851640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb55851640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60456/0 (socket says 172.21.15.148:60456) 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.003+0000 7fbb55851640 1 -- 172.21.15.148:0/521102342 learned_addr learned my addr 172.21.15.148:0/521102342 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55050640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb501f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.013 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb56052640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5015de80 0x7fbb500b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55851640 1 -- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 msgr2=0x7fbb501f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55851640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb501f3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55851640 1 -- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5015de80 msgr2=0x7fbb500b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55851640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5015de80 0x7fbb500b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55851640 1 -- 172.21.15.148:0/521102342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500afe50 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb55851640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500c8240 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c400 tx=0x7fbb4c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c0052d0 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c005490 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb501f4320 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c007260 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.004+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb501f48e0 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.005+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb50073a00 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.006+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb4c007400 con 0x7fbb500cadb0 2024-04-25T10:08:22.014 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.006+0000 7fbb467fc640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb3406b4d0 0x7fbb3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.006+0000 7fbb55050640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb3406b4d0 0x7fbb3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.006+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbb4c0895a0 con 0x7fbb500cadb0 2024-04-25T10:08:22.015 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.007+0000 7fbb55050640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb3406b4d0 0x7fbb3406d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.010+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb4c0585d0 con 0x7fbb500cadb0 2024-04-25T10:08:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.189+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb500cc770 con 0x7fbb3406b4d0 2024-04-25T10:08:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.189+0000 7fbb467fc640 1 -- 172.21.15.148:0/521102342 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbb500cc770 con 0x7fbb3406b4d0 2024-04-25T10:08:22.195 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.191+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb3406b4d0 msgr2=0x7fbb3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.191+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb3406b4d0 0x7fbb3406d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.191+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 msgr2=0x7fbb500c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.191+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500c8240 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c400 tx=0x7fbb4c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 shutdown_connections 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fbb3406b4d0 0x7fbb3406d990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb5015de80 0x7fbb500b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb500ccfe0 0x7fbb501f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 --2- 172.21.15.148:0/521102342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb500cadb0 0x7fbb500c8240 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 >> 172.21.15.148:0/521102342 conn(0x7fbb500b2390 msgr2=0x7fbb500b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 shutdown_connections 2024-04-25T10:08:22.198 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.192+0000 7fbb57ab5640 1 -- 172.21.15.148:0/521102342 wait complete. 2024-04-25T10:08:22.209 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:08:22.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:22.312 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 Processor -- start 2024-04-25T10:08:22.313 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 -- start start 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a2c15dc20 0x7f2a2c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c15e000 con 0x7f2a2c161710 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c0d1de0 con 0x7f2a2c15dc20 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.307+0000 7f2a30c07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c161c60 con 0x7f2a2c0d32f0 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2a59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a2c15dc20 0x7f2a2c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.314 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:40906/0 (socket says 172.21.15.148:40906) 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 -- 172.21.15.148:0/1618564151 learned_addr learned my addr 172.21.15.148:0/1618564151 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 -- 172.21.15.148:0/1618564151 >> v2:172.21.15.188:3300/0 conn(0x7f2a2c15dc20 msgr2=0x7f2a2c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 --2- 172.21.15.148:0/1618564151 >> v2:172.21.15.188:3300/0 conn(0x7f2a2c15dc20 0x7f2a2c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 -- 172.21.15.148:0/1618564151 >> v1:172.21.15.188:6789/0 conn(0x7f2a2c161710 legacy=0x7f2a2c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.315 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.308+0000 7f2a2ad9d640 1 -- 172.21.15.148:0/1618564151 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c161f40 con 0x7f2a2c0d32f0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.309+0000 7f2a2ad9d640 1 --2- 172.21.15.148:0/1618564151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0cebb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009e50 tx=0x7f2a2402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ab60d6f81a54ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.309+0000 7f2a29d9b640 1 -- 172.21.15.148:0/1618564151 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24036070 con 0x7f2a2c0d32f0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.309+0000 7f2a29d9b640 1 -- 172.21.15.148:0/1618564151 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24031600 con 0x7f2a2c0d32f0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.309+0000 7f2a29d9b640 1 -- 172.21.15.148:0/1618564151 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240319d0 con 0x7f2a2c0d32f0 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 -- 172.21.15.148:0/1618564151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 msgr2=0x7f2a2c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.316 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 --2- 172.21.15.148:0/1618564151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0cebb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009e50 tx=0x7f2a2402a710 comp rx=0 tx=0).stop 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 -- 172.21.15.148:0/1618564151 shutdown_connections 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 --2- 172.21.15.148:0/1618564151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0cebb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 --2- 172.21.15.148:0/1618564151 >> v2:172.21.15.188:3300/0 conn(0x7f2a2c15dc20 0x7f2a2c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 -- 172.21.15.148:0/1618564151 >> 172.21.15.148:0/1618564151 conn(0x7f2a2c0d0420 msgr2=0x7f2a2c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 -- 172.21.15.148:0/1618564151 shutdown_connections 2024-04-25T10:08:22.317 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.310+0000 7f2a30c07640 1 -- 172.21.15.148:0/1618564151 wait complete. 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.311+0000 7f2a30c07640 1 Processor -- start 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.311+0000 7f2a30c07640 1 -- start start 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a30c07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a30c07640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a2c15dc20 0x7f2a2c0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a30c07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c161710 0x7f2a2c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a30c07640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0b5e50 con 0x7f2a2c15dc20 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a30c07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c162500 con 0x7f2a2c161710 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a30c07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0b5fd0 con 0x7f2a2c0d32f0 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a2c15dc20 0x7f2a2c0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a2c15dc20 0x7f2a2c0ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60468/0 (socket says 172.21.15.148:60468) 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 -- 172.21.15.148:0/496521639 learned_addr learned my addr 172.21.15.148:0/496521639 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a2a59c640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a2ad9d640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c161710 0x7f2a2c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 -- 172.21.15.148:0/496521639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 msgr2=0x7f2a2c0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0d32f0 0x7f2a2c0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 -- 172.21.15.148:0/496521639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c161710 msgr2=0x7f2a2c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c161710 0x7f2a2c0cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 -- 172.21.15.148:0/496521639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0ccf90 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.312+0000 7f2a23fff640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a2c15dc20 0x7f2a2c0ad100 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c3c0 tx=0x7f2a1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.313+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a140052d0 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.313+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a14005490 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.313+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c0cd270 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.313+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14007260 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.313+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c15ce90 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.314+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a2c14cb30 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.314+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a14007400 con 0x7f2a2c15dc20 2024-04-25T10:08:22.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.314+0000 7f2a22ffd640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a0c06b4d0 0x7f2a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2a140894d0 con 0x7f2a2c15dc20 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a2a59c640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a0c06b4d0 0x7f2a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a22ffd640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f2a0c071a10 0x7f2a0c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 --> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a1401dec0 con 0x7f2a0c071a10 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a14090050 con 0x7f2a2c15dc20 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a2ad9d640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f2a0c071a10 0x7f2a0c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a2a59c640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a0c06b4d0 0x7f2a0c06d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009aa0 tx=0x7f2a1c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.315+0000 7f2a2ad9d640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f2a0c071a10 0x7f2a0c073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.316+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== osd.1 v2:172.21.15.148:6800/3547190582 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2a1401dec0 con 0x7f2a0c071a10 2024-04-25T10:08:22.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.343+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 --> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a2c073a00 con 0x7f2a0c071a10 2024-04-25T10:08:22.349 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.343+0000 7f2a22ffd640 1 -- 172.21.15.148:0/496521639 <== osd.1 v2:172.21.15.148:6800/3547190582 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2a2c073a00 con 0x7f2a0c071a10 2024-04-25T10:08:22.349 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:22.349 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:22.349 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f2a0c071a10 msgr2=0x7f2a0c073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6800/3547190582,v1:172.21.15.148:6801/3547190582] conn(0x7f2a0c071a10 0x7f2a0c073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a0c06b4d0 msgr2=0x7f2a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f2a0c06b4d0 0x7f2a0c06d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009aa0 tx=0x7f2a1c009190 comp rx=0 tx=0).stop 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a2c15dc20 msgr2=0x7f2a2c0ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 --2- 172.21.15.148:0/496521639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a2c15dc20 0x7f2a2c0ad100 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c3c0 tx=0x7f2a1400c880 comp rx=0 tx=0).stop 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a2ad9d640 1 -- 172.21.15.148:0/496521639 reap_dead start 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 shutdown_connections 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 >> 172.21.15.148:0/496521639 conn(0x7f2a2c0d0420 msgr2=0x7f2a2c0d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 shutdown_connections 2024-04-25T10:08:22.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.344+0000 7f2a30c07640 1 -- 172.21.15.148:0/496521639 wait complete. 2024-04-25T10:08:22.360 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:08:22.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.455+0000 7fa4029c1640 1 Processor -- start 2024-04-25T10:08:22.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.455+0000 7fa4029c1640 1 -- start start 2024-04-25T10:08:22.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa4029c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3fc15dc20 0x7fa3fc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa4029c1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3fc0d32f0 0x7fa3fc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.464 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa4029c1640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc15e000 con 0x7fa3fc161710 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa4029c1640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc0d1de0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa4029c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc161c60 con 0x7fa3fc15dc20 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa400f5e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fa3fc0d32f0 0x7fa3fc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa3fb7fe640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fa3fc161710 0x7fa3fc06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44406/0 (socket says 172.21.15.148:44406) 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa3fb7fe640 1 -- 172.21.15.148:0/3433962386 learned_addr learned my addr 172.21.15.148:0/3433962386 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.456+0000 7fa3fbfff640 1 --2- 172.21.15.148:0/3433962386 >> v2:172.21.15.188:3300/0 conn(0x7fa3fc15dc20 0x7fa3fc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa400f5e640 1 -- 172.21.15.148:0/3433962386 >> v2:172.21.15.188:3300/0 conn(0x7fa3fc15dc20 msgr2=0x7fa3fc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa400f5e640 1 --2- 172.21.15.148:0/3433962386 >> v2:172.21.15.188:3300/0 conn(0x7fa3fc15dc20 0x7fa3fc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa400f5e640 1 -- 172.21.15.148:0/3433962386 >> v1:172.21.15.148:6789/0 conn(0x7fa3fc161710 legacy=0x7fa3fc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa3faffd640 1 -- 172.21.15.148:0/3433962386 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319971441 0 0) 0x7fa3fc15e000 con 0x7fa3fc161710 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa400f5e640 1 -- 172.21.15.148:0/3433962386 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc161f40 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa3fbfff640 1 --2- 172.21.15.148:0/3433962386 >> v2:172.21.15.188:3300/0 conn(0x7fa3fc15dc20 0x7fa3fc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:22.465 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.457+0000 7fa400f5e640 1 --2- 172.21.15.148:0/3433962386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc0cebb0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0d1520 tx=0x7fa3f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df21d624d07af68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa3faffd640 1 -- 172.21.15.148:0/3433962386 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4007740 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa3faffd640 1 -- 172.21.15.148:0/3433962386 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f40078e0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa3faffd640 1 -- 172.21.15.148:0/3433962386 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4007cb0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 -- 172.21.15.148:0/3433962386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 msgr2=0x7fa3fc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 --2- 172.21.15.148:0/3433962386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc0cebb0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0d1520 tx=0x7fa3f402a680 comp rx=0 tx=0).stop 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 -- 172.21.15.148:0/3433962386 shutdown_connections 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 --2- 172.21.15.148:0/3433962386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc0cebb0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 --2- 172.21.15.148:0/3433962386 >> v2:172.21.15.188:3300/0 conn(0x7fa3fc15dc20 0x7fa3fc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 -- 172.21.15.148:0/3433962386 >> 172.21.15.148:0/3433962386 conn(0x7fa3fc0d0420 msgr2=0x7fa3fc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 -- 172.21.15.148:0/3433962386 shutdown_connections 2024-04-25T10:08:22.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.458+0000 7fa4029c1640 1 -- 172.21.15.148:0/3433962386 wait complete. 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.459+0000 7fa4029c1640 1 Processor -- start 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 -- start start 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc1f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15dc20 0x7fa3fc1f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc161710 0x7fa3fc156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0b5e50 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc162500 con 0x7fa3fc15dc20 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa4029c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc0b5fd0 con 0x7fa3fc161710 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.460+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc1f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc1f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60486/0 (socket says 172.21.15.148:60486) 2024-04-25T10:08:22.469 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 -- 172.21.15.148:0/1600036868 learned_addr learned my addr 172.21.15.148:0/1600036868 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa400f5e640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc161710 0x7fa3fc156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fb7fe640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15dc20 0x7fa3fc1f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 -- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc161710 msgr2=0x7fa3fc156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc161710 0x7fa3fc156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 -- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15dc20 msgr2=0x7fa3fc1f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15dc20 0x7fa3fc1f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 -- 172.21.15.148:0/1600036868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc160260 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fb7fe640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15dc20 0x7fa3fc1f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3fbfff640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc1f4140 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d7d0 tx=0x7fa3f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0004030 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.470 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f00041d0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc1f4680 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00096a0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.461+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc155ec0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.462+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc073a00 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.463+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3f0009840 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.463+0000 7fa3f8ff9640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e806b500 0x7fa3e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.464+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa3f008a3c0 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.464+0000 7fa3fb7fe640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e806b500 0x7fa3e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.472 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.464+0000 7fa3fb7fe640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e806b500 0x7fa3e806d9c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc15fec0 tx=0x7fa3ec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.468+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3fc1f4680 con 0x7fa3fc0d32f0 2024-04-25T10:08:22.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.639+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3fc1606a0 con 0x7fa3e806b500 2024-04-25T10:08:22.645 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.639+0000 7fa3f8ff9640 1 -- 172.21.15.148:0/1600036868 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa3fc1606a0 con 0x7fa3e806b500 2024-04-25T10:08:22.645 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e806b500 msgr2=0x7fa3e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e806b500 0x7fa3e806d9c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc15fec0 tx=0x7fa3ec009040 comp rx=0 tx=0).stop 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 msgr2=0x7fa3fc1f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc1f4140 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d7d0 tx=0x7fa3f000dc90 comp rx=0 tx=0).stop 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 shutdown_connections 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fa3e806b500 0x7fa3e806d9c0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3fc161710 0x7fa3fc156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3fc15dc20 0x7fa3fc1f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 --2- 172.21.15.148:0/1600036868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3fc0d32f0 0x7fa3fc1f4140 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.641+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 >> 172.21.15.148:0/1600036868 conn(0x7fa3fc0d0420 msgr2=0x7fa3fc0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.642+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 shutdown_connections 2024-04-25T10:08:22.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.642+0000 7fa4029c1640 1 -- 172.21.15.148:0/1600036868 wait complete. 2024-04-25T10:08:22.658 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:08:22.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:22.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.755+0000 7f45f8015640 1 Processor -- start 2024-04-25T10:08:22.761 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.755+0000 7f45f8015640 1 -- start start 2024-04-25T10:08:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f8015640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45f015dc20 0x7f45f00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.764 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f8015640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f45f0161710 0x7f45f00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f8015640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f015e000 con 0x7f45f00b9330 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f8015640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f00d1de0 con 0x7f45f015dc20 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f8015640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f0161c60 con 0x7f45f0161710 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f5db1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45f015dc20 0x7f45f00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f5db1640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f45f015dc20 0x7f45f00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60502/0 (socket says 172.21.15.148:60502) 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.756+0000 7f45f5db1640 1 -- 172.21.15.148:0/2817494172 learned_addr learned my addr 172.21.15.148:0/2817494172 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.757+0000 7f45f55b0640 1 --2- 172.21.15.148:0/2817494172 >> v2:172.21.15.188:3300/0 conn(0x7f45f0161710 0x7f45f00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.757+0000 7f45f5db1640 1 -- 172.21.15.148:0/2817494172 >> v2:172.21.15.188:3300/0 conn(0x7f45f0161710 msgr2=0x7f45f00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.757+0000 7f45f5db1640 1 --2- 172.21.15.148:0/2817494172 >> v2:172.21.15.188:3300/0 conn(0x7f45f0161710 0x7f45f00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.757+0000 7f45f5db1640 1 -- 172.21.15.148:0/2817494172 >> v1:172.21.15.188:6789/0 conn(0x7f45f00b9330 legacy=0x7f45f006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.757+0000 7f45f5db1640 1 -- 172.21.15.148:0/2817494172 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0161f40 con 0x7f45f015dc20 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.757+0000 7f45f5db1640 1 --2- 172.21.15.148:0/2817494172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00ce670 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f45f006da90 tx=0x7f45ec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62312168864dceac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.765 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f4daf640 1 -- 172.21.15.148:0/2817494172 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037070 con 0x7f45f015dc20 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f4daf640 1 -- 172.21.15.148:0/2817494172 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec032440 con 0x7f45f015dc20 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f4daf640 1 -- 172.21.15.148:0/2817494172 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec032810 con 0x7f45f015dc20 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 -- 172.21.15.148:0/2817494172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 msgr2=0x7f45f00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 --2- 172.21.15.148:0/2817494172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00ce670 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f45f006da90 tx=0x7f45ec02a230 comp rx=0 tx=0).stop 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 -- 172.21.15.148:0/2817494172 shutdown_connections 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 --2- 172.21.15.148:0/2817494172 >> v2:172.21.15.188:3300/0 conn(0x7f45f0161710 0x7f45f00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 --2- 172.21.15.148:0/2817494172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00ce670 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 -- 172.21.15.148:0/2817494172 >> 172.21.15.148:0/2817494172 conn(0x7f45f00d0420 msgr2=0x7f45f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 -- 172.21.15.148:0/2817494172 shutdown_connections 2024-04-25T10:08:22.766 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.758+0000 7f45f8015640 1 -- 172.21.15.148:0/2817494172 wait complete. 2024-04-25T10:08:22.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.759+0000 7f45f8015640 1 Processor -- start 2024-04-25T10:08:22.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.759+0000 7f45f8015640 1 -- start start 2024-04-25T10:08:22.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f8015640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f00b9330 0x7f45f014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f8015640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f8015640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45f0161710 0x7f45f00b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f8015640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f00b5e50 con 0x7f45f015dc20 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f8015640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f0162500 con 0x7f45f00b9330 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f8015640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f00b5fd0 con 0x7f45f0161710 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f55b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f55b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60514/0 (socket says 172.21.15.148:60514) 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f55b0640 1 -- 172.21.15.148:0/2532289628 learned_addr learned my addr 172.21.15.148:0/2532289628 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f65b2640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45f0161710 0x7f45f00b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.760+0000 7f45f5db1640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f00b9330 0x7f45f014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f55b0640 1 -- 172.21.15.148:0/2532289628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45f0161710 msgr2=0x7f45f00b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f55b0640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45f0161710 0x7f45f00b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.770 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f55b0640 1 -- 172.21.15.148:0/2532289628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f00b9330 msgr2=0x7f45f014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f55b0640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45f00b9330 0x7f45f014ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f55b0640 1 -- 172.21.15.148:0/2532289628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f015e420 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f55b0640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00cd190 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f45e400d620 tx=0x7f45e400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400b620 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e4019070 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e400ba00 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f015e700 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.761+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f015ecc0 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.762+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45f0073a00 con 0x7f45f015dc20 2024-04-25T10:08:22.771 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.762+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45e400bba0 con 0x7f45f015dc20 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45e27fc640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f45e408a3a0 con 0x7f45f015dc20 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45f5db1640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f45d406b4d0 0x7f45d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45e27fc640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7f45d4071a10 0x7f45d4073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 --> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45e4017e50 con 0x7f45d4071a10 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e4091050 con 0x7f45f015dc20 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45f65b2640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7f45d4071a10 0x7f45d4073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45f5db1640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f45d406b4d0 0x7f45d406d990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f45ec034000 tx=0x7f45ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.772 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.763+0000 7f45f65b2640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7f45d4071a10 0x7f45d4073e50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.773 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.764+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== osd.2 v2:172.21.15.148:6808/1311261585 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f45e4017e50 con 0x7f45d4071a10 2024-04-25T10:08:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.777+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 --> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f45f00d16b0 con 0x7f45d4071a10 2024-04-25T10:08:22.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.777+0000 7f45e27fc640 1 -- 172.21.15.148:0/2532289628 <== osd.2 v2:172.21.15.148:6808/1311261585 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f45f00d16b0 con 0x7f45d4071a10 2024-04-25T10:08:22.783 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:22.783 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:22.783 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:22.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7f45d4071a10 msgr2=0x7f45d4073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6808/1311261585,v1:172.21.15.148:6809/1311261585] conn(0x7f45d4071a10 0x7f45d4073e50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f45d406b4d0 msgr2=0x7f45d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f45d406b4d0 0x7f45d406d990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f45ec034000 tx=0x7f45ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 msgr2=0x7f45f00cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 --2- 172.21.15.148:0/2532289628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45f015dc20 0x7f45f00cd190 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f45e400d620 tx=0x7f45e400dae0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f65b2640 1 -- 172.21.15.148:0/2532289628 reap_dead start 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 shutdown_connections 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 >> 172.21.15.148:0/2532289628 conn(0x7f45f00d0420 msgr2=0x7f45f00b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 shutdown_connections 2024-04-25T10:08:22.785 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.778+0000 7f45f8015640 1 -- 172.21.15.148:0/2532289628 wait complete. 2024-04-25T10:08:22.794 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:08:22.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.886+0000 7fb696c91640 1 Processor -- start 2024-04-25T10:08:22.892 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.886+0000 7fb696c91640 1 -- start start 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb696c91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb69015de80 0x7fb6900cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb696c91640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6900cefb0 0x7fb6900cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb696c91640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900afc80 con 0x7fb6900cf980 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb696c91640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900ce4d0 con 0x7fb6900cefb0 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb696c91640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900ce7b0 con 0x7fb69015de80 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6900cefb0 0x7fb6900cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fb6900cefb0 0x7fb6900cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60516/0 (socket says 172.21.15.148:60516) 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 -- 172.21.15.148:0/1669433799 learned_addr learned my addr 172.21.15.148:0/1669433799 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb69522e640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb6900cf980 0x7fb69006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44412/0 (socket says 172.21.15.148:44412) 2024-04-25T10:08:22.894 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb694a2d640 1 --2- 172.21.15.148:0/1669433799 >> v2:172.21.15.188:3300/0 conn(0x7fb69015de80 0x7fb6900cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 -- 172.21.15.148:0/1669433799 >> v2:172.21.15.188:3300/0 conn(0x7fb69015de80 msgr2=0x7fb6900cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 --2- 172.21.15.148:0/1669433799 >> v2:172.21.15.188:3300/0 conn(0x7fb69015de80 0x7fb6900cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 -- 172.21.15.148:0/1669433799 >> v1:172.21.15.148:6789/0 conn(0x7fb6900cf980 legacy=0x7fb69006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68ffff640 1 -- 172.21.15.148:0/1669433799 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900cea90 con 0x7fb6900cefb0 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb68f7fe640 1 -- 172.21.15.148:0/1669433799 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881845267 0 0) 0x7fb6900afc80 con 0x7fb6900cf980 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.887+0000 7fb694a2d640 1 --2- 172.21.15.148:0/1669433799 >> v2:172.21.15.188:3300/0 conn(0x7fb69015de80 0x7fb6900cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.888+0000 7fb68ffff640 1 --2- 172.21.15.148:0/1669433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6900cefb0 0x7fb6900cf440 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fb6900d1800 tx=0x7fb67c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633506f24fe385f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.888+0000 7fb68f7fe640 1 -- 172.21.15.148:0/1669433799 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c037070 con 0x7fb6900cefb0 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.888+0000 7fb68f7fe640 1 -- 172.21.15.148:0/1669433799 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67c032440 con 0x7fb6900cefb0 2024-04-25T10:08:22.895 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.888+0000 7fb68f7fe640 1 -- 172.21.15.148:0/1669433799 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c032810 con 0x7fb6900cefb0 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 -- 172.21.15.148:0/1669433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6900cefb0 msgr2=0x7fb6900cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 --2- 172.21.15.148:0/1669433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6900cefb0 0x7fb6900cf440 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fb6900d1800 tx=0x7fb67c02a270 comp rx=0 tx=0).stop 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 -- 172.21.15.148:0/1669433799 shutdown_connections 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 --2- 172.21.15.148:0/1669433799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6900cefb0 0x7fb6900cf440 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 --2- 172.21.15.148:0/1669433799 >> v2:172.21.15.188:3300/0 conn(0x7fb69015de80 0x7fb6900cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 -- 172.21.15.148:0/1669433799 >> 172.21.15.148:0/1669433799 conn(0x7fb6900b2390 msgr2=0x7fb6900b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 -- 172.21.15.148:0/1669433799 shutdown_connections 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.889+0000 7fb696c91640 1 -- 172.21.15.148:0/1669433799 wait complete. 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.890+0000 7fb696c91640 1 Processor -- start 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.890+0000 7fb696c91640 1 -- start start 2024-04-25T10:08:22.898 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb696c91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6900cefb0 0x7fb6901494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb696c91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6900cf980 0x7fb6900c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb696c91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 0x7fb6900c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb696c91640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b3030 con 0x7fb69015de80 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb696c91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b2eb0 con 0x7fb6900cf980 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb696c91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c5600 con 0x7fb6900cefb0 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 0x7fb6900c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 0x7fb6900c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60518/0 (socket says 172.21.15.148:60518) 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 -- 172.21.15.148:0/1154124024 learned_addr learned my addr 172.21.15.148:0/1154124024 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb694a2d640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6900cefb0 0x7fb6901494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb68ffff640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6900cf980 0x7fb6900c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 -- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6900cefb0 msgr2=0x7fb6901494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6900cefb0 0x7fb6901494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 -- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6900cf980 msgr2=0x7fb6900c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6900cf980 0x7fb6900c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 -- 172.21.15.148:0/1154124024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690158f00 con 0x7fb69015de80 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb694a2d640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6900cefb0 0x7fb6901494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb68ffff640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6900cf980 0x7fb6900c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.891+0000 7fb69522e640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 0x7fb6900c8ed0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c7d0 tx=0x7fb68800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.892+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb688018070 con 0x7fb69015de80 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.892+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800a640 con 0x7fb69015de80 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.892+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6901591e0 con 0x7fb69015de80 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.892+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800aa70 con 0x7fb69015de80 2024-04-25T10:08:22.900 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.892+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6901597a0 con 0x7fb69015de80 2024-04-25T10:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.893+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb690073a00 con 0x7fb69015de80 2024-04-25T10:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.893+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb68801b070 con 0x7fb69015de80 2024-04-25T10:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.893+0000 7fb68d7fa640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb67406b4d0 0x7fb67406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.894+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb6880896b0 con 0x7fb69015de80 2024-04-25T10:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.894+0000 7fb694a2d640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb67406b4d0 0x7fb67406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.894+0000 7fb694a2d640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb67406b4d0 0x7fb67406d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb68400a990 tx=0x7fb684008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:22.898+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb688058650 con 0x7fb69015de80 2024-04-25T10:08:23.076 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.070+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb69015d050 con 0x7fb67406b4d0 2024-04-25T10:08:23.077 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.070+0000 7fb68d7fa640 1 -- 172.21.15.148:0/1154124024 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb69015d050 con 0x7fb67406b4d0 2024-04-25T10:08:23.077 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb67406b4d0 msgr2=0x7fb67406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb67406b4d0 0x7fb67406d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb68400a990 tx=0x7fb684008040 comp rx=0 tx=0).stop 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 msgr2=0x7fb6900c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 0x7fb6900c8ed0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c7d0 tx=0x7fb68800cc90 comp rx=0 tx=0).stop 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 shutdown_connections 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb67406b4d0 0x7fb67406d990 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb69015de80 0x7fb6900c8ed0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6900cf980 0x7fb6900c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.072+0000 7fb696c91640 1 --2- 172.21.15.148:0/1154124024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6900cefb0 0x7fb6901494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.073+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 >> 172.21.15.148:0/1154124024 conn(0x7fb6900b2390 msgr2=0x7fb69015c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.073+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 shutdown_connections 2024-04-25T10:08:23.079 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.073+0000 7fb696c91640 1 -- 172.21.15.148:0/1154124024 wait complete. 2024-04-25T10:08:23.089 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:08:23.090 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:23.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.185+0000 7f49c9e9c640 1 Processor -- start 2024-04-25T10:08:23.191 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.185+0000 7f49c9e9c640 1 -- start start 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c9e9c640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49c4070dc0 0x7f49c406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c9e9c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49c40711a0 0x7f49c40b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c9e9c640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c40d1770 con 0x7f49c40b4830 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c9e9c640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c41622b0 con 0x7f49c4070dc0 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c9e9c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40b10f0 con 0x7f49c40711a0 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49c4070dc0 0x7f49c406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c37fe640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f49c4070dc0 0x7f49c406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60520/0 (socket says 172.21.15.148:60520) 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c3fff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f49c40b4830 0x7f49c40b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44416/0 (socket says 172.21.15.148:44416) 2024-04-25T10:08:23.194 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c37fe640 1 -- 172.21.15.148:0/3105845426 learned_addr learned my addr 172.21.15.148:0/3105845426 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c37fe640 1 -- 172.21.15.148:0/3105845426 >> v2:172.21.15.188:3300/0 conn(0x7f49c40711a0 msgr2=0x7f49c40b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c2ffd640 1 --2- 172.21.15.148:0/3105845426 >> v2:172.21.15.188:3300/0 conn(0x7f49c40711a0 0x7f49c40b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c37fe640 1 --2- 172.21.15.148:0/3105845426 >> v2:172.21.15.188:3300/0 conn(0x7f49c40711a0 0x7f49c40b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c37fe640 1 -- 172.21.15.148:0/3105845426 >> v1:172.21.15.148:6789/0 conn(0x7f49c40b4830 legacy=0x7f49c40b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.186+0000 7f49c27fc640 1 -- 172.21.15.148:0/3105845426 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964450431 0 0) 0x7f49c40d1770 con 0x7f49c40b4830 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.187+0000 7f49c37fe640 1 -- 172.21.15.148:0/3105845426 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4162740 con 0x7f49c4070dc0 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.187+0000 7f49c2ffd640 1 --2- 172.21.15.148:0/3105845426 >> v2:172.21.15.188:3300/0 conn(0x7f49c40711a0 0x7f49c40b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.187+0000 7f49c37fe640 1 --2- 172.21.15.148:0/3105845426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c4070dc0 0x7f49c406ccd0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f49c404efc0 tx=0x7f49b402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198eb0fde7d17f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.187+0000 7f49c27fc640 1 -- 172.21.15.148:0/3105845426 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4037070 con 0x7f49c4070dc0 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.187+0000 7f49c27fc640 1 -- 172.21.15.148:0/3105845426 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4032440 con 0x7f49c4070dc0 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.187+0000 7f49c27fc640 1 -- 172.21.15.148:0/3105845426 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4032810 con 0x7f49c4070dc0 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3105845426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c4070dc0 msgr2=0x7f49c406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 --2- 172.21.15.148:0/3105845426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c4070dc0 0x7f49c406ccd0 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f49c404efc0 tx=0x7f49b402a150 comp rx=0 tx=0).stop 2024-04-25T10:08:23.195 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3105845426 shutdown_connections 2024-04-25T10:08:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 --2- 172.21.15.148:0/3105845426 >> v2:172.21.15.188:3300/0 conn(0x7f49c40711a0 0x7f49c40b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 --2- 172.21.15.148:0/3105845426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c4070dc0 0x7f49c406ccd0 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3105845426 >> 172.21.15.148:0/3105845426 conn(0x7f49c40b3530 msgr2=0x7f49c40b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3105845426 shutdown_connections 2024-04-25T10:08:23.196 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.188+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3105845426 wait complete. 2024-04-25T10:08:23.199 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.189+0000 7f49c9e9c640 1 Processor -- start 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.189+0000 7f49c9e9c640 1 -- start start 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c9e9c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c4070dc0 0x7f49c40cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c9e9c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49c40711a0 0x7f49c415a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c9e9c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c40b4830 0x7f49c40cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c9e9c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4161f40 con 0x7f49c40b4830 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c9e9c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4161c60 con 0x7f49c4070dc0 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c9e9c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4071630 con 0x7f49c40711a0 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c40b4830 0x7f49c40cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.200 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c40b4830 0x7f49c40cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60522/0 (socket says 172.21.15.148:60522) 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 -- 172.21.15.148:0/3197692670 learned_addr learned my addr 172.21.15.148:0/3197692670 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c2ffd640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49c40711a0 0x7f49c415a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c37fe640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c4070dc0 0x7f49c40cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 -- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49c40711a0 msgr2=0x7f49c415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49c40711a0 0x7f49c415a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 -- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c4070dc0 msgr2=0x7f49c40cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c4070dc0 0x7f49c40cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 -- 172.21.15.148:0/3197692670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4147350 con 0x7f49c40b4830 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c37fe640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c4070dc0 0x7f49c40cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.190+0000 7f49c3fff640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c40b4830 0x7f49c40cabb0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00c700 tx=0x7f49bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.191+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc018070 con 0x7f49c40b4830 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.191+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc00ce10 con 0x7f49c40b4830 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.191+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00a860 con 0x7f49c40b4830 2024-04-25T10:08:23.201 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.191+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c4147630 con 0x7f49c40b4830 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.191+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c4147bf0 con 0x7f49c40b4830 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.192+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49c4073a00 con 0x7f49c40b4830 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.192+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49bc01b070 con 0x7f49c40b4830 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.192+0000 7f49abfff640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49b006b4d0 0x7f49b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49c37fe640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49b006b4d0 0x7f49b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f49bc0896e0 con 0x7f49c40b4830 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49abfff640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f49b0071a10 0x7f49b0073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 --> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49b0074520 con 0x7f49b0071a10 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49bc090050 con 0x7f49c40b4830 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49c2ffd640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f49b0071a10 0x7f49b0073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49c37fe640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49b006b4d0 0x7f49b006d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f49b4034000 tx=0x7f49b402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.202 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.193+0000 7f49c2ffd640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f49b0071a10 0x7f49b0073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.203 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.194+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== osd.3 v2:172.21.15.148:6819/1795140808 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f49b0074520 con 0x7f49b0071a10 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 --> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f49c40b08c0 con 0x7f49b0071a10 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49abfff640 1 -- 172.21.15.148:0/3197692670 <== osd.3 v2:172.21.15.148:6819/1795140808 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f49c40b08c0 con 0x7f49b0071a10 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:23.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f49b0071a10 msgr2=0x7f49b0073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6819/1795140808,v1:172.21.15.148:6821/1795140808] conn(0x7f49b0071a10 0x7f49b0073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49b006b4d0 msgr2=0x7f49b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f49b006b4d0 0x7f49b006d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f49b4034000 tx=0x7f49b402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c40b4830 msgr2=0x7f49c40cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 --2- 172.21.15.148:0/3197692670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49c40b4830 0x7f49c40cabb0 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00c700 tx=0x7f49bc00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c3fff640 1 -- 172.21.15.148:0/3197692670 reap_dead start 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 shutdown_connections 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.207+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 >> 172.21.15.148:0/3197692670 conn(0x7f49c40b3530 msgr2=0x7f49c40cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.208+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 shutdown_connections 2024-04-25T10:08:23.219 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.208+0000 7f49c9e9c640 1 -- 172.21.15.148:0/3197692670 wait complete. 2024-04-25T10:08:23.224 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:08:23.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.316+0000 7ff7691b7640 1 Processor -- start 2024-04-25T10:08:23.323 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff7691b7640 1 -- start start 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff7691b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff76415dc20 0x7ff7640ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff7691b7640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76406cda0 con 0x7ff764161710 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff7691b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76415e000 con 0x7ff7640d32f0 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff7691b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640d1de0 con 0x7ff76415dc20 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff76259c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7ff764161710 0x7ff76406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:44432/0 (socket says 172.21.15.148:44432) 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.317+0000 7ff76259c640 1 -- 172.21.15.148:0/1447247103 learned_addr learned my addr 172.21.15.148:0/1447247103 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.318+0000 7ff762d9d640 1 --2- 172.21.15.148:0/1447247103 >> v2:172.21.15.188:3300/0 conn(0x7ff76415dc20 0x7ff7640ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.318+0000 7ff761d9b640 1 -- 172.21.15.148:0/1447247103 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957122886 0 0) 0x7ff76406cda0 con 0x7ff764161710 2024-04-25T10:08:23.324 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.318+0000 7ff761d9b640 1 -- 172.21.15.148:0/1447247103 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c003520 con 0x7ff764161710 2024-04-25T10:08:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.318+0000 7ff762d9d640 1 -- 172.21.15.148:0/1447247103 >> v1:172.21.15.188:6789/0 conn(0x7ff7640d32f0 legacy=0x7ff76406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.318+0000 7ff762d9d640 1 -- 172.21.15.148:0/1447247103 >> v1:172.21.15.148:6789/0 conn(0x7ff764161710 legacy=0x7ff76406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.325 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.318+0000 7ff762d9d640 1 -- 172.21.15.148:0/1447247103 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff764161c60 con 0x7ff76415dc20 2024-04-25T10:08:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff762d9d640 1 --2- 172.21.15.148:0/1447247103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff76415dc20 0x7ff7640ce670 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff7580097a0 tx=0x7ff75802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340c710ee9741406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff761d9b640 1 -- 172.21.15.148:0/1447247103 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758037070 con 0x7ff76415dc20 2024-04-25T10:08:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff761d9b640 1 -- 172.21.15.148:0/1447247103 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758032440 con 0x7ff76415dc20 2024-04-25T10:08:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff761d9b640 1 -- 172.21.15.148:0/1447247103 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758032810 con 0x7ff76415dc20 2024-04-25T10:08:23.327 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 -- 172.21.15.148:0/1447247103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff76415dc20 msgr2=0x7ff7640ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 --2- 172.21.15.148:0/1447247103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff76415dc20 0x7ff7640ce670 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff7580097a0 tx=0x7ff75802a230 comp rx=0 tx=0).stop 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 -- 172.21.15.148:0/1447247103 shutdown_connections 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 --2- 172.21.15.148:0/1447247103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff76415dc20 0x7ff7640ce670 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 -- 172.21.15.148:0/1447247103 >> 172.21.15.148:0/1447247103 conn(0x7ff7640d0420 msgr2=0x7ff7640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 -- 172.21.15.148:0/1447247103 shutdown_connections 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.319+0000 7ff7691b7640 1 -- 172.21.15.148:0/1447247103 wait complete. 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.321+0000 7ff7691b7640 1 Processor -- start 2024-04-25T10:08:23.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.321+0000 7ff7691b7640 1 -- start start 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 0x7ff7640ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76415dc20 0x7ff764160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff764161710 0x7ff764152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff764162500 con 0x7ff7640d32f0 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff764162220 con 0x7ff764161710 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7640b5e50 con 0x7ff76415dc20 2024-04-25T10:08:23.331 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 0x7ff7640ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 0x7ff7640ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:60532/0 (socket says 172.21.15.148:60532) 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 -- 172.21.15.148:0/4251306658 learned_addr learned my addr 172.21.15.148:0/4251306658 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 -- 172.21.15.148:0/4251306658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76415dc20 msgr2=0x7ff764160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76415dc20 0x7ff764160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 -- 172.21.15.148:0/4251306658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff764161710 msgr2=0x7ff764152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff764161710 0x7ff764152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 -- 172.21.15.148:0/4251306658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640aa8d0 con 0x7ff7640d32f0 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff762d9d640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 0x7ff7640ab970 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7ff758035040 tx=0x7ff7580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758037040 con 0x7ff7640d32f0 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758030500 con 0x7ff7640d32f0 2024-04-25T10:08:23.332 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7640aabb0 con 0x7ff7640d32f0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758030910 con 0x7ff7640d32f0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.322+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7640ab170 con 0x7ff7640d32f0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.323+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7640cd880 con 0x7ff7640d32f0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.324+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff758044020 con 0x7ff7640d32f0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.324+0000 7ff74b7fe640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff75006b4d0 0x7ff75006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.324+0000 7ff76259c640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff75006b4d0 0x7ff75006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.324+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff7580aa2f0 con 0x7ff7640d32f0 2024-04-25T10:08:23.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.325+0000 7ff76259c640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff75006b4d0 0x7ff75006d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff7640d19f0 tx=0x7ff754007930 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.328+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff758079320 con 0x7ff7640d32f0 2024-04-25T10:08:23.502 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.496+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7640d16b0 con 0x7ff75006b4d0 2024-04-25T10:08:23.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.497+0000 7ff74b7fe640 1 -- 172.21.15.148:0/4251306658 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff7640d16b0 con 0x7ff75006b4d0 2024-04-25T10:08:23.503 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff75006b4d0 msgr2=0x7ff75006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff75006b4d0 0x7ff75006d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff7640d19f0 tx=0x7ff754007930 comp rx=0 tx=0).stop 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 msgr2=0x7ff7640ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 0x7ff7640ab970 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7ff758035040 tx=0x7ff7580046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 shutdown_connections 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7ff75006b4d0 0x7ff75006d990 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff764161710 0x7ff764152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff76415dc20 0x7ff764160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 --2- 172.21.15.148:0/4251306658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7640d32f0 0x7ff7640ab970 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.498+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 >> 172.21.15.148:0/4251306658 conn(0x7ff7640d0420 msgr2=0x7ff7640c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.499+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 shutdown_connections 2024-04-25T10:08:23.505 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.499+0000 7ff7691b7640 1 -- 172.21.15.148:0/4251306658 wait complete. 2024-04-25T10:08:23.515 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:08:23.515 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:23.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.610+0000 7f06b3e3a640 1 Processor -- start 2024-04-25T10:08:23.616 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.610+0000 7f06b3e3a640 1 -- start start 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b3e3a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06ac15b7d0 0x7f06ac0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b3e3a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06ac0b5620 0x7f06ac0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b3e3a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac06cf50 con 0x7f06ac06c2d0 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b3e3a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac0ce530 con 0x7f06ac0b5620 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b3e3a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac0ce810 con 0x7f06ac15b7d0 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b13d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06ac0b5620 0x7f06ac0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b23d7640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f06ac06c2d0 0x7f06ac06c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35816/0 (socket says 172.21.15.148:35816) 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b13d5640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f06ac0b5620 0x7f06ac0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42890/0 (socket says 172.21.15.148:42890) 2024-04-25T10:08:23.617 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b23d7640 1 -- 172.21.15.148:0/950946900 learned_addr learned my addr 172.21.15.148:0/950946900 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b1bd6640 1 --2- 172.21.15.148:0/950946900 >> v2:172.21.15.188:3300/0 conn(0x7f06ac15b7d0 0x7f06ac0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b13d5640 1 -- 172.21.15.148:0/950946900 >> v2:172.21.15.188:3300/0 conn(0x7f06ac15b7d0 msgr2=0x7f06ac0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b13d5640 1 --2- 172.21.15.148:0/950946900 >> v2:172.21.15.188:3300/0 conn(0x7f06ac15b7d0 0x7f06ac0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b0bd4640 1 -- 172.21.15.148:0/950946900 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972058634 0 0) 0x7f06ac06cf50 con 0x7f06ac06c2d0 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b13d5640 1 -- 172.21.15.148:0/950946900 >> v1:172.21.15.148:6789/0 conn(0x7f06ac06c2d0 legacy=0x7f06ac06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b13d5640 1 -- 172.21.15.148:0/950946900 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0ceaf0 con 0x7f06ac0b5620 2024-04-25T10:08:23.618 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.611+0000 7f06b1bd6640 1 --2- 172.21.15.148:0/950946900 >> v2:172.21.15.188:3300/0 conn(0x7f06ac15b7d0 0x7f06ac0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:23.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.612+0000 7f06b13d5640 1 --2- 172.21.15.148:0/950946900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0b5ab0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f06ac08c0d0 tx=0x7f069c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda288253ca9ff2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.612+0000 7f06b0bd4640 1 -- 172.21.15.148:0/950946900 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c037070 con 0x7f06ac0b5620 2024-04-25T10:08:23.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.612+0000 7f06b0bd4640 1 -- 172.21.15.148:0/950946900 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c032440 con 0x7f06ac0b5620 2024-04-25T10:08:23.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.612+0000 7f06b0bd4640 1 -- 172.21.15.148:0/950946900 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c032810 con 0x7f06ac0b5620 2024-04-25T10:08:23.622 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 -- 172.21.15.148:0/950946900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 msgr2=0x7f06ac0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 --2- 172.21.15.148:0/950946900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0b5ab0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f06ac08c0d0 tx=0x7f069c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 -- 172.21.15.148:0/950946900 shutdown_connections 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 --2- 172.21.15.148:0/950946900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0b5ab0 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 --2- 172.21.15.148:0/950946900 >> v2:172.21.15.188:3300/0 conn(0x7f06ac15b7d0 0x7f06ac0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 -- 172.21.15.148:0/950946900 >> 172.21.15.148:0/950946900 conn(0x7f06ac0d0800 msgr2=0x7f06ac0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 -- 172.21.15.148:0/950946900 shutdown_connections 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.613+0000 7f06b3e3a640 1 -- 172.21.15.148:0/950946900 wait complete. 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 Processor -- start 2024-04-25T10:08:23.623 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 -- start start 2024-04-25T10:08:23.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06ac06c2d0 0x7f06ac0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac15b7d0 0x7f06ac15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.626 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0cf6d0 con 0x7f06ac0b5620 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.614+0000 7f06b3e3a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0cf550 con 0x7f06ac15b7d0 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b3e3a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac15bbb0 con 0x7f06ac06c2d0 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42902/0 (socket says 172.21.15.148:42902) 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 -- 172.21.15.148:0/2804396987 learned_addr learned my addr 172.21.15.148:0/2804396987 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 -- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06ac06c2d0 msgr2=0x7f06ac0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06ac06c2d0 0x7f06ac0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 -- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac15b7d0 msgr2=0x7f06ac15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06ac15b7d0 0x7f06ac15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 -- 172.21.15.148:0/2804396987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0cb1b0 con 0x7f06ac0b5620 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b13d5640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0cc520 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f069c009890 tx=0x7f069c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c037040 con 0x7f06ac0b5620 2024-04-25T10:08:23.627 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c042070 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac0cb480 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c030a50 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.615+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac0cba40 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.616+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f06ac073a00 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f069c030bf0 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06a27fc640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f069006b4d0 0x7f069006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06b1bd6640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f069006b4d0 0x7f069006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f069c0aab60 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06a27fc640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f0690071a10 0x7f0690073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 --> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f069c008eb0 con 0x7f0690071a10 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.617+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f069c0af2c0 con 0x7f06ac0b5620 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.618+0000 7f06b1bd6640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f069006b4d0 0x7f069006d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.618+0000 7f06b23d7640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f0690071a10 0x7f0690073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.628 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.618+0000 7f06b23d7640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f0690071a10 0x7f0690073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.629 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.619+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== osd.4 v2:172.21.15.188:6800/3303846211 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f069c008eb0 con 0x7f0690071a10 2024-04-25T10:08:23.637 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.631+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 --> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f06ac0b3240 con 0x7f0690071a10 2024-04-25T10:08:23.638 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.632+0000 7f06a27fc640 1 -- 172.21.15.148:0/2804396987 <== osd.4 v2:172.21.15.188:6800/3303846211 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f06ac0b3240 con 0x7f0690071a10 2024-04-25T10:08:23.638 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:23.638 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:23.638 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:23.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.632+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f0690071a10 msgr2=0x7f0690073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.639 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.632+0000 7f06b3e3a640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.188:6800/3303846211,v1:172.21.15.188:6801/3303846211] conn(0x7f0690071a10 0x7f0690073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f069006b4d0 msgr2=0x7f069006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f069006b4d0 0x7f069006d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f06a8006fd0 tx=0x7f06a8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 msgr2=0x7f06ac0cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 --2- 172.21.15.148:0/2804396987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac0b5620 0x7f06ac0cc520 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f069c009890 tx=0x7f069c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b23d7640 1 -- 172.21.15.148:0/2804396987 reap_dead start 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 shutdown_connections 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 >> 172.21.15.148:0/2804396987 conn(0x7f06ac0d0800 msgr2=0x7f06ac15ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 shutdown_connections 2024-04-25T10:08:23.640 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.633+0000 7f06b3e3a640 1 -- 172.21.15.148:0/2804396987 wait complete. 2024-04-25T10:08:23.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:08:23.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.742+0000 7f0732237640 1 Processor -- start 2024-04-25T10:08:23.748 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.742+0000 7f0732237640 1 -- start start 2024-04-25T10:08:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.743+0000 7f0732237640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.743+0000 7f0732237640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f072c0c70d0 0x7f072c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.743+0000 7f0732237640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0c75e0 con 0x7f072c0cfb40 2024-04-25T10:08:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.743+0000 7f0732237640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c0c8680 con 0x7f072c0c70d0 2024-04-25T10:08:23.749 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.743+0000 7f0732237640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c78c0 con 0x7f072c15bb50 2024-04-25T10:08:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f0722ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f072c0c70d0 0x7f072c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f0722ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f072c0c70d0 0x7f072c0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.148:56102/0 (socket says 172.21.15.148:56102) 2024-04-25T10:08:23.750 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c0c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.148:36990/0 (socket says 172.21.15.148:36990) 2024-04-25T10:08:23.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f0722ffd640 1 -- 172.21.15.148:0/3655016827 learned_addr learned my addr 172.21.15.148:0/3655016827 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:23.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072b7fe640 1 -- 172.21.15.148:0/3655016827 >> v2:172.21.15.188:3300/0 conn(0x7f072c0c70d0 msgr2=0x7f072c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072b7fe640 1 --2- 172.21.15.148:0/3655016827 >> v2:172.21.15.188:3300/0 conn(0x7f072c0c70d0 0x7f072c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072affd640 1 -- 172.21.15.148:0/3655016827 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869246815 0 0) 0x7f072c0c75e0 con 0x7f072c0cfb40 2024-04-25T10:08:23.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072b7fe640 1 -- 172.21.15.148:0/3655016827 >> v1:172.21.15.188:6789/0 conn(0x7f072c0cfb40 legacy=0x7f072c070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.751 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.744+0000 7f072b7fe640 1 -- 172.21.15.148:0/3655016827 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0c7ba0 con 0x7f072c15bb50 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.745+0000 7f072b7fe640 1 --2- 172.21.15.148:0/3655016827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c0c6b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f071c0097a0 tx=0x7f071c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142c23d86b2e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.745+0000 7f072affd640 1 -- 172.21.15.148:0/3655016827 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c037070 con 0x7f072c15bb50 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.745+0000 7f072affd640 1 -- 172.21.15.148:0/3655016827 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c032440 con 0x7f072c15bb50 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.745+0000 7f072affd640 1 -- 172.21.15.148:0/3655016827 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c032810 con 0x7f072c15bb50 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 -- 172.21.15.148:0/3655016827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 msgr2=0x7f072c0c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 --2- 172.21.15.148:0/3655016827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c0c6b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f071c0097a0 tx=0x7f071c02a210 comp rx=0 tx=0).stop 2024-04-25T10:08:23.752 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 -- 172.21.15.148:0/3655016827 shutdown_connections 2024-04-25T10:08:23.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 --2- 172.21.15.148:0/3655016827 >> v2:172.21.15.188:3300/0 conn(0x7f072c0c70d0 0x7f072c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 --2- 172.21.15.148:0/3655016827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c0c6b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 -- 172.21.15.148:0/3655016827 >> 172.21.15.148:0/3655016827 conn(0x7f072c06f500 msgr2=0x7f072c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 -- 172.21.15.148:0/3655016827 shutdown_connections 2024-04-25T10:08:23.753 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.746+0000 7f0732237640 1 -- 172.21.15.148:0/3655016827 wait complete. 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.747+0000 7f0732237640 1 Processor -- start 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.747+0000 7f0732237640 1 -- start start 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0732237640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 0x7f072c15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0732237640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c0cfb40 0x7f072c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0732237640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0732237640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c15f6d0 con 0x7f072c0c70d0 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0732237640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c8160 con 0x7f072c0cfb40 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0732237640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c15f850 con 0x7f072c15bb50 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 0x7f072c15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 0x7f072c15ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42918/0 (socket says 172.21.15.148:42918) 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 -- 172.21.15.148:0/292600867 learned_addr learned my addr 172.21.15.148:0/292600867 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072bfff640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0722ffd640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c0cfb40 0x7f072c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.755 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 -- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 msgr2=0x7f072c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 -- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c0cfb40 msgr2=0x7f072c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c0cfb40 0x7f072c0c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 -- 172.21.15.148:0/292600867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0c9e80 con 0x7f072c0c70d0 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072bfff640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f0722ffd640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c0cfb40 0x7f072c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.748+0000 7f072b7fe640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 0x7f072c15ee00 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f071c0097a0 tx=0x7f071c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.749+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c037040 con 0x7f072c0c70d0 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.749+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c004230 con 0x7f072c0c70d0 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.749+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0ca120 con 0x7f072c0c70d0 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.749+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c003610 con 0x7f072c0c70d0 2024-04-25T10:08:23.756 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.749+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0ca6e0 con 0x7f072c0c70d0 2024-04-25T10:08:23.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.750+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c073a00 con 0x7f072c0c70d0 2024-04-25T10:08:23.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.750+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f071c0037b0 con 0x7f072c0c70d0 2024-04-25T10:08:23.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.751+0000 7f0728ff9640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f070c06b500 0x7f070c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.757 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.751+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f071c0ab190 con 0x7f072c0c70d0 2024-04-25T10:08:23.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.751+0000 7f0722ffd640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f070c06b500 0x7f070c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.758 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.751+0000 7f0722ffd640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f070c06b500 0x7f070c06d9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f072c0cf740 tx=0x7f0714008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.760 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.755+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f071c07a1c0 con 0x7f072c0c70d0 2024-04-25T10:08:23.930 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.924+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f072c15fd10 con 0x7f070c06b500 2024-04-25T10:08:23.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.925+0000 7f0728ff9640 1 -- 172.21.15.148:0/292600867 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f072c15fd10 con 0x7f070c06b500 2024-04-25T10:08:23.931 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f070c06b500 msgr2=0x7f070c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f070c06b500 0x7f070c06d9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f072c0cf740 tx=0x7f0714008040 comp rx=0 tx=0).stop 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 msgr2=0x7f072c15ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 0x7f072c15ee00 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f071c0097a0 tx=0x7f071c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 shutdown_connections 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f070c06b500 0x7f070c06d9c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f072c15bb50 0x7f072c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f072c0cfb40 0x7f072c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 --2- 172.21.15.148:0/292600867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f072c0c70d0 0x7f072c15ee00 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 >> 172.21.15.148:0/292600867 conn(0x7f072c06f500 msgr2=0x7f072c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 shutdown_connections 2024-04-25T10:08:23.933 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:23.927+0000 7f0732237640 1 -- 172.21.15.148:0/292600867 wait complete. 2024-04-25T10:08:23.943 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:08:23.943 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:24.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.038+0000 7f4e68a5d640 1 Processor -- start 2024-04-25T10:08:24.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.038+0000 7f4e68a5d640 1 -- start start 2024-04-25T10:08:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e68a5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4e640cc4f0 0x7f4e640cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e68a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 0x7f4e640cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e68a5d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e64072280 con 0x7f4e640cdb20 2024-04-25T10:08:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e68a5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e64071270 con 0x7f4e640cc4f0 2024-04-25T10:08:24.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e68a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640714c0 con 0x7f4e6415b980 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e5bfff640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4e640cdb20 0x7f4e6415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35824/0 (socket says 172.21.15.148:35824) 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e5bfff640 1 -- 172.21.15.148:0/2027510432 learned_addr learned my addr 172.21.15.148:0/2027510432 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 0x7f4e640cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593397978 0 0) 0x7f4e64072280 con 0x7f4e640cdb20 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e6259c640 1 --2- 172.21.15.148:0/2027510432 >> v2:172.21.15.188:3300/0 conn(0x7f4e640cc4f0 0x7f4e640cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e50003520 con 0x7f4e640cdb20 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1483677445 0 0) 0x7f4e50003520 con 0x7f4e640cdb20 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 msgr2=0x7f4e640cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 0x7f4e640cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 >> v2:172.21.15.188:3300/0 conn(0x7f4e640cc4f0 msgr2=0x7f4e640cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 --2- 172.21.15.148:0/2027510432 >> v2:172.21.15.188:3300/0 conn(0x7f4e640cc4f0 0x7f4e640cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640717a0 con 0x7f4e640cdb20 2024-04-25T10:08:24.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f4e4c001690 con 0x7f4e640cdb20 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 >> v1:172.21.15.148:6789/0 conn(0x7f4e640cdb20 legacy=0x7f4e6415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e500039b0 0x7f4e50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 0x7f4e50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64071270 con 0x7f4e50004d40 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640714c0 con 0x7f4e500039b0 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5000b9c0 con 0x7f4e6415b980 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.039+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e6259c640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e500039b0 0x7f4e50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e5bfff640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 0x7f4e50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e62d9d640 1 -- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 msgr2=0x7f4e50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6415b980 0x7f4e50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e62d9d640 1 -- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e500039b0 msgr2=0x7f4e50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e500039b0 0x7f4e50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e62d9d640 1 -- 172.21.15.148:0/2027510432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50003520 con 0x7f4e50004d40 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.040+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e500051c0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009fc0 tx=0x7f4e50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d1595dfb44f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c009c10 con 0x7f4e50004d40 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c036070 con 0x7f4e50004d40 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e61d9b640 1 -- 172.21.15.148:0/2027510432 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c031880 con 0x7f4e50004d40 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e68a5d640 1 -- 172.21.15.148:0/2027510432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 msgr2=0x7f4e500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e68a5d640 1 --2- 172.21.15.148:0/2027510432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e500051c0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009fc0 tx=0x7f4e50005830 comp rx=0 tx=0).stop 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e62d9d640 1 -- 172.21.15.148:0/2027510432 reap_dead start 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e68a5d640 1 -- 172.21.15.148:0/2027510432 shutdown_connections 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e68a5d640 1 -- 172.21.15.148:0/2027510432 >> 172.21.15.148:0/2027510432 conn(0x7f4e6406abe0 msgr2=0x7f4e6406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e68a5d640 1 -- 172.21.15.148:0/2027510432 shutdown_connections 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.041+0000 7f4e68a5d640 1 -- 172.21.15.148:0/2027510432 wait complete. 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.042+0000 7f4e68a5d640 1 Processor -- start 2024-04-25T10:08:24.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.042+0000 7f4e68a5d640 1 -- start start 2024-04-25T10:08:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e68a5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e64156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e68a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6414b410 0x7f4e641565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e68a5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e64156b00 0x7f4e641f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e68a5d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640cdfb0 con 0x7f4e50004d40 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e68a5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64071d60 con 0x7f4e64156b00 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e68a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640ce130 con 0x7f4e6414b410 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e64156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e64156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42942/0 (socket says 172.21.15.148:42942) 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 -- 172.21.15.148:0/1485025916 learned_addr learned my addr 172.21.15.148:0/1485025916 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e5bfff640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6414b410 0x7f4e641565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e64156b00 0x7f4e641f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 -- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6414b410 msgr2=0x7f4e641565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e6414b410 0x7f4e641565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 -- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e64156b00 msgr2=0x7f4e641f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e64156b00 0x7f4e641f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.043+0000 7f4e6259c640 1 -- 172.21.15.148:0/1485025916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6414edc0 con 0x7f4e50004d40 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e64156b00 0x7f4e641f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e6259c640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e64156080 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f4e54011960 tx=0x7f4e54011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400faf0 con 0x7f4e50004d40 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5400fc90 con 0x7f4e50004d40 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6414e240 con 0x7f4e50004d40 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54014440 con 0x7f4e50004d40 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.044+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6414e800 con 0x7f4e50004d40 2024-04-25T10:08:24.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.045+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e64073a00 con 0x7f4e50004d40 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.045+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e54012040 con 0x7f4e50004d40 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5affd640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e4406b500 0x7f4e4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4e5408a720 con 0x7f4e50004d40 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5bfff640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e4406b500 0x7f4e4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5affd640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7f4e44071a40 0x7f4e44073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 --> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e54025ec0 con 0x7f4e44071a40 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e54091050 con 0x7f4e50004d40 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e5bfff640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e4406b500 0x7f4e4406d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4e6414b850 tx=0x7f4e4c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7f4e44071a40 0x7f4e44073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.046+0000 7f4e62d9d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7f4e44071a40 0x7f4e44073e80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.047+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== osd.5 v2:172.21.15.188:6808/3945194105 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4e54025ec0 con 0x7f4e44071a40 2024-04-25T10:08:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.060+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 --> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4e6406d3a0 con 0x7f4e44071a40 2024-04-25T10:08:24.066 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e5affd640 1 -- 172.21.15.148:0/1485025916 <== osd.5 v2:172.21.15.188:6808/3945194105 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4e6406d3a0 con 0x7f4e44071a40 2024-04-25T10:08:24.067 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:24.067 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:24.067 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7f4e44071a40 msgr2=0x7f4e44073e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.188:6808/3945194105,v1:172.21.15.188:6809/3945194105] conn(0x7f4e44071a40 0x7f4e44073e80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e4406b500 msgr2=0x7f4e4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4e4406b500 0x7f4e4406d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4e6414b850 tx=0x7f4e4c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 msgr2=0x7f4e64156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 --2- 172.21.15.148:0/1485025916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e50004d40 0x7f4e64156080 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7f4e54011960 tx=0x7f4e54011e20 comp rx=0 tx=0).stop 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e62d9d640 1 -- 172.21.15.148:0/1485025916 reap_dead start 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 shutdown_connections 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 >> 172.21.15.148:0/1485025916 conn(0x7f4e6406abe0 msgr2=0x7f4e6415d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 shutdown_connections 2024-04-25T10:08:24.068 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.061+0000 7f4e68a5d640 1 -- 172.21.15.148:0/1485025916 wait complete. 2024-04-25T10:08:24.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:08:24.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.170+0000 7f09af77e640 1 Processor -- start 2024-04-25T10:08:24.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.170+0000 7f09af77e640 1 -- start start 2024-04-25T10:08:24.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09af77e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a8070dc0 0x7f09a806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09af77e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09a80711a0 0x7f09a80b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09af77e640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f09a80b4830 0x7f09a80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09af77e640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a81622b0 con 0x7f09a80b4830 2024-04-25T10:08:24.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09af77e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a80b10f0 con 0x7f09a80711a0 2024-04-25T10:08:24.177 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09af77e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80d1770 con 0x7f09a8070dc0 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f09a80b4830 0x7f09a80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f09a80b4830 0x7f09a80b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42946/0 (socket says 172.21.15.148:42946) 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 -- 172.21.15.148:0/3785830995 learned_addr learned my addr 172.21.15.148:0/3785830995 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 -- 172.21.15.148:0/3785830995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a8070dc0 msgr2=0x7f09a806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 --2- 172.21.15.148:0/3785830995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a8070dc0 0x7f09a806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 -- 172.21.15.148:0/3785830995 >> v2:172.21.15.188:3300/0 conn(0x7f09a80711a0 msgr2=0x7f09a80b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 --2- 172.21.15.148:0/3785830995 >> v2:172.21.15.188:3300/0 conn(0x7f09a80711a0 0x7f09a80b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.178 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.171+0000 7f09add1b640 1 -- 172.21.15.148:0/3785830995 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8162740 con 0x7f09a80b4830 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.172+0000 7f09add1b640 1 --2- 172.21.15.148:0/3785830995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80b4cc0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f09a806f540 tx=0x7f09a002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40af9b6b8f4c3886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.172+0000 7f099ffff640 1 -- 172.21.15.148:0/3785830995 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0036070 con 0x7f09a80b4830 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.172+0000 7f099ffff640 1 -- 172.21.15.148:0/3785830995 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0031600 con 0x7f09a80b4830 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.172+0000 7f099ffff640 1 -- 172.21.15.148:0/3785830995 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00319d0 con 0x7f09a80b4830 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 -- 172.21.15.148:0/3785830995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 msgr2=0x7f09a80b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 --2- 172.21.15.148:0/3785830995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80b4cc0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f09a806f540 tx=0x7f09a002a5b0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 -- 172.21.15.148:0/3785830995 shutdown_connections 2024-04-25T10:08:24.180 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 --2- 172.21.15.148:0/3785830995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80b4cc0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 --2- 172.21.15.148:0/3785830995 >> v2:172.21.15.188:3300/0 conn(0x7f09a80711a0 0x7f09a80b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 --2- 172.21.15.148:0/3785830995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a8070dc0 0x7f09a806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 -- 172.21.15.148:0/3785830995 >> 172.21.15.148:0/3785830995 conn(0x7f09a80b3530 msgr2=0x7f09a80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 -- 172.21.15.148:0/3785830995 shutdown_connections 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.173+0000 7f09af77e640 1 -- 172.21.15.148:0/3785830995 wait complete. 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.174+0000 7f09af77e640 1 Processor -- start 2024-04-25T10:08:24.181 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.174+0000 7f09af77e640 1 -- start start 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.174+0000 7f09af77e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a8070dc0 0x7f09a8148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80711a0 0x7f09a80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8161f40 con 0x7f09a80b4830 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8161c60 con 0x7f09a8070dc0 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8071600 con 0x7f09a80711a0 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42956/0 (socket says 172.21.15.148:42956) 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 -- 172.21.15.148:0/947597958 learned_addr learned my addr 172.21.15.148:0/947597958 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 -- 172.21.15.148:0/947597958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80711a0 msgr2=0x7f09a80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.184 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80711a0 0x7f09a80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 -- 172.21.15.148:0/947597958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a8070dc0 msgr2=0x7f09a8148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a8070dc0 0x7f09a8148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 -- 172.21.15.148:0/947597958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80cae00 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09add1b640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80cef40 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009e50 tx=0x7f09a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0041020 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0007780 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0036040 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a80cb0e0 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.175+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a80cbbc0 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.176+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a8073a00 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.177+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09a0044020 con 0x7f09a80b4830 2024-04-25T10:08:24.185 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.177+0000 7f099dffb640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f098c06b4d0 0x7f098c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.177+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f09a00aada0 con 0x7f09a80b4830 2024-04-25T10:08:24.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.177+0000 7f09ad51a640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f098c06b4d0 0x7f098c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.186 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.178+0000 7f09ad51a640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f098c06b4d0 0x7f098c06d990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f09a8155d20 tx=0x7f09a4006c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.187 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.181+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09a004b030 con 0x7f09a80b4830 2024-04-25T10:08:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.351+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f09a80cabb0 con 0x7f098c06b4d0 2024-04-25T10:08:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.351+0000 7f099dffb640 1 -- 172.21.15.148:0/947597958 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f09a80cabb0 con 0x7f098c06b4d0 2024-04-25T10:08:24.357 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:08:24.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f098c06b4d0 msgr2=0x7f098c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f098c06b4d0 0x7f098c06d990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f09a8155d20 tx=0x7f09a4006c90 comp rx=0 tx=0).stop 2024-04-25T10:08:24.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 msgr2=0x7f09a80cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80cef40 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009e50 tx=0x7f09a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:24.359 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 shutdown_connections 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f098c06b4d0 0x7f098c06d990 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09a80b4830 0x7f09a80cef40 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09a80711a0 0x7f09a80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 --2- 172.21.15.148:0/947597958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09a8070dc0 0x7f09a8148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 >> 172.21.15.148:0/947597958 conn(0x7f09a80b3530 msgr2=0x7f09a806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 shutdown_connections 2024-04-25T10:08:24.360 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.353+0000 7f09af77e640 1 -- 172.21.15.148:0/947597958 wait complete. 2024-04-25T10:08:24.370 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:08:24.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:24.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.465+0000 7f4bf986a640 1 Processor -- start 2024-04-25T10:08:24.471 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.465+0000 7f4bf986a640 1 -- start start 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf986a640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bf415de80 0x7f4bf40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf986a640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40afc80 con 0x7f4bf40cefb0 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf986a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40ce4d0 con 0x7f4bf40ccfe0 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf986a640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf40ce7b0 con 0x7f4bf415de80 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bf415de80 0x7f4bf40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf27fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f4bf40cefb0 0x7f4bf40cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35834/0 (socket says 172.21.15.148:35834) 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf27fc640 1 -- 172.21.15.148:0/2190573436 learned_addr learned my addr 172.21.15.148:0/2190573436 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:24.473 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.466+0000 7f4bf2ffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f4bf415de80 0x7f4bf40cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42968/0 (socket says 172.21.15.148:42968) 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf2ffd640 1 -- 172.21.15.148:0/2190573436 >> v1:172.21.15.188:6789/0 conn(0x7f4bf40ccfe0 legacy=0x7f4bf40cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf2ffd640 1 -- 172.21.15.148:0/2190573436 >> v1:172.21.15.148:6789/0 conn(0x7f4bf40cefb0 legacy=0x7f4bf40cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf2ffd640 1 -- 172.21.15.148:0/2190573436 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40cea90 con 0x7f4bf415de80 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf2ffd640 1 --2- 172.21.15.148:0/2190573436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40cadb0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009d90 tx=0x7f4be402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f78c465256a3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf1ffb640 1 -- 172.21.15.148:0/2190573436 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007740 con 0x7f4bf415de80 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf1ffb640 1 -- 172.21.15.148:0/2190573436 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40078e0 con 0x7f4bf415de80 2024-04-25T10:08:24.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.467+0000 7f4bf1ffb640 1 -- 172.21.15.148:0/2190573436 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007cb0 con 0x7f4bf415de80 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 -- 172.21.15.148:0/2190573436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 msgr2=0x7f4bf40cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 --2- 172.21.15.148:0/2190573436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40cadb0 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009d90 tx=0x7f4be402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 -- 172.21.15.148:0/2190573436 shutdown_connections 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 --2- 172.21.15.148:0/2190573436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40cadb0 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 -- 172.21.15.148:0/2190573436 >> 172.21.15.148:0/2190573436 conn(0x7f4bf40b2390 msgr2=0x7f4bf40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 -- 172.21.15.148:0/2190573436 shutdown_connections 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.468+0000 7f4bf986a640 1 -- 172.21.15.148:0/2190573436 wait complete. 2024-04-25T10:08:24.475 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.469+0000 7f4bf986a640 1 Processor -- start 2024-04-25T10:08:24.476 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.469+0000 7f4bf986a640 1 -- start start 2024-04-25T10:08:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf986a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bf40ccfe0 0x7f4bf40c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.477 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf986a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40cefb0 0x7f4bf4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf986a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf986a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40cfb50 con 0x7f4bf415de80 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf986a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf40b2eb0 con 0x7f4bf40cefb0 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf986a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf415ce90 con 0x7f4bf40ccfe0 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42982/0 (socket says 172.21.15.148:42982) 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 -- 172.21.15.148:0/3364207438 learned_addr learned my addr 172.21.15.148:0/3364207438 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf27fc640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40cefb0 0x7f4bf4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf2ffd640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bf40ccfe0 0x7f4bf40c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 -- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bf40ccfe0 msgr2=0x7f4bf40c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bf40ccfe0 0x7f4bf40c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 -- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40cefb0 msgr2=0x7f4bf4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40cefb0 0x7f4bf4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.478 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.470+0000 7f4bf37fe640 1 -- 172.21.15.148:0/3364207438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf415d170 con 0x7f4bf415de80 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.471+0000 7f4bf37fe640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40c9f30 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f4bec001eb0 tx=0x7f4bec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.471+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec017070 con 0x7f4bf415de80 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.471+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec0052d0 con 0x7f4bf415de80 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.471+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec003810 con 0x7f4bf415de80 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.471+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf40c82e0 con 0x7f4bf415de80 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.471+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf41533f0 con 0x7f4bf415de80 2024-04-25T10:08:24.479 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.472+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bf415d3b0 con 0x7f4bf415de80 2024-04-25T10:08:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.472+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bec01a070 con 0x7f4bf415de80 2024-04-25T10:08:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bdbfff640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4be006b4d0 0x7f4be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4bec0893d0 con 0x7f4bf415de80 2024-04-25T10:08:24.480 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bf2ffd640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4be006b4d0 0x7f4be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bdbfff640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7f4be0071a10 0x7f4be0073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be0074520 con 0x7f4be0071a10 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bec090050 con 0x7f4bf415de80 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bf27fc640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7f4be0071a10 0x7f4be0073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bf2ffd640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4be006b4d0 0x7f4be006d990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4be4034000 tx=0x7f4be4033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.473+0000 7f4bf27fc640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7f4be0071a10 0x7f4be0073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.481 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.474+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== osd.6 v2:172.21.15.188:6816/1362851547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4be0074520 con 0x7f4be0071a10 2024-04-25T10:08:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.487+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 --> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4bf4073a00 con 0x7f4be0071a10 2024-04-25T10:08:24.493 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bdbfff640 1 -- 172.21.15.148:0/3364207438 <== osd.6 v2:172.21.15.188:6816/1362851547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4bf4073a00 con 0x7f4be0071a10 2024-04-25T10:08:24.494 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:24.494 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:24.494 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7f4be0071a10 msgr2=0x7f4be0073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.188:6816/1362851547,v1:172.21.15.188:6817/1362851547] conn(0x7f4be0071a10 0x7f4be0073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4be006b4d0 msgr2=0x7f4be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f4be006b4d0 0x7f4be006d990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4be4034000 tx=0x7f4be4033040 comp rx=0 tx=0).stop 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 msgr2=0x7f4bf40c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 --2- 172.21.15.148:0/3364207438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bf415de80 0x7f4bf40c9f30 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f4bec001eb0 tx=0x7f4bec00c890 comp rx=0 tx=0).stop 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf37fe640 1 -- 172.21.15.148:0/3364207438 reap_dead start 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 shutdown_connections 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 >> 172.21.15.148:0/3364207438 conn(0x7f4bf40b2390 msgr2=0x7f4bf40cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 shutdown_connections 2024-04-25T10:08:24.495 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.488+0000 7f4bf986a640 1 -- 172.21.15.148:0/3364207438 wait complete. 2024-04-25T10:08:24.504 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:08:24.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.597+0000 7fef474ee640 1 Processor -- start 2024-04-25T10:08:24.603 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.597+0000 7fef474ee640 1 -- start start 2024-04-25T10:08:24.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef474ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef4006c330 0x7fef400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef474ee640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400d1d50 con 0x7fef4006c7e0 2024-04-25T10:08:24.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef474ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400d2020 con 0x7fef40158230 2024-04-25T10:08:24.606 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef474ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef40158610 con 0x7fef4006c330 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef45a8b640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fef4006c7e0 0x7fef4006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35844/0 (socket says 172.21.15.148:35844) 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef45a8b640 1 -- 172.21.15.148:0/1862352476 learned_addr learned my addr 172.21.15.148:0/1862352476 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611897089 0 0) 0x7fef400d1d50 con 0x7fef4006c7e0 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef44a89640 1 --2- 172.21.15.148:0/1862352476 >> v2:172.21.15.188:3300/0 conn(0x7fef4006c330 0x7fef400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef2c003520 con 0x7fef4006c7e0 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402315986 0 0) 0x7fef2c003520 con 0x7fef4006c7e0 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 >> v2:172.21.15.188:3300/0 conn(0x7fef4006c330 msgr2=0x7fef400b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 --2- 172.21.15.148:0/1862352476 >> v2:172.21.15.188:3300/0 conn(0x7fef4006c330 0x7fef400b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 >> v1:172.21.15.188:6789/0 conn(0x7fef40158230 legacy=0x7fef400d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400d2300 con 0x7fef4006c7e0 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fef380019f0 con 0x7fef4006c7e0 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 >> v1:172.21.15.148:6789/0 conn(0x7fef4006c7e0 legacy=0x7fef4006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.607 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c004800 0x7fef2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c0051c0 0x7fef2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40158610 con 0x7fef2c0039b0 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c00c110 con 0x7fef2c0051c0 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c00c310 con 0x7fef2c004800 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.598+0000 7fef44a89640 1 --2- 172.21.15.148:0/1862352476 >> v2:172.21.15.188:3300/0 conn(0x7fef4006c330 0x7fef400b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef4528a640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef45a8b640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c0051c0 0x7fef2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef44a89640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c004800 0x7fef2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef4528a640 1 -- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c004800 msgr2=0x7fef2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef4528a640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c004800 0x7fef2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef4528a640 1 -- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c0051c0 msgr2=0x7fef2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef4528a640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c0051c0 0x7fef2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef45a8b640 1 -- 172.21.15.148:0/1862352476 reap_dead start 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.599+0000 7fef4528a640 1 -- 172.21.15.148:0/1862352476 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c003520 con 0x7fef2c0039b0 2024-04-25T10:08:24.608 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef4528a640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef2c003dd0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fef3c001750 tx=0x7fef2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a922c272ad8f8747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c001370 con 0x7fef2c0039b0 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c0052d0 con 0x7fef2c0039b0 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef37fff640 1 -- 172.21.15.148:0/1862352476 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c02abc0 con 0x7fef2c0039b0 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 -- 172.21.15.148:0/1862352476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 msgr2=0x7fef2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef2c003dd0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fef3c001750 tx=0x7fef2c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 -- 172.21.15.148:0/1862352476 shutdown_connections 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 --2- 172.21.15.148:0/1862352476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef2c003dd0 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 -- 172.21.15.148:0/1862352476 >> 172.21.15.148:0/1862352476 conn(0x7fef40061600 msgr2=0x7fef400619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 -- 172.21.15.148:0/1862352476 shutdown_connections 2024-04-25T10:08:24.609 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.600+0000 7fef474ee640 1 -- 172.21.15.148:0/1862352476 wait complete. 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.601+0000 7fef474ee640 1 Processor -- start 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.601+0000 7fef474ee640 1 -- start start 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef474ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef40155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef474ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef40159e00 0x7fef400cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef474ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef400cb0f0 0x7fef40149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef474ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef400d25e0 con 0x7fef2c0039b0 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef474ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4006bea0 con 0x7fef400cb0f0 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef474ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4015e320 con 0x7fef40159e00 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef40155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef40155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43008/0 (socket says 172.21.15.148:43008) 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 -- 172.21.15.148:0/718586798 learned_addr learned my addr 172.21.15.148:0/718586798 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 -- 172.21.15.148:0/718586798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef40159e00 msgr2=0x7fef400cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef40159e00 0x7fef400cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.612 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 -- 172.21.15.148:0/718586798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef400cb0f0 msgr2=0x7fef40149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef400cb0f0 0x7fef40149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.602+0000 7fef4528a640 1 -- 172.21.15.148:0/718586798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef401f3b20 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.603+0000 7fef4528a640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef40155f70 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fef3c000c00 tx=0x7fef3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.603+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c03f020 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.603+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c005480 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.603+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef401f3de0 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.603+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c007a30 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.603+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef401f43a0 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.604+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef401f4a60 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.605+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef3c042020 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.605+0000 7fef35ffb640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef2406b4d0 0x7fef2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.605+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fef3c0a9c20 con 0x7fef2c0039b0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.605+0000 7fef44a89640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef2406b4d0 0x7fef2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.613 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.606+0000 7fef44a89640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef2406b4d0 0x7fef2406d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fef30006e20 tx=0x7fef30006d70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.609+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef401f4a60 con 0x7fef2c0039b0 2024-04-25T10:08:24.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.783+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fef4006de50 con 0x7fef2406b4d0 2024-04-25T10:08:24.789 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.783+0000 7fef35ffb640 1 -- 172.21.15.148:0/718586798 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fef4006de50 con 0x7fef2406b4d0 2024-04-25T10:08:24.789 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:08:24.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef2406b4d0 msgr2=0x7fef2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef2406b4d0 0x7fef2406d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fef30006e20 tx=0x7fef30006d70 comp rx=0 tx=0).stop 2024-04-25T10:08:24.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 msgr2=0x7fef40155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef40155f70 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fef3c000c00 tx=0x7fef3c001120 comp rx=0 tx=0).stop 2024-04-25T10:08:24.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 shutdown_connections 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fef2406b4d0 0x7fef2406d990 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef400cb0f0 0x7fef40149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef40159e00 0x7fef400cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 --2- 172.21.15.148:0/718586798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef2c0039b0 0x7fef40155f70 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.785+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 >> 172.21.15.148:0/718586798 conn(0x7fef40061600 msgr2=0x7fef4006f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.786+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 shutdown_connections 2024-04-25T10:08:24.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.786+0000 7fef474ee640 1 -- 172.21.15.148:0/718586798 wait complete. 2024-04-25T10:08:24.802 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:08:24.802 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:08:24.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.897+0000 7f209cb89640 1 Processor -- start 2024-04-25T10:08:24.903 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.897+0000 7f209cb89640 1 -- start start 2024-04-25T10:08:24.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f209cb89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f2098166980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f209cb89640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20980ceba0 con 0x7f2098144d20 2024-04-25T10:08:24.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f209cb89640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20980cee80 con 0x7f20981454b0 2024-04-25T10:08:24.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f209cb89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20980b4830 con 0x7f20980714d0 2024-04-25T10:08:24.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f209659c640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f2098144d20 0x7f20980ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35848/0 (socket says 172.21.15.148:35848) 2024-04-25T10:08:24.907 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f209659c640 1 -- 172.21.15.148:0/1553979299 learned_addr learned my addr 172.21.15.148:0/1553979299 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285341096 0 0) 0x7f20980ceba0 con 0x7f2098144d20 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003520 con 0x7f2098144d20 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2096d9d640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f2098166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3008283061 0 0) 0x7f2084003520 con 0x7f2098144d20 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 msgr2=0x7f2098166980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f2098166980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 >> v1:172.21.15.188:6789/0 conn(0x7f20981454b0 legacy=0x7f2098070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980cf160 con 0x7f2098144d20 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f2088001670 con 0x7f2098144d20 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 >> v1:172.21.15.148:6789/0 conn(0x7f2098144d20 legacy=0x7f20980ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.908 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20840039e0 0x7f2084003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f2084004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f20840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980b4830 con 0x7f2084004800 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208400bad0 con 0x7f20840039e0 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.898+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208400bcd0 con 0x7f20980714d0 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f2084004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f209659c640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20840039e0 0x7f2084003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f2096d9d640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f20840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 -- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 msgr2=0x7f20840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f20840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 -- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20840039e0 msgr2=0x7f2084003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20840039e0 0x7f2084003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.909 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 -- 172.21.15.148:0/1553979299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084003520 con 0x7f2084004800 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f2096d9d640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20980714d0 0x7f20840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.899+0000 7f208ffff640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f2084004c80 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f2080001720 tx=0x7f2084005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a93a2003a957fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080001370 con 0x7f2084004800 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20800052d0 con 0x7f2084004800 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f2095d9b640 1 -- 172.21.15.148:0/1553979299 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208002ab90 con 0x7f2084004800 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f209cb89640 1 -- 172.21.15.148:0/1553979299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 msgr2=0x7f2084004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f209cb89640 1 --2- 172.21.15.148:0/1553979299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f2084004c80 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7f2080001720 tx=0x7f2084005830 comp rx=0 tx=0).stop 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f2096d9d640 1 -- 172.21.15.148:0/1553979299 reap_dead start 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f209cb89640 1 -- 172.21.15.148:0/1553979299 shutdown_connections 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f209cb89640 1 -- 172.21.15.148:0/1553979299 >> 172.21.15.148:0/1553979299 conn(0x7f20980b3530 msgr2=0x7f20980b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f209cb89640 1 -- 172.21.15.148:0/1553979299 shutdown_connections 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.900+0000 7f209cb89640 1 -- 172.21.15.148:0/1553979299 wait complete. 2024-04-25T10:08:24.910 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.901+0000 7f209cb89640 1 Processor -- start 2024-04-25T10:08:24.911 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 -- start start 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f20981f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20981f1a40 0x7f2098151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2098152260 0x7f20981557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098145280 con 0x7f2084004800 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098145100 con 0x7f2098152260 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209cb89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20980b5cb0 con 0x7f20981f1a40 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f20981f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f20981f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43028/0 (socket says 172.21.15.148:43028) 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f209659c640 1 -- 172.21.15.148:0/2993798870 learned_addr learned my addr 172.21.15.148:0/2993798870 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f2096d9d640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2098152260 0x7f20981557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.914 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.902+0000 7f208ffff640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20981f1a40 0x7f2098151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209659c640 1 -- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20981f1a40 msgr2=0x7f2098151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209659c640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20981f1a40 0x7f2098151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209659c640 1 -- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2098152260 msgr2=0x7f20981557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209659c640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2098152260 0x7f20981557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209659c640 1 -- 172.21.15.148:0/2993798870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20981518c0 con 0x7f2084004800 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f2096d9d640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2098152260 0x7f20981557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f208ffff640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20981f1a40 0x7f2098151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209659c640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f20981f1500 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f2088004f40 tx=0x7f20880074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800f750 con 0x7f2084004800 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2088018070 con 0x7f2084004800 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2098152770 con 0x7f2084004800 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800fb10 con 0x7f2084004800 2024-04-25T10:08:24.915 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.903+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209814fe10 con 0x7f2084004800 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.904+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f209806b8d0 con 0x7f2084004800 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.904+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f208800fcb0 con 0x7f2084004800 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208effd640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207806b4d0 0x7f207806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f208808a080 con 0x7f2084004800 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208ffff640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207806b4d0 0x7f207806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208effd640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f2078071a10 0x7f2078073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 --> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2088009e50 con 0x7f2078071a10 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2088091050 con 0x7f2084004800 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f2096d9d640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f2078071a10 0x7f2078073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.905+0000 7f208ffff640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207806b4d0 0x7f207806d990 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2080032000 tx=0x7f208002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.906+0000 7f2096d9d640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f2078071a10 0x7f2078073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.916 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.906+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== osd.7 v2:172.21.15.188:6824/678316061 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2088009e50 con 0x7f2078071a10 2024-04-25T10:08:24.925 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.919+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 --> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2098073a00 con 0x7f2078071a10 2024-04-25T10:08:24.926 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.920+0000 7f208effd640 1 -- 172.21.15.148:0/2993798870 <== osd.7 v2:172.21.15.188:6824/678316061 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2098073a00 con 0x7f2078071a10 2024-04-25T10:08:24.926 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-25T10:08:24.926 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:08:24.926 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-25T10:08:24.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.920+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f2078071a10 msgr2=0x7f2078073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.920+0000 7f209cb89640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.188:6824/678316061,v1:172.21.15.188:6825/678316061] conn(0x7f2078071a10 0x7f2078073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207806b4d0 msgr2=0x7f207806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.927 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f207806b4d0 0x7f207806d990 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2080032000 tx=0x7f208002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 msgr2=0x7f20981f1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 --2- 172.21.15.148:0/2993798870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2084004800 0x7f20981f1500 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7f2088004f40 tx=0x7f20880074f0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f2096d9d640 1 -- 172.21.15.148:0/2993798870 reap_dead start 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 shutdown_connections 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 >> 172.21.15.148:0/2993798870 conn(0x7f20980b3530 msgr2=0x7f209813df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 shutdown_connections 2024-04-25T10:08:24.928 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:24.921+0000 7f209cb89640 1 -- 172.21.15.148:0/2993798870 wait complete. 2024-04-25T10:08:24.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:08:25.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.029+0000 7fd5c591d640 1 Processor -- start 2024-04-25T10:08:25.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.029+0000 7fd5c591d640 1 -- start start 2024-04-25T10:08:25.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5c591d640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd5c015dac0 0x7fd5c00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5c591d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5c00d0ab0 0x7fd5c00b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5c591d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c00d2420 con 0x7fd5c00d1f30 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5c591d640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00d3030 con 0x7fd5c015dac0 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5c591d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00d2700 con 0x7fd5c00d0ab0 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5beffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd5c015dac0 0x7fd5c00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5be7fc640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fd5c00d1f30 0x7fd5c00d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35864/0 (socket says 172.21.15.148:35864) 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5beffd640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7fd5c015dac0 0x7fd5c00d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43044/0 (socket says 172.21.15.148:43044) 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5be7fc640 1 -- 172.21.15.148:0/421476316 learned_addr learned my addr 172.21.15.148:0/421476316 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5bf7fe640 1 --2- 172.21.15.148:0/421476316 >> v2:172.21.15.188:3300/0 conn(0x7fd5c00d0ab0 0x7fd5c00b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5beffd640 1 -- 172.21.15.148:0/421476316 >> v2:172.21.15.188:3300/0 conn(0x7fd5c00d0ab0 msgr2=0x7fd5c00b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5beffd640 1 --2- 172.21.15.148:0/421476316 >> v2:172.21.15.188:3300/0 conn(0x7fd5c00d0ab0 0x7fd5c00b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5beffd640 1 -- 172.21.15.148:0/421476316 >> v1:172.21.15.148:6789/0 conn(0x7fd5c00d1f30 legacy=0x7fd5c00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5bdffb640 1 -- 172.21.15.148:0/421476316 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250966998 0 0) 0x7fd5c00d2420 con 0x7fd5c00d1f30 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.030+0000 7fd5beffd640 1 -- 172.21.15.148:0/421476316 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00d29e0 con 0x7fd5c015dac0 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.031+0000 7fd5beffd640 1 --2- 172.21.15.148:0/421476316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c015dac0 0x7fd5c00d19f0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00cf780 tx=0x7fd5b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11b0017a2dd63dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.031+0000 7fd5bdffb640 1 -- 172.21.15.148:0/421476316 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0037070 con 0x7fd5c015dac0 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.031+0000 7fd5bdffb640 1 -- 172.21.15.148:0/421476316 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0032440 con 0x7fd5c015dac0 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.031+0000 7fd5bdffb640 1 -- 172.21.15.148:0/421476316 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0032810 con 0x7fd5c015dac0 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 -- 172.21.15.148:0/421476316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c015dac0 msgr2=0x7fd5c00d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 --2- 172.21.15.148:0/421476316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c015dac0 0x7fd5c00d19f0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00cf780 tx=0x7fd5b002a230 comp rx=0 tx=0).stop 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 -- 172.21.15.148:0/421476316 shutdown_connections 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 --2- 172.21.15.148:0/421476316 >> v2:172.21.15.188:3300/0 conn(0x7fd5c00d0ab0 0x7fd5c00b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 --2- 172.21.15.148:0/421476316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c015dac0 0x7fd5c00d19f0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 -- 172.21.15.148:0/421476316 >> 172.21.15.148:0/421476316 conn(0x7fd5c0061c30 msgr2=0x7fd5c006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 -- 172.21.15.148:0/421476316 shutdown_connections 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.032+0000 7fd5c591d640 1 -- 172.21.15.148:0/421476316 wait complete. 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.033+0000 7fd5c591d640 1 Processor -- start 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.033+0000 7fd5c591d640 1 -- start start 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5c591d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5c00d0ab0 0x7fd5c01509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5c591d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 0x7fd5c00b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5c591d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c015dac0 0x7fd5c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5c591d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00b97d0 con 0x7fd5c00d1f30 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5c591d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c015dea0 con 0x7fd5c015dac0 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5c591d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00b9950 con 0x7fd5c00d0ab0 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 0x7fd5c00b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 0x7fd5c00b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43054/0 (socket says 172.21.15.148:43054) 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 -- 172.21.15.148:0/605460848 learned_addr learned my addr 172.21.15.148:0/605460848 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5beffd640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5c00d0ab0 0x7fd5c01509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5bf7fe640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c015dac0 0x7fd5c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 -- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5c00d0ab0 msgr2=0x7fd5c01509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5c00d0ab0 0x7fd5c01509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 -- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c015dac0 msgr2=0x7fd5c00b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c015dac0 0x7fd5c00b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5be7fc640 1 -- 172.21.15.148:0/605460848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00b3d30 con 0x7fd5c00d1f30 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.034+0000 7fd5bf7fe640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c015dac0 0x7fd5c00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.035+0000 7fd5be7fc640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 0x7fd5c00b33c0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400c500 tx=0x7fd5b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.035+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4007480 con 0x7fd5c00d1f30 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.035+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b40057a0 con 0x7fd5c00d1f30 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.035+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4007620 con 0x7fd5c00d1f30 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.035+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c00b4010 con 0x7fd5c00d1f30 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.035+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c00b45d0 con 0x7fd5c00d1f30 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.036+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c00b4ab0 con 0x7fd5c00d1f30 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.036+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5b4005b50 con 0x7fd5c00d1f30 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.037+0000 7fd5a7fff640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5ac06b4d0 0x7fd5ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.037+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd5b4089c70 con 0x7fd5c00d1f30 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.037+0000 7fd5beffd640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5ac06b4d0 0x7fd5ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.037+0000 7fd5beffd640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5ac06b4d0 0x7fd5ac06d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0034000 tx=0x7fd5b002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.041+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5c00b4ab0 con 0x7fd5c00d1f30 2024-04-25T10:08:25.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.218+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5c0073a00 con 0x7fd5ac06b4d0 2024-04-25T10:08:25.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.218+0000 7fd5a7fff640 1 -- 172.21.15.148:0/605460848 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd5c0073a00 con 0x7fd5ac06b4d0 2024-04-25T10:08:25.224 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:08:25.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5ac06b4d0 msgr2=0x7fd5ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.226 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5ac06b4d0 0x7fd5ac06d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0034000 tx=0x7fd5b002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 msgr2=0x7fd5c00b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 0x7fd5c00b33c0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400c500 tx=0x7fd5b400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 shutdown_connections 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fd5ac06b4d0 0x7fd5ac06d990 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c015dac0 0x7fd5c00b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00d1f30 0x7fd5c00b33c0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 --2- 172.21.15.148:0/605460848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5c00d0ab0 0x7fd5c01509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 >> 172.21.15.148:0/605460848 conn(0x7fd5c0061c30 msgr2=0x7fd5c006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 shutdown_connections 2024-04-25T10:08:25.227 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.220+0000 7fd5c591d640 1 -- 172.21.15.148:0/605460848 wait complete. 2024-04-25T10:08:25.237 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:25.333 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.327+0000 7f1ccd04d640 1 Processor -- start 2024-04-25T10:08:25.334 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.327+0000 7f1ccd04d640 1 -- start start 2024-04-25T10:08:25.335 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.328+0000 7f1ccd04d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80d0170 0x7f1cc80ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.328+0000 7f1ccd04d640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80c8030 con 0x7f1cc815da70 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.328+0000 7f1ccd04d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80c8310 con 0x7f1cc80ce700 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.328+0000 7f1ccd04d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80ceb50 con 0x7f1cc80d0170 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.328+0000 7f1cc6d9d640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f1cc815da70 0x7f1cc80c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:35878/0 (socket says 172.21.15.148:35878) 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.328+0000 7f1cc6d9d640 1 -- 172.21.15.148:0/229311425 learned_addr learned my addr 172.21.15.148:0/229311425 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc659c640 1 --2- 172.21.15.148:0/229311425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80d0170 0x7f1cc80ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175097065 0 0) 0x7f1cc80c8030 con 0x7f1cc815da70 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb0003520 con 0x7f1cc815da70 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786772197 0 0) 0x7f1cc80c8310 con 0x7f1cc80ce700 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc80c8030 con 0x7f1cc80ce700 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc659c640 1 -- 172.21.15.148:0/229311425 >> v1:172.21.15.188:6789/0 conn(0x7f1cc80ce700 legacy=0x7f1cc806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc659c640 1 -- 172.21.15.148:0/229311425 >> v1:172.21.15.148:6789/0 conn(0x7f1cc815da70 legacy=0x7f1cc80c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.336 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.329+0000 7f1cc659c640 1 -- 172.21.15.148:0/229311425 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80c85f0 con 0x7f1cc80d0170 2024-04-25T10:08:25.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1cc659c640 1 --2- 172.21.15.148:0/229311425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80d0170 0x7f1cc80ce1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097d0 tx=0x7f1cb802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a202c772c0dff1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8037070 con 0x7f1cc80d0170 2024-04-25T10:08:25.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8032440 con 0x7f1cc80d0170 2024-04-25T10:08:25.337 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1cc5d9b640 1 -- 172.21.15.148:0/229311425 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80327f0 con 0x7f1cc80d0170 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 -- 172.21.15.148:0/229311425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80d0170 msgr2=0x7f1cc80ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/229311425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80d0170 0x7f1cc80ce1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097d0 tx=0x7f1cb802a210 comp rx=0 tx=0).stop 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 -- 172.21.15.148:0/229311425 shutdown_connections 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/229311425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80d0170 0x7f1cc80ce1c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 -- 172.21.15.148:0/229311425 >> 172.21.15.148:0/229311425 conn(0x7f1cc80b33e0 msgr2=0x7f1cc80b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 -- 172.21.15.148:0/229311425 shutdown_connections 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.330+0000 7f1ccd04d640 1 -- 172.21.15.148:0/229311425 wait complete. 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.332+0000 7f1ccd04d640 1 Processor -- start 2024-04-25T10:08:25.338 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.332+0000 7f1ccd04d640 1 -- start start 2024-04-25T10:08:25.340 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80ce700 0x7f1cc815e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80d0170 0x7f1cc81f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 0x7f1cc81f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc815de50 con 0x7f1cc815da70 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc806be10 con 0x7f1cc80d0170 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc806ca40 con 0x7f1cc80ce700 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 0x7f1cc81f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 0x7f1cc81f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43062/0 (socket says 172.21.15.148:43062) 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 -- 172.21.15.148:0/1366863173 learned_addr learned my addr 172.21.15.148:0/1366863173 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc6d9d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80ce700 0x7f1cc815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc659c640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80d0170 0x7f1cc81f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 -- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80ce700 msgr2=0x7f1cc815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80ce700 0x7f1cc815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.341 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 -- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80d0170 msgr2=0x7f1cc81f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80d0170 0x7f1cc81f68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 -- 172.21.15.148:0/1366863173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc815e740 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1cc759e640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 0x7f1cc81f6df0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c790 tx=0x7f1cc000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003b70 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0003d10 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc815eb90 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000aa50 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.333+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc815f150 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.334+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8073990 con 0x7f1cc815da70 2024-04-25T10:08:25.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.335+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cc000abf0 con 0x7f1cc815da70 2024-04-25T10:08:25.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.335+0000 7f1caf7fe640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cb406b4d0 0x7f1cb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.335+0000 7f1cc6d9d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cb406b4d0 0x7f1cb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.335+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1cc0089f10 con 0x7f1cc815da70 2024-04-25T10:08:25.343 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.336+0000 7f1cc6d9d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cb406b4d0 0x7f1cb406d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0011e0 tx=0x7f1cbc007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.340+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cc0058860 con 0x7f1cc815da70 2024-04-25T10:08:25.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.507+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1cc815e970 con 0x7f1cb406b4d0 2024-04-25T10:08:25.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.508+0000 7f1caf7fe640 1 -- 172.21.15.148:0/1366863173 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13395 (secure 0 0 0) 0x7f1cc815e970 con 0x7f1cb406b4d0 2024-04-25T10:08:25.514 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:25.514 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cb406b4d0 msgr2=0x7f1cb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cb406b4d0 0x7f1cb406d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0011e0 tx=0x7f1cbc007d30 comp rx=0 tx=0).stop 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 msgr2=0x7f1cc81f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 0x7f1cc81f6df0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c790 tx=0x7f1cc000cc50 comp rx=0 tx=0).stop 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 shutdown_connections 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f1cb406b4d0 0x7f1cb406d990 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cc815da70 0x7f1cc81f6df0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1cc80d0170 0x7f1cc81f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 --2- 172.21.15.148:0/1366863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1cc80ce700 0x7f1cc815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 >> 172.21.15.148:0/1366863173 conn(0x7f1cc80b33e0 msgr2=0x7f1cc81582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.516 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 shutdown_connections 2024-04-25T10:08:25.517 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:25.510+0000 7f1ccd04d640 1 -- 172.21.15.148:0/1366863173 wait complete. 2024-04-25T10:08:25.526 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2024-04-25T10:08:25.197914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":474692,"kb_used_data":21256,"kb_used_omap":12,"kb_used_meta":453299,"kb_avail":754500028,"statfs":{"total":773094113280,"available":772608028672,"internally_reserved":0,"allocated":21766144,"data_stored":10844024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":464178736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12582912,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-8,"num_write_kb":-12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.513526"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":509,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-04-25T10:08:17.085397+0000","last_change":"2024-04-25T09:54:12.187273+0000","last_active":"2024-04-25T10:08:17.085397+0000","last_peered":"2024-04-25T10:08:17.085397+0000","last_clean":"2024-04-25T10:08:17.085397+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:08:17.085397+0000","last_undegraded":"2024-04-25T10:08:17.085397+0000","last_fullsized":"2024-04-25T10:08:17.085397+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:54:11.073286+0000","last_clean_scrub_stamp":"2024-04-25T09:54:11.073286+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:31.197519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56484,"kb_used_data":3092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315356,"statfs":{"total":96636764160,"available":96578924544,"internally_reserved":0,"allocated":3166208,"data_stored":1798279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55900,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315940,"statfs":{"total":96636764160,"available":96579522560,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51744,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320096,"statfs":{"total":96636764160,"available":96583778304,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60064,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82208,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289632,"statfs":{"total":96636764160,"available":96552583168,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55904,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315936,"statfs":{"total":96636764160,"available":96579518464,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55904,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315936,"statfs":{"total":96636764160,"available":96579518464,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575022,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56484,"kb_used_data":3092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315356,"statfs":{"total":96636764160,"available":96578924544,"internally_reserved":0,"allocated":3166208,"data_stored":1798279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:25.530 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T09:54:11.073286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=9, tm_min=54, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=8, tm_sec=20, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:08:25.530 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:08:26.314 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:26.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:45.531 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:45.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.634+0000 7f032dcc6640 1 Processor -- start 2024-04-25T10:08:45.641 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.634+0000 7f032dcc6640 1 -- start start 2024-04-25T10:08:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f032dcc6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0328154ee0 0x7f03280cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f032dcc6640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03280ce590 0x7f032806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f032dcc6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0328071480 con 0x7f03280cfcf0 2024-04-25T10:08:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f032dcc6640 1 -- --> v2:172.21.15.148:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0328072090 con 0x7f03280ce590 2024-04-25T10:08:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f032dcc6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03280d0150 con 0x7f0328154ee0 2024-04-25T10:08:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03280ce590 0x7f032806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 --2- >> v2:172.21.15.148:3300/0 conn(0x7f03280ce590 0x7f032806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43174/0 (socket says 172.21.15.148:43174) 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 -- 172.21.15.148:0/770309259 learned_addr learned my addr 172.21.15.148:0/770309259 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f03277fe640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0328154ee0 0x7f03280cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 -- 172.21.15.148:0/770309259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0328154ee0 msgr2=0x7f03280cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0328154ee0 0x7f03280cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 -- 172.21.15.148:0/770309259 >> v1:172.21.15.188:6789/0 conn(0x7f03280cfcf0 legacy=0x7f03280cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f0327fff640 1 -- 172.21.15.148:0/770309259 --> v2:172.21.15.148:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328071760 con 0x7f03280ce590 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.635+0000 7f03277fe640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0328154ee0 0x7f03280cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.636+0000 7f0327fff640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03280ce590 0x7f032806d6f0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f0320009e50 tx=0x7f032002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc930886bfe1613d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.636+0000 7f03267fc640 1 -- 172.21.15.148:0/770309259 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320036070 con 0x7f03280ce590 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.636+0000 7f03267fc640 1 -- 172.21.15.148:0/770309259 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320031600 con 0x7f03280ce590 2024-04-25T10:08:45.644 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.636+0000 7f03267fc640 1 -- 172.21.15.148:0/770309259 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200319d0 con 0x7f03280ce590 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 -- 172.21.15.148:0/770309259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03280ce590 msgr2=0x7f032806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03280ce590 0x7f032806d6f0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f0320009e50 tx=0x7f032002a710 comp rx=0 tx=0).stop 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 -- 172.21.15.148:0/770309259 shutdown_connections 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03280ce590 0x7f032806d6f0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 --2- 172.21.15.148:0/770309259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0328154ee0 0x7f03280cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 -- 172.21.15.148:0/770309259 >> 172.21.15.148:0/770309259 conn(0x7f032806abe0 msgr2=0x7f032806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 -- 172.21.15.148:0/770309259 shutdown_connections 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.637+0000 7f032dcc6640 1 -- 172.21.15.148:0/770309259 wait complete. 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.638+0000 7f032dcc6640 1 Processor -- start 2024-04-25T10:08:45.647 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.638+0000 7f032dcc6640 1 -- start start 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f032dcc6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03280ce590 0x7f0328159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f032dcc6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03280cfcf0 0x7f03281f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f032dcc6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 0x7f0328153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f032dcc6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328071d20 con 0x7f0328154ee0 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f032dcc6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328071230 con 0x7f03280cfcf0 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f032dcc6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03281552c0 con 0x7f03280ce590 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f0327fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 0x7f0328153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f0327fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 0x7f0328153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43188/0 (socket says 172.21.15.148:43188) 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f0327fff640 1 -- 172.21.15.148:0/3226449166 learned_addr learned my addr 172.21.15.148:0/3226449166 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f03277fe640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03280ce590 0x7f0328159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.639+0000 7f0326ffd640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03280cfcf0 0x7f03281f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f0327fff640 1 -- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03280ce590 msgr2=0x7f0328159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.648 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f0327fff640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03280ce590 0x7f0328159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f0327fff640 1 -- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03280cfcf0 msgr2=0x7f03281f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f0327fff640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03280cfcf0 0x7f03281f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f0327fff640 1 -- 172.21.15.148:0/3226449166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280a8780 con 0x7f0328154ee0 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f0327fff640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 0x7f0328153e40 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f0320009f70 tx=0x7f0320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032003f020 con 0x7f0328154ee0 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320037040 con 0x7f0328154ee0 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0328157ba0 con 0x7f0328154ee0 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320036070 con 0x7f0328154ee0 2024-04-25T10:08:45.649 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.640+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0328158440 con 0x7f0328154ee0 2024-04-25T10:08:45.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.641+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03280ce3b0 con 0x7f0328154ee0 2024-04-25T10:08:45.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.642+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0320042020 con 0x7f0328154ee0 2024-04-25T10:08:45.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.642+0000 7f030ffff640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f031406b4d0 0x7f031406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.642+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f03200aa550 con 0x7f0328154ee0 2024-04-25T10:08:45.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.642+0000 7f03277fe640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f031406b4d0 0x7f031406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.650 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.642+0000 7f03277fe640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f031406b4d0 0x7f031406d990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f032815db10 tx=0x7f0318008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.653 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.646+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03280ce3b0 con 0x7f0328154ee0 2024-04-25T10:08:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.812+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 --> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0328073990 con 0x7f031406b4d0 2024-04-25T10:08:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.813+0000 7f030ffff640 1 -- 172.21.15.148:0/3226449166 <== mgr.4113 v2:172.21.15.148:6832/2250922117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13386 (secure 0 0 0) 0x7f0328073990 con 0x7f031406b4d0 2024-04-25T10:08:45.820 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-25T10:08:45.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f031406b4d0 msgr2=0x7f031406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f031406b4d0 0x7f031406d990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f032815db10 tx=0x7f0318008040 comp rx=0 tx=0).stop 2024-04-25T10:08:45.822 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 msgr2=0x7f0328153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 0x7f0328153e40 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f0320009f70 tx=0x7f0320004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 shutdown_connections 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f031406b4d0 0x7f031406d990 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0328154ee0 0x7f0328153e40 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03280cfcf0 0x7f03281f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 --2- 172.21.15.148:0/3226449166 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03280ce590 0x7f0328159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 >> 172.21.15.148:0/3226449166 conn(0x7f032806abe0 msgr2=0x7f03280af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 shutdown_connections 2024-04-25T10:08:45.823 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.815+0000 7f032dcc6640 1 -- 172.21.15.148:0/3226449166 wait complete. 2024-04-25T10:08:45.833 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2024-04-25T10:08:45.201665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":474676,"kb_used_data":21256,"kb_used_omap":12,"kb_used_meta":453299,"kb_avail":754500044,"statfs":{"total":773094113280,"available":772608045056,"internally_reserved":0,"allocated":21766144,"data_stored":10844024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":464178736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002263"},"pg_stats":[{"pgid":"1.0","version":"203'119","reported_seq":520,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-04-25T10:08:25.530681+0000","last_change":"2024-04-25T10:08:25.530716+0000","last_active":"2024-04-25T10:08:25.530681+0000","last_peered":"2024-04-25T10:08:25.530681+0000","last_clean":"2024-04-25T10:08:25.530681+0000","last_became_active":"2024-04-25T09:54:12.186354+0000","last_became_peered":"2024-04-25T09:54:12.186354+0000","last_unstale":"2024-04-25T10:08:25.530681+0000","last_undegraded":"2024-04-25T10:08:25.530681+0000","last_fullsized":"2024-04-25T10:08:25.530681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'119","last_scrub_stamp":"2024-04-25T10:08:25.530592+0000","last_deep_scrub":"203'119","last_deep_scrub_stamp":"2024-04-25T10:08:25.530592+0000","last_clean_scrub_stamp":"2024-04-25T10:08:25.530592+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:55.359139+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575026,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56476,"kb_used_data":3092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315364,"statfs":{"total":96636764160,"available":96578932736,"internally_reserved":0,"allocated":3166208,"data_stored":1798279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575026,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55900,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315940,"statfs":{"total":96636764160,"available":96579522560,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51744,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320096,"statfs":{"total":96636764160,"available":96583778304,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575026,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60064,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311776,"statfs":{"total":96636764160,"available":96575258624,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82208,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94289632,"statfs":{"total":96636764160,"available":96552583168,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81590727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55904,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315936,"statfs":{"total":96636764160,"available":96579518464,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575026,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55904,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315936,"statfs":{"total":96636764160,"available":96579518464,"internally_reserved":0,"allocated":2572288,"data_stored":1207911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575026,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56476,"kb_used_data":3092,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315364,"statfs":{"total":96636764160,"available":96578932736,"internally_reserved":0,"allocated":3166208,"data_stored":1798279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:45.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:08:45.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.924+0000 7fb564407640 1 Processor -- start 2024-04-25T10:08:45.932 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.924+0000 7fb564407640 1 -- start start 2024-04-25T10:08:45.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb564407640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb55c0ced80 0x7fb55c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb564407640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 0x7fb55c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb564407640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c15e000 con 0x7fb55c15dc20 2024-04-25T10:08:45.935 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb564407640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55c0d1de0 con 0x7fb55c0ced80 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb564407640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c161c60 con 0x7fb55c161710 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb5621a3640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7fb55c15dc20 0x7fb55c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41176/0 (socket says 172.21.15.148:41176) 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.925+0000 7fb5621a3640 1 -- 172.21.15.148:0/3580923115 learned_addr learned my addr 172.21.15.148:0/3580923115 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625689932 0 0) 0x7fb55c15e000 con 0x7fb55c15dc20 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb544003520 con 0x7fb55c15dc20 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 0x7fb55c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5619a2640 1 --2- 172.21.15.148:0/3580923115 >> v2:172.21.15.188:3300/0 conn(0x7fb55c0ced80 0x7fb55c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1093261569 0 0) 0x7fb544003520 con 0x7fb55c15dc20 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 msgr2=0x7fb55c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 0x7fb55c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.936 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 >> v2:172.21.15.188:3300/0 conn(0x7fb55c0ced80 msgr2=0x7fb55c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 --2- 172.21.15.148:0/3580923115 >> v2:172.21.15.188:3300/0 conn(0x7fb55c0ced80 0x7fb55c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c161f40 con 0x7fb55c15dc20 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7fb554001670 con 0x7fb55c15dc20 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5619a2640 1 --2- 172.21.15.148:0/3580923115 >> v2:172.21.15.188:3300/0 conn(0x7fb55c0ced80 0x7fb55c0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 >> v1:172.21.15.148:6789/0 conn(0x7fb55c15dc20 legacy=0x7fb55c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5440039b0 0x7fb544003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 0x7fb544004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0d1de0 con 0x7fb544004d40 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c161c60 con 0x7fb5440039b0 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400ba70 con 0x7fb55c161710 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5619a2640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 0x7fb544004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.937 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5621a3640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5440039b0 0x7fb544003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 -- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 msgr2=0x7fb544004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c161710 0x7fb544004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 -- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5440039b0 msgr2=0x7fb544003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5440039b0 0x7fb544003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5629a4640 1 -- 172.21.15.148:0/3580923115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544003520 con 0x7fb544004d40 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.926+0000 7fb5621a3640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5440039b0 0x7fb544003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.927+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00ba60 tx=0x7fb544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a293feb4ba26979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.927+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c030070 con 0x7fb544004d40 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.927+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c02ae10 con 0x7fb544004d40 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.927+0000 7fb5611a1640 1 -- 172.21.15.148:0/3580923115 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00b600 con 0x7fb544004d40 2024-04-25T10:08:45.938 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb564407640 1 -- 172.21.15.148:0/3580923115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb544004d40 msgr2=0x7fb5440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb564407640 1 --2- 172.21.15.148:0/3580923115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb544004d40 0x7fb5440051c0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00ba60 tx=0x7fb544005830 comp rx=0 tx=0).stop 2024-04-25T10:08:45.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb5629a4640 1 -- 172.21.15.148:0/3580923115 reap_dead start 2024-04-25T10:08:45.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb564407640 1 -- 172.21.15.148:0/3580923115 shutdown_connections 2024-04-25T10:08:45.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb564407640 1 -- 172.21.15.148:0/3580923115 >> 172.21.15.148:0/3580923115 conn(0x7fb55c0d0420 msgr2=0x7fb55c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb564407640 1 -- 172.21.15.148:0/3580923115 shutdown_connections 2024-04-25T10:08:45.939 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.928+0000 7fb564407640 1 -- 172.21.15.148:0/3580923115 wait complete. 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.929+0000 7fb564407640 1 Processor -- start 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.929+0000 7fb564407640 1 -- start start 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb544004d40 0x7fb55c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c152b70 0x7fb55c0b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 0x7fb55c0b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0b5e50 con 0x7fb55c152ff0 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c162500 con 0x7fb544004d40 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0b5fd0 con 0x7fb55c152b70 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 0x7fb55c0b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 0x7fb55c0b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43208/0 (socket says 172.21.15.148:43208) 2024-04-25T10:08:45.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 -- 172.21.15.148:0/3717588811 learned_addr learned my addr 172.21.15.148:0/3717588811 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 -- 172.21.15.148:0/3717588811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c152b70 msgr2=0x7fb55c0b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c152b70 0x7fb55c0b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 -- 172.21.15.148:0/3717588811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb544004d40 msgr2=0x7fb55c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb544004d40 0x7fb55c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 -- 172.21.15.148:0/3717588811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0cca40 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb5629a4640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 0x7fb55c0b8c30 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fb54c02bfd0 tx=0x7fb54c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00b600 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c02ac10 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c0cbec0 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004260 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.930+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c0cc480 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.931+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c073a00 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.932+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb54c045030 con 0x7fb55c152ff0 2024-04-25T10:08:45.943 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.932+0000 7fb55affd640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb54806b4d0 0x7fb54806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.932+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb54c0aa130 con 0x7fb55c152ff0 2024-04-25T10:08:45.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.932+0000 7fb5621a3640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb54806b4d0 0x7fb54806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.933+0000 7fb5621a3640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb54806b4d0 0x7fb54806d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0d43d0 tx=0x7fb554000ef0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.944 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:45.936+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb54c04c020 con 0x7fb55c152ff0 2024-04-25T10:08:46.112 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.105+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fb55c0d16b0 con 0x7fb55c152ff0 2024-04-25T10:08:46.115 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.109+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7fb54c0790d0 con 0x7fb55c152ff0 2024-04-25T10:08:46.117 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.110+0000 7fb55affd640 1 -- 172.21.15.148:0/3717588811 <== mon.0 v2:172.21.15.148:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb54c0789d0 con 0x7fb55c152ff0 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb54806b4d0 msgr2=0x7fb54806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb54806b4d0 0x7fb54806d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0d43d0 tx=0x7fb554000ef0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 msgr2=0x7fb55c0b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 0x7fb55c0b8c30 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fb54c02bfd0 tx=0x7fb54c004730 comp rx=0 tx=0).stop 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 shutdown_connections 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7fb54806b4d0 0x7fb54806d990 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb55c152ff0 0x7fb55c0b8c30 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.119 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb55c152b70 0x7fb55c0b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 --2- 172.21.15.148:0/3717588811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb544004d40 0x7fb55c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 >> 172.21.15.148:0/3717588811 conn(0x7fb55c0d0420 msgr2=0x7fb55c0c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 shutdown_connections 2024-04-25T10:08:46.120 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.112+0000 7fb564407640 1 -- 172.21.15.148:0/3717588811 wait complete. 2024-04-25T10:08:46.130 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:46.131 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.221+0000 7f7120117640 1 Processor -- start 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.221+0000 7f7120117640 1 -- start start 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f7120117640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 0x7f71181625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f7120117640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71180cc8e0 0x7f711815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f7120117640 1 -- --> v1:172.21.15.148:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711815f390 con 0x7f71180d1810 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f7120117640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180cce50 con 0x7f71180cc8e0 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f7120117640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7118161af0 con 0x7f711815dc90 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711e6b4640 1 --1- >> v1:172.21.15.148:6789/0 conn(0x7f71180d1810 0x7f71180cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.148:6789/0 says I am v1:172.21.15.148:41186/0 (socket says 172.21.15.148:41186) 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711e6b4640 1 -- 172.21.15.148:0/2709831011 learned_addr learned my addr 172.21.15.148:0/2709831011 (peer_addr_for_me v1:172.21.15.148:0/0) 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 <== mon.0 v1:172.21.15.148:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594016384 0 0) 0x7f7110001490 con 0x7f71180d1810 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711deb3640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 0x7f71181625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 --> v1:172.21.15.148:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7104003520 con 0x7f71180d1810 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711d6b2640 1 --2- 172.21.15.148:0/2709831011 >> v2:172.21.15.188:3300/0 conn(0x7f71180cc8e0 0x7f711815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 <== mon.0 v1:172.21.15.148:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746705467 0 0) 0x7f7104003520 con 0x7f71180d1810 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 msgr2=0x7f71181625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 0x7f71181625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 >> v2:172.21.15.188:3300/0 conn(0x7f71180cc8e0 msgr2=0x7f711815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 --2- 172.21.15.148:0/2709831011 >> v2:172.21.15.188:3300/0 conn(0x7f71180cc8e0 0x7f711815ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 --> v1:172.21.15.148:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118161dd0 con 0x7f71180d1810 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 <== mon.0 v1:172.21.15.148:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1485415081 0 0) 0x7f7110001bc0 con 0x7f71180d1810 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 >> v1:172.21.15.148:6789/0 conn(0x7f71180d1810 legacy=0x7f71180cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71040039b0 0x7f7104003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.222+0000 7f711ceb1640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 0x7f7104004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711ceb1640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7104004d40 0x7f71040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180cce50 con 0x7f71040039b0 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118161af0 con 0x7f7104004d40 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710400b9c0 con 0x7f711815dc90 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711deb3640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71040039b0 0x7f7104003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711e6b4640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7104004d40 0x7f71040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711d6b2640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 0x7f7104004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711deb3640 1 -- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 msgr2=0x7f7104004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711deb3640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711815dc90 0x7f7104004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711deb3640 1 -- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7104004d40 msgr2=0x7f71040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711deb3640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7104004d40 0x7f71040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.223+0000 7f711deb3640 1 -- 172.21.15.148:0/2709831011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104003520 con 0x7f71040039b0 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f711deb3640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71040039b0 0x7f7104003dd0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f7114009ac0 tx=0x7f7104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0627c95e23c8e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114036070 con 0x7f71040039b0 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7114031400 con 0x7f71040039b0 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f711ceb1640 1 -- 172.21.15.148:0/2709831011 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114009690 con 0x7f71040039b0 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f7120117640 1 -- 172.21.15.148:0/2709831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71040039b0 msgr2=0x7f7104003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f7120117640 1 --2- 172.21.15.148:0/2709831011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71040039b0 0x7f7104003dd0 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f7114009ac0 tx=0x7f7104005830 comp rx=0 tx=0).stop 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f711e6b4640 1 -- 172.21.15.148:0/2709831011 reap_dead start 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f7120117640 1 -- 172.21.15.148:0/2709831011 shutdown_connections 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f7120117640 1 -- 172.21.15.148:0/2709831011 >> 172.21.15.148:0/2709831011 conn(0x7f711806f590 msgr2=0x7f711806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f7120117640 1 -- 172.21.15.148:0/2709831011 shutdown_connections 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.224+0000 7f7120117640 1 -- 172.21.15.148:0/2709831011 wait complete. 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 Processor -- start 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 -- start start 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71040039b0 0x7f71180c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71180b71d0 0x7f71180b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 0x7f71181f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711815e070 con 0x7f71180b4a20 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180cd130 con 0x7f71040039b0 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f7120117640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711815e1f0 con 0x7f71180b71d0 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f711e6b4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 0x7f71181f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f711e6b4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 0x7f71181f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:43216/0 (socket says 172.21.15.148:43216) 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.226+0000 7f711e6b4640 1 -- 172.21.15.148:0/3397905654 learned_addr learned my addr 172.21.15.148:0/3397905654 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711deb3640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71040039b0 0x7f71180c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711d6b2640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71180b71d0 0x7f71180b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711e6b4640 1 -- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71180b71d0 msgr2=0x7f71180b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711e6b4640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71180b71d0 0x7f71180b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711e6b4640 1 -- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71040039b0 msgr2=0x7f71180c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711e6b4640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71040039b0 0x7f71180c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711e6b4640 1 -- 172.21.15.148:0/3397905654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71181f4eb0 con 0x7f71180b4a20 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711deb3640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71040039b0 0x7f71180c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711d6b2640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71180b71d0 0x7f71180b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f711e6b4640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 0x7f71181f4130 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f71180d2920 tx=0x7f711000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110017020 con 0x7f71180b4a20 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f71100077a0 con 0x7f71180b4a20 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71180b66f0 con 0x7f71180b4a20 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71100133f0 con 0x7f71180b4a20 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.227+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71180b7630 con 0x7f71180b4a20 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.228+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118073990 con 0x7f71180b4a20 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.228+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f711001a020 con 0x7f71180b4a20 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.229+0000 7f710e7fc640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70fc06b500 0x7f70fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.229+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f711008a7d0 con 0x7f71180b4a20 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.229+0000 7f711deb3640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70fc06b500 0x7f70fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.229+0000 7f711deb3640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70fc06b500 0x7f70fc06d9c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7114033000 tx=0x7f7114008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.242 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.233+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7110021020 con 0x7f71180b4a20 2024-04-25T10:08:46.445 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.438+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71180c9350 con 0x7f71180b4a20 2024-04-25T10:08:46.446 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.439+0000 7f710e7fc640 1 -- 172.21.15.148:0/3397905654 <== mon.0 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+3718 (secure 0 0 0) 0x7f711815f390 con 0x7f71180b4a20 2024-04-25T10:08:46.446 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-25T10:08:46.446 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":121,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9298,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/2652029769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2652029769},{"type":"v1","addr":"172.21.15.188:6834","nonce":2652029769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":9433,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3302278780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":3302278780},{"type":"v1","addr":"172.21.15.188:6835","nonce":3302278780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115},{"gid":11616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2099698830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2099698830},{"type":"v1","addr":"172.21.15.148:6835","nonce":2099698830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":12111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2891846384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2891846384},{"type":"v1","addr":"172.21.15.148:6837","nonce":2891846384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":121}],"filesystems":[]} 2024-04-25T10:08:46.446 INFO:teuthology.orchestra.run.smithi148.stderr:dumped fsmap epoch 121 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70fc06b500 msgr2=0x7f70fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70fc06b500 0x7f70fc06d9c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7114033000 tx=0x7f7114008030 comp rx=0 tx=0).stop 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 msgr2=0x7f71181f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 0x7f71181f4130 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f71180d2920 tx=0x7f711000af80 comp rx=0 tx=0).stop 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 shutdown_connections 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:6832/2250922117,v1:172.21.15.148:6833/2250922117] conn(0x7f70fc06b500 0x7f70fc06d9c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.448 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71180b4a20 0x7f71181f4130 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71180b71d0 0x7f71180b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 --2- 172.21.15.148:0/3397905654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71040039b0 0x7f71180c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 >> 172.21.15.148:0/3397905654 conn(0x7f711806f590 msgr2=0x7f711806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 shutdown_connections 2024-04-25T10:08:46.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-25T10:08:46.441+0000 7f7120117640 1 -- 172.21.15.148:0/3397905654 wait complete. 2024-04-25T10:08:46.459 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:08:46.459 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:08:46.459 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:46.459 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.468 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:46.468 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:46.468 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.517 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:46.517 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.529 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:46.529 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:46.529 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.580 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:46.580 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:08:46.581 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:08:46.581 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.611 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:08:46.612 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:08:46.612 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.678 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:08:46.679 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:08:46.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.747 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:08:46.747 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:08:46.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.819 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:08:46.819 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:08:46.819 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.852 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:08:46.852 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:08:46.852 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:46.931 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:08:46.932 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:08:46.932 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.002 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:08:47.003 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:08:47.003 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.083 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:08:47.083 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:08:47.084 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:08:47.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.109 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:08:47.110 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:08:47.110 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.148 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:08:47.148 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:08:47.149 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:08:47.149 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.161 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:08:47.162 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:08:47.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.200 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:08:47.201 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:08:47.201 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:47.252 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:08:47.253 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:08:47.253 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T10:08:47.292 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:08:47.292 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:08:47.493 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:08:47.493 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:08:47.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:08:47.599 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:08:47.656 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:08:47.657 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:08:47.716 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T10:08:47.716 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:08:47.861 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T10:08:47.861 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:08:47.936 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T10:08:47.936 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:08:47.977 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T10:08:47.978 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:08:48.079 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:08:48.080 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/data/mon.a.tgz 2024-04-25T10:08:48.080 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-04-25T10:08:48.095 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.otnvaTAgmx 2024-04-25T10:08:48.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.otnvaTAgmx 2024-04-25T10:08:48.272 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.otnvaTAgmx 2024-04-25T10:08:48.353 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.otnvaTAgmx is 532KB 2024-04-25T10:08:48.454 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.otnvaTAgmx 2024-04-25T10:08:48.469 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/data/mon.b.tgz 2024-04-25T10:08:48.469 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-04-25T10:08:48.487 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.Ot1oUzg4dz 2024-04-25T10:08:48.487 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Ot1oUzg4dz 2024-04-25T10:08:48.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.Ot1oUzg4dz 2024-04-25T10:08:48.743 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.Ot1oUzg4dz is 542KB 2024-04-25T10:08:48.841 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.Ot1oUzg4dz 2024-04-25T10:08:48.855 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/data/mon.c.tgz 2024-04-25T10:08:48.855 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-04-25T10:08:48.910 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.SD28pXXFUS 2024-04-25T10:08:48.910 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SD28pXXFUS 2024-04-25T10:08:49.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.SD28pXXFUS 2024-04-25T10:08:49.156 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.SD28pXXFUS is 542KB 2024-04-25T10:08:49.256 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.SD28pXXFUS 2024-04-25T10:08:49.271 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:08:49.271 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:08:49.274 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:08:49.353 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:08:49.389 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:08:49.389 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:08:49.391 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/remote/smithi148/crash 2024-04-25T10:08:49.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:08:49.417 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/remote/smithi188/crash 2024-04-25T10:08:49.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:08:49.452 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:08:49.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:49.462 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:49.540 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:49.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:49.588 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:08:49.588 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:08:49.607 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34853.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41266.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34853.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34853.log.gz 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.41266.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41266.log.gz 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:08:49.685 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40874.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.40874.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.1.40874.log.gz 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34940.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.c.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34900.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34940.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34940.log.gz 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34900.log: 94.0% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34900.log.gz 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34769.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40879.log 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34769.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34769.log.gz 2024-04-25T10:08:49.686 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.40879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40879.log.gz 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34980.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34729.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34809.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34980.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34980.log.gz 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41270.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.34809.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34809.log.gz 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35020.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.41270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41270.log.gz 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr: --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:08:49.687 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.35020.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35020.log.gz 2024-04-25T10:08:49.711 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.b.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:08:49.897 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80412.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62712.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48464.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62712.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80412.log.gz 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85512.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3%/var/log/ceph/ceph-client.admin.48464.log: -- replaced with /var/log/ceph/ceph-client.admin.62712.log.gz 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57380.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48464.log.gz 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100531.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82118.log 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85512.log.gz 2024-04-25T10:08:49.898 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99678.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57380.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100531.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.82118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48968.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82118.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80580.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99678.log: /var/log/ceph/ceph-client.admin.48968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68643.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63892.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.99678.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84693.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80580.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63892.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68643.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114038.log 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63892.log.gz 2024-04-25T10:08:49.899 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106862.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106568.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84693.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113783.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114038.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108754.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106862.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113783.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106568.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104059.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108754.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113783.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75651.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104059.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.108754.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43138.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104059.log.gz 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89711.log 2024-04-25T10:08:49.900 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75651.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107599.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43138.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49196.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47792.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89711.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48296.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107599.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47792.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.49196.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77377.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102035.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47792.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48296.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90596.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102035.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77377.log.gz 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50297.log 2024-04-25T10:08:49.901 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102035.log.gz 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63295.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50297.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90596.log.gz -5 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68039.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63295.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50297.log.gz 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66530.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63295.log.gz 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96856.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68039.log.gz 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42598.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96856.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66530.log.gz 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77527.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96856.log.gz 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65118.log 2024-04-25T10:08:49.902 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42598.log.gz 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113205.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65118.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.77527.log.gz --verbose 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.admin.62366.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113205.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65118.log.gz 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67562.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62366.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113205.log.gz 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50129.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62366.log.gz/var/log/ceph/ceph-client.admin.67562.log: 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50129.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67562.log.gz 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90848.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49070.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50129.log.gz 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62101.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90848.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107473.log 2024-04-25T10:08:49.903 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62101.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90848.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106946.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48176.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89111.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62101.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107473.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48176.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106946.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63193.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48176.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79461.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81273.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101909.log 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63193.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89111.log.gz 2024-04-25T10:08:49.904 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75861.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101909.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.81273.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.79880.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.79461.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101909.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84609.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.75861.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85302.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81147.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79880.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46175.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84609.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85302.log.gz 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75735.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86636.log 2024-04-25T10:08:49.905 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81147.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75735.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46175.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48422.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86636.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75735.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97477.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86636.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58123.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89753.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48422.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46805.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97477.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89753.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58123.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43096.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46805.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89753.log.gz 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112521.log 2024-04-25T10:08:49.906 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43096.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46805.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104484.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43096.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107780.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86274.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107780.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104484.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43485.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.112521.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107780.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67818.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86274.log: /var/log/ceph/ceph-client.admin.43485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97079.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86274.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67818.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43485.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112934.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67818.log.gz 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102586.log 2024-04-25T10:08:49.907 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103578.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97079.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112934.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.102586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102586.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44692.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86941.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103704.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103578.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105286.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44692.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91693.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86941.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105286.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103704.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89453.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.105286.log.gz 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80847.log 2024-04-25T10:08:49.908 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57726.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91693.log.gz 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80847.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.89453.log.gz -- 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.105094.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105477.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80847.log.gz 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105094.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57726.log.gz 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60773.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105094.log.gz 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.105477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92380.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105477.log.gz 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76475.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80058.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113733.log 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76475.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60773.log.gz 2024-04-25T10:08:49.909 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42766.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.92380.log.gz 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76475.log.gz 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113733.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80058.log.gz 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40960.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113733.log.gz/var/log/ceph/ceph-client.admin.42766.log: 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58291.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42766.log.gz 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.40960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108118.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99809.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40960.log.gz 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108118.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.58291.log.gz --verbose 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.admin.84567.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108118.log.gz/var/log/ceph/ceph-client.admin.99809.log: 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94657.log 2024-04-25T10:08:49.910 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81823.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94657.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84567.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100657.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.99809.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84735.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94657.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81823.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90418.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100657.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97333.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84735.log: /var/log/ceph/ceph-client.admin.90418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96730.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84735.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90418.log.gz 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85260.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95257.log 2024-04-25T10:08:49.911 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97333.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46763.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96730.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95257.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85260.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81065.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46763.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.95257.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112197.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47183.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46763.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81065.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112197.log: /var/log/ceph/ceph-client.admin.47183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102921.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43486.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47183.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100363.log 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.112197.log.gz 2024-04-25T10:08:49.912 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107729.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44156.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107729.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100363.log.gz 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94549.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.102921.log.gz 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107729.log.gz 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107685.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94549.log: gzip 88.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44156.log.gz -- 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.44398.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65353.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107685.log.gz 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.94549.log.gz 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56919.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108635.log 2024-04-25T10:08:49.913 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44398.log.gz 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56919.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65353.log.gzgzip 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107557.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113911.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1%/var/log/ceph/ceph-client.admin.107557.log: -- replaced with /var/log/ceph/ceph-client.admin.56919.log.gz 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63151.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108635.log.gz 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65679.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107557.log.gz 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63151.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113911.log.gz -5 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106013.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65679.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63151.log.gz 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105822.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65679.log.gz 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77335.log 2024-04-25T10:08:49.914 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106013.log: /var/log/ceph/ceph-client.admin.105822.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106013.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86899.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.77335.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.admin.50339.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105822.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60091.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50339.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.0.87362.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86899.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50339.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.87362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99541.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87362.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43222.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60091.log.gz 2024-04-25T10:08:49.915 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99541.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.84956.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99541.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99874.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83865.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99874.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84956.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96086.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.83865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42540.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.99874.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96086.log.gz 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48506.log 2024-04-25T10:08:49.916 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42540.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83865.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91142.log 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42540.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41656.log 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48506.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44650.log/var/log/ceph/ceph-client.admin.91142.log: 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41362.log 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41656.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91142.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74951.log 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44650.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45046.log 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41656.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.74951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101823.log 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41362.log.gz 2024-04-25T10:08:49.917 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46721.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101823.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45046.log.gz 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61369.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.74951.log.gz 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46721.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.101823.log.gz --verbose 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.admin.43487.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95394.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46721.log.gz 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81487.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43487.log.gz 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61369.log.gz 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57600.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95394.log: /var/log/ceph/ceph-client.admin.81487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50673.log 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95394.log.gz 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57600.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81487.log.gzgzip 2024-04-25T10:08:49.918 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62838.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57600.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97123.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76132.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50673.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68375.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62838.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76132.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97123.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95824.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110833.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76132.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95824.log: gzip -5 --verbose 89.5% -- /var/log/ceph/ceph-client.admin.48050.log -- replaced with /var/log/ceph/ceph-client.admin.68375.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47420.log 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95824.log.gz 2024-04-25T10:08:49.919 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111939.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47420.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.48050.log.gz --verbose 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.110833.log.gz 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47420.log.gz 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103975.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111939.log.gz 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5/var/log/ceph/ceph-osd.2.log: --verbose -- /var/log/ceph/ceph-client.admin.80328.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64790.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76769.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103975.log.gz 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80328.log.gz 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106526.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.64790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64790.log.gz 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63762.log 2024-04-25T10:08:49.920 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106526.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76769.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58981.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106526.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86403.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58981.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63762.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100573.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58981.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105677.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86403.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103274.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105677.log: 89.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.100573.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101270.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103274.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105677.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42142.log 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103274.log.gz 2024-04-25T10:08:49.921 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79659.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101270.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106180.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79659.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43814.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79659.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95115.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43814.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106180.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89177.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.42142.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43814.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84914.log 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95115.log.gz 2024-04-25T10:08:49.922 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57558.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67688.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57558.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84914.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49154.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89177.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47309.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57558.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67688.log: /var/log/ceph/ceph-client.admin.49154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61237.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67688.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47309.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.49154.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.admin.60604.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61237.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47309.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109443.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110707.log 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109443.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61237.log.gz 2024-04-25T10:08:49.923 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106736.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60604.log.gz 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109443.log.gz 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81571.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110707.log.gz 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94971.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81571.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106736.log.gz 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62586.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64113.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81571.log.gz 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62586.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94971.log.gz 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62586.log.gzgzip 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64239.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114164.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log 2024-04-25T10:08:49.924 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64113.log.gz 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114164.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64239.log.gz 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110899.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43488.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114164.log.gzgzip 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80622.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110899.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63253.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110899.log.gz 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46847.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80622.log.gz 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61692.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63253.log.gz 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62450.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61692.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.46847.log.gz -- 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.74819.log 2024-04-25T10:08:49.925 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58939.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62450.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.74819.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61692.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89369.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96995.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58939.log.gz/var/log/ceph/ceph-client.admin.89369.log: 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.74819.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97249.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89369.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67774.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96995.log: /var/log/ceph/ceph-client.admin.97249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108922.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67774.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97249.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96995.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67774.log.gz 2024-04-25T10:08:49.926 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108838.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86773.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108922.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108838.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43483.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108838.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85638.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86773.log: /var/log/ceph/ceph-client.admin.43483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65978.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86773.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85638.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43483.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43012.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85638.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.65978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79964.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65978.log.gz 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103452.log 2024-04-25T10:08:49.927 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79964.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43012.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66072.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103452.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.79964.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.91268.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84483.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103452.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49610.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91268.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84483.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.66072.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106138.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49610.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84483.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87067.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49610.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76216.log 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106138.log.gz 2024-04-25T10:08:49.928 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91863.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76216.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.87067.log.gz -5 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.107908.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91863.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76216.log.gz 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80805.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91863.log.gz 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68566.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107908.log.gz 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41614.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.68566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112850.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80805.log.gz 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68566.log.gz 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45004.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8%/var/log/ceph/ceph-client.admin.112850.log: -- replaced with /var/log/ceph/ceph-client.admin.41614.log.gz 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:08:49.929 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84183.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112850.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43474.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45004.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84351.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84183.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89537.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84351.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43474.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43180.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104714.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89537.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.84351.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48218.log 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89537.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43180.log.gz 2024-04-25T10:08:49.930 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57894.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104714.log: /var/log/ceph/ceph-client.admin.48218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62922.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104714.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48218.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44776.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57894.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44920.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113996.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62922.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44920.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44776.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108455.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113996.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44920.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80496.log 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.113996.log.gz 2024-04-25T10:08:49.931 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.108455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108455.log.gz 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48800.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59319.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42928.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80496.log.gz 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59319.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48800.log.gz 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80370.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42928.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59319.log.gz 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65351.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80370.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42928.log.gz 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65635.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80370.log.gz 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44440.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65635.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.65351.log.gz -- 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.63985.log 2024-04-25T10:08:49.932 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65635.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97291.log 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44440.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57684.log 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114290.log 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63985.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66656.log 89.1% 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97291.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114290.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57684.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62408.log 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114290.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62324.log 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103494.log 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66656.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62408.log.gz 2024-04-25T10:08:49.933 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40875.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62324.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76957.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.40875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86491.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103494.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40875.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109495.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76957.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47225.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75777.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86491.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109495.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61927.log 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47225.log.gz 2024-04-25T10:08:49.934 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57642.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61927.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.75777.log.gz 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47876.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91226.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47876.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57642.log.gz 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61927.log.gz 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62167.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47876.log.gz 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95478.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62167.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91226.log.gz 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41172.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95478.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.62167.log.gz --verbose 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.0.68562.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95478.log.gz 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107198.log 2024-04-25T10:08:49.935 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41172.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.68562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95352.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68562.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111481.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106442.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107198.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111481.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95352.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65721.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111481.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75389.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65721.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106442.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92128.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102729.log 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92128.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65721.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75389.log.gz 2024-04-25T10:08:49.936 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107030.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92128.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100057.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.102729.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103662.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80889.log 87.9% 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107030.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103662.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100057.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83799.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103662.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48380.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.83799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106306.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80889.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107293.log 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48380.log.gz 2024-04-25T10:08:49.937 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106306.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83799.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77718.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107293.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.106306.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58897.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107293.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.77718.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.0.77718.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.60525.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113522.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60525.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58897.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102395.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113522.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60525.log.gz -5 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97375.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113522.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79796.log 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102395.log.gz 2024-04-25T10:08:49.938 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63678.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97375.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49526.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90722.log 89.3% 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79796.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63678.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104402.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49526.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44158.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104402.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90722.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61739.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104402.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104628.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44158.log.gz 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83931.log 2024-04-25T10:08:49.939 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81613.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61739.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104628.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.83931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87366.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95656.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.87366.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83931.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81613.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87366.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59067.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67053.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95656.log: /var/log/ceph/ceph-client.admin.59067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95866.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95656.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67053.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59067.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101739.log 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95866.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67053.log.gz 2024-04-25T10:08:49.940 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84872.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95866.log.gz 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113247.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101739.log.gz 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85764.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84872.log.gz 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113247.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90502.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113247.log.gz 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.3.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85764.log.gz 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90764.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90239.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90764.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.90502.log.gz 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86731.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101781.log 2024-04-25T10:08:49.941 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90764.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90239.log: /var/log/ceph/ceph-client.admin.86731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49484.log 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90239.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90197.log 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101781.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86731.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49484.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101781.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109798.log 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86108.log 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49484.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90197.log: /var/log/ceph/ceph-client.admin.109798.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90197.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79395.log 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95950.log 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109798.log.gz/var/log/ceph/ceph-client.admin.86108.log: 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79395.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.86108.log.gz 2024-04-25T10:08:49.942 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91735.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96389.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95950.log: /var/log/ceph/ceph-client.admin.91735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61861.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95950.log.gz 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.79395.log.gz 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91735.log.gz/var/log/ceph/ceph-client.admin.96389.log: 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48590.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89411.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96389.log.gz 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61861.log: /var/log/ceph/ceph-client.admin.48590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97561.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48590.log.gz 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107950.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61861.log.gz 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59487.log 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97561.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89411.log.gz 2024-04-25T10:08:49.943 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84777.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97561.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59487.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107950.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106222.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.59487.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57338.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.84777.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:gzip/var/log/ceph/ceph-client.admin.106222.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.104586.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59901.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106222.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104586.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57338.log.gz 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80664.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67137.log 2024-04-25T10:08:49.944 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59901.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104586.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68207.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59901.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80664.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66698.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67137.log: /var/log/ceph/ceph-client.admin.68207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109023.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67137.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66698.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68207.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91958.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66698.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46000.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85040.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91958.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109023.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.2.46000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46000.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43822.log 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91958.log.gz 2024-04-25T10:08:49.945 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110353.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85040.log: /var/log/ceph/ceph-client.admin.43822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114122.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85040.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110353.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43822.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76915.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110353.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100405.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62059.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76915.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114122.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44836.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76915.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100405.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44482.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62059.log: /var/log/ceph/ceph-client.admin.44836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77083.log 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62059.log.gz 2024-04-25T10:08:49.946 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44482.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44836.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64071.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44482.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59963.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34690.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64071.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77083.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64071.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5/var/log/ceph/ceph-client.admin.34690.log: --verbose -- /var/log/ceph/ceph-client.admin.46931.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49822.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59963.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34690.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104017.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46931.log: /var/log/ceph/ceph-client.admin.46469.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:08:49.947 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46931.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104017.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63379.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.c.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104017.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106694.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61795.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63379.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75567.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106694.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109359.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111439.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61795.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97037.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75567.log.gz 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80202.log 2024-04-25T10:08:49.948 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109359.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111439.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105178.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97037.log: /var/log/ceph/ceph-client.admin.80202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111665.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80202.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97037.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96944.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90095.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105178.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111665.log: /var/log/ceph/ceph-client.admin.96944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103933.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.111665.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90095.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65034.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96944.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90095.log.gz 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104143.log 2024-04-25T10:08:49.949 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65076.log 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65034.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103933.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79701.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65034.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65076.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104143.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67400.log 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65076.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58249.log 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114080.log 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79701.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67400.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103890.log 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58249.log.gz 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59744.log 2024-04-25T10:08:49.950 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97519.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103890.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114080.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.59744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67011.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59744.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103890.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61079.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97519.log: /var/log/ceph/ceph-client.admin.67011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63636.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97519.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61079.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67011.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61175.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61079.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58846.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.63636.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61175.log.gz 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60905.log 2024-04-25T10:08:49.951 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44020.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58846.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85933.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81359.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44020.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85933.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.60905.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81359.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85933.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106484.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81359.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92000.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63474.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106484.log: /var/log/ceph/ceph-client.admin.92000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42808.log 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106484.log.gz 2024-04-25T10:08:49.952 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103316.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92000.log.gz 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63474.log.gz 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67480.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.103316.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42808.log.gz 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103316.log.gz 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103190.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91100.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67480.log.gz 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92296.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103190.log.gz/var/log/ceph/ceph-client.admin.91100.log: 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48134.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91100.log.gz/var/log/ceph/ceph-client.admin.92296.log: 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44524.log 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92296.log.gz 2024-04-25T10:08:49.953 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64197.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48134.log: /var/log/ceph/ceph-client.admin.44524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101474.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64197.log: 87.9% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48134.log.gz 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44524.log.gz 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64197.log.gz 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64281.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5/var/log/ceph/ceph-client.admin.101474.log: --verbose -- /var/log/ceph/ceph-client.admin.68123.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64281.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101474.log.gzgzip 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62964.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64281.log.gz 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58802.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68123.log: /var/log/ceph/ceph-client.admin.62964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89243.log 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68123.log.gz 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58802.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62964.log.gz 2024-04-25T10:08:49.954 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58802.log.gz 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42886.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113289.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89243.log.gz 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101993.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42886.log.gz/var/log/ceph/ceph-client.admin.113289.log: 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106820.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113289.log.gz 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45571.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114248.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9%/var/log/ceph/ceph-client.admin.106820.log: -- replaced with /var/log/ceph/ceph-client.admin.101993.log.gz 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.45571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45571.log.gz 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106820.log.gz 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101350.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58165.log 2024-04-25T10:08:49.955 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101350.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114248.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63720.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101350.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109672.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58165.log: /var/log/ceph/ceph-client.admin.63720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94741.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109672.log: 89.2% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63720.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58165.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109672.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90137.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94741.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77041.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104322.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90137.log.gz 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77419.log 2024-04-25T10:08:49.956 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104322.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77041.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104322.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112263.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114516.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77419.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65238.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114516.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112263.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114516.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107156.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60839.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107156.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65238.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107156.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49357.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103148.log 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49357.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60839.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49357.log.gz 2024-04-25T10:08:49.957 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105756.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91310.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103148.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85218.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91310.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.91310.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.105756.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47588.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90460.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85218.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92042.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47588.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90460.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.90460.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47750.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92042.log.gz 2024-04-25T10:08:49.958 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59277.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49866.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47750.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76395.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101951.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59277.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110097.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101951.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76395.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101951.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75221.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100183.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110097.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99636.log 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100183.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75221.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100183.log.gz 2024-04-25T10:08:49.959 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67997.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66824.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99636.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81403.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66824.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67997.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66824.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77293.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95013.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81403.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61993.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95013.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77293.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95013.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96900.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48008.log 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96900.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61993.log.gz 2024-04-25T10:08:49.960 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96900.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81739.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68081.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96814.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68081.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81739.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68081.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112724.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96814.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85891.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112724.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48548.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76641.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76557.log 2024-04-25T10:08:49.961 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85891.log.gz 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59361.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76557.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76641.log.gz 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76557.log.gz 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110395.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50596.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59361.log.gz 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103410.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.50596.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110395.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.0.50596.log.gz 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41446.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60219.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103410.log.gz/var/log/ceph/ceph-client.admin.41446.log: 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43306.log 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60219.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41446.log.gz 2024-04-25T10:08:49.962 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60219.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95215.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95740.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43306.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102329.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95740.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95215.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95740.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40794.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109546.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.40794.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.102329.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40794.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66908.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4%/var/log/ceph/ceph-client.admin.66908.log: -- replaced with /var/log/ceph/ceph-client.admin.109546.log.gz 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84057.log 2024-04-25T10:08:49.963 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49736.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66908.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103788.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109630.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84057.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101867.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109630.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103788.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109630.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67221.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107393.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101867.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91651.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107393.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67221.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107393.log.gz 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114332.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47462.log 2024-04-25T10:08:49.964 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91651.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62754.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1%/var/log/ceph/ceph-client.admin.47462.log: -- replaced with /var/log/ceph/ceph-client.admin.114332.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47462.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49568.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47504.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62754.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46343.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47504.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49568.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47504.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101115.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64155.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46343.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62544.log 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64155.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101115.log.gz 2024-04-25T10:08:49.965 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64155.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99583.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76725.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62544.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66394.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76725.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99583.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76725.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76174.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77167.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66394.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108034.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76174.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77167.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77167.log.gz 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105700.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90638.log 2024-04-25T10:08:49.966 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108034.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105700.log: gzip -5 90.5% --verbose -- /var/log/ceph/ceph-client.admin.100141.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105700.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102077.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100141.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90638.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100141.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75102.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84099.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102077.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62282.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84099.log: 88.8% 91.3% -- replaced with /var/log/ceph/ceph-client.admin.75102.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84099.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111981.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104884.log 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62282.log.gz 2024-04-25T10:08:49.967 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86449.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104884.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.111981.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104884.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100099.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90890.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86449.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86815.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90890.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100099.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90890.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96625.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97603.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86815.log: /var/log/ceph/ceph-client.admin.96625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86815.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96625.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113018.log 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97603.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.97603.log.gz 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92651.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64992.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113018.log: /var/log/ceph/ceph-client.0.92651.log: 0.0% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113018.log.gz 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.0.92651.log.gz 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96688.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46217.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64992.log: /var/log/ceph/ceph-client.admin.96688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64992.log.gz 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96688.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111355.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103536.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46217.log: /var/log/ceph/ceph-client.admin.111355.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46217.log.gz 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89327.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111355.log.gz 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113331.log 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89327.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103536.log.gz 2024-04-25T10:08:49.969 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.89327.log.gz 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77251.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66226.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77251.log: /var/log/ceph/ceph-client.admin.113331.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77251.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113331.log.gz 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46511.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65352.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66226.log: /var/log/ceph/ceph-client.admin.46511.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66226.log.gz 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110181.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46511.log.gz 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42682.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110181.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65352.log.gz 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86575.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110181.log.gz 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111800.log 2024-04-25T10:08:49.970 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86575.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42682.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86575.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66614.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80763.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66614.log: /var/log/ceph/ceph-client.admin.111800.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111800.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66614.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65354.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80763.log: /var/log/ceph/ceph-client.admin.65354.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80763.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83997.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65354.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75483.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.83997.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83997.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46889.log 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75483.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75483.log.gz 2024-04-25T10:08:49.971 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107864.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46889.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46889.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46259.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107864.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107864.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74885.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46259.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46259.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76300.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.74885.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74885.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86028.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76300.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76300.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46595.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86028.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86028.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94825.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46595.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46595.log.gz 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43819.log 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94825.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94825.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68249.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43819.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43819.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86983.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68249.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68249.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86533.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86983.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86983.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75819.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86533.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86533.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95436.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75819.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75819.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95436.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95436.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101516.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66310.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48092.log 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101516.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101516.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28293.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48092.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48092.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60285.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.tmp-client.admin.28293.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28293.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97207.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60285.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60285.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96263.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97207.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97207.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104926.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96263.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96263.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113060.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104926.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104926.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89927.log 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113060.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113060.log.gz 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89927.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89927.log.gz 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76048.log 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.40917.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85807.log 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76048.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76048.log.gz 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44151.log 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85807.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85807.log.gz 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58333.log 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44151.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44151.log.gz 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58333.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58333.log.gz 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80538.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41488.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102245.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80538.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80538.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106610.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102245.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102245.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62796.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106610.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106610.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41044.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62796.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62796.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109935.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41044.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41044.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109233.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109935.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109935.log.gz 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2024-04-25T10:08:49.976 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109233.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109233.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43481.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104968.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44734.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96137.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104968.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104968.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111702.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96137.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96137.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90334.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111702.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111702.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46679.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90334.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90334.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60347.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46679.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46679.log.gz 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47630.log 2024-04-25T10:08:49.977 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60347.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60347.log.gz 2024-04-25T10:08:50.088 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:08:50.093 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:08:50.211 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59193.log 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47630.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47630.log.gz 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107822.log 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59193.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59193.log.gz 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108880.log 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107822.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107822.log.gz 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107072.log 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108880.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108880.log.gz 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113825.log 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107072.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107072.log.gz 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41086.log 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113825.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113825.log.gz 2024-04-25T10:08:50.212 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90932.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41086.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41086.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66740.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90932.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90932.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109840.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66740.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66740.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108268.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109840.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109840.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108268.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108268.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91489.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43264.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75347.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91489.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91489.log.gz 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108160.log 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75347.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75347.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108160.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108160.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99939.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46637.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41530.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99939.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.99939.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92086.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41530.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41530.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75609.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92086.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92086.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113954.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75609.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75609.log.gz 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109065.log 2024-04-25T10:08:50.214 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113954.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113954.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95782.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109065.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109065.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61613.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95782.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.95782.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48632.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61613.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61613.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110139.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48632.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48632.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110139.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110139.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95698.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76599.log 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95698.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95698.log.gz 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104670.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76599.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76599.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65974.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80286.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104670.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.65974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65974.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41276.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60651.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80286.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95055.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60651.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41276.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60651.log.gz 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100320.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59151.log 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95055.log: /var/log/ceph/ceph-client.admin.100320.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95055.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100320.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109317.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63067.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59151.log: /var/log/ceph/ceph-client.admin.109317.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59151.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105906.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109317.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107114.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105906.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63067.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.105906.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107114.log: gzip 88.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.87109.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107114.log.gz 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87109.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67911.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.87109.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112329.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108076.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67911.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63109.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108076.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.112329.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108076.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104758.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86190.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63109.log: /var/log/ceph/ceph-client.admin.104758.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63109.log.gz 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104758.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64599.log 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59553.log 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86190.log: /var/log/ceph/ceph-client.admin.64599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81655.log 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59553.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86190.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59553.log.gz 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64599.log.gz 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101644.log 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112892.log 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101644.log: /var/log/ceph/ceph-client.admin.81655.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81655.log.gz 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.101644.log.gz 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97417.log 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80706.log 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112892.log: /var/log/ceph/ceph-client.admin.97417.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112892.log.gz 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97417.log.gz 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85849.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95520.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85849.log: /var/log/ceph/ceph-client.admin.80706.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80706.log.gz 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85849.log.gz 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110749.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105473.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95520.log: /var/log/ceph/ceph-client.admin.110749.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95520.log.gz 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110749.log.gz 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109149.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58428.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.105473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105473.log.gz 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105010.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109149.log.gz 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77461.log 2024-04-25T10:08:50.220 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105010.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58428.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.105010.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47099.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112640.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47099.log: /var/log/ceph/ceph-client.admin.77461.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47099.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84998.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77461.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85428.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112640.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101432.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85428.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84998.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85428.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60157.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90376.log 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101432.log: /var/log/ceph/ceph-client.admin.60157.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60157.log.gz 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101432.log.gz 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48716.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102582.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90376.log.gz 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94867.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.102582.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48716.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.0.102582.log.gz 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84141.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76258.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94867.log.gz 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102663.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76258.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84141.log.gz 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76258.log.gz 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94699.log 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.102663.log: /var/log/ceph/ceph-client.admin.44608.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94699.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.102663.log.gz 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94699.log.gz 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91907.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92774.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44608.log.gz 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92774.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91907.log.gz 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92774.log.gz 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92655.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76831.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.92655.log: /var/log/ceph/ceph-client.admin.49694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92655.log.gz 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57296.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42414.log 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76831.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81231.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42414.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57296.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42414.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77714.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66572.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81231.log: /var/log/ceph/ceph-client.0.77714.log: 0.0% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81231.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.0.77714.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50045.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104842.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66572.log: /var/log/ceph/ceph-client.admin.50045.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66572.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50045.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91589.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104842.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104842.log.gz 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103232.log 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79838.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91589.log: /var/log/ceph/ceph-client.admin.103232.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91589.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103232.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111090.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114434.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79838.log: /var/log/ceph/ceph-client.0.111090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111090.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66488.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4%/var/log/ceph/ceph-client.admin.114434.log: -- replaced with /var/log/ceph/ceph-client.admin.79838.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114434.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64786.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85386.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66488.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.64786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64786.log.gz 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80985.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100225.log 2024-04-25T10:08:50.225 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85386.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110309.log 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80985.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104800.log 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110309.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100225.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111642.log 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110309.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81189.log 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111642.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104800.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.111642.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5/var/log/ceph/ceph-client.admin.81189.log: --verbose -- /var/log/ceph/ceph-client.admin.102203.log 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81189.log.gz 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112047.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112113.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112047.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102203.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44878.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112113.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.112047.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59109.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2%/var/log/ceph/ceph-client.admin.44878.log: -- replaced with /var/log/ceph/ceph-client.admin.112113.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113691.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59109.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44878.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48758.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59109.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109588.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113691.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48758.log.gz 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42372.log 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5/var/log/ceph/ceph-client.admin.109588.log: --verbose -- /var/log/ceph/ceph-client.admin.91184.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42372.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109588.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96305.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91184.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42372.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91184.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67305.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104526.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67305.log: /var/log/ceph/ceph-client.admin.96305.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96305.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67305.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109714.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.104526.log: /var/log/ceph/ceph-client.admin.75693.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104526.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110437.log 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75693.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109714.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75693.log.gz 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85470.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85344.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110437.log: /var/log/ceph/ceph-client.admin.85470.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110437.log.gz 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85470.log.gz 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59235.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85344.log.gz 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100905.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46385.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59235.log.gz 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46973.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101031.log 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46973.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100905.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46973.log.gz 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85176.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79922.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101031.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47708.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79922.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85176.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79922.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110265.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112479.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47708.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57852.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112479.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110265.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112479.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47546.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97860.log 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47546.log: /var/log/ceph/ceph-client.admin.57852.log: 88.3% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57852.log.gz 2024-04-25T10:08:50.230 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47546.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106652.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114474.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.97860.log: /var/log/ceph/ceph-client.admin.106652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97860.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103830.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106652.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114474.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114474.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64533.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102119.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103830.log: /var/log/ceph/ceph-client.admin.64533.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103830.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86316.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102119.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64533.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102119.log.gz 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100817.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86316.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112413.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100817.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100817.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.112413.log: /var/log/ceph/ceph-client.admin.75305.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47141.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112413.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57936.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108202.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75305.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110665.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108202.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57936.log.gz 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41216.log 2024-04-25T10:08:50.232 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110665.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108202.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110665.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48674.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81445.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41216.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48674.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48674.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58590.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46004.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81445.log: /var/log/ceph/ceph-client.admin.58590.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81445.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58590.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63337.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.2.46004.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75036.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.2.46004.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67095.log 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75036.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63337.log.gz 2024-04-25T10:08:50.233 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose 91.3% -- replaced with /var/log/ceph/ceph-client.admin.75036.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr: -- /var/log/ceph/ceph-client.admin.108569.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100699.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67095.log: /var/log/ceph/ceph-client.admin.108569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67095.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66968.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100699.log: 91.0% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100699.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108569.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85554.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48926.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66968.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66352.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48926.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85554.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48926.log.gz 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111523.log 2024-04-25T10:08:50.234 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112682.log 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66352.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112976.log 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112682.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111523.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112682.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107515.log 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94417.log 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107515.log: /var/log/ceph/ceph-client.admin.112976.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112976.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107515.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102287.log 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42498.log 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94417.log: /var/log/ceph/ceph-client.admin.102287.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102287.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.94417.log.gz 2024-04-25T10:08:50.235 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110521.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42498.log: gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42498.log.gz 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: /var/log/ceph/ceph-client.admin.91777.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67869.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110521.log: /var/log/ceph/ceph-client.admin.91777.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110521.log.gz 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91777.log.gzgzip 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.97856.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96179.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.97856.log: /var/log/ceph/ceph-client.admin.67869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97856.log.gz 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67869.log.gz 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49276.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46553.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96179.log.gz 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76090.log 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46553.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49276.log.gz 2024-04-25T10:08:50.236 INFO:teuthology.orchestra.run.smithi148.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46553.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103620.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101688.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76090.log: /var/log/ceph/ceph-client.admin.103620.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76090.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103620.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84309.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67953.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84309.log: /var/log/ceph/ceph-client.admin.101688.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84309.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101688.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60029.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108459.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67953.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94783.log 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.108459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108459.log.gz 2024-04-25T10:08:50.237 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60029.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89495.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66866.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94783.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89495.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89495.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109275.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95908.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66866.log: /var/log/ceph/ceph-client.admin.109275.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66866.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109275.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114206.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75263.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114206.log: /var/log/ceph/ceph-client.admin.95908.log: 87.8% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95908.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114206.log.gz 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57980.log 2024-04-25T10:08:50.238 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45566.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75263.log: /var/log/ceph/ceph-client.admin.57980.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75263.log.gz 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57980.log.gz 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47267.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.45566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45566.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45700.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92446.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45700.log: /var/log/ceph/ceph-client.admin.47267.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45700.log.gz 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47267.log.gz 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113648.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58081.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92446.log: /var/log/ceph/ceph-client.admin.113648.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92446.log.gz 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58758.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113648.log.gz/var/log/ceph/ceph-client.admin.58081.log: 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97669.log 2024-04-25T10:08:50.239 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58081.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96347.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97669.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58758.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96034.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96347.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97669.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96347.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49112.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57422.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49112.log: /var/log/ceph/ceph-client.admin.96034.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96034.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49112.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42456.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57422.log: -- /var/log/ceph/ceph-client.admin.89285.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57422.log.gz 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84225.log 2024-04-25T10:08:50.240 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42456.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89285.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89285.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64449.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56853.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84225.log.gz/var/log/ceph/ceph-client.admin.64449.log: 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109401.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5%/var/log/ceph/ceph-client.admin.56853.log: -- replaced with /var/log/ceph/ceph-client.admin.64449.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91394.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56853.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58716.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109401.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62670.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91394.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86688.log 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62670.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58716.log.gz 2024-04-25T10:08:50.241 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62670.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41318.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104185.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86688.log: /var/log/ceph/ceph-client.admin.41318.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86688.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76873.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5%/var/log/ceph/ceph-client.admin.104185.log: -- replaced with /var/log/ceph/ceph-client.admin.41318.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104185.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66782.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111895.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76873.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.66782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113869.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111895.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66782.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111895.log.gz 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110791.log 2024-04-25T10:08:50.242 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113869.log: /var/log/ceph/ceph-client.admin.110791.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113869.log.gz 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110791.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64950.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67646.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64950.log: /var/log/ceph/ceph-client.admin.34587.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64950.log.gz 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76006.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50592.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67646.log: /var/log/ceph/ceph-client.admin.76006.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67646.log.gz 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76006.log.gz 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101600.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.50592.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102987.log 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.0.50592.log.gz 2024-04-25T10:08:50.243 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61518.log 2024-04-25T10:08:50.244 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102987.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101600.log.gz 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.102987.log.gz 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47918.log 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96221.log 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61518.log: /var/log/ceph/ceph-client.admin.47918.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61518.log.gz 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47918.log.gz 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43054.log 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96221.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96221.log.gz 2024-04-25T10:08:50.245 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111758.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49399.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43054.log: /var/log/ceph/ceph-client.admin.111758.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43054.log.gz 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107992.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49399.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111758.log.gz 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49399.log.gz 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62880.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.107992.log: -- /var/log/ceph/ceph-client.admin.44962.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107992.log.gz 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90680.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44962.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62880.log.gz 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44962.log.gz 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101157.log 2024-04-25T10:08:50.246 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85680.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90680.log: /var/log/ceph/ceph-client.admin.101157.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90680.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 89.7% -5 --verbose -- /var/log/ceph/ceph-client.admin.89861.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101157.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79527.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89861.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85680.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 91.2% -- replaced with /var/log/ceph/ceph-client.admin.89861.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103053.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79527.log: /var/log/ceph/ceph-client.admin.50213.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.79527.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111397.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75953.log 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103053.log.gz 2024-04-25T10:08:50.247 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109107.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75953.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111397.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75953.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57768.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109756.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109107.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41404.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109756.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57768.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109756.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104101.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44160.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41404.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49917.log 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44160.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104101.log.gz 2024-04-25T10:08:50.248 INFO:teuthology.orchestra.run.smithi148.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44160.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66268.log 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87025.log 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49917.log: /var/log/ceph/ceph-client.admin.66268.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49917.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85596.log 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66268.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.87025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85596.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87025.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85596.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110223.log 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42640.log: /var/log/ceph/ceph-client.admin.110223.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110223.log.gz 2024-04-25T10:08:50.249 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84651.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102771.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100489.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102771.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84651.log.gz 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.102771.log.gz 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43821.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57516.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100489.log.gz/var/log/ceph/ceph-client.admin.43821.log: 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43821.log.gz 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64323.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112808.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57516.log: /var/log/ceph/ceph-client.admin.64323.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57516.log.gz 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64323.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105989.log 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.250 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112808.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112808.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77125.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49959.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105989.log: /var/log/ceph/ceph-client.admin.77125.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105989.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90974.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr: 88.9%/var/log/ceph/ceph-client.admin.49959.log: -- replaced with /var/log/ceph/ceph-client.admin.77125.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49959.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86358.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90974.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.90974.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102837.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101073.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86358.log: /var/log/ceph/ceph-client.admin.102837.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.86358.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105052.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr: 91.2%/var/log/ceph/ceph-client.admin.101073.log: -- replaced with /var/log/ceph/ceph-client.admin.102837.log.gz 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90053.log 2024-04-25T10:08:50.251 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105052.log: -- replaced with /var/log/ceph/ceph-client.admin.101073.log.gz 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60430.log 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105052.log.gz 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94909.log 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60430.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90053.log.gz 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60430.log.gz 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86857.log 2024-04-25T10:08:50.252 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81931.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94909.log: /var/log/ceph/ceph-client.admin.86857.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94909.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80160.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86857.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81931.log: gzip -5 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81931.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/ceph-client.admin.75903.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80160.log: /var/log/ceph/ceph-client.admin.75903.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80160.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96772.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.40606.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75903.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111853.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96772.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96772.log.gz 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46133.log 2024-04-25T10:08:50.253 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101558.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111853.log: /var/log/ceph/ceph-client.admin.46133.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111853.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107641.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46133.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.101558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111086.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.107641.log: 88.8% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101558.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr: -- replaced with /var/log/ceph/ceph-client.admin.107641.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63014.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57810.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.111086.log: /var/log/ceph/ceph-client.admin.63014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111086.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46427.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63014.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57810.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57810.log.gz 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105136.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2024-04-25T10:08:50.254 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46427.log: /var/log/ceph/ceph-client.admin.105136.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46427.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105136.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87175.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.104227.log: /var/log/ceph/ceph-client.admin.87175.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87175.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34326.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43818.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.34326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61013.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43818.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34326.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43818.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106988.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63848.log 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106988.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61013.log.gz 2024-04-25T10:08:50.255 INFO:teuthology.orchestra.run.smithi148.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106988.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41002.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63848.log: /var/log/ceph/ceph-mds.a.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63848.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110479.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41002.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41002.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50171.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110479.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110479.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76683.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50171.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50171.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76999.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76683.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76683.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77209.log 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.76999.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76999.log.gz 2024-04-25T10:08:50.256 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85134.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.77209.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77209.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82122.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85134.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85134.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90806.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.82122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.82122.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106264.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90806.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90806.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95992.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106264.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106264.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65787.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95992.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95992.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58207.log 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65787.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65787.log.gz 2024-04-25T10:08:50.257 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106778.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58207.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58207.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89579.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106778.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67179.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89579.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.89579.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96526.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67179.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67179.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100863.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96526.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96526.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113606.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100863.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100863.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91819.log 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113606.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113606.log.gz 2024-04-25T10:08:50.258 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63804.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91819.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.91819.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89969.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63804.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63804.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100989.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89969.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89969.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94615.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100989.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100989.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92732.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94615.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94615.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61303.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92732.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92732.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96431.log 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61303.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61303.log.gz 2024-04-25T10:08:50.259 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45088.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.96431.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96431.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114374.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.45088.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.45088.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91352.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.114374.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114374.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80454.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91352.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91352.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42970.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80454.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80454.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65195.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42970.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42970.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68309.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65195.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65195.log.gz 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113120.log 2024-04-25T10:08:50.260 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68309.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.68309.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89645.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113120.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113120.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110015.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89645.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89645.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106904.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110015.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110015.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106904.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106904.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92170.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43806.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84417.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92170.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92170.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90011.log 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84417.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.84417.log.gz 2024-04-25T10:08:50.261 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92212.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.90011.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.90011.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65501.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92212.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92212.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49652.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65501.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65501.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61435.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49652.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49652.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95614.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.61435.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61435.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108796.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95614.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95614.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67604.log 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.108796.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108796.log.gz 2024-04-25T10:08:50.262 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63943.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67604.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67604.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63943.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63943.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49441.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42330.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59025.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49441.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49441.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57135.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59025.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59025.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79593.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57135.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57135.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85722.log 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.79593.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79593.log.gz 2024-04-25T10:08:50.263 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86232.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.85722.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.85722.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100947.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.86232.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86232.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60971.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100947.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100947.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91016.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60971.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60971.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102161.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91016.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91016.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92338.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.102161.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102161.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64027.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92338.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92338.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81529.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64027.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64027.log.gz 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50405.log 2024-04-25T10:08:50.264 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81529.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81529.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81315.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50405.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50405.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48842.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81315.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81315.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59445.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48842.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48842.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58632.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59445.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59445.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92254.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58632.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58632.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68165.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.92254.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92254.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91058.log 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.68165.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68165.log.gz 2024-04-25T10:08:50.265 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48338.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.91058.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91058.log.gz 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81865.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.48338.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48338.log.gz 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47015.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81865.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.81865.log.gz 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65593.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47015.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47015.log.gz 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58674.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.65593.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65593.log.gz 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84525.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58674.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58674.log.gz 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63554.log 2024-04-25T10:08:50.266 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84525.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84525.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47834.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.63554.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63554.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40698.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.47834.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47834.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43823.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.40698.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40698.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59740.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.43823.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43823.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89795.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.0.59740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59740.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80244.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89795.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.89795.log.gz 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57069.log 2024-04-25T10:08:50.267 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80244.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80244.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112766.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.57069.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57069.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49778.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.112766.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112766.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109191.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.49778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49778.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113163.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.109191.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109191.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113564.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113163.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113163.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100615.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.113564.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113564.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99744.log 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100615.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100615.log.gz 2024-04-25T10:08:50.268 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-04-25T10:08:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.99744.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.99744.log.gz 2024-04-25T10:08:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81781.log 2024-04-25T10:08:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41722.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-04-25T10:08:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50255.log 2024-04-25T10:08:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81781.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81781.log.gz 2024-04-25T10:08:50.281 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46301.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50255.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50255.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67730.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.46301.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46301.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95173.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67730.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67730.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50087.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.95173.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95173.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103746.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.50087.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50087.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41128.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.103746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103746.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97165.log 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.41128.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41128.log.gz 2024-04-25T10:08:50.282 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75525.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.97165.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97165.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64407.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.75525.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75525.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58039.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64407.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64407.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42724.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58039.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58039.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89045.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.42724.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42724.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60707.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.89045.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.89045.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64491.log 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.60707.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60707.log.gz 2024-04-25T10:08:50.283 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105220.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64491.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64491.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80100.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.105220.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.105220.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59403.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.80100.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80100.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58508.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.59403.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59403.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110563.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.58508.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58508.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100447.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110563.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110563.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94483.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100447.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100447.log.gz 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64365.log 2024-04-25T10:08:50.284 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.94483.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.94483.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67263.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.64365.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64365.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62628.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.67263.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67263.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106400.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.62628.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62628.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56985.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.106400.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106400.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110623.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.56985.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56985.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100741.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.110623.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110623.log.gz 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84267.log 2024-04-25T10:08:50.285 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.100741.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100741.log.gz 2024-04-25T10:08:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111600.log 2024-04-25T10:08:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.84267.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84267.log.gz 2024-04-25T10:08:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81697.log 2024-04-25T10:08:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.111600.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111600.log.gz 2024-04-25T10:08:50.286 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.81697.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81697.log.gz 2024-04-25T10:08:51.180 INFO:teuthology.orchestra.run.smithi148.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:08:53.777 INFO:teuthology.orchestra.run.smithi188.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:08:54.043 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:08:54.047 INFO:teuthology.orchestra.run.smithi188.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:08:54.078 INFO:teuthology.orchestra.run.smithi148.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:08:54.226 INFO:teuthology.orchestra.run.smithi188.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:08:55.170 INFO:teuthology.orchestra.run.smithi188.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:08:55.171 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-04-25T10:08:55.172 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m5.526s 2024-04-25T10:08:55.172 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m23.150s 2024-04-25T10:08:55.172 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.720s 2024-04-25T10:08:55.512 INFO:teuthology.orchestra.run.smithi148.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:08:55.930 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:08:56.150 INFO:teuthology.orchestra.run.smithi148.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:08:56.161 INFO:teuthology.orchestra.run.smithi148.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:09:28.339 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:09:28.341 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-25T10:09:28.341 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m38.722s 2024-04-25T10:09:28.341 INFO:teuthology.orchestra.run.smithi148.stderr:user 1m7.228s 2024-04-25T10:09:28.341 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m2.612s 2024-04-25T10:09:28.341 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:09:28.342 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/remote/smithi148/log 2024-04-25T10:09:28.343 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:09:38.244 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/remote/smithi188/log 2024-04-25T10:09:38.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:09:41.512 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:09:41.522 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:09:41.522 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:09:41.525 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:09:41.608 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:09:41.617 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:09:41.618 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:09:41.621 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:09:41.634 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:09:41.637 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:09:41.639 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:09:41.639 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-25T10:09:41.639 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +1440us[+1436us] +/- 68ms 2024-04-25T10:09:41.639 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1054us[-1054us] +/- 63ms 2024-04-25T10:09:41.639 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 110 -182us[ -180us] +/- 83ms 2024-04-25T10:09:41.639 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:09:41.641 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:09:41.641 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-25T10:09:41.642 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +1392us[+1392us] +/- 68ms 2024-04-25T10:09:41.642 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1096us[-1096us] +/- 63ms 2024-04-25T10:09:41.642 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -224us[ -224us] +/- 84ms 2024-04-25T10:09:41.642 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:09:41.643 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:09:41.652 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:09:41.653 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:09:41.680 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:09:41.775 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:09:41.843 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:09:41.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:41.873 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:09:41.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:41.902 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:09:41.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:09:41.948 INFO:teuthology.task.internal:Duration was 1296.803036 seconds 2024-04-25T10:09:41.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:09:41.981 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:09:41.981 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:09:41.983 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:09:42.018 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:09:42.023 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:09:42.389 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:09:42.389 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-04-25T10:09:42.389 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:09:42.448 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-04-25T10:09:42.449 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:09:42.473 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:09:42.473 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:09:42.491 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:09:42.539 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:09:42.540 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:09:42.557 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:09:42.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:09:42.705 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:09:42.706 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:09:42.729 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:09:42.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:09:42.763 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:09:42.772 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:09:42.791 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-04-25T10:09:42.815 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-04-25T10:09:42.827 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:09:42.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:42.858 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:09:42.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:42.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:09:42.890 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:09:42.891 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/remote/smithi148 2024-04-25T10:09:42.891 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:09:42.933 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673039/remote/smithi188 2024-04-25T10:09:42.933 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:09:42.964 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:09:42.964 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:09:42.970 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:09:43.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:09:43.025 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:09:43.025 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:09:43.049 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:09:43.050 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:09:43.052 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:09:43.064 INFO:teuthology.orchestra.run.smithi148.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:09 /home/ubuntu/cephtest 2024-04-25T10:09:43.068 INFO:teuthology.orchestra.run.smithi188.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:09 /home/ubuntu/cephtest 2024-04-25T10:09:43.069 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:09:43.083 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:09:43.153 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} duration: 1296.8030359745026 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:09:43.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:09:43.234 INFO:teuthology.run:pass